srdjantot
Posts: 3
Joined: Fri Jan 27, 2017 12:52 pm

Fixed OTA updates for kiosk rpi

Fri Jan 27, 2017 1:49 pm

Hi guys,

I have created simple kiosk setup using Arch Linux + Qt5 for showing some web pages. Everything is working great and I'm very happy with the setup. It is currently working on about 10 raspberrys.

Right now I would like to make some bullet proof update workflow that will allow me to easily revert changes in case of unpredicted problems.
I was thinking to setup one raspberry for testing. I will update and test it... if everything is ok, I will make somekind of "snapshot" that will be uploaded to other systems somehow. I could make 2 root partitions and switch back and forth on every update or I could use overlayfs for root partition and just select appropriate image for booting.

Do you have some suggestions?

epoch1970
Posts: 5144
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: Fixed OTA updates for kiosk rpi

Sat Jan 28, 2017 11:57 am

I guess I would use containers or something like that, because if you push by mistake a faulty update that makes the system crash, you wont be able to flip-flop between versions.
A few months ago I could have LXD running a thick VM instance (with direct access to devices) using an ubuntu image. I couldn't do the same with Raspbian. Nowadays I see plenty of mentions of Docker on Raspbian, so perhaps there is an easy way.

(If you don't care for this error case, I think you could also have the devices do a net boot in order to distribute the updated image. Then you wouldn't need the flip-flop mechanism.)
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

srdjantot
Posts: 3
Joined: Fri Jan 27, 2017 12:52 pm

Re: Fixed OTA updates for kiosk rpi

Sun Jan 29, 2017 2:40 pm

Thanks for replying.
Net boot was my first choice, but I need to boot even without network connection, so it's a no go.
Dockerizing system is great idea. I think I'll go with it if I don't find better solution.
I've done some researching and I really like the way Chrome OS is doing updates. Basically, everything is setup like this: one recovery partition and 2 sets of kernel + rootfs for system A and B. Both kernel partitions have some flags (successfully booted, retry count, and sone others) which determines what system (A or B) will be booted. Recovery is started if both systems are unbootable.

I will try to do something similar. I'll make one fat partition for recovery system. Raspberry will boot to it by default. Recovery system will look for 2 other fat partitions (system A and B). It will check for last update time, has the system ever booted successfully, how many times it failed to boot, etc, and will reboot into one of those partitions based on collected data.
Recovery partition will have only firmware, kernel and initramfs. Each system partition will have firmware, kernel, initramfs and squashfs image for read only rootfs.
Booted system will check for updates occasionally over https and when it found new system image, it will rsync it over to other system partition. That's basically it. I'll need to cover some corner cases like partial transfer, corrupted fs, updating recovery partition, saving configuration...

epoch1970
Posts: 5144
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: Fixed OTA updates for kiosk rpi

Sun Jan 29, 2017 5:19 pm

Interesting, thanks.
How do you write "I failed to boot"? How do you reboot? With an external watchdog?
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

srdjantot
Posts: 3
Joined: Fri Jan 27, 2017 12:52 pm

Re: Fixed OTA updates for kiosk rpi

Sun Jan 29, 2017 6:31 pm

I'll record successful boots by setting flag that is clear on next reboot by recovery.

I think that hardware + software watchdog will suffice. In case of system freeze, someone will cut the power and start it again.

User avatar
DougieLawson
Posts: 39182
Joined: Sun Jun 16, 2013 11:19 pm
Location: A small cave in deepest darkest Basingstoke, UK
Contact: Website Twitter

Re: Fixed OTA updates for kiosk rpi

Sun Jan 29, 2017 7:01 pm

srdjantot wrote:I'll record successful boots by setting flag that is clear on next reboot by recovery.
sudo apt-get install uptimed
and it's done for you.
Note: Any requirement to use a crystal ball or mind reading will result in me ignoring your question.

Criticising any questions is banned on this forum.

Any DMs sent on Twitter will be answered next month.
All non-medical doctors are on my foes list.

rduo
Posts: 5
Joined: Fri Dec 14, 2018 11:42 pm

Re: Fixed OTA updates for kiosk rpi

Mon Jul 22, 2019 5:22 pm

There are third party tools that can do this (atomic A/B system updates).

For example, the Mender OTA update manager has a tutorial on this for Raspbian here: https://hub.mender.io/t/raspberry-pi-3- ... b-raspbian

Return to “Advanced users”