Raspberry Pi Forums

Hello All

I deployed a raspberry pi at a remote location to gather data. I was wondering how I would keep it alive after a power failure or shutdown. Right now, for some unexplained reasons, it shutdowns once in a while and I have to ask someone to reboot it. How would I keep it alive automatically?

Thanks.

An idea, maybe not the most eloquent:

Does it need to be on all the time? Because if not, then you could unplug it automatically once a day at a specific time, for example. Just have a Cron shutdown it down safely before power-off. Plug it into one of those cheap pluggable timers that will cut the power off and back on after 15 mins (or whatever other interval). This way, if it ever crashes or shutsdown in an unintentional way, at least it will reboot at the time that you have set the daily power out (and conseqeunt power-on) at. This would, naturally, mean rebooting it everyday, something that you might not want.

Or maybe even: have another pi monitor the first one, and if the first one crashes, have the other pi restart it or take over?

Power failure isn't a problem, when the power is restored your RPi will boot from the SDCard.

shutdown -h now (or halt) is a harder you either need to interrupt the power or you need to briefly short the P6 (A/B models) or RUN (A+/B+) header on the board to restart it.

shutdown -r now (or reboot) should restart as long as nothing hangs in the shutdown.

You can also do very bad things like: http://www.linuxjournal.com/content/rebooting-magic-way but that's only to be done with extreme prejudice. It is NOT recommended. I have a stripped down version of /sbin/shutdown (c program) that does something similar without bothering to terminate tasks in any kind of orderly fashion, just hit it hard with a very big & heavy blunt stick.

Using a time switch to force a power interruption every 24 hours (or once a week if you have a digital time switch) is one way. But there is a small amount of risk that you could (in exceptional circumstances) trash your SDCard by doing that.

Doing a scheduled shutdown -r now (from crontab) is another option, but as before that relies on nothing hanging during the shutdown phase.

CbeSolar wrote:....for some unexplained reasons, it shutdowns once in a while and I have to ask someone to reboot it....

I'd concentrate on finding the problem. If this remote location involves a lot of travel, build a duplicate system for local testing. Review system logs and write your own diagnostics into any software you have written (e.g. time-stamp start/restarts, time-stamp a heart beat every 30s, and save any errors that your program catches).

Its probably software related, but my second choice would be your power supply. As already mentioned, power-outs are not normally a big problem, because when the power is re-applied, the Pi just reboots. If the SD card gets trashed (I've never had this problem) then the system probably won't recover.

However, if your supply is marginal (say its giving you 4.7 to 4.8V) it may dip below the required minimum at some point and lock the system up.

Once you have a reliable system (up-time measured in months or years), you could add a hardware watch-dog. This might be a cheap Picaxe that keeps the supply to the Pi turned on as long as the Pi keeps kicking it. If the kicks stop, the dog sleeps for (say) 10s removing Pi power. Then it starts again and power is resumed.

My birdbox has a variation on this theme: http://captainbodgit.blogspot.co.uk/201 ... oller.html

Oh, one more thing, don't over-clock your Pi unless it is really necessary.

I too have a Pi running on another continent.
My application is a controller for my heater/cooler in my second home in Texas. There a numer of things you need to consider, as the other posters have alluded too as well.

First priority is to get a reliable power supply for the Pi. One that continues to provide clean power during a temporary glitch through a UPS function, and that also provides a clean and secure method to powerdown the Pi when the power is gone for a longer period, and to provide a clean window of power to let the Pi start. This is easy to be overlooked, but you don't want to have a power glitch while the Pi is booting. The Pi should boot and powerdown cleanly, and should always restart itself.

For my system I designed such a power supply from scratch, using normally available components and no seperate controller. You can have a look here : http://www.raspberrypi.org/forums/viewtopic.php?t=50470
It is a little long winded because the post goes through the design process, but in the end, you'll learn to avoid the pitfalls and see the final solution. This has now been operating for more than a year without a hitch.

Once you have solved this you can address the next couple of challenges. The second one is to make sure your application continues to run without a crash. Carefull design and using try-except's helps. Seperating time critical loops into a seperate thread can help to avoid endless loops or other hung issues. In case there is a hang or crash, you need to restart the application automatically. I do that with a seperate watchdog program that gets launched by cron. It checks if there is one instance of the program running (not more, and not less) and otherwise it will restart the application. This is made easier if you run your application as a daemon.

Make sure that your application can be restarted cleanly at all times, so this probably means that you need to preserve the latest state. I do that by putting that information in a file by the use of json structures.

The third element is when your application gets into a hung state or endless loop. You need to have a watchdog function in your code and in the the watchdog itself to catch that. I do that by having my application write the time into a file. The watchdog program reads that and compares if the delta is too great, in which case it will restart the application.

The fourth possibility is that OS will hang. I use the Debian/Pi hardware watchdog timer for that situation. I've never seen this happen, but still.

Lastly, if your application communicates over the internet, you need to add code to check the availability of that link, and either restart the internet interface, or if all else fails, restart the Pi.

A last remaining issue deals with the life-time of the SD Card. I started this about two years ago, when the SD card was a weak link. This is now much improved, but I still use a RAM disk for my files and logs. Also make sure you have a large SD card so the wear-leveling can do its job.

Of course, a Pi was never designed to be used under these constraints, but with some carefull hardware and software design, you can make this work reliably enough.
Hope this helps, paulv