Setting up a Pi 3 for the long haul?

[W. H. Heydt]

Buy a UPS! UPS filters out brown outs, surges, and power outages. Probably the #1 reason why a pie can die, is because the power is not reliable and the memory card gets corrupted. You could go down and buy a cheap $50 dollar APC and keep a few pie's running for a few days if that's all what was plugged into it.

An alternative here is the MoPower kit. It mounts directly on the Pis own GPIO block. His software will send a timely shutdown signal to Raspbian. It is also capable of restarting the Pi when lean power is restored. Other features include an RTC and the ability to have a short period of power backup (long enough to do a proper shutdown and allow wear leveling to complete) using supercaps instead of batteries. Moses will probably pop in now that his device has been mentioned.

[JackElliott]

Temperature. On this project I get:

Code: Select all

$ vcgencmd measure_temp
temp=38.4'C

This with the little black heatsinks in a fan-blown case, and at 15% cpu load.

[ian_g]

love the suggestions. I don't know if I am trolling I probably am.You are basically saying that you should buy a really really good sd card that costs more than the pi rather than say that the sd card is the implicit weakness of a pi.

[davidcoton]

Buy a UPS! UPS filters out brown outs, surges, and power outages. Probably the #1 reason why a pie can die, is because the power is not reliable and the memory card gets corrupted. You could go down and buy a cheap $50 dollar APC and keep a few pie's running for a few days if that's all what was plugged into it.

Or if you are really adventurous, you could hack together a lithium ion battery ups for a raspberry pi. Maybe even hook it up to solar!

Just a word of caution about the standard computer UPSs. They are designed to run with the load of a desktop PC. If you try to support the (comparatively) low-current Pi, it may just switch off. Also, a cheap UPS may not switch from mains to battery quickly enough to protect your system, whether it is a Pi or something else. So a (not-so-cheap) UPS is fine if it is already supporting the minimum load it is designed for, but buying a cheap mains UPS for a Pi might be a big mistake.

A low-voltage solution (either one designed specifically for the Pi, or a Power Bank that can charge while providing power) is a better approach for a Pi-only UPS.

You are basically saying that you should buy a really really good sd card that costs more than the pi rather than say that the sd card is the implicit weakness of a pi

The Pi is designed as a low-cost computer. That means that many of the design decisions favour low-cost, at the expense of survivability under "difficult" circumstances. The same is true of equivalently priced SD Cards, PSUs, and whatever else you connect to your Pi. Remember that "there is no such thing as a free lunch", and "you get what you pay for". Sometimes you pay a high price for low cost parts (Pi Zeros on auction sites), rarely you can find a real bargain on higher quality parts. This thread has explained some of the issues relevant to the OP, the bottom line is that the Pi is a brilliant low-cost SBC, but to make it suitable for high-reliability applications requires more understanding and expenditure. That is not to say that it is a bad approach, and very much not that you could do the job cheaper with a different computer. But there is a limit -- I would not design a safety-critical system around a Pi, and even company mission critical systems require careful thought as to whether the Pi is the best haedware for the task.

[CarlRJ]

love the suggestions. I don't know if I am trolling I probably am.You are basically saying that you should buy a really really good sd card that costs more than the pi rather than say that the sd card is the implicit weakness of a pi.

Yes, you have been trolling this thread. Starting off by implying the Pi was a toy computer that shouldn't be used for any serious purpose because it might fail (anyone with actual serious computer experience would know that all computer components will fail - it's just a matter of when - and take appropriate steps to have recovery plans in place, which are reasonably scaled to the importance of the system at hand). The cards in my Pi's, most of which have been running 24/7 for the past year (with occasional reboots for OS upgrades and such), were all in the neighborhood of $15 each (I've had precisely one SD card failure in a Pi in the past 3 years, early on, with a generic card, after far too many accidents where the power got yanked out). Having the microSD card as the default boot media is absolutely a weakness, yes, but it is also brilliant in many ways (cheap, widely available, easy to format/"burn" in other computers, and easy to change to a completely different use or project by swapping cards). More importantly, all computers have some sort of weakness. (The Pi's main failing, BTW, is limited IO bandwidth to/from the CPU via the single USB2 interface.) Just because the Pi, too, has weaknesses, is no reason to start jumping up and down shouting, "Fail! Fail! Fail!"

The OP's problem could be solved with a much higher likelihood of running continuously for the next ten years, using rack mount server grade hardware, redundant power supplies, UPSes, and support contracts. For a modest sum of, say, $10k. I don't think that's what the OP is looking for. With a Pi, you can get most (certainly not all) of that reliability, with perhaps a couple of short incidents of downtime, for perhaps $100. It sounds like that's much closer to what the OP is seeking. Sneering and saying the Pi isn't perfect, as your first post did, doesn't really help things.

I've never understood the "well, the computer only cost $35 so all the added bits have to be really cheap or else I've been ripped off somehow" line of reasoning. Yes, you wouldn't put a Formula 1 race engine in a cheap used subcompact, but that's hardly the level of impedance mismatch involved in putting a Sandisk or Samsung microSD card into a Pi. The genius of the Pi is that you can have a Linux system with enough CPU, GPU, and IO, to do all sorts of interesting things, and they're cheap enough to experiment with and to dedicate to specific tasks, and low power enough to leave running (I have half a dozen Pi's on 24/7 at home - I wouldn't do that with x86 boxes).

[W. H. Heydt]

Just a word of caution about the standard computer UPSs. They are designed to run with the load of a desktop PC. If you try to support the (comparatively) low-current Pi, it may just switch off. Also, a cheap UPS may not switch from mains to battery quickly enough to protect your system, whether it is a Pi or something else. So a (not-so-cheap) UPS is fine if it is already supporting the minimum load it is designed for, but buying a cheap mains UPS for a Pi might be a big mistake.

I routinely run Pis from small consumer grade UPSes. In fact there's one next to my desk set up that way right now. It's a 250VA unit with a Pi and a monitor connected to it (the monitor is off, by the way) and the Pi is staying up just fine, thank you (I just ssh'd into it...yup, it's up). If have seen that and other Pis on consumer grade UPSes operate across power drops without issue. I suspect the "minimum load" issue may be another case of "it used to be that...." My oldest UPS (1KVA, 30+ years old, on it's third set of 2 30Ah batteries) specified a switchover time of 4ms. It would be hard to imagine that modern ones would do noticeably worse. In any event, the DC conversion in a PSU has to cope with AC input and smooth that out, which implies a fair amount of capacitance and that probably covers enough time to let the UPS switch to running on battery.

If this is a concern, one can always get the hardware ($35 for the Pi, $40 for a small UPS) and test it. Set up a Pi running on the UPS and pull the plug. Is the Pi still running? In a commercial setting, that's a cheap test.

[davidcoton]

Just a word of caution about the standard computer UPSs. They are designed to run with the load of a desktop PC. If you try to support the (comparatively) low-current Pi, it may just switch off. Also, a cheap UPS may not switch from mains to battery quickly enough to protect your system, whether it is a Pi or something else. So a (not-so-cheap) UPS is fine if it is already supporting the minimum load it is designed for, but buying a cheap mains UPS for a Pi might be a big mistake.

I routinely run Pis from small consumer grade UPSes. In fact there's one next to my desk set up that way right now. It's a 250VA unit with a Pi and a monitor connected to it (the monitor is off, by the way) and the Pi is staying up just fine, thank you (I just ssh'd into it...yup, it's up). If have seen that and other Pis on consumer grade UPSes operate across power drops without issue. I suspect the "minimum load" issue may be another case of "it used to be that...." My oldest UPS (1KVA, 30+ years old, on it's third set of 2 30Ah batteries) specified a switchover time of 4ms. It would be hard to imagine that modern ones would do noticeably worse. In any event, the DC conversion in a PSU has to cope with AC input and smooth that out, which implies a fair amount of capacitance and that probably covers enough time to let the UPS switch to running on battery.

If this is a concern, one can always get the hardware ($35 for the Pi, $40 for a small UPS) and test it. Set up a Pi running on the UPS and pull the plug. Is the Pi still running? In a commercial setting, that's a cheap test.

Let me be clear, I am quite sure some UPSs are entirely suitable. But (from experience using a UPS for a 60W emergency light) not all. I've also seen a PC crash at switchover (different UPS, IIRC). It's a case of Buyer Beware, just because it's called a UPS doesn't mean it will deliver uninterrupted power in all circumstances. Before buying, check the specs. Then as you say, do the test. Repeat the test at regular intervals if it's important (batteries eventually die, switchover circuits stick, etc).

And, when you test a UPS, don't do what I saw one "professional" do. We had a database server, still on load. He "knew" the UPS would work, so he pulled the plug. The UPS thought otherwise. Result: crashed server, lost work, lost time to restart server and restore backup, lost contract for so-called professional.

[achrn]

You are basically saying that you should buy a really really good sd card that costs more than the pi rather than say that the sd card is the implicit weakness of a pi.

Can you elaborate or explain this comment some more please? I'm struggling to understand what point you're making.

So what if anyone suggested spending more on the SDcard than the Pi? I have plenty of servers under my control where the storage subsystem cost more than the rest put together (at least ten actually, I think - seven in the UK, three overseas). I also have a pi where the sound sub-system connected cost more than the rest put together (case and PSU included), and the screen on my one pi connected to a screen cost more than the rest too. But actually no, I don't think anyone is suggesting that. A Sandisk Ultra 32GB (one of the suggested brands, as high a grade within the brand as I ever buy, massively more space than required) is available from reputable sellers for about 17 quid - half the price of a pi 3B.

Your suggestion is to use a hard disk instead - but surely that will cost ... more than the pi itself?

[peterlite]

There are a few posts on flash memory life. An expensive controller/memory might erase and write 8K blocks while a cheap device erases and writes 64K blocks. It does not make a bit of difference writing photographs. It makes a big difference when updating a database or a tiny file. Plus your super reliable database will first write to the log file then write the data then update each index then update the log file to record the data as written. Your one update to change one bit in the data will end up as 4 rewrites by 64K, 256K in total. That is called write amplification. Divide your 8 TB write limit by 256K.

[JackElliott]

Leaving aside the stir of excitement caused by that troll, I wanted to get back to the issue of powering off a Pi without allowing the the SD card to perform its maintenance procedures, which risks corrupting the card. Is this a problem with a Pi that has been set to boot and operate from flash memory? Can you yank the power without breaking anything?

[W. H. Heydt]

Leaving aside the stir of excitement caused by that troll, I wanted to get back to the issue of powering off a Pi without allowing the the SD card to perform its maintenance procedures, which risks corrupting the card. Is this a problem with a Pi that has been set to boot and operate from flash memory? Can you yank the power without breaking anything?

Setting aside the fact that SD cards *are* flash memory... In theory, yes, that could break things. But then, so could an abrupt power drop to a rotating rust (HDD) as well. One way to reduce the chances would be to enable filesystem journaling. Note that I say "reduce" not "eliminate" as journaling also requires writing to the file system.

In short, there is no perfect protection against file system corruption.

[JackElliott]

Thank you. A couple of questions. Well, three, actually.

1. Where do I go to learn more about this "journaling" of which you speak? I'd like to learn how this can help protect my little Pi project from an abrupt loss of power.

2. Right, USB "thumb drives" are flash, like SD cards. But people routinely pull them from computer ports without first ejecting them, and unless some write is waiting to happen, they usually don't have a problem. Bad practice, yes, but are good USB flash drives less susceptible to rude power loss/yanking than equally-good SD card?

3. I have a question about the MoPower UPS, and perhaps there is a better place than here to ask it? Anyway, this may already be on the MoPower web page but it's not clear to me: In order to give the flash drive (SD or USB) sufficient time to perform shutdown procedures, can (or is) the MoPower be programmed to issue a shutdown command to the Pi several minutes before the MoPower removes power from the Pi?

Thanks for the help.

[W. H. Heydt]

Thank you. A couple of questions. Well, three, actually.

1. Where do I go to learn more about this "journaling" of which you speak? I'd like to learn how this can help protect my little Pi project from an abrupt loss of power.

As it applies to EXT4, start here: https://en.wikipedia.org/wiki/Ext4 If you're asking about journaling generally....Google is your friend. Journaling filesystems have been around for a long time. The general idea is the same as for databases. It's a way to recover from crashes. You restore from backup and the roll the journal file forward until you are current.

2. Right, USB "thumb drives" are flash, like SD cards. But people routinely pull them from computer ports without first ejecting them, and unless some write is waiting to happen, they usually don't have a problem. Bad practice, yes, but are good USB flash drives less susceptible to rude power loss/yanking than equally-good SD card?

Windows writes immediately. For performance reasons, unix and Linux do not. Therefore, at any random time, there are probably "dirty" blocks in memory. When the OS runs out of buffer space or a sync command is issued, then all dirty (that is, modified) blocks are written. It's pretty common to have the system do a sync command about every 30 seconds, but you can do one manually (or in a program) whenever you want to flush any new data to the mass storage device.

3. I have a question about the MoPower UPS, and perhaps there is a better place than here to ask it? Anyway, this may already be on the MoPower web page but it's not clear to me: In order to give the flash drive (SD or USB) sufficient time to perform shutdown procedures, can (or is) the MoPower be programmed to issue a shutdown command to the Pi several minutes before the MoPower removes power from the Pi?

Thanks for the help.

Actually...several seconds (say, 10 to 15) would be quite adequate. Watch the activity light on a Pi when doing a shutdown to see how long it takes. The MoPower UPS uses an Atmel processor and Moses has done a lot of clever programming for it, including enough delay to give the SD card time to finish any wear leveling activity before actually cutting the power.

As it happens, people running Pis are a bit spoiled with the ease of shutting them down. It used to be that one would announce a shutdown at least a few minutes ahead of time, then take the system down to single user mode, then sync the system and only at that point halt the system and then kill the power. A couple of other old habits one developed were, when editing the rule was "save early, and often" and when logging off, use "sync; sync; logout" where the reason for the double sync was that sync would return immediately and not wait for the actual process to complete. The second sync couldn't do that until the first one completed.

[davidcoton]

Thank you. A couple of questions. Well, three, actually.

I'll try and answer just one:

2. Right, USB "thumb drives" are flash, like SD cards. But people routinely pull them from computer ports without first ejecting them, and unless some write is waiting to happen, they usually don't have a problem. Bad practice, yes, but are good USB flash drives less susceptible to rude power loss/yanking than equally-good SD card?

USB sticks on PCs are usually used as portable data stores, while the Pi SDCard is used for the OS. Writes of data are fairly predictable, most people don't pull them while a write is still in progress. But OS writes can happen any time, so pulling the card or cutting power without shutdown is far more likely to interrupt a write. My guess is that this difference in use accounts for the apparently better survival of USB drives compared with Pi SDCards. There is also some evidence that some SDCards are more resilient than others, and that good ones can survive a lot of power fails -- see MosesPi's posts about his tests.

[JackElliott]

Thank you. A couple of questions. Well, three, actually.

1. Where do I go to learn more about this "journaling" of which you speak? I'd like to learn how this can help protect my little Pi project from an abrupt loss of power.

As it applies to EXT4, start here: https://en.wikipedia.org/wiki/Ext4 If you're asking about journaling generally....Google is your friend. Journaling filesystems have been around for a long time. The general idea is the same as for databases. It's a way to recover from crashes. You restore from backup and the roll the journal file forward until you are current.

Ah, so journaling is built into the ext4 filesystem. Didn't know that. Is there anything a fellow needs to enable? Will the device take advantage of the journaling to attempt to recover from an Ugly Incident?

It sounds like the out-of-the-box MoPower UPS is going to be on my shopping list.

As it happens, people running Pis are a bit spoiled with the ease of shutting them down. It used to be that one would announce a shutdown at least a few minutes ahead of time, then take the system down to single user mode, then sync the system and only at that point halt the system and then kill the power.

Say -- that does sound like fun!

[W. H. Heydt]

Say -- that does sound like fun!

Peopel who complain about how many seconds it takes to boot a Pi have never spent 30 minutes watching a mianframe boot.

[mosespi]

3. I have a question about the MoPower UPS, and perhaps there is a better place than here to ask it? Anyway, this may already be on the MoPower web page but it's not clear to me: In order to give the flash drive (SD or USB) sufficient time to perform shutdown procedures, can (or is) the MoPower be programmed to issue a shutdown command to the Pi several minutes before the MoPower removes power from the Pi?

Hi JackElliott,

I am the creator of the MoPower UPS. I read the RPI forums, and usually read through certain search keywords. If I see my UPS mentioned I try to reply back. You can always email me direct, mopower -at- allspectrum.com

The MoPower client software has a programmed delay from when the OS signals a full shutdown until power is cut. This has been, since day one, hardcoded to 5 seconds. I have never had or heard of anyone having trouble with the value, 5 seconds is a LONG time for a computer! In any case, you can edit it to whatever you like (it might be limited to 255, I have to check) although I would leave it alone.

On Jessie, once the client software is installed you can modify the following file:
/lib/systemd/system-shutdown/mopower-ups-timed-shutdown.sh
and edit the following (number 5), in two places..
printf '\rTIMED_SHUTDOWN=5\r' > $TTY

As a note, systemd executes files in that folder right before final shut down has finished (don't ask me what that means), and this coincides with the fast shutdown LED blink of the Pi.

I always try to use good quality SD cards, but my experience tells me that power is the number one reliability issue with the Pi. When powered properly, I've seen multi-year uptimes and operation in fairly harsh conditions without trouble.

Regards,
-Moses

[ian_g]

Wooden spoon is here to mix the pot

Ok @Originalposter
Understand Murphy's Law
Pick a timeframe for failure.then plan for it.

Now instead of all the hardware stuff you've just heard now think software.

1) Notification (I've failed do something)
2) I need to update (think unattended updateshttps://wiki.debian.org/UnattendedUpgrades)(The comedy value isn't high enough)
3) I need to troll a raspi thread
4) Now pick your granny (not that one! the one that likes a drink and can't use a smartphone) and prepare to document how to deal with the failure because she'll be in charge when it happens

[JackElliott]

Thanks, Moses. Two MoPower UPSs on order.

I was riding my self-built electric bicycle yesterday and reflected on how I have over $1,000 invested in it for the motor & controller, the big Li-ion battery pack, my smart charger, decent wheels, upgraded headlight, a nice rack, etc., etc., and all of them bolted to a used frame I picked up at the bike shop for $70. The frame is perfectly adequate. A Pi with some high-end flash memory, a nifty outboard UPS, and other enhancements to get it to do what I want it to do strikes me as the same kind of thing. Any doofus with a credit card can buy an expensive e-bike or some rackmount enterprise-grade equipment, but where's the fun or pride in that?

[tekn0]

I have been using this case for a while now with no overheating issues. It's cheap and functions as a large heat sync. https://nostalgiatechs.com/heatsync-case-pi-amazon