sd card life running a pi 24/7

[unclejed613]

so back in december i bought one of the pi kits from Maker shed. which came with a Kingston SD4/4GB SDHC card. i set up the Pi as a web server for a tor onion site. except for downtime for backups of the SD card, the web server has been running 24/7 for almost 3 months. this last weekend, the Pi crashed, and after i restored the card from a backup, got it up and running again, but noticed there were some errors in some of the HTML files. i just bought a replacement card, and the pi is running again. when i did a fsck.ext4 of the original SD card (the linux partition, not the boot partition), there were many random errors, i reformatted the card, and it is about 100Megs smaller now. i think i'm going to buy several spare SD cards, since running 24/7 is probably kind of hard on the card. i'm now pretty much satisfied with the setup i have, and won't be going through the cycle of "let's try installing this.... i don't like it so uninstall it.... this looks like it might be better, so install it... yeah that works...". so the new SD card might last a bit longer than 3 months... yes it seems to be running much smoother now.... the periodic slamming of the CPU seems to have gone away (it had started doing that about the middle of last week... slamming the CPU for 15-30 seconds once every 5 minutes)

[Aydan]

In such a scenario you'll have to be careful with proper handling of wear leeveling.

#1: run trim periodically (e.g. once a day)
this erases unused blocks and puts them back into the wear leveling pool. The FS driver doesn't do this automatically, except if you mount with the discard option, but this will lower write performance.

#2: leave enough free space on your card as wear leveling pool.

Regards
Aydan

[RaTTuS]

if you are doing writes to the SD card [logs storing stuff] then minimise that so it does not ...

[poing]

In this thread: http://www.raspberrypi.org/phpBB3/viewt ... 66&t=37213 someone else is complaining on the same issue with a 4GB Kingston card. I use 8GB Sandisk cards (blue with red, old type) and never had a card corruption, even while often 'pulling the plug' without proper shutdown. So maybe the Kingston cards are not that suitable for use in the Pi.

[Mortimer]

Poing, isn't more likely that your 8GB card has more room for wear levelling than unclejed613's 4GB card.

I stopped using my 4GB cards in the RPi, because I thought the amount of spare space was too low, and started using 16GB cards instead. I am beginning to think this might have been a good move.

[Aydan]

The easiest way to not run into wear leveling problems is to not partition the whole card.
So even if all blocks on the partition are in use (meaning non-zero contents, not disk full) you'll have blocks available "outside" for the wear leveling pool.
You'll just have to make sure that the blocks outside of the partition really are zeroed out.

Regards
Aydan

[poing]

Poing, isn't more likely that your 8GB card has more room for wear levelling than unclejed613's 4GB card.

I'm far from an expert on wear leveling, but the OP states that he found errors in his HTML files. These are static so I doubt very much the corruption was due to wear

Assuming he had 2GB left on his card and each block can be written to 4,000 times the theoretical amount of data that can be written is 8TB. I've no clue about his setup, but in 'almost three month' that would come down to writing roughly 100GB per day to the card.

Assuming only writing lines to a log file and blocks of 512KB the amount of write operations would need to be almost 200,000 per day in the same period for the card to wear out.

[Mortimer]

If the HTML files do not occupy a whole block, they could get rewritten to a new block, when they share the block with a file that is getting rewritten, modified, deleted or whatever. If the new block is worn out, you might get a corrupted HTML file. Just because a file is never changed on flash storage, doesn't mean it won't get moved around. And if the trim function isn't being used, then the number of block receiving wear could be reduced somewhat.

[poing]

I didn't know a block could contain parts of more than one file? I thought the reason between 'real occupied disk space' and 'real file size' was because a block can only be used by on file and if the file is smaller than the block it uses the size of the block instead of it's own size.

[Mortimer]

You could be right, but the answer could lie in something called "Static Wear Levelling". The first wear levelling algorithms tried to even wear by changing the flash memory block allocated to a logical block address (LBA) when data was being written. Thus a file that never changed protected the blocks it used from ever being worn out. Newer algorithms incorporate "Static Wear Levelling", which means that files that don't change are periodically copied to different blocks so that the blocks they occupied can be cycled through the wear levelling pool. Thus a non-changing file is not necessarily protected from corruption as a card nears the end of its useful life.

[Kronalias]

Interesting discussion.

FWIW I've been running two Pi's 24/7 - one since just before Xmas 2012 (you can prolly guess why), and one from about a month ago. All(?) my transient files are held in ram - either in a 32MB ramdisk I've made, or another chunk of ram that's used for all the /var/log stuff. I'm not using fstrim, primarily because I'd not thought of it...

Both Pi's are headless, and both use 4GB SD cards.

The first one crashes (or more precisely reboots on a kernel panic or when the wifi drops and cannot reconnect) a few times a day. It also runs a webserver ...
http://kronalias.no-ip.org/
.... so the restarts could be caused by traffic(?). Its card is a Transcend 4GB Class 6 SDHC.

The second Pi is not accessible to the outside world (it's on the dark web), and its uptime is faultless.
That one uses a Hama 4GB Class 10 SDHC card.

Both cards have been tested with fsck, as in I've put the cards in a different PC, made sure they're not mounted, and executed these commands on both cards:
sudo fsck /dev/mmcblk0p1 -v
sudo fsck /dev/mmcblk0p2 -v
They're fine.

HTH

BTW, I'm assuming that the Class is the number inside the Big C on the physical card label, but I could well be wrong on this one.

[unclejed613]

actually, when the pi crashed last saturday, there were libraries used by Apache that had got corrupted, as well as an ogv file i was using as part of an html5 experiment. when i re-flashed the card, i had removed another large file (an mp4 file that i had used for testing omxplayer) before copying my web data directories across (i back up just the web directories once or twice a week using gftp, and do a whole-card backup once a month). so the oder of where certain files were on the card had been reshuffled a bit. after the crash, the pi would boot ok, but complain that one of the apache libraries was corrupted, so apache wouldn't start. so i re-flashed the card and reinstalled the web data. as i browsed through the wep page, to make sure all the links and files were ok, i saw one that had corrupted data in it. tnx for the info about trim. can i run that without unmounting the partitions?


edit:
trim
bash: trim: command not found

[unclejed613]

nvm... kronalias mentioned fstrim.... found it, reading the manpage now

[abishur]

out of curiosity, what sort of of turbo settings (if any) do you have enabled?

[inerlogic]

there's a tutorial for switching the system over to a USB HDD....
i've got RASPBMC and a development Pi running from old portable USB drives, powering the Pis and drives from powered Hubs.....

sorry, too lazy to google the link for you, but... yeh.... solid state memory has a finite lifespan (more so than HDDs) so if you're going to be running a server or moving large files around frequently.... use your head...

[unclejed613]

no turbo settings, running "plain vanilla".

i did notice that the main partition is ext4. this uses journaling. this means a fair amount of writes during normal operation. wouldn't ext2 that doesn't have journaling have been a better choice for a flash drive?

[forumisto]

how we can make full backup of SD card without powering off the RPi?
It's interesting if we need to replace the SD card and don't reinstall and reconfigure the programs

[Aydan]

trim
bash: trim: command not found

The command is called fstrim and you run it while mounted.
The first time after boot it will "trim" all blocks unused by the fs, no matter if they're zeroed/trimmed already or not. Depending on the speed of the SD card this can take a few minutes.
subsequent runs only trim blocks freed after the last run.

Regards
Aydan

[unclejed613]

i'm not sure if you could dd the partitions to an NFS share, that might be one way to make a backup while the pi is running of it's possible....

you can copy all of the contents while the pi is running, but dd'ing it while it's running might not be such a good idea.... you are still writing log files, writing to swap etc (plus, if i understand correctly the SD card's memory manager is operating at the same time, moving stuff around for wear leveling), and you might get a corrupted image just because of that.

[Aydan]

writing to swap

this shouldn't matter since swap is reinitialized after boot.

plus, if i understand correctly the SD card's memory manager is operating at the same time, moving stuff around for wear leveling, and you might get a corrupted image just because of that.

even if an sd card did static wear leveling (moving long unchanged blocks around in it's spare time), which is highly doubtful, it would be transparent (invisible outside the card). So no corruption there.

Even so, having a slightly corrupted image is still better than having no backup at all, and who cares about a corrupted logfile as long as the "production" data is OK.

Regards
Aydan