Minimising SD card writes

[smithnerd]

I'm trying to minimise unnecessary writes to the SD card.

I have nulled .bash_history, which keeps the current shell history in memory, but never writes it anywhere, thus:

Code: Select all

pi$ ln -sf /dev/null ~/.bash_history

I don't really want to use NFS, so I installed 'sshfs' and added 'pi' to the 'fuser' group. This gives me a working subdirectory, mounted from my Linux laptop, from which to write code:

Code: Select all

pi$ mkdir ~/laptop_mnt

pi$ sshfs piuser@laptop: ./laptop_mnt

Does anyone have any other ideas?

[lavers]

As an alternative to linking .bash_history to /dev/null, you can just set the HISTFILE shell variable (in your .bashrc file for example) to blank, which will cause Bash to not write a history file.

I haven't got my Pi yet, so I don't know if distributions like Raspbian already do this, but you might want to mount your SD card with the 'noatime' option to prevent it from writing access times on files every time you access them. Additionally, maybe turn down or off any log files you can find, or redirect them somewhere else.

[RRvW]

have you also looked at the /tmp and /var/lock directorys? Most linux distros mount them as tmpfs.

[kadamski]

Isn't bash default behavior to only write history to HISTFILE when exiting from shell? And since it's usually not really done often, does this "optimization" make much sense?

[Atilla]

Indeed, bash does save only on exit, unless you force it to, which you can easily confirm by murdering a terminal and checking your history afterwards.

[AndrewS]

So obvious it hopefully doesn't need suggesting, but make sure you've disabled swap.

If you want to really minimise the number of writes to the SD card, you could transfer your entire rootfs over to a USB (flash- or hard-) drive. Only the /boot partition needs to stay on SD.

And have a look at this thread www.raspberrypi.org/phpBB3/viewtopic.php?f=29&t=10775

[gdt]

I'd rather think that /var/log/* is the main source of writes, especially since the as-shipped /etc/rsyslog.conf has some files written a line at a line.

You could simply configure rsyslog to discard all messages.

What would be better requires some programming. Configure rsyslog to log to a UNIX domain socket (see the omucsock output module), write a daemon to listen on that socket and keep the last N KB of messages, have a client program which will retrieve and print the recent messages cached by the daemon. And yes, there a strong parallel here with the kernel's "dmesg" utility.

[gdt]

(Note to moderators: it would be nice if the configuration below became the default on the RPi's images, as it will make people's SD cards last longer. If you could bring it to the attention of the person responsible for such things, then that would be appreciated).

I've just had a play, and you can make rsyslog send to /dev/kmsg (which is readable by dmesg).

Edit /etc/rsyslog.conf
Remove "$ModLoad imklog" since we don't want to log kernel messages back to the kernel, and thus loop infinitely.
Similarly, alter the line starting "kern.*" to be "kern.* ~" (ie, discard kernel messages, don't fret as they are still in the kernel's message ring buffer).
Now for every line that writes to a file, replace the filename with "|/dev/kmsg". For example, the line
daemon.* -/var/log/daemon.log
becomes
daemon.* |/dev/kmsg

Restart rsyslog with
/etc/init.d/rsyslog restart

Now there should be no syslog messages written to /var/log.

To see the most recent syslogged messages enter the command
dmesg

When cleaning up, do not delete files in /var/syslog. Rather say "echo -n > /var/log/syslog". This removes the contents of the file, but keeps the file's ownership and permissions, which can be important if it is something other than rsyslog which was writing to the file.

Note that this isn't the best way to write to the kernel's log buffer (the kernel isn't told the message's attributes, such as "facility" or "priority" and it incorrectly classifies the message as kernel space rather than user space). To do it properly a rsyslog output module should be written. Also this isn't the most efficient approach -- having the kernel maintain the log is necessarily less efficient than doing it from user space. But it's not like the RPi writes enough messages that we overly care about the message's metadata or its efficiency. We rather take the reduced writes to the SD card, both so that the RPi runs more quickly and so the SD card lasts longer.

[obarthelemy]

I had started a whole topic about that a while back: http://www.raspberrypi.org/phpBB3/viewt ... +for+flash

Haven't worked on it since then though, sorry.