using logrotate and ramlog

[Holy One]

I was just wondering if anyone had tried this?

basically 99% of all logs are unimportant and reducing the writes to the sdcard would be nice. so using a small ramdisk for logging, that is only written out when the system is shut down or after a set period of time seems like a good idea.

using logrotate to cap the size of those logs even better?

more info: http://archive09.linux.com/feature/141231 && http://www.debian-administration.org/articles/117

[Elect]

Is this the project you are referring to? http://www.tremende.com/ramlog/
I've tried to install ramlog, but dpkg ended with an error. But I'm really interested in working with ramlog as i'm not very happy with writing often to the SD card.

Momentarily i'm using a 1 MB RAM filesystem for my MRTG images. This works well, although i'm losing everything when it restarts. But it's better then buying a new SD card.

[Elect]

Got Ramlog to work and wrote a little tutorial. It seemed that I missed a package in order to Ramlog.

http://piserv.nl/blog/index.php/using-t ... pberry-pi/

[AlterSchwede]

Your ramlog tutorial is not available. Can you post it here?
I have some problems with rsyslog and wonder how you solved it...

Code: Select all

Oct 14 11:28:03 Starting ramlog-tmpfs 2.0.0: Error: /var/log is in use... [fail]
Oct 14 11:28:05 The list of open files: (You need to close below daemons if you want to start/stop ramlog manually)

COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF  NODE NAME
rsyslogd 1805 root    1w   REG  179,2    45580 66966 /var/log/syslog
...

[AlterSchwede]

Update:
This is what is needed:

Code: Select all

sudo apt-get install rsync
sudo apt-get install lsof
wget http://www.tremende.com/ramlog/download/ramlog_2.0.0_all.deb
sudo dpkg -i ramlog_2.0.0_all.deb
# you need to add the following in /etc/init.d/ramlog:
# X-Start-Before: rsyslog
# X-Stop-After: rsyslog
# to the ###BEGIN INIT INFO block in /etc/init.d/ramlog
# and perhaps add more services to the list  (lighthttp, samba etc. if you are using these), e.g.
 X-Start-Before: rsyslog lighthttp
# and do
sudo insserv
...
# reboot twice
...
# check:
cat /var/log/ramlog
# that every think is OK
# and
sudo /etc/init.d/ramlog status
# that ramlog is running

[freemanbubu]

Hi,

I try to use Ramlog on a fresh install, but i'm stuck

ramlog always start after rsyslog, so /var/log is locked....

I do this (with insserv command and reboot after all modifications)
- add in /etc/init.d/ramlog

Code: Select all

X-Start-Before: rsyslog

==> fail

- add in /etc/init.d/rsyslog

Code: Select all

Required-Start: ramlog

===> fail again

Can anyone help me ?

here my ramlog logs:

Oct 25 21:42:55 Starting ramlog-tmpfs 2.0.0: [fail]
Oct 25 21:44:21 Starting ramlog-tmpfs 2.0.0: Error: /var/log is in use... [fail]
Oct 25 21:44:22 The list of open files: (You need to close below daemons if you want to start/stop ramlog manually)

COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
rsyslogd 1801 root 1w REG 179,2 347486 4426 /var/log/syslog
rsyslogd 1801 root 2w REG 179,2 311681 1594 /var/log/kern.log
rsyslogd 1801 root 5w REG 179,2 18904 1592 /var/log/debug
rsyslogd 1801 root 6w REG 179,2 286188 1600 /var/log/messages
rsyslogd 1801 root 7w REG 179,2 27470 4902 /var/log/daemon.log
rsyslogd 1801 root 8w REG 179,2 21113 4903 /var/log/auth.log
console-k 2019 root 8w REG 179,2 8679 4899 /var/log/ConsoleKit/history
Oct 25 21:44:22 Test result: ramlog cannot be started or stopped at the moment.

[freemanbubu]

Solved

I miss a parameter in /etc/default/ramlog....

The boot/stop order works fine

Oct 26 10:39:31 Starting ramlog-tmpfs 2.0.0: [ OK ]
Oct 26 10:45:21 Saving logs to hdd: [ OK ]
Oct 26 10:45:22 Stopping ramlog: [ OK ]
Oct 26 10:45:33 Starting ramlog-tmpfs 2.0.0: [ OK ]
Oct 26 10:47:29 Saving logs to hdd: [ OK ]
Oct 26 10:47:32 Stopping ramlog: [ OK ]
Oct 26 10:47:42 Starting ramlog-tmpfs 2.0.0: [ OK ]

[joshmosh]

Hi,
after having successfully installed ramlog and after fiddling with the order of daemons during boot (some entries here were quite helpful, thank you), I am now stuck with a message that ramlog could not find a shutdown flag file. Googling for shutdown flag file did not give me further clues.

Is there a friendly soul who can give me further hints ?

Thank you all in advance
Joshi

[freemanbubu]

Hi,

I had the same issue....
Ramlog start correctly, but the "shutdown flag" error occurs when I try to stop it....

When ramlog process try to stop, if rsyslog is still running, /var/log is lock and ramlog fail to stop.
I solved it by adding ramlog in the "stop order" of daemon "rssylog".

In /etc/init.d/rsyslog, I add :

Code: Select all

# Required-Start:    ramlog
# Required-Stop:     ramlog

Required start : ramlog start before rsyslog
Required stop : rsyslog stop before ramlog

Try this

[joshmosh]

Hi freemanbubu,
thank you for your kind help, but this trick did not solve my problem
Still the same error mesage

But anyway: thanks a lot !

Cheers
Joshi

[freemanbubu]

You add this in the LSBInit script, a the top of /etc/init.d/rsyslog ?

like this example (not actual rsyslog script):

Code: Select all

### BEGIN INIT INFO
# Provides:          scriptname
# Required-Start:    $remote_fs $syslog ramlog
# Required-Stop:     $remote_fs $syslog ramlog
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Start daemon at boot time
# Description:       Enable service provided by daemon.
### END INIT INFO

without uncomment, just add "ramlog" at end of lines Required-start & required-stop

[joshmosh]

Yes, I added it exactly there and did not uncomment it. The only difference is that I did not put ramlog at the end but rather in the beginning. Don't thing that this will make a difference, but I'll give it a try and put it at the end.
From my experiments with the other daemons: do you think it is worthwhile to add ramlog in the respective files of the other daemons (cups, lighttpd, php-fpm,samba) ? I had to include those in the X-Before section of ramlog.

Thank you and cheers
Joshi

[freemanbubu]

I think process order don't make any difference.

About edit all init.d script, I allready try this, but my ramlog config file was wrong....so I can't tell now if it works or not....
I can't access my Pi now, but I will try this later.

Can you post outpout of command:

Code: Select all

sudo service ramlog stop

[joshmosh]

You are right, process order did not change anything.
I have put 'ramlog' in all Required Start and Required Stop sections of the daemons (samba, lighttpd, syslog, php5-fmp, cups), but without any effect.
So I have doubts that (at least for my type of problem) the Required Start/Stop thing changes anything ...

Here is the output of 'sudo service ramlog stop':

ramlog: Cannot find shutdown flag file. ... (warning).
ramlog: Cannot find shutdown flag file.Stopping ramlog: Error: ramlog is not running failed!

Cheers
Joshi

[freemanbubu]

... I think I misunderstand something:
Did your ramlog process start ? You have an error when you try to stop it, but does it start ?

can you post output of:

Code: Select all

sudo service ramlog start

and

Code: Select all

df -h

thanks


Here my actual config :

/etc/init.d/rsyslog

Code: Select all

#! /bin/sh
### BEGIN INIT INFO
# Provides:          rsyslog
# Required-Start:    $remote_fs $time ramlog
# Required-Stop:     umountnfs $time ramlog
# X-Stop-After:      sendsigs
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: enhanced syslogd
# Description:       Rsyslog is an enhanced multi-threaded syslogd.
#                    It is quite compatible to stock sysklogd and can be
#                    used as a drop-in replacement.
### END INIT INFO

/etc/init.d/ramlog

Code: Select all

### BEGIN INIT INFO
# Provides: ramlog
# X-Start-Before:
# Required-Start:
# Should-Start: $local_fs
# Required-Stop:
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: moves /var/log into ramdisk
# Description: ramlog daemon moves /var/log to ramdisk on startup and copies it back to harddrive on shutdown or restart
### END INIT INFO

df -h command (with ramlog running, see ramlog-tmpfs):

Code: Select all

pi@raspberrypi ~ $ df -h
Filesystem      Size  Used Avail Use% Mounted on
rootfs          7.4G  1.6G  5.5G  23% /
/dev/root       7.4G  1.6G  5.5G  23% /
devtmpfs        117M     0  117M   0% /dev
tmpfs            24M  212K   24M   1% /run
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs            47M     0   47M   0% /run/shm
/dev/mmcblk0p1   56M   34M   23M  60% /boot
/dev/root       7.4G  1.6G  5.5G  23% /var/log.hdd
ramlog-tmpfs     10M  3.0M  7.1M  30% /var/log

And ramlog config /etc/default/ramlog

Code: Select all

#Ramlog service configuration

RAMDISKTYPE=0           #0 = tmpfs (can be swapped)
                        #1 = ramfs (no max size in older kernels, cannot be swapped, not SELinux friendly)
                        #2 = old kernel ramdisk
                        # (kernel ramdisk needs kernel parameter "ramdisk_size=" in kilobytes as default kernel ramdisk size
                        #  is usually lower than the size of the /var/log)

TMPFS_RAMFS_SIZE=10m    #Maximum size of memory to be used by tmpfs or ramfs.
                        # The value can be percentage of total RAM or size in megabytes - for example:
                        # TMPFS_RAMFS_SIZE=40%
                        # TMPFS_RAMFS_SIZE=100m
                        # Empty value means default tmpfs/ramfs size: TMPFS_RAMFS_SIZE=
                        # For more options please check 'man mount', section 'Mount options for tmpfs'
                        # (by the way - ramfs supports size limit in newer kernels as well despite man says there are no mount options)
                        # It has only effect if RAMDISKTYPE=0 or 1

KERNEL_RAMDISK_SIZE=MAX #Kernel ramdisk size in kilobytes or MAX to use entire ramdisk. It has only effect if RAMDISKTYPE=2

LOGGING=1               #0=off, 1=on
LOGNAME=ramlog          #name of the ramlog log file (makes sense if LOGGING=1)

VERBOSE=1               #0=off, 1=on (if 1, teststartstop puts detials to the logs and it is called after start or stop fails)

[joshmosh]

ramlog does not start. In /var/log/ramlog it says: ramlog start [fail]
df -h shows that there is no ramlog
Thank you for posting your /etc/init.d/ramlog. You don't have entries under X-Start-Before whereas I have all my daemons listed there. I will now remove these entries - we'll see what will happen.

Will be back in some minutes.

Joshi

[joshmosh]

Good morning
Still no success. I will now make a fresh start with an image I saved before installing the ramlog.
I have a habit to make an image everytime before I start to install new major things.

Perhaps all the fiddling has damaged something.

Unfortunately today I will be ocupied with other things, so I very likely cannot spend too much time on this. Wille be back tomorrow with more results - hopefully.

Have a nice day
Joshi

[freemanbubu]

ok ok,

You can save your actual image, and rebuild a new system from website iso (to start from a fresh install)

Do the config during first boot (raspi-config), then:

Code: Select all

sudo apt-get update
sudo apt-get upgrade
sudo wget http://www.tremende.com/ramlog/download/ramlog_2.0.0_all.deb
sudo dpkg -i ramlog_2.0.0_all.deb
sudo reboot
sudo reboot

need to reboot twice....

Then, look with df -h if ramlog is OK
If ramlog don't start, do

Code: Select all

sudo servcie ramlog start

In the output, you will see what process use /var/log

Then, insert in these process (/etc/init.d/) "ramlog" in Required-Start and Required-Stop
Reboot twice, and check if all is OK.

If it's working, save your config files, restore your image, and compare....
I don't understand why ramlog never works, you do all configs and modifications possible....

[joshmosh]

Good evening,
it is really a good evening, because it works now
Obviously the system was in a sort of limbo state because of my many unsuccessful attempts.
Now I have put in my daemons in X-Start-Before as well as in Required Start. df -h shows the ram disk.
Thank you very much for your helpful hints.
Now I will drink a glass of wine to your health

Have a nice evening
Joshi

[freemanbubu]

Hi

great news !
FYI, I allow 10m to ramlog, it's enough for "basic" usage : web, sql, php, mumble

Have a nice day

(i'm french, I hope you drink french wine )

[joshmosh]

Good evening to you,
I currently have a ramlog size of 16 M - probably a bit of a waste. I will keep an eye on it over the next few days ...

I am German and my preferred wines come from Italy, South Africa and France - in that order, sorry. France has a number of outstanding wines, but they are rather expensive, at least here in Germany.
Right now I am sipping a splendid Vino Nobile de Montepulciano - I hope you can forgive me

My next project is to use the Raspi as a replacement for my Apple TV. Unfortunately I have an ATV3, which at the moment cannot be jailbroken at therefore is somewhat crippled. If this works reasonably well, then I will order my second Raspi

Have a nice evening - with a glas of good French wine

Santé !
Joshi

[Thijxx]

I also like the Vino Nobile de Montepulciano! But thats offtopic.

Ramlog won't start. At one point I noticed the RAM disc using the "df -h" command but that's also not there anymore. Guess there is something wrong in the 'ramlog' file...

My /etc/init.d/ramlog file:

Code: Select all

### BEGIN INIT INFO
# Provides: ramlog
# Required-Start: 
# X-Start-Before: rsyslog transmission-daemon openvpn pptpd shellinabox
# X-Stop-After: 
# Should-Start: $local_fs
# Required-Stop:
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: moves /var/log into ramdisk
# Description: ramlog daemon moves /var/log to ramdisk on startup and copies it back to harddrive on shutdown or$
### END INIT INFO
                                                                                                                 
############################################################
# Ramlog default variables - please change variables in 
# /etc/sysconfig/ramlog on Fedora/CentOS systems and in
# /etc/default/ramlog on Ubuntu/Debian systems

RAMDISKTYPE=0
TMPFS_RAMFS_SIZE=30m
KERNEL_RAMDISK_SIZE=MAX
LOGGING=1
LOGNAME=ramlog
VERBOSE=1
CONFIGFILE=/etc/init.d/ramlog
 

My /etc/init.d/rsyslog file:

Code: Select all

### BEGIN INIT INFO
# Provides:          rsyslog
# Required-Start:    $remote_fs $time ramlog
# Required-Stop:     umountnfs $time ramlog
# X-Stop-After:      sendsigs
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: enhanced syslogd
# Description:       Rsyslog is an enhanced multi-threaded syslogd.
#                    It is quite compatible to stock sysklogd and can be 
#                    used as a drop-in replacement.
### END INIT INFO

My /var/log/ramlog file:

Code: Select all

Nov 16 16:58:11 ramlog: Cannot find shutdown flag file.                  [warning]                               
Nov 16 16:58:11 Stopping ramlog: Error: ramlog is not running            [fail]                                  
Nov 16 16:58:20 Starting ramlog-tmpfs 2.0.0:                             [fail]                                  

The whole X-Start-Before, Requiered-Start etc. is not clear to me. I'll start Googling/reading about it and maybe you guys know how to configure it the right way.

Thanks

[freemanbubu]

can you try this config for /init.d.ramlog:

Code: Select all

### BEGIN INIT INFO
# Provides: ramlog
# X-Start-Before:
# Required-Start:
# Should-Start: $local_fs
# Required-Stop:
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: moves /var/log into ramdisk
# Description: ramlog daemon moves /var/log to ramdisk on startup and copies it back to harddrive on shutdown or restart
### END INIT INFO

And keep your actual /init.d/rsyslog config ?

[Thijxx]

Thanks for your reply.

I've changed the config, after a couple of reboots, nothing has changed.

What I did find is that when I start Ramlog manually, it can't find the /log mount it should have created (at boot-time?) so maybe it's an issue with credentials.

[freemanbubu]

/var/log is used by many different processes

If ramlog start after one of them, he can't create a "virtual drive" with this folder (currently in use when try to create the symbolic link)
So df -h command don't show the virtual drive.

When you start ramlog manually, the output show what processes use /var/log.

Add "ramlog" in these processes /etc/init.d/** (# Required-Start & # Required-Stop fields)


Can you post the output when you start ramlog manually ?