systemd-journald.service crashing

[spartacus]

this is the second time this happened. last time (several weeks ago), it did this for about four hours before it started working normally. now it's doing it again. i'm guessing it's caused by the journal reaching its size limit.

Code: Select all

Aug 19 19:13:35.343249 capture systemd[1]: Started Trigger Flushing of Journal to Persistent Storage.
Aug 19 19:13:35.169993 capture systemd[1]: Starting Trigger Flushing of Journal to Persistent Storage...
Aug 19 19:13:35.302764 capture systemd-journal[2195]: Journal started
Aug 19 19:13:35.300654 capture systemd[1]: Started Journal Service.
Aug 19 19:13:35.300022 capture systemd[1]: Starting Journal Service...
Aug 19 19:13:35.299376 capture systemd[1]: Stopping Journal Service...
Aug 19 19:13:35.278515 capture systemd[1]: systemd-journald.service has no holdoff time, scheduling restart.
Aug 19 19:13:35.277890 capture systemd[1]: Unit systemd-journald.service entered failed state.
Aug 19 19:13:35.276972 capture systemd-journald[2190]: Received SIGTERM from PID 1 (systemd).
Aug 19 19:13:35.247825 capture systemd-journal[2195]: Permanent journal is using 1.4G (max allowed 1.4G, trying to leave 2.1G free of 11.1G available â curren
Aug 19 19:13:35.067754 capture systemd-journal[2190]: Journal stopped
Aug 19 19:13:34.467490 capture systemd[1]: Starting Journal Service...
Aug 19 19:12:52.199890 capture systemd[1]: systemd-journald.service watchdog timeout!
Aug 19 19:11:52.187706 capture systemd[1]: Started Trigger Flushing of Journal to Persistent Storage.
Aug 19 19:11:52.062911 capture systemd[1]: Starting Trigger Flushing of Journal to Persistent Storage...
Aug 19 19:10:21.699890 capture systemd[1]: systemd-journald.service watchdog timeout!
Aug 19 19:09:21.836882 capture systemd[1]: Started Trigger Flushing of Journal to Persistent Storage.
Aug 19 19:09:21.542730 capture systemd[1]: Starting Trigger Flushing of Journal to Persistent Storage...
Aug 19 19:11:52.273791 capture systemd-journal[2190]: Journal started
Aug 19 19:11:52.271931 capture systemd[1]: Started Journal Service.
Aug 19 19:11:52.271545 capture systemd[1]: Starting Journal Service...
Aug 19 19:11:52.271157 capture systemd[1]: Stopping Journal Service...
Aug 19 19:11:52.270736 capture systemd[1]: systemd-journald.service has no holdoff time, scheduling restart.
Aug 19 19:11:52.270318 capture systemd[1]: Unit systemd-journald.service entered failed state.
Aug 19 19:11:52.269833 capture systemd[1]: systemd-journald.service: main process exited, code=killed, status=9/KILL
Aug 19 19:11:52.269457 capture systemd[1]: Starting Journal Service...
Aug 19 19:11:52.269038 capture systemd[1]: systemd-journald.service stop-sigterm timed out. Killing.
Aug 19 19:11:52.268353 capture systemd[1]: Starting Journal Service...
Aug 19 19:11:52.224614 capture systemd-journal[2190]: Permanent journal is using 1.4G (max allowed 1.4G, trying to leave 2.1G free of 11.1G available â curren
Aug 19 19:09:21.910100 capture systemd-journal[2186]: Permanent journal is using 1.4G (max allowed 1.4G, trying to leave 2.1G free of 11.1G available â curren
Aug 19 19:06:51.153373 capture systemd[1]: Started Trigger Flushing of Journal to Persistent Storage.
Aug 19 19:06:51.045583 capture systemd[1]: Starting Trigger Flushing of Journal to Persistent Storage...
Aug 19 19:05:20.699875 capture systemd[1]: systemd-journald.service watchdog timeout!
Aug 19 19:06:51.282268 capture systemd[1]: Started Journal Service.
Aug 19 19:06:51.281820 capture systemd[1]: Starting Journal Service...
Aug 19 19:06:51.281353 capture systemd[1]: Stopping Journal Service...
Aug 19 19:06:51.280882 capture systemd[1]: systemd-journald.service has no holdoff time, scheduling restart.
Aug 19 19:06:51.280428 capture systemd[1]: Unit systemd-journald.service entered failed state.
Aug 19 19:06:51.279955 capture systemd[1]: systemd-journald.service: main process exited, code=killed, status=9/KILL
Aug 19 19:06:51.279507 capture systemd[1]: Starting Journal Service...
Aug 19 19:06:51.278902 capture systemd[1]: systemd-journald.service stop-sigterm timed out. Killing.
Aug 19 19:06:51.278134 capture systemd[1]: Starting Journal Service...
Aug 19 19:06:51.249129 capture systemd-journal[2161]: Journal started
Aug 19 19:06:51.225621 capture systemd-journal[2161]: Permanent journal is using 1.4G (max allowed 1.4G, trying to leave 2.1G free of 11.1G available â curren

i'm using 'journalctl -r'. not sure why some of the lines aren't in chronological order.

[sdjf]

I have seen both types of things happen. My wild guess about events being out of order is that if the system is busy writing something long, and another event happens during that write, that second event will be able to write to the log when the other is done. At least, that is my impression.

If you really need to keep all those old logs, consider zipping them up, or moving them to external storage, that is what I do when I run out of space.

And take a good look at pacman's cache - do you need packages there, now that we have the Arch Rollback Machine? Or move them to external storage.