macho
Posts: 13
Joined: Mon Dec 14, 2015 7:28 pm

rsync failure due to memory issue - workaround?

Sun Dec 10, 2017 11:35 pm

I'm doing a big backup with rsync and seeing it fail with a "broken pipe" error message. After the failure:

$ dmesg | grep rsync | tail -n 2
[6902561.445347] Out of memory: Kill process 19484 (rsync) score 421 or sacrifice child
[6902561.445399] Killed process 19485 (rsync) total-vm:270936kB, anon-rss:56840kB, file-rss:0kB, shmem-rss:0kB

The source includes a large number of hard links (incremental backups), and I am using rysnc's -H flag to copy them, which I assume is causing the problem. I have rsync'ed smaller portions of the source and am now at the point where it runs without copying essentially anything, and still runs out of memory.

Is there any workaround for this? Can I increase swap space or something?

Thanks!

MaxK1
Posts: 1043
Joined: Sun Aug 26, 2012 11:34 pm

Re: rsync failure due to memory issue - workaround?

Mon Dec 11, 2017 6:43 am

What model Pi and how much memory are you using for the GPU? What else do you have running?
You are in a maze of twisty little passages, all alike.
When General Failure and Major Disaster get together, Private Parts usually suffers.

macho
Posts: 13
Joined: Mon Dec 14, 2015 7:28 pm

Re: rsync failure due to memory issue - workaround?

Wed Feb 07, 2018 9:07 pm

Sorry for the slow reply.

It's an original Raspberry Pi (i.e. not 2 or 3) and I believe it's a model B rather than model A, since it has an ethernet port. Probably not A+, though not sure about that. Here's the info I can think of providing:

$ cat /proc/cpuinfo
processor : 0
model name : ARMv6-compatible processor rev 7 (v6l)
BogoMIPS : 697.95
Features : half thumb fastmult vfp edsp java tls
CPU implementer : 0x41
CPU architecture: 7
CPU variant : 0x0
CPU part : 0xb76
CPU revision : 7

Hardware : BCM2835
Revision : 000e
Serial : 00000000db73afdf
$ free -m
total used free shared buff/cache available
Mem: 434 185 48 17 199 174
Swap: 1023 179 844

The main other thing I have running is apache, for an owncloud server hosted on the pi, and then I have some probably less intensive daemons also running like mpd and transmission. Even the apache server isn't that active, since I'm the only user of the system.

Does that information help?

Return to “Troubleshooting”