rsync failure due to memory issue - workaround?
Posted: Sun Dec 10, 2017 11:35 pm
I'm doing a big backup with rsync and seeing it fail with a "broken pipe" error message. After the failure:
$ dmesg | grep rsync | tail -n 2
[6902561.445347] Out of memory: Kill process 19484 (rsync) score 421 or sacrifice child
[6902561.445399] Killed process 19485 (rsync) total-vm:270936kB, anon-rss:56840kB, file-rss:0kB, shmem-rss:0kB
The source includes a large number of hard links (incremental backups), and I am using rysnc's -H flag to copy them, which I assume is causing the problem. I have rsync'ed smaller portions of the source and am now at the point where it runs without copying essentially anything, and still runs out of memory.
Is there any workaround for this? Can I increase swap space or something?
Thanks!
$ dmesg | grep rsync | tail -n 2
[6902561.445347] Out of memory: Kill process 19484 (rsync) score 421 or sacrifice child
[6902561.445399] Killed process 19485 (rsync) total-vm:270936kB, anon-rss:56840kB, file-rss:0kB, shmem-rss:0kB
The source includes a large number of hard links (incremental backups), and I am using rysnc's -H flag to copy them, which I assume is causing the problem. I have rsync'ed smaller portions of the source and am now at the point where it runs without copying essentially anything, and still runs out of memory.
Is there any workaround for this? Can I increase swap space or something?
Thanks!