Raspberry Pi Forums

Posted: **Sat Dec 28, 2013 12:46 pm**

When I seeding, aria2 progress will be killed for high memory usage.

I didn't know that the kernel will kill progress who is using too much memory. I thought kernel would just return NULL when she don't have more memory for a new malloc call.

I think this would be a common problem for Raspberry Pi servers. Do we have to rewrite the server program to meet Raspberry Pi's memory capability? I'll post a same post on aria2 forum to ask for help.

Code: Select all

Dec 28 16:07:48 raspberrypi kernel: [12343.878390] aria2c invoked oom-killer: gfp_mask=0x2084d0, order=0, oom_score_adj=0
Dec 28 16:07:48 raspberrypi kernel: [12343.878430] CPU: 0 PID: 1828 Comm: aria2c Not tainted 3.10.24+ #614
Dec 28 16:07:48 raspberrypi kernel: [12343.878505] [<c00139f8>] (unwind_backtrace+0x0/0xf0) from [<c0010d5c>] (show_stack+0x10/0x14)
Dec 28 16:07:48 raspberrypi kernel: [12343.878549] [<c0010d5c>] (show_stack+0x10/0x14) from [<c03fcdd8>] (dump_header.isra.13+0x74/0x1b0)
Dec 28 16:07:48 raspberrypi kernel: [12343.878587] [<c03fcdd8>] (dump_header.isra.13+0x74/0x1b0) from [<c0099c24>] (oom_kill_process+0x298/0x41c)
Dec 28 16:07:48 raspberrypi kernel: [12343.878620] [<c0099c24>] (oom_kill_process+0x298/0x41c) from [<c009a22c>] (out_of_memory+0x26c/0x2b8)
Dec 28 16:07:48 raspberrypi kernel: [12343.878655] [<c009a22c>] (out_of_memory+0x26c/0x2b8) from [<c009df50>] (__alloc_pages_nodemask+0x808/0x83c)
Dec 28 16:07:48 raspberrypi kernel: [12343.878699] [<c009df50>] (__alloc_pages_nodemask+0x808/0x83c) from [<c00b2c44>] (__pte_alloc+0x20/0x188)
Dec 28 16:07:48 raspberrypi kernel: [12343.878734] [<c00b2c44>] (__pte_alloc+0x20/0x188) from [<c00b5e98>] (handle_mm_fault+0xd0/0xe8)
Dec 28 16:07:48 raspberrypi kernel: [12343.878769] [<c00b5e98>] (handle_mm_fault+0xd0/0xe8) from [<c0403b5c>] (do_page_fault+0x224/0x3c4)
Dec 28 16:07:48 raspberrypi kernel: [12343.878799] [<c0403b5c>] (do_page_fault+0x224/0x3c4) from [<c000834c>] (do_DataAbort+0x34/0x98)
Dec 28 16:07:48 raspberrypi kernel: [12343.878838] [<c000834c>] (do_DataAbort+0x34/0x98) from [<c04025bc>] (__dabt_usr+0x3c/0x40)
Dec 28 16:07:48 raspberrypi kernel: [12343.878857] Exception stack(0xddb37fb0 to 0xddb37ff8)
Dec 28 16:07:48 raspberrypi kernel: [12343.878877] 7fa0:                                     00004019 00020381 23bfec68 23c02c80
Dec 28 16:07:48 raspberrypi kernel: [12343.878899] 7fc0: 23bfec70 00004019 b6b81258 23bfec68 00000398 23bff000 00004028 00024000
Dec 28 16:07:48 raspberrypi kernel: [12343.878919] 7fe0: 00000001 bee34998 b6b17d7c b6ac6474 60000010 ffffffff
Dec 28 16:07:48 raspberrypi kernel: [12343.878931] Mem-info:
Dec 28 16:07:48 raspberrypi kernel: [12343.878942] Normal per-cpu:
Dec 28 16:07:48 raspberrypi kernel: [12343.878956] CPU    0: hi:  186, btch:  31 usd:  81
Dec 28 16:07:48 raspberrypi kernel: [12343.878986] active_anon:58433 inactive_anon:57606 isolated_anon:0
Dec 28 16:07:48 raspberrypi kernel: [12343.878986]  active_file:37 inactive_file:539 isolated_file:0
Dec 28 16:07:48 raspberrypi kernel: [12343.878986]  unevictable:2 dirty:0 writeback:0 unstable:0
Dec 28 16:07:48 raspberrypi kernel: [12343.878986]  free:3440 slab_reclaimable:410 slab_unreclaimable:1057
Dec 28 16:07:48 raspberrypi kernel: [12343.878986]  mapped:71 shmem:3 pagetables:444 bounce:0
Dec 28 16:07:48 raspberrypi kernel: [12343.878986]  free_cma:1398
Dec 28 16:07:48 raspberrypi kernel: [12343.879054] Normal free:13760kB min:8192kB low:10240kB high:12288kB active_anon:233732kB inactive_anon:230424kB active_file:148kB inactive_file:2156kB unevictable:8kB isolated(anon):0kB isolated(file):0kB present:507904kB managed:480420kB mlocked:8kB dirty:0kB writeback:0kB mapped:284kB shmem:12kB slab_reclaimable:1640kB slab_unreclaimable:4228kB kernel_stack:1056kB pagetables:1776kB unstable:0kB bounce:0kB free_cma:5592kB writeback_tmp:0kB pages_scanned:15475 all_unreclaimable? yes
Dec 28 16:07:48 raspberrypi kernel: [12343.879071] lowmem_reserve[]: 0 0
Dec 28 16:07:48 raspberrypi kernel: [12343.879087] Normal: 1432*4kB (UEC) 0*8kB 0*16kB 1*32kB (R) 1*64kB (R) 0*128kB 1*256kB (R) 1*512kB (R) 1*1024kB (R) 1*2048kB (R) 1*4096kB (R) = 13760kB
Dec 28 16:07:48 raspberrypi kernel: [12343.879154] 673 total pagecache pages
Dec 28 16:07:48 raspberrypi kernel: [12343.879170] 94 pages in swap cache
Dec 28 16:07:48 raspberrypi kernel: [12343.879183] Swap cache stats: add 26383, delete 26289, find 275/343
Dec 28 16:07:48 raspberrypi kernel: [12343.879193] Free swap  = 48kB
Dec 28 16:07:48 raspberrypi kernel: [12343.879202] Total swap = 102396kB
Dec 28 16:07:48 raspberrypi kernel: [12343.911996] 126976 pages of RAM
Dec 28 16:07:48 raspberrypi kernel: [12343.912023] 3639 free pages
Dec 28 16:07:48 raspberrypi kernel: [12343.912034] 2739 reserved pages
Dec 28 16:07:48 raspberrypi kernel: [12343.912042] 1011 slab pages
Dec 28 16:07:48 raspberrypi kernel: [12343.912051] 144 pages shared
Dec 28 16:07:48 raspberrypi kernel: [12343.912060] 106 pages swap cached
Dec 28 16:07:48 raspberrypi kernel: [12343.912071] [ pid ]   uid  tgid total_vm      rss nr_ptes swapents oom_score_adj name
Dec 28 16:07:48 raspberrypi kernel: [12343.912116] [  156]     0   156      721        0       5      133         -1000 udevd
Dec 28 16:07:48 raspberrypi kernel: [12343.912197] [  287]     0   287      720        0       5      135         -1000 udevd
Dec 28 16:07:48 raspberrypi kernel: [12343.912219] [  288]     0   288      720        0       5      134         -1000 udevd
Dec 28 16:07:48 raspberrypi kernel: [12343.912275] [ 1526]     0  1526     1224        0       5      428             0 dhclient
Dec 28 16:07:48 raspberrypi kernel: [12343.912298] [ 1828]     0  1828   141151   115676     278    23390             0 aria2c
Dec 28 16:07:48 raspberrypi kernel: [12343.912318] [ 1879]     0  1879     6993       58       7       58             0 rsyslogd
Dec 28 16:07:48 raspberrypi kernel: [12343.912338] [ 1932]   104  1932      794       17       5       50             0 dbus-daemon
Dec 28 16:07:48 raspberrypi kernel: [12343.912372] [ 1947]     0  1947     2473       18       8       99             0 nmbd
Dec 28 16:07:48 raspberrypi kernel: [12343.912393] [ 1968]   107  1968     1321       17       6       36             0 dnsmasq
Dec 28 16:07:48 raspberrypi kernel: [12343.912412] [ 1970]     0  1970     4705       67      12      102             0 smbd
Dec 28 16:07:48 raspberrypi kernel: [12343.912431] [ 1989]     0  1989     4834       30      11      137             0 smbd
Dec 28 16:07:48 raspberrypi kernel: [12343.912451] [ 2050]     0  2050     1086       18       6       23             0 cron
Dec 28 16:07:48 raspberrypi kernel: [12343.912470] [ 2087]   102  2087     1378       50       6       57             0 ntpd
Dec 28 16:07:48 raspberrypi kernel: [12343.912491] [ 2113]     0  2113     6600       19       7       11             0 rngd
Dec 28 16:07:48 raspberrypi kernel: [12343.912510] [ 2135]     0  2135     1553        6       5      100         -1000 sshd
Dec 28 16:07:48 raspberrypi kernel: [12343.912530] [ 2178] 65534  2178      504        4       5       26             0 thd
Dec 28 16:07:48 raspberrypi kernel: [12343.912549] [ 2184]    33  2184     1667       71       7      139             0 lighttpd
Dec 28 16:07:48 raspberrypi kernel: [12343.912578] [ 2210]     0  2210     1073        0       6       32             0 getty
Dec 28 16:07:48 raspberrypi kernel: [12343.912598] [ 2211]     0  2211     1073        0       6       32             0 getty
Dec 28 16:07:48 raspberrypi kernel: [12343.912620] [ 2212]     0  2212     1073        0       6       32             0 getty
Dec 28 16:07:48 raspberrypi kernel: [12343.912639] [ 2213]     0  2213     1073        0       6       32             0 getty
Dec 28 16:07:48 raspberrypi kernel: [12343.912658] [ 2214]     0  2214     1073        0       4       32             0 getty
Dec 28 16:07:48 raspberrypi kernel: [12343.912677] [ 2215]     0  2215     1073        0       6       32             0 getty
Dec 28 16:07:48 raspberrypi kernel: [12343.912696] [ 2216]     0  2216      516        0       5       31             0 getty
Dec 28 16:07:48 raspberrypi kernel: [12343.912717] [ 2230]     0  2230     6885        0      10      268             0 console-kit-dae
Dec 28 16:07:48 raspberrypi kernel: [12343.912738] [ 2297]     0  2297     5571        0       8      119             0 polkitd
Dec 28 16:07:48 raspberrypi kernel: [12343.912860] Out of memory: Kill process 1828 (aria2c) score 900 or sacrifice child
Dec 28 16:07:48 raspberrypi kernel: [12343.912882] Killed process 1828 (aria2c) total-vm:564604kB, anon-rss:462516kB, file-rss:188kB

Posted: **Sat Dec 28, 2013 9:43 pm**

Hi, do you know freebsd jails?

Ok,
man ulimit

and check renice of process

Pol

Posted: **Sun Dec 29, 2013 9:18 am**

allfox,

When a program calls malloc it will get an OK return value even if there is not enough memory available. Later when the program tries to use that memory it will get a page fault, or whatever error it is, if there is not enough memory.

How can this be?

Linux uses "virtual memory". Processes do not use real memory addresses but virtual addresses, these are mapped to real addresses in RAM and may well be moved around as the program runs. If more memory is required that is physically available "pages" of RAM might be written out to a swap file and the actual RAM used by another processes temporarily.

Raspian does not have a swap file by default as that causes a lot of wear on the SD card and would be very slow anyway.

I have sometimes enabled a swap file on a USB connected hard drive. It worked quite well.

Linux has an "Out Of Memory Killer" feature that will try to kill a run away process that is "leaking" lots of RAM. As far as I know Linux can sometimes get this wrong and kill some other "well behaved" process instead but normally it does a good job.

Posted: **Sun Dec 29, 2013 9:34 am**

Heater wrote:allfox,

When a program calls malloc it will get an OK return value even if there is not enough memory available. Later when the program tries to use that memory it will get a page fault, or whatever error it is, if there is not enough memory.

How can this be?

Linux uses "virtual memory". Processes do not use real memory addresses but virtual addresses, these are mapped to real addresses in RAM and may well be moved around as the program runs. If more memory is required that is physically available "pages" of RAM might be written out to a swap file and the actual RAM used by another processes temporarily.

Raspian does not have a swap file by default as that causes a lot of wear on the SD card and would be very slow anyway.

I have sometimes enabled a swap file on a USB connected hard drive. It worked quite well.

Linux has an "Out Of Memory Killer" feature that will try to kill a run away process that is "leaking" lots of RAM. As far as I know Linux can sometimes get this wrong and kill some other "well behaved" process instead but normally it does a good job.

Thanks for explanation.

So the Out of Memory Killer normally only kill process which is leaking memory? I got it~

I also got a reply from aria2's author, he didn't encounter these issue on desktop systems, but had heared them on embedded devices, such as rpi or openwrt.

He also asked me if I compiled my aria2 myself, for it could be possiable that I built a bad binary. I do compiled my own aria2, so now I switched to default aria2 from raspbian repo. Up to now, it works well.

Posted: **Sun Dec 29, 2013 3:08 pm**

Well, perhaps an application is buggy and leaks lot's of memory over time. But perhaps also it is just a fact that the app needs a lot of memory when running normally. That may be in the nature of what it does or the lazy way it was written:)

I guess you are about to find out...

Posted: **Sun Dec 29, 2013 4:30 pm**

Heater wrote:Well, perhaps an application is buggy and leaks lot's of memory over time. But perhaps also it is just a fact that the app needs a lot of memory when running normally. That may be in the nature of what it does or the lazy way it was written:)

I guess you are about to find out...

Aria2 advertise herself as lightweight(http://aria2.sourceforge.net/), so she can't be written in a lazy way. Also, I think a lazy programmer won't boring keep helping user on a low volume forum.(https://sourceforge.net/apps/phpbb/aria2/index.php)

I know two facts: 1 - The default aria 1.15.1 came with Raspbian works well. 2 - The author didn't encounter leaking on a desktop system with aria 1.18.1.
I'll try to find something. Frankly, I even don't know where the problem is. The author asked me to post dependency libs to him. I guess I'll help him.

Raspberry Pi Forums

Is there a way to limit program's memory usage?

Is there a way to limit program's memory usage?

Re: Is there a way to limit program's memory usage?

Re: Is there a way to limit program's memory usage?

Re: Is there a way to limit program's memory usage?

Re: Is there a way to limit program's memory usage?

Re: Is there a way to limit program's memory usage?