Sickbeard and Sabnzbd usage, one big issue


14 posts
by Furyio » Wed Jun 20, 2012 6:14 pm
I have my RaspberryPi setup not with torrents, but with NZBs .

I have Sickbeard running perfectly and it snatches a new episode of my TV show from the various NZB index sites, and then forwards it to SabNZBD to download (also working eprfectly).

The problems arise after a short period of downloading.

It appears that the download just freezes. The PI doesn't crash, it just doesnt really push through at any decent speeds. It goes from a really nice 6-10mb a sec dl to 0 and just sits there.

The SD card has plenty of room, but I have the setup as follows currently( for a full run)

a) Sickbeard pulls the TV show and snatches the NZB file and forwards to SAB
b) SAB starts the download
c) When download is complete, sickbeard script runs to process the file, and move it correctly to my NAS ( configured fine and working)
d) SAB deletes the remaining files/folders after the post-processing is complete.

But I guess I'm hitting snags with the downloads loosing their speed until they just sit there. I've assumed that maybe the problem is the write speed to the SD card is not fast enough. I can hookup a USB HDD and have the file temporaliy downloaded there until post processing.

anyone had any success with getting the above working as such?
Raspberry Pi Debian Squeeze
IRC channels : #raspi.ie #d3.ie #att #airsofter.ie
Posts: 21
Joined: Tue Jun 05, 2012 7:02 am
Location: Dublin,Ireland
by dom » Wed Jun 20, 2012 6:37 pm
Any errors in dmesg log?
Any sabnzbd errors in log?
Could be running out of memory and a process is getting killed. Make sure you are using the 224M split, and swap is enabled.
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4012
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge
by Furyio » Wed Jun 20, 2012 7:58 pm
What is the dmesg log?

I'll check them both and post to pastebin
Raspberry Pi Debian Squeeze
IRC channels : #raspi.ie #d3.ie #att #airsofter.ie
Posts: 21
Joined: Tue Jun 05, 2012 7:02 am
Location: Dublin,Ireland
by dom » Wed Jun 20, 2012 8:11 pm
Furyio wrote:What is the dmesg log?

Type in dmesg. Logging stuff gets printed. It will often include kernel error messages.
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4012
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge
by Furyio » Wed Jun 20, 2012 8:28 pm
Raspberry pi logs
http://pastebin.com/xQfbCfPK

For some reason my sabnzbd wont start properly now, I know I had a crash the other day from something else but seems to have forgotten everything.

Might un install and reinstall everything during the week and see what happens.
Then I'll come back with logs etc.
Raspberry Pi Debian Squeeze
IRC channels : #raspi.ie #d3.ie #att #airsofter.ie
Posts: 21
Joined: Tue Jun 05, 2012 7:02 am
Location: Dublin,Ireland
by dom » Wed Jun 20, 2012 8:32 pm
Okay, well your sdcard is throwing lots of errors. That's problem the source of your problems.

If you are starting again, use Debian Wheezy:
http://www.raspberrypi.org/archives/1435

it has newer firmware with better compatibility with sdcards (and speed improvements).
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4012
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge
by Furyio » Wed Jun 20, 2012 8:39 pm
O well it seems that SAB started working.

http://pastebin.com/LQpzCgVF

Wheezy you say, I wouldnt mind setting it up again, I guess the issue is that I use my Pi as an IRC bouncer, and have a pretty highly customised irssi config, wonder if I can back it up and load it back onto the new formatted card.

I'll give wheezy a bang sure and let you know how I get on

Thanks
Raspberry Pi Debian Squeeze
IRC channels : #raspi.ie #d3.ie #att #airsofter.ie
Posts: 21
Joined: Tue Jun 05, 2012 7:02 am
Location: Dublin,Ireland
by eddie4 » Sun Jun 24, 2012 12:30 pm
You should try setting it up whit NZBGet. It's way more efficient as it is written in C++ instead of python. Although it is less use friendly. There are two packages one is the downloader one is the web interface. bottom line if you have little to no linux skill don't try it but it is a better solution then sabNZBd. And it's supported by Sickbeared and Couchpotato.
Posts: 8
Joined: Sun Jun 24, 2012 12:27 pm
by estaga » Mon Jun 25, 2012 10:19 pm
Exact same issue here,

This issue should not happen irrespective of which NZB handler is installed ("it is just a Linux box"), so I'll ignore the above comment and focus on the issue experienced rather than software that can avoid the issue if that's allright.

In my instance, SABNzbd likes to run a while and then the Raspberry crashes after some time. (I see the Torrent thread has similar issues).

I'm not really in the mood to update to Wheezy due to the amount of config already done on my image (and surely the firmware+kernel would be the issue, not a particular package), however, I've just done a rpi-update ( https://github.com/Hexxeh/rpi-update) today (to update the Firmware & Kernel (to try and get rid of the issue) and it's still present with kernel 3.1.9+ #135, although I would say it definitely takes longer before crashing.

Setup:
ModelB, Wired Ethernet, Headless, 16gb SD, typically no Swap, however I can reproduce the issue with or without swap on a 5gb swap partition on the SD card (yes I know it's bad but wanted to eliminate other options), ~11Gb left for rootfs and whatever else.

Checking syslog (tail -f /var/log/syslog) I can see that the ethernet module is having some weird issues after SABNzbd starts downloading, however the timing of the printing of the errors does not co-inside with a crash:

Code: Select all
Jun 25 22:32:25 pi01 kernel: [  230.096566] smsc95xx 1-1.1:1.0: eth0: kevent 2 may have been dropped
Jun 25 22:32:25 pi01 kernel: [  230.096593] smsc95xx 1-1.1:1.0: eth0: kevent 2 may have been dropped
Jun 25 22:32:25 pi01 kernel: [  230.096631] smsc95xx 1-1.1:1.0: eth0: kevent 2 may have been dropped
Jun 25 22:32:25 pi01 kernel: [  230.096660] smsc95xx 1-1.1:1.0: eth0: kevent 2 may have been dropped


This happens quite a lot, about 200k times today alone and around ~1500 times per second when it happens:
Code: Select all
root@pi01:~# grep "smsc95xx 1-1.1:1.0: eth0: kevent 2 may have been dropped" /var/log/syslog | wc -l
194065
root@pi01:~# grep smsc95xx /var/log/syslog | grep "21:27:09" | wc -l
1516


I also noticed, that before I mounted my swap, that I would get several of the following exceptions in the syslog. Whilst I do agree that it is evident to memory exhaustion I do have to say that the following log lines does NOT show (and still crashes) with my swap partition on.. (excuse the length of the trace):

Code: Select all
Jun 25 22:25:32 pi01 kernel: [ 4111.086368] IP: queue_glue: no memory for gluing queue cb9fc140
Jun 25 22:25:32 pi01 kernel: [ 4111.086496] IP: queue_glue: no memory for gluing queue cb9fc140
Jun 25 22:25:34 pi01 kernel: [ 4113.921740] warn_alloc_failed: 105231 callbacks suppressed
Jun 25 22:25:34 pi01 kernel: [ 4113.921773] sabnzbdplus: page allocation failure: order:3, mode:0x20
Jun 25 22:25:34 pi01 kernel: [ 4113.921843] [<c00153d4>] (unwind_backtrace+0x0/0xfc) from [<c03fba24>] (dump_stack+0x20
/0x24)
Jun 25 22:25:34 pi01 kernel: [ 4113.921894] [<c03fba24>] (dump_stack+0x20/0x24) from [<c00c272c>] (warn_alloc_failed+0x
c4/0x120)
Jun 25 22:25:34 pi01 kernel: [ 4113.921938] [<c00c272c>] (warn_alloc_failed+0xc4/0x120) from [<c00c586c>] (__alloc_page
s_nodemask+0x46c/0x6a8)
Jun 25 22:25:34 pi01 kernel: [ 4113.921983] [<c00c586c>] (__alloc_pages_nodemask+0x46c/0x6a8) from [<c00f2de4>] (cache_
alloc_refill+0x330/0x620)
Jun 25 22:25:34 pi01 kernel: [ 4113.922020] [<c00f2de4>] (cache_alloc_refill+0x330/0x620) from [<c00f3418>] (__kmalloc+
0x198/0x1ac)
Jun 25 22:25:34 pi01 kernel: [ 4113.922057] [<c00f3418>] (__kmalloc+0x198/0x1ac) from [<c0343794>] (pskb_expand_head+0x
168/0x2d4)
Jun 25 22:25:34 pi01 kernel: [ 4113.922102] [<c0343794>] (pskb_expand_head+0x168/0x2d4) from [<c037894c>] (ip_defrag+0x
618/0xbe4)
Jun 25 22:25:34 pi01 kernel: [ 4113.922139] [<c037894c>] (ip_defrag+0x618/0xbe4) from [<c0377b04>] (ip_local_deliver+0x
8c/0xc8)
Jun 25 22:25:34 pi01 kernel: [ 4113.922175] [<c0377b04>] (ip_local_deliver+0x8c/0xc8) from [<c03773e0>] (ip_rcv_finish+
0x12c/0x394)
Jun 25 22:25:34 pi01 kernel: [ 4113.922209] [<c03773e0>] (ip_rcv_finish+0x12c/0x394) from [<c0377d90>] (ip_rcv+0x250/0x
310)
Jun 25 22:25:34 pi01 kernel: [ 4113.922253] [<c0377d90>] (ip_rcv+0x250/0x310) from [<c034b53c>] (__netif_receive_skb+0x
2a4/0x510)
Jun 25 22:25:34 pi01 kernel: [ 4113.922291] [<c034b53c>] (__netif_receive_skb+0x2a4/0x510) from [<c034b844>] (process_b
acklog+0x9c/0x154)
Jun 25 22:25:34 pi01 kernel: [ 4113.922329] [<c034b844>] (process_backlog+0x9c/0x154) from [<c034bd04>] (net_rx_action+
0x124/0x2d8)
Jun 25 22:25:34 pi01 kernel: [ 4113.922380] [<c034bd04>] (net_rx_action+0x124/0x2d8) from [<c003489c>] (__do_softirq+0x
c0/0x230)
Jun 25 22:25:34 pi01 kernel: [ 4113.922419] [<c003489c>] (__do_softirq+0xc0/0x230) from [<c0034ef4>] (irq_exit+0xa8/0xb4)
Jun 25 22:25:34 pi01 kernel: [ 4113.922466] [<c0034ef4>] (irq_exit+0xa8/0xb4) from [<c000f110>] (handle_IRQ+0x44/0x94)
Jun 25 22:25:34 pi01 kernel: [ 4113.922502] [<c000f110>] (handle_IRQ+0x44/0x94) from [<c0008498>] (asm_do_IRQ+0x18/0x1c)
Jun 25 22:25:34 pi01 kernel: [ 4113.922544] [<c0008498>] (asm_do_IRQ+0x18/0x1c) from [<c03fef80>] (__irq_usr+0x40/0xc0)
Jun 25 22:25:34 pi01 kernel: [ 4113.922567] Exception stack(0xc747ffb0 to 0xc747fff8)
Jun 25 22:25:34 pi01 kernel: [ 4113.922592] ffa0:                                     0250cdb0 405bd240 b9aee1f4 00000000
Jun 25 22:25:34 pi01 kernel: [ 4113.922623] ffc0: 0250cdb0 405bd240 b9aee1f4 0395c910 00000000 021e6810 0250cdb0 0020f27c
Jun 25 22:25:34 pi01 kernel: [ 4113.922650] ffe0: 00053564 47e48584 00054480 00053568 60000010 ffffffff
Jun 25 22:25:34 pi01 kernel: [ 4113.922667] Mem-info:
Jun 25 22:25:34 pi01 kernel: [ 4113.922679] Normal per-cpu:
Jun 25 22:25:34 pi01 kernel: [ 4113.922695] CPU    0: hi:   90, btch:  15 usd:  13
Jun 25 22:25:34 pi01 kernel: [ 4113.922727] active_anon:5579 inactive_anon:12770 isolated_anon:14
Jun 25 22:25:34 pi01 kernel: [ 4113.922736]  active_file:3496 inactive_file:11135 isolated_file:30
Jun 25 22:25:34 pi01 kernel: [ 4113.922745]  unevictable:0 dirty:480 writeback:649 unstable:0
Jun 25 22:25:34 pi01 kernel: [ 4113.922753]  free:4387 slab_reclaimable:1206 slab_unreclaimable:6620
Jun 25 22:25:34 pi01 kernel: [ 4113.922762]  mapped:1004 shmem:64 pagetables:467 bounce:0
Jun 25 22:25:34 pi01 kernel: [ 4113.922811] Normal free:17548kB min:8192kB low:10240kB high:12288kB active_anon:22316kB inactive_anon:51080kB active_file:13984kB inactive_file:44540kB unevictable:0kB isolated(anon):56kB isolated(file):120kB present:195072kB mlocked:0kB dirty:1920kB writeback:2596kB mapped:4016kB shmem:256kB slab_reclaimable:4824kB slab_unreclaimable:26480kB kernel_stack:1840kB pagetables:1868kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Jun 25 22:25:34 pi01 kernel: [ 4113.922864] lowmem_reserve[]: 0 0
Jun 25 22:25:34 pi01 kernel: [ 4113.922885] Normal: 85*4kB 809*8kB 645*16kB 1*32kB 0*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 17548kB
Jun 25 22:25:34 pi01 kernel: [ 4113.922960] 17948 total pagecache pages
Jun 25 22:25:34 pi01 kernel: [ 4113.922973] 3221 pages in swap cache
Jun 25 22:25:34 pi01 kernel: [ 4113.922988] Swap cache stats: add 29469, delete 26248, find 21595/25155
Jun 25 22:25:34 pi01 kernel: [ 4113.923004] Free swap  = 5055568kB
Jun 25 22:25:34 pi01 kernel: [ 4113.923016] Total swap = 5081084kB
Jun 25 22:25:34 pi01 kernel: [ 4113.932617] 49152 pages of RAM
Jun 25 22:25:34 pi01 kernel: [ 4113.932632] 4803 free pages
Jun 25 22:25:34 pi01 kernel: [ 4113.932643] 2126 reserved pages
Jun 25 22:25:34 pi01 kernel: [ 4113.932654] 7826 slab pages
Jun 25 22:25:34 pi01 kernel: [ 4113.932664] 6231 pages shared
Jun 25 22:25:34 pi01 kernel: [ 4113.932675] 3221 pages swap cached
Jun 25 22:25:34 pi01 kernel: [ 4113.932834] IP: queue_glue: no memory for gluing queue cacef180
Jun 25 22:25:34 pi01 kernel: [ 4113.933073] sabnzbdplus: page allocation failure: order:3, mode:0x20
Jun 25 22:25:34 pi01 kernel: [ 4113.933145] [<c00153d4>] (unwind_backtrace+0x0/0xfc) from [<c03fba24>] (dump_stack+0x20/0x24)
Jun 25 22:25:34 pi01 kernel: [ 4113.933196] [<c03fba24>] (dump_stack+0x20/0x24) from [<c00c272c>] (warn_alloc_failed+0xc4/0x120)
Jun 25 22:25:34 pi01 kernel: [ 4113.933240] [<c00c272c>] (warn_alloc_failed+0xc4/0x120) from [<c00c586c>] (__alloc_pages_nodemask+0x46c/0x6a8)
Jun 25 22:25:34 pi01 kernel: [ 4113.933283] [<c00c586c>] (__alloc_pages_nodemask+0x46c/0x6a8) from [<c00f2de4>] (cache_alloc_refill+0x330/0x620)
Jun 25 22:25:34 pi01 kernel: [ 4113.933321] [<c00f2de4>] (cache_alloc_refill+0x330/0x620) from [<c00f3418>] (__kmalloc+0x198/0x1ac)
Jun 25 22:25:34 pi01 kernel: [ 4113.933357] [<c00f3418>] (__kmalloc+0x198/0x1ac) from [<c0343794>] (pskb_expand_head+0x168/0x2d4)
Jun 25 22:25:34 pi01 kernel: [ 4113.933402] [<c0343794>] (pskb_expand_head+0x168/0x2d4) from [<c037894c>] (ip_defrag+0x618/0xbe4)
Jun 25 22:25:34 pi01 kernel: [ 4113.933438] [<c037894c>] (ip_defrag+0x618/0xbe4) from [<c0377b04>] (ip_local_deliver+0x8c/0xc8)
Jun 25 22:25:34 pi01 kernel: [ 4113.933474] [<c0377b04>] (ip_local_deliver+0x8c/0xc8) from [<c03773e0>] (ip_rcv_finish+0x12c/0x394)
Jun 25 22:25:34 pi01 kernel: [ 4113.933509] [<c03773e0>] (ip_rcv_finish+0x12c/0x394) from [<c0377d90>] (ip_rcv+0x250/0x310)
Jun 25 22:25:34 pi01 kernel: [ 4113.933552] [<c0377d90>] (ip_rcv+0x250/0x310) from [<c034b53c>] (__netif_receive_skb+0x2a4/0x510)
Jun 25 22:25:34 pi01 kernel: [ 4113.933590] [<c034b53c>] (__netif_receive_skb+0x2a4/0x510) from [<c034b844>] (process_backlog+0x9c/0x154)
Jun 25 22:25:34 pi01 kernel: [ 4113.933627] [<c034b844>] (process_backlog+0x9c/0x154) from [<c034bd04>] (net_rx_action+0x124/0x2d8)
Jun 25 22:25:34 pi01 kernel: [ 4113.933676] [<c034bd04>] (net_rx_action+0x124/0x2d8) from [<c003489c>] (__do_softirq+0xc0/0x230)
Jun 25 22:25:34 pi01 kernel: [ 4113.933717] [<c003489c>] (__do_softirq+0xc0/0x230) from [<c0034ef4>] (irq_exit+0xa8/0xb4)
Jun 25 22:25:34 pi01 kernel: [ 4113.933763] [<c0034ef4>] (irq_exit+0xa8/0xb4) from [<c000f110>] (handle_IRQ+0x44/0x94)
Jun 25 22:25:34 pi01 kernel: [ 4113.933799] [<c000f110>] (handle_IRQ+0x44/0x94) from [<c0008498>] (asm_do_IRQ+0x18/0x1c)
Jun 25 22:25:34 pi01 kernel: [ 4113.933841] [<c0008498>] (asm_do_IRQ+0x18/0x1c) from [<c03fef80>] (__irq_usr+0x40/0xc0)
Jun 25 22:25:34 pi01 kernel: [ 4113.933864] Exception stack(0xc747ffb0 to 0xc747fff8)
Jun 25 22:25:34 pi01 kernel: [ 4113.933887] ffa0:                                     0250cdb0 405bd240 b9aee1f4 00000000
Jun 25 22:25:34 pi01 kernel: [ 4113.933917] ffc0: 0250cdb0 405bd240 b9aee1f4 0395c910 00000000 021e6810 0250cdb0 0020f27c
Jun 25 22:25:34 pi01 kernel: [ 4113.933944] ffe0: 00053564 47e48584 00054480 00053568 60000010 ffffffff
Jun 25 22:25:34 pi01 kernel: [ 4113.933961] Mem-info:
Jun 25 22:25:34 pi01 kernel: [ 4113.933972] Normal per-cpu:
Jun 25 22:25:34 pi01 kernel: [ 4113.933988] CPU    0: hi:   90, btch:  15 usd:  13
Jun 25 22:25:34 pi01 kernel: [ 4113.934020] active_anon:5579 inactive_anon:12770 isolated_anon:14
Jun 25 22:25:34 pi01 kernel: [ 4113.934030]  active_file:3496 inactive_file:11135 isolated_file:30
Jun 25 22:25:34 pi01 kernel: [ 4113.934038]  unevictable:0 dirty:480 writeback:649 unstable:0
Jun 25 22:25:34 pi01 kernel: [ 4113.934047]  free:4387 slab_reclaimable:1206 slab_unreclaimable:6620
Jun 25 22:25:34 pi01 kernel: [ 4113.934056]  mapped:1004 shmem:64 pagetables:467 bounce:0
Jun 25 22:25:34 pi01 kernel: [ 4113.934105] Normal free:17548kB min:8192kB low:10240kB high:12288kB active_anon:22316kB inactive_anon:51080kB active_file:13984kB inactive_file:44540kB unevictable:0kB isolated(anon):56kB isolated(file):120kB present:195072kB mlocked:0kB dirty:1920kB writeback:2596kB mapped:4016kB shmem:256kB slab_reclaimable:4824kB slab_unreclaimable:26480kB kernel_stack:1840kB pagetables:1868kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
Jun 25 22:25:34 pi01 kernel: [ 4113.934159] lowmem_reserve[]: 0 0
Jun 25 22:25:34 pi01 kernel: [ 4113.934180] Normal: 85*4kB 809*8kB 645*16kB 1*32kB 0*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 17548kB
Jun 25 22:25:34 pi01 kernel: [ 4113.934253] 17948 total pagecache pages
Jun 25 22:25:34 pi01 kernel: [ 4113.934266] 3221 pages in swap cache
Jun 25 22:25:34 pi01 kernel: [ 4113.934281] Swap cache stats: add 29469, delete 26248, find 21595/25155
Jun 25 22:25:34 pi01 kernel: [ 4113.934296] Free swap  = 5055568kB
Jun 25 22:25:34 pi01 kernel: [ 4113.934308] Total swap = 5081084kB
Jun 25 22:25:34 pi01 kernel: [ 4113.943836] 49152 pages of RAM
Jun 25 22:25:34 pi01 kernel: [ 4113.943850] 4803 free pages
Jun 25 22:25:34 pi01 kernel: [ 4113.943861] 2126 reserved pages
Jun 25 22:25:34 pi01 kernel: [ 4113.943873] 7826 slab pages
Jun 25 22:25:34 pi01 kernel: [ 4113.943883] 6231 pages shared
Jun 25 22:25:34 pi01 kernel: [ 4113.943894] 3221 pages swap cached
Jun 25 22:25:34 pi01 kernel: [ 4113.944053] IP: queue_glue: no memory for gluing queue cacef180
Jun 25 22:25:35 pi01 kernel: [ 4113.967631] kworker/0:2: page allocation failure: order:3, mode:0x20
Jun 25 22:25:35 pi01 kernel: [ 4113.967714] [<c00153d4>] (unwind_backtrace+0x0/0xfc) from [<c03fba24>] (dump_stack+0x20/0x24)
Jun 25 22:25:35 pi01 kernel: [ 4113.967767] [<c03fba24>] (dump_stack+0x20/0x24) from [<c00c272c>] (warn_alloc_failed+0xc4/0x120)
Jun 25 22:25:35 pi01 kernel: [ 4113.967812] [<c00c272c>] (warn_alloc_failed+0xc4/0x120) from [<c00c586c>] (__alloc_pa-1.1:1.0: eth0: kevent 2 may have been dropped
Jun 25 22:25:35 pi01 kernel: [ 4114.457641]-1.1:1.0: eth0: kevent 2 may have been dropped
Jun 25 22:25:35 pi01 kernel: [ 4114.460773] smsc95xx 1-1.1:1.0: eth0: kevent 2 may have been dropped


As mentioned, the above does NOT result in a crash of the Raspberry Pi, however, I'm sure similar traces to the above in kernel 3.1.9+ #125 just before a crash.

From all of the above though it seems like investigations should be in the direction of how memory is allocated, the smsc95xx module and kernel event scheduler?

I guess a good starting point would be to have people tail the syslog (as above) and see if they see the same lines as I do during a normal SABNZbd download and/or before a crash.
Posts: 1
Joined: Mon Jun 25, 2012 11:03 am
by guisacouto » Sat Jun 30, 2012 4:08 pm
Just to confirm. I now don't have the kernel panic when downloading torrents, the client just crashes.

The logs are similar:
hundreds of these lines
Code: Select all
smsc95xx 1-1.1:1.0: eth0: kevent 2 may have been dropped


In the end I get a message saying that I was out of memory and that transmission-daemon was killed.

I'm sorry if I'm mixing usenet and torrents matter but now I think it's kind of proven that its the same issue, and that the real problem is not program/protocol specific, but something related with high network usage
Posts: 35
Joined: Mon Feb 20, 2012 12:46 am
by dom » Sat Jun 30, 2012 4:13 pm
You probably have a line like:
vm.min_free_kbytes = 8192
in /etc/sysctl.conf.

Can you try making it larger (e.g. 16384)
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4012
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge
by guisacouto » Sat Jun 30, 2012 4:44 pm
I do have that line but with 21MB of RAM.
Thats exactly why my system keeps working. If this line is < 21 (I've tried with 16) the all system crashes instead of just the transmission daemon. This is probably because the kernel needs more than 16 MB in this situation, and if I don't limit the min_kernel_bytes to 21MB then when the kernel needs memory it won't have it and then there is full system crash.
If it does have the 21MB (or some value between 16 and 21, I haven't found it), then the kernel can work, and it is able to stop the growth of memory needed by transmission, by killing its process. I have tested this with htop opened and I can see that transmission starts ok for a while with the system consuming a total of 50MB.. at some moment it starts to slowly rase the memory usage to it's limit (128, however it only detects 120 in htop).
When I have the min_free_kbytes set to 21MB, I see the transmission-daemon getting killed when the system is using ~98MB which makes total sense 120MB - 98MB = 22 ~ 21 = min_free_kbytes.

This is the dmesg content:

Code: Select all
smsc95xx 1-1.1:1.0: eth0: kevent 2 may have been dropped
.........(more of these lines)......
smsc95xx 1-1.1:1.0: eth0: kevent 2 may have been dropped
smsc95xx 1-1.1:1.0: eth0: kevent 2 may have been dropped
transmission-da invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0
transmission-da cpuset=/ mems_allowed=0
Backtrace:
Function entered at [<c0011870>] from [<c0478188>]
 r6:00000000 r5:00000000 r4:c7ba4420 r3:00000000
Function entered at [<c0478170>] from [<c00737fc>]
Function entered at [<c0073774>] from [<c0073abc>]
Function entered at [<c0073a44>] from [<c0074150>]
Function entered at [<c007406c>] from [<c007752c>]
Function entered at [<c0076eb8>] from [<c0072a50>]
Function entered at [<c00727e4>] from [<c0088d0c>]
Function entered at [<c0088c98>] from [<c008b8dc>]
Function entered at [<c008b860>] from [<c008c0fc>]
Function entered at [<c008c044>] from [<c0015dcc>]
Function entered at [<c0015c2c>] from [<c0008404>]
Function entered at [<c00083c8>] from [<c000e218>]
Exception stack(0xc699ffb0 to 0xc699fff8)
ffa0:                                     00000000 415fec40 415fecc0 415fed40
ffc0: 415fec38 415fec24 4170e348 4170e3d0 0010f268 fffffe70 40d82850 415fec20
ffe0: 00000001 415febf0 4047b578 4048f960 20000010 ffffffff
 r8:0010f268 r7:4170e3d0 r6:ffffffff r5:20000010 r4:4048f960
Mem-info:
Normal per-cpu:
CPU    0: hi:   42, btch:   7 usd:  32
active_anon:0 inactive_anon:12 isolated_anon:23
 active_file:20 inactive_file:30 isolated_file:0
 unevictable:0 dirty:0 writeback:20 unstable:0
 free:4097 slab_reclaimable:352 slab_unreclaimable:25394

 mapped:4 shmem:0 pagetables:188 bounce:0
Normal free:16388kB min:16384kB low:20480kB high:24576kB active_anon:0kB inactive_anon:48kB active_file:80kB inactive_file:120kB unevictable:0kB isolated(anon):92kB isolated(file):0kB present:130048kB mlocked:0kB dirty:0kB writeback:80kB mapped:16kB shmem:0kB slab_reclaimable:1408kB slab_unreclaimable:101576kB kernel_stack:632kB pagetables:752kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1018 all_unreclaimable? yes
lowmem_reserve[]: 0 0
Normal: 283*4kB 257*8kB 221*16kB 278*32kB 0*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 16388kB
81 total pagecache pages
31 pages in swap cache
Swap cache stats: add 37707, delete 37676, find 6567/11693
Free swap  = 881672kB
Total swap = 917500kB
32768 pages of RAM
4271 free pages
1873 reserved pages
25746 slab pages
23 pages shared
31 pages swap cached
[ pid ]   uid  tgid total_vm      rss cpu oom_adj oom_score_adj name
[   90]     0    90      904        0   0     -17         -1000 systemd-udevd
[  148]     0   148      748        0   0       0             0 mount.ntfs
[  218]     0   218     1024        0   0       0             0 syslog-ng
[  219]     0   219     1761        0   0       0             0 syslog-ng
[  269]     0   269      692        0   0       0             0 crond
[  277]     0   277      986        0   0       0             0 lircd
[  283]     0   283     1561        0   0     -17         -1000 sshd
[  284]     0   284      903        0   0       0             0 login
[  285]     0   285      436        0   0       0             0 agetty
[  286]     0   286      436        0   0       0             0 agetty
[  287]     0   287      436        0   0       0             0 agetty
[  300]  1000   300     1313        0   0       0             0 xbmc
[  316]  1000   316    61207       17   0       0             0 xbmc.bin
[  346]     0   346      480        0   0       0             0 dhcpcd
[  348]     0   348     2452        0   0       0             0 sshd
[  350]  1000   350     2452        0   0       0             0 sshd
[  351]  1000   351     2042        0   0       0             0 zsh
[  403]    87   403      873        0   0       0             0 ntpd
[  404]     0   404      915        0   0       0             0 ntpd
[  433]  1000   433    16491        9   0       0             0 transmission-da
Out of memory: Kill process 433 (transmission-da) score 17 or sacrifice child
Killed process 433 (transmission-da) total-vm:65964kB, anon-rss:28kB, file-rss:8kB


best regards
Posts: 35
Joined: Mon Feb 20, 2012 12:46 am
by niko1986 » Mon Jul 02, 2012 9:23 pm
Used tips here and a few other places and just found a setup which has managed to download, par and unrar a 1.5 GB file for the first time without crashing multiple times and badly corrupted files.

Re: Running on RaspberryPi
by zapt0 » June 30th, 2012, 6:36 pm
- 32M Article Cache Limit (You can use 64M or even more - which lowers the load, but I found the pi occasionally starts to swap with this setting.)
- Enable "Pause Downloading During Post-Processing"

http://forums.sabnzbd.org/viewtopic.php ... 0&start=15

Then I followed this guide http://ubuntuforums.org/showthread.php?t=992706 to setup a cpulimit daemon to limit any process exceeding a set amount of CPU utilisation. ATM I've been using 40%, say sabnzbd and par2 are running unrar obviously wouldn't be running. 80% utilisation should leave enough room for other processes and kernel work.
Posts: 12
Joined: Thu Jun 28, 2012 10:21 am
by Dameek » Wed Jul 18, 2012 3:32 pm
I'm glad you had success....I was just getting ready to start working on a similar setup :mrgreen:
Posts: 1
Joined: Tue Feb 14, 2012 8:51 pm