elatllat
Posts: 1338
Joined: Sat Dec 17, 2011 5:05 pm

setting to stabilize the official image

Mon Aug 13, 2012 12:56 am

Please add "smsc95xx.turbo_mode=N" to /boot/cmdline.txt in the official image, it prevents the rpi from crashing under load.
Also prevents the log from filling with this:

Code: Select all

smsc95xx 1-1.1:1.0: eth0: kevent 2 may have been dropped
4k or USB3 : hardkernel.com

FAQ : raspberrypi.stackexchange.com

https://google.com/search?q=site%3Awww.raspberrypi.org+search+first

https://www.raspberrypi.org/forums/search.php?search_id=unanswered

El_Presidente
Posts: 36
Joined: Tue Apr 03, 2012 7:02 am
Location: Bolton, UK

Re: setting to stabilize the official image

Tue Aug 14, 2012 4:19 pm

Can the logic behind this be explained - i.e. what does the default (which I assume is 'Y') cause the Pi and/or processes to do?
-----------------------------------------------------------------------------------------
Never trust a man, who when left alone with a tea cosey, doesn't try it on.

elatllat
Posts: 1338
Joined: Sat Dec 17, 2011 5:05 pm

Re: setting to stabilize the official image

Tue Aug 14, 2012 4:30 pm

El_Presidente wrote:[What does the default cause the rpi to do?]
Maybe you missed the
elatllat wrote:crashing under load
and
elatllat wrote:[log filed with "kevent 2 may have been dropped"]
part of the OP
4k or USB3 : hardkernel.com

FAQ : raspberrypi.stackexchange.com

https://google.com/search?q=site%3Awww.raspberrypi.org+search+first

https://www.raspberrypi.org/forums/search.php?search_id=unanswered

El_Presidente
Posts: 36
Joined: Tue Apr 03, 2012 7:02 am
Location: Bolton, UK

Re: setting to stabilize the official image

Tue Aug 14, 2012 4:43 pm

I should have explained myself better - I know what the setting aims to cure, but not why the default setting would have been causing the crash under load and error logging.

In other words, what is going on inside the Pi when the turbo mode is on?
-----------------------------------------------------------------------------------------
Never trust a man, who when left alone with a tea cosey, doesn't try it on.

elatllat
Posts: 1338
Joined: Sat Dec 17, 2011 5:05 pm

Re: setting to stabilize the official image

Tue Aug 14, 2012 5:51 pm

El_Presidente wrote:I should have explained myself better - I know what the setting aims to cure, but not why the default setting would have been causing the crash under load and error logging.

In other words, what is going on inside the Pi when the turbo mode is on?
Yes that is a good question but they already know there are driver problems, and are working on them.

The point of this thread is to update the default build with a super quick fix that will reduce mass crash-frustration :)
4k or USB3 : hardkernel.com

FAQ : raspberrypi.stackexchange.com

https://google.com/search?q=site%3Awww.raspberrypi.org+search+first

https://www.raspberrypi.org/forums/search.php?search_id=unanswered

User avatar
Burngate
Posts: 4925
Joined: Thu Sep 29, 2011 4:34 pm
Location: Berkshire UK
Contact: Website

Re: setting to stabilize the official image

Wed Aug 15, 2012 4:41 pm

elatllat wrote:... they already know there are driver problems, and are working on them.
They know, but we would like to know more.

Magic incantations are ok, but some of the background would be interesting

elatllat
Posts: 1338
Joined: Sat Dec 17, 2011 5:05 pm

Re: setting to stabilize the official image

Wed Aug 15, 2012 5:10 pm

Burngate wrote:Magic incantations are ok, but some of the background would be interesting

I have not read (nor can I because some of it is closed source) every line of code on the rpi so it's mostly "magic incantations" to me, but Google said turbo_mode Enables multiple frames per Rx transaction.
http://lxr.free-electrons.com/source/dr ... smsc95xx.c
So clearly there is a problem with that feature in either the ethernet or usb code.
("SMSC95XX USB 2.0 Ethernet Devices")

Do you want to know anything specific?
4k or USB3 : hardkernel.com

FAQ : raspberrypi.stackexchange.com

https://google.com/search?q=site%3Awww.raspberrypi.org+search+first

https://www.raspberrypi.org/forums/search.php?search_id=unanswered

robwriter
Posts: 114
Joined: Sat Feb 25, 2012 6:26 pm

Re: setting to stabilize the official image

Wed Aug 15, 2012 5:10 pm

I don't get random crashes.

elatllat
Posts: 1338
Joined: Sat Dec 17, 2011 5:05 pm

Re: setting to stabilize the official image

Wed Aug 15, 2012 5:25 pm

robwriter wrote:I don't get random crashes.
It's not "random" it's "under load" as stated in the OP.
A torrent client saving to disk is a good example because it stresses CPU and the USB with IO from eth0 and sda.
Copying lage files over the network and other compound loads are also reported to cause crashing.
4k or USB3 : hardkernel.com

FAQ : raspberrypi.stackexchange.com

https://google.com/search?q=site%3Awww.raspberrypi.org+search+first

https://www.raspberrypi.org/forums/search.php?search_id=unanswered

El_Presidente
Posts: 36
Joined: Tue Apr 03, 2012 7:02 am
Location: Bolton, UK

Re: setting to stabilize the official image

Thu Aug 16, 2012 5:13 pm

OK - so I've done my own Googling and find that the turbo mode is meant to make the Ethernet more efficient but at the cost of using more Kernel memory. Putting this together with comments from around the Pi forum, it looks like the turbo mode grabs too much resource when heavily using the network, leading to kevent drops and page allocation issues. Others believe that because the Ethernet is effectively a port off the USB 3-port hub built into the Pi, it is flooding the whole USB subsystem, thus being a contributor to overall crap USB experience as well as kernel panic/crash.

Disabling turbo mode will result in lower Ethernet performance, but less stress on kernel memory and lower hit rate on the USB interrupt processing, but I'm not sure if anyone has produced proper before/after stats yet.

That will do for me. Position understood.
-----------------------------------------------------------------------------------------
Never trust a man, who when left alone with a tea cosey, doesn't try it on.

chrisw2
Posts: 106
Joined: Sat Apr 07, 2012 11:22 am
Location: Manchester, UK

Re: setting to stabilize the official image

Thu Aug 16, 2012 5:25 pm

Thanks, El_Presidente.

User avatar
Burngate
Posts: 4925
Joined: Thu Sep 29, 2011 4:34 pm
Location: Berkshire UK
Contact: Website

Re: setting to stabilize the official image

Thu Aug 16, 2012 5:27 pm

El_Presidente wrote:That will do for me. Position understood.
And me. Thanks

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5097
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: setting to stabilize the official image

Wed Sep 26, 2012 7:24 pm

Has anyone found anything that is worse with:
smsc95xx.turbo_mode=N

I'd assumed it slows down the network performance, but iperf says it is just as fast.

Any downside to this?

Ignas
Posts: 7
Joined: Wed Jul 18, 2012 2:05 pm
Location: Šiauliai, Lithuania
Contact: Website

Re: setting to stabilize the official image

Wed Sep 26, 2012 7:32 pm

Been using this setting for a month now. Pi is running rtorrent 24/7. No crashes, dmesg is clear.

Download speed was a disappointment (at around 2MB/s), but now with turbo mode present I can max out my old wrt54gl (~5-6MB/s). Still stable.

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5097
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: setting to stabilize the official image

Wed Sep 26, 2012 7:45 pm

Ignas wrote:Been using this setting for a month now. Pi is running rtorrent 24/7. No crashes, dmesg is clear.

Download speed was a disappointment (at around 2MB/s), but now with turbo mode present I can max out my old wrt54gl (~5-6MB/s). Still stable.
Any difference in download speed with and without:
smsc95xx.turbo_mode=N

Ignas
Posts: 7
Joined: Wed Jul 18, 2012 2:05 pm
Location: Šiauliai, Lithuania
Contact: Website

Re: setting to stabilize the official image

Wed Sep 26, 2012 8:05 pm

dom wrote:
Ignas wrote:Been using this setting for a month now. Pi is running rtorrent 24/7. No crashes, dmesg is clear.

Download speed was a disappointment (at around 2MB/s), but now with turbo mode present I can max out my old wrt54gl (~5-6MB/s). Still stable.
Any difference in download speed with and without:
smsc95xx.turbo_mode=N
Well, to be honest I've never really tried to reach max download speed without smsc95xx.turbo_mode=N. And I think it's impossible without crashing the Pi itself. I was keeping download limit at 200-300KB/s to prevent dmesg from flooding with "kevent 2 may have been dropped" messages. I would try now, but my Pi is in the remote location and I don't have the luxury of power cycling if anything goes wrong. I've read that loads of "kevent 2 may have been dropped" will eventually lead to the crash of Pi.

elatllat
Posts: 1338
Joined: Sat Dec 17, 2011 5:05 pm

Re: setting to stabilize the official image

Wed Sep 26, 2012 8:12 pm

dom wrote:Has anyone found anything that is worse with:
smsc95xx.turbo_mode=N

I'd assumed it slows down the network performance, but iperf says it is just as fast.

Any downside to this?
No down side that I have seen.
I have not explicitly tested if there is any speed difference because without this setting there is crazy ping times, paket loss, and crashing.
I am able to torrent, decrypt, and stram 2 HD movies off the raspberrypi using this setting.
I get 3MiB/s using scp but ssh eating all the CPU is the limiting factor there.
I get 7 or 8MiB/s using pv over smbfs (external drives: kworker(65%), smbd(28%), etc max out the CPU) sdcard smbd only leaves ~5% idle.

Note that I am stuck using an old kernel:
http://www.raspberrypi.org/phpBB3/viewt ... 5&p=181911
but it seems to have only positive effects the ~205 times it's been used:
https://www.google.com/search?hl=en&q=s ... rbo_mode=N"
4k or USB3 : hardkernel.com

FAQ : raspberrypi.stackexchange.com

https://google.com/search?q=site%3Awww.raspberrypi.org+search+first

https://www.raspberrypi.org/forums/search.php?search_id=unanswered

elnoziya
Posts: 1
Joined: Fri Oct 12, 2012 8:59 am

Re: setting to stabilize the official image

Fri Oct 12, 2012 9:10 am

Hi all:

I have been bearing with transmission-daemon crashes till turning off turbo mode, it seems to be working fine after that. However right now my feeling is that rpi is working much slower. In raspi-config I am using "none" as overclocking option, which one are you using?

elatllat
Posts: 1338
Joined: Sat Dec 17, 2011 5:05 pm

Re: setting to stabilize the official image

Fri Oct 12, 2012 12:39 pm

elnoziya wrote:feeling...slower...overclocking?
ask in another thread how to test and prove your 'feeling'. i'm not overclocking and it is also a different thread, please keep this one clean.
4k or USB3 : hardkernel.com

FAQ : raspberrypi.stackexchange.com

https://google.com/search?q=site%3Awww.raspberrypi.org+search+first

https://www.raspberrypi.org/forums/search.php?search_id=unanswered

milhouse
Posts: 613
Joined: Mon Jan 16, 2012 12:59 pm

Re: setting to stabilize the official image

Wed Nov 07, 2012 8:56 pm

I raised issue #153 to report problems with allocation failures while running Transmission with one or two concurrent downloads, and the suggestion to set smsc95xx.turbo_mode=N has fixed the errors (and associated kevent 2 floods) - I've tested with 5 concurrent torrent downloads and there were no problems (512MB board). There is a slight reduction in maximum network throughput, to the order of about 1.5MB/s, but this is acceptable given the alternative.

User avatar
eshatohhin
Posts: 11
Joined: Tue May 15, 2012 12:27 pm
Location: Estonia

Re: setting to stabilize the official image

Sun Dec 30, 2012 6:00 pm

Setting vm.min_free_kbytes to 16384 in /etc/sysctl.conf made my pi more stable under load.

Source: http://www.nerdoncoffee.com/?p=195
http://blogs.linbit.com/p/28/kernel-min_free_kbytes/

milhouse
Posts: 613
Joined: Mon Jan 16, 2012 12:59 pm

Re: setting to stabilize the official image

Mon Dec 31, 2012 12:44 pm

eshatohhin wrote:Setting vm.min_free_kbytes to 16384 in /etc/sysctl.conf made my pi more stable under load.
See the issue I linked above - even with 32MB (32768) allocated for vm.min_free you'll still get problems. Given the Pi only has 256MB/512MB of RAM, with a chunk of that allocated to the GPU, throwing more RAM at this problem isn't really a solution, even if it succeeded.

While disabling turbo mode does reduce network performance by up 1.5MB/s it does at least ensure 100% network reliability even with only 1536 (1.5MB) allocated to vm.min_free.

elatllat
Posts: 1338
Joined: Sat Dec 17, 2011 5:05 pm

Re: setting to stabilize the official image

Mon Sep 30, 2013 7:57 pm

Tested the latest stable build and it seems

Code: Select all

smsc95xx.turbo_mode=N
is still required for stability and is still not default.

Code: Select all

cat /etc/issue
Raspbian GNU/Linux 7 \n \l
uname -a
Linux raspbnas 3.6.11+ #538 PREEMPT Fri Aug 30 20:42:08 BST 2013 armv6l GNU/Linux
4k or USB3 : hardkernel.com

FAQ : raspberrypi.stackexchange.com

https://google.com/search?q=site%3Awww.raspberrypi.org+search+first

https://www.raspberrypi.org/forums/search.php?search_id=unanswered

jdb
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1677
Joined: Thu Jul 11, 2013 2:37 pm

Re: setting to stabilize the official image

Mon Sep 30, 2013 8:36 pm

Can you post dmesg logs or similar during your "instability"?

Several changes have been made recently that should alleviate the occurrence of these messages.

Note: The "kevent 2" messages are simply as a result of kernel memory allocation failures. The way that usbnet works, it will try initially to allocate a large contiguous section of memory and progressively reduce it in size until the allocation succeeds. There should be no obvious impact apart from userspace program slowdowns.

Edit: Your kernel was built on Aug 30th - please try with latest rpi-update kernel.
Rockets are loud.
https://astro-pi.org

harshalvora
Posts: 6
Joined: Sat Apr 19, 2014 12:02 pm

Re: setting to stabilize the official image

Wed May 07, 2014 2:56 pm

On Kernel 3.14.0, I still get
smsc95xx 1-1.1:1.0: eth0: kevent 2 may have been dropped
Before this I see some kernel Warning crash related to ieee80211 module.

Is it recommended to set
"smsc95xx.turbo_mode=N"
even in the latest kernel?

Return to “General discussion”

Who is online

Users browsing this forum: PatroSK and 64 guests