8000 interrupts per second when idle!


140 posts   Page 4 of 6   1, 2, 3, 4, 5, 6
by dom » Thu Aug 23, 2012 9:40 am
We have the FIQ patch working on the 2.91 USB driver. Something in the update to 2.94 broke it.
We've got someone investigating.
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4042
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge
by samsamsam » Mon Aug 27, 2012 9:48 am
dom wrote:We have the FIQ patch working on the 2.91 USB driver. Something in the update to 2.94 broke it.
We've got someone investigating.



Is there any progress with this patch?
Posts: 36
Joined: Fri Aug 17, 2012 11:36 am
by jamesh » Mon Aug 27, 2012 7:07 pm
Bank holiday weekend in the UK - so probably not much work done the last three days. Patience is required!
Soon to be employed engineer - Hurrah! Volunteer at the Raspberry Pi Foundation, helper at PiAcademy September 2014.
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 11895
Joined: Sat Jul 30, 2011 7:41 pm
by samsamsam » Mon Aug 27, 2012 7:29 pm
Oki :) I am patient. Oki :) I am patient. I just asked...
Posts: 36
Joined: Fri Aug 17, 2012 11:36 am
by samsamsam » Fri Aug 31, 2012 8:48 pm
Hello,
Does someone still working on this patch to port it to the new driver?
Posts: 36
Joined: Fri Aug 17, 2012 11:36 am
by dom » Fri Aug 31, 2012 10:10 pm
samsamsam wrote:Hello,
Does someone still working on this patch to port it to the new driver?


Good news. Costas (one of our interns) found the problem earlier today. I'm just testing, and if it looks good will push it out tomorrow.
Looks to give about 10% performance boost to the Pi, without the side effects of the SOF patch.
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4042
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge
by dom » Sat Sep 01, 2012 12:06 am
FIQ patch is pushed. Get it from rpi-update. It is disabled by default, but add:
dwc_otg.fiq_fix_enable=1
to cmdline.txt to test. It should give about 10% extra performance to ARM when USB is not busy. It shouldn't change behaviour of USB.
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4042
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge
by samsamsam » Sat Sep 01, 2012 8:04 am
Super news :) I am going to test this patch :)

Thank you very much! :)
Posts: 36
Joined: Fri Aug 17, 2012 11:36 am
by dom » Sat Sep 01, 2012 10:43 am
You can check the number of interrupts per second with and without this patch with:
Code: Select all
sh -c "cat /proc/interrupts | grep dwc && sleep 10 && cat /proc/interrupts | grep dwc" | awk '{ if (s==0) s=$2 } END { print ($2-s)/10 }'


You should get 8000+ when the patch is disabled.
You should get 600+ when the patch is enabled. This will vary based on USB devices connected and how busy they are. When USB is running flat out you may see 8000+.
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4042
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge
by samsamsam » Sat Sep 01, 2012 11:23 am
Thanks for info.
I will test this patch tonight, however without patch I get:
Code: Select all
root@raspberry-pi:/etc/oscam# sh -c "cat /proc/interrupts | grep dwc && sleep 10 && cat /proc/interrupts | grep dwc" | awk '{ if (s==0) s=$2 } END { print ($2-s)/10 }'
3564

Is it possible? I use network and 4 ftdiToRs232 converters and 1GB pendrive with root partition.

vmstat give me similar results
Code: Select all
root@raspberry-pi:/etc/oscam# vmstat
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 0  0      0 205592   3880  17240    0    0     0     0 3742  161  1  1 95  2
Posts: 36
Joined: Fri Aug 17, 2012 11:36 am
by dom » Sat Sep 01, 2012 12:00 pm
samsamsam wrote:Is it possible? I use network and 4 ftdiToRs232 converters and 1GB pendrive with root partition.


I don't think so. You haven't enabled SOF reduction have you?
Is it any different with the USB devices unplugged?
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4042
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge
by samsamsam » Sat Sep 01, 2012 12:25 pm
My command line:
Code: Select all
Kernel command line: dma.dmachans=0x3c bcm2708_fb.fbwidth=656 bcm2708_fb.fbheight=416 bcm2708.boardrev=0x2 bcm2708.serial=0xec6c4cc5 smsc95xx.macaddr=B8:27:EB:6C:4C:C5 dwc_otg.microframe_schedule=1 smsc95xx.turbo_mode=N sdhci-bcm2708.missing_status=0 sdhci-bcm2708.sync_after_dma=0 dwc_otg.lpm_enable=0 console=tty1 root=/dev/sda1 rootfstype=ext4 rootwait dwc_otg.speed=1


Maybe it is related to parameter dwc_otg.speed=1 or smsc95xx.turbo_mode=N? I can not test what happens when I unplug device now but tonight I will test this also.

I must use dwc_otg.speed=1 because without this my FTDI devices always restart. It is probably similar problem to the: http://www.mail-archive.com/linux-usb-u ... 14694.html
Posts: 36
Joined: Fri Aug 17, 2012 11:36 am
by dom » Sat Sep 01, 2012 12:39 pm
samsamsam wrote:My command line:
Code: Select all
Kernel command line: dma.dmachans=0x3c bcm2708_fb.fbwidth=656 bcm2708_fb.fbheight=416 bcm2708.boardrev=0x2 bcm2708.serial=0xec6c4cc5 smsc95xx.macaddr=B8:27:EB:6C:4C:C5 dwc_otg.microframe_schedule=1 smsc95xx.turbo_mode=N sdhci-bcm2708.missing_status=0 sdhci-bcm2708.sync_after_dma=0 dwc_otg.lpm_enable=0 console=tty1 root=/dev/sda1 rootfstype=ext4 rootwait dwc_otg.speed=1


Maybe it is related to parameter dwc_otg.speed=1 or smsc95xx.turbo_mode=N? I can not test what happens when I unplug device now but tonight I will test this also.

I must use dwc_otg.speed=1 because without this my FTDI devices always restart. It is probably similar problem to the: http://www.mail-archive.com/linux-usb-u ... 14694.html


Yes, dwc_otg.speed=1 limits speed to full speed (rather than USB 2's high speed) protocol, and so only 1000 interrupts/sec when idle.
This patch will have much less benefit if you are using dwc_otg.speed=1.
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4042
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge
by samsamsam » Sat Sep 01, 2012 12:45 pm
Yes I know :) However I think that the patch will make my RPI more stable. Although right now my RPI is very stable after night battles :)

OFFTOP

What do you think can I restore the root partition on SD card since the patch: "low-latency mode to sdcard driver"?

If yes what value should be set for the parameter sdhci-bcm2708.missing_status?
Posts: 36
Joined: Fri Aug 17, 2012 11:36 am
by dom » Sat Sep 01, 2012 1:00 pm
samsamsam wrote:What do you think can I restore the root partition on SD card since the patch: "low-latency mode to sdcard driver"?

If yes what value should be set for the parameter sdhci-bcm2708.missing_status?


sdhci-bcm2708.missing_status is not recommended now, just leave it off command line. The sdhci-bcm2708.enable_llm is now enabled by default and fixes the USB latency problem caused by sdcard a safer way.
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4042
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge
by samsamsam » Sat Sep 01, 2012 1:14 pm
Thank you very much for your help:) I will move my root partition to the SD and test these two patches:)

Thanks once again :)
Posts: 36
Joined: Fri Aug 17, 2012 11:36 am
by samsamsam » Sun Sep 02, 2012 7:37 am
Stats with new kernel and root partition moved to SD card:

Code: Select all
root@raspberry-pi:~# vmstat
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 0  0      0 179380  27148  17624    0    0     0     0 3316  141  2  2 93  4


Code: Select all
Kernel command line: dma.dmachans=0x3c bcm2708_fb.fbwidth=656 bcm2708_fb.fbheight=416 bcm2708.boardrev=0x2 bcm2708.serial=0xec6c4cc5 smsc95xx.macaddr=B8:27:EB:6C:4C:C5 dwc_otg.microframe_schedule=1 smsc95xx.turbo_mode=N dwc_otg.fiq_fix_enable=1 sdhci-bcm2708.sync_after_dma=0 dwc_otg.lpm_enable=0 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 rootwait dwc_otg.speed=1


So far, the system is stable :)
Posts: 36
Joined: Fri Aug 17, 2012 11:36 am
by Koenkk » Sun Sep 02, 2012 12:18 pm
Not working for me, getting a kp at boot (details: http://i.imgur.com/PfJG0.jpg). I ran rpi-update. cmdline.txt is containing : dwc_otg.fiq_fix_enable=1. Kernel: 3.2.27+ #102 PREEMPT Sat Sep 1 01:00:50 BST 2012 armv6l.
Posts: 148
Joined: Sun Jul 29, 2012 10:22 pm
Location: Netherlands
by dom » Sun Sep 02, 2012 12:28 pm
Koenkk wrote:Not working for me, getting a kp at boot (details: http://i.imgur.com/PfJG0.jpg). I ran rpi-update. cmdline.txt is containing : dwc_otg.fiq_fix_enable=1. Kernel: 3.2.27+ #102 PREEMPT Sat Sep 1 01:00:50 BST 2012 armv6l.

Looks like it can't find rootfs. Is your rootfs on sdcard, or are you using nfs/usb?
Does dwc_otg.fiq_fix_enable=0 allow it to boot? Does reverting kernel allow it to boot?
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4042
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge
by samsamsam » Sun Sep 02, 2012 12:31 pm
cmdline.txt should contain at least:
Code: Select all
dwc_otg.fiq_fix_enable=1 sdhci-bcm2708.sync_after_dma=0 dwc_otg.lpm_enable=0 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 rootwait
Posts: 36
Joined: Fri Aug 17, 2012 11:36 am
by Koenkk » Sun Sep 02, 2012 12:33 pm
samsamsam wrote:cmdline.txt should contain at least:
Code: Select all
dwc_otg.fiq_fix_enable=1 sdhci-bcm2708.sync_after_dma=0 dwc_otg.lpm_enable=0 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 rootwait


working, thanks for this uber fast response!!
Posts: 148
Joined: Sun Jul 29, 2012 10:22 pm
Location: Netherlands
by rtramon » Sun Sep 02, 2012 12:46 pm
FIQ patch is pushed. Get it from rpi-update. It is disabled by default, but add:
dwc_otg.fiq_fix_enable=1
to cmdline.txt to test. It should give about 10% extra performance to ARM when USB is not busy. It shouldn't change behaviour of USB.


Updated firmware with rip-update set the cmdline option, and after some time my headless system becomes inaccessible with the following error in /var/log/messages:

Sep 2 14:37:34 raspi kernel: [ 8406.167322] smsc95xx 1-1.1:1.0: eth0: Failed to w
rite HW_CFG_LRST_ bit in HW_CFG register, ret = -110
Sep 2 14:37:40 raspi kernel: [ 8412.167492] smsc95xx 1-1.1:1.0: eth0: Failed to w
rite register index 0x00000014
(repeated many times)

cmdline.txt:
dwc_otg.lpm_enable=0 root=/dev/mmcblk0p2 rootfstype=ext4 rootwait sdhci-bcm2708.sync_after_dma=0 dwc_otg.microframe_schedule=1 dwc_otg.fiq_fix_enable=1

Also interupts per seconds is 44K! like normally on my system using pl2303 usb to rs232 converter reading serial output from small measurement system.
Posts: 10
Joined: Sat Jul 07, 2012 10:06 am
Location: 't Gooi
by samsamsam » Sun Sep 02, 2012 1:02 pm
You can try to add line
Code: Select all
vm.min_free_kbytes = 16384
at the and of file /etc/sysctl.conf

And add parameter smsc95xx.turbo_mode=N to the cmdline.txt
Posts: 36
Joined: Fri Aug 17, 2012 11:36 am
by dom » Sun Sep 02, 2012 1:03 pm
rtramon wrote:Sep 2 14:37:34 raspi kernel: [ 8406.167322] smsc95xx 1-1.1:1.0: eth0: Failed to w
rite HW_CFG_LRST_ bit in HW_CFG register, ret = -110
Sep 2 14:37:40 raspi kernel: [ 8412.167492] smsc95xx 1-1.1:1.0: eth0: Failed to w
rite register index 0x00000014
(repeated many times)

cmdline.txt:
dwc_otg.lpm_enable=0 root=/dev/mmcblk0p2 rootfstype=ext4 rootwait sdhci-bcm2708.sync_after_dma=0 dwc_otg.microframe_schedule=1 dwc_otg.fiq_fix_enable=1


If you change to dwc_otg.fiq_fix_enable=0 does the problem go away?
If you change to dwc_otg.microframe_schedule=0 does the problem go away?
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4042
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge
by bitwelder » Sun Sep 02, 2012 8:32 pm
Hello,
I applied the patch and in my case (nothing attached to USB) interrupts went from 8000+ to 290.

Question: is there any known 'side effect' to this FIQ patch, or any conditions where it may not be convenient to activate it?
Posts: 3
Joined: Fri Aug 03, 2012 2:10 pm