dylan1280
Posts: 2
Joined: Mon Apr 23, 2018 10:20 pm

Interrupt differences between 32-bit and 64-bit kernels

Tue Apr 24, 2018 2:55 pm

I have a Raspberry Pi 3 Model B V1.2 that I am evaluating for real time processing. When testing on other ARM and x86 platforms I have found it helpful to set the smp_affinity of IRQs to core 0 while running the real time process on the remaining cores. My process requires the 64-bit architecture. I have read that it is not possible to change the affinity of individual IRQs.

Code: Select all

[email protected]:~# echo 1 > /proc/irq/41/smp_affinity
bash: echo: write error: Input/output error

I have noticed that on the 32-bit kernel interrupts mostly occur on core 0 and on the 64-bit kernel interrupts are more evenly distributed among all cores. Is there is any known explanation for this difference? Outputs from dmesg and /proc/interrupts below:

64-bit Kernel:

Code: Select all

[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 4.14.34-v8+ ([email protected]) (gcc version 5.4.1 20161213 (Linaro GCC 5.4-2017.01)) #2 SMP PREEMPT Wed Apr 18 18:06:57 EDT 2018
[    0.000000] Boot CPU: AArch64 Processor [410fd034]
[    0.000000] Machine model: Raspberry Pi 3 Model B
[    0.000000] efi: Getting EFI parameters from FDT:
[    0.000000] efi: UEFI not found.
[    0.000000] cma: Reserved 8 MiB at 0x000000003a400000
[    0.000000] On node 0 totalpages: 242688
[    0.000000]   DMA zone: 3792 pages used for memmap
[    0.000000]   DMA zone: 0 pages reserved
[    0.000000]   DMA zone: 242688 pages, LIFO batch:31
[    0.000000] random: fast init done
[    0.000000] percpu: Embedded 22 pages/cpu @ffffffc03b379000 s51712 r8192 d30208 u90112
[    0.000000] pcpu-alloc: s51712 r8192 d30208 u90112 alloc=22*4096
[    0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 
[    0.000000] Detected VIPT I-cache on CPU0
[    0.000000] CPU features: enabling workaround for ARM erratum 845719
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 238896
[    0.000000] Kernel command line: ip=dhcp root=/dev/mmcblk0p2 rw noirqbalance isolcpus=1-3 rcu_nocbs=1-3 nohz_full=1-3 irqaffinity=0 initrd=0x02100000,0x54c1eb

Code: Select all

           CPU0       CPU1       CPU2       CPU3       
  2:   58143081       2840       2729       2675  bcm2836-timer   1 Edge      arch_timer
  6:       7661       6643       7137       7070  ARMCTRL-level   1 Edge      3f00b880.mailbox
  7:          1          0          0          0  ARMCTRL-level   2 Edge      VCHIQ doorbell
 15:    7959396    7647122    7956896    7666630  ARMCTRL-level  64 Edge      dwc_otg, dwc_otg_pcd, dwc_otg_hcd:usb1
 41:  121832918  121367565  121771070  121552960  ARMCTRL-level  41 Edge      dwc_otg_sim-fiq
 48:          0          0          0          0  ARMCTRL-level  48 Edge      bcm2708_fb dma
 50:          0          0          0          0  ARMCTRL-level  50 Edge      DMA IRQ
 52:          0          0          0          0  ARMCTRL-level  52 Edge      DMA IRQ
 53:          0          0          0          0  ARMCTRL-level  53 Edge      DMA IRQ
 56:        541        601        538        587  ARMCTRL-level  56 Edge      DMA IRQ
 61:          0          0          0          0  ARMCTRL-level  61 Edge      bcm2835-auxirq
 71:        154        134        140        173  ARMCTRL-level  88 Edge      mmc0
 77:        212        247        213        219  ARMCTRL-level  94 Edge      mmc1
IPI0:    485616       1822       1747       1710       Rescheduling interrupts
IPI1:         6       1295       1274       1252       Function call interrupts
IPI2:         0          0          0          0       CPU stop interrupts
IPI3:         0          0          0          0       CPU stop (for crash dump) interrupts
IPI4:         0          0          0          0       Timer broadcast interrupts
IPI5:    548890      85226      85043      85336       IRQ work interrupts
IPI6:         0          0          0          0       CPU wake-up interrupts
Err:          0

32-bit Kernel:

Code: Select all

[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 4.14.34-v7+ ([email protected]) (gcc version 4.9.3 (crosstool-NG crosstool-ng-1.22.0-88-g8460611)) #1110 SMP Mon Apr 16 15:18:51 BST 2018
[    0.000000] CPU: ARMv7 Processor [410fd034] revision 4 (ARMv7), cr=10c5383d
[    0.000000] CPU: div instructions available: patching division code
[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
[    0.000000] OF: fdt: Machine model: Raspberry Pi 3 Model B Rev 1.2
[    0.000000] Memory policy: Data cache writealloc
[    0.000000] cma: Reserved 8 MiB at 0x3ac00000
[    0.000000] On node 0 totalpages: 242688
[    0.000000] free_area_init_node: node 0, pgdat 80c84e40, node_mem_map ba3a1000
[    0.000000]   Normal zone: 2133 pages used for memmap
[    0.000000]   Normal zone: 0 pages reserved
[    0.000000]   Normal zone: 242688 pages, LIFO batch:31
[    0.000000] random: fast init done
[    0.000000] percpu: Embedded 17 pages/cpu @ba34a000 s38720 r8192 d22720 u69632
[    0.000000] pcpu-alloc: s38720 r8192 d22720 u69632 alloc=17*4096
[    0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 240555
[    0.000000] Kernel command line: 8250.nr_uarts=0 bcm2708_fb.fbwidth=1920 bcm2708_fb.fbheight=1200 bcm2708_fb.fbswap=1 vc_mem.mem_base=0x3ec00000 vc_mem.mem_size=0x40000000  dwc_otg.lpm_enable=0 console=ttyS0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait quiet splash plymouth.ignore-serial-consoles nohz_full=1-3 rcu_nocbs=1-3 noirqbalance isolcpus=1-3

Code: Select all

           CPU0       CPU1       CPU2       CPU3       
 16:          0          0          0          0  bcm2836-timer   0 Edge      arch_timer
 17:      48398          8          8          8  bcm2836-timer   1 Edge      arch_timer
 21:          0          0          0          0  bcm2836-pmu   9 Edge      arm-pmu
 23:        241          0          0          0  ARMCTRL-level   1 Edge      3f00b880.mailbox
 24:          7          0          0          0  ARMCTRL-level   2 Edge      VCHIQ doorbell
 46:          0          0          0          0  ARMCTRL-level  48 Edge      bcm2708_fb dma
 48:          0          0          0          0  ARMCTRL-level  50 Edge      DMA IRQ
 50:          0          0          0          0  ARMCTRL-level  52 Edge      DMA IRQ
 51:        755          0          0          0  ARMCTRL-level  53 Edge      DMA IRQ
 54:      10496          0          0          0  ARMCTRL-level  56 Edge      DMA IRQ
 59:          0          0          0          0  ARMCTRL-level  61 Edge      bcm2835-auxirq
 62:     260789          0          0          0  ARMCTRL-level  64 Edge      dwc_otg, dwc_otg_pcd, dwc_otg_hcd:usb1
 86:       1129          0          0          0  ARMCTRL-level  88 Edge      mmc0
 87:       5141          0          0          0  ARMCTRL-level  89 Edge      uart-pl011
 92:      67226          0          0          0  ARMCTRL-level  94 Edge      mmc1
FIQ:              usb_fiq
IPI0:          0          0          0          0  CPU wakeup interrupts
IPI1:          0          0          0          0  Timer broadcast interrupts
IPI2:         25        136        136        134  Rescheduling interrupts
IPI3:          0          6          5          4  Function call interrupts
IPI4:          0          0          0          0  CPU stop interrupts
IPI5:        541          0          0          0  IRQ work interrupts
IPI6:          0          0          0          0  completion interrupts
Err:          0

swahren
Posts: 21
Joined: Mon Sep 19, 2016 5:24 pm
Location: Germany

Re: Interrupt differences between 32-bit and 64-bit kernels

Sun Apr 29, 2018 9:38 am

As you noticed the smp_affinity isn't implemented yet. The difference in the behaviour comes from this patch:

https://github.com/raspberrypi/linux/co ... a99570d782

dylan1280
Posts: 2
Joined: Mon Apr 23, 2018 10:20 pm

Re: Interrupt differences between 32-bit and 64-bit kernels

Tue May 01, 2018 4:33 pm

Thank you! This patch was exactly what I was looking for.

Return to “Linux Kernel”

Who is online

Users browsing this forum: No registered users and 0 guests