yaw moo
Posts: 57
Joined: Sun Nov 17, 2019 9:08 pm

calling a timer interrupt handler many thousands of times per second

Fri May 07, 2021 11:53 pm

Hi all,

I am looking at openbsd kernel source code.

I am noticing during usage on a raspberry pi 4 that the system as a whole can get very unresponsive. For example when typing commands on the shell I often have to wait many seconds before they actually execute.

If I understand the code correctly, it seems to have an interrupt handler that is called several thousands of times per second on a raspberry pi 4 on each core.

Could that be one main reason for the system not being very responsive?

cleverca22
Posts: 3947
Joined: Sat Aug 18, 2012 2:33 pm

Re: calling a timer interrupt handler many thousands of times per second

Sat May 08, 2021 12:24 am

Screenshot_2021-05-07_21-19-42.jpg
Screenshot_2021-05-07_21-19-42.jpg (236.96 KiB) Viewed 1076 times
when my pi400 is idle, most of the interrupts are firing at 30hz or less, but there is a vsync irq firing at a perfect 60hz
Screenshot_2021-05-07_21-19-25.png
Screenshot_2021-05-07_21-19-25.png (86.99 KiB) Viewed 1076 times
but when its active, i can get brief spikes of up to 7000 irq/sec, from sources like the mmc controller
i have also seen that the gigabit ethernet controller can fire enough interrupts that the cpu gets swamped, and it cant maintain full gigabit transfers

the graphs are just measuring the rate of change for each counter under /proc/interrupts
you may want to look and see if openbsd can report similar counters, and try to graph them over time, to see which irq could be to blame

jayben
Posts: 301
Joined: Mon Aug 19, 2019 9:56 pm

Re: calling a timer interrupt handler many thousands of times per second

Sat May 08, 2021 9:15 am

Nice graphs, how are they produced?

yaw moo
Posts: 57
Joined: Sun Nov 17, 2019 9:08 pm

Re: calling a timer interrupt handler many thousands of times per second

Sat May 08, 2021 10:38 am

@cleverca thanks. I am doing some very crude measurements, but the interrupt handler seems to be called many thousands per second every time, not only as a spike.

Could that block the system? For example, just logging some text to the console at thousand times per second takes longer than the timer interval and so it completely blocks the machine.

cleverca22
Posts: 3947
Joined: Sat Aug 18, 2012 2:33 pm

Re: calling a timer interrupt handler many thousands of times per second

Sat May 08, 2021 2:05 pm

jayben wrote:
Sat May 08, 2021 9:15 am
Nice graphs, how are they produced?
grafana and prometheus have support to collect data from /proc/interrupts out of the box
you just have to tell grafana how to graph the data, and your done!

Return to “Bare metal, Assembly language”