yaw moo
Posts: 57
Joined: Sun Nov 17, 2019 9:08 pm

aarch64 counter/timer, CNTFRQ_EL0 and CNTPCT_EL0

Sun Apr 11, 2021 5:04 pm

Hi all,
I am looking for some details about aarch64 timers/counters. What I am seeing is that CNTFRQ_EL0 is 0 and CNTPCT_EL0 seems to increment at about 2 Mhz.

I believe that I have to setup CNTFRQ_EL0 myself but what is the exact frequency at which CNTPCT_EL0 increments?

Also, is there a counter on armv8 (preferably) or on the bcm2711 that counts the actual cpu clock cycles? I mean something that varies its frequency together with the cpu clock based on energy requirements and similar.

EDIT: I have seen that there is a counter which seems based on the cpu clock in the arm timer. What is the exact frequency of the arm timer? I am using a prescaler of 0 and it looks like it is in the region of 200Mhz.

Thanks :-)

bzt
Posts: 630
Joined: Sat Oct 14, 2017 9:57 pm

Re: aarch64 counter/timer, CNTFRQ_EL0 and CNTPCT_EL0

Sun Apr 11, 2021 8:19 pm

yaw moo wrote:
Sun Apr 11, 2021 5:04 pm
Hi all,
I am looking for some details about aarch64 timers/counters. What I am seeing is that CNTFRQ_EL0 is 0 and CNTPCT_EL0 seems to increment at about 2 Mhz.

I believe that I have to setup CNTFRQ_EL0 myself but what is the exact frequency at which CNTPCT_EL0 increments?

Also, is there a counter on armv8 (preferably) or on the bcm2711 that counts the actual cpu clock cycles? I mean something that varies its frequency together with the cpu clock based on energy requirements and similar.

EDIT: I have seen that there is a counter which seems based on the cpu clock in the arm timer. What is the exact frequency of the arm timer? I am using a prescaler of 0 and it looks like it is in the region of 200Mhz.

Thanks :-)
I believe you're mixing different timers. No wonder, there are several, and at some point all of them are called the ARM Timer in one doc or another.

The one with the system register CNTPCT by default uses the CPU clock frequency (if you leave CNTFRQ as is, because the firmware sets it to the CPU clock's freq). But this has no prescaler.

Another one, on the ARM QA7 chip, is the ARM Timer clock (section 3.1 and 4.2, address 0x4000_0000), which has a prescaler, and can be configured to be external crystal dependent or CPU clock (APB) dependent as well.

So for CPU clock dependent timer, I'd suggest CNTPCT without touching CNTFRQ or the clock at 0x4000_0000 with APB setting. Read the linked ARM QA7 spec, you can find lots of useful information there.

Cheers,
bzt

cleverca22
Posts: 3540
Joined: Sat Aug 18, 2012 2:33 pm

Re: aarch64 counter/timer, CNTFRQ_EL0 and CNTPCT_EL0

Mon Apr 12, 2021 12:22 am

bzt wrote:
Sun Apr 11, 2021 8:19 pm
The one with the system register CNTPCT by default uses the CPU clock frequency (if you leave CNTFRQ as is, because the firmware sets it to the CPU clock's freq). But this has no prescaler.
my understanding is that the CNTFRQ register is basically useless, it has zero control over how the timer behaves

the only purpose it has, is for the firmware to tell the OS what freq the silicon is driving the timer at, so the OS can compute how many ticks to wait for a given time delay

yaw moo
Posts: 57
Joined: Sun Nov 17, 2019 9:08 pm

Re: aarch64 counter/timer, CNTFRQ_EL0 and CNTPCT_EL0

Mon Apr 12, 2021 12:52 am

@cleverca22 @bzt

I understand what you are writing. And yes, the CNTFRQ register looks like is just there to write a frequency to it that the rest of my system can use when it wants to know.

I guess the actual frequencies are implementation dependent.

I just want to know what are the exact frequencies those specific timers run at on a raspberry pi 4.

By repeatedly printing out the counter values I have a rough estimate.

Is there any documentation that clearly states what those frequencies are?

cleverca22
Posts: 3540
Joined: Sat Aug 18, 2012 2:33 pm

Re: aarch64 counter/timer, CNTFRQ_EL0 and CNTPCT_EL0

Mon Apr 12, 2021 1:03 am

i need to get around to studying the arm local timer in more depth, i recently had some fun where code expecting INTERRUPT_ARM_LOCAL_CNTPNSIRQ to fire, was just silently hanging
after diging around for a while, i discovered that its now firing under INTERRUPT_ARM_LOCAL_CNTPSIRQ

the cause, my code was running in secure mode when originally it was designed to run under non-secure mode!

ive also see some arm timer prescaler configs in the hw registers, so there could be a number of things going on that arent clearly documented

yaw moo
Posts: 57
Joined: Sun Nov 17, 2019 9:08 pm

Re: aarch64 counter/timer, CNTFRQ_EL0 and CNTPCT_EL0

Mon Apr 12, 2021 1:26 am

yep, the prescaler apparently has a non-zero default value. I set it to zero to get the approximate figures I mentioned.

Does anyone know what the actual frequencies are?

This should not be obscured by any intellectual property issue in principle as it's just a frequency and I am sure someone knows the exact value...

cleverca22
Posts: 3540
Joined: Sat Aug 18, 2012 2:33 pm

Re: aarch64 counter/timer, CNTFRQ_EL0 and CNTPCT_EL0

Mon Apr 12, 2021 1:32 am

i can think of 3 possibilities
1: the raw crystal (19.2mhz or 54mhz, depending on the model)
2: a flat 1mhz
3: the raw arm clock (which can vary depending on cpu load)

do your measurements come close to any of those?

yaw moo
Posts: 57
Joined: Sun Nov 17, 2019 9:08 pm

Re: aarch64 counter/timer, CNTFRQ_EL0 and CNTPCT_EL0

Mon Apr 12, 2021 1:54 am

Like I wrote earlier, CNTPCT_EL0 is around 2/3 MHZ, the arm timer (with prescaler set to 0) is around 200Mhz, the system timer is 1Mhz.

These are rough estimates and may be plain wrong, apart from the system timer which I believe is documented to be 1Mhz.

bzt
Posts: 630
Joined: Sat Oct 14, 2017 9:57 pm

Re: aarch64 counter/timer, CNTFRQ_EL0 and CNTPCT_EL0

Mon Apr 12, 2021 11:40 am

yaw moo wrote:
Mon Apr 12, 2021 12:52 am
I guess the actual frequencies are implementation dependent.
Yes, they are. For the BCM2711 it should be 54Mhz by default (but configurable).
yaw moo wrote:
Mon Apr 12, 2021 12:52 am
I just want to know what are the exact frequencies those specific timers run at on a raspberry pi 4.
For the ARM QA7 timer, configurable. Either 19MHz (external crystal) or APB (which is ARM Clock freq / 2). You can also configure the increment value, so by setting that to 2 the timer will be updated on every other CPU cycles, but could contain a proper value (or you could just shift the counter's value by one to get the CPU cycles).
yaw moo wrote:
Mon Apr 12, 2021 12:52 am
Is there any documentation that clearly states what those frequencies are?
Well, for the crystal source and the BCM ST it is fixed, but for the ARM clock it is configurable. In run-time to get the actual ARM clock freq you should use the property mailbox with Clock ID being 3. Hope this helps.

Cheers,
bzt

cleverca22
Posts: 3540
Joined: Sat Aug 18, 2012 2:33 pm

Re: aarch64 counter/timer, CNTFRQ_EL0 and CNTPCT_EL0

Mon Apr 12, 2021 12:08 pm

how is it switched between the raw crystal and the APB clock?

yaw moo
Posts: 57
Joined: Sun Nov 17, 2019 9:08 pm

Re: aarch64 counter/timer, CNTFRQ_EL0 and CNTPCT_EL0

Mon Apr 12, 2021 2:29 pm

Hi,
thanks for your help :-)

What is the QA7 timer ? And what does APB stand for? Sorry I am a bit confused by the terminology here, where do I read about those?

To clarify the terminology I use ( I bet it's already clear for you guys, I am just trying to map what you write to what I learnt :-) ):

I am using aarch64 only, on a raspberry pi 4.

The CNTPCT_EL0 is part of armv8 architecture. I read about it in the armv8 architecture programming guide.
The System Timer and the ARM timers are documented in the bcm2711 manual.

The system timer seems to be fixed at 1Mhz and that agrees with my experience.

Regarding the CNTPCT_EL0, it looks like it's about 2/3 Mhz and the ARM timer around 200Mhz.
To see this, I repeatedly print all the counter values and CNTPCT_EL0 increments at least 2 times the speed of the system timer and the arm timer about 200 times.

The only tweak I do on those timers is I set the prescaler of the arm counter to 0 ( which should result in a divisor of 1 if I am not wrong ). This is done by writing 0 into bits 16-23 of the control register at 0x408. Its default value would have been 0x3e. I also enable the arm timer counter register at 0x420 by setting bit 9 of the control register and I make the counter 32 bits by setting bit 1.

I am a bit puzzled at getting about 2Mhz for CNTPCT_EL0 and about 200Mhz for the arm timer as they don't seem related to the cpu clock.

From all I know, the raspberry pi 4 default cpu clock is 1500Mhz. I don't get any counter that is that fast. Also, the arm timer is a 32 bit value and so if it incremented at 1500Mhz it would go to its max value and back to 0 very quickly.

Also, I am getting nothing close to the 19Mhz and 54Mhz that you mention.

cleverca22
Posts: 3540
Joined: Sat Aug 18, 2012 2:33 pm

Re: aarch64 counter/timer, CNTFRQ_EL0 and CNTPCT_EL0

Mon Apr 12, 2021 2:48 pm

Screenshot_2021-04-12_11-37-33.png
Screenshot_2021-04-12_11-37-33.png (37.2 KiB) Viewed 426 times
this is from the cortex-a7 docs (rpi2's core)

16 irq's come out of the arm cluster, 4 different events, with 4 instances each, since each event is per-cpu
a 64bit raw counter goes back into the arm cluster


its up to the implementor (broadcom) to make a 64bit counter, and feed it with some freq, then route that full 64bit value into the arm cluster
and wire those 16 irq's out, into the irq controller, which goes back into the arm cluster

armv8 (pi3/pi4) made the timer a required feature, and added more instances to better fit the EL system

the physical timer is just comparing against that raw 64bit counter
the virtual timer has an offset set by the hypervisor, so the guest will think it the timer started at 0

most of the above is based on asking questions on irc, i still need to find the right keyword to search for on arm.com, to find the official docs

yaw moo
Posts: 57
Joined: Sun Nov 17, 2019 9:08 pm

Re: aarch64 counter/timer, CNTFRQ_EL0 and CNTPCT_EL0

Mon Apr 12, 2021 3:09 pm

thanks :-)

So as far as we know there is no documentation that says what those frequencies are, right?

Are there any forums, irc or other places that you can recommend where I can ask those kinds of questions?
Last edited by yaw moo on Mon Apr 12, 2021 4:16 pm, edited 2 times in total.

bzt
Posts: 630
Joined: Sat Oct 14, 2017 9:57 pm

Re: aarch64 counter/timer, CNTFRQ_EL0 and CNTPCT_EL0

Mon Apr 12, 2021 4:07 pm

cleverca22 wrote:
Mon Apr 12, 2021 12:08 pm
how is it switched between the raw crystal and the APB clock?
From the spec, page 9, see attachment armqa7.png and page 91 on attachment bcm2711.png.
yaw moo wrote:
Mon Apr 12, 2021 2:29 pm
What is the QA7 timer ? And what does APB stand for? Sorry I am a bit confused by the terminology here, where do I read about those?
Read the doc I've linked, ARM QA7 and also BCM2711. The former is for the RPi3, but I'm sure the same (or very similar) circuit exists on RPi4 too. I'm talking about the same interface that routes CNTVIRQ, CNTHPIRQ and the other interrupts.
yaw moo wrote:
Mon Apr 12, 2021 2:29 pm
this is from the cortex-a7 docs (rpi2's core)
On RPi2, but for RPi3 it does not use the ARM spec for timers, it uses a chip described by the aforementioned spec instead. Not sure about RPi4 though, but this doc in section 6.5.2. ARM_LOCAL page 90 talks about registers with uncanny resemblance of the QA7 registers (the base address is 0x4c0000000).

ps: It's amazing how good looking the BCM2711 is as compared to previous docs :-D

Cheers,
bzt
Attachments
bcm2711.png
bcm2711.png (27.65 KiB) Viewed 398 times
armqa7.png
armqa7.png (29.89 KiB) Viewed 398 times
Last edited by bzt on Mon Apr 12, 2021 4:18 pm, edited 1 time in total.

yaw moo
Posts: 57
Joined: Sun Nov 17, 2019 9:08 pm

Re: aarch64 counter/timer, CNTFRQ_EL0 and CNTPCT_EL0

Mon Apr 12, 2021 4:15 pm

@bzt cool thanks a lot :-)

the more I get into this the more it gets weird. After digging a bit deeper, the arm timer "counter" increments with frequency sys_clk/(prescale+1). On the other hand the arm timer itself with frequency apb_clock/(pre_divider+1).

Are sys_clk and apb_clock the same? If not, why on earth would they feed the counter and the timer of the same peripheral with two distinct clocks?

On the positive side, this is consistent with the documented 250Mhz system clock at startup which corresponds to my rough estimate of about 200Mhz for the arm timer "counter".

The exact frequency of CNTPCT_EL0 is still a mystery to me.

bzt
Posts: 630
Joined: Sat Oct 14, 2017 9:57 pm

Re: aarch64 counter/timer, CNTFRQ_EL0 and CNTPCT_EL0

Mon Apr 12, 2021 4:30 pm

yaw moo wrote:
Mon Apr 12, 2021 4:15 pm
@bzt cool thanks a lot :-)
You're welcome!
yaw moo wrote:
Mon Apr 12, 2021 4:15 pm
the more I get into this the more it gets weird.
Yeah, that's because there are many timers, and they don't have a canonical name. All of them are called the ARM timer sometimes, so if you think it's weird, you're probably mixing the descriptions of two different timers :-) I know what you feel, it was confusing to me too! I've tried to collect all timers in this post.
yaw moo wrote:
Mon Apr 12, 2021 4:15 pm
Are sys_clk and apb_clock the same? If not, why on earth would they feed the counter and the timer of the same peripheral with two distinct clocks?
Well, on RPi3 APB is half the speed of the ARM freq. The reason for the two different source is, one is high precision, but with configurable freq, while the other is using a fixed freq, but less accurate. It's up to the programmer to chose which source suits their needs the best.
yaw moo wrote:
Mon Apr 12, 2021 4:15 pm
The exact frequency of CNTPCT_EL0 is still a mystery to me.
I believe that should be 1/CNTFRQ_EL0? CNTPCT should be incremented exactly CNTFRQ times in a sec (but I guess it's not 100% accurate).

Cheers,
bzt

yaw moo
Posts: 57
Joined: Sun Nov 17, 2019 9:08 pm

Re: aarch64 counter/timer, CNTFRQ_EL0 and CNTPCT_EL0

Mon Apr 12, 2021 5:59 pm

CNTFRQ_EL0 should show the frequency of CNTPCT_EL0 if I am not wrong.

Problem is, it's set to 0.

cleverca22
Posts: 3540
Joined: Sat Aug 18, 2012 2:33 pm

Re: aarch64 counter/timer, CNTFRQ_EL0 and CNTPCT_EL0

Mon Apr 12, 2021 6:11 pm

yaw moo wrote:
Mon Apr 12, 2021 5:59 pm
CNTFRQ_EL0 should show the frequency of CNTPCT_EL0 if I am not wrong.

Problem is, it's set to 0.
on the rpi, that gets set by the arm stub that is loaded to arm physical 0

but if your using kernel_old=1, the armstub wont be ran, so the registers are left at the default state

https://github.com/raspberrypi/tools/bl ... #L110-L112
this code will load it with either 19.2mhz or 54mhz, depending on the model

some nearby code will also adjust the prescaler and decide between crystal or APB

yaw moo
Posts: 57
Joined: Sun Nov 17, 2019 9:08 pm

Re: aarch64 counter/timer, CNTFRQ_EL0 and CNTPCT_EL0

Mon Apr 12, 2021 10:51 pm

Oh ok. I think I get it now. Thanks :-)

I am doing some setup like in the arm stubs in my own code so I might as well set CNTFRQ_EL0 up.

By the way, LOCAL_PRESCALER is at offset 8 and is not documented in the bcm2711 manual as far as I can tell. Is it documented in the previous manuals?

Return to “Bare metal, Assembly language”