somomos
Posts: 3
Joined: Mon Jul 01, 2019 6:21 am

Bus Aliases and the L2 Cache

Mon Jul 01, 2019 7:16 am

Hello,

The diagram on page 5 of the BCM2835 ARM Peripherals document shows four address ranges that can be used to access SDRAM from the VC bus. The two high bits distinguish the four ranges and control caching behavior. These are the ranges described:

Code: Select all

0x0xxxxxxx - L1 and L2 Cached
0x4xxxxxxx - L2 Non-allocating and Coherent
0x8xxxxxxx - L2 Allocating and Coherent
0xCxxxxxxx - Direct Uncached
(https://www.raspberrypi.org/app/uploads ... herals.pdf)

However, Eric Anholt wrote the following. Notice the L2 behavior for aliases 0x4 and 0x8 is swapped:

Code: Select all

0x0... L1 and L2 cache allocating and coherent
0x4... L1 non-allocating, but coherent. L2 allocating and coherent
0x8... L1 non-allocating, but coherent. L2 non-allocating, but coherent
0xc... SDRAM alias. Cache is bypassed. Not L1 or L2 allocating or coherent
(https://patchwork.kernel.org/patch/6329571/)

Eric's take is corroborated by the comments describing memory allocation flags on the firmware wiki:

Code: Select all

MEM_FLAG_NORMAL = 0 << 2, /* normal allocating alias. Don't use from ARM */
MEM_FLAG_DIRECT = 1 << 2, /* 0xC alias uncached */
MEM_FLAG_COHERENT = 2 << 2, /* 0x8 alias. Non-allocating in L2 but coherent */
MEM_FLAG_L1_NONALLOCATING = (MEM_FLAG_DIRECT | MEM_FLAG_COHERENT), /* Allocating in L2 */
https://github.com/raspberrypi/firmware ... -interface

Is the Peripheral specification wrong?
Also, the L2 cache described here is the system L2 cache serving the entire VC4, right?
What's the relationship between that cache and the L2 cache serving the TMUs?
Does the latter go through the former?

Thank you.

LdB
Posts: 1172
Joined: Wed Dec 07, 2016 2:29 pm

Re: Bus Aliases and the L2 Cache

Mon Jul 01, 2019 9:44 am

I suspect you have yourself confused you are comparing apples to oranges.

On one hand you seem to be talking about >>> ARM memory addresses <<<

Code: Select all

0x0xxxxxxx - L1 and L2 Cached
0x4xxxxxxx - L2 Non-allocating and Coherent
0x8xxxxxxx - L2 Allocating and Coherent
0xCxxxxxxx - Direct Uncached
In the next instance you are talking about >>> VC4 FLAGS <<< for the VC4 GL pipeline Allocate Memory function

Code: Select all

MEM_FLAG_NORMAL = 0 << 2, /* normal allocating alias. Don't use from ARM */
MEM_FLAG_DIRECT = 1 << 2, /* 0xC alias uncached */
MEM_FLAG_COHERENT = 2 << 2, /* 0x8 alias. Non-allocating in L2 but coherent */
MEM_FLAG_L1_NONALLOCATING = (MEM_FLAG_DIRECT | MEM_FLAG_COHERENT), /* Allocating in L2 
I can assure you when you allocate memory for the VC4 it is all in the VC4 memory it owns. Which means on a default setup Pi3 for
example it will be between 0x3B400 0000 - 0x3F00 0000 the flags just sets up the block characteristics. So does that make
it clearer? Likely the flags are VC4 flags for it's MMU and unlikely to have anything to do with the ARM and it's memory address above.

somomos
Posts: 3
Joined: Mon Jul 01, 2019 6:21 am

Re: Bus Aliases and the L2 Cache

Mon Jul 01, 2019 8:50 pm

On one hand you seem to be talking about >>> ARM memory addresses <<<
These are VC4 bus addresses. The lower 30 bits map to physical memory while the higher 2 bits control how the VC4 caches are used.
In the next instance you are talking about >>> VC4 FLAGS <<< for the VC4 GL pipeline Allocate Memory function
I'm aware that these are flags. I cited this example not for the flags themselves but for the comments after the flags. Specifically, the comment on this line contradicts the BCM2835 ARM Peripherals document, which claims that memory accesses via the bus alias 0x8 will allocate in the L2 cache, but corroborates Erik Anholt's description, which states that such accesses will not allocate.

Code: Select all

MEM_FLAG_COHERENT = 2 << 2, /* 0x8 alias. Non-allocating in L2 but coherent */
Which is true?
Also, these four ranges are called aliases. I assume this means once I allocate VC4 memory, I can access that memory from the VC4 bus using any of the aliases. Except, if that's right, I don't understand the need for the flags. Maybe that assumption is wrong?

LdB
Posts: 1172
Joined: Wed Dec 07, 2016 2:29 pm

Re: Bus Aliases and the L2 Cache

Tue Jul 02, 2019 3:46 am

There is no contradiction you have an ARM with it's MMU and you have a VC4 with it's MMU ... they just happen to share the same memory but it ends at that point. If it helps Erik Anholt will generally comment from the VC4 and it's MMU point of view he is working with the GL pipeline, he rarely talks from the ARM point of view.

At the moment you are treating the VC4 like it's a peripheral driven from the ARM it is the other way around the ARM is the co-processor. Much of the VC4 memory flags are about the VC4 merging images or running shader code it has nothing to do with the ARM at all and it will be oblivious to it. Yes you can allocate an access area for the ARM but when running a GL pipeline that is about 1% of the allocations you are doing the bulk are for textures, tiles and binning and shader code and stuff for the GL pipeline. So the tag message memory allocations made are predominantly for the VC4 itself the ARM doesn't come into it and most of those alignment must be 128bit aligned because they are for the VC4 being a 128bit processor.

So you have two independant MMU and you trying to join them together because they happen to share the same bus is what is confusing. All I can say is if VC4 with it's MMU was on one board and the ARM with its MMU was on another would you be confused?

If you want to exchange data between the ARM and VC4 when the GL pipeline is running both processors must be configured to the shared area.

somomos
Posts: 3
Joined: Mon Jul 01, 2019 6:21 am

Re: Bus Aliases and the L2 Cache

Tue Jul 02, 2019 6:46 am

My question is about the VC4 and its MMU. I'm not concerned with the ARM in this case. I also understand that the ARM is the co-processor and not vice versa. Although I'm in the process of figuring out how the VC4 works, I don't believe I'm as confused as you think I am. Perhaps I haven't ask my question clearly enough? I'd like to know exactly how the bus aliases work when addressing memory from the QPUs.

LdB
Posts: 1172
Joined: Wed Dec 07, 2016 2:29 pm

Re: Bus Aliases and the L2 Cache

Tue Jul 02, 2019 9:13 am

So if you are not confused then please stop referring to the "diagram on page 5 of the BCM2835 ARM Peripherals document".
That is the 1:1 mapping on the ARM on the left the right hand side is the virtual map on page 5
None of that has any relevance at all to the VC4 (except setup) and completely ARM view only.
So if we remove that ARM documentation (top paragraph of your original post) which has nothing to do with VC4 what is confusing?

So VC4 side the one you are apparently interested in, you can only change the memory either directly by running code on the VC4 or from the ARM via that mailbox and those other flags. Eric told you that !!!
There exists a tiny MMU, configurable only by the VC (running the closed firmware), which maps from the ARM's physical addresses to bus
addresses.

Eric Anholt stated how the VC4 controls how the ARM sees the bus, so read it again and stop trying to refer it to diagram 5 and it will probably not be confusing. Anyhow I will leave you with it and good luck with it all.

Return to “Bare metal, Assembly language”