What is the last address in ram on the pi 3?

4 posts
by Makogan » Thu Jun 15, 2017 3:42 am
Since the pi 3 supposedly has 1GB of memory I expected the last address to be at 0x3FFFFFFC. However things like the mailbox (which is at 0x3F00B880) are way below this limit. It'd make no sense to have gaps in memory just to put memory mapped peripherals, so I must assume actual ram ends somewhere before 0x30000000, so where exactly is the last ram address? What is the last place I can safely read and write from?
Posts: 67
Joined: Tue May 16, 2017 9:17 pm
by LdB » Thu Jun 15, 2017 4:15 am
It's a little bit trickier than that, you are sort of referring to what the BootLoader defaults at. I was hoping Ultibo or David may answer this as they are more technically across this. The PI isn't a microprocessor with a simple fixed memory map like you are trying to treat it.

Anyhow lets put it in background ... open the peripheral data sheet and goto page 5. The diagram is turned sideways because it needs the width under the section 1.2 Address map, 1.2.1 Diagrammatic overview
https://www.raspberrypi.org/app/uploads ... herals.pdf

See the two memory management units (MMU's) one for the VC and one for the ARM.

It also shows you the arm MMU can virtualize memory anywhere between from 0x00000000 to 0xFFFFFFFF to the ARM.

So technically the memory map can change by changing the arm MMU, so all I can really talk about is what the Bootloader defaults with.

So the default is shown on that figure at the top which says size of Physical memory set in arm_loader 0x40000000 .. hopefully you see it.

Now the next complication there is some sort of granularity around the memory settings in the MMU. I don't know the in's and out's of the MMU but for whatever reason around that the IO space for the Pi2/3 gets mapped inside that physical memory not outside it. The IO space is 1Mb so technically you only get 1023Mb of memory on a Pi2/3 1Mb of your memory can not be accessed because the IO maps onto it.

If you look at the official documentation:
https://www.raspberrypi.org/documentati ... /README.md
No arm MMU details and I don't know if it has ever been released

So the default setup on the bootloader on the Pi3 places the last memory byte at 0x3EFFFFFF being the byte below the IO space start at 0x3F000000.

However it doesn't end there you will find the memory repeats there is memory at 0x40000000+

Now look at the diagram on the VC side and you will find the memory marked as '4' Alias - L2 cache coherrent (non allocating).
Look above and that and you will see it maps to 0x80000000 and 0xC0000000.

This harks back to the having to OR and AND NOT 0xC0000000 onto the pointer on your framebuffer. Yes the VC can and does access the same memory as well.

So the memory is also accessed by cache and the VC.

To be really technical the VC is the most important processor the ARM is a coprocessor but it seems weird to think like that because all our code runs on the ARM. I am not even sure the ARM has direct access to the physical hardware given the way it is setup I suspect it just sees the VC bus.

So the problem I was having with your question is it is very hard to answer because all that detail needs to be relayed ... in that light can you refine what you are asking :-)

The offical word on memory:
https://www.raspberrypi.org/documentati ... dresses.md
Posts: 311
Joined: Wed Dec 07, 2016 2:29 pm
by dwelch67 » Thu Jun 15, 2017 10:04 am
I have no facts on this. The original arm peripheral doc shows a pretty good picture including there is a split, back then I remember it being right in half, there is still going to be a split be it in half or a quarter or some fraction is for the GPU and the rest for us.

Understand/remember in the 32 bit x86 days (which we are still transitioning out of) the memory configuration there may place your video memory hole within the lower 4gig, (as well as the 1 or 2 gig pcie hole), and if you have more ram than that you simply lose that ram. Eventually the BIOSes were not defaulting to I dont know so lets assume a 32 bit operating system, and will allow those windows to move up above the end of ram. In this case we have completely different hardware but since we are behind a virtual address space into the VC or GPU or real address space they could very well just prevent us from accessing all the ram and place the peripherals within that lower 1GB space. Which it appears they have.

With where the peripheral space starts, I think it is very clear that the absolute best we are going to get is 0x3F000000 bytes of ram for the ARM, without a lot of work. I know folks here have talked about using the various quarters of the memory space, as it appears to loop around in 0x40000000 byte chunks, the 0 alias, the 4 alias, the 8 alias and C alias as shown in that same picture. I assume that still applies and if I remember right you need to use the right space for writing pixels to the frame buffer and/or using the right space for mailbox communication. The picture implies the arm is mapped into the direct access area (uncached), but I am pretty sure I have even tried going in 0x40000000 chunks in the arm space, perhaps that is an illusion and they simply ignore the upper bits or perhaps in the arm space they really do map us into the GPU space differently.

Someone here probably already knows off hand how much ram we get by default if we dont mess with config.txt, old days ATAGS these days not sure if they still support those or use newer methods for bootloader communication to linux to describe how much ram it has. Nor do I know off hand the miniumum amount the GPU/VC needs/wants/demands.

The minimum amount
of memory which can be given to the GPU is 32MB, but that will restrict the multimedia
performance; for example, 32MB does not provide enough buffering for the GPU to do
1080p30 video decoding.

and the text goes on to explain more. I believe them when they tell us they simply cut out the old arm and put another in. if that is really true then the backend mmu the vc/arm mmu in the picture hasnt changed. Now software on both ends has (gpu side and arm side) but I assume they still have the space on the VC/GPU side in quarters and we have to live with 0x3F000000 as our max amount of memory. Perhaps by now they have opened up the register space for that mmu not that we could necessarily access it from the arm, nor would we want to trash/crash the gpu firmware by messing with it.

Re-reading your original question. Actually it does make sense, we have been living with this on x86 systems for a very long time now, because they still support 32 bit operating systems and dont want to just crash on boot all the time for folks not understanding all of this (the vast majority of the population) the formerly 1gb pcie window now transitioning to 2gb (all pcie peripherals in the system have to fit in that window per the bios) would be placed in the lower 4GB even if the pcie controller supported a 64 bit mode. This is why for the longest time when you bought 4, 8, 16GB of ram your computer would report 3, 7 or 15, you simply threw away up to 25% of that ram you just upgraded to because of how the BIOS allocates your space. So this makes perfect sense. the other fact here is we share that ram with the GPU so they can simply draw the line between arm and gpu at the 0x3F000000 point or lower.

Do I know facts on this? Nope, going on some pictures and experience and assumptions. Pretty obvious why the 0x20xxxxxx space moved to 0x3Fxxxxxx was to make more room for ram, they should have thought of that day one, sounds like the Bill Gates quote on 640K, but didnt, would perhaps be nice if they gave us a config.txt setting that for the BCM2835 the peripherals were also mapped at 0x3Fxxxxxx so we didnt have to if-then-else runtime and could instead have compile time addresses. Of course within the arm's mmu we could just map those up to that space.

Anyway I suspect the next step is to see what the gpu bootloader tells us (linux) how much ram the ARM has on boot. And if you play with config.txt settings does it ever allow above 0x3F000000 if it does I expect you cant access it as you hit the peripherals, perhaps there are pockets above/around the peripherals where it punches through to ram, but I wouldnt rely on that if you even find it...
Posts: 726
Joined: Sat May 26, 2012 5:32 pm
by Makogan » Fri Jun 16, 2017 12:04 am
Thank you both for your replies. Since A) I don't have much time, and B) I am a computer scientist, not a computer engineer and have very little knowledge or experience working directly with hardware (other than using OpenGL to play around with a video card). I decided to just play it safe and keep my stuff under 0x30000000, My super simplistic malloc() implementation hasn't caused any problems yet, so I think this is good enough for a learning project.
Posts: 67
Joined: Tue May 16, 2017 9:17 pm