jasonkleban
Posts: 1
Joined: Sat Aug 25, 2018 3:09 am

Rust types for mailbox buffers & tags

Sat Aug 25, 2018 3:15 am

I'm trying learn rust and native coding and bare-metal and raspberry pi all at once. I'm targeting aarch64 rpi3 (https://github.com/JasonKleban/rust-rasp) and trying to figure out the rustiest way to make its onboard ACT led blink. Actually, I haven't gotten it to blink at all yet - I think my build may be right but I wouldn't be able to tell yet because there's of course no outward indication until I can expect to see that light blinking, which I don't yet have the right code for.

I found some information about how to do this but the examples/tutorials are in c (https://github.com/vanvught/rpidmx512/blob/master/lib-bcm2835/src/bcm2837_gpio_virt.c) or asm (https://adamransom.github.io/posts/raspberry-pi-bare-metal-part-1.html). There's a particular way we need to ask the GPU to control the ACT led for us - with a mailbox protocol. The mailbox is mapped to a particular address and we write arbitrarily-sized, word-aligned, and word-padded data to send messages. The sizes of the data are included in the data.

I'm **aware** of rust's `#[repr(align(4))]` and dynamically-sized-types, but I don't know how we might even represent the padding at the end (as documented) to reserve exactly the space needed. There's also an "end of tags" symbol, 0x0, expected after the last tag in the arbitrary-sized sequence of tags in a buffer but in rust such a dynamically-sized-type can only be last in the struct.

By my reading, I think the buffer full of tags is to be allocated on the heap (aligned(4) so that bits needed for other purposes can be assumed to be 0) and that only the address gets placed on the mailbox. I don't understand what happens after that. So we're writing a single word being the memory address of the buffer? I think is an atomic operation but I don't know how atomic works in shared memory on independent processors.

And then how do we know that the item has been processed and that we can deallocate the buffer? If the answer is that the GPU is writing back to our mailbox to say "DONE" then how does IT know that we've seen its message so that it can deallocate that "DONE" message?

Also, perhaps the GPU on an RPI3 has access to all the memory, but I don't think it is always the case. So how can rust hint to the allocator that the memory must be allocated in a particular range?

How can protocols like this be modeled nicely as rust types? Or does it have to be modeled in rust as `&mut [u8]` with very unsafe function fiddling all the bytes to make up the buffer?

I don't care to write in c, I'm drawn to rust. I don't care to learn rust on x86_64, I'm drawn to arm/embedded. I don't care to write for an OS - I don't want long boot times, system maintenance, user accounts, or startup scripts. I definitely don't want to write python. Other than the OS and the python-emphasis, I like the rpi3 because it seems like I should be able to hook up ready-made peripherals to it and figure out how to control motors and sensors on it from bare metal once I understand the basics - I AM interest if there's a better, cheap, modern, rust-compatible, bare-metal platform for robotics than rpi though - I want to write bare-metal rust (ideally for 64-bit ARMv8+ because it seems clean and power-efficient but also very capable) mostly to control gpio pins (I think) connected to motor controllers and simple sensors. I'm really bad at electronics, so I don't know how to make my own peripherals.

LdB
Posts: 857
Joined: Wed Dec 07, 2016 2:29 pm

Re: Rust types for mailbox buffers & tags

Sat Aug 25, 2018 4:10 pm

The activating of the LED on the Pi3 and the memory details are all available by the Raspberry Pi mailbox

The activity led you need to use tag 0x0003004, port 130, 1 = on 0 = off
The arm memory available on tag 0x00010005
https://github.com/raspberrypi/firmware ... -interface

I believe that is the rust code sample to accesses the mailbox
https://github.com/andre-richter/rust-r ... rc/mbox.rs

The key part not to forget is that all messages for exchange thru the mailbox must be 16byte aligned

Code: Select all

#[repr(align(16))]
It matters not if you are 32bit or 64bit the mailbox function and requirements remain the same.

bzt
Posts: 161
Joined: Sat Oct 14, 2017 9:57 pm

Re: Rust types for mailbox buffers & tags

Sun Aug 26, 2018 9:16 am

jasonkleban wrote:
Sat Aug 25, 2018 3:15 am
By my reading, I think the buffer full of tags is to be allocated on the heap (aligned(4) so that bits needed for other purposes can be assumed to be 0) and that only the address gets placed on the mailbox. I don't understand what happens after that. So we're writing a single word being the memory address of the buffer? I think is an atomic operation but I don't know how atomic works in shared memory on independent processors.
In short, the buffer does not need to be allocated on the heap, it can be anywhere provided it's 16 bytes aligned. Because of the alignment, the least significant bits of the address is always zero. You have to put the required channel there, and write that into the mailbox's MMIO register atomically.
And then how do we know that the item has been processed and that we can deallocate the buffer? If the answer is that the GPU is writing back to our mailbox to say "DONE" then how does IT know that we've seen its message so that it can deallocate that "DONE" message?
You have to poll the mailbox's MMIO register to see if your message was already processed or not. Until it's done, the CPU must not modify the buffer in any way (not even by another core). To avoid this, I suggest to have a different buffer for each core. The GPU writes the response to the same buffer, so no memory allocation / deallocation takes place at all.

The communication is always one way:
CPU puts a request (one or more tags) in the buffer
CPU writes the buffer's address OR'd with the channel into the mailbox MMIO, initiating the CPU->GPU transfer
GPU starts processing the request, and places the response into the same buffer
CPU polls the mailbox's MMIO register to see if the GPU->CPU transfer has finished

Therefore the GPU does not need to know when or if the response message has been processed, so the GPU simply doesn't care about the "DONE" message (which is stored in the same buffer as the request, no allocation / deallocation). There's no GPU->CPU transfer without a CPU->GPU transfer first.

Now if the CPU allocates the buffer dynamically, then it is the CPU's responsibility to deallocate that buffer after it has processed the response message (again, the GPU doesn't care when the CPU has finished with processing the response). But I'd not recommend that, because mailbox can use the same buffer over and over again, so it's a waste of resources to allocate and deallocate the buffer every time. It's better to allocate a properly aligned buffer on start up (maybe per core), and just write the tags into on each mailbox call.

In a multithreaded environment you may want to allocate the buffer every time (to have a separate buffer for each thread), but it's questionable it worth the effort. I'm not sure the GPU can handle paralell mailbox requests. Technically the interface allows that (since you have to check if you can write to the MMIO, there's a mandatory synchronisation), but what would happen if two threads changes the screen resolution to different dimensions at the same time? The GPU must serialize the mailbox messages anyway, and you must provide an atomic MMIO write. IMHO having static buffers for each core is much simplier, and avoiding concurrent mailbox calls (within one core) is not that bad for performance, considering a normal application uses the mailbox only occassionaly.

Hope this helps,
bzt

Return to “Bare metal, Assembly language”

Who is online

Users browsing this forum: No registered users and 5 guests