mvidakovicns
Posts: 6
Joined: Sun Mar 25, 2018 11:26 am

Re: Yet Another Bare Metal Tutorial for the RPi3

Sun Apr 01, 2018 5:28 pm

Hello,

I have modified the raspi-tutorial 13_debugger demo (https://github.com/bztsrc/raspi3-tutori ... 3_debugger) to work with Timer Interrupts. I wanted to create a code which would be called every 1 microsecond, and I wanted that function to be called from the Timer Interrupt.

I would like to check if I am doing this the proper way.

First of all, here is the modified main.c file:

Code: Select all

// set up serial console
uart_init();
// set up timer perod (1 microsecond)
timer_irq_setup(1);
// enable interrupts
enable_irq();

// echo everything back
while(1) {
  uart_send(uart_getc()); 
}
The timer_irq_setup() function is used to set up the timer this way:

Code: Select all

unsigned int divisor;
// Make sure clock is stopped, illegal to change anything while running
// #define ARMTIMER_CONTROL ((volatile unsigned int*)(MMIO_BASE+0xB408))
// #define RPI_ARMTIMER_CTRL_ENABLE        ( 1 << 7 )
*ARMTIMER_CONTROL &= ~(RPI_ARMTIMER_CTRL_ENABLE); 
// Get GPU clock (it varies between 200-450Mhz)
mbox[0] = (5 + 3) * 4;                  // length of the message
mbox[1] = MBOX_REQUEST;         // this is a request message
  
mbox[2] = MBOX_TAG_GET_CLOCK_RATE;   // #define MBOX_TAG_GET_CLOCK_RATE	0x00030002
mbox[3] = 8;                    // buffer size
mbox[4] = 8;
mbox[5] = 4;                    // CLOCK ID: CORE
mbox[6] = 0;                    // Clock Frequency

mbox[7] = MBOX_TAG_LAST;

// send the message to the GPU and receive answer
if (mbox_call(MBOX_CH_PROP)) {
	
  printf("\nclock freq: %d Hz\n", mbox[6]);
	
  // The prescaler divider is set to 250 (based on GPU=250MHz to give 1Mhz clock)
  mbox[6] /= 250;												
  // Divisor we would need at current clock speed
  divisor = ((unsigned long int)period_in_us * mbox[6]) / 1000000;			
  printf("\nclock divisor: %d\n", divisor);
  
  // Enable the timer interrupt IRQ
  // #define IRQ_ENABLE_BASIC_IRQS    ((volatile unsigned int*)(MMIO_BASE+0xB218))
  // #define RPI_BASIC_ARM_TIMER_IRQ         (1 << 0)
  *IRQ_ENABLE_BASIC_IRQS |=  RPI_BASIC_ARM_TIMER_IRQ;
  // Set the load value to divisor
  // #define ARMTIMER_LOAD    ((volatile unsigned int*)(MMIO_BASE+0xB400))
  *ARMTIMER_LOAD = divisor;	

  // #define ARMTIMER_CONTROL ((volatile unsigned int*)(MMIO_BASE+0xB408))
  // #define RPI_ARMTIMER_CTRL_23BIT         ( 1 << 1 )
  // #define RPI_ARMTIMER_CTRL_PRESCALE_1     ( 0 << 2 )
  // #define RPI_ARMTIMER_CTRL_IRQ_ENABLE    ( 1 << 5 )
  // #define RPI_ARMTIMER_CTRL_ENABLE        ( 1 << 7 )
  *ARMTIMER_CONTROL |= RPI_ARMTIMER_CTRL_23BIT | RPI_ARMTIMER_CTRL_PRESCALE_1 | RPI_ARMTIMER_CTRL_IRQ_ENABLE | RPI_ARMTIMER_CTRL_ENABLE; 
  printf("irq_setup finished. Control: %x", *ARMTIMER_CONTROL);
}
The enable_irq() function enables interrupts this way:

Code: Select all

asm volatile("msr daifclr,#2");
Then I have modified original vectors in the start.s file:

Code: Select all

_vectors:
    // synchronous
    .align  7
    //str     x30, [sp, #-16]!     // push x30
    //bl      dbg_saveregs
    //mov     x0, #0
    //bl      dbg_decodeexc
    //bl      dbg_main
    eret

    // IRQ
    .align  7
    //str     x30, [sp, #-16]!     // push x30
    //bl      dbg_saveregs
    //mov     x0, #1
    //bl      dbg_decodeexc
    str   x0, [sp, #-16]!         // push {x0}
    str   x1, [sp, #-16]!         // push {x0}
    str   x2, [sp, #-16]!         // push {x0}
    str   x3, [sp, #-16]!         // push {x0}
    str   x4, [sp, #-16]!         // push {x0}
    str   w0, [sp, #-16]!         // push {x0}
    str   w1, [sp, #-16]!         // push {x0}
    str   w2, [sp, #-16]!         // push {x0}
    str   w3, [sp, #-16]!         // push {x0}
    str   w4, [sp, #-16]!         // push {x0}
    bl      dbg_main
    ldr   w4, [sp], #16           // pop {x0}
    ldr   w3, [sp], #16           // pop {x0}
    ldr   w2, [sp], #16           // pop {x0}
    ldr   w1, [sp], #16           // pop {x0}
    ldr   w0, [sp], #16           // pop {x0}
    ldr   x4, [sp], #16           // pop {x0}
    ldr   x3, [sp], #16           // pop {x0}
    ldr   x2, [sp], #16           // pop {x0}
    ldr   x1, [sp], #16           // pop {x0}
    ldr   x0, [sp], #16           // pop {x0}
    
    eret

    // FIQ
    .align  7
    //str     x30, [sp, #-16]!     // push x30
    //bl      dbg_saveregs
    //mov     x0, #2
    //bl      dbg_decodeexc
    //bl      dbg_main
    eret

    // SError
    .align  7
    //str     x30, [sp, #-16]!     // push x30
    //bl      dbg_saveregs
    //mov     x0, #3
    //bl      dbg_decodeexc
    //bl      dbg_main
    eret
    

As you can see, I have commented out all other causes for the exception, and left only the IRQ. Before calling the dbg_main() function, I have backed-up all registers that were used in that function (I have left the generated assembler files to see which registers were used).

Here is the dbg_main() function:

Code: Select all

void dbg_main()
{
  // clear pending irq
  // #define ARMTIMER_ACQ     ((volatile unsigned int*)(MMIO_BASE+0xB40C))
  *ARMTIMER_ACQ = 1; 
  ...
}
This dbg_main() function is being called every microsecond and I set some GPIO pins there.
My question is: is this the right way to have the timer-generated interrupt code witten in C, for this bare metal platform?

Best regards,
Milan
Last edited by mvidakovicns on Sun Apr 01, 2018 7:15 pm, edited 6 times in total.

mvidakovicns
Posts: 6
Joined: Sun Mar 25, 2018 11:26 am

Java-based client for the raspi3-tutorial loader (Rasbootin64)

Sun Apr 01, 2018 7:05 pm

Hi everyone,

I have created a Java program which can be used to send files to the raspi3-tutorial loader (https://github.com/bztsrc/raspi3-tutori ... spbootin64).

I have posted the program on the github:
https://github.com/milanvidakovic/Raspbootin64Client

This Java program tries to connect to the serial port (given as the first command line parameter) and then tries to upload the given kernel8.img file (as the second command line parameter) to the Raspbootin64 loader. It looks like this:

Code: Select all

java -jar Raspbootin64Client.jar COM3 C:\Temp\kernel8.img
Or, you can start this program like this:

Code: Select all

java -jar Raspbootin64Client.jar gui
This will start the GUI and then you can do the same, but this time, you can use the built-in terminal to see what is RPI sending, and to type text to be sent via serial back to the RPI.

If the upload was successfull, the loader will start the uploaded program.

bzt
Posts: 177
Joined: Sat Oct 14, 2017 9:57 pm

Re: Yet Another Bare Metal Tutorial for the RPi3

Tue Apr 03, 2018 5:31 pm

Hey Milan,

Thank you very much for your java code, I've linked that from my tutorials. :-)

About the timer code, it looks okay to me, but you should consider that executing the debugger every microsec is not very useful. Either you should not re-enable the irq until you finished with the debugger, or (as long as you have only one interrupt) you could disable interrupts and enable them on debugger exit. Or your timer isr should do nothing more than increase a counter. Answering your question's C part, that's the correct way. Gcc has some support to define the vector in C (as well as for fiq handler), but I think it's clearer and more compatible to have a small asm wrapper which sets up arguments and calls a common C handler.

Cheers,
bzt

mvidakovicns
Posts: 6
Joined: Sun Mar 25, 2018 11:26 am

Re: Yet Another Bare Metal Tutorial for the RPi3

Wed Apr 04, 2018 4:39 pm

Hi,
Thank you for your comments and linking my code.

Regarding the timer interrupt, I have changed period from 1 microsecond to 20 microseconds.
I have just one interrupt and I have recycled your dbg_main() to be called every 20 microseconds. Here is my dbg_main():

Code: Select all

void dbg_main()
{
  // clear pending irq
  // #define ARMTIMER_ACQ     ((volatile unsigned int*)(MMIO_BASE+0xB40C))
  *ARMTIMER_ACQ = 1; 
  ...
}
Now, what exactly *ARMTIMER_ACQ = 1 does?
Does it re-enable interrupt, or, it disables interrupt? I haven't found useful information about "clearing pending interrupts". If I omit that code from the dbg_main(), there will be no more interrupts. Therefore, I conclude that it re-enables interrupt to happen again.

Should I put the

Code: Select all

*ARMTIMER_ACQ = 1;
at the end of my interrupt routine (dbg_main())?

How could I achieve the other your suggestion to "disable interrupts and enable them on debugger exit"? Could you give me some example?

I am not experienced with ARM or low level RPI programming so I would appreciate help.

Best regards,
Milan

bzt
Posts: 177
Joined: Sat Oct 14, 2017 9:57 pm

Re: Yet Another Bare Metal Tutorial for the RPi3

Thu Apr 05, 2018 1:59 pm

Hi,

In order to get IRQs working on any architecture, you'll have to do the following things:
1. program the peripheral to generate IRQs
2. program the IRQ Controller to route the IRQ from the peripheral to a specific CPU core (routes can be hardwired)
3. enable interrupts on CPU core so that when it receives an IRQ, it should run an interrupt service routine
4. the ISR should process what it want to process, then acknowledge the IRQ in the controller
5. if required, acknowledge the IRQ on the peripheral
6. the CPU must re-enable interrupts

Now depending on the architecture and the peripheral, some parts are done automatically. On x86 for example the programmable interrupt timer does not require acknowledge, but the real time clock peripheral does. Regardless both timers needs it's IRQ to be acknowledged in the IRQ controller (PIC/APIC). Also the CPU interrupts are disabled automatically when the ISR is called, and they are re-enabled automatically when the CPU flags are restored from the stack.
Now on AArch64 the basic principles are the same. As I've already wrote, I didn't had the time to dig deep into the BCM, so I'm describing more variations. When the BCM timer generates an IRQ, it has to be routed to a CPU core. There's an Interrupt Controller peripheral on the BCM chip, see it's documentation. Your timer_irq_setup() should enable it (point 2) as well as IRQ generation in the timer peripheral (point 1). Whether a CPU can receive an IRQ and interrupt normal code flow, is controlled by a flag, that's what your enable_irq() code sets. I don't know whether this flag is cleared when the exception handler (ISR) is executed, but if so, you have to re-enable it in the ISR. That ACQ you asked about is an acknowledge in the timer peripheral. I don't know if one need to acknowledge an IRQ in the controller too on RPi, but if so, you'll have to do that in the ISR too.

Now when to acknowledge and re-enable interrupts depends on what you want to achive. Basically there are four schools:
1. non-reentrant: the exception handler must be atomic, non-interruptible. Therefore you should acknowledge IRQ (in the peripheral and the controller) and re-enable interrupts (in the CPU) at the end of the ISR. Stable, easy to code, but not so good at performance. DOS is a good example.
2. reentrant: you allow to interrupt an ISR by a new IRQ. To code this, you'll want to acknowledge and re-enable interrupts as soon as possible in the ISR. Really good performance, but could be compatibility and stability issues. Linux is an example.
3. stacking: when an IRQ is received, the handler stacks the call. The real handler routine is called independently. That way when an IRQ handler is executed when a new IRQ fires, it's not handled, but stacked an control is given back to the original IRQ handler as soon as possible. Minix uses this approach (or at least Minix2 did for sure).
4. multually-reentrant: most tricky way, but it's stable and has a good performance too. There you'll enable all *other* IRQs and re-enable interrupts as soon as possible, but acknowledge the IRQ when it's fully handled, at the end of the ISR. This is a re-entrant solution, where different kind of IRQ handlers may interrupt a specific IRQ handler. For example the timer interrupt handler never will be interrupted by the same timer interrupt; but it may be interrupted by a keyboard interrupt. Because a handler can be only interrupted by a different kind, no locking needed and reasonably easy to write to be stabe, yet it has a good performance. Usually used in micro-kernels.

As you can see, it's really up to you. Disabling interrupts in CPU will stop all IRQs firing, so does the lack of acknowledgement in the IRQ controller. On the other hand masking specific IRQs in the controller and delaying acknowliding IRQ in the peripheral only stops a certain IRQ from firing again, but allows other IRQs.

Cheers,
bzt

mvidakovicns
Posts: 6
Joined: Sun Mar 25, 2018 11:26 am

Re: Yet Another Bare Metal Tutorial for the RPi3

Thu Apr 05, 2018 3:54 pm

Hi,
Thanks for the detailed description. I have done the first way and it appears to work. I have re-enabled the interrupts at the end of the routine and it works.
Best regards,
Milan

bzt
Posts: 177
Joined: Sat Oct 14, 2017 9:57 pm

Re: Yet Another Bare Metal Tutorial for the RPi3

Wed Apr 11, 2018 11:03 am

Hi Milan,

I've found a very good tutorial which also covers timer interrupts in depth: RPi OS tutorials

Cheers,
bzt

mvidakovicns
Posts: 6
Joined: Sun Mar 25, 2018 11:26 am

Re: Yet Another Bare Metal Tutorial for the RPi3

Sun Apr 15, 2018 10:12 am

Hi,
Thanks for the tutorial!
Best regards,
Milan

Jekberg
Posts: 3
Joined: Tue May 01, 2018 4:14 pm

Re: Yet Another Bare Metal Tutorial for the RPi3

Tue May 01, 2018 4:40 pm

Hello,

I have been trying to add a few extra lines of uart_puts to test a few things, however, whenever I add another line, it seems that my Raspberry PI 3 B+ is not able to print anything.

Please help!

stewartnjohnston
Posts: 1
Joined: Fri Jun 01, 2018 11:49 pm

Re: Yet Another Bare Metal Tutorial for the RPi3

Fri Jun 01, 2018 11:58 pm

Thanks for the tutorial!

I'm having trouble building the cross compile env. Could you please help? I get the following error when making the binutils

aarch64/binutils-2.30$ make -j4
make[1]: Entering directory '/home/stewart/Documents/RaspberryPI/KernelHacking/aarch64/binutils-2.30'
make[1]: Nothing to be done for 'all-target'.
Configuring in ./intl
Configuring in ./isl
Configuring in ./etc
Configuring in ./libiberty
configure: error: cannot find sources () in .././isl
Makefile:5750: recipe for target 'configure-isl' failed
make[1]: *** [configure-isl] Error 1
make[1]: *** Waiting for unfinished jobs....

When list the binutils-2.30 directory I see:

lrwxrwxrwx 1 stewart stewart 11 Jun 1 18:59 isl -> ../isl-0.18

so the link is there.

What do you think the problem is?

Thanks for your help.

Stewart

LdB
Posts: 872
Joined: Wed Dec 07, 2016 2:29 pm

Re: Yet Another Bare Metal Tutorial for the RPi3

Sat Jun 02, 2018 6:56 am

It tells you what the problem is it can't locate the isl source files .. did you download the full package or just the library?

dsap
Posts: 1
Joined: Mon Jul 23, 2018 8:30 am

Re: Yet Another Bare Metal Tutorial for the RPi3

Mon Jul 23, 2018 8:32 am

I've got the same issue too, trying to make the binutils.

configure: error: cannot find sources () in .././isl

I have followed the tutorial steps one by one precisely and double checked them. Can somebody help?

bzt
Posts: 177
Joined: Sat Oct 14, 2017 9:57 pm

Re: Yet Another Bare Metal Tutorial for the RPi3

Tue Jul 24, 2018 9:21 am

This topic is not about cross-compilers, maybe you should ask your question on your OS' support forum. Try to google "gcc cross-compiler on Ubuntu" or something similar. Hopefully they can help you. (I'm sorry, but I can't help. For me, on my distro, those instructions work, so I can't reproduce your issue. I've compiled 7.2 and 7.3 too without any hickups.) Googling the exact error message could be helpful.

Don't get me wrong guys, but having a cross-compiler is just a prerequirement. If you have trouble with that, are you sure you want to do bare metal programming? I mean compiling a cross-compiler produces error messages, so you should know what's wrong and how to fix. On bare metal there are no error messages at all, you have to figure out everything on your own. Are you sure you're up to that? (No offense meant, just a friendly advice. Bare metal programming is fundamentally different and harder than application programming). Or maybe you should consider using a different distro (Debian maybe?) or one which uses rolling-release (Gentoo, Arch, etc.). Ubuntu is not ideal for developers, and it's reportedly have issues with gcc compilation.

Good luck,
bzt

laroche
Posts: 8
Joined: Fri Jul 06, 2018 7:30 am

Re: Yet Another Bare Metal Tutorial for the RPi3

Tue Jul 24, 2018 11:51 am

If this is really specific to ubuntu, you can also use a pre-compiled compiler
via

Code: Select all

apt-get install g++-aarch64-linux-gnu
Add symlinks or change the Makefiles to use this compiler for the
Bare Metal Tutorial.

best regards,

Florian La Roche

maldus
Posts: 24
Joined: Fri Dec 15, 2017 8:36 am

Re: Yet Another Bare Metal Tutorial for the RPi3

Thu Aug 02, 2018 2:09 pm

How should I proceed to run your examples on a real Rpi3?
I'm trying with 03_uart with no success. It runs fine on qemu.

This is the content of my sd card:

Code: Select all

bootcode.bin  config.txt  kernel8.img  start.elf
where kernel8.img is the example.

These are the contents of config.txt

Code: Select all

arm_64bit=1
disable_commandline_tags=1
enable_uart=1
kernel=kernel8.img
I'm quite sure those are wrong somehow, but I can't find a good reference on it. The Uart stays silent of course.

bzt
Posts: 177
Joined: Sat Oct 14, 2017 9:57 pm

Re: Yet Another Bare Metal Tutorial for the RPi3

Thu Aug 02, 2018 7:03 pm

Hi,

Don't use config.txt. I wrote the tutorials in a way to work on the default configuration. If you have kernel8.img only, you can be sure your RPi will boot in AArch64 mode. Second, make sure you've connected the serial cable properly (boot raspbian Linux and test the serial connection there with minicom). Third, make sure your partitioning and FAT filesystem are OK.

Also I think you might need fixup.dat as well. I've downloaded these files:

Code: Select all

wget -q https://raw.githubusercontent.com/raspberrypi/firmware/master/boot/LICENCE.broadcom
wget -q https://raw.githubusercontent.com/raspberrypi/firmware/master/boot/bootcode.bin
wget -q https://raw.githubusercontent.com/raspberrypi/firmware/master/boot/fixup.dat
wget -q https://raw.githubusercontent.com/raspberrypi/firmware/master/boot/start.elf
I'm using mkfs.vfat -F 16, and I've wrote a small tool in C to create GPT and map the first partition into PMBR with type 0xE, but fdisk in DOS mode should be fine too. I suggest to use at least 8Mb, I found that RPi refuses to boot from smaller partitions. Also it is important that the FAT partition must be the first (you can override that by programming the PWM, but it's tricky for a beginner).

If you still experience problems, I suggest to "dd" a raspbian image on an SD card, delete the config.txt and *.img files. Copy your kernel8.img onto, and try to boot that. This way was reported to work by several users who had problems with properly partitioning and formatting the SD card.

Cheers,
bzt

maldus
Posts: 24
Joined: Fri Dec 15, 2017 8:36 am

Re: Yet Another Bare Metal Tutorial for the RPi3

Thu Aug 02, 2018 8:22 pm

Thank you, I'll try that asap. Meanwhile...
I wrote the tutorials in a way to work on the default configuration
I'm curious, could you elaborate on this? I was under the impression that config.txt was a strong requirement to boot an aarch64 kernel. What is the "default configuration" you are referring to? Is is simply loading the kernel at 0x8000 (I've noticed the not so many others aarch64 examples always add some option to load it at 0x0)?

I've booted A32 kernels on the same sd card, so I'm fairly sure the partitioning is ok (I'll double check if it still doesn't work). I'll also add fixup.dat

EDIT:
Success! Your example worked on the first try. I had to fiddle around with my linker script to make my personal code work on hardware; I was lacking

Code: Select all

. = ALIGN(16)
before defining

Code: Select all

__bss_start
.
Still partially confused on why it was getting stuck because of that. I know addresses should be word aligned, but isn't a word 64bits in aarch64? Then (assuming my understanding is correct) the alignment should be to 8 (8 bytes = 64 bits?). Oh well, I'll study the reference manual for ARMv8

bzt
Posts: 177
Joined: Sat Oct 14, 2017 9:57 pm

Re: Yet Another Bare Metal Tutorial for the RPi3

Fri Aug 03, 2018 10:28 am

maldus wrote:
Thu Aug 02, 2018 8:22 pm
I'm curious, could you elaborate on this? I was under the impression that config.txt was a strong requirement to boot an aarch64 kernel. What is the "default configuration" you are referring to? Is is simply loading the kernel at 0x8000 (I've noticed the not so many others aarch64 examples always add some option to load it at 0x0)?
Yes, it used to be. But thanks to the great work of the Raspberry Pi developers here, start.elf now supports AArch64 without config.txt. By default configuration I meant that, the configuration environment on boot without any tweek in config.txt. That is: if you have kernel8.img, that will be loaded at 0x80000 (and not 0x8000), and will be started in AArch64 mode at EL2. Also no peripherals are expected to be configured (like enable_uart for example), so my tutorials initiliaze them on their own.
I've booted A32 kernels on the same sd card, so I'm fairly sure the partitioning is ok (I'll double check if it still doesn't work). I'll also add fixup.dat
Good, I wrote to check the partitions because there's no error message if start.elf can't be loaded or start.elf can't find or load the kernel (which could seem as a freeze). The lack of the rainbow splash could be a hint here, but I'm afraid that's all.
EDIT:
Success! Your example worked on the first try. I had to fiddle around with my linker script to make my personal code work on hardware; I was lacking

Code: Select all

. = ALIGN(16)
before defining

Code: Select all

__bss_start
.
Still partially confused on why it was getting stuck because of that. I know addresses should be word aligned, but isn't a word 64bits in aarch64? Then (assuming my understanding is correct) the alignment should be to 8 (8 bytes = 64 bits?). Oh well, I'll study the reference manual for ARMv8
If you run into an unaligned exception with no exception handler configured, chances are good that it results in a data abort or instruction fetch exception, causing another exception, leading to an infinite exception loop. And no, sometimes word alignment means 16 bytes, like in the stack, and it seems for the bss too. It depends on which instruction is used to fetch or store data in the bss segment. It is strange though, because I've compiled my tutorials with these linker scripts without any problems (with both gcc 7.2 and 7.3). Recently I had unaligned instruction exception with the FAT BPB struct, so I had tested all my tutorials on qemu and on real hw thoroughly.

Cheers,
bzt

ps: congrats for your success! :-)

AngusMcFire
Posts: 3
Joined: Sun Aug 05, 2018 8:44 pm

Re: Yet Another Bare Metal Tutorial for the RPi3

Sun Aug 05, 2018 8:55 pm

Hi,

I altered/reimplemented the MMU example to use 48 bit of Virtual addresses with 4 layers of address translation.
It works fine on QEMU, but on the real raspberry pi 3 it is not working. The raspberry gets stuck after enabling the MMU:

asm volatile ("msr sctlr_el1, %0;" : : "r" (tmp_var));

Is there some kind of limitation on the arm-cortexA53 on the raspberry pi 3 i missed? Because you also are just using 39Bit virtual addresses.

PS: Thanks for the examples, they work very nice out of the box.

Best Regards
Angus

bzt
Posts: 177
Joined: Sat Oct 14, 2017 9:57 pm

Re: Yet Another Bare Metal Tutorial for the RPi3

Mon Aug 06, 2018 10:24 am

Well, you have to configure the virtual address size. In my tutorial I query the physical address size from a system register (mmu.c:129), and use that value for virtual address size (IPS) too at line 144. The address size you cover with mapping (T0SZ, T1SZ) must be smaller than that, otherwise an error occurs.

I'm just curous, why do you need 48 bits? That seems extremely large (specially for a system that only has 1G RAM). 3 levels with 4k pages allows you to map 1T of memory (512G in user space, 512G for kernel space). On x86_64 systems, which usually have lot more RAM (typically 16G, maybe 512G in servers) I could understand, but on a microcomputer like RPi? Just for the records, most x86 CPU manufacturers implement only 48 bits instead of the architectural 56 bits limit, because they think 48 bits is so f*ing huge it will never be reached in the foreseeable future.

Cheers,
bzt

AngusMcFire
Posts: 3
Joined: Sun Aug 05, 2018 8:44 pm

Re: Yet Another Bare Metal Tutorial for the RPi3

Mon Aug 06, 2018 10:52 am

Hi,

thanks for the fast reply, i need the 48Bit address space because i am porting a x86_64 OS as part of my bachelor thesis, i am well aware this is quite overkill for a 1GB system:-)

For the configuration i do basically the same as you do, except using T0SZ=16 and T1SZ=16, I set the IPS the same way you do.
I am quite worried because it works perfectly on the Qemu, i checked with the gdb, all the memory regions are where i mapped them, but i can not get the thing working on the real Raspberry Pi 3. I also found no example on how to do this with 48Bit, i just relied on your example and the datasheet...

Thanks for the fast response.

BR
Angus

bzt
Posts: 177
Joined: Sat Oct 14, 2017 9:57 pm

Re: Yet Another Bare Metal Tutorial for the RPi3

Thu Aug 09, 2018 1:30 pm

I see. Since qemu is an emulator, it can emulate any address size it likes. On a real hardware, memory address bus has to be hardwired, and if you have less than 48 bits (reported by the system register), then I'm afraid not much you can do about it.

Think it this way: you have 4 level worth of tables here too, but the highest table is stored directly in registers (TTBRx), and only the first and last element in the table are used; others between considered uncanonical addresses. The fact that you tell the MMU to use 3 levels for both T0SZ and T1SZ is just an implementation detail.

The 4 levels then would be:

Code: Select all

PML[0] and PML[511] == in TTBR0 and TTBR1 registers (512G)
PDPE == level 1 (1G)
PDE == level 2 (2M)
PTE == level 3 (4k)
Btw, the value 16 seems way too small to me. I suggest to google a bit, there's a very good description on infocenter.arm.com, with tables on valid address sizes / TxSZ values. I can't find it right now, but I'm sure the link was already posted on this forum. http://infocenter.arm.com/help/topic/co ... index.html and http://infocenter.arm.com/help/topic/co ... p3_trm.pdf are useful too. (Btw the latter states that 40 bits is the max for IPS on AArch64).

Cheers,
bzt

AngusMcFire
Posts: 3
Joined: Sun Aug 05, 2018 8:44 pm

Re: Yet Another Bare Metal Tutorial for the RPi3

Mon Aug 13, 2018 6:14 pm

Hi,

thanks for the response, i see now the maximum bits are limited based on the IPS of the CPU.

There is a quite good docu.
https://armv8-ref.codingbelief.com/en/c ... tages.html

I think i have to work with 39 Bit :| it will do somehow

Anyway million thanks for the examples and you help,

BR angus

dpotop
Posts: 55
Joined: Mon Nov 24, 2014 2:14 pm

Need docs!

Tue Oct 09, 2018 1:33 pm

Hello,

First of all, thanks a lot for these tutorials. They are excellent.

Concerning your "power" tutorial: you mention that most components of the PPi3 card can be shut down individually. But where can I get the doc on that?

Best,
Dpotop
dpotop

bzt
Posts: 177
Joined: Sat Oct 14, 2017 9:57 pm

Re: Yet Another Bare Metal Tutorial for the RPi3

Fri Oct 19, 2018 2:36 pm

Hi dpotop,

The code is simple, you use a Mailbox call on the property channel for that. Code is here. This is a full shutdown, iterates on all devices. If you are interested in the device ids, read the docs here.

It worth mentioning that you cannot power off VCC pins, so if you have a device powered from there, you'll have to implement a way to cut the power (for example with a relay between VCC and the device, controlled by another, data GPIO pin). I really hope that in the upcoming versions of Raspberry Pi they will allow to turn off VCC pins just like the other pins from software.

Cheers,
bzt

Return to “Bare metal, Assembly language”