jnc100
Posts: 54
Joined: Wed Feb 20, 2013 10:10 pm

New Raspberry Pi bootloader (rpi-boot)

Sun Feb 24, 2013 10:17 pm

I've recently obtained a RPi and have been playing with porting my OS to it. As a side effect of my experiments with it I've come up with a small second stage bootloader that I've released under a MIT licence in the hope that it may prove useful.

Its main selling point is that it provides the loaded kernel with 'BIOS-like' functions to aid debugging and hardware access during early kernel initialisation. Specifically, it provides a printf() function for writing text to the framebuffer, read-only filesystem access and timer support. Note that all the drivers are not optimised and are designed to be replaced by OS ones once the OS is more fully loaded. For example the block devices use synchronous PIO calls only, the filesystem drivers do not perform caching, the console driver is slow to scroll (as making it quicker would involve using up one of the RPi's DMA channels which the kernel itself may want to use).

It supports parsing a command file (/boot/rpi-boot.cfg) loaded from the SD card and can load kernels from both ext2 and FAT partitions. It currently supports loading Multiboot compliant kernels in ELF or a.out format and standard ELF kernels.

In Multiboot mode it supports loadable kernel modules, providing a list of devices and a memory map ala E820 (but actually interpreted from the ATAGs supplied by the first stage bootloader).

Currently unsupported features:
- USB support (work in progress)
- Network boot (requires USB)
- Interactive boot (requires USB)
- Ability to ask for a particular framebuffer mode in the Multiboot header (work in progress)

For the source see here, precompiled binaries and a test kernel are available here and further documentation is available at README and MULTIBOOT-ARM.

Please note this is a work in progress and bug reports are most welcome either on this thread or via email. In particular the ext2 code has only been tested in an emulator (but no SD writes are made so will not damage your filesystem).

Regards,
John

User avatar
DexOS
Posts: 876
Joined: Wed May 16, 2012 6:32 pm
Contact: Website

Re: New Raspberry Pi bootloader (rpi-boot)

Mon Feb 25, 2013 12:01 am

Great work, i am sure this will help a lot of coders, not tested yet, but will let you know, if i have any problems.
Dex.
Batteries not included, Some assembly required.

Pate
Posts: 115
Joined: Tue Feb 05, 2013 9:04 am
Location: Finland

Re: New Raspberry Pi bootloader (rpi-boot)

Mon Feb 25, 2013 11:45 am

Thanks for making this available, looks like a really good starting point for various bare metal test projects!

I was just browsing around the source code (while at work, having a bit of a time to spend while testers are testing my code :) ), and noticed something a little bit curious in timer.c. It looks to me like the usleep() calls register_timer() which mallocs a structure whose pointer is returned to usleep, but I don't see the allocated memory getting freed anywhere?

Just thought I'd mention this in case it really is a bug, and if not, sorry for posting before getting a proper understanding of how your memory allocation routines work. :oops:

Pate
Now working on piro: http://piro.patrickaalto.com
See my rpix86 project at http://rpix86.patrickaalto.com

jnc100
Posts: 54
Joined: Wed Feb 20, 2013 10:10 pm

Re: New Raspberry Pi bootloader (rpi-boot)

Tue Feb 26, 2013 12:31 am

Pate wrote:It looks to me like the usleep() calls register_timer() which mallocs a structure whose pointer is returned to usleep, but I don't see the allocated memory getting freed anywhere?
Many thanks for pointing this memory leak out. It has now been fixed.

Regards,
John

User avatar
DexOS
Posts: 876
Joined: Wed May 16, 2012 6:32 pm
Contact: Website

Re: New Raspberry Pi bootloader (rpi-boot)

Tue Feb 26, 2013 5:07 am

I have test both already compiled and compiled by me and in both case on my pi, it gets as far as
"MAIN: device list:
Which it list, but that it it just does nothing after that.
My PI is the 256 ver.
Batteries not included, Some assembly required.

jnc100
Posts: 54
Joined: Wed Feb 20, 2013 10:10 pm

Re: New Raspberry Pi bootloader (rpi-boot)

Tue Feb 26, 2013 11:06 am

DexOS wrote:I have test both already compiled and compiled by me and in both case on my pi, it gets as far as
"MAIN: device list:
Which it list, but that it it just does nothing after that.
My PI is the 256 ver.
Thanks for the test, although that is disappointing. When you say it lists the devices it should say something like emmc0_0, emmc0_1, ... emmc0_n for each supported partition on the SD card.

I've only tested on a 512 MiB version but given where your error occurs I don't think this is the likely problem. The next step after listing the devices is searching for a configuration file, so I guess it hangs somewhere in the directory parsing code suggesting its a SD driver or filesystem driver issue.

I've uploaded a debug version too here (or check out the latest svn and compile with 'CFLAGS=-DDEBUG2 make') which displays far more debugging output and may help to track down where the problem is. I'd be most grateful if you could try this on your Pi.

Regards,
John

User avatar
DexOS
Posts: 876
Joined: Wed May 16, 2012 6:32 pm
Contact: Website

Re: New Raspberry Pi bootloader (rpi-boot)

Tue Feb 26, 2013 11:11 pm

Hi
Yes it prints "emmc0_0, emmc0_1"
And know how you feel, theres nothing worse than everything works fine on you test machine and then not on others, as that was a selling point of the PI, as in if it works on one, it will work on all.
And sure i will test with the debug ver and get back to you.
Dex
Batteries not included, Some assembly required.

User avatar
DexOS
Posts: 876
Joined: Wed May 16, 2012 6:32 pm
Contact: Website

Re: New Raspberry Pi bootloader (rpi-boot)

Wed Feb 27, 2013 1:16 am

Ok done some test, i am not at home so i do not have my Uart2usb cable and the info flash up the screen to fast to read, but i could see enough to see it could not read some clusters.
So after some tests its seem to be something to do with the Fat, the card is 2 GB in size and it as a 50mb partition thats fat32 which it seem to have a problem with, once i formated it to fat16 it works fine.
Does the code get the Fat type from partition size (as i thing, MS say thats the way to detect FAT type) if so that could be a thing to look into.
Hope this help, any more info let me know.
Dex.
As a side note, why not add the USB driver from
http://www.cl.cam.ac.uk/freshers/raspbe ... put01.html
Batteries not included, Some assembly required.

jnc100
Posts: 54
Joined: Wed Feb 20, 2013 10:10 pm

Re: New Raspberry Pi bootloader (rpi-boot)

Wed Feb 27, 2013 5:04 pm

Thanks for testing it. Looks like I have to debug the FAT32 code somewhat. You're right, the way I was determining the FAT type was too oversimplistic (I was dividing total sectors rather than data sectors by sectors per cluster to get the total cluster count). I have now fixed this in the repository but won't get a chance to properly test it for a few days I'm afraid. I'd imagine though that there are probably some other bugs in the FAT32 code too which are preventing it working for you.

As regards USB yes I was looking at that driver and will hopefully get a chance to test it next week sometime. Again, many thanks for the bug reports.

Regards,
John.

User avatar
DexOS
Posts: 876
Joined: Wed May 16, 2012 6:32 pm
Contact: Website

Re: New Raspberry Pi bootloader (rpi-boot)

Fri Mar 01, 2013 12:59 am

What is the format for booting flat bin file, where is it loaded to and do you need anything in a header etc.
I know it should be above 1MB, but how does it know where to load.
Thanks
Batteries not included, Some assembly required.

jnc100
Posts: 54
Joined: Wed Feb 20, 2013 10:10 pm

Re: New Raspberry Pi bootloader (rpi-boot)

Fri Mar 01, 2013 11:09 am

If there is a valid multiboot header in the flat binary with the appropriate 'a.out' fields filled in then it will use these. If not, then the following steps are taken:

1) If the configuration file contains the line binary_load_addr <address> it will set the inbuilt value of 'binary_load_addr' to 'address'
2) If the configuration file contains the line entry_addr <address> it will set the inbuilt value of 'entry_addr' to address
3)If the configuration file contains the line kernel <kernel_file> and kernel_file is not an ELF or linux kernel then it will first attempt to determine the load address:
a) If binary_load_address is set see if it is available (i.e. > 1 MiB, binary_load_address + length < memory size, ensure it doesn't overlap with any modules), if not error
b) If it isn't set find the first free address (this is generally 1 MiB assuming no modules have been loaded first)
4) The kernel will be loaded to this address
5) If entry_addr is not set then set it to the kernel load address
6) After the kernel line in the configuration file (but before the boot line) you can change the entry address with the entry_addr <address> option.
7) The kernel is booted with the boot configuration line

For example:

Code: Select all

binary_load_addr 0x400000
entry_addr 0x401000
kernel /my_kernel.bin
boot
At least that is the theory. Unfortunately the 'binary_load_addr' and 'entry_addr' configuration commands will currently give a not implemented error purely because I have been focusing on ELF load so far and haven't got round to the (albeit trivial) task of writing an atoi() function, although this should appear in the repository in the next few days.

So in summary for now, the configuration file:

Code: Select all

kernel /my_kernel.bin
boot
will load my_kernel.bin to the 1 MiB address and start executing at 1 MiB, with the registers set up as per the README.

Regards,
John.

User avatar
DexOS
Posts: 876
Joined: Wed May 16, 2012 6:32 pm
Contact: Website

Re: New Raspberry Pi bootloader (rpi-boot)

Fri Mar 01, 2013 9:04 pm

Thanks you for that detailed info, i will wait for the up dates.
As i have tried this:

Code: Select all

kernel /my_kernel.bin
boot 
ORG is 0x100000, but i get a -1 error it does detect the file and tries to determine load address, but fails.

One thing that would be useful if you get time, would be a stand alone module like how the Baking Pi USB driver is implemented, for read/write fat file sys, that out puts memory.map info, that only includes emmc and block plus fat etc.
This would be very useful to ASM programmers to include in there OS, as writing full fat in assembly is a pain.
Thanks again, Dex.
Batteries not included, Some assembly required.

jnc100
Posts: 54
Joined: Wed Feb 20, 2013 10:10 pm

Re: New Raspberry Pi bootloader (rpi-boot)

Fri Mar 01, 2013 9:21 pm

DexOS wrote:Thanks you for that detailed info, i will wait for the up dates.
I've just added them to the repository, tested with a flat binary loaded from a fat16 filesystem with load address 0x400000 and entry point 0x400428.
DexOS wrote:it does detect the file and tries to determine load address, but fails.
If the error is from the 'boot' command then this is an old version which didn't correctly set the entry address to be the start of the binary. Also fixed.
DexOS wrote:One thing that would be useful if you get time, would be a stand alone module like how the Baking Pi USB driver is implemented, for read/write fat file sys, that out puts memory.map info, that only includes emmc and block plus fat etc.
This would be very useful to ASM programmers to include in there OS, as writing full fat in assembly is a pain.
I'll try (and probably include ext2 too). What interface functions would you expect this to have? It'd probably need a working malloc though.
DexOS wrote:Thanks again, Dex.
You're welcome. I appreciate the feedback.

Regards,
John.

User avatar
DexOS
Posts: 876
Joined: Wed May 16, 2012 6:32 pm
Contact: Website

Re: New Raspberry Pi bootloader (rpi-boot)

Fri Mar 01, 2013 9:41 pm

Thanks, i will test the update ver and let you know how it went.

Function: just these "fopen/fread/fclose/opendir/readdir/closedir" .
I want to load it to a set address (which i can change by recompiling) and from the memory.map info, use those address to call the functions, thats how i use the USB driver.
Batteries not included, Some assembly required.

jnc100
Posts: 54
Joined: Wed Feb 20, 2013 10:10 pm

Re: New Raspberry Pi bootloader (rpi-boot)

Fri Mar 01, 2013 10:23 pm

If a static library is okay, then I've just uploaded a version which you can try building with 'make libfs.a'. You need to call 'void libfs_init()' first to detect the SD.

It requires (at least) printf() (which could do nothing), malloc() (which is required to return 4-byte aligned pointers), free(), memcpy(), memset(), strlen(), strncpy(), errno, memory_barrier() (which is a standard ARMv6 memory barrier - possibly could do nothing), __aeabi_uidiv() and __aeabi_uidivmod() (the last two from libgcc).

This will supply fopen, fclose, fseek, opendir, readdir, closedir and usleep (I have to pull in the timer functions to get the sd driver to support appropriate delays).

Regards,
John.

User avatar
DexOS
Posts: 876
Joined: Wed May 16, 2012 6:32 pm
Contact: Website

Re: New Raspberry Pi bootloader (rpi-boot)

Fri Mar 01, 2013 11:22 pm

jnc100 wrote:If a static library is okay, then I've just uploaded a version which you can try building with 'make libfs.a'. You need to call 'void libfs_init()' first to detect the SD.

It requires (at least) printf() (which could do nothing), malloc() (which is required to return 4-byte aligned pointers), free(), memcpy(), memset(), strlen(), strncpy(), errno, memory_barrier() (which is a standard ARMv6 memory barrier - possibly could do nothing), __aeabi_uidiv() and __aeabi_uidivmod() (the last two from libgcc).

This will supply fopen, fclose, fseek, opendir, readdir, closedir and usleep (I have to pull in the timer functions to get the sd driver to support appropriate delays).

Regards,
John.
That sounds great thanks, that would be very useful to a lot of OS dev's.

I test the new ver you modded, but for some reason the kernel.img was not getting loaded, i tried both the compiled and i compiled from source and i get the same, just this on booting http://archlinux.spider007.net/RPi/debug-screen.jpg
Hope this helps.
Dex.
Batteries not included, Some assembly required.

jnc100
Posts: 54
Joined: Wed Feb 20, 2013 10:10 pm

Re: New Raspberry Pi bootloader (rpi-boot)

Sat Mar 02, 2013 10:59 am

I've just fixed that - it was caused by an infinite loop between uart_putc() -> usleep() -> malloc() -> sbrk() -> printf() -> uart_putc() on startup caused by a recent commit.

Anyway, I'm now back with my development machine and RPi so can properly test commits before pushing them (which can only be a good thing).

Regards,
John.

User avatar
DexOS
Posts: 876
Joined: Wed May 16, 2012 6:32 pm
Contact: Website

Re: New Raspberry Pi bootloader (rpi-boot)

Sun Mar 03, 2013 12:52 am

jnc100 wrote:I've just fixed that - it was caused by an infinite loop between uart_putc() -> usleep() -> malloc() -> sbrk() -> printf() -> uart_putc() on startup caused by a recent commit.

Anyway, I'm now back with my development machine and RPi so can properly test commits before pushing them (which can only be a good thing).

Regards,
John.
Just tested the new ver, works great.
I was able to load and run a flat binary file.
Thanks.
Batteries not included, Some assembly required.

Joeboy
Posts: 24
Joined: Wed Oct 31, 2012 11:59 am

Re: New Raspberry Pi bootloader (rpi-boot)

Sun Mar 03, 2013 10:07 am

Thanks Dex and jnc100, am busy with real life at the moment but I'll be checking this out as soon as I have some time.

jnc100
Posts: 54
Joined: Wed Feb 20, 2013 10:10 pm

Re: New Raspberry Pi bootloader (rpi-boot)

Sun Mar 03, 2013 9:52 pm

DexOS wrote:Just tested the new ver, works great.
I was able to load and run a flat binary file.
Thanks.
Glad to hear it (finally) works. By the way, are you loading from a FAT32 or 16 partition?

Regards,
John.

User avatar
DexOS
Posts: 876
Joined: Wed May 16, 2012 6:32 pm
Contact: Website

Re: New Raspberry Pi bootloader (rpi-boot)

Sun Mar 03, 2013 10:39 pm

jnc100 wrote:
DexOS wrote:Just tested the new ver, works great.
I was able to load and run a flat binary file.
Thanks.
Glad to hear it (finally) works. By the way, are you loading from a FAT32 or 16 partition?

Regards,
John.
I did the test on a fat16, but i will reformat and test fat32 and let you know.
[EDIT] Yes it also works on fat32 too, great work.

Also it does not matter about the standalone fat driver, as i have decide to bite the bullet and code the fat driver in ASM.
Regards Dex.
Batteries not included, Some assembly required.

tgritchie
Posts: 19
Joined: Fri Jun 01, 2012 4:07 pm
Location: United Kingdom

Re: New Raspberry Pi bootloader (rpi-boot)

Fri Mar 08, 2013 10:12 am

Hello jnc100 - I checked out your bootloader, built it and tested it with few problems on a 4 GB Transcend SDCard. Many thanks for this contribution to the Raspi alternative OS effort. But I ran into some issues when I tried it on a Traveler 2GB card. It seems that on this card the code is having difficulty reading the MBR (among other things). Here is the debug printout. I note you have posted other messages about SDcard issues and as I am not an expert I was wondering if you could cast any light on what is happening here.
  • ATAG_CORE
    flags
    00000000
    pagesize
    00000000
    rootdev
    00000000

    ATAG_MEM
    start
    00000000
    size
    0c000000

    Unknown ATAG
    54410009

    fb_init, fb_addr:
    0d385000
    Successfully set up frame buffer
    Welcome to Rpi bootloader
    ARM system type is c42
    DWC_USB: beginning core reset
    DWC_USB: core reset complete
    DWC_USB: Initialized host controller dwc_usb0
    EMMC: vendor 99, sdversion 2, slot_status 0
    EMMC: resetting controller
    EMMC: control0: 00000000, control1: 00000000
    EMMC: capabilities: 0000000000000000
    EMMC: checking for an inserted card
    EMMC: status: 1fff0000
    EMMC: setting clock rate
    EMMC: control0: 00000000, control1: 00070103
    EMMC: enabling SD clock
    SD: sent CMD0 done
    SD: sent CMD8, received response: 000001aa
    SD: sending ACMD41: CMD55: done, response 00000120, ACMD41: done
    SD: ACMD41 response: 00ff8001
    SD: card not yet ready
    SD: sending ACMD41: CMD55: done, response 00000120, ACMD41: done
    SD: ACMD41 response: 80ff8000
    SD: card identified: OCR: ff80, 1.8v support: 0, SDHC support: 0
    SD: card CID: 0002544d534430324738a5979c32008a
    SD: CMD3 response: e7f40520
    SD: RCA: e7f4
    SD: found a valid SD card
    SD: setup successful (status 3)
    MBR: reading block 0 from device emmc0
    SD: read() obtaining status register: status 4
    SD: read() card ready, reading from block 0
    SD: received error interrupt: 00028001
    SD: error CMD17 response: 1b00
    MBR: block_read failed (-1)
    MAIN: device list:
    VFS: unable to determine device name when parsing /boot/rpi_boot.cfg
    VFS: unable to determine device name when parsing /boot/rpi-boot.cfg
    VFS: unable to determine device name when parsing /boot/grub/grub.cfg
    MAIN: No bootloader configuration file found

Note that this is only one example of the errors that are happening. Sometimes it doesn't even get as far as reading the MBR and just says something like SD cad unusable. I have however used the card for *standard* Linux kernel booting

Regards
Trevor

jnc100
Posts: 54
Joined: Wed Feb 20, 2013 10:10 pm

Re: New Raspberry Pi bootloader (rpi-boot)

Fri Mar 08, 2013 11:57 am

Thanks for the error report. According to the interrupt register you've posted, the card reported a CRC error. From the specs I'm unsure exactly as to what you're supposed to do in this circumstance, but I guess a number of retries are probably in order +/- resetting the card (which I don't currently do). I'd imagine linux does something along these lines. I'll try to add a re-try attempt in case of CRC error later and see whether that fixes it. Otherwise I suppose it may be a timing issue.

For what its worth I'll probably add support for ATAG_CMDLINE (0x54410009) too.

Regards,
John.

tgritchie
Posts: 19
Joined: Fri Jun 01, 2012 4:07 pm
Location: United Kingdom

Re: New Raspberry Pi bootloader (rpi-boot)

Fri Mar 08, 2013 1:00 pm

OK - I'll look forward to it. I guess retries on CRC errors is a good thing though I should point out that this card may be a bit flaky in general as, although I have booted linux from it I have also noticed some mysterious quirks while doing so. Things like corrupt files, file system going read only and stuff like that.

HTH
Trevor

jnc100
Posts: 54
Joined: Wed Feb 20, 2013 10:10 pm

Re: New Raspberry Pi bootloader (rpi-boot)

Fri Mar 08, 2013 9:46 pm

I've just uploaded a version which retries a further 2 times when it receives a CRC check error (either command or data) from the CMD17 (block read) command only. Given you were having problems at various points before this it's probable there were similar errors at other points which I haven't yet written the retry logic for, but if you could keep rebooting with this card until you get to the CMD17 stage again I'd be grateful.

Regards,
John.

Return to “Bare metal, Assembly language”