Pate
Posts: 115
Joined: Tue Feb 05, 2013 9:04 am
Location: Finland

Re: New Raspberry Pi bootloader (rpi-boot)

Sat Mar 09, 2013 11:17 am

Okay, I finally got around to testing this myself. Here's what I did:
1) Copied everything from my working Raspbian /boot directory to root directory of my old Transcend 2GB MicroSD card (which already contains about 1.5GB other data).
2) Downloaded your "rpi-boot-latest" package and extracted into the root of the SDcard,
3) Put the MicroSD card into and SD adapter and put it into my RPi.

The result is:

Code: Select all

Welcome to Rpi bootloader
ARM system type is c42
DWC_USB: Initialized host controller dwc_usb0
EMC: vendor 99, sdversion 2, slot_status 0
SD: card not yet ready
SD: found a valid SD card
I suspect it should do something else after that?

I did not check the SDcard for errors, might some bad directory entries cause it not being able to continue? I have used the same SDcard for a couple of years in my DSx86 tests, and it can very well have all sorts of file system problems. Is there some verbose/debug option, or how would I get the debug printout mentioned above in the thread?

Thanks!
Now working on piro: http://piro.patrickaalto.com
See my rpix86 project at http://rpix86.patrickaalto.com

jnc100
Posts: 54
Joined: Wed Feb 20, 2013 10:10 pm

Re: New Raspberry Pi bootloader (rpi-boot)

Sat Mar 09, 2013 12:05 pm

Pate wrote:Is there some verbose/debug option, or how would I get the debug printout mentioned above in the thread?
The debug version is available from here or by compiling with 'CFLAGS=-DDEBUG2 make'

Have you been able to successfully boot Raspian off this card?

Regards,
John.

Pate
Posts: 115
Joined: Tue Feb 05, 2013 9:04 am
Location: Finland

Re: New Raspberry Pi bootloader (rpi-boot)

Sat Mar 09, 2013 12:30 pm

Thanks for the quick reply! The debug version shows the following at the end:

Code: Select all

SD: found a valid SD card
SD: setup successful (status 3)
MBR: reading block 0 from device emmc0
SD: read() obtaining status register: status 4
SD: read() card ready, reading from block 0
SD: read command complete, waiting for data (current INTERRUPT: 00000000, STATUS: 01ff0206)
SD: awaiting buffer read ready interrupt SD: interrupt 00208000
I suspect your question about having succesfully booted Raspbian from this card is the key here. :oops:
I actually did read your README, instructing us to copy the files to an SD card containing an existing Raspbian installation, but I decided to see what happens when I don't do that. So, just to confirm, your boot loader does need something sensible to be in the MBR of the SDcard (which in my case probably is empty)? I guess I'll next install a 2GB Raspbian distribution to the card and if that boots only then try again with your files.

Edit: Okay, I installed a proper Raspbian Wheezy installation to the SDcard, and that boots fine now. However, the same thing still happens with your boot loader.

Pate
Now working on piro: http://piro.patrickaalto.com
See my rpix86 project at http://rpix86.patrickaalto.com

jnc100
Posts: 54
Joined: Wed Feb 20, 2013 10:10 pm

Re: New Raspberry Pi bootloader (rpi-boot)

Sat Mar 09, 2013 1:57 pm

Pate wrote:The debug version shows the following at the end:

Code: Select all

...
SD: awaiting buffer read ready interrupt SD: interrupt 00208000
This is the controller signalling that the CRC check on the data transferred failed (unlike the CRC failing on the command response which happened in the case above). I've added some extra checks into the latest version just uploaded that retries on failure in this case.
Pate wrote:So, just to confirm, your boot loader does need something sensible to be in the MBR of the SDcard (which in my case probably is empty)?
No, it just requires a valid partition table (which any tool to create partitions should have done already)
Pate wrote:Edit: Okay, I installed a proper Raspbian Wheezy installation to the SDcard, and that boots fine now. However, the same thing still happens with your boot loader.
Thanks, was just checking that it wasn't an incompatibility between your card and the reader in the Pi (which appears to be temperamental at the best of times).

Regards,
John.

tgritchie
Posts: 19
Joined: Fri Jun 01, 2012 4:07 pm
Location: United Kingdom

Re: New Raspberry Pi bootloader (rpi-boot)

Sat Mar 09, 2013 2:49 pm

Hello again - I just checked out revision 67 and there seems to be more consistency in the log data between power cycles and indeed some retries of CMD17 but I think it still fails to read the MBR :( Here is the log:
  • fb_init, fb_addr:
    0d385000
    Successfully set up frame buffer
    Welcome to Rpi bootloader
    ARM system type is c42
    DWC_USB: beginning core reset
    DWC_USB: core reset complete
    DWC_USB: beginning post reset init
    DWC_USB: completed post reset init
    DWC_USB: Initialized host controller dwc_usb0
    EMMC: vendor 99, sdversion 2, slot_status 0
    EMMC: resetting controller
    EMMC: control0: 00000000, control1: 00000000
    EMMC: capabilities: 0000000000000000
    EMMC: checking for an inserted card
    EMMC: status: 1fff0000
    EMMC: setting clock rate
    EMMC: control0: 00000000, control1: 00070103
    EMMC: enabling SD clock
    SD: sent CMD0 done
    SD: sent CMD8, received response: 000001aa
    SD: sending ACMD41: CMD55: done, response 00000120, ACMD41: done
    SD: ACMD41 response: 01ff8001
    SD: card not yet ready
    SD: sending ACMD41: CMD55: done, response 00000120, ACMD41: done
    SD: ACMD41 response: 80ff8000
    SD: card identified: OCR: ff80, 1.8v support: 0, SDHC support: 0
    SD: card CID: 0002544d534430324738a5979c32008a
    SD: CMD3 response: e7f40520
    SD: RCA: e7f4
    SD: found a valid SD card
    SD: setup successful (status 3)
    MBR: reading block 0 from device emmc0
    SD: read() obtaining status register: status 4
    SD: read() card ready, reading from block 0
    SD: received error interrupt: 00028001 CMC_CRC
    SD: error invalid CMD17 response: 1b00
    SD: read() obtaining status register: status 4
    SD: read() card ready, reading from block 0
    SD: received error interrupt: 00028001 CMC_CRC
    SD: error invalid CMD17 response: 1b00
    SD: read() obtaining status register: status 4
    SD: read() card ready, reading from block 0
    SD: received error interrupt: 00028001 CMC_CRC
    SD: error invalid CMD17 response: 1b00
    MBR: block_read failed (-1)
    MAIN: device list:
    VFS: unable to determine device name when parsing /boot/rpi_boot.cfg
    VFS: unable to determine device name when parsing /boot/rpi-boot.cfg
    VFS: unable to determine device name when parsing /boot/grub/grub.cfg
    MAIN: No bootloader configuration file found
Just to be sure I restored the linux kernel to kernel.img and that still seems to work correctly. I can post the startup log if you like

Trevor

Pate
Posts: 115
Joined: Tue Feb 05, 2013 9:04 am
Location: Finland

Re: New Raspberry Pi bootloader (rpi-boot)

Sat Mar 09, 2013 6:39 pm

Thanks for the new version!

It now loops three times, each time ending with the same INTERRUPT 00208000 and then "timeout. Resetting card.".

The end result is the same as what tgritchie gets above.

Hope this helps, it really does seem like the card reader system is pretty tricky to get working with the multitude of SD card types out there. Good luck! :)

Pate
Now working on piro: http://piro.patrickaalto.com
See my rpix86 project at http://rpix86.patrickaalto.com

jnc100
Posts: 54
Joined: Wed Feb 20, 2013 10:10 pm

Re: New Raspberry Pi bootloader (rpi-boot)

Sat Mar 23, 2013 11:55 pm

I've just pushed a new version with a complete rewrite of the sd card code, which accurately follows the SD Host Controller Simplified Spec. I hope it will fix many of the problems people have been having.

Regards,
John.

Pate
Posts: 115
Joined: Tue Feb 05, 2013 9:04 am
Location: Finland

Re: New Raspberry Pi bootloader (rpi-boot)

Sun Mar 24, 2013 9:36 am

Hi, thanks for the new version!

Sadly, looks like it does not improve things much for me.. It loops a few times, then:

Code: Select all

SD: error issuing command: interrupts 00208001: DATA_CRC
SD: error sending READ_SINGLE_BLOCK command, error = 00200000. Giving up.
MBR: block_read failed(-1)
MAIN: device list:
VFS: unable to determine device name when parsing /boot/rpi_boot.cfg
VFS: unable to determine device name when parsing /boot/rpi-boot.cfg
VFS: unable to determine device name when parsing /boot/grub/grub.cfg
MAIN: No bootloader configuration file found
Too bad, this would really be nice to get working.

Pate
Now working on piro: http://piro.patrickaalto.com
See my rpix86 project at http://rpix86.patrickaalto.com

jnc100
Posts: 54
Joined: Wed Feb 20, 2013 10:10 pm

Re: New Raspberry Pi bootloader (rpi-boot)

Sun Mar 24, 2013 3:18 pm

The only way I can reproduce the DATA_CRC error you are getting is to put a bogus block size value in BLKSIZECNT, so I have just pushed a version which explicitly sets it before every read. It probably won't fix your problem but I guess its worth a try.

Other than that I'm working on a DMA version which may fix things.

Regards,
John.

Pate
Posts: 115
Joined: Tue Feb 05, 2013 9:04 am
Location: Finland

Re: New Raspberry Pi bootloader (rpi-boot)

Mon Mar 25, 2013 1:20 pm

Hi!

Sorry it took me a while to get around to test your latest version. You are correct, it did not fix the problem.

Let's hope the DMA version works better. :)

Pate
Now working on piro: http://piro.patrickaalto.com
See my rpix86 project at http://rpix86.patrickaalto.com

jnc100
Posts: 54
Joined: Wed Feb 20, 2013 10:10 pm

Re: New Raspberry Pi bootloader (rpi-boot)

Tue Apr 02, 2013 2:15 pm

I've been able to fix a CRC error experienced by one forum member by sorting out the SD clock rate (particularly by slowing it down for his old card) and the changes have now been uploaded. It may well fix CRC errors that other people are experiencing.

Regards,
John.

Pate
Posts: 115
Joined: Tue Feb 05, 2013 9:04 am
Location: Finland

Re: New Raspberry Pi bootloader (rpi-boot)

Wed Apr 03, 2013 4:07 am

Great, seems to work fine here now!

I first tested with the debug version, but it generated tons and tons of text. I managed to read a lot of "read succesfully" messages, so I switched to the non-debug version. It shows (after a little while):

Code: Select all

Welcome to the test kernel
Multiboot magic: 2badb002
Running on machine type: c42
I am assuming this is the correct behaviour?

Very nice, thanks for your hard work in getting this working also on our old/slow SD cards! :)

Pate
Now working on piro: http://piro.patrickaalto.com
See my rpix86 project at http://rpix86.patrickaalto.com

jnc100
Posts: 54
Joined: Wed Feb 20, 2013 10:10 pm

Re: New Raspberry Pi bootloader (rpi-boot)

Wed Apr 03, 2013 6:32 am

Glad to hear it's working.
Pate wrote:

Code: Select all

Welcome to the test kernel
Multiboot magic: 2badb002
Running on machine type: c42
Indeed it is. The test kernel just demonstrates some basic functionality of the system. The reason its slow to load is mainly in the slowness of the clear screen function.

Regards,
John.

User avatar
DexOS
Posts: 876
Joined: Wed May 16, 2012 6:32 pm
Contact: Website

Re: New Raspberry Pi bootloader (rpi-boot)

Wed Apr 03, 2013 12:24 pm

@jnc100, Great work, are you testing each card and slowing clock down or up accordingly ? or just slow it down any way, as the mailbox has good easy to use functions, for getting max clock speed for sd card.
Batteries not included, Some assembly required.

tufty
Posts: 1456
Joined: Sun Sep 11, 2011 2:32 pm

Re: New Raspberry Pi bootloader (rpi-boot)

Wed Apr 03, 2013 12:59 pm

DexOS wrote:@jnc100, Great work, are you testing each card and slowing clock down or up accordingly ? or just slow it down any way, as the mailbox has good easy to use functions, for getting max clock speed for sd card.
Looks like the former, see http://www.johncronin.org.uk/svn/rpi-boot/emmc.c

jnc100
Posts: 54
Joined: Wed Feb 20, 2013 10:10 pm

Re: New Raspberry Pi bootloader (rpi-boot)

Wed Apr 03, 2013 3:26 pm

DexOS wrote:@jnc100, Great work, are you testing each card and slowing clock down or up accordingly ? or just slow it down any way, as the mailbox has good easy to use functions, for getting max clock speed for sd card.
From what I'm learning deciding on the correct frequency for SD cards is tricky.

The specs define two SD clock frequencies - fOD and fPP used for initialization and data transfer respectively. The exact value for these is hidden away in the simplified specs and despite many days of re-reading and searching I've only just found out whilst in the process of writing the reply that the value fOD should probably be 400 kHz. fPP depends on the card - it can be 25 MHz (default), 50 MHz (high speed mode), or in UHS-I mode it can be 25, 50, 100 or 208 MHz.

The official way of setting the clock speed is to use the clock control register at offset 0x2c. The value you put in here can be determined in two ways. First, some controllers contain a set of 'preset' registers with values to read and put in the clock control register defined for each operating mode. Unfortunately the Arasan circuitry doesn't provide this. The second way is to use one of the clock control modes (either 10-bit divided clock mode or programmable clock mode) to divide the base clock up to an appropriate value.

To do this, you need to know both the base clock rate provided to the eMMC controller and the value of 'M' (if you are using programmable clock mode). On most host controllers, both of these are found in the capabilities register, which again is not present in the Arasan chip. Thus without knowledge of the magic 'M' value, it is impossible to calculate a divisor using the programmable clock mode (perhaps this knowledge could be gleaned from the linux module sources?). If the base clock rate is not found in the capabilities register, the SDHCI docs state it should be found 'by other means'. In the case of the Broadcom chip we can use the mailbox interface to query the clock rate supplied to the eMMC controller and then calculate our divisor from that.

The boot-up value of the base eMMC clock in the RPi is currently 100 MHz (as I have recently discovered). What I was previously doing (based upon code found somewhere in these forums) was using programmable clock mode to set the SD clock frequency to 'base clock * M / 2' which if we assume the value of 'M' is 1 equates to 50 MHz, for higher values of 'M' it is even quicker, which obviously wouldn't work with anything but the most modern cards. I think the reason the code was provided here was that that I think (based upon reading old code lying around) the base eMMC clock was 50 MHz in previous firmware versions (thus giving a valid SD clock of 25 MHz) but has since been increased.

The new version of my code does not change the base eMMC clock but simply reads its value and calculates an appropriate divisor for 10-bit programmable clock mode. To get things working I currently set all cards to run at 12.5 MHz (but enable 4-bit mode and UHS-I for quicker transfers if they are available) although I imagine what I should probably do is use 400 kHz for the identification phase and then switch up to 25 MHz for data transfers however as things seem to work for everyone now I'm reluctant to make this change.

I do not know what would happen if you change the clock frequency by using the mailbox interface intstead of the internal divider within the eMMC circuitry, as this would affect the clock rate at which the host interfaces with the rest of the ARM and Broadcom peripherals, as well as the SD clock rate (which is the rate the host interfaces with the card). Also, you are supposed to switch the SD card clock off when changing its rate and then re-enable it afterwards, so using the in-built divider may be easier anyway (as you can do it all with writing to the same register).

Regards,
John.

tgritchie
Posts: 19
Joined: Fri Jun 01, 2012 4:07 pm
Location: United Kingdom

Re: New Raspberry Pi bootloader (rpi-boot)

Fri Apr 05, 2013 8:25 am

jnc100 wrote:I've been able to fix a CRC error experienced by one forum member by sorting out the SD clock rate (particularly by slowing it down for his old card) and the changes have now been uploaded. It may well fix CRC errors that other people are experiencing.

Regards,
John.

Hello John - Like others I have rebuilt the latest version and it now works a treat even on the *flaky* Traveler SD card I mentioned some while ago :D. Thanks very much again for this effort. One feature I don't believe is available yet that you might like to consider is remote download of test kernels - either over the serial port for small ones or perhaps over Ethernet if/when that route has driver code available. The potential problem I have been encountering is mechanical stress of the SD card connector. In another context I have been using some code from dwelch67 to alleviate this *sd card dance* as he calls it. I may investigate incorporating some parts of his code into your bootloader to see if I can reduce the number of card swaps I have to make. Perhaps you know of a better way - I'd be grateful for your thoughts

Cheers
Trevor

jnc100
Posts: 54
Joined: Wed Feb 20, 2013 10:10 pm

Re: New Raspberry Pi bootloader (rpi-boot)

Fri Apr 05, 2013 11:01 am

I understand the problems of the constant SD switching all too well - I have already snapped the corner off my Raspbian SD card with all the changes in making this bootloader (its now reasonably symmetrical). I plan on ethernet support at some time to give a PXE-like interface, but obviously this relies on me getting USB support and then a driver for the RPi ethernet adapter working first. Others have suggested using a separate ethernet adapter connected to the GPIO pins which may be easier although obviously not everyone would have that. However, once I get USB working, my first task would probably be USB mass storage support first.

There is a small bootloader by mrvn which transfers kernels over the serial protocol, see https://github.com/mrvn/raspbootin.

I was considering adding support for this to rpi-boot but haven't yet decided on a serial protocol to use. Current possibilities are probably YMODEM or TFTP over SLIP/PPP. This is a long way down the line yet though as I don't even have a cable to hook up the UART to USB at the moment.

edit: if mrvn is willing, I may update his raspbootcom server to support sending multiple files across the link (e.g. for modules), possibly by changing the identification string from \003\003\003 to something like \003\003\004 to identify a different version of his protocol.

Regards,
John.

tgritchie
Posts: 19
Joined: Fri Jun 01, 2012 4:07 pm
Location: United Kingdom

Re: New Raspberry Pi bootloader (rpi-boot)

Fri Apr 05, 2013 4:17 pm

jnc100 wrote:I understand the problems of the constant SD switching all too well - I have already snapped the corner off my Raspbian SD card with all the changes in making this bootloader (its now reasonably symmetrical).
Don't I know it. Mine are heading the same way :(
jnc100 wrote: However, once I get USB working, my first task would probably be USB mass storage support first.
I don't know how good it is but you may like to look at some USB code that looks useful from Alex Chadwick at https://github.com/Chadderz121/csud
jnc100 wrote:There is a small bootloader by mrvn which transfers kernels over the serial protocol, see https://github.com/mrvn/raspbootin.
I was considering adding support for this to rpi-boot but haven't yet decided on a serial protocol to use. Current possibilities are probably YMODEM or TFTP over SLIP/PPP. This is a long way down the line yet though as I don't even have a cable to hook up the UART to USB at the moment.
I looked briefly at this and may investigate it further. The code I have already mentioned (and indeed used as I do have a connection to the Raspi UART) from dwelch67 is here https://github.com/dwelch67/raspberrypi and at least one of his bootloaders uses Xmodem and, while relatively simple, works very well.

As regards the minimum one would have to do (apart from the file transfer protocol) to minimise or even eradicate the SD card dance, would I be correct in assuming that the ability to understand the FAT filesystem and possibly edit a few files in it would be enough?

Trevor

jnc100
Posts: 54
Joined: Wed Feb 20, 2013 10:10 pm

Re: New Raspberry Pi bootloader (rpi-boot)

Sat Apr 06, 2013 4:40 pm

tgritchie wrote:
jnc100 wrote: However, once I get USB working, my first task would probably be USB mass storage support first.
I don't know how good it is but you may like to look at some USB code that looks useful from Alex Chadwick at https://github.com/Chadderz121/csud
Thanks, I was aware of that, but it would require some modification to get it working with either ethernet or mass storage devices at present.
tgritchie wrote:The code I have already mentioned (and indeed used as I do have a connection to the Raspi UART) from dwelch67 is here https://github.com/dwelch67/raspberrypi and at least one of his bootloaders uses Xmodem and, while relatively simple, works very well.
The only issue with using XMODEM is that you have to constantly switch between an XMODEM client and a terminal program (for debug) on your host - the raspbootin idea is quite elegant in that the host runs a program which is both a file transfer server and a serial terminal and switches between the two by detecting a magic sequence of bytes (\003\003\003) in the data stream.

I've added some preliminary support for other functions to the protocol https://github.com/jncronin/rpi-boot/bl ... spbootin.c which support CRC checks of the sent information and the ability to browse directories on the server and download parts of a particular file (so you could have your rpi-boot.cfg config file, kernel and modules on the server, and then use the fread() function to load further files as required from either the server or the SD).
tgritchie wrote:As regards the minimum one would have to do (apart from the file transfer protocol) to minimise or even eradicate the SD card dance, would I be correct in assuming that the ability to understand the FAT filesystem and possibly edit a few files in it would be enough?
I don't think FAT write support is a must for this (you can just change the files on the server instead) although it is planned for a future release.

Regards,
John.

tgritchie
Posts: 19
Joined: Fri Jun 01, 2012 4:07 pm
Location: United Kingdom

Re: New Raspberry Pi bootloader (rpi-boot)

Sun Apr 07, 2013 9:20 pm

Hello John - some more thoughts and queries below
jnc100 wrote: I've added some preliminary support for other functions to the protocol https://github.com/jncronin/rpi-boot/bl ... spbootin.c which support CRC checks of the sent information and the ability to browse directories on the server and download parts of a particular file (so you could have your rpi-boot.cfg config file, kernel and modules on the server, and then use the fread() function to load further files as required from either the server or the SD).
I notice this is code is on github. Have you moved development there or is the svn repository on your web site still the main source?
jnc100 wrote: ..
tgritchie wrote:As regards the minimum one would have to do (apart from the file transfer protocol) to minimise or even eradicate the SD card dance, would I be correct in assuming that the ability to understand the FAT filesystem and possibly edit a few files in it would be enough?
I don't think FAT write support is a must for this (you can just change the files on the server instead) although it is planned for a future release.

Regards,
John.
Fair enough for ongoing development of custom kernels etc. but ideally I'd like not to have to do the SD card dance at all - even when I want to reinstate the full linux kernel. I think I worked out that the minimum you would need to do this is for your rpi-boot code to be able to swap the names of two .img files. Do you agree?

Trevor

jnc100
Posts: 54
Joined: Wed Feb 20, 2013 10:10 pm

Re: New Raspberry Pi bootloader (rpi-boot)

Sun Apr 07, 2013 9:41 pm

tgritchie wrote:I notice this is code is on github. Have you moved development there or is the svn repository on your web site still the main source?
They are both kept automatically in sync. I had some requests to move to github to make it easier for people to commit patches but keep it on my own server with svn to have it rebuild the image files whenever a new version is pushed.
tgritchie wrote:Fair enough for ongoing development of custom kernels etc. but ideally I'd like not to have to do the SD card dance at all - even when I want to reinstate the full linux kernel. I think I worked out that the minimum you would need to do this is for your rpi-boot code to be able to swap the names of two .img files. Do you agree?
Now I understand what you meant. A rename function is easy as long as long file names aren't required as these require embedding code pages and also have patent issues.

Regards,
John.

Return to “Bare metal, Assembly language”