128MB RS-MMC can't boot w/ any config - 7 blink at best

[FalconFour]

So, this is some seriously advanced digging I've pulled here... Google gives me a blank stare (of irrelevant crap) for every possible way I can put this.

I'm developing a little tool using an original Raspberry Pi model A and B, which use the full size SD cards. The OS I've put together is based on Tiny Core Linux (piCore). It takes up a total of about 117 MB with all stuff, including a development environment for recompiling (as I don't have a cross compiler setup on my PC). So a 128MB RS-MMC was practically a perfect fit for it. I'd been developing the system using a 1GB Kingston RS-MMC card, which has worked great. However, today I went out and picked up 3x 128MB RS-MMCs, and when I cloned it over to one of them, all hell broke loose.

It's a single partition setup (FAT16), so I first just copied the files into the new card. Booting up gives me a burst of "ACT", followed the rainbow screen, followed by 7 ACT LED flashes. This supposedly means "can't find kernel.img", but that is totally false - kernel.img is right there, just as it was with the 1GB card.
When I try loading the FAT files from an earlier Raspbian (2012-07-15, I think the first?), it flashes 4 times instead.
If I load the (root) files from NOOBS 1.5 onto it, I get 7 flashes and the rainbow awesomeness screen.

Basically the same thing no matter what I do, though the same things I do should always be booting it fine.


Back to the original (piCore), if I delete config.txt, I get 7 flashes. If I add "boot_delay=2", I get 7 flashes. If I add "bootcode_delay=20", I get a loooooooooooong delay... then 7 flashes.

So, it can read config.txt, bootcode.bin, and maybe even start.elf, but somewhere, something gets lost reading kernel.img. Except, maybe, it doesn't get lost *reading* it, but *finding* it? Since start.elf and various other boot components aren't open source and don't seem to be that well documented, I can't seem to find any useful info. Basically, the best I get is "it always works if you do it right, so don't ask questions and you'll be fine". Eh.

I really, really don't want to have to use 8GB SD cards for this. Just such a massive waste.

[rpdom]

If you are using the latest firmware there is a new "sdhost" driver that improves access to SD cards, but could be causing problems with your MMC cards.

Try adding "dtoverlay=mmc" to your config.txt to use the old driver.

[gregeric]

I'm not sure at what point the dtoverlay=mmc influences which of the two peripherals are used; the internal bootrom uses the sdhost hardware block to load bootcode.bin, and I suspect it's not until after the linux kernel is loaded that that line has any effect, so that may not help. It might just be down to the start.elf firmware revision. It will be interesting to see if that line does help.

[FalconFour]

Curious about two things there:

One, you mention "firmware". I've seen that before, but usually as a very old (original Pi) reference to "the SD card image" itself. Since I'd tried the original July 2012 firmware as well, isn't that effectively ruled out? Or is there other firmware on the board itself, somewhere?

Two, I have Device Tree disabled (as much as I could, with "device_tree=" in the config and "dtparm=..." commented out), as it seems to interfere with the PiTFT kernel driver being loaded via modprobe after booting. I only guess that because after several hours of debugging, I finally got it to work after adding that line. Isn't device tree a post-boot kernel thing, or could it be low level enough to mess with the boot process?

I'll give it a try either way - it's back at the shop and I'm back at home for the night. Will post back later for sure!

Ah! Also, good details on how the boot process gets started. So, the chip bootloader knows how to access the SD card to find that hardcoded path, "bootcode.bin", right? Maybe, much like a PC BIOS, it continues using that device access all the way through bootcode.bin, config.txt, then pulls up start.elf which has its own driver code that fails. Maybe on the right track there?

[gregeric]

There's a great write-up here of the boot process & device tree stuff.

[rpdom]

Well, I use a couple of 32MB and 128MB MMC cards(ok, not RS-MMC, but AFAIK those are just smaller versions) occasionally. The very early "firmware" (the initial bootcode.bin, start.elf etc) worked fine with those, then they stopped working for a bit, then started working again.

I leave device tree enabled. That's the way things are going.

[FalconFour]

The more I read about device tree, the less I seem to understand about it.

For embedded projects, I hate the bloat of Raspbian. It takes a good 45-60 seconds to boot up to a login prompt, which is just staggeringly unacceptable for project use - as it loads all tons of unnecessary network support, user data management, international keyboard/language options, system services, etc... though I'm not a real Linux geek myself, I found piCore to be better suited there. With some tweaking, it's up on its feet with the LCD driver and a splash screen loaded within 7 seconds of power-up, and running my app (written in C++ with framebuffer .bin's being un-gzipped and dumped to /dev/fb0 as an interface) within about 20 seconds. Mounting all the compressed extensions takes a while, but that's about it. Raspbian is like... dude, I write code for a living on Arduino Pro Mini's and Unos, not Python or Scratch... keep it simple!

(sorry, I had to get that out. )

That, and piCore is a little unfriendly to hacking the base RAM image - with the fbtft package I found, it's a .img-format rootfs package, and I can't figure out how to mount it to even view its contents, let alone modify it*. So, tinkering with device tree drivers and services was far down the "crap I really didn't want to wrestle with" path, as I doubt reading the dtree is even implemented in that tiny system. The module wanted to be called with certain parameters (I think "name=pitft rotate=90 fps=15"), and I didn't know how to get it to be called into action early enough into the boot - so it's just part of the startup script to load it.
* - I also tried Googling that, and got "just mount -o loop the thing", but when I try that, mount goes "nope, invalid, gfy". So I dunno.

All that said, I plow onward in the description of "device tree", and it almost sounds like you can't really do without it, as it sets GPIOs and stuff. Which is weird, since I couldn't get my LCD working without "crippling" it. I feel like it was interfering with the SPI interface of the kernel somehow...

It does give me the impression that it might work out to add that line... hell, I can't even find a config.txt in NOOBS - seems like it uses a different loading architecture (no start.elf) - and no mention of device tree at all in 2013-09-21's Raspbian files.

[gregeric]

From what you have written, you clearly know what you are doing, but Raspbian & the firmware blobs bootcode.bin & start.elf have changed a bit (for the better) since 2013!

[GTR2Fan]

For embedded projects, I hate the bloat of Raspbian. It takes a good 45-60 seconds to boot up to a login prompt, which is just staggeringly unacceptable for project use.

In fairness to Raspbian Jessie (non-Lite), it is designed to be pretty much a full-blown desktop OS. Run it on a modestly overclocked modern Pi, the Pi2B, and it can be at the desktop in around 12 seconds. Not knocking Windows, but my overclocked 3.6GHz quadcore C2D PC with the OS on an SSD takes longer than that to launch Win10 64-bit from a cold boot. Horses for courses.

[FalconFour]

OK, I have it here now - tried with my existing base piCore files, and modified the config.txt as so:

Code: Select all

# For more options and information see 
# http://www.raspberrypi.org/documentation/configuration/config-txt.md
# Some settings may impact device functionality. See link above for details

#initramfs 6.1.gz followkernel
initramfs rootfs_fbtft.img followkernel
kernel=kernel_fbtft.img

# uncomment if you get no picture on HDMI for a default "safe" mode
#hdmi_safe=1

# uncomment this if your display has a black border of unused pixels visible
# and your display can output without overscan
disable_overscan=1

# uncomment the following to adjust overscan. Use positive numbers if console
# goes off screen, and negative if there is too much border
#overscan_left=16
#overscan_right=16
#overscan_top=16
#overscan_bottom=16

# uncomment to force a console size. By default it will be display's size minus
# overscan
framebuffer_width=320
framebuffer_height=240

# uncomment if hdmi display is not detected and composite is being output
#hdmi_force_hotplug=1

# uncomment to force a specific HDMI mode (this will force VGA)
#hdmi_group=1
#hdmi_mode=1

# uncomment to force a HDMI mode rather than DVI. This can make audio work in
# DMT (computer monitor) modes
#hdmi_drive=2

# uncomment to increase signal to HDMI, if you have interference, blanking, or
# no display
#config_hdmi_boost=4

# uncomment for composite PAL
#sdtv_mode=2

#uncomment to overclock the arm. 700 MHz is the default.
#arm_freq=800

#----------------------------------------------------
dtparam=i2c_arm=on,spi=off
dtoverlay=mmc
#device_tree=

Contents of SD card root:

Code: Select all

 Volume in drive D is PICORE
 Volume Serial Number is F2AD-5D8D

 Directory of D:\

03/30/2015  04:23 AM             6,154 fixup.dat
03/30/2015  04:23 AM             2,365 fixup_cd.dat
03/30/2015  04:23 AM             9,210 fixup_x.dat
02/16/2015  02:57 PM               137 issue.txt
03/26/2015  04:06 PM         4,040,856 kernel.img
01/25/2016  07:32 PM         8,804,844 kernel_fbtft.img
02/15/2015  11:20 AM             1,447 LICENCE.broadcom
08/08/2015  02:13 AM            23,288 mzt280
01/07/2015  10:34 AM             5,691 README
06/24/2015  08:22 AM                46 RELEASE
01/25/2016  07:33 PM         7,762,836 rootfs_fbtft.img
03/30/2015  04:23 AM         3,540,280 start.elf
03/30/2015  04:23 AM           586,616 start_cd.elf
03/30/2015  04:23 AM         4,635,752 start_x.elf
03/30/2015  04:33 AM                41 .firmware_revision
06/24/2015  08:13 AM         3,851,710 6.1.gz
01/01/1980  12:00 AM         3,855,218 6.1.gz.new
03/30/2015  04:23 AM             4,423 bcm2708-rpi-b.dtb
03/30/2015  04:23 AM             4,702 bcm2708-rpi-b-plus.dtb
03/30/2015  04:23 AM             5,690 bcm2709-rpi-2-b.dtb
03/30/2015  04:23 AM            17,864 bootcode.bin
01/27/2016  08:50 PM               130 cmdline.txt
01/28/2016  03:47 PM             1,513 config.txt
02/15/2015  11:20 AM            18,693 COPYING.linux
01/27/2016  10:11 PM                75 fbtft.sh
01/27/2016  09:18 PM    <DIR>          overlays
01/28/2016  03:50 PM                 0 copypasta.txt
              26 File(s)     37,179,581 bytes
               1 Dir(s)      85,295,104 bytes free

(I don't have the TCE extensions or my app folders loaded, as I have to prune a few things to make it fit)

And 7 flashes. Is there anything I can try to see what point in the boot it's actually failing?

[sbp]

Hi
I'm also using piCore on many different cards, and have never seen a problem as you have here.
If I were you I would download the newest piCore.img and burn to your problematic card.
If it can boot you know that the card is OK, and you can start looking where in the process of copying your stuff to the new card is failing.
Steen

[FalconFour]

Tried that too, with piCore-6.1.img, the one I started with in the very beginning.

7 flashes.

And I mean, these MMCs are dated 2007, and they roar by at some 12MB/s write rate, which is really quite surprising since they can be fully written in some 15 seconds flat. They're "dual voltage" cards by AData:



Kinda generic, I know. But it gets to be disturbing that there are absolutely no utilities I can find for Windows that will show me low-level details of the MMC/SD card, such as the controller, capabilities, version, etc., so I can compare the Kingston card that works with this one that doesn't.

[GTR2Fan]

I'm losing track now without reading back through the entire thread. Have you tried Raspbian Jessie Lite?

[gregeric]

I don't think there is anything you can do to fix this. Bootcode.bin is successfully loaded, which is bare metal Videocore code whose last job is to load & execute start.elf. That contains the ThreadX OS which the Videocore runs. I guess the ThreadX sdhost/emmc driver is not compatible with your MMC card, thus failing to read the kernel.img off it.

[GTR2Fan]

I really, really don't want to have to use 8GB SD cards for this. Just such a massive waste.

It may be a waste of potential storage space, but is that really worth worrying about when a genuine Sandisk 8GB Class 10 card (40MB/s reads, 10MB/s writes) with a 10-year warranty can be had for under £4 with free delivery?

https://mobymemory.com/uk/sandisk-ultra ... oCMu3w_wcB

[rpdom]

It may be a waste of potential storage space, but is that really worth worrying about when a genuine Sandisk 8GB Class 10 card (40MB/s reads, 10MB/s writes) with a 10-year warranty can be had for under £4 with free delivery?

That reminds of the time, just over ten years ago, when I was running a Seti@Home farm of Linux PCs. I stripped everything to the minimum I could to keep the cost down - except for the CPU.

They all net booted from a central server. Because the network cards I used didn't have boot ROMs, I made floppy disks with the boot code on.

For a couple of the systems I didn't have floppy disk drives spare, so I used CD-ROMs instead. A whole CD with an image of a 1.44MB floppy, with about 56KB of code on it

The farm ran very well for the few years I participated in the project. It also kept my house warm in winter (so much so that I needed to leave a window open).

As for the SD cards... if I can reuse and old card rather than buying a new one, then that is a saving. Even four quid is money and I have a very nice 32MB MMC card sitting right here

[mimi123]

OK, I have it here now - tried with my existing base piCore files, and modified the config.txt as so:

Code: Select all

# For more options and information see 
# http://www.raspberrypi.org/documentation/configuration/config-txt.md
# Some settings may impact device functionality. See link above for details

#initramfs 6.1.gz followkernel
initramfs rootfs_fbtft.img followkernel
kernel=kernel_fbtft.img

# uncomment if you get no picture on HDMI for a default "safe" mode
#hdmi_safe=1

# uncomment this if your display has a black border of unused pixels visible
# and your display can output without overscan
disable_overscan=1

# uncomment the following to adjust overscan. Use positive numbers if console
# goes off screen, and negative if there is too much border
#overscan_left=16
#overscan_right=16
#overscan_top=16
#overscan_bottom=16

# uncomment to force a console size. By default it will be display's size minus
# overscan
framebuffer_width=320
framebuffer_height=240

# uncomment if hdmi display is not detected and composite is being output
#hdmi_force_hotplug=1

# uncomment to force a specific HDMI mode (this will force VGA)
#hdmi_group=1
#hdmi_mode=1

# uncomment to force a HDMI mode rather than DVI. This can make audio work in
# DMT (computer monitor) modes
#hdmi_drive=2

# uncomment to increase signal to HDMI, if you have interference, blanking, or
# no display
#config_hdmi_boost=4

# uncomment for composite PAL
#sdtv_mode=2

#uncomment to overclock the arm. 700 MHz is the default.
#arm_freq=800

#----------------------------------------------------
dtparam=i2c_arm=on,spi=off
dtoverlay=mmc
#device_tree=

Contents of SD card root:

Code: Select all

 Volume in drive D is PICORE
 Volume Serial Number is F2AD-5D8D

 Directory of D:\

03/30/2015  04:23 AM             6,154 fixup.dat
03/30/2015  04:23 AM             2,365 fixup_cd.dat
03/30/2015  04:23 AM             9,210 fixup_x.dat
02/16/2015  02:57 PM               137 issue.txt
03/26/2015  04:06 PM         4,040,856 kernel.img
01/25/2016  07:32 PM         8,804,844 kernel_fbtft.img
02/15/2015  11:20 AM             1,447 LICENCE.broadcom
08/08/2015  02:13 AM            23,288 mzt280
01/07/2015  10:34 AM             5,691 README
06/24/2015  08:22 AM                46 RELEASE
01/25/2016  07:33 PM         7,762,836 rootfs_fbtft.img
03/30/2015  04:23 AM         3,540,280 start.elf
03/30/2015  04:23 AM           586,616 start_cd.elf
03/30/2015  04:23 AM         4,635,752 start_x.elf
03/30/2015  04:33 AM                41 .firmware_revision
06/24/2015  08:13 AM         3,851,710 6.1.gz
01/01/1980  12:00 AM         3,855,218 6.1.gz.new
03/30/2015  04:23 AM             4,423 bcm2708-rpi-b.dtb
03/30/2015  04:23 AM             4,702 bcm2708-rpi-b-plus.dtb
03/30/2015  04:23 AM             5,690 bcm2709-rpi-2-b.dtb
03/30/2015  04:23 AM            17,864 bootcode.bin
01/27/2016  08:50 PM               130 cmdline.txt
01/28/2016  03:47 PM             1,513 config.txt
02/15/2015  11:20 AM            18,693 COPYING.linux
01/27/2016  10:11 PM                75 fbtft.sh
01/27/2016  09:18 PM    <DIR>          overlays
01/28/2016  03:50 PM                 0 copypasta.txt
              26 File(s)     37,179,581 bytes
               1 Dir(s)      85,295,104 bytes free

(I don't have the TCE extensions or my app folders loaded, as I have to prune a few things to make it fit)

And 7 flashes. Is there anything I can try to see what point in the boot it's actually failing?

The Big and Nasty Blob(also known as start.elf) tries to use SDIO instead of MMC to read the kernel from SD

[FalconFour]

The Big and Nasty Blob(also known as start.elf) tries to use SDIO instead of MMC to read the kernel from SD

Is there a chance in frozen hell this could ever be corrected somehow, so start.elf can actually use MMC to read it? Right now I just feel remarkably stupid that I bought these MMC cards ($7.50 ea) from a local shop, hoping to be able to give these obsolete relics some purpose in life, only to find they won't work. Plus, my application needs the SD card to be pre-installed and not end up broken-off during shipping back and forth... I never made mention of the SD card in my instruction video since they should never even need to know it exists

Ah well... maybe it'll get fixed some day, somehow, but I guess I'm just carrying on with other cards for now.

[DougieLawson]

What happens if you add

Code: Select all

dtoverlay=mmc

to config.txt in your first primary FAT partition?

[gregeric]

I think we've been there already - 7 flashes. Nothing can be done to influence it, save getting @Dom to fix it.

I've a 16MiB MMC which the Pi won't load custom or kosher bootcode.bin from.

[FalconFour]

Yep, tried the dtoverlay=mmc - still 7 flashes. I also Googled it and dug up additional parameters - like force_pio and overclock_50. I tried force_pio=1, still same. And I found that MMC should run at 20MHz, so I tried overclock_50=20, same thing.

But I dug into how Device Tree works, and I didn't find a matching "mmc-overlay.dtb" file in /overlays/, so I wonder if my piCore image simply doesn't have it. So, I'm downloading the latest Raspbian image (instead of NOOBS) to see if it'll make a difference. In the NOOBS distro, I found that os/Raspbian/boot.tar.xz contains a file, /overlays/mmc-overlay.dtb, as expected.

So, a little more tinkering coming up in a moment...

update: Nope... once again, extracted all the Raspbian root files (2015-11-21), tweaked config.txt to add the line "dtoverlay=mmc", no go. Same with adding overclock_50=20, and force_pio=1. All the same 7 flashes.

I do think the official description of the flash codes might be worth updating for this scenario, though...

[FalconFour]

Worth calling out in its own reply...

Aside from blink codes, is there any other way to get debug info out of the start.elf file? I have a small army of debug tools (Arduino for I/O parsing, a Bus Pirate, a cheap Chinese jtagice knockoff, scope, and I think even a PC-connected logic analyzer), at my disposal... though not a whole lot of experience/software/utilities interfacing with something like the Pi. If so, any tips on where to start?

[mimi123]

Worth calling out in its own reply...

Aside from blink codes, is there any other way to get debug info out of the start.elf file? I have a small army of debug tools (Arduino for I/O parsing, a Bus Pirate, a cheap Chinese jtagice knockoff, scope, and I think even a PC-connected logic analyzer), at my disposal... though not a whole lot of experience/software/utilities interfacing with something like the Pi. If so, any tips on where to start?

There is a leaked non-stripped firmware build somewhere.

[mimi123]

What happens if you add

Code: Select all

dtoverlay=mmc

to config.txt in your first primary FAT partition?

start.elf doesn't care about DT at all, those are interpreted by the Linux kernel

[mimi123]

The Big and Nasty Blob(also known as start.elf) tries to use SDIO instead of MMC to read the kernel from SD

Is there a chance in frozen hell this could ever be corrected somehow, so start.elf can actually use MMC to read it? Right now I just feel remarkably stupid that I bought these MMC cards ($7.50 ea) from a local shop, hoping to be able to give these obsolete relics some purpose in life, only to find they won't work. Plus, my application needs the SD card to be pre-installed and not end up broken-off during shipping back and forth... I never made mention of the SD card in my instruction video since they should never even need to know it exists

Ah well... maybe it'll get fixed some day, somehow, but I guess I'm just carrying on with other cards for now.

Wonder if I have a chance to do it, there is a non-stripped firmware build after all(cracked MPEG-2 code verification using it). I think it will be plain old VCOS or FileX(didn't verify if that was used) hooking