LizardLad_1
Posts: 126
Joined: Sat Jan 13, 2018 12:29 am

Bare metal hang when reading from SD Card?

Sun Jul 01, 2018 5:29 am

Hello, so my code is at https://github.com/OllieLollie1/Raspi3-Kernel. When I reach line 25 of fat.c the kernel doesn't progress this line is dereferencing a pointer this pointer cannot be NULL because an addition occurs I have no idea what could be going wrong. Other pointer dereferences occur and they don't hang. One of the possibilities are that my power supply isn't powerful enough and the whole pi hangs however I am not sure why it occurs only there. When I don't try and read from the SD card everything works as expected. My code is mostly based of btz's code according to them they got it working on a real raspberry pi 3. My code works in QEMU however not on real hardware. My power supply is a Samsung EP-TA20HWE with an output of 5.0V at 2.0A.

Schnoogle
Posts: 43
Joined: Sun Feb 11, 2018 4:47 pm

Re: Bare metal hang when reading from SD Card?

Sun Jul 01, 2018 7:00 am

Hi there,

if I‘m Not mistaken the line 25 is accessing the content of the address master_boot_record+0x1C6 ...
This is actually not a 4 Byte aligned address. As far as I know the RPi(ARM) is initially only accepting proper aligned memory access. You could allow un-aligned access during your boot up. I’m not able to grab some code for you that does this, but I guess this is easy to find here in the forum or on the ARM doc’s.

BR Schnoogle

bzt
Posts: 178
Joined: Sat Oct 14, 2017 9:57 pm

Re: Bare metal hang when reading from SD Card?

Sun Jul 01, 2018 9:45 am

Hi,

@Schnoogle: Gcc usually does a deceant job generating code for reading non-aligned addresses, but not all the time.

@LizardLad_1: If the problem occurs every time exactly at the same location then I doubt it's a power issue.

I can confirm that my tutorial works on real RPi and in the latest qemu. Although I've found gcc generating misaligned memory access here. What I came up with: I've read more than a word from a word aligned address, shifted and masked to get only what I needed.

I'd suggest to try my pre-compiled kernel8.img, and one compiled with your toolchain. If the former works but not the latter, then it's definitely a gcc bug.

bzt

(P.S.: bztsrc is not them, it's only me and my sources :-) Although I had contributions for the tutorials, the fat.c code was entirely written by me)

LizardLad_1
Posts: 126
Joined: Sat Jan 13, 2018 12:29 am

Re: Bare metal hang when reading from SD Card?

Mon Jul 02, 2018 2:52 am

Hi,

I don't have access to a UART so I can't actually test on the pi. If you have spare time would you please compile my code and upload it to see if it is just my compiler?
(P.S.: bztsrc is not them, it's only me and my sources :-) Although I had contributions for the tutorials, the fat.c code was entirely written by me)
Sorry I was just trying to not use a gendered pronoun.

bzt
Posts: 178
Joined: Sat Oct 14, 2017 9:57 pm

Re: Bare metal hang when reading from SD Card?

Mon Jul 02, 2018 9:13 am

LizardLad_1 wrote:
Mon Jul 02, 2018 2:52 am
(P.S.: bztsrc is not them, it's only me and my sources :-) Although I had contributions for the tutorials, the fat.c code was entirely written by me)
Sorry I was just trying to not use a gendered pronoun.
Well, it's not gendered, it's plural :-) I'm a single person. Btw, you should not be afraid to use he/she despite of the madness going on lately, the English language has them for a reason. Remember, until the end of time it's gonna need two people of different gender to make a child, no law can change that and no man nor woman should be ashamed for that. And it's okay, I appreciate you tried to be polite.

Back on topic, do this:
1. put two print string calls before the line in question (A and B). That way you can test your print routine should print both and there's no problem with the printing code (some kind of overflow for example which causes the nth print to freeze).
2. move print B after the line in question. If B is not printed, this will proove that stop is caused by that line indeed.
3. try to use a delay before you print A. If you can't see the text A on screen either, then your problem is not related to the actual code (and could be a power issue, although unlikely). If your problem "moves" with the modification (now the problematic instruction is at different address and called later during execution), then try adding a simple exception handler that prints ESR and ELR system registers.
4. check if your mbr is aligned in memory (use readelf on kernel8.elf). If not, use __attribute__ on your buffer. I'm using the "_end" label to load the mbr after the kernel8 in memory, which is known to be aligned. Your master_boot_record_array may not be. Also check if your unortodox reference really works. Should be "master_boot_record=(unsigned char*)&master_boot_record_array;" and not "&(master_boot_record_array[0])", although that should return the address of the first byte in the array, which is the same (in theory at least).
5. if the problem still exists, read an uint64_t from an aligned address (mbr+0x1C0), shift it down by 48 bits, and mask it by "anding" with 0xFFFF, then reading uint64_t from +0x1C8 masking by 0xFFFF, shift it up by 16 and "orring". (So read the first least significant word in one memory read, then read the second, most signifact one in another)

bzt

User avatar
Paeryn
Posts: 2170
Joined: Wed Nov 23, 2011 1:10 am
Location: Sheffield, England

Re: Bare metal hang when reading from SD Card?

Mon Jul 02, 2018 4:38 pm

bzt wrote:
Mon Jul 02, 2018 9:13 am
LizardLad_1 wrote:
Mon Jul 02, 2018 2:52 am
(P.S.: bztsrc is not them, it's only me and my sources :-) Although I had contributions for the tutorials, the fat.c code was entirely written by me)
Sorry I was just trying to not use a gendered pronoun.
Well, it's not gendered, it's plural :-) I'm a single person.
They and them are the singular non-gender forms of he/she and him/her when one person of unknown gender is being referred to, has been for hundreds of years.
She who travels light — forgot something.

LizardLad_1
Posts: 126
Joined: Sat Jan 13, 2018 12:29 am

Re: Bare metal hang when reading from SD Card?

Tue Jul 03, 2018 12:27 am

Hi,

I'm not sure whether this is is aligned or not the address of the master_boot_record_array is
61: 0000000000084410 512 OBJECT GLOBAL DEFAULT 4 master_boot_record_array

And the _end symbol has functions after it so if you are loading the mbr into the _end symbol it is overwriting some code. This follows _end

Code: Select all

   130: 0000000000084720     0 NOTYPE  GLOBAL DEFAULT    4 _end
   131: 0000000000084648     4 OBJECT  GLOBAL DEFAULT    4 pitch
   132: 0000000000084628     8 OBJECT  GLOBAL DEFAULT    4 EndOfProgram
   133: 0000000000084718     8 OBJECT  GLOBAL DEFAULT    4 sd_hv
   134: 00000000000800a4     0 NOTYPE  GLOBAL DEFAULT    1 _hang
   135: 0000000000081ae8    72 FUNC    GLOBAL DEFAULT    1 gpio_output
   136: 0000000000082968   240 FUNC    GLOBAL DEFAULT    1 sd_int
   137: 0000000000081788    20 FUNC    GLOBAL DEFAULT    1 clocks_init
   138: 0000000000081e58   412 FUNC    GLOBAL DEFAULT    1 fat_getcluster
   139: 0000000000081558    52 FUNC    GLOBAL DEFAULT    1 strlen
   140: 0000000000081708   124 FUNC    GLOBAL DEFAULT    1 get_cpu_max_clock
   141: 0000000000084630     8 OBJECT  GLOBAL DEFAULT    4 TotalSpaceAvaliable
   142: 0000000000084638     8 OBJECT  GLOBAL DEFAULT    4 StartOfProgram
   143: 00000000000800d0   144 FUNC    GLOBAL DEFAULT    1 dynamic_memory_alloc_init
   144: 000000000008464c     4 OBJECT  GLOBAL DEFAULT    4 height
   145: 0000000000084650     4 OBJECT  GLOBAL DEFAULT    4 width
I havn't yet implemented the code to print out the exception registers however I believe misalignment to be the issue. I tried to add __atribute__((aligned(4))) however that didn't change anything.

bzt
Posts: 178
Joined: Sat Oct 14, 2017 9:57 pm

Re: Bare metal hang when reading from SD Card?

Wed Jul 04, 2018 4:44 am

@Paeryn: I'm not a native speaker sorry, it seems I have totally forgotten about singular they :oops: Wikipedia on English grammar also says "The plural pronoun they (and its derived forms them, their, etc.) can also be used to refer to one person".

@LizardLad_1: 0x84410+0x1C6 = 0x845D6. What's the remainder if you divide that by 8? Not zero, so it's not aligned. Shouldn't be a problem though, gcc should come around automatically. I can only suggest the same read aligned/shift/mask trick I suggested before.

"And the _end symbol has functions after it so if you are loading the mbr into the _end symbol it is overwriting some code."
That's just plain wrong. Your linker script should define the _end label *AFTER* everything. This is a clear indicator that something is definitely not right with your build environment. Why do you have extra text sections? First, figure that out and then you'll have a chance to solve the alignment issue.

EDIT: Btw, taking a closer look on your output, are you 100% sure _end label is not your last label? :-)

LizardLad_1
Posts: 126
Joined: Sat Jan 13, 2018 12:29 am

Re: Bare metal hang when reading from SD Card?

Wed Jul 04, 2018 10:13 am

bzt wrote: EDIT: Btw, taking a closer look on your output, are you 100% sure _end label is not your last label? :-)
Wow didn't even realize. I will try the read aligned shift mask now.

I also think you mean mod division by eight.

EDIT: When you say shift down does that mean left or right?

LizardLad_1
Posts: 126
Joined: Sat Jan 13, 2018 12:29 am

Re: Bare metal hang when reading from SD Card?

Wed Jul 04, 2018 11:42 pm

Code: Select all

temp_read <<= 48;
temp_read &= 0xFFFF;
uint64_t temp_read_second_part = *((uint64_t *)master_boot_record+0x1C8);
temp_read_second_part &= 0xFFFF;
temp_read_second_part >>= 16;
partitionlba = (temp_read|temp_read_second_part);
I believe this is what you meant when you said
bzt wrote: 5. if the problem still exists, read an uint64_t from an aligned address (mbr+0x1C0), shift it down by 48 bits, and mask it by "anding" with 0xFFFF, then reading uint64_t from +0x1C8 masking by 0xFFFF, shift it up by 16 and "orring". (So read the first least significant word in one memory read, then read the second, most signifact one in another)
However I am now getting the error that it is the wrong filesystem type?

bzt
Posts: 178
Joined: Sat Oct 14, 2017 9:57 pm

Re: Bare metal hang when reading from SD Card?

Thu Jul 05, 2018 12:45 pm

Almost there.

Code: Select all

// Assuming we have 0x12345678 at 0x1C6.
// ---read the lower word---
// because the word we want is at 0x1C6, remove 6 bytes, keep only the upper 2
l = (*(uint64_t*)(mbr+0x1C0)) >> 48;
// l is now 0x5678
// ---read upper word---
// the word is at 0x1C8, so no need to shift, but we have more than 2 bytes
// so cut off the upper 6 bytes
h = (*(uint64_t*)(mbr+0x1C8) & 0xFFFF;
// h is now 0x1234
// ---combine---
// this means the lower word is ok, but upper word has to be shifted into position, otherwise
// we would get invalid value, 0x1234 | 0x5678 is not what we want.
partitionlba = (h<<16) | l;
// partitionlba is now 0x12340000 | 0x5678 = 0x12345678
About wrong filesystem type: there could be several reasons. The code checks the magic bytes "FAT16" or "FAT32" to tell whether the VBR is valid. Either you have not read the correct sector into memory, or your fat creation tool did not placed the magic bytes (unlikely). But since your partitionlba is invalid, chances are good that you have simply read the wrong sector. To verify this, print out partitionlba, and you should use

Code: Select all

fdisk yourimage.dd
p
this will show the starting sector number for your first partition. That should match the value in the variable partitionlba. If not, your code is incorrect.
About the second case (missing magic bytes): the Linux tool mkfs.fat generates those magic bytes, and also the official rasbian image has them. To check, you should open your image file in a hex editor, and jump to the offset startingsector*512. For example using fdisk on the raspbian-stretch-lite image reveals 8192 as starting sector, therefore I have to use 8192*512=0x400000 as offset. There I can see a valid FAT VBR record, with the magic bytes "FAT32" at 0x400052. (The bytes "mkfs.fat" at 0x400002 also tells me that the rasbian image was created on Linux with the mkfs.fat tool.)

Just an interesting addition, I could have used those magic bytes to tell FAT16 and FAT32 apart, but I didn't. Instead I've used a safer solution, checked the 16 bit sector per cluster field. If that's zero, then it must be a FAT32 file system.

bzt

LizardLad_1
Posts: 126
Joined: Sat Jan 13, 2018 12:29 am

Re: Bare metal hang when reading from SD Card?

Fri Jul 06, 2018 12:56 am

Hi bzt,

I tried your code however it still fails I ran it in QEMU to test the output of it compared to the output of

Code: Select all

*((unsigned int *)((unsigned long)master_boot_record + 0x1C6))
and the values were completely different if you wanted to see the code it is in my GitHub repo the link to which is at the top of this thread. the value returned by your code was 00000000 and the value returned by the above line is 00002000. I'm honestly not sure what has gone wrong.

laroche
Posts: 8
Joined: Fri Jul 06, 2018 7:30 am

Re: Bare metal hang when reading from SD Card?

Fri Jul 06, 2018 8:36 am

Hello bzt,

I tried to check if my gcc 7.3.0 from Ubuntu 18.04LTS also miscomppiles your fat.c source code.

If I add the following lines into fat.c:

Code: Select all

unsigned int test0(bpb_t *bpb)
{
    return ((*((unsigned int *)&bpb->nf) >> 8) & 0xFFFF) * sizeof(fatdir_t);
}
unsigned int test1(bpb_t *bpb)
{
    return bpb->nr * sizeof(fatdir_t);
}
This compiles to the following same output as seen with aarch64-linux-gnu-objdump -d fat.o:

Code: Select all

0000000000000000 <test0>:
   0:   78411000        ldurh   w0, [x0, #17]
   4:   531b6800        lsl     w0, w0, #5
   8:   d65f03c0        ret
   c:   d503201f        nop

0000000000000010 <test1>:
  10:   78411000        ldurh   w0, [x0, #17]
  14:   531b6800        lsl     w0, w0, #5
  18:   d65f03c0        ret
  1c:   d503201f        nop
Seems to generate the same code. What compiler version has problems on your side?

best regards,

Florian La Roche

bzt
Posts: 178
Joined: Sat Oct 14, 2017 9:57 pm

Re: Bare metal hang when reading from SD Card?

Fri Jul 06, 2018 3:50 pm

@LizzardLad_1: you have the shifts in the wrong way. Otherwise I don't get it. If you can't read directly, how come you can read it when you compare at line 29? I'd like to point out that 0x2000 (8192) is the correct value, so if you can compare that and print that in line 36 then you definitely can read that address without generating an unaligned exception. So what's going on? Also, @laroche gave the idea, what version of gcc are you using? Try the latest one!

@laroche: hey thanks, good to know! Gcc is full of bugs, but I like that it's Open Source and people are fixing it constantly! I can't remember the exact version, it was in the first half of January, so must have been 7.2.x. I'll check if they have also fixed the other bug I run into regarding referencing multiple struct fields in arrays in one expression, like in "a[idx].field = b[idx2].field2". Thanks!

bzt
Posts: 178
Joined: Sat Oct 14, 2017 9:57 pm

Re: Bare metal hang when reading from SD Card?

Fri Jul 06, 2018 9:49 pm

@laroche: Okay, I have compiled the latest gcc for aarch64-elf target and tested my tutorials a lot. Yeah, gcc 7.3.0 definitely changed. But unfortunately not in a good direction. The alignment problem is not fixed at all, rather my workaround is now optimized away and gcc uses the same faulty code for it that I wanted to avoid...

But every cloud has a silver lining, I've came up with a simple workaround that also fixes @LizardLad_1's problem too:
https://github.com/bztsrc/raspi3-tutori ... /fat.c#L94

Really dirty hack, I've used a byte array, but at least it's not optimized away, and byte access never triggers unaligned exception.

Cheers,
bzt

LizardLad_1
Posts: 126
Joined: Sat Jan 13, 2018 12:29 am

Re: Bare metal hang when reading from SD Card?

Fri Jul 06, 2018 10:42 pm

I was running it in QEMU as to not generate an exception. This was deliberate because I wanted to compare the outputs. I am using gcc 7.2.1 because I am running Fedora 26 I will need to upgrade to 28 to get the newest version of gcc I was looking to do that some time this week.

LizardLad_1
Posts: 126
Joined: Sat Jan 13, 2018 12:29 am

Re: Bare metal hang when reading from SD Card?

Fri Jul 06, 2018 11:27 pm

Thank you sooo much bzt I was able to get it working with output to the framebuffer.

laroche
Posts: 8
Joined: Fri Jul 06, 2018 7:30 am

Re: Bare metal hang when reading from SD Card?

Sun Jul 08, 2018 7:12 am

Hello bzt,

I have added a pull request to https://github.com/bztsrc/raspi3-tutorial. I think you have forgotten one more place in fat.c that can be
miscompiled by gcc.

Due to the miscompiles, I now use "clang -target aarch64-unknown-eabihf -march=armv8-a+nosimd+nofp".
Adding a utils.c with memcmp() and a few minor other changes was enough to get this up and running. Seems like clang produces
often much smaller code on arm64.

best regards,

Florian La Roche

bzt
Posts: 178
Joined: Sat Oct 14, 2017 9:57 pm

Re: Bare metal hang when reading from SD Card?

Sun Jul 08, 2018 10:37 am

@laroche: Thanks a lot, I forgot that one! Fixed!

I haven't merged your pull request btw, because I don't agree with the rest of the modifications.
1. Reading HCR_EL2 is needed. Strange things happen if you don't read back the value you set. Don't ask me why, it's just one of those things I've empirically figured out and which are seemingly unnecessary and so common when interfacing with low-level hardware. I suppose there's a shadow latch or something?
2. CurrentEL is a 32-bit register, so "unsigned int el" is fine.

About clang, yeah, I'm also playing with the though of switching. I had concerns about it's maturity, but since FreeBSD has switched to it successfully, it must be good enough.

Thank you for your contribution! I appreciate it!
bzt

LizardLad_1
Posts: 126
Joined: Sat Jan 13, 2018 12:29 am

Re: Bare metal hang when reading from SD Card?

Sun Jul 08, 2018 10:47 am

It appears that _end isn't always aligned. In my latest compile it was at an address that is 4 byte aligned but not 8 byte aligned. Can the pi access 4 byte aligned addresses?

laroche
Posts: 8
Joined: Fri Jul 06, 2018 7:30 am

Re: Bare metal hang when reading from SD Card?

Sun Jul 08, 2018 12:34 pm

Hello Lizard*,

I use the following linker change to make sure bss end is aligned to 8 bytes and _end is
aligned to 4096 bytes:

Code: Select all

--- a/0F_executionlevel/link.ld
+++ b/0F_executionlevel/link.ld
@@ -35,8 +35,10 @@ SECTIONS
         __bss_start = .;
         *(.bss .bss.*)
         *(COMMON)
+        . = ALIGN(8);
         __bss_end = .;
     }
+    . = ALIGN(4096);
     _end = .;
 
    /DISCARD/ : { *(.comment) *(.gnu*) *(.note*) *(.eh_frame*) }
best regards,

Florian La Roche

laroche
Posts: 8
Joined: Fri Jul 06, 2018 7:30 am

Re: Bare metal hang when reading from SD Card?

Sun Jul 08, 2018 12:39 pm

Hello bzr,

thanks for merging in changes and overall a big thanks for your great repo.

About: CurrentEL: The compiler issues a read into a 64bit register for "CurrentEL".
Using a "unsigned int" works fine for gcc, but "clang" is more restrictive and won't
compile this line of code. All other calls to mrs/msr work fine, this is the only place
32bit is used in your repository.

I've put out another merge request for mbox.c and mmu.c to reduce code size
even further a little bit.

Thanks a lot,
best regards,

Florian La Roche

laroche
Posts: 8
Joined: Fri Jul 06, 2018 7:30 am

Re: Bare metal hang when reading from SD Card?

Mon Jul 09, 2018 4:25 pm

There is one more location in fat.c with unaligned data. Also, instead of of using an extra
(unsigned char*) pointer into the data, I've changed the code to use consecutive "unsigned char"
entries. The code looks much cleaner with this.

Patch is available at: https://github.com/bztsrc/raspi3-tutori ... 6602938049

best regards,

Florian La Roche

bzt
Posts: 178
Joined: Sat Oct 14, 2017 9:57 pm

Re: Bare metal hang when reading from SD Card?

Wed Jul 11, 2018 3:20 pm

@laroche: thank you for your contribution! I've merged your pull request, only with really minor modifications. I've recompiled the kernel images, everythings seems good!

About CurrentEL and Clang: that's a perfectly valid argument! I've changed the type to "long".

Cheers,
bzt

Return to “Bare metal, Assembly language”