hippy
Posts: 9703
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Loading .uf2 via 'picotool' versus MSD

Tue Apr 13, 2021 2:34 pm

I have a .uf2 which appears to be loaded differently when loaded via 'picotool' and the MSD.

Basically I am adding data to an existing .uf2 (MicroPython), contiguous with the executable at 0x10046300, at a later 4K boundary of 0x10080000, with another block at 0x10090000 which points to the start of the contiguous block at 0x10046300.

I have some MicroPython code which reports what's in those loaded areas -

Code: Select all

from machine import mem8

def Dump(adr, size):
  n = (mem8[adr+7]<<24) +(mem8[adr+6]<<16) + (mem8[adr+5]<<8) + mem8[adr+4]
  print("{} : {}".format(hex(adr+4), hex(n)))
  print("{} ...".format(hex(adr)))
  s = ""
  for pc in range(adr, adr + size):
    n = mem8[pc]
    if n < 0x20 or n > 0x7E : s = s + "<"+hex(n)[2:]+">"
    else                    : s = s + chr(n)
  print(s)

#      12345678
Dump(0x10046300, 32)
Dump(0x10080000, 32)
Dump(0x10090000, 32)
When I load that .uf2 with 'picotool load -x' it all works, shows "test.py" and other expected data -

Code: Select all

0x10046304 : 0x1b
0x10046300 ...
<ff><ff><ff><ff><1b><0><0><0>test.py<0>print('in test.p
0x10080004 : 0x1b
0x10080000 ...
<ff><ff><ff><ff><1b><0><0><0>test.py<0>print('in test.p
0x10090004 : 0x10046300
0x10090000 ...
<0>c<4><10><0>c<4><10><0>c<4><10><0>c<4><10><0>c<4><10><0>c<4><10><0>c<4><10><0>c<4><10>
But, when loaded via BOOTSEL MSD it doesn't work fully, "test.py' changed to "yabba.py" to force a change ...

Code: Select all

0x10046304 : 0x1c
0x10046300 ...
<ff><ff><ff><ff><1c><0><0><0>yabba.py<0>print('in test.
0x10080004 : 0x18
0x10080000 ...
<ff><ff><ff><ff><18><0><0><0>pab`  p<0><0>p`hd  !h  dap$ 
0x10090004 : 0x10046300
0x10090000 ...
<0>c<4><10><0>c<4><10><0>c<4><10><0>c<4><10><0>c<4><10><0>c<4><10><0>c<4><10><0>c<4><10>
That 0x10080000 block is corrupted. Examination of the .uf2 file shows it is correct within that. And, without changing anything, exact same .uf2, when reloaded using 'picotool -load x' it works, 0x10080000 is as it should be ...

Code: Select all

0x10046304 : 0x1c
0x10046300 ...
<ff><ff><ff><ff><1c><0><0><0>yabba.py<0>print('in test.
0x10080004 : 0x1c
0x10080000 ...
<ff><ff><ff><ff><1c><0><0><0>yabba.py<0>print('in test.
0x10090004 : 0x10046300
0x10090000 ...
<0>c<4><10><0>c<4><10><0>c<4><10><0>c<4><10><0>c<4><10><0>c<4><10><0>c<4><10><0>c<4><10>
Does anyone have an explanation for this behaviour ?

hippy
Posts: 9703
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Loading .uf2 via 'picotool' versus MSD

Tue Apr 13, 2021 2:53 pm

Using a humanly readable timestamp for a filename - After upload via MSD, only 0x10046300 is correct ...

Code: Select all

0x10046304 : 0x27
0x10046300 ...
<ff><ff><ff><ff>'<0><0><0>2021-04-13 15:48:10<0>prin
0x10080004 : 0x4
0x10080000 ...
<ff><ff><ff><ff><4><0><0><0>0 " !<0> !<0> <0>  0  (  <0> 2 $
0x10090004 : 0x10046300
0x10090000 ...
<0>c<4><10><0>c<4><10><0> <0><10><0><0><4><0><0> <0><10><0>"<0><10><0>!<0><0><0><0><0><0>
After 'picotool load -x', everything is correct ...

Code: Select all

0x10046304 : 0x27
0x10046300 ...
<ff><ff><ff><ff>'<0><0><0>2021-04-13 15:48:10<0>prin
0x10080004 : 0x27
0x10080000 ...
<ff><ff><ff><ff>'<0><0><0>2021-04-13 15:48:10<0>prin
0x10090004 : 0x10046300
0x10090000 ...
<0>c<4><10><0>c<4><10>2021-04-13 15:48:10<0><0><0><0><0>
If it helps visualise what is shown above, my .uf2 is much like ...

Code: Select all

           .------------------.
0x10000000 | xxxxxxxxxxxxxxxx |   x = Original .uf2
           | xxxxxxxxxxxxxxxx |   _ = FF FF FF FF
0x10046200 | xxxxxxxxxxxxx000 |   a = len(t+A)+1
0x10046300 | _at.AAAAAa_00000 |   t = timestamp
           |------------------|   . = chr(0)
0x10080000 | _at.AAAAAa_00000 |   A = text
           |------------------|   p = Ptr to 0x10064300
0x10090000 | ppt0000000000000 |   0 = Padding chr(0)
           `------------------'

kilograham
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 493
Joined: Fri Apr 12, 2019 11:00 am
Location: austin tx

Re: Loading .uf2 via 'picotool' versus MSD

Tue Apr 13, 2021 6:43 pm

it doesn't look like picotool validates block_num field, whereas the bootrom does; so that is a possible difference (both in range, and also duplicates)

hippy
Posts: 9703
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Loading .uf2 via 'picotool' versus MSD

Wed Apr 14, 2021 1:04 am

I don't think it is block number because I update the block numbers in the entire file, in this case to be three more than they were. It's not just a brute-force append. My UF2 tools verify head, tail and family is correct, sequence numbers increment by one, all block numbers match and are the number of blocks used, that every block payload is 256 bytes long, that there's no change in flags.

Also it is; it loads, Boot Rom closes MSD, executes the flashed image. The Boot Rom code suggests to me that, if an upload fails, the file doesn't conform, it leaves the MSD open, resets, and waits for another .uf2 to be copied in. The close and execute suggest it found the file acceptable.

What is rather disconcerting is that Flash areas are corrupted. If they stayed the same I would presume they hadn't been written to, if all 0xFF would assume erased. But this looks like wrong data was written to those areas. Luckily this is just data, not executable code.

kilograham
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 493
Joined: Fri Apr 12, 2019 11:00 am
Location: austin tx

Re: Loading .uf2 via 'picotool' versus MSD

Wed Apr 14, 2021 7:30 pm

ok, well open an issue in pico-bootrom on github and attach a repro UF2

hippy
Posts: 9703
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Loading .uf2 via 'picotool' versus MSD

Wed Apr 14, 2021 8:24 pm

kilograham wrote:
Wed Apr 14, 2021 7:30 pm
ok, well open an issue in pico-bootrom on github
Can't do that I am afraid.
kilograham wrote:
Wed Apr 14, 2021 7:30 pm
and attach a repro UF2
I've attached that here ...
x.zip
(214.22 KiB) Downloaded 5 times

kilograham
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 493
Joined: Fri Apr 12, 2019 11:00 am
Location: austin tx

Re: Loading .uf2 via 'picotool' versus MSD

Wed Apr 14, 2021 8:35 pm

hippy wrote:
Wed Apr 14, 2021 8:24 pm
kilograham wrote:
Wed Apr 14, 2021 7:30 pm
ok, well open an issue in pico-bootrom on github
Can't do that I am afraid.
Don't be afraid.

hippy
Posts: 9703
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Loading .uf2 via 'picotool' versus MSD

Wed Apr 14, 2021 8:56 pm

kilograham wrote:
Wed Apr 14, 2021 8:35 pm
Don't be afraid.
:P

But just to be clear, it's not possible for me to do that.

One option I thought of for diagnosing my issues was to build the Boot Rom loader as a user-program, run that with debugging enabled. But I fell at the first hurdle, compiling the 'git clone' as is. I get close but no cigar ...

Code: Select all

[ 39%] Linking C executable bootrom.elf
/usr/lib/gcc/arm-none-eabi/7.3.1/../../../arm-none-eabi/bin/ld: bootrom.elf section `.text' will not fit in region `ROM'
/usr/lib/gcc/arm-none-eabi/7.3.1/../../../arm-none-eabi/bin/ld: region `ROM' overflowed by 16384 bytes
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/bootrom.dir/build.make:285: bootrom.elf] Error 1
make[1]: *** [CMakeFiles/Makefile2:923: CMakeFiles/bootrom.dir/all] Error 2
make: *** [Makefile:84: all] Error 2
Do you know if what I need to change to make it work as a user program is documented anywhere ?

kilograham
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 493
Joined: Fri Apr 12, 2019 11:00 am
Location: austin tx

Re: Loading .uf2 via 'picotool' versus MSD

Thu Apr 15, 2021 12:56 am

yeah, you need to use gcc 9.3.1 to build the B1 bootrom (the code doesn't fit with 10.x as you see)

There is no public code for running the bootrom as a program, but it is possible if you are feeling keen. I will however take a look at your UF2 as it is concerning!

hippy
Posts: 9703
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Loading .uf2 via 'picotool' versus MSD

Thu Apr 15, 2021 10:27 am

kilograham wrote:
Thu Apr 15, 2021 12:56 am
yeah, you need to use gcc 9.3.1 to build the B1 bootrom (the code doesn't fit with 10.x as you see)
I don't believe I am using GCC 10.x

The GCC which I have is 8.3.0 (Raspbian 8.3.0-6+rpi1), 'cmake' reports the arm-none-eabi compiler for building Pico executables as 7.3.1.
kilograham wrote:
Thu Apr 15, 2021 12:56 am
There is no public code for running the bootrom as a program, but it is possible if you are feeling keen.
I am not keen, but would like to have it working, or at least know why it's not working, so am willing to expend some effort. It's more finding out what I should be doing to make it work which is the current stumbling block.
kilograham wrote:
Thu Apr 15, 2021 12:56 am
I will however take a look at your UF2 as it is concerning!
Thanks. My suspicion falls on 'virtual_disk.c', in 'vd_write_block' which calls '_update_current_uf2_info' then '_write_uf2_page'. The task scheduling of actual writes seems reasonable enough on paper but I guess the issue could be there.

The 'vd_write_block' checks are simple enough, checking the basic characteristic of a block, which I am sure I am meeting. '_update_current_uf2_info' I haven't got to grips with yet, am not sure what's it's doing or why it's doing what it does. It seemed easier to let it run and see what it does rather than guess what it would be doing, should be doing on paper. It seems to me it should work but perhaps doesn't, the standard feature of a bug if there is one.

I don't want to spend a lot of time just figuring out that one part does work how it's meant to, but appreciate the nature of debugging, rule out what does work and the fault lies in something else.

I did think about pulling those routines out and wrapping those as a Pi user program I but am not sure what else I need, haven't achieved that yet. It means having to figure out how to do things I have never done in C before so isn't as easy as it could be for me.

The ideal for me would be being able to see a list of 'Wrote this Flash page'. That would let me see if my file is being passed through or not. whether the bug is above that, perhaps a hardware related issue, or below that, my file or a filtering issue. Just knowing what it was writing to 0x10080000 would be a good start; whether that is what my file says it should be or not.

kilograham
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 493
Joined: Fri Apr 12, 2019 11:00 am
Location: austin tx

Re: Loading .uf2 via 'picotool' versus MSD

Thu Apr 15, 2021 12:16 pm

on the subject of GCC versions; you pretty much need to use whatever compiler is listed in the README.md as space is so tight (I no that gcc 10 makes a binary that is too big which is why I guess that)

yeah, as you said above if the Pico reboots, then it should have passed all the checks.

cleverca22
Posts: 3587
Joined: Sat Aug 18, 2012 2:33 pm

Re: Loading .uf2 via 'picotool' versus MSD

Thu Apr 15, 2021 12:26 pm

kilograham wrote:
Thu Apr 15, 2021 12:16 pm
yeah, as you said above if the Pico reboots, then it should have passed all the checks.
isnt the reboot triggered by the currrent block# matching the total-block#?

i can see how it might malfunction if extra blocks are appended, and it reboots before getting the newly added part
hippy wrote:
Wed Apr 14, 2021 1:04 am
My UF2 tools verify head, tail and family is correct, sequence numbers increment by one, all block numbers match and are the number of blocks used, that every block payload is 256 bytes long, that there's no change in flags.
but that shouldnt be the case for this file

hippy
Posts: 9703
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Loading .uf2 via 'picotool' versus MSD

Thu Apr 15, 2021 2:05 pm

kilograham wrote:
Thu Apr 15, 2021 12:16 pm
on the subject of GCC versions; you pretty much need to use whatever compiler is listed in the README.md as space is so tight (I no that gcc 10 makes a binary that is too big which is why I guess that)
The README.md only says "It was built in debug mode using an gcc-arm-none-eabi version 9.2.1", a statement of fact rather than instruction, as enigmatic as the original wording as to which 'cmake' to install on Windows.

I know, I know; report it on github.
cleverca22 wrote:
Thu Apr 15, 2021 12:26 pm
isnt the reboot triggered by the currrent block# matching the total-block#?
Almost. It seems the reboot happens when it has written as many valid blocks as there were expected to be, which effectively amounts to the same thing, but perhaps not in all cases -

https://github.com/raspberrypi/pico-boo ... isk.c#L163

Not understanding the thinking and reasoning behind what is being done in the code is one of the difficulties in trying to figure out what may be going on or going wrong. It seems the code is building 4K Flash Pages from 256 byte UF2 Blocks, triggering an asynchronous task to write those, a callback rebooting when it considers the last necessary UF2 Block was written.

I'm wondering if there isn't some mismatch between Flash Pages being written and number of UF2 Blocks written, or some race condition, causing a premature reboot before a last block is written.

Or it could be later UF2 Blocks are being included in earlier Flash Pages, are being written to the wrong place. But 'something' does seem to be written to the correct place which corrupts what was there.

In reading the comments at lines 172 to 189 I'm also wondering if there isn't some edge case which has been overlooked, which it doesn't cater for, which I am inadvertently falling into. I'm wondering if there isn't some "stepping on toes" going on, if it is as robust as it was thought to be ?

I don't know and it's all rather difficult to diagnose when one has to create a UF2 file, drag and drop flash it using File Manager, see what the end result appears to be. It's the age old 'I post a letter; it never arrives' black box problem. We know what's meant to happen, but discovering why that doesn't can be a challenge, have numerous causes. Walking the exact path is usually the best way of seeing where things diverge from expectation.

It's perhaps worth noting that I am not looking for a code fix, if that is what is required. It's burned into ROM so can't be fixed there, and even if fixed in later versions, 'millions of Pico' will remain unfixed. I'm looking to know what to avoid, how to work round that in my UF2 generation if that is required, desiring to know why I must do that if I have to.

kilograham
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 493
Joined: Fri Apr 12, 2019 11:00 am
Location: austin tx

Re: Loading .uf2 via 'picotool' versus MSD

Thu Apr 15, 2021 2:43 pm

I tried it on both a B0, and B1 board, and in each case the only difference in what is written with picotool vs MSD is that picotool fills unused parts of erased 4K pages with zeros whereas bootrom does not.

I suggest using pictool save -a foo.bin to save out what is in the flash for comparison. Also use the flash_nuke.uf2 to clear the flash between runs (to see if that helps).

I am using Ubuntu 20. Given that the reason for the UF2 file format's existence is to deal with the unpredictable ways/orders in which an OS might write file blocks to the media, it would certainly be worth noting what OS/method you are using to copy to the flash, it is possible there is an issue with this. I'd still be slightly surprised if there is a bootrom bug here as this code has been heavily stress tested with out of order and duplicate block writes.
.

cleverca22
Posts: 3587
Joined: Sat Aug 18, 2012 2:33 pm

Re: Loading .uf2 via 'picotool' versus MSD

Thu Apr 15, 2021 4:25 pm

kilograham wrote:
Thu Apr 15, 2021 2:43 pm
I am using Ubuntu 20. Given that the reason for the UF2 file format's existence is to deal with the unpredictable ways/orders in which an OS might write file blocks to the media, it would certainly be worth noting what OS/method you are using to copy to the flash, it is possible there is an issue with this. I'd still be slightly surprised if there is a bootrom bug here as this code has been heavily stress tested with out of order and duplicate block writes.
as an extra data-point, also try `cat foo.uf2 > /dev/sdX`, while not mounted, make sure you get the right device

UF2 doesnt care if your using an fs or not, just that the 512 byte uf2 packets get written as 512 byte sectors

hippy
Posts: 9703
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Loading .uf2 via 'picotool' versus MSD

Thu Apr 15, 2021 7:01 pm

Thanks for the suggestions. I am updating my UF2 hacking program so I can more easily create stuff which is easier to debug, and will also allow otters to come along for the ride, no need to share half a meg UF2's around.

So there should be some respite while I bash that into shape, assess what I am seeing :)

Return to “General”