mikeabuilder
Posts: 19
Joined: Thu Feb 11, 2021 12:34 am

Where are my files?

Thu Feb 25, 2021 5:28 pm

2 questions

First question - My micropython program has written to a file on the pico, and it has read it back, so I believe the file really exists. When I connect the pico to my windows computer via USB (button pressed) and look at the pico, I don't see the file. Is there a way to access the file from windows? Maybe putting the file in a particular location in the directory structure from micropython? I can see them from Thonny, but is there a more "normal" way?

Second question - Similarly, if I do the same with C, should my file show up in the same places? In this case I cannot see a file I've written in C when looking at the pico storage using Thonny.

lars_the_bear
Posts: 229
Joined: Thu Jan 28, 2021 8:13 pm
Contact: Website

Re: Where are my files?

Thu Feb 25, 2021 6:23 pm

How have you written a file in C? The C SDK doesn't have working file-handling functions. To write files in C you need to implement a filesystem yourself, either in flash or on some external storage.

As for Python -- I presume that Python also writes files to a purely internal filesystem. I wouldn't expect them to show up on a USB host. But I confess that haven't actually looked.

Kevin.

hippy
Posts: 10530
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Where are my files?

Thu Feb 25, 2021 7:32 pm

mikeabuilder wrote:
Thu Feb 25, 2021 5:28 pm
First question - My micropython program has written to a file on the pico, and it has read it back, so I believe the file really exists.
The file is held in the flash chip in an area MicroPython uses as an internal file system. You can access that and its files within MicroPython, Thonny and command line tools such a 'rshell' and 'ampy'. It's also possible to see those as if in a Linux directory, with File Manager, but that's a work in progress and will likely only be for Linux.

CircuitPython does automatically expose its internal file system as a mountable drive.

For C; it's whatever you implement.

mikeabuilder
Posts: 19
Joined: Thu Feb 11, 2021 12:34 am

Re: Where are my files?

Thu Feb 25, 2021 9:08 pm

Thanks Kevin - no file system explains why there do not seem to be any files. I've been banging through trying to learn C/C++ at the same time as learning the pico itself. If one of my beginner tasks in C is going to be "write a file system", I think I'll be flipping back to micropython.

lars_the_bear
Posts: 229
Joined: Thu Jan 28, 2021 8:13 pm
Contact: Website

Re: Where are my files?

Thu Feb 25, 2021 10:04 pm

mikeabuilder wrote:
Thu Feb 25, 2021 9:08 pm
Thanks Kevin - no file system explains why there do not seem to be any files. I've been banging through trying to learn C/C++ at the same time as learning the pico itself. If one of my beginner tasks in C is going to be "write a file system", I think I'll be flipping back to micropython.
No! Don't do it! ;)

Implementing a complete filesystem is a non-trivial undertaking but, happily, there are several implementations readily available. I'm using LittleFS as, I believe, the MicroPython implementation does. It's easy enough to use -- it's just a matter of supplying LittleFS with the C functions that will read and write the actual blocks of data from flash. All the hard stuff, LittleFS already does. I'm not saying it's easy to implement but, if I can do it, it can't be rocket science.

Kevin

User avatar
nick.mccloud
Posts: 1280
Joined: Sat Feb 04, 2012 4:18 pm

Re: Where are my files?

Thu Feb 25, 2021 11:28 pm

mikeabuilder wrote:
Thu Feb 25, 2021 9:08 pm
Thanks Kevin - no file system explains why there do not seem to be any files. I've been banging through trying to learn C/C++ at the same time as learning the pico itself. If one of my beginner tasks in C is going to be "write a file system", I think I'll be flipping back to micropython.
The Pico isn't a USB Flash drive - it mimics one for ease of programming - if it was meant to show you your files, particularly the C ones, that would be available out the box. You code in C on your computer and then transfer over the .UF2 file when you've compiled. That file isn't your original C text, so there isn't going to be a C file on the Pico for you to get off it at a later date.

So, no point starting out with trying to write a file system to get a copy of a file you won't have on the Pico anyway but do have on your computer.
Pico/RP2040 ≠ Arduino
Pico = hot rod kit car, Arduino = hot rod kit car wrapped in cotton wool with buoyancy aids & parachute

lars_the_bear
Posts: 229
Joined: Thu Jan 28, 2021 8:13 pm
Contact: Website

Re: Where are my files?

Fri Feb 26, 2021 8:44 am

nick.mccloud wrote:
Thu Feb 25, 2021 11:28 pm
So, no point starting out with trying to write a file system to get a copy of a file you won't have on the Pico anyway but do have on your computer.
Ah, yes. It didn't occur to me that the OP might have been talking about source files. I assumed he meant files he'd created (or tried to create) from a C program. As you say, C sources are unlikely to end up on the device anyway. Different programming model entirely.

Kevin.

hippy
Posts: 10530
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Where are my files?

Fri Feb 26, 2021 11:28 am

mikeabuilder wrote:
Thu Feb 25, 2021 5:28 pm
In this case I cannot see a file I've written in C when looking at the pico storage using Thonny.
I also missed the implication of what you were saying - as said; it doesn't work that way, you install the Pico C SDK and compile your C on your Pi or host computer, copy the generated .uf2 file to the Pico. The C source code is never sent to the Pico.

There are some C interpreters and it would be possible to have some sort of C compiling firmware on a Pico. Someone could create a "CircuitC" similar to "CircuitPython" which would expose its file system and allow C files to be dropped on that but it's only going to be a minimal C implementation, won't support a full Pico C SDK.

Using a Pico as a standalone development system, like Home Computers of the past, could be useful in education, for hobbyists, or anyone wanting to run code on a Pico without having to install any SDK. It already exists for Python and such a scheme could be adopted for other languages.

User avatar
nick.mccloud
Posts: 1280
Joined: Sat Feb 04, 2012 4:18 pm

Re: Where are my files?

Fri Feb 26, 2021 11:34 am

hippy wrote:
Fri Feb 26, 2021 11:28 am
Using a Pico as a standalone development system, like Home Computers of the past, could be useful in education, for hobbyists, or anyone wanting to run code on a Pico without having to install any SDK. It already exists for Python and such a scheme could be adopted for other languages.
Not tried it yet, but I think the Fuzix mentioned else where on the forum does this - a mini-OS 'nix clonish.

See viewtopic.php?f=144&t=304223
Pico/RP2040 ≠ Arduino
Pico = hot rod kit car, Arduino = hot rod kit car wrapped in cotton wool with buoyancy aids & parachute

hippy
Posts: 10530
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Where are my files?

Fri Feb 26, 2021 6:00 pm

nick.mccloud wrote:
Fri Feb 26, 2021 11:34 am
Not tried it yet, but I think the Fuzix mentioned else where on the forum does this - a mini-OS 'nix clonish.
Fuzix looks quite amazing but I also haven't tried it yet; don't currently have an SD card adaptor to wire-up.

I was thinking of something less impressive, something self-contained, which could run without requiring an external SD Card.

Crook
Posts: 51
Joined: Mon Jun 04, 2012 5:57 pm

Re: Where are my files?

Fri Feb 26, 2021 8:11 pm

lars_the_bear wrote:
Thu Feb 25, 2021 6:23 pm
How have you written a file in C? The C SDK doesn't have working file-handling functions. To write files in C you need to implement a filesystem yourself, either in flash or on some external storage.

Kevin.
I can create a file and read it back in from the filename. It must be saving this somewhere?

lars_the_bear
Posts: 229
Joined: Thu Jan 28, 2021 8:13 pm
Contact: Website

Re: Where are my files?

Fri Feb 26, 2021 9:14 pm

Crook wrote:
Fri Feb 26, 2021 8:11 pm
I can create a file and read it back in from the filename. It must be saving this somewhere?
I have absolutely no idea how that works. I believe all the standard file-handling functions, like open(), read(), etc., have been replaced by do-nothing stubs. Are you sure you aren't just getting the appearance of saving because the data is just hanging around in memory? Are your file contents still there when after you power-cycle the board?

Kevin.

ejolson
Posts: 8025
Joined: Tue Mar 18, 2014 11:47 am

Re: Where are my files?

Tue Jul 27, 2021 6:07 pm

mikeabuilder wrote:
Thu Feb 25, 2021 9:08 pm
Thanks Kevin - no file system explains why there do not seem to be any files. I've been banging through trying to learn C/C++ at the same time as learning the pico itself. If one of my beginner tasks in C is going to be "write a file system", I think I'll be flipping back to micropython.
There is a C library called LittleFS which implements a robust filesystem for use on microcontrollers. It takes about 15 lines of code to customize it so it works on the Pico. In my opinion LittleFS is such an obvious choice (I think it's used behind the scenes with Python) that it should be included in the SDK.

An example of the minimal customisation needed for LittleFS appears in the source code for Pocfel

viewtopic.php?f=144&t=316321

Another example is at

https://github.com/lurk101/pico-littlefs

I was intimidated at first, but using it turned out easier than expected. If you have any trouble making LittleFS work, just ask.

kilograham
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 865
Joined: Fri Apr 12, 2019 11:00 am
Location: austin tx

Re: Where are my files?

Tue Jul 27, 2021 6:51 pm

I added an issue here
https://github.com/raspberrypi/pico-sdk/issues/531

we probably won't bundle littlefs, but we might make a cmake wrapper library such that if you have littlefs checked out you can add it trivially by adding a single dependency (similar to tinyusb)

hippy
Posts: 10530
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Where are my files?

Wed Jul 28, 2021 11:36 am

ejolson wrote:
Tue Jul 27, 2021 6:07 pm
There is a C library called LittleFS which implements a robust filesystem for use on microcontrollers. It takes about 15 lines of code to customize it so it works on the Pico. In my opinion LittleFS is such an obvious choice (I think it's used behind the scenes with Python) that it should be included in the SDK.
LittleFS is great, is what is used by MicroPython, but it's not so great if one wants a file system which one wishes to expose as MSD where FatFS is a lot more appropriate. That's why CiruitPython uses FatFS rather than LittleFS.

Both have their advantages and disadvantages so it would seem best if both could be supported by the Pico SDK. Something which would allow one or the other to be easily selected but also allow both to be used in some circumstances.

lurk101
Posts: 877
Joined: Mon Jan 27, 2020 2:35 pm
Location: Cumming, GA (US)

Re: Where are my files?

Wed Jul 28, 2021 12:29 pm

hippy wrote:
Wed Jul 28, 2021 11:36 am
Both have their advantages and disadvantages so it would seem best if both could be supported by the Pico SDK. Something which would allow one or the other to be easily selected but also allow both to be used in some circumstances.
I can create, format, and populate a fake FAT filesystem under Linux, then burn it to Pico flash as a binary blob. I can't do that with LFS yet, so that's a big + for FAT. A FUSE version of LFS would do the trick.

ejolson
Posts: 8025
Joined: Tue Mar 18, 2014 11:47 am

Re: Where are my files?

Wed Jul 28, 2021 5:15 pm

lurk101 wrote:
Wed Jul 28, 2021 12:29 pm
hippy wrote:
Wed Jul 28, 2021 11:36 am
Both have their advantages and disadvantages so it would seem best if both could be supported by the Pico SDK. Something which would allow one or the other to be easily selected but also allow both to be used in some circumstances.
I can create, format, and populate a fake FAT filesystem under Linux, then burn it to Pico flash as a binary blob. I can't do that with LFS yet, so that's a big + for FAT. A FUSE version of LFS would do the trick.
Fuzix populates a LittleFS image and then copies it to the Pico.

lurk101
Posts: 877
Joined: Mon Jan 27, 2020 2:35 pm
Location: Cumming, GA (US)

Re: Where are my files?

Thu Jul 29, 2021 3:32 am

ejolson wrote:
Wed Jul 28, 2021 5:15 pm
lurk101 wrote:
Wed Jul 28, 2021 12:29 pm
hippy wrote:
Wed Jul 28, 2021 11:36 am
Both have their advantages and disadvantages so it would seem best if both could be supported by the Pico SDK. Something which would allow one or the other to be easily selected but also allow both to be used in some circumstances.
I can create, format, and populate a fake FAT filesystem under Linux, then burn it to Pico flash as a binary blob. I can't do that with LFS yet, so that's a big + for FAT. A FUSE version of LFS would do the trick.
Fuzix populates a LittleFS image and then copies it to the Pico.
What do you mean by 'populates'? Fuzix runs on the Pico doesn't it? I'm looking for a way to bulk transfer a filesystem with files from a Linux box.

ejolson
Posts: 8025
Joined: Tue Mar 18, 2014 11:47 am

Re: Where are my files?

Thu Jul 29, 2021 4:24 am

lurk101 wrote:
Thu Jul 29, 2021 3:32 am
ejolson wrote:
Wed Jul 28, 2021 5:15 pm
lurk101 wrote:
Wed Jul 28, 2021 12:29 pm

I can create, format, and populate a fake FAT filesystem under Linux, then burn it to Pico flash as a binary blob. I can't do that with LFS yet, so that's a big + for FAT. A FUSE version of LFS would do the trick.
Fuzix populates a LittleFS image and then copies it to the Pico.
What do you mean by 'populates'? Fuzix runs on the Pico doesn't it? I'm looking for a way to bulk transfer a filesystem with files from a Linux box.
It seems I was mistaken. I thought David Given said he had used LittleFS for the built-in flash, but it seems the regular Fuzix filesystem is also used there. At any rate, since Fuzix is not self hosting, the root filesystem has to be built on the development machine and then transferred to flash on the Pico.

If I wanted to do something similar with LittleFS, I would create a LittleFS filesystem in RAM on the development machine with the same block and erase size as the built-in flash on the Pico, copy the data to it, write the filesystem image out to disk and then copy the LittleFS image to the Pico.

Note that when Pocfel is compiled for Linux it actually does create a LittleFS in RAM and later writes that out to a file called pocfel.lfs when the program exits.

The file pocfel.lfs could, in theory, be flashed to the Pico at 1MB and any save game files read back from the image on the Pico later. I haven't actually tried it, though.

viewtopic.php?f=144&t=316321

Transferring the filesystem the other way from the Pico to the host, as asked in this thread, seems more difficult.

lurk101
Posts: 877
Joined: Mon Jan 27, 2020 2:35 pm
Location: Cumming, GA (US)

Re: Where are my files?

Thu Jul 29, 2021 6:48 am

I've got a Linux user littlefs file system (FUSE3) halfway done on a Pi4. I prefer this approach with the littlefs system implemented as a binary file since it mounts like any other file system and is manipulated with standard shell commands. The binary file system file can be extracted from or programmed to pico flash with gdb and openocd. Works both ways.

ejolson
Posts: 8025
Joined: Tue Mar 18, 2014 11:47 am

Re: Where are my files?

Thu Jul 29, 2021 12:05 pm

lurk101 wrote:
Thu Jul 29, 2021 6:48 am
I've got a Linux user littlefs file system (FUSE3) halfway done on a Pi4. I prefer this approach with the littlefs system implemented as a binary file since it mounts like any other file system and is manipulated with standard shell commands. The binary file system file can be extracted from or programmed to pico flash with gdb and openocd. Works both ways.
For the sake of the person making the original post and others, do you know where the LittleFS image starts in flash for micro Python and its length? It would be great to have a recipe to copy that filesystem image back to a PC so the files contained inside could be examined.

I guess ampy and rshell are the standard ways to get those files.

lurk101
Posts: 877
Joined: Mon Jan 27, 2020 2:35 pm
Location: Cumming, GA (US)

Re: Where are my files?

Thu Jul 29, 2021 2:50 pm

ejolson wrote:
Thu Jul 29, 2021 12:05 pm
lurk101 wrote:
Thu Jul 29, 2021 6:48 am
I've got a Linux user littlefs file system (FUSE3) halfway done on a Pi4. I prefer this approach with the littlefs system implemented as a binary file since it mounts like any other file system and is manipulated with standard shell commands. The binary file system file can be extracted from or programmed to pico flash with gdb and openocd. Works both ways.
For the sake of the person making the original post and others, do you know where the LittleFS image starts in flash for micro Python and its length? It would be great to have a recipe to copy that filesystem image back to a PC so the files contained inside could be examined.

I guess ampy and rshell are the standard ways to get those files.
My aversion to interpreted languages has kept me away from Python. I don't know how it sizes the FS or where it puts it. My guess is at the top of flash.

lurk101
Posts: 877
Joined: Mon Jan 27, 2020 2:35 pm
Location: Cumming, GA (US)

Re: Where are my files?

Thu Jul 29, 2021 3:48 pm

kilograham wrote:
Tue Jul 27, 2021 6:51 pm
I added an issue here
https://github.com/raspberrypi/pico-sdk/issues/531

we probably won't bundle littlefs, but we might make a cmake wrapper library such that if you have littlefs checked out you can add it trivially by adding a single dependency (similar to tinyusb)
Sounds good....

hippy
Posts: 10530
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Where are my files?

Thu Jul 29, 2021 7:09 pm

ejolson wrote:
Thu Jul 29, 2021 12:05 pm
For the sake of the person making the original post and others, do you know where the LittleFS image starts in flash for micro Python and its length?
It varies from board to board, is build defined, user configurable, set according to Flash fitted and firmware size. Standard boards are defined in -

~/pico/micropython/ports/rp2/boards

I believe it is always the case that the file system start address is 'top of Flash minus file system length'. For the Pico with 2MB of Flash. "MICROPY_HW_FLASH_STORAGE_BYTES" is set to 1408 * 1024, starts at 0x100A0000. I don't know if the root entry is at the top or bottom.

I have mentioned on the MicroPython forum that I think this is a ridiculous way of doing it, believe MicroPython should work it out for itself as CircuitPython does. Making it user defined not only means a different build for each board but has led to an Adafruit board presenting just 1.4MB of file system even though it has 8MB of Flash fitted with over 6MB of Flash completely wasted.
ejolson wrote:
Thu Jul 29, 2021 12:05 pm
I guess ampy and rshell are the standard ways to get those files.
Yes, though 'mpremote' would be the recommended way these days.

Return to “SDK”