User avatar
HermannSW
Posts: 4010
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Pico MicroPython with networking over USB

Mon Apr 19, 2021 1:07 am

Thread "Pico HTTPS GET through ESP01 serial2wifi" provided some Pico MicroPython networking via connected ESP01:
viewtopic.php?f=146&t=307704


I just built prototypical fork from @incognitum's (C SDK) pico-webserver with networking over USB, showing that networking stuff can completely run from core1:
https://github.com/Hermann-SW2/pico-web ... -statement
Image


Below (official Raspberry display) screenshot from 192.168.7.1 webserver running on Pico (without MicroPython currently) while Pico MicroPython is running as well is the target. Likely REPL over UART because USB is needed for networking. Of course running lwip networing on core1 does not allow for thread in Pico MicroPython ...
Pico_uPython.scrot.png
Pico_uPython.scrot.png
Pico_uPython.scrot.png (226.21 KiB) Viewed 594 times

P.S:
I will work on this after wakeup from hospital operation anesthesia in a few hours from now ...
https://stamm-wilbrandt.de/2wheel_balancing_robot
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://github.com/Hermann-SW/raspiraw
https://stamm-wilbrandt.de/en/Raspberry_camera.html

hippy
Posts: 9664
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Pico MicroPython with networking over USB

Mon Apr 19, 2021 10:46 am

Hoping everything goes well, you have a swift recovery, and are back soon.

hippy
Posts: 9664
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Pico MicroPython with networking over USB

Mon Apr 19, 2021 10:58 am

HermannSW wrote:
Mon Apr 19, 2021 1:07 am
Below (official Raspberry display) screenshot from 192.168.7.1 webserver running on Pico (without MicroPython currently) while Pico MicroPython is running as well is the target. Likely REPL over UART because USB is needed for networking. Of course running lwip networing on core1 does not allow for thread in Pico MicroPython ...
From Gadget Mode experience with the Pi I recall it is possible to have a composite device which supports RNDIS and CDC/ACM. That should mean it's just a matter of twiddling 'tusb_configure.h' and 'tusb_ports.c' to present that, plus adding the hooks into TinyUSB to handle it.

It should be possible to run the network stack, plus web server and network services, concurrently with the REPL / MicroPython program in core 0 using callbacks, but I have no idea how that would be done, how to do it.

Getting it to work with MicroPython, however done, is a big step, provides a foundation for everything to come. So well done and thanks. The rest is optimisation, improvement.

User avatar
HermannSW
Posts: 4010
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Pico MicroPython with networking over USB

Tue Apr 20, 2021 8:47 am

HermannSW wrote:
Mon Apr 19, 2021 1:07 am
...
Likely REPL over UART because USB is needed for networking. Of course running lwip networing on core1 does not allow for thread in Pico MicroPython ...Image

P.S:
I will work on this after wakeup from hospital operation anesthesia in a few hours from now ...
Yesterday I had the problem that GPIOs of Pi4B with official display in RB-LCD-7P4 case are not accessible from the outside. I have no tools here in hospital, but was able to get the job of exposing Pi4B GPIOs done with just a hospital scissors, endurance and force:
viewtopic.php?f=108&p=1853981#p1853981

Today I connected Pico UART to Pi4B GND+GPIO14+GPIO15 and verified by flashing pico-examples/hello_world/serial .uf2 to Pico with minicom session, that /dev/serial0 works. The black cable from RUN pin to "not connected" acts as switch allowing to reset the Pico (by connecting "not connected" cable side to GND). serial hellow_world confirmed that this works as well, output of "Hello, world!" rows in minicom stops while that cable side is connected to GND. In case Bootsel switch is pressed and release while RUN is connected to GND, Pico gets mounted as storage device to Pi4B after the black cable side is disconnected from GND. Now I will continue to integrate networking(+webserver) to Pico Micropython, and make REPL show up over UART and disable REPL over USB in order to not interfere with networing over USB (lwip on core1):
20210420_101029.10%.jpg
20210420_101029.10%.jpg
20210420_101029.10%.jpg (20.92 KiB) Viewed 473 times
https://stamm-wilbrandt.de/2wheel_balancing_robot
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://github.com/Hermann-SW/raspiraw
https://stamm-wilbrandt.de/en/Raspberry_camera.html

User avatar
HermannSW
Posts: 4010
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Pico MicroPython with networking over USB

Tue Apr 20, 2021 8:30 pm

I have first working firmware.uf2(!), but code is too bad to commit to my new micropython fork.

The webpage displayed is not the one from my modified pico-webserver.
I had to add stuff to CMakeLists.txt in an ugly way,
I got duplicate symbols due to not-perfect integration, just commented out some lines.

But:
lwip networking runs on core1, and as can be seen Pico website http://192.168.7.1 shows in Pi4B browser.
UART MicroPython REPL works, MicroPython allowed me to turn Pico led on (right).
A lot cleanup needed tomorrow, but a good start.
I got ram overflow and did reduce gc_heap to 182KB from 192KB to make everything fit into Pico RAM.

16MP smartphone photo: https://stamm-wilbrandt.de/en/forum/mp/ ... 220347.jpg
Scaled to 15%:
20210420_220347.15%.jpg
20210420_220347.15%.jpg
20210420_220347.15%.jpg (48.06 KiB) Viewed 438 times
After "make clean", "make" took 8 minutes on Pi4B SDcard to build.
After "make clean", "make -j4" took 5 minutes on Pi4B SDcard build.

If you want to try out, flash this (620KB) firmware.uf2 to your Pico that is connected via UART (GP0/GP1/GND to GPIO15/GPIO14/GND) and USB:
https://stamm-wilbrandt.de/en/forum/mp/firmware.uf2

Then verify that your Pi gets USB ip address 192.168.7.2 with ifconfig.
Open http://192.168.7.1 in your browser.
Open REPL via "minicom -o -D /dev/serial0".

Please let me know of any issues you find.


P.S:
After cleanup and commit+push, work on Pico MicroPython usocket module is needed.

P.P.S
firmware.uf2 built with "make clean && make" and "make clean && make -j4" are identical.
https://stamm-wilbrandt.de/2wheel_balancing_robot
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://github.com/Hermann-SW/raspiraw
https://stamm-wilbrandt.de/en/Raspberry_camera.html

User avatar
HermannSW
Posts: 4010
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Pico MicroPython with networking over USB

Wed Apr 21, 2021 9:47 am

Webpage changes (in fs subdirectory) can be verified without flashing by "chromium-browser fs/index.html" easily. When satisfied with the edits, "rm build-PICO/CMakeFiles/lwipallapps.dir/home/pi/pico/micropython/lib/lwip/src/apps/http/fs.c.obj" allows for fast (13 seconds) recreation of new firmware.uf2 after webpage changes:

Code: Select all

i@raspberrypi4B:~/pico/micropython/ports/rp2 $ vi fs/index.html 
pi@raspberrypi4B:~/pico/micropython/ports/rp2 $ ./regen-fsdata.sh 
Regenerating fsdata.c

 makefsdata - HTML to C source converter
     by Jim Pettinato               - circa 2003 
     extended by Simon Goldschmidt  - 2009 

HTTP 1.0 header will be statically included.
  Processing all files in directory fs and subdirectories...

processing /index.html...
processing /pico.png...
processing /uPython.png...

Creating target file...


Processed 3 files - done.

Done
pi@raspberrypi4B:~/pico/micropython/ports/rp2 $ rm build-PICO/CMakeFiles/lwipallapps.dir/home/pi/pico/micropython/lib/lwip/src/apps/http/fs.c.obj
pi@raspberrypi4B:~/pico/micropython/ports/rp2 $ time make -j4
[ -d build-PICO ] || cmake -S . -B build-PICO -DPICO_BUILD_DOCS=0 -DMICROPY_BOARD=PICO
make -s -C build-PICO
[  0%] Performing build step for 'ELF2UF2Build'
[  1%] Built target bs2_default
[  2%] Building C object CMakeFiles/lwipallapps.dir/home/pi/pico/micropython/lib/lwip/src/apps/http/fs.c.obj
[  3%] Built target bs2_default_padded_checksummed_asm
[ 24%] Built target lwipcore
[100%] Built target elf2uf2
[ 24%] No install step for 'ELF2UF2Build'
[ 24%] Completed 'ELF2UF2Build'
[ 25%] Built target ELF2UF2Build
[ 26%] Linking C static library liblwipallapps.a
[ 33%] Built target lwipallapps
[ 33%] Generating genhdr/mpversion.h
[ 33%] Generating frozen_content.c
[ 33%] Generating genhdr/mpversion.h
[ 33%] Generating frozen_content.c
[ 33%] Linking CXX executable firmware.elf
   text	   data	    bss	    dec	    hex	filename
 538268	     88	 225344	 763700	  ba734	/home/pi/pico/micropython/ports/rp2/build-PICO/firmware.elf
[100%] Built target firmware

real	0m12.530s
user	0m4.779s
sys	0m1.759s
pi@raspberrypi4B:~/pico/micropython/ports/rp2 $ 

Now the correct webpage gets used:

Code: Select all

pi@raspberrypi4B:~/pico/micropython/ports/rp2 $ grep fsdata lwipopts.h 
#define HTTPD_FSDATA_FILE               "../../../../../ports/rp2/fsdata.c"
pi@raspberrypi4B:~/pico/micropython/ports/rp2 $ 

Screenshot taken with "sleep 8 && scrot" command in terminal:
2021-04-21-114354_800x480_scrot.png
2021-04-21-114354_800x480_scrot.png
2021-04-21-114354_800x480_scrot.png (162.95 KiB) Viewed 371 times
https://stamm-wilbrandt.de/2wheel_balancing_robot
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://github.com/Hermann-SW/raspiraw
https://stamm-wilbrandt.de/en/Raspberry_camera.html

User avatar
HermannSW
Posts: 4010
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Pico MicroPython with networking over USB

Wed Apr 21, 2021 4:54 pm

I am done with cleanup and commited+pushed the changes to my micropython fork with this commit (it shows nicely what all needed to be changed and where):
https://github.com/Hermann-SW/micropyth ... 192ab2a123


In case you just want to try without building MicroPython yourself, here is prebuilt firmware.uf2:
https://github.com/Hermann-SW/micropyth ... rmware.uf2


For building MicroPython for the Pico with USB networking from source
https://datasheets.raspberrypi.org/pico ... pdf#page=5

lwip submodule needs to be inited as well:
git submodule update --init -- lib/pico-sdk lib/tinyusb lib/lwip

I added fork mission statement as well:
https://github.com/Hermann-SW/micropyth ... -statement
2021-04-21-185328_800x480_scrot.png
2021-04-21-185328_800x480_scrot.png
2021-04-21-185328_800x480_scrot.png (75.26 KiB) Viewed 334 times
https://stamm-wilbrandt.de/2wheel_balancing_robot
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://github.com/Hermann-SW/raspiraw
https://stamm-wilbrandt.de/en/Raspberry_camera.html

hippy
Posts: 9664
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Pico MicroPython with networking over USB

Wed Apr 21, 2021 7:25 pm

HermannSW wrote:
Wed Apr 21, 2021 4:54 pm
I am done with cleanup and commited+pushed the changes to my micropython fork with this commit (it shows nicely what all needed to be changed and where):
https://github.com/Hermann-SW/micropyth ... 192ab2a123
Thanks for that. I am currently building and linking from source, but not calling lwip or configuring USB, to progress incrementally. That ...

lwipopts.h

Code: Select all

#define HTTPD_FSDATA_FILE    "../../../../../ports/rp2/fsdata.c"
Is something of a PITA being hardwired to a particular port, the official RP2 port, which I much prefer to leave intact; I replicate 'rp2' then use that.

I am not sure what the ideal solution would be to cater for different port directories and wanting to build multiple web servers from the same port but with different content served. But, as a start, I think 'regen-fsdata.sh' and the generated 'fsdata.c' would be best placed in the 'fs' directory. That way there is just a single self-contained directory for content, and that directory can go anywhere, have any name.

User avatar
HermannSW
Posts: 4010
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Pico MicroPython with networking over USB

Thu Apr 22, 2021 7:23 am

hippy wrote:
Wed Apr 21, 2021 7:25 pm
That ...

lwipopts.h

Code: Select all

#define HTTPD_FSDATA_FILE    "../../../../../ports/rp2/fsdata.c"
Is something of a PITA being hardwired to a particular port, the official RP2 port, which I much prefer to leave intact; I replicate 'rp2' then use that.
It was "../../../../fsdata.c" before in @incognitum's pico-webserver, and my focus is more on providing Pico MicroPython usocket so that any usocket code "just runs" on the Pico without change.
I think 'regen-fsdata.sh' and the generated 'fsdata.c' would be best placed in the 'fs' directory. That way there is just a single self-contained directory for content, and that directory can go anywhere, have any name.
I don't think so, since I want to handle lwip as submodule and canges inside submodule is not something I like or know how to handle:

Code: Select all

pi@raspberrypi4B:~/pico/micropython $ find lib/lwip -name fs
lib/lwip/src/apps/http/fs
pi@raspberrypi4B:~/pico/micropython $ 
pi@raspberrypi4B:~/pico/micropython $ sed -n "/lwip/,/sava/p" .gitmodules 
[submodule "lib/lwip"]
	path = lib/lwip
	url = https://git.savannah.gnu.org/r/lwip.git
pi@raspberrypi4B:~/pico/micropython $ 
https://stamm-wilbrandt.de/2wheel_balancing_robot
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://github.com/Hermann-SW/raspiraw
https://stamm-wilbrandt.de/en/Raspberry_camera.html

hippy
Posts: 9664
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Pico MicroPython with networking over USB

Thu Apr 22, 2021 10:57 am

HermannSW wrote:
Thu Apr 22, 2021 7:23 am
hippy wrote:
Wed Apr 21, 2021 7:25 pm
I think 'regen-fsdata.sh' and the generated 'fsdata.c' would be best placed in the 'fs' directory. That way there is just a single self-contained directory for content, and that directory can go anywhere, have any name.
I don't think so, since I want to handle lwip as submodule and canges inside submodule is not something I like or know how to handle:
AFAICT you wouldn't need to alter anything in lwip itself; the local 'lwipopts.h' already tells lwip where to find its content data outside of its own library directory. It's just a matter of making sure 'lwipopts.h' specifies the right location, and that 'fsdata.c' content is there. This works for me, 'pico-1', 'pico-2' etc, all having different content -

Code: Select all

#define HTTPD_FSDATA_FILE    "/home/pi/picowebserver/pico-1/fsdata.c"
If there is only one 'fs', only one 'fsdata.c' it is difficult to build firmware for different Pico with different content.

Update : I have just had a thought that it may not matter; we shouldn't have to build 'fsdata.c' when content can be whatever is placed on the MicroPython file system. As a stop-gap I can live with 'fsdata.c'.

User avatar
HermannSW
Posts: 4010
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Pico MicroPython with networking over USB

Thu Apr 22, 2021 12:07 pm

What do I want to do with Pico Micropython when I will have usocket running (with lwIP over USB)?

For example using the official two MicroPython network http[⁠s] client examples. Unfortunately both official examples work for google.com host only.

Today I identified root cause (examples miss to present "Host" header most webservers require to see):
https://forum.micropython.org/viewtopic ... 364#p57364

The commit has the details of the changes needed for working http and https examples against arbitrary webservers:
https://github.com/Hermann-SW/micropyth ... cf85fbb86d

I tested the corrected examples with Micropython Unix port as well as python3.
Here is micropython https example execution:
mp_corrected_examples.png
mp_corrected_examples.png
mp_corrected_examples.png (80.08 KiB) Viewed 243 times

Now I "only" need to get usocket running for the Pico ...
https://stamm-wilbrandt.de/2wheel_balancing_robot
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://github.com/Hermann-SW/raspiraw
https://stamm-wilbrandt.de/en/Raspberry_camera.html

User avatar
HermannSW
Posts: 4010
Joined: Fri Jul 22, 2016 9:09 pm
Location: Eberbach, Germany
Contact: Website Twitter YouTube

Re: Pico MicroPython with networking over USB

Sat Apr 24, 2021 3:09 pm

HermannSW wrote:
Thu Apr 22, 2021 12:07 pm
Now I "only" need to get usocket running for the Pico ...
Micropython http client examples are corrected, but I had problems to add usocket support to my existing micropython fork with networking and webserver running on core1, especially compile problems wrt lwip.

Yesterday I learned about native C code programming for Micropython (.mpy) and how to support Pico with a 1line change in micropython/py/dynruntime.mk. Today I found out how to create a module with multiple member functions and multiple integer and string arguments. I did commit the example and dynruntime.mk to my branch, more details and commit link in this posting:
viewtopic.php?f=146&t=310282&p=1856106#p1856106

"peek" screenrecorder animation demonstrates building "string.c" and using module's "len()" and "substr()" member functions:
Image


Now I will create "usocket_native.c" module and import that in new "usocket.py" module. I will start with API of empty member functions found here:
https://github.com/thonny/thonny/blob/m ... usocket.py

First function I will try to get working is "usocket.getaddrinfo()" ...
https://stamm-wilbrandt.de/2wheel_balancing_robot
https://stamm-wilbrandt.de/en#raspcatbot
https://github.com/Hermann-SW/Raspberry_v1_camera_global_external_shutter
https://github.com/Hermann-SW/raspiraw
https://stamm-wilbrandt.de/en/Raspberry_camera.html

Return to “MicroPython”