hippy
Posts: 3882
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Diskless booting Zeroes from Windows

Mon Oct 08, 2018 9:09 pm

Pirate Python

https://github.com/pimoroni/PiratePython

Argh, 'tis interesting that. Be well worth having a play with, me hearties. It's perhaps a bit too heavyweight for my liking, basically a trimmed-down Raspbian rather than a buildroot, so boot times are rather long. But it would be a good stepping-stone to full Raspbian diskless booting if we can get this doing that.

Unzipped the download, copied and pasted to an empty SD Card, plugged that into my Pi Zero W, plugged the Pi into a Windows 10 system via USB ( no PSU for the Zero W ), and off it went. Seemed to stall at the rainbow screen for a while but then ran okay. Some weirdness on the attached Pi monitor but hey ho; "alpha".

Windows 10 recognised the two serial ports, and MSD. It said "there was a problem" with the new removable disk but was fine after that, and I recall maybe the same with 'rpiboot -d msd' the first time. WiFi came up which I can ping okay.

USB networking is also okay. There's a usb0 interface on 10.0.99.1 and I can ping that from Windows 10. Interestingly it's a "Remote NDIS Compatible Device" rather than "RNDIS". Whatever it is; it's working!

The really good news is I can edit the main.py on the MSD and when it's saved, changes are detected and run on the Pi. I thought it was meant to be impossible to have MSD accessible from both sides of the fence. Obviously not. But they do admit to it being hacky and hoping to move to MTP instead. I actually hope not because that to me is a retrograde step as XP doesn't support MTP.

As it is it's pretty much what I want for my App Engines, virtually my imagined "PI macro:bit". So maybe that means a walk away from diskless booting and into making Pirate Python do exactly as I'd like. Perhaps for a while, before burnout sets in.

If someone does want to push forward themselves; the priority is getting gadget mode to present serial, networking and MSD interfaces at the same time, which can be recognised and used by Windows. That should, in theory, only involve editing -

~/gpioexpander/gpioexpand/board/overlay/etc/init.d/S99gpioext

Without networking; NFS, SSH and anything similar is stalled.

hippy
Posts: 3882
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Diskless booting Zeroes from Windows

Tue Oct 09, 2018 12:17 am

hippy wrote:
Mon Oct 08, 2018 9:09 pm
So maybe that means a walk away from diskless booting and into making Pirate Python do exactly as I'd like.
Or maybe not. That seems to be a binary only release and might as well be closed source for the lack of instructions on how to create a build other than a few cryptic posts in various places.

I mounted the initrd and there's nothing of much use in there I can see so did the same with kernel.img and that was pretty much the same. I've no idea where things I'd expect to see are. Life's too short and getting shorter by the day.

hippy
Posts: 3882
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Diskless booting Zeroes from Windows

Tue Oct 09, 2018 7:55 pm

Got a little further with PiratePython, have figured out it uses 'multistrap' and have built the system files and packages but haven't yet figured out how to fully translate the developer's 'build' file into something which works for me.

Nope; by now I wasn't at all surprised that running the provided 'build' script didn't work.

Another sideways-venture straight into a forest, though it does seem multistrap may be a better and easier to use build system than buildroot, so perhaps a worthwhile one.

And 'multistrap' runs on the Pi, doesn't require WSL, uses genuine Raspbian repositories rather than cross-compiling packages from source, which is a major advantage and much quicker to do.

And also progress with buildroot and the 'wchar_t' issue building Python. Seems that having enabled WCHAR support for the Buildroot Toolchan one needs to use 'make clean' before running 'make', and of course another './build.sh'.

And that takes it's time. But after 8 hours rebuilding the toolchain and packages; Python 2.7.13 is running on the disklessly booted Pi.

I also had to 'sudo apt-get install mercurial' because 'hg' is required for whatever combination of packages I selected to include in the build. And re-fix fakeroot because 'make clean' scrubbed that previous fix.

I didn't make a note of what I selected as packages but I will go through and see which commonly used script interpreters build and don't, see what other fixes are needed to make those work.

hippy
Posts: 3882
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Diskless booting Zeroes from Windows

Tue Oct 09, 2018 11:47 pm

Buildroot-2017.02 supported scripting and interpreter languages when built using the currently described WSL set-up for a Pi Zero.

Note it's an either-or choice for Python 2 or Python 3, Lua or LuaJit. It is not possible to select both together.

Working

Ficl - 4.1.0
Haserl - 0.9.35
Lua - 5.3.4
MicroPython - 1.8.7 (+libs)
Perl - 5.24.1
Python - 2.7.13 (+libs)
Ruby - 2.4.0

Not working yet

LuaJit : Toolchain issues
PHP - 7.1.1 : Compiled but seemingly not in image

4th, JamVM, JimTcl, Mono and Tcl : All report they need 'javac' installing on WSL but that may be a bug, a latch-up, as it reports that now even when only using stuff which previously built. Ho hum. Eleventh Circle.

Need to test

Python 3

Cannot be built

NodeJS : No toolchain support

Not tested

Enscript, Erlang, Gauche, Guile, NoarVM

hippy
Posts: 3882
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Diskless booting Zeroes from Windows

Wed Oct 10, 2018 12:34 am

hippy wrote:
Tue Oct 09, 2018 11:47 pm
4th, JamVM, JimTcl, Mono and Tcl : All report they need 'javac' installing on WSL but that may be a bug, a latch-up, as it reports that now even when only using stuff which previously built. Ho hum. Eleventh Circle.
And now everything's broken. It seems to be mixing ARM and i586 builds and unsurprisingly failing. Looks like a complete reinstall and start from the beginning again is the best way to proceed.

No great surprise. Welcome to the Twelfth Circle.

hippy
Posts: 3882
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Diskless booting Zeroes from Windows

Wed Oct 10, 2018 11:42 pm

I reinstalled from scratch and offset the pain of having to do so against it helping confirm the documentation I am writing matches reality. That's coming along nicely and it's a simpler process when one knows what to do and the best order to do it in.

I did have 'another aside thought' having moved the 'diskless boot image' to SD Card just to check that it was bootable from there ( it is ) -

Once it's booting from SD Card it no longer appears as a device which is USB bootable when plugged into the PC. That's fair enough.

But I was wondering if there were something 'gadget mode' where it could fake itself to appear as if a USB bootable device while running as normal, could take what rpiboot throws at it by way of bootcode.bin and pivot to that ?

It would probably mean altering rpiboot to allow additional VID/PID pairs to be specified so it matches what's connected because I don't think a Pi ( or anything ) can have two VID/PID pairs at a time to allow matching what rpiboot expects by default.

That would save having to keep unplugging and re-plugging the Pi, removing and reinserting SD Cards when they have a bootcode.bin on them, having to login, delete files and having to 'reboot' or whatever may work.

User avatar
thagrol
Posts: 868
Joined: Fri Jan 13, 2012 4:41 pm
Location: Darkest Somerset, UK
Contact: Website

Re: Diskless booting Zeroes from Windows

Thu Oct 11, 2018 2:51 pm

I read somewhere on the forumn that if a bootable SD card is present a Pi will always boot from that in preference to USB, network, etc. booting.

That said, there must be a way to surpess booting from SD/eMMC as otherwise it would only be possible to program a CM/CM3 once. I guess there's something in hardware on the CM carriet board that handles this and possibly in the Videocore boot code for these boards.
Note to self: don't feed the trolls
If you believe "L'enfer, c'est les autres" (Hell is other people) have you considered that it may be of your own making?

mikerr
Posts: 2728
Joined: Thu Jan 12, 2012 12:46 pm
Location: UK
Contact: Website

Re: Diskless booting Zeroes from Windows

Thu Oct 11, 2018 3:46 pm

thagrol wrote:
Thu Oct 11, 2018 2:51 pm
I read somewhere on the forumn that if a bootable SD card is present a Pi will always boot from that in preference to USB, network, etc. booting.
Pi generally always tries SD first, but if no bootcode.bin is present, it moves to the next device, full bootflow here:
https://www.raspberrypi.org/documentati ... ootflow.md
Android app - Raspi Card Imager - download and image SD cards - No PC required !

hippy
Posts: 3882
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Diskless booting Zeroes from Windows

Thu Oct 11, 2018 5:37 pm

Yes; that's the problem. A Pi booting from SD Card, never comes up as a bootable device, comes up as a standard Pi ( I'm not interested in that case ), or comes up in Gadget Mode, presents itself to the PC for networking etc, but has a VID/PID which rpiboot doesn't recognise, so won't upload to. And, even if it did there's nothing running on the Pi which would understand what rpiboot was trying to do.

I thought further on it and it should be simpler than I first believed.

It would be easy enough (famous last words ) if the Pi exposes a virtual serial console and an MSD device serving out the full card. Delete '/bootcode.bin' via MSD. Send a "reboot" via the serial. So should be equally easy to have a simple app running on the Pi which does just that triggered by some side channel, a modified WoL packet perhaps, or a TCP/IP server waiting for a "do it" command.

On the rpiboot side, all it needs to do is trigger the Pi to delete '/bootcode.bin' and reboot via that side-channel, and then it's business as usual, waiting for the Pi to appear as a bootable device.

For the CM I believe there's a hardware jumper to adjust. I guess that disables the eMMC which causes a powering-up Pi to fall through to its bootable from a PC mode.

fruitoftheloom
Posts: 17603
Joined: Tue Mar 25, 2014 12:40 pm

Re: Diskless booting Zeroes from Windows

Thu Oct 11, 2018 5:44 pm

hippy wrote:
Thu Oct 11, 2018 5:37 pm
Yes; that's the problem. A Pi booting from SD Card, never comes up as a bootable device, comes up as a standard Pi ( I'm not interested in that case ), or comes up in Gadget Mode, presents itself to the PC for networking etc, but has a VID/PID which rpiboot doesn't recognise, so won't upload to. And, even if it did there's nothing running on the Pi which would understand what rpiboot was trying to do.

I thought further on it and it should be simpler than I first believed.

It would be easy enough (famous last words ) if the Pi exposes a virtual serial console and an MSD device serving out the full card. Delete '/bootcode.bin' via MSD. Send a "reboot" via the serial. So should be equally easy to have a simple app running on the Pi which does just that triggered by some side channel, a modified WoL packet perhaps, or a TCP/IP server waiting for a "do it" command.

On the rpiboot side, all it needs to do is trigger the Pi to delete '/bootcode.bin' and reboot via that side-channel, and then it's business as usual, waiting for the Pi to appear as a bootable device.

For the CM I believe there's a hardware jumper to adjust. I guess that disables the eMMC which causes a powering-up Pi to fall through to its bootable from a PC mode.

https://www.raspberrypi.org/documentati ... lashing.md
Adieu

hippy
Posts: 3882
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Diskless booting Zeroes from Windows

Thu Oct 11, 2018 7:49 pm

Still having a struggle trying to smash PiratePython's Gadget Configuration which works when used -

https://github.com/pimoroni/PiratePytho ... python-usb

into GPIO Expander's configuration which doesn't work for networking -

https://github.com/raspberrypi/gpioexpa ... S99gpioext

My current thinking is that what PiratePython wants isn't set-up as required by the GPIO Expander build I'm subverting, and the GPIO Expander build doesn't allow what is needed to make things work.

I've just about given up on trying to fix it myself, and without network connectivity between the Pi and PC I don't think it's really that useful.

PiratePython's build seems to be a lot better, and 'multisplit' a better option to use, but trying to build PiratePython myself is proving equally difficult.

I'll post what documentation I have, which will allow others to get to where I am, and I'll see if anyone is able to sort networking out before throwing in the towel.


hippy
Posts: 3882
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Diskless booting Zeroes from Windows

Fri Oct 12, 2018 1:07 pm

Re-booting a Pi into a bootable mode does seem possible from proof of concept on a Raspbian Pi 3B.

This is 'rpibootd.py' the side channel server which runs on a Pi which can be used to put the Pi back into bootable mode ...

Code: Select all

#!/usr/bin/python

RPI_REBOOT_PORT    = 1234
RPI_REBOOT_COMMAND = "rpiboot"

import BaseHTTPServer
import os
import threading
import time
import urllib

def RebootThread():
  os.popen("sudo cp /boot/bootcode.bin /boot/bootcode.bin.copy")
  time.sleep(1)
  os.popen("sudo reboot")

class HttpHandler(BaseHTTPServer.BaseHTTPRequestHandler):
  def log_message(self, format, *args):
    pass
  def do_GET(self):
    filename = urllib.unquote(self.path)
    self.send_response( 200 )
    self.send_header("Content-type", "text/html")
    self.end_headers()
    if filename == "/"+RPI_REBOOT_COMMAND:
      threading.Thread(target=RebootThread).start()
      self.wfile.write("OKAY\n"+self.client_address[0]+"\n"+filename+"\n")
    else:
      self.wfile.write("FAIL\n"+self.client_address[0]+"\n"+filename+"\n")

BaseHTTPServer.HTTPServer(("", RPI_REBOOT_PORT), HttpHandler).serve_forever()
Run that on the Pi at start-up with -

Code: Select all

python rpibootd.py &
Then to put the Pi back into bootable mode, from any PC -

Code: Select all

wget http://192.168.0.130:1234/rpiboot
Then just run 'rpiboot' as normally.

That 'wget' can be integrated into 'rpiboot' itself, and there are ways for that to determine which IP address needs to be used. Then we would have an 'rpiboot' which can upload to a Pi in bootable mode or any which has a network connection and is running the 'rpibootd' server.

That "RebootThread()" would need to be written to actually delete the 'bootcode.bin' and be tailored for the Pi set-up it is actually running on, and the entire server might be better written in C so there's no need for Python on the booted Pi.

Obviously the whole thing can be tightened up, challenge and response added if necessary, but this was just a proof of concept.

hippy
Posts: 3882
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Diskless booting Zeroes from Windows

Fri Oct 12, 2018 2:47 pm

Hallelujah !

At last some success with networking. I have finally managed to get it to come up with most of PiratePython's stuff, two serial ports and a Remote NDIS network with Window's drivers automatically installed and working, but no MSD yet.

I can login via the first serial port. Nothing on the second because I think that's for PiratePython diagnostics with no getty running on it.

After some configuration of Windows itself I can ping the remote booted Pi and it can ping the PC.

I think the PiratePython MSD setup is expecting something which I am not creating so borks everything when attempting to create it. I tried some MSD creation examples from the web but no joy there.

Still; I'm a lot further on than yesterday and a lot less despondent about it all.

I'll backup what's working, post that and maybe someone can figure out how MSD can be wedged into that.

hippy
Posts: 3882
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Diskless booting Zeroes from Windows

Fri Oct 12, 2018 5:57 pm

This is the './gpioexpand/board/overlay/etc/init.d/S99gpioext' I am currently using. Two serial plus RNDIS, no MSD yet -

Code: Select all

#!/bin/sh

case "$1" in
  start)
        GADGET=/sys/kernel/config/usb_gadget/g1

        SERIAL="$(grep Serial /proc/cpuinfo | sed 's/Serial\s*: 0000\(\w*\)/\1/')"
        MAC="$(echo ${SERIAL} | sed 's/\(\w\w\)/:\1/g' | cut -b 2-)"
        MAC_HST="12$(echo ${MAC} | cut -b 3-)"
        MAC_DEV="02$(echo ${MAC} | cut -b 3-)"

        mount -t configfs none /sys/kernel/config
        mkdir -p $GADGET
        (cd $GADGET

        # FIXME: obtain proper USB ID instead of using f055 (FOSS)
        echo 0xf055 > idVendor
        echo 0x0001 > idProduct

        echo "0x0200" > bcdUSB
        echo "0x02" > bDeviceClass
        echo "0x00" > bDeviceSubClass
        echo "0x3066" > bcdDevice
        echo "1" > os_desc/use
        echo "0xcd" > os_desc/b_vendor_code
        echo "MSFT100" > os_desc/qw_sign

        mkdir strings/0x409
        echo $SERIAL > strings/0x409/serialnumber
        
# Need to change below ...
echo "Pimoroni Ltd." > strings/0x409/manufacturer
echo "PiratePython" > strings/0x409/product
# Need to change above

        mkdir configs/c.1
        mkdir configs/c.1/strings/0x409
        echo "CDC 2xACM+Mass Storage+RNDIS" > configs/c.1/strings/0x409/configuration
        echo 500 > configs/c.1/MaxPower

        mkdir functions/acm.GS0
        mkdir functions/acm.GS1

        mkdir functions/rndis.usb0
        echo "RNDIS" > functions/rndis.usb0/os_desc/interface.rndis/compatible_id
        echo "5162001" > functions/rndis.usb0/os_desc/interface.rndis/sub_compatible_id
        echo $MAC_HST > functions/rndis.usb0/host_addr
        echo $MAC_DEV > functions/rndis.usb0/dev_addr

# The lines below breaks things ...
# mkdir functions/mass_storage.piratepython
# echo "/dev/mmcblk0p1" > functions/mass_storage.piratepython/lun.0/file
# echo 0 > functions/mass_storage.piratepython/stall
# echo 0 > functions/mass_storage.piratepython/lun.0/cdrom
# echo 0 > functions/mass_storage.piratepython/lun.0/nofua
# echo 1 > functions/mass_storage.piratepython/lun.0/removable
# echo "PiratePython" > functions/mass_storage.piratepython/lun.0/inquiry_string
# Broken above

        ln -s functions/rndis.usb0 configs/c.1
        ln -s configs/c.1 os_desc

        echo "20980000.usb" > UDC

        sleep 2

        echo "" > UDC

        ln -s functions/acm.GS0 configs/c.1
        ln -s functions/acm.GS1 configs/c.1

# The line below breaks things ...
# ln -s functions/mass_storage.piratepython configs/c.1
# Broken above

        echo "0x00" > bDeviceClass

        echo "20980000.usb" > UDC

        )

        # Set led0 trigger to none, so apps can use it for other purposes
        echo none > /sys/class/leds/led0/trigger
        echo 255 > /sys/class/leds/led0/brightness

        ifconfig usb0 192.168.0.99 up

        echo
        uname -a
        echo
        ifconfig -a
 	echo
        ;;
  stop)
	;;
  restart|reload)
	"$0" stop
	"$0" start
	;;
   *)
	echo "Usage: $0 {start|stop|restart}"
	exit 1
esac

exit $?

ejolson
Posts: 2008
Joined: Tue Mar 18, 2014 11:47 am

Re: Diskless booting Zeroes from Windows

Fri Oct 12, 2018 9:49 pm

hippy wrote:
Fri Oct 12, 2018 2:47 pm
At last some success with networking. I have finally managed to get it to come up with most of PiratePython's stuff, two serial ports and a Remote NDIS network with Window's drivers automatically installed and working, but no MSD yet.
Given your success with Windows XP, I decided to try Windows Vista. For a preliminary test, I used the Mathematica USB dongle prepared in this post. After plugging the Zero into a USB port, Windows popped up a screen saying it needed an RNDIS driver. Strangely enough, the correct driver is obtained by clicking on

Update Driver Software
->Browse my computer for driver software
->Let me pick from a list of device drivers on my computer
->Microsoft Corporation
->Microsoft Windows Mobile Remote Adapter

After installing the driver software, the Mathematica dongle used DHCP to assign an address to the USB network device and I was able to login over ssh immediately. This verifies that the USB Ethernet gadget is working on Windows. Next I'll try mounting a Windows share on the Zero hosted by the Windows machine.

ejolson
Posts: 2008
Joined: Tue Mar 18, 2014 11:47 am

Re: Diskless booting Zeroes from Windows

Fri Oct 12, 2018 11:16 pm

ejolson wrote:
Fri Oct 12, 2018 9:49 pm
Next I'll try mounting a Windows share on the Zero hosted by the Windows machine.
After turning off the Windows firewall and enabling file sharing with passwords, I was able to mount a CIFS share from the Windows machine over the USB Gadget onto the Pi using a command such as

# mkdir /share
# mount -t cifs -o workgroup=X,username=Y,vers=2.0,dir_mode=0700,file_mode=0700 //192.168.7.2/Users/Y/Pi /share

After this I changed to the /share directory and created an ext4 filesystem using loop back with the commands

# cd /share
# rm -f root.ext4
# touch root.ext4
# dd bs=1024K seek=8192 count=0 if=/dev/zero of=root.ext4
# losetup /dev/loop1 root.ext4
# mke2fs -t ext4 -L ROOT /dev/loop1
# losetup -d /dev/loop1

Then I mounted the filesystem using

# mkdir /newroot
# mount -o loop /share/root.ext4 /newroot

and am able to read and write files to the newly created ext4 filesystem.

The next step is to copy the contents of a Raspbian SD card onto root.ext. Then it should be possible for rpiboot to load an initial RAM disk that pivots root onto root.ext4 accessed as a Windows share mounted over the USB Gadget.

hippy
Posts: 3882
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Diskless booting Zeroes from Windows

Fri Oct 12, 2018 11:46 pm

ejolson wrote:
Fri Oct 12, 2018 9:49 pm
Given your success with Windows XP, I decided to try Windows Vista.
Many thanks for your efforts. I had completely forgotten to check how what I've got so far works with XP! I'll see if I can replicate what you have achieved.

hippy
Posts: 3882
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Diskless booting Zeroes from Windows

Sat Oct 13, 2018 12:55 pm

What is it good for ?

Well one thing is trying out various monitor modes for a stubborn monitor which doesn't work with hdmi_safe or the default. Just go to the 'rpiboot from directory', open config.txt in WordPad, edit hdmi_group/hdmi_mode, save, re-run rpiboot which uploads the new configuration. No need to even close WordPad between edits.

Have managed to work out how to get Lua and Python scripts, other files and pre-compiled apps, into the build so they are there to be used when the Pi is booted. Launching them at start-up is next once I've completed a 'make clean' on Buildroot to install full Python.

So, apart from loose ends, getting MSD, file sharing, Wi-Fi and Bluetooth working, as a disklessly booted application engine it's all done in most respect. I have not yet decided how much of what's missing I'll fix myself or leave for others to figure out.

There's still the challenge of being able to disklessly boot multiple Pi boards. And I haven't even looked at disklessly booting a full Raspbian.

Any way, with my monitor now working, 'make clean' running, I'm off to the shops and I'll see what pops into my mind on that excursion.

ejolson
Posts: 2008
Joined: Tue Mar 18, 2014 11:47 am

Re: Diskless booting Zeroes from Windows

Sat Oct 13, 2018 8:52 pm

ejolson wrote:
Fri Oct 12, 2018 11:16 pm
The next step is to copy the contents of a Raspbian SD card onto root.ext.
To further check compatibility I downloaded the rpiboot.exe Windows installer from this blog and installed it. The installer claimed there were signing difficulties with the drivers, but I clicked to continue anyway.

I connected a cardless Pi Zero and ran something like

C:>rpiboot -d boot

on the Windows computer. Here boot is a subdirectory copied from the head node in this super-cheap cluster that contains an initial RAM disk and the other usual things. The result shows the kernel and RAM disk are loading and verifiess rpiboot.exe is compatible with Vista.

At this point Raspbian fails to continue booting because it is trying to mount root over NFS rather than as an ext4 image file looped over CIFS. After the initial RAM filesystem is updated, I expect to have Raspbian running diskless with a root filesystem hosted by the Windows computer.

hippy
Posts: 3882
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Diskless booting Zeroes from Windows

Sat Oct 13, 2018 11:54 pm

ejolson wrote:
Sat Oct 13, 2018 8:52 pm
At this point Raspbian fails to continue booting because it is trying to mount root over NFS rather than as an ext4 image file looped over CIFS. After the initial RAM filesystem is updated, I expect to have Raspbian running diskless with a root filesystem hosted by the Windows computer.
I really do hope you achieve that, and it probably is just a case of the Pi being able to see the host in a manner it can cope with. It really should be fairly straight forward. Probably is if one knows what one's doing !

I have to admit that, despite the frustrations, the Pi is proving its remit. I've learned an awful lot about many things, Pi, Linux and Windows, these last few weeks.

I over-imagined the issue of getting a disklessly booted Pi back into bootable mode. There's no bootcode.bin on SD Card so a simple "reboot" has it back to appearing as being bootable ! Trigger that via a server as above and job done.

ejolson
Posts: 2008
Joined: Tue Mar 18, 2014 11:47 am

Re: Diskless booting Zeroes from Windows

Sun Oct 14, 2018 9:11 pm

ejolson wrote:
Sat Oct 13, 2018 8:52 pm
After the initial RAM filesystem is updated, I expect to have Raspbian running diskless with a root filesystem hosted by the Windows computer.
I fixed the initial RAM filesystem so it mounts a Windows share containing an ext4 image file and then pivots root to that image using loopback. This results in a Pi Zero without a SD card running Raspbian where the root filesystem is accessed over the USB Ethernet gadget from a Windows PC.

I'm not sure how Windows file sharing compares in reliability to iSCSI or a NBD, but at least it's built in to Windows so there isn't any need to install additional software. I'll clean things up a bit and then post a detailed description.

hippy
Posts: 3882
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Diskless booting Zeroes from Windows

Mon Oct 15, 2018 12:31 am

ejolson wrote:
Sun Oct 14, 2018 9:11 pm
This results in a Pi Zero without a SD card running Raspbian where the root filesystem is accessed over the USB Ethernet gadget from a Windows PC.
That is excellent news.
ejolson wrote:
Sun Oct 14, 2018 9:11 pm
I'm not sure how Windows file sharing compares in reliability to iSCSI or a NBD
I have not noticed any problems whenever I have used Windows file sharing from XP through Windows 10.

The hard part I find is usually in sharing a folder out in a way that other Windows OS's actually see it. And Windows repeatedly stalling on loading or saving files as it goes to check if previous connected shares exists when they don't.

Once both sides are talking everything has always been rock solid for me -- which is possibly tempting fate !

hippy
Posts: 3882
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Diskless booting Zeroes from Windows

Mon Oct 15, 2018 12:53 pm

Finally reached the Thirteenth Level. Tried to turn monitor blanking off and that has screwed everything. Backed that change out but it's now not generating any config.txt and booting hangs. So it's a choice of trying a 6 hour 'make clean' hoping that fixes thing with a potential 18 hour all-in effort if not, or go for the 12 hour start from scratch again. Ho Hum.

Using 'multistrap' seems a far better way of doing all this than buildroot, taking minutes on a Pi to build rather than hours on a 64-bit X86 PC. Unfortunately there's no help forthcoming on that so far, just silence. Same too on building rpiboot for Windows.

To be honest; I've pretty much had enough. Doing anything out of the ordinary with Linux is, as others have noted, an exercise in frustration and you're mostly on your own. You either know what you are doing, have joined the inner-circle, or have to learn, and that's a major struggle. Documentation is often lacking, plain wrong, out of date, and even more damaging than useful. I imagine most people who succeed got there through working in teams where there's experience, hand-holding and mentoring available. If you're on your own; good luck, you'll need it.

Even if I do the re-build there's still more to do to get to where I want to be. With every effort to get there as equally frustrating as the rest of the journey, I can't see progress being any easier. So I think that's probably it, my adventure with this over for now. Time for a different course. I'll take another look at 'multisplit', maybe a more traditional buildroot, or back to ultibo, or something else.

Anyway, an update of where it's at -

Working

Windows rpiboot
Two virtual serial interfaces
Remote NDIS interface
Virtual COM port access to command line
Framework for app use created
Can load and run Lua, MicroPython, Python scripts
Can run TCP/IP servers which are accessible from host
Can easily coerce non-bootable Pi back to bootable mode

Not working - Tried but failed

Telnet access to Pi
MSD presentation
Access to SD Card / Non-volatile storage
Wi-Fi networking / Connecting to router
Stop screen from blanking
Changing cmdline.txt
Changing config.txt
Building rpiboot.exe for Windows

To Do

Serial from App to virtual COM Port
Serial from App to Pi UART
Pi GPIO interfacing
Writing to frame buffer
Sort out time and date settings
Sort out RNDIS IP Addressing / bridging / ICS
Document Windows RNDIS networking settings
Access for Pi to network shares
Pi sharing out a network share
Detecting MSD writes ala PiratePython
Bluetooth
Booting multiple Pi
Booting multiple Pi when USB ports unknown
IPv6

ejolson
Posts: 2008
Joined: Tue Mar 18, 2014 11:47 am

Re: Diskless booting Zeroes from Windows

Wed Oct 17, 2018 1:56 am

hippy wrote:
Mon Oct 15, 2018 12:53 pm
To be honest; I've pretty much had enough. Doing anything out of the ordinary with Linux is, as others have noted, an exercise in frustration and you're mostly on your own.
I seem to be experiencing the opposite problem: doing ordinary things with Windows.

I currently have the Raspberry Pi Zero booting Raspbian cardless with root mounted from a Windows share. I want to access the Internet from the Pi using a NAT setup provided by internet connection sharing ICS on Windows. When Raspbian boots it first requests a DHCP lease. It takes a while for Windows to recognize the new USB Ethernet gadget and serve a lease, but it does so after about 40 seconds, which is still quite nice. As soon as Raspbian gets the lease it mounts the root filesystem, pivots root and begins the usual Raspbian initialization process. At this point dhcpcd asks for the IP lease again; however, this time the built-in DHCP server that is part of Windows ICS returns a different IP number than the one it returned only seconds before. Note that the MAC address of the Ethernet gadget and hostname were the same for both requests. Fortunately, upon receiving a second IP number the Zero switches to multicast mode and now responds to both IP numbers. In particular, the network connection for the root mount is not dropped, again a good thing.

The problem with this setup is that the lease for the first IP number is not renewed and looks like it will expire in 7 days. Moreover, the lease for the second IP number is updated every 5 minutes and Windows appears to switch it before it expires for no apparent reason. While bridge mode could be used for home networks with a NAT firewall already in place, it is still the case that ICS is correct for use with computers directly connected to a cable modem or any other network that doesn't allow additional Pi Zeros to be bridged in.

Do you know how to view the currently active leases served by the built-in DHCP server in Windows ICS? Searches on the internet seem to indicate that it is not possible. Do you know how to configure the built-in DHCP server on Windows ICS so it always serves the same IP number to the same MAC address? Again there is no information available how to do this.

Return to “Advanced users”