XAPBob
Posts: 91
Joined: Tue Jan 03, 2012 2:40 pm

Network booting the Pi3...

Wed Sep 07, 2016 10:17 am

Over the summer break I had a PiB and a Pi3 with me, and I got the Pi2 working as a DHCP Proxy to boot the Pi3.

Brining them home I have a somewhat different network...
I have a Sophos UTM as my main network device (firewall/router/DNS/DHCP/NTP) but it doesn't have a TFTPd server, or an NFS server.

Aha, I have a PiZero - maybe I can connect it via USB???
Turns out - yes I can! :)

So now I have a network boot setup, so that I can boot and install debian across the network. Excellent.

I added an NFS server, and replicated the new file system onto the PiZero. That works - I can connect to that filesystem from my laptop, easy as pi :roll: sorry...


So I know that netbooting is possible on the network (because I already use it) So my DHCP 'next-server' and 'filename' (pxelinux.0) options are good for NON Pi boots.
And I know that the NFS gear should work...

My guess therefore is that there is something in my normal DHCP configuration which isn't working for the Pi - does anyone have a working full dhcp pi booting config I could see?

I'm hoping I don't need to either configure every mac address I have into either 'x86' or 'Pi' groups, and that I don't have to choose between PXE and Pi booting...

XAPBob
Posts: 91
Joined: Tue Jan 03, 2012 2:40 pm

Re: Network booting the Pi3...

Wed Sep 07, 2016 10:51 am

OK - dug a little deeper, thanks to a comment on the Announcement page linking me to this site:
http://www.do-it-neat.com/opensuse-leap ... ot-teil-1/

Which suggests Option 43 as a required field.
Sophos wants it in colon separated hex, but that's doable (52:61:73:70:62:65:72:72:79:20:50:69:20:42:6f:6f:74:20:20:20)

The Pi just triggered an avalanche of TFTP traffic, no NFS yet though (I'll need to check the IP address in the cmdline.txt file)

Also had to set up a NAT rule - looks like the Pi assumes that the TFTP server resides on the IP from which DHCPd responded.


Avalanche:
192.168.7.0/24 is the network where the Pi is booting, 192.168.70.2 is the PiZero
This is the packet capture from the firewall - both interfaces, limited to ports 67,68,69,111,2049

Code: Select all

11:54:13.189631 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from b8:27:eb:3b:76:09, length 320
11:54:14.191069 IP 192.168.7.1.67 > 192.168.7.169.68: BOOTP/DHCP, Reply, length 300
11:54:14.191357 IP 192.168.7.169.49152 > 192.168.7.1.69:  21 RRQ "bootcode.bin" octet 
11:54:14.191632 IP 192.168.7.169.49152 > 192.168.70.2.69:  21 RRQ "bootcode.bin" octet
11:54:14.269830 IP 192.168.7.169.49152 > 192.168.7.1.69:  20 RRQ "bootsig.bin" octet
11:54:14.269946 IP 192.168.7.169.49152 > 192.168.70.2.69:  20 RRQ "bootsig.bin" octet
11:54:14.379052 IP 192.168.7.169.49153 > 192.168.7.1.69:  27 RRQ "5f3b7609/start.elf" octet 
11:54:14.379310 IP 192.168.7.169.49153 > 192.168.70.2.69:  27 RRQ "5f3b7609/start.elf" octet
11:54:14.403065 IP 192.168.7.169.49154 > 192.168.7.1.69:  21 RRQ "autoboot.txt" octet 
11:54:14.403224 IP 192.168.7.169.49154 > 192.168.70.2.69:  21 RRQ "autoboot.txt" octet
11:54:14.409678 IP 192.168.7.169.49155 > 192.168.7.1.69:  19 RRQ "config.txt" octet
11:54:14.409899 IP 192.168.7.169.49155 > 192.168.70.2.69:  19 RRQ "config.txt" octet
11:54:14.424430 IP 192.168.7.169.49156 > 192.168.7.1.69:  21 RRQ "recovery.elf" octet
11:54:14.424590 IP 192.168.7.169.49156 > 192.168.70.2.69:  21 RRQ "recovery.elf" octet
11:54:14.431018 IP 192.168.7.169.49157 > 192.168.7.1.69:  18 RRQ "start.elf" octet 
11:54:14.431171 IP 192.168.7.169.49157 > 192.168.70.2.69:  18 RRQ "start.elf" octet
11:54:15.548565 IP 192.168.7.169.49158 > 192.168.7.1.69:  9 RRQ "" octet
11:54:15.548973 IP 192.168.7.169.49158 > 192.168.70.2.69:  9 RRQ "" octet

The tail end of the capture from the PiZero:

Code: Select all

10:54:14.426724 IP 192.168.7.169.49157 > 192.168.70.2.tftp:  18 RRQ "start.elf" octet
10:54:14.433901 IP 192.168.70.2.48180 > 192.168.7.169.49157: UDP, length 516
10:54:14.434588 IP 192.168.7.169.49157 > 192.168.70.2.48180: UDP, length 4
.......<two lines repeat>.......
10:54:14.543418 IP 192.168.7.169.49157 > 192.168.70.2.48180: UDP, length 4
10:54:14.543551 IP 192.168.70.2.48180 > 192.168.7.169.49157: UDP, length 516
10:54:15.544530 IP 192.168.7.169.49158 > 192.168.70.2.tftp:  9 RRQ "" octet  
10:54:15.545554 IP 192.168.70.2.48180 > 192.168.7.169.49157: UDP, length 516
10:54:15.549100 IP 192.168.70.2.42605 > 192.168.7.169.49158: UDP, length 17
10:54:17.333277 IP 192.168.70.2.ntp > 192.168.7.1.ntp: NTPv4, Client, length 48
10:54:17.333892 IP 192.168.7.1.ntp > 192.168.70.2.ntp: NTPv4, Server, length 48
10:54:17.547766 IP 192.168.70.2.48180 > 192.168.7.169.49157: UDP, length 516
10:54:21.552031 IP 192.168.70.2.48180 > 192.168.7.169.49157: UDP, length 516
10:54:29.560344 IP 192.168.70.2.48180 > 192.168.7.169.49157: UDP, length 516
10:54:32.559417 IP 192.168.70.1 > 192.168.70.2: ICMP host 192.168.7.169 unreachable, length 552
10:54:45.576600 IP 192.168.70.2.48180 > 192.168.7.169.49157: UDP, length 516
10:54:48.575383 IP 192.168.70.1 > 192.168.70.2: ICMP host 192.168.7.169 unreachable, length 552
Mind you - the ACT led is still blinking... short, short, short, long.
Repeat...

Suggesting either
3 flashes: start.elf not found
4 flashes: start.elf not launch-able (corrupt)

User avatar
DougieLawson
Posts: 36322
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
Contact: Website Twitter

Re: Network booting the Pi3...

Wed Sep 07, 2016 5:15 pm

I got this working but not quite using the published instructions. I did it my own way so that I can use the USB MSD that I'd already got booting on my RPi3B as the network root drive for my RPi3B when it boots using PXE.

I started with an SDCard and ran the magic boot to update the OTP flags.
Got the RPi3 booting from a USB stick, by following the published instructions.
https://www.raspberrypi.org/documentati ... des/msd.md

For network booting I've pulled that USB stick out of my RPi3B and mounted it in my RPi2B. I've mounted that with this /etc/fstab

Code: Select all

proc            /proc           proc    defaults          0       0
/dev/mmcblk0p1  /boot           vfat    defaults          0       2
/dev/mmcblk0p2  /               ext4    defaults,noatime  0       1
# Mount the network boot USB stick
/dev/sda2 /tftp ext4 defaults,noatime 0 1
# Mount /boot as /tftp/boot so that it gets shared out by dnsmasq
/dev/sda1 /tftp/boot vfat defaults 0 0
# a swapfile is not a swap partition, no line here
#   use  dphys-swapfile swap[on|off]  for that
the trick here is to mount the ext4 filesystem on /tftp then mount the fat one on top of that at /tftp/boot.

Updated the /etc/dhcpcd.conf to give the RPi2B a fixed IP address (that's outside my normal network)

Code: Select all

interface eth0
static ip_address=10.1.2.99/24
#static routers=192.168.1.138
#static domain_name_servers=192.168.1.11 8.8.8.8
Updated /tftp/boot/cmdline.txt with

Code: Select all

dwc_otg.lpm_enable=0 console=tty1 root=/dev/nfs nfsroot=10.1.2.99:/tftp elevator=deadline rootwait ip=dhcp rw
note the matching IP address for my RPi2B. This one will get a DHCP address assigned by dnsmasq on the RPi2B. That can be done after you've shutdown the USB booted RPi3B because the /tftp filesystem is mounted read/write on my RPi2B.

Install dnsmasq on the RPI2B and configure it with

Code: Select all

port=0
dhcp-range=10.1.2.10,10.1.2.40,300h
log-dhcp
enable-tftp
tftp-root=/tftp/boot
pxe-service=0,"Raspberry Pi Boot"
the trick here is using the /boot from the USB stick as the root for the tftp server.

Install nfs-kernel-server and configure it with

Code: Select all

/tftp *(rw,sync,no_subtree_check,no_root_squash)
that allows the network booted system to use the network filesystem.

Disconnect everything from your regular network, connect RPi2B (10.1.2.99) to your RPi3B with a cable (or in my case with a five port switch). Boot the RPi2B. Boot the RPi3B see the rainbow screen appear, see the desktop appear and grab a beer to celebrate a job well done.

One thing that also gives me is I can logon to my RPi2B from my laptop, just by plugging it into the 5-port switch. dnsmasq on the RPi2B hands out a second DHCP address 10.1.2.xx/24 to my laptop and I know the RPi2B has got a fixed IP address 10.1.2.99.

Once I can logon to my RPi2B I can look at the dnsmasq logs to determine the address assigned for my RPI3B and can log in to it. My RPi3B as a touch screen, anyway but this gives a headless option.

It also doesn't need any mucking about with tcpdump (I did that first time I tried the network boot, but didn't need it for this configuration). Because the tftp fileserver is the same RPi2B as NFS server it has a single fixed IP address and is fully self-contained.

If anyone is in Winchester next week at the Roebuck PH on Wed @ 20:00 I'll be showing a live demo.
Note: Having anything humorous in your signature is completely banned on this forum. Wear a tin-foil hat and you'll get a ban.

Any DMs sent on Twitter will be answered next month.

This is a doctor free zone.

XAPBob
Posts: 91
Joined: Tue Jan 03, 2012 2:40 pm

Re: Network booting the Pi3...

Wed Sep 07, 2016 6:06 pm

Yeah - managed it with dnsmasq - but I have a tftpd-hpa on my home network for other systems...

If I kill the x86 netboot code on the DHCP server and put the proxy dhcp server in place then it boots correctly... Very perturbing...

mikerr
Posts: 2782
Joined: Thu Jan 12, 2012 12:46 pm
Location: UK
Contact: Website

Re: Network booting the Pi3...

Sat Sep 17, 2016 11:43 am

XAPBob wrote: looks like the Pi assumes that the TFTP server resides on the IP from which DHCPd responded.
specify option 66 to point to your tftp server IP,
otherwise pi will assume it's the same as DHCP server.
Android app - Raspi Card Imager - download and image SD cards - No PC required !

XAPBob
Posts: 91
Joined: Tue Jan 03, 2012 2:40 pm

Re: Network booting the Pi3...

Mon Sep 19, 2016 11:27 am

sname is already specified, but I added option 66 as well...

No difference in behaviour unfortunately.
I might just have to bite the bullet and virtualise the UTM :(

FM81
Posts: 518
Joined: Wed Apr 17, 2013 4:33 pm

Re: Network booting the Pi3...

Mon Sep 19, 2016 2:07 pm

Have you (for testing purposes only) inserted a sd-card with only latest bootcode.bin on it?
(If it works in this case, your RPi3 doesn't seem to work together with your switch/hub/router.)
See also here: viewtopic.php?f=29&t=156830

Greetings, FM_81
A: What does the command 'cat /dev/urandom', can you tell me please?
B: Yeah, that's very simple: It feeds your cat with radioactive material!

Return to “Troubleshooting”