network booting with TFTP server IP != DHCP server IP

[whocares_ido]

Hello,

I am currently struggling with an advanced network booting setup. Basic network booting works for me.

I understand that the Pi does not follow the usual PXE/DHCP rules. It took me a while to figure out that it ignores the filename statement. I do not like it but there are workarounds so this is not an issue.

My DHCP server (dhcpd) specifies a next-server (see log below) that contains the IP address of the TFTP server (192.168.160.22). This works fine for regular x86 PXE clients. The Pi does not take this into account and tries to fetch the files via TFTP from the IP that is running the DHCP server (192.168.161.1) but not from the real TFTP server (192.168.160.22).

Is there a way to tell the Pi to use a different TFTP server IP? Which DHCP option do I have to specify?

Thank you for your help!

My setup:
eth0 of Pi3 with bootcode.bin on SD card (IP 192.168.161.23 in the log below) <-> eth1 of DHCP server (IP: 192.168.161.1)

eth0 of DHCP server box (IP: 192.168.160.?) <-> tftp server (IP: 192.168.160.22)

Code: Select all

17:51:45.622864 IP (tos 0x0, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 348)
    0.0.0.0.68 > 255.255.255.255.67: [no cksum] BOOTP/DHCP, Request from b8:27:eb:ab:cd:ef, length 320, xid 0x26f30339, Flags [none] (0x0000)
	  Client-Ethernet-Address b8:27:eb:ab:cd:ef
	  Vendor-rfc1048 Extensions
	    Magic Cookie 0x63825363
	    DHCP-Message Option 53, length 1: Discover
	    Parameter-Request Option 55, length 12: 
	      Vendor-Option, Vendor-Class, BF, Option 128
	      Option 129, Option 130, Option 131, Option 132
	      Option 133, Option 134, Option 135, TFTP
	    ARCH Option 93, length 2: 0
	    NDI Option 94, length 3: 1.2.1
	    GUID Option 97, length 17: 0.109.228.137.45.109.228.137.45.109.228.137.45.109.228.137.45
	    Vendor-Class Option 60, length 32: "PXEClient:Arch:00000:UNDI:002001"
	    END Option 255, length 0
17:51:45.632253 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.161.23 tell 192.168.161.1, length 28
17:51:46.631645 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.161.23 tell 192.168.161.1, length 28
17:51:46.634642 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
    192.168.161.1.67 > 192.168.161.23.68: [udp sum ok] BOOTP/DHCP, Reply, length 300, xid 0x26f30339, Flags [none] (0x0000)
	  Your-IP 192.168.161.23
	  Server-IP 192.168.160.22
	  Client-Ethernet-Address b8:27:eb:ab:cd:ef
	  file "dummy"
	  Vendor-rfc1048 Extensions
	    Magic Cookie 0x63825363
	    DHCP-Message Option 53, length 1: Offer
	    Server-ID Option 54, length 4: 192.168.161.1
	    Lease-Time Option 51, length 4: 600
	    Vendor-Option Option 43, length 20: 82.97.115.112.98.101.114.114.121.32.80.105.32.66.111.111.116.32.32.32
	    Subnet-Mask Option 1, length 4: 255.255.255.0
	    END Option 255, length 0
	    PAD Option 0, length 0, occurs 16
17:51:46.634832 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.161.1 tell 192.168.161.23, length 46
17:51:46.634951 IP (tos 0x0, ttl 64, id 65181, offset 0, flags [DF], proto ICMP (1), length 48)
    192.168.161.1 > 192.168.161.23: ICMP echo request, id 25814, seq 0, length 28
17:51:46.635192 ARP, Ethernet (len 6), IPv4 (len 4), Reply 192.168.161.1 is-at 00:00:e8:00:00:01, length 28
17:51:46.635348 IP (tos 0x0, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 55)
    192.168.161.23.49153 > 192.168.161.1.69: [no cksum]  27 RRQ "12345678/start.elf" octet
17:51:46.635618 IP (tos 0xc0, ttl 64, id 65279, offset 0, flags [none], proto ICMP (1), length 83)
    192.168.161.1 > 192.168.161.23: ICMP 192.168.161.1 udp port 69 unreachable, length 63
	IP (tos 0x0, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 55)
    192.168.161.23.49153 > 192.168.161.1.69: [no cksum]  27 RRQ "12345678/start.elf" octet

[whocares_ido]

It looks to me like this is broken in the current implementation of bootcode.bin and there is currently no way to make this work with a TFTP server in a different subnet.

The DHCP discover packet sent by the Pi includes the parameter request list (option 55). This list does not contain the paremeter request list item 3 (router). That is why the information about the default gateway is not included in the DHCP offer packet sent by the DHCP server. It was not requested, so it is not provided.

Without the information about the default gateway, it is not possible to contact a TFTP server in a different subnet.

It is a pity that this does not work.

[mfa298]

With the isc-dhcp-server you can update the parameter list to get the dhcp server to send back other options even if not requested, although whether the client will act on them is a different matter.

With the ISC dhcp server I did hack the config in the past where the client assumed it had to get the boot files from the dhcp server rather than a separate tftp server. I did this by abusing the server id parameter and setting it to the tftp server rather than the dhcp server. If the Pi isn't requesting/using the router address then this may not help if the tftp server is on a different subnet but it might at least help if the tftp server is on the same network.

[mikerr]

Use option 66 to specify tftp server ?

[whocares_ido]

Thank you both for the quick feedback.

forcing ISC DHCP server to provide the option routers is worth trying although I do not expect the Pi to use the information about the default gateway if it is not requesting it. Will try and get back to you.

option 66 can indeed be used to specify a different tftp server. The problem is that the PI assumes that this tftp server can be accessed without going through the gateway (I do see an ARP request for the IP address of the TFTP server although it is not on the same subnet).

[whocares_ido]

Even if I force the DHCP server to provide information about the default gateway, the pi does not try to go through the gateway. It assumes that the TFTP server can be reached without going through the gateway although it is not in the same subnet.

So this looks like a real bug of bootcode.bin to me. Do you agree?

Code: Select all

21:22:40.960177 IP (tos 0x0, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 348)
    0.0.0.0.68 > 255.255.255.255.67: [no cksum] BOOTP/DHCP, Request from b8:27:eb:ab:cd:ef, length 320, xid 0x26f30339, Flags [none] (0x0000)
          Client-Ethernet-Address b8:27:eb:ab:cd:ef
          Vendor-rfc1048 Extensions
            Magic Cookie 0x63825363
            DHCP-Message Option 53, length 1: Discover
            Parameter-Request Option 55, length 12:
              Vendor-Option, Vendor-Class, BF, Option 128
              Option 129, Option 130, Option 131, Option 132
              Option 133, Option 134, Option 135, TFTP
            ARCH Option 93, length 2: 0
            NDI Option 94, length 3: 1.2.1
            GUID Option 97, length 17: 0.109.228.137.45.109.228.137.45.109.228.137.45.109.228.137.45
            Vendor-Class Option 60, length 32: "PXEClient:Arch:00000:UNDI:002001"
            END Option 255, length 0
21:22:40.967011 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.161.23 tell 192.168.161.1, length 28
21:22:41.957742 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.161.23 tell 192.168.161.1, length 28
21:22:41.969236 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 334)
    192.168.161.1.67 > 192.168.161.23.68: [udp sum ok] BOOTP/DHCP, Reply, length 306, xid 0x26f30339, Flags [none] (0x0000)
          Your-IP 192.168.161.23
          Server-IP 192.168.160.22
          Client-Ethernet-Address b8:27:eb:ab:cd:ef
          file "bios/lpxelinux.0"
          Vendor-rfc1048 Extensions
            Magic Cookie 0x63825363
            DHCP-Message Option 53, length 1: Offer
            Server-ID Option 54, length 4: 192.168.161.1
            Lease-Time Option 51, length 4: 600
            Vendor-Option Option 43, length 20: 82.97.115.112.98.101.114.114.121.32.80.105.32.66.111.111.116.32.32.32
            TFTP Option 66, length 14: "192.168.160.22"
            Subnet-Mask Option 1, length 4: 255.255.255.0
            Default-Gateway Option 3, length 4: 192.168.161.1
            END Option 255, length 0
21:22:41.969443 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.160.22 tell 192.168.161.23, length 46
21:22:41.969587 IP (tos 0x0, ttl 64, id 25496, offset 0, flags [DF], proto ICMP (1), length 48)
    192.168.161.1 > 192.168.161.23: ICMP echo request, id 24723, seq 0, length 28
21:22:46.977727 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.161.23 tell 192.168.161.1, length 28
21:22:47.977733 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.161.23 tell 192.168.161.1, length 28
21:22:48.977771 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.161.23 tell 192.168.161.1, length 28

I have also noticed that the Pi does not even send the DHCPREQUEST packet and therefore does not comply with the DHCP protocol.

[antonio.c.mariani]

It really is a pity the tftp server could not be on a different subnet. Indeed it no seems to a problem with the bootcode.bin, but with the PXE boot code that comes with the Raspberry. Floris talk about that at https://www.raspberrypi.org/blog/piserv ... nt-1375110.

This is not a common behavior of the PXE boot code. Anyone knows if this problem/bug was fixed on Raspberry Pi 3 model B+?