Hothep
Posts: 6
Joined: Wed Apr 01, 2020 10:57 am

PXE-BOOT RPi4. What am i missing?

Wed Apr 01, 2020 11:28 am

So im trying to pxe-boot my RPi4 from a RB( mikrotik/routerboard) router but i cant seem to get it working. I have a server with a nfs-share and dnsmasq, but I dont want to use the dnsmasq-proxy (or dnsmasq at all), I want to use my current DHCP-server to send out whats needed. I understand this is possible on isc-dhcpd so i dont see why it should not be possible my mikrotik-router.

I started with a captured DHCP-OFFER packet from the working setup (dnsmasq-proxy and mikrotik dhcp) to see what the packet should include and i have managed to format my outgoing packet from my router to include the same information:

Code: Select all

Dynamic Host Configuration Protocol (Offer)
    Message type: Boot Reply (2)
    Hardware type: Ethernet (0x01)
    Hardware address length: 6
    Hops: 0
    Transaction ID: 0x0263d70c
    Seconds elapsed: 0
    Bootp flags: 0x0000 (Unicast)
        0... .... .... .... = Broadcast flag: Unicast
        .000 0000 0000 0000 = Reserved flags: 0x0000
    Client IP address: 0.0.0.0
    Your (client) IP address: 192.168.88.240
    Next server IP address: 192.168.88.6
    Relay agent IP address: 0.0.0.0
    Client MAC address: Raspberr_0a:ef:79 (dc:a6:32:0a:ef:79)
    Client hardware address padding: 00000000000000000000
    Server host name not given
    Boot file name not given
    Magic cookie: DHCP
    Option: (53) DHCP Message Type (Offer)
        Length: 1
        DHCP: Offer (2)
    Option: (54) DHCP Server Identifier (192.168.88.1)
        Length: 4
        DHCP Server Identifier: 192.168.88.1
    Option: (51) IP Address Lease Time
        Length: 4
        IP Address Lease Time: (600s) 10 minutes
    Option: (1) Subnet Mask (255.255.255.0)
        Length: 4
        Subnet Mask: 255.255.255.0
    Option: (3) Router
        Length: 4
        Router: 192.168.88.1
    Option: (43) Vendor-Specific Information (PXEClient)
        Length: 32
        Option 43 Suboption: (6) PXE discovery control
            Length: 1
            discovery control: 0x03, Disable Broadcast, Disable Multicast
                .... ...1 = Disable Broadcast: Set
                .... ..1. = Disable Multicast: Set
                .... .0.. = Serverlist only: Not set
                .... 0... = Bootstrap override: Not set
        Option 43 Suboption: (10) PXE menu prompt
            Length: 4
            menu prompt: 00505845
                Timeout: 0
                Prompt: PXE
        Option 43 Suboption: (9) PXE boot menu
            Length: 20
            boot menu: 00001152617370626572727920506920426f6f74
                Type: Local boot (0)
                Length: 17
                Description: Raspberry Pi Boot
        PXE Client End: 255
    Option: (60) Vendor class identifier
        Length: 9
        Vendor class identifier: PXEClient
    Option: (66) TFTP Server Name
        Length: 15
        TFTP Server Name: 192.168.88.6/24
    Option: (67) Bootfile name
        Length: 10
        Bootfile name: /tftpboot/
    Option: (255) End
        Option End: 255
From what Ive read the important bits are option 43 and option 60, but all I get is a two ARP:s for the nfs-server and a DHCP-ACK.
I have included option 97 but its not being sent out in the packet, is that the problem?
Do i need to satisfy the entire "Parameter Request List" (DHCP option 55) from the DHCP-discover?!
In my world, the RPi is not satisfied with the information in the current DHCP-OFFER and therefore will not boot.
help is greatly appreciated :)
Last edited by Hothep on Wed Apr 01, 2020 12:02 pm, edited 1 time in total.

User avatar
dickon
Posts: 1410
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: PXE-BOOT RPi4. What am i missing?

Wed Apr 01, 2020 11:40 am

Code: Select all

option option-43 code 43 = text;
option option-66 code 66 = text;

shared-network "br0" {
	subnet 172.29.23.0 netmask 255.255.255.0 {
		next-server 172.29.23.1;
		option tftp-server-name "172.29.23.1";
		host tellypi {
			hardware ethernet xx:xx:xx:xx:xx:xx;
			option root-path "telly-buster/";
			option option-43 "Raspberry Pi Boot   ";
			option option-66 "172.29.23.1";
		}
	}
}
is what I have (in relevant part) in /etc/dhcp/dhcpd.conf. You'll need a tftp server to serve the boot files: see https://www.raspberrypi.org/forums/view ... 7#p1546337 for how I manage all this.

Hothep
Posts: 6
Joined: Wed Apr 01, 2020 10:57 am

Re: PXE-BOOT RPi4. What am i missing?

Wed Apr 01, 2020 12:01 pm

My server is running tftp and i dont see how the RPi could be served bootfiles with the dnsmasq-dhcp-proxy, but not when i include (what i think is) the same options directly from the dhcp-server

I have included both option 43 (configured as the working part from the dnsmasq-proxy packet) and option 66 aswell as the "next server" option.
Both are pointing to the same ip which is the interface of the servers nfs-share (the same that worked with dnsmasq.)

https://hackaday.com/2019/11/11/network ... -the-pi-4/ Is the guide i followed to get it to work with dnsmasq but i use the "2020-02-13-raspbian-buster.img" image.

User avatar
dickon
Posts: 1410
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: PXE-BOOT RPi4. What am i missing?

Wed Apr 01, 2020 12:06 pm

Have you seen any tftp requests?

Hothep
Posts: 6
Joined: Wed Apr 01, 2020 10:57 am

Re: PXE-BOOT RPi4. What am i missing?

Wed Apr 01, 2020 12:27 pm

I do not. But im not sniffing at either endpoint (the server or the PRi), I will install a sniffer on the server to see if it can present more information.
I didnt see any TFTP-packets from my sniffing-points (a computer on the same network and the router) when I managed to get it to boot. Some unicast-thing i suppose.

User avatar
dickon
Posts: 1410
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: PXE-BOOT RPi4. What am i missing?

Wed Apr 01, 2020 12:31 pm

Yes. It's the Trivial File Transfer Protocol, which runs over UDP/IP and is extremely common for network booting (virtually everything uses it, from x86 PCs doing PXE, the Pi, U-Boot-based bootloaders, and Cisco routers, to name but a few), so you're unlikely to see anything else sniffing the network unless you've fiddled with your switch.

Check /var/log/daemon.log (or whereever you've got your tftpd logging to) for requests.

Hothep
Posts: 6
Joined: Wed Apr 01, 2020 10:57 am

Re: PXE-BOOT RPi4. What am i missing?

Wed Apr 01, 2020 1:09 pm

no tftp requests from the RPi. nothing at all, but i opened the nfs-share on my windows-machine and it shows in the log. So i take it, its is working. Same if i enter an unshared location (if the variable passed through dhcp /tftpboot/ was wrong ) it shows in the log.

rpc.mountd[1921]: authenticated mount request from 192.168.88.245:973 /nfs/raspi1 (/nfs/raspi1)

timg236
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 504
Joined: Thu Jun 21, 2018 4:30 pm

Re: PXE-BOOT RPi4. What am i missing?

Wed Apr 01, 2020 1:13 pm

You may have more luck starting with a known good server configuration, get that working before rolling your own

https://github.com/raspberrypi/piserver

The trailing spaces in the option43 might be the problem but realistically I doubt anyone is going to debug a custom dnsmasq config file, duplicate the setup and debug it.

User avatar
dickon
Posts: 1410
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: PXE-BOOT RPi4. What am i missing?

Wed Apr 01, 2020 1:15 pm

Good stuff. Anything from here on in is a userland problem -- plug a monitor into it and see what it's saying. I'd remove the 'quiet' on the commandline, too; it's a very, very annoying default.

Hothep
Posts: 6
Joined: Wed Apr 01, 2020 10:57 am

Re: PXE-BOOT RPi4. What am i missing?

Wed Apr 01, 2020 1:45 pm

timg236 wrote:
Wed Apr 01, 2020 1:13 pm
You may have more luck starting with a known good server configuration, get that working before rolling your own

https://github.com/raspberrypi/piserver

The trailing spaces in the option43 might be the problem but realistically I doubt anyone is going to debug a custom dnsmasq config file, duplicate the setup and debug it.
Yes, I was on to this issue as well, but i have not copied the config from dnsmasq (that made it boot). I copied the hex values from the output (DHCP-OFFER) of the dnsmasq-config, option 43 in the dhcp-packet, and inserted that as option 43 in my dhcp-server. The option 43 block are the same in both packets, so unless there is a reason RPi should require trailing spaces in one packet but not the other when they contain the same information, i think i have ruled that out :(
I will take a look at that piserver tho, it might very well be satisfying my needs and make this easier but ive spent so much time on this, i would VERY much like to get it to work.

dickon wrote: Good stuff. Anything from here on in is a userland problem -- plug a monitor into it and see what it's saying. I'd remove the 'quiet' on the commandline, too; it's a very, very annoying default.
A monitor to the RPi?
I get nothing, no signal at all. should i?
I was thinking it wont post anything because there is no driver or anything that can produce display output before it has downloaded the boot-files from the server. I get output and its working when it booted from the dnsmasq-setup.

timg236
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 504
Joined: Thu Jun 21, 2018 4:30 pm

Re: PXE-BOOT RPi4. What am i missing?

Wed Apr 01, 2020 1:52 pm

The latest beta 2020-03-19 does a sub-string match and will ignore trailing spaces but IIRC older versions may have required an exact match

User avatar
dickon
Posts: 1410
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: PXE-BOOT RPi4. What am i missing?

Wed Apr 01, 2020 2:04 pm

Hothep wrote:
Wed Apr 01, 2020 1:45 pm
dickon wrote: Good stuff. Anything from here on in is a userland problem -- plug a monitor into it and see what it's saying. I'd remove the 'quiet' on the commandline, too; it's a very, very annoying default.
A monitor to the RPi?
I get nothing, no signal at all. should i?
I was thinking it wont post anything because there is no driver or anything that can produce display output before it has downloaded the boot-files from the server. I get output and its working when it booted from the dnsmasq-setup.
If you've got as far as an NFS mount, then yes, you should. You should see no signal until it's loaded start4.elf, then the usual rainbow while the kernel (and any initrd) are obtained, then a kernel boot as usual.
timg236 wrote:
Wed Apr 01, 2020 1:52 pm
The latest beta 2020-03-19 does a sub-string match and will ignore trailing spaces but IIRC older versions may have required an exact match
That's been booting my Pi 4 since I flashed the first network boot ROM to it, back when you announced it. I just copied the stanza from an existing Pi 1-3+ version, and it worked.

Hothep
Posts: 6
Joined: Wed Apr 01, 2020 10:57 am

Re: PXE-BOOT RPi4. What am i missing?

Thu Apr 02, 2020 12:40 am

There is no NFS-mount happening. I can mount the NFS-share from my windows machine to make sure it´s in order but the RPi does not even request a mount.

Im not sure why the trailing spaces could be the issue considering the circumstances. IOW the RPi PXE-boots when i use dnsmasq to send the DHCP-information but not with a DHCP packet containing the same information sent from my router. Since im not even getting a mount-request im suspecting there is something wrong with the initial parts of the process. Correct me if im wrong but I think this process starts with a DHCP-offer containing information satisfying some not-very-pxe-like requirements the RPi has. for example the "Raspberry Pi Boot ". If i dont get the DHCP-offer correct it cant even attempt to boot. and its not

HiassofT
Posts: 282
Joined: Fri Jun 30, 2017 10:07 pm
Location: Salzburg, Austria
Contact: Website

Re: PXE-BOOT RPi4. What am i missing?

Thu Apr 02, 2020 9:15 am

The contents of option 66 (tftp server name) look odd:

Code: Select all

    Option: (66) TFTP Server Name
        Length: 15
        TFTP Server Name: 192.168.88.6/24
If present this should contain the IP address of the dhcp server (eg 192.168.88.6, without the /24 at the end). IIRC DNS isn't implemented in the firmware so we can't use a name here (as in RFC).

Some time ago timg236 told me this:
The TFTP server ip can come from

1) The EEPROM config - takes precedence
2) DHCP option 66 (option 67 to specify the filename isn't used because there isn't a single file-name for boot)
3) If option 43 is specified for PXE "Raspberry Pi boot" then DHCP server id from the offer is used (option 54)
I'm successfully netbooting my RPi4s with just option 66 set (i.e. not using 43) as I have the tftp server not on my router (which is providing DHCP, DNS and NTP only).

When comparing working/non-working packets also look at option 54 - that could take precedence over option 66.

so long,

Hias

User avatar
dickon
Posts: 1410
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: PXE-BOOT RPi4. What am i missing?

Thu Apr 02, 2020 10:35 am

Hothep wrote:
Thu Apr 02, 2020 12:40 am
There is no NFS-mount happening. I can mount the NFS-share from my windows machine to make sure it´s in order but the RPi does not even request a mount.
My mistake. I misparsed your earlier post.
Im not sure why the trailing spaces could be the issue considering the circumstances. IOW the RPi PXE-boots when i use dnsmasq to send the DHCP-information but not with a DHCP packet containing the same information sent from my router. Since im not even getting a mount-request im suspecting there is something wrong with the initial parts of the process. Correct me if im wrong but I think this process starts with a DHCP-offer containing information satisfying some not-very-pxe-like requirements the RPi has. for example the "Raspberry Pi Boot ". If i dont get the DHCP-offer correct it cant even attempt to boot. and its not
The Pi does not, strictly speaking, PXE boot. That's an Intel term for a peculiar ethernet card ROM, DHCP / TFTP first-stage bootstrap, and set of BIOS services to provide an API to boot DOS. Since the '90s, DOS has been replaced by pxelinux.0 or the like.

The Pi network boots. It needs a pretty unremarkable DHCP response, and loads $serial/bootcode.bin (3+) or $serial/start4.elf (4(+?)), which then reads the rest of the usual config files as if they were on the SD card. Without logs from the tftpd and dhcpd, you're probably doomed at this point.

The fragment of /etc/dhcp/dhcpd.conf I posted above works well enough for me. Otherwise, stick to the supported mechanisms for doing this until you can sort the logging out on your servers.
HiassofT wrote:
Thu Apr 02, 2020 9:15 am
The contents of option 66 (tftp server name) look odd:

Code: Select all

    Option: (66) TFTP Server Name
        Length: 15
        TFTP Server Name: 192.168.88.6/24
Well spotted. That's broken.

Return to “Advanced users”