fd_
Posts: 66
Joined: Thu Oct 25, 2018 7:35 am

Setting up network bridge under Buster

Mon Jul 15, 2019 4:16 pm

I'm trying to set up a bridge between two wired network interfaces, eth0, which is connected to a local network, and usb0, which is a device I'd like to provide with an Internet connection via the bridge. In addition to creating the bridge, I want the Raspberry Pi to remain accessible from the network.

Under Debian Stretch, I used to achieve that by getting eth0's IP address, setting eth0 to 0.0.0.0, then creating the bridge, adding the two interfaces and assigning the IP address from step 1 to the bridge:

Code: Select all

ifconfig eth0 0.0.0.0 down
brctl addbr bridge_usb
brctl addif bridge_usb eth0
ifconfig eth0 0.0.0.0 up
ifconfig bridge_usb up
# Point 1
ifconfig bridge_usb 0.0.0.0 down
ifconfig usb0 0.0.0.0 down
brctl addif bridge_usb usb0
ifconfig usb0 up
ifconfig bridge_usb up
ifconfig bridge_usb $PUBLIC_IP
(Please note a few steps might be redundant here, I extracted these from a more complex script)

However, under Buster, I cannot get it to work. The same steps correctly set up the bridge and assign it the IP address, but there are multiple problems:
1) As soon as eth0 is contained in the bridge (Point 1 above), the Pi itself loses Internet access (eg ping doesn't work anymore)
2) It seems the eth0 interface automatically restores its IP address to $PUBLIC_IP after a few seconds, even when it is contained in the bridge
3) I'm unable to access the Pi via sshing to $PUBLIC_IP, and I assume that's because there are now 2 interfaces with $PUBLIC_IP, one of which is in the bridge
4) If I don't manually assign an address to the bridge interface, it seems that the bridge interface bridge_usb receives an IP address from my local network's DHCP server. This behavior seems to differ from Stretch.

Does anyone have any idea what could be the reason for the difference in behaviour from Stretch to Buster? What do I need to do to restore Stretch behavior for network bridges?

epoch1970
Posts: 3887
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: Setting up network bridge under Buster

Mon Jul 15, 2019 5:12 pm

Little should have changed between Stretch and Buster regarding networking.
The second part (bridged config) of the official access point documentation shows how to setup a bridge.
The configuration in question defines an IP address for the bridge obtained via DHCP by dhcpcd. This should fit your need I believe, so basically the only departure from the documentation would be for you to replace wlan0 with usb0.
Editing both /etc/dhcpcd.conf and /etc/network/interfaces is necessary.
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

fd_
Posts: 66
Joined: Thu Oct 25, 2018 7:35 am

Re: Setting up network bridge under Buster

Mon Jul 15, 2019 6:54 pm

epoch1970 wrote:
Mon Jul 15, 2019 5:12 pm
Little should have changed between Stretch and Buster regarding networking.
The second part (bridged config) of the official access point documentation shows how to setup a bridge.
The configuration in question defines an IP address for the bridge obtained via DHCP by dhcpcd. This should fit your need I believe, so basically the only departure from the documentation would be for you to replace wlan0 with usb0.
Editing both /etc/dhcpcd.conf and /etc/network/interfaces is necessary.
Thanks for your input!

Well, I'd like to configure the bridge manually (without modifying configuration files), so that I can add new devices when they are attached. Is that possible with the configuration files? And also, is the bridge accessible from outside with that setup?

Anyway, the same manual setup I described above worked under Stretch, so it must be some change in Buster.

epoch1970
Posts: 3887
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: Setting up network bridge under Buster

Mon Jul 15, 2019 9:23 pm

No, no. There is (was) something in your Stretch install that made the script work, but seeing eth0 come back from the dead and being reassigned an IP and routes is typical dhcpcd behavior, and that has not changed between Stretch and Buster.
  • There is the old way of configuring networking under Debian, "ifupdown", with the /etc/network/interfaces file. This design is completely imperative and stateful. You can only ifup or ifdown what's in /etc/network/interfaces; You can only "ifdown" what was previously "ifup"-ed. Even a simple manual "ifconfig up" (or down) can throw a wrench in ifupdown's plans... Ifupdown knows only about itself.
    Consider that when ifupdown was designed, USB network interfaces, wifi, virtual machines barely existed (if at all)
  • There is the current way of configuring network, which is event-driven. Systemd-networkd does this, but with Raspbian, dhcpcd5 is used instead. They share the same design: when a kernel event is fired regarding an interface (cable unplugged, interface up...), dhcpcd reads its config and configures or deconfigures the interface.
    Dhcpcd itself isn't stateful, it reacts faithfully to what the kernel says, and so sticks much better to the reality of the system.
  • There is a downside to the event-driven approach: it works better for simple devices. Complex hierarchical devices like a bridge are not handled by dhcpcd. Dhcpcd can fetch the IP address and routes of a bridge interface, but that's all.
  • The 2 designs are incompatible by nature. They can be made to live side-by-side, barely.
You want to configure a bridge essentially manually.
  • The first thing is to "denyinterfaces" every potential bridge member interface in dhcpcd.conf, or disable the dhcpcd service altogether. If you don't do that, dhcpcd will mess with your network config every time an interface or cable is added/removed.
  • Once you've done that, you can go the fully manual (scripted) way, or use /etc/network/interfaces which offers a nice framework. Of course you can also mix approaches: start with ifup, modify the bridge manually, undo your mods, run ifdown. Ifupdown doesn't know, it doesn't care...
Thanks for reading all this (I did review and cut some text!)
If you still can't believe nothing has changed between Stretch and Buster, find back your Stretch install and look if dhcpcd was activated or configured in a special way...
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

fd_
Posts: 66
Joined: Thu Oct 25, 2018 7:35 am

Re: Setting up network bridge under Buster

Thu Jul 18, 2019 10:37 am

epoch1970 wrote:
Mon Jul 15, 2019 9:23 pm
No, no. There is (was) something in your Stretch install that made the script work, but seeing eth0 come back from the dead and being reassigned an IP and routes is typical dhcpcd behavior, and that has not changed between Stretch and Buster.
  • There is the old way of configuring networking under Debian, "ifupdown", with the /etc/network/interfaces file. This design is completely imperative and stateful. You can only ifup or ifdown what's in /etc/network/interfaces; You can only "ifdown" what was previously "ifup"-ed. Even a simple manual "ifconfig up" (or down) can throw a wrench in ifupdown's plans... Ifupdown knows only about itself.
    Consider that when ifupdown was designed, USB network interfaces, wifi, virtual machines barely existed (if at all)
  • There is the current way of configuring network, which is event-driven. Systemd-networkd does this, but with Raspbian, dhcpcd5 is used instead. They share the same design: when a kernel event is fired regarding an interface (cable unplugged, interface up...), dhcpcd reads its config and configures or deconfigures the interface.
    Dhcpcd itself isn't stateful, it reacts faithfully to what the kernel says, and so sticks much better to the reality of the system.
  • There is a downside to the event-driven approach: it works better for simple devices. Complex hierarchical devices like a bridge are not handled by dhcpcd. Dhcpcd can fetch the IP address and routes of a bridge interface, but that's all.
  • The 2 designs are incompatible by nature. They can be made to live side-by-side, barely.
You want to configure a bridge essentially manually.
  • The first thing is to "denyinterfaces" every potential bridge member interface in dhcpcd.conf, or disable the dhcpcd service altogether. If you don't do that, dhcpcd will mess with your network config every time an interface or cable is added/removed.
  • Once you've done that, you can go the fully manual (scripted) way, or use /etc/network/interfaces which offers a nice framework. Of course you can also mix approaches: start with ifup, modify the bridge manually, undo your mods, run ifdown. Ifupdown doesn't know, it doesn't care...
Thanks for reading all this (I did review and cut some text!)
If you still can't believe nothing has changed between Stretch and Buster, find back your Stretch install and look if dhcpcd was activated or configured in a special way...
Thanks for your details explanation!

The dhcpcd service is running under the Stretch image, just as it is on the Buster image:

Code: Select all

Stretch:
[email protected](rw):~$ systemctl status dhcpcd
● dhcpcd.service - dhcpcd on all interfaces
   Loaded: loaded (/lib/systemd/system/dhcpcd.service; enabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/dhcpcd.service.d
           └─wait.conf
   Active: active (running) since Mon 2019-07-15 23:17:28 CEST; 2 days ago
  Process: 251 ExecStart=/usr/lib/dhcpcd5/dhcpcd -q -w (code=exited, status=0/SUCCESS)
 Main PID: 936 (dhcpcd)
   CGroup: /system.slice/dhcpcd.service
           └─936 /sbin/dhcpcd -q -w

Jul 15 23:17:22 zero dhcpcd[251]: eth0: probing address 192.168.178.133/24
Jul 15 23:17:27 zero dhcpcd[251]: eth0: leased 192.168.178.133 for 864000 seconds
Jul 15 23:17:27 zero dhcpcd[251]: eth0: adding route to 192.168.178.0/24
Jul 15 23:17:27 zero dhcpcd[251]: eth0: adding default route via 192.168.178.1
Jul 15 23:17:28 zero dhcpcd[251]: Too few arguments.
Jul 15 23:17:28 zero dhcpcd[251]: /sbin/resolvconf: 88: /lib/resolvconf/unbound: cannot create /var/cache/unbound/resolvconf_resolvers.conf: Read-only file system
Jul 15 23:17:28 zero dhcpcd[251]: Too few arguments.
Jul 15 23:17:28 zero dhcpcd[251]: forked to background, child pid 936
Jul 15 23:17:28 zero systemd[1]: Started dhcpcd on all interfaces.
Jul 15 23:17:32 zero dhcpcd[936]: eth0: no IPv6 Routers available
Warning: dhcpcd.service changed on disk. Run 'systemctl daemon-reload' to reload units.

Code: Select all

Buster:
[email protected]:~ $ systemctl status dhcpcd
Warning: The unit file, source configuration file or drop-ins of dhcpcd.service changed on disk. Run 'systemctl daemon-reload' to reload units.
● dhcpcd.service - dhcpcd on all interfaces
   Loaded: loaded (/lib/systemd/system/dhcpcd.service; enabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/dhcpcd.service.d
           └─wait.conf
   Active: active (running) since Thu 2019-07-18 11:13:29 BST; 20min ago
  Process: 251 ExecStart=/usr/lib/dhcpcd5/dhcpcd -q -w (code=exited, status=0/SUCCESS)
 Main PID: 360 (dhcpcd)
   Memory: 1.1M
   CGroup: /system.slice/dhcpcd.service
           └─360 /sbin/dhcpcd -q -w

Jul 18 11:13:23 raspberrypi dhcpcd[251]: eth0: adding address fe80::3982:de65:9981:bb07
Jul 18 11:13:23 raspberrypi dhcpcd[251]: eth0: rebinding lease of 192.168.178.133
Jul 18 11:13:23 raspberrypi dhcpcd[251]: eth0: soliciting an IPv6 router
Jul 18 11:13:23 raspberrypi dhcpcd[251]: eth0: probing address 192.168.178.133/24
Jul 18 11:13:29 raspberrypi dhcpcd[251]: eth0: leased 192.168.178.133 for 864000 seconds
Jul 18 11:13:29 raspberrypi dhcpcd[251]: eth0: adding route to 192.168.178.0/24
Jul 18 11:13:29 raspberrypi dhcpcd[251]: eth0: adding default route via 192.168.178.1
Jul 18 11:13:29 raspberrypi dhcpcd[251]: forked to background, child pid 360
Jul 18 11:13:29 raspberrypi systemd[1]: Started dhcpcd on all interfaces.
Jul 18 11:13:36 raspberrypi dhcpcd[360]: eth0: no IPv6 Routers available
The /etc/network/interfaces file is the same for both systems. The /etc/dhcpcd.conf file differs in that I set up the Buster image to assign a static IP address to the Wifi interface:

Code: Select all

interface wlan0
    static ip_address=192.168.4.1/24
    nohook wpa_supplicant
But that should only affect the Wifi interface, I suppose?

epoch1970
Posts: 3887
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: Setting up network bridge under Buster

Thu Jul 18, 2019 10:52 am

Well, what you added to dhcpcd.conf affects wlan0, for sure.
If you did not block dhcpcd from taking care of eth0 -among others-, it will. That is its default behavior: fetch an address via DHCP for any unconfigured interface that comes up.
Add "denyinterfaces eth1 wlan1 usb0 ..." to the beginning of dhcpcd.conf if you need to block dhcpcd from trying to take care of some interfaces.
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

fd_
Posts: 66
Joined: Thu Oct 25, 2018 7:35 am

Re: Setting up network bridge under Buster

Thu Jul 18, 2019 11:03 am

epoch1970 wrote:
Thu Jul 18, 2019 10:52 am
Well, what you added to dhcpcd.conf affects wlan0, for sure.
If you did not block dhcpcd from taking care of eth0 -among others-, it will. That is its default behavior: fetch an address via DHCP for any unconfigured interface that comes up.
Add "denyinterfaces eth1 wlan1 usb0 ..." to the beginning of dhcpcd.conf if you need to block dhcpcd from trying to take care of some interfaces.
Just tried that (denyinterfaces eth0), and it indeed seems to solve the problem. Thank you very much for your help!
The only question that remains is why it worked on Stretch without modifying the dhcpcd configuration. Probably the difference is actually the wlan0 change in the dhcpcd config, not the Stretch vs Buster? Still, since I'm testing on a Pi Zero (without W), I wonder how a config for the wifi interface influences the other interfaces.

fd_
Posts: 66
Joined: Thu Oct 25, 2018 7:35 am

Re: Setting up network bridge under Buster

Thu Jul 18, 2019 2:00 pm

In my script, I now disable the dhcp client (systemctl stop dhcpcd) and remember the default gateway before setting up the bridge. Once it's set up, I manually assign it the IP address and set up a default route with the bridge device (ip route add default via $DEFAULT_GATEWAY dev $BRIDGE_NAME src $PUBLIC_IP). This seems to restore the functionality I had on my old Stretch image.

epoch1970
Posts: 3887
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: Setting up network bridge under Buster

Thu Jul 18, 2019 2:34 pm

Well done!
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

Return to “Troubleshooting”