Fedora 29 update broke hostapd on RPi 3B+ ....suggestions?

[thatseattleguy]

Setup: RPi 3B+ happily running Fedora 29, serving as an access point using hostapd. Setup is typical (eth0 and wlan0 bridged via br0). All interfaces are all started (sucessfully, without errors) using network-scripts. Not using Network Manager nor is any GUI running - this is a headless server in runlevel 3.

All was good with this....until about two weeks ago when I did a routine "dnf update" to get the latest and greatest and then rebooted and...then hostapd stopped working. Well, mostly: I can still connect to the access point, and I can still ssh into the RPi using it as my access point, so the wireless side of hostapd is all fine, as is local traffic into and out of the server via wireless. But I cannot get any traffic from hostapd's wireless client except ICMP (pings) to forward out of the server into the gateway - it's like tcp forwarding or NAT just went away. (Yes, before you ask, /proc/sys/net/ipv4/ip_forward is set to 1.) And nothing in dmesg or the logs seems to be an error that's germane.

I have a strong suspicion the issue is iptables/firewalld related, even though I didn't change any firewall rules myself. Fedora throws up LOTS of rules in the default configuration, so it's a little daunting to figure out what might have changed after the update that would have broken the access point functionality.

Anyone else seeing this kind of thing after a recent update? Or have suggestions for a debug strategy through the rule set that can narrow things down? Happy to post the iptables rules if that would aid in the troubleshooting. Thanks!

[epoch1970]

General comments, as I don't know Fedora.

If you do not have any "master" interface besides br0, i.e. in addition to the eth0 and wlan0 bridge members, this machine is not routing anything, it is only a bridge. In which case ip_forwarding and NAT are irrelevant.

Excessive firewalling of traffic to/from br0 is a possibility.
My "debug" strategy when firewalls are possibly involved is simple: shutdown external (Internet) access, then lift entirely the firewall(s), and check if the problem is still there. Once everything works, reboot and check it still works. Finally bring up firewalling piece by piece and watch for the moment traffic stops flowing...

[thatseattleguy]

Epoch, merci. I killed all firewall rules with...

Code: Select all

iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
iptables -t nat -F
iptables -t mangle -F
iptables -F
iptables -X

....and now the system works again as an access point. However, that leaves the system wide open, and so only confirms the issue is (as suspected) iptables, not what in its rules configuration - rules that used to work, but don't now - got changed due the update.

(Fedora's firewalld is based on "zones" and while it's powerful, it's also arcane, and I was hoping not to have to dig deeply into that if someone else had already solved the generic issue.)

If you do not have any "master" interface besides br0, i.e. in addition to the eth0 and wlan0 bridge members, this machine is not routing anything, it is only a bridge. In which case ip_forwarding and NAT are irrelevant.

Well, perhaps it's just terminology, but somewhere routing is happening here using just those three interfaces - because otherwise multiple users could not connect simultaneously to it as an access point and still access the Internet independently. As they can...
So perhaps the routing and NATing is taking place entirely within hostapd's code stack - but it's definitely still routing, n'est-ce pas?

[epoch1970]

Well, perhaps it's just terminology, but somewhere routing is happening here using just those three interfaces - because otherwise multiple users could not connect simultaneously to it as an access point and still access the Internet independently. As they can...
So perhaps the routing and NATing is taking place entirely within hostapd's code stack - but it's definitely still routing, n'est-ce pas?

Je ne crois pas...
- Take a physical router: it always has 2 interfaces or more, because routing is something that happens at the border of 2 networks. "broadcast domain" is another name for a network.
- Take a physical network switch: it can have no IP address at all, and yet it does put in contact machines in the network. That is because the switch works within the broadcast domain and uses MAC addresses to make communication possible. A software bridge is the equivalent of a network switch. Interface members of the bridge (e.g. eth0, wlan0) are the equivalent of bridge ports.
Considering a machine connected to a single network, IP forwarding (routing), NAT do not make sense. And in the belly of a bridge (switch) nothing counts but MAC addresses and ports. Normally iptables will not deal at all with traffic happening within a bridge. Rarely the old ebtables, iptables-like bridge filtering package is used.

I hope this futile theoretical point is now firmly established. Now let me brush over the practical situation
The damned firewall sits "in front" of br0 and somehow blocks traffic in and/or out of br0. You certainly can setup filtering in absence of routing ("personal firewall") and this is what happens here I think.

I don't use firewalld. Lucky me.
Their zone concept I something I find quite useful, that you can find implemented for example in Shorewall.

What is a zone?
A network zone defines the level of trust for network connections. This is a one to many relation, which means that a connection can only be part of one zone, but a zone can be used for many network connections.
The zone defines the firewall features that are enabled in this zone:
* Predefined services
...

So. Network interfaces belong to a zone (n to 1). Zones have policies defined according to the nature of the zone, e.g. "dmz = filter in and out from and to wan", "lan = filter in from wan, block any to and from dmz" etc. Policies get implemented as firewall rules over the interfaces in the zone.
In practice, you can consider a "zone" is just a symbolic name for an IP network.

The firewalld thing can be remote-controlled by another thing called D-Bus. Lucky you.
Many processes, including GUI widgets can connect with D-Bus. That makes the firewall more dynamic/granular in its configuration, and possibly able to work as an "application firewall" (per-process rules, à la Windows)

I surmise firewalld is in "safe" mode after the update, and there is a knob you need to turn somewhere in order to reenable traffic. Buried in a GUI or within an interminable JSON file, perhaps...

[thatseattleguy]

Grazi for the deep theoretical musings. I blame using OpenBSD + pf for too many years as my firewalling tools of first resort - never took the time to get back to the Linux/Redhat/Fedora way of doing things. I probably need to dig into iptables / firewalld soon...if only as it's what my customers will most likely be using (at least the ones not just throwing an expensive firewall appliance at their problems and hoping they all go away).

In any case, I digress; your writings got me thinking and researching, and I've temporarily "solved" my own situation simply with one line:

Code: Select all

/usr/sbin/iptables -I FORWARD -m physdev --physdev-is-bridged --physdev-in wlan0 --physdev-out eth0 -j ACCEPT

That forces an ACCEPT into the forwarding rules for the two interfaces in question, and after doing only that, hostapd works perfectly again - no other changes in Fedora's iptables rules were needed. It has the feel of clobbering the problem with an meat ax rather than slicing it with a scalpel, but until I learn more, I can live with it. Hoping someone else using Fedora will chime in with a better option, someday.

merci.