Class B bridged network

[mrgenie]

I'm not sure if this is the right section, so forum moderator, feel free to move it if it is.

I'll first explain what I've been doing thus far.
I have a continental spanning class B network 172.22.0.0
The main network, with the oVPN Server, is running on 172.22.56.0/16
first oVPN client is running 172.22.57.0/16
second client is running 172.22.58.0/16
There are thus 20 subnetworks as clients.

All clients use dd-wrt.

Connection run as TAP devices and get oVPN client Ip's in the range 172.22.56.50-79
The main network gives thus local clients per dhcp IP's in the range of 172.22.56.100 -> 199
the oVPN clients get Ip's in the range 172.22.56.50 -> 79

This works splendid! But to get good vpn speed of 60+ Mbps i need to buy routers in the range of 200+ USD/piece.

As you can imagine, I wanted to try it with a raspberry pi 3, which has similar hardware compared to routers, even better in many aspects.

In the raspbian 4.4 version on the raspberry I have the wlan0 acting as access point.
the eth0 is running static ip 192.168.2.10 and makes a connection through the router 192.168.2.1

tap0 comes up.
with the raspbian I can access the whole class B network.

with the wlan0 access point I can get into the internet over eth0

but what I can't seem to get working, is getting clients on the wlan0 to access the tap0

the wlan network I tried everything from
192.168.3.0 network to 172.22.69.0
I tried 255.255.0.0 as well as 255.255.255.0

i tried
sudo iptables -A FORWARD -i eth0 -j ACCEPT
sudo iptables -A FORWARD -i wlan0 -j ACCEPT
sudo iptables -A FORWARD -i tap0 -j ACCEPT
sudo iptables -t nat -A POSTROUTING -o wlan0 -j MASQUERADE
sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
sudo iptables -t nat -A POSTROUTING -o tap0 -j MASQUERADE

and other combinations.

Now I can't get it to work. There's no possible way for me visible to get the clients behind the access point to be able to access the tap0 class B network.

Also, what is wondering me..
the oVPN on the raspbian makes a oVPN connection getting an IP in the oVPN Range of 172.22.56.50-79 and then it gets a 2nd IP from 172.22.56.100-199!!

No dd-wrt oVPN client gets such a strange thing.
In the end, the tap0 thus has an ip from the internal DHCP from the main network and not from the oVPN server.
Although "something" gets an ip from the oVPN DHCP as well.

strange behavior in the raspbian which dd-wrt has not.

Anyway, this doesn't disturb the raspbian to access the main and other client networks.

what is another interesting thing.

When I have wlan0 access point with the ip 172.22.69.1/24 the raspian can browse the network/windows network over the tap0 with the 172.22.0.0/16 without problems.

when I have wlan- access point with the ip 172.22.69.1/16 the raspian can NOT brosw this network at the other end of the tap0 anymore, but it can ping those devices on the other network! Just netbios or windows/network not working anymore.

I read a lot about setting up networks with raspbian, debian, ubuntu and 99,999% of the people online simply make connections to VPN services for hide-my-ass services using TUN devices.. and not access the other ends network anyway.

But I want a really private tap VPN layered 2, just as with dd-wrt it's working perfect, and I need this to be done on the raspbian.

I'm pretty sure it's possible.

And bridging wlan0/eth0 is working.
accessing tap0 class B network on the other end is also working.

I just need to figure out how to get from wlan0 to tap0?

Anyone with a good idea what I'm missing?

[epoch1970]

I don't know what the problem can be.
I am doing it; kernel-level vpn bridging with openvpn on multiple sites, with Pi3 as local gateway, tunnel endpoint, dns/dhcp server, and access point... It works fine for a few clients.
I use the p2p-style Ovpn config, meaning I don't use any bridge-me-this, ntp-me-that fancy openvpn features. It only uses a tap and sets up the tunnel. I take care of the rest (bridging, dnsmasq for DNS/DHCP, a few static routes, hostapd...)
So in my /etc/network/interface file I use a bridge that includes: wlan0, tap0, and a VLAN for ethernet access (eg eth0.1234). I give the bridge a static IP on the split network (from any site, pinging those addresses allows to know if a tunnel is down.)

I suppose you're using some "advanced" features of openvpn. One question though: since you are bridging and want to include the wireless clients, why do you insist on giving wlan0 an address? tap0 doesn't have an address, does it?

One last word. You should persist, because in my experience the Pi 3+raspi combination is very reliable, and it outperforms/outlives consumer-grade internet connections easily.

EDIT. Ok, I have a nice TL;DR for you'se guys and gals interested in a split private network, with Pi 3 as VPN endpoint and platform for a few key services. This is extracted from a Pi on "Site0", it calls another machine on "Site1" where the setup is 99% symmetrical.

Code: Select all

/etc/network/interfaces (partial, edited):

# Site0 LAN: The primary network interface, VLAN 1234
auto eth0.1234
iface eth0.1234 inet manual

# Site0 LAN: OpenVPN tap device
auto tap0
iface tap0 inet manual
pre-up ip tuntap add dev tap0 mode tap
# Don't let the MAC be random - Locally managed addr
# with low MSB so that the bridge adopts it.
pre-up ip link set dev tap0 address 02:ff:01:02:03:04
post-down ip tuntap del dev tap0 mode tap

# Site0 LAN: Wifi 2.4GHz b/g/n AP
# Builtin Broadcom BCM43438 chipset
# Special name defined in /etc/udev/rules.d/70-network_interfaces.rules
auto wap0
iface wap0 inet manual

# Site0 LAN: L3 bridge
# Gets the hw address of iface member tap0 (lowest MAC of all)
auto orgbr
iface orgbr inet static
bridge-stp on
bridge-maxwait 2
bridge-fd 2
bridge-ports tap0 wap0 eth0.1234
# L3 for our service node
address 172.16.0.255
network 172.16.0.0
broadcast 172.16.255.255
netmask 255.255.0.0
# We have a default route at eth0
#gateway 172.16.0.255
dns-nameservers 127.0.0.1
dns-domain site0.org
dns-search site0.org org

/etc/hostapd/hostapd.conf (partial, edited):
# BCM43438 is *not* HT40 capable.
ssid=site0.org
interface=wap0
bridge=orgbr
max_num_sta=25
wpa_psk_file=/etc/hostapd/hostapd.psk

/etc/openvpn/site0-to-site1.conf (partial, edited):
remote gw-site1.org 1234 		# calling site1 local gw port 1234
proto udp					# UDP is a more rugged tunnel fabric
float						# Accept unexpected IPs
lport 1194					# local port 1194
client					# We're a client
dev tap0					# tap0 is our iface
mtu-disc maybe			# Adaptive MTU - unsupported on Darwin
comp-lzo					# Adaptive compression
passtos					# Try to keep QoS through tunnel
fast-io					# Try to accelerate the bridge
resolv-retry infinite 			# try forever to resolve server name
persist-tun				# upon restarts...
persist-key
#do not persist-ip, it persists ip *and* port, no good under NAT/PAT
ping-timer-rem				# Don't ping until connected to remote
keepalive 30 120			# Ping every 30s - Inactivity restart 120s
mute 10					# Don't log messages repeated x times
# Routing - DNS
script-security 2			# Allow calling external scripts
up-delay					# Don't up until we've got a tunnel
up "/etc/openvpn/staticroutes.sh add"
down "/etc/openvpn/staticroutes.sh del"
# TLS security **REDACTED**
cipher AES-128-CBC			# Fast secure portable enough
verb 1					# Don't kill the SD

/etc/dsnmasq.conf (partial,edited):
# DHCP and DNS listening interface
interface=orgbr
bind-interfaces
# Don't respond to lease requests from the other side of the bridge or the LAN port
no-dhcp-interface=tap0
no-dhcp-interface=eth0
# We have a DNS+DHCP server on gw.site0.org and another on gw.site1.org
domain=site0.org,172.16.0.0,172.16.0.255
expand-hosts
domain-needed
# 172.16.200.1 is the internal authoritative server for org
# (bind9, delegates back to local sites gws running dnsmasq)
server=/org/172.16.200.1
server=/16.172.in-addr.arpa/172.16.200.1
# Local-only domains answered from static hosts file or DHCP leases.
local=/site0.org/
# Leases are tagged as an additional fencing measure on the
# bridged network. Ebtables weeds off unwanted lease
# requests coming across the bridge.
dhcp-authoritative
dhcp-range=set:site0,172.16.0.100,172.16.0.254,1h
dhcp-lease-max=154
dhcp-leasefile=/var/lib/misc/dnsmasq.leases
# We refuse to respond to clients known to belong to other sites
dhcp-ignore=tag:site1,tag:site2,tag:site3
dhcp-option=3,172.16.0.255		# Gateway
dhcp-option=6,172.16.0.255		# DNS
dhcp-option=27,1				# "all subnets are local" (same MTU)
dhcp-option=28,172.16.255.255	# Broadcast addr
dhcp-option=42,172.16.0.255		# NTP server
dhcp-option=option:domain-search,site0.org,org

/etc/rc.local (partial,edited):
# Local gw: we NAT between Site0 LAN and the local LAN
echo 1 > /proc/sys/net/ipv4/ip_forward
# IP filtering: *REDACTED*
# Block DHCP offers to clients reaching from other sites
# We have dhcp servers on each side. Requests for leases do leak but responses can be blocked
/sbin/ebtables -A INPUT --in-interface tap+ --protocol ipv4 --ip-protocol udp --ip-destination-port 67:68 -j DROP
/sbin/ebtables -A INPUT --in-interface tap+ --protocol ipv4 --ip-protocol udp --ip-source-port 67:68 -j DROP
/sbin/ebtables -A FORWARD --out-interface tap+ --protocol ipv4 --ip-protocol udp --ip-destination-port 67:68 -j DROP
/sbin/ebtables -A FORWARD --out-interface tap+ --protocol ipv4 --ip-protocol udp --ip-source-port 67:68 -j DROP

HTH.

[mrgenie]

Thank you for your reply. I already have some more ideas to track down how to solve my needs.

"So in my /etc/network/interface file I use a bridge that includes: wlan0, tap0, and a VLAN for ethernet access (eg eth0.1234). I give the bridge a static IP on the split network (from any site, pinging those addresses allows to know if a tunnel is down.)"

but here we have our differences already. On my default raspbian setup, the settings in /etc/network/interfaces isn't even used!

it is using /etc/dhcpcd.conf

Also I want the raspbian to be used as "client" bridging a whole client network behind the raspbian to the remote vpn network.

I see from your example, you're using the raspbian as vpn server, and bridge the clients. I'm not sure if that's the same.

From what I can tell, openvpn on raspbian has a problem with the netmask.
the remote opvenvpn is using 255.255.0.0 thus a class B
on dd-wrt all local networks and the remote are using class B, so every client can see every client everwhere, including the main network with the vpn server without any special routing tables needed. I found out tap is much easier for that then setting up a tun device, because some features need a layer 2 network, whereas tun only gives layer 3.

but on the raspbian, unlike dd-wrt routers, the moment I use layer 3 I can connect to the remote openvpn at least with raspbian itself, as it will open a layer 2 to the remote vpn network but a layer 3 to the local wlan clients. Thus the local wlan clients can never connect on layer 2 to the remote network.
so I need the local wlan clients also to use layer 2 network, but when I do that, even the raspbian can't use the remote network on layer 2 anymore and defaults back to layer 3...

as if raspbian has a problem using layer 2 network on 2 different devices

anyway, not giving up just yet

[mrgenie]

"One question though: since you are bridging and want to include the wireless clients, why do you insist on giving wlan0 an address?"
Good one. Indeed, it shouldn't be needed.. was just out of habit to make sure wlan contacts can connect to the rasp pi as well, if needed. but propbably won't be needed.

tap0 doesn't have an address, does it?

Yes it does. It gets one from the 172.22.56.0/16 dhcp server on the other end. That's what's so strange. All dd-wrt routers don't get an IP, but raspbian ovpn does get one.

[epoch1970]

On my default raspbian setup, the settings in /etc/network/interfaces isn't even used!

it is using /etc/dhcpcd.conf

Yes. I saw to that by removing dhcpcd. I am deviant.

Also I want the raspbian to be used as "client" bridging a whole client network behind the raspbian to the remote vpn network.
I see from your example, you're using the raspbian as vpn server, and bridge the clients. I'm not sure if that's the same.

My Pi 3 is the local network gateway. It manages everybody behind it (local clients on site0.org) and allows them to either go out to the internet by natting the request over to its eth0 interface (which is a client of some local router), or to go to the other side of the bridge, see their buddies at site1.org. Screen sharing, zeroconf and all that jazz, computer to computer regardless of site.

OpenVPN has a relatively weak (and recent) notion of what a "server" and what a "client" is. It started as a p2p application, one config and one process per link. I use this and it works well. Even with the tap0, tap1, tap2, ... config1, config2 ... needed to run many tunnels if you need to direct-connect one site to multiple sites.
Peer-to-peer multi-site (any to any, no single point of failure) is desirable, however IRL there are sites with poor connection, sites where the router is locked-down, sites where you have a nice and cozy IT environment. So there is a good chance that your final network design looks like a star, possibly a snowflake, and not like a mesh. So the Pis don't need to run that many tunnels, hence a p2p config style setup for ovpn is manageable.

...but on the raspbian, unlike dd-wrt routers, the moment I use layer 3 I can connect to the remote openvpn at least with raspbian itself, as it will open a layer 2 to the remote vpn network but a layer 3 to the local wlan clients. Thus the local wlan clients can never connect on layer 2 to the remote network...

I would say this is the difference between your setup and mine: you're asking openvpn to manage the bridge. I have no experience with server-bridge... Does it create a br0 device you can see with ifconfig -a ? If so, you could try to use an up script that would do "brctl addif br0 wlan0" or maybe simply "service hostapd restart" (assuming "bridge=br0" is set in hostapd.conf) and you should be in business.
If it doesn't create a bridge device, then I don't know, sorry.

anyway, not giving up just yet

Good!

[mrgenie]

Ok,

got it to work now for the clients on wlan0

the oVPN is connected properly
and all the clients inside wlan0 are now on layer 2 connected to the rest of the network.

except one thing, the Raspbian itself can't connect either to the remote network or it's client
nor to it's own network or clients..

it can connect to the internet, establish connection to the remote network on tap0
and then bridge wlan0 and tap0.

without bridging wlan0 and tap0, it can access the remote network and clients..
once i bridge it, it can't

this is not a big problem of course as you can imagine, as bridging is actually what I want anyway.

But it is weird don't you think?

You have access to tap0 network
you have access to wlan0 network

you bridge them, and everyone on tap0 and wlan0 are now in 1 big networking environment, but you, the bridge, are excluded.

I must be missing something still, is this intended functionality with bridging?

[epoch1970]

Wow, progress!

you bridge them, and everyone on tap0 and wlan0 are now in 1 big networking environment, but you, the bridge, are excluded.

If it were an L2 bridge (having no IP address) then every client could go buddy-buddy but the machine running the bridge would have no access to them, and vice-versa.
If the bridge is L3, assuming there is no problem with network settings or some iptable rule lurking somewhere, then the machine running the bridge has an IP on the bridged network, and it can participate. Everybody will be able to ping it.

I never hesitate to reboot when things seem wonky after I've made network changes.

[mrgenie]

Latest update, this time wanted to keep it running all night to be sure no strange things would happen:

put static ip on eth0: 192.168.2.125/24
static IP on wlan0: 192.168.2.126/24 (this is actually bogus, because I don't want to use this anyway but for whatever reason, if I don't give it any IP, the Pi3 itself will loose connectivity on the remote network)
tap0: 172.22.56.81/16
br0: 172.22.56.82/16

with these ip/netmasks openvpn connection is up and running all night, no hiccups, and both clients on the wlan as the pi3 itself can access the remote clients and vice-versa.

the clients in the wlan get Ip's in the range of 172.22.69.0/16 so they can't use local internet, only internet remote over the openvpn and they can't access the wlan access point itself, only through the openvpn tunnel..
But if I give wlan0 an IP like: 172.22.69.1/16, then openvpn will establish connection properly, but the pi3 losing connectivity to the remote network.

[mrgenie]

oh, and before I forget.

openVPN speed downstream is around 35Mbps (55Mbps is the maximum of my internet)
upstream is 11Mbps (which is the maximum of my internet)

I'm also not sure if 35Mbps is the limit of the pi3, because clients on the wlan (which I'm testing for speed)
wlan itself cost performance and wlan is known for not being most accurate when it comes to bandwidth

I've ordered a 1Gbps USB adapter to test this on tap0 bridged with eth1 networking

anyway, it's running stable and doing what it should do. But my IP/netmasking is kinda unlogical to me. I'm sure it's still not perfect. So will continue working on it and test different configurations.

also, for bridging I used bridge-utils, mainly because I used them previously. I understood these utils are not "modern" anymore and I should move to iproute2? anyone having experience with these:

https://dougvitale.wordpress.com/2011/1 ... lacements/?

or is it still safe to use the old commands?

[epoch1970]

oh, and before I forget.

openVPN speed downstream is around 35Mbps (55Mbps is the maximum of my internet)
upstream is 11Mbps (which is the maximum of my internet)

I'm also not sure if 35Mbps is the limit of the pi3, because clients on the wlan (which I'm testing for speed)
wlan itself cost performance and wlan is known for not being most accurate when it comes to bandwidth

I've ordered a 1Gbps USB adapter to test this on tap0 bridged with eth1 networking

anyway, it's running stable and doing what it should do. But my IP/netmasking is kinda unlogical to me. I'm sure it's still not perfect. So will continue working on it and test different configurations.

also, for bridging I used bridge-utils, mainly because I used them previously. I understood these utils are not "modern" anymore and I should move to iproute2? anyone having experience with these:

https://dougvitale.wordpress.com/2011/1 ... lacements/?

or is it still safe to use the old commands?

I will read again you post but I still can't picture exactly what's the deal with all those interfaces needing an address. As you've seen in my confs, I don't bother with addresses, basically.
It's no problem to use the old brctl commands.

If you want to test max throughput, you can bridge a network over a segment of a local network (3 local nets in total), and then differentiate speed when reading/writing in RAM or reading/writing to the SD. When testing over a local network you can set cipher=none and see if encryption has an impact.

I haven't bothered running speed tests, but the 30Mbps looks like what I'm seeing over a local test tunnel. OpenVPN uses only one core, but I think the bottleneck is IOs on the SD. Under such a load the Pi meditates a bit before responding to other tasks, but it is definitely responsive.
30Mpbs ain't too shabby, plus in case of ADSL links throughput is asymmetrical so it way above the upload bandwidth.

Another way to look at it. 3MB/s, roughly 30Mbps, isn't that about average transfer speed to a Pi over a LAN, for something like FTP?

[mrgenie]

@epoch1970 I agree with you 100%.

But this is just an adventure right now for me, so I want to try out everything and anything

[JPWhite]

I'm not sure if this is the right section, so forum moderator, feel free to move it if it is.

I'll first explain what I've been doing thus far.
I have a continental spanning class B network 172.22.0.0
The main network, with the oVPN Server, is running on 172.22.56.0/16
first oVPN client is running 172.22.57.0/16
second client is running 172.22.58.0/16
There are thus 20 subnetworks as clients.

All clients use dd-wrt.

Connection run as TAP devices and get oVPN client Ip's in the range 172.22.56.50-79
The main network gives thus local clients per dhcp IP's in the range of 172.22.56.100 -> 199
the oVPN clients get Ip's in the range 172.22.56.50 -> 79

This works splendid! But to get good vpn speed of 60+ Mbps i need to buy routers in the range of 200+ USD/piece.

As you can imagine, I wanted to try it with a raspberry pi 3, which has similar hardware compared to routers, even better in many aspects.

In the raspbian 4.4 version on the raspberry I have the wlan0 acting as access point.
the eth0 is running static ip 192.168.2.10 and makes a connection through the router 192.168.2.1

tap0 comes up.
with the raspbian I can access the whole class B network.

with the wlan0 access point I can get into the internet over eth0

but what I can't seem to get working, is getting clients on the wlan0 to access the tap0

the wlan network I tried everything from
192.168.3.0 network to 172.22.69.0
I tried 255.255.0.0 as well as 255.255.255.0

i tried
sudo iptables -A FORWARD -i eth0 -j ACCEPT
sudo iptables -A FORWARD -i wlan0 -j ACCEPT
sudo iptables -A FORWARD -i tap0 -j ACCEPT
sudo iptables -t nat -A POSTROUTING -o wlan0 -j MASQUERADE
sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
sudo iptables -t nat -A POSTROUTING -o tap0 -j MASQUERADE

and other combinations.

Now I can't get it to work. There's no possible way for me visible to get the clients behind the access point to be able to access the tap0 class B network.

Also, what is wondering me..
the oVPN on the raspbian makes a oVPN connection getting an IP in the oVPN Range of 172.22.56.50-79 and then it gets a 2nd IP from 172.22.56.100-199!!

No dd-wrt oVPN client gets such a strange thing.
In the end, the tap0 thus has an ip from the internal DHCP from the main network and not from the oVPN server.
Although "something" gets an ip from the oVPN DHCP as well.

strange behavior in the raspbian which dd-wrt has not.

Anyway, this doesn't disturb the raspbian to access the main and other client networks.

what is another interesting thing.

When I have wlan0 access point with the ip 172.22.69.1/24 the raspian can browse the network/windows network over the tap0 with the 172.22.0.0/16 without problems.

when I have wlan- access point with the ip 172.22.69.1/16 the raspian can NOT brosw this network at the other end of the tap0 anymore, but it can ping those devices on the other network! Just netbios or windows/network not working anymore.

I read a lot about setting up networks with raspbian, debian, ubuntu and 99,999% of the people online simply make connections to VPN services for hide-my-ass services using TUN devices.. and not access the other ends network anyway.

But I want a really private tap VPN layered 2, just as with dd-wrt it's working perfect, and I need this to be done on the raspbian.

I'm pretty sure it's possible.

And bridging wlan0/eth0 is working.
accessing tap0 class B network on the other end is also working.

I just need to figure out how to get from wlan0 to tap0?

Anyone with a good idea what I'm missing?

Have you considered running DD-WRT on an old x86 computer? It would at least match the rest of your routers with similar software hopefully increasing reliability/compatibility. It'll use 100x times the power tho .

If you are dead set on a RPi, look into putting OpenWrt on it.

[Cancelor]

...
you bridge them, and everyone on tap0 and wlan0 are now in 1 big networking environment, but you, the bridge, are excluded.

I must be missing something still, is this intended functionality with bridging?

A bridge is like a two port switch, it works at Layer 2 of the OSI model and uses MAC addresses.

Routers are OSI model Layer 3 devices, and forward data depending on the Network address, eg IP address.

[mrgenie]

Well I got most sorted out now.

everything is now working.

This time everything. Full class B network

I did check on openWRT btw, but it's not for raspberry pi 3. the pi2 is max supported.

Also, i wanted to fully understand the iptables, bridging, etc..

best way to learn to understand, is to do it yourself.

I know how dd-wrt to configure everything properly. But then you still don't know how it's working. You only know how to configure it.

While using raspbian, you still don't know all the bits and bytes about it, but first on raspbian it's where you start to learn the basics.

But of course, even though it's getting more into details, it's still only scratching the underlying methods and techniques, i know that.

On the other hand of course, no need to reinvent the wheel.

I wonder the statement routers only work on layer 3. How come, that dd-wrt makes use of ebtables if it's only layer 3?
I always thought ebtables run at layer 2, but then again, I might be wrong, still learning along the way.

[Cancelor]

Yes, it can be confusing when routers do some bridging things and bridges do some routing things.

A good diagram can help. http://www.escotal.com/Images/Network%20parts/osi.gif

Some further reading:-
https://en.wikipedia.org/wiki/Bridging_(networking)
https://en.wikipedia.org/wiki/Router_(computing)

[Lesita]

Yes,and now we can only build a Router on other Linux systems. It needs software Packs (rpm).I can't find a valid OPENWRT firmware for Raspberry 3 .maybe some time in further releases.