AsynchronousGalaxy
Posts: 16
Joined: Tue Oct 30, 2018 1:57 am

Ethernet Connection Replacement Using Two Raspberry Pis

Tue Oct 30, 2018 2:57 am

Summary of Project.

I understand this is a long post to get into, but I would like to provide as much information as possible to the awesome people that help me.

I have two devices that are connected via an Ethernet Connection. Let's call them Host and Client.

I would like to make the host and client connection wireless, but I do not have access to the source code for the devices. Therefore, I am unable to change the IP addressing / network configuration that the devices have.

Through an Awesome program, Wireshark, I was able to plug each end into a raspberry pi and decode the packets being sent from each side. from my limited networking experience, I can at least see that this is what appears to be a standard 802.11 TCP/IP protocol.

Also, Upon inspection of the cable that connects the Host and Client devices, it appears to be a "4-wire twisted pair connection" Meaning that there are two sets of +/- wires as opposed to the newer and more modern Ethernet connections. Please see
https://en.wikipedia.org/wiki/Ethernet_ ... ingle-pair
for reference to the type of cable that I plan to replace.

My Idea is to plug the Host into a Raspberry Pi(AP Mode) and Plug the client into a Raspberry Pi(Normal Client Mode) and with Bridging, be able to make the Host and Client "Think" they are still connected via direct Ethernet connection.

For feasibility's sake, lets assume that the wireless connection can support the level of traffic between the two devices without any worry of a slow connection.


Here's the current setup I am using to try and achieve this.

Host Raspberry Pi Configuration: (Raspberry Pi 3B+)
  • I Have followed the Guide In order to setup the Host Pi as an Access Point that has a bridges wlan0 to Eth0
  • In this configuration, I can ping the Host device from the Host Raspberry Pi
  • I have tested this configuration with and without the iptable entries. But I think I should not use the iptable entries as they only provide single direction communication? Versus the bridge allowing dual direction?

Client Raspberry Pi Configuration: (Raspberry Pi 3B)
  • From the guide above, I have tried just setting up the bridge between wlan0 and eth0 as well as just statically assigning IP addresses.
  • In these configurations, I can ping the Client Device from the client raspberry pi

My problem is that when I connect the Client Raspberry Pi to the Host Raspberry Pi's Access Point, I cannot ping one Raspberry Pi from the other, and subsequently, I cannot ping one device from another.

I am not understanding why In this configuration, Even though the Client Raspberry Pi shows to be associated with the Host raspberry Pi's Access Point, I cannot ping One from the other.

My Main problem now is that the Raspberry Pi's appear to be connected, but they will not ping each other.

Please see diagram below: Using the

Code: Select all

ifconfig
command, the following IP addresses are assigned accordingly under

Code: Select all

/etc/dhcpcd.conf
Image

epoch1970
Posts: 4140
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: Ethernet Connection Replacement Using Two Raspberry Pis

Tue Oct 30, 2018 6:02 pm

You can't bridge over wireless because bridging of STA mode (client) devices is not allowed by the WiFi standard.
Non-compliant devices could do that, typically a "gaming adapter" or "travel router" can bridge a single ethernet client over its wifi interface in client mode.

You can work around the problem by tunnelling the bridge over the wifi link, with something like openvpn or gre (gretap)
viewtopic.php?t=204361
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

AsynchronousGalaxy
Posts: 16
Joined: Tue Oct 30, 2018 1:57 am

Re: Ethernet Connection Replacement Using Two Raspberry Pis

Tue Oct 30, 2018 8:16 pm

Thank you for the reply.

From what you're telling me, the raspberry pi OS would not allow me to do such a thing as it is not part of the standard. Does this mean that the Raspberry Pi could potentially do this operation (breaking the standard?) anyways? I am not necessarily looking to uphold any kind of standard here. Or is it that the hardware just does't support that type of routing?


I have seen this post you mention but wasn't sure how it would apply to me. But I am starting to see the light.

First, this is a private and closed-off connection. there is no need to access the outside world for now. I would like to use your diagram as reference to what I am trying to do, with some modifications.

Code: Select all

++ Host +--------------------+  ++ Client+------------+
|             +--------+      |  |      +-----+       |
|             |Router  | )))) WiFi (((( |RPI 3|       |
|             |WiFi AP |      |  |      |Ovpn |       |
|             |RPI3    |      |  | .11 +--+--+        |
|10.201.20.1  +----+---+      |  |  bridge | Ethernet |
|                  | Ethernet |  |         |          |
|          +-------           |  |         |          |
|   bridge |                  |  |         |          |
|       +--+--+               |  |      +--+---+      |
|       |Host |               |  |      |Client|      |
|   .12 +-----+               |  |   .8 +------+      |
|                             |  |                    |
+-----------------------------+  +--------------------+

Would the above be achievable using two raspberry pi devices? Right now the only option I have is to use Raspberry Pi's to "Replace this Ethernet connection". This looks like a solid route to go if you think it would work...

epoch1970
Posts: 4140
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: Ethernet Connection Replacement Using Two Raspberry Pis

Tue Oct 30, 2018 10:26 pm

The posted setup works, within reasonable limits: you need *good to very good* wifi otherwise it doesn't work well. The wifi link acts as the switching fabric in a physical switch, here... With choppy wifi you'll get a broken ethernet switch and the machines connected to it will moan.

You can try the non-compliant adapter option. Perhaps some wireless adapters based on Atheros chips would work for that on Pi, but between sourcing the right model and building the right driver... getting and testing with a travel router (in lieu of the Pi on the client side) looks easier and cheaper.

The built-in wifi interfaces in the Pis don't have the driver or AFAIK the firmware that would make client bridging possible, so your only option is tunnelling if you want to stick to the Pi platform on both sides.
In your drawing you left the bridge next to the "host" (.12) box. In fact the bridge for this side has to be in the Pi (.1) In that bridge you would have eth0 (for your "host"), tap0 (for openvpn) and wlan0 (for the remote Pi). I think that should work.

Some comments if you want to work out a solution based on the post:
  • Along your way, check your routing tables in the Pis, use "brctl showmacs <bridge_name>", read carefully the logs from openvpn.
  • OpenVPN is a kitchen sink. In this case, only a tiny fraction of its features are required. Do not substantially change the posted configuration until you get to a working bridge. Do not add options if you don't know what they are used for.
  • I expected the cipher provided by WPA on the wifi link would suffice, so the openvpn tunnel is not ciphered. But if you prefer you can cipher the tunnel, too; A shared secret (aka "static key") would be suitable in this case (no need for certs/private keys). Network latency would suffer a bit.
  • You can do without STP in the bridges; I left it there in case people would expand the network topology. Or leave it configured, it doesn't hurt much.
  • During the setup phase, work with both Pis next to each other. You need wifi to be at 100%. Between routing, bridging and openvpn, there are enough dials to check and knobs to turn already...
Good luck ;)
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

AsynchronousGalaxy
Posts: 16
Joined: Tue Oct 30, 2018 1:57 am

Re: Ethernet Connection Replacement Using Two Raspberry Pis

Wed Oct 31, 2018 4:26 am

Wow, Awesome! Thank you for all the helpful information. I will be trying to implement this over the next couple days. I may return here for help given I have never used OpenVPN.

I am aware of what it is doing and I think I understand the process.

Again, thanks for the point in the right direction, I'll report back with my findings!

AsynchronousGalaxy
Posts: 16
Joined: Tue Oct 30, 2018 1:57 am

Re: Ethernet Connection Replacement Using Two Raspberry Pis

Wed Oct 31, 2018 5:11 pm

Okay, Where to start.

First I decided to try and follow your post for setting up openVPN. The instructions were clear and make sense. As i followed along, I came across a line for setting the mac address. This is from the Server Side Configuration on PC A (Raspberry Pi Host)

Code: Select all

pre-up ip link set dev tap0 address fe:01:ae:eb:12:34
For this address, I ran

Code: Select all

ifconfig
and used the MAC address of eth0 on the Raspberry Pi.

Other than that one change, I used the Server Side Configuration for PC A on the Raspberry Pi host and the Client side Configuration for RPI3 on the Raspberry Pi client. I followed the instructions verbatim.

Using the command: brctl showmacs,

Code: Select all

Raspberry Pi Host Side:
[email protected]:~$ brctl showmacs br0

Response:
port no     mac addr                 is local?       aging timer
1           b8:27:eb:fe:23:ba        yes             0.00
1           e6:f8:f2:ad:10:d7        no              12.49
 
The e6:f8:f2:ad:10:d7 mac address belongs to the host device I am trying to create the bridge for. So we can at least see that it is plugged into the raspberry pi eth0 interface and the raspberry pi is recognizing the device.

Looking at the device itself, it appears to have aquired an address of: 169.254.161.17, which appears to be some sort of default...signifying that DHCP is not configured correctly? The device normally has a static IP of 10.201.20.12 which I need to keep. I have turned on DHCP on this device just to try and get some pinging between the raspberry pi host and the host device.

Code: Select all

Raspberry Pi Client Side:
[email protected]:~$ brctl showmacs br0

Response:
port no     mac addr                 is local?       aging timer
1           b8:27:eb:a0:12:6e        yes             0.00
1           e6:1e:c8:b2:75:d1        yes             0.00
 
The e6:1e:c8:b2:75:d1 mac address belongs to the client device I am trying to create the bridge for. So again, we can at least see that it is plugged into the raspberry pi eth0 interface and the raspberry pi is recognizing the device.

I cannot change the IP of this device unfortunately. On the client device, there is no option for dhcp. I must use the address 10.201.20.8/1 for the client device.

At this point I think a good first test would be able to ping the Host device connected via eth0 to the raspberry pi host / ping the client device from the raspberry pi client. At this point pinging on the host side returns no network and pinging on the client side never returns anything. returns destination unreachable.

I might add too, that the client is still connected to my wifi while the host no longer has a wlan0 interface.


Apart from the setup, Another question I have includes the connection between the two raspberry pis. Wouldn't I need to setup an AP on the Raspberry Pi Host for the Raspberry Pi client to connect to?

Would setting up an AP using hostapd affect the Raspberry Pi Host configuration?

epoch1970
Posts: 4140
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: Ethernet Connection Replacement Using Two Raspberry Pis

Wed Oct 31, 2018 7:52 pm

Progress, that's good.
AsynchronousGalaxy wrote:
Wed Oct 31, 2018 5:11 pm

Code: Select all

pre-up ip link set dev tap0 address fe:01:ae:eb:12:34
For this address, I ... used the MAC address of eth0 on the Raspberry Pi.
Wrong move ;) Stealing the MAC of an existing interface on the same machine is not going to work and could have upset the system.
There is more: you don't *need* that MAC spoofing line. If you keep it and want to make it your own, choose a MAC starting with fe:...
(I'll spare you the explanation, but the point was to avoid the new br0 device on PC A getting a different IP address than eth0 used to be given.)

More generally, the starting point of my post was: "you have a LAN with DHCP and a bridged wifi network, but you need to include an ethernet device on the remote side of your LAN".
  • I assumed you had on the host side a working network: dnsmasq (DHCP), hostapd (AP), a bridge with eth0 and wlan0 as members.
  • I assumed you had made an astute choice of server network, so that from that Pi you could ping the "host" device through br0, via eth0, into the host device. And back.
  • I assumed your remote Pi on the "client" side was able to connect to wifi and acquire an IP address on the same network, and could ping the other Pi and the device on the "host" side.
And so all you required was adding a bridge in the Pi client with eth0 into it, openvpn on top of that in both bridges.

Looks like I was wrong so lets rewind:
  • "host" side first:
    1. First you have to see if a DHCP server is present or not, needed or not.
      • If not needed, use a static IP address configuration for the Pi and the black box.
      • If needed and not present, you'll add a DHCP server on the Pi: dnsmasq. Set the Pi to a static address, because the DHCP server machine can't deliver itself an address.
      • If needed and already present somewhere else on the network, use DHCP on the Pi (and make sure the DHCP server's IP pool does not include the IP address the black box device wants to use.)
    2. Now configure the black box device and eth0 in the Pi so that they can ping each other. Simple ping, from both hosts (if the black box offers a ping utility). Don't go further until you ping.
    3. Now, replace eth0 with br0 on the Pi (with eth0 in it) and ping again successfully
    4. Add hostapd on the Pi, configure it with "bridge=br0" and "interface=wlan0", start it, check hostapd is happy, see that br0 now has 2 members wlan0 and eth0. Check you can still ping the black box.
  • "client" side next:
    1. configure wlan0 and wpa_supplicant.conf (assuming you use WPA, the reasonable choice), see it authenticates with hostapd on the Pi (look at /var/log/syslog or "iwconfig")
    2. Wlan0 having an IP address on the right network (configured manually or acquired via DHCP), check you can ping the remote Pi and the black box. And the other way around.
Once you're there:
  • "host" side first:
    1. Add "tap0" to the bridge,
    2. install the openvpn "server" (listener) on the Pi, see in /var/log/syslog that it starts ok
    3. Check pinging the black box and the remote Pi again (exactly as before)
  • "client" side:
    1. add the anonymous bridge on the Pi with members eth0 and tap0
    2. install the openvpn "client" (caller), start it up, see it connects to the remote Pi ok
    3. Check pinging the black box and the Pi on the "host" side (exactly as before)
    4. Check the MAC addresses in the bridge and normally you should see the client device's MAC address
Finally, the reveal:
  • On the host side, check the MAC addresses in the bridge. The MAC of the remote "client" device should be listed.
  • From the Pi and from the black box, try pinging/accessing the remote "client" device. This time, that happens over the tunnel.
  • Rejoice and enjoy.
If it should work but it doesn't, try connecting a "regular" ethernet device behind the Pi on the client side. If it doesn't work with a computer, something is wrong somewhere in the setup.
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

AsynchronousGalaxy
Posts: 16
Joined: Tue Oct 30, 2018 1:57 am

Re: Ethernet Connection Replacement Using Two Raspberry Pis

Thu Nov 01, 2018 4:37 pm

Thanks for backing up with me there, Before moving to the VPN side of things, I would like to confirm a couple of things. I'm trying to understand the low level and the high level here.

Raspberry Pi Host AP:
  • Can ping Host Black Box with Br0. At this point, eth0 is the only if on the bridge. br0 has a static address
  • hostapd(AP) starts at Boot and wlan0 is correctly added to br0. Now, br0 contains both eth0 and wlan0
  • Can connect to Host Raspberry Pi AP from laptop(Win10), DHCP Address automatically assigned and correct network is used.
  • Can ping both the Host Raspberry Pi and the Host Black box from Laptop(Win10)
  • Can ping Client Raspberry Pi from Host Raspberry Pi
  • WooHoo!
Raspberry Pi Client:
  • Can connect to Raspberry Pi Host AP, no DHCP Address assigned....(Why Does the DHCP work when the laptop connects, but not when another raspberry pi connects?)
  • I then change dhcpcd.conf to have a static address on wlan0, I can then ping the Host Raspberry Pi AP as well as the Host Black Box....okay fine. Static is no problem
  • If I then assign a static address to eth0, I cannot ping The host Raspberry Pi anymore.
    • ping Raspberry Pi Host AP: From 10.201.20.24(eth0..shouldnt this be wlan0?) destination unreachable
    • ping Client Black Box: failed at first...but as i was typing this, I checked again and is now success.
This appears to be setup correctly....it just bothers me that From the Client Raspberry Pi, I can only ping the Host Raspberry Pi & Host Black box if the eth0 interface is not statically assigned. I cannot seem to ping both the Host devices and the client black box from the client raspberry pi at the same time.

This is what I am doing currently to setup a static address. This is from the Raspberry Pi Client side. I know i need more information. but not sure what is necessary.

Code: Select all

[email protected]:~ $sudo nano /etc/dhcpcd.conf

Scroll to bottom....

interface wlan0
static ip_address=10.201.25
static routers=10.201.20.1

interface eth0
static ip_address=10.201.20.24

save & exit

I will continue to experiment. It appears that I am getting close. I will commence VPN addition on the Host Raspberry Pi in the meantime.

epoch1970
Posts: 4140
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: Ethernet Connection Replacement Using Two Raspberry Pis

Thu Nov 01, 2018 6:58 pm

Ok, still more progress ;)
The situation on the host Pi looks good, but the Pi client doesn't fare so well it seems.
Is it close enough to the AP? Does the AP filter by MAC address or something like that?
If the answer is no and if possible, reinstalling a clean copy of the OS on the SD should work wonders. Obtaining an IP address over DHCP via wifi should not be a problem.

I'm not sure why you try giving an IP address to eth0 in the client Pi.
Eth0 should have no IP address, it will be a bridge member. Bridge members shall never have IP addresses. On both Pis, make sure you "denyinterfaces" in dhcpcd.conf so that it doesn't give IP or IPv4LL addresses to these interfaces.

In the end the client Pi will have a bridge, which itself has no IP address either ("anonymous bridge"). The routing table in the client Pi will be simple: out via wifi for everything routable.
The client device will think it is connected to a switch along with other machines, oblivious to the actual setup. It will have no idea it is physically connected to the client Pi (and conversely). If the client device tries reaching the client Pi, that will be through the bridge, through the tunnel, into the other bridge, and out the AP, over wifi...
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

AsynchronousGalaxy
Posts: 16
Joined: Tue Oct 30, 2018 1:57 am

Re: Ethernet Connection Replacement Using Two Raspberry Pis

Thu Nov 01, 2018 7:41 pm

Okay, Made some progress with the VPN!

I think I am 99 percent there. In an effort to finish this thing, I will provide as much info as I can.

The system that I am trying to make wireless also gives me error messages when the host black box is not connected properly to the client black box.
When I started, the error that i was getting was:
Client Connect error 10035: (10.201.20.8, 1)
Now that I have the VPN somewhat working, I now get the error message:
No data is received, recv
So to me this means we have a connection....to some degree!!!???

Please see the configuration dump below....I know it has to be something small I am missing.

Again, thank you so much for taking the time. I am really learning a lot here!

Host Raspberry Pi AP:
  • Can ping Client Raspberry Pi br0(10.201.20.18)
  • Can ping Client Black Box(10.201.20.8)
  • OMG I CAN PING THE REMOTE BLACK BOX!!!
*The tx and rx packet numbers are arbitrary, but just to show activity, notice ovpn0 has no RX packets?

Code: Select all

[email protected]:~ $ ifconfig 

br0:
	inet 10.201.20.1
	ether 36:2e:1a:fe:0a:bd
	RX packets: 10546
	TX packets: 9398
	
dummy0
	inet none
	ether b2:62:46:fe:4f:3b
	RX packets: 0
	TX packets: 0

eth0
	inet none
	ether b8:27:eb:a0:12:6e
	RX packets: 1467
	TX packets: 815
	
lo
	inet 127.0.0.1
	ether none
	RX packets: 13442
	TX packets: 13442
	
ovpn0
	inet 10.10.10.10
	ether 36:2e:1a:fe:0a:bd
	RX packets: 0
	TX packets: 146
	
tap0
	inet 169.254.247.146
	ether d2:41:7c:c4:64:b4
	RX packets: 4957
	TX packets: 5891
	
wlan0
	inet none
	ether b8:27:eb:f5:47:3b
	RX packets: 5240
	TX packets: 6184
	
[email protected]:~ $ brctl show

br0:
interfaces->eth0, ovpn0, tap0, wlan0
STP enabled no


OpenVPN configuration:
  • bridge-srv.status shows some number of bytes for each read and write of TUN/TAP and TCP/UDP
  • bridge-srv.pid = 493
Contents of bridge-srv.conf

Code: Select all

local 10.10.10.10
dev tap0
ifconfig-nowarn
passtos
fast-io
persist-tun
ping-timer-rem
keepalive 10 30
cipher none (This is what turns off authentication, thus no longer requiring CA)
mute 10
verb 1
writepid /run/openvpn/bridge-srv.pid

contents of /etc/network/interfaces

Code: Select all

auto br0
iface br0 manual

pre-up ip tuntap add dev tap0 mode tap
pre-up ip link set dev tap0 up
pre-up ip link set dev eth0 up
bridge-bridgeprio 65534
bridge-maxwait 2
bridge-fd 2

post-up ip link add name ovpn0 type dummy
post-up ip addr add 10.10.10.10/32 dev opvn0

bridge_ports eth0 wlan0 tap0 ovpn0

contents of /etc/dhcpcd.conf

Code: Select all

option ntp_servers
require dhcp_server_identifier
slaac private

denyinterfaces wlan0 eth0

interface br0
static ip_address=10.201.20.1

Client Raspberry Pi:
  • Can ping Host raspberry pi
  • Can ping Host Black box
  • For some reason, I always have to re add br0 and add eth0 / tap0 on each reboot...
*The tx and rx packet numbers are arbitrary, but just to show activity, notice ovpn0 has no RX packets?

Code: Select all

[email protected]:~ $ ifconfig 

br0:
	inet 10.201.20.15
	ether 2a:12:a6:46:96:4e
	RX packets: 3591
	TX packets: 3554

eth0
	inet 169.254.166.27
	ether b8:27:eb:fe:23:ba
	RX packets: 339
	TX packets: 239
	
lo
	inet 127.0.0.1
	ether none
	RX packets: 0
	TX packets: 0
	
tap0
	inet 169.254.67.136
	ether 2a:12:a6:46:96:4e
	RX packets: 3672
	TX packets: 3922
	
wlan0
	inet 10.201.20.20 (DHCP assigned from Host raspberry)
	ether b8:27:eb:ab:76:ef
	RX packets: 3827
	TX packets: 4055
	
[email protected]:~ $ brctl show

br0:
interfaces->eth0, tap0
STP enabled no


OpenVPN configuration:
  • bridge-clt.status shows some number of bytes for each read and write of TUN/TAP and TCP/UDP
  • bridge-clt.pid = 411
Contents of bridge-clt.conf

Code: Select all

remote10.10.10.10
dev tap0
ifconfig-nowarn
passtos
fast-io
persist-tun
ping-timer-rem
keepalive 10 30
cipher none (This is what turns off authentication, thus no longer requiring CA)
mute 10
verb 1
writepid /run/openvpn/bridge-clt.pid

contents of /etc/network/interfaces

Code: Select all

auto br0
iface br0 manual

pre-up ip tuntap add dev tap0 mode tap
post-down ip tuntap del dev tap0 mode tap
pre-up ip link set dev tap0 up
pre-up ip link set dev eth0 up
bridge-bridgeprio 65534
bridge-maxwait 2
bridge-fd 2

bridge_ports eth0 tap0 

contents of /etc/dhcpcd.conf
Notice the difference in this file versus the Host Raspberry pi....I think this may be a problem...I think I should also denyinterfaces here too?

Code: Select all

hostname
clientid
persistent
option rapid_commit
option interface_mtu
require dhcp_server_identifier
slaac private
Last edited by AsynchronousGalaxy on Fri Nov 02, 2018 1:47 am, edited 1 time in total.

epoch1970
Posts: 4140
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: Ethernet Connection Replacement Using Two Raspberry Pis

Fri Nov 02, 2018 8:58 am

Your client side is clearly wrong. In my post, the bridge in the client has no IP address.

I will repeat that again. The client bridge is an anonymous bridge. It has no IP address configured. The client bridge is a foreign network device that happens to be hosted by the Pi.
The only IP configured in the Pi client is for wlan0.

You apparently also have IPv4LL (169.254.x.x.) addresses on bridge members in the host or client Pis. This makes me think you did not "denyinterfaces" properly. When an interface has an IP address, the system creates routes to and from that interface. These routes can interfere with the trick we are trying to pull here.

In the original post there is the output of "ip -4 addr show" and "ip route show" on both sides. These show all the configured IPv4 addresses and all the configured routes on both sides. Try to get to similar results.

Looks like you're getting close.
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

AsynchronousGalaxy
Posts: 16
Joined: Tue Oct 30, 2018 1:57 am

Re: Ethernet Connection Replacement Using Two Raspberry Pis

Fri Nov 02, 2018 3:33 pm

Okay, I see this issue.

I will recreate your post in my terminal and share the output.

I think I have correctly denied interfaces now...

I have seen the address "10.0.0.0" What does this mean? For example in ip route show.

Also, you mentioned use of a routing table. What would this entail?


Host Raspberry Pi AP

Code: Select all

$ brctl show br0
bridge name   bridge id          STP enabled   interfaces
br0           ffff.b827ebfe23ba   no           eth0
                                               tap0
                                               wlan0

$ netstat -4an | grep 1194
udp        0      10.10.10.10:1194            0.0.0.0:*

$ ip -4 addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
5: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    inet 10.201.20.1/8 brd 10.255.255.255 scope global br0
       valid_lft forever preferred_lft forever
 7: ovpn0: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN group default qlen 1000
    inet 10.10.10.10/32  scope global ovpn0
       valid_lft forever preferred_lft forever

$ ip route show
10.0.0.0/8 dev br0 proto kernel scope link src 10.201.20.1 metric 205






Client Raspberry Pi

Code: Select all

$ brctl show br0
bridge name   bridge id          STP enabled   interfaces
br0           ffff.b827ebfe23ba   no           eth0
                                               tap0

$ netstat -4an | grep 1194
udp        0      0 0.0.0.0:1194            0.0.0.0:*

$ ip -4 addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    inet 10.201.20.20/8 brd 10.255.255.255 scope global wlan0
       valid_lft forever preferred_lft forever

$ ip route show
default via 10.201.20.1 dev wlan0 src 10.201.20.20 metric 303 
10.0.0.0/8 dev wlan0 proto kernel scope link src 10.201.20.20 metric 303
10.10.10.10 dev wlan0 scope link


epoch1970
Posts: 4140
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: Ethernet Connection Replacement Using Two Raspberry Pis

Fri Nov 02, 2018 6:25 pm

Ok,
there is a little routing table issue, in that the dummy0 interface 10.10.10.10/32 resides on the same network as the LAN: 10.201.20.0/8. The LAN uses the whole 10. private IP block, you can probably add 16M Pis to it before you run out of addresses ;)

I suggest moving dummy0 to a free network, eg 192.168.100.10/32. Files and changes:

Host, /etc/network/interfaces

Code: Select all

# IP address used by the OpenVPN client and server.
post-up ip link add name ovpn0 type dummy
post-up ip addr add 192.168.100.10/32 dev ovpn0
Host, /etc/openvpn/bridge-svr.conf

Code: Select all

# Server config
local 192.168.100.10 # Our interface address
Client, /lib/dhcpcd/dhcpcd-hooks/40-ovpn-static-route

Code: Select all

# Route to the local OpenVPN server
if [ "$interface" = "wlan0" ]; then
 ip route add 192.168.100.10/32 dev "$interface"
fi
Client, /etc/openvpn/bridge-clt.conf

Code: Select all

# Client config
remote 192.168.100.10 # Address defined at server side
Apart of that I'd say all looks fine.

Side note: I see the "host" Pi has no default route, it cannot exit to the Internet. The "client" Pi has the "host" Pi as its gateway, so it'll go there but not further. Is this by design?
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

AsynchronousGalaxy
Posts: 16
Joined: Tue Oct 30, 2018 1:57 am

Re: Ethernet Connection Replacement Using Two Raspberry Pis

Fri Nov 02, 2018 7:39 pm

Alright!!! I am up and running over here. It is working!

There are a few issues though.

The client Pi freezes.....and on-top of that, sometimes on reboot, only the lo and wlan0 interfaces are up. If I reboot/ power cycle enough, the interfaces come back....

Perhaps I need a fresh install.

Going to keep playing around with this. I'll keep you updated.


Also, the pi's are right next to each other. What are some things that might increase or decrease speed?

epoch1970
Posts: 4140
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: Ethernet Connection Replacement Using Two Raspberry Pis

Fri Nov 02, 2018 7:55 pm

Congratulations.
As I noted in my post, with this setup your Pis works hard, there is a lot of traffic going over wifi and over eth as well. Plus a little bit of CPU needed for the tunnel. Make sure you're using proper PSUs.
And again, you've built yourself a virtual network switch with a wifi link serving as the communication bus inside it... You need good wifi, esp. if the black-box machines are expecting LAN-like quality of service.
If there is a lot of wifi networks around you, try to move to a less crowded channel (...) basically your choices are 1, 6, 11.
Apart that, I would open a terminal and run "tail -f /var/log/syslog" to see if anything unusual happens on the machines, and also check with "top" who's using most CPU; you want openvpn to be on top. CPU load overall shouldn't be high, something like 25% IIRC.

The best way to check if the setup works ok is to put a "normal" computer behind the client Pi. You can add a physical switch behind the client Pi and connect a PC next to the black box device.
Last edited by epoch1970 on Fri Nov 02, 2018 8:16 pm, edited 1 time in total.
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

AsynchronousGalaxy
Posts: 16
Joined: Tue Oct 30, 2018 1:57 am

Re: Ethernet Connection Replacement Using Two Raspberry Pis

Fri Nov 02, 2018 8:01 pm

Yes, I am using the recommended 2.5A PSU

Have desktop turned off on both units. Only using console with auto-login.

In this case, would it be advisable to use an external wifi dongle?

The system is not very "modern" I would think that with the 4 wire twisted pair would not be able to compete with the Raspberry. I wonder if using a pi3b+ would work well being that its using the AC standard?

epoch1970
Posts: 4140
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: Ethernet Connection Replacement Using Two Raspberry Pis

Fri Nov 02, 2018 8:23 pm

We're crossing posts...
a) We do not know what these boxes do, perhaps they take liberties with ethernet, like sending giant frames. Seems unlikely, but possible. Testing with a normal PC is the best way to make sure the contraption works.

b) If you look at the CPU load, I guess you'll see it is not very high. A 3B+ would probably help due to better bandwidth on its network interfaces. But I don't have a 3B+, so you're on your own going for the wifi AC version. I don't see why it wouldn't work (somewhat faster) but as you know, things happen.
Also, 5GHz is faster and less crowded, but its signal does not go through obstacles as well as 2.4GHz. If you have line of sight 5GHz will surely be faster, across 2 walls, all bets are off.

BTW you're my first customer on that recipe :) I hope you can make your setup reliable and it serves you well.
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

epoch1970
Posts: 4140
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: Ethernet Connection Replacement Using Two Raspberry Pis

Fri Nov 02, 2018 8:36 pm

As for an external dongle, I do not know. In my opinion, the built-in wifi on Pi3 works fine. It doesn't know every trick in the book, but what it does, it does ok.
Another adapter might be better, or could be worse. An external dongle would use the USB link, as ethernet does, and that is not a plus.
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

AsynchronousGalaxy
Posts: 16
Joined: Tue Oct 30, 2018 1:57 am

Re: Ethernet Connection Replacement Using Two Raspberry Pis

Fri Nov 02, 2018 8:40 pm

It is seeming more and more likely that the boxes are sending huge frames. I wanted to keep this raspberry pi related, so I stuck to the "black box" method of describing things.

I may just end up doing a fresh os install on both pis and recreating the setup. For sanity sake and for practice.

I also have access to two 3b+ devices so I would also like to try and recreate this on them as well.

I would be more than happy to share some of the details of the black box, but it would have to be outside of the forum as I would like to keep this post raspberry pi related for others who would like to achieve the same effect.

If you're interested, we can exchange info and I can describe the black box more to you.

A little background, I am a college student studying computer science and I am volunteering for a startup. I was able to choose a project to work on and this one sounded extremely interesting to me.

Either way, I understand if you are busy and do not have the time. You have already helped SO much. :D

epoch1970
Posts: 4140
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: Ethernet Connection Replacement Using Two Raspberry Pis

Sat Nov 03, 2018 10:29 am

Hum. I think I forgot the elephant in the room:

Code: Select all

iw wlan0 set power_save off
Make sure to stick that in a "pre-up" command on each Pi (I suspect the "host" side with hostapd will mostly benefit from it), and anywhere else you want.
I'm so used to pepper my Pis with this instruction that I forgot to include it in the recipe.

I hope this is the magic fix (at least on Pi 3b) you need to make the setup reliable.

You could add to my original post with your corrections and a Pi, Raspbian Stretch setup for the "host"/"house" side. That might help some motivated someone in the future. I chose openvpn because it is ported to many platforms and I know it well, but a full Pi setup would be more adequate.
A Pi3B+ update could also have some value.
More interesting perhaps, and since we potentially have the same hw/OS on each side, describe the same trick using more regular IP-in-IP facilities like GRE (with gretap). Extra bonus points if you roll the whole setup in containers to make it super easy to run.

Glad I could be of help. For entrepreneurship, science, Pi, and beyond :)
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

AsynchronousGalaxy
Posts: 16
Joined: Tue Oct 30, 2018 1:57 am

Re: Ethernet Connection Replacement Using Two Raspberry Pis

Sat Nov 03, 2018 6:47 pm

Thank you for the added tip.

Of course, once I get this going reliably I will definitely update with clear instructions and also, if I can, how to perform this setup on the 3b+.

I may have some more questions as I go so stay tuned!

AsynchronousGalaxy
Posts: 16
Joined: Tue Oct 30, 2018 1:57 am

Re: Ethernet Connection Replacement Using Two Raspberry Pis

Sat Nov 03, 2018 10:56 pm

Update.

Everything is working fantastic. But there are a few caveats at this point. I will try to describe what is going on in an effort to perhaps correct this.

First, I have dramatically increased the performance of the system by adding a simple / standard desktop switch between the client raspberry pi and the client black box.

This makes me think that there is some hardware characteristic of the Client black box that is causing issues at the low level. I know all my bridging / tunneling is correct because when I use the switch and the interfaces all raise correctly, the system goes back to working like it did when I had the Host / Client black box directly connected. (this is great, I am there!)

The Host Raspberry Pi AP can be shutdown and booted / auto configured without any user input. I will say at this point it seems there are no issues with the Host Raspberry Pi.

The Client Raspberry Pi however is a little more hands on at this point. When I boot the Client Raspberry Pi it seems that 6 or 7 times out of 10 that i get the error message:

Failed to raise network interfaces: see systemctl status networking.service

When is use the command above to look at the status, It responds with this error.

failed to raise br0 (the bridge)

Other times, the client boots and all network interfaces raise properly- there are no issues.

I have tried this with other raspberry pi 3b devices and still get the same effect.

It seems like there may be some sort of sequence of events that sometimes happen when they need to happen and other times do not. I cannot seem to find out how to make this issue repeatable, I will continue to recreate it and try to find out the cause.

I am hoping that you can give me some pointers of different things to try.

epoch1970
Posts: 4140
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: Ethernet Connection Replacement Using Two Raspberry Pis

Tue Nov 06, 2018 3:38 pm

Sounds great :)
Your Pi client woes sound like a run off the mill (Raspbian) setup issue, perhaps you want to post your client-side dhcpcd.conf and interfaces files in full?
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

AsynchronousGalaxy
Posts: 16
Joined: Tue Oct 30, 2018 1:57 am

Re: Ethernet Connection Replacement Using Two Raspberry Pis

Tue Nov 06, 2018 4:29 pm

Raspberry Pi Client Config

dhcpcd.conf

Code: Select all

hostname
clientid
persistent
option rapid-commit
option interface_mtu
require dhcp_server_identifier
slaac private
denyinterfaces br0 tap0 eth0
interfaces

Code: Select all

source-directory /etc/network/interfaces.d

allow-hotplug eth0
iface eth0 inet manual

auto br0
iface br0 inet manual
pre-up iw dev wlan0 set power_save off
pre-up ip tuntap add dev tap0 mode tap
post-down ip tuntap del dev tap0 mode tap
pre-up ip link set dev tap0 up
pre-up ip link set dev eth0 up
bridge-bridgeprio 65535
bridge-maxwait 2
bridge-fd 2
bridge-ports eth0 tap0

epoch1970
Posts: 4140
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: Ethernet Connection Replacement Using Two Raspberry Pis

Tue Nov 06, 2018 7:37 pm

Maybe move “denyinterfaces br0 tap0 eth0” to the first line of the file?
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

Return to “Troubleshooting”