bls
Posts: 184
Joined: Mon Oct 22, 2018 11:25 pm

Pi4/Buster router performance

Sat Jul 13, 2019 11:34 pm

One of the changes in the Pi4 that I was really excited about was the much higher performance of the Ethernet and USB3 ports. I set up my Pi4 as a router and did some testing.

I set up the following configuration:

x86 Box <----> GB Switch <---> Pi4 Dongle [Pi4] Ethernet <---> GB Switch <---> NFS server

I plugged an SSD into a Pi4 USB 3.0 port so that the slow SD card would not be a limiting factor for testing.

First I tested the speed of the Ethernet and the USB 3.0 GB Ethernet dongle (Cable Matters, if it matters :)) using the above configuration. The Pi4 hardwired Ethernet generally got about 920Mb/sec in my tests (copying a large ISO from an NFS share). The Ethernet dongle was a bit slower, clocking in at about 880Mb. I suppose part of the performance dropoff for the dongle could be due to it and the SSD both being on USB 3 ports, but it's still quite good!

Next I set up iptables to enable routing, and enabled IP forwarding:

iptables --table nat --append POSTROUTING --out-interface eth0 -j MASQUERADE
iptables --append FORWARD --in-interface eth1 -j ACCEPT
sysctl net.ipv4.ip_forward=1

I haven't done exhaustive testing, but I'm seeing about 550Mb/sec through the Pi4 as router. I was a bit surprised, since this is a fairly large drop from the measured raw port capabilities.

Has anyone else done any similar testing with a P4? Is there any Raspbian tuning that I could be looking at to bring up the router throughput?

epoch1970
Posts: 3364
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: Pi4/Buster router performance

Sun Jul 14, 2019 9:56 am

I don’t have a pi4, sorry (and sad).

Did you try bridging the interfaces and see how fast (UDP, preferably) transfers go?
If you get a result in the 800mbps range then I would try UDP, routed, without masquerading (with static routes to the pi4 router)
If you get a result in the 600mbps range then I don’t know what to say.
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

bls
Posts: 184
Joined: Mon Oct 22, 2018 11:25 pm

Re: Pi4/Buster router performance

Sun Jul 14, 2019 11:18 pm

epoch1970 wrote: I don’t have a pi4, sorry (and sad).

Did you try bridging the interfaces and see how fast (UDP, preferably) transfers go?
If you get a result in the 800mbps range then I would try UDP, routed, without masquerading (with static routes to the pi4 router)
If you get a result in the 600mbps range then I don’t know what to say.
Excellent suggestion! And sorry that you don't have a pi4, but at the moment mine is air-cooled sitting on top of a pi3 case, looking a bit out of place :lol:

My configuration is as described in the base note, with the iperf3 server running on the x86 NFS server.

UDP: iperf3 x86 to iperf3 server 1.05Mb/sec
UDP: iperf3 server to x86 (via -R) 1.05Mb/sec
TCP: iperf3 x86 to iperf3 server 780 Mb/sec
TCP: iperf3 server to x86 (via -R) 600Mb/sec

Definitely looks like TCP is the bottleneck. A full router replacement will require TCP, but luckily, my internet connection speed is less than the pi4's capability, and the Pi will probably get fast enough before my internet catches up with it :lol:

On to more extensive testing while I decide on a pi case/heat manager.

epoch1970
Posts: 3364
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: Pi4/Buster router performance

Mon Jul 15, 2019 8:28 am

That looks better. So that is iperf routed without masquerade, UDP or TCP, right?

On a LAN I would prefer to run NFS (all I can in fact) over UDP, but that’s an aside.

The drop in performance over TCP can probably be remedied via sysctl, there are many kernel tunables related to TCP buffers, etc.
I trust this Arch doc could be useful in that endeavor; or just wait a few months until a student tunes their pi4 up to 11 when connected to the University network and posts about it...
There are also various NFS performance recipes you can find on the net, although mostly dealing with the server side (and slowly losing relevance I believe).
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

Return to “Networking and servers”