WireGuard on the Raspberry Pi

[ejolson]

Recently a way to encrypt network connections called WireGuard was announced that

• operates like IPSec at the kernel level for efficiency;
• uses 4000 lines of code instead of 600,000 lines.

Here is a summary of results for WireGuard using the Pi 4B and two Xeon E5-1650 servers.

Code: Select all

                 Direct       WireGuard       Relative 
              iperf3  ping   iperf3  ping   iperf3  ping
Pi4B to Xeon    936  0.182     704  0.874     75%    20%
Xeon to Xeon    935  0.207     896  0.481     96%    43%

Without WireGuard the iperf3 and ping results for the Pi and Xeon are about the same. Bandwidth as measured by iperf3 for the Pi slowed down more over WireGuard than the Xeon, likely due to differences in CPU processing speed. The ping times for the Pi over WireGuard were typically the same as the Xeon to Xeon case at about 0.4ms; however, the average ping time was greater than 0.8ms in each test involving the Pi because there was at least one ping which took more than 2ms. Does anyone know why?

Since WireGuard is not included in the latest version of Raspbian Buster, I installed from source using

Code: Select all

$ wget https://git.zx2c4.com/WireGuard/snapshot/WireGuard-0.0.20190905.tar.xz
$ tar Jxf WireGuard-0.0.20190905.tar.xz
$ cd WireGuard-0.0.20190905
$ cd src
$ make -j8
$ sudo bash
# make install

As root I then followed the quick start instructions to set up a wg0 interface that is part of a virtual private network along with two Xeon E5-1650 servers.

The Pi 4B to Xeon iperf3 runs were

Code: Select all

$ iperf3 -c 192.168.177.2 ; # Pi 4B to Xeon over WireGuard
Connecting to host 192.168.177.2, port 5201
[  5] local 192.168.177.4 port 57466 connected to 192.168.177.2 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  83.4 MBytes   700 Mbits/sec    0    417 KBytes       
[  5]   1.00-2.00   sec  84.1 MBytes   705 Mbits/sec    0    417 KBytes       
[  5]   2.00-3.00   sec  84.3 MBytes   707 Mbits/sec    0    417 KBytes       
[  5]   3.00-4.00   sec  85.3 MBytes   715 Mbits/sec    0    458 KBytes       
[  5]   4.00-5.00   sec  84.5 MBytes   709 Mbits/sec    0    458 KBytes       
[  5]   5.00-6.00   sec  83.5 MBytes   701 Mbits/sec    0    458 KBytes       
[  5]   6.00-7.00   sec  83.8 MBytes   703 Mbits/sec    0    458 KBytes       
[  5]   7.00-8.00   sec  85.5 MBytes   717 Mbits/sec    0    458 KBytes       
[  5]   8.00-9.00   sec  83.9 MBytes   704 Mbits/sec    0    458 KBytes       
[  5]   9.00-10.00  sec  83.8 MBytes   703 Mbits/sec    0    458 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   842 MBytes   706 Mbits/sec    0             sender
[  5]   0.00-10.04  sec   841 MBytes   703 Mbits/sec                  receiver

iperf Done.
$ iperf3 -c d0.wulf ; # Pi 4B to Xeon direct no WireGuard
Connecting to host d0.wulf, port 5201
[  5] local 192.168.174.145 port 55916 connected to 192.168.174.150 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   111 MBytes   935 Mbits/sec    1    366 KBytes       
[  5]   1.00-2.00   sec   112 MBytes   937 Mbits/sec    0    366 KBytes       
[  5]   2.00-3.00   sec   112 MBytes   940 Mbits/sec    0    366 KBytes       
[  5]   3.00-4.00   sec   112 MBytes   936 Mbits/sec    0    366 KBytes       
[  5]   4.00-5.00   sec   112 MBytes   939 Mbits/sec    0    366 KBytes       
[  5]   5.00-6.00   sec   112 MBytes   938 Mbits/sec    0    366 KBytes       
[  5]   6.00-7.00   sec   112 MBytes   937 Mbits/sec    0    366 KBytes       
[  5]   7.00-8.00   sec   112 MBytes   937 Mbits/sec    0    366 KBytes       
[  5]   8.00-9.00   sec   112 MBytes   937 Mbits/sec    0    366 KBytes       
[  5]   9.00-10.00  sec   112 MBytes   938 Mbits/sec    0    366 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  1.09 GBytes   938 Mbits/sec    1             sender
[  5]   0.00-10.04  sec  1.09 GBytes   933 Mbits/sec                  receiver

iperf Done.

and the Xeon to Xeon iperf3 runs

Code: Select all

$ iperf3 -c 192.168.177.2 ; # Xeon to Xeon over WireGuard
Connecting to host 192.168.177.2, port 5201
[  5] local 192.168.177.3 port 34170 connected to 192.168.177.2 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   108 MBytes   902 Mbits/sec    0    314 KBytes       
[  5]   1.00-2.00   sec   107 MBytes   897 Mbits/sec    0    361 KBytes       
[  5]   2.00-3.00   sec   107 MBytes   897 Mbits/sec    0    383 KBytes       
[  5]   3.00-4.00   sec   107 MBytes   897 Mbits/sec    0    401 KBytes       
[  5]   4.00-5.00   sec   107 MBytes   898 Mbits/sec    0    419 KBytes       
[  5]   5.00-6.00   sec   107 MBytes   896 Mbits/sec    0    419 KBytes       
[  5]   6.00-7.00   sec   107 MBytes   894 Mbits/sec    0    419 KBytes       
[  5]   7.00-8.00   sec   107 MBytes   896 Mbits/sec    0    419 KBytes       
[  5]   8.00-9.00   sec   107 MBytes   897 Mbits/sec    0    419 KBytes       
[  5]   9.00-10.00  sec   107 MBytes   897 Mbits/sec    0    419 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  1.04 GBytes   897 Mbits/sec    0             sender
[  5]   0.00-10.00  sec  1.04 GBytes   896 Mbits/sec                  receiver

iperf Done.
$ iperf3 -c d0.wulf ; # Xeon to Xeon direct no WireGuard
Connecting to host d0.wulf, port 5201
[  5] local 192.168.174.151 port 35774 connected to 192.168.174.150 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   113 MBytes   947 Mbits/sec    0    369 KBytes       
[  5]   1.00-2.00   sec   112 MBytes   937 Mbits/sec    0    369 KBytes       
[  5]   2.00-3.00   sec   111 MBytes   935 Mbits/sec    0    369 KBytes       
[  5]   3.00-4.00   sec   111 MBytes   932 Mbits/sec    0    369 KBytes       
[  5]   4.00-5.00   sec   111 MBytes   934 Mbits/sec    0    369 KBytes       
[  5]   5.00-6.00   sec   112 MBytes   937 Mbits/sec    0    369 KBytes       
[  5]   6.00-7.00   sec   111 MBytes   935 Mbits/sec    0    369 KBytes       
[  5]   7.00-8.00   sec   111 MBytes   931 Mbits/sec    0    369 KBytes       
[  5]   8.00-9.00   sec   111 MBytes   934 Mbits/sec    0    397 KBytes       
[  5]   9.00-10.00  sec   112 MBytes   937 Mbits/sec    0    397 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  1.09 GBytes   936 Mbits/sec    0             sender
[  5]   0.00-10.00  sec  1.09 GBytes   934 Mbits/sec                  receiver

iperf Done.

The Pi 4B to Xeon ping runs were

Code: Select all

$ ping -c 5 192.168.177.2 # ; Pi 4B to Xeon over WireGuard
PING 192.168.177.2 (192.168.177.2) 56(84) bytes of data.
64 bytes from 192.168.177.2: icmp_seq=1 ttl=64 time=0.512 ms
64 bytes from 192.168.177.2: icmp_seq=2 ttl=64 time=0.434 ms
64 bytes from 192.168.177.2: icmp_seq=3 ttl=64 time=2.51 ms
64 bytes from 192.168.177.2: icmp_seq=4 ttl=64 time=0.453 ms
64 bytes from 192.168.177.2: icmp_seq=5 ttl=64 time=0.468 ms

--- 192.168.177.2 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 138ms
rtt min/avg/max/mdev = 0.434/0.874/2.505/0.816 ms
$ ping -c 5 d0.wulf # ; Pi 4B to Xeon direct no WireGuard
PING d0.wulf (192.168.174.150) 56(84) bytes of data.
64 bytes from d0.wulf (192.168.174.150): icmp_seq=1 ttl=64 time=0.190 ms
64 bytes from d0.wulf (192.168.174.150): icmp_seq=2 ttl=64 time=0.140 ms
64 bytes from d0.wulf (192.168.174.150): icmp_seq=3 ttl=64 time=0.176 ms
64 bytes from d0.wulf (192.168.174.150): icmp_seq=4 ttl=64 time=0.194 ms
64 bytes from d0.wulf (192.168.174.150): icmp_seq=5 ttl=64 time=0.211 ms

--- d0.wulf ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 125ms
rtt min/avg/max/mdev = 0.140/0.182/0.211/0.025 ms

and the Xeon to Xeon ping runs

Code: Select all

$ ping -c 5 192.168.177.2 ; # Xeon to Xeon over WireGuard
PING 192.168.177.2 (192.168.177.2) 56(84) bytes of data.
64 bytes from 192.168.177.2: icmp_seq=1 ttl=64 time=0.485 ms
64 bytes from 192.168.177.2: icmp_seq=2 ttl=64 time=0.476 ms
64 bytes from 192.168.177.2: icmp_seq=3 ttl=64 time=0.471 ms
64 bytes from 192.168.177.2: icmp_seq=4 ttl=64 time=0.515 ms
64 bytes from 192.168.177.2: icmp_seq=5 ttl=64 time=0.460 ms

--- 192.168.177.2 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 53ms
rtt min/avg/max/mdev = 0.460/0.481/0.515/0.027 ms
$ ping -c 5 d0.wulf ; # Xeon to Xeon direct no WireGuard
PING d0.wulf (192.168.174.150) 56(84) bytes of data.
64 bytes from d0.wulf (192.168.174.150): icmp_seq=1 ttl=64 time=0.223 ms
64 bytes from d0.wulf (192.168.174.150): icmp_seq=2 ttl=64 time=0.226 ms
64 bytes from d0.wulf (192.168.174.150): icmp_seq=3 ttl=64 time=0.218 ms
64 bytes from d0.wulf (192.168.174.150): icmp_seq=4 ttl=64 time=0.220 ms
64 bytes from d0.wulf (192.168.174.150): icmp_seq=5 ttl=64 time=0.150 ms

--- d0.wulf ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 58ms
rtt min/avg/max/mdev = 0.150/0.207/0.226/0.031 ms

[fjleon]

you don't need to download from source. latest version is on debian unstable and you can use apt pinning. just go to the wireguard website and they explain everything.

you can also use angristan's script to configure everything automatically, although i did have to change one line in the script because it doesn't detect debian properly

[ejolson]

you don't need to download from source. latest version is on debian unstable and you can use apt pinning. just go to the wireguard website and they explain everything.

you can also use angristan's script to configure everything automatically, although i did have to change one line in the script because it doesn't detect debian properly

I am using Raspbian Buster on the Pi. Since the WireGuard module has to match the running kernel, I don't know how a package from Debian unstable helps.

Using a setup script provided by a third party seems to make auditable security more difficult. Fortunately, WireGuard is easy to set up by hand.

Have you tested WireGuard with any other model of Pi?

[epoch1970]

Wg works on 3B, “fast enough”.

[ejolson]

Wg works on 3B, “fast enough”.

Thanks for posting. Given the runs I just performed with the 3B+, since the maximum speed of the 3B is 100 Mbit, I expect performance of the 3B with WireGuard is about the same as without.

I've updated the table to include Pi 3B+ WireGuard results.

Code: Select all

                 Direct       WireGuard       Relative 
              iperf3  ping   iperf3  ping   iperf3  ping
Pi4B to Xeon    936  0.182     704  0.874     75%    20%
Pi3B+ to Xeon   293  0.353     272  1.511     93%    23%
Xeon to Xeon    935  0.207     896  0.481     96%    43%

The decrease in bandwidth when using WireGuard on the 3B+ is much less noticeable than with the 4B. This apparently reflects differences in the balance of processor to networking speeds of the 3B+ compared to the 4B. While the ping results with the Pi 3B+ were generally around 0.7ms, an occasional ping time greater than 4ms led to an average ping time much greater than without WireGuard in place. Again, it would be interesting to know what causes this.

The Pi 3B+ to Xeon iperf3 runs were

Code: Select all

$ iperf3 -c 192.168.177.2 ; # Pi 3B+ to Xeon over WireGuard
Connecting to host 192.168.177.2, port 5201
[  5] local 192.168.177.4 port 33132 connected to 192.168.177.2 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  33.1 MBytes   278 Mbits/sec    0    452 KBytes       
[  5]   1.00-2.00   sec  32.2 MBytes   270 Mbits/sec    0    452 KBytes       
[  5]   2.00-3.00   sec  32.6 MBytes   273 Mbits/sec    0    452 KBytes       
[  5]   3.00-4.00   sec  32.6 MBytes   273 Mbits/sec    0    452 KBytes       
[  5]   4.00-5.00   sec  32.3 MBytes   271 Mbits/sec    0    452 KBytes       
[  5]   5.00-6.00   sec  32.6 MBytes   274 Mbits/sec    0    452 KBytes       
[  5]   6.00-7.00   sec  32.4 MBytes   272 Mbits/sec    0    452 KBytes       
[  5]   7.00-8.00   sec  32.6 MBytes   273 Mbits/sec    0    452 KBytes       
[  5]   8.00-9.00   sec  32.6 MBytes   273 Mbits/sec    0    452 KBytes       
[  5]   9.00-10.00  sec  32.6 MBytes   273 Mbits/sec    0    452 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   325 MBytes   273 Mbits/sec    0             sender
[  5]   0.00-10.06  sec   325 MBytes   271 Mbits/sec                  receiver

iperf Done.
$ iperf3 -c d0.wulf ; # Pi 3B+ to Xeon direct no WireGuard
Connecting to host d0.wulf, port 5201
[  5] local 192.168.174.144 port 46690 connected to 192.168.174.150 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  35.9 MBytes   301 Mbits/sec    0    481 KBytes       
[  5]   1.00-2.00   sec  35.2 MBytes   295 Mbits/sec    0    488 KBytes       
[  5]   2.00-3.00   sec  34.7 MBytes   291 Mbits/sec    0    488 KBytes       
[  5]   3.00-4.00   sec  34.7 MBytes   292 Mbits/sec    0    488 KBytes       
[  5]   4.00-5.00   sec  34.9 MBytes   292 Mbits/sec    0    488 KBytes       
[  5]   5.00-6.00   sec  35.2 MBytes   295 Mbits/sec    0    488 KBytes       
[  5]   6.00-7.00   sec  34.8 MBytes   292 Mbits/sec    0    488 KBytes       
[  5]   7.00-8.00   sec  34.8 MBytes   292 Mbits/sec    0    488 KBytes       
[  5]   8.00-9.00   sec  35.1 MBytes   294 Mbits/sec    0    488 KBytes       
[  5]   9.00-10.00  sec  34.8 MBytes   292 Mbits/sec    0    488 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   350 MBytes   294 Mbits/sec    0             sender
[  5]   0.00-10.05  sec   349 MBytes   292 Mbits/sec                  receiver

iperf Done.

and the Pi 3B+ to Xeon ping runs were

Code: Select all

$ ping -c 5 192.168.177.2 ; # Pi 3B+ to Xeon over WireGuard
PING 192.168.177.2 (192.168.177.2) 56(84) bytes of data.
64 bytes from 192.168.177.2: icmp_seq=1 ttl=64 time=0.712 ms
64 bytes from 192.168.177.2: icmp_seq=2 ttl=64 time=0.748 ms
64 bytes from 192.168.177.2: icmp_seq=3 ttl=64 time=0.743 ms
64 bytes from 192.168.177.2: icmp_seq=4 ttl=64 time=4.69 ms
64 bytes from 192.168.177.2: icmp_seq=5 ttl=64 time=0.662 ms

--- 192.168.177.2 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 144ms
rtt min/avg/max/mdev = 0.662/1.511/4.690/1.589 ms
$ ping -c 5 d0.wulf ; # Pi 3B+ to Xeon direct no WireGuard
PING d0.wulf (192.168.174.150) 56(84) bytes of data.
64 bytes from d0.wulf (192.168.174.150): icmp_seq=1 ttl=64 time=0.371 ms
64 bytes from d0.wulf (192.168.174.150): icmp_seq=2 ttl=64 time=0.348 ms
64 bytes from d0.wulf (192.168.174.150): icmp_seq=3 ttl=64 time=0.349 ms
64 bytes from d0.wulf (192.168.174.150): icmp_seq=4 ttl=64 time=0.313 ms
64 bytes from d0.wulf (192.168.174.150): icmp_seq=5 ttl=64 time=0.385 ms

--- d0.wulf ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 140ms
rtt min/avg/max/mdev = 0.313/0.353/0.385/0.027 ms

[ejolson]

I installed an RTL8153 USB Gigabit Ethernet Adapter in a Pi Zero and updated the table to include the corresponding WireGuard results.

Code: Select all

                  Direct       WireGuard       Relative 
              iperf3  ping   iperf3  ping   iperf3  ping
Pi4B to Xeon    936  0.182     704  0.874     75%    20%
Pi3B+ to Xeon   293  0.353     272  1.511     93%    23%
Zero to Xeon    212  0.866      36  1.541     17%    56%
Xeon to Xeon    935  0.207     896  0.481     96%    43%

The substantial decrease in bandwidth when using WireGuard on the Zero is likely because of the slower CPU. Ping results, on the other hand, showed less of a slowdown than for any of the other cases. Moreover, they were steady without the occasional delays observed on the 3B+ and 4B. As the Zero has a single core processor, could the ping delays observed on the other models result from multiprocessor scheduling moving WireGuard to a different core?

The Pi Zero to Xeon iperf3 runs were

Code: Select all

$ iperf3 -c 192.168.177.2 ; # Zero to Xeon over WireGuard
Connecting to host 192.168.177.2, port 5201
[  5] local 192.168.177.4 port 51346 connected to 192.168.177.2 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  4.10 MBytes  34.4 Mbits/sec    0   60.1 KBytes       
[  5]   1.00-2.00   sec  3.92 MBytes  32.9 Mbits/sec    0   64.1 KBytes       
[  5]   2.00-3.00   sec  3.68 MBytes  30.9 Mbits/sec    0   70.8 KBytes       
[  5]   3.00-4.00   sec  4.17 MBytes  34.8 Mbits/sec    0   70.8 KBytes       
[  5]   4.00-5.00   sec  4.17 MBytes  35.1 Mbits/sec    0   70.8 KBytes       
[  5]   5.00-6.00   sec  4.35 MBytes  36.5 Mbits/sec    0    107 KBytes       
[  5]   6.00-7.00   sec  4.54 MBytes  38.0 Mbits/sec    0    155 KBytes       
[  5]   7.00-8.00   sec  4.54 MBytes  38.1 Mbits/sec    0    231 KBytes       
[  5]   8.00-9.01   sec  4.60 MBytes  38.3 Mbits/sec    0    231 KBytes       
[  5]   9.01-10.00  sec  4.60 MBytes  38.8 Mbits/sec    0    231 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  42.7 MBytes  35.8 Mbits/sec    0             sender
[  5]   0.00-10.04  sec  42.1 MBytes  35.2 Mbits/sec                  receiver

iperf Done.
$ iperf3 -c d0.wulf ; # Zero to Xeon direct no WireGuard
Connecting to host d0.wulf, port 5201
[  5] local 192.168.174.205 port 49322 connected to 192.168.174.150 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.04   sec  26.3 MBytes   212 Mbits/sec    0   67.9 KBytes       
[  5]   1.04-2.03   sec  25.0 MBytes   211 Mbits/sec    0   67.9 KBytes       
[  5]   2.03-3.03   sec  23.8 MBytes   200 Mbits/sec    0   74.9 KBytes       
[  5]   3.03-4.04   sec  26.2 MBytes   217 Mbits/sec    0   74.9 KBytes       
[  5]   4.04-5.01   sec  25.0 MBytes   216 Mbits/sec    0   74.9 KBytes       
[  5]   5.01-6.03   sec  26.2 MBytes   217 Mbits/sec    0   74.9 KBytes       
[  5]   6.03-7.04   sec  26.2 MBytes   216 Mbits/sec    0   74.9 KBytes       
[  5]   7.04-8.08   sec  25.0 MBytes   203 Mbits/sec    0    122 KBytes       
[  5]   8.08-9.05   sec  25.0 MBytes   216 Mbits/sec    0    122 KBytes       
[  5]   9.05-10.01  sec  25.0 MBytes   217 Mbits/sec    0    122 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.01  sec   254 MBytes   213 Mbits/sec    0             sender
[  5]   0.00-10.04  sec   254 MBytes   212 Mbits/sec                  receiver

iperf Done.

and the Pi Zero to Xeon ping runs were

Code: Select all

$ ping -c 5 192.168.177.2 ; # Zero to Xeon over WireGuard
PING 192.168.177.2 (192.168.177.2) 56(84) bytes of data.
64 bytes from 192.168.177.2: icmp_seq=1 ttl=64 time=1.65 ms
64 bytes from 192.168.177.2: icmp_seq=2 ttl=64 time=1.63 ms
64 bytes from 192.168.177.2: icmp_seq=3 ttl=64 time=1.55 ms
64 bytes from 192.168.177.2: icmp_seq=4 ttl=64 time=1.27 ms
64 bytes from 192.168.177.2: icmp_seq=5 ttl=64 time=1.61 ms

--- 192.168.177.2 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 11ms
rtt min/avg/max/mdev = 1.270/1.541/1.645/0.139 ms
$ ping -c 5 d0.wulf ; # Zero to Xeon direct no WireGuard
PING d0.wulf (192.168.174.150) 56(84) bytes of data.
64 bytes from d0.wulf (192.168.174.150): icmp_seq=1 ttl=64 time=0.880 ms
64 bytes from d0.wulf (192.168.174.150): icmp_seq=2 ttl=64 time=0.789 ms
64 bytes from d0.wulf (192.168.174.150): icmp_seq=3 ttl=64 time=0.889 ms
64 bytes from d0.wulf (192.168.174.150): icmp_seq=4 ttl=64 time=0.878 ms
64 bytes from d0.wulf (192.168.174.150): icmp_seq=5 ttl=64 time=0.896 ms

--- d0.wulf ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 9ms
rtt min/avg/max/mdev = 0.789/0.866/0.896/0.047 ms

[ejolson]

I installed this 64-bit Gentoo image on the Raspberry Pi 4B and updated the table to include the corresponding WireGuard results.

Code: Select all

                  Direct       WireGuard       Relative 
              iperf3  ping   iperf3  ping   iperf3  ping
4B64 to Xeon    934  0.192     760  0.476     81%    40%
Pi4B to Xeon    936  0.182     704  0.874     75%    20%
Pi3B+ to Xeon   293  0.353     272  1.511     93%    23%
Zero to Xeon    212  0.866      36  1.541     17%    56%
Xeon to Xeon    935  0.207     896  0.481     96%    43%

Note that the iperf3 bandwidth measurements are about 10% faster in 64-bit mode than 32-bit.

The 64-bit Pi 4B to Xeon iperf3 runs were

Code: Select all

$ iperf3 -c 192.168.177.2 ; # 4B64 to Xeon over WireGuard
Connecting to host 192.168.177.2, port 5201
[  5] local 192.168.177.5 port 48926 connected to 192.168.177.2 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  91.2 MBytes   765 Mbits/sec    0    399 KBytes       
[  5]   1.00-2.00   sec  91.9 MBytes   771 Mbits/sec    0    437 KBytes       
[  5]   2.00-3.00   sec  91.4 MBytes   766 Mbits/sec    0    457 KBytes       
[  5]   3.00-4.00   sec  85.6 MBytes   718 Mbits/sec    0    457 KBytes       
[  5]   4.00-5.00   sec  93.2 MBytes   782 Mbits/sec    0    457 KBytes       
[  5]   5.00-6.00   sec  91.7 MBytes   769 Mbits/sec    0    457 KBytes       
[  5]   6.00-7.00   sec  91.9 MBytes   771 Mbits/sec    0    457 KBytes       
[  5]   7.00-8.00   sec  90.7 MBytes   761 Mbits/sec    0    457 KBytes       
[  5]   8.00-9.00   sec  90.9 MBytes   762 Mbits/sec    0    457 KBytes       
[  5]   9.00-10.00  sec  90.4 MBytes   759 Mbits/sec    0    457 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   909 MBytes   762 Mbits/sec    0             sender
[  5]   0.00-10.03  sec   908 MBytes   759 Mbits/sec                  receiver

iperf Done.
$ iperf3 -c d0.wulf ; # 4B64 to Xeon direct no WireGuard
Connecting to host d0.wulf, port 5201
[  5] local 192.168.174.145 port 60542 connected to 192.168.174.150 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   111 MBytes   935 Mbits/sec    0    349 KBytes       
[  5]   1.00-2.00   sec   112 MBytes   938 Mbits/sec    0    349 KBytes       
[  5]   2.00-3.00   sec   112 MBytes   941 Mbits/sec    0    366 KBytes       
[  5]   3.00-4.00   sec   112 MBytes   936 Mbits/sec    0    366 KBytes       
[  5]   4.00-5.00   sec   111 MBytes   935 Mbits/sec    0    366 KBytes       
[  5]   5.00-6.00   sec   112 MBytes   939 Mbits/sec    0    366 KBytes       
[  5]   6.00-7.00   sec   111 MBytes   935 Mbits/sec    0    366 KBytes       
[  5]   7.00-8.00   sec   112 MBytes   938 Mbits/sec    0    366 KBytes       
[  5]   8.00-9.00   sec   112 MBytes   938 Mbits/sec    0    366 KBytes       
[  5]   9.00-10.00  sec   111 MBytes   928 Mbits/sec    0    366 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  1.09 GBytes   936 Mbits/sec    0             sender
[  5]   0.00-10.04  sec  1.09 GBytes   932 Mbits/sec                  receiver

iperf Done.

and the 64-bit Pi 4B to Xeon ping runs were

Code: Select all

$ ping -c 5 192.168.177.2 ; # 4B64 to Xeon over WireGuard
PING 192.168.177.2 (192.168.177.2) 56(84) bytes of data.
64 bytes from 192.168.177.2: icmp_seq=1 ttl=64 time=0.504 ms
64 bytes from 192.168.177.2: icmp_seq=2 ttl=64 time=0.464 ms
64 bytes from 192.168.177.2: icmp_seq=3 ttl=64 time=0.514 ms
64 bytes from 192.168.177.2: icmp_seq=4 ttl=64 time=0.468 ms
64 bytes from 192.168.177.2: icmp_seq=5 ttl=64 time=0.432 ms

--- 192.168.177.2 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4091ms
rtt min/avg/max/mdev = 0.432/0.476/0.514/0.029 ms
$ ping -c 5 d0.wulf ; # 4B64 to Xeon direct no WireGuard
PING d0.wulf (192.168.174.150) 56(84) bytes of data.
64 bytes from d0.wulf (192.168.174.150): icmp_seq=1 ttl=64 time=0.197 ms
64 bytes from d0.wulf (192.168.174.150): icmp_seq=2 ttl=64 time=0.196 ms
64 bytes from d0.wulf (192.168.174.150): icmp_seq=3 ttl=64 time=0.188 ms
64 bytes from d0.wulf (192.168.174.150): icmp_seq=4 ttl=64 time=0.197 ms
64 bytes from d0.wulf (192.168.174.150): icmp_seq=5 ttl=64 time=0.186 ms

--- d0.wulf ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4056ms
rtt min/avg/max/mdev = 0.186/0.192/0.197/0.004 ms

[pianoquintet]

How could you achieve 272Mbps on a 3B when its eth port is limited to 100Mbps?

[epoch1970]

How could you achieve 272Mbps on a 3B when its eth port is limited to 100Mbps?

Because ejolson tested a 3B+?

[ejolson]

How could you achieve 272Mbps on a 3B when its eth port is limited to 100Mbps?

Because ejolson tested a 3B+?

That's right. On the other hand, I don't understand how these timings from the WireGuard website achieve 1011 Mbps over gigabit Ethernet.

The 3B+ is an upgraded version of the 3B mostly known for better thermal and power management. However, the 3B+ also happens to have gigabit Ethernet connected internally to the same USB2 subsystem as before. As a result, networking on the Pi 3B+ achieves between 200 and 300 Mbit--a bit faster than the original model but nowhere near the wire speed of gigabit Ethernet.

Does anyone have numbers for how WireGuard performs over 10 gigabit Ethernet on more modern hardware?

[Yammers]

How do i set this up to connec to have my laptop at one end running wireguard and then connecting to raspi remotley elsewere also running wire gaud, which it the client and which is the server in this case

[epoch1970]

How do i set this up to connec to have my laptop at one end running wireguard and then connecting to raspi remotley elsewere also running wire gaud, which it the client and which is the server in this case

The client initiates the connection to a known public IP address (or name) and identifies itself with its key.
The server accepts connections from known keys and possibly unknown public addresses. The server is most probably behind a router/firewall performing incoming NAT (and "dyn-dns" domain name updates.)

See https://www.wireguard.com/#cryptokey-routing (and titles above and below) to visualise how server nodes differ in configuration from clients.

[ejolson]

How do i set this up to connec to have my laptop at one end running wireguard and then connecting to raspi remotley elsewere also running wire gaud, which it the client and which is the server in this case

The client initiates the connection to a known public IP address (or name) and identifies itself with its key.
The server accepts connections from known keys and possibly unknown public addresses. The server is most probably behind a router/firewall performing incoming NAT (and "dyn-dns" domain name updates.)

See https://www.wireguard.com/#cryptokey-routing (and titles above and below) to visualise how server nodes differ in configuration from clients.

Another WireGuard configuration example is described in the post

https://www.raspberrypi.org/forums/view ... 2#p1544032

That particular case specifically covers connecting two subnets together using two Raspberry Pi computers with static IP addresses. However, if the downstream machine is instead a notebook computer with a dynamic address, simply omit it's IP number from the upstream WireGuard configuration file and create a dummy device to represent the local area network on that side. Instead of adding a dummy network device, it would also be possible to change the upstream routing tables to allow forwarding directly to the laptop's WireGuard device.