Pi 4B: question on OpenVPN performance or speed [RESULTS]

[Josez]

Hello everyone,

I have a 100/10 mbit internet connection.

I'm currently using a Pi 3B+, connected to a local VPN provider using OpenVPN .

The speeds I'm getting with "speedtest-cli" are:
Download: 50 to 60 mbit/s
Upload: 10 mbit/s

When disconeected from the VPN provider I'm getting the full 100 mbits/s download .

I was wondering what speeds are Pi 4B users out there getting?

Anyone can share their experience? Thanks in advance!

[dav4]

I'm also looking for answer for this question. I checked that new CPU doesn't support for AES unfortunately, but vpn performance should be better than on 3b anyway

[jerrm]

Below are the results of "openssl speed" from a Pi4b. In my tests, these are 1.5x to 3x Pi3b+ speeds - but my the 3b+ wasn't running buster. Run "openssl speed" on your current Pi and compare the algorithms used for your VPN connection.

It should give a pretty good idea of what to expect if processing power on your end is the bottleneck.

Code: Select all

OpenSSL 1.1.1c  28 May 2019
built on: Thu May 30 15:27:48 2019 UTC
options:bn(64,32) rc4(char) des(long) aes(partial) blowfish(ptr)
compiler: gcc -fPIC -pthread -Wa,--noexecstack -Wall -Wa,--noexecstack -g -O2 -fdebug-prefix-map=/build/openssl-hL5TK7/openssl-1.1.1c=. -fstack-protector-strong -Wformat -Werror=format-security -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -DNDEBUG -Wdate-time -D_FORTIFY_SOURCE=2
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
md2                  0.00         0.00         0.00         0.00         0.00         0.00
mdc2                 0.00         0.00         0.00         0.00         0.00         0.00
md4              18976.05k    68880.04k   197145.94k   371034.58k   494515.54k   505235.44k
md5              42312.39k   109290.79k   206835.24k   268217.45k   292884.55k   293016.92k
hmac(md5)        14184.84k    48178.60k   129296.38k   224497.79k   284063.06k   289183.10k
sha1             32685.40k    80003.90k   144859.14k   183286.07k   197640.19k   198975.19k
rmd160           13876.69k    39992.98k    85210.54k   118740.65k   133944.66k   134583.64k
rc4             147047.00k   160990.53k   165141.50k   165968.21k   166371.33k   166494.21k
des cbc          28232.93k    29151.53k    29374.29k    29444.44k    29455.70k    29496.66k
des ede3         10516.35k    10711.12k    10740.01k    10731.18k    10701.48k    10747.90k
idea cbc             0.00         0.00         0.00         0.00         0.00         0.00
seed cbc         35363.27k    36874.01k    37119.83k    37235.37k    37374.63k    37317.29k
rc2 cbc          20581.02k    20965.95k    21298.35k    21241.86k    21206.36k    21211.82k
rc5-32/12 cbc        0.00         0.00         0.00         0.00         0.00         0.00
blowfish cbc     46464.31k    51695.30k    53206.61k    53726.21k    54091.78k    54252.89k
cast cbc         43275.08k    45860.10k    46465.19k    46731.26k    46822.74k    46858.24k
aes-128 cbc      75167.95k    80639.13k    84397.38k    84942.17k    85284.18k    85235.03k
aes-192 cbc      66048.10k    70575.55k    72953.86k    73448.45k    73736.19k    73717.08k
aes-256 cbc      58917.59k    62444.86k    64341.25k    64755.03k    64954.37k    64738.65k
camellia-128 cbc    52014.86k    55537.49k    56329.05k    56628.57k    56781.48k    56743.25k
camellia-192 cbc    41130.31k    43302.68k    43750.91k    43960.66k    44056.58k    44029.27k
camellia-256 cbc    41130.09k    43301.74k    43781.29k    43958.95k    44051.11k    44029.27k
sha256           23358.50k    54125.76k    95411.88k   116913.15k   126580.14k   126649.96k
sha512            5026.49k    20147.29k    28161.03k    38076.07k    42664.16k    42860.54k
whirlpool         3185.02k     6565.59k    10713.69k    12731.39k    13473.11k    13527.72k
aes-128 ige      70113.85k    77668.33k    80956.42k    81725.78k    81769.81k    81936.38k
aes-192 ige      62110.45k    67903.04k    70487.21k    71046.49k    71002.79k    71117.48k
aes-256 ige      55755.31k    60432.21k    62414.76k    62925.82k    62810.79k    63114.02k
ghash            57781.16k    62202.77k    64009.56k    64476.84k    64613.03k    64531.11k
rand              1768.25k     6467.14k    19381.48k    38464.81k    52560.75k    53995.97k
                  sign    verify    sign/s verify/s
rsa  512 bits 0.000319s 0.000029s   3137.9  34984.0
rsa 1024 bits 0.001964s 0.000102s    509.1   9817.4
rsa 2048 bits 0.014245s 0.000393s     70.2   2543.0
rsa 3072 bits 0.045799s 0.000867s     21.8   1152.8
rsa 4096 bits 0.104375s 0.001531s      9.6    653.1
rsa 7680 bits 0.646250s 0.005308s      1.5    188.4
rsa 15360 bits 4.993333s 0.021095s      0.2     47.4
                  sign    verify    sign/s verify/s
dsa  512 bits 0.000521s 0.000373s   1919.3   2683.7
dsa 1024 bits 0.001481s 0.001303s    675.3    767.2
dsa 2048 bits 0.005277s 0.004895s    189.5    204.3
                              sign    verify    sign/s verify/s
 160 bits ecdsa (secp160r1)   0.0011s   0.0009s    924.7   1168.4
 192 bits ecdsa (nistp192)   0.0015s   0.0012s    651.3    844.0
 224 bits ecdsa (nistp224)   0.0022s   0.0017s    446.7    593.5
 256 bits ecdsa (nistp256)   0.0002s   0.0006s   4133.1   1602.4
 384 bits ecdsa (nistp384)   0.0081s   0.0056s    123.1    180.1
 521 bits ecdsa (nistp521)   0.0201s   0.0134s     49.7     74.5
 163 bits ecdsa (nistk163)   0.0017s   0.0033s    594.9    302.4
 233 bits ecdsa (nistk233)   0.0033s   0.0065s    304.7    154.6
 283 bits ecdsa (nistk283)   0.0059s   0.0116s    168.7     85.9
 409 bits ecdsa (nistk409)   0.0139s   0.0272s     71.8     36.7
 571 bits ecdsa (nistk571)   0.0326s   0.0636s     30.7     15.7
 163 bits ecdsa (nistb163)   0.0018s   0.0035s    554.1    283.7
 233 bits ecdsa (nistb233)   0.0036s   0.0071s    278.3    141.6
 283 bits ecdsa (nistb283)   0.0066s   0.0130s    151.5     77.1
 409 bits ecdsa (nistb409)   0.0158s   0.0310s     63.2     32.2
 571 bits ecdsa (nistb571)   0.0373s   0.0728s     26.8     13.7
 256 bits ecdsa (brainpoolP256r1)   0.0030s   0.0024s    338.1    423.9
 256 bits ecdsa (brainpoolP256t1)   0.0029s   0.0022s    342.2    457.3
 384 bits ecdsa (brainpoolP384r1)   0.0081s   0.0061s    123.5    163.9
 384 bits ecdsa (brainpoolP384t1)   0.0081s   0.0056s    123.4    177.9
 512 bits ecdsa (brainpoolP512r1)   0.0176s   0.0129s     56.8     77.8
 512 bits ecdsa (brainpoolP512t1)   0.0175s   0.0118s     57.0     85.0
                              op      op/s
 160 bits ecdh (secp160r1)   0.0010s    989.0
 192 bits ecdh (nistp192)   0.0015s    686.3
 224 bits ecdh (nistp224)   0.0021s    474.4
 256 bits ecdh (nistp256)   0.0004s   2344.1
 384 bits ecdh (nistp384)   0.0078s    129.0
 521 bits ecdh (nistp521)   0.0192s     52.1
 163 bits ecdh (nistk163)   0.0016s    618.2
 233 bits ecdh (nistk233)   0.0032s    315.3
 283 bits ecdh (nistk283)   0.0058s    173.4
 409 bits ecdh (nistk409)   0.0135s     74.1
 571 bits ecdh (nistk571)   0.0316s     31.6
 163 bits ecdh (nistb163)   0.0017s    576.3
 233 bits ecdh (nistb233)   0.0035s    285.8
 283 bits ecdh (nistb283)   0.0064s    155.6
 409 bits ecdh (nistb409)   0.0154s     65.0
 571 bits ecdh (nistb571)   0.0362s     27.6
 256 bits ecdh (brainpoolP256r1)   0.0028s    359.5
 256 bits ecdh (brainpoolP256t1)   0.0028s    358.6
 384 bits ecdh (brainpoolP384r1)   0.0077s    129.1
 384 bits ecdh (brainpoolP384t1)   0.0077s    130.0
 512 bits ecdh (brainpoolP512r1)   0.0168s     59.6
 512 bits ecdh (brainpoolP512t1)   0.0167s     59.7
 253 bits ecdh (X25519)   0.0005s   2012.7
 448 bits ecdh (X448)   0.0025s    406.5
                              sign    verify    sign/s verify/s
 253 bits EdDSA (Ed25519)   0.0002s   0.0006s   4844.2   1763.3
 456 bits EdDSA (Ed448)   0.0013s   0.0026s    784.1    380.8

[thatchunkylad198966]

I pay for 200mbps and OpenVPN was giving around 50mbps. NOT connected to the VPN and I max out at 226mbps.

[epoch1970]

Openvpn has always been I/O-bound on Pi, never CPU-bound (excluding tinfoil-hat-sized cypher blocks perhaps)
I understand Pi4 Ethernet networking effective bandwidth is more than 4 times that of 3b+
I think it is safe to assume near wire speed for tunnels over fast symmetric links, 100/100, maybe 200/200.

[Josez]

A nice colleage of mine lent me a Pi 4B 2G Ram, and we tested OpenVPN with it.

Internet connection used for testing: ~1Gb/s down (Pi 4B connected to the router via eth0).

Download speed test results using "speedtest-cli":

* No vpn: ~550 Mb/s
* With vpn: ~60 Mb/s

I can only observe that the Pi 4B does not seem provide an improvement on the throughput of OpenVPN compared to the Pi 3B+.

But please note that this is not a thorough test by any means, this is just a simple user test.

Notes:
The VPN provider is the same one I used with the P3 B+ mentioned on my original post.
The speedtst-cli server used on the tests is the same server, which is actually a server that belongs to the VPN provider we used during testing.
Traceroute to the speedtest server outside the VPN connection was about 12 hops, while it was 3 hops when connected to the VPN provider.
Fresh Raspbian Buster, updated today, latest openvpn version.
CPU usage did not seem to peak during the downloads, monitored with htop.

Please feel free to ask any questions you may have, maybe helps us with this test.

[jerrm]

It doesn't appear the Pi is the bottleneck?

Does the vpn provider claim any sort of top speed? Maybe 60mpbs is all their giving. What are the results from a traditional PC?

[Josez]

On my Windows PC I get 100 mbit/s, which is the max I have at home.

Thus it is not a limit on the VPN provider.

Tested with same VPN provider, connected to the same vpn server and same UDP protocol and port, using their windows app (which is an electron app that runs openvpn in the background). Same speedtest server as used in the tests with the Pi 4 and Pi 3 but via the speedtest.net site.

[epoch1970]

It might be worth repeating that speedtest essentially tests your ISP network: how well you are connected to it, how well it is connected to the rest of the world.
The test, AFAIK, downloads/uploads data from files. In the case of a Pi running off an SD, performance would then normally hover around the SD’s max transfer speeds.
(I think this is what’s happening here)

Also, asymmetrical links can widely influence results. For TCP, if the uplink used to send acknowledgment of packet reception gets full, actual download speed will hover well below theoretical download speed.
In the special case of a VPN, if a server is at home behind an asymmetric link, max download speed for a client will probably be governed by the upload bandwidth of the server’s connection.

[Josez]

It might be worth repeating that speedtest essentially tests your ISP network: how well you are connected to it, how well it is connected to the rest of the world.
The test, AFAIK, downloads/uploads data from files. In the case of a Pi running off an SD, performance would then normally hover around the SD’s max transfer speeds.
(I think this is what’s happening here)

Also, asymmetrical links can widely influence results. For TCP, if the uplink used to send acknowledgment of packet reception gets full, actual download speed will hover well below theoretical download speed.
In the special case of a VPN, if a server is at home behind an asymmetric link, max download speed for a client will probably be governed by the upload bandwidth of the server’s connection.

The download speed without a vpn connection is fine.

It's when a vpn connection is present that the speeds slow down.

So SD card write speed or uplink speed should not be the limiting factor here when things work "fine" outside of a vpn connection.

[epoch1970]

Right. So here it’s not the SD.
You say cpu load is fine. According to “top”, how much cpu does the openvpn process take, approximately?

[Josez]

On the Pi 3B+, load is usually high on the core that is executing openvpn, it reaches about 95%, it could remain stable at 85%+ on downloads which take more time than speedtest.

On the Pi 4B, I observed it reaching between 60%-70% on one core, that's why I mentioned the load seemed fine. Unfortunately I dont have the Pi 4 to run more tests. I'm sorry, I'm not sure how useful my feedback is here.

I'm not sure how accurate a measurement is when observing htop since it updates at intervals, maybe I missed a peak here or there.

[epoch1970]

Well, clearly the pi3b+ is cpu bound and probably the pi4 was too or very close from it...
What cipher do you use?

[jerrm]

Just to be sure, I'm assuming this is all wired ethernet without the vagaries wireless.

Since the 3+ appears cpu bound, I would have expected the 4 to give some improvement even if it didn't reach the same level as the PC.

I'd be curious of a Linux PC's performance vs Windows. It wouldn't surprise me if their Windows config has tweaks the generic openvpn client config doesn't. Maybe try the Linux config file with the standard OpenVPN windows client (https://openvpn.net/community-downloads/).

[Josez]

Yes, all tests with wired ethernet.

I compared the windows openvpn config from my provider vs the one for linux, the only major difference was the sndbuf and rcvbuf, they were much bigger on the windows config. I tried those higher values on the linux config, it didn't improve the speeds.

I'm not sure what you mean by the below, can you please elaborate? Thanks.

I'd be curious of a Linux PC's performance vs Windows. It wouldn't surprise me if their Windows config has tweaks the generic openvpn client config doesn't. Maybe try the Linux config file with the standard OpenVPN windows client (https://openvpn.net/community-downloads/).

[Josez]

I have done the very same tests now with a Udoo x86 board, with the latest Ubuntu.
https://www.udoo.org/udoo-x86/

Speedtest without VPN connection: 97 Mb/s
Speedtest with VPN connection: 96 Mb/s

This rules out that the low speed seen with the Raspberry boards could be due to Linux and/or with the provider.

Please let me know if you need further info on this test.

[jerrm]

At a loss. The Udoo probably AES in hardware, but if the pi is at 70% on the CPU there should be headroom to improve, even if it never reaches the same max speed.

My experience is that OpenVPN is not shy about eating CPU when needed.

[epoch1970]

Does the OP use AES?

[jerrm]

Does the OP use AES?

It doesn't really matter. I would assume that is the most likely preferred protocol since it has the widest support for hardware acceleration. Acceleration could explain some of the difference, but if the OpenVPN core isn't maxed out, the Pi should be able to do better.

[jerrm]

Please let me know if you need further info on this test.

What are the versions of speedtest-cli on each Linux machines (speedtest-cli --version).

I doubt it would make that much of a difference, but try installing the latest speedtest-cli using pip:

Code: Select all

sudo apt-get remove speedtest-cli
hash -r
sudo pip install speedtest-cli

Just saw a post suggesting the version could make a difference - it doesn't for me, but it's an easy thing to check.

[epoch1970]

Well if the type of cipher doesn’t matter on a thread claiming to offer VPN performance [RESULTS] I guess there is nothing more to say.
Also I believe throughput should be tested in a repeatable environment, ie on an Ethernet LAN and not directly over Internet.
Whatever.

[Josez]

Well if the type of cipher doesn’t matter on a thread claiming to offer VPN performance [RESULTS] I guess there is nothing more to say.
Also I believe throughput should be tested in a repeatable environment, ie on an Ethernet LAN and not directly over Internet.
Whatever.

Yes, cipher used is AES-256-GCM.

Why do you think this should be tested in a test environment as opposed to real life application? As a final user, I care about the real life application.

[Josez]

Please let me know if you need further info on this test.

What are the versions of speedtest-cli on each Linux machines (speedtest-cli --version).

I doubt it would make that much of a difference, but try installing the latest speedtest-cli using pip:

Code: Select all

sudo apt-get remove speedtest-cli
hash -r
sudo pip install speedtest-cli

Just saw a post suggesting the version could make a difference - it doesn't for me, but it's an easy thing to check.

I am using v1.0. I see it's and old version. I could try changing version, but why would the version matter if using the same version the Pi maxes out without VPN, and it gets half the speed with openvpn?

[epoch1970]

Well if the type of cipher doesn’t matter on a thread claiming to offer VPN performance [RESULTS] I guess there is nothing more to say.
Also I believe throughput should be tested in a repeatable environment, ie on an Ethernet LAN and not directly over Internet.
Whatever.

Yes, cipher used is AES-256-GCM.

Why do you think this should be tested in a test environment as opposed to real life application? As a final user, I care about the real life application.

Because then your results only have meaning for yourself. Which is fine, but then don’t publish them as anything else but anecdotal data.

[Josez]

Because then your results only have meaning for yourself. Which is fine, but then don’t publish them as anything else but anecdotal data.

I'm sorry I don't agree that these results would only have meaning for myself. These results should have meaning for anyone who wishes to use a Pi 4 to connect to a VPN provider, to a real life one, on the internet, example as a VPN gateway.

I tested as much as it is to my disposal, this is by no means a full comprehensive ultimate VPN test, but a real one nonetheless.