popopi
Posts: 5
Joined: Wed Aug 12, 2020 3:31 pm

rPi 4: Network speed capping to 100mbit from time to time

Wed Aug 12, 2020 3:51 pm

Hello,
I run a small NAS in my rPi 4. All the equipment involved is 1000mbps (switch, router, cat 6 cabling, USB3 WD easystore).

However, from time to time, I notice that my rPi4 4Gb caps the ethernet speed to 100mbps without any evident reason.

This limitation is seem both in Samba network transfers (limited to 11,2MB/s) as in the applications that run in the Pi (speedtest won't go past 100mbps in a 300mbps cabled connection).
Capped Network Pi4.png
Capped Network Pi4.png (12.8 KiB) Viewed 257 times
Speedtest.png
Speedtest.png (20.15 KiB) Viewed 257 times
After a reboot, sometimes the network speeds go back to normal.
Normal Network Pi4.png
Normal Network Pi4.png (11.93 KiB) Viewed 257 times
I've gone through a bunch of posts relating similar issues, but couldn't find a definitive method to solve this issue (and why this happens, which is a mystery).

This is installation is very recent (only a few days).

Should I just set a cron job to reboot the machine everyday?

epoch1970
Posts: 5438
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: rPi 4: Network speed capping to 100mbit from time to time

Wed Aug 12, 2020 4:30 pm

Speed test is relevant to your ISP connection, nothing else.
Samba involves storage I/Os in addition to network I/Os and I think there are some options that can speed up transfers. Something to do with “large_files” or big blocks... I don’t ever use samba, this is just something I picked from a thread. I would look at the config file and man page for samba.

Also, if power is lacking it seems possible the machine negotiates 100mbps instead of 1000.
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

dustnbone
Posts: 326
Joined: Tue Nov 05, 2019 2:49 am

Re: rPi 4: Network speed capping to 100mbit from time to time

Wed Aug 12, 2020 6:15 pm

Have you tested other cables? 100mbps only requires 2 pairs of wire, while 1000mbps requires 4 pairs. If there's an intermittent fault in a conductor/connector it's possible that it's automatically falling back to 100mbps, rather than fail entirely.

Low voltage is also a possibility as mentioned above, but much more difficult to rule out than a bad cable.

User avatar
mooblie
Posts: 207
Joined: Fri Oct 14, 2016 2:07 pm
Location: The Scottish Highlands

Re: rPi 4: Network speed capping to 100mbit from time to time

Wed Aug 12, 2020 6:49 pm

This command should tell you the current negotiated speed:

Code: Select all

sudo ethtool eth0
...although I'm not sure how often, or under what circumstances, the information is re-sampled though.

trejan
Posts: 2332
Joined: Tue Jul 02, 2019 2:28 pm

Re: rPi 4: Network speed capping to 100mbit from time to time

Wed Aug 12, 2020 6:52 pm

Run "dmesg -T | grep Link" and see if there are lots of Link is Up/Down messages. It'll say what speed it is negotiating as well.

User avatar
rpdom
Posts: 17445
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: rPi 4: Network speed capping to 100mbit from time to time

Wed Aug 12, 2020 6:52 pm

mooblie wrote:
Wed Aug 12, 2020 6:49 pm
This command should tell you the current negotiated speed:

Code: Select all

sudo ethtool eth0
...although I'm not sure how often, or under what circumstances, the information is re-sampled though.
It should pick up any changes immediately.

You don't need sudo to use ethtool to query an interface BTW, only if you want to make any changes to the settings.
Unreadable squiggle

pidd
Posts: 932
Joined: Fri May 29, 2020 8:29 pm
Location: Birkenhead, Wirral, UK
Contact: Website

Re: rPi 4: Network speed capping to 100mbit from time to time

Wed Aug 12, 2020 7:23 pm

If you can bypass the switch, try that, some aren't so good on their power-saving intelligence.

popopi
Posts: 5
Joined: Wed Aug 12, 2020 3:31 pm

Re: rPi 4: Network speed capping to 100mbit from time to time

Wed Aug 12, 2020 9:21 pm

pidd wrote: If you can bypass the switch, try that, some aren't so good on their power-saving intelligence.
I see. I can't test it right away, since there's only one cable leading to the router and many other devices rely on that switch... But I'll keep that in mind.
trejan wrote: Run "dmesg -T | grep Link" and see if there are lots of Link is Up/Down messages. It'll say what speed it is negotiating as well.
After the reboot it has kept a 1Gbps connection.

Code: Select all

pi@raspberrypi:~ $ dmesg -T | grep Link
[Wed Aug 12 16:42:02 2020] sdhci-iproc fe340000.emmc2: Linked as a consumer to r                                       egulator.3
[Wed Aug 12 16:42:02 2020] sdhci-iproc fe340000.emmc2: Linked as a consumer to r                                       egulator.4
[Wed Aug 12 16:42:21 2020] bcmgenet fd580000.genet eth0: Link is Down
[Wed Aug 12 16:42:25 2020] bcmgenet fd580000.genet eth0: Link is Up - 1Gbps/Full                                        - flow control rx/tx
These events happened 6 hours ago.
If the connection goes down again, I'll run the command to see if it negotiates 100Mbps and when it happens.
dustnbone wrote: Have you tested other cables? 100mbps only requires 2 pairs of wire, while 1000mbps requires 4 pairs. If there's an intermittent fault in a conductor/connector it's possible that it's automatically falling back to 100mbps, rather than fail entirely.

Low voltage is also a possibility as mentioned above, but much more difficult to rule out than a bad cable.
I haven't tested another cable. It's a short 50cm cable to the switch and I could try another one. However, it's working fine for 6 hours now, and I'm using the full band of the gigabit without intermitence, as you can see in the log above.
epoch1970 wrote: Speed test is relevant to your ISP connection, nothing else.
Samba involves storage I/Os in addition to network I/Os and I think there are some options that can speed up transfers. Something to do with “large_files” or big blocks... I don’t ever use samba, this is just something I picked from a thread. I would look at the config file and man page for samba.

Also, if power is lacking it seems possible the machine negotiates 100mbps instead of 1000.
I see. After a reboot the speedtest got 300mbit as expected.
Samba is also working fine with big and small transfers, now capping at 110MB/s, instead of the 11MB/s before the reboot.

trejan
Posts: 2332
Joined: Tue Jul 02, 2019 2:28 pm

Re: rPi 4: Network speed capping to 100mbit from time to time

Wed Aug 12, 2020 9:28 pm

popopi wrote:
Wed Aug 12, 2020 9:21 pm
If the connection goes down again, I'll run the command to see if it negotiates 100Mbps and when it happens.
Run

Code: Select all

sudo zgrep Link /var/log/messages*
Ignore the ones with the low numbers in the brackets as that would be when the Pi is booting. If you want to filter those out then

Code: Select all

sudo zgrep Link /var/log/messages* | grep -v "[   "

popopi
Posts: 5
Joined: Wed Aug 12, 2020 3:31 pm

Re: rPi 4: Network speed capping to 100mbit from time to time

Wed Aug 12, 2020 9:34 pm

trejan wrote:
Wed Aug 12, 2020 9:28 pm
popopi wrote:
Wed Aug 12, 2020 9:21 pm
If the connection goes down again, I'll run the command to see if it negotiates 100Mbps and when it happens.
Run

Code: Select all

sudo zgrep Link /var/log/messages*
Ignore the ones with the low numbers in the brackets as that would be when the Pi is booting. If you want to filter those out then

Code: Select all

sudo zgrep Link /var/log/messages* | grep -v "[   "
The log mentions a 100Mbps link in August 9, but no mention of it more recently (even though it's possible that it's been running 100Mbit for a few days without me noticing).

Aug 12 16:42~ is the reboot.

Code: Select all

pi@raspberrypi:~ $ sudo zgrep Link /var/log/messages*
/var/log/messages:Aug  9 21:52:51 raspberrypi kernel: [169752.111493] bcmgenet fd580000.genet eth0: Link is Down
/var/log/messages:Aug  9 21:52:53 raspberrypi kernel: [169754.191589] bcmgenet fd580000.genet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
/var/log/messages:Aug  9 21:52:58 raspberrypi kernel: [169759.391655] bcmgenet fd580000.genet eth0: Link is Down
/var/log/messages:Aug  9 21:53:11 raspberrypi kernel: [169772.911773] bcmgenet fd580000.genet eth0: Link is Up - 100Mbps/Full - flow control rx/tx
/var/log/messages:Aug 12 16:42:05 raspberrypi kernel: [    0.669977] sdhci-iproc fe340000.emmc2: Linked as a consumer to regulator.3
/var/log/messages:Aug 12 16:42:05 raspberrypi kernel: [    0.672204] sdhci-iproc fe340000.emmc2: Linked as a consumer to regulator.4
/var/log/messages:Aug 12 16:42:06 raspberrypi kernel: [   19.274082] bcmgenet fd580000.genet eth0: Link is Down
/var/log/messages:Aug 12 16:42:11 raspberrypi kernel: [   23.434110] bcmgenet fd580000.genet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
/var/log/messages.1:May 27 08:24:30 raspberrypi kernel: [    0.701078] sdhci-iproc fe340000.emmc2: Linked as a consumer to regulator.3
/var/log/messages.1:May 27 08:24:30 raspberrypi kernel: [    0.703534] sdhci-iproc fe340000.emmc2: Linked as a consumer to regulator.4
/var/log/messages.1:May 27 08:24:33 raspberrypi kernel: [    9.848053] bcmgenet fd580000.genet eth0: Link is Down
/var/log/messages.1:May 27 08:24:37 raspberrypi kernel: [   14.018198] bcmgenet fd580000.genet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
/var/log/messages.1:Aug  7 17:50:00 raspberrypi kernel: [    0.701209] sdhci-iproc fe340000.emmc2: Linked as a consumer to regulator.3
/var/log/messages.1:Aug  7 17:50:00 raspberrypi kernel: [    0.703759] sdhci-iproc fe340000.emmc2: Linked as a consumer to regulator.4
/var/log/messages.1:Aug  7 17:50:02 raspberrypi kernel: [    7.928103] bcmgenet fd580000.genet eth0: Link is Down
/var/log/messages.1:Aug  7 17:50:06 raspberrypi kernel: [   12.088282] bcmgenet fd580000.genet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
/var/log/messages.1:Aug  7 18:03:50 raspberrypi kernel: [    0.701131] sdhci-iproc fe340000.emmc2: Linked as a consumer to regulator.3
/var/log/messages.1:Aug  7 18:03:50 raspberrypi kernel: [    0.703668] sdhci-iproc fe340000.emmc2: Linked as a consumer to regulator.4
/var/log/messages.1:Aug  7 18:03:51 raspberrypi kernel: [   29.208086] bcmgenet fd580000.genet eth0: Link is Down
/var/log/messages.1:Aug  7 18:03:56 raspberrypi kernel: [   33.368146] bcmgenet fd580000.genet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
/var/log/messages.1:Aug  7 18:23:06 raspberrypi kernel: [ 1170.090135] bcmgenet fd580000.genet eth0: Link is Down
/var/log/messages.1:Aug  7 18:23:10 raspberrypi kernel: [ 1173.210029] bcmgenet fd580000.genet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
/var/log/messages.1:Aug  7 18:23:17 raspberrypi kernel: [ 1180.490029] bcmgenet fd580000.genet eth0: Link is Down
/var/log/messages.1:Aug  7 18:23:20 raspberrypi kernel: [ 1183.610043] bcmgenet fd580000.genet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
/var/log/messages.1:Aug  7 18:23:24 raspberrypi kernel: [ 1187.770115] bcmgenet fd580000.genet eth0: Link is Down
/var/log/messages.1:Aug  7 18:23:27 raspberrypi kernel: [ 1190.890267] bcmgenet fd580000.genet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
/var/log/messages.1:Aug  7 18:23:36 raspberrypi kernel: [ 1199.210060] bcmgenet fd580000.genet eth0: Link is Down
/var/log/messages.1:Aug  7 18:23:38 raspberrypi kernel: [ 1201.290084] bcmgenet fd580000.genet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
/var/log/messages.1:Aug  7 20:17:44 raspberrypi kernel: [ 8047.729320] bcmgenet fd580000.genet eth0: Link is Down
/var/log/messages.1:Aug  7 20:17:48 raspberrypi kernel: [ 8051.889421] bcmgenet fd580000.genet eth0: Link is Up - 1Gbps/Full - flow control rx/tx
/var/log/messages.1:Aug  7 22:43:45 raspberrypi kernel: [    0.669969] sdhci-iproc fe340000.emmc2: Linked as a consumer to regulator.3
/var/log/messages.1:Aug  7 22:43:45 raspberrypi kernel: [    0.672241] sdhci-iproc fe340000.emmc2: Linked as a consumer to regulator.4
/var/log/messages.1:Aug  7 22:43:47 raspberrypi kernel: [   19.274001] bcmgenet fd580000.genet eth0: Link is Down
/var/log/messages.1:Aug  7 22:43:51 raspberrypi kernel: [   23.434035] bcmgenet fd580000.genet eth0: Link is Up - 1Gbps/Full - flow control rx/tx

trejan
Posts: 2332
Joined: Tue Jul 02, 2019 2:28 pm

Re: rPi 4: Network speed capping to 100mbit from time to time

Wed Aug 12, 2020 9:59 pm

popopi wrote:
Wed Aug 12, 2020 9:34 pm
The log mentions a 100Mbps link in August 9, but no mention of it more recently (even though it's possible that it's been running 100Mbit for a few days without me noticing).

Aug 12 16:42~ is the reboot.
Yeah. It looks like it dropped to 100Mbps from the 9th at ~21:52 and stayed like that until you rebooted it on the 12th. Once the Ethernet interface has negotiated a speed, it won't renegotiate unless the connection is lost so it will stay at 100Mbps. There were also two periods of bad connection on the 7th at 18:23 and 20:17.

Did anybody knock the Pi, move network cabling or touch the switch at any of these times? I've not noticed any problems with negotiation with the Pi 4 and I don't remember seeing any posts about it recently either.

dustnbone
Posts: 326
Joined: Tue Nov 05, 2019 2:49 am

Re: rPi 4: Network speed capping to 100mbit from time to time

Wed Aug 12, 2020 10:09 pm

I'd try another cable for sure. If there's a faulty connection somewhere the difference between working and not could literally be a few degrees of thermal expansion/contraction, or a tiny alteration in the way the cable bends.

It takes seconds to swap a cable, and that is by far the most common cause of the symptoms you are describing.

popopi
Posts: 5
Joined: Wed Aug 12, 2020 3:31 pm

Re: rPi 4: Network speed capping to 100mbit from time to time

Thu Aug 13, 2020 12:16 am

dustnbone wrote: I'd try another cable for sure. If there's a faulty connection somewhere the difference between working and not could literally be a few degrees of thermal expansion/contraction, or a tiny alteration in the way the cable bends.

It takes seconds to swap a cable, and that is by far the most common cause of the symptoms you are describing.
I'll do some testing for a few days and observe! Thanks.
trejan wrote: Did anybody knock the Pi, move network cabling or touch the switch at any of these times? I've not noticed any problems with negotiation with the Pi 4 and I don't remember seeing any posts about it recently either.
Not that I remember.
It's interesting to see that it negotiated 1Gbps and a few seconds later, lost connection and negotiated 100Mbps.

As a workaround, maybe I could set a cron job to force renegotiations once in a while (how to do this?)...

trejan
Posts: 2332
Joined: Tue Jul 02, 2019 2:28 pm

Re: rPi 4: Network speed capping to 100mbit from time to time

Thu Aug 13, 2020 12:58 am

popopi wrote:
Thu Aug 13, 2020 12:16 am
It's interesting to see that it negotiated 1Gbps and a few seconds later, lost connection and negotiated 100Mbps.
It usually indicates a problem with interference or a bad connection. Enough errors accumulate that it drops the link. Tries bring up the link again so starts auto-negotiation again. Negotiates 1Gbps. More errors occur and the link drops again. This time auto-negotiation decides against 1Gbps because it is still seeing a bad connection and drops down to 100Mbps which is slower + only uses 2 pairs out of the 4 available.

You've rebooted now so wiped the interface counters already so can't check what happened previously. Run "sudo ethtool -S eth0" and look at counters. You're mainly interested in errors, fcs, unknown, align and carrier.
popopi wrote:
Thu Aug 13, 2020 12:16 am
As a workaround, maybe I could set a cron job to force renegotiations once in a while (how to do this?)...
This will drop and bring up the link again

Code: Select all

/sbin/ip link set eth0 down; /sbin/ip link set eth0 up
There is no guarantee that it will negotiate 1Gbps. You can limit its choices to 1Gbps only but if the link really is bad then it won't ever come up. Do not disable auto-negotiation entirely as it is required for gigabit.

I'd strongly recommend finding the cause of the bad signal though and not using a cronjob. There will be a noticeable pause on all connections every time you run this and some connections may disconnect entirely. If you do want to run this as a cronjob then wrap it with a call to ethtool to check the current speed.

popopi
Posts: 5
Joined: Wed Aug 12, 2020 3:31 pm

Re: rPi 4: Network speed capping to 100mbit from time to time

Thu Aug 13, 2020 2:55 am

trejan wrote:
Thu Aug 13, 2020 12:58 am
You've rebooted now so wiped the interface counters already so can't check what happened previously. Run "sudo ethtool -S eth0" and look at counters. You're mainly interested in errors, fcs, unknown, align and carrier.
This is what I get from that command. I'm not familiar with the details, but "errors" and "dropped" indicators seem to be on 0.

I'll keep an eye on the hardware side of things, change the ethernet cable from the Pi to the switch if the speed falls down again, and maybe even pull a long cable directly to the router.

Code: Select all

pi@raspberrypi:~ $ sudo ethtool -S eth0
NIC statistics:
     rx_packets: 8274067
     tx_packets: 50949228
     rx_bytes: 1178937720
     tx_bytes: 3560508426
     rx_errors: 0
     tx_errors: 0
     rx_dropped: 0
     tx_dropped: 0
     multicast: 31230
     rx_64_octets: 7480415
     rx_65_127_oct: 143866
     rx_128_255_oct: 137204
     rx_256_511_oct: 47149
     rx_512_1023_oct: 12629
     rx_1024_1518_oct: 462037
     rx_vlan_1519_1522_oct: 0
     rx_1522_2047_oct: 0
     rx_2048_4095_oct: 0
     rx_4096_9216_oct: 0
     rx_pkts: 8283300
     rx_bytes: 1212631640
     rx_multicast: 40305
     rx_broadcast: 5001
     rx_fcs: 0
     rx_control: 8560
     rx_pause: 8560
     rx_unknown: 0
     rx_align: 0
     rx_outrange: 0
     rx_code: 0
     rx_carrier: 0
     rx_oversize: 0
     rx_jabber: 0
     rx_mtu_err: 0
     rx_good_pkts: 8283300
     rx_unicast: 8237994
     rx_ppp: 0
     rx_crc: 0
     tx_64_octets: 73817
     tx_65_127_oct: 158087
     tx_128_255_oct: 83078
     tx_256_511_oct: 45993
     tx_512_1023_oct: 51432
     tx_1024_1518_oct: 50536821
     tx_vlan_1519_1522_oct: 0
     tx_1522_2047_oct: 0
     tx_2048_4095_oct: 0
     tx_4096_9216_oct: 0
     tx_pkts: 50949228
     tx_multicast: 5503
     tx_broadcast: 16033
     tx_pause: 0
     tx_control: 0
     tx_fcs_err: 0
     tx_oversize: 0
     tx_defer: 0
     tx_excess_defer: 0
     tx_single_col: 0
     tx_multi_col: 0
     tx_late_col: 0
     tx_excess_col: 0
     tx_frags: 0
     tx_total_col: 0
     tx_jabber: 0
     tx_bytes: 3764782774
     tx_good_pkts: 50949228
     tx_unicast: 50927692
     rx_runt_pkts: 0
     rx_runt_valid_fcs: 0
     rx_runt_inval_fcs_align: 0
     rx_runt_bytes: 0
     rbuf_ovflow_cnt: 0
     rbuf_err_cnt: 0
     mdf_err_cnt: 11475
     alloc_rx_buff_failed: 0
     rx_dma_failed: 0
     tx_dma_failed: 0
     txq0_packets: 24781553
     txq0_bytes: 3076378443
     rxq0_bytes: 0
     rxq0_packets: 0
     rxq0_errors: 0
     rxq0_dropped: 0
     txq1_packets: 407168
     txq1_bytes: 515681016
     rxq1_bytes: 0
     rxq1_packets: 0
     rxq1_errors: 0
     rxq1_dropped: 0
     txq2_packets: 207279
     txq2_bytes: 252793197
     rxq2_bytes: 0
     rxq2_packets: 0
     rxq2_errors: 0
     rxq2_dropped: 0
     txq3_packets: 25295997
     txq3_bytes: 3704350890
     rxq3_bytes: 0
     rxq3_packets: 0
     rxq3_errors: 0
     rxq3_dropped: 0
     txq16_packets: 257231
     txq16_bytes: 306272176
     rxq16_bytes: 1178937794
     rxq16_packets: 8274068
     rxq16_errors: 0
     rxq16_dropped: 0
pi@raspberrypi:~ $

Return to “Troubleshooting”