smishra12345
Posts: 1
Joined: Tue Jul 02, 2019 11:24 pm

Increasing MTU on Pi 4

Tue Jul 02, 2019 11:27 pm

I tried increasing the MTU on Pi 4 from the default 1500 but it would not let me do so.

With a Gigabit Ethernet, higher MTU values would be very helpful.

Can we recompile the kernel/driver and increase the MTU ?

epoch1970
Posts: 5896
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: Increasing MTU on Pi 4

Wed Jul 03, 2019 2:22 pm

I don’t recall MTU as a Kconfig parameter so recompiling the kernel is possibly not an option.
In any case defaulting to jumbo seems a questionable choice since mtu 9000 (?) is not a standard and regular Ethernet devices will not be able to handle the large payload and will drop it...
Changing the mtu per interface with the “ip” command seems a better idea.

This being said if the Ethernet hardware cannot handle large frames, the point is moot. Do you know the max frame size supported by the Ethernet adapter on pi4? (I don’t.)
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

trejan
Posts: 3062
Joined: Tue Jul 02, 2019 2:28 pm

Re: Increasing MTU on Pi 4

Thu Jul 04, 2019 1:52 am

smishra12345 wrote:
Tue Jul 02, 2019 11:27 pm
Can we recompile the kernel/driver and increase the MTU ?
The Broadcom GENET driver doesn't have any support for setting MTU so it'll involve more than just recompiling it assuming the hardware even supports > 1500. The init_umac function does set a maximum frame length but I've no idea what happens if you increase that number and whether you need to mess about the ring buffers or DMA configuration as well.

jdb
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 2474
Joined: Thu Jul 11, 2013 2:37 pm

Re: Increasing MTU on Pi 4

Thu Jul 04, 2019 10:32 am

In theory the hardware supports up to 16k packet sizes but increasing the MTU will, as you say, need tweaks to the packet buffer configuration. My hunch is that the buffers need to end up 4k page-sized for compatibility with the rest of Linux which means jumbo frames span more than 1 tx or rx buffer. The driver already supports generic segmentation offload so I assume it can handle multi-buffer tx.
Rockets are loud.
https://astro-pi.org

Muny
Posts: 1
Joined: Wed Sep 11, 2019 10:17 pm

Re: Increasing MTU on Pi 4

Wed Sep 11, 2019 10:19 pm

Has anyone had any success with increasing the MTU on their Pi 4?
I'm trying to get larger packets working so I can get better performance out of my GigE vision camera.

akrantz01
Posts: 1
Joined: Fri Nov 22, 2019 12:28 am

Re: Increasing MTU on Pi 4

Fri Nov 22, 2019 12:30 am

Has there been any progress on this? I am having the same issue when trying to set up BATMAN-adv.

User avatar
ehem
Posts: 61
Joined: Mon Sep 09, 2019 1:17 am

Re: Increasing MTU on Pi 4

Fri Nov 22, 2019 2:35 am

Well, does Raspian ship with the "ifupdown" package? Have you tried `ifdown eth0`, then in /etc/network/interfaces, under "iface eth0" adding "mtu 9000" and then doing `ifup eth0`?

If your network uses DHCP, then DHCP option 26 can be used to specify the frame size of the network (I can confirm this works with at least some DHCP client programs). A bigger issue is many switches have a fixed limit of 1500 bytes. You will need a noticeably higher-end switch for 9K packets. Mine is at the bottom end of this category and sending a jumbo frame after a long gap of no jumbo frames for a while causes it to temporarily drop all packets for a bit (OS bug on a switch, wonderful).

jdb wrote:
Thu Jul 04, 2019 10:32 am
In theory the hardware supports up to 16k packet sizes but increasing the MTU will, as you say, need tweaks to the packet buffer configuration. My hunch is that the buffers need to end up 4k page-sized for compatibility with the rest of Linux which means jumbo frames span more than 1 tx or rx buffer. The driver already supports generic segmentation offload so I assume it can handle multi-buffer tx.
I was wanting to claim the standards for gigabit ethernet required all gigabit computer NICs to support jumbo frames, yet a quick search suggests this may not be the case. I'm wondering if perhaps the hardware of gigabit ethernet NICs is required to support jumbo frames, but many drivers do not...

trejan
Posts: 3062
Joined: Tue Jul 02, 2019 2:28 pm

Re: Increasing MTU on Pi 4

Fri Nov 22, 2019 3:22 am

akrantz01 wrote:
Fri Nov 22, 2019 12:30 am
Has there been any progress on this? I am having the same issue when trying to set up BATMAN-adv.
No change. The bcmgenet driver in the latest kernel doesn't have any support for setting the MTU.

trejan
Posts: 3062
Joined: Tue Jul 02, 2019 2:28 pm

Re: Increasing MTU on Pi 4

Fri Nov 22, 2019 3:29 am

ehem wrote:
Fri Nov 22, 2019 2:35 am
I was wanting to claim the standards for gigabit ethernet required all gigabit computer NICs to support jumbo frames, yet a quick search suggests this may not be the case. I'm wondering if perhaps the hardware of gigabit ethernet NICs is required to support jumbo frames, but many drivers do not...
There is no requirement to support jumbo frames. I've seen controllers that only supported the stock 1500 MTU and some that only supported "baby jumbo" where it'll take 1518-1522. Even the controllers that do support jumbo frames don't have any guarantee as to how big it is with some as low as 2000.

wang_Bob
Posts: 1
Joined: Wed May 06, 2020 5:55 am

Re: Increasing MTU on Pi 4

Wed May 06, 2020 6:04 am

Have you sloved the problem?
Now,I'm trying to connect a gigabit camera, but the frame was badly missing.

trejan
Posts: 3062
Joined: Tue Jul 02, 2019 2:28 pm

Re: Increasing MTU on Pi 4

Wed May 06, 2020 1:36 pm

wang_Bob wrote:
Wed May 06, 2020 6:04 am
Have you sloved the problem?
Now,I'm trying to connect a gigabit camera, but the frame was badly missing.
No. The bcmgenet driver still doesn't have any support for changing MTU.

thomaage
Posts: 1
Joined: Fri May 22, 2020 5:04 am

Re: Increasing MTU on Pi 4

Fri May 22, 2020 5:11 am

Muny wrote:
Wed Sep 11, 2019 10:19 pm
Has anyone had any success with increasing the MTU on their Pi 4?
I'm trying to get larger packets working so I can get better performance out of my GigE vision camera.
Did you find a solution to this? I'm trying to solve the same problem. Would a USB3 to Ethernet converter do the trick?

waryishe
Posts: 2
Joined: Sat May 23, 2020 7:36 pm

Re: Increasing MTU on Pi 4

Sat May 23, 2020 8:00 pm

If your willing to get your hands dirty and compile a kernel of your own, you can get jumbo frames working. In the file include/linux/if_vlan.h change the lines

Code: Select all

#define VLAN_ETH_DATA_LEN	1500
#define VLAN_ETH_FRAME_LEN	1518
to

Code: Select all

#define VLAN_ETH_DATA_LEN	9000
#define VLAN_ETH_FRAME_LEN	9018
In include/uapi/linux/if_ether.h change

Code: Select all

#define ETH_DATA_LEN	1500
#define ETH_FRAME_LEN	1514
to

Code: Select all

#define ETH_DATA_LEN	9000
#define ETH_FRAME_LEN	9014
and finally change drivers/net/ethernet/broadcom/genet/bcmgenet.c

Code: Select all

#define RX_BUF_LENGTH		2048
to

Code: Select all

#define RX_BUF_LENGTH		10240
compile and you will have fully functional jumbo frames. I used 9000 as the limit because that works with all the gear on my network. Beware that you are changeing the default mtu for EVERYTHING. Some other network drivers will not like that. But if all you are using is the on-board ethernet, this should work. Iperf gives me 990Mbit after the changes. Very close to line speed.

SMAHH
Posts: 1
Joined: Wed Jun 03, 2020 1:27 pm

Re: Increasing MTU on Pi 4

Wed Jun 03, 2020 2:53 pm

Thank you very much for the information.
It seems to be that entries help to fix the issue partly.
I have done some more tests after I have compiled a new kernel with those entries.

I have tried to transfer image data form a GigE camera via UDP protocol. It works only up to 2000 bytes packet size even MTU is set to 9000 on pi.

I had a look with wireshark at the switch port pi connected to and on the PI. I can see the packet gets damaged on PI when I send the packet > 2000bytes.

Please have a look into the attached files.
Attachments
PI_MTU9000.zip
(116.39 KiB) Downloaded 42 times
Capture.PNG
Capture.PNG (100.32 KiB) Viewed 3728 times

waryishe
Posts: 2
Joined: Sat May 23, 2020 7:36 pm

Re: Increasing MTU on Pi 4

Tue Jun 09, 2020 5:54 pm

I'm not sure why it is having trouble after 2000 byte size? Are you sure its going over the ethernet and not wifi? Wifi has a mtu around 2k. Try running this command on the pi

Code: Select all

 ping -M do -s 8972 IPADDRESS
Replace IPADDRESS with a computer on your network. If you get normal ping responses then it's not the network drivers issue. Also are you running in 32 bit or 64 bit mode? I only checked it in 64 bit, but i didn't think that would make a difference.

User avatar
dickon
Posts: 1876
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: Increasing MTU on Pi 4

Tue Jun 09, 2020 7:44 pm

Ethernet has an MTU of 1500 bytes. Anything more is considered 'jumbo frames', and may or may not be supported. 9000B is common, but I'm not at all surprised to see a ~2000B limit. Gives you 48B to play with for metadata.
As it is apparently board policy to disallow any criticism of anything, as it appears to criticise something is to criticise all the users of that something, I will no longer be commenting in threads which are not directly relevant to my uses of the Pi.

JohnDuffy
Posts: 19
Joined: Tue Jul 14, 2020 7:04 pm

Re: Increasing MTU on Pi 4

Tue Jul 14, 2020 7:13 pm

Thanks for posting the information on how to change the MTU setting in the kernel.

I've made the changes, compiled a new kernel and very happily have an MTU of 9000.

I was wondering where the 10240 value comes from for the RX_BUF_LENGTH? Is this the maximum that can be set for the driver? And I was wondering whether, for maximum performance, this value (or a multiple) should be reflected in any kernel TCP parameters, such as 'tcp_rmem'? Or are they unrelated?

Kind regards

iandk
Posts: 59
Joined: Tue Sep 10, 2019 4:42 pm

Re: Increasing MTU on Pi 4

Fri Oct 09, 2020 8:05 pm

Any plans to support this out of the box?

User avatar
DougieLawson
Posts: 40510
Joined: Sun Jun 16, 2013 11:19 pm
Location: A small cave in deepest darkest Basingstoke, UK
Contact: Website Twitter

Re: Increasing MTU on Pi 4

Sat Oct 10, 2020 12:08 pm

iandk wrote:
Fri Oct 09, 2020 8:05 pm
Any plans to support this out of the box?
Is packet fragmentation a big problem for you? My network works with MTU 1500 on every device.
If you support it out of the box you can't set big packets as default because that will break some folk's networks.
Any language using left-hand whitespace for syntax is ridiculous

Any DMs sent on Twitter will be answered next month.
Fake doctors - are all on my foes list.

Any requirement to use a crystal ball or mind reading will result in me ignoring your question.

User avatar
geerlingguy
Posts: 195
Joined: Sun Feb 15, 2015 3:43 am
Location: St. Louis, MO, USA
Contact: Website Twitter YouTube

Re: Increasing MTU on Pi 4

Wed Nov 04, 2020 4:27 am

Just wanted to follow up and say thanks for posting this—I did this while testing the CM4 with a 4-interface Intel PCIe NIC (plus the internal interface), and went from getting a maximum of 3.4 Gbps (with overclock) across all interfaces, to getting 4.15 Gbps! (details are in this GitHub issue for those interested).
The question is not whether something should be done on a Raspberry Pi, it is whether it can be done on a Raspberry Pi.

User avatar
graysky
Posts: 155
Joined: Fri Apr 05, 2013 11:43 am
Location: /run/user/1000
Contact: Website

Re: Increasing MTU on Pi 4

Sat Nov 14, 2020 2:44 pm

Was wondering if anyone else tried testing different MTU values for throughput on a RPi4. Based on the post above by waryishe, I made this patch to enable them in the 5.4.y series of kernel.

Upon booting into 5.4.75 with the patch applied, I ran some tests using rsync to copy 10 x 330 MB unique files from my x86_64 box to the RPi4 using different MTU values on each end, 1500, 4000, and 9000. With 10 files per test, I could get some stats around the time it took. What I found was that a value of 1500 gave the fastest, 4000 was slower (about 12%), and 9000 was slower yet (about 34%).

Here is the script I used to do generate the unique files and the timing of the rsync transfer if others want to play with it.

Conclusion: mtu values of 4000 and 9000 are inferior to 1500 for copying files of 330 MB.

Stats on the time which is in seconds:
For 1500:

Code: Select all

# NumSamples = 10; Min = 4.73; Max = 5.33
# Mean = 4.911205; Variance = 0.040310; SD = 0.200774; Median 4.818305
For 4000:

Code: Select all

# NumSamples = 10; Min = 5.31; Max = 5.95
# Mean = 5.481886; Variance = 0.039424; SD = 0.198554; Median 5.415320
For 9000:

Code: Select all

# NumSamples = 10; Min = 6.57; Max = 6.58
# Mean = 6.577341; Variance = 0.000012; SD = 0.003519; Median 6.577062

kolsi
Posts: 53
Joined: Wed Jan 23, 2019 10:40 am

Re: Increasing MTU on Pi 4

Fri Nov 20, 2020 10:43 am

I applied the patch to kernel 5.9. Now MTU is 9000 bytes but I still cannot transfer packets larger than 2000 bytes. We have a camera with high frame rate and higher MTU is required for efficient result. Is there a solution to use higher packet size?

Return to “Advanced users”