ejolson
Posts: 3082
Joined: Tue Mar 18, 2014 11:47 am

Jumbo Packets in Kernel 4.14.34+ Gadget Mode

Sun Apr 29, 2018 1:03 am

I have been doing some tests with Pi Zeros with the USB gadget mode Ethernet driver. With kernel 4.9.59+ on a Pi Zero I am able to set jumbo packets for the USB networking device via a command such as

Code: Select all

# uname -a
Linux snail 4.9.59+ #1047 Sun Oct 29 11:47:10 GMT 2017 armv6l GNU/Linux
# ip link set usb0 mtu 7418
# ifconfig usb0
usb0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 7418
        inet 192.168.7.33  netmask 255.255.255.0  broadcast 192.168.7.255
        inet6 fe80::34:33ff:fe3c:5021  prefixlen 64  scopeid 0x20<link>
        ether 02:34:33:3c:50:21  txqueuelen 1000  (Ethernet)
        RX packets 1452322  bytes 3050230089 (2.8 GiB)
        RX errors 0  dropped 24  overruns 0  frame 0
        TX packets 1431037  bytes 2731689230 (2.5 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
however with kernel 4.14.34+ on a Pi Zero W I obtain the error

Code: Select all

# uname -a
Linux giraffe 4.14.34+ #1110 Mon Apr 16 14:51:42 BST 2018 armv6l GNU/Linux
# ip link set usb0 mtu 7418
RTNETLINK answers: Invalid argument
# ifconfig usb0
usb0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.7.1  netmask 255.255.255.0  broadcast 192.168.7.255
        inet6 fe80::46:c0ff:fea8:701  prefixlen 64  scopeid 0x20<link>
        ether 02:46:c0:a8:07:01  txqueuelen 1000  (Ethernet)
        RX packets 499  bytes 42230 (41.2 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 329  bytes 38397 (37.4 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
Has anyone else noticed that jumbo packets no longer work for the USB gadget mode Ethernet driver?

d_older
Posts: 117
Joined: Mon Jun 25, 2012 5:04 pm
Location: East Yorkshire, UK

Re: Jumbo Packets in Kernel 4.14.34+ Gadget Mode

Sun Apr 29, 2018 8:19 am

Hi ejolson,

I hadn't noticed that one but I have found that the combined ethernet and acm mode doesn't work either - using either the "legacy" g_cdc module or the "new" configfs method. The interfaces appear to the host but the ethernet doesn't seem to connect and the serial appears unidirectional.

Hoping that these may be related to your issue and some kernel hacker can find the solution.

Dave

d_older
Posts: 117
Joined: Mon Jun 25, 2012 5:04 pm
Location: East Yorkshire, UK

Re: Jumbo Packets in Kernel 4.14.34+ Gadget Mode

Mon Apr 30, 2018 5:25 pm

Hi ejolson,

To answer the question posed - I see exactly the same response on the PI ZeroW with 4.14.34+

RTNETLINK answers: Invalid argument
and

Code: Select all

usb0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.2.216  netmask 255.255.255.0  broadcast 192.168.2.255
.....
However attempting to set the mtu from the other end ( a Debian 9.4 x86 box) appears to change the interface

Code: Select all

enp0s20f0u13: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.2.239  netmask 255.255.255.0  broadcast 0.0.0.0
....

[email protected]:/home/data/test# ip link set enp0s20f0u13 mtu 7418
[email protected]:/home/data/test# ifconfig 
enp0s20f0u13: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 7418
        inet 192.168.2.239  netmask 255.255.255.0  broadcast 0.0.0.0
....
but the Zero end still reports 1500

Code: Select all

usb0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.2.216  netmask 255.255.255.0  broadcast 192.168.2.255
.....
Do you know how to check the mtu actually in use?

Dave

ejolson
Posts: 3082
Joined: Tue Mar 18, 2014 11:47 am

Re: Jumbo Packets in Kernel 4.14.34+ Gadget Mode

Mon Apr 30, 2018 6:00 pm

d_older wrote:
Mon Apr 30, 2018 5:25 pm
Do you know how to check the mtu actually in use?
What kernel version is your Debian PC using? I'm running 4.12.x on my PC and also have no problem setting the packet size to Jumbo for the host usb0 device.

As far as I know Jumbo needs to be set for both the receiver and the sender to work. The result I get is that on the Zero the buffers are still 1500 bytes; on the host the buffers are Jumbo. If the bandwidth as measured by iperf or iperf3 didn't noticeably increase, then it didn't work. I'll check the iperf results anyway, just to be sure.

People regularly set Jumbo packets on physical Ethernet devices. When they do, both computers need to agree before Jumbo packets are used. I have assumed the same is true for the USB Ethernet gadget, but as you've pointed out this assumption could be wrong.
Last edited by ejolson on Mon Apr 30, 2018 6:14 pm, edited 1 time in total.

d_older
Posts: 117
Joined: Mon Jun 25, 2012 5:04 pm
Location: East Yorkshire, UK

Re: Jumbo Packets in Kernel 4.14.34+ Gadget Mode

Mon Apr 30, 2018 6:13 pm

Hi,

stock debian 9.4 64

Code: Select all

Linux debian-8 4.9.0-6-amd64 #1 SMP Debian 4.9.82-1+deb9u3 (2018-03-02) x86_64 GNU/Linux
Dave

ejolson
Posts: 3082
Joined: Tue Mar 18, 2014 11:47 am

Re: Jumbo Packets in Kernel 4.14.34+ Gadget Mode

Mon Apr 30, 2018 6:49 pm

d_older wrote:
Mon Apr 30, 2018 6:13 pm
stock debian 9.4 64

Code: Select all

Linux debian-8 4.9.0-6-amd64 #1 SMP Debian 4.9.82-1+deb9u3 (2018-03-02) x86_64 GNU/Linux
Thanks. Interesting that it is again older than a 4.14.x kernel. I wonder how a 4.14.x kernel would behave on a regular PC and whether this is an upstream regression or only related to Raspbian.

Trying to set the MTU on the Pi to 7418, having it fail and proceding by setting the MTU on PC to 7418 results in the same bandwidth as with the default settings. In particular, I obtain

Code: Select all

PC Computer (MTU 7418) -> Pi Zero (MTU 1500)
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec   170 MBytes   143 Mbits/sec    0             sender
[  4]   0.00-10.00  sec   169 MBytes   142 Mbits/sec                  receiver

Pi Zero (MTU 1500) -> PC Computer (MTU 7418)
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.01  sec   111 MBytes  93.1 Mbits/sec    0             sender
[  4]   0.00-10.01  sec   111 MBytes  93.1 Mbits/sec                  receiver
Setting the MTU to 500 on both ends works and results in significantly slower communication.

Code: Select all

PC Computer (MTU 500)-> Pi Zero (MTU 500)
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  72.7 MBytes  61.0 Mbits/sec  1223             sender
[  4]   0.00-10.00  sec  71.0 MBytes  59.6 Mbits/sec                  receiver

Pi Zero (MTU 500) -> PC Computer (MTU 500)
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.23  sec  39.4 MBytes  32.3 Mbits/sec    0             sender
[  4]   0.00-10.23  sec  39.4 MBytes  32.3 Mbits/sec                  receiver
Note that the Zero can't keep up with the rate at which the PC sends the 500 byte packets and we end up with 1223 retransmissions. Decreasing performance, while interesting, isn't as useful as increasing performance. With the previous kernel, it was possible to speed things up by using Jumbo packets on both ends. This regression currently affects any person connecting a Pi Zero as a Ethernet gadget to a PC and logging in over VNC. The Pi Zero Mathematica Dongle is an example of such a setup. It also affects clustering Pi Zero computers as done in the Super-Cheap Computing Cluster for Learning. It would be great if the person responsible for the latest Raspbian kernel could look into this.

Return to “General discussion”