LAN7515 -- USB bus contention issues?

[tkaiser]

Anyone here with the new board in his hands able to show some NAS performance numbers? Unfortunately the provided iperf scores (330 Mbps) are pretty much irrelevant for use cases where network and IO access is happening at the same time.

With the old LAN9514 and its Fast Ethernet we were able to get somewhat stable 9-10 MB/s but now with the LAN7515 and Gigabit Ethernet I fear we run into contention issues. At least that's my observation with external Gigabit Ethernet adapters so far. As soon as access to disk happens at the same time performance could even be lower compared to Fast Ethernet.

Anyone able to provide some numbers? The simple push of a huge file (more than 1GB to ensure filesystem buffers/caches need to be flushed) from Windows Explorer to a Samba Share on the RPi 3+ should be already sufficient.

[gregeric]

Don't overlook that the onboard enhanced 5MHz WiFi, on a bus orthogonal to USB, will give you ~100Mbps concurrent with USB disk access.

[jamesh]

Anyone here with the new board in his hands able to show some NAS performance numbers? Unfortunately the provided iperf scores (330 Mbps) are pretty much irrelevant for use cases where network and IO access is happening at the same time.

With the old LAN9514 and its Fast Ethernet we were able to get somewhat stable 9-10 MB/s but now with the LAN7515 and Gigabit Ethernet I fear we run into contention issues. At least that's my observation with external Gigabit Ethernet adapters so far. As soon as access to disk happens at the same time performance could even be lower compared to Fast Ethernet.

Anyone able to provide some numbers? The simple push of a huge file (more than 1GB to ensure filesystem buffers/caches need to be flushed) from Windows Explorer to a Samba Share on the RPi 3+ should be already sufficient.

Not sure why you think performance might be slower. Ultimately the USB2 port to the SoC is the limiting factor, increasing the ethernet speed will simply use up more of that bandwidth until there isn't any left.

Anyway, buy one, test it. Post results here. Plenty in the supply chain so you should get it fairly quickly.

[tkaiser]

Not sure why you think performance might be slower.

Since I tested it. That's RPi 3 with Fast Ethernet:


That's the same RPi 3 with an RTL8153 USB3 Gigabit Ethernet dongle (same thing on PCs or other USB3 equipped ARM boards is good for fully saturating Gigabit Ethernet at +940 Mbits/sec):


The numbers are telling: sequential write performance and also handling with filesystem objects in general (open/close, locking, directory enumeration) is worse than what I got with Fast Ethernet. While read performance is shown around ~30MB/s this is only the result of the used benchmark testing with file sizes too small. 300 MB fit into RPi 3 DRAM so that was only testing 'page cache to network' and not 'disk to network'.

I'm just wondering whether LAN7515 can cope with that. Since it's an integrated hub and GbE controller maybe it could try to prevent contention issues?

[tkaiser]

Don't overlook that the onboard enhanced 5MHz WiFi, on a bus orthogonal to USB, will give you ~100Mbps concurrent with USB disk access.

Nope. At least not when you talk about the numbers presented in the announcement blog post: https://www.raspberrypi.org/blog/raspbe ... le-now-35/

The numbers shown there are the '1m away from AP with nothing in between' throughput. Those conditions where you can use always also a really short Ethernet cable. Once we switch to real-world scenarios performance drops a lot.

When I did some tests last year with my RPi 3 it looked like this:
* RPi 3 being 50cm away from AP: 37.4 Mbits/s on average and over 3000 retransmits when testing in RX direction.
* 4 meters away, 2 walls in between, wall angle approx. 30°: 20.2 Mbits/s on average, a lot of retransmits and already ICMP packet losses.
* 5 meters away, 2 walls in between, different direction compared to prior location: 12 Mbits/s on average, a lot of retransmits and working remotely over SSH felt pretty sluggish.

That was 2.4GHz band which is close to unusable in my area in the evening hours and on the weekends since overcrowded (I spotted ~150 wireless networks back then, only 10% using 5 GHz). So with B+ while being able to escape the overcrowded 2.4GHz band the situation with 5GHz might look better due to less congestion (neighbours 'polluting the air') but of course the same performance drops occur once there is some distance between board and AP and stuff in between. Also the longer waves in 2.4GHz band are better suited to longer ranges and transmission through walls and solid objects.

And unfortunately we get on the new board not even 2x2 MIMO but only a single antenna solution. In my tests MIMO was always key to performance (and also reliability with larger distances).

So no, I'm really not thinking about accessing disk storage through Wi-Fi since it will be a lot slower and less reliable than Fast Ethernet with all the older RPi.

And I've to admit that I'm a bit surprised that nobody seems to have done some serious testing already. Those iperf numbers shown are more or less irrelevant. Wireless numbers since using 'ideal conditions', Gigabit Ethernet numbers not taking 'shared bus' into account.

Edit: just to be clear: I do not complain about Wi-Fi situation with RPi 3 B+ since dual band capability is highly appreciated to escape overcrowded 2.4GHz band. It's just that those '100 Mbits/sec' do not reflect reality (Wi-Fi is shared medium and 'environment' matters more than anything) and that this onboard Wi-Fi is both too slow and too unreliable. I know how I can get sufficient Wi-Fi with any SBC but those RTL8812AU or RTL8814AU USB dongles with 2 or even 4 antennas have both high power demands and are pretty expensive.

[jamesh]

The latest wireless firmware should improve performance of the Wifi on the Pi3B.

Also note that the quality of the linux driver for the USB ethernet device can have a dramatic effect on the results. I'm not familiar with that chip so cannot really say how good it is. I also suspect data has a slightly longer path from the USB connected dongle than from the onboard device, but again, testing would be needed.

The only real test is to get Pi3+ and test it.

[tkaiser]

Also note that the quality of the linux driver for the USB ethernet device can have a dramatic effect on the results. I'm not familiar with that chip so cannot really say how good it is.

RTL8153 is a pretty good choice and RealTek did a great job to get the driver for this hardware upstreamed (part of rtl8152 driver and IIRC in mainline kernel since 3.10 or 3.12). I was testing with your kernel 4.9 on the RPi 3 and as long as only irrelevant stuff is tested (iperf) performance is as expected (somewhere above 300 Mbits/sec -- the USB2 bottleneck). The problems start once network and disk access happen at the same time (easy to test, just copy a 2 GB file over network to RPi and see what happens).

So I assume your answer means no one at Pi towers tested any real world GbE scenario so far? 'Real world' since those RPi users keen on higher network performance are most probably those dreaming about a faster RPi NAS (for everone else GbE is not that interesting since no improvement).

BTW: I did a quick test with the same dongle on another SBC last year simulating RPi design: Both RTL8153 and a JMS567 enclosure with an SSD were connected to an USB2 hub connected to one USB2 port of the board (cheap Allwinner thing). When repeating the test I tried it once with UAS and the other time with the old and inefficient BOT (differences explained for example here). Problem was much worse with BOT compared to UAS. Unfortunately RPi kernel and/or hardware still do not support UAS

[jamesh]

Also note that the quality of the linux driver for the USB ethernet device can have a dramatic effect on the results. I'm not familiar with that chip so cannot really say how good it is.

RTL8153 is a pretty good choice and RealTek did a great job to get the driver for this hardware upstreamed (part of rtl8152 driver and IIRC in mainline kernel since 3.10 or 3.12). I was testing with your kernel 4.9 on the RPi 3 and as long as only irrelevant stuff is tested (iperf) performance is as expected (somewhere above 300 Mbits/sec -- the USB2 bottleneck). The problems start once network and disk access happen at the same time (easy to test, just copy a 2 GB file over network to RPi and see what happens).

So I assume your answer means no one at Pi towers tested any real world GbE scenario so far? 'Real world' since those RPi users keen on higher network performance are most probably those dreaming about a faster RPi NAS (for everone else GbE is not that interesting since no improvement).

BTW: I did a quick test with the same dongle on another SBC last year simulating RPi design: Both RTL8153 and a JMS567 enclosure with an SSD were connected to an USB2 hub connected to one USB2 port of the board (cheap Allwinner thing). When repeating the test I tried it once with UAS and the other time with the old and inefficient BOT (differences explained for example here). Problem was much worse with BOT compared to UAS. Unfortunately RPi kernel and/or hardware still do not support UAS

I personally haven't done any testing, as not on the 3+ test programme. No idea what tests were done.

[milhouse]

Nope. At least not when you talk about the numbers presented in the announcement blog post: https://www.raspberrypi.org/blog/raspbe ... le-now-35/

The numbers shown there are the '1m away from AP with nothing in between' throughput. Those conditions where you can use always also a really short Ethernet cable. Once we switch to real-world scenarios performance drops a lot.

Nope. The tested Pi3/Pi3+ were at the far end of a room, across a hallway, from a closet that contained a Netgear DGND4000 access point. So, 2 walls/partitions and a distance of 20-25 feet or so separated the AP and the test devices, in a wireless environment with at least two dozen visible WiFi networks.

[HiassofT]

I did a quick test on my 3B+ and measured transfer speeds of slightly below 20MB/sec on GBit ethernet when copying a 10GB file via Samba from/to a 2.5" USB HDD attached to the RPi. So that's about what's to be expected from the USB2.0 limit.

RPI 3B+ was running LibreELEC 8.2.4 with Samba 4.6.14, the other end was my Debian Stretch x86_64 PC running kernel 4.14 with an OS-level cifs mount (mount -t cifs -o vers=3.0...). I tested with dd which reported a write speed to RPi3B+ of 18.4MB/sec and a read speed from RPi3B+ of 19.5MB/sec.

so long,

Hias

[tkaiser]

Nope. At least not when you talk about the numbers presented in the announcement blog post: https://www.raspberrypi.org/blog/raspbe ... le-now-35/

The numbers shown there are the '1m away from AP with nothing in between' throughput. Those conditions where you can use always also a really short Ethernet cable. Once we switch to real-world scenarios performance drops a lot.

Nope. The tested Pi3/Pi3+ were at the far end of a room, across a hallway, from a closet that contained a Netgear DGND4000 access point. So, 2 walls/partitions and a distance of 20-25 feet or so separated the AP and the test devices, in a wireless environment with at least two dozen visible WiFi networks.

Thank you, that's good to know. Then the updated Wi-Fi firmware @jamesh already mentioned might be the reason since one year ago in my tests 2 walls significantly reduced throughput. But of course now I have to ask about the '1m away from AP with nothing in between' numbers

A quick check suggest your DGND4000 is only 802.11n capable? Would be interesting to see 802.11ac numbers as well...

[tkaiser]

I tested with dd which reported a write speed to RPi3B+ of 18.4MB/sec and a read speed from RPi3B+ of 19.5MB/sec.

Thank you for the numbers! Yeah, as expected and most probably I would really have to test myself when trying to dig deeper (testing more than just sequential transfer speeds with huge files and so on)

[Rascas]

I don't know about RPi 3B+ new lan chip, I didn't received mine yet, but some time ago I tested one RPi 3 with a USB 3.0 to Gigabit ethernet on Raspbian Jessie and I had more or less the same results as Hiassoft with his RPi 3B+.
Iperf got about 300Mbits but on a real world scenario I got transfers speeds of about 19MB/s on big files ( 1GB+), less on much smaller files as expected, that is normal. This was transfering to a USB hard drive connected to the Pi, ext4 formated, via samba. So more or less a 2X increase in performance compared to RPi 3 Fast Ethernet which is nice for many people. I did not notice any kind of problem/contention at the time, and I expect the same is true for the RPi 3B+, the USB 2.0 bridge is the limiter factor in both cases.