Page 1 of 1

Ethernet/USB bottleneck

Posted: Wed Jan 13, 2016 11:49 am
by nicholsonr
Hi Guys,

I purchased myself a Raspberry Pi (the latest model) media centre kit.
I set this up as a nice small NAS (much cheaper than one too) but didn't realise (my bad) that the ethernet interface shares with the USB 2.0 interface.
Causing a theoretical max transfer rate of 11mbytes per second (at the moment I'm getting 4 roughly, which isn't too bad considering my cabling and setup etc).

What I'm wondering is, is there any way around this, or potential hardware add-ons that could overcome or increase the transfer rate of this?
Failing that, is this going to be considered in a future raspberry pi release?
(More than happy to buy another model Pi if these interfaces are split/rates increased in a future build).

Re: Ethernet/USB bottleneck

Posted: Wed Jan 13, 2016 12:09 pm
by blc
The SoC the Pi uses was originally designed for mobile phones and media devices, rather than (comparitively) high-speed internconnectivity. The SoC only actually has 1 USB port, the LAN chip has a built in USB hub which is why the Model B can have 4 ports and ethernet. It's not possible to work around this on any current Pi model.

As for a "future" version of the Pi... that's something that's been asked about since day 1 :). The short answer is that the SoC -- and it's features -- is not likely to change any time soon.

Re: Ethernet/USB bottleneck

Posted: Wed Jan 13, 2016 12:14 pm
by rpdom
nicholsonr wrote:Causing a theoretical max transfer rate of 11mbytes per second
You won't get faster than 11MB/s from any 100M ethernet interface anyway. 4MBytes is a little slow though. I usually get a bit higher than that.

It also depends what you are using the other USB ports for. If it's just a mouse and keyboard it won't matter much as they will just take up a tiny bit of the USB bandwidth and there is enough to spare from the 480Mb/s that USB uses.

Re: Ethernet/USB bottleneck

Posted: Wed Jan 13, 2016 12:14 pm
by fruitoftheloom
nicholsonr wrote:Hi Guys,

I purchased myself a Raspberry Pi (the latest model) media centre kit.
I set this up as a nice small NAS (much cheaper than one too) but didn't realise (my bad) that the ethernet interface shares with the USB 2.0 interface.
Causing a theoretical max transfer rate of 11mbytes per second (at the moment I'm getting 4 roughly, which isn't too bad considering my cabling and setup etc).

What I'm wondering is, is there any way around this, or potential hardware add-ons that could overcome or increase the transfer rate of this?
Failing that, is this going to be considered in a future raspberry pi release?
(More than happy to buy another model Pi if these interfaces are split/rates increased in a future build).
There is a way to squeeze more I/O by using a USB3 USB Ethernet Adaptor:

http://www.midwesternmac.com/blogs/jeff ... networking

Re: Ethernet/USB bottleneck

Posted: Wed Jan 13, 2016 12:22 pm
by nicholsonr
Wow,

Thanks for the quick reply guys wasn't expecting one so soon.
So at current I only have ethernet connected, and a usb going to a hard drive enclosure that seats 4 hard drives (externally powered), with 2x 4TB GPT drives in it, all hooked up to a router via Cat6 cabling.
Guess i'm holding out on a different Ethernet/USB Interface IO setup.
Also, tested my hard drive enclosure, it transfers at rates approx 100mb's which is much higher than I've seen in past transfer rates on my PC.

So considering this, seems pointless to use a USB3.0 to ethernet adapter or any such as it's using the same IO.

All good though, here's hoping a new model is in the works to overcome this bottleneck.
Overall it is pretty quick for such a small miniature PC.

Edit: @Fruitoftheloom - How would the gigabit adapter work? It would still tie in with the USB ports, or by sticking to 1 interface increase the transfer speed (e.g. no ethernet in use, just 2x USB)?

Re: Ethernet/USB bottleneck

Posted: Wed Jan 13, 2016 12:30 pm
by fruitoftheloom
nicholsonr wrote: All good though, here's hoping a new model is in the works to overcome this bottleneck.
Overall it is pretty quick for such a small miniature PC.
Considering RPT have just released the ZERO and the possibility of 2 new models based on existing architecture over the next year or two, doubtful will be seeing USB3 anytime soon..........

USB2 is fast enough to satisfy the aims of the RPF in the Learning and Education arena and going into Space.....

Re: Ethernet/USB bottleneck

Posted: Wed Jan 13, 2016 12:40 pm
by fruitoftheloom
nicholsonr wrote:Edit: @Fruitoftheloom - How would the gigabit adapter work? It would still tie in with the USB ports, or by sticking to 1 interface increase the transfer speed (e.g. no ethernet in use, just 2x USB)?
If you have a Gigabit LAN the LAN Gateway will auto-negiate the max speed of interfaces, it will look at the RPi and the Ethernet Chipset will state 10/100 so that is what will be offered.

Using a USB3 Ethernet the Chipset inside will state 10/100/1000 so now your LAN Gateway will offer those speeds.


The LAN has no interest in your USB BUS which the Ethernet Chipset is connected to.......

http://www.microchip.com/wwwproducts/De ... ct=LAN9514

Re: Ethernet/USB bottleneck

Posted: Wed Jan 13, 2016 12:42 pm
by RaTTuS
USB3 GIG ethernet works fine - you get more than 12MB/s [not got the figures to hand]

if your external hard drives are formatted ntfs you will get slower speeds - if they are ext3/4 then you will get much faster ones

Re: Ethernet/USB bottleneck

Posted: Wed Jan 13, 2016 12:49 pm
by mic_s
There is a way to squeeze more I/O by using a USB3 USB Ethernet Adaptor
Alternative :
usb 2.0 to Gigabit Ethernet
Microchip Lan7500-ABZJ (current chip price : 3.80 $ in 5k+ quantities, + Magnetics + Cristal + ...)
http://ww1.microchip.com/downloads/en/D ... 01734A.pdf

Speed is limited by USB 2.0: 480 Mbit per second
You dosn't reach Ethernet-Gigabit-speed. As demonstrated some 220 Gigabit per second is easy reachable.

Re: Ethernet/USB bottleneck

Posted: Wed Jan 13, 2016 12:54 pm
by karrika
I measured the download speed with an external USB HUB + Ethernet to be 13.64Mbits/sec. The SPI based ENC28J60 download was just 4.12Mbits/sec. These numbers were from PiZero.

A local electronics shop offers a RTL8153 based USB-Ethernet combination with 3 USB ports plus 10/100/1000 Ethernet for €25. Has anyone tried this?

https://www.verkkokauppa.com/fi/product ... thernetsov

Re: Ethernet/USB bottleneck

Posted: Wed Jan 13, 2016 1:00 pm
by fruitoftheloom
karrika wrote:I measured the download speed with an external USB HUB + Ethernet to be 13.64Mbits/sec. The SPI based ENC28J60 download was just 4.12Mbits/sec. These numbers were from PiZero.

A local electronics shop offers a RTL8153 based USB-Ethernet combination with 3 USB ports plus 10/100/1000 Ethernet for €25. Has anyone tried this?

https://www.verkkokauppa.com/fi/product ... thernetsov
Never tested but you may be better with the USB2 RTL8152, considering USB3 issues if using with the RPi ZERO

viewtopic.php?f=91&t=132434#p882934


Though the OP was more interested in maybe a USB3 Gigabit Ethernet Adapter and not a HUB :?

Re: Ethernet/USB bottleneck

Posted: Wed Jan 13, 2016 1:05 pm
by gregeric
Take the network traffic off the USB bus altogether: https://hackaday.io/project/8678-rpi-wifi-hat

The mmc interface on the Pi runs at 41.6MHz x 4bits = ~20MiB/s (160MiB/s). Stick a decent SDIO WiFi solution on there and off you go...

Re: Ethernet/USB bottleneck

Posted: Wed Jan 13, 2016 1:10 pm
by nicholsonr
Sorry for my cluelessness here,
So as some comments are suggesting and I just want to clear up.
If I get a USB3 to Gigabit ethernet adapter, plug this into the RPi's USB port and the Rj45 ethernet cable going to the router, into this adapter, this will theoretically increase the speed slightly?
The reason I'm using NTFS is these drives were originally in use by windows and I really cannot be bothered formatting and re-transferring terabytes of data.
If I can even squeeze a little bit more out that would help with streaming.

Re: Ethernet/USB bottleneck

Posted: Wed Jan 13, 2016 1:18 pm
by fruitoftheloom
nicholsonr wrote:Sorry for my cluelessness here,
So as some comments are suggesting and I just want to clear up.
If I get a USB3 to Gigabit ethernet adapter, plug this into the RPi's USB port and the Rj45 ethernet cable going to the router, into this adapter, this will theoretically increase the speed slightly?
The reason I'm using NTFS is these drives were originally in use by windows and I really cannot be bothered formatting and re-transferring terabytes of data.
If I can even squeeze a little bit more out that would help with streaming.
Do you have a Gigabit LAN Network ie is your Router 10/100/1000 ?

If so you should be able to obtain better I/O speeds as per my earlier Link :D

Re: Ethernet/USB bottleneck

Posted: Thu Jan 14, 2016 2:48 am
by nicholsonr
Hi All,

Just realised, the small router I purchased only has a megabit connection (stupid me).
I might try using the USB over a Wi-Fi interface like a 802.11ac over 5ghz band which my router supports.
Opinions people, would this theoretically work as it would then only be limited by the usb2.0 bus?

Re: Ethernet/USB bottleneck

Posted: Thu Jan 14, 2016 5:13 am
by skspurling
The only reason for the Gig adapter is to exceed 100Mbps. The usb 2 bus is good for more than 100Mbps, but gig adapters are only made as usb 3 devices for obvious reasons. One time I thought I had found a usb 2 gig adapter (was testing network performance tools on a pi) but it was a Chinese misprint. The onboard 100Mb ethernet gave me about 70Mbps on an iperf test to a quad core windows laptop with built in gig ethernet. That was for an older B model. I am sure that was more of a cpu/os/driver issue. 70's not too bad for a 100Mb with no tuning. A gig adaptor was topping out around 120Mbps, IIRC.

NTFS needs to go. Fat is better, but linux has never really liked NTFS. Fat or a native file system would perform markedly better. It seems wasn't that long ago that NTFS was recommended to only be used in read only mode in the linux kernel to avoid data corruption.

Re: Ethernet/USB bottleneck

Posted: Thu Jan 14, 2016 6:02 am
by rpdom
skspurling wrote:NTFS needs to go.
Agreed. NTFS is bad news.
Fat is better
For speed, yes. However it does have issues, like the 4GB maximum file size.

A native ext4 file system would be much faster.

Re: Ethernet/USB bottleneck

Posted: Thu Jan 14, 2016 6:41 am
by ejolson
rpdom wrote:
skspurling wrote:NTFS needs to go.
Agreed. NTFS is bad news.
I would not be surprised if switching from NTFS to EXT4 gave a 50 percent improvement in file server performance. However, if you are streaming video and don't have enough bandwidth, the usual solution is to stream video at lower resolution and bit rates.

Re: Ethernet/USB bottleneck

Posted: Thu Jan 14, 2016 7:01 am
by nicholsonr
Hi Guys,

As per my previous post,
Was wondering if anyone had tried a 5ghz 801.11ac Wi-Fi dongle, and seen if this provides more performance over USB than a LAN cable using a 100mbit connection?

Re: Ethernet/USB bottleneck

Posted: Thu Jan 14, 2016 9:10 am
by RaTTuS
nicholsonr wrote:Hi Guys,

As per my previous post,
Was wondering if anyone had tried a 5ghz 801.11ac Wi-Fi dongle, and seen if this provides more performance over USB than a LAN cable using a 100mbit connection?
I believe that 5Ghz wifi is illegal in the UK
wi-fi in general is slower than wired
run top and see what your CPU is like
ext4 is what you really want

Re: Ethernet/USB bottleneck

Posted: Thu Jan 14, 2016 1:19 pm
by Gergmchairy
5Ghz wifi is illegal in the UK
That is must certainly incorrect, 5Ghz is a standard wifi frequency range - all new 802.11ac routers use it as standard.

Throughput on these is (theoretically) up to 6.7gbit/s - although how you do the backhaul at that speed in a home environment i'm not sure !!!

Greg

Re: Ethernet/USB bottleneck

Posted: Thu Jan 14, 2016 1:28 pm
by RaTTuS
yeah I'm wrong on that - it's to do with power
http://www.solwise.co.uk/downloads/file ... uk5ghz.pdf

Re: Ethernet/USB bottleneck

Posted: Thu Jan 14, 2016 5:13 pm
by ejolson
nicholsonr wrote:Hi Guys,

As per my previous post,
Was wondering if anyone had tried a 5ghz 801.11ac Wi-Fi dongle, and seen if this provides more performance over USB than a LAN cable using a 100mbit connection?
As WiFi is a broadcast medium where every connected device interferes with every other, putting the Pi on WiFi may result in packet collisions, dropped packets and further slowdown if the device you are streaming to is also on WiFi. However, packet collisions may not be significant and you are right that the speed could also go faster. It would be interesting to try.

Re: Ethernet/USB bottleneck

Posted: Sat Jan 16, 2016 2:51 am
by nicholsonr
Might give the Wi-Fi thing a try.
No other option due to megabit router, and I'm not smart enough to any other WiFi hacks to the unit.