Posts: 235
Joined: Fri Oct 27, 2017 11:14 pm

Copying files, packet size?

Sat Oct 27, 2018 3:19 pm

Hi there. I'm pretty sure this is really a network working issue, but on the off chance it's something to do with the pis....

If I set off 4 windows machines copying the same large file from a nas they all copy at 25% (speed) (ish) until one has finished, then the other 2 jump to 33%, and so on. You know what I mean even if I'm not being very accurate.

I'm trying with same with some pis, a mixture of 3B and 3B+. The 3B+ finish sooner, this I would expect, and pretty speedy they are too, however as each finishes there are a few 3Bs left copying away for ages. This is a reasonable file, 144mb, which isn't enormous, but it's a decent test.

So as each pi finishes the copy operation the full bandwidth is not given 'back to the network' to be shared between the others. Now, is this the fault of the pi? - The OS knows the packet size has shrunk and doesn't bother trying to get more. Or is it the network hardware? - Having sized down the packes to suit the switch(s) don't attempt to raise them again?

Is this the difference between managed and unmanaged? I have run several miles of cat5 cable in my time and setup many a switch, but I haven't got a clue how they actually work!!

Can anyone shed light?

Many thanks.


Re: Copying files, packet size?

Sat Oct 27, 2018 5:39 pm

My guess is that the PCs are saturating either the switch or NAS on a first come first served or round robin basis where as the poor old Pis cannot push that much through the ethernet ports.

The Pi 3B+ has faster ethernet but its still driven via the shared inbuilt USB hub...

Have a look at these tests to see how the Pis perform compared to each other :) ... net-speed/

User avatar
Posts: 452
Joined: Sat Jun 22, 2013 1:36 pm
Location: Guang Dong, China

Re: Copying files, packet size?

Sat Oct 27, 2018 9:04 pm


Are you asking this:
There is an NAS serving files to multiple clients concurrently. However,
1 - Why some clients get more bandwidth, almost all, while the others only get a straw to drain?
2 - Why slow clients won't speed up, even after those faster finished?

My GUESS is that, you need a better NAS machine.

Long story to short: Try setting a Pi as the NAS, and use the Fair Queue queuing discipline on it. The problems might be solved.

About question-1, when the NAS machine's queuing discipline is First in First out, then it could happen.
The word queuing discipline is a Linux terminology, and can be short as qdisc. BSDs call it ALTQ, or ALTernate Queueing.
It's the rule, deciding when multiple packets want to get out of the same network interface, who goes first.

The most simple rule, is First in First out. So the packets comes earlier would have a priority to use the interface first. However, the problem is that, some clients could send their TCP ACK packet faster, to make the NAS send them data faster. So such clients could occupy the interface almost completely. This is not because these clients are bad, just because they start the transfer earlier, so their TCP transfer window get grown to a bigger one.

To solve question-1, the NAS need to have the capability to run a more advanced queuing discipline. In Linux, there is a qdisc called fq, aka Fair Queue.
When facing multiple packets, this qdisc would arrange them into different queues, by default each TCP session, aka SOCKET, have a queue. And then dequeue these queues in a round robin fashion, so none of them would beat the others away.
Documentation could be seen with "man tc-fq"

About question-2, I think it's because the NAS's TCP congestion control algorithm is not trying to test the path size after some time.
There is a part in TCP, also trying to make the network works more fair. It is the TCP congestion control.
Note that, this one is about TCP, the layer 4, end to end fairness. While the qdisc is at layer 2, the single link fairness.
This TCP congestion control should work like this: when the transfer starts, it walks slow. And, trying to raise its speed. Until to a point, it finds out the path reached its capability. So it keep this speed. After a timeout, it could try the speed test again, to make sure if the path condition is changing.

My guess is that your NAS's TCP congestion control is not doing this timeout thing. It won't re test the link condition. If this is the case, then it's not normal, it's a faulty TCP implementation.

To solve all these two problems. I think you could try setting up a Pi as the NAS.
Then you could check your queuing discipline with the command "ip link", see the qdisc part.
You could change the qdisc to fq with "sudo tc qdisc add dev eth0 root fq"

By default, Raspbian uses the CUBIC TCP congestion control algorithm. It's already a very good one, Linux use it, Windows 10 use it either.
You could see it with " cat /proc/sys/net/ipv4/tcp_congestion_control"

I'm not going to make things more difficult, as it's already very difficult. My university didn't teach me these.
So I think we don't need to change Pi's congestion control.

It's a 144 mega bytes file after all, I think a Pi NAS would be sufficient.

Return to “General discussion”