solderdot
Posts: 43
Joined: Wed Oct 09, 2019 2:11 pm

Severely varying speeds on NAS

Sat Feb 22, 2020 7:03 am

Hi community,
I created a little NAS on my Pi4: 3 M.2 SSDs connected via SATA-USB-Enclosure to a powered USB hub which is connected to one of the Pi4's USB3 ports. On those SSDs a RAID5 is installed and it is accessible from the outside via SMB (read-only) and SSH (read-write), This works quite fine.
One thing I do not really understand is that the write performance when doing backups from my windows PC varies in the copy process. What I do is using a PowerShell script for copying VeraCrypt volumes, i.e. a few multi-gigabyte-files. For the actual copying I use this line:

Code: Select all

pscp -sftp -pw <password> $_srcFilePath $_sshFilePath
This gives me output on the console, indicating the current transfer speed. And here I see the speed varying severely. Normally it starts at around 19000kB/s, then goes down to around 110 kB/s and then the speed increases again. Depending on the file size the copy process ends in the middle of the increasing phase, e.g. at 2000 kB/s for a smaller file or 3000 kB/s for a medium sized file and 10000 kB/s for a huge file.
This is perfectly reproducible.
I wonder where this comes from. I know the Raspi4 has some trouble with USB3 connected drives but it is my understanding that this causes constantly degraded access speed. The proposed quirk I did not yet try out.
The source of the data is my laptop, which is connected to the router via WIFI. The Raspi is connected to the router via ethernet/cable. The laptop does not have an ethernet port, WIFI is the only available network connection. Maybe WiFi is the limiting factor?
I thought about trying workin locally on the Raspi, i.e. copying a huge file onto the RAID from a local source (e.g. SD-card). However the normal copy command does not yield intermediate speed information, as pscp in powershell does, so that is not really helpful here.
Any ideas how I could debug deeper here?
Thanks in advance
-- Solderdot

epoch1970
Posts: 4481
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: Severely varying speeds on NAS

Sat Feb 22, 2020 5:45 pm

Backup speed is sensitive to the number and size of files/directories in the target.
You'll always get better speed transferring the zip or tar of a tree compared to transferring the tree itself. That's a rule of thumb, some protocols are more severely affected than others.

If you're seeing speed tank during transfer of a single large file, either you have a network issue, or your Raid 5 setup doesn't deliver as expected.
Copy said big file on an USB drive and try a local copy to check how the drive array behaves.
(Raid 5 is slow on writes.)

You can append "time" to about any command and that will give you the time spent running the command. But a good ol' stopwatch works just as well if you're dealing with long processes.

Code: Select all

$ time sleep 5

real	0m5.005s
user	0m0.004s
sys	0m0.000s
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

solderdot
Posts: 43
Joined: Wed Oct 09, 2019 2:11 pm

Re: Severely varying speeds on NAS

Sun Feb 23, 2020 8:36 pm

OK, I guess, I nailed it.
I figured out that I can use rsync to get in-flight information of transfer speeds while the copying is in process. With that I did as follows:
  • I copied a big file from the SD-card to the NAS. Speeds did not vary significantly. This test leaves out the network completely.
  • I copied the same big file from a thumb drive connected to the same router to which the Raspi is attached. Speeds are pretty constant. This test utilizes the network but only the ethernet connection, leaving out WIFI.
So I concluded that the actual copying is not causing the varying transfer speed, and it is not the ethernet connection.
So it may be the WIFI connection causing the trouble. Lacking a PC with an ethernet port I cannot falsify that assumption. Just to be sure the effect is still in place I copied the same big file from the PC via Powershell pscp -sftp. Speeds vary severely. As expected.
Then I figured out that pscp also supports a different protocol. Instead of SFTP protocol SCP protocol can be used. So I tried that. And now the transfer speed is pretty stable on a high level.
Apparently the SFTP protocol is causing the trouble. I cannot say whether it is the implementation on the Raspi or on Win10 causing that effect.
For me the problem is solved with switching from SFTP to SCP. However I would be interested in understanding why SFTP has that effect. Does anybody have an idea?
Best regards
-- Solderdot

Return to “Troubleshooting”