Aurion
Posts: 6
Joined: Wed Oct 10, 2012 7:49 pm

Two USB3 drives cannot read and write simulteneously from one to another

Thu May 13, 2021 3:32 am

Hi,

I have two powered USB3 hard-drives that are both formatted with EXT4 plugged to the blue USB3 ports.

When I launch rsync or cp from one drive to another, the speeds are inconsistent. I see with "iotop" that whenever read is going on, writes stop/slowdown and vice versa.

Have a look:
https://www.youtube.com/watch?v=iOG1Y4DiRVM

What could be causing that? It didn't do that when one of them was NTFS.

Thanks for your help!

drtechno
Posts: 237
Joined: Fri Apr 09, 2021 6:33 pm

Re: Two USB3 drives cannot read and write simulteneously from one to another

Thu May 13, 2021 5:01 pm

One of the drives is not writing fast enough and the interrupt buffer is called (ext4-.env-conversion) Which can be a sign of interference of some kind, low power, or just a slow usb drive.

Aurion
Posts: 6
Joined: Wed Oct 10, 2012 7:49 pm

Re: Two USB3 drives cannot read and write simulteneously from one to another

Thu May 13, 2021 10:36 pm

It seems that they can both reach around 100MB/s, individually, but not together,

Also, the CPU is never maxed out or throttling. The pi is overclocked to 2.1ghz with active cooling.

Low power on to the pi could be an issue, I don't have a way to measure that. I am using a 3a power supply though. Also, The drives are powered.

I've read about usb-storage and uas. One of the drive uses usb-storage, could that cause conflicts? The only issue with the old usb-storage I found about would be low speeds, not bad stability though.

This certainly never happens on my other PCs.

pidd
Posts: 2072
Joined: Fri May 29, 2020 8:29 pm
Location: Wirral, UK
Contact: Website

Re: Two USB3 drives cannot read and write simulteneously from one to another

Thu May 13, 2021 11:24 pm

Rsync is pretty much bound to operate with alternate reads then writes, it has to read a block to compare it before deciding whether to write it or not. Its designed to reduce network traffic when syncing between separate computers.

You could attempt to saturate the bus by using parallel to run multiple cp on a file list.

ejolson
Posts: 7237
Joined: Tue Mar 18, 2014 11:47 am

Re: Two USB3 drives cannot read and write simulteneously from one to another

Fri May 14, 2021 2:06 am

Aurion wrote:
Thu May 13, 2021 3:32 am
Hi,

I have two powered USB3 hard-drives that are both formatted with EXT4 plugged to the blue USB3 ports.

When I launch rsync or cp from one drive to another, the speeds are inconsistent. I see with "iotop" that whenever read is going on, writes stop/slowdown and vice versa.

Have a look:
https://www.youtube.com/watch?v=iOG1Y4DiRVM

What could be causing that? It didn't do that when one of them was NTFS.

Thanks for your help!
With drives made of spinning disks, reads and writes are collected by the Linux buffer cache to avoid unnecessary seeks of the mechanical heads. With SSDs, writes are buffered in much the same way to avoid write amplification due to the block erase size. Though good in general, caching data also results in lumpy performance when copying large files from one place to another. There are some tuning parameters available to balance cache pressure against other factors in the Linux kernel.

My recollection is the lumpiness when copying large files started with the 2.2.x Linux kernels and wasn't a problem in the earlier 1.0.x, 1.2.x and 2.0.x kernels. It never bothered me enough to try changing it and anyway the trade-off would make other operations slower and less efficient.

I think the effect doesn't occur with NTFS because that filesystem can't make use of the Linux buffer cache.

Aurion
Posts: 6
Joined: Wed Oct 10, 2012 7:49 pm

Re: Two USB3 drives cannot read and write simulteneously from one to another

Mon May 17, 2021 7:44 pm

I have the same problem with cp on a single file. So, this does not look like an rsync linked behavior:
https://youtu.be/MJlu4StK5Dc

I see the Linux cache is only used with EXT4, probably not NTFS. I see the cache usage moving a lot when I refresh "free -m" during the copy activity. I would like to hear about those tuning parameters to balance cache pressure.

File transfers to those drives via SMB do not exhibit that behavior and run at a stable 80MB/s write.

ejolson
Posts: 7237
Joined: Tue Mar 18, 2014 11:47 am

Re: Two USB3 drives cannot read and write simulteneously from one to another

Mon May 17, 2021 8:31 pm

Aurion wrote:
Mon May 17, 2021 7:44 pm
I have the same problem with cp on a single file. So, this does not look like an rsync linked behavior:
https://youtu.be/MJlu4StK5Dc

I see the Linux cache is only used with EXT4, probably not NTFS. I see the cache usage moving a lot when I refresh "free -m" during the copy activity. I would like to hear about those tuning parameters to balance cache pressure.

File transfers to those drives via SMB do not exhibit that behavior and run at a stable 80MB/s write.
It looks like some Linux kernel parameters for changing cache pressure are described in

https://gist.github.com/marianposaceanu/5629740

There is probably more official documentation somewhere in the kernel sources, but this looks like enough to start experimenting with.

Return to “Troubleshooting”