Slow HDD Transfer Speed


9 posts
by JaSauders » Wed Dec 12, 2012 3:03 pm
Hi there. I'm doing a little project with two Raspberry Pi's to work as file servers. Instead of one server eating up ~380 watts with a RAID1 mirror, I'm using two Raspberry Pi's, each with a 500GB 7200 RPM external hard drive with their own dedicated power supplies in an effort to have one server as the primary and the secondary server gets rsync'd to once a day. I'm using a Sandisk Extreme Class 10 card on each of the Pi's. Each Pi is using Raspbian Linux, fully updated, terminal only mode. Overclocked moderately to 900MHz with only 16MB RAM dedicated to GPU.

When I rsync data from one Pi to another, I was getting an average speed of 700 KB/s. Thinking this was some sort of network bottleneck, I simply took the HDD from Pi 2 and put it on Pi 1, which means Pi 1 was thereby hosting both of the USB HDD's. I then rsync'd the data from one drive to another locally on one Pi. Still, I had terrible performance of about 700 KB/s.

I already have SSH keys set up. When I ran it over SSH, the command was:
rsync -az --delete --progress /media/storage/ root@192.168.1.191:/media/storage_II/

And when I had both drives in a single Pi, I ran:
rsync -az --delete --progress /media/storage/ /media/storage_II/


Each Pi is connected by a tablet charger of some sort. One is for an iPad (5v, 2a) and the other is for a Galaxy Tab (5v, 2.1a), so they should be getting ample power.

I then got frustrated, so I plugged both of the hard drives into my desktop. I rsync'd all of the data at about 18-20 MB/s, significantly faster than the 700 KB/s I was getting before when using the Pi's.

As a result, I'm questioning some things here. Is this the intended performance of the Raspberry Pi? I understood that there was some sort of relationship between the LAN and USB chips, thereby sharing bandwidth hub style, making USB performance seem significantly slower than the traditional USB 2.0 speeds we're used to. Is that likely the case here? If it is, fine, but if there's something I can do to speed things up I'd certainly like to.

Appreciate the insight!
Posts: 4
Joined: Wed Dec 12, 2012 2:50 pm
by Aydan » Wed Dec 12, 2012 4:01 pm
I've had really bad HDD performance as well and it was due to a USB-hub problem.
For me it was the problem that the 7-port hub is actually two 4 port bubs where one hub is connected to a port of the other hub.
If I connect mass storage to the second tier hub the performance is really bad. On the first tier hub I get about 20MB/s data transfer.
Since you're using network at the same time this will about halve the transfer rate, because the LAN-chip uses the same USB host port on the CPU

what does
Code: Select all
hdparm -tT /dev/sdx
give you?

regards
Aydan
Posts: 198
Joined: Fri Apr 13, 2012 11:48 am
Location: Germany, Lake Constance
by JaSauders » Wed Dec 12, 2012 4:59 pm
/dev/sda:
Timing cached reads: 124 MB in 2.03 seconds = 61.13 MB/sec
Timing buffered disk reads: 54 MB in 3.08 seconds = 17.53 MB/sec
Posts: 4
Joined: Wed Dec 12, 2012 2:50 pm
by JaSauders » Wed Dec 12, 2012 5:34 pm
For what it's worth, I just rsync'd some data to my SD card from my desktop computer. It came through at an average of about 3-4 MB/s, which is quite a bit faster than the 700 KB/s I was getting on the USB HDDs.

I also tried to take the two Pi thing out of the equation and checked the transfer speed between my desktop and the one Raspberry Pi. I used both in a push/pull fashion, meaning I transferred from Pi-to-desktop and desktop-to-Pi. Still, an average of 700 KB/s. Anything involving the Pi itself having to be involved in the situation, regardless of it being the server or the client, results in very similar findings.

I did the hdparm against my SD card, and this is what I got:

/dev/mmcblk0:
Timing cached reads: 114 MB in 2.04 seconds = 55.97 MB/sec
Timing buffered disk reads: 60 MB in 3.10 seconds = 19.34 MB/sec

I assume that while my HDDs seem to be faster in the hdparm results, the fact is they're still behind the USB bus restriction of the Pi, which sounds like is the root cause. It's completely okay if this is the issue, I just thought I was doing something incorrectly. Considering how low power these gizmos are, they can certainly take as much time as they need to complete nightly sync's. I don't intend to have to do any 300GB nightly syncs like I'm doing right now to get the initial copy done, so I'm sure it'll be significantly faster for what my intended uses are in the future.
Posts: 4
Joined: Wed Dec 12, 2012 2:50 pm
by JaSauders » Wed Dec 12, 2012 7:14 pm
Hmm, I think the compression factor in rsync may have been enough to really push it over the edge. I was running rsync -az blahblahblah, the z being compression. I just initiated the command again with rsync -az and it was transferring a massive ISO file. It was averaging about 560 KB/s. I tried again with just rsync -a, same command otherwise, same ISO, and I was averaging about 2.8 MB/s. Wow. I guess the Pi having to handle the compression factor was enough to throttle it quite a bit.
Posts: 4
Joined: Wed Dec 12, 2012 2:50 pm
by beardy247 » Sun Jan 06, 2013 10:05 pm
I've noticed exactly the same behaviour with the xbmc project I've been tinkering with. If I copy a file to a samba share from my windows machine I get a vastly different transfer rate for shares on the SD card compared to the external hd, roughly 4MB/s vs 700 KB/s. Local file copies to the external drive seems slow too, though I've not had chance to investigate any further.
Posts: 2
Joined: Sun Jan 06, 2013 9:54 pm
by beardy247 » Mon Jan 07, 2013 12:09 pm
I've done a bit of reading round and I suspect NTFS may be the route of my problem. If I can find some time I'll reformat the external HD to ext4 and see if that makes any difference.
Posts: 2
Joined: Sun Jan 06, 2013 9:54 pm
by HimbeerTorte » Tue Apr 09, 2013 12:29 pm
beardy247 wrote:I've done a bit of reading round and I suspect NTFS may be the route of my problem. If I can find some time I'll reformat the external HD to ext4 and see if that makes any difference.

Did you have any chance to test this?
I am having a similar problem. I have built a NAS with one Raspberry PI (B) with 2 external USB HDDs (NTFS). No matter if I copy files with rsync from HDD1 to HDD2 or if I copy files from Windows to HDD1 or HDD2, I can never get above 2 MB/s write speed.
Testing the speed of both HDD 1 and HDD2 with dd gives me at least 8 MB/s for write...

I am still not completely sure what I can expect from the RasPi. I am running at the same time deluge and openvpn consuming quite a bit of the CPU. But even if I stop both it has no influence on the transfer rate of the HDDs.
Any suggestion?
Posts: 1
Joined: Tue Apr 09, 2013 11:10 am
by yonester » Fri Apr 19, 2013 2:42 am
I'm building a similar setup. Basically I've made the Raspberry a NAS controller and hooked it up to two external USB drives (Western Digital Elements 2TB). Here are the numbers I got using rsync without compression (i.e., rsync -avvi --progress).

USB
ext4 to NTFS -- 1.66MB/s
NTFS to ext4 -- 4.1MB/s
ext4 to ext4 -- 4.6MB/s

Network Write
Laptop to Raspberry over SAMBA* -- 6.1MB/s
Laptop to Raspberry over SSH -- 3.3 MB/s
Laptop to Raspberry over RSH** -- 4.8 MB/s

Network Read
Raspberry to laptop over SAMBA* -- 4.8 MB/s
Raspberry to laptop over SSH -- 2.8 MB/s
Raspberry to laptop over RSH -- 4.3 MB/s

* For this test case I simply mounted the remote directory as a local one and rsynced as if it was a local copy. Note that this disabled differential-copy, which would reduce performance for existing files that need updating. But it also disables SSH, which increases transfer speed.

** This test kept crashing after about 100MB, so I stopped trying and recorded the result where it was before the crash.

I also tested file sharing in Windows 7 by mapping the network path to a local drive letter and simply dragging and dropping files:

File Sharing
Windows to Raspberry -- 5.6 MB/s
Raspberry to Windows -- 6.9 MB/s

The assumption here is that Windows and rsync are calculating and reporting transfer rate the same way, though the numbers seem sensible from what I observed.

Usual caveats apply; your mileage may vary.
Posts: 1
Joined: Thu Apr 11, 2013 5:08 pm