Raspberry Pi 4 NAS

[jkirby0610]

So, I am running into a bit of an issue... maybe someone can help me out.

I decided to use my Raspberry Pi 4m, with 4 GB of ram running the latest Raspian Lite image as of (4/22/2020), as a NAS for my Media server. I wasn't expecting it to be super fast but right now I have a 5TB Western Digital USB 3 external hard drive, with a 7200rpm drive in it, attached directly to one of the USB 3 ports on the Raspberry pi. When it was attached to my mac mini (Plex Server) it was giving me about 34mbps read and write speeds over the network. Like the mac mini, the Pi 4 has a gigabit NIC so I was expecting about the same speed. I am currently seeing about 256 Kbps (yes you read that right). I have ruled out the potential of a bottleneck in the network with Shaper rules in the network to allow up to 100Mbps speeds with the NAS. So the only thing I can figure is there is some throttling happening on the pi... I have made sure that there isn't currently any throttleing happening on the pi due to temperature, I added a pi fan just to make sure and it has maintained a cool 54*C and ran "vcgencmd get_throttled" and it returned "throttled=0x20000" which means at one point it was throttled but is no longer.

That is about the extent of my troubleshooting capability when it comes to this kind of thing... any suggestions would be appreciated.

[ejolson]

So, I am running into a bit of an issue... maybe someone can help me out.

I decided to use my Raspberry Pi 4m, with 4 GB of ram running the latest Raspian Lite image as of (4/22/2020), as a NAS for my Media server. I wasn't expecting it to be super fast but right now I have a 5TB Western Digital USB 3 external hard drive, with a 7200rpm drive in it, attached directly to one of the USB 3 ports on the Raspberry pi. When it was attached to my mac mini (Plex Server) it was giving me about 34mbps read and write speeds over the network. Like the mac mini, the Pi 4 has a gigabit NIC so I was expecting about the same speed. I am currently seeing about 256 Kbps (yes you read that right). I have ruled out the potential of a bottleneck in the network with Shaper rules in the network to allow up to 100Mbps speeds with the NAS. So the only thing I can figure is there is some throttling happening on the pi... I have made sure that there isn't currently any throttleing happening on the pi due to temperature, I added a pi fan just to make sure and it has maintained a cool 54*C and ran "vcgencmd get_throttled" and it returned "throttled=0x20000" which means at one point it was throttled but is no longer.

That is about the extent of my troubleshooting capability when it comes to this kind of thing... any suggestions would be appreciated.

Have you tried plugging it in to one of the USB2 ports to see if it's faster?

[procount]

Have you read this sticky post -> viewtopic.php?f=28&t=245931

[jkirby0610]

Have you tried plugging it in to one of the USB2 ports to see if it's faster?

Yes and no it’s not faster at all... I ran iperf -s on the computer I am transferring data from and iperf -c on the pi got 851 Mbits/sec so obviously the Gbit speeds are there.

I even ran a few tests on the pi to the Hard drive to make sure that wasn’t the bottleneck

lsusb -t
/: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 5000M
|__ Port 2: Dev 2, If 0, Class=Hub, Driver=hub/4p, 5000M
|__ Port 2: Dev 3, If 0, Class=Mass Storage, Driver=uas, 5000M
|__ Port 4: Dev 4, If 0, Class=Mass Storage, Driver=usb-storage, 5000M

sudo hdparm -I /dev/sda

/dev/sda:
SG_IO: bad/missing sense data, sb[]: 70 00 05 00 00 00 00 0a 00 00 00 00 24 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

ATA device, with non-removable media
Standards:
Likely used: 1
Configuration:
Logical max current
cylinders 0 0
heads 0 0
sectors/track 0 0
--
Logical/Physical Sector size: 512 bytes
device size with M = 1024*1024: 0 MBytes
device size with M = 1000*1000: 0 MBytes
cache/buffer size = unknown
Capabilities:
IORDY not likely
Cannot perform double-word IO
R/W multiple sector transfer: not supported
DMA: not supported
PIO: pio0

Oh and yes I have read that sticky and while it may work for what they were trying to it didn’t work for me.

I even disabled write barrier to see if that might help with speeds, nope that caused the drive to disappear from the Pi all together.

Thank you for your kind suggestions.

[procount]

What is the output of

Code: Select all

lsusb

and

Code: Select all

sudo hdparm -t /dev/sda1

[jkirby0610]

What is the output of

Code: Select all

lsusb

and

Code: Select all

sudo hdparm -t /dev/sda1

So I put a powered USB3 hub on it, thinking maybe it it was just under powered, that stoped the Pi from booting completely so I removed the power and it boots. After it boots I plug the power back in and it operates normally. But I am still getting the seriously slow speeds.

Here is the out put from what you asked for with a little modification to show a little more.

Code: Select all

$ lsusb -t
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 5000M
    |__ Port 2: Dev 2, If 0, Class=Hub, Driver=hub/4p, 5000M
        |__ Port 2: Dev 3, If 0, Class=Mass Storage, Driver=uas, 5000M
        |__ Port 4: Dev 4, If 0, Class=Mass Storage, Driver=usb-storage, 5000M

and

Code: Select all

sudo hdparm -t /dev/sda

/dev/sda:
SG_IO: bad/missing sense data, sb[]:  70 00 05 00 00 00 00 0a 00 00 00 00 24 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 Timing buffered disk reads: 464 MB in  3.01 seconds = 154.20 MB/sec

[procount]

Those are subtly different commands to the ones I asked for.

[jkirby0610]

Those are subtly different commands to the ones I asked for.

Sorry about that, here is what I get when I run the commands you asked for, I was just thinking I may give you a little more info to see if maybe that would help. This is for an instance of OMV I have installed on the Raspian Lite OS, if that helps any.

Code: Select all

$ lsusb
Bus 002 Device 004: ID 0480:a006 Toshiba America Inc External Disk 1.5TB
Bus 002 Device 003: ID 0bc2:3322 Seagate RSS LLC SRD0NF2 [Expansion Desktop Drive]
Bus 002 Device 002: ID 05e3:0626 Genesys Logic, Inc. 
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 05e3:0610 Genesys Logic, Inc. 4-port hub
Bus 001 Device 002: ID 2109:3431 VIA Labs, Inc. Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

The drive that's attached is actually a 5TB not a 1.5TB, I have no idea where it would be getting that from, and I do have a 500GB drive attached as well but I have not tried writing anything to that one just yet. That one is less important as it is just going to be my ISO Library for my PVE instance but I expect I will have the same issues on that one.

As for the results of "sudo hdparm -t /dev/sda1" the directory doesn't exist.

Code: Select all

sudo hdparm -t /dev/sda1
/dev/sda1: No such file or directory

[procount]

I took /dev/sda from your second post "sudo hdparm -I /dev/sda".
I assumed it would be partitioned and formatted so that /dev/sda1 would exist.
If /dev/sda1 is not correct, please substitute for the device name of a suitable partition on your 5TB HDD.

Edit: It appears to be a SEAGATE HDD. So I assume it was just a simple mistake to describe it as a Western Digital HDD?

[jkirby0610]

I took /dev/sda from your second post "sudo hdparm -I /dev/sda".
I assumed it would be partitioned and formatted so that /dev/sda1 would exist.
If /dev/sda1 is not correct, please substitute for the device name of a suitable partition on your 5TB HDD.

Maybe this might help... Looks like the partitions were mounted as LVM's... that could be an issue...

sudo lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 4.6T 0 disk
└─HS01-HomeShare 254:1 0 4.6T 0 lvm /srv/dev-disk-by-label-HomeDrive
sdb 8:16 0 465.8G 0 disk
└─NS01-NetShare 254:0 0 465.8G 0 lvm /srv/dev-disk-by-label-NetDrive
mmcblk0 179:0 0 29.8G 0 disk
├─mmcblk0p1 179:1 0 2.4G 0 part
├─mmcblk0p2 179:2 0 1K 0 part
├─mmcblk0p5 179:5 0 32M 0 part
├─mmcblk0p6 179:6 0 256M 0 part /boot
└─mmcblk0p7 179:7 0 27.1G 0 part /

[jkirby0610]

Edit: It appears to be a SEAGATE HDD. So I assume it was just a simple mistake to describe it as a Western Digital HDD?

All I know is that the enclosure it's in is branded as a WD.

[jkirby0610]

I took /dev/sda from your second post "sudo hdparm -I /dev/sda".
I assumed it would be partitioned and formatted so that /dev/sda1 would exist.
If /dev/sda1 is not correct, please substitute for the device name of a suitable partition on your 5TB HDD.

Edit: It appears to be a SEAGATE HDD. So I assume it was just a simple mistake to describe it as a Western Digital HDD?

So I went back and looked for how that LVM was mounted and found it was mounted as /dev/dm-1 and here is the output from that:

Code: Select all

sudo hdparm -t /dev/dm-1

/dev/dm-1:
SG_IO: bad/missing sense data, sb[]:  70 00 05 00 00 00 00 0a 00 00 00 00 24 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 Timing buffered disk reads: 540 MB in  3.00 seconds = 179.70 MB/sec

[procount]

That makes more sense.
Unfortunately, I'm not familiar enough with LVM to say how to use hdparm with it, or whether hdparm does work with it (some google fu required).

If you've tried adding:

Code: Select all

usb-storage.quirks=0bc2:3322:u

to cmdline.txt and rebooting, another option would be to try:

Code: Select all

usb-storage.quirks=0bc2:3322:t

[jkirby0610]

That makes more sense.
Unfortunately, I'm not familiar enough with LVM to say how to use hdparm with it, or whether hdparm does work with it (some google fu required).

If you've tried adding:

Code: Select all

usb-storage.quirks=0bc2:3322:u

to cmdline.txt and rebooting, another option would be to try:

Code: Select all

usb-storage.quirks=0bc2:3322:t

Thank you, I will give that a try.

[procount]

A little Google suggested that LVM logical volumes can also be found at /dev/mapper/XXXX where XXXX related to your volume group. That might also be worth a try with hdparm. But I'm only suggesting using hdparm -t to get a local read performace of your HDD without the network affecting it.
I have a similar Seagate 6TB HDD (0bc2: ab38) but the hdparm result is more like 98MB/s rather than 256KB/s for comparison.

[jkirby0610]

A little Google suggested that LVM logical volumes can also be found at /dev/mapper/XXXX where XXXX related to your volume group. That might also be worth a try with hdparm. But I'm only suggesting using hdparm -t to get a local read performance of your HDD without the network affecting it.
I have a similar Seagate 6TB HDD (0bc2: ab38) but the hdparm result is more like 98MB/s rather than 256KB/s for comparison.

You are a genius!!! That's the bottleneck!!! I was thinking hard drive performance was the issue...

Code: Select all

sudo hdparm -t /dev/mapper/HS01-HomeShare 

/dev/mapper/HS01-HomeShare:
 Timing buffered disk reads:   4 MB in  4.08 seconds = 1003.12 kB/sec

So now to figure out why that's the case...

EDIT:
So I ran the same thing on the other drive and got the following.

Code: Select all

sudo hdparm -t /dev/mapper/NS01-NetShare 

/dev/mapper/NS01-NetShare:
 Timing buffered disk reads: 138 MB in  3.01 seconds =  45.86 MB/sec

Which is closer to what I was expecting. They are both LVM's sitting on EXT4 formatted drives. Any suggestions on where to start looking?

[procount]

Nope. That's the limit of my knowledge I'm afraid.
Any joy with the quirks?
I think the 'u' parameter is to turn off UAS access, which may get it to work better for some unsupported drives.
Some drives are automatically blacklisted for UAS, so the 't' parameter marks them as ok to use UAS, AIUI.

[jkirby0610]

Nope. That's the limit of my knowledge I'm afraid.
Any joy with the quirks?
I think the 'u' parameter is to turn off UAS access, which may get it to work better for some unsupported drives.
Some drives are automatically blacklisted for UAS, so the 't' parameter marks them as ok to use UAS, AIUI.

Thank you for your help getting this far, I changed the parameter to "t" and the speeds got even slower... down to 176kB/sec, so I may just turn that back to "u". I am probably going to try wiping it and starting from scratch... good thing I have the time lol.

[procount]

Fair enough. It was worth a try. Glad to have helped you a little further down the road. Good luck!

[jkirby0610]

So I am still having an issue with this, but I am not sure what is going on now as I have eliminated the network, and the pi as being the issue... maybe it now has to do with the source drive which makes no sense to me.

Results of commands on the Pi

Code: Select all

$ sudo lsblk
NAME             MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda                8:0    0   4.6T  0 disk 
└─HS01-HomeShare 254:1    0   4.6T  0 lvm  /srv/dev-disk-by-label-HomeDrive
sdb                8:16   0 465.8G  0 disk 
└─NS01-NetShare  254:0    0 465.8G  0 lvm  /srv/dev-disk-by-label-NetDrive
mmcblk0          179:0    0  29.8G  0 disk 
├─mmcblk0p1      179:1    0   2.4G  0 part 
├─mmcblk0p2      179:2    0     1K  0 part 
├─mmcblk0p5      179:5    0    32M  0 part 
├─mmcblk0p6      179:6    0   256M  0 part /boot
└─mmcblk0p7      179:7    0  27.1G  0 part /

The drive in question is "sda" which has the LVM HS01 mounted.

Here I test the read time directly to sda and note that the speed registered on the Pi itself is ~107 MB/sec:

Code: Select all

sudo hdparm -t /dev/sda

/dev/sda:
SG_IO: bad/missing sense data, sb[]:  70 00 05 00 00 00 00 0a 00 00 00 00 24 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 Timing buffered disk reads: 330 MB in  3.06 seconds = 107.81 MB/sec

Here I test the HS01 mounted as an LVM on the system and again note the speed on the Pi itself is ~152 MB/sec:

Code: Select all

sudo hdparm -t /dev/mapper/HS01-HomeShare

/dev/mapper/HS01-HomeShare:
SG_IO: bad/missing sense data, sb[]:  70 00 05 00 00 00 00 0a 00 00 00 00 24 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 Timing buffered disk reads: 458 MB in  3.01 seconds = 152.40 MB/sec

So I tested the network with the source system as the "server" and the Pi as the "client", again please note the network speeds as they are showing between 664 Mbits/sec and 912 Mbits/sec and yes I understand the difference between MB/sec and Mbits/sec:

Source:

Code: Select all

$ iperf -s -N
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size:  128 KByte (default)
------------------------------------------------------------
[  4] local 10.10.1.11 port 5001 connected with 10.10.1.3 port 44378
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.0 sec  1.06 GBytes   912 Mbits/sec
[  4] local 10.10.1.11 port 5001 connected with 10.10.1.3 port 44382
------------------------------------------------------------
Client connecting to 10.10.1.3, TCP port 5001
TCP window size:  129 KByte (default)
------------------------------------------------------------
[  6] local 10.10.1.11 port 54799 connected with 10.10.1.3 port 5001
[  6]  0.0-10.0 sec  1.02 GBytes   878 Mbits/sec
[  4]  0.0-10.0 sec   794 MBytes   664 Mbits/sec

Client:

Code: Select all

iperf -c 10.10.1.11 -d
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size:  128 KByte (default)
------------------------------------------------------------
------------------------------------------------------------
Client connecting to 10.10.1.11, TCP port 5001
TCP window size:  179 KByte (default)
------------------------------------------------------------
[  5] local 10.10.1.3 port 44382 connected with 10.10.1.11 port 5001
[  4] local 10.10.1.3 port 5001 connected with 10.10.1.11 port 54799
[ ID] Interval       Transfer     Bandwidth
[  5]  0.0-10.0 sec   794 MBytes   665 Mbits/sec
[  4]  0.0-10.0 sec  1.02 GBytes   877 Mbits/sec

Based on the results of the above I should be able to transfer a 1GB file to the Pi in less than 30 seconds. I am looking for a way to test the source system to see if maybe the read capability on it may be the chokepoint in this whole setup.

The Source system is a Mac mini and conducting a rsync from the Mac to the Pi I am seeing between 128 and 256 KB/sec transfer speeds... any ideas would be welcome but not necessary seeing as it's not a Raspberry Pi. Lol. Thank you for getting me as far as I am now and potentially eliminating the Pi as the reason for the slow upload speed.

Edit: I miss calculated the potential transfer speed of a 1GB file as being 10 seconds... I believe it was actually closer to 20 to 30 seconds.

[jkirby0610]

Guess it may have been something to do with my Mac mini because I rebooted it and all of a sudden everything started moving and I am getting about 56 MB/sec transfers to the NAS from it. Thank you all for your help with this.

[cpc464]

Hi OP

Glad that a reboot fixed this. The slow speeds you were getting were pretty insane. I tested a Pi 4 for NAS use and got over 60 MB/s write, over 100 MB/s read. So it's pretty nimble, to say the least.

Jim