MartinPC
Posts: 3
Joined: Tue Nov 12, 2019 11:35 pm

Storage-addressing limits on 4GB Raspberry Pi 4 Model B?

Wed Nov 13, 2019 12:31 am

I'm planning to cobble together a poor man's NAS using OpenMediaVault running on a 4GB Raspberry Pi 4 Model B with 2 or 4 ext4-formatted, single-partition USB 3.0 drives as JBOD. I would like to use "big" drives, initially starting with 2 x 8TB external backup drives (which are cheap right now), but ultimately upgrading to bona fide NAS drives in a 2 x or 4 x 12TB configuration (when prices come down). I'm trying to avoid hidden "gotchas," both at the outset and further down the road, so I have some questions:

What are Pi4B's own inherent, real-world storage-addressing limits, on a per-partition, per-drive, per-USB-port, and aggregate basis? (By "real-world," I mean without a significant performance hit.) In other words, within the confines of "reasonable" performance:

* What's the largest partition I can use?

* What's the largest hard drive I can use?

* What's the largest amount of storage I can address through a single USB 3.0 port, via a hub or dock?

* What's the largest amount of total, aggregate storage I can address through both of the Pi4B's USB 3.0 ports, whether directly or via hubs or docks?

I'm inquiring about OpenMediaVault's storage-addressing limits on OMV's own forums, but if anyone happens to know that OMV's limits are lower than the Pi4B's, I'm all ears.

Many thanks to anyone who takes the time to think about my questions and ventures at shot at answering them!

W. H. Heydt
Posts: 11100
Joined: Fri Mar 09, 2012 7:36 pm
Location: Vallejo, CA (US)

Re: Storage-addressing limits on 4GB Raspberry Pi 4 Model B?

Wed Nov 13, 2019 3:50 am

This https://en.wikipedia.org/wiki/Ext4 explains everything you need.

MartinPC
Posts: 3
Joined: Tue Nov 12, 2019 11:35 pm

Re: Storage-addressing limits on 4GB Raspberry Pi 4 Model B?

Thu Nov 14, 2019 10:06 pm

Thanks, but I'm interested in the 4GB Pi4B's real-world hardware storage-addressing limits, not ext4's theoretical file-system limits. Hardware bottlenecks are common in computers and peripherals and are often hard to ferret out in advance. For example, I have a USB 3.0 ExpressCard adapter on an old laptop that delivers only half the speed a true USB 3.0 port would deliver because the laptop shortchanged the ExpressCard port's resources. I have a 6TB external USB 3.0 hard drive with a single NTFS partition that works just fine when directly connected to a computer, but if I plug it into my Netgear router's USB 3.0 "network storage" port, the router can't reliability read and manage it. The Pi3B has a "gigabit" Ethernet port, but actual throughput is limited to around 60% of true gigabit speeds. Those are the kinds of "hidden gotchas" I'm looking to find out about.

jdb
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 2160
Joined: Thu Jul 11, 2013 2:37 pm

Re: Storage-addressing limits on 4GB Raspberry Pi 4 Model B?

Thu Nov 14, 2019 10:14 pm

There is no hardware reason why the Pi4B cannot address a maximum-size ext4 filesystem. Or any previous Pi, for that matter.
Rockets are loud.
https://astro-pi.org

dustnbone
Posts: 102
Joined: Tue Nov 05, 2019 2:49 am

Re: Storage-addressing limits on 4GB Raspberry Pi 4 Model B?

Thu Nov 14, 2019 11:25 pm

There's no limit aside from the limits of the filesystem you're planning to use. The Pi4 is for the most part a full functioning Linux computer, not really a fair comparison to an old Windows laptop or a NAS appliance.

As for the USB3 speed, on the Pi4 I've seen full speed from every SSD I've connected to it, solidly over 3Gbps pushing close to 4. The Ethernet I've seen over 900Mbps, though some recent firmware updates might have knocked that down a touch. I haven't tested since they came out but my remote X sessions frequently push over 700Mbps bursts in normal use.

My concern would be more with maintaining data integrity with such large volumes operating without any kind of redundancy. 12TB is a lot of data, I would be pretty nervous keeping anything of value on a single unprotected volume.

At least some kind of mirror setup (Maybe ZFS) and automatic notification of a drive failure, or even an online hot spare would probably be enough to at least allow me to sleep at night.

EDIT: A few more notes

The USB bandwidth is shared among all the USB ports, you basically have one USB3 host port, and each pair of USB ports functions as a 2 port hub. There is a limit to how many layers of hubs can be cascaded in total, I think it's 4 but I'm not sure. So using both USB3 ports would allow twice as many theoretical maximum cascaded ports but it's not going to increase the total bandwidth available.

dickon
Posts: 568
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: Storage-addressing limits on 4GB Raspberry Pi 4 Model B?

Fri Nov 15, 2019 12:46 am

MartinPC wrote:
Thu Nov 14, 2019 10:06 pm
I have a 6TB external USB 3.0 hard drive with a single NTFS partition that works just fine when directly connected to a computer, but if I plug it into my Netgear router's USB 3.0 "network storage" port, the router can't reliability read and manage it. The Pi3B has a "gigabit" Ethernet port, but actual throughput is limited to around 60% of true gigabit speeds. Those are the kinds of "hidden gotchas" I'm looking to find out about.
Speed != storage limits. You should have no trouble with storage size limits on any Pi, as they can all run modern kernels with all the latest protocol sets, which have impractical size limits as they're so big. Size issues are a protocol limitation, usually constrained by bitfields at the lowest-level binary representation of either the transport or the filesystem, and aren't a problem with modern SCSI stacks and filesystems (and bear in mind that broadly all modern block devices talk some variant or other of SCSI, and this includes USB mass-storage).

Speed? Yes, that's a thing. But not what you originally asked.

Your router may well have issues with your USB HD because the filesystem is NTFS. Nothing other than Windows reliably drives NTFS -- it's an horrendous fileystem that Microsoft cobbled together when it became clear that FAT32 wasn't going to cut it in a 'modern' context, in the mid-1990s -- and your router is probably running either Linux, WindRiver/QNX, or possibly a *BSD. Basically, it isn't an interoperable filesystem: use FAT32 / exFAT if you truely need it to be, or something actually proper if you don't. Personally, I recommend ZFS, but that isn't without its own issues (mostly around compiling the damned thing for the kernel you want to run it, it really, *really* wants 64b, and it's utterly unsuited to removable media).

User avatar
rpdom
Posts: 15600
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: Storage-addressing limits on 4GB Raspberry Pi 4 Model B?

Fri Nov 15, 2019 5:55 am

The only practical limit for storage size that I am aware of is that some USB-SATA adaptors won't handle devices over something like 2TB (or is it 4TB? Don't know, 2TB is the largest I've got and that works fine on all my PI)

wildestpixel
Posts: 24
Joined: Sun May 26, 2019 7:45 pm

Re: Storage-addressing limits on 4GB Raspberry Pi 4 Model B?

Fri Nov 15, 2019 6:46 am

MartinPC wrote:
Wed Nov 13, 2019 12:31 am

* What's the largest partition I can use? The same as any ext4 volume capacity less a few MB

* What's the largest hard drive I can use? The largest you can support in the power envelope of the pi or an external PSU

* What's the largest amount of storage I can address through a single USB 3.0 port, via a hub or dock? The same as the answer above

* What's the largest amount of total, aggregate storage I can address through both of the Pi4B's USB 3.0 ports, whether directly or via hubs or docks? Double the answer above or more if attached to a hub

I'm inquiring about OpenMediaVault's storage-addressing limits on OMV's own forums, but if anyone happens to know that OMV's limits are lower than the Pi4B's, I'm all ears.

Many thanks to anyone who takes the time to think about my questions and ventures at shot at answering them!
Not sure you are all ears given your first response and 2nd post (total post count = 2). No hidden gotchas as you put it. The limits are your limits and the perception is altered by real knowledge rather than unaccounted for doubt.
To infinity and the pond

Pi4B 4GB heading up the small armada

MartinPC
Posts: 3
Joined: Tue Nov 12, 2019 11:35 pm

Re: Storage-addressing limits on 4GB Raspberry Pi 4 Model B?

Sun Nov 17, 2019 1:11 am

I am in fact all ears! I was nonresponsive for a while because I got unexpectedly busy with other things and then fell very ill. I'm still bedridden and running a fever, so if what I write here is not entirely lucid, please cut me some slack.

Regarding unaccounted-for doubt, I should have made it clear that I'm a pending Windows refugee, have only recently started using Linux, and can fairly be called a "nontechnical user." I have no background in coding or network administration, limited, second-hand familiarity with Linux file systems, and no experience whatsoever with NAS. I've assigned static IP addresses in my router, set up some firewall rules using Gufw, and attempted to sync the data on my Windows laptop and Linux laptops using Samba -- which I subsequently learned is outright defective in the Ubuntu-18.04-based distros I'm currently using! -- and that's about the extent of my technical prowess. Feel free to consider me a clueless noob.

I appreciate all of the feedback I've received in this thread. I'm reassured to learn that there should be no significant hardware bottlenecks with the Pi4B, other than possibly/probably a four-USB-device cap (which shouldn't be a problem for me).

Having experienced both the IBM "Deathstar" and WD "click of death" fiascos first hand, I would never rely on a single drive. For some twenty years, I've had two internal drives in my computers and used an "initial clone + daily differential clone" redundancy strategy in 32-bit Windows XP, using xxclone, and a "monthly clone + interim real-time data-file/config-file sync" redundancy strategy in 64-bit Windows 7, using Macrium Reflect and FreeFileSync/RealTimeSync. I've had system drives suffer mechanical failure or irreparable corruption several times, but in all that time I've never had to reinstall and reconfigure my OS and apps from scratch, and I haven't lost any data. Moreover, I was able to get back up and running in a matter of minutes. (I have configured FreeFileSync to create an age- and number-capped archive of timestamped versioned backups of selected files when they get overwritten or deleted by a sync operation. This allows me to recover from file corruption, editing screw-ups, and unwanted deletions -- each of which has happened to me more than once.)

For external backup, I've been -- depending on folder contents -- mirroring, updating, or syncing my local data and configuration files to two external USB drives. (I sync my local versioned-backup archives, as well.) This has become very tedious, which is a leading reason I'm interested in setting up an NAS. The other leading reason is that my data has grown to the point that I'm approaching the reasonable capacity of a 2TB internal drive and would like to offload my big data files (e.g., videos) to external media that is "always" accessible from all of my computers and that is reasonably fast to write to and read from.

I'm definitely going to want drive redundancy in my NAS, same as for my system drives. If I use OpenMediaVault's software RAID 1 mirroring, I'd get instantaneous redundancy but also instantaneous replication of corruption. Moreover, there is a non-negligible write-speed penalty (~15MBs?) compared to writing to a single unmirrored drive. And finally, in case my Pi4B NAS dies or gets corrupted, I insist on being able to to pull either drive, directly attach it to any Linux computer, and read from and write to it without additional hurdles. I'm guessing this should be possible (with ext4, at least), but it's something I would need to nail down before even considering a software RAID 1 mirror. I think it's more likely that I would run my NAS drives as JBOD and periodically rsync the primary drive to the secondary one. I would not have instantaneous redundancy for the most recently modified/added files, but I would have a window of opportunity to intervene if I notice that something is wrong on the primary NAS drive.

NOTE regarding ext4 vs. XFS vs. ZFS: My biggest data file is under 3GB, and the only time I'm going to be writing large numbers of biggish files (videos, for the most part) over an extended period of time is during the initial offload from computer to NAS. I'm not sure whether XFS presents a big advantage over ext4, which is more universally supported out of the box and which supposedly recovers better from power failures as well. (My NAS will be attached to the same UPS/surge-protector I use for my modem and router, but as with hard drives, I've learned not to count on UPSes' being 100% reliable.) As for ZFS, I'm under the impression that it's not possible to just pull a drive, directly attach it to a computer, and use it. (Also, doesn't it eat up a lot of RAM?) If I'm wrong, please let me know.

At any rate, I think my most important questions have been answered. The Pi4B will be able to handle what I want to do hardware-wise. Thanks very much to everyone who took the time to reply -- I'm grateful and apologize that I wasn't able to check back in and say so earlier.

Return to “General discussion”