Geordon
Posts: 22
Joined: Fri Dec 27, 2013 6:29 am

What is the preferred FS format for high-cap external drives?

Sat May 25, 2019 8:01 am

I have an 8 TB drive that I will be using on my Pi for network storage. Is there a preferred or recommended FS format for this? My network computers are Mac, but my understanding is that for a networked drive, the client computer is generally agnostic as far as the server FS goes.

tpyo kingg
Posts: 537
Joined: Mon Apr 09, 2018 5:26 pm
Location: N. Finland

Re: What is the preferred FS format for high-cap external drives?

Sat May 25, 2019 10:11 am

The native format, in this case EXT4, is usually best. For what it's worth, the last I checked even with added software OS could only access EXT systems safely read-only. So if you think you must some day connect the external drive directly to your OS X machine then you might use HFS+ instead. I see HFS utilities and other support there in the Raspian repository but have not tried them on the Raspberry Pi. I've used them on other machines in the past quite well. However, for HFS to work, journalling must be turned off, something which might be a bit difficult on new versions of OS X. Again, the native format may be best.

Another option which may be of interest could be ZFS because of its file-level error control. However, I have not tested that on Raspbian. It works quite well on FreeBSD 12 for the Raspberry Pi, if it is tuned a little. I've tried that and it seems to work well. It could be an option if you are worried about files curdling eventually. But using FreeBSD would eliminate the option of using wireless.

Geordon
Posts: 22
Joined: Fri Dec 27, 2013 6:29 am

Re: What is the preferred FS format for high-cap external drives?

Sat May 25, 2019 5:38 pm

Thank you for your insight. I appreciate it! Looks like I need to do some more research to make a final determination.

epoch1970
Posts: 2770
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: What is the preferred FS format for high-cap external drives?

Sat May 25, 2019 6:52 pm

Hfsplus non journaled works but I wouldn’t trust nearly 8TB of data to it. Sometimes it can break, eg sudden poweroff or loss of power on USB a tad too early at shutdown, the built in fsck cannot repair the drive, so you have to hook it to a Mac and perform the repair. In my opinion, it is just ok, not great.
Isn’t ntfs supported on Mac? If so that could be a solution for a native FS.

Otherwise you’re stuck with remote transfers and local repair on the Pi. Ext4 is ok, but again 8TB is a lot. Btrfs is supposed to put both ext4 and ZFS to rest. It has snapshotting (copy on write) like ZFS, which is great for containers/virtualization. I’ve never used BTRFS, I tend to stick w/ ext3/ext4 or go with ZoL (ZFS on Linux) which works reliably.
However the last 32 bit binary of ZoL is many years old; their basic target is x86_64, machines with gobs of RAM and at least one fast data link (for cache and log).
I’ve never used ZoL on Pi, I am surprised it runs at all.

In the end I suppose the choice is between btrfs and ntfs (assuming native OSX support).

Personal POV, I am wary of drives bigger than 2TB, where copy/cloning/repair can get so long that the drive (or one of its siblings) can die under the strain before the operation has had time to complete. Pi is totally inadequate for volume aggregation (a la lvm,raid, zfs), but it can handle filesystem aggregation (a la unionfs, mhddfs, aufs, mergerfs) just fine.
Years ago I did attach 3 USB ext4 drives to a Pi2 (2x2 + 1 TB later on). Mhddfs provides a merged, RO view of the directories and data is served over the network. Write access is on the underlying drives only. I still would choose that option for serving mostly static data.
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

jahboater
Posts: 4183
Joined: Wed Feb 04, 2015 6:38 pm

Re: What is the preferred FS format for high-cap external drives?

Sat May 25, 2019 6:59 pm

epoch1970 wrote:
Sat May 25, 2019 6:52 pm
Isn’t ntfs supported on Mac? If so that could be a solution for a native FS.
I didn't think NTFS was native for Mac or Linux? I thought it was a Windows FS.
epoch1970 wrote:
Sat May 25, 2019 6:52 pm
Ext4 is ok, but again 8TB is a lot.
ext4 can handle up to 1 exbibyte (EiB) (which is 2^60 bytes of data), so 8TB should be easy.

LTolledo
Posts: 1371
Joined: Sat Mar 17, 2018 7:29 am

Re: What is the preferred FS format for high-cap external drives?

Sat May 25, 2019 9:54 pm

Go EXT4!
"Don't come to me with 'issues' for I don't know how to deal with those
Come to me with 'problems' and I'll help you find solutions"

Some people be like:
"Help me! Am drowning! But dont you dare touch me nor come near me!"

Andyroo
Posts: 2834
Joined: Sat Jun 16, 2018 12:49 am
Location: Lincs U.K.

Re: What is the preferred FS format for high-cap external drives?

Sat May 25, 2019 10:44 pm

jahboater wrote:
Sat May 25, 2019 6:59 pm
epoch1970 wrote:
Sat May 25, 2019 6:52 pm
Isn’t ntfs supported on Mac? If so that could be a solution for a native FS.
I didn't think NTFS was native for Mac or Linux? I thought it was a Windows FS.
epoch1970 wrote:
Sat May 25, 2019 6:52 pm
Ext4 is ok, but again 8TB is a lot.
ext4 can handle up to 1 exbibyte (EiB) (which is 2^60 bytes of data), so 8TB should be easy.
By default a Mac can read NTFS drives natively but you either have a separate program (Paragon is the normal choice) or a hack to allow write. Never understood why Apple did not enable the hack - walled garden I assume.

You could look at exFAT - R/W support is native in MAC and PC and drivers are available for Linux if not built into Raspbian if you need to move the drive around.
Need Pi spray - these things are breeding in my house...

tpyo kingg
Posts: 537
Joined: Mon Apr 09, 2018 5:26 pm
Location: N. Finland

Re: What is the preferred FS format for high-cap external drives?

Sun May 26, 2019 3:06 am

Andyroo wrote:
Sat May 25, 2019 10:44 pm
Never understood why Apple did not enable the hack - walled garden I assume.
Some of that but mostly NTFS is an unreliable format. Also, since Apple operates mostly from the US, the fact that NTFS is loaded down with software patents is an additional barrier. Apple would have to pay extra licensing for each and every device they ship, simply for having the potential to use NTFS and would create even more divergence (expense) between iOS and OS X. Those software patents would not be an issue for a European company, because software is not patentable in Europe, but then again there would still be the technical problems of that format.
Andyroo wrote:
Sat May 25, 2019 10:44 pm
You could look at exFAT - R/W support is native in MAC and PC and drivers are available for Linux if not built into Raspbian if you need to move the drive around.
ExFAT is a continuation of a bad legacy from the 1980s and could be said to be designed to lose data. I certainly wouldn't use it for large drives, the larger the drive the greater the chance that some part of the file system has gone bad or at least needs defragging. And it, too, is heavily encumbered by software patents when used inside the US market. So whatever Linux support for ExFAT happens to exist cannot be an official part of any distro.

BeauSlim
Posts: 30
Joined: Mon Jul 31, 2017 10:02 am

Re: What is the preferred FS format for high-cap external drives?

Wed May 29, 2019 10:14 pm

Do EXT4 and share it over the network. Install samba (SMB/CIFS) and NFS and netatalk (AFS) for broad network compatibility. Install avahi (zeroconf/mDNS/Bonjour) and configure for all 3 file sharing protocols. Install iTunes on Windows to get Bonjour (you can uninstall iTunes after).

My gear is a mix of macOS, Linux and Windows. I haven't found a good cross-platofrm filesystem. NTFS and EXFAT are slow on non-Windows boxes. ZFS is fantastic for mac/linux but won't work with Windows and a Pi doesn't have the RAM. Apple EFS under linux in read/write mode will eventually corrupt your data. The same goes for EXT under macOS and Windows, although the commercial tools from Paragon, etc., might be okay.

tpyo kingg
Posts: 537
Joined: Mon Apr 09, 2018 5:26 pm
Location: N. Finland

Re: What is the preferred FS format for high-cap external drives?

Fri May 31, 2019 4:18 am

BeauSlim wrote:
Wed May 29, 2019 10:14 pm
ZFS is fantastic for mac/linux but won't work with Windows and a Pi doesn't have the RAM.
ZFS should work with as little as 64MB of RAM, at least that is what some of the specialists have claimed for recent versions. However, there are a few options which might have to be tuned, such as arc_max, kmem_size, and making sure that de-duplication is off.

ZFS should run fine with either Samba or netatalk but it would be important to see how much RAM those use, too.

epoch1970
Posts: 2770
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: What is the preferred FS format for high-cap external drives?

Fri May 31, 2019 10:52 am

tpyo kingg wrote:
Fri May 31, 2019 4:18 am
ZFS should work with as little as 64MB of RAM, at least that is what some of the specialists have claimed for recent versions.
Do you have a link?

The official FAQ still has this:
https://github.com/zfsonlinux/zfs/wiki/FAQ#hardware-requirements wrote:The suggested hardware requirements are:
  • ECC memory. This isn't really a requirement, but it's highly recommended.
  • 8GB+ of memory for the best performance. It's perfectly possible to run with 2GB or less (and people do), but you'll need more if using deduplication.
Even in the unlikely event ZoL decided to take over the linux embedded world, performance over a single USB 2.0 link would still be terrible.
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

tpyo kingg
Posts: 537
Joined: Mon Apr 09, 2018 5:26 pm
Location: N. Finland

Re: What is the preferred FS format for high-cap external drives?

Fri May 31, 2019 12:55 pm

epoch1970 wrote:
Fri May 31, 2019 10:52 am
tpyo kingg wrote:
Fri May 31, 2019 4:18 am
ZFS should work with as little as 64MB of RAM, at least that is what some of the specialists have claimed for recent versions.
Do you have a link?
Not a direct one. I've read other requirements and adjusted the configurations and done some experimenting with it on the Raspberry Pi in FreeBSD 12. Other reference pages mention lower memory requirements and Allan Jude, who is an OpenZFS developer and a co-author of "FreeBSD Mastery: ZFS" and "FreeBSD Mastery: Advanced ZFS", spoke Wednesay about it in the current episode of BSD Now in the section starting at 1h 9m of the MP3.

Anything over USB would not perform quickly, but it may be good enough. It only has to read as fast as the transfer or playback requires. I would agree that ZFS is not useful, IMHO, for general IoT. (The Raspberry Pi is in a weird gray zone, very overpowered for IoT but rather underpowered for a server or desktop) However, ZFS is appropriate and, perhaps, preferential for NAS-related activties, especially where large amounts of data are to be stored. Bits do flip from time to time, even on less dense media than an 8TB drive, and the file level error checking which it has helps mitigate that.

And, again, I am talking about OpenZFS on FreeBSD on the Raspberry Pi, not necessarily ZoL. I won't be able to look again at either until Wednesday or later, my 3B+ is running as part of an exhibit until then. So if you have a spare microSD card and enough curiosity to check for yourself sooner ...

Edit: The ZFS Tuning Guide has an example reported to work well for a machine with 768MB RAM. Again that would be on FreeBSD. I'm not a fan of FreeBSD, just that it is sometimes the right tool for the job. A year from now ZoL will have passed it and will probably be what to use, given the direction of change and the rate of change in the area of OpenZFS. tldr; ext4

epoch1970
Posts: 2770
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: What is the preferred FS format for high-cap external drives?

Fri May 31, 2019 5:56 pm

tpyo kingg wrote:
Fri May 31, 2019 12:55 pm
Allan Jude, who is an OpenZFS developer and a co-author of "FreeBSD Mastery: ZFS" and "FreeBSD Mastery: Advanced ZFS", spoke Wednesay about [lower memory requirements] in the current episode of BSD Now in the section starting at 1h 9m of the MP3.
Super nerdy talk! The actual moment they speak about min RAM requirements is a bit later on (at least it seemed so) ;)
And so I stand corrected, Pi will run ZFS along your applications.

Snapshotting is interesting in any situation, IoT included. Some OTA update systems work at file level (OStree, snappy somewhat) and assume the platform offers a stable and reliable filesystem. Docker has a btrfs volume driver. ZFS in linux (and not only on linux) would be great.

Assuming an array of "high-cap" drives, actual throughput would be poor. A corollary of having an array over USB is that drives can get easily/accidentally detached. I don't want to see the speed of a resilver operation in this case...
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

Return to “Advanced users”