piman314 wrote: ↑
Mon Sep 17, 2018 7:45 am
No, it's hung off the back in two USB JBOD containers
How many disks? Is there any hardware RAID built into the JBOD containers?
Assuming 24MB/sec sync speed (simulataneous reading and writing over USB) it would seem to take about twelve days for a Pi to synchronise a bunch of disks using Linux software RAID.
If no hardware RAID is available, I would recommend using the two JBOD containers to create a mirror-redundant disk pool for BTRFS. Use a command such as
# mkfs.btrfs -L Backup -m raid1 -d raid1 /dev/sda1 /dev/sdb1 ...
where the ... are replaced by the remaining disks available. Note that the advantage of integrating RAID functionality into the filesystem is that only allocated sectors need to be mirrored--a big saving when working with 24TB arrays.
Depending on how many disks you have and their perceived reliability, it might be tempting to try out the experimental RAID6 functionality. Unfortunately RAID5 and RAID6 have known bugs even with the most recent kernels. More information is available here
Development of BTRFS has been slow, but continuous, in recent years. The filesystem is complicated and apparently requires significant technical skill to develope. At the same time people have been porting Sun ZFS to work on Linux.
I do not have the technical understanding to fully characterise the design tradeoffs between BTRFS and ZFS. In the case of Raspberry Pi, ZFS deduplication requires more memory than is available, whereas BTRFS has batch duplication which could be performed by plugging the array into a capable PC once a month. For this reason BTRFS seems a better choice.
Compared to Linux software RAID, BTRFS seems less likely to accidentally remove a disk from the array. If not all drives are online, BTRFS won't mount the remaining ones unless a force option is specified. Linux RAID, on the other hand, immediately removes the missing drives and then mounts the array in degraded mode. I think the behaviour of BTRFS is preferable, especially when the disks are connected through USB.
Since your primary backup is already using BTRFS, were you particularly wanting to do something different with the older array?