Page 1 of 1

can you do a raid with pis?

Posted: Sun Oct 13, 2019 6:28 am
by bob432626
hello I know you can do a raid on hard drives and I thought I found a video where you can do that with pis as well but I can't find it anymore, pretty much all the pis worked together as 1 computer, the main goal was to have each pi share the CPU Prosser, it would work kind of like a raid 5 if I am correct that's where all of the hard drives work as 1 massive hard drive and if one goes down it is still safe because they have a sort of protection or something like that (don't flame i have not looked in to raids all that much)

Re: can you do a raid with pis?

Posted: Sun Oct 13, 2019 8:08 am
by fruitoftheloom
bob432626 wrote:
Sun Oct 13, 2019 6:28 am
hello I know you can do a raid on hard drives and I thought I found a video where you can do that with pis as well but I can't find it anymore, pretty much all the pis worked together as 1 computer, the main goal was to have each pi share the CPU Prosser, it would work kind of like a raid 5 if I am correct that's where all of the hard drives work as 1 massive hard drive and if one goes down it is still safe because they have a sort of protection or something like that (don't flame i have not looked in to raids all that much)

Parallel / Distributed Computing is not the same as Hard Drive Raid, time for a Google:


Distributed Computing:

https://www.pidramble.com/


RAID:

https://unix.stackexchange.com/question ... rage-media

Re: can you do a raid with pis?

Posted: Sun Oct 13, 2019 10:38 am
by LTolledo
One can do RAID with RPi... it can be done.... software RAID.....done that before... as RAID1

....but I cannot recommend it if the drives are USB based....


.... however ...if a hardware RAID HAT will be available (if ever)... and able to acquire and test it for myself....then maybe I may recommend it....

Re: can you do a raid with pis?

Posted: Sun Oct 13, 2019 3:02 pm
by ejolson
bob432626 wrote:
Sun Oct 13, 2019 6:28 am
hello I know you can do a raid on hard drives and I thought I found a video where you can do that with pis as well but I can't find it anymore, pretty much all the pis worked together as 1 computer, the main goal was to have each pi share the CPU Prosser, it would work kind of like a raid 5 if I am correct that's where all of the hard drives work as 1 massive hard drive and if one goes down it is still safe because they have a sort of protection or something like that (don't flame i have not looked in to raids all that much)
To use many Pi computers together for fault-tolerant storage, you need some sort of distributed filesystem. At the moment the popular choices seem to be Gluster, Andrew and Ceph. In not sure which run on Raspbian, but since all three are open source and work with Linux, it is likely all can be made somehow to run on the Raspberry Pi.

In many ways the most interesting is Ceph, partly because it's the newest. None are particularly easy, in my opinion, and all require at least three computers--possibly more--to work at all. While it makes more practical sense for home use to buy a cheap NAS, in certain large-scale commercial environments distributed filesystems are a necessity and learning how they work could be educational.

If you have success with a bunch of Pi computers and any of these distributed file systems, it would be great to share how you set things up on this forum. I would be quite interested and suspect there are others who would be too.

Re: can you do a raid with pis?

Posted: Sun Oct 13, 2019 4:54 pm
by BeauSlim
The simplest RAID solution would probably be to use a hardware-RAID USB disk enclosure like the "RAID" products from Orico and Mediasonic et al. I don't own one so I haven't tried one with the Pi.

Multi-JBOD USB enclosures do work with software-raid via mdadm or btrfs or zfs but software RAID over USB is very much not recommended by experts due to reliability problems and (having tried it) performance tends to be terrible anyway.

Always remember that RAID IS NOT A BACKUP.

Re: can you do a raid with pis?

Posted: Sun Oct 13, 2019 6:06 pm
by ejolson
The original post where
bob432626 wrote:
Sun Oct 13, 2019 6:28 am
pretty much all the pis worked together as 1 computer
makes me think the question is not about traditional RAID with a single Pi but how to set up a fail-safe distributed filesystem using multiple Pi computers.

It is widely documented how to configure a single server with a redundant array of disks. In my opinion RAID works well up to about 100TB. For petabyte storage requirements, however, a distributed approach may be preferable for greater resiliency as well as bandwidth.

Ceph generally requires separate computers for the monitor, manager, metadata server and the online storage devices. A realistic setup might consist of three monitors, two managers, two metadata servers and three or more online storage devices. That's already at least 10 computers, which is quite a few just to set up an environment for learning about things at home.

Luckily, many people have the budget to buy and provide the electricity for that many Raspberry Pi computers--even the 4GB model. This allows someone, who wouldn't have had an opportunity otherwise, to become an expert in setting up and maintaining high-performance distributed filesystems.

At the same time, it might be more productive to set a Pi-controlled drip sprinkler system up for growing tomatoes in the garden.

Re: can you do a raid with pis?

Posted: Sun Oct 13, 2019 6:53 pm
by bob432626
fruitoftheloom wrote:
Sun Oct 13, 2019 8:08 am
bob432626 wrote:
Sun Oct 13, 2019 6:28 am
hello I know you can do a raid on hard drives and I thought I found a video where you can do that with pis as well but I can't find it anymore, pretty much all the pis worked together as 1 computer, the main goal was to have each pi share the CPU Prosser, it would work kind of like a raid 5 if I am correct that's where all of the hard drives work as 1 massive hard drive and if one goes down it is still safe because they have a sort of protection or something like that (don't flame i have not looked in to raids all that much)

Parallel / Distributed Computing is not the same as Hard Drive Raid, time for a Google:


Distributed Computing:

https://www.pidramble.com/


RAID:

https://unix.stackexchange.com/question ... rage-media
THANK YOU, you understand what i mean in this

Re: can you do a raid with pis?

Posted: Mon Oct 14, 2019 6:35 am
by Ernst
bob432626 wrote:
Sun Oct 13, 2019 6:28 am
hello I know you can do a raid on hard drives and I thought I found a video where you can do that with pis as well but I can't find it anymore, pretty much all the pis worked together as 1 computer, the main goal was to have each pi share the CPU Prosser, it would work kind of like a raid 5 if I am correct that's where all of the hard drives work as 1 massive hard drive and if one goes down it is still safe because they have a sort of protection or something like that (don't flame i have not looked in to raids all that much)
https://media.ccc.de/v/1428-ceph-explai ... pberry-pis

https://bryanapperson.com/blog/the-defi ... pberry-pi/