sidious
Posts: 7
Joined: Sun Jun 08, 2014 7:26 pm

Setup with +3 HDDs and backup solution

Sun Jun 08, 2014 9:02 pm

Hello

I have been swinging my mind back and forth for years now how I should build/buy an own low-powered, not so expensive server.
Raspberry Pi has come across my mind multiple of times before and even now my thoughts are always landing back to Raspberry Pi whenever I am searching for microserver alternatives.
There are some obstacles that clouds my final judgement and I hope, maybe, someone could help me finalize my theoratically setup into a working practical one....

For starters I was thinking a fileserver (of course) but I would like a minimum of 4 x 3TB external HDDs. Reliability is more vital than transfer speed but as long as I can stream a full HD movie through Apple TV 2 then I am more than settled.

For Backup I would like a 5th External HDD which I would manually do the backupping of only specifically selected folders. It could be once a week or once a month, depending how often I change the files on the server.

Backup QUESTION:
The Raspberry Pi have two USB ports. One port would be occupied by the USB HUB. Is it possible to use the other port to automatically back up whenever I insert the 5th HDD with the help of, maybe, Rsync or some kind similiar?

I see one problem with my theoratical setup -
There will be 6 adapters (4 adapters for each external HDD, 1 for USB hub and 1 for RaspPi) and lots of cords. Optimally would be if one adapter could rule them all, or at least all of the adapters for the external HDDs.

QUESTION:
Can so many HDDs even be mounted correctly? I heard there is an issue with the current in the Raspberry Pi's USB port?
I have only seen a setup of no more than two HDDs.

I like this setup because there are no fans = no noise and the HDD Elements is very low powered when idle.


Extra stuff I would like is more software related and should work out somehow, I suppose...
- iTunes server (already built in with RPiForked-Daapd, if I understand correctly)
- Torrent downloading (BitTorrent)
- Browsing files with mobile device
- Access files outside of home network
- PHP webserver for website building projects (I am currently using Wamp locally on my PC)


What I have so far:
1 HDD (WD Elements) USB 2.0
1 USB hub, 4 ports. USB 2.0
General devices:
Apple TV 2, PC, iPad, iPhones


Thank you in advance.

User avatar
DougieLawson
Posts: 33785
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
Contact: Website

Re: Setup with +3 HDDs and backup solution

Mon Jun 09, 2014 6:13 am

With five hard disks you could use lvm with a RAID array.
Microprocessor, Raspberry Pi & Arduino Hacker
Mainframe database troubleshooter
MQTT Evangelist
Twitter: @DougieLawson

2012-18: 1B*5, 2B*2, B+, A+, Z, ZW, 3Bs*3, 3B+

Any DMs sent on Twitter will be answered next month.

sidious
Posts: 7
Joined: Sun Jun 08, 2014 7:26 pm

Re: Setup with +3 HDDs and backup solution

Mon Jun 09, 2014 12:50 pm

Yes, perhaps I could. :)
But I would like more to have 4 HDDs separately as each partition and the fifth HDD for purely backup of specified folders.

For Rsync backup I know you can set up a Cron job to make a scheduled backup. But can it also be configured that when a HDD is inserted into one of the RasPi's USB ports, the desired backup will start automatically?

Otherwise I am open for suggestions of what already works seamlessly by most of you guys.

Just to confirm.. Four HDDs (with their own adapter) connected to an USB hub will work just fine?
I at least can't see the limit.

User avatar
RaTTuS
Posts: 10079
Joined: Tue Nov 29, 2011 11:12 am
Location: North West UK

Re: Setup with +3 HDDs and backup solution

Mon Jun 09, 2014 1:06 pm

sidious wrote:Yes, perhaps I could. :)
But I would like more to have 4 HDDs separately as each partition and the fifth HDD for purely backup of specified folders.

For Rsync backup I know you can set up a Cron job to make a scheduled backup. But can it also be configured that when a HDD is inserted into one of the RasPi's USB ports, the desired backup will start automatically?

Otherwise I am open for suggestions of what already works seamlessly by most of you guys.

Just to confirm.. Four HDDs (with their own adapter) connected to an USB hub will work just fine?
I at least can't see the limit.
depending on USB resources you can have as many HD as you want...
you can configure the RPI to launch a script when a HD is plugged in
which will mount it in the correct location and start a script
the script can be anything including a rsync line.

your only real limit would be the speed of the USB port
How To ask Questions :- http://www.catb.org/esr/faqs/smart-questions.html
WARNING - some parts of this post may be erroneous YMMV

1QC43qbL5FySu2Pi51vGqKqxy3UiJgukSX
Covfefe

Chrishudsonrose
Posts: 5
Joined: Sun Jun 15, 2014 6:10 pm

Re: Setup with +3 HDDs and backup solution

Sun Jun 15, 2014 7:45 pm

Just thought I would mention, I have 5 raspberry pi's one of which is named PINAS !

Pinas is my fourth attempt at building a NAS server after experimenting with various Pi operating systems, hard drive configurations and hard drive formatting schemes. Pinas serves as a storage medium for my multimedia - and currently is storing 400+ movies, 39,000 mp3s, a few hundred TV shows and several thousand photos. These files are accessed (streamed) to three Pi's running Openelec XBMC, connected to two TVs and a DLP projector. In addition the files are accessed from my ipad and my Mrs iPad mini via the FB file browser app, my Samsung Galaxy Tab, my Samsung Galaxy Note phone, two Windows laptops and my Linux Mint Netbook. All perform perfectly.

Pinas is currently connected to an 8 port powered USB hub, which has a 2.5 amp power supply, which powers the Pi itself. Pinas has 5 hard drives connected to it via the USB hub, these consist of 3 x 1TB - 3.5 inch drives each in their own mains powered USB case. In addition there are also 2 x 750 GB - 2.5 inch drives connected to the USB hub. The software on pinas is Wheezy from which I removed all the bloated addons and reduced the software build size. Pinas boots from a 256mb SD Card and the Wheezy O/S runs on an 8gb USB stick plugged into the second USB port on the Pi. This allows the o/s to load and run faster and reduces the risk of a corrupted O/S card on power failure. The disks are formatted using the windows EXTFS - which in testing proved to be the fasted file system that would serve up the files, AND allow the drive to be removed from Pinas and hooked up to a Windows 7 PC for maintainence or dumping lots of data to it from my laptop. In addition should pinas fail - I can pull the drives and access the data on my windows laptops. Pinas runs both SSH allowing me to run a terminal session into it from various devices, and also runs tightvnc so I can run the wheezy desktop from a variety of devices. Samba delivers all the hard drives file shares out to my network, via a 100mb cable connection to a Case 8 port switch. Some devices (such as two of the XBMC Pi's) are hard wired into my network - but most devices access Pinas via my two 5 GHz wireless access points (the 2.4 GHz spectrum in my area is SWAMPED with neighbours access points). In addition I have a VPN gateway and pinas runs SFTP. This allows me to sit in my hotel room and stream music from home onto my ipad or FTP large files down to any device I have on me via wireless or 3G.

Three drives are used for data and the other two for backing up the data thus providing a level of redundancy - backups are done using RSYNC and a CRON job in the form :

0 3 * * * rsync -av --delete /media/PINAS1/movies/ /media/PINAS2/movies/

Which in this case at 3am copies any new files from hard drive PINAS1 to hard drive PINAS2. I run several RSYNC jobs in the early hours of the morning to complete my backups.

I'm not saying pinas is perfect - but it does exactly what I needed it to do - the cost was low, Pi was about £28 second hand on EBay and the 8 port hub with its PSU was about £10 on EBay + the cost of the drives EBay again. This was my 4th attempt at a NAS server and my speed testing shows I'm getting about 57 Mbs through both my wired and wireless connections, I have no problems streaming HD movies from the server to my devices and my home cinema projector.

I was thinking of posting a step by step how to but figured everyone would already know how to do the above so never got round to it. But if anyone wants to ask anything feel free ?

Cheers

Chris

HiroProtagonist
Posts: 212
Joined: Sat Jun 29, 2013 9:45 am

Re: Setup with +3 HDDs and backup solution

Tue Jun 17, 2014 12:08 am

sidious wrote: For starters I was thinking a fileserver (of course) but I would like a minimum of 4 x 3TB external HDDs. Reliability is more vital than transfer speed but as long as I can stream a full HD movie through Apple TV 2 then I am more than settled.

For Backup I would like a 5th External HDD which I would manually do the backupping of only specifically selected folders. It could be once a week or once a month, depending how often I change the files on the server.

Backup QUESTION:
The Raspberry Pi have two USB ports. One port would be occupied by the USB HUB. Is it possible to use the other port to automatically back up whenever I insert the 5th HDD with the help of, maybe, Rsync or some kind similiar?
Yes, though powering a hard drive via the Pi's USB won't work, so you either need to connect via a powered hub, or use a self-powered drive.
QUESTION:
Can so many HDDs even be mounted correctly? I heard there is an issue with the current in the Raspberry Pi's USB port?
I have only seen a setup of no more than two HDDs.
This is Not A Problem. The number of drives you can mount is only limited by the number you can connect & power.

HiroProtagonist
Posts: 212
Joined: Sat Jun 29, 2013 9:45 am

Re: Setup with +3 HDDs and backup solution

Tue Jun 17, 2014 12:11 am

Chrishudsonrose wrote:The software on pinas is Wheezy from which I removed all the bloated addons and reduced the software build size.
I have a similar setup serving NFS and DVB-T to my client XBMC boxes, but I'm running fairly vanilla Wheezy. I'd be interested to know what you stripped out.

sidious
Posts: 7
Joined: Sun Jun 08, 2014 7:26 pm

Re: Setup with +3 HDDs and backup solution

Mon Jun 30, 2014 7:48 pm

Thanks all for your replies.

to Chrishudsonrose
Especially thanks for your most elaborate post. Your setup is almost exactly what I am looking for, except that I will be using an Apple TV as media player.

I like your idea to have a seperate SD card boot and the OS on an USB thumb drive to reduce the risk for any corruption on power failure that may happen.
I also like the idea very much that the HDDs are formatted as Windows compatible file system.

I have difficulties finding a guide that covers all these details together.
If you would make a guide, it would be the first I use as a waypoint for creating my own server.

Thanks

jangaledhaval
Posts: 1
Joined: Fri Jul 20, 2018 4:24 pm

Re: Setup with +3 HDDs and backup solution

Fri Jul 20, 2018 4:32 pm

Chrishudsonrose
Chrishudsonrose wrote: Just thought I would mention, I have 5 raspberry pi's one of which is named PINAS !

Pinas is my fourth attempt at building a NAS server after experimenting with various Pi operating systems, hard drive configurations and hard drive formatting schemes. Pinas serves as a storage medium for my multimedia - and currently is storing 400+ movies, 39,000 mp3s, a few hundred TV shows and several thousand photos. These files are accessed (streamed) to three Pi's running Openelec XBMC, connected to two TVs and a DLP projector. In addition the files are accessed from my ipad and my Mrs iPad mini via the FB file browser app, my Samsung Galaxy Tab, my Samsung Galaxy Note phone, two Windows laptops and my Linux Mint Netbook. All perform perfectly.

Pinas is currently connected to an 8 port powered USB hub, which has a 2.5 amp power supply, which powers the Pi itself. Pinas has 5 hard drives connected to it via the USB hub, these consist of 3 x 1TB - 3.5 inch drives each in their own mains powered USB case. In addition there are also 2 x 750 GB - 2.5 inch drives connected to the USB hub. The software on pinas is Wheezy from which I removed all the bloated addons and reduced the software build size. Pinas boots from a 256mb SD Card and the Wheezy O/S runs on an 8gb USB stick plugged into the second USB port on the Pi. This allows the o/s to load and run faster and reduces the risk of a corrupted O/S card on power failure. The disks are formatted using the windows EXTFS - which in testing proved to be the fasted file system that would serve up the files, AND allow the drive to be removed from Pinas and hooked up to a Windows 7 PC for maintainence or dumping lots of data to it from my laptop. In addition should pinas fail - I can pull the drives and access the data on my windows laptops. Pinas runs both SSH allowing me to run a terminal session into it from various devices, and also runs tightvnc so I can run the wheezy desktop from a variety of devices. Samba delivers all the hard drives file shares out to my network, via a 100mb cable connection to a Case 8 port switch. Some devices (such as two of the XBMC Pi's) are hard wired into my network - but most devices access Pinas via my two 5 GHz wireless access points (the 2.4 GHz spectrum in my area is SWAMPED with neighbours access points). In addition I have a VPN gateway and pinas runs SFTP. This allows me to sit in my hotel room and stream music from home onto my ipad or FTP large files down to any device I have on me via wireless or 3G.

Three drives are used for data and the other two for backing up the data thus providing a level of redundancy - backups are done using RSYNC and a CRON job in the form :

0 3 * * * rsync -av --delete /media/PINAS1/movies/ /media/PINAS2/movies/

Which in this case at 3am copies any new files from hard drive PINAS1 to hard drive PINAS2. I run several RSYNC jobs in the early hours of the morning to complete my backups.

I'm not saying pinas is perfect - but it does exactly what I needed it to do - the cost was low, Pi was about £28 second hand on EBay and the 8 port hub with its PSU was about £10 on EBay + the cost of the drives EBay again. This was my 4th attempt at a NAS server and my speed testing shows I'm getting about 57 Mbs through both my wired and wireless connections, I have no problems streaming HD movies from the server to my devices and my home cinema projector.

I was thinking of posting a step by step how to but figured everyone would already know how to do the above so never got round to it. But if anyone wants to ask anything feel free ?

Cheers

Chris
Hi Chris,

Can you give me the step by step for this process. Also some more information on which OS you used to run it as NAS and Media server.

Thank You
Dhaval

User avatar
default_user8
Posts: 533
Joined: Mon Nov 18, 2013 3:11 am

Re: Setup with +3 HDDs and backup solution

Sat Jul 21, 2018 2:31 am

I use the nextcloudpi image booting from a 80gb hard drive with plexmediaserver installed. I use a 1tb drive as storage then rsync a mirror image to a second 1tb drive every Monday morning ar 3am for redundancy. Works really well on all fronts, i use nextcloud as a dropbox replacement and serve media to my smart tv's.
Two heads are better than one, unless one's a goat head.

ejolson
Posts: 1884
Joined: Tue Mar 18, 2014 11:47 am

Re: Setup with +3 HDDs and backup solution

Sat Jul 21, 2018 7:05 pm

jangaledhaval wrote:
Fri Jul 20, 2018 4:32 pm
Chrishudsonrose
Chrishudsonrose wrote:I was thinking of posting a step by step how to but figured everyone would already know how to do the above so never got round to it. But if anyone wants to ask anything feel free ?
Hi Chris,

Can you give me the step by step for this process. Also some more information on which OS you used to run it as NAS and Media server.

Thank You
Dhaval
Welcome to the forum!

It has been a year and half since the person you have asked a question to has logged logged into this forum. While they could be reading as a guest, chances are they will not see your question soon if at all.

My recommendation would be to start with the current version of the Raspbian operating system, connect a hard disk and then install Samba. Use Samba to create a Windows share and you now have network attached storage. If you run into difficulty, start a new thread with a title indicating the difficulty and provide as many details as you can.

Return to “Networking and servers”

Who is online

Users browsing this forum: No registered users and 13 guests