Jawani
Posts: 2
Joined: Tue Nov 12, 2019 9:47 pm

Is a Raspberry Pi ideal for downloading data?

Thu Nov 14, 2019 6:27 pm

Hello everyone,,
I would like to make backups of data I have from a remote server periodically (a few hundred GBs).

I was thinking of setting a Pi up, SSHing in to it, and SFTP'ing the data from the server to the Pi to back it up. The data would be saved to an external HDD connected to the Pi. I like the idea of uisng a Pi because I would be happy to leave it running 24/7 so I could make backups at any time.reversephonelookup.onl/ curated.onl/nba-reddit/

Would a Pi work well for this?

I was thinking the bottlenecks would be the Internet connection, the HDD, and the USB port. Would there be any other bottlenecks I haven't thought of that would make the Pi not great for handling downloads?

Thanks in advance. Any thoughts would be appreciated.
Last edited by Jawani on Fri Nov 15, 2019 3:17 am, edited 1 time in total.

User avatar
rpdom
Posts: 15580
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: Is a Raspberry Pi ideal for downloading data?

Thu Nov 14, 2019 6:58 pm

For a Pi 4B 1GB, the bottleneck would be the internet connection.

Also, look at using rsync for the backup as it will only copy files that have changed.

rbscebu
Posts: 38
Joined: Sat Dec 06, 2014 12:34 am
Location: Cebu, Philippines

Re: Is a Raspberry Pi ideal for downloading data?

Fri Nov 15, 2019 5:40 am

Jawani, if you are currently using a USB 2 for connection of your backup HDD then you will experience little to no difference in speed when using an RPi3B or RPi3B+. The main differences between the 3B and the 3B+ are processor speed (1.2GHz -v-- 1.4GHz) and WiFi (802.11b/g/n -v- dual-band 802.11ac).

If you are use to USB 3 access speed to your backup HDD, you will need the RPi4B (1GB RAM should be plenty for your backup arrangement). The 3B and 3B+ are only USB 2 while the 4B includes 2 x USB 3 ports.

As stated above, your limiting factor will probably be your internet download speed.
RPi4B 4GB with Raspian Buster
RPi3B+ with Raspian Buster
RPi0W with Raspian Buster

dustnbone
Posts: 87
Joined: Tue Nov 05, 2019 2:49 am

Re: Is a Raspberry Pi ideal for downloading data?

Fri Nov 15, 2019 7:19 am

I'm pretty sure in all Pi configurations involving a USB drive you're going to be limited by your internet connection. For simply writing files to a drive even 256MB of RAM is tons, and I'm pretty sure even the weakest Pi CPU can handle the TLS overhead if you're going encrypted.

For sure use rsync or similar to minimize network traffic.

jahboater
Posts: 4834
Joined: Wed Feb 04, 2015 6:38 pm

Re: Is a Raspberry Pi ideal for downloading data?

Fri Nov 15, 2019 8:35 am

dustnbone wrote:
Fri Nov 15, 2019 7:19 am
and I'm pretty sure even the weakest Pi CPU can handle the TLS overhead if you're going encrypted.
It can, but I measured considerably better performance with a Pi3 compared to a Pi Zero, which I presumed was the more powerful CPU handling the encryption better.

The Pi4 has the GiGe ethernet directly connected to the SoC (rather than shared with the USB) and USB3, so is the obvious choice.
The SD card interface runs at double speed too.

jb63
Posts: 131
Joined: Mon May 11, 2015 6:41 pm
Location: Chicago, USA
Contact: Website

Re: Is a Raspberry Pi ideal for downloading data?

Sat Nov 16, 2019 9:14 am

Rather than attaching a hard drive to the PI USB port, you might be better off using a drive with NAS interface ... way faster than the USB 2.0 ports on the RPI 3. You’d have to invest in a wired router/switch and a network (NAS) enabled hard drive. If the backed up data is ‘very important’ (should be) ..... then get a RAID NAS drive, and configure the RAID for redundancy, especially that you’ll be writing often to the drive.

User avatar
rpdom
Posts: 15580
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: Is a Raspberry Pi ideal for downloading data?

Sat Nov 16, 2019 9:41 am

jb63 wrote:
Sat Nov 16, 2019 9:14 am
Rather than attaching a hard drive to the PI USB port, you might be better off using a drive with NAS interface ... way faster than the USB 2.0 ports on the RPI 3. You’d have to invest in a wired router/switch and a network (NAS) enabled hard drive. If the backed up data is ‘very important’ (should be) ..... then get a RAID NAS drive, and configure the RAID for redundancy, especially that you’ll be writing often to the drive.
Seeing as the Ethernet port on the Pi 3 is connected via the USB 2.0 port, I suspect it won't be any faster than a directly connected drive without the additional network packet overheads.

jb63
Posts: 131
Joined: Mon May 11, 2015 6:41 pm
Location: Chicago, USA
Contact: Website

Re: Is a Raspberry Pi ideal for downloading data?

Sat Nov 16, 2019 2:12 pm

Good point, I must have missed that. My logic then applies to the RPI4.

Return to “Beginners”