Page 1 of 1

Is a Raspberry Pi ideal for downloading data?

Posted: Thu Nov 14, 2019 6:27 pm
by Jawani
Hello everyone,,
I would like to make backups of data I have from a remote server periodically (a few hundred GBs).

I was thinking of setting a Pi up, SSHing in to it, and SFTP'ing the data from the server to the Pi to back it up. The data would be saved to an external HDD connected to the Pi. I like the idea of uisng a Pi because I would be happy to leave it running 24/7 so I could make backups at any time.reversephonelookup.onl/ curated.onl/nba-reddit/

Would a Pi work well for this?

I was thinking the bottlenecks would be the Internet connection, the HDD, and the USB port. Would there be any other bottlenecks I haven't thought of that would make the Pi not great for handling downloads?

Thanks in advance. Any thoughts would be appreciated.

Re: Is a Raspberry Pi ideal for downloading data?

Posted: Thu Nov 14, 2019 6:58 pm
by rpdom
For a Pi 4B 1GB, the bottleneck would be the internet connection.

Also, look at using rsync for the backup as it will only copy files that have changed.

Re: Is a Raspberry Pi ideal for downloading data?

Posted: Fri Nov 15, 2019 5:40 am
by rbscebu
Jawani, if you are currently using a USB 2 for connection of your backup HDD then you will experience little to no difference in speed when using an RPi3B or RPi3B+. The main differences between the 3B and the 3B+ are processor speed (1.2GHz -v-- 1.4GHz) and WiFi (802.11b/g/n -v- dual-band 802.11ac).

If you are use to USB 3 access speed to your backup HDD, you will need the RPi4B (1GB RAM should be plenty for your backup arrangement). The 3B and 3B+ are only USB 2 while the 4B includes 2 x USB 3 ports.

As stated above, your limiting factor will probably be your internet download speed.

Re: Is a Raspberry Pi ideal for downloading data?

Posted: Fri Nov 15, 2019 7:19 am
by dustnbone
I'm pretty sure in all Pi configurations involving a USB drive you're going to be limited by your internet connection. For simply writing files to a drive even 256MB of RAM is tons, and I'm pretty sure even the weakest Pi CPU can handle the TLS overhead if you're going encrypted.

For sure use rsync or similar to minimize network traffic.

Re: Is a Raspberry Pi ideal for downloading data?

Posted: Fri Nov 15, 2019 8:35 am
by jahboater
dustnbone wrote:
Fri Nov 15, 2019 7:19 am
and I'm pretty sure even the weakest Pi CPU can handle the TLS overhead if you're going encrypted.
It can, but I measured considerably better performance with a Pi3 compared to a Pi Zero, which I presumed was the more powerful CPU handling the encryption better.

The Pi4 has the GiGe ethernet directly connected to the SoC (rather than shared with the USB) and USB3, so is the obvious choice.
The SD card interface runs at double speed too.

Re: Is a Raspberry Pi ideal for downloading data?

Posted: Sat Nov 16, 2019 9:14 am
by jb63
Rather than attaching a hard drive to the PI USB port, you might be better off using a drive with NAS interface ... way faster than the USB 2.0 ports on the RPI 3. You’d have to invest in a wired router/switch and a network (NAS) enabled hard drive. If the backed up data is ‘very important’ (should be) ..... then get a RAID NAS drive, and configure the RAID for redundancy, especially that you’ll be writing often to the drive.

Re: Is a Raspberry Pi ideal for downloading data?

Posted: Sat Nov 16, 2019 9:41 am
by rpdom
jb63 wrote:
Sat Nov 16, 2019 9:14 am
Rather than attaching a hard drive to the PI USB port, you might be better off using a drive with NAS interface ... way faster than the USB 2.0 ports on the RPI 3. You’d have to invest in a wired router/switch and a network (NAS) enabled hard drive. If the backed up data is ‘very important’ (should be) ..... then get a RAID NAS drive, and configure the RAID for redundancy, especially that you’ll be writing often to the drive.
Seeing as the Ethernet port on the Pi 3 is connected via the USB 2.0 port, I suspect it won't be any faster than a directly connected drive without the additional network packet overheads.

Re: Is a Raspberry Pi ideal for downloading data?

Posted: Sat Nov 16, 2019 2:12 pm
by jb63
Good point, I must have missed that. My logic then applies to the RPI4.