Simple & Secure Onsite + Offsite backups with a Raspberry Pi


2 posts
by nsmith6 » Fri Jul 19, 2013 2:56 pm
I have quite a few PCs at home, each with varying amounts of important data that I wouldn't want to lose. After researching many options for on-site + off-site backups, I have settled on a solution using my raspberry pi as the off-site backup server, along with a handful of free tools to automate the process.

I run primarily Windows PCs at my home, so that's what this backup guide will focus on, but most, if not all, of the tools I use are available for mac & linux, or have suitable alternatives on those platforms.

Harware used:

-- A Raspberry Pi running the distro of your choice (I use Raspbian).
-- An external drive, which you will connect to the Pi (your encrypted off-site backups will be stored here)
-- Someone kind enough to let you plug the Pi+external drive into their router at their house/apartment, etc.
--an always-on PC on your local network, with enough storage for your backups (for on-site backups).

Software used:

-- Bittorrent Sync
-- Boxcryptor Classic 1.6
-- AsComp Backup Maker (there are many alternatives to this, will cover later in the guide)

Overview:

The following is an overview of the complete setup. The sections below this one will go through how to set everything up in step-by-step detail.

I have the raspberry pi + external drive set up at a relative's house, behind their firewall. The Pi is running Raspbian -- I have installed Bittorrent Sync on it, and have a single folder on the external drive, called "Backups", setup as a sync folder.

At home, I have a desktop PC running Windows XP, functioning as my on-site backup server (This is just my desktop PC, it doesn't have to be used solely as a backup server. The only requirement being that it is always on). On this machine, I have Bittorrent Sync installed, along with Boxcryptor Classic.

Also at home, I have multiple machines that need to be backed up (backup clients). Each machine to be backed up has AsComp Backup Maker installed on it.

Each backup client is configured to send its backups to a shared folder on the local backup server (my desktop pc). This backup target folder is actually a folder within my Boxcryptor store on the local server. This is the on-site backup.

The local server is setup to sync the boxcrytor encrypted filesystem, via bittorrent sync, with the off-site raspberry pi. This is the off-site backup.

If some of this is unclear, don't worry! Read on and it will (hopefully) become clear as I go into greater detail.


Why I chose this setup:

1. Security -- the only technology used for syncing data across the internet in this setup is Bittorrent Sync. Bittorrent sync utilizes strong encryption for all data transferred. In addition, the only data we are transmitting over the internet is encrypted locally (by Boxcryptor) before being sent, and is stored encrypted while at rest on the Raspberry Pi off-site.

2. Ease of setup -- Bittorrent sync uses NAT traversal, so no manual port mapping is required. It just works.

3. Good blend between security and usability -- By using Boxcryptor for the encryption, we get an off-site backup that is fully encrypted 100% of the time, and we get a local backup that is stored in an encrypted filesystem, but is available to the local network while the backup server is running.


Stage 1: Setting up the Raspberry Pi as an off-site backup server.

Installing Raspbian
There are many good guides available for installing Raspbian. (like this one http://elinux.org/RPi_Easy_SD_Card_Setup#Using_Windows_7_or_Windows_XP or this one http://elinux.org/RPi_Easy_SD_Card_Setup#Using_Mac_OSX ). If you do not already have Rasbian installed and running on your Pi, follow the linked guide and come back here when finished. Make sure you have set a strong password for the "pi" user, and I'd recommend enabling SSH so that you can remotely connect to the pi once it is off-site. You can change your password & enable SSH easily by running "sudo raspi-config" from a terminal, and selecting the appropriate menu options.

Mounting the external hard drive at boot.

You should now be booted into Raspbian on your Pi. The next step is to ensure that the external hard drive will be mounted and accessible automatically each time the Pi reboots. Open a terminal window.

Make sure the external hard drive is not plugged in to the Pi, and run the command
Code: Select all
sudo blkid
from the terminal. It will return a list of all connected drives. Now, plug in your external hard drive to one of the Pi's USB ports. After 20 seconds or so, re-run the same command,
Code: Select all
sudo blkid
. You should now see a new entry in this list, which will correspond to your external hard drive. What we're after is the external drive's UUID, the long hex number that looks like UUID="123456789ABCDEF". Write that number down, or paste it into an empty text document.

Next we need to create the hard drive's "mount point". This is just the folder where the contents of the hard drive will be accessible to the OS. To create an empty directory at "/media/external", run the following command at the terminal:

Code: Select all
 sudo mkdir /media/external


Now, we need to tell Raspbian to mount the hard drive automatically, at /media/external. We do this by adding a line to a file called fstab. The following command will open fstab in an editor where we can make this change.

Code: Select all
sudo nano /etc/fstab


Using your arrow keys, go down to a blank line at the bottom of the file and add the following:

Code: Select all
UUID=XXXXXXXXXXXXXXXX /media/external ntfs rw,nosuid,nodev,default_permissions 0 0


Be sure to replace the XXXXXXXXXXXXX with the UUID of your device that we notated earlier.

To save these changes, hit CTRL+O on your keyboard. It will give you the option to change the file's name, simply hit Enter to overwrite the old fstab.

Finally, to close the editor, hit CTRL+X on your keyboard. You should now be back to the terminal. Our hard drive will now automatically mount at /media/external each time the Pi boots up.

When finished, type
Code: Select all
reboot
at the terminal to reboot the Pi now.

Installing Bittorrent Sync on the Pi

Rather than re-invent the wheel, I'll just link you to this guide http://blog.bittorrent.com/2013/05/23/how-i-created-my-own-personal-cloud-using-bittorrent-sync-owncloud-and-raspberry-pi/ on installing Bittorrent Sync. All we're interested in is "STEP 2: Installing BitTorrent Sync". When you complete step 2 on the linked page, come back here.

Now you should have Bittorrent Sync installed and auto-starting at boot. Next we need to create a folder on the external hard drive that will contain your backups.

At a terminal, type
Code: Select all
sudo mkdir /media/external/backups


That's all for now, we'll finish the bittorrent sync setup later.


Stage 2: Configuring your local backup server.

What I'm calling the "local backup server" could be a number of things. In my case, it is an always-on desktop pc running Windows XP, with a large internal hard drive. But you could also simply use an external hard drive connected directly to the computer you are wanting to backup (effectively combining the backup client and local server into one machine). My instructions are going to reference my setup (client PC backing up over the local network to a second machine), but it should be easy for you to work out how these steps would change in alternate setups.

Installing Boxcryptor Classic

This section will make more sense if I first explain what Boxcryptor is. Boxcryptor is an encrypted filesystem implementation, similar to EncFS. When you install Boxcryptor, you will designate a folder on your computer that will contain this encrypted filesystem (for illustrative purposes, let's call this location D:\Encrypted_folder). Additionally, Boxcryptor will mount a virtual drive on your system (the default drive letter is Z:). This virtual drive Z: is where you place the plain-text files that you want to be encrypted. While your system is running, the files are available in their unencrypted format at the Z: drive location. However, the entire filesystem is still encrypted in "D:\Encrypted_folder". This allows us to set our backup software to back up over the local network to the unencrypted Z: drive location, and then sync "D:\Encrypted_folder" to our off-site Raspberry Pi using Bittorrent Sync.

Continuing with the how-to...

On the machine that will house your local backups (local backup server), install Boxcryptor Classic. Go to https://www.boxcryptor.com/en/boxcryptor-classic , download the program and install it, accepting all defaults.

Double click the Boxcryptor icon on your Desktop, and select "Create a new Boxcryptor Classic Folder" from the menu that appears. Click Next.

On the next screen, select "Custom". A new field will appear, click "Browse". You need to select a location where your files will be stored in their encrypted state. Assuming the drive where you will store your local backups is drive D:\, browse to the D:\ drive, click the "Make New Folder" button, and call it "Encrypted_folder", or similar. Select the newly created directory, and click "OK".

Click "Next"

You should now be on the "Choose Drive Letter" screen. It will default to drive Z, unless that drive is already taken by your system. Note the Drive letter, and click Next.

Next you will be asked to create a password. It is CRITICAL that you select a strong password, as this will be the basis for the encryption used on all your files. I'd recommend at least 20 characters with UPPER case, lower case, numbers, and symbols. Ideally, use a password generator like the one found here https://www.grc.com/passwords.htm

It is also critical that you store this password in a safe place. If you lose or forget this password, your encrypted backups are unrecoverable. Type in your password twice, make sure "Remember this password" is checked, and click Finish.

Finally, on the Congratulations screen, you will be given the option to "Create a backup of [your] configuration file". This is also critical, so click the button and save it. If you lose your backup configuration file, your encrypted backups will be unrecoverable. I highly recommend emailing the configuration file to a web-based email account, or uploading it to a cloud provider like dropbox for safe-keeping. No one will be able to access your encrypted backups unless they have BOTH your config file AND your password.

Click Close on the Boxcryptor Window.

Sharing your Boxcryptor drive over your network.

Next, we need to share your Boxcryptor drive (most likely Z:), over your local network, so that all the backup clients can backup to it. This requires one additional change to Boxcryptor.

Look for the Boxcryptor icon in the notification area of the Windows taskbar. It looks like a padlock. Right-click on the padlock and select "Preferences".

In the preferences menu, click "Advanced Mode" at the top-right.

Next, right-click on your boxcryptor drive (should be the only item listed). Click Edit.

A new screen will open -- make no changes, and just click "Next".

Finally, on the "Modify Advanced Options" screen, check the boxes titled "Mount for all users" and "Enable long paths".

Click Finish.

Close the Boxcryptor Window.

Now, open a "My Computer" window by holding down the "Windows" key on your keyboard and pushing "E" (Windows+E). You should see your Boxcryptor Z: drive listed. Right-click on this drive, and choose "Properties". You need to share the drive out, as well as set appropriate security settings. This can differ depending on the version of Windows you are running.

On XP, click the "Sharing" tab, click "Share this folder", and then click the "Permissions" button.

In the Permissions window, click on the "Everyone" object, and click the "Allow" button under "Full Control". NOTE: Setting "Everyone" = "Full Control" will allow any device connected to your local network to see your unencrypted backup files. I have my home network firewalled off from guest users, so that isn't a concern for me, but if you ever let others connect to your wifi, you will probably want to tighten down on this security. Those settings are beyond the scope of this tutorial, but you'll need to do something like create a backup user and give that user full control to the share, and then use that user's credentials to login to the share from your client computers.

Click Ok, Ok, until the windows are all closed. Your Z: drive is now shared on the network.


Install Bittorrent Sync.

As mentioned previously, Bittorrent Sync will be used to securely transfer your encrypted filesystem containing your backups to your Raspberry Pi for off-site storage.

Go to http://labs.bittorrent.com/experiments/sync.html , download and install the program, accepting all the defaults. It should now be running in your system tray. Open Bittorrent Sync by double-clicking on its system tray icon.

Click "Add a Sync Folder"

On the "Add Folder" window, click "Generate". This creates the secret key that is the basis of Bittorrent Sync's secure transport technology. No need to write this number down, just leave it in the box.

Click the "Browse" button. Browse to the folder that we specified during Boxcryptor setup to contain the encrypted filesystem (D:\Encrypted_Folder). Click "Select Folder"

Verify that the folder path is correct, then click Ok.

Finally, in the main Bittorrent Sync window, right-click on the sync folder you just created, and select "Show Folder Preferences". Copy and paste the "Read Only Secret" to a blank text document. We'll use that in the next step.


Stage 3: Configuring Bittorrent Sync on the Raspberry Pi

Back on your Raspberry Pi, open up the web browser, and browse to http://127.0.0.1:8888

This will load the Bittorrent Sync config page. Click the "Add Folder" button on the top-right of that page.

In the "Secret" text box, input the "Read Only Secret" that we copied into the notepad from the previous section.

In the "Path" section, type or browse to /media/external/backups

Click "Add"

That's it! We're finished with all of the backend stuff. All that's left is to install the backup software of your choice, and have it back up to the Z: drive share over the network.


Stage 4: Backup Clients

I'm not going to go into a ton of detail here, simply because there are many backup clients and the choice is yours as to which to use. I'm using AsComp Backup Maker https://www.ascomp.de/en/products/show/product/backupmaker , as it is free and has the features that I was looking for. Any backup software that supports backing up over the network will work just fine. If your backup clients are running a Pro version of Windows 7, you can even use the built-in Windows Backup software, which works great.

Other options include:

Duplicati: http://www.duplicati.com/
Areca Backup: http://www.areca-backup.org/

I would recommend choosing a solution that supports incremental backups -- this will greatly reduce the amount of data that is transiting the internet (which will save your bandwidth).

If anyone wants more details on setting up the backup software to backup to \\server\Z , let me know and I will provide.

I hope this tutorial was helpful to you. I tried to be as explicit as possible to reduce any guess-work. I'm sure I made some mistakes as well -- let me know and I'll update the post.

--Nathan
Posts: 1
Joined: Thu Jul 18, 2013 4:10 pm
by Britman » Mon Sep 30, 2013 11:28 pm
Excellent tutorial, works a charm. However I have a question. If say I setup the Pi with BTSync at home on my network and then take it to where it's going to be finally located won't the IP change as it'll be on a different network. Or is it better to setup BTSync when the Pi is in it's final location?

Thanks.
Mark.
Posts: 1
Joined: Mon Sep 30, 2013 11:22 pm