Cooperative backup


7 posts
by texasdex » Wed May 02, 2012 10:24 am
So, I have had this idea for a Pi-based project involving cooperative backup.

Off-site backup is really the only thing that will save you if your house burns down, if your house is robbed, etc.  However it's normally a huge pain.  You have to have a separate site, you have to transport tapes or other media.  Home users never do it because it's too much trouble.  But the advent of decent home broadband would make it possible to do automatically, if only you had an offsite location.

My idea is for a pair of Pies, one at your house, and one at a friend's house, each with a disk attached.  You backup your files to the one in your house (using Time Machine or something similar), and then they are regularly synced to the remote Pi.  In exchange for having a Raspberry Pi and hard disk in their house, they get to do exactly the same thing: their files are all safely backed up to your house.

How much interest there would be in developing this into something installable and even user-friendly, perhaps even an 'appliance'?  Does anyone have suggestions or ideas for this?  Is anyone else doing something like this?

I'm developing a specification for how a finished product might work (since I'm a software engineer).  Here's what I have so far:

When you first install it, you would be able to 'pair' it with another Pi.  SSH keys would be generated and copied, etc.  Perhaps you could have the option to set it so that your files are encrypted while on the remote machine?  Not sure how encryption would work since most rsync-like processes require the destination to be unencrypted.  Anyways, it would walk you through setting up some dynamic DNS service so that the machines would be able to talk to eachother.  Maybe it has a web interface?  Perhaps it could use uPnP to poke holes in the firewall?

After initial configuration the Pi would appear as a simple (and kind of slow) NAS, which you can copy files to.  The user would set up a backup program to copy their the contents of their computer(s) to it.  It would optionally use some kind of versioning scheme (I'm thinking rsnapshot or Git) to allow you to recover previous versions of files.  Not sure how the previous snapshots would be visible to the user... perhaps part of the web interface?  This would depend a lot on the versioning scheme.
Posts: 11
Joined: Mon Aug 29, 2011 3:18 pm
by bredman » Wed May 02, 2012 10:51 am
Debian already includes duplicity to allow backup to a local or remote or cloud destination. The best front-end for duplicity is dejadup. File versioning is built into the file manager.

If you and your friend want to do cooperative backup, you just need to trade your IP addresses (assuming they are static) or your domain names (using a service like no-ip.com)

If you don't have a friend, you could backup to a free ftp server or Amazon S3.
Posts: 1413
Joined: Tue Jan 17, 2012 2:38 pm
by plugwash » Wed May 02, 2012 12:41 pm
Personally I think you would be better off starting with a linux based NAS device than with a Pi, the CPU may not be as powerful and the cost may be a bit higher but it will likely have native SATA, native ethernet and will come with the enclosure and PSU issues already dealt with.
Forum Moderator
Forum Moderator
Posts: 2359
Joined: Wed Dec 28, 2011 11:45 pm
by Pirx-Danford » Wed May 02, 2012 1:40 pm
Well I think its a sweet idea for everyone who wants to have his own distributed storage system.

There are a ton of open source solutions out there for either backing data up or accessing it, also services like dropbox are okay for many people.

This leaves the group of people out that are 1) not willing to hand over their files to a service provider and 2) not nerdy enough to get a solution done themselves.

If you are willing to put the effort that is needed into it you should be able to set up a perfect out of the box duplication solution.

There will be people that are willing to buy such devices, but how big this market is… no clue to that xD

Still I like the idea a lot – its like a private dropbox just for yourself!
Posts: 62
Joined: Thu Oct 06, 2011 2:05 pm
by chickey » Wed May 02, 2012 2:52 pm
Checkout crashplan as this allows you to backup to a friend for free and vice versa and it allows you to backup other machines to your crashplan server as well.  It needs some tinkering to get it to run on arm and debian but i had this running headless on a dockstar although it did give the CPU a kicking.

http://www.crashplan.com/

This was the script i used to install which should work on a Pi with debian

http://geekfun.com/wp-content/.....an.sh_.txt

Col
Posts: 61
Joined: Thu Jan 05, 2012 11:17 am
by bonelifer » Fri May 04, 2012 4:53 am
Crashplan is definately the way to go if the computers are at different sites. I have a pogoplug and have Crashplan installed on it with ArchLinuxARM using the instructions below. They should be helpful minus the Pogoplug specific stuff:

http://www.opticality.com/blog.....oplug-pro/
User avatar
Posts: 122
Joined: Thu Jan 12, 2012 4:01 am
Location: http://www.phpbb.com/community/memberlist.php?mode=viewprofile&u=147848
by jonrogersuk » Wed Jun 27, 2012 10:28 am
I agree, Crashplan is definitely the way to go. I used the instructions from that opticality link and got CrashPlan working on my Pi - if you want instructions specific to the Pi, look at my blog post: http://www.jonrogers.co.uk/2012/05/cras ... pberry-pi/
Posts: 6
Joined: Thu Jan 26, 2012 11:30 am