Well after some playing (and signing up) with S3, I'm now confident that rdup can be used with S3. The basic mode of operation is as follows:

  1. Create a backup with rdup -c
  2. Save the output somewhere
  3. Upload it to Amazon

Backup:

rdup -c /dev/null dirs/to/backup > rdup-bin-file
s3-upload rdup-bin-file
  1. Download the backup file
  2. Put it through rdup-snap -c

Restore:

s3-down rdup-bin-file
cat rdup-bin-file | rdup-snap -c -b /tmp/restore

How this works under Ubuntu/Debian with Perl

The Amazon Perl API is already available with Ubuntu. Just

apt-get install libnet-amazon-s3-perl libwww-perl libxml-simple-perl

and you will get all the stuff you need to start developing. Note that perldoc Net::Amazon::S3 is something you really need to read. Also be sure to read the S3 docs from Amazon.

Generic code

S3 works with buckets in which you can place objects (which should be smaller than 5 GB). So before you can do anything you will need to create a bucket.

All Perl files that follow need to have the following code at the start

#!/usr/bin/perl

use strict;
use warnings;
use Net::Amazon::S3;

Next you will need to create a connection with Amazon, this done with your 'access-key-id' and your 'secret-access-key' which I'm not going to tell of course. Code look like this

my $s3 = Net::Amazon::S3->new( 
{   
aws_access_key_id => "my_public_key"
aws_secret_access_key => "my_private_key",
}   
);

Create a bucket

This step needs to be done once. In code:

my $response = $s3->add_bucket( { bucket => "rdup-test1" } ) or 
    die $s3->err . ": " . $s3->errstr;

Upload a file to Amazon

Code:

my $localname = $ARGV[0];
my $bucket = $s3->bucket("rdup-test1");
$bucket->add_key_filename('rdup-backup-200805-23.bin', $localname,
{ content_type => 'application/binary', },
) or die $s3->err . ": " . $s3->errstr;

The arguments for add_key_filename are

  1. remote filename
  2. local filename
  3. hash reference; in this case with the content_type

Download a file from Amazon

Code:

my $localname = $ARGV[0];
my $bucket = $s3->bucket("rdup-test1");
$response = $bucket->get_key_filename('rdup-backup-200805-23.bin',
'GET', $localname) or
    die $s3->err . ": " . $s3->errstr;

The arguments for get_key_filename are:

  1. remote filename
  2. method, any else than GET allowed?
  3. local filename

Webinterfaces

There are also S3 web interfaces out there which will let you point and click to manage your files. I have no experience with any of them, but they seem like a nice idea. You can at least manage your backup files and buckets from your browser; rdup will never have fancy support for this as it has nothing to do with backing up.

Conclusion

You want cheap, off-site backups? With encryption. You can have to now. For rdup-0.6.1 I will add an rdup-s3 tool to automate some of these things. For now you will need to write your own stuff and use the following commands to backup

rdup -c /dev/null ~/mydir | rdup-crypt my-key > rdup-crypted.output
<upload to amazon>
<ready>

And these to restore

<download from amazon>
cat rdup-crypted.output | rdup-crypt -d my-key | rdup-snap -c -b \
/tmp/restore

Ain't that cool?!

Tags: rdup, perl

0 comments

Comments are closed

If you really, really want to comment, please mail miek@miek.nl.

0 comments in moderator queue