rdup - encrypted/zipped/remote backups

The only backup program that doesn't make backups!

rdup 1.1.14 released (minor bugfixes)

rdup is a platform for backups. It provides a list of files to backup and the necessary mechanisms to process them. It delegates the encryption, compression, transfer and packaging to other utilities in a true Unix-way.

nicehacks.org

and

rdup is a flexible and clean solution for backups, incremental and full backups. [It] allows [for] encryption, compression, path encryption, remote backups and many more features due to its clean and orthogonal design. For the lazy backup person, there is rdup-simple that helps you quite far. The software is quite mature, given that it is in constant development since 2005 (or so) and builds on an older predecessor (hdup).

delicious.com/sven1999

but

Simple backup can be made with rdup-simple ~/bin /tmp/backup

rdup is a simple tool which creates lists of files that are to be backed up. All files types under Unix are supported: regular files, symlinks, hardlinks, devices, sockets, etc.

rdup-1.1.14.tar.bz2 (sha1: 49dc7570122bfa362f36a26a2ffa8bfe8ad55182)
github
git clone http://miek.nl/git/rdup.git
gitweb

Git source code checkout

If you want the full git tree use the following:

git clone http://www.miek.nl/git/rdup.git

ChangeLog for 1.1.14

This is a minor bugfix release

19 Jan 2013: 1.1.14 Miek Gieben <rdup@miek.nl>
   * [rdup] check for the existence of ._rdup_. files and parse its contents
     to restore the actual files' ownership (Miek Gieben)
   * [rdup-up] save user/group info in ._rdup_. files when chown(1)ing has failed.
     Useful when backing up via sshfs. See rdup(1) "Non-root backups" (Miek Gieben)
   * [rdup-up] add -u flag to suppress creating these ._rdup_. files. 
     (useful when restoring) (Miek Gieben)
   * [rdup] add -u flag to suppress interpreting these ._rdup_. files. (Miek Gieben)
   * [rdup-tr] compile fix when compiling with gcc 4.7 (reported by D. L.)
   * [rdup-tr] honor the TMPDIR environment variable (Miek Gieben)
   * [rdup] Don't look at the device ID when deciding if a file needs to be backed up 
     (See https://github.com/miekg/rdup/issues/3)

Check this previously written blog item for some general release notes concerning the 1.1.x releases.

If you have any question, please check the rdup archive first. For contact information see the bottom of this page.

Backups

I have a vision for backups, they:

should be simple to perform
should be simple to setup
should allow for easy recovery (for administrators and end-users)
must allow for incremental backups
must allow for encryption
must allow for remote storage
must be flexible
should be file system agnostic

A lot of backup programs only provide a subset of these requirements or focus on one or two items. Others will provide everything and make setting up and usage a pain.

I believe a backup program should not invent the wheel again and again. This means that you should use ssh for file transfers and not write a new remote/secure authentication mechanism. Sounds familiar? Indeed: KISS.

Taking these requirements and thinking them through to the max, one comes to the conclusion that the best backup program.... does not backup anything.

Hence rdup.

rdup

rdup will only print a list of files that are changed/removed since the last time rdup ran.

Subsequent programs in a Unix pipe line can be used to actually implement to backup scheme. After a run a new (internal) filelist is written. New runs will print out only those files that have actually changed since the last run, thereby making incremental backups possible.

In rdup there is a shell script called rdup-simple, with this script you can easily make backups.

rdup-simple ~/bin /tmp/backup

Will create a date-based backup structure in /tmp/backup. Subsequent runs will update that structure.

/tmp/backup/$HOSTNAME         # base of the backup
/tmp/backup/$HOSTNAME/201001      # year and month (Jan)
/tmp/backup/$HOSTNAME/201001/01   # first day
/tmp/backup/$HOSTNAME/201001/02   # second day

Each day is a full view of your files, data usage is minimized by utilizing hard links.

Difference with `rsync`

Rsync does not support encryption, rdup does. But this comes at a price, rsync deals really well with large files and little changes. rsync will then only transmit the changes; rdup will transmit the entire file. In the age of fast networks and big harddisks I don't consider this a big disadvantage, but you should be aware of it.

Power to the user

Need to create a compressed tar archive? This one-liner will do:

rdup /dev/null ~/bin | rdup-tr -Otar | gzip -c -f > /tmp/rdup.tar.gz

Want to created a compressed tar file, with all the files in it reversed?

rdup -Ptac /dev/null ~/bin | rdup-tr -Otar | gzip -c -f >\
/tmp/rdup.rev.tar.gz

Want GnuPG encrypted archive and store it on another machine?

rdup -Pgpg,-e,-q,--default-recipient-self /dev/null ~/bin | \
ssh -C root@backupmachine rdup-up -t /vol/backup/crypt/$HOSTNAME

In words:

generate the file list with rdup;
transform the file list into an GnuPG encrypted rdup archive;
ssh to the remote backup machine and transfer the archive to it;
unpack the archive to /vol/backup/crypt/$HOSTNAME.

rdup includes rdup-simple which you can use for your own backups. You can create your own scripts to make a fully custom backup. A (short) list of scripts is published here. With rdup-tr and rdup-up this will not really be needed.

Dependencies

rdup is written in C. It depends on

glib library (glib-dev package)
libpcre3

rdup-tr has extra dependencies

libarchive (libarchive1 and libarachive-dev package)
libnettle (optional, simple encryption library)

rdup-up has the same dependencies as rdup.

Contact

If you have any question, please check the rdup archive first.

Questions, patches, bug reports and general discussions can be directed to rdup@miek.nl (in English).

List info can be found at rdup mailman page.

Due to all the spam all non subscriber mail is discarded. So please subscribe before sending.

You can also mail me personally, see about for that.