rdup11

The only backup program that doesn't make backups!

rdup 1.1.1 released !

rdup is a platform for backups. It provides a list of files to backup and the necessary mechanisms to process them. It delegates the encryption, compression, transfer and packaging to other utilities in a true Unix-way. nicehacks.org

rdup is a simple tool which creates lists of files that are to be backed up. All files types under Unix are supported: regular files, symlinks, hardlinks, devices, sockets, etc.

download 1.1.1

Source of rdup:

Git source code checkout

If you want the full git tree use the following:

git clone http://www.miek.nl/git/rdup.git
cd rdup
git checkout -b 11dev origin/11dev

Release Notes for 1.1.1

Check this previously written blog item.

ChangeLog for 1.1.1

release date: 2 Mar 2010
  • tests: Skip gpg test when gpg is not found (Tom)
  • rdup: Rename strmode() to rdup_strmode for FreeBSD (Charlie Kester)
  • rdup: Dont descend when toplevel directory is a symlink (same behavoir as Gnu tar) (Reported by Charlie Kester)
  • rdup: reap our childeren with waitpid() (Oliver Dain)
  • build: Building fix on rpm distros (SuSE/Fedora) (Sven Hartrumpf)
  • build: add --enable-debug to built debug build

Backups

I have a vision for backups, they:

  • should be simple to perform
  • should be simple to setup
  • should allow for easy recovery (for administrators and end-users)
  • must allow for incremental backups
  • must allow for encryption
  • must allow for remote storage
  • must be flexible
  • should be file system agnostic

A lot of backup programs only provide a subset of these requirements or focus on one or two items. Others will provide everything and make setting up and usage a pain.

I believe a backup program should not invent the wheel again and again. This means that you should use ssh for file transfers and not write a new remote/secure authentication mechanism. Sounds familiar? Indeed: KISS.

Taking these requirements and thinking them through to the max, one comes to the conclusion that the best backup program.... does not backup anything.

Hence rdup.

rdup

rdup will only print a list of files that are changed/removed since the last time rdup ran.

Subsequent programs in a Unix pipe line can be used to actually implement to backup scheme. After a run a new (internal) filelist is written. New runs will print out only those files that have actually changed since the last run, thereby making incremental backups possible.

In rdup there is a shell script called rdup-simple, with this script you can easily make backups.

rdup-simple ~/bin /tmp/backup

Will create a date-based backup structure in /tmp/backup. Subsequent runs will update that structure. 

/tmp/backup/$HOSTNAME         # base of the backup
/tmp/backup/$HOSTNAME/201001      # year and month (Jan)
/tmp/backup/$HOSTNAME/201001/01   # first day
/tmp/backup/$HOSTNAME/201001/02   # second day

Each day is a full view of your files, data usage is minimized by utilizing hard links.

Difference with rsync

Rsync does not support encryption, rdup does. But this comes at a price, rsync deals really well with large files and little changes. rsync will then only transmit the changes; rdup will transmit the entire file. In the age of fast networks and big harddisks I don't consider this a big disadvantage, but you should be aware of it.

Power to the user

Need to create a compressed tar archive? This one-liner will do:

rdup /dev/null ~/bin | rdup-tr -Otar | gzip -c -f > /tmp/rdup.tar.gz

Want to created a compressed tar file, with all the files in it reversed?

rdup -Ptac /dev/null ~/bin | rdup-tr -Otar | gzip -c -f >\
/tmp/rdup.rev.tar.gz

Want GnuPG encrypted archive and store it on another machine?

rdup -Pgpg,-e,-q,--default-recipient-self /dev/null ~/bin | \
ssh -C root@backupmachine rdup-up -t /vol/backup/crypt/$HOSTNAME

In words:

  1. generate the file list with rdup;
  2. transform the file list into an GnuPG encrypted rdup archive;
  3. ssh to the remote backup machine and transfer the archive to it;
  4. unpack the archive to /vol/backup/crypt/$HOSTNAME.

Rdup includes a sample script rdup-simple.sh which you can use for your own backups, but feel free to build your own.

A (short) list of scripts is published here

Dependencies

rdup is written in C. It depends on

  • glib library (glib-dev package)
  • libpcre3

rdup-tr has one extra dependency

  • libarchive (libarchive1 and libarachive-dev package)
  • libnettle (optional, simple encryption library)

rdup-up has the same dependencies as rdup.

Contact

Questions, patches, bug reports and general discussions can be directed to rdup@miek.nl (in English).

List info can be found at rdup mailman page.

Due to all the spam all non subscriber mail is discarded. So please subscribe before sending.

You can also mail me personally, see about for that.

External links