rdup included some utilities for manipulating it's output. These utilities have been split-off since the rdup 0.6.2 release. Some of the tools included were:

• rdup-gzip, compress the output
• rdup-mcrypt, encrypt the output
• rdup-gpg, encrypt with gpg

And more.

Then it occurred to me... all these tools do essentially the same, wouldn't it be nicer to put this all in one program?

So right now I'm working on rdup-tr, "rdup-transform", which can turn rdup output into tar, cpio, pax or rdup. It relies on libarchive for the heavy lifting. But it works with a twist. You can tell rdup-tr to set up a pipeline of child processes and all the files will be pushed through these processes. This is where you can do interesting things, like compression and encryption. :-)

It is not finished, but you can already find it in the subversion repository of rdup.

How does it work?

rdup-tr will have the following synopsis:

rdup-tr [-c] -Pcmd1,opt11,...opt15 [-Pcmd2,opt21,...,opt25]... -O FMT

Where FMT can be rdup, tar or cpio. The -P flag is used to pipe the files content through the cmd mentioned (and possibly its options).

All filenames to be read are given on STDIN to rdup-tr, all output of rdup-tr is sent to STDOUT (and should be redirected to a file).

Suppose I don't want to touch the output, then I can do the following, (with -c rdup-tr will output to a tty)

rdup-tr -c -Pcat -Ordup

The command above will mimic the following shell statements

% for file in $files; do 
   cat $file > /tmp/tmpfile
   cat /tmp/tmpfile | rdup-tr -c -Ordup
  done

Then /bin/cat will be used to post-process the files. With compression it will be more fun, like so

rdup /dev/null ~ | rdup-tr -Pgzip,-c -Ordup > compressed-output.rdup

This will pipe the files' content through gzip -c and will re-create the normal rdup output.

Some samples

After some coding the following works, but don't mind the debugging statements.

% rdup-tr -V
rdup-tr 0.6.4

So lets play

% rdup-tr -Pcat,-n > /tmp/mytar
** rdup-tr: Child seen cat
** rdup-tr: Childeren 1
rdup.c
** rdup-tr: Handling child #0 of 1
** rdup-tr: Duping tmpfile
** rdup-tr: Exec cat
** rdup-tr: Childs alive; returning
** rdup-tr: Parent pipe write fd# 6
child.c
** rdup-tr: Handling child #0 of 1
** rdup-tr: Duping tmpfile
** rdup-tr: Exec cat
** rdup-tr: Childs alive; returning
** rdup-tr: Parent pipe write fd# 6
^D

Okay, lets see what we have

% tar tf mytar 
rdup.c
child.c

% tar xf mytar

% cat rdup.c | head -5       
 1  /* 
 2   * Copyright (c) 2005 - 2008 Miek Gieben
 3   * See LICENSE for the license
 4   */
 5

Yes! It has been transformed by running it through cat -n.

I only need to make it generate rdup-like output and maybe speed it up a bit. Often used commands should be made a builtin, just as in shells.