When doing a remote backup you are probably using a non-root account and ssh. As using the root account with ssh is considered bad practise. This has the consequence that all files created "at the other side" have their ownership set to the user that is used for the transfer. In my case this means all files and directories are of miekg/miekg.

There are a few ways around this:

1. Use root to transfer the files. Only this user can change the ownership to other users.

2. Maybe there is some SELinux or a capability which can give ordinary user the ability to change file ownership?

3. Store an unpacked rdup archive. This is just like using tar.

4. Store the original ownership data in a separate file and re-insert this info upon a restore.

Item 1. and 2. speak for them self, so I want to detail 3. and 4.

3. ala tar

With not unpacking I mean just leaving the original rdup -c output, for a full backup this is (with added compression):

rdup -c /dev/null /home | gzip > /tmp/home.rdup.gz

Now you can transfer home.rdup.gz anywhere you want. Unpacking/restoring becomes:

zcat home.rdup.gz | rdup-up -t /saved_files

This is exactly the same as using tar and it also has the same problem. The archive file can become very large. I'm using the same technique however to store backups on Amazon's S3.

4. Separate storage of ownership

It is possible to have an unpacked archive at a remote system and being able to restore this with the original ownership information intact.

I'm still pondering on how to integrate this nicely in rdup-simple. Ideas welcome!

To make it all work we would need the following:

• a list of all the files and their ownership
• a means to update this list with every incremental backup
• a means to re-insert this information when restoring.

creating ownership list

This can be done with the (new! in the not yet released 1.0.0 version) rdup-up -vv switch. This will tell rdup-up to dump a file of uid/gid and filenames to standard output. It looks like this:

rdup -c /dev/null ~/bin | rdup-up -vvnt /tmp/back
+ 0 0 /tmp/back/home
+ 1000 1000 /tmp/back/home/miekg
+ 1000 1000 /tmp/back/home/miekg/bin
+ 1000 1000 /tmp/back/home/miekg/bin/apache2-ssl-certificate
+ 1000 1000 /tmp/back/home/miekg/bin/cx
+ 1000 1000 /tmp/back/home/miekg/bin/docpurge
+ 1000 1000 /tmp/back/home/miekg/bin/eee
+ 1000 1000 /tmp/back/home/miekg/bin/gb.pl
+ 1000 1000 /tmp/back/home/miekg/bin/gconf
...

The output is formatted as: +|- uid gid filename .

During the backup you must save this list somewhere.

updating the list with each incremental

Naturally when you make an incremental dump this list should also be updated as files get removed and others get added. So for now, Perl to the rescue. The following script reads 2 rdup-up -vv created lists and will merge them to a new one.

uid-merge:

#!/usr/bin/perl -wl
use strict;

my %l;
die "Need a conv file (rdup-up -vv output)\n" if ($#ARGV == -1);
open CONV, $ARGV[0] or die "Can not open $ARGV[0]\n";
shift;

while (<CONV>) {
chomp;
my ($p, $uid, $gid, $name) = split / /, $_, 4;
$l{$name} = [ $uid, $gid ];
}
close CONV;

# read another conv file and merge it with the previous one
# - -> remove entry
# + -> overwrite OR add entry
while (<>) {
chomp;
my ($p, $uid, $gid, $name) = split / /, $_, 4;

if ( defined $l{$name} ) {
    # we got something in the previous list
    if ($p eq "+") {
    $l{$name} = [ $uid, $gid ];
    } else {
    delete $l{$name};
    }
} else {
    # nothing in previous list
    if ($p eq "+") {
    $l{$name} = [ $uid, $gid ];
    }
}
}

foreach (keys %l) {
print "+ @{$l{$_}} $_";
}

Usage:

uid-merge saved-file < incremental-update > saved-file2`
mv saved-file2 saved-file

And saved-file is your new list.

re-inserting this information during a restore

Now we want to fool rdup so that the correct uid/gid information is inserted upon a restore. The following script will just do that. It will read the standard rdup output and will replace the uid/gid info with the one from saved-list. Let's recall how standard rdup output looks:

rdup /dev/null ~/bin | sed -n '4,5p'
+- 0775 1001 1001 39 902 /home/miekg/bin/apache2-ssl-certificate
+- 0775 1001 1001 18 14 /home/miekg/bin/cx

Now uid-conv (displayed below) comes in. In our case it will replace those 1001's with the (correct) values of 1000.

rdup /dev/null ~/bin | uid-conv saved-file | sed -n '4,5p'
+- 0775 1000 1000 39 902 /home/miekg/bin/apache2-ssl-certificate
+- 0775 1000 1000 18 14 /home/miekg/bin/cx

uid-conv:

#!/usr/bin/perl -wl
use strict;

my %l;
die "Need a conv file (rdup-up -vv output)\n" if ($#ARGV == -1);
open CONV, $ARGV[0] or die "Can not open $ARGV[0]\n";
shift;

while (<CONV>) {
chomp;
my ($p, $uid, $gid, $name) = split / /, $_, 4;
$l{$name} = [ $uid, $gid ];
}
close CONV;

while (<>) {
chomp;
my @e = split / /, $_, 7;
if ( defined $l{$e[6]} ) {
    ($e[2], $e[3]) = @{$l{$e[6]}}
}
print "@e";
}

putting it all together

For the sake of simplicity only a full dump is shown.

Making the full backup (from mymachine to backuphost):

rdup -c /dev/null ~/bin | ssh miekg@backuphost \
'rdup-up -vv -t /backups/mymachine > /backups/my.list'

We can forgot about uid-merge as we only do a full backup.

Restore (from backuphost to mymachine). Note this MUST connect as root to mymachine otherwise you have the same problem as when you created the backup...So for the restore you must enable root-access in ssh. But after the restore you can disable it again.

rdup /dev/null /backups/mymachine | \
uid-conv /backups/my.list | rdup-tr | \
ssh root@mymachine 'rdup-up -t /home/restore'

A few things happen here. We let rdup (without the -c switch!) create a file list of the files in the backup. Then we use uid-conv to re-insert the correct uid/gid information. Next we give the file list to rdup-tr which will read the files' content from disk. So after rdup-tr has run we have correct meta data (i.e. correct ownership) and the file data. This is then given to ssh which will connect to mymachine where rdup-up is run which restores it to /home/restore.

And presto.