The upcoming release for rdup will be 0.9.0, after that I'm declaring a feature freeze.

The roadmap up to 1.0.0 will be

• Feature freeze. All cool, new ideas will have to wait until post 1.0.0
• Thus only bugfixes
• No internal filelist changes

Post 1.0.0 will be

• i18n stuff - framework is there, just need to be used
• ...

A time schedule is hard to give, but I want kill any remaining bugs and issues in the 0.9.x series and then release an 1.0.0

If after an 0.9.x release nobody complains for a month, that 0.9.x release will be re-released as 1.0.0.

I've started rdup development in

% git log | tail -6 | grep -i '^date'
Date:   Thu Dec 22 09:16:14 2005 +0000

2005! So February/March 2009 seems like a decent time to release the 1.0 version :-) More than three years of coding for a program for which the core algorithm was written in three days...Christmas 2005 was good: beer, coffee, music and coding.

Today if removed two annoyances I had with VIM.

Command mistyping

I often mistype the following:

:w! becomes :W!

or

:q! becomes :Q!

Which is annoying because W and Q do not mean anything, and I don't write or VIM does not quit. VIM has a nifty feature called commands which you can (re)define or add new commands.

Lets try some to fix this, in a running vi:

:com -bang W write!

or

:com -bang Q quit!

Now you can write :Q! and you will force quit.

But it will now always use the force variant because the bang (!). Well of course VIM would not be VIM, if there weren't a solution:

:com -bang W write<bang>
:com -bang Q write<bang>

Mode has changed

This was the other annoyance

Warning: Mode of file "{filename}" has changed since editing started

Kill it with set autoread in your .vimrc. See :help W16 too.

I'm starting to like git more and more. Today I had the pleasure of receiving the first patch for rdup made by git.

According to Tom Hendrikx (who send the patch). It was easy:

$ git pull
$ $EDITOR <whatever>
$ git commit
$ git format-patch -1
$ git send-email *.patch

And at my side:

$ mutt
<save the mail in ~/patch>
$ cd git/rdup
$ git-am --signoff ~/patch

Ready, done, fixed.

Thanks for John Goerzen for nudging me in the right direction.

When you backup to a remote machine you loose the ownership information on your files, because you are logging in a as a normal user with - say - uid 1000. Then all files created are of uid 1000.

(for gid the same holds, but I'll skip that for this story).

rdup does have this information, but it is not used when doing a remote backup. For local backups this normally is not a problem because that runs as root. And the root user can set the ownership.

So, how can we fix this? We would need the original user and group information. The best spot for this would be rdup's internal file list. Unfortunately this info is not saved there. This is why I'm going to change that for the next rdup release (0.8.0).

If the user/group info is stored in the file, you can do the following.

1. Transfer this file to you backup medium
2. Convert it to a normal rdup list
3. Feed this list to rdup-tr, which read the file back from disk
4. Feed rdup-tr's output to rdup-up to re-create the files; rdup-up must then run with root permissions.

Or in code:

Step 1.

rdup -N timestamp LIST ~/bin | rdup-tr | \
ssh miekg@remotehost rdup-up -t /tmp/boe/backup
scp LIST miekg@remotehost /tmp/boe/backup

Step 2 + 3 + 4.

As root:

ssh miekg@remotehost "path-tr.pl < LIST | rdup-tr -c" | rdup-up /new

I have not test this (yet) but it seems like a workable solution. The code of path-tr.pl is listed below. Again: there is still no uid/gid information in the internal filelist, so currently this WILL NOT WORK.

This stuff is still in the early stages and eventually I will incorporate it in rdup-simple.

path-tr.pl

For converting an internal rdup list in normal rdup output. Here we always substitute 1000/1000 for the uid/gid information and I'm prefixing it with the backup path /vol/backup in the variable $prefix so the rdup-tr may actually find the files.

#!/usr/bin/perl

use Fcntl ':mode';
use feature 'switch';

use warnings;
use strict;

# convert rdup's internal list to rdup output
# if you save this list on your backup. You
# can backup as a normal user and save the
# original user/group info.
#
# If you feed this to rdup-tr you get your
# backup back in the original state
# (otherwise all files would user/user, where
# user is the user running the ssh)

# Internal list looks like: (may change)
# # mode dev inode linktype pathlen filesize path
# 16877 2309 2 * 5 4096 /home
#
# rdup output looks like
# #type perm user/group pathlen filesize path
# +d 0755 0 0 5 0 /home

use constant { 
MODE    => 0,
DEV     => 1,
INODE   => 2,
LINK    => 3,
PLEN    => 4,
FSIZE   => 5,
PATH    => 6
};

my $prefix = "/vol/backup";
my $prelen = length($prefix);

my @p;
<>;
while (<>) {
chomp;
@p = split / /, $_, 7;

if (S_ISDIR($p[MODE])) { $p[FSIZE] = 0; }

if (defined $prefix) {
    $p[PLEN] += $prelen;
    $p[PATH] =  $prefix . $p[PATH];
}

# plusmin
print "+";

# type, trying out the new Perl 5.10 stuff
given ($p[MODE]) {
    when (S_ISDIR $_) {
    print "d";
    }

    when (S_ISREG $_) {
    if ($p[LINK] eq 'h') { print "h"; }
    if ($p[LINK] eq '*') { print "-"; }
    }

    when (S_ISLNK $_) {
    print "l";
    }

    # ...
}

# perms
printf " %04o", $p[MODE] & 07777;

# user/group (don't have that info (yet))
print " 1000 1000";

# pathlen
printf " %s %s %s\n", $p[PLEN], $p[FSIZE], $p[PATH];
}

The following might be helpful to others too. I was trying to setup a new raid1 device from two partitions /dev/sda4 and /dev/sdb4. I wanted to do this the "right way" and use UUID everywhere, i.e. in /etc/mdadm/mdadm.conf and in /etc/fstab.

I hit a few snags along the way.

create the array

# mdadm --create --verbose /dev/md6 --level=1 --raid-devices=2 \
/dev/sda4 /dev/sdb4

Get the uuid mdadm uses:

# mdadm --detail  /dev/md6 | grep UUID
  UUID : dc9aba5e:ed1a70d4:770765d8:b0f56d86 (local to host elektron2)

Check. Add that to /etc/mdadm/mdadm.conf:

# grep md6 < /etc/mdadm/mdadm.conf
ARRAY /dev/md6 level=raid1 num-devices=2 UUID=dc9aba5e:ed1a70d4:770765d8:b0f56d86

Okay, that should take care of the array. Now I wanted to get it into the fstab

blkid

The magic command here is blkid. Let's run it:

# blkid | grep md6

Huh, nothing.... hmmm. Okay maybe because there is no filesystem on it, it won't show up?

# mke2fs -j /dev/md0
# blkid | grep md6
/dev/md6: UUID="d4776336-0d4c-42e6-9b7f-a668522f3f40" SEC_TYPE="ext2" TYPE="ext3"

Great, that did the trick. The UUID you see here can be put in fstab. It is a completely different one than mdadm uses.

I've added the following to the fstab

# /dev/md6
UUID=d4776336-0d4c-42e6-9b7f-a668522f3f40 /vol ext3 relatime  0  2

Ok, lets mount it:

# mount /vol
mount: special device /dev/disk/by-uuid/d4776336-0d4c-42e6-9b7f-a668522f3f40 does not exist

WTF? So blkid is spitting out a UUID and udev hasn't updated the links in /dev.... okay. Some helpfull bug reports , here and here.

The one from Debian had the missing command:

# echo add > /sys/block/md6/uevent

And low and behold, the symlink shows up in /dev/disk/by-uuid/

# mount /vol
# cd /vol
# ls
lost+found

Done.

1. I'm very happy with Eeebuntu. Everything works perfectly: power management, disabling/enabling Wifi, special function keys, etc. No more audio skips - even with PulseAudio. Battery lifetime is now up to 3 or 4 hours (don't know yet, haven't been in the train that long).
2. It's mdadm --detail /dev/md0, not mdadm /dev/md0 --detail
3. unetbootin needs a FAT formatted USB stick. With ext* I could not get it to work.
4. Ubuntu's expert server install is tedious

Found this nice code plugin someone wrote. I'm testing it a little in some code segments below.

This is not as fancy as what the code display wordpress can do. As you cut&paste this code you will also copy the line numbers.

But heh, we all know vi. With the following search-and-replace you can remove them again:

:%s/^[0-9]* //

Some examples

1 int main(int c) {
2     printf("This is nice");
3     exit(1);
4 }

1 int main(int c) {
2     printf("This is nice");
3     exit(1);
4 }

A new release, now managed with git.

Quick download link.

For the development of rdup I'm moving to git. I'm doing this for several reasons.

1. Everybody is moving to git :)
2. gitweb is much better than the standard subversion web stuff. I can point to specific revision, which is not possible with the subversion setup I had.
3. git is much faster than svn.
4. And of course the distributed nature of git.

So for the 0.7.1 release of rdup I'm doing a last subversion release and a git release. After that I will be fully converted to git.

Repositories

I will keep the subversion repository online. If you want to play with git you can go to the gitweb interface or pull/clone from /git/rdup.git

I finally have my glass fiber Internet connection. Right now I'm figuring out how to actually implement my home network.

In the meantime I've bought a Netgear Router (WNR3500FS if I'm not mistaken), and it rules. ;-)

In the documentation it actually says:

Advanced users (Linux, Mac, etc) read the manual setup document to configure your router.

And it included the GPL and the LGPL in the paper documentation. Which they have to as the stuff runs Linux, but still.

And the router looks cool and works well. Also if you turn on the router is has a default IP address of 192.168.1.1, not something stupid like 10.0.0.138, like other devices.

What more do you want? I'm very happy to see a manufacturer deliver these kinds of products.

In bash (and other shells) you can use the [[ construct

From the bash manpage:

[[ expression ]]
          Return a status of 0 or 1 depending on  the  evaluation  of  the
          conditional  expression expression.  Expressions are composed of
          the primaries described  below  under  CONDITIONAL  EXPRESSIONS.

          Word  splitting  and pathname expansion are not performed on the

          words between the [[ and  ]];  tilde  expansion,  parameter  and
          variable  expansion, arithmetic expansion, command substitution,
          process substitution, and quote removal are  performed.   Condi‐
          tional operators such as -f must be unquoted to be recognized as
          primaries.

So no word splitting and pathname expansion.

A little example

First we have some setting up to do

$ mkdir ~/tmp/zz
$ touch ~/tmp/zz/111
$ echo ~/tmp/zz/*
/home/miekg/tmp/zz

Well, lets test [[

$ if [[  tmp/zz/* = tmp/zz/111 ]]; then echo OK; else echo NOK; fi
NOK

NOK - that's correct, because [[ will not do pathname expansion.

The other way around

$ if [[  tmp/zz/111 = tmp/zz/* ]]; then echo OK; else echo NOK; fi
OK

Huh, OK?

Why

To see why bash behaves as it does, the best explanation comes from the ksh manual. Yes, the K shell. In the section 'Conditional Expressions' of ksh's manual page

string = pattern        True, if string matches pattern.

Aha, so the right side of a = test is a pattern and a such it is expanded even though we put it in a [[ construct.

In the bash manual we just have

string1 == string2
True if the strings are equal.  = may be used
in place of == for strict POSIX compliance.

Which is not entirely correct.

In zsh (which is of course the shell) in the section 'CONDITIONAL EXPRESSIONS' it is also correct

string = pattern
string == pattern
    true if string matches pattern.  The ‘==’ form is the  preferred
    one.   The  ‘=’ form is for backward compatibility and should be
    considered obsolete.

So this might be something worth considering.

rdup 0.7.0 did not compile on Solaris (as I never tested this), but it turns out that this can be fixed with some simple tweaks.

These fixes have just (5 minutes ago) been committed to subversion. So if you check out the latest trunk it should compile on Solaris.

To make it compile I had to uncomment some GNU_GETTEXT_ macros in configure.ac, but I not really using them anyway.

There was also some nagging about major() and minor() which turned out to be a missing #include <sys/sysmacros.h>.

A colleague of mine had some nice improvements to gitvi.

Right now gitvi can be edited by itself, because the magic sequence $Hash$ is escaped like this

sed -i -e 's/\$[H]ash:.*\$/$H''ash$/' "$base"

(This was Ton's idea)

Some other improvements like a -m MSG switch which allows you to enter a commit message for all files you are editing.

And of course it then also needs to have a -h switch.

You can find the improved version here

Re-selection of a visual

When you use control-v or shift-v in vim to do a visual selection and then use (for instance) y to copy the selected text, you loose the selection.

More often than not you want to do another thing with the selection you just had. How?

gv

Copy to clipboard

Make a selection and use

"*yy

Now the selection is 'under your mouse'. If think I'm going to bind this sequence to something more easy, like *Y.

When entering a comment you can use the following simple bb tags:

• [b] - for bold text
• [i] - for italic text
• [u] - for underlined text
• [s] - for strike through text

These can all be used like so

[b]this must be bold text[/b]

Further more we have

• [h] - for a header

  header

• [code] - for a code sequence

  echo "hello you"

• [quote] - for quoting somebody

  You said

• [url] - for links

  [url]http://www.miek.nl[/url]

  Or something more advanced

  [url=http://miek.nl]Miek's site[/url]

Wikipedia has more information about bb tags.