You are using the ControlMaster feature of ssh and you are having problems remembering which shell is the master?

That's why I've made zsh display this in the prompt. You get a m@ when a connection is a master connection and an @ if it is a slave.

setting up

The check if we are a master works by creating a file which has $SSH_CLIENT as the name, as the following snippet shows.

# ssh stuff
# check if we are the master or a slave connection
# this must be sourced before .zprompt is

# print @-sign before the hostname if we got into this host via SSH
if [ -z $SSH_TTY ]; then
ZSSH=
else
if [[ ! -e ~/.ssh/${SSH_CLIENT// /_} ]]; then
    # file does not exist, we are a master
    touch ~/.ssh/${SSH_CLIENT// /_}
    ZSSH="m@"
else
    # slave connection
    ZSSH="@"
fi
fi

tearing down

Make zsh remove the file when it exits.

TRAPEXIT() {
echo $PWD > ~/.zpwd
# $ZSSH has @ for slaves and M@ for masters
[[ $ZSSH == "m@" ]] && rm ~/.ssh/${SSH_CLIENT// /_}
}

You can now use $ZSSH in your prompt.

On this last day of 2008 I'm releasing rdup 0.6.4. This is pretty much 0.6.4-rc1 with a few minor documentation tweaks.

One little shell script that has come back which is rdup-ln.sh which originally was called rdup-snap-link and is used in rdup-simple to figure out what kind of backup (incremental/full) should be made. In rdup-backups(7) a sample implementation for your own rdup-simple is listed, as I've dropped rdup-simple from rdup.

The changelog for this release:

• rdup-up: further updates and refinements
• rdup-up: added strippath() function to strip leading path components
• doc: minor updates to the manual pages, added a rdup-simple like script to rdup-backups.7
• rdup-ln.sh: a shell script for checking and copying a previous rdup backup. Depends on GNU date and GNU cp. Installs into /usr/lib/rdup/. Previously a small script called rdup-snap-link did this.

Quick download link

What is next?

• I'm contemplating releasing rdup 1.0.0, with rdup, rdup-tr and rdup-up my feeling is that rdup is complete
• for rdup development I'm going to move to 'git'

The script in using git and vi together was a little bit borked, as too many git repo's were created. Hence a new and improved version. This one will look up the directory tree to spot an existing git repository - is nothing found a new one will be created in $PWD.

code

#!/bin/zsh
# a wrapper around git and vi
# expands $Hash$ to $Hash: file short_hash epoch committer $
# git checkout HEAD $file? when would I need this
# TODO: spaces in filename

[[ ! -x =git ]] && exit 1

who=$SUDO_USER
who=${who:-$LOGNAME}

function search_git_dir {
    gpath="$1"

    [[ -d "$gpath/.git" ]] && echo "$gpath" && return
    [[ -z "$gpath" ]] && echo "" && return

    # strip that last path component and try again
    search_git_dir "${gpath%/*}"
}

for file in "$@"; do
dir=$(dirname $file); cd $dir
base=$(basename $file)

if [[ -z $(search_git_dir $PWD) ]]; then
    # make a new one in $PWD
    git init || exit 1
else
    #echo FOUND ONE
fi
chmod +w $base 2> /dev/null

if vi $base; then
    [[ ! -e $base ]] && exit 0
    git add $base
    # collapse $Hash: id $ line
    sed -i -e 's/\$Hash:.*\$/$Hash$/' $base
    git commit $base
fi

id=$(git-show -s --pretty=format:$base\ %h\ %ct\ $who%n -- $base) 
[[ -z $id ]] && exit 1

# re-add $Hash: sha1hash$ line
sed -i -e 's/\$Hash\$'/\$Hash:\ $id\ \$/ $base

chmod a-w $base 2> /dev/null
done

manpage

And ofcourse the manpage.

'\" t
.TH GITVI 1 "27 Dec 2008" "0.1.0" "gitvi"

.SH NAME
gitvi \- a wrapper around git and vi(m)

.SH SYNOPSIS
gitvi [\fIFILE\fR]

.SH DESCRIPTION
Edit files and commit them to a git repository. If FILE is not
given nothing is done.

All the directories from the current one up to the root are
search for a previous .git directory. If no directory is found
a new git repository is created in the current directory. A
new repository will never be created in the root directory.
.PP
The special sequence $Hash$ is expanded by \fBgit-vi\fR to 

.RS
$Hash: sha1hash filename epoch committer $
.RE
.PP
This expanded $Hash$ syntax is \fInot\fR commited to git, this
is done after the commit has taken place.

.SH OPTIONS
There are no options as of yet.

.SH AUTHOR
Miek Gieben <miek@miek.nl>

.SH SEE ALSO
vim(1), git(1).

A long while back I used vi together with rcs to manage files in /etc. This worked, but I found rcs to be clunky. So welcome to the 21st century and my git + vi script.

It is a wrapper around git, which will create a new git repository, add the file and commit it when changed.

The script will also expand $Hash$ to

$Hash: basename-of-file short-commit-hash epoch committer`

a typical example is

$Hash: motd.tail 2ebb8e9 1230404370 miekg $

This is somewhat the same as the $Id$ tag from svn/CVS, but with a different tag. I always found this tagging from subversion useful, esp. when you -- as a human -- needs to tell if a file is under version control.

To avoid accidental edits the file is set to read-only.

I think I'm gonna dump cfengine and just use this script, cfengine is great, but somewhat of a overkill for my home setup.

Sourcecode of the script

The script uses zsh, but can be easily converted to bash or any other shell.

#!/bin/zsh
# a wrapper around git and vi
# expands $Hash$ to $Hash: file short_hash epoch committer $
# git checkout HEAD $file? when would I need this

[[ ! -x =git ]] && exit 1

who=$SUDO_USER
who=${who:-$LOGNAME}

for file in "$@"; do

dir=$(dirname $file); cd $dir

chmod +w $file 2> /dev/null

if [[ ! -d .git ]]; then
    git init || exit 1
fi

if vi $file; then
    [[ ! -e $file ]] && exit 0
    git add $file
    # collapse $Hash: id $ line
    sed -i -e 's/\$Hash:.*\$/$Hash$/' $file
    git commit $file
fi

id=$(git-show -s --pretty=format:$(basename $file)\ %h\ %ct\ $who%n -- $file) 
[[ -z $id ]] && exit 1

# re-add $Hash: sha1hash$ line
sed -i -e 's/\$Hash\$'/\$Hash:\ $id\ \$/ $file

chmod a-w $file 2> /dev/null
done

update

I just thought of something, what if there already is a git repository in a higher directory....Hmmm. This means the script should look for a git repository higher up and work from there....

rdup 0.6.4-rc1 has just been released. I've decided to call it an -rc1 release because of the number changes as there are two new programs:

• rdup-tr, a general transformation util which supersedes rdup-crypt, rdup-gzip and rdup-gpg (and it can do more)
• rdup-up, a replacement of rdup-snap

I haven't tested rdup-up to the fullest, so there may be problems when updating an existing directory structure. If there problems I love to hear about them. rdup-tr is two weeks older than rdup-up, so it has seen some more testing :) Both tools are implemented in C. This means there is no Perl dependency left for creating backups/using rdup.

See these blog entries for more information:

• rdup-tr
• rdup-up

There were also some minor bugfixes in rdup, concerning the ouput of suid/sgid bits and handling of the regular expression linked list.

Quick download link

The full ChangeLog for this release:

17 Dec 2008: 0.6.4-rc1 Miek <miek@miek.nl>
  - major feature enhancements, two new programs
+ rdup-tr: transform rdup output to something else
+ rdup-up: update a directory tree with rdup archive

 * rdup: Added '%N' to the format string, which prints the
   path, but in case of soft- and hardlinks it will only
   print the link's name not the '-> target' part
 * rdup: rdup.1.in had a long deserved make-over
 * rdup: Print symlinks correctly when there named on the cmd line
 * rdup: regexp: don't use g_slist_next, but k->next in gfunc.c
 * rdup: Suid/sgid/sticky bit output was corrected
 * rdup-tr: Added rdup-tr; a tool to convert rdup output to
   (modified) rdup output, tar, pax or cpio files
 * rdup-tr: Don't compile rdup-tr if libarchive is not found
 * rdup-tr: rdup-tr.1 manpage
 * rdup-up: Added rdup-up; C version of rdup-snap
 * rdup-up: Added rdup-up.1 manpage
 * rdup-up: Support for fifos, pipes, devices
 * updated rdup-backups.7 as general info about backups with rdup
 * Set the copyright to 2009 already

When all goes well, I expect to release rdup 0.6.4 before the end of the year.

I was getting sick and tired of the Perl implementations, hence rdup-tr and now rdup-up. With rdup-up you can update a directory tree with an rdup archive. So right now there are three programs that make up the rdup-suite.

• rdup - this creates a file list (possible with file content rdup -c)
• rdup-tr - transform an rdup list to an archive (possible rdup -c)
• rdup-up - update the filesystem

The only Perl program left is rdup-s3 for uploading an archive to Amazon's S3 service. I'm not sure how to deal with this situation. Maybe I will fold the rdup-utilities back into the main rdup archive again. Or create a contrib/ directory where these will live? Don't know yet.

One things that will not come back is rdup-simple, or at least not in the form it was. The main reason for this is that you are so flexible with rdup, rdup-tr and rdup-up that an extra script is an overkill, IMO.

Again a few examples

A compressed dump of /bin to /vol/backup

rdup /dev/null /bin | rdup-tr -Pgzip,-c,-f |\
rdup-up -v /vol/backup

Or, you want to go remote?

rdup /dev/null /bin | rdup-tr -Pgzip,-c,-f \
-Popenssl,enc,-e,-des-cbc,-k,mysecret | \
ssh -C user@remotehost rdup-up /vol/backup/myhost/today/

Then, don't forget the encryption. Restoring is actually reversing the command line:

rdup /dev/null /vol/backup/myhost/today | \
rdup-tr -Popenssl,enc,-e,-des-cbc,-k,mysecret \
-Pgzip,-c,-f |\
ssh -C user@myhost rdup-up /bin

A full backup with compressed and encrypted files in it?

./rdup -R /dev/null ~/bin | \
./rdup-tr -Pgzip,-c -Pmcrypt,-F,-f,keyfile,-q -Otar | \
tar xvCf /tmp/A -

And believe it or not (this is on AMD64), rdup is 32K is size and rdup-tr is 20K in size (stripped ofcourse).

Source code is available in the subversion repository of rdup.

rdup included some utilities for manipulating it's output. These utilities have been split-off since the rdup 0.6.2 release. Some of the tools included were:

• rdup-gzip, compress the output
• rdup-mcrypt, encrypt the output
• rdup-gpg, encrypt with gpg

And more.

Then it occurred to me... all these tools do essentially the same, wouldn't it be nicer to put this all in one program?

So right now I'm working on rdup-tr, "rdup-transform", which can turn rdup output into tar, cpio, pax or rdup. It relies on libarchive for the heavy lifting. But it works with a twist. You can tell rdup-tr to set up a pipeline of child processes and all the files will be pushed through these processes. This is where you can do interesting things, like compression and encryption. :-)

It is not finished, but you can already find it in the subversion repository of rdup.

How does it work?

rdup-tr will have the following synopsis:

rdup-tr [-c] -Pcmd1,opt11,...opt15 [-Pcmd2,opt21,...,opt25]... -O FMT

Where FMT can be rdup, tar or cpio. The -P flag is used to pipe the files content through the cmd mentioned (and possibly its options).

All filenames to be read are given on STDIN to rdup-tr, all output of rdup-tr is sent to STDOUT (and should be redirected to a file).

Suppose I don't want to touch the output, then I can do the following, (with -c rdup-tr will output to a tty)

rdup-tr -c -Pcat -Ordup

The command above will mimic the following shell statements

% for file in $files; do 
   cat $file > /tmp/tmpfile
   cat /tmp/tmpfile | rdup-tr -c -Ordup
  done

Then /bin/cat will be used to post-process the files. With compression it will be more fun, like so

rdup /dev/null ~ | rdup-tr -Pgzip,-c -Ordup > compressed-output.rdup

This will pipe the files' content through gzip -c and will re-create the normal rdup output.

Some samples

After some coding the following works, but don't mind the debugging statements.

% rdup-tr -V
rdup-tr 0.6.4

So lets play

% rdup-tr -Pcat,-n > /tmp/mytar
** rdup-tr: Child seen cat
** rdup-tr: Childeren 1
rdup.c
** rdup-tr: Handling child #0 of 1
** rdup-tr: Duping tmpfile
** rdup-tr: Exec cat
** rdup-tr: Childs alive; returning
** rdup-tr: Parent pipe write fd# 6
child.c
** rdup-tr: Handling child #0 of 1
** rdup-tr: Duping tmpfile
** rdup-tr: Exec cat
** rdup-tr: Childs alive; returning
** rdup-tr: Parent pipe write fd# 6
^D

Okay, lets see what we have

% tar tf mytar 
rdup.c
child.c

% tar xf mytar

% cat rdup.c | head -5       
 1  /* 
 2   * Copyright (c) 2005 - 2008 Miek Gieben
 3   * See LICENSE for the license
 4   */
 5

Yes! It has been transformed by running it through cat -n.

I only need to make it generate rdup-like output and maybe speed it up a bit. Often used commands should be made a builtin, just as in shells.