still don't have a title

Rebuilding the Archive Using Cowbuilder

Did you ever want to rebuild the archive, but setting up this whole wanna-build/sbuild/buildd stuff somewhat scared you? Then this small script might be for you. All you need is cowbuilder, the rights to build inside a cowbuilder environment and this script.

When you call it with a list of packages as argument it will build every package in a clean cowbuilder environment, saving the buildlog (succeeded-$package or failed-$package) and cleaning up the rest afterwards.

To speed up things you can run several instances of this script concurrently – lockfiles will prevent the script from building the same package more than once.

It is even safe to kill the script and restart it later. It will automatically skip all packages where a logfile (succeded|failed)-$package already exists. All you have to do after you killed the script is to remove the lockfile in the LOGDIR and the $package-directory in the BUILDDIR.

#!/bin/sh
# Version: 2007-02-20

LIST=$1

LOGDIR=$(pwd)/"logs"
BUILDDIR=$(pwd)/"build"
BASEPATH="/var/cache/pbuilder/base-testing.cow"

function build {
    PACKAGE=$1

    mkdir -p $BUILDDIR/$PACKAGE
    cd $BUILDDIR/$PACKAGE

    apt-get source $PACKAGE &> /dev/null
    cd $(find . -type d ! -name .)
    pdebuild --pbuilder cowbuilder --use-pdebuild-internal --buildresult .. -- --basepath $BASEPATH &> $LOGDIR/.$PACKAGE

    if [ $? -eq 0 ]; then
        mv $LOGDIR/.$PACKAGE $LOGDIR/succeeded-$PACKAGE
    else
        mv $LOGDIR/.$PACKAGE $LOGDIR/failed-$PACKAGE
    fi

    cd $BUILDDIR
    rm -rf $PACKAGE
}


[ -d $LOGDIR ] || mkdir $LOGDIR

if [ ! -e $LIST ]; then
    echo "No Buildlist given."
    exit 1
fi

while read package; do
    # Create Lockfile (ln -s is atomic and fails if link already exists)
    ln -s $package $LOGDIR/.$package.lock 2> /dev/null || continue

    # Build the package if not already built
    if [ ! -e $LOGDIR/succeeded-$package -a ! -e $LOGDIR/failed-$package ]; then
        echo $(date +%F\ %T) Building $package...
        build $package
    fi

    # Remove Lockfile
    rm -f $LOGDIR/.$package.lock
done < $LIST

I know my bash scripting skills are shitty, so please send patches after you laughed ;)

I’m currently running two instances concurrently for 3 days on my Pentium 2,4GHz HT box and have rebuilt ~70% of the testing archive so far.

It would be cool to distribute the rebuilt across a few computers via ssh, so that they’re all building on the same list but using their own cowbuilders. It should be sufficient if all computers shared the same LOGDIR across the network, but I’m not sure how to do this. NFS does not the trick since it cannot guarantee the atomicity of the ln-command.