[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Incremental svn dump script

From: Neil Gunton <neil_at_nilspace.com>
Date: 2004-09-12 18:25:11 CEST


I don't know if this is a silly thing that I'm trying to do, or maybe I
have just misunderstood the docs. However, I thought I'd raise it in
case it's useful to other people.

I want to be able to back up my subversion repository regularly to
another machine, preferably using rsync. However I don't want to copy
the whole thing, since it currently runs to over 10MB gzipped. Thus I
would like to do a nightly incremental dump and then just copy the (very
small) changes.

The 'svnadmin dump' command requires you to enter the range of revision
numbers (X:Y) for the dump, so you need to know what the last revision
was that was dumped. Unless I missed something in the docs there doesn't
seem to be any easy automatic way to do this within the toolset that
subversion supplies. So, I wrote a little Perl script that keeps a note
of the last version that was dumped in the same directory as where the
files are dumped to.

The script basically does the following:

1. Take two parameters: The repo dir, and the destination dump dir.
2. Use svnlook to get the current revision (youngest).
3. Compare this to the number stored in the 'last_revision' file in dump
4. If youngest is > last_revision then do command:
   svnadmin dump -r $next_revision:$youngest $repo_path --incremental |
gzip -c > ...
    The output is stored in $next_revision-$youngest.dump.gz.
5. The last_revision file is updated with the 'youngest' rev number.

So, for example, if your repository is currently at version 60, and this
is the first time you've run the script, then you might create a
directory at /mirror/svn and run:

dump_svn /var/svn /mirror/svn

The script would dump the repository into /mirror/svn/0-60.dump.gz, and
put '60' into /mirror/svn/last_revision.

If you then committed a few revisions so that the current revision is
now 65, and ran the script again, then it would incrementally dump revs
61 to 65 into /mirror/svn/61-65.dump.gz, and /mirror/svn/last_revision
would now contain 65.

So, the /mirror/svn dir has a list of files that incrementally track the
changes to the repository. Only the new changes are written, which makes
this a good candidate for rsync backup.

Here's the script, let me know if this is useful or if I am simply doing
something really redundant or stupid... incidentally, this has been
tested on Linux, I guess it could be adjusted to run on Windows but I
haven't tried.



p.s. Look out for linebreaks introduced by the email program...

#!/usr/bin/perl -w
# dump_svn v1.0
# Incremental dump of subversion repository to a specified dump
# Syntax: dump_svn <repository dir path> <dest dir path>
# Uses 'last_revision' file in dump dir to keep track of last rev dumped
# Intended for running on same machine as repository
# By Neil Gunton 2004
# This program can be used and distributed freely

use strict;

my $last_rev_file = 'last_revision';

# Check param
my ($repo_path, $dest_path) = @ARGV
or die "Syntax: dump_svn <repository dir path> <dest dir path>";

# Delete any trailing slash on dest path
$dest_path =~ s/\/$//;

# Use svnlook to see what the current version is
my $youngest = `svnlook youngest $repo_path`;
chomp ($youngest);

# Read last revision saved
my $last_revision = 0;
if (-e "$dest_path/$last_rev_file")
    open (FILE, "< $dest_path/$last_rev_file")
        or die "open for read failed for $dest_path/$last_rev_file: $!\n";
    $last_revision = <FILE>;
    chomp $last_revision;
    close (FILE);

# Dump if youngest revision is younger than last dump
if ($youngest > $last_revision)
    my $next_revision = $last_revision ? $last_revision + 1 : 0;
    my $command = "svnadmin dump -r $next_revision:$youngest $repo_path
--incremental" .
                  " | gzip -c > $dest_path/$next_revision-$youngest.dump.gz";
    print "$command\n";
    system ($command);

# Save latest revision
open (FILE, "> $dest_path/$last_rev_file") or die "open for write failed
for $dest_path/$last_rev_file: $!\n";
print FILE $youngest;
close FILE;

# Finish

To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Received on Sun Sep 12 18:25:44 2004

This is an archived mail posted to the Subversion Users mailing list.