[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Feature request: a real backup/restore system for svn.

From: Branko Čibej <brane_at_xbc.nu>
Date: 2004-06-18 04:26:59 CEST

Ben Collins-Sussman wrote:

>After fighting with issue 1817 for a few days, Karl and I have come to
>the conclusion that 'svnadmin hotcopy' is a mess, and we should
>deprecate it.... that is, remove it altogether in 2.0. Here's why:
>
> * it has a race condition with db logfile autoremoval, which is
> something that people have really come to depend on. In theory,
> the worst thing that can happen in the race condition is that the
> user sees an error and just tries 'hotcopy' again. In practice,
> who knows? We've not done any real stress-testing.
>
>
That's not a ptoblem. At worst, we can detect if automatic log file
removal is in force (DB_ENV->get_flags() & DB_LOG_AUTOREMOVE) and fail
unconditionally if it is.

> * it's completely specific to BDB. fsfs needs no such thing, and a
> future SQL backend will already have its own tools for backup.
>
>
So what? Face it: real-life backup strategies, at least those that are
efficient, will _always_ be backend-specific. For example, nobody in
their right mind would use some Subversion tool to back up their SQL
database -- they'll use the database's own backup tools. "svnadmin
hotcopy" simply happens to be a BDB-specific backup tool.

> * people currently view it as "the correct way" to make a full backup.
>
>
Well, it is, given the right circumstances.

The only problem that I do see with it is that it mixes repository
backups and filesystem backups. The FS backup is backend-specific. The
repository backup probably doesn't need any special tools, anyway
(except possibly for the DAV activity database, which we don't handle
correctly and is transient anyway).

>We think that Subversion should provide admins with a single, robust
>backup/restore solution which "just works"
>
A single solution will never "just work" in all circumstances. For
example, there's a huge difference between "hot" (zero-downtime) backups
and "ordinary" backups. Not everyone needs hot backups, but those who do
won't be happy if they have to live with dumps instead.

>(won't throw random retry errors),
>
Like I said, we can simply declare that hotcopy and automatic log file
removal don't work together, and leave it at that. That's quite acceptable.

>and is independent of repository back-end. We're probably
>talking about a smart script which
>
> * utilizes 'svnadmin dump/load' as the main mechanism for backup
> and restore,
>
> * knows when to use --incremental or not,
>
> * uses the 1.1 --deltas switch for dumping,
>
> * remembers what it's done before, and manages a collection of full
> and incremental dumpfiles,
>
> * can intelligently restore from the collection of dumpfiles.
>
>
You do realise that, in order to do this right, you're not looking at a
"smart script" but at a project the size (if not complexity) of cvs2svn?
Because managing such dumpfiles isn't going to be trivial.

Not to mention that this scheme has to take revprop changes into account.

>I'd like to file this as a feature request, but I thought it made
>sense to run it by the list for opinions first.
>
>
A comprehensive backup system will have some mix of the following features:

    * Full and incremental backups
    * Both offline and online (hot) backups
    * Backend-specific and backend-agnostic solutions

Just as an example of what I'm talking about: some databases allow you
to make incremental hot backups. These are very popular with users of
these databases. Your single solution won't provide them. So instead,
I'd suggest thinking about how to integrate Subversion backups with
existing backup tools. It can be as simple as "shut down svnserve before
starting a backup", or as complicated as "Install tool so-and-so to get
Oracle zero-downtime backup integration". Or anything in between. But
it's not a Subversion feature.

-- 
Brane Čibej   <brane_at_xbc.nu>   http://www.xbc.nu/brane/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Fri Jun 18 04:28:54 2004

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.