Proposal for svnadmin copy command

I. Purpose
-----------------------------
     To make hot copies of subversion repositories. Primarily to allow any file
based backup tool to be used on Subversion by backing up copies created by this
command.

II. Hot-backup.py
-----------------------------
     Right now there is hot-backup.py script to do hot backups. It keeps 64 (by
default) most recent backups. Backups are named by the youngest revision in
repository with sequence number for cases when repository's youngest revision
has not changed since the last backup.

     The following algorithm is employed, to do backups:

Step 1: Get youngest revision. If revision exists find largest available
sequence # for that revision.

Step2: Copy the whole repository.

Step3: Recopy all Berkeley log files.

Step4: Recover the copy repository.

Step5: Check if lock file (specific to hot-backup.py) exists, if so then quit.

Step6: In the original repository remove all unused log files.

Step7: Remove lock.

Step8: Delete all but the newest N backups.

III. Concerns regarding Hot-backup.py
-----------------------------

      Between Step2 and Step3 log files are copied twice. If one were to copy
tables first and logs second it would be compliant with Berkeley hot backup
docs. Nothing bad, but inefficient.

     In Step6 there is a potential for a log file to be deleted before it was
ever copied. I described a way that could happen in one of my previous
emails and I am copying it here:

> I have a concern about unconditionally deleting unused log files in the
> source repository. Imagine the following scenario:
> While logs a being copied (step 3) there is repository activity that
> causes the latest log to go from used to the unused state (e.g. reached
> maximum size limit). If at (step 5) we just delete unused logs we would
> delete a log that was not copied fully.

IV. New svnadmin copy command
-----------------------------
     Here is plan I have regarding implementation of such command. The proposed
implementation supports parameter that specifies whether copied unused log files
will be deleted.

1a. (Source) Acquire shared lock on db.lock. Indicates that we working with db env.

1b. (Source) Acquire shared (no log archiving) or exclusive (log archiving) lock
on dblogs.lock.

2. (Source -> Copy) Copy repository recursively with the following exceptions:
   db - Has to be copied separately, in specific sequence.

   locks - Do not copy locks, recreate them. E.g lock file could be locked
exclusively and thus cannot be copied.

   dav - Do not copy, transient information (please correct me, if my
understanding of this area is wrong).

3a. (Copy) Recreate locks directory.

3b. (Copy) Acquire exclusive lock on db.lock.

4a. (Copy) Recreate db directory.

4b. (Source -> Copy) Copy tables.

4c. (Source -> Copy) Copy all logs files.

4d. (Copy) Run recovery on the copy.

4e. (Source) Optionally delete unused log files from source repository that are
identical in the copy. (See comment about lost logs above)

5a. (Source) Release dblogs.lock

5b. (Source) Release db.lock

5c. (Copy) Release db.lock

Sincerely,
Vladimir Berezniker

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Wed Aug 13 20:04:11 2003