RE: Support for filesystem snapshots (?)

> From: Stefan Sperling [mailto:stsp_at_elego.de]
> On Mon, Aug 02, 2010 at 12:42:31PM -0400, Vallon, Justin wrote:
> > I did see that discussion, but it seems to contradict with the claim
> > that the database operations are transactional.
>
> This is a frequent misunderstanding of the "atomic commit" concept.

I suppose that I am assuming that the repository operations would all be transactional. Without transactional guarantees, you can have database corruption if a modification is interrupted.

> E.g. Subversion's FSFS needs to create a revision file from the commit's
> transaction, and move the finalized revision file into place.
> After the revision file has been moved into place successfully, FSFS also
> updates the svn:date revision property and moves the revision properties file
> into place (or copies revprop data into an sqlite database if you use
> revprop packing). Then, it updates the 'current' file which contains the
> number of the current HEAD revision. If you use representation sharing to
> save disk space, the commit may involve further updates to yet another
> sqlite database.
>
> All these actions need to complete in order to have a consistent state.
>
> If you're interested in seeing the code that does this, look at the
> svn_fs_fs__commit() and commit_body() functions in
> http://svn.apache.org/repos/asf/subversion/trunk/subversion/libsvn_fs_fs/fs_fs.c

I see this is executed with a FS write lock. My concern would be focused on the interaction between the commit code and any rollback code. For example, if the commit dies (any any point during the commit), what will be required to insure that the repository behaves as if the commit never started? Will a repo cleanup be required; will the next committer cleanup the partial rev automatically (ie: overwrite stale files); will the repo be hopelessly inconsistent?

A partially committed transaction still leaves the repository in a consistent state, so long as subsequent writers (or a special cleanup process) can undo the internal "damage", if any, made the partial commit.

> The safest way is creating a hotcopy of the live repository, and taking
> a copy of the hotcopy (by taking a snapshot of the filesystem the hotcopy
> resides in, or whatever). It's not the most efficient way of doing it,
> but the most portable and safest.

This solves the problem if you have control over both the hotcopy and the backup. In a large environment, this means that the filesystem backup needs to be synchronized with the hotcopy, or else, the hotcopy itself might run *during* the backup, and would yield an inconsistent hotcopy (hotcopy-in-progress).

> You may be lucky and always get a consistent repository state by taking
> filesystem-level snapshots of a live repository, but there's no guarantee
> that the very latest commit will always be in a consistent state.

There will be two outcomes:

1) The in-progress transaction can always be rolled back (transparently or through the use of a recovery command), leaving an intact repository

2) The repository might be corrupted

If svn cannot guarantee outcome 1, then we are left with outcome 2, and that we can be left with a corrupt repository after a commit failure.

-- 
-Justin