proposed: move FSFS protorevs out of the transactions dir

The access patterns of the files in the FSFS repository fall into
three distinct patterns:

* The proto-revision files (transactions/<txnid>.txn/rev)

  These are appended to during the course of the transaction, and
  eventually renamed into place into the revs directory. These can be
  arbitrarily long. I don't believe that any normal Subversion
  operation even reads from the txn file (even though
  open_and_seek_transaction exists, I don't think anything will
  actually call it, though feel free to correct me on that).

* Everything else in the transaction directory
  (transactions/<txnid>.txn/rev)

  For example, txn noderevs, directory lists, properties, etc. These
  are repeatedly read and written during the transaction, and are
  generally relatively short.

* The revision data (revs/*)

  This is read-only.

(Yeah, there are some other random things, but these are the biggies.)

Administrators may wish to use different forms of storage for these
different kinds of data. For example, the actual revision data (and
the current and txn-current files) may want to live on a network file
share; the transaction-specific data may want to live on local disk in
the server (which requires using a server setup that guarantees that
all operations on the same transaction use the same server).

This can easily be accomplished by symlinking "db/transactions/" to
local disk. (Well, at least, since r28578 fixed a bug in cross-device
copies.) This lets you keep the noderev/directory list/txnprops/etc
read/write cycles on local disk. On the other hand, it'll also keep
the (potentially large) protorev file, which is only appended to
during commit, on local disk, and then the move-into-place during the
write-locked part of the commit becomes a non-constant-time copy. And
the benefit of having the transaction data on local disk just doesn't
seem to apply to the append-only proto-rev.

Thus, it seems reasonable to allow administrators to use symlinks in
this way to move the non-protorev txn data to a different filesystem
without moving the protorevs. I suggest the attached patch, which
moves the protorev (and its lock) into db/txn-protorevs/. We're
already bumping the filesystem format in 1.5...

[[[
In already-bumped FSFS version 3, store the protorev and its lock
separately from the rest of the transaction metadata, to make it easy
for administrators to put the non-protorev metadata on a different
filesystem from the immutable filesystem.

* subversion/libsvn_fs_fs/fs.h
  (PATH_TXN_PROTOS_DIR, PATH_EXT_REV, PATH_EXT_REV_LOCK): New.
  (PATH_REV, PATH_REV_LOCK): Move later and document as legacy.
  (SVN_FS_FS__MIN_PROTOREVS_DIR_FORMAT): New.

* subversion/libsvn_fs_fs/fs_fs.c
  (path_txn_proto_rev, path_txn_proto_rev_lock): Return the new path
   if the format is new enough.
  (svn_fs_fs__hotcopy): Make an empty txn-protorevs directory if the
   format supports it.
  (svn_fs_fs__purge_txn): If the format is new enough, try to delete
   the protorev file and its lock after removing the txn directory.
  (svn_fs_fs__create): If the format is new enough, create
   txn-protorevs directory.

* subversion/libsvn_fs_fs/structure
  (Layout of the FS directory): Document txn-protorevs.
  (Filesystem formats): Document difference between versions.
  (Transaction layout): Document difference between versions.
]]]

-- 
David Glasser | glasser_at_davidglasser.net | http://www.davidglasser.net/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org