Performance of "svn lock *"

One of WANdisco's customers brought the performance of "svn lock *" to
my attention. There is a network issue over HTTP and a filesystem issue
in the repository.

When locking multiple files over HTTP the client sends a separate LOCK
request for each file and the round trip delays add up. Also the
bandwidth overhead, all the HTTP headers, is high: using one request per
path it is not an efficient way to transport paths.

On the repository side creating or removing a lock involves writing an
index file for each parent directory in addition to handling the lock
file itself. To lock N separate files in a directory '/A/B/C' involves
writing N times the index files for '/', '/A', '/A/B' and '/A/B/C' as
well as handling the N lock files. To lock N files at depth D we do
O(N*D) writes but only modify O(N+D) distinct files; it doesn't scale
very well.

We already pass multiple paths into svn_ra_lock so we could address part
of the network problem by rewriting some serf code to make it pipeline
the LOCK requests. That would have the advantage of working with older
servers but to solve all the problems we need to make HTTP more like the
svn protocol: send a single request (perhaps POST instead of LOCK?) for
the repository root and pass all the paths in the body of the request.

Once we have all the paths arriving at the server in one request we can
add new FS APIs to lock/unlock multiple paths, then sort the paths and
write each index file only once.

I'm not quite sure how hooks would behave. We would need to run all the
pre-lock hooks first and some could fail. We could drop the paths that
fail and pass the rest to the FS layer, or perhaps fail the whole
operation if any pre-lock fails. Either way the FS layer may fail to
lock some of the paths for various reasons (non-existant, already
locked, etc.) and so the final set of locks could be smaller than the
set of paths. Finally we run post-lock for the subset of paths that are
locked.

There is also an FS atomic issue to consider. The current single path
API can be interrupted between writing index files and handling the lock
file, but the result is the single path is locked or unlocked with any
"broken" index files being invisible to the user (athough the "broken"
index files may cause more work for the server). A multiple path API
could result in user visible changes to a subset of the paths. I think
that would be OK but it needs a bit more thought.

I've also noticed that we don't fsync any files when writing locks into
the repository. I'm not sure if this is deliberate or not but if we
were to start calling fsync then the filesystem issue would become more
important.

-- 
Philip Martin | Subversion Committer
WANdisco // *Non-Stop Data*