[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Transactionless report processing

From: Greg Hudson <ghudson_at_MIT.EDU>
Date: 2003-12-30 01:27:05 CET

Are you tired of all this "stabilization?" Do you long for the
halcyon days of thousand-line code bombs which rearchitect whole
swaths of the system? If so, issue #1679 is for you!
http://subversion.tigris.org/issues/show_bug.cgi?id=1679 contains a
working copy of my new transactionless reporter implementation. I
will present the remainder of this message in the form of a FAQ,
'cause it's more leet that way. (Nevermind that some of these
"frequently" asked questions have never, in fact, been asked before.)

Q: What does it do?

A: Right now, when you perform an update, switch, status, or diff
operation against the repository, the repository builds one or two
transactions in the FS database to reflect the source and possibly the
desired target state of your working copy. This new implementation
processes those operations without writing the FS database at all.

Q: How does it do that?

A: It writes the report out to a temporary file; then, when the report
is finished, it reads report entries back out, using them to virtually
modify the source and target revisions. Because reports arrive in
depth-first order, we don't need random access to them as long as we
make sure to recurse into reported directories before recursing into
anything else.

Q: What's the point?

A: It would be nice to be able to perform read-only Subversion
operations on a read-only database. Unfortunately, it appears that
locking prevents us from doing that with a repository someone else
might be writing to, but perhaps it might be possible to arrange it
for an unchanging repository.

Also, it should be possible to make a server scale better if it
doesn't have to make synchronous writes to the repository for
read-only operation. With the new code, if you put /tmp on an MFS and
your repository fits into buffer cache, you should be able to achieve
almost zero disk activity even in the face of a high load of checkouts
and updates. I plan to make a test harness and run some performance
tests over the next few days to see how much of a benefit this is.

Q: But our API is frozen, and you couldn't possibly be using
svn_repos_dir_delta as it currently stands. So what did you do?

A: Ah, very perceptive. The new reporter.c contains a heavily
modified copy of delta.c.

Q: Wait, did you just duplicate a thousand lines of code?

A: Yes... but with the reporter no longer using svn_repos_dir_delta(),
there is only one call to it left in the source tree, in mod_dav_svn,
and that could easily be modified to use the reporter. So
svn_repos_dir_delta() should be able to wither and die, although we
have to continue to provide it until svn 2.0.

Also, I got rid of five hundred lines of code in the process (the old
reporter logic for creating transactions), so it's not so bad.

Q: svn_repos_dir_delta was a pretty hairy function. Isn't it even
hairier if you have to figure in the complexity of processing a
report?

A: Yes... but I took pains to make it as simple and non-redundant as
possible. If you take a look at the code, let me know if it's

Q: Okay, I'm convinced by your suave and enthusiastic marketing. How
do I try it out?

A: All the new code lives into reporter.c, but because I hate you all,
I made the code depend on the patches in issue #999 and issue #1677.
So, fetch those patches and apply them first, then sub in reporter.c.

Q: update_tests 15 is failing with the new code. Isn't it funny that
ghudson code is failing on a test for a ghudson bug?

A: Ha freakin' ha. The test failure is cosmetic; because the new code
opens and closes the root where the old code didn't, the output from
"svn update" is slightly different. For reasons I haven't yet
determined, if I modify the test case, "./update_tests.py 15" passes
but "make check" still fails with the same error, which is really
bizarre.

Q: This is pretty funny! You should quit your day job and write FAQs
for a living!

A: Okay, that wasn't even a question. I'm going away now.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Tue Dec 30 01:27:33 2003

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.