Here's an update re recent progress on the performance front:
Recently there was a lot of discussion about how the two network RA
layers were doing too many turnarounds during common operations (e.g.,
checkouts, updates). The number of turnarounds was proportional to
the number of items in the checkout -- badness.
Greg Hudson solved this for ra_svn, by implementing pipelining in
r7532. But for ra_dav, true pipelining is not yet feasible, unless we
want to rewrite Neon. So instead, Ben and I added a new attribute to
our report request, telling the server to send back file contents with
its custom update report (which is used for checkouts as well as
updates). Formerly, the server only sent a "skeleton delta" (skelta),
telling the client what files needed to be fetched from the server.
The client would then issue a standard httpd GET request for each file
in turn.
After our change, there are no more GET requests, hence, no more
turnarounds after the full [non-skelta] delta starts to arrive. The
delta contains all the necessary data, and the client can process it
streamily to create the appropriate working copy. (The patch is
attached to issue #1429, by the way.)
This change results in an appx 300% for remote checkout, over a link
with average 81 ms latency -- the Chicago office's T1 line to
CollabNet HQ in California. Here are some timed checkouts of the
Subversion r7428 tree, over CVS, stock Subversion, and Subversion with
the patch:
CVS SVN r7531 SVN r7531 + patch
---------- ---------- -----------------
1m21.733s 4m46.052s 1m1.295s
1m21.528s 4m21.998s 0m50.425s
1m24.678s 4m12.330s 0m55.898s
1m23.955s 3m52.263s 0m56.361s
1m25.752s 4m38.493s 0m51.013s
This is know as a "win" :-).
Some caveats:
1. Now we have an authorization problem. Because no GET requests
are being issued, mod_authz_svn doesn't work, becuase the
standard Apache authorization mechanism never kicks in. We will
find a way to solve this (probably some sort of generic authz
callback). Thoughts welcome.
2. The above patch only sends file contents right now, not props.
Things will slow down some when we add properties into the
delta, but we don't expect it to go back to 4 minutes, even with
translation. Especially not since r7502 (ghudson's translation
speedups).
3. On the other hand, even with the patch, the client is still
using the Neon compatibility shim code, which gathers all the
cdata in memory and then hands it to end_element() as a single
string. This is both a scalability problem, and waste of time,
and when we complete this patch we will undo the shim code and
use a real (streamy) cdata handler.
So that's the status. Watch issue #1429 for progress.
-Karl
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Mon Oct 27 23:54:21 2003