[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

issue #1429 progress: ra_dav changing analogously to ra_svn

From: <kfogel_at_collab.net>
Date: 2003-10-27 23:15:16 CET

Here's an update re recent progress on the performance front:

Recently there was a lot of discussion about how the two network RA
layers were doing too many turnarounds during common operations (e.g.,
checkouts, updates). The number of turnarounds was proportional to
the number of items in the checkout -- badness.

Greg Hudson solved this for ra_svn, by implementing pipelining in
r7532. But for ra_dav, true pipelining is not yet feasible, unless we
want to rewrite Neon. So instead, Ben and I added a new attribute to
our report request, telling the server to send back file contents with
its custom update report (which is used for checkouts as well as
updates). Formerly, the server only sent a "skeleton delta" (skelta),
telling the client what files needed to be fetched from the server.
The client would then issue a standard httpd GET request for each file
in turn.

After our change, there are no more GET requests, hence, no more
turnarounds after the full [non-skelta] delta starts to arrive. The
delta contains all the necessary data, and the client can process it
streamily to create the appropriate working copy. (The patch is
attached to issue #1429, by the way.)

This change results in an appx 300% for remote checkout, over a link
with average 81 ms latency -- the Chicago office's T1 line to
CollabNet HQ in California. Here are some timed checkouts of the
Subversion r7428 tree, over CVS, stock Subversion, and Subversion with
the patch:

       CVS SVN r7531 SVN r7531 + patch
    ---------- ---------- -----------------
    1m21.733s 4m46.052s 1m1.295s
    1m21.528s 4m21.998s 0m50.425s
    1m24.678s 4m12.330s 0m55.898s
    1m23.955s 3m52.263s 0m56.361s
    1m25.752s 4m38.493s 0m51.013s

This is know as a "win" :-).

Some caveats:

   1. Now we have an authorization problem. Because no GET requests
      are being issued, mod_authz_svn doesn't work, becuase the
      standard Apache authorization mechanism never kicks in. We will
      find a way to solve this (probably some sort of generic authz
      callback). Thoughts welcome.

   2. The above patch only sends file contents right now, not props.
      Things will slow down some when we add properties into the
      delta, but we don't expect it to go back to 4 minutes, even with
      translation. Especially not since r7502 (ghudson's translation

   3. On the other hand, even with the patch, the client is still
      using the Neon compatibility shim code, which gathers all the
      cdata in memory and then hands it to end_element() as a single
      string. This is both a scalability problem, and waste of time,
      and when we complete this patch we will undo the shim code and
      use a real (streamy) cdata handler.

So that's the status. Watch issue #1429 for progress.


To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Mon Oct 27 23:54:21 2003

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.