[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

[PATCH] BDB speedup (was: FSFS speedup)

From: Branko ─îibej <brane_at_xbc.nu>
Date: 2005-02-15 17:32:51 CET

Peter N. Lundblad wrote:

>OK. Here we go. On my tests it is a big speedup. I tested on FSFS with and
>without xdelta when the dump was loaded. It seems like we win a lot even
>if vdelta was used to create the repo. Anyway, feel free to fill in the
>missing row above:-)
>
>
Well, I've done something similar to the BDB combiner, except that it
only pre-expands windows that are truly self-compressed. At the moment,
this makes hardly any difference. However, it paves the way for storing
fulltexts as self-compressed vdeltas in BDB, too, just as FSFS does it.
This could significantly reduce the size of BDB repositories, especially
where there are many active branches.

[[[
Treat self-compressed delta windows as virutal fulltexts in the
delta combiner; expand them first instead of combining them with
the rest of the delta chain.

* subversion/libsvn_fs_base/reps-strings.c (compose_handler_baton):
   New member source_buf; holds expanded window data.
  (compose_handler): When handling a self-compressed window, expand
   it instead of combining it with the existing (combined) window.
  (rep_undeltify_range): If available, use the expanded window from
   the baton instead of the fulltext. Remove empty_buf.
]]]

-- Brane

Index: subversion/libsvn_fs_base/reps-strings.c
===================================================================
--- subversion/libsvn_fs_base/reps-strings.c (revision 13027)
+++ subversion/libsvn_fs_base/reps-strings.c (working copy)
@@ -150,6 +150,11 @@
   svn_txdelta_window_t *window;
   apr_pool_t *window_pool;
 
+ /* If the incoming window was self-compressed, and the combined WINDOW
+ exists from previous iterations, SOURCE_BUF will point to the
+ expanded self-compressed window. */
+ char *source_buf;
+
   /* The trail for this operation. WINDOW_POOL will be a child of
      TRAIL->pool. No allocations will be made from TRAIL->pool itself. */
   trail_t *trail;
@@ -178,18 +183,38 @@
   if (!cb->init && !window)
     return SVN_NO_ERROR;
 
+ /* We should never get here if we've already expanded a
+ self-compressed window. */
+ assert (!cb->source_buf);
+
   if (cb->window)
     {
- /* Combine the incoming window with whatever's in the baton. */
- apr_pool_t *composite_pool = svn_pool_create (cb->trail->pool);
- svn_txdelta_window_t *composite;
+ if (window && (window->sview_len == 0 || window->src_ops == 0))
+ {
+ /* This is a self-compressed window. Don't combine it with
+ the others, because the combiner may go quadratic. Instead,
+ explode it here and signal that the combination has
+ ended. */
+ apr_size_t source_len = window->tview_len;
+ assert (cb->window->sview_len == source_len);
+ cb->source_buf = apr_palloc (cb->window_pool, source_len);
+ svn_txdelta__apply_instructions (window, "",
+ cb->source_buf, &source_len);
+ cb->done = TRUE;
+ }
+ else
+ {
+ /* Combine the incoming window with whatever's in the baton. */
+ apr_pool_t *composite_pool = svn_pool_create (cb->trail->pool);
+ svn_txdelta_window_t *composite;
 
- composite = svn_txdelta__compose_windows (window, cb->window,
- composite_pool);
- svn_pool_destroy (cb->window_pool);
- cb->window = composite;
- cb->window_pool = composite_pool;
- cb->done = (composite->sview_len == 0 || composite->src_ops == 0);
+ composite = svn_txdelta__compose_windows (window, cb->window,
+ composite_pool);
+ svn_pool_destroy (cb->window_pool);
+ cb->window = composite;
+ cb->window_pool = composite_pool;
+ cb->done = (composite->sview_len == 0 || composite->src_ops == 0);
+ }
     }
   else if (window)
     {
@@ -320,8 +345,14 @@
 
       /* cb.window is the combined delta window. Read the source text
          into a buffer. */
- if (fulltext && cb.window->sview_len > 0 && cb.window->src_ops > 0)
+ if (cb.source_buf)
         {
+ /* The combiner already created the source text from a
+ self-compressed window. */
+ source_buf = cb.source_buf;
+ }
+ else if (fulltext && cb.window->sview_len > 0 && cb.window->src_ops > 0)
+ {
           apr_size_t source_len = cb.window->sview_len;
           source_buf = apr_palloc (cb.window_pool, source_len);
           SVN_ERR (svn_fs_bdb__string_read
@@ -331,8 +362,7 @@
         }
       else
         {
- static char empty_buf[] = "";
- source_buf = empty_buf; /* Won't read anything from here. */
+ source_buf = ""; /* Won't read anything from here. */
         }
 
       if (offset > 0)
@@ -1491,4 +1521,3 @@
 
   return SVN_NO_ERROR;
 }
-

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Tue Feb 15 17:32:06 2005

This is an archived mail posted to the Subversion Dev mailing list.