[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

[PATCH] Drastically reduce the number of opened ra_sessions during merge.

From: Lieven Govaerts <svnlgo_at_mobsol.be>
Date: Sun, 05 Oct 2008 14:51:39 +0200

While profiling the merge code I noticed that svn opens a huge number of
connections with a serious performance impact. Attached patch will
drstically reduce the number of sessions and connections opened during
merge.

Example of a simple merge of a few revs from trunk to 1.5.x_at_33458 with
svn 1.5.x_at_33458 over ra_neon:

$ time subversion/svn/svn merge -c --non-interactive
33201,33212,33369,33447,32968,32975,33447 $SVN/trunk ../1.5.x

--- Merging r33201 into '../1.5.x':
C ../1.5.x/subversion/tests/cmdline/copy_tests.py
U ../1.5.x/subversion/libsvn_wc/copy.c
--- Merging r33212 into '../1.5.x':
G ../1.5.x/subversion/tests/cmdline/copy_tests.py
Skipped missing target: '../1.5.x/subversion/libsvn_subr/dirent_uri.c'
--- Merging r32968 into '../1.5.x':
U ../1.5.x/subversion/tests/cmdline/merge_tests.py
--- Merging r32975 into '../1.5.x':
G ../1.5.x/subversion/tests/cmdline/merge_tests.py
C ../1.5.x/subversion/libsvn_client/merge.c

real 1m14.014s
user 0m2.980s
sys 0m1.664s

Nr. of sessions: 43
Nr. of connections: 61
Nr. of requests: 358

Svn 1.5 With attached patch:

$ time subversion/svn/svn merge -c --non-interactive
33201,33212,33369,33447,32968,32975,33447 $SVN/trunk ../1.5.x

real 1m5.987s
user 0m2.536s
sys 0m0.532s

Nr. of sessions: 3
Nr. of connections: 21
Nr. of requests: 296

The performance win is 12% over http, it'll be much higher for https
and/or NTLM/Kerberos authz.

Attached patch makes the merge code reuse ra_sessions - if possible - in
two places. Note: the ensure_ra_session_url function is only needed if
in do_merge the merge sources can be from different repositories, which
I doubt can be the case.

I'd like to get this patch reviewed by someone familiar with the merge
code before I commit.

Lieven

[[[
Reuse opened ra_sessions where possible during merge.

* subversion/libsvn_client/merge.c
  (ensure_ra_session_url): New function.
  (do_merge): When merging from multiple merge sources, reuse the open
   sessions when the sources are from the same repository.
  (populate_remaining_ranges): Reuse the ra_session passed into this
   function.
]]]

Index: subversion/libsvn_client/merge.c
===================================================================
--- subversion/libsvn_client/merge.c (revision 33458)
+++ subversion/libsvn_client/merge.c (working copy)
@@ -2969,7 +2969,8 @@ populate_remaining_ranges(apr_array_header_t *chil
       SVN_ERR(get_full_mergeinfo(&(child->pre_merge_mergeinfo),
                                  &(child->implicit_mergeinfo), child_entry,
                                  &(child->indirect_mergeinfo),
- svn_mergeinfo_inherited, NULL, child->path,
+ svn_mergeinfo_inherited, ra_session,
+ child->path,
                                  MAX(revision1, revision2),
                                  MIN(revision1, revision2),
                                  adm_access, merge_b->ctx, pool));
@@ -6085,7 +6086,38 @@ do_directory_merge(const char *url1,
   return err;
 }
 
+/** Ensure the caller receives a RA_SESSION object to URL. This function will
+ * reuse RA_SESSION if it is not NULL and is opened to the same repository as
+ * URL is pointing to. Otherwise a new session object will be created.
+ */
+static svn_error_t *
+ensure_ra_session_url(svn_ra_session_t **ra_session,
+ const char *url,
+ svn_client_ctx_t *ctx,
+ apr_pool_t *pool)
+{
+ svn_error_t *err = SVN_NO_ERROR;
 
+ if (*ra_session)
+ {
+ const char *old_session_url;
+ err = svn_client__ensure_ra_session_url(&old_session_url,
+ *ra_session,
+ url,
+ pool);
+ }
+
+ if ((err && err->apr_err == SVN_ERR_RA_ILLEGAL_URL) || ! *ra_session)
+ {
+ err = svn_client__open_ra_session_internal(ra_session, url,
+ NULL, NULL, NULL,
+ FALSE, TRUE, ctx, pool);
+ }
+ SVN_ERR(err);
+
+ return SVN_NO_ERROR;
+}
+
 /* Drive a merge of MERGE_SOURCES into working copy path TARGET (with
    associated TARGET_ENTRY and ADM_ACCESS baton).
 
@@ -6131,6 +6163,7 @@ do_merge(apr_array_header_t *merge_sources,
   const char *diff3_cmd;
   int i;
   svn_boolean_t checked_mergeinfo_capability = FALSE;
+ svn_ra_session_t *ra_session1 = NULL, *ra_session2 = NULL;
 
   /* Check from some special conditions when in record-only mode
      (which is a merge-tracking thing). */
@@ -6201,7 +6234,6 @@ do_merge(apr_array_header_t *merge_sources,
         APR_ARRAY_IDX(merge_sources, i, merge_source_t *);
       const char *url1, *url2;
       svn_revnum_t rev1, rev2;
- svn_ra_session_t *ra_session1, *ra_session2;
 
       svn_pool_clear(subpool);
 
@@ -6216,13 +6248,9 @@ do_merge(apr_array_header_t *merge_sources,
       if ((strcmp(url1, url2) == 0) && (rev1 == rev2))
         continue;
 
- /* Establish RA sessions to our URLs. */
- SVN_ERR(svn_client__open_ra_session_internal(&ra_session1, url1,
- NULL, NULL, NULL,
- FALSE, TRUE, ctx, subpool));
- SVN_ERR(svn_client__open_ra_session_internal(&ra_session2, url2,
- NULL, NULL, NULL,
- FALSE, TRUE, ctx, subpool));
+ /* Establish RA sessions to our URLs, reuse where possible. */
+ SVN_ERR(ensure_ra_session_url(&ra_session1, url1, ctx, pool));
+ SVN_ERR(ensure_ra_session_url(&ra_session2, url2, ctx, pool));
 
       /* Populate the portions of the merge context baton that need to
          be reset for each merge source iteration. */

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe_at_subversion.tigris.org
For additional commands, e-mail: dev-help_at_subversion.tigris.org
Received on 2008-10-05 17:44:38 CEST

This is an archived mail posted to the Subversion Dev mailing list.