[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: svn propget -R on the root of a large repository == boom!

From: Peter Wemm <peter_at_wemm.org>
Date: Mon, 21 Apr 2008 17:16:29 -0700

On Mon, Apr 21, 2008 at 4:11 PM, David Glasser <glasser_at_davidglasser.net> wrote:
> On Mon, Apr 21, 2008 at 4:06 PM, Peter Wemm <peter_at_wemm.org> wrote:
> > On Mon, Apr 21, 2008 at 1:37 PM, C. Michael Pilato <cmpilato_at_collab.net> wrote:
> > > Karl Fogel wrote:
> > >
> > > > Karl Fogel <kfogel_at_red-bean.com> writes:
> > > >
> > > > > "Peter Wemm" <peter_at_wemm.org> writes:
> > > > >
> > > > > > Suppose you have a large svn repository and people who like inviting
> > > > > > trouble. An example of 'large' in my case is the output (fsfs) of
> > > > > > cvs2svn of the freebsd.org src repository. There are only 180K
> > > > > > changes, but there are many thousands of branches and tags and copies.
> > > > > > The namespace is quite large and convoluted.
> > > > > >
> > > > > > Running 'svn propget -R svn:author file:///path/to/repo' is a good
> > > > > > way to find out how your machine handles running out of swap space...
> > > > > > In my case, the svn process had reached *19GB* before something broke.
> > > > > >
> > > > > [...patch...]
> > > > >
> > > >
> > > > Peter, I've committed that patch in r30743. It seemed right. But your
> > > > feedback is still important. If the patch doesn't solve the problem,
> > > > then either more needs to be done, or something different needs to be
> > > > done, or perhaps both.
> > > >
> > >
> > > Patch looks good here. You don't need a massive dataset to see that before
> > > the patch memory usage grows, and afterward it doesn't. I trust Peter's
> > > testing will confirm as much.
> >
> > Yes, it ran to completion with no drama at all. It still used a few
> > hundred megs of ram, but that's no big deal given what I asked it to
> > do.
>
> That's still ridiculously large. Does this patch reduce it further (and work)?
>
>
> Index: prop_commands.c
> ===================================================================
> --- prop_commands.c (revision 30744)
> +++ prop_commands.c (working copy)
> @@ -678,11 +678,13 @@
>
> svn_string_t *val = apr_hash_get(prop_hash, propname,
> APR_HASH_KEY_STRING);
> if (val)
> - val = svn_string_dup(val, perm_pool);
> + {
>
>
> + val = svn_string_dup(val, perm_pool);
>
> - apr_hash_set(props,
> - svn_path_join(target_prefix, target_relative, perm_pool),
> - APR_HASH_KEY_STRING, val);
> + apr_hash_set(props,
> + svn_path_join(target_prefix, target_relative, perm_pool),
> + APR_HASH_KEY_STRING, val);
> + }
> }
>
> if (depth >= svn_depth_files

Yes, svn's vsize is effectively static now for the entire 10 minute
run. vsize increased from 43M to 44M in the last 30 seconds. As
expected, it produces no output.

-- 
Peter Wemm - peter_at_wemm.org; peter_at_FreeBSD.org; peter_at_yahoo-inc.com
"All of this is for nothing if we don't go to the stars" - JMS/B5
"If Java had true garbage collection, most programs would delete
themselves upon execution." -- Robert Sewell
**WANTED TO BUY: Garmin Streetpilot 2650 or 2660. Not later model! **
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe_at_subversion.tigris.org
For additional commands, e-mail: dev-help_at_subversion.tigris.org
Received on 2008-04-22 07:15:52 CEST

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.