[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Streamy FS writes found detrimental.

From: Daniel Berlin <dan_at_dberlin.org>
Date: 2002-02-27 02:55:59 CET

On Tue, 26 Feb 2002, Greg Stein wrote:

> On Tue, Feb 26, 2002 at 08:36:12PM -0500, Daniel Berlin wrote:
> > >
> > > And if we (by default) prune the logs at each commit, then we're quite fine.
> >
> > We should prune the logs after every 10 megs of commits (there are only
> > two places we do database puts, so this is trivial. It's only new data and
> > replaces that store full text in the log).
> >
> > Just keep a counter somewhere or something.
> The real bulk is caused by 'strings', so the record keeping ought to be
> quite easy. Although, recall that we create/destroy DB environments all the
> time. That 10 meg could come across 100 different requests and DB open/close
> operations. Where's the counter go then? :-)
Well, we can cleanup at open and close time too, eliminating the problem
Besides, that's why i said "somewhere". I meant keep it
persistent by putting it "somewhere".
It doesn't really matter where.

The problem with doing it every commit is that it actually does a
little real work (looking for checkpoints in the logfiles, etc).
Then again, maybe i'm just used to imports where commits are very

Do we have any stats on how often a commit occurs in the subversion db?

> > > I'm not really worried about the log stuff.
> >
> > I'm worried of what it's symptomatic of, rather than the actual issue
> > itself.
> Right. I took a quick look through strings-table.c to see the kinds of
> things that would cause record rewrites. I think we have room for
> improvement, but I couldn't really trace it back far enough to find the
> various use cases and whether we could optimize some of it.
> > Although, i'm with Greg Hudson that it leaves a bad taste in my mouth
> > about BDB.
> > Really.
> BDB itself, or the fact that we're using it?
BDB itself.
Using it was a good choice, given the options available, which are
"something or nothing", basically.

> As has been said many a time, (IMO) using it was a good choice. While we
> might not have the same log issues, we would have a whole different class of
> problems. (and heck, if we wanted hot backup and stuff, we might even have
> similar log problems, but just with the homegrown stuff)
> That said, I *do* want to see us move to a more pluggable database backend.
> But that is a big bulk of work, destined for SVN 2.0. (although *gentle*
> patches to move the FS towards that pluggable design are welcome)
> > The least they could have done is provide a way to tell it that you only
> > want x amount of space ever taken up by logs over more than one
> > transaction (since you can't do anything about the log size for a
> > single transaction), and have bdb take care of it.
> >
> > Instead, they just shove the whole issue off to the user, and barely
> > mention it in the manual that it could become a problem.
> True, true.
> Cheers,
> -g

To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Sat Oct 21 14:37:10 2006

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.