[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Streamy FS writes found detrimental.

From: Greg Stein <gstein_at_lyra.org>
Date: 2002-02-27 02:50:26 CET

On Tue, Feb 26, 2002 at 08:36:12PM -0500, Daniel Berlin wrote:
> >
> > And if we (by default) prune the logs at each commit, then we're quite fine.
> We should prune the logs after every 10 megs of commits (there are only
> two places we do database puts, so this is trivial. It's only new data and
> replaces that store full text in the log).
> Just keep a counter somewhere or something.

The real bulk is caused by 'strings', so the record keeping ought to be
quite easy. Although, recall that we create/destroy DB environments all the
time. That 10 meg could come across 100 different requests and DB open/close
operations. Where's the counter go then? :-)

> > I'm not really worried about the log stuff.
> I'm worried of what it's symptomatic of, rather than the actual issue
> itself.

Right. I took a quick look through strings-table.c to see the kinds of
things that would cause record rewrites. I think we have room for
improvement, but I couldn't really trace it back far enough to find the
various use cases and whether we could optimize some of it.

> Although, i'm with Greg Hudson that it leaves a bad taste in my mouth
> about BDB.
> Really.

BDB itself, or the fact that we're using it?

As has been said many a time, (IMO) using it was a good choice. While we
might not have the same log issues, we would have a whole different class of
problems. (and heck, if we wanted hot backup and stuff, we might even have
similar log problems, but just with the homegrown stuff)

That said, I *do* want to see us move to a more pluggable database backend.
But that is a big bulk of work, destined for SVN 2.0. (although *gentle*
patches to move the FS towards that pluggable design are welcome)

> The least they could have done is provide a way to tell it that you only
> want x amount of space ever taken up by logs over more than one
> transaction (since you can't do anything about the log size for a
> single transaction), and have bdb take care of it.
> Instead, they just shove the whole issue off to the user, and barely
> mention it in the manual that it could become a problem.

True, true.


Greg Stein, http://www.lyra.org/
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Sat Oct 21 14:37:10 2006

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.