On Tue, 26 Feb 2002, Greg Stein wrote:
> On Tue, Feb 26, 2002 at 08:36:12PM -0500, Daniel Berlin wrote:
> > >
> > > And if we (by default) prune the logs at each commit, then we're quite fine.
> >
> > We should prune the logs after every 10 megs of commits (there are only
> > two places we do database puts, so this is trivial. It's only new data and
> > replaces that store full text in the log).
> >
> > Just keep a counter somewhere or something.
>
> The real bulk is caused by 'strings', so the record keeping ought to be
> quite easy. Although, recall that we create/destroy DB environments all the
> time. That 10 meg could come across 100 different requests and DB open/close
> operations. Where's the counter go then? :-)
Well, we can cleanup at open and close time too, eliminating the problem
entirely.
Besides, that's why i said "somewhere". I meant keep it
persistent by putting it "somewhere".
It doesn't really matter where.
The problem with doing it every commit is that it actually does a
little real work (looking for checkpoints in the logfiles, etc).
Then again, maybe i'm just used to imports where commits are very
frequent.
Do we have any stats on how often a commit occurs in the subversion db?
>
> > > I'm not really worried about the log stuff.
> >
> > I'm worried of what it's symptomatic of, rather than the actual issue
> > itself.
>
> Right. I took a quick look through strings-table.c to see the kinds of
> things that would cause record rewrites. I think we have room for
> improvement, but I couldn't really trace it back far enough to find the
> various use cases and whether we could optimize some of it.
>
> > Although, i'm with Greg Hudson that it leaves a bad taste in my mouth
> > about BDB.
> > Really.
>
> BDB itself, or the fact that we're using it?
BDB itself.
Using it was a good choice, given the options available, which are
"something or nothing", basically.
:)
>
> As has been said many a time, (IMO) using it was a good choice. While we
> might not have the same log issues, we would have a whole different class of
> problems. (and heck, if we wanted hot backup and stuff, we might even have
> similar log problems, but just with the homegrown stuff)
>
> That said, I *do* want to see us move to a more pluggable database backend.
> But that is a big bulk of work, destined for SVN 2.0. (although *gentle*
> patches to move the FS towards that pluggable design are welcome)
>
> > The least they could have done is provide a way to tell it that you only
> > want x amount of space ever taken up by logs over more than one
> > transaction (since you can't do anything about the log size for a
> > single transaction), and have bdb take care of it.
> >
> > Instead, they just shove the whole issue off to the user, and barely
> > mention it in the manual that it could become a problem.
>
> True, true.
>
> Cheers,
> -g
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Sat Oct 21 14:37:10 2006