[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Streamy FS writes found detrimental.

From: Daniel Berlin <dan_at_dberlin.org>
Date: 2002-02-26 20:52:13 CET

On Tue, 2002-02-26 at 13:23, cmpilato@collab.net wrote:
> Here are the contents of my ~/misc directory:
>
> 885124 Feb 7 09:57 somefile.jpg
> 4096 Jan 10 2001 CVS
> 124485 Oct 14 19:46 somefile.html
> 23376888 Dec 3 13:18 somefile.zip
> 87040 Sep 14 14:24 somefile.xls
> 647150 Feb 12 12:24 somefile.exe
>
> I created a brand new repository, turned on pool debugging output, and
> tried to import this directory (and the little CVS dongle attached to
> it). I wanted to prove to myself that we were actually doing better
> in the pool usage area by writing streamily to the FS. Here are my
> findings:
>
> - First I used the new streamy-filesystem-writing code, and the
> results were really disappointing. In fact, they are incomplete,
> because since we are doing streamy writes, we are hitting the
> database many, many, MANY (and did I mention "many"?) more times.
> That means more Berkeley DB log files. Lots more. Like, 1.74 gigs
> more before I ran out of disk space and the import aborted.
> Multiple times. Not a fluke here.
>
> One Point Seven Four Freakin' Gigabytes To Import A Thirty Meg Tree!!!
>

Up the log buffer size, and the cache size, or yer gonna get screwed.
Royally most likely.

I still have the patch to auto-remove logfiles in a thread in my tree,
so i don't have to worry about this stuff.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Sat Oct 21 14:37:10 2006

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.