[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: performance issue with large number of files

From: Garrett Rooney <rooneg_at_electricjellyfish.net>
Date: 2006-08-04 16:36:25 CEST

On 8/4/06, Mark Weaver <mark-clist@npsl.co.uk> wrote:
> I've got a directory with about 12k files totally about 1gb that is
> checked into svn. A svn ci in this folder is excruciatingly slow.
> Partly this is size (~2gb of read, presumably diffing the files), but
> mostly this is rewriting the entries file (about 3.2Mb written for each
> file diffed => ~40gb of writes). Client version is reported as svn,
> version 1.3.2 (r19776).
>
> Is there anything that can be done to mitigate this? e.g. get client to
> check on file modification time rather than diffing; and/or dealing with
> "entries" in a more scalable fashion? I note it doesn't read the
> entries file each time, is the write per file really needed?

Uhh, it does use file modification time, if the mtime hasn't changed
it shouldn't diff the file. Not sure if dealing with the entries file
can be improved or not, it's always possible though.

Honestly though, perhaps having a directory that contains 12k files in
it isn't the best idea in the world... Splitting it up would almost
certainly improve things.

-garrett

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Received on Fri Aug 4 16:38:02 2006

This is an archived mail posted to the Subversion Users mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.