[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Combining administrative files (was Re: Lots of files)

From: Patrick Dean Rusk <PRusk_at_foliage.com>
Date: 2004-11-05 18:16:43 CET

"Patrick Dean Rusk" <PRusk@foliage.com> wrote...
> That said, this discussion does get me thinking about a possible
> performance enhancement in combining some of those administrative files.
> I'll raise it on the Developer's list.

    Ha! I didn't realize this trail was already on the Developer's list.

    This thought may have already been thoroughly discussed before, so
someone can just say so if that's true.

    The "Lots of Files" thread pointed out that the many administrative
files in .svn directories are quite small, yet they take up lots of space on
disk because of minimum file sizes (due to block sizes and, on Windows, MFT
entry sizes). I assume that some number of those files are accessed every
time an "svn up" is performed (the operation that I would consider the most
important to tune performance for).

    Suppose a number of those files are combined into one file. For
instance, if
        dir-prop-base
        dir-props
        dir-wcprops
        empty-file
        format
were all combined into one file (analogous to a .ZIP file without
compression), that file would often fit into one block, presumably requiring
only one directory lookup and one I/O operation to read in. The savings in
I/O operations (particularly physical I/O when they are not already cached)
would very well outweigh the additional parsing required of the file and the
extra number of times the file needs to be written when its component pieces
change.

    Even the files in prop-base, props, and wcprops might benefit from
concatenation, though that would need to be approached gingerly given that
there is 1 per file under version control, so it could have scalability
issues.

    The potential performance benefits might only apply to "raw" svn updates
being done for the first time since a reboot, where none of the files are
yet cached in memory.

Patrick Rusk

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Fri Nov 5 18:17:19 2004

This is an archived mail posted to the Subversion Dev mailing list.