[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Lots of files

From: Patrick Dean Rusk <PRusk_at_foliage.com>
Date: 2004-11-05 17:55:58 CET

"John Peacock" <jpeacock@rowman.com> wrote in message
news:418BA991.6010003@rowman.com...
> Patrick Dean Rusk wrote:
>
> > Some quick calculations with your numbers suggest that you have a
2048
> > byte block size on your disk, which means the small administrative files
> > maintained by both CVS and SVN are taking up much more space on disk
than
> > their internal bytes account for.
>
> Except that is not how NTFS operates. If the file contents are small
> enough, and the blocksize large enough, the contents of the file will be
> store in the MFT Directory Entry, not in a seperate file in the
> filesystem. The size of the MFT block is based on the blocksize of the
> underlying volume, so in cases of small files, it is actually _more_
> efficient to have a larger blocksize:
>
> http://www.cs.wpi.edu/~cs4513/b01/week2-windowsfs/week2-windowsfs.html

    You are right to point out the role of the MFT. However, the comment
about it being more efficient to have a larger blocksize if you have small
files really depends upon the average size of those small files. If they are
on average 900 byte files, I agree with you that you would be better off
with a 1024 block size (necessitating only 1 in MFT) than a 512 block
(necessitating possibly 3 -- 1 for MFT, 2 for file). However, if the files
average a very small size (like the many SVN admin files that are <200 bytes
on average), you're better off with a smaller block size. (Though,
apparently, the MFT block size is between 1K and 4K, not as small as 512
bytes.)

    So, I'll amend what I said earlier.

"Some quick calculations with your numbers suggest that you have an MFT
entry size of 2048 bytes on your disk, which means the small administrative
files maintained by both CVS and SVN are taking up much more space on disk
than their internal bytes account for. Since that's the minimum size Windows
will allocate to any file (including empty ones, I believe), either:
1) don't worry, be happy
2) suck it up and buy more disk space
3) delete a few MP3s."

    That said, this discussion does get me thinking about a possible
performance enhancement in combining some of those administrative files.
I'll raise it on the Developer's list.

    BTW, I checked one of my ".svn/dir-prop-base" files and it was 187 bytes
taking up 4096 bytes on disk. It would seem like the MFT entry size on my
laptop's disk might actually be 4K. I'm a bit surprised by that.

Pat

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Fri Nov 5 17:56:28 2004

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.