[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: [bug report] SVN design issue: checkouts horribly space-inefficient

From: Marcus Rohrmoser <mrohrmoser_at_gmx-gmbh.de>
Date: 2005-06-23 18:03:55 CEST

Is your concern related to that?
"In recent years, disk space has become outrageously cheap and abundant, but network bandwidth has
not. Therefore, the Subversion working copy has been optimized around the scarcer resource.

The .svn administrative directory serves the same purpose as the CVS directory, except that it also
stores read-only, “pristine” copies of your files."
http://svnbook.red-bean.com/en/1.1/svn-book.html#svn-ap-a-sect-3

        M

Marc Mutz schrieb:
> Hi,
>
> I just noticed while checking out some KDE SVN modules that my inodes were
> quickly depleted. After lots of searching I found that not my huge maildir
> archive is the culprit, it's the .svn directory with it's props/prop-base
> directories containing each one file for each object (file or dir) stored in
> the repository. On a typical ext2/3 filesystem, this leads to a waste of
> almost 98% of space:
>
> $ cd kdelibs:
> $ du -sh .svn/props
> 96K .svn/props
> $ wc -c .svn/props/* | grep total
> 1714 total
>
> This means, that - absent tail-end optimizations in the filesystem (which
> ext2/3 doesn't have) - the space efficiency is 1.75% (1714/96K).
>
> Same for prop-base.
>
> Now, the amount of data this represents is
> $ du -sch .svn/text-base/* | grep total
> 272K total
> $ wc -c .svn/text-base/*|grep total
> 224236 total
> -> 80.5% efficiency.
>
> Now, the bad thing is that props and prop-base are identical:
> $ for i in $(cd .svn/props/; echo *.svn-work ); do
> diff-u .svn/props/$i .svn/prop-base/${i/-work/-base}
> done
>
> So a lot of inodes could be spared by simply hard-linking the files in those
> dirs. brings up the efficiency to 3.5%. Same thing could (optionally) be done
> for text-base, on the assumption that most editors break that link during
> saving.
>
> That would halve the overall space inefficency.
>
> As it is now, svn uses about 4x more inodes for the same checkout as cvs. I
> expected 50% both in space and inodes, due to the offline-diff capability.
>
> In ten years of using Unix, I've never been _near_ the inode limit, though
> I've often been permanently in 90%+ disk-full mode.
>
> I hope you can apply these simple "optimizations" in the next release. They
> typically speed up diff's, too, see the difference between
> cp -ra old new-1
> cp -la old new-2
> time diff -ur old new-1
> time diff -ur old new-2
>
> Marc

Received on Thu Jun 23 18:35:11 2005

This is an archived mail posted to the Subversion Users mailing list.