[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: [bug report] SVN design issue: checkouts horribly space-inefficient

From: <kfogel_at_collab.net>
Date: 2005-06-23 20:27:01 CEST

Marcus Rohrmoser <mrohrmoser@gmx-gmbh.de> writes:
> Is your concern related to that?
> "In recent years, disk space has become outrageously cheap and
> abundant, but network bandwidth has
> not. Therefore, the Subversion working copy has been optimized
> around the scarcer resource.
>
> The .svn administrative directory serves the same purpose as the CVS directory, except that it also
> stores read-only, “pristine” copies of your files."
> http://svnbook.red-bean.com/en/1.1/svn-book.html#svn-ap-a-sect-3

This bug isn't about disk space. It's about inodes -- indivisible
storage block allocation units. We're using them up too fast. You
might have a lot of space in your moving van, but a few chairs can
make it all unusable unless you arrange them right.

-Karl

> Marc Mutz schrieb:
> > Hi,
> >
> > I just noticed while checking out some KDE SVN modules that my inodes were
> > quickly depleted. After lots of searching I found that not my huge maildir
> > archive is the culprit, it's the .svn directory with it's props/prop-base
> > directories containing each one file for each object (file or dir) stored in
> > the repository. On a typical ext2/3 filesystem, this leads to a waste of
> > almost 98% of space:
> >
> > $ cd kdelibs:
> > $ du -sh .svn/props
> > 96K .svn/props
> > $ wc -c .svn/props/* | grep total
> > 1714 total
> >
> > This means, that - absent tail-end optimizations in the filesystem (which
> > ext2/3 doesn't have) - the space efficiency is 1.75% (1714/96K).
> >
> > Same for prop-base.
> >
> > Now, the amount of data this represents is
> > $ du -sch .svn/text-base/* | grep total
> > 272K total
> > $ wc -c .svn/text-base/*|grep total
> > 224236 total
> > -> 80.5% efficiency.
> >
> > Now, the bad thing is that props and prop-base are identical:
> > $ for i in $(cd .svn/props/; echo *.svn-work ); do
> > diff-u .svn/props/$i .svn/prop-base/${i/-work/-base}
> > done
> >
> > So a lot of inodes could be spared by simply hard-linking the files in those
> > dirs. brings up the efficiency to 3.5%. Same thing could (optionally) be done
> > for text-base, on the assumption that most editors break that link during
> > saving.
> >
> > That would halve the overall space inefficency.
> >
> > As it is now, svn uses about 4x more inodes for the same checkout as cvs. I
> > expected 50% both in space and inodes, due to the offline-diff capability.
> >
> > In ten years of using Unix, I've never been _near_ the inode limit, though
> > I've often been permanently in 90%+ disk-full mode.
> >
> > I hope you can apply these simple "optimizations" in the next release. They
> > typically speed up diff's, too, see the difference between
> > cp -ra old new-1
> > cp -la old new-2
> > time diff -ur old new-1
> > time diff -ur old new-2
> >
> > Marc

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Received on Thu Jun 23 21:19:07 2005

This is an archived mail posted to the Subversion Users mailing list.