[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: ideas to make svn update faster.

From: Branko Čibej <brane_at_xbc.nu>
Date: 2005-05-07 20:47:29 CEST

Thomas Zander wrote:

>I'm a kde developer and as such lived through the cvs-svn conversion of the
>kde codebase. I even blogged a bit about that.. [1]
>One of the things I notice is that svn update is not faster then cvs update,
>which is contrary to expectations since there should be a global tree
>revision, so it should be faster then the cvs which has a revision per
I can't imagine how you reached that conclusion. How the revisions are
numbered has no bearing on update speed.

>I was told on #svn that this is due to the mixed revisions stuff.
Yup, and the fact that SVN has to do a lot more work than CVS, because
of directory versioning.

> [2] Which
>I fully understand. Looking at strace output I notice that svn could be a
>lot faster (do less writes) if svn was to be more optimistic about version
With "more optimistic" == "wrong", unfortunately...

>kdelibs has ~8800 files and 378 dirs. At any time maybe 10 files have a
>different version then the rest (hell; let it be 10%). That means that
>around 370 .svn/entries files have been written with the only change being
>a new version number in the name="" entry that is equal to just about all
>the other dirs in the project.
>A simple optimalisation would be to remove the directory-version number (the
>one in the xml entry-tag with 'name=""') when it has the same one as the
>parent dir.
Have you actually measured what percentage of update time it takes to
write those 378 entries files, or are you simply guessing that this is
the bottleneck?

>Its probably not goint to be as simple as that (since you update subdirs
>seperately), but I'm pretty sure that a lot less xml's have to be written
>if you follow the route that the normal state is a dir having the same
>version as its parent. Only when that fails do you need to do extra work.
>Being optimistic about version changes; I'd call that.
Well, the first question that pops to mind is, how do you tell that the
equal-version assumption is wrong, unless you record the dir's version

>Now; there is probably going to be a lot of opinions on the above subject;
>and I'd like to point out that svn really needs speed optimalisations; I
>have seen a LOT of complaints about this issue in the KDE switchover.
>Remember that if you find the above suggestion technically less-then-ideal.
Certainly SVN needs speed optimizations. But I think you're approaching
them exactly the wrong way around. The thing to do is to measure where
the bottlenecks are, and strace is far from enough for that.

>The strace also showed me things like;
>* the .svn/format file is opened 5 times for each directory.
We know about that, and we already have a (tentative) plan to remove the
format file and put the format information into the entries file.

>I would think
>that with auto-upgrades only one (the root dir) should be enough.
That, of course, is again an oversimplification. You can't make
assumptions about the state of subdirectories in the working copy.

> Saving
>5*378 -1 open-files for me. :)
>* .svn/lock files being created in every subdir is not needed if you check
>parent dirs that also have a .svn (and maybe the same root).
What you think of as the "root" of the working copy is a figment of the
imagination. It's quite valid to have two SVN processes fiddle in
parallel with two subtrees in the WC. A third SVN working from a common
root of those two subtrees could zap the WC if it didn't try to lock it
recursively first.

>So you create one in the dir you typed 'svn up' in and if someone types svn
>up in a subdir it will change dir to parent and check for a lock file until
>it either finds it (in this case it will, and abort) or it will leave the
>This will save a _lot_ of file-creation and removal afterwards.
So, you're saying that we should check locks upwards in the working
copy, not downwards. Interesting idea. I'd not want to guess what
happens if you have symlinked working copies.

-- Brane

To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Sat May 7 20:48:01 2005

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.