[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: ideas to make svn update faster.

From: Branko Čibej <brane_at_xbc.nu>
Date: 2005-05-07 20:47:29 CEST

Thomas Zander wrote:

>Hi;
>I'm a kde developer and as such lived through the cvs-svn conversion of the
>kde codebase. I even blogged a bit about that.. [1]
>
>One of the things I notice is that svn update is not faster then cvs update,
>which is contrary to expectations since there should be a global tree
>revision, so it should be faster then the cvs which has a revision per
>file.
>
>
I can't imagine how you reached that conclusion. How the revisions are
numbered has no bearing on update speed.

>I was told on #svn that this is due to the mixed revisions stuff.
>
Yup, and the fact that SVN has to do a lot more work than CVS, because
of directory versioning.

> [2] Which
>I fully understand. Looking at strace output I notice that svn could be a
>lot faster (do less writes) if svn was to be more optimistic about version
>numbers.
>
>
With "more optimistic" == "wrong", unfortunately...

>kdelibs has ~8800 files and 378 dirs. At any time maybe 10 files have a
>different version then the rest (hell; let it be 10%). That means that
>around 370 .svn/entries files have been written with the only change being
>a new version number in the name="" entry that is equal to just about all
>the other dirs in the project.
>A simple optimalisation would be to remove the directory-version number (the
>one in the xml entry-tag with 'name=""') when it has the same one as the
>parent dir.
>
>
Have you actually measured what percentage of update time it takes to
write those 378 entries files, or are you simply guessing that this is
the bottleneck?

>Its probably not goint to be as simple as that (since you update subdirs
>seperately), but I'm pretty sure that a lot less xml's have to be written
>if you follow the route that the normal state is a dir having the same
>version as its parent. Only when that fails do you need to do extra work.
>Being optimistic about version changes; I'd call that.
>
>
Well, the first question that pops to mind is, how do you tell that the
equal-version assumption is wrong, unless you record the dir's version
number?

>Now; there is probably going to be a lot of opinions on the above subject;
>and I'd like to point out that svn really needs speed optimalisations; I
>have seen a LOT of complaints about this issue in the KDE switchover.
>Remember that if you find the above suggestion technically less-then-ideal.
>
>
Certainly SVN needs speed optimizations. But I think you're approaching
them exactly the wrong way around. The thing to do is to measure where
the bottlenecks are, and strace is far from enough for that.

>The strace also showed me things like;
>* the .svn/format file is opened 5 times for each directory.
>
We know about that, and we already have a (tentative) plan to remove the
format file and put the format information into the entries file.

>I would think
>that with auto-upgrades only one (the root dir) should be enough.
>
That, of course, is again an oversimplification. You can't make
assumptions about the state of subdirectories in the working copy.

> Saving
>5*378 -1 open-files for me. :)
>
>
>* .svn/lock files being created in every subdir is not needed if you check
>parent dirs that also have a .svn (and maybe the same root).
>
>
What you think of as the "root" of the working copy is a figment of the
imagination. It's quite valid to have two SVN processes fiddle in
parallel with two subtrees in the WC. A third SVN working from a common
root of those two subtrees could zap the WC if it didn't try to lock it
recursively first.

>So you create one in the dir you typed 'svn up' in and if someone types svn
>up in a subdir it will change dir to parent and check for a lock file until
>it either finds it (in this case it will, and abort) or it will leave the
>checkout.
>This will save a _lot_ of file-creation and removal afterwards.
>
>
So, you're saying that we should check locks upwards in the working
copy, not downwards. Interesting idea. I'd not want to guess what
happens if you have symlinked working copies.

-- Brane

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Received on Sat May 7 20:48:01 2005

This is an archived mail posted to the Subversion Dev mailing list.