Re: ideas to make svn update faster.

On Saturday 07 May 2005 23:28, Branko Čibej wrote:
> Thomas Zander wrote:
> Well, you /did/ say you didn't want to write the changed entries files
> if the "shape" of the directory didn't change. So the directory revision
> numbers in the entries file wouldn't be updated.
>
> If that's not what you meant, then I am now thorougly confused.

Let me quote myself;
> A simple optimalisation would be to remove the directory-version number
> (the one in the xml entry-tag with 'name=""') when it has the same one as
> the parent dir.
The effect would indeed be that you don't need the write the changed entries
if the shape of the dir didn't change.
Well; naturally you'd have to do it ones, to remove the version number, but
all successive updates won't need to anymore.

Hope that makes the idea clearer.
Do you still think this will create inconsistencies?

> >>>But, if you did the profiling part; I'd be happy to compare notes! :)
> >>
> >>Oh no -- that's your job, part of the task of convincing us you're
> >> right. :)
> >
> >Maybe you'd believe a co-developer better;
>
> This is not about whom I believe, but about you proving that your
> assertions are correct.

By quoting previous done work (by a co-developer, no less) saying the that
IO is indeed the bottleneck; didn't I just do that?
Your reasoning makes me afraid that the only prove you will believe are
tests that you ran yourself anyway.

> >Ahh; yeah; I think my solution will be full of race conditions, why
> > didn't I think of that when I wrote it.
> >Wait; its not written yet; so you can still make sure it doesn't have
> > any! Micro-bugfixing is not quite usefull at this stage; is it? :-)
>
> If you're impying that resolving race conditions is micro-bugfixing,
> then I strongly suggest you stop right there an do another think. I hope
> you're not implying that.

I'm not exactly new to programming/multithreading etc; with over 15 years of
experience and the last 6 years doing multithreaded IO and UI work (in
Java) for a living, I think I know how to avoid race conditions.
The global design phase isn't it (unless maybe with zero experience).

> >Ofcourse you need to first create an optimistic lock file and then you
> > start checking for subdirs / superdirs that have one.
>
> The result is that you actually have to traverse the whole subtree,
> doing "stat"s instead of "creat"s, *and* you'd have to walk upwards, too
> (which we currently don't do).

1) stats in the same dir as reads on the entries file you are going to be
reading anyway is optimised by the filesystem and gives you near zero
overhead.
Note that svn walks the tree now as well; but it actually opens the locks
for write instead of statting them. If you followed the filesystem classes
you'd know that the roundtrips goes from 1 to 3 (or even 5, depending on
filesytem being journalling or not).
2) Ignoring a subdir which is already locked in an update session brings
this back to 0.
3) traversing directories up instead of recursively traversing directories
down is by definition less work and tests may have to show if it fits in
the gain of now writing; but I'm pretty confident it will for all but the
most extreme cases.

> That's an interesting approach to
> optimisation. :)
If you can't take new ideas seriously, just tell me to shove it and I'll go
away, no problem.
I did expect a more mature approuch from you, though.

-- 
Thomas Zander