[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

RE: [TSVN] RFC: New cache scheme

From: Will Dean <svn_at_indcomp.co.uk>
Date: 2005-01-21 09:48:44 CET

At 09:01 21/01/2005 +0100, you wrote:

>Hi Will,
>
>This sounds *really* promising. Reading it makes me want to descale my C and
>start working on it :-)

I'm afraid it's in C++!

>...
> > 5. Keep track of the modification time of files which are
> > cached, and the
>I don't understand how this is going to work. How do you want to keep track
>of the times? Do you want to cycle through your cache entries slowly and
>compare the cached data to the disk data? Do you want to hook into
>windows-something so the cache gets notified if files / folders change?

No, it's currently in a 'pull' mode, where it records the timestamps of
things so that when the shell asks for the status of the item, it can check
if the value it's just found in the cache is useful.

There is a completely opposite way around which is to use the OS to tell us
when something changes, then update the cache and tell the shell that it's
changed. This seems like it would be better, but there are several
problems with this:

1. I don't think that the FindFirstChangeBlahBlah functions are suitable
for tracking a very large number of directories.
2. The shell itself uses FindFirstChangeBlahBlah to track items in
currently-displayed folders. (This is how it knows to ask for new status
when the text of a file changes.) If we were doing exactly the same
thing, there would be a race between us as to if the cache got updated
before the shell made its own request. This wouldn't really matter, except
that the shell has some kind of time-period after and overlay update during
which it ignores other update notifications.
3. There are NTFS change journal notification techniqes which might be
better than the FindFirstxxx functions, but they'd still suffer from the
problem in '2', and they won't work on non NTFS drives.

>You can stop the recursion at the point when you found the 'maximum'
>dominant SVN Status for a folder.

That's a good point - unfortunately, I think that the most dominant status
are things like 'conflicted' and 'obstructed', which are relatively
uncommon and are not really likely to save you much walking. But it's a
good idea.

>You are of course going to notice when you
>are recursing into a folder that is already cached (with its dominant status
>known), so you can stop there too.

Yes, if you can be confident that *it* has walked all its children!

>Entering a folder should also propagate the status up towards the wc root
>until it hits something 'stronger'

Yes, this is how it works.

> > 3. Issue shell-update requests for folders as their recursive status
> > becomes known.
>Neat :-)

Well, nearly, but of course the shell discards some notifications (because
it thinks they've been sent too close together, which is tedious.)

I am not far off hooking the system call GetTickCount before calling the
shell update function and lying to it about the time!

>Yep, I always switch it on if I demo TortoiseSVN and tell people that it's
>only usable for small projects, so they should switch it off :-|

I got it working here last night off the new cache, and it's
beautiful. It's just quite difficult to keep it up-to-date.

>A cached WC status tree can be 'entered' anytime at any place inside the
>tree. This can help us a lot in keeping information up to date.
>Just out of interest. How much memory does a cached file/folder status
>consume?

Its path length plus a few bytes. In fact, I'm still storing the whole
path of all directory members, which is completely unnecessary (and
slows-down the search), as they're all based on the same common root.

Even like this, I did a big walk around my 'client tree', which has a lot
of files in, with a fair bit versioned, and the shell process was only
using around 9-10Meg. I don't think memory's a big issue for this, though
we probably do need to have some kind of limiting system for people with
very large WCs.

>(x) Here!

Great, thanks.

Cheers,

Will

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tortoisesvn.tigris.org
For additional commands, e-mail: dev-help@tortoisesvn.tigris.org
Received on Fri Jan 21 09:49:42 2005

This is an archived mail posted to the TortoiseSVN Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.