[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: [TSVN] RFC: New cache scheme

From: Will Dean <svn_at_indcomp.co.uk>
Date: 2005-01-21 13:37:08 CET

At 13:02 21/01/2005 +0100, you wrote:

>Do you mean you want to start the process each time you want to fetch
>a status? Sure you would first check if the process is running before
>trying to start it again. Or would you only try to start it if you
>don't get an answer?

If the process isn't running, you can't open the pipe (or if it was already
open, writes and reads to the pipe fail with an error).

Currently if the read or write fails, I just close the pipe.
Before any status fetch, I verify that the pipe was opened - if it isn't,
then I try and open it. So far, this doesn't require any timeouts.

What I was thinking of is that if this 'open' fails, we start the
process. We could either then poll for opening the pipe (which would need
a timeout), or just not bother, and require that the user F5's again, to
have another go at connecting.

>How big would be the timeout then?

It would have to be short, wouldn't it, but enough time to try and get the
cache process started.

>Or starting it only when the shell dll is loaded?

I was just expecting to start it when it was first asked for (though we
could *also* auto-run it at boot time, to reduce delays the first time an
explorer window is opened.

>What about having the status cache implemented as a windows service?

I suspect that the only advantage of having it as a windows service would
be that it could run when no-one was logged on. In exchange for this
(questionable) benefit, we would suddenly have a whole load of extra
complication. It would possibly run as a different user to the shell, which
might be a security issue I'd rather not deal with.

>The status info (if you store the whole svn_wc_status_t struct) is
>more than just 40 bytes. You have the filename in there, the URL it
>points to and UUID strings...

I currently just store svn_wc_status_t (I don't deep-copy 'entry'), which
is 32 bytes. This is enough to do icon overlays and some of the
column-provider columns, but not others.

Deep-copying 'entry' is a bore, and I would probably do something like
you've done in the shell extension, and just pick out the bits which are

As you say, this would start to make the per-file storage larger.

>But we can just assume that whoever has very big working copies also
>must have enough RAM available - I think that's reasonable.

That's my view, too. If I was seeing TSVNCache get up into the tens of
megs, I would be worried about memory. As it's nowhere near this, I'm not.

>I think I can agree with that.

Wow. I think that's the first thing we've agreed on all week. And it's Friday.

>Problem with a garbage collector is that while it is running, it has
>to be exclusively. I.e. all other requests to the cache have to wait
>until the collector is finished. So depending on how long it takes to
>take out the garbage you have a shell either not responding or showing
>the wrong overlays...

Well, not necessarily - for example, if the only garbage we collected was
the list of file data which is stored for each directory, then the garbage
collector could just lock one directory, check if it needed clearing, and
then unlock it. That would be a very short amount of time to hold the
lock, and would only affect someone who asked for that directory at that

>True. If it's fast enough with even recursive overlays turned on, then
>there's no reason to have an option to turn that off.

Well, I think it's fast enough to be useable on a local drive, but I'm sure
people will still want to turn it off!

>Since the cache has an IPC interface, a simple "cache status"
>dialog/property page could ask the cache process for specific
>information like memory use, number of hold files, ...

Yes, someone else suggested this. I'm torn as to whether it's easier to
add more U/I to (for example) TortoiseProc and IPC the data accross, or
whether it's better to add U/I to the cache itself.

Last night I thought the latter was better, but I'm being turned towards
the former...

>As you already mentioned, that can't be done unless we provide two
>different extension dlls - one for the overlays/columns and one for
>the property page.

But that wouldn't be a bad thing, really, would it?

>One problem we might encounter with this approach which I haven't
>mentioned yet:
>You can't fetch the status while another Subversion process is
>accessing the files (e.g. for an update/commit/...). This wasn't a big
>problem with the current approach, since once you start
>TortoiseProc.exe, the explorer already fetched the status and shows
>the overlays. It is however a problem if you have multiple instances
>of explorer open showing the same working copy folder and then you an
>update/commit/... - then sometimes even with the current approach the
>overlays get lost (showing "unversioned") because the working copy is

Right. Does svn_client_status return a specific, easily detectable error
in this situation?

>With a separate process fetching the status, maybe even in a separate
>thread it's much more likely to get into such a situation where you
>try to fetch the status of a locked working copy.

It's already doing this with TortoiseProc actually in 1.1.3, because I
flush the shell-update list after 5-second of nothing being added to it
(this was a mistake, really), and this creates a whole slew of
SHChangeNotifies, which trigger Explorer into asking for status.

I think I should get rid of the 5-second timeout (which was daft, making
the whole thing very complicated and requiring another thread), and just
flush the queue when TortoiseProc exits, (which happens anyway).

>Can you commit what you already have so far? I'd like to have a look
>at it and help out where I can...

I'll commit it into the WillDev branch. I'd welcome comments, but please
don't commit anything in there without talking to me first, because it
might well cause conflicts...

It's very rough....



To unsubscribe, e-mail: dev-unsubscribe@tortoisesvn.tigris.org
For additional commands, e-mail: dev-help@tortoisesvn.tigris.org
Received on Fri Jan 21 13:38:23 2005

This is an archived mail posted to the TortoiseSVN Dev mailing list.