[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: [TSVN] Very long time to crawl WC.

From: Stefan Küng <tortoisesvn_at_gmail.com>
Date: 2005-11-24 18:05:04 CET

Peter McNab wrote:
> I have decided to collate and clarify all the symptoms and end result
> into one email.

Thanks, that helps a lot with keeping the info together.

> I have checked out a 115Mb WC which occupies 149 Mb of disk space.
> Win2K and r5015. (NTFS partition) dual core 3Ghz processor.
>
> The cache is crawling the resultant WC with very low CPU usage and
> taking a very long time. > 1/2 hour
> What may be contributing to the time taken is a batch of "Invalidating and
> refreshing folder" messages that follow the writing of each folder full
> of files. The
> process eventually comes to an end, unlike the problem just fixed or
> maybe partially fixed.
>
> A screenshot of the cache window is attached. It's a midstream sample,
> something I'm familiar with in another context. If you cannot replicate
> thus and need further data gathering I'm available.
>
> ----------- sent in reply re 100%CPU and clarified ----------
> Further observations are:-.
>
> The difference between yesterday's r5002 behavior and todays r5015
> behavior:-
> is that the crawling does eventually get through the WC.
>
> The problem is only occurring during checkout of a new WC, there is no
> problem crawling the new WC once it has allarrived or WCs that have been
> on the machine for months.

During checkout, Subversion repeatedly writes versioned files and also
writes the 'entries' file in the .svn folder. Both writes trigger the
file system watcher which then tells the crawler to crawl the
corresponding folder again because a write to those files means the
status might have changed.

So I guess I have to find a way to make the crawler delay the crawling
during checkouts until a whole folder has been checked out.
/me back to the drawing board to find a solution...

> ---------- New Info -----------
>
> On deleting a fresh checkout it looks like the cache isn't releasing as
> much memory as was acquired while crawling the new checkout.
> On a subsequent checkout without rebooting, cache memory use is
> initially stable then climbs past the total used during the previous
> checkout of the same repo.
>
> While deleting the big 149Mb WC a whole bunch of "Invalidating
> ...folder" messages were flowing too and cache memory used went up over
> 35Mb.

Seems the memory of folders which don't exist anymore isn't freed :(

> Finally the screen went blank and would only refresh where controls were
> responding to mouseover type events.
> Fortunately "Start" "Restart" were among them.

That would be the normal behavior of the cache, isn't it?

> So, for big WCs this looks like a real problem.
>
> Sorry, why is it always me?

Why you? *I* have to fix the bugs ;)

Stefan

-- 
        ___
   oo  // \\      "De Chelonian Mobile"
  (_,\/ \_/ \     TortoiseSVN
    \ \_/_\_/>    The coolest Interface to (Sub)Version Control
    /_/   \_\     http://tortoisesvn.tigris.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tortoisesvn.tigris.org
For additional commands, e-mail: dev-help@tortoisesvn.tigris.org
Received on Thu Nov 24 18:05:34 2005

This is an archived mail posted to the TortoiseSVN Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.