[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Poor performance checking out working copy with many files (Windows SVN 1.6.2)

From: Parashkev Penev <parashkev.penev_at_mimoon.de>
Date: Mon, 8 Jun 2009 09:39:02 +0200

Hello,

There is already a ticket filed for your trouble together with a proof-
of-concept patch. I do no remember the exact number, but search for
"subversion file locking" and you should find it easily.

Br

Paro

On 05.06.2009, at 21:07, Geoff Rowell wrote:

> On Fri, Jun 5, 2009 at 15:02, Andy Levy <andy.levy_at_gmail.com> wrote:
>> On Fri, Jun 5, 2009 at 14:40, Geoff Rowell <geoff.rowell_at_varolii.com>
> wrote:
>>> I thought I'd share some difficulties that I encountered when
> getting a
>>> working copy from a repository with a large number of files.
>>>
>>>
>>>
>>> Background
>>>
>>> --------------
>>>
>>> The repository is running on a Windows 2003 unit. It's using
> CollabNet
>>> Subversion Server 1.5.6 (Apache 2.2). It has a small number of
> revisions
>>> (~130).
>>>
>>> The repository contains a large number of small audio recordings. In
> several
>>> cases there are over 5000 audio files in a folder.
>>>
>>> My laptop is a Windows Vista system using CollabNet Subversion
> Client 1.6.2.
>>>
>>>
>>>
>>> Working copy details:
>>>
>>> - Size = 1.86 GB
>>>
>>> - Size on disk = 3.19 GB
>>>
>>> - # files = 480,000
>>>
>>> - # folders = 860
>>>
>>>
>>>
>>> Problem
>>>
>>> ----------
>>>
>>> I needed to get a working copy on my laptop in order to add/update
> some
>>> audio files. Since I knew this would be a performance challenge, I
> suspended
>>> the TSVNCache process and attempted the checkout from the command
> line.
>>> While it initially flew right along, eventually it slowed down to
> the point
>>> where every file was taking about 30 seconds. I could tell from the
> reported
>>> pathnames that the checkout was going to take days (weeks?).
>>>
>>>
>>>
>>> I decided to do some experiments to refine where the problem was.
>>>
>>>
>>>
>>> I have an Ubuntu Linux box running Subversion 1.5.5. I used that to
> get a
>>> working copy. It flew along at a steady rate and finished in 40
> minutes. I
>>> archived the Linux working copy, transferred it to my laptop and
> unpacked
>>> it. If the Windows file system had been a bottleneck, the unpack
> would have
>>> shown a performance problem. It didn't. It unpacked it at a steady
> rate in a
>>> couple of hours.
>>>
>>>
>>>
>>> I wanted to determine if the problem would occur when using a
> Subversion 1.5
>>> client under Windows, but I immediately ran into some problem masked
> by
>>> issue #3102. That prevented me from investigating this further.
>>>
>>>
>>>
>>> Luckily, I have the perfectly usable working copy, transferred from
> my Linux
>>> box, that can be used for modifications. My typical commits of a few
> files
>>> at a time don't suffer performance problems.
>>>
>>>
>>>
>>> Thought I'd share this experience in hopes that something could be
> done to
>>> improve the performance.
>>
>> There probably isn't much that can be done. NTFS and Windows do not
>> handle large numbers of files/directories in a single directory very
>> well, period. It's not so much a Subversion/library issue as it is
>> poor filesystem performance on that scale.
>>
>> Did you disable your on-access virus scanner while doing this
>> checkout? That's usually a real killer.
>>
>
> That was my first thought, but when the unpack (7-Zip) of the working
> copy
> archive didn't experience this problem I considered it to be unlikely.
>
> ---
> Geoff Rowell | SET Architect & SCC Administrator
> Varolii Corporation | 781.482.2125 | FAX 781.482.2199 |
> geoff.rowell_at_varolii.com
>
>
>
> This email and any files transmitted with it are confidential and
> intended solely for the use of the individual or entity to whom they
> are addressed. If you have received this email in error please
> notify the sender.
>
> ------------------------------------------------------
> http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=2359808
>
> To unsubscribe from this discussion, e-mail: [users-unsubscribe_at_subversion.tigris.org
> ].

------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=2360212

To unsubscribe from this discussion, e-mail: [users-unsubscribe_at_subversion.tigris.org].
Received on 2009-06-08 09:44:37 CEST

This is an archived mail posted to the Subversion Users mailing list.