On Fri, Jun 5, 2009 at 14:40, Geoff Rowell <geoff.rowell_at_varolii.com> wrote:
> I thought I’d share some difficulties that I encountered when getting a
> working copy from a repository with a large number of files.
>
>
>
> Background
>
> --------------
>
> The repository is running on a Windows 2003 unit. It’s using CollabNet
> Subversion Server 1.5.6 (Apache 2.2). It has a small number of revisions
> (~130).
>
> The repository contains a large number of small audio recordings. In several
> cases there are over 5000 audio files in a folder.
>
> My laptop is a Windows Vista system using CollabNet Subversion Client 1.6.2.
>
>
>
> Working copy details:
>
> - Size = 1.86 GB
>
> - Size on disk = 3.19 GB
>
> - # files = 480,000
>
> - # folders = 860
>
>
>
> Problem
>
> ----------
>
> I needed to get a working copy on my laptop in order to add/update some
> audio files. Since I knew this would be a performance challenge, I suspended
> the TSVNCache process and attempted the checkout from the command line.
> While it initially flew right along, eventually it slowed down to the point
> where every file was taking about 30 seconds. I could tell from the reported
> pathnames that the checkout was going to take days (weeks?).
>
>
>
> I decided to do some experiments to refine where the problem was.
>
>
>
> I have an Ubuntu Linux box running Subversion 1.5.5. I used that to get a
> working copy. It flew along at a steady rate and finished in 40 minutes. I
> archived the Linux working copy, transferred it to my laptop and unpacked
> it. If the Windows file system had been a bottleneck, the unpack would have
> shown a performance problem. It didn’t. It unpacked it at a steady rate in a
> couple of hours.
>
>
>
> I wanted to determine if the problem would occur when using a Subversion 1.5
> client under Windows, but I immediately ran into some problem masked by
> issue #3102. That prevented me from investigating this further.
>
>
>
> Luckily, I have the perfectly usable working copy, transferred from my Linux
> box, that can be used for modifications. My typical commits of a few files
> at a time don’t suffer performance problems.
>
>
>
> Thought I’d share this experience in hopes that something could be done to
> improve the performance.
There probably isn't much that can be done. NTFS and Windows do not
handle large numbers of files/directories in a single directory very
well, period. It's not so much a Subversion/library issue as it is
poor filesystem performance on that scale.
Did you disable your on-access virus scanner while doing this
checkout? That's usually a real killer.
------------------------------------------------------
http://subversion.tigris.org/ds/viewMessage.do?dsForumId=1065&dsMessageId=2359804
To unsubscribe from this discussion, e-mail: [users-unsubscribe_at_subversion.tigris.org].
Received on 2009-06-05 21:02:30 CEST