Re: checkout performance

We run our server on a 2.8HGz Xeon processor running SuSE Enterprise
Linux 9, running apache2.0.49 and subversion 1.2.3.

Our authentication system uses mod_auth_kerb to refer to a Windows 2000
domain controller, and authz_svn to control access to various parts of
the repository. There seems to be a bottleneck with the interaction
between mod_auth_krb and mod_authz_svn, where the authentication
information is not a cached, but checked for EACH FILE in the repository
that a transaction looks at.

I found that something like 4 DNS lookups were also being performed
for each file, and fixed most of that by hardcoding IP addresses into
the krb.conf configuration file. This has reduced the server CPU load,
but checkouts and log operations are still much slower than we would
like, and something like 10x slower than using svn: protocol, with no
authz_svn (and no auth_kerb). We like mod_auth_kerb for the single sign
on ability it gives us, but the lack of credential caching is really
killing our performance. We have experimented with bdb and fsfs
backends, but found no definitive performance differences, so you might
be able to gain a little there, but not much.

In all likelihood, your problems are related to authentication and
authorisation - maybe mod_auth_ldap would be better for you?

Tony Butt

CEA Technologies
Canberra, Australia

Res Pons wrote:
> Juergen,
>
> To make you feel better: I serve Subversion on a super fast Linux
> Enterprise server with 4 Xeon processors, tons of disk space, and tons
> of RAM. All the users' desktop clients are Pentium4 @3-4GHZ with WinXP
> + 512MB to 1GB of RAM. And a complete fresh check out still takes
> 20-25 minutes for a 250MB project! Sometimes my patience just runs
> out. Not to mention it's extremely CPU intensive and takes my PC
> hostage. I literally cannot do anything until SVN is done checking
> out. NOW, consider yourself lucky :)
>
>
>
>
> ----Original Message Follows----
> From: Kjell H Andersen <kha-svn@lentic.net>
> To: Ryan Schmidt <subversion-2006Q1@ryandesign.com>
> CC: Juergen Richtsfeld <juergenr@segue.at>, users@subversion.tigris.org
> Subject: Re: checkout performance
> Date: Mon, 19 Dec 2005 15:08:27 +0100
> MIME-Version: 1.0
> Received: from tigris.org ([64.125.133.100]) by
> bay0-mc3-f3.bay0.hotmail.com with Microsoft SMTPSVC(6.0.3790.211);
> Mon, 19 Dec 2005 06:15:35 -0800
> Received: (qmail 13122 invoked by uid 5000); 19 Dec 2005 14:09:57 -0000
> Received: (qmail 12974 invoked from network); 19 Dec 2005 14:09:37 -0000
> X-Message-Info: JGTYoYF78jFSuACWoi/6zybCOlS8nnQQqeMXsbFvV/Y=
> Mailing-List: contact users-help@subversion.tigris.org; run by ezmlm
> Precedence: bulk
> X-No-Archive: yes
> list-help: <mailto:users-help@subversion.tigris.org>
> list-unsubscribe: <mailto:users-unsubscribe@subversion.tigris.org>
> list-post: <mailto:users@subversion.tigris.org>
> Delivered-To: mailing list users@subversion.tigris.org
> User-Agent: Thunderbird 1.5 (Windows/20051025)
> References: <3145C5555D21AD408CFA3A4BFAC77B976B0082@atmail1.segue.com>
> <593D3588-D352-4F3D-8770-65B98B3C2B40@ryandesign.com>
> Return-Path: users-return-42597-pons32=hotmail.com@subversion.tigris.org
> X-OriginalArrivalTime: 19 Dec 2005 14:15:36.0035 (UTC)
> FILETIME=[AE33CB30:01C604A6]
>
> I've had some experience with large check-out/in. The problem isn't
> the size itself, but the number of files.It looks like most file
> systems has a noticeable overhead for each file it has to create. I've
> had some problems related to this when checking out a working copy to
> a nfs mount. NFS performs really poor when trying to perform
> operations on many files.
>
> But to answer your question, I don't think 10-12 minutes are that much
> considering the number of files. Remember that all later operations
> will only be performed on a small amount of data. If you keep your
> working copy on a network drive, you can probably improve performance
> by moving it to your local drive, but that might lead to some backup
> issues..
>
> Kjell
>
>
> Ryan Schmidt wrote:
>> On Dec 19, 2005, at 13:18, Juergen Richtsfeld wrote:
>>
>>> I'm just about to try around with subversion. But it seems to be
>>> quite slow.
>>>
>>> my server has about 1GHZ, 1GB ram, windows 2000 server.
>>> my repository has about 350 MB in about 11000 files. my repository
>>> was created with default settings (svn 1.2) (bdb). my client is a P4
>>> 3Ghz with 1 gig ram.
>>>
>>> doing a clean checkout takes about 12 minutes with apache and about
>>> 10 minutes with svnserve (100 mbit lan).
>>>
>>> I'm currently testing with fsfs. are my measurements normal? i think
>>> it's quite slow. the problem is, we're currently using ms
>>> sourcesafe, which is much faster (but lacks lot of features).
>>
>> I don't know if 10 or 12 minutes is normal for a working copy of that
>> size. I have heard that checkouts on Windows take longer than on
>> other operating systems. But the important thing to remember is that
>> you only do a full checkout very rarely, usually just when you first
>> start working with Subversion. From then on you can just update the
>> existing working copy, which should be much faster.
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
>> For additional commands, e-mail: users-help@subversion.tigris.org
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>
> _________________________________________________________________
> FREE pop-up blocking with the new MSN Toolbar – get it now!
> http://toolbar.msn.click-url.com/go/onm00200415ave/direct/01/
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: users-help@subversion.tigris.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org