[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: [Issue 4176] wcng slow on network disks

From: Mark Phippard <markphip_at_gmail.com>
Date: Mon, 25 Jun 2012 11:43:51 -0400

On Mon, Jun 25, 2012 at 11:34 AM, Bert Huijben <bert_at_qqmail.nl> wrote:

>> markphip_at_tigris.org writes:
>>
>> > http://subversion.tigris.org/issues/show_bug.cgi?id=4176
>>
>> > I think we need to find a way to include this patch.  I would suggest
>> > a new runtime configuration option or an environment variable.
>> > Something like "Allow concurrent client access".
>>
>> There are problems with the patch.  One is that it is incompatible with
>> the legacy access baton API, so an application using that API will need
>> to have exclusive locking disabled.  Another problem is the current
>> implementaion results long-lived SQLite handles so programs using the
>> newer API run into problems if with multiple/long-lived contexts.
>>
>> I think the solution is to default to non-exclusive locking and to allow
>> an application to ask for exclusive locking.  Then the command line
>> client can be patched to ask for exclusive locking.  And we provide a
>> config option that allows the user to prevent the command line client
>> for asking for exclusive locking so that anyone who relies on the
>> existing 1.7 behaviour can still get it.
>
> I don't think this should be the default for any client.
>
> If we make it the default for 'svn' we can just start calling it Subversion
> 2.0 as it breaks all the existing multi client behavior.
>
> I don't see a problem with adding a '--exclusive' option or something (or
> adding it to the config usable via --config-option), or whatever but I don't
> think we should make it default on any platform.

No one has suggested it be the default. Philip said the opposite. I
thought he was just suggesting we ought to create a way via our API
for a client to ask for the behavior. We can then decide on what
optional way to let the svn client ask for it makes the most sense,
such as an --exclusive option or runtime config etc.

> Making it default in our testsuite is just as bad as just creating a much
> simpler exclusive working copy format. Why bother with using a sql backend
> if there can be only a single user?
>
> The problem is/was that Subversion is slow on NFS.
>
> As far as I can tell we never intended it to be fast on that filesystem. The
> locks make us slower, especially on NFS and in some ways Unix, while on
> Windows we always had locks. (The unix folks just didn't care. Just like
> Apache Httpd currently doesn't care about some really slow code on Windows
> right now).
>
> My guess is that the reactions today are triggered by the slow query problem
> on users_at_s.a.o.
>
> This query had a ?2 = '' check which made us perform a full table scan
> before I fixed all those cases for 1.8.
>
> So making this call exclusive as suggested by issue #4176 would still make
> us read the entire 1.5 GB database over NFS (which I assume is mostly filled
> by NODES), while the proper query fix just makes us read the index and a
> single row.
>
> Assuming 100 Mbit the delete is probably back to something like 15-30
> minutes by making it exclusive and to something like 1-3 seconds (but could
> be a fraction of a second) with the query fix and a recent Sqlite.
>
> The exclusive fix might optimize it a bit further, but at a considerable
> compatibility cost. The million to one fix is in using the index.

Your comments here are kind of hard to follow. Obviously it makes
sense to optimize our SQL usage as much as possible to make the client
as fast as possible. When I talked to you in Berlin about this you
said that even with all of the changes you have made, trunk is still
much slower on NFS than 1.6. But if a client could use this
--exclusive option it becomes faster or at least the same as 1.6. I
am just saying that we should punch through an option so that users in
this situation have an option they can turn to. But given that this
will never be the default behavior, improving the SQL usage is still
the most worthwhile option. Are you concerned that if we add this
sort of a "bail out" option that people will stop trying to improve
the SQL? I agree that would be bad.

-- 
Thanks
Mark Phippard
http://markphip.blogspot.com/
Received on 2012-06-25 17:44:23 CEST

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.