On Fri, Oct 26, 2012 at 4:10 PM, Philip Martin
<philip.martin_at_wandisco.com> wrote:
> Johan Corveleyn <jcorvel_at_gmail.com> writes:
>
>> - This 10 second timeout, for a read operation that is blocked by some
>> write transaction: is this implemented somewhere in the svn source
>> code (if so, where?), or is this some configuration setting for the
>> sqlite connection or something like that?
>
> It's hard-coded, see BUSY_TIMEOUT in subversion/libsvn_subr/sqlite.c
Okay, thanks for the pointer.
>> - If a read operation (with a "shared lock" I presume) squeezes in
>> between write transactions of a large write operation, and that read
>> takes a long time, I suppose the next write transaction of that large
>> operation will also block, with the same timeout?
>
> Yes, I think so.
Okay, sounds logical.
>> - Does this also work for two writers? Or do they block out each other
>> with the working copy lock (wc_locks table)?
>
> The wc locks will exclude multiple writers providing all writers take a
> wc lock but I don't know whether all writers do that. I suppose we
> could add some code to check.
Okay. It would be good to at least review the code a bit for this. But
I admit I don't have the time to do this myself, so I'm not pushing
for anything here :-).
>> - With your remark about the workqueue making the read fail, is this
>> how a read operation may proceed?
>> * Try to get a shared sqlite lock for reading. Block for max. 10s if
>> the db is locked for writing (error out if timed out).
>> * When db reading is allowed (shared lock acquired), first read the
>> work_queue. If it's not empty: error out.
>> * Finally, read what we want to read, and get out.
>
> Yes.
Thanks for confirming.
It would be nice if the second limitation (non-empty work_queue) could
be lifted. Bert suggested that this would be possible in this post:
http://svn.haxx.se/dev/archive-2012-10/0234.shtml
If this limitation would be gone, it sounds like concurrency (one
writer, multiple readers) would work quite well, as long as individual
database operations don't take too long (as in "we try to avoid
unbounded large db transactions, e.g. by splitting things up per
directory if possible").
As I said in a previous post in this thread: it would be nice if we'd
have two clear modes of operation: (1) allow for maximum amount of
concurrency, at the cost of some performance, and (2) go for maximum
performance, but ignore concurrency (i.e. the new "exclusive mode").
I mean, this higher level distinction between two modes of operation
could go beyond merely choosing the locking mode of the database. It
could also for instance determine whether a particular query is done
with one big bulk query on the entire database, or a query that is
split per directory.
>> (and the above sequence may be part of a larger read operation that
>> e.g. does a query per directory, so this sequence may be repeated
>> several times, perhaps interleaved with some large write operation
>> that intermittently takes a write lock for things done per directory
>> or somesuch)
>>
>> - All things considered, the "old" non-exclusive mode sounds quite
>> non-deterministic. I mean, isn't this very hard for large GUI clients,
>> IDE plugins, ... to handle this?
>
> Probably, but I've not written any so I don't know.
Any GUI client / plugin devs / ... want to comment? TortoiseSVN,
SmartSVN, AnkhSVN, Subclipse, ...? How do you cope with the above
behavior? Do you do anything more besides just using a busy timeout
for the sqlite lock? What about the failure mode of running into a
non-empty work_queue (because of some write-in-progress) after
acquiring a lock?
If this is hard to handle in practice, and a long running
multi-threaded client has to wait anyway for an entire write operation
to be finished, I'm wondering if it would be much easier for such
clients to also always use the "exclusive locking" mode. It would make
most operations faster, and they have to wait anyway, so ... what
would be the added value of still using non-exclusive mode?
--
Johan
Received on 2012-10-31 11:27:37 CET