RE: concurrency and slowdowns
From: Dan Kearns <Dan.Kearns_at_Siebel.com>
Date: 2004-06-22 18:50:00 CEST
> > Is there any progress on fixing concurrency related
db_stat reports that the maximum number of locks ever held simultaneously is under 200. I suspect the advice on bumping up locks may be obsolete since it looks like the behavior of svn was changed at some point to require fewer. I have it set to 10,000 just in case though ;-)
This sounds very interesting, but also like it would need a couple releases to stabilize. Is the idea that one could put the repository in AFS and have apache as a front end running in multiple clustered or distributed servers?
I also wonder if just using "dirty read" isolation level for updates would help. I am still mostly ignorant of what is actually happening in the code, but it looks like the only mutable records which would ever be read are the node contents themselves and the flags which describe whether they are deltified, so if the contents and the flags are stored as one write by commit code, dirty reads could cut down on the locking required inside a transaction with no particularly bad side effects.
Just to add a data point here... I was using db4.1 (svn 1.0.4) since it was the gentoo default. I just switched to db4.2 (still 1.0.4 for svn), and though the problems are still there, both the performance and concurrency support are vastly improved. There are still deadlocks, but they don't seem to be taking whole client operations down with them now, so I'm cautiously optimistic...
With 4.1, strace on the processes which had slowed down to a crawl showed that basically every process was in some sort of backoff loop (a bunch of selects being used as timeouts with exponentially increasing times). This probably wasn't the svn code, it was probably bdb either waiting on locks or waiting on deadlock detection. If I had a wild guess, I'd say they changed their detection and waiting strategy in a way which subversion inadvertently benefitted from. So the bottom line is that it probably wasn't subversion's retry loop which was making the problem any worse.
This is an archived mail posted to the Subversion Users mailing list.