[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: wc_db performance (was: wc_db API discussion)

From: Paul Burba <ptburba_at_gmail.com>
Date: Tue, 15 Mar 2011 18:22:05 -0400

On Mon, Mar 14, 2011 at 2:36 PM, Hyrum K Wright <hyrum_at_hyrumwright.org> wrote:
> On Sat, Mar 12, 2011 at 6:47 AM, Stefan Sperling <stsp_at_elego.de> wrote:
>> On Fri, Mar 11, 2011 at 10:43:46PM -0500, Greg Stein wrote:
>>> 2011/3/11 Branko Čibej <brane_at_e-reka.si>:
>>> >...
>>> > For the second task, I think the first order of business is to change
>>> > the wc-db tree crawler to do one query instead of zillions, or at least,
>>> > where several queries are required, to do them all in one transaction.
>>>
>>> stsp has been working this recently. Killing the node walker, and
>>> moving to table scans.
>>
>> Yes. So far, I've been working in the revision status code.
>> There are two problems left to fix before I'll move on to the next task:
>>
>>  - There are API layering issues (wc_db.c calls into node.c).
>>   This is related to the API discussions in the other thread
>>   so I'll follow up there.
>
> Before reading this thread, I saw the call into node.c, and have
> subsequently removed it.
>
>>  - The revision status code issues about 5 separate queries,
>>   which aren't combined via a transaction and don't use temporary tables.
>>   This is no worse than the previous code using the node walker,
>>   obviously :)  But I'll look at fixing this so that the results
>>   returned correspond to the state of the DB as of the time the
>>   svn_wc__db_revision_status() call was made.
>
> I wrapped this API in a txn in r1081510.
>
>> For others who want to jump in and help, here is a list of places
>> where the node walker is still being used. I'm not sure if we can
>> eliminate it everywhere before release, but each of these should
>> be looked at to see whether we can use an alternative approach to
>> increase performance:
>>
>>  subversion/libsvn_client/changelist.c
>>  subversion/libsvn_client/commit_util.c
>>  subversion/libsvn_client/info.c
>>  subversion/libsvn_client/merge.c
>>  subversion/libsvn_client/mergeinfo.c
>>  subversion/libsvn_client/prop_commands.c
>>   (This should be propget and propset. Proplist is already using
>>    queries involving temporary tables. Rewriting propget on top
>>    of the proplist code would be easy.

Is anyone working on propget? I'll take stab at that if not. It
would be useful for dealing with merge.c:merge_reintegrate_locked's
use of svn_wc__node_walk_children().

Paul

>> Propset needs more work.)
>>  subversion/libsvn_client/ra.c
>>  subversion/libsvn_wc/update_editor.c
>
> I'll take a gander at some of these, too.  (But I'm not entirely sure
> what I' gandering at or for...)
>
> -Hyrum
>
Received on 2011-03-15 23:37:24 CET

This is an archived mail posted to the Subversion Dev mailing list.