[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: wc_db performance (was: wc_db API discussion)

From: Hyrum K Wright <hyrum_at_hyrumwright.org>
Date: Mon, 14 Mar 2011 13:36:58 -0500

On Sat, Mar 12, 2011 at 6:47 AM, Stefan Sperling <stsp_at_elego.de> wrote:
> On Fri, Mar 11, 2011 at 10:43:46PM -0500, Greg Stein wrote:
>> 2011/3/11 Branko Čibej <brane_at_e-reka.si>:
>> >...
>> > For the second task, I think the first order of business is to change
>> > the wc-db tree crawler to do one query instead of zillions, or at least,
>> > where several queries are required, to do them all in one transaction.
>>
>> stsp has been working this recently. Killing the node walker, and
>> moving to table scans.
>
> Yes. So far, I've been working in the revision status code.
> There are two problems left to fix before I'll move on to the next task:
>
>  - There are API layering issues (wc_db.c calls into node.c).
>   This is related to the API discussions in the other thread
>   so I'll follow up there.

Before reading this thread, I saw the call into node.c, and have
subsequently removed it.

>  - The revision status code issues about 5 separate queries,
>   which aren't combined via a transaction and don't use temporary tables.
>   This is no worse than the previous code using the node walker,
>   obviously :)  But I'll look at fixing this so that the results
>   returned correspond to the state of the DB as of the time the
>   svn_wc__db_revision_status() call was made.

I wrapped this API in a txn in r1081510.

> For others who want to jump in and help, here is a list of places
> where the node walker is still being used. I'm not sure if we can
> eliminate it everywhere before release, but each of these should
> be looked at to see whether we can use an alternative approach to
> increase performance:
>
>  subversion/libsvn_client/changelist.c
>  subversion/libsvn_client/commit_util.c
>  subversion/libsvn_client/info.c
>  subversion/libsvn_client/merge.c
>  subversion/libsvn_client/mergeinfo.c
>  subversion/libsvn_client/prop_commands.c
>   (This should be propget and propset. Proplist is already using
>    queries involving temporary tables. Rewriting propget on top
>    of the proplist code would be easy. Propset needs more work.)
>  subversion/libsvn_client/ra.c
>  subversion/libsvn_wc/update_editor.c

I'll take a gander at some of these, too. (But I'm not entirely sure
what I' gandering at or for...)

-Hyrum
Received on 2011-03-14 19:48:38 CET

This is an archived mail posted to the Subversion Dev mailing list.

This site is subject to the Apache Privacy Policy and the Apache Public Forum Archive Policy.