On Tue, Mar 15, 2011 at 6:22 PM, Paul Burba <ptburba_at_gmail.com> wrote:
> On Mon, Mar 14, 2011 at 2:36 PM, Hyrum K Wright <hyrum_at_hyrumwright.org> wrote:
>> On Sat, Mar 12, 2011 at 6:47 AM, Stefan Sperling <stsp_at_elego.de> wrote:
>>> On Fri, Mar 11, 2011 at 10:43:46PM -0500, Greg Stein wrote:
>>>> 2011/3/11 Branko Čibej <brane_at_e-reka.si>:
>>>> >...
>>>> > For the second task, I think the first order of business is to change
>>>> > the wc-db tree crawler to do one query instead of zillions, or at least,
>>>> > where several queries are required, to do them all in one transaction.
>>>>
>>>> stsp has been working this recently. Killing the node walker, and
>>>> moving to table scans.
>>>
>>> Yes. So far, I've been working in the revision status code.
>>> There are two problems left to fix before I'll move on to the next task:
>>>
>>> - There are API layering issues (wc_db.c calls into node.c).
>>> This is related to the API discussions in the other thread
>>> so I'll follow up there.
>>
>> Before reading this thread, I saw the call into node.c, and have
>> subsequently removed it.
>>
>>> - The revision status code issues about 5 separate queries,
>>> which aren't combined via a transaction and don't use temporary tables.
>>> This is no worse than the previous code using the node walker,
>>> obviously :) But I'll look at fixing this so that the results
>>> returned correspond to the state of the DB as of the time the
>>> svn_wc__db_revision_status() call was made.
>>
>> I wrapped this API in a txn in r1081510.
>>
>>> For others who want to jump in and help, here is a list of places
>>> where the node walker is still being used. I'm not sure if we can
>>> eliminate it everywhere before release, but each of these should
>>> be looked at to see whether we can use an alternative approach to
>>> increase performance:
>>>
>>> subversion/libsvn_client/changelist.c
>>> subversion/libsvn_client/commit_util.c
>>> subversion/libsvn_client/info.c
>>> subversion/libsvn_client/merge.c
>>> subversion/libsvn_client/mergeinfo.c
>>> subversion/libsvn_client/prop_commands.c
>>> (This should be propget and propset. Proplist is already using
>>> queries involving temporary tables. Rewriting propget on top
>>> of the proplist code would be easy.
>
> Is anyone working on propget? I'll take stab at that if not.
Done, see r1082658.
Not surprisingly, this makes a big difference in the performance of
svn propget -R. What is possibly a little(?) surprising is that it's
faster than 1.6 too:
Using a checkout of
https://svn.apache.org/repos/asf/subversion/branches@1082531 at depth
immediates and setting these branches to infinite depth, 1.0.x 1.1.x
1.2.x 1.3.x 1.4.x 1.5.x 1.6.x, gives us reasonably(?) large WC:
272 MB
14,044 files
1,428 Directories
svn pg svn:mergeinfo -Rv on the root
Version Elapsed Time Peak Working Set Memory
------- ------------ -----------------------
1.6.x_at_1082537 00:00:03.859 14,400 K
trunk_at_1082591 00:01:15.979 12,004 K
trunk_at_1082591 (PATCHED) 00:00:00.581 15,448 K
Paul
> It
> would be useful for dealing with merge.c:merge_reintegrate_locked's
> use of svn_wc__node_walk_children().
>
> Paul
>
>>> Propset needs more work.)
>>> subversion/libsvn_client/ra.c
>>> subversion/libsvn_wc/update_editor.c
>>
>> I'll take a gander at some of these, too. (But I'm not entirely sure
>> what I' gandering at or for...)
>>
>> -Hyrum
>>
>
Received on 2011-03-17 20:41:41 CET