Re: 1.6.11 tarballs up for testing/signing

On Fri, Apr 16, 2010 at 2:41 PM, Paul Burba <ptburba_at_gmail.com> wrote:
> On Fri, Apr 16, 2010 at 11:45 AM, C. Michael Pilato <cmpilato_at_collab.net> wrote:
>> Stefan Sperling wrote:
>>> On Fri, Apr 16, 2010 at 09:45:16AM -0400, Paul Burba wrote:
>>>> I just double checked this test, running with both a debug and release
>>>> build, both pass.
>>>>
>>>> You probably already noticed, but the dry-run and full-merge outputs
>>>> are semantically equivalent.  The only differences are the ordering of
>>>> the notifications and that the
>>>>
>>>>   "--- Merging r5 through r9 into
>>>> 'svn-test-work/working_copies/merge_tests-63.files':"
>>>>
>>>> notification appears twice.  This strikes me more as a limitation in
>>>> our test suite, rather than a problem with Subversion.
>>>
>>> Yes, it does not seem to be a serious problem.
>>> It's not release blocker.
>>> I do not compile serf support into the subversion binaries for OpenBSD,
>>> so in any case this test failure won't affect svn users on this platform.
>>>
>>> However, it's interesting that the test fails reliably for me with
>>> fsfs and serf, and passes reliably with bdb and serf.
>>> That's kind of odd. Maybe the problem is related to the fact that
>>> I build APR and Subversion without support for threads? I guess ordering
>>> of notifications with serf is timing dependent? In what way do fsfs
>>> and bdb differ in this respect? Should we try to hunt down the cause of
>>> this or just dismiss it as noise and see if it pops up again later?
>>
>> Actually, I'm really surprised by this result, and can't think of a good
>> explanation even with Serf's ordering ... characteristics.
>>
>> Paul, I thought the merge process always handled subtrees before their
>> parents.  And their two different editor drives altogether.  Is that not the
>> case?
>
> Hi Mike (you'll want to be sitting down for this),
>
> Not necessarily.  Multiple editor drives are dictated by what
> revisions the WC target, taken as a whole, need.  The mere presence of
> subtrees with explicit mergeinfo doesn't imply anything re the number
> of editor drives.  If the needed ranges are contiguous then there is
> only one editor drive, if not, then there are multiple editor drives.
> Hopefully the following examples will show what I mean (examples with
> trunk_at_934970):
>
> 1) Discrete Ranges --> Multiple Editor Drives
>
> We have a branch 'A_COPY' with some mergeinfo on the root and on a subtree:
>
>  >svn pl -vR A_COPY
>  Properties on 'A_COPY':
>    svn:mergeinfo
>      /A:5
>  Properties on 'A_COPY\D':
>    svn:mergeinfo
>      /A/D:3
>
> The revisions eligible to be merged to A_COPY are not contiguous:
>
>  >svn mergeinfo --show-revs eligible -R ^^/A A_COPY
>  r4
>  r6
>
> So when we do the merge there are two editor drives:
>
>  >svn merge ^^/A A_COPY
>
> XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
> XX
> XX  EDITOR DRIVE
> XX
> XX  Merge ^/A_at_2 ^/A_at_4
> XX
> XX  A_COPY/D already has r3, so it only needs r4
> XX  No need for a separate editor drive, we just use
> XX  our svn_ra_reporter3_t->set_path callback to let
> XX  the repos know this subtree is already at r3
> XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
>
>  --- Merging r4 into 'A_COPY\D':
>  U    A_COPY\D\G\rho
>
> XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
> XX
> XX  What about r5?
> XX
> XX  r5 would be inoperative if merged to A_COPY, as it
> XX  affects only A_COPY/B/E/beta and this was already
> XX  merged.  This is one difference between trunk and
> XX  1.6.x, as the former does away with inoperative
> XX  editor drives - see
> XX http://svn.haxx.se/dev/archive-2009-08/0045.shtml
> XX and r878767
> XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
>
> XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
> XX
> XX  EDITOR DRIVE
> XX
> XX  Merge ^/A_at_5 ^/A_at_7
> XX
> XX  No such set_path trickery needed here, r6 is the
> XX  the only operative rev and it affects
> XX  A_COPY/D/H/omega, so effectively the "whole" tree
> XX  rooted at A_COPY needs it.
> XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
>
>  --- Merging r6 through r7 into 'A_COPY\D':
>  U    A_COPY\D\H\omega
>
> XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
> XX  No more editor drives, the following is simply
> XX  merge tracking house-keeping.
> XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
>
>  --- Recording mergeinfo for merge of r2 through r7 into 'A_COPY':
>   U   A_COPY
>  --- Recording mergeinfo for merge of r2 through r7 into 'A_COPY\D':
>   U   A_COPY\D
>  --- Eliding mergeinfo from 'A_COPY\D':
>   U   A_COPY\D
>
> 2) One Contiguous Range --> One Editor Drive
>
> Again, we have a branch 'A_COPY' with some mergeinfo on the root and
> on a subtree:
>
>  >svn pl -vR
>  Properties on 'A_COPY':
>    svn:mergeinfo
>      /A:4
>  Properties on 'A_COPY\D\H\psi':
>    svn:mergeinfo
>      /A/D/H/psi:2-6
>
> But this time, the eligible ranges are contiguous:
>
>  >svn mergeinfo --show-revs eligible -R ^^/A A_COPY
>  r5*
>  r6*
>
> (The -R and '*' are a new 1.7 feature of the svn mergeinfo subcommand.
>  Without -R, it would examine only the mergeinfo on the target A_COPY
> and conclude that r3,r5,and r6 were needed.  With -R it realizes that
> r3 is already fully merged into the only subtree it affects
> A_COPY/D/H/psi, and that only r5 and r6 are needed and further that
> they are needed only by other subtrees - though in this case the
> "other" subtree is actually the remainder of the tree rooted at A_COPY
> less A_COPY\D\H\psi.)
>
>  >svn merge ^^/A A_COPY
>
> XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
> XX
> XX  EDITOR DRIVE
> XX
> XX  Merge ^/A_at_4 ^/A_at_7
> XX
> XX  A_COPY/D/H/psi already has r4:7, so it only needs
> XX  r7.  Again, no need for a separate editor drive, we
> XX  let the repos know this subtree is already at r6
> XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
>
>  --- Merging r5 through r7 into 'A_COPY':
>  U    A_COPY\D\H\omega
>  U    A_COPY\B\E\beta
>
> XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
> XX  Merge tracking house-keeping.
> XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
>
>  --- Recording mergeinfo for merge of r2 through r7 into 'A_COPY':
>   U   A_COPY
>
>
> SIDEBAR: I didn't show this, but there is an inoperative editor drive
> of ^/A_at_1 ^/A_at_2 at the start of both these examples (r2 was the copy of
> A to A_COPY).  Apparently the subtree-mergeinfo branch work is not
> picking up the fact that this is inoperative.  I'll look into that.

My mistake re the sidebar; this is working as expected.

The merge starts with the assumption "merge everything since the
branch was copied". This means a starting editor drive of:

  ^/A_at_1 ^/A_at_7 into A_COPY

Then it accounts for all the explicit mergeinfo on or under the
target, which recall is:

  Properties on 'A_COPY':
    svn:mergeinfo
      /A:5
  Properties on 'A_COPY\D':
    svn:mergeinfo
      /A/D:3

This means that A_COPY needs:

  r2-4,6-7

And A_COPY/D needs:

  r2,4-7

NOTE: The editor target is *always* A_COPY, we don't ever run editor
drives targeting A_COPY's subtrees. Rather, we use a combination of
set_path calls and multiple editor drives to work around any situation
where we can't simply merge a single range to the root. So when I say
"A_COPY/D needs...", I don't mean to imply that 'D' is the editor
target. Sorry if that has caused any confusion.

Anyhow, the subtree-mergeinfo logic notes that A_COPY/D needs r5, but
the root doesn't. For the reasons detailed here,
http://svn.apache.org/viewvc/subversion/branches/subtree-mergeinfo/notes/subtree-mergeinfo/the-performance-problem.txt?revision=878504&view=markup&pathrev=878767,
we want to remove inoperative ranges required by *subtrees*. In this
example that means r5 in A_COPY/D, which leaves these required revs*:

  A_COPY
    r2-4,6-7
  A_COPY/D
    r2,4,6-7

The first editor drive starts at the oldest starting revision that
needs to be merged to all subtrees (r2) and it ends with the youngest
remaining revision that needs to be merge to all subtrees (also r2):

  ^/A_at_1 ^/A_at_2 into A_COPY

That editor drive takes place and the remaining revs are now:

  A_COPY
    r3-4,6-7
  A_COPY/D
    r4,6-7

Following the same logic, the editor drives continue till the merge is done:

  ^/A_at_2 ^/A_at_4 into A_COPY
  ^/A_at_5 ^/A_at_7 into A_COPY

So two semi-obvious questions:

Q1) Since it is inoperative, why bother with the editor drive of ^/A_at_1
^/A_at_2 into A_COPY in the first place?

A1) We could use logic similar to that which removes inoperative
subtree ranges (see
libsvn_client/merge.c:remove_noop_subtree_ranges()) to also remove
inoperative ranges from the root. But determining what to remove
isn't free, it requires use of svn_ra_get_log2(). So it is probably a
wash in terms of performance.

Q2) Wait, r3 and r5 are both inoperative! Why not simply do one
editor drive of of ^/A_at_1 ^/A_at_7 into A_COPY?

A2) In this case that would make sense. Granted, the use case is a
bit unusual: "Do subtree merges that effectively merge all changes to
a branch, keeping it in sync with 'trunk'. Then decide to stop this
practice and merge all available revs to the root of the branch." We
could keep track of all the inoperative ranges we encounter in
remove_noop_subtree_ranges() and then plug them back into the
remaining ranges of every subtree if (cue hand-waving) that would
reduce the number of editor drives. I need to investigate this more
before I commit to anything, if it isn't obvious already, this can get
a bit complicated. I'll start a new thread when I figure out what, if
anything, we might want to do here.

Paul

* In this case removing r5 from A_COPY/D's required revisions doesn't
buy us anything, nor does it effect the number of editor drives. If
we didn't remove r5, the third editor drive would have started at a
different rev,

  ^/A_at_1 ^/A_at_2 into A_COPY
  ^/A_at_2 ^/A_at_4 into A_COPY
  ^/A_at_4 ^/A_at_7 into A_COPY

but since r5 is inoperative, there is no effect on the WC.
Received on 2010-04-19 18:32:52 CEST