Finalizing what's in 1.5 (was: Getting closer to a 1.5 release?)

Karl Fogel <kfogel_at_red-bean.com> writes:
> ...there are some big decisions we haven't made yet, about exactly
> what goes into 1.5. That's going to need a separate thread (sorry),
> and I'll post about it right after this. We really need to nail
> down what's going in, whether we're shipping with an SQLite
> dependency or not, etc.

Okay, we just had a long IRC conversation about this (see transcript
below). To summarize: we need to decide what's going into 1.5, and
here's a list of open questions:

   1. The #2897 (reflective merges) branch. Do we need to ship with
      it for 1.5, or should we punt and say this problem is too
      complex to put into the initial release of merge tracking. The
      latter way would mean adjusting user expectations accordingly,
      carefully documenting what 1.5 does and doesn't handle.

      This possibility is no slight on Kamesh, by the way: he's
      putting in a ton of work on the #2897 branch, and reflective
      merge handling should get folded in someday. The only question
      is whether we can afford more delay on 1.5 in particular.
      
      We do not currently have a clear answer about whether to try to
      incorporate that work into 1.5. The branch isn't getting the
      review it needs, because there's so much else going on. Should
      we try to make a decision now? (I won't try to hide the fact
      that I'm worried about merging a branch that big without
      substantial review; and I'm worried about delaying 1.5 more. On
      the other hand, if the feature is close to done and relatively
      stable, I don't want to turn it down, either.)

      This is the major outstanding question for 1.5. We need to
      settle it ASAP.

   2. Related to the above: there's the question of whether that
      branch truly needs SQLite or whether (as glasser thinks) it can
      be done in a more FSFSy way. This question is not itself a
      release-definer: it could go either way, just noting it here.

   3. Finishing up and merging the reintegrate branch. glasser and I
      are working on that, remaining time should be measured in a
      small number of weeks. I think this branch is a pretty safe bet
      (it's a much smaller problem than #2897), though glasser and I
      need to chat about a couple of items in the
      'reintegrate-branch-TODO' file.

   4. Solution to the 'cp -g' conundrum: well, we came to a pretty
      convincing agreement in IRC. This isn't a hard problem
      technically, it's just a matter of making a decision. I think
      we have made one (see transcript below), and probably the best
      course is just to commit it and make sure no one objects
      violently.

   5. 'log -g' and 'blame -g' need to be fixed; hwright and cmpilato
      are both working on that. I think there aren't any difficult
      decisions to make here, it's just that they need to be fixed.
      This shouldn't take longer than (say) the reintegrate work;
      Hyrum, would you agree?

In summary: our biggest open question is #2897. We really need to
make a decision about that soon. It affects our ability to branch 1.5
more than any other issue.

Here's the IRC transcript:

<kfogel> So I see a few major questions still outstanding:

<kfogel> 1. Is the #2897 work going into 1.5?

<kfogel> 2. If so, will it depend on SQLite or not?

<kfogel> 3. (related) Are we trying to ship without SQLite dep?

<markphip> 3. we need it for the node origins cache

<kfogel> 4. How much work left on reintegrate branch?

<kfogel> (I'll say "EOT" when done listing, btw.)

<glasser> markphip: no we don't, that's dirt simple to reimplement
           in a more FSFSy way

<kfogel> 5. The 'cp -g' situation. This is more of a plain
           decision than an implementation question. I'll expand on
           it in a moment; it shouldn't take long, it's just a
           discussion.

<glasser> hwright: dannyb was suggesting otherwise when i saw him
           the other day :)

<markphip> glasser: excellent!

<kfogel> 6. log -g and blame -g need to be fixed, but that seems
           to be happening pretty quickly (yay, hwright and cmpilato)

<kfogel> Okay, EOT.

<kfogel> Is that list missing anything major?

<hwright> glasser: yay for subversion!

<kfogel> (Of course, there's the 19 or so issues marked "1.5" in
           the tracker, but I'm going for overview of big things
           here.)

<hwright> (the process, not the software)

<markphip> Since you gave EOT ... is 5 still an issue? I thought
           that went away back when cmpilato made his changes

<kfogel> markphip: yes, it's still an issue.

<glasser> 1: Um. I can't tell to what degree Kamesh's branch is "an
           easy obvious fix" or a big ball of hair, and nobody else is
           looking

<kfogel> Before I go into details why, I'd like to get overall
           response to 1-6, so let's table that for a moment.

* kfogel was referring to cp -g just now

<markphip> I was under the impression glasser knew how to remove the
           need for SQLite if we did #1

<kfogel> glasser: that's what I'm worried about. My instinct is
           "big ball of hair" (no slight on Kamesh there, it's a hard
           problem), and that we want to just defer that
           feature/bugfix until after 1.5, and adjust expectations
           accordingly.

<glasser> 2: I think I can make it not depend on SQLite. And if we
           don't put that in at all, we definitely don't need SQLite

<kfogel> whew -- both halves of that statement are reassuring

<glasser> kfogel: Yeah. Like, the core of what he's doing is
           actually pretty localized. On the other hand, it's a
           subtle heuristic, dunno how easy to implement

<kfogel> glasser: I'm sorry so much is on your shoulders, btw :-(.
           Just personally a difficult time for me, a lot of my life
           is consumed with moving logistics right now, so I can't
           exceed my 20 hrs/wk easily.

<markphip> the node origins cache seems pretty critical for
           performance. So I'd be interested to hear how that can be
           replaced

<glasser> 3: I don't care about deps or not, I just care about using
           the DB correctly. If we ship wtih SQLite dep we need to
           rewrite all the SQLite code anyway to not be broken
           (copies/deletes). I know how, I'd just rather not if
           unnecessary

<kfogel> *nod* ok

<glasser> kfogel: Oh, understood. And I'm spending a bit of time on
           non-1.5 stuff at work (not to mention outside of work)
           right now anyway to give myself a breather

<markphip> 3. I agree. having the dep is not the issue

<glasser> 4: I dunno, I think we're almost ready to merge back. I
           kind of think we could do it right now. It might even help
           Hyrum with the log/blame stuff

<markphip> removing it would be great of course

<glasser> Though I haven't looked at our TODOs in the past couple of
           days

<kfogel> ok -- looks like I was more concerned about the pure
           sqlite dep question than anyone else, but I'm happy to
           relax about that.

<kfogel> glasser: I'm chipping away at them, but I think I will
           need your help for some. I'll take a look now while
           you/others are typing

<markphip> I think a lot of people mistook the work glasser was doing
           as a desire to remove the dependency, when he was really
           just fixing the problems

<glasser> 5: Well, we have a current decision. cmpilato even added
           some doc about it to the "svn cp --help" yesterday (sadly
           the text isn't necessarily... constructive? but I'm not
           sure how to do it better)

<kfogel> markphip: (yes, that may have happened.)

<markphip> has there been any recent threads about cp -g ? I really
           thought cmpilato just removed the need for it completely

<kfogel> Okay, here is my understanding about cp -g:

<kfogel> (and I'll check this with cmpilato after I type it, as
           he's sitting right next to me)

<glasser> He removed the need. In return we get "mergeinfo gets
           corrupted in a way that nobody not on IRC right now will
           ever understand". I'm not convinced that I care too much,
           though

<kfogel> Currently, 'svn cp' with no '-g' will be a purely local
           operation, but will not preserve mergeinfo in certain
           circumstances. With -g, it does (or just may?) contact the
           repository, but it will always preserve all mergeinfo;
           however, you'd have to remember to type -g.

* kfogel checks with cmpilato now

<kfogel> Mike says he removed the option entirely!

* kfogel listens to mike

<glasser> Dumb question: how did "cp -g" ever even work? Wouldn't
           there have been a race condition between the parent's
           mergeinfo when you type "svn cp -g" and whne you actually
           commit anyway?

<markphip> it is based on what you copied though, right? And there
           is no race there

<kfogel> Mike and I are settling on a solution here; it might be
           nice to let you all in on it :-).

<markphip> Issue 2897 is our only remaining P1

<kfogel> I'm not going to go into details of what 'svn cp -g' *did*
           precisely, except to say that it preserved some mergeinfo
           by contacting the server (I don't know all the details).

<kfogel> But Mike and I propose this:

<kfogel> (one sec, sorry)

<kfogel> mike is joining

<kfogel> typa typa typa

<cmpilato> Here's the proposal: we never re-add -g to 'svn cp';
           instead, we make that behavior the default for 'svn cp'.
           then, we add --ignore-mergeinfo (or somesuch) which
           actually can apply to *all four* forms of 'svn cp' (i think
           there's valid use-cases for such)

<kfogel> (the priority this proposal is aiming for, as soon as mike
           gets done typing, is that the most important thing is for
           cp to be correct by default, and local-only operation can
           be a user-chosen decision if necessary, I think)

<cmpilato> yes, this changes the default behavior of 'svn cp WC WC'
           w.r.t. repos contact.

<cmpilato> no, we don't care.

<kfogel> Amen. bikeshed: --local-mergeinfo-only for the opt, but
           whatever

<cmpilato> because we value correctness, and merge tracking is a big
           enough change that the whole world gets to be rocked a
           little bit anyway.

<glasser> If we can wrap the "can't contact server" error in a "try
           --local-only" message, then I'm very +1

<kfogel> (glasser: after this discussion, I have a couple of quick
           questions for you re reint, that may help me take care of
           some of the other todo items so you can concentrate on
           2897; more on that right after this cp discussion)

* cmpilato doesn't like the current do-the-best-you-can behavior.

<pburba >if we really don't care then about the change in default
           WC->WC copy behavior then +1 here too. That was ever the
           *only* objection as I recall.

<glasser> (I mean, the big downside is if the "svn cp" is actually
           being run by some tool N levels away from a user-editable
           command-line)

<cmpilato> yes, that's true, but none of our subcommands ever
           *promise* to be offline-only.

<kfogel> I think we sort of care, but not enough to tolerate
           incorrect default behavior. Yes, it's too bad we lose the
           local-only behavior, but that's an okay price to pay. We
           need to make sure to fix up the error messages as glasser
           says, and note this prominently in the release notes.

<cmpilato> we aren't violating promises in this way.

<hwright> so, would 'svn cp' still create mergeinfo? or would the
           current behavior stick?

<cmpilato> that some commands didn't need to contact the repo before
           turned out to be beneficial (and a selling point), but...

<kfogel> (and note that we're free to *restore* local-only behavior
           in future releases, when our wc can cache more data
           perhaps)

<cmpilato> 'svn cp' would not *create* mergeinfo; though it might
           duplicate some from the source/source's-parents

<kfogel> glasser: for "some tool", read "TortoiseSVN", and we work
           closely enough with them that we can make sure they know
           what to do.

<markphip> would it always contact the repository?

<cmpilato> svn cp would always contact the repository unless
           WC-WC-style and asked not to contact the repos.

<cmpilato> (basically)

<markphip> I thought people considered that not acceptable?

<cmpilato> i was one of those people.

<glasser> kfogel: Tortoise is fine, I'm thinking of other things
           that are doing a higher-level op that happens to include a
           cp in the middle. But whatever, maybe it'll be OK

<cmpilato> people change.

<glasser> If the answer is "these people need to rewrite their
           tools, oh well", then, mm, ok

<kfogel> glasser: yeah, there will be fallout. But having
           mergeinfo discovered later to be incorrect will produce
           much worse fallout, in the long run.

<kfogel> because the latter is a time bomb, where as the former
           hits you right up front, and has a clear immediate
           workaround (the new option).

<glasser> agreed

<cmpilato> +1

<markphip> I'm slightly less convinced

<kfogel> now, that error will only come after an N-second or
           N-minute timeout while it tries to contact the repos from
           your airplane seat...

<pburba >cmpilato: We'd contact the repos even if the copied source
           path has explicit mergeinfo?

<markphip> if the information is potentially critical, then I am not
           sure a user should have the option to do it incorrectly.
           If contacting the repository is a show-stopper, it seems
           like we need a way to defer it or something as we had
           discussed way back when

<kfogel> markphip: we can potentially fix this decision in later
           releases. But we can't go back and fix people's mergeinfo
           if it's wrong.

<cmpilato> pburba: oh! good point! no, that'd be silly.

<kfogel> markphip: I agree that would be a better solution, but
           trying to implement it for 1.5 will cause too much delay,
           that's all. Let's defer that.

<pburba >cmpilato: I think the if the copy source has a WC parent
           with explicit mergeinfo to inherit we'd be in the same boat
           (no need to copy). But then I think about mixed rev WC
           (something glasser mentioned a while back) and get sick.

<cmpilato> pburba: but i'm not sure we can trust anything other than
           the exact copy source -- not its parents.

<cmpilato> oh. heh. you were in the same town i was, there.

<cmpilato> i'd feel much better about saying outright "we only trust
           the copy source"

<cmpilato> much less fuzzy and complex to explain to folks.

* pburba meant "no need to copy" -->"no need to contact repos" if that wasnt obvious

<markphip> kfogel: thanks, that is a good point. This solution could
           be improved later without changing the interface

<kfogel> very much, yeah

<glasser> kfogel: Oh! I have a super huge reason why this change is
           good :)

<kfogel> hit us!

<glasser> kfogel: because otherwise every wc-wc copy (which in
           practice is a rename) introduces mergeinfo on the kid that
           differs from its parents

<glasser> which breaks --reintegrate :)

* kfogel ponders that

<kfogel> really?

<kfogel> every one?

<glasser> maybe not quite every

<cmpilato> even same-dir copies?

<glasser> (for the record, I am still of the opinion of "if you ever
           end up with explicit mergeinfo that isn't at the top of a
           branch, and you're a relatively simple project, not some
           enormous beast, you are wrong")

<glasser> (eg, "open source projects smaller than mozilla shouldn't
           have mixed mergeinfo")

<kfogel> glasser: I think that's probably true almost always, sure.
           But it's hard to predict usage patterns; there may be ways
           of working we haven't thought of.

<kfogel> okay, it sounds like we have a general agreement here,
           anyway (and I doubt the list is going to reverse that).
           glasser, can I poke you about some reint stuff briefly?

<glasser> (Or to put it another way: the metnal model of mergeinfo
           when it's only at the root of branches is really easy to
           explain to people and comprehend. Nested mergeinfo is
           stupidly complex)

<markphip> help me out here ... I am working in one of these "sane
           branches". I rename/move some files (refactoring - which
           someday we will handle) does this need to create mergeinfo
           on those files?

<glasser> sure

<glasser> I'm lunching soon though

<kfogel> glasser: oh, let's answer markphip first, I can hit you up
           after lunch.

<kfogel> better to get to the bottom of cp questions

<kfogel> markphip: I believe the answer is the explicit empty
           mergeinfo needs to be created?

<kfogel> I am SO not sure about that, though.

<kfogel> or rather, explicit mergeinfo that either preserves the
           source's mergeinfo, or is empty.

<kfogel> it doesn't want to inherit from the new parent, since that
           would be invalid

* kfogel waits for someone to tell him he's insane

<markphip> I ask because I do not think simple open source projects
           would consider that they are doing anything strange or
           wrong

<kfogel> well, copying isn't strange or wrong

<kfogel> subversion should do the right thing w/ mergeinfo under
           the hood, without bothering the user, is the wohle point
           here.

<glasser> markphip: i think the easy answer is "if all mergeinfo is
           at the root, and you're allowed to contact the server, then
           it all elides away"

<markphip> if it creates mergeinfo that makes things look like you
           did something strange or wrong though ??

<kfogel> markphip: not sure what you maen

<markphip> glasser: well that is what I am getting at then. Are we
           saying we would "fix" this problem

<markphip> kfogel: if we create mergeinfo on the files, it starts to
           resemble a branch where cherry picking happened

<markphip> when all they really did was "normal" development

<markphip> we ideally want a system that encourages and preserves
           mergeinfo at the branch root

<kfogel> markphip: I think glasser's point is that contacting the
           server allows us to avoid setting that explicit mergeinfo
           (empty or otherwise) in most cases.

<kfogel> Which I agree with, if that's what he's saying.

<markphip> kfogel: right, and that was really what my original
           question was

<kfogel> (Heck, I agree with it even if he's not saying it!)

<kfogel> ok

<markphip> I agree that should be our goal

<kfogel> so the answer satisfies you, I'm presuming :-)

<markphip> yes

<kfogel> cool

<markphip> if the root of your WC is the only place that contains
           mergeinfo, could we not optimize this and not contact the
           server?

<kfogel> glasser: I'll hit you up after lunch about the reint stuff

<markphip> too much to check?

<kfogel> (and then I think I'll be ready to post a State of the
           Onion mail)

<hwright> kfogel: re 6: there may be a bit of design work, now that
           we don't have a way to detect branching copies.

<kfogel> markphip: I think the deal is, there are some
           circumstances where the wc will not have all the info we
           need, and in those circumstances, we'll need to contact the
           server.

<hwright> but I don't think it to be overly complex.

<kfogel> hwright: ooooh. uh, okay (shiver)

<kfogel> I hear "design work" and worry.

<kfogel> You're talking about blame, not log?

<hwright> well, not *that* kind of design work, more like "how does
           this really impact us, and what do we have to do about it?"
           design work.

<kfogel> ah, okay

<hwright> both will need a look-see.

<kfogel> ok

<hwright> witness the failing blame test on trunk right now. :(

<kfogel> I think enough information has been gathered here. I'm
           gonna go grab lunch and then try to post the promised
           followup to Mark on dev@.

<kfogel> hwright: *nod* this is separate from the 'log -g output
           format' discussion, though?

<hwright> seperate, but that discussion is related.

<kfogel> (There, I think the final answer is that our non-xml
           cmdline client log -g output simply cannot be very
           intuitive, and we'll have to live with that)

<hwright> because we can't filter history in front of branching
           revisions, our log output now becomes very verbose, hence
           the discussion.

<kfogel> s/in front of/earlier than/ ?

<hwright> yes

<kfogel> there might be ways the client could bunch stuff up and
           then spew it after removing redundancies, but that would
           lower responsiveness.

<kfogel> hmmm

<hwright> yup. cmpilato and I had a conversation about this last
           night, which prompted his mail.

<hwright> we were both pretty tired, though, so the details are a
           bit fuzzy. :P

<kfogel> ok

<kfogel> brain full, stomach empty

<kfogel> back l8r

<hwright> same.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe_at_subversion.tigris.org
For additional commands, e-mail: dev-help_at_subversion.tigris.org
Received on 2008-01-10 19:39:06 CET