Re: Release Management for 1.6.x

On Tue, Sep 16, 2008 at 9:46 PM, Hyrum K. Wright
<hyrum_wright_at_mail.utexas.edu> wrote:

> Since the 1.5.x release process fiasco, I've been doing a lot of thinking about
> how to avoid that situation in 1.6.x. These are some of my ideas, and
> ultimately they form the basis of how I'm planning on proceeding with the 1.6.0
> release.
>
> Branching 1.6.x will happen sometime during the first couple weeks of November.
> This is flexible, but we should try to be as close to that time frame as
> possible. If features aren't quite ready for release by that time, we'll can
> pull 'em and they can wait for 1.7. We already have enough new features to
> justify a release by the end of the year.
>
> As soon as the branch happens, I'll roll 1.6.0-pre1. I'll continue with weekly
> -preX releases until there are no known issues, as which point I'll roll
> 1.6.0-rc1. If we don't have any known issues when we branch, I'll skip the
> -preX series. The purpose of the -preX series is to get code from the branch
> released soon and often, even if we are still working on stabilizing known
> issues. Hopefully your testing environments are well scripted by now. :)
>
> Once 1.6.0-rc1 is released, I'll follow the standard soak process, with a final
> RC the week before release. Release Candidates should be just that: code which
> is ready for release, and which we'd feel perfectly comfortable releasing as the
> final release. Let's not have 11 "release candidates" before we find one we can
> actually release.
>
> I don't think these ideas are radically different than we've previously done,
> just a little bit more formalized. Feedback is welcomed and encouraged.

I am +1 to the plan because I think getting the code out in people's
hands on a regular basis will be good. That said, I do not believe
these changes by themselves would do anything to change what happened
last time. Take a close look at what happened:

There were 11 release candidates. But 5 were never released because
our own test suite failed and one had to be reissued two days later.
In addition the last 4 release candidates came out within the last two
weeks. This tells me a couple things:

1) We need to fix our process that leads up the RC so that test
results do not come as a surprise. Ideally we could make better use
of buildbot and have it do a complete test run on the branch on a
regular basis. But maybe there are other options. The -preN releases
are not going to help unless people are going to run all the tests.
Given that there is nothing stopping us from doing it without these
tarballs, I do not think releasing them is going to fix this. I still
think they are a good idea, they just are not going to fix this alone.

2) We did not get any real community feedback until we were
threatening to do the final release. I do not think there is much we
can do about this, it is something that I think is just an unknown
factor we always have to deal with. There are a lot of tools and
scripts that use our libraries and we know our test suite is weak in
this area. I expect this to be a continuing problem in future
releases. That said, if our release process becomes more efficient,
we may find that a lot of these things start getting reported and
fixed in ".1" releases because we got the release out before community
people started to really test it.

With the exception of all the embarrassing failures to get the RC's
released, I do not think the cycle was as abnormal as people think.
There were essentially 3 RC cycles. The last cycle just had a lot of
bad timing as new API compatibility issues kept surfacing on the same
days we were doing a "final RC". With just different luck these could
easily have all come in together and gone out in one RC.

A product as widely used as SVN is never going to be released in one
RC, at least not unless our tests start covering a lot more of our
API. While I do not think we should plan to have multiple RC's, I
think we should still expect it is going to happen and not feel like
we failed if it does.

-- 
Thanks
Mark Phippard
http://markphip.blogspot.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe_at_subversion.tigris.org
For additional commands, e-mail: dev-help_at_subversion.tigris.org