[svn.haxx.se] · SVN Dev · SVN Users · SVN Org · TSVN Dev · TSVN Users · Subclipse Dev · Subclipse Users · this month's index

Re: Improving our release process

From: Julian Foad <julianfoad_at_btopenworld.com>
Date: Fri, 30 Aug 2013 09:37:33 +0100 (BST)

Ben Reser wrote:
> 1) Decrease the amount of testing done by the every commit buildbots on the
> branches.  This bots should be a quick sanity check, does it build and does it
> pass basic tests.
> 2) Create a buildbot setup that generates a tarball on each branch nightly (if
> there were any changes).  It would do this in the same process that a RM would
> use (and eventually would become the process for producing the tarball).
> 3) This tarball would then have various build bots run tests off it.  These
> tests would be much more extensive.  In particular it would run cross version
> (SERVER_MINOR_VERSION), packing, sharding and other variations of our test
> suite that frankly probably don't get as much attention now as they should.
> 4) Eventually make it so that the RM asks the buildbot to produce the release
> tarball, including running the nightly buildbots against it.  RM then downloads
> the tarball, compares it to the branch_at_revision for differences looking for any
> unexpected diffs, and then signs it.  It gets put up for everyone to approve.
> 5) Humans would no longer do much testing.  We'd have buildbots for the major
> platforms, which would have been included by our manual testing from the past.
> The build bots would be running far more extensive tests than we ever would be
> likely to run on our own.  So the only testing we'd need to do is verifying
> that the builds match the source we expect and whatever sanity testing we felt
> like we wanted to do.

One more thing we'd need to to before signing, of course: verify that the buildbot test runs have completed to our satisfaction (right tarball, sufficient platforms, etc.).

> This should lower the bar dramatically for getting tests/signatures out.
> Humans don't need to do this boring and repetitive work.  It should let
> everyone spend more time on more important issues than manual testing.
> Infra had offered us some more buildbot resources here recently.  This seems
> like the logical way to use them.

+1 to all you said.

- Julian
Received on 2013-08-30 10:38:38 CEST

This is an archived mail posted to the Subversion Dev mailing list.