Re: [RELEASES] Relaxing the release signature requirements

On Sat, Aug 12, 2017 at 7:35 PM, Daniel Shahaf <d.s_at_daniel.shahaf.name> wrote:
> There has been some random discussion about relaxing the release
> signature requirements.
>
> Our hard requirements are:
>
> - One signature [ASF policy]
> - Three +1 votes [ASF policy]
>
> Our current policy is:
>
> - One RM signature
> - Three unix +1s with signatures
> - Three windows +1s with signatures
> + Traditionally these would be seven separate persons, but nowadays we
> are less strict about this.
>
> [The above is facts. The below is opinion.]
>
> Properties we want the new policy to have:
>
> - Fewer than seven people involved
> - Testing on all platforms
> - Complies with ASF policy
>
> So, how about:
>
> - Allowing the RM to cast a +1.
>
> I think this shouldn't be automatic; the RM does not automatically
> cast a +1 on account of having rolled the tarballs, rather, the RM
> only casts a +1 if he specifically sends a "Summary: +1 to release"
> vote indicating he'd tested the tarballs [which he'd produced] the
> usual way.
>
> The rationale is that _rolling_ a tarball is entirely different to
> _testing_ it.
>
> - Requiring fewer than three +1s per platform.
>
> E.g., we could require just two windows +1s and two unix +1s.
>
> If we do this, I would prefer to see the two unix +1s from different
> unix variants. (We have at least four devs on debian/ubuntu, and I
> don't think four +1s one these two are equal to two +1s from two more
> different unixoid platforms.) Likewise on windows, I suppose, but I
> don't know that platform's variations well enough to have an opinion.
>
> These two changes together would mean that only four people would be
> needed to make a release: two devs per platform, one of whom doubles as
> an RM. We could even theoretically manage a release with only three
> developers, if one of them tested on two platforms, two tested on one
> platform each, and one of the three acted as RM — but having fewer
> people involved increases the risk of overlooking some showstopper.
>
> To be concrete, here's the suggestion again without annotations:
>
> - A tarball is rolled by the RM.
>
> - The RM signs the tarball before uploading to /dist/dev.
>
> - The tarball is tested by at least two windows developers, who
> SHOULD use different variants of Windows, and receives their +1
> votes and signatures. The RM MAY be one of these developers.
>
> - The tarball is tested by at least two unix developers, who
> SHOULD use different Unixoid platforms, and receives their +1
> votes and signatures. The RM MAY be one of these developers.
>
> - There MAY be more than two testers per platform. The testers need
> not be committers.
>
> - There SHOULD be at least four different testers.
>
> - Developers MAY sign the tarball without testing it, only if they
> have verified that it matches the tag [with the expected differences].
>
> - The release timelines (at least 72 hours and preferably more for
> testing/votes, then 24 hours for the mirrors) are unchanged.
>
> WDYT?

Overall, I like your proposal. Reducing it to 2+2 sounds quite reasonable to me.

What do you mean with "The testers need not be committers"? Do you
mean the *extra* testers from the sentence before ("There MAY be more
than two testers per platform")? I.e. anyone who tests and sends their
results to the list provides extra value, but the two binding votes we
need (per platform) have to come from committers / pmc members, right?

I'm a bit worried about the extra requirement of "SHOULD use different
unixoid/windows platforms" (and I'm not sure what it would mean for
Windows: different versions (Seven vs. 10)? 32-bit vs. 64-bit?
Different VS versions?). We don't have that requirement right now, so
this would raise the bar from where we are now. At present, in theory
the 3 unix sigs can all come from Ubuntu, and the 3 windows sigs can
all come from the exact same Windows version.

Of course you said "SHOULD", so it's more a preference than a
requirement ... maybe it's not a problem in practice. If the two
binding unix votes come from the same platform, will we accept the
vote as valid?

What about the bindings? I've never built / tested them myself, so my
votes are less "complete" than some of the other devs. Yet it counts
for the current 3 sigs requirement (fortunately other devs do test the
bindings). In the new scheme: what if both Windows sigs would not
include the bindings? Should we require at least one vote per platform
to include the bindings?

-- 
Johan