On 2017-07-27 at 15:04 +0100, Jeremy Harris wrote:
> I don't think that would be good, as it maintains the illusion
> that those builds are actually tested - which, lacking buildfarm
> or other participation, I have no confidence in.
The "or other participation" is the clincher here. In the past, we've
said that if someone tests a Release Candidate on a given platform then
we support the platform but if nobody tests an RC then we don't:
https://github.com/Exim/exim/wiki/EximReleasePolicy#release-candidates--buggy-releases
That's not a binding document and "how we've done things" is not "how we
must do things". Very much not.
The test suite requires pretty much root access by the Exim code on the
entire system with broad sudo. That requires a dedicated OS instance in
most sane environments. For closed-source platforms, that requires
paying money for licenses, etc, and is harder to justify.
So I will typically test that the Darwin stuff "works for me" because I
use a macOS laptop but I don't usually invoke the full test suite there.
I don't have spare equipment to provide a dedicated build-farm animal
for this OS.
Given the security-critical boundary nature of Exim, the goal of "should
have full test suite passes" is a good one. So the switch to "should be
in the buildfarm" is hard to argue against. It's probably the right
move for Exim. The people doing the work, which is mostly not me,
should be the ones making this call, so I shrug and accept it.
But a little more clarity around "buildfarm" vs "someone tests RCs"
would be good, to know whether I need to check files out of git history
to do a laptop build in future.
-Phil
