--On 16 January 2007 10:13:05 +0000 Philip Hazel <ph10@???>
wrote:
> On Mon, 15 Jan 2007, Ian Eiloart wrote:
>
>> I'm getting the hang of this now. I've created a MacPorts installer,
>> which makes me a copy of the test suite minus all the tests. I found a
>> few problems doing this:
>
> I'm not surprised! The test suite wasn't designed to be test-free. Hmm.
> It must be Somebody's Law that any software one releases always gets
> used in a completely unexpected manner by someone, thereby causing
> "issues". :-)
:-) Well, it just seemed to me that creating a suite of tests for my config
file from scratch would be somewhat like reinventing the wheel. I hope you
don't think I'm complaining that software is faulty - just pointing out
some ways in which it could be made easier to repurpose the code in the
suite.
I've already created a number of tests for my config file, and I'm
extremely happy that it's very easy to create the tests. Even better,
creating the baseline output files is trivial with the test-suite using a
known working config file.
>> 1. The build would have been easier if the Makefile supported 'make
>> install'.
>
> There is no concept of "installing" the test suite - it is supposed to
> be run where you unpack it.
Yes. This is only an issue because I'm using MacPorts, which downloads and
unpacks packages in a scratch space. The next thing it does (by default) is
run "make install". Actually, since I don't need the provided tests for the
current purpose, it's not a big deal.
> Partly this is because I expect sysadmins
> not to want to make it publicly available,
I'm not installing on a publicly available machine. Only sysadmins can log
in to this machine.
> but partly also because there
> is a different release with each Exim release, so there is also no
> concept of "the current test suite". Rather you have "the test suite
> that corresponds to Exim release x.y".
>
>> 2. It would be nice to be able to specify a group to use. Normally I run
>> exim under group 'mail', so I had to create the group 'exim'.
>
> As it says in the README
>
> (5) Exim must be built with its user and group specified at build time,
> and with certain minimum facilities, namely:
Ah, so I need to fix my installer to specify a group. I thought it did, but
actually it's relying on the fact that the "exim" user's primary group is
"mail". The Exim build configuration file seems to condone this:
"# Many sites define a user called "exim", with an appropriate default
group,
# and use
#
EXIM_USER=exim
#
# while leaving EXIM_GROUP unspecified (commented out)."
>> 7. I'll usually want to run my tests all on the same config file, so it
>> would be easier to specify a default config file. Otherwise, I can just
>> create tons of links, I suppose.
>
> You can, of course, do what you want by just having a single test script
> that contains lots of different calls to Exim. The design of the suite
> is that each top-level test has a different config file. All the tests
> of that config are then located within the single script.
Yes, that's true. However, I like the granularity and organisational
discipline that separating tests give me. I've settled for putting several
similar tests into each script. Therefore, if I change - say - an ACL,
it'll be easy for me to find and work on the tests for that ACL.
At the moment, my tests work quite quickly, but I need to create many more,
so separating the tests will allow me to run only selected tests.
I'm creating 2-hop links, so that I can switch the target between my
installed and test versions of my config file easily. So, all my testsuite
"config files" are links to another link. That link can be pointed at the
config file that I want to test. It's inelegant, perhaps, but adequate.
--
Ian Eiloart
IT Services, University of Sussex
