Re: [exim-dev] Preliminary testing of a new Exim test suite

Top Page
Delete this message
Reply to this message
Author: Philip Hazel
Date:  
To: Daniel Tiefnig
CC: exim-dev
Subject: Re: [exim-dev] Preliminary testing of a new Exim test suite
On Thu, 29 Sep 2005, Daniel Tiefnig wrote:

> > Which release of Exim? If not 4.53, that would explain that.
>
> It is 4.53-RC2... I'll retry with the release version.


No need. That explains that anomaly.

> Actually, it doesn't have an A record either. The hostname maps to
> 127.0.0.1 via /etc/hosts, though. It's a workstation in a dynamic
> environment. (192.168.x)
>
> > That would certainly explain this output. I'm not quite sure how to
> > deal with that. One possibility is to make it a requirement for
> > running the test suite.
>
> That doesn't sound good to me, but YMMV.


It doesn't sound too good to me either. Working round this will require
changes to Exim as well as to the test suite, but I think they will
improve the testing interface, so I think I will do them while we are
still at this very early stage. In any case, there are a few minor
problems with 4.53 that suggest 4.54 should be created quite soon.

> That would be better, yes. E.g. with nodes behind a NAT device you may
> have 10.x addresses bound on mailservers. Same for mailservers with
> additional internal interfaces, of course, so this doesn't sound too
> unusual. Or does this only concern the "primary" interface?


This concerns only DNS lookup tests, not the actual IP addresses in use.
I want to be sure that any "fake" address I look up doesn't turn out to
be in the same network as the actual address of the host. I think if I
use an impossible network, this will be OK. I'll go and look at assigned
network numbers to find something suitable among the unused addresses.

> Besides, the test suite will be used on development boxes mostly,
> wouldn't it? So one should maybe try to cope with "all" possible network
> setups anyway. (Maybe even no network at all, at least for test that
> doesn't test network functionalities...?)


I haven't split off tests that require no network into separate
directories, as this would make for an even more complicated structure.
I figured that anybody testing an MTA would probably want to test its
networking features. Similarly, the tests won't run if a certain basic
level of functionality isn't compiled in - the "normal" routers and
transports, for example. Of course it could be done, but I did not think
the effort was worth it. I'll think about the networking issue. Maybe I
can come up with a way of skipping tests dynamically rather than have
them in different directories.

> Hmm, not really. But in test-stderr-server i get:
> exim: incompatible command-line options or arguments
>
> I modified runtest to print out the exim command it is executing, and
> got the following:
> /home/tiefnig/exim/exim-testsuite-0.00/eximdir/exim -t
> -DEXIM_PATH=/home/tiefnig/exim/exim-testsuite-0.00/eximdir/exim -C
> /home/tiefnig/exim/exim-testsuite-0.00/test-config -DSERVER=server -bd
> -oX 1225 >>test-stdout 2>>test-stderr
>
> "-t" doen't like "-bd" and vice versa.


Hmm, I wonder how that happened. -t is used for sending a bounce
message. I'll try to figure out that one.

Thanks,
Philip

-- 
Philip Hazel            University of Cambridge Computing Service,
ph10@???      Cambridge, England. Phone: +44 1223 334714.
Get the Exim 4 book:    http://www.uit.co.uk/exim-book