Re: [exim-dev] Preliminary testing of a new Exim test suite

Top Page
Delete this message
Reply to this message
Author: Philip Hazel
Date:  
To: John Jetmore
CC: exim-dev, Philip Hazel
Subject: Re: [exim-dev] Preliminary testing of a new Exim test suite
On Sun, 25 Sep 2005, John Jetmore wrote:

> Darwin doesn't support -shared in gcc. The docs I could find said
> -dynamic should work, but it still got the undefined symbol error.


It was trying to build a module that could be loaded for ${dlfunc
testing. I have arranged for it to build this only on systems where
-shared works, currently Linux and FreeBSD. If we find others (with
possibly other options), then can be added. When the module isn't built,
${dlfunc isn't tested, even if it is in the binary.

> test 0001 failed with this:
> LOG: PANIC DIE
> Exim configuration error in line 132 of
> /Volumes/storage/Users/jetmore/Documents/programming/exim/test-env/exim-testsuite-0.00/confs/0001:
> user bin was not found


I've changed the test setting of never_users to use numeric ids.

> LOG: PANIC DIE
> Exim configuration error:
> localhost_number is greater than the maximum allowed value (10)


I've changed the test value so that it is less than the maximum allowed
on base 36 systems.

> 0002 fails with this:
> Basic/0002 Common string expansions
> ===============
> Line 71 of "test-stdout-munged" does not match line 71 of "stdout/0002".
> ----------
> > base62: 0009IX
> ----------
> > base62: 0003D7
> ===============


That showed up a bug in Exim! The ${base62: expansion was changing what
it did for base 36 systems, but the ${base62d: expansion was not. I've
created a patch for Exim, and re-written the test so that it works on
both kinds of system.

> Line 131 of "test-stdout-munged" does not match line 131 of "stdout/0002".
> ----------
> > stat: 32
> ----------
> > stat: 2
> ===============


This one is a mystery. Why are there 32 links to an ordinary directory
on Darwin? Solved by not outputting this particular data from stat.

> Line 531 of "test-stdout-munged" does not match line 531 of "stdout/0002".
> ----------
> >       host_lookup_failed = 0
> ----------
> >       host_lookup_failed = 1
> ===============


This one goes along with

> ===============
> Line 70 of "stderr/0002" occurs before line 72 of "test-stderr-munged".
> ----------
> 1999-03-02 09:44:33 no host name found for IP address 10.11.12.13


It is supposed not to find a host name, but after failing to find it in
its private DNS simulation, Exim was calling gethostbyaddr(), and I
deduce that you have, somewhere in your local DNS, a record for
13.12.11.10.in-addr.arpa. I have arranged for all the tests to set
host_lookup_order = bydns so that it won't ever use gethostbyaddr() when
testing. Hopefully that will fix this one.

> ===============
> Line 4 of "test-stderr-munged" occurs before line 4 of "stderr/0002".
> ----------
> failed to load readline: dlcompat: file "libreadline.so" not found


I guess this might be common in Exims compiled with readline support.
I've arranged to ignore the line.

> ./runtest didn't find build directory named exim-4.53 correctly. comment
> states that exim-n.m is matched, possibly followed by -RCx, but the code
> doesn't match it. Here's the diff to fix:


Thanks! (But I'd actually hit that one myself as well, once I released
4.53.)

> if the host does not have an IPv6 address, pretty much everything breaks.


I think I've fixed that.

> perhaps there should be an option to server on whether to use ipv6 or not?


Indeed. Done that.

> When compiled w/ readline support, test 0002 fails, specifically
> escape: ${escape:CR^MB}
> produces
> > Failed: missing } at end of string
> > B}
> instead of
> > escape: CR\rBS\b


I've changed it to use some high-bit characters for testing escaping,
thereby not confusing readline.

> Sorry for the rambling nature of this email, I was just typing stuff in
> here as I came across it and while working on other things. Hopefully
> it's not too chaotic =).


Not at all. Very useful indeed. Many thanks for the time. I'll be
putting out a new version of the test suite later today, hopefully one
that will run a lot better for you.

Philip

-- 
Philip Hazel            University of Cambridge Computing Service,
ph10@???      Cambridge, England. Phone: +44 1223 334714.