Re: [exim-dev] Preliminary testing of a new Exim test suite

Top Page
Delete this message
Reply to this message
Author: Stefan Kaltenbrunner
Date:  
To: exim-dev
Subject: Re: [exim-dev] Preliminary testing of a new Exim test suite
Philip Hazel wrote:
> On Thu, 1 Dec 2005, Daniel Tiefnig wrote:
>
>
>>According to this, it would be the safest thing to reduce requirements
>>to an absolute minimum. Like require just AES encoding on the server
>>and disable it in the client:
>>
>> {AES}{!AES:3DES}}
>>
>>Should do the trick quite everywhere, shouldn't it?
>
>
> Yes, I think you are right there. I have made that change, and I have
> just refreshed the tarball with the latest sources.
>
> ftp://ftp.csx.cam.ac.uk/pub/software/email/exim/Testing/exim-testsuite-0.00.tar.bz2



did some initial testing with that on an OpenBSD 3.8-current x86 box.
First problem here is that the test-framework does not even compile on
that plattform due to:

gcc -g -O2 -o bin/fakens src/fakens.c
src/fakens.c:100: error: `ns_t_a' undeclared here (not in a function)
src/fakens.c:100: error: initializer element is not constant
src/fakens.c:100: error: (near initialization for `type_list[0].value')
src/fakens.c:100: error: initializer element is not constant
src/fakens.c:100: error: (near initialization for `type_list[0]')
src/fakens.c:101: error: `ns_t_ns' undeclared here (not in a function)
src/fakens.c:101: error: initializer element is not constant
...

the reason for this is that arpa/nameser.h does not contain the
nescessary definitions. hacking src/faken.c to use the old style T_foo
definitions makes it compile at least.
I also added a gcc-OpenBSD case to the configure-check for dynamically
loaded modules.

Actually running the testsuite results in a fair number of errors - some
like:

asic/0135 ultimate address timeout
===============
Lines 5-8 of "test-mainlog-munged" do not match lines 5-8 of "log/0135".
----------
1999-03-02 09:44:33 10HmaX-0005vi-00 V4NET.0.0.1 [V4NET.0.0.1] Invalid
argument
1999-03-02 09:44:33 10HmaX-0005vi-00 == a@??? R=default T=smtp
defer (22): Invalid argument
1999-03-02 09:44:33 10HmaX-0005vi-00 V4NET.0.0.0 [V4NET.0.0.0] Invalid
argument
1999-03-02 09:44:33 10HmaX-0005vi-00 == b@??? R=default T=smtp
defer (22): Invalid argument
----------
1999-03-02 09:44:33 10HmaX-0005vi-00 V4NET.0.0.1 [V4NET.0.0.1] Network Error
1999-03-02 09:44:33 10HmaX-0005vi-00 == a@??? R=default T=smtp
defer (dd): Network Error
1999-03-02 09:44:33 10HmaX-0005vi-00 V4NET.0.0.0 [V4NET.0.0.0] Network Error
1999-03-02 09:44:33 10HmaX-0005vi-00 == b@??? R=default T=smtp
defer (dd): Network Error
===============
Lines 21-22 of "test-mainlog-munged" do not match lines 21-22 of "log/0135".
----------
1999-03-02 09:44:33 10HmaX-0005vi-00 V4NET.0.0.1 [V4NET.0.0.1] Invalid
argument
1999-03-02 09:44:33 10HmaX-0005vi-00 == a@??? R=default T=smtp
defer (22): Invalid argument
----------
1999-03-02 09:44:33 10HmaX-0005vi-00 V4NET.0.0.1 [V4NET.0.0.1] Network Error
1999-03-02 09:44:33 10HmaX-0005vi-00 == a@??? R=default T=smtp
defer (dd): Network Error
===============
Lines 32-33 of "test-mainlog-munged" do not match lines 32-33 of "log/0135".
----------
1999-03-02 09:44:33 10HmbA-0005vi-00 V4NET.0.0.0 [V4NET.0.0.0] Invalid
argument
1999-03-02 09:44:33 10HmbA-0005vi-00 == b@??? R=default T=smtp
defer (22): Invalid argument
----------
1999-03-02 09:44:33 10HmbA-0005vi-00 V4NET.0.0.0 [V4NET.0.0.0] Network Error
1999-03-02 09:44:33 10HmbA-0005vi-00 == b@??? R=default T=smtp
defer (dd): Network Error
===============
3 differences found.
"test-mainlog-munged" contains 37 lines; "log/0135" contains 37 lines.


only seem to be the result of different OS-level errormessages but I
have to go through all in more detail.



Stefan