Re: [exim-dev] Preliminary testing of a new Exim test suite

Top Page
Delete this message
Reply to this message
Author: Daniel Tiefnig
Date:  
To: exim-dev
Subject: Re: [exim-dev] Preliminary testing of a new Exim test suite
Philip Hazel wrote:
> I've updated the test suite in
>
> ftp://ftp.csx.cam.ac.uk/pub/software/email/exim/Testing/exim-testsuite-0.00.tar.bz2
>
> to take account of all the comments so far received. Some more tests
> have been added.


Just a quick run of the test suite, don't have much time to investigate.
Comments are preceeded by "Note: " to make them better distinguishable
from script-output.


Basic/0032 Bounce message for MX->localhost
===============
Line 22 of "mail/0032.CALLER" occurs before line 22 of "test-mail-munged".
----------
    lowest numbered MX record points to local host
===============
1 difference found.
"test-mail-munged" contains 34 lines; "mail/0032.CALLER" contains 35 lines.


Note: Lowest numbered MX record doesn't point to local host here, it
seems. :o)


GnuTLS/2008 TLS client: general
===============
Lines 16-18 of "test-mainlog-munged" do not match line 16 of "log/2008".
----------
1999-03-02 09:44:33 no host name found for IP address ip4.ip4.ip4.ip4
1999-03-02 09:44:33 no host name found for IP address ip4.ip4.ip4.ip4
1999-03-02 09:44:33 10HmbB-0005vi-00 <= CALLER@???
H=(myhost.test.ex) [ip4.ip4.ip4.ip4] P=esmtps
X=TLS-1.0:RSA_AES_256_CBC_SHA1:32 S=sss id=E10HmaY-0005vi-00@???
----------
1999-03-02 09:44:33 10HmbB-0005vi-00 <= CALLER@???
H=the.local.host.name (myhost.test.ex) [ip4.ip4.ip4.ip4] P=esmtps
X=TLS-1.0:RSA_AES_256_CBC_SHA1:32 S=sss id=E10HmaY-0005vi-00@???
===============
1 difference found.

Note: The same as above appears in test 2010.

===============
Line 66 of "test-stderr-munged" does not match line 66 of "stderr/2008".
----------
SMTP<< 250-myhost.test.ex Hello myhost.test.ex [ip4.ip4.ip4.ip4]
----------
SMTP<< 250-myhost.test.ex Hello the.local.host.name [ip4.ip4.ip4.ip4]
===============
Note: The same for "Line 74".

===============
Line 2 of "test-mail-munged" does not match line 2 of "mail/2008.abcd".
----------
Received: from [ip4.ip4.ip4.ip4] (helo=myhost.test.ex)
----------
Received: from the.local.host.name ([ip4.ip4.ip4.ip4] helo=myhost.test.ex)
===============


GnuTLS/2011 TLS client: require_ciphers
===============
From line 10 of "test-mainlog-munged" and line 10 of "log/2011" the
files are different.
----------
1999-03-02 09:44:33 no host name found for IP address ip4.ip4.ip4.ip4
1999-03-02 09:44:33 TLS error on connection from (myhost.test.ex)
[ip4.ip4.ip4.ip4] (gnutls_handshake): A TLS packet with unexpected
length was received.
1999-03-02 09:44:33 10HmaY-0005vi-00 <= CALLER@???
H=localhost (myhost.test.ex) [127.0.0.1] P=esmtps
X=TLS-1.0:RSA_ARCFOUR_SHA1:16 DN="C=UK,L=Cambridge,O=University of
Cambridge,OU=Computing Service,CN=Philip Hazel" S=sss
id=E10HmaX-0005vi-00@???
----------
1999-03-02 09:44:33 TLS error on connection from the.local.host.name
(myhost.test.ex) [ip4.ip4.ip4.ip4] (gnutls_hand shake): A TLS packet
with unexpected length was received.
1999-03-02 09:44:33 10HmaY-0005vi-00 <= CALLER@???
H=localhost (myhost.test.ex) [127.0.0.1] P=esmtps
X=TLS-1.0:RSA_ARCFOUR_SHA1:16 DN="C=UK,L=Cambridge,O=University of
Cambridge,OU=Computing Service,CN=Philip Hazel" S=sss
id=E10HmaX-0005vi-00@???
===============

Note: Similar for test 2012.


GnuTLS/2014 TLS server: mandatory, optional, and revoked certificates
** Subtest 10 (starting at line 107)
** Return code 1 (expected 0)

Note: Stderr is empty, I'll try to look into that when I get time.


GnuTLS/2025 TLS server: tls_require_ciphers
GnuTLS/2026 TLS: logging parameters for 2 different recipients
GnuTLS/2027 TLS server & client: no certificate in client

Note: Have similar problems as 2008 and 2011, with the "no host name
found" messages and the "the.local.host.name" part missing.


The other tests passed successfully. Didn't (yet) try with OpenSSL that
time, though.


lg,
daniel