Being of a heretical nature, I once more plunged into the test suite to
see what needed changing to keep the suite mostly working this time.
Being stupid, I did _not_ drink alcohol first.
0152 and 0437 are my main concerns.
On balance, I'm not holding up the 4.77 release for 0152, just 0437
until I understand what is happening.
* Some improved runtest resiliency to setup problems, to make it
clearer what might have been missed in setup (not a change)
* the revamped versioning was not reflected in the tests; updated
patchexim to handle it
* the new ratelimit features need test 0038 updated to handle the new
output and try testing those features; I've mailed Tony, rather than
tackle that myself tonight, so left 0038 failing.
* Changed runtest to ignore messages about loading modules
* Apparently when I added extra tests for boolean inversion in 4.76
PP/08 I didn't actually update the expected-output file. Oops.
Fixed. (On bright side: I did update the test when I fixed
something)
* My usual "6TESTSUITE/aux-fixed/<nnnn>.data" vs ":TESTSUITE..." glitch
which I need to investigate again, to see what I missed before; for
now, I've again just continued past those
[...]
Ah, ':' the 11th lookup type, with the cache keying using offsets
from ASCII '0', and ord(':') - ord('0') == 10; I probably just don't
have the precise set of lookups loaded into the binary that the tests
prefer.
* 0152: Extra Sender: header inserted??
Test comment is: # control of Sender: (local_from_{suf,pre}fix)
So this is all about Sender: headers and something has changed in the
three tests from 4th last to 2nd last, to cause a Sender: header to
be inserted.
I can't see anything in ChangeLog which would explain this.
Still broken.
* 0288: extra messages:
Berkeley DB error: no absolute path for the current directory: No such file or directory
Needs investigating, but looks sufficiently like a test suite
artifact that I'm not looking now.
* 0358: order inversion of userx/usery results; ignored.
* 0369: looks like we're fixing up /bin/sh -> SHELL but log/0369
doesn't know about it? Consensus change expected output?
Note though that nothing has changed in fixup here.
* Basic/0437 queue run: close lookups before delivery
----------------------------8< cut here >8------------------------------
** Command 1 ("exim", starting at line 2)
** Return code 1 (expected 0)
show stdErr, show stdOut, Continue (without file comparison), or Quit? [Q] e
2011-10-10 02:35:23 failed to expand spool_directory "[...]/${lookup{spool}lsearch{[...]/aux-fixed/0437.ls}}": unknown lookup type "lsearch"
----------------------------8< cut here >8------------------------------
Hrm. I'd be more worried if I weren't using lsearch in my own setup,
with every mail sent from pdp@??? using lsearch to get the data
to authenticate to tahini.
Also, there are lsearch tests in 0002, 0064, 0219, ...
If I invoke the binary manually, the failure recurs, but if I pass -d
then the test passes and I see lsearch working.
* 0548: order inversion, same issue as 0358
* 1003: timeout talking to ::1 ... note that this might be further than
I've gotten in the test suite before, so not sure that it's new.
I retried and it worked, so looks like it's just flaky.
* 1006: trailing whitespace difference! stdout/1006 expects an extra
25 trailing whitespace characters (26 total, there is actually one
normally); no idea why that was expected, think that I'm now
thoroughly into weed-whacking territory
* 1009: ditto
* 2102: OpenSSL output variance; lots of diffs reported but it boils
down to this actual vs observed:
----------------------------8< cut here >8------------------------------
output:
29 SSL info: SSLv3 read server session ticket A
[...]
82 SSL info: SSLv3 read server session ticket A
83 SSL info: SSLv3 read server session ticket A
[...]
120 SSL info: SSLv3 read server session ticket A
expected:
81 SSL info: SSLv3 read finished A
82 SSL info: SSLv3 read finished A
----------------------------8< cut here >8------------------------------
My recollection (backed by checking! woot!) is that these status
notes are provided by the OpenSSL library itself, so I am fully
prepared to believe that this is a 1.0.0<x> vs 0.9.<whatever>
difference.
Ignoring for now, but would appreciate a report from someone else on
this, so we know whether or not to update the expected output.
* 2103: 2 extra "SSLv3 read server session ticket A"
* 2105: 1 extra "SSLv3 read server session ticket A"
* 2106: 1 extra "SSLv3 read server session ticket A"
* 2114: many such variances
* 2118: 1 such variance
* 2119: 1 such variance
* 2122: 1 such variance
* 2124: 2 straight "server session ticket" vs "finished" variances
* 2128: 2 extra "server session ticket" messages
* 2150: 1 extra
* 3460: just when I thought I was free, 1 extra
* 3463: 2 extra
* 3464: 2 extra
* 3650: bombed out test suite, we have a script with no config
available; anyone who uses Dovecot willing to volunteer to write the
confs/3650 file?
Resumed with:
./runtest $PWD/../src/build-FreeBSD-amd64/exim 3651 8999
* 5007 (maildir): Date: header positioning change
----------------------------8< cut here >8------------------------------
4.73:
PP/01 Date: & Message-Id: revert to normally being appended to a message,
only prepend for the Resent-* case. Fixes regression introduced in
Exim 4.70 by NM/22 for Bugzilla 607.
----------------------------8< cut here >8------------------------------
I remember fixing one other case of this, years ago; I'm clearly
further on than ever in the test suit to be seeing this only now.
expected output updated
Aaand: test suit completed!
(for the modules tested in my config).
