On 2012-04-16 21:44, Todd Lyons wrote:
> On Mon, Apr 16, 2012 at 1:21 PM, Jeremy Harris<jgh@???> wrote:
>> So, I'm running fairly clean passes of the testsuite now,
>> but I have a fairly long list of testcase output changes needed
>> to do so.
>
> Can you post a list of what does seem to require changes? I have a
> lot of test cases failing due to strange differences. Here's a good
> example:
>
> Basic/0002 Common string expansions
> ===============f test-stderr-munged with stderr/0002 failed
> Line 177 of "test-stderr-munged" does not match line 177 of "stderr/0002".
> ----------
> 5TESTSUITE/aux-fixed/0002.lsearch
> ----------
> :TESTSUITE/aux-fixed/0002.lsearch
Yup;
http://bugs.exim.org/show_bug.cgi?id=1224
>
> Here's another one, the munged output file seems to be sorted incorrectly maybe?
> Basic/0005 -bs to simple local delivery
> ===============f test-mainlog-munged with log/0005 failed
That one's not on my list. What system are you testing on?
>
>
> I've got lots more that are like that.
>
> If it's too painful to read this with line wrapping, in the future
> I'll use the dreaded rich formatting to keep it looking better.
Personally I prefer plain ascii.
Here's my current list (but I've not filtered out the ones due to 1184 yet,
so use a healthy ping of salt):
# modified: log/0027
# modified: log/1003
# modified: log/3411
# modified: rejectlog/0027
# modified: rejectlog/3411
# modified: stderr/0002
# modified: stderr/0038
# modified: stderr/0085
# modified: stderr/0123
# modified: stderr/0387
# modified: stderr/0393
# modified: stderr/0403
# modified: stderr/0414
# modified: stderr/0437
# modified: stderr/0464
# modified: stderr/0471
# modified: stderr/0484
# modified: stderr/3400
# modified: stderr/3408
# modified: stderr/3410
# modified: stderr/3500
# modified: stdout/0240
# modified: stdout/0345
# modified: stdout/0373
# modified: stdout/1006
# modified: stdout/1009
# modified: stdout/2102
# modified: stdout/2103
# modified: stdout/2105
# modified: stdout/2106
# modified: stdout/2114
# modified: stdout/2118
# modified: stdout/2119
# modified: stdout/2122
# modified: stdout/2124
# modified: stdout/2128
# modified: stdout/2150
# modified: stdout/3460
# modified: stdout/3463
# modified: stdout/3464
I'm running local (if virtualised) filesystems. Two users:
eximtest:x:500:500:foobar:/home/eximtest:/bin/bash
exim:x:501:501:bazbim:/home/exim:/bin/bash
eximtest is a member of both groups.
The exim build has EXIM_USER=ref:exim
I've noticed some flakiness in some of the tests;
eventually it'll need tracking down and making solid.
There's something odd in time-related stuff somewhere.
I'm considering a "retry without adjusting results-files"
addition to the runtest script.
--
Luck,
Jeremy
