Re: [exim-dev] test suite

On Mon, Apr 16, 2012 at 1:21 PM, Jeremy Harris <jgh@???> wrote:
> So, I'm running fairly clean passes of the testsuite now,
> but I have a fairly long list of testcase output changes needed
> to do so.

Can you post a list of what does seem to require changes? I have a
lot of test cases failing due to strange differences. Here's a good
example:

Basic/0002 Common string expansions
===============f test-stderr-munged with stderr/0002 failed
Line 177 of "test-stderr-munged" does not match line 177 of "stderr/0002".
----------
5TESTSUITE/aux-fixed/0002.lsearch
----------
:TESTSUITE/aux-fixed/0002.lsearch
===============
Line 222 of "test-stderr-munged" does not match line 222 of "stderr/0002".
----------
5TESTSUITE/aux-fixed/0002.lsearch
----------
:TESTSUITE/aux-fixed/0002.lsearch
===============
2 differences found.
"test-stderr-munged" contains 363 lines; "stderr/0002" contains 363 lines.


Here's another one, the munged output file seems to be sorted incorrectly maybe?
Basic/0005 -bs to simple local delivery
===============f test-mainlog-munged with log/0005 failed
Lines 2-8 of "test-mainlog-munged" do not match lines 2-8 of "log/0005".
----------
1999-03-02 09:44:33 10HmaY-0005vi-00 <= someone@??? U=CALLER
P=local-smtp S=sss
1999-03-02 09:44:33 10HmaZ-0005vi-00 <= someone@??? U=CALLER
P=local-smtp S=sss
1999-03-02 09:44:33 Start queue run: pid=pppp
1999-03-02 09:44:33 10HmaX-0005vi-00 => CALLER
<CALLER@???> R=localuser T=local_delivery
1999-03-02 09:44:33 10HmaX-0005vi-00 Completed
1999-03-02 09:44:33 10HmaY-0005vi-00 => CALLER
<CALLER@???> R=localuser T=local_delivery
1999-03-02 09:44:33 10HmaY-0005vi-00 Completed
----------
1999-03-02 09:44:33 10HmaX-0005vi-00 => CALLER
<CALLER@???> R=localuser T=local_delivery
1999-03-02 09:44:33 10HmaX-0005vi-00 Completed
1999-03-02 09:44:33 10HmaY-0005vi-00 <= someone@??? U=CALLER
P=local-smtp S=sss
1999-03-02 09:44:33 10HmaY-0005vi-00 => CALLER
<CALLER@???> R=localuser T=local_delivery
1999-03-02 09:44:33 10HmaY-0005vi-00 Completed
1999-03-02 09:44:33 10HmaZ-0005vi-00 <= someone@??? U=CALLER
P=local-smtp S=sss
1999-03-02 09:44:33 Start queue run: pid=pppp
===============
1 difference found.
"test-mainlog-munged" contains 11 lines; "log/0005" contains 11 lines.


I've got lots more that are like that.

If it's too painful to read this with line wrapping, in the future
I'll use the dreaded rich formatting to keep it looking better.

Also, I have tried many different configurations trying to get it to
run all the way through. Anything I do with an nfs mounted home with
root squash didn't work properly. So I added two users, exim and
exim-build, both of which have a home directory on a local drive.
What seemed to work best was:
id exim: uid=480(exim) gid=500(pubwww) groups=500(pubwww)
id exim-build: uid=1000(exim-build) gid=500(pubwww) groups=500(pubwww)

There were some weird issues with it, so I modified it, but it doesn't
seem to work as well (much more strange diff outputs):
id exim: uid=480(exim) gid=500(pubwww) groups=500(pubwww)
id exim-build: uid=1000(exim-build) gid=136(exim-build)
groups=136(exim-build),500(pubwww)

Do you have any suggestions about how to change it to make tests run better?

...Todd
--
Always code as if the guy who ends up maintaining your code will be a
violent psychopath who knows where you live. -- Martin Golding