On Wed, 12 Jul 2006, Martin Mrazik wrote:
> I was playing with the testsuite (latest testsuite-snapshot with exim-4.62) a
> bit and found a few issues. Attached is a proposed patch to solve most of
> them.
Hello Martin,
Thank you for playing with the testsuite. Only a few other people have
done so as yet, so it is nice to get more feedback. Especially when it
comes with patches! In a week or two I'll be thinking about pulling
together a 4.63 release, and at that time I'll be making sure the
test suite is up-to-date. When I do so, I'll deal with the issues you
raise.
> Also I think it would be nice to have an option (-NOINTERACTIVE ?) to run the
> tests in non-interactive mode. My idea is to run the testsuite and at the
> end just collect some statistical data and logs. What do you think about
> this? I'm willing to implement it.
Yes, that is something I have thought about, but have never had the time
to do anything about it. Thanks for the offer.
> vii) From time to time some tests are failing due to bin/server timeout. I
> would consider raising the default timeout to 10s (not included in the patch)
>
> Sometimes test 0461 is failing too and I'm suspecting timeouts in confs.
Yes, I see this too. It is very difficult to test things that depend on
timing in a system where other programs are running at the same time.
> viii) Output formatting problems.
> My ls(1) is using different time/date format than expected and thus # 0345 is
> failing. I would consider making a file like KNOWN_ISSUES - list of tests
> likely to fail due to formatting and/or other errors.
Good idea.
> I have some troubles with 0476 and 0529 as well:
>
> Line 73 of "test-stderr-munged" does not match lines 73-74 of "stderr/0529".
> ----------
> now=1152723074 received_time=1152636673 diff=86401 timeout=259200
> ----------
> now=tttt first_failed=tttt next_try=tttt expired=0
> received_time=tttt diff=tttt timeout=259200
> ===============
> 1 difference found.
>
> Any idea what might be wrong? For me it seems this is just a different output
> formatting.
This is something that has been changed for 4.63. The problem is that
you have used the snapshot test suite with 4.62. I think you will find
that this will work if you use the snapshot test suite with the snapshot
Exim, or the 4.62 test suite with 4.62.
> ix)
> I was not able to find out why tests 0291 and 0104 are failing. In both cases
> the testsuite is expecting the [duplicate, would not be delivered] string but
> apparently my exim is not printing this. Any clue?
> ----------
> c@one
> ----------
> c@one [duplicate, would not be delivered]
Same as above.
Philip
--
Philip Hazel University of Cambridge Computing Service
Get the Exim 4 book: http://www.uit.co.uk/exim-book
