Hello,
On Mon, Apr 20, 2009 at 11:04 PM, W B Hacker <wbh@???> wrote:
> Catalin Constantin wrote:
>>> On Mon, Apr 20, 2009 at 09:47:59PM +0300, Catalin Constantin wrote:
>>>> Even after setting retry_use_local_part, no luck.
>>>> It looks like Lena suggested that this configuration is ignored for
>>>> remote transports.
>>> Yeah, bummer. Not sure why that is.
>>>
>>> Anyway, there's something sufficiently odd about your configuration then.
>>> We send a much larger number of messages to Yahoo than that, and never
>>> run into problems.
>> The odd part in this whole issue is that sending with Postfix from the
>> same system is
>> FAST enough. Trying to switch to Exim (cause of extra config options)
>> ended up with this
>> BIG deliverability issue.
>>> I'd suggest you try and find out why you are getting hit with greylisting
>>> when others who are sending much more are not.
>>>
>> The grey list is something regular for hosts which do not send very
>> high volumes of email.
>> Not being able to proper retry a delivery cause of miss
>> implementations (or no implementation) could be
>> considered a bug ? Should i submit the bug or feature request ?
>>
>> Triggering from cron:
>> # exim -qf
>> is not pretty nice for a workaround.
>>
>>> We handle tens of millions of messages a month and have never run into
>>> problem with remote retries. Now, that being said, we do have split
>>> queues for immediate deliveries, deferred deliveries, and slow delivery
>>> hosts to keep things running tip-top, but still no underlying problems
>>> with retries on a per domain basis.
>> You can consider this "luck" :)
>>
>
> Not 'luck'. Appropriate configuration.
I am pretty sure the configuration is ok.
>
> That you are getting greylisted *repeatedly* indicates you may have some
> other settings that differ from what you ran with Postfix.
Also with Postfix, at the start, the messages get grey listed
(deferred). But Postfix RETRIES the delivery soon enough
to get the message through.
With Exim the only issue is retry. It does not retry to deliver the
email soon enough. Eventually most of the emails will not get through,
because they will grow OLD in the Exim queue.
>
> Peripheral not-quite-right items that might give Yahoo cause to NOT let
> you thru greylisting on second and subsequent attempt, and for longish
> periods thereafter where your FIRST attempt would be accepted.
>
We're an ESP (Email Service Providers). We're in the process of being
Whitelisted.
We send millions of emails monthly. We have on all our OUTGOING mail
servers for now Postfix.
We're investigating Exim because one major thing which we could not
achieve in Postfix, is:
- sender base routing (each customer with it's private outgoing IP -
this is a requirement for whitelisting)
In Postfix, to achieve this the only viable solution for now is to run
multiple instances on the same server (note: the server has several IP
aliases configured).
Now:
> EX:
>
> - do you 'HELO' with a FQDN that is an exact match to that which a
> reverse-lookup recovers?
Of course !
>
> - does your DNS entry have UPPER CASE where lower-case is expected?
All our DNS and reverse DNS are lowercase.
>
> - do you perhaps hit a significant number of non-existent Yahoo accounts?
All emails are gathered via CONFIRMED OPT IN. Each Yahoo permanent
bounce is removed from the list instantly.
So, NO, we don't send to non existing accounts AT ALL!
>
> - are you jumping-off from an IP block that is unwelcome?
Since delivery from Postfix is fine and since we have no Report Spam
complains (we're registered with feedbackloop.yahoo.net and we take
care instant of all Report Spam complaints - we unsubscribe the
email), i presume it's not the IP Block.
>
> - are you stuck with a <tld> country-code that has a reputation as a
> highly probable spam source?
Not at all. We're spam free.
>
> - Is all else about your message headers, content, and attachments
> 'proper' and of a sort above suspicion?
Everything is 100% perfect. @see test (private email).
>
> Feel free to send me a direct test message and I'll see what the
> 'scores' are in my logs - we're generally more strict than Yahoo, AND
> rather more verbose w/r logging.
Test is out on the way.
> Bill Hacker
In general the test scenario is pretty simple.
What is the behavior of Exim in the following situation ?
You have 1000 emails to be delivered to domain: test.tld.
test.tld has one MX: mx.test.tld mx.test.tld is mapped to ONE IP Address.
test.tld does greylisting on a per message basis.
They KEY for greylisting is: SENDER + RECIPIENT.
The greylisting requires minimum 5 minutes of delay.
In Exim after the first attempt to deliver to recipient:
user1@???, will get a 4xx ERROR (temporary defer).
Exim will add in it's retry DB that host:
mx.test.tld is off and it will set the next retry to run in 5 minutes
for example.
In this queue run process, sending to user2@??? will not even be
attempted because mx.test.tld is marked for retry in 5 minutes.
Doing a simple math, this will take Exim at least 5 minutes / EMAIL
for delivery => 5000 minutes.
If the retry database would have as key also the local part
(retry_use_local_part would work also for remote transport), exim will
try to deliver at first all 1000 emails and will set for each email
the retry schedule after 5 minutes.
Doing again a simple math, this will take Exim something between 5 -
10 minutes to deliver all 1000 emails.
Compare 10 with 5000.
I hope i made the scenario pretty clear.
Thank you for your time.
--
Catalin Constantin
Dazoot Software
http://www.dazoot.eu/
