Re: [exim] Verify outgoing email while using a smarthost

Top Page
Delete this message
Reply to this message
Author: Mike Brudenell
Date:  
To: Exim Users
CC: Jason
Subject: Re: [exim] Verify outgoing email while using a smarthost
Hi, Jason -

If a *dnslookup* router decides it can't handle an address it declines (as
described in the *Specification*) and control passes to the next router.
This could be useful for, say, a host with limited Internet connectivity:
its DNS lookups fail but it can then fall through to a *manualroute* router
and send the message off to a host with better connectivity.

If you want a *dnslookup* router to handle "all remaining non-local
domains" and not decline/fall through to the next router you must set the
*no_more* option on it. (See *17.2. Declining addresses by dnslookup
<http://www.exim.org/exim-html-current/doc/html/spec_html/ch-the_dnslookup_router.html#SECTdnslookupdecline>*)
This in effect turns the router's *decline* into a *fail*. (See *3.10 Running
an individual router
<http://www.exim.org/exim-html-current/doc/html/spec_html/ch-how_exim_receives_and_delivers_mail.html#SECTrunindrou>*
)


Would you be better off doing your cleanup earlier, in the ACLs, rather
than trying to force it into the routers? If you do then you can refuse to
accept that recipient address when the message is first trying to enter
your server, as against accepting it and then trying to get rid of it.

For example, perhaps something like this *untested* entry within your
*acl_smtp_rcpt* ACL?

require verify = recipient


1. The *require* says that its conditions *must* be met otherwise it
will reject the recipient address. (I guess you could do "defer ! verify =
recipient" instead if you really wanted to, with the risks Lena points out.)

2. The *verify = recipient* says that the recipient address must verify
by your routers in order to succeed. And you've already got working routers
to do that, I think: your *dnslookup* router is used only during
verification and can check the MX hosts look sensible, but during delivery
you instead throw the message at your smarthost using your
*manualroute* router
for actual delivery.

Note that by default verification only checks out the domain part of the
address; you'd have to add */callout* if you wanted to to a forward
callout to the far mail server and get it to confirm the localpart of the
address is valid too.

*Hmm…*

I've just seen your update saying that the messages *aren't* arriving over
SMTP so that the acl_smtp_rcpt ACL isn't any use to you… True, so is there
any way of you doing the check in an *acl_not_smtp* ACL instead? (Although
do note that the *Specification* says, "Any kind of rejection is treated as
permanent, because there is no way of sending a temporary error for these
kinds of message.")

I need to head home shortly, but will keep pondering; no doubt others might
get back to you in the meantime.


*By the way…*

If you're cleaning up your outbound mail make sure you read up and use
*ignore_target_hosts* as well in your *dnslookup* router. Some rogue
domains point their MX/A/AAAA records to loopback (eg, 0 or 127.0.0.1) or
private IP addresses, so they appear to have entries but in fact they're of
no use. So you want to filter those out from any entries your
*dnslookup* router
retrieves to check.

For example:

*…Main configuration…*
hostlist private_ip_addrs = <; 10.0.0.0/8 ; 172.16.0.0/12 ; 192.168.0.0/16
hostlist loopback_ip_addrs = <; 0.0.0.0 ; 127.0.0.0/8 ; ::1

*…Routers…*
verify_host:

driver = dnslookup

ignore_target_hosts = +loopback_ip_addrs : +private_ip_addrs



Cheers,
Mike B-)

On 3 August 2016 at 18:29, Jason <silo82@???> wrote:

> Thanks, Mike. I'll check out the lookup. So from the way you described
> it, when sending mail (not testing with -bv or -bt), it hits the
> "verify_address" router, does the check, and would fail, but it still
> continues to the next router ("smarthost" in my case) which delivers the
> message? Shouldn't the verify_address router failure halt/defer the
> message and not send it onto the next router? If not, is that possible in
> the router configuration (without a lookup)? What purpose does a
> verify_only router serve if it doesn't defer/fail a message from being
> processed further?
>
> The reason we want to do the check before forwarding to the relay is
> because our relay is Amazon SES, and if we relay mail that's destined to
> NXDOMAINs, for example, then it will be a bounce and you get penalized for
> submitting mail that bounces (we're trying to cleanup email addresses that
> are in our database that are entered incorrectly, but for now, they're
> there). So we're trying to deal with the "bad" mail before it even gets to
> SES.
>
> Thanks,
> -- Jason
>
> On Wed, Aug 3, 2016 at 1:04 PM, Mike Brudenell <mike.brudenell@???>
> wrote:
>
>> Hi,
>>
>> Exim is doing exactly what your routers are currently written to tell it
>> do…
>>
>> Your *verify_address* router is set (by the *verify_only*) to be used
>> only when verifying an address: for example, as you're doing when you use
>> the "-bv" command line option. If the DNS lookup says that the domain
>> doesn't exist then the router declines to handle the message and it falls
>> through to the next router.
>>
>> The next router is *smarthost* which is set not to be used when
>> verifying (because you have given it *no_verify*). So when you're
>> verifying there's no later router that handles the address being verified
>> so you get the "Unrouteable address" result.
>>
>> *However* when you're doing it for real then when *verify_address* finds
>> a domain that doesn't exist in the DNS and declines, the *smarthost* router
>> *does* run and do exactly what you tell it: send the message over to
>> your smarthost. (I've not used *host_find_failed* but suspect it applies
>> if Exim can't find IP addresses for the smarthost(s) you've defined in
>> *route_list*, not for the recipient address' domain.)
>>
>> As it says in at the start of the *Specification* when describing a
>> *manualroute* router (emphasis by underlining mine)…
>>
>> The *manualroute* router is so-called because it provides a way of
>> manually routing an address according to its domain. It is mainly used when
>> you want to route addresses to remote hosts according to your own rules, *bypassing
>> the normal DNS routing that looks up MX records*.
>>
>>
>> I have a feeling that you trying to check in your *smarthost* router whether
>> the MX records for the recipient address' domains are valid might be
>> unusual; it's easy to argue that should be the job of the smarthost rather
>> than yourself!
>>
>> But if you're determined to do it then presumably you could use a
>>
>> *${lookup dnsdb{mx=$domain}{$value}fail}*
>>
>> (Someone else might be able to suggest an easier way of doing the check,
>> but this could be one way.)
>>
>> Read the chapter *File and database lookups
>> <http://www.exim.org/exim-html-current/doc/html/spec_html/ch-file_and_database_lookups.html>*
>> in the *Specification* for more on theis and other types of lookup.
>>
>> Cheers,
>> Mike B-)
>>
>> On 3 August 2016 at 14:10, Jason <silo82@???> wrote:
>>
>>> Hi,
>>> Is it possible for Exim to verify outgoing mail when using a smarthost?
>>> Specifically, I'm looking to have Exim verify (using dnslookup) that the
>>> domain exists for the recipient. If the domain returns NXDOMAIN (or some
>>> other DNS error), defer it, if DNS exists, continue with the routing. I
>>> tried the following:
>>>
>>> # Verify the recipient domain exists in DNS before sending to the relay.
>>> verify_address:
>>> debug_print = "R: verify_address for $domain"
>>> driver = dnslookup
>>> domains = ! +local_domains
>>> verify_only
>>> no_more
>>>
>>>
>>> smarthost:
>>> debug_print = "R: smarthost for $local_part@$domain"
>>> driver = manualroute
>>> domains = ! +local_domains
>>> transport = remote_smtp_smarthost
>>> route_list = * DCsmarthost byname
>>> host_find_failed = defer
>>> same_domain_copy_routing = yes
>>> no_verify
>>> no_more
>>>
>>> And it seems to work during testing:
>>>
>>> # exim -bv someone@???
>>> someone@??? failed to verify: Unrouteable
>>> address
>>> # exim -bv someone@???
>>> someone@??? verified
>>>
>>>
>>> However, when sending a test message to an NXDOMAIN, exim happily
>>> forwards
>>> it to the smarthost:
>>>
>>> 2016-08-03 08:07:28 1bUvtQ-0007bm-Vh <= root@somehost U=root P=local
>>> S=486
>>> T="test"
>>> 2016-08-03 08:07:29 1bUvtQ-0007bm-Vh =>
>>> someone@??? R=smarthost
>>> T=remote_smtp_smarthost
>>> S=505 H=ses-smtp-prod-335357831.us-east-1.elb.amazonaws.com
>>> [50.19.94.229]
>>> X=TLS1.2:DHE_RSA_AES_128_CBC_SHA1:128
>>> DN="C=US,ST=Washington,L=Seattle,O=Amazon.com\, Inc.,CN=
>>> email-smtp.us-east-1.amazonaws.com"
>>> 2016-08-03 08:07:29 1bUvtQ-0007bm-Vh Completed
>>>
>>> Is it possible for Exim to do outbound verification while using a
>>> smarthost? If so, what am I missing?
>>>
>>> Thanks
>>> --
>>> ## List details at https://lists.exim.org/mailman/listinfo/exim-users
>>> ## Exim details at http://www.exim.org/
>>> ## Please use the Wiki with this list - http://wiki.exim.org/
>>
>>
>>
>>
>> --
>> Systems Administrator & Change Manager
>> IT Services, University of York, Heslington, York YO10 5DD, UK
>> Tel: +44-(0)1904-323811
>>
>> Web: www.york.ac.uk/it-services
>> Disclaimer: www.york.ac.uk/docs/disclaimer/email.htm
>>
>
>



--
Systems Administrator & Change Manager
IT Services, University of York, Heslington, York YO10 5DD, UK
Tel: +44-(0)1904-323811

Web: www.york.ac.uk/it-services
Disclaimer: www.york.ac.uk/docs/disclaimer/email.htm