Re: [exim] Help sought with fairly complex DKIM set up and F…

Top Page
Delete this message
Reply to this message
Author: Michael J. Tubby B.Sc. MBCS G8TIC
Date:  
To: Todd Lyons
CC: Exim User List
Subject: Re: [exim] Help sought with fairly complex DKIM set up and Facebook
Todd,

That's interesting, but my users are complaining that they get no
facebook notifications and facebook keeps telling my that my own email
address is invalid:

    http://www.tubby.org/facebook/broken_email.png


presumably because I am rejecting them, i.e. rejecting real facebook
email rather than bogus ones... this would suggest that facebook is broken?


Mike


On 28/02/2014 02:31, Todd Lyons wrote:
> I tend to think that you just happened to pick one of a few that
> failed. On my systems, since Sunday's logrotation, 0.1% if inbound
> messages had failed signatures:
>
> OVZ-CentOS63[root@ivlog52 ~]# grep facebookmail
> /disk1/log/exim/main.log | grep DKIM | wc -l
> 7086
> OVZ-CentOS63[root@ivlog52 ~]# grep facebookmail
> /disk1/log/exim/main.log | grep DKIM | grep -v "verification
> succeeded" | wc -l
> 9
>
> Upon further digging, every one of those 9 emails appear to have been
> forwarded through another mail server:
>
> OVZ-CentOS63[root@ivlog52 ~]# exigrep
> "d=facebookmail\.com.*verification failed" /disk1/log/exim/main.log |
> grep DMARC
> 2014-02-23 02:24:15 1WHOji-00028R-Or DMARC results:
> spf_domain=foxmail.com dmarc_domain=facebookmail.com spf_align=no
> dkim_align=no enforcement='Reject'
> 2014-02-23 02:24:15 1WHOji-00028R-Or H=smtpbg177.qq.com
> [119.147.194.228] Warning: Message from facebookmail.com failed
> sender's DMARC policy, would REJECT
>
> 2014-02-23 19:48:04 1WHf1s-0001tQ-Cs DMARC results:
> spf_domain=gmail.com dmarc_domain=facebookmail.com spf_align=no
> dkim_align=no enforcement='Reject'
> 2014-02-23 19:48:04 1WHf1s-0001tQ-Cs H=mail-bk0-f43.google.com
> [209.85.214.43] Warning: Message from facebookmail.com failed sender's
> DMARC policy, would REJECT
>
> 2014-02-24 03:49:21 1WHmXb-0005Kz-A2 DMARC results:
> spf_domain=foxmail.com dmarc_domain=facebookmail.com spf_align=no
> dkim_align=no enforcement='Reject'
> 2014-02-24 03:49:21 1WHmXb-0005Kz-A2 H=smtpbg177.qq.com
> [119.147.194.228] Warning: Message from facebookmail.com failed
> sender's DMARC policy, would REJECT
>
> 2014-02-24 20:11:18 1WI1rv-00037F-3W DMARC results:
> spf_domain=gmail.com dmarc_domain=facebookmail.com spf_align=no
> dkim_align=no enforcement='Reject'
> 2014-02-24 20:11:18 1WI1rv-00037F-3W H=mail-bk0-f43.google.com
> [209.85.214.43] Warning: Message from facebookmail.com failed sender's
> DMARC policy, would REJECT
>
> 2014-02-25 02:42:59 1WI7yx-0006TS-9J DMARC results:
> spf_domain=foxmail.com dmarc_domain=facebookmail.com spf_align=no
> dkim_align=no enforcement='Reject'
> 2014-02-25 02:42:59 1WI7yx-0006TS-9J H=smtpbg177.qq.com
> [119.147.194.228] Warning: Message from facebookmail.com failed
> sender's DMARC policy, would REJECT
>
> 2014-02-25 19:28:28 1WINg3-0000jb-94 DMARC results:
> spf_domain=gmail.com dmarc_domain=facebookmail.com spf_align=no
> dkim_align=no enforcement='Reject'
> 2014-02-25 19:28:28 1WINg3-0000jb-94 H=mail-bk0-f49.google.com
> [209.85.214.49] Warning: Message from facebookmail.com failed sender's
> DMARC policy, would REJECT
>
> 2014-02-26 03:23:47 1WIV5z-0005F9-55 DMARC results:
> spf_domain=foxmail.com dmarc_domain=facebookmail.com spf_align=no
> dkim_align=no enforcement='Reject'
> 2014-02-26 03:23:47 1WIV5z-0005F9-55 H=smtpbg175.qq.com
> [119.147.194.226] Warning: Message from facebookmail.com failed
> sender's DMARC policy, would REJECT
>
> 2014-02-27 02:58:18 1WIrAs-0007Xw-EV DMARC results:
> spf_domain=foxmail.com dmarc_domain=facebookmail.com spf_align=no
> dkim_align=no enforcement='Reject'
> 2014-02-27 02:58:18 1WIrAs-0007Xw-EV H=smtpbg175.qq.com
> [119.147.194.226] Warning: Message from facebookmail.com failed
> sender's DMARC policy, would REJECT
>
> 2014-02-27 19:55:48 1WJ73b-0004iO-Ad DMARC results:
> spf_domain=gmail.com dmarc_domain=facebookmail.com spf_align=no
> dkim_align=no enforcement='Reject'
> 2014-02-27 19:55:48 1WJ73b-0004iO-Ad H=mail-ea0-f182.google.com
> [209.85.215.182] Warning: Message from facebookmail.com failed
> sender's DMARC policy, would REJECT
>
> ...Todd
>
> On Thu, Feb 27, 2014 at 4:40 PM, Michael J. Tubby B.Sc. MBCS G8TIC
> <mike.tubby@???> wrote:
>> Exim fans,
>>
>> I run some mail relays for a few hundred domains that I look after and
>> want to perform fairly complex DKIM checking - I want to:
>>
>>     * enforce DKIM tests domains that are 'known signers' (google,
>> facebook, etc) and explicitly accept or deny mail based on the result of
>> the DKIM checks - to avoid faked email
>>     * allow through mail with no signatures (obvious)
>>     * support a 'DKIM whitelist' for domains that send with DKIM but
>> have a known fault/problem
>>     * skip checks on hosts/domains we relay for
>>     * skip checks on authenticated connections from MUAs (clients)
>>     * defer if a message that has a signature is not testable, eg.
>> cannot retrieve their DKIM key, key has syntax error, etc.

>>
>>
>> Systems are: Ubuntu 10.04 LTS 32-bit + Exim 4.82 built from source
>>
>>
>>
>> here's my DKIM ACL:
>>
>> ###
>> ### ack_check_dkim: this ACL is used for checking DKIM
>> ###
>>
>> #
>> # acl_m2 set to zero on start for normal/full checks, set to 1 if
>> white-listed
>> #
>>
>> acl_check_dkim:
>>
>>         #
>>         # start of DKIM debug message and clear macro
>>         #
>>         warn    set acl_m2 = 0
>>                 logwrite = DKIM START: domain=$sender_address_domain
>> possible_signer=$dkim_cur_signer status=$dkim_verify_status ${if
>> def:dkim_verify_reason {(reason=$dkim_verify_reason) }}

>>
>>
>>         #
>>         # strict checking on known signers...
>>         #
>>         deny    sender_domains = +dkim_known_signers
>> #               dkim_signers = +dkim_known_signers
>>                 dkim_status = none:invalid:fail
>>                 message = Message from $sender_address_domain (known
>> signer) with invalid or missing signature
>>                 logwrite = DKIM DENY: Rejected $sender_address_domain is
>> known signer (in database) but has invalid/missing signature

>>
>>         accept  sender_domains = +dkim_known_signers
>> #               dkim_signers = +dkim_known_signers
>>                 dkim_status = pass
>>                 logwrite = DKIM PASS: Accepted $sender_address_domain is
>> known signer and has good signature
>>                 add_header = :after_received:X-DKIM-Result:
>> Domain=$sender_address_domain Result=Good and Known Domain

>>
>>
>>         #
>>         # ignore noise where we have no signature
>>         #
>>         accept  dkim_status = none
>> #               logwrite = DKIM SKIP: Skipping DKIM checks - no
>> signature for: $dkim_cur_signer

>>
>>         #
>>         # skip DKIM if domain whitelisted for DKIM, i.e. known good
>> domain that has broken DKIM
>>         #
>>         accept  sender_domains = +dkim_whitelist_domains
>>                 logwrite = DKIM SKIP: Skipping DKIM checks for
>> whitelisted domain: $sender_address_domain
>>                 set acl_m2 = 1

>>
>>         #
>>         # skip DKIM checks on hosts we relay for
>>         #
>>         accept  hosts = +relay_from_hosts
>>                 logwrite = DKIM SKIP: Skipping DKIM checks for relay
>> host: $sender_fullhost

>>
>>
>>         #
>>         # skip DKIM checks on authenticated hosts (that we also relay for)
>>         #
>>         accept  authenticated = *
>>                 logwrite = DKIM SKIP: Skipping DKIM checks for
>> authenticated host: $sender_fullhost

>>
>>
>>         #
>>         # defer when message not testable, e.g. can't get public key, etc.
>>         #
>>         defer   dkim_status = invalid
>>                 message = Message from $sender_address_domain cannot be
>> verified
>>                 logwrite = DKIM DEFER: domain=$sender_address_domain

>>
>>         #
>>         # accept the message (correctly signed)
>>         #
>>         accept  dkim_status = pass
>>                 sender_domains = $sender_address_domain
>>                 dkim_signers = $sender_address_domain
>>                 logwrite = DKIM PASS: domain=$sender_address_domain
>> signer=$dkim_cur_signer status=$dkim_verify_status
>>                 add_header = :after_received:X-DKIM-Result:
>> Domain=$sender_address_domain Result=Signature OK

>>
>>         #
>>         # accept the message EVEN IF the signature FAILS! due to white
>> listing
>>         #
>>         accept  condition = ${if eq {$acl_m2}{1}}
>>                 dkim_status = fail
>>                 sender_domains = $sender_address_domain
>>                 dkim_signers = $sender_address_domain
>>                 logwrite = DKIM FAIL (WHITELISTED):
>> domain=$sender_address_domain status=$dkim_verify_status - DKIM failed
>> but message accepted
>>                 add_header = :after_received:X-DKIM-Result:
>> Domain=$sender_address_domain Result=FAIL (but whitelisted)

>>
>>         #
>>         # deny (strict) when message fails signature test *and* acl_m2 =
>> 0 (not whitelisted)
>>         #
>>         deny    condition = ${if eq {$acl_m2}{0}}
>>                 dkim_status = fail
>>                 sender_domains = $sender_address_domain
>>                 dkim_signers = $sender_address_domain
>>                 message = Message from has invalid DKIM signature
>>                 logwrite = DKIM FAIL (DENY):
>> domain=$sender_address_domain - message rejected!

>>
>>         #
>>         # accept anything else (should never get here)
>>         #
>>         accept  logwrite = DKIM DEFAULT: domain=$sender_address_domain -
>> message accepted (at end of ACL)

>>
>>
>> NB. hostlists and domainlists are read from MySQL tables and are in
>> standard exim form
>>
>>
>>
>>
>>
>> My setup works for the most of the time including Google/Gmail - they
>> are in my "known signers" list:
>>
>> 2014-02-27 23:52:09 CONNECT: Accepting connection from: 209.85.215.196 -
>> not blocked by any RBL
>> 2014-02-27 23:52:09 HELO: Accepted HELO/EHLO mail-ea0-f196.google.com
>> from remote host: 209.85.215.196 (mail-ea0-f196.google.com)
>> 2014-02-27 23:52:09 HELO: Accepted HELO/EHLO mail-ea0-f196.google.com
>> from remote host: 209.85.215.196 (mail-ea0-f196.google.com)
>> 2014-02-27 23:52:09 MAIL: SPF Result=pass (gmail.com /
>> mail-ea0-f196.google.com [209.85.215.196])
>> 2014-02-27 23:52:09 MAIL: Accept from: mike.tubby80@??? host:
>> mail-ea0-f196.google.com [209.85.215.196]
>> 2014-02-27 23:52:09 RCPT: SPF Result2=pass (gmail.com /
>> mail-ea0-f196.google.com [209.85.215.196])
>> 2014-02-27 23:52:09 1WJAkL-0002Ld-Ep DKIM: d=gmail.com s=20120113
>> c=relaxed/relaxed a=rsa-sha256 [verification succeeded]
>> 2014-02-27 23:52:09 1WJAkL-0002Ld-Ep DKIM START: domain=gmail.com
>> possible_signer=gmail.com status=pass
>> 2014-02-27 23:52:09 1WJAkL-0002Ld-Ep DKIM PASS: Accepted gmail.com is
>> known signer and has good signature
>> 2014-02-27 23:52:09 1WJAkL-0002Ld-Ep MIME: Type=multipart/alternative Size=1
>> 2014-02-27 23:52:09 1WJAkL-0002Ld-Ep MIME: Type=text/plain Size=1
>> 2014-02-27 23:52:09 1WJAkL-0002Ld-Ep MIME: Type=text/html Size=1
>> 2014-02-27 23:52:09 1WJAkL-0002Ld-Ep CONTENT: Start ACL with scan profile: 2
>> 2014-02-27 23:52:09 1WJAkL-0002Ld-Ep CONTENT: Couldn't verify HELO/EHLO
>> greeting (mail-ea0-f196.google.com) from remote host: 209.85.215.196
>> (mail-ea0-f196.google.com)
>> 2014-02-27 23:52:09 1WJAkL-0002Ld-Ep CONTENT: SPAM: Enabled in scan
>> profile (will test, reject at 5.0)
>> 2014-02-27 23:52:09 1WJAkL-0002Ld-Ep CONTENT: SPAM Score: -0.4 (/)
>> 2014-02-27 23:52:09 1WJAkL-0002Ld-Ep CONTENT: ClamAV: Enabled in scan
>> profile (will test)
>> 2014-02-27 23:52:09 1WJAkL-0002Ld-Ep CONTENT: Added custom header:
>> X-Scan-Signature: aee9e5eeb35c86f052d502ac97832558
>> 2014-02-27 23:52:09 1WJAkL-0002Ld-Ep CONTENT: Checks completed, content
>> accepted
>> 2014-02-27 23:52:09 1WJAkL-0002Ld-Ep <= mike.tubby80@???
>> H=mail-ea0-f196.google.com [209.85.215.196] P=esmtps X=TLSv1:RC4-SHA:128
>> S=3105
>> id=CAAnpCNJqpST7cjTLyw3m6gR2mhTZWjx_wdGsQu=UBCUD6pDmtA@???
>> T="gmail testing"
>>
>> Google are good guys!
>>
>>
>>
>> Site mrredonline.com are not in my "known signers" and appear to be broken:
>>
>> 2014-02-27 23:55:41 CONNECT: Accepting connection from: 178.33.94.52 -
>> not blocked by any RBL
>> 2014-02-27 23:55:41 HELO: Accepted HELO/EHLO ukb8mx4.mrredonline.com
>> from remote host: 178.33.94.52 (ukb8mx4.mrredonline.com)
>> 2014-02-27 23:55:41 MAIL: SPF Result=neutral (ukb8mx6.mrredonline.com /
>> ukb8mx4.mrredonline.com [178.33.94.52])
>> 2014-02-27 23:55:41 MAIL: Accept from: bounce@???
>> host: ukb8mx4.mrredonline.com [178.33.94.52]
>> 2014-02-27 23:55:41 RCPT: SPF Result2=neutral (ukb8mx6.mrredonline.com /
>> ukb8mx4.mrredonline.com [178.33.94.52])
>> 2014-02-27 23:55:41 1WJAnl-0002M4-4x DKIM: d=ukb8mx6.mrredonline.com
>> s=dkim c=relaxed/relaxed a=rsa-sha1 i=info@???
>> [invalid - public key record (currently?) unavailable]
>> 2014-02-27 23:55:41 1WJAnl-0002M4-4x DKIM START:
>> domain=ukb8mx6.mrredonline.com possible_signer=ukb8mx6.mrredonline.com
>> status=invalid (reason=pubkey_unavailable)
>> 2014-02-27 23:55:41 1WJAnl-0002M4-4x DKIM DEFER:
>> domain=ukb8mx6.mrredonline.com
>> 2014-02-27 23:55:41 1WJAnl-0002M4-4x H=ukb8mx4.mrredonline.com
>> [178.33.94.52] temporarily rejected DKIM : Message from
>> ukb8mx6.mrredonline.com cannot be verified
>>
>> which appears correct - they are a gambling site and appear to be
>> sending our a DKIM header, but probing them with ProtoDave's checker tool:
>>
>>     http://www.protodave.com/tools/dkim-key-checker/

>>
>> they don't have a public key under that selector... so I defer them...
>> seems appropriate to me... I will keep deferring them until they fix
>> their public key and then I might accept them!
>>
>>
>>
>> Amazon are not in my "known signers" and appear to be ok:
>>
>> 2014-02-28 00:01:02 CONNECT: Accepting connection from: 54.240.0.151 -
>> not blocked by any RBL
>> 2014-02-28 00:01:02 HELO: Accepted HELO/EHLO
>> a0-151.smtp-out.eu-west-1.amazonses.com from remote host: 54.240.0.151
>> (a0-151.smtp-out.eu-west-1.amazonses.com)
>> 2014-02-28 00:01:02 MAIL: SPF Result=pass (bounces.amazon.com /
>> a0-151.smtp-out.eu-west-1.amazonses.com [54.240.0.151])
>> 2014-02-28 00:01:02 MAIL: Accept from:
>> 20140228000100daea22bcd6364808b4c0b369d29f3840-C19ZNAY18YA6WZ@???
>> host: a0-151.smtp-out.eu-west-1.amazonses.com [54.240.0.151]
>> 2014-02-28 00:01:02 RCPT: SPF Result2=pass (bounces.amazon.com /
>> a0-151.smtp-out.eu-west-1.amazonses.com [54.240.0.151])
>> 2014-02-28 00:01:12 1WJAt6-0002NM-C8 DKIM: d=amazon.co.uk
>> s=kfypa4gzdotgdqwujmwyfqhv7hoigmat c=relaxed/simple a=rsa-sha256
>> t=1393545660 [verification succeeded]
>> 2014-02-28 00:01:12 1WJAt6-0002NM-C8 DKIM START:
>> domain=bounces.amazon.com possible_signer=amazon.co.uk status=pass
>> 2014-02-28 00:01:12 1WJAt6-0002NM-C8 DKIM DEFAULT:
>> domain=bounces.amazon.com - message accepted (at end of ACL)
>> 2014-02-28 00:01:12 1WJAt6-0002NM-C8 MIME: Type=multipart/mixed Size=47
>> 2014-02-28 00:01:12 1WJAt6-0002NM-C8 MIME: Type=multipart/alternative
>> Size=47
>> 2014-02-28 00:01:12 1WJAt6-0002NM-C8 MIME: Type=text/plain Size=2
>> 2014-02-28 00:01:12 1WJAt6-0002NM-C8 MIME: Type=text/html Size=42
>> 2014-02-28 00:01:12 1WJAt6-0002NM-C8 CONTENT: Start ACL with scan profile: 1
>> 2014-02-28 00:01:12 1WJAt6-0002NM-C8 CONTENT: Couldn't verify HELO/EHLO
>> greeting (a0-151.smtp-out.eu-west-1.amazonses.com) from remote host:
>> 54.240.0.151 (a0-151.smtp-out.eu-west-1.amazonses.com)
>> 2014-02-28 00:01:12 1WJAt6-0002NM-C8 CONTENT: Checks skipped: SPF
>> Whitelisted
>> 2014-02-28 00:01:12 1WJAt6-0002NM-C8 <=
>> 20140228000100daea22bcd6364808b4c0b369d29f3840-C19ZNAY18YA6WZ@???
>> H=a0-151.smtp-out.eu-west-1.amazonses.com [54.240.0.151] P=esmtp S=49226
>> id=0000014475cb4934-183da1b1-d8b2-4c51-9d5c-70409cd1b646-000000@???
>> T="Feb 28: Today's Deal of the Day"
>>
>> if they are know DKIM signing everything then - perhaps I should
>> elevate them to "known signer" status?
>>
>>
>>
>> Paddy Power are not in my "known signers", but the DKIM header is found:
>>
>> 2014-02-27 23:45:28 CONNECT: Accepting connection from: 89.21.232.58 -
>> not blocked by any RBL
>> 2014-02-27 23:45:28 HELO: Accepted HELO/EHLO
>> mail232-58.send.smartfocusdigital.net from remote host: 89.21.232.58
>> (mail232-58.send.smartfocusdigital.net)
>> 2014-02-27 23:45:28 MAIL: Accept from: sports@???
>> host: mail232-58.send.smartfocusdigital.net [89.21.232.58]
>> 2014-02-27 23:45:28 1WJAds-0002J9-84 DKIM: d=ppmail.paddypower.com
>> s=shared_key c=relaxed/relaxed a=rsa-sha1 i=sports@???
>> [invalid - public key record (currently?) unavailable]
>> 2014-02-27 23:45:28 1WJAds-0002J9-84 DKIM START:
>> domain=ppmail.paddypower.com possible_signer=ppmail.paddypower.com
>> status=invalid (reason=pubkey_unavailable)
>> 2014-02-27 23:45:28 1WJAds-0002J9-84 DKIM DEFER:
>> domain=ppmail.paddypower.com
>> 2014-02-27 23:45:28 1WJAds-0002J9-84
>> H=mail232-58.send.smartfocusdigital.net [89.21.232.58] temporarily
>> rejected DKIM : Message from ppmail.paddypower.com cannot be verified
>>
>> but they appear to have no public key?
>>
>>
>>
>> And the killer one... Facebook... they are in my "known signers" but
>> appear to be broken:
>>
>> 2014-02-27 10:30:16 MAIL: SPF Result=pass (facebookmail.com /
>> outmail016.ash2.facebook.com (mx-out.facebook.com) [66.220.155.150])
>> 2014-02-27 10:30:16 MAIL: Accept from:
>> notification+kjdmd_m7uvpd@??? host:
>> outmail016.ash2.facebook.com (mx-out.facebook.com) [66.220.155.150]
>> 2014-02-27 10:30:16 RCPT: SPF Result2=pass (facebookmail.com /
>> outmail016.ash2.facebook.com (mx-out.facebook.com) [66.220.155.150])
>> 2014-02-27 10:30:16 1WIyEK-0006vE-Fw DKIM: d=facebookmail.com
>> s=s1024-2013-q3 c=relaxed/simple a=rsa-sha256 t=1393497014 [verification
>> failed - signature did not verify (headers probably modified in transit)]
>> 2014-02-27 10:30:16 1WIyEK-0006vE-Fw DKIM START: domain=facebookmail.com
>> possible_signer=facebookmail.com status=fail (reason=signature_incorrect)
>> 2014-02-27 10:30:16 1WIyEK-0006vE-Fw DKIM DENY: Rejected
>> facebookmail.com is known signer (in database) but has invalid/missing
>> signature
>> 2014-02-27 10:30:16 1WIyEK-0006vE-Fw H=outmail016.ash2.facebook.com
>> (mx-out.facebook.com) [66.220.155.150] rejected DKIM : Message from
>> facebookmail.com (known signer) with invalid or missing signature
>>
>> am I the only person having problems with Facebook?
>>
>>
>>
>>
>> Questions:
>>
>> * is there anything wrong with my design or implementation?
>>
>> * are there any suggestions for improvements?
>>
>> * specifically in the case of faceboomail.com do I have something broken
>> or is it them?
>>
>> * do I really need to whitelist facebook as a broken DKIM sender to get
>> their mail in?
>>
>>
>>
>> Regards
>>
>>
>> Mike Tubby
>>
>>
>>
>> --
>> ## List details at https://lists.exim.org/mailman/listinfo/exim-users
>> ## Exim details at http://www.exim.org/
>> ## Please use the Wiki with this list - http://wiki.exim.org/
>
>