[exim] Re: dmarc_history_file - incomplete data logged for s…

Top Page
Delete this message
Reply to this message
Author: Victor Ustugov
Date:  
To: Mackenzie Taiaroa via Exim-users
Subject: [exim] Re: dmarc_history_file - incomplete data logged for spf and dkim
Mackenzie Taiaroa via Exim-users wrote on 19.08.2023 6:31:
> Good day Victor,


Hello

> *Building exim with ARC support does not solve the "arc" and
> "arc_policy"fields problem.*
>
> Good to know.
>
>
>
> *patch-src__exim-4.96-fix-opemdmarc-1.4.x-history_file.patch just outputsa
> DKIM selector to the file to fix problems with the
> dmarc_history_fileformat.*
>
> Maybe this continues to be a problem here, I've applied the patch but
> there's no DKIM selector in the history file:
>
> reporter server.hostname.com.au
> received 1692234968
> ipaddr 209.85.215.171
> from gmail.com
> mfrom gmail.com
> spf 0
> dkim gmail.com 0


Check DMARC_API in the Local/Makefile

My patch adds the output of the selector to the dmarc_history_file when
the value is great or equal to 100400.


> pdomain gmail.com
> policy 15
> rua mailto:mailauth-reports@google.com
> pct 100
> adkim 114
> aspf 114
> p 110
> sp 113
> align_dkim 4
> align_spf 4
> action 2
>
> The below error occurs when opendmarc-import tries importing the dkim
> values into the database:


It doesn't matter if there is no DKIM selector in the line of the
dmarc_history_file starting with "dkim".


> -- Unit dmarc-report.service has begun starting up.
> Aug 19 12:11:16 server.hostname.com.au dmarc-report.sh[2843936]: Moving
> opendmarc.dat for import...
> Aug 19 12:11:16 server.hostname.com.au dmarc-report.sh[2843936]: New
> opendmarc.dat initialized, reporting...
> Aug 19 12:11:16 server.hostname.com.au dmarc-report.sh[2843939]:
> opendmarc-import: started at Sat Aug 19 12:11:16 2023
> Aug 19 12:11:16 server.hostname.com.au dmarc-report.sh[2843939]:
> opendmarc-import: connected to database
> Aug 19 12:11:16 server.hostname.com.au dmarc-report.sh[2843939]: Use of
> uninitialized value $dkim_result in string eq at /usr/sbin/opendmarc-import
> line 637, <STDIN> line 8.
> Aug 19 12:11:16 server.hostname.com.au dmarc-report.sh[2843939]:
> opendmarc-import: updating at line 20
> Aug 19 12:11:16 server.hostname.com.au dmarc-report.sh[2843939]:
> opendmarc-import: failed to insert DKIM data: Column 'pass' cannot be null
> Aug 19 12:11:16 server.hostname.com.au dmarc-report.sh[2843939]: Use of
> uninitialized value $dkim_result in string eq at /usr/sbin/opendmarc-import
> line 637, <STDIN> line 27.
> Aug 19 12:11:16 server.hostname.com.au dmarc-report.sh[2843939]:
> opendmarc-import: updating at line 39
> Aug 19 12:11:16 server.hostname.com.au dmarc-report.sh[2843939]:
> opendmarc-import: failed to insert DKIM data: Column 'pass' cannot be null
> Aug 19 12:11:16 server.hostname.com.au dmarc-report.sh[2843939]: Use of
> uninitialized value $dkim_result in string eq at /usr/sbin/opendmarc-import
> line 637, <STDIN> line 46.
> Aug 19 12:11:16 server.hostname.com.au dmarc-report.sh[2843939]:
> opendmarc-import: updating at line 58
> Aug 19 12:11:17 server.hostname.com.au dmarc-report.sh[2843939]:
> opendmarc-import: failed to insert DKIM data: Column 'pass' cannot be null
> Aug 19 12:11:17 server.hostname.com.au dmarc-report.sh[2843939]: Use of
> uninitialized value $dkim_result in string eq at /usr/sbin/opendmarc-import
> line 637, <STDIN> line 65.
> Aug 19 12:11:17 server.hostname.com.au dmarc-report.sh[2843939]:
> opendmarc-import: updating at line 76
> Aug 19 12:11:17 server.hostname.com.au dmarc-report.sh[2843939]:
> opendmarc-import: failed to insert DKIM data: Column 'pass' cannot be null
> Aug 19 12:11:17 server.hostname.com.au dmarc-report.sh[2843939]:
> opendmarc-import: terminating at Sat Aug 19 12:11:17 2023
> Aug 19 12:11:18 server.hostname.com.au dmarc-report.sh[2843941]:
> opendmarc-reports: started at Sat Aug 19 12:11:17 2023
> Aug 19 12:11:18 server.hostname.com.au dmarc-report.sh[2843941]:
> opendmarc-reports: selected 9 domain(s)
> Aug 19 12:11:19 server.hostname.com.au dmarc-report.sh[2843941]:
> opendmarc-reports: terminating at Sat Aug 19 12:11:19 2023
> Aug 19 12:11:19 server.hostname.com.au dmarc-report.sh[2843943]:
> opendmarc-expire: started at Sat Aug 19 12:11:19 2023
> Aug 19 12:11:19 server.hostname.com.au dmarc-report.sh[2843943]:
> opendmarc-expire: connected to database
> Aug 19 12:11:19 server.hostname.com.au dmarc-report.sh[2843943]:
> opendmarc-expire: expiring messages older than 180 day(s)
> Aug 19 12:11:19 server.hostname.com.au dmarc-report.sh[2843943]:
> opendmarc-expire: expiring signatures on expired messages (id < 1)
> Aug 19 12:11:19 server.hostname.com.au dmarc-report.sh[2843943]:
> opendmarc-expire: expiring arcauthresults on expired messages (id < 1)
> Aug 19 12:11:19 server.hostname.com.au dmarc-report.sh[2843943]:
> opendmarc-expire: expiring arcseals on expired messages (id < 1)
> Aug 19 12:11:19 server.hostname.com.au dmarc-report.sh[2843943]:
> opendmarc-expire: expiring request data older than 180 days
> Aug 19 12:11:19 server.hostname.com.au dmarc-report.sh[2843943]:
> opendmarc-expire: terminating at Sat Aug 19 12:11:19 2023
> Aug 19 12:11:19 server.hostname.com.au dmarc-report.sh[2843943]:
> opendmarc-expire: no rows deleted
> Aug 19 12:11:19 server.hostname.com.au dmarc-report.sh[2843936]: DMARC
> reporting finished.
> Aug 19 12:11:19 server.hostname.com.au systemd[1]: dmarc-report.service:
> Succeeded.
>
> *Line 637 of /usr/sbin/opendmarc-import is the 6th line below:*
>
>           case "dkim"           {
>                                         my @dkim_entry;
>                                         push(@dkim_entry, $value);
>                                         push(@dkim_entry, $dkim_selector);
>                                         push(@dkim_entry, $dkim_result);
>                                         if ($dkim_result eq "4" ||
>                                             $dkim_result eq "5")
>                                         {
>                                                 push(@dkim_entry, 1);
>                                         }
>                                         else
>                                         {
>                                                 push(@dkim_entry, 0);
>                                         }
>                                         push(@dkim_data, [ @dkim_entry ]);

>
>                                         $sigcount++;
>                                 }

>
> I successfully generated a report once, which was sent to Microsoft (
> live.co.uk) - unfortunately I don't have a copy of this, however the
> live.co.uk email received was not signed by DKIM so opendmarc-import
> succeeded and a report was generated. All other attempts to generate/send
> reports fail (without error), I suspect because the data set is incomplete
> for the domains where the import has failed so the data doesn't qualify for
> a report to be sent.
>
> Initially I considered I could have made an error when applying the patch,
> however I can see the patch is applied successfully when reviewing the mock
> build results log:
>
> Patch #10 (exim-4.96-fix-opemdmarc-1.4.x-history_file.patch):
> patching file src/dmarc.c
> Hunk #1 succeeded at 479 (offset 1 line).
> + echo 'Patch #8 (exim-4.96-CVE-2022-3620.patch):'
> + /usr/bin/patch --no-backup-if-mismatch -p1 -b --suffix .CVE-2022-3620
> --fuzz=0
> + echo 'Patch #9 (exim-4.96-malformed-address-exit-fix.patch):'
> + /usr/bin/patch --no-backup-if-mismatch -p1 -b --suffix
> .malformed-address-exit-fix --fuzz=0
> + echo 'Patch #10 (exim-4.96-fix-opemdmarc-1.4.x-history_file.patch):'
> + /usr/bin/patch --no-backup-if-mismatch -p1 -b --suffix
> .exim-4.96-fix-opemdmarc-1.4.x-history_file --fuzz=0


Check file src/dmarc.c after patching.

It should contain the following code snippet:


#if DMARC_API >= 100400
    dkim_history_buffer = string_sprintf("%sdkim %s %s %d\n",
dkim_history_buffer,
                                         sig->domain, sig->selector,
dkim_ares_result);
#else
    dkim_history_buffer = string_sprintf("%sdkim %s %d\n",
dkim_history_buffer,
                                         sig->domain, dkim_ares_result);
#endif



If the src/dmarc.c file contains such a piece of code, check the value
of DMARC_API. If you are using libopendmarc 1.4.x then you need the
value 100400.


> Thanks in advance for your help.
>
> All the best,
> Mackenzie
>



--
Best wishes Victor Ustugov
mailto:victor@corvax.kiev.ua
public GnuPG/PGP key: https://victor.corvax.kiev.ua/corvax.asc

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-users.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-users-unsubscribe@???
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/