Gitweb:
http://git.exim.org/exim.git/commitdiff/e7568d513f9b5a853e5cdb5db8b2cc0d53f79981
Commit: e7568d513f9b5a853e5cdb5db8b2cc0d53f79981
Parent: 9471b034a78d53e4bfb951649e9f556cf4e6018c
Author: Todd Lyons <tlyons@???>
AuthorDate: Thu Jul 26 13:31:20 2012 -0700
Committer: Todd Lyons <tlyons@???>
CommitDate: Fri Jul 27 12:52:46 2012 -0700
Bug #198: Add remove_header ACL modifier.
Used patch from Magnus Holmgren dated 2007-02-20.
Added documentation.
Added tests to detect proper operation.
---
doc/doc-docbook/spec.xfpt | 85 +++++++++++++++++++++++++++++++++++++--
doc/doc-txt/ChangeLog | 3 +
doc/doc-txt/NewStuff | 3 +
src/src/acl.c | 39 ++++++++++++++++++
src/src/globals.c | 1 +
src/src/globals.h | 1 +
src/src/receive.c | 34 ++++++++++++++++
src/src/smtp_in.c | 1 +
test/confs/0567 | 91 ++++++++++++++++++++++++++++++++++++++++++
test/log/0567 | 10 +++++
test/mail/0567.rcptok | 45 +++++++++++++++++++++
test/rejectlog/0567 | 2 +
test/scripts/0000-Basic/0567 | 49 ++++++++++++++++++++++
test/stdout/0567 | 8 ++++
14 files changed, 368 insertions(+), 4 deletions(-)
diff --git a/doc/doc-docbook/spec.xfpt b/doc/doc-docbook/spec.xfpt
index 19fb321..1b5c947 100644
--- a/doc/doc-docbook/spec.xfpt
+++ b/doc/doc-docbook/spec.xfpt
@@ -26508,8 +26508,8 @@ duplicates to be written, use the &%logwrite%& modifier instead.
If &%log_message%& is not present, a &%warn%& verb just checks its conditions
and obeys any &"immediate"& modifiers (such as &%control%&, &%set%&,
-&%logwrite%&, and &%add_header%&) that appear before the first failing
-condition. There is more about adding header lines in section
+&%logwrite%&, &%add_header%&, and &%remove_header%&) that appear before the
+first failing condition. There is more about adding header lines in section
&<<SECTaddheadacl>>&.
If any condition on a &%warn%& statement cannot be completed (that is, there is
@@ -26951,6 +26951,12 @@ all the conditions are true, wherever it appears in an ACL command, whereas
effect.
+.vitem &*remove_header*&&~=&~<&'text'&>
+This modifier specifies one or more header names in a colon-separated list
+ that are to be removed from an incoming message, assuming, of course, that
+the message is ultimately accepted. For details, see section &<<SECTremoveheadacl>>&.
+
+
.vitem &*set*&&~<&'acl_name'&>&~=&~<&'value'&>
.cindex "&%set%& ACL modifier"
This modifier puts a value into one of the ACL variables (see section
@@ -27265,7 +27271,7 @@ Remotely submitted, fixups applied: use &`control = submission`&.
.section "Adding header lines in ACLs" "SECTaddheadacl"
.cindex "header lines" "adding in an ACL"
.cindex "header lines" "position of added lines"
-.cindex "&%message%& ACL modifier"
+.cindex "&%add_header%& ACL modifier"
The &%add_header%& modifier can be used to add one or more extra header lines
to an incoming message, as in this example:
.code
@@ -27307,7 +27313,7 @@ passing data between (for example) the MAIL and RCPT ACLs. If you want to do
this, you can use ACL variables, as described in section
&<<SECTaclvariables>>&.
-The &%add_header%& modifier acts immediately it is encountered during the
+The &%add_header%& modifier acts immediately as it is encountered during the
processing of an ACL. Notice the difference between these two cases:
.display
&`accept add_header = ADDED: some text`&
@@ -27356,6 +27362,77 @@ system filter or in a router or transport.
+.section "Removing header lines in ACLs" "SECTremoveheadacl"
+.cindex "header lines" "removing in an ACL"
+.cindex "header lines" "position of removed lines"
+.cindex "&%remove_header%& ACL modifier"
+The &%remove_header%& modifier can be used to remove one or more header lines
+from an incoming message, as in this example:
+.code
+warn message = Remove internal headers
+ remove_header = x-route-mail1 : x-route-mail2
+.endd
+The &%remove_header%& modifier is permitted in the MAIL, RCPT, PREDATA, DATA,
+MIME, and non-SMTP ACLs (in other words, those that are concerned with
+receiving a message). The message must ultimately be accepted for
+&%remove_header%& to have any significant effect. You can use &%remove_header%&
+with any ACL verb, including &%deny%&, though this is really not useful for
+any verb that doesn't result in a delivered message.
+
+More than one header can be removed at the same time by using a colon separated
+list of header names. The header matching is case insensitive. Wildcards are
+not permitted, nor is list expansion performed, so you cannot use hostlists to
+create a list of headers, however both connection and message variable expansion
+are performed (&%$acl_c_*%& and &%$acl_m_*%&), illustrated in this example:
+.code
+warn hosts = +internal_hosts
+ set acl_c_ihdrs = x-route-mail1 : x-route-mail2
+warn message = Remove internal headers
+ remove_header = $acl_c_ihdrs
+.endd
+Removed header lines are accumulated during the MAIL, RCPT, and predata ACLs.
+They are removed from the message before processing the DATA and MIME ACLs.
+There is no harm in attempting to remove the same header twice nor is removing
+a non-existent header. Further header lines to be removed may be accumulated
+during the DATA and MIME ACLs, after which they are removed from the message,
+if present. In the case of non-SMTP messages, headers to be removed are
+accumulated during the non-SMTP ACLs, and are removed from the message after
+all the ACLs have run. If a message is rejected after DATA or by the non-SMTP
+ACL, there really is no effect because there is no logging of what headers
+would have been removed.
+
+.cindex "header lines" "removed; visibility of"
+Header lines are not visible in string expansions until the DATA phase when it
+is received. Any header lines removed in the MAIL, RCPT, and predata ACLs are
+not visible in the DATA ACL and MIME ACLs. Similarly, header lines that are
+removed by the DATA or MIME ACLs are still visible in those ACLs. Because of
+this restriction, you cannot use header lines as a way of controlling data
+passed between (for example) the MAIL and RCPT ACLs. If you want to do this,
+you should instead use ACL variables, as described in section
+&<<SECTaclvariables>>&.
+
+The &%remove_header%& modifier acts immediately as it is encountered during the
+processing of an ACL. Notice the difference between these two cases:
+.display
+&`accept remove_header = X-Internal`&
+&` `&<&'some condition'&>
+
+&`accept `&<&'some condition'&>
+&` remove_header = X-Internal`&
+.endd
+In the first case, the header line is always removed, whether or not the
+condition is true. In the second case, the header line is removed only if the
+condition is true. Multiple occurrences of &%remove_header%& may occur in the
+same ACL statement. All those that are encountered before a condition fails
+are honoured.
+
+&*Warning*&: This facility currently applies only to header lines that are
+present during ACL processing. It does NOT remove header lines that are added
+in a system filter or in a router or transport.
+
+
+
+
.section "ACL conditions" "SECTaclconditions"
.cindex "&ACL;" "conditions; list of"
diff --git a/doc/doc-txt/ChangeLog b/doc/doc-txt/ChangeLog
index 108f605..2021a2a 100644
--- a/doc/doc-txt/ChangeLog
+++ b/doc/doc-txt/ChangeLog
@@ -35,6 +35,9 @@ TL/01 Bugzilla 1258 - Refactor MAIL FROM optional args processing.
TL/02 Add +smtp_confirmation as a default logging option.
+TL/03 Bugzilla 198 - Implement remove_header ACL modifier.
+ Patch by Magnus Holmgren from 2007-02-20.
+
JH/01 Bugzilla 1201 & 304 - New cutthrough-delivery feature, with TLS support.
JH/02 Support "G" suffix to numbers in ${if comparisons.
diff --git a/doc/doc-txt/NewStuff b/doc/doc-txt/NewStuff
index e684344..94307c8 100644
--- a/doc/doc-txt/NewStuff
+++ b/doc/doc-txt/NewStuff
@@ -109,6 +109,9 @@ Version 4.81
11. Routers and transports can now have multiple headers_add and headers_remove
option lines. The concatenated list is used.
+12. New ACL modifier "remove_header" can remove headers before message gets
+ handled by routers/transports.
+
Version 4.80
------------
diff --git a/src/src/acl.c b/src/src/acl.c
index 5cd0c35..3b23a91 100644
--- a/src/src/acl.c
+++ b/src/src/acl.c
@@ -88,6 +88,7 @@ enum { ACLC_ACL,
#ifdef WITH_CONTENT_SCAN
ACLC_REGEX,
#endif
+ ACLC_REMOVE_HEADER,
ACLC_SENDER_DOMAINS,
ACLC_SENDERS,
ACLC_SET,
@@ -150,6 +151,7 @@ static uschar *conditions[] = {
#ifdef WITH_CONTENT_SCAN
US"regex",
#endif
+ US"remove_header",
US"sender_domains", US"senders", US"set",
#ifdef WITH_CONTENT_SCAN
US"spam",
@@ -280,6 +282,7 @@ static uschar cond_expand_at_top[] = {
#ifdef WITH_CONTENT_SCAN
TRUE, /* regex */
#endif
+ TRUE, /* remove_header */
FALSE, /* sender_domains */
FALSE, /* senders */
TRUE, /* set */
@@ -340,6 +343,7 @@ static uschar cond_modifiers[] = {
#ifdef WITH_CONTENT_SCAN
FALSE, /* regex */
#endif
+ TRUE, /* remove_header */
FALSE, /* sender_domains */
FALSE, /* senders */
TRUE, /* set */
@@ -465,6 +469,12 @@ static unsigned int cond_forbids[] = {
(1<<ACL_WHERE_MIME)),
#endif
+ (unsigned int)
+ ~((1<<ACL_WHERE_MAIL)|(1<<ACL_WHERE_RCPT)| /* remove_header */
+ (1<<ACL_WHERE_PREDATA)|(1<<ACL_WHERE_DATA)|
+ (1<<ACL_WHERE_MIME)|(1<<ACL_WHERE_NOTSMTP)|
+ (1<<ACL_WHERE_NOTSMTP_START)),
+
(1<<ACL_WHERE_AUTH)|(1<<ACL_WHERE_CONNECT)| /* sender_domains */
(1<<ACL_WHERE_HELO)|
(1<<ACL_WHERE_MAILAUTH)|(1<<ACL_WHERE_QUIT)|
@@ -1037,6 +1047,31 @@ for (p = q = hstring; *p != 0; )
+/*************************************************
+* Set up removed header line(s) *
+*************************************************/
+
+/* This function is called by the remove_header modifier. The argument is
+treated as a sequence of header names which are added to a colon separated
+list, provided there isn't an identical one already there.
+
+Argument: string of header names
+Returns: nothing
+*/
+
+static void
+setup_remove_header(uschar *hnames)
+{
+if (*hnames != 0)
+ {
+ if (acl_removed_headers == NULL)
+ acl_removed_headers = hnames;
+ else
+ acl_removed_headers = string_sprintf("%s : %s", acl_removed_headers, hnames);
+ }
+}
+
+
/*************************************************
* Handle warnings *
@@ -3294,6 +3329,10 @@ for (; cb != NULL; cb = cb->next)
break;
#endif
+ case ACLC_REMOVE_HEADER:
+ setup_remove_header(arg);
+ break;
+
case ACLC_SENDER_DOMAINS:
{
uschar *sdomain;
diff --git a/src/src/globals.c b/src/src/globals.c
index 21122f0..bcbe12d 100644
--- a/src/src/globals.c
+++ b/src/src/globals.c
@@ -196,6 +196,7 @@ uschar *acl_not_smtp = NULL;
uschar *acl_not_smtp_mime = NULL;
#endif
uschar *acl_not_smtp_start = NULL;
+uschar *acl_removed_headers = NULL;
uschar *acl_smtp_auth = NULL;
uschar *acl_smtp_connect = NULL;
uschar *acl_smtp_data = NULL;
diff --git a/src/src/globals.h b/src/src/globals.h
index 783eb7b..16caa41 100644
--- a/src/src/globals.h
+++ b/src/src/globals.h
@@ -142,6 +142,7 @@ extern uschar *acl_not_smtp; /* ACL run for non-SMTP messages */
extern uschar *acl_not_smtp_mime; /* For MIME parts of ditto */
#endif
extern uschar *acl_not_smtp_start; /* ACL run at the beginning of a non-SMTP session */
+extern uschar *acl_removed_headers; /* Headers deleted by an ACL */
extern uschar *acl_smtp_auth; /* ACL run for AUTH */
extern uschar *acl_smtp_connect; /* ACL run on SMTP connection */
extern uschar *acl_smtp_data; /* ACL run after DATA received */
diff --git a/src/src/receive.c b/src/src/receive.c
index 636913b..7b51805 100644
--- a/src/src/receive.c
+++ b/src/src/receive.c
@@ -936,6 +936,40 @@ add_acl_headers(uschar *acl_name)
{
header_line *h, *next;
header_line *last_received = NULL;
+int sep = ':';
+
+if (acl_removed_headers != NULL)
+ {
+ DEBUG(D_receive|D_acl) debug_printf(">>Headers removed by %s ACL:\n", acl_name);
+
+ for (h = header_list; h != NULL; h = h->next)
+ {
+ int i;
+ uschar *list;
+ BOOL include_header;
+
+ if (h->type == htype_old) continue;
+
+ include_header = TRUE;
+ list = acl_removed_headers;
+
+ int sep = ':'; /* This is specified as a colon-separated list */
+ uschar *s;
+ uschar buffer[128];
+ while ((s = string_nextinlist(&list, &sep, buffer, sizeof(buffer)))
+ != NULL)
+ {
+ int len = Ustrlen(s);
+ if (header_testname(h, s, len, FALSE))
+ {
+ h->type = htype_old;
+ DEBUG(D_receive|D_acl) debug_printf(" %s", h->text);
+ }
+ }
+ }
+ acl_removed_headers = NULL;
+ DEBUG(D_receive|D_acl) debug_printf(">>\n");
+ }
if (acl_added_headers == NULL) return;
DEBUG(D_receive|D_acl) debug_printf(">>Headers added by %s ACL:\n", acl_name);
diff --git a/src/src/smtp_in.c b/src/src/smtp_in.c
index db7f133..b1fea9d 100644
--- a/src/src/smtp_in.c
+++ b/src/src/smtp_in.c
@@ -1041,6 +1041,7 @@ cancel_cutthrough_connection("smtp reset");
message_linecount = 0;
message_size = -1;
acl_added_headers = NULL;
+acl_removed_headers = NULL;
queue_only_policy = FALSE;
rcpt_smtp_response = NULL;
rcpt_smtp_response_same = TRUE;
diff --git a/test/confs/0567 b/test/confs/0567
new file mode 100644
index 0000000..7348b1f
--- /dev/null
+++ b/test/confs/0567
@@ -0,0 +1,91 @@
+# Exim test configuration 0532
+
+CONNECTCOND=
+
+exim_path = EXIM_PATH
+host_lookup_order = bydns
+primary_hostname = myhost.test.ex
+rfc1413_query_timeout = 0s
+spool_directory = DIR/spool
+log_file_path = DIR/spool/log/%slog
+gecos_pattern = ""
+gecos_name = CALLER_NAME
+
+# ----- Main settings -----
+
+acl_smtp_connect = connect
+acl_smtp_mail = mail
+acl_smtp_rcpt = rcpt
+acl_smtp_predata = predata
+acl_smtp_data = data
+acl_not_smtp = notsmtp
+
+qualify_domain = test.ex
+trusted_users = CALLER
+
+hostlist internal_headers = x-mail-2 : x-mail-3
+
+
+# ----- ACL -----
+
+begin acl
+
+connect:
+ accept CONNECTCOND
+
+mail:
+ accept remove_header = x-mail-1
+ senders = mailok@???
+ # Won't work because doesn't expand
+ remove_header = +internal_headers
+ accept
+
+rcpt:
+ accept local_parts = rcptok
+ remove_header = x-rcpt-4 : x-rcpt-2
+ set acl_m_hdr = x-predata-1
+ deny add_header = RCPT: denied $local_part
+
+
+predata:
+ warn remove_header = x-predata-3 : $acl_m_hdr
+ # Won't work because doesn't use wildcards
+ accept remove_header = x-not-*
+
+data:
+ warn log_message = Verified previously removed header X-Rcpt-2
+ condition = ${if eq{$h_x-rcpt-2:}{}}
+ warn remove_header = x-data-1 : x-data-4
+ condition = ${if eq{$h_cond:}{accept}}
+ remove_header = x-data-3
+ # Won't delete this header because condition fails before the modifier
+ warn condition = ${if eq{$h_cond:}{reject}}
+ remove_header = x-data-2
+ warn log_message = Verified removed header X-Data-3 in this ACL still visible
+ condition = ${if !eq{$h_x-data-3:}{}}
+ accept
+
+notsmtp:
+ # Will remove a required header (Date) if told to
+ accept remove_header = x-notsmtp-1 : date
+
+
+# ----- Routers -----
+
+begin routers
+
+r1:
+ driver = accept
+ transport = t1
+
+
+# ----- Transports -----
+
+begin transports
+
+t1:
+ driver = appendfile
+ file = DIR/test-mail/$local_part
+ user = CALLER
+
+# End
diff --git a/test/log/0567 b/test/log/0567
new file mode 100644
index 0000000..1bad60e
--- /dev/null
+++ b/test/log/0567
@@ -0,0 +1,10 @@
+1999-03-02 09:44:33 U=CALLER F=<mailok@???> rejected RCPT <notok@???>
+1999-03-02 09:44:33 10HmaX-0005vi-00 U=CALLER Warning: Verified previously removed header X-Rcpt-2
+1999-03-02 09:44:33 10HmaX-0005vi-00 U=CALLER Warning: Verified removed header X-Data-3 in this ACL still visible
+1999-03-02 09:44:33 10HmaX-0005vi-00 <= mailok@??? U=CALLER P=local-smtp S=sss
+1999-03-02 09:44:33 10HmaX-0005vi-00 => rcptok <rcptok@???> R=r1 T=t1
+1999-03-02 09:44:33 10HmaX-0005vi-00 Completed
+1999-03-02 09:44:33 10HmaY-0005vi-00 <= CALLER@??? U=CALLER P=local S=sss
+1999-03-02 09:44:33 10HmaY-0005vi-00 => rcptok <rcptok@???> R=r1 T=t1
+1999-03-02 09:44:33 10HmaY-0005vi-00 Completed
+1999-03-02 09:44:33 U=CALLER temporarily rejected connection in "connect" ACL: cannot use remove_header condition in connection ACL
diff --git a/test/mail/0567.rcptok b/test/mail/0567.rcptok
new file mode 100644
index 0000000..deffea8
--- /dev/null
+++ b/test/mail/0567.rcptok
@@ -0,0 +1,45 @@
+From mailok@??? Tue Mar 02 09:44:33 1999
+Received: from CALLER by myhost.test.ex with local-smtp (Exim x.yz)
+ (envelope-from <mailok@???>)
+ id 10HmaX-0005vi-00
+ for rcptok@???; Tue, 2 Mar 1999 09:44:33 +0000
+cond: accept
+X-Data-2: Line two
+X-Data-5: Line five
+X-Not-1: Testing wildcard one
+X-Not-2: Testing wildcard two
+X-Rcpt-1: Line six
+X-Rcpt-3: Line eight
+X-Rcpt-5: Line ten
+X-Mail-2: Line twelve
+X-Mail-3: Line thirteen
+X-Mail-4: Line fourteen is also really long, but it won't get
+ removed by these ACL's.
+X-Mail-5: Line fifteen
+X-Predata-5: Line sixteen
+X-Predata-4: Line seventeen
+X-Predata-2: Line nineteen
+X-NotSMTP-1: Line twenty-one
+X-NotSMTP-2: Line twenty-two
+X-NotSMTP-3: Line twenty-three
+Message-Id: <E10HmaX-0005vi-00@???>
+From: mailok@???
+Date: Tue, 2 Mar 1999 09:44:33 +0000
+RCPT: denied notok
+
+Test message
+
+From CALLER@??? Tue Mar 02 09:44:33 1999
+Received: from CALLER by myhost.test.ex with local (Exim x.yz)
+ (envelope-from <CALLER@???>)
+ id 10HmaY-0005vi-00
+ for rcptok@???; Tue, 2 Mar 1999 09:44:33 +0000
+Message-Id: <E10HmaY-0005vi-00@???>
+From: CALLER_NAME <CALLER@???>
+
+Test non-SMTP message. Make sure it doesn't blow up when a header
+it wants to remove is not present. This one also overrides the
+fixup of adding a Date header because we specified to remove it!
+Allow the admin to shoot himself in the foot if he really and
+truly wants to.
+
diff --git a/test/rejectlog/0567 b/test/rejectlog/0567
new file mode 100644
index 0000000..980e2e8
--- /dev/null
+++ b/test/rejectlog/0567
@@ -0,0 +1,2 @@
+1999-03-02 09:44:33 U=CALLER F=<mailok@???> rejected RCPT <notok@???>
+1999-03-02 09:44:33 U=CALLER temporarily rejected connection in "connect" ACL: cannot use remove_header condition in connection ACL
diff --git a/test/scripts/0000-Basic/0567 b/test/scripts/0000-Basic/0567
new file mode 100644
index 0000000..5abd06f
--- /dev/null
+++ b/test/scripts/0000-Basic/0567
@@ -0,0 +1,49 @@
+# remove_header modifier in ACLs
+exim -bs -odi
+mail from:<mailok@???>
+rcpt to:<rcptok@???>
+rcpt to:<notok@???>
+data
+cond: accept
+X-Data-1: Line one
+X-Data-2: Line two
+X-Data-3: Line three
+X-Data-4: Line four
+X-Data-5: Line five
+X-Not-1: Testing wildcard one
+X-Not-2: Testing wildcard two
+X-Rcpt-1: Line six
+X-Rcpt-2: Line seven
+X-Rcpt-3: Line eight
+X-Rcpt-4: Line nine is really long, so long in fact that it wraps
+ around to the next line.
+X-Rcpt-5: Line ten
+X-Mail-1: Line eleven
+X-Mail-2: Line twelve
+X-Mail-3: Line thirteen
+X-Mail-4: Line fourteen is also really long, but it won't get
+ removed by these ACL's.
+X-Mail-5: Line fifteen
+X-Predata-5: Line sixteen
+X-Predata-4: Line seventeen
+X-Predata-3: Line eighteen
+X-Predata-2: Line nineteen
+X-Predata-1: Line twenty
+X-NotSMTP-1: Line twenty-one
+X-NotSMTP-2: Line twenty-two
+X-NotSMTP-3: Line twenty-three
+
+Test message
+.
+quit
+****
+exim -odi rcptok@???
+Test non-SMTP message. Make sure it doesn't blow up when a header
+it wants to remove is not present. This one also overrides the
+fixup of adding a Date header because we specified to remove it!
+Allow the admin to shoot himself in the foot if he really and
+truly wants to.
+****
+exim -bs -odi -DCONNECTCOND="remove_header=CONNECT: won't do this"
+****
+no_msglog_check
diff --git a/test/stdout/0567 b/test/stdout/0567
new file mode 100644
index 0000000..65fd23c
--- /dev/null
+++ b/test/stdout/0567
@@ -0,0 +1,8 @@
+220 myhost.test.ex ESMTP Exim x.yz Tue, 2 Mar 1999 09:44:33 +0000
+250 OK
+250 Accepted
+550 Administrative prohibition
+354 Enter message, ending with "." on a line by itself
+250 OK id=10HmaX-0005vi-00
+221 myhost.test.ex closing connection
+451 Temporary local problem - please try later
