ph10 2004/11/19 15:18:57 GMT
Modified files:
exim-doc/doc-misc WishList
exim-doc/doc-txt ChangeLog NewStuff
exim-src/src/lookups dnsdb.c
exim-test-orig/AutoTest/scripts 082 571
exim-test-orig/AutoTest/stdout 082
Log:
Additions to dnsdb lookups: (a) list of domains (b) change output
separator.
Revision Changes Path
1.7 +0 -7 exim/exim-doc/doc-misc/WishList
1.35 +10 -2 exim/exim-doc/doc-txt/ChangeLog
1.13 +37 -0 exim/exim-doc/doc-txt/NewStuff
1.3 +158 -91 exim/exim-src/src/lookups/dnsdb.c
1.6 +11 -0 exim/exim-test-orig/AutoTest/scripts/082
1.4 +4 -1 exim/exim-test-orig/AutoTest/scripts/571
1.6 +14 -0 exim/exim-test-orig/AutoTest/stdout/082
Index: WishList
===================================================================
RCS file: /home/cvs/exim/exim-doc/doc-misc/WishList,v
retrieving revision 1.6
retrieving revision 1.7
diff -u -r1.6 -r1.7
--- WishList 9 Nov 2004 09:32:58 -0000 1.6
+++ WishList 19 Nov 2004 15:18:57 -0000 1.7
@@ -1,4 +1,4 @@
-$Cambridge: exim/exim-doc/doc-misc/WishList,v 1.6 2004/11/09 09:32:58 ph10 Exp $
+$Cambridge: exim/exim-doc/doc-misc/WishList,v 1.7 2004/11/19 15:18:57 ph10 Exp $
EXIM 4 WISH LIST
----------------
@@ -1411,13 +1411,6 @@
possibility to have multiple conditions in routers. It will be great to also
be able to set variables in routers like in acl's." This is effectively a
radical suggestion for a complete re-design, and is therefore BIG.
-------------------------------------------------------------------------------
-
-(222) 19-Dec-03 S Iterative option for dnsdb
-
-A way of getting a dnsdb lookup to chop off components until something is
-found: e.g. ${lookup dndsb-i{ns=a.b.c.d}} would look for nameservers for
-a.b.c.d, then b.c.d, etc.
------------------------------------------------------------------------------
(223) 22-Dec-03 S Support SOA lookup in dnsdb lookups
Index: ChangeLog
===================================================================
RCS file: /home/cvs/exim/exim-doc/doc-txt/ChangeLog,v
retrieving revision 1.34
retrieving revision 1.35
diff -u -r1.34 -r1.35
--- ChangeLog 19 Nov 2004 09:45:54 -0000 1.34
+++ ChangeLog 19 Nov 2004 15:18:57 -0000 1.35
@@ -1,4 +1,4 @@
-$Cambridge: exim/exim-doc/doc-txt/ChangeLog,v 1.34 2004/11/19 09:45:54 ph10 Exp $
+$Cambridge: exim/exim-doc/doc-txt/ChangeLog,v 1.35 2004/11/19 15:18:57 ph10 Exp $
Change log file for Exim from version 4.21
-------------------------------------------
@@ -149,8 +149,16 @@
per-process caching. But the chance of anyone hitting this buglet seems
very small.
-37. The dnsdb lookup has a new type, "zns", which walks up the domain tree
- until it finds some nameserver records. It should be used with care.
+37. The dnsdb lookup has been extended in a number of ways.
+
+ (1) There is a new type, "zns", which walks up the domain tree until it
+ finds some nameserver records. It should be used with care.
+
+ (2) It is now possible to give a list of domains (or IP addresses) to be
+ looked up.
+
+ (3) It is now possible to specify the separator character for use when
+ multiple records are returned.
Exim version 4.43
Index: NewStuff
===================================================================
RCS file: /home/cvs/exim/exim-doc/doc-txt/NewStuff,v
retrieving revision 1.12
retrieving revision 1.13
diff -u -r1.12 -r1.13
--- NewStuff 19 Nov 2004 09:45:54 -0000 1.12
+++ NewStuff 19 Nov 2004 15:18:57 -0000 1.13
@@ -1,4 +1,4 @@
-$Cambridge: exim/exim-doc/doc-txt/NewStuff,v 1.12 2004/11/19 09:45:54 ph10 Exp $
+$Cambridge: exim/exim-doc/doc-txt/NewStuff,v 1.13 2004/11/19 15:18:57 ph10 Exp $
New Features in Exim
--------------------
@@ -128,6 +128,43 @@
servers for a given domain are on a blacklist. You can probably assume that
the name servers for the high-level domains such as .com or .co.uk are not
going to be on such a list.
+
+13. It is now possible to specify a list of domains or IP addresses to be
+ looked up in a dnsdb lookup. The list is specified in the normal Exim way,
+ with colon as the default separator, but with the ability to change this.
+ For example:
+
+ ${lookup dnsdb{one.domain.com:two.domain.com}}
+ ${lookup dnsdb{a=one.host.com:two.host.com}}
+ ${lookup dnsdb{ptr = <; 1.2.3.4 ; 4.5.6.8}}
+
+ In order to retain backwards compatibility, there is one special case: if
+ the lookup type is PTR and no change of separator is specified, Exim looks
+ to see if the rest of the string is precisely one IPv6 address. In this
+ case, it does not treat it as a list.
+
+ The data from each lookup is concatenated, with newline separators (by
+ default - see 14 below), in the same way that multiple DNS records for a
+ single item are handled.
+
+ The lookup fails only if all the DNS lookups fail. As long as at least one
+ of them yields some data, the lookup succeeds. However, if there is a
+ temporary DNS error for any of them, the lookup defers.
+
+14. It is now possible to specify the character to be used as a separator when
+ a dnsdb lookup returns data from more than one DNS record. The default is a
+ newline. To specify a different character, put '>' followed by the new
+ character at the start of the query. For example:
+
+ ${lookup dnsdb{>: a=h1.test.ex:h2.test.ex}}
+ ${lookup dnsdb{>| mx=<;m1.test.ex;m2.test.ex}}
+
+ It is permitted to specify a space as the separator character. Note that
+ more than one DNS record can be found for a single lookup item; this
+ feature is relevant even when you do not specify a list.
+
+ The same effect could be achieved by wrapping the lookup in ${tr...}; this
+ feature is just a syntactic simplification.
Version 4.43
Index: dnsdb.c
===================================================================
RCS file: /home/cvs/exim/exim-src/src/lookups/dnsdb.c,v
retrieving revision 1.2
retrieving revision 1.3
diff -u -r1.2 -r1.3
--- dnsdb.c 19 Nov 2004 09:45:54 -0000 1.2
+++ dnsdb.c 19 Nov 2004 15:18:57 -0000 1.3
@@ -1,4 +1,4 @@
-/* $Cambridge: exim/exim-src/src/lookups/dnsdb.c,v 1.2 2004/11/19 09:45:54 ph10 Exp $ */
+/* $Cambridge: exim/exim-src/src/lookups/dnsdb.c,v 1.3 2004/11/19 15:18:57 ph10 Exp $ */
/*************************************************
* Exim - an Internet mail transport agent *
@@ -77,7 +77,20 @@
* Find entry point for dnsdb *
*************************************************/
-/* See local README for interface description. */
+/* See local README for interface description. The query in the "keystring" may
+consist of a number of parts.
+
+(a) If the first significant character is '>' then the next character is the
+separator character that is used when multiple records are found. The default
+separator is newline.
+
+(b) If the next sequence of characters is a sequence of letters and digits
+followed by '=', it is interpreted as the name of the DNS record type. The
+default is "A".
+
+(c) Then there follows list of domain names. This is a generalized Exim list,
+which may start with '<' in order to set a specific separator. The default
+separator, as always, is colon. */
int
dnsdb_find(void *handle, uschar *filename, uschar *keystring, int length,
@@ -86,9 +99,10 @@
int rc;
int size = 256;
int ptr = 0;
+int sep = 0;
int type = T_TXT;
-uschar *orig_keystring = keystring;
-uschar *equals = Ustrchr(keystring, '=');
+uschar *outsep = US"\n";
+uschar *equals, *domain;
uschar buffer[256];
/* Because we're the working in the search pool, we try to reclaim as much
@@ -105,12 +119,26 @@
length = length;
do_cache = do_cache;
-/* If the keystring contains an = this is preceded by a type name. */
+/* If the string starts with '>' we change the output separator */
+
+while (isspace(*keystring)) keystring++;
+if (*keystring == '>')
+ {
+ outsep = keystring + 1;
+ keystring += 2;
+ while (isspace(*keystring)) keystring++;
+ }
+
+/* If the keystring contains an = this must be preceded by a valid type name. */
-if (equals != NULL)
+if ((equals = Ustrchr(keystring, '=')) != NULL)
{
- int i;
- int len = equals - keystring;
+ int i, len;
+ uschar *tend = equals;
+
+ while (tend > keystring && isspace(tend[-1])) tend--;
+ len = tend - keystring;
+
for (i = 0; i < sizeof(type_names)/sizeof(uschar *); i++)
{
if (len == Ustrlen(type_names[i]) &&
@@ -120,111 +148,150 @@
break;
}
}
+
if (i >= sizeof(type_names)/sizeof(uschar *))
{
*errmsg = US"unsupported DNS record type";
return DEFER;
}
- keystring += len + 1;
- }
-
-/* If the type is PTR, we have to construct the relevant magic lookup
-key. This code is now in a separate function. */
-
-if (type == T_PTR)
- {
- dns_build_reverse(keystring, buffer);
- keystring = buffer;
+
+ keystring = equals + 1;
+ while (isspace(*keystring)) keystring++;
}
-
-DEBUG(D_lookup) debug_printf("dnsdb key: %s\n", keystring);
-
-/* Initialize the resolver, in case this is the first time it is used
-in this run. Then do the lookup and sort out the result. */
+
+/* Initialize the resolver in case this is the first time it has been used. */
dns_init(FALSE, FALSE);
-rc = dns_special_lookup(&dnsa, keystring, type, NULL);
-
-if (rc == DNS_NOMATCH || rc == DNS_NODATA) return FAIL;
-if (rc != DNS_SUCCEED) return DEFER;
-
-/* If the lookup was a pseudo-type, change it to the correct type for searching
-the returned records; then search for them. */
-
-if (type == T_ZNS) type = T_NS;
-for (rr = dns_next_rr(&dnsa, &dnss, RESET_ANSWERS);
- rr != NULL;
- rr = dns_next_rr(&dnsa, &dnss, RESET_NEXT))
- {
- if (rr->type != type) continue;
-
- /* There may be several addresses from an A6 record. Put newlines between
- them, just as for between several records. */
- if (type == T_A ||
- #ifdef SUPPORT_A6
- type == T_A6 ||
- #endif
- type == T_AAAA)
+/* The remainder of the string must be a list of domains. As long as the lookup
+for at least one of them succeeds, we return success. Failure means that none
+of them were found.
+
+The original implementation did not support a list of domains. Adding the list
+feature is compatible, except in one case: when PTR records are being looked up
+for a single IPv6 address. Fortunately, we can hack in a compatibility feature
+here: If the type is PTR and no list separator is specified, and the entire
+remaining string is valid as an IP address, set an impossible separator so that
+it is treated as one item. */
+
+if (type == T_PTR && keystring[0] != '<' &&
+ string_is_ip_address(keystring, NULL) > 0)
+ sep = -1;
+
+/* Now scan the list and do a lookup for each item */
+
+while ((domain = string_nextinlist(&keystring, &sep, buffer, sizeof(buffer)))
+ != NULL)
+ {
+ uschar rbuffer[256];
+
+ /* If the type is PTR, we have to construct the relevant magic lookup
+ key. This code is now in a separate function. */
+
+ if (type == T_PTR)
{
- dns_address *da;
- for (da = dns_address_from_rr(&dnsa, rr); da != NULL; da = da->next)
- {
- if (ptr != 0) yield = string_cat(yield, &size, &ptr, US"\n", 1);
- yield = string_cat(yield, &size, &ptr, da->address, Ustrlen(da->address));
- }
- continue;
- }
-
- /* Other kinds of record just have one piece of data each. */
-
- if (ptr != 0) yield = string_cat(yield, &size, &ptr, US"\n", 1);
-
- if (type == T_TXT)
- {
- yield = string_cat(yield, &size, &ptr, (uschar *)(rr->data+1),
- (rr->data)[0]);
+ dns_build_reverse(domain, rbuffer);
+ domain = rbuffer;
}
- else /* T_CNAME, T_MX, T_NS, T_PTR */
+
+ DEBUG(D_lookup) debug_printf("dnsdb key: %s\n", domain);
+
+ /* Do the lookup and sort out the result. We use the special
+ lookup function that knows about pseudo types like "zns". If the lookup
+ fails, continue with the next domain. */
+
+ rc = dns_special_lookup(&dnsa, domain, type, NULL);
+
+ if (rc == DNS_NOMATCH || rc == DNS_NODATA) continue;
+ if (rc != DNS_SUCCEED) return DEFER;
+
+ /* If the lookup was a pseudo-type, change it to the correct type for
+ searching the returned records; then search for them. */
+
+ if (type == T_ZNS) type = T_NS;
+ for (rr = dns_next_rr(&dnsa, &dnss, RESET_ANSWERS);
+ rr != NULL;
+ rr = dns_next_rr(&dnsa, &dnss, RESET_NEXT))
{
- uschar s[264];
- uschar *p = (uschar *)(rr->data);
- if (type == T_MX)
+ if (rr->type != type) continue;
+
+ /* There may be several addresses from an A6 record. Put the configured
+ separator between them, just as for between several records. However, A6
+ support is not normally configured these days. */
+
+ if (type == T_A ||
+ #ifdef SUPPORT_A6
+ type == T_A6 ||
+ #endif
+ type == T_AAAA)
{
- int num;
- GETSHORT(num, p); /* pointer is advanced */
- sprintf(CS s, "%d ", num);
- yield = string_cat(yield, &size, &ptr, s, Ustrlen(s));
+ dns_address *da;
+ for (da = dns_address_from_rr(&dnsa, rr); da != NULL; da = da->next)
+ {
+ if (ptr != 0) yield = string_cat(yield, &size, &ptr, outsep, 1);
+ yield = string_cat(yield, &size, &ptr, da->address,
+ Ustrlen(da->address));
+ }
+ continue;
}
- else if (type == T_SRV)
+
+ /* Other kinds of record just have one piece of data each, but there may be
+ several of them, of course. */
+
+ if (ptr != 0) yield = string_cat(yield, &size, &ptr, outsep, 1);
+
+ if (type == T_TXT)
{
- int num, weight, port;
- GETSHORT(num, p); /* pointer is advanced */
- GETSHORT(weight, p);
- GETSHORT(port, p);
- sprintf(CS s, "%d %d %d ", num, weight, port);
- yield = string_cat(yield, &size, &ptr, s, Ustrlen(s));
+ yield = string_cat(yield, &size, &ptr, (uschar *)(rr->data+1),
+ (rr->data)[0]);
}
- rc = dn_expand(dnsa.answer, dnsa.answer + dnsa.answerlen, p,
- (DN_EXPAND_ARG4_TYPE)(s), sizeof(s));
-
- /* If an overlong response was received, the data will have been
- truncated and dn_expand may fail. */
-
- if (rc < 0)
+ else /* T_CNAME, T_MX, T_NS, T_SRV, T_PTR */
{
- log_write(0, LOG_MAIN, "host name alias list truncated for %s",
- orig_keystring);
- break;
+ uschar s[264];
+ uschar *p = (uschar *)(rr->data);
+ if (type == T_MX)
+ {
+ int num;
+ GETSHORT(num, p); /* pointer is advanced */
+ sprintf(CS s, "%d ", num);
+ yield = string_cat(yield, &size, &ptr, s, Ustrlen(s));
+ }
+ else if (type == T_SRV)
+ {
+ int num, weight, port;
+ GETSHORT(num, p); /* pointer is advanced */
+ GETSHORT(weight, p);
+ GETSHORT(port, p);
+ sprintf(CS s, "%d %d %d ", num, weight, port);
+ yield = string_cat(yield, &size, &ptr, s, Ustrlen(s));
+ }
+ rc = dn_expand(dnsa.answer, dnsa.answer + dnsa.answerlen, p,
+ (DN_EXPAND_ARG4_TYPE)(s), sizeof(s));
+
+ /* If an overlong response was received, the data will have been
+ truncated and dn_expand may fail. */
+
+ if (rc < 0)
+ {
+ log_write(0, LOG_MAIN, "host name alias list truncated: type=%s "
+ "domain=%s", dns_text_type(type), domain);
+ break;
+ }
+ else yield = string_cat(yield, &size, &ptr, s, Ustrlen(s));
}
- else yield = string_cat(yield, &size, &ptr, s, Ustrlen(s));
- }
- }
+ } /* Loop for list of returned records */
+ } /* Loop for list of domains */
+/* Reclaim unused memory */
+
+store_reset(yield + ptr + 1);
+
+/* If ptr == 0 we have not found anything. Otherwise, insert the terminating
+zero and return the result. */
+
+if (ptr == 0) return FAIL;
yield[ptr] = 0;
-store_reset(yield + ptr + 1); /* Reclaim unused */
*result = yield;
-
return OK;
}
Index: 082
===================================================================
RCS file: /home/cvs/exim/exim-test-orig/AutoTest/scripts/082,v
retrieving revision 1.5
retrieving revision 1.6
diff -u -r1.5 -r1.6
--- 082 19 Nov 2004 09:45:54 -0000 1.5
+++ 082 19 Nov 2004 15:18:57 -0000 1.6
@@ -441,9 +441,20 @@
ptr=131.111.10.206 ${lookup dnsdb{ptr=131.111.10.206}{$value}fail}
a6=xoanon.ipv6.test.ex ${lookup dnsdb {a6=xoanon.ipv6.test.ex}{$value}{fail}}
a6=prefix.ipv6.test.ex ${lookup dnsdb {a6=prefix.ipv6.test.ex}{$value}{fail}}
+ptr=fe80:0:12:1:a00:20ff:fe86:a062 ${lookup dnsdb {ptr=<;fe80:0:12:1:a00:20ff:fe86:a062}{$value}{fail}}
ptr=fe80:0:12:1:a00:20ff:fe86:a062 ${lookup dnsdb {ptr=fe80:0:12:1:a00:20ff:fe86:a062}{$value}{fail}}
srv=_smtp._tcp.srv01.test.ex ${lookup dnsdb{srv=_smtp._tcp.srv01.test.ex}{$value}{fail}}
srv=_smtp._tcp.nosmtp.test.ex ${lookup dnsdb{srv=_smtp._tcp.nosmtp.test.ex}{$value}{fail}}
+
+# DNS lookups with multiple items
+
+ten-1:ten2 ${lookup dnsdb{a=ten-1.test.ex:ten-2.test.ex}}
+ten-1|ten2 ${lookup dnsdb{a=<|ten-1.test.ex|ten-2.test.ex}}
+mxt1;mxt2 | output ${lookup dnsdb{>|mx=<;mxt1.test.ex;mxt2.test.ex}}
+13/14 rbl ${lookup dnsdb{> 13.12.11.10.rbl.test.ex:13.12.11.10.rbl.test.ex}}
+double ptr ${lookup dnsdb{ptr=10.0.0.1:10.0.0.2}}
+ten-1:bad ${lookup dnsdb{ a = ten-1.test.ex:badbad.test.ex}{$value}{FAILED}}
+bad:bad ${lookup dnsdb{ a = <; badbad.test.ex ; badbad.test.ex}{$value}{FAILED}}
# Extract
Index: 571
===================================================================
RCS file: /home/cvs/exim/exim-test-orig/AutoTest/scripts/571,v
retrieving revision 1.3
retrieving revision 1.4
diff -u -r1.3 -r1.4
--- 571 17 Nov 2004 14:32:25 -0000 1.3
+++ 571 19 Nov 2004 15:18:57 -0000 1.4
@@ -183,10 +183,13 @@
killdaemon
****
0
+sleep 1
+****
+0
exim -DSERVER=server -DSUBMISSION_OPTIONS=/domain=a.b.c/sender_retain -bd -oX 1225
****
0
-sleep 2
+sleep 1
****
0
client 127.0.0.1 1225
Index: 082
===================================================================
RCS file: /home/cvs/exim/exim-test-orig/AutoTest/stdout/082,v
retrieving revision 1.5
retrieving revision 1.6
diff -u -r1.5 -r1.6
--- 082 19 Nov 2004 09:45:54 -0000 1.5
+++ 082 19 Nov 2004 15:18:57 -0000 1.6
@@ -438,8 +438,22 @@
> Failed: lookup of "a6=xoanon.ipv6.test.ex" gave DEFER: unsupported DNS record type
> Failed: lookup of "a6=prefix.ipv6.test.ex" gave DEFER: unsupported DNS record type
> ptr=fe80:0:12:1:a00:20ff:fe86:a062 testptr-arpa.ipv6.test.ex
+> ptr=fe80:0:12:1:a00:20ff:fe86:a062 testptr-arpa.ipv6.test.ex
> srv=_smtp._tcp.srv01.test.ex 0 0 25 ten-1.test.ex
> srv=_smtp._tcp.nosmtp.test.ex 0 0 0
+>
+> # DNS lookups with multiple items
+>
+> ten-1:ten2 10.0.0.1
+10.0.0.2
+> ten-1|ten2 10.0.0.1
+10.0.0.2
+> mxt1;mxt2 | output 5 xoanon.csi.cam.ac.uk|5 not-exist.test.ex
+> 13/14 rbl This is a test blacklisting message This is a test blacklisting message
+> double ptr ten-1.test.ex
+ten-2.test.ex
+> ten-1:bad 10.0.0.1
+> bad:bad FAILED
>
> # Extract
>
