On Thu, 3 Jul 2003, Avleen Vig wrote:
> While working on the next release of Virtual Exim[1], Rossz and I
> stumbled across this little bit of knowledge.. it's odd, maybe someone
> else can explain it? I can only speculate:
>
> Did a little fiddling, and this works on Exim 4.20:
> VIRTUAL_DOMAINS = SELECT DISTINCT concat(domain, ' : ') from <table>
>
> domainlist local_domains = localhost : ${lookup mysql{VIRTUAL_DOMAINS}}
It doesn't seem odd to me. :-) As documented, all domain/host/address
lists are expanded before they are used. That means they can be
dynamically generated by lookups (or any other means), which is exactly
what you are doing. The result of the expansion must be a
colon-separated list.
> Mysql returns the results, on per line.
> Normally you have to put a \ at the end of each line, so Exim knows that
> it should continue searching for a match on the next line.
The \ continuation facility is only for *actual* lines in the
configuration file. When Exim reads the file, it removes the backslashes
and concatenates the lines, before it does any interpretation. Thus,
backslashing does not apply within any subsequently generated data.
> In fact, putting a ":\" at the end of the line actually broke it, and
> caused Exim to think the result was a negative rather than a positive
> match.
It would have treated the backslash as part of the data, and no doubt
got confused, because backslash is interpreted by the string
interpreter. Embedded NL characters, however, will be treated as white
space.
> Something else I just realised:
> If the domains are specified in the config file, Exim will cache the
> results for the next time that questions is asked, eg:
>
> >>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> >>> routing lists-exim@???
> >>> silverwraith.com in "! +local_domains"? no (matched "! +local_domains" - cached)
>
> But it appears this caching isn't happening on replied given by the
> database? If anything, this SHOULD be done, to reduce the number of
> calls to the database, which could end up being expensive is your
> database is slow, or across a network.
There are two different issues here:
(1) Caching of lookups;
(2) Caching of the results of matching against a named list.
1. Caching of lookups:
For lookups, Exim caches only the result of the most recent query for a
given file or database. This goes back to the very early days of Exim,
when there were only lsearch and dbm-type lookups. A single-entry cache
works well for simple cases (e.g. checking "domains" in several
successive routers). I have not re-visited this code for a long time.
I would expect the result of the db lookup to be cached in this sense.
The debug output should show a message of the form "cached data used for
lookup of...". However, if there were other mysql lookups in between,
you won't see this (owing to the one-levelness of Exim's cache).
Exim does cache the connection to the server. I suppose Exim could
maintain a larger cache of queries, but how large should it be? I think
I might argue that a better place for this caching is in the database
server itself (compare DNS name servers).
2. Caching of matches against a named list
This is what you are seeing in the debug output you quoted. If, while
routing an address, the domain is matched against a named list such as
+local_domains, the result is cached, BUT only if the string that makes
up the domain list contains no expansion items. It would be wrong to
cache strings that can potentially be different every time. That is why
this type of caching is not happening when you are using a lookup to
generate the domainlist. At that level, all Exim sees is "match domain
to this list; this list contains a $ so it might be different each
time; therefore don't cache".
I hope this helps.
--
Philip Hazel University of Cambridge Computing Service,
ph10@??? Cambridge, England. Phone: +44 1223 334714.
Get the Exim 4 book: http://www.uit.co.uk/exim-book
