--replying to multiple messages - wrapped up in one reply--
Thanks for all of the suggestions and information! Here are some follow
up thoughts:
(1) I already totally understand about how/why this can be done via
pointing the DNS to a locally server, such as BIND - and then - *if*
BIND is forwarding all queries to google - there can be a "conditional
forwarder" to force the BIND to treat specific types of queries
differently. HOWEVER I always (mistakenly!) thought that this required
pointing that conditional forwarder to a specific IP - so I didn't know
that having that part empty - would get BIND to revert THOSE queries
back to using root hints and then send the query to authoritative
servers - so that new knowledge is helpful. (Thanks Lena!)
(2) I also understand the benefit of caching to DNSBLs. This isn't quite
as big of a deal as it seems... because quality DNSBLs only have a TTL
that lasts for a few minutes, at most! But this is still an issue since
caching still helps in rapid fire situations - where multiple messages
are sent in close succession to multiple users. While this is a valid
point - the problem I'm trying to solve is such a pain sometimes - that
this lack-of-caching issue can often be a "lesser of evils".
(3) Stepping back a moment to look at "the big picture" - Sadly, it is
getting to the point where about 95+% of all email - is handled by about
a dozen large tech companies - and (I'm guessing that...) 50+% is
handled by about 6 tech companies. (Microsoft, Google, Godaddy,
Proofpoint, Mimecast, Sophos) Meanwhile, the number of domains handled
by those who manage their own server - is distinctly LESS than 5% - and
shrinking. I think this is very bad for the industry. I fear that one
day - these large providers will get together and implement standards
that throw the rest of us overboard. Keep in mind that these dozen of so
companies operate a small percentage of the overall MTAs - but they
still handle the mail for 95+% of all domains. Part of the reason for
this - is that running a mail server is getting too complicated! New
layers of expertise and knowledge keeps getting added to the mix -
TLS/SSL, DKIM, SPF, dmarc, PTR records, and customers are starting to
expect near perfect spam filtering these days - often the email admin
wears 50 hats besides managing the mail server - so this is partly the
reason why so many flock to large cloud providers. So, in general, there
needs to a paradigm shift where providers of various hardware and
software run-your-own-mail-server packages... to try to make these
things easier. The problem that I'm having - is when dealing with these
sys admins who wear those 50 hats... they either don't want to hassle
with DNS - or they try to modify the DNS and figure out that their
hoster has that locked down - or their hoster keeps overwriting their
settings.
(4) Along those lines (what I said in #3) - for smallish operations
(perhaps fewer than 5000 mailboxes)... losing a little caching
efficiency is an acceptable tradeoff for being able to provide the
option to click a checkbox that says "use DNSBL provider's DNS servers
directly" - then, if there can be added caching that is limited to the
last few minutes is added - that is even better!
(5) It doesn't sound like this is possible in Exim. THEREFORE - What
would I need to do to build this as a fork (or addon?) to Exim - If I
try to provide a custom build of Exim - would I need to maintain
multiple versions for different OS's? Or could I just provide one single
download for my customers? What would be involved for them to install it
- yet without overritting their existing Exim settings... as a sort of
drop-in upgrade? What language is Exim written in?
(6) Take a look at step 1 of my subscribe page:
https://www.invaluement.com/subscribe/
First, from a marketing standpoint - this is a real buzzkiller! (if a
less technical manager-type - who just wants less spam at a minimal cost
- sees this - they get frustrated and click away!) And this is the new
and improved "simplified" version. NOW - consider that somewhere between
25-50% of all invaluement trials - even AFTER reading "step 1" - STILL
get that wrong! I then have to send a follow-up message trying to
further explain to them how/why their queries can't come via Google or
OpenDNS servers - how to fix that - and even then, half the time, they
still can't get it right. And every month, about 2-3% of existing
customers start suddenly doing this wrong - due to settings reverting
back to Google or OpenDNS. When any of these situations happens - then
the queries from Google or OpenDNS are obviously blocked. But the
problem is worsened by the fact that Google and OpenDNS often do as many
as a dozen retries, thus clogging up my logs with lots of failed queries
- and the sum total of all of these can add up to much resource usage.
Thus, if this "query DNSBL providers' DNS servers directly" were added -
and that caused a few extra non-cached redundant queries to happen -
that are generally otherwise correct - the extra queries from a lack of
caching is a lesser of evils!
Thanks!
--
Rob McEwen
http://www.invaluement.com
+1 (478) 475-9032
