[exim] Strange confusing random slowness

I have a problem that's confusing me and having a hard time tracking it
down. One of my exim servers is randomly slow sending email. The symptom
showed up recently about a week ago around the time I installed 4.66 but
I don't think it's related.

Using thunderbird. I'll try to send an email and it will sit there for
10 to 20 seconds before it sends. Sometimes it sends instantly. I'm also
noticing slowness sending to mailman lists.

The server however is hardly loaded at all, and I actually just upgraded
it to a faster server with more memory but that didn't have any affect
on the problem.

It acts like it's slow DNS related but tried running my own dedicated
DNS server that's very fast and it didn't seem to make a difference.
/etc/resolv is pointing to my DNS server, and I don't think it's DNS but
it behaves the way things would behave if I had a DNS problem. So that's
why I'm confused.

Around the same time however I started having random slowness with a
spamassassin server where processes would back up in memory, but actual
server load levels would not be high. It is as if these processes are
waiting on something.

Anyhow - need a clue. Thanks in advance.