[Exim] smtp auth for remote hosts

I was looking into enabling smtp authentication, but I seem to be having a
problem. When I enable auth_hosts = *, no one can deliver mail to me. I have
log entries that show remote smtp hosts being denied because they did not
authenticate.

Do I have to have separate incoming/outgoing mail servers for smtp auth to
work?

Any help is appreciated.