[exim] How do ISP's restrict access without authentication

Hi,
I was wondering how exactly ISP's - that don't require authentication -
manage to restrict access to their customers only.
I know that Exim can restrict access by IP address, but IP addresses can be
spoofed (and very often are spoofed by automated scanners which search for
SMTP servers that are open in this way).
How then, do ISP's manage to prevent their servers from being abused?
In other words, if an ISP only authenticates based on IP address, then
surely that would leave their server open to abuse.
The answer to this question will help me a lot.
Thanks