[exim] Manual route / lookup troubles

I've been doing some limited testing with Exim for a couple of weeks.

I've got exim running on port 25 & port 26.

Port 26 is protected via a firewall and will only allow connections from
another server that does the spam protection, Exim handles the final
delivery to client servers.

Messages from port 26 and I use a manualroute router to control where to
send the messages, based on a lookup of local_parts and route_list.

That all works fine.

The part I'm having trouble with is, bounce handling. I try and deliver,
if the recipient server rejects the message and I need to process a bounce
I run into a problem for some domains (which I host, and need to manual
route)

For a few trouble domains which must be manually routed, Exim looks up the
MX record and sees that it is listed as the lowest MX record - messages are
frozen and will not deliver. It is true that it is the lowest MX record,
to find the correct destination I need to get the server from the file for
the local_parts file I have.

The bounce does not come in on port 26, so it does not hit my manualroute
and can't figure out where to send the messages.

I tried making a second manualroute router that rather than working on the
port, worked on specific domains in the condition. That did not turn out
very well...

Any suggestions how I might go about resolving the destination server when
I need to use the manualroute?