[exim-dev] [Bug 3085] New: Allow UTF-8 for log output

https://bugs.exim.org/show_bug.cgi?id=3085

            Bug ID: 3085
           Summary: Allow UTF-8 for log output
           Product: Exim
           Version: N/A
          Hardware: All
                OS: Linux
            Status: NEW
          Severity: bug
          Priority: medium
         Component: Logging
          Assignee: unallocated@???
          Reporter: forza@???
                CC: exim-dev@???

This is probably not a bug, but more of a request for comments.

I am logging to syslog instead of files. The syslog is handled by syslog-ng,
and I parse the logfiles with Fail2Ban.

The exim.conf:

### Logging
log_selector            = +all
log_file_path            = syslog
syslog_timestamp        = false
syslog_duplication        = false
syslog_processname        = exim
SYSLOG_LONG_LINES        = yes

No, my issue is that sometimes Fail2Ban fails to read some of the lines and
outputs a warning like this:

2024-03-17T19:23:33.870+00:00 warning fail2ban.filter[2922]: WARNING Error
decoding line from '/var/log/exim.log' with 'UTF-8'.
2024-03-17T19:23:33.870+00:00 warning fail2ban.filter[2922]: WARNING Consider
setting logencoding to appropriate encoding for this jail. Continuing to
process line ignoring invalid characters: b'2024-03-17T19:23:33.698+00:00
notice exim[5673]: [12\\21] F From: "\xbe\xe7\xb9\xcc\xbc\xf8"
<msoony@???>\n'


So, this leads me to my current question. Can Exim be set to output UTF-8
encoded logs to syslog? Apparently, the syslog format according to RFC-5425
says " MSG SHOULD be UNICODE, encoded using UTF-8", but it seems to allow plain
US-ASCII too.

https://datatracker.ietf.org/doc/html/rfc5424#section-6.4

I believe syslog-ng could handle non-UT8 messages, using flags(sanitize-utf8)
on the source, however the manual specifies:

"The HEADER part of the message must be in plain ASCII format, the parameter
values of the STRUCTURED-DATA part must be in UTF-8, while the MSG part should
be in UTF-8. The different parts of the message are explained in the following
sections."

Perhaps I am overthinking all of this. I'd appreciate some thoughts on correct
logging configurations.

--
You are receiving this mail because:
You are on the CC list for the bug.

--
## subscription configuration (requires account):
## https://lists.exim.org/mailman3/postorius/lists/exim-dev.lists.exim.org/
## unsubscribe (doesn't require an account):
## exim-dev-unsubscribe@???
## Exim details at http://www.exim.org/
## Please use the Wiki with this list - http://wiki.exim.org/