Re: [exim-dev] Bad utf-8 in pgsql lookup and mainlog

Startseite
Nachricht löschen
Nachricht beantworten
Autor: Axel Rau
Datum:  
To: exim-dev
Betreff: Re: [exim-dev] Bad utf-8 in pgsql lookup and mainlog

Am 20.07.2013 um 15:45 schrieb Jeremy Harris <jgh@???>:

> On 07/20/2013 01:17 PM, Axel Rau wrote:
>> Recording of utf-8 characters from headers in mainlog and PostgreSQL DB via lookup usually works flawlessly.
>>
>> Occasionally PostgreSQL complains during INSERT of header items or main log events (our log host uses PostgreSQL as bakend) about invalid byte sequence, like here:
>>
>> [1\3] 1V085d-00067H-9X H=mail03.noris.net [62.128.1.223] Warning: ACL "warn" statement skipped: condition test deferred: PGSQL: query failed: ERROR: invalid byte sequence for encoding "UTF8": 0xfc
>>
>> 2013-07-19T10:39:10.005396+00:00 db1 rsyslogd: db error (22021): invalid byte sequence for encoding "UTF8": 0xfc
>> 2013-07-19T10:39:10.005415+00:00 db1 rsyslogd: db error (event): |2013-07-19t10:39:09.991124+00:00|6|2|mx4|exim| [2\3] (PGRES_FATAL_ERROR) (SELECT * FROM record_Reception( '1525916', '1V085d-00067H-9X', 'Staatstheater Nürnberg <info@???>', 'Newsletter Staatstheater Nürnberg', 'none', 'N/A'))
>>
>> Does this come from bad encoding of original mail headers?
>
> It seems likely from the complaint coming from pgsql,
> but you've not shown us any relevant bit of your exim config.

---
  warn        condition = ${if eq {${lookup pgsql {SELECT * FROM record_Reception( \
                        '${quote_pgsql:$acl_m_mail_id_list}', \
                        '${quote_pgsql:$message_exim_id}', \
                        '${quote_pgsql:${if def:h_from:{$h_from:}{$h_sender:}}}', \
                        '${quote_pgsql:$h_Subject:}', \
                        '${quote_pgsql:$dkim_verify_status}', \
                        '${quote_pgsql:N/A}')}}}{t}}
---
record_Reception(…) is a backend function:
---
CREATE OR REPLACE FUNCTION record_Reception(mail_id_list text, msgID text, from_hdr text, subject_hdr text, DKIMresult text, SA_result text) RETURNS BOOLEAN AS $$
    DECLARE
        retval BOOLEAN  := True;
        affected_rows INT;
    BEGIN
                    --RAISE NOTICE '[record_Reception 1: mail_id_list_text = %; Subject = "%s"]', mail_id_list, convert_from(from_hdr, 'LATIN1');
            UPDATE mail
              SET messageID = msgID,
                from_header = from_hdr,
                subject_header =  subject_hdr,
                deliverytime = NOW()
            WHERE id = ANY( CAST(string_to_array( mail_id_list, ',' ) AS BIGINT[]));
        GET DIAGNOSTICS affected_rows := ROW_COUNT;
        IF affected_rows = 0 THEN
                        RAISE NOTICE '[record_Reception 1: mail_id_list_text = %]', mail_id_list;
            retval = False;
        END IF;
        RETURN retval;
    END;
$$ LANGUAGE 'plpgsql';
---


>
> Can you duplicate the situation manually?

No. I would have to create a mail with an invalid byte sequence in a subject header.
Any suggestions?
> Can you identify the lines in your exim config that are responsible?

Yes, see above.

Axel
---
PGP-Key:29E99DD6 ☀ +49 151 2300 9283 ☀ computing @ chaos claudius