[exim-dev] Feature: support for compressed messages on deliv…

Hello Devs...

I am exploring a way to start the conversation or submit a patch myself that would build support for compressed messages on delivery. Dovecot supports a feature to deal with compressed message files (plugin zlib) which allows for all messages to be compressed on the file store. The only caveat is that existing messages or messages not written/imported by Dovecot are not compressed, an external job must do this. It would be nice if/when new mail is delivered from Exim, this can be done automatically.

I first thought it would be as simple as adding a transport_filter on the appendfile delivery transport, however, I ran into some problems. Consider the following transport configuration.

local_delivery:
driver = appendfile
directory = $home/Maildir
delivery_date_add = true
envelope_to_add = true
return_path_add = true
create_directory = true
maildir_format = true
maildirfolder_create_regex = /\.[^/]+$
maildir_tag = ,S=$message_size,W=${eval:$message_size + $message_linecount + 1}
transport_filter = /usr/bin/gzip
transport_filter_timeout = 1m


The above *does* work, and message files are store in gzip format, but the problem is we need to use the original message size and original message_linecount for the message tags. (the former being much more important than the latter, though the linecount would be nice to have also). We also need thie pre-compressed message size to be used when dealing with quotas. (ie: adding messages to maildirsize)

The problem is exim uses the new message size returned from the filter, and rightfully so. But in this case the actual compressed file size is not what we would like, we would like to use the original message size as it was passed to before the transport_filter. The reason for this is to get the maildir S= (& W=) tags correct. Since Dovecot decompresses the messages on the fly before sending them to the clients. So it's imperative that we somehow keep the pre-compressed message size.

I can help out with this, but before I start ripping through the source code perhaps someone would already know how to pull this off easily (maybe even without a patch), or knows the code better than I do that would not require a lot of time. Or looking for suggestions on how best to implement this.

Any input would be great! TIA

-Robert