Hi,
thank you for responding,
Phil Pennock <exim-users@???> (Thu Feb 17 05:41:45 2011):
> On 2011-02-16 at 23:32 +0100, Heiko Schlittermann wrote:
> [ maildirsize recalculations expensive ]
> > How is this solved on other busy sites?
>
> A somewhat equivalent style of metadata caching (but not maildirsize) at
> previous employment: you only care about the metadata at the time a user
> connects to retrieve mail, so if the updates are frequent then rather
> than updating every 5 minutes there's a new email, all night long, you
> really care about having an update at the time of a new connection which
> needs the data.
The metadata (here the accumulated sizes of the mailbox contents)
are needed on the POP|IMAP side to avoid uploads into an already filled
mailbox. But on the external incoming side (we've a two stage incoming)
we need the information about a full mailbox almost immediatly.
> If need be, use a shadow_transport to write a flag-file to another
> directory, or append to a per-injection-host logfile, or whatever, for
> each new mail delivered. The client app can then pick up on those. For
> reliability reasons (shadow_transport fails when mail delivery succeeds)
> you probably want to randomly/daily force a complete scan anyway.
Yes, that's the direction my thoughts already went - not updating the
maildirsize file from within Exim, just sending the message size to some
meta data daemon.
But this would need some extra service running somehwere. It's what I'd
like to avoid.
Currently I'm investigating two directions:
- if I can prevent deliveries into a specific mailbox while
the size recalculation is running [preferred]
- if I can have some staged maildirsize, just appending the
sizes to some "journal" and trigger the recalculations
somehow [here I'm not sure if I just invent another layer
of the same problems]
--
Heiko
