hHello Bill,
W B Hacker <wbh@???> (Mi 02 Feb 2011 04:30:50 CET):
>
> Calc method aside, doesn't that in effect 'reward' those who cannot be bothered
> to expunge, and/or who use their .Trash as a scratch storage area?
>
> I've had IMAP users who tried that with *years* worth of messages...
It's a "policy problem" - policy of the mailsystem could clearly state
that deleted messages are automatically expunged after xxx days.
Of course, this means extra processing power, but it's not timecritical.
> Seems better to INCLUDE those files in the count and penalize the sloppy, not
> exclude and let tem continue poor practice at the expense of the overall user
> community... and your hardware and admin expense as well..
Unfortunely the current maildir++ quota implementations do not have some
soft and hard limits, they just have limits…
Including the marked (for deletion) files is no problem at all. Clients are free to
expunge these messages to free up some space.
But some clients do not use the
1) mark (as deleted)
2) expunge method
they do
1) copy(message->"Trash")
2) mark original (as deleted)
3) expunge original
If these clients try to free up space, they end with "Can't delete
message: Quota full" (or some similiar message). To avoid support
tickets with this approach often the Trash folder is excluded from quota
calculations.
(Unfortunely nobody defined the name of the folder used as trash bin,
fortunely most clients seem to use "Trash".)
> > - Using du(1) isn't an option.
> > - because exim does the recalculation on its own when the magical
> > 5120 bytes size limit of maildirsize is reached
>
> I hear you - but at the end of the day there is a ton of machine code and I/O to
> be executed by *something* any way you cut it. 'Magical' just means *we* don't
> see the auto-fornication motion..
I'm not sure how to understand this. The reason for the "magical" 5120
byte limit of the "maildirsize" file seems to be NFS related
(raceconditions, updates w/o locking, …)
> > - because we're talking about 120k mailboxes on some NFS share
> > (netapp)
> >
> 120,000 Maildir's at what average population? ten files each? fifty each? A
> hundred?
Good question, mostly the worth of about 60 days incoming mails. Some
more detailed number I'll find.
> I would at least use du to get those stats for planning....
Hm. But the numbers will not be too useful, because we expect growing of
the maildirs as soon as the autoexpiration is not in place anymore.
> > - Using du(1) is not necessary at all, because the files already
> > have their size coded in the S=… tag of the file name.
>
> As soon as there is more than one file involved, parsing info from the file
> *names* rather than the fs doesn't seem to me to be an improvement when you have
> to extract, manipulate, calculate for 'many'.
I'd say it *IS* a difference between doing 40k readdir() and 40k stat().
On some test system using an NFS mounted volume I got about these
numbers for the maildir containing 40k messages and re-calculating the
maildirsize file:
0.075s for using the …S=nnnn… aproach (size coded into the filename)
1.4s for doing a stat() on every file
> Do you have any chance of dropping - or at least loosening - quotas in favor of
> allocating the account storage across multiple, separate 'chunks'?
The reason for thinking about quota is'nt of technical nature, it is
part of the customers business model. (The technical and economical challenges of
unlimited mail space have to be faced too, of course…)
> - more storage is usually cheaper than more admin labor/stress
More storage, more backup …
> - separating the 120,000 Maildir into subsets should produce less drama
> if/as/when some part goes offline - even if just for maintenance.
Yes, but - the current system works and will continue to work after
upgrading the bottle neck - the NFS storage. Using maildir quota by
means of the maildirsize file gives us mailbox quota for a low price.
But we're getting off topic. The task is to implent quota. Period. While
finding a lean solution I noticed some missing feature in Exim (and even
found that the Courier IMAP server is not consistent in counting these
<T> tagged messages.)
Thank you for giving me input ☺
--
Heiko
