Hi all,
too make yourself aware of an exim-dovecot-quota problem you may run
into, we need:
https://wiki.dovecot.org/MailLocation/Maildir
which states:
Optimizations
*
maildir_copy_with_hardlinks=yes (default): When copying a message,
do it with hard links whenever possible. This makes the performance
much better, and it's unlikely to have any side effects. Only reason
to disable this is if you're using a filesystem where hard links are
slow (e.g. HFS+).
means, if a client ( imap ) copies mails from one tree segment to
another, dovecot shall use hardlinks instead of a fs-storage copyoperation.
If you are running exim with a quota for imapmailboxes, you can now get
into trouble, as the FS in form of "repquota" and "du" miscalculates the
size of that mailbox, compared to exim. The fs gets a lower total size
of the directory, as it does not take hardlinks into account,*but exim
does*.
Example:
[root@sxxx Maildir]# repquota -a | grep 50544
#50544 -- 1341720 0 0 1341 0 0
and exim counted everything as if it was a real file:
check_dir_size: dir=/mailacct/{usernameinquestion}/Maildir/ sum=1573152669 count=1420
which in this case, resulted in an overflow of the quota and emails got
rejected with "mailbox full".
The issue was caused by a client copy operation of a folder into the
trashcan, which did not result in a delete of the copied folder.
or easier: it copied, but should had use "move" for it.
If you are the admin now, and check the quota or diskusage for that
specific mailbox, you will see a (mostly) much lower du as you quota for
the given mailbox suggests.
You can detect it with ls :
-rw------- 2 51703 exim 4688 28. Jul 14:27
1406550448.H508202P31298.testmailsystem.de:2,STac
-rw------- 1 51703 exim 6228 28. Jul 14:29 1406550593.H760899P1075.testmailsystem.de:2,STac
You notice the "2" in line 1 column 2 ? Thats an indicator, that you
have hardlinks in use.
Problem is, you have no idea where the other one is located. You have to
use "ls -lia" to get the inode referenced for that file:
5913370 -rw------- 1 51703 exim 5174 10. Sep 16:00 1410357617.H134775P24247.testmailsystem.de:2,STac
5904472 -rw------- 2 51703 exim 13042 11. Sep 13:11
1410433911.H351692P31331.testmailsystem.de:2,STac
and search with find for the inodeusage :
find /mailacct/mailbenutzer/ -inum 5904472 -print
Now you need to decide which one of those files gets deleted. If you
have an entire folderfull of hardlinked files, it's easy, as you can
just delete the folder.
The second methode to detect it is "du -s":
"du -sh .[A-Z]* * "
You get a list of the mailbox and a summary of the folders sizes. Now
run "du -s" on each folder manually and compare it to the first output.
You will notice that in the first du output for all folders, the size
a(or more) folders is just a few kb , but when you "du -s" it one by
one, you get a much bigger folder size out of it. That happens, because
du captures the inodes for files it processes, and if you hit a file
with the same inode again, which is a hardlink, it skips it. It can't
skip this, if it only finds one of the hardlinked files :) As a result,
the du -s of the single folder reveals the true "exim counted" size of
the folder.
The problem resides in the nature of hardlinks and how different tools
react to it.
The question is:
Shall Exim be changed to refect the quota-du pov ( i.e. with an option
how to handle hardlinks (default=oldstyle) ) or not?
Before you say, "dovecot caused it, just change the default option
there" remember: the problem already manifested itself on your maildir
storage and changing dovecots config now does not change this. Sooner or
later you may ran into this again.
I vote for a quota option in appendfile transport to skip hardlinks. A
simple (inode,boolean)hash check would do while scanning the mdir content.
What do you think?
best regards,
Marius
For german readers, there is an example case here:
https://marius.bloggt-in-braunschweig.de/2014/11/30/exim-hoehere-quota-durch-hardlinks/
