Dear subscribers,
Our process involves a time critical thawing of frozen messages. The
faster it works the better it is.
We have two groups of servers. The first group can thaw a message in
about 15 ms. The second group can thaw a message within in 76 ms.
Our goal is to make the second server group as fast as the first.
Unfortunately, we are talking about different hardware, different OS,
different Exim versions.
So I guess as my question cannot be very specific, I cannot expect a
specific answer. However, maybe somebody can point me into the right
direction.
Here is a bit more detail information:
Server 1 (old group):
8 CPU cores, 4 GB RAM, CentOS 5.8, 2.6.18-194.26.1.el5PAE
WHM/cPanel 11.30.3 (build 4)
Exim 4.69
Server 2 (new group):
8 CPU cores, 8 GB RAM, CentOS 6.3, 2.6.32-279.5.1.el6.x86_64
WHM/cPanel 11.32.3 (build 23)
Exim 4.77
Running UnixBench on both servers indicates that server 2 is way faster
in all tests except for Process Creation and Shell Scripts when running
8 parallel copies (not when running single copy). I personally don't
think that this is the cause of the slowness but I mention it for the
sake of completeness.
cPanel and Exim are configured as similar as was possible with all spam
checks, RBLs etc. disabled.
For speed testing purposes I use a PHP script which first submits an
email to the queue (frozen).
Then I execute '/usr/sbin/exim -d -Mt <msgId>' and measure the microtime
before and after the call to get the difference.
For each server I performed 5 runs and averaged the results.
Server 1: 14.8 ms
Server 2: 76.2 ms
Note 1: When executing the above command without message ID, times of
both servers are almost identical. So something must happen during the
thaw process which introduces delays.
Note 2: Server 2 had the Perl interpreter started in the Exim
configuration which made it about 20 ms slower than it runs currently -
was at around 90 ms with Perl startup enabled.
Note 3: We monitored port 53 usage with tcpdump on Server 2 to make sure
there are no DNS requests involved causing delays.
Note 4: Each server group has similar hardware and software
configuration. Performance within each group is reproducible. First
group is consistently faster than second group.
Is there anything else which might improve the time on server/group 2?
Could it be caused by changes in between Exim versions - internal
processing changes?
Could it be caused by 32 vs. 64 bit or different kernels?
Any suggestions where to start or which tools could be used to gain
further insight?
Every pointer is highly welcome.
Down below I've attached the trace information for each of the servers.
Thanks,
Volker
---
Server 1:
Exim version 4.69 uid=0 gid=0 pid=1223 D=fbb95cfd
Berkeley DB: Sleepycat Software: Berkeley DB 4.3.29: (July 12, 2010)
Support for: crypteq iconv() IPv6 PAM Perl OpenSSL Content_Scanning
Old_Demime Experimental_SPF Experimental_SRS Experimental_DomainKeys
Experimental_DKIM
Lookups: lsearch wildlsearch nwildlsearch iplsearch dbm dbmnz passwd
Authenticators: cram_md5 dovecot plaintext spa
Routers: accept dnslookup ipliteral manualroute queryprogram redirect
Transports: appendfile/maildir autoreply pipe smtp
Size of off_t: 8
changed uid/gid: forcing real = effective
uid=0 gid=0 pid=1223
auxiliary group list: <none>
seeking password data for user "cpaneleximfilter": cache not available
getpwnam() succeeded uid=504 gid=32013
seeking password data for user "root": cache not available
getpwnam() succeeded uid=0 gid=0
configuration file is /etc/exim.conf
log selectors = 000026fc 0021200d
cwd=/opt/test 4 args: /usr/sbin/exim -d -Mt 1T4Eat-0000Jg-CW
trusted user
admin user
changed uid/gid: privilege not needed
uid=47 gid=12 pid=1223
auxiliary group list: 12
set_process_info: 1223 acting on specified messages
reading spool file 1T4Eat-0000Jg-CW-H
user=root uid=0 gid=0 sender=test@???
sender_local=0 ident=root
Non-recipients:
Empty Tree
---- End of tree ----
recipients_count=1
body_linecount=1 message_linecount=14
Writing spool header file
Size of headers = 518
LOG: MAIN
unfrozen by root
run 0: 12 ms
Server 2:
Exim version 4.77 uid=0 gid=0 pid=20748 D=fbb95cfd
Berkeley DB: Berkeley DB 4.7.25: (November 12, 2010)
Support for: crypteq iconv() IPv6 PAM Perl OpenSSL Content_Scanning DKIM
Old_Demime Experimental_SPF Experimental_SRS
Lookups (built-in): lsearch wildlsearch nwildlsearch iplsearch dbm dbmnz
passwd
Authenticators: cram_md5 dovecot plaintext spa
Routers: accept dnslookup ipliteral manualroute queryprogram redirect
Transports: appendfile/maildir autoreply pipe smtp
Size of off_t: 8
Compiler: GCC [4.4.4 20100726 (Red Hat 4.4.4-13)]
Library version: OpenSSL: Compile: OpenSSL 1.0.0-fips 29 Mar 2010
Runtime: OpenSSL 1.0.0-fips 29 Mar 2010
Library version: PCRE: Compile: 8.12
Runtime: 8.12 2011-01-15
Total 8 lookups
WHITELIST_D_MACROS unset
TRUSTED_CONFIG_LIST: "/etc/exim_trusted_configs"
changed uid/gid: forcing real = effective
uid=0 gid=0 pid=20748
auxiliary group list: <none>
seeking password data for user "cpaneleximfilter": cache not available
getpwnam() succeeded uid=32007 gid=32009
seeking password data for user "root": cache not available
getpwnam() succeeded uid=0 gid=0
configuration file is /etc/exim.conf
log selectors = 000034fc 0029304f
LOG: MAIN
cwd=/opt/test 4 args: /usr/sbin/exim -d -Mt 1T4EbF-0005Ob-CL
trusted user
admin user
changed uid/gid: privilege not needed
uid=47 gid=12 pid=20748
auxiliary group list: 12
set_process_info: 20748 acting on specified messages
reading spool file 1T4EbF-0005Ob-CL-H
user=root uid=0 gid=0 sender=test@???
sender_local=0 ident=root
Non-recipients:
Empty Tree
---- End of tree ----
recipients_count=1
body_linecount=1 message_linecount=14
Writing spool header file
Size of headers = 528
LOG: MAIN
unfrozen by root
run 0: 83 ms
