[exim] Long retry times after successful pipe delivery

Top Page
Delete this message
Reply to this message
Author: Guido.Schneider
Date:  
To: exim-users
Subject: [exim] Long retry times after successful pipe delivery




Hello,
since we changed from Debian Sarge (exim 4.50) to Debian Etch (exim 4.63)
we
observe that messages may waiting for a long time before the next delivery
attempt
is done even after a successfull delivery to the same recipient
(for details see "Test case" description below).
I suppose that the change PH/19 in the exim 4.61 changlog is responsible
for the problem.
1. Is it possible to get the old retry behaviour (may be with a
configuration change ?)


System description:
We are using exim together with a pipe transport that delivers remote
messages
over a slow link (< 9600bps) using an special protocol (no IP).
The delivery using the pipe transport is not queued so the pipe processes
may run a
long time (several 10 minutes).
To avoid a lot of waiting pipe processes they return a temp. failure
(exit code 75 ) if they detect too many running instances.
The main target is to use the link capacity efficiently.
With exim 4.50 in Debian Sarge this solution worked satisfactorily.

2. Is there another way to limit the number of running pipe processes of
our transport
without having other transports blocked ?
I think this happens if I limit the number of queue runners.

Test case:
Send 6 messages to "x@??? ".
Each successful delivery by the pipe process takes 10 min.

Result:
Time until all 6 messages are delivered:
exim 4.50: 104 min (max. 3 min delay between successfull delivery and
next delivery)
exim 4.63: 163 min ( up to 50 min delay between successfull delivery and
next delivery)

Retry rule:
*@partial0-lsearch*;/etc/testhosts * F,15m,3m; G,16h,1h,1.5; F,4d,6h

Router:
testMailRouter:
        debug_print = "R: TestMailRouter for $local_part@$domain"
        driver = manualroute
        transport = test_transport
        domains = ! +local_domains
        route_data = ${lookup{$domain}partial0-lsearch*{/etc/testhosts}}
        require_files = /usr/local/sbin/test_pipe_mail


Transport:
test_transport:
        debug_print = "T: test_pipe for host:$host
domain:$local_part@$domain"
        driver = pipe
        command = "/usr/local/sbin/test_pipe_mail $host $message_id"
      path = "/bin:/usr/bin:/usr/local/bin"
        batch_max = 10000
        use_bsmtp
        message_prefix = "EHLO $primary_hostname \r\n"
            log_output = true
            return_fail_output = true
      use_crlf
      timeout = 4h


/etc/testhosts:
dom1.test: 1
dom2.test: 2

exim -bP:
no_accept_8bitmime
acl_not_smtp =
acl_not_smtp_start =
acl_smtp_auth =
acl_smtp_connect =
acl_smtp_data = acl_check_data
acl_smtp_etrn =
acl_smtp_expn =
acl_smtp_helo =
acl_smtp_mail =
acl_smtp_mailauth =
acl_smtp_predata =
acl_smtp_quit =
acl_smtp_rcpt = acl_check_rcpt
acl_smtp_starttls =
acl_smtp_vrfy =
admin_groups =
no_allow_domain_literals
no_allow_mx_to_ip
no_allow_utf8_domains
auth_advertise_hosts = *
auto_thaw = 0s
bi_command =
bounce_message_file =
bounce_message_text =
bounce_return_body
bounce_return_message
bounce_return_size_limit = 100K
bounce_sender_authentication =
callout_domain_negative_expire = 3h
callout_domain_positive_expire = 1w
callout_negative_expire = 2h
callout_positive_expire = 1d
callout_random_local_part = $primary_hostname-$tod_epoch-testing
check_log_inodes = 0
check_log_space = 0
check_rfc2047_length
check_spool_inodes = 0
check_spool_space = 0
daemon_smtp_ports = smtp
daemon_startup_retries = 9
daemon_startup_sleep = 30s
delay_warning = 1d
delay_warning_condition = ${if or {{
!eq{$h_list-id:$h_list-post:$h_list-subscribe:}{} }{
match{$h_precedence:}{(?i)bulk|list|junk} }{
match{$h_auto-submitted:}{(?i)auto-generated|auto-replied} }} {no}{yes}}
no_deliver_drop_privilege
deliver_queue_load_max =
delivery_date_remove
no_disable_ipv6
dns_again_means_nonexist =
dns_check_names_pattern =
(?i)^(?>(?(1)\.|())[^\W_](?>[a-z0-9/-]*[^\W_])?)+$
dns_csa_search_limit = 5
dns_csa_use_reverse
dns_ipv4_lookup =
dns_retrans = 0s
dns_retry = 0
no_drop_cr
envelope_to_remove
errors_copy =
errors_reply_to =
exim_group = Debian-exim
exim_path = /usr/sbin/exim4
exim_user = Debian-exim
extra_local_interfaces =
extract_addresses_remove_arguments
finduser_retries = 0
freeze_tell = postmaster
gecos_name = $1
gecos_pattern = ^([^,:]*)
header_line_maxsize = 0
header_maxsize = 1048576
headers_charset = UTF-8
helo_accept_junk_hosts =
helo_allow_chars =
helo_lookup_domains = @ : @[]
helo_try_verify_hosts =
helo_verify_hosts =
hold_domains =
host_lookup =
host_lookup_order = bydns:byaddr
host_reject_connection =
hosts_connection_nolog =
hosts_treat_as_local =
ignore_bounce_errors_after = 2d
ignore_fromline_hosts =
no_ignore_fromline_local
keep_malformed = 4d
no_local_from_check
local_from_prefix =
local_from_suffix =
local_interfaces = <; ::0 ; 0.0.0.0
local_scan_path =
local_scan_timeout = 5m
local_sender_retain
localhost_number =
log_file_path = /var/log/exim4/%slog
log_selector =
no_log_timezone
lookup_open_max = 25
max_username_length = 0
message_body_visible = 500
message_id_header_domain =
message_id_header_text =
message_logs
message_size_limit = 50M
no_move_frozen_messages
no_mua_wrapper
never_users =
percent_hack_domains =
pid_file_path = /var/run/exim4/exim.pid
pipelining_advertise_hosts = *
no_preserve_message_logs
primary_hostname = gsvm-etch-texim.localdomain
no_print_topbitchars
process_log_path =
prod_requires_admin
qualify_domain = gsvm-etch-texim.localdomain
qualify_recipient = gsvm-etch-texim.localdomain
queue_domains =
queue_list_requires_admin
no_queue_only
queue_only_file =
queue_only_load =
queue_only_override
queue_run_in_order
queue_run_max = 5
queue_smtp_domains =
receive_timeout = 0s
received_header_text = Received: ${if def:sender_rcvhost {from
$sender_rcvhost\n\t}{${if def:sender_ident {from
${quote_local_part:$sender_ident} }}${if def:sender_helo_name
{(helo=$sender_helo_name)\n\t}}}}by $primary_hostname ${if
def:received_protocol {with $received_protocol}} ${if def:tls_cipher
{($tls_cipher)\n\t}}(Exim $version_number)\n\t${if def:sender_address
{(envelope-from <$sender_address>)\n\t}}id $message_exim_id${if
def:received_for {\n\tfor $received_for}}
received_headers_max = 30
recipient_unqualified_hosts =
recipients_max = 0
no_recipients_max_reject
remote_max_parallel = 2
remote_sort_domains =
retry_data_expire = 1w
retry_interval_max = 1d
return_path_remove
rfc1413_hosts = *
rfc1413_query_timeout = 0s
sender_unqualified_hosts =
smtp_accept_keepalive
smtp_accept_max = 20
smtp_accept_max_nonmail = 10
smtp_accept_max_nonmail_hosts = *
smtp_accept_max_per_connection = 1000
smtp_accept_max_per_host =
smtp_accept_queue = 0
smtp_accept_queue_per_connection = 10
smtp_accept_reserve = 0
smtp_active_hostname =
smtp_banner = $smtp_active_hostname ESMTP Exim $version_number $tod_full
smtp_check_spool_space
smtp_connect_backlog = 20
smtp_enforce_sync
smtp_etrn_command =
smtp_etrn_serialize
smtp_load_reserve =
smtp_max_synprot_errors = 3
smtp_max_unknown_commands = 3
smtp_ratelimit_hosts =
smtp_ratelimit_mail =
smtp_ratelimit_rcpt =
smtp_receive_timeout = 5m
smtp_reserve_hosts =
no_smtp_return_error_details
no_split_spool_directory
spool_directory = /var/spool/exim4
no_strip_excess_angle_brackets
no_strip_trailing_dot
syslog_duplication
syslog_facility =
syslog_processname = exim
syslog_timestamp
system_filter =
system_filter_directory_transport =
system_filter_file_transport =
system_filter_group =
system_filter_pipe_transport =
system_filter_reply_transport =
system_filter_user =
tcp_nodelay
timeout_frozen_after = 1w
timezone =
tls_advertise_hosts =
tls_certificate =
tls_crl =
tls_dhparam =
tls_on_connect_ports =
tls_privatekey =
no_tls_remember_esmtp
tls_require_ciphers =
tls_try_verify_hosts =
tls_verify_certificates =
tls_verify_hosts =
trusted_groups =
trusted_users = uucp
unknown_login =
unknown_username =
untrusted_set_sender = *
uucp_from_pattern =
^From\s+(\S+)\s+(?:[a-zA-Z]{3},?\s+)?(?:[a-zA-Z]{3}\s+\d?\d|\d?\d\s+[a-zA-Z]{3}\s+\d\d(?:\d\d)?)\s+\d\d?:\d\d?
uucp_from_sender = $1
warn_message_file =
write_rejectlog

Additional log files are also available.