This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.
--
--
[ Picked text/plain from multipart/alternative ]
Hi All,
I have spent quite some time researching and developing an add-in to the
excellent Exiscan-ACL that uses more than just the extension to identify a
file type. The result of all this work is FFPA (File FingerPrint Analyser).
After first obtaining Philip's permission to use the Exim list, I'd like to
ask for some (brave) volunteers to help test it. Actually you don't have to
be that brave because not only can test the included FFPA utility first, I
already have it running on a small number of mail hosts (handling an average
8k messages per day) and it seems to be quite stable.
FFPA will be available as patch for Exim 4.30 (and later) and Exiscan-ACL
under the GPL licence.
I've attached the current beta manual so you can get a better idea of what
it can do.
I regret that at the moment I no longer have a home website for it, so all
communications and updates will be via email (not this list!) until I find
one (offers anyone?).
I only have Sun Solaris 8 platforms on which to develop and test FFPA, so I
would be particularly interested in hearing from anyone with other
platforms.
If you'd like to take part or want more info, please feel free to contact me
at the address below, but include [FFPA] in the subject to get my attention!
Thanks for your time,
Tony Sheen
MCI EMEA PostMaster
tony.sheen@???
--
FFPA For Exiscan-ACL - V2.00 - 11-Feb-2004 - BETA BUILD 027
-------------------- ----- -----------
FFPA (File FingerPrint Analyser) v2 is an add-in to the Exiscan-ACL patch for
Exim v4.30 and later.
BETA INFORMATION: An FFPA beta patch takes the form:
'ffpa-2.00-xxx_exiscan-15.patch'
where 'xxx' is the build number. Beta testers should also
see 'ffpa-beta.txt' (if present) for the latest information.
Copyright Statements
--------------------
Exim is Copyright (c) University of Cambridge.
Author: Dr. Philip Hazel
Site: http://www.exim.org
Licence: GPL
Exiscan-ACL is Copyright (c) Tom Kistner
Author: Tom Kistner <tom@???>
Site: http://duncanthrax.net/exiscan-acl
Licence: GPL
FFPA is Copyright (C) Tony Sheen
Author: Tony Sheen
Site: <None at present>
Licence: GPL
Why Use FFPA ?
--------------
Unlike the majority of attachment blockers, FFPA does not rely on the file
extension to decide the file type. It opens each attachment in the message and
performs a number of tests to identify the real file type. Even if an EXE file
is renamed to TXT, it will still be indentified by FFPA.
A successful FFPA detection will stop all further match attempts on an
attachment, otherwise it is passed to each defined FFPA scanner for possible
matching until all scanners have been used.
Requirements For This Version
-----------------------------
Exim: V4.30 (exim-4.30.tar.gz -or- exim-4.30.tar.bz2 etc)
Exiscan: Revision 15 (exiscan-acl-4.30-15.patch) [BETA: SUPPLIED]
FFPA: V2.00 (ffpa-2.00_exiscan-15.patch)
Installation
------------
1) Install a clean copy of the Exim source.
2) Patch Exim with the correct (matching) version of Exiscan-ACL.
3) Patch Exim with the correct (matching) version of FFPA.
4) Configure, compile and install Exim as normal.
5) Edit the Exim configuration file to include the wanted FFPA scanners.
6) Test it...
7) Take a well-earned rest !
As part of the Exim installation, two utility programs are created:
FFPA - A standalone file scanner
HXDUMP - A file to hex converter
See ffpa-utils.txt for more information.
Configuring Exim
----------------
Main configuration section:
FFPA_LOGGING
For the author's statistical purposes, Exiscan-ACL and Exim do not log
sufficient information about a frozen or rejected message. This command
provides a rather terse one-liner for both matched extension blocking and
spam that has exceeded the SpamAssassin threshold. Logging is OFF by
default.
Usage:
FFPA_LOGGING=<option>
where: <option> is one of 'TRUE', 'FALSE', 'YES' or 'NO'
Example:
FFPA_LOGGING=yes
would turn on logging.
The logging format uses the Exim style. Following the standard date, time
and message ID is the reason for the entry - always preceded by the word
'Exiscan' (note the capital E). This is followed by the host name (H=) and
IP address, then the sender's address from the envelope (F=), then the
number of recipients (D[?]=) and their addresses, the size of the message
(S=) and finally the subject (T=).
Examples:
Note that in each of the three examples, all the text would normally be
on a single line. The format has been chosen to allow data extraction
using a single Perl regex.
2004-02-10 15:02:06 1AqZOn-0007YZ-Ur
Exiscan ffpa: found banned file (EXE)
H=mail.borg.com [192.168.27.41]
F='queenie@???'
D[3]=[sevenofnine@??? janeway@??? neelix@???]
S=145621
T='A special present for you'
2004-02-10 18:19:47 1AqcU5-0000Gg-Hl
Exiscan: found spam (14.3/6.00)
H=mail.starfleet.com [192.168.27.12]
F='kirk@???'
D[2]=[scotty@??? sulu@???]
S=2041
T='New orders'
2004-02-10 19:11:53 1AqdIW-0000aC-VW
Exiscan: found banned file (ppt)
H=mail.ds9.com [192.168.27.20]
F='quark@???'
D[1]=[rom@???]
S=28971
T='Slug-o-Cola shipments plan'
Exiscan: found banned file (???)
The (???) is the extension that cause the banning in Exiscan-ACL.
Exiscan ffpa: found banned file (???)
The (???) is the extension that cause the banning in FFPA.
Exiscan: found spam (14.3/6.00)
The first figure is the actual message score and the second is the
threshold in the SpamAssassin configuration file.
S=2041
This figure is the size of the message when it arrived. The size in
Exim's received line (<=) includes the size of any received header,
while the sent lines (=> or ->) includes all other header and/or body
additions. This initial value has been added to allow a variety of
statistics to be calculated.
D[2]=[scotty@??? sulu@???]
In this example there were 2 recipients and each destination address is
separated from the next by a single space.
FFPA_SCANNERS
A single Exim configuration file option in the main configuration section
controls whether FFPA is activated. This has been done for two reasons:
1) To enable a single Exim build to be created for various deployments and
only have FFPA active as and when required.
2) So that the end user can choose which scanners to enable.
Usage:
FFPA_SCANNERS=<options>
where: <options> is a colon separated list of the scanners to activate, or
the word 'all' to run all scanners.
Example:
FFPA_SCANNERS=EXE:RIFF
would enable FFPA scanning for the file types supported by the EXE and RIFF
scanners ONLY. Any other available scanners would not be used.
By default FFPA is turned OFF as no scanners have been requested.
FFPA_USE_SUBTYPES
FFPA normally uses generic extensions for detecting files (see below). To
detect specific types, subtype detection needs to be enabled as it is OFF
by default.
Usage:
FFPA_USE_SUBTYPES=<option>
where: <option> is one of 'TRUE', 'FALSE', 'YES' or 'NO'
Example:
FFPA_USE_SUBTYPES=yes
would allow the detection of subtypes.
Generic Extensions
------------------
Before getting to the ACL configuration, a word about file extensions. FFPA
uses its own style of file extensions in conjunction with the Exiscan-ACL
'demime' command for three main reasons:
1) To distinguish it from normal extensions. Standard and FFPA file extensions
can be freely mixed, so it therefore becomes obvious to a reader of a
configuration file that specific types are being sought using FFPA.
2) To enable files from different operating systems or of different uses to be
independantly detected. The author has found that, for example, the 'pfx'
file extension as used in Windows can be both a DLL and a fax header page,
and the only way to tell the difference is to look inside.
3) To speed up Exim's response when looking for detected file types. As FFPA
doesn't use standard file extensions, it does not have to process them,
thus reducing the processing overhead.
The word 'generic' can have many meanings, but in the context of FFPA it means
a generally accepted file extension.
For instance, the simple DLL file has a huge range of possible file extensions
('dll', 'ocx' and 'ax' to name but a few) but it is simply known to the user
as a DLL file. A generic extension for this type of file whould be 'dll' so a
scanner will return this text in order that Exiscan-ACL can easily catch them
all...
A prefix is used to indicate the type. For instance 'W_' is used to indicate a
Windows type file and 'M_' for a multi-media type file.
In this release, the following generic extensions are supported:
W_EXE - Windows/OS2 EXE files
W_DLL - Windows/OS2 DLL files
W_SYS - Windows/OS2 system files
M_WAV - WAV audio files
ACL Configuration
-----------------
The FFPA functionality is ONLY used FFPA_SCANNERS has been used, -AND- if the
Exiscan-ACL demime facility is actually used in the DATA ACL.
Note that these examples cannot be used as-is, they have to be inserted into a
valid DATA ACL section. See the Exim manual and the Exiscan-ACL documentation
for more information.
Example 1 - Rejecting unwanted types
This example simply rejects all messages that FFPA has detected with EXE
and DLL file type attachments...
deny message = $found_extension files are not accepted here
demime = vbs:bat:W_EXE:W_DLL
Example 2 - Freezing unwanted types
This example accepts a message with an EXE or WAV file attachment (and
attachments with normal com and pif extensions), but freezes them for later
review etc.
WARNING: The 'control' statement MUST follow the 'demime' command or ALL
messages will be frozen !!!
accept message = $found_extension files are not accepted here
demime = com:pif:W_EXE:M_WAV
control = freeze
And that's it really... Yes really!
Processing And Other Overheads
------------------------------
If the demime or malware options of Exiscan-ACL are already in use, then the
FFPA overhead is very small.
If they are not already in use, then a slowdown in the processing of email
messages will be noticeable. The faster the hardware and processors and the
more installed RAM, the less noticeable the processing overheads will be.
As a rule of thumb, allow at least double the currently used disc space for
Exiscan-ACL to unpack and process the messages.
Files That FFPA Does Not Detect
-------------------------------
If FFPA does not detect a file, and either it should have done or an example
for a new scanner is being submitted, please take the following steps.
1) Convert the file to a hex file with HXDUMP.
2) Zip up the results.
3) E-mail the zip file to the author along with some relevant information.
Any message larger than 10Mb will not get through. Sorry, but this is company
policy. In this case, please split the hex file into parts and send those.
This method of submission is preferred just in case FFPA has been updated to
include the file you wish to submit.
Please note that the author is ALWAYS pleased to accept new files and file
types which will be added to the testing archive.
EXE Scanning
------------
The EXE scanner detects a number of different subtypes within the overall EXE
definition. If it detects a DLL, it will report a DLL was found - even though
a DLL is actually just a form of EXE file.
However, bear in mind that it does not discrimate between a Windows EXE and a
DOS EXE file as they both use the same file extension. To stop only DOS EXE
files, subtype scanning would have to be enabled. Use 'ffpa -v' to see which
subtypes are currently supported.
Other Scanners
--------------
The following FFPA scanners are currently under development and at least AVI
should be present in the first public release:
AVI
MID (and RMID) - MIDI audio/music files
WMV
Although MPG/MP3 support is planned for the future, there are a huge number of
MPG file types and subtypes etc, and not all of them are video and/or audio
streams. This fact will delay its appearance (unless some very nice person can
help me out here!).
Comments And Suggestions
------------------------
The author would like to hear from anyone who has any comments or suggestions
on improving FFPA.
Differences To FFPA v1.xx
-------------------------
FFPA V1.xx was written as a stand-alone Exim 'local_scan' function and was
only designed to stop Windows EXE and DLL files. There was no facility to add
further file type scanners.
V1.xx was only used in a very limited number of sites and with the agreement
of those Postmasters, it will no longer supported by the author after the
release of v2.
The FFPA_SCANNERS command in the Exim configuration file is the ONLY one to be
retained. All the others are no longer relevant and should be removed. Their
continued use will cause syntax errors when trying to start Exim. Sorry, but
this is deliberate to ensure they are deleted.
Contact Details
---------------
Tony Sheen
MCI EMEA PostMaster
Please put [FFPA] in the subject of any message relating to FFPA. Thank you.
tony.sheen@???
--
