To some degree, this depends on the type of lookups you are using.
A flat file lookup will always be fastest as long as the flat files have
a relatively small number of entries. "relatively small" varies based on
your hardware, but if you keep it under 4K this will still probably beat
any of the other databases. Flat files are opened, and read sequentially.
Next would be cdb/berkeley db. These types of database files are opened,
and read randomly - potentially lots of seeks back and forth.
LDAP is next. OpenLDAP is basically a network interface on top of
Berkeley DB.
Finally, you have network-based databases such as sqlite and mysql.
Anything going through TCP sockets, and SQL statement parsing, has yet
another layer of overhead. SQL however can efficiently perform certain
types of complex queries that would be difficult to implement any other way.
If you are going to be doing lots of static lookups, where the data
being queried does not often change, you are best off using cdb or
Berkeley DB, and using rsync or the like to get the data to the host.
You can also contemplate storing data in a MySQL database, and
periodically refreshing a CDB database with the mysql content.
Your choice of database may also be determined by how easy it is to
distribute the information to multiple servers, e.g. in a cluster.
snowcrash+exim-users wrote:
> i though this was going to be a bit easier ...
>
> i'm interested in benchmarked performance of flatfile vs
> cdb/sqlite/mysql lookup performance.
>
> i _know_ that there are differences, but how significant? the
> frequent advice is for "short lists" use flat-file, and as the lists
> get longer, move to sql etc etc.
>
> what's the rule-of-thumb for a list being "long"?
>
> i found this list-thread commented by Nigel M., but no follow up (yet):
>
> http://www.exim.org/mail-archives/exim-users/Week-of-Mon-20051128/msg00017.html
>
> any further thoughts or references?
>
> thanks.
>
>
--
<http://www.foreThought.net> Jawaid Bazyar
President
Jawaid.Bazyar@??? <email:Jawaid.Bazyar@???>
ph 303.815.1814
fax 303.815.1001
Communications Simplified
