homeold db
documentation:zdkimfilter zdkimsign redact zaggregate zfilter_db zdkimfilter.conf zdkimgenkey

ZDKIMFILTER
"z" DKIM filter for Courier-MTA using the OpenDKIM Project's library

[Requirements] [Download] [Install] [Set up] [What it does] [Links] [License]

zdkimfilter is a global filter for the Courier-MTA mail system. It either adds or verifies DKIM signatures by calling the functions provided by the OpenDKIM library.

I called it zdkimfilter because I think it's better to run it after any other global filter.

Requirements

Most installations already have OpenSSL, MySQL, and resolver. The Install section below provides minimal hints for installing OpenDBX, OpenDKIM, and zdkimfilter itself.

Download

here are the available 0.* versions:
v0.5 Tue 05 Oct 2010 14:10:11 [gpg signature] [announce]
v0.4 Fri 16 Jul 2010 18:07:16 [gpg signature] [announce]
v0.3 Wed 28 Apr 2010 08:04:14 [gpg signature] [announce]
v0.2 Tue 23 Mar 2010 11:03:43 [gpg signature] [announce]
v0.1 Wed 03 Mar 2010 16:03:59 [gpg signature] [announce]

it is also possible to checkout svn sources.

Install

OpenDBX (optional)

OpenDBX is a lightweight stable wrapper around MySQL. It is needed if you intend to collect statistics about DKIM verifications in a database. More on statistics below.

Download the package from linuxnetworks. For a fuss-free installation, I've configured it like so:

./configure CFLAGS="your choice" \  
   CPPFLAGS="-I/usr/include/mysql" \ this is the most common location...
   LDFLAGS="-L/usr/lib/mysql" \ ... run mysql_config to make sure.
   --with-backends="mysql" \  
   --disable-utils \ unless you want odbx-sql, an alternative to mysql
   --enable-singlelib \ build single library including backends
   --disable-shared YMMV
make  
make install  

OpenDKIM

Download the package from opendkim.org and read its INSTALL. Options I've found worth considering are:

./configure CFLAGS="your choice" \ e.g. whether to use -g, what optimizations, what -march, ...
   --disable-filter \ we want just the library, not the milter
   --enable-stats \ this delivers the opendkim-stats utility
   --with-odbx \ if you have OpenDBX, this additionally delivers opendkim-genstats and opendkim-importstats
   --enable-dkim_reputation \ check out dkim-reputation.org
   --disable-shared YMMV.
make  
make install  

zdkimfilter

Download the latest version, untar,

./configure CFLAGS="your choice" there is few conditional compilation, so you shouldn't need any option
make  
make install if zdkimfilter is running, do courierfilter stop before this.
However, you may want to plan configuration changes before doing so. See next section...

Set Up

After install, navigate to the courier/filters directory where your zdkimfilter.conf has been installed. Edit it. Options are documented in there. Before install, the configuration file is in etc/zdkimfilter.conf.dist, in the untarred directory. You may also read the latest svn source of that file.

On upgrading, the existing configuration file, if any, is left intact, and a dist copy is installed besides it. Compare the files to check for new parameters. You can also display the values parsed from either configuration file as described below.

After modifying the configuration file, you need to restart zdkimfilter for changes to take effect. Use either courierfilter or filterctl for that.

Keys

You may want to change the domain_keys directory, otherwise create a keys subdirectory in the directory you just navigated to, and adjust its ownership and permissions.

Every now and then, you may want to change the selector used for signing.

cd courier/filters/keys                        
opendkim-genkey -s march -t -d example.com this creates a new key for the march selector.
edit march.txt recall instructions in OpenDKIM's INSTALL (section LARGE KEYS) for publishing the public key in the DNS
rm march.txt after copying and pasting the public key to example.com's zone file, you don't need this file any more
ln -s -n march.private example.com this link enables signing for example.com. You may want to wait until your public key is available at secondary DNS servers before doing so.

A chown command is implied for each file creation above, unless you are logged in as the mail user. opendkim-genkey is smart enough to set proper permissions.

Courier settings

Courier may rewrite a mail file upon receiving it, before running global filters. DKIM signatures are likely to break in such case. If you have other global filters that may break DKIM signatures, you should probably run two dkimfilters, one for verification and one for signing. Please discuss this on courier-users list.

To avoid rewritings by Courier, edit esmtpd in Courier's sysconfig directory. Add

MIME=none never rewrite messages

You may still want to allow some rewriting for outgoing messages. It can be done for users submitting to port 587 by editing esmtpd-msa and override the value of MIME

MIME=some rewrite some messages, e.g. to ensure that a Content-Transfer-Encoding field is present.
NOTE: the MIME environment variable is documented in submit(8), but does not have a prepared stanza in esmtpd or esmtpd-msa. Those files are sourced, thus one might as well write unexport MIME to reset it in esmtpd-msa; setting it to some, or other unrecognized value, has the same effect. Courier needs to be restarted for those settings to take effect.

ADSP

Since version 0.4, the default installed conf file disables ADSP behavior. This is ruled by the parameter no_author_domain. When upgrading from an earlier version, consider setting that variable to true so as to avoid possible message loss.

What it does

Signing

Once installed, zdkimfilter checks whether the message has been accepted with RELAYCLIENT permission. If so, and there is an authenticated user, a domain for signing is searched in the user's id. If not found there, the configured default_domain is used. If a private key can be found in the domain_keys directory, the filter will sign with it. The selector is found either reading the symbolic link or using defaults.

Verifying

If the message had no RELAYCLIENT setting, the filter will verify and write an Authentication-Results (A-R) field on top of the header. If the first Received-SPF fields found in the message have a "pass", that info will also be repeated in A-R. Only one DKIM signature is reported in A-R. In presence of multiple valid signatures, the filter chooses it in this order:

  1. Author's domain signature (after the From header field.)
  2. Sender's domain signature (after Received-SPF for MAILFROM.)
  3. Other (possibly helo-related).

If the author's domain is invalid, or if its policy mandates a specific behavior for messages without a valid author's domain signature and none is found in the message, the filter honors that policy. This would be a relief for phishing, if ADSP worked: Phishing with bait service@paypal.com can be blocked this way, and it is reliable since PayPal reserved that domain for transactional mail. For other cases, senders can be whitelisted. Alternatively, the filter can be configured to never honor ADSP, but add a line for the dkim-adsp method in its A-R field inside the message. The latter alternative is recommended, and is the configuration distributed by default since version 0.4. In addition, the filter may add A-R methods x-dwl after looking up the signer in Spamhaus, and x-dkim-rep according to the configured dkim-reputation thresholds. Authentication-Results should be checked by mail clients, enabled as specified by RFC 5451.

Use in scripting

Command line usage has been conceived for debugging and testing, so it is not very user-friendly. The directory it gets installed to is usually not in PATH. However, using zdkimfilter, it is possible to prepare scripts that sign and/or verify a mail file offline. The final order of header fields will differ, according to the sequence of events.

zdkimfilter command line args:
  -f config-filename      override the default zdkimfilter.conf
  --help                  print this stuff and exit
  --version               print version string and exit
  -tN file...             scan rest of args as N ctl and mail file(s)
  --batch-test            enter batch test mode

Basically, you need to create a dummy ctlfile and an input file containing four lines:

  1. the name of a mail file, header and body,
  2. the name of the ctlfile,
  3. an empty line, and
  4. the keyword exit.

Control File Format is documented in the relevant section of Courier Mail Queue. We only need three record types:

u (msgsource),
if this is not authsmtp then the message is verified, rather than signed;
i (authname),
needed for signing, the signing domain is either after the '@' in this address or in the conf file;
M (msgid),
this is only used for logging (to stderr).

Recall that the record identifier is case sensitive, and don't name ctl and mail files like test*, sig*, sleep, and exit*, as that may conflict with batch mode keywords. Given a mail.eml file and a key.private, you can filter mail before sending like so:

#! /bin/sh
# link domain to selector, if needed
ln -n -s key.private example.com

# custom conf file, if needed
printf 'domain_keys = .\n' > conf

# sign mail.eml using postmaster@example.com as the login id
printf 'uauthsmtp\nipostmaster@example.com\nMtestsigning\n' > ctlfile
printf 'mail.eml\nctlfile\n\nexit' | /your/path/to/zdkimfilter -f conf --batch-test > /dev/null 2>log

# verify signature, if needed
# Received field is needed: its "by localhost with" provides the authserv-id
printf 'Received: from localhost by localhost with local\n' > mail2.eml
cat mail.eml >> mail2.eml
printf 'usmtp\nMtestverifying\n' > ctlfile
printf 'mail2.eml\nctlfile\n\nexit' | /your/path/to/zdkimfilter -f conf --batch-test > /dev/null 2>>log

# send it
cat mail2.eml | sendmail

A parsed printout of a conf file can be obtained like so:

$ echo test1 | /your/path/to/zdkimfilter -f conf --batch-test

The remaining batch mode keywords are only useful for testing and are not documented; their usage can be deduced from the testsuite, if needed. Likewise, I don't document the similar option -tN file..., which is only useful for attaching the gdb debugger to the forked working copy of the filter.

Statistics (since OpenDKIM v2.2.0 and zdkimfilter v0.5)

OpenDKIM.org keeps data about DKIM verifications. You may contribute to that data by submitting statistics files generated by zdkimfilter. You may feed your data to your own database, reusing their schema, if you have OpenDBX. You may extract daily statistics from those plain text stats files, e.g. to learn whether it's safe to enable ADSP.

To enable writing the stats file, set the stats_file parameter. The filter will write a tab-separated line for each verified message, except in a few cases, e.g. when the message's From is invalid; and one line for each DKIM signature. The format of the file is compatible with that used by opendkim-stats, so you can use that command for a human-readable version of its content.

Two utilities come with zdkimfilter for managing that file.

zdkimstats-wait signals (USR1) zdkimfilter to reopen stats
files and then waits until it can acquire a write lock on
the renamed file; for use in postrotate clauses.
Command line args:
 [-f] filename            renamed file to wait for
  -t seconds              timeout (default: wait forever)
  --help                  print this stuff and exit
  --version               print version string and exit
zdkimstats-anon reads stdin assuming that it is in the
OpenDKIM stats format, checks it, optionally appends counts
and/or ADSP failures to the respective files, and rewrites
data to stdout, possibly anonymized; for use before mailing
data to OpenDKIM (see http://www.opendkim.org/.)
Command line args:
 [-p] prefix              secret string for MD5 digests
  -x domain ...           don't anonymize data about messages
                          signed by these domains
  -a adsp-track           append a line for each adsp domain
  -c count-file           append there one line with totals
  --help                  print this stuff and exit
  --version               print version string and exit

At the time, etc/logrotate.example showed their usage. zdkimstats-anon may provide additional daily statistics, but its main role it to massage the stat file in order to get rid of minor format differences that may exist, anonymizing as needed.

The format of the stats file and the schema are described in the stats directory of the OpenDKIM distribution. The one-time setup required is as follows:

MySQL:
--as root user, create database and user
CREATE DATABASE opendkim;
CREATE USER 'opendkim'@'localhost' IDENTIFIED BY 'password';
GRANT ALL ON opendkim.* TO 'opendkim'@'localhost';

--as opendkim user,
mysql -u opendkim -ppassword opendkim < mkdb.mysql

The database can then be populated from rotated stats files like so:

zdkimstats-anon < /var/log/zdkimstats/stats.0 | opendkim-importstats -ppassword

Links
Signing software that has been mentioned on the courier-users mailing list, and related discussion

License

GPLv3

Keep in mind that DKIM relies on patented technology that may require a separate licence.

As far as software copyright is concerned, zdkimfilter is free software: you can redistribute it and/or modify it under the terms of the GNU General Public Licence as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

zdkimfilter is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public Licence for more details.

As an additional permission under GNU GPLv3 section 7,

If you modify zdkimfilter, or any covered work, by linking or combining it with software developed by The OpenDKIM Project and its contributors, containing parts covered by the applicable licence, the licensor of zdkimfilter grants you additional permission to convey the resulting work.

Copyright (C) 2010 Alessandro Vesely