AmavisNewSQL 0.7.0

AmavisNewSQL - Amavis-New+SQL+SpamAssassin+Quarantine DB plugin for SquirrelMail

Author: Jared Watkins <jared <at-or-about> watkins.net>

If you like this program.. see my wishlist over at amazon... Anyone who gifts me something
will have their name added to the Acknowledgements section. Earn your place of glory today. =]

http://www.amazon.com/gp/registry/registry.html/102-4175228-0195360?%5Fencoding=UTF8&id=1J21CZXG6QJ44

The AmavisNewSQL SquirrelMail plugin lets users change a pre-defined set of
SpamAssassin settings when those settings are stored in a SQL DB
rather than a config file. It also allows you to use a quarantine database
for questionable mail. This plugin was designed with enterprise use in mind.




REQUIREMENTS

o SquirrelMail 1.4x

o PEAR Supported Database (tested with Postgres and MySQL)

o Pear Net/SMTP (for resending quarantine messages)
o Pear Log (for the process_bsmtp.php)

o htmlMimeMail Class (included, for digest mailings)

o This plugin can not use the native SquirrelMail options database
  so you will need a database just for these amavis-new settings.
  (see the included schema file)

o This plugin makes use of a quarantine system for messages that would normally
  be blocked.  Please use the provided process_bsmtp.php script from this
  package instead of the one that ships with spamassassin.. Please see the
  section on the Quarantine DB settings for how to configure it.





INSTALLATION

o These directions assume you already have a working install of amavis-new
  with spamassassin, SquirrelMail and the database installed.

o Install PEAR modules - Usually just 'pear install <module>'

o Unpack in the plugins directory of your SM installation.

o Copy config.php.dist to config.php and edit config.php to change the
  DSN and domain names for your webmail URL and digest mailings.

o Create the amavis user account in your database

o Create the database called amavis owned by the amavis user.

  NOTE - If using Postgres be sure to set the encoding to SQL_ASCII
         ie: CREATE DATABASE "amavis" WITH TEMPLATE = template1 ENCODING = 0;

o Import the schema file in the utils directory appropriate for your DB
   - Default schema files are provided for Postgres and MySQL

o Edit cleanquarantine.php and set the path info.

o Edit generatedigest.php and set the path info.

o If you don't want to run the cron scripts directly from the plugin directory
  or if you are running them from a different system

  - Create the directory /usr/local/amavisnewsql
  - Copy (or symlink) the following things to /usr/local/amavisnewsql
    - process_bsmtp.php
    - cleanquarantine.php
    - generatedigest.php
    - config.php
    - amavisnewsql.class.php

o Edit the process_bsmtp.php script and change the BASEINCLUDE setting to
  match your directory. This script must be located on the same system with your
  amavis-new install since it picks up quarantined messages and places
  them in the database.


o Set up cron jobs for cleanquarantine.php, process_bsmtp.php and
  generatedigest.php (suggested entries listed below)

o In amavisd.conf change the default sql statement for
  $sql_select_white_black_list to the following:

$sql_select_white_black_list = 'SELECT wb FROM wblist'.
  ' WHERE (rid=?) AND (wblist.email IN (%k))'.
  ' ORDER BY wblist.priority DESC';

o Also in amavisd.conf be sure to enable sql lookups with
  @lookup_sql_dsn to match your database settings. 

o Enable the amavisnewsql plugin.





QUARANTINE DB

This plugin supports a spam/virus quarantine where messages
can be stored and reviewed.. released or automaticly deleted.

Here is how it works...

1. If a message crosses the user settable upper threshold amavis-new
   will stash it in a quarantine folder (/var/virusmails by default)

2. A cron script included with this plugin (process_bsmtp.pl)
   will scan that directory and place all messages into the
   database.

3. Another cron script (generatedigest.php) will scan the database once a day
   and send a 'digest' email to each user showing what is in their
   quarantine.  The frequencyof this mailing is settable for each user.
   The digest will only contain the sender address, subject, date and SA score.

4. If mail sits in the quarantine long enough.. it will be cleaned up
   by a third cron script (cleanquarantine.php) which will remove
   everything older than a certain number of days.. settable for each user.




Suggested cron entries are:

*/5 * * * * root /usr/local/amavisnewsql/process_bsmtp.php
0 15 * * * root /usr/local/amavisnewsql/cleanquarantine.php
30 16 * * * root /usr/local/amavisnewsql/generatedigest.php

I use 4:30pm so that people may review their quarantine before the work day
is over and still have a high chance of seeing everything from that day.



AMAVIS-NEW - Other Config Changes

   o These are the settings/changes for amavisnew.conf file to make use
     of this plugin

     o Use the bsmtp methods:

        $virus_quarantine_method = "bsmtp:$QUARANTINEDIR/virus-%i-%n.msmtp";
        $spam_quarantine_method = "bsmtp:$QUARANTINEDIR/spam-%b-%i-%n.msmtp";

     $virus_quarantine_to = 'virus-quarantine';

     Other settings may be changed by setting various options in the policy
     table of the database.





DIGEST MAILINGS

If you want to let your users know what is sitting in their mail
quarantine from time to time.. use the generatedigest.php script.

Edit the first line of generatedigest.php to point to the location
of your php binary.

To format and send mail messages I use the htmlMimeMail class
which I include in this tarball. The original can be found at
http://www.phpguru.org/mime.mail.html

To install the class do this as root:

  mv htmlMimeMail-2.5.1 /usr/local/lib/php

  chown -R root.root /usr/local/lib/php/htmlMimeMail-2.5.1/

  ln -s /usr/local/lib/php/htmlMimeMail-2.5.1 /usr/local/lib/php/htmlMimeMail


The script will lookup the value of the 'digest' field in the 'users' table
from the amavis database. It will mail Monthly users on the first of the
month.. and Weekly users on Friday at whatever time you set in the cron job.





SUPPORTED SPAMASSASSIN SETTINGS

Any of the published settings for sql lookups may be used.. but my
web interface only gives access to change a subset of these. For
a more complete Admin tool please look for my upcoming <insert admin
tool name> which will allow more control.

o whitelist / blacklist
o tag2 Score Level
o Kill (Quarantine) Score Level






FAQ

Q: How is this plugin different from the other many SpamAssassin plugins for SM?

A:    This plugin is specifically for use with Amavis-New which calls
   SpamAssassin. The database schema is very different between the two setups
   and at the time of writing this there was not an amavis specific plugin
   out there.  This also allows you to use a database quarantine.

Q: How can I adjust the default policy settings?

A:    These defaults are set on the database columns.. using your admin
   tool of choice... simply change the default on each column.

Q: What if I don't want to use the quarantine database?

A:    If you do not wish to use the quarantine database you must change things in two
   places.   In the config file find variable $CONFIG["use_quarantine"] and change
   it to false. Also.. in the database.. on the policy table you will need to remove
   the default setting for column spam_quarantine_to and delete that column for policy id 2
   (from the default schema). That will ensure new users don't have their mail
   quarantined.

Q: Where can I find more about how/why things are done?

A: If not answered here.. check table comments in the database or read the source..
   My coding is not /that/ hard to follow.




KNOWN BUGS

I'm sure there are some.. but none my room full of smoking monkeys
have found so far.


TODO

Complete the code (and test) to support master/slave replicated databases.
  This is looking more difficult every time I look at it...  =\


See my (jimmydigital) other project(s) at freshmeat:
http://freshmeat.net/~jimmydigital/



ACKNOWLEDGMENTS

Thanks to the SquirrelMail team for builing such a great app and
for all the work they do to keep it running.

Thanks to various people of irc for helping me kick around ideas.

Thanks to all those who have sent in patches or suggested improvements.



I would like to acknowledge the following people for specific contributions
of code or goodies:

o Peter Collinson for help modifying process_bsmtp.pl (Used in previous versions)

o Stefano Ruberti for adding multi language support

o Phil Schilling -  Thanks for the DVD

o Wolfgang - Thanks for the two books!

. . . Your name here . . .





