LinuxReviws.org --get your your Linux knowledge
> Linux Reviews > Howtos >

Setting up spam scanning under postfix/Debian

Setting up spamassassin (SA) and razor to be called site-wide from postfix while also running RAVantivirus

Written by Chris Evans Email me via mailto.pl only
Version 2b. 1.vi.03 This work is licensed under a Creative Commons License. Creative Commons License

Acknowledgements and disclaimer

Without generous help from Wietse Venema, Jeffrey Taylor, Scott Fernandes and Justin Mason I don't think I'd have persevered and got his working. However, this document is my responsibility and errors in it are mine. For all that I hope it helps someone, I have to disclaim all indirect responsibility for problems anyone may hit following this as some sort of route map: this isn't my paying job and I don't earn enough to be worth suing!

Why write this?

Because I found it hard to achieve this. There were some things I did that weren't intelligent in retrospect. However, I'm not a very stupid person and I maintain a low-load ADSL server to run SMTP traffic and some Email lists and don't usually waste two or three days achieving something using my beloved open source systems. Trying to do this I found lists tended not to answer my big questions or to do so in ways that didn't really help me and that documentation on the www and in list archives was often confusing or contradictory.

Eventually I found it was really quite easy to get what I wanted but the problems I faced were real and have come about because software in this area (anti-spamming and security) has developed so fast. We have the usual problem that code writers mostly leave documentation to others or perhaps prefer not to document some things for fear that their documentation will be out of date so soon or because they believe in security through obscurity for such things. (I sympathise a little with STO but it won't work.)

Since I wrote this I've found a very impressive similar document but oriented to people wanting per-user filtering and using procmail to link to SA for that. It's by William Stearns and is at: http://www.opensourcedigest.com/issue2/spamassassin.html#spamassassin

First cautionary note

Get postfix running fine for you before you start any of this. Make sure it delivers internal and external Email as you want. If you have antiviral s'ware as I have (and if you haven't, what kind of a mail server are you...?), get that running fine too. Make a copy of master.cf somewhere safe (and perhaps all of /etc/postfix) before you start.

Second cautionary note

I have tried to separate the generic from the specific in what follows but will have failed. I am most unlikely to have succeeded for everyone. What is specific will go out of date rapidly as the programs involved evolve. My day job is as a researcher and psychotherapist and is busy. I'd love to answer questions on this document but make zero guarantee that I'll manage.

Third cautionary note

What follows reads as if it were the sequence you go through to do all this. It isn't. It is the sequence that makes clear what's going on. Right at the end there's the sequence you go through if you want to follow what I did in the quickest fashion. Don't cheat and jump straight there though!

What I wanted and my setup

I run a server behind a firewall which sees an ADSL connection to the internet. My server (www.psyctc.org) runs ecartis which handles some small Email lists for psychotherapy charities etc. The server also provides a very antiquated set of web pages and my and my family's own Email. Everything is Debian stable (currently Debian3.0, known to its friends as "Woody") on ancient i386 type hardware (mostly early pentium). I wanted to add a flexible antispam system (spamassassin (SA)) to replace or supplement my use of postfix's superb antispam ("UCE control") options, most of which I use but which I was failing to update fast enough manually to keep spam influx as low as I wanted. I was using RAVantivirus for my antiviral scanning and wanted to keep that.

O.K. Here's the listing of what I started with:

  • Debian 3.0 stable (Woody)
  • Postfix 1.1.11 from Debian stable distribution
  • RAVantivirus 8.4.2
  • spamassassin from Debian stable (2.2.0?)
  • razor1 from Debian stable (1.?)
  • all the various sensible tools (Emacs for editing for me), Perl, CPAN etc.

Summary of how postfix calls filters

3rd cautionary note: I believe this has changed a bit with the shift up to Postfix 2.x and I haven't yet gone there so bear that in mind.
I didn't really understand the complexity of postfix's multiple parts and queues. You have to go and read the postfix overview documentation and particularly http://www.postfix.org/big-picture.html to understand properly that it has two main incoming queues, one from the machine it's running on (pickup) and one (smtpd) which collects incoming smtp traffic from the internet. These can each be asked to feed messages to a filter and this is controlled by giving them arguments where they are controlled within /etc/postfix/master.cf. I had never really read the introduction to master.cf carefully having given up on it. It's actually very clear in retrospect. Here's Wietse Venema's comment block at the start of my master.cf:
#
# Postfix master process configuration file.  Each line describes how
# a mailer component program should be run. The fields that make up
# each line are described below. A "-" field value requests that a
# default value be used for that field.
#
# Service: any name that is valid for the specified transport type
# (the next field).  With INET transports, a service is specified as
# host:port.  The host part (and colon) may be omitted. Either host
# or port may be given in symbolic form or in numeric form. Examples
# for the SMTP server:  localhost:smtp receives mail via the loopback
# interface only; 10025 receives mail on port 10025.
#
# Transport type: "inet" for Internet sockets, "unix" for UNIX-domain
# sockets, "fifo" for named pipes.
#
# Private: whether or not access is restricted to the mail system.
# Default is private service.  Internet (inet) sockets can't be private.
#
# Unprivileged: whether the service runs with root privileges or as
# the owner of the Postfix system (the owner name is controlled by the
# mail_owner configuration variable in the main.cf file).
#
# Chroot: whether or not the service runs chrooted to the mail queue
# directory (pathname is controlled by the queue_directory configuration
# variable in the main.cf file). Presently, all Postfix daemons can run
# chrooted, except for the pipe and local daemons. The files in the
# examples/chroot-setup subdirectory describe how to set up a Postfix
# chroot environment for your type of machine.
#
# Wakeup time: automatically wake up the named service after the
# specified number of seconds.  Specify 0 for no wakeup. Presently,
# only the local pickup and queue manager daemons need a wakeup timer.
#
# Max procs: the maximum number of processes that may execute this
# service simultaneously. Default is to use a globally configurable
# limit (the default_process_limit configuration parameter in main.cf).
#
# Command + args: the command to be executed. The command name is
# relative to the Postfix program directory (pathname is controlled by
# the program_directory configuration variable). Adding one or more
# -v options turns on verbose logging for that service; adding a -D
# option enables symbolic debugging (see the debugger_command variable
# in the main.cf configuration file).
#
# In order to use the "uucp" message tranport below, set up entries
# in the transport table.
#
# In order to use the "cyrus" message transport below, configure it
# in main.cf as the mailbox_transport.
#
# SPECIFY ONLY PROGRAMS THAT ARE WRITTEN TO RUN AS POSTFIX DAEMONS.
# ALL DAEMONS SPECIFIED HERE MUST SPEAK A POSTFIX-INTERNAL PROTOCOL.

All that describes this crucial block of master.cf which defines the services that the postfix master daemon will maintain...

#
# ==========================================================================
# service type	private	unpriv	chroot	wakeup	maxproc	command + args
# 		(yes)	(yes)	(yes)	(never)	(50)
# ==========================================================================
smtp	  inet	n	-	-	-	-	smtpd
pickup	  fifo	n	-	-	60	1	pickup
cleanup	  unix	n	-	-	-	0	cleanup
qmgr	  fifo	n	-	-	300	1	qmgr
rewrite	  unix	-	-	-	-	-	trivial-rewrite
bounce	  unix	-	-	-	-	0	bounce
defer	  unix	-	-	-	-	0	bounce
smtp	  unix	-	-	-	-	-	smtp
showq     unix	n	-	-	-	-	showq
error     unix	-	-	-	-	-	error
local	  unix	-	n	n	-	-	local
cyrus	  unix	-	n	n	-	-	pipe
    flags=R user=cyrus argv=/usr/sbin/cyrdeliver -e -m ${extension} ${user}
uucp	  unix	-	n	n	-	-	pipe
    flags=F user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail ($recipient)
ifmail    unix  -       n       n       -       -       pipe
    flags=F user=ftn argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient)
bsmtp     unix  -       n       n       -       -       pipe
    flags=F. user=foo argv=/usr/local/sbin/bsmtp -f $sender $nexthop $recipient

flush	  unix	n	-	-	1000?	0	flush

The text is actually very clear but only when you've really started to understand the anatomy of postfix (recheck http://www.postfix.org/big-picture.html for the diagram of the anatomy). The other thing that was puzzling me (this was stupid) was that I couldn't find a file FILTER_README.gz which some people referred to. That's because Debian packages the postfix README files in a separate deb: postfix-doc which will install that excellent tutorial in /usr/share/doc/postfix if you get it from stable.

There are actually two main ways that postfix can send messages to programs such as spam or virus filters. One is to send them out to a port. This is the solution that RAVantivirus uses and isn't "filtering" in postfix terminology, just a different delivery system. RAV do this by running a daemon (ravpostfix listening for messages from postfix on 10025 and returning them on 10026. for Here's how RAV do the latter part of that in master.cf:

#rav-begin: RAV AntiVirus Configuration
127.0.0.1:10026 inet n - n - 100 smtpd
                -o content_filter=
		-o myhostname=dummy.domain.name
		-o local_recipient_maps=				
#rav-end

and how it does the other bit of that in main.cf:

#rav-begin: RAV AntiVirus Configuration added automatically by RAV installation for 8.4.2
content_filter=smtp:[127.0.0.1]:10025
#rav-end
I'm explaining this, and that it's quite different from how I've got spamassassin filtering, because I think I was unduly nervous that I might cause nasty interactions between whatever I did and what RAV was doing. Unless you tamper with the lines that RAV (or your own antiviral) have added, I don't think there's any risk.

O.K. so what is filtering? It involves defining a filter as a postfix service which you can give any arbitrary name to. However, "filter" seems a sensible name and here's the definition I've now got in master.cf with the acknowledgement.

## filter instruction based on http://advosys.ca/papers/postfix-filtering.html
filter    unix  -	n	n	-	-	pipe    
	flags=Rq user=filter argv=/home/filter/filter.sh -f ${sender} -- ${recipient}
## end of filter definition

I still don't really understand the flags but user=filter defines the user that will run the program and argv=/home/filter/filter.sh is the program itself which should be executable by that user.

O.K. so you need to adduser filter and edit /etc/passwd to make sure it's not a user who can log on (the /bin/false bit below:

filter:x:1012:1012:,,,:/home/filter:/bin/false
I've done chmod 750 /home/filter/filter.sh to get:
   4 -rwxr-x---    1 filter   filter         67 May  1 22:12 filter.sh
You're advised to make sure that filter is the only member of group filter as well to help keep things secure.

   (I know: I'll come to what /home/filter/filter.sh comprises in a minute!)

That filter is called by adding arguments calling it in the smptd line in master.cf:

# ==========================================================================
# service type	private	unpriv	chroot	wakeup	maxproc	command + args
# 		(yes)	(yes)	(yes)	(never)	(50)
# ==========================================================================
smtp	  inet	n	-	-	-	-	smtpd -o content_filter=filter:
pickup	  fifo	n	-	-	60	1	pickup

The arguments are that -o content_filter=filter: which invoke the filter definition I've just described and ensure that every message coming in through the smptd collection from the internet gets sent through the filter. (I think you should be able to do exactly the same for pickup but it doesn't work for me and I don't really need it so ...):

O.K. and here we have again the definition of the filter:

## filter instruction based on http://advosys.ca/papers/postfix-filtering.html
filter    unix  -	n	n	-	-	pipe    
	flags=Rq user=filter argv=/home/filter/filter.sh -f ${sender} -- ${recipient}
## end of filter definition

The last bits of the filter definition -f ${sender} -- ${recipient} are going to be used by /home/filter/filter.sh to ensure that the message passes on through the filter and back into postfix from the sender and to the recipient. /home/filter/filter.sh will use postfix's sendmail replacement /usr/sbin/sendmail to do this.

O.K. that brings us to what's in /home/filter/filter.sh. I have a terrible confession here: I found this somewhere on the www where it was clear the author was wanting it used, but I can't remember where so if it's yours, contact me and I'll acknowledge your generosity properly.

#!/bin/bash
/usr/sbin/spamc | /usr/sbin/sendmail -i "$@"
exit $?
There are other ways to pump things through spamassassin and I started out with another script which called spamassassin itself but, though that worked for me with SA 2.20, it didn't with SA 2.53 and I hadn't the energy to work out why as spamc/spamd is a better combination being more CPU and memory efficient than calling SA itself.

O.K. that brings us to spamassassin itself and spamc/spamd.

Debian stable packages SA 2.20 but it's clear that you'll get more successful antispam action if you move to the latest version which at the time of writing is 2.53. It wasn't that difficult to do this. If you go to the spamassassin web site there are clear instructions on dowloading which you can do either using CRAN (worked for me only after I upgraded CRAN to 1.70 I think) or by pulling down the zipped tarball, gunziping that, and tar -xvf extracting the files and directories.

There was a problem with this: the instructions suggest that you can test SA as follows:

  - Test it:

      spamassassin -t < sample-nonspam.txt > nonspam.out
      spamassassin -t < sample-spam.txt > spam.out

    Verify (using a text viewer, ie. "less" or "notepad") that nonspam.out
    has not been tagged as spam, and that spam.out has.  The files should
    contain the full text and headers of the messages, the "spam.out"
    message should be annotated with "****SPAM****" in the subject line
    and a report from SpamAssassin, and there should be no errors when you
    run the commands.

In fact the default behaviour of SA is no longer to use that "****SPAM****" but to add some X headers so what I see in the spam.out file is:

X-Spam-Flag: YES
X-Spam-Status: Yes, hits=9.8 required=5.0
        tests=DATE_IN_PAST_12_24,DRASTIC_REDUCED,HOME_EMPLOYMENT,
              INVALID_DATE,INVALID_MSGID,NO_REAL_NAME,ONCE_IN_LIFETIME,
              RAZOR2_CHECK,REMOVE_SUBJ,UNDISC_RECIPS
        version=2.53
X-Spam-Level: *********
X-Spam-Checker-Version: SpamAssassin 2.53 1.174.2.15-2003-03-30-exp
Using spamassassin -D -t < sample-spam.txt > spam.out will give you a reassuring debug listing of what SA is doing as it checks that message.

O.K. Now move spamassassin and spamc and spamd (you'll find them in a spamd subdirectory in the unpacked tarball) to /usr/sbin/ and from the spamd subdirectory edit debian-rc-script.sh to change the line:

# Change to one to enable spamd
ENABLED=0
to ENABLED=1 as it says! Then move it: mv debian-rc-script.sh /etc/default/spamd.conf and also do: mv debian-rc-script.sh /etc/init.d/spamassassin and make sure it's executable and owned by root and then excute it like any other init.d script:
/etc/init.d/spamassassin start and use ps aux | grep spam to check it's invoked spamd, the daemonised spamassassin, properly:
root     11548  0.0  1.8 19672 2316 ?        S    May01   0:07 /usr/sbin/spamd -c -d --pidfile=/var/run/spamd.pid
root     20818  0.0  0.3  1336  436 pts/3    S    11:40   0:00 grep spam
O.K. we now have a single instance of spamassassin running permanently within the spamd daemon and awaiting messages to process. Messages are passed to spamd to be processed by the other program the spamc/d pairing: spamc. The two actually have a chat with each other using a protocol rather like http (the details are in the documentation in the spamd subdirectory though the vital information about integrating spamc/d into postfix sadly aren't). This is more efficient than having to launch perl and an instance of spamassassin for every message. I'm running spamd as root as I think my box is pretty safe, you can find instructions to run it as something else (filter?) but it seems pretty safe to me as it switches down to whatever user (filter in my case) called spamc when it receives the call from spamc. Don't blame me if I'm inviting you to share bad practice here please.

So where does Razor fit in?

Vipul's razor, http://razor.sourceforge.net/ like SA, seems to me to be a brilliant but very poorly documented program that enables SA to call on an internet wide repository of spam fingerprints and other information. I feel pretty strongly that you would be well advised to allow SA to call razor which is its default expectation. Again, the version of razor in Debian stable is out of date, again, downloading from http://razor.sourceforge.net/ is easy and the instructions to unpack and install it are fine and worked for me (get the order of the two tarballs right). For me razor seemed to install where it was needed and razor-admin -register from my main non-root user identity gave me a very unmemorable razor username and password that razor then uses automatically (as far as I understand things).

In the last week or so since I got things running and had a bit more time I've realised that there may be a lot of politics here that maybe I should sleuth out at some point. I'm not sure whether razor2 is fully open source, certainly there's a rival product, pyzor, written in Python, which argues that razor and razor's communication protocols aren't open and that pyzor was therefore needed. I don't know anything about the changes between razor1 and razor2 but they are probably major. There is also one other distributed database: DCC and SA can be set up to call 0,1,2 or all three of razor2, pyzor and DCC. One day maybe I'll get my head around the differences.

Make sure your trusted senders don't get tested for spam

All configuration of SA is done in /etc/mail/spamassassin/local.cf. It makes good sense to whitelist your trusted senders like this:
# Add your own customisations to this file.  See 'man Mail::SpamAssassin::Conf'
# for details of what can be tweaked.
# 
rewrite_subject 0 # change this to one if you want the old ****SPAM**** subject line from SA
use_terse_report 0
whitelist_from "*bounce@psyctc.org"
whitelist_from "*firewall@psyctc.org"
whitelist_from "*store@psyctc.org"
...

I'm running behind a firewall. Will that make for problems with razor?

The official answer from the razor FAQ is:
Q: I have a firewall. What ports do I need to open in order for Razor2 to work?

    Outgoing TCP port 2703 (Razor2) and 7 (Echo). Razor2 uses TCP pings to discover what servers are closest to it.
I found that it was also trying to use 2702 outgoing so I opened that too. I've not seen it try to use anything else.

I'm running logcheck, this is clogging my logs

If you have a high throughput server you'll want to put something like this into /etc/logcheck/logcheck.ignore (change aaa\.bbb\.ccc\.ddd to your server's IP address!)
tcpspy\[.*\]: connect: user root, local aaa\.bbb\.ccc\.ddd:.*, remote 209\.204\.62\.150:2702
tcpspy\[.*\]: disconnect: user root, local aaa\.bbb\.ccc\.ddd:.*, remote 209\.204\.62\.150:2702

spamd\[.*\]: info: setuid to filter succeeded
spamd\[.*]: processing message <.*> for filter:1012\.
tcpspy\[.*\]: connect: user filter, local aaa\.bbb\.ccc\.ddd:.*, remote 216\.52\.3\.10:2703
tcpspy\[.*\]: disconnect: user filter, local aaa\.bbb\.ccc\.ddd:.*, remote 216\.52\.3\.10:2703
tcpspy\[.*\]: connect: user root, local 127\.0\.0\.1:783, remote 127\.0\.0\.1:.*
tcpspy\[.*\]: disconnect: user root, local 127\.0\.0\.1:783, remote 127\.0\.0\.1:.*
tcpspy\[.*\]: connect: user root, local 127\.0\.0\.1:.*, remote 127\.0\.0\.1:783
tcpspy\[.*\]: disconnect: user root, local 127\.0\.0\.1:.*, remote 127\.0\.0\.1:783
tcpspy\[.*\]: connect: user filter, local 127\.0\.0\.1:.*, remote 127\.0\.0\.1:783
tcpspy\[.*\]: disconnect: user filter, local 127\.0\.0\.1:.*, remote 127\.0\.0\.1:783
(I think the 783 communications are between spamd and spamc). You'll need different addresses if you're in a different part of the internet as razor is pretty sophisticated about finding the nearest servers and using those. You may also find it uses different addresses over time. I think I've added a couple more since I copied that lot in above.

So in what order should I do all this?

  1. Get postfix and any antiviral working fine for you
  2. Make backups of their config files, particularly /etc/postfix
  3. adduser filter and lock it out from logging on
  4. if you have installed spamassassin and razor from Debian stable, remove them
  5. get razor from its site of origin and install it
  6. get spamassassin from its site of origin and test it (remember note about instructions being out of date in docs.)
  7. configure spamc/spamd config and init script files into /etc/default/spamassin and /etc/init.d/spamassassin
  8. launch spamd and check it's running
  9. create /home/filter/filter.sh as per instructions above including setting its permissions
  10. edit master.cf on the smtpd line and below there to create the filter invocation
  11. add trusted senders to /etc/mail/spamassassin/local.cf (but not your own testing address!)
  12. test everything with messages
  13. if you have users you can't tell to enjoy SA working, read up how to give them an opt out (fortunately I didn't have to do this)
  14. brace yourself to have to change lots of this every time each program changes ... I will try to log my experiences below

Space reserved for the continuing saga

What's all this "Bayes" stuff?

Oh goody, something I know something about. The Reverend Bayes was an English cleric (19th Century I think) who came up with a rational basis of working out how probability estimates should logically change as new evidence is accumulated. SA can learn what you receive that's spam and what's not ("ham"). To achieve this you need to feed it messages to learn from using the program sa-learn.

I have to work on a Windoze system and have used Pegasus for my mail for something like 15 years. Fortunately I've been keeping a collection of spam and it wasn't that difficult to chose a large folder of non-spam. Dimitri Furman from the SA list pointed out something I'd overlooked which is that pegasus offers to create new folders in mbox format if you ask it (thanks Dimitri). That made it easy to create an mbox folder for ham and one for spam. They always have names in the form UNX*.MBX and it's fairly easy by date or size to work out which is ham and which spam. Transfer them to your Linux box and issue:
sa-learn --ham --mbox hamfilename
sa-learn --spam --mbox spamfilename
Seemed to consume a lot of CPU but worked fine.

One little thing I'm unsure of

I wasn't completely sure that SA was obeying the whitelist_from instructions I'd put in /etc/mail/spamassassin/local.cf. It may be it was obeying them but it was still reporting on all mail from those addresses (and still is) which seems odd. However, I had the impression from running it in debug mode that it might be reading /root/.spamassassin/user_prefs instead so I have made a logical link back to /etc/mail/spamassassin/local.cf for good measure.

Adding spam statistics monitoring using Spamstats

Spamstats is a perl script that does the difficult business of linking up the lines that spamd and postfix put into /var/log/mail.log to give you statistics on your spam and ham like this:


File /var/log/mail.log : from May 11 08:07:41 to May 18 07:55:22

Total number of emails processed by the spam filter : 1293
Number of spams : 18 ( 1.39%)
Number of clean messages : 1275 ( 98.61%)
Average message analysis time : 7.62 seconds
Average spam analysis time : 2.62 seconds
Average clean message analysis time : 7.72 seconds
Average message score : -6.39
Average spam score : 9.72
Average clean message score : -6.74
Total spam volume : 11 kbytes
Total clean volume : 1552 kbytes

Installing Spamstats, which was written by Vincent Deffontaines, was easy. Pull the file down from http://www.gryzor.com/tools. I used: wget http://www.gryzor.com/tools/spamstats-0.4b.tar.gz. Then unzip, tar -xvf and chmod 755 and you have a working executable.

There was one wrinkle that Vincent solved for me. We both run syslog-ng and this may be syslog-ng or Debian specific but you have to restart the spamd daemon each time syslog restarts itself otherwise it doesn't get spamd's logging. I've done this by adapting /etc/init.d/syslog-ng by making sure it invokes klogd and spamd:

#! /bin/sh
#
# skeleton	example file to build /etc/init.d/ scripts.
#		This file should be used to construct scripts for /etc/init.d.
#
#		Written by Miquel van Smoorenburg .
#		Modified for Debian GNU/Linux
#		by Ian Murdock .
#
# Version:	@(#)skeleton  1.8  03-Mar-1998  miquels@cistron.nl
#
# This file was customized by SZALAY Attila 

test -f /sbin/syslog-ng || exit 0

case "$1" in
  start)
	echo -n "Starting system logging: syslog-ng"
	start-stop-daemon --start --quiet --exec /sbin/syslog-ng
        /etc/init.d/klogd restart
	/etc/init.d/spamassassin restart
	echo "."
	;;
  stop)
	echo -n "Stopping system logging: syslog-ng"
	start-stop-daemon --stop --quiet --pidfile /var/run/syslog-ng.pid
	echo "."
	;;
  reload|force-reload)
	start-stop-daemon --stop --signal 1 --quiet --pidfile /var/run/syslog-ng.pid
        /etc/init.d/klogd restart
	/etc/init.d/spamassassin restart
	;;
  restart)
	echo -n "Stopping system logging: syslog-ng"
	start-stop-daemon --stop --quiet --pidfile /var/run/syslog-ng.pid
	echo "."
	sleep 1
	echo -n "Starting system logging: syslog-ng"
	start-stop-daemon --start --quiet --exec /sbin/syslog-ng
        /etc/init.d/klogd restart
	/etc/init.d/spamassassin restart
	echo "."
	;;
  *)
	echo "Usage: /etc/init.d/syslog-ng {start|stop|restart|reload|force-reload}" >&2
	exit 1
	;;
esac

exit 0


I've also invoked Spamstats from cron (in everything that follows, you'll have to tweak at least bits that read xxxx and yyyy to sensible directory locations for you!):
55 7 * * * /home/xxxxxxxxxxxx/scripts/cron_spamstats
and that script is:

#!/bin/sh
#
# script that calls spamstats to generate html spam statistics for previous day's mail log

numericdate=`date +%Y%m%d`

# first test to see that this hasn't already been done by /etc/logrotate.d/syslog-ng prerotate 
# call to my script that produces weekly and four-weekly sliding spam statistics
if ! [ -e /var/psyctc-ssl/spamstats/$numericdate.html ]
    then
	# make html file of results:
	/usr/bin/perl /home/xxxxxx/spamstats/spamstats-0.4b/spamstats0.4b.pl \
	    -file /var/log/mail.log -duration 86400 -html > /var/yyyyyy/spamstats/$numericdate.html
	# mail that to self
	/usr/bin/mpack -s "psyctc.org spamstats daily report $numericdate" \
	    /var/yyyyyy/spamstats/$numericdate.html chris@psyctc.org
fi

The "-duration 86400" gets you the stats for the previous 24 hours.

Finally, I decided that I wanted weekly and sliding 4 weekly stats too. All done with a script:

#!/bin/sh
#
# script that calls spamstats to generate html spam statistics for previous week's mail log
# called from prerotate line in /etc/logrotate.d/syslog-ng

numericdate=`date +%Y%m%d`

# process latest log looking just for last 24 hours (otherwise the routine crontab call to do this
# will get nonsense results as it will be looking at last few minutes if it ends up coming after the 
# rotation, and it's difficult to control time so it won't)
/usr/bin/perl /home/xxxxxxx/spamstats/spamstats-0.4b/spamstats0.4b.pl \
    -file /var/log/mail.log -duration 86400 -html > /var/yyyyyyy/spamstats/$numericdate.html
# mail that to self
/usr/bin/mpack -s "psyctc.org spamstats daily report $numericdate" \
    /var/yyyyyyy/spamstats/$numericdate.html chris@psyctc.org

# process latest log this time allowing all records through to get stats for last week
/usr/bin/perl /home/xxxxxxx/spamstats/spamstats-0.4b/spamstats0.4b.pl \
   -file /var/log/mail.log -html > /var/yyyyyyy/spamstats.weekly/$numericdate.html
# mail that to self
/usr/bin/mpack -s "psyctc.org spamstats weekly report $numericdate" \
   /var/yyyyyyy/spamstats.weekly/$numericdate.html chris@psyctc.org

# now concatenate and get the four week sliding stats
cat /var/log/mail.log /var/log/mail.log.1 /var/log/mail.log.2 /var/log/mail.log.3 > /var/log/mail.log.all
# and use
/usr/bin/perl /home/xxxxxxx/spamstats/spamstats-0.4b/spamstats0.4b.pl \
   -file /var/log/mail.log.all -html > /var/yyyyyyy/spamstats.4weekly/$numericdate.html
# mail that to self
/usr/bin/mpack -s "psyctc.org spamstats 4 week report $numericdate" \
   /var/yyyyyyy/spamstats.4weekly/$numericdate.html chris@psyctc.org

rm /var/log/mail.log.all



That gets called with a prerotate line in /etc/logrotate.d/syslog-ng:

/var/log/auth.log {
   rotate 4
   missingok
   notifempty
   weekly
   compress
}

/var/log/cron.log {
   rotate 4
   weekly
   missingok
   notifempty
   compress
}

/var/log/daemon.log {
   rotate 7
   weekly
   missingok
   notifempty
   compress
}

/var/log/debug {
   rotate 4
   weekly
   missingok
   notifempty
   compress
}

/var/log/kern.log {
   rotate 4
   weekly
   missingok
   notifempty
   compress
}

/var/log/lpr.log {
   rotate 4
   weekly
   missingok
   notifempty
   compress
}

/var/log/mail.err {
   rotate 4
   weekly
   missingok
   notifempty
   compress
}

/var/log/mail.info {
   rotate 4
   weekly
   missingok
   notifempty
   compress
}

/var/log/mail.log {
   rotate 4
   weekly
   missingok
   notifempty
   prerotate
	/home/xxxxxxx/scripts/spamstats.weekly
   endscript
}

/var/log/mail.warn {
   rotate 4
   weekly
   missingok
   notifempty
   compress
}

/var/log/messages {
   rotate 4
   weekly
   missingok
   notifempty
   compress
}

/var/log/ppp.log {
   rotate 4
   weekly
   missingok
   notifempty
   compress
}

/var/log/user.log {
   rotate 4
   weekly
   missingok
   notifempty
   compress
}

/var/log/uucp.log {
   rotate 4
   missingok
   notifempty
   weekly
   compress
}

/var/log/syslog {
   rotate 7
   daily
   compress
   postrotate
      /etc/init.d/syslog-ng reload >/dev/null
   endscript
}

 

Written by Chris Evans Email me via mailto.pl only
Version 2. 17.v.03

Maintained at (original):

Meet new people