[Greylist-users] Some more data.

[Scott Nelson]

I was asked for an update, so here goes;

I'm currently running a test of greylisting on some of my spamtrap
addresses.  
Caveats: the addresses are presumed to only receive spam, and they
are not a "random uncorrelated" sample, but it is hoped that the
correlations do not contaminate the results.  
Any of those assumptions could be wrong, and the possibility that 
I've made some horrible error should not be discounted.

I split 200 addresses into four group of 50 each,
by sorting alphabetically, and then alternating down the list.
I call them spam0, spam1, spam2 and spam3.
spam2 and spam3 were greylisting, spam0 and spam1 were not.
I'm using a very generous retry window - between 30 seconds and 36 hours.

>From Sun, 13 Jul 2003 23:59:13 to Fri, 1 Aug 2003 10:20:00 (about two weeks) 
Total RCPTs
Spam0 - 310
Spam1 - 494
Spam2 - 1330
Spam3 - 1900

Spam0 - 310 Accepted, 290 unique.
Spam1 - 494 Accepted, 404 unique.
Spam2 - 268 Accepted, 129 unique.
Spam3 - 120 Accepted, 111 unique.

spam0 - 0 Defered
spam1 - 0 Defered
spam2 - 1062 Defered
spam3 - 1780 Defered


Spam2 had 3 repeats within 30 seconds.
Spam3 had 5 repeats within 30 seconds.
There were no repeats over the 36 hour window.

(I've now switched spam1 to greylisting, and spam2 to not greylist,
 and when a similar amount of time has passed, I'll post those numbers.)



Parsed results;
Note that these addresses get considerably less spam than the
"average" mailbox, and the numbers are relatively small,
so the error margins are probably large.  Still, a good rule of
thumb is +/- the square root of the number which translates 
to +/- 10% for these numbers.

If we do /nothing/ but greylisting, it stops between 80-95% of all spam attempts.
Blocking semi-legitimate mailing lists (for example, equalamail.deliveroffers.com)
increases this to 91-97%.  In theory "unsubscribing" should give the same
benefit, but I have tried to unsubscribe from many of these jokers without
success.  

However, there appears to be a significant increase in the number of attempts,
and greylisting may only result in a 50-70% decrease in total spam volume.
I'm still trying to determine this, and lots more data needs to be
collected.


I'm still parsing the blacklist data, (I mistakenly assumed that they
all returned similar values for similar meanings) but it seems to be
about the same as without greylisting.  That is, roughly 10% of the
IPs that pass are listed, varying widely based on how agressive
the blacklist is in listing.

Since these are spam trap addresses, I have no idea what the
false positive rates these blacklists might be.


************

On a related subject, I've deployed greylisting on about 175 "real world"
servers.  So far I've received more compliments than complaints,
which is unusual for a change of any sort, even a bug fix.

The complaints:
Two sources needed to be whitelisted (one is running GroupWise, 
the other is running SLmail)  
One user was running a custom mailer to stuff his account, and
requested that greylisting be turned off.
7 users noticed that there was a delay on some of their mail,
but didn't seem concerned about it.

20 people noticed a reduction in the amount of spam, and were thankful.

The vast majority made no comment at all.



Scott Nelson <scott at spamwolf.com>