[Greylist-users] Central whitelist database

[Eirik Oeverby]

Hi,

On Tue, 24 Jun 2003 16:42:09 -0500 (CDT)
Evan Harris <eharris at puremagic.com> wrote:

> > However there are many things that can be done before it gets to
> > that. As a preparation, I have been toying with the idea of setting
> > up some kind of central whitelist, a database that can be queried
> > and/or subscribed to by people running MTAs and who wishes to make
> > the burden upon their users as small as possible.
> 
[...]
> I haven't decided what way I'd like to do it.  I would rather stay
> away from something that requires connecting to a "master" server. 
> How to manage updates is also a problem, but I'll get to that in a
> bit.

I wouldn't like that either. But there are ways that could be
investigated. I am imagining a kind of organization 'owning' the
greylisting database, which anyone can query and/or replicate. One could
have several servers spread around for load-balancing, etc.
One possible approach would be that anyone who wishes to *submit*
information to the database would have to become a subscriber, with
verifiable contact information etc. to avoid spammers submitting bad
data. One could still employ some kind of 'voting' system, requiring
several subscribers to submit the same or similiar data in order for it
to be accepted into the database.
In principle, anyone among these subscribers could be a 'mirror' of the
master database, and appear on a list of hosts that can be used by new
users. Once this project matures, a kindo f init/install script could be
run for the greylisting engine on new MTAs that would select, from the
global list of database hosts, a number of hosts to use for querying -
for example based on pingtime, distance (hops), etc.

> In the meantime, I'd be happy to host a simple whitelist here
> alongside the paper and code, if people want to submit their entries
> to me offlist.
>
> I don't think an automatic system based on traffic patterns is a good
> idea. It is too easy for that type of system to be abused by spammers.
> What would stop them from installing a bunch of systems with some sort
> of fake client to submit "good" reports for their own IP's.

See above.

> Luckily, the sites that should need whitelisting should be pretty
> small, so checking them by hand shouldn't be much of a problem.

True, but those will represent a large number of the mails normal users
receive. To have them in a central whitelist database would make a *lot*
of mail pass right through all the greylists, eliminating the wait for a
lot of users a lot of the time.
For me, this initial delay for receiving mail is one of the biggest
problems I have with greylisting. And for most of the users I'm serving
through my MTAs this will most definitely be noticed. Someone used a
notion resembling 'using email as a bloated instant messenger system' or
something - that's how it's often used. With this initial delay, a
'conversation' via e-Mail won't even have a chance to get started ;)

> If there's a script to sync databases, then each MX can run from it's
> own DB as long as it syncs occasionally (every half hour?), and that
> issue is removed.

That's a good start, indeed. But before anyone starts building any kind
of central database, I believe (as a DBA of profession) the database
model should be revised and normalized. It should also (if it isn't
already) be designed in such a way that "any" database engine can be
used, MySQL, PostgreSQL, DB2, SQLServer (yea i know), Oracle, Informix,
SAPdb ... Give people choice.


/Eirik
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://lists.puremagic.com/pipermail/greylist-users/attachments/20030626/3921c5f4/attachment.bin