Re: Stupid anti-spam testings...

Re: Stupid anti-spam testings...

[Marc MERLIN]

[Philip Hazel bcced]
[Blue-labs postmaster bcced too]

From: Matti Aarnio <matti.aarnio@zmailer.org>
> On Mon, Sep 02, 2002 at 04:28:37PM -0600, Andreas Dilger wrote:
> ...

Hi folks. Since I've been involved in code in exim, spamassassin, and was
the postmaster for sf.net, I thought I could give some info on all this.

> > > Folks,  when you deploy that kind of testers, DO VERIFY THAT THEY
> > > HAVE SANE CACHES!  A positive result shall be cached for at least
> > > two hours, a negative result shall be cached for at least 30 minutes.
> > 
> > Do you know if this is one of the default checks from spamassassin?
 
There  is no  code in  spamassassin  that does  SMTP callbacks  and I  would
advocate against it if someone were to think about adding it. Callbacks need
to be  done once,  at SMTP  time, not  later for  each receipient  when it's
already too late to bounce the mail if the env from is non reachable.
 
>   No idea.  I have seen these coming from Exim 4.10, Exim-something,
>   some sendmail milter (whatever that is), etc..

Ok.
So, exim, has  since version 3.2x (I think, might  be 3.3x) something called
SMTP callbacks (now SMTP callouts).
It's an optional  feature where the server does a  NULL SMTP connection back
to the  envelope and header  sender to make sure  that the servers  would be
willing to accept mail back for those.

Goals are:
1) do not accept mail that you cannot bounce (i.e. limit dead mail/double
   bounces in your spool)
2) make sure that mail you accept can be answered (bad header from)
3) Tell  people that  their envelope  is  broken (you  can only  do that  by
   rejecting the SMTP connection)
4) Refuse mail from morons who refuse null MAIL FROMs
5) and that's only a byproduct: refuse some junk mail/spam

If you want to know how it works in details, I've explained it here:
http://sourceforge.net/docman/display_doc.php?docid=6747&group_id=1 

I think someone may eventually have written an SMTP milter to do the same
(I suggested it to Eric Allman when he talked at SVLUG last year)

>   Apparently the idea (which I have thought of long ago, and rejected
>   as incomplete) has caught, and has multiple implementations...

Not too many yet, and it is turned off by default on exim, however many are
turning it on (for better or for worse)

>   I can easily reduce the load impact it causes to vger by
>   running the smtp server in "accept everything" mode without
>   analyzing local addresses for existence.  With a bit more
>   work I can throw in local cache..  (which I probably have to do..)
 
Hopefully it won't come to that.
 
> > If it is possible to track what tool is causing the problem and fixing
> > the default setup of that tool at the source, it will probably solve
> > 99% of the problems in one go (after the list knows to which version
> > they should upgrade).

As soon  as Philip Hazel initially  came up with the  first implementation I
know  in exim,  I  suggested to  him  that he  should  _really_ implement  a
positive and negative cache.
Worse, if the header  and envelope from are the same,  the current code will
still open  two SMTP connections back  to check them both,  even though they
are equal

Unfortunately Philip hasn't had the time to implement this yet, and others
(like me) haven't had the time to do it either and contribute code back.

>   Raise some noise all around, there are multiple implementations
>   of the idea.  Some even with syntactically invalid tester codes
>   (spaces put in place where they don't belong in RFC 821/2821);
>   "works with sendmail" is NOT synonymous to "is syntactically 
>   correct."

I only know of the exim implementation, and I believe that one is correct.

>   - a mister at blue-labs.org  runs some sendmail-milter which
>     does testing with invalid protocol syntax

I think I know who it is, he may have written it after he saw the code I put
on sf.net, or he may have gotten it from somewhere else, I'll let him know.

>   - usw-sf-list1.sourceforge.net  use probably their own code
>     usw-sf-fw2.sourceforge.net too...  possibly more systems there..

I don't remember the  network layout, and it may have  changed since I left,
but mail  goes to  one host  which then dispatches  it to  sf lists  or some
users.sf.net alias.
I did  change the  code a  little bit  on lists.sf.net  to do  an additional
postmaster callback  because we got  tired of  having to support  people who
filled our  spool with crap and  didn't accept Email for  postmaster when we
tried to contact them and work things out.
The code uses the existing SMTP connection,  so it issues an extra RCPT, but
that's very little extra load.

Note  too that  users.sf.net users  aren't really  supposed to  subscribe to
mailing  lists  with  those  aliases. The users.sf.net  alias  is  simply  a
redirector you can use and advertize for your projects.
Now in  practise, some  sf.net users are  apparently subscribing  to mailing
lists anyway, but the mail server wasn't designed with that in mind.

Please note too  that I am not  employed by VA Software anymore,  so you can
flame  me,  but  I  won't  be  able  to  do  much  to  fix  it. I  know  the
implementation is imperfect,  I was only postmaster for sf.net  on my "spare
time" since no one else was there to do  it, but it was the best we could do
to handle junk mail, bad bounces,  and spam send to several hundred thousand
users all running on mail server, and with no full time mail admin.
I know some things  that can be done, I'll list a few  here, but I never got
to upgrade to exim 4 with a  callback exclude setup (doable with exim 3 too)
because I  was layed off a  couple of weeks  before I'd have rolled  out the
implementation.

To contact the sf.net team, you can submit a sf.net support request here:
https://sourceforge.net/tracker/?func=add&group_id=1&atid=200001
If you  are not satisfied,  you can also contact  the site director  at this
address: pat@sf.net

>   - quetz.demon.co.uk tests from  Exim 4.10
>   - somebody.symons.net tests from Exim 3.35
 
I'm surprized you're not seeing more exim users.
 
>   Right now something like 5-7 different systems are doing it.
>   Try to imagine when all 3500 targets do it...  BRRRRR...
>   (Sure, VGER can handle it, no problem, but it is that much
>    wasted cycles, and network traffic...)

Ok, so here's my opinion on this since I studied callbacks extensively
before rolling them out on sf.net and hacking the code:

1) we've had  3 or 4  complaints in 2 years,  mostly from people  who didn't
   like the extra lines in their mail logs
2) The SMTP callback does take a few resources, but they are rather minimal.
   If you  send lots  of mail,  you typically have  the resources  to handle
   callbacks from remote sites.
3) Yes, it can get  bad for a list server when  many receipients start doing
   it


What should be done about this:

1) Positive caching with a cache of at least a day, if not a week, should be
   in every SMTP callback implementation

2) I'm looking at setting up a DNS list of list servers so that people don't
   do callbacks on them. With exim 4, you could then query that list and not
   do callbacks if the sending list server's IP is in that list.
   Something can probably be done to exclude envelope from domains too.
   If you are willing to help me with this, let me know

3) A 3rd, and lesser solution, is to agree on a mail header that tells
   receiving servers not to do callbacks.
   a) can obviously  be faked by spammers,  but I still think  it'd work for
      quite  a while:  spammers  who  are smart  enough  already fake  valid
      envelope and header froms, so they wouldn't have to add the header
   b) exim can trivially add a header on outgoing mail and check for it
      before doing  callbacks on the  receiving side,  but I don't  know how
      trivial  it is  for other  MTAs  of sending  list servers  to add  the
      header.

Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems & security ....
                                      .... what McDonalds is to gourmet cooking 
Home page: http://marc.merlins.org/   |   Finger marc_f@merlins.org for PGP key

Re: Stupid anti-spam testings...

[Alan Cox]

Matti, it seems a more productive approach would be to make sure that 
vger deliberately trips the idiot spam filters into failing. That way
people will rapidly leave the list or fix their systems.

Alan

Re: Stupid anti-spam testings...

[Marc MERLIN]

[BTW, I didn't specify, please CC me on answers, I'm not on lk anymore]

On Tue, Sep 03, 2002 at 09:05:16AM +0200, Rogier Wolff wrote:
> >    a) can obviously  be faked by spammers,  but I still think  it'd work for
> >       quite  a while:  spammers  who  are smart  enough  already fake  valid
> >       envelope and header froms, so they wouldn't have to add the header
> 
> This would quickly prevent detection of the problems that you
> intended to catch: Someone will blindly add the header, and before
> you know it, they fill in a bad "sender" somewhere....

That's what I just said, if they  bother to add the extra header, they could
just as well fake the envelope from  to point to some yahoo account, without
needing to set the said header.

On Tue, Sep 03, 2002 at 05:27:42AM -0700, Kevin P. Fleming wrote:
> This won't work; sender verify callouts (at least in Exim) are done at 
> MAIL FROM: time, I believe, in addition to after the headers have been 
> delivered.
 
In exim 3 yes, in exim 4, you can do what I explained
(sorry, I should have been specific about which version)

Details on the exim-users list if needed.
 
> I think caching is the way to go, no doubt. If noone else mentions that 
> they're working on it this week, I'll do so on Friday.

Caching is indeed the number 1 solution, a BL 

On Tue, Sep 03, 2002 at 04:14:04PM +0100, Alan Cox wrote:
> Matti, it seems a more productive approach would be to make sure that 
> vger deliberately trips the idiot spam filters into failing. That way
> people will rapidly leave the list or fix their systems.

1) exim callbacks are _not_ spam filters, they almost garantee that you only
   accept mail that you can reply to and/or bounce. If your envelope from
   points to  foohost.domain.tld with some non  routable IP, my only  way to
   tell you that I won't be able to  send you bounces is to refuse your mail
   at SMTP time.
   If you send mail from a non existant account (by mistake), same thing.
   SMTP callbacks just happen to also catch a lot of badly formatted spam,
   but that's not their main goal

2) There aren't many ways you can cause exim callbacks to fail without
   refusing real mail.
   (you can look at the callback rate in your logs and block the IPs, that's
   about it)

Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems & security ....
                                      .... what McDonalds is to gourmet cooking 
Home page: http://marc.merlins.org/   |   Finger marc_f@merlins.org for PGP key

Re: Stupid anti-spam testings...

[Henning P. Schmiedehausen]

Alan Cox <alan@lxorguk.ukuu.org.uk> writes:

>Matti, it seems a more productive approach would be to make sure that 
>vger deliberately trips the idiot spam filters into failing. That way
>people will rapidly leave the list or fix their systems.

Internet Mail Wars at eleven.

	Regards
		Henning

-- 
Dipl.-Inf. (Univ.) Henning P. Schmiedehausen       -- Geschaeftsfuehrer
INTERMETA - Gesellschaft fuer Mehrwertdienste mbH     hps@intermeta.de

Am Schwabachgrund 22  Fon.: 09131 / 50654-0   info@intermeta.de
D-91054 Buckenhof     Fax.: 09131 / 50654-20   

Stupid anti-spam testings...

[Matti Aarnio]

Quite a many of vger's recipients are doing return-path verification
testing for SMTP's MAIL FROM address.

I would not mind that, EXCEPT that those bloody stupid things don't
have any sane caches at all!    VGER is sending 300+ messages per
day to 3500+ recipients of linux-kernel list EVERY DAY, and every
outgoing message is now getting oodles of those probes!

Folks,  when you deploy that kind of testers, DO VERIFY THAT THEY
HAVE SANE CACHES!  A positive result shall be cached for at least
two hours, a negative result shall be cached for at least 30 minutes.

That would send a dozen back-probes towards vger from recipient
system, instead of present 300+ !


/Matti Aarnio -- who considers some cures worse than the original problem...

Re: Stupid anti-spam testings...

[Andreas Dilger]

On Sep 03, 2002  00:50 +0300, Matti Aarnio wrote:
> Quite a many of vger's recipients are doing return-path verification
> testing for SMTP's MAIL FROM address.
> 
> I would not mind that, EXCEPT that those bloody stupid things don't
> have any sane caches at all!    VGER is sending 300+ messages per
> day to 3500+ recipients of linux-kernel list EVERY DAY, and every
> outgoing message is now getting oodles of those probes!
> 
> Folks,  when you deploy that kind of testers, DO VERIFY THAT THEY
> HAVE SANE CACHES!  A positive result shall be cached for at least
> two hours, a negative result shall be cached for at least 30 minutes.

Do you know if this is one of the default checks from spamassassin?
I would imagine that a lot of people (including myself) have it
installed, so it is possible that it (or some other widely-used tool)
now does this sort of check out-of-the-box, and the people who are
installing them have no idea about the kind of load it generates on vger.
I doubt that there are a large number of people who are independently
misconfiguring their mail setup this way

If it is possible to track what tool is causing the problem and fixing
the default setup of that tool at the source, it will probably solve
99% of the problems in one go (after the list knows to which version
they should upgrade).

Cheers, Andreas
--
Andreas Dilger
http://www-mddsp.enel.ucalgary.ca/People/adilger/
http://sourceforge.net/projects/ext2resize/

Re: Stupid anti-spam testings...

[Matti Aarnio]

On Mon, Sep 02, 2002 at 04:28:37PM -0600, Andreas Dilger wrote:
...
> > Folks,  when you deploy that kind of testers, DO VERIFY THAT THEY
> > HAVE SANE CACHES!  A positive result shall be cached for at least
> > two hours, a negative result shall be cached for at least 30 minutes.
> 
> Do you know if this is one of the default checks from spamassassin?

  No idea.  I have seen these coming from Exim 4.10, Exim-something,
  some sendmail milter (whatever that is), etc..

  Apparently the idea (which I have thought of long ago, and rejected
  as incomplete) has caught, and has multiple implementations...

> I would imagine that a lot of people (including myself) have it
> installed, so it is possible that it (or some other widely-used tool)
> now does this sort of check out-of-the-box, and the people who are
> installing them have no idea about the kind of load it generates on vger.
> I doubt that there are a large number of people who are independently
> misconfiguring their mail setup this way

  I can easily reduce the load impact it causes to vger by
  running the smtp server in "accept everything" mode without
  analyzing local addresses for existence.  With a bit more
  work I can throw in local cache..  (which I probably have to do..)

> If it is possible to track what tool is causing the problem and fixing
> the default setup of that tool at the source, it will probably solve
> 99% of the problems in one go (after the list knows to which version
> they should upgrade).

  Raise some noise all around, there are multiple implementations
  of the idea.  Some even with syntactically invalid tester codes
  (spaces put in place where they don't belong in RFC 821/2821);
  "works with sendmail" is NOT synonymous to "is syntactically 
  correct."

  - a mister at blue-labs.org  runs some sendmail-milter which
    does testing with invalid protocol syntax
  - usw-sf-list1.sourceforge.net  use probably their own code
    usw-sf-fw2.sourceforge.net too...  possibly more systems there..
  - quetz.demon.co.uk tests from  Exim 4.10
  - somebody.symons.net tests from Exim 3.35

  Right now something like 5-7 different systems are doing it.
  Try to imagine when all 3500 targets do it...  BRRRRR...
  (Sure, VGER can handle it, no problem, but it is that much
   wasted cycles, and network traffic...)

> Cheers, Andreas
> Andreas Dilger

/Matti Aarnio

Re: Stupid anti-spam testings...

[Davide Libenzi]

On Tue, 3 Sep 2002, Matti Aarnio wrote:

> On Mon, Sep 02, 2002 at 04:28:37PM -0600, Andreas Dilger wrote:
> ...
> > > Folks,  when you deploy that kind of testers, DO VERIFY THAT THEY
> > > HAVE SANE CACHES!  A positive result shall be cached for at least
> > > two hours, a negative result shall be cached for at least 30 minutes.
> >
> > Do you know if this is one of the default checks from spamassassin?
>
>   No idea.  I have seen these coming from Exim 4.10, Exim-something,
>   some sendmail milter (whatever that is), etc..
>
>   Apparently the idea (which I have thought of long ago, and rejected
>   as incomplete) has caught, and has multiple implementations...

Personally i don't think this kind of tests are going to reduce the spam
that much but a simple lookup in the subscribers database might help
reducing the smtp-test traffic only for non-subscriber addresses.



- Davide

Re: Stupid anti-spam testings...

[Barry K. Nathan]

On Tue, Sep 03, 2002 at 02:32:30AM +0300, Matti Aarnio wrote:
>   some sendmail milter (whatever that is), etc..

I don't know if this helps, but it might:
http://www.milter.org/about.shtml

-Barry K. Nathan <barryn@pobox.com>

Re: Stupid anti-spam testings...

[Henning P. Schmiedehausen]

Matti Aarnio <matti.aarnio@zmailer.org> writes:

>  some sendmail milter (whatever that is), etc..

mail filter. The filtering API of sendmail 8.12+. You can e.g. attach
SpamAssassin here...

	Regards
		Henning

-- 
Dipl.-Inf. (Univ.) Henning P. Schmiedehausen       -- Geschaeftsfuehrer
INTERMETA - Gesellschaft fuer Mehrwertdienste mbH     hps@intermeta.de

Am Schwabachgrund 22  Fon.: 09131 / 50654-0   info@intermeta.de
D-91054 Buckenhof     Fax.: 09131 / 50654-20