VGER's filters..

VGER's filters..

[Matti Aarnio]

VGER runs email processing with two layers of filters.
That we need any such thing is due to the sorry state
of email (all manner of spamming all around).

VGER has web-pages where various aspects of the system
are shown, _including_ present filter-rules in Majordomo.
  (  http://vger.kernel.org/  and onwards.. )


We have added also some synchronous filters into VGER's
MTA, so that incoming email gets rejected VERBOSELY to
its sender, when couple common cases are encountered.


  How do these filters work, then ?

Our filters are line-based one-match keyword trigger thingies.
Majordomo 1.x does not have any sort of scoring system.   Nor
have we had much interest in integrating something else, like
SpamAssassin, into our MTA environment to make scorings.


We are treating things like messages of TEXT/PLAIN type with
BASE64 encoded content, or messages with HTML in them  as
obfuscated and potentially spam.   Our rather simple filters
don't decode BASE64 (nor QP, but our MTA decodes that).


I recall that I have myself tried to use Hotmail,  and found
quite easily the setups so that my outgoing email will never
have HTML in them.  -- Current version of HM does not appear
to send HTML, nor did I find any settings for it.


Current Yahoo does not send HTML attachments either, unless poster
WANTS to send HTML by activating "Allow HTML tags" thingie at
right underside of the message body entry box.  Turning that
off will not send HTML.  Plain and simple.

(Making these tests took me about an hour, most of the time to get
thru all those foobar verifiers.)



With Yahoo I had at first immense problems to get any email from them,
as their SMTP email sender uses INVALID protocol:

<<-  MAIL FROM: <yahoo-dev-null@yahoo-inc.com>
->>  501 5.1.7 strangeness between ':' and '<': <yahoo-dev-null@yahoo-inc.com>

When you read really carefully RFC 821 / 2821 syntax about that,
you will see that it does not allow space in that place.
Sendmail does, and that has forced others to extend the syntax alike.

That happens only during the registration if alternate address is given.
Actual web-mail sending works as it should.

Yahoo is the only legitimate email source doing that of what I have seen.
(Tons of spammers do it, of course.)


/Matti Aarnio  --  one of   co-postmasters of vger.kernel.org

Re: VGER's filters..

[H. Peter Anvin]

Followup to:  <20030404181054.GT29167@mea-ext.zmailer.org>
By author:    Matti Aarnio <matti.aarnio@zmailer.org>
In newsgroup: linux.dev.kernel
> 
> With Yahoo I had at first immense problems to get any email from them,
> as their SMTP email sender uses INVALID protocol:
> 
> <<-  MAIL FROM: <yahoo-dev-null@yahoo-inc.com>
> ->>  501 5.1.7 strangeness between ':' and '<': <yahoo-dev-null@yahoo-inc.com>
> When you read really carefully RFC 821 / 2821 syntax about that,
> you will see that it does not allow space in that place.
> Sendmail does, and that has forced others to extend the syntax alike.
> 

Sendmail, and a whole bunch of other mailers, have taken the more
liberal approach of allowing any RFC 822-compliant address in this
place (which is a *lot* more liberal than an RFC 821-compliant
reverse-path.)  This is consistent with the "be liberal in what you
accept, conservative in what you send" philosophy of network
interoperability.

I suspect in Sendmail it naturally falls out of using a single set of
canonicalization rules for all syntax.

	-hpa
-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
Architectures needed: ia64 m68k mips64 ppc ppc64 s390 s390x sh v850 x86-64

Re: VGER's filters..

[Matti Aarnio]

On Fri, Apr 04, 2003 at 12:36:20PM -0800, H. Peter Anvin wrote:
> By author:    Matti Aarnio <matti.aarnio@zmailer.org>
> > <<-  MAIL FROM: <yahoo-dev-null@yahoo-inc.com>
> > ->>  501 5.1.7 strangeness between ':' and '<': <yahoo-dev-null@yahoo-inc.com>
...
> Sendmail, and a whole bunch of other mailers, have taken the more
> liberal approach of allowing any RFC 822-compliant address in this
> place (which is a *lot* more liberal than an RFC 821-compliant
> reverse-path.)  This is consistent with the "be liberal in what you
> accept, conservative in what you send" philosophy of network
> interoperability.

Definitely.  VGER is running fairly liberal mode, while some
other systems I run in extremely strict mode.   As a result,
vger lets in spams, which could be blocked by running strict.

> I suspect in Sendmail it naturally falls out of using a single set
> of canonicalization rules for all syntax.

Nope, the protocol line parser in original version was simple,
it reused same code for "MAIL FROM:" and "RCPT TO:" as for
"From:" and "To:", therefore with old sendmails you could
do in SMTP:
   MAIL FROM: < foo@bar >
   MAIL FROM:   foo@bar
and I think even:
   MAIL FROM: Example User foo@bar

There really was no SMTP protocol parser, there was just something
resembling it on surface.   (And tons of security problems...)

> 	-hpa

/Matti Aarnio

Re: VGER's filters..

[H. Peter Anvin]

Followup to:  <20030404210530.GV29167@mea-ext.zmailer.org>
By author:    Matti Aarnio <matti.aarnio@zmailer.org>
In newsgroup: linux.dev.kernel
> 
> > I suspect in Sendmail it naturally falls out of using a single set
> > of canonicalization rules for all syntax.
> 
> Nope, the protocol line parser in original version was simple,
> it reused same code for "MAIL FROM:" and "RCPT TO:" as for
> "From:" and "To:",...
> 

It's that what I said?

	-hpa
-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
Architectures needed: ia64 m68k mips64 ppc ppc64 s390 s390x sh v850 x86-64