Time to forbid non-subscribers from posting to the list?

Time to forbid non-subscribers from posting to the list?

[Alexey Zaytsev]

Hello, list.

What are the objections to makeing lkml and other lists at vget
subscribers-only?
Non-subscribers messages could still be allowed after moderation.
I get 1/4 of my spam from lkml, and see no benefit from allowing
non-subscribers to freely post to the list. If you are not subscribed,
you just have to wait until your mail gets approved by the moderator,
and it is not hard to subscribe anyway.

Re: Time to forbid non-subscribers from posting to the list?

[Jeff Garzik]

Alexey Zaytsev wrote:
> Hello, list.
> 
> What are the objections to makeing lkml and other lists at vget
> subscribers-only?
> Non-subscribers messages could still be allowed after moderation.
> I get 1/4 of my spam from lkml, and see no benefit from allowing
> non-subscribers to freely post to the list. If you are not subscribed,
> you just have to wait until your mail gets approved by the moderator,
> and it is not hard to subscribe anyway.

The kernel developers who need to keep the barrier to bug reports low 
like the current policy.

Get a good spam filter, I only get 1-2 pieces a day in my LKML folder.

	Jeff

RE: Time to forbid non-subscribers from posting to the list?

[David Schwartz]

> The kernel developers who need to keep the barrier to bug reports low
> like the current policy.
>
> Get a good spam filter, I only get 1-2 pieces a day in my LKML folder.
>
> 	Jeff

	How is everyone individually spam filtering better than one central spam
filter? More likelihood that at least one relevent person will get the bug
report? Certainly a single central spam filter can get more resources aimed
at it to make sure it doesn't suppress anything important.

	DS

Re: Time to forbid non-subscribers from posting to the list?

[jdow]

From: "David Schwartz" <davids@webmaster.com>
> 
>> The kernel developers who need to keep the barrier to bug reports low
>> like the current policy.
>>
>> Get a good spam filter, I only get 1-2 pieces a day in my LKML folder.
>>
>> Jeff
> 
> How is everyone individually spam filtering better than one central spam
> filter? More likelihood that at least one relevent person will get the bug
> report? Certainly a single central spam filter can get more resources aimed
> at it to make sure it doesn't suppress anything important.

If you have the luxury of the ability to write personalized rules and
whitelist entries for SpamAssassin it can become a startlingly good
filtering system. And you can tailor the filtering for individual 
sources with meta rules. Processing large numbers of messages through
BAYES and large numbers of rules gets quite time consuming, perhaps
more than vger might want to handle. Processing only a small number
of email accounts for trustworthy people allows one the luxury of
custom rules and individual BAYES filtering. Since a lot of what
one person might consider to be spam is another person's ham this is
a good thing. (And SURBL is a good thing, too. It is remarkably
reliable as long as you're not one of the first receiving a particular
piece of junk. The SpamAssassin "RulesEmporium" has some very nice
anti-spam rule sets, too.)

{^_^}   Joanne

Re: Time to forbid non-subscribers from posting to the list?

[Matthias Andree]

On Tue, 08 Aug 2006, jdow wrote:

> If you have the luxury of the ability to write personalized rules and
> whitelist entries for SpamAssassin it can become a startlingly good
> filtering system. And you can tailor the filtering for individual 
> sources with meta rules. Processing large numbers of messages through
> BAYES and large numbers of rules gets quite time consuming, perhaps

Bayes and distributed filtering in SpamAssassin, although it integrates
nicely with the scoring, is so painfully slow that I've ditched it after
a short test drive. Systems such as bogofilter, spamprobe or qsf are way
faster - and can also look at tags that SpamAssassin (in local non-bayes
mode) may have added to the header.

-- 
Matthias Andree

Re: Time to forbid non-subscribers from posting to the list?

[Erik Mouw]

On Wed, Aug 09, 2006 at 10:51:49AM +0200, Matthias Andree wrote:
> Bayes and distributed filtering in SpamAssassin, although it integrates
> nicely with the scoring, is so painfully slow that I've ditched it after
> a short test drive. Systems such as bogofilter, spamprobe or qsf are way
> faster - and can also look at tags that SpamAssassin (in local non-bayes
> mode) may have added to the header.

Spamassassin is slow if you call it at every message cause the Perl
startup and init is quite expensive. If you run the spamassassin daemon
(spamd) and use the spamassassin client (spamc) it's a lot faster.

Over here it can cope with lots of mailing list traffic and even more
spam (>30000 messages a day on an Athlon 2400+).


Erik

-- 
+-- Erik Mouw -- www.harddisk-recovery.com -- +31 70 370 12 90 --
| Lab address: Delftechpark 26, 2628 XH, Delft, The Netherlands

RE: Time to forbid non-subscribers from posting to the list?

[Lee Revell]

On Tue, 2006-08-08 at 12:00 -0700, David Schwartz wrote:
> > The kernel developers who need to keep the barrier to bug reports low
> > like the current policy.
> >
> > Get a good spam filter, I only get 1-2 pieces a day in my LKML folder.
> >
> > 	Jeff
> 
> 	How is everyone individually spam filtering better than one central spam
> filter? More likelihood that at least one relevent person will get the bug
> report? Certainly a single central spam filter can get more resources aimed
> at it to make sure it doesn't suppress anything important.

There IS one central spam filter already, but apparently a 100:1
ham:spam ratio is not good enough for some people.  Do you have any idea
what this list would look like if there were no filtering at all?

At some point you just have to say it's good enough, that to filter any
more aggressively would entail an unacceptable risk of false positives,
and that the small minority for whom a single spam is intolerable can do
their own filtering.

Lee

Re: Time to forbid non-subscribers from posting to the list?

[Folkert van Heusden]

> > The kernel developers who need to keep the barrier to bug reports low
> > like the current policy.
> > Get a good spam filter, I only get 1-2 pieces a day in my LKML folder.
> 	How is everyone individually spam filtering better than one central spam
> filter? More likelihood that at least one relevent person will get the bug
> report? Certainly a single central spam filter can get more resources aimed
> at it to make sure it doesn't suppress anything important.

What about just using the spamhaus.org blocklist at vger? Stops quite a
bit of spam over here (http://keetweej.vanheusden.com/nspam_graph.png).


Folkert van Heusden

www.vanheusden.com/multitail - multitail is tail on steroids. multiple
               windows, filtering, coloring, anything you can think of
----------------------------------------------------------------------
Phone: +31-6-41278122, PGP-key: 1F28D8AE, www.vanheusden.com

Re: Time to forbid non-subscribers from posting to the list?

[Matti Aarnio]

On Wed, Aug 09, 2006 at 04:34:30PM +0200, Folkert van Heusden wrote:
> > > The kernel developers who need to keep the barrier to bug reports low
> > > like the current policy.
> > > Get a good spam filter, I only get 1-2 pieces a day in my LKML folder.
> > 	How is everyone individually spam filtering better than one central spam
> > filter? More likelihood that at least one relevent person will get the bug
> > report? Certainly a single central spam filter can get more resources aimed
> > at it to make sure it doesn't suppress anything important.
> 
> What about just using the spamhaus.org blocklist at vger? Stops quite a
> bit of spam over here (http://keetweej.vanheusden.com/nspam_graph.png).

I have seen these lists classify major ISP relays as spam sources(*),
even classify VGER as one.  Their maintenance standards are varying,
some demand ridiculous things out of DNS zone SOA timers, some are
otherwise retarded in their "we are the world police, beware or be
sorry"..   and then they simply evaporate into the bit heaven.

(*) ISP user's main relays are spam fan-out sources way more often
than system keepers would like, but very few MTAs provide rate-limits
for anonymous ( = "non autenticated" ) users to keep a high-jacked
Windows machine from being effective spam-sources and utterly killing
the ISP relay..  (See "ASTA Recommendation".)
(Limiting spam-sending to 60 messages per hour of 240 rcpt per hour
can still get the relay to spam lists, but it won't flood internal
queues as badly as completely unlimited feed rates.)


Spamhouse and Spamcop have long(er) existence compared to most
DNS BLs, but still I am utterly worried...
("Many times burned, forever distrustful..")

> Folkert van Heusden
> www.vanheusden.com/multitail - multitail is tail on steroids. multiple
>                windows, filtering, coloring, anything you can think of

  /Matti Aarnio

Re: Time to forbid non-subscribers from posting to the list?

[D. Hazelton]

On Wednesday 09 August 2006 11:08, Matti Aarnio wrote:
> On Wed, Aug 09, 2006 at 04:34:30PM +0200, Folkert van Heusden wrote:
> > > > The kernel developers who need to keep the barrier to bug reports low
> > > > like the current policy.
> > > > Get a good spam filter, I only get 1-2 pieces a day in my LKML
> > > > folder.
> > >
> > > 	How is everyone individually spam filtering better than one central
> > > spam filter? More likelihood that at least one relevent person will get
> > > the bug report? Certainly a single central spam filter can get more
> > > resources aimed at it to make sure it doesn't suppress anything
> > > important.
> >
> > What about just using the spamhaus.org blocklist at vger? Stops quite a
> > bit of spam over here (http://keetweej.vanheusden.com/nspam_graph.png).
>
> I have seen these lists classify major ISP relays as spam sources(*),
> even classify VGER as one.  Their maintenance standards are varying,
> some demand ridiculous things out of DNS zone SOA timers, some are
> otherwise retarded in their "we are the world police, beware or be
> sorry"..   and then they simply evaporate into the bit heaven.

Agreed fully - I've seen this many times myself.

> (*) ISP user's main relays are spam fan-out sources way more often
> than system keepers would like, but very few MTAs provide rate-limits
> for anonymous ( = "non autenticated" ) users to keep a high-jacked
> Windows machine from being effective spam-sources and utterly killing
> the ISP relay..  (See "ASTA Recommendation".)
> (Limiting spam-sending to 60 messages per hour of 240 rcpt per hour
> can still get the relay to spam lists, but it won't flood internal
> queues as badly as completely unlimited feed rates.)

Postfix and several other MTA's (most notably sendmail) come configured to act 
as totally open MTA's. Most ISP's have locked down on this, but with zombie 
machines on the networks sending mail it doesn't work all that well. Rate 
limiting per-machine outgoing mail is a solution that can keep the servers 
functional, but I have very recently seen a lot of spam in my "spambox" email 
address that has massive TO, CC and BCC lists attached to evade the rate 
limiting.

I run spambot locally and have my local server set to filter spam at the 
server level, but I still see spam coming in from my other mail accounts - 
LKML is, by far, not the source of a lot of the spam that hit's my system. 
(Actually, the source of most of the spam are the two account I have with my 
broadband provider - and they claim to have the same level of spam-filters on 
their servers that I do locally)

The recent spate of "request to join list openbsd-xxxx" I saw seems to have 
been a childish prank from some BSD zealot. (That's a personal opinion - not 
a statement of fact. I don't need a flamewar) Seeing as this thread appears 
to have begun right after the first batch of those appeared in my inbox from 
vger I'm guessing that that might have been the "SPAM" the gentleman who 
started this thread was complaining about.

> Spamhouse and Spamcop have long(er) existence compared to most
> DNS BLs, but still I am utterly worried...
> ("Many times burned, forever distrustful..")

Exactly. My memory fails me currently, but there was also a big flap over a 
national ISP blocking all incoming mail from european domains. That was the 
sort of over-reaction you often see with DNS BL's and their kin. Truthfully, 
all a piece of mailing list software really needs is a good set of filters 
that each message must pass before being turned around and sent out to the 
people subscribed. A lot of spam has recently gotten extremely poor in it's 
spelling - so filters would miss stuff like this unless the filtering 
software itself had some excellent heuristics.

No, I didn't just propose adding filtering to LKML and VGER and then shoot it 
down - rather, I pointed out a flaw with the traditional filtering model. A 
solution to this would be to add filters that examine the mail headers 
themselves, since I am certain that some of the software used for sending 
spam does leave a signature in the headers.

One signature that comes to mind is the "Grab two words from a dictionary file 
and smash them together with a letter to form a valid looking name" tactic 
that seems to have become, lately, a preferred method for the zombie 
spambots.  To filter for those messages one just needs a good dictionary file 
(the Aspell American or British English dictionary comes to mind) and 
software that can match the two words against that list. I could *easily* 
write that in PERL in under 10 minutes.

These are suggestions only. I am most definately *not* offering to do anything 
- my plate is full and seems like it will remain that way for the forseeable 
future.

DRH

Re: Time to forbid non-subscribers from posting to the list?

[David Lang]

On Wed, 9 Aug 2006, D. Hazelton wrote:

>
> Postfix and several other MTA's (most notably sendmail) come configured to act
> as totally open MTA's.

sendmail shifted it's default from default open to default closed several years 
ago.

David Lang

Re: Time to forbid non-subscribers from posting to the list?

[Wakko Warner]

Matti Aarnio wrote:
> I have seen these lists classify major ISP relays as spam sources(*),
> even classify VGER as one.  Their maintenance standards are varying,
> some demand ridiculous things out of DNS zone SOA timers, some are
> otherwise retarded in their "we are the world police, beware or be
> sorry"..   and then they simply evaporate into the bit heaven.

Be nice if those that get listed and have real spammer problems actually
clean up.  Some don't care.

> Spamhouse and Spamcop have long(er) existence compared to most
> DNS BLs, but still I am utterly worried...
> ("Many times burned, forever distrustful..")

Personally, I use several RBLs, spamhaus and spamcop being 2 of those.

If it's possible, one suggesting I have is to filter out some HELO strings.
I have 3 of those in my suggestion:
1) strings consisting of either the IP or the hostname of the receiving
server (vger.kernel.org I assume)
2) any helo that does not contain a dot.  I do not know the impact on the
list if that were used
3) tom.com and 163.com.  I have noticed that these are very common in some
of the spam on the list and others that I am on.

These are only suggestions and I do not expect anyone to blindly use them. 
I do use these methods and others, however, I am the only user of the mail
server in which I'm sending this email.

-- 
 Lab tests show that use of micro$oft causes cancer in lab animals
 Got Gas???

Re: Time to forbid non-subscribers from posting to the list?

[Neil Brown]

On Tuesday August 8, alexey.zaytsev@gmail.com wrote:
> Hello, list.
> 
> What are the objections to makeing lkml and other lists at vget
> subscribers-only?

Yes.  Many.  I think this is in the FAQ. (hhmm.. just looked, it isn't exactly).

> Non-subscribers messages could still be allowed after moderation.
> I get 1/4 of my spam from lkml, and see no benefit from allowing
> non-subscribers to freely post to the list. If you are not subscribed,
> you just have to wait until your mail gets approved by the moderator,
> and it is not hard to subscribe anyway.

We want to barrier to posting to be low so that people will post bug
reports.  We want to hear about bug reports. really really.

Were you volunteering to be a moderator?  What sort of minimum delay
would you guarantee :-)

NeilBrown

Re: Time to forbid non-subscribers from posting to the list?

[Alexey Zaytsev]

On 8/8/06, Neil Brown <neilb@suse.de> wrote:
> On Tuesday August 8, alexey.zaytsev@gmail.com wrote:
> > Hello, list.
> >
> > What are the objections to makeing lkml and other lists at vget
> > subscribers-only?
>
> Yes.  Many.  I think this is in the FAQ. (hhmm.. just looked, it isn't exactly).
>
> > Non-subscribers messages could still be allowed after moderation.
> > I get 1/4 of my spam from lkml, and see no benefit from allowing
> > non-subscribers to freely post to the list. If you are not subscribed,
> > you just have to wait until your mail gets approved by the moderator,
> > and it is not hard to subscribe anyway.
>
> We want to barrier to posting to be low so that people will post bug
> reports.  We want to hear about bug reports. really really.
>
> Were you volunteering to be a moderator?  What sort of minimum delay
> would you guarantee :-)

If we had a large moderators group, we could do mail processing within minutes.
I'm not familiar with any mail list systems, is the mail validation
done with a web-interface?
If we could get incoming traffic delivered to the moderator's mailbox,
rather than appear on the web interface, it would be not hard for him
to ack/nack certain e-mails. We could even get the traffic split
betweem moderators, e.g. one gets N e-mails, afters processing them,
he gets an other N e-mails, and if he does not process theese e-mails
within M hours, they are sent to an other moderator. But here it's
only my imagination, maybe people with some mailing list
administration experience could give more ideas. And yes, if the
moderators group is large enough, I'm ready to come in.

>
> NeilBrown
>

Re: Time to forbid non-subscribers from posting to the list?

[Willy Tarreau]

On Tue, Aug 08, 2006 at 03:41:15PM +0400, Alexey Zaytsev wrote:
> On 8/8/06, Neil Brown <neilb@suse.de> wrote:
> >On Tuesday August 8, alexey.zaytsev@gmail.com wrote:
> >> Hello, list.
> >>
> >> What are the objections to makeing lkml and other lists at vget
> >> subscribers-only?
> >
> >Yes.  Many.  I think this is in the FAQ. (hhmm.. just looked, it isn't 
> >exactly).
> >
> >> Non-subscribers messages could still be allowed after moderation.
> >> I get 1/4 of my spam from lkml, and see no benefit from allowing
> >> non-subscribers to freely post to the list. If you are not subscribed,
> >> you just have to wait until your mail gets approved by the moderator,
> >> and it is not hard to subscribe anyway.
> >
> >We want to barrier to posting to be low so that people will post bug
> >reports.  We want to hear about bug reports. really really.

100% agreed. Many (I mean MANY) posters add a "PS: please CC me as I'm
not subscribed" line. This means whe might lose their reports while they're
often the most important ones.

Another problem is that many of us post from multiple addresses (work, home,
kernel.org, ...), and this must be taken into account to. Next, some spammers
use *valid* posters addresses, so those will pass through your filters anyway.
I'm already getting several spams a day pretending to emerge from several
LKML posters, so spamlists are also built from large mailing lists.

> >Were you volunteering to be a moderator?  What sort of minimum delay
> >would you guarantee :-)
> 
> If we had a large moderators group, we could do mail processing within 
> minutes.

We already have this large moderators group : all the users. I really
don't mind getting that small rate of spams in my LKML folder. Those
are very easy to identify at first glance, it will be harder when they
will begin with [PATCH] or things like this. You need 15 seconds to
hit the "D" key to remove 30 of them, that's OK.

> I'm not familiar with any mail list systems, is the mail validation
> done with a web-interface?
> If we could get incoming traffic delivered to the moderator's mailbox,
> rather than appear on the web interface, it would be not hard for him
> to ack/nack certain e-mails. We could even get the traffic split
> betweem moderators, e.g. one gets N e-mails, afters processing them,
> he gets an other N e-mails, and if he does not process theese e-mails
> within M hours, they are sent to an other moderator. But here it's
> only my imagination, maybe people with some mailing list
> administration experience could give more ideas. And yes, if the
> moderators group is large enough, I'm ready to come in.

Your description is generally interesting, but would require a lot of
work, and I'm not sure you'll find many kind people as you who would
devote a great part of their time to moderate such a list.

Regards,
Willy

Re: Time to forbid non-subscribers from posting to the list?

[Andi Kleen]

"Alexey Zaytsev" <alexey.zaytsev@gmail.com> writes:

> Hello, list.
> 
> What are the objections to makeing lkml and other lists at vget
> subscribers-only?

You would make bug reports impossible from normal people who 
don't want to subscribe fully. It would totally wreck the
development model.

-Andi

Re: Time to forbid non-subscribers from posting to the list?

[Alexey Zaytsev]

On 08 Aug 2006 13:23:50 +0200, Andi Kleen <ak@suse.de> wrote:
> "Alexey Zaytsev" <alexey.zaytsev@gmail.com> writes:
>
> > Hello, list.
> >
> > What are the objections to makeing lkml and other lists at vget
> > subscribers-only?
>
> You would make bug reports impossible from normal people who
> don't want to subscribe fully. It would totally wreck the
> development model.
If they don't want to subscribe, they can just report to the list as
usual, theyr mail will be only slightly delayed because of moderation.
We could even use some sort of white lists, if a user's mail was once
approved, all his further mail will be accepthed without moderation.

>
> -Andi
>

Re: Time to forbid non-subscribers from posting to the list?

[Jes Sorensen]

>>>>> "Alexey" == Alexey Zaytsev <alexey.zaytsev@gmail.com> writes:

Alexey> On 08 Aug 2006 13:23:50 +0200, Andi Kleen <ak@suse.de> wrote:
>> You would make bug reports impossible from normal people who don't
>> want to subscribe fully. It would totally wreck the development
>> model.
Alexey> If they don't want to subscribe, they can just report to the
Alexey> list as usual, theyr mail will be only slightly delayed
Alexey> because of moderation.  We could even use some sort of white
Alexey> lists, if a user's mail was once approved, all his further
Alexey> mail will be accepthed without moderation.

At 400-500 mails per day, who is going to handle the moderation? Sure
only a portion would be held back, but there's plenty other work to do
and we want fast turnaround for user bug reports, so if a user is
asked a question he/she can respond quickly with more details.

Moderation is a bad idea, right up there with C++ kernel modules, and
splitting the source code into smaller tarballs. For some reason they
all seem to get proposed again and again on a semi regular basis. If
people would just read the archives.....

Just install a proper spam filter like everyone else.

Jes

Re: Time to forbid non-subscribers from posting to the list?

[Alexey Zaytsev]

On 08 Aug 2006 07:55:33 -0400, Jes Sorensen <jes@sgi.com> wrote:
> >>>>> "Alexey" == Alexey Zaytsev <alexey.zaytsev@gmail.com> writes:
>
> Alexey> On 08 Aug 2006 13:23:50 +0200, Andi Kleen <ak@suse.de> wrote:
> >> You would make bug reports impossible from normal people who don't
> >> want to subscribe fully. It would totally wreck the development
> >> model.
> Alexey> If they don't want to subscribe, they can just report to the
> Alexey> list as usual, theyr mail will be only slightly delayed
> Alexey> because of moderation.  We could even use some sort of white
> Alexey> lists, if a user's mail was once approved, all his further
> Alexey> mail will be accepthed without moderation.
>
> At 400-500 mails per day, who is going to handle the moderation? Sure
> only a portion would be held back, but there's plenty other work to do
> and we want fast turnaround for user bug reports, so if a user is
> asked a question he/she can respond quickly with more details.

I'm sure not all the 500 mails are from non-subscribers, and, as I
told, we could split
the traffic between a large number of moderators.

>
> Moderation is a bad idea, right up there with C++ kernel modules, and
> splitting the source code into smaller tarballs. For some reason they
> all seem to get proposed again and again on a semi regular basis. If
> people would just read the archives.....

Ok, if this idea was discussed repeatedly, I'll giveup.

>
> Just install a proper spam filter like everyone else.
>
> Jes
>

Re: Time to forbid non-subscribers from posting to the list?

[Matti Aarnio]

On Tue, Aug 08, 2006 at 04:03:07PM +0400, Alexey Zaytsev wrote:
> On 08 Aug 2006 07:55:33 -0400, Jes Sorensen <jes@sgi.com> wrote:
> >>>>>> "Alexey" == Alexey Zaytsev <alexey.zaytsev@gmail.com> writes:
> >Alexey> On 08 Aug 2006 13:23:50 +0200, Andi Kleen <ak@suse.de> wrote:
> >>> You would make bug reports impossible from normal people who don't
> >>> want to subscribe fully. It would totally wreck the development
> >>> model.
> >Alexey> If they don't want to subscribe, they can just report to the
> >Alexey> list as usual, theyr mail will be only slightly delayed
> >Alexey> because of moderation.  We could even use some sort of white
> >Alexey> lists, if a user's mail was once approved, all his further
> >Alexey> mail will be accepthed without moderation.
> >
> >At 400-500 mails per day, who is going to handle the moderation? Sure
> >only a portion would be held back, but there's plenty other work to do
> >and we want fast turnaround for user bug reports, so if a user is
> >asked a question he/she can respond quickly with more details.
> 
> I'm sure not all the 500 mails are from non-subscribers, and, as I
> told, we could split the traffic between a large number of moderators.

We already have a rather well working filter that very rarely rejects
any valid emails.   That is around 300 to 900 per day (yes, we do collect
them all into daily message folders.)

Postings by non-members are rare - and even for those regular posters
that don't want to receive but do want to send we could have a simple
"subscribe to linux-posters -list" rule.  Nevertheless that does not
validate message poster person to be list member, just that the email
address in the "From:" header is known.   (I have been receiving for
years some email virus that claims to be coming from Alan Cox...)

So, having a well working web-based "approve these" filter (something
totally different from what Mailman does - that does not work with
tens of thousands of messages in the moderation queue) that gets only
a very small subset of things could work -- but whom would you trust
to be the moderator ?  If anybody could become one (for ease to get
people to do it), could they still do similar stupidities as these
recent "lets subscribe  linux-kernel  -list on many email lists"
attacks have tried to be ?

Having two-tiered moderation could work -- taboo-filters reject
pure junk, and that does not arrive to voluntary moderators at
all. Then rest can be processed by volunteers, and all that they
get (via dedicated linux-list-moderators  -list or some such) is
an email every 3 hours of "NN messages in moderation queue" along
with queue management tool URL.  No authentication, just queue
processing, and the tool shows only beginnings of ten oldest
messages in the queue with some smart javascript to optionally hide
99% of message headers (and to reveal them when somebody wants to
peek inside to know what is happening..)

That way the burden shouldn't be too bad (burden that will kill such
moderation effort) while likely valid messages still do get thru.

If they need help (horrible flood of junk passed thru pre-filters)
they can issue a help request to system postmasters to tune the
pre-filters and to use some heavy-duty tools to shove away most
junk from the queue.

Any volunteers to write couple perl tools ?


> >Just install a proper spam filter like everyone else.
> >Jes

/Matti Aarnio

Re: Time to forbid non-subscribers from posting to the list?

[jdow]

From: "Jes Sorensen" <jes@sgi.com>
>>>>>> "Alexey" == Alexey Zaytsev <alexey.zaytsev@gmail.com> writes:

> Just install a proper spam filter like everyone else.

There is a modest problem with that, Jes. What looks like spam in
most emails is often ham for this list, particularly with regards
to bug reports and patches.

With SpamAssasssin I have made an ad-hoc fix that works quite nicely
for the several "open" mailing lists I am on.

The first fix is a generic one I made once I was sure the BAYES
filter was well trained with ham and spam such that it seldom if
ever made a real mistake at the extremes of the score ranges. I
then increased the BAYES_95 rule score slightly. And have been
able to raise BAYES_99 to a full threshold of 5 score without
any false positives. YMMV here. But it does help on personal
installations.

First I developed a "feature" detection scheme for each of the lists.
Not all of them are straight forward for detection. Each list has its
own __L_RULE.... These rules are combined into a __LIST_RELAY meta
rule. Finally I create a series of meta rules that fire if BAYES_XX
and __LIST_RELAY are true. These raise the high BAYES scores and
lower the low BAYES scores.

Today I noticed one single spam out of 263 waiting messages when I
got to this mailing list. It was in a foreign language so I simply
fed it to BAYES to raise it from BAYES_80 to BAYES_99. Since it's
score was already 4.5 "it won't happen again."

I thought I'd pass along this "list amplifier" score trick for
SpamAssassin because it appears some folks here are seriously
troubled by the spam getting through.

(SUSE runs SpamAssassin. Unfortunately their BAYES is horridly mis-
trained. So it misfires left and right. I wish they'd fix that. Quite
often lists require some level of tweaking on the BAYES scores when
processing list messages.)

{^_^}   Joanne

Re: Time to forbid non-subscribers from posting to the list?

[Andi Kleen]

On Tuesday 08 August 2006 13:47, Alexey Zaytsev wrote:
> On 08 Aug 2006 13:23:50 +0200, Andi Kleen <ak@suse.de> wrote:
> > "Alexey Zaytsev" <alexey.zaytsev@gmail.com> writes:
> >
> > > Hello, list.
> > >
> > > What are the objections to makeing lkml and other lists at vget
> > > subscribers-only?
> >
> > You would make bug reports impossible from normal people who
> > don't want to subscribe fully. It would totally wreck the
> > development model.
> If they don't want to subscribe, they can just report to the list as
> usual, theyr mail will be only slightly delayed because of moderation.
> We could even use some sort of white lists, if a user's mail was once
> approved, all his further mail will be accepthed without moderation.

That would apply to all of my email - and I send a lot.  I'm not actually
subscribed but read it through a newsgroup.

Also I don't think it's practical because of the volume and the turnaround
time and there isn't really that much spam on l-k anyways.

-Andi

Re: Time to forbid non-subscribers from posting to the list?

[Alan Cox]

Ar Maw, 2006-08-08 am 13:23 +0200, ysgrifennodd Andi Kleen:
> You would make bug reports impossible from normal people who 
> don't want to subscribe fully. It would totally wreck the
> development model.

Only if applied without imagination.

Tag subject lines from non subscribes with [nonsub] and everyone can
then decide for themselves.

Re: Time to forbid non-subscribers from posting to the list?

[Willy Tarreau]

On Tue, Aug 08, 2006 at 03:39:16PM +0100, Alan Cox wrote:
> Ar Maw, 2006-08-08 am 13:23 +0200, ysgrifennodd Andi Kleen:
> > You would make bug reports impossible from normal people who 
> > don't want to subscribe fully. It would totally wreck the
> > development model.
> 
> Only if applied without imagination.
> 
> Tag subject lines from non subscribes with [nonsub] and everyone can
> then decide for themselves.

This looks like a very clever yet simple idea (if easy to implement at all) !
While I have no anti-spam and am not annoyed at all by the low spam rate on
LKML, I think this would make my cleaning operations even more effective.

Cheers,
Willy

Re: Time to forbid non-subscribers from posting to the list?

[Lee Revell]

On Tue, 2006-08-08 at 21:16 +0200, Willy Tarreau wrote:
> On Tue, Aug 08, 2006 at 03:39:16PM +0100, Alan Cox wrote:
> > Ar Maw, 2006-08-08 am 13:23 +0200, ysgrifennodd Andi Kleen:
> > > You would make bug reports impossible from normal people who 
> > > don't want to subscribe fully. It would totally wreck the
> > > development model.
> > 
> > Only if applied without imagination.
> > 
> > Tag subject lines from non subscribes with [nonsub] and everyone can
> > then decide for themselves.
> 
> This looks like a very clever yet simple idea (if easy to implement at all) !
> While I have no anti-spam and am not annoyed at all by the low spam rate on
> LKML, I think this would make my cleaning operations even more effective.

That would mean 8 fewer characters of useful information visible in the
subject line.

The spam ratio on LKML is so low that I think this cure would be worse
than the disease.  Why can't the minority of LKML readers who have
absolutely zero tolerance for spam just filter locally?

Lee

Re: Time to forbid non-subscribers from posting to the list?

[Kyle Moffett]

On Aug 08, 2006, at 16:25:16, Lee Revell wrote:
> The spam ratio on LKML is so low that I think this cure would be  
> worse than the disease.  Why can't the minority of LKML readers who  
> have absolutely zero tolerance for spam just filter locally?

Given the volume of valid email on the LKML it took me all of 3 days  
to get a fresh install of my mail client's bayes filter trained to a  
<5% false negative rate and a <0.5% false positive rate.   
Theoretically a similarly tuned bayes filter on vger with a web- 
interface for people to vote on spamminess would allow us to fix any  
remaining problems; but the spam levels are so low already; why bother?

Cheers,
Kyle Moffett

Re: Time to forbid non-subscribers from posting to the list?

[Jan Engelhardt]

>> > Tag subject lines from non subscribes with [nonsub] and everyone can
>> > then decide for themselves.
>> 
>> This looks like a very clever yet simple idea (if easy to implement at all) !
>> While I have no anti-spam and am not annoyed at all by the low spam rate on
>> LKML, I think this would make my cleaning operations even more effective.

+1


>That would mean 8 fewer characters of useful information visible in the
>subject line.

For those who use a graphical client this should not be a problem, pixels 
are plenty. For 80x25 readers, well, I can only suggest to use an index 
listing optimized to your needs. For example, the default pine listing 
left too few space for the subject for me on LKML, so I changed it to be 
like

(index-format=MSGNO STATUS SUBJECT FROM SMARTTIME SIZENARROW)

314   D   | | |-Re: Time to forbid non-subscri Andi Kleen          Tue02pm  (3K)
315     . | |-Re: Time to forbid non-subscribe Alan Cox            Tue03pm  (3K)
316     . |   |-Re: Time to forbid non-subscri Willy Tarreau       Tue09pm  (3K)
317     . |     |-Re: Time to forbid non-subsc Lee Revell          Tue04pm  (3K)
318       |       |-Re: Time to forbid non-sub Kyle Moffett        Tue05pm  (3K)
319   N   |-Re: Time to forbid non-subscribers David Miller        Tue03pm  (2K)

Since a lot of mails are already prefixed with something [], like [PATCH], 
I do not really mind if there's [nonsub], or if you like to keep it short, 
you can make that [NSUB], and it even outperforms [PATCH] wrt. length.

Then it would only take the key sequence ;ts[NSUB]<Enter>adx to kill all 
the nonsub messages. However, I would also be fine with an extra header in 
the mail header saying "this comes from a non-subscriber", the key seq in 
pine would be slightly bigger though, 
";thMajordomo-Info<Enter>nonsub<Enter>adx".


Jan Engelhardt
-- 

Re: Time to forbid non-subscribers from posting to the list?

[Jes Sorensen]

>>>>> "Lee" == Lee Revell <rlrevell@joe-job.com> writes:

Lee> On Tue, 2006-08-08 at 21:16 +0200, Willy Tarreau wrote:
>> This looks like a very clever yet simple idea (if easy to implement
>> at all) !  While I have no anti-spam and am not annoyed at all by
>> the low spam rate on LKML, I think this would make my cleaning
>> operations even more effective.

Lee> That would mean 8 fewer characters of useful information visible
Lee> in the subject line.

Or tag it with "X-Non-Subscriber: Yes" - then those who care can
filter it into a seperate inbox in a jiffy and we don't lose the
information on the subject line.

Cheers,
Jes

Re: Time to forbid non-subscribers from posting to the list?

[Geert Uytterhoeven]

On Wed, 9 Aug 2006, Jes Sorensen wrote:
> >>>>> "Lee" == Lee Revell <rlrevell@joe-job.com> writes:
> 
> Lee> On Tue, 2006-08-08 at 21:16 +0200, Willy Tarreau wrote:
> >> This looks like a very clever yet simple idea (if easy to implement
> >> at all) !  While I have no anti-spam and am not annoyed at all by
> >> the low spam rate on LKML, I think this would make my cleaning
> >> operations even more effective.
> 
> Lee> That would mean 8 fewer characters of useful information visible
> Lee> in the subject line.
> 
> Or tag it with "X-Non-Subscriber: Yes" - then those who care can
> filter it into a seperate inbox in a jiffy and we don't lose the
> information on the subject line.

Nice, and lkml will become much quieter if Andi's emails end up in a separate
inbox ;-)

SCNR...

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds

Re: Time to forbid non-subscribers from posting to the list?

[Athanasius]

On Tue, Aug 08, 2006 at 04:25:16PM -0400, Lee Revell wrote:
> On Tue, 2006-08-08 at 21:16 +0200, Willy Tarreau wrote:
> > On Tue, Aug 08, 2006 at 03:39:16PM +0100, Alan Cox wrote:
> > > Only if applied without imagination.
> > > 
> > > Tag subject lines from non subscribes with [nonsub] and everyone can
> > > then decide for themselves.
> > 
> > This looks like a very clever yet simple idea (if easy to implement at all) !
> > While I have no anti-spam and am not annoyed at all by the low spam rate on
> > LKML, I think this would make my cleaning operations even more effective.
> 
> That would mean 8 fewer characters of useful information visible in the
> subject line.

  There's no reason not to put that tag at the END of the subjectline.
Of course I'm assuming automatic filtering based on it.  Although in
that case it could as easily be a X-NonSub: Yes header.

-Ath
-- 
- Athanasius = Athanasius(at)miggy.org / http://www.miggy.org/
                  Finger athan(at)fysh.org for PGP key
	   "And it's me who is my enemy. Me who beats me up.
Me who makes the monsters. Me who strips my confidence." Paula Cole - ME

Re: Time to forbid non-subscribers from posting to the list?

[David Miller]

From: "Alexey Zaytsev" <alexey.zaytsev@gmail.com>
Date: Tue, 8 Aug 2006 15:07:47 +0400

> What are the objections to makeing lkml and other lists at vget
> subscribers-only?

If you don't like the current policy, you can feel free to unsubscribe
from the mailing list.

When you run your own mailing list, you can set up the posting rules
however you like.

Re: Time to forbid non-subscribers from posting to the list?

[Helge Hafting]

Alexey Zaytsev wrote:
> Hello, list.
>
> What are the objections to makeing lkml and other lists at vget
> subscribers-only?
> Non-subscribers messages could still be allowed after moderation.
> I get 1/4 of my spam from lkml, and see no benefit from allowing
> non-subscribers to freely post to the list. If you are not subscribed,
> you just have to wait until your mail gets approved by the moderator,
> and it is not hard to subscribe anyway. 
This doesn't really prevent spam - the spammers can
subscribe - post spam - unsubscribe
or subscribe from free webmail accounts that they don't care about
anyway.

The moderator setup have the problem that nobody wants to be
moderator.  Today, you are moderator of your own lkml mailbox,
and you hate that. :-/

I don't  filter my lkml mail, yet the spam is not a problem for me.
I receive lkml in a mailbox of its own, and the few spam messages
are obvious from the subject line alone.  So I never read them,
just delete them along with the mass deletion of other uninteresting
messages. (I.e. issues with hw I don't have.)

Most mail systems allow you to filter lkml into a mailbox of its own.
Mail readers like mozilla or mutt lets you skip dubious messages
easily, mozilla even have a filter of its own if you care to use it.

Helge Hafting

Re: Time to forbid non-subscribers from posting to the list?

[Bodo Eggert]

Lee Revell <rlrevell@joe-job.com> wrote:
>> On Tue, Aug 08, 2006 at 03:39:16PM +0100, Alan Cox wrote:

>> > Tag subject lines from non subscribes with [nonsub] and everyone can
>> > then decide for themselves.

> That would mean 8 fewer characters of useful information visible in the
> subject line.

I suggest introducing an X-LKML-Whitelisted: header. If you include a patch,
use a decent Subject ("[BUG]") or if you're subscribed, it's set to 'yes'.
-- 
Ich danke GMX dafür, die Verwendung meiner Adressen mittels per SPF
verbreiteten Lügen zu sabotieren.

http://david.woodhou.se/why-not-spf.html