why does reiserfs list get so much spam?

why does reiserfs list get so much spam?

[Tomasz Chmielewski]

Why does reiserfs list get so much spam?

When I look at the archives, the first impression is that 90% of posts 
on the list are spam messages...

Can't you do something about that?

Lists of other Open Source projects get spam only occasionally.

-- 
Tomek
http://wpkg.org

Re: why does reiserfs list get so much spam?

[Ingo Bormuth]

On 2005-09-16 12:18, Tomasz Chmielewski wrote:
> When I look at the archives, the first impression is that 90% of posts 
> on the list are spam messages...
> Can't you do something about that?

Locally I pipe all mail through spamassassin and hardly see any spam.
I think that should also be applicable for the archive 
(which really is hard to read by now).



-- 
Ingo Bormuth,  voicebox & telefax: +49-12125-10226517        '(~o o~)'
GnuPG key 86326EC9 at http://ibormuth.efil.de/contact  ----ooO--(.)--Ooo----

Re: why does reiserfs list get so much spam?

[Tomasz Chmielewski]

Ingo Bormuth schrieb:
> On 2005-09-16 12:18, Tomasz Chmielewski wrote:
> 
>>When I look at the archives, the first impression is that 90% of posts 
>>on the list are spam messages...
>>Can't you do something about that?
> 
> 
> Locally I pipe all mail through spamassassin and hardly see any spam.
> I think that should also be applicable for the archive 
> (which really is hard to read by now).

The real reason is that it allows posts from non-registered users...
This should really be fixed.
Spam in the archive looks lame :)

I just sent this mail from an email which I never registered to 
send/receive mail from this list.

----------------------------------------------------------------------
TOUR DE POLOGNE: oficjalny serwis >>> http://link.interia.pl/f18b5

Re: why does reiserfs list get so much spam?

[Lexington Luthor]

Tomasz Chmielewski wrote:
> 
> The real reason is that it allows posts from non-registered users...
> This should really be fixed.
> Spam in the archive looks lame :)
> 

Most lists I use allow me to post without subscribing (I use gmane 
mail->news gateway), but this list has the most spam of them all.

I think some simple server-side SA filtering will be enough to catch 
almost all spam, without requiring all posters to register.

LL

Re: why does reiserfs list get so much spam?

[Grzegorz Kulewski]

On Fri, 16 Sep 2005, Tomasz Chmielewski wrote:

> Ingo Bormuth schrieb:
>>  On 2005-09-16 12:18, Tomasz Chmielewski wrote:
>> 
>> > When I look at the archives, the first impression is that 90% of posts on 
>> > the list are spam messages...
>> > Can't you do something about that?
>>
>>
>>  Locally I pipe all mail through spamassassin and hardly see any spam.
>>  I think that should also be applicable for the archive (which really is
>>  hard to read by now).
>
> The real reason is that it allows posts from non-registered users...
> This should really be fixed.
> Spam in the archive looks lame :)
>
> I just sent this mail from an email which I never registered to send/receive 
> mail from this list.

Don't do that. *Many* people subscribed to LKML are posting here and often 
Namesys people CC this list in their posts. This way replies to such 
posts will be lost. It will cause major pain in following such threads.

Instead there are really good techniques to fight spam while not causing 
harm to legitimate users. For example checking if sending host is allowed 
to post mails from specified domain, different bayes filters (including 
adaptive ones), graylisting and so on. I will bet that only first of these 
ideas will kill 50% of spam here (including this from *polcom.net that I 
reported to Hans several months ago - of course this spam does not come 
from our servers but is using fake domain - easy to filter out).


Grzegorz Kulewski

Re: why does reiserfs list get so much spam?

[Tomasz Chmielewski]

Grzegorz Kulewski schrieb:
> On Fri, 16 Sep 2005, Tomasz Chmielewski wrote:
> 
>> Ingo Bormuth schrieb:
>>
>>>  On 2005-09-16 12:18, Tomasz Chmielewski wrote:
>>>
>>> > When I look at the archives, the first impression is that 90% of 
>>> posts on > the list are spam messages...
>>> > Can't you do something about that?
>>>
>>>
>>>  Locally I pipe all mail through spamassassin and hardly see any spam.
>>>  I think that should also be applicable for the archive (which really is
>>>  hard to read by now).
>>
>>
>> The real reason is that it allows posts from non-registered users...
>> This should really be fixed.
>> Spam in the archive looks lame :)
>>
>> I just sent this mail from an email which I never registered to 
>> send/receive mail from this list.
> 
> 
> Don't do that. *Many* people subscribed to LKML are posting here and 
> often Namesys people CC this list in their posts. This way replies to 
> such posts will be lost. It will cause major pain in following such 
> threads.
> 
> Instead there are really good techniques to fight spam while not causing 
> harm to legitimate users. For example checking if sending host is 
> allowed to post mails from specified domain, different bayes filters 
> (including adaptive ones), graylisting and so on. I will bet that only 
> first of these ideas will kill 50% of spam here (including this from 
> *polcom.net that I reported to Hans several months ago - of course this 
> spam does not come from our servers but is using fake domain - easy to 
> filter out).

yeah, but it is to be used by the end-user.

the archive will be still filled with spam.

not everyone who wants to know about reiser subscribes to the list; most 
of the people would just use the archives.


-- 
Tomek
http://wpkg.org

Re: why does reiserfs list get so much spam?

[Gregory Maxwell]

On 9/16/05, Tomasz Chmielewski <mangoo@mch.one.pl> wrote:
> yeah, but it is to be used by the end-user.
> 
> the archive will be still filled with spam.
> 
> not everyone who wants to know about reiser subscribes to the list; most
> of the people would just use the archives.

I actually subbed to the list because I was frustrated by spam in the archive.
Whomever runs the archive should at least just pipe it through spam
assassin. If some legitimate messages are lost from the web archive it
could not be worse than what we have now, an archive made nearly
useless from excessive spam.

Re: why does reiserfs list get so much spam?

[Łukasz Mierzwa]

Dnia Fri, 16 Sep 2005 14:40:22 +0200, Gregory Maxwell <gmaxwell@gmail.com> napisa³:

> On 9/16/05, Tomasz Chmielewski <mangoo@mch.one.pl> wrote:
>> yeah, but it is to be used by the end-user.
>>
>> the archive will be still filled with spam.
>>
>> not everyone who wants to know about reiser subscribes to the list; most
>> of the people would just use the archives.
>
> I actually subbed to the list because I was frustrated by spam in the archive.
> Whomever runs the archive should at least just pipe it through spam
> assassin. If some legitimate messages are lost from the web archive it
> could not be worse than what we have now, an archive made nearly
> useless from excessive spam.
>

If You will look in the headers of messages that You get from this list You will see that there is spamassassin running on thebsh.namesys.com, it's just that it is not configured good enough.

£ukasz Mierzwa

Re: why does reiserfs list get so much spam?

[Grzegorz Kulewski]

On Fri, 16 Sep 2005, Tomasz Chmielewski wrote:

> Grzegorz Kulewski schrieb:
>>  On Fri, 16 Sep 2005, Tomasz Chmielewski wrote:
>> 
>> >  Ingo Bormuth schrieb:
>> > 
>> > >   On 2005-09-16 12:18, Tomasz Chmielewski wrote:
>> > > 
>> > > >  When I look at the archives, the first impression is that 90% of 
>> > >  posts on > the list are spam messages...
>> > > >  Can't you do something about that?
>> > > 
>> > > 
>> > >   Locally I pipe all mail through spamassassin and hardly see any spam.
>> > >   I think that should also be applicable for the archive (which really 
>> > >   is
>> > >   hard to read by now).
>> > 
>> > 
>> >  The real reason is that it allows posts from non-registered users...
>> >  This should really be fixed.
>> >  Spam in the archive looks lame :)
>> > 
>> >  I just sent this mail from an email which I never registered to 
>> >  send/receive mail from this list.
>>
>>
>>  Don't do that. *Many* people subscribed to LKML are posting here and often
>>  Namesys people CC this list in their posts. This way replies to such posts
>>  will be lost. It will cause major pain in following such threads.
>>
>>  Instead there are really good techniques to fight spam while not causing
>>  harm to legitimate users. For example checking if sending host is allowed
>>  to post mails from specified domain, different bayes filters (including
>>  adaptive ones), graylisting and so on. I will bet that only first of these
>>  ideas will kill 50% of spam here (including this from *polcom.net that I
>>  reported to Hans several months ago - of course this spam does not come
>>  from our servers but is using fake domain - easy to filter out).
>
> yeah, but it is to be used by the end-user.

No - it should be filtered at the moment of delivery to mail list software 
before going to subscribed users or to archive. It is done in this way on 
LKML and on all other lists. It is done on this list too but something is 
probably misconfigured or not configured good enough.


> the archive will be still filled with spam.

No. Why? Spam should be dead before it is going anywhere from the list 
(including all potential archives).


> not everyone who wants to know about reiser subscribes to the list; most of 
> the people would just use the archives.

... or are posting or CCing this list without being subscribed

Exactly, so we should not limit posting to only subscribed users. I know 
that some stupid lists are doing so (for example acpi list IIRC) but this 
is pain for me (I am not subscribed) when I want to report yet another 
problem with acpi on my system or when I am replying to thread that is 
CCd to acpi list.


Grzegorz Kulewski

Re: why does reiserfs list get so much spam?

[Bruce Israel]

Here's my suggestion.  I don't know if Hans et al has the time for
it, but the simple solution would be to set the list up as moderated
and subscriber-only posting.  So when a list subscriber posts, it
goes through immediately.  If a non-subscriber posts (e.g. someone
from a different account, a cross-posted message thru LKML, a spam
message) then it doesn't go to the list but instead goes to the
moderator.  The moderator can skip it if it's spam or forward to the
list if it's not.  Further, the moderator can add the sender to a
whitelist so that further messages from the same person aren't
delayed. 

This has the cost of a slight delay with the first message from any
person to the list, while eliminating the spam and minimizing the
amount of maintenance and monitoring the moderator has to do.

Bruce


On Fri, Sep 16, 2005 at 02:46:34PM +0200, ?ukasz Mierzwa wrote:
> Dnia Fri, 16 Sep 2005 14:40:22 +0200, Gregory Maxwell <gmaxwell@gmail.com> 
> napisa?:
> 
> >On 9/16/05, Tomasz Chmielewski <mangoo@mch.one.pl> wrote:
> >>yeah, but it is to be used by the end-user.
> >>
> >>the archive will be still filled with spam.
> >>
> >>not everyone who wants to know about reiser subscribes to the list; most
> >>of the people would just use the archives.
> >
> >I actually subbed to the list because I was frustrated by spam in the 
> >archive.
> >Whomever runs the archive should at least just pipe it through spam
> >assassin. If some legitimate messages are lost from the web archive it
> >could not be worse than what we have now, an archive made nearly
> >useless from excessive spam.
> >
> 
> If You will look in the headers of messages that You get from this list You 
> will see that there is spamassassin running on thebsh.namesys.com, it's 
> just that it is not configured good enough.
> 
> ?ukasz Mierzwa
> 

Re: why does reiserfs list get so much spam?

[Łukasz Mierzwa]

Dnia Fri, 16 Sep 2005 15:52:17 +0200, Ingo Bormuth <ibormuth@efil.de> napisa³:

> On 2005-09-16 14:46, £ukasz Mierzwa wrote:
>>
>> If You will look in the headers of messages that You get from this list You
>> will see that there is spamassassin running on thebsh.namesys.com, it's
>> just that it is not configured good enough.
>>
>
> You're right. I use spamassassin in standard configuration with a
> threshold of 7 plus razor, pyzor and dcc. It does a very good job.
>
>

I'm using on my server MiailScanner+postfix, mailscanner does a great job of integrating mta, spamassassin, clamav and his own checks, 99% of spam I get is identified and moved to subfolder of my inbox. About a week ago I've installed sqlgrey (greylist daemon for postfix) and I am very very happy with it. My mailgraph shows that number of messages marked as spam goes down to zero and numer of messages rejected by server goes up. No need to filter them with spamassassin, they never hit Your mailbox (or maildir :P ).

£ukasz Mierzwa

Re: why does reiserfs list get so much spam?

[David Masover]

£ukasz Mierzwa wrote:
> Dnia Fri, 16 Sep 2005 14:40:22 +0200, Gregory Maxwell
> <gmaxwell@gmail.com> napisa³:
> 
>> On 9/16/05, Tomasz Chmielewski <mangoo@mch.one.pl> wrote:
>>
>>> yeah, but it is to be used by the end-user.
>>>
>>> the archive will be still filled with spam.
>>>
>>> not everyone who wants to know about reiser subscribes to the list; most
>>> of the people would just use the archives.
>>
>>
>> I actually subbed to the list because I was frustrated by spam in the
>> archive.
>> Whomever runs the archive should at least just pipe it through spam
>> assassin. If some legitimate messages are lost from the web archive it
>> could not be worse than what we have now, an archive made nearly
>> useless from excessive spam.
>>
> 
> If You will look in the headers of messages that You get from this list
> You will see that there is spamassassin running on thebsh.namesys.com,
> it's just that it is not configured good enough.

Can spamassassin be configured "good" enough?

I use dspam:

http://www.nuclearelephant.com/projects/dspam/

There are some articles about why dspam has a fundamentally better
design than spamassassin, and why in general statistical filters beat
manual-rule-based ones like spamassassin.

Re: why does reiserfs list get so much spam?

[Tomasz Chmielewski]

David Masover schrieb:

(...)

>>If You will look in the headers of messages that You get from this list
>>You will see that there is spamassassin running on thebsh.namesys.com,
>>it's just that it is not configured good enough.
> 
> 
> Can spamassassin be configured "good" enough?
> 
> I use dspam:
> 
> http://www.nuclearelephant.com/projects/dspam/
> 
> There are some articles about why dspam has a fundamentally better
> design than spamassassin, and why in general statistical filters beat
> manual-rule-based ones like spamassassin.

Yeah spamassassin can work extremely well.

I guess these articles are based on the quality of spamassassin which 
checks spam from this list? :)

And it's not really true that spamassassin is a manual-rule-based filter 
only.


-- 
Tomek
http://wpkg.org

Re: why does reiserfs list get so much spam?

[David Masover]

Tomasz Chmielewski wrote:
> David Masover schrieb:
> 
> (...)
> 
>>> If You will look in the headers of messages that You get from this list
>>> You will see that there is spamassassin running on thebsh.namesys.com,
>>> it's just that it is not configured good enough.
>>
>>
>>
>> Can spamassassin be configured "good" enough?
>>
>> I use dspam:
>>
>> http://www.nuclearelephant.com/projects/dspam/
>>
>> There are some articles about why dspam has a fundamentally better
>> design than spamassassin, and why in general statistical filters beat
>> manual-rule-based ones like spamassassin.
> 
> 
> Yeah spamassassin can work extremely well.
> 
> I guess these articles are based on the quality of spamassassin which 
> checks spam from this list? :)
> 
> And it's not really true that spamassassin is a manual-rule-based filter 
> only.

Right, but the statistical/learning component of spamassassin is just 
that -- a component, to be combined with razor/pyzor, manual rules, and 
anything else they can think of.  I think dspam does a much better job 
at being a statistical filter, and that's all it does -- and that's all 
it needs to.  Some people have reported 99.997% accuracy from dspam, 
beating humans.

Anyway, the articles are about the principle of the thing.  A 
statistical filter will beat a manual one every time, because it's 
faster and better at coming up with rules, and you don't need to update 
your definitions to start filtering the new spam -- just train on two or 
three mails, and you're done.

Re: why does reiserfs list get so much spam?

[Tomasz Chmielewski]

David Masover schrieb:

(...)

>> And it's not really true that spamassassin is a manual-rule-based 
>> filter only.
> 
> 
> Right, but the statistical/learning component of spamassassin is just 
> that -- a component, to be combined with razor/pyzor, manual rules, and 
> anything else they can think of.  I think dspam does a much better job 
> at being a statistical filter, and that's all it does -- and that's all 
> it needs to.  Some people have reported 99.997% accuracy from dspam, 
> beating humans.
> 
> Anyway, the articles are about the principle of the thing.  A 
> statistical filter will beat a manual one every time, because it's 
> faster and better at coming up with rules, and you don't need to update 
> your definitions to start filtering the new spam -- just train on two or 
> three mails, and you're done.

yeah, but this starts to be a bit off-topic now.

in case of reiser list it would be good if it had *any* filter that had 
even 50% accuracy, for a start :)


-- 
Tomek

Re: why does reiserfs list get so much spam?

[Łukasz Mierzwa]

Dnia Fri, 16 Sep 2005 20:51:39 +0200, David Masover <ninja@slaphack.com> napisa³:

> Tomasz Chmielewski wrote:
>> David Masover schrieb:
>>
>> (...)
>>
>>>> If You will look in the headers of messages that You get from this list
>>>> You will see that there is spamassassin running on thebsh.namesys.com,
>>>> it's just that it is not configured good enough.
>>>
>>>
>>>
>>> Can spamassassin be configured "good" enough?
>>>
>>> I use dspam:
>>>
>>> http://www.nuclearelephant.com/projects/dspam/
>>>
>>> There are some articles about why dspam has a fundamentally better
>>> design than spamassassin, and why in general statistical filters beat
>>> manual-rule-based ones like spamassassin.
>>
>>
>> Yeah spamassassin can work extremely well.
>>
>> I guess these articles are based on the quality of spamassassin which
>> checks spam from this list? :)
>>
>> And it's not really true that spamassassin is a manual-rule-based filter
>> only.
>
> Right, but the statistical/learning component of spamassassin is just
> that -- a component, to be combined with razor/pyzor, manual rules, and
> anything else they can think of.  I think dspam does a much better job
> at being a statistical filter, and that's all it does -- and that's all
> it needs to.  Some people have reported 99.997% accuracy from dspam,
> beating humans.
>
> Anyway, the articles are about the principle of the thing.  A
> statistical filter will beat a manual one every time, because it's
> faster and better at coming up with rules, and you don't need to update
> your definitions to start filtering the new spam -- just train on two or
> three mails, and you're done.
>

SpamAssassin uses bayes so You don't need any definitions for SA to detect new spam, it learns itself if message have enough points.
All You have to do is learn SA with sa-learn after instaling.

Re: why does reiserfs list get so much spam?

[David Masover]

£ukasz Mierzwa wrote:
> Dnia Fri, 16 Sep 2005 20:51:39 +0200, David Masover <ninja@slaphack.com> 
> napisa³:
> 
>> Tomasz Chmielewski wrote:
>>
>>> David Masover schrieb:
>>>
>>> (...)
>>>
>>>>> If You will look in the headers of messages that You get from this 
>>>>> list
>>>>> You will see that there is spamassassin running on thebsh.namesys.com,
>>>>> it's just that it is not configured good enough.
>>>>
>>>>
>>>>
>>>>
>>>> Can spamassassin be configured "good" enough?
>>>>
>>>> I use dspam:
>>>>
>>>> http://www.nuclearelephant.com/projects/dspam/
>>>>
>>>> There are some articles about why dspam has a fundamentally better
>>>> design than spamassassin, and why in general statistical filters beat
>>>> manual-rule-based ones like spamassassin.
>>>
>>>
>>>
>>> Yeah spamassassin can work extremely well.
>>>
>>> I guess these articles are based on the quality of spamassassin which
>>> checks spam from this list? :)
>>>
>>> And it's not really true that spamassassin is a manual-rule-based filter
>>> only.
>>
>>
>> Right, but the statistical/learning component of spamassassin is just
>> that -- a component, to be combined with razor/pyzor, manual rules, and
>> anything else they can think of.  I think dspam does a much better job
>> at being a statistical filter, and that's all it does -- and that's all
>> it needs to.  Some people have reported 99.997% accuracy from dspam,
>> beating humans.
>>
>> Anyway, the articles are about the principle of the thing.  A
>> statistical filter will beat a manual one every time, because it's
>> faster and better at coming up with rules, and you don't need to update
>> your definitions to start filtering the new spam -- just train on two or
>> three mails, and you're done.
>>
> 
> SpamAssassin uses bayes so You don't need any definitions for SA to 
> detect new spam, it learns itself if message have enough points.
> All You have to do is learn SA with sa-learn after instaling.

Getting way offtopic, but I'll bite anyway.

All I have to do is 'dspam --error', and not nearly as often as I had to 
do with Spamassassin.  And it trains itself on new emails, just like 
Spamassassin.  It's just much more accurate.