All of lore.kernel.org
 help / color / mirror / Atom feed
* [LARTC] How to fight with encrypted p2p
@ 2007-11-12  1:51 sAwAr
  2007-11-12  3:55 ` Mohan Sundaram
                   ` (16 more replies)
  0 siblings, 17 replies; 18+ messages in thread
From: sAwAr @ 2007-11-12  1:51 UTC (permalink / raw)
  To: lartc



Hi

I believe that whole question is in topic. 
Is there any way to recognize ( and then shape ) p2p traffic which is encrypted?
Modern p2p clients have this ability moreover some of them have this enabled by default. 
Now I'm using ipp2p for iptables but as I know this doesn't recognize encrypted traffic.

Thanks in advance.

Pozdrawiam
Szymon Turkiewicz

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [LARTC] How to fight with encrypted p2p
  2007-11-12  1:51 [LARTC] How to fight with encrypted p2p sAwAr
@ 2007-11-12  3:55 ` Mohan Sundaram
  2007-11-12  7:02 ` David Bierce
                   ` (15 subsequent siblings)
  16 siblings, 0 replies; 18+ messages in thread
From: Mohan Sundaram @ 2007-11-12  3:55 UTC (permalink / raw)
  To: lartc

sAwAr wrote:
> 
> Hi
> 
> I believe that whole question is in topic. 
> Is there any way to recognize ( and then shape ) p2p traffic which is encrypted?
> Modern p2p clients have this ability moreover some of them have this enabled by default. 
> Now I'm using ipp2p for iptables but as I know this doesn't recognize encrypted traffic.
> 
> Thanks in advance.
> 
> Pozdrawiam
> Szymon Turkiewicz
Have not tried this. An idea. P2P initiations are not encrypted AFAIK. 
Thus connections can be marked and related traffic shaped. If initiation 
is also encrypted, then I think we have a serious problem.

Mohan
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [LARTC] How to fight with encrypted p2p
  2007-11-12  1:51 [LARTC] How to fight with encrypted p2p sAwAr
  2007-11-12  3:55 ` Mohan Sundaram
@ 2007-11-12  7:02 ` David Bierce
  2007-11-12 11:17 ` sawar
                   ` (14 subsequent siblings)
  16 siblings, 0 replies; 18+ messages in thread
From: David Bierce @ 2007-11-12  7:02 UTC (permalink / raw)
  To: lartc

Some clients P2P clients are nice about there encryption and negotiate  
encryption ahead of time using plain communication. I.E. Limewire,  
Azureus.  However, some just start TLS and that is all you can see.

Looking at ipp2ps signatures, I don't see anything that leads me to  
believe they track that kind of info.



David Bierce



On Nov 11, 2007, at 9:48 PM, Mohan Sundaram wrote:

> sAwAr wrote:
>> Hi
>> I believe that whole question is in topic. Is there any way to  
>> recognize ( and then shape ) p2p traffic which is encrypted?
>> Modern p2p clients have this ability moreover some of them have  
>> this enabled by default. Now I'm using ipp2p for iptables but as I  
>> know this doesn't recognize encrypted traffic.
>> Thanks in advance.
>> Pozdrawiam
>> Szymon Turkiewicz
> Have not tried this. An idea. P2P initiations are not encrypted  
> AFAIK. Thus connections can be marked and related traffic shaped. If  
> initiation is also encrypted, then I think we have a serious problem.
>
> Mohan
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [LARTC] How to fight with encrypted p2p
  2007-11-12  1:51 [LARTC] How to fight with encrypted p2p sAwAr
  2007-11-12  3:55 ` Mohan Sundaram
  2007-11-12  7:02 ` David Bierce
@ 2007-11-12 11:17 ` sawar
  2007-11-13 11:58 ` Marcin Stanczyk
                   ` (13 subsequent siblings)
  16 siblings, 0 replies; 18+ messages in thread
From: sawar @ 2007-11-12 11:17 UTC (permalink / raw)
  To: lartc

Rtorrent which I use sometimes have ability to completely disable plain text 
communication :

man rtorrent
              allow_incoming  (allow incoming encrypted connections), 
try_outgoing (use encryption for outgoing connections), require (disable 
unencrypted  handshakes),  require_RC4  (also  disable  plaintext               
transmission  after  the initial encrypted handshake), enable_retry (if the 
initial outgoing connection fails, retry with encryption turned on if it was 
off or off if it was on),  prefer_plain text  (choose  plaintext when peer 
offers a choice between plaintext transmission and RC4 encryption, otherwise 
RC4 will be used).

and many other clients have similar abilities.
I'm afraid that full encrypted and enabled by default communication is only a 
matter of time and we will lose this "fight" very soon.

> Some clients P2P clients are nice about there encryption and negotiate
> encryption ahead of time using plain communication. I.E. Limewire,
> Azureus.  However, some just start TLS and that is all you can see.
>
> Looking at ipp2ps signatures, I don't see anything that leads me to
> believe they track that kind of info.
>
>
>
> David Bierce
>
> On Nov 11, 2007, at 9:48 PM, Mohan Sundaram wrote:
> > sAwAr wrote:
> >> Hi
> >> I believe that whole question is in topic. Is there any way to
> >> recognize ( and then shape ) p2p traffic which is encrypted?
> >> Modern p2p clients have this ability moreover some of them have
> >> this enabled by default. Now I'm using ipp2p for iptables but as I
> >> know this doesn't recognize encrypted traffic.
> >> Thanks in advance.
> >> Pozdrawiam
> >> Szymon Turkiewicz
> >
> > Have not tried this. An idea. P2P initiations are not encrypted
> > AFAIK. Thus connections can be marked and related traffic shaped. If
> > initiation is also encrypted, then I think we have a serious problem.
> >
> > Mohan
> > _______________________________________________
> > LARTC mailing list
> > LARTC@mailman.ds9a.nl
> > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [LARTC] How to fight with encrypted p2p
  2007-11-12  1:51 [LARTC] How to fight with encrypted p2p sAwAr
                   ` (2 preceding siblings ...)
  2007-11-12 11:17 ` sawar
@ 2007-11-13 11:58 ` Marcin Stanczyk
  2007-11-13 15:09 ` Grant Taylor
                   ` (12 subsequent siblings)
  16 siblings, 0 replies; 18+ messages in thread
From: Marcin Stanczyk @ 2007-11-13 11:58 UTC (permalink / raw)
  To: lartc

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


> Hi
> 
> I believe that whole question is in topic. 
> Is there any way to recognize ( and then shape ) p2p traffic which is encrypted?
> Modern p2p clients have this ability moreover some of them have this enabled by default. 
> Now I'm using ipp2p for iptables but as I know this doesn't recognize encrypted traffic.
> 
> Thanks in advance.
> 

As far as I am concerned there is no way to recognize encrypted p2p.
I`ve tested ipp2p, and layer7, and also Mikrotik`s 3.0 p2p mark systems
[ which in fact uses modified layer7 ], and none of them cant recognize
this traffic simply because it is enctypted.

- --

[ ... and the Slackware for All... ]

gpg_key: http://szuwar.mayhem.net.pl/keys/szuwar_mayhem-public.gpg



-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (MingW32)

iD8DBQFHOZF96pLYJHrOtfoRAhxNAKCoOEqujQ8ibpeKKLcy4ODHXQmgAgCgh8Ur
k+pLmHu2JzhZmM1mtxD9pL4=DKs2
-----END PGP SIGNATURE-----
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [LARTC] How to fight with encrypted p2p
  2007-11-12  1:51 [LARTC] How to fight with encrypted p2p sAwAr
                   ` (3 preceding siblings ...)
  2007-11-13 11:58 ` Marcin Stanczyk
@ 2007-11-13 15:09 ` Grant Taylor
  2007-11-13 15:37 ` Carl-Daniel Hailfinger
                   ` (11 subsequent siblings)
  16 siblings, 0 replies; 18+ messages in thread
From: Grant Taylor @ 2007-11-13 15:09 UTC (permalink / raw)
  To: lartc

On 11/11/07 19:51, sAwAr wrote:
> Is there any way to recognize ( and then shape ) p2p traffic which is 
> encrypted?  Modern p2p clients have this ability moreover some of 
> them have this enabled by default.  Now I'm using ipp2p for iptables 
> but as I know this doesn't recognize encrypted traffic.

Does this mean that we are down to handling traffic based on the 
sustained stream(s)?  I.e. how long the streams have been active, how 
many packets per second, how many streams a given end point has, speed 
of traffic, average size of packets?

Encrypted or not, I believe all traffic can be somewhat recognized by 
its usage pattern(s).  However there may be more false positives.  We 
may end up recognizing what we know as good and putting the rest at a 
lower class of service.

Thought's / comments / objections / flame wars?



Grant. . . .
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [LARTC] How to fight with encrypted p2p
  2007-11-12  1:51 [LARTC] How to fight with encrypted p2p sAwAr
                   ` (4 preceding siblings ...)
  2007-11-13 15:09 ` Grant Taylor
@ 2007-11-13 15:37 ` Carl-Daniel Hailfinger
  2007-11-13 15:53 ` Grant Taylor
                   ` (10 subsequent siblings)
  16 siblings, 0 replies; 18+ messages in thread
From: Carl-Daniel Hailfinger @ 2007-11-13 15:37 UTC (permalink / raw)
  To: lartc

On 13.11.2007 16:09, Grant Taylor wrote:
> On 11/11/07 19:51, sAwAr wrote:
>> Is there any way to recognize ( and then shape ) p2p traffic which is
>> encrypted?  Modern p2p clients have this ability moreover some of
>> them have this enabled by default.  Now I'm using ipp2p for iptables
>> but as I know this doesn't recognize encrypted traffic.
>
> Does this mean that we are down to handling traffic based on the
> sustained stream(s)?  I.e. how long the streams have been active, how
> many packets per second, how many streams a given end point has, speed
> of traffic, average size of packets?
>
> Encrypted or not, I believe all traffic can be somewhat recognized by
> its usage pattern(s).  However there may be more false positives.  We
> may end up recognizing what we know as good and putting the rest at a
> lower class of service.

Well, you can surely try. But then again, I have been doing research
(publication pending) in traffic-pattern-based detection of VoIP flows
and peer-to-peer connections. While it usually is easy to find a pattern
matching your particular traffic class very well, part of this research
has been dedicated to automatically circumvent these systems. Why that?
Simple. Application evolve to circumvent detection. If you can simulate
that evolution in the lab, you can find out where your detection
algorithms will fail. Of course, that enumeration of possible failure
modes is non-exhaustive.

Bottom line: This is an arms race. Unless you do lots of research and
testing, detection will always be trying to catch up. If detection
manages to catch up, circumvention will advance, but you may have a
small time window where you can enjoy the "win". However, winning
becomes more and more expensive. Clients can expend considerable amount
of CPU power to avoid detection. You don't have that luxury in filter
systems unless you have one filter per client.


Regards,
Carl-Daniel
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [LARTC] How to fight with encrypted p2p
  2007-11-12  1:51 [LARTC] How to fight with encrypted p2p sAwAr
                   ` (5 preceding siblings ...)
  2007-11-13 15:37 ` Carl-Daniel Hailfinger
@ 2007-11-13 15:53 ` Grant Taylor
  2007-11-13 16:32 ` Marco Aurelio
                   ` (9 subsequent siblings)
  16 siblings, 0 replies; 18+ messages in thread
From: Grant Taylor @ 2007-11-13 15:53 UTC (permalink / raw)
  To: lartc

On 11/13/07 09:37, Carl-Daniel Hailfinger wrote:
> Well, you can surely try. But then again, I have been doing research 
> (publication pending) in traffic-pattern-based detection of VoIP 
> flows and peer-to-peer connections. While it usually is easy to find 
> a pattern matching your particular traffic class very well, part of 
> this research has been dedicated to automatically circumvent these 
> systems. Why that?  Simple. Application evolve to circumvent 
> detection. If you can simulate that evolution in the lab, you can 
> find out where your detection algorithms will fail. Of course, that 
> enumeration of possible failure modes is non-exhaustive.
> 
> Bottom line: This is an arms race. Unless you do lots of research and 
> testing, detection will always be trying to catch up. If detection 
> manages to catch up, circumvention will advance, but you may have a 
> small time window where you can enjoy the "win". However, winning 
> becomes more and more expensive. Clients can expend considerable 
> amount of CPU power to avoid detection. You don't have that luxury in 
> filter systems unless you have one filter per client.

All very good points with regard to pattern based detecting P2P (and the 
likes) traffic.  What do you think about recognizing the traffic you do 
want and treating all else as a second or third class citizen.  Or is 
this just a form of net neutrality?  Or really is this entire discussion 
such.  Further does the net neutrality issue apply to companies (read: 
non ISPs) wanting to filter their own internal traffic.

Additionally as an aside will you please provide more information on 
your pending publication?  I'd likely be curious to read (what ever) 
when ever it is published.  Thanks in advance.



Grant. . . .
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [LARTC] How to fight with encrypted p2p
  2007-11-12  1:51 [LARTC] How to fight with encrypted p2p sAwAr
                   ` (6 preceding siblings ...)
  2007-11-13 15:53 ` Grant Taylor
@ 2007-11-13 16:32 ` Marco Aurelio
  2007-11-14  9:42 ` Klaus
                   ` (8 subsequent siblings)
  16 siblings, 0 replies; 18+ messages in thread
From: Marco Aurelio @ 2007-11-13 16:32 UTC (permalink / raw)
  To: lartc

As you might have seen, these are words from ipp2p author:

"""

I have seen some pieces of code from ipoque which can detect encypted bittorrent
and edonkey traffic. Unforunately, this code will not work with
iptables, because it needs
more information about the flow history and the history of an ip address.

Right now, I do not have the time and the money to develop a filter
like this, but
if you are interested in a developement in this direction, please contact me.

"""

I *think* that we need something like a "bittorrent helper" in the
kernel to keep this extra information about the flow history and then
an iptables plugin to match. What do you think? Maybe we could contact
him to know what kind of information is it?


On Nov 12, 2007 9:17 AM, sawar <sawar@interia.pl> wrote:
> Rtorrent which I use sometimes have ability to completely disable plain text
> communication :
>
> man rtorrent
>               allow_incoming  (allow incoming encrypted connections),
> try_outgoing (use encryption for outgoing connections), require (disable
> unencrypted  handshakes),  require_RC4  (also  disable  plaintext
> transmission  after  the initial encrypted handshake), enable_retry (if the
> initial outgoing connection fails, retry with encryption turned on if it was
> off or off if it was on),  prefer_plain text  (choose  plaintext when peer
> offers a choice between plaintext transmission and RC4 encryption, otherwise
> RC4 will be used).
>
> and many other clients have similar abilities.
> I'm afraid that full encrypted and enabled by default communication is only a
> matter of time and we will lose this "fight" very soon.
>
>
> > Some clients P2P clients are nice about there encryption and negotiate
> > encryption ahead of time using plain communication. I.E. Limewire,
> > Azureus.  However, some just start TLS and that is all you can see.
> >
> > Looking at ipp2ps signatures, I don't see anything that leads me to
> > believe they track that kind of info.
> >
> >
> >
> > David Bierce
> >
> > On Nov 11, 2007, at 9:48 PM, Mohan Sundaram wrote:
> > > sAwAr wrote:
> > >> Hi
> > >> I believe that whole question is in topic. Is there any way to
> > >> recognize ( and then shape ) p2p traffic which is encrypted?
> > >> Modern p2p clients have this ability moreover some of them have
> > >> this enabled by default. Now I'm using ipp2p for iptables but as I
> > >> know this doesn't recognize encrypted traffic.
> > >> Thanks in advance.
> > >> Pozdrawiam
> > >> Szymon Turkiewicz
> > >
> > > Have not tried this. An idea. P2P initiations are not encrypted
> > > AFAIK. Thus connections can be marked and related traffic shaped. If
> > > initiation is also encrypted, then I think we have a serious problem.
> > >
> > > Mohan
> > > _______________________________________________
> > > LARTC mailing list
> > > LARTC@mailman.ds9a.nl
> > > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
> >
> > _______________________________________________
> > LARTC mailing list
> > LARTC@mailman.ds9a.nl
> > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>



-- 
Marco Casaroli
SapucaiNet Telecom
+55 35 34712377 ext. 5
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [LARTC] How to fight with encrypted p2p
  2007-11-12  1:51 [LARTC] How to fight with encrypted p2p sAwAr
                   ` (7 preceding siblings ...)
  2007-11-13 16:32 ` Marco Aurelio
@ 2007-11-14  9:42 ` Klaus
  2007-11-14 14:32 ` Sébastien CRAMATTE
                   ` (7 subsequent siblings)
  16 siblings, 0 replies; 18+ messages in thread
From: Klaus @ 2007-11-14  9:42 UTC (permalink / raw)
  To: lartc

About ipp2p,

Right now, the battle against p2p is lost with l7 detection from ipp2p,
l7 filter and others.

Why ?? It is a known fact that pattern matching does not work with full
encrypted P2P handshakes based on DHT key exchange algorithms with byte
padding. You have absolutely no byte pattern and no fixed packet lengths
in the stream. So something like a flow history will fail or might have
a very high false +ve rate.

The thing is that there are proprietary solutions which can detect fully
encrypted p2p streams based on a heuristic approach. (AFAIK ipoque is
selling a proprietary library for this which is integrated in some
firewall vendors). I have not seen any open source development into this
direction.

Klaus, (former) maintainer of ipp2p


Marco Aurelio wrote:
> As you might have seen, these are words from ipp2p author:
> 
> """
> 
> I have seen some pieces of code from ipoque which can detect encypted bittorrent
> and edonkey traffic. Unforunately, this code will not work with
> iptables, because it needs
> more information about the flow history and the history of an ip address.
> 
> Right now, I do not have the time and the money to develop a filter
> like this, but
> if you are interested in a developement in this direction, please contact me.
> 
> """
> 
> I *think* that we need something like a "bittorrent helper" in the
> kernel to keep this extra information about the flow history and then
> an iptables plugin to match. What do you think? Maybe we could contact
> him to know what kind of information is it?
> 
> 
> On Nov 12, 2007 9:17 AM, sawar <sawar@interia.pl> wrote:
>> Rtorrent which I use sometimes have ability to completely disable plain text
>> communication :
>>
>> man rtorrent
>>               allow_incoming  (allow incoming encrypted connections),
>> try_outgoing (use encryption for outgoing connections), require (disable
>> unencrypted  handshakes),  require_RC4  (also  disable  plaintext
>> transmission  after  the initial encrypted handshake), enable_retry (if the
>> initial outgoing connection fails, retry with encryption turned on if it was
>> off or off if it was on),  prefer_plain text  (choose  plaintext when peer
>> offers a choice between plaintext transmission and RC4 encryption, otherwise
>> RC4 will be used).
>>
>> and many other clients have similar abilities.
>> I'm afraid that full encrypted and enabled by default communication is only a
>> matter of time and we will lose this "fight" very soon.
>>
>>
>>> Some clients P2P clients are nice about there encryption and negotiate
>>> encryption ahead of time using plain communication. I.E. Limewire,
>>> Azureus.  However, some just start TLS and that is all you can see.
>>>
>>> Looking at ipp2ps signatures, I don't see anything that leads me to
>>> believe they track that kind of info.
>>>
>>>
>>>
>>> David Bierce
>>>
>>> On Nov 11, 2007, at 9:48 PM, Mohan Sundaram wrote:
>>>> sAwAr wrote:
>>>>> Hi
>>>>> I believe that whole question is in topic. Is there any way to
>>>>> recognize ( and then shape ) p2p traffic which is encrypted?
>>>>> Modern p2p clients have this ability moreover some of them have
>>>>> this enabled by default. Now I'm using ipp2p for iptables but as I
>>>>> know this doesn't recognize encrypted traffic.
>>>>> Thanks in advance.
>>>>> Pozdrawiam
>>>>> Szymon Turkiewicz
>>>> Have not tried this. An idea. P2P initiations are not encrypted
>>>> AFAIK. Thus connections can be marked and related traffic shaped. If
>>>> initiation is also encrypted, then I think we have a serious problem.
>>>>
>>>> Mohan
>>>> _______________________________________________
>>>> LARTC mailing list
>>>> LARTC@mailman.ds9a.nl
>>>> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>>> _______________________________________________
>>> LARTC mailing list
>>> LARTC@mailman.ds9a.nl
>>> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>> _______________________________________________
>> LARTC mailing list
>> LARTC@mailman.ds9a.nl
>> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>>
> 
> 
> 
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [LARTC] How to fight with encrypted p2p
  2007-11-12  1:51 [LARTC] How to fight with encrypted p2p sAwAr
                   ` (8 preceding siblings ...)
  2007-11-14  9:42 ` Klaus
@ 2007-11-14 14:32 ` Sébastien CRAMATTE
  2007-11-14 14:44 ` Sébastien CRAMATTE
                   ` (6 subsequent siblings)
  16 siblings, 0 replies; 18+ messages in thread
From: Sébastien CRAMATTE @ 2007-11-14 14:32 UTC (permalink / raw)
  To: lartc

Klauss,

Could you
Might be you can sponsor the development ...

Regards

Sébastien


Klaus escribió:
> About ipp2p,
>
> Right now, the battle against p2p is lost with l7 detection from ipp2p,
> l7 filter and others.
>
> Why ?? It is a known fact that pattern matching does not work with full
> encrypted P2P handshakes based on DHT key exchange algorithms with byte
> padding. You have absolutely no byte pattern and no fixed packet lengths
> in the stream. So something like a flow history will fail or might have
> a very high false +ve rate.
>
> The thing is that there are proprietary solutions which can detect fully
> encrypted p2p streams based on a heuristic approach. (AFAIK ipoque is
> selling a proprietary library for this which is integrated in some
> firewall vendors). I have not seen any open source development into this
> direction.
>
> Klaus, (former) maintainer of ipp2p
>
>
> Marco Aurelio wrote:
>   
>> As you might have seen, these are words from ipp2p author:
>>
>> """
>>
>> I have seen some pieces of code from ipoque which can detect encypted bittorrent
>> and edonkey traffic. Unforunately, this code will not work with
>> iptables, because it needs
>> more information about the flow history and the history of an ip address.
>>
>> Right now, I do not have the time and the money to develop a filter
>> like this, but
>> if you are interested in a developement in this direction, please contact me.
>>
>> """
>>
>> I *think* that we need something like a "bittorrent helper" in the
>> kernel to keep this extra information about the flow history and then
>> an iptables plugin to match. What do you think? Maybe we could contact
>> him to know what kind of information is it?
>>
>>
>> On Nov 12, 2007 9:17 AM, sawar <sawar@interia.pl> wrote:
>>     
>>> Rtorrent which I use sometimes have ability to completely disable plain text
>>> communication :
>>>
>>> man rtorrent
>>>               allow_incoming  (allow incoming encrypted connections),
>>> try_outgoing (use encryption for outgoing connections), require (disable
>>> unencrypted  handshakes),  require_RC4  (also  disable  plaintext
>>> transmission  after  the initial encrypted handshake), enable_retry (if the
>>> initial outgoing connection fails, retry with encryption turned on if it was
>>> off or off if it was on),  prefer_plain text  (choose  plaintext when peer
>>> offers a choice between plaintext transmission and RC4 encryption, otherwise
>>> RC4 will be used).
>>>
>>> and many other clients have similar abilities.
>>> I'm afraid that full encrypted and enabled by default communication is only a
>>> matter of time and we will lose this "fight" very soon.
>>>
>>>
>>>       
>>>> Some clients P2P clients are nice about there encryption and negotiate
>>>> encryption ahead of time using plain communication. I.E. Limewire,
>>>> Azureus.  However, some just start TLS and that is all you can see.
>>>>
>>>> Looking at ipp2ps signatures, I don't see anything that leads me to
>>>> believe they track that kind of info.
>>>>
>>>>
>>>>
>>>> David Bierce
>>>>
>>>> On Nov 11, 2007, at 9:48 PM, Mohan Sundaram wrote:
>>>>         
>>>>> sAwAr wrote:
>>>>>           
>>>>>> Hi
>>>>>> I believe that whole question is in topic. Is there any way to
>>>>>> recognize ( and then shape ) p2p traffic which is encrypted?
>>>>>> Modern p2p clients have this ability moreover some of them have
>>>>>> this enabled by default. Now I'm using ipp2p for iptables but as I
>>>>>> know this doesn't recognize encrypted traffic.
>>>>>> Thanks in advance.
>>>>>> Pozdrawiam
>>>>>> Szymon Turkiewicz
>>>>>>             
>>>>> Have not tried this. An idea. P2P initiations are not encrypted
>>>>> AFAIK. Thus connections can be marked and related traffic shaped. If
>>>>> initiation is also encrypted, then I think we have a serious problem.
>>>>>
>>>>> Mohan
>>>>> _______________________________________________
>>>>> LARTC mailing list
>>>>> LARTC@mailman.ds9a.nl
>>>>> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>>>>>           
>>>> _______________________________________________
>>>> LARTC mailing list
>>>> LARTC@mailman.ds9a.nl
>>>> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>>>>         
>>> _______________________________________________
>>> LARTC mailing list
>>> LARTC@mailman.ds9a.nl
>>> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>>>
>>>       
>>
>>     
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>
>
>   

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [LARTC] How to fight with encrypted p2p
  2007-11-12  1:51 [LARTC] How to fight with encrypted p2p sAwAr
                   ` (9 preceding siblings ...)
  2007-11-14 14:32 ` Sébastien CRAMATTE
@ 2007-11-14 14:44 ` Sébastien CRAMATTE
  2007-12-02 11:42 ` Andrew Beverley
                   ` (5 subsequent siblings)
  16 siblings, 0 replies; 18+ messages in thread
From: Sébastien CRAMATTE @ 2007-11-14 14:44 UTC (permalink / raw)
  To: lartc

Sorry ... I'm little bite tired ...
I mean  that we might sponsor  Klauss and L7 team to develop this ...

Regards

Sébastien CRAMATTE escribió:
> Klauss,
>
> Could you
> Might be you can sponsor the development ...
>
> Regards
>
> Sébastien
>
>
> Klaus escribió:
>   
>> About ipp2p,
>>
>> Right now, the battle against p2p is lost with l7 detection from ipp2p,
>> l7 filter and others.
>>
>> Why ?? It is a known fact that pattern matching does not work with full
>> encrypted P2P handshakes based on DHT key exchange algorithms with byte
>> padding. You have absolutely no byte pattern and no fixed packet lengths
>> in the stream. So something like a flow history will fail or might have
>> a very high false +ve rate.
>>
>> The thing is that there are proprietary solutions which can detect fully
>> encrypted p2p streams based on a heuristic approach. (AFAIK ipoque is
>> selling a proprietary library for this which is integrated in some
>> firewall vendors). I have not seen any open source development into this
>> direction.
>>
>> Klaus, (former) maintainer of ipp2p
>>
>>
>> Marco Aurelio wrote:
>>   
>>     
>>> As you might have seen, these are words from ipp2p author:
>>>
>>> """
>>>
>>> I have seen some pieces of code from ipoque which can detect encypted bittorrent
>>> and edonkey traffic. Unforunately, this code will not work with
>>> iptables, because it needs
>>> more information about the flow history and the history of an ip address.
>>>
>>> Right now, I do not have the time and the money to develop a filter
>>> like this, but
>>> if you are interested in a developement in this direction, please contact me.
>>>
>>> """
>>>
>>> I *think* that we need something like a "bittorrent helper" in the
>>> kernel to keep this extra information about the flow history and then
>>> an iptables plugin to match. What do you think? Maybe we could contact
>>> him to know what kind of information is it?
>>>
>>>
>>> On Nov 12, 2007 9:17 AM, sawar <sawar@interia.pl> wrote:
>>>     
>>>       
>>>> Rtorrent which I use sometimes have ability to completely disable plain text
>>>> communication :
>>>>
>>>> man rtorrent
>>>>               allow_incoming  (allow incoming encrypted connections),
>>>> try_outgoing (use encryption for outgoing connections), require (disable
>>>> unencrypted  handshakes),  require_RC4  (also  disable  plaintext
>>>> transmission  after  the initial encrypted handshake), enable_retry (if the
>>>> initial outgoing connection fails, retry with encryption turned on if it was
>>>> off or off if it was on),  prefer_plain text  (choose  plaintext when peer
>>>> offers a choice between plaintext transmission and RC4 encryption, otherwise
>>>> RC4 will be used).
>>>>
>>>> and many other clients have similar abilities.
>>>> I'm afraid that full encrypted and enabled by default communication is only a
>>>> matter of time and we will lose this "fight" very soon.
>>>>
>>>>
>>>>       
>>>>         
>>>>> Some clients P2P clients are nice about there encryption and negotiate
>>>>> encryption ahead of time using plain communication. I.E. Limewire,
>>>>> Azureus.  However, some just start TLS and that is all you can see.
>>>>>
>>>>> Looking at ipp2ps signatures, I don't see anything that leads me to
>>>>> believe they track that kind of info.
>>>>>
>>>>>
>>>>>
>>>>> David Bierce
>>>>>
>>>>> On Nov 11, 2007, at 9:48 PM, Mohan Sundaram wrote:
>>>>>         
>>>>>           
>>>>>> sAwAr wrote:
>>>>>>           
>>>>>>             
>>>>>>> Hi
>>>>>>> I believe that whole question is in topic. Is there any way to
>>>>>>> recognize ( and then shape ) p2p traffic which is encrypted?
>>>>>>> Modern p2p clients have this ability moreover some of them have
>>>>>>> this enabled by default. Now I'm using ipp2p for iptables but as I
>>>>>>> know this doesn't recognize encrypted traffic.
>>>>>>> Thanks in advance.
>>>>>>> Pozdrawiam
>>>>>>> Szymon Turkiewicz
>>>>>>>             
>>>>>>>               
>>>>>> Have not tried this. An idea. P2P initiations are not encrypted
>>>>>> AFAIK. Thus connections can be marked and related traffic shaped. If
>>>>>> initiation is also encrypted, then I think we have a serious problem.
>>>>>>
>>>>>> Mohan
>>>>>> _______________________________________________
>>>>>> LARTC mailing list
>>>>>> LARTC@mailman.ds9a.nl
>>>>>> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>>>>>>           
>>>>>>             
>>>>> _______________________________________________
>>>>> LARTC mailing list
>>>>> LARTC@mailman.ds9a.nl
>>>>> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>>>>>         
>>>>>           
>>>> _______________________________________________
>>>> LARTC mailing list
>>>> LARTC@mailman.ds9a.nl
>>>> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>>>>
>>>>       
>>>>         
>>>     
>>>       
>> _______________________________________________
>> LARTC mailing list
>> LARTC@mailman.ds9a.nl
>> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>>
>>
>>   
>>     
>
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>
>
>   

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [LARTC] How to fight with encrypted p2p
  2007-11-12  1:51 [LARTC] How to fight with encrypted p2p sAwAr
                   ` (10 preceding siblings ...)
  2007-11-14 14:44 ` Sébastien CRAMATTE
@ 2007-12-02 11:42 ` Andrew Beverley
  2007-12-03 10:49 ` Gustin Johnson
                   ` (4 subsequent siblings)
  16 siblings, 0 replies; 18+ messages in thread
From: Andrew Beverley @ 2007-12-02 11:42 UTC (permalink / raw)
  To: lartc

> I believe that whole question is in topic. 
> Is there any way to recognize ( and then shape ) p2p traffic which is encrypted?
> Modern p2p clients have this ability moreover some of them have this enabled by default. 
> Now I'm using ipp2p for iptables but as I know this doesn't recognize encrypted traffic.

One way to do this is to look for the style of traffic. For example, I
look for lots of connections from one PC to port numbers above 1024.
This will also incorrectly recognise some other traffic, but on the
whole it works well for me.

The following are some examples using connlimit (now included in vanilla
kernel) and ipset (see http://ipset.netfilter.org/)

# first look for style of traffic and log that client to an ipset
iptables -t mangle -A FORWARD -o ppp0 -p tcp --dport 1024: \
	-m connlimit --connlimit-above 10 -j SET --add-set p2p src
iptables -t mangle -A FORWARD -o ppp0 -p udp --dport 1024: \
	-m connlimit --connlimit-above 10 -j SET --add-set p2p src
iptables -t mangle -A FORWARD -i ppp0 -p tcp --sport 1024: \
	-m connlimit --connlimit-above 10 -j SET --add-set p2p dst
iptables -t mangle -A FORWARD -i ppp0 -p udp --sport 1024: \
	-m connlimit --connlimit-above 10 -j SET --add-set p2p dst

# then shape traffic above port 1024 for those detected clients
iptables -t mangle -A FORWARD -o ppp0 -p tcp --dport 1024: \
	-m set --set p2p dst -j MARK --set-mark 60
iptables -t mangle -A FORWARD -i ppp0 -p tcp --sport 1024: \
	-m set --set p2p dst -j MARK --set-mark 60
iptables -t mangle -A FORWARD -o ppp0 -p udp --dport 1024: \
	-m set --set p2p dst -j MARK --set-mark 60
iptables -t mangle -A FORWARD -i ppp0 -p udp --sport 1024: \
	-m set --set p2p dst -j MARK --set-mark 60


Regards,

Andy Beverley


_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [LARTC] How to fight with encrypted p2p
  2007-11-12  1:51 [LARTC] How to fight with encrypted p2p sAwAr
                   ` (11 preceding siblings ...)
  2007-12-02 11:42 ` Andrew Beverley
@ 2007-12-03 10:49 ` Gustin Johnson
  2007-12-03 19:33 ` Andrew Beverley
                   ` (3 subsequent siblings)
  16 siblings, 0 replies; 18+ messages in thread
From: Gustin Johnson @ 2007-12-03 10:49 UTC (permalink / raw)
  To: lartc

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I believe "fighting" is the wrong approach.  Badly shaping the wrong
traffic is just as bad, if not worse IMO.  An ISP in my neck of the
woods plays havoc with encrypted mail (SMTP + TLS as well as IMAPS) as a
result of their P2P fight.  Needless to say we no longer use them, and
we encourage clients, friends, and colleagues not to as well.  I don't
use P2P but I do use ssh, imaps, sftp, and https daily.  Screwing with
these services is not useful.  Limiting your rules to specific ports is
pretty useless.  This has been done before, and it failed miserably.

For me, if P2P does not belong at all, for instance on a corporate
network, then a default deny on the outbound works much better.  We then
 only allow specific connections on a case by case basis.  For instances
where I am not able to block p2p, I define specific rules for high and
low priority, and leave everything else in the default.  If the end user
wants to use the bulk of his or her bandwidth for P2P, so be it.  Of
course in this case bandwidth accounting is far more useful.

I would also encourage your users to use software that is or can be well
behaved.  Software that allows you set a proper TOS for instance.  If
possible work with the end users.

I have personally found that the best solutions are not tech solutions.
 Having a well defined Acceptable Use Policy, plus a constructive
dialogue with my users has been far more effective than any shaping
routine I/we could come up with.

Ask yourself, what is the problem you are really trying to solve.

Andrew Beverley wrote:
>> I believe that whole question is in topic. 
>> Is there any way to recognize ( and then shape ) p2p traffic which is encrypted?
>> Modern p2p clients have this ability moreover some of them have this enabled by default. 
>> Now I'm using ipp2p for iptables but as I know this doesn't recognize encrypted traffic.
> 
> One way to do this is to look for the style of traffic. For example, I
> look for lots of connections from one PC to port numbers above 1024.
> This will also incorrectly recognise some other traffic, but on the
> whole it works well for me.
> 
> The following are some examples using connlimit (now included in vanilla
> kernel) and ipset (see http://ipset.netfilter.org/)
> 
> # first look for style of traffic and log that client to an ipset
> iptables -t mangle -A FORWARD -o ppp0 -p tcp --dport 1024: \
> 	-m connlimit --connlimit-above 10 -j SET --add-set p2p src
> iptables -t mangle -A FORWARD -o ppp0 -p udp --dport 1024: \
> 	-m connlimit --connlimit-above 10 -j SET --add-set p2p src
> iptables -t mangle -A FORWARD -i ppp0 -p tcp --sport 1024: \
> 	-m connlimit --connlimit-above 10 -j SET --add-set p2p dst
> iptables -t mangle -A FORWARD -i ppp0 -p udp --sport 1024: \
> 	-m connlimit --connlimit-above 10 -j SET --add-set p2p dst
> 
> # then shape traffic above port 1024 for those detected clients
> iptables -t mangle -A FORWARD -o ppp0 -p tcp --dport 1024: \
> 	-m set --set p2p dst -j MARK --set-mark 60
> iptables -t mangle -A FORWARD -i ppp0 -p tcp --sport 1024: \
> 	-m set --set p2p dst -j MARK --set-mark 60
> iptables -t mangle -A FORWARD -o ppp0 -p udp --dport 1024: \
> 	-m set --set p2p dst -j MARK --set-mark 60
> iptables -t mangle -A FORWARD -i ppp0 -p udp --sport 1024: \
> 	-m set --set p2p dst -j MARK --set-mark 60
> 
> 
> Regards,
> 
> Andy Beverley
> 
> 
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHU98owRXgH3rKGfMRAmszAKCIhVoUnfuWDIaWQqwE1WfuSz9sNwCgipFZ
wqrptNaNg3HMFE79AvbQ+fI=gb3i
-----END PGP SIGNATURE-----
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [LARTC] How to fight with encrypted p2p
  2007-11-12  1:51 [LARTC] How to fight with encrypted p2p sAwAr
                   ` (12 preceding siblings ...)
  2007-12-03 10:49 ` Gustin Johnson
@ 2007-12-03 19:33 ` Andrew Beverley
  2007-12-10 13:37 ` the sew
                   ` (2 subsequent siblings)
  16 siblings, 0 replies; 18+ messages in thread
From: Andrew Beverley @ 2007-12-03 19:33 UTC (permalink / raw)
  To: lartc

> I believe "fighting" is the wrong approach.  Badly shaping the wrong
> traffic is just as bad, if not worse IMO.  An ISP in my neck of the
> woods plays havoc with encrypted mail (SMTP + TLS as well as IMAPS) as a
> result of their P2P fight.  Needless to say we no longer use them, and
> we encourage clients, friends, and colleagues not to as well.  I don't
> use P2P but I do use ssh, imaps, sftp, and https daily.  Screwing with
> these services is not useful.

Using the rules in the example previously given specifically steers well clear
of these services.

> Limiting your rules to specific ports is
> pretty useless.  This has been done before, and it failed miserably.

Agreed.

> For me, if P2P does not belong at all, for instance on a corporate
> network, then a default deny on the outbound works much better.  We then
> only allow specific connections on a case by case basis.

I have seen this work very well on corporate networks, and would 
recommend this
approach where possible. Unfortunately though, on a normal home user network,
there are so many different possibilities that this isn't very practical.

> For instances
> where I am not able to block p2p, I define specific rules for high and
> low priority, and leave everything else in the default.  If the end user
> wants to use the bulk of his or her bandwidth for P2P, so be it.  Of
> course in this case bandwidth accounting is far more useful.

Again, this depends on the circumstances. If you only have 2Mbit/s to share
between 100 users then each user cannot have their own 'share' of the
connection. Equally, people downloading in a responsible way are lumped 
into the
same category as p2p users, which is not fair. Bandwidth accounting is a
possibility, and something I haven't investigated.

For those who want to fairly share bandwidth beween users, I would 
recommend the
ESFQ patches. These allow bandwidth sharing to be done on an IP address basis,
rather than per connection. This prevents the hundreds of p2p connections from
drowning out single downloads.

> I would also encourage your users to use software that is or can be well
> behaved.  Software that allows you set a proper TOS for instance.  If
> possible work with the end users.
> I have personally found that the best solutions are not tech solutions.
> Having a well defined Acceptable Use Policy, plus a constructive
> dialogue with my users has been far more effective than any shaping
> routine I/we could come up with.

Agreed. However, in a situation where you have a lot of users coming 
and going,
it is not easy to educate the many hundreds of users.

I guess it all boils down to your own situation. Traffic shaping on a 
corporate
network or on a network where your users are static can be done using 
the above
techniques. However, sharing a small connection between hundreds of regularly
changing users is difficult, and I have found the 'blunt' rules previously
described to work very well with no complaints.

Regards,

Andy Beverley


_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [LARTC] How to fight with encrypted p2p
  2007-11-12  1:51 [LARTC] How to fight with encrypted p2p sAwAr
                   ` (13 preceding siblings ...)
  2007-12-03 19:33 ` Andrew Beverley
@ 2007-12-10 13:37 ` the sew
  2007-12-10 14:09 ` Mario Antonio Garcia
  2007-12-10 14:28 ` the sew
  16 siblings, 0 replies; 18+ messages in thread
From: the sew @ 2007-12-10 13:37 UTC (permalink / raw)
  To: lartc

Hi,

We had similiar problem with p2p, used ipp2p and L7filter together
before and worked well until clients( mostly clever ones) started
getting around it with encryption. We have about 700 wireless clients
hitting our network and our network was taking big knocks with guys
using couple of gigs day on entry level packages.

Was going to use Ipoque, but was quite pricy for us, Only solutions
for us to use a daily limit of eg 500MB, then they get slowed down to
slower speeds, This worked like a charm

Out of interest we used freeradius / pptpd|pppd  with some custom perl
scripts and tc rules

Sew

On Dec 3, 2007 9:33 PM, Andrew Beverley <andy@andybev.com> wrote:
> > I believe "fighting" is the wrong approach.  Badly shaping the wrong
> > traffic is just as bad, if not worse IMO.  An ISP in my neck of the
> > woods plays havoc with encrypted mail (SMTP + TLS as well as IMAPS) as a
> > result of their P2P fight.  Needless to say we no longer use them, and
> > we encourage clients, friends, and colleagues not to as well.  I don't
> > use P2P but I do use ssh, imaps, sftp, and https daily.  Screwing with
> > these services is not useful.
>
> Using the rules in the example previously given specifically steers well clear
> of these services.
>
> > Limiting your rules to specific ports is
> > pretty useless.  This has been done before, and it failed miserably.
>
> Agreed.
>
> > For me, if P2P does not belong at all, for instance on a corporate
> > network, then a default deny on the outbound works much better.  We then
> > only allow specific connections on a case by case basis.
>
> I have seen this work very well on corporate networks, and would
> recommend this
> approach where possible. Unfortunately though, on a normal home user network,
> there are so many different possibilities that this isn't very practical.
>
> > For instances
> > where I am not able to block p2p, I define specific rules for high and
> > low priority, and leave everything else in the default.  If the end user
> > wants to use the bulk of his or her bandwidth for P2P, so be it.  Of
> > course in this case bandwidth accounting is far more useful.
>
> Again, this depends on the circumstances. If you only have 2Mbit/s to share
> between 100 users then each user cannot have their own 'share' of the
> connection. Equally, people downloading in a responsible way are lumped
> into the
> same category as p2p users, which is not fair. Bandwidth accounting is a
> possibility, and something I haven't investigated.
>
> For those who want to fairly share bandwidth beween users, I would
> recommend the
> ESFQ patches. These allow bandwidth sharing to be done on an IP address basis,
> rather than per connection. This prevents the hundreds of p2p connections from
> drowning out single downloads.
>
> > I would also encourage your users to use software that is or can be well
> > behaved.  Software that allows you set a proper TOS for instance.  If
> > possible work with the end users.
> > I have personally found that the best solutions are not tech solutions.
> > Having a well defined Acceptable Use Policy, plus a constructive
> > dialogue with my users has been far more effective than any shaping
> > routine I/we could come up with.
>
> Agreed. However, in a situation where you have a lot of users coming
> and going,
> it is not easy to educate the many hundreds of users.
>
> I guess it all boils down to your own situation. Traffic shaping on a
> corporate
> network or on a network where your users are static can be done using
> the above
> techniques. However, sharing a small connection between hundreds of regularly
> changing users is difficult, and I have found the 'blunt' rules previously
> described to work very well with no complaints.
>
>
> Regards,
>
> Andy Beverley
>
>
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [LARTC] How to fight with encrypted p2p
  2007-11-12  1:51 [LARTC] How to fight with encrypted p2p sAwAr
                   ` (14 preceding siblings ...)
  2007-12-10 13:37 ` the sew
@ 2007-12-10 14:09 ` Mario Antonio Garcia
  2007-12-10 14:28 ` the sew
  16 siblings, 0 replies; 18+ messages in thread
From: Mario Antonio Garcia @ 2007-12-10 14:09 UTC (permalink / raw)
  To: lartc

Thanks for sharing.

Just one question, how are you implementing the daily limit?

Regards,

Mario Antonio

----- Original Message -----
From: "the sew" <sewlist@gmail.com>
To: "Andrew Beverley" <andy@andybev.com>
Cc: lartc@mailman.ds9a.nl
Sent: Monday, December 10, 2007 8:37:07 AM (GMT-0500) America/New_York
Subject: Re: [LARTC] How to fight with encrypted p2p

Hi,

We had similiar problem with p2p, used ipp2p and L7filter together
before and worked well until clients( mostly clever ones) started
getting around it with encryption. We have about 700 wireless clients
hitting our network and our network was taking big knocks with guys
using couple of gigs day on entry level packages.

Was going to use Ipoque, but was quite pricy for us, Only solutions
for us to use a daily limit of eg 500MB, then they get slowed down to
slower speeds, This worked like a charm

Out of interest we used freeradius / pptpd|pppd  with some custom perl
scripts and tc rules

Sew

On Dec 3, 2007 9:33 PM, Andrew Beverley <andy@andybev.com> wrote:
> > I believe "fighting" is the wrong approach.  Badly shaping the wrong
> > traffic is just as bad, if not worse IMO.  An ISP in my neck of the
> > woods plays havoc with encrypted mail (SMTP + TLS as well as IMAPS) as a
> > result of their P2P fight.  Needless to say we no longer use them, and
> > we encourage clients, friends, and colleagues not to as well.  I don't
> > use P2P but I do use ssh, imaps, sftp, and https daily.  Screwing with
> > these services is not useful.
>
> Using the rules in the example previously given specifically steers well clear
> of these services.
>
> > Limiting your rules to specific ports is
> > pretty useless.  This has been done before, and it failed miserably.
>
> Agreed.
>
> > For me, if P2P does not belong at all, for instance on a corporate
> > network, then a default deny on the outbound works much better.  We then
> > only allow specific connections on a case by case basis.
>
> I have seen this work very well on corporate networks, and would
> recommend this
> approach where possible. Unfortunately though, on a normal home user network,
> there are so many different possibilities that this isn't very practical.
>
> > For instances
> > where I am not able to block p2p, I define specific rules for high and
> > low priority, and leave everything else in the default.  If the end user
> > wants to use the bulk of his or her bandwidth for P2P, so be it.  Of
> > course in this case bandwidth accounting is far more useful.
>
> Again, this depends on the circumstances. If you only have 2Mbit/s to share
> between 100 users then each user cannot have their own 'share' of the
> connection. Equally, people downloading in a responsible way are lumped
> into the
> same category as p2p users, which is not fair. Bandwidth accounting is a
> possibility, and something I haven't investigated.
>
> For those who want to fairly share bandwidth beween users, I would
> recommend the
> ESFQ patches. These allow bandwidth sharing to be done on an IP address basis,
> rather than per connection. This prevents the hundreds of p2p connections from
> drowning out single downloads.
>
> > I would also encourage your users to use software that is or can be well
> > behaved.  Software that allows you set a proper TOS for instance.  If
> > possible work with the end users.
> > I have personally found that the best solutions are not tech solutions.
> > Having a well defined Acceptable Use Policy, plus a constructive
> > dialogue with my users has been far more effective than any shaping
> > routine I/we could come up with.
>
> Agreed. However, in a situation where you have a lot of users coming
> and going,
> it is not easy to educate the many hundreds of users.
>
> I guess it all boils down to your own situation. Traffic shaping on a
> corporate
> network or on a network where your users are static can be done using
> the above
> techniques. However, sharing a small connection between hundreds of regularly
> changing users is difficult, and I have found the 'blunt' rules previously
> described to work very well with no complaints.
>
>
> Regards,
>
> Andy Beverley
>
>
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc




_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [LARTC] How to fight with encrypted p2p
  2007-11-12  1:51 [LARTC] How to fight with encrypted p2p sAwAr
                   ` (15 preceding siblings ...)
  2007-12-10 14:09 ` Mario Antonio Garcia
@ 2007-12-10 14:28 ` the sew
  16 siblings, 0 replies; 18+ messages in thread
From: the sew @ 2007-12-10 14:28 UTC (permalink / raw)
  To: lartc

On Dec 10, 2007 4:09 PM, Mario Antonio Garcia <dino@webjogger.net> wrote:
> Thanks for sharing.

No problem


>
> Just one question, how are you implementing the daily limit?

two ways of detecting them:

1st is the /etc/ppp/ip-up.local which executes the script to check
usage against the radius db and shape them on authentication,

Once they logged in , I dont want to kick everone off every few hours
to check usage, I have a "nice" cron job running every 3 hours, to
check every single user against the db and if they reached their
quotas ,they get shaped while being online., Radius stores all info
about the nas in the db, so makes it quite strait forward.

Also nicely added is our reselling guys who maintain the clients get a
report everytime of users who exceeded the limit and can be aware of
which clients is the problem if they phone to complain, Also nice is
that usually the infected pc's gets knocked off first to save alot of
bandwidth

Sew





>
> Regards,
>
> Mario Antonio
>
>
> ----- Original Message -----
> From: "the sew" <sewlist@gmail.com>
> To: "Andrew Beverley" <andy@andybev.com>
> Cc: lartc@mailman.ds9a.nl
> Sent: Monday, December 10, 2007 8:37:07 AM (GMT-0500) America/New_York
> Subject: Re: [LARTC] How to fight with encrypted p2p
>
> Hi,
>
> We had similiar problem with p2p, used ipp2p and L7filter together
> before and worked well until clients( mostly clever ones) started
> getting around it with encryption. We have about 700 wireless clients
> hitting our network and our network was taking big knocks with guys
> using couple of gigs day on entry level packages.
>
> Was going to use Ipoque, but was quite pricy for us, Only solutions
> for us to use a daily limit of eg 500MB, then they get slowed down to
> slower speeds, This worked like a charm
>
> Out of interest we used freeradius / pptpd|pppd  with some custom perl
> scripts and tc rules
>
> Sew
>
> On Dec 3, 2007 9:33 PM, Andrew Beverley <andy@andybev.com> wrote:
> > > I believe "fighting" is the wrong approach.  Badly shaping the wrong
> > > traffic is just as bad, if not worse IMO.  An ISP in my neck of the
> > > woods plays havoc with encrypted mail (SMTP + TLS as well as IMAPS) as a
> > > result of their P2P fight.  Needless to say we no longer use them, and
> > > we encourage clients, friends, and colleagues not to as well.  I don't
> > > use P2P but I do use ssh, imaps, sftp, and https daily.  Screwing with
> > > these services is not useful.
> >
> > Using the rules in the example previously given specifically steers well clear
> > of these services.
> >
> > > Limiting your rules to specific ports is
> > > pretty useless.  This has been done before, and it failed miserably.
> >
> > Agreed.
> >
> > > For me, if P2P does not belong at all, for instance on a corporate
> > > network, then a default deny on the outbound works much better.  We then
> > > only allow specific connections on a case by case basis.
> >
> > I have seen this work very well on corporate networks, and would
> > recommend this
> > approach where possible. Unfortunately though, on a normal home user network,
> > there are so many different possibilities that this isn't very practical.
> >
> > > For instances
> > > where I am not able to block p2p, I define specific rules for high and
> > > low priority, and leave everything else in the default.  If the end user
> > > wants to use the bulk of his or her bandwidth for P2P, so be it.  Of
> > > course in this case bandwidth accounting is far more useful.
> >
> > Again, this depends on the circumstances. If you only have 2Mbit/s to share
> > between 100 users then each user cannot have their own 'share' of the
> > connection. Equally, people downloading in a responsible way are lumped
> > into the
> > same category as p2p users, which is not fair. Bandwidth accounting is a
> > possibility, and something I haven't investigated.
> >
> > For those who want to fairly share bandwidth beween users, I would
> > recommend the
> > ESFQ patches. These allow bandwidth sharing to be done on an IP address basis,
> > rather than per connection. This prevents the hundreds of p2p connections from
> > drowning out single downloads.
> >
> > > I would also encourage your users to use software that is or can be well
> > > behaved.  Software that allows you set a proper TOS for instance.  If
> > > possible work with the end users.
> > > I have personally found that the best solutions are not tech solutions.
> > > Having a well defined Acceptable Use Policy, plus a constructive
> > > dialogue with my users has been far more effective than any shaping
> > > routine I/we could come up with.
> >
> > Agreed. However, in a situation where you have a lot of users coming
> > and going,
> > it is not easy to educate the many hundreds of users.
> >
> > I guess it all boils down to your own situation. Traffic shaping on a
> > corporate
> > network or on a network where your users are static can be done using
> > the above
> > techniques. However, sharing a small connection between hundreds of regularly
> > changing users is difficult, and I have found the 'blunt' rules previously
> > described to work very well with no complaints.
> >
> >
> > Regards,
> >
> > Andy Beverley
> >
> >
> > _______________________________________________
> > LARTC mailing list
> > LARTC@mailman.ds9a.nl
> > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
> >
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>
>
>
>
>
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

^ permalink raw reply	[flat|nested] 18+ messages in thread

end of thread, other threads:[~2007-12-10 14:28 UTC | newest]

Thread overview: 18+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2007-11-12  1:51 [LARTC] How to fight with encrypted p2p sAwAr
2007-11-12  3:55 ` Mohan Sundaram
2007-11-12  7:02 ` David Bierce
2007-11-12 11:17 ` sawar
2007-11-13 11:58 ` Marcin Stanczyk
2007-11-13 15:09 ` Grant Taylor
2007-11-13 15:37 ` Carl-Daniel Hailfinger
2007-11-13 15:53 ` Grant Taylor
2007-11-13 16:32 ` Marco Aurelio
2007-11-14  9:42 ` Klaus
2007-11-14 14:32 ` Sébastien CRAMATTE
2007-11-14 14:44 ` Sébastien CRAMATTE
2007-12-02 11:42 ` Andrew Beverley
2007-12-03 10:49 ` Gustin Johnson
2007-12-03 19:33 ` Andrew Beverley
2007-12-10 13:37 ` the sew
2007-12-10 14:09 ` Mario Antonio Garcia
2007-12-10 14:28 ` the sew

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.