All of lore.kernel.org
 help / color / mirror / Atom feed
* syn DDoS attack solution
@ 2007-05-31 16:19 Bgs
  2007-05-31 19:57 ` R. DuFresne
  2007-05-31 20:08 ` Ric Messier
  0 siblings, 2 replies; 34+ messages in thread
From: Bgs @ 2007-05-31 16:19 UTC (permalink / raw)
  To: netfilter


  Hi all,


  We recently got under a low traffic botnet DDoS attack. All attacker 
nodes opened a single tcp session (just SYN part) and then did nothing. 
This rules out rate limiting solutions and syncookie doesn't help 
either. (Thousands of attacking nodes).


  I'd like to know you thoughts about two possible approaches in solving 
this:

- syn proxy: already used for example by Cisco. The router handles the 
first part of the connection and only routes packets to the client if 
the connection seems good. (Good against single/spoofed incoming SYNs 
but may be used to wait for the first packet with actual data as well).
- Implement a conntrack solution that gives a timestamp to the 
connection. Then this timestamp could be used to drop the connection if 
no data arrives within a configured time limit (good for open inactive 
connections only). Of course appropriate close toward the local client 
has to be done as well.

What's your opinion?

Bye
Bgs


^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: syn DDoS attack solution
  2007-05-31 16:19 syn DDoS attack solution Bgs
@ 2007-05-31 19:57 ` R. DuFresne
  2007-06-01  9:45   ` Bgs
  2007-05-31 20:08 ` Ric Messier
  1 sibling, 1 reply; 34+ messages in thread
From: R. DuFresne @ 2007-05-31 19:57 UTC (permalink / raw)
  To: Bgs; +Cc: netfilter

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thu, 31 May 2007, Bgs wrote:

>
> Hi all,
>
>
> We recently got under a low traffic botnet DDoS attack. All attacker nodes 
> opened a single tcp session (just SYN part) and then did nothing. This rules 
> out rate limiting solutions and syncookie doesn't help either. (Thousands of 
> attacking nodes).
>
>
> I'd like to know you thoughts about two possible approaches in solving this:
>
> - syn proxy: already used for example by Cisco. The router handles the first 
> part of the connection and only routes packets to the client if the 
> connection seems good. (Good against single/spoofed incoming SYNs but may be 
> used to wait for the first packet with actual data as well).
> - Implement a conntrack solution that gives a timestamp to the connection. 
> Then this timestamp could be used to drop the connection if no data arrives 
> within a configured time limit (good for open inactive connections only). Of 
> course appropriate close toward the local client has to be done as well.
>
> What's your opinion?
>


That you knowledge ot TCP/IP is poor.



Thanks,

Ron DuFresne
- -- 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
         admin & senior security consultant:  sysinfo.com
                         http://sysinfo.com
Key fingerprint = 9401 4B13 B918 164C 647A  E838 B2DF AFCC 94B0 6629

...We waste time looking for the perfect lover
instead of creating the perfect love.

                 -Tom Robbins <Still Life With Woodpecker>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)

iD8DBQFGXyiqst+vzJSwZikRAih2AJ0SY0npal/EqdwWK+/4SYOGaXY/uACggRtl
uhPlH5F7JzCJGHf2RgryJT4=
=Y3xX
-----END PGP SIGNATURE-----


^ permalink raw reply	[flat|nested] 34+ messages in thread

* RE: syn DDoS attack solution
  2007-05-31 16:19 syn DDoS attack solution Bgs
  2007-05-31 19:57 ` R. DuFresne
@ 2007-05-31 20:08 ` Ric Messier
  2007-06-01  1:20   ` Tony.Ho
  2007-06-01  9:44   ` Bgs
  1 sibling, 2 replies; 34+ messages in thread
From: Ric Messier @ 2007-05-31 20:08 UTC (permalink / raw)
  To: 'Bgs', netfilter


Bgs writes:
> 
>   We recently got under a low traffic botnet DDoS attack. All attacker
> nodes opened a single tcp session (just SYN part) and then did nothing.
> This rules out rate limiting solutions and syncookie doesn't help
> either. (Thousands of attacking nodes).
> 

This is simply a SYN flood attack. It may or may not be a botnet (though
saying botnet makes it sound sexier :-) ). A decent SYN flood attack tool
would randomize the source address anyway. 

You should try reading the following as a starting point:

http://www.securityfocus.com/infocus/1729

Your second suggestion has been implemented in the TCP/IP stack forever. The
article above gives guidance on how to tune it in a Linux implementation.

Ric



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: syn DDoS attack solution
  2007-05-31 20:08 ` Ric Messier
@ 2007-06-01  1:20   ` Tony.Ho
  2007-06-01  9:44   ` Bgs
  1 sibling, 0 replies; 34+ messages in thread
From: Tony.Ho @ 2007-06-01  1:20 UTC (permalink / raw)
  To: Ric Messier, netfilter

I have a thought about this.
I can use ipset and iptables on a bridge firewall.

ipt_recent module compares the SYN package and ACK package's TTL. If not 
match then drop.
ipt_hashlimit module stores the concurrent connections. When the 
connections exceed the threshold iptables would store the IP in ipset.
ipset's iptree modules can store the IP in a fixed time. When a IP which 
is in the iptree's list comes the firewall iptables would TARPIT its tcp 
connection.

Is this setting effective?


Ric Messier wrote:
> Bgs writes:
>   
>>   We recently got under a low traffic botnet DDoS attack. All attacker
>> nodes opened a single tcp session (just SYN part) and then did nothing.
>> This rules out rate limiting solutions and syncookie doesn't help
>> either. (Thousands of attacking nodes).
>>
>>     
>
> This is simply a SYN flood attack. It may or may not be a botnet (though
> saying botnet makes it sound sexier :-) ). A decent SYN flood attack tool
> would randomize the source address anyway. 
>
> You should try reading the following as a starting point:
>
> http://www.securityfocus.com/infocus/1729
>
> Your second suggestion has been implemented in the TCP/IP stack forever. The
> article above gives guidance on how to tune it in a Linux implementation.
>
> Ric
>
>
>
>   



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: syn DDoS attack solution
  2007-05-31 20:08 ` Ric Messier
  2007-06-01  1:20   ` Tony.Ho
@ 2007-06-01  9:44   ` Bgs
  2007-06-01 15:01     ` Ric Messier
  1 sibling, 1 reply; 34+ messages in thread
From: Bgs @ 2007-06-01  9:44 UTC (permalink / raw)
  To: Ric Messier; +Cc: netfilter


> This is simply a SYN flood attack. It may or may not be a botnet (though
> saying botnet makes it sound sexier :-) ). A decent SYN flood attack tool
> would randomize the source address anyway. 

Some more info about the attack: All IPs were real IPs otherway the tcp 
handshake wouldn't have made it. The attacker IPs were also consistent. 
They also new about the blocked IPs as after a new bunch of blocked IPs 
we fared OK then they added another bunch new IPs... we played this for 
quite some time...

All connections were in the ESTABLISHED state.

> 
> You should try reading the following as a starting point:
> 
> http://www.securityfocus.com/infocus/1729
> 
> Your second suggestion has been implemented in the TCP/IP stack forever. The
> article above gives guidance on how to tune it in a Linux implementation.

That part is about syncookies, backlog queue and half open timeouts. 
None of them applies here as all connections are legitimate in terms of 
SYN packets and tcp handshake.

How is the handling of ESTABLISHED connections implemented in the TCP/IP 
stack?


^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: syn DDoS attack solution
  2007-05-31 19:57 ` R. DuFresne
@ 2007-06-01  9:45   ` Bgs
  0 siblings, 0 replies; 34+ messages in thread
From: Bgs @ 2007-06-01  9:45 UTC (permalink / raw)
  To: R. DuFresne; +Cc: netfilter


>> What's your opinion?
>>
> 
> 
> That you knowledge ot TCP/IP is poor.

*clap*

> 
> 
> 
> Thanks,
> 
> Ron DuFresne
> - -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>         admin & senior security consultant:  sysinfo.com
>                         http://sysinfo.com
> Key fingerprint = 9401 4B13 B918 164C 647A  E838 B2DF AFCC 94B0 6629
> 
> ...We waste time looking for the perfect lover
> instead of creating the perfect love.
> 
>                 -Tom Robbins <Still Life With Woodpecker>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.5 (GNU/Linux)
> 
> iD8DBQFGXyiqst+vzJSwZikRAih2AJ0SY0npal/EqdwWK+/4SYOGaXY/uACggRtl
> uhPlH5F7JzCJGHf2RgryJT4=
> =Y3xX
> -----END PGP SIGNATURE-----


^ permalink raw reply	[flat|nested] 34+ messages in thread

* RE: syn DDoS attack solution
  2007-06-01  9:44   ` Bgs
@ 2007-06-01 15:01     ` Ric Messier
  2007-06-01 15:37       ` Bgs
  2007-06-01 21:34       ` Martijn Lievaart
  0 siblings, 2 replies; 34+ messages in thread
From: Ric Messier @ 2007-06-01 15:01 UTC (permalink / raw)
  To: 'Bgs'; +Cc: netfilter


Bgs writes:
> 
> Some more info about the attack: All IPs were real IPs otherway the tcp
> handshake wouldn't have made it. The attacker IPs were also consistent.
> They also new about the blocked IPs as after a new bunch of blocked IPs
> we fared OK then they added another bunch new IPs... we played this for
> quite some time...
> 
> All connections were in the ESTABLISHED state.
> 

Then your original description was incorrect or at least inadequate. It has
nothing to do with SYN as originally suggested since an ESTABLISHED
connection has blown past SYN, through SYN/ACK and by ACK. It has completed
the TCP handshake, as you note above. A SYN attack/flood would stop after
sending the initial SYN and leave the connection half-open to exhaust the
half-open buffers.

> 
> How is the handling of ESTABLISHED connections implemented in the
> TCP/IP
> stack?

There is likely a timer somewhere to time out connections that are just
hanging around doing nothing. You'd have to dig around to find it and turn
it down. You could also use something like tcpkill to get rid of them for
you. 


Ric




^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: syn DDoS attack solution
  2007-06-01 15:01     ` Ric Messier
@ 2007-06-01 15:37       ` Bgs
  2007-06-02 19:07         ` R. DuFresne
  2007-06-01 21:34       ` Martijn Lievaart
  1 sibling, 1 reply; 34+ messages in thread
From: Bgs @ 2007-06-01 15:37 UTC (permalink / raw)
  To: Ric Messier; +Cc: netfilter


Sorry if my first mail was misleading, I only slept about 5-6 hours this 
week in all :/

>> How is the handling of ESTABLISHED connections implemented in the
>> TCP/IP
>> stack?
> 
> There is likely a timer somewhere to time out connections that are just
> hanging around doing nothing. You'd have to dig around to find it and turn
> it down. You could also use something like tcpkill to get rid of them for
> you. 

What I'd like to achieve is different from the normal tcp timeouts. I 
only want a different (quite short in terms of everyday tcp traffic) 
timeout for connections that have not sent any data after the tcp 
handshake. The connections that did any traffic would become normal or 
'legitimate' and the usual tcp timeouts would apply.


About the netlink approach, here is what I was thinking about. I have 
never coded in the netlink/netfilter space so I'm completely in the dark 
if this can be done at all or not:

Client sends SYN to server. It arrives to the firewall. An info is sent 
to the userspace portion and the SYN packet is DROPed. The user space 
program logs the SYN packet and sends out an appropriately constructed 
SYN/ACK to the client. Client sends the ACK. The info is sent to util 
which logs the packet and puts the connection in allowed state as it 
knows from it's own db that the tcp handshake was ok. At this point it 
either starts the next phase or waits for the first data packet and 
starts the next phase then (allowing the data packet after it). The next 
phase is: userland replays the SYN handshake with the server thus making 
it receptive to the tcp stream to come. Through netlinks firewall part 
it would allow all subsequent traffic.

The main point is almost identical to the syn proxy thingie in bsd and 
ios, the main difference would be the ability to add some policy about 
when and how to let the traffic through to the server. For example with 
delayed replay, connections in ESTABLISHED state but without data would 
be dropped from the replay db stopping the culprit at the firewall.

Just a thought... can it be done (technically) or should I simply go to 
bed :)

Regards
Bgs





^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: syn DDoS attack solution
  2007-06-01 15:01     ` Ric Messier
  2007-06-01 15:37       ` Bgs
@ 2007-06-01 21:34       ` Martijn Lievaart
  2007-06-01 21:37         ` Martijn Lievaart
                           ` (3 more replies)
  1 sibling, 4 replies; 34+ messages in thread
From: Martijn Lievaart @ 2007-06-01 21:34 UTC (permalink / raw)
  To: Ric Messier; +Cc: netfilter

Ric Messier wrote:
> Bgs writes:
>   
>> Some more info about the attack: All IPs were real IPs otherway the tcp
>> handshake wouldn't have made it. The attacker IPs were also consistent.
>> They also new about the blocked IPs as after a new bunch of blocked IPs
>> we fared OK then they added another bunch new IPs... we played this for
>> quite some time...
>>
>> All connections were in the ESTABLISHED state.
>>
>>     
>
> Then your original description was incorrect or at least inadequate. It has
> nothing to do with SYN as originally suggested since an ESTABLISHED
> connection has blown past SYN, through SYN/ACK and by ACK. It has completed
> the TCP handshake, as you note above. A SYN attack/flood would stop after
> sending the initial SYN and leave the connection half-open to exhaust the
> half-open buffers.
>   

An connection is in the ESTABLISHED state once a packet has been seen. 
So once the SYN is seen, the state is ESTABLISHED.

M4



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: syn DDoS attack solution
  2007-06-01 21:34       ` Martijn Lievaart
@ 2007-06-01 21:37         ` Martijn Lievaart
  2007-06-01 21:38         ` Ric Messier
                           ` (2 subsequent siblings)
  3 siblings, 0 replies; 34+ messages in thread
From: Martijn Lievaart @ 2007-06-01 21:37 UTC (permalink / raw)
  Cc: Ric Messier, netfilter

Martijn Lievaart wrote:
> Ric Messier wrote:
>> Bgs writes:
>>  
>>> Some more info about the attack: All IPs were real IPs otherway the tcp
>>> handshake wouldn't have made it. The attacker IPs were also consistent.
>>> They also new about the blocked IPs as after a new bunch of blocked IPs
>>> we fared OK then they added another bunch new IPs... we played this for
>>> quite some time...
>>>
>>> All connections were in the ESTABLISHED state.
>>>
>>>     
>>
>> Then your original description was incorrect or at least inadequate. 
>> It has
>> nothing to do with SYN as originally suggested since an ESTABLISHED
>> connection has blown past SYN, through SYN/ACK and by ACK. It has 
>> completed
>> the TCP handshake, as you note above. A SYN attack/flood would stop 
>> after
>> sending the initial SYN and leave the connection half-open to exhaust 
>> the
>> half-open buffers.
>>   
>
> An connection is in the ESTABLISHED state once a packet has been seen. 
> So once the SYN is seen, the state is ESTABLISHED.
>

Ah scratch that. You're talking about open connections, not ipfilter 
state matching.

M4



^ permalink raw reply	[flat|nested] 34+ messages in thread

* RE: syn DDoS attack solution
  2007-06-01 21:34       ` Martijn Lievaart
  2007-06-01 21:37         ` Martijn Lievaart
@ 2007-06-01 21:38         ` Ric Messier
  2007-06-01 23:09         ` Ethy H. Brito
  2007-06-19 18:22         ` R. DuFresne
  3 siblings, 0 replies; 34+ messages in thread
From: Ric Messier @ 2007-06-01 21:38 UTC (permalink / raw)
  To: 'Martijn Lievaart'; +Cc: netfilter

Martijn Lievaart writes:
> 
> Ric Messier wrote:
> > Then your original description was incorrect or at least inadequate.
> It has
> > nothing to do with SYN as originally suggested since an ESTABLISHED
> > connection has blown past SYN, through SYN/ACK and by ACK. It has
> completed
> > the TCP handshake, as you note above. A SYN attack/flood would stop
> after
> > sending the initial SYN and leave the connection half-open to exhaust
> the
> > half-open buffers.
> >
> 
> An connection is in the ESTABLISHED state once a packet has been seen.
> So once the SYN is seen, the state is ESTABLISHED.
> 

Not last time I checked. That may be true to some degree in iptables but in
netstat, an ESTABLISHED connection is one that has made it through the
handshake process. Otherwise, it's in SYN_RECV state. 

Ric




^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: syn DDoS attack solution
  2007-06-01 21:34       ` Martijn Lievaart
  2007-06-01 21:37         ` Martijn Lievaart
  2007-06-01 21:38         ` Ric Messier
@ 2007-06-01 23:09         ` Ethy H. Brito
  2007-06-19 18:22         ` R. DuFresne
  3 siblings, 0 replies; 34+ messages in thread
From: Ethy H. Brito @ 2007-06-01 23:09 UTC (permalink / raw)
  To: netfilter

On Fri, 01 Jun 2007 23:34:02 +0200
Martijn Lievaart <m@rtij.nl> wrote:

> > Then your original description was incorrect or at least inadequate. It has
> > nothing to do with SYN as originally suggested since an ESTABLISHED
> > connection has blown past SYN, through SYN/ACK and by ACK. It has completed
> > the TCP handshake, as you note above. A SYN attack/flood would stop after
> > sending the initial SYN and leave the connection half-open to exhaust the
> > half-open buffers.
> >   
> 
> An connection is in the ESTABLISHED state once a packet has been seen. 
> So once the SYN is seen, the state is ESTABLISHED.

I think you meant "So once the SYN is seen, the state is NEW".

The state will change to ESTABLISHED as soon as netfiletr sees the SYN+ACK
response.

My 2 cents.
Cheers.

Ethy




^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: syn DDoS attack solution
  2007-06-01 15:37       ` Bgs
@ 2007-06-02 19:07         ` R. DuFresne
  2007-06-04 14:22           ` Ric Messier
  2007-06-04 17:24           ` Martin McKeay
  0 siblings, 2 replies; 34+ messages in thread
From: R. DuFresne @ 2007-06-02 19:07 UTC (permalink / raw)
  To: Bgs; +Cc: Ric Messier, netfilter

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Fri, 1 Jun 2007, Bgs wrote:

>
> Sorry if my first mail was misleading, I only slept about 5-6 hours this week 
> in all :/
>
>>> How is the handling of ESTABLISHED connections implemented in the
>>> TCP/IP
>>> stack?
>> 
>> There is likely a timer somewhere to time out connections that are just
>> hanging around doing nothing. You'd have to dig around to find it and turn
>> it down. You could also use something like tcpkill to get rid of them for
>> you. 
>
> What I'd like to achieve is different from the normal tcp timeouts. I only 
> want a different (quite short in terms of everyday tcp traffic) timeout for 
> connections that have not sent any data after the tcp handshake. The 
> connections that did any traffic would become normal or 'legitimate' and the 
> usual tcp timeouts would apply.
>
>
> About the netlink approach, here is what I was thinking about. I have never 
> coded in the netlink/netfilter space so I'm completely in the dark if this 
> can be done at all or not:
>
> Client sends SYN to server. It arrives to the firewall. An info is sent to 
> the userspace portion and the SYN packet is DROPed. The user space program 
> logs the SYN packet and sends out an appropriately constructed SYN/ACK to the 
> client. Client sends the ACK. The info is sent to util which logs the packet 
> and puts the connection in allowed state as it knows from it's own db that 
> the tcp handshake was ok. At this point it either starts the next phase or 
> waits for the first data packet and starts the next phase then (allowing the 
> data packet after it). The next phase is: userland replays the SYN handshake 
> with the server thus making it receptive to the tcp stream to come. Through 
> netlinks firewall part it would allow all subsequent traffic.
>
> The main point is almost identical to the syn proxy thingie in bsd and ios, 
> the main difference would be the ability to add some policy about when and 
> how to let the traffic through to the server. For example with delayed 
> replay, connections in ESTABLISHED state but without data would be dropped 
> from the replay db stopping the culprit at the firewall.
>
> Just a thought... can it be done (technically) or should I simply go to bed 
> :)
>



DDOS attacks work against resources, the tcp/ip stack <half opens (syn's) 
left hanging>, memory <cache of data in your syn-flood db>, drive space 
<system logs with overzealous logging of packet flow in limited space, or 
how much disk is devoted to a syn-flood db>, the size of your pipe...


In any but the most simplistic low bandwith attacks there is little one 
can do in such cases but either ride out the storm or go upstream for help in 
resolution <mist often a block of the damaging traffic>.  Even an 
semi-decent firewall defense against a simple low bandwith syn flood will 
need to be totally rework for defense in the case of a simple lowbandwidth 
ping flood, etc.  And once the attack level is amplified above the flow 
capabilities of your pipe, all bets are off.  In a serious flooding attack 
the firewall simply become a stopgate from preventing work on the local 
net from being affected.  There have been and will continue to be some 
rather decently funded companies with some fairly decent pipes wiped out 
of business or their internet presence closed up due to some of these 
kinds of attacks over extended periods of time.  Goverments across the 
globe have had internet services disrupted for extended periods. 
Microsoft has had to relocate servers to new net/ip addresses to divert 
the flow from such attacks and stay somewhat online...


Best way to avoid such problems is to not get into a whose prick is bigger 
contest with some kid in IRC that controls a 10,000-100,000 or larger 
series of zombies in their botnet.

It's the nature of the game, all in the design...



Thanks,

Ron DuFresne
- -- 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
         admin & senior security consultant:  sysinfo.com
                         http://sysinfo.com
Key fingerprint = 9401 4B13 B918 164C 647A  E838 B2DF AFCC 94B0 6629

...We waste time looking for the perfect lover
instead of creating the perfect love.

                 -Tom Robbins <Still Life With Woodpecker>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)

iD8DBQFGYcADst+vzJSwZikRApiLAJ444UDiM3HZnoNLO7ECHocT31r88wCeMhmS
Zv2jS1v6fCcb3gLbx9+KqHQ=
=iZ/G
-----END PGP SIGNATURE-----


^ permalink raw reply	[flat|nested] 34+ messages in thread

* RE: syn DDoS attack solution
  2007-06-02 19:07         ` R. DuFresne
@ 2007-06-04 14:22           ` Ric Messier
  2007-06-04 17:24           ` Martin McKeay
  1 sibling, 0 replies; 34+ messages in thread
From: Ric Messier @ 2007-06-04 14:22 UTC (permalink / raw)
  To: 'R. DuFresne', 'Bgs'; +Cc: netfilter


R. DuFresne writes:
> There have been and will continue to be some
> rather decently funded companies with some fairly decent pipes wiped
> out
> of business or their internet presence closed up due to some of these
> kinds of attacks over extended periods of time.  Goverments across the
> globe have had internet services disrupted for extended periods.
> Microsoft has had to relocate servers to new net/ip addresses to divert
> the flow from such attacks and stay somewhat online...
> 

I won't even tell you what we ended up having to do to get the first large,
commercial DDoS targets back online (Stacheldraht, anyone). The great thing
about DoS attacks is that they tend to be self-propagating. In order to get
them to go away, you generally end up doing things that kills legitimate
connections. Just a casualty of war. Either that or you ride them out.
Invariably, you have to go upstream to get help in stopping them if they are
worth noticing.

Ric




^ permalink raw reply	[flat|nested] 34+ messages in thread

* RE: syn DDoS attack solution
  2007-06-02 19:07         ` R. DuFresne
  2007-06-04 14:22           ` Ric Messier
@ 2007-06-04 17:24           ` Martin McKeay
  2007-06-04 23:16             ` R. DuFresne
  2007-06-05  8:29             ` Bgs
  1 sibling, 2 replies; 34+ messages in thread
From: Martin McKeay @ 2007-06-04 17:24 UTC (permalink / raw)
  To: R. DuFresne, Bgs; +Cc: Ric Messier, netfilter

I'm glad you summed up the technical aspects of the conversation so far.
A DDoS attack is a serious problem, no matter what form it takes.

I have one question for you: do you really believe that there's no
defense other than not waking the lion, that the battle is already lost?
That doesn't paint a pretty picture to me.  Are you basing this on
published work or personal experience?  Which I guess makes two
questions, so I'll quit while I'm ahead.

Martin

Martin McKeay, CISSP, GSNA
Cobia Product Evangelist
StillSecure
martin@stillsecure.com
707-495-7926
http://www.cobiablog.com

-- R. DuFrense wrote --


DDOS attacks work against resources, the tcp/ip stack <half opens
(syn's) left hanging>, memory <cache of data in your syn-flood db>,
drive space <system logs with overzealous logging of packet flow in
limited space, or how much disk is devoted to a syn-flood db>, the size
of your pipe...


In any but the most simplistic low bandwith attacks there is little one
can do in such cases but either ride out the storm or go upstream for
help in resolution <mist often a block of the damaging traffic>.  Even
an semi-decent firewall defense against a simple low bandwith syn flood
will need to be totally rework for defense in the case of a simple
lowbandwidth ping flood, etc.  And once the attack level is amplified
above the flow capabilities of your pipe, all bets are off.  In a
serious flooding attack the firewall simply become a stopgate from
preventing work on the local net from being affected.  There have been
and will continue to be some rather decently funded companies with some
fairly decent pipes wiped out of business or their internet presence
closed up due to some of these kinds of attacks over extended periods of
time.  Goverments across the globe have had internet services disrupted
for extended periods. 
Microsoft has had to relocate servers to new net/ip addresses to divert
the flow from such attacks and stay somewhat online...


Best way to avoid such problems is to not get into a whose prick is
bigger contest with some kid in IRC that controls a 10,000-100,000 or
larger series of zombies in their botnet.

It's the nature of the game, all in the design...



Thanks,

Ron DuFresne
- --
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
         admin & senior security consultant:  sysinfo.com
                         http://sysinfo.com Key fingerprint = 9401 4B13
B918 164C 647A  E838 B2DF AFCC 94B0 6629

...We waste time looking for the perfect lover instead of creating the
perfect love.

                 -Tom Robbins <Still Life With Woodpecker> -----BEGIN
PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)

iD8DBQFGYcADst+vzJSwZikRApiLAJ444UDiM3HZnoNLO7ECHocT31r88wCeMhmS
Zv2jS1v6fCcb3gLbx9+KqHQ=
=iZ/G
-----END PGP SIGNATURE-----



^ permalink raw reply	[flat|nested] 34+ messages in thread

* RE: syn DDoS attack solution
  2007-06-04 17:24           ` Martin McKeay
@ 2007-06-04 23:16             ` R. DuFresne
  2007-06-05  8:29             ` Bgs
  1 sibling, 0 replies; 34+ messages in thread
From: R. DuFresne @ 2007-06-04 23:16 UTC (permalink / raw)
  To: Martin McKeay; +Cc: Ric Messier, netfilter

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



For the most part, both, a yes to each of your questions.  And as many 
stated earlier, the only recourse is to go upstream for assistance in the 
case of a large scaled attack.

To get an idea of the issues involved in simply trying to locate the 
source<s> of an attack, try tracing back towards the offending addresses, 
taking into account the ability to spoof attack addresses, and 
understanding the concepts of packet mangling...


Thanks,

Ron DuFresne


On Mon, 4 Jun 2007, Martin McKeay wrote:

> I'm glad you summed up the technical aspects of the conversation so far.
> A DDoS attack is a serious problem, no matter what form it takes.
>
> I have one question for you: do you really believe that there's no
> defense other than not waking the lion, that the battle is already lost?
> That doesn't paint a pretty picture to me.  Are you basing this on
> published work or personal experience?  Which I guess makes two
> questions, so I'll quit while I'm ahead.
>
> Martin
>
> Martin McKeay, CISSP, GSNA
> Cobia Product Evangelist
> StillSecure
> martin@stillsecure.com
> 707-495-7926
> http://www.cobiablog.com
>
> -- R. DuFrense wrote --
>
>
> DDOS attacks work against resources, the tcp/ip stack <half opens
> (syn's) left hanging>, memory <cache of data in your syn-flood db>,
> drive space <system logs with overzealous logging of packet flow in
> limited space, or how much disk is devoted to a syn-flood db>, the size
> of your pipe...
>
>
> In any but the most simplistic low bandwith attacks there is little one
> can do in such cases but either ride out the storm or go upstream for
> help in resolution <mist often a block of the damaging traffic>.  Even
> an semi-decent firewall defense against a simple low bandwith syn flood
> will need to be totally rework for defense in the case of a simple
> lowbandwidth ping flood, etc.  And once the attack level is amplified
> above the flow capabilities of your pipe, all bets are off.  In a
> serious flooding attack the firewall simply become a stopgate from
> preventing work on the local net from being affected.  There have been
> and will continue to be some rather decently funded companies with some
> fairly decent pipes wiped out of business or their internet presence
> closed up due to some of these kinds of attacks over extended periods of
> time.  Goverments across the globe have had internet services disrupted
> for extended periods.
> Microsoft has had to relocate servers to new net/ip addresses to divert
> the flow from such attacks and stay somewhat online...
>
>
> Best way to avoid such problems is to not get into a whose prick is
> bigger contest with some kid in IRC that controls a 10,000-100,000 or
> larger series of zombies in their botnet.
>
> It's the nature of the game, all in the design...
>
>
>
> Thanks,
>
> Ron DuFresne
> - --
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>         admin & senior security consultant:  sysinfo.com
>                         http://sysinfo.com Key fingerprint = 9401 4B13
> B918 164C 647A  E838 B2DF AFCC 94B0 6629
>
> ...We waste time looking for the perfect lover instead of creating the
> perfect love.
>
>                 -Tom Robbins <Still Life With Woodpecker> -----BEGIN
> PGP SIGNATURE-----
> Version: GnuPG v1.4.5 (GNU/Linux)
>
> iD8DBQFGYcADst+vzJSwZikRApiLAJ444UDiM3HZnoNLO7ECHocT31r88wCeMhmS
> Zv2jS1v6fCcb3gLbx9+KqHQ=
> =iZ/G
> -----END PGP SIGNATURE-----
>
>

- -- 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
         admin & senior security consultant:  sysinfo.com
                         http://sysinfo.com
Key fingerprint = 9401 4B13 B918 164C 647A  E838 B2DF AFCC 94B0 6629

...We waste time looking for the perfect lover
instead of creating the perfect love.

                 -Tom Robbins <Still Life With Woodpecker>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)

iD8DBQFGZJ1Qst+vzJSwZikRAnOnAKCq3f7tiM9ynbYWbovDP4wmrCO6OACgvzIR
LQDjBIOKqpaz7N/8yMH3yhc=
=n3I0
-----END PGP SIGNATURE-----


^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: syn DDoS attack solution
  2007-06-04 17:24           ` Martin McKeay
  2007-06-04 23:16             ` R. DuFresne
@ 2007-06-05  8:29             ` Bgs
  2007-06-05 14:16               ` Steven M Campbell
  1 sibling, 1 reply; 34+ messages in thread
From: Bgs @ 2007-06-05  8:29 UTC (permalink / raw)
  To: Martin McKeay; +Cc: Ric Messier, netfilter

  You can have defense against many kind of ddos attacks but victory is 
not sure at all. Take the case for example when a very large number of 
distributed bots issues many but slow SYN/ACK bounce attacks or plain 
protocol connections to your site. If they do it 'right' you will end up 
with up to millions of sources doing 'ordinary' things with random 
sources. No source will ever trigger anything above an average user. One 
important step in taking ddos seriously was when the first ISP went 
broke because it was a target.

  So take up the fight when it happens. Most attackers are not 
resourceful enough (either by available hw/bots or technical knowledge), 
so on the long term you can usually win. But loosing the war is always a 
possibility no matter how good you are...

Martin McKeay wrote:
> I'm glad you summed up the technical aspects of the conversation so far.
> A DDoS attack is a serious problem, no matter what form it takes.
> 
> I have one question for you: do you really believe that there's no
> defense other than not waking the lion, that the battle is already lost?
> That doesn't paint a pretty picture to me.  Are you basing this on
> published work or personal experience?  Which I guess makes two
> questions, so I'll quit while I'm ahead.
> 
> Martin
> 
> Martin McKeay, CISSP, GSNA
> Cobia Product Evangelist
> StillSecure
> martin@stillsecure.com
> 707-495-7926
> http://www.cobiablog.com
> 
> -- R. DuFrense wrote --
> 
> 
> DDOS attacks work against resources, the tcp/ip stack <half opens
> (syn's) left hanging>, memory <cache of data in your syn-flood db>,
> drive space <system logs with overzealous logging of packet flow in
> limited space, or how much disk is devoted to a syn-flood db>, the size
> of your pipe...
> 
> 
> In any but the most simplistic low bandwith attacks there is little one
> can do in such cases but either ride out the storm or go upstream for
> help in resolution <mist often a block of the damaging traffic>.  Even
> an semi-decent firewall defense against a simple low bandwith syn flood
> will need to be totally rework for defense in the case of a simple
> lowbandwidth ping flood, etc.  And once the attack level is amplified
> above the flow capabilities of your pipe, all bets are off.  In a
> serious flooding attack the firewall simply become a stopgate from
> preventing work on the local net from being affected.  There have been
> and will continue to be some rather decently funded companies with some
> fairly decent pipes wiped out of business or their internet presence
> closed up due to some of these kinds of attacks over extended periods of
> time.  Goverments across the globe have had internet services disrupted
> for extended periods. 
> Microsoft has had to relocate servers to new net/ip addresses to divert
> the flow from such attacks and stay somewhat online...
> 
> 
> Best way to avoid such problems is to not get into a whose prick is
> bigger contest with some kid in IRC that controls a 10,000-100,000 or
> larger series of zombies in their botnet.
> 
> It's the nature of the game, all in the design...
> 
> 
> 
> Thanks,
> 
> Ron DuFresne
> - --
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>          admin & senior security consultant:  sysinfo.com
>                          http://sysinfo.com Key fingerprint = 9401 4B13
> B918 164C 647A  E838 B2DF AFCC 94B0 6629
> 
> ...We waste time looking for the perfect lover instead of creating the
> perfect love.
> 
>                  -Tom Robbins <Still Life With Woodpecker> -----BEGIN
> PGP SIGNATURE-----
> Version: GnuPG v1.4.5 (GNU/Linux)
> 
> iD8DBQFGYcADst+vzJSwZikRApiLAJ444UDiM3HZnoNLO7ECHocT31r88wCeMhmS
> Zv2jS1v6fCcb3gLbx9+KqHQ=
> =iZ/G
> -----END PGP SIGNATURE-----
> 


^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: syn DDoS attack solution
  2007-06-05  8:29             ` Bgs
@ 2007-06-05 14:16               ` Steven M Campbell
  2007-06-05 15:22                 ` ..prevention, was: " Arnt Karlsen
  0 siblings, 1 reply; 34+ messages in thread
From: Steven M Campbell @ 2007-06-05 14:16 UTC (permalink / raw)
  To: Bgs; +Cc: Ric Messier, netfilter

And, most important for folks here, do egress filtering on your 
firewall!    Help prevent zombie machines on your own networks from 
being a problem, you can't stop your end users from bringing infections 
into your network but you can control their spread.

Bgs wrote:
>  You can have defense against many kind of ddos attacks but victory is 
> not sure at all. Take the case for example when a very large number of 
> distributed bots issues many but slow SYN/ACK bounce attacks or plain 
> protocol connections to your site. If they do it 'right' you will end 
> up with up to millions of sources doing 'ordinary' things with random 
> sources. No source will ever trigger anything above an average user. 
> One important step in taking ddos seriously was when the first ISP 
> went broke because it was a target.
>
>  So take up the fight when it happens. Most attackers are not 
> resourceful enough (either by available hw/bots or technical 
> knowledge), so on the long term you can usually win. But loosing the 
> war is always a possibility no matter how good you are...
>
> Martin McKeay wrote:



^ permalink raw reply	[flat|nested] 34+ messages in thread

* ..prevention, was: syn DDoS attack solution
  2007-06-05 14:16               ` Steven M Campbell
@ 2007-06-05 15:22                 ` Arnt Karlsen
  2007-06-05 15:40                     ` Steven M Campbell
  0 siblings, 1 reply; 34+ messages in thread
From: Arnt Karlsen @ 2007-06-05 15:22 UTC (permalink / raw)
  To: netfilter

On Tue, 05 Jun 2007 10:16:40 -0400, Steven wrote in message 
<46657048.4040600@SCampbell.net>:

> And, most important for folks here, do egress filtering on your 
> firewall!    Help prevent zombie machines on your own networks from 
> being a problem, you can't stop your end users from bringing infections 
> into your network but you can control their spread.

..what tricks _are_ out there?  Set up some kinda p0f deamon and 
cut 'n tarpit any and all Wintendo network traffic attempts?  
Or even feed them LROS thru ActiveX if they need firm hints?

-- 
..med vennlig hilsen = with Kind Regards from Arnt... ;o)
...with a number of polar bear hunters in his ancestry...
  Scenarios always come in sets of three: 
  best case, worst case, and just in case.



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: ..prevention, was: syn DDoS attack solution
@ 2007-06-05 15:40                     ` Steven M Campbell
  2007-06-05 18:34                       ` Arnt Karlsen
  0 siblings, 1 reply; 34+ messages in thread
From: Steven M Campbell @ 2007-06-05 15:40 UTC (permalink / raw)
  To: Arnt Karlsen; +Cc: netfilter

Arnt Karlsen wrote:
> On Tue, 05 Jun 2007 10:16:40 -0400, Steven wrote in message 
> <46657048.4040600@SCampbell.net>:
>
>   
>> And, most important for folks here, do egress filtering on your 
>> firewall!    Help prevent zombie machines on your own networks from 
>> being a problem, you can't stop your end users from bringing infections 
>> into your network but you can control their spread.
>>     
>
> ..what tricks _are_ out there?  Set up some kinda p0f deamon and 
> cut 'n tarpit any and all Wintendo network traffic attempts?  
> Or even feed them LROS thru ActiveX if they need firm hints?
>
>   

Not really very tricky, limit outbound traffic to what is needed.   Do 
all of your workstations need UDP ports outbound?  Smtp?    For a lot of 
sites the average workstations internet requirements are very small, 
especially if proxys are used for SMTP,HTTP,FTP, etc.  Just by blocking 
most of the end users from direct internet access (or at least to a 
small set of outbound protocol/ports) we render those machines pretty 
useless to the bad guys. 

It does, however, become very tricky if it's not done up front.  It's 
really really tough to figure out what the requirements are if anyone 
could historically do anything they wanted.    It's far better to 
seriously restrict things up front and put in the exceptions as you find 
them.   Sites that have historically allowed all outbound traffic are a 
two fold problem, it's hard to fix and they are exactly the sorts of 
sites the bad guys like to use.



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: ..prevention, was: syn DDoS attack solution
  2007-06-05 15:40                     ` Steven M Campbell
@ 2007-06-05 18:34                       ` Arnt Karlsen
  0 siblings, 0 replies; 34+ messages in thread
From: Arnt Karlsen @ 2007-06-05 18:34 UTC (permalink / raw)
  To: Steven M Campbell; +Cc: netfilter

On Tue, 05 Jun 2007 11:40:36 -0400, Steven wrote in message 
<030f37e8000023b8@intmail01.analysts.com>:

> Arnt Karlsen wrote:
> > On Tue, 05 Jun 2007 10:16:40 -0400, Steven wrote in message 
> > <46657048.4040600@SCampbell.net>:
> >
> >   
> > > And, most important for folks here, do egress filtering on your 
> > > firewall!    Help prevent zombie machines on your own networks from 
> > > being a problem, you can't stop your end users from bringing infections 
> > > into your network but you can control their spread.
> > >     
> >
> > ..what tricks _are_ out there?  Set up some kinda p0f deamon and 
> > cut 'n tarpit any and all Wintendo network traffic attempts?  
> > Or even feed them LROS thru ActiveX if they need firm hints?
> > 
> 
> Not really very tricky, limit outbound traffic to what is needed.   Do 
> all of your workstations need UDP ports outbound?  Smtp?    For a lot of 
> sites the average workstations internet requirements are very small, 
> especially if proxys are used for SMTP,HTTP,FTP, etc.  Just by blocking 
> most of the end users from direct internet access (or at least to a 
> small set of outbound protocol/ports) we render those machines pretty 
> useless to the bad guys. 

..no problem, GNU/Linux only /25 shop here.

..but I also want to deny all windroids access to internet and make 
them fetch and use some safe os off my (Debian etc) lan mirror, before 
I allow them outbound.  Just a wee wifi spot biz idea, but if Microsoft
can't secure their "OS", HTFAISTDI?  ;o)

> It does, however, become very tricky if it's not done up front.  It's 
> really really tough to figure out what the requirements are if anyone 
> could historically do anything they wanted.    It's far better to 
> seriously restrict things up front and put in the exceptions as you find 
> them.   Sites that have historically allowed all outbound traffic are a 
> two fold problem, it's hard to fix and they are exactly the sorts of 
> sites the bad guys like to use.

..aye, but easy if it's your net, just enforce your ban on Wintendo.  
Now, I'm GNU+Linux only shop and have no recent Wintendo experience
since early 1998, I had Wintendo95 and ran away, so I look for pointers
on "ID their OS first, then chk if they have paid my bill and deserve
access."


-- 
..med vennlig hilsen = with Kind Regards from Arnt... ;o)
...with a number of polar bear hunters in his ancestry...
  Scenarios always come in sets of three: 
  best case, worst case, and just in case.



^ permalink raw reply	[flat|nested] 34+ messages in thread

* RE: ..prevention, was: syn DDoS attack solution
@ 2007-06-07 18:41                     ` Martin McKeay
  2007-06-08  1:40                       ` Arnt Karlsen
  2007-06-12 18:10                       ` Martin McKeay
  0 siblings, 2 replies; 34+ messages in thread
From: Martin McKeay @ 2007-06-07 18:41 UTC (permalink / raw)
  To: netfilter

I writing a blog post about the rules for blocking the bogon IP's
(unused IP networks) using the firewall module of our Cobia,  I was
considering including the equivalent iptables rules.  Would this be of
interest to anyone?


Martin McKeay, CISSP, GSNA
Cobia Product Evangelist
StillSecure
martin@stillsecure.com
707-495-7926
http://www.cobiablog.com

-----Original Message-----
From: netfilter-bounces@lists.netfilter.org
[mailto:netfilter-bounces@lists.netfilter.org] On Behalf Of Steven M
Campbell
Sent: Tuesday, June 05, 2007 9:41 AM
To: Arnt Karlsen
Cc: netfilter@lists.netfilter.org
Subject: Re: ..prevention, was: syn DDoS attack solution

Arnt Karlsen wrote:
> On Tue, 05 Jun 2007 10:16:40 -0400, Steven wrote in message
> <46657048.4040600@SCampbell.net>:
>
>   
>> And, most important for folks here, do egress filtering on your 
>> firewall!    Help prevent zombie machines on your own networks from 
>> being a problem, you can't stop your end users from bringing 
>> infections into your network but you can control their spread.
>>     
>
> ..what tricks _are_ out there?  Set up some kinda p0f deamon and cut 
> 'n tarpit any and all Wintendo network traffic attempts?
> Or even feed them LROS thru ActiveX if they need firm hints?
>
>   

Not really very tricky, limit outbound traffic to what is needed.   Do 
all of your workstations need UDP ports outbound?  Smtp?    For a lot of

sites the average workstations internet requirements are very small,
especially if proxys are used for SMTP,HTTP,FTP, etc.  Just by blocking
most of the end users from direct internet access (or at least to a
small set of outbound protocol/ports) we render those machines pretty
useless to the bad guys. 

It does, however, become very tricky if it's not done up front.  It's
really really tough to figure out what the requirements are if anyone 
could historically do anything they wanted.    It's far better to 
seriously restrict things up front and put in the exceptions as you find

them.   Sites that have historically allowed all outbound traffic are a 
two fold problem, it's hard to fix and they are exactly the sorts of
sites the bad guys like to use.




^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: ..prevention, was: syn DDoS attack solution
  2007-06-07 18:41                     ` Martin McKeay
@ 2007-06-08  1:40                       ` Arnt Karlsen
  2007-06-12 18:10                       ` Martin McKeay
  1 sibling, 0 replies; 34+ messages in thread
From: Arnt Karlsen @ 2007-06-08  1:40 UTC (permalink / raw)
  To: Martin McKeay; +Cc: netfilter

On Thu, 7 Jun 2007 12:41:44 -0600, Martin wrote in message 
<5C9E8CCEEB81ED498AC0C3B0054704F302A2379F@webmail.latis.com>:

> I writing a blog post about the rules for blocking the bogon IP's
> (unused IP networks) using the firewall module of our Cobia,  I was
> considering including the equivalent iptables rules.  Would this be of
> interest to anyone?

..yup.  Is _why_ I posted.
 
> Martin McKeay, CISSP, GSNA
> Cobia Product Evangelist
> StillSecure
> martin@stillsecure.com
> 707-495-7926
> http://www.cobiablog.com
> 
> Arnt Karlsen wrote:
> >
> > ..what tricks _are_ out there?  Set up some kinda p0f deamon and cut 
> > 'n tarpit any and all Wintendo network traffic attempts?
> > Or even feed them LROS thru ActiveX if they need firm hints?


-- 
..med vennlig hilsen = with Kind Regards from Arnt... ;o)
...with a number of polar bear hunters in his ancestry...
  Scenarios always come in sets of three: 
  best case, worst case, and just in case.



^ permalink raw reply	[flat|nested] 34+ messages in thread

* RE: ..prevention, was: syn DDoS attack solution
  2007-06-07 18:41                     ` Martin McKeay
  2007-06-08  1:40                       ` Arnt Karlsen
@ 2007-06-12 18:10                       ` Martin McKeay
  2007-06-12 22:44                         ` R. DuFresne
  1 sibling, 1 reply; 34+ messages in thread
From: Martin McKeay @ 2007-06-12 18:10 UTC (permalink / raw)
  To: netfilter

As promised, I wrote a blog entry on the blocking the bogon IP's and
included the full set of Iptables rules.
http://cobia.typepad.com/cobia_community_blog/how_to_use_cobia/index.htm
l

Thanks,

Martin    


Martin McKeay, CISSP, GSNA
Cobia Product Evangelist
StillSecure
martin@stillsecure.com
707-495-7926
http://www.cobiablog.com

-----Original Message-----
From: netfilter-bounces@lists.netfilter.org
[mailto:netfilter-bounces@lists.netfilter.org] On Behalf Of Martin
McKeay
Sent: Thursday, June 07, 2007 12:42 PM
To: netfilter@lists.netfilter.org
Subject: RE: ..prevention, was: syn DDoS attack solution

I writing a blog post about the rules for blocking the bogon IP's
(unused IP networks) using the firewall module of our Cobia,  I was
considering including the equivalent iptables rules.  Would this be of
interest to anyone?


Martin McKeay, CISSP, GSNA
Cobia Product Evangelist
StillSecure
martin@stillsecure.com
707-495-7926
http://www.cobiablog.com

-----Original Message-----
From: netfilter-bounces@lists.netfilter.org
[mailto:netfilter-bounces@lists.netfilter.org] On Behalf Of Steven M
Campbell
Sent: Tuesday, June 05, 2007 9:41 AM
To: Arnt Karlsen
Cc: netfilter@lists.netfilter.org
Subject: Re: ..prevention, was: syn DDoS attack solution

Arnt Karlsen wrote:
> On Tue, 05 Jun 2007 10:16:40 -0400, Steven wrote in message
> <46657048.4040600@SCampbell.net>:
>
>   
>> And, most important for folks here, do egress filtering on your 
>> firewall!    Help prevent zombie machines on your own networks from 
>> being a problem, you can't stop your end users from bringing 
>> infections into your network but you can control their spread.
>>     
>
> ..what tricks _are_ out there?  Set up some kinda p0f deamon and cut 
> 'n tarpit any and all Wintendo network traffic attempts?
> Or even feed them LROS thru ActiveX if they need firm hints?
>
>   

Not really very tricky, limit outbound traffic to what is needed.   Do 
all of your workstations need UDP ports outbound?  Smtp?    For a lot of

sites the average workstations internet requirements are very small,
especially if proxys are used for SMTP,HTTP,FTP, etc.  Just by blocking
most of the end users from direct internet access (or at least to a
small set of outbound protocol/ports) we render those machines pretty
useless to the bad guys. 

It does, however, become very tricky if it's not done up front.  It's
really really tough to figure out what the requirements are if anyone 
could historically do anything they wanted.    It's far better to 
seriously restrict things up front and put in the exceptions as you find

them.   Sites that have historically allowed all outbound traffic are a 
two fold problem, it's hard to fix and they are exactly the sorts of
sites the bad guys like to use.





^ permalink raw reply	[flat|nested] 34+ messages in thread

* RE: ..prevention, was: syn DDoS attack solution
  2007-06-12 18:10                       ` Martin McKeay
@ 2007-06-12 22:44                         ` R. DuFresne
  2007-06-13 14:24                           ` Martin McKeay
  0 siblings, 1 reply; 34+ messages in thread
From: R. DuFresne @ 2007-06-12 22:44 UTC (permalink / raw)
  To: Martin McKeay; +Cc: netfilter

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


With references to updating the continually changing bogon IP's, and how 
to add rules for those changes?

Thanks,

Ron DuFresne


On Tue, 12 Jun 2007, Martin McKeay wrote:

> As promised, I wrote a blog entry on the blocking the bogon IP's and
> included the full set of Iptables rules.
> http://cobia.typepad.com/cobia_community_blog/how_to_use_cobia/index.htm
> l
>
> Thanks,
>
> Martin
>
>
> Martin McKeay, CISSP, GSNA
> Cobia Product Evangelist
> StillSecure
> martin@stillsecure.com
> 707-495-7926
> http://www.cobiablog.com
>
> -----Original Message-----
> From: netfilter-bounces@lists.netfilter.org
> [mailto:netfilter-bounces@lists.netfilter.org] On Behalf Of Martin
> McKeay
> Sent: Thursday, June 07, 2007 12:42 PM
> To: netfilter@lists.netfilter.org
> Subject: RE: ..prevention, was: syn DDoS attack solution
>
> I writing a blog post about the rules for blocking the bogon IP's
> (unused IP networks) using the firewall module of our Cobia,  I was
> considering including the equivalent iptables rules.  Would this be of
> interest to anyone?
>
>
> Martin McKeay, CISSP, GSNA
> Cobia Product Evangelist
> StillSecure
> martin@stillsecure.com
> 707-495-7926
> http://www.cobiablog.com
>
> -----Original Message-----
> From: netfilter-bounces@lists.netfilter.org
> [mailto:netfilter-bounces@lists.netfilter.org] On Behalf Of Steven M
> Campbell
> Sent: Tuesday, June 05, 2007 9:41 AM
> To: Arnt Karlsen
> Cc: netfilter@lists.netfilter.org
> Subject: Re: ..prevention, was: syn DDoS attack solution
>
> Arnt Karlsen wrote:
>> On Tue, 05 Jun 2007 10:16:40 -0400, Steven wrote in message
>> <46657048.4040600@SCampbell.net>:
>>
>>
>>> And, most important for folks here, do egress filtering on your
>>> firewall!    Help prevent zombie machines on your own networks from
>>> being a problem, you can't stop your end users from bringing
>>> infections into your network but you can control their spread.
>>>
>>
>> ..what tricks _are_ out there?  Set up some kinda p0f deamon and cut
>> 'n tarpit any and all Wintendo network traffic attempts?
>> Or even feed them LROS thru ActiveX if they need firm hints?
>>
>>
>
> Not really very tricky, limit outbound traffic to what is needed.   Do
> all of your workstations need UDP ports outbound?  Smtp?    For a lot of
>
> sites the average workstations internet requirements are very small,
> especially if proxys are used for SMTP,HTTP,FTP, etc.  Just by blocking
> most of the end users from direct internet access (or at least to a
> small set of outbound protocol/ports) we render those machines pretty
> useless to the bad guys.
>
> It does, however, become very tricky if it's not done up front.  It's
> really really tough to figure out what the requirements are if anyone
> could historically do anything they wanted.    It's far better to
> seriously restrict things up front and put in the exceptions as you find
>
> them.   Sites that have historically allowed all outbound traffic are a
> two fold problem, it's hard to fix and they are exactly the sorts of
> sites the bad guys like to use.
>
>
>
>
>

- -- 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
         admin & senior security consultant:  sysinfo.com
                         http://sysinfo.com
Key fingerprint = 9401 4B13 B918 164C 647A  E838 B2DF AFCC 94B0 6629

...We waste time looking for the perfect lover
instead of creating the perfect love.

                 -Tom Robbins <Still Life With Woodpecker>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)

iD8DBQFGbyHYst+vzJSwZikRAsrtAKCEycOkf9ZeXynCnUDfAn5g+OoLpACfc+pZ
f1EvxxUjYYdHtR9Qo1+SMsc=
=xmHn
-----END PGP SIGNATURE-----


^ permalink raw reply	[flat|nested] 34+ messages in thread

* RE: ..prevention, was: syn DDoS attack solution
  2007-06-12 22:44                         ` R. DuFresne
@ 2007-06-13 14:24                           ` Martin McKeay
  2007-06-13 17:26                             ` Arnt Karlsen
  0 siblings, 1 reply; 34+ messages in thread
From: Martin McKeay @ 2007-06-13 14:24 UTC (permalink / raw)
  To: R. DuFresne; +Cc: netfilter

It's manual at this point.  The bogons aren't changing that often to the
best of my knowledge, maybe 3-4 times a year at most.  

Martin 


Martin McKeay, CISSP, GSNA
Cobia Product Evangelist
StillSecure
martin@stillsecure.com
707-495-7926
http://www.cobiablog.com

-----Original Message-----
From: R. DuFresne [mailto:dufresne@sysinfo.com] 
Sent: Tuesday, June 12, 2007 4:45 PM
To: Martin McKeay
Cc: netfilter@lists.netfilter.org
Subject: RE: ..prevention, was: syn DDoS attack solution

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


With references to updating the continually changing bogon IP's, and how
to add rules for those changes?

Thanks,

Ron DuFresne


On Tue, 12 Jun 2007, Martin McKeay wrote:

> As promised, I wrote a blog entry on the blocking the bogon IP's and 
> included the full set of Iptables rules.
> http://cobia.typepad.com/cobia_community_blog/how_to_use_cobia/index.h
> tm
> l
>


^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: ..prevention, was: syn DDoS attack solution
  2007-06-13 14:24                           ` Martin McKeay
@ 2007-06-13 17:26                             ` Arnt Karlsen
  2007-06-13 17:44                               ` Martin McKeay
  0 siblings, 1 reply; 34+ messages in thread
From: Arnt Karlsen @ 2007-06-13 17:26 UTC (permalink / raw)
  To: Martin McKeay; +Cc: netfilter

On Wed, 13 Jun 2007 08:24:24 -0600, Martin wrote in message 
<5C9E8CCEEB81ED498AC0C3B0054704F302A8410E@webmail.latis.com>:

> It's manual at this point.  The bogons aren't changing that often to the
> best of my knowledge, maybe 3-4 times a year at most.  

..rare enough to forgen't all about it then.  Cron job?

-- 
..med vennlig hilsen = with Kind Regards from Arnt... ;o)
...with a number of polar bear hunters in his ancestry...
  Scenarios always come in sets of three: 
  best case, worst case, and just in case.



^ permalink raw reply	[flat|nested] 34+ messages in thread

* RE: ..prevention, was: syn DDoS attack solution
  2007-06-13 17:26                             ` Arnt Karlsen
@ 2007-06-13 17:44                               ` Martin McKeay
  2007-06-14 19:43                                 ` R. DuFresne
  0 siblings, 1 reply; 34+ messages in thread
From: Martin McKeay @ 2007-06-13 17:44 UTC (permalink / raw)
  To: Arnt Karlsen; +Cc: netfilter

I'm not directly involved in the firewall for the corporation in my
current postion, but as the Security Manager at my last employer, one of
the things we did was set up a schedule of regular firewall reviews by a
committee consisting of the firewall admin, the network admin and
myself. We had a check list of the minimum requirements for the
firewalls, including the bogons and a final deny all rule.  It was a
e-commerce software/hosting company, so we had over 30 firewalls we
managed.  Our entire IT department, including myself and the IT Director
was 7 people, so you can guess this wasn't something people liked making
time for, but it saved us a more than once when someone had made a
mistake in the firewall configuration.

I like the idea of an automatic update, but I think it's more important
to have regular peer review of the firewall configuration.  It's worked
well for me in the past

Martin


Martin McKeay, CISSP, GSNA
Cobia Product Evangelist
StillSecure
martin@stillsecure.com
707-495-7926
http://www.cobiablog.com

-----Original Message-----
From: Arnt Karlsen [mailto:arnt@c2i.net] 
Sent: Wednesday, June 13, 2007 11:26 AM
To: Martin McKeay
Cc: R. DuFresne; netfilter@lists.netfilter.org
Subject: Re: ..prevention, was: syn DDoS attack solution

On Wed, 13 Jun 2007 08:24:24 -0600, Martin wrote in message
<5C9E8CCEEB81ED498AC0C3B0054704F302A8410E@webmail.latis.com>:

> It's manual at this point.  The bogons aren't changing that often to 
> the best of my knowledge, maybe 3-4 times a year at most.

..rare enough to forgen't all about it then.  Cron job?

--
..med vennlig hilsen = with Kind Regards from Arnt... ;o) ...with a
number of polar bear hunters in his ancestry...
  Scenarios always come in sets of three: 
  best case, worst case, and just in case.



^ permalink raw reply	[flat|nested] 34+ messages in thread

* RE: ..prevention, was: syn DDoS attack solution
  2007-06-13 17:44                               ` Martin McKeay
@ 2007-06-14 19:43                                 ` R. DuFresne
  2007-06-14 20:13                                   ` supportnew
  0 siblings, 1 reply; 34+ messages in thread
From: R. DuFresne @ 2007-06-14 19:43 UTC (permalink / raw)
  To: Martin McKeay; +Cc: Arnt Karlsen, netfilter

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, 13 Jun 2007, Martin McKeay wrote:

> I'm not directly involved in the firewall for the corporation in my
> current postion, but as the Security Manager at my last employer, one of
> the things we did was set up a schedule of regular firewall reviews by a
> committee consisting of the firewall admin, the network admin and
> myself. We had a check list of the minimum requirements for the
> firewalls, including the bogons and a final deny all rule.  It was a
> e-commerce software/hosting company, so we had over 30 firewalls we
> managed.  Our entire IT department, including myself and the IT Director
> was 7 people, so you can guess this wasn't something people liked making
> time for, but it saved us a more than once when someone had made a
> mistake in the firewall configuration.
>
> I like the idea of an automatic update, but I think it's more important
> to have regular peer review of the firewall configuration.  It's worked
> well for me in the past
>


That's kinda a lame cop-out for not wanting to update your online 
documentation and work up some scripts to accompany it.  Such a review 
does not have to negate an automated function to deall with blocking bogon 
blocks nor spam controls.  And automating such tasks is a good way to make 
sure rules and info are uptodate and currnet, even prior to a review.


Thanks,

Ron DuFresne
- -- 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
         admin & senior security consultant:  sysinfo.com
                         http://sysinfo.com
Key fingerprint = 9401 4B13 B918 164C 647A  E838 B2DF AFCC 94B0 6629

...We waste time looking for the perfect lover
instead of creating the perfect love.

                 -Tom Robbins <Still Life With Woodpecker>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)

iD8DBQFGcZp1st+vzJSwZikRAuv+AKDRGgGHaO010LFNxEqB5lYQoFzY4QCdEA4L
5QXNf/5/W6UUAyGLgH7O7+M=
=uzzl
-----END PGP SIGNATURE-----


^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: ..prevention, was: syn DDoS attack solution
  2007-06-14 19:43                                 ` R. DuFresne
@ 2007-06-14 20:13                                   ` supportnew
  0 siblings, 0 replies; 34+ messages in thread
From: supportnew @ 2007-06-14 20:13 UTC (permalink / raw)
  To: netfilter

R. DuFresne wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Wed, 13 Jun 2007, Martin McKeay wrote:
>
>> I'm not directly involved in the firewall for the corporation in my
>> current postion, but as the Security Manager at my last employer, one of
>> the things we did was set up a schedule of regular firewall reviews by a
>> committee consisting of the firewall admin, the network admin and
>> myself. We had a check list of the minimum requirements for the
>> firewalls, including the bogons and a final deny all rule.  It was a
>> e-commerce software/hosting company, so we had over 30 firewalls we
>> managed.  Our entire IT department, including myself and the IT Director
>> was 7 people, so you can guess this wasn't something people liked making
>> time for, but it saved us a more than once when someone had made a
>> mistake in the firewall configuration.
>>
>> I like the idea of an automatic update, but I think it's more important
>> to have regular peer review of the firewall configuration.  It's worked
>> well for me in the past
>>
>
>
> That's kinda a lame cop-out for not wanting to update your online 
> documentation and work up some scripts to accompany it.  Such a review 
> does not have to negate an automated function to deall with blocking 
> bogon blocks nor spam controls.  And automating such tasks is a good 
> way to make sure rules and info are uptodate and currnet, even prior 
> to a review.
>
>
> Thanks,
>
> Ron DuFresne
> - -- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>         admin & senior security consultant:  sysinfo.com
>                         http://sysinfo.com
> Key fingerprint = 9401 4B13 B918 164C 647A  E838 B2DF AFCC 94B0 6629
>
> ...We waste time looking for the perfect lover
> instead of creating the perfect love.
>
>                 -Tom Robbins <Still Life With Woodpecker>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.5 (GNU/Linux)
>
> iD8DBQFGcZp1st+vzJSwZikRAuv+AKDRGgGHaO010LFNxEqB5lYQoFzY4QCdEA4L
> 5QXNf/5/W6UUAyGLgH7O7+M=
> =uzzl
> -----END PGP SIGNATURE-----
>
>
Hi All,

For your info , the APF iptables firewall script has a bogon blocker , 
and also a cron mechanism to update the bogon list.

It also has some other excellent features such as

reactive address blocking (RAB), next generation in-line intrusion 
prevention
packet flow rate limiting that prevents abuse on the most widely abused 
protocol, icmp
dshield.org block list support to ban networks exhibiting suspicious 
activity
advanced packet sanity checks to make sure traffic coming and going 
meets the strictest of standards
filter attacks such as fragmented UDP, port zero floods, stuffed 
routing, arp poisoning and more
configurable kernel hooks (ties) to harden the system further to 
syn-flood attacks & routing abuses

I would suggest you have a look at the script.

Kev


^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: syn DDoS attack solution
  2007-06-01 21:34       ` Martijn Lievaart
                           ` (2 preceding siblings ...)
  2007-06-01 23:09         ` Ethy H. Brito
@ 2007-06-19 18:22         ` R. DuFresne
  2007-06-19 22:19           ` Robert Nichols
  3 siblings, 1 reply; 34+ messages in thread
From: R. DuFresne @ 2007-06-19 18:22 UTC (permalink / raw)
  To: Martijn Lievaart; +Cc: Ric Messier, netfilter

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Fri, 1 Jun 2007, Martijn Lievaart wrote:

> Ric Messier wrote:
>> Bgs writes:
>> 
>>> Some more info about the attack: All IPs were real IPs otherway the tcp
>>> handshake wouldn't have made it. The attacker IPs were also consistent.
>>> They also new about the blocked IPs as after a new bunch of blocked IPs
>>> we fared OK then they added another bunch new IPs... we played this for
>>> quite some time...
>>> 
>>> All connections were in the ESTABLISHED state.
>>>
>>> 
>> 
>> Then your original description was incorrect or at least inadequate. It has
>> nothing to do with SYN as originally suggested since an ESTABLISHED
>> connection has blown past SYN, through SYN/ACK and by ACK. It has completed
>> the TCP handshake, as you note above. A SYN attack/flood would stop after
>> sending the initial SYN and leave the connection half-open to exhaust the
>> half-open buffers.
>> 
>
> An connection is in the ESTABLISHED state once a packet has been seen. So 
> once the SYN is seen, the state is ESTABLISHED.
>


No, it is in state "new" with a mere syn sent.


Thanks,

Ron DuFresne
- -- 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
         admin & senior security consultant:  sysinfo.com
                         http://sysinfo.com
Key fingerprint = 9401 4B13 B918 164C 647A  E838 B2DF AFCC 94B0 6629

...We waste time looking for the perfect lover
instead of creating the perfect love.

                 -Tom Robbins <Still Life With Woodpecker>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)

iD8DBQFGeB7/st+vzJSwZikRAmbnAKDPbPVQcBsCzAkmoETaYT61EHVOAgCgwl0P
gTWFY+m1/+x1np1D7Rr1ulA=
=Eq4t
-----END PGP SIGNATURE-----


^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: syn DDoS attack solution
  2007-06-19 18:22         ` R. DuFresne
@ 2007-06-19 22:19           ` Robert Nichols
  2007-06-19 23:02             ` Pascal Hambourg
  2007-06-21 18:26             ` R. DuFresne
  0 siblings, 2 replies; 34+ messages in thread
From: Robert Nichols @ 2007-06-19 22:19 UTC (permalink / raw)
  To: netfilter

R. DuFresne wrote:
>
> On Fri, 1 Jun 2007, Martijn Lievaart wrote:
>>
>> An connection is in the ESTABLISHED state once a packet has been seen. 
>> So once the SYN is seen, the state is ESTABLISHED.
>>
> 
> 
> No, it is in state "new" with a mere syn sent.

You have to specify whether you are talking about the TCP connection
status or the conntrack status.  A mere SYN is sufficient to make an
ESTABLISHED status in conntrack.  If that were not true, then when
I send a TCP SYN packet the SYN/ACK would never make it back through
my firewall.

-- 
Bob Nichols     "NOSPAM" is really part of my email address.
                 Do NOT delete it.



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: syn DDoS attack solution
  2007-06-19 22:19           ` Robert Nichols
@ 2007-06-19 23:02             ` Pascal Hambourg
  2007-06-21 18:26             ` R. DuFresne
  1 sibling, 0 replies; 34+ messages in thread
From: Pascal Hambourg @ 2007-06-19 23:02 UTC (permalink / raw)
  To: netfilter

Hello,

Robert Nichols a écrit :
> R. DuFresne wrote:
>>
>> On Fri, 1 Jun 2007, Martijn Lievaart wrote:
>>>
>>> An connection is in the ESTABLISHED state once a packet has been 
>>> seen. So once the SYN is seen, the state is ESTABLISHED.
>>
>> No, it is in state "new" with a mere syn sent.

Correct.

> You have to specify whether you are talking about the TCP connection
> status or the conntrack status.  A mere SYN is sufficient to make an
> ESTABLISHED status in conntrack.

Nope. The ESTABLISHED state is reached when traffic has been seen in 
both "directions" (original and reply).

> If that were not true, then when
> I send a TCP SYN packet the SYN/ACK would never make it back through
> my firewall.

Why not ? The connection goes into the ESTABLISHED state as soon as the 
SYN/ACK is seen, and this state applies to the SYN/ACK packet itself.


^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: syn DDoS attack solution
  2007-06-19 22:19           ` Robert Nichols
  2007-06-19 23:02             ` Pascal Hambourg
@ 2007-06-21 18:26             ` R. DuFresne
  1 sibling, 0 replies; 34+ messages in thread
From: R. DuFresne @ 2007-06-21 18:26 UTC (permalink / raw)
  To: Robert Nichols; +Cc: netfilter

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Tue, 19 Jun 2007, Robert Nichols wrote:

> R. DuFresne wrote:
>> 
>> On Fri, 1 Jun 2007, Martijn Lievaart wrote:
>>> 
>>> An connection is in the ESTABLISHED state once a packet has been seen. So 
>>> once the SYN is seen, the state is ESTABLISHED.
>>> 
>> 
>> 
>> No, it is in state "new" with a mere syn sent.
>
> You have to specify whether you are talking about the TCP connection
> status or the conntrack status.  A mere SYN is sufficient to make an
> ESTABLISHED status in conntrack.  If that were not true, then when
> I send a TCP SYN packet the SYN/ACK would never make it back through
> my firewall.
>
>

remember, you fiter upon the state, new established and related <ignoring 
invalid here>.  I was correct, you need to review the filtering contexts <smile>.


Thanks,

Ron DuFresne
- -- 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
         admin & senior security consultant:  sysinfo.com
                         http://sysinfo.com
Key fingerprint = 9401 4B13 B918 164C 647A  E838 B2DF AFCC 94B0 6629

...We waste time looking for the perfect lover
instead of creating the perfect love.

                 -Tom Robbins <Still Life With Woodpecker>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)

iD8DBQFGesLxst+vzJSwZikRAtJXAJ4xGDmYV/xjS1rRQ1RwLUqNY5ulnQCbBkrL
oOttmX+tQwr8Ot1SSesPRBg=
=KF5d
-----END PGP SIGNATURE-----


^ permalink raw reply	[flat|nested] 34+ messages in thread

end of thread, other threads:[~2007-06-21 18:26 UTC | newest]

Thread overview: 34+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2007-05-31 16:19 syn DDoS attack solution Bgs
2007-05-31 19:57 ` R. DuFresne
2007-06-01  9:45   ` Bgs
2007-05-31 20:08 ` Ric Messier
2007-06-01  1:20   ` Tony.Ho
2007-06-01  9:44   ` Bgs
2007-06-01 15:01     ` Ric Messier
2007-06-01 15:37       ` Bgs
2007-06-02 19:07         ` R. DuFresne
2007-06-04 14:22           ` Ric Messier
2007-06-04 17:24           ` Martin McKeay
2007-06-04 23:16             ` R. DuFresne
2007-06-05  8:29             ` Bgs
2007-06-05 14:16               ` Steven M Campbell
2007-06-05 15:22                 ` ..prevention, was: " Arnt Karlsen
2007-06-05 15:40                   ` Steven M Campbell
2007-06-05 15:40                     ` Steven M Campbell
2007-06-05 18:34                       ` Arnt Karlsen
2007-06-07 18:41                     ` Martin McKeay
2007-06-08  1:40                       ` Arnt Karlsen
2007-06-12 18:10                       ` Martin McKeay
2007-06-12 22:44                         ` R. DuFresne
2007-06-13 14:24                           ` Martin McKeay
2007-06-13 17:26                             ` Arnt Karlsen
2007-06-13 17:44                               ` Martin McKeay
2007-06-14 19:43                                 ` R. DuFresne
2007-06-14 20:13                                   ` supportnew
2007-06-01 21:34       ` Martijn Lievaart
2007-06-01 21:37         ` Martijn Lievaart
2007-06-01 21:38         ` Ric Messier
2007-06-01 23:09         ` Ethy H. Brito
2007-06-19 18:22         ` R. DuFresne
2007-06-19 22:19           ` Robert Nichols
2007-06-19 23:02             ` Pascal Hambourg
2007-06-21 18:26             ` R. DuFresne

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.