Aren't these connections ESTABILISHED? (2nd take)

Aren't these connections ESTABILISHED? (2nd take)

[Gioele Barabucci]

I spend the last weeks doing experiments with iptables but still I have
problems with connections that should be ESTABILISHED but are not.

Postfix does some DNS lookups on the DNS server (69.93.28.254). After a bit,
iptables forget that the connection is ESTABILISHED and DROPs the reply.

My logs are full of dropped packets like these
05:32:33  69.93.28.254 53  myIP 2755  UDP
05:32:33  69.93.28.254 53  myIP 2755  UDP
05:32:53  69.93.28.254 53  myIP 2758  UDP
05:32:53  69.93.28.254 53  myIP 2758  UDP
05:33:13  69.93.28.254 53  myIP 2760  UDP
05:33:13  69.93.28.254 53  myIP 2760  UDP
05:33:34  69.93.28.254 53  myIP 2760  UDP
05:33:34  69.93.28.254 53  myIP 2760  UDP
05:33:34  69.93.28.254 53  myIP 2760  UDP
05:33:34  69.93.28.254 53  myIP 2760  UDP
05:38:08  69.93.28.254 53  myIP 2761  UDP
05:38:08  69.93.28.254 53  myIP 2761  UDP

Here is my ruleset (BTW, I did not test much the "limit SMTP trafic", do you
think that it is correct?)

iptables -F
iptables -X

echo "Default policies"
iptables -P INPUT DROP
iptables -P OUTPUT ACCEPT
iptables -P FORWARD DROP

echo "Passthrough for known good trafic"
iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -i lo -j ACCEPT #accept internal connections

echo "Allow only legal connection"
iptables -A INPUT -p tcp ! --syn -m state --state NEW -j LOG --log-level
"debug" --log-ip-options --log-tcp-options --log-prefix 'iptables INPUT
DROP !SYN '
iptables -A INPUT -p tcp ! --syn -m state --state NEW -j DROP

echo "Limit smtp traffic"
iptables -I INPUT -p tcp --dport smtp -i eth0 -m state --state NEW -m recent
--set
#iptables -I INPUT -p tcp --dport smtp -i eth0 -m state --state NEW -m
recent --update --seconds 30 --hitcount 4 -j LOG --log-level "debug"
--log-ip-options --log-tcp-options --log-prefix 'iptables INPUT DROP limit
'
iptables -I INPUT -p tcp --dport smtp -i eth0 -m state --state NEW -m recent
--update --seconds 30 --hitcount 4 -j DROP

echo "Exceptions for INPUT"
iptables -A INPUT -p tcp --dport ssh -j ACCEPT
iptables -A INPUT -p tcp --dport smtp -j ACCEPT
iptables -A INPUT -p tcp --dport pop3 -j ACCEPT
iptables -A INPUT -p tcp --dport imap -j ACCEPT
iptables -A INPUT -p tcp --dport http -j ACCEPT
iptables -A INPUT -p icmp -j ACCEPT


-- 
Gioele <dev@gioelebarabucci.com>

Re: Aren't these connections ESTABILISHED? (2nd take)

[/dev/rob0]

On Saturday 2005-October-01 14:12, Gioele Barabucci wrote:
> I spend the last weeks doing experiments with iptables but still I
> have problems with connections that should be ESTABILISHED but are

You have it spelled correctly in the script, but perhaps you should 
check again. ESTABLISHED != ESTABILISHED. I am not sure if iptables 
would complain about that or not, but it's always safest to spell 
things correctly. :)

> not.
>
> Postfix does some DNS lookups on the DNS server (69.93.28.254). After

FWIW: my Postfices do tons of DNS lookups, so much so, that I would 
never run without a caching nameserver on the same machine.

> a bit, iptables forget that the connection is ESTABILISHED and DROPs
> the reply.

When that happens you might want to check the conntrack table. Perhaps 
even script something to run from -j ULOG when a packet is dropped.

Is anything not working? I have a feeling these are just occasional 
strays that ip_conntrack isn't catching for some reason.

> My logs are full of dropped packets like these
> 05:32:33  69.93.28.254 53  myIP 2755  UDP

You *are* getting these from netfilter logs, correct? You have just 
removed all the superfluous information for readability?

> Here is my ruleset (BTW, I did not test much the "limit SMTP trafic",
> do you think that it is correct?)
snip
> echo "Limit smtp traffic"
> iptables -I INPUT -p tcp --dport smtp -i eth0 -m state --state NEW -m
> recent --set
> iptables -I INPUT -p tcp --dport smtp -i eth0 -m state --state NEW -m
> recent --update --seconds 30 --hitcount 4 -j DROP

I have not yet used -m recent. Without RTFM it looks like you are 
wanting to limit to limit any IP to 4 new connections per 30 second 
period. If the problem is dictionary attacks be advised that this might 
not help at all. The attacker could be attempting as many as 
smtpd_recipient_limit (default 1000) usernames in a single session.

Also, I'm not sure it would do anything at all, because there cannot be 
that many --state NEW connections in such a short time. Conntrack would 
call those "RELATED". I think you should try --syn, not --state NEW.
-- 
    mail to this address is discarded unless "/dev/rob0"
    or "not-spam" is in Subject: header

Re: Aren't these connections ESTABILISHED? (2nd take)

[Henrik Nordstrom]

On Sat, 1 Oct 2005, Gioele Barabucci wrote:

> Postfix does some DNS lookups on the DNS server (69.93.28.254). After a bit,
> iptables forget that the connection is ESTABILISHED and DROPs the reply.

Probably hits the UDP conntrack timeout. The default is only 30 seconds to 
receive a reply to the initial packet, and DNS can take longer..

Regards
Henrik

Re: Aren't these connections ESTABILISHED? (2nd take)

[Henrik Nordstrom]

On Sat, 1 Oct 2005, /dev/rob0 wrote:

> Also, I'm not sure it would do anything at all, because there cannot be
> that many --state NEW connections in such a short time. Conntrack would
> call those "RELATED". I think you should try --syn, not --state NEW.

The syn part is correct, but not RELATED.

each time a new connection is seen (unique source/destination/ports) the 
first packet is NEW, simply by the fact that the connection is not yet 
known to conntrack.

conntrack calls syn retransmits on already accepted connections as 
ESTABLISHED.


RELATED is "NEW" on other traffic flows which forms a known related 
connection to a already known connection. For example the data channel in 
the FTP protocol.


NEW is not related to syn, even if most TCP packets with state NEW is syn 
packets. Any packet from a TCP (or UDP) session not yet known to conntrack 
is NEW, even a TCP RST packet.


Regards
Henrik

Re: Aren't these connections ESTABILISHED? (2nd take)

[Robert Nichols]

Henrik Nordstrom wrote:
> each time a new connection is seen (unique source/destination/ports) the 
> first packet is NEW, simply by the fact that the connection is not yet 
> known to conntrack.

Actually, I find that an unsolicited TCP SYN/ACK packet matches state
INVALID and does not match state NEW.  Here is my log rule:

    -A Incoming -m state --state INVALID -m limit --limit 30/hour
       --limit-burst 20 -j LOG --log-prefix "FW-Inv: " --log-level 6

here is a typical log entry:

    Oct  1 18:45:59 omega-3a kernel: FW-Inv: IN=eth1 OUT=
       MAC=00:0c:6e:f9:ba:69:00:01:5c:22:c2:42:08:00 SRC=69.25.58.145
       DST=xx.xx.xx.xx LEN=44 TOS=0x00 PREC=0x00 TTL=50 ID=0 DF PROTO=TCP
       SPT=5555 DPT=35674 WINDOW=5840 RES=0x00 ACK SYN URGP=0

and here is the packet that produced it:

Frame 4265 (60 bytes on wire, 60 bytes captured)
Ethernet II, Src: 00:01:5c:22:c2:42 (00:01:5c:22:c2:42), Dst: 
00:0c:6e:f9:ba:69 (00:0c:6e:f9:ba:69)
Internet Protocol, Src: 69.25.58.145 (69.25.58.145), Dst: xx.xx.xx.xx 
(xx.xx.xx.xx)
     Version: 4
     Header length: 20 bytes
     Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00)
     Total Length: 44
     Identification: 0x0000 (0)
     Flags: 0x04 (Don't Fragment)
     Fragment offset: 0
     Time to live: 50
     Protocol: TCP (0x06)
     Header checksum: 0x1126 [correct]
     Source: 69.25.58.145 (69.25.58.145)
     Destination: xx.xx.xx.xx (xx.xx.xx.xx)
Transmission Control Protocol, Src Port: 5555 (5555), Dst Port: 35674 
(35674), Seq: 0, Ack: 1, Len: 0
     Source port: 5555 (5555)
     Destination port: 35674 (35674)
     Sequence number: 0    (relative sequence number)
     Acknowledgement number: 1    (relative ack number)
     Header length: 24 bytes
     Flags: 0x0012 (SYN, ACK)
     Window size: 5840
     Checksum: 0x5337 [correct]
     Options: (4 bytes)

0000  00 0c 6e f9 ba 69 00 01 5c 22 c2 42 08 00 45 00   ..n..i..\".B..E.
0010  00 2c 00 00 40 00 32 06 11 26 45 19 3a 91 xx xx   .,..@.2..&E.:...
0020  xx xx 15 b3 8b 5a 85 0e c0 80 00 00 0f cc 60 12   .....Z........`.
0030  16 d0 53 37 00 00 02 04 05 b4 00 00               ..S7........

According to ethereal, there had been no previous communication with the
host that sent the packet.  The "FW-Inv: " prefix is unique to this
rule.

I receive a handful of packets like that each day. I see the same thing
with packets having a bad TCP checksum.  They also get classified as
INVALID.

Kernel version is 2.6.12 on an i686.  IPtables version 1.2.11.

-- 
Bob Nichols         Yes, "NOSPAM" is really part of my email address.

Re: Aren't these connections ESTABILISHED? (2nd take)

[Henrik Nordstrom]

On Sat, 1 Oct 2005, Robert Nichols wrote:

> Actually, I find that an unsolicited TCP SYN/ACK packet matches state
> INVALID and does not match state NEW.

Right.. this is one of very few exceptions (SYN/ACK, FIN, RST) where 
otherwise valid TCP packets gets handled as INVALID if there is no 
existing session. A lonely SYN/ACK is nearly always "bad", and in worst 
case the SYN will need to be retransmitted for the session to get 
established when it is for real.

I am not entirely sure about the reasoning why a lone FIN is seen as 
invalid. The comment only says "Too late and no reason to do anything...". 
Also not entirely sure why a lone RST is seen as invalid. Both FIN and RST 
carries valuable meaning when resuming forgotten sessions.

> I receive a handful of packets like that each day. I see the same thing
> with packets having a bad TCP checksum.  They also get classified as
> INVALID.

Packets with bad checksums should be INVALID as these can by definition 
not be part of a valid session (the packets are not valid).

Regards
Henrik

Re: Aren't these connections ESTABILISHED? (2nd take)

[Jozsef Kadlecsik]

On Sun, 2 Oct 2005, Henrik Nordstrom wrote:

> I am not entirely sure about the reasoning why a lone FIN is seen as
> invalid. The comment only says "Too late and no reason to do anything...".
> Also not entirely sure why a lone RST is seen as invalid. Both FIN and RST
> carries valuable meaning when resuming forgotten sessions.

If a FIN receives which does not belong to any existing connection in
the conntrack table, which state should we assign to the "new" connection
"established" by the FIN? Was it the first FIN (half-closed session) or
the second FIN from the other direction?

I dunno how a lone RST could signal to pick up a connection.

Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlec@sunserv.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary

Re: Aren't these connections ESTABILISHED? (2nd take)

[Henrik Nordstrom]

On Sun, 2 Oct 2005, Jozsef Kadlecsik wrote:

> If a FIN receives which does not belong to any existing connection in
> the conntrack table, which state should we assign to the "new" connection
> "established" by the FIN? Was it the first FIN (half-closed session) or
> the second FIN from the other direction?

If you assume it's the second then it's a good approximation, and 
sufficient to push the TCP on the receiving host into a state from which 
there is a guaranteed timeout as it's then only waiting on itself to 
finish what it is doing.

If the FIN is dropped then communication may stall indefinitely unless the 
protocol is using an application level timeout or TCP keepalive.

> I dunno how a lone RST could signal to pick up a connection.

It should be the same as a RST in an existing connection, no more, not 
much less. It has some value to the receiving host telling it to shut down 
a matching TCP. But this is considerably less important than the FIN as 
the RST is very unreliable anyway and protocols using initiated RST (not 
in direct response to a received packet) ashould have appropriate 
application level timeouts on their connections.

Regards
Henrik

Re: Aren't these connections ESTABILISHED? (2nd take)

[Gioele Barabucci]

Henrik Nordstrom wrote:
> Probably hits the UDP conntrack timeout. The default is only 30 seconds to
> receive a reply to the initial packet, and DNS can take longer..
Is there a way to set this timeout to an higher value?

Is ip_ct_udp_timeout, or ip_ct_udp_timeout_stream, the correct sysctl key to
set?
Is there a iptables option to set longer timeout for a specific kind for
connection?

-- 
Gioele <dev@gioelebarabucci.com>

Re: Aren't these connections ESTABILISHED? (2nd take)

[Gioele Barabucci]

/dev/rob0 wrote:
> On Saturday 2005-October-01 14:12, Gioele Barabucci wrote:
>> I spend the last weeks doing experiments with iptables but still I
>> have problems with connections that should be ESTABILISHED but are
> 
> You have it spelled correctly in the script, but perhaps you should
> check again. ESTABLISHED != ESTABILISHED. I am not sure if iptables
> would complain about that or not, but it's always safest to spell
> things correctly. :)
Couldn't english borrow more from Saxons and less from Latins. There are so
many spell-alike words in italian and english :(
I'll tell you a secret: I got it wrong the first time I wrote the script :)

>> a bit, iptables forget that the connection is ESTABILISHED and DROPs
>> the reply.
> When that happens you might want to check the conntrack table. Perhaps
> even script something to run from -j ULOG when a packet is dropped.
Can I dump the packet in a file?

> Is anything not working? I have a feeling these are just occasional
> strays that ip_conntrack isn't catching for some reason.
Sometimes the SPF policy daemon that waits for the DNS dies. But... couldn't
this be the the cause and not the effect?
If an application start up a query and dies before the server replies, would
I see packets like the ones I see in my log?

>> Here is my ruleset (BTW, I did not test much the "limit SMTP trafic",
>> do you think that it is correct?)
>> echo "Limit smtp traffic"
> If the problem is dictionary attacks be advised that this might
> not help at all. The attacker could be attempting as many as
> smtpd_recipient_limit (default 1000) usernames in a single session.
It is to make Postfix handle few messages at a time. I removed these rules,
I'll think something better later.

> Also, I'm not sure it would do anything at all, because there cannot be
> that many --state NEW connections in such a short time. Conntrack would
> call those "RELATED". I think you should try --syn, not --state NEW.
I tried with netcat and the fifth connection was DROP'd. I'll investigate
about the differences between --syn and --state NEW. Thanks for the
corrections.


-- 
Gioele <dev@gioelebarabucci.com>

Re: Aren't these connections ESTABILISHED? (2nd take)

[/dev/rob0]

On Saturday 2005-October-01 21:16, Henrik Nordstrom wrote:
> On Sat, 1 Oct 2005, /dev/rob0 wrote:
> > Also, I'm not sure it would do anything at all, because there
> > cannot be that many --state NEW connections in such a short time.
> > Conntrack would call those "RELATED". I think you should try --syn,
> > not --state NEW.
>
> The syn part is correct, but not RELATED.

I know 2 things: RTFM and experience. From The Fine Manual: " ... 
RELATED meaning that the packet is starting a new connection, but is 
associated with an existing connection, such as an FTP data transfer, 
or an ICMP error."

How is this association known to conntrack?

Experience tells me that my little ssh attack blocking ploy using -m 
limit did not work if a --state RELATED,ESTABLISHED -j ACCEPT rule 
preceded it, but it did/does work if the rule matches only --state 
ESTABLISHED.

My inference therefrom was that the association is determined from the 
IP addresses. For example if my MTA makes an outbound connection to 
send mail to a remote site, and that site does an identd query, we have 
a RELATED connection.

> each time a new connection is seen (unique source/destination/ports)
> the first packet is NEW, simply by the fact that the connection is
> not yet known to conntrack.

TFM does appear to contradict this opinion. I'm unable to check the 
source code for the definite answer. (Unable to understand the source, 
I mean.) If you have done so please do clear up these issues for us.

> conntrack calls syn retransmits on already accepted connections as
> ESTABLISHED.

Yes, since conntrack itself is protocol-agnostic it won't know or care 
about TCP flags.

> RELATED is "NEW" on other traffic flows which forms a known related
> connection to a already known connection. For example the data
> channel in the FTP protocol.

Here again conntrack is protocol-agnostic. You and I know that with FTP 
we talk to the server on its 21/tcp, and when we try to GET from it we 
will expect the server to connect back to our port 20/tcp.

I do not believe that conntrack knows that. How it sees an FTP data 
connection is probably no different from my SMTP->out in<-identd 
example. It knows it has an ESTABLISHED connection between those two 
addresses, but not of any protocol relationship (because in this 
example there is none; identd checking is not defined in SMTP.)

> NEW is not related to syn, even if most TCP packets with state NEW is
> syn packets. Any packet from a TCP (or UDP) session not yet known to
> conntrack is NEW, even a TCP RST packet.

Sure, conntrack won't care about protocol features. But precisely how 
are you suggesting the determination of RELATED state is made?
-- 
    mail to this address is discarded unless "/dev/rob0"
    or "not-spam" is in Subject: header

Re: Aren't these connections ESTABILISHED? (2nd take)

[Jozsef Kadlecsik]

On Sun, 2 Oct 2005, Henrik Nordstrom wrote:

> On Sun, 2 Oct 2005, Jozsef Kadlecsik wrote:
>
> > If a FIN receives which does not belong to any existing connection in
> > the conntrack table, which state should we assign to the "new" connection
> > "established" by the FIN? Was it the first FIN (half-closed session) or
> > the second FIN from the other direction?
>
> If you assume it's the second then it's a good approximation, and
> sufficient to push the TCP on the receiving host into a state from which
> there is a guaranteed timeout as it's then only waiting on itself to
> finish what it is doing.

But that relies on the assumption that the receiver side wants to close
the session as well. If it enters the CLOSE_WAIT state instead, the
connection will hang anyway in spite of letting through the FIN and
assuming the LAST_ACK state.

However how would conntrack loose an (established) connection? Or are we
speaking of loading in conntrack "on the fly" when there are already
established connections flowing through the firewall? That's doable
but hairy and unreliable anyway due to the lost window scaling parameters.

> If the FIN is dropped then communication may stall indefinitely unless the
> protocol is using an application level timeout or TCP keepalive.
>
> > I dunno how a lone RST could signal to pick up a connection.
>
> It should be the same as a RST in an existing connection, no more, not
> much less. It has some value to the receiving host telling it to shut down
> a matching TCP. But this is considerably less important than the FIN as
> the RST is very unreliable anyway and protocols using initiated RST (not
> in direct response to a received packet) ashould have appropriate
> application level timeouts on their connections.

Somehow I have got bad feelings on passing random RST segments.

It is assumed (I think) that the hung connection is in the established
state. That does not hurt the host, does not pose as a possible DoS
situation or bottleneck. It also does not prevent establishing a new
connection over the hung one, we make special efforts for enabling that.

Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlec@sunserv.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary

Re: Aren't these connections ESTABILISHED? (2nd take)

[Jozsef Kadlecsik]

On Sun, 2 Oct 2005, /dev/rob0 wrote:

> On Saturday 2005-October-01 21:16, Henrik Nordstrom wrote:
> > On Sat, 1 Oct 2005, /dev/rob0 wrote:
> > > Also, I'm not sure it would do anything at all, because there
> > > cannot be that many --state NEW connections in such a short time.
> > > Conntrack would call those "RELATED". I think you should try --syn,
> > > not --state NEW.
> >
> > The syn part is correct, but not RELATED.
>
> I know 2 things: RTFM and experience. From The Fine Manual: " ...
> RELATED meaning that the packet is starting a new connection, but is
> associated with an existing connection, such as an FTP data transfer,
> or an ICMP error."
>
> How is this association known to conntrack?

There are protocol-dependent helper modules, like ip_conntrack_ftp which
job is to figure out the data channel parameters (RELATED connections)
from the commands and messages issued/passed on the command channel.

> Experience tells me that my little ssh attack blocking ploy using -m
> limit did not work if a --state RELATED,ESTABLISHED -j ACCEPT rule
> preceded it, but it did/does work if the rule matches only --state
> ESTABLISHED.

Without knowing the exact rules one cannot explain why you found that.

> My inference therefrom was that the association is determined from the
> IP addresses. For example if my MTA makes an outbound connection to
> send mail to a remote site, and that site does an identd query, we have
> a RELATED connection.

No, there is no helper module for such a task.

> > each time a new connection is seen (unique source/destination/ports)
> > the first packet is NEW, simply by the fact that the connection is
> > not yet known to conntrack.
>
> TFM does appear to contradict this opinion. I'm unable to check the
> source code for the definite answer. (Unable to understand the source,
> I mean.) If you have done so please do clear up these issues for us.
>
> > conntrack calls syn retransmits on already accepted connections as
> > ESTABLISHED.
>
> Yes, since conntrack itself is protocol-agnostic it won't know or care
> about TCP flags.

Then we could eliminate ip_conntrack_proto_tcp.c, don't we? ;-)

> > RELATED is "NEW" on other traffic flows which forms a known related
> > connection to a already known connection. For example the data
> > channel in the FTP protocol.
>
> Here again conntrack is protocol-agnostic. You and I know that with FTP
> we talk to the server on its 21/tcp, and when we try to GET from it we
> will expect the server to connect back to our port 20/tcp.
>
> I do not believe that conntrack knows that. How it sees an FTP data
> connection is probably no different from my SMTP->out in<-identd
> example.

It *is* completely different. Building up an FTP data channel is well
defined in RFC 959 and supported by a helper module in netfilter.
SMTP/POP/IMAP/... and ident lookups form a much loose relationship, there
is no "ident" helper.

Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlec@sunserv.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary

Re: Aren't these connections ESTABILISHED? (2nd take)

[Jozsef Kadlecsik]

On Sun, 2 Oct 2005, Jozsef Kadlecsik wrote:

> On Sun, 2 Oct 2005, Henrik Nordstrom wrote:
>
> > On Sun, 2 Oct 2005, Jozsef Kadlecsik wrote:
> >
> > > If a FIN receives which does not belong to any existing connection in
> > > the conntrack table, which state should we assign to the "new" connection
> > > "established" by the FIN? Was it the first FIN (half-closed session) or
> > > the second FIN from the other direction?
> >
> > If you assume it's the second then it's a good approximation, and
> > sufficient to push the TCP on the receiving host into a state from which
> > there is a guaranteed timeout as it's then only waiting on itself to
> > finish what it is doing.
>
> But that relies on the assumption that the receiver side wants to close
> the session as well. If it enters the CLOSE_WAIT state instead, the
> connection will hang anyway in spite of letting through the FIN and
> assuming the LAST_ACK state.

Hmmm, I must correct myself: the conntrack entry might get timed out, but
then seeing the pure ACKs it picks up the connection again. Still:

> However how would conntrack loose an (established) connection? Or are we
> speaking of loading in conntrack "on the fly" when there are already
> established connections flowing through the firewall? That's doable
> but hairy and unreliable anyway due to the lost window scaling parameters.

Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlec@sunserv.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary

Re: Aren't these connections ESTABILISHED? (2nd take)

[/dev/rob0]

On Sunday 2005-October-02 15:38, Jozsef Kadlecsik wrote:
> > I know 2 things: RTFM and experience. From The Fine Manual: " ...
> > RELATED meaning that the packet is starting a new connection, but
> > is associated with an existing connection, such as an FTP data
> > transfer, or an ICMP error."
> >
> > How is this association known to conntrack?
>
> There are protocol-dependent helper modules, like ip_conntrack_ftp
> which job is to figure out the data channel parameters (RELATED
> connections) from the commands and messages issued/passed on the
> command channel.

Are you saying that RELATED is only known through the use of helper 
modules?

> > Experience tells me that my little ssh attack blocking ploy using
> > -m limit did not work if a --state RELATED,ESTABLISHED -j ACCEPT
> > rule preceded it, but it did/does work if the rule matches only
> > --state ESTABLISHED.
>
> Without knowing the exact rules one cannot explain why you found
> that.

I had what amounted to this:
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p tcp --dport ssh -j Ssh
-A Ssh -m limit --limit 3/m --limit-burst 3 -j ACCEPT
-A Ssh -j DROP

That does nothing to deter the SSH attack bots. This works, however:
-A INPUT -p tcp --dport ssh -j Ssh
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A Ssh -m state --state ESTABLISHED -j ACCEPT
-A Ssh -m limit --limit 3/m --limit-burst 3 -j ACCEPT
-A Ssh -j DROP

The fundamental difference of course is that --state RELATED are not 
accepted before the --limit.

An SSH attack bot is a brute force attempt to gain shell access through 
common usernames and passwords. They scan for open sshd's on 22/tcp and 
then do this. Each connection (--syn) tries one username/password 
combination. They keep coming back with dozens of these whenever an 
open sshd has been found. If you run sshd open to the Internet you 
probably have seen these attacks in your logs.

Each subsequent --syn packet has the same source IP, and the same 
destination IP and port, as the first one which is by now in --state 
ESTABLISHED.

So you are telling me that those are not --state RELATED? My firewall 
disagrees. The rules above are blocking SSH brute force attacks after 
the 3rd attempt.

> > My inference therefrom was that the association is determined from
> > the IP addresses. For example if my MTA makes an outbound
> > connection to send mail to a remote site, and that site does an
> > identd query, we have a RELATED connection.
>
> No, there is no helper module for such a task.

Okay, so I think we need some more explicit wording in the 
documentation. What is RELATED? I have not found anything which 
explains it completely.
-- 
    mail to this address is discarded unless "/dev/rob0"
    or "not-spam" is in Subject: header

Re: Aren't these connections ESTABILISHED? (2nd take)

[Henrik Nordstrom]

On Sun, 2 Oct 2005, Jozsef Kadlecsik wrote:

> But that relies on the assumption that the receiver side wants to close
> the session as well. If it enters the CLOSE_WAIT state instead, the
> connection will hang anyway in spite of letting through the FIN and
> assuming the LAST_ACK state.

Yes, but at that stage it is not a network problem, only a badly designed 
application who do not shut down when the other end closes the connection.

CLOSE_WAIT is waiting for the local application to close the local end of 
the connection.

> However how would conntrack loose an (established) connection? Or are we
> speaking of loading in conntrack "on the fly" when there are already
> established connections flowing through the firewall? That's doable
> but hairy and unreliable anyway due to the lost window scaling parameters.

Yes. Or more realisticly reboot of the firewall/router or failover between 
two firewalls/routers not using ctsync.

> Somehow I have got bad feelings on passing random RST segments.

And as I said it is of very little value.

Regards
Henrik

Re: Aren't these connections ESTABILISHED? (2nd take)

[Henrik Nordstrom]

On Sun, 2 Oct 2005, Jozsef Kadlecsik wrote:

> Hmmm, I must correct myself: the conntrack entry might get timed out, but
> then seeing the pure ACKs it picks up the connection again. Still:
>
>> However how would conntrack loose an (established) connection?

One case where the timeouts allows the above quite easily even if 
conntrack is running all the time is a half-closed connection. Here 
conntrack has entered FIN_WAIT with only a 2 minutes timeout.

Regards
Henrik

Re: Aren't these connections ESTABILISHED? (2nd take)

[Henrik Nordstrom]

On Sun, 2 Oct 2005, /dev/rob0 wrote:

> I know 2 things: RTFM and experience. From The Fine Manual: " ...
> RELATED meaning that the packet is starting a new connection, but is
> associated with an existing connection, such as an FTP data transfer,
> or an ICMP error."
>
> How is this association known to conntrack?

By a application protocol helper module registering an expectation of a 
related connection.

The manual is actually slightly incorrect. ICMP errors normally gets 
sorted as ESTABLISHED, not RELATED.

> Experience tells me that my little ssh attack blocking ploy using -m
> limit did not work if a --state RELATED,ESTABLISHED -j ACCEPT rule
> preceded it, but it did/does work if the rule matches only --state
> ESTABLISHED.

This is very odd and should not happen. SSH connections can by definition 
never be RELATED.

> My inference therefrom was that the association is determined from the
> IP addresses. For example if my MTA makes an outbound connection to
> send mail to a remote site, and that site does an identd query, we have
> a RELATED connection.

Nope. That will be a NEW session. Uless ofcourse if you write a conntrack 
application helper module which registers the ident expectation when your 
MTA makes an outbound SMTP request. Not difficult (only a couple of lines 
of code, not counting the common parts looking the same in all these 
helpers) but not normally done.

> Here again conntrack is protocol-agnostic. You and I know that with FTP
> we talk to the server on its 21/tcp, and when we try to GET from it we
> will expect the server to connect back to our port 20/tcp.

conntrack is not protocol agnostic, only the spirit of conntrack is (it 
should track connections, not police the traffic). It knows a great deal 
of TCP, and there is application helper modules for FTP and a number of 
other application protocols needing RELATED connections.

It is these application protocol helper modules which registers the 
expected connections (i.e. FTP data channel) and making them RELATED 
instead of NEW when the connection is seen. When the FTP application 
protocol helper module sees a PORT command or PASV reply it registers the 
connection specified in these as an expected connection RELATED to this 
FTP control connection, and the SYN on that connection will be seen as 
RELATED instead of NEW.

Regards
Henrik

Re: Aren't these connections ESTABILISHED? (2nd take)

[Henrik Nordstrom]

On Sun, 2 Oct 2005, /dev/rob0 wrote:

> Are you saying that RELATED is only known through the use of helper
> modules?

Yes.

Regards
Henrik

Re: Aren't these connections ESTABILISHED? (2nd take)

[Jozsef Kadlecsik]

On Mon, 3 Oct 2005, Henrik Nordstrom wrote:

> > How is this association known to conntrack?
>
> By a application protocol helper module registering an expectation of a
> related connection.
>
> The manual is actually slightly incorrect. ICMP errors normally gets
> sorted as ESTABLISHED, not RELATED.

According to the source code the manual is correct and ICMP error messages
are always RELATED (or INVALID).

Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlec@sunserv.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary