conntrack and ICMP echo replies not showing as ESTABLISHED

conntrack and ICMP echo replies not showing as ESTABLISHED

[Oliver O'Boyle]

Hi,

Running on Ubuntu 16.04LTS with a fairly basic iptables config (see below).

I'm sure I'm missing something obvious but I am not seeing echo
replies showing as ESTABLISHED when running:

sudo conntrack -E

I see this:

[NEW] icmp     1 30 src=192.168.144.164 dst=192.168.144.254 type=8
code=0 id=1 [UNREPLIED] src=192.168.144.254 dst=192.168.144.164 type=0
code=0 id=1
[UPDATE] icmp     1 30 src=192.168.144.164 dst=192.168.144.254 type=8
code=0 id=1 src=192.168.144.254 dst=192.168.144.164 type=0 code=0 id=1

But I expected to see something more like line 3 where it shows the
connection is ESTABLISHED:

[NEW] tcp      6 120 SYN_SENT src=192.168.144.164 dst=52.12.252.111
sport=10032 dport=443 [UNREPLIED] src=52.72.252.111 dst=216.46.2.101
sport=443 dport=10032
[UPDATE] tcp      6 60 SYN_RECV src=192.168.144.164 dst=52.12.252.111
sport=10032 dport=443 src=52.72.252.111 dst=216.46.2.101 sport=443
dport=10032
[UPDATE] tcp      6 432000 ESTABLISHED src=192.168.144.164
dst=52.12.252.111 sport=10032 dport=443 src=52.72.252.111
dst=216.46.2.101 sport=443 dport=10032 [ASSURED]


I'm new to netfilter/iptables so I may be missing something or may
just be expecting something that doesn't happen.

Can someone point me in the right direction, please?

Thanks for the help!

O.

FILTER TABLE:

Chain INPUT (policy DROP 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source
  destination
1      162  9900 ACCEPT     all  --  any    any     anywhere
  anywhere             ctstate RELATED,ESTABLISHED
2        0     0 ACCEPT     all  --  lo     any     anywhere
  anywhere             ctstate NEW
3        2   120 ACCEPT     icmp --  any    any     anywhere
  anywhere             ctstate NEW
4        0     0 ACCEPT     tcp  --  ens192 any     anywhere
  anywhere             tcp dpt:ssh
5       51  4426 LOG        all  --  any    any     anywhere
  anywhere             limit: avg 5/min burst 5 LOG level debug prefix
"iptables denied: "
6     1986  226K DROP       all  --  any    any     anywhere
  anywhere

Chain FORWARD (policy ACCEPT 2 packets, 80 bytes)
num   pkts bytes target     prot opt in     out     source
  destination
1     3359 1788K ACCEPT     all  --  ens160 ens192  anywhere
  anywhere             ctstate RELATED,ESTABLISHED
2     4887  896K ACCEPT     all  --  ens192 ens160  anywhere
  anywhere             ctstate NEW,RELATED,ESTABLISHED

Chain OUTPUT (policy ACCEPT 214 packets, 41776 bytes)
num   pkts bytes target     prot opt in     out     source
  destination
1       13   780 ACCEPT     icmp --  any    any     anywhere
  anywhere             ctstate NEW,RELATED,ESTABLISHED

NAT TABLE:

Chain PREROUTING (policy ACCEPT 2587 packets, 269K bytes)
num   pkts bytes target     prot opt in     out     source
  destination

Chain INPUT (policy ACCEPT 2 packets, 120 bytes)
num   pkts bytes target     prot opt in     out     source
  destination

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source
  destination

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source
  destination
1      429 22583 MASQUERADE  all  --  any    ens160  anywhere
   anywhere

RE: conntrack and ICMP echo replies not showing as ESTABLISHED

[André Paulsberg-Csibi (IBM Consultant)]

As far as I can tell - ESTABLISHED - is only for session based protocols like TCP .
You will not see that for UDP or ICMP , as far as CONNTRACK is concerned .

IN the ruleset it will fall under the STATE of ESTABLISHED , as IPTABLES understands that this is direct return traffic .
However , like UDP it has no real "IP state" outside of IPTABLES and my guess this is why it not refered to in that state in CONNTRACK .


Best regards
André Paulsberg-Csibi
Senior Network Engineer 
IBM Services AS

-----Original Message-----
From: netfilter-owner@vger.kernel.org [mailto:netfilter-owner@vger.kernel.org] On Behalf Of Oliver O'Boyle
Sent: Tuesday, January 2, 2018 10:05 PM
To: netfilter@vger.kernel.org
Subject: conntrack and ICMP echo replies not showing as ESTABLISHED

Hi,

Running on Ubuntu 16.04LTS with a fairly basic iptables config (see below).

I'm sure I'm missing something obvious but I am not seeing echo replies showing as ESTABLISHED when running:

sudo conntrack -E

I see this:

[NEW] icmp     1 30 src=192.168.144.164 dst=192.168.144.254 type=8
code=0 id=1 [UNREPLIED] src=192.168.144.254 dst=192.168.144.164 type=0
code=0 id=1
[UPDATE] icmp     1 30 src=192.168.144.164 dst=192.168.144.254 type=8
code=0 id=1 src=192.168.144.254 dst=192.168.144.164 type=0 code=0 id=1

But I expected to see something more like line 3 where it shows the connection is ESTABLISHED:

[NEW] tcp      6 120 SYN_SENT src=192.168.144.164 dst=52.12.252.111
sport=10032 dport=443 [UNREPLIED] src=52.72.252.111 dst=216.46.2.101
sport=443 dport=10032
[UPDATE] tcp      6 60 SYN_RECV src=192.168.144.164 dst=52.12.252.111
sport=10032 dport=443 src=52.72.252.111 dst=216.46.2.101 sport=443
dport=10032
[UPDATE] tcp      6 432000 ESTABLISHED src=192.168.144.164
dst=52.12.252.111 sport=10032 dport=443 src=52.72.252.111
dst=216.46.2.101 sport=443 dport=10032 [ASSURED]


I'm new to netfilter/iptables so I may be missing something or may just be expecting something that doesn't happen.

Can someone point me in the right direction, please?

Thanks for the help!

O.

FILTER TABLE:

Chain INPUT (policy DROP 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source
  destination
1      162  9900 ACCEPT     all  --  any    any     anywhere
  anywhere             ctstate RELATED,ESTABLISHED
2        0     0 ACCEPT     all  --  lo     any     anywhere
  anywhere             ctstate NEW
3        2   120 ACCEPT     icmp --  any    any     anywhere
  anywhere             ctstate NEW
4        0     0 ACCEPT     tcp  --  ens192 any     anywhere
  anywhere             tcp dpt:ssh
5       51  4426 LOG        all  --  any    any     anywhere
  anywhere             limit: avg 5/min burst 5 LOG level debug prefix
"iptables denied: "
6     1986  226K DROP       all  --  any    any     anywhere
  anywhere

Chain FORWARD (policy ACCEPT 2 packets, 80 bytes)
num   pkts bytes target     prot opt in     out     source
  destination
1     3359 1788K ACCEPT     all  --  ens160 ens192  anywhere
  anywhere             ctstate RELATED,ESTABLISHED
2     4887  896K ACCEPT     all  --  ens192 ens160  anywhere
  anywhere             ctstate NEW,RELATED,ESTABLISHED

Chain OUTPUT (policy ACCEPT 214 packets, 41776 bytes)
num   pkts bytes target     prot opt in     out     source
  destination
1       13   780 ACCEPT     icmp --  any    any     anywhere
  anywhere             ctstate NEW,RELATED,ESTABLISHED

NAT TABLE:

Chain PREROUTING (policy ACCEPT 2587 packets, 269K bytes)
num   pkts bytes target     prot opt in     out     source
  destination

Chain INPUT (policy ACCEPT 2 packets, 120 bytes)
num   pkts bytes target     prot opt in     out     source
  destination

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source
  destination

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source
  destination
1      429 22583 MASQUERADE  all  --  any    ens160  anywhere
   anywhere
--
To unsubscribe from this list: send the line "unsubscribe netfilter" in the body of a message to majordomo@vger.kernel.org More majordomo info at  https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fvger.kernel.org%2Fmajordomo-info.html&data=02%7C01%7C%7Ccb9c5741e81048a958b508d55224923c%7C40cc2915e2834a2794716bdd7ca4c6e1%7C1%7C0%7C636505239381260666&sdata=OcUhnFtFPurBJphvLKbaxe5fVENe5Fs8eSxUtbfKGuY%3D&reserved=0

Re: conntrack and ICMP echo replies not showing as ESTABLISHED

[Neal P. Murphy]

On Tue, 2 Jan 2018 21:56:07 +0000
André Paulsberg-Csibi (IBM Consultant)         <Andre.Paulsberg-Csibi@evry.com> wrote:

> As far as I can tell - ESTABLISHED - is only for session based protocols like TCP .
> You will not see that for UDP or ICMP , as far as CONNTRACK is concerned .  

With netfilter, there are five connection states: NEW, RELATED, ESTABLISHED, INVALID and RAW.
  - A NEW packet is the first packet of a new peer-to-peer communication connection
    (a conn), be it TCP, SCTP, UDP, GRE, or any other protocol.
  - A RELATED packet is the first packet of a new conn that netfilter determined is
    related to an existing conn (the data conn of an FTP conn, for example).
  - When two-way communication is established with a reply packet, the conn's state
    changes to ESTABLISHED.
  - INVALID packets are those that netfilter has received but has no idea what to do
    with them; they are packets that can only belong to an ESTABLISHED conn but it can
    find no such conn in its database.
  - I think RAW packets are those that netfilter has been told not to process; but I'm
    not sure of this as I've never had reason to use RAW packets.

In netfilter, 'connection' is not related to connection-oriented protocols. It has to do with the relationship--the logical connection--between two endpoints on a LAN or on some internetwork of them. It is much like two people talking on a walkie-talkie, two people exchanging TXT MSGs, or two people talking on a phone that has a circuit-switched connection set up between them.

RE: conntrack and ICMP echo replies not showing as ESTABLISHED

[André Paulsberg-Csibi (IBM Consultant)]

I think here are some semantics confused ( could be my fault ).

My statement is meant to explain that the term "ESTABLISHED" used in CONNTRACK , does not match the state "ESTABLISHED" used in iptables rules-set .
ESTABLISHED in iptables ruleset simply means there is an entry directly related to the packet that is being examined in the connection table .

ESTABLISHED in the conntrack TERM is only used for TCP as here the packets have multiple timers as at least TCP is session/connection based protocol ,
I am not aware of other protocols who have other times and multiple sets of session/connection states in netfilter/iptables/conntrack and using this TERM in same manner .

RELATED is actually not the FIRST/NEW packet of new connection , it is any packet ( first second or later ) from any system that could be indirectly connected to any one connection table entry .
Including ICMP messages like type 11 ( TTL exceeded ) typically sent from every router on the way until you reached the end target destination when doing a traceroute .

When conntrack deals with other none TCP it uses only UNREPLIED / ASSURED and not the term ESTABLISHED
And for ICMP it does not use ASSURED , only UNREPLIED ( which is removed after the first reply )

As far as I can tell there are 6 states for the connection table part ( 7 if you include those NOT/NEVER there )
NEW
ESTABLISHED
RELATED
INVALID
UNTRACKED
CLOSED

For me these semantics is also not so self explanatory , specially since the terms somewhat overlap and are missing for others .
( but now maybe this is a bit more clear ) If not here is more details http://www.iptables.info/en/connection-state.html


Best regards
André Paulsberg-Csibi
Senior Network Engineer 
IBM Services AS

-----Original Message-----
From: netfilter-owner@vger.kernel.org [mailto:netfilter-owner@vger.kernel.org] On Behalf Of Neal P. Murphy
Sent: Wednesday, January 3, 2018 12:37 AM
Cc: netfilter@vger.kernel.org
Subject: Re: conntrack and ICMP echo replies not showing as ESTABLISHED

On Tue, 2 Jan 2018 21:56:07 +0000
André Paulsberg-Csibi (IBM Consultant)         <Andre.Paulsberg-Csibi@evry.com> wrote:

> As far as I can tell - ESTABLISHED - is only for session based protocols like TCP .
> You will not see that for UDP or ICMP , as far as CONNTRACK is concerned .  

With netfilter, there are five connection states: NEW, RELATED, ESTABLISHED, INVALID and RAW.
  - A NEW packet is the first packet of a new peer-to-peer communication connection
    (a conn), be it TCP, SCTP, UDP, GRE, or any other protocol.
  - A RELATED packet is the first packet of a new conn that netfilter determined is
    related to an existing conn (the data conn of an FTP conn, for example).
  - When two-way communication is established with a reply packet, the conn's state
    changes to ESTABLISHED.
  - INVALID packets are those that netfilter has received but has no idea what to do
    with them; they are packets that can only belong to an ESTABLISHED conn but it can
    find no such conn in its database.
  - I think RAW packets are those that netfilter has been told not to process; but I'm
    not sure of this as I've never had reason to use RAW packets.

In netfilter, 'connection' is not related to connection-oriented protocols. It has to do with the relationship--the logical connection--between two endpoints on a LAN or on some internetwork of them. It is much like two people talking on a walkie-talkie, two people exchanging TXT MSGs, or two people talking on a phone that has a circuit-switched connection set up between them.
--
To unsubscribe from this list: send the line "unsubscribe netfilter" in the body of a message to majordomo@vger.kernel.org More majordomo info at  https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fvger.kernel.org%2Fmajordomo-info.html&data=02%7C01%7C%7C8cd63c84f69240d4239308d5523e4c6d%7C40cc2915e2834a2794716bdd7ca4c6e1%7C1%7C0%7C636505349884829535&sdata=hwc8E8kipzNl9HDW3SWhZfc9w2WgFoOXHH43uV5ugNk%3D&reserved=0

Re: conntrack and ICMP echo replies not showing as ESTABLISHED

[Oliver O'Boyle]

All,

Thanks for the replies, your help is much appreciated.

Andre, the link you included in your last message is one of the ones
I've been referencing and it'sm in the fact, the one that's causing
the confusion.

a) I agree that ICMP is a connectionless protocol and we would not
normally expect to see an established two-way session with it, such as
we would see with TCP.
b) As per Neal's and Pascal's comments above, I agree that
Netfilter/Conntrack impose a connected (or unconnected) state between
two endpoints/streams which is what's seen in the output of 'sudo
contrack -E'.

But I'm still not getting the result I'm hoping to see. Maybe it's
just not possible, though. I'll re-phrase what I'm looking for to see
if there's a different avenue for answers:

Referencing the following quote from the link Andre sent: 'When the
firewall sees a request packet, it considers it as NEW. When the host
sends a reply packet to the request it is considered ESTABLISHED.'

I'd like to see which packets are considered 'ESTABLISHED'. Or, I'd
like a clear confirmation that the connection stream these ESTABLISHED
packets are part of is considered "approved/allowed" to pass. I don't
care which tool I use to see these. If the approval I'm looking for is
in the form of a missing statement (meaning that the
packet/stream/connection is considered allowed/established because
there is no statement that explicitly declares a
packet/stream/connection as NOT allowed/established) then I can work
with that too, I just need confirmation that this is the case.

Thanks!
Oliver



On Wed, Jan 3, 2018 at 5:13 AM, André Paulsberg-Csibi (IBM Consultant)
<Andre.Paulsberg-Csibi@evry.com> wrote:
> I think here are some semantics confused ( could be my fault ).
>
> My statement is meant to explain that the term "ESTABLISHED" used in CONNTRACK , does not match the state "ESTABLISHED" used in iptables rules-set .
> ESTABLISHED in iptables ruleset simply means there is an entry directly related to the packet that is being examined in the connection table .
>
> ESTABLISHED in the conntrack TERM is only used for TCP as here the packets have multiple timers as at least TCP is session/connection based protocol ,
> I am not aware of other protocols who have other times and multiple sets of session/connection states in netfilter/iptables/conntrack and using this TERM in same manner .
>
> RELATED is actually not the FIRST/NEW packet of new connection , it is any packet ( first second or later ) from any system that could be indirectly connected to any one connection table entry .
> Including ICMP messages like type 11 ( TTL exceeded ) typically sent from every router on the way until you reached the end target destination when doing a traceroute .
>
> When conntrack deals with other none TCP it uses only UNREPLIED / ASSURED and not the term ESTABLISHED
> And for ICMP it does not use ASSURED , only UNREPLIED ( which is removed after the first reply )
>
> As far as I can tell there are 6 states for the connection table part ( 7 if you include those NOT/NEVER there )
> NEW
> ESTABLISHED
> RELATED
> INVALID
> UNTRACKED
> CLOSED
>
> For me these semantics is also not so self explanatory , specially since the terms somewhat overlap and are missing for others .
> ( but now maybe this is a bit more clear ) If not here is more details http://www.iptables.info/en/connection-state.html
>
>
> Best regards
> André Paulsberg-Csibi
> Senior Network Engineer
> IBM Services AS
>
> -----Original Message-----
> From: netfilter-owner@vger.kernel.org [mailto:netfilter-owner@vger.kernel.org] On Behalf Of Neal P. Murphy
> Sent: Wednesday, January 3, 2018 12:37 AM
> Cc: netfilter@vger.kernel.org
> Subject: Re: conntrack and ICMP echo replies not showing as ESTABLISHED
>
> On Tue, 2 Jan 2018 21:56:07 +0000
> André Paulsberg-Csibi (IBM Consultant)         <Andre.Paulsberg-Csibi@evry.com> wrote:
>
>> As far as I can tell - ESTABLISHED - is only for session based protocols like TCP .
>> You will not see that for UDP or ICMP , as far as CONNTRACK is concerned .
>
> With netfilter, there are five connection states: NEW, RELATED, ESTABLISHED, INVALID and RAW.
>   - A NEW packet is the first packet of a new peer-to-peer communication connection
>     (a conn), be it TCP, SCTP, UDP, GRE, or any other protocol.
>   - A RELATED packet is the first packet of a new conn that netfilter determined is
>     related to an existing conn (the data conn of an FTP conn, for example).
>   - When two-way communication is established with a reply packet, the conn's state
>     changes to ESTABLISHED.
>   - INVALID packets are those that netfilter has received but has no idea what to do
>     with them; they are packets that can only belong to an ESTABLISHED conn but it can
>     find no such conn in its database.
>   - I think RAW packets are those that netfilter has been told not to process; but I'm
>     not sure of this as I've never had reason to use RAW packets.
>
> In netfilter, 'connection' is not related to connection-oriented protocols. It has to do with the relationship--the logical connection--between two endpoints on a LAN or on some internetwork of them. It is much like two people talking on a walkie-talkie, two people exchanging TXT MSGs, or two people talking on a phone that has a circuit-switched connection set up between them.
> --
> To unsubscribe from this list: send the line "unsubscribe netfilter" in the body of a message to majordomo@vger.kernel.org More majordomo info at  https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fvger.kernel.org%2Fmajordomo-info.html&data=02%7C01%7C%7C8cd63c84f69240d4239308d5523e4c6d%7C40cc2915e2834a2794716bdd7ca4c6e1%7C1%7C0%7C636505349884829535&sdata=hwc8E8kipzNl9HDW3SWhZfc9w2WgFoOXHH43uV5ugNk%3D&reserved=0



-- 
:o@>

Re: conntrack and ICMP echo replies not showing as ESTABLISHED

[Humberto Jucá]

Hi,

Try something like this:
iptables -t raw -I PREROUTING -p icmp -j TRACE

After this, use the "tail -f" command in your syslog or messages file:
tail -f /va

2018-01-03 11:01 GMT-03:00 Oliver O'Boyle <oliver.oboyle@gmail.com>:
> All,
>
> Thanks for the replies, your help is much appreciated.
>
> Andre, the link you included in your last message is one of the ones
> I've been referencing and it'sm in the fact, the one that's causing
> the confusion.
>
> a) I agree that ICMP is a connectionless protocol and we would not
> normally expect to see an established two-way session with it, such as
> we would see with TCP.
> b) As per Neal's and Pascal's comments above, I agree that
> Netfilter/Conntrack impose a connected (or unconnected) state between
> two endpoints/streams which is what's seen in the output of 'sudo
> contrack -E'.
>
> But I'm still not getting the result I'm hoping to see. Maybe it's
> just not possible, though. I'll re-phrase what I'm looking for to see
> if there's a different avenue for answers:
>
> Referencing the following quote from the link Andre sent: 'When the
> firewall sees a request packet, it considers it as NEW. When the host
> sends a reply packet to the request it is considered ESTABLISHED.'
>
> I'd like to see which packets are considered 'ESTABLISHED'. Or, I'd
> like a clear confirmation that the connection stream these ESTABLISHED
> packets are part of is considered "approved/allowed" to pass. I don't
> care which tool I use to see these. If the approval I'm looking for is
> in the form of a missing statement (meaning that the
> packet/stream/connection is considered allowed/established because
> there is no statement that explicitly declares a
> packet/stream/connection as NOT allowed/established) then I can work
> with that too, I just need confirmation that this is the case.
>
> Thanks!
> Oliver
>
>
>
> On Wed, Jan 3, 2018 at 5:13 AM, André Paulsberg-Csibi (IBM Consultant)
> <Andre.Paulsberg-Csibi@evry.com> wrote:
>> I think here are some semantics confused ( could be my fault ).
>>
>> My statement is meant to explain that the term "ESTABLISHED" used in CONNTRACK , does not match the state "ESTABLISHED" used in iptables rules-set .
>> ESTABLISHED in iptables ruleset simply means there is an entry directly related to the packet that is being examined in the connection table .
>>
>> ESTABLISHED in the conntrack TERM is only used for TCP as here the packets have multiple timers as at least TCP is session/connection based protocol ,
>> I am not aware of other protocols who have other times and multiple sets of session/connection states in netfilter/iptables/conntrack and using this TERM in same manner .
>>
>> RELATED is actually not the FIRST/NEW packet of new connection , it is any packet ( first second or later ) from any system that could be indirectly connected to any one connection table entry .
>> Including ICMP messages like type 11 ( TTL exceeded ) typically sent from every router on the way until you reached the end target destination when doing a traceroute .
>>
>> When conntrack deals with other none TCP it uses only UNREPLIED / ASSURED and not the term ESTABLISHED
>> And for ICMP it does not use ASSURED , only UNREPLIED ( which is removed after the first reply )
>>
>> As far as I can tell there are 6 states for the connection table part ( 7 if you include those NOT/NEVER there )
>> NEW
>> ESTABLISHED
>> RELATED
>> INVALID
>> UNTRACKED
>> CLOSED
>>
>> For me these semantics is also not so self explanatory , specially since the terms somewhat overlap and are missing for others .
>> ( but now maybe this is a bit more clear ) If not here is more details http://www.iptables.info/en/connection-state.html
>>
>>
>> Best regards
>> André Paulsberg-Csibi
>> Senior Network Engineer
>> IBM Services AS
>>
>> -----Original Message-----
>> From: netfilter-owner@vger.kernel.org [mailto:netfilter-owner@vger.kernel.org] On Behalf Of Neal P. Murphy
>> Sent: Wednesday, January 3, 2018 12:37 AM
>> Cc: netfilter@vger.kernel.org
>> Subject: Re: conntrack and ICMP echo replies not showing as ESTABLISHED
>>
>> On Tue, 2 Jan 2018 21:56:07 +0000
>> André Paulsberg-Csibi (IBM Consultant)         <Andre.Paulsberg-Csibi@evry.com> wrote:
>>
>>> As far as I can tell - ESTABLISHED - is only for session based protocols like TCP .
>>> You will not see that for UDP or ICMP , as far as CONNTRACK is concerned .
>>
>> With netfilter, there are five connection states: NEW, RELATED, ESTABLISHED, INVALID and RAW.
>>   - A NEW packet is the first packet of a new peer-to-peer communication connection
>>     (a conn), be it TCP, SCTP, UDP, GRE, or any other protocol.
>>   - A RELATED packet is the first packet of a new conn that netfilter determined is
>>     related to an existing conn (the data conn of an FTP conn, for example).
>>   - When two-way communication is established with a reply packet, the conn's state
>>     changes to ESTABLISHED.
>>   - INVALID packets are those that netfilter has received but has no idea what to do
>>     with them; they are packets that can only belong to an ESTABLISHED conn but it can
>>     find no such conn in its database.
>>   - I think RAW packets are those that netfilter has been told not to process; but I'm
>>     not sure of this as I've never had reason to use RAW packets.
>>
>> In netfilter, 'connection' is not related to connection-oriented protocols. It has to do with the relationship--the logical connection--between two endpoints on a LAN or on some internetwork of them. It is much like two people talking on a walkie-talkie, two people exchanging TXT MSGs, or two people talking on a phone that has a circuit-switched connection set up between them.
>> --
>> To unsubscribe from this list: send the line "unsubscribe netfilter" in the body of a message to majordomo@vger.kernel.org More majordomo info at  https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fvger.kernel.org%2Fmajordomo-info.html&data=02%7C01%7C%7C8cd63c84f69240d4239308d5523e4c6d%7C40cc2915e2834a2794716bdd7ca4c6e1%7C1%7C0%7C636505349884829535&sdata=hwc8E8kipzNl9HDW3SWhZfc9w2WgFoOXHH43uV5ugNk%3D&reserved=0
>
>
>
> --
> :o@>
> --
> To unsubscribe from this list: send the line "unsubscribe netfilter" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: conntrack and ICMP echo replies not showing as ESTABLISHED

[Oliver O'Boyle]

Thanks, Humberto. That certainly allows me to see what policy and rule
is being applied. Thanks!

If anyone is aware of a book or blog or something that explains the
state stuff discussed above without ANY ambiguity at all, I'd love to
read it!

Oliver

On Wed, Jan 3, 2018 at 9:34 AM, Humberto Jucá <betolj@gmail.com> wrote:
> Hi,
>
> Try something like this:
> iptables -t raw -I PREROUTING -p icmp -j TRACE
>
> After this, use the "tail -f" command in your syslog or messages file:
> tail -f /va
>
> 2018-01-03 11:01 GMT-03:00 Oliver O'Boyle <oliver.oboyle@gmail.com>:
>> All,
>>
>> Thanks for the replies, your help is much appreciated.
>>
>> Andre, the link you included in your last message is one of the ones
>> I've been referencing and it'sm in the fact, the one that's causing
>> the confusion.
>>
>> a) I agree that ICMP is a connectionless protocol and we would not
>> normally expect to see an established two-way session with it, such as
>> we would see with TCP.
>> b) As per Neal's and Pascal's comments above, I agree that
>> Netfilter/Conntrack impose a connected (or unconnected) state between
>> two endpoints/streams which is what's seen in the output of 'sudo
>> contrack -E'.
>>
>> But I'm still not getting the result I'm hoping to see. Maybe it's
>> just not possible, though. I'll re-phrase what I'm looking for to see
>> if there's a different avenue for answers:
>>
>> Referencing the following quote from the link Andre sent: 'When the
>> firewall sees a request packet, it considers it as NEW. When the host
>> sends a reply packet to the request it is considered ESTABLISHED.'
>>
>> I'd like to see which packets are considered 'ESTABLISHED'. Or, I'd
>> like a clear confirmation that the connection stream these ESTABLISHED
>> packets are part of is considered "approved/allowed" to pass. I don't
>> care which tool I use to see these. If the approval I'm looking for is
>> in the form of a missing statement (meaning that the
>> packet/stream/connection is considered allowed/established because
>> there is no statement that explicitly declares a
>> packet/stream/connection as NOT allowed/established) then I can work
>> with that too, I just need confirmation that this is the case.
>>
>> Thanks!
>> Oliver
>>
>>
>>
>> On Wed, Jan 3, 2018 at 5:13 AM, André Paulsberg-Csibi (IBM Consultant)
>> <Andre.Paulsberg-Csibi@evry.com> wrote:
>>> I think here are some semantics confused ( could be my fault ).
>>>
>>> My statement is meant to explain that the term "ESTABLISHED" used in CONNTRACK , does not match the state "ESTABLISHED" used in iptables rules-set .
>>> ESTABLISHED in iptables ruleset simply means there is an entry directly related to the packet that is being examined in the connection table .
>>>
>>> ESTABLISHED in the conntrack TERM is only used for TCP as here the packets have multiple timers as at least TCP is session/connection based protocol ,
>>> I am not aware of other protocols who have other times and multiple sets of session/connection states in netfilter/iptables/conntrack and using this TERM in same manner .
>>>
>>> RELATED is actually not the FIRST/NEW packet of new connection , it is any packet ( first second or later ) from any system that could be indirectly connected to any one connection table entry .
>>> Including ICMP messages like type 11 ( TTL exceeded ) typically sent from every router on the way until you reached the end target destination when doing a traceroute .
>>>
>>> When conntrack deals with other none TCP it uses only UNREPLIED / ASSURED and not the term ESTABLISHED
>>> And for ICMP it does not use ASSURED , only UNREPLIED ( which is removed after the first reply )
>>>
>>> As far as I can tell there are 6 states for the connection table part ( 7 if you include those NOT/NEVER there )
>>> NEW
>>> ESTABLISHED
>>> RELATED
>>> INVALID
>>> UNTRACKED
>>> CLOSED
>>>
>>> For me these semantics is also not so self explanatory , specially since the terms somewhat overlap and are missing for others .
>>> ( but now maybe this is a bit more clear ) If not here is more details http://www.iptables.info/en/connection-state.html
>>>
>>>
>>> Best regards
>>> André Paulsberg-Csibi
>>> Senior Network Engineer
>>> IBM Services AS
>>>
>>> -----Original Message-----
>>> From: netfilter-owner@vger.kernel.org [mailto:netfilter-owner@vger.kernel.org] On Behalf Of Neal P. Murphy
>>> Sent: Wednesday, January 3, 2018 12:37 AM
>>> Cc: netfilter@vger.kernel.org
>>> Subject: Re: conntrack and ICMP echo replies not showing as ESTABLISHED
>>>
>>> On Tue, 2 Jan 2018 21:56:07 +0000
>>> André Paulsberg-Csibi (IBM Consultant)         <Andre.Paulsberg-Csibi@evry.com> wrote:
>>>
>>>> As far as I can tell - ESTABLISHED - is only for session based protocols like TCP .
>>>> You will not see that for UDP or ICMP , as far as CONNTRACK is concerned .
>>>
>>> With netfilter, there are five connection states: NEW, RELATED, ESTABLISHED, INVALID and RAW.
>>>   - A NEW packet is the first packet of a new peer-to-peer communication connection
>>>     (a conn), be it TCP, SCTP, UDP, GRE, or any other protocol.
>>>   - A RELATED packet is the first packet of a new conn that netfilter determined is
>>>     related to an existing conn (the data conn of an FTP conn, for example).
>>>   - When two-way communication is established with a reply packet, the conn's state
>>>     changes to ESTABLISHED.
>>>   - INVALID packets are those that netfilter has received but has no idea what to do
>>>     with them; they are packets that can only belong to an ESTABLISHED conn but it can
>>>     find no such conn in its database.
>>>   - I think RAW packets are those that netfilter has been told not to process; but I'm
>>>     not sure of this as I've never had reason to use RAW packets.
>>>
>>> In netfilter, 'connection' is not related to connection-oriented protocols. It has to do with the relationship--the logical connection--between two endpoints on a LAN or on some internetwork of them. It is much like two people talking on a walkie-talkie, two people exchanging TXT MSGs, or two people talking on a phone that has a circuit-switched connection set up between them.
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe netfilter" in the body of a message to majordomo@vger.kernel.org More majordomo info at  https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fvger.kernel.org%2Fmajordomo-info.html&data=02%7C01%7C%7C8cd63c84f69240d4239308d5523e4c6d%7C40cc2915e2834a2794716bdd7ca4c6e1%7C1%7C0%7C636505349884829535&sdata=hwc8E8kipzNl9HDW3SWhZfc9w2WgFoOXHH43uV5ugNk%3D&reserved=0
>>
>>
>>
>> --
>> :o@>
>> --
>> To unsubscribe from this list: send the line "unsubscribe netfilter" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
:o@>

Re: conntrack and ICMP echo replies not showing as ESTABLISHED

[Oliver O'Boyle]

This article helped clear up some questions/misconceptions I was having:

https://i-admin.blogspot.ca/2014/02/caveats-about-linux-connection-tracking.html

I now better understand that connection tracking is enabled by default
and that the matches for NEW and ESTABLISHED etc... use the connection
tracking information already being collected and updated in the
PREROUTING and OUTPUT chains.

The level of detail visible in the sudo conntrack -L and -E commands
make sense now and are directly related to the layer 4 protocols in
use (TCP/UDP/ICMP etc...), hence only TCP sessions will show as
established sessions because it's the only connection-oriented
protocol of the three listed.

In a nutshell, "state" is determined via the data collected by and
rules determined by connection tracking. Connection tracking is mostly
just a set of timeouts, thresholds, and verifications that help us
determine if a packet is "likely" or, ideally "mostly guaranteed" to
be part of a known/expected/established layer 4 session. These
timeouts, thresholds, and verifications can be seen by catting the
various files in /proc/sys/net/netfilter (on Ubuntu 16.04LTS). The
conntrack module applies logic to these values which is what
determines the state of a layer 4 session based on the relationship of
packets to one-another, along with the data contained in their fields
and to source and destination endpoints.

Using --ctstate matches in rules simply allows us to tap into the
connection tracking data to make rules more specific and therefore
more secure.

Thanks for the help guys, much appreciated.

Oliver

On Wed, Jan 3, 2018 at 10:15 AM, Oliver O'Boyle <oliver.oboyle@gmail.com> wrote:
> Thanks, Humberto. That certainly allows me to see what policy and rule
> is being applied. Thanks!
>
> If anyone is aware of a book or blog or something that explains the
> state stuff discussed above without ANY ambiguity at all, I'd love to
> read it!
>
> Oliver
>
> On Wed, Jan 3, 2018 at 9:34 AM, Humberto Jucá <betolj@gmail.com> wrote:
>> Hi,
>>
>> Try something like this:
>> iptables -t raw -I PREROUTING -p icmp -j TRACE
>>
>> After this, use the "tail -f" command in your syslog or messages file:
>> tail -f /va
>>
>> 2018-01-03 11:01 GMT-03:00 Oliver O'Boyle <oliver.oboyle@gmail.com>:
>>> All,
>>>
>>> Thanks for the replies, your help is much appreciated.
>>>
>>> Andre, the link you included in your last message is one of the ones
>>> I've been referencing and it'sm in the fact, the one that's causing
>>> the confusion.
>>>
>>> a) I agree that ICMP is a connectionless protocol and we would not
>>> normally expect to see an established two-way session with it, such as
>>> we would see with TCP.
>>> b) As per Neal's and Pascal's comments above, I agree that
>>> Netfilter/Conntrack impose a connected (or unconnected) state between
>>> two endpoints/streams which is what's seen in the output of 'sudo
>>> contrack -E'.
>>>
>>> But I'm still not getting the result I'm hoping to see. Maybe it's
>>> just not possible, though. I'll re-phrase what I'm looking for to see
>>> if there's a different avenue for answers:
>>>
>>> Referencing the following quote from the link Andre sent: 'When the
>>> firewall sees a request packet, it considers it as NEW. When the host
>>> sends a reply packet to the request it is considered ESTABLISHED.'
>>>
>>> I'd like to see which packets are considered 'ESTABLISHED'. Or, I'd
>>> like a clear confirmation that the connection stream these ESTABLISHED
>>> packets are part of is considered "approved/allowed" to pass. I don't
>>> care which tool I use to see these. If the approval I'm looking for is
>>> in the form of a missing statement (meaning that the
>>> packet/stream/connection is considered allowed/established because
>>> there is no statement that explicitly declares a
>>> packet/stream/connection as NOT allowed/established) then I can work
>>> with that too, I just need confirmation that this is the case.
>>>
>>> Thanks!
>>> Oliver
>>>
>>>
>>>
>>> On Wed, Jan 3, 2018 at 5:13 AM, André Paulsberg-Csibi (IBM Consultant)
>>> <Andre.Paulsberg-Csibi@evry.com> wrote:
>>>> I think here are some semantics confused ( could be my fault ).
>>>>
>>>> My statement is meant to explain that the term "ESTABLISHED" used in CONNTRACK , does not match the state "ESTABLISHED" used in iptables rules-set .
>>>> ESTABLISHED in iptables ruleset simply means there is an entry directly related to the packet that is being examined in the connection table .
>>>>
>>>> ESTABLISHED in the conntrack TERM is only used for TCP as here the packets have multiple timers as at least TCP is session/connection based protocol ,
>>>> I am not aware of other protocols who have other times and multiple sets of session/connection states in netfilter/iptables/conntrack and using this TERM in same manner .
>>>>
>>>> RELATED is actually not the FIRST/NEW packet of new connection , it is any packet ( first second or later ) from any system that could be indirectly connected to any one connection table entry .
>>>> Including ICMP messages like type 11 ( TTL exceeded ) typically sent from every router on the way until you reached the end target destination when doing a traceroute .
>>>>
>>>> When conntrack deals with other none TCP it uses only UNREPLIED / ASSURED and not the term ESTABLISHED
>>>> And for ICMP it does not use ASSURED , only UNREPLIED ( which is removed after the first reply )
>>>>
>>>> As far as I can tell there are 6 states for the connection table part ( 7 if you include those NOT/NEVER there )
>>>> NEW
>>>> ESTABLISHED
>>>> RELATED
>>>> INVALID
>>>> UNTRACKED
>>>> CLOSED
>>>>
>>>> For me these semantics is also not so self explanatory , specially since the terms somewhat overlap and are missing for others .
>>>> ( but now maybe this is a bit more clear ) If not here is more details http://www.iptables.info/en/connection-state.html
>>>>
>>>>
>>>> Best regards
>>>> André Paulsberg-Csibi
>>>> Senior Network Engineer
>>>> IBM Services AS
>>>>
>>>> -----Original Message-----
>>>> From: netfilter-owner@vger.kernel.org [mailto:netfilter-owner@vger.kernel.org] On Behalf Of Neal P. Murphy
>>>> Sent: Wednesday, January 3, 2018 12:37 AM
>>>> Cc: netfilter@vger.kernel.org
>>>> Subject: Re: conntrack and ICMP echo replies not showing as ESTABLISHED
>>>>
>>>> On Tue, 2 Jan 2018 21:56:07 +0000
>>>> André Paulsberg-Csibi (IBM Consultant)         <Andre.Paulsberg-Csibi@evry.com> wrote:
>>>>
>>>>> As far as I can tell - ESTABLISHED - is only for session based protocols like TCP .
>>>>> You will not see that for UDP or ICMP , as far as CONNTRACK is concerned .
>>>>
>>>> With netfilter, there are five connection states: NEW, RELATED, ESTABLISHED, INVALID and RAW.
>>>>   - A NEW packet is the first packet of a new peer-to-peer communication connection
>>>>     (a conn), be it TCP, SCTP, UDP, GRE, or any other protocol.
>>>>   - A RELATED packet is the first packet of a new conn that netfilter determined is
>>>>     related to an existing conn (the data conn of an FTP conn, for example).
>>>>   - When two-way communication is established with a reply packet, the conn's state
>>>>     changes to ESTABLISHED.
>>>>   - INVALID packets are those that netfilter has received but has no idea what to do
>>>>     with them; they are packets that can only belong to an ESTABLISHED conn but it can
>>>>     find no such conn in its database.
>>>>   - I think RAW packets are those that netfilter has been told not to process; but I'm
>>>>     not sure of this as I've never had reason to use RAW packets.
>>>>
>>>> In netfilter, 'connection' is not related to connection-oriented protocols. It has to do with the relationship--the logical connection--between two endpoints on a LAN or on some internetwork of them. It is much like two people talking on a walkie-talkie, two people exchanging TXT MSGs, or two people talking on a phone that has a circuit-switched connection set up between them.
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe netfilter" in the body of a message to majordomo@vger.kernel.org More majordomo info at  https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fvger.kernel.org%2Fmajordomo-info.html&data=02%7C01%7C%7C8cd63c84f69240d4239308d5523e4c6d%7C40cc2915e2834a2794716bdd7ca4c6e1%7C1%7C0%7C636505349884829535&sdata=hwc8E8kipzNl9HDW3SWhZfc9w2WgFoOXHH43uV5ugNk%3D&reserved=0
>>>
>>>
>>>
>>> --
>>> :o@>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe netfilter" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>
>
> --
> :o@>



-- 
:o@>

Re: conntrack and ICMP echo replies not showing as ESTABLISHED

[Pascal Hambourg]

Le 02/01/2018 à 22:56, André Paulsberg-Csibi (IBM Consultant) a écrit :
> As far as I can tell - ESTABLISHED - is only for session based protocols like TCP .
> You will not see that for UDP or ICMP , as far as CONNTRACK is concerned .
> 
> IN the ruleset it will fall under the STATE of ESTABLISHED , as IPTABLES understands that this is direct return traffic .
> However , like UDP it has no real "IP state" outside of IPTABLES and my guess this is why it not refered to in that state in CONNTRACK .

You could not be more wrong. Connection tracking handles state for UDP 
and ICMP. Iptables just checks the packet states assigned by conntrack.

> -----Original Message-----
> 
> I'm sure I'm missing something obvious but I am not seeing echo replies showing as ESTABLISHED when running:
> 
> sudo conntrack -E

Do not confuse packet state and connection state.
conntrack -E shows connection tracking events, not packet states.