No sign of INVALID packet , LOGS DROP but not reason

No sign of INVALID packet , LOGS DROP but not reason

[André Paulsberg-Csibi (IBM Consultant)]

Hi ,

 
I have come across something that I am starting to think is a bug ,

but before I start upgrading and other works lets see if I missed something !


I have log entries like these

May 28 10:47:13 zotac kernel: INVALID-STATE IN=vlan0 OUT= MAC=# SRC=189.222.120.167 DST=# LEN=40 TOS=0x00 PREC=0x00 TTL=116 ID=5745 PROTO=TCP SPT=21735 DPT=56715 WINDOW=0 RES=0x00 ACK RST URGP=0


I have used

conntrack -E -o timestamp

and added logging with

echo 255 > /proc/sys/net/netfilter/nf_conntrack_log_invalid


 from what I can see there is no "kernel: nf_ct_tcp: " entries at the moment of the DROP of ACK RST

and there is an entry in conntrack for this session that should allow ACK RST to terminate that session .


when I do :

zotac:~ # journalctl | grep nf_ct | grep " ACK RST " | grep -v " ACK RST FIN "
May 26 22:35:31 zotac kernel: nf_ct_tcp: invalid RST IN= OUT= SRC=# DST=81.233.185.232 LEN=40 TOS=0x00 PREC=0x00 TTL=127 ID=14841 PROTO=TCP SPT=7905 DPT=56206 SEQ=2244837322 ACK=835716258 WINDOW=0 RES=0x00 ACK RST URGP=0

I only find ONE result , but when I do :

zotac:~ # journalctl | grep INVALID | grep " ACK RST " | grep -v " ACK RST FIN " | grep "May 2[678]" | wc
   1590   38480  412611


I should have atleast 1000 + more nf_ct log entries to match all my INVALID ACK RST log entries .


I have tried to spot some issues with TCPDUMPs , but all packets seems like normal ACK RST when I try to get same result "manually" by sending SYN packets "I just used "telnet IP PORT" to a port I found in my log ...

I see the ACK RST telling me the port is blocked and I can't seem to find any issues with the packet !



Best regards
André Paulsberg-Csibi
Senior Network Engineer
Fault Handling
IBM Services AS
andre.paulsberg-csibi@evry.com

 
   

Re: No sign of INVALID packet , LOGS DROP but not reason

[Noel Kuntze]

[-- Attachment #1.1: Type: text/plain, Size: 2267 bytes --]

On 29.05.2016 12:42, André Paulsberg-Csibi (IBM Consultant) wrote:
> Hi ,
>
>  
> I have come across something that I am starting to think is a bug ,
>
> but before I start upgrading and other works lets see if I missed something !
>
>
> I have log entries like these
>
> May 28 10:47:13 zotac kernel: INVALID-STATE IN=vlan0 OUT= MAC=# SRC=189.222.120.167 DST=# LEN=40 TOS=0x00 PREC=0x00 TTL=116 ID=5745 PROTO=TCP SPT=21735 DPT=56715 WINDOW=0 RES=0x00 ACK RST URGP=0
>
>
> I have used
>
> conntrack -E -o timestamp
>
> and added logging with
>
> echo 255 > /proc/sys/net/netfilter/nf_conntrack_log_invalid
>
>
>  from what I can see there is no "kernel: nf_ct_tcp: " entries at the moment of the DROP of ACK RST
>
> and there is an entry in conntrack for this session that should allow ACK RST to terminate that session .
>
>
> when I do :
>
> zotac:~ # journalctl | grep nf_ct | grep " ACK RST " | grep -v " ACK RST FIN "
> May 26 22:35:31 zotac kernel: nf_ct_tcp: invalid RST IN= OUT= SRC=# DST=81.233.185.232 LEN=40 TOS=0x00 PREC=0x00 TTL=127 ID=14841 PROTO=TCP SPT=7905 DPT=56206 SEQ=2244837322 ACK=835716258 WINDOW=0 RES=0x00 ACK RST URGP=0
>
> I only find ONE result , but when I do :
>
> zotac:~ # journalctl | grep INVALID | grep " ACK RST " | grep -v " ACK RST FIN " | grep "May 2[678]" | wc
>    1590   38480  412611
>
>
> I should have atleast 1000 + more nf_ct log entries to match all my INVALID ACK RST log entries .
>
>
> I have tried to spot some issues with TCPDUMPs , but all packets seems like normal ACK RST when I try to get same result "manually" by sending SYN packets "I just used "telnet IP PORT" to a port I found in my log ...
>
> I see the ACK RST telling me the port is blocked and I can't seem to find any issues with the packet !
>
>
>
> Best regards
> André Paulsberg-Csibi
> Senior Network Engineer
> Fault Handling
> IBM Services AS
> andre.paulsberg-csibi@evry.com
Hello André,

The TCP flag combination "ACK RST URGP=0" is invalid, I think. The RFC (RFC 6093) about the urgent pointer
is pretty clear about that.

-- 

Mit freundlichen Grüßen/Kind Regards,
Noel Kuntze

GPG Key ID: 0x63EC6658
Fingerprint: 23CA BB60 2146 05E7 7278 6592 3839 298F 63EC 6658



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

RE: No sign of INVALID packet , LOGS DROP but not reason

[André Paulsberg-Csibi (IBM Consultant)]

Thank you for that answer , I will try to verify/check out if that is the reason .
It seems you may be correct , as it makes no sense to set any URG flag with zero data .
( I may have just "ignored" it mentally as the URGP is set to ZERO , and I may have assumed that is the same as no flag )

Either way I think that nf_ct should have made a log entry for this , which it doesn't 
( and even if I missed something ) that does seem like "something" that should be fixed :)


Best regards
André Paulsberg-Csibi
Senior Network Engineer 
Fault Handling
IBM Services AS
andre.paulsberg-csibi@evry.com
M +47 9070 5988




-----Original Message-----
From: netfilter-owner@vger.kernel.org [mailto:netfilter-owner@vger.kernel.org] On Behalf Of Noel Kuntze
Sent: 29. mai 2016 19:52
To: André Paulsberg-Csibi (IBM Consultant) <Andre.Paulsberg-Csibi@evry.com>; netfilter@vger.kernel.org
Subject: Re: No sign of INVALID packet , LOGS DROP but not reason

On 29.05.2016 12:42, André Paulsberg-Csibi (IBM Consultant) wrote:
> Hi ,
>
>  
> I have come across something that I am starting to think is a bug ,
>
> but before I start upgrading and other works lets see if I missed something !
>
>
> I have log entries like these
>
> May 28 10:47:13 zotac kernel: INVALID-STATE IN=vlan0 OUT= MAC=# SRC=189.222.120.167 DST=# LEN=40 TOS=0x00 PREC=0x00 TTL=116 ID=5745 PROTO=TCP SPT=21735 DPT=56715 WINDOW=0 RES=0x00 ACK RST URGP=0
>
>
> I have used
>
> conntrack -E -o timestamp
>
> and added logging with
>
> echo 255 > /proc/sys/net/netfilter/nf_conntrack_log_invalid
>
>
>  from what I can see there is no "kernel: nf_ct_tcp: " entries at the moment of the DROP of ACK RST
>
> and there is an entry in conntrack for this session that should allow ACK RST to terminate that session .
>
>
> when I do :
>
> zotac:~ # journalctl | grep nf_ct | grep " ACK RST " | grep -v " ACK RST FIN "
> May 26 22:35:31 zotac kernel: nf_ct_tcp: invalid RST IN= OUT= SRC=# DST=81.233.185.232 LEN=40 TOS=0x00 PREC=0x00 TTL=127 ID=14841 PROTO=TCP SPT=7905 DPT=56206 SEQ=2244837322 ACK=835716258 WINDOW=0 RES=0x00 ACK RST URGP=0
>
> I only find ONE result , but when I do :
>
> zotac:~ # journalctl | grep INVALID | grep " ACK RST " | grep -v " ACK RST FIN " | grep "May 2[678]" | wc
>    1590   38480  412611
>
>
> I should have atleast 1000 + more nf_ct log entries to match all my INVALID ACK RST log entries .
>
>
> I have tried to spot some issues with TCPDUMPs , but all packets seems like normal ACK RST when I try to get same result "manually" by sending SYN packets "I just used "telnet IP PORT" to a port I found in my log ...
>
> I see the ACK RST telling me the port is blocked and I can't seem to find any issues with the packet !
>
>
>
> Best regards
> André Paulsberg-Csibi
> Senior Network Engineer
> Fault Handling
> IBM Services AS
> andre.paulsberg-csibi@evry.com
Hello André,

The TCP flag combination "ACK RST URGP=0" is invalid, I think. The RFC (RFC 6093) about the urgent pointer
is pretty clear about that.

-- 

Mit freundlichen Grüßen/Kind Regards,
Noel Kuntze

GPG Key ID: 0x63EC6658
Fingerprint: 23CA BB60 2146 05E7 7278 6592 3839 298F 63EC 6658