* Re: Possible conntrack/kernel bug - not catching certain ICMP packets [not found] ` <4E241AB0.5060603@wildgooses.com> @ 2011-07-18 12:16 ` Jan Engelhardt 2011-07-20 21:45 ` Ed W 0 siblings, 1 reply; 7+ messages in thread From: Jan Engelhardt @ 2011-07-18 12:16 UTC (permalink / raw) To: Ed W; +Cc: Netfilter, Netfilter Developer Mailing List On Monday 2011-07-18 13:36, Ed W wrote: >Hi, any chance I could get some of the netfilter gurus to have a look at >this? Sent it to nf-dev... >Essentially we have a reproducible situation where conntrack is >(apparently) relating a packet to a connection, but not incrementing the >connection counters? > >Thanks for any help figuring out a fix? > >Ed W > > >On 15/07/2011 12:45, Ed W wrote: >> Hi, This is related to a previous thread, but more complete problem >> statement below: >> >> I notice that I can get an ICMP packet to bypass parts of conntrack >> under the following conditions >> >> - Send a UDP packet that triggers some kind of UDP reply >> - Close the listening UDP socket before that reply arrives >> - Kernel generates an ICMP unreachable response which does not appear to >> be tracked (as expected) by conntrack >> >> Tested with kernel 2.6.38.4 + iptables 1.4.11.1 >> >> Reproduce this easily like so: >> >> nslookup www.yahoo.co.uk 8.8.8.8 & sleep 0.001 && killall nslookup >> >> (where the sleep obviously needs to be smaller than your DNS RTT >> lookup time. Obviously substitute nslookup/dig as appropriate...) >> >> My results >> >> # conntrack -E >> >> [NEW] udp 17 30 src=10.141.86.7 dst=8.8.8.8 sport=60721 >> dport=53 [UNREPLIED] src=8.8.8.8 dst=10.141.86.7 sport=53 dport=60721 >> [UPDATE] udp 17 29 src=10.141.86.7 dst=8.8.8.8 sport=60721 >> dport=53 src=8.8.8.8 dst=10.141.86.7 sport=53 dport=60721 >> [DESTROY] udp 17 src=10.141.86.7 dst=8.8.8.8 sport=60721 dport=53 >> packets=1 bytes=66 src=8.8.8.8 dst=10.141.86.7 sport=53 dport=60721 >> packets=1 bytes=110 >> >> # tcpdump >> >> 11:26:35.072564 IP 10.141.86.7.60721 > 8.8.8.8.domain: 2+ PTR? >> 8.8.8.8.in-addr.arpa. (38) >> 11:26:35.351804 IP 8.8.8.8.domain > 10.141.86.7.60721: 2 1/0/0 PTR >> google-public-dns-a.google.com. (82) >> 11:26:35.352110 IP 10.141.86.7 > 8.8.8.8: ICMP 10.141.86.7 udp port >> 60721 unreachable, length 118 >> >> >> # iptables -A OUTPUT -j LOGMARK >> # iptables -A OUTPUT -j LOG >> >> Jul 15 11:26:35 localhost kern.warn kernel: [ 6676.964396] iif=0 >> hook=OUTPUT nfmark=0x0 secmark=0x0 classify=0x0 ctdir=ORIGINAL >> ct=0xcf3d5060 ctmark=0x0 ctstate=NEW ctstatus= lifetime=6346s >> Jul 15 11:26:35 localhost kern.warn kernel: [ 6676.964396] IN= OUT=ppp1 >> SRC=10.141.86.7 DST=8.8.8.8 LEN=66 TOS=0x00 PREC=0x00 TTL=64 ID=19971 DF >> PROTO=UDP SPT=60721 DPT=53 LEN=46 >> Jul 15 11:26:35 localhost kern.warn kernel: [ 6677.249312] iif=0 >> hook=OUTPUT nfmark=0x0 secmark=0x0 classify=0x0 ctdir=ORIGINAL >> ct=0xcf3d5060 ctmark=0x0 ctstate=RELATED ctstatus=SEEN_REPLY,CONFIRMED >> lifetime=4294937s >> Jul 15 11:26:35 localhost kern.warn kernel: [ 6677.249426] IN= OUT=ppp1 >> SRC=10.141.86.7 DST=8.8.8.8 LEN=138 TOS=0x00 PREC=0xC0 TTL=64 ID=18412 >> PROTO=ICMP TYPE=3 CODE=3 [SRC=8.8.8.8 DST=10.141.86.7 LEN=110 TOS=0x00 >> PREC=0x00 TTL=46 ID=3897 PROTO=UDP SPT=53 DPT=60721 LEN=90 >> >> >> Notice that logmark seems to show that the ctstatus on the ICMP packet >> is SEEN_REPLY, but conntrack -E shows only packets=1? tcpdump shows >> that the ICMP packet did indeed go out >> >> Could someone with more knowledge of conntrack please investigate further? >> >> Thanks >> >> Ed W >> -- >> To unsubscribe from this list: send the line "unsubscribe netfilter" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html > >-- >To unsubscribe from this list: send the line "unsubscribe netfilter" in >the body of a message to majordomo@vger.kernel.org >More majordomo info at http://vger.kernel.org/majordomo-info.html > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Possible conntrack/kernel bug - not catching certain ICMP packets 2011-07-18 12:16 ` Possible conntrack/kernel bug - not catching certain ICMP packets Jan Engelhardt @ 2011-07-20 21:45 ` Ed W 2011-07-21 6:16 ` Patrick McHardy 0 siblings, 1 reply; 7+ messages in thread From: Ed W @ 2011-07-20 21:45 UTC (permalink / raw) To: Jan Engelhardt; +Cc: Netfilter Developer Mailing List Please, could someone take a look at this possible conntrack issue? Even just acknowledging that it's reproducible on a selection of recent kernels or architectures would help? Thanks Ed W >> Essentially we have a reproducible situation where conntrack is >> (apparently) relating a packet to a connection, but not incrementing the >> connection counters? >> >> Thanks for any help figuring out a fix? >> >> Ed W >> >> >> On 15/07/2011 12:45, Ed W wrote: >>> Hi, This is related to a previous thread, but more complete problem >>> statement below: >>> >>> I notice that I can get an ICMP packet to bypass parts of conntrack >>> under the following conditions >>> >>> - Send a UDP packet that triggers some kind of UDP reply >>> - Close the listening UDP socket before that reply arrives >>> - Kernel generates an ICMP unreachable response which does not appear to >>> be tracked (as expected) by conntrack >>> >>> Tested with kernel 2.6.38.4 + iptables 1.4.11.1 >>> >>> Reproduce this easily like so: >>> >>> nslookup www.yahoo.co.uk 8.8.8.8 & sleep 0.001 && killall nslookup >>> >>> (where the sleep obviously needs to be smaller than your DNS RTT >>> lookup time. Obviously substitute nslookup/dig as appropriate...) >>> >>> My results >>> >>> # conntrack -E >>> >>> [NEW] udp 17 30 src=10.141.86.7 dst=8.8.8.8 sport=60721 >>> dport=53 [UNREPLIED] src=8.8.8.8 dst=10.141.86.7 sport=53 dport=60721 >>> [UPDATE] udp 17 29 src=10.141.86.7 dst=8.8.8.8 sport=60721 >>> dport=53 src=8.8.8.8 dst=10.141.86.7 sport=53 dport=60721 >>> [DESTROY] udp 17 src=10.141.86.7 dst=8.8.8.8 sport=60721 dport=53 >>> packets=1 bytes=66 src=8.8.8.8 dst=10.141.86.7 sport=53 dport=60721 >>> packets=1 bytes=110 >>> >>> # tcpdump >>> >>> 11:26:35.072564 IP 10.141.86.7.60721 > 8.8.8.8.domain: 2+ PTR? >>> 8.8.8.8.in-addr.arpa. (38) >>> 11:26:35.351804 IP 8.8.8.8.domain > 10.141.86.7.60721: 2 1/0/0 PTR >>> google-public-dns-a.google.com. (82) >>> 11:26:35.352110 IP 10.141.86.7 > 8.8.8.8: ICMP 10.141.86.7 udp port >>> 60721 unreachable, length 118 >>> >>> >>> # iptables -A OUTPUT -j LOGMARK >>> # iptables -A OUTPUT -j LOG >>> >>> Jul 15 11:26:35 localhost kern.warn kernel: [ 6676.964396] iif=0 >>> hook=OUTPUT nfmark=0x0 secmark=0x0 classify=0x0 ctdir=ORIGINAL >>> ct=0xcf3d5060 ctmark=0x0 ctstate=NEW ctstatus= lifetime=6346s >>> Jul 15 11:26:35 localhost kern.warn kernel: [ 6676.964396] IN= OUT=ppp1 >>> SRC=10.141.86.7 DST=8.8.8.8 LEN=66 TOS=0x00 PREC=0x00 TTL=64 ID=19971 DF >>> PROTO=UDP SPT=60721 DPT=53 LEN=46 >>> Jul 15 11:26:35 localhost kern.warn kernel: [ 6677.249312] iif=0 >>> hook=OUTPUT nfmark=0x0 secmark=0x0 classify=0x0 ctdir=ORIGINAL >>> ct=0xcf3d5060 ctmark=0x0 ctstate=RELATED ctstatus=SEEN_REPLY,CONFIRMED >>> lifetime=4294937s >>> Jul 15 11:26:35 localhost kern.warn kernel: [ 6677.249426] IN= OUT=ppp1 >>> SRC=10.141.86.7 DST=8.8.8.8 LEN=138 TOS=0x00 PREC=0xC0 TTL=64 ID=18412 >>> PROTO=ICMP TYPE=3 CODE=3 [SRC=8.8.8.8 DST=10.141.86.7 LEN=110 TOS=0x00 >>> PREC=0x00 TTL=46 ID=3897 PROTO=UDP SPT=53 DPT=60721 LEN=90 >>> >>> >>> Notice that logmark seems to show that the ctstatus on the ICMP packet >>> is SEEN_REPLY, but conntrack -E shows only packets=1? tcpdump shows >>> that the ICMP packet did indeed go out >>> >>> Could someone with more knowledge of conntrack please investigate further? >>> >>> Thanks >>> >>> Ed W >>> -- >>> To unsubscribe from this list: send the line "unsubscribe netfilter" in >>> the body of a message to majordomo@vger.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >> >> -- >> To unsubscribe from this list: send the line "unsubscribe netfilter" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> > > -- > To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Possible conntrack/kernel bug - not catching certain ICMP packets 2011-07-20 21:45 ` Ed W @ 2011-07-21 6:16 ` Patrick McHardy 2011-07-21 8:43 ` Ed W 0 siblings, 1 reply; 7+ messages in thread From: Patrick McHardy @ 2011-07-21 6:16 UTC (permalink / raw) To: Ed W; +Cc: Jan Engelhardt, Netfilter Developer Mailing List On 20.07.2011 23:45, Ed W wrote: > Please, could someone take a look at this possible conntrack issue? > Even just acknowledging that it's reproducible on a selection of recent > kernels or architectures would help? > It's expected behaviour since ICMP packets related to an existing connection don't refresh the connection and are not accounted. I don't have an opinion on whether they should be accounted, I guess you could argue both ways. > >>> Essentially we have a reproducible situation where conntrack is >>> (apparently) relating a packet to a connection, but not incrementing the >>> connection counters? >>> >>> Thanks for any help figuring out a fix? >>> >>> Ed W >>> >>> >>> On 15/07/2011 12:45, Ed W wrote: >>>> Hi, This is related to a previous thread, but more complete problem >>>> statement below: >>>> >>>> I notice that I can get an ICMP packet to bypass parts of conntrack >>>> under the following conditions >>>> >>>> - Send a UDP packet that triggers some kind of UDP reply >>>> - Close the listening UDP socket before that reply arrives >>>> - Kernel generates an ICMP unreachable response which does not appear to >>>> be tracked (as expected) by conntrack >>>> >>>> Tested with kernel 2.6.38.4 + iptables 1.4.11.1 >>>> >>>> Reproduce this easily like so: >>>> >>>> nslookup www.yahoo.co.uk 8.8.8.8 & sleep 0.001 && killall nslookup >>>> >>>> (where the sleep obviously needs to be smaller than your DNS RTT >>>> lookup time. Obviously substitute nslookup/dig as appropriate...) >>>> >>>> My results >>>> >>>> # conntrack -E >>>> >>>> [NEW] udp 17 30 src=10.141.86.7 dst=8.8.8.8 sport=60721 >>>> dport=53 [UNREPLIED] src=8.8.8.8 dst=10.141.86.7 sport=53 dport=60721 >>>> [UPDATE] udp 17 29 src=10.141.86.7 dst=8.8.8.8 sport=60721 >>>> dport=53 src=8.8.8.8 dst=10.141.86.7 sport=53 dport=60721 >>>> [DESTROY] udp 17 src=10.141.86.7 dst=8.8.8.8 sport=60721 dport=53 >>>> packets=1 bytes=66 src=8.8.8.8 dst=10.141.86.7 sport=53 dport=60721 >>>> packets=1 bytes=110 >>>> >>>> # tcpdump >>>> >>>> 11:26:35.072564 IP 10.141.86.7.60721 > 8.8.8.8.domain: 2+ PTR? >>>> 8.8.8.8.in-addr.arpa. (38) >>>> 11:26:35.351804 IP 8.8.8.8.domain > 10.141.86.7.60721: 2 1/0/0 PTR >>>> google-public-dns-a.google.com. (82) >>>> 11:26:35.352110 IP 10.141.86.7 > 8.8.8.8: ICMP 10.141.86.7 udp port >>>> 60721 unreachable, length 118 >>>> >>>> >>>> # iptables -A OUTPUT -j LOGMARK >>>> # iptables -A OUTPUT -j LOG >>>> >>>> Jul 15 11:26:35 localhost kern.warn kernel: [ 6676.964396] iif=0 >>>> hook=OUTPUT nfmark=0x0 secmark=0x0 classify=0x0 ctdir=ORIGINAL >>>> ct=0xcf3d5060 ctmark=0x0 ctstate=NEW ctstatus= lifetime=6346s >>>> Jul 15 11:26:35 localhost kern.warn kernel: [ 6676.964396] IN= OUT=ppp1 >>>> SRC=10.141.86.7 DST=8.8.8.8 LEN=66 TOS=0x00 PREC=0x00 TTL=64 ID=19971 DF >>>> PROTO=UDP SPT=60721 DPT=53 LEN=46 >>>> Jul 15 11:26:35 localhost kern.warn kernel: [ 6677.249312] iif=0 >>>> hook=OUTPUT nfmark=0x0 secmark=0x0 classify=0x0 ctdir=ORIGINAL >>>> ct=0xcf3d5060 ctmark=0x0 ctstate=RELATED ctstatus=SEEN_REPLY,CONFIRMED >>>> lifetime=4294937s >>>> Jul 15 11:26:35 localhost kern.warn kernel: [ 6677.249426] IN= OUT=ppp1 >>>> SRC=10.141.86.7 DST=8.8.8.8 LEN=138 TOS=0x00 PREC=0xC0 TTL=64 ID=18412 >>>> PROTO=ICMP TYPE=3 CODE=3 [SRC=8.8.8.8 DST=10.141.86.7 LEN=110 TOS=0x00 >>>> PREC=0x00 TTL=46 ID=3897 PROTO=UDP SPT=53 DPT=60721 LEN=90 >>>> >>>> >>>> Notice that logmark seems to show that the ctstatus on the ICMP packet >>>> is SEEN_REPLY, but conntrack -E shows only packets=1? tcpdump shows >>>> that the ICMP packet did indeed go out >>>> >>>> Could someone with more knowledge of conntrack please investigate further? >>>> >>>> Thanks >>>> >>>> Ed W >>>> -- >>>> To unsubscribe from this list: send the line "unsubscribe netfilter" in >>>> the body of a message to majordomo@vger.kernel.org >>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>> >>> -- >>> To unsubscribe from this list: send the line "unsubscribe netfilter" in >>> the body of a message to majordomo@vger.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>> >> >> -- >> To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html > > -- > To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Possible conntrack/kernel bug - not catching certain ICMP packets 2011-07-21 6:16 ` Patrick McHardy @ 2011-07-21 8:43 ` Ed W 2011-07-21 9:14 ` Patrick McHardy 0 siblings, 1 reply; 7+ messages in thread From: Ed W @ 2011-07-21 8:43 UTC (permalink / raw) To: Patrick McHardy; +Cc: Jan Engelhardt, Netfilter Developer Mailing List On 21/07/2011 07:16, Patrick McHardy wrote: > It's expected behaviour since ICMP packets related to an existing > connection don't refresh the connection and are not accounted. > I don't have an opinion on whether they should be accounted, I > guess you could argue both ways. Thanks for the feedback. I guess I was hoping that conntrack could be used for accurate bandwidth accounting, however, it seems to ignore this type of packet, so it's count is going to deviate from a simple interface byte counter? I don't see the argument for *not* counting the bytes from the ICMP packet though? Surely the goal of conntrack is that everything is scooped into some connection? It seems like in this case conntrack labels this packet as belonging to the connection, BUT doesn't update the packet or byte counts - this seems like a half and half situation? Thanks for replying - interested to hear the arguments against refreshing byte counts given that conntrack has already marked it as related? Cheers Ed W ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Possible conntrack/kernel bug - not catching certain ICMP packets 2011-07-21 8:43 ` Ed W @ 2011-07-21 9:14 ` Patrick McHardy 2011-07-25 23:45 ` Jan Engelhardt 0 siblings, 1 reply; 7+ messages in thread From: Patrick McHardy @ 2011-07-21 9:14 UTC (permalink / raw) To: Ed W; +Cc: Jan Engelhardt, Netfilter Developer Mailing List On 21.07.2011 10:43, Ed W wrote: > On 21/07/2011 07:16, Patrick McHardy wrote: >> It's expected behaviour since ICMP packets related to an existing >> connection don't refresh the connection and are not accounted. >> I don't have an opinion on whether they should be accounted, I >> guess you could argue both ways. > > Thanks for the feedback. > > I guess I was hoping that conntrack could be used for accurate bandwidth > accounting, however, it seems to ignore this type of packet, so it's > count is going to deviate from a simple interface byte counter? Yes, but it's going to do that anyways since there are also packets which can't be tracked, invalid packets, etc. Also conntrack doesn't account for link layer headers and only for IPv4/v6 packets. > I don't see the argument for *not* counting the bytes from the ICMP > packet though? Surely the goal of conntrack is that everything is > scooped into some connection? It seems like in this case conntrack > labels this packet as belonging to the connection, BUT doesn't update > the packet or byte counts - this seems like a half and half situation? Well, you could just as well argue that someone is only interested in the amount of bytes/packets which were actually transfered within a connection. Since you mentioned interface counters, we also don't account for ARP requests triggered by a connection. I agree that it would be good to account for these packet *somewhere*. > Thanks for replying - interested to hear the arguments against > refreshing byte counts given that conntrack has already marked it as > related? Well, basically the related ICMP packets bypass protocol related tracking, which is of course correct since they obviously shouldn't be handled by f.i. TCP tracking, but this is where the accounting happens. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Possible conntrack/kernel bug - not catching certain ICMP packets 2011-07-21 9:14 ` Patrick McHardy @ 2011-07-25 23:45 ` Jan Engelhardt 2011-07-26 13:00 ` Patrick McHardy 0 siblings, 1 reply; 7+ messages in thread From: Jan Engelhardt @ 2011-07-25 23:45 UTC (permalink / raw) To: Patrick McHardy; +Cc: Ed W, Netfilter Developer Mailing List On Thursday 2011-07-21 11:14, Patrick McHardy wrote: >On 21.07.2011 10:43, Ed W wrote: >> On 21/07/2011 07:16, Patrick McHardy wrote: >>> It's expected behaviour since ICMP packets related to an existing >>> connection don't refresh the connection and are not accounted. >>> I don't have an opinion on whether they should be accounted, I >>> guess you could argue both ways. >> >> Thanks for the feedback. >> >> I guess I was hoping that conntrack could be used for accurate bandwidth >> accounting, however, it seems to ignore this type of packet, so it's >> count is going to deviate from a simple interface byte counter? > >Yes, but it's going to do that anyways since there are also packets >which can't be tracked, invalid packets, etc. Also conntrack doesn't >account for link layer headers and only for IPv4/v6 packets. While toying around, I found that if an skb is classified as RELATED, skb->nfct->master always points to skb->nfct itself. Is that a bug or something? Should it not point to the origin CT? ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Possible conntrack/kernel bug - not catching certain ICMP packets 2011-07-25 23:45 ` Jan Engelhardt @ 2011-07-26 13:00 ` Patrick McHardy 0 siblings, 0 replies; 7+ messages in thread From: Patrick McHardy @ 2011-07-26 13:00 UTC (permalink / raw) To: Jan Engelhardt; +Cc: Ed W, Netfilter Developer Mailing List On 26.07.2011 01:45, Jan Engelhardt wrote: > > On Thursday 2011-07-21 11:14, Patrick McHardy wrote: >> On 21.07.2011 10:43, Ed W wrote: >>> On 21/07/2011 07:16, Patrick McHardy wrote: >>>> It's expected behaviour since ICMP packets related to an existing >>>> connection don't refresh the connection and are not accounted. >>>> I don't have an opinion on whether they should be accounted, I >>>> guess you could argue both ways. >>> >>> Thanks for the feedback. >>> >>> I guess I was hoping that conntrack could be used for accurate bandwidth >>> accounting, however, it seems to ignore this type of packet, so it's >>> count is going to deviate from a simple interface byte counter? >> >> Yes, but it's going to do that anyways since there are also packets >> which can't be tracked, invalid packets, etc. Also conntrack doesn't >> account for link layer headers and only for IPv4/v6 packets. > > While toying around, I found that if an skb is classified as RELATED, > skb->nfct->master always points to skb->nfct itself. Is that a bug > or something? Should it not point to the origin CT? For RELATED connections expected by a helper? That would be wrong, it should point to the real master. ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2011-07-26 12:42 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <4E202844.30602@wildgooses.com>
[not found] ` <4E241AB0.5060603@wildgooses.com>
2011-07-18 12:16 ` Possible conntrack/kernel bug - not catching certain ICMP packets Jan Engelhardt
2011-07-20 21:45 ` Ed W
2011-07-21 6:16 ` Patrick McHardy
2011-07-21 8:43 ` Ed W
2011-07-21 9:14 ` Patrick McHardy
2011-07-25 23:45 ` Jan Engelhardt
2011-07-26 13:00 ` Patrick McHardy
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).