nf_conntrack_count versus '/proc/net/nf_conntrack | wc -l' count

nf_conntrack_count versus '/proc/net/nf_conntrack | wc -l' count

[Afi Gjermund]

Hi all,

I am running into an odd issue where the kernel begins to drop packets
because the connection tracking table is full. (I am running
2.6.26.5).

A 'cat /proc/sys/net/netfilter/nf_conntrack_count' says 4096.  But if
I do a 'cat /proc/net/nf_conntrack | wc -l' then it says 4.

A tail on /var/log/messages has:

Feb 15 16:45:33 titan user.warn kernel: __ratelimit: 20 messages suppressed
Feb 15 16:45:33 titan user.warn kernel: nf_conntrack: table full,
dropping packet.

I have an interface that allows me to do a 'nf_conntrack_flush' in
kernel, but that seems to only clear the table of
/proc/net/nf_conntrack and not the count the kernel has.

I need a way to ensure that at say 95% there is a way to clear the
"actual" table as well as the count...whichever that may be.  There
seems to be a disconnect between the implementation versus my
understanding of how the system is working. Any help is appreciated.

Afi

Re: nf_conntrack_count versus '/proc/net/nf_conntrack | wc -l' count

[Patrick McHardy]

Afi Gjermund wrote:
> Hi all,
> 
> I am running into an odd issue where the kernel begins to drop packets
> because the connection tracking table is full. (I am running
> 2.6.26.5).
> 
> A 'cat /proc/sys/net/netfilter/nf_conntrack_count' says 4096.  But if
> I do a 'cat /proc/net/nf_conntrack | wc -l' then it says 4.
> 
> A tail on /var/log/messages has:
> 
> Feb 15 16:45:33 titan user.warn kernel: __ratelimit: 20 messages suppressed
> Feb 15 16:45:33 titan user.warn kernel: nf_conntrack: table full,
> dropping packet.
> 
> I have an interface that allows me to do a 'nf_conntrack_flush' in
> kernel, but that seems to only clear the table of
> /proc/net/nf_conntrack and not the count the kernel has.
> 
> I need a way to ensure that at say 95% there is a way to clear the
> "actual" table as well as the count...whichever that may be.  There
> seems to be a disconnect between the implementation versus my
> understanding of how the system is working. Any help is appreciated.

Conntracks might exist and not be in the global table anymore,
f.i. when referenced by a packet. The difference in your case
seems pretty extreme, so I'd guess that packets are leaked
somewhere.

Re: nf_conntrack_count versus '/proc/net/nf_conntrack | wc -l' count

[Jan Engelhardt]

On Monday 2010-02-15 18:29, Patrick McHardy wrote:
>Afi Gjermund wrote:
>> 
>> I am running into an odd issue where the kernel begins to drop packets
>> because the connection tracking table is full. (I am running
>> 2.6.26.5).
>> 
>> A 'cat /proc/sys/net/netfilter/nf_conntrack_count' says 4096.  But if
>> I do a 'cat /proc/net/nf_conntrack | wc -l' then it says 4.
>
>Conntracks might exist and not be in the global table anymore,
>f.i. when referenced by a packet. The difference in your case
>seems pretty extreme, so I'd guess that packets are leaked
>somewhere.

So, that would make for 4092 expected connections then?

Afi, what would `conntrack -L expect` give?

Re: nf_conntrack_count versus '/proc/net/nf_conntrack | wc -l' count

[Afi Gjermund]

On Mon, Feb 15, 2010 at 9:46 AM, Jan Engelhardt <jengelh@medozas.de> wrote:
>
> On Monday 2010-02-15 18:29, Patrick McHardy wrote:
>>Afi Gjermund wrote:
>>>
>>> I am running into an odd issue where the kernel begins to drop packets
>>> because the connection tracking table is full. (I am running
>>> 2.6.26.5).
>>>
>>> A 'cat /proc/sys/net/netfilter/nf_conntrack_count' says 4096.  But if
>>> I do a 'cat /proc/net/nf_conntrack | wc -l' then it says 4.
>>
>>Conntracks might exist and not be in the global table anymore,
>>f.i. when referenced by a packet. The difference in your case
>>seems pretty extreme, so I'd guess that packets are leaked
>>somewhere.
>
> So, that would make for 4092 expected connections then?
>
> Afi, what would `conntrack -L expect` give?
>

Jan, I am running this on an embedded system and will have to
cross-compile the userspace tools and get back to you.  One thing to
note is, I have stopped any traffic flowing through the device, and
yet I am still receiving the kernel drop messages.  Any change its
connection tracking on the loopback? ( I use the loopback for IPC ).

Re: nf_conntrack_count versus '/proc/net/nf_conntrack | wc -l' count

[Jan Engelhardt]

On Monday 2010-02-15 19:04, Afi Gjermund wrote:
>>>>
>>>> I am running into an odd issue where the kernel begins to drop packets
>>>> because the connection tracking table is full. (I am running
>>>> 2.6.26.5).
>>>>
>>>> A 'cat /proc/sys/net/netfilter/nf_conntrack_count' says 4096.  But if
>>>> I do a 'cat /proc/net/nf_conntrack | wc -l' then it says 4.
>>>
>>>Conntracks might exist and not be in the global table anymore,
>>>f.i. when referenced by a packet. The difference in your case
>>>seems pretty extreme, so I'd guess that packets are leaked
>>>somewhere.
>>
>> So, that would make for 4092 expected connections then?
>>
>> Afi, what would `conntrack -L expect` give?
   (meant: conntrack -L expect | wc -l)

>One thing to
>note is, I have stopped any traffic flowing through the device, and
>yet I am still receiving the kernel drop messages.  Any change its
>connection tracking on the loopback? ( I use the loopback for IPC ).

Yes. conntrack does not care about what interface packets
come in or go out on. Unless it's NOTRACKed, it's counted.

Re: nf_conntrack_count versus '/proc/net/nf_conntrack | wc -l' count

[Afi Gjermund]

On Mon, Feb 15, 2010 at 11:00 AM, Jan Engelhardt <jengelh@medozas.de> wrote:
>
> On Monday 2010-02-15 19:04, Afi Gjermund wrote:
>>>>>
>>>>> I am running into an odd issue where the kernel begins to drop packets
>>>>> because the connection tracking table is full. (I am running
>>>>> 2.6.26.5).
>>>>>
>>>>> A 'cat /proc/sys/net/netfilter/nf_conntrack_count' says 4096.  But if
>>>>> I do a 'cat /proc/net/nf_conntrack | wc -l' then it says 4.
>>>>
>>>>Conntracks might exist and not be in the global table anymore,
>>>>f.i. when referenced by a packet. The difference in your case
>>>>seems pretty extreme, so I'd guess that packets are leaked
>>>>somewhere.
>>>
>>> So, that would make for 4092 expected connections then?
>>>
>>> Afi, what would `conntrack -L expect` give?
>   (meant: conntrack -L expect | wc -l)
>
>>One thing to
>>note is, I have stopped any traffic flowing through the device, and
>>yet I am still receiving the kernel drop messages.  Any change its
>>connection tracking on the loopback? ( I use the loopback for IPC ).
>
> Yes. conntrack does not care about what interface packets
> come in or go out on. Unless it's NOTRACKed, it's counted.
>
Okay, after some tinkering I was able to get the userspace application
going.  The results, however, do not seem that helpful.

root@00:00:10:73:77:64 ~# ./conntrack -L expect | wc -l
conntrack v0.9.14 (conntrack-tools): 0 expectations have been shown.
0

root@titan ~# ./conntrack -L conntrack
udp      17 179 src=0.0.0.0 dst=0.0.0.0 sport=0 dport=0 packets=89099
bytes=12968758 src=0.0.0.0 dst=0.0.0.0 sport=0 dport=0 packets=110358
bytes=17041625 [ASSURED] mark=0 use=1
udp      17 179 src=0.0.0.0 dst=0.0.0.0 sport=0 dport=0 packets=87867
bytes=12816098 src=0.0.0.0 dst=0.0.0.0 sport=0 dport=0 packets=107497
bytes=16573614 [ASSURED] mark=0 use=1
conntrack v0.9.14 (conntrack-tools): 2 flow entries have been shown.

root@titan ~# cat /proc/sys/net/netfilter/nf_conntrack_count
4096

root@titan ~# ./conntrack -S
entries                 4096
searched                2525930
found                   418064263
new                     295548
invalid                 1509
ignore                  233351
delete                  291452
delete_list             280256
insert                  280258
insert_failed           0
drop                    69960
early_drop              173185
icmp_error              45
expect_new              0
expect_create           0
expect_delete           0

Re: nf_conntrack_count versus '/proc/net/nf_conntrack | wc -l' count

[Afi Gjermund]

On Mon, Feb 15, 2010 at 11:30 AM, Afi Gjermund <afigjermund@gmail.com> wrote:
> On Mon, Feb 15, 2010 at 11:00 AM, Jan Engelhardt <jengelh@medozas.de> wrote:
>>
>> On Monday 2010-02-15 19:04, Afi Gjermund wrote:
>>>>>>
>>>>>> I am running into an odd issue where the kernel begins to drop packets
>>>>>> because the connection tracking table is full. (I am running
>>>>>> 2.6.26.5).
>>>>>>
>>>>>> A 'cat /proc/sys/net/netfilter/nf_conntrack_count' says 4096.  But if
>>>>>> I do a 'cat /proc/net/nf_conntrack | wc -l' then it says 4.
>>>>>
>>>>>Conntracks might exist and not be in the global table anymore,
>>>>>f.i. when referenced by a packet. The difference in your case
>>>>>seems pretty extreme, so I'd guess that packets are leaked
>>>>>somewhere.
>>>>
>>>> So, that would make for 4092 expected connections then?
>>>>
>>>> Afi, what would `conntrack -L expect` give?
>>   (meant: conntrack -L expect | wc -l)
>>
>>>One thing to
>>>note is, I have stopped any traffic flowing through the device, and
>>>yet I am still receiving the kernel drop messages.  Any change its
>>>connection tracking on the loopback? ( I use the loopback for IPC ).
>>
>> Yes. conntrack does not care about what interface packets
>> come in or go out on. Unless it's NOTRACKed, it's counted.
>>
> Okay, after some tinkering I was able to get the userspace application
> going.  The results, however, do not seem that helpful.
>
> root@00:00:10:73:77:64 ~# ./conntrack -L expect | wc -l
> conntrack v0.9.14 (conntrack-tools): 0 expectations have been shown.
> 0
>
> root@titan ~# ./conntrack -L conntrack
> udp      17 179 src=0.0.0.0 dst=0.0.0.0 sport=0 dport=0 packets=89099
> bytes=12968758 src=0.0.0.0 dst=0.0.0.0 sport=0 dport=0 packets=110358
> bytes=17041625 [ASSURED] mark=0 use=1
> udp      17 179 src=0.0.0.0 dst=0.0.0.0 sport=0 dport=0 packets=87867
> bytes=12816098 src=0.0.0.0 dst=0.0.0.0 sport=0 dport=0 packets=107497
> bytes=16573614 [ASSURED] mark=0 use=1
> conntrack v0.9.14 (conntrack-tools): 2 flow entries have been shown.
>
> root@titan ~# cat /proc/sys/net/netfilter/nf_conntrack_count
> 4096
>
> root@titan ~# ./conntrack -S
> entries                 4096
> searched                2525930
> found                   418064263
> new                     295548
> invalid                 1509
> ignore                  233351
> delete                  291452
> delete_list             280256
> insert                  280258
> insert_failed           0
> drop                    69960
> early_drop              173185
> icmp_error              45
> expect_new              0
> expect_create           0
> expect_delete           0
>
Oops..wrong machine...

root@titan ~# ./conntrack -L expect | wc -l
conntrack v0.9.14 (conntrack-tools): 0 expectations have been shown.
0

Re: nf_conntrack_count versus '/proc/net/nf_conntrack | wc -l' count

[Eric Dumazet]

Le lundi 15 février 2010 à 11:30 -0800, Afi Gjermund a écrit :
> root@titan ~# ./conntrack -L conntrack
> udp      17 179 src=0.0.0.0 dst=0.0.0.0 sport=0 dport=0 packets=89099
> bytes=12968758 src=0.0.0.0 dst=0.0.0.0 sport=0 dport=0 packets=110358
> bytes=17041625 [ASSURED] mark=0 use=1
> udp      17 179 src=0.0.0.0 dst=0.0.0.0 sport=0 dport=0 packets=87867
> bytes=12816098 src=0.0.0.0 dst=0.0.0.0 sport=0 dport=0 packets=107497
> bytes=16573614 [ASSURED] mark=0 use=1
> conntrack v0.9.14 (conntrack-tools): 2 flow entries have been shown.
> 

This looks strange...


--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: nf_conntrack_count versus '/proc/net/nf_conntrack | wc -l' count

[Jan Engelhardt]

On Monday 2010-02-15 21:04, Eric Dumazet wrote:
>Le lundi 15 février 2010 à 11:30 -0800, Afi Gjermund a écrit :
>> root@titan ~# ./conntrack -L conntrack
>> udp      17 179 src=0.0.0.0 dst=0.0.0.0 sport=0 dport=0 packets=89099
>> bytes=12968758 src=0.0.0.0 dst=0.0.0.0 sport=0 dport=0 packets=110358
>> bytes=17041625 [ASSURED] mark=0 use=1
>> udp      17 179 src=0.0.0.0 dst=0.0.0.0 sport=0 dport=0 packets=87867
>> bytes=12816098 src=0.0.0.0 dst=0.0.0.0 sport=0 dport=0 packets=107497
>> bytes=16573614 [ASSURED] mark=0 use=1
>> conntrack v0.9.14 (conntrack-tools): 2 flow entries have been shown.
>> 
>
>This looks strange...

Could it be that there are ct entries in other namespaces that
conntrack -L and /proc/net/nf_conntrack does not show,
but which nf_conntrack_count counts?
If the procfs files are netns safe at all..
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: nf_conntrack_count versus '/proc/net/nf_conntrack | wc -l' count

[Afi Gjermund]

On Mon, Feb 15, 2010 at 12:33 PM, Jan Engelhardt <jengelh@medozas.de> wrote:
> On Monday 2010-02-15 21:04, Eric Dumazet wrote:
>>Le lundi 15 février 2010 à 11:30 -0800, Afi Gjermund a écrit :
>>> root@titan ~# ./conntrack -L conntrack
>>> udp      17 179 src=0.0.0.0 dst=0.0.0.0 sport=0 dport=0 packets=89099
>>> bytes=12968758 src=0.0.0.0 dst=0.0.0.0 sport=0 dport=0 packets=110358
>>> bytes=17041625 [ASSURED] mark=0 use=1
>>> udp      17 179 src=0.0.0.0 dst=0.0.0.0 sport=0 dport=0 packets=87867
>>> bytes=12816098 src=0.0.0.0 dst=0.0.0.0 sport=0 dport=0 packets=107497
>>> bytes=16573614 [ASSURED] mark=0 use=1
>>> conntrack v0.9.14 (conntrack-tools): 2 flow entries have been shown.
>>>
>>
>>This looks strange...
>
> Could it be that there are ct entries in other namespaces that
> conntrack -L and /proc/net/nf_conntrack does not show,
> but which nf_conntrack_count counts?
> If the procfs files are netns safe at all..
>
On my 2.6.26.5 kernel I do not have CONFIG_NAMESPACES set.
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: nf_conntrack_count versus '/proc/net/nf_conntrack | wc -l' count

[Eric Dumazet]

Le lundi 15 février 2010 à 21:33 +0100, Jan Engelhardt a écrit :
> On Monday 2010-02-15 21:04, Eric Dumazet wrote:
> >Le lundi 15 février 2010 à 11:30 -0800, Afi Gjermund a écrit :
> >> root@titan ~# ./conntrack -L conntrack
> >> udp      17 179 src=0.0.0.0 dst=0.0.0.0 sport=0 dport=0 packets=89099
> >> bytes=12968758 src=0.0.0.0 dst=0.0.0.0 sport=0 dport=0 packets=110358
> >> bytes=17041625 [ASSURED] mark=0 use=1
> >> udp      17 179 src=0.0.0.0 dst=0.0.0.0 sport=0 dport=0 packets=87867
> >> bytes=12816098 src=0.0.0.0 dst=0.0.0.0 sport=0 dport=0 packets=107497
> >> bytes=16573614 [ASSURED] mark=0 use=1
> >> conntrack v0.9.14 (conntrack-tools): 2 flow entries have been shown.
> >> 
> >
> >This looks strange...
> 
> Could it be that there are ct entries in other namespaces that
> conntrack -L and /proc/net/nf_conntrack does not show,
> but which nf_conntrack_count counts?
> If the procfs files are netns safe at all..

Well, its an embedded platform, I doubt it is namespace enabled :)

(and kernel version is 2.6.26.5, not yet namespace ready for conntrack)



--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: nf_conntrack_count versus '/proc/net/nf_conntrack | wc -l' count

[Eric Dumazet]

Le lundi 15 février 2010 à 13:08 -0800, Afi Gjermund a écrit :
> >
> On my 2.6.26.5 kernel I do not have CONFIG_NAMESPACES set.

could you post the result of 'netstat -s' ?


--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: nf_conntrack_count versus '/proc/net/nf_conntrack | wc -l' count

[Afi Gjermund]

On Mon, Feb 15, 2010 at 1:52 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> Le lundi 15 février 2010 à 13:08 -0800, Afi Gjermund a écrit :
>> >
>> On my 2.6.26.5 kernel I do not have CONFIG_NAMESPACES set.
>
> could you post the result of 'netstat -s' ?
>
>
>

Unfortunately the Busybox version of netstat doesn't have the statistics option.
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: nf_conntrack_count versus '/proc/net/nf_conntrack | wc -l' count

[Eric Dumazet]

Le lundi 15 février 2010 à 14:00 -0800, Afi Gjermund a écrit :
> On Mon, Feb 15, 2010 at 1:52 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> > Le lundi 15 février 2010 à 13:08 -0800, Afi Gjermund a écrit :
> >> >
> >> On my 2.6.26.5 kernel I do not have CONFIG_NAMESPACES set.
> >
> > could you post the result of 'netstat -s' ?
> >
> >
> >
> 
> Unfortunately the Busybox version of netstat doesn't have the statistics option.

ok then :)

cat /proc/net/snmp


--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: nf_conntrack_count versus '/proc/net/nf_conntrack | wc -l' count

[Afi Gjermund]

On Mon, Feb 15, 2010 at 2:02 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> Le lundi 15 février 2010 à 14:00 -0800, Afi Gjermund a écrit :
>> On Mon, Feb 15, 2010 at 1:52 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
>> > Le lundi 15 février 2010 à 13:08 -0800, Afi Gjermund a écrit :
>> >> >
>> >> On my 2.6.26.5 kernel I do not have CONFIG_NAMESPACES set.
>> >
>> > could you post the result of 'netstat -s' ?
>> >
>> >
>> >
>>
>> Unfortunately the Busybox version of netstat doesn't have the statistics option.
>
> ok then :)
>
> cat /proc/net/snmp
>
>
>
Ip: Forwarding DefaultTTL InReceives InHdrErrors InAddrErrors
ForwDatagrams InUnknownProtos InDiscards InDelivers OutRequests
OutDiscards OutNoRoutes ReasmTimeout ReasmReqds ReasmOKs ReasmFails
FragOKs FragFails FragCreates
Ip: 2 64 137517215 0 33726 0 0 0 136714167 151150681 186 53 0 0 0 0 0 0 0
Icmp: InMsgs InErrors InDestUnreachs InTimeExcds InParmProbs
InSrcQuenchs InRedirects InEchos InEchoReps InTimestamps
InTimestampReps InAddrMasks InAddrMaskReps OutMsgs OutErrors
OutDestUnreachs OutTimeExcds OutParmProbs OutSrcQuenchs O
utRedirects OutEchos OutEchoReps OutTimestamps OutTimestampReps
OutAddrMasks OutAddrMaskReps
Icmp: 35 0 35 0 0 0 0 0 0 0 0 0 0 437 0 437 0 0 0 0 0 0 0 0 0 0
IcmpMsg: InType3 OutType3
IcmpMsg: 35 437
Tcp: RtoAlgorithm RtoMin RtoMax MaxConn ActiveOpens PassiveOpens
AttemptFails EstabResets CurrEstab InSegs OutSegs RetransSegs InErrs
OutRsts
Tcp: 1 200 120000 -1 0 139 0 80 0 17587 19167 63 0 11984
Udp: InDatagrams NoPorts InErrors OutDatagrams RcvbufErrors SndbufErrors
Udp: 136619682 437 0 151131067 0 0
UdpLite: InDatagrams NoPorts InErrors OutDatagrams RcvbufErrors SndbufErrors
UdpLite: 0 0 0 0 0 0
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: nf_conntrack_count versus '/proc/net/nf_conntrack | wc -l' count

[Afi Gjermund]

On Mon, Feb 15, 2010 at 2:10 PM, Afi Gjermund <afigjermund@gmail.com> wrote:
> On Mon, Feb 15, 2010 at 2:02 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
>> Le lundi 15 février 2010 à 14:00 -0800, Afi Gjermund a écrit :
>>> On Mon, Feb 15, 2010 at 1:52 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
>>> > Le lundi 15 février 2010 à 13:08 -0800, Afi Gjermund a écrit :
>>> >> >
>>> >> On my 2.6.26.5 kernel I do not have CONFIG_NAMESPACES set.
>>> >
>>> > could you post the result of 'netstat -s' ?
>>> >
>>> >
>>> >
>>>
>>> Unfortunately the Busybox version of netstat doesn't have the statistics option.
>>
>> ok then :)
>>
>> cat /proc/net/snmp
>>
>>
>>
> Ip: Forwarding DefaultTTL InReceives InHdrErrors InAddrErrors
> ForwDatagrams InUnknownProtos InDiscards InDelivers OutRequests
> OutDiscards OutNoRoutes ReasmTimeout ReasmReqds ReasmOKs ReasmFails
> FragOKs FragFails FragCreates
> Ip: 2 64 137517215 0 33726 0 0 0 136714167 151150681 186 53 0 0 0 0 0 0 0
> Icmp: InMsgs InErrors InDestUnreachs InTimeExcds InParmProbs
> InSrcQuenchs InRedirects InEchos InEchoReps InTimestamps
> InTimestampReps InAddrMasks InAddrMaskReps OutMsgs OutErrors
> OutDestUnreachs OutTimeExcds OutParmProbs OutSrcQuenchs O
> utRedirects OutEchos OutEchoReps OutTimestamps OutTimestampReps
> OutAddrMasks OutAddrMaskReps
> Icmp: 35 0 35 0 0 0 0 0 0 0 0 0 0 437 0 437 0 0 0 0 0 0 0 0 0 0
> IcmpMsg: InType3 OutType3
> IcmpMsg: 35 437
> Tcp: RtoAlgorithm RtoMin RtoMax MaxConn ActiveOpens PassiveOpens
> AttemptFails EstabResets CurrEstab InSegs OutSegs RetransSegs InErrs
> OutRsts
> Tcp: 1 200 120000 -1 0 139 0 80 0 17587 19167 63 0 11984
> Udp: InDatagrams NoPorts InErrors OutDatagrams RcvbufErrors SndbufErrors
> Udp: 136619682 437 0 151131067 0 0
> UdpLite: InDatagrams NoPorts InErrors OutDatagrams RcvbufErrors SndbufErrors
> UdpLite: 0 0 0 0 0 0
>
I am still trying to figure out why the nf_conntrack_count differs
from the table system.  I decided I would use the conntrack userspace
tools.
Both of my NICs are unplugged with no other userspace applications
running to affect connection tracking counts.


root@titan ~# date
Thu Feb 18 17:35:21 UTC 2010

root@titan ~# ./conntrack -C conntrack
351

root@titan ~# date
Thu Feb 18 17:35:24 UTC 2010

root@titan ~# ./conntrack -F conntrack
conntrack v0.9.14 (conntrack-tools): connection tracking table has been emptied.

root@titan ~# date
Thu Feb 18 17:35:31 UTC 2010

root@titan ~# ./conntrack -C conntrack
351

root@titan ~# date
Thu Feb 18 17:35:36 UTC 2010

Shouldn't the value after the flush be 0? The traffic that has created
this mess is from a REDIRECT rule in the PREROUTING chain of the 'nat'
table.
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: nf_conntrack_count versus '/proc/net/nf_conntrack | wc -l' count

[Eric Dumazet]

Le jeudi 18 février 2010 à 09:40 -0800, Afi Gjermund a écrit :
> I am still trying to figure out why the nf_conntrack_count differs
> from the table system.  I decided I would use the conntrack userspace
> tools.
> Both of my NICs are unplugged with no other userspace applications
> running to affect connection tracking counts.
> 
> 
> root@titan ~# date
> Thu Feb 18 17:35:21 UTC 2010
> 
> root@titan ~# ./conntrack -C conntrack
> 351
> 
> root@titan ~# date
> Thu Feb 18 17:35:24 UTC 2010
> 
> root@titan ~# ./conntrack -F conntrack
> conntrack v0.9.14 (conntrack-tools): connection tracking table has been emptied.
> 
> root@titan ~# date
> Thu Feb 18 17:35:31 UTC 2010
> 
> root@titan ~# ./conntrack -C conntrack
> 351
> 
> root@titan ~# date
> Thu Feb 18 17:35:36 UTC 2010
> 
> Shouldn't the value after the flush be 0? The traffic that has created
> this mess is from a REDIRECT rule in the PREROUTING chain of the 'nat'
> table.

Could you post a copy of these rules ?

Thanks


--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: nf_conntrack_count versus '/proc/net/nf_conntrack | wc -l' count

[Afi Gjermund]

On Thu, Feb 18, 2010 at 9:51 AM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> Le jeudi 18 février 2010 à 09:40 -0800, Afi Gjermund a écrit :
>> I am still trying to figure out why the nf_conntrack_count differs
>> from the table system.  I decided I would use the conntrack userspace
>> tools.
>> Both of my NICs are unplugged with no other userspace applications
>> running to affect connection tracking counts.
>>
>>
>> root@titan ~# date
>> Thu Feb 18 17:35:21 UTC 2010
>>
>> root@titan ~# ./conntrack -C conntrack
>> 351
>>
>> root@titan ~# date
>> Thu Feb 18 17:35:24 UTC 2010
>>
>> root@titan ~# ./conntrack -F conntrack
>> conntrack v0.9.14 (conntrack-tools): connection tracking table has been emptied.
>>
>> root@titan ~# date
>> Thu Feb 18 17:35:31 UTC 2010
>>
>> root@titan ~# ./conntrack -C conntrack
>> 351
>>
>> root@titan ~# date
>> Thu Feb 18 17:35:36 UTC 2010
>>
>> Shouldn't the value after the flush be 0? The traffic that has created
>> this mess is from a REDIRECT rule in the PREROUTING chain of the 'nat'
>> table.
>
> Could you post a copy of these rules ?
>
> Thanks
>
>
>
iptables -t nat -A PREROUTING -p tcp -s X.X.X.X -d X.X.X.X --sport X
--dport X -j REDIRECT --to-port X
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: nf_conntrack_count versus '/proc/net/nf_conntrack | wc -l' count

[Eric Dumazet]

Le jeudi 18 février 2010 à 09:55 -0800, Afi Gjermund a écrit :
> On Thu, Feb 18, 2010 at 9:51 AM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> > Le jeudi 18 février 2010 à 09:40 -0800, Afi Gjermund a écrit :
> >> I am still trying to figure out why the nf_conntrack_count differs
> >> from the table system.  I decided I would use the conntrack userspace
> >> tools.
> >> Both of my NICs are unplugged with no other userspace applications
> >> running to affect connection tracking counts.
> >>
> >>
> >> root@titan ~# date
> >> Thu Feb 18 17:35:21 UTC 2010
> >>
> >> root@titan ~# ./conntrack -C conntrack
> >> 351
> >>
> >> root@titan ~# date
> >> Thu Feb 18 17:35:24 UTC 2010
> >>
> >> root@titan ~# ./conntrack -F conntrack
> >> conntrack v0.9.14 (conntrack-tools): connection tracking table has been emptied.
> >>
> >> root@titan ~# date
> >> Thu Feb 18 17:35:31 UTC 2010
> >>
> >> root@titan ~# ./conntrack -C conntrack
> >> 351
> >>
> >> root@titan ~# date
> >> Thu Feb 18 17:35:36 UTC 2010
> >>
> >> Shouldn't the value after the flush be 0? The traffic that has created
> >> this mess is from a REDIRECT rule in the PREROUTING chain of the 'nat'
> >> table.
> >
> > Could you post a copy of these rules ?
> >
> > Thanks
> >
> >
> >
> iptables -t nat -A PREROUTING -p tcp -s X.X.X.X -d X.X.X.X --sport X
> --dport X -j REDIRECT --to-port X

Yes I understood you were using such rules, but I cannot understand how
it can trigger without real nics being plugged. So I asked you some
details, apprently you dont want to provide them and prefer to hide from
us :)




--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: nf_conntrack_count versus '/proc/net/nf_conntrack | wc -l' count

[Douglas Diniz]

I'm facing the same problem. I'm working in a embedded system with
kernel 2.6.20-6. When I send a ping (or any other protocol) through
eth0 to eth1 (or vice versa) the conntrack count isn't decremented. If
I send the ping through any other interface (eth0 to wifi, eth1 to
wifi, wifi to eth0 and wifi to eth1) I have no problem.
The problem seems to be only between the ethernet interfaces.
I debug the netfilter and I saw that when the problem occurs the "use"
variable inside conntract structure in > 1, so this variable is only
decremented by 1, not reaching in 0, and then the destroy_conntrack
function is not called.

So I think that the problem is more low level, and some events aren't
reaching netfilter, and the "use"variable isn't decremented properly.

This could be a problem with the ethernet driver?

Thanks....

On Thu, Feb 18, 2010 at 3:51 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> Le jeudi 18 février 2010 à 09:40 -0800, Afi Gjermund a écrit :
>> I am still trying to figure out why the nf_conntrack_count differs
>> from the table system.  I decided I would use the conntrack userspace
>> tools.
>> Both of my NICs are unplugged with no other userspace applications
>> running to affect connection tracking counts.
>>
>>
>> root@titan ~# date
>> Thu Feb 18 17:35:21 UTC 2010
>>
>> root@titan ~# ./conntrack -C conntrack
>> 351
>>
>> root@titan ~# date
>> Thu Feb 18 17:35:24 UTC 2010
>>
>> root@titan ~# ./conntrack -F conntrack
>> conntrack v0.9.14 (conntrack-tools): connection tracking table has been emptied.
>>
>> root@titan ~# date
>> Thu Feb 18 17:35:31 UTC 2010
>>
>> root@titan ~# ./conntrack -C conntrack
>> 351
>>
>> root@titan ~# date
>> Thu Feb 18 17:35:36 UTC 2010
>>
>> Shouldn't the value after the flush be 0? The traffic that has created
>> this mess is from a REDIRECT rule in the PREROUTING chain of the 'nat'
>> table.
>
> Could you post a copy of these rules ?
>
> Thanks
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: nf_conntrack_count versus '/proc/net/nf_conntrack | wc -l' count

[Afi Gjermund]

On Thu, Feb 18, 2010 at 10:07 AM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> Le jeudi 18 février 2010 à 09:55 -0800, Afi Gjermund a écrit :
>> On Thu, Feb 18, 2010 at 9:51 AM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
>> > Le jeudi 18 février 2010 à 09:40 -0800, Afi Gjermund a écrit :
>> >> I am still trying to figure out why the nf_conntrack_count differs
>> >> from the table system.  I decided I would use the conntrack userspace
>> >> tools.
>> >> Both of my NICs are unplugged with no other userspace applications
>> >> running to affect connection tracking counts.
>> >>
>> >>
>> >> root@titan ~# date
>> >> Thu Feb 18 17:35:21 UTC 2010
>> >>
>> >> root@titan ~# ./conntrack -C conntrack
>> >> 351
>> >>
>> >> root@titan ~# date
>> >> Thu Feb 18 17:35:24 UTC 2010
>> >>
>> >> root@titan ~# ./conntrack -F conntrack
>> >> conntrack v0.9.14 (conntrack-tools): connection tracking table has been emptied.
>> >>
>> >> root@titan ~# date
>> >> Thu Feb 18 17:35:31 UTC 2010
>> >>
>> >> root@titan ~# ./conntrack -C conntrack
>> >> 351
>> >>
>> >> root@titan ~# date
>> >> Thu Feb 18 17:35:36 UTC 2010
>> >>
>> >> Shouldn't the value after the flush be 0? The traffic that has created
>> >> this mess is from a REDIRECT rule in the PREROUTING chain of the 'nat'
>> >> table.
>> >
>> > Could you post a copy of these rules ?
>> >
>> > Thanks
>> >
>> >
>> >
>> iptables -t nat -A PREROUTING -p tcp -s X.X.X.X -d X.X.X.X --sport X
>> --dport X -j REDIRECT --to-port X
>
> Yes I understood you were using such rules, but I cannot understand how
> it can trigger without real nics being plugged. So I asked you some
> details, apprently you dont want to provide them and prefer to hide from
> us :)
>
>
>
>
>
Lol, sorry. The X values are dynamic and depend on what network the
device happens to be on, as well as the ephemeral source port.

iptables -t nat -A PREROUTING -p tcp -s 172.168.8.45 -d 172.168.8.200
--sport 4351 --dport 4500 -j REDIRECT --to-port 45001
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: nf_conntrack_count versus '/proc/net/nf_conntrack | wc -l' count

[Patrick McHardy]

Afi Gjermund wrote:
> On Thu, Feb 18, 2010 at 10:07 AM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
>>>>> Shouldn't the value after the flush be 0? The traffic that has created
>>>>> this mess is from a REDIRECT rule in the PREROUTING chain of the 'nat'
>>>>> table.
>>>> Could you post a copy of these rules ?
>>>>
>>> iptables -t nat -A PREROUTING -p tcp -s X.X.X.X -d X.X.X.X --sport X
>>> --dport X -j REDIRECT --to-port X
>> Yes I understood you were using such rules, but I cannot understand how
>> it can trigger without real nics being plugged. So I asked you some
>> details, apprently you dont want to provide them and prefer to hide from
>> us :)
>>
> Lol, sorry. The X values are dynamic and depend on what network the
> device happens to be on, as well as the ephemeral source port.
> 
> iptables -t nat -A PREROUTING -p tcp -s 172.168.8.45 -d 172.168.8.200
> --sport 4351 --dport 4500 -j REDIRECT --to-port 45001

NAT is unlikely to be the cause since its widely used and there
are no other reports of leaks. Please describe your full setup,
especially things like traffic scheduling, network devices,
userspace queueing etc etc.

Re: nf_conntrack_count versus '/proc/net/nf_conntrack | wc -l' count

[Patrick McHardy]

Douglas Diniz wrote:
> I'm facing the same problem. I'm working in a embedded system with
> kernel 2.6.20-6. When I send a ping (or any other protocol) through
> eth0 to eth1 (or vice versa) the conntrack count isn't decremented. If
> I send the ping through any other interface (eth0 to wifi, eth1 to
> wifi, wifi to eth0 and wifi to eth1) I have no problem.
> The problem seems to be only between the ethernet interfaces.
> I debug the netfilter and I saw that when the problem occurs the "use"
> variable inside conntract structure in > 1, so this variable is only
> decremented by 1, not reaching in 0, and then the destroy_conntrack
> function is not called.
> 
> So I think that the problem is more low level, and some events aren't
> reaching netfilter, and the "use"variable isn't decremented properly.
> 
> This could be a problem with the ethernet driver?

Yes, although you'd likely notice other effects like running
out of memory if it was leaking the packets.

Re: nf_conntrack_count versus '/proc/net/nf_conntrack | wc -l' count

[Douglas Diniz]

I did some tests removing all nat modules, leaving only the conntrack
core module and the problem still occurs.

I'm working in this problem for more than a week I stiil didnt found the cause.

On Thu, Feb 18, 2010 at 4:22 PM, Patrick McHardy <kaber@trash.net> wrote:
> Douglas Diniz wrote:
>> I'm facing the same problem. I'm working in a embedded system with
>> kernel 2.6.20-6. When I send a ping (or any other protocol) through
>> eth0 to eth1 (or vice versa) the conntrack count isn't decremented. If
>> I send the ping through any other interface (eth0 to wifi, eth1 to
>> wifi, wifi to eth0 and wifi to eth1) I have no problem.
>> The problem seems to be only between the ethernet interfaces.
>> I debug the netfilter and I saw that when the problem occurs the "use"
>> variable inside conntract structure in > 1, so this variable is only
>> decremented by 1, not reaching in 0, and then the destroy_conntrack
>> function is not called.
>>
>> So I think that the problem is more low level, and some events aren't
>> reaching netfilter, and the "use"variable isn't decremented properly.
>>
>> This could be a problem with the ethernet driver?
>
> Yes, although you'd likely notice other effects like running
> out of memory if it was leaking the packets.
>

Re: nf_conntrack_count versus '/proc/net/nf_conntrack | wc -l' count

[Afi Gjermund]

On Thu, Feb 18, 2010 at 10:19 AM, Patrick McHardy <kaber@trash.net> wrote:
> Afi Gjermund wrote:
>> On Thu, Feb 18, 2010 at 10:07 AM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
>>>>>> Shouldn't the value after the flush be 0? The traffic that has created
>>>>>> this mess is from a REDIRECT rule in the PREROUTING chain of the 'nat'
>>>>>> table.
>>>>> Could you post a copy of these rules ?
>>>>>
>>>> iptables -t nat -A PREROUTING -p tcp -s X.X.X.X -d X.X.X.X --sport X
>>>> --dport X -j REDIRECT --to-port X
>>> Yes I understood you were using such rules, but I cannot understand how
>>> it can trigger without real nics being plugged. So I asked you some
>>> details, apprently you dont want to provide them and prefer to hide from
>>> us :)
>>>
>> Lol, sorry. The X values are dynamic and depend on what network the
>> device happens to be on, as well as the ephemeral source port.
>>
>> iptables -t nat -A PREROUTING -p tcp -s 172.168.8.45 -d 172.168.8.200
>> --sport 4351 --dport 4500 -j REDIRECT --to-port 45001
>
> NAT is unlikely to be the cause since its widely used and there
> are no other reports of leaks. Please describe your full setup,
> especially things like traffic scheduling, network devices,
> userspace queueing etc etc.
>

The device has 2 network interfaces that are configured in a bridge
(eth0,eth1).  The traffic scheduling has not been changed from the
default kernel configuration.

Problem path:
The problem I am seeing is that my tcp connections enter the
/proc/net/nf_conntrack table, then disappear over time but the
nf_conntrack_count never decreases.  Over time, the nf_conntrack_count
hits the 4096 nf_conntrack_max and the kernel begins to drop packets
even though the /proc/net/nf_conntrack table is not full (has < 100
connections).

In testing I decided to set the nf_conntrack_max to 100, and fill the
table via the connections above.  Then remove both ethernet cables to
ensure no new connections could be made.  I also set the
nf_conntrack_tcp_timeout_established to 60 seconds.  I left this for 2
hours and saw that the /proc/net/nf_conntrack table was empty while
the nf_conntrack_count was still 100.

I also created a kernel module that calls the nf_conntrack_flush()
function, this seems to only clear the /proc/net/nf_conntrack table,
but not the count. If I also do an atomic_set(&nf_conntrack_count,0)
then (obviously) the count becomes 0.  It is as if the connections are
being removed from the table, but the count is not being decremented,
which I am not sure why.  As far as I understand it, they should be in
sync.

Re: nf_conntrack_count versus '/proc/net/nf_conntrack | wc -l' count

[Afi Gjermund]

On Thu, Feb 18, 2010 at 11:39 AM, Afi Gjermund <afigjermund@gmail.com> wrote:
> On Thu, Feb 18, 2010 at 10:19 AM, Patrick McHardy <kaber@trash.net> wrote:
>> Afi Gjermund wrote:
>>> On Thu, Feb 18, 2010 at 10:07 AM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
>>>>>>> Shouldn't the value after the flush be 0? The traffic that has created
>>>>>>> this mess is from a REDIRECT rule in the PREROUTING chain of the 'nat'
>>>>>>> table.
>>>>>> Could you post a copy of these rules ?
>>>>>>
>>>>> iptables -t nat -A PREROUTING -p tcp -s X.X.X.X -d X.X.X.X --sport X
>>>>> --dport X -j REDIRECT --to-port X
>>>> Yes I understood you were using such rules, but I cannot understand how
>>>> it can trigger without real nics being plugged. So I asked you some
>>>> details, apprently you dont want to provide them and prefer to hide from
>>>> us :)
>>>>
>>> Lol, sorry. The X values are dynamic and depend on what network the
>>> device happens to be on, as well as the ephemeral source port.
>>>
>>> iptables -t nat -A PREROUTING -p tcp -s 172.168.8.45 -d 172.168.8.200
>>> --sport 4351 --dport 4500 -j REDIRECT --to-port 45001
>>
>> NAT is unlikely to be the cause since its widely used and there
>> are no other reports of leaks. Please describe your full setup,
>> especially things like traffic scheduling, network devices,
>> userspace queueing etc etc.
>>
>
> The device has 2 network interfaces that are configured in a bridge
> (eth0,eth1).  The traffic scheduling has not been changed from the
> default kernel configuration.
>
> Problem path:
> The problem I am seeing is that my tcp connections enter the
> /proc/net/nf_conntrack table, then disappear over time but the
> nf_conntrack_count never decreases.  Over time, the nf_conntrack_count
> hits the 4096 nf_conntrack_max and the kernel begins to drop packets
> even though the /proc/net/nf_conntrack table is not full (has < 100
> connections).
>
> In testing I decided to set the nf_conntrack_max to 100, and fill the
> table via the connections above.  Then remove both ethernet cables to
> ensure no new connections could be made.  I also set the
> nf_conntrack_tcp_timeout_established to 60 seconds.  I left this for 2
> hours and saw that the /proc/net/nf_conntrack table was empty while
> the nf_conntrack_count was still 100.
>
> I also created a kernel module that calls the nf_conntrack_flush()
> function, this seems to only clear the /proc/net/nf_conntrack table,
> but not the count. If I also do an atomic_set(&nf_conntrack_count,0)
> then (obviously) the count becomes 0.  It is as if the connections are
> being removed from the table, but the count is not being decremented,
> which I am not sure why.  As far as I understand it, they should be in
> sync.
>

I have found the issue that was causing this problem.  A userspace
application that was using the NFQueue mechanism to queue data to
userspace was returning a verdict of STOLEN on the first UDP packet
seen.  This appears to have been leaving entries in the connection
table that could not be flushed via nf_conntrack_flush().  When
changing the verdict to DROP, the problem no longer existed.

This was found as I noticed the Timer value of the connections within
the table remained at 3000 (30 in nf_conntrack_udp_timeout x 100).

Feb 18 22:56:31 titan user.info kernel:  ===========================
Table Dump =========================
Feb 18 22:56:31 titan user.info kernel:  ---- Set ----
Feb 18 22:56:31 titan user.info kernel: Timer is : 3000
Feb 18 22:56:31 titan user.info kernel:  tuple dump: IP_CT_DIR_ORIGINAL
Feb 18 22:56:31 titan user.info kernel:
Feb 18 22:56:31 titan user.warn kernel:  tuple c321cc70: l3num 2
protonum 17  srcIP 172.16.8.45 srcPort 4858  -> dstIP 172.16.8.7
dstPort 45001
Feb 18 22:56:31 titan user.info kernel:  tuple dump: IP_CT_DIR_REPLY
Feb 18 22:56:31 titan user.info kernel:
Feb 18 22:56:31 titan user.warn kernel:  tuple c321cca8: l3num 2
protonum 17  srcIP 172.16.8.7 srcPort 45001  -> dstIP 172.16.8.45
dstPort 4858
Feb 18 22:56:31 titan user.info kernel:  ---- End Set ----
Feb 18 22:56:31 titan user.info kernel:  ===========================
End Table Dump =========================
Feb 18 22:57:03 titan user.info kernel:  ===========================
Table Dump =========================
Feb 18 22:57:03 titan user.info kernel:  ---- Set ----
Feb 18 22:57:03 titan user.info kernel: Timer is : 3000
Feb 18 22:57:03 titan user.info kernel:  tuple dump: IP_CT_DIR_ORIGINAL
Feb 18 22:57:03 titan user.info kernel:
Feb 18 22:57:03 titan user.warn kernel:  tuple c321cc70: l3num 2
protonum 17  srcIP 172.16.8.45 srcPort 4858  -> dstIP 172.16.8.7
dstPort 45001
Feb 18 22:57:03 titan user.info kernel:  tuple dump: IP_CT_DIR_REPLY
Feb 18 22:57:03 titan user.info kernel:
Feb 18 22:57:03 titan user.warn kernel:  tuple c321cca8: l3num 2
protonum 17  srcIP 172.16.8.7 srcPort 45001  -> dstIP 172.16.8.45
dstPort 4858
Feb 18 22:57:03 titan user.info kernel:  ---- End Set ----
Feb 18 22:57:03 titan user.info kernel:  ===========================
End Table Dump =========================

Thank you all for your help!  Hopefully this may help other people as well.

Afi

Re: nf_conntrack_count versus '/proc/net/nf_conntrack | wc -l' count

[Eric Dumazet]

Le jeudi 18 février 2010 à 16:53 -0800, Afi Gjermund a écrit :

> I have found the issue that was causing this problem.  A userspace
> application that was using the NFQueue mechanism to queue data to
> userspace was returning a verdict of STOLEN on the first UDP packet
> seen.  This appears to have been leaving entries in the connection
> table that could not be flushed via nf_conntrack_flush().  When
> changing the verdict to DROP, the problem no longer existed.
> 
> This was found as I noticed the Timer value of the connections within
> the table remained at 3000 (30 in nf_conntrack_udp_timeout x 100).
> 
> Feb 18 22:56:31 titan user.info kernel:  ===========================
> Table Dump =========================
> Feb 18 22:56:31 titan user.info kernel:  ---- Set ----
> Feb 18 22:56:31 titan user.info kernel: Timer is : 3000
> Feb 18 22:56:31 titan user.info kernel:  tuple dump: IP_CT_DIR_ORIGINAL
> Feb 18 22:56:31 titan user.info kernel:
> Feb 18 22:56:31 titan user.warn kernel:  tuple c321cc70: l3num 2
> protonum 17  srcIP 172.16.8.45 srcPort 4858  -> dstIP 172.16.8.7
> dstPort 45001
> Feb 18 22:56:31 titan user.info kernel:  tuple dump: IP_CT_DIR_REPLY
> Feb 18 22:56:31 titan user.info kernel:
> Feb 18 22:56:31 titan user.warn kernel:  tuple c321cca8: l3num 2
> protonum 17  srcIP 172.16.8.7 srcPort 45001  -> dstIP 172.16.8.45
> dstPort 4858
> Feb 18 22:56:31 titan user.info kernel:  ---- End Set ----
> Feb 18 22:56:31 titan user.info kernel:  ===========================
> End Table Dump =========================
> Feb 18 22:57:03 titan user.info kernel:  ===========================
> Table Dump =========================
> Feb 18 22:57:03 titan user.info kernel:  ---- Set ----
> Feb 18 22:57:03 titan user.info kernel: Timer is : 3000
> Feb 18 22:57:03 titan user.info kernel:  tuple dump: IP_CT_DIR_ORIGINAL
> Feb 18 22:57:03 titan user.info kernel:
> Feb 18 22:57:03 titan user.warn kernel:  tuple c321cc70: l3num 2
> protonum 17  srcIP 172.16.8.45 srcPort 4858  -> dstIP 172.16.8.7
> dstPort 45001
> Feb 18 22:57:03 titan user.info kernel:  tuple dump: IP_CT_DIR_REPLY
> Feb 18 22:57:03 titan user.info kernel:
> Feb 18 22:57:03 titan user.warn kernel:  tuple c321cca8: l3num 2
> protonum 17  srcIP 172.16.8.7 srcPort 45001  -> dstIP 172.16.8.45
> dstPort 4858
> Feb 18 22:57:03 titan user.info kernel:  ---- End Set ----
> Feb 18 22:57:03 titan user.info kernel:  ===========================
> End Table Dump =========================
> 
> Thank you all for your help!  Hopefully this may help other people as well.
> 
> Afi

Thanks Afi for providing us more info :)

Patrick, If a user application asks NF_STOLEN, we leak the skb.
As the entry is freed, there is no way this skb can be found again.

What do you think of following patch ?
Or should we ignore NF_STOLEN status from user, to let packet still
queued ?

Thanks

[PATCH] nf_queue: fix NF_STOLEN skb leak

commit 3bc38712e3a6e059 (handle NF_STOP and unknown verdicts in
nf_reinject) was a partial fix to packet leaks.

If user asks NF_STOLEN status, we must free the skb as well.

Reported-by: Afi Gjermund <afigjermund@gmail.com>
Signed-off-by: Eric DUmazet <eric.dumazet@gmail.com>
---
diff --git a/net/netfilter/nf_queue.c b/net/netfilter/nf_queue.c
index 3a6fd77..ba095fd 100644
--- a/net/netfilter/nf_queue.c
+++ b/net/netfilter/nf_queue.c
@@ -265,7 +265,6 @@ void nf_reinject(struct nf_queue_entry *entry, unsigned int verdict)
 		local_bh_disable();
 		entry->okfn(skb);
 		local_bh_enable();
-	case NF_STOLEN:
 		break;
 	case NF_QUEUE:
 		if (!__nf_queue(skb, elem, entry->pf, entry->hook,
@@ -273,6 +272,7 @@ void nf_reinject(struct nf_queue_entry *entry, unsigned int verdict)
 				verdict >> NF_VERDICT_BITS))
 			goto next_hook;
 		break;
+	case NF_STOLEN:
 	default:
 		kfree_skb(skb);
 	}


--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: nf_conntrack_count versus '/proc/net/nf_conntrack | wc -l' count

[Patrick McHardy]

Eric Dumazet wrote:
> Le jeudi 18 février 2010 à 16:53 -0800, Afi Gjermund a écrit :
> 
> Thanks Afi for providing us more info :)
> 
> Patrick, If a user application asks NF_STOLEN, we leak the skb.
> As the entry is freed, there is no way this skb can be found again.
> 
> What do you think of following patch ?
> Or should we ignore NF_STOLEN status from user, to let packet still
> queued ?

I think dropping the packet is the expected behaviour.

> [PATCH] nf_queue: fix NF_STOLEN skb leak
> 
> commit 3bc38712e3a6e059 (handle NF_STOP and unknown verdicts in
> nf_reinject) was a partial fix to packet leaks.
> 
> If user asks NF_STOLEN status, we must free the skb as well.
> 
>

Applied, thanks Eric.
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html