[0/3] conntrack event kernel issues

[0/3] conntrack event kernel issues

[Fabian Hugelshofer]

Hi,

I plan to use netfilter conntrack events to count the number of
connections which timed out without having received a reply. To make
this as efficient as possible I only want to monitor destroy events. I
have noticed three issues with that and wrote patches to solve them.
They follow in separate mails.

Please let me know what you think about them and if they could be useful
for other people as well.

patch1: export ct->status on all conntrack events
patch2: set SEEN_REPLY before destroying a conntrack on TCP RST
patch3: new status flag SEEN_RELATED


Regards,

Fabian

Re: [0/3] conntrack event kernel issues

[Patrick McHardy]

Fabian Hugelshofer wrote:
> Hi,
> 
> I plan to use netfilter conntrack events to count the number of
> connections which timed out without having received a reply. To make
> this as efficient as possible I only want to monitor destroy events. I
> have noticed three issues with that and wrote patches to solve them.
> They follow in separate mails.
> 
> Please let me know what you think about them and if they could be useful
> for other people as well.
> 
> patch1: export ct->status on all conntrack events
> patch2: set SEEN_REPLY before destroying a conntrack on TCP RST
> patch3: new status flag SEEN_RELATED

I can't imagine other uses for this than the one you described,
especially for 2 and 3. Patch 3 also adds code in a hot path,
so unless someone can present good arguments in favour of these
patches, I don't really want to apply them.

Re: [0/3] conntrack event kernel issues

[Fabian Hugelshofer]

Patrick McHardy wrote:
>> patch1: export ct->status on all conntrack events
>> patch2: set SEEN_REPLY before destroying a conntrack on TCP RST
>> patch3: new status flag SEEN_RELATED
> 
> I can't imagine other uses for this than the one you described,
> especially for 2 and 3. Patch 3 also adds code in a hot path,
> so unless someone can present good arguments in favour of these
> patches, I don't really want to apply them.

This is what I had expected. Especially for patch 3 I know that it's 
very unlikely to be integrated because of its limited use and the 
changes it makes.

For not exporting the connection status on a destroy event I see no 
reason. The information is there and should be exported. Might also be 
interesting to have the EXPECTED or ASSURED flags.

Then there is this thing with the TCP RST. I think, that the event data 
should be accurate. If the status is returned (with patch 1), then it 
should have the SEEN_REPLY flag set. Another issue is that the 
accounting counters are not updated. IMHO this should be done as well 
(is not in my patches).

Re: [0/3] conntrack event kernel issues

[Patrick McHardy]

Fabian Hugelshofer wrote:
> Patrick McHardy wrote:
>>> patch1: export ct->status on all conntrack events
>>> patch2: set SEEN_REPLY before destroying a conntrack on TCP RST
>>> patch3: new status flag SEEN_RELATED
>>
>> I can't imagine other uses for this than the one you described,
>> especially for 2 and 3. Patch 3 also adds code in a hot path,
>> so unless someone can present good arguments in favour of these
>> patches, I don't really want to apply them.
> 
> This is what I had expected. Especially for patch 3 I know that it's 
> very unlikely to be integrated because of its limited use and the 
> changes it makes.
> 
> For not exporting the connection status on a destroy event I see no 
> reason. The information is there and should be exported. Might also be 
> interesting to have the EXPECTED or ASSURED flags.

Yes, that one I'm fine with.

> Then there is this thing with the TCP RST. I think, that the event data 
> should be accurate. If the status is returned (with patch 1), then it 
> should have the SEEN_REPLY flag set. Another issue is that the 
> accounting counters are not updated. IMHO this should be done as well 
> (is not in my patches).

Fully agreed about the counters. About the SEEN_REPLY bit - that
depends on how you define its meaning. So far its only set if
a valid reply for the connection is seen - which a RST isn't.

Re: [0/3] conntrack event kernel issues

[Pablo Neira Ayuso]

Patrick McHardy wrote:
> Fabian Hugelshofer wrote:
>> Patrick McHardy wrote:
>>>> patch1: export ct->status on all conntrack events
>>>> patch2: set SEEN_REPLY before destroying a conntrack on TCP RST
>>>> patch3: new status flag SEEN_RELATED
>>>
>>> I can't imagine other uses for this than the one you described,
>>> especially for 2 and 3. Patch 3 also adds code in a hot path,
>>> so unless someone can present good arguments in favour of these
>>> patches, I don't really want to apply them.
>>
>> This is what I had expected. Especially for patch 3 I know that it's 
>> very unlikely to be integrated because of its limited use and the 
>> changes it makes.
>>
>> For not exporting the connection status on a destroy event I see no 
>> reason. The information is there and should be exported. Might also be 
>> interesting to have the EXPECTED or ASSURED flags.
> 
> Yes, that one I'm fine with.

Indeed. It makes sense.

>> Then there is this thing with the TCP RST. I think, that the event 
>> data should be accurate. If the status is returned (with patch 1), 
>> then it should have the SEEN_REPLY flag set. Another issue is that the 
>> accounting counters are not updated. IMHO this should be done as well 
>> (is not in my patches).
> 
> Fully agreed about the counters. About the SEEN_REPLY bit - that
> depends on how you define its meaning. So far its only set if
> a valid reply for the connection is seen - which a RST isn't.

I'm bit lost about the thing related with the counters, what do you mean?

-- 
"Los honestos son inadaptados sociales" -- Les Luthiers

Re: [0/3] conntrack event kernel issues

[Pablo Neira Ayuso]

Pablo Neira Ayuso wrote:
> Patrick McHardy wrote:
>> Fabian Hugelshofer wrote:
>>> Patrick McHardy wrote:
>>>>> patch1: export ct->status on all conntrack events
>>>>> patch2: set SEEN_REPLY before destroying a conntrack on TCP RST
>>>>> patch3: new status flag SEEN_RELATED
>>>>
>>>> I can't imagine other uses for this than the one you described,
>>>> especially for 2 and 3. Patch 3 also adds code in a hot path,
>>>> so unless someone can present good arguments in favour of these
>>>> patches, I don't really want to apply them.
>>>
>>> This is what I had expected. Especially for patch 3 I know that it's 
>>> very unlikely to be integrated because of its limited use and the 
>>> changes it makes.
>>>
>>> For not exporting the connection status on a destroy event I see no 
>>> reason. The information is there and should be exported. Might also 
>>> be interesting to have the EXPECTED or ASSURED flags.
>>
>> Yes, that one I'm fine with.
> 
> Indeed. It makes sense.
> 
>>> Then there is this thing with the TCP RST. I think, that the event 
>>> data should be accurate. If the status is returned (with patch 1), 
>>> then it should have the SEEN_REPLY flag set. Another issue is that 
>>> the accounting counters are not updated. IMHO this should be done as 
>>> well (is not in my patches).
>>
>> Fully agreed about the counters. About the SEEN_REPLY bit - that
>> depends on how you define its meaning. So far its only set if
>> a valid reply for the connection is seen - which a RST isn't.
> 
> I'm bit lost about the thing related with the counters, what do you mean?

Oh! I see the TCP RST.

-- 
"Los honestos son inadaptados sociales" -- Les Luthiers

Re: [0/3] conntrack event kernel issues

[Fabian Hugelshofer]

Patrick McHardy wrote:
>> Then there is this thing with the TCP RST. I think, that the event 
>> data should be accurate. If the status is returned (with patch 1), 
>> then it should have the SEEN_REPLY flag set. Another issue is that the 
>> accounting counters are not updated. IMHO this should be done as well 
>> (is not in my patches).
> 
> Fully agreed about the counters. About the SEEN_REPLY bit - that
> depends on how you define its meaning. So far its only set if
> a valid reply for the connection is seen - which a RST isn't.

I consider a RST as a valid reply for a connection. It matches the tuple 
and is in reply direction (ctinfo is set to IS_REPLY). Further the TCP 
protocol handler returns no error. The SEEN_REPLY bit is actually set at 
the end of nf_conntrack_in() but the destroy event is already triggered 
by the TCP handler.

Increasing the counters in reply direction without setting the 
SEEN_REPLY bit seems a bit weird to me.

Re: [0/3] conntrack event kernel issues

[Patrick McHardy]

Fabian Hugelshofer wrote:
> Patrick McHardy wrote:
>>> Then there is this thing with the TCP RST. I think, that the event 
>>> data should be accurate. If the status is returned (with patch 1), 
>>> then it should have the SEEN_REPLY flag set. Another issue is that 
>>> the accounting counters are not updated. IMHO this should be done as 
>>> well (is not in my patches).
>>
>> Fully agreed about the counters. About the SEEN_REPLY bit - that
>> depends on how you define its meaning. So far its only set if
>> a valid reply for the connection is seen - which a RST isn't.
> 
> I consider a RST as a valid reply for a connection. It matches the tuple 
> and is in reply direction (ctinfo is set to IS_REPLY). Further the TCP 
> protocol handler returns no error. The SEEN_REPLY bit is actually set at 
> the end of nf_conntrack_in() but the destroy event is already triggered 
> by the TCP handler.
> 
> Increasing the counters in reply direction without setting the 
> SEEN_REPLY bit seems a bit weird to me.


OK, you've convinced me. If you send me a patch to update
the counters properly and set the SEEN_REPLY bit, I'll
apply it.