Re: [PATCH] add TCP protocol state event groups

[Pablo Neira Ayuso <pablo@netfilter.org>]

Patrick McHardy wrote:
> Pablo Neira Ayuso wrote:
>> Patrick McHardy wrote:
>>
>>>> Well, why just save a couple of groups if we've got 2^32 event groups?
>>>> Moreover, per protocol state seems to me the most fine-grain and
>>>> flexible solution. Depending on the replication schema I might be
>>>> interested in different states.
>>>
>>> Its not only about saving groups. A scheme like this only makes
>>> sense if you introduce groups for every tiny bit, otherwise you
>>> need to subscribe to the "global" group anyway to get the remaining
>>> "unclassified" events you're interested in. And that not only uses
>>> a lot of groups, it also requires dispatching the same event to
>>> potentially many groups. I'm interested, do you already use this
>>> feature in conntrackd? If yes, how do you deal with UDP etc. that
>>> you didn't introduce new groups for?
> 
> You skipped my question.

Sorry, I'm lazy :-)

>> The patch is incomplete but you can guess the schema. Consider a patch
>> to add the following groups:
>>
>> NFNLGRP_CONNTRACK_PROTO_UDP_NEW,
>> NFNLGRP_CONNTRACK_PROTO_UDP_UPDATE,
>> NFNLGRP_CONNTRACK_PROTO_UDP_DESTROY,
>> NFNLGRP_CONNTRACK_PROTO_ICMP_NEW,
>> NFNLGRP_CONNTRACK_PROTO_ICMP_UPDATE,
>> NFNLGRP_CONNTRACK_PROTO_ICMP_DESTROY,
>> NFNLGRP_CONNTRACK_PROTO_SCTP_...,
>> NFNLGRP_CONNTRACK_PROTO_UNCLASSIFIED_NEW,
>> NFNLGRP_CONNTRACK_PROTO_UNCLASSIFIED_UPDATE,
>> NFNLGRP_CONNTRACK_PROTO_UNCLASSIFIED_DESTROY
>>
>> At a glance I see the function netlink_update_listeners that is O(n),
>> that could be the bottleneck if we have a lot of groups. Still
>> objections with this approach?
> 
> So far I presume conntrackd still listens to the global group. So
> I would like to know how many groups we'll end up with until this
> scheme is really fine-grained enough that it allows conntrackd
> to avoid listening to the global group and what types of events
> it would ignore (so I can understand the performance improvement
> this might yield). The above still only contains protocol specific
> groups.

OK, let me develop the issue a bit more. In my initial experiments, I
used the patch to replicate only TCP connections, so conntrackd was not
listening the global group but just TCP state events. I currently have a
clause that looks like:

Replicate ESTABLISHED TIME_WAIT for TCP

Thus, only established and time_wait states are replicated. My plan is
to add similar clauses for other protocols so:

Replicate ESTABLISHED for STCP
Replicate NEW DESTROY for UDP
Replica NEW DESTROY for UNCLASSIFIED

The sysadmin manually may tune the states that it wants to replicate to
reduce the CPU overhead.

-- 
The dawn of the fourth age of Linux firewalling is coming; a time of
great struggle and heroic deeds -- J.Kadlecsik got inspired by J.Morris