nfnetlink and conntrack extension question

nfnetlink and conntrack extension question

[KOVACS Krisztian]

Dear Netfilter developers,

We're working on releasing a GPL version of our modular proxy software
again, this time complete with our custom iptables target and matcher
modules that implement much of our policy evaluation in iptables. (Think
something like being able to define sets of rules that are evaluated
using a best-match algorithm instead of linear evaluation like iptables
does.)

The kernel space is obviously under GPL, and we'd be very glad to make
it as easy to install as possible -- having to apply kernel patches and
recompile your distro kernel is out of question for most of the
potential users.

Unfortunately upstream submission is not really an option, since the
module itself is very much tied to the userspace and there's really no
point in having it in the kernel unless you want to use Zorp. (I thought
a lot about potential use cases without the userspace proxy, but really
couldn't come up with anything meaningful.)

There are two things remaining that prevent us doing simple out-of-tree
kernel module builds:

     1. We use nfnetlink for the userspace->kernelspace communication.
        This works beautifully, however, since NFNL_SUBSYS_COUNT is a
        compile-time constant there's no way of registering a subsystem
        with an ID not known at compile time.
     2. Pretty much the same happens with conntrack extensions: we use a
        conntrack extension to store cached results of policy
        evaluation. Unfortunatly here we have an NF_CT_EXT_NUM sized
        static array storing the registered conntrack extensions.

My question is whether or not removing those limitations and allowing
runtime registration of both nfnetlink subsystems and conntrack
extensions would be acceptable upstream? That way out-of-tree modules
could possibly use those features without having to resort to patching
and recompiling the kernel.

TiA,

-- 
KOVACS Krisztian

Re: nfnetlink and conntrack extension question

[Jan Engelhardt]

On Wednesday 2011-11-30 16:54, KOVACS Krisztian wrote:
>
>There are two things remaining that prevent us doing simple out-of-tree
>kernel module builds:
>
>     1. We use nfnetlink for the userspace->kernelspace communication.
>        This works beautifully, however, since NFNL_SUBSYS_COUNT is a
>        compile-time constant there's no way of registering a subsystem
>        with an ID not known at compile time.

You can't even snatch a number because the arrays are just sized
__foobar_max-ly, and those may all be used.

>My question is whether or not removing those limitations and allowing
>runtime registration of both nfnetlink subsystems and conntrack
>extensions would be acceptable upstream? That way out-of-tree modules
>could possibly use those features without having to resort to patching
>and recompiling the kernel.

As for 1, you can use genetlink, just as I do for the copy of ipset
in xtables-addons. Being forced to use nfnetlink has been point of
much discussion and ultimately, nobody was able to provide a
technical reason on why nfnetlink is better.

The last argument made by nfnl proponents was that NETLINK_NETFILTER
was free of multicast messages from other uninteresting subsystems
(e.g. block layer), but I am questioning on whether genl really
forces you to receive (and then ignore) uninteresting mcast messages,
given one has to explicitly subscribe to nlgroups in the first place
anyway.

Re: nfnetlink and conntrack extension question

[Pablo Neira Ayuso]

On Wed, Nov 30, 2011 at 04:54:13PM +0100, KOVACS Krisztian wrote:
> Dear Netfilter developers,
> 
> We're working on releasing a GPL version of our modular proxy software
> again, this time complete with our custom iptables target and matcher
> modules that implement much of our policy evaluation in iptables. (Think
> something like being able to define sets of rules that are evaluated
> using a best-match algorithm instead of linear evaluation like iptables
> does.)
> 
> The kernel space is obviously under GPL, and we'd be very glad to make
> it as easy to install as possible -- having to apply kernel patches and
> recompile your distro kernel is out of question for most of the
> potential users.
> 
> Unfortunately upstream submission is not really an option, since the
> module itself is very much tied to the userspace and there's really no
> point in having it in the kernel unless you want to use Zorp. (I thought
> a lot about potential use cases without the userspace proxy, but really
> couldn't come up with anything meaningful.)
> 
> There are two things remaining that prevent us doing simple out-of-tree
> kernel module builds:
> 
>      1. We use nfnetlink for the userspace->kernelspace communication.
>         This works beautifully, however, since NFNL_SUBSYS_COUNT is a
>         compile-time constant there's no way of registering a subsystem
>         with an ID not known at compile time.
>      2. Pretty much the same happens with conntrack extensions: we use a
>         conntrack extension to store cached results of policy
>         evaluation. Unfortunatly here we have an NF_CT_EXT_NUM sized
>         static array storing the registered conntrack extensions.
> 
> My question is whether or not removing those limitations and allowing
> runtime registration of both nfnetlink subsystems and conntrack
> extensions would be acceptable upstream? That way out-of-tree modules
> could possibly use those features without having to resort to patching
> and recompiling the kernel.

My opinion: I think that it is hard to justify any sort of change in
mainstream to make life easier to out-of-tree modules.

We want things in mainstream.

P.S: Any chance to integrate your development with nfgrep? I didn't
find any time to release the source code yet. I expect to find some
spare time during the end of year holidays. If I have access to that
GPL code we can think of some sort of integration nfgrep <-> zorp.

Re: nfnetlink and conntrack extension question

[Pablo Neira Ayuso]

On Wed, Nov 30, 2011 at 06:05:13PM +0100, Jan Engelhardt wrote:
> 
> On Wednesday 2011-11-30 16:54, KOVACS Krisztian wrote:
> >
> >There are two things remaining that prevent us doing simple out-of-tree
> >kernel module builds:
> >
> >     1. We use nfnetlink for the userspace->kernelspace communication.
> >        This works beautifully, however, since NFNL_SUBSYS_COUNT is a
> >        compile-time constant there's no way of registering a subsystem
> >        with an ID not known at compile time.
> 
> You can't even snatch a number because the arrays are just sized
> __foobar_max-ly, and those may all be used.
> 
> >My question is whether or not removing those limitations and allowing
> >runtime registration of both nfnetlink subsystems and conntrack
> >extensions would be acceptable upstream? That way out-of-tree modules
> >could possibly use those features without having to resort to patching
> >and recompiling the kernel.
> 
> As for 1, you can use genetlink, just as I do for the copy of ipset
> in xtables-addons. Being forced to use nfnetlink has been point of
> much discussion and ultimately, nobody was able to provide a
> technical reason on why nfnetlink is better.

Well, few differences. With genetlink:

* you have to send a message to look up for the ID first (to guess the
  multicast group and subsystem IDs).

* you don't know how many users will using the genetlink bus. You'll
  have to share the bandwidth with them.

And to finish, for netfilter, the policy is to use the NETLINK_NEFILTER
bus which is where *all* netfilter subsystem reside.

genetlink is a bit more bloated than nfnetlink IMO.

I like the idea that you cannot register out-of-tree modules using
nfnetlink. I like things in mainstream (or look for some alternative
to get them it).

> The last argument made by nfnl proponents was that NETLINK_NETFILTER
> was free of multicast messages from other uninteresting subsystems
> (e.g. block layer), but I am questioning on whether genl really
> forces you to receive (and then ignore) uninteresting mcast messages,
> given one has to explicitly subscribe to nlgroups in the first place
> anyway.

No, if you subscribe to mcast messages that you're interested, you
won't get messages for other multicast groups.

Re: nfnetlink and conntrack extension question

[Jan Engelhardt]

On Wednesday 2011-11-30 19:22, Pablo Neira Ayuso wrote:
>> 
>> As for 1, you can use genetlink, just as I do for the copy of ipset
>> in xtables-addons. Being forced to use nfnetlink has been point of
>> much discussion and ultimately, nobody was able to provide a
>> technical reason on why nfnetlink is better.
>
>Well, few differences. With genetlink:
>
>* you have to send a message to look up for the ID first (to guess the
>  multicast group and subsystem IDs).

That is to be expected and rather ok (usernames<->UID mapping is
the very same issue in many a program).

Adding myself, from the previous discussions,

* one needs to listen for ID changes

=> I hope that genetlink can be tuned so as to pin modules while
any socket to them is open, so that there will be no change.

>* you don't know how many users will using the genetlink bus. You'll
>  have to share the bandwidth with them.

That is rather a non-argument - you won't know many nfnetlink users
there will be either. But anyway, this would only apply to mcast
messages, and then only when you actually subscribed, does it not?

Re: nfnetlink and conntrack extension question

[KOVACS Krisztian]

Hi,

On Sun 04 Dec 2011 10:06:12 PM CET, Jan Engelhardt wrote:
> On Wednesday 2011-11-30 19:22, Pablo Neira Ayuso wrote:
>>> As for 1, you can use genetlink, just as I do for the copy of ipset
>>> in xtables-addons. Being forced to use nfnetlink has been point of
>>> much discussion and ultimately, nobody was able to provide a
>>> technical reason on why nfnetlink is better.

Thanks for the suggestion, guys. We could definitely use genetlink 
instead of nfnetlink for the userspace-kernelspace communication stuff.

However, this still leaves us with the second problem -- registering a 
conntrack extension without recompiling the conntrack module. That one 
is still more or less a show-stopper for us, unless we implement our 
own state table effectively duplicating the features of the conntrack 
hash and do a lookup every time we need policy information. Neither of 
these options is particularly nice.

-- 
KOVACS Krisztian

Re: nfnetlink and conntrack extension question

[KOVACS Krisztian]

Hey Pablo,

On 11/30/2011 07:09 PM, Pablo Neira Ayuso wrote:
> P.S: Any chance to integrate your development with nfgrep? I didn't
> find any time to release the source code yet. I expect to find some
> spare time during the end of year holidays. If I have access to that
> GPL code we can think of some sort of integration nfgrep <-> zorp.

Well, the code itself isn't really about doing content
processing/filtering in kernel-space, but implementing a rule-based
policy engine (think of something like iptables with best-match
evaluation) that effectively makes decisions about which user-space
proxies, if any, are used for processing a connection.

So at the moment I think that while nfgrep matching could be integrated
into the code and would provide some really nice additional features, it
wouldn't be a replacement for the stuff we're doing.

nfgrep's really cool, though, thanks for your work Pablo!

-- 
KOVACS Krisztian