NF structural changes (was: dangerous?)

NF structural changes (was: dangerous?)

[Jan Engelhardt]

Hello Ccs,


As a result of the "dangerous? Setting mark in nat table" thread, I have 
just drawn this idea of mine down, where:

 - filter and mangle tables are merged
 - filtering comes early, even before nat
   (so that, for example, packets do not needlessy be MARKed, mangled, etc.)

Pic: http://jengelh.hopto.org/GFX0/nf_proposal.png (or .svg)
(Yes, it's reminiscient of http://imagestream.com/~josh/PacketFlow-new.png )

Comments welcome.


Jan
-- 

Re: NF structural changes (was: dangerous?)

[Patrick McHardy]

Jan Engelhardt wrote:
> As a result of the "dangerous? Setting mark in nat table" thread, I have 
> just drawn this idea of mine down, where:
> 
>  - filter and mangle tables are merged

We need to preserve compatibility (meaning we must keep the rerouting
in mangle), so I don't see the advantage over simply removing the
restrictions to mangle.

>  - filtering comes early, even before nat
>    (so that, for example, packets do not needlessy be MARKed, mangled,
etc.)

Would probably make life easier (no more -m conntrack --ctorigdst),
but I don't see how this could be changed without breaking rulesets.

Re: NF structural changes (was: dangerous?)

[Jan Engelhardt]

On Mar 14 2007 22:05, Patrick McHardy wrote:
>Jan Engelhardt wrote:
>> As a result of the "dangerous? Setting mark in nat table" thread, I have 
>> just drawn this idea of mine down, where:
>> 
>>  - filter and mangle tables are merged
>
>We need to preserve compatibility (meaning we must keep the rerouting
>in mangle), so I don't see the advantage over simply removing the
>restrictions to mangle.

I think I kept that, did not I? Basically, I took PacketFlow-new, and
removed the filter table - which leaves rerouting working, and turned the
mangle into a filter/mangle (which automatically satisfies the 2nd point).
Or could you please enlighten me a bit what exactly you meant?

>
>>  - filtering comes early, even before nat
>>    (so that, for example, packets do not needlessy be MARKed, mangled,
>etc.)
>
>Would probably make life easier (no more -m conntrack --ctorigdst),

--ctorigdst is still needed (or at least, it's a benefit), in the
filter/mangle-INPUT box and filter/mangle-POSTROUTING.

>but I don't see how this could be changed without breaking rulesets.

Right. Of course this is a problem. To that end, I can really only say
that a divide needs to be made someday. People lived through ipfwadm,
ipchains and now iptables (and a change from one to another required
ruleset changes too). There also have been "minor" annoyances, such as
double-NAT removal (IIRC), and NAT of local outgoing connections (which
left the "warn_if_extra_mangle" function till today). Then support for
multiple ranges got removed in SNAT/DNAT and friends, perhaps breaking a
few rulesets again. The latest breakage was the switch from L3-dependent
to L3-independent. Though POM and external modules are not as supported as
mainline matches/targets, they are useful for a number of people, did not
get compiled, and hence, broke rulesets again. I have compilation fixes
for ROUTE, SYSRQ, ipp2p, time, u32, layer7 to make them work with L3-ind.
tproxy has not been updated yet by balabit.

Let's start a new netfilter that can coexist [at the Kconfig level] with
what people know, therefore making them aware of changes. ("You choose
CONFIG_NETFILTER2, you know your scripts will break.") Perhaps call it
CONFIG_PKTTABLES ;-)



Jan
-- 

Re: NF structural changes (was: dangerous?)

[Amin Azez]

Jan Engelhardt wrote:
> On Mar 14 2007 22:05, Patrick McHardy wrote:
>   
>> Jan Engelhardt wrote:
>>     
>>> As a result of the "dangerous? Setting mark in nat table" thread, I have 
>>> just drawn this idea of mine down, where:
>>>
>>>  - filter and mangle tables are merged
>>>       
>> We need to preserve compatibility (meaning we must keep the rerouting
>> in mangle), so I don't see the advantage over simply removing the
>> restrictions to mangle.
>>     
>
> I think I kept that, did not I? Basically, I took PacketFlow-new, and
> removed the filter table - which leaves rerouting working, and turned the
> mangle into a filter/mangle (which automatically satisfies the 2nd point).
> Or could you please enlighten me a bit what exactly you meant?
>
>   
I think its fine. I still miss filter/mangle after nat on pre-routing 
and re-suggest merging nat as well.

As for user-compatability, introduce a new character as a legal chain 
name, maybe @ and then let:
iptables -t XXX -[AID] YYY
refer to chain YYY@XXX
and let there be a global table whose contents when flushed default to:
-A PREROUTING -j PREROUTING@mangle
-A PREROUTING -m ctstate --ctstate new -j PREROUTING@nat
etc

Thus the old framework can easily be represented but a single global 
table can be used instead for those who can (or insist on) managing it.

>>>  - filtering comes early, even before nat
>>>    (so that, for example, packets do not needlessy be MARKed, mangled,
>>>       
>> etc.)
>>
>> Would probably make life easier (no more -m conntrack --ctorigdst),
>>     
>
> --ctorigdst is still needed (or at least, it's a benefit), in the
> filter/mangle-INPUT box and filter/mangle-POSTROUTING.
>   
it's also very useful. We also need a ctstate match to decide which ct 
flow a packet is from.
>
> Let's start a new netfilter that can coexist [at the Kconfig level] with
> what people know, therefore making them aware of changes. ("You choose
> CONFIG_NETFILTER2, you know your scripts will break.") Perhaps call it
> CONFIG_PKTTABLES ;-)
>
>   
hmmm, yeah, I've been talking about preserving the user interface, but 
you are right that the different targets and matches have a kernel 
interface that may change. (I'm thinking of nat as was recently hinted). 
But I'm hoping that if the global table has a rich enough context 
(available data and actions) that it can fake the environment of a 
target from the global table.

But at least lets merge mangle and filter in the medium term (and put 
one after nat on prerouting?)

Sam

Sam

Re: NF structural changes (was: dangerous?)

[Jan Engelhardt]

On Mar 14 2007 21:48, Amin Azez wrote:
> I think its fine. I still miss filter/mangle after nat on pre-routing and
> re-suggest merging nat as well.

Hm ok. It is not as if you could not abuse nat _today_ for filtering...
(Yes, it makes perfectly sense to merge nat as well, now that I have 
given it some more thought.)

> As for user-compatability, introduce a new character as a legal chain name,
> maybe @ and then let:
> iptables -t XXX -[AID] YYY
> refer to chain YYY@XXX
> and let there be a global table whose contents when flushed default to:
> -A PREROUTING -j PREROUTING@mangle
> -A PREROUTING -m ctstate --ctstate new -j PREROUTING@nat
> etc

Wanna merge the 'raw' table too - or would that cause increased coding
complexity in netfilter?
The same goes for brouting, which probably needs to stay separate from 
the 'main' table. If they all can be merged, the better.

> But at least lets merge mangle and filter in the medium term (and put 
> one after nat on prerouting?)

Here is the upgraded schematic graph. I have dubbed the 
filter/mangle/nat combo table 'main'.

http://jengelh.hopto.org/GFX0/nf_proposal2.png (.svg)



Jan
-- 

Re: NF structural changes (was: dangerous?)

[Henrik Nordstrom]

[-- Attachment #1: Type: text/plain, Size: 533 bytes --]

tor 2007-03-15 klockan 03:24 +0100 skrev Jan Engelhardt:

> Wanna merge the 'raw' table too - or would that cause increased coding
> complexity in netfilter?

To do that connection tracking needs significant changes.

the main purpose of raw is to hit packets before connection tracking,
while everything else is after connection tracking.

But yes, it's possible by just adding a set of additional
hooks/entrychains to the "filter" table. But thats not a very big
difference from having them split..

Regards
Henrik

[-- Attachment #2: Detta är en digitalt signerad meddelandedel --]
[-- Type: application/pgp-signature, Size: 307 bytes --]

Re: NF structural changes (was: dangerous?)

[Patrick McHardy]

Jan Engelhardt wrote:
> Right. Of course this is a problem. To that end, I can really only say
> that a divide needs to be made someday.

I totally disagree. iptables has tons of problems, I'm guessing that
a year from now we will have a successor. Breaking compatibility at
this point in its lifetime is not acceptable, especially on such a
fundamental level. I really wish people would stop wasteing time
trying to redesign iptables, its just not worth it.

Re: NF structural changes (was: dangerous?)

[Amin Azez]

Patrick McHardy wrote:
> Jan Engelhardt wrote:
>   
>> Right. Of course this is a problem. To that end, I can really only say
>> that a divide needs to be made someday.
>>     
>
> I totally disagree. iptables has tons of problems, I'm guessing that
> a year from now we will have a successor. Breaking compatibility at
> this point in its lifetime is not acceptable, especially on such a
> fundamental level. 
quite likely we will come to that conclusion but our discussion might 
result in private patches that don't hit the kernel.
> I really wish people would stop wasteing time
> trying to redesign iptables, its just not worth it.
>   
maybe, but it looks so promising coming from a position of wasting time 
trying to work with it :-)
- and you know it's so much more fun to tweak someone elses work than 
start from scratch...

Sam

Re: NF structural changes (was: dangerous?)

[Krzysztof Oledzki]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 809 bytes --]



On Wed, 14 Mar 2007, Jan Engelhardt wrote:

> Hello Ccs,
>
>
> As a result of the "dangerous? Setting mark in nat table" thread, I have
> just drawn this idea of mine down, where:
>
> - filter and mangle tables are merged

Please don't. As you probably know "-j MARK" is not a terminating target, 
so after a "-j MARK" you often need additional "-j ACCEPT". The same is 
about TTL, SET, etc. But this ACCEPT does not mean that this packed is 
going to be accepted, only that a mangling process is finished.

> - filtering comes early, even before nat
>   (so that, for example, packets do not needlessy be MARKed, mangled, etc.)

Hm, it is very useful to be able to filter packed based on a real 
destignation, that is only known _after_ dnat.

Best regards,

 				Krzysztof Olędzki

Re: NF structural changes (was: dangerous?)

[Jan Engelhardt]

On Mar 14 2007 22:41, Krzysztof Oledzki wrote:
>> 
>> - filter and mangle tables are merged
>
> Please don't. As you probably know "-j MARK" is not a terminating target, so
> after a "-j MARK" you often need additional "-j ACCEPT". The same is about TTL,
> SET, etc. But this ACCEPT does not mean that this packed is going to be
> accepted, only that a mangling process is finished.

Since MARK is proposed to anyway be available in the filter table (patch
posted by Sam at
http://lists.netfilter.org/pipermail/netfilter-devel/2007-March/027309.html
and comment
http://lists.netfilter.org/pipermail/netfilter-devel/2007-March/027310.html),
people will already have to deal with it when using MARK during filtering.

Since what I was proposing is a change big enough to break rulesets "hard"
(subjective wording), people sticking with the potentially new scheme
would anyway know what awaited them. 

>> - filtering comes early, even before nat
>> (so that, for example, packets do not needlessy be MARKed, mangled,
>> etc.)
>
> Hm, it is very useful to be able to filter packed based on a real
> destignation, that is only known _after_ dnat.

Somehow yes :-/
I'll post an updated png soon.


Jan
--