Re: nf_conntrack

Re: nf_conntrack

[Davide Guerri]

On Friday 20 February 2004 13:58, you wrote:
> Hi,
>
> From: Davide Guerri <tato.net@tin.it>
> Date: Fri, 20 Feb 2004 09:42:13 +0100
>
> > Hi, and sorry for my poor English.
>
> Don't worry. me too.
>
> > I'm very interested in this project since I'm writing a NA(P)T-PT (as
> > described in RFC 2766) implementation for Linux. Actually I'm a student
> > (computer science, in Rome) and this will be my PhD thesis.
>
> Great.
>
> > What happened to your code? I've downloaded USAGI Linux kernel, but I've
> > found only your previous IPv6 conntrack. (very good work however!)
>
> nf_conntrack is not in USAGI linux26. Of cause, I'll commit it to USAGI
> tree.
>
> After I sent nf_conntrack, Mr. Harald Welte, who is a member of Netfilter
> Core Team, suggested that L3 part of conntrack (tuple, too) should be
> allocated dynamically.
>
> Then I wrote "dynamic version" and sent it last year (December), but I
> suspend this work due to other work for a while. Fortunately, I could
> resume to this work. I'll evaluate 2 implementations, and decide which
> implementation is adopted.
>
> After that, I'll commit nf_conntrack to USAGI tree.

That's a great news.
I've tryed the second version of your nf_conntrack. It seems to work very 
well!

> > Is there some documentation on conntrack? I'm reading the existing NAT
> > code, but it's quite difficult to understand. It's difficult for me
> > because I don't yet completely understand the general behaviour of
> > netfilter conntrack and the role of its tricky data structures.
>
> Unfortunately, there isn't documentation about nf_conntrack.
> But its scheme is mostly same as ip_conntrack. The differences are tuple,
> new functions and arguments for Layer 3 header handling, and new module
> for Layer 3 protocols.
>
> BTW, do you read Documentations at Netfilter Project web page ?
>
> 	http://www.netfilter.org/documentation/index.html
>
> "Netfilter Hacking HOWTO" may help you.

In the (very good) "Netfilter Hacking HOWTO" there are some infos on NAT, but 
I would need more detail.
However there is no problem, I'll study the netfilter code. ;)

> > I've also downloaded the nf_conntrack patch included in your post on
> > netfilter-devel, I've applied it to a vanilla 2.6.0 Kernel and I'm going
> > to try it. Prior to read that code, I would ask you if it can handle
> > mixed IPv6-IPv4 tuples.
>
> Thank you for using nf_conntrack. But you cannot handle mixed tuples
> (ex. src = IPv6, dst = IPv4) in now.

Sorry, I've expressed myself badly! What I need is to change the expected 
reply of the conntrack from an IPv4 tuple to an IPv6 tuple and vice versa. 
Hence what I would to know is if nf_conntrack can handles original-direction 
tuples of one type and reply-direction tuple of the other type (so that it 
will be possible to track a "NA(P)T-PT'ed connection").

> You need to write NA(P)T-PT code based on nf_conntrack. Maybe IPv4 NAT
> codes help you. The biggest issue is how pass packets from IPv4 stack to
> IPv6 stack and vice versa. I imagine that addresses in packets are mangled
> at only NF_IP{6}_PRE_ROUNTING, and then called netif_rx().

That was my first idea. But I've seen some code that makes use 
ip_route_me_harder() + ip_output() for injecting IPv4 packets, and 
ip6_route_me_harder() + ip6_output for IPv6 packets.
I fear that with netif_rx() traslated packet would be tracked again, and this 
is not what I need. (I'm sorry but I'm still studying netfilter and conntrack 
(and English too ;) ) so it's likely that I'm wrong).

> BTW, if allowed, could you Cc to netfilter-devel for other folks who
> interested in this area ?

I've just done it ;)

> Regards,
>
> -----------------------------------------------------------------
> Yasuyuki KOZAKAI @ USAGI Project <yasuyuki.kozakai@toshiba.co.jp>

Thank you very much Yasuyuki for your work and for your quick reply!

Best regards,
Davide Guerri.

-- 
T@1t

Re: nf_conntrack

[Yasuyuki Kozakai]

Hi,

From: Davide Guerri <tato.net@tin.it>
Date: Sun, 22 Feb 2004 11:10:28 +0100

> > > I've also downloaded the nf_conntrack patch included in your post on
> > > netfilter-devel, I've applied it to a vanilla 2.6.0 Kernel and I'm going
> > > to try it. Prior to read that code, I would ask you if it can handle
> > > mixed IPv6-IPv4 tuples.
> >
> > Thank you for using nf_conntrack. But you cannot handle mixed tuples
> > (ex. src = IPv6, dst = IPv4) in now.
> 
> Sorry, I've expressed myself badly! What I need is to change the expected 
> reply of the conntrack from an IPv4 tuple to an IPv6 tuple and vice versa. 
> Hence what I would to know is if nf_conntrack can handles original-direction 
> tuples of one type and reply-direction tuple of the other type (so that it 
> will be possible to track a "NA(P)T-PT'ed connection").

You're right. I mistook.

Please assume that initiator is IPv4 node. Likely IPv4 NAT, nf_conntrack
create conntrack which has following tuples.

          | Src	           |Dst
-------------------------------------------
          | Address | Port |Address | Port
-------------------------------------------
Original  | A4	    | a	   |B4      | b
Reply	  | B4	    | b	   |A4      | a


NA(P)T-PT module will select unique tuple, and change tuple of conntrack.

          | Src	           |Dst
-------------------------------------------
          | Address | Port |Address | Port
-------------------------------------------
Original  | A4	    | a	   |B4      | b
Reply	  | B'6	    | b    |A'6     | a'

# A4, B4   : IPv4 addr
# A'6, B'6 : IPv6 addr

nf_conntrack doesn't need to care about weather tuple has changed or not.
It just find conntrack which has tuple matched with IPv{4,6} header of packet,
and update states of it.

Handling tuple and conntrack in NA(P)T-PT are same as IPv4 NAT.
The differences are creating IPv{4,6} header and passing to other network stack.
And please care about relation of IPv4 NAT and NA(P)T-PT. These are exclusive.

> > You need to write NA(P)T-PT code based on nf_conntrack. Maybe IPv4 NAT
> > codes help you. The biggest issue is how pass packets from IPv4 stack to
> > IPv6 stack and vice versa. I imagine that addresses in packets are mangled
> > at only NF_IP{6}_PRE_ROUNTING, and then called netif_rx().
> 
> That was my first idea. But I've seen some code that makes use 
> ip_route_me_harder() + ip_output() for injecting IPv4 packets, and 
> ip6_route_me_harder() + ip6_output for IPv6 packets.
> I fear that with netif_rx() traslated packet would be tracked again, and this 
> is not what I need. (I'm sorry but I'm still studying netfilter and conntrack 
> (and English too ;) ) so it's likely that I'm wrong).

Sorry, I'm confusing. Which path should packets go through ? we need to design
so that users can filter/log packets before and/or after translating.
At least, translated packets should go through NF_IP{6}_FORWARD, I think.

> > BTW, if allowed, could you Cc to netfilter-devel for other folks who
> > interested in this area ?
> 
> I've just done it ;)

Thanks!

-----------------------------------------------------------------
Yasuyuki KOZAKAI @ USAGI Project <yasuyuki.kozakai@toshiba.co.jp>

Re: nf_conntrack

[Henrik Nordstrom]

On Mon, 23 Feb 2004, Yasuyuki Kozakai wrote:

> nf_conntrack doesn't need to care about weather tuple has changed or not.
> It just find conntrack which has tuple matched with IPv{4,6} header of packet,
> and update states of it.

But conntrack do care about the protocol and is only implemented for 
IPv4.

Assuming there is a 1-1 map IPv4<->IPv6 (after any IPv4 NAT applied by
iptables) then the translation is probably best implemented as a virtual
tunnel like device outside of netfilter. IPv4 packets routed to this
tunnel gets translated into IPv6 and then reinjected to the other stack as
input packets and vise versa. Beware of loops.

In early implementation you can use the TUN device and run the translation
in userspace. In final implemenenation you want to implement a new virtual
device running the translation in kernel land to avoid extra copying of
the packet data and context switches..

Regards
Henrik

Re: nf_conntrack

[Davide]

Hi Yasuyuki,

On Monday 23 February 2004 06:27, you wrote:

> > > You need to write NA(P)T-PT code based on nf_conntrack. Maybe IPv4 NAT
> > > codes help you. The biggest issue is how pass packets from IPv4 stack
> > > to IPv6 stack and vice versa. I imagine that addresses in packets are
> > > mangled at only NF_IP{6}_PRE_ROUNTING, and then called netif_rx().
> >
> > That was my first idea. But I've seen some code that makes use
> > ip_route_me_harder() + ip_output() for injecting IPv4 packets, and
> > ip6_route_me_harder() + ip6_output for IPv6 packets.
> > I fear that with netif_rx() traslated packet would be tracked again, and
> > this is not what I need. (I'm sorry but I'm still studying netfilter and
> > conntrack (and English too ;) ) so it's likely that I'm wrong).
>
> Sorry, I'm confusing. Which path should packets go through ? we need to
> design so that users can filter/log packets before and/or after
> translating. At least, translated packets should go through
> NF_IP{6}_FORWARD, I think.

I agree with you, users should be able to filter/log NAT(P)T-PT'ed packets. 
However there are some (bad) problems.
At present time your conntrack code doesn't handle IPv6 fragments, but I guess  
that when it will do, IPv6 datagrams will be defragmented as in current 
conntrack. So I assume that NA(P)T-PT code will see only entire l3 packets.

One problem is that I should know if original datagrams had a fragment header 
or not. NA(P)T-PT should produce IPv4 packets with DF flag set for IPv6 
non-fragment datagrams and IPv4 packets with DF not set for IPv6 datagrams 
with a fragment header. So, although on one hand I need conntrack, on the 
other hand I should see original datagrams.
A similar problem turns out on IPv4-to-IPv6 translation. When I receive IPv4 
datagrams with DF not set, I should fragment them so that their IPv6 
translation fits in 1280 octects. If DF flag is set, in order to allow 
pathMTU discovery, I should preserve their size and I should translate them 
into IPv6 datagram without a fragmentation header (except for IPv4 fragments 
with DF set...).
I'm trying to say (with my terrible English) that a NA(P)T-PT implementation 
should have more control over IPv6/IPv4 packets: since conntrack 
defragments/fragments them it's impossible to be RFC 2766 compliant.

For these reasons I'm tempted to use the approach proposed by Henrik (virtual 
network device).

Please correct me if I've made any mistakes.

BTW: I'm very happy that your conntrack will be put into patch-o-matic-ng ;)

Best regards,
Davide

-- 
T@1t

Re: nf_conntrack

[Yasuyuki Kozakai]

[-- Attachment #1: Type: Text/Plain, Size: 3797 bytes --]

Hi, Davide,

Sorry for late replying.

From: Davide <tatonet@tiscali.it>
Date: Thu, 26 Feb 2004 13:20:08 +0100

> > Sorry, I'm confusing. Which path should packets go through ? we need to
> > design so that users can filter/log packets before and/or after
> > translating. At least, translated packets should go through
> > NF_IP{6}_FORWARD, I think.
> 
> I agree with you, users should be able to filter/log NAT(P)T-PT'ed packets. 
> However there are some (bad) problems.
> At present time your conntrack code doesn't handle IPv6 fragments, but I guess  
> that when it will do, IPv6 datagrams will be defragmented as in current 
> conntrack. So I assume that NA(P)T-PT code will see only entire l3 packets.

Yes. That is missing piece. Of cause, I'll implement that. But I haven't
decide how to handle fragmented IPv6 packets yet.

ip6_conntrack, which is written by me and tracks only IPv6 packets, clone
fragmented packets and reassemble them (note: not linearize). After tracking,
reassembled packet is discard and pass original fragmented packets to next
network process. Of cause, original fragmented skbs points to same conntrack.

I adopted this way to prevent from disturbing other IPv6 codes(ex. Path MTU
Discovery.)

But I don't know this is best. Maybe NA(P)T-PT helper module wants reassembled
packet to rewrite data in payload (ex FTP).
				   
> One problem is that I should know if original datagrams had a fragment header 
> or not. NA(P)T-PT should produce IPv4 packets with DF flag set for IPv6 
> non-fragment datagrams and IPv4 packets with DF not set for IPv6 datagrams 
> with a fragment header. So, although on one hand I need conntrack, on the 
> other hand I should see original datagrams.
> A similar problem turns out on IPv4-to-IPv6 translation. When I receive IPv4 
> datagrams with DF not set, I should fragment them so that their IPv6 
> translation fits in 1280 octects. If DF flag is set, in order to allow 
> pathMTU discovery, I should preserve their size and I should translate them 
> into IPv6 datagram without a fragmentation header (except for IPv4 fragments 
> with DF set...).
> I'm trying to say (with my terrible English) that a NA(P)T-PT implementation 
> should have more control over IPv6/IPv4 packets: since conntrack 
> defragments/fragments them it's impossible to be RFC 2766 compliant.

IPv4 header has DF bit field and it's not changed. so there is no problem
even if reassembled by conntrack in the case of IPv4-to-IPv6.

There is problem in the case of IPv6-to-IPv4. I was going to implement so that
nf_conntrack removes Fragment Header in the case that payload length of
reassembled packet is > 65535 - 8 .

ok, I need to be more care about handling fragmented packet.

But in any case, original(not reassembled) IPv6 packets are needed to be
passed next network.
And I think it's natural that behavior for IPv4 and IPv6 is synchronized.
This mean that IPv4 packets isn't reassembled.

In this case, benefits are
	- all modules can process fragmented packets with any style.
	- all modules can prevent from disturbing other network code.

drawbacks are
	- User Interface will be changed. current ip_conntrack reassemble at
	  PRE_ROUTING, and refragment at POST_ROUTING. so users assume that
	  packets are reassembled when filtering.

	- if multiple module want reassembled packets, reassembling is occurred
	  many times.

The 1st drawback can be solved at ip{6}_tables in kernel.
But I don't any idea about 2nd issue in now.

> For these reasons I'm tempted to use the approach proposed by Henrik (virtual 
> network device).

I have timely news. Recently, I heard that kernel implementation of NAT-PT is
already created. Maybe this implementation doesn't use netfilter.
virtual device ???

I'll attach the message at usagi-users ML.


[-- Attachment #2: nat-pt.txt --]
[-- Type: Text/Plain, Size: 3230 bytes --]

Received: from mx2.toshiba.co.jp (mx2.toshiba.co.jp [133.199.160.163])
	by isl.rdc.toshiba.co.jp (8.11.7/8.11.6/1.4) with ESMTP id i1REpf128100;
	Fri, 27 Feb 2004 23:51:41 +0900 (JST)
Received: from tsb-sgw2.toshiba.co.jp by toshiba.co.jp id XAA17776; Fri, 27 Feb 2004 23:51:41 +0900 (JST)
Received: from inet-tsb5.toshiba.co.jp 
	by tsb-sgw2.toshiba.co.jp  with ESMTP id i1REpe8U014622;
	Fri, 27 Feb 2004 23:51:40 +0900 (JST)
Received: from linux6.nezu.wide.ad.jp (linux6.nezu.wide.ad.jp [203.178.142.218])
	by inet-tsb5.toshiba.co.jp  with ESMTP id i1REpd4v018298
	(version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO);
	Fri, 27 Feb 2004 23:51:39 +0900 (JST)
Received: from nezu.linux-ipv6.org (localhost [127.0.0.1])
	by linux6.nezu.wide.ad.jp (8.12.3/8.12.3/Debian-6.6) with ESMTP id i1REEPAA016856;
	Fri, 27 Feb 2004 23:14:25 +0900
Received: from web40301.mail.yahoo.com (web40301.mail.yahoo.com [66.218.78.80])
	by linux6.nezu.wide.ad.jp (8.12.3/8.12.3/Debian-6.6) with SMTP id i1REEFAA016836
	for <usagi-users@linux-ipv6.org>; Fri, 27 Feb 2004 23:14:15 +0900
Received: from [193.220.224.6] by web40301.mail.yahoo.com via HTTP; Fri, 27 Feb 2004 04:27:28 PST
Date: Fri, 27 Feb 2004 04:27:28 -0800 (PST)
From: sunil arora <korona_lite@yahoo.com>
Reply-To: usagi-users@linux-ipv6.org
Subject: (usagi-users 02823) regarding NAT-PT implementation in linux kernel
To: yhsiung@zyxel.com, usagi-users@linux-ipv6.org
Cc: wangping@tyut.edu.cn
Message-Id: <20040227122728.59974.qmail@web40301.mail.yahoo.com>
X-ML-Name: usagi-users
X-Mail-Count: 02823
X-MLServer: fml [fml 4.0.3 release (20011202/4.0.3)]; post only (only members can post)
X-ML-Info: If you have a question, send e-mail with the body
	"help" (without quotes) to the address usagi-users-ctl@linux-ipv6.org;
	help=<mailto:usagi-users-ctl@linux-ipv6.org?body=help>
X-Spam-Status: No, hits=0.0 required=5.0 tests= version=2.20
X-Spam-Flag: No
X-Spam-Probability: 0.499972
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Precedence: bulk
Lines: 29
List-Id: usagi-users.linux-ipv6.org
List-Software: fml [fml 4.0.3 release (20011202/4.0.3)]
List-Post: <mailto:usagi-users@linux-ipv6.org>
List-Owner: <mailto:usagi-users-admin@linux-ipv6.org>
List-Help: <mailto:usagi-users-ctl@linux-ipv6.org?body=help>
List-Unsubscribe: <mailto:usagi-users-ctl@linux-ipv6.org?body=unsubscribe>
X-UIDL: -DF"!+7-"!</7!!<<d!!

Hi all,
I am really sorry for late response. Actually was busy
with mid-semester exms here!

Our Nat-PT implementation in linux kernel is almost
over and we are giving it final testing. It will take
roughly 25 days to get the finally tested patch
available for kernel 2.4.22.
I hope you guys can bear with us. And We will be
releasing a floppy based version of our distribution
as well, that is also almost ready.

I apologise for not making the code available right
away.
There is also a userland application developed by some
instituition in Korea which performs NAT-PT. I read an
article in LJ discussing a NAT-PT set up ..some months
back.I will send the pointer to their link in the next
mail.

-sun



__________________________________
Do you Yahoo!?
Get better spam protection with Yahoo! Mail.
http://antispam.yahoo.com/tools

Re: nf_conntrack

[Davide Guerri]

Hi Yasuyuki,

On Monday 01 March 2004 07:08, you wrote:
> Hi, Davide,
>
> Sorry for late replying.

Don't worry, thank you very much for your reply ;)

> > > Sorry, I'm confusing. Which path should packets go through ? we need to
> > > design so that users can filter/log packets before and/or after
> > > translating. At least, translated packets should go through
> > > NF_IP{6}_FORWARD, I think.
> >
> > I agree with you, users should be able to filter/log NAT(P)T-PT'ed
> > packets. However there are some (bad) problems.
> > At present time your conntrack code doesn't handle IPv6 fragments, but I
> > guess that when it will do, IPv6 datagrams will be defragmented as in
> > current conntrack. So I assume that NA(P)T-PT code will see only entire
> > l3 packets.
>
> Yes. That is missing piece. Of cause, I'll implement that. But I haven't
> decide how to handle fragmented IPv6 packets yet.
>
> ip6_conntrack, which is written by me and tracks only IPv6 packets, clone
> fragmented packets and reassemble them (note: not linearize). After
> tracking, reassembled packet is discard and pass original fragmented
> packets to next network process. Of cause, original fragmented skbs points
> to same conntrack.
>
> I adopted this way to prevent from disturbing other IPv6 codes(ex. Path MTU
> Discovery.)
>
> But I don't know this is best. Maybe NA(P)T-PT helper module wants
> reassembled packet to rewrite data in payload (ex FTP).

Yes, that is a good way, but there is another issue. Since conntrack will be 
used to implement a statefull IPv6 filtering, we need to be sure that what is 
received by destination host is what the firewall have seen. 
I know that partially overlapped IPv6 fragments should never appear (since 
only the source host can perform fragmentation), but in general we don't know 
if the destination host will accept or discard them.
If the firewall lets them pass, we couldn't be sure of the order in which they 
will be received by the destination ( think, for instance, to the well-known 
IPv4 overlapped fragments attack).

Personally I think that a firewall must discard overlapped (IPv4/IPv6) 
fragments, but this is only my personal point of view.

> > One problem is that I should know if original datagrams had a fragment
> > header or not. NA(P)T-PT should produce IPv4 packets with DF flag set for
> > IPv6 non-fragment datagrams and IPv4 packets with DF not set for IPv6
> > datagrams with a fragment header. So, although on one hand I need
> > conntrack, on the other hand I should see original datagrams.
> > A similar problem turns out on IPv4-to-IPv6 translation. When I receive
> > IPv4 datagrams with DF not set, I should fragment them so that their IPv6
> > translation fits in 1280 octects. If DF flag is set, in order to allow
> > pathMTU discovery, I should preserve their size and I should translate
> > them into IPv6 datagram without a fragmentation header (except for IPv4
> > fragments with DF set...).
> > I'm trying to say (with my terrible English) that a NA(P)T-PT
> > implementation should have more control over IPv6/IPv4 packets: since
> > conntrack defragments/fragments them it's impossible to be RFC 2766
> > compliant.
>
> IPv4 header has DF bit field and it's not changed. so there is no problem
> even if reassembled by conntrack in the case of IPv4-to-IPv6.

Actually ip_conntrack ignore DF flag in fragmented IPv4 fragments. 
This leads to a problem described in this bug report:
https://bugzilla.netfilter.org/cgi-bin/bugzilla/show_bug.cgi?id=48
In this report I've proposed a solution; however I must admit that the 
implementation I've proposed it is not good. (However I don't know if that 
idea can be useful for you).
I think we must avoid that problem in IPv6 connection tracking since in this 
case the consequences are more serious.

> There is problem in the case of IPv6-to-IPv4. I was going to implement so
> that nf_conntrack removes Fragment Header in the case that payload length
> of reassembled packet is > 65535 - 8 .
>
> ok, I need to be more care about handling fragmented packet.

If there were no problems it wouldn't be amusing! ;))

> But in any case, original(not reassembled) IPv6 packets are needed to be
> passed next network.
> And I think it's natural that behavior for IPv4 and IPv6 is synchronized.
> This mean that IPv4 packets isn't reassembled.
>
> In this case, benefits are
> 	- all modules can process fragmented packets with any style.
> 	- all modules can prevent from disturbing other network code.
>
> drawbacks are
> 	- User Interface will be changed. current ip_conntrack reassemble at
> 	  PRE_ROUTING, and refragment at POST_ROUTING. so users assume that
> 	  packets are reassembled when filtering.
>
> 	- if multiple module want reassembled packets, reassembling is occurred
> 	  many times.
>
> The 1st drawback can be solved at ip{6}_tables in kernel.
> But I don't any idea about 2nd issue in now.

I agree with you, IPv6 and IPv4 conntrack behaviours should be synchronized.
This is an hard issue: any solutions will require many changes in present 
netfilter modules...

> > For these reasons I'm tempted to use the approach proposed by Henrik
> > (virtual network device).
>
> I have timely news. Recently, I heard that kernel implementation of NAT-PT
> is already created. Maybe this implementation doesn't use netfilter.
> virtual device ???
>
> I'll attach the message at usagi-users ML.

Thank you very much. I've also noticed another NA(P)T-PT implementation in the 
Linux kernel (http://naptpt.sourceforge.net/). However this implementation 
doesn't handle fragment (both for IPv4 and IPv6), and use a poor connection 
tracking for TCP based on activity instead of the state: it removes a TCP 
connection from tracked connections if there is no activity for a certain 
period of time (this is good for UDP and ICMP but not for TCP).
Moreover this implementation bypasses all netfilter code (it intercepts 
packets in PRE_ROUTING and uses ip_output() to send translated packets).

I'm really in trouble: I have got the code for the tranlation, I have got a 
precise idea of how to translate packets (even in case of fragmentation) but 
I can't decide where to translate... ;))

Best regards,
Davide Guerri.

-- 
T@1t

Re: nf_conntrack

[Davide]

On Friday 20 February 2004 13:58, you wrote:
> Hi,
>
> From: Davide Guerri <tato.net@tin.it>
> Date: Fri, 20 Feb 2004 09:42:13 +0100
>
> > Hi, and sorry for my poor English.
>
> Don't worry. me too.
>
> > I'm very interested in this project since I'm writing a NA(P)T-PT (as
> > described in RFC 2766) implementation for Linux. Actually I'm a student
> > (computer science, in Rome) and this will be my PhD thesis.
>
> Great.
>
> > What happened to your code? I've downloaded USAGI Linux kernel, but I've
> > found only your previous IPv6 conntrack. (very good work however!)
>
> nf_conntrack is not in USAGI linux26. Of cause, I'll commit it to USAGI
> tree.
>
> After I sent nf_conntrack, Mr. Harald Welte, who is a member of Netfilter
> Core Team, suggested that L3 part of conntrack (tuple, too) should be
> allocated dynamically.
>
> Then I wrote "dynamic version" and sent it last year (December), but I
> suspend this work due to other work for a while. Fortunately, I could
> resume to this work. I'll evaluate 2 implementations, and decide which
> implementation is adopted.
>
> After that, I'll commit nf_conntrack to USAGI tree.

That's a great news.
I've tryed the second version of your nf_conntrack. It seems to work very 
well!

> > Is there some documentation on conntrack? I'm reading the existing NAT
> > code, but it's quite difficult to understand. It's difficult for me
> > because I don't yet completely understand the general behaviour of
> > netfilter conntrack and the role of its tricky data structures.
>
> Unfortunately, there isn't documentation about nf_conntrack.
> But its scheme is mostly same as ip_conntrack. The differences are tuple,
> new functions and arguments for Layer 3 header handling, and new module
> for Layer 3 protocols.
>
> BTW, do you read Documentations at Netfilter Project web page ?
>
> 	http://www.netfilter.org/documentation/index.html
>
> "Netfilter Hacking HOWTO" may help you.

In the (very good) "Netfilter Hacking HOWTO" there are some infos on NAT, but 
I would need more detail.
However there is no problem, I'll study the netfilter code. ;)

> > I've also downloaded the nf_conntrack patch included in your post on
> > netfilter-devel, I've applied it to a vanilla 2.6.0 Kernel and I'm going
> > to try it. Prior to read that code, I would ask you if it can handle
> > mixed IPv6-IPv4 tuples.
>
> Thank you for using nf_conntrack. But you cannot handle mixed tuples
> (ex. src = IPv6, dst = IPv4) in now.

Sorry, I've expressed myself badly! What I need is to change the expected 
reply of the conntrack from an IPv4 tuple to an IPv6 tuple and vice versa. 
Hence what I would to know is if nf_conntrack can handles original-direction 
tuples of one type and reply-direction tuple of the other type (so that it 
will be possible to track a "NA(P)T-PT'ed connection").

> You need to write NA(P)T-PT code based on nf_conntrack. Maybe IPv4 NAT
> codes help you. The biggest issue is how pass packets from IPv4 stack to
> IPv6 stack and vice versa. I imagine that addresses in packets are mangled
> at only NF_IP{6}_PRE_ROUNTING, and then called netif_rx().

That was my first idea. But I've seen some code that makes use 
ip_route_me_harder() + ip_output() for injecting IPv4 packets, and 
ip6_route_me_harder() + ip6_output for IPv6 packets.
I fear that with netif_rx() traslated packet would be tracked again, and this 
is not what I need. (I'm sorry but I'm still studying netfilter and conntrack 
(and English too ;) ) so it's likely that I'm wrong).

> BTW, if allowed, could you Cc to netfilter-devel for other folks who
> interested in this area ?

I've just done it ;)

> Regards,
>
> -----------------------------------------------------------------
> Yasuyuki KOZAKAI @ USAGI Project <yasuyuki.kozakai@toshiba.co.jp>

Thank you very much Yasuyuki for your work and for your quick reply!

Best regards,
Davide Guerri.

-- 
T@1t

Re: nf_conntrack [was Re: [PATCH 1/4] RFC: fast string matching infrastrure for netfilter]

[Patrick McHardy]

Jozsef Kadlecsik wrote:

>On Mon, 10 Jan 2005, Patrick McHardy wrote:
>
>
>>I have actually given up keeping nf_conntrack up to date currently, but
>>I hope we can now really put ip_conntrack in maintenance mode and
>>concentrate
>>on nf_conntrack. Any chance you want to base this on the nf_conntrack
>>patch ?
>>
>
>Actually, what is the opinion that nf_conntrack uses the union of IPv4 and
>IPv6 addresses in the tuples?
>
In my opinion that's something that could also be improved later.

>The infrastructure is there in the patch to support IPv4/IPv6 conntrack
>separatedly in spite of the common code and not to waste so many bytes at
>every IPv4 connections for the sake of supporting IPv6.
>
You mean the get_features stuff and seperate caches ? I'm don't like this
part very much, I thing Rusty's "structure extension stuff" (ct_extend)
is a nicer way to do this, although its probably not useable for the tuples.

> Shouldn't there be
>put more effort into this long standing issue, before swithcing over it?
>  
>
I think we should put ip_conntrack in maintenance mode, than we can
resync nf_conntrack and make changes like this before we submit it.

Regards
Patrick

Re: nf_conntrack [was Re: [PATCH 1/4] RFC: fast string matching infrastrure for netfilter]

[Harald Welte]

[-- Attachment #1: Type: text/plain, Size: 2839 bytes --]

On Mon, Jan 10, 2005 at 09:31:45PM +0100, Patrick McHardy wrote:
> >Actually, what is the opinion that nf_conntrack uses the union of IPv4 and
> >IPv6 addresses in the tuples?
> >
> In my opinion that's something that could also be improved later.

Agreed.

> >The infrastructure is there in the patch to support IPv4/IPv6 conntrack
> >separatedly in spite of the common code and not to waste so many bytes at
> >every IPv4 connections for the sake of supporting IPv6.
>
> You mean the get_features stuff and seperate caches ? I'm don't like this
> part very much, I thing Rusty's "structure extension stuff" (ct_extend)
> is a nicer way to do this, although its probably not useable for the tuples.

Unfortunately I have to disagree.  I think I pointed it out in an
earlier email, but I'm not sure whether it was on IRC or whether you've
been on the list of recipients.

the nf_conntrack approach of having multiple slab caches of differently
sized conntrack structures is better from a performance point of view.

If you look at the typical applications, people will usually

a) either use NAT on most/all of their connections
b) or not use NAT on any of their connections

Now why don't they (the 'b' users) recompile their kernel? Because they
use vendor-kernels, and they compile with everything enabled.

So what we end up if we use rusty's scheme for nat private data, is that
we have at least one additional allocation, and two disjunct seperate
pieces of memory (which will probably also introduce yet another cache
miss).   This is acceptable for rare exceptional cases, not for 99.9% of
your connections.

So even if Rusty's architecture and API is cleaner, I'm very much in
favour of the nf_conntrack design and Yasuyuki's current
implementation.

Now you can argue about conntrack/nat helper private data.  Here my
argument is weaker, since normally you have something like < 1% of your
connections with a helper.  For this I would be willing to accept the
additional allocation and whatever.

But, then think of a firewall in front of an FTP (IRC/...) Server, and
your traffic pattern immediately looks like 50% of your connections (not
talking about traffic) have helper private data... and my argument
becomes stronger again.

> I think we should put ip_conntrack in maintenance mode, than we can
> resync nf_conntrack and make changes like this before we submit it.

At last, we again agree :)

-- 
- Harald Welte <laforge@netfilter.org>             http://www.netfilter.org/
============================================================================
  "Fragmentation is like classful addressing -- an interesting early
   architectural error that shows how much experimentation was going
   on while IP was being designed."                    -- Paul Vixie

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: nf_conntrack [was Re: [PATCH 1/4] RFC: fast string matching infrastrure for netfilter]

[Patrick McHardy]

Harald Welte wrote:

>On Mon, Jan 10, 2005 at 09:31:45PM +0100, Patrick McHardy wrote:
>  
>
>>>The infrastructure is there in the patch to support IPv4/IPv6 conntrack
>>>separatedly in spite of the common code and not to waste so many bytes at
>>>every IPv4 connections for the sake of supporting IPv6.
>>>      
>>>
>>You mean the get_features stuff and seperate caches ? I'm don't like this
>>part very much, I thing Rusty's "structure extension stuff" (ct_extend)
>>is a nicer way to do this, although its probably not useable for the tuples.
>>    
>>
>
>Unfortunately I have to disagree.  I think I pointed it out in an
>earlier email, but I'm not sure whether it was on IRC or whether you've
>been on the list of recipients.
>
Hmm, I can't recall talking about this earlier.

>the nf_conntrack approach of having multiple slab caches of differently
>sized conntrack structures is better from a performance point of view.
>
>If you look at the typical applications, people will usually
>
>a) either use NAT on most/all of their connections
>b) or not use NAT on any of their connections
>
>Now why don't they (the 'b' users) recompile their kernel? Because they
>use vendor-kernels, and they compile with everything enabled.
>
>So what we end up if we use rusty's scheme for nat private data, is that
>we have at least one additional allocation, and two disjunct seperate
>pieces of memory (which will probably also introduce yet another cache
>miss).   This is acceptable for rare exceptional cases, not for 99.9% of
>your connections.
>
>So even if Rusty's architecture and API is cleaner, I'm very much in
>favour of the nf_conntrack design and Yasuyuki's current
>implementation.
>  
>
But unlike ct_extend, the get_features stuff is very unflexible. The
way it is currently implemented it only works for a single dynamically
allocated part without leaving holes, and it even it if could handle more,
they all need to be determined at allocation time, so a worst-case 
assumption
has to be made for anything depending on the ruleset. This is especially
a concern with modules and distribution-kernels, NAT could be loaded at any
time, so the memory needs to be allocated in advance.

>Now you can argue about conntrack/nat helper private data.  Here my
>argument is weaker, since normally you have something like < 1% of your
>connections with a helper.  For this I would be willing to accept the
>additional allocation and whatever.
>
>But, then think of a firewall in front of an FTP (IRC/...) Server, and
>your traffic pattern immediately looks like 50% of your connections (not
>talking about traffic) have helper private data... and my argument
>becomes stronger again.
>  
>
Sure, but helpers are slow anyway. The additional allocation is a one-time
effort, the two disjunct pieces of memory shouldn't hurt too much since
nf_conntrack is very large and covers multiple cache-lines already. Rusty
mentioned he got struct ip_conntrack down to ~150b, this makes it possible
to have a lot more active connections without exceeding the cache.

>>I think we should put ip_conntrack in maintenance mode, than we can
>>resync nf_conntrack and make changes like this before we submit it.
>>    
>>
>
>At last, we again agree :)
>  
>
One more thing I was thinking about. If we put ip_conntrack in maintenance
mode and probably deprecate it at some time, I see little reason to remove
ipchains compat just now. Why not just let it fade out together with
ip_conntrack once nf_conntrack is in ? I haven't checked if it was because
of incompatibilities or required changes with Rusty's latest work, if so
that would be a good reason.

Regards
Patrick

Re: nf_conntrack

[Yasuyuki KOZAKAI]

Hi,

From: Patrick McHardy <kaber@trash.net>
Date: Fri, 14 Jan 2005 03:45:44 +0100

> Harald Welte wrote:
> 
> >the nf_conntrack approach of having multiple slab caches of differently
> >sized conntrack structures is better from a performance point of view.
> >
> >If you look at the typical applications, people will usually
> >
> >a) either use NAT on most/all of their connections
> >b) or not use NAT on any of their connections
> >
> >Now why don't they (the 'b' users) recompile their kernel? Because they
> >use vendor-kernels, and they compile with everything enabled.
> >
> >So what we end up if we use rusty's scheme for nat private data, is that
> >we have at least one additional allocation, and two disjunct seperate
> >pieces of memory (which will probably also introduce yet another cache
> >miss).   This is acceptable for rare exceptional cases, not for 99.9% of
> >your connections.
> >
> >So even if Rusty's architecture and API is cleaner, I'm very much in
> >favour of the nf_conntrack design and Yasuyuki's current
> >implementation.
> >  
> >
> But unlike ct_extend, the get_features stuff is very unflexible. The
> way it is currently implemented it only works for a single dynamically
> allocated part without leaving holes, and it even it if could handle more,
> they all need to be determined at allocation time, so a worst-case 
> assumption
> has to be made for anything depending on the ruleset. This is especially
> a concern with modules and distribution-kernels, NAT could be loaded at any
> time, so the memory needs to be allocated in advance.

ct_extend is definitely flexible. But at the same time, I agree to use
multiple slab caches if we can improve performance.

I haven't deeply read ct_extend codes, but can we combine it and cache scheme ?

For example, allocate conntrack from cache at first. We can roughly find out
its size by checking l3 protocol, loaded modules, finding helpers which want
to inspect packet, and so on. And if it is not enough, we can allocate more
piecies.

Anyway, I think we have many chance to optimize it later.

So why don't we port ct_extend to nf_conntrack and then optimize it with slab
cache ?

-----------------------------------------------------------------
Yasuyuki KOZAKAI @ USAGI Project <yasuyuki.kozakai@toshiba.co.jp>

nf_conntrack

[Yasuyuki Kozakai]

[-- Attachment #1: Type: Text/Plain, Size: 809 bytes --]


Hi, All,

This is other version of L3 independent connection tracking.

In previous version, the l3 part in tuple is represented by "union" of IPv4
and IPv6 address.

And in this version, size of the l3 part in tuple varies with layer 3 protocol.
Then it's more complex than the previous version, but can save memory spaces
where many IPv4 connections are created.

I'll measure the speed of 2 version of nf_conntrack from now.

BTW, Harald, Does this satisfy your requirements ?
And I don't know how large impact to cacheline when using "union".
Do you have any measurements ?

----------------------------------------
Yasuyuki KOZAKAI

Communication Platform Laboratory,
Corporate Research & Development Center,
Toshiba Corporation

yasuyuki.kozakai@toshiba.co.jp
----------------------------------------


[-- Attachment #2: nfct-dynamic-l3part.patch.bz2 --]
[-- Type: Application/Octet-Stream, Size: 26409 bytes --]

Re: nf_conntrack

[Harald Welte]

[-- Attachment #1: Type: text/plain, Size: 3377 bytes --]

[explicitly Cc'ing rusty and jozsef since they did a lot of conntrack
work]

On Wed, Dec 24, 2003 at 10:43:20AM +0900, Yasuyuki Kozakai wrote:
> 
> Hi, All,
> 
> This is other version of L3 independent connection tracking.

Thank you very much.  Sorry for this extremely late reply, but I missed
your email from december and just noted it after the current follow-up
on this issue.

> In previous version, the l3 part in tuple is represented by "union" of IPv4
> and IPv6 address.
> 
> And in this version, size of the l3 part in tuple varies with layer 3
> protocol.  Then it's more complex than the previous version, but can
> save memory spaces where many IPv4 connections are created.

Yes, this is definitely something we need.  Although I still don't like
the idea of having to explicitly allocate three pieces of memory
(2*tuple, 1*nf_conntrack) per new connection.  This will hurt connection
setup rate very badly, I expect :(

What I initially had in mind (but don't know if it can be implemented in
any reasonable way) was:

- have a static part of struct np_conntrack without tuples
- this static part contains a bitmask
- this bitmask determines which 'extension structures' are present at
  the end
- possible 'extension structures' might be:
	- ipv4 orig/reply tuple
	- ipv6 orig/reply tuple
	- ipv4 l3 protocol (and helper) data
	- ipv6 l3 protocol (and helper) data

I know it sounds very tricky.  We always guarantee the ordering of such
extension structures.  The nf_conntrack core would maintain a list of
slab caches for every reasonable combination of extensions structures.  
The extension structures are always allocated in one chunk gogether with
the static nf_conntrack 'header'.

Extension structures would always be accessed via pre-defined macros
(which just add an offset depending on the bitmask)

Even thinking further, this would enable us to put more parts than just
the tuple into those extensions (let's say the nat or helper union).
This in turn means that we can make conntrack structures for the normal
case (ipv4 only, no helper, no nat) even smaller.  

I'm not sure if it's worth performance-wise.  The benefits of such a
solution would be:

1) still one chunk of memory for any conntrack
	- one allocation at connection setup time
	- conntrack+tuple still physically close in memory instead of
	  having to dereference every tuple 
2) potentially smaller conntracks even in the current 'normal' case
   (one of our long-term wishes)

As disadvantage I can see
1) difficult implementation with lots of macros/offset calculation
2) we'd have to determine which slab we came from at destroy() time
   [maybe save cachep in nf_contrack?]
3) slightly higher computational cost when we want to access extension
   structures (like tuples), since relative offset has to be calculated
   at runtime.

What do you think?  

Rusty, Jozsef: Do you think it's worth the effort?
 
> Yasuyuki KOZAKAI

-- 
- Harald Welte <laforge@netfilter.org>             http://www.netfilter.org/
============================================================================
  "Fragmentation is like classful addressing -- an interesting early
   architectural error that shows how much experimentation was going
   on while IP was being designed."                    -- Paul Vixie

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: nf_conntrack

[Yasuyuki Kozakai]

Hi, Harald, thank you for replying.

Sory about long message and poor english.

> > In previous version, the l3 part in tuple is represented by "union" of IPv4
> > and IPv6 address.
> > 
> > And in this version, size of the l3 part in tuple varies with layer 3
> > protocol.  Then it's more complex than the previous version, but can
> > save memory spaces where many IPv4 connections are created.
> 
> Yes, this is definitely something we need.  Although I still don't like
> the idea of having to explicitly allocate three pieces of memory
> (2*tuple, 1*nf_conntrack) per new connection.  This will hurt connection
> setup rate very badly, I expect :(

No.

When L3 protocol modules are registered, my implementation of
"dynamic version" calculate the size of 2 tuple + nf_conntrack, and
create caches. So

	$ cat /proc/slabinfo |grep nf_conntrack

prints	"nf_conntrack:ipv4" and "nf_conntrack:ipv6".

1 chunk includes static part of nf_conntrack and 2 tuples.
But memory spaces for helper, nat are statically allocated.
Because I wanted to see the impact of dynamic allocating memory space
for tuple, at first.

> What I initially had in mind (but don't know if it can be implemented in
> any reasonable way) was:
> 
> - have a static part of struct np_conntrack without tuples
> - this static part contains a bitmask
> - this bitmask determines which 'extension structures' are present at
>   the end
> - possible 'extension structures' might be:
> 	- ipv4 orig/reply tuple
> 	- ipv6 orig/reply tuple
> 	- ipv4 l3 protocol (and helper) data
> 	- ipv6 l3 protocol (and helper) data

Your idea about extension structure (L3/helper/nat union) is acceptable.

But I think tuple of IPv{4,6} should be represented by one structure
-- "struct nf_conntrack_tuple".

Your idea seems that ip_conntrack_tuple and ip6_conntrack_tuple are defined
individually. Please see nf_conntrack_proto_tcp.c in my implementation.
All functions can handle both tuples for IPv{4,6} . But if ip_conntrack_tuple
and ip6_conntrack_tuple are defined individually, we have to define
2 version of each functions which do same thing and have just different
type of argument "tuple".

	ex) ip_tcp_print_conntrack(ip_conntrack_tuple *tuple);
	    ip6_tcp_print_conntrack(ip6_conntrack_tuple *tuple);

This does't mean "Layer 3 independent".

BTW, unfortunately, I can't spend time for this work this 2 months.
Recently I changed my code to fairly compare 2 implementations, and roughly
evaluated. As I expected, the performance of "dynamic version" is more worse
than "static version".

So I concluded it is desirable to be statically defined the structure of tuple.

> I know it sounds very tricky.  We always guarantee the ordering of such
> extension structures.  The nf_conntrack core would maintain a list of
> slab caches for every reasonable combination of extensions structures.  
> The extension structures are always allocated in one chunk gogether with
> the static nf_conntrack 'header'.
> 
> Extension structures would always be accessed via pre-defined macros
> (which just add an offset depending on the bitmask)

It is efficient to keep the pointer to extension structure when allocating
nf_conntrack instead of calculating offset every access to it.

> Even thinking further, this would enable us to put more parts than just
> the tuple into those extensions (let's say the nat or helper union).
> This in turn means that we can make conntrack structures for the normal
> case (ipv4 only, no helper, no nat) even smaller.  
> 
> I'm not sure if it's worth performance-wise.  The benefits of such a
> solution would be:
> 
> 1) still one chunk of memory for any conntrack
> 	- one allocation at connection setup time
> 	- conntrack+tuple still physically close in memory instead of
> 	  having to dereference every tuple 
> 2) potentially smaller conntracks even in the current 'normal' case
>    (one of our long-term wishes)
> 
> As disadvantage I can see
> 1) difficult implementation with lots of macros/offset calculation
> 2) we'd have to determine which slab we came from at destroy() time
>    [maybe save cachep in nf_contrack?]
> 3) slightly higher computational cost when we want to access extension
>    structures (like tuples), since relative offset has to be calculated
>    at runtime.
> 
> What do you think?  

I think processing speed is more important for connection tracking than
saving memory space. But you got a point that unused memory space should
not be allocated.

my opinions are

1. It is desirable to be statically defined the structure of tuple.
   Because
	- tuple should be represented by one structure -- nf_conntrack_tuple
	  to get tcp/udp/icmp part in tuple independent of Layer 3 protocol.

	- I can't agree with dynamic structure of tuple due to performance
	  issue and difficulty of handling.

2. Your idea about extension structure (L3/helper/nat union) is acceptable.
   I expect that this impact to performance is small.But I worry about
   slowdown in the case of "nat helper union". When allocating conntrack,
   it is difficult to figure out whether conntrack will be processed or not
   by nat helper module. So I expect that it is needed to reallocate and copy
   conntrack before nat helper module processes it.

   On the other hand, there is no problem about "nat union" (= ip_nat_info).
   Because if nat module is loaded, all conntrack certainly have "null-binding"
   or "binding" which specified by user. So conntrack core module always
   need to allocate memory space for "nat union" in this case.

Regards,

-----------------------------------------------------------------
Yasuyuki KOZAKAI @ USAGI Project <yasuyuki.kozakai@toshiba.co.jp>

Re: nf_conntrack

[Harald Welte]

[-- Attachment #1: Type: text/plain, Size: 5048 bytes --]

On Wed, Feb 25, 2004 at 09:47:26PM +0900, Yasuyuki Kozakai wrote:
> When L3 protocol modules are registered, my implementation of
> "dynamic version" calculate the size of 2 tuple + nf_conntrack, and
> create caches. So

Oh sorry, I should have noticed that.  

> 	$ cat /proc/slabinfo |grep nf_conntrack
> 
> prints	"nf_conntrack:ipv4" and "nf_conntrack:ipv6".
> 
> 1 chunk includes static part of nf_conntrack and 2 tuples.
> But memory spaces for helper, nat are statically allocated.
> Because I wanted to see the impact of dynamic allocating memory space
> for tuple, at first.

I think this is fine.  The 'extension structures' would be a further
optimization that are totally unrelated to ipv4/ipv6 generalization and
don't really fit this discussion then.

> But I think tuple of IPv{4,6} should be represented by one structure
> -- "struct nf_conntrack_tuple".

agreed.

> Your idea seems that ip_conntrack_tuple and ip6_conntrack_tuple are defined
> individually. Please see nf_conntrack_proto_tcp.c in my implementation.
> All functions can handle both tuples for IPv{4,6} . But if ip_conntrack_tuple
> and ip6_conntrack_tuple are defined individually, we have to define
> 2 version of each functions which do same thing and have just different
> type of argument "tuple".

I see, again agreed.

> This does't mean "Layer 3 independent".

;) well at least not for layer 4 protocol helpers.

> BTW, unfortunately, I can't spend time for this work this 2 months.

No problem, we're all busy with other stuff anyway.  But I think we
should finish the mising parts (I've read some FIXME's with regard to
helpers, are they still valid?) at about the time that 2.7.x forks off. 

Could you write a small TODO list containing the Items that will yet

btw: I will put your current patch into patch-o-matic-ng. This means it
will probably get some more users.

> So I concluded it is desirable to be statically defined the structure
> of tuple.

Yes, indeed.

> It is efficient to keep the pointer to extension structure when allocating
> nf_conntrack instead of calculating offset every access to it.

Yes, that is an option.

> I think processing speed is more important for connection tracking than
> saving memory space. But you got a point that unused memory space should
> not be allocated.

The issue is mostly about wasting cache lines.  Memory is cheap, but
cache is very limited... ip_conntrack is currently 304 bytes of size.
If we could somehow get this down to 256 bytes, we would save a whole 
cache line per connection, that's quite a lot.

Also, we have increasing demands for adding information to the conntrack
table.  Some people want to add counters in order to facilitate
connection based accounting.

> my opinions are
> 
> 1. It is desirable to be statically defined the structure of tuple.

ok, you convinced me ;)

> 2. Your idea about extension structure (L3/helper/nat union) is acceptable.
>    I expect that this impact to performance is small.But I worry about
>    slowdown in the case of "nat helper union". When allocating conntrack,
>    it is difficult to figure out whether conntrack will be processed or not
>    by nat helper module. So I expect that it is needed to reallocate and copy
>    conntrack before nat helper module processes it.

it's actually not that difficult.  When we allocate a conntrack, we
already know the tuple.  And from the tuple, we already know if there is
a conntrack helper.  So let's just assume if iptable_nat is loaded and
we have a conntrack helper responsible for this tuple, we will later
need the nat helper union.  There are few protocols that only have a nat
helper but no conntrack helper.  In that case one would have to either
re-allocate, or force them to have a dummy conntrack helper.

We really should optimize for the non-nat non-helper case, since it is
by far the most common case.  It's just a pity that we're currently
wasting so much memory/cache for 90% of all connections that don't even

>    On the other hand, there is no problem about "nat union" (= ip_nat_info).
>    Because if nat module is loaded, all conntrack certainly have
>    "null-binding" or "binding" which specified by user. So conntrack
>    core module always need to allocate memory space for "nat union" in
>    this case.

Yes, but what happens in cases where you alrady have entries in the
conntrack table at the time of loading iptable_nat ?  Then the old
conntracks don't have the nat union, ... - but I think these are minor
issues that could be resolved.

> Regards,

Thanks for your feedback.

-- 
- Harald Welte <laforge@netfilter.org>             http://www.netfilter.org/
============================================================================
  "Fragmentation is like classful addressing -- an interesting early
   architectural error that shows how much experimentation was going
   on while IP was being designed."                    -- Paul Vixie

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: nf_conntrack

[Yasuyuki Kozakai]

Hi, Harald,

From: Harald Welte <laforge@netfilter.org>
Date: Wed, 25 Feb 2004 16:05:54 +0100

> > 1 chunk includes static part of nf_conntrack and 2 tuples.
> > But memory spaces for helper, nat are statically allocated.
> > Because I wanted to see the impact of dynamic allocating memory space
> > for tuple, at first.
> 
> I think this is fine.  The 'extension structures' would be a further
> optimization that are totally unrelated to ipv4/ipv6 generalization and
> don't really fit this discussion then.

I see.

> > BTW, unfortunately, I can't spend time for this work this 2 months.
> 
> No problem, we're all busy with other stuff anyway.  But I think we
> should finish the mising parts (I've read some FIXME's with regard to
> helpers, are they still valid?) at about the time that 2.7.x forks off. 
> 
> Could you write a small TODO list containing the Items that will yet

ok.
	1. change to allocate nf_conntrack with various size depended
	   on features.
	2. implement the code handling fragmented packets
	3. support for various helper. At least, helpers in current main
	   kernel.
	4. apply fixes and changes for latest ip_conntrack.
	5. wrapper for current ip_conntrack and ip_nat, if I have any time.
	   (or is it more efficient to change all nat and helper modules ?)

> btw: I will put your current patch into patch-o-matic-ng. This means it
> will probably get some more users.

Thank you. Please notice that I'll implement 1 in TODO list based on
"static size of nf_conntrack" version. So I'll change many codes.

> > I think processing speed is more important for connection tracking than
> > saving memory space. But you got a point that unused memory space should
> > not be allocated.
> 
> The issue is mostly about wasting cache lines.  Memory is cheap, but
> cache is very limited... ip_conntrack is currently 304 bytes of size.
> If we could somehow get this down to 256 bytes, we would save a whole 
> cache line per connection, that's quite a lot.

Oh, sorry about missunderstading. I got it.

> it's actually not that difficult.  When we allocate a conntrack, we
> already know the tuple.  And from the tuple, we already know if there is
> a conntrack helper.  So let's just assume if iptable_nat is loaded and
> we have a conntrack helper responsible for this tuple, we will later
> need the nat helper union.  There are few protocols that only have a nat
> helper but no conntrack helper.  In that case one would have to either
> re-allocate, or force them to have a dummy conntrack helper.

I agree.

> >    On the other hand, there is no problem about "nat union" (= ip_nat_info).
> >    Because if nat module is loaded, all conntrack certainly have
> >    "null-binding" or "binding" which specified by user. So conntrack
> >    core module always need to allocate memory space for "nat union" in
> >    this case.
> 
> Yes, but what happens in cases where you alrady have entries in the
> conntrack table at the time of loading iptable_nat ?  Then the old
> conntracks don't have the nat union, ... - but I think these are minor
> issues that could be resolved.

I imagine that bit flags is checked and conntrack is reallocated if needed.
The cost is small because reallocating is needed only once per conntrack
processed by nat helper.

In the case that nat module is unloaded, cache isn't destroyed until
all nat enabled conntrack is free.

Thank you for your help and suggestions.

-----------------------------------------------------------------
Yasuyuki KOZAKAI @ USAGI Project <yasuyuki.kozakai@toshiba.co.jp>

Re: nf_conntrack

[Jozsef Kadlecsik]

Hi,

On Tue, 24 Feb 2004, Harald Welte wrote:

> > In previous version, the l3 part in tuple is represented by "union" of IPv4
> > and IPv6 address.
> >
> > And in this version, size of the l3 part in tuple varies with layer 3
> > protocol.  Then it's more complex than the previous version, but can
> > save memory spaces where many IPv4 connections are created.
>
> Yes, this is definitely something we need.  Although I still don't like
> the idea of having to explicitly allocate three pieces of memory
> (2*tuple, 1*nf_conntrack) per new connection.  This will hurt connection
> setup rate very badly, I expect :(

I don't like allocating in chunks either.

> What I initially had in mind (but don't know if it can be implemented in
> any reasonable way) was:
>
> - have a static part of struct np_conntrack without tuples
> - this static part contains a bitmask
> - this bitmask determines which 'extension structures' are present at
>   the end
> - possible 'extension structures' might be:
> 	- ipv4 orig/reply tuple
> 	- ipv6 orig/reply tuple
> 	- ipv4 l3 protocol (and helper) data
> 	- ipv6 l3 protocol (and helper) data
>
> I know it sounds very tricky.  We always guarantee the ordering of such
> extension structures.  The nf_conntrack core would maintain a list of
> slab caches for every reasonable combination of extensions structures.
> The extension structures are always allocated in one chunk gogether with
> the static nf_conntrack 'header'.
>
> Extension structures would always be accessed via pre-defined macros
> (which just add an offset depending on the bitmask)
>
> Even thinking further, this would enable us to put more parts than just
> the tuple into those extensions (let's say the nat or helper union).
> This in turn means that we can make conntrack structures for the normal
> case (ipv4 only, no helper, no nat) even smaller.
>
> I'm not sure if it's worth performance-wise.  The benefits of such a
> solution would be:
>
> 1) still one chunk of memory for any conntrack
> 	- one allocation at connection setup time
> 	- conntrack+tuple still physically close in memory instead of
> 	  having to dereference every tuple
> 2) potentially smaller conntracks even in the current 'normal' case
>    (one of our long-term wishes)
>
> As disadvantage I can see
> 1) difficult implementation with lots of macros/offset calculation
> 2) we'd have to determine which slab we came from at destroy() time
>    [maybe save cachep in nf_contrack?]
> 3) slightly higher computational cost when we want to access extension
>    structures (like tuples), since relative offset has to be calculated
>    at runtime.
>
> What do you think?

Definitely this is the way we should go.

Even NAT-PT could be fitted into it, with the overhead of allocating
new conntrack entries at NAT-PT. Maybe Henrik's suggestion to use a
special tun device would be more optimal from performance point of view,
with the cost of two conntrack entries for each protocol-translated
session.

Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlec@sunserv.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary