Use connection tracking to store private data for rate shaping algorithms?

Use connection tracking to store private data for rate shaping algorithms?

[Dirk]

Hello,


We are developing some some more advanced TCP rate shaping algorithms
( http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1261834&tag=1 )
and would like to implement it as a (GPL) kernel module. The goal is
to have a high performant module that can 'shape' traffic for 10 000's
of hosts and users on individual base and also minimize shaping
overhead (drop/retransmits/latency) on the network.

Our current solution uses a custom developed iptables target that
already performs traffic policing for 10'000s of simultaneous users (
http://tnc2007.terena.org/programme/presentations/showbeb6.html ).
Since policing is not an ideal way to regulate bandwidth, we want to
improve it :-)


However, in order to function correctly, we need to keep some extra
information for each flow (both tcp and udp) through the device:
- few statistics (two 64bit integers)
- shaping/rate state information (four 64bit integers)

 This information should be set/accessible from a kernel tc qdisc module.

My idea was to use the connection tracking framework to keep track of
connection states (which is also required by the algorithms) and
somehow extend it to also store the extra information.

I have found there is an extension infrastructure for nf_conntrack (
http://www.mail-archive.com/git-commits-head@vger.kernel.org/msg15798.html
), but it does not seem a module is intended to register itself
without modifying nf_ct_ext_id in /nf_conntrack_extend.h. Since that
would require a kernel recompile, it is not really an option.


Since we're not (yet) familiar with the connection tracking code: what
would be the best way to accomplish this?



Thanks in advance,
Dirk Janssens

Re: Use connection tracking to store private data for rate shaping algorithms?

[Pablo Neira Ayuso]

On 30/11/10 16:18, Dirk wrote:
> Hello,
> 
> 
> We are developing some some more advanced TCP rate shaping algorithms
> ( http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1261834&tag=1 )
> and would like to implement it as a (GPL) kernel module. The goal is
> to have a high performant module that can 'shape' traffic for 10 000's
> of hosts and users on individual base and also minimize shaping
> overhead (drop/retransmits/latency) on the network.
> 
> Our current solution uses a custom developed iptables target that
> already performs traffic policing for 10'000s of simultaneous users (
> http://tnc2007.terena.org/programme/presentations/showbeb6.html ).
> Since policing is not an ideal way to regulate bandwidth, we want to
> improve it :-)
> 
> 
> However, in order to function correctly, we need to keep some extra
> information for each flow (both tcp and udp) through the device:
> - few statistics (two 64bit integers)
> - shaping/rate state information (four 64bit integers)
> 
>  This information should be set/accessible from a kernel tc qdisc module.
> 
> My idea was to use the connection tracking framework to keep track of
> connection states (which is also required by the algorithms) and
> somehow extend it to also store the extra information.
> 
> I have found there is an extension infrastructure for nf_conntrack (
> http://www.mail-archive.com/git-commits-head@vger.kernel.org/msg15798.html
> ), but it does not seem a module is intended to register itself
> without modifying nf_ct_ext_id in /nf_conntrack_extend.h. Since that
> would require a kernel recompile, it is not really an option.
> 
> Since we're not (yet) familiar with the connection tracking code: what
> would be the best way to accomplish this?

Then, you should do similar to this patch to add a new ct extension in
kernel-space:

http://www.spinics.net/lists/netfilter-devel/msg15320.html

Re: Use connection tracking to store private data for rate shaping algorithms?

[Patrick McHardy]

Am 30.11.2010 16:18, schrieb Dirk:
> Hello,
> 
> 
> We are developing some some more advanced TCP rate shaping algorithms
> ( http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1261834&tag=1 )
> and would like to implement it as a (GPL) kernel module. The goal is
> to have a high performant module that can 'shape' traffic for 10 000's
> of hosts and users on individual base and also minimize shaping
> overhead (drop/retransmits/latency) on the network.

Sounds interesting.

> Our current solution uses a custom developed iptables target that
> already performs traffic policing for 10'000s of simultaneous users (
> http://tnc2007.terena.org/programme/presentations/showbeb6.html ).
> Since policing is not an ideal way to regulate bandwidth, we want to
> improve it :-)
> 
> 
> However, in order to function correctly, we need to keep some extra
> information for each flow (both tcp and udp) through the device:
> - few statistics (two 64bit integers)
> - shaping/rate state information (four 64bit integers)
> 
>  This information should be set/accessible from a kernel tc qdisc module.
> 
> My idea was to use the connection tracking framework to keep track of
> connection states (which is also required by the algorithms) and
> somehow extend it to also store the extra information.

That sounds reasonable. I've implemented TCP rate control about 9
years ago and did the same.

> I have found there is an extension infrastructure for nf_conntrack (
> http://www.mail-archive.com/git-commits-head@vger.kernel.org/msg15798.html
> ), but it does not seem a module is intended to register itself
> without modifying nf_ct_ext_id in /nf_conntrack_extend.h. Since that
> would require a kernel recompile, it is not really an option.
> 
> Since we're not (yet) familiar with the connection tracking code: what
> would be the best way to accomplish this?

You can't add data to the conntrack structure (neither directly nor
through extensions) without recompiling the kernel. I'd suggest to
use ct extensions, require a recompile for now and then merge your
module upstream once it's finished.