Select chain from set?

Select chain from set?

[Oskar Berggren]

Hello,

is it possible to have iptables query a set of some sort to quickly
look up a chain to jump to for a specific packet?

For example the set would contain a mapping from ip-address ->
chain-name. Then the destination address is looked up in the set and
iptables would jump to the specified chain.

Is it possible, or how difficult would it be to implement something
like this? I'd imagine it would be implemented as a target, that would
have the ability to "redirect" to another target, which would be a
chain.


/Oskar

Re: Select chain from set?

[Martin Millnert]

[-- Attachment #1: Type: text/plain, Size: 1801 bytes --]

On Tue, 2009-04-28 at 11:21 +0200, Oskar Berggren wrote:
> Hello,
> 
> is it possible to have iptables query a set of some sort to quickly
> look up a chain to jump to for a specific packet?
> 
> For example the set would contain a mapping from ip-address ->
> chain-name. Then the destination address is looked up in the set and
> iptables would jump to the specified chain.
> 
> Is it possible, or how difficult would it be to implement something
> like this? I'd imagine it would be implemented as a target, that would
> have the ability to "redirect" to another target, which would be a
> chain.
> 
> 
> /Oskar

Hi Oskar,

to my knowledge there is no current such functionality. But I second the
usefulness of such a function. It would be especially useful to replace
traditional "match trees" of today, to reach down to a per
customer-chain for inwards and outwards matching.

It seems like it would be beneficial to let it be very similar to the
ipset of today, in that you would be very well of in having multiple
such sets to jump to as a target.  Thus you could easily use normal
matchings to decide whether or not to perform this sort of look-up.
  It would be useful if each such set would contain the possibility to
match on either source address/netmask and destination address/netmask.

This all begs the question on how effective some tree structure with -g
is implemented, to figure out how much of a performance benefit such a
new target would have over a treelike chain structure.  

It would indeed be useful to have it all as a single target however,
save a lot of rules and also be much more readable.

Is it possible to easily extend ipset to perform this?

More input appreciated.

Regards,
-- 
Martin Millnert <millnert@csbnet.se>

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 835 bytes --]

Re: Select chain from set?

[Покотиленко Костик]

В Вто, 28/04/2009 в 14:27 +0200, Martin Millnert пишет:
> On Tue, 2009-04-28 at 11:21 +0200, Oskar Berggren wrote:
> > Hello,
> > 
> > is it possible to have iptables query a set of some sort to quickly
> > look up a chain to jump to for a specific packet?
> > 
> > For example the set would contain a mapping from ip-address ->
> > chain-name. Then the destination address is looked up in the set and
> > iptables would jump to the specified chain.
> > 
> > Is it possible, or how difficult would it be to implement something
> > like this? I'd imagine it would be implemented as a target, that would
> > have the ability to "redirect" to another target, which would be a
> > chain.
> > 
> > 
> > /Oskar
> 
> Hi Oskar,
> 
> to my knowledge there is no current such functionality. But I second the
> usefulness of such a function. It would be especially useful to replace
> traditional "match trees" of today, to reach down to a per
> customer-chain for inwards and outwards matching.
> 
> It seems like it would be beneficial to let it be very similar to the
> ipset of today, in that you would be very well of in having multiple
> such sets to jump to as a target.  Thus you could easily use normal
> matchings to decide whether or not to perform this sort of look-up.
>   It would be useful if each such set would contain the possibility to
> match on either source address/netmask and destination address/netmask.

This functionality is more related to a special target than ipset
feature. -j CLASSIFY does half of the work and as to me is the most
ideal to be taken as the base for the new target.

> This all begs the question on how effective some tree structure with -g
> is implemented, to figure out how much of a performance benefit such a
> new target would have over a treelike chain structure.  

If we compare many linear -g with just one function gettarget(ip) the
different is many/one. Tree-like -g structure would save most
comparitions, but is hard to write for every task. Function-like target
is real fast and fully automatic, the only disadvantage is in fact it
doesn't exist :)

> It would indeed be useful to have it all as a single target however,
> save a lot of rules and also be much more readable.

Also, something is telling me that if the deal is comming to write new
target it is good idea to review the whole task as it probably can be
solved in a more effective way.

-- 
Покотиленко Костик <casper@meteor.dp.ua>

Re: Select chain from set?

[Susan Hinrichs]

I also agree that a runtime structure to track traffic attributes and
match them to targets would be great.  I created my own match-tree table
generator to achieve a similar effect.  It works, but updating large
static structures can be rather time consuming and fragile.

I have a question about the '-g' terminology used by Casper and Oscar.
Is this a new piece of functionality?  Or are you talking about the
--goto option?

Susan

> > This all begs the question on how effective some tree structure with -g
> > is implemented, to figure out how much of a performance benefit such a
> > new target would have over a treelike chain structure.  
> 
> If we compare many linear -g with just one function gettarget(ip) the
> different is many/one. Tree-like -g structure would save most
> comparitions, but is hard to write for every task. Function-like target
> is real fast and fully automatic, the only disadvantage is in fact it
> doesn't exist :)
> 

Re: Select chain from set?

[Покотиленко Костик]

В Вто, 28/04/2009 в 10:39 -0500, Susan Hinrichs пишет:
> I also agree that a runtime structure to track traffic attributes and
> match them to targets would be great.  I created my own match-tree table
> generator to achieve a similar effect.  It works, but updating large
> static structures can be rather time consuming and fragile.

Can you share details?

> I have a question about the '-g' terminology used by Casper and Oscar.
> Is this a new piece of functionality?  Or are you talking about the
> --goto option?

Personaly, I was talking about --goto.

> > > This all begs the question on how effective some tree structure with -g
> > > is implemented, to figure out how much of a performance benefit such a
> > > new target would have over a treelike chain structure.  
> > 
> > If we compare many linear -g with just one function gettarget(ip) the
> > different is many/one. Tree-like -g structure would save most
> > comparitions, but is hard to write for every task. Function-like target
> > is real fast and fully automatic, the only disadvantage is in fact it
> > doesn't exist :)
> > 
> 
> 
-- 
Покотиленко Костик <casper@meteor.dp.ua>

Re: Select chain from set?

[Susan Hinrichs]

On Wed, 2009-04-29 at 11:11 +0300, Покотиленко Костик wrote:
> В Вто, 28/04/2009 в 10:39 -0500, Susan Hinrichs пишет:
> > I also agree that a runtime structure to track traffic attributes and
> > match them to targets would be great.  I created my own match-tree table
> > generator to achieve a similar effect.  It works, but updating large
> > static structures can be rather time consuming and fragile.
> 
> Can you share details?
> 

Sure, I have a tool that takes a list of IP's, MACs, or marks, and
builds a prefix-based binary tree of the data.  It generates the tree in
linked chains.  It operates in bulk and incremental model.

I didn't consider the gotos in generating the output, but I can see now
that it might simplify the tree flow quite a bit.

I've done some very basic latency tests using ping, and the tree
introduced less than 2/3 the latency vs the a linear case for around 800
elements, and just 20% more latency than an empty iptable policy.

Re: Select chain from set?

[Покотиленко Костик]

В Чтв, 30/04/2009 в 10:52 -0500, Susan Hinrichs пишет:
> On Wed, 2009-04-29 at 11:11 +0300, Покотиленко Костик wrote:
> > В Вто, 28/04/2009 в 10:39 -0500, Susan Hinrichs пишет:
> > > I also agree that a runtime structure to track traffic attributes and
> > > match them to targets would be great.  I created my own match-tree table
> > > generator to achieve a similar effect.  It works, but updating large
> > > static structures can be rather time consuming and fragile.
> > 
> > Can you share details?
> > 
> 
> Sure, I have a tool that takes a list of IP's, MACs, or marks, and
> builds a prefix-based binary tree of the data.  It generates the tree in
> linked chains.  It operates in bulk and incremental model.

What is the purpose of this?

> I didn't consider the gotos in generating the output, but I can see now
> that it might simplify the tree flow quite a bit.
> 
> I've done some very basic latency tests using ping, and the tree
> introduced less than 2/3 the latency vs the a linear case for around 800
> elements, and just 20% more latency than an empty iptable policy.

-- 
Покотиленко Костик <casper@meteor.dp.ua>

Re: Select chain from set?

[Susan Hinrichs]

On Thu, 2009-05-07 at 13:07 +0300, Покотиленко Костик wrote:
> В Чтв, 30/04/2009 в 10:52 -0500, Susan Hinrichs пишет:
> > On Wed, 2009-04-29 at 11:11 +0300, Покотиленко Костик wrote:
> > > В Вто, 28/04/2009 в 10:39 -0500, Susan Hinrichs пишет:
> > > > I also agree that a runtime structure to track traffic attributes and
> > > > match them to targets would be great.  I created my own match-tree table
> > > > generator to achieve a similar effect.  It works, but updating large
> > > > static structures can be rather time consuming and fragile.
> > > 
> > > Can you share details?
> > > 
> > 
> > Sure, I have a tool that takes a list of IP's, MACs, or marks, and
> > builds a prefix-based binary tree of the data.  It generates the tree in
> > linked chains.  It operates in bulk and incremental model.
> 
> What is the purpose of this?

The tree lets you efficiently match a packet and do something unique for
each "client" or "grouping".  So set a mark or set a class ID or update
a unique recent set. As was noted in this thread earlier, ipset lets you
efficiently match a packet basic on an address, but it doesn't let you
do anything unique for each match. 

Re: Select chain from set?

[Thomas Jacob]

On Thu, 2009-05-07 at 08:44 -0500, Susan Hinrichs wrote:
> On Thu, 2009-05-07 at 13:07 +0300, Покотиленко Костик wrote:
> > В Чтв, 30/04/2009 в 10:52 -0500, Susan Hinrichs пишет:
> > > On Wed, 2009-04-29 at 11:11 +0300, Покотиленко Костик wrote:
> > > > В Вто, 28/04/2009 в 10:39 -0500, Susan Hinrichs пишет:
> > > > > I also agree that a runtime structure to track traffic attributes and
> > > > > match them to targets would be great.  I created my own match-tree table
> > > > > generator to achieve a similar effect.  It works, but updating large
> > > > > static structures can be rather time consuming and fragile.
> > > > 
> > > > Can you share details?
> > > > 
> > > 
> > > Sure, I have a tool that takes a list of IP's, MACs, or marks, and
> > > builds a prefix-based binary tree of the data.  It generates the tree in
> > > linked chains.  It operates in bulk and incremental model.
> > 
> > What is the purpose of this?
> 
> The tree lets you efficiently match a packet and do something unique for
> each "client" or "grouping".  So set a mark or set a class ID or update
> a unique recent set. As was noted in this thread earlier, ipset lets you
> efficiently match a packet basic on an address, but it doesn't let you
> do anything unique for each match. 

Nftables will let you do that in the future

http://lwn.net/Articles/324251/

Re: Select chain from set?

[Susan Hinrichs]

On Thu, 2009-05-07 at 16:01 +0200, Thomas Jacob wrote:
> On Thu, 2009-05-07 at 08:44 -0500, Susan Hinrichs wrote:
> > On Thu, 2009-05-07 at 13:07 +0300, Покотиленко Костик wrote:
> > > В Чтв, 30/04/2009 в 10:52 -0500, Susan Hinrichs пишет:
> > > > On Wed, 2009-04-29 at 11:11 +0300, Покотиленко Костик wrote:
> > > > > В Вто, 28/04/2009 в 10:39 -0500, Susan Hinrichs пишет:
> > > > > > I also agree that a runtime structure to track traffic attributes and
> > > > > > match them to targets would be great.  I created my own match-tree table
> > > > > > generator to achieve a similar effect.  It works, but updating large
> > > > > > static structures can be rather time consuming and fragile.
> > > > > 
> > > > > Can you share details?
> > > > > 
> > > > 
> > > > Sure, I have a tool that takes a list of IP's, MACs, or marks, and
> > > > builds a prefix-based binary tree of the data.  It generates the tree in
> > > > linked chains.  It operates in bulk and incremental model.
> > > 
> > > What is the purpose of this?
> > 
> > The tree lets you efficiently match a packet and do something unique for
> > each "client" or "grouping".  So set a mark or set a class ID or update
> > a unique recent set. As was noted in this thread earlier, ipset lets you
> > efficiently match a packet basic on an address, but it doesn't let you
> > do anything unique for each match. 
> 
> Nftables will let you do that in the future
> 
> http://lwn.net/Articles/324251/
> 

Great! Looking forward to it.  The dictionaries look great.  I'll have
to start playing with the first version on a test machine.  Do you know
what kind of MAC address support there is?  Similar to the source mac
support in iptables?

Re: Select chain from set?

[Thomas Jacob]

On Thu, 2009-05-07 at 09:31 -0500, Susan Hinrichs wrote:
> > Nftables will let you do that in the future
> > 
> > http://lwn.net/Articles/324251/
> > 
> 
> Great! Looking forward to it.  The dictionaries look great.  I'll have
> to start playing with the first version on a test machine.  Do you know
> what kind of MAC address support there is?  Similar to the source mac
> support in iptables?

Sorry, can't help you there. It's probably best to ask concrete question
about the planned features of nftables on netfilter-devel list, AFAIK
it's still under heavy development. But as far as I understand the
dictionary concept, mac address matching should become available at
one point. 

" Sets (as everything else) operate on generic
data and thus can be used for any kind of match."

Until nftables becomes stable enough for production I myself am sticking
with the chain tree approach (and ipset for simple IP match sets)

Re: Select chain from set?

[Покотиленко Костик]

В Чтв, 07/05/2009 в 16:01 +0200, Thomas Jacob пишет:
> On Thu, 2009-05-07 at 08:44 -0500, Susan Hinrichs wrote:
> > On Thu, 2009-05-07 at 13:07 +0300, Покотиленко Костик wrote:
> > > В Чтв, 30/04/2009 в 10:52 -0500, Susan Hinrichs пишет:
> > > > On Wed, 2009-04-29 at 11:11 +0300, Покотиленко Костик wrote:
> > > > > В Вто, 28/04/2009 в 10:39 -0500, Susan Hinrichs пишет:
> > > > > > I also agree that a runtime structure to track traffic attributes and
> > > > > > match them to targets would be great.  I created my own match-tree table
> > > > > > generator to achieve a similar effect.  It works, but updating large
> > > > > > static structures can be rather time consuming and fragile.
> > > > > 
> > > > > Can you share details?
> > > > > 
> > > > 
> > > > Sure, I have a tool that takes a list of IP's, MACs, or marks, and
> > > > builds a prefix-based binary tree of the data.  It generates the tree in
> > > > linked chains.  It operates in bulk and incremental model.
> > > 
> > > What is the purpose of this?
> > 
> > The tree lets you efficiently match a packet and do something unique for
> > each "client" or "grouping".  So set a mark or set a class ID or update
> > a unique recent set. As was noted in this thread earlier, ipset lets you
> > efficiently match a packet basic on an address, but it doesn't let you
> > do anything unique for each match. 
> 
> Nftables will let you do that in the future
> 
> http://lwn.net/Articles/324251/

This might save our world :)
I would say from the anounce this is what you have wanted from iptables.

Thanks for link.

-- 
Покотиленко Костик <casper@meteor.dp.ua>