[nf_tables] suggestion: system-wide sets

[nf_tables] suggestion: system-wide sets

[Arturo Borrero Gonzalez]

Hi there!

I can't remember this subject being discussed in the near past.

Why not using system-wide sets? Or at least, family-wide sets?

From the user point of view, I think is very interesting to define
sets that can be used in rules all across the ruleset, same set in
different tables and families.

Being family-wide is what ipset does, and I'm sure is what most people
will expect.

On the other hand, I understand this would be a major change at this point.

-- 
Arturo Borrero González
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [nf_tables] suggestion: system-wide sets

[Patrick McHardy]

On Thu, Feb 27, 2014 at 01:15:36PM +0100, Arturo Borrero Gonzalez wrote:
> Hi there!
> 
> I can't remember this subject being discussed in the near past.
> 
> Why not using system-wide sets? Or at least, family-wide sets?
> 
> >From the user point of view, I think is very interesting to define
> sets that can be used in rules all across the ruleset, same set in
> different tables and families.

Yeah, I agree. I think family wide sets and global (AF_UNSPEC) sets should
bet quite easy to add. However there's the question how to expose them
in the nft list table output. The idea is to be able to recreate the
current ruleset, including sets and elements, by parsing the output of
nft list table. If we don't include sets, the user will have to seperately
save and restore them. OTOH if we simply include global and AF-specific
sets, they will be restored once for each table and this will fail on
the second table.

Any other ideas?

> Being family-wide is what ipset does, and I'm sure is what most people
> will expect.
> 
> On the other hand, I understand this would be a major change at this point.

Should be fine. I'm thinking of keeping the table specific sets and also
having per AF and AF_UNSPEC sets. For maps we should restrict this feature
to data maps since verdicts may need a table context.

Re: [nf_tables] suggestion: system-wide sets

[Arturo Borrero Gonzalez]

On 27 February 2014 15:02, Patrick McHardy <kaber@trash.net> wrote:
> On Thu, Feb 27, 2014 at 01:15:36PM +0100, Arturo Borrero Gonzalez wrote:
>> Hi there!
>>
>> I can't remember this subject being discussed in the near past.
>>
>> Why not using system-wide sets? Or at least, family-wide sets?
>>
>> >From the user point of view, I think is very interesting to define
>> sets that can be used in rules all across the ruleset, same set in
>> different tables and families.
>
> Yeah, I agree. I think family wide sets and global (AF_UNSPEC) sets should
> bet quite easy to add. However there's the question how to expose them
> in the nft list table output. The idea is to be able to recreate the
> current ruleset, including sets and elements, by parsing the output of
> nft list table. If we don't include sets, the user will have to seperately
> save and restore them. OTOH if we simply include global and AF-specific
> sets, they will be restored once for each table and this will fail on
> the second table.
>
> Any other ideas?

idea 1:

print global sets when listing every time a table is listed.

To backup the ruleset, the user must iterate over the list of
tables/AF (nft list table $AF $TABLE), so the global sets get printed
several times.

when restoring, try to add the global set to the kernel.
If the set already exist, return EEXIST from the kernel, and let nft
ignore that if operating with 'nft -f'.

idea 2:

add `nft list ruleset', which prints all tables in all AF, including
global sets.
Global sets are only printed in this 'ruleset' view.

Still, the ignore EEXIST errors from the kernel for globals sets.

>
>> Being family-wide is what ipset does, and I'm sure is what most people
>> will expect.
>>
>> On the other hand, I understand this would be a major change at this point.
>
> Should be fine. I'm thinking of keeping the table specific sets and also
> having per AF and AF_UNSPEC sets. For maps we should restrict this feature
> to data maps since verdicts may need a table context.

I agree.

-- 
Arturo Borrero González
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [nf_tables] suggestion: system-wide sets

[Patrick McHardy]

On Thu, Feb 27, 2014 at 03:46:12PM +0100, Arturo Borrero Gonzalez wrote:
> On 27 February 2014 15:02, Patrick McHardy <kaber@trash.net> wrote:
> > On Thu, Feb 27, 2014 at 01:15:36PM +0100, Arturo Borrero Gonzalez wrote:
> >> Hi there!
> >>
> >> I can't remember this subject being discussed in the near past.
> >>
> >> Why not using system-wide sets? Or at least, family-wide sets?
> >>
> >> >From the user point of view, I think is very interesting to define
> >> sets that can be used in rules all across the ruleset, same set in
> >> different tables and families.
> >
> > Yeah, I agree. I think family wide sets and global (AF_UNSPEC) sets should
> > bet quite easy to add. However there's the question how to expose them
> > in the nft list table output. The idea is to be able to recreate the
> > current ruleset, including sets and elements, by parsing the output of
> > nft list table. If we don't include sets, the user will have to seperately
> > save and restore them. OTOH if we simply include global and AF-specific
> > sets, they will be restored once for each table and this will fail on
> > the second table.
> >
> > Any other ideas?
> 
> idea 1:
> 
> print global sets when listing every time a table is listed.
> 
> To backup the ruleset, the user must iterate over the list of
> tables/AF (nft list table $AF $TABLE), so the global sets get printed
> several times.
> 
> when restoring, try to add the global set to the kernel.
> If the set already exist, return EEXIST from the kernel, and let nft
> ignore that if operating with 'nft -f'.

Yeah, that might work. However, how do we know that this it contains the
correct elements?

> idea 2:
> 
> add `nft list ruleset', which prints all tables in all AF, including
> global sets.
> Global sets are only printed in this 'ruleset' view.
> 
> Still, the ignore EEXIST errors from the kernel for globals sets.

Also something worth considering. I'm not sure about this yet, let me
think about this a bit.

> >> Being family-wide is what ipset does, and I'm sure is what most people
> >> will expect.
> >>
> >> On the other hand, I understand this would be a major change at this point.
> >
> > Should be fine. I'm thinking of keeping the table specific sets and also
> > having per AF and AF_UNSPEC sets. For maps we should restrict this feature
> > to data maps since verdicts may need a table context.
> 
> I agree.

Re: [nf_tables] suggestion: system-wide sets

[Patrick Schaaf]

On Thursday 27 February 2014 14:02:30 Patrick McHardy wrote:

> Yeah, I agree. I think family wide sets and global (AF_UNSPEC) sets should
> bet quite easy to add. However there's the question how to expose them
> in the nft list table output. The idea is to be able to recreate the
> current ruleset, including sets and elements, by parsing the output of
> nft list table. If we don't include sets, the user will have to seperately
> save and restore them. OTOH if we simply include global and AF-specific
> sets, they will be restored once for each table and this will fail on
> the second table.
> 
> Any other ideas?

First of all I'd like to note that this needs-separate-saving, is exactly the 
situation we have right now with ipset, so it is not something totally 
unknown. Recreating a table that references nonexisting global sets, would 
fail, just like loading / restoring iptables rules that reference nonexisting 
ipsets, fails right now.

Alternatively - but I'm not sure how good an idea that would be - couldn't 
such nonexisting set references somehow create "forward declarations" to 
permit loading in any order, and threat as-yet-undefined sets as empty?

An additional issue that I imagine we'd have, is set names clashing between 
global and per-table sets. To this end, maybe it would be useful to have a 
syntactic means to differentiate the two cases when referencing sets. Maybe 
append a second '@' to reference global sets?

nfd add rule ip input ip saddr @sharedset@

Rule listing could be a bit flexibly when just a plain set name is given, 
showing per table or global sets as they exist.

Just some thoughts I have on the issue, looking from the outside. Use as you 
see fit :)

best regards
  Patrick

Re: [nf_tables] suggestion: system-wide sets

[Patrick McHardy]

On Mon, Mar 03, 2014 at 11:32:14AM +0100, Patrick Schaaf wrote:
> On Thursday 27 February 2014 14:02:30 Patrick McHardy wrote:
> 
> > Yeah, I agree. I think family wide sets and global (AF_UNSPEC) sets should
> > bet quite easy to add. However there's the question how to expose them
> > in the nft list table output. The idea is to be able to recreate the
> > current ruleset, including sets and elements, by parsing the output of
> > nft list table. If we don't include sets, the user will have to seperately
> > save and restore them. OTOH if we simply include global and AF-specific
> > sets, they will be restored once for each table and this will fail on
> > the second table.
> > 
> > Any other ideas?
> 
> First of all I'd like to note that this needs-separate-saving, is exactly the 
> situation we have right now with ipset, so it is not something totally 
> unknown. Recreating a table that references nonexisting global sets, would 
> fail, just like loading / restoring iptables rules that reference nonexisting 
> ipsets, fails right now.

Right, but it makes atomic replacements of the ruleset either asymetrical
to saving them or simply impossible. Sets are considered a part of the rulset.

> Alternatively - but I'm not sure how good an idea that would be - couldn't 
> such nonexisting set references somehow create "forward declarations" to 
> permit loading in any order, and threat as-yet-undefined sets as empty?

That would mean any typo in set names would create an empty set instead of
an error. I don't think this is a good idea.

> An additional issue that I imagine we'd have, is set names clashing between 
> global and per-table sets. To this end, maybe it would be useful to have a 
> syntactic means to differentiate the two cases when referencing sets. Maybe 
> append a second '@' to reference global sets?
> 
> nfd add rule ip input ip saddr @sharedset@

The alternative would be scoping, we need to look up in multiple scopes
anyways. AF-specific overrides global.

> Rule listing could be a bit flexibly when just a plain set name is given, 
> showing per table or global sets as they exist.

Yeah, that definitely makes sense, we don't want to dump all global sets
if they're not even used within a table. Actually even easier would be
to not show them at all outside of the AF_UNSPEC family and require the
user to restore in the correct order when using global sets. We do support
file inclusions etc. that make this easy to handle. A full dump (all AFs)
would simply begin with AF_UNSPEC and be restorable without any manual
intervention.

> Just some thoughts I have on the issue, looking from the outside. Use as you 
> see fit :)

Thanks.