nft list sets changed behavior

nft list sets changed behavior

[nft.ogxzcrqhuhgchbvxcs4j7wws]

After updating to Debian 12 my tools relying on 'nft -j list sets' fail. 
It now does not include the elements in those lists like it did on 11.

Is this hidden behind an 'anti-terse' flag now?

Web searches, the changelog on the site, and manpages didn't reveal 
anything.

Re: nft list sets changed behavior

[Kerin Millar]

On Sun, 18 Jun 2023 02:42:46 +0200
nft.ogxzcrqhuhgchbvxcs4j7wws@qmail.sunbirdgrove.com wrote:

> After updating to Debian 12 my tools relying on 'nft -j list sets' fail. 
> It now does not include the elements in those lists like it did on 11.
> 
> Is this hidden behind an 'anti-terse' flag now?
> 
> Web searches, the changelog on the site, and manpages didn't reveal 
> anything.

Running `nft list sets` doesn't list elements either. Unfortunately, whatever the intended behaviour may be, it is not documented at all. The nft(8) page states that the "list" verb should "display the elements in the specified set", which can only describe the "list set [family] table set" syntax, rather than the "list sets [family]" syntax.

I agree with the implication that to have "list sets" include the elements, unless -t is given, would be more consistent and logical. I would suggest filing a bug for further discussion.

-- 
Kerin Millar

Re: nft list sets changed behavior

[nft.ogxzcrqhuhgchbvxcs4j7wws]

Thanks for your reply. On Debian 11 it did list the elements for at 
least a year, that's why I'm surprised about this change. 0.9.8 vs. 1.0.6.

I'll look into filing a bug.

On 6/18/23 13:22, Kerin Millar wrote:
> On Sun, 18 Jun 2023 02:42:46 +0200
> nft.ogxzcrqhuhgchbvxcs4j7wws@qmail.sunbirdgrove.com wrote:
> 
>> After updating to Debian 12 my tools relying on 'nft -j list sets' fail.
>> It now does not include the elements in those lists like it did on 11.
>>
>> Is this hidden behind an 'anti-terse' flag now?
>>
>> Web searches, the changelog on the site, and manpages didn't reveal
>> anything.
> 
> Running `nft list sets` doesn't list elements either. Unfortunately, whatever the intended behaviour may be, it is not documented at all. The nft(8) page states that the "list" verb should "display the elements in the specified set", which can only describe the "list set [family] table set" syntax, rather than the "list sets [family]" syntax.
> 
> I agree with the implication that to have "list sets" include the elements, unless -t is given, would be more consistent and logical. I would suggest filing a bug for further discussion.
> 

Re: nft list sets changed behavior

[Florian Westphal]

moving to nf-devel

nft.ogxzcrqhuhgchbvxcs4j7wws@qmail.sunbirdgrove.com <nft.ogxzcrqhuhgchbvxcs4j7wws@qmail.sunbirdgrove.com> wrote:
> Thanks for your reply. On Debian 11 it did list the elements for at least a
> year, that's why I'm surprised about this change. 0.9.8 vs. 1.0.6.
> 
> I'll look into filing a bug.

No need, consider the bug filed.

Pablo, we have a behaviour change in
"nft -j list sets".

1.0.0:
nft -j list sets : lists sets with elements.
nft list sets : no elements.

1.0.1+:
nft -j list sets : no elements.
nft list sets : no elements.

So 1.0.1+ it at least consistent, no set elements
are shown.

But it breaks at least one user setup:
> > > After updating to Debian 12 my tools relying on 'nft -j list sets' fail.
> > > It now does not include the elements in those lists like it did on 11.

I see three possible solutions:
1 - accept the breakage.
2 - repair the inconsistency so we get 1.0.0 and
    earlier behaviour back.
3 - make "list sets" *always* include set elements,
    unless --terse was given.

Thoughts? I'd go with 3, I dislike the
different behaviour that 2) implies and we already
have --terse, we just need to make use of it here.

I'd even favour 1 over 2.

This change came with
commit a1a6b0a5c3c4b4b305fa34a77932ee1c6452d1c8
cache: finer grain cache population for list commands

so it would be easy to resolve, e.g.:

diff --git a/src/cache.c b/src/cache.c
--- a/src/cache.c
+++ b/src/cache.c
@@ -235,6 +235,8 @@ static unsigned int evaluate_cache_list(struct nft_ctx *nft, struct cmd *cmd,
        case CMD_OBJ_SETS:
        case CMD_OBJ_MAPS:
                flags |= NFT_CACHE_TABLE | NFT_CACHE_SET;
+               if (!nft_output_terse(&nft->output))
+                       flags |= NFT_CACHE_SETELEM;
                break;
        case CMD_OBJ_FLOWTABLE:
                if (filter &&

Re: nft list sets changed behavior

[nft.ogxzcrqhuhgchbvxcs4j7wws]

 > No need, consider the bug filed.

Thank you!

 > Thoughts? I'd go with 3

If you care for input on -devel, I'd also go with 3, that's intuitive 
like the rest. (nft -t list set inet filter foo, nft -t list ruleset)

I already switched to looping over it, but when you need all data 
anyway, it's a few less calls.

Re: nft list sets changed behavior

[Pablo Neira Ayuso]

On Sun, Jun 18, 2023 at 03:35:09PM +0200, Florian Westphal wrote:
> moving to nf-devel
> 
> nft.ogxzcrqhuhgchbvxcs4j7wws@qmail.sunbirdgrove.com <nft.ogxzcrqhuhgchbvxcs4j7wws@qmail.sunbirdgrove.com> wrote:
[...]
> > > > After updating to Debian 12 my tools relying on 'nft -j list sets' fail.
> > > > It now does not include the elements in those lists like it did on 11.
> 
> I see three possible solutions:
> 1 - accept the breakage.
> 2 - repair the inconsistency so we get 1.0.0 and
>     earlier behaviour back.
> 3 - make "list sets" *always* include set elements,
>     unless --terse was given.
> 
> Thoughts? I'd go with 3, I dislike the
> different behaviour that 2) implies and we already
> have --terse, we just need to make use of it here.

I'd go with 3 too, so --terse is honored.

Re: nft list sets changed behavior

[Kerin Millar]

On Sun, 18 Jun 2023 17:21:38 +0200
Pablo Neira Ayuso <pablo@netfilter.org> wrote:

> On Sun, Jun 18, 2023 at 03:35:09PM +0200, Florian Westphal wrote:
> > moving to nf-devel
> > 
> > nft.ogxzcrqhuhgchbvxcs4j7wws@qmail.sunbirdgrove.com <nft.ogxzcrqhuhgchbvxcs4j7wws@qmail.sunbirdgrove.com> wrote:
> [...]
> > > > > After updating to Debian 12 my tools relying on 'nft -j list sets' fail.
> > > > > It now does not include the elements in those lists like it did on 11.
> > 
> > I see three possible solutions:
> > 1 - accept the breakage.
> > 2 - repair the inconsistency so we get 1.0.0 and
> >     earlier behaviour back.
> > 3 - make "list sets" *always* include set elements,
> >     unless --terse was given.
> > 
> > Thoughts? I'd go with 3, I dislike the
> > different behaviour that 2) implies and we already
> > have --terse, we just need to make use of it here.
> 
> I'd go with 3 too, so --terse is honored.

I think so too. While there is a theoretical risk of breaking someone's script in the case that they were relying upon the 'new' behaviour, the present behaviour makes very little sense.

-- 
Kerin Millar