Re: [PATCH nft 1/2] evaluate: Perform set evaluation on implicitly declared (anonymous) sets

[Pablo Neira Ayuso <pablo@netfilter.org>]

On Tue, May 26, 2020 at 08:01:21PM +0200, Stefano Brivio wrote:
> On Tue, 26 May 2020 19:34:19 +0200
> Pablo Neira Ayuso <pablo@netfilter.org> wrote:
> 
> > On Tue, May 26, 2020 at 07:17:25PM +0200, Stefano Brivio wrote:
> > > On Tue, 26 May 2020 18:54:16 +0200
> > > Pablo Neira Ayuso <pablo@netfilter.org> wrote:
> > >   
> > > > On Sun, May 24, 2020 at 03:00:26PM +0200, Stefano Brivio wrote:  
> > > > > If a set is implicitly declared, set_evaluate() is not called as a
> > > > > result of cmd_evaluate_add(), because we're adding in fact something
> > > > > else (e.g. a rule). Expression-wise, evaluation still happens as the
> > > > > implicit set expression is eventually found in the tree and handled
> > > > > by expr_evaluate_set(), but context-wise evaluation (set_evaluate())
> > > > > is skipped, and this might be relevant instead.
> > > > > 
> > > > > This is visible in the reported case of an anonymous set including
> > > > > concatenated ranges:
> > > > > 
> > > > >   # nft add rule t c ip saddr . tcp dport { 192.0.2.1 . 20-30 } accept
> > > > >   BUG: invalid range expression type concat
> > > > >   nft: expression.c:1160: range_expr_value_low: Assertion `0' failed.
> > > > >   Aborted
> > > > > 
> > > > > because we reach do_add_set() without properly evaluated flags and
> > > > > set description, and eventually end up in expr_to_intervals(), which
> > > > > can't handle that expression.
> > > > > 
> > > > > Explicitly call set_evaluate() as we add anonymous sets into the
> > > > > context, and instruct the same function to skip expression-wise set
> > > > > evaluation if the set is anonymous, as that happens later anyway as
> > > > > part of the general tree evaluation.
> > > > > 
> > > > > Reported-by: Pablo Neira Ayuso <pablo@netfilter.org>
> > > > > Reported-by: Phil Sutter <phil@nwl.cc>
> > > > > Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
> > > > > ---
> > > > >  src/evaluate.c | 5 ++++-
> > > > >  1 file changed, 4 insertions(+), 1 deletion(-)
> > > > > 
> > > > > diff --git a/src/evaluate.c b/src/evaluate.c
> > > > > index 506f2c6a257e..ee019bc98480 100644
> > > > > --- a/src/evaluate.c
> > > > > +++ b/src/evaluate.c
> > > > > @@ -76,6 +76,7 @@ static void key_fix_dtype_byteorder(struct expr *key)
> > > > >  	datatype_set(key, set_datatype_alloc(dtype, key->byteorder));
> > > > >  }
> > > > >  
> > > > > +static int set_evaluate(struct eval_ctx *ctx, struct set *set);
> > > > >  static struct expr *implicit_set_declaration(struct eval_ctx *ctx,
> > > > >  					     const char *name,
> > > > >  					     struct expr *key,
> > > > > @@ -107,6 +108,8 @@ static struct expr *implicit_set_declaration(struct eval_ctx *ctx,
> > > > >  		list_add_tail(&cmd->list, &ctx->cmd->list);
> > > > >  	}
> > > > >  
> > > > > +	set_evaluate(ctx, set);    
> > > > 
> > > > Hm, set_evaluate() populates the cache with the anonymous set in this
> > > > case, see set_lookup() + sed_add_hash().  
> > > 
> > > While checking what parts of set_evaluate() we should skip for anonymous
> > > sets, I thought it made sense to keep that, simply because I didn't see
> > > any value in making that a special case. Is the __set* stuff polluting?  
> > 
> > Yes, it's just adding a __set%d to the cache.
> > 
> > > Any other bad consequence I missed? Or you would skip that just because
> > > it's useless?  
> > 
> > I did not find any command that triggers any problem right now. I just
> > think we should not add an anonymous set to the cache.
> 
> Okay, I see.
> 
> > BTW, are not set->desc.field_len and set->key->field_len duplicated
> > fields? Same thing with field_count.
> 
> Yes, they are, but:
> 
> > Probably it should be possible to simplify this by using
> > set->key->field* instead? So set_evaluate() is not required to
> > transfer the fields.
> 
> ...even if we use those, we still need to call expr_evaluate_concat()
> (with the same logic as implemented by set_evaluate()) to fill the
> set->key fields in.
> 
> Conceptually, I think that set_evaluate() should apply just in the same
> way no matter how sets are created, minus expression evaluation and
> caching. Taking selecting bits out looks a bit fragile/inconsistent to
> me. Maybe I'm biased by the fact it was relatively complicated for me
> to narrow down this particular issue.

OK, let's just not add this anonymous set to the cache.

There is also a test line in tests/py that needs to be turned on that
Phil mentioned (no need for new tests/shell file).

There is a memleak kicking in a few tests after this patch. Turn on
ASAN and run tests/shell/ to catch it.

So re-spin and send v2, thanks.