Re: nftables rules not matching after upgrading from 0.7 to 0.8

["Anders K. Pedersen | Cohaesio" <akp@cohaesio.com>]

On ons, 2017-10-25 at 20:20 +0200, Anders K. Pedersen | Cohaesio wrote:
> On ons, 2017-10-25 at 19:57 +0200, Florian Westphal wrote:
> > Anders K. Pedersen | Cohaesio <akp@cohaesio.com> wrote:
> > > After upgrading to nftables 0.8 (from 0.7) on one of my systems,
> > > I've
> > > experiences several cases where rules that used to work fine with
> > > 0.7
> > > sometimes doesn't match anymore with 0.8 (it's not consistent -
> > > sometimes the rules do match with 0.8).
> > > 
> > > The rule chains end with a log statement before rejecting or
> > > dropping
> > > the packets, and I can see in the log that everything is as
> > > expected
> > > and the rules should match. After downgrading to nftables 0.7
> > > everything works again.
> > 
> > Are those errors restricted to a particular table family, chain or
> > protocol?
> 
> So far I've only registered it for IPv4 input filtering for TCP, but
> that's also most of the traffic on this system, so I'm not sure that
> it's limited to that.
> 
> As mentioned, it's not consistent. A rule that has worked fine could
> suddenly stop working without any rule set changes for days. Some
> times
> it has helped to just reload the exact same rule set. Other times
> changing
> 
> tcp dport { imap2, imaps } flow table imap \
>                 { ip saddr & 255.255.255.240 \
>                   timeout 5m limit rate 10/minute } \
>         counter accept
> 
> to
> 
> tcp dport { imap2, imaps } flow table imap \
>         counter accept
> 
> or
> 
> tcp dport { domain, http, https, 8080, 8443, 9091 } \
>         meta iif eth1 accept
> 
> to
> 
> tcp dport { domain, http, https, 8080, 8443, 9091 } \
>         meta iif eth1 counter accept
> 
> has resolved it, but it feels like it wasn't really due to the
> changes,
> but more random luck.
> 
> > At least for ip family all packets will have pkt->tprot_set true,
> > so I don't see how meta would break there.
> > 
> > >  ip filter input 
> > > -  [ payload load 1b @ network header + 9 => reg 1 ]
> > > +  [ meta load l4proto => reg 1 ]
> > > 
> > > and
> > > 
> > >  ip6 filter input 
> > > -  [ payload load 1b @ network header + 6 => reg 1 ]
> > > +  [ meta load l4proto => reg 1 ]
> > > 
> > > for many of the rules. I believe this is intentional and correct
> > > by
> > > itself, so I guess the problem is in the kernel (the system is
> > > currently running 4.13.8), where "meta l4proto" sometimes isn't
> > > what it
> > > should be.
> > 
> > Weird.  We changed the implicit l4 dependencies to meta l4proto
> > because
> > in ipv6 case this will skip extension headers.
> > 
> > IIRC ipv4 was only changed to keep it more simple.
> > (also nft_set_pktinfo_ipv4 sets
> >          pkt->tprot_set = true;
> >          pkt->tprot = iph->protocol;
> > 
> > so I fail to see why it should not be identical in all cases).
> 
> Any testing, I can do?

One more thing, I just noticed is that if I try to use nftables 0.7 to
dump the rule set that was loaded with 0.7, I get the following error:

# nft list ruleset
nft: payload.c:550: payload_expr_expand: Assertion `desc->base == expr->payload.base' failed.
Aborted

If I use 0.8 to dump the rule set that was loaded with 0.7, I get the
correct rule set except for a difference with regards to sets and maps
that use interfaces like:

--- nft-0.7-0.8 rule set loaded with 0.7 and dumped with 0.8
+++ nft-0.8     same rule set loaded and dumped with 0.8
@@ -9,12 +9,12 @@
        chain prerouting {
                type filter hook prerouting priority -100; policy accept;
                iif "lo" accept
-               ct mark set iif map { 33554432 : 0x00000001, 67108864 : 0x00000002 }
+               ct mark set iif map { "eth0" : 0x00000001, "eth2" : 0x00000002 }
                iif "eth1" jump prerouting_internal
-               iif { 33554432, 67108864 } ip saddr { 0.0.0.0/8, 10.0.0.0/8, 127.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 224.0.0.0-255.255.255.255 } counter packets 6 bytes 705 drop
+               iif { "eth0", "eth2" } ip saddr { 0.0.0.0/8, 10.0.0.0/8, 127.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 224.0.0.0-255.255.255.255 } counter packets 0 bytes 0 drop

--
Regards,
Anders