Analyzing firewall rules programmatically

Linux Netfilter discussions
 help / color / mirror / Atom feed

* Analyzing firewall rules programmatically
@ 2024-01-30 20:13 Timo Lindfors
  2024-02-01 20:34 ` Kerin Millar
  0 siblings, 1 reply; 6+ messages in thread
From: Timo Lindfors @ 2024-01-30 20:13 UTC (permalink / raw)
  To: Netfilter list

Hi,

I spent some time trying to figure out how to programmatically answer 
questions like "Does the ruleset R allow reaching service listening on port P from 
interface I?" or "Does the ruleset R1 allow something that ruleset R2 does 
not allow?".

I tried to find prior work on this area and was able to find "Verified 
iptables Firewall Analysis and Verification" [1] that uses 
the Isabelle proof assistant for iptables rules and "Automated Analysis 
and Debugging of Network Connectivity Policies" [2, 3] that uses Z3 for 
some simple Azure firewall ACL rules.

I could not find anything relevant for netfilter so I began experimenting 
a bit by writing a python function that tries to simulate nftables 
behavior given a ruleset and a packet. I used output of "nft --json list 
ruleset" to and soon noticed that it doesn't quite contain all the 
necessary information for this task.

For example, the default ufw configuration uses iptables to setup

Chain ufw-not-local (1 references)
  pkts bytes target     prot opt in     out     source               destination
   779 46756 RETURN     0    --  *      *       0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type LOCAL

which shows up as

$ sudo nft -a list ruleset | grep "handle 247"
 		fib daddr type local counter packets 772 bytes 46336

in in the "normal" output. However, in the JSON output critical 
information (mainly "LOCAL") is entirely missing:

$ sudo nft -a --json list ruleset | jq . | grep -B5 -A19 '"handle": 247'
     {
       "rule": {
         "family": "ip",
         "table": "filter",
         "chain": "ufw-not-local",
         "handle": 247,
         "expr": [
           {
             "xt": {
               "type": "match",
               "name": "addrtype"
             }
           },
           {
             "counter": {
               "packets": 778,
               "bytes": 46696
             }
           },
           {
             "return": null
           }
         ]
       }
     },

I understand that ufw is using a compatibility interface. Nevertheless, 
I'm in need of a way to analyze firewall configurations of large number of 
real-world systems programmatically. Can you suggest how I should approach 
this? I can think of at least the following options:

1) Fix the JSON output.

2) Ignore the JSON output and try to parse the output of "nft list 
ruleset".

3) Use JSON output of most of the stuff but fill the gaps by also 
parsing "nft list ruleset".

4) Try to parse the raw netlink traffic seen in "nft --debug=netlink list 
ruleset".

4) Create separate tools for parsing iptables and netfilter rules and 
hope that no system mixes these two.

[1] https://link.springer.com/article/10.1007/s10817-017-9445-1
[2] https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/secguru.pdf
[3] https://github.com/Z3Prover/FirewallChecker

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: Analyzing firewall rules programmatically
  2024-01-30 20:13 Analyzing firewall rules programmatically Timo Lindfors
@ 2024-02-01 20:34 ` Kerin Millar
  2024-02-02 19:05   ` Timo Lindfors
  0 siblings, 1 reply; 6+ messages in thread
From: Kerin Millar @ 2024-02-01 20:34 UTC (permalink / raw)
  To: Timo Lindfors; +Cc: Netfilter list

On Tue, 30 Jan 2024 22:13:26 +0200 (EET)
Timo Lindfors <timo.lindfors@iki.fi> wrote:

> Hi,
> 
> I spent some time trying to figure out how to programmatically answer 
> questions like "Does the ruleset R allow reaching service listening on port P from 
> interface I?" or "Does the ruleset R1 allow something that ruleset R2 does 
> not allow?".
> 
> I tried to find prior work on this area and was able to find "Verified 
> iptables Firewall Analysis and Verification" [1] that uses 
> the Isabelle proof assistant for iptables rules and "Automated Analysis 
> and Debugging of Network Connectivity Policies" [2, 3] that uses Z3 for 
> some simple Azure firewall ACL rules.
> 
> I could not find anything relevant for netfilter so I began experimenting 
> a bit by writing a python function that tries to simulate nftables 
> behavior given a ruleset and a packet. I used output of "nft --json list 
> ruleset" to and soon noticed that it doesn't quite contain all the 
> necessary information for this task.
> 
> For example, the default ufw configuration uses iptables to setup
> 
> Chain ufw-not-local (1 references)
>   pkts bytes target     prot opt in     out     source               destination
>    779 46756 RETURN     0    --  *      *       0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type LOCAL
> 
> which shows up as
> 
> $ sudo nft -a list ruleset | grep "handle 247"
>  		fib daddr type local counter packets 772 bytes 46336

Given that the rule is supposed to have been created with iptables-nft, I am surprised that it is presented in that way at all (the line you are showing doesn't match "handle 247" either).

# uname -r; nft -V | head -n1
6.7.2-arch1-2
nftables v1.0.9 (Old Doc Yak #3)
# nft flush ruleset
# iptables-nft -A INPUT -m addrtype --dst-type LOCAL
# nft list ruleset 2>/dev/null | sed -n 4p
		xt match "addrtype" counter packets 8 bytes 778

> 
> in in the "normal" output. However, in the JSON output critical 
> information (mainly "LOCAL") is entirely missing:
> 
> $ sudo nft -a --json list ruleset | jq . | grep -B5 -A19 '"handle": 247'
>      {
>        "rule": {
>          "family": "ip",
>          "table": "filter",
>          "chain": "ufw-not-local",
>          "handle": 247,
>          "expr": [
>            {
>              "xt": {
>                "type": "match",
>                "name": "addrtype"
>              }
>            },
>            {
>              "counter": {
>                "packets": 778,
>                "bytes": 46696
>              }
>            },
>            {
>              "return": null
>            }
>          ]
>        }
>      },
> 
> 
> I understand that ufw is using a compatibility interface. Nevertheless, 

Indeed, the output you are getting is symptomatic of the presence of xtables glue. The output of nft list ruleset won't be round-trip safe.

> I'm in need of a way to analyze firewall configurations of large number of 
> real-world systems programmatically. Can you suggest how I should approach 
> this? I can think of at least the following options:
> 
> 1) Fix the JSON output.

The xtables glue seems so disjoint that I wonder whether it would ever be fixed.

> 
> 2) Ignore the JSON output and try to parse the output of "nft list 
> ruleset".

This would not be reliable either.

> 
> 3) Use JSON output of most of the stuff but fill the gaps by also 
> parsing "nft list ruleset".

Nor this.

> 
> 4) Try to parse the raw netlink traffic seen in "nft --debug=netlink list 
> ruleset".

This doesn't seem very useful either.

# nft --debug=netlink list ruleset 2>/dev/null | head -n3
ip filter INPUT 2
  [ match name addrtype rev 1 ]
  [ counter pkts 287 bytes 59823 ]

> 
> 4) Create separate tools for parsing iptables and netfilter rules and 
> hope that no system mixes these two.

As things stand, this seems more realistic to me. Perhaps one of the developers would have a better idea, though.

-- 
Kerin Millar

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: Analyzing firewall rules programmatically
  2024-02-01 20:34 ` Kerin Millar
@ 2024-02-02 19:05   ` Timo Lindfors
  2024-02-02 20:55     ` Kerin Millar
  2024-02-03 11:45     ` Florian Westphal
  0 siblings, 2 replies; 6+ messages in thread
From: Timo Lindfors @ 2024-02-02 19:05 UTC (permalink / raw)
  To: Netfilter list



On Thu, 1 Feb 2024, Kerin Millar wrote:
> Given that the rule is supposed to have been created with iptables-nft, I am surprised that it is presented in that way at all (the line you are showing doesn't match "handle 247" either).

ah, I might have made an error while trying to format the lines for 80 
characters. Here's a fresh new situation:

$ sudo nft -a list ruleset | grep "handle 125"
# Warning: table ip filter is managed by iptables-nft, do not touch!
 		fib daddr type broadcast counter packets 33859 bytes 6322835 jump ufw-skip-to-policy-input # handle 125
# Warning: table ip6 filter is managed by iptables-nft, do not touch!
 	chain ufw6-user-logging-output { # handle 125
# Warning: table ip nat is managed by iptables-nft, do not touch!
# Warning: table ip mangle is managed by iptables-nft, do not touch!

$ sudo nft -a --json list ruleset | jq . | grep -B5 -A19 '"handle": 125'
     {
       "rule": {
         "family": "ip",
         "table": "filter",
         "chain": "ufw-after-input",
         "handle": 125,
         "expr": [
           {
             "xt": {
               "type": "match",
               "name": "addrtype"
             }
           },
           {
             "counter": {
               "packets": 33859,
               "bytes": 6322835
             }
           },
           {
             "jump": {
               "target": "ufw-skip-to-policy-input"
             }
           }
         ]



$ sudo iptables -nvL | grep 33859
33859 6323K ufw-skip-to-policy-input  0    --  *      *       0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type BROADCAST



> # uname -r; nft -V | head -n1
> 6.7.2-arch1-2
> nftables v1.0.9 (Old Doc Yak #3)
> # nft flush ruleset
> # iptables-nft -A INPUT -m addrtype --dst-type LOCAL
> # nft list ruleset 2>/dev/null | sed -n 4p
> 		xt match "addrtype" counter packets 8 bytes 778

I get different output on Debian 12:

# uname -r; nft -V | head -n1
6.1.0-17-amd64
nftables v1.0.6 (Lester Gooch #5)
# nft flush ruleset
# iptables-nft -A INPUT -m addrtype --dst-type LOCAL
# nft list ruleset 2>/dev/null | sed -n 4p
 		fib daddr type local counter packets 24 bytes 16144

Is this perhaps a regression?

>> 4) Create separate tools for parsing iptables and netfilter rules and
>> hope that no system mixes these two.
>
> As things stand, this seems more realistic to me. Perhaps one of the developers would have a better idea, though.

Ok, thanks for the insights. If I built the tool for netfilter first, 
which of the three formats should I try to analyze? netlink, text or json?

-Timo

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: Analyzing firewall rules programmatically
  2024-02-02 19:05   ` Timo Lindfors
@ 2024-02-02 20:55     ` Kerin Millar
  2024-02-03 11:45     ` Florian Westphal
  1 sibling, 0 replies; 6+ messages in thread
From: Kerin Millar @ 2024-02-02 20:55 UTC (permalink / raw)
  To: Timo Lindfors, Netfilter list

On Fri, 2 Feb 2024, at 7:05 PM, Timo Lindfors wrote:
> On Thu, 1 Feb 2024, Kerin Millar wrote:
>> Given that the rule is supposed to have been created with iptables-nft, I am surprised that it is presented in that way at all (the line you are showing doesn't match "handle 247" either).
>
> ah, I might have made an error while trying to format the lines for 80 
> characters. Here's a fresh new situation:
>
> $ sudo nft -a list ruleset | grep "handle 125"
> # Warning: table ip filter is managed by iptables-nft, do not touch!
>  		fib daddr type broadcast counter packets 33859 bytes 6322835 jump 

This gives the initial impression that iptables-nft was enlightened enough to generate a native nft rule for its input. It can do that in some cases and not others.

> ufw-skip-to-policy-input # handle 125
> # Warning: table ip6 filter is managed by iptables-nft, do not touch!
>  	chain ufw6-user-logging-output { # handle 125
> # Warning: table ip nat is managed by iptables-nft, do not touch!
> # Warning: table ip mangle is managed by iptables-nft, do not touch!
>
> $ sudo nft -a --json list ruleset | jq . | grep -B5 -A19 '"handle": 125'
>      {
>        "rule": {
>          "family": "ip",
>          "table": "filter",
>          "chain": "ufw-after-input",
>          "handle": 125,
>          "expr": [
>            {
>              "xt": {
>                "type": "match",
>                "name": "addrtype"
>              }
>            },
>            {
>              "counter": {
>                "packets": 33859,
>                "bytes": 6322835
>              }
>            },
>            {
>              "jump": {
>                "target": "ufw-skip-to-policy-input"
>              }
>            }
>          ]

Yet, the above output indicates the use of an xtables match. Here's how it should look for a native rule.

# nft 'table t { chain c { fib daddr type broadcast counter; }; }'
# nft -j list ruleset
{
  "nftables": [
    {
      "metainfo": {
        "version": "1.0.9",
        "release_name": "Old Doc Yak #3",
        "json_schema_version": 1
      }
    },
    {
      "table": {
        "family": "ip",
        "name": "t",
        "handle": 2
      }
    },
    {
      "chain": {
        "family": "ip",
        "table": "t",
        "name": "c",
        "handle": 1
      }
    },
    {
      "rule": {
        "family": "ip",
        "table": "t",
        "chain": "c",
        "handle": 2,
        "expr": [
          {
            "match": {
              "op": "==",
              "left": {
                "fib": {
                  "result": "type",
                  "flags": [
                    "daddr"
                  ]
                }
              },
              "right": "broadcast"
            }
          },
          {
            "counter": {
              "packets": 0,
              "bytes": 0
            }
          }
        ]
      }
    }
  ]
}

>
>
>
> $ sudo iptables -nvL | grep 33859
> 33859 6323K ufw-skip-to-policy-input  0    --  *      *       0.0.0.0/0 
>            0.0.0.0/0            ADDRTYPE match dst-type BROADCAST
>
>
>
>> # uname -r; nft -V | head -n1
>> 6.7.2-arch1-2
>> nftables v1.0.9 (Old Doc Yak #3)
>> # nft flush ruleset
>> # iptables-nft -A INPUT -m addrtype --dst-type LOCAL
>> # nft list ruleset 2>/dev/null | sed -n 4p
>> 		xt match "addrtype" counter packets 8 bytes 778
>
> I get different output on Debian 12:
>
> # uname -r; nft -V | head -n1
> 6.1.0-17-amd64
> nftables v1.0.6 (Lester Gooch #5)
> # nft flush ruleset
> # iptables-nft -A INPUT -m addrtype --dst-type LOCAL
> # nft list ruleset 2>/dev/null | sed -n 4p
>  		fib daddr type local counter packets 24 bytes 16144
>
> Is this perhaps a regression?

It might be that iptables-nft is supposed to generate a native rule and that we are both experiencing a regression in that respect.

On the other hand, it might be the opposite. That is, even though iptables-nft could, in principle, generate a native rule for "-m addrtype --dst-type BROADCAST", it might be that it has never been able to and that the output of "nft list ruleset" is simply misleading in your case.

Presently, it is unclear to me.

>
>>> 4) Create separate tools for parsing iptables and netfilter rules and
>>> hope that no system mixes these two.
>>
>> As things stand, this seems more realistic to me. Perhaps one of the developers would have a better idea, though.
>
> Ok, thanks for the insights. If I built the tool for netfilter first, 
> which of the three formats should I try to analyze? netlink, text or json?

It's difficult for me to make a recommendation because I only fully understand the output of iptables-save, nft list ruleset, nft -j list ruleset etc. However, I would say that structured data is helpful, meaning that I would prefer "nft -j list ruleset" over "nft list ruleset".

-- 
Kerin Millar

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: Analyzing firewall rules programmatically
  2024-02-02 19:05   ` Timo Lindfors
  2024-02-02 20:55     ` Kerin Millar
@ 2024-02-03 11:45     ` Florian Westphal
  2024-02-05 12:35       ` Timo Lindfors
  1 sibling, 1 reply; 6+ messages in thread
From: Florian Westphal @ 2024-02-03 11:45 UTC (permalink / raw)
  To: Timo Lindfors; +Cc: Netfilter list

Timo Lindfors <timo.lindfors@iki.fi> wrote:
> > # uname -r; nft -V | head -n1
> > 6.7.2-arch1-2
> > nftables v1.0.9 (Old Doc Yak #3)
> > # nft flush ruleset
> > # iptables-nft -A INPUT -m addrtype --dst-type LOCAL
> > # nft list ruleset 2>/dev/null | sed -n 4p
> > 		xt match "addrtype" counter packets 8 bytes 778
> 
> I get different output on Debian 12:
> 
> # uname -r; nft -V | head -n1
> 6.1.0-17-amd64
> nftables v1.0.6 (Lester Gooch #5)
> # nft flush ruleset
> # iptables-nft -A INPUT -m addrtype --dst-type LOCAL
> # nft list ruleset 2>/dev/null | sed -n 4p
> 		fib daddr type local counter packets 24 bytes 16144
> 
> Is this perhaps a regression?

More likely that the former nft is compiled without xtables support,
the latter nft binary asks iptables-translate for a textual
nft-equivalent repesentation of the addrtype match.

> Ok, thanks for the insights. If I built the tool for netfilter first, which
> of the three formats should I try to analyze? netlink, text or json?

Text is rather unstable, I would not rely on it.

json ought to be stable, netlink is stable (its the api after all).

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: Analyzing firewall rules programmatically
  2024-02-03 11:45     ` Florian Westphal
@ 2024-02-05 12:35       ` Timo Lindfors
  0 siblings, 0 replies; 6+ messages in thread
From: Timo Lindfors @ 2024-02-05 12:35 UTC (permalink / raw)
  To: Netfilter list



On Sat, 3 Feb 2024, Florian Westphal wrote:
> Text is rather unstable, I would not rely on it.
>
> json ought to be stable, netlink is stable (its the api after all).

Thanks. I implemented my first prototype on the JSON output, and only for 
netfilter for now. You can give it a ruleset, desired verdict and extra
constraints and it will find a packet that achieves the desired verdict if 
it exists.

Here are some examples:

nftables_analysis$ for i in test_data/*.nft; do echo "#### $i"; cat $i; ./find_satisfying_packet.py --json-ruleset $i.json -c tcp.dport==1234; done

#### test_data/complex1.nft
#!/usr/sbin/nft -f

table inet my_filter {
     chain input {
         type filter hook input priority 0; policy accept;
 	meta nfproto ipv6 tcp sport 53 meta length == 200 iif > 0 ip6 saddr fe80::fca5:f7ff:febc:97a4 tcp dport 1234 accept
 	tcp dport 1234 drop
     }
}
Satified
ip.daddr=-1
ip.saddr=-1
ip6.daddr=0
ip6.saddr=338288524927261089672224126588925286308
meta.iif=1
meta.l4proto=6
meta.length=200
meta.nfproto=10
tcp.dport=1234
tcp.sport=53
udp.dport=-1
udp.sport=-1
#### test_data/complex2.nft
#!/usr/sbin/nft -f

table inet my_filter {
     chain input {
         type filter hook input priority 0; policy accept;
 	meta nfproto ipv6 tcp dport 1234 accept
 	tcp dport 1234 drop
     }
}
Satified
ip.daddr=-1
ip.saddr=-1
ip6.daddr=0
ip6.saddr=0
meta.iif=0
meta.l4proto=6
meta.length=0
meta.nfproto=10
tcp.dport=1234
tcp.sport=0
udp.dport=-1
udp.sport=-1
#### test_data/source_iface.nft
#!/usr/sbin/nft -f

table inet my_filter {
     chain input {
         type filter hook input priority 0; policy accept;
         iif enp0s31f6 tcp dport 1234 accept
 	tcp dport 1234 drop
     }
}
Satified
ip.daddr=0
ip.saddr=0
ip6.daddr=-1
ip6.saddr=-1
meta.iif=0
meta.l4proto=6
meta.length=0
meta.nfproto=2
tcp.dport=1234
tcp.sport=0
udp.dport=-1
udp.sport=-1
#### test_data/source_ip2.nft
#!/usr/sbin/nft -f

table inet my_filter {
     chain input {
         type filter hook input priority 0; policy accept;
 	tcp dport 1234 jump helper
 	tcp dport 1234 accept
     }
     chain helper {
         ip saddr 10.61.7.4 accept
 	tcp dport 1234 drop
     }
}
Satified
ip.daddr=0
ip.saddr=171771652
ip6.daddr=-1
ip6.saddr=-1
meta.iif=0
meta.l4proto=6
meta.length=0
meta.nfproto=2
tcp.dport=1234
tcp.sport=0
udp.dport=-1
udp.sport=-1
#### test_data/source_ip.nft
#!/usr/sbin/nft -f

table inet my_filter {
     chain input {
         type filter hook input priority 0; policy accept;
         tcp dport 1234 ip saddr 10.61.7.4 accept
 	tcp dport 1234 drop
     }
}
Satified
ip.daddr=0
ip.saddr=171771652
ip6.daddr=-1
ip6.saddr=-1
meta.iif=0
meta.l4proto=6
meta.length=0
meta.nfproto=2
tcp.dport=1234
tcp.sport=0
udp.dport=-1
udp.sport=-1
#### test_data/source_port_ipv6.nft
#!/usr/sbin/nft -f

table ip my_filter {
     chain input {
         type filter hook input priority 0; policy accept;
 	tcp dport 1234 drop
     }
}

table ip6 my_filter {
     chain input {
         type filter hook input priority 0; policy accept;
         tcp dport 1234 tcp sport 53 accept
 	tcp dport 1234 drop
     }
}
Satified
ip.daddr=-1
ip.saddr=-1
ip6.daddr=0
ip6.saddr=0
meta.iif=0
meta.l4proto=6
meta.length=0
meta.nfproto=10
tcp.dport=1234
tcp.sport=53
udp.dport=-1
udp.sport=-1
#### test_data/source_port.nft
#!/usr/sbin/nft -f

table inet my_filter {
     chain input {
         type filter hook input priority 0; policy accept;
         tcp dport 1234 tcp sport 53 accept
 	tcp dport 1234 drop
     }
}
Satified
ip.daddr=0
ip.saddr=0
ip6.daddr=-1
ip6.saddr=-1
meta.iif=0
meta.l4proto=6
meta.length=0
meta.nfproto=2
tcp.dport=1234
tcp.sport=53
udp.dport=-1
udp.sport=-1
#### test_data/source_port_range.nft
#!/usr/sbin/nft -f

table inet my_filter {
     chain input {
         type filter hook input priority 0; policy accept;
 	tcp sport {50-55} tcp dport 1234 accept
 	tcp dport 1234 drop
     }
}
Satified
ip.daddr=0
ip.saddr=0
ip6.daddr=-1
ip6.saddr=-1
meta.iif=0
meta.l4proto=6
meta.length=0
meta.nfproto=2
tcp.dport=1234
tcp.sport=50
udp.dport=-1
udp.sport=-1


I put the code to https://github.com/lindi2/nftables_analysis

(MIT license, do whatever you want but I would appreciate bug fixes so 
that I can make it more reliable and maybe implement iptables support 
later as well)

-Timo


^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2024-02-05 12:35 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-01-30 20:13 Analyzing firewall rules programmatically Timo Lindfors
2024-02-01 20:34 ` Kerin Millar
2024-02-02 19:05   ` Timo Lindfors
2024-02-02 20:55     ` Kerin Millar
2024-02-03 11:45     ` Florian Westphal
2024-02-05 12:35       ` Timo Lindfors

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox