Nftables + ALG + Linux 6.1.0-10-amd64 …?... is it a kown Problem?

Nftables + ALG + Linux 6.1.0-10-amd64 …?... is it a kown Problem?

[toml]

Good morning

After updating my server from Bullseye to Bookworm I noticed that my
NFT rules seem to be causing problems lately. Upon closer inspection,
the running process came to a virtual halt with a kernel-panic-message
in the journal. I was able to somehow narrow down the cause of the
error to the application layer gateway - because it was running without
it. Interestingly, I was able to open other consoles during the crash
with ctrl-alt-f2/5, with (except for the network) apparently full
functionality, to restart the machine. However, that Shutdown didn't
work in the normal way either, but only with clear coercion: 'systemctl
poweroff -f -f'.  Finally I reverted back to Bullseye, because
stability was more important to me.

I then looked at the problem further in a Bookworm-VM, same problem,
same cause, but without this dramatic effect. It seems, my problem is
the ALG-FTP and the passive mode during file transfer... this obviously
doesn't work anymore with Linux 6.1.0-10-amd64 at the moment. On my VM
it looks like this:

# cat /etc/modules-load.d/modules.conf
    # /etc/modules: kernel modules to load at boot time.
    #
    # This file contains the names of kernel modules 
    # that should be loaded at boot time, one per line. 
    # Lines beginning with "#" are ignored.

    nf_conntrack
    nf_conntrack_ftp
    nf_conntrack_tftp

# journalctl -b | grep -i conntrack
    Jul 30 20:33:31 ftps systemd-modules-load[238]: 
    Inserted module 'nf_conntrack'
    Jul 30 20:33:31 ftps systemd-modules-load[238]: 
    Inserted module 'nf_conntrack_ftp'
    Jul 30 20:33:31 ftps systemd-modules-load[238]: 
    Inserted module 'nf_conntrack_tftp'

# lsmod | grep nf_
    nf_reject_ipv4         16384  1 nft_reject_ipv4
    nf_tables             290816  119 nft_reject_ipv4,nft_ct,nft_reject
    nfnetlink              20480  1 nf_tables
    nf_conntrack_tftp      20480  0
    nf_conntrack_ftp       24576  0
    nf_conntrack          188416  3 
nf_conntrack_tftp,nft_ct,nf_conntrack_ftp
    nf_defrag_ipv6         24576  1 nf_conntrack
    nf_defrag_ipv4         16384  1 nf_conntrack
    libcrc32c              16384  2 nf_conntrack,nf_tables

# ls /proc/sys/net/netfilter/net.netfilter.nf_conntrack_helper
    ls: File not found

I cannot enable the conntrack-helper with '1'.

Does anyone know if this is a known problem? What irritates me about
the whole thing is the fact, that connection/file transfer with 'active
mode' still seems to work, despite missing (!) open ports 1024-65535. 
That is, I am currently at a loss.

Best regards
Thomas

(Translated with deepl)

Re: Nftables + ALG + Linux 6.1.0-10-amd64 …?... is it a kown Problem?

[Florian Westphal]

toml <toml@thlu.de> wrote:
> After updating my server from Bullseye to Bookworm I noticed that my
> NFT rules seem to be causing problems lately. Upon closer inspection,
> the running process came to a virtual halt with a kernel-panic-message
> in the journal.

Please report this panic message.

> # ls /proc/sys/net/netfilter/net.netfilter.nf_conntrack_helper
>     ls: File not found
> 
> I cannot enable the conntrack-helper with '1'.

This workaround was removed.

You need to assign the helper to use in your nftables ruleset.

ct helper ftp-standard {
    type "ftp" protocol tcp
}

And then, from a prerouting chain:

tcp dport 21 ct helper set "ftp-standard"

You might need to do this from output too if you need
this to work from the machine itself as well.

Re: Nftables + ALG + Linux 6.1.0-10-amd64 …?... is it a kown Problem?

[toml]

Hello Florian

Thank you for your response! 

Am Montag, dem 31.07.2023 um 15:36 +0200 schrieb Florian Westphal:
> You need to assign the helper to use in your nftables ruleset.
> 

> You might need to do this from output too if you need
> this to work from the machine itself as well.

That is my test rule. My Server is both, FTP-Server (Web-Cams) and FTP-
Client for various uploads.

If I understand correctly, incoming packets (as FTP-Server) will first
activate the helper in prerouting. The input rules then allow the
control channel port 21 and the helper the (related) data channel port
n.

For outgoing packets (as FTP-Client) first the helper is activated in
the output chain, then port 21 is allowed again, the helper handles the
related data channel. 

Have I understood this correctly?


table ip filter {
  ct helper ftp-helper {
      type "ftp" protocol tcp
      l3proto ip
  }

  chain prerouting {
      type filter hook prerouting priority -100; policy accept;
      ct state 0x8 tcp dport 21 ct helper set "ftp-helper"
  }

  chain input {
      type filter hook input priority 0; policy accept;
      tcp dport 21 accept
      ct helper "ftp" accept
  }

  chain output {
      type filter hook output priority 0; policy accept;
      ct state 0x8 tcp dport 21 ct helper set "ftp-helper"
      ct state 0x8 tcp dport 21 accept
      ct helper "ftp" accept
  }
}

Best Regards
Thomas

Re: Nftables + ALG + Linux 6.1.0-10-amd64 …?... is it a kown Problem?

[Florian Westphal]

toml <toml@thlu.de> wrote:
> Am Montag, dem 31.07.2023 um 15:36 +0200 schrieb Florian Westphal:
> > You need to assign the helper to use in your nftables ruleset.
> > 
> 
> > You might need to do this from output too if you need
> > this to work from the machine itself as well.
> 
> That is my test rule. My Server is both, FTP-Server (Web-Cams) and FTP-
> Client for various uploads.
> 
> If I understand correctly, incoming packets (as FTP-Server) will first
> activate the helper in prerouting. The input rules then allow the
> control channel port 21 and the helper the (related) data channel port
> n.
>
> For outgoing packets (as FTP-Client) first the helper is activated in
> the output chain, then port 21 is allowed again, the helper handles the
> related data channel. 
> 
> Have I understood this correctly?

Sounds about right, helper assignment looks correct to me.

Re: Nftables + ALG + Linux 6.1.0-10-amd64 …?... is it a kown Problem?

[toml]

Am Dienstag, dem 01.08.2023 um 22:11 +0200 schrieb Florian Westphal:
> > Have I understood this correctly?
> 
> Sounds about right, helper assignment looks correct to me.

Thank you!   :-)

Best Regards
Thomas

Re: Nftables + ALG + Linux 6.1.0-10-amd64 …?... is it a kown Problem?

[toml]

Am Dienstag, dem 01.08.2023 um 22:11 +0200 schrieb Florian Westphal:

> Sounds about right, helper assignment looks correct to me.

Unfortunately, the example from my previous post does not work, when I tighten the filter to prevent unwanted traffic. For outgoing FTP I always get the message:
"Could not open data connection to port nnnnn: Connection refused".

I then just created a log entry before the general-drop (like policy) takes effect. And surprise, S-Port 20 is the Problem. This Package going dropped.
Aug 03 13:54:12 ftps kernel: Debugging: IN=enp1s0 OUT= MAC=0a:cc:00:22:44:0a:e0:28:zv:11:5f:ff:gg:00 SRC=222.178.19.133 DST=10.1.1.4 LEN=60 TOS=0x00 PREC=0x00 TTL
1=55 ID=30979 DF PROTO=TCP SPT=20 DPT=38909 WINDOW=64240 RES=0x00 SYN URGP=0 

The Package is marked with "SYN", but seems it is not related to the connection via Port 21.  But, also open for SYN in general, also suitable to establish a another connection as friendly. This was not necessary until now with the old ALG-model in Bullseye. Could it be, that I myself and my expectation am the source of the Problem here? 
The following sample works. But I don't know, is it that, what I want... a closed filter with explicit permissions? 

table ip filter {
        ct helper ftp-helper {
                type "ftp" protocol tcp
                l3proto ip
        }

        chain prerouting {
                type filter hook prerouting priority -100; policy accept;
                ct state 0x8 tcp dport 21 ct helper set "ftp-helper"
        }

        chain input {
                type filter hook input priority 0; policy accept;
                iifname "lo" accept
                ct state 0x2,0x4 accept
                ip protocol 1 accept
                meta pkttype { 1, 2 } accept
                tcp sport 20 accept
                tcp dport 21 accept
                ct helper "ftp" accept
                log prefix "Debugging: "
                ip protocol 6 counter packets 0 bytes 0 reject with tcp reset
                counter packets 0 bytes 0 reject
        }

        chain output {
                type filter hook output priority 0; policy accept;
                oifname "lo" accept
                ct state 0x2,0x4 accept
                meta pkttype { 0, 1, 2 } accept
                ip protocol 1 accept
                ct state 0x8 tcp dport 21 ct helper set "ftp-helper"
                tcp dport 21 accept
                ct helper "ftp" accept
                counter packets 0 bytes 0 reject with icmp 13
        }
}


Best Regards
Thomas

Re: Nftables + ALG + Linux 6.1.0-10-amd64 …?... is it a kown Problem?

[Florian Westphal]

toml <toml@thlu.de> wrote:
> Am Dienstag, dem 01.08.2023 um 22:11 +0200 schrieb Florian Westphal:
> 
> > Sounds about right, helper assignment looks correct to me.
> 
> Unfortunately, the example from my previous post does not work, when I tighten the filter to prevent unwanted traffic. For outgoing FTP I always get the message:
> "Could not open data connection to port nnnnn: Connection refused".

The helper is not active for that connection.

>         chain output {
>                 type filter hook output priority 0; policy accept;
>                 oifname "lo" accept
>                 ct state 0x2,0x4 accept
>                 meta pkttype { 0, 1, 2 } accept
>                 ip protocol 1 accept
>                 ct state 0x8 tcp dport 21 ct helper set "ftp-helper"

adding a 'counter' after set "ftp-helper" will show that its never set.

I suspect packets are eaten by the preceeding pkttype rule.
You can test via

nft insert rule ip filter output tcp flags syn tcp dport 21 meta nftrace set 1

then run 'nft monitor' and try to connect to a tcp server.