Flowtables ignore timeout settings in recent kernels

Flowtables ignore timeout settings in recent kernels

[Vladimir Smelhaus]

[-- Attachment #1: Type: text/plain, Size: 1101 bytes --]

Netfilter ignores the timeout settings for a flowtable

# sysctl -a -r flowtable
net.netfilter.nf_flowtable_tcp_timeout = 30
net.netfilter.nf_flowtable_udp_timeout = 30

Situation. A long udp connection (tunnel) with some data flowing through a router. The connection is sent to a flowtable on the router. It's a few packets per second, more here and there, a pause here and there, and so on over and over. The pauses are 
minimal and are also limited by the tunnel settings to be no longer than 25 seconds. Everything is satisfying to make the connection last continuously in the flowtable and not reappear in forward. However, the connection keeps dropping out of the 
flowtable. It stays in the flowtable (offloaded) for a second at most and then it is kicked out, back to forward.

In an attached test script you can see counters that should be zero but are not. If I watch the normal packet flow on a particular router, I can see packets in the conntrack table that should be OFFLOAD as ASSURED.

Tested in kernel 6.5.6. In an old(er) kernel 5.10 it works as expected.

Regards

Vladimir Smelhaus

[-- Attachment #2: test_flowtable.sh --]
[-- Type: application/x-shellscript, Size: 10791 bytes --]

Re: Flowtables ignore timeout settings in recent kernels

[Bagas Sanjaya]

[-- Attachment #1: Type: text/plain, Size: 1710 bytes --]

On Sun, Oct 15, 2023 at 09:56:14PM +0200, Vladimir Smelhaus wrote:
> Netfilter ignores the timeout settings for a flowtable
> 
> # sysctl -a -r flowtable
> net.netfilter.nf_flowtable_tcp_timeout = 30
> net.netfilter.nf_flowtable_udp_timeout = 30
> 
> Situation. A long udp connection (tunnel) with some data flowing through a
> router. The connection is sent to a flowtable on the router. It's a few
> packets per second, more here and there, a pause here and there, and so on
> over and over. The pauses are minimal and are also limited by the tunnel
> settings to be no longer than 25 seconds. Everything is satisfying to make
> the connection last continuously in the flowtable and not reappear in
> forward. However, the connection keeps dropping out of the flowtable. It
> stays in the flowtable (offloaded) for a second at most and then it is
> kicked out, back to forward.
> 
> In an attached test script you can see counters that should be zero but are not. If I watch the normal packet flow on a particular router, I can see packets in the conntrack table that should be OFFLOAD as ASSURED.
> 
> Tested in kernel 6.5.6. In an old(er) kernel 5.10 it works as expected.
> 

Then please perform bisection to find a culprit that introduces your
regression (see Documentation/admin-guide/bug-bisect.rst in the kernel
sources for reference). Also, it'd been great if you also post the
reproducer script inline (within your email) instead, as some MUAs
(like mutt that I'm using now) may ignore the attachment.

Anyway, thanks for the regression report. I'm adding it to regzbot:

#regzbot ^introduced: v5.10..v6.5

-- 
An old man doll... just what I always wanted! - Clara

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: Flowtables ignore timeout settings in recent kernels

[Pablo Neira Ayuso]

On Mon, Oct 16, 2023 at 07:20:06AM +0700, Bagas Sanjaya wrote:
> On Sun, Oct 15, 2023 at 09:56:14PM +0200, Vladimir Smelhaus wrote:
> > Netfilter ignores the timeout settings for a flowtable
> > 
> > # sysctl -a -r flowtable
> > net.netfilter.nf_flowtable_tcp_timeout = 30
> > net.netfilter.nf_flowtable_udp_timeout = 30
> > 
> > Situation. A long udp connection (tunnel) with some data flowing through a
> > router. The connection is sent to a flowtable on the router. It's a few
> > packets per second, more here and there, a pause here and there, and so on
> > over and over. The pauses are minimal and are also limited by the tunnel
> > settings to be no longer than 25 seconds. Everything is satisfying to make
> > the connection last continuously in the flowtable and not reappear in
> > forward. However, the connection keeps dropping out of the flowtable. It
> > stays in the flowtable (offloaded) for a second at most and then it is
> > kicked out, back to forward.
> > 
> > In an attached test script you can see counters that should be zero but are not. If I watch the normal packet flow on a particular router, I can see packets in the conntrack table that should be OFFLOAD as ASSURED.
> > 
> > Tested in kernel 6.5.6. In an old(er) kernel 5.10 it works as expected.
> > 
> 
> Then please perform bisection to find a culprit that introduces your
> regression (see Documentation/admin-guide/bug-bisect.rst in the kernel
> sources for reference). Also, it'd been great if you also post the
> reproducer script inline (within your email) instead, as some MUAs
> (like mutt that I'm using now) may ignore the attachment.
> 
> Anyway, thanks for the regression report. I'm adding it to regzbot:
> 
> #regzbot ^introduced: v5.10..v6.5

Fix here:

https://patchwork.ozlabs.org/project/netfilter-devel/patch/20231024193815.1987-1-pablo@netfilter.org/

it is a bug from Jun 2023, regression was introduced in the v6.5
development cycle.

Thanks for reporting.

Re: Flowtables ignore timeout settings in recent kernels

[Bagas Sanjaya]

[-- Attachment #1: Type: text/plain, Size: 2305 bytes --]

On Tue, Oct 24, 2023 at 10:11:44PM +0200, Pablo Neira Ayuso wrote:
> On Mon, Oct 16, 2023 at 07:20:06AM +0700, Bagas Sanjaya wrote:
> > On Sun, Oct 15, 2023 at 09:56:14PM +0200, Vladimir Smelhaus wrote:
> > > Netfilter ignores the timeout settings for a flowtable
> > > 
> > > # sysctl -a -r flowtable
> > > net.netfilter.nf_flowtable_tcp_timeout = 30
> > > net.netfilter.nf_flowtable_udp_timeout = 30
> > > 
> > > Situation. A long udp connection (tunnel) with some data flowing through a
> > > router. The connection is sent to a flowtable on the router. It's a few
> > > packets per second, more here and there, a pause here and there, and so on
> > > over and over. The pauses are minimal and are also limited by the tunnel
> > > settings to be no longer than 25 seconds. Everything is satisfying to make
> > > the connection last continuously in the flowtable and not reappear in
> > > forward. However, the connection keeps dropping out of the flowtable. It
> > > stays in the flowtable (offloaded) for a second at most and then it is
> > > kicked out, back to forward.
> > > 
> > > In an attached test script you can see counters that should be zero but are not. If I watch the normal packet flow on a particular router, I can see packets in the conntrack table that should be OFFLOAD as ASSURED.
> > > 
> > > Tested in kernel 6.5.6. In an old(er) kernel 5.10 it works as expected.
> > > 
> > 
> > Then please perform bisection to find a culprit that introduces your
> > regression (see Documentation/admin-guide/bug-bisect.rst in the kernel
> > sources for reference). Also, it'd been great if you also post the
> > reproducer script inline (within your email) instead, as some MUAs
> > (like mutt that I'm using now) may ignore the attachment.
> > 
> > Anyway, thanks for the regression report. I'm adding it to regzbot:
> > 
> > #regzbot ^introduced: v5.10..v6.5
> 
> Fix here:
> 
> https://patchwork.ozlabs.org/project/netfilter-devel/patch/20231024193815.1987-1-pablo@netfilter.org/
> 
> it is a bug from Jun 2023, regression was introduced in the v6.5
> development cycle.
> 

Telling regzbot:

#regzbot fix: netfilter: nf_flow_table: GC pushes back packets to classic path

Thanks.

-- 
An old man doll... just what I always wanted! - Clara

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]