Clarification on the functioning of "ct count"

Clarification on the functioning of "ct count"

[Avinash Dige]

[-- Attachment #1.1: Type: text/plain, Size: 4649 bytes --]

I have questions about how exactly using "ct count" or "ct count over" in
dynamic nft sets works. I want to implement nft rules for the following use
cases:
1. Limit the max TCP sessions from one host to 5000.
2. Limit the max TCP sessions to one host to 5000.

I tried using 'ct count' for the 1st use case. But I have questions about
it.
Documentation:
https://wiki.nftables.org/wiki-nftables/index.php/Meters#Doing_connlimit_with_nft
I tried the example from the above link: (For testing, just changed the "*ct
count over 20*" part to "*ct count over 3*" & for my use case I would
change it to 5000.)
```











*table ip filter {    set my_connlimit {            type ipv4_addr
  size 65535            flags dynamic    }    chain output {
type filter hook output priority filter; policy accept;            ct state
new add @my_connlimit { ip saddr ct count over 3 } counter drop    }}*
```

I did a setup of 3 machines (including VMs).
*Host machine: 192.168.56.1*
Then on virtualbox, I set up 2 VMs.

*Machine 1: 192.168.56.101Machine 2: 192.168.56.103*
I put the above nft rules on "Machine 1".
I have recorded videos of the tests I conducted.
In the video:
i. Left half of the screen: Top 4 terminal windows are from the "host
machine".
ii. Left half of the screen: Bottom 4 terminal windows are from the
"Machine 2".
iii. Right half of the screen: Output on "Machine 1" for the commands:
'sudo conntrack -L' & 'sudo nft list table ip filter'. We can monitor the
actual conntrack tables entries on "Machine 1" & the nft set elements in
the my_connlimit set on "Machine 1".
Then I tried the following test- "Test 1".
    From "Host machine", initialised 4 ssh connections to "Machine 1".
(Watch the top 4 terminal windows on the left half of the screen)
    Expectation: 192.168.56.1 should get added to the my_connlimit set
after the number of connections from 192.168.56.1 exceeds 3.
    Actual observation: 192.168.56.1 does not get added to the my_connlimit
set. Although the conntrack table correctly shows 4 new entries from
192.168.56.1.
I have recorded & attached the video of the test:* test1.webm* (Watch the
right half of the screen for conntrack table & nft set outputs on "Machine
1")

Then I tried the following test- "Test 2".
    From "Machine 2", initialised 4 ssh connections to "Machine 1". (Watch
the bottom 4 terminal windows on the left half of the screen)
    Expectation: 192.168.56.103 should get added to the my_connlimit set
after the number of connections from 192.168.56.103 exceeds 3.
    Actual observation: 192.168.56.103 does not get added to the
my_connlimit set. Although the conntrack table correctly shows 4 new
entries from 192.168.56.103.
I have recorded & attached the video of the test: *test2.webm* (Watch the
right half of the screen for conntrack table & nft set outputs on "Machine
1")

# I. So I am having trouble understanding & need a clarification on how
exactly using "ct count" or "ct count over" in dynamic nft sets work.
    The documentation says- 'For each packet matching this rule, it adds an
element to the set whose key is 'ip saddr' and it allows a maximum number
of 20 established connections to such IP source addresses.'
    Does that mean-
    1. For a particular source IP address, when the first packet matches
this rule, its 'ip saddr ct count over 20' would get added to the
my_connlimit set.
    2. When the ct count for this source IP address goes over 20, we would
start dropping these packets.
# II. What changes do I need to do to make the rule work so that I can
track connection count individually for separate IP addresses? I need to do
this without having separate rules for each IP address.
    Having separate nft rules for each IP address manually works (Rules
like below)
    ```
*    meta l4proto tcp ct state new ip saddr 192.168.56.1 ct count over 5000
add @my_connlimit { ip saddr } counter *




*    meta l4proto tcp ct state new ip saddr 192.168.56.102 ct count over
5000 add @my_connlimit { ip saddr } counter     meta l4proto tcp ct state
new ip saddr 192.168.56.103 ct count over 5000 add @my_connlimit { ip saddr
} counter     meta l4proto tcp ct state new ip saddr 192.168.56.104 ct
count over 5000 add @my_connlimit { ip saddr } counter     meta l4proto tcp
ct state new ip saddr 192.168.56.105 ct count over 5000 add @my_connlimit {
ip saddr } counter     ```*
    But this is not a feasible solution as the number of clients increases.
Thus I am looking at a rule like `*meta l4proto tcp ct state new add
@my_connlimit { ip saddr ct count over 5000 } counter drop*` to work, but
it is not working as expected.


-- 


*Avinash Dige9028317335*

[-- Attachment #1.2: Type: text/html, Size: 5679 bytes --]

[-- Attachment #2: test2.webm --]
[-- Type: video/webm, Size: 1125978 bytes --]

[-- Attachment #3: test1.webm --]
[-- Type: video/webm, Size: 994189 bytes --]

Clarification on the functioning of "ct count"

[Avinash Dige]

[-- Attachment #1: Type: text/plain, Size: 4146 bytes --]

Posting this again without HTML formatting in case the first mail has
been rejected. I have questions about how exactly using "ct count" or
"ct count over" in dynamic nft sets works. I want to implement nft
rules for the following use cases:
1. Limit the max TCP sessions from one host to 5000.
2. Limit the max TCP sessions to one host to 5000.

I tried using 'ct count' for the 1st use case. I have questions about
how exactly using "ct count" or "ct count over" in dynamic nft sets
work.
https://wiki.nftables.org/wiki-nftables/index.php/Meters#Doing_connlimit_with_nft
I tried the example from the above link: (For testing, just changed
the "ct count over 20" part to "ct count over 3" & for my use case I
would change it to 5000.)
'''
table ip filter {
set my_connlimit {
type ipv4_addr
size 65535
flags dynamic
}

chain output {
type filter hook output priority filter; policy accept;
ct state new add @my_connlimit { ip saddr ct count over 3 } counter drop
}
}
'''
I have a setup of 3 machines (VMs).
Host machine: 192.168.56.1
Then on virtualbox, I set up 2 VMs.
Machine 1: 192.168.56.101
Machine 2: 192.168.56.103
I put the above nft rules on "Machine 1".
I have recorded videos.
Left half of the screen: Top 4 terminal windows are from the "host machine".
Left half of the screen: Bottom 4 terminal windows are from the "Machine 2".
Right half of the screen: Output on "Machine 1" for the commands:
'sudo conntrack -L' & 'sudo nft list table ip filter'. We can monitor
the actual conntrack tables entries on "Machine 1" & the nft set
elements in the my_connlimit set on "Machine 1".
Then I tried the following test- "Test 1".
From "Host machine", initialised 4 ssh connections to "Machine 1".
(Watch the top 4 terminal windows on the left half of the screen)
Expectation: 192.168.56.1 should get added to the my_connlimit set
after the number of connections from 192.168.56.1 exceeds 3.
Actual observation: 192.168.56.1 does not get added to the my_connlimit set.
I have recorded a video of the test: test1.webm (Watch the right half
of the screen)

Then I tried the following test- "Test 2".
From "Machine 2", initialised 4 ssh connections to "Machine 1". (Watch
the bottom 4 terminal windows on the left half of the screen)
Expectation: 192.168.56.103 should get added to the my_connlimit set
after the number of connections from 192.168.56.103 exceeds 3.
Actual observation: 192.168.56.103 does not get added to the my_connlimit set.
I have recorded a video of the test: test1.webm (Watch the right half
of the screen, that's the c)

I. I am having trouble understanding how exactly using "ct count" or
"ct count over" in dynamic nft sets work.
The documentation says- 'For each packet matching this rule, it adds
an element to the set whose key is 'ip saddr' and it allows a maximum
number of 20 established connections to such IP source addresses.'
Does that mean-
1. For a particular source IP address, when the first packet matches
this rule, its 'ip saddr ct count over 20' would get added to the
my_connlimit set.
2. When the ct count for this source IP address goes over 20, we would
start dropping these packets.
II. What changes do I need to do to make the rule work so that I can
track connection count individually for separate IP addresses? I need
to do this without having separate rules for each IP address.
Having separate nft rules for each IP address manually works (Rules like below)
'''
meta l4proto tcp ct state new ip saddr 192.168.56.102 ct count over
5000 add @my_connlimit { ip saddr } counter
meta l4proto tcp ct state new ip saddr 192.168.56.103 ct count over
5000 add @my_connlimit { ip saddr } counter
meta l4proto tcp ct state new ip saddr 192.168.56.104 ct count over
5000 add @my_connlimit { ip saddr } counter
meta l4proto tcp ct state new ip saddr 192.168.56.105 ct count over
5000 add @my_connlimit { ip saddr } counter
'''
But this is not a feasible solution as the number of clients
increases. Thus I am looking at a rule like 'meta l4proto tcp ct state
new add @my_connlimit { ip saddr ct count over 5000 } counter drop' to
work, but it is not working as expected.


-- 

Avinash Dige
9028317335

[-- Attachment #2: test2.webm --]
[-- Type: video/webm, Size: 1125978 bytes --]

[-- Attachment #3: test1.webm --]
[-- Type: video/webm, Size: 994189 bytes --]

Re: Clarification on the functioning of "ct count"

[Pablo Neira Ayuso]

Hi,

On Mon, Sep 30, 2024 at 01:09:11PM +0530, Avinash Dige wrote:
> Posting this again without HTML formatting in case the first mail has
> been rejected. I have questions about how exactly using "ct count" or
> "ct count over" in dynamic nft sets works. I want to implement nft
> rules for the following use cases:
> 1. Limit the max TCP sessions from one host to 5000.
> 2. Limit the max TCP sessions to one host to 5000.
> 
> I tried using 'ct count' for the 1st use case. I have questions about
> how exactly using "ct count" or "ct count over" in dynamic nft sets
> work.
> https://wiki.nftables.org/wiki-nftables/index.php/Meters#Doing_connlimit_with_nft
> I tried the example from the above link: (For testing, just changed
> the "ct count over 20" part to "ct count over 3" & for my use case I
> would change it to 5000.)
> '''
> table ip filter {
> set my_connlimit {
> type ipv4_addr
> size 65535
> flags dynamic
> }
> 
> chain output {
> type filter hook output priority filter; policy accept;
> ct state new add @my_connlimit { ip saddr ct count over 3 } counter drop
> }
> }
> '''
> I have a setup of 3 machines (VMs).
> Host machine: 192.168.56.1
> Then on virtualbox, I set up 2 VMs.
> Machine 1: 192.168.56.101
> Machine 2: 192.168.56.103

What is your topology to reach the VMs? Is this a bridged or routed?

> I put the above nft rules on "Machine 1".
> I have recorded videos.
> Left half of the screen: Top 4 terminal windows are from the "host machine".
> Left half of the screen: Bottom 4 terminal windows are from the "Machine 2".
> Right half of the screen: Output on "Machine 1" for the commands:
> 'sudo conntrack -L' & 'sudo nft list table ip filter'. We can monitor
> the actual conntrack tables entries on "Machine 1" & the nft set
> elements in the my_connlimit set on "Machine 1".
> Then I tried the following test- "Test 1".
> From "Host machine", initialised 4 ssh connections to "Machine 1".
> (Watch the top 4 terminal windows on the left half of the screen)
> Expectation: 192.168.56.1 should get added to the my_connlimit set
> after the number of connections from 192.168.56.1 exceeds 3.

- Your ruleset is in "Machine 1" (one of the VMs)
- You start ssh from "Host machine" to "Machine 1"
- Your ruleset in "Machine 1" is not correct, you use the output chain:

        type filter hook output priority filter; policy accept;

  it seems your intention is to use input instead:

        type filter hook input priority filter; policy accept;

  add this to "Machine 1" instead:

-o-
flush ruleset

table ip filter {
        set my_connlimit {
                type ipv4_addr
                size 65535
                flags dynamic
        }

        chain input {
                type filter hook input priority filter; policy accept;
                ct state new add @my_connlimit { ip saddr ct count over 3 } counter drop
        }
}
-o-

> Actual observation: 192.168.56.1 does not get added to the my_connlimit set.
> I have recorded a video of the test: test1.webm (Watch the right half
> of the screen)

Right, because your ruleset uses the output chain, not the input
chain to restrict the maximum number of incoming connection to
"Machine 1".

> Then I tried the following test- "Test 2".
> From "Machine 2", initialised 4 ssh connections to "Machine 1". (Watch
> the bottom 4 terminal windows on the left half of the screen)
> Expectation: 192.168.56.103 should get added to the my_connlimit set
> after the number of connections from 192.168.56.103 exceeds 3.
> Actual observation: 192.168.56.103 does not get added to the my_connlimit set.
> I have recorded a video of the test: test1.webm (Watch the right half
> of the screen, that's the c)
> 
> I. I am having trouble understanding how exactly using "ct count" or
> "ct count over" in dynamic nft sets work.
> The documentation says- 'For each packet matching this rule, it adds
> an element to the set whose key is 'ip saddr' and it allows a maximum
> number of 20 established connections to such IP source addresses.'
> Does that mean-
> 1. For a particular source IP address, when the first packet matches
> this rule, its 'ip saddr ct count over 20' would get added to the
> my_connlimit set.
> 2. When the ct count for this source IP address goes over 20, we would
> start dropping these packets.
> II. What changes do I need to do to make the rule work so that I can
> track connection count individually for separate IP addresses? I need
> to do this without having separate rules for each IP address.

No.

> Having separate nft rules for each IP address manually works (Rules like below)
> '''
> meta l4proto tcp ct state new ip saddr 192.168.56.102 ct count over
> 5000 add @my_connlimit { ip saddr } counter
> meta l4proto tcp ct state new ip saddr 192.168.56.103 ct count over
> 5000 add @my_connlimit { ip saddr } counter
> meta l4proto tcp ct state new ip saddr 192.168.56.104 ct count over
> 5000 add @my_connlimit { ip saddr } counter
> meta l4proto tcp ct state new ip saddr 192.168.56.105 ct count over
> 5000 add @my_connlimit { ip saddr } counter
> '''

No, this is not correct.

> But this is not a feasible solution as the number of clients
> increases. Thus I am looking at a rule like 'meta l4proto tcp ct state
> new add @my_connlimit { ip saddr ct count over 5000 } counter drop' to
> work, but it is not working as expected.
> 
> 
> -- 
> 
> Avinash Dige
> 9028317335