Re: Clarification on the functioning of "ct count"

[Pablo Neira Ayuso <pablo@netfilter.org>]

Hi,

On Mon, Sep 30, 2024 at 01:09:11PM +0530, Avinash Dige wrote:
> Posting this again without HTML formatting in case the first mail has
> been rejected. I have questions about how exactly using "ct count" or
> "ct count over" in dynamic nft sets works. I want to implement nft
> rules for the following use cases:
> 1. Limit the max TCP sessions from one host to 5000.
> 2. Limit the max TCP sessions to one host to 5000.
> 
> I tried using 'ct count' for the 1st use case. I have questions about
> how exactly using "ct count" or "ct count over" in dynamic nft sets
> work.
> https://wiki.nftables.org/wiki-nftables/index.php/Meters#Doing_connlimit_with_nft
> I tried the example from the above link: (For testing, just changed
> the "ct count over 20" part to "ct count over 3" & for my use case I
> would change it to 5000.)
> '''
> table ip filter {
> set my_connlimit {
> type ipv4_addr
> size 65535
> flags dynamic
> }
> 
> chain output {
> type filter hook output priority filter; policy accept;
> ct state new add @my_connlimit { ip saddr ct count over 3 } counter drop
> }
> }
> '''
> I have a setup of 3 machines (VMs).
> Host machine: 192.168.56.1
> Then on virtualbox, I set up 2 VMs.
> Machine 1: 192.168.56.101
> Machine 2: 192.168.56.103

What is your topology to reach the VMs? Is this a bridged or routed?

> I put the above nft rules on "Machine 1".
> I have recorded videos.
> Left half of the screen: Top 4 terminal windows are from the "host machine".
> Left half of the screen: Bottom 4 terminal windows are from the "Machine 2".
> Right half of the screen: Output on "Machine 1" for the commands:
> 'sudo conntrack -L' & 'sudo nft list table ip filter'. We can monitor
> the actual conntrack tables entries on "Machine 1" & the nft set
> elements in the my_connlimit set on "Machine 1".
> Then I tried the following test- "Test 1".
> From "Host machine", initialised 4 ssh connections to "Machine 1".
> (Watch the top 4 terminal windows on the left half of the screen)
> Expectation: 192.168.56.1 should get added to the my_connlimit set
> after the number of connections from 192.168.56.1 exceeds 3.

- Your ruleset is in "Machine 1" (one of the VMs)
- You start ssh from "Host machine" to "Machine 1"
- Your ruleset in "Machine 1" is not correct, you use the output chain:

        type filter hook output priority filter; policy accept;

  it seems your intention is to use input instead:

        type filter hook input priority filter; policy accept;

  add this to "Machine 1" instead:

-o-
flush ruleset

table ip filter {
        set my_connlimit {
                type ipv4_addr
                size 65535
                flags dynamic
        }

        chain input {
                type filter hook input priority filter; policy accept;
                ct state new add @my_connlimit { ip saddr ct count over 3 } counter drop
        }
}
-o-

> Actual observation: 192.168.56.1 does not get added to the my_connlimit set.
> I have recorded a video of the test: test1.webm (Watch the right half
> of the screen)

Right, because your ruleset uses the output chain, not the input
chain to restrict the maximum number of incoming connection to
"Machine 1".

> Then I tried the following test- "Test 2".
> From "Machine 2", initialised 4 ssh connections to "Machine 1". (Watch
> the bottom 4 terminal windows on the left half of the screen)
> Expectation: 192.168.56.103 should get added to the my_connlimit set
> after the number of connections from 192.168.56.103 exceeds 3.
> Actual observation: 192.168.56.103 does not get added to the my_connlimit set.
> I have recorded a video of the test: test1.webm (Watch the right half
> of the screen, that's the c)
> 
> I. I am having trouble understanding how exactly using "ct count" or
> "ct count over" in dynamic nft sets work.
> The documentation says- 'For each packet matching this rule, it adds
> an element to the set whose key is 'ip saddr' and it allows a maximum
> number of 20 established connections to such IP source addresses.'
> Does that mean-
> 1. For a particular source IP address, when the first packet matches
> this rule, its 'ip saddr ct count over 20' would get added to the
> my_connlimit set.
> 2. When the ct count for this source IP address goes over 20, we would
> start dropping these packets.
> II. What changes do I need to do to make the rule work so that I can
> track connection count individually for separate IP addresses? I need
> to do this without having separate rules for each IP address.

No.

> Having separate nft rules for each IP address manually works (Rules like below)
> '''
> meta l4proto tcp ct state new ip saddr 192.168.56.102 ct count over
> 5000 add @my_connlimit { ip saddr } counter
> meta l4proto tcp ct state new ip saddr 192.168.56.103 ct count over
> 5000 add @my_connlimit { ip saddr } counter
> meta l4proto tcp ct state new ip saddr 192.168.56.104 ct count over
> 5000 add @my_connlimit { ip saddr } counter
> meta l4proto tcp ct state new ip saddr 192.168.56.105 ct count over
> 5000 add @my_connlimit { ip saddr } counter
> '''

No, this is not correct.

> But this is not a feasible solution as the number of clients
> increases. Thus I am looking at a rule like 'meta l4proto tcp ct state
> new add @my_connlimit { ip saddr ct count over 5000 } counter drop' to
> work, but it is not working as expected.
> 
> 
> -- 
> 
> Avinash Dige
> 9028317335