[nftables] netdev rate limiting | timeouts rfq

[nftables] netdev rate limiting | timeouts rfq

[ѽ҉ᶬḳ℠]

[-- Attachment #1: Type: text/plain, Size: 1991 bytes --]

kernel 5.9.0-rc6 armv7l | nft 0.9.6

------

(1) netdev rate limiting

/table netdev filter {//
//        set glv4 {//
//                type ipv4_addr//
//                size 65535//
//                flags dynamic,timeout//
//                counter//
//                timeout 15m//
//                gc-interval 10s//
//        }//
//        chain input {//
//                type filter hook ingress device "eth2" priority -500;
policy accept;//
//                ip saddr @glv4 counter packets 0 bytes 0 log prefix
"nd glv4 DROP" flags all drop//
//        }

/Then however

/nft add rule netdev filter input ip protocol tcp { ip saddr limit rate
over 50/second burst 15 packets } update @glv4 { ip saddr }/

producing:

Error: syntax error, unexpected saddr, expecting comma or '}'
add rule netdev filter input ip protocol tcp { ip saddr limit rate over
50/second burst 15 packets } update @glv4 { ip saddr }
^^^^^

Is rate limiting not meant for the netdev chain or where do I go wrong
with the syntax?

----

(2) timeouts

Like in the above statement is the timeout applicable to the entire set,
i.e. the entire gets wiped after 15 min, or the timeout applies
selectively to each entry?

Perusing
https://www.netfilter.org/projects/nftables/manpage.html#lbANhttps://www.netfilter.org/projects/nftables/manpage.html#lbAN

/nft add rule ip filter input tcp flags syn tcp dport ssh meter flood
size 128000 { ip saddr *timeout 10s* limit rate over 10/second} add
@blackhole { ip saddr *timeout 1m* } drop

/I fail to comprehend what the first statement (/timeout 10s/) achieves;
what is the purpose?

As for the second statement (/timeout 1m/) I am wondering whether it
would clash with or supersede the statement in the set (e.g. /timeout
15m/ as in the above example)?
//



[-- Attachment #2: OpenPGP_0xF4F735931F05C5CE.asc --]
[-- Type: application/pgp-keys, Size: 3112 bytes --]

Re: [nftables] netdev rate limiting | timeouts rfq

[Pablo Neira Ayuso]

On Tue, Sep 22, 2020 at 10:49:00PM +0000, ѽ҉ᶬḳ℠ wrote:
> kernel 5.9.0-rc6 armv7l | nft 0.9.6
> 
> ------
> 
> (1) netdev rate limiting
> 
> /table netdev filter {//
> //        set glv4 {//
> //                type ipv4_addr//
> //                size 65535//
> //                flags dynamic,timeout//
> //                counter//
> //                timeout 15m//
> //                gc-interval 10s//
> //        }//
> //        chain input {//
> //                type filter hook ingress device "eth2" priority -500;
> policy accept;//
> //                ip saddr @glv4 counter packets 0 bytes 0 log prefix
> "nd glv4 DROP" flags all drop//
> //        }
> 
> /Then however
> 
> /nft add rule netdev filter input ip protocol tcp { ip saddr limit rate
> over 50/second burst 15 packets } update @glv4 { ip saddr }/

table ip filter {
        set glv4 {
                type ipv4_addr
                size 65535
                flags dynamic,timeout
                timeout 15m
                gc-interval 10s
        }

        chain input {
                ip protocol tcp update @glv4 { ip saddr limit rate over 50/second burst 15 packets }
        }
}

This reads: update set 'glv4' with ip saddr using limit rate over
50/second burst 15 packets.

You cannot combine counter and limit yet though.

> producing:
> 
> Error: syntax error, unexpected saddr, expecting comma or '}'
> add rule netdev filter input ip protocol tcp { ip saddr limit rate over
> 50/second burst 15 packets } update @glv4 { ip saddr }
> ^^^^^
> 
> Is rate limiting not meant for the netdev chain or where do I go wrong
> with the syntax?
> 
> ----
> 
> (2) timeouts
> 
> Like in the above statement is the timeout applicable to the entire set,
> i.e. the entire gets wiped after 15 min, or the timeout applies
> selectively to each entry?
> 
> Perusing
> https://www.netfilter.org/projects/nftables/manpage.html#lbANhttps://www.netfilter.org/projects/nftables/manpage.html#lbAN
> 
> /nft add rule ip filter input tcp flags syn tcp dport ssh meter flood
> size 128000 { ip saddr *timeout 10s* limit rate over 10/second} add
> @blackhole { ip saddr *timeout 1m* } drop
> 
> /I fail to comprehend what the first statement (/timeout 10s/) achieves;
> what is the purpose?
> 
> As for the second statement (/timeout 1m/) I am wondering whether it
> would clash with or supersede the statement in the set (e.g. /timeout
> 15m/ as in the above example)?

timeout in your rule (if specified), eg.

... update @xyz { ip saddr *timeout 1m* }

overrides the global set timeout policy.

Re: [nftables] netdev rate limiting | timeouts rfq

[ѽ҉ᶬḳ℠]

On 23/09/2020 10:30, Pablo Neira Ayuso wrote:
> On Tue, Sep 22, 2020 at 10:49:00PM +0000, ѽ҉ᶬḳ℠ wrote:
>> kernel 5.9.0-rc6 armv7l | nft 0.9.6
>>
>> ------
>>
>> (1) netdev rate limiting
>>
>> /table netdev filter {//
>> //        set glv4 {//
>> //                type ipv4_addr//
>> //                size 65535//
>> //                flags dynamic,timeout//
>> //                counter//
>> //                timeout 15m//
>> //                gc-interval 10s//
>> //        }//
>> //        chain input {//
>> //                type filter hook ingress device "eth2" priority -500;
>> policy accept;//
>> //                ip saddr @glv4 counter packets 0 bytes 0 log prefix
>> "nd glv4 DROP" flags all drop//
>> //        }
>>
>> /Then however
>>
>> /nft add rule netdev filter input ip protocol tcp { ip saddr limit rate
>> over 50/second burst 15 packets } update @glv4 { ip saddr }/
> table ip filter {
>          set glv4 {
>                  type ipv4_addr
>                  size 65535
>                  flags dynamic,timeout
>                  timeout 15m
>                  gc-interval 10s
>          }
>
>          chain input {
>                  ip protocol tcp update @glv4 { ip saddr limit rate over 50/second burst 15 packets }
>          }
> }
>
> This reads: update set 'glv4' with ip saddr using limit rate over
> 50/second burst 15 packets.
>
> You cannot combine counter and limit yet though.

It seems that updating a set and dropping the packet in the same rules 
does not work either, e.g.

ip protocol tcp update @glv4 { ip saddr limit rate over 50/second burst 
15 packets } log prefix "foo " drop

would seem to only update the named set but is neither logging nor 
dropping the frame.

>
>> producing:
>>
>> Error: syntax error, unexpected saddr, expecting comma or '}'
>> add rule netdev filter input ip protocol tcp { ip saddr limit rate over
>> 50/second burst 15 packets } update @glv4 { ip saddr }
>> ^^^^^
>>
>> Is rate limiting not meant for the netdev chain or where do I go wrong
>> with the syntax?
>>
>> ----
>>
>> (2) timeouts
>>
>> Like in the above statement is the timeout applicable to the entire set,
>> i.e. the entire gets wiped after 15 min, or the timeout applies
>> selectively to each entry?
>>
>> Perusing
>> https://www.netfilter.org/projects/nftables/manpage.html#lbANhttps://www.netfilter.org/projects/nftables/manpage.html#lbAN
>>
>> /nft add rule ip filter input tcp flags syn tcp dport ssh meter flood
>> size 128000 { ip saddr *timeout 10s* limit rate over 10/second} add
>> @blackhole { ip saddr *timeout 1m* } drop
>>
>> /I fail to comprehend what the first statement (/timeout 10s/) achieves;
>> what is the purpose?
>>
>> As for the second statement (/timeout 1m/) I am wondering whether it
>> would clash with or supersede the statement in the set (e.g. /timeout
>> 15m/ as in the above example)?
> timeout in your rule (if specified), eg.
>
> ... update @xyz { ip saddr *timeout 1m* }
>
> overrides the global set timeout policy.

Re: [nftables] netdev rate limiting | timeouts rfq

[Pablo Neira Ayuso]

On Mon, Sep 28, 2020 at 11:02:00AM +0000, ѽ҉ᶬḳ℠ wrote:
[...]
> It seems that updating a set and dropping the packet in the same rules does
> not work either, e.g.
> 
> ip protocol tcp update @glv4 { ip saddr limit rate over 50/second burst 15
> packets } log prefix "foo " drop
>
> would seem to only update the named set but is neither logging nor dropping
> the frame.

This example is working fine here:

table ip x {
        set glv4 {
                typeof ip saddr
                size 65535
                flags dynamic,timeout
                timeout 15m
        }

        chain y {
                type filter hook output priority filter; policy accept;
                ip protocol tcp update @glv4 { ip daddr limit rate over 1/minute burst 2 packets } log prefix "foo " drop
        }
}

hping3 -S x.x.x.x -p 80 -c 1
HPING x.x.x.x (enp0s25 x.x.x.x): S set, 40 headers + 0 data bytes
len=46 ip=x.x.x.x ttl=53 DF id=0 sport=80 flags=SA seq=0 win=29200 rtt=31.8 ms

--- x.x.x.x hping statistic ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 31.8/31.8/31.8 ms

hping3 -S x.x.x.x -p 80 -c 1
HPING x.x.x.x (enp0s25 x.x.x.x): S set, 40 headers + 0 data bytes
[send_ip] sendto: Operation not permitted

[137712.162113] foo IN= OUT=enp0s25 SRC=10.141.1.219 DST=x.x.x.x LEN=40 TOS=0x00 PREC=0x00 TTL=64 ID=57787 PROTO=TCP SPT=2904 DPT=80 WINDOW=512 RES=0x00 SYN URGP=0

Re: [nftables] netdev rate limiting | timeouts rfq

[ѽ҉ᶬḳ℠]

On 28/09/2020 13:52, Pablo Neira Ayuso wrote:
> On Mon, Sep 28, 2020 at 11:02:00AM +0000, ѽ҉ᶬḳ℠ wrote:
> [...]
>> It seems that updating a set and dropping the packet in the same rules does
>> not work either, e.g.
>>
>> ip protocol tcp update @glv4 { ip saddr limit rate over 50/second burst 15
>> packets } log prefix "foo " drop
>>
>> would seem to only update the named set but is neither logging nor dropping
>> the frame.
> This example is working fine here:
>
> table ip x {
>          set glv4 {
>                  typeof ip saddr
>                  size 65535
>                  flags dynamic,timeout
>                  timeout 15m
>          }
>
>          chain y {
>                  type filter hook output priority filter; policy accept;
>                  ip protocol tcp update @glv4 { ip daddr limit rate over 1/minute burst 2 packets } log prefix "foo " drop
>          }
> }
>
> hping3 -S x.x.x.x -p 80 -c 1
> HPING x.x.x.x (enp0s25 x.x.x.x): S set, 40 headers + 0 data bytes
> len=46 ip=x.x.x.x ttl=53 DF id=0 sport=80 flags=SA seq=0 win=29200 rtt=31.8 ms
>
> --- x.x.x.x hping statistic ---
> 1 packets transmitted, 1 packets received, 0% packet loss
> round-trip min/avg/max = 31.8/31.8/31.8 ms
>
> hping3 -S x.x.x.x -p 80 -c 1
> HPING x.x.x.x (enp0s25 x.x.x.x): S set, 40 headers + 0 data bytes
> [send_ip] sendto: Operation not permitted
>
> [137712.162113] foo IN= OUT=enp0s25 SRC=10.141.1.219 DST=x.x.x.x LEN=40 TOS=0x00 PREC=0x00 TTL=64 ID=57787 PROTO=TCP SPT=2904 DPT=80 WINDOW=512 RES=0x00 SYN URGP=0
>

In the below example it does not print any logs and since that stanza is 
right prior drop I reckon that since the log printing does not happen 
the frame dropping does neither.

table inet filter {
         set n_wan {
                 type ifname
                 flags constant
                 elements = { "eth2", "pppoe-wan" }
         }
         set glv4 {
                 type ipv4_addr
                 size 65535
                 flags dynamic,timeout
                 timeout 7d
                 gc-interval 10s
         }
         flowtable fte {
                 hook ingress priority -100
                 devices = { eth2 }
                 counter
         }
         flowtable ftw {
                 hook ingress priority -100
                 devices = { pppoe-wan }
                 counter
         }
         chain forward {
                 type filter hook forward priority filter; policy drop;
                 icmpv6 type echo-request accept
                 ct state established,related accept
                 iifname != @n_wan accept
                 tcp flags syn tcp option maxseg size set rt mtu
                 ip protocol { tcp, udp } flow add @fte
                 ip6 nexthdr { tcp, udp } flow add @fte
                 ip protocol { tcp, udp } flow add @ftw
                 ip6 nexthdr { tcp, udp } flow add @ftw
         }
         chain prerouting {
                 type filter hook prerouting priority mangle; policy accept;
                 iifname @n_wan goto wan_pr
         }
         chain wan_pr {
                 ct state { new } update @glv4 { ip saddr limit rate 
over 15/day burst 3 packets } log prefix "glv4 DROP: " flags all drop
         }
}

Re: [nftables] netdev rate limiting | timeouts rfq

[Pablo Neira Ayuso]

On Mon, Sep 28, 2020 at 12:08:00PM +0000, ѽ҉ᶬḳ℠ wrote:
> In the below example it does not print any logs and since that stanza is
> right prior drop I reckon that since the log printing does not happen the
> frame dropping does neither.
> 
> table inet filter {
[...]
>         chain wan_pr {
>                 ct state { new } update @glv4 { ip saddr limit rate over
> 15/day burst 3 packets } log prefix "glv4 DROP: " flags all drop
>         }
> }

ct state new is restricting to match only on the first packet of the
connection.

Re: [nftables] netdev rate limiting | timeouts rfq

[ѽ҉ᶬḳ℠]

On 28/09/2020 14:24, Pablo Neira Ayuso wrote:
> On Mon, Sep 28, 2020 at 12:08:00PM +0000, ѽ҉ᶬḳ℠ wrote:
>> In the below example it does not print any logs and since that stanza is
>> right prior drop I reckon that since the log printing does not happen the
>> frame dropping does neither.
>>
>> table inet filter {
> [...]
>>          chain wan_pr {
>>                  ct state { new } update @glv4 { ip saddr limit rate over
>> 15/day burst 3 packets } log prefix "glv4 DROP: " flags all drop
>>          }
>> }
> ct state new is restricting to match only on the first packet of the
> connection.

Right, it should be then rather something with ct count, like:


ct state new update @glv4 { ip saddr ct count over 15 } log flags all 
prefix "glv4 DROP: " drop


but that produces:

Error: Could not process rule: Not supported

??

Re: [nftables] netdev rate limiting | timeouts rfq

[Pablo Neira Ayuso]

On Mon, Sep 28, 2020 at 01:10:00PM +0000, ѽ҉ᶬḳ℠ wrote:
> On 28/09/2020 14:24, Pablo Neira Ayuso wrote:
> > On Mon, Sep 28, 2020 at 12:08:00PM +0000, ѽ҉ᶬḳ℠ wrote:
> > > In the below example it does not print any logs and since that stanza is
> > > right prior drop I reckon that since the log printing does not happen the
> > > frame dropping does neither.
> > > 
> > > table inet filter {
> > [...]
> > >          chain wan_pr {
> > >                  ct state { new } update @glv4 { ip saddr limit rate over
> > > 15/day burst 3 packets } log prefix "glv4 DROP: " flags all drop
> > >          }
> > > }
> > ct state new is restricting to match only on the first packet of the
> > connection.
> 
> Right, it should be then rather something with ct count, like:
> 
> ct state new update @glv4 { ip saddr ct count over 15 } log flags all prefix
> "glv4 DROP: " drop
> 
> but that produces:
> 
> Error: Could not process rule: Not supported

What would you like to achieve in first place?

This thread is starting to diverge from the original question.

Re: [nftables] netdev rate limiting | timeouts rfq

[ѽ҉ᶬḳ℠]

On 28/09/2020 17:43, Pablo Neira Ayuso wrote:
> On Mon, Sep 28, 2020 at 01:10:00PM +0000, ѽ҉ᶬḳ℠ wrote:
>> On 28/09/2020 14:24, Pablo Neira Ayuso wrote:
>>> On Mon, Sep 28, 2020 at 12:08:00PM +0000, ѽ҉ᶬḳ℠ wrote:
>>>> In the below example it does not print any logs and since that stanza is
>>>> right prior drop I reckon that since the log printing does not happen the
>>>> frame dropping does neither.
>>>>
>>>> table inet filter {
>>> [...]
>>>>           chain wan_pr {
>>>>                   ct state { new } update @glv4 { ip saddr limit rate over
>>>> 15/day burst 3 packets } log prefix "glv4 DROP: " flags all drop
>>>>           }
>>>> }
>>> ct state new is restricting to match only on the first packet of the
>>> connection.
>> Right, it should be then rather something with ct count, like:
>>
>> ct state new update @glv4 { ip saddr ct count over 15 } log flags all prefix
>> "glv4 DROP: " drop
>>
>> but that produces:
>>
>> Error: Could not process rule: Not supported
> What would you like to achieve in first place?
>
> This thread is starting to diverge from the original question.

It is still about (rate) limiting offending saddrs with sets, probably 
in a staged approach, say log and drop saddrs like

* ct state update @glv4 { new , invalid } count over 50 timeout 1s
* ct state update @glv4 { new , invalid } count over 75 timeout 1h
* ct state update @glv4 { new , invalid } count over 150 timeout 1d

similar to what been mentioned here 
https://www.spinics.net/lists/netfilter/msg58506.html

where is is mentioned:

This example above also requires kernel >= 4.18

This node currently deploys kernel 5.9.0-rc7 and thus I am surprised 
about the error printing not supported, also with log stanza removed 
from the rule and add instead of update for the set handling.

Re: [nftables] netdev rate limiting | timeouts rfq

[Pablo Neira Ayuso]

On Mon, Sep 28, 2020 at 04:03:00PM +0000, ѽ҉ᶬḳ℠ wrote:
> 
> On 28/09/2020 17:43, Pablo Neira Ayuso wrote:
> > On Mon, Sep 28, 2020 at 01:10:00PM +0000, ѽ҉ᶬḳ℠ wrote:
> > > On 28/09/2020 14:24, Pablo Neira Ayuso wrote:
> > > > On Mon, Sep 28, 2020 at 12:08:00PM +0000, ѽ҉ᶬḳ℠ wrote:
> > > > > In the below example it does not print any logs and since that stanza is
> > > > > right prior drop I reckon that since the log printing does not happen the
> > > > > frame dropping does neither.
> > > > > 
> > > > > table inet filter {
> > > > [...]
> > > > >           chain wan_pr {
> > > > >                   ct state { new } update @glv4 { ip saddr limit rate over
> > > > > 15/day burst 3 packets } log prefix "glv4 DROP: " flags all drop
> > > > >           }
> > > > > }
> > > > ct state new is restricting to match only on the first packet of the
> > > > connection.
> > > Right, it should be then rather something with ct count, like:
> > > 
> > > ct state new update @glv4 { ip saddr ct count over 15 } log flags all prefix
> > > "glv4 DROP: " drop
> > > 
> > > but that produces:
> > > 
> > > Error: Could not process rule: Not supported
> > What would you like to achieve in first place?
> > 
> > This thread is starting to diverge from the original question.
> 
> It is still about (rate) limiting offending saddrs with sets, probably in a
> staged approach, say log and drop saddrs like
> 
> * ct state update @glv4 { new , invalid } count over 50 timeout 1s
> * ct state update @glv4 { new , invalid } count over 75 timeout 1h
> * ct state update @glv4 { new , invalid } count over 150 timeout 1d

Sorry I don't understand what you're trying to express with this
syntax.

Re: [nftables] netdev rate limiting | timeouts rfq

[ѽ҉ᶬḳ℠]

On 28/09/2020 18:23, Pablo Neira Ayuso wrote:
> On Mon, Sep 28, 2020 at 04:03:00PM +0000, ѽ҉ᶬḳ℠ wrote:
>> On 28/09/2020 17:43, Pablo Neira Ayuso wrote:
>>> On Mon, Sep 28, 2020 at 01:10:00PM +0000, ѽ҉ᶬḳ℠ wrote:
>>>> On 28/09/2020 14:24, Pablo Neira Ayuso wrote:
>>>>> On Mon, Sep 28, 2020 at 12:08:00PM +0000, ѽ҉ᶬḳ℠ wrote:
>>>>>> In the below example it does not print any logs and since that stanza is
>>>>>> right prior drop I reckon that since the log printing does not happen the
>>>>>> frame dropping does neither.
>>>>>>
>>>>>> table inet filter {
>>>>> [...]
>>>>>>            chain wan_pr {
>>>>>>                    ct state { new } update @glv4 { ip saddr limit rate over
>>>>>> 15/day burst 3 packets } log prefix "glv4 DROP: " flags all drop
>>>>>>            }
>>>>>> }
>>>>> ct state new is restricting to match only on the first packet of the
>>>>> connection.
>>>> Right, it should be then rather something with ct count, like:
>>>>
>>>> ct state new update @glv4 { ip saddr ct count over 15 } log flags all prefix
>>>> "glv4 DROP: " drop
>>>>
>>>> but that produces:
>>>>
>>>> Error: Could not process rule: Not supported
>>> What would you like to achieve in first place?
>>>
>>> This thread is starting to diverge from the original question.
>> It is still about (rate) limiting offending saddrs with sets, probably in a
>> staged approach, say log and drop saddrs like
>>
>> * ct state update @glv4 { new , invalid } count over 50 timeout 1s
>> * ct state update @glv4 { new , invalid } count over 75 timeout 1h
>> * ct state update @glv4 { new , invalid } count over 150 timeout 1d
> Sorry I don't understand what you're trying to express with this
> syntax.

To get a flexible evaluation period for the count value:


* ct state { new , invalid } update @glv4 { ip saddr ct count over 50  
timeout 1s } log flags all prefix "glv4 DROP: " drop

update set element for any saddr that exceeds the count of 50 within 1 s 
for ct state new | invalid


* ct state { new , invalid } update @glv4 { ip saddr ct count over 75  
timeout 1s } log flags all prefix "glv4 DROP: " drop

update set element for any saddr that exceeds the count of 75 within 1 h 
for ct state new | invalid


* ct state { new , invalid } update @glv4 { ip saddr ct count over 75  
timeout 1s } log flags all prefix "glv4 DROP: " drop

update set element for any saddr that exceeds the count of 150 within 1 
d for ct state new | invalid

Re: [nftables] netdev rate limiting | timeouts rfq

[Pablo Neira Ayuso]

On Mon, Sep 28, 2020 at 04:47:00PM +0000, ѽ҉ᶬḳ℠ wrote:
> To get a flexible evaluation period for the count value:
> 
> * ct state { new , invalid } update @glv4 { ip saddr ct count over 50 
> timeout 1s } log flags all prefix "glv4 DROP: " drop
> 
> update set element for any saddr that exceeds the count of 50 within 1 s for
> ct state new | invalid
> 
> 
> * ct state { new , invalid } update @glv4 { ip saddr ct count over 75 
> timeout 1s } log flags all prefix "glv4 DROP: " drop
> 
> update set element for any saddr that exceeds the count of 75 within 1 h for
> ct state new | invalid
> 
> 
> * ct state { new , invalid } update @glv4 { ip saddr ct count over 75 
> timeout 1s } log flags all prefix "glv4 DROP: " drop
> 
> update set element for any saddr that exceeds the count of 150 within 1 d
> for ct state new | invalid

Thanks, these are looking better, although still not correct.

Two issues:

* 'ct count' relies on the connection tracking table. This is counting
  the number of existing connections in this table according to your
  key, ie. ip saddr. You do not have to specify timeouts here because
  it is the connection tracking time that governs when the conntrack
  entries expire.

* You have to use 'add' instead of 'update'. Update makes sense to
  refresh timeouts when they are in place, but there is no timeouts in
  this case.

Therefore, make sure you define the dynamic set with not timeouts at
all when combining this with ct count.

Using 'update' in your rule with ct count and/or 'timeout' in your set
definition will make you hit "Operation not supported".

Error reporting will get better sooner or later to provide more hints
on why this makes no sense, meanwhile, documentation will probably
help fill the gap.

[nftables] netdev rate limiting | timeouts rfq

[ѽ҉ᶬḳ℠]

On 28/09/2020 19:01, Pablo Neira Ayuso wrote:
> On Mon, Sep 28, 2020 at 04:47:00PM +0000, ѽ҉ᶬḳ℠ wrote:
>> To get a flexible evaluation period for the count value:
>>
>> * ct state { new , invalid } update @glv4 { ip saddr ct count over 50
>> timeout 1s } log flags all prefix "glv4 DROP: " drop
>>
>> update set element for any saddr that exceeds the count of 50 within 1 s for
>> ct state new | invalid
>>
>>
>> * ct state { new , invalid } update @glv4 { ip saddr ct count over 75
>> timeout 1s } log flags all prefix "glv4 DROP: " drop
>>
>> update set element for any saddr that exceeds the count of 75 within 1 h for
>> ct state new | invalid
>>
>>
>> * ct state { new , invalid } update @glv4 { ip saddr ct count over 75
>> timeout 1s } log flags all prefix "glv4 DROP: " drop
>>
>> update set element for any saddr that exceeds the count of 150 within 1 d
>> for ct state new | invalid
> Thanks, these are looking better, although still not correct.
>
> Two issues:
>
> * 'ct count' relies on the connection tracking table. This is counting
>    the number of existing connections in this table according to your
>    key, ie. ip saddr. You do not have to specify timeouts here because
>    it is the connection tracking time that governs when the conntrack
>    entries expire.

suppose those are stipulated in net.netfilter.nf_conntrack_[proto]_timeout ?

This is a bit inflexible though, an offender might not probe within such 
conntrack stipulated timeout but say in infrequent intervals, one would 
have to stipulate a count of 1 to catch such saddr at any given time 
(conntrack  timeout), which would be fine for ct state new but could 
lead to some false positive with ct state invalid.

Moreover, if not specifying a timeout for the set then how is the saddr 
element going to be removed ever? As far as I comprehend there are two 
parameters (whichever yields first) for elements being purged (save for 
manual purging)  from a set:

* size - if exceeded oldest elements being purged first

or

* timeout - if counted down to zero element being purged.

With no timeout in the set elements may live forever, which is not 
really wanted for a dynamic sort of set.


> * You have to use 'add' instead of 'update'. Update makes sense to
>    refresh timeouts when they are in place, but there is no timeouts in
>    this case.
>
> Therefore, make sure you define the dynamic set with not timeouts at
> all when combining this with ct count.
>
> Using 'update' in your rule with ct count and/or 'timeout' in your set
> definition will make you hit "Operation not supported".

Having the set then as:

   set glv4 {
     type ipv4_addr
     size 65535
     flags dynamic
     counter
   }

and the rule in pre-routing chain

ct state new add @glv4 { ip saddr ct count over 25 }

and still been hitting:

  Error: Could not process rule: Not supported

until removed 'counter' from the set, then reading

   set glv4 {
     type ipv4_addr
     size 65535
     flags dynamic
   }


> Error reporting will get better sooner or later to provide more hints
> on why this makes no sense, meanwhile, documentation will probably
> help fill the gap.

Re: [nftables] netdev rate limiting | timeouts rfq

[Pablo Neira Ayuso]

On Mon, Sep 28, 2020 at 05:38:00PM +0000, ѽ҉ᶬḳ℠ wrote:
> On 28/09/2020 19:01, Pablo Neira Ayuso wrote:
> > On Mon, Sep 28, 2020 at 04:47:00PM +0000, ѽ҉ᶬḳ℠ wrote:
> > > To get a flexible evaluation period for the count value:
> > > 
> > > * ct state { new , invalid } update @glv4 { ip saddr ct count over 50
> > > timeout 1s } log flags all prefix "glv4 DROP: " drop
> > > 
> > > update set element for any saddr that exceeds the count of 50 within 1 s for
> > > ct state new | invalid
> > > 
> > > 
> > > * ct state { new , invalid } update @glv4 { ip saddr ct count over 75
> > > timeout 1s } log flags all prefix "glv4 DROP: " drop
> > > 
> > > update set element for any saddr that exceeds the count of 75 within 1 h for
> > > ct state new | invalid
> > > 
> > > 
> > > * ct state { new , invalid } update @glv4 { ip saddr ct count over 75
> > > timeout 1s } log flags all prefix "glv4 DROP: " drop
> > > 
> > > update set element for any saddr that exceeds the count of 150 within 1 d
> > > for ct state new | invalid
> > Thanks, these are looking better, although still not correct.
> > 
> > Two issues:
> > 
> > * 'ct count' relies on the connection tracking table. This is counting
> >    the number of existing connections in this table according to your
> >    key, ie. ip saddr. You do not have to specify timeouts here because
> >    it is the connection tracking time that governs when the conntrack
> >    entries expire.
> 
> suppose those are stipulated in net.netfilter.nf_conntrack_[proto]_timeout ?
> 
> This is a bit inflexible though, an offender might not probe within such
> conntrack stipulated timeout but say in infrequent intervals, one would have
> to stipulate a count of 1 to catch such saddr at any given time (conntrack 
> timeout), which would be fine for ct state new but could lead to some false
> positive with ct state invalid.
> 
> Moreover, if not specifying a timeout for the set then how is the saddr
> element going to be removed ever? As far as I comprehend there are two
> parameters (whichever yields first) for elements being purged (save for
> manual purging)  from a set:
> 
> * size - if exceeded oldest elements being purged first
> 
> or
> 
> * timeout - if counted down to zero element being purged.
> 
> With no timeout in the set elements may live forever, which is not really
> wanted for a dynamic sort of set.

That's not how it works: ct count implements a garbage collector to
purge old list of connections whose key is ip saddr.

http://workshop.netfilter.org/2018/wiki/images/2/23/Connlimit.pdf

The connection tracking table entries are released once they expire.
Connection in TIME_WAIT and CLOSE state are already assumed to be
close, hence not being counted.

> > * You have to use 'add' instead of 'update'. Update makes sense to
> >    refresh timeouts when they are in place, but there is no timeouts in
> >    this case.
> > 
> > Therefore, make sure you define the dynamic set with not timeouts at
> > all when combining this with ct count.
> > 
> > Using 'update' in your rule with ct count and/or 'timeout' in your set
> > definition will make you hit "Operation not supported".
> 
> Having the set then as:
> 
>   set glv4 {
>     type ipv4_addr
>     size 65535
>     flags dynamic
>     counter
>   }
> 
> and the rule in pre-routing chain
> 
> ct state new add @glv4 { ip saddr ct count over 25 }
> 
> and still been hitting:
> 
>  Error: Could not process rule: Not supported
> 
> until removed 'counter' from the set, then reading
> 
>   set glv4 {
>     type ipv4_addr
>     size 65535
>     flags dynamic
>   }

The counter is a stateful expression and ct count is also a stateful
expression too.

You can only use one single stateful expression per set set element at
this stage.

Re: [nftables] netdev rate limiting | timeouts rfq

[ѽ҉ᶬḳ℠]

On 28/09/2020 19:56, Pablo Neira Ayuso wrote:
> On Mon, Sep 28, 2020 at 05:38:00PM +0000, ѽ҉ᶬḳ℠ wrote:
>> On 28/09/2020 19:01, Pablo Neira Ayuso wrote:
>>> On Mon, Sep 28, 2020 at 04:47:00PM +0000, ѽ҉ᶬḳ℠ wrote:
>>>> To get a flexible evaluation period for the count value:
>>>>
>>>> * ct state { new , invalid } update @glv4 { ip saddr ct count over 50
>>>> timeout 1s } log flags all prefix "glv4 DROP: " drop
>>>>
>>>> update set element for any saddr that exceeds the count of 50 within 1 s for
>>>> ct state new | invalid
>>>>
>>>>
>>>> * ct state { new , invalid } update @glv4 { ip saddr ct count over 75
>>>> timeout 1s } log flags all prefix "glv4 DROP: " drop
>>>>
>>>> update set element for any saddr that exceeds the count of 75 within 1 h for
>>>> ct state new | invalid
>>>>
>>>>
>>>> * ct state { new , invalid } update @glv4 { ip saddr ct count over 75
>>>> timeout 1s } log flags all prefix "glv4 DROP: " drop
>>>>
>>>> update set element for any saddr that exceeds the count of 150 within 1 d
>>>> for ct state new | invalid
>>> Thanks, these are looking better, although still not correct.
>>>
>>> Two issues:
>>>
>>> * 'ct count' relies on the connection tracking table. This is counting
>>>     the number of existing connections in this table according to your
>>>     key, ie. ip saddr. You do not have to specify timeouts here because
>>>     it is the connection tracking time that governs when the conntrack
>>>     entries expire.
>> suppose those are stipulated in net.netfilter.nf_conntrack_[proto]_timeout ?
>>
>> This is a bit inflexible though, an offender might not probe within such
>> conntrack stipulated timeout but say in infrequent intervals, one would have
>> to stipulate a count of 1 to catch such saddr at any given time (conntrack
>> timeout), which would be fine for ct state new but could lead to some false
>> positive with ct state invalid.
>>
>> Moreover, if not specifying a timeout for the set then how is the saddr
>> element going to be removed ever? As far as I comprehend there are two
>> parameters (whichever yields first) for elements being purged (save for
>> manual purging)  from a set:
>>
>> * size - if exceeded oldest elements being purged first
>>
>> or
>>
>> * timeout - if counted down to zero element being purged.
>>
>> With no timeout in the set elements may live forever, which is not really
>> wanted for a dynamic sort of set.
> That's not how it works: ct count implements a garbage collector to
> purge old list of connections whose key is ip saddr.
>
> http://workshop.netfilter.org/2018/wiki/images/2/23/Connlimit.pdf
>
> The connection tracking table entries are released once they expire.
> Connection in TIME_WAIT and CLOSE state are already assumed to be
> close, hence not being counted.

How is that in less abstract (sort of layman) terms?

nft_connlimit stores conntrack objects from the conntrack table and once 
those objects expire in the conntrack table said garbage collector is 
retiring elements from the set?

With 'gc-interval'  one could then exercise some sort of control over 
the element's lifespan in the ct state set?


>
>>> * You have to use 'add' instead of 'update'. Update makes sense to
>>>     refresh timeouts when they are in place, but there is no timeouts in
>>>     this case.
>>>
>>> Therefore, make sure you define the dynamic set with not timeouts at
>>> all when combining this with ct count.
>>>
>>> Using 'update' in your rule with ct count and/or 'timeout' in your set
>>> definition will make you hit "Operation not supported".
>> Having the set then as:
>>
>>    set glv4 {
>>      type ipv4_addr
>>      size 65535
>>      flags dynamic
>>      counter
>>    }
>>
>> and the rule in pre-routing chain
>>
>> ct state new add @glv4 { ip saddr ct count over 25 }
>>
>> and still been hitting:
>>
>>   Error: Could not process rule: Not supported
>>
>> until removed 'counter' from the set, then reading
>>
>>    set glv4 {
>>      type ipv4_addr
>>      size 65535
>>      flags dynamic
>>    }
> The counter is a stateful expression and ct count is also a stateful
> expression too.
>
> You can only use one single stateful expression per set set element at
> this stage.

That is curious/confusing (me) because adding 'counter' to the rule does 
not throw an error:

ct state new add @glv4 { ip saddr ct count over 25 } counter

Re: [nftables] netdev rate limiting | timeouts rfq

[ѽ҉ᶬḳ℠]

On 28/09/2020 20:15, ѽ҉ᶬḳ℠ wrote:
>
> On 28/09/2020 19:56, Pablo Neira Ayuso wrote:
>> On Mon, Sep 28, 2020 at 05:38:00PM +0000, ѽ҉ᶬḳ℠ wrote:
>>> On 28/09/2020 19:01, Pablo Neira Ayuso wrote:
>>>> On Mon, Sep 28, 2020 at 04:47:00PM +0000, ѽ҉ᶬḳ℠ wrote:
>>>>> To get a flexible evaluation period for the count value:
>>>>>
>>>>> * ct state { new , invalid } update @glv4 { ip saddr ct count over 50
>>>>> timeout 1s } log flags all prefix "glv4 DROP: " drop
>>>>>
>>>>> update set element for any saddr that exceeds the count of 50 
>>>>> within 1 s for
>>>>> ct state new | invalid
>>>>>
>>>>>
>>>>> * ct state { new , invalid } update @glv4 { ip saddr ct count over 75
>>>>> timeout 1s } log flags all prefix "glv4 DROP: " drop
>>>>>
>>>>> update set element for any saddr that exceeds the count of 75 
>>>>> within 1 h for
>>>>> ct state new | invalid
>>>>>
>>>>>
>>>>> * ct state { new , invalid } update @glv4 { ip saddr ct count over 75
>>>>> timeout 1s } log flags all prefix "glv4 DROP: " drop
>>>>>
>>>>> update set element for any saddr that exceeds the count of 150 
>>>>> within 1 d
>>>>> for ct state new | invalid
>>>> Thanks, these are looking better, although still not correct.
>>>>
>>>> Two issues:
>>>>
>>>> * 'ct count' relies on the connection tracking table. This is counting
>>>>     the number of existing connections in this table according to your
>>>>     key, ie. ip saddr. You do not have to specify timeouts here 
>>>> because
>>>>     it is the connection tracking time that governs when the conntrack
>>>>     entries expire.
>>> suppose those are stipulated in 
>>> net.netfilter.nf_conntrack_[proto]_timeout ?
>>>
>>> This is a bit inflexible though, an offender might not probe within 
>>> such
>>> conntrack stipulated timeout but say in infrequent intervals, one 
>>> would have
>>> to stipulate a count of 1 to catch such saddr at any given time 
>>> (conntrack
>>> timeout), which would be fine for ct state new but could lead to 
>>> some false
>>> positive with ct state invalid.
>>>
>>> Moreover, if not specifying a timeout for the set then how is the saddr
>>> element going to be removed ever? As far as I comprehend there are two
>>> parameters (whichever yields first) for elements being purged (save for
>>> manual purging)  from a set:
>>>
>>> * size - if exceeded oldest elements being purged first
>>>
>>> or
>>>
>>> * timeout - if counted down to zero element being purged.
>>>
>>> With no timeout in the set elements may live forever, which is not 
>>> really
>>> wanted for a dynamic sort of set.
>> That's not how it works: ct count implements a garbage collector to
>> purge old list of connections whose key is ip saddr.
>>
>> http://workshop.netfilter.org/2018/wiki/images/2/23/Connlimit.pdf
>>
>> The connection tracking table entries are released once they expire.
>> Connection in TIME_WAIT and CLOSE state are already assumed to be
>> close, hence not being counted.
>
> How is that in less abstract (sort of layman) terms?
>
> nft_connlimit stores conntrack objects from the conntrack table and 
> once those objects expire in the conntrack table said garbage 
> collector is retiring elements from the set?
>
> With 'gc-interval'  one could then exercise some sort of control over 
> the element's lifespan in the ct state set?
>

Just figured gc-interval does not work for ct state based sets. With no 
timeout control and TIME_WAIT and CLOSE state not counting there is no 
use for ct state based dynamic sets to gather meaningful data about 
offenders, even saddr ct count over 0 there is hardly any element in the 
set and if there is it is just too short-lived.

>
>>
>>>> * You have to use 'add' instead of 'update'. Update makes sense to
>>>>     refresh timeouts when they are in place, but there is no 
>>>> timeouts in
>>>>     this case.
>>>>
>>>> Therefore, make sure you define the dynamic set with not timeouts at
>>>> all when combining this with ct count.
>>>>
>>>> Using 'update' in your rule with ct count and/or 'timeout' in your set
>>>> definition will make you hit "Operation not supported".
>>> Having the set then as:
>>>
>>>    set glv4 {
>>>      type ipv4_addr
>>>      size 65535
>>>      flags dynamic
>>>      counter
>>>    }
>>>
>>> and the rule in pre-routing chain
>>>
>>> ct state new add @glv4 { ip saddr ct count over 25 }
>>>
>>> and still been hitting:
>>>
>>>   Error: Could not process rule: Not supported
>>>
>>> until removed 'counter' from the set, then reading
>>>
>>>    set glv4 {
>>>      type ipv4_addr
>>>      size 65535
>>>      flags dynamic
>>>    }
>> The counter is a stateful expression and ct count is also a stateful
>> expression too.
>>
>> You can only use one single stateful expression per set set element at
>> this stage.
>
> That is curious/confusing (me) because adding 'counter' to the rule 
> does not throw an error:
>
> ct state new add @glv4 { ip saddr ct count over 25 } counter
>
>
>
>
>