Counters for individual elements in maps and sets?

Counters for individual elements in maps and sets?

[Tomas Mudrunka]

Hello,
i've figured it's possible to simplify my rules by using maps and sets 
instead of using individual rules, but i need to account traffic for 
each address in the map separately. Maybe this can be implemented using 
flags in map/set, so i will be able to enable it like this:

	map prometheus {
		type ipv4_addr : classid;
		flags interval, counter;
		elements = {
			1.1.1.2 : 2:2222 counter packets 10 bytes 5120,
			1.1.1.3 : 3:3333 counter packets 3 bytes 489
		}
	}

just to make it clear, the same map without counters looks like this 
right now:

	map prometheus {
		type ipv4_addr : classid
		flags interval
		elements = {
			1.1.1.2 : 2:2222,
			1.1.1.3 : 3:3333
		}
	}


Will it be ever possible to have per element counters for maps and sets?

Thanks

-- 
S pozdravem
Best regards
      Tomáš Mudruňka - SPOJE.NET s.r.o.

Re: Counters for individual elements in maps and sets?

[Arturo Borrero Gonzalez]

On 7 November 2017 at 14:09, Tomas Mudrunka <mudrunka@spoje.net> wrote:
> Hello,
> i've figured it's possible to simplify my rules by using maps and sets
> instead of using individual rules, but i need to account traffic for each
> address in the map separately. Maybe this can be implemented using flags in
> map/set, so i will be able to enable it like this:
>
>         map prometheus {
>                 type ipv4_addr : classid;
>                 flags interval, counter;
>                 elements = {
>                         1.1.1.2 : 2:2222 counter packets 10 bytes 5120,
>                         1.1.1.3 : 3:3333 counter packets 3 bytes 489
>                 }
>         }
>
> just to make it clear, the same map without counters looks like this right
> now:
>
>         map prometheus {
>                 type ipv4_addr : classid
>                 flags interval
>                 elements = {
>                         1.1.1.2 : 2:2222,
>                         1.1.1.3 : 3:3333
>                 }
>         }
>
>
> Will it be ever possible to have per element counters for maps and sets?
>

you are probably looking for something like this:
https://wiki.nftables.org/wiki-nftables/index.php/Flow_tables

Re: Counters for individual elements in maps and sets?

[Pablo Neira Ayuso]

On Tue, Nov 07, 2017 at 07:44:13PM +0100, Arturo Borrero Gonzalez wrote:
> On 7 November 2017 at 14:09, Tomas Mudrunka <mudrunka@spoje.net> wrote:
> > Hello,
> > i've figured it's possible to simplify my rules by using maps and sets
> > instead of using individual rules, but i need to account traffic for each
> > address in the map separately. Maybe this can be implemented using flags in
> > map/set, so i will be able to enable it like this:
> >
> >         map prometheus {
> >                 type ipv4_addr : classid;
> >                 flags interval, counter;
> >                 elements = {
> >                         1.1.1.2 : 2:2222 counter packets 10 bytes 5120,
> >                         1.1.1.3 : 3:3333 counter packets 3 bytes 489
> >                 }
> >         }
> >
> > just to make it clear, the same map without counters looks like this right
> > now:
> >
> >         map prometheus {
> >                 type ipv4_addr : classid
> >                 flags interval
> >                 elements = {
> >                         1.1.1.2 : 2:2222,
> >                         1.1.1.3 : 3:3333
> >                 }
> >         }
> >
> >
> > Will it be ever possible to have per element counters for maps and sets?
> >
> 
> you are probably looking for something like this:
> https://wiki.nftables.org/wiki-nftables/index.php/Flow_tables

This looks like a different usecase we don't support yet, that doesn't
fit into flow tables.

There's a ticket in bugzilla asking for something like this, we plan
to add support for this indeed.

Re: Counters for individual elements in maps and sets?

[Tomas Mudrunka]

Dne 2017-11-10 08:07, Pablo Neira Ayuso napsal:
> On Tue, Nov 07, 2017 at 07:44:13PM +0100, Arturo Borrero Gonzalez 
> wrote:
>> On 7 November 2017 at 14:09, Tomas Mudrunka <mudrunka@spoje.net> 
>> wrote:
>> > Hello,
>> > i've figured it's possible to simplify my rules by using maps and sets
>> > instead of using individual rules, but i need to account traffic for each
>> > address in the map separately. Maybe this can be implemented using flags in
>> > map/set, so i will be able to enable it like this:
>> >
>> >         map prometheus {
>> >                 type ipv4_addr : classid;
>> >                 flags interval, counter;
>> >                 elements = {
>> >                         1.1.1.2 : 2:2222 counter packets 10 bytes 5120,
>> >                         1.1.1.3 : 3:3333 counter packets 3 bytes 489
>> >                 }
>> >         }
>> >
>> > just to make it clear, the same map without counters looks like this right
>> > now:
>> >
>> >         map prometheus {
>> >                 type ipv4_addr : classid
>> >                 flags interval
>> >                 elements = {
>> >                         1.1.1.2 : 2:2222,
>> >                         1.1.1.3 : 3:3333
>> >                 }
>> >         }
>> >
>> >
>> > Will it be ever possible to have per element counters for maps and sets?
>> >
>> 
>> you are probably looking for something like this:
>> https://wiki.nftables.org/wiki-nftables/index.php/Flow_tables
> 
> This looks like a different usecase we don't support yet, that doesn't
> fit into flow tables.
> 
> There's a ticket in bugzilla asking for something like this, we plan
> to add support for this indeed.


I guess that this can be done using flow tables, HOWEVER in such case 
each packet has to be matched two times. Once for classification and 
once for traffic accounting.

Similar problem is with sets and maps. AFAIK you can't really use one 
map for multiple rules.
Eg.: I wish to have single map that would contain ip adress key, then ip 
adress for NAT, class for TC and counter.

So i can use that map as if it was set. So i can have rule that will 
just ACCEPT anything that is in the map, another rule that will do NAT 
to different IP stored in the same map and set classid of that packet 
according to same map. And finaly update counters on that element in 
map.

This would require that maps can have multiple columns (sorta like SQL 
DB) and possibly multiple keys.
Right now, you can have map that matches multiple "keys" eg. using "ip 
saddr . tcp dport", but what if i want to add another rule, that would 
use the same map to match just by "ip saddr". this is not possible and i 
have to make another map for that.

Having this option to define both multiple keys and multiple values in 
single element of map might simplify and even speed up packet processing 
when used well. (let's say that i want to define map that would map NAT 
from ip:port to another ip:port. right now i have to match two maps. one 
for ip:port to ip and one for same ip:port to port).

-- 
S pozdravem
Best regards
      Tomáš Mudruňka - SPOJE.NET s.r.o.

Re: Counters for individual elements in maps and sets?

[Raymond Burkholder]

On 11/10/2017 09:09 AM, Tomas Mudrunka wrote:
> Dne 2017-11-10 08:07, Pablo Neira Ayuso napsal:
>> On Tue, Nov 07, 2017 at 07:44:13PM +0100, Arturo Borrero Gonzalez wrote:
>>> On 7 November 2017 at 14:09, Tomas Mudrunka <mudrunka@spoje.net> wrote:
>>> > Hello,
>>> > i've figured it's possible to simplify my rules by using maps and sets
>>> > instead of using individual rules, but i need to account traffic 
>>> for each
>>> > address in the map separately. Maybe this can be implemented using 
>>> flags in
>>> > map/set, so i will be able to enable it like this:
>>> >
>>>
>>> you are probably looking for something like this:
>>> https://wiki.nftables.org/wiki-nftables/index.php/Flow_tables
>>
>> This looks like a different usecase we don't support yet, that doesn't
>> fit into flow tables.
>>
> I guess that this can be done using flow tables, HOWEVER in such case 
> each packet has to be matched two times. Once for classification and 
> once for traffic accounting.
> 

Doing a free association here, in case it is of interest, use connection 
tracking in Open vSwitch.  It deals with flows, makes use of connection 
tracking, and maintains flow statistics.

It is an interesting technology combination, and I am working to convert 
infrastructure for which I am responsible to this style of security: 
security rules are built into the routing and switching mechanism at 
endpoints as well as midpoints.

There are many other related concepts which spring to mind for those 
interested in this mix.

--
Raymond Burkholder
https://blog.raymond.burkholder.net


-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.