PF_PACKET RPS like using bpf

PF_PACKET RPS like using bpf

[Alexandre Cassen]

Hello,

While implementing a large scale L2 processing soft, there is a need for 
high perf ingress handling.

In short: using BPF socket_filter prog with a hash func over ethernet 
address to distribute load across multiple PF_PACKET sockets: mainly 
filtering out non related hkey packets. It simplify userland app by 
using multiple PF_PACKET sockets in dedicated pthread.

Longer discussion + quick source code illustration here:

https://github.com/acassen/bpf-pfpacket-rps


My question would be related to sockmap. I tried to figure out how to 
perform the same policing design using sockmap. And cant find my way 
around it :/ If bpf verdict prog can get socket fd from __sk_buff then 
it could drives the same hash/policing decision.

Is it possible to get socket fd from __sk_buff from sockmap verdict prog ?

regs,
Alexandre

Re: PF_PACKET RPS like using bpf

[Stanislav Fomichev]

On 03/14, Alexandre Cassen wrote:
> Hello,
> 
> While implementing a large scale L2 processing soft, there is a need for
> high perf ingress handling.
> 
> In short: using BPF socket_filter prog with a hash func over ethernet
> address to distribute load across multiple PF_PACKET sockets: mainly
> filtering out non related hkey packets. It simplify userland app by using
> multiple PF_PACKET sockets in dedicated pthread.
> 
> Longer discussion + quick source code illustration here:
> 
> https://github.com/acassen/bpf-pfpacket-rps
> 
> 
> My question would be related to sockmap. I tried to figure out how to
> perform the same policing design using sockmap. And cant find my way around
> it :/ If bpf verdict prog can get socket fd from __sk_buff then it could
> drives the same hash/policing decision.
> 
> Is it possible to get socket fd from __sk_buff from sockmap verdict prog ?

Don't have an answer about sockmap, but I'd suggest you look at AF_XDP
if you want the best perf out there with raw frames.
IIRC, it just recently got a mode where you can redirect into multiple
sockets (== consumer threads):

https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/commit/?id=2863d665ea41

Re: PF_PACKET RPS like using bpf

[Alexandre Cassen]

On 15/03/2024 18:31, Stanislav Fomichev wrote:
> On 03/14, Alexandre Cassen wrote:
>> Hello,
>>
>> While implementing a large scale L2 processing soft, there is a need for
>> high perf ingress handling.
>>
>> In short: using BPF socket_filter prog with a hash func over ethernet
>> address to distribute load across multiple PF_PACKET sockets: mainly
>> filtering out non related hkey packets. It simplify userland app by using
>> multiple PF_PACKET sockets in dedicated pthread.
>>
>> Longer discussion + quick source code illustration here:
>>
>> https://github.com/acassen/bpf-pfpacket-rps
>>
>>
>> My question would be related to sockmap. I tried to figure out how to
>> perform the same policing design using sockmap. And cant find my way around
>> it :/ If bpf verdict prog can get socket fd from __sk_buff then it could
>> drives the same hash/policing decision.
>>
>> Is it possible to get socket fd from __sk_buff from sockmap verdict prog ?
> 
> Don't have an answer about sockmap, but I'd suggest you look at AF_XDP
> if you want the best perf out there with raw frames.
> IIRC, it just recently got a mode where you can redirect into multiple
> sockets (== consumer threads):
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/commit/?id=2863d665ea41

thanks for replying, __sk_buff bpf mirror doesnt export struct sock as 
present into kernel skbuff, so no options but extending.
AF_XDP is an option. Will extend test code with it later on.

Re: PF_PACKET RPS like using bpf

[Stanislav Fomichev]

On 03/15, Alexandre Cassen wrote:
> 
> 
> On 15/03/2024 18:31, Stanislav Fomichev wrote:
> > On 03/14, Alexandre Cassen wrote:
> > > Hello,
> > > 
> > > While implementing a large scale L2 processing soft, there is a need for
> > > high perf ingress handling.
> > > 
> > > In short: using BPF socket_filter prog with a hash func over ethernet
> > > address to distribute load across multiple PF_PACKET sockets: mainly
> > > filtering out non related hkey packets. It simplify userland app by using
> > > multiple PF_PACKET sockets in dedicated pthread.
> > > 
> > > Longer discussion + quick source code illustration here:
> > > 
> > > https://github.com/acassen/bpf-pfpacket-rps
> > > 
> > > 
> > > My question would be related to sockmap. I tried to figure out how to
> > > perform the same policing design using sockmap. And cant find my way around
> > > it :/ If bpf verdict prog can get socket fd from __sk_buff then it could
> > > drives the same hash/policing decision.
> > > 
> > > Is it possible to get socket fd from __sk_buff from sockmap verdict prog ?
> > 
> > Don't have an answer about sockmap, but I'd suggest you look at AF_XDP
> > if you want the best perf out there with raw frames.
> > IIRC, it just recently got a mode where you can redirect into multiple
> > sockets (== consumer threads):
> > 
> > https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/commit/?id=2863d665ea41
> 
> thanks for replying, __sk_buff bpf mirror doesnt export struct sock as
> present into kernel skbuff, so no options but extending.
> AF_XDP is an option. Will extend test code with it later on.

We do export bpf_sock in __sk_buff:

	__bpf_md_ptr(struct bpf_sock *, sk);

But it has a very minimal set of 'struct sock' fields exported.

Re: PF_PACKET RPS like using bpf

[Yonghong Song]

On 3/15/24 11:30 AM, Stanislav Fomichev wrote:
> On 03/15, Alexandre Cassen wrote:
>>
>> On 15/03/2024 18:31, Stanislav Fomichev wrote:
>>> On 03/14, Alexandre Cassen wrote:
>>>> Hello,
>>>>
>>>> While implementing a large scale L2 processing soft, there is a need for
>>>> high perf ingress handling.
>>>>
>>>> In short: using BPF socket_filter prog with a hash func over ethernet
>>>> address to distribute load across multiple PF_PACKET sockets: mainly
>>>> filtering out non related hkey packets. It simplify userland app by using
>>>> multiple PF_PACKET sockets in dedicated pthread.
>>>>
>>>> Longer discussion + quick source code illustration here:
>>>>
>>>> https://github.com/acassen/bpf-pfpacket-rps
>>>>
>>>>
>>>> My question would be related to sockmap. I tried to figure out how to
>>>> perform the same policing design using sockmap. And cant find my way around
>>>> it :/ If bpf verdict prog can get socket fd from __sk_buff then it could
>>>> drives the same hash/policing decision.
>>>>
>>>> Is it possible to get socket fd from __sk_buff from sockmap verdict prog ?
>>> Don't have an answer about sockmap, but I'd suggest you look at AF_XDP
>>> if you want the best perf out there with raw frames.
>>> IIRC, it just recently got a mode where you can redirect into multiple
>>> sockets (== consumer threads):
>>>
>>> https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/commit/?id=2863d665ea41
>> thanks for replying, __sk_buff bpf mirror doesnt export struct sock as
>> present into kernel skbuff, so no options but extending.
>> AF_XDP is an option. Will extend test code with it later on.
> We do export bpf_sock in __sk_buff:
>
> 	__bpf_md_ptr(struct bpf_sock *, sk);
>
> But it has a very minimal set of 'struct sock' fields exported.

You could use bpf_core_cast() in bpf_core_read.h to cast 'struct bpf_sock' to kernel 'struct sock'.

>

Re: PF_PACKET RPS like using bpf

[Alexandre Cassen]

On 18/03/2024 17:40, Yonghong Song wrote:
> 
> On 3/15/24 11:30 AM, Stanislav Fomichev wrote:
>> On 03/15, Alexandre Cassen wrote:
>>>
>>> On 15/03/2024 18:31, Stanislav Fomichev wrote:
>>>> On 03/14, Alexandre Cassen wrote:
>>>>> Hello,
>>>>>
>>>>> While implementing a large scale L2 processing soft, there is a 
>>>>> need for
>>>>> high perf ingress handling.
>>>>>
>>>>> In short: using BPF socket_filter prog with a hash func over ethernet
>>>>> address to distribute load across multiple PF_PACKET sockets: mainly
>>>>> filtering out non related hkey packets. It simplify userland app by 
>>>>> using
>>>>> multiple PF_PACKET sockets in dedicated pthread.
>>>>>
>>>>> Longer discussion + quick source code illustration here:
>>>>>
>>>>> https://github.com/acassen/bpf-pfpacket-rps
>>>>>
>>>>>
>>>>> My question would be related to sockmap. I tried to figure out how to
>>>>> perform the same policing design using sockmap. And cant find my 
>>>>> way around
>>>>> it :/ If bpf verdict prog can get socket fd from __sk_buff then it 
>>>>> could
>>>>> drives the same hash/policing decision.
>>>>>
>>>>> Is it possible to get socket fd from __sk_buff from sockmap verdict 
>>>>> prog ?
>>>> Don't have an answer about sockmap, but I'd suggest you look at AF_XDP
>>>> if you want the best perf out there with raw frames.
>>>> IIRC, it just recently got a mode where you can redirect into multiple
>>>> sockets (== consumer threads):
>>>>
>>>> https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/commit/?id=2863d665ea41
>>> thanks for replying, __sk_buff bpf mirror doesnt export struct sock as
>>> present into kernel skbuff, so no options but extending.
>>> AF_XDP is an option. Will extend test code with it later on.
>> We do export bpf_sock in __sk_buff:
>>
>>     __bpf_md_ptr(struct bpf_sock *, sk);
>>
>> But it has a very minimal set of 'struct sock' fields exported.
> 
> You could use bpf_core_cast() in bpf_core_read.h to cast 'struct 
> bpf_sock' to kernel 'struct sock'.


good idea, other option could be to userland set SO_MARK one time and 
get it back from bpf prog via bpf_sock.mark to drive policing decision