XDP for Kernel by-pass?

XDP for Kernel by-pass?

[tom.barbette]

Hi all,

Can XDP be used for "skbuff bypass"? Like a rx_handler but closer to the driver? I'm not very familiar with eBPF, could it be used to pass the kernel to any kernel module or userspace application?

It looks to me that bpf_func in struct bpf_prog could be any function that we could use to deviate XDP and use it for that purpose. Correct?

Kernel app like Click or OVS could benefit from that as they don't use most skbuff facilities.

Thanks,

Tom

Re: XDP for Kernel by-pass?

[David Miller]

From: tom.barbette@ulg.ac.be
Date: Thu, 13 Jul 2017 21:34:17 +0200 (CEST)

> Can XDP be used for "skbuff bypass"? Like a rx_handler but closer to
> the driver? I'm not very familiar with eBPF, could it be used to
> pass the kernel to any kernel module or userspace application?

Absolutely not.

> It looks to me that bpf_func in struct bpf_prog could be any
> function that we could use to deviate XDP and use it for that
> purpose. Correct?

The list of functions and data that a bpf program can access is
extremely strictly controlled.

Only specific "BPF helper" functions can be invoked, and only specific
pieces of data can be accessed in very strictly controlled ways.

It is specifically designed to not be a bypass facility, but rather
a mechanism by which one can implement arbitrary policy and software
programmable networking.

Things like the OVS switching plane, or a bridge, or whatever
forwarding facility you can imagine, can be fully implemented with
eBPF and the latest work being done by John Fastabend.

It is not necessary to access "arbitrary kernel modules and data"
in order to facilitate this.

Re: XDP for Kernel by-pass?

[Daniel Borkmann]

On 07/13/2017 09:34 PM, tom.barbette@ulg.ac.be wrote:
> Hi all,
>
> Can XDP be used for "skbuff bypass"? Like a rx_handler but closer to the driver? I'm not very familiar with eBPF, could it be used to pass the kernel to any kernel module or userspace application?
>
> It looks to me that bpf_func in struct bpf_prog could be any function that we could use to deviate XDP and use it for that purpose. Correct?
>
> Kernel app like Click or OVS could benefit from that as they don't use most skbuff facilities.

You can do a lot of things with eBPF already e.g. take a look at the kernel
BPF sample code or BPF selftests e.g. test_l4lb.c, test_xdp.c, etc, or
projects like cilium that get compiled from C to BPF bytecode. BPF and XDP
is heavily tied together and part of the core kernel, so modular code/hooks
like third party kernel modules are not possible. Also f.e. wrt to user space
one of the use cases is that you don't need to cross boundaries in the first
place when you preprocess in XDP and e.g. apply policy to drop unrelated
packets early while remaining ones are forwarded to containers, etc. There's
wip to have an OVS data path in BPF/XDP. I currently don't find the slides,
but Quentin did a good coverage on it from the last workshop [1], so worth
to read up a bit on the topic.

   [1] https://qmonnet.github.io/whirl-offload/2017/03/03/iovisor-summit-summary/

> Thanks,
>
> Tom
>

Re: XDP for Kernel by-pass?

[Joe Stringer]

On 13 July 2017 at 13:44, Daniel Borkmann <daniel@iogearbox.net> wrote:
> On 07/13/2017 09:34 PM, tom.barbette@ulg.ac.be wrote:
>>
>> Hi all,
>>
>> Can XDP be used for "skbuff bypass"? Like a rx_handler but closer to the
>> driver? I'm not very familiar with eBPF, could it be used to pass the kernel
>> to any kernel module or userspace application?
>>
>> It looks to me that bpf_func in struct bpf_prog could be any function that
>> we could use to deviate XDP and use it for that purpose. Correct?
>>
>> Kernel app like Click or OVS could benefit from that as they don't use
>> most skbuff facilities.
>
>
> You can do a lot of things with eBPF already e.g. take a look at the kernel
> BPF sample code or BPF selftests e.g. test_l4lb.c, test_xdp.c, etc, or
> projects like cilium that get compiled from C to BPF bytecode. BPF and XDP
> is heavily tied together and part of the core kernel, so modular code/hooks
> like third party kernel modules are not possible. Also f.e. wrt to user
> space
> one of the use cases is that you don't need to cross boundaries in the first
> place when you preprocess in XDP and e.g. apply policy to drop unrelated
> packets early while remaining ones are forwarded to containers, etc. There's
> wip to have an OVS data path in BPF/XDP. I currently don't find the slides,
> but Quentin did a good coverage on it from the last workshop [1], so worth
> to read up a bit on the topic.
>
>   [1]
> https://qmonnet.github.io/whirl-offload/2017/03/03/iovisor-summit-summary/

Right, from OVS perspective this is a pretty good summary of the
current state: it's still an uphill battle with regards to verifier
complexity, and there's reasonably large unsolved problems such as
connection tracking; megaflows isn't there; and we haven't got to a
point where we can observe performance parity with existing in-kernel
solutions yet. That's just things we're aware of right now. In some
perfect would it would be simple as dragging and dropping existing
kernel code into the eBPF environment, but I'm sure that one of the
first things you'll learn on this xdp-newbies list is that everything
you want to put in eBPF/XDP needs to be rewritten from scratch with
the verifier in mind.

Re: XDP for Kernel by-pass?

[tom.barbette]

Thanks all of you for your very clear answers.

Then I would like to ask why so close to removing the need for DPDK, Netmap and friends it has been avoided on purpose? I know this question has been a little touchy in the past. But I'm really wondering about the motivation behind "BPF-only". 

If the goal is to want programs to be executed in a safe environment, do you believe that full IPS programs like Snort (which is also a use case for kernel by-pass as it implements its own stack) would even be one day implementable in a eeeBPF program?

Netmap hooks exactly where XDP does. We would just need a little shared-memory module that hooks in XDP and give all the raw packets to a userspace program. And then bye bye DPDK, Netmap, PF_RING and the others and we forget about those crazy years where everybody had his own framework to bypass the kernel.

Tom

----- Mail original -----
> De: "Joe Stringer" <joe@ovn.org>
> À: "Daniel Borkmann" <daniel@iogearbox.net>
> Cc: "tom barbette" <tom.barbette@ulg.ac.be>, xdp-newbies@vger.kernel.org
> Envoyé: Jeudi 13 Juillet 2017 23:07:07
> Objet: Re: XDP for Kernel by-pass?

> On 13 July 2017 at 13:44, Daniel Borkmann <daniel@iogearbox.net> wrote:
>> On 07/13/2017 09:34 PM, tom.barbette@ulg.ac.be wrote:
>>>
>>> Hi all,
>>>
>>> Can XDP be used for "skbuff bypass"? Like a rx_handler but closer to the
>>> driver? I'm not very familiar with eBPF, could it be used to pass the kernel
>>> to any kernel module or userspace application?
>>>
>>> It looks to me that bpf_func in struct bpf_prog could be any function that
>>> we could use to deviate XDP and use it for that purpose. Correct?
>>>
>>> Kernel app like Click or OVS could benefit from that as they don't use
>>> most skbuff facilities.
>>
>>
>> You can do a lot of things with eBPF already e.g. take a look at the kernel
>> BPF sample code or BPF selftests e.g. test_l4lb.c, test_xdp.c, etc, or
>> projects like cilium that get compiled from C to BPF bytecode. BPF and XDP
>> is heavily tied together and part of the core kernel, so modular code/hooks
>> like third party kernel modules are not possible. Also f.e. wrt to user
>> space
>> one of the use cases is that you don't need to cross boundaries in the first
>> place when you preprocess in XDP and e.g. apply policy to drop unrelated
>> packets early while remaining ones are forwarded to containers, etc. There's
>> wip to have an OVS data path in BPF/XDP. I currently don't find the slides,
>> but Quentin did a good coverage on it from the last workshop [1], so worth
>> to read up a bit on the topic.
>>
>>   [1]
>> https://qmonnet.github.io/whirl-offload/2017/03/03/iovisor-summit-summary/
> 
> Right, from OVS perspective this is a pretty good summary of the
> current state: it's still an uphill battle with regards to verifier
> complexity, and there's reasonably large unsolved problems such as
> connection tracking; megaflows isn't there; and we haven't got to a
> point where we can observe performance parity with existing in-kernel
> solutions yet. That's just things we're aware of right now. In some
> perfect would it would be simple as dragging and dropping existing
> kernel code into the eBPF environment, but I'm sure that one of the
> first things you'll learn on this xdp-newbies list is that everything
> you want to put in eBPF/XDP needs to be rewritten from scratch with
> the verifier in mind.

Re: XDP for Kernel by-pass?

[Daniel Borkmann]

On 07/14/2017 10:46 AM, tom.barbette@ulg.ac.be wrote:
[...]
> Netmap hooks exactly where XDP does. We would just need a little shared-memory module that hooks in XDP and give all the raw packets to a userspace program.

There is work in progress for a faster AF_PACKET interface for RX/TX, which would
be a much better fit given AF_PACKET's purpose. Some RFC patches have been posted
here [1][2] some time ago.

  [1] http://patchwork.ozlabs.org/patch/720937/
  [2] http://patchwork.ozlabs.org/patch/720936/

> Tom