Re: [RFC 0/5] add XDP support to mt76x2e/mt76x0e drivers

Re: [RFC 0/5] add XDP support to mt76x2e/mt76x0e drivers

[Jesper Dangaard Brouer]

On Wed, 28 Nov 2018 13:36:26 +0100
Toke Høiland-Jørgensen <toke-LJ9M9ZcSy1A@public.gmane.org> wrote:

> Lorenzo Bianconi <lorenzo.bianconi-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> writes:
> 
> >> Lorenzo Bianconi <lorenzo.bianconi-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> writes:
> >>   
> >> > This series is intended as a playground to start experimenting/developing
> >> > with XDP/eBPF over WiFi and collect ideas/concerns about it.
> >> > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported
> >> > actions are:
> >> > - XDP_PASS
> >> > - XDP_ABORTED
> >> > - XDP_DROP
> >> > Introduce ndo_bpf mac80211 callback in order to to load a bpf
> >> > program into low level driver XDP rx hook.
> >> > This series has been tested through a simple bpf program (available here:
> >> > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats)
> >> > used to count frame types received by the device.
> >> > Possible eBPF use cases could be:
> >> > - implement new statistics through bpf maps
> >> > - implement fast packet filtering (e.g in monitor mode)
> >> > - ...  
> >
> > Hi Kalle,
> >  
> >> 
> >> This is most likely a stupid question, but why do this in the driver and
> >> not in mac80211 so that all drivers could benefit from it? I guess there
> >> are reasons for that, I just can't figure that out.  
> 
> XDP achieves its speedup by running the eBPF program inside the driver
> NAPI loop, before the kernel even touches the data in any other capacity
> (and in particular, before it allocates an SKB). Which kinda means the
> hook needs to be in the driver... Could be a fallback in mac80211,
> though; although we'd have to figure out how that interacts with Generic
> XDP.
> 
> > This is an early stage implementation, at this point I would collect
> > other people opinions/concerns about using bpf/xdp directly on 802.11
> > frames.  
> 
> Thanks for looking into this!
> 
> I have two concerns with running XDP on 802.11 frames:
> 
> 1. It makes it more difficult to add other XDP actions (such as
>    REDIRECT), as the XDP program would then have to make sure that the
>    outer packet headers are removed before, say, redirecting the packet
>    out of an ethernet interface. Also, if we do add redirect, we would
>    be bypassing mac80211 entirely; to what extent would that mess up
>    internal state?
> 
> 2. UI consistency; suddenly, the user needs to know which kind of
>    frames to expect, and XDP program reuse becomes more difficult. This
>    may be unavoidable given the nature of XDP, but some thought needs to
>    go into this. Especially since we wouldn't necessarily be consistent
>    between WiFi drivers (there are fullmac devices that remove 802.11
>    headers before sending up the frame, right?).
> 
> 
> Adding in Jesper; maybe he has some thoughts on this?

Today XDP assumes the frame is an Ethernet frame.  With WiFi I guess
this assumption change, right?
  I worry a bit about this, as XDP is all about performance, and I don't
want to add performance regressions, by requiring all XDP programs or
core-code to having to check-frame-type before proceeding. That said, I
do think it is doable, without adding performance regressions.

Option #1 is to move the check-frame-type to setup time.  By either
having frame-type be part of eBPF prog, or supply frame-type as option
XDP attach call.  And then reject attaching XDP prog to a device, where
the expected frame-type does not match.

Option#2, leave it up to eBPF-programmer if they want to add runtime
checks.  By extending xdp_rxq_info with frame-type (default to
Ethernet), which allow the eBPF-programmer choose to write a generic
XDP program that both work on Ethernet and WiFi, or skip-check as they
know this will e.g. only run on Wifi.  (Note xdp_rxq_info is static
read-only info per RX-queue, will all Wifi frames have same frame-type?.


Also consider what happens in case of XDP_REDIRECT, from a Wifi NIC to
an Ethernet NIC.  It would of-cause be cool to get this working cross,
Wifi-Ethernet.

Option#3 is to say, Wifi XDP is so different that we should create a
new (enum) bpf_prog_type.  And then still see if we can leverage some
of the same core-code (as long as it doesn't slowdown performance).

-- 
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Principal Kernel Engineer at Red Hat
  LinkedIn: http://www.linkedin.com/in/brouer

Re: [RFC 0/5] add XDP support to mt76x2e/mt76x0e drivers

[Lorenzo Bianconi]

> On Wed, 28 Nov 2018 13:36:26 +0100
> Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> 
> > Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes:
> > 
> > >> Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes:
> > >>   
> > >> > This series is intended as a playground to start experimenting/developing
> > >> > with XDP/eBPF over WiFi and collect ideas/concerns about it.
> > >> > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported
> > >> > actions are:
> > >> > - XDP_PASS
> > >> > - XDP_ABORTED
> > >> > - XDP_DROP
> > >> > Introduce ndo_bpf mac80211 callback in order to to load a bpf
> > >> > program into low level driver XDP rx hook.
> > >> > This series has been tested through a simple bpf program (available here:
> > >> > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats)
> > >> > used to count frame types received by the device.
> > >> > Possible eBPF use cases could be:
> > >> > - implement new statistics through bpf maps
> > >> > - implement fast packet filtering (e.g in monitor mode)
> > >> > - ...  
> > >
> > > Hi Kalle,
> > >  
> > >> 
> > >> This is most likely a stupid question, but why do this in the driver and
> > >> not in mac80211 so that all drivers could benefit from it? I guess there
> > >> are reasons for that, I just can't figure that out.  
> > 
> > XDP achieves its speedup by running the eBPF program inside the driver
> > NAPI loop, before the kernel even touches the data in any other capacity
> > (and in particular, before it allocates an SKB). Which kinda means the
> > hook needs to be in the driver... Could be a fallback in mac80211,
> > though; although we'd have to figure out how that interacts with Generic
> > XDP.
> > 
> > > This is an early stage implementation, at this point I would collect
> > > other people opinions/concerns about using bpf/xdp directly on 802.11
> > > frames.  
> > 
> > Thanks for looking into this!
> > 
> > I have two concerns with running XDP on 802.11 frames:
> > 
> > 1. It makes it more difficult to add other XDP actions (such as
> >    REDIRECT), as the XDP program would then have to make sure that the
> >    outer packet headers are removed before, say, redirecting the packet
> >    out of an ethernet interface. Also, if we do add redirect, we would
> >    be bypassing mac80211 entirely; to what extent would that mess up
> >    internal state?
> > 
> > 2. UI consistency; suddenly, the user needs to know which kind of
> >    frames to expect, and XDP program reuse becomes more difficult. This
> >    may be unavoidable given the nature of XDP, but some thought needs to
> >    go into this. Especially since we wouldn't necessarily be consistent
> >    between WiFi drivers (there are fullmac devices that remove 802.11
> >    headers before sending up the frame, right?).
> > 
> > 
> > Adding in Jesper; maybe he has some thoughts on this?

Hi Jesper,

> 
> Today XDP assumes the frame is an Ethernet frame.  With WiFi I guess
> this assumption change, right?

yes correct, SoftMAC devices report 802.11 frames to the stack

>   I worry a bit about this, as XDP is all about performance, and I don't
> want to add performance regressions, by requiring all XDP programs or
> core-code to having to check-frame-type before proceeding. That said, I
> do think it is doable, without adding performance regressions.
> 
> Option #1 is to move the check-frame-type to setup time.  By either
> having frame-type be part of eBPF prog, or supply frame-type as option
> XDP attach call.  And then reject attaching XDP prog to a device, where
> the expected frame-type does not match.
> 

I guess it will be enough to avoid loading a 'non-WiFi' bpf program on a 802.11
netdevice (and vice versa). We could add a flag (or something similar) in
XDP_SETUP_PROG section of netdev_bpf data structure and use ieee80211_ptr
netdevice pointer in order to guarantee that the bpf program will work on the
expected 'frame-type'

> Option#2, leave it up to eBPF-programmer if they want to add runtime
> checks.  By extending xdp_rxq_info with frame-type (default to
> Ethernet), which allow the eBPF-programmer choose to write a generic
> XDP program that both work on Ethernet and WiFi, or skip-check as they
> know this will e.g. only run on Wifi.  (Note xdp_rxq_info is static
> read-only info per RX-queue, will all Wifi frames have same frame-type?.
> 

802.11 standards define three frame subtype (data, management and control).
Subtypes could be detected parsing 802.11 header

> 
> Also consider what happens in case of XDP_REDIRECT, from a Wifi NIC to
> an Ethernet NIC.  It would of-cause be cool to get this working cross,
> Wifi-Ethernet.
> 

Very cool :) On tx side the driver will accept standard ethernet frames in
ndo_xdp_xmit pointer

> Option#3 is to say, Wifi XDP is so different that we should create a
> new (enum) bpf_prog_type.  And then still see if we can leverage some
> of the same core-code (as long as it doesn't slowdown performance).
> 

Do you think that Option#3 will be more 'future-proof' respect to Option#1?

Regards,
Lorenzo

> -- 
> Best regards,
>   Jesper Dangaard Brouer
>   MSc.CS, Principal Kernel Engineer at Red Hat
>   LinkedIn: http://www.linkedin.com/in/brouer

Re: [RFC 0/5] add XDP support to mt76x2e/mt76x0e drivers

[Toke Høiland-Jørgensen]

Lorenzo Bianconi <lorenzo.bianconi-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> writes:

>> On Wed, 28 Nov 2018 13:36:26 +0100
>> Toke Høiland-Jørgensen <toke-LJ9M9ZcSy1A@public.gmane.org> wrote:
>> 
>> > Lorenzo Bianconi <lorenzo.bianconi-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> writes:
>> > 
>> > >> Lorenzo Bianconi <lorenzo.bianconi-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> writes:
>> > >>   
>> > >> > This series is intended as a playground to start experimenting/developing
>> > >> > with XDP/eBPF over WiFi and collect ideas/concerns about it.
>> > >> > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported
>> > >> > actions are:
>> > >> > - XDP_PASS
>> > >> > - XDP_ABORTED
>> > >> > - XDP_DROP
>> > >> > Introduce ndo_bpf mac80211 callback in order to to load a bpf
>> > >> > program into low level driver XDP rx hook.
>> > >> > This series has been tested through a simple bpf program (available here:
>> > >> > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats)
>> > >> > used to count frame types received by the device.
>> > >> > Possible eBPF use cases could be:
>> > >> > - implement new statistics through bpf maps
>> > >> > - implement fast packet filtering (e.g in monitor mode)
>> > >> > - ...  
>> > >
>> > > Hi Kalle,
>> > >  
>> > >> 
>> > >> This is most likely a stupid question, but why do this in the driver and
>> > >> not in mac80211 so that all drivers could benefit from it? I guess there
>> > >> are reasons for that, I just can't figure that out.  
>> > 
>> > XDP achieves its speedup by running the eBPF program inside the driver
>> > NAPI loop, before the kernel even touches the data in any other capacity
>> > (and in particular, before it allocates an SKB). Which kinda means the
>> > hook needs to be in the driver... Could be a fallback in mac80211,
>> > though; although we'd have to figure out how that interacts with Generic
>> > XDP.
>> > 
>> > > This is an early stage implementation, at this point I would collect
>> > > other people opinions/concerns about using bpf/xdp directly on 802.11
>> > > frames.  
>> > 
>> > Thanks for looking into this!
>> > 
>> > I have two concerns with running XDP on 802.11 frames:
>> > 
>> > 1. It makes it more difficult to add other XDP actions (such as
>> >    REDIRECT), as the XDP program would then have to make sure that the
>> >    outer packet headers are removed before, say, redirecting the packet
>> >    out of an ethernet interface. Also, if we do add redirect, we would
>> >    be bypassing mac80211 entirely; to what extent would that mess up
>> >    internal state?
>> > 
>> > 2. UI consistency; suddenly, the user needs to know which kind of
>> >    frames to expect, and XDP program reuse becomes more difficult. This
>> >    may be unavoidable given the nature of XDP, but some thought needs to
>> >    go into this. Especially since we wouldn't necessarily be consistent
>> >    between WiFi drivers (there are fullmac devices that remove 802.11
>> >    headers before sending up the frame, right?).
>> > 
>> > 
>> > Adding in Jesper; maybe he has some thoughts on this?
>
> Hi Jesper,
>
>> 
>> Today XDP assumes the frame is an Ethernet frame.  With WiFi I guess
>> this assumption change, right?
>
> yes correct, SoftMAC devices report 802.11 frames to the stack
>
>>   I worry a bit about this, as XDP is all about performance, and I don't
>> want to add performance regressions, by requiring all XDP programs or
>> core-code to having to check-frame-type before proceeding. That said, I
>> do think it is doable, without adding performance regressions.
>> 
>> Option #1 is to move the check-frame-type to setup time.  By either
>> having frame-type be part of eBPF prog, or supply frame-type as option
>> XDP attach call.  And then reject attaching XDP prog to a device, where
>> the expected frame-type does not match.
>> 
>
> I guess it will be enough to avoid loading a 'non-WiFi' bpf program on
> a 802.11 netdevice (and vice versa). We could add a flag (or something
> similar) in XDP_SETUP_PROG section of netdev_bpf data structure and
> use ieee80211_ptr netdevice pointer in order to guarantee that the bpf
> program will work on the expected 'frame-type'

Yeah, a flag would be good; we've been discussing that for other XDP use
cases; it's not a done deal yet, but I think it would be useful.
>
>> Option#2, leave it up to eBPF-programmer if they want to add runtime
>> checks.  By extending xdp_rxq_info with frame-type (default to
>> Ethernet), which allow the eBPF-programmer choose to write a generic
>> XDP program that both work on Ethernet and WiFi, or skip-check as they
>> know this will e.g. only run on Wifi.  (Note xdp_rxq_info is static
>> read-only info per RX-queue, will all Wifi frames have same frame-type?.
>> 
>
> 802.11 standards define three frame subtype (data, management and control).
> Subtypes could be detected parsing 802.11 header
>
>> 
>> Also consider what happens in case of XDP_REDIRECT, from a Wifi NIC to
>> an Ethernet NIC.  It would of-cause be cool to get this working cross,
>> Wifi-Ethernet.
>> 
>
> Very cool :) On tx side the driver will accept standard ethernet frames in
> ndo_xdp_xmit pointer

How do you envision that will work with drivers that build software
802.11 frames? The TX hook would have to be in mac80211 somewhere,
wouldn't it?

>> Option#3 is to say, Wifi XDP is so different that we should create a
>> new (enum) bpf_prog_type.  And then still see if we can leverage some
>> of the same core-code (as long as it doesn't slowdown performance).
>> 
>
> Do you think that Option#3 will be more 'future-proof' respect to
> Option#1?

My feeling is that WiFi devices are not sufficiently different to
warrant a whole new program type. We risk combinatorial explosion for
all the stuff that is the same, but now need to be tested for two (or N)
types...

-Toke

Re: [RFC 0/5] add XDP support to mt76x2e/mt76x0e drivers

[Michał Kazior]

On Thu, 29 Nov 2018 at 14:31, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
[...]
> >> Option#3 is to say, Wifi XDP is so different that we should create a
> >> new (enum) bpf_prog_type.  And then still see if we can leverage some
> >> of the same core-code (as long as it doesn't slowdown performance).
> >>
> >
> > Do you think that Option#3 will be more 'future-proof' respect to
> > Option#1?
>
> My feeling is that WiFi devices are not sufficiently different to
> warrant a whole new program type. We risk combinatorial explosion for
> all the stuff that is the same, but now need to be tested for two (or N)
> types...

I'm not sure if my understanding is correct, but XDP seems like it can
(and intends to be able to?) act as a general purpose packet
accelerator (REDIRECT action was mentioned so I'm inferring..). In
such case you'll need to be able to perform transformations on packets
too, e.g. strip/prepend vlan tags, gre headers and what have you. The
802.3 <-> 802.11 conversion could be treated on equal terms.


Michał

Re: [RFC 0/5] add XDP support to mt76x2e/mt76x0e drivers

[Toke Høiland-Jørgensen]

Michał Kazior <kazikcz@gmail.com> writes:

> On Thu, 29 Nov 2018 at 14:31, Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> [...]
>> >> Option#3 is to say, Wifi XDP is so different that we should create a
>> >> new (enum) bpf_prog_type.  And then still see if we can leverage some
>> >> of the same core-code (as long as it doesn't slowdown performance).
>> >>
>> >
>> > Do you think that Option#3 will be more 'future-proof' respect to
>> > Option#1?
>>
>> My feeling is that WiFi devices are not sufficiently different to
>> warrant a whole new program type. We risk combinatorial explosion for
>> all the stuff that is the same, but now need to be tested for two (or N)
>> types...
>
> I'm not sure if my understanding is correct, but XDP seems like it can
> (and intends to be able to?) act as a general purpose packet
> accelerator (REDIRECT action was mentioned so I'm inferring..). In
> such case you'll need to be able to perform transformations on packets
> too, e.g. strip/prepend vlan tags, gre headers and what have you. The
> 802.3 <-> 802.11 conversion could be treated on equal terms.

Sure, that is perfectly possible. It just needs to be implemented by the
eBPF program being loaded; and there is a facility to shrink/expand the
packet size for encapsulation processing. It's just that since currently
all interfaces process Ethernet frames, this frame type assumption has
kinda been built in. So something needs to be done to handle that.

The minimum is just to ignore the issue and make it up to the program
writer to handle this or risk breaking things, but something a bit
friendlier (such as an ability for the loaded program to indicate which
frame type it is prepared to handle, which can be sanity-checked by the
loader) might be a good idea... Especially if different WiFi devices
will emit different frame types (so we can't just go "it's a WiFi
device, you should know this").

Then there's the additional issue that since mac80211 and/or the driver
may have internal state, we need to expose appropriate helpers for an
XDP program to be able to update this as it's processing packets (before
bypassing the stack, for instance).

-Toke

Re: [RFC 0/5] add XDP support to mt76x2e/mt76x0e drivers

[Lorenzo Bianconi]

> Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes:
> 
> >> On Wed, 28 Nov 2018 13:36:26 +0100
> >> Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> >> 
> >> > Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes:
> >> > 
> >> > >> Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes:
> >> > >>   
> >> > >> > This series is intended as a playground to start experimenting/developing
> >> > >> > with XDP/eBPF over WiFi and collect ideas/concerns about it.
> >> > >> > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported
> >> > >> > actions are:
> >> > >> > - XDP_PASS
> >> > >> > - XDP_ABORTED
> >> > >> > - XDP_DROP
> >> > >> > Introduce ndo_bpf mac80211 callback in order to to load a bpf
> >> > >> > program into low level driver XDP rx hook.
> >> > >> > This series has been tested through a simple bpf program (available here:
> >> > >> > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats)
> >> > >> > used to count frame types received by the device.
> >> > >> > Possible eBPF use cases could be:
> >> > >> > - implement new statistics through bpf maps
> >> > >> > - implement fast packet filtering (e.g in monitor mode)
> >> > >> > - ...  
> >> > >
> >> > > Hi Kalle,
> >> > >  
> >> > >> 
> >> > >> This is most likely a stupid question, but why do this in the driver and
> >> > >> not in mac80211 so that all drivers could benefit from it? I guess there
> >> > >> are reasons for that, I just can't figure that out.  
> >> > 
> >> > XDP achieves its speedup by running the eBPF program inside the driver
> >> > NAPI loop, before the kernel even touches the data in any other capacity
> >> > (and in particular, before it allocates an SKB). Which kinda means the
> >> > hook needs to be in the driver... Could be a fallback in mac80211,
> >> > though; although we'd have to figure out how that interacts with Generic
> >> > XDP.
> >> > 
> >> > > This is an early stage implementation, at this point I would collect
> >> > > other people opinions/concerns about using bpf/xdp directly on 802.11
> >> > > frames.  
> >> > 
> >> > Thanks for looking into this!
> >> > 
> >> > I have two concerns with running XDP on 802.11 frames:
> >> > 
> >> > 1. It makes it more difficult to add other XDP actions (such as
> >> >    REDIRECT), as the XDP program would then have to make sure that the
> >> >    outer packet headers are removed before, say, redirecting the packet
> >> >    out of an ethernet interface. Also, if we do add redirect, we would
> >> >    be bypassing mac80211 entirely; to what extent would that mess up
> >> >    internal state?
> >> > 
> >> > 2. UI consistency; suddenly, the user needs to know which kind of
> >> >    frames to expect, and XDP program reuse becomes more difficult. This
> >> >    may be unavoidable given the nature of XDP, but some thought needs to
> >> >    go into this. Especially since we wouldn't necessarily be consistent
> >> >    between WiFi drivers (there are fullmac devices that remove 802.11
> >> >    headers before sending up the frame, right?).
> >> > 
> >> > 
> >> > Adding in Jesper; maybe he has some thoughts on this?
> >
> > Hi Jesper,
> >
> >> 
> >> Today XDP assumes the frame is an Ethernet frame.  With WiFi I guess
> >> this assumption change, right?
> >
> > yes correct, SoftMAC devices report 802.11 frames to the stack
> >
> >>   I worry a bit about this, as XDP is all about performance, and I don't
> >> want to add performance regressions, by requiring all XDP programs or
> >> core-code to having to check-frame-type before proceeding. That said, I
> >> do think it is doable, without adding performance regressions.
> >> 
> >> Option #1 is to move the check-frame-type to setup time.  By either
> >> having frame-type be part of eBPF prog, or supply frame-type as option
> >> XDP attach call.  And then reject attaching XDP prog to a device, where
> >> the expected frame-type does not match.
> >> 
> >
> > I guess it will be enough to avoid loading a 'non-WiFi' bpf program on
> > a 802.11 netdevice (and vice versa). We could add a flag (or something
> > similar) in XDP_SETUP_PROG section of netdev_bpf data structure and
> > use ieee80211_ptr netdevice pointer in order to guarantee that the bpf
> > program will work on the expected 'frame-type'
> 
> Yeah, a flag would be good; we've been discussing that for other XDP use
> cases; it's not a done deal yet, but I think it would be useful.

Do you think something wifi specific is ok (e.g bool wifi) or do you prefer
something more general (e.g u32 frame_type)?

> >
> >> Option#2, leave it up to eBPF-programmer if they want to add runtime
> >> checks.  By extending xdp_rxq_info with frame-type (default to
> >> Ethernet), which allow the eBPF-programmer choose to write a generic
> >> XDP program that both work on Ethernet and WiFi, or skip-check as they
> >> know this will e.g. only run on Wifi.  (Note xdp_rxq_info is static
> >> read-only info per RX-queue, will all Wifi frames have same frame-type?.
> >> 
> >
> > 802.11 standards define three frame subtype (data, management and control).
> > Subtypes could be detected parsing 802.11 header
> >
> >> 
> >> Also consider what happens in case of XDP_REDIRECT, from a Wifi NIC to
> >> an Ethernet NIC.  It would of-cause be cool to get this working cross,
> >> Wifi-Ethernet.
> >> 
> >
> > Very cool :) On tx side the driver will accept standard ethernet frames in
> > ndo_xdp_xmit pointer
> 
> How do you envision that will work with drivers that build software
> 802.11 frames? The TX hook would have to be in mac80211 somewhere,
> wouldn't it?

In order to perform 802.3 --> 802.11 xdp forwarding my current idea is is to have
ndo_xdp_xmit pointer in mac80211 that will forward the frame to the low-level driver
(more or less what I did in the RFC series to upload the bpf program to mt76).
We will probably need to pass some info to the driver from mac80211 (e.g sequence
number or hw key idx to use)

> 
> >> Option#3 is to say, Wifi XDP is so different that we should create a
> >> new (enum) bpf_prog_type.  And then still see if we can leverage some
> >> of the same core-code (as long as it doesn't slowdown performance).
> >> 
> >
> > Do you think that Option#3 will be more 'future-proof' respect to
> > Option#1?
> 
> My feeling is that WiFi devices are not sufficiently different to
> warrant a whole new program type. We risk combinatorial explosion for
> all the stuff that is the same, but now need to be tested for two (or N)
> types...

Agree

Regards,
Lorenzo

> 
> -Toke

Re: [RFC 0/5] add XDP support to mt76x2e/mt76x0e drivers

[Toke Høiland-Jørgensen]

Lorenzo Bianconi <lorenzo.bianconi-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> writes:

>> Lorenzo Bianconi <lorenzo.bianconi-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> writes:
>> 
>> >> On Wed, 28 Nov 2018 13:36:26 +0100
>> >> Toke Høiland-Jørgensen <toke-LJ9M9ZcSy1A@public.gmane.org> wrote:
>> >> 
>> >> > Lorenzo Bianconi <lorenzo.bianconi-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> writes:
>> >> > 
>> >> > >> Lorenzo Bianconi <lorenzo.bianconi-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> writes:
>> >> > >>   
>> >> > >> > This series is intended as a playground to start experimenting/developing
>> >> > >> > with XDP/eBPF over WiFi and collect ideas/concerns about it.
>> >> > >> > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported
>> >> > >> > actions are:
>> >> > >> > - XDP_PASS
>> >> > >> > - XDP_ABORTED
>> >> > >> > - XDP_DROP
>> >> > >> > Introduce ndo_bpf mac80211 callback in order to to load a bpf
>> >> > >> > program into low level driver XDP rx hook.
>> >> > >> > This series has been tested through a simple bpf program (available here:
>> >> > >> > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats)
>> >> > >> > used to count frame types received by the device.
>> >> > >> > Possible eBPF use cases could be:
>> >> > >> > - implement new statistics through bpf maps
>> >> > >> > - implement fast packet filtering (e.g in monitor mode)
>> >> > >> > - ...  
>> >> > >
>> >> > > Hi Kalle,
>> >> > >  
>> >> > >> 
>> >> > >> This is most likely a stupid question, but why do this in the driver and
>> >> > >> not in mac80211 so that all drivers could benefit from it? I guess there
>> >> > >> are reasons for that, I just can't figure that out.  
>> >> > 
>> >> > XDP achieves its speedup by running the eBPF program inside the driver
>> >> > NAPI loop, before the kernel even touches the data in any other capacity
>> >> > (and in particular, before it allocates an SKB). Which kinda means the
>> >> > hook needs to be in the driver... Could be a fallback in mac80211,
>> >> > though; although we'd have to figure out how that interacts with Generic
>> >> > XDP.
>> >> > 
>> >> > > This is an early stage implementation, at this point I would collect
>> >> > > other people opinions/concerns about using bpf/xdp directly on 802.11
>> >> > > frames.  
>> >> > 
>> >> > Thanks for looking into this!
>> >> > 
>> >> > I have two concerns with running XDP on 802.11 frames:
>> >> > 
>> >> > 1. It makes it more difficult to add other XDP actions (such as
>> >> >    REDIRECT), as the XDP program would then have to make sure that the
>> >> >    outer packet headers are removed before, say, redirecting the packet
>> >> >    out of an ethernet interface. Also, if we do add redirect, we would
>> >> >    be bypassing mac80211 entirely; to what extent would that mess up
>> >> >    internal state?
>> >> > 
>> >> > 2. UI consistency; suddenly, the user needs to know which kind of
>> >> >    frames to expect, and XDP program reuse becomes more difficult. This
>> >> >    may be unavoidable given the nature of XDP, but some thought needs to
>> >> >    go into this. Especially since we wouldn't necessarily be consistent
>> >> >    between WiFi drivers (there are fullmac devices that remove 802.11
>> >> >    headers before sending up the frame, right?).
>> >> > 
>> >> > 
>> >> > Adding in Jesper; maybe he has some thoughts on this?
>> >
>> > Hi Jesper,
>> >
>> >> 
>> >> Today XDP assumes the frame is an Ethernet frame.  With WiFi I guess
>> >> this assumption change, right?
>> >
>> > yes correct, SoftMAC devices report 802.11 frames to the stack
>> >
>> >>   I worry a bit about this, as XDP is all about performance, and I don't
>> >> want to add performance regressions, by requiring all XDP programs or
>> >> core-code to having to check-frame-type before proceeding. That said, I
>> >> do think it is doable, without adding performance regressions.
>> >> 
>> >> Option #1 is to move the check-frame-type to setup time.  By either
>> >> having frame-type be part of eBPF prog, or supply frame-type as option
>> >> XDP attach call.  And then reject attaching XDP prog to a device, where
>> >> the expected frame-type does not match.
>> >> 
>> >
>> > I guess it will be enough to avoid loading a 'non-WiFi' bpf program on
>> > a 802.11 netdevice (and vice versa). We could add a flag (or something
>> > similar) in XDP_SETUP_PROG section of netdev_bpf data structure and
>> > use ieee80211_ptr netdevice pointer in order to guarantee that the bpf
>> > program will work on the expected 'frame-type'
>> 
>> Yeah, a flag would be good; we've been discussing that for other XDP use
>> cases; it's not a done deal yet, but I think it would be useful.
>
> Do you think something wifi specific is ok (e.g bool wifi) or do you prefer
> something more general (e.g u32 frame_type)?

My thought was a feature flag where the program can set a flag which
means "I expect 802.11 frames", and the driver can set a flag saying "I
emit 802.11 frames", and if those two flags don't match, the verifier
can refuse to load the program. This would not be fool-proof (an XDP
program can still corrupt things if written incorrectly), but it would
at least protect against the most obvious mistakes.

>> >> Option#2, leave it up to eBPF-programmer if they want to add runtime
>> >> checks.  By extending xdp_rxq_info with frame-type (default to
>> >> Ethernet), which allow the eBPF-programmer choose to write a generic
>> >> XDP program that both work on Ethernet and WiFi, or skip-check as they
>> >> know this will e.g. only run on Wifi.  (Note xdp_rxq_info is static
>> >> read-only info per RX-queue, will all Wifi frames have same frame-type?.
>> >> 
>> >
>> > 802.11 standards define three frame subtype (data, management and control).
>> > Subtypes could be detected parsing 802.11 header
>> >
>> >> 
>> >> Also consider what happens in case of XDP_REDIRECT, from a Wifi NIC to
>> >> an Ethernet NIC.  It would of-cause be cool to get this working cross,
>> >> Wifi-Ethernet.
>> >> 
>> >
>> > Very cool :) On tx side the driver will accept standard ethernet frames in
>> > ndo_xdp_xmit pointer
>> 
>> How do you envision that will work with drivers that build software
>> 802.11 frames? The TX hook would have to be in mac80211 somewhere,
>> wouldn't it?
>
> In order to perform 802.3 --> 802.11 xdp forwarding my current idea is
> is to have ndo_xdp_xmit pointer in mac80211 that will forward the
> frame to the low-level driver (more or less what I did in the RFC
> series to upload the bpf program to mt76). We will probably need to
> pass some info to the driver from mac80211 (e.g sequence number or hw
> key idx to use)

So this means that the driver will need to do the 802.11 encapsulation?
I guess we could have a fallback implementation in mac80211; but there
is possibly quite a bit of refactoring needed to make the existing code
work without an skb. Also, we need to think about queueing; I'm not sure
it's a good idea to have redirected frames bypass the TXQs...

-Toke

Re: [RFC 0/5] add XDP support to mt76x2e/mt76x0e drivers

[Lorenzo Bianconi]

> Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes:
> 
> >> Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes:
> >> 
> >> >> On Wed, 28 Nov 2018 13:36:26 +0100
> >> >> Toke Høiland-Jørgensen <toke@toke.dk> wrote:
> >> >> 

[...]

> >> >
> >> > I guess it will be enough to avoid loading a 'non-WiFi' bpf program on
> >> > a 802.11 netdevice (and vice versa). We could add a flag (or something
> >> > similar) in XDP_SETUP_PROG section of netdev_bpf data structure and
> >> > use ieee80211_ptr netdevice pointer in order to guarantee that the bpf
> >> > program will work on the expected 'frame-type'
> >> 
> >> Yeah, a flag would be good; we've been discussing that for other XDP use
> >> cases; it's not a done deal yet, but I think it would be useful.
> >
> > Do you think something wifi specific is ok (e.g bool wifi) or do you prefer
> > something more general (e.g u32 frame_type)?
> 
> My thought was a feature flag where the program can set a flag which
> means "I expect 802.11 frames", and the driver can set a flag saying "I
> emit 802.11 frames", and if those two flags don't match, the verifier
> can refuse to load the program. This would not be fool-proof (an XDP
> program can still corrupt things if written incorrectly), but it would
> at least protect against the most obvious mistakes.

I guess we can use iee80211_ptr in dev_xdp_install to double check if it is
allowed to upload a 802.11 (or 802.3) bpf program

> 
> >> >> Option#2, leave it up to eBPF-programmer if they want to add runtime
> >> >> checks.  By extending xdp_rxq_info with frame-type (default to
> >> >> Ethernet), which allow the eBPF-programmer choose to write a generic
> >> >> XDP program that both work on Ethernet and WiFi, or skip-check as they
> >> >> know this will e.g. only run on Wifi.  (Note xdp_rxq_info is static
> >> >> read-only info per RX-queue, will all Wifi frames have same frame-type?.
> >> >> 
> >> >
> >> > 802.11 standards define three frame subtype (data, management and control).
> >> > Subtypes could be detected parsing 802.11 header
> >> >
> >> >> 
> >> >> Also consider what happens in case of XDP_REDIRECT, from a Wifi NIC to
> >> >> an Ethernet NIC.  It would of-cause be cool to get this working cross,
> >> >> Wifi-Ethernet.
> >> >> 
> >> >
> >> > Very cool :) On tx side the driver will accept standard ethernet frames in
> >> > ndo_xdp_xmit pointer
> >> 
> >> How do you envision that will work with drivers that build software
> >> 802.11 frames? The TX hook would have to be in mac80211 somewhere,
> >> wouldn't it?
> >
> > In order to perform 802.3 --> 802.11 xdp forwarding my current idea is
> > is to have ndo_xdp_xmit pointer in mac80211 that will forward the
> > frame to the low-level driver (more or less what I did in the RFC
> > series to upload the bpf program to mt76). We will probably need to
> > pass some info to the driver from mac80211 (e.g sequence number or hw
> > key idx to use)
> 
> So this means that the driver will need to do the 802.11 encapsulation?
> I guess we could have a fallback implementation in mac80211; but there
> is possibly quite a bit of refactoring needed to make the existing code
> work without an skb. Also, we need to think about queueing; I'm not sure
> it's a good idea to have redirected frames bypass the TXQs...
> 

good point :)

Regards,
Lorenzo

> -Toke

Re: [RFC 0/5] add XDP support to mt76x2e/mt76x0e drivers

[Toke Høiland-Jørgensen]

Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes:

>> Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes:
>> 
>> >> Lorenzo Bianconi <lorenzo.bianconi@redhat.com> writes:
>> >> 
>> >> >> On Wed, 28 Nov 2018 13:36:26 +0100
>> >> >> Toke Høiland-Jørgensen <toke@toke.dk> wrote:
>> >> >> 
>
> [...]
>
>> >> >
>> >> > I guess it will be enough to avoid loading a 'non-WiFi' bpf program on
>> >> > a 802.11 netdevice (and vice versa). We could add a flag (or something
>> >> > similar) in XDP_SETUP_PROG section of netdev_bpf data structure and
>> >> > use ieee80211_ptr netdevice pointer in order to guarantee that the bpf
>> >> > program will work on the expected 'frame-type'
>> >> 
>> >> Yeah, a flag would be good; we've been discussing that for other XDP use
>> >> cases; it's not a done deal yet, but I think it would be useful.
>> >
>> > Do you think something wifi specific is ok (e.g bool wifi) or do you prefer
>> > something more general (e.g u32 frame_type)?
>> 
>> My thought was a feature flag where the program can set a flag which
>> means "I expect 802.11 frames", and the driver can set a flag saying "I
>> emit 802.11 frames", and if those two flags don't match, the verifier
>> can refuse to load the program. This would not be fool-proof (an XDP
>> program can still corrupt things if written incorrectly), but it would
>> at least protect against the most obvious mistakes.
>
> I guess we can use iee80211_ptr in dev_xdp_install to double check if it is
> allowed to upload a 802.11 (or 802.3) bpf program

Yeah, I think it's more an issue of convincing the wider XDP community
that support for feature flags is in fact needed ;)

-Toke