Re: XDP seeking input from NIC hardware vendors

[Jakub Kicinski via iovisor-dev <iovisor-dev-9jONkmmOlFHEE9lA1F8Ukti2O/JbrIOy@public.gmane.org>]

On Fri, 8 Jul 2016 17:19:43 +0200, Jesper Dangaard Brouer wrote:
> On Fri, 8 Jul 2016 14:44:53 +0100 Jakub Kicinski <jakub.kicinski-wFxRvT7yatFl57MIdRCFDg@public.gmane.org> wrote:
> > On Thu, 7 Jul 2016 19:22:12 -0700, Alexei Starovoitov wrote:
> > > > If the goal is to just separate XDP traffic from non-XDP traffic
> > > > you could accomplish this with a combination of SR-IOV/macvlan to
> > > > separate the device queues into multiple netdevs and then run XDP
> > > > on just one of the netdevs. Then use flow director (ethtool) or
> > > > 'tc cls_u32/flower' to steer traffic to the netdev. This is how
> > > > we support multiple networking stacks on one device by the way it
> > > > is called the bifurcated driver. Its not too far of a stretch to
> > > > think we could offload some simple XDP programs to program the
> > > > splitting of traffic instead of cls_u32/flower/flow_director and
> > > > then you would have a stack of XDP programs. One running in
> > > > hardware and a set running on the queues in software.      
> > > 
> > >
> > > the above sounds like much better approach then Jesper/mine
> > > prog_per_ring stuff.
> > >
> > > If we can split the nic via sriov and have dedicated netdev via VF
> > > just for XDP that's way cleaner approach. I guess we won't need to
> > > do xdp_rxqmask after all.    
> > 
> > +1
> > 
> > I was thinking about using eBPF to direct to NIC queues but concluded
> > that doing a redirect to a VF is cleaner.  Especially if the PF driver
> > supports VF representatives we could potentially just use
> > bpf_redirect(VFR netdev) and the VF doesn't even have to be handled by
> > the same stack.  
> 
> I actually disagree.
> 
> I _do_ want to use the "filter" part of eBPF to direct to NIC queues, and
> then run a single/specific XDP program on that queue.
> 
> Why to I want this?
> 
> This part of solving a very fundamental CS problem (early demux), when
> wanting to support Zero-copy on RX.  The basic problem that the NIC
> driver need to map RX pages into the RX ring, prior to receiving
> packets. Thus, we need HW support to steer packets, for gaining enough
> isolation (e.g between tenants domains) for allowing zero-copy.
> 
> 
> Based on the flexibility of the HW-filter, the granularity achievable
> for isolation (e.g. application specific) is much more flexible.  Than
> splitting up the entire NIC with SR-IOV, VFs or macvlans.

I think of SR-IOV VFs a way of grouping queues.  If HW is capable of
directing to a queue it's usually capable of directing to a VF as well.
And the VF could have all other traffic disabled so you would get only
packets directed to it by the (BPF) filter - same as you would for the
queue.  Does that make sense for zero copy apps?