Re: XDP seeking input from NIC hardware vendors

[John Fastabend via iovisor-dev <iovisor-dev-9jONkmmOlFHEE9lA1F8Ukti2O/JbrIOy@public.gmane.org>]

On 16-07-07 07:22 PM, Alexei Starovoitov wrote:
> On Thu, Jul 07, 2016 at 03:18:11PM +0000, Fastabend, John R wrote:
>> Hi Jesper,
>>
>> I have done some previous work on proprietary systems where we
>> used hardware to do the classification/parsing then passed a cookie to the
>> software which used the cookie to lookup a program to run on the packet.
>> When your programs are structured as a bunch of parsing followed by some
>> actions this can provide real performance benefits. Also a lot of
>> existing hardware supports this today assuming you use headers the
>> hardware "knows" about. It's a natural model for hardware that uses a
>> parser followed by tcam/cam/sram/etc lookup tables.

> looking at bpf programs written in plumgrid, facebook and cisco
> with full certainty I can assure that parse/action split doesn't exist.
> Parsing is always interleaved with lookups and actions.
> cpu spends a tiny fraction of time doing parsing. Lookups are the heaviest.

What is heavy about a lookup? Is it the key generation? The key
generation can be provided by the hardware is what I was really alluding
to. If your data structures are ebpf maps though its probably a hash
or array table and the benefit of leveraging hardware would likely be
much better if/when there are software structures for LPM or wildcard
lookups.

> Trying to split single logical program into parsing/after_parse stages
> has no pracitcal benefit.
> 
>> If the goal is to just separate XDP traffic from non-XDP traffic
>> you could accomplish this with a combination of SR-IOV/macvlan to separate
>> the device queues into multiple netdevs and then run XDP on just one of
>> the netdevs. Then use flow director (ethtool) or 'tc cls_u32/flower' to
>> steer traffic to the netdev. This is how we support multiple networking
>> stacks on one device by the way it is called the bifurcated driver. Its
>> not too far of a stretch to think we could offload some simple XDP
>> programs to program the splitting of traffic instead of
>> cls_u32/flower/flow_director and then you would have a stack of XDP
>> programs. One running in hardware and a set running on the queues in
>> software.
> 
> the above sounds like much better approach then Jesper/mine prog_per_ring stuff.
> If we can split the nic via sriov and have dedicated netdev via VF just for XDP that's way cleaner approach.
> I guess we won't need to do xdp_rxqmask after all.
> 

Right and this works today so all it would require is adding the XDP
engine code to the VF drivers. Which should be relatively straight
forward if you have the PF driver working.

.John