From mboxrd@z Thu Jan 1 00:00:00 1970 From: Alexei Starovoitov via iovisor-dev Subject: Re: XDP seeking input from NIC hardware vendors Date: Thu, 7 Jul 2016 19:22:12 -0700 Message-ID: <20160708022210.GA12244@ast-mbp.thefacebook.com> References: <20160707124245.6d95635a@redhat.com> Reply-To: Alexei Starovoitov Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Cc: Jakub Kicinski , "netdev-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" , "iovisor-dev-9jONkmmOlFHEE9lA1F8Ukti2O/JbrIOy@public.gmane.org" , Edward Cree , Simon Horman , Rana Shahout , Or Gerlitz , Ari Saha To: "Fastabend, John R" Return-path: Content-Disposition: inline In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: iovisor-dev-bounces-9jONkmmOlFHEE9lA1F8Ukti2O/JbrIOy@public.gmane.org Errors-To: iovisor-dev-bounces-9jONkmmOlFHEE9lA1F8Ukti2O/JbrIOy@public.gmane.org List-Id: netdev.vger.kernel.org On Thu, Jul 07, 2016 at 03:18:11PM +0000, Fastabend, John R wrote: > Hi Jesper, > > I have done some previous work on proprietary systems where we used hardware to do the classification/parsing then passed a cookie to the software which used the cookie to lookup a program to run on the packet. When your programs are structured as a bunch of parsing followed by some actions this can provide real performance benefits. Also a lot of existing hardware supports this today assuming you use headers the hardware "knows" about. It's a natural model for hardware that uses a parser followed by tcam/cam/sram/etc lookup tables. looking at bpf programs written in plumgrid, facebook and cisco with full certainty I can assure that parse/action split doesn't exist. Parsing is always interleaved with lookups and actions. cpu spends a tiny fraction of time doing parsing. Lookups are the heaviest. Trying to split single logical program into parsing/after_parse stages has no pracitcal benefit. > If the goal is to just separate XDP traffic from non-XDP traffic you could accomplish this with a combination of SR-IOV/macvlan to separate the device queues into multiple netdevs and then run XDP on just one of the netdevs. Then use flow director (ethtool) or 'tc cls_u32/flower' to steer traffic to the netdev. This is how we support multiple networking stacks on one device by the way it is called the bifurcated driver. Its not too far of a stretch to think we could offload some simple XDP programs to program the splitting of traffic instead of cls_u32/flower/flow_director and then you would have a stack of XDP programs. One running in hardware and a set running on the queues in software. the above sounds like much better approach then Jesper/mine prog_per_ring stuff. If we can split the nic via sriov and have dedicated netdev via VF just for XDP that's way cleaner approach. I guess we won't need to do xdp_rxqmask after all.