Re: bifurcated driver

[Vincent JARDIN <vincent.jardin-pdR9zngts4EAvxtiuMwx3w@public.gmane.org>]

+Or

On 05/11/2014 23:48, Zhou, Danny wrote:
> Hi Thomas,
>
> Thanks for sharing the links to ibverbs, I will take a close look at it and compare it to bifurcated driver. My take
> after a rough review is that idea is very much similar, but bifurcated driver implementation is generic for any
> Ethernet device based on existing af_packet mechanism, with extension of exchanging the messages between
> user space and kernel space driver.
>
> I have an internal document to summary the pros and cons of below solutions, except for ibvers, but
> will be adding it shortly.
>
> - igb_uio
> - uio_pci_generic
> - VFIO
> - bifurcated driver
>
> Short answers to your questions:
>> 	- upstream status
> Adding IOMMU based memory protection and generic descriptor description support now, into version 2
> kernel patches.
>
>> 	- usable with kernel netdev
> af_packet based, and relevant patchset will be submitted to netdev for sure.
>
>> 	- usable in a vm
> No, it does no coexist with SRIOV for number of reasons. but if you pass-through a PF to a VM, it works perfect.
>
>> 	- usable for Ethernet
> It could work with all Ethernet NICs, as flow director is available and NIC driver support new net_ops to split off
> queue pairs for user space.
>
>> 	- hardware requirements
> No specific hardware requirements. All mainstream NICs have multiple qpairs and flow director support.
>
>> 	- security protection
> Leverage IOMMU to provide memory protection on Intel platform. Other archs provide similar memory protection
> mechanism, so we only use arch-agnostic DMA memory allocation APIs in kernel to support memory protection.
>
>> 	- performance
> DPDK native performance on user space queues, as long as drop_en is enabled to avoid head-of-line blocking.
>
> -Danny
>
>> -----Original Message-----
>> From: Thomas Monjalon [mailto:thomas.monjalon-pdR9zngts4EAvxtiuMwx3w@public.gmane.org]
>> Sent: Wednesday, November 05, 2014 9:01 PM
>> To: Zhou, Danny
>> Cc: dev-VfR2kkLFssw@public.gmane.org; Fastabend, John R
>> Subject: Re: [dpdk-dev] bifurcated driver
>>
>> Hi Danny,
>>
>> 2014-10-31 17:36, O'driscoll, Tim:
>>> Bifurcated Driver (Danny.Zhou-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org)
>>
>> Thanks for the presentation of bifurcated driver during the community call.
>> I asked if you looked at ibverbs and you wanted a link to check.
>> The kernel module is here:
>> 	http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/drivers/infiniband/core
>> The userspace library:
>> 	http://git.kernel.org/cgit/libs/infiniband/libibverbs.git
>>
>> Extract from Kconfig:
>> "
>> config INFINIBAND_USER_ACCESS
>> 	tristate "InfiniBand userspace access (verbs and CM)"
>> 	select ANON_INODES
>> 	---help---
>> 	  Userspace InfiniBand access support.  This enables the
>> 	  kernel side of userspace verbs and the userspace
>> 	  communication manager (CM).  This allows userspace processes
>> 	  to set up connections and directly access InfiniBand
>> 	  hardware for fast-path operations.  You will also need
>> 	  libibverbs, libibcm and a hardware driver library from
>> 	  <http://www.openfabrics.org/git/>.
>> "
>>
>> It seems to be close to the bifurcated driver needs.
>> Not sure if it can solve the security issues if there is no dedicated MMU
>> in the NIC.
>>
>> I feel we should sum up pros and cons of
>> 	- igb_uio
>> 	- uio_pci_generic
>> 	- VFIO
>> 	- ibverbs
>> 	- bifurcated driver
>> I suggest to consider these criterias:
>> 	- upstream status
>> 	- usable with kernel netdev
>> 	- usable in a vm
>> 	- usable for ethernet
>> 	- hardware requirements
>> 	- security protection
>> 	- performance
>>
>> --
>> Thomas