From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jason Gunthorpe Subject: Re: [PATCH V4 3/5] vDPA: introduce vDPA bus Date: Thu, 20 Feb 2020 15:14:16 +0000 Message-ID: <20200220151412.GV23930@mellanox.com> References: <20200220061141.29390-1-jasowang@redhat.com> <20200220061141.29390-4-jasowang@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: <20200220061141.29390-4-jasowang@redhat.com> Content-Language: en-US Content-ID: <56560667045D7F4389E12517EA86F97B@eurprd05.prod.outlook.com> Sender: netdev-owner@vger.kernel.org To: Jason Wang Cc: "mst@redhat.com" , "linux-kernel@vger.kernel.org" , "kvm@vger.kernel.org" , "virtualization@lists.linux-foundation.org" , "netdev@vger.kernel.org" , "tiwei.bie@intel.com" , "maxime.coquelin@redhat.com" , "cunming.liang@intel.com" , "zhihong.wang@intel.com" , "rob.miller@broadcom.com" , "xiao.w.wang@intel.com" , "haotian.wang@sifive.com" , "lingshan.zhu@intel.com" , "eperezma@redhat.com" , "lulu@redhat.com" List-Id: virtualization@lists.linuxfoundation.org On Thu, Feb 20, 2020 at 02:11:39PM +0800, Jason Wang wrote: > vDPA device is a device that uses a datapath which complies with the > virtio specifications with vendor specific control path. vDPA devices > can be both physically located on the hardware or emulated by > software. vDPA hardware devices are usually implemented through PCIE > with the following types: >=20 > - PF (Physical Function) - A single Physical Function > - VF (Virtual Function) - Device that supports single root I/O > virtualization (SR-IOV). Its Virtual Function (VF) represents a > virtualized instance of the device that can be assigned to different > partitions > - ADI (Assignable Device Interface) and its equivalents - With > technologies such as Intel Scalable IOV, a virtual device (VDEV) > composed by host OS utilizing one or more ADIs. Or its equivalent > like SF (Sub function) from Mellanox. >=20 > From a driver's perspective, depends on how and where the DMA > translation is done, vDPA devices are split into two types: >=20 > - Platform specific DMA translation - From the driver's perspective, > the device can be used on a platform where device access to data in > memory is limited and/or translated. An example is a PCIE vDPA whose > DMA request was tagged via a bus (e.g PCIE) specific way. DMA > translation and protection are done at PCIE bus IOMMU level. > - Device specific DMA translation - The device implements DMA > isolation and protection through its own logic. An example is a vDPA > device which uses on-chip IOMMU. >=20 > To hide the differences and complexity of the above types for a vDPA > device/IOMMU options and in order to present a generic virtio device > to the upper layer, a device agnostic framework is required. >=20 > This patch introduces a software vDPA bus which abstracts the > common attributes of vDPA device, vDPA bus driver and the > communication method (vdpa_config_ops) between the vDPA device > abstraction and the vDPA bus driver. This allows multiple types of > drivers to be used for vDPA device like the virtio_vdpa and vhost_vdpa > driver to operate on the bus and allow vDPA device could be used by > either kernel virtio driver or userspace vhost drivers as: >=20 > virtio drivers vhost drivers > | | > [virtio bus] [vhost uAPI] > | | > virtio device vhost device > virtio_vdpa drv vhost_vdpa drv > \ / > [vDPA bus] > | > vDPA device > hardware drv > | > [hardware bus] > | > vDPA hardware I still don't like this strange complexity, vhost should have been layered on top of the virtio device instead of adding an extra bus just for vdpa. However, I don't see any technical problems with this patch now. Thanks, Jason