From mboxrd@z Thu Jan 1 00:00:00 1970 From: Lu Baolu Subject: Re: [PATCH v3 0/8] vfio/mdev: IOMMU aware mediated device Date: Wed, 17 Oct 2018 10:10:26 +0800 Message-ID: References: <20181012051632.26064-1-baolu.lu@linux.intel.com> <5BC1AC09.1060507@huawei.com> <5BC454DF.6010109@huawei.com> <40fc685e-a0be-5e54-2de7-6cd87c36dd80@linux.intel.com> <5BC69830.707@huawei.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Cc: baolu.lu@linux.intel.com, kevin.tian@intel.com, ashok.raj@intel.com, tiwei.bie@intel.com, Jean-Philippe Brucker , sanjay.k.kumar@intel.com, iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, yi.y.sun@intel.com, jacob.jun.pan@intel.com, kvm@vger.kernel.org To: Xu Zaibo , Joerg Roedel , David Woodhouse , Alex Williamson , Kirti Wankhede Return-path: In-Reply-To: <5BC69830.707@huawei.com> Content-Language: en-US Sender: linux-kernel-owner@vger.kernel.org List-Id: kvm.vger.kernel.org Hi, On 10/17/18 10:02 AM, Xu Zaibo wrote: > Hi, > > On 2018/10/16 9:21, Lu Baolu wrote: >> Hi, >> >> On 10/15/2018 04:50 PM, Xu Zaibo wrote: >>> Hi, >>> >>> On 2018/10/15 10:48, Lu Baolu wrote: >>>> Hi, >>>> >>>> On 10/13/2018 04:25 PM, Xu Zaibo wrote: >>>>> Hi, >>>>> >>>>> On 2018/10/12 13:16, Lu Baolu wrote: >>>>>> Hi, >>>>>> >>>>>> The Mediate Device is a framework for fine-grained physical device >>>>>> sharing across the isolated domains. Currently the mdev framework >>>>>> is designed to be independent of the platform IOMMU support. As the >>>>>> result, the DMA isolation relies on the mdev parent device in a >>>>>> vendor specific way. >>>>>> >>>>>> There are several cases where a mediated device could be protected >>>>>> and isolated by the platform IOMMU. For example, Intel vt-d rev3.0 >>>>>> [1] introduces a new translation mode called 'scalable mode', which >>>>>> enables PASID-granular translations. The vt-d scalable mode is the >>>>>> key ingredient for Scalable I/O Virtualization [2] [3] which allows >>>>>> sharing a device in minimal possible granularity (ADI - Assignable >>>>>> Device Interface). >>>>>> >>>>>> A mediated device backed by an ADI could be protected and isolated >>>>>> by the IOMMU since 1) the parent device supports tagging an unique >>>>>> PASID to all DMA traffic out of the mediated device; and 2) the DMA >>>>>> translation unit (IOMMU) supports the PASID granular translation. >>>>>> We can apply IOMMU protection and isolation to this kind of devices >>>>>> just as what we are doing with an assignable PCI device. >>>>>> >>>>>> In order to distinguish the IOMMU-capable mediated devices from those >>>>>> which still need to rely on parent devices, this patch set adds two >>>>>> new members in struct mdev_device. >>>>>> >>>>>> * iommu_device >>>>>>    - This, if set, indicates that the mediated device could >>>>>>      be fully isolated and protected by IOMMU via attaching >>>>>>      an iommu domain to this device. If empty, it indicates >>>>>>      using vendor defined isolation. >>>>>> >>>>>> * iommu_domain >>>>>>    - This is a place holder for an iommu domain. A domain >>>>>>      could be store here for later use once it has been >>>>>>      attached to the iommu_device of this mdev. >>>>>> >>>>>> Below helpers are added to set and get above iommu device >>>>>> and iommu domain pointers in mdev core implementation. >>>>>> >>>>>> * mdev_set/get_iommu_device(dev, iommu_device) >>>>>>    - Set or get the iommu device which represents this mdev >>>>>>      in IOMMU's device scope. Drivers don't need to set the >>>>>>      iommu device if it uses vendor defined isolation. >>>>>> >>>>>> * mdev_set/get_iommu_domain(domain) >>>>>>    - A iommu domain which has been attached to the iommu >>>>>>      device in order to protect and isolate the mediated >>>>>>      device will be kept in the mdev data structure and >>>>>>      could be retrieved later. >>>>>> >>>>>> The mdev parent device driver could opt-in that the mdev could be >>>>>> fully isolated and protected by the IOMMU when the mdev is being >>>>>> created by invoking mdev_set_iommu_device() in its @create(). >>>>> I just cannot understand here, how to get an iommu_device while I >>>>> create mediated >>>>> device in my parent device driver? >>>> >>>> When you are creating an mdev in your parent driver, you should know >>>> which PCI device this mdev belonging to. >>>> >>> >>> So, generally, I can set the parent device as mdev's iommu_device? >>> If that, however, Mdev already holds its parent device. So, I just >>> figure what >>> differences between Mdev's parent device and iommu_device are. >>>>> >>>>> And why not reuse the device of MDEV instread of adding a new >>>>> device here? >>>> >>>> iommu_device in the mdev_device structure represents the PCI device >>>> that represents this mdev in iommu's device scope. IOMMU is only aware >>>> of pci devices, it's not aware of mdev device. >>> >>> Could I understand like that: IOMMU can be aware of the parent device >>> of Mdev? >>> And more, I am doubting the necessary of iommu_device in Mdev. >>> >> >> The "mdev parent device" and "mdev iommu device" are different although >> they might be the same in practice. "mdev parent device" represents the >> device who created the mdev. "mdev iommu device" represents the device >> who shares the device context entry in iommu tables. >> >> "mdev iommu device" is always a PCI/PCIe device since IOMMU always use >> source id (bus:dev:func) to walk the device context table. But there is >> no limitation on who can create an mdev, right? >> > Actually, I am not sure. > > My understanding: > The DMA address will be issued by the parent device with PASID or > something like that to IOMMU > facilities. However, the translation units such as iommu （PASID/page > .etx）tables are from another > device node.  I cannot figure out how to control this in hardware level, > or whether there will be > conflicts between the DMA transation of iommu_device and parent device. Yes. That's the reason why these two devices are same in practice. But conceptually, they might be different. Best regards, Lu Baolu