Re: [PATCH v6 00/20] Add VFIO mediated device support and DEV-MSI support for the idxd driver

Dave Jiang
2021-06-11T18:21:52Z

On 6/8/2021 9:02 AM, Dave Jiang wrote:
>
> On 6/7/2021 12:11 PM, Jason Gunthorpe wrote:
>> On Mon, Jun 07, 2021 at 11:13:04AM -0700, Dave Jiang wrote:
>>
>>> So in step 1, we 'tag' the wq to be dedicated to guest usage and put
>>> the
>>> hardware wq into enable state. For a dedicated mode wq, we can
>>> definitely
>>> just register directly and skip the mdev step. For a shared wq mode,
>>> we can
>>> have multiple mdev running on top of a single wq. So we need some
>>> way to
>>> create more mdevs. We can either go with the existing established
>>> creation
>>> path by mdev, or invent something custom for the driver as Jason
>>> suggested
>>> to accomodate additional virtual devices for guests. We implemented
>>> the mdev
>>> path originally with consideration of mdev is established and has a
>>> known
>>> interface already.
>> It sounds like you could just as easially have a 'create new vfio'
>> file under the idxd sysfs.. Especially since you already have a bus
>> and dynamic vfio specific things being created on this bus.
>
> Will explore this and using of 'struct vfio_device' without mdev.
>
Hi Jason. I hacked the idxd driver to remove mdev association and use
vfio_device directly. Ran into some issues. Specifically mdev does some
special handling when it comes to iommu domain. When we hit
vfio_iommu_type1_attach_group(), there's a branch in there for
mdev_bus_type. It sets the group with mdev_group flag, which later has
effect of special handling for iommu_attach_group. And in addition, it
ends up switching the bus to pci_bus_type before iommu_domain_alloc() is
called.  Do we need to provide similar type of handling for vfio_device
that are not backed by an entire PCI device like vfio_pci? Not sure it's
the right thing to do to attach these devices to pci_bus_type directly.