Re: [PATCH v4 00/14] iommufd: Add vIOMMU infrastructure (Part-2: vDEVICE)

[Nicolin Chen <nicolinc@nvidia.com>]

On Fri, Oct 25, 2024 at 03:54:44PM +1100, Alexey Kardashevskiy wrote:
> On 22/10/24 11:20, Nicolin Chen wrote:
> > Following the previous vIOMMU series, this adds another vDEVICE structure,
> > representing the association from an iommufd_device to an iommufd_viommu.
> > This gives the whole architecture a new "v" layer:
> >    _______________________________________________________________________
> >   |                      iommufd (with vIOMMU/vDEVICE)                    |
> >   |                        _____________      _____________               |
> >   |                       |             |    |             |              |
> >   |      |----------------|    vIOMMU   |<---|   vDEVICE   |<------|      |
> >   |      |                |             |    |_____________|       |      |
> >   |      |     ______     |             |     _____________     ___|____  |
> >   |      |    |      |    |             |    |             |   |        | |
> >   |      |    | IOAS |<---|(HWPT_PAGING)|<---| HWPT_NESTED |<--| DEVICE | |
> >   |      |    |______|    |_____________|    |_____________|   |________| |
> >   |______|________|______________|__________________|_______________|_____|
> >          |        |              |                  |               |
> >    ______v_____   |        ______v_____       ______v_____       ___v__
> >   |   struct   |  |  PFN  |  (paging)  |     |  (nested)  |     |struct|
> >   |iommu_device|  |------>|iommu_domain|<----|iommu_domain|<----|device|
> >   |____________|   storage|____________|     |____________|     |______|
> > 
> > This vDEVICE object is used to collect and store all vIOMMU-related device
> > information/attributes in a VM. As an initial series for vDEVICE, add only
> > the virt_id to the vDEVICE, which is a vIOMMU specific device ID in a VM:
> > e.g. vSID of ARM SMMUv3, vDeviceID of AMD IOMMU, and vID of Intel VT-d to
> > a Context Table. This virt_id helps IOMMU drivers to link the vID to a pID
> > of the device against the physical IOMMU instance. This is essential for a
> > vIOMMU-based invalidation, where the request contains a device's vID for a
> > device cache flush, e.g. ATC invalidation.
> > 
> > Therefore, with this vDEVICE object, support a vIOMMU-based invalidation,
> > by reusing IOMMUFD_CMD_HWPT_INVALIDATE for a vIOMMU object to flush cache
> > with a given driver data.
> > 
> > As for the implementation of the series, add driver support in ARM SMMUv3
> > for a real world use case.
> > 
> > This series is on Github:
> > https://github.com/nicolinc/iommufd/commits/iommufd_viommu_p2-v4
> > 
> > For testing, try this "with-rmr" branch:
> > https://github.com/nicolinc/iommufd/commits/iommufd_viommu_p2-v4-with-rmr
> 
> Is there any real example of a .vdevice_alloc hook, besides the
> selftests? It is not in iommufd_viommu_p2-v4-with-rmr, hence the
> question. I am trying to sketch something with this new machinery and
> less guessing would be nice. Thanks,

No, I am actually dropping that one, and moving the vdevice struct
to the private header, as there seems to be no use case:
https://lore.kernel.org/linux-iommu/ZxsSYbK3gqyC84U7@Asurada-Nvidia/
https://lore.kernel.org/linux-iommu/ZxsTAANTTuQzQ9HR@Asurada-Nvidia/

Do you need vdevice_alloc in the driver for your sketch?

Thanks
Nicolin