Re: [PATCH v4 00/14] iommufd: Add vIOMMU infrastructure (Part-2: vDEVICE)

[Nicolin Chen <nicolinc@nvidia.com>]

On Fri, Oct 25, 2024 at 04:32:10PM +1100, Alexey Kardashevskiy wrote:
> External email: Use caution opening links or attachments
> 
> 
> On 25/10/24 15:59, Nicolin Chen wrote:
> > On Fri, Oct 25, 2024 at 03:54:44PM +1100, Alexey Kardashevskiy wrote:
> > > On 22/10/24 11:20, Nicolin Chen wrote:
> > > > Following the previous vIOMMU series, this adds another vDEVICE structure,
> > > > representing the association from an iommufd_device to an iommufd_viommu.
> > > > This gives the whole architecture a new "v" layer:
> > > >     _______________________________________________________________________
> > > >    |                      iommufd (with vIOMMU/vDEVICE)                    |
> > > >    |                        _____________      _____________               |
> > > >    |                       |             |    |             |              |
> > > >    |      |----------------|    vIOMMU   |<---|   vDEVICE   |<------|      |
> > > >    |      |                |             |    |_____________|       |      |
> > > >    |      |     ______     |             |     _____________     ___|____  |
> > > >    |      |    |      |    |             |    |             |   |        | |
> > > >    |      |    | IOAS |<---|(HWPT_PAGING)|<---| HWPT_NESTED |<--| DEVICE | |
> > > >    |      |    |______|    |_____________|    |_____________|   |________| |
> > > >    |______|________|______________|__________________|_______________|_____|
> > > >           |        |              |                  |               |
> > > >     ______v_____   |        ______v_____       ______v_____       ___v__
> > > >    |   struct   |  |  PFN  |  (paging)  |     |  (nested)  |     |struct|
> > > >    |iommu_device|  |------>|iommu_domain|<----|iommu_domain|<----|device|
> > > >    |____________|   storage|____________|     |____________|     |______|
> > > > 
> > > > This vDEVICE object is used to collect and store all vIOMMU-related device
> > > > information/attributes in a VM. As an initial series for vDEVICE, add only
> > > > the virt_id to the vDEVICE, which is a vIOMMU specific device ID in a VM:
> > > > e.g. vSID of ARM SMMUv3, vDeviceID of AMD IOMMU, and vID of Intel VT-d to
> > > > a Context Table. This virt_id helps IOMMU drivers to link the vID to a pID
> > > > of the device against the physical IOMMU instance. This is essential for a
> > > > vIOMMU-based invalidation, where the request contains a device's vID for a
> > > > device cache flush, e.g. ATC invalidation.
> > > > 
> > > > Therefore, with this vDEVICE object, support a vIOMMU-based invalidation,
> > > > by reusing IOMMUFD_CMD_HWPT_INVALIDATE for a vIOMMU object to flush cache
> > > > with a given driver data.
> > > > 
> > > > As for the implementation of the series, add driver support in ARM SMMUv3
> > > > for a real world use case.
> > > > 
> > > > This series is on Github:
> > > > https://github.com/nicolinc/iommufd/commits/iommufd_viommu_p2-v4
> > > > 
> > > > For testing, try this "with-rmr" branch:
> > > > https://github.com/nicolinc/iommufd/commits/iommufd_viommu_p2-v4-with-rmr
> > > 
> > > Is there any real example of a .vdevice_alloc hook, besides the
> > > selftests? It is not in iommufd_viommu_p2-v4-with-rmr, hence the
> > > question. I am trying to sketch something with this new machinery and
> > > less guessing would be nice. Thanks,
> > 
> > No, I am actually dropping that one, and moving the vdevice struct
> > to the private header, as there seems to be no use case:
> 
> Why keep it then?

We need that structure to store per-vIOMMU virtual ID. Hiding it
in the core only means we need to provide another vIOMMU APIs for
drivers to look up the ID, v.s. exposing it for drivers to access
directly.

Thanks
Nicolin