Re: [PATCH v2 04/19] iommufd: Allow pt_id to carry viommu_id for IOMMU_HWPT_ALLOC

[Nicolin Chen <nicolinc@nvidia.com>]

On Fri, Sep 27, 2024 at 02:23:16AM +0000, Tian, Kevin wrote:

> > > > > Does it mean each vIOMMU of VM can only have
> > > > > one s2 HWPT?
> > > >
> > > > Giving some examples here:
> > > >  - If a VM has 1 vIOMMU, there will be 1 vIOMMU object in the
> > > >    kernel holding one S2 HWPT.
> > > >  - If a VM has 2 vIOMMUs, there will be 2 vIOMMU objects in the
> > > >    kernel that can hold two different S2 HWPTs, or share one S2
> > > >    HWPT (saving memory).
> > > >
> > >
> > > this is not consistent with previous discussion.
> > >
> > > even for 1 vIOMMU per VM there could be multiple vIOMMU objects
> > > created in the kernel in case the devices connected to the VM-visible
> > > vIOMMU locate behind different physical SMMUs.
> > >
> > > we don't expect one vIOMMU object to span multiple physical ones.
> >
> > I think it's consistent, yet we had different perspectives for a
> > virtual IOMMU instance in the VM: Jason's suggested design for a
> > VM is to have 1-to-1 mapping between virtual IOMMU instances and
> > physical IOMMU instances. So, one vIOMMU is backed by one pIOMMU
> > only, i.e. one vIOMMU object in the kernel.
> >
> > Your case seems to be the model where a VM has one giant virtual
> > IOMMU instance backed by multiple physical IOMMUs, in which case
> > all the passthrough devices, regardless their associated pIOMMUs,
> > are connected to this shared virtual IOMMU. And yes, this shared
> > virtual IOMMU can have multiple vIOMMU objects.
> 
> yes.
> 
> sorry that I should not use "inconsistent" in the last reply. It's more
> about completeness for what the design allows. 😊

No worries. I'll add more narratives to the next version, likely
with another detailed update to the iommufd documentation. This
discussion made me realize that we need to clearly write it down.

Thanks!
Nic