Re: [PATCH v2 14/18] iommu/arm-smmu-v3: Support domains with shared CDs

From: Jason Gunthorpe <jgg@nvidia.com>
To: Baolu Lu <baolu.lu@linux.intel.com>
Cc: Michael Shavit <mshavit@google.com>,
	Will Deacon <will@kernel.org>,
	Robin Murphy <robin.murphy@arm.com>,
	Joerg Roedel <joro@8bytes.org>,
	jean-philippe@linaro.org, nicolinc@nvidia.com,
	linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH v2 14/18] iommu/arm-smmu-v3: Support domains with shared CDs
Date: Thu, 8 Jun 2023 10:39:28 -0300	[thread overview]
Message-ID: <ZIHaEFJFVQqvJvmU@nvidia.com> (raw)
In-Reply-To: <f0a691fa-d050-f457-9c8d-0ae340eab58f@linux.intel.com>

On Thu, Jun 08, 2023 at 10:39:23AM +0800, Baolu Lu wrote:
> On 6/7/23 7:59 PM, Jason Gunthorpe wrote:
> > On Wed, Jun 07, 2023 at 12:06:07AM +0530, Michael Shavit wrote:
> > > > What we definately shouldn't do is try to have different SVA
> > > > iommu_domain's pointing at the same ASID. That is again making SVA
> > > > special, which we are trying to get away from 😄
> > > Fwiw, this change is preserving the status-quo in that regard;
> > > arm-smmu-v3-sva.c is already doing this. But yes, I agree that
> > > resolving the limitation is a better long term solution... and
> > > something I can try to look at further.
> > I suppose we also don't really have a entirely clear picture what
> > allocating multiple SVA domains should even do in the iommu driver.
> > 
> > The driver would like to share the ASID, but things are much cleaner
> > for everything if the driver model has ASID 1:1 with the iommu_domain.
> 
> This means that each ASID should be mapped to a single IOMMU domain.
> This is conceptually right as iommu_domain represents a hardware page
> table. For SVA, it's an mm_struct.
> 
> So in my mind, each sva_domain should have a 1:1 relationship with an
> mm_struct.

If we want to support multiple iommu drivers then we should support
multiple iommu_domains per mm_struct so that each driver can have its
own. In this world if each instance wants its own iommu_domain it is
not a big deal.

Drivers that can share iommu_domains across instances should probably
also share sva iommu_domains across instances.

Most real systems have only one iommu driver and we'd like the good
iommu drivers to be able to share domains across instances, so we'd
expect only 1 iommu_domain per mm struct.

Jason