Re: [PATCH RFC v2 02/11] iommu: Add iommu_group_singleton_lockdown()

From: Jason Gunthorpe via iommu <iommu@lists.linux-foundation.org>
To: Yi Liu <yi.l.liu@intel.com>
Cc: "Tian, Kevin" <kevin.tian@intel.com>,
	"Raj, Ashok" <ashok.raj@intel.com>,
	Robin Murphy <robin.murphy@arm.com>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	Christoph Hellwig <hch@infradead.org>,
	Jean-Philippe Brucker <jean-philippe@linaro.com>,
	"iommu@lists.linux-foundation.org"
	<iommu@lists.linux-foundation.org>,
	"Pan, Jacob jun" <jacob.jun.pan@intel.com>,
	Will Deacon <will@kernel.org>
Subject: Re: [PATCH RFC v2 02/11] iommu: Add iommu_group_singleton_lockdown()
Date: Fri, 1 Apr 2022 08:52:12 -0300	[thread overview]
Message-ID: <20220401115212.GJ2120790@nvidia.com> (raw)
In-Reply-To: <81ea9536-ba36-deef-b849-6b407b5409e0@intel.com>

On Fri, Apr 01, 2022 at 02:20:23PM +0800, Yi Liu wrote:
> 
> 
> On 2022/3/29 19:42, Jason Gunthorpe wrote:
> > On Tue, Mar 29, 2022 at 08:42:13AM +0000, Tian, Kevin wrote:
> > 
> > > btw I'm not sure whether this is what SVA requires. IIRC the problem with
> > > SVA is because PASID TLP prefix is not counted in PCI packet routing thus
> > > a DMA target address with PASID might be treated as P2P if the address
> > > falls into the MMIO BAR of other devices in the group. This is why the
> > > original code needs to strictly apply SVA in a group containing a single
> > > device, instead of a group attached by a single driver, unless we want to
> > > reserve those MMIO ranges in CPU VA space.
> > 
> > I think it is not such a good idea to mix up group with this test
> > 
> > Here you want to say that all TLPs from the RID route to the host
> > bridge - ie ACS is on/etc. This is subtly different from a group with
> > a single device. Specifically it is an immutable property of the
> > fabric and doesn't change after hot plug events.
> 
> so the group size can be immutable for specific topology. right? I think for
> non-multi-function devices plugged behind an PCIE bridge which has enabled
> ACS, such devices should have their own groups. Under such topology the
> group size should be 1 constantly. May just enable SVA for such devices.

Like I said, you should stop thinking about group size.

You need to know that 100% of TLPs translate through the IOMMU to
enable SVA, nothing less will do, and that property has nothing to do
with group size.

> > ie if we have a singleton group that doesn't have ACS and someone
> > hotplugs in another device on a bridge, then our SVA is completely
> > broken and we get data corruption.
> 
> I think this may be a device plugged in a PCIE-to-PCI bridge, and then
> hotplug a device to this bridge. The group size is variable. right? Per my
> understanding, maybe such a bridge cannot support PASID Prefix at all, hence
> no SVA support for such devices.

Any PCIE-to-PCIE bridge will do, don't ned to involve legacy PCI here
to have hotplug problems.

Jason
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu