Re: [PATCH v1 3/3] iommu/amd: Optimize the IOMMU queue flush

[Joerg Roedel <joro-zLv9SwRftAIdnm+yROfE0A@public.gmane.org>]

On Wed, Jun 21, 2017 at 05:09:31PM -0400, Jan Vesely wrote:
> On Wed, 2017-06-21 at 12:01 -0500, Tom Lendacky wrote:
> > On 6/21/2017 11:20 AM, Jan Vesely wrote:
> > > Hi Arindam,
> > > 
> > > has this patch been replaced by Joerg's "[PATCH 0/7] iommu/amd:
> > > Optimize iova queue flushing" series?
> > 
> > Yes, Joerg's patches replaced this patch.  He applied just the first two
> > patches of this series.
> 
> Joerg's patches applied on top of 4.10.17 do not solve my issue (do I
> need the first two patches of this series?). the machine still hangs on
> boot with a flood of IOMMU wait loop timed out messages.
> 
> on the other hand patch 3/3 v1 applied on top of 4.10.17 fixes the
> problem and the machine boots successfully

Interesting. I did some measurements on the IOTLB flush-rate with my
network load-test. This test is designed to heavily excerise the IOMMU
map/unmap path and thus cause many IOTLB invalidations too.

Results are:

	Current upstream v4.12-rc6:	~147000 flushes/s
	With Toms patches:		  ~5900 flushes/s
	With my patches:		  ~1200 flushes/s

So while Toms patches also get the flush-rate down significantly, it is
even lower with my patches. This indicates that the problem is
triggerable even with low flush rates.

But I have no idea why it still triggers with my patches, but not with
Toms. The approaches follow the same idea of only flushing domains that
have map/unmap operations on them.

I really think we need the patch to blacklist ATS on these GPUs
upstream.

Regards,

	Joerg