From: Niklas Schnelle <schnelle@linux.ibm.com>
To: Joerg Roedel <joro@8bytes.org>,
Matthew Rosato <mjrosato@linux.ibm.com>,
Will Deacon <will@kernel.org>,
Wenjia Zhang <wenjia@linux.ibm.com>,
Robin Murphy <robin.murphy@arm.com>,
Jason Gunthorpe <jgg@ziepe.ca>
Cc: Gerd Bayer <gbayer@linux.ibm.com>,
Julian Ruess <julianr@linux.ibm.com>,
Pierre Morel <pmorel@linux.ibm.com>,
Alexandra Winter <wintera@linux.ibm.com>,
Heiko Carstens <hca@linux.ibm.com>,
Vasily Gorbik <gor@linux.ibm.com>,
Alexander Gordeev <agordeev@linux.ibm.com>,
Christian Borntraeger <borntraeger@linux.ibm.com>,
Sven Schnelle <svens@linux.ibm.com>,
Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>,
Hector Martin <marcan@marcan.st>, Sven Peter <sven@svenpeter.dev>,
Alyssa Rosenzweig <alyssa@rosenzweig.io>,
David Woodhouse <dwmw2@infradead.org>,
Lu Baolu <baolu.lu@linux.intel.com>,
Andy Gross <agross@kernel.org>,
Bjorn Andersson <andersson@kernel.org>,
Konrad Dybcio <konrad.dybcio@linaro.org>,
Yong Wu <yong.wu@mediatek.com>,
Matthias Brugger <matthias.bgg@gmail.com>,
AngeloGioacchino Del Regno
<angelogioacchino.delregno@collabora.com>,
Gerald Schaefer <gerald.schaefer@linux.ibm.com>,
Orson Zhai <orsonzhai@gmail.com>,
Baolin Wang <baolin.wang@linux.alibaba.com>,
Chunyan Zhang <zhang.lyra@gmail.com>,
Chen-Yu Tsai <wens@csie.org>,
Jernej Skrabec <jernej.skrabec@gmail.com>,
Samuel Holland <samuel@sholland.org>,
Thierry Reding <thierry.reding@gmail.com>,
Krishna Reddy <vdumpa@nvidia.com>,
Jonathan Hunter <jonathanh@nvidia.com>,
Niklas Schnelle <schnelle@linux.ibm.com>,
Jonathan Corbet <corbet@lwn.net>,
linux-s390@vger.kernel.org, netdev@vger.kernel.org,
linux-kernel@vger.kernel.org, iommu@lists.linux.dev,
asahi@lists.linux.dev, linux-arm-kernel@lists.infradead.org,
linux-arm-msm@vger.kernel.org,
linux-mediatek@lists.infradead.org, linux-sunxi@lists.linux.dev,
linux-tegra@vger.kernel.org, linux-doc@vger.kernel.org
Subject: [PATCH v12 4/6] iommu/s390: Disable deferred flush for ISM devices
Date: Fri, 25 Aug 2023 12:11:19 +0200 [thread overview]
Message-ID: <20230825-dma_iommu-v12-4-4134455994a7@linux.ibm.com> (raw)
In-Reply-To: <20230825-dma_iommu-v12-0-4134455994a7@linux.ibm.com>
ISM devices are virtual PCI devices used for cross-LPAR communication.
Unlike real PCI devices ISM devices do not use the hardware IOMMU but
inspects IOMMU translation tables directly on IOTLB flush (s390 RPCIT
instruction).
ISM devices keep their DMA allocations static and only very rarely DMA
unmap at all. For each IOTLB flush that occurs after unmap the ISM
devices will however inspect the area of the IOVA space indicated by the
flush. This means that for the global IOTLB flushes used by the flush
queue mechanism the entire IOVA space would be inspected. In principle
this would be fine, albeit potentially unnecessarily slow, it turns out
however that ISM devices are sensitive to seeing IOVA addresses that are
currently in use in the IOVA range being flushed. Seeing such in-use
IOVA addresses will cause the ISM device to enter an error state and
become unusable.
Fix this by claiming IOMMU_CAP_DEFERRED_FLUSH only for non-ISM devices.
This makes sure IOTLB flushes only cover IOVAs that have been unmapped
and also restricts the range of the IOTLB flush potentially reducing
latency spikes.
Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
---
drivers/iommu/s390-iommu.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/iommu/s390-iommu.c b/drivers/iommu/s390-iommu.c
index f6d6c60e5634..8310180a102c 100644
--- a/drivers/iommu/s390-iommu.c
+++ b/drivers/iommu/s390-iommu.c
@@ -315,11 +315,13 @@ static struct s390_domain *to_s390_domain(struct iommu_domain *dom)
static bool s390_iommu_capable(struct device *dev, enum iommu_cap cap)
{
+ struct zpci_dev *zdev = to_zpci_dev(dev);
+
switch (cap) {
case IOMMU_CAP_CACHE_COHERENCY:
return true;
case IOMMU_CAP_DEFERRED_FLUSH:
- return true;
+ return zdev->pft != PCI_FUNC_TYPE_ISM;
default:
return false;
}
--
2.39.2
next prev parent reply other threads:[~2023-08-25 10:13 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-08-25 10:11 [PATCH v12 0/6] iommu/dma: s390 DMA API conversion and optimized IOTLB flushing Niklas Schnelle
2023-08-25 10:11 ` [PATCH v12 1/6] iommu: Allow .iotlb_sync_map to fail and handle s390's -ENOMEM return Niklas Schnelle
2023-08-25 10:11 ` [PATCH v12 2/6] s390/pci: prepare is_passed_through() for dma-iommu Niklas Schnelle
2023-08-25 10:11 ` [PATCH v12 3/6] s390/pci: Use dma-iommu layer Niklas Schnelle
2023-08-25 10:11 ` Niklas Schnelle [this message]
2023-08-25 18:23 ` [PATCH v12 4/6] iommu/s390: Disable deferred flush for ISM devices Matthew Rosato
2023-08-25 10:11 ` [PATCH v12 5/6] iommu/dma: Allow a single FQ in addition to per-CPU FQs Niklas Schnelle
2023-09-11 12:06 ` Niklas Schnelle
2023-08-25 10:11 ` [PATCH v12 6/6] iommu/dma: Use a large flush queue and timeout for shadow_on_flush Niklas Schnelle
2023-08-25 18:26 ` [PATCH v12 0/6] iommu/dma: s390 DMA API conversion and optimized IOTLB flushing Matthew Rosato
2023-09-05 16:09 ` Robin Murphy
2023-09-25 9:56 ` Joerg Roedel
2023-09-26 15:04 ` Joerg Roedel
2023-09-26 16:08 ` Jason Gunthorpe
2023-09-27 8:55 ` Niklas Schnelle
2023-09-27 9:26 ` Robin Murphy
2023-09-27 9:55 ` Joerg Roedel
2023-09-27 11:24 ` Niklas Schnelle
2023-09-27 13:20 ` Niklas Schnelle
2023-09-27 14:31 ` Niklas Schnelle
2023-09-27 15:24 ` Niklas Schnelle
2023-09-27 15:40 ` Jason Gunthorpe
2023-09-27 16:16 ` Robin Murphy
2023-09-27 20:25 ` Matthew Rosato
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230825-dma_iommu-v12-4-4134455994a7@linux.ibm.com \
--to=schnelle@linux.ibm.com \
--cc=agordeev@linux.ibm.com \
--cc=agross@kernel.org \
--cc=alyssa@rosenzweig.io \
--cc=andersson@kernel.org \
--cc=angelogioacchino.delregno@collabora.com \
--cc=asahi@lists.linux.dev \
--cc=baolin.wang@linux.alibaba.com \
--cc=baolu.lu@linux.intel.com \
--cc=borntraeger@linux.ibm.com \
--cc=corbet@lwn.net \
--cc=dwmw2@infradead.org \
--cc=gbayer@linux.ibm.com \
--cc=gerald.schaefer@linux.ibm.com \
--cc=gor@linux.ibm.com \
--cc=hca@linux.ibm.com \
--cc=iommu@lists.linux.dev \
--cc=jernej.skrabec@gmail.com \
--cc=jgg@ziepe.ca \
--cc=jonathanh@nvidia.com \
--cc=joro@8bytes.org \
--cc=julianr@linux.ibm.com \
--cc=konrad.dybcio@linaro.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-arm-msm@vger.kernel.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mediatek@lists.infradead.org \
--cc=linux-s390@vger.kernel.org \
--cc=linux-sunxi@lists.linux.dev \
--cc=linux-tegra@vger.kernel.org \
--cc=marcan@marcan.st \
--cc=matthias.bgg@gmail.com \
--cc=mjrosato@linux.ibm.com \
--cc=netdev@vger.kernel.org \
--cc=orsonzhai@gmail.com \
--cc=pmorel@linux.ibm.com \
--cc=robin.murphy@arm.com \
--cc=samuel@sholland.org \
--cc=suravee.suthikulpanit@amd.com \
--cc=sven@svenpeter.dev \
--cc=svens@linux.ibm.com \
--cc=thierry.reding@gmail.com \
--cc=vdumpa@nvidia.com \
--cc=wenjia@linux.ibm.com \
--cc=wens@csie.org \
--cc=will@kernel.org \
--cc=wintera@linux.ibm.com \
--cc=yong.wu@mediatek.com \
--cc=zhang.lyra@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).