From: Robin Murphy <robin.murphy@arm.com>
To: Jason Gunthorpe <jgg@ziepe.ca>, Niklas Schnelle <schnelle@linux.ibm.com>
Cc: Joerg Roedel <joro@8bytes.org>,
Matthew Rosato <mjrosato@linux.ibm.com>,
Will Deacon <will@kernel.org>,
Wenjia Zhang <wenjia@linux.ibm.com>,
Gerd Bayer <gbayer@linux.ibm.com>,
Julian Ruess <julianr@linux.ibm.com>,
Pierre Morel <pmorel@linux.ibm.com>,
Alexandra Winter <wintera@linux.ibm.com>,
Heiko Carstens <hca@linux.ibm.com>,
Vasily Gorbik <gor@linux.ibm.com>,
Alexander Gordeev <agordeev@linux.ibm.com>,
Christian Borntraeger <borntraeger@linux.ibm.com>,
Sven Schnelle <svens@linux.ibm.com>,
Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>,
Hector Martin <marcan@marcan.st>, Sven Peter <sven@svenpeter.dev>,
Alyssa Rosenzweig <alyssa@rosenzweig.io>,
David Woodhouse <dwmw2@infradead.org>,
Lu Baolu <baolu.lu@linux.intel.com>,
Andy Gross <agross@kernel.org>,
Bjorn Andersson <andersson@kernel.org>,
Konrad Dybcio <konrad.dybcio@linaro.org>,
Yong Wu <yong.wu@mediatek.com>,
Matthias Brugger <matthias.bgg@gmail.com>,
AngeloGioacchino Del Regno
<angelogioacchino.delregno@collabora.com>,
Gerald Schaefer <gerald.schaefer@linux.ibm.com>,
Orson Zhai <orsonzhai@gmail.com>,
Baolin Wang <baolin.wang@linux.alibaba.com>,
Chunyan Zhang <zhang.lyra@gmail.com>,
Chen-Yu Tsai <wens@csie.org>,
Jernej Skrabec <jernej.skrabec@gmail.com>,
Samuel Holland <samuel@sholland.org>,
Thierry Reding <thierry.reding@gmail.com>,
Krishna Reddy <vdumpa@nvidia.com>,
Jonathan Hunter <jonathanh@nvidia.com>,
Jonathan Corbet <corbet@lwn.net>,
linux-s390@vger.kernel.org, netdev@vger.kernel.org,
linux-kernel@vger.kernel.org, iommu@lists.linux.dev,
asahi@lists.linux.dev, linux-arm-kernel@lists.infradead.org,
linux-arm-msm@vger.kernel.org,
linux-mediatek@lists.infradead.org, linux-sunxi@lists.linux.dev,
linux-tegra@vger.kernel.org, linux-doc@vger.kernel.org
Subject: Re: [PATCH v12 0/6] iommu/dma: s390 DMA API conversion and optimized IOTLB flushing
Date: Wed, 27 Sep 2023 17:16:23 +0100 [thread overview]
Message-ID: <96c0892f-20e6-db1f-f310-9b09d419aa6a@arm.com> (raw)
In-Reply-To: <20230927154009.GN13795@ziepe.ca>
On 27/09/2023 4:40 pm, Jason Gunthorpe wrote:
> On Wed, Sep 27, 2023 at 05:24:20PM +0200, Niklas Schnelle wrote:
>
>> Ok, another update. On trying it out again this problem actually also
>> occurs when applying this v12 on top of v6.6-rc3 too. Also I guess
>> unlike my prior thinking it probably doesn't occur with
>> iommu.forcedac=1 since that still allows IOVAs below 4 GiB and we might
>> be the only ones who don't support those. From my point of view this
>> sounds like a mlx5_core issue they really should call
>> dma_set_mask_and_coherent() before their first call to
>> dma_alloc_coherent() not after. So I guess I'll send a v13 of this
>> series rebased on iommu/core and with an additional mlx5 patch and then
>> let's hope we can get that merged in a way that doesn't leave us with
>> broken ConnectX VFs for too long.
>
> Yes, OK. It definitely sounds wrong that mlx5 is doing dma allocations before
> setting it's dma_set_mask_and_coherent(). Please link to this thread
> and we can get Leon or Saeed to ack it for Joerg.
>
> (though wondering why s390 is the only case that ever hit this?)
Probably because most systems happen to be able to satisfy the
allocation within the default 32-bit mask - the whole bottom 4GB of IOVA
space being reserved is pretty atypical.
TBH it makes me wonder the opposite - how this ever worked on s390
before? And I think the answer to that is "by pure chance", since upon
inspection the existing s390_pci_dma_ops implementation appears to pay
absolutely no attention to the device's DMA masks whatsoever :(
Robin.
next prev parent reply other threads:[~2023-09-27 16:16 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-08-25 10:11 [PATCH v12 0/6] iommu/dma: s390 DMA API conversion and optimized IOTLB flushing Niklas Schnelle
2023-08-25 10:11 ` [PATCH v12 1/6] iommu: Allow .iotlb_sync_map to fail and handle s390's -ENOMEM return Niklas Schnelle
2023-08-25 10:11 ` [PATCH v12 2/6] s390/pci: prepare is_passed_through() for dma-iommu Niklas Schnelle
2023-08-25 10:11 ` [PATCH v12 3/6] s390/pci: Use dma-iommu layer Niklas Schnelle
2023-08-25 10:11 ` [PATCH v12 4/6] iommu/s390: Disable deferred flush for ISM devices Niklas Schnelle
2023-08-25 18:23 ` Matthew Rosato
2023-08-25 10:11 ` [PATCH v12 5/6] iommu/dma: Allow a single FQ in addition to per-CPU FQs Niklas Schnelle
2023-09-11 12:06 ` Niklas Schnelle
2023-08-25 10:11 ` [PATCH v12 6/6] iommu/dma: Use a large flush queue and timeout for shadow_on_flush Niklas Schnelle
2023-08-25 18:26 ` [PATCH v12 0/6] iommu/dma: s390 DMA API conversion and optimized IOTLB flushing Matthew Rosato
2023-09-05 16:09 ` Robin Murphy
2023-09-25 9:56 ` Joerg Roedel
2023-09-26 15:04 ` Joerg Roedel
2023-09-26 16:08 ` Jason Gunthorpe
2023-09-27 8:55 ` Niklas Schnelle
2023-09-27 9:26 ` Robin Murphy
2023-09-27 9:55 ` Joerg Roedel
2023-09-27 11:24 ` Niklas Schnelle
2023-09-27 13:20 ` Niklas Schnelle
2023-09-27 14:31 ` Niklas Schnelle
2023-09-27 15:24 ` Niklas Schnelle
2023-09-27 15:40 ` Jason Gunthorpe
2023-09-27 16:16 ` Robin Murphy [this message]
2023-09-27 20:25 ` Matthew Rosato
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=96c0892f-20e6-db1f-f310-9b09d419aa6a@arm.com \
--to=robin.murphy@arm.com \
--cc=agordeev@linux.ibm.com \
--cc=agross@kernel.org \
--cc=alyssa@rosenzweig.io \
--cc=andersson@kernel.org \
--cc=angelogioacchino.delregno@collabora.com \
--cc=asahi@lists.linux.dev \
--cc=baolin.wang@linux.alibaba.com \
--cc=baolu.lu@linux.intel.com \
--cc=borntraeger@linux.ibm.com \
--cc=corbet@lwn.net \
--cc=dwmw2@infradead.org \
--cc=gbayer@linux.ibm.com \
--cc=gerald.schaefer@linux.ibm.com \
--cc=gor@linux.ibm.com \
--cc=hca@linux.ibm.com \
--cc=iommu@lists.linux.dev \
--cc=jernej.skrabec@gmail.com \
--cc=jgg@ziepe.ca \
--cc=jonathanh@nvidia.com \
--cc=joro@8bytes.org \
--cc=julianr@linux.ibm.com \
--cc=konrad.dybcio@linaro.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-arm-msm@vger.kernel.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mediatek@lists.infradead.org \
--cc=linux-s390@vger.kernel.org \
--cc=linux-sunxi@lists.linux.dev \
--cc=linux-tegra@vger.kernel.org \
--cc=marcan@marcan.st \
--cc=matthias.bgg@gmail.com \
--cc=mjrosato@linux.ibm.com \
--cc=netdev@vger.kernel.org \
--cc=orsonzhai@gmail.com \
--cc=pmorel@linux.ibm.com \
--cc=samuel@sholland.org \
--cc=schnelle@linux.ibm.com \
--cc=suravee.suthikulpanit@amd.com \
--cc=sven@svenpeter.dev \
--cc=svens@linux.ibm.com \
--cc=thierry.reding@gmail.com \
--cc=vdumpa@nvidia.com \
--cc=wenjia@linux.ibm.com \
--cc=wens@csie.org \
--cc=will@kernel.org \
--cc=wintera@linux.ibm.com \
--cc=yong.wu@mediatek.com \
--cc=zhang.lyra@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).