public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed
From: Jason Gunthorpe <jgg@nvidia.com>
To: Robin Murphy <robin.murphy@arm.com>
Cc: Nicolin Chen <nicolinc@nvidia.com>,
	dan.j.williams@intel.com, "Tian, Kevin" <kevin.tian@intel.com>,
	Jonathan Cameron <jonathan.cameron@huawei.com>,
	"will@kernel.org" <will@kernel.org>,
	"bhelgaas@google.com" <bhelgaas@google.com>,
	"joro@8bytes.org" <joro@8bytes.org>,
	"praan@google.com" <praan@google.com>,
	"baolu.lu@linux.intel.com" <baolu.lu@linux.intel.com>,
	"miko.lenczewski@arm.com" <miko.lenczewski@arm.com>,
	"linux-arm-kernel@lists.infradead.org"
	<linux-arm-kernel@lists.infradead.org>,
	"iommu@lists.linux.dev" <iommu@lists.linux.dev>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
	"linux-cxl@vger.kernel.org" <linux-cxl@vger.kernel.org>
Subject: Re: [PATCH RFCv1 1/3] PCI: Allow ATS to be always on for CXL.cache capable devices
Date: Fri, 20 Feb 2026 09:51:24 -0400	[thread overview]
Message-ID: <20260220135124.GA1894399@nvidia.com> (raw)
In-Reply-To: <6a842339-3f0a-48db-830a-326add917519@arm.com>

On Fri, Feb 20, 2026 at 01:22:49PM +0000, Robin Murphy wrote:

> But is that an issue? Until the device has a driver, surely it shouldn't be
> expected to send interrupts at all, much less depend on them being received
> and understood by Linux? The MSI cookie is only populated once a driver
> actually requests some MSI vectors (since it doesn't know what ITS
> address(es) may or may not need mapping), so an empty DMA domain is still no
> better than a true blocking domain in this regard anyway.

Oh, the issue is the driver_managed_dma flag.

In this mode we do bind a driver but the iommu callbacks at driver
bind are not called anymore because that flag says the driver itself
will call them later.

Things like PCI port driver that never issue DMA at all will set the
flag and never make any calls, while still expecting interrupts to
work.

This is why the other option is to rework this somewhat so these
drivers still make call in to the iommu and can get an interrupt
setup.

All of this is only for multi-device groups where we want to ignore
some bad grouping with VFIO on old HW without sufficient ACS. Thinking
about it some more I suspect this entire concept has been broken from
day 1 in VFIO on ARM. If the iommu_group has two members, port driver
and a VFIO device then:

 The port driver will start first, install the ITS page in the DMA
 domain, VFIO will start second an switch the domain to BLOCKED, then
 to PAGING, and the ITS mapping used by the port driver will be lost.

And nobody will notice this has happened because the interrupts in the
port driver are only used for RAS IIRC so the net effect is your
system doesn't print AERs anymore.

Jason

  reply	other threads:[~2026-02-20 13:51 UTC|newest]

Thread overview: 60+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-01-17  4:56 [PATCH RFCv1 0/3] Allow ATS to be always on for certain ATS-capable devices Nicolin Chen
2026-01-17  4:56 ` [PATCH RFCv1 1/3] PCI: Allow ATS to be always on for CXL.cache capable devices Nicolin Chen
2026-01-19 17:58   ` Jason Gunthorpe
2026-01-21  8:01   ` Tian, Kevin
2026-01-21 10:03     ` Jonathan Cameron
2026-01-21 13:03       ` Jason Gunthorpe
2026-01-22  1:17         ` Baolu Lu
2026-01-22 13:15           ` Jason Gunthorpe
2026-01-22  5:44         ` dan.j.williams
2026-01-22 13:14           ` Jason Gunthorpe
2026-01-22 16:29             ` Nicolin Chen
2026-01-22 16:58               ` Jason Gunthorpe
2026-01-22 19:46             ` dan.j.williams
2026-01-27  8:10               ` Tian, Kevin
2026-01-27 15:04                 ` Jason Gunthorpe
2026-01-28  0:49                   ` dan.j.williams
2026-01-28 13:05                     ` Jason Gunthorpe
2026-02-03  5:13                       ` Nicolin Chen
2026-02-03 14:33                         ` Jason Gunthorpe
2026-02-03 17:45                           ` Nicolin Chen
2026-02-03 17:55                             ` Jason Gunthorpe
2026-02-03 18:50                               ` Nicolin Chen
2026-02-04 13:21                                 ` Jason Gunthorpe
2026-02-03 18:59                               ` Robin Murphy
2026-02-03 19:24                                 ` Nicolin Chen
2026-02-03 23:16                                 ` Jason Gunthorpe
2026-02-04 12:18                                   ` Robin Murphy
2026-02-04 13:20                                     ` Jason Gunthorpe
2026-02-18 22:56                               ` Nicolin Chen
2026-02-19 14:37                                 ` Jason Gunthorpe
2026-02-19 16:53                                   ` Nicolin Chen
2026-02-19 17:41                                     ` Jason Gunthorpe
2026-02-20  4:52                                       ` Nicolin Chen
2026-02-20 12:50                                         ` Jason Gunthorpe
2026-02-20 13:22                                           ` Robin Murphy
2026-02-20 13:51                                             ` Jason Gunthorpe [this message]
2026-02-20 14:45                                               ` Robin Murphy
2026-02-26 15:10                                                 ` Jason Gunthorpe
2026-02-20 18:49                                           ` Nicolin Chen
2026-02-24 14:38                                             ` Jason Gunthorpe
2026-01-28  0:57                   ` Tian, Kevin
2026-01-28 13:11                     ` Jason Gunthorpe
2026-01-29  3:28                       ` Tian, Kevin
2026-01-22 10:24         ` Alejandro Lucero Palau
2026-01-17  4:56 ` [PATCH RFCv1 2/3] PCI: Allow ATS to be always on for non-CXL NVIDIA GPUs Nicolin Chen
2026-01-19 18:00   ` Jason Gunthorpe
2026-01-19 18:09     ` Nicolin Chen
2026-01-17  4:56 ` [PATCH RFCv1 3/3] iommu/arm-smmu-v3: Allow ATS to be always on Nicolin Chen
2026-01-19 20:06   ` Jason Gunthorpe
2026-01-26 12:39   ` Will Deacon
2026-01-26 17:20     ` Jason Gunthorpe
2026-01-26 18:40       ` Nicolin Chen
2026-01-26 19:16         ` Jason Gunthorpe
2026-01-26 18:49       ` Robin Murphy
2026-01-26 19:09         ` Jason Gunthorpe
2026-01-27 13:10           ` Will Deacon
2026-01-27 13:26             ` Robin Murphy
2026-01-27 13:50               ` Will Deacon
2026-01-27 14:49                 ` Jason Gunthorpe
2026-01-26 18:21     ` Nicolin Chen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260220135124.GA1894399@nvidia.com \
    --to=jgg@nvidia.com \
    --cc=baolu.lu@linux.intel.com \
    --cc=bhelgaas@google.com \
    --cc=dan.j.williams@intel.com \
    --cc=iommu@lists.linux.dev \
    --cc=jonathan.cameron@huawei.com \
    --cc=joro@8bytes.org \
    --cc=kevin.tian@intel.com \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-cxl@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-pci@vger.kernel.org \
    --cc=miko.lenczewski@arm.com \
    --cc=nicolinc@nvidia.com \
    --cc=praan@google.com \
    --cc=robin.murphy@arm.com \
    --cc=will@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox