Re: [PATCH v4 11/24] iommu: Add iommu_report_device_broken() to quarantine a broken device

[Jason Gunthorpe <jgg@nvidia.com>]

On Tue, May 19, 2026 at 03:30:45PM -0700, Nicolin Chen wrote:
> On Tue, May 19, 2026 at 04:16:26PM -0300, Jason Gunthorpe wrote:
> > On Tue, May 19, 2026 at 11:29:23AM -0700, Nicolin Chen wrote:
> > > On Tue, May 19, 2026 at 09:07:37AM -0300, Jason Gunthorpe wrote:
> > > > On Mon, May 18, 2026 at 08:38:54PM -0700, Nicolin Chen wrote:
> > > Then, the core needs to block the device using the similar routine
> > > to the reset prepare(). And that needs to hold group->mutex, so it
> > > needs an async worker.
> > > 
> > > Do you see a much simpler way?
> > 
> > Put the work on the dev_iommu and forget about rcu.
> > 
> > But this is all probably better as some later series if at all. The
> > driver can block the ATS and the expectation is something will FLR the
> > device. The FLR will set the blocking and then restore the
> > domain. None of this async work seems functionally necessary, though
> > it would be a nice to have. Lets focus on the bare minimum here it, it
> > is already a difficult enough problem without tacking on these
> > extras..
> 
> OK. So you are suggesting a quarantine at the driver-level only:
> 
> 1. Driver detects ATC_INV timeout during an invalidation.
> 2. Driver retries the commands to identify the master.

I might argue to push even this out to a followup series given it is
complex and I suspect it becomes much simpler after the batch
removal...

> 3. Driver calls pci_disable_ats() and clears STE.EATS.
> 4. Driver marks domain->invs ATS entries as BROKEN.
>    (optional since pci_disable_ats() is done?)

We need to stop sending invs otherwise there will be trouble making
forward progress.

> 5. Driver sets master->ats_broken to fence concurrent attach:
>    arm_smmu_write_ste() and arm_smmu_ats_supported().

Not sure this is needed, if we race some attach then the attach will
re-set EATS, get another timeout and clear EATS. Doesn't seem worth
trying to optimize for.

> 6. Something external triggers an FLR (sysfs or AER).
> 7. FLR goes through pci_dev_reset_iommu_prepare()/done(). done()
>    reverts 3+4 and calls the reset_device_done callback clearing
>    master->ats_broken (5).

It should restore core/driver/hw synchronization of EATS and the
pci_enable_ats() by installing a blocking domain. Then it can go on to
re-attach a translating domain and everything is back to correct.

We do need to push a pci error event (didn't see that in this series)
so the driver can catch it and start the FLR process. I suppose that
will still need to bounce through a workqueue, and once you have that
it can also set the blocked domain prior to calling out to the driver.

Jason