Re: [PATCH rc] iommu/arm-smmu-v3: Drain in-flight fault handlers

[Will Deacon <will@kernel.org>]

On Fri, Mar 06, 2026 at 04:17:23PM -0800, Nicolin Chen wrote:
> From: Malak Marrid <mmarrid@nvidia.com>
> 
> When a device is switching away from a domain, either through a detach or a
> replace operation, it must drain its IOPF queue that only contains the page
> requests for the old domain.
> 
> Currently, the IOPF infrastructure is used by master->stall_enabled. So the
> stalled transaction for the old domain should be resumed/terminated. Fix it
> properly.
> 
> Fixes: cfea71aea921 ("iommu/arm-smmu-v3: Put iopf enablement in the domain attach path")
> Cc: stable@vger.kernel.org
> Co-developed-by: Barak Biber <bbiber@nvidia.com>
> Signed-off-by: Barak Biber <bbiber@nvidia.com>
> Co-developed-by: Stefan Kaestle <skaestle@nvidia.com>
> Signed-off-by: Stefan Kaestle <skaestle@nvidia.com>
> Signed-off-by: Malak Marrid <mmarrid@nvidia.com>
> Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
> ---
>  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 11 ++++++++++-
>  1 file changed, 10 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
> index 4d00d796f0783..2176ee8bec767 100644
> --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
> +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
> @@ -2843,6 +2843,12 @@ static int arm_smmu_enable_iopf(struct arm_smmu_master *master,
>  	if (master->iopf_refcount) {
>  		master->iopf_refcount++;
>  		master_domain->using_iopf = true;
> +		/*
> +		 * If the device is already on the IOPF queue (domain replace),
> +		 * drain in-flight fault handlers so nothing will hold the old
> +		 * domain when the core switches the attach handle.
> +		 */
> +		iopf_queue_flush_dev(master->dev);

So this drains the iopf workqueue, but don't you still have a race with
the hardware generating a fault on the old domain and then that only
showing up once you've switched to the new one? What is the actual
problem you're trying to solve with this patch?

>  		return 0;
>  	}
>  
> @@ -2866,8 +2872,11 @@ static void arm_smmu_disable_iopf(struct arm_smmu_master *master,
>  		return;
>  
>  	master->iopf_refcount--;
> -	if (master->iopf_refcount == 0)
> +	if (master->iopf_refcount == 0) {
> +		/* Drain in-flight fault handlers before removing device */
> +		iopf_queue_flush_dev(master->dev);
>  		iopf_queue_remove_device(master->smmu->evtq.iopf, master->dev);

Why doesn't iopf_queue_remove_device() handle the draining? Is there a
case where you _don't_ want to drain the faults on the disable path?

Will