From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ACB5F30BF67 for ; Wed, 18 Mar 2026 22:02:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.172 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773871359; cv=none; b=hrCOJvZmAxTo23tF2C2N/yXVi4xqa5ftvCl9bQVSSXk8by6CPsGWoz0noJd+Ms17ItVYwY+wxMwCULCO8pUcgOup6DqW/ZPUqd5GatZPUNRAcxJ8aJxuB6BNJF5t0hqnZ3pSVhZXVjfuSm9MJbCMY/LDVT91qafkKyHzCo3Ow4o= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773871359; c=relaxed/simple; bh=HqUIH1+qdjXuqXvG79xkAqR86UdDeaKrxocv3JQHwGg=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=BUYp0dOZ2qtoKOxhcO9pBG9beWbkMGPosFKQZ9MH5XoCilJxH7AzNa6LnuhiiEI5QyYTBc9yloIY4rvsw9eTwrMsVN8wej0Nyto1aw0ZOXHTZ4Wc7UJIzzpkH5zIzPlc89Uuv0JjZXN93b2pGKMeoHkdbxODa18cvPVNE9RFG44= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=eh2nmujY; arc=none smtp.client-ip=209.85.214.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="eh2nmujY" Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-2b052562254so29675ad.0 for ; Wed, 18 Mar 2026 15:02:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1773871358; x=1774476158; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=KWL0RuLJN0YCYwd7OUvYKgvM4/v9ZFsPnPl8Rwt0VYI=; b=eh2nmujY1+JiEXO20gy7et8TPWw+HZaW/edYuU5H1qHbdBauEFxmzOWxAxs1n3MYxy TFSLelX8u8eJ1StTYqhmScSOCo1qJY6v0muq1igV0xGPkNUbLjffsPFHO8ytIDeueGqn TKkiXT2LYVWRulTnlh+jFH8FWICssTxFi8RcVBFvI/wE9XErIENlllAqehyZPuyqhLZk l3im/ln6jujOA6Mgmu5Yl953SN/uHKZX7x3yDfP2nTrwy37687qPVIUc9WAq5OFr7H5O 1JSlCkAHywNtGz2Uup/MAVTJR03CSxqvrt+ykz8XVshL3alqvdmEyqHKhojRUI8kM+Tn A4MA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1773871358; x=1774476158; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KWL0RuLJN0YCYwd7OUvYKgvM4/v9ZFsPnPl8Rwt0VYI=; b=KhTxuWXENvJcv0je8lQDjE7TuCZ4Tf/uW02iEtg1PLsNolTDBUxB7SpYtKAfuFsWuB duwSmVbVrhwe0uDt4mmDLlBGdmFRVNzSQEFkteJ1yVs5spLd+6tCUQRZmhsO0HQCYIEw jpB/6AmXacGF2yUeZmQeHAFPdTwvg3ZFV1CmlHDhq6/K3lpr8AFKQJldKR4YmmTU39z0 TsG2jT5zcFXe3DnPnYbMQ7xJav5cNJFPIH5pFyVdOho4775OdCYM5cHb7Rhba6p4FSQM 4BPRGhT2YhM8cNrNUBkxYPYNc6RvOw3G+dVohAlSNp9d0ZMAIUIYWJzxP4czZcN7MUri 1pTA== X-Forwarded-Encrypted: i=1; AJvYcCWaMmQo5ADwDtNzoZ7HV57Qvg2T3u49NkPXfY3XZL22g02ZMqYX3QlUTatmBA42lV8d7oLuaLo56xmR@vger.kernel.org X-Gm-Message-State: AOJu0Yyz98ghJRm+7hq2qr1X3SNumzV5praReNuNnEom6tlzMLid8SVF wQ/DdNTpM7u2zRpbX10CPKSWlC3imioVVwjLx0vBp5ReTjxZur6eXwqmMaHTVltaSg== X-Gm-Gg: ATEYQzzOc+YEzPhqKKdIzD1vhIJT3bkpBohU1lYZhlytF/RJYYbBPhA/UOr40iZRBtc AYQgnN/ll5xE5XywUvNaf7JBdbocWglxLgyoMxwSYm8VFZKa+wHUXgv6uWqWqU5vgB8XOCdjKdf iqrWaCD6/quf/F4Igj7xJ7F8VysXy7Rq8tEZoZM6ytD6yQmQP21NCV68u8hIh9nIVQsL4QE5A4H Z9o5bg2WWE3PNaBquMAivTOtHwp2Ugj7Md7El8LAX2Z8HH0mW9CZwz3SpqaoticazRCKd7wfF/x /rieKmt3KazXXOQJTqY7U0X4x/MxHh3qiuEObO+/HfA1kAJ4fQa+jQp7E0EY1IDT5XD3tfJ/rCv Fhe2ipvDsd3zNv+TKvMNGNkZmy+n2qjjZrLmV+CiXOys8WgfGr08qZL6VF9rVNqmowE/1Fbz7RC MRfKJA8E29EedibMmjoTeAy7QIjdmarJBQrztwIUZT1+H4Yp8KPc2J3woPthehGg== X-Received: by 2002:a17:902:ebc5:b0:2a7:4151:2c74 with SMTP id d9443c01a7336-2b077df2de7mr1074425ad.16.1773871357349; Wed, 18 Mar 2026 15:02:37 -0700 (PDT) Received: from google.com (168.136.83.34.bc.googleusercontent.com. [34.83.136.168]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-35bc62e4b53sm576347a91.7.2026.03.18.15.02.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Mar 2026 15:02:36 -0700 (PDT) Date: Wed, 18 Mar 2026 22:02:32 +0000 From: Samiullah Khawaja To: Nicolin Chen Cc: will@kernel.org, robin.murphy@arm.com, joro@8bytes.org, bhelgaas@google.com, jgg@nvidia.com, rafael@kernel.org, lenb@kernel.org, praan@google.com, baolu.lu@linux.intel.com, xueshuai@linux.alibaba.com, kevin.tian@intel.com, linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-acpi@vger.kernel.org, linux-pci@vger.kernel.org, vsethi@nvidia.com Subject: Re: [PATCH v2 4/7] iommu/arm-smmu-v3: Mark ATC invalidate timeouts via lockless bitmap Message-ID: References: <0c5525367cc67ccc84a675544d1d9f8462704065.1773774441.git.nicolinc@nvidia.com> Precedence: bulk X-Mailing-List: linux-acpi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: <0c5525367cc67ccc84a675544d1d9f8462704065.1773774441.git.nicolinc@nvidia.com> Hi Nicolin, On Tue, Mar 17, 2026 at 12:15:37PM -0700, Nicolin Chen wrote: >An ATC invalidation timeout is a fatal error. While the SMMUv3 hardware is >aware of the timeout via a GERROR interrupt, the driver thread issuing the >commands lacks a direct mechanism to verify whether its specific batch was >the cause or not, as polling the CMD_SYNC status doesn't natively return a >failure code, making it very difficult to coordinate per-device recovery. > >Introduce an atc_sync_timeouts bitmap in the cmdq structure to bridge this >gap. When the ISR detects an ATC timeout, set the bit corresponding to the >physical CMDQ index of the faulting CMD_SYNC command. > >On the issuer side, after polling completes (or times out), test and clear >its dedicated bit. If set, override any generic timeout, return -ETIMEDOUT >to trigger device quarantine. > >Signed-off-by: Nicolin Chen >--- > drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 1 + > drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 20 +++++++++++++++++++- > 2 files changed, 20 insertions(+), 1 deletion(-) > >diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h >index 36de2b0b2ebe6..3eb12a34b086a 100644 >--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h >+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h >@@ -633,6 +633,7 @@ struct arm_smmu_cmdq { > atomic_long_t *valid_map; > atomic_t owner_prod; > atomic_t lock; >+ unsigned long *atc_sync_timeouts; > bool (*supports_cmd)(struct arm_smmu_cmdq_ent *ent); > }; > >diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c >index 01030ffd2fe23..9c8972ebc94f9 100644 >--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c >+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c >@@ -445,7 +445,10 @@ void __arm_smmu_cmdq_skip_err(struct arm_smmu_device *smmu, > * at the CMD_SYNC. Attempt to complete other pending commands > * by repeating the CMD_SYNC, though we might well end up back > * here since the ATC invalidation may still be pending. >+ * >+ * Mark the faulty batch in the bitmap for the issuer to match. > */ >+ set_bit(Q_IDX(&q->llq, cons), cmdq->atc_sync_timeouts); > return; > case CMDQ_ERR_CERROR_ILL_IDX: > default: >@@ -895,9 +898,19 @@ int arm_smmu_cmdq_issue_cmdlist(struct arm_smmu_device *smmu, > > /* 5. If we are inserting a CMD_SYNC, we must wait for it to complete */ > if (sync) { >+ u32 sync_prod; >+ > llq.prod = queue_inc_prod_n(&llq, n); >+ sync_prod = llq.prod; >+ > ret = arm_smmu_cmdq_poll_until_sync(smmu, cmdq, &llq); >- if (ret) { >+ if (test_and_clear_bit(Q_IDX(&llq, sync_prod), >+ cmdq->atc_sync_timeouts)) { This will not be set if a software timeout (1 second) occurs. Do you know if the ATC timeout of Arm sMMUv3 is less than the software timeout in the driver? If not maybe we can handle the software timeout here also as the cmdlist is already known? Thanks, Sami >+ dev_err_ratelimited(smmu->dev, >+ "CMD_SYNC for ATC_INV timeout at prod=0x%08x\n", >+ sync_prod); >+ ret = -ETIMEDOUT; >+ } else if (ret) { > dev_err_ratelimited(smmu->dev, > "CMD_SYNC timeout at 0x%08x [hwprod 0x%08x, hwcons 0x%08x]\n", > llq.prod, >@@ -4458,6 +4471,11 @@ int arm_smmu_cmdq_init(struct arm_smmu_device *smmu, > if (!cmdq->valid_map) > return -ENOMEM; > >+ cmdq->atc_sync_timeouts = >+ devm_bitmap_zalloc(smmu->dev, nents, GFP_KERNEL); >+ if (!cmdq->atc_sync_timeouts) >+ return -ENOMEM; >+ > return 0; > } > >-- >2.43.0 > >