Linux-ARM-Kernel Archive on lore.kernel.org
 help / color / mirror / Atom feed
From: John Garry <john.garry@arm.com>
To: Nicolin Chen <nicolinc@nvidia.com>, will@kernel.org, jgg@nvidia.com
Cc: robin.murphy@arm.com, joro@8bytes.org, praan@google.com,
	kevin.tian@intel.com, smostafa@google.com,
	linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev,
	linux-kernel@vger.kernel.org, jamien@nvidia.com, kas@kernel.org,
	Cristian Prundeanu <cpru@amazon.com>,
	Breno Leitao <leitao@kernel.org>,
	xueshuai@linux.alibaba.com
Subject: Re: [PATCH v12 10/12] iommu/arm-smmu-v3: Retain CR0_SMMUEN during kdump device reset
Date: Mon, 16 Nov 2026 12:19:32 +0000	[thread overview]
Message-ID: <1ee06735-d651-411f-bcc9-783e9c016c84@arm.com> (raw)
In-Reply-To: <f458ce5587ad991c85e65ee12e8f5b271e5f6745.1793593923.git.nicolinc@nvidia.com>

On 11/2/26 05:19, Nicolin Chen wrote:
> When ARM_SMMU_OPT_KDUMP_ADOPT is detected, do not disable SMMUEN and skip
> the CR1/CR2/STRTAB_BASE update sequence in arm_smmu_device_reset(). Those
> register writes are all CONSTRAINED UNPREDICTABLE while CR0_SMMUEN==1, so
> leaving them intact lets in-flight DMAs continue to be translated by the
> adopted stream table.
> 
> Initialize 'enables' to 0, so it can carry the retained CR0 fields in the
> kdump case, clearing only the queue enable bits. Then, preserve them when
> enabling the command queue.
> 
> The retained CR0 keeps the crashed kernel's ATSCHK too, which selects the
> fast (0) or the safe (1) mode for the ATS translated traffic. Switching to
> the safe mode would start checking the in-flight traffic against STE.EATS
> fields that the crashed kernel never set up for a check, aborting the very
> DMAs being carried.
> 
> Clear latched gerror bits if necessary.
> 
> Reviewed-by: Kevin Tian <kevin.tian@intel.com>
> Reviewed-by: Pranjal Shrivastava <praan@google.com>
> Tested-by: Breno Leitao <leitao@kernel.org>
> Tested-by: Cristian Prundeanu <cpru@amazon.com>
> Tested-by: Shuai Xue <xueshuai@linux.alibaba.com>
> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
> Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>

Looks ok regardless of small comments below, so FWIW:

Reviewed-by: John Garry <john.garry@arm.com>

> ---
>   drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 57 +++++++++++++++++++--
>   1 file changed, 53 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
> index d45ce33334ce2..1c53c690a2b02 100644
> --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
> +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
> @@ -5145,10 +5145,39 @@ static void arm_smmu_write_strtab(struct arm_smmu_device *smmu)
>   static int arm_smmu_device_reset(struct arm_smmu_device *smmu)
>   {
>   	int ret;
> -	u32 reg, enables;
> +	u32 reg, enables = 0;
>   
> -	/* Clear CR0 and sync (disables SMMU and queue processing) */
>   	reg = readl_relaxed(smmu->base + ARM_SMMU_CR0);
> +
> +	/*
> +	 * In a kdump case (set when CR0_SMMUEN=1 and !GERROR_SFM_ERR), retain
> +	 * all the live CR0 fields, e.g. CR0_SMMUEN to avoid aborting in-flight
> +	 * DMA and CR0_ATSCHK to carry on the ATS-check policy, while clearing
> +	 * only the queue enable bits for this kernel to take over the queues.
> +	 *
> +	 * According to spec, updating STRTAB_BASE/CR1/CR2 when CR0_SMMUEN=1 is
> +	 * CONSTRAINED UNPREDICTABLE. So, skip those register updates and rely
> +	 * on the adopted stream table from the crashed kernel.
> +	 */
> +	if (smmu->options & ARM_SMMU_OPT_KDUMP_ADOPT) {
> +		dev_info(smmu->dev,
> +			 "kdump: retaining SMMUEN for in-flight DMA\n");
> +		enables = reg & ~(CR0_CMDQEN | CR0_EVTQEN | CR0_PRIQEN);
> +		/*
> +		 * Disable queues since arm_smmu_device_disable() is skipped.
> +		 * CR0 fields are independent per spec, so the queue enable bits
> +		 * can be cleared while retaining SMMUEN=1.
> +		 */
> +		ret = arm_smmu_write_reg_sync(smmu, enables, ARM_SMMU_CR0,
> +					      ARM_SMMU_CR0ACK);
> +		if (ret) {
> +			dev_err(smmu->dev, "failed to disable queues\n");

"kdump: failed to disable queues\n"?

And it would be nice to say or hint what happens next.

> +			return ret;
> +		}
> +		goto reset_queues;

nit: goto could be avoided, but that is a taste thing

> +	}
> +
> +	/* Clear CR0 and sync (disables SMMU and queue processing) */
>   	if (reg & CR0_SMMUEN) {
>   		dev_warn(smmu->dev, "SMMU currently enabled! Resetting...\n");
>   		arm_smmu_update_gbpa(smmu, GBPA_ABORT, 0);
> @@ -5178,12 +5207,27 @@ static int arm_smmu_device_reset(struct arm_smmu_device *smmu)
>   	/* Stream table */
>   	arm_smmu_write_strtab(smmu);
>   
> +reset_queues:
> +	/*
> +	 * GERROR bits are latched. Read after queue disabling so that unhandled
> +	 * errors would be visible. Ack everything prior to re-enabling the CMDQ
> +	 * as a stale CMDQ_ERR would halt the CMDQ and new command will timeout.
> +	 * Acking SFM_ERR is defined too, although it would not exit the SFM.
> +	 */
> +	if (is_kdump_kernel()) {
> +		u32 gerror = readl_relaxed(smmu->base + ARM_SMMU_GERROR);
> +		u32 gerrorn = readl_relaxed(smmu->base + ARM_SMMU_GERRORN);
> +
> +		if ((gerror ^ gerrorn) & GERROR_ERR_MASK)
> +			writel(gerror, smmu->base + ARM_SMMU_GERRORN);
> +	}
> +
>   	/* Command queue */
>   	writeq_relaxed(smmu->cmdq.q.q_base, smmu->base + ARM_SMMU_CMDQ_BASE);
>   	writel_relaxed(smmu->cmdq.q.llq.prod, smmu->base + ARM_SMMU_CMDQ_PROD);
>   	writel_relaxed(smmu->cmdq.q.llq.cons, smmu->base + ARM_SMMU_CMDQ_CONS);
>   
> -	enables = CR0_CMDQEN;
> +	enables |= CR0_CMDQEN;
>   	ret = arm_smmu_write_reg_sync(smmu, enables, ARM_SMMU_CR0,
>   				      ARM_SMMU_CR0ACK);
>   	if (ret) {
> @@ -5239,7 +5283,12 @@ static int arm_smmu_device_reset(struct arm_smmu_device *smmu)
>   		}
>   	}
>   
> -	if (smmu->features & ARM_SMMU_FEAT_ATS) {
> +	/*
> +	 * In a kdump adopt case, retain the crashed kernel's ATS-check policy
> +	 * captured above rather than forcing it on.
> +	 */
> +	if (!(smmu->options & ARM_SMMU_OPT_KDUMP_ADOPT) &&
> +	    (smmu->features & ARM_SMMU_FEAT_ATS)) {
>   		enables |= CR0_ATSCHK;
>   		ret = arm_smmu_write_reg_sync(smmu, enables, ARM_SMMU_CR0,
>   					      ARM_SMMU_CR0ACK);



  reply	other threads:[~2026-11-16 12:20 UTC|newest]

Thread overview: 22+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-02  5:19 [PATCH v12 00/12] iommu/arm-smmu-v3: Adopt the crashed kernel's stream table for kdump Nicolin Chen
2026-11-02  5:19 ` [PATCH v12 01/12] iommu: Pass in NULL old_domain when attaching a deferred device Nicolin Chen
2026-11-02  5:19 ` [PATCH v12 02/12] iommu/arm-smmu-v3: Init the vmid_map ida before the stream table setup Nicolin Chen
2026-11-16  9:30   ` John Garry
2026-11-02  5:19 ` [PATCH v12 03/12] iommu/arm-smmu-v3: Make the ASID space per SMMU instance Nicolin Chen
2026-11-16  9:46   ` John Garry
2026-11-02  5:19 ` [PATCH v12 04/12] iommu/arm-smmu-v3: Pack the MSI vectors of the used sources Nicolin Chen
2026-11-16 10:29   ` John Garry
2026-11-02  5:19 ` [PATCH v12 05/12] iommu/arm-smmu-v3: Add ARM_SMMU_FEAT_EVTQ for the event queue Nicolin Chen
2026-11-16 10:59   ` John Garry
2026-11-02  5:19 ` [PATCH v12 06/12] iommu/arm-smmu-v3: Disable the EVTQ and the PRIQ in a kdump kernel Nicolin Chen
2026-11-16 11:08   ` John Garry
2026-11-02  5:19 ` [PATCH v12 07/12] iommu/arm-smmu-v3: Add ARM_SMMU_OPT_KDUMP_ADOPT for " Nicolin Chen
2026-11-02  5:19 ` [PATCH v12 08/12] iommu/arm-smmu-v3-kexec: Reserve crashed kernel's ASIDs and VMIDs Nicolin Chen
2026-11-02  5:19 ` [PATCH v12 09/12] iommu/arm-smmu-v3-kexec: Implement is_attach_deferred() Nicolin Chen
2026-11-16 12:03   ` John Garry
2026-11-02  5:19 ` [PATCH v12 10/12] iommu/arm-smmu-v3: Retain CR0_SMMUEN during kdump device reset Nicolin Chen
2026-11-16 12:19   ` John Garry [this message]
2026-11-02  5:19 ` [PATCH v12 11/12] iommu/arm-smmu-v3: Skip RMR bypass for kdump adoption Nicolin Chen
2026-11-16 12:26   ` John Garry
2026-11-02  5:19 ` [PATCH v12 12/12] iommu/arm-smmu-v3: Detect ARM_SMMU_OPT_KDUMP_ADOPT in probe() Nicolin Chen
2026-11-16 12:32   ` John Garry

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1ee06735-d651-411f-bcc9-783e9c016c84@arm.com \
    --to=john.garry@arm.com \
    --cc=cpru@amazon.com \
    --cc=iommu@lists.linux.dev \
    --cc=jamien@nvidia.com \
    --cc=jgg@nvidia.com \
    --cc=joro@8bytes.org \
    --cc=kas@kernel.org \
    --cc=kevin.tian@intel.com \
    --cc=leitao@kernel.org \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=nicolinc@nvidia.com \
    --cc=praan@google.com \
    --cc=robin.murphy@arm.com \
    --cc=smostafa@google.com \
    --cc=will@kernel.org \
    --cc=xueshuai@linux.alibaba.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox