Re: [PATCH v6 2/6] iommu/arm-smmu-v3: Add CS_NONE quirk

From: Nicolin Chen <nicolinc@nvidia.com>
To: Jason Gunthorpe <jgg@nvidia.com>
Cc: <will@kernel.org>, <robin.murphy@arm.com>, <joro@8bytes.org>,
	<thierry.reding@gmail.com>, <vdumpa@nvidia.com>,
	<jonathanh@nvidia.com>, <linux-kernel@vger.kernel.org>,
	<iommu@lists.linux.dev>, <linux-arm-kernel@lists.infradead.org>,
	<linux-tegra@vger.kernel.org>
Subject: Re: [PATCH v6 2/6] iommu/arm-smmu-v3: Add CS_NONE quirk
Date: Tue, 30 Apr 2024 09:30:43 -0700	[thread overview]
Message-ID: <ZjEcs4rY1HpPz4Oa@Asurada-Nvidia> (raw)
In-Reply-To: <20240430142201.GQ941030@nvidia.com>

On Tue, Apr 30, 2024 at 11:22:01AM -0300, Jason Gunthorpe wrote:
> On Mon, Apr 29, 2024 at 09:43:45PM -0700, Nicolin Chen wrote:
> > The CMDQV extension in NVIDIA Tegra241 SoC only supports CS_NONE in the
> > CS field of CMD_SYNC. Add a quirk flag to accommodate that.
> > 
> > Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
> > ---
> >  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 10 ++++++++--
> >  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h |  4 ++++
> >  2 files changed, 12 insertions(+), 2 deletions(-)
> 
> This seems fine, other than the misplaced hunk
> 
> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
> 
> But it might be tidier like the below. There is already a function
> that is called to build the sync that has the q, just build it
> directly there and avoid going through the ent?

Yea, and looks like we can have a patch tidying the existing sync
building function, and then another one adding CS_NONE.

>  static void arm_smmu_cmdq_build_sync_cmd(u64 *cmd, struct arm_smmu_device *smmu,
> -					 struct arm_smmu_queue *q, u32 prod)
> +					 struct arm_smmu_queue *q, u32 prod,
> +					 bool msi)
>  {
...
> +	if (!msi || !(smmu->options & ARM_SMMU_OPT_MSIPOLL)) {
> +		cmd[0] |= FIELD_PREP(CMDQ_SYNC_0_CS, CMDQ_SYNC_0_CS_SEV);
> +		return;
> +	}
...
> @@ -402,9 +395,6 @@ void __arm_smmu_cmdq_skip_err(struct arm_smmu_device *smmu,
>  	u64 cmd[CMDQ_ENT_DWORDS];
>  	u32 cons = readl_relaxed(q->cons_reg);
>  	u32 idx = FIELD_GET(CMDQ_CONS_ERR, cons);
> -	struct arm_smmu_cmdq_ent cmd_sync = {
> -		.opcode = CMDQ_OP_CMD_SYNC,
> -	};
>  
>  	dev_err(smmu->dev, "CMDQ error (cons 0x%08x): %s\n", cons,
>  		idx < ARRAY_SIZE(cerror_str) ?  cerror_str[idx] : "Unknown");
> @@ -437,11 +427,8 @@ void __arm_smmu_cmdq_skip_err(struct arm_smmu_device *smmu,
>  	for (i = 0; i < ARRAY_SIZE(cmd); ++i)
>  		dev_err(smmu->dev, "\t0x%016llx\n", (unsigned long long)cmd[i]);
>  
> -	if (q->quirks & CMDQ_QUIRK_SYNC_CS_NONE_ONLY)
> -		cmd_sync.sync.cs_none = true;
> -
>  	/* Convert the erroneous command into a CMD_SYNC */
> -	arm_smmu_cmdq_build_cmd(cmd, &cmd_sync);
> +	arm_smmu_cmdq_build_sync_cmd(cmd, smmu, q, 0, false);
>  
>  	queue_write(Q_ENT(q, cons), cmd, q->ent_dwords);

Here is the only caller for "msi=false". Maybe we could just do:
+	arm_smmu_cmdq_build_sync_cmd(cmd, smmu, q, cons);

So, no need of "bool msi"? It would slightly change the behavior
though, a SYNC for ARM_SMMU_OPT_MSIPOLL should be still a SYNC.

Thanks
Nicolin