Re: [PATCH v9 4/6] iommu/arm-smmu-v3: Add CS_NONE quirk for CONFIG_TEGRA241_CMDQV

[Nicolin Chen <nicolinc@nvidia.com>]

On Fri, Jul 05, 2024 at 11:10:47AM -0700, Nicolin Chen wrote:
> Hi Will,
> 
> On Fri, Jul 05, 2024 at 04:27:21PM +0100, Will Deacon wrote:
> > On Tue, Jul 02, 2024 at 01:10:19PM -0700, Nicolin Chen wrote:
> > > On Tue, Jul 02, 2024 at 12:47:14PM -0700, Nicolin Chen wrote:
> > > > @@ -345,6 +345,11 @@ static void arm_smmu_cmdq_build_sync_cmd(u64 *cmd, struct arm_smmu_device *smmu,
> > > >             FIELD_PREP(CMDQ_SYNC_0_MSH, ARM_SMMU_SH_ISH) |
> > > >             FIELD_PREP(CMDQ_SYNC_0_MSIATTR, ARM_SMMU_MEMATTR_OIWB);
> > > >
> > > > +   if (cmdq->type == TEGRA241_VCMDQ) {
> > > > +           cmd[0] |= FIELD_PREP(CMDQ_SYNC_0_CS, CMDQ_SYNC_0_CS_NONE);
> > > > +           return;
> > > > +   }
> > > > +
> > > >     if (!(smmu->options & ARM_SMMU_OPT_MSIPOLL)) {
> > > >             cmd[0] |= FIELD_PREP(CMDQ_SYNC_0_CS, CMDQ_SYNC_0_CS_SEV);
> > > >             return;
> > > > @@ -690,7 +695,8 @@ static int arm_smmu_cmdq_poll_until_sync(struct arm_smmu_device *smmu,
> > > >                                     struct arm_smmu_cmdq *cmdq,
> > > >                                     struct arm_smmu_ll_queue *llq)
> > > >  {
> > > > -   if (smmu->options & ARM_SMMU_OPT_MSIPOLL)
> > > > +   if (smmu->options & ARM_SMMU_OPT_MSIPOLL &&
> > > > +       cmdq->type != TEGRA241_VCMDQ) {
> > > >             return __arm_smmu_cmdq_poll_until_msi(smmu, cmdq, llq);
> > > >
> > > > --------------------------------------------------------------
> > > >
> > > > Would you prefer this one? I feel CMDQ_QUIRK_SYNC_CS_NONE_ONLY
> > > > is more general looking though..
> > >
> > > And we would need some additional lines of comments for the two
> > > pieces above, explaining why TEGRA241_VCMDQ type needs the first
> > > one while bypasses the second one. Again, it feels even worse :(
> > 
> > I hacked the code around a bit this afternoon. Please can you see if:
> > 
> > https://git.kernel.org/pub/scm/linux/kernel/git/will/linux.git/log/?h=for-nicolin/grace-vcmdq-wip
> > 
> > does roughly what you need?
> 
> I appreciate the patch. Yet, we cannot use IORT's model field.
> This would need to go through IORT documentation, for A. And B,
> we had a very long discussion with ARM (Robin was there) years
> ago, and concluded that this CMDQV would not be a model in IORT
> but a DSDT node as an extension. So, this is firm...
> 
> With that, we cannot avoid an unconditional hard-coding tegra
> function call even if we switch to an impl design:
> 
> +static int acpi_smmu_impl_init(u32 model, struct arm_smmu_device *smmu)
> +{
> +	/*
> +	 * unconditional go through ACPI table to detect if there is a tegra241
> +	 * implementation that extends SMMU with a CMDQV. The probe() will fill
> +	 * the smmu->impl pointer upon success. Otherwise, fall back to regular
> +	 * SMMU CMDQ.
> +	 */
> +	tegra241_impl_acpi_probe(smmu);
> +	return 0;
> +}
> 
> As for arm_smmu_cmdq_needs_busy_polling, it doesn't really look
> very optimal to me. But if you insist on having an smmu option,
> we still have to take in the PATCH-3 in this series, enforcing
> an arm_smmu_cmdq_build_sync_cmd() call in the IRQ handler too.
> So, it would eventually look like [attachment].

Please ignore the attachment. Since we are adding arm_smmu_impl,
I figure that we could add an arm_smmu_cmdq_impl too. There's an
another small feature that I didn't implement in this v9, while
being able to benefit from a cmdq impl now.

The impl can also hold a boolean busy_polling, so we won't need
a global smmu option.

I will send a new version asap, though I am not sure if we can
still make it to this cycle that we hoped for :-/

Thanks
Nicolin