Re: [PATCH v11 2/9] iommu/arm-smmu-v3: Enforce arm_smmu_cmdq_build_sync_cmd

[Nicolin Chen <nicolinc@nvidia.com>]

On Fri, Aug 16, 2024 at 02:53:31PM +0100, Will Deacon wrote:
> On Tue, Aug 06, 2024 at 07:11:47PM -0700, Nicolin Chen wrote:
> > There is an existing arm_smmu_cmdq_build_sync_cmd() so the driver should
> > call it at all places other than going through arm_smmu_cmdq_build_cmd()
> > separately. This helps the following patch that adds a CS_NONE option.
> >
> > Note that this changes the type of CMD_SYNC in __arm_smmu_cmdq_skip_err,
> > in ARM_SMMU_OPT_MSIPOLL=true case, from previously a non-MSI one to now
> > an MSI one that is proven to still work using a hacking test:
> >   nvme: Adding to iommu group 10
> >   nvme: --------hacking-----------
> >   arm-smmu-v3: unexpected global error reported (0x00000001),
> >                this could be serious
> >   arm-smmu-v3: CMDQ error (cons 0x01000022): Illegal command
> >   arm-smmu-v3: skipping command in error state:
> >   arm-smmu-v3:  0x0000000000000000
> >   arm-smmu-v3:  0x0000000000000000
> >   nvme: -------recovered----------
> >   nvme nvme0: 72/0/0 default/read/poll queues
> >    nvme0n1: p1 p2
> 
> Hmm, I'm still a little wary of this. The only reason we emit a CMD_SYNC
> in __arm_smmu_cmdq_skip_err() is because the SMMU doesn't have a NOP
> command. So I'd really like the SYNC to have as little functionality
> as possible in this case.
> 
> I think we could either propapate the 'cs' field down to
> arm_smmu_cmdq_build_cmd() or just open-code the command-creation in
> __arm_smmu_cmdq_skip_err().

OK. Let's go with your approach then:
+       if (arm_smmu_cmdq_needs_busy_polling(smmu, cmdq))
+               u64p_replace_bits(cmd, CMDQ_SYNC_0_CS_NONE, CMDQ_SYNC_0_CS);

Thanks
Nicolin