Re: [PATCH v1 3/4] iommu/arm-smmu-v3: Decouple vmid from S2 nest_parent domain

[Nicolin Chen <nicolinc@nvidia.com>]

On Wed, Mar 05, 2025 at 01:01:57PM -0400, Jason Gunthorpe wrote:
> On Tue, Mar 04, 2025 at 09:04:02PM -0800, Nicolin Chen wrote:
> > @@ -2249,10 +2249,22 @@ static void arm_smmu_tlb_inv_context(void *cookie)
> >  	 */
> >  	if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) {
> >  		arm_smmu_tlb_inv_asid(smmu, smmu_domain->cd.asid);
> > -	} else {
> > +	} else if (!smmu_domain->nest_parent) {
> >  		cmd.opcode	= CMDQ_OP_TLBI_S12_VMALL;
> >  		cmd.tlbi.vmid	= smmu_domain->s2_cfg.vmid;
> >  		arm_smmu_cmdq_issue_cmd_with_sync(smmu, &cmd);
> > +	} else {
> > +		struct arm_vsmmu *vsmmu, *next;
> > +		unsigned long flags;
> > +
> > +		cmd.opcode = CMDQ_OP_TLBI_S12_VMALL;
> > +		spin_lock_irqsave(&smmu_domain->vsmmus.lock, flags);
> > +		list_for_each_entry_safe(vsmmu, next, &smmu_domain->vsmmus.list,
> > +					 vsmmus_elm) {
> > +			cmd.tlbi.vmid = vsmmu->vmid;
> > +			arm_smmu_cmdq_issue_cmd_with_sync(smmu, &cmd);

Just noticed that here should be vsmmu->smmu.

> > +		}
> > +		spin_unlock_irqrestore(&smmu_domain->vsmmus.lock, flags);
> >  	}
> 
> I see.. So this just makes a 3rd classification of invalidation
> protocol that uses a spinlock and linked list
>
> >  	arm_smmu_atc_inv_domain(smmu_domain, 0, 0);
> 
> This is no good, arm_smmu_atc_inv_domain() is invalidating against the
> instance that created the domain.

Oh right... we might need an arm_smmu_atc_inv_all() that takes
an smmu pointer. This might have some performance downgrade as
you worried about though.

> IMHO if you do this you should set domain->iommu = NULL to indicate
> that the iommu is non-valid in this mode to catch issues.

You mean smmu_domain->smmu pointer right?

Thanks
Nicolin