Re: [PATCH v4 02/13] iommu/arm-smmu-v3: Add smmu_s1_cfg to smmu_master

[Jason Gunthorpe <jgg@nvidia.com>]

On Thu, Jul 27, 2023 at 07:22:05PM +0800, Michael Shavit wrote:
> Sorry for the delay; I'm trying to refactor these patches now.
> 
> > I think the master should have a pointer to the iommu_domain that owns
> > the STE or if NULL the master should assign its internal CD table to
> > the STE.
> Just to clarify, does the nested domain patch series require writing
> CDs into the user-space's CD table using arm_smmu_write_ctx_desc()?

No, CD entries in nested CD tables are written by userspace only.

> Or is there any other requirement for writing a CD into a
> domain-owned CD table from arm_smmu_write_ctx_desc?

Not that I know of.

The only time the kernel writes a CD entry is to the shared CD table
stored in the master.

> 1. The CD entries STALL bit value in arm_smmu_write_ctx_desc depends
> on the master (e.g. if STALL_FORCE is set on the smmu device). This
> could potentially be encoded in arm_smmu_ctx_desc_cfg, at which point
> that CD table is only attachable to masters with the same
> stall_enabled value.

For cleanness I would orgnize it like this.

> 2. arm_smmu_write_ctx_desc must sync the CD for the attached master(s)
> in the middle of writing CD entry.
>
> This is all easier to handle in arm_smmu_write_ctx_desc if the table
> is always owned by the master.

I think it is fine if you start with a shared CD table being 1:1 with
a single master.

Making the CD table shared between masters (eg for multi-device-group)
is a micro-optimization, and I'm not sure we have workloads where it
is worthwhile. We already block PASID support for multi-device-group.

Though resizable CD table is probably a better place to put efforts.

Jason