Re: [RFC PATCH v4 03/16] iommu/arm-smmu-v3: Add initial pSMMU realm viommu plumbing

[Jason Gunthorpe <jgg@ziepe.ca>]

On Wed, Sep 02, 2026 at 06:45:42PM +0530, Aneesh Kumar K.V wrote:

> To reiterate, for this configuration:
> 
> - The viommu will use a stage-1 bypass configuration.
> - A new IOMMU_VIOMMU_TYPE_ARM_REALM_SMMUV3 type will create the viommu.
>   The psmmu will be activated at this point to avoid creating psmmu
>   objects early. We will reference-count it to ensure that the same
>   psmmu is shared across realm guests.
> - Creating a vdevice will invoke SMC_RMI_PSMMU_ST_L2_CREATE.
> 
> I am unclear about the vdev_create suggestion. Creating a vdevice
> requires an RD, which is created later in the flow above. How do you
> suggest linking vdevice_alloc to vdev_create?

I was thinking we'd make sure the KVM is associated with the viommu
and/or possibly the S2 domain. That was always sort of broadly the
idea in this space. There are several topics unrelated to CC that
needed this.
In any case, when you create the iommufd viommu you should also do
VSMMU_CREATE which needs the RD. It seems reasonable to assume the RD
is available during vdevice create.

[There is an aside here I will mention: several other use cases need
this idea of an "external" domain where the HWPT would be created but
not controlled by iommufd or the iommu subsystem. Xen and Hyperv for
example. There is probably some merit in thinking more about exactly
what the nested parent domain should be for this viommu, but it isn't
critical.]

> From the RMM's perspective, the sequence is as follows:
>
> viommu alloc
>
> [   rmm ] SMC_RMI_PSMMU_ACTIVATE            2b400000 8819bb000 > RMI_INCOMPLETE 0 10008
> [   rmm ]       SMC_RMI_OP_MEM_DONATE       0 881bac098 1 > RMI_INCOMPLETE 2 10004
> [   rmm ]       SMC_RMI_OP_MEM_DONATE       0 881bac098 1 > RMI_INCOMPLETE 1 10004
> [   rmm ]       SMC_RMI_OP_MEM_DONATE       0 881bac098 1 > RMI_INCOMPLETE 1 0
> [   rmm ] L1 StrTab: PA 0x881baa000 VA 0x80003c0000 size 0x2000
> [   rmm ] CMDQ: PA 0x88066a000 VA 0x80003c2000
> [   rmm ] EVTQ: PA 0x8815c5000 VA 0x80003c3000
> [   rmm ] PSMMU 0x2b400000 activated
> [   rmm ]       SMC_RMI_OP_CONTINUE         0 0 > RMI_SUCCESS 0 0
>
> [   rmm ] SMC_RMI_PSMMU_ST_L2_CREATE        2b400000 300 > RMI_INCOMPLETE 0 10004
> [   rmm ]       SMC_RMI_OP_MEM_DONATE       0 881bac098 1 > RMI_INCOMPLETE 1 0
> [   rmm ] smmu->strtab_base[12] 0x0 @0x80003c0060
> [   rmm ] L1STD[12] 0x8819bb007 for SID 0x300: L2 table VA 0x80003d0000 PA 0x8819bb000
> [   rmm ]       SMC_RMI_OP_CONTINUE         0 0 > RMI_SUCCESS 0 0

What was this one for during viommu alloc?

> vdevice alloc
> 
> [   rmm ] SMC_RMI_PSMMU_ST_L2_CREATE        2b400000 200 > RMI_INCOMPLETE 0 10004
> [   rmm ]       SMC_RMI_OP_MEM_DONATE       0 882648098 1 > RMI_INCOMPLETE 1 0
> [   rmm ] smmu->strtab_base[8] 0x0 @0x80003c0040
> [   rmm ] L1STD[8] 0x882506007 for SID 0x200: L2 table VA 0x80003cc000 PA 0x882506000
> [   rmm ]       SMC_RMI_OP_CONTINUE         0 0 > RMI_SUCCESS 0 0
> 
> RD gets allocated here
> 
> [   rmm ] SMC_RMI_REALM_CREATE              881b03000 8819a1000 > RMI_INCOMPLETE 0 24
> [   rmm ]       SMC_RMI_OP_MEM_DONATE       0 881605098 9 > RMI_INCOMPLETE 9 0
> [   rmm ]       SMC_RMI_OP_CONTINUE         0 0 > RMI_SUCCESS 0 0

Why? The VMM needs to create the kvm before starting the iommu stuff.

I would expect that KVM knows it is going to a realm very early on? I
had assumed the RD would also be created early by KVM?

What triggers RD creation? Why can't the VMM do it earlier?

Your followup said:
  - The realm is created during viommu allocation, ensuring that the
    vSMMU can be created here.

Do you mean the SMMUv3 driver triggers RD creation? That feels
wrong. Did you mean the VMM just does it earlier?

The draft in the next message looks promising, did you discover any
other gotchas when exploring it?

When you repost this can you take some care to explain the general
idea of this modelling in the commit messages so AMD and Intel can
confirm they can use it for both their non-viommu and viommu cases?

Thanks,
Jason