Re: [PATCH v8 12/14] iommu/arm-smmu-v3: Introduce struct arm_smmu_vmaster

[Jason Gunthorpe <jgg@nvidia.com>]

On Mon, Mar 17, 2025 at 11:49:14AM -0700, Nicolin Chen wrote:
> On Mon, Mar 17, 2025 at 12:44:23PM -0300, Jason Gunthorpe wrote:
> > On Tue, Mar 11, 2025 at 10:43:08AM -0700, Nicolin Chen wrote:
> > > > > +int arm_smmu_attach_prepare_vmaster(struct arm_smmu_attach_state *state,
> > > > > +				    struct arm_smmu_nested_domain *nested_domain)
> > > > > +{
> > > > > +	struct arm_smmu_vmaster *vmaster;
> > > > > +	unsigned long vsid;
> > > > > +	int ret;
> > > > > +
> > > > > +	iommu_group_mutex_assert(state->master->dev);
> > > > > +
> > > > > +	/* Skip invalid vSTE */
> > > > > +	if (!(nested_domain->ste[0] & cpu_to_le64(STRTAB_STE_0_V)))
> > > > > +		return 0;
> > > > 
> > > > Ok, and we don't need to set 'state->vmaster' in this case because we
> > > > only report stage-1 faults back to the vSMMU?
> > > 
> > > This is a good question that I didn't ask myself hard enough..
> > > 
> > > I think we should probably drop it. An invalid STE should trigger
> > > a C_BAD_STE event that is in the supported vEVENT list. I'll run
> > > some test before removing this line from v9.
> > 
> > It won't trigger C_BAD_STE, recall Robin was opposed to thatm so we have this:
> > 
> > static void arm_smmu_make_nested_domain_ste(
> > 	struct arm_smmu_ste *target, struct arm_smmu_master *master,
> > 	struct arm_smmu_nested_domain *nested_domain, bool ats_enabled)
> > {
> > 	unsigned int cfg =
> > 		FIELD_GET(STRTAB_STE_0_CFG, le64_to_cpu(nested_domain->ste[0]));
> > 
> > 	/*
> > 	 * Userspace can request a non-valid STE through the nesting interface.
> > 	 * We relay that into an abort physical STE with the intention that
> > 	 * C_BAD_STE for this SID can be generated to userspace.
> > 	 */
> > 	if (!(nested_domain->ste[0] & cpu_to_le64(STRTAB_STE_0_V)))
> > 		cfg = STRTAB_STE_0_CFG_ABORT;
> > 
> > So, in the case of a non-valid STE, and a device access, the HW will
> > generate one of the translation faults and that will be forwarded.
> > 
> > Some software component will have to transform those fault events into
> > C_BAD_STE for the VM.
> 
> Hmm, double checked the spec. It does say that C_BAD_STE would be
> triggered:
> 
> " V, bit [0] STE Valid.
>   [...]
>   Device transactions that select an STE with this field configured
>   to 0 are terminated with an abort reported back to the device and
>   a C_BAD_STE event is recorded."
> 
> I also did a hack test unsetting the V bit in the kernel. Then, the
> HW did report C_BAD_STE (0x4) back to the VM (via vEVENTQ).

Yes, I expect that C_BAD_STE will forward just fine.

But, as above, it should never be generated by HW because the
hypervisor kernel will never install a bad STE, we detect that and
convert it to abort.

Jason