Re: [PATCH 3/4] iommu/sva: Support reservation of global PASIDs

[Jason Gunthorpe <jgg@nvidia.com>]

On Mon, Mar 06, 2023 at 06:48:43PM +0000, Luck, Tony wrote:
> >> ENQCMDS does not have the restriction of using a single CPU MSR to store
> >> PASIDs, PASID is supplied to the instruction operand. 
> >
> > Huh? That isn't what it says in the programming manual. It says the
> > PASID only comes from the IA32_PASID msr and the only two operands are
> > the destination MMIO and the memory source for the rest of the payload.
> 
> Jason,
> 
> Two different instructions with only one letter different in the name.
> 
> ENQCMD - ring 3 instruction. The PASID is inserted into the descriptor
> pushed to the device from the IA32_PASID MSR.
> 
> ENQCMDS - ring 0 instruction (see that trailing "S" for Supervisor mode).
> In this case the submitter can include any PASID value they want in the
> in-memory copy of the descriptor and ENQCMDS will pass that to the
> device.

Ah, well, my comment wasn't talking about ENQCMDS :)

If ENQCMDS can take in an arbitary PASID then there is no
justification here to use the global allocator.

The rational is more like:

 IDXD uses PASIDs that come from the SVA allocator. It needs to create
 an internal kernel-only PASID that is non-overlapping so allow the SVA
 allocator to reserve PASIDs for driver use.

 IDXD has to use the global SVA PASID allocator beacuse its userspace
 will use ENQCMD which requires global PASIDs.

Jason