Re: [PATCH rc] iommu: Skip PASID validation for devices without PASID capability

[Vasant Hegde <vasant.hegde@amd.com>]

Jason,


On 4/24/2025 6:01 PM, Jason Gunthorpe wrote:
> On Thu, Apr 24, 2025 at 12:08:56PM +0530, Vasant Hegde wrote:
> 
>>> What the iommu driver should do when set_dev_pasid is called for a non-
>>> PASID device?
> 
> That's a good point, maybe the core code should filter that out based
> on max_pasids? I think we do run into trouble here because the drivers
> are allocating PASID table space based on max_pasids so the non-pasid
> device should fail to add the pasid. Tushar, you should have hit this
> in your testing???
> 
> We also have a problem setting up the default domain - it won't
> compute IOMMU_HWPT_ALLOC_PASID properly across the group. If the
> no-pasid device probes first then PASID will be broken on the group.
> 
> Tushar isn't hitting this because ARM always uses a PASID compatible
> domain today, but it will not work on AMD.
> 
> That's a huge pain to deal with :\

Agree. That will complicate things.

Just to be clear, I gave some of the AMD GPU as an example of group where we
have both PASID, non-PASID devices in same group. But currently AMDGPU is not
using PASID. But currently I am not looking for supporting SVA for amdgpu with
such configs.


> 
>> Per device max_pasids check should cover that right?
> 
> The driver shouldn't be doing this though, if the driver is told to
> make a pasid then it should make a pasid.. The driver can fail
> attaching a pasid to a device that is over the device's max_pasid.
> 
>> FYI. One example of such device is some of the AMD GPUs which has
>> both VGA and audio in same group. while VGA supports PASID, audio is
>> not. This used to work fine when we had AMD IOMMU PASID specific
>> driver. GPUs stopped using PASIDs in upstream kernel. So I didn't
>> look into this part in details.
> 
> Uhhh.. That sounds like a worse problem, the only way you should end
> up with same group is if the ACS flags are missing on the GPU so Linux
> assumes the VGA and audio can loopback to each other internally.
> 
> That should completely block PASID support on the GPU side due the
> wrong routing. We can't have a hole in the PASID address space where
> the audio BAR is.
> 
> I suppose the HW doesn't actually behave this way but since it doesn't
> have the right ACS flags the SW doesn't know? Guessing..

Honestly I have no idea. Since they had stopped using PASID support I never
digged into the details!

-Vasant