Re: [RFC PATCH 3/5] iommu/arm-smmu-v3: Implement pm_runtime & system sleep ops

[Nicolin Chen <nicolinc@nvidia.com>]

On Tue, Apr 15, 2025 at 08:47:03PM +0000, Pranjal Shrivastava wrote:
> On Mon, Apr 14, 2025 at 02:26:19PM -0700, Nicolin Chen wrote:
> > > On a system with vCMDQs, there are currently one standard SMMU
> > > CMDQ and two vCMDQs, i.e. totally 3 cmdqs that could be picked
> > > in this context. Perhaps SMMU might need a list of cmdqs that
> > > any new allocated cmdq must be added to, so we can iterate all
> > > the cmdqs in the list?
> > 
> > Another tricky thing: if a vcmdq is assigned to a guest VM, it
> > won't be added to the "list" above. Should we wait for them or
> > simply rely on VMM sending a suspend signal to all its VMs?
> > 
> 
> Hmm... I haven't looked at the vCMDQ series (of the vIOMMU) yet but I'm
> assuming that's what we're talking about here?
> 
> I'm not sure if sending a signal to the VMM would help if we have some
> stalled transactions? I guess we'll need to somehow figure out if
> there's some PCI device assigned that's using that vIOMMU/vCMDQ ? 

If all VMs go into suspend prior to the host machine and all
guest-owned vcmdqs are ensured to be empty, we would be fine.

But this is difficult to tell, since a VMM might not signal
a suspend to all guests VMs. And even if it does, not all the
guest kernels might have implemented a suspend routine in the
guest vcmdq driver draining all guest-owned vcmdqs.

> Maybe we can smartly flush only the PCIe ATC inv and pri resp commands?

Yes, I suspect the host driver will have to do something like
that for the reason that I mentioned.

Thanks
Nicolin