Re: [PATCH v6 07/10] iommu/arm-smmu-v3: Implement pm_runtime & system sleep ops

[Pranjal Shrivastava <praan@google.com>]

On Fri, Apr 24, 2026 at 12:16:39PM -0300, Jason Gunthorpe wrote:
> On Thu, Apr 23, 2026 at 10:24:30PM -0700, Daniel Mentz wrote:
> > On Wed, Apr 22, 2026 at 6:41 AM Pranjal Shrivastava <praan@google.com> wrote:
> > > > > +       /* Clear the STOP FLAG to allow CMDQ Submissions */
> > > >
> > > > Clearing the STOP flag here seems premature. I propose the following sequence:
> > > >
> > > > * Set up command queue (SMMU_CR1, SMMU_CMDQ_BASE etc.)
> > > > * Enable command queue
> > > > * Clear STOP flag
> > > > * Enable SMMU
> > > >
> > > > By clearing the STOP flag, you allow arm_smmu_cmdq_issue_cmdlist() to
> > > > write to SMMU_CMDQ_PROD which then races against
> > > > arm_smmu_device_reset() which also writes to SMMU_CMDQ_PROD.
> > > >
> > >
> > > I agree, IIUC you're suggesting to clear the stop flag before the
> > > TLBI_ALL and CFGI_ALL commands are issued by device_reset? We'll need
> > > to clear the flag before device_reset issues the  TLBI_ALL AND CFGI_ALL.
> > > Thus the sequence should be:
> > >
> > > * Set up CMDQ
> > > * Enable CMDQ
> > > * Clear STOP flag
> > > * Issue TLBI_ALL + CFGI_ALL
> > > * Enable SMMU
> > >
> > > Right?
> > 
> > Yes
> 
> This is making a very similar fencing argument to how Nicolin's series
> works.
> 

The invs array series? 

> After clearing stop but before enablie SMMU you have to acquire all
> concurrent threads updated data structure memory into this thread
> because as soon as enable SMMU happens it will start reading that
> memory and it must be 'either we see the latest copy' or 'we see an
> old copy and guarenteed to see the invalidation'
> 
> Exactly the same as the reasoning used for attach with the
> invalidation list. So I think you should look at the barrier design
> there and ensure it exists here too.
> 

Hmm.. I see your point. We need to ensure that any structure updates
(like STE/CD or page table changes) made by concurrent threads are 
globally visible before we send the MMIO write to enable the SMMU,
i.e. we're talking about memory ordering between other CPUs and
the Resume CPU (CPU running resume), without a barrier on the Resume
CPU, the SMMUEN=1 write could potentially beat a client thread's memory
update. If the SMMU is enabled and fetches a stale configuration before
the RAM update lands, it could cache an inconsistent state.

I guess I'll add an smp_mb() before the SMMUEN enable to act as a
system-wide synchronization point.

Thanks,
Praan