Re: [PATCH v9 6/7] iommu/arm-smmu-v3: Add arm_smmu_invs based arm_smmu_domain_inv_range()

[Nicolin Chen <nicolinc@nvidia.com>]

On Tue, Jan 27, 2026 at 03:19:38PM -0400, Jason Gunthorpe wrote:
> On Tue, Jan 27, 2026 at 10:37:44AM -0800, Nicolin Chen wrote:
> > On Tue, Jan 27, 2026 at 02:23:48PM -0400, Jason Gunthorpe wrote:
> > > On Tue, Jan 27, 2026 at 10:07:09AM -0800, Nicolin Chen wrote:
> > > > > My understanding has been that this invalidation can run from an IRQ
> > > > > context - we permit the use of the DMA API from an interrupt handler?
> > > > > 
> > > > > I though that for rwsem the read side does not require the _irqsave,
> > > > > even if it is in an irq context, unless the write side runs from an
> > > > > IRQ. 
> > > > 
> > > > Hmm, is "rwsem" a typo? Because it's rwlock_t, which is spinlock :-/
> > > 
> > > Yeah, sorry
> > > 
> > > > > Here the write side always runs from a process context.
> > > > > 
> > > > > So the write side will block the IRQ which ensures we don't spin
> > > > > during read in an IRQ.
> > > > 
> > > > And, does write_lock_irqsave() disable global IRQ or local IRQ only?
> > > > 
> > > > Documentation/locking/locktypes.rst mentions "local_irq_disable()"..
> > > 
> > > It will only disable the local IRQ, since it is a spin type lock an IRQ on
> > > another CPU can spin until it is unlocked.
> > > 
> > > The main issue is if this CPU takes an IRQ while the write side is
> > > locked and spins, then it will never unlock.
> > 
> > Yea, that sounds unsafe. I'll send a v11 with read_lock_irqsave().
> 
> I'm explaining why it is safe now, the write side takes the irqsave so
> the above can't happen.

Sorry, I misunderstood..

> There is no case where the read side needs to block IRQ because if the
> read side succeeds, an IRQ happens and tries to take another read
> side, it will succeed not spin.

Yea, I also went a bit deeper.

It seems to depend on the CONFIG_QUEUED_RWLOCKS (ARM sets =y)

252 config QUEUED_RWLOCKS
253         def_bool y if ARCH_USE_QUEUED_RWLOCKS
254         depends on SMP && !PREEMPT_RT

where a reader will not get blocked in our particular use case:

21 void __lockfunc queued_read_lock_slowpath(struct qrwlock *lock)
22 {
23         /*
24          * Readers come here when they cannot get the lock without waiting
25          */
26         if (unlikely(in_interrupt())) {
27                 /*
28                  * Readers in interrupt context will get the lock immediately
29                  * if the writer is just waiting (not holding the lock yet),
30                  * so spin with ACQUIRE semantics until the lock is available
31                  * without waiting in the queue.
32                  */
33                 atomic_cond_read_acquire(&lock->cnts, !(VAL & _QW_LOCKED));
34                 return;

And I don't see any non-hackable way for CONFIG_QUEUED_RWLOCKS=n
unless CONFIG_PREEMPT_RT=y, which would be a different ball game
that I assume SMMUv3 might not be completely compatible with.

Thanks
Nicolin