Re: [PATCH v2 1/2] iommu/arm-smmu-v3: Fix CMDQ timeout warning

[Jacob Pan <jacob.pan@linux.microsoft.com>]

Hi Nicolin,

On Thu, 30 Oct 2025 15:41:57 -0700
Nicolin Chen <nicolinc@nvidia.com> wrote:

> On Mon, Oct 20, 2025 at 03:43:52PM -0700, Jacob Pan wrote:
> > @@ -785,10 +748,33 @@ static int arm_smmu_cmdq_issue_cmdlist(struct
> > arm_smmu_device *smmu, do {
> >  		u64 old;
> >  
> > +		queue_poll_init(smmu, &qp);
> >  		while (!queue_has_space(&llq, n + sync)) {
> > +			unsigned long iflags;
> > +
> >  			local_irq_restore(flags);
> > -			if
> > (arm_smmu_cmdq_poll_until_not_full(smmu, cmdq, &llq))
> > -				dev_err_ratelimited(smmu->dev,
> > "CMDQ timeout\n");
> > +			/*
> > +			 * Try to update our copy of cons by
> > grabbing exclusive cmdq access. If
> > +			 * that fails, spin until somebody else
> > updates it for us.
> > +			 */
> > +			if
> > (arm_smmu_cmdq_exclusive_trylock_irqsave(cmdq, iflags)) {
> > +				WRITE_ONCE(cmdq->q.llq.cons,
> > readl_relaxed(cmdq->q.cons_reg));
> > +
> > arm_smmu_cmdq_exclusive_unlock_irqrestore(cmdq, iflags);
> > +				llq.val =
> > READ_ONCE(cmdq->q.llq.val);
> > +				local_irq_save(flags);  
> 
> I don't quite get the reason why it moves queue_poll_init() and
> add local_irq_save(). It's quite different than what the driver
> has, so it's nicer to explain in the commit message at least.

Let me add the following to the commit message.

The original code has three nested while loops,
do {
	while (!queue_has_space(&llq, n + sync)) {
		// inside arm_smmu_cmdq_poll_until_not_full
		queue_poll_init(smmu, &qp);
		do {
			if(!queue_full(llq))
				break;
			ret = queue_poll(&qp);

		}while (!ret);
	}
	check exit condition

} while (1);

Now, with this patch we reduced to two nested while loops and
calling queue_has_space() only without checking queue_full.

do {
	queue_poll_init(smmu, &qp);
	while (!queue_has_space(&llq, n + sync)) {
		ret = queue_poll(&qp);
		if (ret == -ETIMEDOUT) {
			dev_err();
			queue_poll_init(smmu, &qp);
		} 
	}
	check exit condition
} while (1);

An additional queue_poll_init is added outside inner while loop to arm
the timer. We can merge the two queue_poll_init with a local bool
variable to track whether init is needed, but IMHO it is not any better.

Adding local_irq_save() just to make sure it pairs up with
local_irq_restore(), no functional changes.

> I still feel that we could just replace the _until_not_full()
> with a _until_has_space()?
Since the current code uses three nested while loops, replacing the
inner _until_not_full() function means means retaining all three nested
while loops and calling queue_has_space in two places - once in the
middle while loop then again in this _until_has_space() function.

I tried to extract the inner loop into a function but it requires
passing in irqflags to restore. Not pretty.