iommu.lists.linux-foundation.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v3] iommu/arm-smmu-v3: Shrink command/event/PRI queues in kdump kernel
@ 2026-07-06  8:47 Kiryl Shutsemau (Meta)
  2026-07-15 11:46 ` Kiryl Shutsemau
  2026-07-28 10:16 ` Will Deacon
  0 siblings, 2 replies; 13+ messages in thread
From: Kiryl Shutsemau (Meta) @ 2026-07-06  8:47 UTC (permalink / raw)
  To: Will Deacon, Robin Murphy, Joerg Roedel
  Cc: Jason Gunthorpe, Nicolin Chen, Pranjal Shrivastava, Breno Leitao,
	Kyle McMartin, Usama Arif, linux-arm-kernel, iommu, linux-kernel,
	Kiryl Shutsemau (Meta)

All SMMU queues are sized from the maxima the hardware advertises in IDR1,
which can be several megabytes each, and are allocated at probe. The kdump
kernel already disables the event and PRI queues (arm_smmu_device_reset()
drops CR0_EVTQEN/CR0_PRIQEN) but still allocates them at full size. On
systems with many SMMUv3 instances that cost is paid per instance and adds
up to tens of megabytes of coherent DMA in the capture kernel.

A kdump capture kernel runs from a small crashkernel reservation and only
has to drive the few devices used to save the dump, so deep queues serve
no purpose. The queues are not on the DMA data path, so dump throughput is
unaffected; a shallower command queue only bounds how many commands may be
in flight before a sync, which does not matter for the capture kernel's
small device count and modest I/O.

Clamp every queue to a single page when is_kdump_kernel() is true. Doing
it in arm_smmu_init_one_queue() covers the command, event and PRI queues
in one place. The command queue still holds at least one batch plus a sync
(256 entries on a 4K-page kernel, well above CMDQ_BATCH_ENTRIES), so
command batching keeps working.

Suggested-by: Kyle McMartin <jkkm@meta.com>
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Breno Leitao <leitao@debian.org>
Reviewed-by: Pranjal Shrivastava <praan@google.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
Tested-by: Nicolin Chen <nicolinc@nvidia.com>
---
v3:
 - Reword the commit message: all SMMU queues are sized from IDR1, and the
   event/PRI queues, though disabled in kdump, are still allocated
   (Pranjal Shrivastava).
 - Collect Reviewed-by from Pranjal, Jason and Nicolin; Tested-by from
   Nicolin.

 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index e8d7dbe495f0..a4ec4a59e527 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -4414,6 +4414,20 @@ int arm_smmu_init_one_queue(struct arm_smmu_device *smmu,
 {
 	size_t qsz;
 
+	/*
+	 * A kdump capture kernel runs from a small crashkernel reservation and
+	 * only has to drive the few devices used to save the dump, so there is
+	 * no point sizing the queues for the (multi-megabyte) maxima the
+	 * hardware advertises. Clamp each queue to a single page. ent_sz_shift
+	 * is the log2 of the entry size in bytes (dwords * 8).
+	 */
+	if (is_kdump_kernel()) {
+		u32 ent_sz_shift = ilog2(dwords) + 3;
+
+		q->llq.max_n_shift = min(q->llq.max_n_shift,
+					 PAGE_SHIFT - ent_sz_shift);
+	}
+
 	do {
 		qsz = ((1 << q->llq.max_n_shift) * dwords) << 3;
 		q->base = dmam_alloc_coherent(smmu->dev, qsz, &q->base_dma,
-- 
2.54.0


^ permalink raw reply related	[flat|nested] 13+ messages in thread

* Re: [PATCH v3] iommu/arm-smmu-v3: Shrink command/event/PRI queues in kdump kernel
  2026-07-06  8:47 [PATCH v3] iommu/arm-smmu-v3: Shrink command/event/PRI queues in kdump kernel Kiryl Shutsemau (Meta)
@ 2026-07-15 11:46 ` Kiryl Shutsemau
  2026-07-27  9:30   ` Kiryl Shutsemau
  2026-07-28 10:16 ` Will Deacon
  1 sibling, 1 reply; 13+ messages in thread
From: Kiryl Shutsemau @ 2026-07-15 11:46 UTC (permalink / raw)
  To: Will Deacon, Robin Murphy, Joerg Roedel
  Cc: Jason Gunthorpe, Nicolin Chen, Pranjal Shrivastava, Breno Leitao,
	Kyle McMartin, Usama Arif, linux-arm-kernel, iommu, linux-kernel

On Mon, Jul 06, 2026 at 09:47:08AM +0100, Kiryl Shutsemau (Meta) wrote:
> All SMMU queues are sized from the maxima the hardware advertises in IDR1,
> which can be several megabytes each, and are allocated at probe. The kdump
> kernel already disables the event and PRI queues (arm_smmu_device_reset()
> drops CR0_EVTQEN/CR0_PRIQEN) but still allocates them at full size. On
> systems with many SMMUv3 instances that cost is paid per instance and adds
> up to tens of megabytes of coherent DMA in the capture kernel.
> 
> A kdump capture kernel runs from a small crashkernel reservation and only
> has to drive the few devices used to save the dump, so deep queues serve
> no purpose. The queues are not on the DMA data path, so dump throughput is
> unaffected; a shallower command queue only bounds how many commands may be
> in flight before a sync, which does not matter for the capture kernel's
> small device count and modest I/O.
> 
> Clamp every queue to a single page when is_kdump_kernel() is true. Doing
> it in arm_smmu_init_one_queue() covers the command, event and PRI queues
> in one place. The command queue still holds at least one batch plus a sync
> (256 entries on a 4K-page kernel, well above CMDQ_BATCH_ENTRIES), so
> command batching keeps working.
> 
> Suggested-by: Kyle McMartin <jkkm@meta.com>
> Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
> Reviewed-by: Breno Leitao <leitao@debian.org>
> Reviewed-by: Pranjal Shrivastava <praan@google.com>
> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
> Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
> Tested-by: Nicolin Chen <nicolinc@nvidia.com>

Gentle ping?

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH v3] iommu/arm-smmu-v3: Shrink command/event/PRI queues in kdump kernel
  2026-07-15 11:46 ` Kiryl Shutsemau
@ 2026-07-27  9:30   ` Kiryl Shutsemau
  0 siblings, 0 replies; 13+ messages in thread
From: Kiryl Shutsemau @ 2026-07-27  9:30 UTC (permalink / raw)
  To: Will Deacon, Robin Murphy, Joerg Roedel
  Cc: Jason Gunthorpe, Nicolin Chen, Pranjal Shrivastava, Breno Leitao,
	Kyle McMartin, Usama Arif, linux-arm-kernel, iommu, linux-kernel

On Wed, Jul 15, 2026 at 12:46:13PM +0100, Kiryl Shutsemau wrote:
> On Mon, Jul 06, 2026 at 09:47:08AM +0100, Kiryl Shutsemau (Meta) wrote:
> > All SMMU queues are sized from the maxima the hardware advertises in IDR1,
> > which can be several megabytes each, and are allocated at probe. The kdump
> > kernel already disables the event and PRI queues (arm_smmu_device_reset()
> > drops CR0_EVTQEN/CR0_PRIQEN) but still allocates them at full size. On
> > systems with many SMMUv3 instances that cost is paid per instance and adds
> > up to tens of megabytes of coherent DMA in the capture kernel.
> > 
> > A kdump capture kernel runs from a small crashkernel reservation and only
> > has to drive the few devices used to save the dump, so deep queues serve
> > no purpose. The queues are not on the DMA data path, so dump throughput is
> > unaffected; a shallower command queue only bounds how many commands may be
> > in flight before a sync, which does not matter for the capture kernel's
> > small device count and modest I/O.
> > 
> > Clamp every queue to a single page when is_kdump_kernel() is true. Doing
> > it in arm_smmu_init_one_queue() covers the command, event and PRI queues
> > in one place. The command queue still holds at least one batch plus a sync
> > (256 entries on a 4K-page kernel, well above CMDQ_BATCH_ENTRIES), so
> > command batching keeps working.
> > 
> > Suggested-by: Kyle McMartin <jkkm@meta.com>
> > Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
> > Reviewed-by: Breno Leitao <leitao@debian.org>
> > Reviewed-by: Pranjal Shrivastava <praan@google.com>
> > Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
> > Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
> > Tested-by: Nicolin Chen <nicolinc@nvidia.com>
> 
> Gentle ping?

Hey folks,

Can we get the patch applied?

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH v3] iommu/arm-smmu-v3: Shrink command/event/PRI queues in kdump kernel
  2026-07-06  8:47 [PATCH v3] iommu/arm-smmu-v3: Shrink command/event/PRI queues in kdump kernel Kiryl Shutsemau (Meta)
  2026-07-15 11:46 ` Kiryl Shutsemau
@ 2026-07-28 10:16 ` Will Deacon
  2026-07-28 13:53   ` Kiryl Shutsemau
  1 sibling, 1 reply; 13+ messages in thread
From: Will Deacon @ 2026-07-28 10:16 UTC (permalink / raw)
  To: Kiryl Shutsemau (Meta)
  Cc: Robin Murphy, Joerg Roedel, Jason Gunthorpe, Nicolin Chen,
	Pranjal Shrivastava, Breno Leitao, Kyle McMartin, Usama Arif,
	linux-arm-kernel, iommu, linux-kernel

On Mon, Jul 06, 2026 at 09:47:08AM +0100, Kiryl Shutsemau (Meta) wrote:
> All SMMU queues are sized from the maxima the hardware advertises in IDR1,
> which can be several megabytes each, and are allocated at probe. The kdump
> kernel already disables the event and PRI queues (arm_smmu_device_reset()
> drops CR0_EVTQEN/CR0_PRIQEN) but still allocates them at full size. On
> systems with many SMMUv3 instances that cost is paid per instance and adds
> up to tens of megabytes of coherent DMA in the capture kernel.
> 
> A kdump capture kernel runs from a small crashkernel reservation and only
> has to drive the few devices used to save the dump, so deep queues serve
> no purpose. The queues are not on the DMA data path, so dump throughput is
> unaffected; a shallower command queue only bounds how many commands may be
> in flight before a sync, which does not matter for the capture kernel's
> small device count and modest I/O.
> 
> Clamp every queue to a single page when is_kdump_kernel() is true. Doing
> it in arm_smmu_init_one_queue() covers the command, event and PRI queues
> in one place. The command queue still holds at least one batch plus a sync
> (256 entries on a 4K-page kernel, well above CMDQ_BATCH_ENTRIES), so
> command batching keeps working.

Wouldn't we be better of not allocating unused queues in the first place?
That's what this patch does:

https://lore.kernel.org/r/0b035c53cb401acde8244b805d4b6a0312b83708.1782799827.git.nicolinc@nvidia.com

so help reviewing that series would be much appreciated!

If you want to reduce the cmdq size, I'd prefer a cmdline option rather
than special-casing kdump (as I've had other folks ask about configuring
the cmdq size for other reasons).

Will

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH v3] iommu/arm-smmu-v3: Shrink command/event/PRI queues in kdump kernel
  2026-07-28 10:16 ` Will Deacon
@ 2026-07-28 13:53   ` Kiryl Shutsemau
  2026-08-04 14:02     ` Will Deacon
  0 siblings, 1 reply; 13+ messages in thread
From: Kiryl Shutsemau @ 2026-07-28 13:53 UTC (permalink / raw)
  To: Will Deacon
  Cc: Robin Murphy, Joerg Roedel, Jason Gunthorpe, Nicolin Chen,
	Pranjal Shrivastava, Breno Leitao, Kyle McMartin, Usama Arif,
	linux-arm-kernel, iommu, linux-kernel

On Tue, Jul 28, 2026 at 11:16:10AM +0100, Will Deacon wrote:
> On Mon, Jul 06, 2026 at 09:47:08AM +0100, Kiryl Shutsemau (Meta) wrote:
> > All SMMU queues are sized from the maxima the hardware advertises in IDR1,
> > which can be several megabytes each, and are allocated at probe. The kdump
> > kernel already disables the event and PRI queues (arm_smmu_device_reset()
> > drops CR0_EVTQEN/CR0_PRIQEN) but still allocates them at full size. On
> > systems with many SMMUv3 instances that cost is paid per instance and adds
> > up to tens of megabytes of coherent DMA in the capture kernel.
> > 
> > A kdump capture kernel runs from a small crashkernel reservation and only
> > has to drive the few devices used to save the dump, so deep queues serve
> > no purpose. The queues are not on the DMA data path, so dump throughput is
> > unaffected; a shallower command queue only bounds how many commands may be
> > in flight before a sync, which does not matter for the capture kernel's
> > small device count and modest I/O.
> > 
> > Clamp every queue to a single page when is_kdump_kernel() is true. Doing
> > it in arm_smmu_init_one_queue() covers the command, event and PRI queues
> > in one place. The command queue still holds at least one batch plus a sync
> > (256 entries on a 4K-page kernel, well above CMDQ_BATCH_ENTRIES), so
> > command batching keeps working.
> 
> Wouldn't we be better of not allocating unused queues in the first place?
> That's what this patch does:
> 
> https://lore.kernel.org/r/0b035c53cb401acde8244b805d4b6a0312b83708.1782799827.git.nicolinc@nvidia.com

Maybe I'm reading it wrong, but I couldn't find the allocation change in
there. That patch seems to only touch arm_smmu_device_reset() -- skipping
the EVTQ/PRIQ base/prod/cons writes and the CR0_EVTQEN/CR0_PRIQEN enables
instead of the current enable-then-mask-out.

I went through the rest of the series too and didn't see
arm_smmu_init_queues() or arm_smmu_init_one_queue() touched anywhere, so
as far as I can tell the kdump kernel still allocates full-size
evtq/priq, and the cmdq is not affected either.

Is the allocation side handled somewhere else that I've missed?

> so help reviewing that series would be much appreciated!
> 
> If you want to reduce the cmdq size, I'd prefer a cmdline option rather
> than special-casing kdump (as I've had other folks ask about configuring
> the cmdq size for other reasons).

Fair enough for the cmdq. I'm less sure what to do about evtq/priq: they
get disabled in the kdump kernel anyway, so I don't know what a size knob
would mean for them, other than every kdump config having to pass it to
get back the memory.

Would skipping their allocation when is_kdump_kernel() be acceptable on
its own, or would you rather see it done differently?

-- 
  Kiryl Shutsemau / Kirill A. Shutemov

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH v3] iommu/arm-smmu-v3: Shrink command/event/PRI queues in kdump kernel
  2026-07-28 13:53   ` Kiryl Shutsemau
@ 2026-08-04 14:02     ` Will Deacon
  2026-08-04 14:17       ` Jason Gunthorpe
  0 siblings, 1 reply; 13+ messages in thread
From: Will Deacon @ 2026-08-04 14:02 UTC (permalink / raw)
  To: Kiryl Shutsemau
  Cc: Robin Murphy, Joerg Roedel, Jason Gunthorpe, Nicolin Chen,
	Pranjal Shrivastava, Breno Leitao, Kyle McMartin, Usama Arif,
	linux-arm-kernel, iommu, linux-kernel

On Tue, Jul 28, 2026 at 02:53:16PM +0100, Kiryl Shutsemau wrote:
> On Tue, Jul 28, 2026 at 11:16:10AM +0100, Will Deacon wrote:
> > On Mon, Jul 06, 2026 at 09:47:08AM +0100, Kiryl Shutsemau (Meta) wrote:
> > > All SMMU queues are sized from the maxima the hardware advertises in IDR1,
> > > which can be several megabytes each, and are allocated at probe. The kdump
> > > kernel already disables the event and PRI queues (arm_smmu_device_reset()
> > > drops CR0_EVTQEN/CR0_PRIQEN) but still allocates them at full size. On
> > > systems with many SMMUv3 instances that cost is paid per instance and adds
> > > up to tens of megabytes of coherent DMA in the capture kernel.
> > > 
> > > A kdump capture kernel runs from a small crashkernel reservation and only
> > > has to drive the few devices used to save the dump, so deep queues serve
> > > no purpose. The queues are not on the DMA data path, so dump throughput is
> > > unaffected; a shallower command queue only bounds how many commands may be
> > > in flight before a sync, which does not matter for the capture kernel's
> > > small device count and modest I/O.
> > > 
> > > Clamp every queue to a single page when is_kdump_kernel() is true. Doing
> > > it in arm_smmu_init_one_queue() covers the command, event and PRI queues
> > > in one place. The command queue still holds at least one batch plus a sync
> > > (256 entries on a 4K-page kernel, well above CMDQ_BATCH_ENTRIES), so
> > > command batching keeps working.
> > 
> > Wouldn't we be better of not allocating unused queues in the first place?
> > That's what this patch does:
> > 
> > https://lore.kernel.org/r/0b035c53cb401acde8244b805d4b6a0312b83708.1782799827.git.nicolinc@nvidia.com
> 
> Maybe I'm reading it wrong, but I couldn't find the allocation change in
> there. That patch seems to only touch arm_smmu_device_reset() -- skipping
> the EVTQ/PRIQ base/prod/cons writes and the CR0_EVTQEN/CR0_PRIQEN enables
> instead of the current enable-then-mask-out.
> 
> I went through the rest of the series too and didn't see
> arm_smmu_init_queues() or arm_smmu_init_one_queue() touched anywhere, so
> as far as I can tell the kdump kernel still allocates full-size
> evtq/priq, and the cmdq is not affected either.
> 
> Is the allocation side handled somewhere else that I've missed?

Sorry, yes, it would need extending to avoid allocating the queues now
that they're not used at all. I was hoping you and Nicolin could work
together on that.

> > If you want to reduce the cmdq size, I'd prefer a cmdline option rather
> > than special-casing kdump (as I've had other folks ask about configuring
> > the cmdq size for other reasons).
> 
> Fair enough for the cmdq. I'm less sure what to do about evtq/priq: they
> get disabled in the kdump kernel anyway, so I don't know what a size knob
> would mean for them, other than every kdump config having to pass it to
> get back the memory.
> 
> Would skipping their allocation when is_kdump_kernel() be acceptable on
> its own, or would you rather see it done differently?

With Nicolin's patch, we can just avoid allocating the priq and the evtq
entirely in the kdump case. We can then add a cmdline option to control
the maximum size of the cmdq, which is useful regardless of kdump.

Will

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH v3] iommu/arm-smmu-v3: Shrink command/event/PRI queues in kdump kernel
  2026-08-04 14:02     ` Will Deacon
@ 2026-08-04 14:17       ` Jason Gunthorpe
  2026-08-04 15:23         ` Robin Murphy
  2026-08-04 16:04         ` Nicolin Chen
  0 siblings, 2 replies; 13+ messages in thread
From: Jason Gunthorpe @ 2026-08-04 14:17 UTC (permalink / raw)
  To: Will Deacon
  Cc: Kiryl Shutsemau, Robin Murphy, Joerg Roedel, Nicolin Chen,
	Pranjal Shrivastava, Breno Leitao, Kyle McMartin, Usama Arif,
	linux-arm-kernel, iommu, linux-kernel

On Tue, Aug 04, 2026 at 03:02:46PM +0100, Will Deacon wrote:
> > Would skipping their allocation when is_kdump_kernel() be acceptable on
> > its own, or would you rather see it done differently?
> 
> With Nicolin's patch, we can just avoid allocating the priq and the evtq
> entirely in the kdump case. We can then add a cmdline option to control
> the maximum size of the cmdq, which is useful regardless of kdump.

I would still prefer kdump do this re-sizing automatically, even if we
do add a commandline. It makes it easire to deploy than having to know
secret command lines :\

So functionally I still think this patch is fine, though yes it points
out more things in Nicolin's work too.

Jason

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH v3] iommu/arm-smmu-v3: Shrink command/event/PRI queues in kdump kernel
  2026-08-04 14:17       ` Jason Gunthorpe
@ 2026-08-04 15:23         ` Robin Murphy
  2026-08-04 16:04         ` Nicolin Chen
  1 sibling, 0 replies; 13+ messages in thread
From: Robin Murphy @ 2026-08-04 15:23 UTC (permalink / raw)
  To: Jason Gunthorpe, Will Deacon
  Cc: Kiryl Shutsemau, Joerg Roedel, Nicolin Chen, Pranjal Shrivastava,
	Breno Leitao, Kyle McMartin, Usama Arif, linux-arm-kernel, iommu,
	linux-kernel

On 04/08/2026 3:17 pm, Jason Gunthorpe wrote:
> On Tue, Aug 04, 2026 at 03:02:46PM +0100, Will Deacon wrote:
>>> Would skipping their allocation when is_kdump_kernel() be acceptable on
>>> its own, or would you rather see it done differently?
>>
>> With Nicolin's patch, we can just avoid allocating the priq and the evtq
>> entirely in the kdump case. We can then add a cmdline option to control
>> the maximum size of the cmdq, which is useful regardless of kdump.
> 
> I would still prefer kdump do this re-sizing automatically, even if we
> do add a commandline. It makes it easire to deploy than having to know
> secret command lines :\

Right, but if we implement a general command-line option/module 
parameter/whatever with a variable that needs a default value for when 
it's not overridden, then it's even more trivial to initialise *that* 
default value based on further conditions as desired.

Plus implementing the more dynamic mechanism to begin with is arguably 
nicer than hard-coding a kdump special case in what is effectively the 
wrong place, since semantically what we really want to 
override/generalise are the max_n_shift initialisations in 
arm_smmu_device_hw_probe() itself.

Thanks,
Robin.

> So functionally I still think this patch is fine, though yes it points
> out more things in Nicolin's work too.
> 
> Jason


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH v3] iommu/arm-smmu-v3: Shrink command/event/PRI queues in kdump kernel
  2026-08-04 14:17       ` Jason Gunthorpe
  2026-08-04 15:23         ` Robin Murphy
@ 2026-08-04 16:04         ` Nicolin Chen
  2026-08-04 16:19           ` Jason Gunthorpe
  1 sibling, 1 reply; 13+ messages in thread
From: Nicolin Chen @ 2026-08-04 16:04 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Will Deacon, Kiryl Shutsemau, Robin Murphy, Joerg Roedel,
	Pranjal Shrivastava, Breno Leitao, Kyle McMartin, Usama Arif,
	linux-arm-kernel, iommu, linux-kernel

On Tue, Aug 04, 2026 at 11:17:04AM -0300, Jason Gunthorpe wrote:
> On Tue, Aug 04, 2026 at 03:02:46PM +0100, Will Deacon wrote:
> > > Would skipping their allocation when is_kdump_kernel() be acceptable on
> > > its own, or would you rather see it done differently?
> > 
> > With Nicolin's patch, we can just avoid allocating the priq and the evtq
> > entirely in the kdump case. We can then add a cmdline option to control
> > the maximum size of the cmdq, which is useful regardless of kdump.
> 
> I would still prefer kdump do this re-sizing automatically, even if we
> do add a commandline. It makes it easire to deploy than having to know
> secret command lines :\
> 
> So functionally I still think this patch is fine, though yes it points
> out more things in Nicolin's work too.

I will send a v10 of mine, to skip the EVTQ/PRIQ allocations.

Thanks
Nicolin

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH v3] iommu/arm-smmu-v3: Shrink command/event/PRI queues in kdump kernel
  2026-08-04 16:04         ` Nicolin Chen
@ 2026-08-04 16:19           ` Jason Gunthorpe
  2026-08-04 16:45             ` Nicolin Chen
  0 siblings, 1 reply; 13+ messages in thread
From: Jason Gunthorpe @ 2026-08-04 16:19 UTC (permalink / raw)
  To: Nicolin Chen
  Cc: Will Deacon, Kiryl Shutsemau, Robin Murphy, Joerg Roedel,
	Pranjal Shrivastava, Breno Leitao, Kyle McMartin, Usama Arif,
	linux-arm-kernel, iommu, linux-kernel

On Tue, Aug 04, 2026 at 09:04:32AM -0700, Nicolin Chen wrote:
> On Tue, Aug 04, 2026 at 11:17:04AM -0300, Jason Gunthorpe wrote:
> > On Tue, Aug 04, 2026 at 03:02:46PM +0100, Will Deacon wrote:
> > > > Would skipping their allocation when is_kdump_kernel() be acceptable on
> > > > its own, or would you rather see it done differently?
> > > 
> > > With Nicolin's patch, we can just avoid allocating the priq and the evtq
> > > entirely in the kdump case. We can then add a cmdline option to control
> > > the maximum size of the cmdq, which is useful regardless of kdump.
> > 
> > I would still prefer kdump do this re-sizing automatically, even if we
> > do add a commandline. It makes it easire to deploy than having to know
> > secret command lines :\
> > 
> > So functionally I still think this patch is fine, though yes it points
> > out more things in Nicolin's work too.
> 
> I will send a v10 of mine, to skip the EVTQ/PRIQ allocations.

I think Robin's point is to structure the v10 so you introduce a bunch
of tunables, including something like 0 size to disable a feature.

Then have kdump default those tunables to the right thing

Jason

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH v3] iommu/arm-smmu-v3: Shrink command/event/PRI queues in kdump kernel
  2026-08-04 16:19           ` Jason Gunthorpe
@ 2026-08-04 16:45             ` Nicolin Chen
  2026-08-04 16:59               ` Jason Gunthorpe
  0 siblings, 1 reply; 13+ messages in thread
From: Nicolin Chen @ 2026-08-04 16:45 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Will Deacon, Kiryl Shutsemau, Robin Murphy, Joerg Roedel,
	Pranjal Shrivastava, Breno Leitao, Kyle McMartin, Usama Arif,
	linux-arm-kernel, iommu, linux-kernel

On Tue, Aug 04, 2026 at 01:19:18PM -0300, Jason Gunthorpe wrote:
> On Tue, Aug 04, 2026 at 09:04:32AM -0700, Nicolin Chen wrote:
> > On Tue, Aug 04, 2026 at 11:17:04AM -0300, Jason Gunthorpe wrote:
> > > On Tue, Aug 04, 2026 at 03:02:46PM +0100, Will Deacon wrote:
> > > > > Would skipping their allocation when is_kdump_kernel() be acceptable on
> > > > > its own, or would you rather see it done differently?
> > > > 
> > > > With Nicolin's patch, we can just avoid allocating the priq and the evtq
> > > > entirely in the kdump case. We can then add a cmdline option to control
> > > > the maximum size of the cmdq, which is useful regardless of kdump.
> > > 
> > > I would still prefer kdump do this re-sizing automatically, even if we
> > > do add a commandline. It makes it easire to deploy than having to know
> > > secret command lines :\
> > > 
> > > So functionally I still think this patch is fine, though yes it points
> > > out more things in Nicolin's work too.
> > 
> > I will send a v10 of mine, to skip the EVTQ/PRIQ allocations.
> 
> I think Robin's point is to structure the v10 so you introduce a bunch
> of tunables, including something like 0 size to disable a feature.
>
> Then have kdump default those tunables to the right thing

But in kdump, EVTQ/PRIQ are disabled, so my series should have
skipped their allocations regardless of the tunables.

So, I think the tunables can be done in a separate series. And
its use case might not be confined to kdump?

Nicolin

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH v3] iommu/arm-smmu-v3: Shrink command/event/PRI queues in kdump kernel
  2026-08-04 16:45             ` Nicolin Chen
@ 2026-08-04 16:59               ` Jason Gunthorpe
  2026-08-04 17:39                 ` Nicolin Chen
  0 siblings, 1 reply; 13+ messages in thread
From: Jason Gunthorpe @ 2026-08-04 16:59 UTC (permalink / raw)
  To: Nicolin Chen
  Cc: Will Deacon, Kiryl Shutsemau, Robin Murphy, Joerg Roedel,
	Pranjal Shrivastava, Breno Leitao, Kyle McMartin, Usama Arif,
	linux-arm-kernel, iommu, linux-kernel

On Tue, Aug 04, 2026 at 09:45:01AM -0700, Nicolin Chen wrote:
> On Tue, Aug 04, 2026 at 01:19:18PM -0300, Jason Gunthorpe wrote:
> > On Tue, Aug 04, 2026 at 09:04:32AM -0700, Nicolin Chen wrote:
> > > On Tue, Aug 04, 2026 at 11:17:04AM -0300, Jason Gunthorpe wrote:
> > > > On Tue, Aug 04, 2026 at 03:02:46PM +0100, Will Deacon wrote:
> > > > > > Would skipping their allocation when is_kdump_kernel() be acceptable on
> > > > > > its own, or would you rather see it done differently?
> > > > > 
> > > > > With Nicolin's patch, we can just avoid allocating the priq and the evtq
> > > > > entirely in the kdump case. We can then add a cmdline option to control
> > > > > the maximum size of the cmdq, which is useful regardless of kdump.
> > > > 
> > > > I would still prefer kdump do this re-sizing automatically, even if we
> > > > do add a commandline. It makes it easire to deploy than having to know
> > > > secret command lines :\
> > > > 
> > > > So functionally I still think this patch is fine, though yes it points
> > > > out more things in Nicolin's work too.
> > > 
> > > I will send a v10 of mine, to skip the EVTQ/PRIQ allocations.
> > 
> > I think Robin's point is to structure the v10 so you introduce a bunch
> > of tunables, including something like 0 size to disable a feature.
> >
> > Then have kdump default those tunables to the right thing
> 
> But in kdump, EVTQ/PRIQ are disabled, so my series should have
> skipped their allocations regardless of the tunables.
> 
> So, I think the tunables can be done in a separate series. And
> its use case might not be confined to kdump?

I think the ask is to do the tunables first and change your series to
default the tunables instead of doing direct kdump overrides.

IDK that anyone wants commandline tunables, but it doesn't seem like a
big task

Jason

^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH v3] iommu/arm-smmu-v3: Shrink command/event/PRI queues in kdump kernel
  2026-08-04 16:59               ` Jason Gunthorpe
@ 2026-08-04 17:39                 ` Nicolin Chen
  0 siblings, 0 replies; 13+ messages in thread
From: Nicolin Chen @ 2026-08-04 17:39 UTC (permalink / raw)
  To: Jason Gunthorpe
  Cc: Will Deacon, Kiryl Shutsemau, Robin Murphy, Joerg Roedel,
	Pranjal Shrivastava, Breno Leitao, Kyle McMartin, Usama Arif,
	linux-arm-kernel, iommu, linux-kernel

On Tue, Aug 04, 2026 at 01:59:05PM -0300, Jason Gunthorpe wrote:
> On Tue, Aug 04, 2026 at 09:45:01AM -0700, Nicolin Chen wrote:
> > On Tue, Aug 04, 2026 at 01:19:18PM -0300, Jason Gunthorpe wrote:
> > > On Tue, Aug 04, 2026 at 09:04:32AM -0700, Nicolin Chen wrote:
> > > > On Tue, Aug 04, 2026 at 11:17:04AM -0300, Jason Gunthorpe wrote:
> > > > > On Tue, Aug 04, 2026 at 03:02:46PM +0100, Will Deacon wrote:
> > > > > > > Would skipping their allocation when is_kdump_kernel() be acceptable on
> > > > > > > its own, or would you rather see it done differently?
> > > > > > 
> > > > > > With Nicolin's patch, we can just avoid allocating the priq and the evtq
> > > > > > entirely in the kdump case. We can then add a cmdline option to control
> > > > > > the maximum size of the cmdq, which is useful regardless of kdump.
> > > > > 
> > > > > I would still prefer kdump do this re-sizing automatically, even if we
> > > > > do add a commandline. It makes it easire to deploy than having to know
> > > > > secret command lines :\
> > > > > 
> > > > > So functionally I still think this patch is fine, though yes it points
> > > > > out more things in Nicolin's work too.
> > > > 
> > > > I will send a v10 of mine, to skip the EVTQ/PRIQ allocations.
> > > 
> > > I think Robin's point is to structure the v10 so you introduce a bunch
> > > of tunables, including something like 0 size to disable a feature.
> > >
> > > Then have kdump default those tunables to the right thing
> > 
> > But in kdump, EVTQ/PRIQ are disabled, so my series should have
> > skipped their allocations regardless of the tunables.
> > 
> > So, I think the tunables can be done in a separate series. And
> > its use case might not be confined to kdump?
> 
> I think the ask is to do the tunables first and change your series to
> default the tunables instead of doing direct kdump overrides.

You mean the driver can override the tunables to 0 in kdump mode,
right?

Well, we are skipping a lot of things in kdump mode already; it's
a very small change to skip EVTQ/PRIQ even without the tunables.

If Will prefers this merging sequence, I wouldn't mind changing my
series to overriding the tunables.

Thanks
Nicolin

^ permalink raw reply	[flat|nested] 13+ messages in thread

end of thread, other threads:[~2026-08-04 17:39 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-07-06  8:47 [PATCH v3] iommu/arm-smmu-v3: Shrink command/event/PRI queues in kdump kernel Kiryl Shutsemau (Meta)
2026-07-15 11:46 ` Kiryl Shutsemau
2026-07-27  9:30   ` Kiryl Shutsemau
2026-07-28 10:16 ` Will Deacon
2026-07-28 13:53   ` Kiryl Shutsemau
2026-08-04 14:02     ` Will Deacon
2026-08-04 14:17       ` Jason Gunthorpe
2026-08-04 15:23         ` Robin Murphy
2026-08-04 16:04         ` Nicolin Chen
2026-08-04 16:19           ` Jason Gunthorpe
2026-08-04 16:45             ` Nicolin Chen
2026-08-04 16:59               ` Jason Gunthorpe
2026-08-04 17:39                 ` Nicolin Chen

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).