From: "Kiryl Shutsemau (Meta)" <kas@kernel.org>
To: Will Deacon <will@kernel.org>,
Robin Murphy <robin.murphy@arm.com>,
Joerg Roedel <joro@8bytes.org>
Cc: Jason Gunthorpe <jgg@nvidia.com>,
Nicolin Chen <nicolinc@nvidia.com>,
Pranjal Shrivastava <praan@google.com>,
Mostafa Saleh <smostafa@google.com>,
Thierry Reding <thierry.reding@kernel.org>,
Krishna Reddy <vdumpa@nvidia.com>,
Jonathan Hunter <jonathanh@nvidia.com>,
Breno Leitao <leitao@debian.org>, Kyle McMartin <jkkm@meta.com>,
Usama Arif <usama.arif@linux.dev>,
kernel-team@meta.com, linux-arm-kernel@lists.infradead.org,
iommu@lists.linux.dev, linux-tegra@vger.kernel.org,
linux-kernel@vger.kernel.org,
"Kiryl Shutsemau (Meta)" <kas@kernel.org>
Subject: [PATCH v4 2/2] iommu/arm-smmu-v3: Default queue depths to one page in a kdump kernel
Date: Wed, 2 Sep 2026 13:17:24 +0100 [thread overview]
Message-ID: <20260902121724.3494954-3-kas@kernel.org> (raw)
In-Reply-To: <20260902121724.3494954-1-kas@kernel.org>
All three queues are sized from the maxima the hardware advertises in IDR1
and allocated at probe, reaching megabytes of coherent DMA each. The
capture kernel already disables two of them: arm_smmu_device_reset() drops
CR0_EVTQEN and CR0_PRIQEN. It still allocates both at full size.
A kdump capture kernel runs from a small crashkernel reservation, and a
system with several SMMUv3 instances pays the full cost per instance. Tens
of megabytes go to queues that either serve the handful of devices used to
save the dump or are switched off outright, and that is memory the dump
itself needs.
Default all three depths to one page worth of entries when
is_kdump_kernel(). The queues carry commands and fault records rather than
DMA data, so dump throughput is unaffected. A shallower command queue only
bounds how many commands may be in flight before a sync, which does not
matter for the few devices that save the dump.
An explicit cmdq_entries still wins, so a capture kernel that wants a
deeper command queue can ask for one on the command line.
Suggested-by: Kyle McMartin <jkkm@meta.com>
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Assisted-by: Claude-Code:claude-opus-5
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 25 +++++++++++++++------
1 file changed, 18 insertions(+), 7 deletions(-)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index b371e9ddbfcc..a786219f40ff 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -4422,7 +4422,10 @@ static struct iommu_dirty_ops arm_smmu_dirty_ops = {
* arm_smmu_queue_max_n_shift() - pick the log2 depth of a queue
* @hw_shift: log2 depth the hardware allows, capped for natural alignment
* @ent_sz_shift: log2 of the queue entry size in bytes
- * @want: number of entries asked for, or zero to use @hw_shift
+ * @want: number of entries asked for, or zero to use the default
+ *
+ * The default is @hw_shift, except in a kdump capture kernel, which defaults
+ * to one page worth of entries.
*
* @want is rounded down to a power of two. It never sizes a queue below one
* page, because coherent DMA is page granular: a shallower queue occupies the
@@ -4432,11 +4435,16 @@ static struct iommu_dirty_ops arm_smmu_dirty_ops = {
static u32 arm_smmu_queue_max_n_shift(u32 hw_shift, u32 ent_sz_shift, u32 want)
{
u32 page_shift = PAGE_SHIFT - ent_sz_shift;
+ u32 want_shift;
- if (!want)
+ if (want)
+ want_shift = max(ilog2(want), page_shift);
+ else if (is_kdump_kernel())
+ want_shift = page_shift;
+ else
return hw_shift;
- return min(hw_shift, max(ilog2(want), page_shift));
+ return min(hw_shift, want_shift);
}
/*
@@ -5208,10 +5216,13 @@ static int arm_smmu_device_hw_probe(struct arm_smmu_device *smmu)
return -ENXIO;
}
- smmu->evtq.q.llq.max_n_shift = min_t(u32, EVTQ_MAX_SZ_SHIFT,
- FIELD_GET(IDR1_EVTQS, reg));
- smmu->priq.q.llq.max_n_shift = min_t(u32, PRIQ_MAX_SZ_SHIFT,
- FIELD_GET(IDR1_PRIQS, reg));
+ hw_shift = min_t(u32, EVTQ_MAX_SZ_SHIFT, FIELD_GET(IDR1_EVTQS, reg));
+ smmu->evtq.q.llq.max_n_shift =
+ arm_smmu_queue_max_n_shift(hw_shift, EVTQ_ENT_SZ_SHIFT, 0);
+
+ hw_shift = min_t(u32, PRIQ_MAX_SZ_SHIFT, FIELD_GET(IDR1_PRIQS, reg));
+ smmu->priq.q.llq.max_n_shift =
+ arm_smmu_queue_max_n_shift(hw_shift, PRIQ_ENT_SZ_SHIFT, 0);
/* SID/SSID sizes */
smmu->ssid_bits = FIELD_GET(IDR1_SSIDSIZE, reg);
--
2.54.0
prev parent reply other threads:[~2026-09-02 12:18 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-09-02 12:17 [PATCH v4 0/2] iommu/arm-smmu-v3: Make the queue depths tunable, and shrink them in a kdump kernel Kiryl Shutsemau (Meta)
2026-09-02 12:17 ` [PATCH v4 1/2] iommu/arm-smmu-v3: Add a cmdq_entries module parameter Kiryl Shutsemau (Meta)
2026-09-02 17:22 ` Nicolin Chen
2026-09-03 14:15 ` Kiryl Shutsemau
2026-09-03 15:09 ` Nicolin Chen
2026-09-04 14:28 ` Kiryl Shutsemau
2026-09-02 12:17 ` Kiryl Shutsemau (Meta) [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260902121724.3494954-3-kas@kernel.org \
--to=kas@kernel.org \
--cc=iommu@lists.linux.dev \
--cc=jgg@nvidia.com \
--cc=jkkm@meta.com \
--cc=jonathanh@nvidia.com \
--cc=joro@8bytes.org \
--cc=kernel-team@meta.com \
--cc=leitao@debian.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-tegra@vger.kernel.org \
--cc=nicolinc@nvidia.com \
--cc=praan@google.com \
--cc=robin.murphy@arm.com \
--cc=smostafa@google.com \
--cc=thierry.reding@kernel.org \
--cc=usama.arif@linux.dev \
--cc=vdumpa@nvidia.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox