[PATCH AUTOSEL 6.17] iommu/vt-d: Remove LPIG from page group response descriptor

[PATCH AUTOSEL 6.17] iommu/vt-d: Remove LPIG from page group response descriptor

[Sasha Levin]

From: Lu Baolu <baolu.lu@linux.intel.com>

[ Upstream commit 4402e8f39d0bfff5c0a5edb5e1afe27a56545e11 ]

Bit 66 in the page group response descriptor used to be the LPIG (Last
Page in Group), but it was marked as Reserved since Specification 4.0.
Remove programming on this bit to make it consistent with the latest
specification.

Existing hardware all treats bit 66 of the page group response descriptor
as "ignored", therefore this change doesn't break any existing hardware.

Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Link: https://lore.kernel.org/r/20250901053943.1708490-1-baolu.lu@linux.intel.com
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---

LLM Generated explanations, may be completely bogus:

YES
- VT-d responses no longer program bit 66 (`QI_PGRP_LPIG`), which the
  Intel spec has marked reserved since 4.0; both the macro removal in
  `drivers/iommu/intel/iommu.h:455-463` and the descriptor writes in
  `drivers/iommu/intel/prq.c:149-155` and
  `drivers/iommu/intel/prq.c:372-395` now guarantee the reserved bit is
  left zero. That brings the driver back into spec compliance and avoids
  undefined behaviour on newer hardware that enforces the “reserved must
  be zero” rule for page-request responses.
- Without this change the kernel still reflected `req->lpig` into the
  response descriptor (`drivers/iommu/intel/prq.c:149-155`, `372-395`
  before the patch), so a “last-page” response would carry a ‘1’ in a
  field the spec now forbids. VT-d PRQ handshakes are sensitive: if the
  IOMMU rejects the response, devices stall waiting for completion, so
  this is a real bug for any implementation following the latest spec.
  Existing hardware already ignores the bit, so clearing it cannot
  regress older systems.
- The fix is tightly scoped to the Intel VT-d page-request path, keeps
  the driver’s outward behaviour (e.g. still reporting
  `IOMMU_FAULT_PAGE_REQUEST_LAST_PAGE` to device drivers at
  `drivers/iommu/intel/prq.c:187-194`), and has no dependencies beyond
  trivial code motion. Backporting simply drops the `QI_PGRP_LPIG()`
  usage in the equivalent response paths (older stable trees have the
  same logic in `svm.c`), so the risk of regression is minimal while the
  upside is support for spec-compliant hardware.

 drivers/iommu/intel/iommu.h | 1 -
 drivers/iommu/intel/prq.c   | 7 ++-----
 2 files changed, 2 insertions(+), 6 deletions(-)

diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h
index 2c261c069001c..21b2c3f85ddc5 100644
--- a/drivers/iommu/intel/iommu.h
+++ b/drivers/iommu/intel/iommu.h
@@ -462,7 +462,6 @@ enum {
 #define QI_PGRP_PASID(pasid)	(((u64)(pasid)) << 32)
 
 /* Page group response descriptor QW1 */
-#define QI_PGRP_LPIG(x)		(((u64)(x)) << 2)
 #define QI_PGRP_IDX(idx)	(((u64)(idx)) << 3)
 
 
diff --git a/drivers/iommu/intel/prq.c b/drivers/iommu/intel/prq.c
index 52570e42a14c0..ff63c228e6e19 100644
--- a/drivers/iommu/intel/prq.c
+++ b/drivers/iommu/intel/prq.c
@@ -151,8 +151,7 @@ static void handle_bad_prq_event(struct intel_iommu *iommu,
 			QI_PGRP_PASID_P(req->pasid_present) |
 			QI_PGRP_RESP_CODE(result) |
 			QI_PGRP_RESP_TYPE;
-	desc.qw1 = QI_PGRP_IDX(req->prg_index) |
-			QI_PGRP_LPIG(req->lpig);
+	desc.qw1 = QI_PGRP_IDX(req->prg_index);
 
 	qi_submit_sync(iommu, &desc, 1, 0);
 }
@@ -379,19 +378,17 @@ void intel_iommu_page_response(struct device *dev, struct iopf_fault *evt,
 	struct iommu_fault_page_request *prm;
 	struct qi_desc desc;
 	bool pasid_present;
-	bool last_page;
 	u16 sid;
 
 	prm = &evt->fault.prm;
 	sid = PCI_DEVID(bus, devfn);
 	pasid_present = prm->flags & IOMMU_FAULT_PAGE_REQUEST_PASID_VALID;
-	last_page = prm->flags & IOMMU_FAULT_PAGE_REQUEST_LAST_PAGE;
 
 	desc.qw0 = QI_PGRP_PASID(prm->pasid) | QI_PGRP_DID(sid) |
 			QI_PGRP_PASID_P(pasid_present) |
 			QI_PGRP_RESP_CODE(msg->code) |
 			QI_PGRP_RESP_TYPE;
-	desc.qw1 = QI_PGRP_IDX(prm->grpid) | QI_PGRP_LPIG(last_page);
+	desc.qw1 = QI_PGRP_IDX(prm->grpid);
 	desc.qw2 = 0;
 	desc.qw3 = 0;
 
-- 
2.51.0

[PATCH AUTOSEL 6.17-5.10] iommu/amd: Skip enabling command/event buffers for kdump

[Sasha Levin]

From: Ashish Kalra <ashish.kalra@amd.com>

[ Upstream commit 9be15fbfc6c5c89c22cf6e209f66ea43ee0e58bb ]

After a panic if SNP is enabled in the previous kernel then the kdump
kernel boots with IOMMU SNP enforcement still enabled.

IOMMU command buffers and event buffer registers remain locked and
exclusive to the previous kernel. Attempts to enable command and event
buffers in the kdump kernel will fail, as hardware ignores writes to
the locked MMIO registers as per AMD IOMMU spec Section 2.12.2.1.

Skip enabling command buffers and event buffers for kdump boot as they
are already enabled in the previous kernel.

Reviewed-by: Vasant Hegde <vasant.hegde@amd.com>
Tested-by: Sairaj Kodilkar <sarunkod@amd.com>
Signed-off-by: Ashish Kalra <ashish.kalra@amd.com>
Link: https://lore.kernel.org/r/576445eb4f168b467b0fc789079b650ca7c5b037.1756157913.git.ashish.kalra@amd.com
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---

LLM Generated explanations, may be completely bogus:

YES

- What it fixes
  - Kdump boot after a panic with AMD SNP: IOMMU command/event buffer
    base registers remain locked to the previous kernel, so programming
    them in the crash kernel is ignored (per AMD IOMMU spec 2.12.2.1).
    This prevents enabling command/event buffers and breaks IOMMU
    operation in the crash kernel.

- Key changes
  - Skips writing command buffer base in kdump:
    `drivers/iommu/amd/init.c:824-833`. The write to
    `MMIO_CMD_BUF_OFFSET` is now gated by `if (!is_kdump_kernel())`,
    while still resetting and enabling the ring via
    `amd_iommu_reset_cmd_buffer()` (`drivers/iommu/amd/init.c:835`).
  - Skips writing event buffer base in kdump:
    `drivers/iommu/amd/init.c:884-892`. Similarly, the write to
    `MMIO_EVT_BUF_OFFSET` is skipped in kdump; head/tail registers are
    cleared and logging enabled (`drivers/iommu/amd/init.c:894-899`).

- Why it’s correct and low risk
  - The driver already reuses/remaps the previous kernel’s buffers in
    kdump:
    - Event buffer remap from existing MMIO base:
      `drivers/iommu/amd/init.c:987-996`.
    - Command buffer remap from existing MMIO base:
      `drivers/iommu/amd/init.c:998-1006`.
    - Kdump buffer provisioning path:
      `drivers/iommu/amd/init.c:1039-1050`.
  - With those remaps, `iommu->cmd_buf` and `iommu->evt_buf` point to
    the same memory the hardware is locked to, so skipping the base
    register writes is necessary and safe; the driver still resets
    head/tail and enables the features so operation resumes as expected.
  - This matches existing kdump policy to avoid touching locked
    registers, e.g. device table programming is skipped in kdump:
    `drivers/iommu/amd/init.c:409`.
  - Scope is small, localized to AMD IOMMU init paths, and guarded by
    `is_kdump_kernel()`, so normal boots are unaffected.

- User impact and stability criteria
  - Fixes a real reliability bug in crash dump scenarios on SNP-enabled
    systems; improves kdump robustness without adding features or
    architectural changes.
  - Changes are minimal and well-contained; only affect the kdump path.
  - No ABI or interface changes; limited to initialization register
    programming avoidance in kdump.

- Dependencies/considerations for backport
  - Ensure the kdump remap paths for command/event/CWB buffers are
    present so that `iommu->cmd_buf` and `iommu->evt_buf` reference the
    pre-existing hardware buffer addresses (see
    `drivers/iommu/amd/init.c:987-996`, `998-1006`, `1039-1050`). This
    commit relies on those existing mechanisms; backport alongside those
    if they’re not already in the target stable tree.

Given the above, this is a good stable backport candidate: important
bugfix, minimal risk, and confined to the kdump path for AMD IOMMU.

 drivers/iommu/amd/init.c | 28 +++++++++++++++++++---------
 1 file changed, 19 insertions(+), 9 deletions(-)

diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c
index 309951e57f301..d0cd40ee0dec6 100644
--- a/drivers/iommu/amd/init.c
+++ b/drivers/iommu/amd/init.c
@@ -815,11 +815,16 @@ static void iommu_enable_command_buffer(struct amd_iommu *iommu)
 
 	BUG_ON(iommu->cmd_buf == NULL);
 
-	entry = iommu_virt_to_phys(iommu->cmd_buf);
-	entry |= MMIO_CMD_SIZE_512;
-
-	memcpy_toio(iommu->mmio_base + MMIO_CMD_BUF_OFFSET,
-		    &entry, sizeof(entry));
+	if (!is_kdump_kernel()) {
+		/*
+		 * Command buffer is re-used for kdump kernel and setting
+		 * of MMIO register is not required.
+		 */
+		entry = iommu_virt_to_phys(iommu->cmd_buf);
+		entry |= MMIO_CMD_SIZE_512;
+		memcpy_toio(iommu->mmio_base + MMIO_CMD_BUF_OFFSET,
+			    &entry, sizeof(entry));
+	}
 
 	amd_iommu_reset_cmd_buffer(iommu);
 }
@@ -870,10 +875,15 @@ static void iommu_enable_event_buffer(struct amd_iommu *iommu)
 
 	BUG_ON(iommu->evt_buf == NULL);
 
-	entry = iommu_virt_to_phys(iommu->evt_buf) | EVT_LEN_MASK;
-
-	memcpy_toio(iommu->mmio_base + MMIO_EVT_BUF_OFFSET,
-		    &entry, sizeof(entry));
+	if (!is_kdump_kernel()) {
+		/*
+		 * Event buffer is re-used for kdump kernel and setting
+		 * of MMIO register is not required.
+		 */
+		entry = iommu_virt_to_phys(iommu->evt_buf) | EVT_LEN_MASK;
+		memcpy_toio(iommu->mmio_base + MMIO_EVT_BUF_OFFSET,
+			    &entry, sizeof(entry));
+	}
 
 	/* set head and tail to zero manually */
 	writel(0x00, iommu->mmio_base + MMIO_EVT_HEAD_OFFSET);
-- 
2.51.0

[PATCH AUTOSEL 6.17] iommu/amd: Reuse device table for kdump

[Sasha Levin]

From: Ashish Kalra <ashish.kalra@amd.com>

[ Upstream commit 38e5f33ee3596f37ee8d1e694073a17590904004 ]

After a panic if SNP is enabled in the previous kernel then the kdump
kernel boots with IOMMU SNP enforcement still enabled.

IOMMU device table register is locked and exclusive to the previous
kernel. Attempts to copy old device table from the previous kernel
fails in kdump kernel as hardware ignores writes to the locked device
table base address register as per AMD IOMMU spec Section 2.12.2.1.

This causes the IOMMU driver (OS) and the hardware to reference
different memory locations. As a result, the IOMMU hardware cannot
process the command which results in repeated "Completion-Wait loop
timed out" errors and a second kernel panic: "Kernel panic - not
syncing: timer doesn't work through Interrupt-remapped IO-APIC".

Reuse device table instead of copying device table in case of kdump
boot and remove all copying device table code.

Reviewed-by: Vasant Hegde <vasant.hegde@amd.com>
Tested-by: Sairaj Kodilkar <sarunkod@amd.com>
Signed-off-by: Ashish Kalra <ashish.kalra@amd.com>
Link: https://lore.kernel.org/r/3a31036fb2f7323e6b1a1a1921ac777e9f7bdddc.1756157913.git.ashish.kalra@amd.com
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---

LLM Generated explanations, may be completely bogus:

YES

- Fixes a real kdump crash on AMD systems with SNP: When the first
  kernel panics with SNP enforced, the AMD IOMMU device-table base
  register remains locked (per spec), so the crash kernel’s attempts to
  program a new device table base are ignored. This leaves the OS and
  hardware pointing at different device tables, causing command timeouts
  and a second panic. The change makes the crash kernel reuse the
  previous kernel’s device table instead of copying it and trying to
  reprogram the base register.

- Minimal, targeted scope in AMD IOMMU init: All changes are confined to
  the AMD IOMMU initialization path and are conditional to kdump/“pre-
  enabled translation” scenarios. Normal boot paths are unaffected.

- Core behavior changes that address the bug:
  - Skip programming the DEV table base in kdump:
    drivers/iommu/amd/init.c:401–416 adds an early return in
    `iommu_set_device_table()` for `is_kdump_kernel()`. This avoids
    writing to the locked `MMIO_DEV_TABLE_OFFSET`, exactly the condition
    that was breaking kdump (hardware ignores the write).
  - Reuse instead of copy the prior device table:
    drivers/iommu/amd/init.c:1136–1177 implements
    `__reuse_device_table()` which reads the old table base from
    hardware, clears SME C-bit as needed, and maps it via
    `iommu_memremap()`. The segment-level wrapper `reuse_device_table()`
    at drivers/iommu/amd/init.c:1179–1204 ensures reuse happens only
    once per PCI segment (same as the old “copy” logic, but now purely
    reuse).
  - Driver now adopts the remapped table: in the success path, the
    driver frees its freshly allocated table and replaces it with the
    remapped one (drivers/iommu/amd/init.c:2916–2920). It logs “Reused
    DEV table from previous kernel.” (drivers/iommu/amd/init.c:2914).
  - Robust failure handling when reuse is required: If reuse fails while
    the IOMMU was pre-enabled (the problematic kdump case) and SNP is
    present, the code bails out early with a `BUG_ON()` to prevent
    subsequent hangs/timeouts that lead to a secondary panic
    (drivers/iommu/amd/init.c:2899–2913). This is appropriate for a
    crash kernel context.
  - Correct freeing/unmapping for kdump allocations: In kdump paths,
    previously allocated memory is unmmapped rather than freed from the
    page allocator (e.g., `free_dev_table()` uses `memunmap()` under
    kdump in drivers/iommu/amd/init.c:650–657; similarly for
    `old_dev_tbl_cpy` at drivers/iommu/amd/init.c:2902–2904). This
    matches the new “reuse/remap” strategy.

- Why reuse is necessary vs copying: The old approach copied contents
  into a new table and then tried to reprogram the base register to
  point at it. In kdump with SNP, the base register remains locked to
  the prior kernel’s table; hardware keeps using the old table while the
  OS uses the new copy, causing “Completion-Wait loop timed out” and
  eventually a timer-related panic. Reusing the same memory location
  aligns OS and hardware references immediately and resolves the failure
  mode.

- Removed copy-time fixups are safe in this model: The old copy path
  reserved domain IDs and masked out GCR3/GV bits while copying. With
  reuse, the crash kernel updates DTEs in-place and the attach path
  handles necessary state transitions:
  - New domain associations overwrite the DTE’s `domid` and flush the
    old domain’s TLB if needed (drivers/iommu/amd/iommu.c:2096–2126).
    This mitigates the need to pre-reserve legacy domain IDs.
  - GCR3/guest state is set appropriately when attaching domains via
    `set_dte_gcr3_table()` and associated code in the attach/update path
    (drivers/iommu/amd/iommu.c:2082–2126). This removes the need for ad-
    hoc masking in the copy code.

- Backport risk/considerations:
  - Dependencies: This change relies on `iommu_memremap()`
    (drivers/iommu/amd/init.c:659–681) and the broader kdump reuse
    plumbing already present for the completion wait buffer, command
    buffer, and event buffer (e.g., drivers/iommu/amd/init.c:1089–1177).
    If a target stable branch does not yet have these helpers and reuse
    logic, they should be brought in along with this patch.
  - Kdump-only behavior change in `iommu_set_device_table()`: The
    unconditional `is_kdump_kernel()` early return
    (drivers/iommu/amd/init.c:409–415) is intentional because the base
    register is locked when translation is pre-enabled by the prior
    kernel (i.e., precisely the scenario that matters for kdump). The
    `early_enable_iommus()` logic uses reuse only when
    `amd_iommu_pre_enabled` is true and falls back to full
    initialization otherwise; with SNP and pre-enabled translation, it
    deliberately BUG_ON if reuse fails to avoid the known-timeout/panic
    path (drivers/iommu/amd/init.c:2888–2952).
  - Scope is limited; no architectural changes; affects AMD-IOMMU kdump
    path only.

- Meets stable criteria:
  - Important user-visible bugfix (crash kernel panic on AMD SNP
    systems).
  - Contained to the AMD IOMMU driver init path.
  - No new features or ABI changes.
  - Low regression risk outside kdump; guarded by `is_kdump_kernel()`
    and `amd_iommu_pre_enabled` checks.

Given the severity of the kdump failure and the focused nature of the
fix, this is a strong candidate for stable backport, provided the small
helper/dependency pieces used in this change are included or adapted for
the target branches.

 drivers/iommu/amd/init.c | 104 +++++++++++++--------------------------
 1 file changed, 34 insertions(+), 70 deletions(-)

diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c
index d0cd40ee0dec6..f2991c11867cb 100644
--- a/drivers/iommu/amd/init.c
+++ b/drivers/iommu/amd/init.c
@@ -406,6 +406,9 @@ static void iommu_set_device_table(struct amd_iommu *iommu)
 
 	BUG_ON(iommu->mmio_base == NULL);
 
+	if (is_kdump_kernel())
+		return;
+
 	entry = iommu_virt_to_phys(dev_table);
 	entry |= (dev_table_size >> 12) - 1;
 	memcpy_toio(iommu->mmio_base + MMIO_DEV_TABLE_OFFSET,
@@ -646,7 +649,10 @@ static inline int __init alloc_dev_table(struct amd_iommu_pci_seg *pci_seg)
 
 static inline void free_dev_table(struct amd_iommu_pci_seg *pci_seg)
 {
-	iommu_free_pages(pci_seg->dev_table);
+	if (is_kdump_kernel())
+		memunmap((void *)pci_seg->dev_table);
+	else
+		iommu_free_pages(pci_seg->dev_table);
 	pci_seg->dev_table = NULL;
 }
 
@@ -1127,15 +1133,12 @@ static void set_dte_bit(struct dev_table_entry *dte, u8 bit)
 	dte->data[i] |= (1UL << _bit);
 }
 
-static bool __copy_device_table(struct amd_iommu *iommu)
+static bool __reuse_device_table(struct amd_iommu *iommu)
 {
-	u64 int_ctl, int_tab_len, entry = 0;
 	struct amd_iommu_pci_seg *pci_seg = iommu->pci_seg;
-	struct dev_table_entry *old_devtb = NULL;
-	u32 lo, hi, devid, old_devtb_size;
+	u32 lo, hi, old_devtb_size;
 	phys_addr_t old_devtb_phys;
-	u16 dom_id, dte_v, irq_v;
-	u64 tmp;
+	u64 entry;
 
 	/* Each IOMMU use separate device table with the same size */
 	lo = readl(iommu->mmio_base + MMIO_DEV_TABLE_OFFSET);
@@ -1160,66 +1163,20 @@ static bool __copy_device_table(struct amd_iommu *iommu)
 		pr_err("The address of old device table is above 4G, not trustworthy!\n");
 		return false;
 	}
-	old_devtb = (cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT) && is_kdump_kernel())
-		    ? (__force void *)ioremap_encrypted(old_devtb_phys,
-							pci_seg->dev_table_size)
-		    : memremap(old_devtb_phys, pci_seg->dev_table_size, MEMREMAP_WB);
-
-	if (!old_devtb)
-		return false;
 
-	pci_seg->old_dev_tbl_cpy = iommu_alloc_pages_sz(
-		GFP_KERNEL | GFP_DMA32, pci_seg->dev_table_size);
+	/*
+	 * Re-use the previous kernel's device table for kdump.
+	 */
+	pci_seg->old_dev_tbl_cpy = iommu_memremap(old_devtb_phys, pci_seg->dev_table_size);
 	if (pci_seg->old_dev_tbl_cpy == NULL) {
-		pr_err("Failed to allocate memory for copying old device table!\n");
-		memunmap(old_devtb);
+		pr_err("Failed to remap memory for reusing old device table!\n");
 		return false;
 	}
 
-	for (devid = 0; devid <= pci_seg->last_bdf; ++devid) {
-		pci_seg->old_dev_tbl_cpy[devid] = old_devtb[devid];
-		dom_id = old_devtb[devid].data[1] & DEV_DOMID_MASK;
-		dte_v = old_devtb[devid].data[0] & DTE_FLAG_V;
-
-		if (dte_v && dom_id) {
-			pci_seg->old_dev_tbl_cpy[devid].data[0] = old_devtb[devid].data[0];
-			pci_seg->old_dev_tbl_cpy[devid].data[1] = old_devtb[devid].data[1];
-			/* Reserve the Domain IDs used by previous kernel */
-			if (ida_alloc_range(&pdom_ids, dom_id, dom_id, GFP_ATOMIC) != dom_id) {
-				pr_err("Failed to reserve domain ID 0x%x\n", dom_id);
-				memunmap(old_devtb);
-				return false;
-			}
-			/* If gcr3 table existed, mask it out */
-			if (old_devtb[devid].data[0] & DTE_FLAG_GV) {
-				tmp = (DTE_GCR3_30_15 | DTE_GCR3_51_31);
-				pci_seg->old_dev_tbl_cpy[devid].data[1] &= ~tmp;
-				tmp = (DTE_GCR3_14_12 | DTE_FLAG_GV);
-				pci_seg->old_dev_tbl_cpy[devid].data[0] &= ~tmp;
-			}
-		}
-
-		irq_v = old_devtb[devid].data[2] & DTE_IRQ_REMAP_ENABLE;
-		int_ctl = old_devtb[devid].data[2] & DTE_IRQ_REMAP_INTCTL_MASK;
-		int_tab_len = old_devtb[devid].data[2] & DTE_INTTABLEN_MASK;
-		if (irq_v && (int_ctl || int_tab_len)) {
-			if ((int_ctl != DTE_IRQ_REMAP_INTCTL) ||
-			    (int_tab_len != DTE_INTTABLEN_512 &&
-			     int_tab_len != DTE_INTTABLEN_2K)) {
-				pr_err("Wrong old irq remapping flag: %#x\n", devid);
-				memunmap(old_devtb);
-				return false;
-			}
-
-			pci_seg->old_dev_tbl_cpy[devid].data[2] = old_devtb[devid].data[2];
-		}
-	}
-	memunmap(old_devtb);
-
 	return true;
 }
 
-static bool copy_device_table(void)
+static bool reuse_device_table(void)
 {
 	struct amd_iommu *iommu;
 	struct amd_iommu_pci_seg *pci_seg;
@@ -1227,17 +1184,17 @@ static bool copy_device_table(void)
 	if (!amd_iommu_pre_enabled)
 		return false;
 
-	pr_warn("Translation is already enabled - trying to copy translation structures\n");
+	pr_warn("Translation is already enabled - trying to reuse translation structures\n");
 
 	/*
 	 * All IOMMUs within PCI segment shares common device table.
-	 * Hence copy device table only once per PCI segment.
+	 * Hence reuse device table only once per PCI segment.
 	 */
 	for_each_pci_segment(pci_seg) {
 		for_each_iommu(iommu) {
 			if (pci_seg->id != iommu->pci_seg->id)
 				continue;
-			if (!__copy_device_table(iommu))
+			if (!__reuse_device_table(iommu))
 				return false;
 			break;
 		}
@@ -2916,8 +2873,8 @@ static void early_enable_iommu(struct amd_iommu *iommu)
  * This function finally enables all IOMMUs found in the system after
  * they have been initialized.
  *
- * Or if in kdump kernel and IOMMUs are all pre-enabled, try to copy
- * the old content of device table entries. Not this case or copy failed,
+ * Or if in kdump kernel and IOMMUs are all pre-enabled, try to reuse
+ * the old content of device table entries. Not this case or reuse failed,
  * just continue as normal kernel does.
  */
 static void early_enable_iommus(void)
@@ -2925,18 +2882,25 @@ static void early_enable_iommus(void)
 	struct amd_iommu *iommu;
 	struct amd_iommu_pci_seg *pci_seg;
 
-	if (!copy_device_table()) {
+	if (!reuse_device_table()) {
 		/*
-		 * If come here because of failure in copying device table from old
+		 * If come here because of failure in reusing device table from old
 		 * kernel with all IOMMUs enabled, print error message and try to
 		 * free allocated old_dev_tbl_cpy.
 		 */
-		if (amd_iommu_pre_enabled)
-			pr_err("Failed to copy DEV table from previous kernel.\n");
+		if (amd_iommu_pre_enabled) {
+			pr_err("Failed to reuse DEV table from previous kernel.\n");
+			/*
+			 * Bail out early if unable to remap/reuse DEV table from
+			 * previous kernel if SNP enabled as IOMMU commands will
+			 * time out without DEV table and cause kdump boot panic.
+			 */
+			BUG_ON(check_feature(FEATURE_SNP));
+		}
 
 		for_each_pci_segment(pci_seg) {
 			if (pci_seg->old_dev_tbl_cpy != NULL) {
-				iommu_free_pages(pci_seg->old_dev_tbl_cpy);
+				memunmap((void *)pci_seg->old_dev_tbl_cpy);
 				pci_seg->old_dev_tbl_cpy = NULL;
 			}
 		}
@@ -2946,7 +2910,7 @@ static void early_enable_iommus(void)
 			early_enable_iommu(iommu);
 		}
 	} else {
-		pr_info("Copied DEV table from previous kernel.\n");
+		pr_info("Reused DEV table from previous kernel.\n");
 
 		for_each_pci_segment(pci_seg) {
 			iommu_free_pages(pci_seg->dev_table);
-- 
2.51.0

[PATCH AUTOSEL 6.17-5.15] iommu/vt-d: Replace snprintf with scnprintf in dmar_latency_snapshot()

[Sasha Levin]

From: Seyediman Seyedarab <ImanDevel@gmail.com>

[ Upstream commit 75c02a037609f34db17e91be195cedb33b61bae0 ]

snprintf() returns the number of bytes that would have been written, not
the number actually written. Using this for offset tracking can cause
buffer overruns if truncation occurs.

Replace snprintf() with scnprintf() to ensure the offset stays within
bounds.

Since scnprintf() never returns a negative value, and zero is not possible
in this context because 'bytes' starts at 0 and 'size - bytes' is
DEBUG_BUFFER_SIZE in the first call, which is large enough to hold the
string literals used, the return value is always positive. An integer
overflow is also completely out of reach here due to the small and fixed
buffer size. The error check in latency_show_one() is therefore
unnecessary. Remove it and make dmar_latency_snapshot() return void.

Signed-off-by: Seyediman Seyedarab <ImanDevel@gmail.com>
Link: https://lore.kernel.org/r/20250731225048.131364-1-ImanDevel@gmail.com
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---

LLM Generated explanations, may be completely bogus:

YES – This fixes a real memory-corruption bug in the Intel VT-d debugfs
latency snapshot and the change is tight, self-contained, and low risk
for stable.

- `drivers/iommu/intel/perf.c:116-160` now uses `scnprintf` for every
  append into `debug_buf`. Previously `snprintf` advanced the `bytes`
  cursor by the would-have-been length; once the counters grew large
  enough to truncate a write, `bytes` could exceed `size`, and the next
  `snprintf(str + bytes, size - bytes, …)` would wrap the length
  argument and scribble past the 1 KB buffer. That overflow is a latent
  kernel memory corruption reachable from the `dmar_perf_latency`
  debugfs file. With `scnprintf`, the offset can no longer run past the
  remaining space, eliminating the corruption risk.
- `drivers/iommu/intel/debugfs.c:659-666` drops the dead `< 0` error
  handling and simply prints the buffer, matching the new `void` return
  semantics and avoiding bogus “Failed to get latency snapshot”
  messages.
- `drivers/iommu/intel/perf.h:37-70` updates the prototype and the
  `!CONFIG_DMAR_PERF` stub accordingly so all callers build cleanly; no
  other interfaces or architectures are touched.

The bug exists in all trees that have Intel IOMMU debugfs
(`CONFIG_INTEL_IOMMU_DEBUGFS` selects this code) and can be triggered by
routine use once latency counters accumulate large values. The fix is
entirely in debug/perf code, introduces no behavioural changes beyond
removing the overflow, and carries negligible regression risk.
Backporting is recommended.

 drivers/iommu/intel/debugfs.c | 10 ++--------
 drivers/iommu/intel/perf.c    | 10 ++++------
 drivers/iommu/intel/perf.h    |  5 ++---
 3 files changed, 8 insertions(+), 17 deletions(-)

diff --git a/drivers/iommu/intel/debugfs.c b/drivers/iommu/intel/debugfs.c
index 5aa7f46a420b5..38790ff50977c 100644
--- a/drivers/iommu/intel/debugfs.c
+++ b/drivers/iommu/intel/debugfs.c
@@ -661,17 +661,11 @@ DEFINE_SHOW_ATTRIBUTE(ir_translation_struct);
 static void latency_show_one(struct seq_file *m, struct intel_iommu *iommu,
 			     struct dmar_drhd_unit *drhd)
 {
-	int ret;
-
 	seq_printf(m, "IOMMU: %s Register Base Address: %llx\n",
 		   iommu->name, drhd->reg_base_addr);
 
-	ret = dmar_latency_snapshot(iommu, debug_buf, DEBUG_BUFFER_SIZE);
-	if (ret < 0)
-		seq_puts(m, "Failed to get latency snapshot");
-	else
-		seq_puts(m, debug_buf);
-	seq_puts(m, "\n");
+	dmar_latency_snapshot(iommu, debug_buf, DEBUG_BUFFER_SIZE);
+	seq_printf(m, "%s\n", debug_buf);
 }
 
 static int latency_show(struct seq_file *m, void *v)
diff --git a/drivers/iommu/intel/perf.c b/drivers/iommu/intel/perf.c
index adc4de6bbd88e..dceeadc3ee7cd 100644
--- a/drivers/iommu/intel/perf.c
+++ b/drivers/iommu/intel/perf.c
@@ -113,7 +113,7 @@ static char *latency_type_names[] = {
 	"     svm_prq"
 };
 
-int dmar_latency_snapshot(struct intel_iommu *iommu, char *str, size_t size)
+void dmar_latency_snapshot(struct intel_iommu *iommu, char *str, size_t size)
 {
 	struct latency_statistic *lstat = iommu->perf_statistic;
 	unsigned long flags;
@@ -122,7 +122,7 @@ int dmar_latency_snapshot(struct intel_iommu *iommu, char *str, size_t size)
 	memset(str, 0, size);
 
 	for (i = 0; i < COUNTS_NUM; i++)
-		bytes += snprintf(str + bytes, size - bytes,
+		bytes += scnprintf(str + bytes, size - bytes,
 				  "%s", latency_counter_names[i]);
 
 	spin_lock_irqsave(&latency_lock, flags);
@@ -130,7 +130,7 @@ int dmar_latency_snapshot(struct intel_iommu *iommu, char *str, size_t size)
 		if (!dmar_latency_enabled(iommu, i))
 			continue;
 
-		bytes += snprintf(str + bytes, size - bytes,
+		bytes += scnprintf(str + bytes, size - bytes,
 				  "\n%s", latency_type_names[i]);
 
 		for (j = 0; j < COUNTS_NUM; j++) {
@@ -156,11 +156,9 @@ int dmar_latency_snapshot(struct intel_iommu *iommu, char *str, size_t size)
 				break;
 			}
 
-			bytes += snprintf(str + bytes, size - bytes,
+			bytes += scnprintf(str + bytes, size - bytes,
 					  "%12lld", val);
 		}
 	}
 	spin_unlock_irqrestore(&latency_lock, flags);
-
-	return bytes;
 }
diff --git a/drivers/iommu/intel/perf.h b/drivers/iommu/intel/perf.h
index df9a36942d643..1d4baad7e852e 100644
--- a/drivers/iommu/intel/perf.h
+++ b/drivers/iommu/intel/perf.h
@@ -40,7 +40,7 @@ void dmar_latency_disable(struct intel_iommu *iommu, enum latency_type type);
 bool dmar_latency_enabled(struct intel_iommu *iommu, enum latency_type type);
 void dmar_latency_update(struct intel_iommu *iommu, enum latency_type type,
 			 u64 latency);
-int dmar_latency_snapshot(struct intel_iommu *iommu, char *str, size_t size);
+void dmar_latency_snapshot(struct intel_iommu *iommu, char *str, size_t size);
 #else
 static inline int
 dmar_latency_enable(struct intel_iommu *iommu, enum latency_type type)
@@ -64,9 +64,8 @@ dmar_latency_update(struct intel_iommu *iommu, enum latency_type type, u64 laten
 {
 }
 
-static inline int
+static inline void
 dmar_latency_snapshot(struct intel_iommu *iommu, char *str, size_t size)
 {
-	return 0;
 }
 #endif /* CONFIG_DMAR_PERF */
-- 
2.51.0

[PATCH AUTOSEL 6.17-6.6] iommu/apple-dart: Clear stream error indicator bits for T8110 DARTs

[Sasha Levin]

From: Hector Martin <marcan@marcan.st>

[ Upstream commit ecf6508923f87e4597228f70cc838af3d37f6662 ]

These registers exist and at least on the t602x variant the IRQ only
clears when theses are cleared.

Signed-off-by: Hector Martin <marcan@marcan.st>
Signed-off-by: Janne Grunau <j@jannau.net>
Reviewed-by: Sven Peter <sven@kernel.org>
Reviewed-by: Neal Gompa <neal@gompa.dev>
Link: https://lore.kernel.org/r/20250826-dart-t8110-stream-error-v1-1-e33395112014@jannau.net
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---

LLM Generated explanations, may be completely bogus:

YES

- Fixes a real bug: On T8110-class DARTs, the interrupt only deasserts
  when per‑stream error indicator bits are cleared. Without clearing
  them, the IRQ can remain asserted, causing repeated interrupts and
  potential system instability.
- Minimal, contained change: Adds one register define and a short clear
  loop in the T8110 IRQ handler only.
  - Adds `DART_T8110_ERROR_STREAMS` at `0x1c0` (drivers/iommu/apple-
    dart.c:125).
  - In `apple_dart_t8110_irq`, after acknowledging the error status
    (`writel(error, ...)`), clears all stream error indicator words:
    `for (int i = 0; i < BITS_TO_U32(dart->num_streams); i++)
    writel(U32_MAX, dart->regs + DART_T8110_ERROR_STREAMS + 4 * i);`
    (drivers/iommu/apple-dart.c:1093, drivers/iommu/apple-dart.c:1094,
    drivers/iommu/apple-dart.c:1095).
- Scoped to T8110 only: The handler is registered only for the T8110
  hardware variant (drivers/iommu/apple-dart.c:1298), so it does not
  affect other Apple DART generations.
- Safe by design:
  - `num_streams` is read from hardware (`DART_T8110_PARAMS4_NUM_SIDS`)
    and bounded by `DART_MAX_STREAMS` (drivers/iommu/apple-dart.c:1152,
    drivers/iommu/apple-dart.c:1153, drivers/iommu/apple-dart.c:1156,
    drivers/iommu/apple-dart.c:1161).
  - `BITS_TO_U32(...)` ensures the correct number of 32‑bit words are
    cleared.
  - The driver already uses the same write‑all‑ones bitmap pattern for
    stream operations (e.g., enabling all streams) showing these
    registers are W1C bitmaps and that this access pattern is
    established and safe (drivers/iommu/apple-dart.c:485,
    drivers/iommu/apple-dart.c:486).
- No architectural changes or API/ABI effects: This is a straightforward
  IRQ acknowledgment fix limited to the Apple DART IOMMU driver.
- User impact: Prevents stuck/level interrupts and interrupt floods on
  affected Apple SoCs (e.g., t602x using T8110 DART), improving
  stability and correctness.

Given it is a clear bug fix, small and localized, with low regression
risk and meaningful user impact, it is a strong candidate for
backporting to stable kernels that include the T8110 DART support.

 drivers/iommu/apple-dart.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/drivers/iommu/apple-dart.c b/drivers/iommu/apple-dart.c
index 190f28d766151..8b1272b7bb44a 100644
--- a/drivers/iommu/apple-dart.c
+++ b/drivers/iommu/apple-dart.c
@@ -122,6 +122,8 @@
 #define DART_T8110_ERROR_ADDR_LO 0x170
 #define DART_T8110_ERROR_ADDR_HI 0x174
 
+#define DART_T8110_ERROR_STREAMS 0x1c0
+
 #define DART_T8110_PROTECT 0x200
 #define DART_T8110_UNPROTECT 0x204
 #define DART_T8110_PROTECT_LOCK 0x208
@@ -1077,6 +1079,9 @@ static irqreturn_t apple_dart_t8110_irq(int irq, void *dev)
 		error, stream_idx, error_code, fault_name, addr);
 
 	writel(error, dart->regs + DART_T8110_ERROR);
+	for (int i = 0; i < BITS_TO_U32(dart->num_streams); i++)
+		writel(U32_MAX, dart->regs + DART_T8110_ERROR_STREAMS + 4 * i);
+
 	return IRQ_HANDLED;
 }
 
-- 
2.51.0

[PATCH AUTOSEL 6.17] iommu/amd: Add support to remap/unmap IOMMU buffers for kdump

[Sasha Levin]

From: Ashish Kalra <ashish.kalra@amd.com>

[ Upstream commit f32fe7cb019861f585b40bff4c3daf237b9af294 ]

After a panic if SNP is enabled in the previous kernel then the kdump
kernel boots with IOMMU SNP enforcement still enabled.

IOMMU completion wait buffers (CWBs), command buffers and event buffer
registers remain locked and exclusive to the previous kernel. Attempts
to allocate and use new buffers in the kdump kernel fail, as hardware
ignores writes to the locked MMIO registers as per AMD IOMMU spec
Section 2.12.2.1.

This results in repeated "Completion-Wait loop timed out" errors and a
second kernel panic: "Kernel panic - not syncing: timer doesn't work
through Interrupt-remapped IO-APIC"

The list of MMIO registers locked and which ignore writes after failed
SNP shutdown are mentioned in the AMD IOMMU specifications below:

Section 2.12.2.1.
https://docs.amd.com/v/u/en-US/48882_3.10_PUB

Reuse the pages of the previous kernel for completion wait buffers,
command buffers, event buffers and memremap them during kdump boot
and essentially work with an already enabled IOMMU configuration and
re-using the previous kernel’s data structures.

Reusing of command buffers and event buffers is now done for kdump boot
irrespective of SNP being enabled during kdump.

Re-use of completion wait buffers is only done when SNP is enabled as
the exclusion base register is used for the completion wait buffer
(CWB) address only when SNP is enabled.

Reviewed-by: Vasant Hegde <vasant.hegde@amd.com>
Tested-by: Sairaj Kodilkar <sarunkod@amd.com>
Signed-off-by: Ashish Kalra <ashish.kalra@amd.com>
Link: https://lore.kernel.org/r/ff04b381a8fe774b175c23c1a336b28bc1396511.1756157913.git.ashish.kalra@amd.com
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---

LLM Generated explanations, may be completely bogus:

YES

- Fixes a real kdump failure: After a panic with SNP enabled, AMD IOMMU
  MMIO registers for completion-wait buffer (CWB), command buffer, and
  event buffer remain locked to the previous kernel. The kdump kernel’s
  attempts to program new buffers are ignored, leading to “Completion-
  Wait loop timed out” and a second panic (“timer doesn't work through
  Interrupt-remapped IO-APIC”). The change reuses the previous kernel’s
  buffers and remaps them into the kdump kernel instead of reprogramming
  the locked registers.

What the change does
- New encrypted-aware remap helper: `iommu_memremap()` clears the SME
  mask and maps with `ioremap_encrypted()` when host memory encryption
  is active, otherwise uses `memremap(MEMREMAP_WB)`
  (drivers/iommu/amd/init.c:719).
- Reuse/remap buffers in kdump:
  - Event buffer remap from hardware register: reads
    `MMIO_EVT_BUF_OFFSET` to get paddr and remaps
    (drivers/iommu/amd/init.c:987).
  - Command buffer remap from hardware register: reads
    `MMIO_CMD_BUF_OFFSET` and remaps (drivers/iommu/amd/init.c:998).
  - CWB handling: If SNP is enabled, read `MMIO_EXCL_BASE_OFFSET` and
    remap the CWB; otherwise allocate a fresh CWB (consistent with spec
    that EXCL_BASE is only used for CWB with SNP)
    (drivers/iommu/amd/init.c:1009).
  - One orchestrating entry point `alloc_iommu_buffers()` chooses remap
    vs allocate strictly based on `is_kdump_kernel()`
    (drivers/iommu/amd/init.c:1031).
- Avoids writes to locked MMIO base registers in kdump:
  `iommu_enable_command_buffer()` and `iommu_enable_event_buffer()` skip
  programming base/length registers when `is_kdump_kernel()` and only
  reset head/tail and enable the features (drivers/iommu/amd/init.c:818,
  drivers/iommu/amd/init.c:878).
- Stores the physical CWB address once: Adds `cmd_sem_paddr` to `struct
  amd_iommu` (drivers/iommu/amd/amd_iommu_types.h:795). It is
  initialized on allocation or remap (drivers/iommu/amd/init.c:978,
  drivers/iommu/amd/init.c:1019) and then used directly when building
  completion-wait commands (drivers/iommu/amd/iommu.c:1195). This
  removes the need to resolve a virtual address that may be a remapped
  legacy physical address.
- Proper unmapping on teardown in kdump: Introduces unmap variants
  (`unmap_cwwb_sem`, `unmap_command_buffer`, `unmap_event_buffer`) and
  uses them conditionally via `free_iommu_buffers()` based on
  `is_kdump_kernel()` (drivers/iommu/amd/init.c:1031,
  drivers/iommu/amd/init.c:1757). Using `memunmap()` is correct for
  these mappings since `memunmap()` detects ioremap-backed regions and
  calls `iounmap()` (kernel/iomem.c:120).

Why it fits stable criteria
- User-visible bugfix: Prevents a second panic and restores kdump
  operation on AMD systems when the previous kernel had SNP enabled.
- Contained change: All changes are within the AMD IOMMU driver and its
  init/enable paths, with kdump-specific behavior guarded by
  `is_kdump_kernel()` and SNP checks. Normal boots remain on the
  original allocation path, with only a benign addition of caching
  `cmd_sem_paddr`.
- No architectural overhaul: Adds a small field and helper functions,
  plus remap/unmap paths. The IOMMU programming model remains unchanged;
  the kdump path just avoids touching registers the hardware purposely
  locks.
- Low regression risk:
  - Non-kdump boots: The existing flow still allocates buffers and
    programs MMIO registers as before. The only functional change is
    that completion-wait now uses the cached physical address
    `cmd_sem_paddr` (drivers/iommu/amd/iommu.c:1195), which is set at
    allocation time (drivers/iommu/amd/init.c:978).
  - Kdump boots: Writes to locked base/length registers are avoided;
    HEAD/TAIL resets and enables remain, which are the only needed
    touches (drivers/iommu/amd/init.c:818,
    drivers/iommu/amd/init.c:878).
  - Memory encryption correctness: `iommu_memremap()` clears the SME
    mask when deriving the true physical address for mapping
    (drivers/iommu/amd/init.c:719). Unmap correctness is ensured by
    `memunmap()`’s use of `iounmap()` for ioremap-backed regions
    (kernel/iomem.c:120).
- No feature additions: Strictly a robustness fix for crash kernels
  interacting with SNP-locked IOMMU hardware.
- Critical subsystem touch vs. mitigations: Although AMD IOMMU is
  critical, the change isolates the special handling to kdump/SNP cases
  and avoids altering normal runtime behavior.

Net: This is a targeted, necessary fix for a severe kdump failure mode
on AMD platforms with SNP. The changes are well-scoped, guarded, and
follow kernel mapping/unmapping conventions. It’s a good candidate for
stable backport.

 drivers/iommu/amd/amd_iommu_types.h |   5 +
 drivers/iommu/amd/init.c            | 152 +++++++++++++++++++++++++---
 drivers/iommu/amd/iommu.c           |   2 +-
 3 files changed, 146 insertions(+), 13 deletions(-)

diff --git a/drivers/iommu/amd/amd_iommu_types.h b/drivers/iommu/amd/amd_iommu_types.h
index 95f63c5f6159f..a698a2e7ce2a6 100644
--- a/drivers/iommu/amd/amd_iommu_types.h
+++ b/drivers/iommu/amd/amd_iommu_types.h
@@ -792,6 +792,11 @@ struct amd_iommu {
 	u32 flags;
 	volatile u64 *cmd_sem;
 	atomic64_t cmd_sem_val;
+	/*
+	 * Track physical address to directly use it in build_completion_wait()
+	 * and avoid adding any special checks and handling for kdump.
+	 */
+	u64 cmd_sem_paddr;
 
 #ifdef CONFIG_AMD_IOMMU_DEBUGFS
 	/* DebugFS Info */
diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c
index ba9e582a8bbe5..309951e57f301 100644
--- a/drivers/iommu/amd/init.c
+++ b/drivers/iommu/amd/init.c
@@ -710,6 +710,26 @@ static void __init free_alias_table(struct amd_iommu_pci_seg *pci_seg)
 	pci_seg->alias_table = NULL;
 }
 
+static inline void *iommu_memremap(unsigned long paddr, size_t size)
+{
+	phys_addr_t phys;
+
+	if (!paddr)
+		return NULL;
+
+	/*
+	 * Obtain true physical address in kdump kernel when SME is enabled.
+	 * Currently, previous kernel with SME enabled and kdump kernel
+	 * with SME support disabled is not supported.
+	 */
+	phys = __sme_clr(paddr);
+
+	if (cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT))
+		return (__force void *)ioremap_encrypted(phys, size);
+	else
+		return memremap(phys, size, MEMREMAP_WB);
+}
+
 /*
  * Allocates the command buffer. This buffer is per AMD IOMMU. We can
  * write commands to that buffer later and the IOMMU will execute them
@@ -942,8 +962,91 @@ static int iommu_init_ga_log(struct amd_iommu *iommu)
 static int __init alloc_cwwb_sem(struct amd_iommu *iommu)
 {
 	iommu->cmd_sem = iommu_alloc_4k_pages(iommu, GFP_KERNEL, 1);
+	if (!iommu->cmd_sem)
+		return -ENOMEM;
+	iommu->cmd_sem_paddr = iommu_virt_to_phys((void *)iommu->cmd_sem);
+	return 0;
+}
+
+static int __init remap_event_buffer(struct amd_iommu *iommu)
+{
+	u64 paddr;
+
+	pr_info_once("Re-using event buffer from the previous kernel\n");
+	paddr = readq(iommu->mmio_base + MMIO_EVT_BUF_OFFSET) & PM_ADDR_MASK;
+	iommu->evt_buf = iommu_memremap(paddr, EVT_BUFFER_SIZE);
+
+	return iommu->evt_buf ? 0 : -ENOMEM;
+}
+
+static int __init remap_command_buffer(struct amd_iommu *iommu)
+{
+	u64 paddr;
 
-	return iommu->cmd_sem ? 0 : -ENOMEM;
+	pr_info_once("Re-using command buffer from the previous kernel\n");
+	paddr = readq(iommu->mmio_base + MMIO_CMD_BUF_OFFSET) & PM_ADDR_MASK;
+	iommu->cmd_buf = iommu_memremap(paddr, CMD_BUFFER_SIZE);
+
+	return iommu->cmd_buf ? 0 : -ENOMEM;
+}
+
+static int __init remap_or_alloc_cwwb_sem(struct amd_iommu *iommu)
+{
+	u64 paddr;
+
+	if (check_feature(FEATURE_SNP)) {
+		/*
+		 * When SNP is enabled, the exclusion base register is used for the
+		 * completion wait buffer (CWB) address. Read and re-use it.
+		 */
+		pr_info_once("Re-using CWB buffers from the previous kernel\n");
+		paddr = readq(iommu->mmio_base + MMIO_EXCL_BASE_OFFSET) & PM_ADDR_MASK;
+		iommu->cmd_sem = iommu_memremap(paddr, PAGE_SIZE);
+		if (!iommu->cmd_sem)
+			return -ENOMEM;
+		iommu->cmd_sem_paddr = paddr;
+	} else {
+		return alloc_cwwb_sem(iommu);
+	}
+
+	return 0;
+}
+
+static int __init alloc_iommu_buffers(struct amd_iommu *iommu)
+{
+	int ret;
+
+	/*
+	 * Reuse/Remap the previous kernel's allocated completion wait
+	 * command and event buffers for kdump boot.
+	 */
+	if (is_kdump_kernel()) {
+		ret = remap_or_alloc_cwwb_sem(iommu);
+		if (ret)
+			return ret;
+
+		ret = remap_command_buffer(iommu);
+		if (ret)
+			return ret;
+
+		ret = remap_event_buffer(iommu);
+		if (ret)
+			return ret;
+	} else {
+		ret = alloc_cwwb_sem(iommu);
+		if (ret)
+			return ret;
+
+		ret = alloc_command_buffer(iommu);
+		if (ret)
+			return ret;
+
+		ret = alloc_event_buffer(iommu);
+		if (ret)
+			return ret;
+	}
+
+	return 0;
 }
 
 static void __init free_cwwb_sem(struct amd_iommu *iommu)
@@ -951,6 +1054,38 @@ static void __init free_cwwb_sem(struct amd_iommu *iommu)
 	if (iommu->cmd_sem)
 		iommu_free_pages((void *)iommu->cmd_sem);
 }
+static void __init unmap_cwwb_sem(struct amd_iommu *iommu)
+{
+	if (iommu->cmd_sem) {
+		if (check_feature(FEATURE_SNP))
+			memunmap((void *)iommu->cmd_sem);
+		else
+			iommu_free_pages((void *)iommu->cmd_sem);
+	}
+}
+
+static void __init unmap_command_buffer(struct amd_iommu *iommu)
+{
+	memunmap((void *)iommu->cmd_buf);
+}
+
+static void __init unmap_event_buffer(struct amd_iommu *iommu)
+{
+	memunmap(iommu->evt_buf);
+}
+
+static void __init free_iommu_buffers(struct amd_iommu *iommu)
+{
+	if (is_kdump_kernel()) {
+		unmap_cwwb_sem(iommu);
+		unmap_command_buffer(iommu);
+		unmap_event_buffer(iommu);
+	} else {
+		free_cwwb_sem(iommu);
+		free_command_buffer(iommu);
+		free_event_buffer(iommu);
+	}
+}
 
 static void iommu_enable_xt(struct amd_iommu *iommu)
 {
@@ -1655,9 +1790,7 @@ static void __init free_sysfs(struct amd_iommu *iommu)
 static void __init free_iommu_one(struct amd_iommu *iommu)
 {
 	free_sysfs(iommu);
-	free_cwwb_sem(iommu);
-	free_command_buffer(iommu);
-	free_event_buffer(iommu);
+	free_iommu_buffers(iommu);
 	amd_iommu_free_ppr_log(iommu);
 	free_ga_log(iommu);
 	iommu_unmap_mmio_space(iommu);
@@ -1821,14 +1954,9 @@ static int __init init_iommu_one_late(struct amd_iommu *iommu)
 {
 	int ret;
 
-	if (alloc_cwwb_sem(iommu))
-		return -ENOMEM;
-
-	if (alloc_command_buffer(iommu))
-		return -ENOMEM;
-
-	if (alloc_event_buffer(iommu))
-		return -ENOMEM;
+	ret = alloc_iommu_buffers(iommu);
+	if (ret)
+		return ret;
 
 	iommu->int_enabled = false;
 
diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c
index eb348c63a8d09..05a9ab3da1a3e 100644
--- a/drivers/iommu/amd/iommu.c
+++ b/drivers/iommu/amd/iommu.c
@@ -1195,7 +1195,7 @@ static void build_completion_wait(struct iommu_cmd *cmd,
 				  struct amd_iommu *iommu,
 				  u64 data)
 {
-	u64 paddr = iommu_virt_to_phys((void *)iommu->cmd_sem);
+	u64 paddr = iommu->cmd_sem_paddr;
 
 	memset(cmd, 0, sizeof(*cmd));
 	cmd->data[0] = lower_32_bits(paddr) | CMD_COMPL_WAIT_STORE_MASK;
-- 
2.51.0