[PATCH 00/20] [PULL REQUEST] Intel IOMMU updates for v7.3

[PATCH 00/20] [PULL REQUEST] Intel IOMMU updates for v7.3

[Lu Baolu]

Hi Joerg,

The following changes are queued for v7.3-rc1. This update introduces
new features, quick fixes and several refactorings, including:

- Consolidation of complex enablement logic into a clean, priority-based
  state machine.
- Support for the DMA_REMAP_OPT_OUT flag from the VT-d v5.2
  specification.
- An update to cache_tag_flush_devtlb_psi() to use full-range constants
  instead of modifying shared variables for CACHE_TAG_NESTING_DEVTLB.
- A fix for the UCTP context-table slot when copying root entries.
- Fixes for several pre-existing issues reported by Sashiko.
- General code cleanup and refinement.

These patches are based on v7.2-rc6. The full series is also available
on GitHub:

https://github.com/LuBaolu/intel-iommu/commits/iommu/vt-d/next

Please consider pulling this series into the iommu/vt-d branch.

Best regards,
baolu

Desnes Nunes (1):
  iommu/vt-d: Fix UCTP context table slot when copying root entries

Dmitry Antipov (1):
  iommu/vt-d: Use kstrtoint_from_user() in dmar_perf_latency_write()

Guanghui Feng (1):
  iommu/vt-d: Fix CACHE_TAG_NESTING_DEVTLB polluting shared variables in
    flush loop

Kevin Tian (9):
  iommu/vt-d: Fix no_iommu to disable platform opt-in
  iommu/vt-d: Force requesting ACS when tboot is enabled
  iommu/vt-d: Remove dead code when CONFIG_INTEL_IOMMU is not set
  iommu/vt-d: Consolidate dmar policy management and force_on logic
  iommu/vt-d: Use dmar_can_force_on() for platform opt-in
  iommu/vt-d: Call dmar_can_force_on() for tboot opt-in
  iommu/vt-d: Remove the 'force_on' variable
  iommu/vt-d: Remove dmar_disabled
  iommu/vt-d: Support the new DMA_REMAP_OPT_OUT flag bit

Li RongQing (1):
  iommu/vt-d: Use logical OR operator for privilege mode check

Lu Baolu (6):
  iommu/vt-d: Cache max domain ID to avoid redundant calculation
  iommu/vt-d: Fix shift overflow in qi_desc_dev_iotlb_pasid()
  iommu/vt-d: Clear Present bit before tearing down copied context entry
  iommu/vt-d: Fix iopf_refcount leak on RID domain replacement
  iommu/vt-d: Tear down scalable-mode context on probe failure
  iommu/vt-d: Flush context cache with correct SID when tearing down
    aliases

ZhaoJinming (1):
  iommu/vt-d: Fix copied_tables bitmap leak on error in
    copy_translation_tables

 drivers/iommu/intel/iommu.h   |  89 +++++++++++++++++-----
 include/linux/dmar.h          |   1 +
 drivers/iommu/intel/cache.c   |   5 +-
 drivers/iommu/intel/debugfs.c |  16 +---
 drivers/iommu/intel/dmar.c    |  96 ++++++++++++++++++++---
 drivers/iommu/intel/iommu.c   | 139 +++++++++++++++++++---------------
 drivers/iommu/intel/pasid.c   |   9 ++-
 drivers/iommu/intel/prq.c     |   2 +-
 drivers/iommu/intel/svm.c     |   2 +-
 9 files changed, 250 insertions(+), 109 deletions(-)

-- 
2.43.0

[PATCH 01/20] iommu/vt-d: Fix UCTP context table slot when copying root entries

[Lu Baolu]

From: Desnes Nunes <desnesn@redhat.com>

When translation is already enabled at boot (e.g. kdump), the vt-d driver
copies context tables from the previous kernel's root table. In scalable
mode, buses that only populate the upper root half (UCTP, devfn >= 0x80)
should be written to ctxt_tbls[tbl_idx + 1] through copy_context_table().
However, the current copy path always uses tbl[tbl_idx + 0] in this situa-
tion. Since idx wraps to 0 at devfn 0x80 due to a zeroed LCTP, new_ce for
LCTP will be NULL and keep pos equals to 0. Thus, UCTP entries will be co-
pied into tbl[tbl_idx + 0] instead of tbl[tbl_idx + 1], and written after-
wards to root_entry[bus].lo instead of .hi in copy_translation_tables().

In short, devices on bus 0x80 with devfn >= 0x80 fail DMA with fault 0x39,
which will break drivers running in kernels with translation pre-enabled.
This fixes NO_PASID DMAR faults for UCTP-only buses such as:

DMAR: [DMA Read NO_PASID] Request device [80:14.0] fault addr 0xe81759000
      [fault reason 0x39] SM: Present bit in Root Entry is clear

For instance, this fault yielded to locking issues between systemd and
xHCI, blocking a system's reboot after a vmcore was captured with kdump:

 systemd-udevd[246]: usb3: Worker [255] processing SEQNUM=2193 is taking a long time
 dracut-initqueue[277]: Timed out while waiting for udev queue to empty.
 systemd-udevd[246]: usb3: Worker [255] processing SEQNUM=2193 killed
 systemd-udevd[246]: usb3: Worker [255] terminated by signal 9 (KILL).
 ...
 kdump[569]: saving vmcore complete
 ...
 systemd-shutdown[1]: Rebooting.
 INFO: task kworker/0:1:11 blocked for more than 122 seconds.
       Not tainted 7.0.0-clean #1
 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
 task:kworker/0:1 state:D stack:0 pid:11 tgid:11 ppid:2 task_flags:0x4208160 flags:0x00080000
 Workqueue: usb_hub_wq hub_event
 Call Trace:
  <TASK>
  __schedule+0x299/0x5c0
  schedule+0x27/0x80
  schedule_timeout+0xbd/0x100
  __wait_for_common+0x97/0x1b0
  ? __pfx_schedule_timeout+0x10/0x10
  xhci_alloc_dev+0x9e/0x2b0
  usb_alloc_dev+0x7a/0x3b0
  hub_port_connect+0x285/0x960
  hub_port_connect_change+0x94/0x290
  port_event+0x4bb/0x840
  hub_event+0x141/0x460
  process_one_work+0x196/0x390
  worker_thread+0x1af/0x320
  ? __pfx_worker_thread+0x10/0x10
  kthread+0xe3/0x120
  ? __pfx_kthread+0x10/0x10
  ret_from_fork+0x199/0x260
  ? __pfx_kthread+0x10/0x10
  ret_from_fork_asm+0x1a/0x30
  </TASK>
 INFO: task systemd-shutdow:1 blocked for more than 122 seconds.
       Not tainted 7.0.0-clean #1
 "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
 task:systemd-shutdow state:D stack:0 pid:1 tgid:1 ppid:0 task_flags:0x400100 flags:0x00080000
 Call Trace:
  <TASK>
  __schedule+0x299/0x5c0
  schedule+0x27/0x80
  schedule_preempt_disabled+0x15/0x30
  __mutex_lock.constprop.0+0x547/0xac0
  device_shutdown+0xac/0x1b0
  kernel_restart+0x3a/0x70
  __do_sys_reboot+0x147/0x240
  do_syscall_64+0x11b/0x6a0
  ? handle_mm_fault+0x110/0x350
  ? do_user_addr_fault+0x206/0x680
  ? irqentry_exit+0x7a/0x4d0
  entry_SYSCALL_64_after_hwframe+0x76/0x7e
 RIP: 0033:0x7fe2958da917
 RSP: 002b:00007ffc5c458618 EFLAGS: 00000206 ORIG_RAX: 00000000000000a9
 RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fe2958da917
 RDX: 0000000001234567 RSI: 0000000028121969 RDI: 00000000fee1dead
 RBP: 00007ffc5c458790 R08: 0000000000000069 R09: 00000000ffffffff
 R10: 0000000000000000 R11: 0000000000000206 R12: 0000000000000000
 R13: 0000000000000000 R14: 00007ffc5c4588b8 R15: 0000000000000000
  </TASK>
 INFO: task systemd-shutdow:1 is blocked on a mutex likely owned by task kworker/0:1:11.

Fixes: 091d42e43d21 ("iommu/vt-d: Copy translation tables from old kernel")
Signed-off-by: Desnes Nunes <desnesn@redhat.com>
Tested-by: Tao Liu <ltao@redhat.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
---
 drivers/iommu/intel/iommu.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index 849d06dfe1ae..cf5f92619943 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -1446,7 +1446,7 @@ static int copy_context_table(struct intel_iommu *iommu,
 			      struct context_entry **tbl,
 			      int bus, bool ext)
 {
-	int tbl_idx, pos = 0, idx, devfn, ret = 0, did;
+	int tbl_idx, tbl_slot = 0, idx, devfn, ret = 0, did;
 	struct context_entry *new_ce = NULL, ce;
 	struct context_entry *old_ce = NULL;
 	struct root_entry re;
@@ -1462,10 +1462,9 @@ static int copy_context_table(struct intel_iommu *iommu,
 		if (idx == 0) {
 			/* First save what we may have and clean up */
 			if (new_ce) {
-				tbl[tbl_idx] = new_ce;
+				tbl[tbl_idx + tbl_slot] = new_ce;
 				__iommu_flush_cache(iommu, new_ce,
 						    VTD_PAGE_SIZE);
-				pos = 1;
 			}
 
 			if (old_ce)
@@ -1487,6 +1486,9 @@ static int copy_context_table(struct intel_iommu *iommu,
 				}
 			}
 
+			/* Track if saving UCTP or LCTP entries in scalable mode */
+			tbl_slot = ext && devfn >= 0x80 ? 1 : 0;
+
 			ret = -ENOMEM;
 			old_ce = memremap(old_ce_phys, PAGE_SIZE,
 					MEMREMAP_WB);
@@ -1515,7 +1517,7 @@ static int copy_context_table(struct intel_iommu *iommu,
 		new_ce[idx] = ce;
 	}
 
-	tbl[tbl_idx + pos] = new_ce;
+	tbl[tbl_idx + tbl_slot] = new_ce;
 
 	__iommu_flush_cache(iommu, new_ce, VTD_PAGE_SIZE);
 
-- 
2.43.0

[PATCH 02/20] iommu/vt-d: Use logical OR operator for privilege mode check

[Lu Baolu]

From: Li RongQing <lirongqing@baidu.com>

Replace bitwise OR (|) with logical OR (||) in the privilege mode
validation check. While both operators produce the same result for
boolean values (0 or 1), using logical OR is semantically correct and
makes the intent clearer.

No functional change, but improves code readability.

Signed-off-by: Li RongQing <lirongqing@baidu.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
---
 drivers/iommu/intel/prq.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/iommu/intel/prq.c b/drivers/iommu/intel/prq.c
index 586055e51bb2..dddd3b51b657 100644
--- a/drivers/iommu/intel/prq.c
+++ b/drivers/iommu/intel/prq.c
@@ -223,7 +223,7 @@ static irqreturn_t prq_event_thread(int irq, void *d)
 			goto prq_advance;
 		}
 
-		if (unlikely(req->pm_req && (req->rd_req | req->wr_req))) {
+		if (unlikely(req->pm_req && (req->rd_req || req->wr_req))) {
 			pr_err("IOMMU: %s: Page request in Privilege Mode\n",
 			       iommu->name);
 			goto bad_req;
-- 
2.43.0

[PATCH 03/20] iommu/vt-d: Fix CACHE_TAG_NESTING_DEVTLB polluting shared variables in flush loop

[Lu Baolu]

From: Guanghui Feng <guanghuifeng@linux.alibaba.com>

In cache_tag_flush_range(), the CACHE_TAG_NESTING_DEVTLB case modifies the
shared local variables 'addr' and 'mask' before falling through to
CACHE_TAG_DEVTLB. This causes all subsequent CACHE_TAG_DEVTLB entries in
the same loop iteration to incorrectly use the full-range flush parameters
(addr=0, mask=MAX_AGAW_PFN_WIDTH) instead of the precisely calculated PSI
range. This is not the intended behavior, as regular DEVTLB entries should
always perform targeted range-based invalidation.

Fix this by having CACHE_TAG_NESTING_DEVTLB directly call
cache_tag_flush_devtlb_psi() with the full-range constants and break,
instead of modifying shared variables and falling through. This ensures
CACHE_TAG_DEVTLB always uses the original calculated addr and mask for
precise range flush.

Signed-off-by: Guanghui Feng <guanghuifeng@linux.alibaba.com>
Signed-off-by: Guixin Liu <kanie@linux.alibaba.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
---
 drivers/iommu/intel/cache.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/drivers/iommu/intel/cache.c b/drivers/iommu/intel/cache.c
index fdc88817709f..26a758b0f501 100644
--- a/drivers/iommu/intel/cache.c
+++ b/drivers/iommu/intel/cache.c
@@ -454,9 +454,8 @@ void cache_tag_flush_range(struct dmar_domain *domain, unsigned long start,
 			 * affected by a change in S2. So just flush the entire
 			 * device cache.
 			 */
-			addr = 0;
-			mask = MAX_AGAW_PFN_WIDTH;
-			fallthrough;
+			cache_tag_flush_devtlb_psi(domain, tag, 0, MAX_AGAW_PFN_WIDTH);
+			break;
 		case CACHE_TAG_DEVTLB:
 			cache_tag_flush_devtlb_psi(domain, tag, addr, mask);
 			break;
-- 
2.43.0

[PATCH 04/20] iommu/vt-d: Use kstrtoint_from_user() in dmar_perf_latency_write()

[Lu Baolu]

From: Dmitry Antipov <dmantipov@yandex.ru>

Simplify 'dmar_perf_latency_write()' by using the convenient
'kstrtoint_from_user()'.

Signed-off-by: Dmitry Antipov <dmantipov@yandex.ru>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
---
 drivers/iommu/intel/debugfs.c | 16 ++++------------
 1 file changed, 4 insertions(+), 12 deletions(-)

diff --git a/drivers/iommu/intel/debugfs.c b/drivers/iommu/intel/debugfs.c
index 21e4e465ca58..d87408ebb830 100644
--- a/drivers/iommu/intel/debugfs.c
+++ b/drivers/iommu/intel/debugfs.c
@@ -690,19 +690,11 @@ static ssize_t dmar_perf_latency_write(struct file *filp,
 {
 	struct dmar_drhd_unit *drhd;
 	struct intel_iommu *iommu;
-	int counting;
-	char buf[64];
+	int ret, counting;
 
-	if (cnt > 63)
-		cnt = 63;
-
-	if (copy_from_user(&buf, ubuf, cnt))
-		return -EFAULT;
-
-	buf[cnt] = 0;
-
-	if (kstrtoint(buf, 0, &counting))
-		return -EINVAL;
+	ret = kstrtoint_from_user(ubuf, cnt, 0, &counting);
+	if (ret)
+		return ret;
 
 	switch (counting) {
 	case 0:
-- 
2.43.0

[PATCH 05/20] iommu/vt-d: Fix no_iommu to disable platform opt-in

[Lu Baolu]

From: Kevin Tian <kevin.tian@intel.com>

If user explicitly requests to disable iommu (via "iommu=off" or
"intel_iommu=off"), there is no reason to force enabling it due
to platform opt-in (for external-facing devices). User should be
aware of any security implication of doing so.

"intel_iommu=off" implements this policy by setting no_platform_optin
to skip platform opt-in in platform_optin_force_iommu().

However, "iommu=off" (no_iommu=1) doesn't set no_platform_optin
hence is broken in this aspect:

  - detect_intel_iommu() doesn't request ACS if no_iommu=1
  - platform_optin_force_iommu() forces iommu on if external-facing
    devices exist and no_platform_optin is not set

This leads to a bad configuration with ACS disabled while DMA
remapping is enabled.

Instead of setting no_platform_optin (will soon be removed) for
no_iommu=1, directly check no_iommu in platform_optin_force_iommu().

Fixes: 89a6079df791 ("iommu/vt-d: Force IOMMU on for platform opt in hint")
Cc: stable@vger.kernel.org
Signed-off-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
---
 drivers/iommu/intel/iommu.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index cf5f92619943..1a1f27a51063 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -2484,10 +2484,11 @@ static bool has_external_pci(void)
 
 static int __init platform_optin_force_iommu(void)
 {
-	if (!dmar_platform_optin() || no_platform_optin || !has_external_pci())
+	if (no_iommu || !dmar_platform_optin() || no_platform_optin ||
+	    !has_external_pci())
 		return 0;
 
-	if (no_iommu || dmar_disabled)
+	if (dmar_disabled)
 		pr_info("Intel-IOMMU force enabled due to platform opt in\n");
 
 	/*
@@ -2498,7 +2499,6 @@ static int __init platform_optin_force_iommu(void)
 		iommu_set_default_passthrough(false);
 
 	dmar_disabled = 0;
-	no_iommu = 0;
 
 	return 1;
 }
-- 
2.43.0

[PATCH 06/20] iommu/vt-d: Force requesting ACS when tboot is enabled

[Lu Baolu]

From: Kevin Tian <kevin.tian@intel.com>

Currently the conditions of requesting ACS in detect_intel_iommu()
don't include tboot, leading to a possible misconfiguration with ACS
disabled (e.g. due to user opts) while iommu is later forced on by
tboot_force_iommu().

Fix it by checking tboot in detect_intel_iommu().

Fixes: 5d990b627537 ("PCI: add pci_request_acs")
Cc: stable@vger.kernel.org
Signed-off-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
---
 drivers/iommu/intel/iommu.h |  2 ++
 drivers/iommu/intel/dmar.c  | 15 +++++++++++++--
 drivers/iommu/intel/iommu.c |  2 +-
 3 files changed, 16 insertions(+), 3 deletions(-)

diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h
index 775f1c4ae346..2cee36138d6e 100644
--- a/drivers/iommu/intel/iommu.h
+++ b/drivers/iommu/intel/iommu.h
@@ -1354,6 +1354,7 @@ static inline bool ecmd_has_pmu_essential(struct intel_iommu *iommu)
 
 extern int dmar_disabled;
 extern int intel_iommu_enabled;
+extern int intel_iommu_tboot_noforce;
 #else
 static inline int iommu_calculate_agaw(struct intel_iommu *iommu)
 {
@@ -1366,6 +1367,7 @@ static inline int iommu_calculate_max_sagaw(struct intel_iommu *iommu)
 #define dmar_disabled	(1)
 #define intel_iommu_enabled (0)
 #define intel_iommu_sm (0)
+#define intel_iommu_tboot_noforce (0)
 #endif
 
 static inline const char *decode_prq_descriptor(char *str, size_t size,
diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c
index 767ec092accd..e32685402f74 100644
--- a/drivers/iommu/intel/dmar.c
+++ b/drivers/iommu/intel/dmar.c
@@ -915,6 +915,18 @@ dmar_validate_one_drhd(struct acpi_dmar_header *entry, void *arg)
 	return 0;
 }
 
+static bool dmar_required(void)
+{
+	/* tboot supersedes any user/platform opt */
+	if (!intel_iommu_tboot_noforce && tboot_enabled())
+		return true;
+
+	if (!no_iommu && (!dmar_disabled || dmar_platform_optin()))
+		return true;
+
+	return false;
+}
+
 void __init detect_intel_iommu(void)
 {
 	int ret;
@@ -928,8 +940,7 @@ void __init detect_intel_iommu(void)
 	if (!ret)
 		ret = dmar_walk_dmar_table((struct acpi_table_dmar *)dmar_tbl,
 					   &validate_drhd_cb);
-	if (!ret && !no_iommu && !iommu_detected &&
-	    (!dmar_disabled || dmar_platform_optin())) {
+	if (!ret && !iommu_detected && dmar_required()) {
 		iommu_detected = 1;
 		/* Make sure ACS will be enabled */
 		pci_request_acs();
diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index 1a1f27a51063..4d03d9a517de 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -57,7 +57,7 @@ static int rwbf_quirk;
  * (used when kernel is launched w/ TXT)
  */
 static int force_on = 0;
-static int intel_iommu_tboot_noforce;
+int intel_iommu_tboot_noforce;
 static int no_platform_optin;
 
 #define ROOT_ENTRY_NR (VTD_PAGE_SIZE/sizeof(struct root_entry))
-- 
2.43.0

[PATCH 07/20] iommu/vt-d: Remove dead code when CONFIG_INTEL_IOMMU is not set

[Lu Baolu]

From: Kevin Tian <kevin.tian@intel.com>

Those are leftovers and unreachable now: the entire intel directory
is built only when CONFIG_INTEL_IOMMU is set.

Signed-off-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
---
 drivers/iommu/intel/iommu.h | 15 ---------------
 1 file changed, 15 deletions(-)

diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h
index 2cee36138d6e..785aa3b62055 100644
--- a/drivers/iommu/intel/iommu.h
+++ b/drivers/iommu/intel/iommu.h
@@ -1340,7 +1340,6 @@ static inline bool intel_domain_is_ss_paging(struct dmar_domain *domain)
 	return domain->domain.ops == &intel_ss_paging_domain_ops;
 }
 
-#ifdef CONFIG_INTEL_IOMMU
 extern int intel_iommu_sm;
 int iommu_calculate_agaw(struct intel_iommu *iommu);
 int iommu_calculate_max_sagaw(struct intel_iommu *iommu);
@@ -1355,20 +1354,6 @@ static inline bool ecmd_has_pmu_essential(struct intel_iommu *iommu)
 extern int dmar_disabled;
 extern int intel_iommu_enabled;
 extern int intel_iommu_tboot_noforce;
-#else
-static inline int iommu_calculate_agaw(struct intel_iommu *iommu)
-{
-	return 0;
-}
-static inline int iommu_calculate_max_sagaw(struct intel_iommu *iommu)
-{
-	return 0;
-}
-#define dmar_disabled	(1)
-#define intel_iommu_enabled (0)
-#define intel_iommu_sm (0)
-#define intel_iommu_tboot_noforce (0)
-#endif
 
 static inline const char *decode_prq_descriptor(char *str, size_t size,
 		u64 dw0, u64 dw1, u64 dw2, u64 dw3)
-- 
2.43.0

[PATCH 08/20] iommu/vt-d: Consolidate dmar policy management and force_on logic

[Lu Baolu]

From: Kevin Tian <kevin.tian@intel.com>

Currently the dmar on/off is carried by multiple variables (no_iommu,
dmar_disabled, no_platform_optin, etc.) with error-prone force_on logic
scattered in multiple places.

Unify/centralize the policy/priority management for various force_on
scenarios.

No functional impact except one case - "intel_iommu=off" sets
no_platform_optin which is checked in platform_optin_force_iommu()
but not in detect_intel_iommu(), leading to ACS unnecessarily requested
when iommu could not be forced on later. Now with the unified logic
this becomes more consistent.

Signed-off-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
---
 drivers/iommu/intel/iommu.h | 45 ++++++++++++++++++++++++++++
 drivers/iommu/intel/dmar.c  | 58 ++++++++++++++++++++++++++++++++++---
 drivers/iommu/intel/iommu.c |  7 +++++
 3 files changed, 106 insertions(+), 4 deletions(-)

diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h
index 785aa3b62055..dd2376a079b9 100644
--- a/drivers/iommu/intel/iommu.h
+++ b/drivers/iommu/intel/iommu.h
@@ -1351,6 +1351,51 @@ static inline bool ecmd_has_pmu_essential(struct intel_iommu *iommu)
 		DMA_ECMD_ECCAP3_ESSENTIAL;
 }
 
+enum dmar_force_on {
+	DMAR_FORCEON_PLATFORM,
+	DMAR_FORCEON_TBOOT
+};
+
+/*
+ * On policies are positive, with more positive value being stronger.
+ * Off policies are negative, with more negative value being stronger.
+ *
+ * 'dmar' here refers to DMA remapping instead of the dmar/iommu unit.
+ *
+ * - DMAR_FORCE_ON:
+ *     force to turn on (e.g. by tboot or platform opt-in).
+ *
+ * - DMAR_ON:
+ *     turn on by build configuration (CONFIG_INTEL_IOMMU_DEFAULT_ON=on)
+ *     or user opts ("intel_iommu=on").
+ *
+ * - DMAR_DEFAULT_OFF
+ *     turn off by build configuration (CONFIG_INTEL_IOMMU_DEFAULT_ON=off).
+ *
+ * - DMAR_USER_OFF
+ *     turn off by user opts ("intel_iommu=off" or "iommu=off").
+ *
+ * - '0' is invalid, compared to decide the on/off policy
+ *
+ */
+#define DMAR_FORCE_ON		2
+#define DMAR_ON			1
+#define DMAR_DEFAULT_OFF	-1
+#define DMAR_USER_OFF		-2
+extern int dmar_policy;
+
+static inline bool dmar_policy_on(void)
+{
+	return dmar_policy > 0;
+}
+
+static inline bool dmar_policy_off(void)
+{
+	return dmar_policy < 0;
+}
+
+bool dmar_can_force_on(enum dmar_force_on force_on);
+
 extern int dmar_disabled;
 extern int intel_iommu_enabled;
 extern int intel_iommu_tboot_noforce;
diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c
index e32685402f74..bc2f6597eb27 100644
--- a/drivers/iommu/intel/dmar.c
+++ b/drivers/iommu/intel/dmar.c
@@ -915,14 +915,61 @@ dmar_validate_one_drhd(struct acpi_dmar_header *entry, void *arg)
 	return 0;
 }
 
+/*
+ * Centralized helper for deciding the force_on policy
+ *
+ * dmar off policies (for DMA Remapping) are defined from stronger
+ * (more negative values) to weaker (less negative values).
+ *
+ * When a force_on type is passed in, it is associated to a reference
+ * level for comparison. force_on is permitted when dmar is in a
+ * off policy less negative than the reference level (if the policy is
+ * on then the check is always true).
+ *
+ * For supported force_on types:
+ *
+ * - DMAR_FORCEON_TBOOT: tboot strictly requires DMA remapping for secure
+ *   boot hence supersedes any user opts ("iommu=off" or "intel_iommu=off")
+ *   and weaker off policies.
+ *
+ * - DMAR_FORCEON_PLATFORM: external-facing devices requires DMA
+ *   remapping to prevent malicious downstream external devices from
+ *   composing DMA attacks. force_on is permitted only if dmar policy is
+ *   off by build configurations (CONFIG_INTEL_IOMMU_DEFAULT_ON=off).
+ *
+ * In a nutshell, "trusted boot environment" is considered stronger than
+ * "user choices", which in turn is stronger than "platform opt-in hint".
+ */
+bool dmar_can_force_on(enum dmar_force_on force_on)
+{
+	int level;
+
+	switch (force_on) {
+	case DMAR_FORCEON_TBOOT:
+		level = DMAR_USER_OFF;
+		break;
+	case DMAR_FORCEON_PLATFORM:
+		level = DMAR_DEFAULT_OFF;
+		break;
+	default:
+		level = INT_MAX;
+		pr_warn("Unsupported force_on type (%d)\n", force_on);
+		break;
+	}
+
+	return dmar_policy >= level;
+}
+
 static bool dmar_required(void)
 {
-	/* tboot supersedes any user/platform opt */
+	if (dmar_policy_on())
+		return true;
+
 	if (!intel_iommu_tboot_noforce && tboot_enabled())
-		return true;
+		return dmar_can_force_on(DMAR_FORCEON_TBOOT);
 
-	if (!no_iommu && (!dmar_disabled || dmar_platform_optin()))
-		return true;
+	if (dmar_platform_optin())
+		return dmar_can_force_on(DMAR_FORCEON_PLATFORM);
 
 	return false;
 }
@@ -936,6 +983,9 @@ void __init detect_intel_iommu(void)
 	};
 
 	down_write(&dmar_global_lock);
+	if (no_iommu)
+		dmar_policy = DMAR_USER_OFF;
+
 	ret = dmar_table_detect();
 	if (!ret)
 		ret = dmar_walk_dmar_table((struct acpi_table_dmar *)dmar_tbl,
diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index 4d03d9a517de..2f0cd1714923 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -199,6 +199,11 @@ static LIST_HEAD(dmar_satc_units);
 
 static void intel_iommu_domain_free(struct iommu_domain *domain);
 
+#ifdef CONFIG_INTEL_IOMMU_DEFAULT_ON
+int dmar_policy = DMAR_ON;
+#else
+int dmar_policy = DMAR_DEFAULT_OFF;
+#endif
 int dmar_disabled = !IS_ENABLED(CONFIG_INTEL_IOMMU_DEFAULT_ON);
 int intel_iommu_sm = IS_ENABLED(CONFIG_INTEL_IOMMU_SCALABLE_MODE_DEFAULT_ON);
 
@@ -240,9 +245,11 @@ static int __init intel_iommu_setup(char *str)
 
 	while (*str) {
 		if (!strncmp(str, "on", 2)) {
+			dmar_policy = DMAR_ON;
 			dmar_disabled = 0;
 			pr_info("IOMMU enabled\n");
 		} else if (!strncmp(str, "off", 3)) {
+			dmar_policy = DMAR_USER_OFF;
 			dmar_disabled = 1;
 			no_platform_optin = 1;
 			pr_info("IOMMU disabled\n");
-- 
2.43.0

[PATCH 09/20] iommu/vt-d: Use dmar_can_force_on() for platform opt-in

[Lu Baolu]

From: Kevin Tian <kevin.tian@intel.com>

So the policy of requesting ACS in detect_intel_iommu() is consistent
with that in platform_optin_force_iommu().

While at it, remove no_platform_optin which is unnecessary now.

Signed-off-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
---
 drivers/iommu/intel/iommu.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index 2f0cd1714923..ce0794e82e55 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -58,7 +58,6 @@ static int rwbf_quirk;
  */
 static int force_on = 0;
 int intel_iommu_tboot_noforce;
-static int no_platform_optin;
 
 #define ROOT_ENTRY_NR (VTD_PAGE_SIZE/sizeof(struct root_entry))
 
@@ -251,7 +250,6 @@ static int __init intel_iommu_setup(char *str)
 		} else if (!strncmp(str, "off", 3)) {
 			dmar_policy = DMAR_USER_OFF;
 			dmar_disabled = 1;
-			no_platform_optin = 1;
 			pr_info("IOMMU disabled\n");
 		} else if (!strncmp(str, "igfx_off", 8)) {
 			disable_igfx_iommu = 1;
@@ -2491,20 +2489,23 @@ static bool has_external_pci(void)
 
 static int __init platform_optin_force_iommu(void)
 {
-	if (no_iommu || !dmar_platform_optin() || no_platform_optin ||
-	    !has_external_pci())
+	if (!dmar_platform_optin() || !dmar_can_force_on(DMAR_FORCEON_PLATFORM))
 		return 0;
 
-	if (dmar_disabled)
-		pr_info("Intel-IOMMU force enabled due to platform opt in\n");
+	if (!has_external_pci())
+		return 0;
 
 	/*
 	 * If Intel-IOMMU is disabled by default, we will apply identity
 	 * map for all devices except those marked as being untrusted.
 	 */
-	if (dmar_disabled)
+	if (dmar_policy_off()) {
+		pr_info("Intel-IOMMU force enabled due to platform opt in\n");
 		iommu_set_default_passthrough(false);
+	}
 
+	/* No concurrent access to dmar_policy at this point. */
+	dmar_policy = DMAR_FORCE_ON;
 	dmar_disabled = 0;
 
 	return 1;
-- 
2.43.0

[PATCH 10/20] iommu/vt-d: Call dmar_can_force_on() for tboot opt-in

[Lu Baolu]

From: Kevin Tian <kevin.tian@intel.com>

So the policy of requesting ACS in detect_intel_iommu() is consistent
with that in tboot_force_iommu().

Though tboot is the strongest override so far, dmar_can_force_on() may
return false due to future extensions. In this case panic the kernel,
as is already done when failing to initialize DMA remapping for tboot.

No functional impact at this point.

Signed-off-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
---
 drivers/iommu/intel/iommu.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index ce0794e82e55..5eb80aeec274 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -2550,12 +2550,17 @@ static int __init probe_acpi_namespace_devices(void)
 
 static __init int tboot_force_iommu(void)
 {
-	if (!tboot_enabled())
+	if (!tboot_enabled() || intel_iommu_tboot_noforce)
 		return 0;
 
-	if (no_iommu || dmar_disabled)
+	if (!dmar_can_force_on(DMAR_FORCEON_TBOOT))
+		panic("tboot: Failed to force IOMMU on\n");
+
+	if (dmar_policy_off())
 		pr_warn("Forcing Intel-IOMMU to enabled\n");
 
+	/* No concurrent access to dmar_policy at this point. */
+	dmar_policy = DMAR_FORCE_ON;
 	dmar_disabled = 0;
 	no_iommu = 0;
 
@@ -2572,8 +2577,7 @@ int __init intel_iommu_init(void)
 	 * Intel IOMMU is required for a TXT/tboot launch or platform
 	 * opt in, so enforce that.
 	 */
-	force_on = (!intel_iommu_tboot_noforce && tboot_force_iommu()) ||
-		    platform_optin_force_iommu();
+	force_on = tboot_force_iommu() || platform_optin_force_iommu();
 
 	down_write(&dmar_global_lock);
 	if (dmar_table_init()) {
-- 
2.43.0

[PATCH 11/20] iommu/vt-d: Remove the 'force_on' variable

[Lu Baolu]

From: Kevin Tian <kevin.tian@intel.com>

The force_on variable is now redundant - same information captured
by "dmar_policy == DMAR_FORCE_ON". Replace all force_on checks
with dmar_policy_force_on().

Signed-off-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
---
 drivers/iommu/intel/iommu.h |  5 +++++
 drivers/iommu/intel/iommu.c | 37 +++++++++++++++++--------------------
 2 files changed, 22 insertions(+), 20 deletions(-)

diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h
index dd2376a079b9..40da2ed7e254 100644
--- a/drivers/iommu/intel/iommu.h
+++ b/drivers/iommu/intel/iommu.h
@@ -1394,6 +1394,11 @@ static inline bool dmar_policy_off(void)
 	return dmar_policy < 0;
 }
 
+static inline bool dmar_policy_force_on(void)
+{
+	return dmar_policy == DMAR_FORCE_ON;
+}
+
 bool dmar_can_force_on(enum dmar_force_on force_on);
 
 extern int dmar_disabled;
diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index 5eb80aeec274..118c62e859fd 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -53,10 +53,9 @@ static int rwbf_quirk;
 #define rwbf_required(iommu)	(rwbf_quirk || cap_rwbf((iommu)->cap))
 
 /*
- * set to 1 to panic kernel if can't successfully enable VT-d
- * (used when kernel is launched w/ TXT)
+ * Skip forcing iommu on and avoid tboot-related kernel panics during
+ * initialization when set to 1 (via intel_iommu=tboot_noforce).
  */
-static int force_on = 0;
 int intel_iommu_tboot_noforce;
 
 #define ROOT_ENTRY_NR (VTD_PAGE_SIZE/sizeof(struct root_entry))
@@ -1713,7 +1712,7 @@ static int __init init_dmars(void)
 			 * we always have to disable PMRs or DMA may fail on
 			 * this device
 			 */
-			if (force_on)
+			if (dmar_policy_force_on())
 				iommu_disable_protect_mem_regions(iommu);
 			continue;
 		}
@@ -1805,7 +1804,7 @@ static int init_iommu_hw(void)
 			 * we always have to disable PMRs or DMA may fail on
 			 * this device
 			 */
-			if (force_on)
+			if (dmar_policy_force_on())
 				iommu_disable_protect_mem_regions(iommu);
 			continue;
 		}
@@ -1866,7 +1865,7 @@ static void iommu_resume(void *data)
 	unsigned long flag;
 
 	if (init_iommu_hw()) {
-		if (force_on)
+		if (dmar_policy_force_on())
 			panic("tboot: IOMMU setup failed, DMAR can not resume!\n");
 		else
 			WARN(1, "IOMMU setup failed, DMAR can not resume!\n");
@@ -2134,7 +2133,7 @@ static int intel_iommu_add(struct dmar_drhd_unit *dmaru)
 		/*
 		 * we always have to disable PMRs or DMA may fail on this device
 		 */
-		if (force_on)
+		if (dmar_policy_force_on())
 			iommu_disable_protect_mem_regions(iommu);
 		return 0;
 	}
@@ -2487,13 +2486,13 @@ static bool has_external_pci(void)
 	return false;
 }
 
-static int __init platform_optin_force_iommu(void)
+static void __init platform_optin_force_iommu(void)
 {
 	if (!dmar_platform_optin() || !dmar_can_force_on(DMAR_FORCEON_PLATFORM))
-		return 0;
+		return;
 
 	if (!has_external_pci())
-		return 0;
+		return;
 
 	/*
 	 * If Intel-IOMMU is disabled by default, we will apply identity
@@ -2507,8 +2506,6 @@ static int __init platform_optin_force_iommu(void)
 	/* No concurrent access to dmar_policy at this point. */
 	dmar_policy = DMAR_FORCE_ON;
 	dmar_disabled = 0;
-
-	return 1;
 }
 
 static int __init probe_acpi_namespace_devices(void)
@@ -2548,10 +2545,10 @@ static int __init probe_acpi_namespace_devices(void)
 	return 0;
 }
 
-static __init int tboot_force_iommu(void)
+static __init void tboot_force_iommu(void)
 {
 	if (!tboot_enabled() || intel_iommu_tboot_noforce)
-		return 0;
+		return;
 
 	if (!dmar_can_force_on(DMAR_FORCEON_TBOOT))
 		panic("tboot: Failed to force IOMMU on\n");
@@ -2563,8 +2560,6 @@ static __init int tboot_force_iommu(void)
 	dmar_policy = DMAR_FORCE_ON;
 	dmar_disabled = 0;
 	no_iommu = 0;
-
-	return 1;
 }
 
 int __init intel_iommu_init(void)
@@ -2577,17 +2572,19 @@ int __init intel_iommu_init(void)
 	 * Intel IOMMU is required for a TXT/tboot launch or platform
 	 * opt in, so enforce that.
 	 */
-	force_on = tboot_force_iommu() || platform_optin_force_iommu();
+	tboot_force_iommu();
+	if (!dmar_policy_force_on())
+		platform_optin_force_iommu();
 
 	down_write(&dmar_global_lock);
 	if (dmar_table_init()) {
-		if (force_on)
+		if (dmar_policy_force_on())
 			panic("tboot: Failed to initialize DMAR table\n");
 		goto out_free_dmar;
 	}
 
 	if (dmar_dev_scope_init() < 0) {
-		if (force_on)
+		if (dmar_policy_force_on())
 			panic("tboot: Failed to initialize DMAR device scope\n");
 		goto out_free_dmar;
 	}
@@ -2641,7 +2638,7 @@ int __init intel_iommu_init(void)
 
 	ret = init_dmars();
 	if (ret) {
-		if (force_on)
+		if (dmar_policy_force_on())
 			panic("tboot: Failed to initialize DMARs\n");
 		pr_err("Initialization failed\n");
 		goto out_free_dmar;
-- 
2.43.0

[PATCH 12/20] iommu/vt-d: Remove dmar_disabled

[Lu Baolu]

From: Kevin Tian <kevin.tian@intel.com>

It's replaced by dmar_policy_off() now, covering both "iommu=off"
and "intel_iommu=off". Also remove unnecessary checks on no_iommu,
leaving only one exception in intel_iommu_init() which skips debugfs
init for "iommu=off" but not "intel_iommu=off". Keep it to avoid
surprise for now.

Signed-off-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
---
 drivers/iommu/intel/iommu.h | 1 -
 drivers/iommu/intel/iommu.c | 9 ++-------
 drivers/iommu/intel/svm.c   | 2 +-
 3 files changed, 3 insertions(+), 9 deletions(-)

diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h
index 40da2ed7e254..9805edb8c7df 100644
--- a/drivers/iommu/intel/iommu.h
+++ b/drivers/iommu/intel/iommu.h
@@ -1401,7 +1401,6 @@ static inline bool dmar_policy_force_on(void)
 
 bool dmar_can_force_on(enum dmar_force_on force_on);
 
-extern int dmar_disabled;
 extern int intel_iommu_enabled;
 extern int intel_iommu_tboot_noforce;
 
diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index 118c62e859fd..a996b72aca57 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -202,7 +202,6 @@ int dmar_policy = DMAR_ON;
 #else
 int dmar_policy = DMAR_DEFAULT_OFF;
 #endif
-int dmar_disabled = !IS_ENABLED(CONFIG_INTEL_IOMMU_DEFAULT_ON);
 int intel_iommu_sm = IS_ENABLED(CONFIG_INTEL_IOMMU_SCALABLE_MODE_DEFAULT_ON);
 
 int intel_iommu_enabled = 0;
@@ -244,11 +243,9 @@ static int __init intel_iommu_setup(char *str)
 	while (*str) {
 		if (!strncmp(str, "on", 2)) {
 			dmar_policy = DMAR_ON;
-			dmar_disabled = 0;
 			pr_info("IOMMU enabled\n");
 		} else if (!strncmp(str, "off", 3)) {
 			dmar_policy = DMAR_USER_OFF;
-			dmar_disabled = 1;
 			pr_info("IOMMU disabled\n");
 		} else if (!strncmp(str, "igfx_off", 8)) {
 			disable_igfx_iommu = 1;
@@ -2371,7 +2368,7 @@ void intel_iommu_shutdown(void)
 	struct dmar_drhd_unit *drhd;
 	struct intel_iommu *iommu = NULL;
 
-	if (no_iommu || dmar_disabled)
+	if (dmar_policy_off())
 		return;
 
 	/*
@@ -2505,7 +2502,6 @@ static void __init platform_optin_force_iommu(void)
 
 	/* No concurrent access to dmar_policy at this point. */
 	dmar_policy = DMAR_FORCE_ON;
-	dmar_disabled = 0;
 }
 
 static int __init probe_acpi_namespace_devices(void)
@@ -2558,7 +2554,6 @@ static __init void tboot_force_iommu(void)
 
 	/* No concurrent access to dmar_policy at this point. */
 	dmar_policy = DMAR_FORCE_ON;
-	dmar_disabled = 0;
 	no_iommu = 0;
 }
 
@@ -2602,7 +2597,7 @@ int __init intel_iommu_init(void)
 	if (!no_iommu)
 		intel_iommu_debugfs_init();
 
-	if (no_iommu || dmar_disabled) {
+	if (dmar_policy_off()) {
 		/*
 		 * We exit the function here to ensure IOMMU's remapping and
 		 * mempool aren't setup, which means that the IOMMU's PMRs
diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c
index 726f7b6d0bff..432e35723ce9 100644
--- a/drivers/iommu/intel/svm.c
+++ b/drivers/iommu/intel/svm.c
@@ -115,7 +115,7 @@ static int intel_iommu_sva_supported(struct device *dev)
 	struct device_domain_info *info = dev_iommu_priv_get(dev);
 	struct intel_iommu *iommu;
 
-	if (!info || dmar_disabled)
+	if (!info || dmar_policy_off())
 		return -EINVAL;
 
 	iommu = info->iommu;
-- 
2.43.0

[PATCH 13/20] iommu/vt-d: Support the new DMA_REMAP_OPT_OUT flag bit

[Lu Baolu]

From: Kevin Tian <kevin.tian@intel.com>

Some BIOS already provides config options to expose/hide VT-d units
as a whole to/from system software. A new demand is to allow exposing
VT-d units but requesting system software to disable DMA remapping
while sustaining interrupt remapping. This can be communicated now by
setting the new DMA_REMAP_OPT_OUT flag bit in the DMAR table, as
introduced in VT-d spec v5.2 (section 8.1, DMA Remapping Reporting
Structure).

Introduce a new off policy (DMAR_FW_OFF) for DMA_REMAP_OPT_OUT. As
the strongest off policy, it cannot be overridden by user opts or
any force_on types. If tboot is enabled in the meantime, kernel will
panic. It is user responsibility to configure BIOS properly.

One cleanup is left for future - the DMAR flag is parsed multiple
times, in detect_intel_iommu(), dmar_platform_optin() (which can be
called at run-time), etc. Caching it is a cleaner way.

Signed-off-by: Kevin Tian <kevin.tian@intel.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
---
 drivers/iommu/intel/iommu.h |  4 ++++
 include/linux/dmar.h        |  1 +
 drivers/iommu/intel/dmar.c  | 34 ++++++++++++++++++++++++----------
 3 files changed, 29 insertions(+), 10 deletions(-)

diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h
index 9805edb8c7df..656cd311ed9a 100644
--- a/drivers/iommu/intel/iommu.h
+++ b/drivers/iommu/intel/iommu.h
@@ -1375,6 +1375,9 @@ enum dmar_force_on {
  * - DMAR_USER_OFF
  *     turn off by user opts ("intel_iommu=off" or "iommu=off").
  *
+ * - DMAR_FW_OFF
+ *     turn off due to firmware opt-out (DMAR_REMAP_OPT_OUT)
+ *
  * - '0' is invalid, compared to decide the on/off policy
  *
  */
@@ -1382,6 +1385,7 @@ enum dmar_force_on {
 #define DMAR_ON			1
 #define DMAR_DEFAULT_OFF	-1
 #define DMAR_USER_OFF		-2
+#define DMAR_FW_OFF		-3
 extern int dmar_policy;
 
 static inline bool dmar_policy_on(void)
diff --git a/include/linux/dmar.h b/include/linux/dmar.h
index 692b2b445761..63e35df2cef4 100644
--- a/include/linux/dmar.h
+++ b/include/linux/dmar.h
@@ -24,6 +24,7 @@ struct acpi_dmar_header;
 #define DMAR_INTR_REMAP		0x1
 #define DMAR_X2APIC_OPT_OUT	0x2
 #define DMAR_PLATFORM_OPT_IN	0x4
+#define DMAR_REMAP_OPT_OUT	0x8
 
 struct intel_iommu;
 
diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c
index bc2f6597eb27..33bfaeafa7c6 100644
--- a/drivers/iommu/intel/dmar.c
+++ b/drivers/iommu/intel/dmar.c
@@ -930,7 +930,9 @@ dmar_validate_one_drhd(struct acpi_dmar_header *entry, void *arg)
  *
  * - DMAR_FORCEON_TBOOT: tboot strictly requires DMA remapping for secure
  *   boot hence supersedes any user opts ("iommu=off" or "intel_iommu=off")
- *   and weaker off policies.
+ *   and weaker off policies. But if firmware forces DMA remapping off (by
+ *   setting DMAR_REMAP_OPT_OUT in the DMAR table), no force_on is allowed.
+ *   Firmware settings must be changed to unblock tboot.
  *
  * - DMAR_FORCEON_PLATFORM: external-facing devices requires DMA
  *   remapping to prevent malicious downstream external devices from
@@ -939,6 +941,7 @@ dmar_validate_one_drhd(struct acpi_dmar_header *entry, void *arg)
  *
  * In a nutshell, "trusted boot environment" is considered stronger than
  * "user choices", which in turn is stronger than "platform opt-in hint".
+ * But they are all meaningless when it's forced off by "firmware".
  */
 bool dmar_can_force_on(enum dmar_force_on force_on)
 {
@@ -976,31 +979,42 @@ static bool dmar_required(void)
 
 void __init detect_intel_iommu(void)
 {
-	int ret;
 	struct dmar_res_callback validate_drhd_cb = {
 		.cb[ACPI_DMAR_TYPE_HARDWARE_UNIT] = &dmar_validate_one_drhd,
 		.ignore_unhandled = true,
 	};
+	struct acpi_table_dmar *dmar;
+	int ret;
 
 	down_write(&dmar_global_lock);
 	if (no_iommu)
 		dmar_policy = DMAR_USER_OFF;
 
 	ret = dmar_table_detect();
-	if (!ret)
-		ret = dmar_walk_dmar_table((struct acpi_table_dmar *)dmar_tbl,
-					   &validate_drhd_cb);
-	if (!ret && !iommu_detected && dmar_required()) {
+	if (!ret) {
+		dmar = (struct acpi_table_dmar *)dmar_tbl;
+		ret = dmar_walk_dmar_table(dmar, &validate_drhd_cb);
+	}
+
+	if (ret)
+		goto out;
+
+	if (dmar->flags & DMAR_REMAP_OPT_OUT) {
+		dmar_policy = DMAR_FW_OFF;
+		pr_info("Firmware forces DMA remapping off\n");
+		pr_info("Any user opt or tboot/platform force_on will be ignored\n");
+	}
+
+	if (!iommu_detected && dmar_required()) {
 		iommu_detected = 1;
 		/* Make sure ACS will be enabled */
 		pci_request_acs();
 	}
 
-	if (!ret) {
-		x86_init.iommu.iommu_init = intel_iommu_init;
-		x86_platform.iommu_shutdown = intel_iommu_shutdown;
-	}
+	x86_init.iommu.iommu_init = intel_iommu_init;
+	x86_platform.iommu_shutdown = intel_iommu_shutdown;
 
+out:
 	if (dmar_tbl) {
 		acpi_put_table(dmar_tbl);
 		dmar_tbl = NULL;
-- 
2.43.0

[PATCH 14/20] iommu/vt-d: Cache max domain ID to avoid redundant calculation

[Lu Baolu]

The cap_ndoms() helper calculates the maximum available domain ID from
the value of capability register, which can be inefficient if called
repeatedly. Cache the maximum supported domain ID in max_domain_id field
during initialization to avoid redundant calls to cap_ndoms() throughout
the IOMMU driver.

No functionality change.

Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: Xu Yilun <yilun.xu@linux.intel.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
---
 drivers/iommu/intel/iommu.h |  1 +
 drivers/iommu/intel/dmar.c  |  1 +
 drivers/iommu/intel/iommu.c | 10 +++++-----
 3 files changed, 7 insertions(+), 5 deletions(-)

diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h
index 656cd311ed9a..c00f44db0020 100644
--- a/drivers/iommu/intel/iommu.h
+++ b/drivers/iommu/intel/iommu.h
@@ -700,6 +700,7 @@ struct intel_iommu {
 	/* mutex to protect domain_ida */
 	struct mutex	did_lock;
 	struct ida	domain_ida; /* domain id allocator */
+	unsigned long	max_domain_id;
 	unsigned long	*copied_tables; /* bitmap of copied tables */
 	spinlock_t	lock; /* protect context, domain ids */
 	struct root_entry *root_entry; /* virtual address */
diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c
index 33bfaeafa7c6..c854173f6be9 100644
--- a/drivers/iommu/intel/dmar.c
+++ b/drivers/iommu/intel/dmar.c
@@ -1174,6 +1174,7 @@ static int alloc_iommu(struct dmar_drhd_unit *drhd)
 	spin_lock_init(&iommu->lock);
 	ida_init(&iommu->domain_ida);
 	mutex_init(&iommu->did_lock);
+	iommu->max_domain_id = cap_ndoms(iommu->cap);
 
 	ver = readl(iommu->reg + DMAR_VER_REG);
 	pr_info("%s: reg_base_addr %llx ver %d:%d cap %llx ecap %llx\n",
diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index a996b72aca57..5ea584b76f77 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -1047,7 +1047,7 @@ int domain_attach_iommu(struct dmar_domain *domain, struct intel_iommu *iommu)
 	}
 
 	num = ida_alloc_range(&iommu->domain_ida, IDA_START_DID,
-			      cap_ndoms(iommu->cap) - 1, GFP_KERNEL);
+			      iommu->max_domain_id - 1, GFP_KERNEL);
 	if (num < 0) {
 		pr_err("%s: No free domain ids\n", iommu->name);
 		goto err_unlock;
@@ -1111,7 +1111,7 @@ static void copied_context_tear_down(struct intel_iommu *iommu,
 	did_old = context_domain_id(context);
 	context_clear_entry(context);
 
-	if (did_old < cap_ndoms(iommu->cap)) {
+	if (did_old < iommu->max_domain_id) {
 		iommu->flush.flush_context(iommu, did_old,
 					   PCI_DEVID(bus, devfn),
 					   DMA_CCMD_MASK_NOBIT,
@@ -1511,7 +1511,7 @@ static int copy_context_table(struct intel_iommu *iommu,
 			continue;
 
 		did = context_domain_id(&ce);
-		if (did >= 0 && did < cap_ndoms(iommu->cap))
+		if (did >= 0 && did < iommu->max_domain_id)
 			ida_alloc_range(&iommu->domain_ida, did, did, GFP_KERNEL);
 
 		set_context_copied(iommu, bus, devfn);
@@ -2431,7 +2431,7 @@ static ssize_t domains_supported_show(struct device *dev,
 				      struct device_attribute *attr, char *buf)
 {
 	struct intel_iommu *iommu = dev_to_intel_iommu(dev);
-	return sysfs_emit(buf, "%ld\n", cap_ndoms(iommu->cap));
+	return sysfs_emit(buf, "%ld\n", iommu->max_domain_id);
 }
 static DEVICE_ATTR_RO(domains_supported);
 
@@ -2442,7 +2442,7 @@ static ssize_t domains_used_show(struct device *dev,
 	unsigned int count = 0;
 	int id;
 
-	for (id = 0; id < cap_ndoms(iommu->cap); id++)
+	for (id = 0; id < iommu->max_domain_id; id++)
 		if (ida_exists(&iommu->domain_ida, id))
 			count++;
 
-- 
2.43.0

[PATCH 15/20] iommu/vt-d: Fix copied_tables bitmap leak on error in copy_translation_tables

[Lu Baolu]

From: ZhaoJinming <zhaojinming@uniontech.com>

The iommu->copied_tables bitmap was introduced by the IOMMU live
update series to track which context entries have been copied from
the previous kernel.  The allocation via bitmap_zalloc() was added
inside copy_translation_tables(), but the error paths were not
updated to free it:

  1. When old_rt_phys is 0 (invalid root table address)
  2. When memremap(old_rt_phys) fails
  3. When kcalloc for ctxt_tbls fails (goto out_unmap, which only
     unmaps old_rt without releasing the bitmap)

The bitmap is only cleaned up by free_dmar_iommu(), which is
called from the free_iommu error label in init_dmars().  However,
when copy_translation_tables() fails, init_dmars() does not jump
to free_iommu -- it logs the error, falls through, and continues
with the next IOMMU.  As a result, copied_tables is leaked.

Fix this by converting the two early returns to goto a new
err_free_bitmap label, and by making out_unmap fall through to
it so that the bitmap is always freed on any error path.  The
success path performs memunmap(old_rt) inline and returns 0
directly, since copied_tables must remain allocated for
subsequent use.

Signed-off-by: ZhaoJinming <zhaojinming@uniontech.com>
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
---
 drivers/iommu/intel/iommu.c | 19 +++++++++++++------
 1 file changed, 13 insertions(+), 6 deletions(-)

diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index 5ea584b76f77..7098a6bf6a40 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -1557,12 +1557,16 @@ static int copy_translation_tables(struct intel_iommu *iommu)
 		return -ENOMEM;
 
 	old_rt_phys = rtaddr_reg & VTD_PAGE_MASK;
-	if (!old_rt_phys)
-		return -EINVAL;
+	if (!old_rt_phys) {
+		ret = -EINVAL;
+		goto err_free_bitmap;
+	}
 
 	old_rt = memremap(old_rt_phys, PAGE_SIZE, MEMREMAP_WB);
-	if (!old_rt)
-		return -ENOMEM;
+	if (!old_rt) {
+		ret = -ENOMEM;
+		goto err_free_bitmap;
+	}
 
 	/* This is too big for the stack - allocate it from slab */
 	ctxt_table_entries = ext ? 512 : 256;
@@ -1606,11 +1610,14 @@ static int copy_translation_tables(struct intel_iommu *iommu)
 
 	__iommu_flush_cache(iommu, iommu->root_entry, PAGE_SIZE);
 
-	ret = 0;
+	memunmap(old_rt);
+	return 0;
 
 out_unmap:
 	memunmap(old_rt);
-
+err_free_bitmap:
+	bitmap_free(iommu->copied_tables);
+	iommu->copied_tables = NULL;
 	return ret;
 }
 
-- 
2.43.0

[PATCH 16/20] iommu/vt-d: Fix shift overflow in qi_desc_dev_iotlb_pasid()

[Lu Baolu]

Callers request a full Device-TLB flush by passing MAX_AGAW_PFN_WIDTH
(64 - VTD_PAGE_SHIFT == 52) as @size_order.  Two shifts in
qi_desc_dev_iotlb_pasid() are not prepared for a value that large:

  unsigned long mask = 1UL << (VTD_PAGE_SHIFT + size_order - 1);
  ...
  if (!IS_ALIGNED(addr, VTD_PAGE_SIZE << size_order))

The first evaluates to 1UL << 63.  On 32-bit builds this is undefined
behaviour; in practice x86 masks the shift count to 5 bits, so the
expression yields 1UL << 31 and ~mask becomes 0x7fffffff.  That value is
zero-extended when it is applied to the 64-bit descriptor, so

  desc->qw1 &= ~mask;

clears qw1[63:32] as well as bit 31.  The ADDR field, which had just been
filled with ones to request the widest possible range, collapses to
0x7ffff000.  As the S bit remains set, hardware decodes the least
significant zero bit of ADDR and invalidates only 2GiB instead of the
entire address space.  Device-TLB entries above that boundary survive the
unmap, leaving an ATS-capable device able to keep accessing memory that
has already been freed.

The second shift, VTD_PAGE_SIZE << size_order, is 1UL << 64 and is
therefore undefined on 64-bit builds too.  On x86_64 the shift count
masks to zero, IS_ALIGNED(addr, 1) is trivially true and the sanity check
silently degrades into a no-op.

Compute both quantities in 64-bit and clamp @size_order to the largest
range the ADDR field can encode.  Capping at 63 - VTD_PAGE_SHIFT keeps
the intended "flush everything" behaviour: qw1[62:12] is set, bit 62 is
cleared as the size indicator and the S bit is set.  The non-PASID
variant qi_desc_dev_iotlb() already uses 1ULL and is unaffected.

Fixes: f701c9f36bcb7 ("iommu/vt-d: Factor out invalidation descriptor composition")
Cc: stable@vger.kernel.org
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260623060122.3796325-1-guanghuifeng%40linux.alibaba.com
Assisted-by: Claude:claude-opus-5
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Reviewed-by: Samiullah Khawaja <skhawaja@google.com>
---
 drivers/iommu/intel/iommu.h | 16 ++++++++++++----
 1 file changed, 12 insertions(+), 4 deletions(-)

diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h
index c00f44db0020..8a59c7c9d0a6 100644
--- a/drivers/iommu/intel/iommu.h
+++ b/drivers/iommu/intel/iommu.h
@@ -1105,12 +1105,20 @@ static inline void qi_desc_dev_iotlb_pasid(u16 sid, u16 pfsid, u32 pasid,
 					   unsigned int size_order,
 					   struct qi_desc *desc)
 {
-	unsigned long mask = 1UL << (VTD_PAGE_SHIFT + size_order - 1);
-
 	desc->qw0 = QI_DEV_EIOTLB_PASID(pasid) | QI_DEV_EIOTLB_SID(sid) |
 		QI_DEV_EIOTLB_QDEP(qdep) | QI_DEIOTLB_TYPE |
 		QI_DEV_IOTLB_PFSID(pfsid);
 
+	/*
+	 * The invalidation range is encoded in the ADDR field, which only
+	 * covers bits 63:12.  Clamp @size_order so that callers asking for a
+	 * full flush (e.g. with MAX_AGAW_PFN_WIDTH) do not overflow the
+	 * shifts below.  The clamped value still spans the whole range that
+	 * the descriptor is able to express.
+	 */
+	if (size_order > 63 - VTD_PAGE_SHIFT)
+		size_order = 63 - VTD_PAGE_SHIFT;
+
 	/*
 	 * If S bit is 0, we only flush a single page. If S bit is set,
 	 * The least significant zero bit indicates the invalidation address
@@ -1120,7 +1128,7 @@ static inline void qi_desc_dev_iotlb_pasid(u16 sid, u16 pfsid, u32 pasid,
 	 * Max Invs Pending (MIP) is set to 0 for now until we have DIT in
 	 * ECAP.
 	 */
-	if (!IS_ALIGNED(addr, VTD_PAGE_SIZE << size_order))
+	if (!IS_ALIGNED(addr, BIT_ULL(VTD_PAGE_SHIFT + size_order)))
 		pr_warn_ratelimited("Invalidate non-aligned address %llx, order %d\n",
 				    addr, size_order);
 
@@ -1136,7 +1144,7 @@ static inline void qi_desc_dev_iotlb_pasid(u16 sid, u16 pfsid, u32 pasid,
 		desc->qw1 |= GENMASK_ULL(size_order + VTD_PAGE_SHIFT - 1,
 					VTD_PAGE_SHIFT);
 		/* Clear size_order bit to indicate size */
-		desc->qw1 &= ~mask;
+		desc->qw1 &= ~BIT_ULL(VTD_PAGE_SHIFT + size_order - 1);
 		/* Set the S bit to indicate flushing more than 1 page */
 		desc->qw1 |= QI_DEV_EIOTLB_SIZE;
 	}
-- 
2.43.0

[PATCH 17/20] iommu/vt-d: Clear Present bit before tearing down copied context entry

[Lu Baolu]

copied_context_tear_down() zeroes the 128-bit context entry with
context_clear_entry() while the Present bit is still set, and only then
issues the context-cache and IOTLB invalidations.  This leaves a window
in which hardware can fetch a torn entry, with some fields already zeroed
while Present is still set, leading to unpredictable behaviour or
spurious faults.  While x86 provides strong write ordering, the compiler
may reorder the writes to the two 64-bit halves of the entry, and the
hardware fetch is not guaranteed to be atomic with respect to multiple
CPU writes.

There is no cacheline flush before the invalidation either, so on an
IOMMU without coherent access to the context table the zeroed entry may
not be visible to hardware at the point the invalidation is submitted.

Apply the same ownership handshake described in the VT-d spec, Section
6.5.3.3 ("Guidance to Software for Invalidations"): clear only the Present
bit, flush it out to the IOMMU, perform the invalidations, and only then
zero the remainder of the entry.

Fixes: c7191984e5aad ("iommu/vt-d: Factor out helpers from domain_context_mapping_one()")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260602233426.357499-1-baolu.lu%40linux.intel.com
Assisted-by: Claude:claude-opus-5
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
---
 drivers/iommu/intel/iommu.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index 7098a6bf6a40..000b81cf4a7a 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -1109,7 +1109,8 @@ static void copied_context_tear_down(struct intel_iommu *iommu,
 	assert_spin_locked(&iommu->lock);
 
 	did_old = context_domain_id(context);
-	context_clear_entry(context);
+	context_clear_present(context);
+	__iommu_flush_cache(iommu, context, sizeof(*context));
 
 	if (did_old < iommu->max_domain_id) {
 		iommu->flush.flush_context(iommu, did_old,
@@ -1120,6 +1121,9 @@ static void copied_context_tear_down(struct intel_iommu *iommu,
 					 DMA_TLB_DSI_FLUSH);
 	}
 
+	context_clear_entry(context);
+	__iommu_flush_cache(iommu, context, sizeof(*context));
+
 	clear_context_copied(iommu, bus, devfn);
 }
 
-- 
2.43.0

[PATCH 18/20] iommu/vt-d: Fix iopf_refcount leak on RID domain replacement

[Lu Baolu]

intel_iommu_attach_device() enables IOPF for the new domain but never
disables it for the old one.  device_block_translation(), called at the
start of the function, tears down translation but does not touch any IOPF
state; blocking_domain_attach_dev() has to call iopf_for_domain_remove()
explicitly before invoking it for exactly this reason.

identity_domain_attach_dev() has the same problem.  Its comment claims
that no PRI handling is needed because the device has been put in the
blocking state, but the blocking state and the IOPF reference count are
independent of each other.

As a result, replacing a domain that has an iopf_handler with another
domain at RID level leaks a reference in info->iopf_refcount.  The count
never drops back to zero, so iopf_queue_remove_device() is never called
and iommu_disable_pci_pri() triggers its WARN_ON(info->iopf_refcount)
when the device is released.

The PASID paths already handle this correctly by way of
iopf_for_domain_replace(); convert the two RID paths to do the same.
Using the replace helper rather than a bare remove keeps the enable
before the disable, so the reference count does not transiently reach
zero and evict the device from the IOPF queue.

Fixes: 17fce9d2336d ("iommu/vt-d: Put iopf enablement in domain attach path")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260602233426.357499-1-baolu.lu%40linux.intel.com
Assisted-by: Claude:claude-opus-5
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
---
 drivers/iommu/intel/iommu.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index 000b81cf4a7a..9003783d02bc 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -3152,13 +3152,13 @@ static int intel_iommu_attach_device(struct iommu_domain *domain,
 	if (ret)
 		return ret;
 
-	ret = iopf_for_domain_set(domain, dev);
+	ret = iopf_for_domain_replace(domain, old, dev);
 	if (ret)
 		return ret;
 
 	ret = dmar_domain_attach_device(to_dmar_domain(domain), dev);
 	if (ret)
-		iopf_for_domain_remove(domain, dev);
+		iopf_for_domain_replace(old, domain, dev);
 
 	return ret;
 }
@@ -3861,10 +3861,13 @@ static int identity_domain_attach_dev(struct iommu_domain *domain,
 		return 0;
 
 	/*
-	 * No PRI support with the global identity domain. No need to enable or
-	 * disable PRI in this path as the iommu has been put in the blocking
-	 * state.
+	 * The identity domain has no iopf_handler, so no IOPF reference is
+	 * taken for it.  The reference held by the old domain must still be
+	 * released here; putting the device in the blocking state above does
+	 * not affect the IOPF reference count.
 	 */
+	iopf_for_domain_remove(old, dev);
+
 	if (sm_supported(iommu))
 		ret = intel_pasid_setup_pass_through(iommu, dev, IOMMU_NO_PASID);
 	else
-- 
2.43.0

[PATCH 19/20] iommu/vt-d: Tear down scalable-mode context on probe failure

[Lu Baolu]

intel_pasid_setup_sm_context() walks a PCI device’s DMA aliases via
pci_for_each_dma_alias() and programs a scalable-mode context entry for
each RID. For a device with a dma_alias_mask, the callback is invoked
once for the device’s own RID and once for each alias bit, all with the
same pci_dev, so device_pasid_table_setup() runs for multiple RIDs.

pci_for_each_dma_alias() stops at the first callback error. Therefore, a
failure partway through the walk can leave context entries for already
processed RIDs present and still pointing to the device’s PASID table.

On this error path, intel_iommu_probe_device() currently jumps directly
to intel_pasid_free_table(), which frees the PASID table without
first tearing down those context entries. The IOMMU may then walk a
present context entry whose PASID table pointer references freed
memory.

intel_iommu_release_device() already performs teardown before freeing the
table. Apply the same ordering on the probe failure path.

device_pasid_table_teardown() safely handles RIDs that were never
programmed: iommu_context_addr() returns NULL when no context table has
been allocated, and clearing the Present bit of an already non-present
entry is a no-op. So unwind is safe for both the alias that failed and
any aliases not yet reached.

Fixes: 301f1a80487fd ("iommu/vt-d: Setup scalable mode context entry in probe path")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260602233426.357499-1-baolu.lu%40linux.intel.com
Assisted-by: Claude:claude-opus-5
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
---
 drivers/iommu/intel/iommu.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index 9003783d02bc..489bab4999fb 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -3333,6 +3333,7 @@ static struct iommu_device *intel_iommu_probe_device(struct device *dev)
 
 	return &iommu->iommu;
 free_table:
+	intel_pasid_teardown_sm_context(dev);
 	intel_pasid_free_table(dev);
 clear_rbtree:
 	device_rbtree_remove(info);
-- 
2.43.0

[PATCH 20/20] iommu/vt-d: Flush context cache with correct SID when tearing down aliases

[Lu Baolu]

domain_context_clear_one() and device_pasid_table_teardown() are both
invoked once per DMA alias of a device. Each function locates the context
entry using the bus/devfn pair provided by the pci_for_each_dma_alias()
callback, then calls intel_context_flush_no_pasid(), which constructs a
device-selective context-cache invalidation from info->bus and
info->devfn (that is, always the requester ID of the device itself).

As a result, for every alias other than the device’s own RID, the context
entry that was just cleared in memory is never invalidated in the context
cache. Hardware may continue using that stale cached entry. In the
scalable-mode teardown path, intel_pasid_free_table() can then free the
PASID directory still referenced by that stale entry, allowing the IOMMU
to walk freed memory.

Fix this by passing the source ID of the entry being torn down to
intel_context_flush_no_pasid(), instead of deriving it from @info.

Fixes: f90584f4beb84 ("iommu/vt-d: Add helper to flush caches for context change")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://sashiko.dev/#/patchset/20260602233426.357499-1-baolu.lu%40linux.intel.com
Assisted-by: Claude:claude-opus-5
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Reviewed-by: Samiullah Khawaja <skhawaja@google.com>
---
 drivers/iommu/intel/iommu.h | 2 +-
 drivers/iommu/intel/iommu.c | 2 +-
 drivers/iommu/intel/pasid.c | 9 ++++++---
 3 files changed, 8 insertions(+), 5 deletions(-)

diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h
index 8a59c7c9d0a6..7f01620bf3a1 100644
--- a/drivers/iommu/intel/iommu.h
+++ b/drivers/iommu/intel/iommu.h
@@ -1249,7 +1249,7 @@ void cache_tag_flush_range_np(struct dmar_domain *domain, unsigned long start,
 			      unsigned long end);
 
 void intel_context_flush_no_pasid(struct device_domain_info *info,
-				  struct context_entry *context, u16 did);
+				  struct context_entry *context, u16 did, u16 sid);
 
 int intel_iommu_enable_prq(struct intel_iommu *iommu);
 int intel_iommu_finish_prq(struct intel_iommu *iommu);
diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index 489bab4999fb..6d81644c66bc 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -1251,7 +1251,7 @@ static void domain_context_clear_one(struct device_domain_info *info, u8 bus, u8
 	context_clear_present(context);
 	__iommu_flush_cache(iommu, context, sizeof(*context));
 	spin_unlock(&iommu->lock);
-	intel_context_flush_no_pasid(info, context, did);
+	intel_context_flush_no_pasid(info, context, did, PCI_DEVID(bus, devfn));
 	context_clear_entry(context);
 	__iommu_flush_cache(iommu, context, sizeof(*context));
 }
diff --git a/drivers/iommu/intel/pasid.c b/drivers/iommu/intel/pasid.c
index 81353fd46b37..e4f24d3f19a6 100644
--- a/drivers/iommu/intel/pasid.c
+++ b/drivers/iommu/intel/pasid.c
@@ -751,7 +751,7 @@ static void device_pasid_table_teardown(struct device *dev, u8 bus, u8 devfn)
 	context_clear_present(context);
 	__iommu_flush_cache(iommu, context, sizeof(*context));
 	spin_unlock(&iommu->lock);
-	intel_context_flush_no_pasid(info, context, did);
+	intel_context_flush_no_pasid(info, context, did, PCI_DEVID(bus, devfn));
 	context_clear_entry(context);
 	__iommu_flush_cache(iommu, context, sizeof(*context));
 }
@@ -955,9 +955,12 @@ static void __context_flush_dev_iotlb(struct device_domain_info *info)
  * This helper can only be used when IOMMU is working in the legacy mode or
  * IOMMU is in scalable mode but all PASID table entries of the device are
  * non-present.
+ *
+ * @sid identifies the context entry that was modified, which may be a DMA
+ * alias of @info->dev rather than its own requester ID.
  */
 void intel_context_flush_no_pasid(struct device_domain_info *info,
-				  struct context_entry *context, u16 did)
+				  struct context_entry *context, u16 did, u16 sid)
 {
 	struct intel_iommu *iommu = info->iommu;
 
@@ -967,7 +970,7 @@ void intel_context_flush_no_pasid(struct device_domain_info *info,
 	 * when operating in scalable mode. Therefore the @did value doesn't
 	 * matter in scalable mode.
 	 */
-	iommu->flush.flush_context(iommu, did, PCI_DEVID(info->bus, info->devfn),
+	iommu->flush.flush_context(iommu, did, sid,
 				   DMA_CCMD_MASK_NOBIT, DMA_CCMD_DEVICE_INVL);
 
 	/*
-- 
2.43.0

答复: [外部邮件] [PATCH 03/20] iommu/vt-d: Fix CACHE_TAG_NESTING_DEVTLB polluting shared variables in flush loop

[Li,Rongqing]

> From: Guanghui Feng <guanghuifeng@linux.alibaba.com>
> 
> In cache_tag_flush_range(), the CACHE_TAG_NESTING_DEVTLB case modifies the
> shared local variables 'addr' and 'mask' before falling through to
> CACHE_TAG_DEVTLB. This causes all subsequent CACHE_TAG_DEVTLB entries in
> the same loop iteration to incorrectly use the full-range flush parameters
> (addr=0, mask=MAX_AGAW_PFN_WIDTH) instead of the precisely calculated PSI
> range. This is not the intended behavior, as regular DEVTLB entries should always
> perform targeted range-based invalidation.
> 
> Fix this by having CACHE_TAG_NESTING_DEVTLB directly call
> cache_tag_flush_devtlb_psi() with the full-range constants and break, instead of
> modifying shared variables and falling through. This ensures
> CACHE_TAG_DEVTLB always uses the original calculated addr and mask for
> precise range flush.
> 
> Signed-off-by: Guanghui Feng <guanghuifeng@linux.alibaba.com>
> Signed-off-by: Guixin Liu <kanie@linux.alibaba.com>
> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
> ---
>  drivers/iommu/intel/cache.c | 5 ++---
>  1 file changed, 2 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/iommu/intel/cache.c b/drivers/iommu/intel/cache.c index
> fdc88817709f..26a758b0f501 100644
> --- a/drivers/iommu/intel/cache.c
> +++ b/drivers/iommu/intel/cache.c
> @@ -454,9 +454,8 @@ void cache_tag_flush_range(struct dmar_domain
> *domain, unsigned long start,
>  			 * affected by a change in S2. So just flush the entire
>  			 * device cache.
>  			 */
> -			addr = 0;
> -			mask = MAX_AGAW_PFN_WIDTH;
> -			fallthrough;
> +			cache_tag_flush_devtlb_psi(domain, tag, 0,
> MAX_AGAW_PFN_WIDTH);
> +			break;
>  		case CACHE_TAG_DEVTLB:
>  			cache_tag_flush_devtlb_psi(domain, tag, addr, mask);
>  			break;
> --
> 2.43.0

This patch introduces a subtle side effect on the tracing logic later in this function. 
At the end of cache_tag_flush_range(), trace_cache_tag_flush_range(tag, start, end, addr, mask) is called to log 
the flush operation. 

With this patch: bypassed the assignment and used break, addr and mask retain their original range values. 
This causes the tracepoint to log an incorrect, smaller range while the actual hardware execution was a full-range flush.

My patch has not this issue:
https://lore.kernel.org/linux-iommu/20260605003950.1720-1-lirongqing@baidu.com/

thanks

[Li,Rongqing] 

Re: 答复: [外部邮件] [PATCH 03/20] iommu/vt-d: Fix CACHE_TAG_NESTING_DEVTLB polluting shared variables in flush loop

[Baolu Lu]

On 8/4/2026 11:16 AM, Li,Rongqing wrote:
> 
> 
>> From: Guanghui Feng <guanghuifeng@linux.alibaba.com>
>>
>> In cache_tag_flush_range(), the CACHE_TAG_NESTING_DEVTLB case modifies the
>> shared local variables 'addr' and 'mask' before falling through to
>> CACHE_TAG_DEVTLB. This causes all subsequent CACHE_TAG_DEVTLB entries in
>> the same loop iteration to incorrectly use the full-range flush parameters
>> (addr=0, mask=MAX_AGAW_PFN_WIDTH) instead of the precisely calculated PSI
>> range. This is not the intended behavior, as regular DEVTLB entries should always
>> perform targeted range-based invalidation.
>>
>> Fix this by having CACHE_TAG_NESTING_DEVTLB directly call
>> cache_tag_flush_devtlb_psi() with the full-range constants and break, instead of
>> modifying shared variables and falling through. This ensures
>> CACHE_TAG_DEVTLB always uses the original calculated addr and mask for
>> precise range flush.
>>
>> Signed-off-by: Guanghui Feng <guanghuifeng@linux.alibaba.com>
>> Signed-off-by: Guixin Liu <kanie@linux.alibaba.com>
>> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
>> ---
>>   drivers/iommu/intel/cache.c | 5 ++---
>>   1 file changed, 2 insertions(+), 3 deletions(-)
>>
>> diff --git a/drivers/iommu/intel/cache.c b/drivers/iommu/intel/cache.c index
>> fdc88817709f..26a758b0f501 100644
>> --- a/drivers/iommu/intel/cache.c
>> +++ b/drivers/iommu/intel/cache.c
>> @@ -454,9 +454,8 @@ void cache_tag_flush_range(struct dmar_domain
>> *domain, unsigned long start,
>>   			 * affected by a change in S2. So just flush the entire
>>   			 * device cache.
>>   			 */
>> -			addr = 0;
>> -			mask = MAX_AGAW_PFN_WIDTH;
>> -			fallthrough;
>> +			cache_tag_flush_devtlb_psi(domain, tag, 0,
>> MAX_AGAW_PFN_WIDTH);
>> +			break;
>>   		case CACHE_TAG_DEVTLB:
>>   			cache_tag_flush_devtlb_psi(domain, tag, addr, mask);
>>   			break;
>> --
>> 2.43.0
> 
> This patch introduces a subtle side effect on the tracing logic later in this function.
> At the end of cache_tag_flush_range(), trace_cache_tag_flush_range(tag, start, end, addr, mask) is called to log
> the flush operation.
> 
> With this patch: bypassed the assignment and used break, addr and mask retain their original range values.
> This causes the tracepoint to log an incorrect, smaller range while the actual hardware execution was a full-range flush.

The tracepoint here records what the caller requested: a specific cache-
invalidation type for a specific range. In this helper, we may widen the
invalidation range for implementation reasons (as described in the
comments), but that does not change the caller’s original intent.
Therefore, this tracepoint should log the caller-requested range.

If we want to observe the actual invalidation range sent to hardware,
that is already covered by the qi_submit trace event, which logs the
real invalidation descriptors submitted by the driver.

> 
> My patch has not this issue:
> https://lore.kernel.org/linux-iommu/20260605003950.1720-1-lirongqing@baidu.com/
Thanks,
baolu

Re: [PATCH 16/20] iommu/vt-d: Fix shift overflow in qi_desc_dev_iotlb_pasid()

[Baolu Lu]

On 8/4/2026 10:37 AM, Lu Baolu wrote:
> Callers request a full Device-TLB flush by passing MAX_AGAW_PFN_WIDTH
> (64 - VTD_PAGE_SHIFT == 52) as @size_order.  Two shifts in
> qi_desc_dev_iotlb_pasid() are not prepared for a value that large:
> 
>    unsigned long mask = 1UL << (VTD_PAGE_SHIFT + size_order - 1);
>    ...
>    if (!IS_ALIGNED(addr, VTD_PAGE_SIZE << size_order))
> 
> The first evaluates to 1UL << 63.  On 32-bit builds this is undefined
> behaviour; in practice x86 masks the shift count to 5 bits, so the
> expression yields 1UL << 31 and ~mask becomes 0x7fffffff.  That value is
> zero-extended when it is applied to the 64-bit descriptor, so
> 
>    desc->qw1 &= ~mask;
> 
> clears qw1[63:32] as well as bit 31.  The ADDR field, which had just been
> filled with ones to request the widest possible range, collapses to
> 0x7ffff000.  As the S bit remains set, hardware decodes the least
> significant zero bit of ADDR and invalidates only 2GiB instead of the
> entire address space.  Device-TLB entries above that boundary survive the
> unmap, leaving an ATS-capable device able to keep accessing memory that
> has already been freed.
> 
> The second shift, VTD_PAGE_SIZE << size_order, is 1UL << 64 and is
> therefore undefined on 64-bit builds too.  On x86_64 the shift count
> masks to zero, IS_ALIGNED(addr, 1) is trivially true and the sanity check
> silently degrades into a no-op.
> 
> Compute both quantities in 64-bit and clamp @size_order to the largest
> range the ADDR field can encode.  Capping at 63 - VTD_PAGE_SHIFT keeps
> the intended "flush everything" behaviour: qw1[62:12] is set, bit 62 is
> cleared as the size indicator and the S bit is set.  The non-PASID
> variant qi_desc_dev_iotlb() already uses 1ULL and is unaffected.
> 
> Fixes: f701c9f36bcb7 ("iommu/vt-d: Factor out invalidation descriptor composition")
> Cc: stable@vger.kernel.org
> Reported-by: Sashiko <sashiko-bot@kernel.org>
> Closes: https://sashiko.dev/#/patchset/20260623060122.3796325-1-guanghuifeng%40linux.alibaba.com
> Assisted-by: Claude:claude-opus-5
> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
> Reviewed-by: Samiullah Khawaja <skhawaja@google.com>
> ---
>   drivers/iommu/intel/iommu.h | 16 ++++++++++++----
>   1 file changed, 12 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h
> index c00f44db0020..8a59c7c9d0a6 100644
> --- a/drivers/iommu/intel/iommu.h
> +++ b/drivers/iommu/intel/iommu.h
> @@ -1105,12 +1105,20 @@ static inline void qi_desc_dev_iotlb_pasid(u16 sid, u16 pfsid, u32 pasid,
>   					   unsigned int size_order,
>   					   struct qi_desc *desc)
>   {
> -	unsigned long mask = 1UL << (VTD_PAGE_SHIFT + size_order - 1);
> -
>   	desc->qw0 = QI_DEV_EIOTLB_PASID(pasid) | QI_DEV_EIOTLB_SID(sid) |
>   		QI_DEV_EIOTLB_QDEP(qdep) | QI_DEIOTLB_TYPE |
>   		QI_DEV_IOTLB_PFSID(pfsid);
>   
> +	/*
> +	 * The invalidation range is encoded in the ADDR field, which only
> +	 * covers bits 63:12.  Clamp @size_order so that callers asking for a
> +	 * full flush (e.g. with MAX_AGAW_PFN_WIDTH) do not overflow the
> +	 * shifts below.  The clamped value still spans the whole range that
> +	 * the descriptor is able to express.
> +	 */
> +	if (size_order > 63 - VTD_PAGE_SHIFT)
> +		size_order = 63 - VTD_PAGE_SHIFT;
> +

Sashiko reported a critical issue with this change. I will drop this
patch from the series and spend more time investigating and reworking
the fix.

https://sashiko.dev/#/patchset/20260804023714.3080506-1-baolu.lu%40linux.intel.com

>   	/*
>   	 * If S bit is 0, we only flush a single page. If S bit is set,
>   	 * The least significant zero bit indicates the invalidation address
> @@ -1120,7 +1128,7 @@ static inline void qi_desc_dev_iotlb_pasid(u16 sid, u16 pfsid, u32 pasid,
>   	 * Max Invs Pending (MIP) is set to 0 for now until we have DIT in
>   	 * ECAP.
>   	 */
> -	if (!IS_ALIGNED(addr, VTD_PAGE_SIZE << size_order))
> +	if (!IS_ALIGNED(addr, BIT_ULL(VTD_PAGE_SHIFT + size_order)))
>   		pr_warn_ratelimited("Invalidate non-aligned address %llx, order %d\n",
>   				    addr, size_order);
>   
> @@ -1136,7 +1144,7 @@ static inline void qi_desc_dev_iotlb_pasid(u16 sid, u16 pfsid, u32 pasid,
>   		desc->qw1 |= GENMASK_ULL(size_order + VTD_PAGE_SHIFT - 1,
>   					VTD_PAGE_SHIFT);
>   		/* Clear size_order bit to indicate size */
> -		desc->qw1 &= ~mask;
> +		desc->qw1 &= ~BIT_ULL(VTD_PAGE_SHIFT + size_order - 1);
>   		/* Set the S bit to indicate flushing more than 1 page */
>   		desc->qw1 |= QI_DEV_EIOTLB_SIZE;
>   	}

Thanks,
baolu

Re: 答复: [外部邮件] [PATCH 03/20] iommu/vt-d: Fix CACHE_TAG_NESTING_DEVTLB polluting shared variables in flush loop

[Baolu Lu]

On 8/4/2026 1:28 PM, Baolu Lu wrote:
> On 8/4/2026 11:16 AM, Li,Rongqing wrote:
>>
>>
>>> From: Guanghui Feng <guanghuifeng@linux.alibaba.com>
>>>
>>> In cache_tag_flush_range(), the CACHE_TAG_NESTING_DEVTLB case 
>>> modifies the
>>> shared local variables 'addr' and 'mask' before falling through to
>>> CACHE_TAG_DEVTLB. This causes all subsequent CACHE_TAG_DEVTLB entries in
>>> the same loop iteration to incorrectly use the full-range flush 
>>> parameters
>>> (addr=0, mask=MAX_AGAW_PFN_WIDTH) instead of the precisely calculated 
>>> PSI
>>> range. This is not the intended behavior, as regular DEVTLB entries 
>>> should always
>>> perform targeted range-based invalidation.
>>>
>>> Fix this by having CACHE_TAG_NESTING_DEVTLB directly call
>>> cache_tag_flush_devtlb_psi() with the full-range constants and break, 
>>> instead of
>>> modifying shared variables and falling through. This ensures
>>> CACHE_TAG_DEVTLB always uses the original calculated addr and mask for
>>> precise range flush.
>>>
>>> Signed-off-by: Guanghui Feng <guanghuifeng@linux.alibaba.com>
>>> Signed-off-by: Guixin Liu <kanie@linux.alibaba.com>
>>> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
>>> ---
>>>   drivers/iommu/intel/cache.c | 5 ++---
>>>   1 file changed, 2 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/drivers/iommu/intel/cache.c b/drivers/iommu/intel/ 
>>> cache.c index
>>> fdc88817709f..26a758b0f501 100644
>>> --- a/drivers/iommu/intel/cache.c
>>> +++ b/drivers/iommu/intel/cache.c
>>> @@ -454,9 +454,8 @@ void cache_tag_flush_range(struct dmar_domain
>>> *domain, unsigned long start,
>>>                * affected by a change in S2. So just flush the entire
>>>                * device cache.
>>>                */
>>> -            addr = 0;
>>> -            mask = MAX_AGAW_PFN_WIDTH;
>>> -            fallthrough;
>>> +            cache_tag_flush_devtlb_psi(domain, tag, 0,
>>> MAX_AGAW_PFN_WIDTH);
>>> +            break;
>>>           case CACHE_TAG_DEVTLB:
>>>               cache_tag_flush_devtlb_psi(domain, tag, addr, mask);
>>>               break;
>>> -- 
>>> 2.43.0
>>
>> This patch introduces a subtle side effect on the tracing logic later 
>> in this function.
>> At the end of cache_tag_flush_range(), 
>> trace_cache_tag_flush_range(tag, start, end, addr, mask) is called to log
>> the flush operation.
>>
>> With this patch: bypassed the assignment and used break, addr and mask 
>> retain their original range values.
>> This causes the tracepoint to log an incorrect, smaller range while 
>> the actual hardware execution was a full-range flush.
> 
> The tracepoint here records what the caller requested: a specific cache-
> invalidation type for a specific range. In this helper, we may widen the
> invalidation range for implementation reasons (as described in the
> comments), but that does not change the caller’s original intent.
> Therefore, this tracepoint should log the caller-requested range.
> 
> If we want to observe the actual invalidation range sent to hardware,
> that is already covered by the qi_submit trace event, which logs the
> real invalidation descriptors submitted by the driver.

I will add the following in the commit message:

"
This change slightly affects trace_cache_tag_flush_range() behavior.
Previously, after addr/mask were overwritten, the tracepoint could
record a full-range flush even when the caller requested a narrower
range.

The tracepoint should reflect caller intent. Although this helper may
widen the actual hardware invalidation range for implementation reasons,
that does not change what the caller requested, so logging the requested
range is the correct behavior.

If the actual invalidation range sent to hardware is needed, it is
already visible via the qi_submit trace event, which records the
invalidation descriptors emitted by the driver.
"

Thanks,
baolu