From: Lu Baolu <baolu.lu@linux.intel.com>
To: iommu@lists.linux.dev, x86@kernel.org,
linux-coco@lists.linux.dev, kvm@vger.kernel.org
Cc: Jason Gunthorpe <jgg@ziepe.ca>, Kevin Tian <kevin.tian@intel.com>,
Kiryl Shutsemau <kas@kernel.org>,
Rick Edgecombe <rick.p.edgecombe@intel.com>,
yilun.xu@linux.intel.com, xiaoyao.li@intel.com,
Chao Gao <chao.gao@intel.com>,
Binbin Wu <binbin.wu@linux.intel.com>,
linux-kernel@vger.kernel.org, Lu Baolu <baolu.lu@linux.intel.com>
Subject: [PATCH v3 6/9] iommu/vt-d: Refuse system suspend while acquired by the TDX module
Date: Thu, 12 Nov 2026 12:30:51 +0800 [thread overview]
Message-ID: <20261112043054.2664857-7-baolu.lu@linux.intel.com> (raw)
In-Reply-To: <20261112043054.2664857-1-baolu.lu@linux.intel.com>
While tdx-host has acquired an IOMMU, the VT-d driver blocks its GCMD
writes. TDX disables S3 by clearing acpi_suspend_lowlevel, but that is
only checked after the syscore suspend callbacks have run, so an S3
request, e.g. "echo mem > /sys/power/state", still runs iommu_suspend()
and then iommu_resume():
suspend_enter()
syscore_suspend()
iommu_suspend() <- runs on the acquired IOMMU
suspend_ops->enter() = acpi_suspend_enter()
if (!acpi_suspend_lowlevel) <- TDX's "S3 disabled" check
return -ENOSYS
syscore_resume()
iommu_resume() <- runs on the acquired IOMMU
Register a PM notifier for each DMAR unit that supports trusted DMA and
refuse PM_SUSPEND_PREPARE with -EBUSY while the unit is acquired. The
suspend then fails before tasks are frozen and before syscore callbacks
run.
A check at PM_SUSPEND_PREPARE alone leaves a window: user space is only
frozen after the notifiers run, so a device connect may still acquire
the unit in between. Close it with a per-unit tdxc_suspending flag, set
when the notifier lets the suspend proceed and cleared at
PM_POST_SUSPEND. dmar_tdxc_acquire() fails with -EBUSY while it is set.
Both the notifier and dmar_tdxc_acquire() hold the unit's tdxc_lock, so
the check and the acquisition can't interleave.
The notifiers are registered at init only for units advertising
ECAP.TDXCS. Such units are never hot-added or hot-removed, so the set
doesn't change afterwards and the notifiers are never unregistered.
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
---
drivers/iommu/intel/iommu.h | 1 +
include/linux/dmar.h | 4 ++-
drivers/iommu/intel/dmar.c | 53 +++++++++++++++++++++++++++++++++++++
drivers/iommu/intel/iommu.c | 1 +
4 files changed, 58 insertions(+), 1 deletion(-)
diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h
index 26135248675d..f7554df8cd2d 100644
--- a/drivers/iommu/intel/iommu.h
+++ b/drivers/iommu/intel/iommu.h
@@ -1363,6 +1363,7 @@ extern int rwbf_quirk;
int iommu_calculate_agaw(struct intel_iommu *iommu);
int iommu_calculate_max_sagaw(struct intel_iommu *iommu);
int ecmd_submit_sync(struct intel_iommu *iommu, u8 ecmd, u64 oa, u64 ob);
+void dmar_tdxc_pm_init(void);
static inline bool ecmd_has_pmu_essential(struct intel_iommu *iommu)
{
diff --git a/include/linux/dmar.h b/include/linux/dmar.h
index a39ca13f13dc..8b407bc6d179 100644
--- a/include/linux/dmar.h
+++ b/include/linux/dmar.h
@@ -47,7 +47,9 @@ struct dmar_drhd_unit {
u8 ignored:1; /* ignore drhd */
u8 include_all:1;
u8 gfx_dedicated:1; /* graphic dedicated */
- struct mutex tdxc_lock; /* serializes acquire/release */
+ struct mutex tdxc_lock; /* serializes acquire/release/suspend */
+ struct notifier_block tdxc_pm_nb;
+ bool tdxc_suspending; /* protected by tdxc_lock */
struct intel_iommu *iommu;
};
diff --git a/drivers/iommu/intel/dmar.c b/drivers/iommu/intel/dmar.c
index 91826705a06b..ad6b22601446 100644
--- a/drivers/iommu/intel/dmar.c
+++ b/drivers/iommu/intel/dmar.c
@@ -28,6 +28,7 @@
#include <linux/iommu.h>
#include <linux/numa.h>
#include <linux/limits.h>
+#include <linux/suspend.h>
#include <asm/irq_remapping.h>
#include "iommu.h"
@@ -2683,6 +2684,9 @@ int dmar_tdxc_acquire(struct pci_dev *pdev,
return -EINVAL;
guard(mutex)(&drhd->tdxc_lock);
+ if (drhd->tdxc_suspending)
+ return -EBUSY;
+
/* Still set if TDH.IOMMU.CLEAR failed and tdx-host was reloaded. */
if (iommu->tdxc_acquired)
return -EBUSY;
@@ -2737,3 +2741,52 @@ void dmar_tdxc_release(struct pci_dev *pdev,
dmar_tdxc_unreserve_did(drhd->iommu);
}
EXPORT_SYMBOL_FOR_MODULES(dmar_tdxc_release, "tdx-host");
+
+/*
+ * TDX only fails S3 after syscore suspend/resume have run, and those
+ * would hit the blocked GCMD paths. Refuse the suspend up front instead.
+ */
+static int dmar_tdxc_pm_notify(struct notifier_block *nb,
+ unsigned long action, void *data)
+{
+ struct dmar_drhd_unit *drhd =
+ container_of(nb, struct dmar_drhd_unit, tdxc_pm_nb);
+
+ guard(mutex)(&drhd->tdxc_lock);
+
+ switch (action) {
+ case PM_SUSPEND_PREPARE:
+ if (drhd->iommu->tdxc_acquired)
+ return notifier_from_errno(-EBUSY);
+ drhd->tdxc_suspending = true;
+ break;
+ case PM_POST_SUSPEND:
+ drhd->tdxc_suspending = false;
+ break;
+ }
+
+ return NOTIFY_DONE;
+}
+
+void __init dmar_tdxc_pm_init(void)
+{
+ struct dmar_drhd_unit *drhd;
+ struct intel_iommu *iommu;
+
+ down_read(&dmar_global_lock);
+ for_each_active_iommu(iommu, drhd) {
+ if (!ecap_tdxcs(iommu->ecap))
+ continue;
+
+ drhd->tdxc_pm_nb.notifier_call = dmar_tdxc_pm_notify;
+ /*
+ * CPU hotplug takes dmar_global_lock under the PM notifier chain
+ * lock. Drop it to avoid a lock order splat; TDXCS units are never
+ * hot-removed, so @drhd stays valid.
+ */
+ up_read(&dmar_global_lock);
+ register_pm_notifier(&drhd->tdxc_pm_nb);
+ down_read(&dmar_global_lock);
+ }
+ up_read(&dmar_global_lock);
+}
diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index 69e8c2c8680a..d8b9610ae791 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -2041,6 +2041,7 @@ static struct syscore iommu_syscore = {
static void __init init_iommu_pm_ops(void)
{
register_syscore(&iommu_syscore);
+ dmar_tdxc_pm_init();
}
#else
--
2.43.0
next prev parent reply other threads:[~2026-11-12 4:44 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-12 4:30 [PATCH v3 0/9] coco/tdx-host: Introduce trusted DMA initialization support Lu Baolu
2026-11-12 4:30 ` [PATCH v3 1/9] x86/virt/tdx: Add SEAMCALL wrappers for IOMMU setup/clear Lu Baolu
2026-11-12 4:30 ` [PATCH v3 2/9] x86/virt/tdx: Read global metadata for trusted IOMMU Lu Baolu
2026-11-12 4:30 ` [PATCH v3 3/9] x86/virt/tdx: Export Dynamic PAMT helpers to tdx-host Lu Baolu
2026-11-12 4:30 ` [PATCH v3 4/9] iommu/vt-d: Add interface for tdx-host when enabling trusted DMA Lu Baolu
2026-11-12 4:30 ` [PATCH v3 5/9] iommu/vt-d: Block GCMD and ECMD while acquired by the TDX module Lu Baolu
2026-11-12 4:30 ` Lu Baolu [this message]
2026-11-12 4:30 ` [PATCH v3 7/9] coco/tdx-host: Add skeleton to support PCI link encryption Lu Baolu
2026-11-18 7:16 ` Chao Gao
2026-11-12 4:30 ` [PATCH v3 8/9] coco/tdx-host: Rename tdx-host.c to main.c Lu Baolu
2026-11-12 4:30 ` [PATCH v3 9/9] coco/tdx-host: Enable trusted DMA on IOMMUs on device connect Lu Baolu
2026-11-18 8:11 ` Chao Gao
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261112043054.2664857-7-baolu.lu@linux.intel.com \
--to=baolu.lu@linux.intel.com \
--cc=binbin.wu@linux.intel.com \
--cc=chao.gao@intel.com \
--cc=iommu@lists.linux.dev \
--cc=jgg@ziepe.ca \
--cc=kas@kernel.org \
--cc=kevin.tian@intel.com \
--cc=kvm@vger.kernel.org \
--cc=linux-coco@lists.linux.dev \
--cc=linux-kernel@vger.kernel.org \
--cc=rick.p.edgecombe@intel.com \
--cc=x86@kernel.org \
--cc=xiaoyao.li@intel.com \
--cc=yilun.xu@linux.intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.