* [PATCH v2 06/19] PCI/TSM: Add Device Security (TVM Guest) LOCK operation support
From: Dan Williams @ 2026-03-03 0:01 UTC (permalink / raw)
To: linux-coco, linux-pci
Cc: gregkh, aik, aneesh.kumar, yilun.xu, bhelgaas, alistair23, lukas,
jgg
In-Reply-To: <20260303000207.1836586-1-dan.j.williams@intel.com>
PCIe Trusted Execution Environment Device Interface Security Protocol
(TDISP) has two distinct sets of operations. The first, already enabled in
driver/pci/tsm.c, enables the VMM to authenticate the physical function
(PCIe Component Measurement and Authentication (CMA)), establish a secure
message passing session (DMTF SPDM), and establish physical link security
(PCIe Integrity and Data Encryption (IDE)). The second set of operations
lets a VM manage the security state of assigned devices (TEE Device
Interfaces (TDIs)). Implement the LOCK/UNLOCK operations in pci_tsm_ops, to
be followed with an ACCEPT operation.
- lock(): Transition the device to the TDISP state. In this mode
the device is responsible for validating that it is in a secure
configuration and will transition to the TDISP ERROR state if those
settings are modified. Device Security Manager (DSM) and the TEE
Security Manager (TSM) enforce that the device is not permitted to issue
T=1 traffic in this mode.
- unlock(): From the RUN state the only other TDISP states that can be
moved to are ERROR or UNLOCKED. Voluntarily move the device to the
UNLOCKED state.
Co-developed-by: Xu Yilun <yilun.xu@linux.intel.com>
Signed-off-by: Xu Yilun <yilun.xu@linux.intel.com>
Co-developed-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
Signed-off-by: Aneesh Kumar K.V (Arm) <aneesh.kumar@kernel.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
Documentation/ABI/testing/sysfs-bus-pci | 34 +++-
Documentation/ABI/testing/sysfs-class-tsm | 19 ++
Documentation/driver-api/pci/tsm.rst | 44 +++++
include/linux/device.h | 1 +
include/linux/pci-tsm.h | 21 ++-
drivers/pci/tsm.c | 213 +++++++++++++++++++++-
drivers/virt/coco/tsm-core.c | 41 +++++
7 files changed, 363 insertions(+), 10 deletions(-)
diff --git a/Documentation/ABI/testing/sysfs-bus-pci b/Documentation/ABI/testing/sysfs-bus-pci
index b767db2c52cb..1ed77b9402a6 100644
--- a/Documentation/ABI/testing/sysfs-bus-pci
+++ b/Documentation/ABI/testing/sysfs-bus-pci
@@ -647,13 +647,16 @@ Description:
Encryption) establishment. Reads from this attribute return the
name of the connected TSM or the empty string if not
connected. A TSM device signals its readiness to accept PCI
- connection via a KOBJ_CHANGE event.
+ connection via a KOBJ_CHANGE event. This is a "link" TSM
+ attribute, see Documentation/ABI/testing/sysfs-class-tsm.
What: /sys/bus/pci/devices/.../tsm/disconnect
Contact: linux-coco@lists.linux.dev
Description:
(WO) Write the name of the TSM device that was specified
- to 'connect' to teardown the connection.
+ to 'connect' to teardown the connection. This is a
+ "link" TSM attribute, see
+ Documentation/ABI/testing/sysfs-class-tsm.
What: /sys/bus/pci/devices/.../tsm/dsm
Contact: linux-coco@lists.linux.dev
@@ -702,3 +705,30 @@ Description:
When present and the tsm/ attribute directory is present, the
authenticated attribute is an alias for the device 'connect'
state. See the 'tsm/connect' attribute for more details.
+ This is a "link" TSM attribute, see
+ Documentation/ABI/testing/sysfs-class-tsm.
+
+What: /sys/bus/pci/devices/.../tsm/lock
+Contact: linux-coco@lists.linux.dev
+Description:
+ (RW) Write the name of a TSM (TEE Security Manager) device
+ instance to this file to request that the platform transition
+ the PCIe device configuration to the TDISP LOCKED state. This
+ puts the device in a state where security sensitive
+ configuration setting can not be changed without transitioning
+ the device the PCIe TDISP ERROR state. Reads from this attribute
+ return the name of the TSM device instance that has arranged for
+ the device to be locked, or the empty string if not locked. A
+ TSM class device signals its readiness for lock requests via a
+ KOBJ_CHANGE event. Writes fail with EBUSY if this device is
+ bound to a driver. This is a "devsec" TSM attribute, see
+ Documentation/ABI/testing/sysfs-class-tsm. See
+ Documentation/driver-api/pci/tsm.rst for more details.
+
+What: /sys/bus/pci/devices/.../tsm/unlock
+Contact: linux-coco@lists.linux.dev
+Description:
+ (WO) Write the name of the TSM device that was specified to
+ 'lock' to teardown the connection. Writes fail with EBUSY if
+ this device is bound to a driver. This is a "devsec" TSM
+ attribute, see Documentation/ABI/testing/sysfs-class-tsm.
diff --git a/Documentation/ABI/testing/sysfs-class-tsm b/Documentation/ABI/testing/sysfs-class-tsm
index 1ddb8f357961..76704501f082 100644
--- a/Documentation/ABI/testing/sysfs-class-tsm
+++ b/Documentation/ABI/testing/sysfs-class-tsm
@@ -20,3 +20,22 @@ Description:
Documentation/ABI/testing/sysfs-devices-pci-host-bridge for the
description of the pciDDDD:BB/streamH.R.E symlink and the
pciDDDD:BB/available_secure_streams attribute.
+
+What: /sys/class/tsm/tsmN/pci_mode
+Contact: linux-coco@lists.linux.dev
+Description:
+ (RO) A TSM with PCIe TDISP capability can be in one of two
+ modes.
+
+ "link": typically for a hypervisor (VMM) to authenticate,
+ establish a secure session, and setup link
+ encryption.
+
+ "devsec": typically for a confidential guest (TVM) to
+ transition assigned devices through the TDISP
+ state machine UNLOCKED->LOCKED->RUN.
+
+ See the "tsm/" entries in
+ Documentation/ABI/testing/sysfs-bus-pci for the available PCI
+ device attributes when a TSM with the given "pci_mode" is
+ registered.
diff --git a/Documentation/driver-api/pci/tsm.rst b/Documentation/driver-api/pci/tsm.rst
index 232b92bec93f..e9b7ac70b404 100644
--- a/Documentation/driver-api/pci/tsm.rst
+++ b/Documentation/driver-api/pci/tsm.rst
@@ -5,6 +5,50 @@
PCI Trusted Execution Environment Security Manager (TSM)
========================================================
+Overview
+========
+
+A "TSM", as detailed by PCIe r7.0 section 11 "TEE Device Interface
+Security Protocol (TDISP)", is an entity within the platform's Trusted
+Computing Base (TCB) that enforces security policies on the host. It
+serves to mitigate a threat model where devices may be under the control
+of an adversary. The adversarial threats are:
+
+- Identity: Device may be mimicking a legitimate device identity / firmware
+- Physical: link may be under observation, or control (reorder / drop data)
+- Virtual: Device MMIO presented to a guest may not actually map the
+ device, device DMA may be redirected.
+
+In Linux a "tsm" is a broader concept. It is a class device interface to
+mitigate one or more of the above threats. A "tsm driver" registers a
+tsm device that publishes either the 'tsm/connect' or
+'tsm/{lock,accept}' set of attributes for the PCIe device. The typical
+expectation is that 'tsm/{lock,accept}' is published by a guest "tsm
+driver" to mitigate "Virtual" threats. The 'tsm/connect' interface is
+published by a host "tsm driver" to mitigate "Identity" and/or
+"Physical" threats.
+
+Device Interface LOCK
+=====================
+The lock operation facilitated by tsm/lock (see
+Documentation/ABI/testing/sysfs-bus-pci) places the device in a mode
+where any security sensitive changes to the device configuration results
+in the device transitioning to the ERROR state. The device presents
+signed evidence of its LOCK state to the kernel through the tsm driver.
+The relying party is responsible for verifying not only the evidence but
+that the device is trusted to maintain those attested values while
+locked. Accepting the locked configuration also asserts that device is
+trusted to cease TCB interactions (send T=1 DMA / accept T=1 MMIO TLPs)
+when it is next unlocked by STOP. The TSM is responsible for enforcing
+that the device is not unlocked within the interval between evidence
+collection and acceptance, by correlating the evidence from LOCK to the
+subsequent RUN request.
+
+While the PCIe specification allows for the device to operate outside
+the TCB when locked, depending on the TSM architecture implementation,
+T=0 DMA from the device may be blocked until the device is next
+unlocked.
+
Subsystem Interfaces
====================
diff --git a/include/linux/device.h b/include/linux/device.h
index 4470365d772b..f131623f39d9 100644
--- a/include/linux/device.h
+++ b/include/linux/device.h
@@ -911,6 +911,7 @@ static inline void device_unlock(struct device *dev)
}
DEFINE_GUARD(device, struct device *, device_lock(_T), device_unlock(_T))
+DEFINE_GUARD_COND(device, _intr, device_lock_interruptible(_T), _RET == 0)
static inline void device_lock_assert(struct device *dev)
{
diff --git a/include/linux/pci-tsm.h b/include/linux/pci-tsm.h
index a6435aba03f9..2a896b83bff9 100644
--- a/include/linux/pci-tsm.h
+++ b/include/linux/pci-tsm.h
@@ -68,7 +68,8 @@ struct pci_tsm_ops {
* @unlock: destroy TSM context and return device to UNLOCKED state
*
* Context: @lock and @unlock run under pci_tsm_rwsem held for write to
- * sync with TSM unregistration and each other
+ * sync with TSM unregistration and each other. All operations run under
+ * the device lock for mutual exclusion with driver attach and detach.
*/
struct_group_tagged(pci_tsm_devsec_ops, devsec_ops,
struct pci_tsm *(*lock)(struct tsm_dev *tsm_dev,
@@ -106,6 +107,13 @@ struct pci_tdi {
* sub-function (SR-IOV virtual function, or non-function0
* multifunction-device), or a downstream endpoint (PCIe upstream switch-port as
* DSM).
+ *
+ * For devsec operations it serves to indicate that the function / TDI has been
+ * locked to a given TSM.
+ *
+ * The common expectation is that there is only ever one TSM, but this is not
+ * enforced. The implementation only enforces that a device can be "connected"
+ * to a TSM instance or "locked" to a different TSM.
*/
struct pci_tsm {
struct pci_dev *pdev;
@@ -126,6 +134,14 @@ struct pci_tsm_pf0 {
struct pci_doe_mb *doe_mb;
};
+/**
+ * struct pci_tsm_devsec - context for tracking private/accepted PCI resources
+ * @base_tsm: generic core "tsm" context
+ */
+struct pci_tsm_devsec {
+ struct pci_tsm base_tsm;
+};
+
/* physical function0 and capable of 'connect' */
static inline bool is_pci_tsm_pf0(struct pci_dev *pdev)
{
@@ -206,6 +222,8 @@ int pci_tsm_link_constructor(struct pci_dev *pdev, struct pci_tsm *tsm,
struct tsm_dev *tsm_dev);
int pci_tsm_pf0_constructor(struct pci_dev *pdev, struct pci_tsm_pf0 *tsm,
struct tsm_dev *tsm_dev);
+int pci_tsm_devsec_constructor(struct pci_dev *pdev, struct pci_tsm_devsec *tsm,
+ struct tsm_dev *tsm_dev);
void pci_tsm_pf0_destructor(struct pci_tsm_pf0 *tsm);
int pci_tsm_doe_transfer(struct pci_dev *pdev, u8 type, const void *req,
size_t req_sz, void *resp, size_t resp_sz);
@@ -216,6 +234,7 @@ void pci_tsm_tdi_constructor(struct pci_dev *pdev, struct pci_tdi *tdi,
ssize_t pci_tsm_guest_req(struct pci_dev *pdev, enum pci_tsm_req_scope scope,
sockptr_t req_in, size_t in_len, sockptr_t req_out,
size_t out_len, u64 *tsm_code);
+struct pci_tsm_devsec *to_pci_tsm_devsec(struct pci_tsm *tsm);
#else
static inline int pci_tsm_register(struct tsm_dev *tsm_dev)
{
diff --git a/drivers/pci/tsm.c b/drivers/pci/tsm.c
index 5fdcd7f2e820..259e75092618 100644
--- a/drivers/pci/tsm.c
+++ b/drivers/pci/tsm.c
@@ -9,6 +9,7 @@
#define dev_fmt(fmt) "PCI/TSM: " fmt
#include <linux/bitfield.h>
+#include <linux/ioport.h>
#include <linux/pci.h>
#include <linux/pci-doe.h>
#include <linux/pci-tsm.h>
@@ -63,6 +64,26 @@ static struct pci_tsm_pf0 *to_pci_tsm_pf0(struct pci_tsm *tsm)
return container_of(pf0->tsm, struct pci_tsm_pf0, base_tsm);
}
+static inline bool is_devsec(struct pci_dev *pdev)
+{
+ return pdev->tsm && pdev->tsm->dsm_dev == NULL &&
+ pdev->tsm->tdi == NULL;
+}
+
+/* 'struct pci_tsm_devsec' wraps 'struct pci_tsm' when ->tdi == ->dsm == NULL */
+struct pci_tsm_devsec *to_pci_tsm_devsec(struct pci_tsm *tsm)
+{
+ struct pci_dev *pdev = tsm->pdev;
+
+ if (!is_devsec(pdev) || !has_tee(pdev)) {
+ pci_WARN_ONCE(pdev, 1, "invalid context object\n");
+ return NULL;
+ }
+
+ return container_of(tsm, struct pci_tsm_devsec, base_tsm);
+}
+EXPORT_SYMBOL_GPL(to_pci_tsm_devsec);
+
static void tsm_remove(struct pci_tsm *tsm)
{
struct pci_dev *pdev;
@@ -536,6 +557,125 @@ static ssize_t dsm_show(struct device *dev, struct device_attribute *attr,
}
static DEVICE_ATTR_RO(dsm);
+/**
+ * pci_tsm_unlock() - Transition TDI from LOCKED/RUN to UNLOCKED
+ * @pdev: TDI device to unlock
+ *
+ * Returns void, requires all callers to have satisfied dependencies like making
+ * sure the device is locked and detached from its driver.
+ */
+static void pci_tsm_unlock(struct pci_dev *pdev)
+{
+ lockdep_assert_held_write(&pci_tsm_rwsem);
+ device_lock_assert(&pdev->dev);
+
+ if (dev_WARN_ONCE(&pdev->dev, pdev->dev.driver,
+ "unlock attempted on driver attached device\n"))
+ return;
+
+ device_cc_reject(&pdev->dev);
+ to_pci_tsm_ops(pdev->tsm)->unlock(pdev->tsm);
+ pdev->tsm = NULL;
+}
+
+static int pci_tsm_lock(struct pci_dev *pdev, struct tsm_dev *tsm_dev)
+{
+ const struct pci_tsm_ops *ops = tsm_dev->pci_ops;
+ struct pci_tsm *tsm;
+ int rc;
+
+ ACQUIRE(device_intr, lock)(&pdev->dev);
+ if ((rc = ACQUIRE_ERR(device_intr, &lock)))
+ return rc;
+
+ if (pdev->dev.driver)
+ return -EBUSY;
+
+ tsm = ops->lock(tsm_dev, pdev);
+ if (IS_ERR(tsm))
+ return PTR_ERR(tsm);
+
+ pdev->tsm = tsm;
+ return 0;
+}
+
+static ssize_t lock_store(struct device *dev, struct device_attribute *attr,
+ const char *buf, size_t len)
+{
+ struct pci_dev *pdev = to_pci_dev(dev);
+ int rc, id;
+
+ rc = sscanf(buf, "tsm%d\n", &id);
+ if (rc != 1)
+ return -EINVAL;
+
+ ACQUIRE(rwsem_write_kill, lock)(&pci_tsm_rwsem);
+ if ((rc = ACQUIRE_ERR(rwsem_write_kill, &lock)))
+ return rc;
+
+ if (pdev->tsm)
+ return -EBUSY;
+
+ struct tsm_dev *tsm_dev __free(put_tsm_dev) = find_tsm_dev(id);
+ if (!is_devsec_tsm(tsm_dev))
+ return -ENXIO;
+
+ rc = pci_tsm_lock(pdev, tsm_dev);
+ if (rc)
+ return rc;
+ return len;
+}
+
+static ssize_t lock_show(struct device *dev, struct device_attribute *attr,
+ char *buf)
+{
+ struct pci_dev *pdev = to_pci_dev(dev);
+ struct tsm_dev *tsm_dev;
+ int rc;
+
+ ACQUIRE(rwsem_read_intr, lock)(&pci_tsm_rwsem);
+ if ((rc = ACQUIRE_ERR(rwsem_read_intr, &lock)))
+ return rc;
+
+ if (!pdev->tsm)
+ return sysfs_emit(buf, "\n");
+
+ tsm_dev = pdev->tsm->tsm_dev;
+ return sysfs_emit(buf, "%s\n", dev_name(&tsm_dev->dev));
+}
+static DEVICE_ATTR_RW(lock);
+
+static ssize_t unlock_store(struct device *dev, struct device_attribute *attr,
+ const char *buf, size_t len)
+{
+ struct pci_dev *pdev = to_pci_dev(dev);
+ struct tsm_dev *tsm_dev;
+ int rc;
+
+ ACQUIRE(rwsem_write_kill, lock)(&pci_tsm_rwsem);
+ if ((rc = ACQUIRE_ERR(rwsem_write_kill, &lock)))
+ return rc;
+
+ if (!pdev->tsm)
+ return -EINVAL;
+
+ tsm_dev = pdev->tsm->tsm_dev;
+ if (!sysfs_streq(buf, dev_name(&tsm_dev->dev)))
+ return -EINVAL;
+
+ ACQUIRE(device_intr, dev_lock)(&pdev->dev);
+ if ((rc = ACQUIRE_ERR(device_intr, &dev_lock)))
+ return rc;
+
+ if (pdev->dev.driver)
+ return -EBUSY;
+
+ pci_tsm_unlock(pdev);
+
+ return len;
+}
+static DEVICE_ATTR_WO(unlock);
+
/* The 'authenticated' attribute is exclusive to the presence of a 'link' TSM */
static bool pci_tsm_link_group_visible(struct kobject *kobj)
{
@@ -561,6 +701,13 @@ static bool pci_tsm_link_group_visible(struct kobject *kobj)
}
DEFINE_SIMPLE_SYSFS_GROUP_VISIBLE(pci_tsm_link);
+static bool pci_tsm_devsec_group_visible(struct kobject *kobj)
+{
+ struct pci_dev *pdev = to_pci_dev(kobj_to_dev(kobj));
+
+ return pci_tsm_devsec_count && has_tee(pdev);
+}
+
/*
* 'link' and 'devsec' TSMs share the same 'tsm/' sysfs group, so the TSM type
* specific attributes need individual visibility checks.
@@ -592,12 +739,19 @@ static umode_t pci_tsm_attr_visible(struct kobject *kobj,
}
}
+ if (pci_tsm_devsec_group_visible(kobj)) {
+ if (attr == &dev_attr_lock.attr ||
+ attr == &dev_attr_unlock.attr)
+ return attr->mode;
+ }
+
return 0;
}
static bool pci_tsm_group_visible(struct kobject *kobj)
{
- return pci_tsm_link_group_visible(kobj);
+ return pci_tsm_link_group_visible(kobj) ||
+ pci_tsm_devsec_group_visible(kobj);
}
DEFINE_SYSFS_GROUP_VISIBLE(pci_tsm);
@@ -606,6 +760,8 @@ static struct attribute *pci_tsm_attrs[] = {
&dev_attr_disconnect.attr,
&dev_attr_bound.attr,
&dev_attr_dsm.attr,
+ &dev_attr_lock.attr,
+ &dev_attr_unlock.attr,
NULL
};
@@ -734,6 +890,29 @@ int pci_tsm_link_constructor(struct pci_dev *pdev, struct pci_tsm *tsm,
}
EXPORT_SYMBOL_GPL(pci_tsm_link_constructor);
+/**
+ * pci_tsm_devsec_constructor() - devsec TSM context initialization
+ * @pdev: The PCI device
+ * @tsm: context to initialize
+ * @tsm_dev: Platform TEE Security Manager, initiator of security operations
+ */
+int pci_tsm_devsec_constructor(struct pci_dev *pdev, struct pci_tsm_devsec *tsm,
+ struct tsm_dev *tsm_dev)
+{
+ struct pci_tsm *pci_tsm = &tsm->base_tsm;
+
+ if (!is_devsec_tsm(tsm_dev))
+ return -EINVAL;
+
+ pci_tsm->dsm_dev = NULL;
+ pci_tsm->tdi = NULL;
+ pci_tsm->pdev = pdev;
+ pci_tsm->tsm_dev = tsm_dev;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(pci_tsm_devsec_constructor);
+
/**
* pci_tsm_pf0_constructor() - common 'struct pci_tsm_pf0' (DSM) initialization
* @pdev: Physical Function 0 PCI device (as indicated by is_pci_tsm_pf0())
@@ -761,6 +940,13 @@ void pci_tsm_pf0_destructor(struct pci_tsm_pf0 *pf0_tsm)
}
EXPORT_SYMBOL_GPL(pci_tsm_pf0_destructor);
+static void devsec_sysfs_enable(struct pci_dev *pdev)
+{
+ pci_dbg(pdev, "TEE I/O Device capability detected (TDISP)\n");
+
+ sysfs_update_group(&pdev->dev.kobj, &pci_tsm_attr_group);
+}
+
int pci_tsm_register(struct tsm_dev *tsm_dev)
{
struct pci_dev *pdev = NULL;
@@ -782,8 +968,10 @@ int pci_tsm_register(struct tsm_dev *tsm_dev)
for_each_pci_dev(pdev)
if (is_pci_tsm_pf0(pdev))
link_sysfs_enable(pdev);
- } else if (is_devsec_tsm(tsm_dev)) {
- pci_tsm_devsec_count++;
+ } else if (is_devsec_tsm(tsm_dev) && pci_tsm_devsec_count++ == 0) {
+ for_each_pci_dev(pdev)
+ if (has_tee(pdev))
+ devsec_sysfs_enable(pdev);
}
return 0;
@@ -818,6 +1006,9 @@ static void __pci_tsm_destroy(struct pci_dev *pdev, struct tsm_dev *tsm_dev)
if (is_link_tsm(tsm_dev) && is_pci_tsm_pf0(pdev) && !pci_tsm_link_count)
link_sysfs_disable(pdev);
+ if (is_devsec_tsm(tsm_dev) && !pci_tsm_devsec_count)
+ sysfs_update_group(&pdev->dev.kobj, &pci_tsm_attr_group);
+
/* Nothing else to do if this device never attached to the departing TSM */
if (!tsm)
return;
@@ -828,10 +1019,18 @@ static void __pci_tsm_destroy(struct pci_dev *pdev, struct tsm_dev *tsm_dev)
else if (tsm_dev != tsm->tsm_dev)
return;
- if (is_link_tsm(tsm_dev) && is_pci_tsm_pf0(pdev))
- pci_tsm_disconnect(pdev);
- else
- pci_tsm_fn_exit(pdev);
+ /* Disconnect DSMs, unlock assigned TDIs, or cleanup DSM subfunctions */
+ if (is_link_tsm(tsm_dev)) {
+ if (is_pci_tsm_pf0(pdev))
+ pci_tsm_disconnect(pdev);
+ else
+ pci_tsm_fn_exit(pdev);
+ }
+
+ if (is_devsec_tsm(tsm_dev) && has_tee(pdev)) {
+ guard(device)(&pdev->dev);
+ pci_tsm_unlock(pdev);
+ }
}
void pci_tsm_destroy(struct pci_dev *pdev)
diff --git a/drivers/virt/coco/tsm-core.c b/drivers/virt/coco/tsm-core.c
index 3c99c38cfaa5..231462d60379 100644
--- a/drivers/virt/coco/tsm-core.c
+++ b/drivers/virt/coco/tsm-core.c
@@ -54,6 +54,45 @@ static struct tsm_dev *alloc_tsm_dev(struct device *parent)
return no_free_ptr(tsm_dev);
}
+static ssize_t pci_mode_show(struct device *dev, struct device_attribute *attr,
+ char *buf)
+{
+ struct tsm_dev *tsm_dev = container_of(dev, struct tsm_dev, dev);
+ const struct pci_tsm_ops *ops = tsm_dev->pci_ops;
+
+ if (ops->connect)
+ return sysfs_emit(buf, "link\n");
+ if (ops->lock)
+ return sysfs_emit(buf, "devsec\n");
+ return sysfs_emit(buf, "none\n");
+}
+static DEVICE_ATTR_RO(pci_mode);
+
+static umode_t tsm_pci_visible(struct kobject *kobj, struct attribute *attr, int n)
+{
+ struct device *dev = container_of(kobj, struct device, kobj);
+ struct tsm_dev *tsm_dev = container_of(dev, struct tsm_dev, dev);
+
+ if (tsm_dev->pci_ops)
+ return attr->mode;
+ return 0;
+}
+
+static struct attribute *tsm_pci_attrs[] = {
+ &dev_attr_pci_mode.attr,
+ NULL
+};
+
+static const struct attribute_group tsm_pci_group = {
+ .attrs = tsm_pci_attrs,
+ .is_visible = tsm_pci_visible,
+};
+
+static const struct attribute_group *tsm_pci_groups[] = {
+ &tsm_pci_group,
+ NULL
+};
+
static struct tsm_dev *tsm_register_pci_or_reset(struct tsm_dev *tsm_dev,
struct pci_tsm_ops *pci_ops)
{
@@ -70,6 +109,7 @@ static struct tsm_dev *tsm_register_pci_or_reset(struct tsm_dev *tsm_dev,
device_unregister(&tsm_dev->dev);
return ERR_PTR(rc);
}
+ sysfs_update_group(&tsm_dev->dev.kobj, &tsm_pci_group);
/* Notify TSM userspace that PCI/TSM operations are now possible */
kobject_uevent(&tsm_dev->dev.kobj, KOBJ_CHANGE);
@@ -214,6 +254,7 @@ static int __init tsm_init(void)
if (IS_ERR(tsm_class))
return PTR_ERR(tsm_class);
+ tsm_class->dev_groups = tsm_pci_groups;
tsm_class->dev_release = tsm_release;
return 0;
}
--
2.52.0
^ permalink raw reply related
* [PATCH v2 04/19] modules: Document the global async_probe parameter
From: Dan Williams @ 2026-03-03 0:01 UTC (permalink / raw)
To: linux-coco, linux-pci
Cc: gregkh, aik, aneesh.kumar, yilun.xu, bhelgaas, alistair23, lukas,
jgg, Saravana Kannan, Luis Chamberlain
In-Reply-To: <20260303000207.1836586-1-dan.j.williams@intel.com>
In preparation for adding another /sys/module/module/parameters entry,
document the existing async_probe parameter.
Cc: Saravana Kannan <saravanak@google.com>
Cc: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
Documentation/ABI/stable/sysfs-module | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/Documentation/ABI/stable/sysfs-module b/Documentation/ABI/stable/sysfs-module
index 41b1f16e8795..397c5c850894 100644
--- a/Documentation/ABI/stable/sysfs-module
+++ b/Documentation/ABI/stable/sysfs-module
@@ -45,3 +45,13 @@ Date: Jun 2005
Description:
If the module source has MODULE_VERSION, this file will contain
the version of the source code.
+
+What: /sys/module/module/parameters/async_probe
+Description:
+ (RW) Emits "1" if drivers from loadable modules attempt async
+ probing by default. Emits "0" if drivers from loadable modules
+ attempt synchronous probing by default. This value is overridden
+ (in priority order) by: the module's built-in "PROBE_FORCE_*"
+ requests, the "driver_async_probe=..." kernel command line, the
+ "async_probe" module option, then this default. Write a valid
+ boolean value to toggle this policy.
--
2.52.0
^ permalink raw reply related
* [PATCH v2 05/19] device core: Autoprobe considered harmful?
From: Dan Williams @ 2026-03-03 0:01 UTC (permalink / raw)
To: linux-coco, linux-pci
Cc: gregkh, aik, aneesh.kumar, yilun.xu, bhelgaas, alistair23, lukas,
jgg, Christoph Hellwig, Marek Szyprowski, Robin Murphy,
Roman Kisel, Samuel Ortiz, Rafael J. Wysocki, Danilo Krummrich
In-Reply-To: <20260303000207.1836586-1-dan.j.williams@intel.com>
The threat model of PCI Trusted Execution Environment Device Interface
Security Protocol (TDISP), is that an adversary may be impersonating the
device's identity, redirecting the device's MMIO interface, and/or
snooping/manipulating the physical link. Outside of PCI TDISP, PCI ATS
(that allows IOMMU bypass) comes to mind as another threat vector that
warrants additional device verification beyond whether ACPI enumerates the
device as "internal" [1].
The process of verifying a device ranges from the traditional default
"accept everything" to gathering signed evidence from a locked device,
shipping it to a relying party and acting on that disposition. That policy
belongs in userspace. A natural way for userspace to get a control point
for verifying a device before use is when the driver for the device comes
from a module.
For deployments that are concerned about adversarial devices, introduce a
mechanism to disable autoprobe. When a driver originates from a module,
consult that driver's autoprobe policy at initial device or driver attach.
Note that with TDISP, unaccepted devices do not have access to private
memory (so called "T=0" mode). However, a deployment may still not want to
operate more than a handful of boot devices until confirming the system
device topology with a verifier.
Yes, this is a security vs convenience tradeoff. Yes, devices with
non-modular drivers are out of scope. Yes, there are known regression cases
for subsystems where device objects are expected to auto-attach outside of
fatal probe failure. For navigating regressions, a per-module "autoprobe"
option is included to allow fine grained policy.
Cc: Christoph Hellwig <hch@lst.de>
Cc: Jason Gunthorpe <jgg@nvidia.com>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Robin Murphy <robin.murphy@arm.com>
Cc: Roman Kisel <romank@linux.microsoft.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Samuel Ortiz <sameo@rivosinc.com>
Cc: Alexey Kardashevskiy <aik@amd.com>
Cc: Xu Yilun <yilun.xu@linux.intel.com>
Cc: "Aneesh Kumar K.V" <aneesh.kumar@kernel.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: "Rafael J. Wysocki" <rafael@kernel.org>
Cc: Danilo Krummrich <dakr@kernel.org>
Link: http://lore.kernel.org/6971b9406d069_1d33100df@dwillia2-mobl4.notmuch [1]
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
drivers/base/Kconfig | 24 ++++++++++++++++++++++++
Documentation/ABI/stable/sysfs-module | 10 ++++++++++
drivers/base/base.h | 1 +
include/linux/module.h | 14 ++++++++++++++
drivers/base/bus.c | 7 ++++++-
drivers/base/dd.c | 26 +++++++++++++++++++++++---
kernel/module/main.c | 13 +++++++++++++
7 files changed, 91 insertions(+), 4 deletions(-)
diff --git a/drivers/base/Kconfig b/drivers/base/Kconfig
index d4743bf978ec..7c1da5df9745 100644
--- a/drivers/base/Kconfig
+++ b/drivers/base/Kconfig
@@ -253,4 +253,28 @@ config CONFIDENTIAL_DEVICES
depends on ARCH_HAS_CC_PLATFORM
bool
+config MODULES_AUTOPROBE
+ bool "Automatic probe of drivers from modules"
+ default y
+ help
+ Say Y for the typical and traditional Linux behavior of automatically
+ attaching devices to drivers when a module is loaded.
+
+ Say N to opt into a threat model where userspace verification of a
+ device is required before driver attach. This includes Confidential
+ Computing use cases where the device needs to have its configuration
+ locked and verified by a relying party. It also includes use cases
+ like leaving devices with Address Translation (IOMMU protection
+ bypass) capability disabled until userspace attests the device and
+ binds a driver.
+
+ This default value can be overridden by the "autoprobe" module option.
+ Note that some subsystems may not be prepared for autoprobe to be
+ disabled, take care to test your selected drivers. Built-in drivers are
+ unaffected by this policy and will autoprobe unless the bus itself has
+ disabled autoprobe.
+
+ If in doubt, say Y. The N case is only for expert configurations, and
+ selective "autoprobe=0" in modprobe policy is the common expectation.
+
endmenu
diff --git a/Documentation/ABI/stable/sysfs-module b/Documentation/ABI/stable/sysfs-module
index 397c5c850894..1085d0942b17 100644
--- a/Documentation/ABI/stable/sysfs-module
+++ b/Documentation/ABI/stable/sysfs-module
@@ -55,3 +55,13 @@ Description:
requests, the "driver_async_probe=..." kernel command line, the
"async_probe" module option, then this default. Write a valid
boolean value to toggle this policy.
+
+What: /sys/module/module/parameters/modules_autoprobe
+Description:
+ (RW) Emits "1" if drivers from loadable modules automatically
+ attach to their devices. Emits "0" if userspace is responsible
+ to attach devices to drivers post module load, or device
+ arrival. This value defaults to CONFIG_MODULES_AUTOPROBE compile
+ to configuration and is overridden by either a bus's autoprobe
+ policy or the per-module "autoprobe" option. Write a valid
+ boolean value to toggle this policy.
diff --git a/drivers/base/base.h b/drivers/base/base.h
index 1ae9a1679504..908ba366b8d2 100644
--- a/drivers/base/base.h
+++ b/drivers/base/base.h
@@ -230,6 +230,7 @@ void device_block_probing(void);
void device_unblock_probing(void);
void deferred_probe_extend_timeout(void);
void driver_deferred_probe_trigger(void);
+bool driver_autoprobe(struct device_driver *drv);
const char *device_get_devnode(const struct device *dev, umode_t *mode,
kuid_t *uid, kgid_t *gid, const char **tmp);
diff --git a/include/linux/module.h b/include/linux/module.h
index d80c3ea57472..7db34ef0400c 100644
--- a/include/linux/module.h
+++ b/include/linux/module.h
@@ -450,6 +450,7 @@ struct module {
#endif
bool async_probe_requested;
+ bool autoprobe;
/* Exception table */
unsigned int num_exentries;
@@ -761,6 +762,14 @@ static inline bool module_requested_async_probing(struct module *module)
return module && module->async_probe_requested;
}
+static inline bool module_requested_autoprobe(struct module *module)
+{
+ /* Built-in modules autoprobe by default. */
+ if (!module)
+ return true;
+ return module->autoprobe;
+}
+
static inline bool is_livepatch_module(struct module *mod)
{
#ifdef CONFIG_LIVEPATCH
@@ -865,6 +874,11 @@ static inline bool module_requested_async_probing(struct module *module)
return false;
}
+static inline bool module_requested_autoprobe(struct module *module)
+{
+ /* Built-in modules autoprobe by default. */
+ return true;
+}
static inline void set_module_sig_enforced(void)
{
diff --git a/drivers/base/bus.c b/drivers/base/bus.c
index 9eb7771706f0..26ca98cd2a74 100644
--- a/drivers/base/bus.c
+++ b/drivers/base/bus.c
@@ -677,6 +677,11 @@ static ssize_t uevent_store(struct device_driver *drv, const char *buf,
}
static DRIVER_ATTR_WO(uevent);
+bool driver_autoprobe(struct device_driver *drv)
+{
+ return module_requested_autoprobe(drv->owner);
+}
+
/**
* bus_add_driver - Add a driver to the bus.
* @drv: driver.
@@ -711,7 +716,7 @@ int bus_add_driver(struct device_driver *drv)
goto out_unregister;
klist_add_tail(&priv->knode_bus, &sp->klist_drivers);
- if (sp->drivers_autoprobe) {
+ if (sp->drivers_autoprobe && driver_autoprobe(drv)) {
error = driver_attach(drv);
if (error)
goto out_del_list;
diff --git a/drivers/base/dd.c b/drivers/base/dd.c
index 349f31bedfa1..926e120b3cc4 100644
--- a/drivers/base/dd.c
+++ b/drivers/base/dd.c
@@ -917,6 +917,12 @@ struct device_attach_data {
* driver, we'll encounter one that requests asynchronous probing.
*/
bool have_async;
+
+ /*
+ * On initial device arrival driver attach is subject to
+ * driver_autoprobe() policy.
+ */
+ bool initial_probe;
};
static int __device_attach_driver(struct device_driver *drv, void *_data)
@@ -926,6 +932,13 @@ static int __device_attach_driver(struct device_driver *drv, void *_data)
bool async_allowed;
int ret;
+ /*
+ * At initial probe of a newly arrived device, honor the policy to defer
+ * attachment to explicit userspace bind request.
+ */
+ if (data->initial_probe && !driver_autoprobe(drv))
+ return 0;
+
ret = driver_match_device(drv, dev);
if (ret == 0) {
/* no match */
@@ -998,8 +1011,13 @@ static void __device_attach_async_helper(void *_dev, async_cookie_t cookie)
put_device(dev);
}
-static int __device_attach(struct device *dev, bool allow_async)
+#define DEVICE_ATTACH_F_ASYNC BIT(0)
+#define DEVICE_ATTACH_F_INITIAL BIT(1)
+
+static int __device_attach(struct device *dev, unsigned long flags)
{
+ bool allow_async = flags & DEVICE_ATTACH_F_ASYNC;
+ bool initial_probe = flags & DEVICE_ATTACH_F_INITIAL;
int ret = 0;
bool async = false;
@@ -1023,6 +1041,7 @@ static int __device_attach(struct device *dev, bool allow_async)
.dev = dev,
.check_async = allow_async,
.want_async = false,
+ .initial_probe = initial_probe,
};
if (dev->parent)
@@ -1071,7 +1090,7 @@ static int __device_attach(struct device *dev, bool allow_async)
*/
int device_attach(struct device *dev)
{
- return __device_attach(dev, false);
+ return __device_attach(dev, 0);
}
EXPORT_SYMBOL_GPL(device_attach);
@@ -1083,7 +1102,8 @@ void device_initial_probe(struct device *dev)
return;
if (sp->drivers_autoprobe)
- __device_attach(dev, true);
+ __device_attach(dev, DEVICE_ATTACH_F_INITIAL |
+ DEVICE_ATTACH_F_ASYNC);
subsys_put(sp);
}
diff --git a/kernel/module/main.c b/kernel/module/main.c
index 710ee30b3bea..3fca2bc3217d 100644
--- a/kernel/module/main.c
+++ b/kernel/module/main.c
@@ -3001,6 +3001,10 @@ void flush_module_init_free_work(void)
static bool async_probe;
module_param(async_probe, bool, 0644);
+/* Default value for module->autoprobe */
+bool modules_autoprobe = IS_ENABLED(CONFIG_MODULES_AUTOPROBE);
+module_param(modules_autoprobe, bool, 0644);
+
/*
* This is where the real work happens.
*
@@ -3304,6 +3308,14 @@ static int unknown_module_param_cb(char *param, char *val, const char *modname,
return 0;
}
+ if (strcmp(param, "autoprobe") == 0) {
+ bool autoprobe;
+
+ if (kstrtobool(val, &autoprobe) >= 0)
+ mod->autoprobe = autoprobe;
+ return 0;
+ }
+
/* Check for magic 'dyndbg' arg */
ret = ddebug_dyndbg_module_param_cb(param, val, modname);
if (ret != 0)
@@ -3473,6 +3485,7 @@ static int load_module(struct load_info *info, const char __user *uargs,
goto bug_cleanup;
mod->async_probe_requested = async_probe;
+ mod->autoprobe = modules_autoprobe;
/* Module is ready to execute: parsing args may do that. */
after_dashes = parse_args(mod->name, mod->args, mod->kp, mod->num_kp,
--
2.52.0
^ permalink raw reply related
* [PATCH v2 03/19] device core: Introduce confidential device acceptance
From: Dan Williams @ 2026-03-03 0:01 UTC (permalink / raw)
To: linux-coco, linux-pci
Cc: gregkh, aik, aneesh.kumar, yilun.xu, bhelgaas, alistair23, lukas,
jgg, Christoph Hellwig, Jason Gunthorpe, Marek Szyprowski,
Robin Murphy, Roman Kisel, Samuel Ortiz, Rafael J. Wysocki,
Danilo Krummrich
In-Reply-To: <20260303000207.1836586-1-dan.j.williams@intel.com>
An "accepted" device is one that is allowed to access private memory within
a Trusted Computing Boundary (TCB). The concept of "acceptance" is distinct
from other device properties like usb_device::authorized, or
tb_switch::authorized. The entry to the accepted state is a violent
operation in which the device will reject MMIO requests that are not
encrypted, and the device enters a new IOMMU protection domain to allow it
to access addresses that were previously off-limits.
Subsystems like the DMA mapping layer, that need to modify their behavior
based on the accept state, may only have access to the base 'struct
device'. It is also likely that the concept of TCB acceptance grows beyond
PCI devices over time. For these reasons, introduce the concept of
acceptance in 'struct device_private' which is device common, but only
suitable for buses and built-in infrastructure to consume.
Cc: Christoph Hellwig <hch@lst.de>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Robin Murphy <robin.murphy@arm.com>
Cc: Roman Kisel <romank@linux.microsoft.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Samuel Ortiz <sameo@rivosinc.com>
Cc: Alexey Kardashevskiy <aik@amd.com>
Cc: Xu Yilun <yilun.xu@linux.intel.com>
Cc: "Aneesh Kumar K.V" <aneesh.kumar@kernel.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: "Rafael J. Wysocki" <rafael@kernel.org>
Cc: Danilo Krummrich <dakr@kernel.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
drivers/base/Kconfig | 4 +++
drivers/base/Makefile | 1 +
drivers/base/base.h | 9 +++++++
include/linux/device.h | 22 ++++++++++++++++
drivers/base/coco.c | 58 ++++++++++++++++++++++++++++++++++++++++++
5 files changed, 94 insertions(+)
create mode 100644 drivers/base/coco.c
diff --git a/drivers/base/Kconfig b/drivers/base/Kconfig
index 1786d87b29e2..d4743bf978ec 100644
--- a/drivers/base/Kconfig
+++ b/drivers/base/Kconfig
@@ -249,4 +249,8 @@ config FW_DEVLINK_SYNC_STATE_TIMEOUT
command line option on every system/board your kernel is expected to
work on.
+config CONFIDENTIAL_DEVICES
+ depends on ARCH_HAS_CC_PLATFORM
+ bool
+
endmenu
diff --git a/drivers/base/Makefile b/drivers/base/Makefile
index 8074a10183dc..e11052cd5253 100644
--- a/drivers/base/Makefile
+++ b/drivers/base/Makefile
@@ -27,6 +27,7 @@ obj-$(CONFIG_GENERIC_MSI_IRQ) += platform-msi.o
obj-$(CONFIG_GENERIC_ARCH_TOPOLOGY) += arch_topology.o
obj-$(CONFIG_GENERIC_ARCH_NUMA) += arch_numa.o
obj-$(CONFIG_ACPI) += physical_location.o
+obj-$(CONFIG_CONFIDENTIAL_DEVICES) += coco.o
obj-y += test/
diff --git a/drivers/base/base.h b/drivers/base/base.h
index b68355f5d6e3..1ae9a1679504 100644
--- a/drivers/base/base.h
+++ b/drivers/base/base.h
@@ -119,8 +119,13 @@ struct driver_type {
* @dead: This device is currently either in the process of or has been
* removed from the system. Any asynchronous events scheduled for this
* device should exit without taking any action.
+ * @cc_accepted: track the TEE acceptance state of the device for deferred
+ * probing, MMIO mapping type, and SWIOTLB bypass for private memory DMA.
*
* Nothing outside of the driver core should ever touch these fields.
+ *
+ * All bitfield flags are manipulated under device_lock() to avoid
+ * read-modify-write collisions.
*/
struct device_private {
struct klist klist_children;
@@ -136,6 +141,10 @@ struct device_private {
struct driver_type driver_type;
#endif
u8 dead:1;
+#ifdef CONFIG_CONFIDENTIAL_DEVICES
+ u8 cc_accepted:1;
+#endif
+
};
#define to_device_private_parent(obj) \
container_of(obj, struct device_private, knode_parent)
diff --git a/include/linux/device.h b/include/linux/device.h
index 0be95294b6e6..4470365d772b 100644
--- a/include/linux/device.h
+++ b/include/linux/device.h
@@ -1191,6 +1191,28 @@ static inline bool device_link_test(const struct device_link *link, u32 flags)
return !!(link->flags & flags);
}
+/* Confidential Device state helpers */
+#ifdef CONFIG_CONFIDENTIAL_DEVICES
+int device_cc_accept(struct device *dev);
+int device_cc_reject(struct device *dev);
+bool device_cc_accepted(struct device *dev);
+#else
+static inline int device_cc_accept(struct device *dev)
+{
+ lockdep_assert_held(&dev->mutex);
+ return 0;
+}
+static inline int device_cc_reject(struct device *dev)
+{
+ lockdep_assert_held(&dev->mutex);
+ return 0;
+}
+static inline bool device_cc_accepted(struct device *dev)
+{
+ return false;
+}
+#endif /* CONFIG_CONFIDENTIAL_DEVICES */
+
/* Create alias, so I can be autoloaded. */
#define MODULE_ALIAS_CHARDEV(major,minor) \
MODULE_ALIAS("char-major-" __stringify(major) "-" __stringify(minor))
diff --git a/drivers/base/coco.c b/drivers/base/coco.c
new file mode 100644
index 000000000000..2589745530e3
--- /dev/null
+++ b/drivers/base/coco.c
@@ -0,0 +1,58 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2026 Intel Corporation */
+#include <linux/device.h>
+#include <linux/lockdep.h>
+#include "base.h"
+
+/*
+ * Confidential devices implement encrypted + integrity protected MMIO and have
+ * the ability to issue DMA to encrypted + integrity protected System RAM
+ * (private memory). The device_cc_*() helpers aid buses in setting the
+ * acceptance state and the DMA mapping subsystem augmenting behavior in the
+ * presence of accepted devices.
+ */
+
+/**
+ * device_cc_accept(): Mark a device as able to access private memory
+ * @dev: device to accept
+ *
+ * Confidential bus drivers use this helper to accept devices. For example, PCI
+ * has a sysfs ABI to accept devices after relying party attestation.
+ *
+ * Given that moving a device into confidential / private operation implicates
+ * changes to MMIO mapping attributes and DMA mappings, the transition must be
+ * done while the device is idle (driver detached).
+ */
+int device_cc_accept(struct device *dev)
+{
+ lockdep_assert_held(&dev->mutex);
+
+ if (dev->driver)
+ return -EBUSY;
+ dev->p->cc_accepted = 1;
+
+ return 0;
+}
+
+int device_cc_reject(struct device *dev)
+{
+ lockdep_assert_held(&dev->mutex);
+
+ if (dev->driver)
+ return -EBUSY;
+ dev->p->cc_accepted = 0;
+
+ return 0;
+}
+
+/**
+ * device_cc_accepted(): Fetch the device's ability to access private memory
+ * @dev: device to check
+ *
+ * Mechanisms like swiotlb and dma_alloc() need to augment their behavior in the
+ * presence of accepted devices.
+ */
+bool device_cc_accepted(struct device *dev)
+{
+ return dev->p->cc_accepted;
+}
--
2.52.0
^ permalink raw reply related
* [PATCH v2 02/19] device core: Fix kernel-doc warnings in base.h
From: Dan Williams @ 2026-03-03 0:01 UTC (permalink / raw)
To: linux-coco, linux-pci
Cc: gregkh, aik, aneesh.kumar, yilun.xu, bhelgaas, alistair23, lukas,
jgg
In-Reply-To: <20260303000207.1836586-1-dan.j.williams@intel.com>
In preparation for adding new fields to 'struct device_private' fix up
existing kernel-doc warnings in this header file of the form:
Warning: drivers/base/base.h:59 struct member 'subsys' not described in
'subsys_private'
Warning: drivers/base/base.h:59 struct member 'devices_kset' not described
in 'subsys_private'
Warning: drivers/base/base.h:59 struct member 'interfaces' not described in
'subsys_private'
Warning: drivers/base/base.h:59 struct member 'mutex' not described in
'subsys_private'
...which are simple replacements of " - " with ": ".
Add new descriptions for these previously undescribed fields:
Warning: drivers/base/base.h:58 struct member 'drivers_autoprobe' not
described in 'subsys_private'
Warning: drivers/base/base.h:117 struct member 'deferred_probe_reason' not
described in 'device_private'
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
drivers/base/base.h | 79 +++++++++++++++++++++++----------------------
1 file changed, 41 insertions(+), 38 deletions(-)
diff --git a/drivers/base/base.h b/drivers/base/base.h
index 430cbefbc97f..b68355f5d6e3 100644
--- a/drivers/base/base.h
+++ b/drivers/base/base.h
@@ -13,27 +13,28 @@
#include <linux/notifier.h>
/**
- * struct subsys_private - structure to hold the private to the driver core portions of the bus_type/class structure.
- *
- * @subsys - the struct kset that defines this subsystem
- * @devices_kset - the subsystem's 'devices' directory
- * @interfaces - list of subsystem interfaces associated
- * @mutex - protect the devices, and interfaces lists.
- *
- * @drivers_kset - the list of drivers associated
- * @klist_devices - the klist to iterate over the @devices_kset
- * @klist_drivers - the klist to iterate over the @drivers_kset
- * @bus_notifier - the bus notifier list for anything that cares about things
- * on this bus.
- * @bus - pointer back to the struct bus_type that this structure is associated
- * with.
+ * struct subsys_private - structure to hold the private to the driver core
+ * portions of the bus_type/class structure.
+ * @subsys: the struct kset that defines this subsystem
+ * @devices_kset: the subsystem's 'devices' directory
+ * @interfaces: list of subsystem interfaces associated
+ * @mutex: protect the devices, and interfaces lists.
+ * @drivers_kset: the list of drivers associated
+ * @klist_devices: the klist to iterate over the @devices_kset
+ * @klist_drivers: the klist to iterate over the @drivers_kset
+ * @bus_notifier: the bus notifier list for anything that cares about things
+ * on this bus.
+ * @drivers_autoprobe: gate whether new devices are automatically attached to
+ * registered drivers, or new drivers automatically attach
+ * to existing devices.
+ * @bus: pointer back to the struct bus_type that this structure is associated
+ * with.
* @dev_root: Default device to use as the parent.
- *
- * @glue_dirs - "glue" directory to put in-between the parent device to
- * avoid namespace conflicts
- * @class - pointer back to the struct class that this structure is associated
- * with.
- * @lock_key: Lock class key for use by the lock validator
+ * @glue_dirs: "glue" directory to put in-between the parent device to
+ * avoid namespace conflicts
+ * @class: pointer back to the struct class that this structure is associated
+ * with.
+ * @lock_key: Lock class key for use by the lock validator
*
* This structure is the one that is the actual kobject allowing struct
* bus_type/class to be statically allocated safely. Nothing outside of the
@@ -98,24 +99,26 @@ struct driver_type {
#endif
/**
- * struct device_private - structure to hold the private to the driver core portions of the device structure.
- *
- * @klist_children - klist containing all children of this device
- * @knode_parent - node in sibling list
- * @knode_driver - node in driver list
- * @knode_bus - node in bus list
- * @knode_class - node in class list
- * @deferred_probe - entry in deferred_probe_list which is used to retry the
- * binding of drivers which were unable to get all the resources needed by
- * the device; typically because it depends on another driver getting
- * probed first.
- * @async_driver - pointer to device driver awaiting probe via async_probe
- * @device - pointer back to the struct device that this structure is
- * associated with.
- * @driver_type - The type of the bound Rust driver.
- * @dead - This device is currently either in the process of or has been
- * removed from the system. Any asynchronous events scheduled for this
- * device should exit without taking any action.
+ * struct device_private - structure to hold the private to the driver core
+ * portions of the device structure.
+ * @klist_children: klist containing all children of this device
+ * @knode_parent: node in sibling list
+ * @knode_driver: node in driver list
+ * @knode_bus: node in bus list
+ * @knode_class: node in class list
+ * @deferred_probe: entry in deferred_probe_list which is used to retry the
+ * binding of drivers which were unable to get all the
+ * resources needed by the device; typically because it depends
+ * on another driver getting probed first.
+ * @async_driver: pointer to device driver awaiting probe via async_probe
+ * @deferred_probe_reason: capture the -EPROBE_DEFER message emitted with
+ * dev_err_probe() for later retrieval via debugfs
+ * @device: pointer back to the struct device that this structure is
+ * associated with.
+ * @driver_type: The type of the bound Rust driver.
+ * @dead: This device is currently either in the process of or has been
+ * removed from the system. Any asynchronous events scheduled for this
+ * device should exit without taking any action.
*
* Nothing outside of the driver core should ever touch these fields.
*/
--
2.52.0
^ permalink raw reply related
* [PATCH v2 00/19] PCI/TSM: TEE I/O infrastructure
From: Dan Williams @ 2026-03-03 0:01 UTC (permalink / raw)
To: linux-coco, linux-pci
Cc: gregkh, aik, aneesh.kumar, yilun.xu, bhelgaas, alistair23, lukas,
jgg, Andy Lutomirski, Arnd Bergmann, Borislav Petkov,
Christoph Hellwig, Danilo Krummrich, Dave Hansen, Donald Hunter,
H. Peter Anvin, Ingo Molnar, Jakub Kicinski, Jason Gunthorpe,
Luis Chamberlain, Marek Szyprowski, Peter Zijlstra,
Rafael J. Wysocki, Robin Murphy, Roman Kisel, Samuel Ortiz,
Saravana Kannan, Suzuki K Poulose, Thomas Gleixner,
Thomas Gleixner
Changes since v1 [1]:
- Add a netlink ABI for conveying device attestation evidence and
interface reports
- Add a module autoprobe policy proposal
- Add simulated device evidence support to samples/devsec/
- Add MMIO resource evaluation from a TDISP device interface report
- Include device_cc_accepted() proposals for DMA setup
- Restore a lookup mechanism from tsm class device to all established
streams
- Clarify TEE vs Confidential vs Private in drivers/base/coco.c (Greg)
- Move 'cc_accepted' to an additional bitfield next to 'dead' (Greg)
- Drop device_cc_probe() proposal (Jason)
[1]: http://lore.kernel.org/20250827035259.1356758-1-dan.j.williams@intel.com
---
Overview
========
TEE I/O starts with the premise that devices are adversarial. That
threat model needs a series of new ABIs and mechanisms. The x86 changes
and the samples/devsec/ implementation in this set serve to have a
consumer for all of these proposed mechanisms.
1/ Userspace needs to be able to intercept driver attach. If a relying
party does not endorse the system talking to a given device then
userspace needs a control point to decline device operation. Module
policy is suitable for that policy mechanism. See "device core:
Autoprobe considered harmful?"
2/ Userspace needs to be able to gather evidence that validates the
device's identity, configuration, and active mappings of MMIO and DMA.
See "PCI/TSM: Add 'evidence' support"
3/ To gather and act on device evidence a device needs a "lock"
mechanism to hold a stable configuration, and an "accept" mechanism to
bring the device into operation after relying party validation. See
"PCI/TSM: Add Device Security (TVM Guest) LOCK operation support" and
"PCI/TSM: Add Device Security (TVM Guest) ACCEPT operation support".
4/ Drivers must be unmodified (1): ioremap() requests must automatically
determine whether a resource range is mapped as encrypted or not. See
"x86, ioremap, resource: Support IORES_DESC_ENCRYPTED for encrypted PCI
MMIO". TODO: test unencrypted ranges in the middle of a PCI device BAR
that is otherwise encrypted (MSI-X table case).
5/ Drivers must be unmodified (2): dma_alloc_coherent() and dma_map()
need to bypass swiotlb and potentially modify DMA handles when a device
is accepted to DMA direct to private memory. See "x86, swiotlb: Teach
swiotlb to skip 'accepted' devices" and "x86, dma: Allow accepted
devices to map private memory".
Note an example SEV-TIO implementation of the lock+accept operations is
out for review here [2] (based on older baseline of tsm.git#staging).
[2]: http://lore.kernel.org/20260225053806.3311234-1-aik@amd.com
On PCI/TSM Netlink and Rust SPDM
================================
The PCI/TSM netlink proposal is a result of the discussion from the Rust
SPDM proposal [3]. That thread discussed the merits of an SPDM netlink
ABI that multicasts signature events and a ".cma" keyring to
authenticate PCI devices. The PCI/TSM netlink proposal diverges
significantly based on the following assumptions:
1/ Device acceptance decisions are based on evidence material beyond
whether the device publishes a valid root certificate (kernel SPDM
library proposal).
2/ Automatic device identity revalidation after reset is secondary to
initial device acceptance. It is follow-on work that can be achieved
without a ".cma" ring. For example, cache a hash of the device
certificate chain and / or measurements. Otherwise, mere identity
revalidation is insufficient for PCI TDISP.
3/ Device evidence mutates based on userspace taking action on the
device state. For example, the device interface report is not available
until post "lock". The result, the netlink interface must be on demand,
not implicit multicast. PCI/TSM evidence conveyance is a netlink
"dump" request.
The proposal for how the native kernel SPDM support would interact with
the PCI/TSM implementation is via an "spdm-tsm" driver. An "spdm-tsm"
driver allows for userspace policy to select between a kernel native
"spdm-tsm" and "$platform-tsm" as only one TSM can have a session
established at a time.
[3]: http://lore.kernel.org/20260211032935.2705841-1-alistair.francis@wdc.com
On PCI/TSM Netlink and guest request
====================================
One of the open questions is whether pci_tsm_guest_req() should be used
to convey device evidence to guests. In other words, if the core kernel
understands 'struct pci_tsm_evidence' in a common way across
architectures, why not implement a common transport and save
pci_tsm_guest_req() for other ancillary messages that are indeed
implementation specific?
This all passes a tools/testing/devsec/devsec.sh run. It wants a rebase
on v7.0-rc2. It is pushed out as new tag, devsec-20260302, in the
tsm.git#staging tree. The Maturity Map [4] has been updated accordingly.
[4]: https://git.kernel.org/pub/scm/linux/kernel/git/devsec/tsm.git/tree/Documentation/driver-api/pci/tsm.rst?h=staging
Dan Williams (19):
PCI/TSM: Report active IDE streams per host bridge
device core: Fix kernel-doc warnings in base.h
device core: Introduce confidential device acceptance
modules: Document the global async_probe parameter
device core: Autoprobe considered harmful?
PCI/TSM: Add Device Security (TVM Guest) LOCK operation support
PCI/TSM: Add Device Security (TVM Guest) ACCEPT operation support
PCI/TSM: Add "evidence" support
PCI/TSM: Support creating encrypted MMIO descriptors via TDISP Report
x86, swiotlb: Teach swiotlb to skip "accepted" devices
x86, dma: Allow accepted devices to map private memory
x86, ioremap, resource: Support IORES_DESC_ENCRYPTED for encrypted PCI
MMIO
samples/devsec: Introduce a PCI device-security bus + endpoint sample
samples/devsec: Add sample IDE establishment
samples/devsec: Add sample TSM bind and guest_request flows
samples/devsec: Introduce a "Device Security TSM" sample driver
tools/testing/devsec: Add a script to exercise samples/devsec/
samples/devsec: Add evidence support
tools/testing/devsec: Add basic evidence retrieval validation
drivers/base/Kconfig | 28 +
drivers/pci/Kconfig | 2 +
samples/Kconfig | 19 +
drivers/base/Makefile | 1 +
drivers/pci/Makefile | 2 +-
drivers/pci/tsm/Makefile | 9 +
samples/Makefile | 1 +
samples/devsec/Makefile | 16 +
Documentation/ABI/stable/sysfs-module | 20 +
Documentation/ABI/testing/sysfs-bus-pci | 47 +-
Documentation/ABI/testing/sysfs-class-tsm | 32 +
Documentation/ABI/testing/sysfs-faux-devsec | 15 +
Documentation/driver-api/pci/tsm.rst | 44 ++
Documentation/netlink/specs/pci-tsm.yaml | 151 ++++
drivers/base/base.h | 89 ++-
drivers/pci/tsm/netlink.h | 23 +
include/linux/device.h | 23 +
include/linux/ioport.h | 2 +
include/linux/module.h | 14 +
include/linux/pci-ide.h | 2 +
include/linux/pci-tsm.h | 121 ++-
include/linux/swiotlb.h | 15 +-
include/linux/tsm.h | 3 +
include/uapi/linux/pci-tsm-netlink.h | 101 +++
samples/devsec/devsec.h | 48 ++
arch/x86/kernel/pci-dma.c | 2 +-
arch/x86/mm/ioremap.c | 49 +-
arch/x86/mm/mem_encrypt.c | 5 +-
drivers/base/bus.c | 7 +-
drivers/base/coco.c | 58 ++
drivers/base/dd.c | 26 +-
drivers/pci/ide.c | 4 +
drivers/pci/{tsm.c => tsm/core.c} | 532 ++++++++++++-
drivers/pci/tsm/evidence.c | 274 +++++++
drivers/pci/tsm/netlink.c | 43 ++
drivers/virt/coco/tsm-core.c | 138 ++++
kernel/dma/swiotlb.c | 1 +
kernel/module/main.c | 13 +
samples/devsec/bus.c | 784 ++++++++++++++++++++
samples/devsec/common.c | 160 ++++
samples/devsec/link_tsm.c | 432 +++++++++++
samples/devsec/pci.c | 39 +
samples/devsec/tsm.c | 131 ++++
tools/testing/devsec/devsec.sh | 280 +++++++
MAINTAINERS | 6 +-
45 files changed, 3736 insertions(+), 76 deletions(-)
create mode 100644 drivers/pci/tsm/Makefile
create mode 100644 samples/devsec/Makefile
create mode 100644 Documentation/ABI/testing/sysfs-faux-devsec
create mode 100644 Documentation/netlink/specs/pci-tsm.yaml
create mode 100644 drivers/pci/tsm/netlink.h
create mode 100644 include/uapi/linux/pci-tsm-netlink.h
create mode 100644 samples/devsec/devsec.h
create mode 100644 drivers/base/coco.c
rename drivers/pci/{tsm.c => tsm/core.c} (63%)
create mode 100644 drivers/pci/tsm/evidence.c
create mode 100644 drivers/pci/tsm/netlink.c
create mode 100644 samples/devsec/bus.c
create mode 100644 samples/devsec/common.c
create mode 100644 samples/devsec/link_tsm.c
create mode 100644 samples/devsec/pci.c
create mode 100644 samples/devsec/tsm.c
create mode 100755 tools/testing/devsec/devsec.sh
base-commit: c2012263047689e495e81c96d7d5b0586299578d
--
2.52.0
^ permalink raw reply
* [PATCH v2 01/19] PCI/TSM: Report active IDE streams per host bridge
From: Dan Williams @ 2026-03-03 0:01 UTC (permalink / raw)
To: linux-coco, linux-pci
Cc: gregkh, aik, aneesh.kumar, yilun.xu, bhelgaas, alistair23, lukas,
jgg
In-Reply-To: <20260303000207.1836586-1-dan.j.williams@intel.com>
The first attempt at an ABI for this failed to account for naming
collisions across host bridges:
Commit a4438f06b1db ("PCI/TSM: Report active IDE streams")
Revive this ABI with a per host bridge link that appears at first stream
creation for a given host bridge and disappears after the last stream is
removed.
For systems with many host bridge objects it allows:
ls /sys/class/tsm/tsmN/pci*/stream*
...to find all the host bridges with active streams without first iterating
over all host bridges. Yilun notes that is handy to have this short cut [1]
and from an administrator perspective it helps with inventory for
constrained stream resources.
Link: http://lore.kernel.org/aXLtILY85oMU5qlb@yilunxu-OptiPlex-7050 [1]
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
Documentation/ABI/testing/sysfs-class-tsm | 13 +++
include/linux/pci-ide.h | 2 +
include/linux/tsm.h | 3 +
drivers/pci/ide.c | 4 +
drivers/virt/coco/tsm-core.c | 97 +++++++++++++++++++++++
5 files changed, 119 insertions(+)
diff --git a/Documentation/ABI/testing/sysfs-class-tsm b/Documentation/ABI/testing/sysfs-class-tsm
index 2949468deaf7..1ddb8f357961 100644
--- a/Documentation/ABI/testing/sysfs-class-tsm
+++ b/Documentation/ABI/testing/sysfs-class-tsm
@@ -7,3 +7,16 @@ Description:
signals when the PCI layer is able to support establishment of
link encryption and other device-security features coordinated
through a platform tsm.
+
+What: /sys/class/tsm/tsmN/pciDDDD:BB
+Contact: linux-pci@vger.kernel.org
+Description:
+ (RO) When a PCIe host bridge has established a secure connection
+ via a TSM to an endpoint, this symlink appears. It facilitates a
+ TSM instance scoped view of PCIe Link Encryption and Secure
+ Session resource consumption across host bridges. The symlink
+ appears when a host bridge has 1 or more IDE streams established
+ with this TSM, and disappears when that number returns to 0. See
+ Documentation/ABI/testing/sysfs-devices-pci-host-bridge for the
+ description of the pciDDDD:BB/streamH.R.E symlink and the
+ pciDDDD:BB/available_secure_streams attribute.
diff --git a/include/linux/pci-ide.h b/include/linux/pci-ide.h
index ae07d9f699c0..381a1bf22a95 100644
--- a/include/linux/pci-ide.h
+++ b/include/linux/pci-ide.h
@@ -82,6 +82,7 @@ struct pci_ide_regs {
* @host_bridge_stream: allocated from host bridge @ide_stream_ida pool
* @stream_id: unique Stream ID (within Partner Port pairing)
* @name: name of the established Selective IDE Stream in sysfs
+ * @tsm_dev: For TSM established IDE, the TSM device context
*
* Negative @stream_id values indicate "uninitialized" on the
* expectation that with TSM established IDE the TSM owns the stream_id
@@ -93,6 +94,7 @@ struct pci_ide {
u8 host_bridge_stream;
int stream_id;
const char *name;
+ struct tsm_dev *tsm_dev;
};
/*
diff --git a/include/linux/tsm.h b/include/linux/tsm.h
index 22e05b2aac69..a3b7ab668eff 100644
--- a/include/linux/tsm.h
+++ b/include/linux/tsm.h
@@ -123,4 +123,7 @@ int tsm_report_unregister(const struct tsm_report_ops *ops);
struct tsm_dev *tsm_register(struct device *parent, struct pci_tsm_ops *ops);
void tsm_unregister(struct tsm_dev *tsm_dev);
struct tsm_dev *find_tsm_dev(int id);
+struct pci_ide;
+int tsm_ide_stream_register(struct pci_ide *ide);
+void tsm_ide_stream_unregister(struct pci_ide *ide);
#endif /* __TSM_H */
diff --git a/drivers/pci/ide.c b/drivers/pci/ide.c
index 23f554490539..9629f3ceb213 100644
--- a/drivers/pci/ide.c
+++ b/drivers/pci/ide.c
@@ -11,6 +11,7 @@
#include <linux/pci_regs.h>
#include <linux/slab.h>
#include <linux/sysfs.h>
+#include <linux/tsm.h>
#include "pci.h"
@@ -372,6 +373,9 @@ void pci_ide_stream_release(struct pci_ide *ide)
if (ide->partner[PCI_IDE_EP].enable)
pci_ide_stream_disable(pdev, ide);
+ if (ide->tsm_dev)
+ tsm_ide_stream_unregister(ide);
+
if (ide->partner[PCI_IDE_RP].setup)
pci_ide_stream_teardown(rp, ide);
diff --git a/drivers/virt/coco/tsm-core.c b/drivers/virt/coco/tsm-core.c
index 8712df8596a1..3c99c38cfaa5 100644
--- a/drivers/virt/coco/tsm-core.c
+++ b/drivers/virt/coco/tsm-core.c
@@ -4,10 +4,12 @@
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
#include <linux/tsm.h>
+#include <linux/pci.h>
#include <linux/device.h>
#include <linux/module.h>
#include <linux/cleanup.h>
#include <linux/pci-tsm.h>
+#include <linux/pci-ide.h>
static struct class *tsm_class;
static DEFINE_IDA(tsm_ida);
@@ -104,6 +106,100 @@ void tsm_unregister(struct tsm_dev *tsm_dev)
}
EXPORT_SYMBOL_GPL(tsm_unregister);
+static DEFINE_XARRAY(tsm_ide_streams);
+static DEFINE_MUTEX(tsm_ide_streams_lock);
+
+/* tracker for the bridge symlink when the bridge has any streams */
+struct tsm_ide_stream {
+ struct tsm_dev *tsm_dev;
+ struct pci_host_bridge *bridge;
+ struct kref kref;
+};
+
+static struct tsm_ide_stream *create_streams(struct tsm_dev *tsm_dev,
+ struct pci_host_bridge *bridge)
+{
+ int rc;
+
+ struct tsm_ide_stream *streams __free(kfree) =
+ kzalloc(sizeof(*streams), GFP_KERNEL);
+ if (!streams)
+ return NULL;
+
+ streams->tsm_dev = tsm_dev;
+ streams->bridge = bridge;
+ kref_init(&streams->kref);
+ rc = xa_insert(&tsm_ide_streams, (unsigned long)bridge, streams,
+ GFP_KERNEL);
+ if (rc)
+ return NULL;
+
+ rc = sysfs_create_link(&tsm_dev->dev.kobj, &bridge->dev.kobj,
+ dev_name(&bridge->dev));
+ if (rc) {
+ xa_erase(&tsm_ide_streams, (unsigned long)bridge);
+ return NULL;
+ }
+
+ return no_free_ptr(streams);
+}
+
+int tsm_ide_stream_register(struct pci_ide *ide)
+{
+ struct tsm_ide_stream *streams;
+ struct pci_dev *pdev = ide->pdev;
+ struct pci_tsm *tsm = pdev->tsm;
+ struct tsm_dev *tsm_dev = tsm->tsm_dev;
+ struct pci_host_bridge *bridge = pci_find_host_bridge(pdev->bus);
+
+ guard(mutex)(&tsm_ide_streams_lock);
+ streams = xa_load(&tsm_ide_streams, (unsigned long)bridge);
+ if (streams)
+ kref_get(&streams->kref);
+ else
+ streams = create_streams(tsm_dev, bridge);
+
+ if (!streams)
+ return -ENOMEM;
+ ide->tsm_dev = tsm_dev;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(tsm_ide_stream_register);
+
+static void destroy_streams(struct kref *kref)
+{
+ struct tsm_ide_stream *streams =
+ container_of(kref, struct tsm_ide_stream, kref);
+ struct tsm_dev *tsm_dev = streams->tsm_dev;
+ struct pci_host_bridge *bridge = streams->bridge;
+
+ lockdep_assert_held(&tsm_ide_streams_lock);
+ sysfs_remove_link(&tsm_dev->dev.kobj, dev_name(&bridge->dev));
+ xa_erase(&tsm_ide_streams, (unsigned long)bridge);
+ kfree(streams);
+}
+
+void tsm_ide_stream_unregister(struct pci_ide *ide)
+{
+ struct tsm_ide_stream *streams;
+ struct tsm_dev *tsm_dev = ide->tsm_dev;
+ struct pci_dev *pdev = ide->pdev;
+ struct pci_host_bridge *bridge = pci_find_host_bridge(pdev->bus);
+
+ guard(mutex)(&tsm_ide_streams_lock);
+ streams = xa_load(&tsm_ide_streams, (unsigned long)bridge);
+ /* catch API abuse */
+ if (dev_WARN_ONCE(&tsm_dev->dev,
+ !streams || streams->tsm_dev != tsm_dev,
+ "no IDE streams associated with %s\n",
+ dev_name(&bridge->dev)))
+ return;
+ kref_put(&streams->kref, destroy_streams);
+ ide->tsm_dev = NULL;
+}
+EXPORT_SYMBOL_GPL(tsm_ide_stream_unregister);
+
static void tsm_release(struct device *dev)
{
struct tsm_dev *tsm_dev = container_of(dev, typeof(*tsm_dev), dev);
@@ -126,6 +222,7 @@ module_init(tsm_init)
static void __exit tsm_exit(void)
{
class_destroy(tsm_class);
+ xa_destroy(&tsm_ide_streams);
}
module_exit(tsm_exit)
--
2.52.0
^ permalink raw reply related
* Re: [PATCH kernel 4/9] dma/swiotlb: Stop forcing SWIOTLB for TDISP devices
From: dan.j.williams @ 2026-03-02 23:53 UTC (permalink / raw)
To: Jason Gunthorpe, dan.j.williams
Cc: Robin Murphy, Alexey Kardashevskiy, x86, linux-kernel, kvm,
linux-pci, Thomas Gleixner, Ingo Molnar, Borislav Petkov,
Dave Hansen, H. Peter Anvin, Sean Christopherson, Paolo Bonzini,
Andy Lutomirski, Peter Zijlstra, Bjorn Helgaas, Marek Szyprowski,
Andrew Morton, Catalin Marinas, Michael Ellerman, Mike Rapoport,
Tom Lendacky, Ard Biesheuvel, Neeraj Upadhyay, Ashish Kalra,
Stefano Garzarella, Melody Wang, Seongman Lee, Joerg Roedel,
Nikunj A Dadhania, Michael Roth, Suravee Suthikulpanit,
Andi Kleen, Kuppuswamy Sathyanarayanan, Tony Luck,
David Woodhouse, Greg Kroah-Hartman, Denis Efremov, Geliang Tang,
Piotr Gregor, Michael S. Tsirkin, Alex Williamson, Arnd Bergmann,
Jesse Barnes, Jacob Pan, Yinghai Lu, Kevin Brodsky,
Jonathan Cameron, Aneesh Kumar K.V (Arm), Xu Yilun, Herbert Xu,
Kim Phillips, Konrad Rzeszutek Wilk, Stefano Stabellini,
Claire Chang, linux-coco, iommu
In-Reply-To: <20260228002808.GO44359@ziepe.ca>
Jason Gunthorpe wrote:
> On Wed, Feb 25, 2026 at 12:57:01PM -0800, dan.j.williams@intel.com wrote:
> > > (since a device that's trusted to access private memory
> > > isn't necessarily prohibited from still also accessing shared memory as
> > > well), hmmm...
> >
> > The specification allows it, but Linux DMA mapping core is not yet ready
> > for it. So the expectation to start is that the device loses access to
> > its original shared IOMMU mappings when converted to private operation.
>
> Yes, the underlying translation changes, but no, it doesn't loose DMA
> access to any shared pages, it just goes through the T=1 IOMMU now.
Yes, what I meant to say is that Linux may need to be prepared for
implementations that do not copy over the shared mappings. At least for
early staging / minimum viable implementation for first merge.
> The T=1 IOMMU will still have them mapped on all three platforms
> AFAIK.
Oh, I thought SEV-TIO had trouble with this, if this is indeed the case,
great, ignore my first comment.
> On TDX/CCA the CPU and IOMMU S2 tables are identical, so of
> course the shared pages are mapped. On AMD there is only one IOMMU so
> the page must also be mapped or non-TDISP is broken.
>
> When this TDISP awareness is put in the DMA API it needs to be done in
> a way that allows DMA_ATTR_CC_DECRYPTED to keep working for TDISP
> devices.
>
> This is important because we are expecting these sorts of things to
> work as part of integrating non-TDISP RDMA devices into CC guests. We
> can't loose access to the shared pages that are shared with the
> non-TDISP devices...
Ok, I need to go look at this DMA_ATTR_CC_DECRYPTED proposal...
I have a v2 of a TEE I/O set going out shortly and sounds like it will
need a rethink for this attribute proposal for v3. I think it still helps to
have combo sets at this stage so the whole lifecycle is visible in one
set, but it is nearly at the point of being too big a set to consider in
one sitting.
> > So on ARM where shared addresses are high, it is future work to figure
> > out how an accepted device might also access shared mappings outside the
> > device's dma_mask.
>
> ARM has a "solution" right now. The location of the high bit is
> controlled by the VMM and the VMM cannot create a CC VM where the IPA
> space exceeds the dma_mask of any assigned device.
>
> Thus the VMM must limit the total available DRAM to fit within the HW
> restrictions.
>
> Hopefully TDX can do the same.
TDX does not have the same problem, but the ARM "solution" seems
reasonable for now.
^ permalink raw reply
* Re: [PATCH v2 3/7] x86/sev: add support for RMPOPT instruction
From: Dave Hansen @ 2026-03-02 23:15 UTC (permalink / raw)
To: Kalra, Ashish, tglx, mingo, bp, dave.hansen, x86, hpa, seanjc,
peterz, thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, KPrateek.Nayak, Tycho.Andersen,
Nathan.Fontenot, jackyli, pgonda, rientjes, jacobhxu, xin,
pawan.kumar.gupta, babu.moger, dyoung, nikunj, john.allen, darwi,
linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <5e857228-4c1f-41aa-a03b-21ec401da434@amd.com>
On 3/2/26 15:09, Kalra, Ashish wrote:
>>> +static void rmpopt_all_physmem(void)
>>> +{
>>> + if (rmpopt_task)
>>> + wake_up_process(rmpopt_task);
>>> +}
>> Wait a sec, doesn't this just run all the time? It'll be doing an RMPOPT
>> on some forever.
> The rmpopt_kthread will be sleeping, till it is woken explicitly by wake_up_process() here.
Sorry for the noise, I totally parsed that bit wrong.
^ permalink raw reply
* Re: [PATCH v2 2/7] x86/sev: add support for enabling RMPOPT
From: Kalra, Ashish @ 2026-03-02 23:11 UTC (permalink / raw)
To: Dave Hansen, tglx, mingo, bp, dave.hansen, x86, hpa, seanjc,
peterz, thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, KPrateek.Nayak, Tycho.Andersen,
Nathan.Fontenot, jackyli, pgonda, rientjes, jacobhxu, xin,
pawan.kumar.gupta, babu.moger, dyoung, nikunj, john.allen, darwi,
linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <86bcca73-0c1b-405f-a72d-debb3c696333@intel.com>
On 3/2/2026 5:00 PM, Dave Hansen wrote:
> On 3/2/26 14:55, Kalra, Ashish wrote:
>>> What's wrong with:
>>>
>>> u64 rmpopt_base = pa_start | MSR_AMD64_RMPOPT_ENABLE;
>>> ...
>>> for_each_online_cpu(cpu)
>>> wrmsrq_on_cpu(cpu, MSR_AMD64_RMPOPT_BASE, rmpopt_base);
>>>
>>> Then there's at least no ugly casting.
>>>
>> RMOPT_BASE MSRs don't need to be set serially, therefore, by
>> using the cpu_online_mask and on_each_cpu_mask(), we can setup the MSRs
>> concurrently and in parallel. Using for_each_online_cpu() will be slower than
>> doing on_each_cpu_mask() as it doesn't send IPIs in parallel, right.
>
> If that's the case and you *need* performance, then please go add a
> wrmsrq_on_cpumask() function to do things in parallel.
Sure.
Thanks,
Ashish
^ permalink raw reply
* Re: [PATCH v2 3/7] x86/sev: add support for RMPOPT instruction
From: Kalra, Ashish @ 2026-03-02 23:09 UTC (permalink / raw)
To: Dave Hansen, tglx, mingo, bp, dave.hansen, x86, hpa, seanjc,
peterz, thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, KPrateek.Nayak, Tycho.Andersen,
Nathan.Fontenot, jackyli, pgonda, rientjes, jacobhxu, xin,
pawan.kumar.gupta, babu.moger, dyoung, nikunj, john.allen, darwi,
linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <a1701ab4-d80f-496c-bdb3-5d94d2d2f673@intel.com>
Hello Dave,
On 3/2/2026 4:57 PM, Dave Hansen wrote:
>> + set_current_state(TASK_INTERRUPTIBLE);
>> + schedule();
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static void rmpopt_all_physmem(void)
>> +{
>> + if (rmpopt_task)
>> + wake_up_process(rmpopt_task);
>> +}
>
> Wait a sec, doesn't this just run all the time? It'll be doing an RMPOPT
> on some forever.
The rmpopt_kthread will be sleeping, till it is woken explicitly by wake_up_process() here.
When the schedule() function is called with the state as TASK_INTERRUPTIBLE or TASK_UNINTERRUPTIBLE,
an additional step is performed: the currently executing process is moved off the run queue before
another process is scheduled. The effect of this is the executing process goes to sleep,
as it no longer is on the run queue. Hence, it never is scheduled by the scheduler.
The thread would then be woken up by calling wake_up_process().
I believe, this is probably the simplest way of sleeping and waking in the kernel.
Thanks,
Ashish
^ permalink raw reply
* Re: [PATCH v2 1/7] x86/cpufeatures: Add X86_FEATURE_AMD_RMPOPT feature flag
From: Dave Hansen @ 2026-03-02 23:00 UTC (permalink / raw)
To: Ashish Kalra, tglx, mingo, bp, dave.hansen, x86, hpa, seanjc,
peterz, thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, KPrateek.Nayak, Tycho.Andersen,
Nathan.Fontenot, jackyli, pgonda, rientjes, jacobhxu, xin,
pawan.kumar.gupta, babu.moger, dyoung, nikunj, john.allen, darwi,
linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <219ebbd57ac1d99fc5ea055431f7a8396021c2c2.1772486459.git.ashish.kalra@amd.com>
On 3/2/26 13:35, Ashish Kalra wrote:
> From: Ashish Kalra <ashish.kalra@amd.com>
>
> Add a flag indicating whether RMPOPT instruction is supported.
>
> RMPOPT is a new instruction designed to minimize the performance
> overhead of RMP checks on the hypervisor and on non-SNP guests by
> allowing RMP checks to be skipped when 1G regions of memory are known
> not to contain any SEV-SNP guest memory.
>
> For more information on the RMPOPT instruction, see the AMD64 RMPOPT
> technical documentation. [1]
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
^ permalink raw reply
* Re: [PATCH v2 2/7] x86/sev: add support for enabling RMPOPT
From: Dave Hansen @ 2026-03-02 23:00 UTC (permalink / raw)
To: Kalra, Ashish, tglx, mingo, bp, dave.hansen, x86, hpa, seanjc,
peterz, thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, KPrateek.Nayak, Tycho.Andersen,
Nathan.Fontenot, jackyli, pgonda, rientjes, jacobhxu, xin,
pawan.kumar.gupta, babu.moger, dyoung, nikunj, john.allen, darwi,
linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <6a50d050-f602-43fd-a44a-cecedd9823eb@amd.com>
On 3/2/26 14:55, Kalra, Ashish wrote:
>> What's wrong with:
>>
>> u64 rmpopt_base = pa_start | MSR_AMD64_RMPOPT_ENABLE;
>> ...
>> for_each_online_cpu(cpu)
>> wrmsrq_on_cpu(cpu, MSR_AMD64_RMPOPT_BASE, rmpopt_base);
>>
>> Then there's at least no ugly casting.
>>
> RMOPT_BASE MSRs don't need to be set serially, therefore, by
> using the cpu_online_mask and on_each_cpu_mask(), we can setup the MSRs
> concurrently and in parallel. Using for_each_online_cpu() will be slower than
> doing on_each_cpu_mask() as it doesn't send IPIs in parallel, right.
If that's the case and you *need* performance, then please go add a
wrmsrq_on_cpumask() function to do things in parallel.
^ permalink raw reply
* Re: [PATCH v2 3/7] x86/sev: add support for RMPOPT instruction
From: Dave Hansen @ 2026-03-02 22:57 UTC (permalink / raw)
To: Ashish Kalra, tglx, mingo, bp, dave.hansen, x86, hpa, seanjc,
peterz, thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, KPrateek.Nayak, Tycho.Andersen,
Nathan.Fontenot, jackyli, pgonda, rientjes, jacobhxu, xin,
pawan.kumar.gupta, babu.moger, dyoung, nikunj, john.allen, darwi,
linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <8dc0198f1261f5ae4b16388fc1ffad5ddb3895f9.1772486459.git.ashish.kalra@amd.com>
That subject could use a wee bit of work.
I'd probably talk about this adding a new kernel thread that does the
optimizations asynchronously.
On 3/2/26 13:36, Ashish Kalra wrote:
> From: Ashish Kalra <ashish.kalra@amd.com>
>
> As SEV-SNP is enabled by default on boot when an RMP table is
> allocated by BIOS, the hypervisor and non-SNP guests are subject to
> RMP write checks to provide integrity of SNP guest memory.
>
> RMPOPT is a new instruction that minimizes the performance overhead of
> RMP checks on the hypervisor and on non-SNP guests by allowing RMP
> checks to be skipped for 1GB regions of memory that are known not to
> contain any SEV-SNP guest memory.
>
> Enable RMPOPT optimizations globally for all system RAM at RMP
> initialization time. RMP checks can initially be skipped for 1GB memory
> ranges that do not contain SEV-SNP guest memory (excluding preassigned
> pages such as the RMP table and firmware pages). As SNP guests are
> launched, RMPUPDATE will disable the corresponding RMPOPT optimizations.
This is heavy on the "what" and light on the "why" and "how".
> diff --git a/arch/x86/virt/svm/sev.c b/arch/x86/virt/svm/sev.c
> index 405199c2f563..c99270dfe3b3 100644
> --- a/arch/x86/virt/svm/sev.c
> +++ b/arch/x86/virt/svm/sev.c
> @@ -19,6 +19,7 @@
> #include <linux/iommu.h>
> #include <linux/amd-iommu.h>
> #include <linux/nospec.h>
> +#include <linux/kthread.h>
>
> #include <asm/sev.h>
> #include <asm/processor.h>
> @@ -122,6 +123,13 @@ static u64 rmp_cfg;
>
> static u64 probed_rmp_base, probed_rmp_size;
>
> +enum rmpopt_function {
> + RMPOPT_FUNC_VERIFY_AND_REPORT_STATUS,
> + RMPOPT_FUNC_REPORT_STATUS
> +};
Shouldn't these go by the instruction definition?
You could even call it rmpopt_rcx or something.
> +static struct task_struct *rmpopt_task;
> +
> static LIST_HEAD(snp_leaked_pages_list);
> static DEFINE_SPINLOCK(snp_leaked_pages_list_lock);
>
> @@ -500,6 +508,61 @@ static bool __init setup_rmptable(void)
> }
> }
>
> +/*
> + * 'val' is a system physical address aligned to 1GB OR'ed with
> + * a function selection. Currently supported functions are 0
> + * (verify and report status) and 1 (report status).
> + */
> +static void rmpopt(void *val)
> +{
> + asm volatile(".byte 0xf2, 0x0f, 0x01, 0xfc"
> + : : "a" ((u64)val & PUD_MASK), "c" ((u64)val & 0x1)
> + : "memory", "cc");
> +}
Doesn't this belong in:
arch/x86/include/asm/special_insns.h
Also, it's not reporting *any* status here, right? So why even talk
about it if the kernel isn't doing any status checks? It just makes it
more confusing.
> +static int rmpopt_kthread(void *__unused)
> +{
> + phys_addr_t pa_start, pa_end;
> +
> + pa_start = ALIGN_DOWN(PFN_PHYS(min_low_pfn), PUD_SIZE);
> + pa_end = ALIGN(PFN_PHYS(max_pfn), PUD_SIZE);
Needs vertical alignment:
pa_start = ALIGN_DOWN(PFN_PHYS(min_low_pfn), PUD_SIZE);
pa_end = ALIGN( PFN_PHYS(max_pfn), PUD_SIZE);
Nit: the architecture says "1GB" regions, not PUD_SIZE. If we ever got
fancy and changed the page tables, this code would break. Why make it
harder on ourselves than it has to be?
> + /* Limit memory scanning to the first 2 TB of RAM */
> + pa_end = (pa_end - pa_start) <= SZ_2T ? pa_end : pa_start + SZ_2T;
That's a rather unfortunate use of ternary form. Isn't this a billion
times more clear?
if (pa_end - pa_start > SZ_2T)
pa_end = pa_start + SZ_2T;
> + while (!kthread_should_stop()) {
> + phys_addr_t pa;
> +
> + pr_info("RMP optimizations enabled on physical address range @1GB alignment [0x%016llx - 0x%016llx]\n",
> + pa_start, pa_end);
This isn't really enabling optimizations. It's trying to enable them,
right? It might fall on its face and fail every time, right?
> + /*
> + * RMPOPT optimizations skip RMP checks at 1GB granularity if this range of
> + * memory does not contain any SNP guest memory.
> + */
> + for (pa = pa_start; pa < pa_end; pa += PUD_SIZE) {
> + /* Bit zero passes the function to the RMPOPT instruction. */
> + on_each_cpu_mask(cpu_online_mask, rmpopt,
> + (void *)(pa | RMPOPT_FUNC_VERIFY_AND_REPORT_STATUS),
> + true);
> +
> + /* Give a chance for other threads to run */
> + cond_resched();
> + }
Could you also put together some proper helpers, please? The
lowest-level helper should look a lot like the instruction reference:
void __rmpopt(u64 rax, u64 rcx)
{
asm volatile(".byte 0xf2, 0x0f, 0x01, 0xfc"
: : "a" (rax), "c" (rcx)
: "memory", "cc");
}
Then you can have a higher-level instruction that shows how you convert
the logical things "physical address" and "rmpopt_function" into the
register arguments:
void rmpopt(unsigned long pa)
{
u64 rax = ALIGN_DOWN(pa & SZ_1GB);
u64 rcx = RMPOPT_FUNC_VERIFY_AND_REPORT_STATUS;
__rmpopt(rax, rcx);
}
There's no need right now to pack and unpack rax/rcx from a pointer. Why
even bother when rcx is a fixed value?
> + set_current_state(TASK_INTERRUPTIBLE);
> + schedule();
> + }
> +
> + return 0;
> +}
> +
> +static void rmpopt_all_physmem(void)
> +{
> + if (rmpopt_task)
> + wake_up_process(rmpopt_task);
> +}
Wait a sec, doesn't this just run all the time? It'll be doing an RMPOPT
on some forever.
^ permalink raw reply
* Re: [PATCH v2 2/7] x86/sev: add support for enabling RMPOPT
From: Kalra, Ashish @ 2026-03-02 22:55 UTC (permalink / raw)
To: Dave Hansen, tglx, mingo, bp, dave.hansen, x86, hpa, seanjc,
peterz, thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, KPrateek.Nayak, Tycho.Andersen,
Nathan.Fontenot, jackyli, pgonda, rientjes, jacobhxu, xin,
pawan.kumar.gupta, babu.moger, dyoung, nikunj, john.allen, darwi,
linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <b5a44f79-8f99-4e61-aaa2-e8aec6f0cf69@intel.com>
Hello Dave,
On 3/2/2026 4:32 PM, Dave Hansen wrote:
>> +static __init void configure_and_enable_rmpopt(void)
>> +{
>> + phys_addr_t pa_start = ALIGN_DOWN(PFN_PHYS(min_low_pfn), PUD_SIZE);
>> +
>> + if (!cpu_feature_enabled(X86_FEATURE_RMPOPT)) {
>> + pr_debug("RMPOPT not supported on this platform\n");
>> + return;
>> + }
>> +
>> + if (!cc_platform_has(CC_ATTR_HOST_SEV_SNP)) {
>> + pr_debug("RMPOPT optimizations not enabled as SNP support is not enabled\n");
>> + return;
>> + }
>
> To be honest, I think those two are just plain noise ^^.
They are basically pr_debug's, so won't really cause noise generally.
>
>> + if (!(rmp_cfg & MSR_AMD64_SEG_RMP_ENABLED)) {
>> + pr_info("RMPOPT optimizations not enabled, segmented RMP required\n");
>> + return;
>> + }
>> +
>> + /*
>> + * Per-CPU RMPOPT tables support at most 2 TB of addressable memory for RMP optimizations.
>> + *
>> + * Set per-core RMPOPT base to min_low_pfn to enable RMP optimization for
>> + * up to 2TB of system RAM on all CPUs.
>> + */
>
> Please at least be consistent with your comments. This is both over 80
> columns *and* not even consistent in the two sentences.
Sure.
>
>> + on_each_cpu_mask(cpu_online_mask, __configure_rmpopt, (void *)pa_start, true);
>> +}
>
> What's wrong with:
>
> u64 rmpopt_base = pa_start | MSR_AMD64_RMPOPT_ENABLE;
> ...
> for_each_online_cpu(cpu)
> wrmsrq_on_cpu(cpu, MSR_AMD64_RMPOPT_BASE, rmpopt_base);
>
> Then there's at least no ugly casting.
>
RMOPT_BASE MSRs don't need to be set serially, therefore, by
using the cpu_online_mask and on_each_cpu_mask(), we can setup the MSRs
concurrently and in parallel. Using for_each_online_cpu() will be slower than
doing on_each_cpu_mask() as it doesn't send IPIs in parallel, right.
Thanks,
Ashish
^ permalink raw reply
* Re: [PATCH v2 2/7] x86/sev: add support for enabling RMPOPT
From: Dave Hansen @ 2026-03-02 22:33 UTC (permalink / raw)
To: Ashish Kalra, tglx, mingo, bp, dave.hansen, x86, hpa, seanjc,
peterz, thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, KPrateek.Nayak, Tycho.Andersen,
Nathan.Fontenot, jackyli, pgonda, rientjes, jacobhxu, xin,
pawan.kumar.gupta, babu.moger, dyoung, nikunj, john.allen, darwi,
linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <85aec55af41957678d214e9629eb6249b064fa87.1772486459.git.ashish.kalra@amd.com>
Oh, and:
[PATCH v2 2/7] x86/sev: add support for enabling RMPOPT
^ Capitalize this, please
^ permalink raw reply
* Re: [PATCH v2 2/7] x86/sev: add support for enabling RMPOPT
From: Dave Hansen @ 2026-03-02 22:32 UTC (permalink / raw)
To: Ashish Kalra, tglx, mingo, bp, dave.hansen, x86, hpa, seanjc,
peterz, thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, KPrateek.Nayak, Tycho.Andersen,
Nathan.Fontenot, jackyli, pgonda, rientjes, jacobhxu, xin,
pawan.kumar.gupta, babu.moger, dyoung, nikunj, john.allen, darwi,
linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <85aec55af41957678d214e9629eb6249b064fa87.1772486459.git.ashish.kalra@amd.com>
On 3/2/26 13:35, Ashish Kalra wrote:
> The new RMPOPT instruction sets bits in a per-CPU RMPOPT table, which
> indicates whether specific 1GB physical memory regions contain SEV-SNP
> guest memory.
Honestly, this is an implementation detail that we don't need to know
about in the kernel. It's also not even factually correct. The
instruction _might_ not set any bits, either because there is SEV-SNP
memory or because it's being run in query mode.
The new RMPOPT instruction helps manage per-CPU RMP optimization
structures inside the CPU. It takes a 1GB-aligned physical
address and either returns the status of the optimizations or
tries to enable the optimizations.
> Per-CPU RMPOPT tables support at most 2 TB of addressable memory for
> RMP optimizations.
>
> Initialize the per-CPU RMPOPT table base to the starting physical
> address. This enables RMP optimization for up to 2 TB of system RAM on
> all CPUs.
The reset looks good.
> diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
> index da5275d8eda6..8e7da03abd5b 100644
> --- a/arch/x86/include/asm/msr-index.h
> +++ b/arch/x86/include/asm/msr-index.h
> @@ -753,6 +753,9 @@
> #define MSR_AMD64_SEG_RMP_ENABLED_BIT 0
> #define MSR_AMD64_SEG_RMP_ENABLED BIT_ULL(MSR_AMD64_SEG_RMP_ENABLED_BIT)
> #define MSR_AMD64_RMP_SEGMENT_SHIFT(x) (((x) & GENMASK_ULL(13, 8)) >> 8)
> +#define MSR_AMD64_RMPOPT_BASE 0xc0010139
> +#define MSR_AMD64_RMPOPT_ENABLE_BIT 0
> +#define MSR_AMD64_RMPOPT_ENABLE BIT_ULL(MSR_AMD64_RMPOPT_ENABLE_BIT)
>
> #define MSR_SVSM_CAA 0xc001f000
>
> diff --git a/arch/x86/virt/svm/sev.c b/arch/x86/virt/svm/sev.c
> index a4f3a364fb65..405199c2f563 100644
> --- a/arch/x86/virt/svm/sev.c
> +++ b/arch/x86/virt/svm/sev.c
> @@ -500,6 +500,41 @@ static bool __init setup_rmptable(void)
> }
> }
>
> +static void __configure_rmpopt(void *val)
> +{
> + u64 rmpopt_base = ((u64)val & PUD_MASK) | MSR_AMD64_RMPOPT_ENABLE;
> +
> + wrmsrq(MSR_AMD64_RMPOPT_BASE, rmpopt_base);
> +}
> +
> +static __init void configure_and_enable_rmpopt(void)
> +{
> + phys_addr_t pa_start = ALIGN_DOWN(PFN_PHYS(min_low_pfn), PUD_SIZE);
> +
> + if (!cpu_feature_enabled(X86_FEATURE_RMPOPT)) {
> + pr_debug("RMPOPT not supported on this platform\n");
> + return;
> + }
> +
> + if (!cc_platform_has(CC_ATTR_HOST_SEV_SNP)) {
> + pr_debug("RMPOPT optimizations not enabled as SNP support is not enabled\n");
> + return;
> + }
To be honest, I think those two are just plain noise ^^.
> + if (!(rmp_cfg & MSR_AMD64_SEG_RMP_ENABLED)) {
> + pr_info("RMPOPT optimizations not enabled, segmented RMP required\n");
> + return;
> + }
> +
> + /*
> + * Per-CPU RMPOPT tables support at most 2 TB of addressable memory for RMP optimizations.
> + *
> + * Set per-core RMPOPT base to min_low_pfn to enable RMP optimization for
> + * up to 2TB of system RAM on all CPUs.
> + */
Please at least be consistent with your comments. This is both over 80
columns *and* not even consistent in the two sentences.
> + on_each_cpu_mask(cpu_online_mask, __configure_rmpopt, (void *)pa_start, true);
> +}
What's wrong with:
u64 rmpopt_base = pa_start | MSR_AMD64_RMPOPT_ENABLE;
...
for_each_online_cpu(cpu)
wrmsrq_on_cpu(cpu, MSR_AMD64_RMPOPT_BASE, rmpopt_base);
Then there's at least no ugly casting.
^ permalink raw reply
* [PATCH v2 7/7] x86/sev: Add debugfs support for RMPOPT
From: Ashish Kalra @ 2026-03-02 21:37 UTC (permalink / raw)
To: tglx, mingo, bp, dave.hansen, x86, hpa, seanjc, peterz,
thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, KPrateek.Nayak, Tycho.Andersen,
Nathan.Fontenot, jackyli, pgonda, rientjes, jacobhxu, xin,
pawan.kumar.gupta, babu.moger, dyoung, nikunj, john.allen, darwi,
linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <cover.1772486459.git.ashish.kalra@amd.com>
From: Ashish Kalra <ashish.kalra@amd.com>
Add a debugfs interface to report per-CPU RMPOPT status across all
system RAM.
To dump the per-CPU RMPOPT status for all system RAM:
/sys/kernel/debug/rmpopt# cat rmpopt-table
Memory @ 0GB: CPU(s): none
Memory @ 1GB: CPU(s): none
Memory @ 2GB: CPU(s): 0-1023
Memory @ 3GB: CPU(s): 0-1023
Memory @ 4GB: CPU(s): none
Memory @ 5GB: CPU(s): 0-1023
Memory @ 6GB: CPU(s): 0-1023
Memory @ 7GB: CPU(s): 0-1023
...
Memory @1025GB: CPU(s): 0-1023
Memory @1026GB: CPU(s): 0-1023
Memory @1027GB: CPU(s): 0-1023
Memory @1028GB: CPU(s): 0-1023
Memory @1029GB: CPU(s): 0-1023
Memory @1030GB: CPU(s): 0-1023
Memory @1031GB: CPU(s): 0-1023
Memory @1032GB: CPU(s): 0-1023
Memory @1033GB: CPU(s): 0-1023
Memory @1034GB: CPU(s): 0-1023
Memory @1035GB: CPU(s): 0-1023
Memory @1036GB: CPU(s): 0-1023
Memory @1037GB: CPU(s): 0-1023
Memory @1038GB: CPU(s): none
Suggested-by: Thomas Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Ashish Kalra <ashish.kalra@amd.com>
---
arch/x86/virt/svm/sev.c | 101 +++++++++++++++++++++++++++++++++++++++-
1 file changed, 100 insertions(+), 1 deletion(-)
diff --git a/arch/x86/virt/svm/sev.c b/arch/x86/virt/svm/sev.c
index 4dd5a525ad32..49bd7ba76169 100644
--- a/arch/x86/virt/svm/sev.c
+++ b/arch/x86/virt/svm/sev.c
@@ -20,6 +20,8 @@
#include <linux/amd-iommu.h>
#include <linux/nospec.h>
#include <linux/kthread.h>
+#include <linux/debugfs.h>
+#include <linux/seq_file.h>
#include <asm/sev.h>
#include <asm/processor.h>
@@ -135,6 +137,13 @@ static DEFINE_SPINLOCK(snp_leaked_pages_list_lock);
static unsigned long snp_nr_leaked_pages;
+static cpumask_t rmpopt_cpumask;
+static struct dentry *rmpopt_debugfs;
+
+struct seq_paddr {
+ phys_addr_t next_seq_paddr;
+};
+
#undef pr_fmt
#define pr_fmt(fmt) "SEV-SNP: " fmt
@@ -515,9 +524,14 @@ static bool __init setup_rmptable(void)
*/
static void rmpopt(void *val)
{
+ bool optimized;
+
asm volatile(".byte 0xf2, 0x0f, 0x01, 0xfc"
- : : "a" ((u64)val & PUD_MASK), "c" ((u64)val & 0x1)
+ : "=@ccc" (optimized)
+ : "a" ((u64)val & PUD_MASK), "c" ((u64)val & 0x1)
: "memory", "cc");
+
+ assign_cpu(smp_processor_id(), &rmpopt_cpumask, optimized);
}
static int rmpopt_kthread(void *__unused)
@@ -563,6 +577,89 @@ static void rmpopt_all_physmem(void)
wake_up_process(rmpopt_task);
}
+/*
+ * start() can be called multiple times if allocated buffer has overflowed
+ * and bigger buffer is allocated.
+ */
+static void *rmpopt_table_seq_start(struct seq_file *seq, loff_t *pos)
+{
+ phys_addr_t end_paddr = ALIGN(PFN_PHYS(max_pfn), PUD_SIZE);
+ struct seq_paddr *p = seq->private;
+
+ if (*pos == 0) {
+ p->next_seq_paddr = ALIGN_DOWN(PFN_PHYS(min_low_pfn), PUD_SIZE);
+ return &p->next_seq_paddr;
+ }
+
+ if (p->next_seq_paddr == end_paddr)
+ return NULL;
+
+ return &p->next_seq_paddr;
+}
+
+static void *rmpopt_table_seq_next(struct seq_file *seq, void *v, loff_t *pos)
+{
+ phys_addr_t end_paddr = ALIGN(PFN_PHYS(max_pfn), PUD_SIZE);
+ phys_addr_t *curr_paddr = v;
+
+ (*pos)++;
+ if (*curr_paddr == end_paddr)
+ return NULL;
+ *curr_paddr += PUD_SIZE;
+
+ return curr_paddr;
+}
+
+static void rmpopt_table_seq_stop(struct seq_file *seq, void *v)
+{
+}
+
+static int rmpopt_table_seq_show(struct seq_file *seq, void *v)
+{
+ phys_addr_t *curr_paddr = v;
+
+ seq_printf(seq, "Memory @%3lluGB: ", *curr_paddr >> PUD_SHIFT);
+
+ cpumask_clear(&rmpopt_cpumask);
+ on_each_cpu_mask(cpu_online_mask, rmpopt,
+ (void *)(*curr_paddr | RMPOPT_FUNC_REPORT_STATUS),
+ true);
+
+ if (cpumask_empty(&rmpopt_cpumask))
+ seq_puts(seq, "CPU(s): none\n");
+ else
+ seq_printf(seq, "CPU(s): %*pbl\n", cpumask_pr_args(&rmpopt_cpumask));
+
+ return 0;
+}
+
+static const struct seq_operations rmpopt_table_seq_ops = {
+ .start = rmpopt_table_seq_start,
+ .next = rmpopt_table_seq_next,
+ .stop = rmpopt_table_seq_stop,
+ .show = rmpopt_table_seq_show
+};
+
+static int rmpopt_table_open(struct inode *inode, struct file *file)
+{
+ return seq_open_private(file, &rmpopt_table_seq_ops, sizeof(struct seq_paddr));
+}
+
+static const struct file_operations rmpopt_table_fops = {
+ .open = rmpopt_table_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = seq_release_private,
+};
+
+static void rmpopt_debugfs_setup(void)
+{
+ rmpopt_debugfs = debugfs_create_dir("rmpopt", arch_debugfs_dir);
+
+ debugfs_create_file("rmpopt-table", 0444, rmpopt_debugfs,
+ NULL, &rmpopt_table_fops);
+}
+
static void __configure_rmpopt(void *val)
{
u64 rmpopt_base = ((u64)val & PUD_MASK) | MSR_AMD64_RMPOPT_ENABLE;
@@ -611,6 +708,8 @@ static __init void configure_and_enable_rmpopt(void)
* optimizations on all physical memory.
*/
rmpopt_all_physmem();
+
+ rmpopt_debugfs_setup();
}
/*
--
2.43.0
^ permalink raw reply related
* [PATCH v2 6/7] KVM: SEV: Implement SEV-SNP specific guest cleanup
From: Ashish Kalra @ 2026-03-02 21:37 UTC (permalink / raw)
To: tglx, mingo, bp, dave.hansen, x86, hpa, seanjc, peterz,
thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, KPrateek.Nayak, Tycho.Andersen,
Nathan.Fontenot, jackyli, pgonda, rientjes, jacobhxu, xin,
pawan.kumar.gupta, babu.moger, dyoung, nikunj, john.allen, darwi,
linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <cover.1772486459.git.ashish.kalra@amd.com>
From: Ashish Kalra <ashish.kalra@amd.com>
Implement the arch-specific cleanup for SEV-SNP via the
kvm_gmem_cleanup() hook. Use this interface to re-enable RMP
optimizations during guest shutdown.
Signed-off-by: Ashish Kalra <ashish.kalra@amd.com>
---
arch/x86/kvm/Kconfig | 1 +
arch/x86/kvm/svm/sev.c | 9 +++++++++
arch/x86/kvm/svm/svm.c | 1 +
arch/x86/kvm/svm/svm.h | 2 ++
4 files changed, 13 insertions(+)
diff --git a/arch/x86/kvm/Kconfig b/arch/x86/kvm/Kconfig
index d916bd766c94..fdfdb7ac6a45 100644
--- a/arch/x86/kvm/Kconfig
+++ b/arch/x86/kvm/Kconfig
@@ -164,6 +164,7 @@ config KVM_AMD_SEV
select HAVE_KVM_ARCH_GMEM_PREPARE
select HAVE_KVM_ARCH_GMEM_INVALIDATE
select HAVE_KVM_ARCH_GMEM_POPULATE
+ select HAVE_KVM_ARCH_GMEM_CLEANUP
help
Provides support for launching encrypted VMs which use Secure
Encrypted Virtualization (SEV), Secure Encrypted Virtualization with
diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
index 3f9c1aa39a0a..4c206e9f70cd 100644
--- a/arch/x86/kvm/svm/sev.c
+++ b/arch/x86/kvm/svm/sev.c
@@ -5109,6 +5109,15 @@ int sev_gmem_max_mapping_level(struct kvm *kvm, kvm_pfn_t pfn, bool is_private)
return level;
}
+void sev_gmem_cleanup(void)
+{
+ /*
+ * Re-enable RMP optimizations once all guest pages are
+ * converted back to shared following guest shutdown.
+ */
+ snp_perform_rmp_optimization();
+}
+
struct vmcb_save_area *sev_decrypt_vmsa(struct kvm_vcpu *vcpu)
{
struct vcpu_svm *svm = to_svm(vcpu);
diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
index 8f8bc863e214..46526ab9ab92 100644
--- a/arch/x86/kvm/svm/svm.c
+++ b/arch/x86/kvm/svm/svm.c
@@ -5260,6 +5260,7 @@ struct kvm_x86_ops svm_x86_ops __initdata = {
.gmem_prepare = sev_gmem_prepare,
.gmem_invalidate = sev_gmem_invalidate,
.gmem_max_mapping_level = sev_gmem_max_mapping_level,
+ .gmem_cleanup = sev_gmem_cleanup,
};
/*
diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h
index ebd7b36b1ceb..443c29c23a6a 100644
--- a/arch/x86/kvm/svm/svm.h
+++ b/arch/x86/kvm/svm/svm.h
@@ -896,6 +896,7 @@ void sev_handle_rmp_fault(struct kvm_vcpu *vcpu, gpa_t gpa, u64 error_code);
int sev_gmem_prepare(struct kvm *kvm, kvm_pfn_t pfn, gfn_t gfn, int max_order);
void sev_gmem_invalidate(kvm_pfn_t start, kvm_pfn_t end);
int sev_gmem_max_mapping_level(struct kvm *kvm, kvm_pfn_t pfn, bool is_private);
+void sev_gmem_cleanup(void);
struct vmcb_save_area *sev_decrypt_vmsa(struct kvm_vcpu *vcpu);
void sev_free_decrypted_vmsa(struct kvm_vcpu *vcpu, struct vmcb_save_area *vmsa);
#else
@@ -928,6 +929,7 @@ static inline int sev_gmem_max_mapping_level(struct kvm *kvm, kvm_pfn_t pfn, boo
{
return 0;
}
+static inline void sev_gmem_cleanup(void) {}
static inline struct vmcb_save_area *sev_decrypt_vmsa(struct kvm_vcpu *vcpu)
{
--
2.43.0
^ permalink raw reply related
* [PATCH v2 5/7] KVM: guest_memfd: Add cleanup interface for guest teardown
From: Ashish Kalra @ 2026-03-02 21:36 UTC (permalink / raw)
To: tglx, mingo, bp, dave.hansen, x86, hpa, seanjc, peterz,
thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, KPrateek.Nayak, Tycho.Andersen,
Nathan.Fontenot, jackyli, pgonda, rientjes, jacobhxu, xin,
pawan.kumar.gupta, babu.moger, dyoung, nikunj, john.allen, darwi,
linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <cover.1772486459.git.ashish.kalra@amd.com>
From: Ashish Kalra <ashish.kalra@amd.com>
Introduce kvm_arch_gmem_cleanup() to perform architecture-specific
cleanups when the last file descriptor for the guest_memfd inode is
closed. This typically occurs during guest shutdown and termination
and allows for final resource release.
Signed-off-by: Ashish Kalra <ashish.kalra@amd.com>
---
arch/x86/include/asm/kvm-x86-ops.h | 1 +
arch/x86/include/asm/kvm_host.h | 1 +
arch/x86/kvm/x86.c | 7 +++++++
include/linux/kvm_host.h | 4 ++++
virt/kvm/Kconfig | 4 ++++
virt/kvm/guest_memfd.c | 8 ++++++++
6 files changed, 25 insertions(+)
diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-x86-ops.h
index de709fb5bd76..ebbecd0c9e4f 100644
--- a/arch/x86/include/asm/kvm-x86-ops.h
+++ b/arch/x86/include/asm/kvm-x86-ops.h
@@ -148,6 +148,7 @@ KVM_X86_OP_OPTIONAL(alloc_apic_backing_page)
KVM_X86_OP_OPTIONAL_RET0(gmem_prepare)
KVM_X86_OP_OPTIONAL_RET0(gmem_max_mapping_level)
KVM_X86_OP_OPTIONAL(gmem_invalidate)
+KVM_X86_OP_OPTIONAL(gmem_cleanup)
#undef KVM_X86_OP
#undef KVM_X86_OP_OPTIONAL
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index ff07c45e3c73..7894cf791fef 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -1962,6 +1962,7 @@ struct kvm_x86_ops {
int (*gmem_prepare)(struct kvm *kvm, kvm_pfn_t pfn, gfn_t gfn, int max_order);
void (*gmem_invalidate)(kvm_pfn_t start, kvm_pfn_t end);
int (*gmem_max_mapping_level)(struct kvm *kvm, kvm_pfn_t pfn, bool is_private);
+ void (*gmem_cleanup)(void);
};
struct kvm_x86_nested_ops {
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 3fb64905d190..d992848942c3 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -14080,6 +14080,13 @@ void kvm_arch_gmem_invalidate(kvm_pfn_t start, kvm_pfn_t end)
kvm_x86_call(gmem_invalidate)(start, end);
}
#endif
+
+#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CLEANUP
+void kvm_arch_gmem_cleanup(void)
+{
+ kvm_x86_call(gmem_cleanup)();
+}
+#endif
#endif
int kvm_spec_ctrl_test_value(u64 value)
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index dde605cb894e..b14143c427eb 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -2607,6 +2607,10 @@ long kvm_gmem_populate(struct kvm *kvm, gfn_t gfn, void __user *src, long npages
void kvm_arch_gmem_invalidate(kvm_pfn_t start, kvm_pfn_t end);
#endif
+#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CLEANUP
+void kvm_arch_gmem_cleanup(void);
+#endif
+
#ifdef CONFIG_KVM_GENERIC_PRE_FAULT_MEMORY
long kvm_arch_vcpu_pre_fault_memory(struct kvm_vcpu *vcpu,
struct kvm_pre_fault_memory *range);
diff --git a/virt/kvm/Kconfig b/virt/kvm/Kconfig
index 267c7369c765..9072ec12d5e7 100644
--- a/virt/kvm/Kconfig
+++ b/virt/kvm/Kconfig
@@ -125,3 +125,7 @@ config HAVE_KVM_ARCH_GMEM_INVALIDATE
config HAVE_KVM_ARCH_GMEM_POPULATE
bool
depends on KVM_GUEST_MEMFD
+
+config HAVE_KVM_ARCH_GMEM_CLEANUP
+ bool
+ depends on KVM_GUEST_MEMFD
diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c
index 017d84a7adf3..2724dd1099f2 100644
--- a/virt/kvm/guest_memfd.c
+++ b/virt/kvm/guest_memfd.c
@@ -955,6 +955,14 @@ static void kvm_gmem_destroy_inode(struct inode *inode)
static void kvm_gmem_free_inode(struct inode *inode)
{
+#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_CLEANUP
+ /*
+ * Finalize cleanup for the inode once the last guest_memfd
+ * reference is released. This usually occurs after guest
+ * termination.
+ */
+ kvm_arch_gmem_cleanup();
+#endif
kmem_cache_free(kvm_gmem_inode_cachep, GMEM_I(inode));
}
--
2.43.0
^ permalink raw reply related
* [PATCH v2 4/7] x86/sev: Add interface to re-enable RMP optimizations.
From: Ashish Kalra @ 2026-03-02 21:36 UTC (permalink / raw)
To: tglx, mingo, bp, dave.hansen, x86, hpa, seanjc, peterz,
thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, KPrateek.Nayak, Tycho.Andersen,
Nathan.Fontenot, jackyli, pgonda, rientjes, jacobhxu, xin,
pawan.kumar.gupta, babu.moger, dyoung, nikunj, john.allen, darwi,
linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <cover.1772486459.git.ashish.kalra@amd.com>
From: Ashish Kalra <ashish.kalra@amd.com>
RMPOPT table is a per-processor table which indicates if 1GB regions of
physical memory are entirely hypervisor-owned or not.
When performing host memory accesses in hypervisor mode as well as
non-SNP guest mode, the processor may consult the RMPOPT table to
potentially skip an RMP access and improve performance.
Events such as RMPUPDATE or SNP_INIT can clear RMP optimizations. Add
an interface to re-enable those optimizations.
Signed-off-by: Ashish Kalra <ashish.kalra@amd.com>
---
arch/x86/include/asm/sev.h | 2 ++
arch/x86/virt/svm/sev.c | 17 +++++++++++++++++
drivers/crypto/ccp/sev-dev.c | 4 ++++
3 files changed, 23 insertions(+)
diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h
index 0e6c0940100f..451fb2b2a0f7 100644
--- a/arch/x86/include/asm/sev.h
+++ b/arch/x86/include/asm/sev.h
@@ -657,6 +657,7 @@ int rmp_make_shared(u64 pfn, enum pg_level level);
void __snp_leak_pages(u64 pfn, unsigned int npages, bool dump_rmp);
void kdump_sev_callback(void);
void snp_fixup_e820_tables(void);
+int snp_perform_rmp_optimization(void);
static inline void snp_leak_pages(u64 pfn, unsigned int pages)
{
__snp_leak_pages(pfn, pages, true);
@@ -677,6 +678,7 @@ static inline void __snp_leak_pages(u64 pfn, unsigned int npages, bool dump_rmp)
static inline void snp_leak_pages(u64 pfn, unsigned int npages) {}
static inline void kdump_sev_callback(void) { }
static inline void snp_fixup_e820_tables(void) {}
+static inline int snp_perform_rmp_optimization(void) { return 0; }
#endif
#endif
diff --git a/arch/x86/virt/svm/sev.c b/arch/x86/virt/svm/sev.c
index c99270dfe3b3..4dd5a525ad32 100644
--- a/arch/x86/virt/svm/sev.c
+++ b/arch/x86/virt/svm/sev.c
@@ -1144,6 +1144,23 @@ int rmp_make_shared(u64 pfn, enum pg_level level)
}
EXPORT_SYMBOL_GPL(rmp_make_shared);
+int snp_perform_rmp_optimization(void)
+{
+ if (!cpu_feature_enabled(X86_FEATURE_RMPOPT))
+ return -EINVAL;
+
+ if (!cc_platform_has(CC_ATTR_HOST_SEV_SNP))
+ return -EINVAL;
+
+ if (!(rmp_cfg & MSR_AMD64_SEG_RMP_ENABLED))
+ return -EINVAL;
+
+ rmpopt_all_physmem();
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(snp_perform_rmp_optimization);
+
void __snp_leak_pages(u64 pfn, unsigned int npages, bool dump_rmp)
{
struct page *page = pfn_to_page(pfn);
diff --git a/drivers/crypto/ccp/sev-dev.c b/drivers/crypto/ccp/sev-dev.c
index 096f993974d1..d84178a232e0 100644
--- a/drivers/crypto/ccp/sev-dev.c
+++ b/drivers/crypto/ccp/sev-dev.c
@@ -1478,6 +1478,10 @@ static int __sev_snp_init_locked(int *error, unsigned int max_snp_asid)
}
snp_hv_fixed_pages_state_update(sev, HV_FIXED);
+
+ /* SNP_INIT clears the RMPOPT table, re-enable RMP optimizations */
+ snp_perform_rmp_optimization();
+
sev->snp_initialized = true;
dev_dbg(sev->dev, "SEV-SNP firmware initialized, SEV-TIO is %s\n",
data.tio_en ? "enabled" : "disabled");
--
2.43.0
^ permalink raw reply related
* [PATCH v2 3/7] x86/sev: add support for RMPOPT instruction
From: Ashish Kalra @ 2026-03-02 21:36 UTC (permalink / raw)
To: tglx, mingo, bp, dave.hansen, x86, hpa, seanjc, peterz,
thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, KPrateek.Nayak, Tycho.Andersen,
Nathan.Fontenot, jackyli, pgonda, rientjes, jacobhxu, xin,
pawan.kumar.gupta, babu.moger, dyoung, nikunj, john.allen, darwi,
linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <cover.1772486459.git.ashish.kalra@amd.com>
From: Ashish Kalra <ashish.kalra@amd.com>
As SEV-SNP is enabled by default on boot when an RMP table is
allocated by BIOS, the hypervisor and non-SNP guests are subject to
RMP write checks to provide integrity of SNP guest memory.
RMPOPT is a new instruction that minimizes the performance overhead of
RMP checks on the hypervisor and on non-SNP guests by allowing RMP
checks to be skipped for 1GB regions of memory that are known not to
contain any SEV-SNP guest memory.
Enable RMPOPT optimizations globally for all system RAM at RMP
initialization time. RMP checks can initially be skipped for 1GB memory
ranges that do not contain SEV-SNP guest memory (excluding preassigned
pages such as the RMP table and firmware pages). As SNP guests are
launched, RMPUPDATE will disable the corresponding RMPOPT optimizations.
Suggested-by: Thomas Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Ashish Kalra <ashish.kalra@amd.com>
---
arch/x86/virt/svm/sev.c | 78 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 78 insertions(+)
diff --git a/arch/x86/virt/svm/sev.c b/arch/x86/virt/svm/sev.c
index 405199c2f563..c99270dfe3b3 100644
--- a/arch/x86/virt/svm/sev.c
+++ b/arch/x86/virt/svm/sev.c
@@ -19,6 +19,7 @@
#include <linux/iommu.h>
#include <linux/amd-iommu.h>
#include <linux/nospec.h>
+#include <linux/kthread.h>
#include <asm/sev.h>
#include <asm/processor.h>
@@ -122,6 +123,13 @@ static u64 rmp_cfg;
static u64 probed_rmp_base, probed_rmp_size;
+enum rmpopt_function {
+ RMPOPT_FUNC_VERIFY_AND_REPORT_STATUS,
+ RMPOPT_FUNC_REPORT_STATUS
+};
+
+static struct task_struct *rmpopt_task;
+
static LIST_HEAD(snp_leaked_pages_list);
static DEFINE_SPINLOCK(snp_leaked_pages_list_lock);
@@ -500,6 +508,61 @@ static bool __init setup_rmptable(void)
}
}
+/*
+ * 'val' is a system physical address aligned to 1GB OR'ed with
+ * a function selection. Currently supported functions are 0
+ * (verify and report status) and 1 (report status).
+ */
+static void rmpopt(void *val)
+{
+ asm volatile(".byte 0xf2, 0x0f, 0x01, 0xfc"
+ : : "a" ((u64)val & PUD_MASK), "c" ((u64)val & 0x1)
+ : "memory", "cc");
+}
+
+static int rmpopt_kthread(void *__unused)
+{
+ phys_addr_t pa_start, pa_end;
+
+ pa_start = ALIGN_DOWN(PFN_PHYS(min_low_pfn), PUD_SIZE);
+ pa_end = ALIGN(PFN_PHYS(max_pfn), PUD_SIZE);
+
+ /* Limit memory scanning to the first 2 TB of RAM */
+ pa_end = (pa_end - pa_start) <= SZ_2T ? pa_end : pa_start + SZ_2T;
+
+ while (!kthread_should_stop()) {
+ phys_addr_t pa;
+
+ pr_info("RMP optimizations enabled on physical address range @1GB alignment [0x%016llx - 0x%016llx]\n",
+ pa_start, pa_end);
+
+ /*
+ * RMPOPT optimizations skip RMP checks at 1GB granularity if this range of
+ * memory does not contain any SNP guest memory.
+ */
+ for (pa = pa_start; pa < pa_end; pa += PUD_SIZE) {
+ /* Bit zero passes the function to the RMPOPT instruction. */
+ on_each_cpu_mask(cpu_online_mask, rmpopt,
+ (void *)(pa | RMPOPT_FUNC_VERIFY_AND_REPORT_STATUS),
+ true);
+
+ /* Give a chance for other threads to run */
+ cond_resched();
+ }
+
+ set_current_state(TASK_INTERRUPTIBLE);
+ schedule();
+ }
+
+ return 0;
+}
+
+static void rmpopt_all_physmem(void)
+{
+ if (rmpopt_task)
+ wake_up_process(rmpopt_task);
+}
+
static void __configure_rmpopt(void *val)
{
u64 rmpopt_base = ((u64)val & PUD_MASK) | MSR_AMD64_RMPOPT_ENABLE;
@@ -533,6 +596,21 @@ static __init void configure_and_enable_rmpopt(void)
* up to 2TB of system RAM on all CPUs.
*/
on_each_cpu_mask(cpu_online_mask, __configure_rmpopt, (void *)pa_start, true);
+
+ rmpopt_task = kthread_create(rmpopt_kthread, NULL, "rmpopt_kthread");
+ if (IS_ERR(rmpopt_task)) {
+ pr_warn("Unable to start RMPOPT kernel thread\n");
+ rmpopt_task = NULL;
+ return;
+ }
+
+ pr_info("RMPOPT worker thread created with PID %d\n", task_pid_nr(rmpopt_task));
+
+ /*
+ * Once all per-CPU RMPOPT tables have been configured, enable RMPOPT
+ * optimizations on all physical memory.
+ */
+ rmpopt_all_physmem();
}
/*
--
2.43.0
^ permalink raw reply related
* [PATCH v2 2/7] x86/sev: add support for enabling RMPOPT
From: Ashish Kalra @ 2026-03-02 21:35 UTC (permalink / raw)
To: tglx, mingo, bp, dave.hansen, x86, hpa, seanjc, peterz,
thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, KPrateek.Nayak, Tycho.Andersen,
Nathan.Fontenot, jackyli, pgonda, rientjes, jacobhxu, xin,
pawan.kumar.gupta, babu.moger, dyoung, nikunj, john.allen, darwi,
linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <cover.1772486459.git.ashish.kalra@amd.com>
From: Ashish Kalra <ashish.kalra@amd.com>
The new RMPOPT instruction sets bits in a per-CPU RMPOPT table, which
indicates whether specific 1GB physical memory regions contain SEV-SNP
guest memory.
Per-CPU RMPOPT tables support at most 2 TB of addressable memory for
RMP optimizations.
Initialize the per-CPU RMPOPT table base to the starting physical
address. This enables RMP optimization for up to 2 TB of system RAM on
all CPUs.
Suggested-by: Thomas Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Ashish Kalra <ashish.kalra@amd.com>
---
arch/x86/include/asm/msr-index.h | 3 +++
arch/x86/virt/svm/sev.c | 37 ++++++++++++++++++++++++++++++++
2 files changed, 40 insertions(+)
diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index da5275d8eda6..8e7da03abd5b 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -753,6 +753,9 @@
#define MSR_AMD64_SEG_RMP_ENABLED_BIT 0
#define MSR_AMD64_SEG_RMP_ENABLED BIT_ULL(MSR_AMD64_SEG_RMP_ENABLED_BIT)
#define MSR_AMD64_RMP_SEGMENT_SHIFT(x) (((x) & GENMASK_ULL(13, 8)) >> 8)
+#define MSR_AMD64_RMPOPT_BASE 0xc0010139
+#define MSR_AMD64_RMPOPT_ENABLE_BIT 0
+#define MSR_AMD64_RMPOPT_ENABLE BIT_ULL(MSR_AMD64_RMPOPT_ENABLE_BIT)
#define MSR_SVSM_CAA 0xc001f000
diff --git a/arch/x86/virt/svm/sev.c b/arch/x86/virt/svm/sev.c
index a4f3a364fb65..405199c2f563 100644
--- a/arch/x86/virt/svm/sev.c
+++ b/arch/x86/virt/svm/sev.c
@@ -500,6 +500,41 @@ static bool __init setup_rmptable(void)
}
}
+static void __configure_rmpopt(void *val)
+{
+ u64 rmpopt_base = ((u64)val & PUD_MASK) | MSR_AMD64_RMPOPT_ENABLE;
+
+ wrmsrq(MSR_AMD64_RMPOPT_BASE, rmpopt_base);
+}
+
+static __init void configure_and_enable_rmpopt(void)
+{
+ phys_addr_t pa_start = ALIGN_DOWN(PFN_PHYS(min_low_pfn), PUD_SIZE);
+
+ if (!cpu_feature_enabled(X86_FEATURE_RMPOPT)) {
+ pr_debug("RMPOPT not supported on this platform\n");
+ return;
+ }
+
+ if (!cc_platform_has(CC_ATTR_HOST_SEV_SNP)) {
+ pr_debug("RMPOPT optimizations not enabled as SNP support is not enabled\n");
+ return;
+ }
+
+ if (!(rmp_cfg & MSR_AMD64_SEG_RMP_ENABLED)) {
+ pr_info("RMPOPT optimizations not enabled, segmented RMP required\n");
+ return;
+ }
+
+ /*
+ * Per-CPU RMPOPT tables support at most 2 TB of addressable memory for RMP optimizations.
+ *
+ * Set per-core RMPOPT base to min_low_pfn to enable RMP optimization for
+ * up to 2TB of system RAM on all CPUs.
+ */
+ on_each_cpu_mask(cpu_online_mask, __configure_rmpopt, (void *)pa_start, true);
+}
+
/*
* Do the necessary preparations which are verified by the firmware as
* described in the SNP_INIT_EX firmware command description in the SNP
@@ -555,6 +590,8 @@ int __init snp_rmptable_init(void)
skip_enable:
cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "x86/rmptable_init:online", __snp_enable, NULL);
+ configure_and_enable_rmpopt();
+
/*
* Setting crash_kexec_post_notifiers to 'true' to ensure that SNP panic
* notifier is invoked to do SNP IOMMU shutdown before kdump.
--
2.43.0
^ permalink raw reply related
* [PATCH v2 1/7] x86/cpufeatures: Add X86_FEATURE_AMD_RMPOPT feature flag
From: Ashish Kalra @ 2026-03-02 21:35 UTC (permalink / raw)
To: tglx, mingo, bp, dave.hansen, x86, hpa, seanjc, peterz,
thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, KPrateek.Nayak, Tycho.Andersen,
Nathan.Fontenot, jackyli, pgonda, rientjes, jacobhxu, xin,
pawan.kumar.gupta, babu.moger, dyoung, nikunj, john.allen, darwi,
linux-kernel, linux-crypto, kvm, linux-coco
In-Reply-To: <cover.1772486459.git.ashish.kalra@amd.com>
From: Ashish Kalra <ashish.kalra@amd.com>
Add a flag indicating whether RMPOPT instruction is supported.
RMPOPT is a new instruction designed to minimize the performance
overhead of RMP checks on the hypervisor and on non-SNP guests by
allowing RMP checks to be skipped when 1G regions of memory are known
not to contain any SEV-SNP guest memory.
For more information on the RMPOPT instruction, see the AMD64 RMPOPT
technical documentation. [1]
Link: https://docs.amd.com/v/u/en-US/69201_1.00_AMD64_RMPOPT_PUB [1]
Signed-off-by: Ashish Kalra <ashish.kalra@amd.com>
---
arch/x86/include/asm/cpufeatures.h | 2 +-
arch/x86/kernel/cpu/scattered.c | 1 +
2 files changed, 2 insertions(+), 1 deletion(-)
diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index dbe104df339b..bce1b2e2a35c 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -76,7 +76,7 @@
#define X86_FEATURE_K8 ( 3*32+ 4) /* Opteron, Athlon64 */
#define X86_FEATURE_ZEN5 ( 3*32+ 5) /* CPU based on Zen5 microarchitecture */
#define X86_FEATURE_ZEN6 ( 3*32+ 6) /* CPU based on Zen6 microarchitecture */
-/* Free ( 3*32+ 7) */
+#define X86_FEATURE_RMPOPT ( 3*32+ 7) /* Support for AMD RMPOPT instruction */
#define X86_FEATURE_CONSTANT_TSC ( 3*32+ 8) /* "constant_tsc" TSC ticks at a constant rate */
#define X86_FEATURE_UP ( 3*32+ 9) /* "up" SMP kernel running on UP */
#define X86_FEATURE_ART ( 3*32+10) /* "art" Always running timer (ART) */
diff --git a/arch/x86/kernel/cpu/scattered.c b/arch/x86/kernel/cpu/scattered.c
index 42c7eac0c387..7ac3818c4502 100644
--- a/arch/x86/kernel/cpu/scattered.c
+++ b/arch/x86/kernel/cpu/scattered.c
@@ -65,6 +65,7 @@ static const struct cpuid_bit cpuid_bits[] = {
{ X86_FEATURE_PERFMON_V2, CPUID_EAX, 0, 0x80000022, 0 },
{ X86_FEATURE_AMD_LBR_V2, CPUID_EAX, 1, 0x80000022, 0 },
{ X86_FEATURE_AMD_LBR_PMC_FREEZE, CPUID_EAX, 2, 0x80000022, 0 },
+ { X86_FEATURE_RMPOPT, CPUID_EDX, 0, 0x80000025, 0 },
{ X86_FEATURE_AMD_HTR_CORES, CPUID_EAX, 30, 0x80000026, 0 },
{ 0, 0, 0, 0, 0 }
};
--
2.43.0
^ permalink raw reply related
* [PATCH v2 0/7] Add RMPOPT support.
From: Ashish Kalra @ 2026-03-02 21:35 UTC (permalink / raw)
To: tglx, mingo, bp, dave.hansen, x86, hpa, seanjc, peterz,
thomas.lendacky, herbert, davem, ardb
Cc: pbonzini, aik, Michael.Roth, KPrateek.Nayak, Tycho.Andersen,
Nathan.Fontenot, jackyli, pgonda, rientjes, jacobhxu, xin,
pawan.kumar.gupta, babu.moger, dyoung, nikunj, john.allen, darwi,
linux-kernel, linux-crypto, kvm, linux-coco
From: Ashish Kalra <ashish.kalra@amd.com>
In the SEV-SNP architecture, hypervisor and non-SNP guests are subject
to RMP checks on writes to provide integrity of SEV-SNP guest memory.
The RMPOPT architecture enables optimizations whereby the RMP checks
can be skipped if 1GB regions of memory are known to not contain any
SNP guest memory.
RMPOPT is a new instruction designed to minimize the performance
overhead of RMP checks for the hypervisor and non-SNP guests.
RMPOPT instruction currently supports two functions. In case of the
verify and report status function the CPU will read the RMP contents,
verify the entire 1GB region starting at the provided SPA is HV-owned.
For the entire 1GB region it checks that all RMP entries in this region
are HV-owned (i.e, not in assigned state) and then accordingly updates
the RMPOPT table to indicate if optimization has been enabled and
provide indication to software if the optimization was successful.
In case of report status function, the CPU returns the optimization
status for the 1GB region.
The RMPOPT table is managed by a combination of software and hardware.
Software uses the RMPOPT instruction to set bits in the table,
indicating that regions of memory are entirely HV-owned. Hardware
automatically clears bits in the RMPOPT table when RMP contents are
changed during RMPUPDATE instruction.
For more information on the RMPOPT instruction, see the AMD64 RMPOPT
technical documentation. [1]
As SNP is enabled by default the hypervisor and non-SNP guests are
subject to RMP write checks to provide integrity of SNP guest memory.
This patch-series adds support to enable RMP optimizations for up to
2TB of system RAM across the system and allow RMPUPDATE to disable
those optimizations as SNP guests are launched.
Support for RAM larger than 2 TB will be added in follow-on series.
This series also introduces a new guest_memfd cleanup interface for
guest teardown, in case of SEV-SNP this interface is used to re-enable
RMP optimizations during guest shutdown and/or termination.
Once 1GB hugetlb guest_memfd support is merged, support for
re-enabling RMPOPT optimizations during 1GB page cleanup will be added
in follow-on series.
Additionally add debugfs interface to report per-CPU RMPOPT status
across all system RAM.
[1] https://docs.amd.com/v/u/en-US/69201_1.00_AMD64_RMPOPT_PUB
v2:
- Drop all NUMA and Socket configuration and enablement support and
enable RMPOPT support for up to 2TB of system RAM.
- Drop get_cpumask_of_primary_threads() and enable per-core RMPOPT
base MSRs and issue RMPOPT instruction on all CPUs.
- Drop the configfs interface to manually re-enable RMP optimizations.
- Add new guest_memfd cleanup interface to automatically re-enable
RMP optimizations during guest shutdown.
- Include references to the public RMPOPT documentation.
- Move debugfs directory for RMPOPT under architecuture specific
parent directory.
Ashish Kalra (7):
x86/cpufeatures: Add X86_FEATURE_AMD_RMPOPT feature flag
x86/sev: add support for enabling RMPOPT
x86/sev: add support for RMPOPT instruction
x86/sev: Add interface to re-enable RMP optimizations.
KVM: guest_memfd: Add cleanup interface for guest teardown
KVM: SEV: Implement SEV-SNP specific guest cleanup
x86/sev: Add debugfs support for RMPOPT
arch/x86/include/asm/cpufeatures.h | 2 +-
arch/x86/include/asm/kvm-x86-ops.h | 1 +
arch/x86/include/asm/kvm_host.h | 1 +
arch/x86/include/asm/msr-index.h | 3 +
arch/x86/include/asm/sev.h | 2 +
arch/x86/kernel/cpu/scattered.c | 1 +
arch/x86/kvm/Kconfig | 1 +
arch/x86/kvm/svm/sev.c | 9 ++
arch/x86/kvm/svm/svm.c | 1 +
arch/x86/kvm/svm/svm.h | 2 +
arch/x86/kvm/x86.c | 7 +
arch/x86/virt/svm/sev.c | 231 +++++++++++++++++++++++++++++
drivers/crypto/ccp/sev-dev.c | 4 +
include/linux/kvm_host.h | 4 +
virt/kvm/Kconfig | 4 +
virt/kvm/guest_memfd.c | 8 +
16 files changed, 280 insertions(+), 1 deletion(-)
--
2.43.0
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox