* [PATCH v5 01/65] linux-headers: Update to Linux v6.8-rc5
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-02-29 6:36 ` [PATCH v5 02/65] RAMBlock: Add support of KVM private guest memfd Xiaoyao Li
` (63 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
Guest memfd support in QEMU requires corresponding KVM guest memfd APIs,
which lands in Linux from v6.8-rc1.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
include/standard-headers/drm/drm_fourcc.h | 10 +-
include/standard-headers/linux/ethtool.h | 41 +++--
.../standard-headers/linux/virtio_config.h | 8 +-
include/standard-headers/linux/virtio_pci.h | 68 +++++++++
include/standard-headers/linux/virtio_pmem.h | 7 +
| 15 +-
| 2 +-
| 5 +
| 5 +
| 5 +
| 5 +
| 5 +
| 40 +++++
| 5 +
| 5 +
| 3 +
| 5 +
| 5 +
| 5 +
| 79 ++++++++++
| 140 +++++++-----------
| 29 +++-
| 1 +
23 files changed, 381 insertions(+), 112 deletions(-)
diff --git a/include/standard-headers/drm/drm_fourcc.h b/include/standard-headers/drm/drm_fourcc.h
index 3afb70160f0b..b72917073d8d 100644
--- a/include/standard-headers/drm/drm_fourcc.h
+++ b/include/standard-headers/drm/drm_fourcc.h
@@ -53,7 +53,7 @@ extern "C" {
* Format modifiers may change any property of the buffer, including the number
* of planes and/or the required allocation size. Format modifiers are
* vendor-namespaced, and as such the relationship between a fourcc code and a
- * modifier is specific to the modifer being used. For example, some modifiers
+ * modifier is specific to the modifier being used. For example, some modifiers
* may preserve meaning - such as number of planes - from the fourcc code,
* whereas others may not.
*
@@ -78,7 +78,7 @@ extern "C" {
* format.
* - Higher-level programs interfacing with KMS/GBM/EGL/Vulkan/etc: these users
* see modifiers as opaque tokens they can check for equality and intersect.
- * These users musn't need to know to reason about the modifier value
+ * These users mustn't need to know to reason about the modifier value
* (i.e. they are not expected to extract information out of the modifier).
*
* Vendors should document their modifier usage in as much detail as
@@ -539,7 +539,7 @@ extern "C" {
* This is a tiled layout using 4Kb tiles in row-major layout.
* Within the tile pixels are laid out in 16 256 byte units / sub-tiles which
* are arranged in four groups (two wide, two high) with column-major layout.
- * Each group therefore consits out of four 256 byte units, which are also laid
+ * Each group therefore consists out of four 256 byte units, which are also laid
* out as 2x2 column-major.
* 256 byte units are made out of four 64 byte blocks of pixels, producing
* either a square block or a 2:1 unit.
@@ -1102,7 +1102,7 @@ drm_fourcc_canonicalize_nvidia_format_mod(uint64_t modifier)
*/
/*
- * The top 4 bits (out of the 56 bits alloted for specifying vendor specific
+ * The top 4 bits (out of the 56 bits allotted for specifying vendor specific
* modifiers) denote the category for modifiers. Currently we have three
* categories of modifiers ie AFBC, MISC and AFRC. We can have a maximum of
* sixteen different categories.
@@ -1418,7 +1418,7 @@ drm_fourcc_canonicalize_nvidia_format_mod(uint64_t modifier)
* Amlogic FBC Memory Saving mode
*
* Indicates the storage is packed when pixel size is multiple of word
- * boudaries, i.e. 8bit should be stored in this mode to save allocation
+ * boundaries, i.e. 8bit should be stored in this mode to save allocation
* memory.
*
* This mode reduces body layout to 3072 bytes per 64x32 superblock with
diff --git a/include/standard-headers/linux/ethtool.h b/include/standard-headers/linux/ethtool.h
index 99fcddf04f88..dfb54eff6f7f 100644
--- a/include/standard-headers/linux/ethtool.h
+++ b/include/standard-headers/linux/ethtool.h
@@ -1266,6 +1266,8 @@ struct ethtool_rxfh_indir {
* hardware hash key.
* @hfunc: Defines the current RSS hash function used by HW (or to be set to).
* Valid values are one of the %ETH_RSS_HASH_*.
+ * @input_xfrm: Defines how the input data is transformed. Valid values are one
+ * of %RXH_XFRM_*.
* @rsvd8: Reserved for future use; see the note on reserved space.
* @rsvd32: Reserved for future use; see the note on reserved space.
* @rss_config: RX ring/queue index for each hash value i.e., indirection table
@@ -1285,7 +1287,8 @@ struct ethtool_rxfh {
uint32_t indir_size;
uint32_t key_size;
uint8_t hfunc;
- uint8_t rsvd8[3];
+ uint8_t input_xfrm;
+ uint8_t rsvd8[2];
uint32_t rsvd32;
uint32_t rss_config[];
};
@@ -1992,6 +1995,15 @@ static inline int ethtool_validate_duplex(uint8_t duplex)
#define WOL_MODE_COUNT 8
+/* RSS hash function data
+ * XOR the corresponding source and destination fields of each specified
+ * protocol. Both copies of the XOR'ed fields are fed into the RSS and RXHASH
+ * calculation. Note that this XORing reduces the input set entropy and could
+ * be exploited to reduce the RSS queue spread.
+ */
+#define RXH_XFRM_SYM_XOR (1 << 0)
+#define RXH_XFRM_NO_CHANGE 0xff
+
/* L2-L4 network traffic flow types */
#define TCP_V4_FLOW 0x01 /* hash or spec (tcp_ip4_spec) */
#define UDP_V4_FLOW 0x02 /* hash or spec (udp_ip4_spec) */
@@ -2128,18 +2140,6 @@ enum ethtool_reset_flags {
* refused. For drivers: ignore this field (use kernel's
* __ETHTOOL_LINK_MODE_MASK_NBITS instead), any change to it will
* be overwritten by kernel.
- * @supported: Bitmap with each bit meaning given by
- * %ethtool_link_mode_bit_indices for the link modes, physical
- * connectors and other link features for which the interface
- * supports autonegotiation or auto-detection. Read-only.
- * @advertising: Bitmap with each bit meaning given by
- * %ethtool_link_mode_bit_indices for the link modes, physical
- * connectors and other link features that are advertised through
- * autonegotiation or enabled for auto-detection.
- * @lp_advertising: Bitmap with each bit meaning given by
- * %ethtool_link_mode_bit_indices for the link modes, and other
- * link features that the link partner advertised through
- * autonegotiation; 0 if unknown or not applicable. Read-only.
* @transceiver: Used to distinguish different possible PHY types,
* reported consistently by PHYLIB. Read-only.
* @master_slave_cfg: Master/slave port mode.
@@ -2181,6 +2181,21 @@ enum ethtool_reset_flags {
* %set_link_ksettings() should validate all fields other than @cmd
* and @link_mode_masks_nwords that are not described as read-only or
* deprecated, and must ignore all fields described as read-only.
+ *
+ * @link_mode_masks is divided into three bitfields, each of length
+ * @link_mode_masks_nwords:
+ * - supported: Bitmap with each bit meaning given by
+ * %ethtool_link_mode_bit_indices for the link modes, physical
+ * connectors and other link features for which the interface
+ * supports autonegotiation or auto-detection. Read-only.
+ * - advertising: Bitmap with each bit meaning given by
+ * %ethtool_link_mode_bit_indices for the link modes, physical
+ * connectors and other link features that are advertised through
+ * autonegotiation or enabled for auto-detection.
+ * - lp_advertising: Bitmap with each bit meaning given by
+ * %ethtool_link_mode_bit_indices for the link modes, and other
+ * link features that the link partner advertised through
+ * autonegotiation; 0 if unknown or not applicable. Read-only.
*/
struct ethtool_link_settings {
uint32_t cmd;
diff --git a/include/standard-headers/linux/virtio_config.h b/include/standard-headers/linux/virtio_config.h
index bfd1ca643e7f..45be0fa1bcdb 100644
--- a/include/standard-headers/linux/virtio_config.h
+++ b/include/standard-headers/linux/virtio_config.h
@@ -52,7 +52,7 @@
* rest are per-device feature bits.
*/
#define VIRTIO_TRANSPORT_F_START 28
-#define VIRTIO_TRANSPORT_F_END 41
+#define VIRTIO_TRANSPORT_F_END 42
#ifndef VIRTIO_CONFIG_NO_LEGACY
/* Do we get callbacks when the ring is completely used, even if we've
@@ -112,4 +112,10 @@
* This feature indicates that the driver can reset a queue individually.
*/
#define VIRTIO_F_RING_RESET 40
+
+/*
+ * This feature indicates that the device support administration virtqueues.
+ */
+#define VIRTIO_F_ADMIN_VQ 41
+
#endif /* _LINUX_VIRTIO_CONFIG_H */
diff --git a/include/standard-headers/linux/virtio_pci.h b/include/standard-headers/linux/virtio_pci.h
index b7fdfd066878..3e2bc2c97e6e 100644
--- a/include/standard-headers/linux/virtio_pci.h
+++ b/include/standard-headers/linux/virtio_pci.h
@@ -175,6 +175,9 @@ struct virtio_pci_modern_common_cfg {
uint16_t queue_notify_data; /* read-write */
uint16_t queue_reset; /* read-write */
+
+ uint16_t admin_queue_index; /* read-only */
+ uint16_t admin_queue_num; /* read-only */
};
/* Fields in VIRTIO_PCI_CAP_PCI_CFG: */
@@ -215,7 +218,72 @@ struct virtio_pci_cfg_cap {
#define VIRTIO_PCI_COMMON_Q_USEDHI 52
#define VIRTIO_PCI_COMMON_Q_NDATA 56
#define VIRTIO_PCI_COMMON_Q_RESET 58
+#define VIRTIO_PCI_COMMON_ADM_Q_IDX 60
+#define VIRTIO_PCI_COMMON_ADM_Q_NUM 62
#endif /* VIRTIO_PCI_NO_MODERN */
+/* Admin command status. */
+#define VIRTIO_ADMIN_STATUS_OK 0
+
+/* Admin command opcode. */
+#define VIRTIO_ADMIN_CMD_LIST_QUERY 0x0
+#define VIRTIO_ADMIN_CMD_LIST_USE 0x1
+
+/* Admin command group type. */
+#define VIRTIO_ADMIN_GROUP_TYPE_SRIOV 0x1
+
+/* Transitional device admin command. */
+#define VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_WRITE 0x2
+#define VIRTIO_ADMIN_CMD_LEGACY_COMMON_CFG_READ 0x3
+#define VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_WRITE 0x4
+#define VIRTIO_ADMIN_CMD_LEGACY_DEV_CFG_READ 0x5
+#define VIRTIO_ADMIN_CMD_LEGACY_NOTIFY_INFO 0x6
+
+struct QEMU_PACKED virtio_admin_cmd_hdr {
+ uint16_t opcode;
+ /*
+ * 1 - SR-IOV
+ * 2-65535 - reserved
+ */
+ uint16_t group_type;
+ /* Unused, reserved for future extensions. */
+ uint8_t reserved1[12];
+ uint64_t group_member_id;
+};
+
+struct QEMU_PACKED virtio_admin_cmd_status {
+ uint16_t status;
+ uint16_t status_qualifier;
+ /* Unused, reserved for future extensions. */
+ uint8_t reserved2[4];
+};
+
+struct QEMU_PACKED virtio_admin_cmd_legacy_wr_data {
+ uint8_t offset; /* Starting offset of the register(s) to write. */
+ uint8_t reserved[7];
+ uint8_t registers[];
+};
+
+struct QEMU_PACKED virtio_admin_cmd_legacy_rd_data {
+ uint8_t offset; /* Starting offset of the register(s) to read. */
+};
+
+#define VIRTIO_ADMIN_CMD_NOTIFY_INFO_FLAGS_END 0
+#define VIRTIO_ADMIN_CMD_NOTIFY_INFO_FLAGS_OWNER_DEV 0x1
+#define VIRTIO_ADMIN_CMD_NOTIFY_INFO_FLAGS_OWNER_MEM 0x2
+
+#define VIRTIO_ADMIN_CMD_MAX_NOTIFY_INFO 4
+
+struct QEMU_PACKED virtio_admin_cmd_notify_info_data {
+ uint8_t flags; /* 0 = end of list, 1 = owner device, 2 = member device */
+ uint8_t bar; /* BAR of the member or the owner device */
+ uint8_t padding[6];
+ uint64_t offset; /* Offset within bar. */
+};
+
+struct virtio_admin_cmd_notify_info_result {
+ struct virtio_admin_cmd_notify_info_data entries[VIRTIO_ADMIN_CMD_MAX_NOTIFY_INFO];
+};
+
#endif
diff --git a/include/standard-headers/linux/virtio_pmem.h b/include/standard-headers/linux/virtio_pmem.h
index fc029de7988e..1a2576d0178e 100644
--- a/include/standard-headers/linux/virtio_pmem.h
+++ b/include/standard-headers/linux/virtio_pmem.h
@@ -14,6 +14,13 @@
#include "standard-headers/linux/virtio_ids.h"
#include "standard-headers/linux/virtio_config.h"
+/* Feature bits */
+/* guest physical address range will be indicated as shared memory region 0 */
+#define VIRTIO_PMEM_F_SHMEM_REGION 0
+
+/* shmid of the shared memory region corresponding to the pmem */
+#define VIRTIO_PMEM_SHMEM_REGION_ID 0
+
struct virtio_pmem_config {
uint64_t start;
uint64_t size;
--git a/linux-headers/asm-generic/unistd.h b/linux-headers/asm-generic/unistd.h
index 756b013fb832..75f00965ab15 100644
--- a/linux-headers/asm-generic/unistd.h
+++ b/linux-headers/asm-generic/unistd.h
@@ -829,8 +829,21 @@ __SYSCALL(__NR_futex_wait, sys_futex_wait)
#define __NR_futex_requeue 456
__SYSCALL(__NR_futex_requeue, sys_futex_requeue)
+#define __NR_statmount 457
+__SYSCALL(__NR_statmount, sys_statmount)
+
+#define __NR_listmount 458
+__SYSCALL(__NR_listmount, sys_listmount)
+
+#define __NR_lsm_get_self_attr 459
+__SYSCALL(__NR_lsm_get_self_attr, sys_lsm_get_self_attr)
+#define __NR_lsm_set_self_attr 460
+__SYSCALL(__NR_lsm_set_self_attr, sys_lsm_set_self_attr)
+#define __NR_lsm_list_modules 461
+__SYSCALL(__NR_lsm_list_modules, sys_lsm_list_modules)
+
#undef __NR_syscalls
-#define __NR_syscalls 457
+#define __NR_syscalls 462
/*
* 32 bit systems traditionally used different
--git a/linux-headers/asm-mips/mman.h b/linux-headers/asm-mips/mman.h
index c6e1fc77c996..9c48d9a21aa0 100644
--- a/linux-headers/asm-mips/mman.h
+++ b/linux-headers/asm-mips/mman.h
@@ -88,7 +88,7 @@
#define MADV_HUGEPAGE 14 /* Worth backing with hugepages */
#define MADV_NOHUGEPAGE 15 /* Not worth backing with hugepages */
-#define MADV_DONTDUMP 16 /* Explicity exclude from the core dump,
+#define MADV_DONTDUMP 16 /* Explicitly exclude from core dump,
overrides the coredump filter bits */
#define MADV_DODUMP 17 /* Clear the MADV_NODUMP flag */
--git a/linux-headers/asm-mips/unistd_n32.h b/linux-headers/asm-mips/unistd_n32.h
index 994b6f008f54..ce2e050a9ba4 100644
--- a/linux-headers/asm-mips/unistd_n32.h
+++ b/linux-headers/asm-mips/unistd_n32.h
@@ -385,5 +385,10 @@
#define __NR_futex_wake (__NR_Linux + 454)
#define __NR_futex_wait (__NR_Linux + 455)
#define __NR_futex_requeue (__NR_Linux + 456)
+#define __NR_statmount (__NR_Linux + 457)
+#define __NR_listmount (__NR_Linux + 458)
+#define __NR_lsm_get_self_attr (__NR_Linux + 459)
+#define __NR_lsm_set_self_attr (__NR_Linux + 460)
+#define __NR_lsm_list_modules (__NR_Linux + 461)
#endif /* _ASM_UNISTD_N32_H */
--git a/linux-headers/asm-mips/unistd_n64.h b/linux-headers/asm-mips/unistd_n64.h
index 41dcf5877a1b..5bfb3733ffdf 100644
--- a/linux-headers/asm-mips/unistd_n64.h
+++ b/linux-headers/asm-mips/unistd_n64.h
@@ -361,5 +361,10 @@
#define __NR_futex_wake (__NR_Linux + 454)
#define __NR_futex_wait (__NR_Linux + 455)
#define __NR_futex_requeue (__NR_Linux + 456)
+#define __NR_statmount (__NR_Linux + 457)
+#define __NR_listmount (__NR_Linux + 458)
+#define __NR_lsm_get_self_attr (__NR_Linux + 459)
+#define __NR_lsm_set_self_attr (__NR_Linux + 460)
+#define __NR_lsm_list_modules (__NR_Linux + 461)
#endif /* _ASM_UNISTD_N64_H */
--git a/linux-headers/asm-mips/unistd_o32.h b/linux-headers/asm-mips/unistd_o32.h
index ae9d334d96e3..02eaecd020ec 100644
--- a/linux-headers/asm-mips/unistd_o32.h
+++ b/linux-headers/asm-mips/unistd_o32.h
@@ -431,5 +431,10 @@
#define __NR_futex_wake (__NR_Linux + 454)
#define __NR_futex_wait (__NR_Linux + 455)
#define __NR_futex_requeue (__NR_Linux + 456)
+#define __NR_statmount (__NR_Linux + 457)
+#define __NR_listmount (__NR_Linux + 458)
+#define __NR_lsm_get_self_attr (__NR_Linux + 459)
+#define __NR_lsm_set_self_attr (__NR_Linux + 460)
+#define __NR_lsm_list_modules (__NR_Linux + 461)
#endif /* _ASM_UNISTD_O32_H */
--git a/linux-headers/asm-powerpc/unistd_32.h b/linux-headers/asm-powerpc/unistd_32.h
index b9b23d66d7d9..bbab08d6ec26 100644
--- a/linux-headers/asm-powerpc/unistd_32.h
+++ b/linux-headers/asm-powerpc/unistd_32.h
@@ -438,6 +438,11 @@
#define __NR_futex_wake 454
#define __NR_futex_wait 455
#define __NR_futex_requeue 456
+#define __NR_statmount 457
+#define __NR_listmount 458
+#define __NR_lsm_get_self_attr 459
+#define __NR_lsm_set_self_attr 460
+#define __NR_lsm_list_modules 461
#endif /* _ASM_UNISTD_32_H */
--git a/linux-headers/asm-powerpc/unistd_64.h b/linux-headers/asm-powerpc/unistd_64.h
index cbb4b3e8f7c2..af34cde70f20 100644
--- a/linux-headers/asm-powerpc/unistd_64.h
+++ b/linux-headers/asm-powerpc/unistd_64.h
@@ -410,6 +410,11 @@
#define __NR_futex_wake 454
#define __NR_futex_wait 455
#define __NR_futex_requeue 456
+#define __NR_statmount 457
+#define __NR_listmount 458
+#define __NR_lsm_get_self_attr 459
+#define __NR_lsm_set_self_attr 460
+#define __NR_lsm_list_modules 461
#endif /* _ASM_UNISTD_64_H */
--git a/linux-headers/asm-riscv/kvm.h b/linux-headers/asm-riscv/kvm.h
index 60d3b21dead7..7499e88a947c 100644
--- a/linux-headers/asm-riscv/kvm.h
+++ b/linux-headers/asm-riscv/kvm.h
@@ -139,6 +139,33 @@ enum KVM_RISCV_ISA_EXT_ID {
KVM_RISCV_ISA_EXT_ZIHPM,
KVM_RISCV_ISA_EXT_SMSTATEEN,
KVM_RISCV_ISA_EXT_ZICOND,
+ KVM_RISCV_ISA_EXT_ZBC,
+ KVM_RISCV_ISA_EXT_ZBKB,
+ KVM_RISCV_ISA_EXT_ZBKC,
+ KVM_RISCV_ISA_EXT_ZBKX,
+ KVM_RISCV_ISA_EXT_ZKND,
+ KVM_RISCV_ISA_EXT_ZKNE,
+ KVM_RISCV_ISA_EXT_ZKNH,
+ KVM_RISCV_ISA_EXT_ZKR,
+ KVM_RISCV_ISA_EXT_ZKSED,
+ KVM_RISCV_ISA_EXT_ZKSH,
+ KVM_RISCV_ISA_EXT_ZKT,
+ KVM_RISCV_ISA_EXT_ZVBB,
+ KVM_RISCV_ISA_EXT_ZVBC,
+ KVM_RISCV_ISA_EXT_ZVKB,
+ KVM_RISCV_ISA_EXT_ZVKG,
+ KVM_RISCV_ISA_EXT_ZVKNED,
+ KVM_RISCV_ISA_EXT_ZVKNHA,
+ KVM_RISCV_ISA_EXT_ZVKNHB,
+ KVM_RISCV_ISA_EXT_ZVKSED,
+ KVM_RISCV_ISA_EXT_ZVKSH,
+ KVM_RISCV_ISA_EXT_ZVKT,
+ KVM_RISCV_ISA_EXT_ZFH,
+ KVM_RISCV_ISA_EXT_ZFHMIN,
+ KVM_RISCV_ISA_EXT_ZIHINTNTL,
+ KVM_RISCV_ISA_EXT_ZVFH,
+ KVM_RISCV_ISA_EXT_ZVFHMIN,
+ KVM_RISCV_ISA_EXT_ZFA,
KVM_RISCV_ISA_EXT_MAX,
};
@@ -157,9 +184,16 @@ enum KVM_RISCV_SBI_EXT_ID {
KVM_RISCV_SBI_EXT_EXPERIMENTAL,
KVM_RISCV_SBI_EXT_VENDOR,
KVM_RISCV_SBI_EXT_DBCN,
+ KVM_RISCV_SBI_EXT_STA,
KVM_RISCV_SBI_EXT_MAX,
};
+/* SBI STA extension registers for KVM_GET_ONE_REG and KVM_SET_ONE_REG */
+struct kvm_riscv_sbi_sta {
+ unsigned long shmem_lo;
+ unsigned long shmem_hi;
+};
+
/* Possible states for kvm_riscv_timer */
#define KVM_RISCV_TIMER_STATE_OFF 0
#define KVM_RISCV_TIMER_STATE_ON 1
@@ -241,6 +275,12 @@ enum KVM_RISCV_SBI_EXT_ID {
#define KVM_REG_RISCV_VECTOR_REG(n) \
((n) + sizeof(struct __riscv_v_ext_state) / sizeof(unsigned long))
+/* Registers for specific SBI extensions are mapped as type 10 */
+#define KVM_REG_RISCV_SBI_STATE (0x0a << KVM_REG_RISCV_TYPE_SHIFT)
+#define KVM_REG_RISCV_SBI_STA (0x0 << KVM_REG_RISCV_SUBTYPE_SHIFT)
+#define KVM_REG_RISCV_SBI_STA_REG(name) \
+ (offsetof(struct kvm_riscv_sbi_sta, name) / sizeof(unsigned long))
+
/* Device Control API: RISC-V AIA */
#define KVM_DEV_RISCV_APLIC_ALIGN 0x1000
#define KVM_DEV_RISCV_APLIC_SIZE 0x4000
--git a/linux-headers/asm-s390/unistd_32.h b/linux-headers/asm-s390/unistd_32.h
index c093e6d5f911..a3ece69d8241 100644
--- a/linux-headers/asm-s390/unistd_32.h
+++ b/linux-headers/asm-s390/unistd_32.h
@@ -429,5 +429,10 @@
#define __NR_futex_wake 454
#define __NR_futex_wait 455
#define __NR_futex_requeue 456
+#define __NR_statmount 457
+#define __NR_listmount 458
+#define __NR_lsm_get_self_attr 459
+#define __NR_lsm_set_self_attr 460
+#define __NR_lsm_list_modules 461
#endif /* _ASM_S390_UNISTD_32_H */
--git a/linux-headers/asm-s390/unistd_64.h b/linux-headers/asm-s390/unistd_64.h
index 114c0569a49a..8c5fd93495ce 100644
--- a/linux-headers/asm-s390/unistd_64.h
+++ b/linux-headers/asm-s390/unistd_64.h
@@ -377,5 +377,10 @@
#define __NR_futex_wake 454
#define __NR_futex_wait 455
#define __NR_futex_requeue 456
+#define __NR_statmount 457
+#define __NR_listmount 458
+#define __NR_lsm_get_self_attr 459
+#define __NR_lsm_set_self_attr 460
+#define __NR_lsm_list_modules 461
#endif /* _ASM_S390_UNISTD_64_H */
--git a/linux-headers/asm-x86/kvm.h b/linux-headers/asm-x86/kvm.h
index 2b3a8f7bd2c0..003fb745347c 100644
--- a/linux-headers/asm-x86/kvm.h
+++ b/linux-headers/asm-x86/kvm.h
@@ -560,4 +560,7 @@ struct kvm_pmu_event_filter {
/* x86-specific KVM_EXIT_HYPERCALL flags. */
#define KVM_EXIT_HYPERCALL_LONG_MODE BIT(0)
+#define KVM_X86_DEFAULT_VM 0
+#define KVM_X86_SW_PROTECTED_VM 1
+
#endif /* _ASM_X86_KVM_H */
--git a/linux-headers/asm-x86/unistd_32.h b/linux-headers/asm-x86/unistd_32.h
index 329649c377be..5c9c329e9390 100644
--- a/linux-headers/asm-x86/unistd_32.h
+++ b/linux-headers/asm-x86/unistd_32.h
@@ -447,6 +447,11 @@
#define __NR_futex_wake 454
#define __NR_futex_wait 455
#define __NR_futex_requeue 456
+#define __NR_statmount 457
+#define __NR_listmount 458
+#define __NR_lsm_get_self_attr 459
+#define __NR_lsm_set_self_attr 460
+#define __NR_lsm_list_modules 461
#endif /* _ASM_UNISTD_32_H */
--git a/linux-headers/asm-x86/unistd_64.h b/linux-headers/asm-x86/unistd_64.h
index 4583606ce684..d9aab7ae87d8 100644
--- a/linux-headers/asm-x86/unistd_64.h
+++ b/linux-headers/asm-x86/unistd_64.h
@@ -369,6 +369,11 @@
#define __NR_futex_wake 454
#define __NR_futex_wait 455
#define __NR_futex_requeue 456
+#define __NR_statmount 457
+#define __NR_listmount 458
+#define __NR_lsm_get_self_attr 459
+#define __NR_lsm_set_self_attr 460
+#define __NR_lsm_list_modules 461
#endif /* _ASM_UNISTD_64_H */
--git a/linux-headers/asm-x86/unistd_x32.h b/linux-headers/asm-x86/unistd_x32.h
index 146d74d8e4b0..63cdd1ee43df 100644
--- a/linux-headers/asm-x86/unistd_x32.h
+++ b/linux-headers/asm-x86/unistd_x32.h
@@ -321,6 +321,11 @@
#define __NR_futex_wake (__X32_SYSCALL_BIT + 454)
#define __NR_futex_wait (__X32_SYSCALL_BIT + 455)
#define __NR_futex_requeue (__X32_SYSCALL_BIT + 456)
+#define __NR_statmount (__X32_SYSCALL_BIT + 457)
+#define __NR_listmount (__X32_SYSCALL_BIT + 458)
+#define __NR_lsm_get_self_attr (__X32_SYSCALL_BIT + 459)
+#define __NR_lsm_set_self_attr (__X32_SYSCALL_BIT + 460)
+#define __NR_lsm_list_modules (__X32_SYSCALL_BIT + 461)
#define __NR_rt_sigaction (__X32_SYSCALL_BIT + 512)
#define __NR_rt_sigreturn (__X32_SYSCALL_BIT + 513)
#define __NR_ioctl (__X32_SYSCALL_BIT + 514)
--git a/linux-headers/linux/iommufd.h b/linux-headers/linux/iommufd.h
index 806d98d09c01..72e8f4b9dd0e 100644
--- a/linux-headers/linux/iommufd.h
+++ b/linux-headers/linux/iommufd.h
@@ -49,6 +49,7 @@ enum {
IOMMUFD_CMD_GET_HW_INFO,
IOMMUFD_CMD_HWPT_SET_DIRTY_TRACKING,
IOMMUFD_CMD_HWPT_GET_DIRTY_BITMAP,
+ IOMMUFD_CMD_HWPT_INVALIDATE,
};
/**
@@ -613,4 +614,82 @@ struct iommu_hwpt_get_dirty_bitmap {
#define IOMMU_HWPT_GET_DIRTY_BITMAP _IO(IOMMUFD_TYPE, \
IOMMUFD_CMD_HWPT_GET_DIRTY_BITMAP)
+/**
+ * enum iommu_hwpt_invalidate_data_type - IOMMU HWPT Cache Invalidation
+ * Data Type
+ * @IOMMU_HWPT_INVALIDATE_DATA_VTD_S1: Invalidation data for VTD_S1
+ */
+enum iommu_hwpt_invalidate_data_type {
+ IOMMU_HWPT_INVALIDATE_DATA_VTD_S1,
+};
+
+/**
+ * enum iommu_hwpt_vtd_s1_invalidate_flags - Flags for Intel VT-d
+ * stage-1 cache invalidation
+ * @IOMMU_VTD_INV_FLAGS_LEAF: Indicates whether the invalidation applies
+ * to all-levels page structure cache or just
+ * the leaf PTE cache.
+ */
+enum iommu_hwpt_vtd_s1_invalidate_flags {
+ IOMMU_VTD_INV_FLAGS_LEAF = 1 << 0,
+};
+
+/**
+ * struct iommu_hwpt_vtd_s1_invalidate - Intel VT-d cache invalidation
+ * (IOMMU_HWPT_INVALIDATE_DATA_VTD_S1)
+ * @addr: The start address of the range to be invalidated. It needs to
+ * be 4KB aligned.
+ * @npages: Number of contiguous 4K pages to be invalidated.
+ * @flags: Combination of enum iommu_hwpt_vtd_s1_invalidate_flags
+ * @__reserved: Must be 0
+ *
+ * The Intel VT-d specific invalidation data for user-managed stage-1 cache
+ * invalidation in nested translation. Userspace uses this structure to
+ * tell the impacted cache scope after modifying the stage-1 page table.
+ *
+ * Invalidating all the caches related to the page table by setting @addr
+ * to be 0 and @npages to be U64_MAX.
+ *
+ * The device TLB will be invalidated automatically if ATS is enabled.
+ */
+struct iommu_hwpt_vtd_s1_invalidate {
+ __aligned_u64 addr;
+ __aligned_u64 npages;
+ __u32 flags;
+ __u32 __reserved;
+};
+
+/**
+ * struct iommu_hwpt_invalidate - ioctl(IOMMU_HWPT_INVALIDATE)
+ * @size: sizeof(struct iommu_hwpt_invalidate)
+ * @hwpt_id: ID of a nested HWPT for cache invalidation
+ * @data_uptr: User pointer to an array of driver-specific cache invalidation
+ * data.
+ * @data_type: One of enum iommu_hwpt_invalidate_data_type, defining the data
+ * type of all the entries in the invalidation request array. It
+ * should be a type supported by the hwpt pointed by @hwpt_id.
+ * @entry_len: Length (in bytes) of a request entry in the request array
+ * @entry_num: Input the number of cache invalidation requests in the array.
+ * Output the number of requests successfully handled by kernel.
+ * @__reserved: Must be 0.
+ *
+ * Invalidate the iommu cache for user-managed page table. Modifications on a
+ * user-managed page table should be followed by this operation to sync cache.
+ * Each ioctl can support one or more cache invalidation requests in the array
+ * that has a total size of @entry_len * @entry_num.
+ *
+ * An empty invalidation request array by setting @entry_num==0 is allowed, and
+ * @entry_len and @data_uptr would be ignored in this case. This can be used to
+ * check if the given @data_type is supported or not by kernel.
+ */
+struct iommu_hwpt_invalidate {
+ __u32 size;
+ __u32 hwpt_id;
+ __aligned_u64 data_uptr;
+ __u32 data_type;
+ __u32 entry_len;
+ __u32 entry_num;
+ __u32 __reserved;
+};
+#define IOMMU_HWPT_INVALIDATE _IO(IOMMUFD_TYPE, IOMMUFD_CMD_HWPT_INVALIDATE)
#endif
--git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h
index 549fea3a978a..17839229b2ac 100644
--- a/linux-headers/linux/kvm.h
+++ b/linux-headers/linux/kvm.h
@@ -16,76 +16,6 @@
#define KVM_API_VERSION 12
-/* *** Deprecated interfaces *** */
-
-#define KVM_TRC_SHIFT 16
-
-#define KVM_TRC_ENTRYEXIT (1 << KVM_TRC_SHIFT)
-#define KVM_TRC_HANDLER (1 << (KVM_TRC_SHIFT + 1))
-
-#define KVM_TRC_VMENTRY (KVM_TRC_ENTRYEXIT + 0x01)
-#define KVM_TRC_VMEXIT (KVM_TRC_ENTRYEXIT + 0x02)
-#define KVM_TRC_PAGE_FAULT (KVM_TRC_HANDLER + 0x01)
-
-#define KVM_TRC_HEAD_SIZE 12
-#define KVM_TRC_CYCLE_SIZE 8
-#define KVM_TRC_EXTRA_MAX 7
-
-#define KVM_TRC_INJ_VIRQ (KVM_TRC_HANDLER + 0x02)
-#define KVM_TRC_REDELIVER_EVT (KVM_TRC_HANDLER + 0x03)
-#define KVM_TRC_PEND_INTR (KVM_TRC_HANDLER + 0x04)
-#define KVM_TRC_IO_READ (KVM_TRC_HANDLER + 0x05)
-#define KVM_TRC_IO_WRITE (KVM_TRC_HANDLER + 0x06)
-#define KVM_TRC_CR_READ (KVM_TRC_HANDLER + 0x07)
-#define KVM_TRC_CR_WRITE (KVM_TRC_HANDLER + 0x08)
-#define KVM_TRC_DR_READ (KVM_TRC_HANDLER + 0x09)
-#define KVM_TRC_DR_WRITE (KVM_TRC_HANDLER + 0x0A)
-#define KVM_TRC_MSR_READ (KVM_TRC_HANDLER + 0x0B)
-#define KVM_TRC_MSR_WRITE (KVM_TRC_HANDLER + 0x0C)
-#define KVM_TRC_CPUID (KVM_TRC_HANDLER + 0x0D)
-#define KVM_TRC_INTR (KVM_TRC_HANDLER + 0x0E)
-#define KVM_TRC_NMI (KVM_TRC_HANDLER + 0x0F)
-#define KVM_TRC_VMMCALL (KVM_TRC_HANDLER + 0x10)
-#define KVM_TRC_HLT (KVM_TRC_HANDLER + 0x11)
-#define KVM_TRC_CLTS (KVM_TRC_HANDLER + 0x12)
-#define KVM_TRC_LMSW (KVM_TRC_HANDLER + 0x13)
-#define KVM_TRC_APIC_ACCESS (KVM_TRC_HANDLER + 0x14)
-#define KVM_TRC_TDP_FAULT (KVM_TRC_HANDLER + 0x15)
-#define KVM_TRC_GTLB_WRITE (KVM_TRC_HANDLER + 0x16)
-#define KVM_TRC_STLB_WRITE (KVM_TRC_HANDLER + 0x17)
-#define KVM_TRC_STLB_INVAL (KVM_TRC_HANDLER + 0x18)
-#define KVM_TRC_PPC_INSTR (KVM_TRC_HANDLER + 0x19)
-
-struct kvm_user_trace_setup {
- __u32 buf_size;
- __u32 buf_nr;
-};
-
-#define __KVM_DEPRECATED_MAIN_W_0x06 \
- _IOW(KVMIO, 0x06, struct kvm_user_trace_setup)
-#define __KVM_DEPRECATED_MAIN_0x07 _IO(KVMIO, 0x07)
-#define __KVM_DEPRECATED_MAIN_0x08 _IO(KVMIO, 0x08)
-
-#define __KVM_DEPRECATED_VM_R_0x70 _IOR(KVMIO, 0x70, struct kvm_assigned_irq)
-
-struct kvm_breakpoint {
- __u32 enabled;
- __u32 padding;
- __u64 address;
-};
-
-struct kvm_debug_guest {
- __u32 enabled;
- __u32 pad;
- struct kvm_breakpoint breakpoints[4];
- __u32 singlestep;
-};
-
-#define __KVM_DEPRECATED_VCPU_W_0x87 _IOW(KVMIO, 0x87, struct kvm_debug_guest)
-
-/* *** End of deprecated interfaces *** */
-
-
/* for KVM_SET_USER_MEMORY_REGION */
struct kvm_userspace_memory_region {
__u32 slot;
@@ -95,6 +25,19 @@ struct kvm_userspace_memory_region {
__u64 userspace_addr; /* start of the userspace allocated memory */
};
+/* for KVM_SET_USER_MEMORY_REGION2 */
+struct kvm_userspace_memory_region2 {
+ __u32 slot;
+ __u32 flags;
+ __u64 guest_phys_addr;
+ __u64 memory_size;
+ __u64 userspace_addr;
+ __u64 guest_memfd_offset;
+ __u32 guest_memfd;
+ __u32 pad1;
+ __u64 pad2[14];
+};
+
/*
* The bit 0 ~ bit 15 of kvm_userspace_memory_region::flags are visible for
* userspace, other bits are reserved for kvm internal use which are defined
@@ -102,6 +45,7 @@ struct kvm_userspace_memory_region {
*/
#define KVM_MEM_LOG_DIRTY_PAGES (1UL << 0)
#define KVM_MEM_READONLY (1UL << 1)
+#define KVM_MEM_GUEST_MEMFD (1UL << 2)
/* for KVM_IRQ_LINE */
struct kvm_irq_level {
@@ -265,6 +209,7 @@ struct kvm_xen_exit {
#define KVM_EXIT_RISCV_CSR 36
#define KVM_EXIT_NOTIFY 37
#define KVM_EXIT_LOONGARCH_IOCSR 38
+#define KVM_EXIT_MEMORY_FAULT 39
/* For KVM_EXIT_INTERNAL_ERROR */
/* Emulate instruction failed. */
@@ -514,6 +459,13 @@ struct kvm_run {
#define KVM_NOTIFY_CONTEXT_INVALID (1 << 0)
__u32 flags;
} notify;
+ /* KVM_EXIT_MEMORY_FAULT */
+ struct {
+#define KVM_MEMORY_EXIT_FLAG_PRIVATE (1ULL << 3)
+ __u64 flags;
+ __u64 gpa;
+ __u64 size;
+ } memory_fault;
/* Fix the size of the union. */
char padding[256];
};
@@ -941,9 +893,6 @@ struct kvm_ppc_resize_hpt {
*/
#define KVM_GET_VCPU_MMAP_SIZE _IO(KVMIO, 0x04) /* in bytes */
#define KVM_GET_SUPPORTED_CPUID _IOWR(KVMIO, 0x05, struct kvm_cpuid2)
-#define KVM_TRACE_ENABLE __KVM_DEPRECATED_MAIN_W_0x06
-#define KVM_TRACE_PAUSE __KVM_DEPRECATED_MAIN_0x07
-#define KVM_TRACE_DISABLE __KVM_DEPRECATED_MAIN_0x08
#define KVM_GET_EMULATED_CPUID _IOWR(KVMIO, 0x09, struct kvm_cpuid2)
#define KVM_GET_MSR_FEATURE_INDEX_LIST _IOWR(KVMIO, 0x0a, struct kvm_msr_list)
@@ -1197,6 +1146,11 @@ struct kvm_ppc_resize_hpt {
#define KVM_CAP_ARM_EAGER_SPLIT_CHUNK_SIZE 228
#define KVM_CAP_ARM_SUPPORTED_BLOCK_SIZES 229
#define KVM_CAP_ARM_SUPPORTED_REG_MASK_RANGES 230
+#define KVM_CAP_USER_MEMORY2 231
+#define KVM_CAP_MEMORY_FAULT_INFO 232
+#define KVM_CAP_MEMORY_ATTRIBUTES 233
+#define KVM_CAP_GUEST_MEMFD 234
+#define KVM_CAP_VM_TYPES 235
#ifdef KVM_CAP_IRQ_ROUTING
@@ -1287,6 +1241,7 @@ struct kvm_x86_mce {
#define KVM_XEN_HVM_CONFIG_EVTCHN_2LEVEL (1 << 4)
#define KVM_XEN_HVM_CONFIG_EVTCHN_SEND (1 << 5)
#define KVM_XEN_HVM_CONFIG_RUNSTATE_UPDATE_FLAG (1 << 6)
+#define KVM_XEN_HVM_CONFIG_PVCLOCK_TSC_UNSTABLE (1 << 7)
struct kvm_xen_hvm_config {
__u32 flags;
@@ -1479,6 +1434,8 @@ struct kvm_vfio_spapr_tce {
struct kvm_userspace_memory_region)
#define KVM_SET_TSS_ADDR _IO(KVMIO, 0x47)
#define KVM_SET_IDENTITY_MAP_ADDR _IOW(KVMIO, 0x48, __u64)
+#define KVM_SET_USER_MEMORY_REGION2 _IOW(KVMIO, 0x49, \
+ struct kvm_userspace_memory_region2)
/* enable ucontrol for s390 */
struct kvm_s390_ucas_mapping {
@@ -1503,20 +1460,8 @@ struct kvm_s390_ucas_mapping {
_IOW(KVMIO, 0x67, struct kvm_coalesced_mmio_zone)
#define KVM_UNREGISTER_COALESCED_MMIO \
_IOW(KVMIO, 0x68, struct kvm_coalesced_mmio_zone)
-#define KVM_ASSIGN_PCI_DEVICE _IOR(KVMIO, 0x69, \
- struct kvm_assigned_pci_dev)
#define KVM_SET_GSI_ROUTING _IOW(KVMIO, 0x6a, struct kvm_irq_routing)
-/* deprecated, replaced by KVM_ASSIGN_DEV_IRQ */
-#define KVM_ASSIGN_IRQ __KVM_DEPRECATED_VM_R_0x70
-#define KVM_ASSIGN_DEV_IRQ _IOW(KVMIO, 0x70, struct kvm_assigned_irq)
#define KVM_REINJECT_CONTROL _IO(KVMIO, 0x71)
-#define KVM_DEASSIGN_PCI_DEVICE _IOW(KVMIO, 0x72, \
- struct kvm_assigned_pci_dev)
-#define KVM_ASSIGN_SET_MSIX_NR _IOW(KVMIO, 0x73, \
- struct kvm_assigned_msix_nr)
-#define KVM_ASSIGN_SET_MSIX_ENTRY _IOW(KVMIO, 0x74, \
- struct kvm_assigned_msix_entry)
-#define KVM_DEASSIGN_DEV_IRQ _IOW(KVMIO, 0x75, struct kvm_assigned_irq)
#define KVM_IRQFD _IOW(KVMIO, 0x76, struct kvm_irqfd)
#define KVM_CREATE_PIT2 _IOW(KVMIO, 0x77, struct kvm_pit_config)
#define KVM_SET_BOOT_CPU_ID _IO(KVMIO, 0x78)
@@ -1533,9 +1478,6 @@ struct kvm_s390_ucas_mapping {
* KVM_CAP_VM_TSC_CONTROL to set defaults for a VM */
#define KVM_SET_TSC_KHZ _IO(KVMIO, 0xa2)
#define KVM_GET_TSC_KHZ _IO(KVMIO, 0xa3)
-/* Available with KVM_CAP_PCI_2_3 */
-#define KVM_ASSIGN_SET_INTX_MASK _IOW(KVMIO, 0xa4, \
- struct kvm_assigned_pci_dev)
/* Available with KVM_CAP_SIGNAL_MSI */
#define KVM_SIGNAL_MSI _IOW(KVMIO, 0xa5, struct kvm_msi)
/* Available with KVM_CAP_PPC_GET_SMMU_INFO */
@@ -1588,8 +1530,6 @@ struct kvm_s390_ucas_mapping {
#define KVM_SET_SREGS _IOW(KVMIO, 0x84, struct kvm_sregs)
#define KVM_TRANSLATE _IOWR(KVMIO, 0x85, struct kvm_translation)
#define KVM_INTERRUPT _IOW(KVMIO, 0x86, struct kvm_interrupt)
-/* KVM_DEBUG_GUEST is no longer supported, use KVM_SET_GUEST_DEBUG instead */
-#define KVM_DEBUG_GUEST __KVM_DEPRECATED_VCPU_W_0x87
#define KVM_GET_MSRS _IOWR(KVMIO, 0x88, struct kvm_msrs)
#define KVM_SET_MSRS _IOW(KVMIO, 0x89, struct kvm_msrs)
#define KVM_SET_CPUID _IOW(KVMIO, 0x8a, struct kvm_cpuid)
@@ -2263,4 +2203,24 @@ struct kvm_s390_zpci_op {
/* flags for kvm_s390_zpci_op->u.reg_aen.flags */
#define KVM_S390_ZPCIOP_REGAEN_HOST (1 << 0)
+/* Available with KVM_CAP_MEMORY_ATTRIBUTES */
+#define KVM_SET_MEMORY_ATTRIBUTES _IOW(KVMIO, 0xd2, struct kvm_memory_attributes)
+
+struct kvm_memory_attributes {
+ __u64 address;
+ __u64 size;
+ __u64 attributes;
+ __u64 flags;
+};
+
+#define KVM_MEMORY_ATTRIBUTE_PRIVATE (1ULL << 3)
+
+#define KVM_CREATE_GUEST_MEMFD _IOWR(KVMIO, 0xd4, struct kvm_create_guest_memfd)
+
+struct kvm_create_guest_memfd {
+ __u64 size;
+ __u64 flags;
+ __u64 reserved[6];
+};
+
#endif /* __LINUX_KVM_H */
--git a/linux-headers/linux/userfaultfd.h b/linux-headers/linux/userfaultfd.h
index 953c75fedae9..4283de22d5b6 100644
--- a/linux-headers/linux/userfaultfd.h
+++ b/linux-headers/linux/userfaultfd.h
@@ -41,7 +41,8 @@
UFFD_FEATURE_WP_HUGETLBFS_SHMEM | \
UFFD_FEATURE_WP_UNPOPULATED | \
UFFD_FEATURE_POISON | \
- UFFD_FEATURE_WP_ASYNC)
+ UFFD_FEATURE_WP_ASYNC | \
+ UFFD_FEATURE_MOVE)
#define UFFD_API_IOCTLS \
((__u64)1 << _UFFDIO_REGISTER | \
(__u64)1 << _UFFDIO_UNREGISTER | \
@@ -50,6 +51,7 @@
((__u64)1 << _UFFDIO_WAKE | \
(__u64)1 << _UFFDIO_COPY | \
(__u64)1 << _UFFDIO_ZEROPAGE | \
+ (__u64)1 << _UFFDIO_MOVE | \
(__u64)1 << _UFFDIO_WRITEPROTECT | \
(__u64)1 << _UFFDIO_CONTINUE | \
(__u64)1 << _UFFDIO_POISON)
@@ -73,6 +75,7 @@
#define _UFFDIO_WAKE (0x02)
#define _UFFDIO_COPY (0x03)
#define _UFFDIO_ZEROPAGE (0x04)
+#define _UFFDIO_MOVE (0x05)
#define _UFFDIO_WRITEPROTECT (0x06)
#define _UFFDIO_CONTINUE (0x07)
#define _UFFDIO_POISON (0x08)
@@ -92,6 +95,8 @@
struct uffdio_copy)
#define UFFDIO_ZEROPAGE _IOWR(UFFDIO, _UFFDIO_ZEROPAGE, \
struct uffdio_zeropage)
+#define UFFDIO_MOVE _IOWR(UFFDIO, _UFFDIO_MOVE, \
+ struct uffdio_move)
#define UFFDIO_WRITEPROTECT _IOWR(UFFDIO, _UFFDIO_WRITEPROTECT, \
struct uffdio_writeprotect)
#define UFFDIO_CONTINUE _IOWR(UFFDIO, _UFFDIO_CONTINUE, \
@@ -222,6 +227,9 @@ struct uffdio_api {
* asynchronous mode is supported in which the write fault is
* automatically resolved and write-protection is un-set.
* It implies UFFD_FEATURE_WP_UNPOPULATED.
+ *
+ * UFFD_FEATURE_MOVE indicates that the kernel supports moving an
+ * existing page contents from userspace.
*/
#define UFFD_FEATURE_PAGEFAULT_FLAG_WP (1<<0)
#define UFFD_FEATURE_EVENT_FORK (1<<1)
@@ -239,6 +247,7 @@ struct uffdio_api {
#define UFFD_FEATURE_WP_UNPOPULATED (1<<13)
#define UFFD_FEATURE_POISON (1<<14)
#define UFFD_FEATURE_WP_ASYNC (1<<15)
+#define UFFD_FEATURE_MOVE (1<<16)
__u64 features;
__u64 ioctls;
@@ -347,6 +356,24 @@ struct uffdio_poison {
__s64 updated;
};
+struct uffdio_move {
+ __u64 dst;
+ __u64 src;
+ __u64 len;
+ /*
+ * Especially if used to atomically remove memory from the
+ * address space the wake on the dst range is not needed.
+ */
+#define UFFDIO_MOVE_MODE_DONTWAKE ((__u64)1<<0)
+#define UFFDIO_MOVE_MODE_ALLOW_SRC_HOLES ((__u64)1<<1)
+ __u64 mode;
+ /*
+ * "move" is written by the ioctl and must be at the end: the
+ * copy_from_user will not read the last 8 bytes.
+ */
+ __s64 move;
+};
+
/*
* Flags for the userfaultfd(2) system call itself.
*/
--git a/linux-headers/linux/vfio.h b/linux-headers/linux/vfio.h
index 8e175ece31ce..b4be37b2255c 100644
--- a/linux-headers/linux/vfio.h
+++ b/linux-headers/linux/vfio.h
@@ -1219,6 +1219,7 @@ enum vfio_device_mig_state {
VFIO_DEVICE_STATE_RUNNING_P2P = 5,
VFIO_DEVICE_STATE_PRE_COPY = 6,
VFIO_DEVICE_STATE_PRE_COPY_P2P = 7,
+ VFIO_DEVICE_STATE_NR,
};
/**
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 02/65] RAMBlock: Add support of KVM private guest memfd
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
2024-02-29 6:36 ` [PATCH v5 01/65] linux-headers: Update to Linux v6.8-rc5 Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-02-29 6:36 ` [PATCH v5 03/65] HostMem: Add mechanism to opt in kvm guest memfd via MachineState Xiaoyao Li
` (62 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
Add KVM guest_memfd support to RAMBlock so both normal hva based memory
and kvm guest memfd based private memory can be associated in one RAMBlock.
Introduce new flag RAM_GUEST_MEMFD. When it's set, it calls KVM ioctl to
create private guest_memfd during RAMBlock setup.
Allocating a new RAM_GUEST_MEMFD flag to instruct the setup of guest memfd
is more flexible and extensible than simply relying on the VM type because
in the future we may have the case that not all the memory of a VM need
guest memfd. As a benefit, it also avoid getting MachineState in memory
subsystem.
Note, RAM_GUEST_MEMFD is supposed to be set for memory backends of
confidential guests, such as TDX VM. How and when to set it for memory
backends will be implemented in the following patches.
Introduce memory_region_has_guest_memfd() to query if the MemoryRegion has
KVM guest_memfd allocated.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
---
Changes in v5:
- Use assert(new_block->guest_memfd < 0) instead of condition check;
- Collect Reviewed-by tag from David;
Changes in v4:
- Add clarification of RAM_GUEST_MEMFD in commit message; (David Hildenbrand)
- refine the return value and error message; (Daniel P. Berrangé)
- remove flags in ram_block_add(); (David Hildenbrand)
Changes in v3:
- rename gmem to guest_memfd;
- close(guest_memfd) when RAMBlock is released; (Daniel P. Berrangé)
- Suqash the patch that introduces memory_region_has_guest_memfd().
---
accel/kvm/kvm-all.c | 25 +++++++++++++++++++++++++
accel/stubs/kvm-stub.c | 5 +++++
include/exec/memory.h | 20 +++++++++++++++++---
include/exec/ram_addr.h | 2 +-
include/exec/ramblock.h | 1 +
include/sysemu/kvm.h | 2 ++
system/memory.c | 5 +++++
system/physmem.c | 24 +++++++++++++++++++++---
8 files changed, 77 insertions(+), 7 deletions(-)
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index a8cecd040ebc..1c14a80284e6 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -91,6 +91,7 @@ bool kvm_msi_use_devid;
static bool kvm_has_guest_debug;
static int kvm_sstep_flags;
static bool kvm_immediate_exit;
+static bool kvm_guest_memfd_supported;
static hwaddr kvm_max_slot_size = ~0;
static const KVMCapabilityInfo kvm_required_capabilites[] = {
@@ -2395,6 +2396,8 @@ static int kvm_init(MachineState *ms)
}
s->as = g_new0(struct KVMAs, s->nr_as);
+ kvm_guest_memfd_supported = kvm_check_extension(s, KVM_CAP_GUEST_MEMFD);
+
if (object_property_find(OBJECT(current_machine), "kvm-type")) {
g_autofree char *kvm_type = object_property_get_str(OBJECT(current_machine),
"kvm-type",
@@ -4094,3 +4097,25 @@ void query_stats_schemas_cb(StatsSchemaList **result, Error **errp)
query_stats_schema_vcpu(first_cpu, &stats_args);
}
}
+
+int kvm_create_guest_memfd(uint64_t size, uint64_t flags, Error **errp)
+{
+ int fd;
+ struct kvm_create_guest_memfd guest_memfd = {
+ .size = size,
+ .flags = flags,
+ };
+
+ if (!kvm_guest_memfd_supported) {
+ error_setg(errp, "KVM doesn't support guest memfd\n");
+ return -1;
+ }
+
+ fd = kvm_vm_ioctl(kvm_state, KVM_CREATE_GUEST_MEMFD, &guest_memfd);
+ if (fd < 0) {
+ error_setg_errno(errp, errno, "Error creating kvm guest memfd");
+ return -1;
+ }
+
+ return fd;
+}
diff --git a/accel/stubs/kvm-stub.c b/accel/stubs/kvm-stub.c
index ca3817288401..8e0eb22e61cf 100644
--- a/accel/stubs/kvm-stub.c
+++ b/accel/stubs/kvm-stub.c
@@ -129,3 +129,8 @@ bool kvm_hwpoisoned_mem(void)
{
return false;
}
+
+int kvm_create_guest_memfd(uint64_t size, uint64_t flags, Error **errp)
+{
+ return -ENOSYS;
+}
diff --git a/include/exec/memory.h b/include/exec/memory.h
index 8626a355b310..679a8476852e 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -243,6 +243,9 @@ typedef struct IOMMUTLBEvent {
/* RAM FD is opened read-only */
#define RAM_READONLY_FD (1 << 11)
+/* RAM can be private that has kvm guest memfd backend */
+#define RAM_GUEST_MEMFD (1 << 12)
+
static inline void iommu_notifier_init(IOMMUNotifier *n, IOMMUNotify fn,
IOMMUNotifierFlag flags,
hwaddr start, hwaddr end,
@@ -1307,7 +1310,8 @@ bool memory_region_init_ram_nomigrate(MemoryRegion *mr,
* @name: Region name, becomes part of RAMBlock name used in migration stream
* must be unique within any device
* @size: size of the region.
- * @ram_flags: RamBlock flags. Supported flags: RAM_SHARED, RAM_NORESERVE.
+ * @ram_flags: RamBlock flags. Supported flags: RAM_SHARED, RAM_NORESERVE,
+ * RAM_GUEST_MEMFD.
* @errp: pointer to Error*, to store an error if it happens.
*
* Note that this function does not do anything to cause the data in the
@@ -1369,7 +1373,7 @@ bool memory_region_init_resizeable_ram(MemoryRegion *mr,
* (getpagesize()) will be used.
* @ram_flags: RamBlock flags. Supported flags: RAM_SHARED, RAM_PMEM,
* RAM_NORESERVE, RAM_PROTECTED, RAM_NAMED_FILE, RAM_READONLY,
- * RAM_READONLY_FD
+ * RAM_READONLY_FD, RAM_GUEST_MEMFD
* @path: the path in which to allocate the RAM.
* @offset: offset within the file referenced by path
* @errp: pointer to Error*, to store an error if it happens.
@@ -1399,7 +1403,7 @@ bool memory_region_init_ram_from_file(MemoryRegion *mr,
* @size: size of the region.
* @ram_flags: RamBlock flags. Supported flags: RAM_SHARED, RAM_PMEM,
* RAM_NORESERVE, RAM_PROTECTED, RAM_NAMED_FILE, RAM_READONLY,
- * RAM_READONLY_FD
+ * RAM_READONLY_FD, RAM_GUEST_MEMFD
* @fd: the fd to mmap.
* @offset: offset within the file referenced by fd
* @errp: pointer to Error*, to store an error if it happens.
@@ -1722,6 +1726,16 @@ static inline bool memory_region_is_romd(MemoryRegion *mr)
*/
bool memory_region_is_protected(MemoryRegion *mr);
+/**
+ * memory_region_has_guest_memfd: check whether a memory region has guest_memfd
+ * associated
+ *
+ * Returns %true if a memory region's ram_block has valid guest_memfd assigned.
+ *
+ * @mr: the memory region being queried
+ */
+bool memory_region_has_guest_memfd(MemoryRegion *mr);
+
/**
* memory_region_get_iommu: check whether a memory region is an iommu
*
diff --git a/include/exec/ram_addr.h b/include/exec/ram_addr.h
index 90676093f5d5..4ebd9ded5e86 100644
--- a/include/exec/ram_addr.h
+++ b/include/exec/ram_addr.h
@@ -109,7 +109,7 @@ long qemu_maxrampagesize(void);
* @mr: the memory region where the ram block is
* @ram_flags: RamBlock flags. Supported flags: RAM_SHARED, RAM_PMEM,
* RAM_NORESERVE, RAM_PROTECTED, RAM_NAMED_FILE, RAM_READONLY,
- * RAM_READONLY_FD
+ * RAM_READONLY_FD, RAM_GUEST_MEMFD
* @mem_path or @fd: specify the backing file or device
* @offset: Offset into target file
* @errp: pointer to Error*, to store an error if it happens
diff --git a/include/exec/ramblock.h b/include/exec/ramblock.h
index 3eb79723c6a8..03b3a3d40a6d 100644
--- a/include/exec/ramblock.h
+++ b/include/exec/ramblock.h
@@ -41,6 +41,7 @@ struct RAMBlock {
QLIST_HEAD(, RAMBlockNotifier) ramblock_notifiers;
int fd;
uint64_t fd_offset;
+ int guest_memfd;
size_t page_size;
/* dirty bitmap used during migration */
unsigned long *bmap;
diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
index fad9a7e8ff30..6cdf82de8372 100644
--- a/include/sysemu/kvm.h
+++ b/include/sysemu/kvm.h
@@ -544,4 +544,6 @@ uint32_t kvm_dirty_ring_size(void);
* reported for the VM.
*/
bool kvm_hwpoisoned_mem(void);
+
+int kvm_create_guest_memfd(uint64_t size, uint64_t flags, Error **errp);
#endif
diff --git a/system/memory.c b/system/memory.c
index a229a79988fc..c756950c0c0f 100644
--- a/system/memory.c
+++ b/system/memory.c
@@ -1850,6 +1850,11 @@ bool memory_region_is_protected(MemoryRegion *mr)
return mr->ram && (mr->ram_block->flags & RAM_PROTECTED);
}
+bool memory_region_has_guest_memfd(MemoryRegion *mr)
+{
+ return mr->ram_block && mr->ram_block->guest_memfd >= 0;
+}
+
uint8_t memory_region_get_dirty_log_mask(MemoryRegion *mr)
{
uint8_t mask = mr->dirty_log_mask;
diff --git a/system/physmem.c b/system/physmem.c
index e3ebc19eefd8..c4fc1e506de8 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -1841,6 +1841,17 @@ static void ram_block_add(RAMBlock *new_block, Error **errp)
}
}
+ if (kvm_enabled() && (new_block->flags & RAM_GUEST_MEMFD)) {
+ assert(new_block->guest_memfd < 0);
+
+ new_block->guest_memfd = kvm_create_guest_memfd(new_block->max_length,
+ 0, errp);
+ if (new_block->guest_memfd < 0) {
+ qemu_mutex_unlock_ramlist();
+ return;
+ }
+ }
+
new_ram_size = MAX(old_ram_size,
(new_block->offset + new_block->max_length) >> TARGET_PAGE_BITS);
if (new_ram_size > old_ram_size) {
@@ -1903,7 +1914,7 @@ RAMBlock *qemu_ram_alloc_from_fd(ram_addr_t size, MemoryRegion *mr,
/* Just support these ram flags by now. */
assert((ram_flags & ~(RAM_SHARED | RAM_PMEM | RAM_NORESERVE |
RAM_PROTECTED | RAM_NAMED_FILE | RAM_READONLY |
- RAM_READONLY_FD)) == 0);
+ RAM_READONLY_FD | RAM_GUEST_MEMFD)) == 0);
if (xen_enabled()) {
error_setg(errp, "-mem-path not supported with Xen");
@@ -1938,6 +1949,7 @@ RAMBlock *qemu_ram_alloc_from_fd(ram_addr_t size, MemoryRegion *mr,
new_block->used_length = size;
new_block->max_length = size;
new_block->flags = ram_flags;
+ new_block->guest_memfd = -1;
new_block->host = file_ram_alloc(new_block, size, fd, !file_size, offset,
errp);
if (!new_block->host) {
@@ -2016,7 +2028,7 @@ RAMBlock *qemu_ram_alloc_internal(ram_addr_t size, ram_addr_t max_size,
Error *local_err = NULL;
assert((ram_flags & ~(RAM_SHARED | RAM_RESIZEABLE | RAM_PREALLOC |
- RAM_NORESERVE)) == 0);
+ RAM_NORESERVE| RAM_GUEST_MEMFD)) == 0);
assert(!host ^ (ram_flags & RAM_PREALLOC));
size = HOST_PAGE_ALIGN(size);
@@ -2028,6 +2040,7 @@ RAMBlock *qemu_ram_alloc_internal(ram_addr_t size, ram_addr_t max_size,
new_block->max_length = max_size;
assert(max_size >= size);
new_block->fd = -1;
+ new_block->guest_memfd = -1;
new_block->page_size = qemu_real_host_page_size();
new_block->host = host;
new_block->flags = ram_flags;
@@ -2050,7 +2063,7 @@ RAMBlock *qemu_ram_alloc_from_ptr(ram_addr_t size, void *host,
RAMBlock *qemu_ram_alloc(ram_addr_t size, uint32_t ram_flags,
MemoryRegion *mr, Error **errp)
{
- assert((ram_flags & ~(RAM_SHARED | RAM_NORESERVE)) == 0);
+ assert((ram_flags & ~(RAM_SHARED | RAM_NORESERVE | RAM_GUEST_MEMFD)) == 0);
return qemu_ram_alloc_internal(size, size, NULL, NULL, ram_flags, mr, errp);
}
@@ -2078,6 +2091,11 @@ static void reclaim_ramblock(RAMBlock *block)
} else {
qemu_anon_ram_free(block->host, block->max_length);
}
+
+ if (block->guest_memfd >= 0) {
+ close(block->guest_memfd);
+ }
+
g_free(block);
}
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 03/65] HostMem: Add mechanism to opt in kvm guest memfd via MachineState
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
2024-02-29 6:36 ` [PATCH v5 01/65] linux-headers: Update to Linux v6.8-rc5 Xiaoyao Li
2024-02-29 6:36 ` [PATCH v5 02/65] RAMBlock: Add support of KVM private guest memfd Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-02-29 6:36 ` [PATCH v5 04/65] trace/kvm: Split address space and slot id in trace_kvm_set_user_memory() Xiaoyao Li
` (61 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
Add a new member "guest_memfd" to memory backends. When it's set
to true, it enables RAM_GUEST_MEMFD in ram_flags, thus private kvm
guest_memfd will be allocated during RAMBlock allocation.
Memory backend's @guest_memfd is wired with @require_guest_memfd
field of MachineState. It avoid looking up the machine in phymem.c.
MachineState::require_guest_memfd is supposed to be set by any VMs
that requires KVM guest memfd as private memory, e.g., TDX VM.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
---
Changes in v4:
- rename "require_guest_memfd" to "guest_memfd" in struct
HostMemoryBackend; (David Hildenbrand)
---
backends/hostmem-file.c | 1 +
backends/hostmem-memfd.c | 1 +
backends/hostmem-ram.c | 1 +
backends/hostmem.c | 1 +
hw/core/machine.c | 5 +++++
include/hw/boards.h | 2 ++
include/sysemu/hostmem.h | 1 +
7 files changed, 12 insertions(+)
diff --git a/backends/hostmem-file.c b/backends/hostmem-file.c
index ac3e433cbddb..3c69db794607 100644
--- a/backends/hostmem-file.c
+++ b/backends/hostmem-file.c
@@ -85,6 +85,7 @@ file_backend_memory_alloc(HostMemoryBackend *backend, Error **errp)
ram_flags |= fb->readonly ? RAM_READONLY_FD : 0;
ram_flags |= fb->rom == ON_OFF_AUTO_ON ? RAM_READONLY : 0;
ram_flags |= backend->reserve ? 0 : RAM_NORESERVE;
+ ram_flags |= backend->guest_memfd ? RAM_GUEST_MEMFD : 0;
ram_flags |= fb->is_pmem ? RAM_PMEM : 0;
ram_flags |= RAM_NAMED_FILE;
return memory_region_init_ram_from_file(&backend->mr, OBJECT(backend), name,
diff --git a/backends/hostmem-memfd.c b/backends/hostmem-memfd.c
index 3923ea9364d5..745ead0034dc 100644
--- a/backends/hostmem-memfd.c
+++ b/backends/hostmem-memfd.c
@@ -55,6 +55,7 @@ memfd_backend_memory_alloc(HostMemoryBackend *backend, Error **errp)
name = host_memory_backend_get_name(backend);
ram_flags = backend->share ? RAM_SHARED : 0;
ram_flags |= backend->reserve ? 0 : RAM_NORESERVE;
+ ram_flags |= backend->guest_memfd ? RAM_GUEST_MEMFD : 0;
return memory_region_init_ram_from_fd(&backend->mr, OBJECT(backend), name,
backend->size, ram_flags, fd, 0, errp);
}
diff --git a/backends/hostmem-ram.c b/backends/hostmem-ram.c
index d121249f0f45..f7d81af783a5 100644
--- a/backends/hostmem-ram.c
+++ b/backends/hostmem-ram.c
@@ -30,6 +30,7 @@ ram_backend_memory_alloc(HostMemoryBackend *backend, Error **errp)
name = host_memory_backend_get_name(backend);
ram_flags = backend->share ? RAM_SHARED : 0;
ram_flags |= backend->reserve ? 0 : RAM_NORESERVE;
+ ram_flags |= backend->guest_memfd ? RAM_GUEST_MEMFD : 0;
return memory_region_init_ram_flags_nomigrate(&backend->mr, OBJECT(backend),
name, backend->size,
ram_flags, errp);
diff --git a/backends/hostmem.c b/backends/hostmem.c
index 81a72ce40b78..eb9682b4a856 100644
--- a/backends/hostmem.c
+++ b/backends/hostmem.c
@@ -277,6 +277,7 @@ static void host_memory_backend_init(Object *obj)
/* TODO: convert access to globals to compat properties */
backend->merge = machine_mem_merge(machine);
backend->dump = machine_dump_guest_core(machine);
+ backend->guest_memfd = machine_require_guest_memfd(machine);
backend->reserve = true;
backend->prealloc_threads = machine->smp.cpus;
}
diff --git a/hw/core/machine.c b/hw/core/machine.c
index 9ac5d5389a6c..651a3d463a06 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
@@ -1192,6 +1192,11 @@ bool machine_mem_merge(MachineState *machine)
return machine->mem_merge;
}
+bool machine_require_guest_memfd(MachineState *machine)
+{
+ return machine->require_guest_memfd;
+}
+
static char *cpu_slot_to_string(const CPUArchId *cpu)
{
GString *s = g_string_new(NULL);
diff --git a/include/hw/boards.h b/include/hw/boards.h
index bcfde8a84d10..2e6156ae96b8 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -36,6 +36,7 @@ bool machine_usb(MachineState *machine);
int machine_phandle_start(MachineState *machine);
bool machine_dump_guest_core(MachineState *machine);
bool machine_mem_merge(MachineState *machine);
+bool machine_require_guest_memfd(MachineState *machine);
HotpluggableCPUList *machine_query_hotpluggable_cpus(MachineState *machine);
void machine_set_cpu_numa_node(MachineState *machine,
const CpuInstanceProperties *props,
@@ -370,6 +371,7 @@ struct MachineState {
char *dt_compatible;
bool dump_guest_core;
bool mem_merge;
+ bool require_guest_memfd;
bool usb;
bool usb_disabled;
char *firmware;
diff --git a/include/sysemu/hostmem.h b/include/sysemu/hostmem.h
index 0e411aaa29e1..04b884bf42a6 100644
--- a/include/sysemu/hostmem.h
+++ b/include/sysemu/hostmem.h
@@ -74,6 +74,7 @@ struct HostMemoryBackend {
uint64_t size;
bool merge, dump, use_canonical_path;
bool prealloc, is_mapped, share, reserve;
+ bool guest_memfd;
uint32_t prealloc_threads;
ThreadContext *prealloc_context;
DECLARE_BITMAP(host_nodes, MAX_NODES + 1);
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 04/65] trace/kvm: Split address space and slot id in trace_kvm_set_user_memory()
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (2 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 03/65] HostMem: Add mechanism to opt in kvm guest memfd via MachineState Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-02-29 6:36 ` [PATCH v5 05/65] kvm: Enable KVM_SET_USER_MEMORY_REGION2 for memslot Xiaoyao Li
` (60 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
The upper 16 bits of kvm_userspace_memory_region::slot are
address space id. Parse it separately in trace_kvm_set_user_memory().
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
accel/kvm/kvm-all.c | 5 +++--
accel/kvm/trace-events | 2 +-
2 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 1c14a80284e6..6c36c452d60b 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -304,8 +304,9 @@ static int kvm_set_user_memory_region(KVMMemoryListener *kml, KVMSlot *slot, boo
ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION, &mem);
slot->old_flags = mem.flags;
err:
- trace_kvm_set_user_memory(mem.slot, mem.flags, mem.guest_phys_addr,
- mem.memory_size, mem.userspace_addr, ret);
+ trace_kvm_set_user_memory(mem.slot >> 16, (uint16_t)mem.slot, mem.flags,
+ mem.guest_phys_addr, mem.memory_size,
+ mem.userspace_addr, ret);
if (ret < 0) {
error_report("%s: KVM_SET_USER_MEMORY_REGION failed, slot=%d,"
" start=0x%" PRIx64 ", size=0x%" PRIx64 ": %s",
diff --git a/accel/kvm/trace-events b/accel/kvm/trace-events
index a25902597b1b..9f599abc172c 100644
--- a/accel/kvm/trace-events
+++ b/accel/kvm/trace-events
@@ -15,7 +15,7 @@ kvm_irqchip_update_msi_route(int virq) "Updating MSI route virq=%d"
kvm_irqchip_release_virq(int virq) "virq %d"
kvm_set_ioeventfd_mmio(int fd, uint64_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%" PRIx64 " val=0x%x assign: %d size: %d match: %d"
kvm_set_ioeventfd_pio(int fd, uint16_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%x val=0x%x assign: %d size: %d match: %d"
-kvm_set_user_memory(uint32_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, int ret) "Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " ret=%d"
+kvm_set_user_memory(uint16_t as, uint16_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, int ret) "AddrSpace#%d Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " ret=%d"
kvm_clear_dirty_log(uint32_t slot, uint64_t start, uint32_t size) "slot#%"PRId32" start 0x%"PRIx64" size 0x%"PRIx32
kvm_resample_fd_notify(int gsi) "gsi %d"
kvm_dirty_ring_full(int id) "vcpu %d"
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 05/65] kvm: Enable KVM_SET_USER_MEMORY_REGION2 for memslot
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (3 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 04/65] trace/kvm: Split address space and slot id in trace_kvm_set_user_memory() Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-02-29 6:36 ` [PATCH v5 06/65] kvm: Introduce support for memory_attributes Xiaoyao Li
` (59 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
From: Chao Peng <chao.p.peng@linux.intel.com>
Switch to KVM_SET_USER_MEMORY_REGION2 when supported by KVM.
With KVM_SET_USER_MEMORY_REGION2, QEMU can set up memory region that
backend'ed both by hva-based shared memory and guest memfd based private
memory.
Signed-off-by: Chao Peng <chao.p.peng@linux.intel.com>
Co-developed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
Changes in v4:
- update KVM_MEM_PRIVATE to KVM_MEM_GUEST_MEMFD; (Isaku)
---
accel/kvm/kvm-all.c | 56 ++++++++++++++++++++++++++++++++++------
accel/kvm/trace-events | 2 +-
include/sysemu/kvm_int.h | 2 ++
3 files changed, 51 insertions(+), 9 deletions(-)
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 6c36c452d60b..cd0aa7545a1f 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -283,35 +283,69 @@ int kvm_physical_memory_addr_from_host(KVMState *s, void *ram,
static int kvm_set_user_memory_region(KVMMemoryListener *kml, KVMSlot *slot, bool new)
{
KVMState *s = kvm_state;
- struct kvm_userspace_memory_region mem;
+ struct kvm_userspace_memory_region2 mem;
+ static int cap_user_memory2 = -1;
int ret;
+ if (cap_user_memory2 == -1) {
+ cap_user_memory2 = kvm_check_extension(s, KVM_CAP_USER_MEMORY2);
+ }
+
+ if (!cap_user_memory2 && slot->guest_memfd >= 0) {
+ error_report("%s, KVM doesn't support KVM_CAP_USER_MEMORY2,"
+ " which is required by guest memfd!", __func__);
+ exit(1);
+ }
+
mem.slot = slot->slot | (kml->as_id << 16);
mem.guest_phys_addr = slot->start_addr;
mem.userspace_addr = (unsigned long)slot->ram;
mem.flags = slot->flags;
+ mem.guest_memfd = slot->guest_memfd;
+ mem.guest_memfd_offset = slot->guest_memfd_offset;
if (slot->memory_size && !new && (mem.flags ^ slot->old_flags) & KVM_MEM_READONLY) {
/* Set the slot size to 0 before setting the slot to the desired
* value. This is needed based on KVM commit 75d61fbc. */
mem.memory_size = 0;
- ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION, &mem);
+
+ if (cap_user_memory2) {
+ ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION2, &mem);
+ } else {
+ ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION, &mem);
+ }
if (ret < 0) {
goto err;
}
}
mem.memory_size = slot->memory_size;
- ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION, &mem);
+ if (cap_user_memory2) {
+ ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION2, &mem);
+ } else {
+ ret = kvm_vm_ioctl(s, KVM_SET_USER_MEMORY_REGION, &mem);
+ }
slot->old_flags = mem.flags;
err:
trace_kvm_set_user_memory(mem.slot >> 16, (uint16_t)mem.slot, mem.flags,
mem.guest_phys_addr, mem.memory_size,
- mem.userspace_addr, ret);
+ mem.userspace_addr, mem.guest_memfd,
+ mem.guest_memfd_offset, ret);
if (ret < 0) {
- error_report("%s: KVM_SET_USER_MEMORY_REGION failed, slot=%d,"
- " start=0x%" PRIx64 ", size=0x%" PRIx64 ": %s",
- __func__, mem.slot, slot->start_addr,
- (uint64_t)mem.memory_size, strerror(errno));
+ if (cap_user_memory2) {
+ error_report("%s: KVM_SET_USER_MEMORY_REGION2 failed, slot=%d,"
+ " start=0x%" PRIx64 ", size=0x%" PRIx64 ","
+ " flags=0x%" PRIx32 ", guest_memfd=%" PRId32 ","
+ " guest_memfd_offset=0x%" PRIx64 ": %s",
+ __func__, mem.slot, slot->start_addr,
+ (uint64_t)mem.memory_size, mem.flags,
+ mem.guest_memfd, (uint64_t)mem.guest_memfd_offset,
+ strerror(errno));
+ } else {
+ error_report("%s: KVM_SET_USER_MEMORY_REGION failed, slot=%d,"
+ " start=0x%" PRIx64 ", size=0x%" PRIx64 ": %s",
+ __func__, mem.slot, slot->start_addr,
+ (uint64_t)mem.memory_size, strerror(errno));
+ }
}
return ret;
}
@@ -466,6 +500,9 @@ static int kvm_mem_flags(MemoryRegion *mr)
if (readonly && kvm_readonly_mem_allowed) {
flags |= KVM_MEM_READONLY;
}
+ if (memory_region_has_guest_memfd(mr)) {
+ flags |= KVM_MEM_GUEST_MEMFD;
+ }
return flags;
}
@@ -1363,6 +1400,9 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml,
mem->ram_start_offset = ram_start_offset;
mem->ram = ram;
mem->flags = kvm_mem_flags(mr);
+ mem->guest_memfd = mr->ram_block->guest_memfd;
+ mem->guest_memfd_offset = (uint8_t*)ram - mr->ram_block->host;
+
kvm_slot_init_dirty_bitmap(mem);
err = kvm_set_user_memory_region(kml, mem, true);
if (err) {
diff --git a/accel/kvm/trace-events b/accel/kvm/trace-events
index 9f599abc172c..e8c52cb9e7a1 100644
--- a/accel/kvm/trace-events
+++ b/accel/kvm/trace-events
@@ -15,7 +15,7 @@ kvm_irqchip_update_msi_route(int virq) "Updating MSI route virq=%d"
kvm_irqchip_release_virq(int virq) "virq %d"
kvm_set_ioeventfd_mmio(int fd, uint64_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%" PRIx64 " val=0x%x assign: %d size: %d match: %d"
kvm_set_ioeventfd_pio(int fd, uint16_t addr, uint32_t val, bool assign, uint32_t size, bool datamatch) "fd: %d @0x%x val=0x%x assign: %d size: %d match: %d"
-kvm_set_user_memory(uint16_t as, uint16_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, int ret) "AddrSpace#%d Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " ret=%d"
+kvm_set_user_memory(uint16_t as, uint16_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, uint32_t fd, uint64_t fd_offset, int ret) "AddrSpace#%d Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " guest_memfd=%d" " guest_memfd_offset=0x%" PRIx64 " ret=%d"
kvm_clear_dirty_log(uint32_t slot, uint64_t start, uint32_t size) "slot#%"PRId32" start 0x%"PRIx64" size 0x%"PRIx32
kvm_resample_fd_notify(int gsi) "gsi %d"
kvm_dirty_ring_full(int id) "vcpu %d"
diff --git a/include/sysemu/kvm_int.h b/include/sysemu/kvm_int.h
index 882e37e12c5b..cad763e240c1 100644
--- a/include/sysemu/kvm_int.h
+++ b/include/sysemu/kvm_int.h
@@ -30,6 +30,8 @@ typedef struct KVMSlot
int as_id;
/* Cache of the offset in ram address space */
ram_addr_t ram_start_offset;
+ int guest_memfd;
+ hwaddr guest_memfd_offset;
} KVMSlot;
typedef struct KVMMemoryUpdate {
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 06/65] kvm: Introduce support for memory_attributes
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (4 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 05/65] kvm: Enable KVM_SET_USER_MEMORY_REGION2 for memslot Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-03-19 2:03 ` Wang, Lei
2024-02-29 6:36 ` [PATCH v5 07/65] physmem: Introduce ram_block_discard_guest_memfd_range() Xiaoyao Li
` (58 subsequent siblings)
64 siblings, 1 reply; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
Introduce the helper functions to set the attributes of a range of
memory to private or shared.
This is necessary to notify KVM the private/shared attribute of each gpa
range. KVM needs the information to decide the GPA needs to be mapped at
hva-based shared memory or guest_memfd based private memory.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
Changes in v4:
- move the check of kvm_supported_memory_attributes to the common
kvm_set_memory_attributes(); (Wang Wei)
- change warn_report() to error_report() in kvm_set_memory_attributes()
and drop the __func__; (Daniel)
---
accel/kvm/kvm-all.c | 44 ++++++++++++++++++++++++++++++++++++++++++++
include/sysemu/kvm.h | 3 +++
2 files changed, 47 insertions(+)
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index cd0aa7545a1f..70d482a2c936 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -92,6 +92,7 @@ static bool kvm_has_guest_debug;
static int kvm_sstep_flags;
static bool kvm_immediate_exit;
static bool kvm_guest_memfd_supported;
+static uint64_t kvm_supported_memory_attributes;
static hwaddr kvm_max_slot_size = ~0;
static const KVMCapabilityInfo kvm_required_capabilites[] = {
@@ -1304,6 +1305,46 @@ void kvm_set_max_memslot_size(hwaddr max_slot_size)
kvm_max_slot_size = max_slot_size;
}
+static int kvm_set_memory_attributes(hwaddr start, hwaddr size, uint64_t attr)
+{
+ struct kvm_memory_attributes attrs;
+ int r;
+
+ if (kvm_supported_memory_attributes == 0) {
+ error_report("No memory attribute supported by KVM\n");
+ return -EINVAL;
+ }
+
+ if ((attr & kvm_supported_memory_attributes) != attr) {
+ error_report("memory attribute 0x%lx not supported by KVM,"
+ " supported bits are 0x%lx\n",
+ attr, kvm_supported_memory_attributes);
+ return -EINVAL;
+ }
+
+ attrs.attributes = attr;
+ attrs.address = start;
+ attrs.size = size;
+ attrs.flags = 0;
+
+ r = kvm_vm_ioctl(kvm_state, KVM_SET_MEMORY_ATTRIBUTES, &attrs);
+ if (r) {
+ error_report("failed to set memory (0x%lx+%#zx) with attr 0x%lx error '%s'",
+ start, size, attr, strerror(errno));
+ }
+ return r;
+}
+
+int kvm_set_memory_attributes_private(hwaddr start, hwaddr size)
+{
+ return kvm_set_memory_attributes(start, size, KVM_MEMORY_ATTRIBUTE_PRIVATE);
+}
+
+int kvm_set_memory_attributes_shared(hwaddr start, hwaddr size)
+{
+ return kvm_set_memory_attributes(start, size, 0);
+}
+
/* Called with KVMMemoryListener.slots_lock held */
static void kvm_set_phys_mem(KVMMemoryListener *kml,
MemoryRegionSection *section, bool add)
@@ -2439,6 +2480,9 @@ static int kvm_init(MachineState *ms)
kvm_guest_memfd_supported = kvm_check_extension(s, KVM_CAP_GUEST_MEMFD);
+ ret = kvm_check_extension(s, KVM_CAP_MEMORY_ATTRIBUTES);
+ kvm_supported_memory_attributes = ret > 0 ? ret : 0;
+
if (object_property_find(OBJECT(current_machine), "kvm-type")) {
g_autofree char *kvm_type = object_property_get_str(OBJECT(current_machine),
"kvm-type",
diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
index 6cdf82de8372..8e83adfbbd19 100644
--- a/include/sysemu/kvm.h
+++ b/include/sysemu/kvm.h
@@ -546,4 +546,7 @@ uint32_t kvm_dirty_ring_size(void);
bool kvm_hwpoisoned_mem(void);
int kvm_create_guest_memfd(uint64_t size, uint64_t flags, Error **errp);
+
+int kvm_set_memory_attributes_private(hwaddr start, hwaddr size);
+int kvm_set_memory_attributes_shared(hwaddr start, hwaddr size);
#endif
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* Re: [PATCH v5 06/65] kvm: Introduce support for memory_attributes
2024-02-29 6:36 ` [PATCH v5 06/65] kvm: Introduce support for memory_attributes Xiaoyao Li
@ 2024-03-19 2:03 ` Wang, Lei
2024-03-20 12:44 ` Xiaoyao Li
0 siblings, 1 reply; 124+ messages in thread
From: Wang, Lei @ 2024-03-19 2:03 UTC (permalink / raw)
To: Xiaoyao Li, Paolo Bonzini, David Hildenbrand, Igor Mammedov,
Eduardo Habkost, Marcel Apfelbaum, Philippe Mathieu-Daudé,
Yanan Wang, Michael S. Tsirkin, Richard Henderson, Ani Sinha,
Peter Xu, Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang
On 2/29/2024 14:36, Xiaoyao Li wrote:> Introduce the helper functions to set the attributes of a range of
> memory to private or shared.
>
> This is necessary to notify KVM the private/shared attribute of each gpa
> range. KVM needs the information to decide the GPA needs to be mapped at
> hva-based shared memory or guest_memfd based private memory.
>
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
> Changes in v4:
> - move the check of kvm_supported_memory_attributes to the common
> kvm_set_memory_attributes(); (Wang Wei)
> - change warn_report() to error_report() in kvm_set_memory_attributes()
> and drop the __func__; (Daniel)
> ---
> accel/kvm/kvm-all.c | 44 ++++++++++++++++++++++++++++++++++++++++++++
> include/sysemu/kvm.h | 3 +++
> 2 files changed, 47 insertions(+)
>
> diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
> index cd0aa7545a1f..70d482a2c936 100644
> --- a/accel/kvm/kvm-all.c
> +++ b/accel/kvm/kvm-all.c
> @@ -92,6 +92,7 @@ static bool kvm_has_guest_debug;
> static int kvm_sstep_flags;
> static bool kvm_immediate_exit;
> static bool kvm_guest_memfd_supported;
> +static uint64_t kvm_supported_memory_attributes;
> static hwaddr kvm_max_slot_size = ~0;
>
> static const KVMCapabilityInfo kvm_required_capabilites[] = {
> @@ -1304,6 +1305,46 @@ void kvm_set_max_memslot_size(hwaddr max_slot_size)
> kvm_max_slot_size = max_slot_size;
> }
>
> +static int kvm_set_memory_attributes(hwaddr start, hwaddr size, uint64_t attr)
> +{
> + struct kvm_memory_attributes attrs;
> + int r;
> +
> + if (kvm_supported_memory_attributes == 0) {
> + error_report("No memory attribute supported by KVM\n");
> + return -EINVAL;
> + }
> +
> + if ((attr & kvm_supported_memory_attributes) != attr) {
> + error_report("memory attribute 0x%lx not supported by KVM,"
> + " supported bits are 0x%lx\n",
> + attr, kvm_supported_memory_attributes);
> + return -EINVAL;
> + }
> +
> + attrs.attributes = attr;
> + attrs.address = start;
> + attrs.size = size;
> + attrs.flags = 0;
> +
> + r = kvm_vm_ioctl(kvm_state, KVM_SET_MEMORY_ATTRIBUTES, &attrs);
> + if (r) {
> + error_report("failed to set memory (0x%lx+%#zx) with attr 0x%lx error '%s'",
> + start, size, attr, strerror(errno));
> + }
> + return r;
> +}
> +
> +int kvm_set_memory_attributes_private(hwaddr start, hwaddr size)
> +{
> + return kvm_set_memory_attributes(start, size, KVM_MEMORY_ATTRIBUTE_PRIVATE);
> +}
> +
> +int kvm_set_memory_attributes_shared(hwaddr start, hwaddr size)
> +{
> + return kvm_set_memory_attributes(start, size, 0);
> +}
> +
> /* Called with KVMMemoryListener.slots_lock held */
> static void kvm_set_phys_mem(KVMMemoryListener *kml,
> MemoryRegionSection *section, bool add)
> @@ -2439,6 +2480,9 @@ static int kvm_init(MachineState *ms)
>
> kvm_guest_memfd_supported = kvm_check_extension(s, KVM_CAP_GUEST_MEMFD);
>
> + ret = kvm_check_extension(s, KVM_CAP_MEMORY_ATTRIBUTES);
> + kvm_supported_memory_attributes = ret > 0 ? ret : 0;
kvm_check_extension() only returns non-negative value, so we can just
kvm_supported_memory_attributes = kvm_check_extension(s, KVM_CAP_MEMORY_ATTRIBUTES);
> +
> if (object_property_find(OBJECT(current_machine), "kvm-type")) {
> g_autofree char *kvm_type = object_property_get_str(OBJECT(current_machine),
> "kvm-type",
> diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
> index 6cdf82de8372..8e83adfbbd19 100644
> --- a/include/sysemu/kvm.h
> +++ b/include/sysemu/kvm.h
> @@ -546,4 +546,7 @@ uint32_t kvm_dirty_ring_size(void);
> bool kvm_hwpoisoned_mem(void);
>
> int kvm_create_guest_memfd(uint64_t size, uint64_t flags, Error **errp);
> +
> +int kvm_set_memory_attributes_private(hwaddr start, hwaddr size);
> +int kvm_set_memory_attributes_shared(hwaddr start, hwaddr size);
> #endif
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 06/65] kvm: Introduce support for memory_attributes
2024-03-19 2:03 ` Wang, Lei
@ 2024-03-20 12:44 ` Xiaoyao Li
0 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-03-20 12:44 UTC (permalink / raw)
To: Wang, Lei, Paolo Bonzini, David Hildenbrand, Igor Mammedov,
Eduardo Habkost, Marcel Apfelbaum, Philippe Mathieu-Daudé,
Yanan Wang, Michael S. Tsirkin, Richard Henderson, Ani Sinha,
Peter Xu, Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang
On 3/19/2024 10:03 AM, Wang, Lei wrote:
> On 2/29/2024 14:36, Xiaoyao Li wrote:> Introduce the helper functions to set the attributes of a range of
>> memory to private or shared.
>>
>> This is necessary to notify KVM the private/shared attribute of each gpa
>> range. KVM needs the information to decide the GPA needs to be mapped at
>> hva-based shared memory or guest_memfd based private memory.
>>
>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>> ---
>> Changes in v4:
>> - move the check of kvm_supported_memory_attributes to the common
>> kvm_set_memory_attributes(); (Wang Wei)
>> - change warn_report() to error_report() in kvm_set_memory_attributes()
>> and drop the __func__; (Daniel)
>> ---
>> accel/kvm/kvm-all.c | 44 ++++++++++++++++++++++++++++++++++++++++++++
>> include/sysemu/kvm.h | 3 +++
>> 2 files changed, 47 insertions(+)
>>
>> diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
>> index cd0aa7545a1f..70d482a2c936 100644
>> --- a/accel/kvm/kvm-all.c
>> +++ b/accel/kvm/kvm-all.c
>> @@ -92,6 +92,7 @@ static bool kvm_has_guest_debug;
>> static int kvm_sstep_flags;
>> static bool kvm_immediate_exit;
>> static bool kvm_guest_memfd_supported;
>> +static uint64_t kvm_supported_memory_attributes;
>> static hwaddr kvm_max_slot_size = ~0;
>>
>> static const KVMCapabilityInfo kvm_required_capabilites[] = {
>> @@ -1304,6 +1305,46 @@ void kvm_set_max_memslot_size(hwaddr max_slot_size)
>> kvm_max_slot_size = max_slot_size;
>> }
>>
>> +static int kvm_set_memory_attributes(hwaddr start, hwaddr size, uint64_t attr)
>> +{
>> + struct kvm_memory_attributes attrs;
>> + int r;
>> +
>> + if (kvm_supported_memory_attributes == 0) {
>> + error_report("No memory attribute supported by KVM\n");
>> + return -EINVAL;
>> + }
>> +
>> + if ((attr & kvm_supported_memory_attributes) != attr) {
>> + error_report("memory attribute 0x%lx not supported by KVM,"
>> + " supported bits are 0x%lx\n",
>> + attr, kvm_supported_memory_attributes);
>> + return -EINVAL;
>> + }
>> +
>> + attrs.attributes = attr;
>> + attrs.address = start;
>> + attrs.size = size;
>> + attrs.flags = 0;
>> +
>> + r = kvm_vm_ioctl(kvm_state, KVM_SET_MEMORY_ATTRIBUTES, &attrs);
>> + if (r) {
>> + error_report("failed to set memory (0x%lx+%#zx) with attr 0x%lx error '%s'",
>> + start, size, attr, strerror(errno));
>> + }
>> + return r;
>> +}
>> +
>> +int kvm_set_memory_attributes_private(hwaddr start, hwaddr size)
>> +{
>> + return kvm_set_memory_attributes(start, size, KVM_MEMORY_ATTRIBUTE_PRIVATE);
>> +}
>> +
>> +int kvm_set_memory_attributes_shared(hwaddr start, hwaddr size)
>> +{
>> + return kvm_set_memory_attributes(start, size, 0);
>> +}
>> +
>> /* Called with KVMMemoryListener.slots_lock held */
>> static void kvm_set_phys_mem(KVMMemoryListener *kml,
>> MemoryRegionSection *section, bool add)
>> @@ -2439,6 +2480,9 @@ static int kvm_init(MachineState *ms)
>>
>> kvm_guest_memfd_supported = kvm_check_extension(s, KVM_CAP_GUEST_MEMFD);
>>
>> + ret = kvm_check_extension(s, KVM_CAP_MEMORY_ATTRIBUTES);
>> + kvm_supported_memory_attributes = ret > 0 ? ret : 0;
>
> kvm_check_extension() only returns non-negative value, so we can just
>
> kvm_supported_memory_attributes = kvm_check_extension(s, KVM_CAP_MEMORY_ATTRIBUTES);
Good catch, I will update it in next version.
>> +
>> if (object_property_find(OBJECT(current_machine), "kvm-type")) {
>> g_autofree char *kvm_type = object_property_get_str(OBJECT(current_machine),
>> "kvm-type",
>> diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
>> index 6cdf82de8372..8e83adfbbd19 100644
>> --- a/include/sysemu/kvm.h
>> +++ b/include/sysemu/kvm.h
>> @@ -546,4 +546,7 @@ uint32_t kvm_dirty_ring_size(void);
>> bool kvm_hwpoisoned_mem(void);
>>
>> int kvm_create_guest_memfd(uint64_t size, uint64_t flags, Error **errp);
>> +
>> +int kvm_set_memory_attributes_private(hwaddr start, hwaddr size);
>> +int kvm_set_memory_attributes_shared(hwaddr start, hwaddr size);
>> #endif
^ permalink raw reply [flat|nested] 124+ messages in thread
* [PATCH v5 07/65] physmem: Introduce ram_block_discard_guest_memfd_range()
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (5 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 06/65] kvm: Introduce support for memory_attributes Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-02-29 6:36 ` [PATCH v5 08/65] kvm: handle KVM_EXIT_MEMORY_FAULT Xiaoyao Li
` (57 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
When memory page is converted from private to shared, the original
private memory is back'ed by guest_memfd. Introduce
ram_block_discard_guest_memfd_range() for discarding memory in
guest_memfd.
Originally-from: Isaku Yamahata <isaku.yamahata@intel.com>
Codeveloped-by: Xiaoyao Li <xiaoyao.li@intel.com>
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
---
Changes in v5:
- Collect Reviewed-by from David;
Changes in in v4:
- Drop ram_block_convert_range() and open code its implementation in the
next Patch.
---
include/exec/cpu-common.h | 2 ++
system/physmem.c | 23 +++++++++++++++++++++++
2 files changed, 25 insertions(+)
diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h
index 9ead1be10057..ccd46ad3e173 100644
--- a/include/exec/cpu-common.h
+++ b/include/exec/cpu-common.h
@@ -166,6 +166,8 @@ typedef int (RAMBlockIterFunc)(RAMBlock *rb, void *opaque);
int qemu_ram_foreach_block(RAMBlockIterFunc func, void *opaque);
int ram_block_discard_range(RAMBlock *rb, uint64_t start, size_t length);
+int ram_block_discard_guest_memfd_range(RAMBlock *rb, uint64_t start,
+ size_t length);
#endif
diff --git a/system/physmem.c b/system/physmem.c
index c4fc1e506de8..8c9368bc99ef 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -3618,6 +3618,29 @@ err:
return ret;
}
+int ram_block_discard_guest_memfd_range(RAMBlock *rb, uint64_t start,
+ size_t length)
+{
+ int ret = -1;
+
+#ifdef CONFIG_FALLOCATE_PUNCH_HOLE
+ ret = fallocate(rb->guest_memfd, FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE,
+ start, length);
+
+ if (ret) {
+ ret = -errno;
+ error_report("%s: Failed to fallocate %s:%" PRIx64 " +%zx (%d)",
+ __func__, rb->idstr, start, length, ret);
+ }
+#else
+ ret = -ENOSYS;
+ error_report("%s: fallocate not available %s:%" PRIx64 " +%zx (%d)",
+ __func__, rb->idstr, start, length, ret);
+#endif
+
+ return ret;
+}
+
bool ramblock_is_pmem(RAMBlock *rb)
{
return rb->flags & RAM_PMEM;
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 08/65] kvm: handle KVM_EXIT_MEMORY_FAULT
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (6 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 07/65] physmem: Introduce ram_block_discard_guest_memfd_range() Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-03-05 9:10 ` Isaku Yamahata
2024-03-19 2:14 ` Wang, Lei
2024-02-29 6:36 ` [PATCH v5 09/65] trace/kvm: Add trace for page convertion between shared and private Xiaoyao Li
` (56 subsequent siblings)
64 siblings, 2 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
From: Chao Peng <chao.p.peng@linux.intel.com>
When geeting KVM_EXIT_MEMORY_FAULT exit, it indicates userspace needs to
do the memory conversion on the RAMBlock to turn the memory into desired
attribute, i.e., private/shared.
Currently only KVM_MEMORY_EXIT_FLAG_PRIVATE in flags is valid when
KVM_EXIT_MEMORY_FAULT happens.
Note, KVM_EXIT_MEMORY_FAULT makes sense only when the RAMBlock has
guest_memfd memory backend.
Note, KVM_EXIT_MEMORY_FAULT returns with -EFAULT, so special handling is
added.
When page is converted from shared to private, the original shared
memory can be discarded via ram_block_discard_range(). Note, shared
memory can be discarded only when it's not back'ed by hugetlb because
hugetlb is supposed to be pre-allocated and no need for discarding.
Signed-off-by: Chao Peng <chao.p.peng@linux.intel.com>
Co-developed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
Changes in v4:
- open-coded ram_block_discard logic;
- change warn_report() to error_report(); (Daniel)
---
accel/kvm/kvm-all.c | 94 ++++++++++++++++++++++++++++++++++++++++-----
1 file changed, 84 insertions(+), 10 deletions(-)
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 70d482a2c936..87e4275932a7 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -2903,6 +2903,68 @@ static void kvm_eat_signals(CPUState *cpu)
} while (sigismember(&chkset, SIG_IPI));
}
+static int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private)
+{
+ MemoryRegionSection section;
+ ram_addr_t offset;
+ MemoryRegion *mr;
+ RAMBlock *rb;
+ void *addr;
+ int ret = -1;
+
+ if (!QEMU_PTR_IS_ALIGNED(start, qemu_host_page_size) ||
+ !QEMU_PTR_IS_ALIGNED(size, qemu_host_page_size)) {
+ return -1;
+ }
+
+ if (!size) {
+ return -1;
+ }
+
+ section = memory_region_find(get_system_memory(), start, size);
+ mr = section.mr;
+ if (!mr) {
+ return -1;
+ }
+
+ if (memory_region_has_guest_memfd(mr)) {
+ if (to_private) {
+ ret = kvm_set_memory_attributes_private(start, size);
+ } else {
+ ret = kvm_set_memory_attributes_shared(start, size);
+ }
+
+ if (ret) {
+ memory_region_unref(section.mr);
+ return ret;
+ }
+
+ addr = memory_region_get_ram_ptr(mr) + section.offset_within_region;
+ rb = qemu_ram_block_from_host(addr, false, &offset);
+
+ if (to_private) {
+ if (rb->page_size != qemu_host_page_size) {
+ /*
+ * shared memory is back'ed by hugetlb, which is supposed to be
+ * pre-allocated and doesn't need to be discarded
+ */
+ return 0;
+ } else {
+ ret = ram_block_discard_range(rb, offset, size);
+ }
+ } else {
+ ret = ram_block_discard_guest_memfd_range(rb, offset, size);
+ }
+ } else {
+ error_report("Convert non guest_memfd backed memory region "
+ "(0x%"HWADDR_PRIx" ,+ 0x%"HWADDR_PRIx") to %s",
+ start, size, to_private ? "private" : "shared");
+ }
+
+ memory_region_unref(section.mr);
+ return ret;
+}
+
int kvm_cpu_exec(CPUState *cpu)
{
struct kvm_run *run = cpu->kvm_run;
@@ -2970,18 +3032,20 @@ int kvm_cpu_exec(CPUState *cpu)
ret = EXCP_INTERRUPT;
break;
}
- fprintf(stderr, "error: kvm run failed %s\n",
- strerror(-run_ret));
+ if (!(run_ret == -EFAULT && run->exit_reason == KVM_EXIT_MEMORY_FAULT)) {
+ fprintf(stderr, "error: kvm run failed %s\n",
+ strerror(-run_ret));
#ifdef TARGET_PPC
- if (run_ret == -EBUSY) {
- fprintf(stderr,
- "This is probably because your SMT is enabled.\n"
- "VCPU can only run on primary threads with all "
- "secondary threads offline.\n");
- }
+ if (run_ret == -EBUSY) {
+ fprintf(stderr,
+ "This is probably because your SMT is enabled.\n"
+ "VCPU can only run on primary threads with all "
+ "secondary threads offline.\n");
+ }
#endif
- ret = -1;
- break;
+ ret = -1;
+ break;
+ }
}
trace_kvm_run_exit(cpu->cpu_index, run->exit_reason);
@@ -3064,6 +3128,16 @@ int kvm_cpu_exec(CPUState *cpu)
break;
}
break;
+ case KVM_EXIT_MEMORY_FAULT:
+ if (run->memory_fault.flags & ~KVM_MEMORY_EXIT_FLAG_PRIVATE) {
+ error_report("KVM_EXIT_MEMORY_FAULT: Unknown flag 0x%" PRIx64,
+ (uint64_t)run->memory_fault.flags);
+ ret = -1;
+ break;
+ }
+ ret = kvm_convert_memory(run->memory_fault.gpa, run->memory_fault.size,
+ run->memory_fault.flags & KVM_MEMORY_EXIT_FLAG_PRIVATE);
+ break;
default:
ret = kvm_arch_handle_exit(cpu, run);
break;
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* Re: [PATCH v5 08/65] kvm: handle KVM_EXIT_MEMORY_FAULT
2024-02-29 6:36 ` [PATCH v5 08/65] kvm: handle KVM_EXIT_MEMORY_FAULT Xiaoyao Li
@ 2024-03-05 9:10 ` Isaku Yamahata
2024-03-07 6:43 ` Xiaoyao Li
2024-03-19 2:14 ` Wang, Lei
1 sibling, 1 reply; 124+ messages in thread
From: Isaku Yamahata @ 2024-03-05 9:10 UTC (permalink / raw)
To: Xiaoyao Li
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti, kvm, qemu-devel, Michael Roth,
Claudio Fontana, Gerd Hoffmann, Isaku Yamahata, Chenyi Qiang,
isaku.yamahata, isaku.yamahata
On Thu, Feb 29, 2024 at 01:36:29AM -0500,
Xiaoyao Li <xiaoyao.li@intel.com> wrote:
> From: Chao Peng <chao.p.peng@linux.intel.com>
>
> When geeting KVM_EXIT_MEMORY_FAULT exit, it indicates userspace needs to
> do the memory conversion on the RAMBlock to turn the memory into desired
> attribute, i.e., private/shared.
>
> Currently only KVM_MEMORY_EXIT_FLAG_PRIVATE in flags is valid when
> KVM_EXIT_MEMORY_FAULT happens.
>
> Note, KVM_EXIT_MEMORY_FAULT makes sense only when the RAMBlock has
> guest_memfd memory backend.
>
> Note, KVM_EXIT_MEMORY_FAULT returns with -EFAULT, so special handling is
> added.
>
> When page is converted from shared to private, the original shared
> memory can be discarded via ram_block_discard_range(). Note, shared
> memory can be discarded only when it's not back'ed by hugetlb because
> hugetlb is supposed to be pre-allocated and no need for discarding.
>
> Signed-off-by: Chao Peng <chao.p.peng@linux.intel.com>
> Co-developed-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>
> ---
> Changes in v4:
> - open-coded ram_block_discard logic;
> - change warn_report() to error_report(); (Daniel)
> ---
> accel/kvm/kvm-all.c | 94 ++++++++++++++++++++++++++++++++++++++++-----
> 1 file changed, 84 insertions(+), 10 deletions(-)
>
> diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
> index 70d482a2c936..87e4275932a7 100644
> --- a/accel/kvm/kvm-all.c
> +++ b/accel/kvm/kvm-all.c
> @@ -2903,6 +2903,68 @@ static void kvm_eat_signals(CPUState *cpu)
> } while (sigismember(&chkset, SIG_IPI));
> }
>
> +static int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private)
> +{
> + MemoryRegionSection section;
> + ram_addr_t offset;
> + MemoryRegion *mr;
> + RAMBlock *rb;
> + void *addr;
> + int ret = -1;
> +
> + if (!QEMU_PTR_IS_ALIGNED(start, qemu_host_page_size) ||
> + !QEMU_PTR_IS_ALIGNED(size, qemu_host_page_size)) {
> + return -1;
> + }
> +
> + if (!size) {
> + return -1;
> + }
> +
> + section = memory_region_find(get_system_memory(), start, size);
> + mr = section.mr;
> + if (!mr) {
> + return -1;
> + }
> +
> + if (memory_region_has_guest_memfd(mr)) {
> + if (to_private) {
> + ret = kvm_set_memory_attributes_private(start, size);
> + } else {
> + ret = kvm_set_memory_attributes_shared(start, size);
> + }
> +
> + if (ret) {
> + memory_region_unref(section.mr);
> + return ret;
> + }
> +
> + addr = memory_region_get_ram_ptr(mr) + section.offset_within_region;
> + rb = qemu_ram_block_from_host(addr, false, &offset);
> +
> + if (to_private) {
> + if (rb->page_size != qemu_host_page_size) {
> + /*
> + * shared memory is back'ed by hugetlb, which is supposed to be
> + * pre-allocated and doesn't need to be discarded
> + */
> + return 0;
The reference count leaks. Add memory_region_unref() is needed.
Otherwise looks good to me.
Reviewed-by: Isaku Yamahata <isaku.yamahata@intel.com>
--
Isaku Yamahata <isaku.yamahata@linux.intel.com>
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 08/65] kvm: handle KVM_EXIT_MEMORY_FAULT
2024-03-05 9:10 ` Isaku Yamahata
@ 2024-03-07 6:43 ` Xiaoyao Li
0 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-03-07 6:43 UTC (permalink / raw)
To: Isaku Yamahata
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti, kvm, qemu-devel, Michael Roth,
Claudio Fontana, Gerd Hoffmann, Isaku Yamahata, Chenyi Qiang,
isaku.yamahata
On 3/5/2024 5:10 PM, Isaku Yamahata wrote:
> On Thu, Feb 29, 2024 at 01:36:29AM -0500,
> Xiaoyao Li <xiaoyao.li@intel.com> wrote:
>
>> From: Chao Peng <chao.p.peng@linux.intel.com>
>>
>> When geeting KVM_EXIT_MEMORY_FAULT exit, it indicates userspace needs to
>> do the memory conversion on the RAMBlock to turn the memory into desired
>> attribute, i.e., private/shared.
>>
>> Currently only KVM_MEMORY_EXIT_FLAG_PRIVATE in flags is valid when
>> KVM_EXIT_MEMORY_FAULT happens.
>>
>> Note, KVM_EXIT_MEMORY_FAULT makes sense only when the RAMBlock has
>> guest_memfd memory backend.
>>
>> Note, KVM_EXIT_MEMORY_FAULT returns with -EFAULT, so special handling is
>> added.
>>
>> When page is converted from shared to private, the original shared
>> memory can be discarded via ram_block_discard_range(). Note, shared
>> memory can be discarded only when it's not back'ed by hugetlb because
>> hugetlb is supposed to be pre-allocated and no need for discarding.
>>
>> Signed-off-by: Chao Peng <chao.p.peng@linux.intel.com>
>> Co-developed-by: Xiaoyao Li <xiaoyao.li@intel.com>
>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>>
>> ---
>> Changes in v4:
>> - open-coded ram_block_discard logic;
>> - change warn_report() to error_report(); (Daniel)
>> ---
>> accel/kvm/kvm-all.c | 94 ++++++++++++++++++++++++++++++++++++++++-----
>> 1 file changed, 84 insertions(+), 10 deletions(-)
>>
>> diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
>> index 70d482a2c936..87e4275932a7 100644
>> --- a/accel/kvm/kvm-all.c
>> +++ b/accel/kvm/kvm-all.c
>> @@ -2903,6 +2903,68 @@ static void kvm_eat_signals(CPUState *cpu)
>> } while (sigismember(&chkset, SIG_IPI));
>> }
>>
>> +static int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private)
>> +{
>> + MemoryRegionSection section;
>> + ram_addr_t offset;
>> + MemoryRegion *mr;
>> + RAMBlock *rb;
>> + void *addr;
>> + int ret = -1;
>> +
>> + if (!QEMU_PTR_IS_ALIGNED(start, qemu_host_page_size) ||
>> + !QEMU_PTR_IS_ALIGNED(size, qemu_host_page_size)) {
>> + return -1;
>> + }
>> +
>> + if (!size) {
>> + return -1;
>> + }
>> +
>> + section = memory_region_find(get_system_memory(), start, size);
>> + mr = section.mr;
>> + if (!mr) {
>> + return -1;
>> + }
>> +
>> + if (memory_region_has_guest_memfd(mr)) {
>> + if (to_private) {
>> + ret = kvm_set_memory_attributes_private(start, size);
>> + } else {
>> + ret = kvm_set_memory_attributes_shared(start, size);
>> + }
>> +
>> + if (ret) {
>> + memory_region_unref(section.mr);
>> + return ret;
>> + }
>> +
>> + addr = memory_region_get_ram_ptr(mr) + section.offset_within_region;
>> + rb = qemu_ram_block_from_host(addr, false, &offset);
>> +
>> + if (to_private) {
>> + if (rb->page_size != qemu_host_page_size) {
>> + /*
>> + * shared memory is back'ed by hugetlb, which is supposed to be
>> + * pre-allocated and doesn't need to be discarded
>> + */
>> + return 0;
>
> The reference count leaks. Add memory_region_unref() is needed.
thanks for catching it. Will fix it in next version.
> Otherwise looks good to me.
> Reviewed-by: Isaku Yamahata <isaku.yamahata@intel.com>
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [PATCH v5 08/65] kvm: handle KVM_EXIT_MEMORY_FAULT
2024-02-29 6:36 ` [PATCH v5 08/65] kvm: handle KVM_EXIT_MEMORY_FAULT Xiaoyao Li
2024-03-05 9:10 ` Isaku Yamahata
@ 2024-03-19 2:14 ` Wang, Lei
2024-03-20 13:49 ` Xiaoyao Li
1 sibling, 1 reply; 124+ messages in thread
From: Wang, Lei @ 2024-03-19 2:14 UTC (permalink / raw)
To: Xiaoyao Li, Paolo Bonzini, David Hildenbrand, Igor Mammedov,
Eduardo Habkost, Marcel Apfelbaum, Philippe Mathieu-Daudé,
Yanan Wang, Michael S. Tsirkin, Richard Henderson, Ani Sinha,
Peter Xu, Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang
On 2/29/2024 14:36, Xiaoyao Li wrote:
> From: Chao Peng <chao.p.peng@linux.intel.com>
>
> When geeting KVM_EXIT_MEMORY_FAULT exit, it indicates userspace needs to
> do the memory conversion on the RAMBlock to turn the memory into desired
> attribute, i.e., private/shared.
>
> Currently only KVM_MEMORY_EXIT_FLAG_PRIVATE in flags is valid when
> KVM_EXIT_MEMORY_FAULT happens.
>
> Note, KVM_EXIT_MEMORY_FAULT makes sense only when the RAMBlock has
> guest_memfd memory backend.
>
> Note, KVM_EXIT_MEMORY_FAULT returns with -EFAULT, so special handling is
> added.
>
> When page is converted from shared to private, the original shared
> memory can be discarded via ram_block_discard_range(). Note, shared
> memory can be discarded only when it's not back'ed by hugetlb because
> hugetlb is supposed to be pre-allocated and no need for discarding.
>
> Signed-off-by: Chao Peng <chao.p.peng@linux.intel.com>
> Co-developed-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>
> ---
> Changes in v4:
> - open-coded ram_block_discard logic;
> - change warn_report() to error_report(); (Daniel)
> ---
> accel/kvm/kvm-all.c | 94 ++++++++++++++++++++++++++++++++++++++++-----
> 1 file changed, 84 insertions(+), 10 deletions(-)
>
> diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
> index 70d482a2c936..87e4275932a7 100644
> --- a/accel/kvm/kvm-all.c
> +++ b/accel/kvm/kvm-all.c
> @@ -2903,6 +2903,68 @@ static void kvm_eat_signals(CPUState *cpu)
> } while (sigismember(&chkset, SIG_IPI));
> }
>
> +static int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private)
> +{
> + MemoryRegionSection section;
> + ram_addr_t offset;
> + MemoryRegion *mr;
> + RAMBlock *rb;
> + void *addr;
> + int ret = -1;
> +
> + if (!QEMU_PTR_IS_ALIGNED(start, qemu_host_page_size) ||
> + !QEMU_PTR_IS_ALIGNED(size, qemu_host_page_size)) {
> + return -1;
> + }
> +
> + if (!size) {
> + return -1;
> + }
> +
> + section = memory_region_find(get_system_memory(), start, size);
> + mr = section.mr;
> + if (!mr) {
> + return -1;
> + }
> +
> + if (memory_region_has_guest_memfd(mr)) {
> + if (to_private) {
> + ret = kvm_set_memory_attributes_private(start, size);
> + } else {
> + ret = kvm_set_memory_attributes_shared(start, size);
> + }
> +
> + if (ret) {
> + memory_region_unref(section.mr);
> + return ret;
> + }
> +
> + addr = memory_region_get_ram_ptr(mr) + section.offset_within_region;
> + rb = qemu_ram_block_from_host(addr, false, &offset);
> +
> + if (to_private) {
> + if (rb->page_size != qemu_host_page_size) {
> + /*
> + * shared memory is back'ed by hugetlb, which is supposed to be
> + * pre-allocated and doesn't need to be discarded
> + */
Nit: comment indentation is broken here.
> + return 0;
> + } else {
> + ret = ram_block_discard_range(rb, offset, size);
> + }
> + } else {
> + ret = ram_block_discard_guest_memfd_range(rb, offset, size);
> + }
> + } else {
> + error_report("Convert non guest_memfd backed memory region "
> + "(0x%"HWADDR_PRIx" ,+ 0x%"HWADDR_PRIx") to %s",
Same as above.
> + start, size, to_private ? "private" : "shared");
> + }
> +
> + memory_region_unref(section.mr);
> + return ret;
> +}
> +
> int kvm_cpu_exec(CPUState *cpu)
> {
> struct kvm_run *run = cpu->kvm_run;
> @@ -2970,18 +3032,20 @@ int kvm_cpu_exec(CPUState *cpu)
> ret = EXCP_INTERRUPT;
> break;
> }
> - fprintf(stderr, "error: kvm run failed %s\n",
> - strerror(-run_ret));
> + if (!(run_ret == -EFAULT && run->exit_reason == KVM_EXIT_MEMORY_FAULT)) {
> + fprintf(stderr, "error: kvm run failed %s\n",
> + strerror(-run_ret));
> #ifdef TARGET_PPC
> - if (run_ret == -EBUSY) {
> - fprintf(stderr,
> - "This is probably because your SMT is enabled.\n"
> - "VCPU can only run on primary threads with all "
> - "secondary threads offline.\n");
> - }
> + if (run_ret == -EBUSY) {
> + fprintf(stderr,
> + "This is probably because your SMT is enabled.\n"
> + "VCPU can only run on primary threads with all "
> + "secondary threads offline.\n");
> + }
> #endif
> - ret = -1;
> - break;
> + ret = -1;
> + break;
> + }
> }
>
> trace_kvm_run_exit(cpu->cpu_index, run->exit_reason);
> @@ -3064,6 +3128,16 @@ int kvm_cpu_exec(CPUState *cpu)
> break;
> }
> break;
> + case KVM_EXIT_MEMORY_FAULT:
> + if (run->memory_fault.flags & ~KVM_MEMORY_EXIT_FLAG_PRIVATE) {
> + error_report("KVM_EXIT_MEMORY_FAULT: Unknown flag 0x%" PRIx64,
> + (uint64_t)run->memory_fault.flags);
> + ret = -1;
> + break;
> + }
> + ret = kvm_convert_memory(run->memory_fault.gpa, run->memory_fault.size,
> + run->memory_fault.flags & KVM_MEMORY_EXIT_FLAG_PRIVATE);
> + break;
> default:
> ret = kvm_arch_handle_exit(cpu, run);
> break;
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 08/65] kvm: handle KVM_EXIT_MEMORY_FAULT
2024-03-19 2:14 ` Wang, Lei
@ 2024-03-20 13:49 ` Xiaoyao Li
0 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-03-20 13:49 UTC (permalink / raw)
To: Wang, Lei, Paolo Bonzini, David Hildenbrand, Igor Mammedov,
Eduardo Habkost, Marcel Apfelbaum, Philippe Mathieu-Daudé,
Yanan Wang, Michael S. Tsirkin, Richard Henderson, Ani Sinha,
Peter Xu, Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang
On 3/19/2024 10:14 AM, Wang, Lei wrote:
> On 2/29/2024 14:36, Xiaoyao Li wrote:
>> From: Chao Peng <chao.p.peng@linux.intel.com>
>>
>> When geeting KVM_EXIT_MEMORY_FAULT exit, it indicates userspace needs to
>> do the memory conversion on the RAMBlock to turn the memory into desired
>> attribute, i.e., private/shared.
>>
>> Currently only KVM_MEMORY_EXIT_FLAG_PRIVATE in flags is valid when
>> KVM_EXIT_MEMORY_FAULT happens.
>>
>> Note, KVM_EXIT_MEMORY_FAULT makes sense only when the RAMBlock has
>> guest_memfd memory backend.
>>
>> Note, KVM_EXIT_MEMORY_FAULT returns with -EFAULT, so special handling is
>> added.
>>
>> When page is converted from shared to private, the original shared
>> memory can be discarded via ram_block_discard_range(). Note, shared
>> memory can be discarded only when it's not back'ed by hugetlb because
>> hugetlb is supposed to be pre-allocated and no need for discarding.
>>
>> Signed-off-by: Chao Peng <chao.p.peng@linux.intel.com>
>> Co-developed-by: Xiaoyao Li <xiaoyao.li@intel.com>
>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>>
>> ---
>> Changes in v4:
>> - open-coded ram_block_discard logic;
>> - change warn_report() to error_report(); (Daniel)
>> ---
>> accel/kvm/kvm-all.c | 94 ++++++++++++++++++++++++++++++++++++++++-----
>> 1 file changed, 84 insertions(+), 10 deletions(-)
>>
>> diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
>> index 70d482a2c936..87e4275932a7 100644
>> --- a/accel/kvm/kvm-all.c
>> +++ b/accel/kvm/kvm-all.c
>> @@ -2903,6 +2903,68 @@ static void kvm_eat_signals(CPUState *cpu)
>> } while (sigismember(&chkset, SIG_IPI));
>> }
>>
>> +static int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private)
>> +{
>> + MemoryRegionSection section;
>> + ram_addr_t offset;
>> + MemoryRegion *mr;
>> + RAMBlock *rb;
>> + void *addr;
>> + int ret = -1;
>> +
>> + if (!QEMU_PTR_IS_ALIGNED(start, qemu_host_page_size) ||
>> + !QEMU_PTR_IS_ALIGNED(size, qemu_host_page_size)) {
>> + return -1;
>> + }
>> +
>> + if (!size) {
>> + return -1;
>> + }
>> +
>> + section = memory_region_find(get_system_memory(), start, size);
>> + mr = section.mr;
>> + if (!mr) {
>> + return -1;
>> + }
>> +
>> + if (memory_region_has_guest_memfd(mr)) {
>> + if (to_private) {
>> + ret = kvm_set_memory_attributes_private(start, size);
>> + } else {
>> + ret = kvm_set_memory_attributes_shared(start, size);
>> + }
>> +
>> + if (ret) {
>> + memory_region_unref(section.mr);
>> + return ret;
>> + }
>> +
>> + addr = memory_region_get_ram_ptr(mr) + section.offset_within_region;
>> + rb = qemu_ram_block_from_host(addr, false, &offset);
>> +
>> + if (to_private) {
>> + if (rb->page_size != qemu_host_page_size) {
>> + /*
>> + * shared memory is back'ed by hugetlb, which is supposed to be
>> + * pre-allocated and doesn't need to be discarded
>> + */
>
> Nit: comment indentation is broken here.
>
>> + return 0;
>> + } else {
>> + ret = ram_block_discard_range(rb, offset, size);
>> + }
>> + } else {
>> + ret = ram_block_discard_guest_memfd_range(rb, offset, size);
>> + }
>> + } else {
>> + error_report("Convert non guest_memfd backed memory region "
>> + "(0x%"HWADDR_PRIx" ,+ 0x%"HWADDR_PRIx") to %s",
>
> Same as above.
>
Fixed.
thanks!
^ permalink raw reply [flat|nested] 124+ messages in thread
* [PATCH v5 09/65] trace/kvm: Add trace for page convertion between shared and private
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (7 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 08/65] kvm: handle KVM_EXIT_MEMORY_FAULT Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-02-29 6:36 ` [PATCH v5 10/65] kvm/memory: Make memory type private by default if it has guest memfd backend Xiaoyao Li
` (55 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
From: Isaku Yamahata <isaku.yamahata@intel.com>
Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
accel/kvm/kvm-all.c | 2 ++
accel/kvm/trace-events | 1 +
2 files changed, 3 insertions(+)
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 87e4275932a7..fe2eb3f06902 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -2912,6 +2912,8 @@ static int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private)
void *addr;
int ret = -1;
+ trace_kvm_convert_memory(start, size, to_private ? "shared_to_private" : "private_to_shared");
+
if (!QEMU_PTR_IS_ALIGNED(start, qemu_host_page_size) ||
!QEMU_PTR_IS_ALIGNED(size, qemu_host_page_size)) {
return -1;
diff --git a/accel/kvm/trace-events b/accel/kvm/trace-events
index e8c52cb9e7a1..31175fed97ce 100644
--- a/accel/kvm/trace-events
+++ b/accel/kvm/trace-events
@@ -31,3 +31,4 @@ kvm_cpu_exec(void) ""
kvm_interrupt_exit_request(void) ""
kvm_io_window_exit(void) ""
kvm_run_exit_system_event(int cpu_index, uint32_t event_type) "cpu_index %d, system_even_type %"PRIu32
+kvm_convert_memory(uint64_t start, uint64_t size, const char *msg) "start 0x%" PRIx64 " size 0x%" PRIx64 " %s"
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 10/65] kvm/memory: Make memory type private by default if it has guest memfd backend
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (8 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 09/65] trace/kvm: Add trace for page convertion between shared and private Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-02-29 6:36 ` [PATCH v5 11/65] *** HACK *** linux-headers: Update headers to pull in TDX API changes Xiaoyao Li
` (54 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
KVM side leaves the memory to shared by default, while may incur the
overhead of paging conversion on the first visit of each page. Because
the expectation is that page is likely to private for the VMs that
require private memory (has guest memfd).
Explicitly set the memory to private when memory region has valid
guest memfd backend.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
accel/kvm/kvm-all.c | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index fe2eb3f06902..0c0719a0303c 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -1451,6 +1451,16 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml,
strerror(-err));
abort();
}
+
+ if (memory_region_has_guest_memfd(mr)) {
+ err = kvm_set_memory_attributes_private(start_addr, slot_size);
+ if (err) {
+ error_report("%s: failed to set memory attribute private: %s\n",
+ __func__, strerror(-err));
+ exit(1);
+ }
+ }
+
start_addr += slot_size;
ram_start_offset += slot_size;
ram += slot_size;
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 11/65] *** HACK *** linux-headers: Update headers to pull in TDX API changes
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (9 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 10/65] kvm/memory: Make memory type private by default if it has guest memfd backend Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-02-29 6:36 ` [PATCH v5 12/65] i386: Introduce tdx-guest object Xiaoyao Li
` (53 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
Pull in recent TDX updates, which are not backwards compatible.
It's just to make this series runnable. It will be updated by script
scripts/update-linux-headers.sh
once TDX support is upstreamed in linux kernel
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
| 86 +++++++++++++++++++++++++++++++
| 100 ++++++++++++++++++++++++++++++++++++
2 files changed, 186 insertions(+)
--git a/linux-headers/asm-x86/kvm.h b/linux-headers/asm-x86/kvm.h
index 003fb745347c..c2c3123f88e7 100644
--- a/linux-headers/asm-x86/kvm.h
+++ b/linux-headers/asm-x86/kvm.h
@@ -562,5 +562,91 @@ struct kvm_pmu_event_filter {
#define KVM_X86_DEFAULT_VM 0
#define KVM_X86_SW_PROTECTED_VM 1
+#define KVM_X86_TDX_VM 2
+#define KVM_X86_SNP_VM 3
+
+/* Trust Domain eXtension sub-ioctl() commands. */
+enum kvm_tdx_cmd_id {
+ KVM_TDX_CAPABILITIES = 0,
+ KVM_TDX_INIT_VM,
+ KVM_TDX_INIT_VCPU,
+ KVM_TDX_EXTEND_MEMORY,
+ KVM_TDX_FINALIZE_VM,
+
+ KVM_TDX_CMD_NR_MAX,
+};
+
+struct kvm_tdx_cmd {
+ /* enum kvm_tdx_cmd_id */
+ __u32 id;
+ /* flags for sub-commend. If sub-command doesn't use this, set zero. */
+ __u32 flags;
+ /*
+ * data for each sub-command. An immediate or a pointer to the actual
+ * data in process virtual address. If sub-command doesn't use it,
+ * set zero.
+ */
+ __u64 data;
+ /*
+ * Auxiliary error code. The sub-command may return TDX SEAMCALL
+ * status code in addition to -Exxx.
+ * Defined for consistency with struct kvm_sev_cmd.
+ */
+ __u64 error;
+};
+
+#define KVM_TDX_CPUID_NO_SUBLEAF ((__u32)-1)
+
+struct kvm_tdx_cpuid_config {
+ __u32 leaf;
+ __u32 sub_leaf;
+ __u32 eax;
+ __u32 ebx;
+ __u32 ecx;
+ __u32 edx;
+};
+
+/* supported_gpaw */
+#define TDX_CAP_GPAW_48 (1 << 0)
+#define TDX_CAP_GPAW_52 (1 << 1)
+
+struct kvm_tdx_capabilities {
+ __u64 attrs_fixed0;
+ __u64 attrs_fixed1;
+ __u64 xfam_fixed0;
+ __u64 xfam_fixed1;
+ __u32 supported_gpaw;
+ __u32 padding;
+ __u64 reserved[251];
+
+ __u32 nr_cpuid_configs;
+ struct kvm_tdx_cpuid_config cpuid_configs[];
+};
+
+struct kvm_tdx_init_vm {
+ __u64 attributes;
+ __u64 mrconfigid[6]; /* sha384 digest */
+ __u64 mrowner[6]; /* sha384 digest */
+ __u64 mrownerconfig[6]; /* sha384 digest */
+ /*
+ * For future extensibility to make sizeof(struct kvm_tdx_init_vm) = 8KB.
+ * This should be enough given sizeof(TD_PARAMS) = 1024.
+ * 8KB was chosen given because
+ * sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES(=256) = 8KB.
+ */
+ __u64 reserved[1004];
+
+ /*
+ * Call KVM_TDX_INIT_VM before vcpu creation, thus before
+ * KVM_SET_CPUID2.
+ * This configuration supersedes KVM_SET_CPUID2s for VCPUs because the
+ * TDX module directly virtualizes those CPUIDs without VMM. The user
+ * space VMM, e.g. qemu, should make KVM_SET_CPUID2 consistent with
+ * those values. If it doesn't, KVM may have wrong idea of vCPUIDs of
+ * the guest, and KVM may wrongly emulate CPUIDs or MSRs that the TDX
+ * module doesn't virtualize.
+ */
+ struct kvm_cpuid2 cpuid;
+};
#endif /* _ASM_X86_KVM_H */
--git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h
index 17839229b2ac..76cd2d772d4f 100644
--- a/linux-headers/linux/kvm.h
+++ b/linux-headers/linux/kvm.h
@@ -167,6 +167,92 @@ struct kvm_xen_exit {
} u;
};
+/* masks for reg_mask to indicate which registers are passed. */
+#define TDX_VMCALL_REG_MASK_RBX BIT_ULL(2)
+#define TDX_VMCALL_REG_MASK_RDX BIT_ULL(3)
+#define TDX_VMCALL_REG_MASK_RSI BIT_ULL(6)
+#define TDX_VMCALL_REG_MASK_RDI BIT_ULL(7)
+#define TDX_VMCALL_REG_MASK_R8 BIT_ULL(8)
+#define TDX_VMCALL_REG_MASK_R9 BIT_ULL(9)
+#define TDX_VMCALL_REG_MASK_R10 BIT_ULL(10)
+#define TDX_VMCALL_REG_MASK_R11 BIT_ULL(11)
+#define TDX_VMCALL_REG_MASK_R12 BIT_ULL(12)
+#define TDX_VMCALL_REG_MASK_R13 BIT_ULL(13)
+#define TDX_VMCALL_REG_MASK_R14 BIT_ULL(14)
+#define TDX_VMCALL_REG_MASK_R15 BIT_ULL(15)
+
+struct kvm_tdx_exit {
+#define KVM_EXIT_TDX_VMCALL 1
+ __u32 type;
+ __u32 pad;
+
+ union {
+ struct kvm_tdx_vmcall {
+ /*
+ * RAX(bit 0), RCX(bit 1) and RSP(bit 4) are reserved.
+ * RAX(bit 0): TDG.VP.VMCALL status code.
+ * RCX(bit 1): bitmap for used registers.
+ * RSP(bit 4): the caller stack.
+ */
+ union {
+ __u64 in_rcx;
+ __u64 reg_mask;
+ };
+
+ /*
+ * Guest-Host-Communication Interface for TDX spec
+ * defines the ABI for TDG.VP.VMCALL.
+ */
+ /* Input parameters: guest -> VMM */
+ union {
+ __u64 in_r10;
+ __u64 type;
+ };
+ union {
+ __u64 in_r11;
+ __u64 subfunction;
+ };
+ /*
+ * Subfunction specific.
+ * Registers are used in this order to pass input
+ * arguments. r12=arg0, r13=arg1, etc.
+ */
+ __u64 in_r12;
+ __u64 in_r13;
+ __u64 in_r14;
+ __u64 in_r15;
+ __u64 in_rbx;
+ __u64 in_rdi;
+ __u64 in_rsi;
+ __u64 in_r8;
+ __u64 in_r9;
+ __u64 in_rdx;
+
+ /* Output parameters: VMM -> guest */
+ union {
+ __u64 out_r10;
+ __u64 status_code;
+ };
+ /*
+ * Subfunction specific.
+ * Registers are used in this order to output return
+ * values. r11=ret0, r12=ret1, etc.
+ */
+ __u64 out_r11;
+ __u64 out_r12;
+ __u64 out_r13;
+ __u64 out_r14;
+ __u64 out_r15;
+ __u64 out_rbx;
+ __u64 out_rdi;
+ __u64 out_rsi;
+ __u64 out_r8;
+ __u64 out_r9;
+ __u64 out_rdx;
+ } vmcall;
+ } u;
+};
+
#define KVM_S390_GET_SKEYS_NONE 1
#define KVM_S390_SKEYS_MAX 1048576
@@ -210,6 +296,7 @@ struct kvm_xen_exit {
#define KVM_EXIT_NOTIFY 37
#define KVM_EXIT_LOONGARCH_IOCSR 38
#define KVM_EXIT_MEMORY_FAULT 39
+#define KVM_EXIT_TDX 40
/* For KVM_EXIT_INTERNAL_ERROR */
/* Emulate instruction failed. */
@@ -466,6 +553,8 @@ struct kvm_run {
__u64 gpa;
__u64 size;
} memory_fault;
+ /* KVM_EXIT_TDX_VMCALL */
+ struct kvm_tdx_exit tdx;
/* Fix the size of the union. */
char padding[256];
};
@@ -1151,6 +1240,8 @@ struct kvm_ppc_resize_hpt {
#define KVM_CAP_MEMORY_ATTRIBUTES 233
#define KVM_CAP_GUEST_MEMFD 234
#define KVM_CAP_VM_TYPES 235
+#define KVM_CAP_MEMORY_MAPPING 236
+#define KVM_CAP_X86_BUS_FREQUENCY_CONTROL 237
#ifdef KVM_CAP_IRQ_ROUTING
@@ -2223,4 +2314,13 @@ struct kvm_create_guest_memfd {
__u64 reserved[6];
};
+#define KVM_MEMORY_MAPPING _IOWR(KVMIO, 0xd5, struct kvm_memory_mapping)
+
+struct kvm_memory_mapping {
+ __u64 base_gfn;
+ __u64 nr_pages;
+ __u64 flags;
+ __u64 source;
+};
+
#endif /* __LINUX_KVM_H */
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 12/65] i386: Introduce tdx-guest object
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (10 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 11/65] *** HACK *** linux-headers: Update headers to pull in TDX API changes Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-02-29 8:19 ` Markus Armbruster
2024-02-29 6:36 ` [PATCH v5 13/65] target/i386: Implement mc->kvm_type() to get VM type Xiaoyao Li
` (52 subsequent siblings)
64 siblings, 1 reply; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
Introduce tdx-guest object which inherits CONFIDENTIAL_GUEST_SUPPORT,
and will be used to create TDX VMs (TDs) by
qemu -machine ...,confidential-guest-support=tdx0 \
-object tdx-guest,id=tdx0
So far, it has no QAPI member/properety decleared and only one internal
member 'attributes' with fixed value 0 that not configurable.
QAPI properties will be added later.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
---
Changes in v4:
- update the new qapi `since` filed from 8.2 to 9.0
Changes in v1
- make @attributes not user-settable
---
configs/devices/i386-softmmu/default.mak | 1 +
hw/i386/Kconfig | 5 +++
qapi/qom.json | 12 +++++++
target/i386/kvm/meson.build | 2 ++
target/i386/kvm/tdx.c | 40 ++++++++++++++++++++++++
target/i386/kvm/tdx.h | 19 +++++++++++
6 files changed, 79 insertions(+)
create mode 100644 target/i386/kvm/tdx.c
create mode 100644 target/i386/kvm/tdx.h
diff --git a/configs/devices/i386-softmmu/default.mak b/configs/devices/i386-softmmu/default.mak
index 598c6646dfc0..9b5ec59d65b0 100644
--- a/configs/devices/i386-softmmu/default.mak
+++ b/configs/devices/i386-softmmu/default.mak
@@ -18,6 +18,7 @@
#CONFIG_QXL=n
#CONFIG_SEV=n
#CONFIG_SGA=n
+#CONFIG_TDX=n
#CONFIG_TEST_DEVICES=n
#CONFIG_TPM_CRB=n
#CONFIG_TPM_TIS_ISA=n
diff --git a/hw/i386/Kconfig b/hw/i386/Kconfig
index a1846be6f761..c0ccf50ac3ef 100644
--- a/hw/i386/Kconfig
+++ b/hw/i386/Kconfig
@@ -10,6 +10,10 @@ config SGX
bool
depends on KVM
+config TDX
+ bool
+ depends on KVM
+
config PC
bool
imply APPLESMC
@@ -26,6 +30,7 @@ config PC
imply QXL
imply SEV
imply SGX
+ imply TDX
imply TEST_DEVICES
imply TPM_CRB
imply TPM_TIS_ISA
diff --git a/qapi/qom.json b/qapi/qom.json
index 2a6e49365a4a..220cc6c98d4b 100644
--- a/qapi/qom.json
+++ b/qapi/qom.json
@@ -895,6 +895,16 @@
'reduced-phys-bits': 'uint32',
'*kernel-hashes': 'bool' } }
+##
+# @TdxGuestProperties:
+#
+# Properties for tdx-guest objects.
+#
+# Since: 9.0
+##
+{ 'struct': 'TdxGuestProperties',
+ 'data': { }}
+
##
# @ThreadContextProperties:
#
@@ -974,6 +984,7 @@
'sev-guest',
'thread-context',
's390-pv-guest',
+ 'tdx-guest',
'throttle-group',
'tls-creds-anon',
'tls-creds-psk',
@@ -1041,6 +1052,7 @@
'secret_keyring': { 'type': 'SecretKeyringProperties',
'if': 'CONFIG_SECRET_KEYRING' },
'sev-guest': 'SevGuestProperties',
+ 'tdx-guest': 'TdxGuestProperties',
'thread-context': 'ThreadContextProperties',
'throttle-group': 'ThrottleGroupProperties',
'tls-creds-anon': 'TlsCredsAnonProperties',
diff --git a/target/i386/kvm/meson.build b/target/i386/kvm/meson.build
index e7850981e62d..26a1ab038513 100644
--- a/target/i386/kvm/meson.build
+++ b/target/i386/kvm/meson.build
@@ -7,6 +7,8 @@ i386_kvm_ss.add(files(
i386_kvm_ss.add(when: 'CONFIG_XEN_EMU', if_true: files('xen-emu.c'))
+i386_kvm_ss.add(when: 'CONFIG_TDX', if_true: files('tdx.c'))
+
i386_system_ss.add(when: 'CONFIG_HYPERV', if_true: files('hyperv.c'), if_false: files('hyperv-stub.c'))
i386_system_ss.add_all(when: 'CONFIG_KVM', if_true: i386_kvm_ss)
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
new file mode 100644
index 000000000000..d3792d4a3d56
--- /dev/null
+++ b/target/i386/kvm/tdx.c
@@ -0,0 +1,40 @@
+/*
+ * QEMU TDX support
+ *
+ * Copyright Intel
+ *
+ * Author:
+ * Xiaoyao Li <xiaoyao.li@intel.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "qom/object_interfaces.h"
+
+#include "tdx.h"
+
+/* tdx guest */
+OBJECT_DEFINE_TYPE_WITH_INTERFACES(TdxGuest,
+ tdx_guest,
+ TDX_GUEST,
+ CONFIDENTIAL_GUEST_SUPPORT,
+ { TYPE_USER_CREATABLE },
+ { NULL })
+
+static void tdx_guest_init(Object *obj)
+{
+ TdxGuest *tdx = TDX_GUEST(obj);
+
+ tdx->attributes = 0;
+}
+
+static void tdx_guest_finalize(Object *obj)
+{
+}
+
+static void tdx_guest_class_init(ObjectClass *oc, void *data)
+{
+}
diff --git a/target/i386/kvm/tdx.h b/target/i386/kvm/tdx.h
new file mode 100644
index 000000000000..415aeb5af746
--- /dev/null
+++ b/target/i386/kvm/tdx.h
@@ -0,0 +1,19 @@
+#ifndef QEMU_I386_TDX_H
+#define QEMU_I386_TDX_H
+
+#include "exec/confidential-guest-support.h"
+
+#define TYPE_TDX_GUEST "tdx-guest"
+#define TDX_GUEST(obj) OBJECT_CHECK(TdxGuest, (obj), TYPE_TDX_GUEST)
+
+typedef struct TdxGuestClass {
+ ConfidentialGuestSupportClass parent_class;
+} TdxGuestClass;
+
+typedef struct TdxGuest {
+ ConfidentialGuestSupport parent_obj;
+
+ uint64_t attributes; /* TD attributes */
+} TdxGuest;
+
+#endif /* QEMU_I386_TDX_H */
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* Re: [PATCH v5 12/65] i386: Introduce tdx-guest object
2024-02-29 6:36 ` [PATCH v5 12/65] i386: Introduce tdx-guest object Xiaoyao Li
@ 2024-02-29 8:19 ` Markus Armbruster
0 siblings, 0 replies; 124+ messages in thread
From: Markus Armbruster @ 2024-02-29 8:19 UTC (permalink / raw)
To: Xiaoyao Li
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Marcelo Tosatti, kvm, qemu-devel, Michael Roth, Claudio Fontana,
Gerd Hoffmann, Isaku Yamahata, Chenyi Qiang
Xiaoyao Li <xiaoyao.li@intel.com> writes:
> Introduce tdx-guest object which inherits CONFIDENTIAL_GUEST_SUPPORT,
> and will be used to create TDX VMs (TDs) by
>
> qemu -machine ...,confidential-guest-support=tdx0 \
> -object tdx-guest,id=tdx0
>
> So far, it has no QAPI member/properety decleared and only one internal
> member 'attributes' with fixed value 0 that not configurable.
>
> QAPI properties will be added later.
>
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> Acked-by: Markus Armbruster <armbru@redhat.com>
I'm happy with the commit message now. Thanks!
^ permalink raw reply [flat|nested] 124+ messages in thread
* [PATCH v5 13/65] target/i386: Implement mc->kvm_type() to get VM type
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (11 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 12/65] i386: Introduce tdx-guest object Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-02-29 6:36 ` [PATCH v5 14/65] i386/tdx: Implement tdx_kvm_init() to initialize TDX VM context Xiaoyao Li
` (51 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
TDX VM requires VM type KVM_X86_TDX_VM to be passed to
kvm_ioctl(KVM_CREATE_VM). Hence implement mc->kvm_type() for i386
architecture.
If tdx-guest object is specified to confidential-guest-support, like,
qemu -machine ...,confidential-guest-support=tdx0 \
-object tdx-guest,id=tdx0,...
it parses VM type as KVM_X86_TDX_VM. Otherwise, it's KVM_X86_DEFAULT_VM.
Also store the vm_type in MachineState for other code to query what the
VM type is.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
Changes in v4:
- fix the build error of kvm_get_vm_type() when --disable-kvm;
---
hw/i386/x86.c | 12 ++++++++++++
include/hw/i386/x86.h | 1 +
target/i386/kvm/kvm.c | 30 ++++++++++++++++++++++++++++++
target/i386/kvm/kvm_i386.h | 1 +
4 files changed, 44 insertions(+)
diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index 807e09bcdbb8..fa7095310f37 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -1389,6 +1389,17 @@ static void machine_set_sgx_epc(Object *obj, Visitor *v, const char *name,
qapi_free_SgxEPCList(list);
}
+static int x86_kvm_type(MachineState *ms, const char *vm_type)
+{
+ X86MachineState *x86ms = X86_MACHINE(ms);
+ int kvm_type;
+
+ kvm_type = kvm_enabled() ? kvm_get_vm_type(ms, vm_type) : 0;
+ x86ms->vm_type = kvm_type;
+
+ return kvm_type;
+}
+
static void x86_machine_initfn(Object *obj)
{
X86MachineState *x86ms = X86_MACHINE(obj);
@@ -1413,6 +1424,7 @@ static void x86_machine_class_init(ObjectClass *oc, void *data)
mc->cpu_index_to_instance_props = x86_cpu_index_to_props;
mc->get_default_cpu_node_id = x86_get_default_cpu_node_id;
mc->possible_cpu_arch_ids = x86_possible_cpu_arch_ids;
+ mc->kvm_type = x86_kvm_type;
x86mc->save_tsc_khz = true;
x86mc->fwcfg_dma_enabled = true;
nc->nmi_monitor_handler = x86_nmi;
diff --git a/include/hw/i386/x86.h b/include/hw/i386/x86.h
index 4dc30dcb4d26..d28e79cc484a 100644
--- a/include/hw/i386/x86.h
+++ b/include/hw/i386/x86.h
@@ -43,6 +43,7 @@ struct X86MachineState {
MachineState parent;
/*< public >*/
+ unsigned int vm_type;
/* Pointers to devices and objects: */
ISADevice *rtc;
diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index ca4e1fb72dd9..898f8fb30c61 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -32,6 +32,7 @@
#include "sysemu/runstate.h"
#include "kvm_i386.h"
#include "sev.h"
+#include "tdx.h"
#include "xen-emu.h"
#include "hyperv.h"
#include "hyperv-proto.h"
@@ -161,6 +162,35 @@ static KVMMSRHandlers msr_handlers[KVM_MSR_FILTER_MAX_RANGES];
static RateLimit bus_lock_ratelimit_ctrl;
static int kvm_get_one_msr(X86CPU *cpu, int index, uint64_t *value);
+static const char *vm_type_name[] = {
+ [KVM_X86_DEFAULT_VM] = "default",
+ [KVM_X86_TDX_VM] = "tdx",
+};
+
+int kvm_get_vm_type(MachineState *ms, const char *vm_type)
+{
+ int kvm_type = KVM_X86_DEFAULT_VM;
+
+ if (ms->cgs && object_dynamic_cast(OBJECT(ms->cgs), TYPE_TDX_GUEST)) {
+ kvm_type = KVM_X86_TDX_VM;
+ }
+
+ /*
+ * old KVM doesn't support KVM_CAP_VM_TYPES and KVM_X86_DEFAULT_VM
+ * is always supported
+ */
+ if (kvm_type == KVM_X86_DEFAULT_VM) {
+ return kvm_type;
+ }
+
+ if (!(kvm_check_extension(KVM_STATE(ms->accelerator), KVM_CAP_VM_TYPES) & BIT(kvm_type))) {
+ error_report("vm-type %s not supported by KVM", vm_type_name[kvm_type]);
+ exit(1);
+ }
+
+ return kvm_type;
+}
+
bool kvm_has_smm(void)
{
return kvm_vm_check_extension(kvm_state, KVM_CAP_X86_SMM);
diff --git a/target/i386/kvm/kvm_i386.h b/target/i386/kvm/kvm_i386.h
index 30fedcffea3e..55fb25fa8e2e 100644
--- a/target/i386/kvm/kvm_i386.h
+++ b/target/i386/kvm/kvm_i386.h
@@ -37,6 +37,7 @@ bool kvm_hv_vpindex_settable(void);
bool kvm_enable_sgx_provisioning(KVMState *s);
bool kvm_hyperv_expand_features(X86CPU *cpu, Error **errp);
+int kvm_get_vm_type(MachineState *ms, const char *vm_type);
void kvm_arch_reset_vcpu(X86CPU *cs);
void kvm_arch_after_reset_vcpu(X86CPU *cpu);
void kvm_arch_do_init_vcpu(X86CPU *cs);
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 14/65] i386/tdx: Implement tdx_kvm_init() to initialize TDX VM context
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (12 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 13/65] target/i386: Implement mc->kvm_type() to get VM type Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-02-29 6:36 ` [PATCH v5 15/65] i386/tdx: Get tdx_capabilities via KVM_TDX_CAPABILITIES Xiaoyao Li
` (50 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
Implement TDX specific ConfidentialGuestSupportClass::kvm_init()
callback, tdx_kvm_init().
Set ms->require_guest_memfd to true to require private guest memfd
allocation for any memory backend.
More TDX specific initialization will be added later.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
Changes in v5:
- remove Acked-by from Gerd since the patch changed due to use
ConfidentialGuestSupportClass::kvm_init();
---
target/i386/kvm/kvm.c | 11 +----------
target/i386/kvm/tdx.c | 13 +++++++++++++
2 files changed, 14 insertions(+), 10 deletions(-)
diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index 898f8fb30c61..52d99d30bdc8 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -2550,16 +2550,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
Error *local_err = NULL;
/*
- * Initialize SEV context, if required
- *
- * If no memory encryption is requested (ms->cgs == NULL) this is
- * a no-op.
- *
- * It's also a no-op if a non-SEV confidential guest support
- * mechanism is selected. SEV is the only mechanism available to
- * select on x86 at present, so this doesn't arise, but if new
- * mechanisms are supported in future (e.g. TDX), they'll need
- * their own initialization either here or elsewhere.
+ * Initialize confidential guest (SEV/TDX) context, if required
*/
if (ms->cgs) {
ret = confidential_guest_kvm_init(ms->cgs, &local_err);
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index d3792d4a3d56..d9a1dd46dc69 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -14,8 +14,18 @@
#include "qemu/osdep.h"
#include "qom/object_interfaces.h"
+#include "hw/i386/x86.h"
#include "tdx.h"
+static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
+{
+ MachineState *ms = MACHINE(qdev_get_machine());
+
+ ms->require_guest_memfd = true;
+
+ return 0;
+}
+
/* tdx guest */
OBJECT_DEFINE_TYPE_WITH_INTERFACES(TdxGuest,
tdx_guest,
@@ -37,4 +47,7 @@ static void tdx_guest_finalize(Object *obj)
static void tdx_guest_class_init(ObjectClass *oc, void *data)
{
+ ConfidentialGuestSupportClass *klass = CONFIDENTIAL_GUEST_SUPPORT_CLASS(oc);
+
+ klass->kvm_init = tdx_kvm_init;
}
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 15/65] i386/tdx: Get tdx_capabilities via KVM_TDX_CAPABILITIES
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (13 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 14/65] i386/tdx: Implement tdx_kvm_init() to initialize TDX VM context Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-03-12 23:02 ` Isaku Yamahata
2024-03-19 2:43 ` Wang, Lei
2024-02-29 6:36 ` [PATCH v5 16/65] i386/tdx: Introduce is_tdx_vm() helper and cache tdx_guest object Xiaoyao Li
` (49 subsequent siblings)
64 siblings, 2 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
KVM provides TDX capabilities via sub command KVM_TDX_CAPABILITIES of
IOCTL(KVM_MEMORY_ENCRYPT_OP). Get the capabilities when initializing
TDX context. It will be used to validate user's setting later.
Since there is no interface reporting how many cpuid configs contains in
KVM_TDX_CAPABILITIES, QEMU chooses to try starting with a known number
and abort when it exceeds KVM_MAX_CPUID_ENTRIES.
Besides, introduce the interfaces to invoke TDX "ioctls" at different
scope (KVM, VM and VCPU) in preparation.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
Changes in v4:
- use {} to initialize struct kvm_tdx_cmd, to avoid memset();
- remove tdx_platform_ioctl() because no user;
Changes in v3:
- rename __tdx_ioctl() to tdx_ioctl_internal()
- Pass errp in get_tdx_capabilities();
changes in v2:
- Make the error message more clear;
changes in v1:
- start from nr_cpuid_configs = 6 for the loop;
- stop the loop when nr_cpuid_configs exceeds KVM_MAX_CPUID_ENTRIES;
---
target/i386/kvm/kvm.c | 2 -
target/i386/kvm/kvm_i386.h | 2 +
target/i386/kvm/tdx.c | 91 +++++++++++++++++++++++++++++++++++++-
3 files changed, 92 insertions(+), 3 deletions(-)
diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index 52d99d30bdc8..0e68e80f4291 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -1685,8 +1685,6 @@ static int hyperv_init_vcpu(X86CPU *cpu)
static Error *invtsc_mig_blocker;
-#define KVM_MAX_CPUID_ENTRIES 100
-
static void kvm_init_xsave(CPUX86State *env)
{
if (has_xsave2) {
diff --git a/target/i386/kvm/kvm_i386.h b/target/i386/kvm/kvm_i386.h
index 55fb25fa8e2e..c3ef46a97a7b 100644
--- a/target/i386/kvm/kvm_i386.h
+++ b/target/i386/kvm/kvm_i386.h
@@ -13,6 +13,8 @@
#include "sysemu/kvm.h"
+#define KVM_MAX_CPUID_ENTRIES 100
+
#ifdef CONFIG_KVM
#define kvm_pit_in_kernel() \
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index d9a1dd46dc69..2b956450a083 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -12,18 +12,107 @@
*/
#include "qemu/osdep.h"
+#include "qemu/error-report.h"
+#include "qapi/error.h"
#include "qom/object_interfaces.h"
+#include "sysemu/kvm.h"
#include "hw/i386/x86.h"
+#include "kvm_i386.h"
#include "tdx.h"
+static struct kvm_tdx_capabilities *tdx_caps;
+
+enum tdx_ioctl_level{
+ TDX_VM_IOCTL,
+ TDX_VCPU_IOCTL,
+};
+
+static int tdx_ioctl_internal(void *state, enum tdx_ioctl_level level, int cmd_id,
+ __u32 flags, void *data)
+{
+ struct kvm_tdx_cmd tdx_cmd = {};
+ int r;
+
+ tdx_cmd.id = cmd_id;
+ tdx_cmd.flags = flags;
+ tdx_cmd.data = (__u64)(unsigned long)data;
+
+ switch (level) {
+ case TDX_VM_IOCTL:
+ r = kvm_vm_ioctl(kvm_state, KVM_MEMORY_ENCRYPT_OP, &tdx_cmd);
+ break;
+ case TDX_VCPU_IOCTL:
+ r = kvm_vcpu_ioctl(state, KVM_MEMORY_ENCRYPT_OP, &tdx_cmd);
+ break;
+ default:
+ error_report("Invalid tdx_ioctl_level %d", level);
+ exit(1);
+ }
+
+ return r;
+}
+
+static inline int tdx_vm_ioctl(int cmd_id, __u32 flags, void *data)
+{
+ return tdx_ioctl_internal(NULL, TDX_VM_IOCTL, cmd_id, flags, data);
+}
+
+static inline int tdx_vcpu_ioctl(void *vcpu_fd, int cmd_id, __u32 flags,
+ void *data)
+{
+ return tdx_ioctl_internal(vcpu_fd, TDX_VCPU_IOCTL, cmd_id, flags, data);
+}
+
+static int get_tdx_capabilities(Error **errp)
+{
+ struct kvm_tdx_capabilities *caps;
+ /* 1st generation of TDX reports 6 cpuid configs */
+ int nr_cpuid_configs = 6;
+ size_t size;
+ int r;
+
+ do {
+ size = sizeof(struct kvm_tdx_capabilities) +
+ nr_cpuid_configs * sizeof(struct kvm_tdx_cpuid_config);
+ caps = g_malloc0(size);
+ caps->nr_cpuid_configs = nr_cpuid_configs;
+
+ r = tdx_vm_ioctl(KVM_TDX_CAPABILITIES, 0, caps);
+ if (r == -E2BIG) {
+ g_free(caps);
+ nr_cpuid_configs *= 2;
+ if (nr_cpuid_configs > KVM_MAX_CPUID_ENTRIES) {
+ error_setg(errp, "%s: KVM TDX seems broken that number of CPUID "
+ "entries in kvm_tdx_capabilities exceeds limit %d",
+ __func__, KVM_MAX_CPUID_ENTRIES);
+ return r;
+ }
+ } else if (r < 0) {
+ g_free(caps);
+ error_setg_errno(errp, -r, "%s: KVM_TDX_CAPABILITIES failed", __func__);
+ return r;
+ }
+ }
+ while (r == -E2BIG);
+
+ tdx_caps = caps;
+
+ return 0;
+}
+
static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
{
MachineState *ms = MACHINE(qdev_get_machine());
+ int r = 0;
ms->require_guest_memfd = true;
- return 0;
+ if (!tdx_caps) {
+ r = get_tdx_capabilities(errp);
+ }
+
+ return r;
}
/* tdx guest */
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* Re: [PATCH v5 15/65] i386/tdx: Get tdx_capabilities via KVM_TDX_CAPABILITIES
2024-02-29 6:36 ` [PATCH v5 15/65] i386/tdx: Get tdx_capabilities via KVM_TDX_CAPABILITIES Xiaoyao Li
@ 2024-03-12 23:02 ` Isaku Yamahata
2024-03-19 2:43 ` Wang, Lei
1 sibling, 0 replies; 124+ messages in thread
From: Isaku Yamahata @ 2024-03-12 23:02 UTC (permalink / raw)
To: Xiaoyao Li
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti, kvm, qemu-devel, Michael Roth,
Claudio Fontana, Gerd Hoffmann, Isaku Yamahata, Chenyi Qiang,
isaku.yamahata, isaku.yamahata
On Thu, Feb 29, 2024 at 01:36:36AM -0500,
Xiaoyao Li <xiaoyao.li@intel.com> wrote:
> KVM provides TDX capabilities via sub command KVM_TDX_CAPABILITIES of
> IOCTL(KVM_MEMORY_ENCRYPT_OP). Get the capabilities when initializing
> TDX context. It will be used to validate user's setting later.
>
> Since there is no interface reporting how many cpuid configs contains in
> KVM_TDX_CAPABILITIES, QEMU chooses to try starting with a known number
> and abort when it exceeds KVM_MAX_CPUID_ENTRIES.
>
> Besides, introduce the interfaces to invoke TDX "ioctls" at different
> scope (KVM, VM and VCPU) in preparation.
>
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
> Changes in v4:
> - use {} to initialize struct kvm_tdx_cmd, to avoid memset();
> - remove tdx_platform_ioctl() because no user;
>
> Changes in v3:
> - rename __tdx_ioctl() to tdx_ioctl_internal()
> - Pass errp in get_tdx_capabilities();
>
> changes in v2:
> - Make the error message more clear;
>
> changes in v1:
> - start from nr_cpuid_configs = 6 for the loop;
> - stop the loop when nr_cpuid_configs exceeds KVM_MAX_CPUID_ENTRIES;
> ---
> target/i386/kvm/kvm.c | 2 -
> target/i386/kvm/kvm_i386.h | 2 +
> target/i386/kvm/tdx.c | 91 +++++++++++++++++++++++++++++++++++++-
> 3 files changed, 92 insertions(+), 3 deletions(-)
>
> diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
> index 52d99d30bdc8..0e68e80f4291 100644
> --- a/target/i386/kvm/kvm.c
> +++ b/target/i386/kvm/kvm.c
> @@ -1685,8 +1685,6 @@ static int hyperv_init_vcpu(X86CPU *cpu)
>
> static Error *invtsc_mig_blocker;
>
> -#define KVM_MAX_CPUID_ENTRIES 100
> -
> static void kvm_init_xsave(CPUX86State *env)
> {
> if (has_xsave2) {
> diff --git a/target/i386/kvm/kvm_i386.h b/target/i386/kvm/kvm_i386.h
> index 55fb25fa8e2e..c3ef46a97a7b 100644
> --- a/target/i386/kvm/kvm_i386.h
> +++ b/target/i386/kvm/kvm_i386.h
> @@ -13,6 +13,8 @@
>
> #include "sysemu/kvm.h"
>
> +#define KVM_MAX_CPUID_ENTRIES 100
> +
> #ifdef CONFIG_KVM
>
> #define kvm_pit_in_kernel() \
> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
> index d9a1dd46dc69..2b956450a083 100644
> --- a/target/i386/kvm/tdx.c
> +++ b/target/i386/kvm/tdx.c
> @@ -12,18 +12,107 @@
> */
>
> #include "qemu/osdep.h"
> +#include "qemu/error-report.h"
> +#include "qapi/error.h"
> #include "qom/object_interfaces.h"
> +#include "sysemu/kvm.h"
>
> #include "hw/i386/x86.h"
> +#include "kvm_i386.h"
> #include "tdx.h"
>
> +static struct kvm_tdx_capabilities *tdx_caps;
> +
> +enum tdx_ioctl_level{
> + TDX_VM_IOCTL,
> + TDX_VCPU_IOCTL,
> +};
> +
> +static int tdx_ioctl_internal(void *state, enum tdx_ioctl_level level, int cmd_id,
> + __u32 flags, void *data)
> +{
> + struct kvm_tdx_cmd tdx_cmd = {};
> + int r;
> +
> + tdx_cmd.id = cmd_id;
> + tdx_cmd.flags = flags;
> + tdx_cmd.data = (__u64)(unsigned long)data;
> +
> + switch (level) {
> + case TDX_VM_IOCTL:
> + r = kvm_vm_ioctl(kvm_state, KVM_MEMORY_ENCRYPT_OP, &tdx_cmd);
> + break;
> + case TDX_VCPU_IOCTL:
> + r = kvm_vcpu_ioctl(state, KVM_MEMORY_ENCRYPT_OP, &tdx_cmd);
> + break;
> + default:
> + error_report("Invalid tdx_ioctl_level %d", level);
> + exit(1);
> + }
> +
> + return r;
> +}
> +
> +static inline int tdx_vm_ioctl(int cmd_id, __u32 flags, void *data)
> +{
> + return tdx_ioctl_internal(NULL, TDX_VM_IOCTL, cmd_id, flags, data);
> +}
> +
> +static inline int tdx_vcpu_ioctl(void *vcpu_fd, int cmd_id, __u32 flags,
> + void *data)
As kvm_vcpu_ioctl(CPUState *cpu, int type, ...) takes CPUState *, this can be
tdx_vcpu_ioctl(CPUState *cpu, ) instead of void *.
I struggled to fin my mistake to pass "int vcpu_fd" to this function.
--
Isaku Yamahata <isaku.yamahata@intel.com>
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 15/65] i386/tdx: Get tdx_capabilities via KVM_TDX_CAPABILITIES
2024-02-29 6:36 ` [PATCH v5 15/65] i386/tdx: Get tdx_capabilities via KVM_TDX_CAPABILITIES Xiaoyao Li
2024-03-12 23:02 ` Isaku Yamahata
@ 2024-03-19 2:43 ` Wang, Lei
1 sibling, 0 replies; 124+ messages in thread
From: Wang, Lei @ 2024-03-19 2:43 UTC (permalink / raw)
To: Xiaoyao Li, Paolo Bonzini, David Hildenbrand, Igor Mammedov,
Eduardo Habkost, Marcel Apfelbaum, Philippe Mathieu-Daudé,
Yanan Wang, Michael S. Tsirkin, Richard Henderson, Ani Sinha,
Peter Xu, Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang
On 2/29/2024 14:36, Xiaoyao Li wrote:> KVM provides TDX capabilities via sub
command KVM_TDX_CAPABILITIES of
> IOCTL(KVM_MEMORY_ENCRYPT_OP). Get the capabilities when initializing
> TDX context. It will be used to validate user's setting later.
>
> Since there is no interface reporting how many cpuid configs contains in
> KVM_TDX_CAPABILITIES, QEMU chooses to try starting with a known number
> and abort when it exceeds KVM_MAX_CPUID_ENTRIES.
>
> Besides, introduce the interfaces to invoke TDX "ioctls" at different
> scope (KVM, VM and VCPU) in preparation.
tdx_platform_ioctl() is dropped because no user so suggest rephrasing this
statement because no KVM scope ioctl interface is introduced in this patch.
>
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
> Changes in v4:
> - use {} to initialize struct kvm_tdx_cmd, to avoid memset();
> - remove tdx_platform_ioctl() because no user;
>
> Changes in v3:
> - rename __tdx_ioctl() to tdx_ioctl_internal()
> - Pass errp in get_tdx_capabilities();
>
> changes in v2:
> - Make the error message more clear;
>
> changes in v1:
> - start from nr_cpuid_configs = 6 for the loop;
> - stop the loop when nr_cpuid_configs exceeds KVM_MAX_CPUID_ENTRIES;
> ---
> target/i386/kvm/kvm.c | 2 -
> target/i386/kvm/kvm_i386.h | 2 +
> target/i386/kvm/tdx.c | 91 +++++++++++++++++++++++++++++++++++++-
> 3 files changed, 92 insertions(+), 3 deletions(-)
>
> diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
> index 52d99d30bdc8..0e68e80f4291 100644
> --- a/target/i386/kvm/kvm.c
> +++ b/target/i386/kvm/kvm.c
> @@ -1685,8 +1685,6 @@ static int hyperv_init_vcpu(X86CPU *cpu)
>
> static Error *invtsc_mig_blocker;
>
> -#define KVM_MAX_CPUID_ENTRIES 100
> -
> static void kvm_init_xsave(CPUX86State *env)
> {
> if (has_xsave2) {
> diff --git a/target/i386/kvm/kvm_i386.h b/target/i386/kvm/kvm_i386.h
> index 55fb25fa8e2e..c3ef46a97a7b 100644
> --- a/target/i386/kvm/kvm_i386.h
> +++ b/target/i386/kvm/kvm_i386.h
> @@ -13,6 +13,8 @@
>
> #include "sysemu/kvm.h"
>
> +#define KVM_MAX_CPUID_ENTRIES 100
> +
> #ifdef CONFIG_KVM
>
> #define kvm_pit_in_kernel() \
> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
> index d9a1dd46dc69..2b956450a083 100644
> --- a/target/i386/kvm/tdx.c
> +++ b/target/i386/kvm/tdx.c
> @@ -12,18 +12,107 @@
> */
>
> #include "qemu/osdep.h"
> +#include "qemu/error-report.h"
> +#include "qapi/error.h"
> #include "qom/object_interfaces.h"
> +#include "sysemu/kvm.h"
>
> #include "hw/i386/x86.h"
> +#include "kvm_i386.h"
> #include "tdx.h"
>
> +static struct kvm_tdx_capabilities *tdx_caps;
> +
> +enum tdx_ioctl_level{
> + TDX_VM_IOCTL,
> + TDX_VCPU_IOCTL,
> +};
> +
> +static int tdx_ioctl_internal(void *state, enum tdx_ioctl_level level, int cmd_id,
> + __u32 flags, void *data)
> +{
> + struct kvm_tdx_cmd tdx_cmd = {};
> + int r;
> +
> + tdx_cmd.id = cmd_id;
> + tdx_cmd.flags = flags;
> + tdx_cmd.data = (__u64)(unsigned long)data;
> +
> + switch (level) {
> + case TDX_VM_IOCTL:
> + r = kvm_vm_ioctl(kvm_state, KVM_MEMORY_ENCRYPT_OP, &tdx_cmd);
> + break;
> + case TDX_VCPU_IOCTL:
> + r = kvm_vcpu_ioctl(state, KVM_MEMORY_ENCRYPT_OP, &tdx_cmd);
> + break;
> + default:
> + error_report("Invalid tdx_ioctl_level %d", level);
> + exit(1);
> + }
> +
> + return r;
> +}
> +
> +static inline int tdx_vm_ioctl(int cmd_id, __u32 flags, void *data)
> +{
> + return tdx_ioctl_internal(NULL, TDX_VM_IOCTL, cmd_id, flags, data);
> +}
> +
> +static inline int tdx_vcpu_ioctl(void *vcpu_fd, int cmd_id, __u32 flags,
> + void *data)
> +{
> + return tdx_ioctl_internal(vcpu_fd, TDX_VCPU_IOCTL, cmd_id, flags, data);
> +}
> +
> +static int get_tdx_capabilities(Error **errp)
> +{
> + struct kvm_tdx_capabilities *caps;
> + /* 1st generation of TDX reports 6 cpuid configs */
> + int nr_cpuid_configs = 6;
> + size_t size;
> + int r;
> +
> + do {
> + size = sizeof(struct kvm_tdx_capabilities) +
> + nr_cpuid_configs * sizeof(struct kvm_tdx_cpuid_config);
> + caps = g_malloc0(size);
> + caps->nr_cpuid_configs = nr_cpuid_configs;
> +
> + r = tdx_vm_ioctl(KVM_TDX_CAPABILITIES, 0, caps);
> + if (r == -E2BIG) {
> + g_free(caps);
> + nr_cpuid_configs *= 2;
> + if (nr_cpuid_configs > KVM_MAX_CPUID_ENTRIES) {
> + error_setg(errp, "%s: KVM TDX seems broken that number of CPUID "
> + "entries in kvm_tdx_capabilities exceeds limit %d",
> + __func__, KVM_MAX_CPUID_ENTRIES);
> + return r;
> + }
> + } else if (r < 0) {
> + g_free(caps);
> + error_setg_errno(errp, -r, "%s: KVM_TDX_CAPABILITIES failed", __func__);
> + return r;
> + }
> + }
> + while (r == -E2BIG);
> +
> + tdx_caps = caps;
> +
> + return 0;
> +}
> +
> static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
> {
> MachineState *ms = MACHINE(qdev_get_machine());
> + int r = 0;
>
> ms->require_guest_memfd = true;
>
> - return 0;
> + if (!tdx_caps) {
> + r = get_tdx_capabilities(errp);
> + }
> +
> + return r;
> }
>
> /* tdx guest */
^ permalink raw reply [flat|nested] 124+ messages in thread
* [PATCH v5 16/65] i386/tdx: Introduce is_tdx_vm() helper and cache tdx_guest object
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (14 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 15/65] i386/tdx: Get tdx_capabilities via KVM_TDX_CAPABILITIES Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-02-29 6:36 ` [PATCH v5 17/65] i386/tdx: Adjust the supported CPUID based on TDX restrictions Xiaoyao Li
` (48 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
It will need special handling for TDX VMs all around the QEMU.
Introduce is_tdx_vm() helper to query if it's a TDX VM.
Cache tdx_guest object thus no need to cast from ms->cgs every time.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
changes in v3:
- replace object_dynamic_cast with TDX_GUEST();
---
target/i386/kvm/tdx.c | 15 ++++++++++++++-
target/i386/kvm/tdx.h | 10 ++++++++++
2 files changed, 24 insertions(+), 1 deletion(-)
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 2b956450a083..756058f2ed4a 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -21,8 +21,16 @@
#include "kvm_i386.h"
#include "tdx.h"
+static TdxGuest *tdx_guest;
+
static struct kvm_tdx_capabilities *tdx_caps;
+/* It's valid after kvm_arch_init()->kvm_tdx_init() */
+bool is_tdx_vm(void)
+{
+ return !!tdx_guest;
+}
+
enum tdx_ioctl_level{
TDX_VM_IOCTL,
TDX_VCPU_IOCTL,
@@ -104,15 +112,20 @@ static int get_tdx_capabilities(Error **errp)
static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
{
MachineState *ms = MACHINE(qdev_get_machine());
+ TdxGuest *tdx = TDX_GUEST(cgs);
int r = 0;
ms->require_guest_memfd = true;
if (!tdx_caps) {
r = get_tdx_capabilities(errp);
+ if (r) {
+ return r;
+ }
}
- return r;
+ tdx_guest = tdx;
+ return 0;
}
/* tdx guest */
diff --git a/target/i386/kvm/tdx.h b/target/i386/kvm/tdx.h
index 415aeb5af746..d5b4c179fbf7 100644
--- a/target/i386/kvm/tdx.h
+++ b/target/i386/kvm/tdx.h
@@ -1,6 +1,10 @@
#ifndef QEMU_I386_TDX_H
#define QEMU_I386_TDX_H
+#ifndef CONFIG_USER_ONLY
+#include CONFIG_DEVICES /* CONFIG_TDX */
+#endif
+
#include "exec/confidential-guest-support.h"
#define TYPE_TDX_GUEST "tdx-guest"
@@ -16,4 +20,10 @@ typedef struct TdxGuest {
uint64_t attributes; /* TD attributes */
} TdxGuest;
+#ifdef CONFIG_TDX
+bool is_tdx_vm(void);
+#else
+#define is_tdx_vm() 0
+#endif /* CONFIG_TDX */
+
#endif /* QEMU_I386_TDX_H */
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 17/65] i386/tdx: Adjust the supported CPUID based on TDX restrictions
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (15 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 16/65] i386/tdx: Introduce is_tdx_vm() helper and cache tdx_guest object Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-05-31 8:47 ` Duan, Zhenzhong
2024-02-29 6:36 ` [PATCH v5 18/65] i386/tdx: Make Intel-PT unsupported for TD guest Xiaoyao Li
` (47 subsequent siblings)
64 siblings, 1 reply; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
According to Chapter "CPUID Virtualization" in TDX module spec, CPUID
bits of TD can be classified into 6 types:
------------------------------------------------------------------------
1 | As configured | configurable by VMM, independent of native value;
------------------------------------------------------------------------
2 | As configured | configurable by VMM if the bit is supported natively
(if native) | Otherwise it equals as native(0).
------------------------------------------------------------------------
3 | Fixed | fixed to 0/1
------------------------------------------------------------------------
4 | Native | reflect the native value
------------------------------------------------------------------------
5 | Calculated | calculated by TDX module.
------------------------------------------------------------------------
6 | Inducing #VE | get #VE exception
------------------------------------------------------------------------
Note:
1. All the configurable XFAM related features and TD attributes related
features fall into type #2. And fixed0/1 bits of XFAM and TD
attributes fall into type #3.
2. For CPUID leaves not listed in "CPUID virtualization Overview" table
in TDX module spec, TDX module injects #VE to TDs when those are
queried. For this case, TDs can request CPUID emulation from VMM via
TDVMCALL and the values are fully controlled by VMM.
Due to TDX module has its own virtualization policy on CPUID bits, it leads
to what reported via KVM_GET_SUPPORTED_CPUID diverges from the supported
CPUID bits for TDs. In order to keep a consistent CPUID configuration
between VMM and TDs. Adjust supported CPUID for TDs based on TDX
restrictions.
Currently only focus on the CPUID leaves recognized by QEMU's
feature_word_info[] that are indexed by a FeatureWord.
Introduce a TDX CPUID lookup table, which maintains 1 entry for each
FeatureWord. Each entry has below fields:
- tdx_fixed0/1: The bits that are fixed as 0/1;
- depends_on_vmm_cap: The bits that are configurable from the view of
TDX module. But they requires emulation of VMM
when configured as enabled. For those, they are
not supported if VMM doesn't report them as
supported. So they need be fixed up by checking
if VMM supports them.
- inducing_ve: TD gets #VE when querying this CPUID leaf. The result is
totally configurable by VMM.
- supported_on_ve: It's valid only when @inducing_ve is true. It represents
the maximum feature set supported that be emulated
for TDs.
By applying TDX CPUID lookup table and TDX capabilities reported from
TDX module, the supported CPUID for TDs can be obtained from following
steps:
- get the base of VMM supported feature set;
- if the leaf is not a FeatureWord just return VMM's value without
modification;
- if the leaf is an inducing_ve type, applying supported_on_ve mask and
return;
- include all native bits, it covers type #2, #4, and parts of type #1.
(it also includes some unsupported bits. The following step will
correct it.)
- apply fixed0/1 to it (it covers #3, and rectifies the previous step);
- add configurable bits (it covers the other part of type #1);
- fix the ones in vmm_fixup;
(Calculated type is ignored since it's determined at runtime).
Co-developed-by: Chenyi Qiang <chenyi.qiang@intel.com>
Signed-off-by: Chenyi Qiang <chenyi.qiang@intel.com>
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
target/i386/cpu.h | 16 +++
target/i386/kvm/kvm.c | 4 +
target/i386/kvm/tdx.c | 263 ++++++++++++++++++++++++++++++++++++++++++
target/i386/kvm/tdx.h | 3 +
4 files changed, 286 insertions(+)
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 952174bb6f52..7bd604f802a1 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -787,6 +787,8 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
/* Support RDFSBASE/RDGSBASE/WRFSBASE/WRGSBASE */
#define CPUID_7_0_EBX_FSGSBASE (1U << 0)
+/* Support for TSC adjustment MSR 0x3B */
+#define CPUID_7_0_EBX_TSC_ADJUST (1U << 1)
/* Support SGX */
#define CPUID_7_0_EBX_SGX (1U << 2)
/* 1st Group of Advanced Bit Manipulation Extensions */
@@ -805,8 +807,12 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
#define CPUID_7_0_EBX_INVPCID (1U << 10)
/* Restricted Transactional Memory */
#define CPUID_7_0_EBX_RTM (1U << 11)
+/* Cache QoS Monitoring */
+#define CPUID_7_0_EBX_PQM (1U << 12)
/* Memory Protection Extension */
#define CPUID_7_0_EBX_MPX (1U << 14)
+/* Resource Director Technology Allocation */
+#define CPUID_7_0_EBX_RDT_A (1U << 15)
/* AVX-512 Foundation */
#define CPUID_7_0_EBX_AVX512F (1U << 16)
/* AVX-512 Doubleword & Quadword Instruction */
@@ -862,10 +868,16 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
#define CPUID_7_0_ECX_AVX512VNNI (1U << 11)
/* Support for VPOPCNT[B,W] and VPSHUFBITQMB */
#define CPUID_7_0_ECX_AVX512BITALG (1U << 12)
+/* Intel Total Memory Encryption */
+#define CPUID_7_0_ECX_TME (1U << 13)
/* POPCNT for vectors of DW/QW */
#define CPUID_7_0_ECX_AVX512_VPOPCNTDQ (1U << 14)
+/* Placeholder for bit 15 */
+#define CPUID_7_0_ECX_FZM (1U << 15)
/* 5-level Page Tables */
#define CPUID_7_0_ECX_LA57 (1U << 16)
+/* MAWAU for MPX */
+#define CPUID_7_0_ECX_MAWAU (31U << 17)
/* Read Processor ID */
#define CPUID_7_0_ECX_RDPID (1U << 22)
/* Bus Lock Debug Exception */
@@ -876,6 +888,8 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
#define CPUID_7_0_ECX_MOVDIRI (1U << 27)
/* Move 64 Bytes as Direct Store Instruction */
#define CPUID_7_0_ECX_MOVDIR64B (1U << 28)
+/* ENQCMD and ENQCMDS instructions */
+#define CPUID_7_0_ECX_ENQCMD (1U << 29)
/* Support SGX Launch Control */
#define CPUID_7_0_ECX_SGX_LC (1U << 30)
/* Protection Keys for Supervisor-mode Pages */
@@ -893,6 +907,8 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
#define CPUID_7_0_EDX_SERIALIZE (1U << 14)
/* TSX Suspend Load Address Tracking instruction */
#define CPUID_7_0_EDX_TSX_LDTRK (1U << 16)
+/* PCONFIG instruction */
+#define CPUID_7_0_EDX_PCONFIG (1U << 18)
/* Architectural LBRs */
#define CPUID_7_0_EDX_ARCH_LBR (1U << 19)
/* AMX_BF16 instruction */
diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index 0e68e80f4291..389b631c03dd 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -520,6 +520,10 @@ uint32_t kvm_arch_get_supported_cpuid(KVMState *s, uint32_t function,
ret |= 1U << KVM_HINTS_REALTIME;
}
+ if (is_tdx_vm()) {
+ tdx_get_supported_cpuid(function, index, reg, &ret);
+ }
+
return ret;
}
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 756058f2ed4a..85d96140b450 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -15,11 +15,129 @@
#include "qemu/error-report.h"
#include "qapi/error.h"
#include "qom/object_interfaces.h"
+#include "standard-headers/asm-x86/kvm_para.h"
#include "sysemu/kvm.h"
+#include "sysemu/sysemu.h"
#include "hw/i386/x86.h"
#include "kvm_i386.h"
#include "tdx.h"
+#include "../cpu-internal.h"
+
+#define TDX_SUPPORTED_KVM_FEATURES ((1U << KVM_FEATURE_NOP_IO_DELAY) | \
+ (1U << KVM_FEATURE_PV_UNHALT) | \
+ (1U << KVM_FEATURE_PV_TLB_FLUSH) | \
+ (1U << KVM_FEATURE_PV_SEND_IPI) | \
+ (1U << KVM_FEATURE_POLL_CONTROL) | \
+ (1U << KVM_FEATURE_PV_SCHED_YIELD) | \
+ (1U << KVM_FEATURE_MSI_EXT_DEST_ID))
+
+typedef struct KvmTdxCpuidLookup {
+ uint32_t tdx_fixed0;
+ uint32_t tdx_fixed1;
+
+ /*
+ * The CPUID bits that are configurable from the view of TDX module
+ * but require VMM's support when wanting to enable them.
+ *
+ * For those bits, they cannot be enabled if VMM (KVM/QEMU) doesn't support
+ * them.
+ */
+ uint32_t depends_on_vmm_cap;
+
+ bool inducing_ve;
+ /*
+ * The maximum supported feature set for given inducing-#VE leaf.
+ * It's valid only when .inducing_ve is true.
+ */
+ uint32_t supported_value_on_ve;
+} KvmTdxCpuidLookup;
+
+ /*
+ * QEMU maintained TDX CPUID lookup tables, which reflects how CPUIDs are
+ * virtualized for guest TDs based on "CPUID virtualization" of TDX spec.
+ *
+ * Note:
+ *
+ * This table will be updated runtime by tdx_caps reported by KVM.
+ *
+ */
+static KvmTdxCpuidLookup tdx_cpuid_lookup[FEATURE_WORDS] = {
+ [FEAT_1_EDX] = {
+ .tdx_fixed0 =
+ BIT(10) /* Reserved */ | BIT(20) /* Reserved */ | CPUID_IA64,
+ .tdx_fixed1 =
+ CPUID_MSR | CPUID_PAE | CPUID_MCE | CPUID_APIC |
+ CPUID_MTRR | CPUID_MCA | CPUID_CLFLUSH | CPUID_DTS,
+ .depends_on_vmm_cap =
+ CPUID_ACPI | CPUID_PBE,
+ },
+ [FEAT_1_ECX] = {
+ .tdx_fixed0 =
+ CPUID_EXT_VMX | CPUID_EXT_SMX | BIT(16) /* Reserved */,
+ .tdx_fixed1 =
+ CPUID_EXT_CX16 | CPUID_EXT_PDCM | CPUID_EXT_X2APIC |
+ CPUID_EXT_AES | CPUID_EXT_XSAVE | CPUID_EXT_RDRAND |
+ CPUID_EXT_HYPERVISOR,
+ .depends_on_vmm_cap =
+ CPUID_EXT_EST | CPUID_EXT_TM2 | CPUID_EXT_XTPR | CPUID_EXT_DCA,
+ },
+ [FEAT_8000_0001_EDX] = {
+ .tdx_fixed1 =
+ CPUID_EXT2_NX | CPUID_EXT2_PDPE1GB | CPUID_EXT2_RDTSCP |
+ CPUID_EXT2_LM,
+ },
+ [FEAT_7_0_EBX] = {
+ .tdx_fixed0 =
+ CPUID_7_0_EBX_TSC_ADJUST | CPUID_7_0_EBX_SGX | CPUID_7_0_EBX_MPX,
+ .tdx_fixed1 =
+ CPUID_7_0_EBX_FSGSBASE | CPUID_7_0_EBX_RTM |
+ CPUID_7_0_EBX_RDSEED | CPUID_7_0_EBX_SMAP |
+ CPUID_7_0_EBX_CLFLUSHOPT | CPUID_7_0_EBX_CLWB |
+ CPUID_7_0_EBX_SHA_NI,
+ .depends_on_vmm_cap =
+ CPUID_7_0_EBX_PQM | CPUID_7_0_EBX_RDT_A,
+ },
+ [FEAT_7_0_ECX] = {
+ .tdx_fixed0 =
+ CPUID_7_0_ECX_FZM | CPUID_7_0_ECX_MAWAU |
+ CPUID_7_0_ECX_ENQCMD | CPUID_7_0_ECX_SGX_LC,
+ .tdx_fixed1 =
+ CPUID_7_0_ECX_MOVDIR64B | CPUID_7_0_ECX_BUS_LOCK_DETECT,
+ .depends_on_vmm_cap =
+ CPUID_7_0_ECX_TME,
+ },
+ [FEAT_7_0_EDX] = {
+ .tdx_fixed1 =
+ CPUID_7_0_EDX_SPEC_CTRL | CPUID_7_0_EDX_ARCH_CAPABILITIES |
+ CPUID_7_0_EDX_CORE_CAPABILITY | CPUID_7_0_EDX_SPEC_CTRL_SSBD,
+ .depends_on_vmm_cap =
+ CPUID_7_0_EDX_PCONFIG,
+ },
+ [FEAT_8000_0008_EBX] = {
+ .tdx_fixed0 =
+ ~CPUID_8000_0008_EBX_WBNOINVD,
+ .tdx_fixed1 =
+ CPUID_8000_0008_EBX_WBNOINVD,
+ },
+ [FEAT_XSAVE] = {
+ .tdx_fixed1 =
+ CPUID_XSAVE_XSAVEOPT | CPUID_XSAVE_XSAVEC |
+ CPUID_XSAVE_XSAVES,
+ },
+ [FEAT_6_EAX] = {
+ .inducing_ve = true,
+ .supported_value_on_ve = CPUID_6_EAX_ARAT,
+ },
+ [FEAT_8000_0007_EDX] = {
+ .inducing_ve = true,
+ .supported_value_on_ve = -1U,
+ },
+ [FEAT_KVM] = {
+ .inducing_ve = true,
+ .supported_value_on_ve = TDX_SUPPORTED_KVM_FEATURES,
+ },
+};
static TdxGuest *tdx_guest;
@@ -31,6 +149,151 @@ bool is_tdx_vm(void)
return !!tdx_guest;
}
+static inline uint32_t host_cpuid_reg(uint32_t function,
+ uint32_t index, int reg)
+{
+ uint32_t eax, ebx, ecx, edx;
+ uint32_t ret = 0;
+
+ host_cpuid(function, index, &eax, &ebx, &ecx, &edx);
+
+ switch (reg) {
+ case R_EAX:
+ ret = eax;
+ break;
+ case R_EBX:
+ ret = ebx;
+ break;
+ case R_ECX:
+ ret = ecx;
+ break;
+ case R_EDX:
+ ret = edx;
+ break;
+ }
+ return ret;
+}
+
+/*
+ * get the configurable cpuid bits (can be set to 0 or 1) reported by TDX module
+ * from tdx_caps.
+ */
+static inline uint32_t tdx_cap_cpuid_config(uint32_t function,
+ uint32_t index, int reg)
+{
+ struct kvm_tdx_cpuid_config *cpuid_c;
+ int ret = 0;
+ int i;
+
+ if (tdx_caps->nr_cpuid_configs <= 0) {
+ return ret;
+ }
+
+ for (i = 0; i < tdx_caps->nr_cpuid_configs; i++) {
+ cpuid_c = &tdx_caps->cpuid_configs[i];
+ /* 0xffffffff in sub_leaf means the leaf doesn't require a sublesf */
+ if (cpuid_c->leaf == function &&
+ (cpuid_c->sub_leaf == 0xffffffff || cpuid_c->sub_leaf == index)) {
+ switch (reg) {
+ case R_EAX:
+ ret = cpuid_c->eax;
+ break;
+ case R_EBX:
+ ret = cpuid_c->ebx;
+ break;
+ case R_ECX:
+ ret = cpuid_c->ecx;
+ break;
+ case R_EDX:
+ ret = cpuid_c->edx;
+ break;
+ default:
+ return 0;
+ }
+ }
+ }
+ return ret;
+}
+
+static FeatureWord get_cpuid_featureword_index(uint32_t function,
+ uint32_t index, int reg)
+{
+ FeatureWord w;
+
+ for (w = 0; w < FEATURE_WORDS; w++) {
+ FeatureWordInfo *f = &feature_word_info[w];
+
+ if (f->type == MSR_FEATURE_WORD || f->cpuid.eax != function ||
+ f->cpuid.reg != reg ||
+ (f->cpuid.needs_ecx && f->cpuid.ecx != index)) {
+ continue;
+ }
+
+ return w;
+ }
+
+ return w;
+}
+
+/*
+ * TDX supported CPUID varies from what KVM reports. Adjust the result by
+ * applying the TDX restrictions.
+ */
+void tdx_get_supported_cpuid(uint32_t function, uint32_t index, int reg,
+ uint32_t *ret)
+{
+ /*
+ * it's KVMM + QEMU 's capabilities of what CPUID bits is supported or
+ * can be emulated as supported.
+ */
+ uint32_t vmm_cap = *ret;
+ FeatureWord w;
+
+ /* Only handle features leaves that recognized by feature_word_info[] */
+ w = get_cpuid_featureword_index(function, index, reg);
+ if (w == FEATURE_WORDS) {
+ return;
+ }
+
+ if (tdx_cpuid_lookup[w].inducing_ve) {
+ *ret &= tdx_cpuid_lookup[w].supported_value_on_ve;
+ return;
+ }
+
+ /*
+ * Include all the native bits as first step. It covers types
+ * - As configured (if native)
+ * - Native
+ * - XFAM related and Attributes realted
+ *
+ * It also has side effect to enable unsupported bits, e.g., the
+ * bits of "fixed0" type while present natively. It's safe because
+ * the unsupported bits will be masked off by .fixed0 later.
+ */
+ *ret |= host_cpuid_reg(function, index, reg);
+
+ /* Adjust according to "fixed" type in tdx_cpuid_lookup. */
+ *ret |= tdx_cpuid_lookup[w].tdx_fixed1;
+ *ret &= ~tdx_cpuid_lookup[w].tdx_fixed0;
+
+ /*
+ * Configurable cpuids are supported unconditionally. It's mainly to
+ * include those configurable regardless of native existence.
+ */
+ *ret |= tdx_cap_cpuid_config(function, index, reg);
+
+ /*
+ * clear the configurable bits that require VMM emulation and VMM doesn't
+ * report the support.
+ */
+ *ret &= ~(tdx_cpuid_lookup[w].depends_on_vmm_cap & ~vmm_cap);
+
+ /* special handling */
+ if (function == 1 && reg == R_ECX && !enable_cpu_pm) {
+ *ret &= ~CPUID_EXT_MONITOR;
+ }
+}
+
enum tdx_ioctl_level{
TDX_VM_IOCTL,
TDX_VCPU_IOCTL,
diff --git a/target/i386/kvm/tdx.h b/target/i386/kvm/tdx.h
index d5b4c179fbf7..f62fe8ece982 100644
--- a/target/i386/kvm/tdx.h
+++ b/target/i386/kvm/tdx.h
@@ -26,4 +26,7 @@ bool is_tdx_vm(void);
#define is_tdx_vm() 0
#endif /* CONFIG_TDX */
+void tdx_get_supported_cpuid(uint32_t function, uint32_t index, int reg,
+ uint32_t *ret);
+
#endif /* QEMU_I386_TDX_H */
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* Re: [PATCH v5 17/65] i386/tdx: Adjust the supported CPUID based on TDX restrictions
2024-02-29 6:36 ` [PATCH v5 17/65] i386/tdx: Adjust the supported CPUID based on TDX restrictions Xiaoyao Li
@ 2024-05-31 8:47 ` Duan, Zhenzhong
2024-06-12 9:23 ` Xiaoyao Li
0 siblings, 1 reply; 124+ messages in thread
From: Duan, Zhenzhong @ 2024-05-31 8:47 UTC (permalink / raw)
To: Xiaoyao Li, Paolo Bonzini, David Hildenbrand, Igor Mammedov,
Eduardo Habkost, Marcel Apfelbaum, Philippe Mathieu-Daudé,
Yanan Wang, Michael S. Tsirkin, Richard Henderson, Ani Sinha,
Peter Xu, Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang
On 2/29/2024 2:36 PM, Xiaoyao Li wrote:
> According to Chapter "CPUID Virtualization" in TDX module spec, CPUID
> bits of TD can be classified into 6 types:
>
> ------------------------------------------------------------------------
> 1 | As configured | configurable by VMM, independent of native value;
> ------------------------------------------------------------------------
> 2 | As configured | configurable by VMM if the bit is supported natively
> (if native) | Otherwise it equals as native(0).
> ------------------------------------------------------------------------
> 3 | Fixed | fixed to 0/1
> ------------------------------------------------------------------------
> 4 | Native | reflect the native value
> ------------------------------------------------------------------------
> 5 | Calculated | calculated by TDX module.
> ------------------------------------------------------------------------
> 6 | Inducing #VE | get #VE exception
> ------------------------------------------------------------------------
>
> Note:
> 1. All the configurable XFAM related features and TD attributes related
> features fall into type #2. And fixed0/1 bits of XFAM and TD
> attributes fall into type #3.
>
> 2. For CPUID leaves not listed in "CPUID virtualization Overview" table
> in TDX module spec, TDX module injects #VE to TDs when those are
> queried. For this case, TDs can request CPUID emulation from VMM via
> TDVMCALL and the values are fully controlled by VMM.
>
> Due to TDX module has its own virtualization policy on CPUID bits, it leads
> to what reported via KVM_GET_SUPPORTED_CPUID diverges from the supported
> CPUID bits for TDs. In order to keep a consistent CPUID configuration
> between VMM and TDs. Adjust supported CPUID for TDs based on TDX
> restrictions.
>
> Currently only focus on the CPUID leaves recognized by QEMU's
> feature_word_info[] that are indexed by a FeatureWord.
>
> Introduce a TDX CPUID lookup table, which maintains 1 entry for each
> FeatureWord. Each entry has below fields:
>
> - tdx_fixed0/1: The bits that are fixed as 0/1;
>
> - depends_on_vmm_cap: The bits that are configurable from the view of
> TDX module. But they requires emulation of VMM
> when configured as enabled. For those, they are
> not supported if VMM doesn't report them as
> supported. So they need be fixed up by checking
> if VMM supports them.
Previously I thought bits configurable for TDX module are emulated by
TDX module,
it looks not. Just curious why doesn't those bits belong to "Inducing
#VE" type?
Then guest can get KVM reported capabilities with tdvmcall directly.
>
> - inducing_ve: TD gets #VE when querying this CPUID leaf. The result is
> totally configurable by VMM.
>
> - supported_on_ve: It's valid only when @inducing_ve is true. It represents
> the maximum feature set supported that be emulated
> for TDs.
This is never used together with depends_on_vmm_cap, maybe one variable
is enough?
>
> By applying TDX CPUID lookup table and TDX capabilities reported from
> TDX module, the supported CPUID for TDs can be obtained from following
> steps:
>
> - get the base of VMM supported feature set;
>
> - if the leaf is not a FeatureWord just return VMM's value without
> modification;
>
> - if the leaf is an inducing_ve type, applying supported_on_ve mask and
> return;
>
> - include all native bits, it covers type #2, #4, and parts of type #1.
> (it also includes some unsupported bits. The following step will
> correct it.)
>
> - apply fixed0/1 to it (it covers #3, and rectifies the previous step);
>
> - add configurable bits (it covers the other part of type #1);
>
> - fix the ones in vmm_fixup;
>
> (Calculated type is ignored since it's determined at runtime).
>
> Co-developed-by: Chenyi Qiang <chenyi.qiang@intel.com>
> Signed-off-by: Chenyi Qiang <chenyi.qiang@intel.com>
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
> target/i386/cpu.h | 16 +++
> target/i386/kvm/kvm.c | 4 +
> target/i386/kvm/tdx.c | 263 ++++++++++++++++++++++++++++++++++++++++++
> target/i386/kvm/tdx.h | 3 +
> 4 files changed, 286 insertions(+)
>
> diff --git a/target/i386/cpu.h b/target/i386/cpu.h
> index 952174bb6f52..7bd604f802a1 100644
> --- a/target/i386/cpu.h
> +++ b/target/i386/cpu.h
> @@ -787,6 +787,8 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
>
> /* Support RDFSBASE/RDGSBASE/WRFSBASE/WRGSBASE */
> #define CPUID_7_0_EBX_FSGSBASE (1U << 0)
> +/* Support for TSC adjustment MSR 0x3B */
> +#define CPUID_7_0_EBX_TSC_ADJUST (1U << 1)
> /* Support SGX */
> #define CPUID_7_0_EBX_SGX (1U << 2)
> /* 1st Group of Advanced Bit Manipulation Extensions */
> @@ -805,8 +807,12 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
> #define CPUID_7_0_EBX_INVPCID (1U << 10)
> /* Restricted Transactional Memory */
> #define CPUID_7_0_EBX_RTM (1U << 11)
> +/* Cache QoS Monitoring */
> +#define CPUID_7_0_EBX_PQM (1U << 12)
> /* Memory Protection Extension */
> #define CPUID_7_0_EBX_MPX (1U << 14)
> +/* Resource Director Technology Allocation */
> +#define CPUID_7_0_EBX_RDT_A (1U << 15)
> /* AVX-512 Foundation */
> #define CPUID_7_0_EBX_AVX512F (1U << 16)
> /* AVX-512 Doubleword & Quadword Instruction */
> @@ -862,10 +868,16 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
> #define CPUID_7_0_ECX_AVX512VNNI (1U << 11)
> /* Support for VPOPCNT[B,W] and VPSHUFBITQMB */
> #define CPUID_7_0_ECX_AVX512BITALG (1U << 12)
> +/* Intel Total Memory Encryption */
> +#define CPUID_7_0_ECX_TME (1U << 13)
> /* POPCNT for vectors of DW/QW */
> #define CPUID_7_0_ECX_AVX512_VPOPCNTDQ (1U << 14)
> +/* Placeholder for bit 15 */
> +#define CPUID_7_0_ECX_FZM (1U << 15)
> /* 5-level Page Tables */
> #define CPUID_7_0_ECX_LA57 (1U << 16)
> +/* MAWAU for MPX */
> +#define CPUID_7_0_ECX_MAWAU (31U << 17)
> /* Read Processor ID */
> #define CPUID_7_0_ECX_RDPID (1U << 22)
> /* Bus Lock Debug Exception */
> @@ -876,6 +888,8 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
> #define CPUID_7_0_ECX_MOVDIRI (1U << 27)
> /* Move 64 Bytes as Direct Store Instruction */
> #define CPUID_7_0_ECX_MOVDIR64B (1U << 28)
> +/* ENQCMD and ENQCMDS instructions */
> +#define CPUID_7_0_ECX_ENQCMD (1U << 29)
> /* Support SGX Launch Control */
> #define CPUID_7_0_ECX_SGX_LC (1U << 30)
> /* Protection Keys for Supervisor-mode Pages */
> @@ -893,6 +907,8 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
> #define CPUID_7_0_EDX_SERIALIZE (1U << 14)
> /* TSX Suspend Load Address Tracking instruction */
> #define CPUID_7_0_EDX_TSX_LDTRK (1U << 16)
> +/* PCONFIG instruction */
> +#define CPUID_7_0_EDX_PCONFIG (1U << 18)
> /* Architectural LBRs */
> #define CPUID_7_0_EDX_ARCH_LBR (1U << 19)
> /* AMX_BF16 instruction */
> diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
> index 0e68e80f4291..389b631c03dd 100644
> --- a/target/i386/kvm/kvm.c
> +++ b/target/i386/kvm/kvm.c
> @@ -520,6 +520,10 @@ uint32_t kvm_arch_get_supported_cpuid(KVMState *s, uint32_t function,
> ret |= 1U << KVM_HINTS_REALTIME;
> }
>
> + if (is_tdx_vm()) {
> + tdx_get_supported_cpuid(function, index, reg, &ret);
> + }
> +
> return ret;
> }
>
> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
> index 756058f2ed4a..85d96140b450 100644
> --- a/target/i386/kvm/tdx.c
> +++ b/target/i386/kvm/tdx.c
> @@ -15,11 +15,129 @@
> #include "qemu/error-report.h"
> #include "qapi/error.h"
> #include "qom/object_interfaces.h"
> +#include "standard-headers/asm-x86/kvm_para.h"
> #include "sysemu/kvm.h"
> +#include "sysemu/sysemu.h"
>
> #include "hw/i386/x86.h"
> #include "kvm_i386.h"
> #include "tdx.h"
> +#include "../cpu-internal.h"
> +
> +#define TDX_SUPPORTED_KVM_FEATURES ((1U << KVM_FEATURE_NOP_IO_DELAY) | \
> + (1U << KVM_FEATURE_PV_UNHALT) | \
> + (1U << KVM_FEATURE_PV_TLB_FLUSH) | \
> + (1U << KVM_FEATURE_PV_SEND_IPI) | \
> + (1U << KVM_FEATURE_POLL_CONTROL) | \
> + (1U << KVM_FEATURE_PV_SCHED_YIELD) | \
> + (1U << KVM_FEATURE_MSI_EXT_DEST_ID))
> +
> +typedef struct KvmTdxCpuidLookup {
> + uint32_t tdx_fixed0;
> + uint32_t tdx_fixed1;
> +
> + /*
> + * The CPUID bits that are configurable from the view of TDX module
> + * but require VMM's support when wanting to enable them.
> + *
> + * For those bits, they cannot be enabled if VMM (KVM/QEMU) doesn't support
> + * them.
> + */
> + uint32_t depends_on_vmm_cap;
> +
> + bool inducing_ve;
> + /*
> + * The maximum supported feature set for given inducing-#VE leaf.
> + * It's valid only when .inducing_ve is true.
> + */
> + uint32_t supported_value_on_ve;
> +} KvmTdxCpuidLookup;
> +
> + /*
> + * QEMU maintained TDX CPUID lookup tables, which reflects how CPUIDs are
> + * virtualized for guest TDs based on "CPUID virtualization" of TDX spec.
> + *
> + * Note:
> + *
> + * This table will be updated runtime by tdx_caps reported by KVM.
> + *
> + */
> +static KvmTdxCpuidLookup tdx_cpuid_lookup[FEATURE_WORDS] = {
> + [FEAT_1_EDX] = {
> + .tdx_fixed0 =
> + BIT(10) /* Reserved */ | BIT(20) /* Reserved */ | CPUID_IA64,
> + .tdx_fixed1 =
> + CPUID_MSR | CPUID_PAE | CPUID_MCE | CPUID_APIC |
> + CPUID_MTRR | CPUID_MCA | CPUID_CLFLUSH | CPUID_DTS,
> + .depends_on_vmm_cap =
> + CPUID_ACPI | CPUID_PBE,
> + },
> + [FEAT_1_ECX] = {
> + .tdx_fixed0 =
> + CPUID_EXT_VMX | CPUID_EXT_SMX | BIT(16) /* Reserved */,
> + .tdx_fixed1 =
> + CPUID_EXT_CX16 | CPUID_EXT_PDCM | CPUID_EXT_X2APIC |
> + CPUID_EXT_AES | CPUID_EXT_XSAVE | CPUID_EXT_RDRAND |
> + CPUID_EXT_HYPERVISOR,
> + .depends_on_vmm_cap =
> + CPUID_EXT_EST | CPUID_EXT_TM2 | CPUID_EXT_XTPR | CPUID_EXT_DCA,
> + },
> + [FEAT_8000_0001_EDX] = {
> + .tdx_fixed1 =
> + CPUID_EXT2_NX | CPUID_EXT2_PDPE1GB | CPUID_EXT2_RDTSCP |
> + CPUID_EXT2_LM,
> + },
> + [FEAT_7_0_EBX] = {
> + .tdx_fixed0 =
> + CPUID_7_0_EBX_TSC_ADJUST | CPUID_7_0_EBX_SGX | CPUID_7_0_EBX_MPX,
> + .tdx_fixed1 =
> + CPUID_7_0_EBX_FSGSBASE | CPUID_7_0_EBX_RTM |
> + CPUID_7_0_EBX_RDSEED | CPUID_7_0_EBX_SMAP |
> + CPUID_7_0_EBX_CLFLUSHOPT | CPUID_7_0_EBX_CLWB |
> + CPUID_7_0_EBX_SHA_NI,
> + .depends_on_vmm_cap =
> + CPUID_7_0_EBX_PQM | CPUID_7_0_EBX_RDT_A,
> + },
> + [FEAT_7_0_ECX] = {
> + .tdx_fixed0 =
> + CPUID_7_0_ECX_FZM | CPUID_7_0_ECX_MAWAU |
> + CPUID_7_0_ECX_ENQCMD | CPUID_7_0_ECX_SGX_LC,
> + .tdx_fixed1 =
> + CPUID_7_0_ECX_MOVDIR64B | CPUID_7_0_ECX_BUS_LOCK_DETECT,
> + .depends_on_vmm_cap =
> + CPUID_7_0_ECX_TME,
> + },
> + [FEAT_7_0_EDX] = {
> + .tdx_fixed1 =
> + CPUID_7_0_EDX_SPEC_CTRL | CPUID_7_0_EDX_ARCH_CAPABILITIES |
> + CPUID_7_0_EDX_CORE_CAPABILITY | CPUID_7_0_EDX_SPEC_CTRL_SSBD,
> + .depends_on_vmm_cap =
> + CPUID_7_0_EDX_PCONFIG,
> + },
> + [FEAT_8000_0008_EBX] = {
> + .tdx_fixed0 =
> + ~CPUID_8000_0008_EBX_WBNOINVD,
> + .tdx_fixed1 =
> + CPUID_8000_0008_EBX_WBNOINVD,
> + },
> + [FEAT_XSAVE] = {
> + .tdx_fixed1 =
> + CPUID_XSAVE_XSAVEOPT | CPUID_XSAVE_XSAVEC |
> + CPUID_XSAVE_XSAVES,
> + },
> + [FEAT_6_EAX] = {
> + .inducing_ve = true,
> + .supported_value_on_ve = CPUID_6_EAX_ARAT,
> + },
> + [FEAT_8000_0007_EDX] = {
> + .inducing_ve = true,
> + .supported_value_on_ve = -1U,
> + },
> + [FEAT_KVM] = {
> + .inducing_ve = true,
> + .supported_value_on_ve = TDX_SUPPORTED_KVM_FEATURES,
> + },
> +};
>
> static TdxGuest *tdx_guest;
>
> @@ -31,6 +149,151 @@ bool is_tdx_vm(void)
> return !!tdx_guest;
> }
>
> +static inline uint32_t host_cpuid_reg(uint32_t function,
> + uint32_t index, int reg)
> +{
> + uint32_t eax, ebx, ecx, edx;
> + uint32_t ret = 0;
> +
> + host_cpuid(function, index, &eax, &ebx, &ecx, &edx);
> +
> + switch (reg) {
> + case R_EAX:
> + ret = eax;
> + break;
> + case R_EBX:
> + ret = ebx;
> + break;
> + case R_ECX:
> + ret = ecx;
> + break;
> + case R_EDX:
> + ret = edx;
> + break;
> + }
> + return ret;
> +}
> +
> +/*
> + * get the configurable cpuid bits (can be set to 0 or 1) reported by TDX module
> + * from tdx_caps.
> + */
> +static inline uint32_t tdx_cap_cpuid_config(uint32_t function,
> + uint32_t index, int reg)
> +{
> + struct kvm_tdx_cpuid_config *cpuid_c;
> + int ret = 0;
> + int i;
> +
> + if (tdx_caps->nr_cpuid_configs <= 0) {
> + return ret;
> + }
> +
> + for (i = 0; i < tdx_caps->nr_cpuid_configs; i++) {
> + cpuid_c = &tdx_caps->cpuid_configs[i];
> + /* 0xffffffff in sub_leaf means the leaf doesn't require a sublesf */
> + if (cpuid_c->leaf == function &&
> + (cpuid_c->sub_leaf == 0xffffffff || cpuid_c->sub_leaf == index)) {
> + switch (reg) {
> + case R_EAX:
> + ret = cpuid_c->eax;
> + break;
> + case R_EBX:
> + ret = cpuid_c->ebx;
> + break;
> + case R_ECX:
> + ret = cpuid_c->ecx;
> + break;
> + case R_EDX:
> + ret = cpuid_c->edx;
> + break;
> + default:
> + return 0;
> + }
> + }
> + }
> + return ret;
> +}
> +
> +static FeatureWord get_cpuid_featureword_index(uint32_t function,
> + uint32_t index, int reg)
> +{
> + FeatureWord w;
> +
> + for (w = 0; w < FEATURE_WORDS; w++) {
> + FeatureWordInfo *f = &feature_word_info[w];
> +
> + if (f->type == MSR_FEATURE_WORD || f->cpuid.eax != function ||
> + f->cpuid.reg != reg ||
> + (f->cpuid.needs_ecx && f->cpuid.ecx != index)) {
> + continue;
> + }
> +
> + return w;
> + }
> +
> + return w;
> +}
> +
> +/*
> + * TDX supported CPUID varies from what KVM reports. Adjust the result by
> + * applying the TDX restrictions.
> + */
> +void tdx_get_supported_cpuid(uint32_t function, uint32_t index, int reg,
> + uint32_t *ret)
> +{
> + /*
> + * it's KVMM + QEMU 's capabilities of what CPUID bits is supported or
> + * can be emulated as supported.
> + */
> + uint32_t vmm_cap = *ret;
> + FeatureWord w;
> +
> + /* Only handle features leaves that recognized by feature_word_info[] */
> + w = get_cpuid_featureword_index(function, index, reg);
> + if (w == FEATURE_WORDS) {
> + return;
> + }
> +
> + if (tdx_cpuid_lookup[w].inducing_ve) {
> + *ret &= tdx_cpuid_lookup[w].supported_value_on_ve;
> + return;
> + }
> +
> + /*
> + * Include all the native bits as first step. It covers types
> + * - As configured (if native)
> + * - Native
> + * - XFAM related and Attributes realted
s/realted/related
> + *
> + * It also has side effect to enable unsupported bits, e.g., the
> + * bits of "fixed0" type while present natively. It's safe because
> + * the unsupported bits will be masked off by .fixed0 later.
> + */
> + *ret |= host_cpuid_reg(function, index, reg);
Looks KVM capabilities are merged with native bits, is this intentional?
Thanks
Zhenzhong
> +
> + /* Adjust according to "fixed" type in tdx_cpuid_lookup. */
> + *ret |= tdx_cpuid_lookup[w].tdx_fixed1;
> + *ret &= ~tdx_cpuid_lookup[w].tdx_fixed0;
> +
> + /*
> + * Configurable cpuids are supported unconditionally. It's mainly to
> + * include those configurable regardless of native existence.
> + */
> + *ret |= tdx_cap_cpuid_config(function, index, reg);
> +
> + /*
> + * clear the configurable bits that require VMM emulation and VMM doesn't
> + * report the support.
> + */
> + *ret &= ~(tdx_cpuid_lookup[w].depends_on_vmm_cap & ~vmm_cap);
> +
> + /* special handling */
> + if (function == 1 && reg == R_ECX && !enable_cpu_pm) {
> + *ret &= ~CPUID_EXT_MONITOR;
> + }
> +}
> +
> enum tdx_ioctl_level{
> TDX_VM_IOCTL,
> TDX_VCPU_IOCTL,
> diff --git a/target/i386/kvm/tdx.h b/target/i386/kvm/tdx.h
> index d5b4c179fbf7..f62fe8ece982 100644
> --- a/target/i386/kvm/tdx.h
> +++ b/target/i386/kvm/tdx.h
> @@ -26,4 +26,7 @@ bool is_tdx_vm(void);
> #define is_tdx_vm() 0
> #endif /* CONFIG_TDX */
>
> +void tdx_get_supported_cpuid(uint32_t function, uint32_t index, int reg,
> + uint32_t *ret);
> +
> #endif /* QEMU_I386_TDX_H */
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 17/65] i386/tdx: Adjust the supported CPUID based on TDX restrictions
2024-05-31 8:47 ` Duan, Zhenzhong
@ 2024-06-12 9:23 ` Xiaoyao Li
2024-06-13 8:26 ` Duan, Zhenzhong
0 siblings, 1 reply; 124+ messages in thread
From: Xiaoyao Li @ 2024-06-12 9:23 UTC (permalink / raw)
To: Duan, Zhenzhong, Paolo Bonzini, David Hildenbrand, Igor Mammedov,
Eduardo Habkost, Marcel Apfelbaum, Philippe Mathieu-Daudé,
Yanan Wang, Michael S. Tsirkin, Richard Henderson, Ani Sinha,
Peter Xu, Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang
On 5/31/2024 4:47 PM, Duan, Zhenzhong wrote:
>
> On 2/29/2024 2:36 PM, Xiaoyao Li wrote:
>> According to Chapter "CPUID Virtualization" in TDX module spec, CPUID
>> bits of TD can be classified into 6 types:
>>
>> ------------------------------------------------------------------------
>> 1 | As configured | configurable by VMM, independent of native value;
>> ------------------------------------------------------------------------
>> 2 | As configured | configurable by VMM if the bit is supported natively
>> (if native) | Otherwise it equals as native(0).
>> ------------------------------------------------------------------------
>> 3 | Fixed | fixed to 0/1
>> ------------------------------------------------------------------------
>> 4 | Native | reflect the native value
>> ------------------------------------------------------------------------
>> 5 | Calculated | calculated by TDX module.
>> ------------------------------------------------------------------------
>> 6 | Inducing #VE | get #VE exception
>> ------------------------------------------------------------------------
>>
>> Note:
>> 1. All the configurable XFAM related features and TD attributes related
>> features fall into type #2. And fixed0/1 bits of XFAM and TD
>> attributes fall into type #3.
>>
>> 2. For CPUID leaves not listed in "CPUID virtualization Overview" table
>> in TDX module spec, TDX module injects #VE to TDs when those are
>> queried. For this case, TDs can request CPUID emulation from VMM via
>> TDVMCALL and the values are fully controlled by VMM.
>>
>> Due to TDX module has its own virtualization policy on CPUID bits, it
>> leads
>> to what reported via KVM_GET_SUPPORTED_CPUID diverges from the supported
>> CPUID bits for TDs. In order to keep a consistent CPUID configuration
>> between VMM and TDs. Adjust supported CPUID for TDs based on TDX
>> restrictions.
>>
>> Currently only focus on the CPUID leaves recognized by QEMU's
>> feature_word_info[] that are indexed by a FeatureWord.
>>
>> Introduce a TDX CPUID lookup table, which maintains 1 entry for each
>> FeatureWord. Each entry has below fields:
>>
>> - tdx_fixed0/1: The bits that are fixed as 0/1;
>>
>> - depends_on_vmm_cap: The bits that are configurable from the view of
>> TDX module. But they requires emulation of VMM
>> when configured as enabled. For those, they are
>> not supported if VMM doesn't report them as
>> supported. So they need be fixed up by checking
>> if VMM supports them.
>
> Previously I thought bits configurable for TDX module are emulated by
> TDX module,
>
> it looks not. Just curious why doesn't those bits belong to "Inducing
> #VE" type?
Because when TD guest queries this type of CPUID leaf, it doesn't get #VE.
The bits in this category are free to be configured as on/off by VMM and
passed to TDX module via TD_PARAM. Once they get configured, they are
queried directly by TD guest without getting #VE.
The problem is whether VMM allows them to be configured freely. E.g.,
for features TME and PCONFIG, they are configurable. However when VMM
configures them to 1, VMM needs to provide the support of related MSRs
of them. If VMM cannot provide such support, VMM should be allow user to
configured to 1.
That's why we have this kind of type.
BTW, we are going to abondan the solution that let QEMU to maintain the
CPUID configurability table in this series. Next version we will come up
with
> Then guest can get KVM reported capabilities with tdvmcall directly.
>
>>
>> - inducing_ve: TD gets #VE when querying this CPUID leaf. The result is
>> totally configurable by VMM.
>>
>> - supported_on_ve: It's valid only when @inducing_ve is true. It
>> represents
>> the maximum feature set supported that be emulated
>> for TDs.
> This is never used together with depends_on_vmm_cap, maybe one variable
> is enough?
>>
>> By applying TDX CPUID lookup table and TDX capabilities reported from
>> TDX module, the supported CPUID for TDs can be obtained from following
>> steps:
>>
>> - get the base of VMM supported feature set;
>>
>> - if the leaf is not a FeatureWord just return VMM's value without
>> modification;
>>
>> - if the leaf is an inducing_ve type, applying supported_on_ve mask and
>> return;
>>
>> - include all native bits, it covers type #2, #4, and parts of type #1.
>> (it also includes some unsupported bits. The following step will
>> correct it.)
>>
>> - apply fixed0/1 to it (it covers #3, and rectifies the previous step);
>>
>> - add configurable bits (it covers the other part of type #1);
>>
>> - fix the ones in vmm_fixup;
>>
>> (Calculated type is ignored since it's determined at runtime).
>>
>> Co-developed-by: Chenyi Qiang <chenyi.qiang@intel.com>
>> Signed-off-by: Chenyi Qiang <chenyi.qiang@intel.com>
>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>> ---
>> target/i386/cpu.h | 16 +++
>> target/i386/kvm/kvm.c | 4 +
>> target/i386/kvm/tdx.c | 263 ++++++++++++++++++++++++++++++++++++++++++
>> target/i386/kvm/tdx.h | 3 +
>> 4 files changed, 286 insertions(+)
>>
>> diff --git a/target/i386/cpu.h b/target/i386/cpu.h
>> index 952174bb6f52..7bd604f802a1 100644
>> --- a/target/i386/cpu.h
>> +++ b/target/i386/cpu.h
>> @@ -787,6 +787,8 @@ uint64_t
>> x86_cpu_get_supported_feature_word(FeatureWord w,
>> /* Support RDFSBASE/RDGSBASE/WRFSBASE/WRGSBASE */
>> #define CPUID_7_0_EBX_FSGSBASE (1U << 0)
>> +/* Support for TSC adjustment MSR 0x3B */
>> +#define CPUID_7_0_EBX_TSC_ADJUST (1U << 1)
>> /* Support SGX */
>> #define CPUID_7_0_EBX_SGX (1U << 2)
>> /* 1st Group of Advanced Bit Manipulation Extensions */
>> @@ -805,8 +807,12 @@ uint64_t
>> x86_cpu_get_supported_feature_word(FeatureWord w,
>> #define CPUID_7_0_EBX_INVPCID (1U << 10)
>> /* Restricted Transactional Memory */
>> #define CPUID_7_0_EBX_RTM (1U << 11)
>> +/* Cache QoS Monitoring */
>> +#define CPUID_7_0_EBX_PQM (1U << 12)
>> /* Memory Protection Extension */
>> #define CPUID_7_0_EBX_MPX (1U << 14)
>> +/* Resource Director Technology Allocation */
>> +#define CPUID_7_0_EBX_RDT_A (1U << 15)
>> /* AVX-512 Foundation */
>> #define CPUID_7_0_EBX_AVX512F (1U << 16)
>> /* AVX-512 Doubleword & Quadword Instruction */
>> @@ -862,10 +868,16 @@ uint64_t
>> x86_cpu_get_supported_feature_word(FeatureWord w,
>> #define CPUID_7_0_ECX_AVX512VNNI (1U << 11)
>> /* Support for VPOPCNT[B,W] and VPSHUFBITQMB */
>> #define CPUID_7_0_ECX_AVX512BITALG (1U << 12)
>> +/* Intel Total Memory Encryption */
>> +#define CPUID_7_0_ECX_TME (1U << 13)
>> /* POPCNT for vectors of DW/QW */
>> #define CPUID_7_0_ECX_AVX512_VPOPCNTDQ (1U << 14)
>> +/* Placeholder for bit 15 */
>> +#define CPUID_7_0_ECX_FZM (1U << 15)
>> /* 5-level Page Tables */
>> #define CPUID_7_0_ECX_LA57 (1U << 16)
>> +/* MAWAU for MPX */
>> +#define CPUID_7_0_ECX_MAWAU (31U << 17)
>> /* Read Processor ID */
>> #define CPUID_7_0_ECX_RDPID (1U << 22)
>> /* Bus Lock Debug Exception */
>> @@ -876,6 +888,8 @@ uint64_t
>> x86_cpu_get_supported_feature_word(FeatureWord w,
>> #define CPUID_7_0_ECX_MOVDIRI (1U << 27)
>> /* Move 64 Bytes as Direct Store Instruction */
>> #define CPUID_7_0_ECX_MOVDIR64B (1U << 28)
>> +/* ENQCMD and ENQCMDS instructions */
>> +#define CPUID_7_0_ECX_ENQCMD (1U << 29)
>> /* Support SGX Launch Control */
>> #define CPUID_7_0_ECX_SGX_LC (1U << 30)
>> /* Protection Keys for Supervisor-mode Pages */
>> @@ -893,6 +907,8 @@ uint64_t
>> x86_cpu_get_supported_feature_word(FeatureWord w,
>> #define CPUID_7_0_EDX_SERIALIZE (1U << 14)
>> /* TSX Suspend Load Address Tracking instruction */
>> #define CPUID_7_0_EDX_TSX_LDTRK (1U << 16)
>> +/* PCONFIG instruction */
>> +#define CPUID_7_0_EDX_PCONFIG (1U << 18)
>> /* Architectural LBRs */
>> #define CPUID_7_0_EDX_ARCH_LBR (1U << 19)
>> /* AMX_BF16 instruction */
>> diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
>> index 0e68e80f4291..389b631c03dd 100644
>> --- a/target/i386/kvm/kvm.c
>> +++ b/target/i386/kvm/kvm.c
>> @@ -520,6 +520,10 @@ uint32_t kvm_arch_get_supported_cpuid(KVMState
>> *s, uint32_t function,
>> ret |= 1U << KVM_HINTS_REALTIME;
>> }
>> + if (is_tdx_vm()) {
>> + tdx_get_supported_cpuid(function, index, reg, &ret);
>> + }
>> +
>> return ret;
>> }
>> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
>> index 756058f2ed4a..85d96140b450 100644
>> --- a/target/i386/kvm/tdx.c
>> +++ b/target/i386/kvm/tdx.c
>> @@ -15,11 +15,129 @@
>> #include "qemu/error-report.h"
>> #include "qapi/error.h"
>> #include "qom/object_interfaces.h"
>> +#include "standard-headers/asm-x86/kvm_para.h"
>> #include "sysemu/kvm.h"
>> +#include "sysemu/sysemu.h"
>> #include "hw/i386/x86.h"
>> #include "kvm_i386.h"
>> #include "tdx.h"
>> +#include "../cpu-internal.h"
>> +
>> +#define TDX_SUPPORTED_KVM_FEATURES ((1U << KVM_FEATURE_NOP_IO_DELAY)
>> | \
>> + (1U << KVM_FEATURE_PV_UNHALT) | \
>> + (1U << KVM_FEATURE_PV_TLB_FLUSH)
>> | \
>> + (1U << KVM_FEATURE_PV_SEND_IPI) | \
>> + (1U << KVM_FEATURE_POLL_CONTROL)
>> | \
>> + (1U <<
>> KVM_FEATURE_PV_SCHED_YIELD) | \
>> + (1U <<
>> KVM_FEATURE_MSI_EXT_DEST_ID))
>> +
>> +typedef struct KvmTdxCpuidLookup {
>> + uint32_t tdx_fixed0;
>> + uint32_t tdx_fixed1;
>> +
>> + /*
>> + * The CPUID bits that are configurable from the view of TDX module
>> + * but require VMM's support when wanting to enable them.
>> + *
>> + * For those bits, they cannot be enabled if VMM (KVM/QEMU)
>> doesn't support
>> + * them.
>> + */
>> + uint32_t depends_on_vmm_cap;
>> +
>> + bool inducing_ve;
>> + /*
>> + * The maximum supported feature set for given inducing-#VE leaf.
>> + * It's valid only when .inducing_ve is true.
>> + */
>> + uint32_t supported_value_on_ve;
>> +} KvmTdxCpuidLookup;
>> +
>> + /*
>> + * QEMU maintained TDX CPUID lookup tables, which reflects how
>> CPUIDs are
>> + * virtualized for guest TDs based on "CPUID virtualization" of TDX
>> spec.
>> + *
>> + * Note:
>> + *
>> + * This table will be updated runtime by tdx_caps reported by KVM.
>> + *
>> + */
>> +static KvmTdxCpuidLookup tdx_cpuid_lookup[FEATURE_WORDS] = {
>> + [FEAT_1_EDX] = {
>> + .tdx_fixed0 =
>> + BIT(10) /* Reserved */ | BIT(20) /* Reserved */ |
>> CPUID_IA64,
>> + .tdx_fixed1 =
>> + CPUID_MSR | CPUID_PAE | CPUID_MCE | CPUID_APIC |
>> + CPUID_MTRR | CPUID_MCA | CPUID_CLFLUSH | CPUID_DTS,
>> + .depends_on_vmm_cap =
>> + CPUID_ACPI | CPUID_PBE,
>> + },
>> + [FEAT_1_ECX] = {
>> + .tdx_fixed0 =
>> + CPUID_EXT_VMX | CPUID_EXT_SMX | BIT(16) /* Reserved */,
>> + .tdx_fixed1 =
>> + CPUID_EXT_CX16 | CPUID_EXT_PDCM | CPUID_EXT_X2APIC |
>> + CPUID_EXT_AES | CPUID_EXT_XSAVE | CPUID_EXT_RDRAND |
>> + CPUID_EXT_HYPERVISOR,
>> + .depends_on_vmm_cap =
>> + CPUID_EXT_EST | CPUID_EXT_TM2 | CPUID_EXT_XTPR |
>> CPUID_EXT_DCA,
>> + },
>> + [FEAT_8000_0001_EDX] = {
>> + .tdx_fixed1 =
>> + CPUID_EXT2_NX | CPUID_EXT2_PDPE1GB | CPUID_EXT2_RDTSCP |
>> + CPUID_EXT2_LM,
>> + },
>> + [FEAT_7_0_EBX] = {
>> + .tdx_fixed0 =
>> + CPUID_7_0_EBX_TSC_ADJUST | CPUID_7_0_EBX_SGX |
>> CPUID_7_0_EBX_MPX,
>> + .tdx_fixed1 =
>> + CPUID_7_0_EBX_FSGSBASE | CPUID_7_0_EBX_RTM |
>> + CPUID_7_0_EBX_RDSEED | CPUID_7_0_EBX_SMAP |
>> + CPUID_7_0_EBX_CLFLUSHOPT | CPUID_7_0_EBX_CLWB |
>> + CPUID_7_0_EBX_SHA_NI,
>> + .depends_on_vmm_cap =
>> + CPUID_7_0_EBX_PQM | CPUID_7_0_EBX_RDT_A,
>> + },
>> + [FEAT_7_0_ECX] = {
>> + .tdx_fixed0 =
>> + CPUID_7_0_ECX_FZM | CPUID_7_0_ECX_MAWAU |
>> + CPUID_7_0_ECX_ENQCMD | CPUID_7_0_ECX_SGX_LC,
>> + .tdx_fixed1 =
>> + CPUID_7_0_ECX_MOVDIR64B | CPUID_7_0_ECX_BUS_LOCK_DETECT,
>> + .depends_on_vmm_cap =
>> + CPUID_7_0_ECX_TME,
>> + },
>> + [FEAT_7_0_EDX] = {
>> + .tdx_fixed1 =
>> + CPUID_7_0_EDX_SPEC_CTRL | CPUID_7_0_EDX_ARCH_CAPABILITIES |
>> + CPUID_7_0_EDX_CORE_CAPABILITY |
>> CPUID_7_0_EDX_SPEC_CTRL_SSBD,
>> + .depends_on_vmm_cap =
>> + CPUID_7_0_EDX_PCONFIG,
>> + },
>> + [FEAT_8000_0008_EBX] = {
>> + .tdx_fixed0 =
>> + ~CPUID_8000_0008_EBX_WBNOINVD,
>> + .tdx_fixed1 =
>> + CPUID_8000_0008_EBX_WBNOINVD,
>> + },
>> + [FEAT_XSAVE] = {
>> + .tdx_fixed1 =
>> + CPUID_XSAVE_XSAVEOPT | CPUID_XSAVE_XSAVEC |
>> + CPUID_XSAVE_XSAVES,
>> + },
>> + [FEAT_6_EAX] = {
>> + .inducing_ve = true,
>> + .supported_value_on_ve = CPUID_6_EAX_ARAT,
>> + },
>> + [FEAT_8000_0007_EDX] = {
>> + .inducing_ve = true,
>> + .supported_value_on_ve = -1U,
>> + },
>> + [FEAT_KVM] = {
>> + .inducing_ve = true,
>> + .supported_value_on_ve = TDX_SUPPORTED_KVM_FEATURES,
>> + },
>> +};
>> static TdxGuest *tdx_guest;
>> @@ -31,6 +149,151 @@ bool is_tdx_vm(void)
>> return !!tdx_guest;
>> }
>> +static inline uint32_t host_cpuid_reg(uint32_t function,
>> + uint32_t index, int reg)
>> +{
>> + uint32_t eax, ebx, ecx, edx;
>> + uint32_t ret = 0;
>> +
>> + host_cpuid(function, index, &eax, &ebx, &ecx, &edx);
>> +
>> + switch (reg) {
>> + case R_EAX:
>> + ret = eax;
>> + break;
>> + case R_EBX:
>> + ret = ebx;
>> + break;
>> + case R_ECX:
>> + ret = ecx;
>> + break;
>> + case R_EDX:
>> + ret = edx;
>> + break;
>> + }
>> + return ret;
>> +}
>> +
>> +/*
>> + * get the configurable cpuid bits (can be set to 0 or 1) reported by
>> TDX module
>> + * from tdx_caps.
>> + */
>> +static inline uint32_t tdx_cap_cpuid_config(uint32_t function,
>> + uint32_t index, int reg)
>> +{
>> + struct kvm_tdx_cpuid_config *cpuid_c;
>> + int ret = 0;
>> + int i;
>> +
>> + if (tdx_caps->nr_cpuid_configs <= 0) {
>> + return ret;
>> + }
>> +
>> + for (i = 0; i < tdx_caps->nr_cpuid_configs; i++) {
>> + cpuid_c = &tdx_caps->cpuid_configs[i];
>> + /* 0xffffffff in sub_leaf means the leaf doesn't require a
>> sublesf */
>> + if (cpuid_c->leaf == function &&
>> + (cpuid_c->sub_leaf == 0xffffffff || cpuid_c->sub_leaf ==
>> index)) {
>> + switch (reg) {
>> + case R_EAX:
>> + ret = cpuid_c->eax;
>> + break;
>> + case R_EBX:
>> + ret = cpuid_c->ebx;
>> + break;
>> + case R_ECX:
>> + ret = cpuid_c->ecx;
>> + break;
>> + case R_EDX:
>> + ret = cpuid_c->edx;
>> + break;
>> + default:
>> + return 0;
>> + }
>> + }
>> + }
>> + return ret;
>> +}
>> +
>> +static FeatureWord get_cpuid_featureword_index(uint32_t function,
>> + uint32_t index, int reg)
>> +{
>> + FeatureWord w;
>> +
>> + for (w = 0; w < FEATURE_WORDS; w++) {
>> + FeatureWordInfo *f = &feature_word_info[w];
>> +
>> + if (f->type == MSR_FEATURE_WORD || f->cpuid.eax != function ||
>> + f->cpuid.reg != reg ||
>> + (f->cpuid.needs_ecx && f->cpuid.ecx != index)) {
>> + continue;
>> + }
>> +
>> + return w;
>> + }
>> +
>> + return w;
>> +}
>> +
>> +/*
>> + * TDX supported CPUID varies from what KVM reports. Adjust the
>> result by
>> + * applying the TDX restrictions.
>> + */
>> +void tdx_get_supported_cpuid(uint32_t function, uint32_t index, int reg,
>> + uint32_t *ret)
>> +{
>> + /*
>> + * it's KVMM + QEMU 's capabilities of what CPUID bits is
>> supported or
>> + * can be emulated as supported.
>> + */
>> + uint32_t vmm_cap = *ret;
>> + FeatureWord w;
>> +
>> + /* Only handle features leaves that recognized by
>> feature_word_info[] */
>> + w = get_cpuid_featureword_index(function, index, reg);
>> + if (w == FEATURE_WORDS) {
>> + return;
>> + }
>> +
>> + if (tdx_cpuid_lookup[w].inducing_ve) {
>> + *ret &= tdx_cpuid_lookup[w].supported_value_on_ve;
>> + return;
>> + }
>> +
>> + /*
>> + * Include all the native bits as first step. It covers types
>> + * - As configured (if native)
>> + * - Native
>> + * - XFAM related and Attributes realted
> s/realted/related
>> + *
>> + * It also has side effect to enable unsupported bits, e.g., the
>> + * bits of "fixed0" type while present natively. It's safe because
>> + * the unsupported bits will be masked off by .fixed0 later.
>> + */
>> + *ret |= host_cpuid_reg(function, index, reg);
>
> Looks KVM capabilities are merged with native bits, is this intentional?
yes, if we change the order, it would be more clear for you I guess.
host_cpuid_reg() | kvm_capabilities
The base is host's native value, while any bit that absent from native
but KVM can emulate is also added to base.
> Thanks
>
> Zhenzhong
>
>> +
>> + /* Adjust according to "fixed" type in tdx_cpuid_lookup. */
>> + *ret |= tdx_cpuid_lookup[w].tdx_fixed1;
>> + *ret &= ~tdx_cpuid_lookup[w].tdx_fixed0;
>> +
>> + /*
>> + * Configurable cpuids are supported unconditionally. It's mainly to
>> + * include those configurable regardless of native existence.
>> + */
>> + *ret |= tdx_cap_cpuid_config(function, index, reg);
>> +
>> + /*
>> + * clear the configurable bits that require VMM emulation and VMM
>> doesn't
>> + * report the support.
>> + */
>> + *ret &= ~(tdx_cpuid_lookup[w].depends_on_vmm_cap & ~vmm_cap);
>> +
>> + /* special handling */
>> + if (function == 1 && reg == R_ECX && !enable_cpu_pm) {
>> + *ret &= ~CPUID_EXT_MONITOR;
>> + }
>> +}
>> +
>> enum tdx_ioctl_level{
>> TDX_VM_IOCTL,
>> TDX_VCPU_IOCTL,
>> diff --git a/target/i386/kvm/tdx.h b/target/i386/kvm/tdx.h
>> index d5b4c179fbf7..f62fe8ece982 100644
>> --- a/target/i386/kvm/tdx.h
>> +++ b/target/i386/kvm/tdx.h
>> @@ -26,4 +26,7 @@ bool is_tdx_vm(void);
>> #define is_tdx_vm() 0
>> #endif /* CONFIG_TDX */
>> +void tdx_get_supported_cpuid(uint32_t function, uint32_t index, int reg,
>> + uint32_t *ret);
>> +
>> #endif /* QEMU_I386_TDX_H */
^ permalink raw reply [flat|nested] 124+ messages in thread* RE: [PATCH v5 17/65] i386/tdx: Adjust the supported CPUID based on TDX restrictions
2024-06-12 9:23 ` Xiaoyao Li
@ 2024-06-13 8:26 ` Duan, Zhenzhong
2024-06-14 1:02 ` Xiaoyao Li
0 siblings, 1 reply; 124+ messages in thread
From: Duan, Zhenzhong @ 2024-06-13 8:26 UTC (permalink / raw)
To: Li, Xiaoyao, Paolo Bonzini, David Hildenbrand, Igor Mammedov,
Eduardo Habkost, Marcel Apfelbaum, Philippe Mathieu-Daudé,
Yanan Wang, Michael S. Tsirkin, Richard Henderson, Ani Sinha,
Peter Xu, Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm@vger.kernel.org, qemu-devel@nongnu.org, Michael Roth,
Claudio Fontana, Gerd Hoffmann, Isaku Yamahata, Qiang, Chenyi
>-----Original Message-----
>From: Li, Xiaoyao <xiaoyao.li@intel.com>
>Subject: Re: [PATCH v5 17/65] i386/tdx: Adjust the supported CPUID based
>on TDX restrictions
>
>On 5/31/2024 4:47 PM, Duan, Zhenzhong wrote:
>>
>> On 2/29/2024 2:36 PM, Xiaoyao Li wrote:
>>> According to Chapter "CPUID Virtualization" in TDX module spec, CPUID
>>> bits of TD can be classified into 6 types:
>>>
>>> ------------------------------------------------------------------------
>>> 1 | As configured | configurable by VMM, independent of native value;
>>> ------------------------------------------------------------------------
>>> 2 | As configured | configurable by VMM if the bit is supported natively
>>> (if native) | Otherwise it equals as native(0).
>>> ------------------------------------------------------------------------
>>> 3 | Fixed | fixed to 0/1
>>> ------------------------------------------------------------------------
>>> 4 | Native | reflect the native value
>>> ------------------------------------------------------------------------
>>> 5 | Calculated | calculated by TDX module.
>>> ------------------------------------------------------------------------
>>> 6 | Inducing #VE | get #VE exception
>>> ------------------------------------------------------------------------
>>>
>>> Note:
>>> 1. All the configurable XFAM related features and TD attributes related
>>> features fall into type #2. And fixed0/1 bits of XFAM and TD
>>> attributes fall into type #3.
>>>
>>> 2. For CPUID leaves not listed in "CPUID virtualization Overview" table
>>> in TDX module spec, TDX module injects #VE to TDs when those are
>>> queried. For this case, TDs can request CPUID emulation from VMM via
>>> TDVMCALL and the values are fully controlled by VMM.
>>>
>>> Due to TDX module has its own virtualization policy on CPUID bits, it
>>> leads
>>> to what reported via KVM_GET_SUPPORTED_CPUID diverges from the
>supported
>>> CPUID bits for TDs. In order to keep a consistent CPUID configuration
>>> between VMM and TDs. Adjust supported CPUID for TDs based on TDX
>>> restrictions.
>>>
>>> Currently only focus on the CPUID leaves recognized by QEMU's
>>> feature_word_info[] that are indexed by a FeatureWord.
>>>
>>> Introduce a TDX CPUID lookup table, which maintains 1 entry for each
>>> FeatureWord. Each entry has below fields:
>>>
>>> - tdx_fixed0/1: The bits that are fixed as 0/1;
>>>
>>> - depends_on_vmm_cap: The bits that are configurable from the view of
>>> TDX module. But they requires emulation of VMM
>>> when configured as enabled. For those, they are
>>> not supported if VMM doesn't report them as
>>> supported. So they need be fixed up by checking
>>> if VMM supports them.
>>
>> Previously I thought bits configurable for TDX module are emulated by
>> TDX module,
>>
>> it looks not. Just curious why doesn't those bits belong to "Inducing
>> #VE" type?
>
>Because when TD guest queries this type of CPUID leaf, it doesn't get #VE.
>
>The bits in this category are free to be configured as on/off by VMM and
>passed to TDX module via TD_PARAM. Once they get configured, they are
>queried directly by TD guest without getting #VE.
>
>The problem is whether VMM allows them to be configured freely. E.g.,
>for features TME and PCONFIG, they are configurable. However when VMM
>configures them to 1, VMM needs to provide the support of related MSRs
>of them. If VMM cannot provide such support, VMM should be allow user to
>configured to 1.
>
>That's why we have this kind of type.
>
>BTW, we are going to abondan the solution that let QEMU to maintain the
>CPUID configurability table in this series. Next version we will come up
>with
>
>> Then guest can get KVM reported capabilities with tdvmcall directly.
>>
>>>
>>> - inducing_ve: TD gets #VE when querying this CPUID leaf. The result is
>>> totally configurable by VMM.
>>>
>>> - supported_on_ve: It's valid only when @inducing_ve is true. It
>>> represents
>>> the maximum feature set supported that be emulated
>>> for TDs.
>> This is never used together with depends_on_vmm_cap, maybe one
>variable
>> is enough?
>>>
>>> By applying TDX CPUID lookup table and TDX capabilities reported from
>>> TDX module, the supported CPUID for TDs can be obtained from following
>>> steps:
>>>
>>> - get the base of VMM supported feature set;
>>>
>>> - if the leaf is not a FeatureWord just return VMM's value without
>>> modification;
>>>
>>> - if the leaf is an inducing_ve type, applying supported_on_ve mask and
>>> return;
>>>
>>> - include all native bits, it covers type #2, #4, and parts of type #1.
>>> (it also includes some unsupported bits. The following step will
>>> correct it.)
>>>
>>> - apply fixed0/1 to it (it covers #3, and rectifies the previous step);
>>>
>>> - add configurable bits (it covers the other part of type #1);
>>>
>>> - fix the ones in vmm_fixup;
>>>
>>> (Calculated type is ignored since it's determined at runtime).
>>>
>>> Co-developed-by: Chenyi Qiang <chenyi.qiang@intel.com>
>>> Signed-off-by: Chenyi Qiang <chenyi.qiang@intel.com>
>>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>>> ---
>>> target/i386/cpu.h | 16 +++
>>> target/i386/kvm/kvm.c | 4 +
>>> target/i386/kvm/tdx.c | 263
>++++++++++++++++++++++++++++++++++++++++++
>>> target/i386/kvm/tdx.h | 3 +
>>> 4 files changed, 286 insertions(+)
>>>
>>> diff --git a/target/i386/cpu.h b/target/i386/cpu.h
>>> index 952174bb6f52..7bd604f802a1 100644
>>> --- a/target/i386/cpu.h
>>> +++ b/target/i386/cpu.h
>>> @@ -787,6 +787,8 @@ uint64_t
>>> x86_cpu_get_supported_feature_word(FeatureWord w,
>>> /* Support RDFSBASE/RDGSBASE/WRFSBASE/WRGSBASE */
>>> #define CPUID_7_0_EBX_FSGSBASE (1U << 0)
>>> +/* Support for TSC adjustment MSR 0x3B */
>>> +#define CPUID_7_0_EBX_TSC_ADJUST (1U << 1)
>>> /* Support SGX */
>>> #define CPUID_7_0_EBX_SGX (1U << 2)
>>> /* 1st Group of Advanced Bit Manipulation Extensions */
>>> @@ -805,8 +807,12 @@ uint64_t
>>> x86_cpu_get_supported_feature_word(FeatureWord w,
>>> #define CPUID_7_0_EBX_INVPCID (1U << 10)
>>> /* Restricted Transactional Memory */
>>> #define CPUID_7_0_EBX_RTM (1U << 11)
>>> +/* Cache QoS Monitoring */
>>> +#define CPUID_7_0_EBX_PQM (1U << 12)
>>> /* Memory Protection Extension */
>>> #define CPUID_7_0_EBX_MPX (1U << 14)
>>> +/* Resource Director Technology Allocation */
>>> +#define CPUID_7_0_EBX_RDT_A (1U << 15)
>>> /* AVX-512 Foundation */
>>> #define CPUID_7_0_EBX_AVX512F (1U << 16)
>>> /* AVX-512 Doubleword & Quadword Instruction */
>>> @@ -862,10 +868,16 @@ uint64_t
>>> x86_cpu_get_supported_feature_word(FeatureWord w,
>>> #define CPUID_7_0_ECX_AVX512VNNI (1U << 11)
>>> /* Support for VPOPCNT[B,W] and VPSHUFBITQMB */
>>> #define CPUID_7_0_ECX_AVX512BITALG (1U << 12)
>>> +/* Intel Total Memory Encryption */
>>> +#define CPUID_7_0_ECX_TME (1U << 13)
>>> /* POPCNT for vectors of DW/QW */
>>> #define CPUID_7_0_ECX_AVX512_VPOPCNTDQ (1U << 14)
>>> +/* Placeholder for bit 15 */
>>> +#define CPUID_7_0_ECX_FZM (1U << 15)
>>> /* 5-level Page Tables */
>>> #define CPUID_7_0_ECX_LA57 (1U << 16)
>>> +/* MAWAU for MPX */
>>> +#define CPUID_7_0_ECX_MAWAU (31U << 17)
>>> /* Read Processor ID */
>>> #define CPUID_7_0_ECX_RDPID (1U << 22)
>>> /* Bus Lock Debug Exception */
>>> @@ -876,6 +888,8 @@ uint64_t
>>> x86_cpu_get_supported_feature_word(FeatureWord w,
>>> #define CPUID_7_0_ECX_MOVDIRI (1U << 27)
>>> /* Move 64 Bytes as Direct Store Instruction */
>>> #define CPUID_7_0_ECX_MOVDIR64B (1U << 28)
>>> +/* ENQCMD and ENQCMDS instructions */
>>> +#define CPUID_7_0_ECX_ENQCMD (1U << 29)
>>> /* Support SGX Launch Control */
>>> #define CPUID_7_0_ECX_SGX_LC (1U << 30)
>>> /* Protection Keys for Supervisor-mode Pages */
>>> @@ -893,6 +907,8 @@ uint64_t
>>> x86_cpu_get_supported_feature_word(FeatureWord w,
>>> #define CPUID_7_0_EDX_SERIALIZE (1U << 14)
>>> /* TSX Suspend Load Address Tracking instruction */
>>> #define CPUID_7_0_EDX_TSX_LDTRK (1U << 16)
>>> +/* PCONFIG instruction */
>>> +#define CPUID_7_0_EDX_PCONFIG (1U << 18)
>>> /* Architectural LBRs */
>>> #define CPUID_7_0_EDX_ARCH_LBR (1U << 19)
>>> /* AMX_BF16 instruction */
>>> diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
>>> index 0e68e80f4291..389b631c03dd 100644
>>> --- a/target/i386/kvm/kvm.c
>>> +++ b/target/i386/kvm/kvm.c
>>> @@ -520,6 +520,10 @@ uint32_t
>kvm_arch_get_supported_cpuid(KVMState
>>> *s, uint32_t function,
>>> ret |= 1U << KVM_HINTS_REALTIME;
>>> }
>>> + if (is_tdx_vm()) {
>>> + tdx_get_supported_cpuid(function, index, reg, &ret);
>>> + }
>>> +
>>> return ret;
>>> }
>>> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
>>> index 756058f2ed4a..85d96140b450 100644
>>> --- a/target/i386/kvm/tdx.c
>>> +++ b/target/i386/kvm/tdx.c
>>> @@ -15,11 +15,129 @@
>>> #include "qemu/error-report.h"
>>> #include "qapi/error.h"
>>> #include "qom/object_interfaces.h"
>>> +#include "standard-headers/asm-x86/kvm_para.h"
>>> #include "sysemu/kvm.h"
>>> +#include "sysemu/sysemu.h"
>>> #include "hw/i386/x86.h"
>>> #include "kvm_i386.h"
>>> #include "tdx.h"
>>> +#include "../cpu-internal.h"
>>> +
>>> +#define TDX_SUPPORTED_KVM_FEATURES ((1U <<
>KVM_FEATURE_NOP_IO_DELAY)
>>> | \
>>> + (1U << KVM_FEATURE_PV_UNHALT) | \
>>> + (1U << KVM_FEATURE_PV_TLB_FLUSH)
>>> | \
>>> + (1U << KVM_FEATURE_PV_SEND_IPI) | \
>>> + (1U << KVM_FEATURE_POLL_CONTROL)
>>> | \
>>> + (1U <<
>>> KVM_FEATURE_PV_SCHED_YIELD) | \
>>> + (1U <<
>>> KVM_FEATURE_MSI_EXT_DEST_ID))
>>> +
>>> +typedef struct KvmTdxCpuidLookup {
>>> + uint32_t tdx_fixed0;
>>> + uint32_t tdx_fixed1;
>>> +
>>> + /*
>>> + * The CPUID bits that are configurable from the view of TDX module
>>> + * but require VMM's support when wanting to enable them.
>>> + *
>>> + * For those bits, they cannot be enabled if VMM (KVM/QEMU)
>>> doesn't support
>>> + * them.
>>> + */
>>> + uint32_t depends_on_vmm_cap;
>>> +
>>> + bool inducing_ve;
>>> + /*
>>> + * The maximum supported feature set for given inducing-#VE leaf.
>>> + * It's valid only when .inducing_ve is true.
>>> + */
>>> + uint32_t supported_value_on_ve;
>>> +} KvmTdxCpuidLookup;
>>> +
>>> + /*
>>> + * QEMU maintained TDX CPUID lookup tables, which reflects how
>>> CPUIDs are
>>> + * virtualized for guest TDs based on "CPUID virtualization" of TDX
>>> spec.
>>> + *
>>> + * Note:
>>> + *
>>> + * This table will be updated runtime by tdx_caps reported by KVM.
>>> + *
>>> + */
>>> +static KvmTdxCpuidLookup tdx_cpuid_lookup[FEATURE_WORDS] = {
>>> + [FEAT_1_EDX] = {
>>> + .tdx_fixed0 =
>>> + BIT(10) /* Reserved */ | BIT(20) /* Reserved */ |
>>> CPUID_IA64,
>>> + .tdx_fixed1 =
>>> + CPUID_MSR | CPUID_PAE | CPUID_MCE | CPUID_APIC |
>>> + CPUID_MTRR | CPUID_MCA | CPUID_CLFLUSH | CPUID_DTS,
>>> + .depends_on_vmm_cap =
>>> + CPUID_ACPI | CPUID_PBE,
>>> + },
>>> + [FEAT_1_ECX] = {
>>> + .tdx_fixed0 =
>>> + CPUID_EXT_VMX | CPUID_EXT_SMX | BIT(16) /* Reserved */,
>>> + .tdx_fixed1 =
>>> + CPUID_EXT_CX16 | CPUID_EXT_PDCM | CPUID_EXT_X2APIC |
>>> + CPUID_EXT_AES | CPUID_EXT_XSAVE | CPUID_EXT_RDRAND |
>>> + CPUID_EXT_HYPERVISOR,
>>> + .depends_on_vmm_cap =
>>> + CPUID_EXT_EST | CPUID_EXT_TM2 | CPUID_EXT_XTPR |
>>> CPUID_EXT_DCA,
>>> + },
>>> + [FEAT_8000_0001_EDX] = {
>>> + .tdx_fixed1 =
>>> + CPUID_EXT2_NX | CPUID_EXT2_PDPE1GB | CPUID_EXT2_RDTSCP
>|
>>> + CPUID_EXT2_LM,
>>> + },
>>> + [FEAT_7_0_EBX] = {
>>> + .tdx_fixed0 =
>>> + CPUID_7_0_EBX_TSC_ADJUST | CPUID_7_0_EBX_SGX |
>>> CPUID_7_0_EBX_MPX,
>>> + .tdx_fixed1 =
>>> + CPUID_7_0_EBX_FSGSBASE | CPUID_7_0_EBX_RTM |
>>> + CPUID_7_0_EBX_RDSEED | CPUID_7_0_EBX_SMAP |
>>> + CPUID_7_0_EBX_CLFLUSHOPT | CPUID_7_0_EBX_CLWB |
>>> + CPUID_7_0_EBX_SHA_NI,
>>> + .depends_on_vmm_cap =
>>> + CPUID_7_0_EBX_PQM | CPUID_7_0_EBX_RDT_A,
>>> + },
>>> + [FEAT_7_0_ECX] = {
>>> + .tdx_fixed0 =
>>> + CPUID_7_0_ECX_FZM | CPUID_7_0_ECX_MAWAU |
>>> + CPUID_7_0_ECX_ENQCMD | CPUID_7_0_ECX_SGX_LC,
>>> + .tdx_fixed1 =
>>> + CPUID_7_0_ECX_MOVDIR64B |
>CPUID_7_0_ECX_BUS_LOCK_DETECT,
>>> + .depends_on_vmm_cap =
>>> + CPUID_7_0_ECX_TME,
>>> + },
>>> + [FEAT_7_0_EDX] = {
>>> + .tdx_fixed1 =
>>> + CPUID_7_0_EDX_SPEC_CTRL |
>CPUID_7_0_EDX_ARCH_CAPABILITIES |
>>> + CPUID_7_0_EDX_CORE_CAPABILITY |
>>> CPUID_7_0_EDX_SPEC_CTRL_SSBD,
>>> + .depends_on_vmm_cap =
>>> + CPUID_7_0_EDX_PCONFIG,
>>> + },
>>> + [FEAT_8000_0008_EBX] = {
>>> + .tdx_fixed0 =
>>> + ~CPUID_8000_0008_EBX_WBNOINVD,
>>> + .tdx_fixed1 =
>>> + CPUID_8000_0008_EBX_WBNOINVD,
>>> + },
>>> + [FEAT_XSAVE] = {
>>> + .tdx_fixed1 =
>>> + CPUID_XSAVE_XSAVEOPT | CPUID_XSAVE_XSAVEC |
>>> + CPUID_XSAVE_XSAVES,
>>> + },
>>> + [FEAT_6_EAX] = {
>>> + .inducing_ve = true,
>>> + .supported_value_on_ve = CPUID_6_EAX_ARAT,
>>> + },
>>> + [FEAT_8000_0007_EDX] = {
>>> + .inducing_ve = true,
>>> + .supported_value_on_ve = -1U,
>>> + },
>>> + [FEAT_KVM] = {
>>> + .inducing_ve = true,
>>> + .supported_value_on_ve = TDX_SUPPORTED_KVM_FEATURES,
>>> + },
>>> +};
>>> static TdxGuest *tdx_guest;
>>> @@ -31,6 +149,151 @@ bool is_tdx_vm(void)
>>> return !!tdx_guest;
>>> }
>>> +static inline uint32_t host_cpuid_reg(uint32_t function,
>>> + uint32_t index, int reg)
>>> +{
>>> + uint32_t eax, ebx, ecx, edx;
>>> + uint32_t ret = 0;
>>> +
>>> + host_cpuid(function, index, &eax, &ebx, &ecx, &edx);
>>> +
>>> + switch (reg) {
>>> + case R_EAX:
>>> + ret = eax;
>>> + break;
>>> + case R_EBX:
>>> + ret = ebx;
>>> + break;
>>> + case R_ECX:
>>> + ret = ecx;
>>> + break;
>>> + case R_EDX:
>>> + ret = edx;
>>> + break;
>>> + }
>>> + return ret;
>>> +}
>>> +
>>> +/*
>>> + * get the configurable cpuid bits (can be set to 0 or 1) reported by
>>> TDX module
>>> + * from tdx_caps.
>>> + */
>>> +static inline uint32_t tdx_cap_cpuid_config(uint32_t function,
>>> + uint32_t index, int reg)
>>> +{
>>> + struct kvm_tdx_cpuid_config *cpuid_c;
>>> + int ret = 0;
>>> + int i;
>>> +
>>> + if (tdx_caps->nr_cpuid_configs <= 0) {
>>> + return ret;
>>> + }
>>> +
>>> + for (i = 0; i < tdx_caps->nr_cpuid_configs; i++) {
>>> + cpuid_c = &tdx_caps->cpuid_configs[i];
>>> + /* 0xffffffff in sub_leaf means the leaf doesn't require a
>>> sublesf */
>>> + if (cpuid_c->leaf == function &&
>>> + (cpuid_c->sub_leaf == 0xffffffff || cpuid_c->sub_leaf ==
>>> index)) {
>>> + switch (reg) {
>>> + case R_EAX:
>>> + ret = cpuid_c->eax;
>>> + break;
>>> + case R_EBX:
>>> + ret = cpuid_c->ebx;
>>> + break;
>>> + case R_ECX:
>>> + ret = cpuid_c->ecx;
>>> + break;
>>> + case R_EDX:
>>> + ret = cpuid_c->edx;
>>> + break;
>>> + default:
>>> + return 0;
>>> + }
>>> + }
>>> + }
>>> + return ret;
>>> +}
>>> +
>>> +static FeatureWord get_cpuid_featureword_index(uint32_t function,
>>> + uint32_t index, int reg)
>>> +{
>>> + FeatureWord w;
>>> +
>>> + for (w = 0; w < FEATURE_WORDS; w++) {
>>> + FeatureWordInfo *f = &feature_word_info[w];
>>> +
>>> + if (f->type == MSR_FEATURE_WORD || f->cpuid.eax != function ||
>>> + f->cpuid.reg != reg ||
>>> + (f->cpuid.needs_ecx && f->cpuid.ecx != index)) {
>>> + continue;
>>> + }
>>> +
>>> + return w;
>>> + }
>>> +
>>> + return w;
>>> +}
>>> +
>>> +/*
>>> + * TDX supported CPUID varies from what KVM reports. Adjust the
>>> result by
>>> + * applying the TDX restrictions.
>>> + */
>>> +void tdx_get_supported_cpuid(uint32_t function, uint32_t index, int
>reg,
>>> + uint32_t *ret)
>>> +{
>>> + /*
>>> + * it's KVMM + QEMU 's capabilities of what CPUID bits is
>>> supported or
>>> + * can be emulated as supported.
>>> + */
>>> + uint32_t vmm_cap = *ret;
>>> + FeatureWord w;
>>> +
>>> + /* Only handle features leaves that recognized by
>>> feature_word_info[] */
>>> + w = get_cpuid_featureword_index(function, index, reg);
>>> + if (w == FEATURE_WORDS) {
>>> + return;
>>> + }
>>> +
>>> + if (tdx_cpuid_lookup[w].inducing_ve) {
>>> + *ret &= tdx_cpuid_lookup[w].supported_value_on_ve;
>>> + return;
>>> + }
>>> +
>>> + /*
>>> + * Include all the native bits as first step. It covers types
>>> + * - As configured (if native)
>>> + * - Native
>>> + * - XFAM related and Attributes realted
>> s/realted/related
>>> + *
>>> + * It also has side effect to enable unsupported bits, e.g., the
>>> + * bits of "fixed0" type while present natively. It's safe because
>>> + * the unsupported bits will be masked off by .fixed0 later.
>>> + */
>>> + *ret |= host_cpuid_reg(function, index, reg);
>>
>> Looks KVM capabilities are merged with native bits, is this intentional?
>
>yes, if we change the order, it would be more clear for you I guess.
>
> host_cpuid_reg() | kvm_capabilities
>
>The base is host's native value, while any bit that absent from native
>but KVM can emulate is also added to base.
Imagine there is a 'type native' bit that's absent from native but KVM emulated,
With above code we pass 1 to tdx module but it wants native 0, is it an issue?
Thanks
Zhenzhong
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 17/65] i386/tdx: Adjust the supported CPUID based on TDX restrictions
2024-06-13 8:26 ` Duan, Zhenzhong
@ 2024-06-14 1:02 ` Xiaoyao Li
0 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-06-14 1:02 UTC (permalink / raw)
To: Duan, Zhenzhong, Paolo Bonzini, David Hildenbrand, Igor Mammedov,
Eduardo Habkost, Marcel Apfelbaum, Philippe Mathieu-Daudé,
Yanan Wang, Michael S. Tsirkin, Richard Henderson, Ani Sinha,
Peter Xu, Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm@vger.kernel.org, qemu-devel@nongnu.org, Michael Roth,
Claudio Fontana, Gerd Hoffmann, Isaku Yamahata, Qiang, Chenyi
On 6/13/2024 4:26 PM, Duan, Zhenzhong wrote:
>>>> + *
>>>> + * It also has side effect to enable unsupported bits, e.g., the
>>>> + * bits of "fixed0" type while present natively. It's safe because
>>>> + * the unsupported bits will be masked off by .fixed0 later.
>>>> + */
>>>> + *ret |= host_cpuid_reg(function, index, reg);
>>> Looks KVM capabilities are merged with native bits, is this intentional?
>> yes, if we change the order, it would be more clear for you I guess.
>>
>> host_cpuid_reg() | kvm_capabilities
>>
>> The base is host's native value, while any bit that absent from native
>> but KVM can emulate is also added to base.
> Imagine there is a 'type native' bit that's absent from native but KVM emulated,
> With above code we pass 1 to tdx module but it wants native 0, is it an issue?
yes, it will have issue but it's not "we pass 1 to tdx_module".
"Native" bit is not configurable in the view of TDX module, and QEMU/KVM
cannot configure it. But it does causes mismatch in above case that QEMU
sees the bit is supported while in the TD the bit is not supported.
This is one of the reason why we are going to drop the solution that
QEMU maintains the CPUID configurability in this series.
^ permalink raw reply [flat|nested] 124+ messages in thread
* [PATCH v5 18/65] i386/tdx: Make Intel-PT unsupported for TD guest
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (16 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 17/65] i386/tdx: Adjust the supported CPUID based on TDX restrictions Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-05-31 9:27 ` Duan, Zhenzhong
2024-02-29 6:36 ` [PATCH v5 19/65] i386/tdx: Update tdx_cpuid_lookup[].tdx_fixed0/1 by tdx_caps.cpuid_config[] Xiaoyao Li
` (46 subsequent siblings)
64 siblings, 1 reply; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
Due to the fact that Intel-PT virtualization support has been broken in
QEMU since Sapphire Rapids generation[1], below warning is triggered when
luanching TD guest:
warning: host doesn't support requested feature: CPUID.07H:EBX.intel-pt [bit 25]
Before Intel-pt is fixed in QEMU, just make Intel-PT unsupported for TD
guest, to avoid the confusing warning.
[1] https://lore.kernel.org/qemu-devel/20230531084311.3807277-1-xiaoyao.li@intel.com/
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
Changes in v4:
- newly added patch;
---
target/i386/kvm/tdx.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 85d96140b450..239170142e4f 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -292,6 +292,11 @@ void tdx_get_supported_cpuid(uint32_t function, uint32_t index, int reg,
if (function == 1 && reg == R_ECX && !enable_cpu_pm) {
*ret &= ~CPUID_EXT_MONITOR;
}
+
+ /* QEMU Intel-pt support is broken, don't advertise Intel-PT */
+ if (function == 7 && reg == R_EBX) {
+ *ret &= ~CPUID_7_0_EBX_INTEL_PT;
+ }
}
enum tdx_ioctl_level{
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* Re: [PATCH v5 18/65] i386/tdx: Make Intel-PT unsupported for TD guest
2024-02-29 6:36 ` [PATCH v5 18/65] i386/tdx: Make Intel-PT unsupported for TD guest Xiaoyao Li
@ 2024-05-31 9:27 ` Duan, Zhenzhong
2024-06-12 9:24 ` Xiaoyao Li
0 siblings, 1 reply; 124+ messages in thread
From: Duan, Zhenzhong @ 2024-05-31 9:27 UTC (permalink / raw)
To: Xiaoyao Li, Paolo Bonzini, David Hildenbrand, Igor Mammedov,
Eduardo Habkost, Marcel Apfelbaum, Philippe Mathieu-Daudé,
Yanan Wang, Michael S. Tsirkin, Richard Henderson, Ani Sinha,
Peter Xu, Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang
On 2/29/2024 2:36 PM, Xiaoyao Li wrote:
> Due to the fact that Intel-PT virtualization support has been broken in
> QEMU since Sapphire Rapids generation[1], below warning is triggered when
> luanching TD guest:
>
> warning: host doesn't support requested feature: CPUID.07H:EBX.intel-pt [bit 25]
>
> Before Intel-pt is fixed in QEMU, just make Intel-PT unsupported for TD
> guest, to avoid the confusing warning.
I guess normal guest has same issue.
Thanks
Zhenzhong
>
> [1] https://lore.kernel.org/qemu-devel/20230531084311.3807277-1-xiaoyao.li@intel.com/
>
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
> Changes in v4:
> - newly added patch;
> ---
> target/i386/kvm/tdx.c | 5 +++++
> 1 file changed, 5 insertions(+)
>
> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
> index 85d96140b450..239170142e4f 100644
> --- a/target/i386/kvm/tdx.c
> +++ b/target/i386/kvm/tdx.c
> @@ -292,6 +292,11 @@ void tdx_get_supported_cpuid(uint32_t function, uint32_t index, int reg,
> if (function == 1 && reg == R_ECX && !enable_cpu_pm) {
> *ret &= ~CPUID_EXT_MONITOR;
> }
> +
> + /* QEMU Intel-pt support is broken, don't advertise Intel-PT */
> + if (function == 7 && reg == R_EBX) {
> + *ret &= ~CPUID_7_0_EBX_INTEL_PT;
> + }
> }
>
> enum tdx_ioctl_level{
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 18/65] i386/tdx: Make Intel-PT unsupported for TD guest
2024-05-31 9:27 ` Duan, Zhenzhong
@ 2024-06-12 9:24 ` Xiaoyao Li
0 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-06-12 9:24 UTC (permalink / raw)
To: Duan, Zhenzhong, Paolo Bonzini, David Hildenbrand, Igor Mammedov,
Eduardo Habkost, Marcel Apfelbaum, Philippe Mathieu-Daudé,
Yanan Wang, Michael S. Tsirkin, Richard Henderson, Ani Sinha,
Peter Xu, Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang
On 5/31/2024 5:27 PM, Duan, Zhenzhong wrote:
>
> On 2/29/2024 2:36 PM, Xiaoyao Li wrote:
>> Due to the fact that Intel-PT virtualization support has been broken in
>> QEMU since Sapphire Rapids generation[1], below warning is triggered when
>> luanching TD guest:
>>
>> warning: host doesn't support requested feature:
>> CPUID.07H:EBX.intel-pt [bit 25]
>>
>> Before Intel-pt is fixed in QEMU, just make Intel-PT unsupported for TD
>> guest, to avoid the confusing warning.
>
> I guess normal guest has same issue.
yeah, just the bug referenced by [1]
> Thanks
>
> Zhenzhong
>
>>
>> [1]
>> https://lore.kernel.org/qemu-devel/20230531084311.3807277-1-xiaoyao.li@intel.com/
>>
>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>> ---
>> Changes in v4:
>> - newly added patch;
>> ---
>> target/i386/kvm/tdx.c | 5 +++++
>> 1 file changed, 5 insertions(+)
>>
>> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
>> index 85d96140b450..239170142e4f 100644
>> --- a/target/i386/kvm/tdx.c
>> +++ b/target/i386/kvm/tdx.c
>> @@ -292,6 +292,11 @@ void tdx_get_supported_cpuid(uint32_t function,
>> uint32_t index, int reg,
>> if (function == 1 && reg == R_ECX && !enable_cpu_pm) {
>> *ret &= ~CPUID_EXT_MONITOR;
>> }
>> +
>> + /* QEMU Intel-pt support is broken, don't advertise Intel-PT */
>> + if (function == 7 && reg == R_EBX) {
>> + *ret &= ~CPUID_7_0_EBX_INTEL_PT;
>> + }
>> }
>> enum tdx_ioctl_level{
^ permalink raw reply [flat|nested] 124+ messages in thread
* [PATCH v5 19/65] i386/tdx: Update tdx_cpuid_lookup[].tdx_fixed0/1 by tdx_caps.cpuid_config[]
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (17 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 18/65] i386/tdx: Make Intel-PT unsupported for TD guest Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-05-31 9:27 ` Duan, Zhenzhong
2024-02-29 6:36 ` [PATCH v5 20/65] i386/tdx: Integrate tdx_caps->xfam_fixed0/1 into tdx_cpuid_lookup Xiaoyao Li
` (45 subsequent siblings)
64 siblings, 1 reply; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
tdx_cpuid_lookup[].tdx_fixed0/1 is QEMU maintained data which reflects
TDX restrictions regrading what bits are fixed by TDX module.
It's retrieved from TDX spec and static. However, TDX may evolve and
change some fixed fields to configurable in the future. Update
tdx_cpuid.lookup[].tdx_fixed0/1 fields by removing the bits that
reported from TDX module as configurable. This can adapt with the
updated TDX (module) automatically.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
target/i386/kvm/tdx.c | 34 ++++++++++++++++++++++++++++++++++
1 file changed, 34 insertions(+)
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 239170142e4f..424c0f3c0fbb 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -377,6 +377,38 @@ static int get_tdx_capabilities(Error **errp)
return 0;
}
+static void update_tdx_cpuid_lookup_by_tdx_caps(void)
+{
+ KvmTdxCpuidLookup *entry;
+ FeatureWordInfo *fi;
+ uint32_t config;
+ FeatureWord w;
+
+ for (w = 0; w < FEATURE_WORDS; w++) {
+ fi = &feature_word_info[w];
+ entry = &tdx_cpuid_lookup[w];
+
+ if (fi->type != CPUID_FEATURE_WORD) {
+ continue;
+ }
+
+ config = tdx_cap_cpuid_config(fi->cpuid.eax,
+ fi->cpuid.needs_ecx ? fi->cpuid.ecx : ~0u,
+ fi->cpuid.reg);
+
+ if (!config) {
+ continue;
+ }
+
+ /*
+ * Remove the configurable bits from tdx_fixed0/1 in case QEMU
+ * maintained fixed0/1 values is outdated to TDX module.
+ */
+ entry->tdx_fixed0 &= ~config;
+ entry->tdx_fixed1 &= ~config;
+ }
+}
+
static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
{
MachineState *ms = MACHINE(qdev_get_machine());
@@ -392,6 +424,8 @@ static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
}
}
+ update_tdx_cpuid_lookup_by_tdx_caps();
+
tdx_guest = tdx;
return 0;
}
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* Re: [PATCH v5 19/65] i386/tdx: Update tdx_cpuid_lookup[].tdx_fixed0/1 by tdx_caps.cpuid_config[]
2024-02-29 6:36 ` [PATCH v5 19/65] i386/tdx: Update tdx_cpuid_lookup[].tdx_fixed0/1 by tdx_caps.cpuid_config[] Xiaoyao Li
@ 2024-05-31 9:27 ` Duan, Zhenzhong
0 siblings, 0 replies; 124+ messages in thread
From: Duan, Zhenzhong @ 2024-05-31 9:27 UTC (permalink / raw)
To: Xiaoyao Li, Paolo Bonzini, David Hildenbrand, Igor Mammedov,
Eduardo Habkost, Marcel Apfelbaum, Philippe Mathieu-Daudé,
Yanan Wang, Michael S. Tsirkin, Richard Henderson, Ani Sinha,
Peter Xu, Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang
On 2/29/2024 2:36 PM, Xiaoyao Li wrote:
> tdx_cpuid_lookup[].tdx_fixed0/1 is QEMU maintained data which reflects
> TDX restrictions regrading what bits are fixed by TDX module.
>
> It's retrieved from TDX spec and static. However, TDX may evolve and
> change some fixed fields to configurable in the future. Update
> tdx_cpuid.lookup[].tdx_fixed0/1 fields by removing the bits that
> reported from TDX module as configurable. This can adapt with the
> updated TDX (module) automatically.
Can the fixed fields evolves to other type, i.e., native?
>
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
> target/i386/kvm/tdx.c | 34 ++++++++++++++++++++++++++++++++++
> 1 file changed, 34 insertions(+)
>
> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
> index 239170142e4f..424c0f3c0fbb 100644
> --- a/target/i386/kvm/tdx.c
> +++ b/target/i386/kvm/tdx.c
> @@ -377,6 +377,38 @@ static int get_tdx_capabilities(Error **errp)
> return 0;
> }
>
> +static void update_tdx_cpuid_lookup_by_tdx_caps(void)
> +{
> + KvmTdxCpuidLookup *entry;
> + FeatureWordInfo *fi;
> + uint32_t config;
> + FeatureWord w;
> +
> + for (w = 0; w < FEATURE_WORDS; w++) {
> + fi = &feature_word_info[w];
> + entry = &tdx_cpuid_lookup[w];
> +
> + if (fi->type != CPUID_FEATURE_WORD) {
> + continue;
> + }
> +
> + config = tdx_cap_cpuid_config(fi->cpuid.eax,
> + fi->cpuid.needs_ecx ? fi->cpuid.ecx : ~0u,
So the check "cpuid_c->sub_leaf == 0xffffffff" in tdx_cap_cpuid_config()
is unnecessary?
Thanks
Zhenzhong
> + fi->cpuid.reg);
> +
> + if (!config) {
> + continue;
> + }
> +
> + /*
> + * Remove the configurable bits from tdx_fixed0/1 in case QEMU
> + * maintained fixed0/1 values is outdated to TDX module.
> + */
> + entry->tdx_fixed0 &= ~config;
> + entry->tdx_fixed1 &= ~config;
> + }
> +}
> +
> static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
> {
> MachineState *ms = MACHINE(qdev_get_machine());
> @@ -392,6 +424,8 @@ static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
> }
> }
>
> + update_tdx_cpuid_lookup_by_tdx_caps();
> +
> tdx_guest = tdx;
> return 0;
> }
^ permalink raw reply [flat|nested] 124+ messages in thread
* [PATCH v5 20/65] i386/tdx: Integrate tdx_caps->xfam_fixed0/1 into tdx_cpuid_lookup
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (18 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 19/65] i386/tdx: Update tdx_cpuid_lookup[].tdx_fixed0/1 by tdx_caps.cpuid_config[] Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-02-29 6:36 ` [PATCH v5 21/65] i386/tdx: Integrate tdx_caps->attrs_fixed0/1 to tdx_cpuid_lookup Xiaoyao Li
` (44 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
KVM requires userspace to pass XFAM configuration via CPUID 0xD leaves.
Convert tdx_caps->xfam_fixed0/1 into corresponding
tdx_cpuid_lookup[].tdx_fixed0/1 field of CPUID 0xD leaves. Thus the
requirement can be applied naturally.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
target/i386/cpu.c | 3 ---
target/i386/cpu.h | 3 +++
target/i386/kvm/tdx.c | 24 ++++++++++++++++++++++++
3 files changed, 27 insertions(+), 3 deletions(-)
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 2666ef380891..b5f0c1080c34 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -1576,9 +1576,6 @@ static const X86RegisterInfo32 x86_reg_info_32[CPU_NB_REGS32] = {
};
#undef REGISTER
-/* CPUID feature bits available in XSS */
-#define CPUID_XSTATE_XSS_MASK (XSTATE_ARCH_LBR_MASK)
-
ExtSaveArea x86_ext_save_areas[XSAVE_STATE_AREA_COUNT] = {
[XSTATE_FP_BIT] = {
/* x87 FP state component is always enabled if XSAVE is supported */
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 7bd604f802a1..782aba921bd5 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -595,6 +595,9 @@ typedef enum X86Seg {
XSTATE_Hi16_ZMM_MASK | XSTATE_PKRU_MASK | \
XSTATE_XTILE_CFG_MASK | XSTATE_XTILE_DATA_MASK)
+/* CPUID feature bits available in XSS */
+#define CPUID_XSTATE_XSS_MASK (XSTATE_ARCH_LBR_MASK)
+
/* CPUID feature words */
typedef enum FeatureWord {
FEAT_1_EDX, /* CPUID[1].EDX */
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 424c0f3c0fbb..24475e6b312e 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -407,6 +407,30 @@ static void update_tdx_cpuid_lookup_by_tdx_caps(void)
entry->tdx_fixed0 &= ~config;
entry->tdx_fixed1 &= ~config;
}
+
+ /*
+ * Because KVM gets XFAM settings via CPUID leaves 0xD, map
+ * tdx_caps->xfam_fixed{0, 1} into tdx_cpuid_lookup[].tdx_fixed{0, 1}.
+ *
+ * Then the enforment applies in tdx_get_configurable_cpuid() naturally.
+ */
+ tdx_cpuid_lookup[FEAT_XSAVE_XCR0_LO].tdx_fixed0 =
+ (uint32_t)~tdx_caps->xfam_fixed0 & CPUID_XSTATE_XCR0_MASK;
+ tdx_cpuid_lookup[FEAT_XSAVE_XCR0_LO].tdx_fixed1 =
+ (uint32_t)tdx_caps->xfam_fixed1 & CPUID_XSTATE_XCR0_MASK;
+ tdx_cpuid_lookup[FEAT_XSAVE_XCR0_HI].tdx_fixed0 =
+ (~tdx_caps->xfam_fixed0 & CPUID_XSTATE_XCR0_MASK) >> 32;
+ tdx_cpuid_lookup[FEAT_XSAVE_XCR0_HI].tdx_fixed1 =
+ (tdx_caps->xfam_fixed1 & CPUID_XSTATE_XCR0_MASK) >> 32;
+
+ tdx_cpuid_lookup[FEAT_XSAVE_XSS_LO].tdx_fixed0 =
+ (uint32_t)~tdx_caps->xfam_fixed0 & CPUID_XSTATE_XSS_MASK;
+ tdx_cpuid_lookup[FEAT_XSAVE_XSS_LO].tdx_fixed1 =
+ (uint32_t)tdx_caps->xfam_fixed1 & CPUID_XSTATE_XSS_MASK;
+ tdx_cpuid_lookup[FEAT_XSAVE_XSS_HI].tdx_fixed0 =
+ (~tdx_caps->xfam_fixed0 & CPUID_XSTATE_XSS_MASK) >> 32;
+ tdx_cpuid_lookup[FEAT_XSAVE_XSS_HI].tdx_fixed1 =
+ (tdx_caps->xfam_fixed1 & CPUID_XSTATE_XSS_MASK) >> 32;
}
static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 21/65] i386/tdx: Integrate tdx_caps->attrs_fixed0/1 to tdx_cpuid_lookup
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (19 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 20/65] i386/tdx: Integrate tdx_caps->xfam_fixed0/1 into tdx_cpuid_lookup Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-02-29 6:36 ` [PATCH v5 22/65] i386/kvm: Move architectural CPUID leaf generation to separate helper Xiaoyao Li
` (43 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
Some bits in TD attributes have corresponding CPUID feature bits. Reflect
the fixed0/1 restriction on TD attributes to their corresponding CPUID
bits in tdx_cpuid_lookup[] as well.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
Changes in v4:
- reverse the meaning of tdx_caps->attr_fixed0, because value 0 of bit
means the bit must be fixed 0.
---
target/i386/cpu-internal.h | 9 +++++++++
target/i386/cpu.c | 9 ---------
target/i386/cpu.h | 2 ++
target/i386/kvm/tdx.c | 21 +++++++++++++++++++++
4 files changed, 32 insertions(+), 9 deletions(-)
diff --git a/target/i386/cpu-internal.h b/target/i386/cpu-internal.h
index 9baac5c0b450..e980f6e3147f 100644
--- a/target/i386/cpu-internal.h
+++ b/target/i386/cpu-internal.h
@@ -20,6 +20,15 @@
#ifndef I386_CPU_INTERNAL_H
#define I386_CPU_INTERNAL_H
+typedef struct FeatureMask {
+ FeatureWord index;
+ uint64_t mask;
+} FeatureMask;
+
+typedef struct FeatureDep {
+ FeatureMask from, to;
+} FeatureDep;
+
typedef enum FeatureWordType {
CPUID_FEATURE_WORD,
MSR_FEATURE_WORD,
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index b5f0c1080c34..78a3f55a856f 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -1443,15 +1443,6 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
},
};
-typedef struct FeatureMask {
- FeatureWord index;
- uint64_t mask;
-} FeatureMask;
-
-typedef struct FeatureDep {
- FeatureMask from, to;
-} FeatureDep;
-
static FeatureDep feature_dependencies[] = {
{
.from = { FEAT_7_0_EDX, CPUID_7_0_EDX_ARCH_CAPABILITIES },
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 782aba921bd5..f86afbdfca96 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -883,6 +883,8 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,
#define CPUID_7_0_ECX_MAWAU (31U << 17)
/* Read Processor ID */
#define CPUID_7_0_ECX_RDPID (1U << 22)
+/* KeyLocker */
+#define CPUID_7_0_ECX_KeyLocker (1U << 23)
/* Bus Lock Debug Exception */
#define CPUID_7_0_ECX_BUS_LOCK_DETECT (1U << 24)
/* Cache Line Demote Instruction */
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 24475e6b312e..144acd8c9912 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -32,6 +32,13 @@
(1U << KVM_FEATURE_PV_SCHED_YIELD) | \
(1U << KVM_FEATURE_MSI_EXT_DEST_ID))
+#define TDX_ATTRIBUTES_MAX_BITS 64
+
+static FeatureMask tdx_attrs_ctrl_fields[TDX_ATTRIBUTES_MAX_BITS] = {
+ [30] = { .index = FEAT_7_0_ECX, .mask = CPUID_7_0_ECX_PKS },
+ [31] = { .index = FEAT_7_0_ECX, .mask = CPUID_7_0_ECX_KeyLocker},
+};
+
typedef struct KvmTdxCpuidLookup {
uint32_t tdx_fixed0;
uint32_t tdx_fixed1;
@@ -383,6 +390,8 @@ static void update_tdx_cpuid_lookup_by_tdx_caps(void)
FeatureWordInfo *fi;
uint32_t config;
FeatureWord w;
+ FeatureMask *fm;
+ int i;
for (w = 0; w < FEATURE_WORDS; w++) {
fi = &feature_word_info[w];
@@ -408,6 +417,18 @@ static void update_tdx_cpuid_lookup_by_tdx_caps(void)
entry->tdx_fixed1 &= ~config;
}
+ for (i = 0; i < ARRAY_SIZE(tdx_attrs_ctrl_fields); i++) {
+ fm = &tdx_attrs_ctrl_fields[i];
+
+ if (~(tdx_caps->attrs_fixed0 & (1ULL << i))) {
+ tdx_cpuid_lookup[fm->index].tdx_fixed0 |= fm->mask;
+ }
+
+ if (tdx_caps->attrs_fixed1 & (1ULL << i)) {
+ tdx_cpuid_lookup[fm->index].tdx_fixed1 |= fm->mask;
+ }
+ }
+
/*
* Because KVM gets XFAM settings via CPUID leaves 0xD, map
* tdx_caps->xfam_fixed{0, 1} into tdx_cpuid_lookup[].tdx_fixed{0, 1}.
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 22/65] i386/kvm: Move architectural CPUID leaf generation to separate helper
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (20 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 21/65] i386/tdx: Integrate tdx_caps->attrs_fixed0/1 to tdx_cpuid_lookup Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-02-29 6:36 ` [PATCH v5 23/65] kvm: Introduce kvm_arch_pre_create_vcpu() Xiaoyao Li
` (42 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
From: Sean Christopherson <sean.j.christopherson@intel.com>
Move the architectural (for lack of a better term) CPUID leaf generation
to a separate helper so that the generation code can be reused by TDX,
which needs to generate a canonical VM-scoped configuration.
Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
target/i386/kvm/kvm.c | 459 +++++++++++++++++++------------------
target/i386/kvm/kvm_i386.h | 3 +
2 files changed, 240 insertions(+), 222 deletions(-)
diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index 389b631c03dd..315998c8f7e5 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -1731,6 +1731,241 @@ static void kvm_init_nested_state(CPUX86State *env)
}
}
+uint32_t kvm_x86_arch_cpuid(CPUX86State *env, struct kvm_cpuid_entry2 *entries,
+ uint32_t cpuid_i)
+{
+ uint32_t limit, i, j;
+ uint32_t unused;
+ struct kvm_cpuid_entry2 *c;
+
+ if (cpuid_i > KVM_MAX_CPUID_ENTRIES) {
+ error_report("exceeded cpuid index (%d) for entries[]", cpuid_i);
+ abort();
+ }
+
+ cpu_x86_cpuid(env, 0, 0, &limit, &unused, &unused, &unused);
+
+ for (i = 0; i <= limit; i++) {
+ if (cpuid_i == KVM_MAX_CPUID_ENTRIES) {
+ fprintf(stderr, "unsupported level value: 0x%x\n", limit);
+ abort();
+ }
+ c = &entries[cpuid_i++];
+
+ switch (i) {
+ case 2: {
+ /* Keep reading function 2 till all the input is received */
+ int times;
+
+ c->function = i;
+ c->flags = KVM_CPUID_FLAG_STATEFUL_FUNC |
+ KVM_CPUID_FLAG_STATE_READ_NEXT;
+ cpu_x86_cpuid(env, i, 0, &c->eax, &c->ebx, &c->ecx, &c->edx);
+ times = c->eax & 0xff;
+
+ for (j = 1; j < times; ++j) {
+ if (cpuid_i == KVM_MAX_CPUID_ENTRIES) {
+ fprintf(stderr, "cpuid_data is full, no space for "
+ "cpuid(eax:2):eax & 0xf = 0x%x\n", times);
+ abort();
+ }
+ c = &entries[cpuid_i++];
+ c->function = i;
+ c->flags = KVM_CPUID_FLAG_STATEFUL_FUNC;
+ cpu_x86_cpuid(env, i, 0, &c->eax, &c->ebx, &c->ecx, &c->edx);
+ }
+ break;
+ }
+ case 0x1f:
+ if (env->nr_dies < 2) {
+ cpuid_i--;
+ break;
+ }
+ /* fallthrough */
+ case 4:
+ case 0xb:
+ case 0xd:
+ for (j = 0; ; j++) {
+ if (i == 0xd && j == 64) {
+ break;
+ }
+
+ c->function = i;
+ c->flags = KVM_CPUID_FLAG_SIGNIFCANT_INDEX;
+ c->index = j;
+ cpu_x86_cpuid(env, i, j, &c->eax, &c->ebx, &c->ecx, &c->edx);
+
+ if (i == 4 && c->eax == 0) {
+ break;
+ }
+ if (i == 0xb && !(c->ecx & 0xff00)) {
+ break;
+ }
+ if (i == 0x1f && !(c->ecx & 0xff00)) {
+ break;
+ }
+ if (i == 0xd && c->eax == 0) {
+ continue;
+ }
+ if (cpuid_i == KVM_MAX_CPUID_ENTRIES) {
+ fprintf(stderr, "cpuid_data is full, no space for "
+ "cpuid(eax:0x%x,ecx:0x%x)\n", i, j);
+ abort();
+ }
+ c = &entries[cpuid_i++];
+ }
+ break;
+ case 0x12:
+ for (j = 0; ; j++) {
+ c->function = i;
+ c->flags = KVM_CPUID_FLAG_SIGNIFCANT_INDEX;
+ c->index = j;
+ cpu_x86_cpuid(env, i, j, &c->eax, &c->ebx, &c->ecx, &c->edx);
+
+ if (j > 1 && (c->eax & 0xf) != 1) {
+ break;
+ }
+
+ if (cpuid_i == KVM_MAX_CPUID_ENTRIES) {
+ fprintf(stderr, "cpuid_data is full, no space for "
+ "cpuid(eax:0x12,ecx:0x%x)\n", j);
+ abort();
+ }
+ c = &entries[cpuid_i++];
+ }
+ break;
+ case 0x7:
+ case 0x14:
+ case 0x1d:
+ case 0x1e: {
+ uint32_t times;
+
+ c->function = i;
+ c->index = 0;
+ c->flags = KVM_CPUID_FLAG_SIGNIFCANT_INDEX;
+ cpu_x86_cpuid(env, i, 0, &c->eax, &c->ebx, &c->ecx, &c->edx);
+ times = c->eax;
+
+ for (j = 1; j <= times; ++j) {
+ if (cpuid_i == KVM_MAX_CPUID_ENTRIES) {
+ fprintf(stderr, "cpuid_data is full, no space for "
+ "cpuid(eax:0x%x,ecx:0x%x)\n", i, j);
+ abort();
+ }
+ c = &entries[cpuid_i++];
+ c->function = i;
+ c->index = j;
+ c->flags = KVM_CPUID_FLAG_SIGNIFCANT_INDEX;
+ cpu_x86_cpuid(env, i, j, &c->eax, &c->ebx, &c->ecx, &c->edx);
+ }
+ break;
+ }
+ default:
+ c->function = i;
+ c->flags = 0;
+ cpu_x86_cpuid(env, i, 0, &c->eax, &c->ebx, &c->ecx, &c->edx);
+ if (!c->eax && !c->ebx && !c->ecx && !c->edx) {
+ /*
+ * KVM already returns all zeroes if a CPUID entry is missing,
+ * so we can omit it and avoid hitting KVM's 80-entry limit.
+ */
+ cpuid_i--;
+ }
+ break;
+ }
+ }
+
+ if (limit >= 0x0a) {
+ uint32_t eax, edx;
+
+ cpu_x86_cpuid(env, 0x0a, 0, &eax, &unused, &unused, &edx);
+
+ has_architectural_pmu_version = eax & 0xff;
+ if (has_architectural_pmu_version > 0) {
+ num_architectural_pmu_gp_counters = (eax & 0xff00) >> 8;
+
+ /* Shouldn't be more than 32, since that's the number of bits
+ * available in EBX to tell us _which_ counters are available.
+ * Play it safe.
+ */
+ if (num_architectural_pmu_gp_counters > MAX_GP_COUNTERS) {
+ num_architectural_pmu_gp_counters = MAX_GP_COUNTERS;
+ }
+
+ if (has_architectural_pmu_version > 1) {
+ num_architectural_pmu_fixed_counters = edx & 0x1f;
+
+ if (num_architectural_pmu_fixed_counters > MAX_FIXED_COUNTERS) {
+ num_architectural_pmu_fixed_counters = MAX_FIXED_COUNTERS;
+ }
+ }
+ }
+ }
+
+ cpu_x86_cpuid(env, 0x80000000, 0, &limit, &unused, &unused, &unused);
+
+ for (i = 0x80000000; i <= limit; i++) {
+ if (cpuid_i == KVM_MAX_CPUID_ENTRIES) {
+ fprintf(stderr, "unsupported xlevel value: 0x%x\n", limit);
+ abort();
+ }
+ c = &entries[cpuid_i++];
+
+ switch (i) {
+ case 0x8000001d:
+ /* Query for all AMD cache information leaves */
+ for (j = 0; ; j++) {
+ c->function = i;
+ c->flags = KVM_CPUID_FLAG_SIGNIFCANT_INDEX;
+ c->index = j;
+ cpu_x86_cpuid(env, i, j, &c->eax, &c->ebx, &c->ecx, &c->edx);
+
+ if (c->eax == 0) {
+ break;
+ }
+ if (cpuid_i == KVM_MAX_CPUID_ENTRIES) {
+ fprintf(stderr, "cpuid_data is full, no space for "
+ "cpuid(eax:0x%x,ecx:0x%x)\n", i, j);
+ abort();
+ }
+ c = &entries[cpuid_i++];
+ }
+ break;
+ default:
+ c->function = i;
+ c->flags = 0;
+ cpu_x86_cpuid(env, i, 0, &c->eax, &c->ebx, &c->ecx, &c->edx);
+ if (!c->eax && !c->ebx && !c->ecx && !c->edx) {
+ /*
+ * KVM already returns all zeroes if a CPUID entry is missing,
+ * so we can omit it and avoid hitting KVM's 80-entry limit.
+ */
+ cpuid_i--;
+ }
+ break;
+ }
+ }
+
+ /* Call Centaur's CPUID instructions they are supported. */
+ if (env->cpuid_xlevel2 > 0) {
+ cpu_x86_cpuid(env, 0xC0000000, 0, &limit, &unused, &unused, &unused);
+
+ for (i = 0xC0000000; i <= limit; i++) {
+ if (cpuid_i == KVM_MAX_CPUID_ENTRIES) {
+ fprintf(stderr, "unsupported xlevel2 value: 0x%x\n", limit);
+ abort();
+ }
+ c = &entries[cpuid_i++];
+
+ c->function = i;
+ c->flags = 0;
+ cpu_x86_cpuid(env, i, 0, &c->eax, &c->ebx, &c->ecx, &c->edx);
+ }
+ }
+
+ return cpuid_i;
+}
+
int kvm_arch_init_vcpu(CPUState *cs)
{
struct {
@@ -1747,8 +1982,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
X86CPU *cpu = X86_CPU(cs);
CPUX86State *env = &cpu->env;
- uint32_t limit, i, j, cpuid_i;
- uint32_t unused;
+ uint32_t cpuid_i;
struct kvm_cpuid_entry2 *c;
uint32_t signature[3];
int kvm_base = KVM_CPUID_SIGNATURE;
@@ -1901,8 +2135,6 @@ int kvm_arch_init_vcpu(CPUState *cs)
c->edx = env->features[FEAT_KVM_HINTS];
}
- cpu_x86_cpuid(env, 0, 0, &limit, &unused, &unused, &unused);
-
if (cpu->kvm_pv_enforce_cpuid) {
r = kvm_vcpu_enable_cap(cs, KVM_CAP_ENFORCE_PV_FEATURE_CPUID, 0, 1);
if (r < 0) {
@@ -1913,224 +2145,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
}
}
- for (i = 0; i <= limit; i++) {
- if (cpuid_i == KVM_MAX_CPUID_ENTRIES) {
- fprintf(stderr, "unsupported level value: 0x%x\n", limit);
- abort();
- }
- c = &cpuid_data.entries[cpuid_i++];
-
- switch (i) {
- case 2: {
- /* Keep reading function 2 till all the input is received */
- int times;
-
- c->function = i;
- c->flags = KVM_CPUID_FLAG_STATEFUL_FUNC |
- KVM_CPUID_FLAG_STATE_READ_NEXT;
- cpu_x86_cpuid(env, i, 0, &c->eax, &c->ebx, &c->ecx, &c->edx);
- times = c->eax & 0xff;
-
- for (j = 1; j < times; ++j) {
- if (cpuid_i == KVM_MAX_CPUID_ENTRIES) {
- fprintf(stderr, "cpuid_data is full, no space for "
- "cpuid(eax:2):eax & 0xf = 0x%x\n", times);
- abort();
- }
- c = &cpuid_data.entries[cpuid_i++];
- c->function = i;
- c->flags = KVM_CPUID_FLAG_STATEFUL_FUNC;
- cpu_x86_cpuid(env, i, 0, &c->eax, &c->ebx, &c->ecx, &c->edx);
- }
- break;
- }
- case 0x1f:
- if (env->nr_dies < 2) {
- cpuid_i--;
- break;
- }
- /* fallthrough */
- case 4:
- case 0xb:
- case 0xd:
- for (j = 0; ; j++) {
- if (i == 0xd && j == 64) {
- break;
- }
-
- c->function = i;
- c->flags = KVM_CPUID_FLAG_SIGNIFCANT_INDEX;
- c->index = j;
- cpu_x86_cpuid(env, i, j, &c->eax, &c->ebx, &c->ecx, &c->edx);
-
- if (i == 4 && c->eax == 0) {
- break;
- }
- if (i == 0xb && !(c->ecx & 0xff00)) {
- break;
- }
- if (i == 0x1f && !(c->ecx & 0xff00)) {
- break;
- }
- if (i == 0xd && c->eax == 0) {
- continue;
- }
- if (cpuid_i == KVM_MAX_CPUID_ENTRIES) {
- fprintf(stderr, "cpuid_data is full, no space for "
- "cpuid(eax:0x%x,ecx:0x%x)\n", i, j);
- abort();
- }
- c = &cpuid_data.entries[cpuid_i++];
- }
- break;
- case 0x12:
- for (j = 0; ; j++) {
- c->function = i;
- c->flags = KVM_CPUID_FLAG_SIGNIFCANT_INDEX;
- c->index = j;
- cpu_x86_cpuid(env, i, j, &c->eax, &c->ebx, &c->ecx, &c->edx);
-
- if (j > 1 && (c->eax & 0xf) != 1) {
- break;
- }
-
- if (cpuid_i == KVM_MAX_CPUID_ENTRIES) {
- fprintf(stderr, "cpuid_data is full, no space for "
- "cpuid(eax:0x12,ecx:0x%x)\n", j);
- abort();
- }
- c = &cpuid_data.entries[cpuid_i++];
- }
- break;
- case 0x7:
- case 0x14:
- case 0x1d:
- case 0x1e: {
- uint32_t times;
-
- c->function = i;
- c->index = 0;
- c->flags = KVM_CPUID_FLAG_SIGNIFCANT_INDEX;
- cpu_x86_cpuid(env, i, 0, &c->eax, &c->ebx, &c->ecx, &c->edx);
- times = c->eax;
-
- for (j = 1; j <= times; ++j) {
- if (cpuid_i == KVM_MAX_CPUID_ENTRIES) {
- fprintf(stderr, "cpuid_data is full, no space for "
- "cpuid(eax:0x%x,ecx:0x%x)\n", i, j);
- abort();
- }
- c = &cpuid_data.entries[cpuid_i++];
- c->function = i;
- c->index = j;
- c->flags = KVM_CPUID_FLAG_SIGNIFCANT_INDEX;
- cpu_x86_cpuid(env, i, j, &c->eax, &c->ebx, &c->ecx, &c->edx);
- }
- break;
- }
- default:
- c->function = i;
- c->flags = 0;
- cpu_x86_cpuid(env, i, 0, &c->eax, &c->ebx, &c->ecx, &c->edx);
- if (!c->eax && !c->ebx && !c->ecx && !c->edx) {
- /*
- * KVM already returns all zeroes if a CPUID entry is missing,
- * so we can omit it and avoid hitting KVM's 80-entry limit.
- */
- cpuid_i--;
- }
- break;
- }
- }
-
- if (limit >= 0x0a) {
- uint32_t eax, edx;
-
- cpu_x86_cpuid(env, 0x0a, 0, &eax, &unused, &unused, &edx);
-
- has_architectural_pmu_version = eax & 0xff;
- if (has_architectural_pmu_version > 0) {
- num_architectural_pmu_gp_counters = (eax & 0xff00) >> 8;
-
- /* Shouldn't be more than 32, since that's the number of bits
- * available in EBX to tell us _which_ counters are available.
- * Play it safe.
- */
- if (num_architectural_pmu_gp_counters > MAX_GP_COUNTERS) {
- num_architectural_pmu_gp_counters = MAX_GP_COUNTERS;
- }
-
- if (has_architectural_pmu_version > 1) {
- num_architectural_pmu_fixed_counters = edx & 0x1f;
-
- if (num_architectural_pmu_fixed_counters > MAX_FIXED_COUNTERS) {
- num_architectural_pmu_fixed_counters = MAX_FIXED_COUNTERS;
- }
- }
- }
- }
-
- cpu_x86_cpuid(env, 0x80000000, 0, &limit, &unused, &unused, &unused);
-
- for (i = 0x80000000; i <= limit; i++) {
- if (cpuid_i == KVM_MAX_CPUID_ENTRIES) {
- fprintf(stderr, "unsupported xlevel value: 0x%x\n", limit);
- abort();
- }
- c = &cpuid_data.entries[cpuid_i++];
-
- switch (i) {
- case 0x8000001d:
- /* Query for all AMD cache information leaves */
- for (j = 0; ; j++) {
- c->function = i;
- c->flags = KVM_CPUID_FLAG_SIGNIFCANT_INDEX;
- c->index = j;
- cpu_x86_cpuid(env, i, j, &c->eax, &c->ebx, &c->ecx, &c->edx);
-
- if (c->eax == 0) {
- break;
- }
- if (cpuid_i == KVM_MAX_CPUID_ENTRIES) {
- fprintf(stderr, "cpuid_data is full, no space for "
- "cpuid(eax:0x%x,ecx:0x%x)\n", i, j);
- abort();
- }
- c = &cpuid_data.entries[cpuid_i++];
- }
- break;
- default:
- c->function = i;
- c->flags = 0;
- cpu_x86_cpuid(env, i, 0, &c->eax, &c->ebx, &c->ecx, &c->edx);
- if (!c->eax && !c->ebx && !c->ecx && !c->edx) {
- /*
- * KVM already returns all zeroes if a CPUID entry is missing,
- * so we can omit it and avoid hitting KVM's 80-entry limit.
- */
- cpuid_i--;
- }
- break;
- }
- }
-
- /* Call Centaur's CPUID instructions they are supported. */
- if (env->cpuid_xlevel2 > 0) {
- cpu_x86_cpuid(env, 0xC0000000, 0, &limit, &unused, &unused, &unused);
-
- for (i = 0xC0000000; i <= limit; i++) {
- if (cpuid_i == KVM_MAX_CPUID_ENTRIES) {
- fprintf(stderr, "unsupported xlevel2 value: 0x%x\n", limit);
- abort();
- }
- c = &cpuid_data.entries[cpuid_i++];
-
- c->function = i;
- c->flags = 0;
- cpu_x86_cpuid(env, i, 0, &c->eax, &c->ebx, &c->ecx, &c->edx);
- }
- }
-
+ cpuid_i = kvm_x86_arch_cpuid(env, cpuid_data.entries, cpuid_i);
cpuid_data.cpuid.nent = cpuid_i;
if (((env->cpuid_version >> 8)&0xF) >= 6
diff --git a/target/i386/kvm/kvm_i386.h b/target/i386/kvm/kvm_i386.h
index c3ef46a97a7b..cbf52c1c6d17 100644
--- a/target/i386/kvm/kvm_i386.h
+++ b/target/i386/kvm/kvm_i386.h
@@ -24,6 +24,9 @@
#define kvm_ioapic_in_kernel() \
(kvm_irqchip_in_kernel() && !kvm_irqchip_is_split())
+uint32_t kvm_x86_arch_cpuid(CPUX86State *env, struct kvm_cpuid_entry2 *entries,
+ uint32_t cpuid_i);
+
#else
#define kvm_pit_in_kernel() 0
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 23/65] kvm: Introduce kvm_arch_pre_create_vcpu()
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (21 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 22/65] i386/kvm: Move architectural CPUID leaf generation to separate helper Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-02-29 6:36 ` [PATCH v5 24/65] i386/tdx: Initialize TDX before creating TD vcpus Xiaoyao Li
` (41 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
Introduce kvm_arch_pre_create_vcpu(), to perform arch-dependent
work prior to create any vcpu. This is for i386 TDX because it needs
call TDX_INIT_VM before creating any vcpu.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
Changes in v3:
- pass @errp to kvm_arch_pre_create_vcpu(); (Per Daniel)
---
accel/kvm/kvm-all.c | 10 ++++++++++
include/sysemu/kvm.h | 1 +
2 files changed, 11 insertions(+)
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 0c0719a0303c..a8a99d48e4ce 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -418,6 +418,11 @@ static int kvm_get_vcpu(KVMState *s, unsigned long vcpu_id)
return kvm_vm_ioctl(s, KVM_CREATE_VCPU, (void *)vcpu_id);
}
+int __attribute__ ((weak)) kvm_arch_pre_create_vcpu(CPUState *cpu, Error **errp)
+{
+ return 0;
+}
+
int kvm_init_vcpu(CPUState *cpu, Error **errp)
{
KVMState *s = kvm_state;
@@ -426,6 +431,11 @@ int kvm_init_vcpu(CPUState *cpu, Error **errp)
trace_kvm_init_vcpu(cpu->cpu_index, kvm_arch_vcpu_id(cpu));
+ ret = kvm_arch_pre_create_vcpu(cpu, errp);
+ if (ret < 0) {
+ goto err;
+ }
+
ret = kvm_get_vcpu(s, kvm_arch_vcpu_id(cpu));
if (ret < 0) {
error_setg_errno(errp, -ret, "kvm_init_vcpu: kvm_get_vcpu failed (%lu)",
diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
index 8e83adfbbd19..82b547848130 100644
--- a/include/sysemu/kvm.h
+++ b/include/sysemu/kvm.h
@@ -341,6 +341,7 @@ int kvm_arch_get_default_type(MachineState *ms);
int kvm_arch_init(MachineState *ms, KVMState *s);
+int kvm_arch_pre_create_vcpu(CPUState *cpu, Error **errp);
int kvm_arch_init_vcpu(CPUState *cpu);
int kvm_arch_destroy_vcpu(CPUState *cpu);
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 24/65] i386/tdx: Initialize TDX before creating TD vcpus
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (22 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 23/65] kvm: Introduce kvm_arch_pre_create_vcpu() Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-02-29 6:36 ` [PATCH v5 25/65] i386/tdx: Add property sept-ve-disable for tdx-guest object Xiaoyao Li
` (40 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
Invoke KVM_TDX_INIT in kvm_arch_pre_create_vcpu() that KVM_TDX_INIT
configures global TD configurations, e.g. the canonical CPUID config,
and must be executed prior to creating vCPUs.
Use kvm_x86_arch_cpuid() to setup the CPUID settings for TDX VM.
Note, this doesn't address the fact that QEMU may change the CPUID
configuration when creating vCPUs, i.e. punts on refactoring QEMU to
provide a stable CPUID config prior to kvm_arch_init().
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
---
Changes in v4:
- mark init_vm with g_autofree() and use QEMU_LOCK_GUARD() to eliminate
the goto labels; (Daniel)
Changes in v3:
- Pass @errp in tdx_pre_create_vcpu() and pass error info to it. (Daniel)
---
accel/kvm/kvm-all.c | 9 +++++++-
target/i386/kvm/kvm.c | 9 ++++++++
target/i386/kvm/meson.build | 2 +-
target/i386/kvm/tdx-stub.c | 8 ++++++++
target/i386/kvm/tdx.c | 41 +++++++++++++++++++++++++++++++++++++
target/i386/kvm/tdx.h | 4 ++++
6 files changed, 71 insertions(+), 2 deletions(-)
create mode 100644 target/i386/kvm/tdx-stub.c
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index a8a99d48e4ce..c9df41efa484 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -431,8 +431,15 @@ int kvm_init_vcpu(CPUState *cpu, Error **errp)
trace_kvm_init_vcpu(cpu->cpu_index, kvm_arch_vcpu_id(cpu));
+ /*
+ * tdx_pre_create_vcpu() may call cpu_x86_cpuid(). It in turn may call
+ * kvm_vm_ioctl(). Set cpu->kvm_state in advance to avoid NULL pointer
+ * dereference.
+ */
+ cpu->kvm_state = s;
ret = kvm_arch_pre_create_vcpu(cpu, errp);
if (ret < 0) {
+ cpu->kvm_state = NULL;
goto err;
}
@@ -440,11 +447,11 @@ int kvm_init_vcpu(CPUState *cpu, Error **errp)
if (ret < 0) {
error_setg_errno(errp, -ret, "kvm_init_vcpu: kvm_get_vcpu failed (%lu)",
kvm_arch_vcpu_id(cpu));
+ cpu->kvm_state = NULL;
goto err;
}
cpu->kvm_fd = ret;
- cpu->kvm_state = s;
cpu->vcpu_dirty = true;
cpu->dirty_pages = 0;
cpu->throttle_us_per_full = 0;
diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index 315998c8f7e5..1664ac49005e 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -2271,6 +2271,15 @@ int kvm_arch_init_vcpu(CPUState *cs)
return r;
}
+int kvm_arch_pre_create_vcpu(CPUState *cpu, Error **errp)
+{
+ if (is_tdx_vm()) {
+ return tdx_pre_create_vcpu(cpu, errp);
+ }
+
+ return 0;
+}
+
int kvm_arch_destroy_vcpu(CPUState *cs)
{
X86CPU *cpu = X86_CPU(cs);
diff --git a/target/i386/kvm/meson.build b/target/i386/kvm/meson.build
index 26a1ab038513..460c5f8f85f3 100644
--- a/target/i386/kvm/meson.build
+++ b/target/i386/kvm/meson.build
@@ -7,7 +7,7 @@ i386_kvm_ss.add(files(
i386_kvm_ss.add(when: 'CONFIG_XEN_EMU', if_true: files('xen-emu.c'))
-i386_kvm_ss.add(when: 'CONFIG_TDX', if_true: files('tdx.c'))
+i386_kvm_ss.add(when: 'CONFIG_TDX', if_true: files('tdx.c'), if_false: files('tdx-stub.c'))
i386_system_ss.add(when: 'CONFIG_HYPERV', if_true: files('hyperv.c'), if_false: files('hyperv-stub.c'))
diff --git a/target/i386/kvm/tdx-stub.c b/target/i386/kvm/tdx-stub.c
new file mode 100644
index 000000000000..b614b46d3f4a
--- /dev/null
+++ b/target/i386/kvm/tdx-stub.c
@@ -0,0 +1,8 @@
+#include "qemu/osdep.h"
+
+#include "tdx.h"
+
+int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
+{
+ return -EINVAL;
+}
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 144acd8c9912..d548ec340285 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -475,6 +475,45 @@ static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
return 0;
}
+int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
+{
+ MachineState *ms = MACHINE(qdev_get_machine());
+ X86CPU *x86cpu = X86_CPU(cpu);
+ CPUX86State *env = &x86cpu->env;
+ g_autofree struct kvm_tdx_init_vm *init_vm = NULL;
+ int r = 0;
+
+ QEMU_LOCK_GUARD(&tdx_guest->lock);
+ if (tdx_guest->initialized) {
+ return r;
+ }
+
+ init_vm = g_malloc0(sizeof(struct kvm_tdx_init_vm) +
+ sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES);
+
+ r = kvm_vm_enable_cap(kvm_state, KVM_CAP_MAX_VCPUS, 0, ms->smp.cpus);
+ if (r < 0) {
+ error_setg(errp, "Unable to set MAX VCPUS to %d", ms->smp.cpus);
+ return r;
+ }
+
+ init_vm->cpuid.nent = kvm_x86_arch_cpuid(env, init_vm->cpuid.entries, 0);
+
+ init_vm->attributes = tdx_guest->attributes;
+
+ do {
+ r = tdx_vm_ioctl(KVM_TDX_INIT_VM, 0, init_vm);
+ } while (r == -EAGAIN);
+ if (r < 0) {
+ error_setg_errno(errp, -r, "KVM_TDX_INIT_VM failed");
+ return r;
+ }
+
+ tdx_guest->initialized = true;
+
+ return 0;
+}
+
/* tdx guest */
OBJECT_DEFINE_TYPE_WITH_INTERFACES(TdxGuest,
tdx_guest,
@@ -487,6 +526,8 @@ static void tdx_guest_init(Object *obj)
{
TdxGuest *tdx = TDX_GUEST(obj);
+ qemu_mutex_init(&tdx->lock);
+
tdx->attributes = 0;
}
diff --git a/target/i386/kvm/tdx.h b/target/i386/kvm/tdx.h
index f62fe8ece982..0df910725b52 100644
--- a/target/i386/kvm/tdx.h
+++ b/target/i386/kvm/tdx.h
@@ -17,6 +17,9 @@ typedef struct TdxGuestClass {
typedef struct TdxGuest {
ConfidentialGuestSupport parent_obj;
+ QemuMutex lock;
+
+ bool initialized;
uint64_t attributes; /* TD attributes */
} TdxGuest;
@@ -28,5 +31,6 @@ bool is_tdx_vm(void);
void tdx_get_supported_cpuid(uint32_t function, uint32_t index, int reg,
uint32_t *ret);
+int tdx_pre_create_vcpu(CPUState *cpu, Error **errp);
#endif /* QEMU_I386_TDX_H */
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 25/65] i386/tdx: Add property sept-ve-disable for tdx-guest object
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (23 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 24/65] i386/tdx: Initialize TDX before creating TD vcpus Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-06-06 10:45 ` Daniel P. Berrangé
2024-02-29 6:36 ` [PATCH v5 26/65] i386/tdx: Make sept_ve_disable set by default Xiaoyao Li
` (39 subsequent siblings)
64 siblings, 1 reply; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
Bit 28 of TD attribute, named SEPT_VE_DISABLE. When set to 1, it disables
EPT violation conversion to #VE on guest TD access of PENDING pages.
Some guest OS (e.g., Linux TD guest) may require this bit as 1.
Otherwise refuse to boot.
Add sept-ve-disable property for tdx-guest object, for user to configure
this bit.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
---
Changes in v4:
- collect Acked-by from Markus
Changes in v3:
- update the comment of property @sept-ve-disable to make it more
descriptive and use new format. (Daniel and Markus)
---
qapi/qom.json | 7 ++++++-
target/i386/kvm/tdx.c | 24 ++++++++++++++++++++++++
2 files changed, 30 insertions(+), 1 deletion(-)
diff --git a/qapi/qom.json b/qapi/qom.json
index 220cc6c98d4b..89ed89b9b46e 100644
--- a/qapi/qom.json
+++ b/qapi/qom.json
@@ -900,10 +900,15 @@
#
# Properties for tdx-guest objects.
#
+# @sept-ve-disable: toggle bit 28 of TD attributes to control disabling
+# of EPT violation conversion to #VE on guest TD access of PENDING
+# pages. Some guest OS (e.g., Linux TD guest) may require this to
+# be set, otherwise they refuse to boot.
+#
# Since: 9.0
##
{ 'struct': 'TdxGuestProperties',
- 'data': { }}
+ 'data': { '*sept-ve-disable': 'bool' } }
##
# @ThreadContextProperties:
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index d548ec340285..806192158c9d 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -32,6 +32,8 @@
(1U << KVM_FEATURE_PV_SCHED_YIELD) | \
(1U << KVM_FEATURE_MSI_EXT_DEST_ID))
+#define TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE BIT_ULL(28)
+
#define TDX_ATTRIBUTES_MAX_BITS 64
static FeatureMask tdx_attrs_ctrl_fields[TDX_ATTRIBUTES_MAX_BITS] = {
@@ -514,6 +516,24 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
return 0;
}
+static bool tdx_guest_get_sept_ve_disable(Object *obj, Error **errp)
+{
+ TdxGuest *tdx = TDX_GUEST(obj);
+
+ return !!(tdx->attributes & TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE);
+}
+
+static void tdx_guest_set_sept_ve_disable(Object *obj, bool value, Error **errp)
+{
+ TdxGuest *tdx = TDX_GUEST(obj);
+
+ if (value) {
+ tdx->attributes |= TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE;
+ } else {
+ tdx->attributes &= ~TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE;
+ }
+}
+
/* tdx guest */
OBJECT_DEFINE_TYPE_WITH_INTERFACES(TdxGuest,
tdx_guest,
@@ -529,6 +549,10 @@ static void tdx_guest_init(Object *obj)
qemu_mutex_init(&tdx->lock);
tdx->attributes = 0;
+
+ object_property_add_bool(obj, "sept-ve-disable",
+ tdx_guest_get_sept_ve_disable,
+ tdx_guest_set_sept_ve_disable);
}
static void tdx_guest_finalize(Object *obj)
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* Re: [PATCH v5 25/65] i386/tdx: Add property sept-ve-disable for tdx-guest object
2024-02-29 6:36 ` [PATCH v5 25/65] i386/tdx: Add property sept-ve-disable for tdx-guest object Xiaoyao Li
@ 2024-06-06 10:45 ` Daniel P. Berrangé
2024-06-12 8:28 ` Xiaoyao Li
0 siblings, 1 reply; 124+ messages in thread
From: Daniel P. Berrangé @ 2024-06-06 10:45 UTC (permalink / raw)
To: Xiaoyao Li
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Eric Blake, Markus Armbruster, Marcelo Tosatti,
kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, Zhenzhong Duan
Copying Zhenzhong Duan as my point relates to the proposed libvirt
TDX patches.
On Thu, Feb 29, 2024 at 01:36:46AM -0500, Xiaoyao Li wrote:
> Bit 28 of TD attribute, named SEPT_VE_DISABLE. When set to 1, it disables
> EPT violation conversion to #VE on guest TD access of PENDING pages.
>
> Some guest OS (e.g., Linux TD guest) may require this bit as 1.
> Otherwise refuse to boot.
>
> Add sept-ve-disable property for tdx-guest object, for user to configure
> this bit.
>
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> Acked-by: Markus Armbruster <armbru@redhat.com>
> ---
> Changes in v4:
> - collect Acked-by from Markus
>
> Changes in v3:
> - update the comment of property @sept-ve-disable to make it more
> descriptive and use new format. (Daniel and Markus)
> ---
> qapi/qom.json | 7 ++++++-
> target/i386/kvm/tdx.c | 24 ++++++++++++++++++++++++
> 2 files changed, 30 insertions(+), 1 deletion(-)
>
> diff --git a/qapi/qom.json b/qapi/qom.json
> index 220cc6c98d4b..89ed89b9b46e 100644
> --- a/qapi/qom.json
> +++ b/qapi/qom.json
> @@ -900,10 +900,15 @@
> #
> # Properties for tdx-guest objects.
> #
> +# @sept-ve-disable: toggle bit 28 of TD attributes to control disabling
> +# of EPT violation conversion to #VE on guest TD access of PENDING
> +# pages. Some guest OS (e.g., Linux TD guest) may require this to
> +# be set, otherwise they refuse to boot.
> +#
> # Since: 9.0
> ##
> { 'struct': 'TdxGuestProperties',
> - 'data': { }}
> + 'data': { '*sept-ve-disable': 'bool' } }
So this exposes a single boolean property that gets mapped into one
specific bit in the TD attributes:
> +
> +static void tdx_guest_set_sept_ve_disable(Object *obj, bool value, Error **errp)
> +{
> + TdxGuest *tdx = TDX_GUEST(obj);
> +
> + if (value) {
> + tdx->attributes |= TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE;
> + } else {
> + tdx->attributes &= ~TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE;
> + }
> +}
If I look at the documentation for TD attributes
https://download.01.org/intel-sgx/latest/dcap-latest/linux/docs/Intel_TDX_DCAP_Quoting_Library_API.pdf
Section "A.3.4. TD Attributes"
I see "TD attributes" is a 64-bit int, with 5 bits currently
defined "DEBUG", "SEPT_VE_DISABLE", "PKS", "PL", "PERFMON",
and the rest currently reserved for future use. This makes me
wonder about our modelling approach into the future ?
For the AMD SEV equivalent we've just directly exposed the whole
field as an int:
'policy' : 'uint32',
For the proposed SEV-SNP patches, the same has been done again
https://lists.nongnu.org/archive/html/qemu-devel/2024-06/msg00536.html
'*policy': 'uint64',
The advantage of exposing individual booleans is that it is
self-documenting at the QAPI level, but the disadvantage is
that every time we want to expose ability to control a new
bit in the policy we have to modify QEMU, libvirt, the mgmt
app above libvirt, and whatever tools the end user has to
talk to the mgmt app.
If we expose a policy int, then newly defined bits only require
a change in QEMU, and everything above QEMU will already be
capable of setting it.
In fact if I look at the proposed libvirt patches, they have
proposed just exposing a policy "int" field in the XML, which
then has to be unpacked to set the individual QAPI booleans
https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/message/WXWXEESYUA77DP7YIBP55T2OPSVKV5QW/
On balance, I think it would be better if QEMU just exposed
the raw TD attributes policy as an uint64 at QAPI, instead
of trying to unpack it to discrete bool fields. This gives
consistency with SEV and SEV-SNP, and with what's proposed
at the libvirt level, and minimizes future changes when
more policy bits are defined.
> +
> /* tdx guest */
> OBJECT_DEFINE_TYPE_WITH_INTERFACES(TdxGuest,
> tdx_guest,
> @@ -529,6 +549,10 @@ static void tdx_guest_init(Object *obj)
> qemu_mutex_init(&tdx->lock);
>
> tdx->attributes = 0;
> +
> + object_property_add_bool(obj, "sept-ve-disable",
> + tdx_guest_get_sept_ve_disable,
> + tdx_guest_set_sept_ve_disable);
> }
>
> static void tdx_guest_finalize(Object *obj)
> --
> 2.34.1
>
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 25/65] i386/tdx: Add property sept-ve-disable for tdx-guest object
2024-06-06 10:45 ` Daniel P. Berrangé
@ 2024-06-12 8:28 ` Xiaoyao Li
2024-06-13 8:35 ` Duan, Zhenzhong
0 siblings, 1 reply; 124+ messages in thread
From: Xiaoyao Li @ 2024-06-12 8:28 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Eric Blake, Markus Armbruster, Marcelo Tosatti,
kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, Zhenzhong Duan
On 6/6/2024 6:45 PM, Daniel P. Berrangé wrote:
> Copying Zhenzhong Duan as my point relates to the proposed libvirt
> TDX patches.
>
> On Thu, Feb 29, 2024 at 01:36:46AM -0500, Xiaoyao Li wrote:
>> Bit 28 of TD attribute, named SEPT_VE_DISABLE. When set to 1, it disables
>> EPT violation conversion to #VE on guest TD access of PENDING pages.
>>
>> Some guest OS (e.g., Linux TD guest) may require this bit as 1.
>> Otherwise refuse to boot.
>>
>> Add sept-ve-disable property for tdx-guest object, for user to configure
>> this bit.
>>
>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
>> Acked-by: Markus Armbruster <armbru@redhat.com>
>> ---
>> Changes in v4:
>> - collect Acked-by from Markus
>>
>> Changes in v3:
>> - update the comment of property @sept-ve-disable to make it more
>> descriptive and use new format. (Daniel and Markus)
>> ---
>> qapi/qom.json | 7 ++++++-
>> target/i386/kvm/tdx.c | 24 ++++++++++++++++++++++++
>> 2 files changed, 30 insertions(+), 1 deletion(-)
>>
>> diff --git a/qapi/qom.json b/qapi/qom.json
>> index 220cc6c98d4b..89ed89b9b46e 100644
>> --- a/qapi/qom.json
>> +++ b/qapi/qom.json
>> @@ -900,10 +900,15 @@
>> #
>> # Properties for tdx-guest objects.
>> #
>> +# @sept-ve-disable: toggle bit 28 of TD attributes to control disabling
>> +# of EPT violation conversion to #VE on guest TD access of PENDING
>> +# pages. Some guest OS (e.g., Linux TD guest) may require this to
>> +# be set, otherwise they refuse to boot.
>> +#
>> # Since: 9.0
>> ##
>> { 'struct': 'TdxGuestProperties',
>> - 'data': { }}
>> + 'data': { '*sept-ve-disable': 'bool' } }
>
> So this exposes a single boolean property that gets mapped into one
> specific bit in the TD attributes:
>
>> +
>> +static void tdx_guest_set_sept_ve_disable(Object *obj, bool value, Error **errp)
>> +{
>> + TdxGuest *tdx = TDX_GUEST(obj);
>> +
>> + if (value) {
>> + tdx->attributes |= TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE;
>> + } else {
>> + tdx->attributes &= ~TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE;
>> + }
>> +}
>
> If I look at the documentation for TD attributes
>
> https://download.01.org/intel-sgx/latest/dcap-latest/linux/docs/Intel_TDX_DCAP_Quoting_Library_API.pdf
>
> Section "A.3.4. TD Attributes"
>
> I see "TD attributes" is a 64-bit int, with 5 bits currently
> defined "DEBUG", "SEPT_VE_DISABLE", "PKS", "PL", "PERFMON",
> and the rest currently reserved for future use. This makes me
> wonder about our modelling approach into the future ?
>
> For the AMD SEV equivalent we've just directly exposed the whole
> field as an int:
>
> 'policy' : 'uint32',
>
> For the proposed SEV-SNP patches, the same has been done again
>
> https://lists.nongnu.org/archive/html/qemu-devel/2024-06/msg00536.html
>
> '*policy': 'uint64',
>
>
> The advantage of exposing individual booleans is that it is
> self-documenting at the QAPI level, but the disadvantage is
> that every time we want to expose ability to control a new
> bit in the policy we have to modify QEMU, libvirt, the mgmt
> app above libvirt, and whatever tools the end user has to
> talk to the mgmt app.
>
> If we expose a policy int, then newly defined bits only require
> a change in QEMU, and everything above QEMU will already be
> capable of setting it.
>
> In fact if I look at the proposed libvirt patches, they have
> proposed just exposing a policy "int" field in the XML, which
> then has to be unpacked to set the individual QAPI booleans
>
> https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/message/WXWXEESYUA77DP7YIBP55T2OPSVKV5QW/
>
> On balance, I think it would be better if QEMU just exposed
> the raw TD attributes policy as an uint64 at QAPI, instead
> of trying to unpack it to discrete bool fields. This gives
> consistency with SEV and SEV-SNP, and with what's proposed
> at the libvirt level, and minimizes future changes when
> more policy bits are defined.
The reasons why introducing individual bit of sept-ve-disable instead of
a raw TD attribute as a whole are that
1. other bits like perfmon, PKS, KL are associated with cpu properties,
e.g.,
perfmon -> pmu,
pks -> pks,
kl -> keylokcer feature that QEMU currently doesn't support
If allowing configuring attribute directly, we need to deal with the
inconsistence between attribute vs cpu property.
2. people need to know the exact bit position of each attribute. I don't
think it is a user-friendly interface to require user to be aware of
such details.
For example, if user wants to create a Debug TD, user just needs to set
'debug=on' for tdx-guest object. It's much more friendly than that user
needs to set the bit 0 of the attribute.
>> +
>> /* tdx guest */
>> OBJECT_DEFINE_TYPE_WITH_INTERFACES(TdxGuest,
>> tdx_guest,
>> @@ -529,6 +549,10 @@ static void tdx_guest_init(Object *obj)
>> qemu_mutex_init(&tdx->lock);
>>
>> tdx->attributes = 0;
>> +
>> + object_property_add_bool(obj, "sept-ve-disable",
>> + tdx_guest_get_sept_ve_disable,
>> + tdx_guest_set_sept_ve_disable);
>> }
>>
>> static void tdx_guest_finalize(Object *obj)
>> --
>> 2.34.1
>>
>
> With regards,
> Daniel
^ permalink raw reply [flat|nested] 124+ messages in thread* RE: [PATCH v5 25/65] i386/tdx: Add property sept-ve-disable for tdx-guest object
2024-06-12 8:28 ` Xiaoyao Li
@ 2024-06-13 8:35 ` Duan, Zhenzhong
2024-06-14 1:04 ` Xiaoyao Li
0 siblings, 1 reply; 124+ messages in thread
From: Duan, Zhenzhong @ 2024-06-13 8:35 UTC (permalink / raw)
To: Li, Xiaoyao, Daniel P. Berrangé
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Eric Blake, Markus Armbruster, Marcelo Tosatti,
kvm@vger.kernel.org, qemu-devel@nongnu.org, Michael Roth,
Claudio Fontana, Gerd Hoffmann, Isaku Yamahata, Qiang, Chenyi
>-----Original Message-----
>From: Li, Xiaoyao <xiaoyao.li@intel.com>
>Subject: Re: [PATCH v5 25/65] i386/tdx: Add property sept-ve-disable for
>tdx-guest object
>
>On 6/6/2024 6:45 PM, Daniel P. Berrangé wrote:
>> Copying Zhenzhong Duan as my point relates to the proposed libvirt
>> TDX patches.
>>
>> On Thu, Feb 29, 2024 at 01:36:46AM -0500, Xiaoyao Li wrote:
>>> Bit 28 of TD attribute, named SEPT_VE_DISABLE. When set to 1, it
>disables
>>> EPT violation conversion to #VE on guest TD access of PENDING pages.
>>>
>>> Some guest OS (e.g., Linux TD guest) may require this bit as 1.
>>> Otherwise refuse to boot.
>>>
>>> Add sept-ve-disable property for tdx-guest object, for user to configure
>>> this bit.
>>>
>>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>>> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
>>> Acked-by: Markus Armbruster <armbru@redhat.com>
>>> ---
>>> Changes in v4:
>>> - collect Acked-by from Markus
>>>
>>> Changes in v3:
>>> - update the comment of property @sept-ve-disable to make it more
>>> descriptive and use new format. (Daniel and Markus)
>>> ---
>>> qapi/qom.json | 7 ++++++-
>>> target/i386/kvm/tdx.c | 24 ++++++++++++++++++++++++
>>> 2 files changed, 30 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/qapi/qom.json b/qapi/qom.json
>>> index 220cc6c98d4b..89ed89b9b46e 100644
>>> --- a/qapi/qom.json
>>> +++ b/qapi/qom.json
>>> @@ -900,10 +900,15 @@
>>> #
>>> # Properties for tdx-guest objects.
>>> #
>>> +# @sept-ve-disable: toggle bit 28 of TD attributes to control disabling
>>> +# of EPT violation conversion to #VE on guest TD access of PENDING
>>> +# pages. Some guest OS (e.g., Linux TD guest) may require this to
>>> +# be set, otherwise they refuse to boot.
>>> +#
>>> # Since: 9.0
>>> ##
>>> { 'struct': 'TdxGuestProperties',
>>> - 'data': { }}
>>> + 'data': { '*sept-ve-disable': 'bool' } }
>>
>> So this exposes a single boolean property that gets mapped into one
>> specific bit in the TD attributes:
>>
>>> +
>>> +static void tdx_guest_set_sept_ve_disable(Object *obj, bool value, Error
>**errp)
>>> +{
>>> + TdxGuest *tdx = TDX_GUEST(obj);
>>> +
>>> + if (value) {
>>> + tdx->attributes |= TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE;
>>> + } else {
>>> + tdx->attributes &= ~TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE;
>>> + }
>>> +}
>>
>> If I look at the documentation for TD attributes
>>
>> https://download.01.org/intel-sgx/latest/dcap-
>latest/linux/docs/Intel_TDX_DCAP_Quoting_Library_API.pdf
>>
>> Section "A.3.4. TD Attributes"
>>
>> I see "TD attributes" is a 64-bit int, with 5 bits currently
>> defined "DEBUG", "SEPT_VE_DISABLE", "PKS", "PL", "PERFMON",
>> and the rest currently reserved for future use. This makes me
>> wonder about our modelling approach into the future ?
>>
>> For the AMD SEV equivalent we've just directly exposed the whole
>> field as an int:
>>
>> 'policy' : 'uint32',
>>
>> For the proposed SEV-SNP patches, the same has been done again
>>
>> https://lists.nongnu.org/archive/html/qemu-devel/2024-
>06/msg00536.html
>>
>> '*policy': 'uint64',
>>
>>
>> The advantage of exposing individual booleans is that it is
>> self-documenting at the QAPI level, but the disadvantage is
>> that every time we want to expose ability to control a new
>> bit in the policy we have to modify QEMU, libvirt, the mgmt
>> app above libvirt, and whatever tools the end user has to
>> talk to the mgmt app.
>>
>> If we expose a policy int, then newly defined bits only require
>> a change in QEMU, and everything above QEMU will already be
>> capable of setting it.
>>
>> In fact if I look at the proposed libvirt patches, they have
>> proposed just exposing a policy "int" field in the XML, which
>> then has to be unpacked to set the individual QAPI booleans
>>
>>
>https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/message/WXWX
>EESYUA77DP7YIBP55T2OPSVKV5QW/
>>
>> On balance, I think it would be better if QEMU just exposed
>> the raw TD attributes policy as an uint64 at QAPI, instead
>> of trying to unpack it to discrete bool fields. This gives
>> consistency with SEV and SEV-SNP, and with what's proposed
>> at the libvirt level, and minimizes future changes when
>> more policy bits are defined.
>
>The reasons why introducing individual bit of sept-ve-disable instead of
>a raw TD attribute as a whole are that
>
>1. other bits like perfmon, PKS, KL are associated with cpu properties,
>e.g.,
>
> perfmon -> pmu,
> pks -> pks,
> kl -> keylokcer feature that QEMU currently doesn't support
>
>If allowing configuring attribute directly, we need to deal with the
>inconsistence between attribute vs cpu property.
What about defining those bits associated with cpu properties reserved
But other bits work as Daniel suggested way.
Thanks
Zhenzhong
>
>2. people need to know the exact bit position of each attribute. I don't
>think it is a user-friendly interface to require user to be aware of
>such details.
>
>For example, if user wants to create a Debug TD, user just needs to set
>'debug=on' for tdx-guest object. It's much more friendly than that user
>needs to set the bit 0 of the attribute.
>
>
>>> +
>>> /* tdx guest */
>>> OBJECT_DEFINE_TYPE_WITH_INTERFACES(TdxGuest,
>>> tdx_guest,
>>> @@ -529,6 +549,10 @@ static void tdx_guest_init(Object *obj)
>>> qemu_mutex_init(&tdx->lock);
>>>
>>> tdx->attributes = 0;
>>> +
>>> + object_property_add_bool(obj, "sept-ve-disable",
>>> + tdx_guest_get_sept_ve_disable,
>>> + tdx_guest_set_sept_ve_disable);
>>> }
>>>
>>> static void tdx_guest_finalize(Object *obj)
>>> --
>>> 2.34.1
>>>
>>
>> With regards,
>> Daniel
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 25/65] i386/tdx: Add property sept-ve-disable for tdx-guest object
2024-06-13 8:35 ` Duan, Zhenzhong
@ 2024-06-14 1:04 ` Xiaoyao Li
2024-06-14 2:49 ` Duan, Zhenzhong
2024-06-14 7:49 ` Daniel P. Berrangé
0 siblings, 2 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-06-14 1:04 UTC (permalink / raw)
To: Duan, Zhenzhong, Daniel P. Berrangé
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Eric Blake, Markus Armbruster, Marcelo Tosatti,
kvm@vger.kernel.org, qemu-devel@nongnu.org, Michael Roth,
Claudio Fontana, Gerd Hoffmann, Isaku Yamahata, Qiang, Chenyi
On 6/13/2024 4:35 PM, Duan, Zhenzhong wrote:
>
>
>> -----Original Message-----
>> From: Li, Xiaoyao <xiaoyao.li@intel.com>
>> Subject: Re: [PATCH v5 25/65] i386/tdx: Add property sept-ve-disable for
>> tdx-guest object
>>
>> On 6/6/2024 6:45 PM, Daniel P. Berrangé wrote:
>>> Copying Zhenzhong Duan as my point relates to the proposed libvirt
>>> TDX patches.
>>>
>>> On Thu, Feb 29, 2024 at 01:36:46AM -0500, Xiaoyao Li wrote:
>>>> Bit 28 of TD attribute, named SEPT_VE_DISABLE. When set to 1, it
>> disables
>>>> EPT violation conversion to #VE on guest TD access of PENDING pages.
>>>>
>>>> Some guest OS (e.g., Linux TD guest) may require this bit as 1.
>>>> Otherwise refuse to boot.
>>>>
>>>> Add sept-ve-disable property for tdx-guest object, for user to configure
>>>> this bit.
>>>>
>>>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>>>> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
>>>> Acked-by: Markus Armbruster <armbru@redhat.com>
>>>> ---
>>>> Changes in v4:
>>>> - collect Acked-by from Markus
>>>>
>>>> Changes in v3:
>>>> - update the comment of property @sept-ve-disable to make it more
>>>> descriptive and use new format. (Daniel and Markus)
>>>> ---
>>>> qapi/qom.json | 7 ++++++-
>>>> target/i386/kvm/tdx.c | 24 ++++++++++++++++++++++++
>>>> 2 files changed, 30 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/qapi/qom.json b/qapi/qom.json
>>>> index 220cc6c98d4b..89ed89b9b46e 100644
>>>> --- a/qapi/qom.json
>>>> +++ b/qapi/qom.json
>>>> @@ -900,10 +900,15 @@
>>>> #
>>>> # Properties for tdx-guest objects.
>>>> #
>>>> +# @sept-ve-disable: toggle bit 28 of TD attributes to control disabling
>>>> +# of EPT violation conversion to #VE on guest TD access of PENDING
>>>> +# pages. Some guest OS (e.g., Linux TD guest) may require this to
>>>> +# be set, otherwise they refuse to boot.
>>>> +#
>>>> # Since: 9.0
>>>> ##
>>>> { 'struct': 'TdxGuestProperties',
>>>> - 'data': { }}
>>>> + 'data': { '*sept-ve-disable': 'bool' } }
>>>
>>> So this exposes a single boolean property that gets mapped into one
>>> specific bit in the TD attributes:
>>>
>>>> +
>>>> +static void tdx_guest_set_sept_ve_disable(Object *obj, bool value, Error
>> **errp)
>>>> +{
>>>> + TdxGuest *tdx = TDX_GUEST(obj);
>>>> +
>>>> + if (value) {
>>>> + tdx->attributes |= TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE;
>>>> + } else {
>>>> + tdx->attributes &= ~TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE;
>>>> + }
>>>> +}
>>>
>>> If I look at the documentation for TD attributes
>>>
>>> https://download.01.org/intel-sgx/latest/dcap-
>> latest/linux/docs/Intel_TDX_DCAP_Quoting_Library_API.pdf
>>>
>>> Section "A.3.4. TD Attributes"
>>>
>>> I see "TD attributes" is a 64-bit int, with 5 bits currently
>>> defined "DEBUG", "SEPT_VE_DISABLE", "PKS", "PL", "PERFMON",
>>> and the rest currently reserved for future use. This makes me
>>> wonder about our modelling approach into the future ?
>>>
>>> For the AMD SEV equivalent we've just directly exposed the whole
>>> field as an int:
>>>
>>> 'policy' : 'uint32',
>>>
>>> For the proposed SEV-SNP patches, the same has been done again
>>>
>>> https://lists.nongnu.org/archive/html/qemu-devel/2024-
>> 06/msg00536.html
>>>
>>> '*policy': 'uint64',
>>>
>>>
>>> The advantage of exposing individual booleans is that it is
>>> self-documenting at the QAPI level, but the disadvantage is
>>> that every time we want to expose ability to control a new
>>> bit in the policy we have to modify QEMU, libvirt, the mgmt
>>> app above libvirt, and whatever tools the end user has to
>>> talk to the mgmt app.
>>>
>>> If we expose a policy int, then newly defined bits only require
>>> a change in QEMU, and everything above QEMU will already be
>>> capable of setting it.
>>>
>>> In fact if I look at the proposed libvirt patches, they have
>>> proposed just exposing a policy "int" field in the XML, which
>>> then has to be unpacked to set the individual QAPI booleans
>>>
>>>
>> https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/message/WXWX
>> EESYUA77DP7YIBP55T2OPSVKV5QW/
>>>
>>> On balance, I think it would be better if QEMU just exposed
>>> the raw TD attributes policy as an uint64 at QAPI, instead
>>> of trying to unpack it to discrete bool fields. This gives
>>> consistency with SEV and SEV-SNP, and with what's proposed
>>> at the libvirt level, and minimizes future changes when
>>> more policy bits are defined.
>>
>> The reasons why introducing individual bit of sept-ve-disable instead of
>> a raw TD attribute as a whole are that
>>
>> 1. other bits like perfmon, PKS, KL are associated with cpu properties,
>> e.g.,
>>
>> perfmon -> pmu,
>> pks -> pks,
>> kl -> keylokcer feature that QEMU currently doesn't support
>>
>> If allowing configuring attribute directly, we need to deal with the
>> inconsistence between attribute vs cpu property.
>
> What about defining those bits associated with cpu properties reserved
> But other bits work as Daniel suggested way.
I don't understand. Do you mean we provide the interface to configure
raw 64 bit attributes while some bits of it are reserved?
> Thanks
> Zhenzhong
>
>>
>> 2. people need to know the exact bit position of each attribute. I don't
>> think it is a user-friendly interface to require user to be aware of
>> such details.
>>
>> For example, if user wants to create a Debug TD, user just needs to set
>> 'debug=on' for tdx-guest object. It's much more friendly than that user
>> needs to set the bit 0 of the attribute.
>>
>>
>>>> +
>>>> /* tdx guest */
>>>> OBJECT_DEFINE_TYPE_WITH_INTERFACES(TdxGuest,
>>>> tdx_guest,
>>>> @@ -529,6 +549,10 @@ static void tdx_guest_init(Object *obj)
>>>> qemu_mutex_init(&tdx->lock);
>>>>
>>>> tdx->attributes = 0;
>>>> +
>>>> + object_property_add_bool(obj, "sept-ve-disable",
>>>> + tdx_guest_get_sept_ve_disable,
>>>> + tdx_guest_set_sept_ve_disable);
>>>> }
>>>>
>>>> static void tdx_guest_finalize(Object *obj)
>>>> --
>>>> 2.34.1
>>>>
>>>
>>> With regards,
>>> Daniel
>
^ permalink raw reply [flat|nested] 124+ messages in thread* RE: [PATCH v5 25/65] i386/tdx: Add property sept-ve-disable for tdx-guest object
2024-06-14 1:04 ` Xiaoyao Li
@ 2024-06-14 2:49 ` Duan, Zhenzhong
2024-06-14 7:49 ` Daniel P. Berrangé
1 sibling, 0 replies; 124+ messages in thread
From: Duan, Zhenzhong @ 2024-06-14 2:49 UTC (permalink / raw)
To: Li, Xiaoyao, Daniel P. Berrangé
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Eric Blake, Markus Armbruster, Marcelo Tosatti,
kvm@vger.kernel.org, qemu-devel@nongnu.org, Michael Roth,
Claudio Fontana, Gerd Hoffmann, Isaku Yamahata, Qiang, Chenyi
>-----Original Message-----
>From: Li, Xiaoyao <xiaoyao.li@intel.com>
>Subject: Re: [PATCH v5 25/65] i386/tdx: Add property sept-ve-disable for
>tdx-guest object
>
>On 6/13/2024 4:35 PM, Duan, Zhenzhong wrote:
>>
>>
>>> -----Original Message-----
>>> From: Li, Xiaoyao <xiaoyao.li@intel.com>
>>> Subject: Re: [PATCH v5 25/65] i386/tdx: Add property sept-ve-disable for
>>> tdx-guest object
>>>
>>> On 6/6/2024 6:45 PM, Daniel P. Berrangé wrote:
>>>> Copying Zhenzhong Duan as my point relates to the proposed libvirt
>>>> TDX patches.
>>>>
>>>> On Thu, Feb 29, 2024 at 01:36:46AM -0500, Xiaoyao Li wrote:
>>>>> Bit 28 of TD attribute, named SEPT_VE_DISABLE. When set to 1, it
>>> disables
>>>>> EPT violation conversion to #VE on guest TD access of PENDING pages.
>>>>>
>>>>> Some guest OS (e.g., Linux TD guest) may require this bit as 1.
>>>>> Otherwise refuse to boot.
>>>>>
>>>>> Add sept-ve-disable property for tdx-guest object, for user to configure
>>>>> this bit.
>>>>>
>>>>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>>>>> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
>>>>> Acked-by: Markus Armbruster <armbru@redhat.com>
>>>>> ---
>>>>> Changes in v4:
>>>>> - collect Acked-by from Markus
>>>>>
>>>>> Changes in v3:
>>>>> - update the comment of property @sept-ve-disable to make it more
>>>>> descriptive and use new format. (Daniel and Markus)
>>>>> ---
>>>>> qapi/qom.json | 7 ++++++-
>>>>> target/i386/kvm/tdx.c | 24 ++++++++++++++++++++++++
>>>>> 2 files changed, 30 insertions(+), 1 deletion(-)
>>>>>
>>>>> diff --git a/qapi/qom.json b/qapi/qom.json
>>>>> index 220cc6c98d4b..89ed89b9b46e 100644
>>>>> --- a/qapi/qom.json
>>>>> +++ b/qapi/qom.json
>>>>> @@ -900,10 +900,15 @@
>>>>> #
>>>>> # Properties for tdx-guest objects.
>>>>> #
>>>>> +# @sept-ve-disable: toggle bit 28 of TD attributes to control disabling
>>>>> +# of EPT violation conversion to #VE on guest TD access of PENDING
>>>>> +# pages. Some guest OS (e.g., Linux TD guest) may require this to
>>>>> +# be set, otherwise they refuse to boot.
>>>>> +#
>>>>> # Since: 9.0
>>>>> ##
>>>>> { 'struct': 'TdxGuestProperties',
>>>>> - 'data': { }}
>>>>> + 'data': { '*sept-ve-disable': 'bool' } }
>>>>
>>>> So this exposes a single boolean property that gets mapped into one
>>>> specific bit in the TD attributes:
>>>>
>>>>> +
>>>>> +static void tdx_guest_set_sept_ve_disable(Object *obj, bool value,
>Error
>>> **errp)
>>>>> +{
>>>>> + TdxGuest *tdx = TDX_GUEST(obj);
>>>>> +
>>>>> + if (value) {
>>>>> + tdx->attributes |= TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE;
>>>>> + } else {
>>>>> + tdx->attributes &= ~TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE;
>>>>> + }
>>>>> +}
>>>>
>>>> If I look at the documentation for TD attributes
>>>>
>>>> https://download.01.org/intel-sgx/latest/dcap-
>>> latest/linux/docs/Intel_TDX_DCAP_Quoting_Library_API.pdf
>>>>
>>>> Section "A.3.4. TD Attributes"
>>>>
>>>> I see "TD attributes" is a 64-bit int, with 5 bits currently
>>>> defined "DEBUG", "SEPT_VE_DISABLE", "PKS", "PL", "PERFMON",
>>>> and the rest currently reserved for future use. This makes me
>>>> wonder about our modelling approach into the future ?
>>>>
>>>> For the AMD SEV equivalent we've just directly exposed the whole
>>>> field as an int:
>>>>
>>>> 'policy' : 'uint32',
>>>>
>>>> For the proposed SEV-SNP patches, the same has been done again
>>>>
>>>> https://lists.nongnu.org/archive/html/qemu-devel/2024-
>>> 06/msg00536.html
>>>>
>>>> '*policy': 'uint64',
>>>>
>>>>
>>>> The advantage of exposing individual booleans is that it is
>>>> self-documenting at the QAPI level, but the disadvantage is
>>>> that every time we want to expose ability to control a new
>>>> bit in the policy we have to modify QEMU, libvirt, the mgmt
>>>> app above libvirt, and whatever tools the end user has to
>>>> talk to the mgmt app.
>>>>
>>>> If we expose a policy int, then newly defined bits only require
>>>> a change in QEMU, and everything above QEMU will already be
>>>> capable of setting it.
>>>>
>>>> In fact if I look at the proposed libvirt patches, they have
>>>> proposed just exposing a policy "int" field in the XML, which
>>>> then has to be unpacked to set the individual QAPI booleans
>>>>
>>>>
>>>
>https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/message/WXWX
>>> EESYUA77DP7YIBP55T2OPSVKV5QW/
>>>>
>>>> On balance, I think it would be better if QEMU just exposed
>>>> the raw TD attributes policy as an uint64 at QAPI, instead
>>>> of trying to unpack it to discrete bool fields. This gives
>>>> consistency with SEV and SEV-SNP, and with what's proposed
>>>> at the libvirt level, and minimizes future changes when
>>>> more policy bits are defined.
>>>
>>> The reasons why introducing individual bit of sept-ve-disable instead of
>>> a raw TD attribute as a whole are that
>>>
>>> 1. other bits like perfmon, PKS, KL are associated with cpu properties,
>>> e.g.,
>>>
>>> perfmon -> pmu,
>>> pks -> pks,
>>> kl -> keylokcer feature that QEMU currently doesn't support
>>>
>>> If allowing configuring attribute directly, we need to deal with the
>>> inconsistence between attribute vs cpu property.
>>
>> What about defining those bits associated with cpu properties reserved
>> But other bits work as Daniel suggested way.
>
>I don't understand. Do you mean we provide the interface to configure
>raw 64 bit attributes while some bits of it are reserved?
Yes, qemu provide raw 64bit attribute but ignore those cpuid related bits
to avoid config conflict.
You can still provide sept-ve-disable and debug properties in qemu if you want,
But Libvirt will not use them, it will use the raw 64bit attribute.
Thanks
Zhenzhong
>
>> Thanks
>> Zhenzhong
>>
>>>
>>> 2. people need to know the exact bit position of each attribute. I don't
>>> think it is a user-friendly interface to require user to be aware of
>>> such details.
>>>
>>> For example, if user wants to create a Debug TD, user just needs to set
>>> 'debug=on' for tdx-guest object. It's much more friendly than that user
>>> needs to set the bit 0 of the attribute.
>>>
>>>
>>>>> +
>>>>> /* tdx guest */
>>>>> OBJECT_DEFINE_TYPE_WITH_INTERFACES(TdxGuest,
>>>>> tdx_guest,
>>>>> @@ -529,6 +549,10 @@ static void tdx_guest_init(Object *obj)
>>>>> qemu_mutex_init(&tdx->lock);
>>>>>
>>>>> tdx->attributes = 0;
>>>>> +
>>>>> + object_property_add_bool(obj, "sept-ve-disable",
>>>>> + tdx_guest_get_sept_ve_disable,
>>>>> + tdx_guest_set_sept_ve_disable);
>>>>> }
>>>>>
>>>>> static void tdx_guest_finalize(Object *obj)
>>>>> --
>>>>> 2.34.1
>>>>>
>>>>
>>>> With regards,
>>>> Daniel
>>
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 25/65] i386/tdx: Add property sept-ve-disable for tdx-guest object
2024-06-14 1:04 ` Xiaoyao Li
2024-06-14 2:49 ` Duan, Zhenzhong
@ 2024-06-14 7:49 ` Daniel P. Berrangé
2024-06-24 15:01 ` Daniel P. Berrangé
1 sibling, 1 reply; 124+ messages in thread
From: Daniel P. Berrangé @ 2024-06-14 7:49 UTC (permalink / raw)
To: Xiaoyao Li
Cc: Duan, Zhenzhong, Paolo Bonzini, David Hildenbrand, Igor Mammedov,
Eduardo Habkost, Marcel Apfelbaum, Philippe Mathieu-Daudé,
Yanan Wang, Michael S. Tsirkin, Richard Henderson, Ani Sinha,
Peter Xu, Cornelia Huck, Eric Blake, Markus Armbruster,
Marcelo Tosatti, kvm@vger.kernel.org, qemu-devel@nongnu.org,
Michael Roth, Claudio Fontana, Gerd Hoffmann, Isaku Yamahata,
Qiang, Chenyi
On Fri, Jun 14, 2024 at 09:04:33AM +0800, Xiaoyao Li wrote:
> On 6/13/2024 4:35 PM, Duan, Zhenzhong wrote:
> >
> >
> > > -----Original Message-----
> > > From: Li, Xiaoyao <xiaoyao.li@intel.com>
> > > Subject: Re: [PATCH v5 25/65] i386/tdx: Add property sept-ve-disable for
> > > tdx-guest object
> > >
> > > On 6/6/2024 6:45 PM, Daniel P. Berrangé wrote:
> > > > Copying Zhenzhong Duan as my point relates to the proposed libvirt
> > > > TDX patches.
> > > >
> > > > On Thu, Feb 29, 2024 at 01:36:46AM -0500, Xiaoyao Li wrote:
> > > > > Bit 28 of TD attribute, named SEPT_VE_DISABLE. When set to 1, it
> > > disables
> > > > > EPT violation conversion to #VE on guest TD access of PENDING pages.
> > > > >
> > > > > Some guest OS (e.g., Linux TD guest) may require this bit as 1.
> > > > > Otherwise refuse to boot.
> > > > >
> > > > > Add sept-ve-disable property for tdx-guest object, for user to configure
> > > > > this bit.
> > > > >
> > > > > Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> > > > > Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> > > > > Acked-by: Markus Armbruster <armbru@redhat.com>
> > > > > ---
> > > > > Changes in v4:
> > > > > - collect Acked-by from Markus
> > > > >
> > > > > Changes in v3:
> > > > > - update the comment of property @sept-ve-disable to make it more
> > > > > descriptive and use new format. (Daniel and Markus)
> > > > > ---
> > > > > qapi/qom.json | 7 ++++++-
> > > > > target/i386/kvm/tdx.c | 24 ++++++++++++++++++++++++
> > > > > 2 files changed, 30 insertions(+), 1 deletion(-)
> > > > >
> > > > > diff --git a/qapi/qom.json b/qapi/qom.json
> > > > > index 220cc6c98d4b..89ed89b9b46e 100644
> > > > > --- a/qapi/qom.json
> > > > > +++ b/qapi/qom.json
> > > > > @@ -900,10 +900,15 @@
> > > > > #
> > > > > # Properties for tdx-guest objects.
> > > > > #
> > > > > +# @sept-ve-disable: toggle bit 28 of TD attributes to control disabling
> > > > > +# of EPT violation conversion to #VE on guest TD access of PENDING
> > > > > +# pages. Some guest OS (e.g., Linux TD guest) may require this to
> > > > > +# be set, otherwise they refuse to boot.
> > > > > +#
> > > > > # Since: 9.0
> > > > > ##
> > > > > { 'struct': 'TdxGuestProperties',
> > > > > - 'data': { }}
> > > > > + 'data': { '*sept-ve-disable': 'bool' } }
> > > >
> > > > So this exposes a single boolean property that gets mapped into one
> > > > specific bit in the TD attributes:
> > > >
> > > > > +
> > > > > +static void tdx_guest_set_sept_ve_disable(Object *obj, bool value, Error
> > > **errp)
> > > > > +{
> > > > > + TdxGuest *tdx = TDX_GUEST(obj);
> > > > > +
> > > > > + if (value) {
> > > > > + tdx->attributes |= TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE;
> > > > > + } else {
> > > > > + tdx->attributes &= ~TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE;
> > > > > + }
> > > > > +}
> > > >
> > > > If I look at the documentation for TD attributes
> > > >
> > > > https://download.01.org/intel-sgx/latest/dcap-
> > > latest/linux/docs/Intel_TDX_DCAP_Quoting_Library_API.pdf
> > > >
> > > > Section "A.3.4. TD Attributes"
> > > >
> > > > I see "TD attributes" is a 64-bit int, with 5 bits currently
> > > > defined "DEBUG", "SEPT_VE_DISABLE", "PKS", "PL", "PERFMON",
> > > > and the rest currently reserved for future use. This makes me
> > > > wonder about our modelling approach into the future ?
> > > >
> > > > For the AMD SEV equivalent we've just directly exposed the whole
> > > > field as an int:
> > > >
> > > > 'policy' : 'uint32',
> > > >
> > > > For the proposed SEV-SNP patches, the same has been done again
> > > >
> > > > https://lists.nongnu.org/archive/html/qemu-devel/2024-
> > > 06/msg00536.html
> > > >
> > > > '*policy': 'uint64',
> > > >
> > > >
> > > > The advantage of exposing individual booleans is that it is
> > > > self-documenting at the QAPI level, but the disadvantage is
> > > > that every time we want to expose ability to control a new
> > > > bit in the policy we have to modify QEMU, libvirt, the mgmt
> > > > app above libvirt, and whatever tools the end user has to
> > > > talk to the mgmt app.
> > > >
> > > > If we expose a policy int, then newly defined bits only require
> > > > a change in QEMU, and everything above QEMU will already be
> > > > capable of setting it.
> > > >
> > > > In fact if I look at the proposed libvirt patches, they have
> > > > proposed just exposing a policy "int" field in the XML, which
> > > > then has to be unpacked to set the individual QAPI booleans
> > > >
> > > >
> > > https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/message/WXWX
> > > EESYUA77DP7YIBP55T2OPSVKV5QW/
> > > >
> > > > On balance, I think it would be better if QEMU just exposed
> > > > the raw TD attributes policy as an uint64 at QAPI, instead
> > > > of trying to unpack it to discrete bool fields. This gives
> > > > consistency with SEV and SEV-SNP, and with what's proposed
> > > > at the libvirt level, and minimizes future changes when
> > > > more policy bits are defined.
> > >
> > > The reasons why introducing individual bit of sept-ve-disable instead of
> > > a raw TD attribute as a whole are that
> > >
> > > 1. other bits like perfmon, PKS, KL are associated with cpu properties,
> > > e.g.,
> > >
> > > perfmon -> pmu,
> > > pks -> pks,
> > > kl -> keylokcer feature that QEMU currently doesn't support
> > >
> > > If allowing configuring attribute directly, we need to deal with the
> > > inconsistence between attribute vs cpu property.
> >
> > What about defining those bits associated with cpu properties reserved
> > But other bits work as Daniel suggested way.
>
> I don't understand. Do you mean we provide the interface to configure raw 64
> bit attributes while some bits of it are reserved?
Just have a mask of what bits are permitted to be set, and report an
error if the user sets non-permitted bits.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 25/65] i386/tdx: Add property sept-ve-disable for tdx-guest object
2024-06-14 7:49 ` Daniel P. Berrangé
@ 2024-06-24 15:01 ` Daniel P. Berrangé
2024-06-26 14:45 ` Xiaoyao Li
0 siblings, 1 reply; 124+ messages in thread
From: Daniel P. Berrangé @ 2024-06-24 15:01 UTC (permalink / raw)
To: Xiaoyao Li, Duan, Zhenzhong, Paolo Bonzini, David Hildenbrand,
Igor Mammedov, Eduardo Habkost, Marcel Apfelbaum,
Philippe Mathieu-Daudé, Yanan Wang, Michael S. Tsirkin,
Richard Henderson, Ani Sinha, Peter Xu, Cornelia Huck, Eric Blake,
Markus Armbruster, Marcelo Tosatti, kvm@vger.kernel.org,
qemu-devel@nongnu.org, Michael Roth, Claudio Fontana,
Gerd Hoffmann, Isaku Yamahata, Qiang, Chenyi
On Fri, Jun 14, 2024 at 08:49:57AM +0100, Daniel P. Berrangé wrote:
> On Fri, Jun 14, 2024 at 09:04:33AM +0800, Xiaoyao Li wrote:
> > On 6/13/2024 4:35 PM, Duan, Zhenzhong wrote:
> > >
> > >
> > > > -----Original Message-----
> > > > From: Li, Xiaoyao <xiaoyao.li@intel.com>
> > > > Subject: Re: [PATCH v5 25/65] i386/tdx: Add property sept-ve-disable for
> > > > tdx-guest object
> > > >
> > > > On 6/6/2024 6:45 PM, Daniel P. Berrangé wrote:
> > > > > Copying Zhenzhong Duan as my point relates to the proposed libvirt
> > > > > TDX patches.
> > > > >
> > > > > On Thu, Feb 29, 2024 at 01:36:46AM -0500, Xiaoyao Li wrote:
> > > > > > Bit 28 of TD attribute, named SEPT_VE_DISABLE. When set to 1, it
> > > > disables
> > > > > > EPT violation conversion to #VE on guest TD access of PENDING pages.
> > > > > >
> > > > > > Some guest OS (e.g., Linux TD guest) may require this bit as 1.
> > > > > > Otherwise refuse to boot.
> > > > > >
> > > > > > Add sept-ve-disable property for tdx-guest object, for user to configure
> > > > > > this bit.
> > > > > >
> > > > > > Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> > > > > > Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> > > > > > Acked-by: Markus Armbruster <armbru@redhat.com>
> > > > > > ---
> > > > > > Changes in v4:
> > > > > > - collect Acked-by from Markus
> > > > > >
> > > > > > Changes in v3:
> > > > > > - update the comment of property @sept-ve-disable to make it more
> > > > > > descriptive and use new format. (Daniel and Markus)
> > > > > > ---
> > > > > > qapi/qom.json | 7 ++++++-
> > > > > > target/i386/kvm/tdx.c | 24 ++++++++++++++++++++++++
> > > > > > 2 files changed, 30 insertions(+), 1 deletion(-)
> > > > > >
> > > > > > diff --git a/qapi/qom.json b/qapi/qom.json
> > > > > > index 220cc6c98d4b..89ed89b9b46e 100644
> > > > > > --- a/qapi/qom.json
> > > > > > +++ b/qapi/qom.json
> > > > > > @@ -900,10 +900,15 @@
> > > > > > #
> > > > > > # Properties for tdx-guest objects.
> > > > > > #
> > > > > > +# @sept-ve-disable: toggle bit 28 of TD attributes to control disabling
> > > > > > +# of EPT violation conversion to #VE on guest TD access of PENDING
> > > > > > +# pages. Some guest OS (e.g., Linux TD guest) may require this to
> > > > > > +# be set, otherwise they refuse to boot.
> > > > > > +#
> > > > > > # Since: 9.0
> > > > > > ##
> > > > > > { 'struct': 'TdxGuestProperties',
> > > > > > - 'data': { }}
> > > > > > + 'data': { '*sept-ve-disable': 'bool' } }
> > > > >
> > > > > So this exposes a single boolean property that gets mapped into one
> > > > > specific bit in the TD attributes:
> > > > >
> > > > > > +
> > > > > > +static void tdx_guest_set_sept_ve_disable(Object *obj, bool value, Error
> > > > **errp)
> > > > > > +{
> > > > > > + TdxGuest *tdx = TDX_GUEST(obj);
> > > > > > +
> > > > > > + if (value) {
> > > > > > + tdx->attributes |= TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE;
> > > > > > + } else {
> > > > > > + tdx->attributes &= ~TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE;
> > > > > > + }
> > > > > > +}
> > > > >
> > > > > If I look at the documentation for TD attributes
> > > > >
> > > > > https://download.01.org/intel-sgx/latest/dcap-
> > > > latest/linux/docs/Intel_TDX_DCAP_Quoting_Library_API.pdf
> > > > >
> > > > > Section "A.3.4. TD Attributes"
> > > > >
> > > > > I see "TD attributes" is a 64-bit int, with 5 bits currently
> > > > > defined "DEBUG", "SEPT_VE_DISABLE", "PKS", "PL", "PERFMON",
> > > > > and the rest currently reserved for future use. This makes me
> > > > > wonder about our modelling approach into the future ?
> > > > >
> > > > > For the AMD SEV equivalent we've just directly exposed the whole
> > > > > field as an int:
> > > > >
> > > > > 'policy' : 'uint32',
> > > > >
> > > > > For the proposed SEV-SNP patches, the same has been done again
> > > > >
> > > > > https://lists.nongnu.org/archive/html/qemu-devel/2024-
> > > > 06/msg00536.html
> > > > >
> > > > > '*policy': 'uint64',
> > > > >
> > > > >
> > > > > The advantage of exposing individual booleans is that it is
> > > > > self-documenting at the QAPI level, but the disadvantage is
> > > > > that every time we want to expose ability to control a new
> > > > > bit in the policy we have to modify QEMU, libvirt, the mgmt
> > > > > app above libvirt, and whatever tools the end user has to
> > > > > talk to the mgmt app.
> > > > >
> > > > > If we expose a policy int, then newly defined bits only require
> > > > > a change in QEMU, and everything above QEMU will already be
> > > > > capable of setting it.
> > > > >
> > > > > In fact if I look at the proposed libvirt patches, they have
> > > > > proposed just exposing a policy "int" field in the XML, which
> > > > > then has to be unpacked to set the individual QAPI booleans
> > > > >
> > > > >
> > > > https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/message/WXWX
> > > > EESYUA77DP7YIBP55T2OPSVKV5QW/
> > > > >
> > > > > On balance, I think it would be better if QEMU just exposed
> > > > > the raw TD attributes policy as an uint64 at QAPI, instead
> > > > > of trying to unpack it to discrete bool fields. This gives
> > > > > consistency with SEV and SEV-SNP, and with what's proposed
> > > > > at the libvirt level, and minimizes future changes when
> > > > > more policy bits are defined.
> > > >
> > > > The reasons why introducing individual bit of sept-ve-disable instead of
> > > > a raw TD attribute as a whole are that
> > > >
> > > > 1. other bits like perfmon, PKS, KL are associated with cpu properties,
> > > > e.g.,
> > > >
> > > > perfmon -> pmu,
> > > > pks -> pks,
> > > > kl -> keylokcer feature that QEMU currently doesn't support
> > > >
> > > > If allowing configuring attribute directly, we need to deal with the
> > > > inconsistence between attribute vs cpu property.
> > >
> > > What about defining those bits associated with cpu properties reserved
> > > But other bits work as Daniel suggested way.
> >
> > I don't understand. Do you mean we provide the interface to configure raw 64
> > bit attributes while some bits of it are reserved?
>
> Just have a mask of what bits are permitted to be set, and report an
> error if the user sets non-permitted bits.
Looking at the IGVM patches, the CnofidentialGuestSupport class is
gaining a "set_guest_policy" method, which takes a "uint64_t policy".
If (when) IGVM support is extended to cover TDX too, then we're
going to need to accept the user providing the policy in integer
format via the IGVM file metadata. This will require adding code
to check for any inconsistency between the policy bitmask, and
the CPU flags.
https://lists.nongnu.org/archive/html/qemu-devel/2024-06/msg04037.html
so to me this is another reason to just expose the policy as an
integer in the QAPI/QOM structure too. Everywhere just wants to
be working with policy in integer format.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 25/65] i386/tdx: Add property sept-ve-disable for tdx-guest object
2024-06-24 15:01 ` Daniel P. Berrangé
@ 2024-06-26 14:45 ` Xiaoyao Li
0 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-06-26 14:45 UTC (permalink / raw)
To: Daniel P. Berrangé, Duan, Zhenzhong, Paolo Bonzini,
David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Eric Blake, Markus Armbruster, Marcelo Tosatti,
kvm@vger.kernel.org, qemu-devel@nongnu.org, Michael Roth,
Claudio Fontana, Gerd Hoffmann, Isaku Yamahata, Qiang, Chenyi
On 6/24/2024 11:01 PM, Daniel P. Berrangé wrote:
> On Fri, Jun 14, 2024 at 08:49:57AM +0100, Daniel P. Berrangé wrote:
>> On Fri, Jun 14, 2024 at 09:04:33AM +0800, Xiaoyao Li wrote:
>>> On 6/13/2024 4:35 PM, Duan, Zhenzhong wrote:
>>>>
>>>>
>>>>> -----Original Message-----
>>>>> From: Li, Xiaoyao <xiaoyao.li@intel.com>
>>>>> Subject: Re: [PATCH v5 25/65] i386/tdx: Add property sept-ve-disable for
>>>>> tdx-guest object
>>>>>
>>>>> On 6/6/2024 6:45 PM, Daniel P. Berrangé wrote:
>>>>>> Copying Zhenzhong Duan as my point relates to the proposed libvirt
>>>>>> TDX patches.
>>>>>>
>>>>>> On Thu, Feb 29, 2024 at 01:36:46AM -0500, Xiaoyao Li wrote:
>>>>>>> Bit 28 of TD attribute, named SEPT_VE_DISABLE. When set to 1, it
>>>>> disables
>>>>>>> EPT violation conversion to #VE on guest TD access of PENDING pages.
>>>>>>>
>>>>>>> Some guest OS (e.g., Linux TD guest) may require this bit as 1.
>>>>>>> Otherwise refuse to boot.
>>>>>>>
>>>>>>> Add sept-ve-disable property for tdx-guest object, for user to configure
>>>>>>> this bit.
>>>>>>>
>>>>>>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>>>>>>> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
>>>>>>> Acked-by: Markus Armbruster <armbru@redhat.com>
>>>>>>> ---
>>>>>>> Changes in v4:
>>>>>>> - collect Acked-by from Markus
>>>>>>>
>>>>>>> Changes in v3:
>>>>>>> - update the comment of property @sept-ve-disable to make it more
>>>>>>> descriptive and use new format. (Daniel and Markus)
>>>>>>> ---
>>>>>>> qapi/qom.json | 7 ++++++-
>>>>>>> target/i386/kvm/tdx.c | 24 ++++++++++++++++++++++++
>>>>>>> 2 files changed, 30 insertions(+), 1 deletion(-)
>>>>>>>
>>>>>>> diff --git a/qapi/qom.json b/qapi/qom.json
>>>>>>> index 220cc6c98d4b..89ed89b9b46e 100644
>>>>>>> --- a/qapi/qom.json
>>>>>>> +++ b/qapi/qom.json
>>>>>>> @@ -900,10 +900,15 @@
>>>>>>> #
>>>>>>> # Properties for tdx-guest objects.
>>>>>>> #
>>>>>>> +# @sept-ve-disable: toggle bit 28 of TD attributes to control disabling
>>>>>>> +# of EPT violation conversion to #VE on guest TD access of PENDING
>>>>>>> +# pages. Some guest OS (e.g., Linux TD guest) may require this to
>>>>>>> +# be set, otherwise they refuse to boot.
>>>>>>> +#
>>>>>>> # Since: 9.0
>>>>>>> ##
>>>>>>> { 'struct': 'TdxGuestProperties',
>>>>>>> - 'data': { }}
>>>>>>> + 'data': { '*sept-ve-disable': 'bool' } }
>>>>>>
>>>>>> So this exposes a single boolean property that gets mapped into one
>>>>>> specific bit in the TD attributes:
>>>>>>
>>>>>>> +
>>>>>>> +static void tdx_guest_set_sept_ve_disable(Object *obj, bool value, Error
>>>>> **errp)
>>>>>>> +{
>>>>>>> + TdxGuest *tdx = TDX_GUEST(obj);
>>>>>>> +
>>>>>>> + if (value) {
>>>>>>> + tdx->attributes |= TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE;
>>>>>>> + } else {
>>>>>>> + tdx->attributes &= ~TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE;
>>>>>>> + }
>>>>>>> +}
>>>>>>
>>>>>> If I look at the documentation for TD attributes
>>>>>>
>>>>>> https://download.01.org/intel-sgx/latest/dcap-
>>>>> latest/linux/docs/Intel_TDX_DCAP_Quoting_Library_API.pdf
>>>>>>
>>>>>> Section "A.3.4. TD Attributes"
>>>>>>
>>>>>> I see "TD attributes" is a 64-bit int, with 5 bits currently
>>>>>> defined "DEBUG", "SEPT_VE_DISABLE", "PKS", "PL", "PERFMON",
>>>>>> and the rest currently reserved for future use. This makes me
>>>>>> wonder about our modelling approach into the future ?
>>>>>>
>>>>>> For the AMD SEV equivalent we've just directly exposed the whole
>>>>>> field as an int:
>>>>>>
>>>>>> 'policy' : 'uint32',
>>>>>>
>>>>>> For the proposed SEV-SNP patches, the same has been done again
>>>>>>
>>>>>> https://lists.nongnu.org/archive/html/qemu-devel/2024-
>>>>> 06/msg00536.html
>>>>>>
>>>>>> '*policy': 'uint64',
>>>>>>
>>>>>>
>>>>>> The advantage of exposing individual booleans is that it is
>>>>>> self-documenting at the QAPI level, but the disadvantage is
>>>>>> that every time we want to expose ability to control a new
>>>>>> bit in the policy we have to modify QEMU, libvirt, the mgmt
>>>>>> app above libvirt, and whatever tools the end user has to
>>>>>> talk to the mgmt app.
>>>>>>
>>>>>> If we expose a policy int, then newly defined bits only require
>>>>>> a change in QEMU, and everything above QEMU will already be
>>>>>> capable of setting it.
>>>>>>
>>>>>> In fact if I look at the proposed libvirt patches, they have
>>>>>> proposed just exposing a policy "int" field in the XML, which
>>>>>> then has to be unpacked to set the individual QAPI booleans
>>>>>>
>>>>>>
>>>>> https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/message/WXWX
>>>>> EESYUA77DP7YIBP55T2OPSVKV5QW/
>>>>>>
>>>>>> On balance, I think it would be better if QEMU just exposed
>>>>>> the raw TD attributes policy as an uint64 at QAPI, instead
>>>>>> of trying to unpack it to discrete bool fields. This gives
>>>>>> consistency with SEV and SEV-SNP, and with what's proposed
>>>>>> at the libvirt level, and minimizes future changes when
>>>>>> more policy bits are defined.
>>>>>
>>>>> The reasons why introducing individual bit of sept-ve-disable instead of
>>>>> a raw TD attribute as a whole are that
>>>>>
>>>>> 1. other bits like perfmon, PKS, KL are associated with cpu properties,
>>>>> e.g.,
>>>>>
>>>>> perfmon -> pmu,
>>>>> pks -> pks,
>>>>> kl -> keylokcer feature that QEMU currently doesn't support
>>>>>
>>>>> If allowing configuring attribute directly, we need to deal with the
>>>>> inconsistence between attribute vs cpu property.
>>>>
>>>> What about defining those bits associated with cpu properties reserved
>>>> But other bits work as Daniel suggested way.
>>>
>>> I don't understand. Do you mean we provide the interface to configure raw 64
>>> bit attributes while some bits of it are reserved?
>>
>> Just have a mask of what bits are permitted to be set, and report an
>> error if the user sets non-permitted bits.
>
> Looking at the IGVM patches, the CnofidentialGuestSupport class is
> gaining a "set_guest_policy" method, which takes a "uint64_t policy".
>
> If (when) IGVM support is extended to cover TDX too, then we're
> going to need to accept the user providing the policy in integer
> format via the IGVM file metadata. This will require adding code
> to check for any inconsistency between the policy bitmask, and
> the CPU flags.
>
> https://lists.nongnu.org/archive/html/qemu-devel/2024-06/msg04037.html
Just have a glance at the IGVM series (which is a news to me). The IGVM
series looks specific to SEV-*. Though some of the callbacks introduced
in ConfidentialGuestSupportClass can be tweaked for TDX, but the
arguments introduced are specific to SEV-*.
Aside from above, are you going to map (sev-*)'s policy to TDX's
attributes? I don't think it is a good idea. It mixes up things with
limited benefit. To me, the common of them is only they are both 64-bit
field.
> so to me this is another reason to just expose the policy as an
> integer in the QAPI/QOM structure too. Everywhere just wants to
> be working with policy in integer format.
I'm not reluctant to expose TD's attribute as a raw 64-bit data in QAPI
structure. I just don't like the idea of the permitted mask, which makes
the 64-bit field incomplete, and makes things complicated. People need
to learn that some bits are configured in attribute directly while other
bits are configured via cpu properties indirectly.
Maybe we can allow the direct configurability of raw 64-bit attribute
and give it highest priority. If inconsistent value is provided via cpu
properties, warn it and let the attribute value overwrite CPU properties?
> With regards,
> Daniel
^ permalink raw reply [flat|nested] 124+ messages in thread
* [PATCH v5 26/65] i386/tdx: Make sept_ve_disable set by default
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (24 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 25/65] i386/tdx: Add property sept-ve-disable for tdx-guest object Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-02-29 6:36 ` [PATCH v5 27/65] i386/tdx: Wire CPU features up with attributes of TD guest Xiaoyao Li
` (38 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
From: Isaku Yamahata <isaku.yamahata@intel.com>
For TDX KVM use case, Linux guest is the most major one. It requires
sept_ve_disable set. Make it default for the main use case. For other use
case, it can be enabled/disabled via qemu command line.
Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
target/i386/kvm/tdx.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 806192158c9d..b6295a644566 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -548,7 +548,7 @@ static void tdx_guest_init(Object *obj)
qemu_mutex_init(&tdx->lock);
- tdx->attributes = 0;
+ tdx->attributes = TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE;
object_property_add_bool(obj, "sept-ve-disable",
tdx_guest_get_sept_ve_disable,
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 27/65] i386/tdx: Wire CPU features up with attributes of TD guest
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (25 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 26/65] i386/tdx: Make sept_ve_disable set by default Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-02-29 6:36 ` [PATCH v5 28/65] i386/tdx: Disable pmu for " Xiaoyao Li
` (37 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
For QEMU VMs, PKS is configured via CPUID_7_0_ECX_PKS and PMU is
configured by x86cpu->enable_pmu. Reuse the existing configuration
interface for TDX VMs.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
target/i386/kvm/tdx.c | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index b6295a644566..262e86fd2c67 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -33,6 +33,8 @@
(1U << KVM_FEATURE_MSI_EXT_DEST_ID))
#define TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE BIT_ULL(28)
+#define TDX_TD_ATTRIBUTES_PKS BIT_ULL(30)
+#define TDX_TD_ATTRIBUTES_PERFMON BIT_ULL(63)
#define TDX_ATTRIBUTES_MAX_BITS 64
@@ -477,6 +479,15 @@ static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
return 0;
}
+static void setup_td_guest_attributes(X86CPU *x86cpu)
+{
+ CPUX86State *env = &x86cpu->env;
+
+ tdx_guest->attributes |= (env->features[FEAT_7_0_ECX] & CPUID_7_0_ECX_PKS) ?
+ TDX_TD_ATTRIBUTES_PKS : 0;
+ tdx_guest->attributes |= x86cpu->enable_pmu ? TDX_TD_ATTRIBUTES_PERFMON : 0;
+}
+
int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
{
MachineState *ms = MACHINE(qdev_get_machine());
@@ -499,6 +510,8 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
return r;
}
+ setup_td_guest_attributes(x86cpu);
+
init_vm->cpuid.nent = kvm_x86_arch_cpuid(env, init_vm->cpuid.entries, 0);
init_vm->attributes = tdx_guest->attributes;
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 28/65] i386/tdx: Disable pmu for TD guest
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (26 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 27/65] i386/tdx: Wire CPU features up with attributes of TD guest Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-04-16 8:32 ` Chenyi Qiang
2024-02-29 6:36 ` [PATCH v5 29/65] i386/tdx: Validate TD attributes Xiaoyao Li
` (36 subsequent siblings)
64 siblings, 1 reply; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
Current KVM doesn't support PMU for TD guest. It returns error if TD is
created with PMU bit being set in attributes.
Disable PMU for TD guest on QEMU side.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
target/i386/kvm/tdx.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 262e86fd2c67..1c12cda002b8 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -496,6 +496,8 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
g_autofree struct kvm_tdx_init_vm *init_vm = NULL;
int r = 0;
+ object_property_set_bool(OBJECT(cpu), "pmu", false, &error_abort);
+
QEMU_LOCK_GUARD(&tdx_guest->lock);
if (tdx_guest->initialized) {
return r;
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* Re: [PATCH v5 28/65] i386/tdx: Disable pmu for TD guest
2024-02-29 6:36 ` [PATCH v5 28/65] i386/tdx: Disable pmu for " Xiaoyao Li
@ 2024-04-16 8:32 ` Chenyi Qiang
2024-04-16 8:55 ` Xiaoyao Li
0 siblings, 1 reply; 124+ messages in thread
From: Chenyi Qiang @ 2024-04-16 8:32 UTC (permalink / raw)
To: Xiaoyao Li, Paolo Bonzini, David Hildenbrand, Igor Mammedov,
Eduardo Habkost, Marcel Apfelbaum, Philippe Mathieu-Daudé,
Yanan Wang, Michael S. Tsirkin, Richard Henderson, Ani Sinha,
Peter Xu, Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata
On 2/29/2024 2:36 PM, Xiaoyao Li wrote:
> Current KVM doesn't support PMU for TD guest. It returns error if TD is
> created with PMU bit being set in attributes.
>
> Disable PMU for TD guest on QEMU side.
>
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
> target/i386/kvm/tdx.c | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
> index 262e86fd2c67..1c12cda002b8 100644
> --- a/target/i386/kvm/tdx.c
> +++ b/target/i386/kvm/tdx.c
> @@ -496,6 +496,8 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
> g_autofree struct kvm_tdx_init_vm *init_vm = NULL;
> int r = 0;
>
> + object_property_set_bool(OBJECT(cpu), "pmu", false, &error_abort);
Is it necessary to output some prompt if the user wants to enable pmu by
"-cpu host,pmu=on"? As in patch 27, it mentions PMU is configured by
x86cpu->enable_pmu, but PMU is actually not support in this series and
will be disabled silently.
> +
> QEMU_LOCK_GUARD(&tdx_guest->lock);
> if (tdx_guest->initialized) {
> return r;
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 28/65] i386/tdx: Disable pmu for TD guest
2024-04-16 8:32 ` Chenyi Qiang
@ 2024-04-16 8:55 ` Xiaoyao Li
0 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-04-16 8:55 UTC (permalink / raw)
To: Chenyi Qiang, Paolo Bonzini, David Hildenbrand, Igor Mammedov,
Eduardo Habkost, Marcel Apfelbaum, Philippe Mathieu-Daudé,
Yanan Wang, Michael S. Tsirkin, Richard Henderson, Ani Sinha,
Peter Xu, Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata
On 4/16/2024 4:32 PM, Chenyi Qiang wrote:
>
>
> On 2/29/2024 2:36 PM, Xiaoyao Li wrote:
>> Current KVM doesn't support PMU for TD guest. It returns error if TD is
>> created with PMU bit being set in attributes.
>>
>> Disable PMU for TD guest on QEMU side.
>>
>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>> ---
>> target/i386/kvm/tdx.c | 2 ++
>> 1 file changed, 2 insertions(+)
>>
>> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
>> index 262e86fd2c67..1c12cda002b8 100644
>> --- a/target/i386/kvm/tdx.c
>> +++ b/target/i386/kvm/tdx.c
>> @@ -496,6 +496,8 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
>> g_autofree struct kvm_tdx_init_vm *init_vm = NULL;
>> int r = 0;
>>
>> + object_property_set_bool(OBJECT(cpu), "pmu", false, &error_abort);
>
> Is it necessary to output some prompt if the user wants to enable pmu by
> "-cpu host,pmu=on"? As in patch 27, it mentions PMU is configured by
> x86cpu->enable_pmu, but PMU is actually not support in this series and
> will be disabled silently.
We do this in QEMU is mainly for KVM, because KVM will fail to init TD
if ATTRIBUTE.PERFMON is set.
It's expected that KVM reports PERFMON in attributes.fixed0 when KVM
cannot provide support of it. Then QEMU will print error message
automatically when validate the attributes.
For QEMU part, next version is going to set the default value of "pmu"
to false in kvm_cpu_max_instance_init(), so that "-cpu host" will not
enable pmu for TDX VMs by default.
I suppose both the KVM and QEMU change will show up in the next version.
>> +
>> QEMU_LOCK_GUARD(&tdx_guest->lock);
>> if (tdx_guest->initialized) {
>> return r;
^ permalink raw reply [flat|nested] 124+ messages in thread
* [PATCH v5 29/65] i386/tdx: Validate TD attributes
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (27 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 28/65] i386/tdx: Disable pmu for " Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-02-29 6:36 ` [PATCH v5 30/65] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig Xiaoyao Li
` (35 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
Validate TD attributes with tdx_caps that fixed-0 bits must be zero and
fixed-1 bits must be set.
Besides, sanity check the attribute bits that have not been supported by
QEMU yet. e.g., debug bit, it will be allowed in the future when debug
TD support lands in QEMU.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
Changes in v3:
- using error_setg() for error report; (Daniel)
---
target/i386/kvm/tdx.c | 29 +++++++++++++++++++++++++++--
1 file changed, 27 insertions(+), 2 deletions(-)
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 1c12cda002b8..d0ad4f57b5d0 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -32,6 +32,7 @@
(1U << KVM_FEATURE_PV_SCHED_YIELD) | \
(1U << KVM_FEATURE_MSI_EXT_DEST_ID))
+#define TDX_TD_ATTRIBUTES_DEBUG BIT_ULL(0)
#define TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE BIT_ULL(28)
#define TDX_TD_ATTRIBUTES_PKS BIT_ULL(30)
#define TDX_TD_ATTRIBUTES_PERFMON BIT_ULL(63)
@@ -479,13 +480,34 @@ static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
return 0;
}
-static void setup_td_guest_attributes(X86CPU *x86cpu)
+static int tdx_validate_attributes(TdxGuest *tdx, Error **errp)
+{
+ if (((tdx->attributes & tdx_caps->attrs_fixed0) | tdx_caps->attrs_fixed1) !=
+ tdx->attributes) {
+ error_setg(errp, "Invalid attributes 0x%lx for TDX VM "
+ "(fixed0 0x%llx, fixed1 0x%llx)",
+ tdx->attributes, tdx_caps->attrs_fixed0,
+ tdx_caps->attrs_fixed1);
+ return -1;
+ }
+
+ if (tdx->attributes & TDX_TD_ATTRIBUTES_DEBUG) {
+ error_setg(errp, "Current QEMU doesn't support attributes.debug[bit 0] for TDX VM");
+ return -1;
+ }
+
+ return 0;
+}
+
+static int setup_td_guest_attributes(X86CPU *x86cpu, Error **errp)
{
CPUX86State *env = &x86cpu->env;
tdx_guest->attributes |= (env->features[FEAT_7_0_ECX] & CPUID_7_0_ECX_PKS) ?
TDX_TD_ATTRIBUTES_PKS : 0;
tdx_guest->attributes |= x86cpu->enable_pmu ? TDX_TD_ATTRIBUTES_PERFMON : 0;
+
+ return tdx_validate_attributes(tdx_guest, errp);
}
int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
@@ -512,7 +534,10 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
return r;
}
- setup_td_guest_attributes(x86cpu);
+ r = setup_td_guest_attributes(x86cpu, errp);
+ if (r) {
+ return r;
+ }
init_vm->cpuid.nent = kvm_x86_arch_cpuid(env, init_vm->cpuid.entries, 0);
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 30/65] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (28 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 29/65] i386/tdx: Validate TD attributes Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-02-29 8:37 ` Markus Armbruster
2024-02-29 6:36 ` [PATCH v5 31/65] i386/tdx: Implement user specified tsc frequency Xiaoyao Li
` (34 subsequent siblings)
64 siblings, 1 reply; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
From: Isaku Yamahata <isaku.yamahata@intel.com>
Three sha384 hash values, mrconfigid, mrowner and mrownerconfig, of a TD
can be provided for TDX attestation. Detailed meaning of them can be
found: https://lore.kernel.org/qemu-devel/31d6dbc1-f453-4cef-ab08-4813f4e0ff92@intel.com/
Allow user to specify those values via property mrconfigid, mrowner and
mrownerconfig. They are all in base64 format.
example
-object tdx-guest, \
mrconfigid=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
mrowner=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
mrownerconfig=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v
Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
Co-developed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
Changes in v5:
- refine the description of QAPI properties and add description of
default value when not specified;
Changes in v4:
- describe more of there fields in qom.json
- free the old value before set new value to avoid memory leak in
_setter(); (Daniel)
Changes in v3:
- use base64 encoding instread of hex-string;
---
qapi/qom.json | 17 ++++++++-
target/i386/kvm/tdx.c | 87 +++++++++++++++++++++++++++++++++++++++++++
target/i386/kvm/tdx.h | 3 ++
3 files changed, 106 insertions(+), 1 deletion(-)
diff --git a/qapi/qom.json b/qapi/qom.json
index 89ed89b9b46e..cac875349a3a 100644
--- a/qapi/qom.json
+++ b/qapi/qom.json
@@ -905,10 +905,25 @@
# pages. Some guest OS (e.g., Linux TD guest) may require this to
# be set, otherwise they refuse to boot.
#
+# @mrconfigid: ID for non-owner-defined configuration of the guest TD,
+# e.g., run-time or OS configuration (base64 encoded SHA384 digest).
+# (A default value 0 of SHA384 is used when absent).
+#
+# @mrowner: ID for the guest TD’s owner (base64 encoded SHA384 digest).
+# (A default value 0 of SHA384 is used when absent).
+#
+# @mrownerconfig: ID for owner-defined configuration of the guest TD,
+# e.g., specific to the workload rather than the run-time or OS
+# (base64 encoded SHA384 digest). (A default value 0 of SHA384 is
+# used when absent).
+#
# Since: 9.0
##
{ 'struct': 'TdxGuestProperties',
- 'data': { '*sept-ve-disable': 'bool' } }
+ 'data': { '*sept-ve-disable': 'bool',
+ '*mrconfigid': 'str',
+ '*mrowner': 'str',
+ '*mrownerconfig': 'str' } }
##
# @ThreadContextProperties:
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index d0ad4f57b5d0..4ce2f1d082ce 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -13,6 +13,7 @@
#include "qemu/osdep.h"
#include "qemu/error-report.h"
+#include "qemu/base64.h"
#include "qapi/error.h"
#include "qom/object_interfaces.h"
#include "standard-headers/asm-x86/kvm_para.h"
@@ -516,6 +517,7 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
X86CPU *x86cpu = X86_CPU(cpu);
CPUX86State *env = &x86cpu->env;
g_autofree struct kvm_tdx_init_vm *init_vm = NULL;
+ size_t data_len;
int r = 0;
object_property_set_bool(OBJECT(cpu), "pmu", false, &error_abort);
@@ -528,6 +530,38 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
init_vm = g_malloc0(sizeof(struct kvm_tdx_init_vm) +
sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES);
+#define SHA384_DIGEST_SIZE 48
+
+ if (tdx_guest->mrconfigid) {
+ g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrconfigid,
+ strlen(tdx_guest->mrconfigid), &data_len, errp);
+ if (!data || data_len != SHA384_DIGEST_SIZE) {
+ error_setg(errp, "TDX: failed to decode mrconfigid");
+ return -1;
+ }
+ memcpy(init_vm->mrconfigid, data, data_len);
+ }
+
+ if (tdx_guest->mrowner) {
+ g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrowner,
+ strlen(tdx_guest->mrowner), &data_len, errp);
+ if (!data || data_len != SHA384_DIGEST_SIZE) {
+ error_setg(errp, "TDX: failed to decode mrowner");
+ return -1;
+ }
+ memcpy(init_vm->mrowner, data, data_len);
+ }
+
+ if (tdx_guest->mrownerconfig) {
+ g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrownerconfig,
+ strlen(tdx_guest->mrownerconfig), &data_len, errp);
+ if (!data || data_len != SHA384_DIGEST_SIZE) {
+ error_setg(errp, "TDX: failed to decode mrownerconfig");
+ return -1;
+ }
+ memcpy(init_vm->mrownerconfig, data, data_len);
+ }
+
r = kvm_vm_enable_cap(kvm_state, KVM_CAP_MAX_VCPUS, 0, ms->smp.cpus);
if (r < 0) {
error_setg(errp, "Unable to set MAX VCPUS to %d", ms->smp.cpus);
@@ -574,6 +608,51 @@ static void tdx_guest_set_sept_ve_disable(Object *obj, bool value, Error **errp)
}
}
+static char * tdx_guest_get_mrconfigid(Object *obj, Error **errp)
+{
+ TdxGuest *tdx = TDX_GUEST(obj);
+
+ return g_strdup(tdx->mrconfigid);
+}
+
+static void tdx_guest_set_mrconfigid(Object *obj, const char *value, Error **errp)
+{
+ TdxGuest *tdx = TDX_GUEST(obj);
+
+ g_free(tdx->mrconfigid);
+ tdx->mrconfigid = g_strdup(value);
+}
+
+static char * tdx_guest_get_mrowner(Object *obj, Error **errp)
+{
+ TdxGuest *tdx = TDX_GUEST(obj);
+
+ return g_strdup(tdx->mrowner);
+}
+
+static void tdx_guest_set_mrowner(Object *obj, const char *value, Error **errp)
+{
+ TdxGuest *tdx = TDX_GUEST(obj);
+
+ g_free(tdx->mrowner);
+ tdx->mrowner = g_strdup(value);
+}
+
+static char * tdx_guest_get_mrownerconfig(Object *obj, Error **errp)
+{
+ TdxGuest *tdx = TDX_GUEST(obj);
+
+ return g_strdup(tdx->mrownerconfig);
+}
+
+static void tdx_guest_set_mrownerconfig(Object *obj, const char *value, Error **errp)
+{
+ TdxGuest *tdx = TDX_GUEST(obj);
+
+ g_free(tdx->mrownerconfig);
+ tdx->mrownerconfig = g_strdup(value);
+}
+
/* tdx guest */
OBJECT_DEFINE_TYPE_WITH_INTERFACES(TdxGuest,
tdx_guest,
@@ -593,6 +672,14 @@ static void tdx_guest_init(Object *obj)
object_property_add_bool(obj, "sept-ve-disable",
tdx_guest_get_sept_ve_disable,
tdx_guest_set_sept_ve_disable);
+ object_property_add_str(obj, "mrconfigid",
+ tdx_guest_get_mrconfigid,
+ tdx_guest_set_mrconfigid);
+ object_property_add_str(obj, "mrowner",
+ tdx_guest_get_mrowner, tdx_guest_set_mrowner);
+ object_property_add_str(obj, "mrownerconfig",
+ tdx_guest_get_mrownerconfig,
+ tdx_guest_set_mrownerconfig);
}
static void tdx_guest_finalize(Object *obj)
diff --git a/target/i386/kvm/tdx.h b/target/i386/kvm/tdx.h
index 0df910725b52..2697e6bdfb1d 100644
--- a/target/i386/kvm/tdx.h
+++ b/target/i386/kvm/tdx.h
@@ -21,6 +21,9 @@ typedef struct TdxGuest {
bool initialized;
uint64_t attributes; /* TD attributes */
+ char *mrconfigid; /* base64 encoded sha348 digest */
+ char *mrowner; /* base64 encoded sha348 digest */
+ char *mrownerconfig; /* base64 encoded sha348 digest */
} TdxGuest;
#ifdef CONFIG_TDX
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* Re: [PATCH v5 30/65] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig
2024-02-29 6:36 ` [PATCH v5 30/65] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig Xiaoyao Li
@ 2024-02-29 8:37 ` Markus Armbruster
2024-02-29 10:50 ` Xiaoyao Li
0 siblings, 1 reply; 124+ messages in thread
From: Markus Armbruster @ 2024-02-29 8:37 UTC (permalink / raw)
To: Xiaoyao Li
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Marcelo Tosatti, kvm, qemu-devel, Michael Roth, Claudio Fontana,
Gerd Hoffmann, Isaku Yamahata, Chenyi Qiang
Xiaoyao Li <xiaoyao.li@intel.com> writes:
> From: Isaku Yamahata <isaku.yamahata@intel.com>
>
> Three sha384 hash values, mrconfigid, mrowner and mrownerconfig, of a TD
> can be provided for TDX attestation. Detailed meaning of them can be
> found: https://lore.kernel.org/qemu-devel/31d6dbc1-f453-4cef-ab08-4813f4e0ff92@intel.com/
>
> Allow user to specify those values via property mrconfigid, mrowner and
> mrownerconfig. They are all in base64 format.
>
> example
> -object tdx-guest, \
> mrconfigid=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
> mrowner=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
> mrownerconfig=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v
>
> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
> Co-developed-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>
> ---
> Changes in v5:
> - refine the description of QAPI properties and add description of
> default value when not specified;
>
> Changes in v4:
> - describe more of there fields in qom.json
> - free the old value before set new value to avoid memory leak in
> _setter(); (Daniel)
>
> Changes in v3:
> - use base64 encoding instread of hex-string;
> ---
> qapi/qom.json | 17 ++++++++-
> target/i386/kvm/tdx.c | 87 +++++++++++++++++++++++++++++++++++++++++++
> target/i386/kvm/tdx.h | 3 ++
> 3 files changed, 106 insertions(+), 1 deletion(-)
>
> diff --git a/qapi/qom.json b/qapi/qom.json
> index 89ed89b9b46e..cac875349a3a 100644
> --- a/qapi/qom.json
> +++ b/qapi/qom.json
> @@ -905,10 +905,25 @@
> # pages. Some guest OS (e.g., Linux TD guest) may require this to
> # be set, otherwise they refuse to boot.
> #
> +# @mrconfigid: ID for non-owner-defined configuration of the guest TD,
> +# e.g., run-time or OS configuration (base64 encoded SHA384 digest).
> +# (A default value 0 of SHA384 is used when absent).
Suggest to drop the parenthesis in the last sentence.
@mrconfigid is a string, so the default value can't be 0. Actually,
it's not just any string, but a base64 encoded SHA384 digest, which
means it must be exactly 96 hex digits. So it can't be "0", either. It
could be
"000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000".
More on this below.
> +#
> +# @mrowner: ID for the guest TD’s owner (base64 encoded SHA384 digest).
> +# (A default value 0 of SHA384 is used when absent).
> +#
> +# @mrownerconfig: ID for owner-defined configuration of the guest TD,
> +# e.g., specific to the workload rather than the run-time or OS
> +# (base64 encoded SHA384 digest). (A default value 0 of SHA384 is
> +# used when absent).
> +#
> # Since: 9.0
> ##
> { 'struct': 'TdxGuestProperties',
> - 'data': { '*sept-ve-disable': 'bool' } }
> + 'data': { '*sept-ve-disable': 'bool',
> + '*mrconfigid': 'str',
> + '*mrowner': 'str',
> + '*mrownerconfig': 'str' } }
>
> ##
> # @ThreadContextProperties:
> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
> index d0ad4f57b5d0..4ce2f1d082ce 100644
> --- a/target/i386/kvm/tdx.c
> +++ b/target/i386/kvm/tdx.c
> @@ -13,6 +13,7 @@
>
> #include "qemu/osdep.h"
> #include "qemu/error-report.h"
> +#include "qemu/base64.h"
> #include "qapi/error.h"
> #include "qom/object_interfaces.h"
> #include "standard-headers/asm-x86/kvm_para.h"
> @@ -516,6 +517,7 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
> X86CPU *x86cpu = X86_CPU(cpu);
> CPUX86State *env = &x86cpu->env;
> g_autofree struct kvm_tdx_init_vm *init_vm = NULL;
> + size_t data_len;
> int r = 0;
>
> object_property_set_bool(OBJECT(cpu), "pmu", false, &error_abort);
> @@ -528,6 +530,38 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
> init_vm = g_malloc0(sizeof(struct kvm_tdx_init_vm) +
> sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES);
>
> +#define SHA384_DIGEST_SIZE 48
> +
> + if (tdx_guest->mrconfigid) {
> + g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrconfigid,
> + strlen(tdx_guest->mrconfigid), &data_len, errp);
> + if (!data || data_len != SHA384_DIGEST_SIZE) {
> + error_setg(errp, "TDX: failed to decode mrconfigid");
> + return -1;
> + }
> + memcpy(init_vm->mrconfigid, data, data_len);
> + }
When @mrconfigid is absent, the property remains null, and this
conditional is not executed. init_vm->mrconfigid[], an array of 6
__u64, remains all zero. How does the kernel treat that?
> +
> + if (tdx_guest->mrowner) {
> + g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrowner,
> + strlen(tdx_guest->mrowner), &data_len, errp);
> + if (!data || data_len != SHA384_DIGEST_SIZE) {
> + error_setg(errp, "TDX: failed to decode mrowner");
> + return -1;
> + }
> + memcpy(init_vm->mrowner, data, data_len);
> + }
> +
> + if (tdx_guest->mrownerconfig) {
> + g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrownerconfig,
> + strlen(tdx_guest->mrownerconfig), &data_len, errp);
> + if (!data || data_len != SHA384_DIGEST_SIZE) {
> + error_setg(errp, "TDX: failed to decode mrownerconfig");
> + return -1;
> + }
> + memcpy(init_vm->mrownerconfig, data, data_len);
> + }
> +
> r = kvm_vm_enable_cap(kvm_state, KVM_CAP_MAX_VCPUS, 0, ms->smp.cpus);
> if (r < 0) {
> error_setg(errp, "Unable to set MAX VCPUS to %d", ms->smp.cpus);
[...]
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 30/65] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig
2024-02-29 8:37 ` Markus Armbruster
@ 2024-02-29 10:50 ` Xiaoyao Li
2024-02-29 13:25 ` Markus Armbruster
0 siblings, 1 reply; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 10:50 UTC (permalink / raw)
To: Markus Armbruster
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Marcelo Tosatti, kvm, qemu-devel, Michael Roth, Claudio Fontana,
Gerd Hoffmann, Isaku Yamahata, Chenyi Qiang
On 2/29/2024 4:37 PM, Markus Armbruster wrote:
> Xiaoyao Li <xiaoyao.li@intel.com> writes:
>
>> From: Isaku Yamahata <isaku.yamahata@intel.com>
>>
>> Three sha384 hash values, mrconfigid, mrowner and mrownerconfig, of a TD
>> can be provided for TDX attestation. Detailed meaning of them can be
>> found: https://lore.kernel.org/qemu-devel/31d6dbc1-f453-4cef-ab08-4813f4e0ff92@intel.com/
>>
>> Allow user to specify those values via property mrconfigid, mrowner and
>> mrownerconfig. They are all in base64 format.
>>
>> example
>> -object tdx-guest, \
>> mrconfigid=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
>> mrowner=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
>> mrownerconfig=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v
>>
>> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
>> Co-developed-by: Xiaoyao Li <xiaoyao.li@intel.com>
>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>>
>> ---
>> Changes in v5:
>> - refine the description of QAPI properties and add description of
>> default value when not specified;
>>
>> Changes in v4:
>> - describe more of there fields in qom.json
>> - free the old value before set new value to avoid memory leak in
>> _setter(); (Daniel)
>>
>> Changes in v3:
>> - use base64 encoding instread of hex-string;
>> ---
>> qapi/qom.json | 17 ++++++++-
>> target/i386/kvm/tdx.c | 87 +++++++++++++++++++++++++++++++++++++++++++
>> target/i386/kvm/tdx.h | 3 ++
>> 3 files changed, 106 insertions(+), 1 deletion(-)
>>
>> diff --git a/qapi/qom.json b/qapi/qom.json
>> index 89ed89b9b46e..cac875349a3a 100644
>> --- a/qapi/qom.json
>> +++ b/qapi/qom.json
>> @@ -905,10 +905,25 @@
>> # pages. Some guest OS (e.g., Linux TD guest) may require this to
>> # be set, otherwise they refuse to boot.
>> #
>> +# @mrconfigid: ID for non-owner-defined configuration of the guest TD,
>> +# e.g., run-time or OS configuration (base64 encoded SHA384 digest).
>> +# (A default value 0 of SHA384 is used when absent).
>
> Suggest to drop the parenthesis in the last sentence.
>
> @mrconfigid is a string, so the default value can't be 0. Actually,
> it's not just any string, but a base64 encoded SHA384 digest, which
> means it must be exactly 96 hex digits. So it can't be "0", either. It
> could be
> "000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000".
I thought value 0 of SHA384 just means it.
That's my fault and my poor english.
> More on this below.
>
>> +#
>> +# @mrowner: ID for the guest TD’s owner (base64 encoded SHA384 digest).
>> +# (A default value 0 of SHA384 is used when absent).
>> +#
>> +# @mrownerconfig: ID for owner-defined configuration of the guest TD,
>> +# e.g., specific to the workload rather than the run-time or OS
>> +# (base64 encoded SHA384 digest). (A default value 0 of SHA384 is
>> +# used when absent).
>> +#
>> # Since: 9.0
>> ##
>> { 'struct': 'TdxGuestProperties',
>> - 'data': { '*sept-ve-disable': 'bool' } }
>> + 'data': { '*sept-ve-disable': 'bool',
>> + '*mrconfigid': 'str',
>> + '*mrowner': 'str',
>> + '*mrownerconfig': 'str' } }
>>
>> ##
>> # @ThreadContextProperties:
>> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
>> index d0ad4f57b5d0..4ce2f1d082ce 100644
>> --- a/target/i386/kvm/tdx.c
>> +++ b/target/i386/kvm/tdx.c
>> @@ -13,6 +13,7 @@
>>
>> #include "qemu/osdep.h"
>> #include "qemu/error-report.h"
>> +#include "qemu/base64.h"
>> #include "qapi/error.h"
>> #include "qom/object_interfaces.h"
>> #include "standard-headers/asm-x86/kvm_para.h"
>> @@ -516,6 +517,7 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
>> X86CPU *x86cpu = X86_CPU(cpu);
>> CPUX86State *env = &x86cpu->env;
>> g_autofree struct kvm_tdx_init_vm *init_vm = NULL;
>> + size_t data_len;
>> int r = 0;
>>
>> object_property_set_bool(OBJECT(cpu), "pmu", false, &error_abort);
>> @@ -528,6 +530,38 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
>> init_vm = g_malloc0(sizeof(struct kvm_tdx_init_vm) +
>> sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES);
>>
>> +#define SHA384_DIGEST_SIZE 48
>> +
>> + if (tdx_guest->mrconfigid) {
>> + g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrconfigid,
>> + strlen(tdx_guest->mrconfigid), &data_len, errp);
>> + if (!data || data_len != SHA384_DIGEST_SIZE) {
>> + error_setg(errp, "TDX: failed to decode mrconfigid");
>> + return -1;
>> + }
>> + memcpy(init_vm->mrconfigid, data, data_len);
>> + }
>
> When @mrconfigid is absent, the property remains null, and this
> conditional is not executed. init_vm->mrconfigid[], an array of 6
> __u64, remains all zero. How does the kernel treat that?
A all-zero SHA384 value is still a valid value, isn't it?
KVM treats it with no difference.
>> +
>> + if (tdx_guest->mrowner) {
>> + g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrowner,
>> + strlen(tdx_guest->mrowner), &data_len, errp);
>> + if (!data || data_len != SHA384_DIGEST_SIZE) {
>> + error_setg(errp, "TDX: failed to decode mrowner");
>> + return -1;
>> + }
>> + memcpy(init_vm->mrowner, data, data_len);
>> + }
>> +
>> + if (tdx_guest->mrownerconfig) {
>> + g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrownerconfig,
>> + strlen(tdx_guest->mrownerconfig), &data_len, errp);
>> + if (!data || data_len != SHA384_DIGEST_SIZE) {
>> + error_setg(errp, "TDX: failed to decode mrownerconfig");
>> + return -1;
>> + }
>> + memcpy(init_vm->mrownerconfig, data, data_len);
>> + }
>> +
>> r = kvm_vm_enable_cap(kvm_state, KVM_CAP_MAX_VCPUS, 0, ms->smp.cpus);
>> if (r < 0) {
>> error_setg(errp, "Unable to set MAX VCPUS to %d", ms->smp.cpus);
>
> [...]
>
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 30/65] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig
2024-02-29 10:50 ` Xiaoyao Li
@ 2024-02-29 13:25 ` Markus Armbruster
2024-02-29 14:14 ` Xiaoyao Li
0 siblings, 1 reply; 124+ messages in thread
From: Markus Armbruster @ 2024-02-29 13:25 UTC (permalink / raw)
To: Xiaoyao Li
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Marcelo Tosatti, kvm, qemu-devel, Michael Roth, Claudio Fontana,
Gerd Hoffmann, Isaku Yamahata, Chenyi Qiang
Xiaoyao Li <xiaoyao.li@intel.com> writes:
> On 2/29/2024 4:37 PM, Markus Armbruster wrote:
>> Xiaoyao Li <xiaoyao.li@intel.com> writes:
>>
>>> From: Isaku Yamahata <isaku.yamahata@intel.com>
>>>
>>> Three sha384 hash values, mrconfigid, mrowner and mrownerconfig, of a TD
>>> can be provided for TDX attestation. Detailed meaning of them can be
>>> found: https://lore.kernel.org/qemu-devel/31d6dbc1-f453-4cef-ab08-4813f4e0ff92@intel.com/
>>>
>>> Allow user to specify those values via property mrconfigid, mrowner and
>>> mrownerconfig. They are all in base64 format.
>>>
>>> example
>>> -object tdx-guest, \
>>> mrconfigid=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
>>> mrowner=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
>>> mrownerconfig=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v
>>>
>>> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
>>> Co-developed-by: Xiaoyao Li <xiaoyao.li@intel.com>
>>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
[...]
>>> diff --git a/qapi/qom.json b/qapi/qom.json
>>> index 89ed89b9b46e..cac875349a3a 100644
>>> --- a/qapi/qom.json
>>> +++ b/qapi/qom.json
>>> @@ -905,10 +905,25 @@
>>> # pages. Some guest OS (e.g., Linux TD guest) may require this to
>>> # be set, otherwise they refuse to boot.
>>> #
>>> +# @mrconfigid: ID for non-owner-defined configuration of the guest TD,
>>> +# e.g., run-time or OS configuration (base64 encoded SHA384 digest).
>>> +# (A default value 0 of SHA384 is used when absent).
>>
>> Suggest to drop the parenthesis in the last sentence.
>>
>> @mrconfigid is a string, so the default value can't be 0. Actually,
>> it's not just any string, but a base64 encoded SHA384 digest, which
>> means it must be exactly 96 hex digits. So it can't be "0", either. It
>> could be
>> "000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000".
>
> I thought value 0 of SHA384 just means it.
>
> That's my fault and my poor english.
"Fault" is too harsh :) It's not as precise as I want our interface
documentation to be. We work together to get there.
>> More on this below.
>>
>>> +#
>>> +# @mrowner: ID for the guest TD’s owner (base64 encoded SHA384 digest).
>>> +# (A default value 0 of SHA384 is used when absent).
>>> +#
>>> +# @mrownerconfig: ID for owner-defined configuration of the guest TD,
>>> +# e.g., specific to the workload rather than the run-time or OS
>>> +# (base64 encoded SHA384 digest). (A default value 0 of SHA384 is
>>> +# used when absent).
>>> +#
>>> # Since: 9.0
>>> ##
>>> { 'struct': 'TdxGuestProperties',
>>> - 'data': { '*sept-ve-disable': 'bool' } }
>>> + 'data': { '*sept-ve-disable': 'bool',
>>> + '*mrconfigid': 'str',
>>> + '*mrowner': 'str',
>>> + '*mrownerconfig': 'str' } }
>>> ##
>>> # @ThreadContextProperties:
>>> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
>>> index d0ad4f57b5d0..4ce2f1d082ce 100644
>>> --- a/target/i386/kvm/tdx.c
>>> +++ b/target/i386/kvm/tdx.c
>>> @@ -13,6 +13,7 @@
>>> #include "qemu/osdep.h"
>>> #include "qemu/error-report.h"
>>> +#include "qemu/base64.h"
>>> #include "qapi/error.h"
>>> #include "qom/object_interfaces.h"
>>> #include "standard-headers/asm-x86/kvm_para.h"
>>> @@ -516,6 +517,7 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
>>> X86CPU *x86cpu = X86_CPU(cpu);
>>> CPUX86State *env = &x86cpu->env;
>>> g_autofree struct kvm_tdx_init_vm *init_vm = NULL;
>>> + size_t data_len;
>>> int r = 0;
>>> object_property_set_bool(OBJECT(cpu), "pmu", false, &error_abort);
>>> @@ -528,6 +530,38 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
>>> init_vm = g_malloc0(sizeof(struct kvm_tdx_init_vm) +
>>> sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES);
>>> +#define SHA384_DIGEST_SIZE 48
>>> +
>>> + if (tdx_guest->mrconfigid) {
>>> + g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrconfigid,
>>> + strlen(tdx_guest->mrconfigid), &data_len, errp);
>>> + if (!data || data_len != SHA384_DIGEST_SIZE) {
>>> + error_setg(errp, "TDX: failed to decode mrconfigid");
>>> + return -1;
>>> + }
>>> + memcpy(init_vm->mrconfigid, data, data_len);
>>> + }
>>
>> When @mrconfigid is absent, the property remains null, and this
>> conditional is not executed. init_vm->mrconfigid[], an array of 6
>> __u64, remains all zero. How does the kernel treat that?
>
> A all-zero SHA384 value is still a valid value, isn't it?
>
> KVM treats it with no difference.
Can you point me to the spot in the kernel where mrconfigid is used?
[...]
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 30/65] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig
2024-02-29 13:25 ` Markus Armbruster
@ 2024-02-29 14:14 ` Xiaoyao Li
2024-03-07 8:39 ` Markus Armbruster
0 siblings, 1 reply; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 14:14 UTC (permalink / raw)
To: Markus Armbruster
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Marcelo Tosatti, kvm, qemu-devel, Michael Roth, Claudio Fontana,
Gerd Hoffmann, Isaku Yamahata, Chenyi Qiang
On 2/29/2024 9:25 PM, Markus Armbruster wrote:
> Xiaoyao Li <xiaoyao.li@intel.com> writes:
>
>> On 2/29/2024 4:37 PM, Markus Armbruster wrote:
>>> Xiaoyao Li <xiaoyao.li@intel.com> writes:
>>>
>>>> From: Isaku Yamahata <isaku.yamahata@intel.com>
>>>>
>>>> Three sha384 hash values, mrconfigid, mrowner and mrownerconfig, of a TD
>>>> can be provided for TDX attestation. Detailed meaning of them can be
>>>> found: https://lore.kernel.org/qemu-devel/31d6dbc1-f453-4cef-ab08-4813f4e0ff92@intel.com/
>>>>
>>>> Allow user to specify those values via property mrconfigid, mrowner and
>>>> mrownerconfig. They are all in base64 format.
>>>>
>>>> example
>>>> -object tdx-guest, \
>>>> mrconfigid=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
>>>> mrowner=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
>>>> mrownerconfig=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v
>>>>
>>>> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
>>>> Co-developed-by: Xiaoyao Li <xiaoyao.li@intel.com>
>>>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>
> [...]
>
>>>> diff --git a/qapi/qom.json b/qapi/qom.json
>>>> index 89ed89b9b46e..cac875349a3a 100644
>>>> --- a/qapi/qom.json
>>>> +++ b/qapi/qom.json
>>>> @@ -905,10 +905,25 @@
>>>> # pages. Some guest OS (e.g., Linux TD guest) may require this to
>>>> # be set, otherwise they refuse to boot.
>>>> #
>>>> +# @mrconfigid: ID for non-owner-defined configuration of the guest TD,
>>>> +# e.g., run-time or OS configuration (base64 encoded SHA384 digest).
>>>> +# (A default value 0 of SHA384 is used when absent).
>>>
>>> Suggest to drop the parenthesis in the last sentence.
>>>
>>> @mrconfigid is a string, so the default value can't be 0. Actually,
>>> it's not just any string, but a base64 encoded SHA384 digest, which
>>> means it must be exactly 96 hex digits. So it can't be "0", either. It
>>> could be
>>> "000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000".
>>
>> I thought value 0 of SHA384 just means it.
>>
>> That's my fault and my poor english.
>
> "Fault" is too harsh :) It's not as precise as I want our interface
> documentation to be. We work together to get there.
>
>>> More on this below.
>>>
>>>> +#
>>>> +# @mrowner: ID for the guest TD’s owner (base64 encoded SHA384 digest).
>>>> +# (A default value 0 of SHA384 is used when absent).
>>>> +#
>>>> +# @mrownerconfig: ID for owner-defined configuration of the guest TD,
>>>> +# e.g., specific to the workload rather than the run-time or OS
>>>> +# (base64 encoded SHA384 digest). (A default value 0 of SHA384 is
>>>> +# used when absent).
>>>> +#
>>>> # Since: 9.0
>>>> ##
>>>> { 'struct': 'TdxGuestProperties',
>>>> - 'data': { '*sept-ve-disable': 'bool' } }
>>>> + 'data': { '*sept-ve-disable': 'bool',
>>>> + '*mrconfigid': 'str',
>>>> + '*mrowner': 'str',
>>>> + '*mrownerconfig': 'str' } }
>>>> ##
>>>> # @ThreadContextProperties:
>>>> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
>>>> index d0ad4f57b5d0..4ce2f1d082ce 100644
>>>> --- a/target/i386/kvm/tdx.c
>>>> +++ b/target/i386/kvm/tdx.c
>>>> @@ -13,6 +13,7 @@
>>>> #include "qemu/osdep.h"
>>>> #include "qemu/error-report.h"
>>>> +#include "qemu/base64.h"
>>>> #include "qapi/error.h"
>>>> #include "qom/object_interfaces.h"
>>>> #include "standard-headers/asm-x86/kvm_para.h"
>>>> @@ -516,6 +517,7 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
>>>> X86CPU *x86cpu = X86_CPU(cpu);
>>>> CPUX86State *env = &x86cpu->env;
>>>> g_autofree struct kvm_tdx_init_vm *init_vm = NULL;
>>>> + size_t data_len;
>>>> int r = 0;
>>>> object_property_set_bool(OBJECT(cpu), "pmu", false, &error_abort);
>>>> @@ -528,6 +530,38 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
>>>> init_vm = g_malloc0(sizeof(struct kvm_tdx_init_vm) +
>>>> sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES);
>>>> +#define SHA384_DIGEST_SIZE 48
>>>> +
>>>> + if (tdx_guest->mrconfigid) {
>>>> + g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrconfigid,
>>>> + strlen(tdx_guest->mrconfigid), &data_len, errp);
>>>> + if (!data || data_len != SHA384_DIGEST_SIZE) {
>>>> + error_setg(errp, "TDX: failed to decode mrconfigid");
>>>> + return -1;
>>>> + }
>>>> + memcpy(init_vm->mrconfigid, data, data_len);
>>>> + }
>>>
>>> When @mrconfigid is absent, the property remains null, and this
>>> conditional is not executed. init_vm->mrconfigid[], an array of 6
>>> __u64, remains all zero. How does the kernel treat that?
>>
>> A all-zero SHA384 value is still a valid value, isn't it?
>>
>> KVM treats it with no difference.
>
> Can you point me to the spot in the kernel where mrconfigid is used?
>
https://github.com/intel/tdx/blob/66a10e258636fa8ec9f5ce687607bf2196a92341/arch/x86/kvm/vmx/tdx.c#L2322
KVM just copy what QEMU provides into its own data structure @td_params.
The format @of td_params is defined by TDX spec, and @td_params needs to
be passed to TDX module when initialize the context of TD via
SEAMCALL(TDH.MNG.INIT):
https://github.com/intel/tdx/blob/66a10e258636fa8ec9f5ce687607bf2196a92341/arch/x86/kvm/vmx/tdx.c#L2450
In fact, all the three SHA384 fields, will be hashed together with some
other fields (in td_params and other content of TD) to compromise the
initial measurement of TD.
TDX module doesn't care the value of td_params->mrconfigid.
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 30/65] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig
2024-02-29 14:14 ` Xiaoyao Li
@ 2024-03-07 8:39 ` Markus Armbruster
2024-03-07 11:24 ` Xiaoyao Li
0 siblings, 1 reply; 124+ messages in thread
From: Markus Armbruster @ 2024-03-07 8:39 UTC (permalink / raw)
To: Xiaoyao Li
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Marcelo Tosatti, kvm, qemu-devel, Michael Roth, Claudio Fontana,
Gerd Hoffmann, Isaku Yamahata, Chenyi Qiang
Xiaoyao Li <xiaoyao.li@intel.com> writes:
> On 2/29/2024 9:25 PM, Markus Armbruster wrote:
>> Xiaoyao Li <xiaoyao.li@intel.com> writes:
>>
>>> On 2/29/2024 4:37 PM, Markus Armbruster wrote:
>>>> Xiaoyao Li <xiaoyao.li@intel.com> writes:
>>>>
>>>>> From: Isaku Yamahata <isaku.yamahata@intel.com>
>>>>>
>>>>> Three sha384 hash values, mrconfigid, mrowner and mrownerconfig, of a TD
>>>>> can be provided for TDX attestation. Detailed meaning of them can be
>>>>> found: https://lore.kernel.org/qemu-devel/31d6dbc1-f453-4cef-ab08-4813f4e0ff92@intel.com/
>>>>>
>>>>> Allow user to specify those values via property mrconfigid, mrowner and
>>>>> mrownerconfig. They are all in base64 format.
>>>>>
>>>>> example
>>>>> -object tdx-guest, \
>>>>> mrconfigid=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
>>>>> mrowner=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
>>>>> mrownerconfig=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v
>>>>>
>>>>> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
>>>>> Co-developed-by: Xiaoyao Li <xiaoyao.li@intel.com>
>>>>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>> [...]
>>
>>>>> diff --git a/qapi/qom.json b/qapi/qom.json
>>>>> index 89ed89b9b46e..cac875349a3a 100644
>>>>> --- a/qapi/qom.json
>>>>> +++ b/qapi/qom.json
>>>>> @@ -905,10 +905,25 @@
>>>>> # pages. Some guest OS (e.g., Linux TD guest) may require this to
>>>>> # be set, otherwise they refuse to boot.
>>>>> #
>>>>> +# @mrconfigid: ID for non-owner-defined configuration of the guest TD,
>>>>> +# e.g., run-time or OS configuration (base64 encoded SHA384 digest).
>>>>> +# (A default value 0 of SHA384 is used when absent).
>>>>
>>>> Suggest to drop the parenthesis in the last sentence.
>>>>
>>>> @mrconfigid is a string, so the default value can't be 0. Actually,
>>>> it's not just any string, but a base64 encoded SHA384 digest, which
>>>> means it must be exactly 96 hex digits. So it can't be "0", either. It
>>>> could be
>>>> "000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000".
>>>
>>> I thought value 0 of SHA384 just means it.
>>>
>>> That's my fault and my poor english.
>>
>> "Fault" is too harsh :) It's not as precise as I want our interface
>> documentation to be. We work together to get there.
>>
>>>> More on this below.
>>>>
>>>>> +#
>>>>> +# @mrowner: ID for the guest TD’s owner (base64 encoded SHA384 digest).
>>>>> +# (A default value 0 of SHA384 is used when absent).
>>>>> +#
>>>>> +# @mrownerconfig: ID for owner-defined configuration of the guest TD,
>>>>> +# e.g., specific to the workload rather than the run-time or OS
>>>>> +# (base64 encoded SHA384 digest). (A default value 0 of SHA384 is
>>>>> +# used when absent).
>>>>> +#
>>>>> # Since: 9.0
>>>>> ##
>>>>> { 'struct': 'TdxGuestProperties',
>>>>> - 'data': { '*sept-ve-disable': 'bool' } }
>>>>> + 'data': { '*sept-ve-disable': 'bool',
>>>>> + '*mrconfigid': 'str',
>>>>> + '*mrowner': 'str',
>>>>> + '*mrownerconfig': 'str' } }
>>>>> ##
>>>>> # @ThreadContextProperties:
>>>>> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
>>>>> index d0ad4f57b5d0..4ce2f1d082ce 100644
>>>>> --- a/target/i386/kvm/tdx.c
>>>>> +++ b/target/i386/kvm/tdx.c
>>>>> @@ -13,6 +13,7 @@
>>>>> #include "qemu/osdep.h"
>>>>> #include "qemu/error-report.h"
>>>>> +#include "qemu/base64.h"
>>>>> #include "qapi/error.h"
>>>>> #include "qom/object_interfaces.h"
>>>>> #include "standard-headers/asm-x86/kvm_para.h"
>>>>> @@ -516,6 +517,7 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
>>>>> X86CPU *x86cpu = X86_CPU(cpu);
>>>>> CPUX86State *env = &x86cpu->env;
>>>>> g_autofree struct kvm_tdx_init_vm *init_vm = NULL;
>>>>> + size_t data_len;
>>>>> int r = 0;
>>>>> object_property_set_bool(OBJECT(cpu), "pmu", false, &error_abort);
>>>>> @@ -528,6 +530,38 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
>>>>> init_vm = g_malloc0(sizeof(struct kvm_tdx_init_vm) +
>>>>> sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES);
>>>>> +#define SHA384_DIGEST_SIZE 48
>>>>> +
>>>>> + if (tdx_guest->mrconfigid) {
>>>>> + g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrconfigid,
>>>>> + strlen(tdx_guest->mrconfigid), &data_len, errp);
>>>>> + if (!data || data_len != SHA384_DIGEST_SIZE) {
>>>>> + error_setg(errp, "TDX: failed to decode mrconfigid");
>>>>> + return -1;
>>>>> + }
>>>>> + memcpy(init_vm->mrconfigid, data, data_len);
>>>>> + }
>>>>
>>>> When @mrconfigid is absent, the property remains null, and this
>>>> conditional is not executed. init_vm->mrconfigid[], an array of 6
>>>> __u64, remains all zero. How does the kernel treat that?
>>>
>>> A all-zero SHA384 value is still a valid value, isn't it?
>>>
>>> KVM treats it with no difference.
>>
>> Can you point me to the spot in the kernel where mrconfigid is used?
>
> https://github.com/intel/tdx/blob/66a10e258636fa8ec9f5ce687607bf2196a92341/arch/x86/kvm/vmx/tdx.c#L2322
>
> KVM just copy what QEMU provides into its own data structure @td_params. The format @of td_params is defined by TDX spec, and @td_params needs to be passed to TDX module when initialize the context of TD via SEAMCALL(TDH.MNG.INIT): https://github.com/intel/tdx/blob/66a10e258636fa8ec9f5ce687607bf2196a92341/arch/x86/kvm/vmx/tdx.c#L2450
>
>
> In fact, all the three SHA384 fields, will be hashed together with some other fields (in td_params and other content of TD) to compromise the initial measurement of TD.
>
> TDX module doesn't care the value of td_params->mrconfigid.
My problem is that I don't understand when and why users would omit the
optional @mrFOO.
I naively expected absent @mrFOO to mean something like "no attestation
of FOO".
But I see that they default to
"000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000".
If this zero value is special and means "no attestation", then we
accidentally get no attestation when whatever is being hashed happens to
hash to this zero value. Unlikely, but possible.
If it's not special, then when and why is the ability to omit it useful?
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 30/65] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig
2024-03-07 8:39 ` Markus Armbruster
@ 2024-03-07 11:24 ` Xiaoyao Li
2024-03-07 13:56 ` Markus Armbruster
0 siblings, 1 reply; 124+ messages in thread
From: Xiaoyao Li @ 2024-03-07 11:24 UTC (permalink / raw)
To: Markus Armbruster
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Marcelo Tosatti, kvm, qemu-devel, Michael Roth, Claudio Fontana,
Gerd Hoffmann, Isaku Yamahata, Chenyi Qiang
On 3/7/2024 4:39 PM, Markus Armbruster wrote:
> Xiaoyao Li <xiaoyao.li@intel.com> writes:
>
>> On 2/29/2024 9:25 PM, Markus Armbruster wrote:
>>> Xiaoyao Li <xiaoyao.li@intel.com> writes:
>>>
>>>> On 2/29/2024 4:37 PM, Markus Armbruster wrote:
>>>>> Xiaoyao Li <xiaoyao.li@intel.com> writes:
>>>>>
>>>>>> From: Isaku Yamahata <isaku.yamahata@intel.com>
>>>>>>
>>>>>> Three sha384 hash values, mrconfigid, mrowner and mrownerconfig, of a TD
>>>>>> can be provided for TDX attestation. Detailed meaning of them can be
>>>>>> found: https://lore.kernel.org/qemu-devel/31d6dbc1-f453-4cef-ab08-4813f4e0ff92@intel.com/
>>>>>>
>>>>>> Allow user to specify those values via property mrconfigid, mrowner and
>>>>>> mrownerconfig. They are all in base64 format.
>>>>>>
>>>>>> example
>>>>>> -object tdx-guest, \
>>>>>> mrconfigid=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
>>>>>> mrowner=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
>>>>>> mrownerconfig=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v
>>>>>>
>>>>>> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
>>>>>> Co-developed-by: Xiaoyao Li <xiaoyao.li@intel.com>
>>>>>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>>> [...]
>>>
>>>>>> diff --git a/qapi/qom.json b/qapi/qom.json
>>>>>> index 89ed89b9b46e..cac875349a3a 100644
>>>>>> --- a/qapi/qom.json
>>>>>> +++ b/qapi/qom.json
>>>>>> @@ -905,10 +905,25 @@
>>>>>> # pages. Some guest OS (e.g., Linux TD guest) may require this to
>>>>>> # be set, otherwise they refuse to boot.
>>>>>> #
>>>>>> +# @mrconfigid: ID for non-owner-defined configuration of the guest TD,
>>>>>> +# e.g., run-time or OS configuration (base64 encoded SHA384 digest).
>>>>>> +# (A default value 0 of SHA384 is used when absent).
>>>>>
>>>>> Suggest to drop the parenthesis in the last sentence.
>>>>>
>>>>> @mrconfigid is a string, so the default value can't be 0. Actually,
>>>>> it's not just any string, but a base64 encoded SHA384 digest, which
>>>>> means it must be exactly 96 hex digits. So it can't be "0", either. It
>>>>> could be
>>>>> "000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000".
>>>>
>>>> I thought value 0 of SHA384 just means it.
>>>>
>>>> That's my fault and my poor english.
>>>
>>> "Fault" is too harsh :) It's not as precise as I want our interface
>>> documentation to be. We work together to get there.
>>>
>>>>> More on this below.
>>>>>
>>>>>> +#
>>>>>> +# @mrowner: ID for the guest TD’s owner (base64 encoded SHA384 digest).
>>>>>> +# (A default value 0 of SHA384 is used when absent).
>>>>>> +#
>>>>>> +# @mrownerconfig: ID for owner-defined configuration of the guest TD,
>>>>>> +# e.g., specific to the workload rather than the run-time or OS
>>>>>> +# (base64 encoded SHA384 digest). (A default value 0 of SHA384 is
>>>>>> +# used when absent).
>>>>>> +#
>>>>>> # Since: 9.0
>>>>>> ##
>>>>>> { 'struct': 'TdxGuestProperties',
>>>>>> - 'data': { '*sept-ve-disable': 'bool' } }
>>>>>> + 'data': { '*sept-ve-disable': 'bool',
>>>>>> + '*mrconfigid': 'str',
>>>>>> + '*mrowner': 'str',
>>>>>> + '*mrownerconfig': 'str' } }
>>>>>> ##
>>>>>> # @ThreadContextProperties:
>>>>>> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
>>>>>> index d0ad4f57b5d0..4ce2f1d082ce 100644
>>>>>> --- a/target/i386/kvm/tdx.c
>>>>>> +++ b/target/i386/kvm/tdx.c
>>>>>> @@ -13,6 +13,7 @@
>>>>>> #include "qemu/osdep.h"
>>>>>> #include "qemu/error-report.h"
>>>>>> +#include "qemu/base64.h"
>>>>>> #include "qapi/error.h"
>>>>>> #include "qom/object_interfaces.h"
>>>>>> #include "standard-headers/asm-x86/kvm_para.h"
>>>>>> @@ -516,6 +517,7 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
>>>>>> X86CPU *x86cpu = X86_CPU(cpu);
>>>>>> CPUX86State *env = &x86cpu->env;
>>>>>> g_autofree struct kvm_tdx_init_vm *init_vm = NULL;
>>>>>> + size_t data_len;
>>>>>> int r = 0;
>>>>>> object_property_set_bool(OBJECT(cpu), "pmu", false, &error_abort);
>>>>>> @@ -528,6 +530,38 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
>>>>>> init_vm = g_malloc0(sizeof(struct kvm_tdx_init_vm) +
>>>>>> sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES);
>>>>>> +#define SHA384_DIGEST_SIZE 48
>>>>>> +
>>>>>> + if (tdx_guest->mrconfigid) {
>>>>>> + g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrconfigid,
>>>>>> + strlen(tdx_guest->mrconfigid), &data_len, errp);
>>>>>> + if (!data || data_len != SHA384_DIGEST_SIZE) {
>>>>>> + error_setg(errp, "TDX: failed to decode mrconfigid");
>>>>>> + return -1;
>>>>>> + }
>>>>>> + memcpy(init_vm->mrconfigid, data, data_len);
>>>>>> + }
>>>>>
>>>>> When @mrconfigid is absent, the property remains null, and this
>>>>> conditional is not executed. init_vm->mrconfigid[], an array of 6
>>>>> __u64, remains all zero. How does the kernel treat that?
>>>>
>>>> A all-zero SHA384 value is still a valid value, isn't it?
>>>>
>>>> KVM treats it with no difference.
>>>
>>> Can you point me to the spot in the kernel where mrconfigid is used?
>>
>> https://github.com/intel/tdx/blob/66a10e258636fa8ec9f5ce687607bf2196a92341/arch/x86/kvm/vmx/tdx.c#L2322
>>
>> KVM just copy what QEMU provides into its own data structure @td_params. The format @of td_params is defined by TDX spec, and @td_params needs to be passed to TDX module when initialize the context of TD via SEAMCALL(TDH.MNG.INIT): https://github.com/intel/tdx/blob/66a10e258636fa8ec9f5ce687607bf2196a92341/arch/x86/kvm/vmx/tdx.c#L2450
>>
>>
>> In fact, all the three SHA384 fields, will be hashed together with some other fields (in td_params and other content of TD) to compromise the initial measurement of TD.
>>
>> TDX module doesn't care the value of td_params->mrconfigid.
>
> My problem is that I don't understand when and why users would omit the
> optional @mrFOO.
When users don't care it and don't have an explicit value for them, they
can omit it. Then a default all-zero value is used.
If making it mandatory field, then users have to explicit pass a
all-zero value when they don't care it.
> I naively expected absent @mrFOO to mean something like "no attestation
> of FOO".
>
> But I see that they default to
> "000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000".
>
> If this zero value is special and means "no attestation", then we
> accidentally get no attestation when whatever is being hashed happens to
> hash to this zero value. Unlikely, but possible.
>
> If it's not special, then when and why is the ability to omit it useful?
>
At some point, the zero value is special, because it is the default
value if no explicit one provided by user. But for TDX point of view, it
is not special. The field is a must for any TD, and whatever value it
is, it will be hashed into MRTD (Build-time Measurement Register) for
later attestation.
TDX architecture defines what fields are always hashed into measurement
and also provide other mechanism to hash optional field into
measurement. All this is known to users of TDX, and users can calculate
the final measurement by itself and compare to what gets reported by TDX
to see they are identical.
For these three fields, they are must-to-have fields to be hashed into
measurement. For user's convenience, we don't want to make it mandatory
input because not everyone cares it and have a specific value to input.
What people needs to know is that, when no explicit value is provided
for these three field, a all-zero value is used.
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 30/65] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig
2024-03-07 11:24 ` Xiaoyao Li
@ 2024-03-07 13:56 ` Markus Armbruster
2024-03-11 1:25 ` Xiaoyao Li
0 siblings, 1 reply; 124+ messages in thread
From: Markus Armbruster @ 2024-03-07 13:56 UTC (permalink / raw)
To: Xiaoyao Li
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Marcelo Tosatti, kvm, qemu-devel, Michael Roth, Claudio Fontana,
Gerd Hoffmann, Isaku Yamahata, Chenyi Qiang
Xiaoyao Li <xiaoyao.li@intel.com> writes:
> On 3/7/2024 4:39 PM, Markus Armbruster wrote:
>> Xiaoyao Li <xiaoyao.li@intel.com> writes:
>>
>>> On 2/29/2024 9:25 PM, Markus Armbruster wrote:
>>>> Xiaoyao Li <xiaoyao.li@intel.com> writes:
>>>>
>>>>> On 2/29/2024 4:37 PM, Markus Armbruster wrote:
>>>>>> Xiaoyao Li <xiaoyao.li@intel.com> writes:
>>>>>>
>>>>>>> From: Isaku Yamahata <isaku.yamahata@intel.com>
>>>>>>>
>>>>>>> Three sha384 hash values, mrconfigid, mrowner and mrownerconfig, of a TD
>>>>>>> can be provided for TDX attestation. Detailed meaning of them can be
>>>>>>> found: https://lore.kernel.org/qemu-devel/31d6dbc1-f453-4cef-ab08-4813f4e0ff92@intel.com/
>>>>>>>
>>>>>>> Allow user to specify those values via property mrconfigid, mrowner and
>>>>>>> mrownerconfig. They are all in base64 format.
>>>>>>>
>>>>>>> example
>>>>>>> -object tdx-guest, \
>>>>>>> mrconfigid=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
>>>>>>> mrowner=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
>>>>>>> mrownerconfig=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v
>>>>>>>
>>>>>>> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
>>>>>>> Co-developed-by: Xiaoyao Li <xiaoyao.li@intel.com>
>>>>>>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>>>> [...]
>>>>
>>>>>>> diff --git a/qapi/qom.json b/qapi/qom.json
>>>>>>> index 89ed89b9b46e..cac875349a3a 100644
>>>>>>> --- a/qapi/qom.json
>>>>>>> +++ b/qapi/qom.json
>>>>>>> @@ -905,10 +905,25 @@
>>>>>>> # pages. Some guest OS (e.g., Linux TD guest) may require this to
>>>>>>> # be set, otherwise they refuse to boot.
>>>>>>> #
>>>>>>> +# @mrconfigid: ID for non-owner-defined configuration of the guest TD,
>>>>>>> +# e.g., run-time or OS configuration (base64 encoded SHA384 digest).
>>>>>>> +# (A default value 0 of SHA384 is used when absent).
>>>>>>
>>>>>> Suggest to drop the parenthesis in the last sentence.
>>>>>>
>>>>>> @mrconfigid is a string, so the default value can't be 0. Actually,
>>>>>> it's not just any string, but a base64 encoded SHA384 digest, which
>>>>>> means it must be exactly 96 hex digits. So it can't be "0", either. It
>>>>>> could be
>>>>>> "000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000".
>>>>>
>>>>> I thought value 0 of SHA384 just means it.
>>>>>
>>>>> That's my fault and my poor english.
>>>>
>>>> "Fault" is too harsh :) It's not as precise as I want our interface
>>>> documentation to be. We work together to get there.
>>>>
>>>>>> More on this below.
>>>>>>
>>>>>>> +#
>>>>>>> +# @mrowner: ID for the guest TD’s owner (base64 encoded SHA384 digest).
>>>>>>> +# (A default value 0 of SHA384 is used when absent).
>>>>>>> +#
>>>>>>> +# @mrownerconfig: ID for owner-defined configuration of the guest TD,
>>>>>>> +# e.g., specific to the workload rather than the run-time or OS
>>>>>>> +# (base64 encoded SHA384 digest). (A default value 0 of SHA384 is
>>>>>>> +# used when absent).
>>>>>>> +#
>>>>>>> # Since: 9.0
>>>>>>> ##
>>>>>>> { 'struct': 'TdxGuestProperties',
>>>>>>> - 'data': { '*sept-ve-disable': 'bool' } }
>>>>>>> + 'data': { '*sept-ve-disable': 'bool',
>>>>>>> + '*mrconfigid': 'str',
>>>>>>> + '*mrowner': 'str',
>>>>>>> + '*mrownerconfig': 'str' } }
>>>>>>> ##
>>>>>>> # @ThreadContextProperties:
>>>>>>> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
>>>>>>> index d0ad4f57b5d0..4ce2f1d082ce 100644
>>>>>>> --- a/target/i386/kvm/tdx.c
>>>>>>> +++ b/target/i386/kvm/tdx.c
>>>>>>> @@ -13,6 +13,7 @@
>>>>>>> #include "qemu/osdep.h"
>>>>>>> #include "qemu/error-report.h"
>>>>>>> +#include "qemu/base64.h"
>>>>>>> #include "qapi/error.h"
>>>>>>> #include "qom/object_interfaces.h"
>>>>>>> #include "standard-headers/asm-x86/kvm_para.h"
>>>>>>> @@ -516,6 +517,7 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
>>>>>>> X86CPU *x86cpu = X86_CPU(cpu);
>>>>>>> CPUX86State *env = &x86cpu->env;
>>>>>>> g_autofree struct kvm_tdx_init_vm *init_vm = NULL;
>>>>>>> + size_t data_len;
>>>>>>> int r = 0;
>>>>>>> object_property_set_bool(OBJECT(cpu), "pmu", false, &error_abort);
>>>>>>> @@ -528,6 +530,38 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
>>>>>>> init_vm = g_malloc0(sizeof(struct kvm_tdx_init_vm) +
>>>>>>> sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES);
>>>>>>> +#define SHA384_DIGEST_SIZE 48
>>>>>>> +
>>>>>>> + if (tdx_guest->mrconfigid) {
>>>>>>> + g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrconfigid,
>>>>>>> + strlen(tdx_guest->mrconfigid), &data_len, errp);
>>>>>>> + if (!data || data_len != SHA384_DIGEST_SIZE) {
>>>>>>> + error_setg(errp, "TDX: failed to decode mrconfigid");
>>>>>>> + return -1;
>>>>>>> + }
>>>>>>> + memcpy(init_vm->mrconfigid, data, data_len);
>>>>>>> + }
>>>>>>
>>>>>> When @mrconfigid is absent, the property remains null, and this
>>>>>> conditional is not executed. init_vm->mrconfigid[], an array of 6
>>>>>> __u64, remains all zero. How does the kernel treat that?
>>>>>
>>>>> A all-zero SHA384 value is still a valid value, isn't it?
>>>>>
>>>>> KVM treats it with no difference.
>>>>
>>>> Can you point me to the spot in the kernel where mrconfigid is used?
>>>
>>> https://github.com/intel/tdx/blob/66a10e258636fa8ec9f5ce687607bf2196a92341/arch/x86/kvm/vmx/tdx.c#L2322
>>>
>>> KVM just copy what QEMU provides into its own data structure @td_params. The format @of td_params is defined by TDX spec, and @td_params needs to be passed to TDX module when initialize the context of TD via SEAMCALL(TDH.MNG.INIT): https://github.com/intel/tdx/blob/66a10e258636fa8ec9f5ce687607bf2196a92341/arch/x86/kvm/vmx/tdx.c#L2450
>>>
>>>
>>> In fact, all the three SHA384 fields, will be hashed together with some other fields (in td_params and other content of TD) to compromise the initial measurement of TD.
>>>
>>> TDX module doesn't care the value of td_params->mrconfigid.
>>
>> My problem is that I don't understand when and why users would omit the
>> optional @mrFOO.
>
> When users don't care it and don't have an explicit value for them, they can omit it. Then a default all-zero value is used.
>
> If making it mandatory field, then users have to explicit pass a all-zero value when they don't care it.
>
>> I naively expected absent @mrFOO to mean something like "no attestation
>> of FOO".
>>
>> But I see that they default to
>> "000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000".
>>
>> If this zero value is special and means "no attestation", then we
>> accidentally get no attestation when whatever is being hashed happens to
>> hash to this zero value. Unlikely, but possible.
>>
>> If it's not special, then when and why is the ability to omit it useful?
>
> At some point, the zero value is special, because it is the default value if no explicit one provided by user. But for TDX point of view, it is not special. The field is a must for any TD, and whatever value it is, it will be hashed into MRTD (Build-time Measurement Register) for later attestation.
>
> TDX architecture defines what fields are always hashed into measurement and also provide other mechanism to hash optional field into measurement. All this is known to users of TDX, and users can calculate the final measurement by itself and compare to what gets reported by TDX to see they are identical.
>
> For these three fields, they are must-to-have fields to be hashed into measurement. For user's convenience, we don't want to make it mandatory input because not everyone cares it and have a specific value to input.
> What people needs to know is that, when no explicit value is provided for these three field, a all-zero value is used.
Alright, the doc comment is not the place to educate me about TDX.
Perhaps we can go with
# @mrconfigid: ID for non-owner-defined configuration of the guest TD,
# e.g., run-time or OS configuration (base64 encoded SHA384 digest).
# Defaults to all zeroes.
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 30/65] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig
2024-03-07 13:56 ` Markus Armbruster
@ 2024-03-11 1:25 ` Xiaoyao Li
2024-03-11 7:19 ` Markus Armbruster
0 siblings, 1 reply; 124+ messages in thread
From: Xiaoyao Li @ 2024-03-11 1:25 UTC (permalink / raw)
To: Markus Armbruster
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Marcelo Tosatti, kvm, qemu-devel, Michael Roth, Claudio Fontana,
Gerd Hoffmann, Isaku Yamahata, Chenyi Qiang
On 3/7/2024 9:56 PM, Markus Armbruster wrote:
> Xiaoyao Li <xiaoyao.li@intel.com> writes:
>
>> On 3/7/2024 4:39 PM, Markus Armbruster wrote:
>>> Xiaoyao Li <xiaoyao.li@intel.com> writes:
>>>
>>>> On 2/29/2024 9:25 PM, Markus Armbruster wrote:
>>>>> Xiaoyao Li <xiaoyao.li@intel.com> writes:
>>>>>
>>>>>> On 2/29/2024 4:37 PM, Markus Armbruster wrote:
>>>>>>> Xiaoyao Li <xiaoyao.li@intel.com> writes:
>>>>>>>
>>>>>>>> From: Isaku Yamahata <isaku.yamahata@intel.com>
>>>>>>>>
>>>>>>>> Three sha384 hash values, mrconfigid, mrowner and mrownerconfig, of a TD
>>>>>>>> can be provided for TDX attestation. Detailed meaning of them can be
>>>>>>>> found: https://lore.kernel.org/qemu-devel/31d6dbc1-f453-4cef-ab08-4813f4e0ff92@intel.com/
>>>>>>>>
>>>>>>>> Allow user to specify those values via property mrconfigid, mrowner and
>>>>>>>> mrownerconfig. They are all in base64 format.
>>>>>>>>
>>>>>>>> example
>>>>>>>> -object tdx-guest, \
>>>>>>>> mrconfigid=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
>>>>>>>> mrowner=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
>>>>>>>> mrownerconfig=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v
>>>>>>>>
>>>>>>>> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
>>>>>>>> Co-developed-by: Xiaoyao Li <xiaoyao.li@intel.com>
>>>>>>>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>>>>> [...]
>>>>>
>>>>>>>> diff --git a/qapi/qom.json b/qapi/qom.json
>>>>>>>> index 89ed89b9b46e..cac875349a3a 100644
>>>>>>>> --- a/qapi/qom.json
>>>>>>>> +++ b/qapi/qom.json
>>>>>>>> @@ -905,10 +905,25 @@
>>>>>>>> # pages. Some guest OS (e.g., Linux TD guest) may require this to
>>>>>>>> # be set, otherwise they refuse to boot.
>>>>>>>> #
>>>>>>>> +# @mrconfigid: ID for non-owner-defined configuration of the guest TD,
>>>>>>>> +# e.g., run-time or OS configuration (base64 encoded SHA384 digest).
>>>>>>>> +# (A default value 0 of SHA384 is used when absent).
>>>>>>>
>>>>>>> Suggest to drop the parenthesis in the last sentence.
>>>>>>>
>>>>>>> @mrconfigid is a string, so the default value can't be 0. Actually,
>>>>>>> it's not just any string, but a base64 encoded SHA384 digest, which
>>>>>>> means it must be exactly 96 hex digits. So it can't be "0", either. It
>>>>>>> could be
>>>>>>> "000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000".
>>>>>>
>>>>>> I thought value 0 of SHA384 just means it.
>>>>>>
>>>>>> That's my fault and my poor english.
>>>>>
>>>>> "Fault" is too harsh :) It's not as precise as I want our interface
>>>>> documentation to be. We work together to get there.
>>>>>
>>>>>>> More on this below.
>>>>>>>
>>>>>>>> +#
>>>>>>>> +# @mrowner: ID for the guest TD’s owner (base64 encoded SHA384 digest).
>>>>>>>> +# (A default value 0 of SHA384 is used when absent).
>>>>>>>> +#
>>>>>>>> +# @mrownerconfig: ID for owner-defined configuration of the guest TD,
>>>>>>>> +# e.g., specific to the workload rather than the run-time or OS
>>>>>>>> +# (base64 encoded SHA384 digest). (A default value 0 of SHA384 is
>>>>>>>> +# used when absent).
>>>>>>>> +#
>>>>>>>> # Since: 9.0
>>>>>>>> ##
>>>>>>>> { 'struct': 'TdxGuestProperties',
>>>>>>>> - 'data': { '*sept-ve-disable': 'bool' } }
>>>>>>>> + 'data': { '*sept-ve-disable': 'bool',
>>>>>>>> + '*mrconfigid': 'str',
>>>>>>>> + '*mrowner': 'str',
>>>>>>>> + '*mrownerconfig': 'str' } }
>>>>>>>> ##
>>>>>>>> # @ThreadContextProperties:
>>>>>>>> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
>>>>>>>> index d0ad4f57b5d0..4ce2f1d082ce 100644
>>>>>>>> --- a/target/i386/kvm/tdx.c
>>>>>>>> +++ b/target/i386/kvm/tdx.c
>>>>>>>> @@ -13,6 +13,7 @@
>>>>>>>> #include "qemu/osdep.h"
>>>>>>>> #include "qemu/error-report.h"
>>>>>>>> +#include "qemu/base64.h"
>>>>>>>> #include "qapi/error.h"
>>>>>>>> #include "qom/object_interfaces.h"
>>>>>>>> #include "standard-headers/asm-x86/kvm_para.h"
>>>>>>>> @@ -516,6 +517,7 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
>>>>>>>> X86CPU *x86cpu = X86_CPU(cpu);
>>>>>>>> CPUX86State *env = &x86cpu->env;
>>>>>>>> g_autofree struct kvm_tdx_init_vm *init_vm = NULL;
>>>>>>>> + size_t data_len;
>>>>>>>> int r = 0;
>>>>>>>> object_property_set_bool(OBJECT(cpu), "pmu", false, &error_abort);
>>>>>>>> @@ -528,6 +530,38 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
>>>>>>>> init_vm = g_malloc0(sizeof(struct kvm_tdx_init_vm) +
>>>>>>>> sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES);
>>>>>>>> +#define SHA384_DIGEST_SIZE 48
>>>>>>>> +
>>>>>>>> + if (tdx_guest->mrconfigid) {
>>>>>>>> + g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrconfigid,
>>>>>>>> + strlen(tdx_guest->mrconfigid), &data_len, errp);
>>>>>>>> + if (!data || data_len != SHA384_DIGEST_SIZE) {
>>>>>>>> + error_setg(errp, "TDX: failed to decode mrconfigid");
>>>>>>>> + return -1;
>>>>>>>> + }
>>>>>>>> + memcpy(init_vm->mrconfigid, data, data_len);
>>>>>>>> + }
>>>>>>>
>>>>>>> When @mrconfigid is absent, the property remains null, and this
>>>>>>> conditional is not executed. init_vm->mrconfigid[], an array of 6
>>>>>>> __u64, remains all zero. How does the kernel treat that?
>>>>>>
>>>>>> A all-zero SHA384 value is still a valid value, isn't it?
>>>>>>
>>>>>> KVM treats it with no difference.
>>>>>
>>>>> Can you point me to the spot in the kernel where mrconfigid is used?
>>>>
>>>> https://github.com/intel/tdx/blob/66a10e258636fa8ec9f5ce687607bf2196a92341/arch/x86/kvm/vmx/tdx.c#L2322
>>>>
>>>> KVM just copy what QEMU provides into its own data structure @td_params. The format @of td_params is defined by TDX spec, and @td_params needs to be passed to TDX module when initialize the context of TD via SEAMCALL(TDH.MNG.INIT): https://github.com/intel/tdx/blob/66a10e258636fa8ec9f5ce687607bf2196a92341/arch/x86/kvm/vmx/tdx.c#L2450
>>>>
>>>>
>>>> In fact, all the three SHA384 fields, will be hashed together with some other fields (in td_params and other content of TD) to compromise the initial measurement of TD.
>>>>
>>>> TDX module doesn't care the value of td_params->mrconfigid.
>>>
>>> My problem is that I don't understand when and why users would omit the
>>> optional @mrFOO.
>>
>> When users don't care it and don't have an explicit value for them, they can omit it. Then a default all-zero value is used.
>>
>> If making it mandatory field, then users have to explicit pass a all-zero value when they don't care it.
>>
>>> I naively expected absent @mrFOO to mean something like "no attestation
>>> of FOO".
>>>
>>> But I see that they default to
>>> "000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000".
>>>
>>> If this zero value is special and means "no attestation", then we
>>> accidentally get no attestation when whatever is being hashed happens to
>>> hash to this zero value. Unlikely, but possible.
>>>
>>> If it's not special, then when and why is the ability to omit it useful?
>>
>> At some point, the zero value is special, because it is the default value if no explicit one provided by user. But for TDX point of view, it is not special. The field is a must for any TD, and whatever value it is, it will be hashed into MRTD (Build-time Measurement Register) for later attestation.
>>
>> TDX architecture defines what fields are always hashed into measurement and also provide other mechanism to hash optional field into measurement. All this is known to users of TDX, and users can calculate the final measurement by itself and compare to what gets reported by TDX to see they are identical.
>>
>> For these three fields, they are must-to-have fields to be hashed into measurement. For user's convenience, we don't want to make it mandatory input because not everyone cares it and have a specific value to input.
>> What people needs to know is that, when no explicit value is provided for these three field, a all-zero value is used.
>
> Alright, the doc comment is not the place to educate me about TDX.
I'm pleased to help you (on or off the maillist) if you have any
question on TDX. :)
> Perhaps we can go with
>
> # @mrconfigid: ID for non-owner-defined configuration of the guest TD,
> # e.g., run-time or OS configuration (base64 encoded SHA384 digest).
> # Defaults to all zeroes.
>
I will update to this in next version.
Huge thanks for your help!
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 30/65] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig
2024-03-11 1:25 ` Xiaoyao Li
@ 2024-03-11 7:19 ` Markus Armbruster
0 siblings, 0 replies; 124+ messages in thread
From: Markus Armbruster @ 2024-03-11 7:19 UTC (permalink / raw)
To: Xiaoyao Li
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Marcelo Tosatti, kvm, qemu-devel, Michael Roth, Claudio Fontana,
Gerd Hoffmann, Isaku Yamahata, Chenyi Qiang
Xiaoyao Li <xiaoyao.li@intel.com> writes:
> On 3/7/2024 9:56 PM, Markus Armbruster wrote:
[...]
>> Alright, the doc comment is not the place to educate me about TDX.
>
> I'm pleased to help you (on or off the maillist) if you have any question on TDX. :)
I can't possibly become knowledgable about QEMU's entire external
interface, it's simply too much. Where I'm ignorant, I try to learn
just enough to ensure the documentation is useful.
I appreciate your kind & patient help with that.
>> Perhaps we can go with
>> # @mrconfigid: ID for non-owner-defined configuration of the guest TD,
>> # e.g., run-time or OS configuration (base64 encoded SHA384 digest).
>> # Defaults to all zeroes.
>>
>
> I will update to this in next version.
> Huge thanks for your help!
You're welcome!
^ permalink raw reply [flat|nested] 124+ messages in thread
* [PATCH v5 31/65] i386/tdx: Implement user specified tsc frequency
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (29 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 30/65] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-02-29 6:36 ` [PATCH v5 32/65] i386/tdx: Set kvm_readonly_mem_enabled to false for TDX VM Xiaoyao Li
` (33 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
Reuse "-cpu,tsc-frequency=" to get user wanted tsc frequency and call VM
scope VM_SET_TSC_KHZ to set the tsc frequency of TD before KVM_TDX_INIT_VM.
Besides, sanity check the tsc frequency to be in the legal range and
legal granularity (required by TDX module).
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
Changes in v3:
- use @errp to report error info; (Daniel)
Changes in v1:
- Use VM scope VM_SET_TSC_KHZ to set the TSC frequency of TD since KVM
side drop the @tsc_khz field in struct kvm_tdx_init_vm
---
target/i386/kvm/kvm.c | 9 +++++++++
target/i386/kvm/tdx.c | 25 +++++++++++++++++++++++++
2 files changed, 34 insertions(+)
diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index 1664ac49005e..4f998b2d6d37 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -793,6 +793,15 @@ static int kvm_arch_set_tsc_khz(CPUState *cs)
int r, cur_freq;
bool set_ioctl = false;
+ /*
+ * TSC of TD vcpu is immutable, it cannot be set/changed via vcpu scope
+ * VM_SET_TSC_KHZ, but only be initialized via VM scope VM_SET_TSC_KHZ
+ * before ioctl KVM_TDX_INIT_VM in tdx_pre_create_vcpu()
+ */
+ if (is_tdx_vm()) {
+ return 0;
+ }
+
if (!env->tsc_khz) {
return 0;
}
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 4ce2f1d082ce..42dbb5ce5c15 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -33,6 +33,9 @@
(1U << KVM_FEATURE_PV_SCHED_YIELD) | \
(1U << KVM_FEATURE_MSI_EXT_DEST_ID))
+#define TDX_MIN_TSC_FREQUENCY_KHZ (100 * 1000)
+#define TDX_MAX_TSC_FREQUENCY_KHZ (10 * 1000 * 1000)
+
#define TDX_TD_ATTRIBUTES_DEBUG BIT_ULL(0)
#define TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE BIT_ULL(28)
#define TDX_TD_ATTRIBUTES_PKS BIT_ULL(30)
@@ -568,6 +571,28 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
return r;
}
+ if (env->tsc_khz && (env->tsc_khz < TDX_MIN_TSC_FREQUENCY_KHZ ||
+ env->tsc_khz > TDX_MAX_TSC_FREQUENCY_KHZ)) {
+ error_setg(errp, "Invalid TSC %ld KHz, must specify cpu_frequency between [%d, %d] kHz",
+ env->tsc_khz, TDX_MIN_TSC_FREQUENCY_KHZ,
+ TDX_MAX_TSC_FREQUENCY_KHZ);
+ return -EINVAL;
+ }
+
+ if (env->tsc_khz % (25 * 1000)) {
+ error_setg(errp, "Invalid TSC %ld KHz, it must be multiple of 25MHz",
+ env->tsc_khz);
+ return -EINVAL;
+ }
+
+ /* it's safe even env->tsc_khz is 0. KVM uses host's tsc_khz in this case */
+ r = kvm_vm_ioctl(kvm_state, KVM_SET_TSC_KHZ, env->tsc_khz);
+ if (r < 0) {
+ error_setg_errno(errp, -r, "Unable to set TSC frequency to %" PRId64 " kHz",
+ env->tsc_khz);
+ return r;
+ }
+
r = setup_td_guest_attributes(x86cpu, errp);
if (r) {
return r;
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 32/65] i386/tdx: Set kvm_readonly_mem_enabled to false for TDX VM
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (30 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 31/65] i386/tdx: Implement user specified tsc frequency Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-02-29 6:36 ` [PATCH v5 33/65] kvm/tdx: Don't complain when converting vMMIO region to shared Xiaoyao Li
` (32 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
TDX only supports readonly for shared memory but not for private memory.
In the view of QEMU, it has no idea whether a memslot is used as shared
memory of private. Thus just mark kvm_readonly_mem_enabled to false to
TDX VM for simplicity.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
target/i386/kvm/tdx.c | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 42dbb5ce5c15..13f069171db7 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -480,6 +480,15 @@ static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
update_tdx_cpuid_lookup_by_tdx_caps();
+ /*
+ * Set kvm_readonly_mem_allowed to false, because TDX only supports readonly
+ * memory for shared memory but not for private memory. Besides, whether a
+ * memslot is private or shared is not determined by QEMU.
+ *
+ * Thus, just mark readonly memory not supported for simplicity.
+ */
+ kvm_readonly_mem_allowed = false;
+
tdx_guest = tdx;
return 0;
}
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 33/65] kvm/tdx: Don't complain when converting vMMIO region to shared
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (31 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 32/65] i386/tdx: Set kvm_readonly_mem_enabled to false for TDX VM Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-02-29 6:36 ` [PATCH v5 34/65] kvm/tdx: Ignore memory conversion to shared of unassigned region Xiaoyao Li
` (31 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
From: Isaku Yamahata <isaku.yamahata@intel.com>
Because vMMIO region needs to be shared region, guest TD may explicitly
convert such region from private to shared. Don't complain such
conversion.
Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
accel/kvm/kvm-all.c | 19 ++++++++++++++++---
1 file changed, 16 insertions(+), 3 deletions(-)
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index c9df41efa484..d533e2611ad8 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -2985,9 +2985,22 @@ static int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private)
ret = ram_block_discard_guest_memfd_range(rb, offset, size);
}
} else {
- error_report("Convert non guest_memfd backed memory region "
- "(0x%"HWADDR_PRIx" ,+ 0x%"HWADDR_PRIx") to %s",
- start, size, to_private ? "private" : "shared");
+ /*
+ * Because vMMIO region must be shared, guest TD may convert vMMIO
+ * region to shared explicitly. Don't complain such case. See
+ * memory_region_type() for checking if the region is MMIO region.
+ */
+ if (!to_private &&
+ !memory_region_is_ram(mr) &&
+ !memory_region_is_ram_device(mr) &&
+ !memory_region_is_rom(mr) &&
+ !memory_region_is_romd(mr)) {
+ ret = 0;
+ } else {
+ error_report("Convert non guest_memfd backed memory region "
+ "(0x%"HWADDR_PRIx" ,+ 0x%"HWADDR_PRIx") to %s",
+ start, size, to_private ? "private" : "shared");
+ }
}
memory_region_unref(section.mr);
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 34/65] kvm/tdx: Ignore memory conversion to shared of unassigned region
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (32 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 33/65] kvm/tdx: Don't complain when converting vMMIO region to shared Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-02-29 6:36 ` [PATCH v5 35/65] memory: Introduce memory_region_init_ram_guest_memfd() Xiaoyao Li
` (30 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
From: Isaku Yamahata <isaku.yamahata@intel.com>
TDX requires vMMIO region to be shared. For KVM, MMIO region is the region
which kvm memslot isn't assigned to (except in-kernel emulation).
qemu has the memory region for vMMIO at each device level.
While OVMF issues MapGPA(to-shared) conservatively on 32bit PCI MMIO
region, qemu doesn't find corresponding vMMIO region because it's before
PCI device allocation and memory_region_find() finds the device region, not
PCI bus region. It's safe to ignore MapGPA(to-shared) because when guest
accesses those region they use GPA with shared bit set for vMMIO. Ignore
memory conversion request of non-assigned region to shared and return
success. Otherwise OVMF is confused and panics there.
Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
accel/kvm/kvm-all.c | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index d533e2611ad8..9dc17a1b5f43 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -2953,6 +2953,18 @@ static int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private)
section = memory_region_find(get_system_memory(), start, size);
mr = section.mr;
if (!mr) {
+ /*
+ * Ignore converting non-assigned region to shared.
+ *
+ * TDX requires vMMIO region to be shared to inject #VE to guest.
+ * OVMF issues conservatively MapGPA(shared) on 32bit PCI MMIO region,
+ * and vIO-APIC 0xFEC00000 4K page.
+ * OVMF assigns 32bit PCI MMIO region to
+ * [top of low memory: typically 2GB=0xC000000, 0xFC00000)
+ */
+ if (!to_private) {
+ return 0;
+ }
return -1;
}
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 35/65] memory: Introduce memory_region_init_ram_guest_memfd()
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (33 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 34/65] kvm/tdx: Ignore memory conversion to shared of unassigned region Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-02-29 6:36 ` [PATCH v5 36/65] i386/tdx: load TDVF for TD guest Xiaoyao Li
` (29 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
Introduce memory_region_init_ram_guest_memfd() to allocate private
guset memfd on the MemoryRegion initialization. It's for the use case of
TDVF, which must be private on TDX case.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
Changes in v5:
- drop memory_region_set_default_private() because this function is
dropped in this v5 series;
---
include/exec/memory.h | 6 ++++++
system/memory.c | 25 +++++++++++++++++++++++++
2 files changed, 31 insertions(+)
diff --git a/include/exec/memory.h b/include/exec/memory.h
index 679a8476852e..1e351f6fc875 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -1603,6 +1603,12 @@ bool memory_region_init_ram(MemoryRegion *mr,
uint64_t size,
Error **errp);
+bool memory_region_init_ram_guest_memfd(MemoryRegion *mr,
+ Object *owner,
+ const char *name,
+ uint64_t size,
+ Error **errp);
+
/**
* memory_region_init_rom: Initialize a ROM memory region.
*
diff --git a/system/memory.c b/system/memory.c
index c756950c0c0f..85a22408e9a4 100644
--- a/system/memory.c
+++ b/system/memory.c
@@ -3606,6 +3606,31 @@ bool memory_region_init_ram(MemoryRegion *mr,
return true;
}
+bool memory_region_init_ram_guest_memfd(MemoryRegion *mr,
+ Object *owner,
+ const char *name,
+ uint64_t size,
+ Error **errp)
+{
+ DeviceState *owner_dev;
+
+ if (!memory_region_init_ram_flags_nomigrate(mr, owner, name, size,
+ RAM_GUEST_MEMFD, errp)) {
+ return false;
+ }
+
+ /* This will assert if owner is neither NULL nor a DeviceState.
+ * We only want the owner here for the purposes of defining a
+ * unique name for migration. TODO: Ideally we should implement
+ * a naming scheme for Objects which are not DeviceStates, in
+ * which case we can relax this restriction.
+ */
+ owner_dev = DEVICE(owner);
+ vmstate_register_ram(mr, owner_dev);
+
+ return true;
+}
+
bool memory_region_init_rom(MemoryRegion *mr,
Object *owner,
const char *name,
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 36/65] i386/tdx: load TDVF for TD guest
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (34 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 35/65] memory: Introduce memory_region_init_ram_guest_memfd() Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-02-29 6:36 ` [PATCH v5 37/65] i386/tdvf: Introduce function to parse TDVF metadata Xiaoyao Li
` (28 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
From: Chao Peng <chao.p.peng@linux.intel.com>
TDVF(OVMF) needs to run at private memory for TD guest. TDX cannot
support pflash device since it doesn't support read-only private memory.
Thus load TDVF(OVMF) with -bios option for TDs.
Use memory_region_init_ram_guest_memfd() to allocate the MemoryRegion
for TDVF because it needs to be located at private memory.
Also store the MemoryRegion pointer of TDVF since the shared ramblock of
it can be discared after it gets copied to private ramblock.
Signed-off-by: Chao Peng <chao.p.peng@linux.intel.com>
Co-developed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
hw/i386/x86.c | 13 +++++++++++--
target/i386/kvm/tdx.c | 7 +++++++
target/i386/kvm/tdx.h | 3 +++
3 files changed, 21 insertions(+), 2 deletions(-)
diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index fa7095310f37..5a0cadc88c4f 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -47,6 +47,7 @@
#include "hw/intc/i8259.h"
#include "hw/rtc/mc146818rtc.h"
#include "target/i386/sev.h"
+#include "kvm/tdx.h"
#include "hw/acpi/cpu_hotplug.h"
#include "hw/irq.h"
@@ -1157,9 +1158,17 @@ void x86_bios_rom_init(MachineState *ms, const char *default_firmware,
(bios_size % 65536) != 0) {
goto bios_error;
}
+
bios = g_malloc(sizeof(*bios));
- memory_region_init_ram(bios, NULL, "pc.bios", bios_size, &error_fatal);
- if (sev_enabled()) {
+ if (is_tdx_vm()) {
+ memory_region_init_ram_guest_memfd(bios, NULL, "pc.bios", bios_size,
+ &error_fatal);
+ tdx_set_tdvf_region(bios);
+ } else {
+ memory_region_init_ram(bios, NULL, "pc.bios", bios_size, &error_fatal);
+ }
+
+ if (sev_enabled() || is_tdx_vm()) {
/*
* The concept of a "reset" simply doesn't exist for
* confidential computing guests, we have to destroy and
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 13f069171db7..7c8e14e3cc58 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -19,6 +19,7 @@
#include "standard-headers/asm-x86/kvm_para.h"
#include "sysemu/kvm.h"
#include "sysemu/sysemu.h"
+#include "exec/ramblock.h"
#include "hw/i386/x86.h"
#include "kvm_i386.h"
@@ -463,6 +464,12 @@ static void update_tdx_cpuid_lookup_by_tdx_caps(void)
(tdx_caps->xfam_fixed1 & CPUID_XSTATE_XSS_MASK) >> 32;
}
+void tdx_set_tdvf_region(MemoryRegion *tdvf_mr)
+{
+ assert(!tdx_guest->tdvf_mr);
+ tdx_guest->tdvf_mr = tdvf_mr;
+}
+
static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
{
MachineState *ms = MACHINE(qdev_get_machine());
diff --git a/target/i386/kvm/tdx.h b/target/i386/kvm/tdx.h
index 2697e6bdfb1d..c021223001a5 100644
--- a/target/i386/kvm/tdx.h
+++ b/target/i386/kvm/tdx.h
@@ -24,6 +24,8 @@ typedef struct TdxGuest {
char *mrconfigid; /* base64 encoded sha348 digest */
char *mrowner; /* base64 encoded sha348 digest */
char *mrownerconfig; /* base64 encoded sha348 digest */
+
+ MemoryRegion *tdvf_mr;
} TdxGuest;
#ifdef CONFIG_TDX
@@ -35,5 +37,6 @@ bool is_tdx_vm(void);
void tdx_get_supported_cpuid(uint32_t function, uint32_t index, int reg,
uint32_t *ret);
int tdx_pre_create_vcpu(CPUState *cpu, Error **errp);
+void tdx_set_tdvf_region(MemoryRegion *tdvf_mr);
#endif /* QEMU_I386_TDX_H */
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 37/65] i386/tdvf: Introduce function to parse TDVF metadata
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (35 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 36/65] i386/tdx: load TDVF for TD guest Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-06-04 6:32 ` Duan, Zhenzhong
2024-02-29 6:36 ` [PATCH v5 38/65] i386/tdx: Parse TDVF metadata for TDX VM Xiaoyao Li
` (27 subsequent siblings)
64 siblings, 1 reply; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
From: Isaku Yamahata <isaku.yamahata@intel.com>
TDX VM needs to boot with its specialized firmware, Trusted Domain
Virtual Firmware (TDVF). QEMU needs to parse TDVF and map it in TD
guest memory prior to running the TDX VM.
A TDVF Metadata in TDVF image describes the structure of firmware.
QEMU refers to it to setup memory for TDVF. Introduce function
tdvf_parse_metadata() to parse the metadata from TDVF image and store
the info of each TDVF section.
TDX metadata is located by a TDX metadata offset block, which is a
GUID-ed structure. The data portion of the GUID structure contains
only an 4-byte field that is the offset of TDX metadata to the end
of firmware file.
Select X86_FW_OVMF when TDX is enable to leverage existing functions
to parse and search OVMF's GUID-ed structures.
Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
Co-developed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
Changes in v1:
- rename tdvf_parse_section_entry() to
tdvf_parse_and_check_section_entry()
Changes in RFC v4:
- rename TDX_METADATA_GUID to TDX_METADATA_OFFSET_GUID
---
hw/i386/Kconfig | 1 +
hw/i386/meson.build | 1 +
hw/i386/tdvf.c | 199 +++++++++++++++++++++++++++++++++++++++++
include/hw/i386/tdvf.h | 51 +++++++++++
4 files changed, 252 insertions(+)
create mode 100644 hw/i386/tdvf.c
create mode 100644 include/hw/i386/tdvf.h
diff --git a/hw/i386/Kconfig b/hw/i386/Kconfig
index c0ccf50ac3ef..4e6c8905f077 100644
--- a/hw/i386/Kconfig
+++ b/hw/i386/Kconfig
@@ -12,6 +12,7 @@ config SGX
config TDX
bool
+ select X86_FW_OVMF
depends on KVM
config PC
diff --git a/hw/i386/meson.build b/hw/i386/meson.build
index b9c1ca39cb05..f09441c1ea54 100644
--- a/hw/i386/meson.build
+++ b/hw/i386/meson.build
@@ -28,6 +28,7 @@ i386_ss.add(when: 'CONFIG_PC', if_true: files(
'port92.c'))
i386_ss.add(when: 'CONFIG_X86_FW_OVMF', if_true: files('pc_sysfw_ovmf.c'),
if_false: files('pc_sysfw_ovmf-stubs.c'))
+i386_ss.add(when: 'CONFIG_TDX', if_true: files('tdvf.c'))
subdir('kvm')
subdir('xen')
diff --git a/hw/i386/tdvf.c b/hw/i386/tdvf.c
new file mode 100644
index 000000000000..ff51f40088f0
--- /dev/null
+++ b/hw/i386/tdvf.c
@@ -0,0 +1,199 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0-or-later
+
+ * Copyright (c) 2020 Intel Corporation
+ * Author: Isaku Yamahata <isaku.yamahata at gmail.com>
+ * <isaku.yamahata at intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/error-report.h"
+
+#include "hw/i386/pc.h"
+#include "hw/i386/tdvf.h"
+#include "sysemu/kvm.h"
+
+#define TDX_METADATA_OFFSET_GUID "e47a6535-984a-4798-865e-4685a7bf8ec2"
+#define TDX_METADATA_VERSION 1
+#define TDVF_SIGNATURE 0x46564454 /* TDVF as little endian */
+
+typedef struct {
+ uint32_t DataOffset;
+ uint32_t RawDataSize;
+ uint64_t MemoryAddress;
+ uint64_t MemoryDataSize;
+ uint32_t Type;
+ uint32_t Attributes;
+} TdvfSectionEntry;
+
+typedef struct {
+ uint32_t Signature;
+ uint32_t Length;
+ uint32_t Version;
+ uint32_t NumberOfSectionEntries;
+ TdvfSectionEntry SectionEntries[];
+} TdvfMetadata;
+
+struct tdx_metadata_offset {
+ uint32_t offset;
+};
+
+static TdvfMetadata *tdvf_get_metadata(void *flash_ptr, int size)
+{
+ TdvfMetadata *metadata;
+ uint32_t offset = 0;
+ uint8_t *data;
+
+ if ((uint32_t) size != size) {
+ return NULL;
+ }
+
+ if (pc_system_ovmf_table_find(TDX_METADATA_OFFSET_GUID, &data, NULL)) {
+ offset = size - le32_to_cpu(((struct tdx_metadata_offset *)data)->offset);
+
+ if (offset + sizeof(*metadata) > size) {
+ return NULL;
+ }
+ } else {
+ error_report("Cannot find TDX_METADATA_OFFSET_GUID");
+ return NULL;
+ }
+
+ metadata = flash_ptr + offset;
+
+ /* Finally, verify the signature to determine if this is a TDVF image. */
+ metadata->Signature = le32_to_cpu(metadata->Signature);
+ if (metadata->Signature != TDVF_SIGNATURE) {
+ error_report("Invalid TDVF signature in metadata!");
+ return NULL;
+ }
+
+ /* Sanity check that the TDVF doesn't overlap its own metadata. */
+ metadata->Length = le32_to_cpu(metadata->Length);
+ if (offset + metadata->Length > size) {
+ return NULL;
+ }
+
+ /* Only version 1 is supported/defined. */
+ metadata->Version = le32_to_cpu(metadata->Version);
+ if (metadata->Version != TDX_METADATA_VERSION) {
+ return NULL;
+ }
+
+ return metadata;
+}
+
+static int tdvf_parse_and_check_section_entry(const TdvfSectionEntry *src,
+ TdxFirmwareEntry *entry)
+{
+ entry->data_offset = le32_to_cpu(src->DataOffset);
+ entry->data_len = le32_to_cpu(src->RawDataSize);
+ entry->address = le64_to_cpu(src->MemoryAddress);
+ entry->size = le64_to_cpu(src->MemoryDataSize);
+ entry->type = le32_to_cpu(src->Type);
+ entry->attributes = le32_to_cpu(src->Attributes);
+
+ /* sanity check */
+ if (entry->size < entry->data_len) {
+ error_report("Broken metadata RawDataSize 0x%x MemoryDataSize 0x%lx",
+ entry->data_len, entry->size);
+ return -1;
+ }
+ if (!QEMU_IS_ALIGNED(entry->address, TARGET_PAGE_SIZE)) {
+ error_report("MemoryAddress 0x%lx not page aligned", entry->address);
+ return -1;
+ }
+ if (!QEMU_IS_ALIGNED(entry->size, TARGET_PAGE_SIZE)) {
+ error_report("MemoryDataSize 0x%lx not page aligned", entry->size);
+ return -1;
+ }
+
+ switch (entry->type) {
+ case TDVF_SECTION_TYPE_BFV:
+ case TDVF_SECTION_TYPE_CFV:
+ /* The sections that must be copied from firmware image to TD memory */
+ if (entry->data_len == 0) {
+ error_report("%d section with RawDataSize == 0", entry->type);
+ return -1;
+ }
+ break;
+ case TDVF_SECTION_TYPE_TD_HOB:
+ case TDVF_SECTION_TYPE_TEMP_MEM:
+ /* The sections that no need to be copied from firmware image */
+ if (entry->data_len != 0) {
+ error_report("%d section with RawDataSize 0x%x != 0",
+ entry->type, entry->data_len);
+ return -1;
+ }
+ break;
+ default:
+ error_report("TDVF contains unsupported section type %d", entry->type);
+ return -1;
+ }
+
+ return 0;
+}
+
+int tdvf_parse_metadata(TdxFirmware *fw, void *flash_ptr, int size)
+{
+ TdvfSectionEntry *sections;
+ TdvfMetadata *metadata;
+ ssize_t entries_size;
+ uint32_t len, i;
+
+ metadata = tdvf_get_metadata(flash_ptr, size);
+ if (!metadata) {
+ return -EINVAL;
+ }
+
+ //load and parse metadata entries
+ fw->nr_entries = le32_to_cpu(metadata->NumberOfSectionEntries);
+ if (fw->nr_entries < 2) {
+ error_report("Invalid number of fw entries (%u) in TDVF", fw->nr_entries);
+ return -EINVAL;
+ }
+
+ len = le32_to_cpu(metadata->Length);
+ entries_size = fw->nr_entries * sizeof(TdvfSectionEntry);
+ if (len != sizeof(*metadata) + entries_size) {
+ error_report("TDVF metadata len (0x%x) mismatch, expected (0x%x)",
+ len, (uint32_t)(sizeof(*metadata) + entries_size));
+ return -EINVAL;
+ }
+
+ fw->entries = g_new(TdxFirmwareEntry, fw->nr_entries);
+ sections = g_new(TdvfSectionEntry, fw->nr_entries);
+
+ if (!memcpy(sections, (void *)metadata + sizeof(*metadata), entries_size)) {
+ error_report("Failed to read TDVF section entries");
+ goto err;
+ }
+
+ for (i = 0; i < fw->nr_entries; i++) {
+ if (tdvf_parse_and_check_section_entry(§ions[i], &fw->entries[i])) {
+ goto err;
+ }
+ }
+ g_free(sections);
+
+ return 0;
+
+err:
+ g_free(sections);
+ fw->entries = 0;
+ g_free(fw->entries);
+ return -EINVAL;
+}
diff --git a/include/hw/i386/tdvf.h b/include/hw/i386/tdvf.h
new file mode 100644
index 000000000000..593341eb2e93
--- /dev/null
+++ b/include/hw/i386/tdvf.h
@@ -0,0 +1,51 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0-or-later
+
+ * Copyright (c) 2020 Intel Corporation
+ * Author: Isaku Yamahata <isaku.yamahata at gmail.com>
+ * <isaku.yamahata at intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef HW_I386_TDVF_H
+#define HW_I386_TDVF_H
+
+#include "qemu/osdep.h"
+
+#define TDVF_SECTION_TYPE_BFV 0
+#define TDVF_SECTION_TYPE_CFV 1
+#define TDVF_SECTION_TYPE_TD_HOB 2
+#define TDVF_SECTION_TYPE_TEMP_MEM 3
+
+#define TDVF_SECTION_ATTRIBUTES_MR_EXTEND (1U << 0)
+#define TDVF_SECTION_ATTRIBUTES_PAGE_AUG (1U << 1)
+
+typedef struct TdxFirmwareEntry {
+ uint32_t data_offset;
+ uint32_t data_len;
+ uint64_t address;
+ uint64_t size;
+ uint32_t type;
+ uint32_t attributes;
+} TdxFirmwareEntry;
+
+typedef struct TdxFirmware {
+ uint32_t nr_entries;
+ TdxFirmwareEntry *entries;
+} TdxFirmware;
+
+int tdvf_parse_metadata(TdxFirmware *fw, void *flash_ptr, int size);
+
+#endif /* HW_I386_TDVF_H */
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* Re: [PATCH v5 37/65] i386/tdvf: Introduce function to parse TDVF metadata
2024-02-29 6:36 ` [PATCH v5 37/65] i386/tdvf: Introduce function to parse TDVF metadata Xiaoyao Li
@ 2024-06-04 6:32 ` Duan, Zhenzhong
0 siblings, 0 replies; 124+ messages in thread
From: Duan, Zhenzhong @ 2024-06-04 6:32 UTC (permalink / raw)
To: Xiaoyao Li, Paolo Bonzini, David Hildenbrand, Igor Mammedov,
Eduardo Habkost, Marcel Apfelbaum, Philippe Mathieu-Daudé,
Yanan Wang, Michael S. Tsirkin, Richard Henderson, Ani Sinha,
Peter Xu, Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang
On 2/29/2024 2:36 PM, Xiaoyao Li wrote:
> From: Isaku Yamahata <isaku.yamahata@intel.com>
>
> TDX VM needs to boot with its specialized firmware, Trusted Domain
> Virtual Firmware (TDVF). QEMU needs to parse TDVF and map it in TD
> guest memory prior to running the TDX VM.
>
> A TDVF Metadata in TDVF image describes the structure of firmware.
> QEMU refers to it to setup memory for TDVF. Introduce function
> tdvf_parse_metadata() to parse the metadata from TDVF image and store
> the info of each TDVF section.
>
> TDX metadata is located by a TDX metadata offset block, which is a
> GUID-ed structure. The data portion of the GUID structure contains
> only an 4-byte field that is the offset of TDX metadata to the end
> of firmware file.
>
> Select X86_FW_OVMF when TDX is enable to leverage existing functions
> to parse and search OVMF's GUID-ed structures.
>
> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
> Co-developed-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
>
> ---
> Changes in v1:
> - rename tdvf_parse_section_entry() to
> tdvf_parse_and_check_section_entry()
> Changes in RFC v4:
> - rename TDX_METADATA_GUID to TDX_METADATA_OFFSET_GUID
> ---
> hw/i386/Kconfig | 1 +
> hw/i386/meson.build | 1 +
> hw/i386/tdvf.c | 199 +++++++++++++++++++++++++++++++++++++++++
> include/hw/i386/tdvf.h | 51 +++++++++++
> 4 files changed, 252 insertions(+)
> create mode 100644 hw/i386/tdvf.c
> create mode 100644 include/hw/i386/tdvf.h
>
> diff --git a/hw/i386/Kconfig b/hw/i386/Kconfig
> index c0ccf50ac3ef..4e6c8905f077 100644
> --- a/hw/i386/Kconfig
> +++ b/hw/i386/Kconfig
> @@ -12,6 +12,7 @@ config SGX
>
> config TDX
> bool
> + select X86_FW_OVMF
> depends on KVM
>
> config PC
> diff --git a/hw/i386/meson.build b/hw/i386/meson.build
> index b9c1ca39cb05..f09441c1ea54 100644
> --- a/hw/i386/meson.build
> +++ b/hw/i386/meson.build
> @@ -28,6 +28,7 @@ i386_ss.add(when: 'CONFIG_PC', if_true: files(
> 'port92.c'))
> i386_ss.add(when: 'CONFIG_X86_FW_OVMF', if_true: files('pc_sysfw_ovmf.c'),
> if_false: files('pc_sysfw_ovmf-stubs.c'))
> +i386_ss.add(when: 'CONFIG_TDX', if_true: files('tdvf.c'))
>
> subdir('kvm')
> subdir('xen')
> diff --git a/hw/i386/tdvf.c b/hw/i386/tdvf.c
> new file mode 100644
> index 000000000000..ff51f40088f0
> --- /dev/null
> +++ b/hw/i386/tdvf.c
> @@ -0,0 +1,199 @@
> +/*
> + * SPDX-License-Identifier: GPL-2.0-or-later
> +
> + * Copyright (c) 2020 Intel Corporation
> + * Author: Isaku Yamahata <isaku.yamahata at gmail.com>
> + * <isaku.yamahata at intel.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License as published by
> + * the Free Software Foundation; either version 2 of the License, or
> + * (at your option) any later version.
> +
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> +
> + * You should have received a copy of the GNU General Public License along
> + * with this program; if not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include "qemu/osdep.h"
> +#include "qemu/error-report.h"
> +
> +#include "hw/i386/pc.h"
> +#include "hw/i386/tdvf.h"
> +#include "sysemu/kvm.h"
> +
> +#define TDX_METADATA_OFFSET_GUID "e47a6535-984a-4798-865e-4685a7bf8ec2"
> +#define TDX_METADATA_VERSION 1
> +#define TDVF_SIGNATURE 0x46564454 /* TDVF as little endian */
> +
> +typedef struct {
> + uint32_t DataOffset;
> + uint32_t RawDataSize;
> + uint64_t MemoryAddress;
> + uint64_t MemoryDataSize;
> + uint32_t Type;
> + uint32_t Attributes;
> +} TdvfSectionEntry;
> +
> +typedef struct {
> + uint32_t Signature;
> + uint32_t Length;
> + uint32_t Version;
> + uint32_t NumberOfSectionEntries;
> + TdvfSectionEntry SectionEntries[];
> +} TdvfMetadata;
> +
> +struct tdx_metadata_offset {
> + uint32_t offset;
> +};
> +
> +static TdvfMetadata *tdvf_get_metadata(void *flash_ptr, int size)
> +{
> + TdvfMetadata *metadata;
> + uint32_t offset = 0;
> + uint8_t *data;
> +
> + if ((uint32_t) size != size) {
> + return NULL;
> + }
> +
> + if (pc_system_ovmf_table_find(TDX_METADATA_OFFSET_GUID, &data, NULL)) {
> + offset = size - le32_to_cpu(((struct tdx_metadata_offset *)data)->offset);
> +
> + if (offset + sizeof(*metadata) > size) {
> + return NULL;
> + }
> + } else {
> + error_report("Cannot find TDX_METADATA_OFFSET_GUID");
> + return NULL;
> + }
> +
> + metadata = flash_ptr + offset;
> +
> + /* Finally, verify the signature to determine if this is a TDVF image. */
> + metadata->Signature = le32_to_cpu(metadata->Signature);
> + if (metadata->Signature != TDVF_SIGNATURE) {
> + error_report("Invalid TDVF signature in metadata!");
> + return NULL;
> + }
> +
> + /* Sanity check that the TDVF doesn't overlap its own metadata. */
> + metadata->Length = le32_to_cpu(metadata->Length);
> + if (offset + metadata->Length > size) {
> + return NULL;
> + }
> +
> + /* Only version 1 is supported/defined. */
> + metadata->Version = le32_to_cpu(metadata->Version);
> + if (metadata->Version != TDX_METADATA_VERSION) {
> + return NULL;
> + }
> +
> + return metadata;
> +}
> +
> +static int tdvf_parse_and_check_section_entry(const TdvfSectionEntry *src,
> + TdxFirmwareEntry *entry)
> +{
> + entry->data_offset = le32_to_cpu(src->DataOffset);
> + entry->data_len = le32_to_cpu(src->RawDataSize);
> + entry->address = le64_to_cpu(src->MemoryAddress);
> + entry->size = le64_to_cpu(src->MemoryDataSize);
> + entry->type = le32_to_cpu(src->Type);
> + entry->attributes = le32_to_cpu(src->Attributes);
> +
> + /* sanity check */
> + if (entry->size < entry->data_len) {
> + error_report("Broken metadata RawDataSize 0x%x MemoryDataSize 0x%lx",
> + entry->data_len, entry->size);
> + return -1;
> + }
> + if (!QEMU_IS_ALIGNED(entry->address, TARGET_PAGE_SIZE)) {
> + error_report("MemoryAddress 0x%lx not page aligned", entry->address);
> + return -1;
> + }
> + if (!QEMU_IS_ALIGNED(entry->size, TARGET_PAGE_SIZE)) {
> + error_report("MemoryDataSize 0x%lx not page aligned", entry->size);
> + return -1;
> + }
> +
> + switch (entry->type) {
> + case TDVF_SECTION_TYPE_BFV:
> + case TDVF_SECTION_TYPE_CFV:
> + /* The sections that must be copied from firmware image to TD memory */
> + if (entry->data_len == 0) {
> + error_report("%d section with RawDataSize == 0", entry->type);
> + return -1;
> + }
> + break;
> + case TDVF_SECTION_TYPE_TD_HOB:
> + case TDVF_SECTION_TYPE_TEMP_MEM:
> + /* The sections that no need to be copied from firmware image */
> + if (entry->data_len != 0) {
> + error_report("%d section with RawDataSize 0x%x != 0",
> + entry->type, entry->data_len);
> + return -1;
> + }
> + break;
> + default:
> + error_report("TDVF contains unsupported section type %d", entry->type);
> + return -1;
> + }
> +
> + return 0;
> +}
> +
> +int tdvf_parse_metadata(TdxFirmware *fw, void *flash_ptr, int size)
> +{
> + TdvfSectionEntry *sections;
> + TdvfMetadata *metadata;
> + ssize_t entries_size;
> + uint32_t len, i;
> +
> + metadata = tdvf_get_metadata(flash_ptr, size);
> + if (!metadata) {
> + return -EINVAL;
> + }
> +
> + //load and parse metadata entries
> + fw->nr_entries = le32_to_cpu(metadata->NumberOfSectionEntries);
> + if (fw->nr_entries < 2) {
> + error_report("Invalid number of fw entries (%u) in TDVF", fw->nr_entries);
> + return -EINVAL;
> + }
> +
> + len = le32_to_cpu(metadata->Length);
metadata->Length is already CPU endian, any reason to convert again?
> + entries_size = fw->nr_entries * sizeof(TdvfSectionEntry);
> + if (len != sizeof(*metadata) + entries_size) {
> + error_report("TDVF metadata len (0x%x) mismatch, expected (0x%x)",
> + len, (uint32_t)(sizeof(*metadata) + entries_size));
> + return -EINVAL;
> + }
> +
> + fw->entries = g_new(TdxFirmwareEntry, fw->nr_entries);
> + sections = g_new(TdvfSectionEntry, fw->nr_entries);
> +
> + if (!memcpy(sections, (void *)metadata + sizeof(*metadata), entries_size)) {
> + error_report("Failed to read TDVF section entries");
> + goto err;
> + }
> +
> + for (i = 0; i < fw->nr_entries; i++) {
> + if (tdvf_parse_and_check_section_entry(§ions[i], &fw->entries[i])) {
> + goto err;
> + }
> + }
> + g_free(sections);
> +
> + return 0;
> +
> +err:
> + g_free(sections);
This can be g_autofree.
Thanks
Zhenzhong
^ permalink raw reply [flat|nested] 124+ messages in thread
* [PATCH v5 38/65] i386/tdx: Parse TDVF metadata for TDX VM
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (36 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 37/65] i386/tdvf: Introduce function to parse TDVF metadata Xiaoyao Li
@ 2024-02-29 6:36 ` Xiaoyao Li
2024-02-29 6:37 ` [PATCH v5 39/65] i386/tdx: Skip BIOS shadowing setup Xiaoyao Li
` (26 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:36 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
After TDVF is loaded to bios MemoryRegion, it needs parse TDVF metadata.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
hw/i386/pc_sysfw.c | 7 +++++++
target/i386/kvm/tdx-stub.c | 5 +++++
target/i386/kvm/tdx.c | 5 +++++
target/i386/kvm/tdx.h | 3 +++
4 files changed, 20 insertions(+)
diff --git a/hw/i386/pc_sysfw.c b/hw/i386/pc_sysfw.c
index b02e28557948..945978ee59f9 100644
--- a/hw/i386/pc_sysfw.c
+++ b/hw/i386/pc_sysfw.c
@@ -37,6 +37,7 @@
#include "hw/block/flash.h"
#include "sysemu/kvm.h"
#include "sev.h"
+#include "kvm/tdx.h"
#define FLASH_SECTOR_SIZE 4096
@@ -254,5 +255,11 @@ void x86_firmware_configure(void *ptr, int size)
}
sev_encrypt_flash(ptr, size, &error_fatal);
+ } else if (is_tdx_vm()) {
+ ret = tdx_parse_tdvf(ptr, size);
+ if (ret) {
+ error_report("failed to parse TDVF for TDX VM");
+ exit(1);
+ }
}
}
diff --git a/target/i386/kvm/tdx-stub.c b/target/i386/kvm/tdx-stub.c
index b614b46d3f4a..a064d583d393 100644
--- a/target/i386/kvm/tdx-stub.c
+++ b/target/i386/kvm/tdx-stub.c
@@ -6,3 +6,8 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
{
return -EINVAL;
}
+
+int tdx_parse_tdvf(void *flash_ptr, int size)
+{
+ return -EINVAL;
+}
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 7c8e14e3cc58..2bb6e9e9c392 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -631,6 +631,11 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
return 0;
}
+int tdx_parse_tdvf(void *flash_ptr, int size)
+{
+ return tdvf_parse_metadata(&tdx_guest->tdvf, flash_ptr, size);
+}
+
static bool tdx_guest_get_sept_ve_disable(Object *obj, Error **errp)
{
TdxGuest *tdx = TDX_GUEST(obj);
diff --git a/target/i386/kvm/tdx.h b/target/i386/kvm/tdx.h
index c021223001a5..30ea3737276a 100644
--- a/target/i386/kvm/tdx.h
+++ b/target/i386/kvm/tdx.h
@@ -6,6 +6,7 @@
#endif
#include "exec/confidential-guest-support.h"
+#include "hw/i386/tdvf.h"
#define TYPE_TDX_GUEST "tdx-guest"
#define TDX_GUEST(obj) OBJECT_CHECK(TdxGuest, (obj), TYPE_TDX_GUEST)
@@ -26,6 +27,7 @@ typedef struct TdxGuest {
char *mrownerconfig; /* base64 encoded sha348 digest */
MemoryRegion *tdvf_mr;
+ TdxFirmware tdvf;
} TdxGuest;
#ifdef CONFIG_TDX
@@ -38,5 +40,6 @@ void tdx_get_supported_cpuid(uint32_t function, uint32_t index, int reg,
uint32_t *ret);
int tdx_pre_create_vcpu(CPUState *cpu, Error **errp);
void tdx_set_tdvf_region(MemoryRegion *tdvf_mr);
+int tdx_parse_tdvf(void *flash_ptr, int size);
#endif /* QEMU_I386_TDX_H */
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 39/65] i386/tdx: Skip BIOS shadowing setup
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (37 preceding siblings ...)
2024-02-29 6:36 ` [PATCH v5 38/65] i386/tdx: Parse TDVF metadata for TDX VM Xiaoyao Li
@ 2024-02-29 6:37 ` Xiaoyao Li
2024-02-29 6:37 ` [PATCH v5 40/65] i386/tdx: Don't initialize pc.rom for TDX VMs Xiaoyao Li
` (25 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:37 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
TDX doesn't support map different GPAs to same private memory. Thus,
aliasing top 128KB of BIOS as isa-bios is not supported.
On the other hand, TDX guest cannot go to real mode, it can work fine
without isa-bios.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
Changes in v1:
- update commit message and comment to clarify
---
hw/i386/x86.c | 25 ++++++++++++++-----------
1 file changed, 14 insertions(+), 11 deletions(-)
diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index 5a0cadc88c4f..61c45dfc14dd 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -1190,17 +1190,20 @@ void x86_bios_rom_init(MachineState *ms, const char *default_firmware,
}
g_free(filename);
- /* map the last 128KB of the BIOS in ISA space */
- isa_bios_size = MIN(bios_size, 128 * KiB);
- isa_bios = g_malloc(sizeof(*isa_bios));
- memory_region_init_alias(isa_bios, NULL, "isa-bios", bios,
- bios_size - isa_bios_size, isa_bios_size);
- memory_region_add_subregion_overlap(rom_memory,
- 0x100000 - isa_bios_size,
- isa_bios,
- 1);
- if (!isapc_ram_fw) {
- memory_region_set_readonly(isa_bios, true);
+ /* For TDX, alias different GPAs to same private memory is not supported */
+ if (!is_tdx_vm()) {
+ /* map the last 128KB of the BIOS in ISA space */
+ isa_bios_size = MIN(bios_size, 128 * KiB);
+ isa_bios = g_malloc(sizeof(*isa_bios));
+ memory_region_init_alias(isa_bios, NULL, "isa-bios", bios,
+ bios_size - isa_bios_size, isa_bios_size);
+ memory_region_add_subregion_overlap(rom_memory,
+ 0x100000 - isa_bios_size,
+ isa_bios,
+ 1);
+ if (!isapc_ram_fw) {
+ memory_region_set_readonly(isa_bios, true);
+ }
}
/* map all the bios at the top of memory */
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 40/65] i386/tdx: Don't initialize pc.rom for TDX VMs
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (38 preceding siblings ...)
2024-02-29 6:37 ` [PATCH v5 39/65] i386/tdx: Skip BIOS shadowing setup Xiaoyao Li
@ 2024-02-29 6:37 ` Xiaoyao Li
2024-02-29 6:37 ` [PATCH v5 41/65] i386/tdx: Track mem_ptr for each firmware entry of TDVF Xiaoyao Li
` (24 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:37 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
For TDX, the address below 1MB are entirely general RAM. No need to
initialize pc.rom memory region for TDs.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
This is more as a workaround of the issue that for q35 machine type, the
real memslot update (which requires memslot deletion )for pc.rom happens
after tdx_init_memory_region. It leads to the private memory ADD'ed
before get lost. I haven't work out a good solution to resolve the
order issue. So just skip the pc.rom setup to avoid memslot deletion.
---
hw/i386/pc.c | 21 ++++++++++++---------
1 file changed, 12 insertions(+), 9 deletions(-)
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index f5ff970acfa0..3f8dd218eb08 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -43,6 +43,7 @@
#include "sysemu/xen.h"
#include "sysemu/reset.h"
#include "kvm/kvm_i386.h"
+#include "kvm/tdx.h"
#include "hw/xen/xen.h"
#include "qapi/qmp/qlist.h"
#include "qemu/error-report.h"
@@ -1028,16 +1029,18 @@ void pc_memory_init(PCMachineState *pcms,
/* Initialize PC system firmware */
pc_system_firmware_init(pcms, rom_memory);
- option_rom_mr = g_malloc(sizeof(*option_rom_mr));
- memory_region_init_ram(option_rom_mr, NULL, "pc.rom", PC_ROM_SIZE,
- &error_fatal);
- if (pcmc->pci_enabled) {
- memory_region_set_readonly(option_rom_mr, true);
+ if (!is_tdx_vm()) {
+ option_rom_mr = g_malloc(sizeof(*option_rom_mr));
+ memory_region_init_ram(option_rom_mr, NULL, "pc.rom", PC_ROM_SIZE,
+ &error_fatal);
+ if (pcmc->pci_enabled) {
+ memory_region_set_readonly(option_rom_mr, true);
+ }
+ memory_region_add_subregion_overlap(rom_memory,
+ PC_ROM_MIN_VGA,
+ option_rom_mr,
+ 1);
}
- memory_region_add_subregion_overlap(rom_memory,
- PC_ROM_MIN_VGA,
- option_rom_mr,
- 1);
fw_cfg = fw_cfg_arch_create(machine,
x86ms->boot_cpus, x86ms->apic_id_limit);
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 41/65] i386/tdx: Track mem_ptr for each firmware entry of TDVF
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (39 preceding siblings ...)
2024-02-29 6:37 ` [PATCH v5 40/65] i386/tdx: Don't initialize pc.rom for TDX VMs Xiaoyao Li
@ 2024-02-29 6:37 ` Xiaoyao Li
2024-02-29 6:37 ` [PATCH v5 42/65] i386/tdx: Track RAM entries for TDX VM Xiaoyao Li
` (23 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:37 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
For each TDVF sections, QEMU needs to copy the content to guest
private memory via KVM API (KVM_TDX_INIT_MEM_REGION).
Introduce a field @mem_ptr for TdxFirmwareEntry to track the memory
pointer of each TDVF sections. So that QEMU can add/copy them to guest
private memory later.
TDVF sections can be classified into two groups:
- Firmware itself, e.g., TDVF BFV and CFV, that located separately from
guest RAM. Its memory pointer is the bios pointer.
- Sections located at guest RAM, e.g., TEMP_MEM and TD_HOB.
mmap a new memory range for them.
Register a machine_init_done callback to do the stuff.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
hw/i386/tdvf.c | 1 +
include/hw/i386/tdvf.h | 7 +++++++
target/i386/kvm/tdx.c | 31 +++++++++++++++++++++++++++++++
3 files changed, 39 insertions(+)
diff --git a/hw/i386/tdvf.c b/hw/i386/tdvf.c
index ff51f40088f0..0a6445705160 100644
--- a/hw/i386/tdvf.c
+++ b/hw/i386/tdvf.c
@@ -189,6 +189,7 @@ int tdvf_parse_metadata(TdxFirmware *fw, void *flash_ptr, int size)
}
g_free(sections);
+ fw->mem_ptr = flash_ptr;
return 0;
err:
diff --git a/include/hw/i386/tdvf.h b/include/hw/i386/tdvf.h
index 593341eb2e93..d880af245a73 100644
--- a/include/hw/i386/tdvf.h
+++ b/include/hw/i386/tdvf.h
@@ -39,13 +39,20 @@ typedef struct TdxFirmwareEntry {
uint64_t size;
uint32_t type;
uint32_t attributes;
+
+ void *mem_ptr;
} TdxFirmwareEntry;
typedef struct TdxFirmware {
+ void *mem_ptr;
+
uint32_t nr_entries;
TdxFirmwareEntry *entries;
} TdxFirmware;
+#define for_each_tdx_fw_entry(fw, e) \
+ for (e = (fw)->entries; e != (fw)->entries + (fw)->nr_entries; e++)
+
int tdvf_parse_metadata(TdxFirmware *fw, void *flash_ptr, int size);
#endif /* HW_I386_TDVF_H */
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 2bb6e9e9c392..1b4bca9cc3cd 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -14,6 +14,7 @@
#include "qemu/osdep.h"
#include "qemu/error-report.h"
#include "qemu/base64.h"
+#include "qemu/mmap-alloc.h"
#include "qapi/error.h"
#include "qom/object_interfaces.h"
#include "standard-headers/asm-x86/kvm_para.h"
@@ -22,6 +23,7 @@
#include "exec/ramblock.h"
#include "hw/i386/x86.h"
+#include "hw/i386/tdvf.h"
#include "kvm_i386.h"
#include "tdx.h"
#include "../cpu-internal.h"
@@ -470,6 +472,33 @@ void tdx_set_tdvf_region(MemoryRegion *tdvf_mr)
tdx_guest->tdvf_mr = tdvf_mr;
}
+static void tdx_finalize_vm(Notifier *notifier, void *unused)
+{
+ TdxFirmware *tdvf = &tdx_guest->tdvf;
+ TdxFirmwareEntry *entry;
+
+ for_each_tdx_fw_entry(tdvf, entry) {
+ switch (entry->type) {
+ case TDVF_SECTION_TYPE_BFV:
+ case TDVF_SECTION_TYPE_CFV:
+ entry->mem_ptr = tdvf->mem_ptr + entry->data_offset;
+ break;
+ case TDVF_SECTION_TYPE_TD_HOB:
+ case TDVF_SECTION_TYPE_TEMP_MEM:
+ entry->mem_ptr = qemu_ram_mmap(-1, entry->size,
+ qemu_real_host_page_size(), 0, 0);
+ break;
+ default:
+ error_report("Unsupported TDVF section %d", entry->type);
+ exit(1);
+ }
+ }
+}
+
+static Notifier tdx_machine_done_notify = {
+ .notify = tdx_finalize_vm,
+};
+
static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
{
MachineState *ms = MACHINE(qdev_get_machine());
@@ -496,6 +525,8 @@ static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
*/
kvm_readonly_mem_allowed = false;
+ qemu_add_machine_init_done_notifier(&tdx_machine_done_notify);
+
tdx_guest = tdx;
return 0;
}
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 42/65] i386/tdx: Track RAM entries for TDX VM
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (40 preceding siblings ...)
2024-02-29 6:37 ` [PATCH v5 41/65] i386/tdx: Track mem_ptr for each firmware entry of TDVF Xiaoyao Li
@ 2024-02-29 6:37 ` Xiaoyao Li
2024-02-29 6:37 ` [PATCH v5 43/65] headers: Add definitions from UEFI spec for volumes, resources, etc Xiaoyao Li
` (22 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:37 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
The RAM of TDX VM can be classified into two types:
- TDX_RAM_UNACCEPTED: default type of TDX memory, which needs to be
accepted by TDX guest before it can be used and will be all-zeros
after being accepted.
- TDX_RAM_ADDED: the RAM that is ADD'ed to TD guest before running, and
can be used directly. E.g., TD HOB and TEMP MEM that needed by TDVF.
Maintain TdxRamEntries[] which grabs the initial RAM info from e820 table
and mark each RAM range as default type TDX_RAM_UNACCEPTED.
Then turn the range of TD HOB and TEMP MEM to TDX_RAM_ADDED since these
ranges will be ADD'ed before TD runs and no need to be accepted runtime.
The TdxRamEntries[] are later used to setup the memory TD resource HOB
that passes memory info from QEMU to TDVF.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
Changes in v3:
- use enum TdxRamType in struct TdxRamEntry; (Isaku)
- Fix the indention; (Daniel)
Changes in v1:
- simplify the algorithm of tdx_accept_ram_range() (Suggested-by: Gerd Hoffman)
(1) Change the existing entry to cover the accepted ram range.
(2) If there is room before the accepted ram range add a
TDX_RAM_UNACCEPTED entry for that.
(3) If there is room after the accepted ram range add a
TDX_RAM_UNACCEPTED entry for that.
---
target/i386/kvm/tdx.c | 111 ++++++++++++++++++++++++++++++++++++++++++
target/i386/kvm/tdx.h | 14 ++++++
2 files changed, 125 insertions(+)
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 1b4bca9cc3cd..98b2cfd40651 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -22,6 +22,7 @@
#include "sysemu/sysemu.h"
#include "exec/ramblock.h"
+#include "hw/i386/e820_memory_layout.h"
#include "hw/i386/x86.h"
#include "hw/i386/tdvf.h"
#include "kvm_i386.h"
@@ -472,11 +473,117 @@ void tdx_set_tdvf_region(MemoryRegion *tdvf_mr)
tdx_guest->tdvf_mr = tdvf_mr;
}
+static void tdx_add_ram_entry(uint64_t address, uint64_t length,
+ enum TdxRamType type)
+{
+ uint32_t nr_entries = tdx_guest->nr_ram_entries;
+ tdx_guest->ram_entries = g_renew(TdxRamEntry, tdx_guest->ram_entries,
+ nr_entries + 1);
+
+ tdx_guest->ram_entries[nr_entries].address = address;
+ tdx_guest->ram_entries[nr_entries].length = length;
+ tdx_guest->ram_entries[nr_entries].type = type;
+ tdx_guest->nr_ram_entries++;
+}
+
+static int tdx_accept_ram_range(uint64_t address, uint64_t length)
+{
+ uint64_t head_start, tail_start, head_length, tail_length;
+ uint64_t tmp_address, tmp_length;
+ TdxRamEntry *e;
+ int i;
+
+ for (i = 0; i < tdx_guest->nr_ram_entries; i++) {
+ e = &tdx_guest->ram_entries[i];
+
+ if (address + length <= e->address ||
+ e->address + e->length <= address) {
+ continue;
+ }
+
+ /*
+ * The to-be-accepted ram range must be fully contained by one
+ * RAM entry.
+ */
+ if (e->address > address ||
+ e->address + e->length < address + length) {
+ return -EINVAL;
+ }
+
+ if (e->type == TDX_RAM_ADDED) {
+ return -EINVAL;
+ }
+
+ break;
+ }
+
+ if (i == tdx_guest->nr_ram_entries) {
+ return -1;
+ }
+
+ tmp_address = e->address;
+ tmp_length = e->length;
+
+ e->address = address;
+ e->length = length;
+ e->type = TDX_RAM_ADDED;
+
+ head_length = address - tmp_address;
+ if (head_length > 0) {
+ head_start = tmp_address;
+ tdx_add_ram_entry(head_start, head_length, TDX_RAM_UNACCEPTED);
+ }
+
+ tail_start = address + length;
+ if (tail_start < tmp_address + tmp_length) {
+ tail_length = tmp_address + tmp_length - tail_start;
+ tdx_add_ram_entry(tail_start, tail_length, TDX_RAM_UNACCEPTED);
+ }
+
+ return 0;
+}
+
+static int tdx_ram_entry_compare(const void *lhs_, const void* rhs_)
+{
+ const TdxRamEntry *lhs = lhs_;
+ const TdxRamEntry *rhs = rhs_;
+
+ if (lhs->address == rhs->address) {
+ return 0;
+ }
+ if (le64_to_cpu(lhs->address) > le64_to_cpu(rhs->address)) {
+ return 1;
+ }
+ return -1;
+}
+
+static void tdx_init_ram_entries(void)
+{
+ unsigned i, j, nr_e820_entries;
+
+ nr_e820_entries = e820_get_num_entries();
+ tdx_guest->ram_entries = g_new(TdxRamEntry, nr_e820_entries);
+
+ for (i = 0, j = 0; i < nr_e820_entries; i++) {
+ uint64_t addr, len;
+
+ if (e820_get_entry(i, E820_RAM, &addr, &len)) {
+ tdx_guest->ram_entries[j].address = addr;
+ tdx_guest->ram_entries[j].length = len;
+ tdx_guest->ram_entries[j].type = TDX_RAM_UNACCEPTED;
+ j++;
+ }
+ }
+ tdx_guest->nr_ram_entries = j;
+}
+
static void tdx_finalize_vm(Notifier *notifier, void *unused)
{
TdxFirmware *tdvf = &tdx_guest->tdvf;
TdxFirmwareEntry *entry;
+ tdx_init_ram_entries();
+
for_each_tdx_fw_entry(tdvf, entry) {
switch (entry->type) {
case TDVF_SECTION_TYPE_BFV:
@@ -487,12 +594,16 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused)
case TDVF_SECTION_TYPE_TEMP_MEM:
entry->mem_ptr = qemu_ram_mmap(-1, entry->size,
qemu_real_host_page_size(), 0, 0);
+ tdx_accept_ram_range(entry->address, entry->size);
break;
default:
error_report("Unsupported TDVF section %d", entry->type);
exit(1);
}
}
+
+ qsort(tdx_guest->ram_entries, tdx_guest->nr_ram_entries,
+ sizeof(TdxRamEntry), &tdx_ram_entry_compare);
}
static Notifier tdx_machine_done_notify = {
diff --git a/target/i386/kvm/tdx.h b/target/i386/kvm/tdx.h
index 30ea3737276a..3fb4069268f6 100644
--- a/target/i386/kvm/tdx.h
+++ b/target/i386/kvm/tdx.h
@@ -15,6 +15,17 @@ typedef struct TdxGuestClass {
ConfidentialGuestSupportClass parent_class;
} TdxGuestClass;
+enum TdxRamType{
+ TDX_RAM_UNACCEPTED,
+ TDX_RAM_ADDED,
+};
+
+typedef struct TdxRamEntry {
+ uint64_t address;
+ uint64_t length;
+ enum TdxRamType type;
+} TdxRamEntry;
+
typedef struct TdxGuest {
ConfidentialGuestSupport parent_obj;
@@ -28,6 +39,9 @@ typedef struct TdxGuest {
MemoryRegion *tdvf_mr;
TdxFirmware tdvf;
+
+ uint32_t nr_ram_entries;
+ TdxRamEntry *ram_entries;
} TdxGuest;
#ifdef CONFIG_TDX
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 43/65] headers: Add definitions from UEFI spec for volumes, resources, etc...
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (41 preceding siblings ...)
2024-02-29 6:37 ` [PATCH v5 42/65] i386/tdx: Track RAM entries for TDX VM Xiaoyao Li
@ 2024-02-29 6:37 ` Xiaoyao Li
2024-02-29 6:37 ` [PATCH v5 44/65] i386/tdx: Setup the TD HOB list Xiaoyao Li
` (21 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:37 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
Add UEFI definitions for literals, enums, structs, GUIDs, etc... that
will be used by TDX to build the UEFI Hand-Off Block (HOB) that is passed
to the Trusted Domain Virtual Firmware (TDVF).
All values come from the UEFI specification [1], PI spec [2] and TDVF
design guide[3].
[1] UEFI Specification v2.1.0 https://uefi.org/sites/default/files/resources/UEFI_Spec_2_10_Aug29.pdf
[2] UEFI PI spec v1.8 https://uefi.org/sites/default/files/resources/UEFI_PI_Spec_1_8_March3.pdf
[3] https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-virtual-firmware-design-guide-rev-1.pdf
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
include/standard-headers/uefi/uefi.h | 198 +++++++++++++++++++++++++++
1 file changed, 198 insertions(+)
create mode 100644 include/standard-headers/uefi/uefi.h
diff --git a/include/standard-headers/uefi/uefi.h b/include/standard-headers/uefi/uefi.h
new file mode 100644
index 000000000000..b15aba796156
--- /dev/null
+++ b/include/standard-headers/uefi/uefi.h
@@ -0,0 +1,198 @@
+/*
+ * Copyright (C) 2020 Intel Corporation
+ *
+ * Author: Isaku Yamahata <isaku.yamahata at gmail.com>
+ * <isaku.yamahata at intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, see <http://www.gnu.org/licenses/>.
+ *
+ */
+
+#ifndef HW_I386_UEFI_H
+#define HW_I386_UEFI_H
+
+/***************************************************************************/
+/*
+ * basic EFI definitions
+ * supplemented with UEFI Specification Version 2.8 (Errata A)
+ * released February 2020
+ */
+/* UEFI integer is little endian */
+
+typedef struct {
+ uint32_t Data1;
+ uint16_t Data2;
+ uint16_t Data3;
+ uint8_t Data4[8];
+} EFI_GUID;
+
+typedef enum {
+ EfiReservedMemoryType,
+ EfiLoaderCode,
+ EfiLoaderData,
+ EfiBootServicesCode,
+ EfiBootServicesData,
+ EfiRuntimeServicesCode,
+ EfiRuntimeServicesData,
+ EfiConventionalMemory,
+ EfiUnusableMemory,
+ EfiACPIReclaimMemory,
+ EfiACPIMemoryNVS,
+ EfiMemoryMappedIO,
+ EfiMemoryMappedIOPortSpace,
+ EfiPalCode,
+ EfiPersistentMemory,
+ EfiUnacceptedMemoryType,
+ EfiMaxMemoryType
+} EFI_MEMORY_TYPE;
+
+#define EFI_HOB_HANDOFF_TABLE_VERSION 0x0009
+
+#define EFI_HOB_TYPE_HANDOFF 0x0001
+#define EFI_HOB_TYPE_MEMORY_ALLOCATION 0x0002
+#define EFI_HOB_TYPE_RESOURCE_DESCRIPTOR 0x0003
+#define EFI_HOB_TYPE_GUID_EXTENSION 0x0004
+#define EFI_HOB_TYPE_FV 0x0005
+#define EFI_HOB_TYPE_CPU 0x0006
+#define EFI_HOB_TYPE_MEMORY_POOL 0x0007
+#define EFI_HOB_TYPE_FV2 0x0009
+#define EFI_HOB_TYPE_LOAD_PEIM_UNUSED 0x000A
+#define EFI_HOB_TYPE_UEFI_CAPSULE 0x000B
+#define EFI_HOB_TYPE_FV3 0x000C
+#define EFI_HOB_TYPE_UNUSED 0xFFFE
+#define EFI_HOB_TYPE_END_OF_HOB_LIST 0xFFFF
+
+typedef struct {
+ uint16_t HobType;
+ uint16_t HobLength;
+ uint32_t Reserved;
+} EFI_HOB_GENERIC_HEADER;
+
+typedef uint64_t EFI_PHYSICAL_ADDRESS;
+typedef uint32_t EFI_BOOT_MODE;
+
+typedef struct {
+ EFI_HOB_GENERIC_HEADER Header;
+ uint32_t Version;
+ EFI_BOOT_MODE BootMode;
+ EFI_PHYSICAL_ADDRESS EfiMemoryTop;
+ EFI_PHYSICAL_ADDRESS EfiMemoryBottom;
+ EFI_PHYSICAL_ADDRESS EfiFreeMemoryTop;
+ EFI_PHYSICAL_ADDRESS EfiFreeMemoryBottom;
+ EFI_PHYSICAL_ADDRESS EfiEndOfHobList;
+} EFI_HOB_HANDOFF_INFO_TABLE;
+
+#define EFI_RESOURCE_SYSTEM_MEMORY 0x00000000
+#define EFI_RESOURCE_MEMORY_MAPPED_IO 0x00000001
+#define EFI_RESOURCE_IO 0x00000002
+#define EFI_RESOURCE_FIRMWARE_DEVICE 0x00000003
+#define EFI_RESOURCE_MEMORY_MAPPED_IO_PORT 0x00000004
+#define EFI_RESOURCE_MEMORY_RESERVED 0x00000005
+#define EFI_RESOURCE_IO_RESERVED 0x00000006
+#define EFI_RESOURCE_MEMORY_UNACCEPTED 0x00000007
+#define EFI_RESOURCE_MAX_MEMORY_TYPE 0x00000008
+
+#define EFI_RESOURCE_ATTRIBUTE_PRESENT 0x00000001
+#define EFI_RESOURCE_ATTRIBUTE_INITIALIZED 0x00000002
+#define EFI_RESOURCE_ATTRIBUTE_TESTED 0x00000004
+#define EFI_RESOURCE_ATTRIBUTE_SINGLE_BIT_ECC 0x00000008
+#define EFI_RESOURCE_ATTRIBUTE_MULTIPLE_BIT_ECC 0x00000010
+#define EFI_RESOURCE_ATTRIBUTE_ECC_RESERVED_1 0x00000020
+#define EFI_RESOURCE_ATTRIBUTE_ECC_RESERVED_2 0x00000040
+#define EFI_RESOURCE_ATTRIBUTE_READ_PROTECTED 0x00000080
+#define EFI_RESOURCE_ATTRIBUTE_WRITE_PROTECTED 0x00000100
+#define EFI_RESOURCE_ATTRIBUTE_EXECUTION_PROTECTED 0x00000200
+#define EFI_RESOURCE_ATTRIBUTE_UNCACHEABLE 0x00000400
+#define EFI_RESOURCE_ATTRIBUTE_WRITE_COMBINEABLE 0x00000800
+#define EFI_RESOURCE_ATTRIBUTE_WRITE_THROUGH_CACHEABLE 0x00001000
+#define EFI_RESOURCE_ATTRIBUTE_WRITE_BACK_CACHEABLE 0x00002000
+#define EFI_RESOURCE_ATTRIBUTE_16_BIT_IO 0x00004000
+#define EFI_RESOURCE_ATTRIBUTE_32_BIT_IO 0x00008000
+#define EFI_RESOURCE_ATTRIBUTE_64_BIT_IO 0x00010000
+#define EFI_RESOURCE_ATTRIBUTE_UNCACHED_EXPORTED 0x00020000
+#define EFI_RESOURCE_ATTRIBUTE_READ_ONLY_PROTECTED 0x00040000
+#define EFI_RESOURCE_ATTRIBUTE_READ_ONLY_PROTECTABLE 0x00080000
+#define EFI_RESOURCE_ATTRIBUTE_READ_PROTECTABLE 0x00100000
+#define EFI_RESOURCE_ATTRIBUTE_WRITE_PROTECTABLE 0x00200000
+#define EFI_RESOURCE_ATTRIBUTE_EXECUTION_PROTECTABLE 0x00400000
+#define EFI_RESOURCE_ATTRIBUTE_PERSISTENT 0x00800000
+#define EFI_RESOURCE_ATTRIBUTE_PERSISTABLE 0x01000000
+#define EFI_RESOURCE_ATTRIBUTE_MORE_RELIABLE 0x02000000
+
+typedef uint32_t EFI_RESOURCE_TYPE;
+typedef uint32_t EFI_RESOURCE_ATTRIBUTE_TYPE;
+
+typedef struct {
+ EFI_HOB_GENERIC_HEADER Header;
+ EFI_GUID Owner;
+ EFI_RESOURCE_TYPE ResourceType;
+ EFI_RESOURCE_ATTRIBUTE_TYPE ResourceAttribute;
+ EFI_PHYSICAL_ADDRESS PhysicalStart;
+ uint64_t ResourceLength;
+} EFI_HOB_RESOURCE_DESCRIPTOR;
+
+typedef struct {
+ EFI_HOB_GENERIC_HEADER Header;
+ EFI_GUID Name;
+
+ /* guid specific data follows */
+} EFI_HOB_GUID_TYPE;
+
+typedef struct {
+ EFI_HOB_GENERIC_HEADER Header;
+ EFI_PHYSICAL_ADDRESS BaseAddress;
+ uint64_t Length;
+} EFI_HOB_FIRMWARE_VOLUME;
+
+typedef struct {
+ EFI_HOB_GENERIC_HEADER Header;
+ EFI_PHYSICAL_ADDRESS BaseAddress;
+ uint64_t Length;
+ EFI_GUID FvName;
+ EFI_GUID FileName;
+} EFI_HOB_FIRMWARE_VOLUME2;
+
+typedef struct {
+ EFI_HOB_GENERIC_HEADER Header;
+ EFI_PHYSICAL_ADDRESS BaseAddress;
+ uint64_t Length;
+ uint32_t AuthenticationStatus;
+ bool ExtractedFv;
+ EFI_GUID FvName;
+ EFI_GUID FileName;
+} EFI_HOB_FIRMWARE_VOLUME3;
+
+typedef struct {
+ EFI_HOB_GENERIC_HEADER Header;
+ uint8_t SizeOfMemorySpace;
+ uint8_t SizeOfIoSpace;
+ uint8_t Reserved[6];
+} EFI_HOB_CPU;
+
+typedef struct {
+ EFI_HOB_GENERIC_HEADER Header;
+} EFI_HOB_MEMORY_POOL;
+
+typedef struct {
+ EFI_HOB_GENERIC_HEADER Header;
+
+ EFI_PHYSICAL_ADDRESS BaseAddress;
+ uint64_t Length;
+} EFI_HOB_UEFI_CAPSULE;
+
+#define EFI_HOB_OWNER_ZERO \
+ ((EFI_GUID){ 0x00000000, 0x0000, 0x0000, \
+ { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 } })
+
+#endif
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 44/65] i386/tdx: Setup the TD HOB list
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (42 preceding siblings ...)
2024-02-29 6:37 ` [PATCH v5 43/65] headers: Add definitions from UEFI spec for volumes, resources, etc Xiaoyao Li
@ 2024-02-29 6:37 ` Xiaoyao Li
2024-02-29 6:37 ` [PATCH v5 45/65] i386/tdx: Populate TDVF private memory via KVM_MEMORY_MAPPING Xiaoyao Li
` (20 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:37 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
The TD HOB list is used to pass the information from VMM to TDVF. The TD
HOB must include PHIT HOB and Resource Descriptor HOB. More details can
be found in TDVF specification and PI specification.
Build the TD HOB in TDX's machine_init_done callback.
Co-developed-by: Isaku Yamahata <isaku.yamahata@intel.com>
Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
Co-developed-by: Sean Christopherson <sean.j.christopherson@intel.com>
Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
Changes in v1:
- drop the code of adding mmio resources since OVMF prepares all the
MMIO hob itself.
---
hw/i386/meson.build | 2 +-
hw/i386/tdvf-hob.c | 147 ++++++++++++++++++++++++++++++++++++++++++
hw/i386/tdvf-hob.h | 24 +++++++
target/i386/kvm/tdx.c | 16 +++++
4 files changed, 188 insertions(+), 1 deletion(-)
create mode 100644 hw/i386/tdvf-hob.c
create mode 100644 hw/i386/tdvf-hob.h
diff --git a/hw/i386/meson.build b/hw/i386/meson.build
index f09441c1ea54..2345383940ab 100644
--- a/hw/i386/meson.build
+++ b/hw/i386/meson.build
@@ -28,7 +28,7 @@ i386_ss.add(when: 'CONFIG_PC', if_true: files(
'port92.c'))
i386_ss.add(when: 'CONFIG_X86_FW_OVMF', if_true: files('pc_sysfw_ovmf.c'),
if_false: files('pc_sysfw_ovmf-stubs.c'))
-i386_ss.add(when: 'CONFIG_TDX', if_true: files('tdvf.c'))
+i386_ss.add(when: 'CONFIG_TDX', if_true: files('tdvf.c', 'tdvf-hob.c'))
subdir('kvm')
subdir('xen')
diff --git a/hw/i386/tdvf-hob.c b/hw/i386/tdvf-hob.c
new file mode 100644
index 000000000000..0da6ff2df576
--- /dev/null
+++ b/hw/i386/tdvf-hob.c
@@ -0,0 +1,147 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0-or-later
+
+ * Copyright (c) 2020 Intel Corporation
+ * Author: Isaku Yamahata <isaku.yamahata at gmail.com>
+ * <isaku.yamahata at intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/log.h"
+#include "qemu/error-report.h"
+#include "e820_memory_layout.h"
+#include "hw/i386/pc.h"
+#include "hw/i386/x86.h"
+#include "hw/pci/pcie_host.h"
+#include "sysemu/kvm.h"
+#include "standard-headers/uefi/uefi.h"
+#include "tdvf-hob.h"
+
+typedef struct TdvfHob {
+ hwaddr hob_addr;
+ void *ptr;
+ int size;
+
+ /* working area */
+ void *current;
+ void *end;
+} TdvfHob;
+
+static uint64_t tdvf_current_guest_addr(const TdvfHob *hob)
+{
+ return hob->hob_addr + (hob->current - hob->ptr);
+}
+
+static void tdvf_align(TdvfHob *hob, size_t align)
+{
+ hob->current = QEMU_ALIGN_PTR_UP(hob->current, align);
+}
+
+static void *tdvf_get_area(TdvfHob *hob, uint64_t size)
+{
+ void *ret;
+
+ if (hob->current + size > hob->end) {
+ error_report("TD_HOB overrun, size = 0x%" PRIx64, size);
+ exit(1);
+ }
+
+ ret = hob->current;
+ hob->current += size;
+ tdvf_align(hob, 8);
+ return ret;
+}
+
+static void tdvf_hob_add_memory_resources(TdxGuest *tdx, TdvfHob *hob)
+{
+ EFI_HOB_RESOURCE_DESCRIPTOR *region;
+ EFI_RESOURCE_ATTRIBUTE_TYPE attr;
+ EFI_RESOURCE_TYPE resource_type;
+
+ TdxRamEntry *e;
+ int i;
+
+ for (i = 0; i < tdx->nr_ram_entries; i++) {
+ e = &tdx->ram_entries[i];
+
+ if (e->type == TDX_RAM_UNACCEPTED) {
+ resource_type = EFI_RESOURCE_MEMORY_UNACCEPTED;
+ attr = EFI_RESOURCE_ATTRIBUTE_TDVF_UNACCEPTED;
+ } else if (e->type == TDX_RAM_ADDED){
+ resource_type = EFI_RESOURCE_SYSTEM_MEMORY;
+ attr = EFI_RESOURCE_ATTRIBUTE_TDVF_PRIVATE;
+ } else {
+ error_report("unknown TDX_RAM_ENTRY type %d", e->type);
+ exit(1);
+ }
+
+ region = tdvf_get_area(hob, sizeof(*region));
+ *region = (EFI_HOB_RESOURCE_DESCRIPTOR) {
+ .Header = {
+ .HobType = EFI_HOB_TYPE_RESOURCE_DESCRIPTOR,
+ .HobLength = cpu_to_le16(sizeof(*region)),
+ .Reserved = cpu_to_le32(0),
+ },
+ .Owner = EFI_HOB_OWNER_ZERO,
+ .ResourceType = cpu_to_le32(resource_type),
+ .ResourceAttribute = cpu_to_le32(attr),
+ .PhysicalStart = cpu_to_le64(e->address),
+ .ResourceLength = cpu_to_le64(e->length),
+ };
+ }
+}
+
+void tdvf_hob_create(TdxGuest *tdx, TdxFirmwareEntry *td_hob)
+{
+ TdvfHob hob = {
+ .hob_addr = td_hob->address,
+ .size = td_hob->size,
+ .ptr = td_hob->mem_ptr,
+
+ .current = td_hob->mem_ptr,
+ .end = td_hob->mem_ptr + td_hob->size,
+ };
+
+ EFI_HOB_GENERIC_HEADER *last_hob;
+ EFI_HOB_HANDOFF_INFO_TABLE *hit;
+
+ /* Note, Efi{Free}Memory{Bottom,Top} are ignored, leave 'em zeroed. */
+ hit = tdvf_get_area(&hob, sizeof(*hit));
+ *hit = (EFI_HOB_HANDOFF_INFO_TABLE) {
+ .Header = {
+ .HobType = EFI_HOB_TYPE_HANDOFF,
+ .HobLength = cpu_to_le16(sizeof(*hit)),
+ .Reserved = cpu_to_le32(0),
+ },
+ .Version = cpu_to_le32(EFI_HOB_HANDOFF_TABLE_VERSION),
+ .BootMode = cpu_to_le32(0),
+ .EfiMemoryTop = cpu_to_le64(0),
+ .EfiMemoryBottom = cpu_to_le64(0),
+ .EfiFreeMemoryTop = cpu_to_le64(0),
+ .EfiFreeMemoryBottom = cpu_to_le64(0),
+ .EfiEndOfHobList = cpu_to_le64(0), /* initialized later */
+ };
+
+ tdvf_hob_add_memory_resources(tdx, &hob);
+
+ last_hob = tdvf_get_area(&hob, sizeof(*last_hob));
+ *last_hob = (EFI_HOB_GENERIC_HEADER) {
+ .HobType = EFI_HOB_TYPE_END_OF_HOB_LIST,
+ .HobLength = cpu_to_le16(sizeof(*last_hob)),
+ .Reserved = cpu_to_le32(0),
+ };
+ hit->EfiEndOfHobList = tdvf_current_guest_addr(&hob);
+}
diff --git a/hw/i386/tdvf-hob.h b/hw/i386/tdvf-hob.h
new file mode 100644
index 000000000000..1b737e946a8d
--- /dev/null
+++ b/hw/i386/tdvf-hob.h
@@ -0,0 +1,24 @@
+#ifndef HW_I386_TD_HOB_H
+#define HW_I386_TD_HOB_H
+
+#include "hw/i386/tdvf.h"
+#include "target/i386/kvm/tdx.h"
+
+void tdvf_hob_create(TdxGuest *tdx, TdxFirmwareEntry *td_hob);
+
+#define EFI_RESOURCE_ATTRIBUTE_TDVF_PRIVATE \
+ (EFI_RESOURCE_ATTRIBUTE_PRESENT | \
+ EFI_RESOURCE_ATTRIBUTE_INITIALIZED | \
+ EFI_RESOURCE_ATTRIBUTE_TESTED)
+
+#define EFI_RESOURCE_ATTRIBUTE_TDVF_UNACCEPTED \
+ (EFI_RESOURCE_ATTRIBUTE_PRESENT | \
+ EFI_RESOURCE_ATTRIBUTE_INITIALIZED | \
+ EFI_RESOURCE_ATTRIBUTE_TESTED)
+
+#define EFI_RESOURCE_ATTRIBUTE_TDVF_MMIO \
+ (EFI_RESOURCE_ATTRIBUTE_PRESENT | \
+ EFI_RESOURCE_ATTRIBUTE_INITIALIZED | \
+ EFI_RESOURCE_ATTRIBUTE_UNCACHEABLE)
+
+#endif
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 98b2cfd40651..fb9c60172fde 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -25,6 +25,7 @@
#include "hw/i386/e820_memory_layout.h"
#include "hw/i386/x86.h"
#include "hw/i386/tdvf.h"
+#include "hw/i386/tdvf-hob.h"
#include "kvm_i386.h"
#include "tdx.h"
#include "../cpu-internal.h"
@@ -473,6 +474,19 @@ void tdx_set_tdvf_region(MemoryRegion *tdvf_mr)
tdx_guest->tdvf_mr = tdvf_mr;
}
+static TdxFirmwareEntry *tdx_get_hob_entry(TdxGuest *tdx)
+{
+ TdxFirmwareEntry *entry;
+
+ for_each_tdx_fw_entry(&tdx->tdvf, entry) {
+ if (entry->type == TDVF_SECTION_TYPE_TD_HOB) {
+ return entry;
+ }
+ }
+ error_report("TDVF metadata doesn't specify TD_HOB location.");
+ exit(1);
+}
+
static void tdx_add_ram_entry(uint64_t address, uint64_t length,
enum TdxRamType type)
{
@@ -604,6 +618,8 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused)
qsort(tdx_guest->ram_entries, tdx_guest->nr_ram_entries,
sizeof(TdxRamEntry), &tdx_ram_entry_compare);
+
+ tdvf_hob_create(tdx_guest, tdx_get_hob_entry(tdx_guest));
}
static Notifier tdx_machine_done_notify = {
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 45/65] i386/tdx: Populate TDVF private memory via KVM_MEMORY_MAPPING
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (43 preceding siblings ...)
2024-02-29 6:37 ` [PATCH v5 44/65] i386/tdx: Setup the TD HOB list Xiaoyao Li
@ 2024-02-29 6:37 ` Xiaoyao Li
2024-02-29 6:37 ` [PATCH v5 46/65] i386/tdx: Call KVM_TDX_INIT_VCPU to initialize TDX vcpu Xiaoyao Li
` (19 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:37 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
From: Isaku Yamahata <isaku.yamahata@intel.com>
TDVF firmware (CODE and VARS) needs to be copied to TD's private
memory, as well as TD HOB and TEMP memory.
If the TDVF section has TDVF_SECTION_ATTRIBUTES_MR_EXTEND set in the
flag, calling KVM_TDX_EXTEND_MEMORY to extend the measurement.
After populating the TDVF memory, the original image located in shared
ramblock can be discarded.
Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
Changes in v1:
- rename variable @metadata to @flags
---
target/i386/kvm/tdx.c | 47 +++++++++++++++++++++++++++++++++++++++++++
1 file changed, 47 insertions(+)
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index fb9c60172fde..dcabe359eda5 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -595,6 +595,8 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused)
{
TdxFirmware *tdvf = &tdx_guest->tdvf;
TdxFirmwareEntry *entry;
+ RAMBlock *ram_block;
+ int r;
tdx_init_ram_entries();
@@ -620,6 +622,51 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused)
sizeof(TdxRamEntry), &tdx_ram_entry_compare);
tdvf_hob_create(tdx_guest, tdx_get_hob_entry(tdx_guest));
+
+ for_each_tdx_fw_entry(tdvf, entry) {
+ struct kvm_memory_mapping mapping = {
+ .base_gfn = entry->address >> 12,
+ .nr_pages = entry->size >> 12,
+ .source = (__u64)entry->mem_ptr,
+ };
+
+ do {
+ r = kvm_vcpu_ioctl(first_cpu, KVM_MEMORY_MAPPING, &mapping);
+ } while (r == -EAGAIN);
+
+ if (r < 0) {
+ error_report("KVM_MEMORY_MAPPING failed %s", strerror(-r));
+ exit(1);
+ }
+
+ if (entry->attributes & TDVF_SECTION_ATTRIBUTES_MR_EXTEND) {
+ mapping = (struct kvm_memory_mapping) {
+ .base_gfn = entry->address >> 12,
+ .nr_pages = entry->size >> 12,
+ };
+
+ do {
+ r = tdx_vm_ioctl(KVM_TDX_EXTEND_MEMORY, 0, &mapping);
+ } while (r == -EAGAIN);
+ if (r < 0) {
+ error_report("KVM_TDX_EXTEND_MEMORY failed %s", strerror(-r));
+ exit(1);
+ }
+ }
+
+ if (entry->type == TDVF_SECTION_TYPE_TD_HOB ||
+ entry->type == TDVF_SECTION_TYPE_TEMP_MEM) {
+ qemu_ram_munmap(-1, entry->mem_ptr, entry->size);
+ entry->mem_ptr = NULL;
+ }
+ }
+
+ /*
+ * TDVF image has been copied into private region above via
+ * KVM_MEMORY_MAPPING. It becomes useless.
+ */
+ ram_block = tdx_guest->tdvf_mr->ram_block;
+ ram_block_discard_range(ram_block, 0, ram_block->max_length);
}
static Notifier tdx_machine_done_notify = {
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 46/65] i386/tdx: Call KVM_TDX_INIT_VCPU to initialize TDX vcpu
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (44 preceding siblings ...)
2024-02-29 6:37 ` [PATCH v5 45/65] i386/tdx: Populate TDVF private memory via KVM_MEMORY_MAPPING Xiaoyao Li
@ 2024-02-29 6:37 ` Xiaoyao Li
2024-02-29 6:37 ` [PATCH v5 47/65] i386/tdx: Finalize TDX VM Xiaoyao Li
` (18 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:37 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
TDX vcpu needs to be initialized by SEAMCALL(TDH.VP.INIT) and KVM
provides vcpu level IOCTL KVM_TDX_INIT_VCPU for it.
KVM_TDX_INIT_VCPU needs the address of the HOB as input. Invoke it for
each vcpu after HOB list is created.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
target/i386/kvm/tdx.c | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index dcabe359eda5..4625f806920e 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -591,6 +591,22 @@ static void tdx_init_ram_entries(void)
tdx_guest->nr_ram_entries = j;
}
+static void tdx_post_init_vcpus(void)
+{
+ TdxFirmwareEntry *hob;
+ CPUState *cpu;
+ int r;
+
+ hob = tdx_get_hob_entry(tdx_guest);
+ CPU_FOREACH(cpu) {
+ r = tdx_vcpu_ioctl(cpu, KVM_TDX_INIT_VCPU, 0, (void *)hob->address);
+ if (r < 0) {
+ error_report("KVM_TDX_INIT_VCPU failed %s", strerror(-r));
+ exit(1);
+ }
+ }
+}
+
static void tdx_finalize_vm(Notifier *notifier, void *unused)
{
TdxFirmware *tdvf = &tdx_guest->tdvf;
@@ -623,6 +639,8 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused)
tdvf_hob_create(tdx_guest, tdx_get_hob_entry(tdx_guest));
+ tdx_post_init_vcpus();
+
for_each_tdx_fw_entry(tdvf, entry) {
struct kvm_memory_mapping mapping = {
.base_gfn = entry->address >> 12,
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 47/65] i386/tdx: Finalize TDX VM
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (45 preceding siblings ...)
2024-02-29 6:37 ` [PATCH v5 46/65] i386/tdx: Call KVM_TDX_INIT_VCPU to initialize TDX vcpu Xiaoyao Li
@ 2024-02-29 6:37 ` Xiaoyao Li
2024-02-29 6:37 ` [PATCH v5 48/65] i386/tdx: handle TDG.VP.VMCALL<SetupEventNotifyInterrupt> Xiaoyao Li
` (17 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:37 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
Invoke KVM_TDX_FINALIZE_VM to finalize the TD's measurement and make
the TD vCPUs runnable once machine initialization is complete.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
target/i386/kvm/tdx.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 4625f806920e..d445d4b70f77 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -685,6 +685,13 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused)
*/
ram_block = tdx_guest->tdvf_mr->ram_block;
ram_block_discard_range(ram_block, 0, ram_block->max_length);
+
+ r = tdx_vm_ioctl(KVM_TDX_FINALIZE_VM, 0, NULL);
+ if (r < 0) {
+ error_report("KVM_TDX_FINALIZE_VM failed %s", strerror(-r));
+ exit(0);
+ }
+ tdx_guest->parent_obj.ready = true;
}
static Notifier tdx_machine_done_notify = {
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 48/65] i386/tdx: handle TDG.VP.VMCALL<SetupEventNotifyInterrupt>
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (46 preceding siblings ...)
2024-02-29 6:37 ` [PATCH v5 47/65] i386/tdx: Finalize TDX VM Xiaoyao Li
@ 2024-02-29 6:37 ` Xiaoyao Li
2024-02-29 6:37 ` [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote> Xiaoyao Li
` (16 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:37 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
From: Isaku Yamahata <isaku.yamahata@intel.com>
For SetupEventNotifyInterrupt, record interrupt vector and the apic id
of the vcpu that received this TDVMCALL.
Later it can inject interrupt with given vector to the specific vcpu
that received SetupEventNotifyInterrupt.
Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
target/i386/kvm/kvm.c | 8 ++++++
target/i386/kvm/tdx-stub.c | 5 ++++
target/i386/kvm/tdx.c | 53 ++++++++++++++++++++++++++++++++++++++
target/i386/kvm/tdx.h | 14 ++++++++++
4 files changed, 80 insertions(+)
diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index 4f998b2d6d37..2748086231d5 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -5413,6 +5413,14 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run)
ret = kvm_xen_handle_exit(cpu, &run->xen);
break;
#endif
+ case KVM_EXIT_TDX:
+ if (!is_tdx_vm()) {
+ error_report("KVM: get KVM_EXIT_TDX for a non-TDX VM.");
+ ret = -1;
+ break;
+ }
+ ret = tdx_handle_exit(cpu, &run->tdx);
+ break;
default:
fprintf(stderr, "KVM: unknown exit reason %d\n", run->exit_reason);
ret = -1;
diff --git a/target/i386/kvm/tdx-stub.c b/target/i386/kvm/tdx-stub.c
index a064d583d393..57cd25793842 100644
--- a/target/i386/kvm/tdx-stub.c
+++ b/target/i386/kvm/tdx-stub.c
@@ -11,3 +11,8 @@ int tdx_parse_tdvf(void *flash_ptr, int size)
{
return -EINVAL;
}
+
+int tdx_handle_exit(X86CPU *cpu, struct kvm_tdx_exit *tdx_exit)
+{
+ return -EINVAL;
+}
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index d445d4b70f77..49f94d9d46f4 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -866,6 +866,56 @@ int tdx_parse_tdvf(void *flash_ptr, int size)
return tdvf_parse_metadata(&tdx_guest->tdvf, flash_ptr, size);
}
+static int tdx_handle_setup_event_notify_interrupt(X86CPU *cpu,
+ struct kvm_tdx_vmcall *vmcall)
+{
+ int vector = vmcall->in_r12;
+
+ if (32 <= vector && vector <= 255) {
+ qemu_mutex_lock(&tdx_guest->lock);
+ tdx_guest->event_notify_vector = vector;
+ tdx_guest->event_notify_apicid = cpu->apic_id;
+ qemu_mutex_unlock(&tdx_guest->lock);
+ vmcall->status_code = TDG_VP_VMCALL_SUCCESS;
+ } else {
+ vmcall->status_code = TDG_VP_VMCALL_INVALID_OPERAND;
+ }
+
+ return 0;
+}
+
+static int tdx_handle_vmcall(X86CPU *cpu, struct kvm_tdx_vmcall *vmcall)
+{
+ vmcall->status_code = TDG_VP_VMCALL_INVALID_OPERAND;
+
+ /* For now handle only TDG.VP.VMCALL leaf defined in TDX GHCI */
+ if (vmcall->type != 0) {
+ error_report("Unknown TDG.VP.VMCALL type 0x%llx subfunction 0x%llx",
+ vmcall->type, vmcall->subfunction);
+ return -1;
+ }
+
+ switch (vmcall->subfunction) {
+ case TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT:
+ return tdx_handle_setup_event_notify_interrupt(cpu, vmcall);
+ default:
+ error_report("Unknown TDG.VP.VMCALL type 0x%llx subfunction 0x%llx",
+ vmcall->type, vmcall->subfunction);
+ return -1;
+ }
+}
+
+int tdx_handle_exit(X86CPU *cpu, struct kvm_tdx_exit *tdx_exit)
+{
+ switch (tdx_exit->type) {
+ case KVM_EXIT_TDX_VMCALL:
+ return tdx_handle_vmcall(cpu, &tdx_exit->u.vmcall);
+ default:
+ error_report("unknown tdx exit type 0x%x", tdx_exit->type);
+ return -1;
+ }
+}
+
static bool tdx_guest_get_sept_ve_disable(Object *obj, Error **errp)
{
TdxGuest *tdx = TDX_GUEST(obj);
@@ -956,6 +1006,9 @@ static void tdx_guest_init(Object *obj)
object_property_add_str(obj, "mrownerconfig",
tdx_guest_get_mrownerconfig,
tdx_guest_set_mrownerconfig);
+
+ tdx->event_notify_vector = -1;
+ tdx->event_notify_apicid = -1;
}
static void tdx_guest_finalize(Object *obj)
diff --git a/target/i386/kvm/tdx.h b/target/i386/kvm/tdx.h
index 3fb4069268f6..b3d4462fe718 100644
--- a/target/i386/kvm/tdx.h
+++ b/target/i386/kvm/tdx.h
@@ -7,6 +7,7 @@
#include "exec/confidential-guest-support.h"
#include "hw/i386/tdvf.h"
+#include "sysemu/kvm.h"
#define TYPE_TDX_GUEST "tdx-guest"
#define TDX_GUEST(obj) OBJECT_CHECK(TdxGuest, (obj), TYPE_TDX_GUEST)
@@ -15,6 +16,14 @@ typedef struct TdxGuestClass {
ConfidentialGuestSupportClass parent_class;
} TdxGuestClass;
+#define TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT 0x10004ULL
+
+#define TDG_VP_VMCALL_SUCCESS 0x0000000000000000ULL
+#define TDG_VP_VMCALL_RETRY 0x0000000000000001ULL
+#define TDG_VP_VMCALL_INVALID_OPERAND 0x8000000000000000ULL
+#define TDG_VP_VMCALL_GPA_INUSE 0x8000000000000001ULL
+#define TDG_VP_VMCALL_ALIGN_ERROR 0x8000000000000002ULL
+
enum TdxRamType{
TDX_RAM_UNACCEPTED,
TDX_RAM_ADDED,
@@ -42,6 +51,10 @@ typedef struct TdxGuest {
uint32_t nr_ram_entries;
TdxRamEntry *ram_entries;
+
+ /* runtime state */
+ uint32_t event_notify_vector;
+ uint32_t event_notify_apicid;
} TdxGuest;
#ifdef CONFIG_TDX
@@ -55,5 +68,6 @@ void tdx_get_supported_cpuid(uint32_t function, uint32_t index, int reg,
int tdx_pre_create_vcpu(CPUState *cpu, Error **errp);
void tdx_set_tdvf_region(MemoryRegion *tdvf_mr);
int tdx_parse_tdvf(void *flash_ptr, int size);
+int tdx_handle_exit(X86CPU *cpu, struct kvm_tdx_exit *tdx_exit);
#endif /* QEMU_I386_TDX_H */
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (47 preceding siblings ...)
2024-02-29 6:37 ` [PATCH v5 48/65] i386/tdx: handle TDG.VP.VMCALL<SetupEventNotifyInterrupt> Xiaoyao Li
@ 2024-02-29 6:37 ` Xiaoyao Li
2024-02-29 8:40 ` Markus Armbruster
` (2 more replies)
2024-02-29 6:37 ` [PATCH v5 50/65] i386/tdx: handle TDG.VP.VMCALL<MapGPA> hypercall Xiaoyao Li
` (15 subsequent siblings)
64 siblings, 3 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:37 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
From: Isaku Yamahata <isaku.yamahata@intel.com>
Add property "quote-generation-socket" to tdx-guest, which is a property
of type SocketAddress to specify Quote Generation Service(QGS).
On request of GetQuote, it connects to the QGS socket, read request
data from shared guest memory, send the request data to the QGS,
and store the response into shared guest memory, at last notify
TD guest by interrupt.
command line example:
qemu-system-x86_64 \
-object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \
-machine confidential-guest-support=tdx0
Note, above example uses vsock type socket because the QGS we used
implements the vsock socket. It can be other types, like UNIX socket,
which depends on the implementation of QGS.
To avoid no response from QGS server, setup a timer for the transaction.
If timeout, make it an error and interrupt guest. Define the threshold of
time to 30s at present, maybe change to other value if not appropriate.
Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
Codeveloped-by: Chenyi Qiang <chenyi.qiang@intel.com>
Signed-off-by: Chenyi Qiang <chenyi.qiang@intel.com>
Codeveloped-by: Xiaoyao Li <xiaoyao.li@intel.com>
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
Changes in v5:
- add more decription of quote-generation-socket property;
Changes in v4:
- merge next patch "i386/tdx: setup a timer for the qio channel";
Changes in v3:
- rename property "quote-generation-service" to "quote-generation-socket";
- change the type of "quote-generation-socket" from str to
SocketAddress;
- squash next patch into this one;
---
qapi/qom.json | 8 +-
target/i386/kvm/meson.build | 2 +-
target/i386/kvm/tdx-quote-generator.c | 170 ++++++++++++++++++++
target/i386/kvm/tdx-quote-generator.h | 95 +++++++++++
target/i386/kvm/tdx.c | 216 ++++++++++++++++++++++++++
target/i386/kvm/tdx.h | 6 +
6 files changed, 495 insertions(+), 2 deletions(-)
create mode 100644 target/i386/kvm/tdx-quote-generator.c
create mode 100644 target/i386/kvm/tdx-quote-generator.h
diff --git a/qapi/qom.json b/qapi/qom.json
index cac875349a3a..7b26b0a0d3aa 100644
--- a/qapi/qom.json
+++ b/qapi/qom.json
@@ -917,13 +917,19 @@
# (base64 encoded SHA384 digest). (A default value 0 of SHA384 is
# used when absent).
#
+# @quote-generation-socket: socket address for Quote Generation
+# Service (QGS). QGS is a daemon running on the host. User in
+# TD guest cannot get TD quoting for attestation if QGS is not
+# provided. So admin should always provide it.
+#
# Since: 9.0
##
{ 'struct': 'TdxGuestProperties',
'data': { '*sept-ve-disable': 'bool',
'*mrconfigid': 'str',
'*mrowner': 'str',
- '*mrownerconfig': 'str' } }
+ '*mrownerconfig': 'str',
+ '*quote-generation-socket': 'SocketAddress' } }
##
# @ThreadContextProperties:
diff --git a/target/i386/kvm/meson.build b/target/i386/kvm/meson.build
index 460c5f8f85f3..d38b8e8e5fca 100644
--- a/target/i386/kvm/meson.build
+++ b/target/i386/kvm/meson.build
@@ -7,7 +7,7 @@ i386_kvm_ss.add(files(
i386_kvm_ss.add(when: 'CONFIG_XEN_EMU', if_true: files('xen-emu.c'))
-i386_kvm_ss.add(when: 'CONFIG_TDX', if_true: files('tdx.c'), if_false: files('tdx-stub.c'))
+i386_kvm_ss.add(when: 'CONFIG_TDX', if_true: files('tdx.c', 'tdx-quote-generator.c'), if_false: files('tdx-stub.c'))
i386_system_ss.add(when: 'CONFIG_HYPERV', if_true: files('hyperv.c'), if_false: files('hyperv-stub.c'))
diff --git a/target/i386/kvm/tdx-quote-generator.c b/target/i386/kvm/tdx-quote-generator.c
new file mode 100644
index 000000000000..057ad09e7e95
--- /dev/null
+++ b/target/i386/kvm/tdx-quote-generator.c
@@ -0,0 +1,170 @@
+/*
+ * QEMU TDX support
+ *
+ * Copyright Intel
+ *
+ * Author:
+ * Xiaoyao Li <xiaoyao.li@intel.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/error-report.h"
+#include "qapi/error.h"
+#include "qapi/qapi-visit-sockets.h"
+
+#include "tdx-quote-generator.h"
+
+typedef struct TdxQuoteGeneratorClass {
+ DeviceClass parent_class;
+} TdxQuoteGeneratorClass;
+
+OBJECT_DEFINE_TYPE(TdxQuoteGenerator, tdx_quote_generator, TDX_QUOTE_GENERATOR, OBJECT)
+
+static void tdx_quote_generator_finalize(Object *obj)
+{
+}
+
+static void tdx_quote_generator_class_init(ObjectClass *oc, void *data)
+{
+}
+
+static void tdx_quote_generator_init(Object *obj)
+{
+}
+
+static void tdx_generate_quote_cleanup(struct tdx_generate_quote_task *task)
+{
+ timer_del(&task->timer);
+
+ g_source_remove(task->watch);
+ qio_channel_close(QIO_CHANNEL(task->sioc), NULL);
+ object_unref(OBJECT(task->sioc));
+
+ /* Maintain the number of in-flight requests. */
+ qemu_mutex_lock(&task->quote_gen->lock);
+ task->quote_gen->num--;
+ qemu_mutex_unlock(&task->quote_gen->lock);
+
+ task->completion(task);
+}
+
+static gboolean tdx_get_quote_read(QIOChannel *ioc, GIOCondition condition,
+ gpointer opaque)
+{
+ struct tdx_generate_quote_task *task = opaque;
+ Error *err = NULL;
+ int ret;
+
+ ret = qio_channel_read(ioc, task->receive_buf + task->receive_buf_received,
+ task->payload_len - task->receive_buf_received, &err);
+ if (ret < 0) {
+ if (ret == QIO_CHANNEL_ERR_BLOCK) {
+ return G_SOURCE_CONTINUE;
+ } else {
+ error_report_err(err);
+ task->status_code = TDX_VP_GET_QUOTE_ERROR;
+ goto end;
+ }
+ }
+
+ task->receive_buf_received += ret;
+ if (ret == 0 || task->receive_buf_received == task->payload_len) {
+ task->status_code = TDX_VP_GET_QUOTE_SUCCESS;
+ goto end;
+ }
+
+ return G_SOURCE_CONTINUE;
+
+end:
+ tdx_generate_quote_cleanup(task);
+ return G_SOURCE_REMOVE;
+}
+
+static gboolean tdx_send_report(QIOChannel *ioc, GIOCondition condition,
+ gpointer opaque)
+{
+ struct tdx_generate_quote_task *task = opaque;
+ Error *err = NULL;
+ int ret;
+
+ ret = qio_channel_write(ioc, task->send_data + task->send_data_sent,
+ task->send_data_size - task->send_data_sent, &err);
+ if (ret < 0) {
+ if (ret == QIO_CHANNEL_ERR_BLOCK) {
+ ret = 0;
+ } else {
+ error_report_err(err);
+ task->status_code = TDX_VP_GET_QUOTE_ERROR;
+ tdx_generate_quote_cleanup(task);
+ goto end;
+ }
+ }
+ task->send_data_sent += ret;
+
+ if (task->send_data_sent == task->send_data_size) {
+ task->watch = qio_channel_add_watch(QIO_CHANNEL(task->sioc), G_IO_IN,
+ tdx_get_quote_read, task, NULL);
+ goto end;
+ }
+
+ return G_SOURCE_CONTINUE;
+
+end:
+ return G_SOURCE_REMOVE;
+}
+
+static void tdx_quote_generator_connected(QIOTask *qio_task, gpointer opaque)
+{
+ struct tdx_generate_quote_task *task = opaque;
+ Error *err = NULL;
+ int ret;
+
+ ret = qio_task_propagate_error(qio_task, &err);
+ if (ret) {
+ error_report_err(err);
+ task->status_code = TDX_VP_GET_QUOTE_QGS_UNAVAILABLE;
+ tdx_generate_quote_cleanup(task);
+ return;
+ }
+
+ task->watch = qio_channel_add_watch(QIO_CHANNEL(task->sioc), G_IO_OUT,
+ tdx_send_report, task, NULL);
+}
+
+#define TRANSACTION_TIMEOUT 30000
+
+static void getquote_expired(void *opaque)
+{
+ struct tdx_generate_quote_task *task = opaque;
+
+ task->status_code = TDX_VP_GET_QUOTE_ERROR;
+ tdx_generate_quote_cleanup(task);
+}
+
+static void setup_get_quote_timer(struct tdx_generate_quote_task *task)
+{
+ int64_t time;
+
+ timer_init_ms(&task->timer, QEMU_CLOCK_VIRTUAL, getquote_expired, task);
+ time = qemu_clock_get_ms(QEMU_CLOCK_VIRTUAL);
+ timer_mod(&task->timer, time + TRANSACTION_TIMEOUT);
+}
+
+void tdx_generate_quote(struct tdx_generate_quote_task *task)
+{
+ struct TdxQuoteGenerator *quote_gen = task->quote_gen;
+ QIOChannelSocket *sioc;
+
+ sioc = qio_channel_socket_new();
+ task->sioc = sioc;
+
+ setup_get_quote_timer(task);
+
+ qio_channel_socket_connect_async(sioc, quote_gen->socket,
+ tdx_quote_generator_connected, task,
+ NULL, NULL);
+}
diff --git a/target/i386/kvm/tdx-quote-generator.h b/target/i386/kvm/tdx-quote-generator.h
new file mode 100644
index 000000000000..54899d44aa6f
--- /dev/null
+++ b/target/i386/kvm/tdx-quote-generator.h
@@ -0,0 +1,95 @@
+#ifndef QEMU_I386_TDX_QUOTE_GENERATOR_H
+#define QEMU_I386_TDX_QUOTE_GENERATOR_H
+
+#include "qom/object_interfaces.h"
+#include "io/channel-socket.h"
+#include "exec/hwaddr.h"
+
+/* tdx quote generation */
+struct TdxQuoteGenerator {
+ Object parent_obj;
+
+ int num;
+ SocketAddress *socket;
+
+ QemuMutex lock;
+};
+
+#define TYPE_TDX_QUOTE_GENERATOR "tdx-quote-generator"
+
+OBJECT_DECLARE_SIMPLE_TYPE(TdxQuoteGenerator, TDX_QUOTE_GENERATOR)
+
+
+#define TDX_GET_QUOTE_STRUCTURE_VERSION 1ULL
+
+#define TDX_VP_GET_QUOTE_SUCCESS 0ULL
+#define TDX_VP_GET_QUOTE_IN_FLIGHT (-1ULL)
+#define TDX_VP_GET_QUOTE_ERROR 0x8000000000000000ULL
+#define TDX_VP_GET_QUOTE_QGS_UNAVAILABLE 0x8000000000000001ULL
+
+/* Limit to avoid resource starvation. */
+#define TDX_GET_QUOTE_MAX_BUF_LEN (128 * 1024)
+#define TDX_MAX_GET_QUOTE_REQUEST 16
+
+#define TDX_GET_QUOTE_HDR_SIZE 24
+
+/* Format of pages shared with guest. */
+struct tdx_get_quote_header {
+ /* Format version: must be 1 in little endian. */
+ uint64_t structure_version;
+
+ /*
+ * GetQuote status code in little endian:
+ * Guest must set error_code to 0 to avoid information leak.
+ * Qemu sets this before interrupting guest.
+ */
+ uint64_t error_code;
+
+ /*
+ * in-message size in little endian: The message will follow this header.
+ * The in-message will be send to QGS.
+ */
+ uint32_t in_len;
+
+ /*
+ * out-message size in little endian:
+ * On request, out_len must be zero to avoid information leak.
+ * On return, message size from QGS. Qemu overwrites this field.
+ * The message will follows this header. The in-message is overwritten.
+ */
+ uint32_t out_len;
+
+ /*
+ * Message buffer follows.
+ * Guest sets message that will be send to QGS. If out_len > in_len, guest
+ * should zero remaining buffer to avoid information leak.
+ * Qemu overwrites this buffer with a message returned from QGS.
+ */
+};
+
+struct tdx_generate_quote_task {
+ hwaddr buf_gpa;
+ hwaddr payload_gpa;
+ uint64_t payload_len;
+
+ char *send_data;
+ uint64_t send_data_size;
+ uint64_t send_data_sent;
+
+ char *receive_buf;
+ uint64_t receive_buf_received;
+
+ uint64_t status_code;
+ struct tdx_get_quote_header hdr;
+
+ QIOChannelSocket *sioc;
+ guint watch;
+ QEMUTimer timer;
+ struct TdxQuoteGenerator *quote_gen;
+
+ void (*completion)(struct tdx_generate_quote_task *task);
+};
+
+void tdx_generate_quote(struct tdx_generate_quote_task *task);
+
+#endif /* QEMU_I386_TDX_QUOTE_GENERATOR_H */
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 49f94d9d46f4..7dfda507cc8c 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -16,18 +16,23 @@
#include "qemu/base64.h"
#include "qemu/mmap-alloc.h"
#include "qapi/error.h"
+#include "qapi/qapi-visit-sockets.h"
#include "qom/object_interfaces.h"
#include "standard-headers/asm-x86/kvm_para.h"
#include "sysemu/kvm.h"
#include "sysemu/sysemu.h"
#include "exec/ramblock.h"
+#include "hw/i386/apic_internal.h"
+#include "hw/i386/apic-msidef.h"
#include "hw/i386/e820_memory_layout.h"
#include "hw/i386/x86.h"
#include "hw/i386/tdvf.h"
#include "hw/i386/tdvf-hob.h"
+#include "hw/pci/msi.h"
#include "kvm_i386.h"
#include "tdx.h"
+#include "tdx-quote-generator.h"
#include "../cpu-internal.h"
#define TDX_SUPPORTED_KVM_FEATURES ((1U << KVM_FEATURE_NOP_IO_DELAY) | \
@@ -866,6 +871,175 @@ int tdx_parse_tdvf(void *flash_ptr, int size)
return tdvf_parse_metadata(&tdx_guest->tdvf, flash_ptr, size);
}
+static void tdx_inject_interrupt(uint32_t apicid, uint32_t vector)
+{
+ int ret;
+
+ if (vector < 32 || vector > 255) {
+ return;
+ }
+
+ MSIMessage msg = {
+ .address = ((apicid & 0xff) << MSI_ADDR_DEST_ID_SHIFT) |
+ (((uint64_t)apicid & 0xffffff00) << 32),
+ .data = vector | (APIC_DM_FIXED << MSI_DATA_DELIVERY_MODE_SHIFT),
+ };
+
+ ret = kvm_irqchip_send_msi(kvm_state, msg);
+ if (ret < 0) {
+ /* In this case, no better way to tell it to guest. Log it. */
+ error_report("TDX: injection %d failed, interrupt lost (%s).\n",
+ vector, strerror(-ret));
+ }
+}
+
+static hwaddr tdx_shared_bit(X86CPU *cpu)
+{
+ return (cpu->phys_bits > 48) ? BIT_ULL(51) : BIT_ULL(47);
+}
+
+static void tdx_get_quote_completion(struct tdx_generate_quote_task *task)
+{
+ int ret;
+
+ if (task->status_code == TDX_VP_GET_QUOTE_SUCCESS) {
+ ret = address_space_write(&address_space_memory, task->payload_gpa,
+ MEMTXATTRS_UNSPECIFIED, task->receive_buf,
+ task->receive_buf_received);
+ if (ret != MEMTX_OK) {
+ error_report("TDX: get-quote: failed to write quote data.\n");
+ } else {
+ task->hdr.out_len = cpu_to_le64(task->receive_buf_received);
+ }
+ }
+ task->hdr.error_code = cpu_to_le32(task->status_code);
+
+ /* Publish the response contents before marking this request completed. */
+ smp_wmb();
+ ret = address_space_write(&address_space_memory, task->buf_gpa,
+ MEMTXATTRS_UNSPECIFIED, &task->hdr,
+ TDX_GET_QUOTE_HDR_SIZE);
+ if (ret != MEMTX_OK) {
+ error_report("TDX: get-quote: failed to update GetQuote header.");
+ }
+
+ tdx_inject_interrupt(tdx_guest->event_notify_apicid,
+ tdx_guest->event_notify_vector);
+
+ g_free(task->send_data);
+ g_free(task->receive_buf);
+ g_free(task);
+}
+
+static int tdx_handle_get_quote(X86CPU *cpu, struct kvm_tdx_vmcall *vmcall)
+{
+ struct tdx_generate_quote_task *task;
+ struct tdx_get_quote_header hdr;
+ hwaddr buf_gpa = vmcall->in_r12;
+ uint64_t buf_len = vmcall->in_r13;
+
+ QEMU_BUILD_BUG_ON(sizeof(struct tdx_get_quote_header) != TDX_GET_QUOTE_HDR_SIZE);
+
+ vmcall->status_code = TDG_VP_VMCALL_INVALID_OPERAND;
+
+ if (buf_len == 0) {
+ return 0;
+ }
+
+ /* GPA must be shared. */
+ if (!(buf_gpa & tdx_shared_bit(cpu))) {
+ return 0;
+ }
+ buf_gpa &= ~tdx_shared_bit(cpu);
+
+ if (!QEMU_IS_ALIGNED(buf_gpa, 4096) || !QEMU_IS_ALIGNED(buf_len, 4096)) {
+ vmcall->status_code = TDG_VP_VMCALL_ALIGN_ERROR;
+ return 0;
+ }
+
+ if (address_space_read(&address_space_memory, buf_gpa, MEMTXATTRS_UNSPECIFIED,
+ &hdr, TDX_GET_QUOTE_HDR_SIZE) != MEMTX_OK) {
+ error_report("TDX: get-quote: failed to read GetQuote header.\n");
+ return -1;
+ }
+
+ if (le64_to_cpu(hdr.structure_version) != TDX_GET_QUOTE_STRUCTURE_VERSION) {
+ return 0;
+ }
+
+ /*
+ * Paranoid: Guest should clear error_code and out_len to avoid information
+ * leak. Enforce it. The initial value of them doesn't matter for qemu to
+ * process the request.
+ */
+ if (le64_to_cpu(hdr.error_code) != TDX_VP_GET_QUOTE_SUCCESS ||
+ le32_to_cpu(hdr.out_len) != 0) {
+ return 0;
+ }
+
+ /* Only safe-guard check to avoid too large buffer size. */
+ if (buf_len > TDX_GET_QUOTE_MAX_BUF_LEN ||
+ le32_to_cpu(hdr.in_len) > buf_len - TDX_GET_QUOTE_HDR_SIZE) {
+ return 0;
+ }
+
+ vmcall->status_code = TDG_VP_VMCALL_SUCCESS;
+ if (!tdx_guest->quote_generator) {
+ hdr.error_code = cpu_to_le64(TDX_VP_GET_QUOTE_QGS_UNAVAILABLE);
+ if (address_space_write(&address_space_memory, buf_gpa,
+ MEMTXATTRS_UNSPECIFIED,
+ &hdr, TDX_GET_QUOTE_HDR_SIZE) != MEMTX_OK) {
+ error_report("TDX: failed to update GetQuote header.\n");
+ return -1;
+ }
+ return 0;
+ }
+
+ qemu_mutex_lock(&tdx_guest->quote_generator->lock);
+ if (tdx_guest->quote_generator->num >= TDX_MAX_GET_QUOTE_REQUEST) {
+ qemu_mutex_unlock(&tdx_guest->quote_generator->lock);
+ vmcall->status_code = TDG_VP_VMCALL_RETRY;
+ return 0;
+ }
+ tdx_guest->quote_generator->num++;
+ qemu_mutex_unlock(&tdx_guest->quote_generator->lock);
+
+ /* Mark the buffer in-flight. */
+ hdr.error_code = cpu_to_le64(TDX_VP_GET_QUOTE_IN_FLIGHT);
+ if (address_space_write(&address_space_memory, buf_gpa,
+ MEMTXATTRS_UNSPECIFIED,
+ &hdr, TDX_GET_QUOTE_HDR_SIZE) != MEMTX_OK) {
+ error_report("TDX: failed to update GetQuote header.\n");
+ return -1;
+ }
+
+ task = g_malloc(sizeof(*task));
+ task->buf_gpa = buf_gpa;
+ task->payload_gpa = buf_gpa + TDX_GET_QUOTE_HDR_SIZE;
+ task->payload_len = buf_len - TDX_GET_QUOTE_HDR_SIZE;
+ task->hdr = hdr;
+ task->quote_gen = tdx_guest->quote_generator;
+ task->completion = tdx_get_quote_completion;
+
+ task->send_data_size = le32_to_cpu(hdr.in_len);
+ task->send_data = g_malloc(task->send_data_size);
+ task->send_data_sent = 0;
+
+ if (address_space_read(&address_space_memory, task->payload_gpa,
+ MEMTXATTRS_UNSPECIFIED, task->send_data,
+ task->send_data_size) != MEMTX_OK) {
+ g_free(task->send_data);
+ return -1;
+ }
+
+ task->receive_buf = g_malloc0(task->payload_len);
+ task->receive_buf_received = 0;
+
+ tdx_generate_quote(task);
+
+ return 0;
+}
+
static int tdx_handle_setup_event_notify_interrupt(X86CPU *cpu,
struct kvm_tdx_vmcall *vmcall)
{
@@ -896,6 +1070,8 @@ static int tdx_handle_vmcall(X86CPU *cpu, struct kvm_tdx_vmcall *vmcall)
}
switch (vmcall->subfunction) {
+ case TDG_VP_VMCALL_GET_QUOTE:
+ return tdx_handle_get_quote(cpu, vmcall);
case TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT:
return tdx_handle_setup_event_notify_interrupt(cpu, vmcall);
default:
@@ -979,6 +1155,40 @@ static void tdx_guest_set_mrownerconfig(Object *obj, const char *value, Error **
tdx->mrownerconfig = g_strdup(value);
}
+static void tdx_guest_get_quote_generation(Object *obj, Visitor *v,
+ const char *name, void *opaque,
+ Error **errp)
+{
+ TdxGuest *tdx = TDX_GUEST(obj);
+
+ visit_type_SocketAddress(v, name, &tdx->quote_generator->socket, errp);
+}
+
+static void tdx_guest_set_quote_generation(Object *obj, Visitor *v,
+ const char *name, void *opaque,
+ Error **errp)
+{
+ TdxGuest *tdx = TDX_GUEST(obj);
+ SocketAddress *sock = NULL;
+ Object *qg_obj;
+ TdxQuoteGenerator *quote_generator;
+
+ if (!visit_type_SocketAddress(v, name, &sock, errp)) {
+ return;
+ }
+
+ if (tdx->quote_generator) {
+ object_unref(tdx->quote_generator);
+ }
+
+ qg_obj = object_new(TYPE_TDX_QUOTE_GENERATOR);
+ quote_generator = TDX_QUOTE_GENERATOR(qg_obj);
+ quote_generator->socket = sock;
+ qemu_mutex_init("e_generator->lock);
+
+ tdx->quote_generator = quote_generator;
+}
+
/* tdx guest */
OBJECT_DEFINE_TYPE_WITH_INTERFACES(TdxGuest,
tdx_guest,
@@ -1007,6 +1217,12 @@ static void tdx_guest_init(Object *obj)
tdx_guest_get_mrownerconfig,
tdx_guest_set_mrownerconfig);
+ tdx->quote_generator = NULL;
+ object_property_add(obj, "quote-generation-socket", "SocketAddress",
+ tdx_guest_get_quote_generation,
+ tdx_guest_set_quote_generation,
+ NULL, NULL);
+
tdx->event_notify_vector = -1;
tdx->event_notify_apicid = -1;
}
diff --git a/target/i386/kvm/tdx.h b/target/i386/kvm/tdx.h
index b3d4462fe718..b7ce793d6037 100644
--- a/target/i386/kvm/tdx.h
+++ b/target/i386/kvm/tdx.h
@@ -9,6 +9,8 @@
#include "hw/i386/tdvf.h"
#include "sysemu/kvm.h"
+#include "tdx-quote-generator.h"
+
#define TYPE_TDX_GUEST "tdx-guest"
#define TDX_GUEST(obj) OBJECT_CHECK(TdxGuest, (obj), TYPE_TDX_GUEST)
@@ -16,6 +18,7 @@ typedef struct TdxGuestClass {
ConfidentialGuestSupportClass parent_class;
} TdxGuestClass;
+#define TDG_VP_VMCALL_GET_QUOTE 0x10002ULL
#define TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT 0x10004ULL
#define TDG_VP_VMCALL_SUCCESS 0x0000000000000000ULL
@@ -55,6 +58,9 @@ typedef struct TdxGuest {
/* runtime state */
uint32_t event_notify_vector;
uint32_t event_notify_apicid;
+
+ /* GetQuote */
+ TdxQuoteGenerator *quote_generator;
} TdxGuest;
#ifdef CONFIG_TDX
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* Re: [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
2024-02-29 6:37 ` [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote> Xiaoyao Li
@ 2024-02-29 8:40 ` Markus Armbruster
2024-02-29 10:54 ` Xiaoyao Li
2024-03-11 9:27 ` Daniel P. Berrangé
2024-10-03 18:08 ` Daniel P. Berrangé
2 siblings, 1 reply; 124+ messages in thread
From: Markus Armbruster @ 2024-02-29 8:40 UTC (permalink / raw)
To: Xiaoyao Li
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti, kvm, qemu-devel, Michael Roth,
Claudio Fontana, Gerd Hoffmann, Isaku Yamahata, Chenyi Qiang
Xiaoyao Li <xiaoyao.li@intel.com> writes:
> From: Isaku Yamahata <isaku.yamahata@intel.com>
>
> Add property "quote-generation-socket" to tdx-guest, which is a property
> of type SocketAddress to specify Quote Generation Service(QGS).
>
> On request of GetQuote, it connects to the QGS socket, read request
> data from shared guest memory, send the request data to the QGS,
> and store the response into shared guest memory, at last notify
> TD guest by interrupt.
>
> command line example:
> qemu-system-x86_64 \
> -object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \
> -machine confidential-guest-support=tdx0
>
> Note, above example uses vsock type socket because the QGS we used
> implements the vsock socket. It can be other types, like UNIX socket,
> which depends on the implementation of QGS.
>
> To avoid no response from QGS server, setup a timer for the transaction.
> If timeout, make it an error and interrupt guest. Define the threshold of
> time to 30s at present, maybe change to other value if not appropriate.
>
> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
> Codeveloped-by: Chenyi Qiang <chenyi.qiang@intel.com>
> Signed-off-by: Chenyi Qiang <chenyi.qiang@intel.com>
> Codeveloped-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
[...]
> diff --git a/qapi/qom.json b/qapi/qom.json
> index cac875349a3a..7b26b0a0d3aa 100644
> --- a/qapi/qom.json
> +++ b/qapi/qom.json
> @@ -917,13 +917,19 @@
> # (base64 encoded SHA384 digest). (A default value 0 of SHA384 is
> # used when absent).
> #
> +# @quote-generation-socket: socket address for Quote Generation
> +# Service (QGS). QGS is a daemon running on the host. User in
> +# TD guest cannot get TD quoting for attestation if QGS is not
> +# provided. So admin should always provide it.
This makes me wonder why it's optional. Can you describe a use case for
*not* specifying @quote-generation-socket?
> +#
> # Since: 9.0
> ##
> { 'struct': 'TdxGuestProperties',
> 'data': { '*sept-ve-disable': 'bool',
> '*mrconfigid': 'str',
> '*mrowner': 'str',
> - '*mrownerconfig': 'str' } }
> + '*mrownerconfig': 'str',
> + '*quote-generation-socket': 'SocketAddress' } }
>
> ##
> # @ThreadContextProperties:
[...]
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
2024-02-29 8:40 ` Markus Armbruster
@ 2024-02-29 10:54 ` Xiaoyao Li
2024-02-29 13:28 ` Markus Armbruster
0 siblings, 1 reply; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 10:54 UTC (permalink / raw)
To: Markus Armbruster
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Marcelo Tosatti, kvm, qemu-devel, Michael Roth, Claudio Fontana,
Gerd Hoffmann, Isaku Yamahata, Chenyi Qiang
On 2/29/2024 4:40 PM, Markus Armbruster wrote:
> Xiaoyao Li <xiaoyao.li@intel.com> writes:
>
>> From: Isaku Yamahata <isaku.yamahata@intel.com>
>>
>> Add property "quote-generation-socket" to tdx-guest, which is a property
>> of type SocketAddress to specify Quote Generation Service(QGS).
>>
>> On request of GetQuote, it connects to the QGS socket, read request
>> data from shared guest memory, send the request data to the QGS,
>> and store the response into shared guest memory, at last notify
>> TD guest by interrupt.
>>
>> command line example:
>> qemu-system-x86_64 \
>> -object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \
>> -machine confidential-guest-support=tdx0
>>
>> Note, above example uses vsock type socket because the QGS we used
>> implements the vsock socket. It can be other types, like UNIX socket,
>> which depends on the implementation of QGS.
>>
>> To avoid no response from QGS server, setup a timer for the transaction.
>> If timeout, make it an error and interrupt guest. Define the threshold of
>> time to 30s at present, maybe change to other value if not appropriate.
>>
>> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
>> Codeveloped-by: Chenyi Qiang <chenyi.qiang@intel.com>
>> Signed-off-by: Chenyi Qiang <chenyi.qiang@intel.com>
>> Codeveloped-by: Xiaoyao Li <xiaoyao.li@intel.com>
>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>
> [...]
>
>> diff --git a/qapi/qom.json b/qapi/qom.json
>> index cac875349a3a..7b26b0a0d3aa 100644
>> --- a/qapi/qom.json
>> +++ b/qapi/qom.json
>> @@ -917,13 +917,19 @@
>> # (base64 encoded SHA384 digest). (A default value 0 of SHA384 is
>> # used when absent).
>> #
>> +# @quote-generation-socket: socket address for Quote Generation
>> +# Service (QGS). QGS is a daemon running on the host. User in
>> +# TD guest cannot get TD quoting for attestation if QGS is not
>> +# provided. So admin should always provide it.
>
> This makes me wonder why it's optional. Can you describe a use case for
> *not* specifying @quote-generation-socket?
Maybe at last when all the TDX support lands on all the components,
attestation will become a must for a TD guest to be usable.
However, at least for today, booting and running a TD guest don't
require attestation. So not provide it, doesn't affect anything
excepting cannot get a Quote.
>> +#
>> # Since: 9.0
>> ##
>> { 'struct': 'TdxGuestProperties',
>> 'data': { '*sept-ve-disable': 'bool',
>> '*mrconfigid': 'str',
>> '*mrowner': 'str',
>> - '*mrownerconfig': 'str' } }
>> + '*mrownerconfig': 'str',
>> + '*quote-generation-socket': 'SocketAddress' } }
>>
>> ##
>> # @ThreadContextProperties:
>
> [...]
>
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
2024-02-29 10:54 ` Xiaoyao Li
@ 2024-02-29 13:28 ` Markus Armbruster
2024-03-07 11:25 ` Xiaoyao Li
0 siblings, 1 reply; 124+ messages in thread
From: Markus Armbruster @ 2024-02-29 13:28 UTC (permalink / raw)
To: Xiaoyao Li
Cc: Markus Armbruster, Paolo Bonzini, David Hildenbrand,
Igor Mammedov, Eduardo Habkost, Marcel Apfelbaum,
Philippe Mathieu-Daudé, Yanan Wang, Michael S. Tsirkin,
Richard Henderson, Ani Sinha, Peter Xu, Cornelia Huck,
Daniel P. Berrangé, Eric Blake, Marcelo Tosatti, kvm,
qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang
Xiaoyao Li <xiaoyao.li@intel.com> writes:
> On 2/29/2024 4:40 PM, Markus Armbruster wrote:
>> Xiaoyao Li <xiaoyao.li@intel.com> writes:
>>
>>> From: Isaku Yamahata <isaku.yamahata@intel.com>
>>>
>>> Add property "quote-generation-socket" to tdx-guest, which is a property
>>> of type SocketAddress to specify Quote Generation Service(QGS).
>>>
>>> On request of GetQuote, it connects to the QGS socket, read request
>>> data from shared guest memory, send the request data to the QGS,
>>> and store the response into shared guest memory, at last notify
>>> TD guest by interrupt.
>>>
>>> command line example:
>>> qemu-system-x86_64 \
>>> -object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \
>>> -machine confidential-guest-support=tdx0
>>>
>>> Note, above example uses vsock type socket because the QGS we used
>>> implements the vsock socket. It can be other types, like UNIX socket,
>>> which depends on the implementation of QGS.
>>>
>>> To avoid no response from QGS server, setup a timer for the transaction.
>>> If timeout, make it an error and interrupt guest. Define the threshold of
>>> time to 30s at present, maybe change to other value if not appropriate.
>>>
>>> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
>>> Codeveloped-by: Chenyi Qiang <chenyi.qiang@intel.com>
>>> Signed-off-by: Chenyi Qiang <chenyi.qiang@intel.com>
>>> Codeveloped-by: Xiaoyao Li <xiaoyao.li@intel.com>
>>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>>
>> [...]
>>
>>> diff --git a/qapi/qom.json b/qapi/qom.json
>>> index cac875349a3a..7b26b0a0d3aa 100644
>>> --- a/qapi/qom.json
>>> +++ b/qapi/qom.json
>>> @@ -917,13 +917,19 @@
>>> # (base64 encoded SHA384 digest). (A default value 0 of SHA384 is
>>> # used when absent).
>>> #
>>> +# @quote-generation-socket: socket address for Quote Generation
>>> +# Service (QGS). QGS is a daemon running on the host. User in
>>> +# TD guest cannot get TD quoting for attestation if QGS is not
>>> +# provided. So admin should always provide it.
>>
>> This makes me wonder why it's optional. Can you describe a use case for
>> *not* specifying @quote-generation-socket?
>
> Maybe at last when all the TDX support lands on all the components, attestation will become a must for a TD guest to be usable.
>
> However, at least for today, booting and running a TD guest don't require attestation. So not provide it, doesn't affect anything excepting cannot get a Quote.
Maybe
# @quote-generation-socket: Socket address for Quote Generation
# Service (QGS). QGS is a daemon running on the host. Without
# it, the guest will not be able to get a TD quote for
# attestation.
[...]
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
2024-02-29 13:28 ` Markus Armbruster
@ 2024-03-07 11:25 ` Xiaoyao Li
0 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-03-07 11:25 UTC (permalink / raw)
To: Markus Armbruster
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Marcelo Tosatti, kvm, qemu-devel, Michael Roth, Claudio Fontana,
Gerd Hoffmann, Isaku Yamahata, Chenyi Qiang
On 2/29/2024 9:28 PM, Markus Armbruster wrote:
> Xiaoyao Li <xiaoyao.li@intel.com> writes:
>
>> On 2/29/2024 4:40 PM, Markus Armbruster wrote:
>>> Xiaoyao Li <xiaoyao.li@intel.com> writes:
>>>
>>>> From: Isaku Yamahata <isaku.yamahata@intel.com>
>>>>
>>>> Add property "quote-generation-socket" to tdx-guest, which is a property
>>>> of type SocketAddress to specify Quote Generation Service(QGS).
>>>>
>>>> On request of GetQuote, it connects to the QGS socket, read request
>>>> data from shared guest memory, send the request data to the QGS,
>>>> and store the response into shared guest memory, at last notify
>>>> TD guest by interrupt.
>>>>
>>>> command line example:
>>>> qemu-system-x86_64 \
>>>> -object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \
>>>> -machine confidential-guest-support=tdx0
>>>>
>>>> Note, above example uses vsock type socket because the QGS we used
>>>> implements the vsock socket. It can be other types, like UNIX socket,
>>>> which depends on the implementation of QGS.
>>>>
>>>> To avoid no response from QGS server, setup a timer for the transaction.
>>>> If timeout, make it an error and interrupt guest. Define the threshold of
>>>> time to 30s at present, maybe change to other value if not appropriate.
>>>>
>>>> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
>>>> Codeveloped-by: Chenyi Qiang <chenyi.qiang@intel.com>
>>>> Signed-off-by: Chenyi Qiang <chenyi.qiang@intel.com>
>>>> Codeveloped-by: Xiaoyao Li <xiaoyao.li@intel.com>
>>>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>>>
>>> [...]
>>>
>>>> diff --git a/qapi/qom.json b/qapi/qom.json
>>>> index cac875349a3a..7b26b0a0d3aa 100644
>>>> --- a/qapi/qom.json
>>>> +++ b/qapi/qom.json
>>>> @@ -917,13 +917,19 @@
>>>> # (base64 encoded SHA384 digest). (A default value 0 of SHA384 is
>>>> # used when absent).
>>>> #
>>>> +# @quote-generation-socket: socket address for Quote Generation
>>>> +# Service (QGS). QGS is a daemon running on the host. User in
>>>> +# TD guest cannot get TD quoting for attestation if QGS is not
>>>> +# provided. So admin should always provide it.
>>>
>>> This makes me wonder why it's optional. Can you describe a use case for
>>> *not* specifying @quote-generation-socket?
>>
>> Maybe at last when all the TDX support lands on all the components, attestation will become a must for a TD guest to be usable.
>>
>> However, at least for today, booting and running a TD guest don't require attestation. So not provide it, doesn't affect anything excepting cannot get a Quote.
>
> Maybe
>
> # @quote-generation-socket: Socket address for Quote Generation
> # Service (QGS). QGS is a daemon running on the host. Without
> # it, the guest will not be able to get a TD quote for
> # attestation.
Thanks! will update to it.
> [...]
>
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
2024-02-29 6:37 ` [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote> Xiaoyao Li
2024-02-29 8:40 ` Markus Armbruster
@ 2024-03-11 9:27 ` Daniel P. Berrangé
2024-03-12 7:44 ` Xiaoyao Li
2024-10-03 18:08 ` Daniel P. Berrangé
2 siblings, 1 reply; 124+ messages in thread
From: Daniel P. Berrangé @ 2024-03-11 9:27 UTC (permalink / raw)
To: Xiaoyao Li
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Eric Blake, Markus Armbruster, Marcelo Tosatti,
kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang
On Thu, Feb 29, 2024 at 01:37:10AM -0500, Xiaoyao Li wrote:
> From: Isaku Yamahata <isaku.yamahata@intel.com>
>
> Add property "quote-generation-socket" to tdx-guest, which is a property
> of type SocketAddress to specify Quote Generation Service(QGS).
>
> On request of GetQuote, it connects to the QGS socket, read request
> data from shared guest memory, send the request data to the QGS,
> and store the response into shared guest memory, at last notify
> TD guest by interrupt.
>
> command line example:
> qemu-system-x86_64 \
> -object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \
Can you illustrate this with 'unix' sockets, not 'vsock'.
It makes no conceptual sense to be using vsock for two
processes on the host to be using vsock to talk to
each other. vsock is only needed for the guest to talk
to the host.
> -machine confidential-guest-support=tdx0
>
> Note, above example uses vsock type socket because the QGS we used
> implements the vsock socket. It can be other types, like UNIX socket,
> which depends on the implementation of QGS.
>
> To avoid no response from QGS server, setup a timer for the transaction.
> If timeout, make it an error and interrupt guest. Define the threshold of
> time to 30s at present, maybe change to other value if not appropriate.
>
> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
> Codeveloped-by: Chenyi Qiang <chenyi.qiang@intel.com>
> Signed-off-by: Chenyi Qiang <chenyi.qiang@intel.com>
> Codeveloped-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
> Changes in v5:
> - add more decription of quote-generation-socket property;
>
> Changes in v4:
> - merge next patch "i386/tdx: setup a timer for the qio channel";
>
> Changes in v3:
> - rename property "quote-generation-service" to "quote-generation-socket";
> - change the type of "quote-generation-socket" from str to
> SocketAddress;
> - squash next patch into this one;
> ---
> qapi/qom.json | 8 +-
> target/i386/kvm/meson.build | 2 +-
> target/i386/kvm/tdx-quote-generator.c | 170 ++++++++++++++++++++
> target/i386/kvm/tdx-quote-generator.h | 95 +++++++++++
> target/i386/kvm/tdx.c | 216 ++++++++++++++++++++++++++
> target/i386/kvm/tdx.h | 6 +
> 6 files changed, 495 insertions(+), 2 deletions(-)
> create mode 100644 target/i386/kvm/tdx-quote-generator.c
> create mode 100644 target/i386/kvm/tdx-quote-generator.h
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
2024-03-11 9:27 ` Daniel P. Berrangé
@ 2024-03-12 7:44 ` Xiaoyao Li
2024-03-12 18:02 ` Isaku Yamahata
2024-03-13 15:31 ` Daniel P. Berrangé
0 siblings, 2 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-03-12 7:44 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Eric Blake, Markus Armbruster, Marcelo Tosatti,
kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang
On 3/11/2024 5:27 PM, Daniel P. Berrangé wrote:
> On Thu, Feb 29, 2024 at 01:37:10AM -0500, Xiaoyao Li wrote:
>> From: Isaku Yamahata <isaku.yamahata@intel.com>
>>
>> Add property "quote-generation-socket" to tdx-guest, which is a property
>> of type SocketAddress to specify Quote Generation Service(QGS).
>>
>> On request of GetQuote, it connects to the QGS socket, read request
>> data from shared guest memory, send the request data to the QGS,
>> and store the response into shared guest memory, at last notify
>> TD guest by interrupt.
>>
>> command line example:
>> qemu-system-x86_64 \
>> -object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \
>
> Can you illustrate this with 'unix' sockets, not 'vsock'.
Are you suggesting only updating the commit message to an example of
unix socket? Or you want the code to test with some unix socket QGS?
(It seems the QGS I got for testing, only supports vsock socket. Because
at the time when it got developed, it was supposed to communicate with
drivers inside TD guest directly not via VMM (KVM+QEMU). Anyway, I will
talk to internal folks to see if any plan to support unix socket.)
> It makes no conceptual sense to be using vsock for two
> processes on the host to be using vsock to talk to
> each other. vsock is only needed for the guest to talk
> to the host.
>
>> -machine confidential-guest-support=tdx0
>>
>> Note, above example uses vsock type socket because the QGS we used
>> implements the vsock socket. It can be other types, like UNIX socket,
>> which depends on the implementation of QGS.
>>
>> To avoid no response from QGS server, setup a timer for the transaction.
>> If timeout, make it an error and interrupt guest. Define the threshold of
>> time to 30s at present, maybe change to other value if not appropriate.
>>
>> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
>> Codeveloped-by: Chenyi Qiang <chenyi.qiang@intel.com>
>> Signed-off-by: Chenyi Qiang <chenyi.qiang@intel.com>
>> Codeveloped-by: Xiaoyao Li <xiaoyao.li@intel.com>
>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>> ---
>> Changes in v5:
>> - add more decription of quote-generation-socket property;
>>
>> Changes in v4:
>> - merge next patch "i386/tdx: setup a timer for the qio channel";
>>
>> Changes in v3:
>> - rename property "quote-generation-service" to "quote-generation-socket";
>> - change the type of "quote-generation-socket" from str to
>> SocketAddress;
>> - squash next patch into this one;
>> ---
>> qapi/qom.json | 8 +-
>> target/i386/kvm/meson.build | 2 +-
>> target/i386/kvm/tdx-quote-generator.c | 170 ++++++++++++++++++++
>> target/i386/kvm/tdx-quote-generator.h | 95 +++++++++++
>> target/i386/kvm/tdx.c | 216 ++++++++++++++++++++++++++
>> target/i386/kvm/tdx.h | 6 +
>> 6 files changed, 495 insertions(+), 2 deletions(-)
>> create mode 100644 target/i386/kvm/tdx-quote-generator.c
>> create mode 100644 target/i386/kvm/tdx-quote-generator.h
>
>
> With regards,
> Daniel
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
2024-03-12 7:44 ` Xiaoyao Li
@ 2024-03-12 18:02 ` Isaku Yamahata
2024-03-13 15:31 ` Daniel P. Berrangé
1 sibling, 0 replies; 124+ messages in thread
From: Isaku Yamahata @ 2024-03-12 18:02 UTC (permalink / raw)
To: Xiaoyao Li
Cc: Daniel P. Berrangé, Paolo Bonzini, David Hildenbrand,
Igor Mammedov, Eduardo Habkost, Marcel Apfelbaum,
Philippe Mathieu-Daudé, Yanan Wang, Michael S. Tsirkin,
Richard Henderson, Ani Sinha, Peter Xu, Cornelia Huck, Eric Blake,
Markus Armbruster, Marcelo Tosatti, kvm, qemu-devel, Michael Roth,
Claudio Fontana, Gerd Hoffmann, Isaku Yamahata, Chenyi Qiang,
isaku.yamahata, isaku.yamahata
On Tue, Mar 12, 2024 at 03:44:32PM +0800,
Xiaoyao Li <xiaoyao.li@intel.com> wrote:
> On 3/11/2024 5:27 PM, Daniel P. Berrangé wrote:
> > On Thu, Feb 29, 2024 at 01:37:10AM -0500, Xiaoyao Li wrote:
> > > From: Isaku Yamahata <isaku.yamahata@intel.com>
> > >
> > > Add property "quote-generation-socket" to tdx-guest, which is a property
> > > of type SocketAddress to specify Quote Generation Service(QGS).
> > >
> > > On request of GetQuote, it connects to the QGS socket, read request
> > > data from shared guest memory, send the request data to the QGS,
> > > and store the response into shared guest memory, at last notify
> > > TD guest by interrupt.
> > >
> > > command line example:
> > > qemu-system-x86_64 \
> > > -object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \
> >
> > Can you illustrate this with 'unix' sockets, not 'vsock'.
>
> Are you suggesting only updating the commit message to an example of unix
> socket? Or you want the code to test with some unix socket QGS?
>
> (It seems the QGS I got for testing, only supports vsock socket. Because at
> the time when it got developed, it was supposed to communicate with drivers
> inside TD guest directly not via VMM (KVM+QEMU). Anyway, I will talk to
> internal folks to see if any plan to support unix socket.)
You can use small utility to proxy sockets for testing purpose. The famous one
is socat or nmap ncat. They support vsock. At least their latest version does.
QGS <-vsock-> socat <-unix domain-> qemu
--
Isaku Yamahata <isaku.yamahata@intel.com>
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
2024-03-12 7:44 ` Xiaoyao Li
2024-03-12 18:02 ` Isaku Yamahata
@ 2024-03-13 15:31 ` Daniel P. Berrangé
2024-03-15 8:41 ` Xiaoyao Li
1 sibling, 1 reply; 124+ messages in thread
From: Daniel P. Berrangé @ 2024-03-13 15:31 UTC (permalink / raw)
To: Xiaoyao Li
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Eric Blake, Markus Armbruster, Marcelo Tosatti,
kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang
On Tue, Mar 12, 2024 at 03:44:32PM +0800, Xiaoyao Li wrote:
> On 3/11/2024 5:27 PM, Daniel P. Berrangé wrote:
> > On Thu, Feb 29, 2024 at 01:37:10AM -0500, Xiaoyao Li wrote:
> > > From: Isaku Yamahata <isaku.yamahata@intel.com>
> > >
> > > Add property "quote-generation-socket" to tdx-guest, which is a property
> > > of type SocketAddress to specify Quote Generation Service(QGS).
> > >
> > > On request of GetQuote, it connects to the QGS socket, read request
> > > data from shared guest memory, send the request data to the QGS,
> > > and store the response into shared guest memory, at last notify
> > > TD guest by interrupt.
> > >
> > > command line example:
> > > qemu-system-x86_64 \
> > > -object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \
> >
> > Can you illustrate this with 'unix' sockets, not 'vsock'.
>
> Are you suggesting only updating the commit message to an example of unix
> socket? Or you want the code to test with some unix socket QGS?
>
> (It seems the QGS I got for testing, only supports vsock socket. Because at
> the time when it got developed, it was supposed to communicate with drivers
> inside TD guest directly not via VMM (KVM+QEMU). Anyway, I will talk to
> internal folks to see if any plan to support unix socket.)
The QGS provided as part of DCAP supports running with both
UNIX sockets and VSOCK, and I would expect QEMU to be made
to work with this, since its is Intel's OSS reference impl.
Exposing QGS to the guest when we only intend for it to be
used by the host QEMU is needlessly expanding the attack
surface.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
2024-03-13 15:31 ` Daniel P. Berrangé
@ 2024-03-15 8:41 ` Xiaoyao Li
0 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-03-15 8:41 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Eric Blake, Markus Armbruster, Marcelo Tosatti,
kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang
On 3/13/2024 11:31 PM, Daniel P. Berrangé wrote:
> On Tue, Mar 12, 2024 at 03:44:32PM +0800, Xiaoyao Li wrote:
>> On 3/11/2024 5:27 PM, Daniel P. Berrangé wrote:
>>> On Thu, Feb 29, 2024 at 01:37:10AM -0500, Xiaoyao Li wrote:
>>>> From: Isaku Yamahata <isaku.yamahata@intel.com>
>>>>
>>>> Add property "quote-generation-socket" to tdx-guest, which is a property
>>>> of type SocketAddress to specify Quote Generation Service(QGS).
>>>>
>>>> On request of GetQuote, it connects to the QGS socket, read request
>>>> data from shared guest memory, send the request data to the QGS,
>>>> and store the response into shared guest memory, at last notify
>>>> TD guest by interrupt.
>>>>
>>>> command line example:
>>>> qemu-system-x86_64 \
>>>> -object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \
>>>
>>> Can you illustrate this with 'unix' sockets, not 'vsock'.
>>
>> Are you suggesting only updating the commit message to an example of unix
>> socket? Or you want the code to test with some unix socket QGS?
>>
>> (It seems the QGS I got for testing, only supports vsock socket. Because at
>> the time when it got developed, it was supposed to communicate with drivers
>> inside TD guest directly not via VMM (KVM+QEMU). Anyway, I will talk to
>> internal folks to see if any plan to support unix socket.)
>
> The QGS provided as part of DCAP supports running with both
> UNIX sockets and VSOCK, and I would expect QEMU to be made
> to work with this, since its is Intel's OSS reference impl.
After synced with internal folks, yes, the QGS I used does support unix
socket. I tested it and it worked.
-object
'{"qom-type":"tdx-guest","id":"tdx","quote-generation-socket":{"type":"unix",
"path":"/var/run/tdx-qgs/qgs.socket"}}'
> Exposing QGS to the guest when we only intend for it to be
> used by the host QEMU is needlessly expanding the attack
> surface.
>
> With regards,
> Daniel
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
2024-02-29 6:37 ` [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote> Xiaoyao Li
2024-02-29 8:40 ` Markus Armbruster
2024-03-11 9:27 ` Daniel P. Berrangé
@ 2024-10-03 18:08 ` Daniel P. Berrangé
2025-04-02 15:26 ` Xiaoyao Li
2 siblings, 1 reply; 124+ messages in thread
From: Daniel P. Berrangé @ 2024-10-03 18:08 UTC (permalink / raw)
To: Xiaoyao Li
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Eric Blake, Markus Armbruster, Marcelo Tosatti,
kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang
On Thu, Feb 29, 2024 at 01:37:10AM -0500, Xiaoyao Li wrote:
> From: Isaku Yamahata <isaku.yamahata@intel.com>
>
> Add property "quote-generation-socket" to tdx-guest, which is a property
> of type SocketAddress to specify Quote Generation Service(QGS).
>
> On request of GetQuote, it connects to the QGS socket, read request
> data from shared guest memory, send the request data to the QGS,
> and store the response into shared guest memory, at last notify
> TD guest by interrupt.
>
> command line example:
> qemu-system-x86_64 \
> -object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \
> -machine confidential-guest-support=tdx0
>
> Note, above example uses vsock type socket because the QGS we used
> implements the vsock socket. It can be other types, like UNIX socket,
> which depends on the implementation of QGS.
>
> To avoid no response from QGS server, setup a timer for the transaction.
> If timeout, make it an error and interrupt guest. Define the threshold of
> time to 30s at present, maybe change to other value if not appropriate.
>
> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
> Codeveloped-by: Chenyi Qiang <chenyi.qiang@intel.com>
> Signed-off-by: Chenyi Qiang <chenyi.qiang@intel.com>
> Codeveloped-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
> index 49f94d9d46f4..7dfda507cc8c 100644
> --- a/target/i386/kvm/tdx.c
> +++ b/target/i386/kvm/tdx.c
> +static int tdx_handle_get_quote(X86CPU *cpu, struct kvm_tdx_vmcall *vmcall)
> +{
> + struct tdx_generate_quote_task *task;
> + struct tdx_get_quote_header hdr;
> + hwaddr buf_gpa = vmcall->in_r12;
> + uint64_t buf_len = vmcall->in_r13;
> +
> + QEMU_BUILD_BUG_ON(sizeof(struct tdx_get_quote_header) != TDX_GET_QUOTE_HDR_SIZE);
> +
> + vmcall->status_code = TDG_VP_VMCALL_INVALID_OPERAND;
> +
> + if (buf_len == 0) {
> + return 0;
> + }
> +
> + /* GPA must be shared. */
> + if (!(buf_gpa & tdx_shared_bit(cpu))) {
> + return 0;
> + }
> + buf_gpa &= ~tdx_shared_bit(cpu);
> +
> + if (!QEMU_IS_ALIGNED(buf_gpa, 4096) || !QEMU_IS_ALIGNED(buf_len, 4096)) {
> + vmcall->status_code = TDG_VP_VMCALL_ALIGN_ERROR;
> + return 0;
> + }
> +
> + if (address_space_read(&address_space_memory, buf_gpa, MEMTXATTRS_UNSPECIFIED,
> + &hdr, TDX_GET_QUOTE_HDR_SIZE) != MEMTX_OK) {
> + error_report("TDX: get-quote: failed to read GetQuote header.\n");
> + return -1;
> + }
> +
> + if (le64_to_cpu(hdr.structure_version) != TDX_GET_QUOTE_STRUCTURE_VERSION) {
> + return 0;
> + }
> +
> + /*
> + * Paranoid: Guest should clear error_code and out_len to avoid information
> + * leak. Enforce it. The initial value of them doesn't matter for qemu to
> + * process the request.
> + */
> + if (le64_to_cpu(hdr.error_code) != TDX_VP_GET_QUOTE_SUCCESS ||
> + le32_to_cpu(hdr.out_len) != 0) {
> + return 0;
> + }
> +
> + /* Only safe-guard check to avoid too large buffer size. */
> + if (buf_len > TDX_GET_QUOTE_MAX_BUF_LEN ||
> + le32_to_cpu(hdr.in_len) > buf_len - TDX_GET_QUOTE_HDR_SIZE) {
> + return 0;
> + }
> +
> + vmcall->status_code = TDG_VP_VMCALL_SUCCESS;
> + if (!tdx_guest->quote_generator) {
> + hdr.error_code = cpu_to_le64(TDX_VP_GET_QUOTE_QGS_UNAVAILABLE);
> + if (address_space_write(&address_space_memory, buf_gpa,
> + MEMTXATTRS_UNSPECIFIED,
> + &hdr, TDX_GET_QUOTE_HDR_SIZE) != MEMTX_OK) {
> + error_report("TDX: failed to update GetQuote header.\n");
> + return -1;
> + }
> + return 0;
> + }
> +
> + qemu_mutex_lock(&tdx_guest->quote_generator->lock);
> + if (tdx_guest->quote_generator->num >= TDX_MAX_GET_QUOTE_REQUEST) {
> + qemu_mutex_unlock(&tdx_guest->quote_generator->lock);
> + vmcall->status_code = TDG_VP_VMCALL_RETRY;
> + return 0;
> + }
> + tdx_guest->quote_generator->num++;
> + qemu_mutex_unlock(&tdx_guest->quote_generator->lock);
> +
> + /* Mark the buffer in-flight. */
> + hdr.error_code = cpu_to_le64(TDX_VP_GET_QUOTE_IN_FLIGHT);
> + if (address_space_write(&address_space_memory, buf_gpa,
> + MEMTXATTRS_UNSPECIFIED,
> + &hdr, TDX_GET_QUOTE_HDR_SIZE) != MEMTX_OK) {
> + error_report("TDX: failed to update GetQuote header.\n");
> + return -1;
> + }
> +
> + task = g_malloc(sizeof(*task));
> + task->buf_gpa = buf_gpa;
> + task->payload_gpa = buf_gpa + TDX_GET_QUOTE_HDR_SIZE;
> + task->payload_len = buf_len - TDX_GET_QUOTE_HDR_SIZE;
> + task->hdr = hdr;
> + task->quote_gen = tdx_guest->quote_generator;
> + task->completion = tdx_get_quote_completion;
> +
> + task->send_data_size = le32_to_cpu(hdr.in_len);
> + task->send_data = g_malloc(task->send_data_size);
> + task->send_data_sent = 0;
> +
> + if (address_space_read(&address_space_memory, task->payload_gpa,
> + MEMTXATTRS_UNSPECIFIED, task->send_data,
> + task->send_data_size) != MEMTX_OK) {
> + g_free(task->send_data);
> + return -1;
> + }
In this method we've received "struct tdx_get_quote_header" from
the guest OS, and the 'hdr.in_len' field in that struct tells us
the payload to read from guest memory. This payload is treated as
opaque by QEMU and sent over the UNIX socket directly to QGS with
no validation of the payload.
The payload is supposed to be a raw TDX report, that QGS will turn
into a quote.
Nothing guarantees that the guest OS has actually given QEMU a
payload that represents a TDX report.
The only validation done in this patch is to check the 'hdr.in_len'
was not ridiculously huge:
#define TDX_GET_QUOTE_MAX_BUF_LEN (128 * 1024)
#define TDX_GET_QUOTE_HDR_SIZE 24
...
/* Only safe-guard check to avoid too large buffer size. */
if (buf_len > TDX_GET_QUOTE_MAX_BUF_LEN ||
le32_to_cpu(hdr.in_len) > buf_len - TDX_GET_QUOTE_HDR_SIZE) {
return 0;
}
IOW, hdr.in_len can be any value between 0 and 131048, and
the payload data read can contain arbitrary bytes.
Over in the QGS code, QGS historically had a socket protocol
taking various messages from the libtdxattest library which
were defined in this:
https://github.com/intel/SGXDataCenterAttestationPrimitives/blob/main/QuoteGeneration/quote_wrapper/qgs_msg_lib/inc/qgs_msg_lib.h
typedef enum _qgs_msg_type_t {
GET_QUOTE_REQ = 0,
GET_QUOTE_RESP = 1,
GET_COLLATERAL_REQ = 2,
GET_COLLATERAL_RESP = 3,
GET_PLATFORM_INFO_REQ = 4,
GET_PLATFORM_INFO_RESP = 5,
QGS_MSG_TYPE_MAX
} qgs_msg_type_t;
typedef struct _qgs_msg_header_t {
uint16_t major_version;
uint16_t minor_version;
uint32_t type;
uint32_t size; // size of the whole message, include this header, in byte
uint32_t error_code; // used in response only
} qgs_msg_header_t;
such messages are processed by the 'get_resp' method in QGS:
https://github.com/intel/SGXDataCenterAttestationPrimitives/blob/main/QuoteGeneration/quote_wrapper/qgs/qgs_ql_logic.cpp#L78
The 1.21 release of DCAP introduced a new "raw" mode in QGS which
just receives the raw 1024 byte packet from the client which is
supposed to be a raw TDX report. This is what this QEMU patch
is relying on IIUC.
The QGS daemon decides whether a client is speaking the formal
protocol, or "raw" mode, by trying to interpret the incoming
data as a 'qgs_msg_header_t' struct. If the header size looks
wrong & it has exactly 1024 bytes, then QGS assumes it has got
a raw TDX report:
https://github.com/intel/SGXDataCenterAttestationPrimitives/blob/main/QuoteGeneration/quote_wrapper/qgs/qgs_server.cpp#L165
This all works if the data QEMU gets from the guest is indeed a
1024 byte raw TDX report, but what happens if we face a malicious
guest ?
AFAICT, the guest OS is able to send a "qgs_msg_header_t" packet
to QEMU, which QEMU blindly passes on to QGS. This allows the
guest OS to invoke any of the three QGS commands - GET_QUOTE_REQ,
GET_COLLATERAL_REQ, or GET_PLATFORM_INFO_REQ. Fortunately I think
those three messages are all safe to invoke, but none the less,
this should not be permitted, as it leaves a wide open door for
possible future exploits.
As mentioned before, I don't know why this raw mode was invented
for QGS, when QEMU itself could just take the guest report and
pack it into the 'GET_QUOTE_REQ' message format and send it to
QGS. This prevents the guest OS from being able to exploit QEMU
to invoke arbirtary QGS messages.
IMHO, QEMU should be made to pack & unpack the TDX report from
the guest into the GET_QUOTE_REQ / GET_QUOTE_RESP messages, and
this "raw" mode should be removed to QGS as it is inherantly
dangerous to have this magic protocol overloading.
Below is a patch on top of this one that illustrates how QEMU
could use the GET_QUOTE_REQ / GET_QUOTE_RESP messages and avoid
the "raw" mode of QGS.
> +
> + task->receive_buf = g_malloc0(task->payload_len);
> + task->receive_buf_received = 0;
> +
> + tdx_generate_quote(task);
> +
> + return 0;
> +}
--- qemu-9.0.0-rc3/target/i386/kvm/tdx-quote-generator.c 2024-10-02 11:05:31.328003392 -0400
+++ qemu-9.0.0/target/i386/kvm/tdx-quote-generator.c 2024-10-03 13:46:25.744775539 -0400
@@ -24,6 +24,61 @@
OBJECT_DEFINE_TYPE(TdxQuoteGenerator, tdx_quote_generator, TDX_QUOTE_GENERATOR, OBJECT)
+const uint32_t QGS_MSG_LIB_MAJOR_VER = 1;
+const uint32_t QGS_MSG_LIB_MINOR_VER = 1;
+
+typedef enum _qgs_msg_type_t {
+ GET_QUOTE_REQ = 0,
+ GET_QUOTE_RESP = 1,
+ GET_COLLATERAL_REQ = 2,
+ GET_COLLATERAL_RESP = 3,
+ GET_PLATFORM_INFO_REQ = 4,
+ GET_PLATFORM_INFO_RESP = 5,
+ QGS_MSG_TYPE_MAX
+} qgs_msg_type_t;
+
+typedef struct _qgs_msg_header_t {
+ uint16_t major_version;
+ uint16_t minor_version;
+ uint32_t type;
+ uint32_t size; // size of the whole message, include this header, in byte
+ uint32_t error_code; // used in response only
+} qgs_msg_header_t;
+
+typedef struct _qgs_msg_get_quote_req_t {
+ qgs_msg_header_t header; // header.type = GET_QUOTE_REQ
+ uint32_t report_size; // cannot be 0
+ uint32_t id_list_size; // length of id_list, in byte, can be 0
+} qgs_msg_get_quote_req_t;
+
+typedef struct _qgs_msg_get_quote_resp_s {
+ qgs_msg_header_t header; // header.type = GET_QUOTE_RESP
+ uint32_t selected_id_size; // can be 0 in case only one id is sent in request
+ uint32_t quote_size; // length of quote_data, in byte
+ uint8_t id_quote[]; // selected id followed by quote
+} qgs_msg_get_quote_resp_t;
+
+const unsigned HEADER_SIZE = 4;
+
+static uint32_t decode_header(const char *buf, size_t len) {
+ if (len < HEADER_SIZE) {
+ return 0;
+ }
+ uint32_t msg_size = 0;
+ for (uint32_t i = 0; i < HEADER_SIZE; ++i) {
+ msg_size = msg_size * 256 + (buf[i] & 0xFF);
+ }
+ return msg_size;
+}
+
+static void encode_header(char *buf, size_t len, uint32_t size) {
+ assert(len >= HEADER_SIZE);
+ buf[0] = ((size >> 24) & 0xFF);
+ buf[1] = ((size >> 16) & 0xFF);
+ buf[2] = ((size >> 8) & 0xFF);
+ buf[3] = (size & 0xFF);
+}
+
static void tdx_quote_generator_finalize(Object *obj)
{
}
@@ -70,9 +125,86 @@
goto end;
}
}
+
+ if (ret == 0) {
+ error_report("End of file before reply received");
+ task->status_code = TDX_VP_GET_QUOTE_ERROR;
+ goto end;
+ }
task->receive_buf_received += ret;
- if (ret == 0 || task->receive_buf_received == task->payload_len) {
+ if (task->receive_buf_received >= HEADER_SIZE) {
+ uint32_t len = decode_header(task->receive_buf,
+ task->receive_buf_received);
+ if (len == 0 ||
+ len > (task->payload_len - HEADER_SIZE)) {
+ error_report("Message len %u must be non-zero & less than %zu",
+ len, (task->payload_len - HEADER_SIZE));
+ task->status_code = TDX_VP_GET_QUOTE_ERROR;
+ goto end;
+ }
+
+ /* Now we know the size, shrink to fit */
+ task->payload_len = HEADER_SIZE + len;
+ task->receive_buf = g_renew(char,
+ task->receive_buf,
+ task->payload_len);
+ }
+
+ if (task->receive_buf_received >= (sizeof(qgs_msg_header_t) + HEADER_SIZE)) {
+ qgs_msg_header_t *hdr = (qgs_msg_header_t *)(task->receive_buf + HEADER_SIZE);
+ if (hdr->major_version != QGS_MSG_LIB_MAJOR_VER ||
+ hdr->minor_version != QGS_MSG_LIB_MINOR_VER) {
+ error_report("Invalid QGS message header version %d.%d\n",
+ hdr->major_version,
+ hdr->minor_version);
+ task->status_code = TDX_VP_GET_QUOTE_ERROR;
+ goto end;
+ }
+ if (hdr->type != GET_QUOTE_RESP) {
+ error_report("Invalid QGS message type %d\n",
+ hdr->type);
+ task->status_code = TDX_VP_GET_QUOTE_ERROR;
+ goto end;
+ }
+ if (hdr->size > (task->payload_len - HEADER_SIZE)) {
+ error_report("QGS message size %d exceeds payload capacity %zu",
+ hdr->size, task->payload_len);
+ task->status_code = TDX_VP_GET_QUOTE_ERROR;
+ goto end;
+ }
+ if (hdr->error_code != 0) {
+ error_report("QGS message error code %d",
+ hdr->error_code);
+ task->status_code = TDX_VP_GET_QUOTE_ERROR;
+ goto end;
+ }
+ }
+ if (task->receive_buf_received >= (sizeof(qgs_msg_get_quote_resp_t) + HEADER_SIZE)) {
+ qgs_msg_get_quote_resp_t *msg = (qgs_msg_get_quote_resp_t *)(task->receive_buf + HEADER_SIZE);
+ if (msg->selected_id_size != 0) {
+ error_report("QGS message selected ID was %d not 0",
+ msg->selected_id_size);
+ task->status_code = TDX_VP_GET_QUOTE_ERROR;
+ goto end;
+ }
+
+ if ((task->payload_len - HEADER_SIZE - sizeof(qgs_msg_get_quote_resp_t)) !=
+ msg->quote_size) {
+ error_report("QGS quote size %d should be %zu",
+ msg->quote_size,
+ (task->payload_len - sizeof(qgs_msg_get_quote_resp_t)));
+ task->status_code = TDX_VP_GET_QUOTE_ERROR;
+ goto end;
+ }
+ }
+
+ if (task->receive_buf_received == task->payload_len) {
+ size_t strip = HEADER_SIZE + sizeof(qgs_msg_get_quote_resp_t);
+ memmove(task->receive_buf,
+ task->receive_buf + strip,
+ task->receive_buf_received - strip);
+ task->receive_buf_received -= strip;
task->status_code = TDX_VP_GET_QUOTE_SUCCESS;
goto end;
}
@@ -158,6 +290,29 @@
{
struct TdxQuoteGenerator *quote_gen = task->quote_gen;
QIOChannelSocket *sioc;
+ qgs_msg_get_quote_req_t msg;
+
+ /* Prepare a QGS message prelude */
+ msg.header.major_version = QGS_MSG_LIB_MAJOR_VER;
+ msg.header.minor_version = QGS_MSG_LIB_MINOR_VER;
+ msg.header.type = GET_QUOTE_REQ;
+ msg.header.size = sizeof(msg) + task->send_data_size;
+ msg.header.error_code = 0;
+ msg.report_size = task->send_data_size;
+ msg.id_list_size = 0;
+
+ /* Make room to add the QGS message prelude */
+ task->send_data = g_renew(char,
+ task->send_data,
+ task->send_data_size + sizeof(msg) + HEADER_SIZE);
+ memmove(task->send_data + sizeof(msg) + HEADER_SIZE,
+ task->send_data,
+ task->send_data_size);
+ memcpy(task->send_data + HEADER_SIZE,
+ &msg,
+ sizeof(msg));
+ encode_header(task->send_data, HEADER_SIZE, task->send_data_size + sizeof(msg));
+ task->send_data_size += sizeof(msg) + HEADER_SIZE;
sioc = qio_channel_socket_new();
task->sioc = sioc;
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
2024-10-03 18:08 ` Daniel P. Berrangé
@ 2025-04-02 15:26 ` Xiaoyao Li
2025-04-02 15:49 ` Daniel P. Berrangé
0 siblings, 1 reply; 124+ messages in thread
From: Xiaoyao Li @ 2025-04-02 15:26 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Eric Blake, Markus Armbruster, Marcelo Tosatti,
kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang
Sorry for the late response.
KVM part of TDX attestation support is submitting again. QEMU part will
follow and we need to settle dowm this topic before QEMU patches submission.
On 10/4/2024 2:08 AM, Daniel P. Berrangé wrote:
> On Thu, Feb 29, 2024 at 01:37:10AM -0500, Xiaoyao Li wrote:
>> From: Isaku Yamahata <isaku.yamahata@intel.com>
>>
>> Add property "quote-generation-socket" to tdx-guest, which is a property
>> of type SocketAddress to specify Quote Generation Service(QGS).
>>
>> On request of GetQuote, it connects to the QGS socket, read request
>> data from shared guest memory, send the request data to the QGS,
>> and store the response into shared guest memory, at last notify
>> TD guest by interrupt.
>>
>> command line example:
>> qemu-system-x86_64 \
>> -object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \
>> -machine confidential-guest-support=tdx0
>>
>> Note, above example uses vsock type socket because the QGS we used
>> implements the vsock socket. It can be other types, like UNIX socket,
>> which depends on the implementation of QGS.
>>
>> To avoid no response from QGS server, setup a timer for the transaction.
>> If timeout, make it an error and interrupt guest. Define the threshold of
>> time to 30s at present, maybe change to other value if not appropriate.
>>
>> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
>> Codeveloped-by: Chenyi Qiang <chenyi.qiang@intel.com>
>> Signed-off-by: Chenyi Qiang <chenyi.qiang@intel.com>
>> Codeveloped-by: Xiaoyao Li <xiaoyao.li@intel.com>
>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>
>
>> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
>> index 49f94d9d46f4..7dfda507cc8c 100644
>> --- a/target/i386/kvm/tdx.c
>> +++ b/target/i386/kvm/tdx.c
>
>> +static int tdx_handle_get_quote(X86CPU *cpu, struct kvm_tdx_vmcall *vmcall)
>> +{
>> + struct tdx_generate_quote_task *task;
>> + struct tdx_get_quote_header hdr;
>> + hwaddr buf_gpa = vmcall->in_r12;
>> + uint64_t buf_len = vmcall->in_r13;
>> +
>> + QEMU_BUILD_BUG_ON(sizeof(struct tdx_get_quote_header) != TDX_GET_QUOTE_HDR_SIZE);
>> +
>> + vmcall->status_code = TDG_VP_VMCALL_INVALID_OPERAND;
>> +
>> + if (buf_len == 0) {
>> + return 0;
>> + }
>> +
>> + /* GPA must be shared. */
>> + if (!(buf_gpa & tdx_shared_bit(cpu))) {
>> + return 0;
>> + }
>> + buf_gpa &= ~tdx_shared_bit(cpu);
>> +
>> + if (!QEMU_IS_ALIGNED(buf_gpa, 4096) || !QEMU_IS_ALIGNED(buf_len, 4096)) {
>> + vmcall->status_code = TDG_VP_VMCALL_ALIGN_ERROR;
>> + return 0;
>> + }
>> +
>> + if (address_space_read(&address_space_memory, buf_gpa, MEMTXATTRS_UNSPECIFIED,
>> + &hdr, TDX_GET_QUOTE_HDR_SIZE) != MEMTX_OK) {
>> + error_report("TDX: get-quote: failed to read GetQuote header.\n");
>> + return -1;
>> + }
>> +
>> + if (le64_to_cpu(hdr.structure_version) != TDX_GET_QUOTE_STRUCTURE_VERSION) {
>> + return 0;
>> + }
>> +
>> + /*
>> + * Paranoid: Guest should clear error_code and out_len to avoid information
>> + * leak. Enforce it. The initial value of them doesn't matter for qemu to
>> + * process the request.
>> + */
>> + if (le64_to_cpu(hdr.error_code) != TDX_VP_GET_QUOTE_SUCCESS ||
>> + le32_to_cpu(hdr.out_len) != 0) {
>> + return 0;
>> + }
>> +
>> + /* Only safe-guard check to avoid too large buffer size. */
>> + if (buf_len > TDX_GET_QUOTE_MAX_BUF_LEN ||
>> + le32_to_cpu(hdr.in_len) > buf_len - TDX_GET_QUOTE_HDR_SIZE) {
>> + return 0;
>> + }
>> +
>> + vmcall->status_code = TDG_VP_VMCALL_SUCCESS;
>> + if (!tdx_guest->quote_generator) {
>> + hdr.error_code = cpu_to_le64(TDX_VP_GET_QUOTE_QGS_UNAVAILABLE);
>> + if (address_space_write(&address_space_memory, buf_gpa,
>> + MEMTXATTRS_UNSPECIFIED,
>> + &hdr, TDX_GET_QUOTE_HDR_SIZE) != MEMTX_OK) {
>> + error_report("TDX: failed to update GetQuote header.\n");
>> + return -1;
>> + }
>> + return 0;
>> + }
>> +
>> + qemu_mutex_lock(&tdx_guest->quote_generator->lock);
>> + if (tdx_guest->quote_generator->num >= TDX_MAX_GET_QUOTE_REQUEST) {
>> + qemu_mutex_unlock(&tdx_guest->quote_generator->lock);
>> + vmcall->status_code = TDG_VP_VMCALL_RETRY;
>> + return 0;
>> + }
>> + tdx_guest->quote_generator->num++;
>> + qemu_mutex_unlock(&tdx_guest->quote_generator->lock);
>> +
>> + /* Mark the buffer in-flight. */
>> + hdr.error_code = cpu_to_le64(TDX_VP_GET_QUOTE_IN_FLIGHT);
>> + if (address_space_write(&address_space_memory, buf_gpa,
>> + MEMTXATTRS_UNSPECIFIED,
>> + &hdr, TDX_GET_QUOTE_HDR_SIZE) != MEMTX_OK) {
>> + error_report("TDX: failed to update GetQuote header.\n");
>> + return -1;
>> + }
>> +
>> + task = g_malloc(sizeof(*task));
>> + task->buf_gpa = buf_gpa;
>> + task->payload_gpa = buf_gpa + TDX_GET_QUOTE_HDR_SIZE;
>> + task->payload_len = buf_len - TDX_GET_QUOTE_HDR_SIZE;
>> + task->hdr = hdr;
>> + task->quote_gen = tdx_guest->quote_generator;
>> + task->completion = tdx_get_quote_completion;
>> +
>> + task->send_data_size = le32_to_cpu(hdr.in_len);
>> + task->send_data = g_malloc(task->send_data_size);
>> + task->send_data_sent = 0;
>> +
>> + if (address_space_read(&address_space_memory, task->payload_gpa,
>> + MEMTXATTRS_UNSPECIFIED, task->send_data,
>> + task->send_data_size) != MEMTX_OK) {
>> + g_free(task->send_data);
>> + return -1;
>> + }
>
> In this method we've received "struct tdx_get_quote_header" from
> the guest OS, and the 'hdr.in_len' field in that struct tells us
> the payload to read from guest memory. This payload is treated as
> opaque by QEMU and sent over the UNIX socket directly to QGS with
> no validation of the payload.
>
> The payload is supposed to be a raw TDX report, that QGS will turn
> into a quote.
>
> Nothing guarantees that the guest OS has actually given QEMU a
> payload that represents a TDX report.
>
> The only validation done in this patch is to check the 'hdr.in_len'
> was not ridiculously huge:
>
> #define TDX_GET_QUOTE_MAX_BUF_LEN (128 * 1024)
>
> #define TDX_GET_QUOTE_HDR_SIZE 24
>
> ...
>
> /* Only safe-guard check to avoid too large buffer size. */
> if (buf_len > TDX_GET_QUOTE_MAX_BUF_LEN ||
> le32_to_cpu(hdr.in_len) > buf_len - TDX_GET_QUOTE_HDR_SIZE) {
> return 0;
> }
>
> IOW, hdr.in_len can be any value between 0 and 131048, and
> the payload data read can contain arbitrary bytes.
>
>
> Over in the QGS code, QGS historically had a socket protocol
> taking various messages from the libtdxattest library which
> were defined in this:
>
> https://github.com/intel/SGXDataCenterAttestationPrimitives/blob/main/QuoteGeneration/quote_wrapper/qgs_msg_lib/inc/qgs_msg_lib.h
>
> typedef enum _qgs_msg_type_t {
> GET_QUOTE_REQ = 0,
> GET_QUOTE_RESP = 1,
> GET_COLLATERAL_REQ = 2,
> GET_COLLATERAL_RESP = 3,
> GET_PLATFORM_INFO_REQ = 4,
> GET_PLATFORM_INFO_RESP = 5,
> QGS_MSG_TYPE_MAX
> } qgs_msg_type_t;
>
> typedef struct _qgs_msg_header_t {
> uint16_t major_version;
> uint16_t minor_version;
> uint32_t type;
> uint32_t size; // size of the whole message, include this header, in byte
> uint32_t error_code; // used in response only
> } qgs_msg_header_t;
>
> such messages are processed by the 'get_resp' method in QGS:
>
> https://github.com/intel/SGXDataCenterAttestationPrimitives/blob/main/QuoteGeneration/quote_wrapper/qgs/qgs_ql_logic.cpp#L78
>
> The 1.21 release of DCAP introduced a new "raw" mode in QGS which
> just receives the raw 1024 byte packet from the client which is
> supposed to be a raw TDX report. This is what this QEMU patch
> is relying on IIUC.
>
>
> The QGS daemon decides whether a client is speaking the formal
> protocol, or "raw" mode, by trying to interpret the incoming
> data as a 'qgs_msg_header_t' struct. If the header size looks
> wrong & it has exactly 1024 bytes, then QGS assumes it has got
> a raw TDX report:
>
> https://github.com/intel/SGXDataCenterAttestationPrimitives/blob/main/QuoteGeneration/quote_wrapper/qgs/qgs_server.cpp#L165
>
> This all works if the data QEMU gets from the guest is indeed a
> 1024 byte raw TDX report, but what happens if we face a malicious
> guest ?
>
> AFAICT, the guest OS is able to send a "qgs_msg_header_t" packet
> to QEMU, which QEMU blindly passes on to QGS. This allows the
> guest OS to invoke any of the three QGS commands - GET_QUOTE_REQ,
> GET_COLLATERAL_REQ, or GET_PLATFORM_INFO_REQ. Fortunately I think
> those three messages are all safe to invoke, but none the less,
> this should not be permitted, as it leaves a wide open door for
> possible future exploits.
>
> As mentioned before, I don't know why this raw mode was invented
> for QGS, when QEMU itself could just take the guest report and
> pack it into the 'GET_QUOTE_REQ' message format and send it to
> QGS. This prevents the guest OS from being able to exploit QEMU
> to invoke arbirtary QGS messages.
I guess the raw mode was introduced due to the design was changed to let
guest kernel to forward to TD report to host QGS via TDVMCALL instead of
guest application communicates with host QGS via vsock, and Linux TD
guest driver doesn't integrate any QGS protocol but just forward the raw
TD report data to KVM.
> IMHO, QEMU should be made to pack & unpack the TDX report from
> the guest into the GET_QUOTE_REQ / GET_QUOTE_RESP messages, and
> this "raw" mode should be removed to QGS as it is inherantly
> dangerous to have this magic protocol overloading.
There is no enforcement that the input data of TDVMCALL.GetQuote is the
raw data of TD report. It is just the current Linux tdx-guest driver of
tsm implementation send the raw data. For other TDX OS, or third-party
driver, they might encapsulate the raw TD report data with QGS message
header. For such cases, if QEMU adds another layer of package, it leads
to the wrong result.
If we are going to pack the input data of GETQUOTE in QEMU, it becomes a
hard requirement from QEMU that the input data of GETQUOTE must be raw
data of TD report.
> Below is a patch on top of this one that illustrates how QEMU
> could use the GET_QUOTE_REQ / GET_QUOTE_RESP messages and avoid
> the "raw" mode of QGS.
>
>> +
>> + task->receive_buf = g_malloc0(task->payload_len);
>> + task->receive_buf_received = 0;
>> +
>> + tdx_generate_quote(task);
>> +
>> + return 0;
>> +}
>
> --- qemu-9.0.0-rc3/target/i386/kvm/tdx-quote-generator.c 2024-10-02 11:05:31.328003392 -0400
> +++ qemu-9.0.0/target/i386/kvm/tdx-quote-generator.c 2024-10-03 13:46:25.744775539 -0400
> @@ -24,6 +24,61 @@
>
> OBJECT_DEFINE_TYPE(TdxQuoteGenerator, tdx_quote_generator, TDX_QUOTE_GENERATOR, OBJECT)
>
> +const uint32_t QGS_MSG_LIB_MAJOR_VER = 1;
> +const uint32_t QGS_MSG_LIB_MINOR_VER = 1;
> +
> +typedef enum _qgs_msg_type_t {
> + GET_QUOTE_REQ = 0,
> + GET_QUOTE_RESP = 1,
> + GET_COLLATERAL_REQ = 2,
> + GET_COLLATERAL_RESP = 3,
> + GET_PLATFORM_INFO_REQ = 4,
> + GET_PLATFORM_INFO_RESP = 5,
> + QGS_MSG_TYPE_MAX
> +} qgs_msg_type_t;
> +
> +typedef struct _qgs_msg_header_t {
> + uint16_t major_version;
> + uint16_t minor_version;
> + uint32_t type;
> + uint32_t size; // size of the whole message, include this header, in byte
> + uint32_t error_code; // used in response only
> +} qgs_msg_header_t;
> +
> +typedef struct _qgs_msg_get_quote_req_t {
> + qgs_msg_header_t header; // header.type = GET_QUOTE_REQ
> + uint32_t report_size; // cannot be 0
> + uint32_t id_list_size; // length of id_list, in byte, can be 0
> +} qgs_msg_get_quote_req_t;
> +
> +typedef struct _qgs_msg_get_quote_resp_s {
> + qgs_msg_header_t header; // header.type = GET_QUOTE_RESP
> + uint32_t selected_id_size; // can be 0 in case only one id is sent in request
> + uint32_t quote_size; // length of quote_data, in byte
> + uint8_t id_quote[]; // selected id followed by quote
> +} qgs_msg_get_quote_resp_t;
> +
> +const unsigned HEADER_SIZE = 4;
> +
> +static uint32_t decode_header(const char *buf, size_t len) {
> + if (len < HEADER_SIZE) {
> + return 0;
> + }
> + uint32_t msg_size = 0;
> + for (uint32_t i = 0; i < HEADER_SIZE; ++i) {
> + msg_size = msg_size * 256 + (buf[i] & 0xFF);
> + }
> + return msg_size;
> +}
> +
> +static void encode_header(char *buf, size_t len, uint32_t size) {
> + assert(len >= HEADER_SIZE);
> + buf[0] = ((size >> 24) & 0xFF);
> + buf[1] = ((size >> 16) & 0xFF);
> + buf[2] = ((size >> 8) & 0xFF);
> + buf[3] = (size & 0xFF);
> +}
> +
> static void tdx_quote_generator_finalize(Object *obj)
> {
> }
> @@ -70,9 +125,86 @@
> goto end;
> }
> }
> +
> + if (ret == 0) {
> + error_report("End of file before reply received");
> + task->status_code = TDX_VP_GET_QUOTE_ERROR;
> + goto end;
> + }
>
> task->receive_buf_received += ret;
> - if (ret == 0 || task->receive_buf_received == task->payload_len) {
> + if (task->receive_buf_received >= HEADER_SIZE) {
> + uint32_t len = decode_header(task->receive_buf,
> + task->receive_buf_received);
> + if (len == 0 ||
> + len > (task->payload_len - HEADER_SIZE)) {
> + error_report("Message len %u must be non-zero & less than %zu",
> + len, (task->payload_len - HEADER_SIZE));
> + task->status_code = TDX_VP_GET_QUOTE_ERROR;
> + goto end;
> + }
> +
> + /* Now we know the size, shrink to fit */
> + task->payload_len = HEADER_SIZE + len;
> + task->receive_buf = g_renew(char,
> + task->receive_buf,
> + task->payload_len);
> + }
> +
> + if (task->receive_buf_received >= (sizeof(qgs_msg_header_t) + HEADER_SIZE)) {
> + qgs_msg_header_t *hdr = (qgs_msg_header_t *)(task->receive_buf + HEADER_SIZE);
> + if (hdr->major_version != QGS_MSG_LIB_MAJOR_VER ||
> + hdr->minor_version != QGS_MSG_LIB_MINOR_VER) {
> + error_report("Invalid QGS message header version %d.%d\n",
> + hdr->major_version,
> + hdr->minor_version);
> + task->status_code = TDX_VP_GET_QUOTE_ERROR;
> + goto end;
> + }
> + if (hdr->type != GET_QUOTE_RESP) {
> + error_report("Invalid QGS message type %d\n",
> + hdr->type);
> + task->status_code = TDX_VP_GET_QUOTE_ERROR;
> + goto end;
> + }
> + if (hdr->size > (task->payload_len - HEADER_SIZE)) {
> + error_report("QGS message size %d exceeds payload capacity %zu",
> + hdr->size, task->payload_len);
> + task->status_code = TDX_VP_GET_QUOTE_ERROR;
> + goto end;
> + }
> + if (hdr->error_code != 0) {
> + error_report("QGS message error code %d",
> + hdr->error_code);
> + task->status_code = TDX_VP_GET_QUOTE_ERROR;
> + goto end;
> + }
> + }
> + if (task->receive_buf_received >= (sizeof(qgs_msg_get_quote_resp_t) + HEADER_SIZE)) {
> + qgs_msg_get_quote_resp_t *msg = (qgs_msg_get_quote_resp_t *)(task->receive_buf + HEADER_SIZE);
> + if (msg->selected_id_size != 0) {
> + error_report("QGS message selected ID was %d not 0",
> + msg->selected_id_size);
> + task->status_code = TDX_VP_GET_QUOTE_ERROR;
> + goto end;
> + }
> +
> + if ((task->payload_len - HEADER_SIZE - sizeof(qgs_msg_get_quote_resp_t)) !=
> + msg->quote_size) {
> + error_report("QGS quote size %d should be %zu",
> + msg->quote_size,
> + (task->payload_len - sizeof(qgs_msg_get_quote_resp_t)));
> + task->status_code = TDX_VP_GET_QUOTE_ERROR;
> + goto end;
> + }
> + }
> +
> + if (task->receive_buf_received == task->payload_len) {
> + size_t strip = HEADER_SIZE + sizeof(qgs_msg_get_quote_resp_t);
> + memmove(task->receive_buf,
> + task->receive_buf + strip,
> + task->receive_buf_received - strip);
> + task->receive_buf_received -= strip;
> task->status_code = TDX_VP_GET_QUOTE_SUCCESS;
> goto end;
> }
> @@ -158,6 +290,29 @@
> {
> struct TdxQuoteGenerator *quote_gen = task->quote_gen;
> QIOChannelSocket *sioc;
> + qgs_msg_get_quote_req_t msg;
> +
> + /* Prepare a QGS message prelude */
> + msg.header.major_version = QGS_MSG_LIB_MAJOR_VER;
> + msg.header.minor_version = QGS_MSG_LIB_MINOR_VER;
> + msg.header.type = GET_QUOTE_REQ;
> + msg.header.size = sizeof(msg) + task->send_data_size;
> + msg.header.error_code = 0;
> + msg.report_size = task->send_data_size;
> + msg.id_list_size = 0;
> +
> + /* Make room to add the QGS message prelude */
> + task->send_data = g_renew(char,
> + task->send_data,
> + task->send_data_size + sizeof(msg) + HEADER_SIZE);
> + memmove(task->send_data + sizeof(msg) + HEADER_SIZE,
> + task->send_data,
> + task->send_data_size);
> + memcpy(task->send_data + HEADER_SIZE,
> + &msg,
> + sizeof(msg));
> + encode_header(task->send_data, HEADER_SIZE, task->send_data_size + sizeof(msg));
> + task->send_data_size += sizeof(msg) + HEADER_SIZE;
>
> sioc = qio_channel_socket_new();
> task->sioc = sioc;
>
>
> With regards,
> Daniel
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
2025-04-02 15:26 ` Xiaoyao Li
@ 2025-04-02 15:49 ` Daniel P. Berrangé
2025-04-03 7:28 ` Xiaoyao Li
0 siblings, 1 reply; 124+ messages in thread
From: Daniel P. Berrangé @ 2025-04-02 15:49 UTC (permalink / raw)
To: Xiaoyao Li
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Eric Blake, Markus Armbruster, Marcelo Tosatti,
kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang
On Wed, Apr 02, 2025 at 11:26:11PM +0800, Xiaoyao Li wrote:
> Sorry for the late response.
>
> KVM part of TDX attestation support is submitting again. QEMU part will
> follow and we need to settle dowm this topic before QEMU patches submission.
>
> On 10/4/2024 2:08 AM, Daniel P. Berrangé wrote:
> > On Thu, Feb 29, 2024 at 01:37:10AM -0500, Xiaoyao Li wrote:
> > > From: Isaku Yamahata <isaku.yamahata@intel.com>
> > >
> > > Add property "quote-generation-socket" to tdx-guest, which is a property
> > > of type SocketAddress to specify Quote Generation Service(QGS).
> > >
> > > On request of GetQuote, it connects to the QGS socket, read request
> > > data from shared guest memory, send the request data to the QGS,
> > > and store the response into shared guest memory, at last notify
> > > TD guest by interrupt.
> > >
> > > command line example:
> > > qemu-system-x86_64 \
> > > -object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \
> > > -machine confidential-guest-support=tdx0
> > >
> > > Note, above example uses vsock type socket because the QGS we used
> > > implements the vsock socket. It can be other types, like UNIX socket,
> > > which depends on the implementation of QGS.
> > >
> > > To avoid no response from QGS server, setup a timer for the transaction.
> > > If timeout, make it an error and interrupt guest. Define the threshold of
> > > time to 30s at present, maybe change to other value if not appropriate.
> > >
> > > Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
> > > Codeveloped-by: Chenyi Qiang <chenyi.qiang@intel.com>
> > > Signed-off-by: Chenyi Qiang <chenyi.qiang@intel.com>
> > > Codeveloped-by: Xiaoyao Li <xiaoyao.li@intel.com>
> > > Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> >
> >
> > > diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
> > > index 49f94d9d46f4..7dfda507cc8c 100644
> > > --- a/target/i386/kvm/tdx.c
> > > +++ b/target/i386/kvm/tdx.c
> >
> > > +static int tdx_handle_get_quote(X86CPU *cpu, struct kvm_tdx_vmcall *vmcall)
> > > +{
> > > + struct tdx_generate_quote_task *task;
> > > + struct tdx_get_quote_header hdr;
> > > + hwaddr buf_gpa = vmcall->in_r12;
> > > + uint64_t buf_len = vmcall->in_r13;
> > > +
> > > + QEMU_BUILD_BUG_ON(sizeof(struct tdx_get_quote_header) != TDX_GET_QUOTE_HDR_SIZE);
> > > +
> > > + vmcall->status_code = TDG_VP_VMCALL_INVALID_OPERAND;
> > > +
> > > + if (buf_len == 0) {
> > > + return 0;
> > > + }
> > > +
> > > + /* GPA must be shared. */
> > > + if (!(buf_gpa & tdx_shared_bit(cpu))) {
> > > + return 0;
> > > + }
> > > + buf_gpa &= ~tdx_shared_bit(cpu);
> > > +
> > > + if (!QEMU_IS_ALIGNED(buf_gpa, 4096) || !QEMU_IS_ALIGNED(buf_len, 4096)) {
> > > + vmcall->status_code = TDG_VP_VMCALL_ALIGN_ERROR;
> > > + return 0;
> > > + }
> > > +
> > > + if (address_space_read(&address_space_memory, buf_gpa, MEMTXATTRS_UNSPECIFIED,
> > > + &hdr, TDX_GET_QUOTE_HDR_SIZE) != MEMTX_OK) {
> > > + error_report("TDX: get-quote: failed to read GetQuote header.\n");
> > > + return -1;
> > > + }
> > > +
> > > + if (le64_to_cpu(hdr.structure_version) != TDX_GET_QUOTE_STRUCTURE_VERSION) {
> > > + return 0;
> > > + }
> > > +
> > > + /*
> > > + * Paranoid: Guest should clear error_code and out_len to avoid information
> > > + * leak. Enforce it. The initial value of them doesn't matter for qemu to
> > > + * process the request.
> > > + */
> > > + if (le64_to_cpu(hdr.error_code) != TDX_VP_GET_QUOTE_SUCCESS ||
> > > + le32_to_cpu(hdr.out_len) != 0) {
> > > + return 0;
> > > + }
> > > +
> > > + /* Only safe-guard check to avoid too large buffer size. */
> > > + if (buf_len > TDX_GET_QUOTE_MAX_BUF_LEN ||
> > > + le32_to_cpu(hdr.in_len) > buf_len - TDX_GET_QUOTE_HDR_SIZE) {
> > > + return 0;
> > > + }
> > > +
> > > + vmcall->status_code = TDG_VP_VMCALL_SUCCESS;
> > > + if (!tdx_guest->quote_generator) {
> > > + hdr.error_code = cpu_to_le64(TDX_VP_GET_QUOTE_QGS_UNAVAILABLE);
> > > + if (address_space_write(&address_space_memory, buf_gpa,
> > > + MEMTXATTRS_UNSPECIFIED,
> > > + &hdr, TDX_GET_QUOTE_HDR_SIZE) != MEMTX_OK) {
> > > + error_report("TDX: failed to update GetQuote header.\n");
> > > + return -1;
> > > + }
> > > + return 0;
> > > + }
> > > +
> > > + qemu_mutex_lock(&tdx_guest->quote_generator->lock);
> > > + if (tdx_guest->quote_generator->num >= TDX_MAX_GET_QUOTE_REQUEST) {
> > > + qemu_mutex_unlock(&tdx_guest->quote_generator->lock);
> > > + vmcall->status_code = TDG_VP_VMCALL_RETRY;
> > > + return 0;
> > > + }
> > > + tdx_guest->quote_generator->num++;
> > > + qemu_mutex_unlock(&tdx_guest->quote_generator->lock);
> > > +
> > > + /* Mark the buffer in-flight. */
> > > + hdr.error_code = cpu_to_le64(TDX_VP_GET_QUOTE_IN_FLIGHT);
> > > + if (address_space_write(&address_space_memory, buf_gpa,
> > > + MEMTXATTRS_UNSPECIFIED,
> > > + &hdr, TDX_GET_QUOTE_HDR_SIZE) != MEMTX_OK) {
> > > + error_report("TDX: failed to update GetQuote header.\n");
> > > + return -1;
> > > + }
> > > +
> > > + task = g_malloc(sizeof(*task));
> > > + task->buf_gpa = buf_gpa;
> > > + task->payload_gpa = buf_gpa + TDX_GET_QUOTE_HDR_SIZE;
> > > + task->payload_len = buf_len - TDX_GET_QUOTE_HDR_SIZE;
> > > + task->hdr = hdr;
> > > + task->quote_gen = tdx_guest->quote_generator;
> > > + task->completion = tdx_get_quote_completion;
> > > +
> > > + task->send_data_size = le32_to_cpu(hdr.in_len);
> > > + task->send_data = g_malloc(task->send_data_size);
> > > + task->send_data_sent = 0;
> > > +
> > > + if (address_space_read(&address_space_memory, task->payload_gpa,
> > > + MEMTXATTRS_UNSPECIFIED, task->send_data,
> > > + task->send_data_size) != MEMTX_OK) {
> > > + g_free(task->send_data);
> > > + return -1;
> > > + }
> >
> > In this method we've received "struct tdx_get_quote_header" from
> > the guest OS, and the 'hdr.in_len' field in that struct tells us
> > the payload to read from guest memory. This payload is treated as
> > opaque by QEMU and sent over the UNIX socket directly to QGS with
> > no validation of the payload.
> >
> > The payload is supposed to be a raw TDX report, that QGS will turn
> > into a quote.
> >
> > Nothing guarantees that the guest OS has actually given QEMU a
> > payload that represents a TDX report.
> >
> > The only validation done in this patch is to check the 'hdr.in_len'
> > was not ridiculously huge:
> >
> > #define TDX_GET_QUOTE_MAX_BUF_LEN (128 * 1024)
> >
> > #define TDX_GET_QUOTE_HDR_SIZE 24
> >
> > ...
> > /* Only safe-guard check to avoid too large buffer size. */
> > if (buf_len > TDX_GET_QUOTE_MAX_BUF_LEN ||
> > le32_to_cpu(hdr.in_len) > buf_len - TDX_GET_QUOTE_HDR_SIZE) {
> > return 0;
> > }
> >
> > IOW, hdr.in_len can be any value between 0 and 131048, and
> > the payload data read can contain arbitrary bytes.
> >
> >
> > Over in the QGS code, QGS historically had a socket protocol
> > taking various messages from the libtdxattest library which
> > were defined in this:
> >
> > https://github.com/intel/SGXDataCenterAttestationPrimitives/blob/main/QuoteGeneration/quote_wrapper/qgs_msg_lib/inc/qgs_msg_lib.h
> >
> > typedef enum _qgs_msg_type_t {
> > GET_QUOTE_REQ = 0,
> > GET_QUOTE_RESP = 1,
> > GET_COLLATERAL_REQ = 2,
> > GET_COLLATERAL_RESP = 3,
> > GET_PLATFORM_INFO_REQ = 4,
> > GET_PLATFORM_INFO_RESP = 5,
> > QGS_MSG_TYPE_MAX
> > } qgs_msg_type_t;
> >
> > typedef struct _qgs_msg_header_t {
> > uint16_t major_version;
> > uint16_t minor_version;
> > uint32_t type;
> > uint32_t size; // size of the whole message, include this header, in byte
> > uint32_t error_code; // used in response only
> > } qgs_msg_header_t;
> >
> > such messages are processed by the 'get_resp' method in QGS:
> >
> > https://github.com/intel/SGXDataCenterAttestationPrimitives/blob/main/QuoteGeneration/quote_wrapper/qgs/qgs_ql_logic.cpp#L78
> >
> > The 1.21 release of DCAP introduced a new "raw" mode in QGS which
> > just receives the raw 1024 byte packet from the client which is
> > supposed to be a raw TDX report. This is what this QEMU patch
> > is relying on IIUC.
> >
> >
> > The QGS daemon decides whether a client is speaking the formal
> > protocol, or "raw" mode, by trying to interpret the incoming
> > data as a 'qgs_msg_header_t' struct. If the header size looks
> > wrong & it has exactly 1024 bytes, then QGS assumes it has got
> > a raw TDX report:
> >
> > https://github.com/intel/SGXDataCenterAttestationPrimitives/blob/main/QuoteGeneration/quote_wrapper/qgs/qgs_server.cpp#L165
> >
> > This all works if the data QEMU gets from the guest is indeed a
> > 1024 byte raw TDX report, but what happens if we face a malicious
> > guest ?
> >
> > AFAICT, the guest OS is able to send a "qgs_msg_header_t" packet
> > to QEMU, which QEMU blindly passes on to QGS. This allows the
> > guest OS to invoke any of the three QGS commands - GET_QUOTE_REQ,
> > GET_COLLATERAL_REQ, or GET_PLATFORM_INFO_REQ. Fortunately I think
> > those three messages are all safe to invoke, but none the less,
> > this should not be permitted, as it leaves a wide open door for
> > possible future exploits.
> >
> > As mentioned before, I don't know why this raw mode was invented
> > for QGS, when QEMU itself could just take the guest report and
> > pack it into the 'GET_QUOTE_REQ' message format and send it to
> > QGS. This prevents the guest OS from being able to exploit QEMU
> > to invoke arbirtary QGS messages.
>
> I guess the raw mode was introduced due to the design was changed to let
> guest kernel to forward to TD report to host QGS via TDVMCALL instead of
> guest application communicates with host QGS via vsock, and Linux TD guest
> driver doesn't integrate any QGS protocol but just forward the raw TD report
> data to KVM.
>
> > IMHO, QEMU should be made to pack & unpack the TDX report from
> > the guest into the GET_QUOTE_REQ / GET_QUOTE_RESP messages, and
> > this "raw" mode should be removed to QGS as it is inherantly
> > dangerous to have this magic protocol overloading.
>
> There is no enforcement that the input data of TDVMCALL.GetQuote is the raw
> data of TD report. It is just the current Linux tdx-guest driver of tsm
> implementation send the raw data. For other TDX OS, or third-party driver,
> they might encapsulate the raw TD report data with QGS message header. For
> such cases, if QEMU adds another layer of package, it leads to the wrong
> result.
If I look at the GHCI spec
https://cdrdv2-public.intel.com/726790/TDX%20Guest-Hypervisor%20Communication%20Interface_1.0_344426_006%20-%2020230311.pdf
In "3.3 TDG.VP.VMCALL<GetQuote>", it indicates the parameter is a
"TDREPORT_STRUCT". IOW, it doesn't look valid to allow the guest to
send arbitrary other data as QGS protocol messages.
> If we are going to pack the input data of GETQUOTE in QEMU, it becomes a
> hard requirement from QEMU that the input data of GETQUOTE must be raw data
> of TD report.
AFAICT it must be a raw TDREPORT_STRUCT per the spec and thus QEMU must not
allow anything different, as that exposes a significantly larger security
attack surface on the QGS daemon.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
2025-04-02 15:49 ` Daniel P. Berrangé
@ 2025-04-03 7:28 ` Xiaoyao Li
2025-04-03 8:10 ` Daniel P. Berrangé
0 siblings, 1 reply; 124+ messages in thread
From: Xiaoyao Li @ 2025-04-03 7:28 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Eric Blake, Markus Armbruster, Marcelo Tosatti,
kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang
On 4/2/2025 11:49 PM, Daniel P. Berrangé wrote:
> On Wed, Apr 02, 2025 at 11:26:11PM +0800, Xiaoyao Li wrote:
>> Sorry for the late response.
>>
>> KVM part of TDX attestation support is submitting again. QEMU part will
>> follow and we need to settle dowm this topic before QEMU patches submission.
>>
>> On 10/4/2024 2:08 AM, Daniel P. Berrangé wrote:
>>> On Thu, Feb 29, 2024 at 01:37:10AM -0500, Xiaoyao Li wrote:
>>>> From: Isaku Yamahata <isaku.yamahata@intel.com>
>>>>
>>>> Add property "quote-generation-socket" to tdx-guest, which is a property
>>>> of type SocketAddress to specify Quote Generation Service(QGS).
>>>>
>>>> On request of GetQuote, it connects to the QGS socket, read request
>>>> data from shared guest memory, send the request data to the QGS,
>>>> and store the response into shared guest memory, at last notify
>>>> TD guest by interrupt.
>>>>
>>>> command line example:
>>>> qemu-system-x86_64 \
>>>> -object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \
>>>> -machine confidential-guest-support=tdx0
>>>>
>>>> Note, above example uses vsock type socket because the QGS we used
>>>> implements the vsock socket. It can be other types, like UNIX socket,
>>>> which depends on the implementation of QGS.
>>>>
>>>> To avoid no response from QGS server, setup a timer for the transaction.
>>>> If timeout, make it an error and interrupt guest. Define the threshold of
>>>> time to 30s at present, maybe change to other value if not appropriate.
>>>>
>>>> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
>>>> Codeveloped-by: Chenyi Qiang <chenyi.qiang@intel.com>
>>>> Signed-off-by: Chenyi Qiang <chenyi.qiang@intel.com>
>>>> Codeveloped-by: Xiaoyao Li <xiaoyao.li@intel.com>
>>>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>>>
>>>
>>>> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
>>>> index 49f94d9d46f4..7dfda507cc8c 100644
>>>> --- a/target/i386/kvm/tdx.c
>>>> +++ b/target/i386/kvm/tdx.c
>>>
>>>> +static int tdx_handle_get_quote(X86CPU *cpu, struct kvm_tdx_vmcall *vmcall)
>>>> +{
>>>> + struct tdx_generate_quote_task *task;
>>>> + struct tdx_get_quote_header hdr;
>>>> + hwaddr buf_gpa = vmcall->in_r12;
>>>> + uint64_t buf_len = vmcall->in_r13;
>>>> +
>>>> + QEMU_BUILD_BUG_ON(sizeof(struct tdx_get_quote_header) != TDX_GET_QUOTE_HDR_SIZE);
>>>> +
>>>> + vmcall->status_code = TDG_VP_VMCALL_INVALID_OPERAND;
>>>> +
>>>> + if (buf_len == 0) {
>>>> + return 0;
>>>> + }
>>>> +
>>>> + /* GPA must be shared. */
>>>> + if (!(buf_gpa & tdx_shared_bit(cpu))) {
>>>> + return 0;
>>>> + }
>>>> + buf_gpa &= ~tdx_shared_bit(cpu);
>>>> +
>>>> + if (!QEMU_IS_ALIGNED(buf_gpa, 4096) || !QEMU_IS_ALIGNED(buf_len, 4096)) {
>>>> + vmcall->status_code = TDG_VP_VMCALL_ALIGN_ERROR;
>>>> + return 0;
>>>> + }
>>>> +
>>>> + if (address_space_read(&address_space_memory, buf_gpa, MEMTXATTRS_UNSPECIFIED,
>>>> + &hdr, TDX_GET_QUOTE_HDR_SIZE) != MEMTX_OK) {
>>>> + error_report("TDX: get-quote: failed to read GetQuote header.\n");
>>>> + return -1;
>>>> + }
>>>> +
>>>> + if (le64_to_cpu(hdr.structure_version) != TDX_GET_QUOTE_STRUCTURE_VERSION) {
>>>> + return 0;
>>>> + }
>>>> +
>>>> + /*
>>>> + * Paranoid: Guest should clear error_code and out_len to avoid information
>>>> + * leak. Enforce it. The initial value of them doesn't matter for qemu to
>>>> + * process the request.
>>>> + */
>>>> + if (le64_to_cpu(hdr.error_code) != TDX_VP_GET_QUOTE_SUCCESS ||
>>>> + le32_to_cpu(hdr.out_len) != 0) {
>>>> + return 0;
>>>> + }
>>>> +
>>>> + /* Only safe-guard check to avoid too large buffer size. */
>>>> + if (buf_len > TDX_GET_QUOTE_MAX_BUF_LEN ||
>>>> + le32_to_cpu(hdr.in_len) > buf_len - TDX_GET_QUOTE_HDR_SIZE) {
>>>> + return 0;
>>>> + }
>>>> +
>>>> + vmcall->status_code = TDG_VP_VMCALL_SUCCESS;
>>>> + if (!tdx_guest->quote_generator) {
>>>> + hdr.error_code = cpu_to_le64(TDX_VP_GET_QUOTE_QGS_UNAVAILABLE);
>>>> + if (address_space_write(&address_space_memory, buf_gpa,
>>>> + MEMTXATTRS_UNSPECIFIED,
>>>> + &hdr, TDX_GET_QUOTE_HDR_SIZE) != MEMTX_OK) {
>>>> + error_report("TDX: failed to update GetQuote header.\n");
>>>> + return -1;
>>>> + }
>>>> + return 0;
>>>> + }
>>>> +
>>>> + qemu_mutex_lock(&tdx_guest->quote_generator->lock);
>>>> + if (tdx_guest->quote_generator->num >= TDX_MAX_GET_QUOTE_REQUEST) {
>>>> + qemu_mutex_unlock(&tdx_guest->quote_generator->lock);
>>>> + vmcall->status_code = TDG_VP_VMCALL_RETRY;
>>>> + return 0;
>>>> + }
>>>> + tdx_guest->quote_generator->num++;
>>>> + qemu_mutex_unlock(&tdx_guest->quote_generator->lock);
>>>> +
>>>> + /* Mark the buffer in-flight. */
>>>> + hdr.error_code = cpu_to_le64(TDX_VP_GET_QUOTE_IN_FLIGHT);
>>>> + if (address_space_write(&address_space_memory, buf_gpa,
>>>> + MEMTXATTRS_UNSPECIFIED,
>>>> + &hdr, TDX_GET_QUOTE_HDR_SIZE) != MEMTX_OK) {
>>>> + error_report("TDX: failed to update GetQuote header.\n");
>>>> + return -1;
>>>> + }
>>>> +
>>>> + task = g_malloc(sizeof(*task));
>>>> + task->buf_gpa = buf_gpa;
>>>> + task->payload_gpa = buf_gpa + TDX_GET_QUOTE_HDR_SIZE;
>>>> + task->payload_len = buf_len - TDX_GET_QUOTE_HDR_SIZE;
>>>> + task->hdr = hdr;
>>>> + task->quote_gen = tdx_guest->quote_generator;
>>>> + task->completion = tdx_get_quote_completion;
>>>> +
>>>> + task->send_data_size = le32_to_cpu(hdr.in_len);
>>>> + task->send_data = g_malloc(task->send_data_size);
>>>> + task->send_data_sent = 0;
>>>> +
>>>> + if (address_space_read(&address_space_memory, task->payload_gpa,
>>>> + MEMTXATTRS_UNSPECIFIED, task->send_data,
>>>> + task->send_data_size) != MEMTX_OK) {
>>>> + g_free(task->send_data);
>>>> + return -1;
>>>> + }
>>>
>>> In this method we've received "struct tdx_get_quote_header" from
>>> the guest OS, and the 'hdr.in_len' field in that struct tells us
>>> the payload to read from guest memory. This payload is treated as
>>> opaque by QEMU and sent over the UNIX socket directly to QGS with
>>> no validation of the payload.
>>>
>>> The payload is supposed to be a raw TDX report, that QGS will turn
>>> into a quote.
>>>
>>> Nothing guarantees that the guest OS has actually given QEMU a
>>> payload that represents a TDX report.
>>>
>>> The only validation done in this patch is to check the 'hdr.in_len'
>>> was not ridiculously huge:
>>>
>>> #define TDX_GET_QUOTE_MAX_BUF_LEN (128 * 1024)
>>>
>>> #define TDX_GET_QUOTE_HDR_SIZE 24
>>>
>>> ...
>>> /* Only safe-guard check to avoid too large buffer size. */
>>> if (buf_len > TDX_GET_QUOTE_MAX_BUF_LEN ||
>>> le32_to_cpu(hdr.in_len) > buf_len - TDX_GET_QUOTE_HDR_SIZE) {
>>> return 0;
>>> }
>>>
>>> IOW, hdr.in_len can be any value between 0 and 131048, and
>>> the payload data read can contain arbitrary bytes.
>>>
>>>
>>> Over in the QGS code, QGS historically had a socket protocol
>>> taking various messages from the libtdxattest library which
>>> were defined in this:
>>>
>>> https://github.com/intel/SGXDataCenterAttestationPrimitives/blob/main/QuoteGeneration/quote_wrapper/qgs_msg_lib/inc/qgs_msg_lib.h
>>>
>>> typedef enum _qgs_msg_type_t {
>>> GET_QUOTE_REQ = 0,
>>> GET_QUOTE_RESP = 1,
>>> GET_COLLATERAL_REQ = 2,
>>> GET_COLLATERAL_RESP = 3,
>>> GET_PLATFORM_INFO_REQ = 4,
>>> GET_PLATFORM_INFO_RESP = 5,
>>> QGS_MSG_TYPE_MAX
>>> } qgs_msg_type_t;
>>>
>>> typedef struct _qgs_msg_header_t {
>>> uint16_t major_version;
>>> uint16_t minor_version;
>>> uint32_t type;
>>> uint32_t size; // size of the whole message, include this header, in byte
>>> uint32_t error_code; // used in response only
>>> } qgs_msg_header_t;
>>>
>>> such messages are processed by the 'get_resp' method in QGS:
>>>
>>> https://github.com/intel/SGXDataCenterAttestationPrimitives/blob/main/QuoteGeneration/quote_wrapper/qgs/qgs_ql_logic.cpp#L78
>>>
>>> The 1.21 release of DCAP introduced a new "raw" mode in QGS which
>>> just receives the raw 1024 byte packet from the client which is
>>> supposed to be a raw TDX report. This is what this QEMU patch
>>> is relying on IIUC.
>>>
>>>
>>> The QGS daemon decides whether a client is speaking the formal
>>> protocol, or "raw" mode, by trying to interpret the incoming
>>> data as a 'qgs_msg_header_t' struct. If the header size looks
>>> wrong & it has exactly 1024 bytes, then QGS assumes it has got
>>> a raw TDX report:
>>>
>>> https://github.com/intel/SGXDataCenterAttestationPrimitives/blob/main/QuoteGeneration/quote_wrapper/qgs/qgs_server.cpp#L165
>>>
>>> This all works if the data QEMU gets from the guest is indeed a
>>> 1024 byte raw TDX report, but what happens if we face a malicious
>>> guest ?
>>>
>>> AFAICT, the guest OS is able to send a "qgs_msg_header_t" packet
>>> to QEMU, which QEMU blindly passes on to QGS. This allows the
>>> guest OS to invoke any of the three QGS commands - GET_QUOTE_REQ,
>>> GET_COLLATERAL_REQ, or GET_PLATFORM_INFO_REQ. Fortunately I think
>>> those three messages are all safe to invoke, but none the less,
>>> this should not be permitted, as it leaves a wide open door for
>>> possible future exploits.
>>>
>>> As mentioned before, I don't know why this raw mode was invented
>>> for QGS, when QEMU itself could just take the guest report and
>>> pack it into the 'GET_QUOTE_REQ' message format and send it to
>>> QGS. This prevents the guest OS from being able to exploit QEMU
>>> to invoke arbirtary QGS messages.
>>
>> I guess the raw mode was introduced due to the design was changed to let
>> guest kernel to forward to TD report to host QGS via TDVMCALL instead of
>> guest application communicates with host QGS via vsock, and Linux TD guest
>> driver doesn't integrate any QGS protocol but just forward the raw TD report
>> data to KVM.
>>
>>> IMHO, QEMU should be made to pack & unpack the TDX report from
>>> the guest into the GET_QUOTE_REQ / GET_QUOTE_RESP messages, and
>>> this "raw" mode should be removed to QGS as it is inherantly
>>> dangerous to have this magic protocol overloading.
>>
>> There is no enforcement that the input data of TDVMCALL.GetQuote is the raw
>> data of TD report. It is just the current Linux tdx-guest driver of tsm
>> implementation send the raw data. For other TDX OS, or third-party driver,
>> they might encapsulate the raw TD report data with QGS message header. For
>> such cases, if QEMU adds another layer of package, it leads to the wrong
>> result.
>
> If I look at the GHCI spec
>
> https://cdrdv2-public.intel.com/726790/TDX%20Guest-Hypervisor%20Communication%20Interface_1.0_344426_006%20-%2020230311.pdf
>
> In "3.3 TDG.VP.VMCALL<GetQuote>", it indicates the parameter is a
> "TDREPORT_STRUCT". IOW, it doesn't look valid to allow the guest to
> send arbitrary other data as QGS protocol messages.
In table 3-7, the description of R12 is
Shared GPA as input - the memory contains a TDREPORT_STRUCT.
The same buffer is used as output - the memory contains a TD Quote.
table 3-10, describes the detailed format of the shared GPA:
starting from offset 24 bytes, it is the "Data"
On input, the data filled by TD with input length. The data should
include TDREPORT_STRUCT. TD should zeroize the remaining buffer to
avoid information leak if size of shared GPA (R13) > Input Length.
It uses the word "contains" and "include", but without "only". So it is
not clear to me.
I will work with internal attestation folks to make it clearer that who
(TD guest or host VMM) is responsible to encapsulate the raw
TDERPORT_STRCUT with QGS MSG protocol, and update the spec accordingly.
>> If we are going to pack the input data of GETQUOTE in QEMU, it becomes a
>> hard requirement from QEMU that the input data of GETQUOTE must be raw data
>> of TD report.
>
> AFAICT it must be a raw TDREPORT_STRUCT per the spec and thus QEMU must not
> allow anything different, as that exposes a significantly larger security
> attack surface on the QGS daemon.
>
> With regards,
> Daniel
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
2025-04-03 7:28 ` Xiaoyao Li
@ 2025-04-03 8:10 ` Daniel P. Berrangé
2025-04-08 2:00 ` Xiaoyao Li
0 siblings, 1 reply; 124+ messages in thread
From: Daniel P. Berrangé @ 2025-04-03 8:10 UTC (permalink / raw)
To: Xiaoyao Li
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Eric Blake, Markus Armbruster, Marcelo Tosatti,
kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang
On Thu, Apr 03, 2025 at 03:28:43PM +0800, Xiaoyao Li wrote:
> On 4/2/2025 11:49 PM, Daniel P. Berrangé wrote:
> > On Wed, Apr 02, 2025 at 11:26:11PM +0800, Xiaoyao Li wrote:
> > >
> > > I guess the raw mode was introduced due to the design was changed to let
> > > guest kernel to forward to TD report to host QGS via TDVMCALL instead of
> > > guest application communicates with host QGS via vsock, and Linux TD guest
> > > driver doesn't integrate any QGS protocol but just forward the raw TD report
> > > data to KVM.
> > >
> > > > IMHO, QEMU should be made to pack & unpack the TDX report from
> > > > the guest into the GET_QUOTE_REQ / GET_QUOTE_RESP messages, and
> > > > this "raw" mode should be removed to QGS as it is inherantly
> > > > dangerous to have this magic protocol overloading.
> > >
> > > There is no enforcement that the input data of TDVMCALL.GetQuote is the raw
> > > data of TD report. It is just the current Linux tdx-guest driver of tsm
> > > implementation send the raw data. For other TDX OS, or third-party driver,
> > > they might encapsulate the raw TD report data with QGS message header. For
> > > such cases, if QEMU adds another layer of package, it leads to the wrong
> > > result.
> >
> > If I look at the GHCI spec
> >
> > https://cdrdv2-public.intel.com/726790/TDX%20Guest-Hypervisor%20Communication%20Interface_1.0_344426_006%20-%2020230311.pdf
> >
> > In "3.3 TDG.VP.VMCALL<GetQuote>", it indicates the parameter is a
> > "TDREPORT_STRUCT". IOW, it doesn't look valid to allow the guest to
> > send arbitrary other data as QGS protocol messages.
>
> In table 3-7, the description of R12 is
>
> Shared GPA as input - the memory contains a TDREPORT_STRUCT.
> The same buffer is used as output - the memory contains a TD Quote.
>
> table 3-10, describes the detailed format of the shared GPA:
>
> starting from offset 24 bytes, it is the "Data"
>
> On input, the data filled by TD with input length. The data should
> include TDREPORT_STRUCT. TD should zeroize the remaining buffer to
> avoid information leak if size of shared GPA (R13) > Input Length.
>
> It uses the word "contains" and "include", but without "only". So it is not
> clear to me.
>
> I will work with internal attestation folks to make it clearer that who (TD
> guest or host VMM) is responsible to encapsulate the raw TDERPORT_STRCUT
> with QGS MSG protocol, and update the spec accordingly.
To be clear, my strong preference is that the spec be updated to only
permit the raw TDREPORT_STRUCT.
IMHO allowing arbitrary QGS MSGs would be a significant host security
weakness, as it exposes a huge amount of the QGS codebase to direct
attack from the guest. QEMU needs to be able to block that attack
vector. Without that, the benefit/value of shuffling of TDREPORTs via
the GetQuote hypercall is largely eliminated, and might as well have
just exposed QGS over VSOCK.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote>
2025-04-03 8:10 ` Daniel P. Berrangé
@ 2025-04-08 2:00 ` Xiaoyao Li
0 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2025-04-08 2:00 UTC (permalink / raw)
To: Daniel P. Berrangé
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Eric Blake, Markus Armbruster, Marcelo Tosatti,
kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang
On 4/3/2025 4:10 PM, Daniel P. Berrangé wrote:
> On Thu, Apr 03, 2025 at 03:28:43PM +0800, Xiaoyao Li wrote:
>> On 4/2/2025 11:49 PM, Daniel P. Berrangé wrote:
>>> On Wed, Apr 02, 2025 at 11:26:11PM +0800, Xiaoyao Li wrote:
>>>>
>>>> I guess the raw mode was introduced due to the design was changed to let
>>>> guest kernel to forward to TD report to host QGS via TDVMCALL instead of
>>>> guest application communicates with host QGS via vsock, and Linux TD guest
>>>> driver doesn't integrate any QGS protocol but just forward the raw TD report
>>>> data to KVM.
>>>>
>>>>> IMHO, QEMU should be made to pack & unpack the TDX report from
>>>>> the guest into the GET_QUOTE_REQ / GET_QUOTE_RESP messages, and
>>>>> this "raw" mode should be removed to QGS as it is inherantly
>>>>> dangerous to have this magic protocol overloading.
>>>>
>>>> There is no enforcement that the input data of TDVMCALL.GetQuote is the raw
>>>> data of TD report. It is just the current Linux tdx-guest driver of tsm
>>>> implementation send the raw data. For other TDX OS, or third-party driver,
>>>> they might encapsulate the raw TD report data with QGS message header. For
>>>> such cases, if QEMU adds another layer of package, it leads to the wrong
>>>> result.
>>>
>>> If I look at the GHCI spec
>>>
>>> https://cdrdv2-public.intel.com/726790/TDX%20Guest-Hypervisor%20Communication%20Interface_1.0_344426_006%20-%2020230311.pdf
>>>
>>> In "3.3 TDG.VP.VMCALL<GetQuote>", it indicates the parameter is a
>>> "TDREPORT_STRUCT". IOW, it doesn't look valid to allow the guest to
>>> send arbitrary other data as QGS protocol messages.
>>
>> In table 3-7, the description of R12 is
>>
>> Shared GPA as input - the memory contains a TDREPORT_STRUCT.
>> The same buffer is used as output - the memory contains a TD Quote.
>>
>> table 3-10, describes the detailed format of the shared GPA:
>>
>> starting from offset 24 bytes, it is the "Data"
>>
>> On input, the data filled by TD with input length. The data should
>> include TDREPORT_STRUCT. TD should zeroize the remaining buffer to
>> avoid information leak if size of shared GPA (R13) > Input Length.
>>
>> It uses the word "contains" and "include", but without "only". So it is not
>> clear to me.
>>
>> I will work with internal attestation folks to make it clearer that who (TD
>> guest or host VMM) is responsible to encapsulate the raw TDERPORT_STRCUT
>> with QGS MSG protocol, and update the spec accordingly.
>
> To be clear, my strong preference is that the spec be updated to only
> permit the raw TDREPORT_STRUCT.
>
> IMHO allowing arbitrary QGS MSGs would be a significant host security
> weakness, as it exposes a huge amount of the QGS codebase to direct
> attack from the guest.
If I remember correctly, the QGS instance keeps the vsock interface so
that TD guest can communicate with QGS directly without going through
host VMM. (I'm not sure if latest QGS implementation still keeps it.)
So QGS should know how to protect itself.
Regarding QEMU avoids from being exploited to forward arbitrary data
from malicious TDX guest to QGS, QEMU can check the beginning of the
data to be a valid QGS MSG header before handing it over to QGS socket.
> QEMU needs to be able to block that attack
> vector. Without that, the benefit/value of shuffling of TDREPORTs via
> the GetQuote hypercall is largely eliminated, and might as well have
> just exposed QGS over VSOCK.
If I remember correctly, one of the reason we changed from TD guest
communicates with QGS directly with vsock to current TDVMCALL(GETQUOTE),
is some of the TD guest environment might not have network stack for
vsock to work.
Anyway, I don't have preference. Either is OK to me. Let's see what
decision the attestation folks will make.
> With regards,
> Daniel
^ permalink raw reply [flat|nested] 124+ messages in thread
* [PATCH v5 50/65] i386/tdx: handle TDG.VP.VMCALL<MapGPA> hypercall
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (48 preceding siblings ...)
2024-02-29 6:37 ` [PATCH v5 49/65] i386/tdx: handle TDG.VP.VMCALL<GetQuote> Xiaoyao Li
@ 2024-02-29 6:37 ` Xiaoyao Li
2024-02-29 6:37 ` [PATCH v5 51/65] i386/tdx: Handle TDG.VP.VMCALL<REPORT_FATAL_ERROR> Xiaoyao Li
` (14 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:37 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
From: Isaku Yamahata <isaku.yamahata@intel.com>
MapGPA is a hypercall to convert GPA from/to private GPA to/from shared GPA.
As the conversion function is already implemented as kvm_convert_memory,
wire it to TDX hypercall exit.
Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
accel/kvm/kvm-all.c | 2 +-
include/sysemu/kvm.h | 2 ++
target/i386/kvm/tdx.c | 54 +++++++++++++++++++++++++++++++++++++++++++
target/i386/kvm/tdx.h | 1 +
4 files changed, 58 insertions(+), 1 deletion(-)
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 9dc17a1b5f43..2c83b6d270f7 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -2930,7 +2930,7 @@ static void kvm_eat_signals(CPUState *cpu)
} while (sigismember(&chkset, SIG_IPI));
}
-static int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private)
+int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private)
{
MemoryRegionSection section;
ram_addr_t offset;
diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
index 82b547848130..78fcf1c3a617 100644
--- a/include/sysemu/kvm.h
+++ b/include/sysemu/kvm.h
@@ -550,4 +550,6 @@ int kvm_create_guest_memfd(uint64_t size, uint64_t flags, Error **errp);
int kvm_set_memory_attributes_private(hwaddr start, hwaddr size);
int kvm_set_memory_attributes_shared(hwaddr start, hwaddr size);
+
+int kvm_convert_memory(hwaddr start, hwaddr size, bool to_private);
#endif
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 7dfda507cc8c..88d245577594 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -898,6 +898,58 @@ static hwaddr tdx_shared_bit(X86CPU *cpu)
return (cpu->phys_bits > 48) ? BIT_ULL(51) : BIT_ULL(47);
}
+/* 64MB at most in one call. What value is appropriate? */
+#define TDX_MAP_GPA_MAX_LEN (64 * 1024 * 1024)
+
+static int tdx_handle_map_gpa(X86CPU *cpu, struct kvm_tdx_vmcall *vmcall)
+{
+ hwaddr shared_bit = tdx_shared_bit(cpu);
+ hwaddr gpa = vmcall->in_r12 & ~shared_bit;
+ bool private = !(vmcall->in_r12 & shared_bit);
+ hwaddr size = vmcall->in_r13;
+ bool retry = false;
+ int ret = 0;
+
+ vmcall->status_code = TDG_VP_VMCALL_INVALID_OPERAND;
+
+ if (!QEMU_IS_ALIGNED(gpa, 4096) || !QEMU_IS_ALIGNED(size, 4096)) {
+ vmcall->status_code = TDG_VP_VMCALL_ALIGN_ERROR;
+ return 0;
+ }
+
+ /* Overflow case. */
+ if (gpa + size < gpa) {
+ return 0;
+ }
+ if (gpa >= (1ULL << cpu->phys_bits) ||
+ gpa + size >= (1ULL << cpu->phys_bits)) {
+ return 0;
+ }
+
+ if (size > TDX_MAP_GPA_MAX_LEN) {
+ retry = true;
+ size = TDX_MAP_GPA_MAX_LEN;
+ }
+
+ if (size > 0) {
+ ret = kvm_convert_memory(gpa, size, private);
+ }
+
+ if (!ret) {
+ if (retry) {
+ vmcall->status_code = TDG_VP_VMCALL_RETRY;
+ vmcall->out_r11 = gpa + size;
+ if (!private) {
+ vmcall->out_r11 |= shared_bit;
+ }
+ } else {
+ vmcall->status_code = TDG_VP_VMCALL_SUCCESS;
+ }
+ }
+
+ return 0;
+}
+
static void tdx_get_quote_completion(struct tdx_generate_quote_task *task)
{
int ret;
@@ -1070,6 +1122,8 @@ static int tdx_handle_vmcall(X86CPU *cpu, struct kvm_tdx_vmcall *vmcall)
}
switch (vmcall->subfunction) {
+ case TDG_VP_VMCALL_MAP_GPA:
+ return tdx_handle_map_gpa(cpu, vmcall);
case TDG_VP_VMCALL_GET_QUOTE:
return tdx_handle_get_quote(cpu, vmcall);
case TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT:
diff --git a/target/i386/kvm/tdx.h b/target/i386/kvm/tdx.h
index b7ce793d6037..b6b8742e79f7 100644
--- a/target/i386/kvm/tdx.h
+++ b/target/i386/kvm/tdx.h
@@ -18,6 +18,7 @@ typedef struct TdxGuestClass {
ConfidentialGuestSupportClass parent_class;
} TdxGuestClass;
+#define TDG_VP_VMCALL_MAP_GPA 0x10001ULL
#define TDG_VP_VMCALL_GET_QUOTE 0x10002ULL
#define TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT 0x10004ULL
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 51/65] i386/tdx: Handle TDG.VP.VMCALL<REPORT_FATAL_ERROR>
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (49 preceding siblings ...)
2024-02-29 6:37 ` [PATCH v5 50/65] i386/tdx: handle TDG.VP.VMCALL<MapGPA> hypercall Xiaoyao Li
@ 2024-02-29 6:37 ` Xiaoyao Li
2024-02-29 6:37 ` [PATCH v5 52/65] i386/tdx: Wire TDX_REPORT_FATAL_ERROR with GuestPanic facility Xiaoyao Li
` (13 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:37 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
TD guest can use TDG.VP.VMCALL<REPORT_FATAL_ERROR> to request termination
with error message encoded in GPRs.
Parse and print the error message, and terminate the TD guest in the
handler.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
target/i386/kvm/tdx.c | 39 +++++++++++++++++++++++++++++++++++++++
target/i386/kvm/tdx.h | 1 +
2 files changed, 40 insertions(+)
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 88d245577594..6cd72a26b970 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -1092,6 +1092,43 @@ static int tdx_handle_get_quote(X86CPU *cpu, struct kvm_tdx_vmcall *vmcall)
return 0;
}
+static int tdx_handle_report_fatal_error(X86CPU *cpu,
+ struct kvm_tdx_vmcall *vmcall)
+{
+ uint64_t error_code = vmcall->in_r12;
+ char *message = NULL;
+
+ if (error_code & 0xffff) {
+ error_report("TDX: REPORT_FATAL_ERROR: invalid error code: "
+ "0x%lx\n", error_code);
+ return -1;
+ }
+
+ /* it has optional message */
+ if (vmcall->in_r14) {
+ uint64_t * tmp;
+
+#define GUEST_PANIC_INFO_TDX_MESSAGE_MAX 64
+ message = g_malloc0(GUEST_PANIC_INFO_TDX_MESSAGE_MAX + 1);
+
+ tmp = (uint64_t *)message;
+ /* The order is defined in TDX GHCI spec */
+ *(tmp++) = cpu_to_le64(vmcall->in_r14);
+ *(tmp++) = cpu_to_le64(vmcall->in_r15);
+ *(tmp++) = cpu_to_le64(vmcall->in_rbx);
+ *(tmp++) = cpu_to_le64(vmcall->in_rdi);
+ *(tmp++) = cpu_to_le64(vmcall->in_rsi);
+ *(tmp++) = cpu_to_le64(vmcall->in_r8);
+ *(tmp++) = cpu_to_le64(vmcall->in_r9);
+ *(tmp++) = cpu_to_le64(vmcall->in_rdx);
+ message[GUEST_PANIC_INFO_TDX_MESSAGE_MAX] = '\0';
+ assert((char *)tmp == message + GUEST_PANIC_INFO_TDX_MESSAGE_MAX);
+ }
+
+ error_report("TD guest reports fatal error. %s\n", message ? : "");
+ return -1;
+}
+
static int tdx_handle_setup_event_notify_interrupt(X86CPU *cpu,
struct kvm_tdx_vmcall *vmcall)
{
@@ -1126,6 +1163,8 @@ static int tdx_handle_vmcall(X86CPU *cpu, struct kvm_tdx_vmcall *vmcall)
return tdx_handle_map_gpa(cpu, vmcall);
case TDG_VP_VMCALL_GET_QUOTE:
return tdx_handle_get_quote(cpu, vmcall);
+ case TDG_VP_VMCALL_REPORT_FATAL_ERROR:
+ return tdx_handle_report_fatal_error(cpu, vmcall);
case TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT:
return tdx_handle_setup_event_notify_interrupt(cpu, vmcall);
default:
diff --git a/target/i386/kvm/tdx.h b/target/i386/kvm/tdx.h
index b6b8742e79f7..86b1f80d6f49 100644
--- a/target/i386/kvm/tdx.h
+++ b/target/i386/kvm/tdx.h
@@ -20,6 +20,7 @@ typedef struct TdxGuestClass {
#define TDG_VP_VMCALL_MAP_GPA 0x10001ULL
#define TDG_VP_VMCALL_GET_QUOTE 0x10002ULL
+#define TDG_VP_VMCALL_REPORT_FATAL_ERROR 0x10003ULL
#define TDG_VP_VMCALL_SETUP_EVENT_NOTIFY_INTERRUPT 0x10004ULL
#define TDG_VP_VMCALL_SUCCESS 0x0000000000000000ULL
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 52/65] i386/tdx: Wire TDX_REPORT_FATAL_ERROR with GuestPanic facility
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (50 preceding siblings ...)
2024-02-29 6:37 ` [PATCH v5 51/65] i386/tdx: Handle TDG.VP.VMCALL<REPORT_FATAL_ERROR> Xiaoyao Li
@ 2024-02-29 6:37 ` Xiaoyao Li
2024-02-29 8:51 ` Markus Armbruster
2024-02-29 6:37 ` [PATCH v5 53/65] pci-host/q35: Move PAM initialization above SMRAM initialization Xiaoyao Li
` (12 subsequent siblings)
64 siblings, 1 reply; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:37 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
Integrate TDX's TDX_REPORT_FATAL_ERROR into QEMU GuestPanic facility
Originated-from: Isaku Yamahata <isaku.yamahata@intel.com>
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
Changes in v5:
- mention additional error information in gpa when it presents;
- refine the documentation; (Markus)
Changes in v4:
- refine the documentation; (Markus)
Changes in v3:
- Add docmentation of new type and struct; (Daniel)
- refine the error message handling; (Daniel)
---
qapi/run-state.json | 31 +++++++++++++++++++++--
system/runstate.c | 58 +++++++++++++++++++++++++++++++++++++++++++
target/i386/kvm/tdx.c | 24 +++++++++++++++++-
3 files changed, 110 insertions(+), 3 deletions(-)
diff --git a/qapi/run-state.json b/qapi/run-state.json
index dd0770b379e5..b71dd1884eb6 100644
--- a/qapi/run-state.json
+++ b/qapi/run-state.json
@@ -483,10 +483,12 @@
#
# @s390: s390 guest panic information type (Since: 2.12)
#
+# @tdx: tdx guest panic information type (Since: 9.0)
+#
# Since: 2.9
##
{ 'enum': 'GuestPanicInformationType',
- 'data': [ 'hyper-v', 's390' ] }
+ 'data': [ 'hyper-v', 's390', 'tdx' ] }
##
# @GuestPanicInformation:
@@ -501,7 +503,8 @@
'base': {'type': 'GuestPanicInformationType'},
'discriminator': 'type',
'data': {'hyper-v': 'GuestPanicInformationHyperV',
- 's390': 'GuestPanicInformationS390'}}
+ 's390': 'GuestPanicInformationS390',
+ 'tdx' : 'GuestPanicInformationTdx'}}
##
# @GuestPanicInformationHyperV:
@@ -564,6 +567,30 @@
'psw-addr': 'uint64',
'reason': 'S390CrashReason'}}
+##
+# @GuestPanicInformationTdx:
+#
+# TDX Guest panic information specific to TDX, as specified in the
+# "Guest-Hypervisor Communication Interface (GHCI) Specification",
+# section TDG.VP.VMCALL<ReportFatalError>.
+#
+# @error-code: TD-specific error code
+#
+# @message: Human-readable error message provided by the guest. Not
+# to be trusted.
+#
+# @gpa: guest-physical address of a page that contains more verbose
+# error information, as zero-terminated string. Present when the
+# "GPA valid" bit (bit 63) is set in @error-code.
+#
+#
+# Since: 9.0
+##
+{'struct': 'GuestPanicInformationTdx',
+ 'data': {'error-code': 'uint64',
+ 'message': 'str',
+ '*gpa': 'uint64'}}
+
##
# @MEMORY_FAILURE:
#
diff --git a/system/runstate.c b/system/runstate.c
index d6ab860ecaa7..3b628c38739d 100644
--- a/system/runstate.c
+++ b/system/runstate.c
@@ -519,6 +519,52 @@ static void qemu_system_wakeup(void)
}
}
+static char* tdx_parse_panic_message(char *message)
+{
+ bool printable = false;
+ char *buf = NULL;
+ int len = 0, i;
+
+ /*
+ * Although message is defined as a json string, we shouldn't
+ * unconditionally treat it as is because the guest generated it and
+ * it's not necessarily trustable.
+ */
+ if (message) {
+ /* The caller guarantees the NUL-terminated string. */
+ len = strlen(message);
+
+ printable = len > 0;
+ for (i = 0; i < len; i++) {
+ if (!(0x20 <= message[i] && message[i] <= 0x7e)) {
+ printable = false;
+ break;
+ }
+ }
+ }
+
+ if (!printable && len) {
+ /* 3 = length of "%02x " */
+ buf = g_malloc(len * 3);
+ for (i = 0; i < len; i++) {
+ if (message[i] == '\0') {
+ break;
+ } else {
+ sprintf(buf + 3 * i, "%02x ", message[i]);
+ }
+ }
+ if (i > 0)
+ /* replace the last ' '(space) to NUL */
+ buf[i * 3 - 1] = '\0';
+ else
+ buf[0] = '\0';
+
+ return buf;
+ }
+
+ return message;
+}
+
void qemu_system_guest_panicked(GuestPanicInformation *info)
{
qemu_log_mask(LOG_GUEST_ERROR, "Guest crashed");
@@ -560,7 +606,19 @@ void qemu_system_guest_panicked(GuestPanicInformation *info)
S390CrashReason_str(info->u.s390.reason),
info->u.s390.psw_mask,
info->u.s390.psw_addr);
+ } else if (info->type == GUEST_PANIC_INFORMATION_TYPE_TDX) {
+ qemu_log_mask(LOG_GUEST_ERROR,
+ " TDX guest reports fatal error:"
+ " error code: 0x%#" PRIx64 " error message:\"%s\"\n",
+ info->u.tdx.error_code,
+ tdx_parse_panic_message(info->u.tdx.message));
+ if (info->u.tdx.error_code & (1ull << 63)) {
+ qemu_log_mask(LOG_GUEST_ERROR, "Additional error information "
+ "can be found at gpa page: 0x%#" PRIx64 "\n",
+ info->u.tdx.gpa);
+ }
}
+
qapi_free_GuestPanicInformation(info);
}
}
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 6cd72a26b970..811a3b81af99 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -20,6 +20,7 @@
#include "qom/object_interfaces.h"
#include "standard-headers/asm-x86/kvm_para.h"
#include "sysemu/kvm.h"
+#include "sysemu/runstate.h"
#include "sysemu/sysemu.h"
#include "exec/ramblock.h"
@@ -1092,11 +1093,26 @@ static int tdx_handle_get_quote(X86CPU *cpu, struct kvm_tdx_vmcall *vmcall)
return 0;
}
+static void tdx_panicked_on_fatal_error(X86CPU *cpu, uint64_t error_code,
+ char *message, uint64_t gpa)
+{
+ GuestPanicInformation *panic_info;
+
+ panic_info = g_new0(GuestPanicInformation, 1);
+ panic_info->type = GUEST_PANIC_INFORMATION_TYPE_TDX;
+ panic_info->u.tdx.error_code = error_code;
+ panic_info->u.tdx.message = message;
+ panic_info->u.tdx.gpa = gpa;
+
+ qemu_system_guest_panicked(panic_info);
+}
+
static int tdx_handle_report_fatal_error(X86CPU *cpu,
struct kvm_tdx_vmcall *vmcall)
{
uint64_t error_code = vmcall->in_r12;
char *message = NULL;
+ uint64_t gpa = -1ull;
if (error_code & 0xffff) {
error_report("TDX: REPORT_FATAL_ERROR: invalid error code: "
@@ -1125,7 +1141,13 @@ static int tdx_handle_report_fatal_error(X86CPU *cpu,
assert((char *)tmp == message + GUEST_PANIC_INFO_TDX_MESSAGE_MAX);
}
- error_report("TD guest reports fatal error. %s\n", message ? : "");
+#define TDX_REPORT_FATAL_ERROR_GPA_VALID BIT_ULL(63)
+ if (error_code & TDX_REPORT_FATAL_ERROR_GPA_VALID) {
+ gpa = vmcall->in_r13;
+ }
+
+ tdx_panicked_on_fatal_error(cpu, error_code, message, gpa);
+
return -1;
}
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* Re: [PATCH v5 52/65] i386/tdx: Wire TDX_REPORT_FATAL_ERROR with GuestPanic facility
2024-02-29 6:37 ` [PATCH v5 52/65] i386/tdx: Wire TDX_REPORT_FATAL_ERROR with GuestPanic facility Xiaoyao Li
@ 2024-02-29 8:51 ` Markus Armbruster
2024-03-07 11:30 ` Xiaoyao Li
0 siblings, 1 reply; 124+ messages in thread
From: Markus Armbruster @ 2024-02-29 8:51 UTC (permalink / raw)
To: Xiaoyao Li
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Marcelo Tosatti, kvm, qemu-devel, Michael Roth, Claudio Fontana,
Gerd Hoffmann, Isaku Yamahata, Chenyi Qiang
Xiaoyao Li <xiaoyao.li@intel.com> writes:
> Integrate TDX's TDX_REPORT_FATAL_ERROR into QEMU GuestPanic facility
>
> Originated-from: Isaku Yamahata <isaku.yamahata@intel.com>
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
> Changes in v5:
> - mention additional error information in gpa when it presents;
> - refine the documentation; (Markus)
>
> Changes in v4:
> - refine the documentation; (Markus)
>
> Changes in v3:
> - Add docmentation of new type and struct; (Daniel)
> - refine the error message handling; (Daniel)
> ---
> qapi/run-state.json | 31 +++++++++++++++++++++--
> system/runstate.c | 58 +++++++++++++++++++++++++++++++++++++++++++
> target/i386/kvm/tdx.c | 24 +++++++++++++++++-
> 3 files changed, 110 insertions(+), 3 deletions(-)
>
> diff --git a/qapi/run-state.json b/qapi/run-state.json
> index dd0770b379e5..b71dd1884eb6 100644
> --- a/qapi/run-state.json
> +++ b/qapi/run-state.json
> @@ -483,10 +483,12 @@
> #
> # @s390: s390 guest panic information type (Since: 2.12)
> #
> +# @tdx: tdx guest panic information type (Since: 9.0)
> +#
> # Since: 2.9
> ##
> { 'enum': 'GuestPanicInformationType',
> - 'data': [ 'hyper-v', 's390' ] }
> + 'data': [ 'hyper-v', 's390', 'tdx' ] }
>
> ##
> # @GuestPanicInformation:
> @@ -501,7 +503,8 @@
> 'base': {'type': 'GuestPanicInformationType'},
> 'discriminator': 'type',
> 'data': {'hyper-v': 'GuestPanicInformationHyperV',
> - 's390': 'GuestPanicInformationS390'}}
> + 's390': 'GuestPanicInformationS390',
> + 'tdx' : 'GuestPanicInformationTdx'}}
>
> ##
> # @GuestPanicInformationHyperV:
> @@ -564,6 +567,30 @@
> 'psw-addr': 'uint64',
> 'reason': 'S390CrashReason'}}
>
> +##
> +# @GuestPanicInformationTdx:
> +#
> +# TDX Guest panic information specific to TDX, as specified in the
> +# "Guest-Hypervisor Communication Interface (GHCI) Specification",
> +# section TDG.VP.VMCALL<ReportFatalError>.
> +#
> +# @error-code: TD-specific error code
> +#
> +# @message: Human-readable error message provided by the guest. Not
> +# to be trusted.
> +#
> +# @gpa: guest-physical address of a page that contains more verbose
> +# error information, as zero-terminated string. Present when the
> +# "GPA valid" bit (bit 63) is set in @error-code.
Uh, peeking at GHCI Spec section 3.4 TDG.VP.VMCALL<ReportFatalError>, I
see operand R12 consists of
bits name description
31:0 TD-specific error code TD-specific error code
Panic – 0x0.
Values – 0x1 to 0xFFFFFFFF
reserved.
62:32 TD-specific extended TD-specific extended error code.
error code TD software defined.
63 GPA Valid Set if the TD specified additional
information in the GPA parameter
(R13).
Is @error-code all of R12, or just bits 31:0?
If it's all of R12, description of @error-code as "TD-specific error
code" is misleading.
If it's just bits 31:0, then 'Present when the "GPA valid" bit (bit 63)
is set in @error-code' is wrong. Could go with 'Only present when the
guest provides this information'.
> +#
> +#
Drop one of these two lines, please.
> +# Since: 9.0
> +##
> +{'struct': 'GuestPanicInformationTdx',
> + 'data': {'error-code': 'uint64',
> + 'message': 'str',
> + '*gpa': 'uint64'}}
> +
> ##
> # @MEMORY_FAILURE:
> #
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 52/65] i386/tdx: Wire TDX_REPORT_FATAL_ERROR with GuestPanic facility
2024-02-29 8:51 ` Markus Armbruster
@ 2024-03-07 11:30 ` Xiaoyao Li
2024-03-07 13:51 ` Markus Armbruster
0 siblings, 1 reply; 124+ messages in thread
From: Xiaoyao Li @ 2024-03-07 11:30 UTC (permalink / raw)
To: Markus Armbruster
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Marcelo Tosatti, kvm, qemu-devel, Michael Roth, Claudio Fontana,
Gerd Hoffmann, Isaku Yamahata, Chenyi Qiang
On 2/29/2024 4:51 PM, Markus Armbruster wrote:
> Xiaoyao Li <xiaoyao.li@intel.com> writes:
>
>> Integrate TDX's TDX_REPORT_FATAL_ERROR into QEMU GuestPanic facility
>>
>> Originated-from: Isaku Yamahata <isaku.yamahata@intel.com>
>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>> ---
>> Changes in v5:
>> - mention additional error information in gpa when it presents;
>> - refine the documentation; (Markus)
>>
>> Changes in v4:
>> - refine the documentation; (Markus)
>>
>> Changes in v3:
>> - Add docmentation of new type and struct; (Daniel)
>> - refine the error message handling; (Daniel)
>> ---
>> qapi/run-state.json | 31 +++++++++++++++++++++--
>> system/runstate.c | 58 +++++++++++++++++++++++++++++++++++++++++++
>> target/i386/kvm/tdx.c | 24 +++++++++++++++++-
>> 3 files changed, 110 insertions(+), 3 deletions(-)
>>
>> diff --git a/qapi/run-state.json b/qapi/run-state.json
>> index dd0770b379e5..b71dd1884eb6 100644
>> --- a/qapi/run-state.json
>> +++ b/qapi/run-state.json
>> @@ -483,10 +483,12 @@
>> #
>> # @s390: s390 guest panic information type (Since: 2.12)
>> #
>> +# @tdx: tdx guest panic information type (Since: 9.0)
>> +#
>> # Since: 2.9
>> ##
>> { 'enum': 'GuestPanicInformationType',
>> - 'data': [ 'hyper-v', 's390' ] }
>> + 'data': [ 'hyper-v', 's390', 'tdx' ] }
>>
>> ##
>> # @GuestPanicInformation:
>> @@ -501,7 +503,8 @@
>> 'base': {'type': 'GuestPanicInformationType'},
>> 'discriminator': 'type',
>> 'data': {'hyper-v': 'GuestPanicInformationHyperV',
>> - 's390': 'GuestPanicInformationS390'}}
>> + 's390': 'GuestPanicInformationS390',
>> + 'tdx' : 'GuestPanicInformationTdx'}}
>>
>> ##
>> # @GuestPanicInformationHyperV:
>> @@ -564,6 +567,30 @@
>> 'psw-addr': 'uint64',
>> 'reason': 'S390CrashReason'}}
>>
>> +##
>> +# @GuestPanicInformationTdx:
>> +#
>> +# TDX Guest panic information specific to TDX, as specified in the
>> +# "Guest-Hypervisor Communication Interface (GHCI) Specification",
>> +# section TDG.VP.VMCALL<ReportFatalError>.
>> +#
>> +# @error-code: TD-specific error code
>> +#
>> +# @message: Human-readable error message provided by the guest. Not
>> +# to be trusted.
>> +#
>> +# @gpa: guest-physical address of a page that contains more verbose
>> +# error information, as zero-terminated string. Present when the
>> +# "GPA valid" bit (bit 63) is set in @error-code.
>
> Uh, peeking at GHCI Spec section 3.4 TDG.VP.VMCALL<ReportFatalError>, I
> see operand R12 consists of
>
> bits name description
> 31:0 TD-specific error code TD-specific error code
> Panic – 0x0.
> Values – 0x1 to 0xFFFFFFFF
> reserved.
> 62:32 TD-specific extended TD-specific extended error code.
> error code TD software defined.
> 63 GPA Valid Set if the TD specified additional
> information in the GPA parameter
> (R13).
>
> Is @error-code all of R12, or just bits 31:0?
>
> If it's all of R12, description of @error-code as "TD-specific error
> code" is misleading.
We pass all of R12 to @error_code.
Here it wants to use "error_code" as generic as the whole R12. Do you
have any better description of it ?
> If it's just bits 31:0, then 'Present when the "GPA valid" bit (bit 63)
> is set in @error-code' is wrong. Could go with 'Only present when the
> guest provides this information'.
>
>> +#
>> +#
>
> Drop one of these two lines, please.
>
>> +# Since: 9.0
>> +##
>> +{'struct': 'GuestPanicInformationTdx',
>> + 'data': {'error-code': 'uint64',
>> + 'message': 'str',
>> + '*gpa': 'uint64'}}
>> +
>> ##
>> # @MEMORY_FAILURE:
>> #
>
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 52/65] i386/tdx: Wire TDX_REPORT_FATAL_ERROR with GuestPanic facility
2024-03-07 11:30 ` Xiaoyao Li
@ 2024-03-07 13:51 ` Markus Armbruster
2024-03-11 1:28 ` Xiaoyao Li
0 siblings, 1 reply; 124+ messages in thread
From: Markus Armbruster @ 2024-03-07 13:51 UTC (permalink / raw)
To: Xiaoyao Li
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Marcelo Tosatti, kvm, qemu-devel, Michael Roth, Claudio Fontana,
Gerd Hoffmann, Isaku Yamahata, Chenyi Qiang
Xiaoyao Li <xiaoyao.li@intel.com> writes:
> On 2/29/2024 4:51 PM, Markus Armbruster wrote:
>> Xiaoyao Li <xiaoyao.li@intel.com> writes:
>>
>>> Integrate TDX's TDX_REPORT_FATAL_ERROR into QEMU GuestPanic facility
>>>
>>> Originated-from: Isaku Yamahata <isaku.yamahata@intel.com>
>>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>>> ---
>>> Changes in v5:
>>> - mention additional error information in gpa when it presents;
>>> - refine the documentation; (Markus)
>>>
>>> Changes in v4:
>>> - refine the documentation; (Markus)
>>>
>>> Changes in v3:
>>> - Add docmentation of new type and struct; (Daniel)
>>> - refine the error message handling; (Daniel)
>>> ---
>>> qapi/run-state.json | 31 +++++++++++++++++++++--
>>> system/runstate.c | 58 +++++++++++++++++++++++++++++++++++++++++++
>>> target/i386/kvm/tdx.c | 24 +++++++++++++++++-
>>> 3 files changed, 110 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/qapi/run-state.json b/qapi/run-state.json
>>> index dd0770b379e5..b71dd1884eb6 100644
>>> --- a/qapi/run-state.json
>>> +++ b/qapi/run-state.json
>>> @@ -483,10 +483,12 @@
>>> #
>>> # @s390: s390 guest panic information type (Since: 2.12)
>>> #
>>> +# @tdx: tdx guest panic information type (Since: 9.0)
>>> +#
>>> # Since: 2.9
>>> ##
>>> { 'enum': 'GuestPanicInformationType',
>>> - 'data': [ 'hyper-v', 's390' ] }
>>> + 'data': [ 'hyper-v', 's390', 'tdx' ] }
>>> ##
>>> # @GuestPanicInformation:
>>> @@ -501,7 +503,8 @@
>>> 'base': {'type': 'GuestPanicInformationType'},
>>> 'discriminator': 'type',
>>> 'data': {'hyper-v': 'GuestPanicInformationHyperV',
>>> - 's390': 'GuestPanicInformationS390'}}
>>> + 's390': 'GuestPanicInformationS390',
>>> + 'tdx' : 'GuestPanicInformationTdx'}}
>>> ##
>>> # @GuestPanicInformationHyperV:
>>> @@ -564,6 +567,30 @@
>>> 'psw-addr': 'uint64',
>>> 'reason': 'S390CrashReason'}}
>>> +##
>>> +# @GuestPanicInformationTdx:
>>> +#
>>> +# TDX Guest panic information specific to TDX, as specified in the
>>> +# "Guest-Hypervisor Communication Interface (GHCI) Specification",
>>> +# section TDG.VP.VMCALL<ReportFatalError>.
>>> +#
>>> +# @error-code: TD-specific error code
>>> +#
>>> +# @message: Human-readable error message provided by the guest. Not
>>> +# to be trusted.
>>> +#
>>> +# @gpa: guest-physical address of a page that contains more verbose
>>> +# error information, as zero-terminated string. Present when the
>>> +# "GPA valid" bit (bit 63) is set in @error-code.
>>
>> Uh, peeking at GHCI Spec section 3.4 TDG.VP.VMCALL<ReportFatalError>, I
>> see operand R12 consists of
>>
>> bits name description
>> 31:0 TD-specific error code TD-specific error code
>> Panic – 0x0.
>> Values – 0x1 to 0xFFFFFFFF
>> reserved.
>> 62:32 TD-specific extended TD-specific extended error code.
>> error code TD software defined.
>> 63 GPA Valid Set if the TD specified additional
>> information in the GPA parameter
>> (R13).
>> Is @error-code all of R12, or just bits 31:0?
>> If it's all of R12, description of @error-code as "TD-specific error
>> code" is misleading.
>
> We pass all of R12 to @error_code.
>
> Here it wants to use "error_code" as generic as the whole R12. Do you have any better description of it ?
Sadly, the spec is of no help: it doesn't name the entire thing, only
the three sub-fields TD-specific error code, TD-specific extended error
code, GPA valid.
We could take the hint, and provide the sub-fields instead:
* @error-code contains the TD-specific error code (bits 31:0)
* @extended-error-code contains the TD-specific extended error code
(bits 62:32)
* we don't need @gpa-valid, because it's the same as "@gpa is present"
If we decide to keep the single member, we do need another name for it.
@error-codes (plural) doesn't exactly feel wonderful, but it gives at
least a subtle hint that it's not just *the* error code.
>> If it's just bits 31:0, then 'Present when the "GPA valid" bit (bit 63)
>> is set in @error-code' is wrong. Could go with 'Only present when the
>> guest provides this information'.
>>
>>> +#
>>> +#
>>
>> Drop one of these two lines, please.
>>
>>> +# Since: 9.0
>>> +##
>>> +{'struct': 'GuestPanicInformationTdx',
>>> + 'data': {'error-code': 'uint64',
>>> + 'message': 'str',
>>> + '*gpa': 'uint64'}}
>>> +
>>> ##
>>> # @MEMORY_FAILURE:
>>> #
>>
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 52/65] i386/tdx: Wire TDX_REPORT_FATAL_ERROR with GuestPanic facility
2024-03-07 13:51 ` Markus Armbruster
@ 2024-03-11 1:28 ` Xiaoyao Li
2024-03-11 7:29 ` Markus Armbruster
0 siblings, 1 reply; 124+ messages in thread
From: Xiaoyao Li @ 2024-03-11 1:28 UTC (permalink / raw)
To: Markus Armbruster
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Marcelo Tosatti, kvm, qemu-devel, Michael Roth, Claudio Fontana,
Gerd Hoffmann, Isaku Yamahata, Chenyi Qiang
On 3/7/2024 9:51 PM, Markus Armbruster wrote:
> Xiaoyao Li <xiaoyao.li@intel.com> writes:
>
>> On 2/29/2024 4:51 PM, Markus Armbruster wrote:
>>> Xiaoyao Li <xiaoyao.li@intel.com> writes:
>>>
>>>> Integrate TDX's TDX_REPORT_FATAL_ERROR into QEMU GuestPanic facility
>>>>
>>>> Originated-from: Isaku Yamahata <isaku.yamahata@intel.com>
>>>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>>>> ---
>>>> Changes in v5:
>>>> - mention additional error information in gpa when it presents;
>>>> - refine the documentation; (Markus)
>>>>
>>>> Changes in v4:
>>>> - refine the documentation; (Markus)
>>>>
>>>> Changes in v3:
>>>> - Add docmentation of new type and struct; (Daniel)
>>>> - refine the error message handling; (Daniel)
>>>> ---
>>>> qapi/run-state.json | 31 +++++++++++++++++++++--
>>>> system/runstate.c | 58 +++++++++++++++++++++++++++++++++++++++++++
>>>> target/i386/kvm/tdx.c | 24 +++++++++++++++++-
>>>> 3 files changed, 110 insertions(+), 3 deletions(-)
>>>>
>>>> diff --git a/qapi/run-state.json b/qapi/run-state.json
>>>> index dd0770b379e5..b71dd1884eb6 100644
>>>> --- a/qapi/run-state.json
>>>> +++ b/qapi/run-state.json
>>>> @@ -483,10 +483,12 @@
>>>> #
>>>> # @s390: s390 guest panic information type (Since: 2.12)
>>>> #
>>>> +# @tdx: tdx guest panic information type (Since: 9.0)
>>>> +#
>>>> # Since: 2.9
>>>> ##
>>>> { 'enum': 'GuestPanicInformationType',
>>>> - 'data': [ 'hyper-v', 's390' ] }
>>>> + 'data': [ 'hyper-v', 's390', 'tdx' ] }
>>>> ##
>>>> # @GuestPanicInformation:
>>>> @@ -501,7 +503,8 @@
>>>> 'base': {'type': 'GuestPanicInformationType'},
>>>> 'discriminator': 'type',
>>>> 'data': {'hyper-v': 'GuestPanicInformationHyperV',
>>>> - 's390': 'GuestPanicInformationS390'}}
>>>> + 's390': 'GuestPanicInformationS390',
>>>> + 'tdx' : 'GuestPanicInformationTdx'}}
>>>> ##
>>>> # @GuestPanicInformationHyperV:
>>>> @@ -564,6 +567,30 @@
>>>> 'psw-addr': 'uint64',
>>>> 'reason': 'S390CrashReason'}}
>>>> +##
>>>> +# @GuestPanicInformationTdx:
>>>> +#
>>>> +# TDX Guest panic information specific to TDX, as specified in the
>>>> +# "Guest-Hypervisor Communication Interface (GHCI) Specification",
>>>> +# section TDG.VP.VMCALL<ReportFatalError>.
>>>> +#
>>>> +# @error-code: TD-specific error code
>>>> +#
>>>> +# @message: Human-readable error message provided by the guest. Not
>>>> +# to be trusted.
>>>> +#
>>>> +# @gpa: guest-physical address of a page that contains more verbose
>>>> +# error information, as zero-terminated string. Present when the
>>>> +# "GPA valid" bit (bit 63) is set in @error-code.
>>>
>>> Uh, peeking at GHCI Spec section 3.4 TDG.VP.VMCALL<ReportFatalError>, I
>>> see operand R12 consists of
>>>
>>> bits name description
>>> 31:0 TD-specific error code TD-specific error code
>>> Panic – 0x0.
>>> Values – 0x1 to 0xFFFFFFFF
>>> reserved.
>>> 62:32 TD-specific extended TD-specific extended error code.
>>> error code TD software defined.
>>> 63 GPA Valid Set if the TD specified additional
>>> information in the GPA parameter
>>> (R13).
>>> Is @error-code all of R12, or just bits 31:0?
>>> If it's all of R12, description of @error-code as "TD-specific error
>>> code" is misleading.
>>
>> We pass all of R12 to @error_code.
>>
>> Here it wants to use "error_code" as generic as the whole R12. Do you have any better description of it ?
>
> Sadly, the spec is of no help: it doesn't name the entire thing, only
> the three sub-fields TD-specific error code, TD-specific extended error
> code, GPA valid.
>
> We could take the hint, and provide the sub-fields instead:
>
> * @error-code contains the TD-specific error code (bits 31:0)
>
> * @extended-error-code contains the TD-specific extended error code
> (bits 62:32)
>
> * we don't need @gpa-valid, because it's the same as "@gpa is present"
>
> If we decide to keep the single member, we do need another name for it.
> @error-codes (plural) doesn't exactly feel wonderful, but it gives at
> least a subtle hint that it's not just *the* error code.
The reason we only defined one single member, is that the
extended-error-code is not used now, and I believe it won't be used in
the near future.
If no objection from others, I will use @error-codes (plural) in the
next version.
>>> If it's just bits 31:0, then 'Present when the "GPA valid" bit (bit 63)
>>> is set in @error-code' is wrong. Could go with 'Only present when the
>>> guest provides this information'.
>>>
>>>> +#
>>>> +#
>>>
>>> Drop one of these two lines, please.
>>>
>>>> +# Since: 9.0
>>>> +##
>>>> +{'struct': 'GuestPanicInformationTdx',
>>>> + 'data': {'error-code': 'uint64',
>>>> + 'message': 'str',
>>>> + '*gpa': 'uint64'}}
>>>> +
>>>> ##
>>>> # @MEMORY_FAILURE:
>>>> #
>>>
>
^ permalink raw reply [flat|nested] 124+ messages in thread* Re: [PATCH v5 52/65] i386/tdx: Wire TDX_REPORT_FATAL_ERROR with GuestPanic facility
2024-03-11 1:28 ` Xiaoyao Li
@ 2024-03-11 7:29 ` Markus Armbruster
2024-03-12 7:27 ` Xiaoyao Li
0 siblings, 1 reply; 124+ messages in thread
From: Markus Armbruster @ 2024-03-11 7:29 UTC (permalink / raw)
To: Xiaoyao Li
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Marcelo Tosatti, kvm, qemu-devel, Michael Roth, Claudio Fontana,
Gerd Hoffmann, Isaku Yamahata, Chenyi Qiang
Xiaoyao Li <xiaoyao.li@intel.com> writes:
> On 3/7/2024 9:51 PM, Markus Armbruster wrote:
>> Xiaoyao Li <xiaoyao.li@intel.com> writes:
>>
>>> On 2/29/2024 4:51 PM, Markus Armbruster wrote:
>>>> Xiaoyao Li <xiaoyao.li@intel.com> writes:
>>>>
>>>>> Integrate TDX's TDX_REPORT_FATAL_ERROR into QEMU GuestPanic facility
>>>>>
>>>>> Originated-from: Isaku Yamahata <isaku.yamahata@intel.com>
>>>>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>>>>> ---
>>>>> Changes in v5:
>>>>> - mention additional error information in gpa when it presents;
>>>>> - refine the documentation; (Markus)
>>>>>
>>>>> Changes in v4:
>>>>> - refine the documentation; (Markus)
>>>>>
>>>>> Changes in v3:
>>>>> - Add docmentation of new type and struct; (Daniel)
>>>>> - refine the error message handling; (Daniel)
>>>>> ---
>>>>> qapi/run-state.json | 31 +++++++++++++++++++++--
>>>>> system/runstate.c | 58 +++++++++++++++++++++++++++++++++++++++++++
>>>>> target/i386/kvm/tdx.c | 24 +++++++++++++++++-
>>>>> 3 files changed, 110 insertions(+), 3 deletions(-)
>>>>>
>>>>> diff --git a/qapi/run-state.json b/qapi/run-state.json
>>>>> index dd0770b379e5..b71dd1884eb6 100644
>>>>> --- a/qapi/run-state.json
>>>>> +++ b/qapi/run-state.json
[...]
>>>>> @@ -564,6 +567,30 @@
>>>>> 'psw-addr': 'uint64',
>>>>> 'reason': 'S390CrashReason'}}
>>>>> +##
>>>>> +# @GuestPanicInformationTdx:
>>>>> +#
>>>>> +# TDX Guest panic information specific to TDX, as specified in the
>>>>> +# "Guest-Hypervisor Communication Interface (GHCI) Specification",
>>>>> +# section TDG.VP.VMCALL<ReportFatalError>.
>>>>> +#
>>>>> +# @error-code: TD-specific error code
>>>>> +#
>>>>> +# @message: Human-readable error message provided by the guest. Not
>>>>> +# to be trusted.
>>>>> +#
>>>>> +# @gpa: guest-physical address of a page that contains more verbose
>>>>> +# error information, as zero-terminated string. Present when the
>>>>> +# "GPA valid" bit (bit 63) is set in @error-code.
>>>>
>>>> Uh, peeking at GHCI Spec section 3.4 TDG.VP.VMCALL<ReportFatalError>, I
>>>> see operand R12 consists of
>>>>
>>>> bits name description
>>>> 31:0 TD-specific error code TD-specific error code
>>>> Panic – 0x0.
>>>> Values – 0x1 to 0xFFFFFFFF
>>>> reserved.
>>>> 62:32 TD-specific extended TD-specific extended error code.
>>>> error code TD software defined.
>>>> 63 GPA Valid Set if the TD specified additional
>>>> information in the GPA parameter
>>>> (R13).
>>>> Is @error-code all of R12, or just bits 31:0?
>>>> If it's all of R12, description of @error-code as "TD-specific error
>>>> code" is misleading.
>>>
>>> We pass all of R12 to @error_code.
>>>
>>> Here it wants to use "error_code" as generic as the whole R12. Do you have any better description of it ?
>>
>> Sadly, the spec is of no help: it doesn't name the entire thing, only
>> the three sub-fields TD-specific error code, TD-specific extended error
>> code, GPA valid.
>>
>> We could take the hint, and provide the sub-fields instead:
>>
>> * @error-code contains the TD-specific error code (bits 31:0)
>>
>> * @extended-error-code contains the TD-specific extended error code
>> (bits 62:32)
>>
>> * we don't need @gpa-valid, because it's the same as "@gpa is present"
>>
>> If we decide to keep the single member, we do need another name for it.
>> @error-codes (plural) doesn't exactly feel wonderful, but it gives at
>> least a subtle hint that it's not just *the* error code.
>
> The reason we only defined one single member, is that the
> extended-error-code is not used now, and I believe it won't be used in
> the near future.
Aha! Then I recommend
* @error-code contains the TD-specific error code (bits 31:0)
* Omit bits 62:32 from the reply; if we later find an actual use for
them, we can add a suitable member
* Omit bit 63, because it's the same as "@gpa is present"
> If no objection from others, I will use @error-codes (plural) in the
> next version.
I recommend to keep the @error-code name, but narrow its value to the
actual error code, i.e. bits 31:0.
>>>> If it's just bits 31:0, then 'Present when the "GPA valid" bit (bit 63)
>>>> is set in @error-code' is wrong. Could go with 'Only present when the
>>>> guest provides this information'.
[...]
^ permalink raw reply [flat|nested] 124+ messages in thread
* Re: [PATCH v5 52/65] i386/tdx: Wire TDX_REPORT_FATAL_ERROR with GuestPanic facility
2024-03-11 7:29 ` Markus Armbruster
@ 2024-03-12 7:27 ` Xiaoyao Li
0 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-03-12 7:27 UTC (permalink / raw)
To: Markus Armbruster
Cc: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Marcelo Tosatti, kvm, qemu-devel, Michael Roth, Claudio Fontana,
Gerd Hoffmann, Isaku Yamahata, Chenyi Qiang
On 3/11/2024 3:29 PM, Markus Armbruster wrote:
> Xiaoyao Li <xiaoyao.li@intel.com> writes:
>
>> On 3/7/2024 9:51 PM, Markus Armbruster wrote:
>>> Xiaoyao Li <xiaoyao.li@intel.com> writes:
>>>
>>>> On 2/29/2024 4:51 PM, Markus Armbruster wrote:
>>>>> Xiaoyao Li <xiaoyao.li@intel.com> writes:
>>>>>
>>>>>> Integrate TDX's TDX_REPORT_FATAL_ERROR into QEMU GuestPanic facility
>>>>>>
>>>>>> Originated-from: Isaku Yamahata <isaku.yamahata@intel.com>
>>>>>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>>>>>> ---
>>>>>> Changes in v5:
>>>>>> - mention additional error information in gpa when it presents;
>>>>>> - refine the documentation; (Markus)
>>>>>>
>>>>>> Changes in v4:
>>>>>> - refine the documentation; (Markus)
>>>>>>
>>>>>> Changes in v3:
>>>>>> - Add docmentation of new type and struct; (Daniel)
>>>>>> - refine the error message handling; (Daniel)
>>>>>> ---
>>>>>> qapi/run-state.json | 31 +++++++++++++++++++++--
>>>>>> system/runstate.c | 58 +++++++++++++++++++++++++++++++++++++++++++
>>>>>> target/i386/kvm/tdx.c | 24 +++++++++++++++++-
>>>>>> 3 files changed, 110 insertions(+), 3 deletions(-)
>>>>>>
>>>>>> diff --git a/qapi/run-state.json b/qapi/run-state.json
>>>>>> index dd0770b379e5..b71dd1884eb6 100644
>>>>>> --- a/qapi/run-state.json
>>>>>> +++ b/qapi/run-state.json
>
> [...]
>
>>>>>> @@ -564,6 +567,30 @@
>>>>>> 'psw-addr': 'uint64',
>>>>>> 'reason': 'S390CrashReason'}}
>>>>>> +##
>>>>>> +# @GuestPanicInformationTdx:
>>>>>> +#
>>>>>> +# TDX Guest panic information specific to TDX, as specified in the
>>>>>> +# "Guest-Hypervisor Communication Interface (GHCI) Specification",
>>>>>> +# section TDG.VP.VMCALL<ReportFatalError>.
>>>>>> +#
>>>>>> +# @error-code: TD-specific error code
>>>>>> +#
>>>>>> +# @message: Human-readable error message provided by the guest. Not
>>>>>> +# to be trusted.
>>>>>> +#
>>>>>> +# @gpa: guest-physical address of a page that contains more verbose
>>>>>> +# error information, as zero-terminated string. Present when the
>>>>>> +# "GPA valid" bit (bit 63) is set in @error-code.
>>>>>
>>>>> Uh, peeking at GHCI Spec section 3.4 TDG.VP.VMCALL<ReportFatalError>, I
>>>>> see operand R12 consists of
>>>>>
>>>>> bits name description
>>>>> 31:0 TD-specific error code TD-specific error code
>>>>> Panic – 0x0.
>>>>> Values – 0x1 to 0xFFFFFFFF
>>>>> reserved.
>>>>> 62:32 TD-specific extended TD-specific extended error code.
>>>>> error code TD software defined.
>>>>> 63 GPA Valid Set if the TD specified additional
>>>>> information in the GPA parameter
>>>>> (R13).
>>>>> Is @error-code all of R12, or just bits 31:0?
>>>>> If it's all of R12, description of @error-code as "TD-specific error
>>>>> code" is misleading.
>>>>
>>>> We pass all of R12 to @error_code.
>>>>
>>>> Here it wants to use "error_code" as generic as the whole R12. Do you have any better description of it ?
>>>
>>> Sadly, the spec is of no help: it doesn't name the entire thing, only
>>> the three sub-fields TD-specific error code, TD-specific extended error
>>> code, GPA valid.
>>>
>>> We could take the hint, and provide the sub-fields instead:
>>>
>>> * @error-code contains the TD-specific error code (bits 31:0)
>>>
>>> * @extended-error-code contains the TD-specific extended error code
>>> (bits 62:32)
>>>
>>> * we don't need @gpa-valid, because it's the same as "@gpa is present"
>>>
>>> If we decide to keep the single member, we do need another name for it.
>>> @error-codes (plural) doesn't exactly feel wonderful, but it gives at
>>> least a subtle hint that it's not just *the* error code.
>>
>> The reason we only defined one single member, is that the
>> extended-error-code is not used now, and I believe it won't be used in
>> the near future.
>
> Aha! Then I recommend
>
> * @error-code contains the TD-specific error code (bits 31:0)
>
> * Omit bits 62:32 from the reply; if we later find an actual use for
> them, we can add a suitable member
>
> * Omit bit 63, because it's the same as "@gpa is present"
>
>> If no objection from others, I will use @error-codes (plural) in the
>> next version.
>
> I recommend to keep the @error-code name, but narrow its value to the
> actual error code, i.e. bits 31:0.
It works for me. I will got this direction in the next version.
>>>>> If it's just bits 31:0, then 'Present when the "GPA valid" bit (bit 63)
>>>>> is set in @error-code' is wrong. Could go with 'Only present when the
>>>>> guest provides this information'.
>
> [...]
>
^ permalink raw reply [flat|nested] 124+ messages in thread
* [PATCH v5 53/65] pci-host/q35: Move PAM initialization above SMRAM initialization
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (51 preceding siblings ...)
2024-02-29 6:37 ` [PATCH v5 52/65] i386/tdx: Wire TDX_REPORT_FATAL_ERROR with GuestPanic facility Xiaoyao Li
@ 2024-02-29 6:37 ` Xiaoyao Li
2024-02-29 6:37 ` [PATCH v5 54/65] q35: Introduce smm_ranges property for q35-pci-host Xiaoyao Li
` (11 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:37 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
From: Isaku Yamahata <isaku.yamahata@intel.com>
In mch_realize(), process PAM initialization before SMRAM initialization so
that later patch can skill all the SMRAM related with a single check.
Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
hw/pci-host/q35.c | 19 ++++++++++---------
1 file changed, 10 insertions(+), 9 deletions(-)
diff --git a/hw/pci-host/q35.c b/hw/pci-host/q35.c
index 0d7d4e3f0860..98d4a7c253a6 100644
--- a/hw/pci-host/q35.c
+++ b/hw/pci-host/q35.c
@@ -568,6 +568,16 @@ static void mch_realize(PCIDevice *d, Error **errp)
/* setup pci memory mapping */
pc_pci_as_mapping_init(mch->system_memory, mch->pci_address_space);
+ /* PAM */
+ init_pam(&mch->pam_regions[0], OBJECT(mch), mch->ram_memory,
+ mch->system_memory, mch->pci_address_space,
+ PAM_BIOS_BASE, PAM_BIOS_SIZE);
+ for (i = 0; i < ARRAY_SIZE(mch->pam_regions) - 1; ++i) {
+ init_pam(&mch->pam_regions[i + 1], OBJECT(mch), mch->ram_memory,
+ mch->system_memory, mch->pci_address_space,
+ PAM_EXPAN_BASE + i * PAM_EXPAN_SIZE, PAM_EXPAN_SIZE);
+ }
+
/* if *disabled* show SMRAM to all CPUs */
memory_region_init_alias(&mch->smram_region, OBJECT(mch), "smram-region",
mch->pci_address_space, MCH_HOST_BRIDGE_SMRAM_C_BASE,
@@ -634,15 +644,6 @@ static void mch_realize(PCIDevice *d, Error **errp)
object_property_add_const_link(qdev_get_machine(), "smram",
OBJECT(&mch->smram));
-
- init_pam(&mch->pam_regions[0], OBJECT(mch), mch->ram_memory,
- mch->system_memory, mch->pci_address_space,
- PAM_BIOS_BASE, PAM_BIOS_SIZE);
- for (i = 0; i < ARRAY_SIZE(mch->pam_regions) - 1; ++i) {
- init_pam(&mch->pam_regions[i + 1], OBJECT(mch), mch->ram_memory,
- mch->system_memory, mch->pci_address_space,
- PAM_EXPAN_BASE + i * PAM_EXPAN_SIZE, PAM_EXPAN_SIZE);
- }
}
uint64_t mch_mcfg_base(void)
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 54/65] q35: Introduce smm_ranges property for q35-pci-host
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (52 preceding siblings ...)
2024-02-29 6:37 ` [PATCH v5 53/65] pci-host/q35: Move PAM initialization above SMRAM initialization Xiaoyao Li
@ 2024-02-29 6:37 ` Xiaoyao Li
2024-10-05 8:15 ` Michael Tokarev
2024-02-29 6:37 ` [PATCH v5 55/65] i386/tdx: Disable SMM for TDX VMs Xiaoyao Li
` (10 subsequent siblings)
64 siblings, 1 reply; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:37 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
From: Isaku Yamahata <isaku.yamahata@linux.intel.com>
Add a q35 property to check whether or not SMM ranges, e.g. SMRAM, TSEG,
etc... exist for the target platform. TDX doesn't support SMM and doesn't
play nice with QEMU modifying related guest memory ranges.
Signed-off-by: Isaku Yamahata <isaku.yamahata@linux.intel.com>
Co-developed-by: Sean Christopherson <sean.j.christopherson@intel.com>
Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
hw/i386/pc_q35.c | 2 ++
hw/pci-host/q35.c | 42 +++++++++++++++++++++++++++------------
include/hw/i386/pc.h | 1 +
include/hw/pci-host/q35.h | 1 +
4 files changed, 33 insertions(+), 13 deletions(-)
diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c
index 45a4102e75f5..dc3f68676ea0 100644
--- a/hw/i386/pc_q35.c
+++ b/hw/i386/pc_q35.c
@@ -212,6 +212,8 @@ static void pc_q35_init(MachineState *machine)
x86ms->above_4g_mem_size, NULL);
object_property_set_bool(phb, PCI_HOST_BYPASS_IOMMU,
pcms->default_bus_bypass_iommu, NULL);
+ object_property_set_bool(phb, PCI_HOST_PROP_SMM_RANGES,
+ x86_machine_is_smm_enabled(x86ms), NULL);
sysbus_realize_and_unref(SYS_BUS_DEVICE(phb), &error_fatal);
/* pci */
diff --git a/hw/pci-host/q35.c b/hw/pci-host/q35.c
index 98d4a7c253a6..0b6cbaed7ed5 100644
--- a/hw/pci-host/q35.c
+++ b/hw/pci-host/q35.c
@@ -179,6 +179,8 @@ static Property q35_host_props[] = {
mch.below_4g_mem_size, 0),
DEFINE_PROP_SIZE(PCI_HOST_ABOVE_4G_MEM_SIZE, Q35PCIHost,
mch.above_4g_mem_size, 0),
+ DEFINE_PROP_BOOL(PCI_HOST_PROP_SMM_RANGES, Q35PCIHost,
+ mch.has_smm_ranges, true),
DEFINE_PROP_BOOL("x-pci-hole64-fix", Q35PCIHost, pci_hole64_fix, true),
DEFINE_PROP_END_OF_LIST(),
};
@@ -214,6 +216,7 @@ static void q35_host_initfn(Object *obj)
/* mch's object_initialize resets the default value, set it again */
qdev_prop_set_uint64(DEVICE(s), PCI_HOST_PROP_PCI_HOLE64_SIZE,
Q35_PCI_HOST_HOLE64_SIZE_DEFAULT);
+
object_property_add(obj, PCI_HOST_PROP_PCI_HOLE_START, "uint32",
q35_host_get_pci_hole_start,
NULL, NULL, NULL);
@@ -476,6 +479,10 @@ static void mch_write_config(PCIDevice *d,
mch_update_pciexbar(mch);
}
+ if (!mch->has_smm_ranges) {
+ return;
+ }
+
if (ranges_overlap(address, len, MCH_HOST_BRIDGE_SMRAM,
MCH_HOST_BRIDGE_SMRAM_SIZE)) {
mch_update_smram(mch);
@@ -494,10 +501,13 @@ static void mch_write_config(PCIDevice *d,
static void mch_update(MCHPCIState *mch)
{
mch_update_pciexbar(mch);
+
mch_update_pam(mch);
- mch_update_smram(mch);
- mch_update_ext_tseg_mbytes(mch);
- mch_update_smbase_smram(mch);
+ if (mch->has_smm_ranges) {
+ mch_update_smram(mch);
+ mch_update_ext_tseg_mbytes(mch);
+ mch_update_smbase_smram(mch);
+ }
/*
* pci hole goes from end-of-low-ram to io-apic.
@@ -538,19 +548,21 @@ static void mch_reset(DeviceState *qdev)
pci_set_quad(d->config + MCH_HOST_BRIDGE_PCIEXBAR,
MCH_HOST_BRIDGE_PCIEXBAR_DEFAULT);
- d->config[MCH_HOST_BRIDGE_SMRAM] = MCH_HOST_BRIDGE_SMRAM_DEFAULT;
- d->config[MCH_HOST_BRIDGE_ESMRAMC] = MCH_HOST_BRIDGE_ESMRAMC_DEFAULT;
- d->wmask[MCH_HOST_BRIDGE_SMRAM] = MCH_HOST_BRIDGE_SMRAM_WMASK;
- d->wmask[MCH_HOST_BRIDGE_ESMRAMC] = MCH_HOST_BRIDGE_ESMRAMC_WMASK;
+ if (mch->has_smm_ranges) {
+ d->config[MCH_HOST_BRIDGE_SMRAM] = MCH_HOST_BRIDGE_SMRAM_DEFAULT;
+ d->config[MCH_HOST_BRIDGE_ESMRAMC] = MCH_HOST_BRIDGE_ESMRAMC_DEFAULT;
+ d->wmask[MCH_HOST_BRIDGE_SMRAM] = MCH_HOST_BRIDGE_SMRAM_WMASK;
+ d->wmask[MCH_HOST_BRIDGE_ESMRAMC] = MCH_HOST_BRIDGE_ESMRAMC_WMASK;
- if (mch->ext_tseg_mbytes > 0) {
- pci_set_word(d->config + MCH_HOST_BRIDGE_EXT_TSEG_MBYTES,
- MCH_HOST_BRIDGE_EXT_TSEG_MBYTES_QUERY);
+ if (mch->ext_tseg_mbytes > 0) {
+ pci_set_word(d->config + MCH_HOST_BRIDGE_EXT_TSEG_MBYTES,
+ MCH_HOST_BRIDGE_EXT_TSEG_MBYTES_QUERY);
+ }
+
+ d->config[MCH_HOST_BRIDGE_F_SMBASE] = 0;
+ d->wmask[MCH_HOST_BRIDGE_F_SMBASE] = 0xff;
}
- d->config[MCH_HOST_BRIDGE_F_SMBASE] = 0;
- d->wmask[MCH_HOST_BRIDGE_F_SMBASE] = 0xff;
-
mch_update(mch);
}
@@ -578,6 +590,10 @@ static void mch_realize(PCIDevice *d, Error **errp)
PAM_EXPAN_BASE + i * PAM_EXPAN_SIZE, PAM_EXPAN_SIZE);
}
+ if (!mch->has_smm_ranges) {
+ return;
+ }
+
/* if *disabled* show SMRAM to all CPUs */
memory_region_init_alias(&mch->smram_region, OBJECT(mch), "smram-region",
mch->pci_address_space, MCH_HOST_BRIDGE_SMRAM_C_BASE,
diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h
index 5065590281a1..91be8e3b35b5 100644
--- a/include/hw/i386/pc.h
+++ b/include/hw/i386/pc.h
@@ -161,6 +161,7 @@ void pc_acpi_smi_interrupt(void *opaque, int irq, int level);
#define PCI_HOST_PROP_PCI_HOLE64_SIZE "pci-hole64-size"
#define PCI_HOST_BELOW_4G_MEM_SIZE "below-4g-mem-size"
#define PCI_HOST_ABOVE_4G_MEM_SIZE "above-4g-mem-size"
+#define PCI_HOST_PROP_SMM_RANGES "smm-ranges"
void pc_pci_as_mapping_init(MemoryRegion *system_memory,
diff --git a/include/hw/pci-host/q35.h b/include/hw/pci-host/q35.h
index bafcbe675214..22fadfa3ed76 100644
--- a/include/hw/pci-host/q35.h
+++ b/include/hw/pci-host/q35.h
@@ -50,6 +50,7 @@ struct MCHPCIState {
MemoryRegion tseg_blackhole, tseg_window;
MemoryRegion smbase_blackhole, smbase_window;
bool has_smram_at_smbase;
+ bool has_smm_ranges;
Range pci_hole;
uint64_t below_4g_mem_size;
uint64_t above_4g_mem_size;
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* Re: [PATCH v5 54/65] q35: Introduce smm_ranges property for q35-pci-host
2024-02-29 6:37 ` [PATCH v5 54/65] q35: Introduce smm_ranges property for q35-pci-host Xiaoyao Li
@ 2024-10-05 8:15 ` Michael Tokarev
0 siblings, 0 replies; 124+ messages in thread
From: Michael Tokarev @ 2024-10-05 8:15 UTC (permalink / raw)
To: Xiaoyao Li, Paolo Bonzini, David Hildenbrand, Igor Mammedov,
Eduardo Habkost, Marcel Apfelbaum, Philippe Mathieu-Daudé,
Yanan Wang, Michael S. Tsirkin, Richard Henderson, Ani Sinha,
Peter Xu, Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang
29.02.2024 09:37, Xiaoyao Li wrote:
> From: Isaku Yamahata <isaku.yamahata@linux.intel.com>
>
> Add a q35 property to check whether or not SMM ranges, e.g. SMRAM, TSEG,
> etc... exist for the target platform. TDX doesn't support SMM and doesn't
> play nice with QEMU modifying related guest memory ranges.
This change introduces a newly reported issue with windows XP:
https://gitlab.com/qemu-project/qemu/-/issues/2608
Can you take a look please? I've no idea if this is a bug in qemu or not.
Thanks,
/mjt
^ permalink raw reply [flat|nested] 124+ messages in thread
* [PATCH v5 55/65] i386/tdx: Disable SMM for TDX VMs
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (53 preceding siblings ...)
2024-02-29 6:37 ` [PATCH v5 54/65] q35: Introduce smm_ranges property for q35-pci-host Xiaoyao Li
@ 2024-02-29 6:37 ` Xiaoyao Li
2024-02-29 6:37 ` [PATCH v5 56/65] i386/tdx: Disable PIC " Xiaoyao Li
` (9 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:37 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
TDX doesn't support SMM and VMM cannot emulate SMM for TDX VMs because
VMM cannot manipulate TDX VM's memory.
Disable SMM for TDX VMs and error out if user requests to enable SMM.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
target/i386/kvm/tdx.c | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 811a3b81af99..c3fadbc5c58e 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -707,11 +707,19 @@ static Notifier tdx_machine_done_notify = {
static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
{
MachineState *ms = MACHINE(qdev_get_machine());
+ X86MachineState *x86ms = X86_MACHINE(ms);
TdxGuest *tdx = TDX_GUEST(cgs);
int r = 0;
ms->require_guest_memfd = true;
+ if (x86ms->smm == ON_OFF_AUTO_AUTO) {
+ x86ms->smm = ON_OFF_AUTO_OFF;
+ } else if (x86ms->smm == ON_OFF_AUTO_ON) {
+ error_setg(errp, "TDX VM doesn't support SMM");
+ return -EINVAL;
+ }
+
if (!tdx_caps) {
r = get_tdx_capabilities(errp);
if (r) {
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 56/65] i386/tdx: Disable PIC for TDX VMs
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (54 preceding siblings ...)
2024-02-29 6:37 ` [PATCH v5 55/65] i386/tdx: Disable SMM for TDX VMs Xiaoyao Li
@ 2024-02-29 6:37 ` Xiaoyao Li
2024-02-29 6:37 ` [PATCH v5 57/65] i386/tdx: Don't allow system reset " Xiaoyao Li
` (8 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:37 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
Legacy PIC (8259) cannot be supported for TDX VMs since TDX module
doesn't allow directly interrupt injection. Using posted interrupts
for the PIC is not a viable option as the guest BIOS/kernel will not
do EOI for PIC IRQs, i.e. will leave the vIRR bit set.
Hence disable PIC for TDX VMs and error out if user wants PIC.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
target/i386/kvm/tdx.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index c3fadbc5c58e..0225a9b79b36 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -720,6 +720,13 @@ static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
return -EINVAL;
}
+ if (x86ms->pic == ON_OFF_AUTO_AUTO) {
+ x86ms->pic = ON_OFF_AUTO_OFF;
+ } else if (x86ms->pic == ON_OFF_AUTO_ON) {
+ error_setg(errp, "TDX VM doesn't support PIC");
+ return -EINVAL;
+ }
+
if (!tdx_caps) {
r = get_tdx_capabilities(errp);
if (r) {
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 57/65] i386/tdx: Don't allow system reset for TDX VMs
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (55 preceding siblings ...)
2024-02-29 6:37 ` [PATCH v5 56/65] i386/tdx: Disable PIC " Xiaoyao Li
@ 2024-02-29 6:37 ` Xiaoyao Li
2024-02-29 6:37 ` [PATCH v5 58/65] i386/tdx: LMCE is not supported for TDX Xiaoyao Li
` (7 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:37 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
TDX CPU state is protected and thus vcpu state cann't be reset by VMM.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
target/i386/kvm/kvm.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index 2748086231d5..b1b0384b0c5c 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -5673,7 +5673,7 @@ bool kvm_has_waitpkg(void)
bool kvm_arch_cpu_check_are_resettable(void)
{
- return !sev_es_enabled();
+ return !sev_es_enabled() && !is_tdx_vm();
}
#define ARCH_REQ_XCOMP_GUEST_PERM 0x1025
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 58/65] i386/tdx: LMCE is not supported for TDX
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (56 preceding siblings ...)
2024-02-29 6:37 ` [PATCH v5 57/65] i386/tdx: Don't allow system reset " Xiaoyao Li
@ 2024-02-29 6:37 ` Xiaoyao Li
2024-02-29 6:37 ` [PATCH v5 59/65] hw/i386: add eoi_intercept_unsupported member to X86MachineState Xiaoyao Li
` (6 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:37 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
LMCE is not supported TDX since KVM doesn't provide emulation for
MSR_IA32_FEAT_CTL.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
target/i386/kvm/kvm-cpu.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/target/i386/kvm/kvm-cpu.c b/target/i386/kvm/kvm-cpu.c
index 9c791b7b0520..8c618869533c 100644
--- a/target/i386/kvm/kvm-cpu.c
+++ b/target/i386/kvm/kvm-cpu.c
@@ -15,6 +15,7 @@
#include "sysemu/sysemu.h"
#include "hw/boards.h"
+#include "tdx.h"
#include "kvm_i386.h"
#include "hw/core/accel-cpu.h"
@@ -60,6 +61,10 @@ static bool lmce_supported(void)
if (kvm_ioctl(kvm_state, KVM_X86_GET_MCE_CAP_SUPPORTED, &mce_cap) < 0) {
return false;
}
+
+ if (is_tdx_vm())
+ return false;
+
return !!(mce_cap & MCG_LMCE_P);
}
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 59/65] hw/i386: add eoi_intercept_unsupported member to X86MachineState
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (57 preceding siblings ...)
2024-02-29 6:37 ` [PATCH v5 58/65] i386/tdx: LMCE is not supported for TDX Xiaoyao Li
@ 2024-02-29 6:37 ` Xiaoyao Li
2024-02-29 6:37 ` [PATCH v5 60/65] hw/i386: add option to forcibly report edge trigger in acpi tables Xiaoyao Li
` (5 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:37 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
Add a new bool member, eoi_intercept_unsupported, to X86MachineState
with default value false. Set true for TDX VM.
Inability to intercept eoi causes impossibility to emulate level
triggered interrupt to be re-injected when level is still kept active.
which affects interrupt controller emulation.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
hw/i386/x86.c | 1 +
include/hw/i386/x86.h | 1 +
target/i386/kvm/tdx.c | 2 ++
3 files changed, 4 insertions(+)
diff --git a/hw/i386/x86.c b/hw/i386/x86.c
index 61c45dfc14dd..6ff2475535bc 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
@@ -1425,6 +1425,7 @@ static void x86_machine_initfn(Object *obj)
x86ms->oem_table_id = g_strndup(ACPI_BUILD_APPNAME8, 8);
x86ms->bus_lock_ratelimit = 0;
x86ms->above_4g_mem_start = 4 * GiB;
+ x86ms->eoi_intercept_unsupported = false;
}
static void x86_machine_class_init(ObjectClass *oc, void *data)
diff --git a/include/hw/i386/x86.h b/include/hw/i386/x86.h
index d28e79cc484a..033f0a34891b 100644
--- a/include/hw/i386/x86.h
+++ b/include/hw/i386/x86.h
@@ -60,6 +60,7 @@ struct X86MachineState {
uint64_t above_4g_mem_start;
/* CPU and apic information: */
+ bool eoi_intercept_unsupported;
unsigned pci_irq_mask;
unsigned apic_id_limit;
uint16_t boot_cpus;
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 0225a9b79b36..b1fb326bd395 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -727,6 +727,8 @@ static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
return -EINVAL;
}
+ x86ms->eoi_intercept_unsupported = true;
+
if (!tdx_caps) {
r = get_tdx_capabilities(errp);
if (r) {
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 60/65] hw/i386: add option to forcibly report edge trigger in acpi tables
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (58 preceding siblings ...)
2024-02-29 6:37 ` [PATCH v5 59/65] hw/i386: add eoi_intercept_unsupported member to X86MachineState Xiaoyao Li
@ 2024-02-29 6:37 ` Xiaoyao Li
2024-02-29 6:37 ` [PATCH v5 61/65] i386/tdx: Don't synchronize guest tsc for TDs Xiaoyao Li
` (4 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:37 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
From: Isaku Yamahata <isaku.yamahata@intel.com>
When level trigger isn't supported on x86 platform,
forcibly report edge trigger in acpi tables.
Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
hw/i386/acpi-build.c | 99 ++++++++++++++++++++++++++++---------------
hw/i386/acpi-common.c | 45 +++++++++++++++-----
2 files changed, 101 insertions(+), 43 deletions(-)
diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 15242b9096b5..8b493a2e9605 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -964,7 +964,8 @@ static void build_dbg_aml(Aml *table)
aml_append(table, scope);
}
-static Aml *build_link_dev(const char *name, uint8_t uid, Aml *reg)
+static Aml *build_link_dev(const char *name, uint8_t uid, Aml *reg,
+ bool level_trigger_unsupported)
{
Aml *dev;
Aml *crs;
@@ -976,7 +977,10 @@ static Aml *build_link_dev(const char *name, uint8_t uid, Aml *reg)
aml_append(dev, aml_name_decl("_UID", aml_int(uid)));
crs = aml_resource_template();
- aml_append(crs, aml_interrupt(AML_CONSUMER, AML_LEVEL, AML_ACTIVE_HIGH,
+ aml_append(crs, aml_interrupt(AML_CONSUMER,
+ level_trigger_unsupported ?
+ AML_EDGE : AML_LEVEL,
+ AML_ACTIVE_HIGH,
AML_SHARED, irqs, ARRAY_SIZE(irqs)));
aml_append(dev, aml_name_decl("_PRS", crs));
@@ -1000,7 +1004,8 @@ static Aml *build_link_dev(const char *name, uint8_t uid, Aml *reg)
return dev;
}
-static Aml *build_gsi_link_dev(const char *name, uint8_t uid, uint8_t gsi)
+static Aml *build_gsi_link_dev(const char *name, uint8_t uid,
+ uint8_t gsi, bool level_trigger_unsupported)
{
Aml *dev;
Aml *crs;
@@ -1013,7 +1018,10 @@ static Aml *build_gsi_link_dev(const char *name, uint8_t uid, uint8_t gsi)
crs = aml_resource_template();
irqs = gsi;
- aml_append(crs, aml_interrupt(AML_CONSUMER, AML_LEVEL, AML_ACTIVE_HIGH,
+ aml_append(crs, aml_interrupt(AML_CONSUMER,
+ level_trigger_unsupported ?
+ AML_EDGE : AML_LEVEL,
+ AML_ACTIVE_HIGH,
AML_SHARED, &irqs, 1));
aml_append(dev, aml_name_decl("_PRS", crs));
@@ -1032,7 +1040,7 @@ static Aml *build_gsi_link_dev(const char *name, uint8_t uid, uint8_t gsi)
}
/* _CRS method - get current settings */
-static Aml *build_iqcr_method(bool is_piix4)
+static Aml *build_iqcr_method(bool is_piix4, bool level_trigger_unsupported)
{
Aml *if_ctx;
uint32_t irqs;
@@ -1040,7 +1048,9 @@ static Aml *build_iqcr_method(bool is_piix4)
Aml *crs = aml_resource_template();
irqs = 0;
- aml_append(crs, aml_interrupt(AML_CONSUMER, AML_LEVEL,
+ aml_append(crs, aml_interrupt(AML_CONSUMER,
+ level_trigger_unsupported ?
+ AML_EDGE : AML_LEVEL,
AML_ACTIVE_HIGH, AML_SHARED, &irqs, 1));
aml_append(method, aml_name_decl("PRR0", crs));
@@ -1074,7 +1084,7 @@ static Aml *build_irq_status_method(void)
return method;
}
-static void build_piix4_pci0_int(Aml *table)
+static void build_piix4_pci0_int(Aml *table, bool level_trigger_unsupported)
{
Aml *dev;
Aml *crs;
@@ -1087,12 +1097,16 @@ static void build_piix4_pci0_int(Aml *table)
aml_append(sb_scope, pci0_scope);
aml_append(sb_scope, build_irq_status_method());
- aml_append(sb_scope, build_iqcr_method(true));
+ aml_append(sb_scope, build_iqcr_method(true, level_trigger_unsupported));
- aml_append(sb_scope, build_link_dev("LNKA", 0, aml_name("PRQ0")));
- aml_append(sb_scope, build_link_dev("LNKB", 1, aml_name("PRQ1")));
- aml_append(sb_scope, build_link_dev("LNKC", 2, aml_name("PRQ2")));
- aml_append(sb_scope, build_link_dev("LNKD", 3, aml_name("PRQ3")));
+ aml_append(sb_scope, build_link_dev("LNKA", 0, aml_name("PRQ0"),
+ level_trigger_unsupported));
+ aml_append(sb_scope, build_link_dev("LNKB", 1, aml_name("PRQ1"),
+ level_trigger_unsupported));
+ aml_append(sb_scope, build_link_dev("LNKC", 2, aml_name("PRQ2"),
+ level_trigger_unsupported));
+ aml_append(sb_scope, build_link_dev("LNKD", 3, aml_name("PRQ3"),
+ level_trigger_unsupported));
dev = aml_device("LNKS");
{
@@ -1101,7 +1115,9 @@ static void build_piix4_pci0_int(Aml *table)
crs = aml_resource_template();
irqs = 9;
- aml_append(crs, aml_interrupt(AML_CONSUMER, AML_LEVEL,
+ aml_append(crs, aml_interrupt(AML_CONSUMER,
+ level_trigger_unsupported ?
+ AML_EDGE : AML_LEVEL,
AML_ACTIVE_HIGH, AML_SHARED,
&irqs, 1));
aml_append(dev, aml_name_decl("_PRS", crs));
@@ -1187,7 +1203,7 @@ static Aml *build_q35_routing_table(const char *str)
return pkg;
}
-static void build_q35_pci0_int(Aml *table)
+static void build_q35_pci0_int(Aml *table, bool level_trigger_unsupported)
{
Aml *method;
Aml *sb_scope = aml_scope("_SB");
@@ -1226,25 +1242,41 @@ static void build_q35_pci0_int(Aml *table)
aml_append(sb_scope, pci0_scope);
aml_append(sb_scope, build_irq_status_method());
- aml_append(sb_scope, build_iqcr_method(false));
+ aml_append(sb_scope, build_iqcr_method(false, level_trigger_unsupported));
- aml_append(sb_scope, build_link_dev("LNKA", 0, aml_name("PRQA")));
- aml_append(sb_scope, build_link_dev("LNKB", 1, aml_name("PRQB")));
- aml_append(sb_scope, build_link_dev("LNKC", 2, aml_name("PRQC")));
- aml_append(sb_scope, build_link_dev("LNKD", 3, aml_name("PRQD")));
- aml_append(sb_scope, build_link_dev("LNKE", 4, aml_name("PRQE")));
- aml_append(sb_scope, build_link_dev("LNKF", 5, aml_name("PRQF")));
- aml_append(sb_scope, build_link_dev("LNKG", 6, aml_name("PRQG")));
- aml_append(sb_scope, build_link_dev("LNKH", 7, aml_name("PRQH")));
+ aml_append(sb_scope, build_link_dev("LNKA", 0, aml_name("PRQA"),
+ level_trigger_unsupported));
+ aml_append(sb_scope, build_link_dev("LNKB", 1, aml_name("PRQB"),
+ level_trigger_unsupported));
+ aml_append(sb_scope, build_link_dev("LNKC", 2, aml_name("PRQC"),
+ level_trigger_unsupported));
+ aml_append(sb_scope, build_link_dev("LNKD", 3, aml_name("PRQD"),
+ level_trigger_unsupported));
+ aml_append(sb_scope, build_link_dev("LNKE", 4, aml_name("PRQE"),
+ level_trigger_unsupported));
+ aml_append(sb_scope, build_link_dev("LNKF", 5, aml_name("PRQF"),
+ level_trigger_unsupported));
+ aml_append(sb_scope, build_link_dev("LNKG", 6, aml_name("PRQG"),
+ level_trigger_unsupported));
+ aml_append(sb_scope, build_link_dev("LNKH", 7, aml_name("PRQH"),
+ level_trigger_unsupported));
- aml_append(sb_scope, build_gsi_link_dev("GSIA", 0x10, 0x10));
- aml_append(sb_scope, build_gsi_link_dev("GSIB", 0x11, 0x11));
- aml_append(sb_scope, build_gsi_link_dev("GSIC", 0x12, 0x12));
- aml_append(sb_scope, build_gsi_link_dev("GSID", 0x13, 0x13));
- aml_append(sb_scope, build_gsi_link_dev("GSIE", 0x14, 0x14));
- aml_append(sb_scope, build_gsi_link_dev("GSIF", 0x15, 0x15));
- aml_append(sb_scope, build_gsi_link_dev("GSIG", 0x16, 0x16));
- aml_append(sb_scope, build_gsi_link_dev("GSIH", 0x17, 0x17));
+ aml_append(sb_scope, build_gsi_link_dev("GSIA", 0x10, 0x10,
+ level_trigger_unsupported));
+ aml_append(sb_scope, build_gsi_link_dev("GSIB", 0x11, 0x11,
+ level_trigger_unsupported));
+ aml_append(sb_scope, build_gsi_link_dev("GSIC", 0x12, 0x12,
+ level_trigger_unsupported));
+ aml_append(sb_scope, build_gsi_link_dev("GSID", 0x13, 0x13,
+ level_trigger_unsupported));
+ aml_append(sb_scope, build_gsi_link_dev("GSIE", 0x14, 0x14,
+ level_trigger_unsupported));
+ aml_append(sb_scope, build_gsi_link_dev("GSIF", 0x15, 0x15,
+ level_trigger_unsupported));
+ aml_append(sb_scope, build_gsi_link_dev("GSIG", 0x16, 0x16,
+ level_trigger_unsupported));
+ aml_append(sb_scope, build_gsi_link_dev("GSIH", 0x17, 0x17,
+ level_trigger_unsupported));
aml_append(table, sb_scope);
}
@@ -1426,6 +1458,7 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
PCMachineState *pcms = PC_MACHINE(machine);
PCMachineClass *pcmc = PC_MACHINE_GET_CLASS(machine);
X86MachineState *x86ms = X86_MACHINE(machine);
+ bool level_trigger_unsupported = x86ms->eoi_intercept_unsupported;
AcpiMcfgInfo mcfg;
bool mcfg_valid = !!acpi_get_mcfg(&mcfg);
uint32_t nr_mem = machine->ram_slots;
@@ -1458,7 +1491,7 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
if (pm->pcihp_bridge_en || pm->pcihp_root_en) {
build_x86_acpi_pci_hotplug(dsdt, pm->pcihp_io_base);
}
- build_piix4_pci0_int(dsdt);
+ build_piix4_pci0_int(dsdt, level_trigger_unsupported);
} else if (q35) {
sb_scope = aml_scope("_SB");
dev = aml_device("PCI0");
@@ -1502,7 +1535,7 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
if (pm->pcihp_bridge_en) {
build_x86_acpi_pci_hotplug(dsdt, pm->pcihp_io_base);
}
- build_q35_pci0_int(dsdt);
+ build_q35_pci0_int(dsdt, level_trigger_unsupported);
}
if (misc->has_hpet) {
diff --git a/hw/i386/acpi-common.c b/hw/i386/acpi-common.c
index 20f19269da40..d93281b13758 100644
--- a/hw/i386/acpi-common.c
+++ b/hw/i386/acpi-common.c
@@ -103,6 +103,7 @@ void acpi_build_madt(GArray *table_data, BIOSLinker *linker,
const CPUArchIdList *apic_ids = mc->possible_cpu_arch_ids(MACHINE(x86ms));
AcpiTable table = { .sig = "APIC", .rev = 3, .oem_id = oem_id,
.oem_table_id = oem_table_id };
+ bool level_trigger_unsupported = x86ms->eoi_intercept_unsupported;
acpi_table_begin(&table, table_data);
/* Local APIC Address */
@@ -122,18 +123,42 @@ void acpi_build_madt(GArray *table_data, BIOSLinker *linker,
IO_APIC_SECONDARY_ADDRESS, IO_APIC_SECONDARY_IRQBASE);
}
- if (x86mc->apic_xrupt_override) {
- build_xrupt_override(table_data, 0, 2,
- 0 /* Flags: Conforms to the specifications of the bus */);
- }
+ if (level_trigger_unsupported) {
+ /* Force edge trigger */
+ if (x86mc->apic_xrupt_override) {
+ build_xrupt_override(table_data, 0, 2,
+ /* Flags: active high, edge triggered */
+ 1 | (1 << 2));
+ }
+
+ for (i = x86mc->apic_xrupt_override ? 1 : 0; i < 16; i++) {
+ build_xrupt_override(table_data, i, i,
+ /* Flags: active high, edge triggered */
+ 1 | (1 << 2));
+ }
+
+ if (x86ms->ioapic2) {
+ for (i = 0; i < 16; i++) {
+ build_xrupt_override(table_data, IO_APIC_SECONDARY_IRQBASE + i,
+ IO_APIC_SECONDARY_IRQBASE + i,
+ /* Flags: active high, edge triggered */
+ 1 | (1 << 2));
+ }
+ }
+ } else {
+ if (x86mc->apic_xrupt_override) {
+ build_xrupt_override(table_data, 0, 2,
+ 0 /* Flags: Conforms to the specifications of the bus */);
+ }
- for (i = 1; i < 16; i++) {
- if (!(x86ms->pci_irq_mask & (1 << i))) {
- /* No need for a INT source override structure. */
- continue;
+ for (i = 1; i < 16; i++) {
+ if (!(x86ms->pci_irq_mask & (1 << i))) {
+ /* No need for a INT source override structure. */
+ continue;
+ }
+ build_xrupt_override(table_data, i, i,
+ 0xd /* Flags: Active high, Level Triggered */);
}
- build_xrupt_override(table_data, i, i,
- 0xd /* Flags: Active high, Level Triggered */);
}
if (x2apic_mode) {
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 61/65] i386/tdx: Don't synchronize guest tsc for TDs
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (59 preceding siblings ...)
2024-02-29 6:37 ` [PATCH v5 60/65] hw/i386: add option to forcibly report edge trigger in acpi tables Xiaoyao Li
@ 2024-02-29 6:37 ` Xiaoyao Li
2024-02-29 6:37 ` [PATCH v5 62/65] i386/tdx: Only configure MSR_IA32_UCODE_REV in kvm_init_msrs() " Xiaoyao Li
` (3 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:37 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
From: Isaku Yamahata <isaku.yamahata@intel.com>
TSC of TDs is not accessible and KVM doesn't allow access of
MSR_IA32_TSC for TDs. To avoid the assert() in kvm_get_tsc, make
kvm_synchronize_all_tsc() noop for TDs,
Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
Reviewed-by: Connor Kuehl <ckuehl@redhat.com>
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
target/i386/kvm/kvm.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index b1b0384b0c5c..83276e23b19f 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -277,7 +277,7 @@ void kvm_synchronize_all_tsc(void)
{
CPUState *cpu;
- if (kvm_enabled()) {
+ if (kvm_enabled() && !is_tdx_vm()) {
CPU_FOREACH(cpu) {
run_on_cpu(cpu, do_kvm_synchronize_tsc, RUN_ON_CPU_NULL);
}
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 62/65] i386/tdx: Only configure MSR_IA32_UCODE_REV in kvm_init_msrs() for TDs
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (60 preceding siblings ...)
2024-02-29 6:37 ` [PATCH v5 61/65] i386/tdx: Don't synchronize guest tsc for TDs Xiaoyao Li
@ 2024-02-29 6:37 ` Xiaoyao Li
2024-02-29 6:37 ` [PATCH v5 63/65] i386/tdx: Skip kvm_put_apicbase() " Xiaoyao Li
` (2 subsequent siblings)
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:37 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
For TDs, only MSR_IA32_UCODE_REV in kvm_init_msrs() can be configured
by VMM, while the features enumerated/controlled by other MSRs except
MSR_IA32_UCODE_REV in kvm_init_msrs() are not under control of VMM.
Only configure MSR_IA32_UCODE_REV for TDs.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
target/i386/kvm/kvm.c | 44 ++++++++++++++++++++++---------------------
1 file changed, 23 insertions(+), 21 deletions(-)
diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index 83276e23b19f..d23f94b77257 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -3270,32 +3270,34 @@ static void kvm_init_msrs(X86CPU *cpu)
CPUX86State *env = &cpu->env;
kvm_msr_buf_reset(cpu);
- if (has_msr_arch_capabs) {
- kvm_msr_entry_add(cpu, MSR_IA32_ARCH_CAPABILITIES,
- env->features[FEAT_ARCH_CAPABILITIES]);
- }
-
- if (has_msr_core_capabs) {
- kvm_msr_entry_add(cpu, MSR_IA32_CORE_CAPABILITY,
- env->features[FEAT_CORE_CAPABILITY]);
- }
-
- if (has_msr_perf_capabs && cpu->enable_pmu) {
- kvm_msr_entry_add_perf(cpu, env->features);
+
+ if (!is_tdx_vm()) {
+ if (has_msr_arch_capabs) {
+ kvm_msr_entry_add(cpu, MSR_IA32_ARCH_CAPABILITIES,
+ env->features[FEAT_ARCH_CAPABILITIES]);
+ }
+
+ if (has_msr_core_capabs) {
+ kvm_msr_entry_add(cpu, MSR_IA32_CORE_CAPABILITY,
+ env->features[FEAT_CORE_CAPABILITY]);
+ }
+
+ if (has_msr_perf_capabs && cpu->enable_pmu) {
+ kvm_msr_entry_add_perf(cpu, env->features);
+ }
+
+ /*
+ * Older kernels do not include VMX MSRs in KVM_GET_MSR_INDEX_LIST, but
+ * all kernels with MSR features should have them.
+ */
+ if (kvm_feature_msrs && cpu_has_vmx(env)) {
+ kvm_msr_entry_add_vmx(cpu, env->features);
+ }
}
if (has_msr_ucode_rev) {
kvm_msr_entry_add(cpu, MSR_IA32_UCODE_REV, cpu->ucode_rev);
}
-
- /*
- * Older kernels do not include VMX MSRs in KVM_GET_MSR_INDEX_LIST, but
- * all kernels with MSR features should have them.
- */
- if (kvm_feature_msrs && cpu_has_vmx(env)) {
- kvm_msr_entry_add_vmx(cpu, env->features);
- }
-
assert(kvm_buf_set_msrs(cpu) == 0);
}
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 63/65] i386/tdx: Skip kvm_put_apicbase() for TDs
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (61 preceding siblings ...)
2024-02-29 6:37 ` [PATCH v5 62/65] i386/tdx: Only configure MSR_IA32_UCODE_REV in kvm_init_msrs() " Xiaoyao Li
@ 2024-02-29 6:37 ` Xiaoyao Li
2024-02-29 6:37 ` [PATCH v5 64/65] i386/tdx: Don't get/put guest state for TDX VMs Xiaoyao Li
2024-02-29 6:37 ` [PATCH v5 65/65] docs: Add TDX documentation Xiaoyao Li
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:37 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
KVM doesn't allow wirting to MSR_IA32_APICBASE for TDs.
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
target/i386/kvm/kvm.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index d23f94b77257..31aed1c9aae0 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -3052,6 +3052,11 @@ void kvm_put_apicbase(X86CPU *cpu, uint64_t value)
{
int ret;
+ /* TODO: Allow accessing guest state for debug TDs. */
+ if (is_tdx_vm()) {
+ return;
+ }
+
ret = kvm_put_one_msr(cpu, MSR_IA32_APICBASE, value);
assert(ret == 1);
}
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 64/65] i386/tdx: Don't get/put guest state for TDX VMs
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (62 preceding siblings ...)
2024-02-29 6:37 ` [PATCH v5 63/65] i386/tdx: Skip kvm_put_apicbase() " Xiaoyao Li
@ 2024-02-29 6:37 ` Xiaoyao Li
2024-02-29 6:37 ` [PATCH v5 65/65] docs: Add TDX documentation Xiaoyao Li
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:37 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
From: Sean Christopherson <sean.j.christopherson@intel.com>
Don't get/put state of TDX VMs since accessing/mutating guest state of
production TDs is not supported.
Note, it will be allowed for a debug TD. Corresponding support will be
introduced when debug TD support is implemented in the future.
Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
target/i386/kvm/kvm.c | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index 31aed1c9aae0..39113718ea14 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -4620,6 +4620,11 @@ int kvm_arch_put_registers(CPUState *cpu, int level)
assert(cpu_is_stopped(cpu) || qemu_cpu_is_self(cpu));
+ /* TODO: Allow accessing guest state for debug TDs. */
+ if (is_tdx_vm()) {
+ return 0;
+ }
+
/*
* Put MSR_IA32_FEATURE_CONTROL first, this ensures the VM gets out of VMX
* root operation upon vCPU reset. kvm_put_msr_feature_control() should also
@@ -4720,6 +4725,12 @@ int kvm_arch_get_registers(CPUState *cs)
if (ret < 0) {
goto out;
}
+
+ /* TODO: Allow accessing guest state for debug TDs. */
+ if (is_tdx_vm()) {
+ return 0;
+ }
+
ret = kvm_getput_regs(cpu, 0);
if (ret < 0) {
goto out;
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread* [PATCH v5 65/65] docs: Add TDX documentation
2024-02-29 6:36 [PATCH v5 00/65] QEMU Guest memfd + QEMU TDX support Xiaoyao Li
` (63 preceding siblings ...)
2024-02-29 6:37 ` [PATCH v5 64/65] i386/tdx: Don't get/put guest state for TDX VMs Xiaoyao Li
@ 2024-02-29 6:37 ` Xiaoyao Li
64 siblings, 0 replies; 124+ messages in thread
From: Xiaoyao Li @ 2024-02-29 6:37 UTC (permalink / raw)
To: Paolo Bonzini, David Hildenbrand, Igor Mammedov, Eduardo Habkost,
Marcel Apfelbaum, Philippe Mathieu-Daudé, Yanan Wang,
Michael S. Tsirkin, Richard Henderson, Ani Sinha, Peter Xu,
Cornelia Huck, Daniel P. Berrangé, Eric Blake,
Markus Armbruster, Marcelo Tosatti
Cc: kvm, qemu-devel, Michael Roth, Claudio Fontana, Gerd Hoffmann,
Isaku Yamahata, Chenyi Qiang, xiaoyao.li
Add docs/system/i386/tdx.rst for TDX support, and add tdx in
confidential-guest-support.rst
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
Changes in v5:
- Add TD attestation section and update the QEMU parameter;
Changes since v1:
- Add prerequisite of private gmem;
- update example command to launch TD;
Changes since RFC v4:
- add the restriction that kernel-irqchip must be split
---
docs/system/confidential-guest-support.rst | 1 +
docs/system/i386/tdx.rst | 143 +++++++++++++++++++++
docs/system/target-i386.rst | 1 +
3 files changed, 145 insertions(+)
create mode 100644 docs/system/i386/tdx.rst
diff --git a/docs/system/confidential-guest-support.rst b/docs/system/confidential-guest-support.rst
index 0c490dbda2b7..66129fbab64c 100644
--- a/docs/system/confidential-guest-support.rst
+++ b/docs/system/confidential-guest-support.rst
@@ -38,6 +38,7 @@ Supported mechanisms
Currently supported confidential guest mechanisms are:
* AMD Secure Encrypted Virtualization (SEV) (see :doc:`i386/amd-memory-encryption`)
+* Intel Trust Domain Extension (TDX) (see :doc:`i386/tdx`)
* POWER Protected Execution Facility (PEF) (see :ref:`power-papr-protected-execution-facility-pef`)
* s390x Protected Virtualization (PV) (see :doc:`s390x/protvirt`)
diff --git a/docs/system/i386/tdx.rst b/docs/system/i386/tdx.rst
new file mode 100644
index 000000000000..8491cdcfa163
--- /dev/null
+++ b/docs/system/i386/tdx.rst
@@ -0,0 +1,143 @@
+Intel Trusted Domain eXtension (TDX)
+====================================
+
+Intel Trusted Domain eXtensions (TDX) refers to an Intel technology that extends
+Virtual Machine Extensions (VMX) and Multi-Key Total Memory Encryption (MKTME)
+with a new kind of virtual machine guest called a Trust Domain (TD). A TD runs
+in a CPU mode that is designed to protect the confidentiality of its memory
+contents and its CPU state from any other software, including the hosting
+Virtual Machine Monitor (VMM), unless explicitly shared by the TD itself.
+
+Prerequisites
+-------------
+
+To run TD, the physical machine needs to have TDX module loaded and initialized
+while KVM hypervisor has TDX support and has TDX enabled. If those requirements
+are met, the ``KVM_CAP_VM_TYPES`` will report the support of ``KVM_X86_TDX_VM``.
+
+Trust Domain Virtual Firmware (TDVF)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Trust Domain Virtual Firmware (TDVF) is required to provide TD services to boot
+TD Guest OS. TDVF needs to be copied to guest private memory and measured before
+the TD boots.
+
+KVM vcpu ioctl ``KVM_MEMORY_MAPPING`` can be used to populates the TDVF content
+into its private memory.
+
+Since TDX doesn't support readonly memslot, TDVF cannot be mapped as pflash
+device and it actually works as RAM. "-bios" option is chosen to load TDVF.
+
+OVMF is the opensource firmware that implements the TDVF support. Thus the
+command line to specify and load TDVF is ``-bios OVMF.fd``
+
+KVM private memory
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+TD's memory (RAM) needs to be able to be transformed between private and shared.
+Its BIOS (OVMF/TDVF) needs to be mapped as private as well. Thus QEMU needs to
+allocate private guest memfd for them via KVM's IOCTL (KVM_CREATE_GUEST_MEMFD),
+which requires KVM is newer enough that reports KVM_CAP_GUEST_MEMFD.
+
+Feature Control
+---------------
+
+Unlike non-TDX VM, the CPU features (enumerated by CPU or MSR) of a TD is not
+under full control of VMM. VMM can only configure part of features of a TD on
+``KVM_TDX_INIT_VM`` command of VM scope ``MEMORY_ENCRYPT_OP`` ioctl.
+
+The configurable features have three types:
+
+- Attributes:
+ - PKS (bit 30) controls whether Supervisor Protection Keys is exposed to TD,
+ which determines related CPUID bit and CR4 bit;
+ - PERFMON (bit 63) controls whether PMU is exposed to TD.
+
+- XSAVE related features (XFAM):
+ XFAM is a 64b mask, which has the same format as XCR0 or IA32_XSS MSR. It
+ determines the set of extended features available for use by the guest TD.
+
+- CPUID features:
+ Only some bits of some CPUID leaves are directly configurable by VMM.
+
+What features can be configured is reported via TDX capabilities.
+
+TDX capabilities
+~~~~~~~~~~~~~~~~
+
+The VM scope ``MEMORY_ENCRYPT_OP`` ioctl provides command ``KVM_TDX_CAPABILITIES``
+to get the TDX capabilities from KVM. It returns a data structure of
+``struct kvm_tdx_capabilites``, which tells the supported configuration of
+attributes, XFAM and CPUIDs.
+
+TD attestation
+--------------
+
+In TD guest, the attestation process is used to verify the TDX guest
+trustworthiness to other entities before provisioning secrets to the guest.
+
+TD attestation is initiated first by calling TDG.MR.REPORT inside TD to get the
+REPORT. Then the REPORT data needs to be converted into a remotely verifiable
+Quote by SGX Quoting Enclave (QE).
+
+A host daemon, Quote Generation Service (QGS), provides the functionality of
+SGX GE. It provides a socket address, to which a TD guest can connect via
+"quote-generation-socket" property. On the request of <GETQUOTE> from TD guest,
+QEMU sends the TDREPORT to QGS via "quote-generation-socket" socket, and gets
+the returning Quoting and return it back to TD guest.
+
+Though "quote-generation-socket" is optional for booting the TD guest, it's a
+must for supporting TD guest atteatation.
+
+Launching a TD (TDX VM)
+-----------------------
+
+To launch a TDX guest, below are new added and required:
+
+.. parsed-literal::
+
+ |qemu_system_x86| \\
+ -object tdx-guest,id=tdx0 \\
+ -machine ...,kernel-irqchip=split,confidential-guest-support=tdx0 \\
+ -bios OVMF.fd \\
+
+If TD attestation support is wanted:
+
+.. parsed-literal::
+
+ |qemu_system_x86| \\
+ -object '{"qom-type":"tdx-guest","id":"tdx0","quote-generation-socket":{"type": "vsock", "cid":"1","port":"1234"}}' \\
+ -machine ...,kernel-irqchip=split,confidential-guest-support=tdx0 \\
+ -bios OVMF.fd \\
+
+Debugging
+---------
+
+Bit 0 of TD attributes, is DEBUG bit, which decides if the TD runs in off-TD
+debug mode. When in off-TD debug mode, TD's VCPU state and private memory are
+accessible via given SEAMCALLs. This requires KVM to expose APIs to invoke those
+SEAMCALLs and resonponding QEMU change.
+
+It's targeted as future work.
+
+restrictions
+------------
+
+ - kernel-irqchip must be split;
+
+ - No readonly support for private memory;
+
+ - No SMM support: SMM support requires manipulating the guset register states
+ which is not allowed;
+
+Live Migration
+--------------
+
+TODO
+
+References
+----------
+
+- `TDX Homepage <https://www.intel.com/content/www/us/en/developer/articles/technical/intel-trust-domain-extensions.html>`__
+
+- `SGX QE <https://github.com/intel/SGXDataCenterAttestationPrimitives/tree/master/QuoteGeneration>`__
diff --git a/docs/system/target-i386.rst b/docs/system/target-i386.rst
index 1b8a1f248abb..4d58cdbc4e06 100644
--- a/docs/system/target-i386.rst
+++ b/docs/system/target-i386.rst
@@ -29,6 +29,7 @@ Architectural features
i386/kvm-pv
i386/sgx
i386/amd-memory-encryption
+ i386/tdx
OS requirements
~~~~~~~~~~~~~~~
--
2.34.1
^ permalink raw reply related [flat|nested] 124+ messages in thread