[PATCH for 10.1 v8 00/55] QEMU TDX support

[PATCH for 10.1 v8 00/55] QEMU TDX support

[Xiaoyao Li]

This is the v8 series of TDX QEMU enabling and the series is also available
at github:
https://github.com/intel-staging/qemu-tdx/tree/tdx-qemu-upstream-v8

To boot TD guest, please always use the latest TDX module (1.5) and OVMF
available.

Note, this series has a dependency on
https://lore.kernel.org/qemu-devel/20241217123932.948789-1-xiaoyao.li@intel.com/

=============
Changes in v8

- Address the comments from Francesco Lavra;
- Add a patch to check KVM_CAP_MAX_VCPUS at vm level;
- Make the default kernel_irqchip to split mode for TDX and refuse to
  boot when other mode requested; (Daniel)
- Drop the manually adjustment of supported CPUID for TDX, which turns
  to be not forward compatible since any newly enabled feature for common KVM
  will automatically be treated as supported for TDX as well. Instead,
  implement TDX's supported CPUID by combining the information from
  directly configurable bits, fixed1 bits, XFAM controlled bits and
  attributes controlled bits. 

===========
future work

- CPU model

  It now only supports booting TD VM with "-cpu host". It is the only
  case that not supposed to hit any warning/error.

  When using named CPU model, even the same model as host, it likely
  hits warning like some feature not supported or some feature enforced
  on. It's a future work to decide if needs to introduce TDX specific
  named CPU models.

- Attestation support

  Atttestation support will be submitted separately after KVM side
  patches being submitted.

- gdb support

  gdb support to debug a TD in off-debug mode is left as future work.

=======
history

v7: https://lore.kernel.org/qemu-devel/20250124132048.3229049-1-xiaoyao.li@intel.com/

v6: https://lore.kernel.org/qemu-devel/20241105062408.3533704-1-xiaoyao.li@intel.com/

Chao Peng (1):
  i386/tdx: load TDVF for TD guest

Isaku Yamahata (5):
  i386/tdx: Make sept_ve_disable set by default
  i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig
  i386/tdvf: Introduce function to parse TDVF metadata
  i386/tdx: Add TDVF memory via KVM_TDX_INIT_MEM_REGION
  i386/tdx: Don't synchronize guest tsc for TDs

Xiaoyao Li (49):
  *** HACK *** linux-headers: Update headers to pull in TDX API changes
  i386: Introduce tdx-guest object
  i386/tdx: Implement tdx_kvm_type() for TDX
  i386/tdx: Implement tdx_kvm_init() to initialize TDX VM context
  i386/tdx: Get tdx_capabilities via KVM_TDX_CAPABILITIES
  i386/tdx: Introduce is_tdx_vm() helper and cache tdx_guest object
  kvm: Introduce kvm_arch_pre_create_vcpu()
  i386/tdx: Initialize TDX before creating TD vcpus
  i386/tdx: Add property sept-ve-disable for tdx-guest object
  i386/tdx: Wire CPU features up with attributes of TD guest
  i386/tdx: Validate TD attributes
  i386/tdx: Set APIC bus rate to match with what TDX module enforces
  i386/tdx: Implement user specified tsc frequency
  i386/tdx: Parse TDVF metadata for TDX VM
  i386/tdx: Don't initialize pc.rom for TDX VMs
  i386/tdx: Track mem_ptr for each firmware entry of TDVF
  i386/tdx: Track RAM entries for TDX VM
  headers: Add definitions from UEFI spec for volumes, resources, etc...
  i386/tdx: Setup the TD HOB list
  i386/tdx: Call KVM_TDX_INIT_VCPU to initialize TDX vcpu
  i386/tdx: Finalize TDX VM
  i386/tdx: Enable user exit on KVM_HC_MAP_GPA_RANGE
  i386/tdx: Handle KVM_SYSTEM_EVENT_TDX_FATAL
  i386/tdx: Wire TDX_REPORT_FATAL_ERROR with GuestPanic facility
  kvm: Check KVM_CAP_MAX_VCPUS at vm level
  i386/cpu: introduce x86_confidential_guest_cpu_instance_init()
  i386/tdx: implement tdx_cpu_instance_init()
  i386/cpu: Introduce enable_cpuid_0x1f to force exposing CPUID 0x1f
  i386/tdx: Force exposing CPUID 0x1f
  i386/tdx: Set kvm_readonly_mem_enabled to false for TDX VM
  i386/tdx: Disable SMM for TDX VMs
  i386/tdx: Disable PIC for TDX VMs
  i386/tdx: Set and check kernel_irqchip mode for TDX
  i386/tdx: Only configure MSR_IA32_UCODE_REV in kvm_init_msrs() for TDs
  i386/apic: Skip kvm_apic_put() for TDX
  cpu: Don't set vcpu_dirty when guest_state_protected
  i386/cgs: Rename *mask_cpuid_features() to *adjust_cpuid_features()
  i386/tdx: Implement adjust_cpuid_features() for TDX
  i386/tdx: Add TDX fixed1 bits to supported CPUIDs
  i386/tdx: Add supported CPUID bits related to TD Attributes
  i386/tdx: Add supported CPUID bits relates to XFAM
  i386/tdx: Add XFD to supported bit of TDX
  i386/tdx: Define supported KVM features for TDX
  i386/cgs: Introduce x86_confidential_guest_check_features()
  i386/tdx: Fetch and validate CPUID of TD guest
  i386/tdx: Don't treat SYSCALL as unavailable
  i386/tdx: Make invtsc default on
  i386/tdx: Validate phys_bits against host value
  docs: Add TDX documentation

 accel/kvm/kvm-all.c                        |   11 +-
 configs/devices/i386-softmmu/default.mak   |    1 +
 docs/system/confidential-guest-support.rst |    1 +
 docs/system/i386/tdx.rst                   |  156 +++
 docs/system/target-i386.rst                |    1 +
 hw/i386/Kconfig                            |    6 +
 hw/i386/kvm/apic.c                         |    5 +
 hw/i386/meson.build                        |    1 +
 hw/i386/pc.c                               |   29 +-
 hw/i386/pc_sysfw.c                         |    7 +
 hw/i386/tdvf-hob.c                         |  130 ++
 hw/i386/tdvf-hob.h                         |   26 +
 hw/i386/tdvf.c                             |  184 +++
 hw/i386/x86-common.c                       |    6 +-
 include/hw/i386/tdvf.h                     |   45 +
 include/standard-headers/uefi/uefi.h       |  187 +++
 include/system/kvm.h                       |    1 +
 linux-headers/asm-x86/kvm.h                |   69 ++
 linux-headers/linux/kvm.h                  |    1 +
 qapi/qom.json                              |   35 +
 qapi/run-state.json                        |   31 +-
 system/runstate.c                          |   65 +
 target/arm/kvm.c                           |    5 +
 target/i386/confidential-guest.h           |   44 +-
 target/i386/cpu.c                          |   73 +-
 target/i386/cpu.h                          |   40 +
 target/i386/host-cpu.c                     |    2 +-
 target/i386/host-cpu.h                     |    1 +
 target/i386/kvm/kvm.c                      |  114 +-
 target/i386/kvm/kvm_i386.h                 |   15 +
 target/i386/kvm/meson.build                |    2 +
 target/i386/kvm/tdx-stub.c                 |   20 +
 target/i386/kvm/tdx.c                      | 1273 ++++++++++++++++++++
 target/i386/kvm/tdx.h                      |   65 +
 target/i386/sev.c                          |    9 +-
 target/loongarch/kvm/kvm.c                 |    4 +
 target/mips/kvm.c                          |    5 +
 target/ppc/kvm.c                           |    5 +
 target/riscv/kvm/kvm-cpu.c                 |    5 +
 target/s390x/kvm/kvm.c                     |    5 +
 40 files changed, 2592 insertions(+), 93 deletions(-)
 create mode 100644 docs/system/i386/tdx.rst
 create mode 100644 hw/i386/tdvf-hob.c
 create mode 100644 hw/i386/tdvf-hob.h
 create mode 100644 hw/i386/tdvf.c
 create mode 100644 include/hw/i386/tdvf.h
 create mode 100644 include/standard-headers/uefi/uefi.h
 create mode 100644 target/i386/kvm/tdx-stub.c
 create mode 100644 target/i386/kvm/tdx.c
 create mode 100644 target/i386/kvm/tdx.h

-- 
2.34.1

[PATCH v8 01/55] *** HACK *** linux-headers: Update headers to pull in TDX API changes

[Xiaoyao Li]

Pull in recent TDX updates, which are not backwards compatible.

It's just to make this series runnable. It will be updated by script

	scripts/update-linux-headers.sh

once TDX support is upstreamed in linux kernel

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
 linux-headers/asm-x86/kvm.h | 69 +++++++++++++++++++++++++++++++++++++
 linux-headers/linux/kvm.h   |  1 +
 2 files changed, 70 insertions(+)

diff --git a/linux-headers/asm-x86/kvm.h b/linux-headers/asm-x86/kvm.h
index 86f2c34e7afa..baca2d479365 100644
--- a/linux-headers/asm-x86/kvm.h
+++ b/linux-headers/asm-x86/kvm.h
@@ -925,4 +925,73 @@ struct kvm_hyperv_eventfd {
 #define KVM_X86_SNP_VM		4
 #define KVM_X86_TDX_VM		5
 
+/* Trust Domain eXtension sub-ioctl() commands. */
+enum kvm_tdx_cmd_id {
+	KVM_TDX_CAPABILITIES = 0,
+	KVM_TDX_INIT_VM,
+	KVM_TDX_INIT_VCPU,
+	KVM_TDX_INIT_MEM_REGION,
+	KVM_TDX_FINALIZE_VM,
+	KVM_TDX_GET_CPUID,
+
+	KVM_TDX_CMD_NR_MAX,
+};
+
+struct kvm_tdx_cmd {
+	/* enum kvm_tdx_cmd_id */
+	__u32 id;
+	/* flags for sub-commend. If sub-command doesn't use this, set zero. */
+	__u32 flags;
+	/*
+	 * data for each sub-command. An immediate or a pointer to the actual
+	 * data in process virtual address.  If sub-command doesn't use it,
+	 * set zero.
+	 */
+	__u64 data;
+	/*
+	 * Auxiliary error code.  The sub-command may return TDX SEAMCALL
+	 * status code in addition to -Exxx.
+	 * Defined for consistency with struct kvm_sev_cmd.
+	 */
+	__u64 hw_error;
+};
+
+struct kvm_tdx_capabilities {
+	__u64 supported_attrs;
+	__u64 supported_xfam;
+	__u64 reserved[254];
+	struct kvm_cpuid2 cpuid;
+};
+
+struct kvm_tdx_init_vm {
+	__u64 attributes;
+	__u64 xfam;
+	__u64 mrconfigid[6];	/* sha384 digest */
+	__u64 mrowner[6];	/* sha384 digest */
+	__u64 mrownerconfig[6];	/* sha384 digest */
+
+	/* The total space for TD_PARAMS before the CPUIDs is 256 bytes */
+	__u64 reserved[12];
+
+	/*
+	 * Call KVM_TDX_INIT_VM before vcpu creation, thus before
+	 * KVM_SET_CPUID2.
+	 * This configuration supersedes KVM_SET_CPUID2s for VCPUs because the
+	 * TDX module directly virtualizes those CPUIDs without VMM.  The user
+	 * space VMM, e.g. qemu, should make KVM_SET_CPUID2 consistent with
+	 * those values.  If it doesn't, KVM may have wrong idea of vCPUIDs of
+	 * the guest, and KVM may wrongly emulate CPUIDs or MSRs that the TDX
+	 * module doesn't virtualize.
+	 */
+	struct kvm_cpuid2 cpuid;
+};
+
+#define KVM_TDX_MEASURE_MEMORY_REGION   _BITULL(0)
+
+struct kvm_tdx_init_mem_region {
+	__u64 source_addr;
+	__u64 gpa;
+	__u64 nr_pages;
+};
+
 #endif /* _ASM_X86_KVM_H */
diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h
index 27181b3dd8fb..bf728042db0c 100644
--- a/linux-headers/linux/kvm.h
+++ b/linux-headers/linux/kvm.h
@@ -369,6 +369,7 @@ struct kvm_run {
 #define KVM_SYSTEM_EVENT_WAKEUP         4
 #define KVM_SYSTEM_EVENT_SUSPEND        5
 #define KVM_SYSTEM_EVENT_SEV_TERM       6
+#define KVM_SYSTEM_EVENT_TDX_FATAL      7
 			__u32 type;
 			__u32 ndata;
 			union {
-- 
2.34.1

Re: [PATCH v8 01/55] *** HACK *** linux-headers: Update headers to pull in TDX API changes

[Zhao Liu]

> 
> diff --git a/linux-headers/asm-x86/kvm.h b/linux-headers/asm-x86/kvm.h
> index 86f2c34e7afa..baca2d479365 100644
> --- a/linux-headers/asm-x86/kvm.h
> +++ b/linux-headers/asm-x86/kvm.h
> @@ -925,4 +925,73 @@ struct kvm_hyperv_eventfd {
>  #define KVM_X86_SNP_VM		4
>  #define KVM_X86_TDX_VM		5
>  
> +/* Trust Domain eXtension sub-ioctl() commands. */

Typo? s/eXtension/Extension/

> +enum kvm_tdx_cmd_id {
> +	KVM_TDX_CAPABILITIES = 0,
> +	KVM_TDX_INIT_VM,
> +	KVM_TDX_INIT_VCPU,
> +	KVM_TDX_INIT_MEM_REGION,
> +	KVM_TDX_FINALIZE_VM,
> +	KVM_TDX_GET_CPUID,
> +
> +	KVM_TDX_CMD_NR_MAX,
> +};
> +
> +struct kvm_tdx_cmd {
> +	/* enum kvm_tdx_cmd_id */
> +	__u32 id;
> +	/* flags for sub-commend. If sub-command doesn't use this, set zero. */

Typo?  s/sub-commend/sub-command/, this line have these 2 cases.

Re: [PATCH v8 01/55] *** HACK *** linux-headers: Update headers to pull in TDX API changes

[Xiaoyao Li]

On 4/18/2025 5:47 PM, Zhao Liu wrote:
>>
>> diff --git a/linux-headers/asm-x86/kvm.h b/linux-headers/asm-x86/kvm.h
>> index 86f2c34e7afa..baca2d479365 100644
>> --- a/linux-headers/asm-x86/kvm.h
>> +++ b/linux-headers/asm-x86/kvm.h
>> @@ -925,4 +925,73 @@ struct kvm_hyperv_eventfd {
>>   #define KVM_X86_SNP_VM		4
>>   #define KVM_X86_TDX_VM		5
>>   
>> +/* Trust Domain eXtension sub-ioctl() commands. */
> 
> Typo? s/eXtension/Extension/
> 
>> +enum kvm_tdx_cmd_id {
>> +	KVM_TDX_CAPABILITIES = 0,
>> +	KVM_TDX_INIT_VM,
>> +	KVM_TDX_INIT_VCPU,
>> +	KVM_TDX_INIT_MEM_REGION,
>> +	KVM_TDX_FINALIZE_VM,
>> +	KVM_TDX_GET_CPUID,
>> +
>> +	KVM_TDX_CMD_NR_MAX,
>> +};
>> +
>> +struct kvm_tdx_cmd {
>> +	/* enum kvm_tdx_cmd_id */
>> +	__u32 id;
>> +	/* flags for sub-commend. If sub-command doesn't use this, set zero. */
> 
> Typo?  s/sub-commend/sub-command/, this line have these 2 cases.
>

They (it and above) were synced from Linux headers. It needs to reported 
to KVM and fixed there.

[PATCH v8 02/55] i386: Introduce tdx-guest object

[Xiaoyao Li]

Introduce tdx-guest object which inherits X86_CONFIDENTIAL_GUEST,
and will be used to create TDX VMs (TDs) by

  qemu -machine ...,confidential-guest-support=tdx0	\
       -object tdx-guest,id=tdx0

It has one QAPI member 'attributes' defined, which allows user to set
TD's attributes directly.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
---
Changes in v7:
 - update QAPI version to 10.0;
 - update to use SPDX tags for license info;
 - update copyright to 2025;

Chanegs in v6:
 - Make tdx-guest inherits X86_CONFIDENTIAL_GUEST;
 - set cgs->require_guest_memfd;
 - allow attributes settable via QAPI;
 - update QAPI version to since 9.2;

Changes in v4:
 - update the new qapi `since` filed from 8.2 to 9.0

Changes in v1
 - make @attributes not user-settable
---
 configs/devices/i386-softmmu/default.mak |  1 +
 hw/i386/Kconfig                          |  5 +++
 qapi/qom.json                            | 15 +++++++++
 target/i386/kvm/meson.build              |  2 ++
 target/i386/kvm/tdx.c                    | 43 ++++++++++++++++++++++++
 target/i386/kvm/tdx.h                    | 21 ++++++++++++
 6 files changed, 87 insertions(+)
 create mode 100644 target/i386/kvm/tdx.c
 create mode 100644 target/i386/kvm/tdx.h

diff --git a/configs/devices/i386-softmmu/default.mak b/configs/devices/i386-softmmu/default.mak
index 4faf2f0315e2..bc0479a7e0a3 100644
--- a/configs/devices/i386-softmmu/default.mak
+++ b/configs/devices/i386-softmmu/default.mak
@@ -18,6 +18,7 @@
 #CONFIG_QXL=n
 #CONFIG_SEV=n
 #CONFIG_SGA=n
+#CONFIG_TDX=n
 #CONFIG_TEST_DEVICES=n
 #CONFIG_TPM_CRB=n
 #CONFIG_TPM_TIS_ISA=n
diff --git a/hw/i386/Kconfig b/hw/i386/Kconfig
index d34ce07b215d..cce9521ba934 100644
--- a/hw/i386/Kconfig
+++ b/hw/i386/Kconfig
@@ -10,6 +10,10 @@ config SGX
     bool
     depends on KVM
 
+config TDX
+    bool
+    depends on KVM
+
 config PC
     bool
     imply APPLESMC
@@ -26,6 +30,7 @@ config PC
     imply QXL
     imply SEV
     imply SGX
+    imply TDX
     imply TEST_DEVICES
     imply TPM_CRB
     imply TPM_TIS_ISA
diff --git a/qapi/qom.json b/qapi/qom.json
index 28ce24cd8d08..c0b61df964ef 100644
--- a/qapi/qom.json
+++ b/qapi/qom.json
@@ -1047,6 +1047,19 @@
             '*host-data': 'str',
             '*vcek-disabled': 'bool' } }
 
+##
+# @TdxGuestProperties:
+#
+# Properties for tdx-guest objects.
+#
+# @attributes: The 'attributes' of a TD guest that is passed to
+#     KVM_TDX_INIT_VM
+#
+# Since: 10.1
+##
+{ 'struct': 'TdxGuestProperties',
+  'data': { '*attributes': 'uint64' } }
+
 ##
 # @ThreadContextProperties:
 #
@@ -1132,6 +1145,7 @@
     'sev-snp-guest',
     'thread-context',
     's390-pv-guest',
+    'tdx-guest',
     'throttle-group',
     'tls-creds-anon',
     'tls-creds-psk',
@@ -1204,6 +1218,7 @@
                                       'if': 'CONFIG_SECRET_KEYRING' },
       'sev-guest':                  'SevGuestProperties',
       'sev-snp-guest':              'SevSnpGuestProperties',
+      'tdx-guest':                  'TdxGuestProperties',
       'thread-context':             'ThreadContextProperties',
       'throttle-group':             'ThrottleGroupProperties',
       'tls-creds-anon':             'TlsCredsAnonProperties',
diff --git a/target/i386/kvm/meson.build b/target/i386/kvm/meson.build
index 3996cafaf29f..466bccb9cb17 100644
--- a/target/i386/kvm/meson.build
+++ b/target/i386/kvm/meson.build
@@ -8,6 +8,8 @@ i386_kvm_ss.add(files(
 
 i386_kvm_ss.add(when: 'CONFIG_XEN_EMU', if_true: files('xen-emu.c'))
 
+i386_kvm_ss.add(when: 'CONFIG_TDX', if_true: files('tdx.c'))
+
 i386_system_ss.add(when: 'CONFIG_HYPERV', if_true: files('hyperv.c'), if_false: files('hyperv-stub.c'))
 
 i386_system_ss.add_all(when: 'CONFIG_KVM', if_true: i386_kvm_ss)
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
new file mode 100644
index 000000000000..ec84ae2947bb
--- /dev/null
+++ b/target/i386/kvm/tdx.c
@@ -0,0 +1,43 @@
+/*
+ * QEMU TDX support
+ *
+ * Copyright (c) 2025 Intel Corporation
+ *
+ * Author:
+ *      Xiaoyao Li <xiaoyao.li@intel.com>
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+#include "qom/object_interfaces.h"
+
+#include "tdx.h"
+
+/* tdx guest */
+OBJECT_DEFINE_TYPE_WITH_INTERFACES(TdxGuest,
+                                   tdx_guest,
+                                   TDX_GUEST,
+                                   X86_CONFIDENTIAL_GUEST,
+                                   { TYPE_USER_CREATABLE },
+                                   { NULL })
+
+static void tdx_guest_init(Object *obj)
+{
+    ConfidentialGuestSupport *cgs = CONFIDENTIAL_GUEST_SUPPORT(obj);
+    TdxGuest *tdx = TDX_GUEST(obj);
+
+    cgs->require_guest_memfd = true;
+    tdx->attributes = 0;
+
+    object_property_add_uint64_ptr(obj, "attributes", &tdx->attributes,
+                                   OBJ_PROP_FLAG_READWRITE);
+}
+
+static void tdx_guest_finalize(Object *obj)
+{
+}
+
+static void tdx_guest_class_init(ObjectClass *oc, void *data)
+{
+}
diff --git a/target/i386/kvm/tdx.h b/target/i386/kvm/tdx.h
new file mode 100644
index 000000000000..f3b725336161
--- /dev/null
+++ b/target/i386/kvm/tdx.h
@@ -0,0 +1,21 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#ifndef QEMU_I386_TDX_H
+#define QEMU_I386_TDX_H
+
+#include "confidential-guest.h"
+
+#define TYPE_TDX_GUEST "tdx-guest"
+#define TDX_GUEST(obj)  OBJECT_CHECK(TdxGuest, (obj), TYPE_TDX_GUEST)
+
+typedef struct TdxGuestClass {
+    X86ConfidentialGuestClass parent_class;
+} TdxGuestClass;
+
+typedef struct TdxGuest {
+    X86ConfidentialGuest parent_obj;
+
+    uint64_t attributes;    /* TD attributes */
+} TdxGuest;
+
+#endif /* QEMU_I386_TDX_H */
-- 
2.34.1

Re: [PATCH v8 02/55] i386: Introduce tdx-guest object

[Daniel P. Berrangé]

On Tue, Apr 01, 2025 at 09:01:12AM -0400, Xiaoyao Li wrote:
> Introduce tdx-guest object which inherits X86_CONFIDENTIAL_GUEST,
> and will be used to create TDX VMs (TDs) by
> 
>   qemu -machine ...,confidential-guest-support=tdx0	\
>        -object tdx-guest,id=tdx0
> 
> It has one QAPI member 'attributes' defined, which allows user to set
> TD's attributes directly.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> Acked-by: Markus Armbruster <armbru@redhat.com>
> ---
> Changes in v7:
>  - update QAPI version to 10.0;
>  - update to use SPDX tags for license info;
>  - update copyright to 2025;
> 
> Chanegs in v6:
>  - Make tdx-guest inherits X86_CONFIDENTIAL_GUEST;
>  - set cgs->require_guest_memfd;
>  - allow attributes settable via QAPI;
>  - update QAPI version to since 9.2;
> 
> Changes in v4:
>  - update the new qapi `since` filed from 8.2 to 9.0
> 
> Changes in v1
>  - make @attributes not user-settable
> ---
>  configs/devices/i386-softmmu/default.mak |  1 +
>  hw/i386/Kconfig                          |  5 +++
>  qapi/qom.json                            | 15 +++++++++
>  target/i386/kvm/meson.build              |  2 ++
>  target/i386/kvm/tdx.c                    | 43 ++++++++++++++++++++++++
>  target/i386/kvm/tdx.h                    | 21 ++++++++++++
>  6 files changed, 87 insertions(+)
>  create mode 100644 target/i386/kvm/tdx.c
>  create mode 100644 target/i386/kvm/tdx.h

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v8 02/55] i386: Introduce tdx-guest object

[Zhao Liu]

>  configs/devices/i386-softmmu/default.mak |  1 +
>  hw/i386/Kconfig                          |  5 +++
>  qapi/qom.json                            | 15 +++++++++
>  target/i386/kvm/meson.build              |  2 ++
>  target/i386/kvm/tdx.c                    | 43 ++++++++++++++++++++++++
>  target/i386/kvm/tdx.h                    | 21 ++++++++++++

SEV.* and confidential-guest.* are all placed in target/i386/.
It's best if all of these can be in the same place.

>  6 files changed, 87 insertions(+)
>  create mode 100644 target/i386/kvm/tdx.c
>  create mode 100644 target/i386/kvm/tdx.h

...

> diff --git a/target/i386/kvm/tdx.h b/target/i386/kvm/tdx.h
> new file mode 100644
> index 000000000000..f3b725336161
> --- /dev/null
> +++ b/target/i386/kvm/tdx.h
> @@ -0,0 +1,21 @@
> +/* SPDX-License-Identifier: GPL-2.0-or-later */
> +
> +#ifndef QEMU_I386_TDX_H
> +#define QEMU_I386_TDX_H

I386_TDX_H is enough... the QEMU prefix is rarely seen in the whole
project.

Others look good,

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

Re: [PATCH v8 02/55] i386: Introduce tdx-guest object

[Xiaoyao Li]

On 4/18/2025 5:17 PM, Zhao Liu wrote:
>>   configs/devices/i386-softmmu/default.mak |  1 +
>>   hw/i386/Kconfig                          |  5 +++
>>   qapi/qom.json                            | 15 +++++++++
>>   target/i386/kvm/meson.build              |  2 ++
>>   target/i386/kvm/tdx.c                    | 43 ++++++++++++++++++++++++
>>   target/i386/kvm/tdx.h                    | 21 ++++++++++++
> 
> SEV.* and confidential-guest.* are all placed in target/i386/.
> It's best if all of these can be in the same place.

I think it's more reasonable to put TDX code under KVM directory since 
TDX depends on KVM.

Regarding SEV, I think the same applies. But it was merged as is and I 
don't see it strongly that it has to move.

>>   6 files changed, 87 insertions(+)
>>   create mode 100644 target/i386/kvm/tdx.c
>>   create mode 100644 target/i386/kvm/tdx.h
> 
> ...
> 
>> diff --git a/target/i386/kvm/tdx.h b/target/i386/kvm/tdx.h
>> new file mode 100644
>> index 000000000000..f3b725336161
>> --- /dev/null
>> +++ b/target/i386/kvm/tdx.h
>> @@ -0,0 +1,21 @@
>> +/* SPDX-License-Identifier: GPL-2.0-or-later */
>> +
>> +#ifndef QEMU_I386_TDX_H
>> +#define QEMU_I386_TDX_H
> 
> I386_TDX_H is enough... the QEMU prefix is rarely seen in the whole
> project.

# git grep -r "#ifndef QEMU_" ./ | wc -l
# 329

I'm not sure if 329 means rarely?

I will just keep it unchanged unless someone objects it strongly.

> Others look good,
> 
> Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

Thanks!

Re: [PATCH v8 02/55] i386: Introduce tdx-guest object

[Daniel P. Berrangé]

On Fri, Apr 18, 2025 at 05:17:02PM +0800, Zhao Liu wrote:
> >  configs/devices/i386-softmmu/default.mak |  1 +
> >  hw/i386/Kconfig                          |  5 +++
> >  qapi/qom.json                            | 15 +++++++++
> >  target/i386/kvm/meson.build              |  2 ++
> >  target/i386/kvm/tdx.c                    | 43 ++++++++++++++++++++++++
> >  target/i386/kvm/tdx.h                    | 21 ++++++++++++
> 
> SEV.* and confidential-guest.* are all placed in target/i386/.
> It's best if all of these can be in the same place.
> 
> >  6 files changed, 87 insertions(+)
> >  create mode 100644 target/i386/kvm/tdx.c
> >  create mode 100644 target/i386/kvm/tdx.h
> 
> ...
> 
> > diff --git a/target/i386/kvm/tdx.h b/target/i386/kvm/tdx.h
> > new file mode 100644
> > index 000000000000..f3b725336161
> > --- /dev/null
> > +++ b/target/i386/kvm/tdx.h
> > @@ -0,0 +1,21 @@
> > +/* SPDX-License-Identifier: GPL-2.0-or-later */
> > +
> > +#ifndef QEMU_I386_TDX_H
> > +#define QEMU_I386_TDX_H
> 
> I386_TDX_H is enough... the QEMU prefix is rarely seen in the whole
> project.

IMHO having a QEMU_ prefix here is "best practice", so don't remove it.

That lots of other QEMU code doesn't follow best practice is unfortunate.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v8 02/55] i386: Introduce tdx-guest object

[Zhao Liu]

> > > index 000000000000..f3b725336161
> > > --- /dev/null
> > > +++ b/target/i386/kvm/tdx.h
> > > @@ -0,0 +1,21 @@
> > > +/* SPDX-License-Identifier: GPL-2.0-or-later */
> > > +
> > > +#ifndef QEMU_I386_TDX_H
> > > +#define QEMU_I386_TDX_H
> > 
> > I386_TDX_H is enough... the QEMU prefix is rarely seen in the whole
> > project.
> 
> IMHO having a QEMU_ prefix here is "best practice", so don't remove it.
> 
> That lots of other QEMU code doesn't follow best practice is unfortunate.

Thanks, now I see!

[PATCH v8 03/55] i386/tdx: Implement tdx_kvm_type() for TDX

[Xiaoyao Li]

TDX VM requires VM type to be KVM_X86_TDX_VM. Implement tdx_kvm_type()
as X86ConfidentialGuestClass->kvm_type.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
Changes in v6:
 - new added patch;
---
 target/i386/kvm/kvm.c |  1 +
 target/i386/kvm/tdx.c | 12 ++++++++++++
 2 files changed, 13 insertions(+)

diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index 6c749d4ee812..4f1cfb529c19 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -191,6 +191,7 @@ static const char *vm_type_name[] = {
     [KVM_X86_SEV_VM] = "SEV",
     [KVM_X86_SEV_ES_VM] = "SEV-ES",
     [KVM_X86_SNP_VM] = "SEV-SNP",
+    [KVM_X86_TDX_VM] = "TDX",
 };
 
 bool kvm_is_vm_type_supported(int type)
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index ec84ae2947bb..d785c1f6d173 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -12,8 +12,17 @@
 #include "qemu/osdep.h"
 #include "qom/object_interfaces.h"
 
+#include "kvm_i386.h"
 #include "tdx.h"
 
+static int tdx_kvm_type(X86ConfidentialGuest *cg)
+{
+    /* Do the object check */
+    TDX_GUEST(cg);
+
+    return KVM_X86_TDX_VM;
+}
+
 /* tdx guest */
 OBJECT_DEFINE_TYPE_WITH_INTERFACES(TdxGuest,
                                    tdx_guest,
@@ -40,4 +49,7 @@ static void tdx_guest_finalize(Object *obj)
 
 static void tdx_guest_class_init(ObjectClass *oc, void *data)
 {
+    X86ConfidentialGuestClass *x86_klass = X86_CONFIDENTIAL_GUEST_CLASS(oc);
+
+    x86_klass->kvm_type = tdx_kvm_type;
 }
-- 
2.34.1

Re: [PATCH v8 03/55] i386/tdx: Implement tdx_kvm_type() for TDX

[Daniel P. Berrangé]

On Tue, Apr 01, 2025 at 09:01:13AM -0400, Xiaoyao Li wrote:
> TDX VM requires VM type to be KVM_X86_TDX_VM. Implement tdx_kvm_type()
> as X86ConfidentialGuestClass->kvm_type.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
> Changes in v6:
>  - new added patch;
> ---
>  target/i386/kvm/kvm.c |  1 +
>  target/i386/kvm/tdx.c | 12 ++++++++++++
>  2 files changed, 13 insertions(+)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v8 03/55] i386/tdx: Implement tdx_kvm_type() for TDX

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:13AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:13 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 03/55] i386/tdx: Implement tdx_kvm_type() for TDX
> X-Mailer: git-send-email 2.34.1
> 
> TDX VM requires VM type to be KVM_X86_TDX_VM. Implement tdx_kvm_type()
> as X86ConfidentialGuestClass->kvm_type.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
> Changes in v6:
>  - new added patch;
> ---
>  target/i386/kvm/kvm.c |  1 +
>  target/i386/kvm/tdx.c | 12 ++++++++++++
>  2 files changed, 13 insertions(+)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 04/55] i386/tdx: Implement tdx_kvm_init() to initialize TDX VM context

[Xiaoyao Li]

Implement TDX specific ConfidentialGuestSupportClass::kvm_init()
callback, tdx_kvm_init().

Mark guest state is proctected for TDX VM.  More TDX specific
initialization will be added later.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
Changes in v6:
 - remove Acked-by from Gerd since the patch changed due to use
   ConfidentialGuestSupportClass::kvm_init();
---
 target/i386/kvm/kvm.c | 11 +----------
 target/i386/kvm/tdx.c | 10 ++++++++++
 2 files changed, 11 insertions(+), 10 deletions(-)

diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index 4f1cfb529c19..1af4710556ad 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -3206,16 +3206,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
     Error *local_err = NULL;
 
     /*
-     * Initialize SEV context, if required
-     *
-     * If no memory encryption is requested (ms->cgs == NULL) this is
-     * a no-op.
-     *
-     * It's also a no-op if a non-SEV confidential guest support
-     * mechanism is selected.  SEV is the only mechanism available to
-     * select on x86 at present, so this doesn't arise, but if new
-     * mechanisms are supported in future (e.g. TDX), they'll need
-     * their own initialization either here or elsewhere.
+     * Initialize confidential guest (SEV/TDX) context, if required
      */
     if (ms->cgs) {
         ret = confidential_guest_kvm_init(ms->cgs, &local_err);
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index d785c1f6d173..4ff94860815d 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -12,9 +12,17 @@
 #include "qemu/osdep.h"
 #include "qom/object_interfaces.h"
 
+#include "hw/i386/x86.h"
 #include "kvm_i386.h"
 #include "tdx.h"
 
+static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
+{
+    kvm_mark_guest_state_protected();
+
+    return 0;
+}
+
 static int tdx_kvm_type(X86ConfidentialGuest *cg)
 {
     /* Do the object check */
@@ -49,7 +57,9 @@ static void tdx_guest_finalize(Object *obj)
 
 static void tdx_guest_class_init(ObjectClass *oc, void *data)
 {
+    ConfidentialGuestSupportClass *klass = CONFIDENTIAL_GUEST_SUPPORT_CLASS(oc);
     X86ConfidentialGuestClass *x86_klass = X86_CONFIDENTIAL_GUEST_CLASS(oc);
 
+    klass->kvm_init = tdx_kvm_init;
     x86_klass->kvm_type = tdx_kvm_type;
 }
-- 
2.34.1

Re: [PATCH v8 04/55] i386/tdx: Implement tdx_kvm_init() to initialize TDX VM context

[Daniel P. Berrangé]

On Tue, Apr 01, 2025 at 09:01:14AM -0400, Xiaoyao Li wrote:
> Implement TDX specific ConfidentialGuestSupportClass::kvm_init()
> callback, tdx_kvm_init().
> 
> Mark guest state is proctected for TDX VM.  More TDX specific
> initialization will be added later.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
> Changes in v6:
>  - remove Acked-by from Gerd since the patch changed due to use
>    ConfidentialGuestSupportClass::kvm_init();
> ---
>  target/i386/kvm/kvm.c | 11 +----------
>  target/i386/kvm/tdx.c | 10 ++++++++++
>  2 files changed, 11 insertions(+), 10 deletions(-)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v8 04/55] i386/tdx: Implement tdx_kvm_init() to initialize TDX VM context

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:14AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:14 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 04/55] i386/tdx: Implement tdx_kvm_init() to initialize
>  TDX VM context
> X-Mailer: git-send-email 2.34.1
> 
> Implement TDX specific ConfidentialGuestSupportClass::kvm_init()
> callback, tdx_kvm_init().
> 
> Mark guest state is proctected for TDX VM.  More TDX specific
> initialization will be added later.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
> Changes in v6:
>  - remove Acked-by from Gerd since the patch changed due to use
>    ConfidentialGuestSupportClass::kvm_init();
> ---
>  target/i386/kvm/kvm.c | 11 +----------
>  target/i386/kvm/tdx.c | 10 ++++++++++
>  2 files changed, 11 insertions(+), 10 deletions(-)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 05/55] i386/tdx: Get tdx_capabilities via KVM_TDX_CAPABILITIES

[Xiaoyao Li]

KVM provides TDX capabilities via sub command KVM_TDX_CAPABILITIES of
IOCTL(KVM_MEMORY_ENCRYPT_OP). Get the capabilities when initializing
TDX context. It will be used to validate user's setting later.

Since there is no interface reporting how many cpuid configs contains in
KVM_TDX_CAPABILITIES, QEMU chooses to try starting with a known number
and abort when it exceeds KVM_MAX_CPUID_ENTRIES.

Besides, introduce the interfaces to invoke TDX "ioctls" at VCPU scope
in preparation.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
Changes in v7:
- refine and unifiy the error handling; (Daniel)

Changes in v6:
- Pass CPUState * to tdx_vcpu_ioctl();
- update commit message to remove platform scope thing;
- dump hw_error when it's non-zero to help debug;

Changes in v4:
- use {} to initialize struct kvm_tdx_cmd, to avoid memset();
- remove tdx_platform_ioctl() because no user;

Changes in v3:
- rename __tdx_ioctl() to tdx_ioctl_internal()
- Pass errp in get_tdx_capabilities();

changes in v2:
  - Make the error message more clear;

changes in v1:
  - start from nr_cpuid_configs = 6 for the loop;
  - stop the loop when nr_cpuid_configs exceeds KVM_MAX_CPUID_ENTRIES;
---
 target/i386/kvm/kvm.c      |   2 -
 target/i386/kvm/kvm_i386.h |   2 +
 target/i386/kvm/tdx.c      | 107 ++++++++++++++++++++++++++++++++++++-
 3 files changed, 108 insertions(+), 3 deletions(-)

diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index 1af4710556ad..b4fa35405fe1 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -1779,8 +1779,6 @@ static int hyperv_init_vcpu(X86CPU *cpu)
 
 static Error *invtsc_mig_blocker;
 
-#define KVM_MAX_CPUID_ENTRIES  100
-
 static void kvm_init_xsave(CPUX86State *env)
 {
     if (has_xsave2) {
diff --git a/target/i386/kvm/kvm_i386.h b/target/i386/kvm/kvm_i386.h
index 88565e8dbac1..ed1e61fb8ba9 100644
--- a/target/i386/kvm/kvm_i386.h
+++ b/target/i386/kvm/kvm_i386.h
@@ -13,6 +13,8 @@
 
 #include "system/kvm.h"
 
+#define KVM_MAX_CPUID_ENTRIES  100
+
 /* always false if !CONFIG_KVM */
 #define kvm_pit_in_kernel() \
     (kvm_irqchip_in_kernel() && !kvm_irqchip_is_split())
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 4ff94860815d..c67be5e618e2 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -10,17 +10,122 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/error-report.h"
+#include "qapi/error.h"
 #include "qom/object_interfaces.h"
 
 #include "hw/i386/x86.h"
 #include "kvm_i386.h"
 #include "tdx.h"
 
+static struct kvm_tdx_capabilities *tdx_caps;
+
+enum tdx_ioctl_level {
+    TDX_VM_IOCTL,
+    TDX_VCPU_IOCTL,
+};
+
+static int tdx_ioctl_internal(enum tdx_ioctl_level level, void *state,
+                              int cmd_id, __u32 flags, void *data,
+                              Error **errp)
+{
+    struct kvm_tdx_cmd tdx_cmd = {};
+    int r;
+
+    const char *tdx_ioctl_name[] = {
+        [KVM_TDX_CAPABILITIES] = "KVM_TDX_CAPABILITIES",
+        [KVM_TDX_INIT_VM] = "KVM_TDX_INIT_VM",
+        [KVM_TDX_INIT_VCPU] = "KVM_TDX_INIT_VCPU",
+        [KVM_TDX_INIT_MEM_REGION] = "KVM_TDX_INIT_MEM_REGION",
+        [KVM_TDX_FINALIZE_VM] = "KVM_TDX_FINALIZE_VM",
+        [KVM_TDX_GET_CPUID] = "KVM_TDX_GET_CPUID",
+    };
+
+    tdx_cmd.id = cmd_id;
+    tdx_cmd.flags = flags;
+    tdx_cmd.data = (__u64)(unsigned long)data;
+
+    switch (level) {
+    case TDX_VM_IOCTL:
+        r = kvm_vm_ioctl(kvm_state, KVM_MEMORY_ENCRYPT_OP, &tdx_cmd);
+        break;
+    case TDX_VCPU_IOCTL:
+        r = kvm_vcpu_ioctl(state, KVM_MEMORY_ENCRYPT_OP, &tdx_cmd);
+        break;
+    default:
+        error_setg(errp, "Invalid tdx_ioctl_level %d", level);
+        return -EINVAL;
+    }
+
+    if (r < 0) {
+        error_setg_errno(errp, -r, "TDX ioctl %s failed, hw_errors: 0x%llx",
+                         tdx_ioctl_name[cmd_id], tdx_cmd.hw_error);
+    }
+    return r;
+}
+
+static inline int tdx_vm_ioctl(int cmd_id, __u32 flags, void *data,
+                               Error **errp)
+{
+    return tdx_ioctl_internal(TDX_VM_IOCTL, NULL, cmd_id, flags, data, errp);
+}
+
+static inline int tdx_vcpu_ioctl(CPUState *cpu, int cmd_id, __u32 flags,
+                                 void *data, Error **errp)
+{
+    return  tdx_ioctl_internal(TDX_VCPU_IOCTL, cpu, cmd_id, flags, data, errp);
+}
+
+static int get_tdx_capabilities(Error **errp)
+{
+    struct kvm_tdx_capabilities *caps;
+    /* 1st generation of TDX reports 6 cpuid configs */
+    int nr_cpuid_configs = 6;
+    size_t size;
+    int r;
+
+    do {
+        Error *local_err = NULL;
+        size = sizeof(struct kvm_tdx_capabilities) +
+                      nr_cpuid_configs * sizeof(struct kvm_cpuid_entry2);
+        caps = g_malloc0(size);
+        caps->cpuid.nent = nr_cpuid_configs;
+
+        r = tdx_vm_ioctl(KVM_TDX_CAPABILITIES, 0, caps, &local_err);
+        if (r == -E2BIG) {
+            g_free(caps);
+            nr_cpuid_configs *= 2;
+            if (nr_cpuid_configs > KVM_MAX_CPUID_ENTRIES) {
+                error_report("KVM TDX seems broken that number of CPUID entries"
+                             " in kvm_tdx_capabilities exceeds limit: %d",
+                             KVM_MAX_CPUID_ENTRIES);
+                error_propagate(errp, local_err);
+                return r;
+            }
+            error_free(local_err);
+        } else if (r < 0) {
+            g_free(caps);
+            error_propagate(errp, local_err);
+            return r;
+        }
+    } while (r == -E2BIG);
+
+    tdx_caps = caps;
+
+    return 0;
+}
+
 static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
 {
+    int r = 0;
+
     kvm_mark_guest_state_protected();
 
-    return 0;
+    if (!tdx_caps) {
+        r = get_tdx_capabilities(errp);
+    }
+
+    return r;
 }
 
 static int tdx_kvm_type(X86ConfidentialGuest *cg)
-- 
2.34.1

Re: [PATCH v8 05/55] i386/tdx: Get tdx_capabilities via KVM_TDX_CAPABILITIES

[Daniel P. Berrangé]

On Tue, Apr 01, 2025 at 09:01:15AM -0400, Xiaoyao Li wrote:
> KVM provides TDX capabilities via sub command KVM_TDX_CAPABILITIES of
> IOCTL(KVM_MEMORY_ENCRYPT_OP). Get the capabilities when initializing
> TDX context. It will be used to validate user's setting later.
> 
> Since there is no interface reporting how many cpuid configs contains in
> KVM_TDX_CAPABILITIES, QEMU chooses to try starting with a known number
> and abort when it exceeds KVM_MAX_CPUID_ENTRIES.
> 
> Besides, introduce the interfaces to invoke TDX "ioctls" at VCPU scope
> in preparation.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
> Changes in v7:
> - refine and unifiy the error handling; (Daniel)
> 
> Changes in v6:
> - Pass CPUState * to tdx_vcpu_ioctl();
> - update commit message to remove platform scope thing;
> - dump hw_error when it's non-zero to help debug;
> 
> Changes in v4:
> - use {} to initialize struct kvm_tdx_cmd, to avoid memset();
> - remove tdx_platform_ioctl() because no user;
> 
> Changes in v3:
> - rename __tdx_ioctl() to tdx_ioctl_internal()
> - Pass errp in get_tdx_capabilities();
> 
> changes in v2:
>   - Make the error message more clear;
> 
> changes in v1:
>   - start from nr_cpuid_configs = 6 for the loop;
>   - stop the loop when nr_cpuid_configs exceeds KVM_MAX_CPUID_ENTRIES;
> ---
>  target/i386/kvm/kvm.c      |   2 -
>  target/i386/kvm/kvm_i386.h |   2 +
>  target/i386/kvm/tdx.c      | 107 ++++++++++++++++++++++++++++++++++++-
>  3 files changed, 108 insertions(+), 3 deletions(-)
> 
> diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
> index 1af4710556ad..b4fa35405fe1 100644
> --- a/target/i386/kvm/kvm.c
> +++ b/target/i386/kvm/kvm.c
> @@ -1779,8 +1779,6 @@ static int hyperv_init_vcpu(X86CPU *cpu)
>  
>  static Error *invtsc_mig_blocker;
>  
> -#define KVM_MAX_CPUID_ENTRIES  100
> -
>  static void kvm_init_xsave(CPUX86State *env)
>  {
>      if (has_xsave2) {
> diff --git a/target/i386/kvm/kvm_i386.h b/target/i386/kvm/kvm_i386.h
> index 88565e8dbac1..ed1e61fb8ba9 100644
> --- a/target/i386/kvm/kvm_i386.h
> +++ b/target/i386/kvm/kvm_i386.h
> @@ -13,6 +13,8 @@
>  
>  #include "system/kvm.h"
>  
> +#define KVM_MAX_CPUID_ENTRIES  100
> +
>  /* always false if !CONFIG_KVM */
>  #define kvm_pit_in_kernel() \
>      (kvm_irqchip_in_kernel() && !kvm_irqchip_is_split())
> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
> index 4ff94860815d..c67be5e618e2 100644
> --- a/target/i386/kvm/tdx.c
> +++ b/target/i386/kvm/tdx.c
> @@ -10,17 +10,122 @@
>   */
>  
>  #include "qemu/osdep.h"
> +#include "qemu/error-report.h"
> +#include "qapi/error.h"
>  #include "qom/object_interfaces.h"
>  
>  #include "hw/i386/x86.h"
>  #include "kvm_i386.h"
>  #include "tdx.h"
>  
> +static struct kvm_tdx_capabilities *tdx_caps;
> +
> +enum tdx_ioctl_level {
> +    TDX_VM_IOCTL,
> +    TDX_VCPU_IOCTL,
> +};
> +
> +static int tdx_ioctl_internal(enum tdx_ioctl_level level, void *state,
> +                              int cmd_id, __u32 flags, void *data,
> +                              Error **errp)
> +{
> +    struct kvm_tdx_cmd tdx_cmd = {};
> +    int r;
> +
> +    const char *tdx_ioctl_name[] = {
> +        [KVM_TDX_CAPABILITIES] = "KVM_TDX_CAPABILITIES",
> +        [KVM_TDX_INIT_VM] = "KVM_TDX_INIT_VM",
> +        [KVM_TDX_INIT_VCPU] = "KVM_TDX_INIT_VCPU",
> +        [KVM_TDX_INIT_MEM_REGION] = "KVM_TDX_INIT_MEM_REGION",
> +        [KVM_TDX_FINALIZE_VM] = "KVM_TDX_FINALIZE_VM",
> +        [KVM_TDX_GET_CPUID] = "KVM_TDX_GET_CPUID",
> +    };
> +
> +    tdx_cmd.id = cmd_id;
> +    tdx_cmd.flags = flags;
> +    tdx_cmd.data = (__u64)(unsigned long)data;
> +
> +    switch (level) {
> +    case TDX_VM_IOCTL:
> +        r = kvm_vm_ioctl(kvm_state, KVM_MEMORY_ENCRYPT_OP, &tdx_cmd);
> +        break;
> +    case TDX_VCPU_IOCTL:
> +        r = kvm_vcpu_ioctl(state, KVM_MEMORY_ENCRYPT_OP, &tdx_cmd);
> +        break;
> +    default:
> +        error_setg(errp, "Invalid tdx_ioctl_level %d", level);
> +        return -EINVAL;
> +    }
> +
> +    if (r < 0) {
> +        error_setg_errno(errp, -r, "TDX ioctl %s failed, hw_errors: 0x%llx",
> +                         tdx_ioctl_name[cmd_id], tdx_cmd.hw_error);
> +    }
> +    return r;
> +}
> +
> +static inline int tdx_vm_ioctl(int cmd_id, __u32 flags, void *data,
> +                               Error **errp)
> +{
> +    return tdx_ioctl_internal(TDX_VM_IOCTL, NULL, cmd_id, flags, data, errp);
> +}
> +
> +static inline int tdx_vcpu_ioctl(CPUState *cpu, int cmd_id, __u32 flags,
> +                                 void *data, Error **errp)
> +{
> +    return  tdx_ioctl_internal(TDX_VCPU_IOCTL, cpu, cmd_id, flags, data, errp);
> +}
> +
> +static int get_tdx_capabilities(Error **errp)
> +{
> +    struct kvm_tdx_capabilities *caps;
> +    /* 1st generation of TDX reports 6 cpuid configs */
> +    int nr_cpuid_configs = 6;
> +    size_t size;
> +    int r;
> +
> +    do {
> +        Error *local_err = NULL;
> +        size = sizeof(struct kvm_tdx_capabilities) +
> +                      nr_cpuid_configs * sizeof(struct kvm_cpuid_entry2);
> +        caps = g_malloc0(size);
> +        caps->cpuid.nent = nr_cpuid_configs;
> +
> +        r = tdx_vm_ioctl(KVM_TDX_CAPABILITIES, 0, caps, &local_err);
> +        if (r == -E2BIG) {
> +            g_free(caps);
> +            nr_cpuid_configs *= 2;
> +            if (nr_cpuid_configs > KVM_MAX_CPUID_ENTRIES) {
> +                error_report("KVM TDX seems broken that number of CPUID entries"
> +                             " in kvm_tdx_capabilities exceeds limit: %d",
> +                             KVM_MAX_CPUID_ENTRIES);
> +                error_propagate(errp, local_err);
> +                return r;
> +            }
> +            error_free(local_err);

IIRC, you'll need  'local_err = NULL' here, otherwise next time around
the loop 'local_err' will be pointing to a free'd error object which
'error_setg' will think is still valid & won't overwrite.

> +        } else if (r < 0) {
> +            g_free(caps);
> +            error_propagate(errp, local_err);
> +            return r;
> +        }
> +    } while (r == -E2BIG);
> +
> +    tdx_caps = caps;
> +
> +    return 0;
> +}
> +


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v8 05/55] i386/tdx: Get tdx_capabilities via KVM_TDX_CAPABILITIES

[Xiaoyao Li]

On 4/2/2025 7:00 PM, Daniel P. Berrangé wrote:
> On Tue, Apr 01, 2025 at 09:01:15AM -0400, Xiaoyao Li wrote:
>> KVM provides TDX capabilities via sub command KVM_TDX_CAPABILITIES of
>> IOCTL(KVM_MEMORY_ENCRYPT_OP). Get the capabilities when initializing
>> TDX context. It will be used to validate user's setting later.
>>
>> Since there is no interface reporting how many cpuid configs contains in
>> KVM_TDX_CAPABILITIES, QEMU chooses to try starting with a known number
>> and abort when it exceeds KVM_MAX_CPUID_ENTRIES.
>>
>> Besides, introduce the interfaces to invoke TDX "ioctls" at VCPU scope
>> in preparation.
>>
>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>> ---
>> Changes in v7:
>> - refine and unifiy the error handling; (Daniel)
>>
>> Changes in v6:
>> - Pass CPUState * to tdx_vcpu_ioctl();
>> - update commit message to remove platform scope thing;
>> - dump hw_error when it's non-zero to help debug;
>>
>> Changes in v4:
>> - use {} to initialize struct kvm_tdx_cmd, to avoid memset();
>> - remove tdx_platform_ioctl() because no user;
>>
>> Changes in v3:
>> - rename __tdx_ioctl() to tdx_ioctl_internal()
>> - Pass errp in get_tdx_capabilities();
>>
>> changes in v2:
>>    - Make the error message more clear;
>>
>> changes in v1:
>>    - start from nr_cpuid_configs = 6 for the loop;
>>    - stop the loop when nr_cpuid_configs exceeds KVM_MAX_CPUID_ENTRIES;
>> ---
>>   target/i386/kvm/kvm.c      |   2 -
>>   target/i386/kvm/kvm_i386.h |   2 +
>>   target/i386/kvm/tdx.c      | 107 ++++++++++++++++++++++++++++++++++++-
>>   3 files changed, 108 insertions(+), 3 deletions(-)
>>
>> diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
>> index 1af4710556ad..b4fa35405fe1 100644
>> --- a/target/i386/kvm/kvm.c
>> +++ b/target/i386/kvm/kvm.c
>> @@ -1779,8 +1779,6 @@ static int hyperv_init_vcpu(X86CPU *cpu)
>>   
>>   static Error *invtsc_mig_blocker;
>>   
>> -#define KVM_MAX_CPUID_ENTRIES  100
>> -
>>   static void kvm_init_xsave(CPUX86State *env)
>>   {
>>       if (has_xsave2) {
>> diff --git a/target/i386/kvm/kvm_i386.h b/target/i386/kvm/kvm_i386.h
>> index 88565e8dbac1..ed1e61fb8ba9 100644
>> --- a/target/i386/kvm/kvm_i386.h
>> +++ b/target/i386/kvm/kvm_i386.h
>> @@ -13,6 +13,8 @@
>>   
>>   #include "system/kvm.h"
>>   
>> +#define KVM_MAX_CPUID_ENTRIES  100
>> +
>>   /* always false if !CONFIG_KVM */
>>   #define kvm_pit_in_kernel() \
>>       (kvm_irqchip_in_kernel() && !kvm_irqchip_is_split())
>> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
>> index 4ff94860815d..c67be5e618e2 100644
>> --- a/target/i386/kvm/tdx.c
>> +++ b/target/i386/kvm/tdx.c
>> @@ -10,17 +10,122 @@
>>    */
>>   
>>   #include "qemu/osdep.h"
>> +#include "qemu/error-report.h"
>> +#include "qapi/error.h"
>>   #include "qom/object_interfaces.h"
>>   
>>   #include "hw/i386/x86.h"
>>   #include "kvm_i386.h"
>>   #include "tdx.h"
>>   
>> +static struct kvm_tdx_capabilities *tdx_caps;
>> +
>> +enum tdx_ioctl_level {
>> +    TDX_VM_IOCTL,
>> +    TDX_VCPU_IOCTL,
>> +};
>> +
>> +static int tdx_ioctl_internal(enum tdx_ioctl_level level, void *state,
>> +                              int cmd_id, __u32 flags, void *data,
>> +                              Error **errp)
>> +{
>> +    struct kvm_tdx_cmd tdx_cmd = {};
>> +    int r;
>> +
>> +    const char *tdx_ioctl_name[] = {
>> +        [KVM_TDX_CAPABILITIES] = "KVM_TDX_CAPABILITIES",
>> +        [KVM_TDX_INIT_VM] = "KVM_TDX_INIT_VM",
>> +        [KVM_TDX_INIT_VCPU] = "KVM_TDX_INIT_VCPU",
>> +        [KVM_TDX_INIT_MEM_REGION] = "KVM_TDX_INIT_MEM_REGION",
>> +        [KVM_TDX_FINALIZE_VM] = "KVM_TDX_FINALIZE_VM",
>> +        [KVM_TDX_GET_CPUID] = "KVM_TDX_GET_CPUID",
>> +    };
>> +
>> +    tdx_cmd.id = cmd_id;
>> +    tdx_cmd.flags = flags;
>> +    tdx_cmd.data = (__u64)(unsigned long)data;
>> +
>> +    switch (level) {
>> +    case TDX_VM_IOCTL:
>> +        r = kvm_vm_ioctl(kvm_state, KVM_MEMORY_ENCRYPT_OP, &tdx_cmd);
>> +        break;
>> +    case TDX_VCPU_IOCTL:
>> +        r = kvm_vcpu_ioctl(state, KVM_MEMORY_ENCRYPT_OP, &tdx_cmd);
>> +        break;
>> +    default:
>> +        error_setg(errp, "Invalid tdx_ioctl_level %d", level);
>> +        return -EINVAL;
>> +    }
>> +
>> +    if (r < 0) {
>> +        error_setg_errno(errp, -r, "TDX ioctl %s failed, hw_errors: 0x%llx",
>> +                         tdx_ioctl_name[cmd_id], tdx_cmd.hw_error);
>> +    }
>> +    return r;
>> +}
>> +
>> +static inline int tdx_vm_ioctl(int cmd_id, __u32 flags, void *data,
>> +                               Error **errp)
>> +{
>> +    return tdx_ioctl_internal(TDX_VM_IOCTL, NULL, cmd_id, flags, data, errp);
>> +}
>> +
>> +static inline int tdx_vcpu_ioctl(CPUState *cpu, int cmd_id, __u32 flags,
>> +                                 void *data, Error **errp)
>> +{
>> +    return  tdx_ioctl_internal(TDX_VCPU_IOCTL, cpu, cmd_id, flags, data, errp);
>> +}
>> +
>> +static int get_tdx_capabilities(Error **errp)
>> +{
>> +    struct kvm_tdx_capabilities *caps;
>> +    /* 1st generation of TDX reports 6 cpuid configs */
>> +    int nr_cpuid_configs = 6;
>> +    size_t size;
>> +    int r;
>> +
>> +    do {
>> +        Error *local_err = NULL;
>> +        size = sizeof(struct kvm_tdx_capabilities) +
>> +                      nr_cpuid_configs * sizeof(struct kvm_cpuid_entry2);
>> +        caps = g_malloc0(size);
>> +        caps->cpuid.nent = nr_cpuid_configs;
>> +
>> +        r = tdx_vm_ioctl(KVM_TDX_CAPABILITIES, 0, caps, &local_err);
>> +        if (r == -E2BIG) {
>> +            g_free(caps);
>> +            nr_cpuid_configs *= 2;
>> +            if (nr_cpuid_configs > KVM_MAX_CPUID_ENTRIES) {
>> +                error_report("KVM TDX seems broken that number of CPUID entries"
>> +                             " in kvm_tdx_capabilities exceeds limit: %d",
>> +                             KVM_MAX_CPUID_ENTRIES);
>> +                error_propagate(errp, local_err);
>> +                return r;
>> +            }
>> +            error_free(local_err);
> 
> IIRC, you'll need  'local_err = NULL' here, otherwise next time around
> the loop 'local_err' will be pointing to a free'd error object which
> 'error_setg' will think is still valid & won't overwrite.

it's set to NULL at the beginning of the do loop.

(It looks I need to declare Error *local_err outside the do loop.)

>> +        } else if (r < 0) {
>> +            g_free(caps);
>> +            error_propagate(errp, local_err);
>> +            return r;
>> +        }
>> +    } while (r == -E2BIG);
>> +
>> +    tdx_caps = caps;
>> +
>> +    return 0;
>> +}
>> +
> 
> 
> With regards,
> Daniel

Re: [PATCH v8 05/55] i386/tdx: Get tdx_capabilities via KVM_TDX_CAPABILITIES

[Daniel P. Berrangé]

On Wed, Apr 02, 2025 at 10:52:38PM +0800, Xiaoyao Li wrote:
> On 4/2/2025 7:00 PM, Daniel P. Berrangé wrote:
> > On Tue, Apr 01, 2025 at 09:01:15AM -0400, Xiaoyao Li wrote:
> > > KVM provides TDX capabilities via sub command KVM_TDX_CAPABILITIES of
> > > IOCTL(KVM_MEMORY_ENCRYPT_OP). Get the capabilities when initializing
> > > TDX context. It will be used to validate user's setting later.
> > > 
> > > Since there is no interface reporting how many cpuid configs contains in
> > > KVM_TDX_CAPABILITIES, QEMU chooses to try starting with a known number
> > > and abort when it exceeds KVM_MAX_CPUID_ENTRIES.
> > > 
> > > Besides, introduce the interfaces to invoke TDX "ioctls" at VCPU scope
> > > in preparation.
> > > 
> > > Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> > > ---
> > > Changes in v7:
> > > - refine and unifiy the error handling; (Daniel)
> > > 
> > > Changes in v6:
> > > - Pass CPUState * to tdx_vcpu_ioctl();
> > > - update commit message to remove platform scope thing;
> > > - dump hw_error when it's non-zero to help debug;
> > > 
> > > Changes in v4:
> > > - use {} to initialize struct kvm_tdx_cmd, to avoid memset();
> > > - remove tdx_platform_ioctl() because no user;
> > > 
> > > Changes in v3:
> > > - rename __tdx_ioctl() to tdx_ioctl_internal()
> > > - Pass errp in get_tdx_capabilities();
> > > 
> > > changes in v2:
> > >    - Make the error message more clear;
> > > 
> > > changes in v1:
> > >    - start from nr_cpuid_configs = 6 for the loop;
> > >    - stop the loop when nr_cpuid_configs exceeds KVM_MAX_CPUID_ENTRIES;
> > > ---
> > >   target/i386/kvm/kvm.c      |   2 -
> > >   target/i386/kvm/kvm_i386.h |   2 +
> > >   target/i386/kvm/tdx.c      | 107 ++++++++++++++++++++++++++++++++++++-
> > >   3 files changed, 108 insertions(+), 3 deletions(-)
> > > 
> > > diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
> > > index 1af4710556ad..b4fa35405fe1 100644
> > > --- a/target/i386/kvm/kvm.c
> > > +++ b/target/i386/kvm/kvm.c
> > > @@ -1779,8 +1779,6 @@ static int hyperv_init_vcpu(X86CPU *cpu)
> > >   static Error *invtsc_mig_blocker;
> > > -#define KVM_MAX_CPUID_ENTRIES  100
> > > -
> > >   static void kvm_init_xsave(CPUX86State *env)
> > >   {
> > >       if (has_xsave2) {
> > > diff --git a/target/i386/kvm/kvm_i386.h b/target/i386/kvm/kvm_i386.h
> > > index 88565e8dbac1..ed1e61fb8ba9 100644
> > > --- a/target/i386/kvm/kvm_i386.h
> > > +++ b/target/i386/kvm/kvm_i386.h
> > > @@ -13,6 +13,8 @@
> > >   #include "system/kvm.h"
> > > +#define KVM_MAX_CPUID_ENTRIES  100
> > > +
> > >   /* always false if !CONFIG_KVM */
> > >   #define kvm_pit_in_kernel() \
> > >       (kvm_irqchip_in_kernel() && !kvm_irqchip_is_split())
> > > diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
> > > index 4ff94860815d..c67be5e618e2 100644
> > > --- a/target/i386/kvm/tdx.c
> > > +++ b/target/i386/kvm/tdx.c
> > > @@ -10,17 +10,122 @@
> > >    */
> > >   #include "qemu/osdep.h"
> > > +#include "qemu/error-report.h"
> > > +#include "qapi/error.h"
> > >   #include "qom/object_interfaces.h"
> > >   #include "hw/i386/x86.h"
> > >   #include "kvm_i386.h"
> > >   #include "tdx.h"
> > > +static struct kvm_tdx_capabilities *tdx_caps;
> > > +
> > > +enum tdx_ioctl_level {
> > > +    TDX_VM_IOCTL,
> > > +    TDX_VCPU_IOCTL,
> > > +};
> > > +
> > > +static int tdx_ioctl_internal(enum tdx_ioctl_level level, void *state,
> > > +                              int cmd_id, __u32 flags, void *data,
> > > +                              Error **errp)
> > > +{
> > > +    struct kvm_tdx_cmd tdx_cmd = {};
> > > +    int r;
> > > +
> > > +    const char *tdx_ioctl_name[] = {
> > > +        [KVM_TDX_CAPABILITIES] = "KVM_TDX_CAPABILITIES",
> > > +        [KVM_TDX_INIT_VM] = "KVM_TDX_INIT_VM",
> > > +        [KVM_TDX_INIT_VCPU] = "KVM_TDX_INIT_VCPU",
> > > +        [KVM_TDX_INIT_MEM_REGION] = "KVM_TDX_INIT_MEM_REGION",
> > > +        [KVM_TDX_FINALIZE_VM] = "KVM_TDX_FINALIZE_VM",
> > > +        [KVM_TDX_GET_CPUID] = "KVM_TDX_GET_CPUID",
> > > +    };
> > > +
> > > +    tdx_cmd.id = cmd_id;
> > > +    tdx_cmd.flags = flags;
> > > +    tdx_cmd.data = (__u64)(unsigned long)data;
> > > +
> > > +    switch (level) {
> > > +    case TDX_VM_IOCTL:
> > > +        r = kvm_vm_ioctl(kvm_state, KVM_MEMORY_ENCRYPT_OP, &tdx_cmd);
> > > +        break;
> > > +    case TDX_VCPU_IOCTL:
> > > +        r = kvm_vcpu_ioctl(state, KVM_MEMORY_ENCRYPT_OP, &tdx_cmd);
> > > +        break;
> > > +    default:
> > > +        error_setg(errp, "Invalid tdx_ioctl_level %d", level);
> > > +        return -EINVAL;
> > > +    }
> > > +
> > > +    if (r < 0) {
> > > +        error_setg_errno(errp, -r, "TDX ioctl %s failed, hw_errors: 0x%llx",
> > > +                         tdx_ioctl_name[cmd_id], tdx_cmd.hw_error);
> > > +    }
> > > +    return r;
> > > +}
> > > +
> > > +static inline int tdx_vm_ioctl(int cmd_id, __u32 flags, void *data,
> > > +                               Error **errp)
> > > +{
> > > +    return tdx_ioctl_internal(TDX_VM_IOCTL, NULL, cmd_id, flags, data, errp);
> > > +}
> > > +
> > > +static inline int tdx_vcpu_ioctl(CPUState *cpu, int cmd_id, __u32 flags,
> > > +                                 void *data, Error **errp)
> > > +{
> > > +    return  tdx_ioctl_internal(TDX_VCPU_IOCTL, cpu, cmd_id, flags, data, errp);
> > > +}
> > > +
> > > +static int get_tdx_capabilities(Error **errp)
> > > +{
> > > +    struct kvm_tdx_capabilities *caps;
> > > +    /* 1st generation of TDX reports 6 cpuid configs */
> > > +    int nr_cpuid_configs = 6;
> > > +    size_t size;
> > > +    int r;
> > > +
> > > +    do {
> > > +        Error *local_err = NULL;
> > > +        size = sizeof(struct kvm_tdx_capabilities) +
> > > +                      nr_cpuid_configs * sizeof(struct kvm_cpuid_entry2);
> > > +        caps = g_malloc0(size);
> > > +        caps->cpuid.nent = nr_cpuid_configs;
> > > +
> > > +        r = tdx_vm_ioctl(KVM_TDX_CAPABILITIES, 0, caps, &local_err);
> > > +        if (r == -E2BIG) {
> > > +            g_free(caps);
> > > +            nr_cpuid_configs *= 2;
> > > +            if (nr_cpuid_configs > KVM_MAX_CPUID_ENTRIES) {
> > > +                error_report("KVM TDX seems broken that number of CPUID entries"
> > > +                             " in kvm_tdx_capabilities exceeds limit: %d",
> > > +                             KVM_MAX_CPUID_ENTRIES);
> > > +                error_propagate(errp, local_err);
> > > +                return r;
> > > +            }
> > > +            error_free(local_err);
> > 
> > IIRC, you'll need  'local_err = NULL' here, otherwise next time around
> > the loop 'local_err' will be pointing to a free'd error object which
> > 'error_setg' will think is still valid & won't overwrite.
> 
> it's set to NULL at the beginning of the do loop.

Oh yes, i missed that, your code is fine as is then.

> (It looks I need to declare Error *local_err outside the do loop.)

Nah, it is fine within the loop given nothing outside needs it.

> 
> > > +        } else if (r < 0) {
> > > +            g_free(caps);
> > > +            error_propagate(errp, local_err);
> > > +            return r;
> > > +        }
> > > +    } while (r == -E2BIG);
> > > +
> > > +    tdx_caps = caps;
> > > +
> > > +    return 0;
> > > +}
> > > +
> > 
> > 
> > With regards,
> > Daniel
> 

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

[PATCH v8 06/55] i386/tdx: Introduce is_tdx_vm() helper and cache tdx_guest object

[Xiaoyao Li]

It will need special handling for TDX VMs all around the QEMU.
Introduce is_tdx_vm() helper to query if it's a TDX VM.

Cache tdx_guest object thus no need to cast from ms->cgs every time.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Isaku Yamahata <isaku.yamahata@intel.com>
---
changes in v3:
- replace object_dynamic_cast with TDX_GUEST();
---
 target/i386/kvm/tdx.c | 15 ++++++++++++++-
 target/i386/kvm/tdx.h | 10 ++++++++++
 2 files changed, 24 insertions(+), 1 deletion(-)

diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index c67be5e618e2..16f67e18ae78 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -18,8 +18,16 @@
 #include "kvm_i386.h"
 #include "tdx.h"
 
+static TdxGuest *tdx_guest;
+
 static struct kvm_tdx_capabilities *tdx_caps;
 
+/* Valid after kvm_arch_init()->confidential_guest_kvm_init()->tdx_kvm_init() */
+bool is_tdx_vm(void)
+{
+    return !!tdx_guest;
+}
+
 enum tdx_ioctl_level {
     TDX_VM_IOCTL,
     TDX_VCPU_IOCTL,
@@ -117,15 +125,20 @@ static int get_tdx_capabilities(Error **errp)
 
 static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
 {
+    TdxGuest *tdx = TDX_GUEST(cgs);
     int r = 0;
 
     kvm_mark_guest_state_protected();
 
     if (!tdx_caps) {
         r = get_tdx_capabilities(errp);
+        if (r) {
+            return r;
+        }
     }
 
-    return r;
+    tdx_guest = tdx;
+    return 0;
 }
 
 static int tdx_kvm_type(X86ConfidentialGuest *cg)
diff --git a/target/i386/kvm/tdx.h b/target/i386/kvm/tdx.h
index f3b725336161..de8ae9196163 100644
--- a/target/i386/kvm/tdx.h
+++ b/target/i386/kvm/tdx.h
@@ -3,6 +3,10 @@
 #ifndef QEMU_I386_TDX_H
 #define QEMU_I386_TDX_H
 
+#ifndef CONFIG_USER_ONLY
+#include CONFIG_DEVICES /* CONFIG_TDX */
+#endif
+
 #include "confidential-guest.h"
 
 #define TYPE_TDX_GUEST "tdx-guest"
@@ -18,4 +22,10 @@ typedef struct TdxGuest {
     uint64_t attributes;    /* TD attributes */
 } TdxGuest;
 
+#ifdef CONFIG_TDX
+bool is_tdx_vm(void);
+#else
+#define is_tdx_vm() 0
+#endif /* CONFIG_TDX */
+
 #endif /* QEMU_I386_TDX_H */
-- 
2.34.1

Re: [PATCH v8 06/55] i386/tdx: Introduce is_tdx_vm() helper and cache tdx_guest object

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:16AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:16 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 06/55] i386/tdx: Introduce is_tdx_vm() helper and cache
>  tdx_guest object
> X-Mailer: git-send-email 2.34.1
> 
> It will need special handling for TDX VMs all around the QEMU.
> Introduce is_tdx_vm() helper to query if it's a TDX VM.
> 
> Cache tdx_guest object thus no need to cast from ms->cgs every time.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> Reviewed-by: Isaku Yamahata <isaku.yamahata@intel.com>
> ---
> changes in v3:
> - replace object_dynamic_cast with TDX_GUEST();
> ---
>  target/i386/kvm/tdx.c | 15 ++++++++++++++-
>  target/i386/kvm/tdx.h | 10 ++++++++++
>  2 files changed, 24 insertions(+), 1 deletion(-)
> 
> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
> index c67be5e618e2..16f67e18ae78 100644
> --- a/target/i386/kvm/tdx.c
> +++ b/target/i386/kvm/tdx.c
> @@ -18,8 +18,16 @@
>  #include "kvm_i386.h"
>  #include "tdx.h"
>  
> +static TdxGuest *tdx_guest;
> +
>  static struct kvm_tdx_capabilities *tdx_caps;
>  
> +/* Valid after kvm_arch_init()->confidential_guest_kvm_init()->tdx_kvm_init() */
> +bool is_tdx_vm(void)
> +{
> +    return !!tdx_guest;
> +}
> +
>  enum tdx_ioctl_level {
>      TDX_VM_IOCTL,
>      TDX_VCPU_IOCTL,
> @@ -117,15 +125,20 @@ static int get_tdx_capabilities(Error **errp)
>  
>  static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
>  {
> +    TdxGuest *tdx = TDX_GUEST(cgs);
>      int r = 0;
>  
>      kvm_mark_guest_state_protected();
>  
>      if (!tdx_caps) {
>          r = get_tdx_capabilities(errp);
> +        if (r) {
> +            return r;
> +        }
>      }
>  
> -    return r;
> +    tdx_guest = tdx;
> +    return 0;
>  }
>  
>  static int tdx_kvm_type(X86ConfidentialGuest *cg)
> diff --git a/target/i386/kvm/tdx.h b/target/i386/kvm/tdx.h
> index f3b725336161..de8ae9196163 100644
> --- a/target/i386/kvm/tdx.h
> +++ b/target/i386/kvm/tdx.h
> @@ -3,6 +3,10 @@
>  #ifndef QEMU_I386_TDX_H
>  #define QEMU_I386_TDX_H
>  
> +#ifndef CONFIG_USER_ONLY
> +#include CONFIG_DEVICES /* CONFIG_TDX */
> +#endif
> +
>  #include "confidential-guest.h"
>  
>  #define TYPE_TDX_GUEST "tdx-guest"
> @@ -18,4 +22,10 @@ typedef struct TdxGuest {
>      uint64_t attributes;    /* TD attributes */
>  } TdxGuest;
>  
> +#ifdef CONFIG_TDX
> +bool is_tdx_vm(void);
> +#else
> +#define is_tdx_vm() 0
> +#endif /* CONFIG_TDX */
> +
 
a little nit: could we rename it as "tdx_enabled"?

Then the cases like these would be neater?

if (sev_enabled() || is_tdx_vm()) {
    ...
}


if (sev_enabled()) {
    ...
} else if (is_tdx_vm()) {
    ...
}

Otherwise,

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

Thanks,
Zhao

Re: [PATCH v8 06/55] i386/tdx: Introduce is_tdx_vm() helper and cache tdx_guest object

[Xiaoyao Li]

On 4/18/2025 5:45 PM, Zhao Liu wrote:
> On Tue, Apr 01, 2025 at 09:01:16AM -0400, Xiaoyao Li wrote:
>> Date: Tue,  1 Apr 2025 09:01:16 -0400
>> From: Xiaoyao Li <xiaoyao.li@intel.com>
>> Subject: [PATCH v8 06/55] i386/tdx: Introduce is_tdx_vm() helper and cache
>>   tdx_guest object
>> X-Mailer: git-send-email 2.34.1
>>
>> It will need special handling for TDX VMs all around the QEMU.
>> Introduce is_tdx_vm() helper to query if it's a TDX VM.
>>
>> Cache tdx_guest object thus no need to cast from ms->cgs every time.
>>
>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
>> Reviewed-by: Isaku Yamahata <isaku.yamahata@intel.com>
>> ---
>> changes in v3:
>> - replace object_dynamic_cast with TDX_GUEST();
>> ---
>>   target/i386/kvm/tdx.c | 15 ++++++++++++++-
>>   target/i386/kvm/tdx.h | 10 ++++++++++
>>   2 files changed, 24 insertions(+), 1 deletion(-)
>>
>> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
>> index c67be5e618e2..16f67e18ae78 100644
>> --- a/target/i386/kvm/tdx.c
>> +++ b/target/i386/kvm/tdx.c
>> @@ -18,8 +18,16 @@
>>   #include "kvm_i386.h"
>>   #include "tdx.h"
>>   
>> +static TdxGuest *tdx_guest;
>> +
>>   static struct kvm_tdx_capabilities *tdx_caps;
>>   
>> +/* Valid after kvm_arch_init()->confidential_guest_kvm_init()->tdx_kvm_init() */
>> +bool is_tdx_vm(void)
>> +{
>> +    return !!tdx_guest;
>> +}
>> +
>>   enum tdx_ioctl_level {
>>       TDX_VM_IOCTL,
>>       TDX_VCPU_IOCTL,
>> @@ -117,15 +125,20 @@ static int get_tdx_capabilities(Error **errp)
>>   
>>   static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
>>   {
>> +    TdxGuest *tdx = TDX_GUEST(cgs);
>>       int r = 0;
>>   
>>       kvm_mark_guest_state_protected();
>>   
>>       if (!tdx_caps) {
>>           r = get_tdx_capabilities(errp);
>> +        if (r) {
>> +            return r;
>> +        }
>>       }
>>   
>> -    return r;
>> +    tdx_guest = tdx;
>> +    return 0;
>>   }
>>   
>>   static int tdx_kvm_type(X86ConfidentialGuest *cg)
>> diff --git a/target/i386/kvm/tdx.h b/target/i386/kvm/tdx.h
>> index f3b725336161..de8ae9196163 100644
>> --- a/target/i386/kvm/tdx.h
>> +++ b/target/i386/kvm/tdx.h
>> @@ -3,6 +3,10 @@
>>   #ifndef QEMU_I386_TDX_H
>>   #define QEMU_I386_TDX_H
>>   
>> +#ifndef CONFIG_USER_ONLY
>> +#include CONFIG_DEVICES /* CONFIG_TDX */
>> +#endif
>> +
>>   #include "confidential-guest.h"
>>   
>>   #define TYPE_TDX_GUEST "tdx-guest"
>> @@ -18,4 +22,10 @@ typedef struct TdxGuest {
>>       uint64_t attributes;    /* TD attributes */
>>   } TdxGuest;
>>   
>> +#ifdef CONFIG_TDX
>> +bool is_tdx_vm(void);
>> +#else
>> +#define is_tdx_vm() 0
>> +#endif /* CONFIG_TDX */
>> +
>   
> a little nit: could we rename it as "tdx_enabled"?
> 
> Then the cases like these would be neater?

When sev support was added, it was seen as a feature for the VMs that 
are created on AMD platform. I think that's why it got called sev_enabled().

But for TDX, it is introduced as a different type of VM in contrast to 
the legacy/normal VMX VMs. We need to pass specific TDX vm type to 
KVM_CREATE_VM. Based on this, is_tdx_vm() was chosen.

I don't think the different name is a big issue, as nobody mentions it 
from the initial RFC to current v8 until you. So again, I will just keep 
it unchanged unless someone objects it strongly.

> if (sev_enabled() || is_tdx_vm()) {
>      ...
> }
> 
> 
> if (sev_enabled()) {
>      ...
> } else if (is_tdx_vm()) {
>      ...
> }
> 
> Otherwise,
> 
> Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

Thanks!

> Thanks,
> Zhao
> 
> 

Re: [PATCH v8 06/55] i386/tdx: Introduce is_tdx_vm() helper and cache tdx_guest object

[Zhao Liu]

> > > +#ifdef CONFIG_TDX
> > > +bool is_tdx_vm(void);
> > > +#else
> > > +#define is_tdx_vm() 0
> > > +#endif /* CONFIG_TDX */
> > > +
> > a little nit: could we rename it as "tdx_enabled"?
> > 
> > Then the cases like these would be neater?
> 
> When sev support was added, it was seen as a feature for the VMs that are
> created on AMD platform. I think that's why it got called sev_enabled().
> 
> But for TDX, it is introduced as a different type of VM in contrast to the
> legacy/normal VMX VMs. We need to pass specific TDX vm type to
> KVM_CREATE_VM. Based on this, is_tdx_vm() was chosen.

But isn't AMD's SEV also defined as the VM type?

static const char *vm_type_name[] = {
    [KVM_X86_DEFAULT_VM] = "default",
    [KVM_X86_SEV_VM] = "SEV",
    [KVM_X86_SEV_ES_VM] = "SEV-ES",
    [KVM_X86_SNP_VM] = "SEV-SNP",
    [KVM_X86_TDX_VM] = "TDX",
};

Functionally, they are part of the coco functionality provided by
different vendors, so it's better thatt both could be in the same place
as much as possible, including file location, naming style. Of course,
it's not a big deal, and it can be cleaned up after merge if needed.

> I don't think the different name is a big issue, as nobody mentions it from
> the initial RFC to current v8 until you.

The Chinese saying: There are a thousand Hamlets in a thousand people's
eyes :-).

Re: [PATCH v8 06/55] i386/tdx: Introduce is_tdx_vm() helper and cache tdx_guest object

[Xiaoyao Li]

On 4/22/2025 10:20 PM, Zhao Liu wrote:
>>>> +#ifdef CONFIG_TDX
>>>> +bool is_tdx_vm(void);
>>>> +#else
>>>> +#define is_tdx_vm() 0
>>>> +#endif /* CONFIG_TDX */
>>>> +
>>> a little nit: could we rename it as "tdx_enabled"?
>>>
>>> Then the cases like these would be neater?
>>
>> When sev support was added, it was seen as a feature for the VMs that are
>> created on AMD platform. I think that's why it got called sev_enabled().
>>
>> But for TDX, it is introduced as a different type of VM in contrast to the
>> legacy/normal VMX VMs. We need to pass specific TDX vm type to
>> KVM_CREATE_VM. Based on this, is_tdx_vm() was chosen.
> 
> But isn't AMD's SEV also defined as the VM type?

The initial SEV support that introduced sev_eanbled(), didn't introduce 
the VM type.

The specific type for SEV* was added later.

> static const char *vm_type_name[] = {
>      [KVM_X86_DEFAULT_VM] = "default",
>      [KVM_X86_SEV_VM] = "SEV",
>      [KVM_X86_SEV_ES_VM] = "SEV-ES",
>      [KVM_X86_SNP_VM] = "SEV-SNP",
>      [KVM_X86_TDX_VM] = "TDX",
> };
> 
> Functionally, they are part of the coco functionality provided by
> different vendors, so it's better thatt both could be in the same place
> as much as possible, including file location, naming style. Of course,
> it's not a big deal, and it can be cleaned up after merge if needed.

yes, cleanup can be a separate work if Paolo doesn't dislike.

>> I don't think the different name is a big issue, as nobody mentions it from
>> the initial RFC to current v8 until you.
> 
> The Chinese saying: There are a thousand Hamlets in a thousand people's
> eyes :-).
> 

[PATCH v8 07/55] kvm: Introduce kvm_arch_pre_create_vcpu()

[Xiaoyao Li]

Introduce kvm_arch_pre_create_vcpu(), to perform arch-dependent
work prior to create any vcpu. This is for i386 TDX because it needs
call TDX_INIT_VM before creating any vcpu.

The specific implemnet of i386 will be added in the future patch.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
Changes in v7:
- Implement stub for all the ARCHes instead of defining it with weak
  attribute; (Philippe)

Changes in v3:
- pass @errp to kvm_arch_pre_create_vcpu(); (Per Daniel)
---
 accel/kvm/kvm-all.c        | 5 +++++
 include/system/kvm.h       | 1 +
 target/arm/kvm.c           | 5 +++++
 target/i386/kvm/kvm.c      | 5 +++++
 target/loongarch/kvm/kvm.c | 4 ++++
 target/mips/kvm.c          | 5 +++++
 target/ppc/kvm.c           | 5 +++++
 target/riscv/kvm/kvm-cpu.c | 5 +++++
 target/s390x/kvm/kvm.c     | 5 +++++
 9 files changed, 40 insertions(+)

diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index f89568bfa397..df9840e53a35 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -540,6 +540,11 @@ int kvm_init_vcpu(CPUState *cpu, Error **errp)
 
     trace_kvm_init_vcpu(cpu->cpu_index, kvm_arch_vcpu_id(cpu));
 
+    ret = kvm_arch_pre_create_vcpu(cpu, errp);
+    if (ret < 0) {
+        goto err;
+    }
+
     ret = kvm_create_vcpu(cpu);
     if (ret < 0) {
         error_setg_errno(errp, -ret,
diff --git a/include/system/kvm.h b/include/system/kvm.h
index ab17c09a551f..d7dfa25493a2 100644
--- a/include/system/kvm.h
+++ b/include/system/kvm.h
@@ -374,6 +374,7 @@ int kvm_arch_get_default_type(MachineState *ms);
 
 int kvm_arch_init(MachineState *ms, KVMState *s);
 
+int kvm_arch_pre_create_vcpu(CPUState *cpu, Error **errp);
 int kvm_arch_init_vcpu(CPUState *cpu);
 int kvm_arch_destroy_vcpu(CPUState *cpu);
 
diff --git a/target/arm/kvm.c b/target/arm/kvm.c
index da30bdbb2349..93f1a7245b3f 100644
--- a/target/arm/kvm.c
+++ b/target/arm/kvm.c
@@ -1874,6 +1874,11 @@ static int kvm_arm_sve_set_vls(ARMCPU *cpu)
 
 #define ARM_CPU_ID_MPIDR       3, 0, 0, 0, 5
 
+int kvm_arch_pre_create_vcpu(CPUState *cpu, Error **errp)
+{
+    return 0;
+}
+
 int kvm_arch_init_vcpu(CPUState *cs)
 {
     int ret;
diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index b4fa35405fe1..1a4dd19e24ab 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -2050,6 +2050,11 @@ full:
     abort();
 }
 
+int kvm_arch_pre_create_vcpu(CPUState *cpu, Error **errp)
+{
+    return 0;
+}
+
 int kvm_arch_init_vcpu(CPUState *cs)
 {
     struct {
diff --git a/target/loongarch/kvm/kvm.c b/target/loongarch/kvm/kvm.c
index 7f63e7c8fe51..5e82dacc677b 100644
--- a/target/loongarch/kvm/kvm.c
+++ b/target/loongarch/kvm/kvm.c
@@ -1071,7 +1071,11 @@ static int kvm_cpu_check_pv_features(CPUState *cs, Error **errp)
             env->pv_features |= BIT(KVM_FEATURE_VIRT_EXTIOI);
         }
     }
+    return 0;
+}
 
+int kvm_arch_pre_create_vcpu(CPUState *cpu, Error **errp)
+{
     return 0;
 }
 
diff --git a/target/mips/kvm.c b/target/mips/kvm.c
index d67b7c1a8ecb..ec53acb51a1f 100644
--- a/target/mips/kvm.c
+++ b/target/mips/kvm.c
@@ -61,6 +61,11 @@ int kvm_arch_irqchip_create(KVMState *s)
     return 0;
 }
 
+int kvm_arch_pre_create_vcpu(CPUState *cpu, Error **errp)
+{
+    return 0;
+}
+
 int kvm_arch_init_vcpu(CPUState *cs)
 {
     CPUMIPSState *env = cpu_env(cs);
diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c
index 992356cb7593..20fabccecd54 100644
--- a/target/ppc/kvm.c
+++ b/target/ppc/kvm.c
@@ -479,6 +479,11 @@ static void kvmppc_hw_debug_points_init(CPUPPCState *cenv)
     }
 }
 
+int kvm_arch_pre_create_vcpu(CPUState *cpu, Error **errp)
+{
+    return 0;
+}
+
 int kvm_arch_init_vcpu(CPUState *cs)
 {
     PowerPCCPU *cpu = POWERPC_CPU(cs);
diff --git a/target/riscv/kvm/kvm-cpu.c b/target/riscv/kvm/kvm-cpu.c
index 4ffeeaa1c953..451c00f17c2f 100644
--- a/target/riscv/kvm/kvm-cpu.c
+++ b/target/riscv/kvm/kvm-cpu.c
@@ -1389,6 +1389,11 @@ static int kvm_vcpu_enable_sbi_dbcn(RISCVCPU *cpu, CPUState *cs)
     return kvm_set_one_reg(cs, kvm_sbi_dbcn.kvm_reg_id, &reg);
 }
 
+int kvm_arch_pre_create_vcpu(CPUState *cpu, Error **errp)
+{
+    return 0;
+}
+
 int kvm_arch_init_vcpu(CPUState *cs)
 {
     int ret = 0;
diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
index 4d56e653ddf6..1f592733f4e2 100644
--- a/target/s390x/kvm/kvm.c
+++ b/target/s390x/kvm/kvm.c
@@ -404,6 +404,11 @@ unsigned long kvm_arch_vcpu_id(CPUState *cpu)
     return cpu->cpu_index;
 }
 
+int kvm_arch_pre_create_vcpu(CPUState *cpu, Error **errp)
+{
+    return 0;
+}
+
 int kvm_arch_init_vcpu(CPUState *cs)
 {
     unsigned int max_cpus = MACHINE(qdev_get_machine())->smp.max_cpus;
-- 
2.34.1

Re: [PATCH v8 07/55] kvm: Introduce kvm_arch_pre_create_vcpu()

[Daniel P. Berrangé]

On Tue, Apr 01, 2025 at 09:01:17AM -0400, Xiaoyao Li wrote:
> Introduce kvm_arch_pre_create_vcpu(), to perform arch-dependent
> work prior to create any vcpu. This is for i386 TDX because it needs
> call TDX_INIT_VM before creating any vcpu.
> 
> The specific implemnet of i386 will be added in the future patch.

s/implemnet of/implementation for'

> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
> Changes in v7:
> - Implement stub for all the ARCHes instead of defining it with weak
>   attribute; (Philippe)
> 
> Changes in v3:
> - pass @errp to kvm_arch_pre_create_vcpu(); (Per Daniel)
> ---
>  accel/kvm/kvm-all.c        | 5 +++++
>  include/system/kvm.h       | 1 +
>  target/arm/kvm.c           | 5 +++++
>  target/i386/kvm/kvm.c      | 5 +++++
>  target/loongarch/kvm/kvm.c | 4 ++++
>  target/mips/kvm.c          | 5 +++++
>  target/ppc/kvm.c           | 5 +++++
>  target/riscv/kvm/kvm-cpu.c | 5 +++++
>  target/s390x/kvm/kvm.c     | 5 +++++
>  9 files changed, 40 insertions(+)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v8 07/55] kvm: Introduce kvm_arch_pre_create_vcpu()

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:17AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:17 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 07/55] kvm: Introduce kvm_arch_pre_create_vcpu()
> X-Mailer: git-send-email 2.34.1
> 
> Introduce kvm_arch_pre_create_vcpu(), to perform arch-dependent
> work prior to create any vcpu. This is for i386 TDX because it needs
> call TDX_INIT_VM before creating any vcpu.
> 
> The specific implemnet of i386 will be added in the future patch.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
> Changes in v7:
> - Implement stub for all the ARCHes instead of defining it with weak
>   attribute; (Philippe)
> 
> Changes in v3:
> - pass @errp to kvm_arch_pre_create_vcpu(); (Per Daniel)
> ---
>  accel/kvm/kvm-all.c        | 5 +++++
>  include/system/kvm.h       | 1 +
>  target/arm/kvm.c           | 5 +++++
>  target/i386/kvm/kvm.c      | 5 +++++
>  target/loongarch/kvm/kvm.c | 4 ++++
>  target/mips/kvm.c          | 5 +++++
>  target/ppc/kvm.c           | 5 +++++
>  target/riscv/kvm/kvm-cpu.c | 5 +++++
>  target/s390x/kvm/kvm.c     | 5 +++++
>  9 files changed, 40 insertions(+)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 08/55] i386/tdx: Initialize TDX before creating TD vcpus

[Xiaoyao Li]

Invoke KVM_TDX_INIT_VM in kvm_arch_pre_create_vcpu() that
KVM_TDX_INIT_VM configures global TD configurations, e.g. the canonical
CPUID config, and must be executed prior to creating vCPUs.

Use kvm_x86_arch_cpuid() to setup the CPUID settings for TDX VM.

Note, this doesn't address the fact that QEMU may change the CPUID
configuration when creating vCPUs, i.e. punts on refactoring QEMU to
provide a stable CPUID config prior to kvm_arch_init().

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
---
Changes in v8:
- Drop the code that initializes cpu->kvm_state before
  kvm_arch_pre_create_vcpu() because it's not needed anymore.

Changes in v7:
- Add comments to explain why KVM_TDX_INIT_VM should retry on -EAGAIN;
- Add retry limit of 10000 times for -EAGAIN on KVM_TDX_INIT_VM;

Changes in v6:
- setup xfam explicitly to fit with new uapi;
- use tdx_caps->cpuid to filter the input of cpuids because now KVM only
  allows the leafs that reported via KVM_TDX_GET_CAPABILITIES;

Changes in v4:
- mark init_vm with g_autofree() and use QEMU_LOCK_GUARD() to eliminate
  the goto labels; (Daniel)
Changes in v3:
- Pass @errp in tdx_pre_create_vcpu() and pass error info to it. (Daniel)
---
 target/i386/kvm/kvm.c       |  16 +++---
 target/i386/kvm/kvm_i386.h  |   5 ++
 target/i386/kvm/meson.build |   2 +-
 target/i386/kvm/tdx-stub.c  |  10 ++++
 target/i386/kvm/tdx.c       | 105 ++++++++++++++++++++++++++++++++++++
 target/i386/kvm/tdx.h       |   6 +++
 6 files changed, 137 insertions(+), 7 deletions(-)
 create mode 100644 target/i386/kvm/tdx-stub.c

diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index 1a4dd19e24ab..a537699bb7df 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -38,6 +38,7 @@
 #include "kvm_i386.h"
 #include "../confidential-guest.h"
 #include "sev.h"
+#include "tdx.h"
 #include "xen-emu.h"
 #include "hyperv.h"
 #include "hyperv-proto.h"
@@ -414,9 +415,9 @@ static uint32_t cpuid_entry_get_reg(struct kvm_cpuid_entry2 *entry, int reg)
 
 /* Find matching entry for function/index on kvm_cpuid2 struct
  */
-static struct kvm_cpuid_entry2 *cpuid_find_entry(struct kvm_cpuid2 *cpuid,
-                                                 uint32_t function,
-                                                 uint32_t index)
+struct kvm_cpuid_entry2 *cpuid_find_entry(struct kvm_cpuid2 *cpuid,
+                                          uint32_t function,
+                                          uint32_t index)
 {
     int i;
     for (i = 0; i < cpuid->nent; ++i) {
@@ -1821,9 +1822,8 @@ static void kvm_init_nested_state(CPUX86State *env)
     }
 }
 
-static uint32_t kvm_x86_build_cpuid(CPUX86State *env,
-                                    struct kvm_cpuid_entry2 *entries,
-                                    uint32_t cpuid_i)
+uint32_t kvm_x86_build_cpuid(CPUX86State *env, struct kvm_cpuid_entry2 *entries,
+                             uint32_t cpuid_i)
 {
     uint32_t limit, i, j;
     uint32_t unused;
@@ -2052,6 +2052,10 @@ full:
 
 int kvm_arch_pre_create_vcpu(CPUState *cpu, Error **errp)
 {
+    if (is_tdx_vm()) {
+        return tdx_pre_create_vcpu(cpu, errp);
+    }
+
     return 0;
 }
 
diff --git a/target/i386/kvm/kvm_i386.h b/target/i386/kvm/kvm_i386.h
index ed1e61fb8ba9..dc696cb7238a 100644
--- a/target/i386/kvm/kvm_i386.h
+++ b/target/i386/kvm/kvm_i386.h
@@ -59,6 +59,11 @@ uint64_t kvm_swizzle_msi_ext_dest_id(uint64_t address);
 void kvm_update_msi_routes_all(void *private, bool global,
                                uint32_t index, uint32_t mask);
 
+struct kvm_cpuid_entry2 *cpuid_find_entry(struct kvm_cpuid2 *cpuid,
+                                          uint32_t function,
+                                          uint32_t index);
+uint32_t kvm_x86_build_cpuid(CPUX86State *env, struct kvm_cpuid_entry2 *entries,
+                             uint32_t cpuid_i);
 #endif /* CONFIG_KVM */
 
 void kvm_pc_setup_irq_routing(bool pci_enabled);
diff --git a/target/i386/kvm/meson.build b/target/i386/kvm/meson.build
index 466bccb9cb17..3f44cdedb758 100644
--- a/target/i386/kvm/meson.build
+++ b/target/i386/kvm/meson.build
@@ -8,7 +8,7 @@ i386_kvm_ss.add(files(
 
 i386_kvm_ss.add(when: 'CONFIG_XEN_EMU', if_true: files('xen-emu.c'))
 
-i386_kvm_ss.add(when: 'CONFIG_TDX', if_true: files('tdx.c'))
+i386_kvm_ss.add(when: 'CONFIG_TDX', if_true: files('tdx.c'), if_false: files('tdx-stub.c'))
 
 i386_system_ss.add(when: 'CONFIG_HYPERV', if_true: files('hyperv.c'), if_false: files('hyperv-stub.c'))
 
diff --git a/target/i386/kvm/tdx-stub.c b/target/i386/kvm/tdx-stub.c
new file mode 100644
index 000000000000..2344433594ea
--- /dev/null
+++ b/target/i386/kvm/tdx-stub.c
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#include "qemu/osdep.h"
+
+#include "tdx.h"
+
+int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
+{
+    return -EINVAL;
+}
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 16f67e18ae78..0afaf739c09f 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -149,6 +149,109 @@ static int tdx_kvm_type(X86ConfidentialGuest *cg)
     return KVM_X86_TDX_VM;
 }
 
+static int setup_td_xfam(X86CPU *x86cpu, Error **errp)
+{
+    CPUX86State *env = &x86cpu->env;
+    uint64_t xfam;
+
+    xfam = env->features[FEAT_XSAVE_XCR0_LO] |
+           env->features[FEAT_XSAVE_XCR0_HI] |
+           env->features[FEAT_XSAVE_XSS_LO] |
+           env->features[FEAT_XSAVE_XSS_HI];
+
+    if (xfam & ~tdx_caps->supported_xfam) {
+        error_setg(errp, "Invalid XFAM 0x%lx for TDX VM (supported: 0x%llx))",
+                   xfam, tdx_caps->supported_xfam);
+        return -1;
+    }
+
+    tdx_guest->xfam = xfam;
+    return 0;
+}
+
+static void tdx_filter_cpuid(struct kvm_cpuid2 *cpuids)
+{
+    int i, dest_cnt = 0;
+    struct kvm_cpuid_entry2 *src, *dest, *conf;
+
+    for (i = 0; i < cpuids->nent; i++) {
+        src = cpuids->entries + i;
+        conf = cpuid_find_entry(&tdx_caps->cpuid, src->function, src->index);
+        if (!conf) {
+            continue;
+        }
+        dest = cpuids->entries + dest_cnt;
+
+        dest->function = src->function;
+        dest->index = src->index;
+        dest->flags = src->flags;
+        dest->eax = src->eax & conf->eax;
+        dest->ebx = src->ebx & conf->ebx;
+        dest->ecx = src->ecx & conf->ecx;
+        dest->edx = src->edx & conf->edx;
+
+        dest_cnt++;
+    }
+    cpuids->nent = dest_cnt++;
+}
+
+int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
+{
+    X86CPU *x86cpu = X86_CPU(cpu);
+    CPUX86State *env = &x86cpu->env;
+    g_autofree struct kvm_tdx_init_vm *init_vm = NULL;
+    Error *local_err = NULL;
+    int retry = 10000;
+    int r = 0;
+
+    QEMU_LOCK_GUARD(&tdx_guest->lock);
+    if (tdx_guest->initialized) {
+        return r;
+    }
+
+    init_vm = g_malloc0(sizeof(struct kvm_tdx_init_vm) +
+                        sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES);
+
+    r = setup_td_xfam(x86cpu, errp);
+    if (r) {
+        return r;
+    }
+
+    init_vm->cpuid.nent = kvm_x86_build_cpuid(env, init_vm->cpuid.entries, 0);
+    tdx_filter_cpuid(&init_vm->cpuid);
+
+    init_vm->attributes = tdx_guest->attributes;
+    init_vm->xfam = tdx_guest->xfam;
+
+    /*
+     * KVM_TDX_INIT_VM gets -EAGAIN when KVM side SEAMCALL(TDH_MNG_CREATE)
+     * gets TDX_RND_NO_ENTROPY due to Random number generation (e.g., RDRAND or
+     * RDSEED) is busy.
+     *
+     * Retry for the case.
+     */
+    do {
+        error_free(local_err);
+        local_err = NULL;
+        r = tdx_vm_ioctl(KVM_TDX_INIT_VM, 0, init_vm, &local_err);
+    } while (r == -EAGAIN && --retry);
+
+    if (r < 0) {
+        if (!retry) {
+            error_report("Hardware RNG (Random Number Generator) is busy "
+                         "occupied by someone (via RDRAND/RDSEED) maliciously, "
+                         "which leads to KVM_TDX_INIT_VM keeping failure "
+                         "due to lack of entropy.");
+        }
+        error_propagate(errp, local_err);
+        return r;
+    }
+
+    tdx_guest->initialized = true;
+
+    return 0;
+}
+
 /* tdx guest */
 OBJECT_DEFINE_TYPE_WITH_INTERFACES(TdxGuest,
                                    tdx_guest,
@@ -162,6 +265,8 @@ static void tdx_guest_init(Object *obj)
     ConfidentialGuestSupport *cgs = CONFIDENTIAL_GUEST_SUPPORT(obj);
     TdxGuest *tdx = TDX_GUEST(obj);
 
+    qemu_mutex_init(&tdx->lock);
+
     cgs->require_guest_memfd = true;
     tdx->attributes = 0;
 
diff --git a/target/i386/kvm/tdx.h b/target/i386/kvm/tdx.h
index de8ae9196163..4e2b5c61ff5b 100644
--- a/target/i386/kvm/tdx.h
+++ b/target/i386/kvm/tdx.h
@@ -19,7 +19,11 @@ typedef struct TdxGuestClass {
 typedef struct TdxGuest {
     X86ConfidentialGuest parent_obj;
 
+    QemuMutex lock;
+
+    bool initialized;
     uint64_t attributes;    /* TD attributes */
+    uint64_t xfam;
 } TdxGuest;
 
 #ifdef CONFIG_TDX
@@ -28,4 +32,6 @@ bool is_tdx_vm(void);
 #define is_tdx_vm() 0
 #endif /* CONFIG_TDX */
 
+int tdx_pre_create_vcpu(CPUState *cpu, Error **errp);
+
 #endif /* QEMU_I386_TDX_H */
-- 
2.34.1

Re: [PATCH v8 08/55] i386/tdx: Initialize TDX before creating TD vcpus

[Daniel P. Berrangé]

On Tue, Apr 01, 2025 at 09:01:18AM -0400, Xiaoyao Li wrote:
> Invoke KVM_TDX_INIT_VM in kvm_arch_pre_create_vcpu() that
> KVM_TDX_INIT_VM configures global TD configurations, e.g. the canonical
> CPUID config, and must be executed prior to creating vCPUs.
> 
> Use kvm_x86_arch_cpuid() to setup the CPUID settings for TDX VM.
> 
> Note, this doesn't address the fact that QEMU may change the CPUID
> configuration when creating vCPUs, i.e. punts on refactoring QEMU to
> provide a stable CPUID config prior to kvm_arch_init().
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> Acked-by: Markus Armbruster <armbru@redhat.com>
> ---
> Changes in v8:
> - Drop the code that initializes cpu->kvm_state before
>   kvm_arch_pre_create_vcpu() because it's not needed anymore.
> 
> Changes in v7:
> - Add comments to explain why KVM_TDX_INIT_VM should retry on -EAGAIN;
> - Add retry limit of 10000 times for -EAGAIN on KVM_TDX_INIT_VM;
> 
> Changes in v6:
> - setup xfam explicitly to fit with new uapi;
> - use tdx_caps->cpuid to filter the input of cpuids because now KVM only
>   allows the leafs that reported via KVM_TDX_GET_CAPABILITIES;
> 
> Changes in v4:
> - mark init_vm with g_autofree() and use QEMU_LOCK_GUARD() to eliminate
>   the goto labels; (Daniel)
> Changes in v3:
> - Pass @errp in tdx_pre_create_vcpu() and pass error info to it. (Daniel)
> ---
>  target/i386/kvm/kvm.c       |  16 +++---
>  target/i386/kvm/kvm_i386.h  |   5 ++
>  target/i386/kvm/meson.build |   2 +-
>  target/i386/kvm/tdx-stub.c  |  10 ++++
>  target/i386/kvm/tdx.c       | 105 ++++++++++++++++++++++++++++++++++++
>  target/i386/kvm/tdx.h       |   6 +++
>  6 files changed, 137 insertions(+), 7 deletions(-)
>  create mode 100644 target/i386/kvm/tdx-stub.c


> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
> index 16f67e18ae78..0afaf739c09f 100644
> --- a/target/i386/kvm/tdx.c
> +++ b/target/i386/kvm/tdx.c

> +int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
> +{
> +    X86CPU *x86cpu = X86_CPU(cpu);
> +    CPUX86State *env = &x86cpu->env;
> +    g_autofree struct kvm_tdx_init_vm *init_vm = NULL;
> +    Error *local_err = NULL;
> +    int retry = 10000;
> +    int r = 0;
> +
> +    QEMU_LOCK_GUARD(&tdx_guest->lock);
> +    if (tdx_guest->initialized) {
> +        return r;
> +    }
> +
> +    init_vm = g_malloc0(sizeof(struct kvm_tdx_init_vm) +
> +                        sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES);
> +
> +    r = setup_td_xfam(x86cpu, errp);
> +    if (r) {
> +        return r;
> +    }
> +
> +    init_vm->cpuid.nent = kvm_x86_build_cpuid(env, init_vm->cpuid.entries, 0);
> +    tdx_filter_cpuid(&init_vm->cpuid);
> +
> +    init_vm->attributes = tdx_guest->attributes;
> +    init_vm->xfam = tdx_guest->xfam;
> +
> +    /*
> +     * KVM_TDX_INIT_VM gets -EAGAIN when KVM side SEAMCALL(TDH_MNG_CREATE)
> +     * gets TDX_RND_NO_ENTROPY due to Random number generation (e.g., RDRAND or
> +     * RDSEED) is busy.
> +     *
> +     * Retry for the case.
> +     */
> +    do {
> +        error_free(local_err);
> +        local_err = NULL;
> +        r = tdx_vm_ioctl(KVM_TDX_INIT_VM, 0, init_vm, &local_err);
> +    } while (r == -EAGAIN && --retry);
> +
> +    if (r < 0) {
> +        if (!retry) {
> +            error_report("Hardware RNG (Random Number Generator) is busy "
> +                         "occupied by someone (via RDRAND/RDSEED) maliciously, "
> +                         "which leads to KVM_TDX_INIT_VM keeping failure "
> +                         "due to lack of entropy.");

This needs to be

     error_append_hint(local_err, ....);

so that this message gets associated with the error object that
is propagated, and the top level will print it all at once.

> +        }
> +        error_propagate(errp, local_err);
> +        return r;
> +    }
> +
> +    tdx_guest->initialized = true;
> +
> +    return 0;
> +}

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v8 08/55] i386/tdx: Initialize TDX before creating TD vcpus

[Xiaoyao Li]

On 4/2/2025 7:41 PM, Daniel P. Berrangé wrote:
> On Tue, Apr 01, 2025 at 09:01:18AM -0400, Xiaoyao Li wrote:
>> Invoke KVM_TDX_INIT_VM in kvm_arch_pre_create_vcpu() that
>> KVM_TDX_INIT_VM configures global TD configurations, e.g. the canonical
>> CPUID config, and must be executed prior to creating vCPUs.
>>
>> Use kvm_x86_arch_cpuid() to setup the CPUID settings for TDX VM.
>>
>> Note, this doesn't address the fact that QEMU may change the CPUID
>> configuration when creating vCPUs, i.e. punts on refactoring QEMU to
>> provide a stable CPUID config prior to kvm_arch_init().
>>
>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
>> Acked-by: Markus Armbruster <armbru@redhat.com>
>> ---
>> Changes in v8:
>> - Drop the code that initializes cpu->kvm_state before
>>    kvm_arch_pre_create_vcpu() because it's not needed anymore.
>>
>> Changes in v7:
>> - Add comments to explain why KVM_TDX_INIT_VM should retry on -EAGAIN;
>> - Add retry limit of 10000 times for -EAGAIN on KVM_TDX_INIT_VM;
>>
>> Changes in v6:
>> - setup xfam explicitly to fit with new uapi;
>> - use tdx_caps->cpuid to filter the input of cpuids because now KVM only
>>    allows the leafs that reported via KVM_TDX_GET_CAPABILITIES;
>>
>> Changes in v4:
>> - mark init_vm with g_autofree() and use QEMU_LOCK_GUARD() to eliminate
>>    the goto labels; (Daniel)
>> Changes in v3:
>> - Pass @errp in tdx_pre_create_vcpu() and pass error info to it. (Daniel)
>> ---
>>   target/i386/kvm/kvm.c       |  16 +++---
>>   target/i386/kvm/kvm_i386.h  |   5 ++
>>   target/i386/kvm/meson.build |   2 +-
>>   target/i386/kvm/tdx-stub.c  |  10 ++++
>>   target/i386/kvm/tdx.c       | 105 ++++++++++++++++++++++++++++++++++++
>>   target/i386/kvm/tdx.h       |   6 +++
>>   6 files changed, 137 insertions(+), 7 deletions(-)
>>   create mode 100644 target/i386/kvm/tdx-stub.c
> 
> 
>> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
>> index 16f67e18ae78..0afaf739c09f 100644
>> --- a/target/i386/kvm/tdx.c
>> +++ b/target/i386/kvm/tdx.c
> 
>> +int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
>> +{
>> +    X86CPU *x86cpu = X86_CPU(cpu);
>> +    CPUX86State *env = &x86cpu->env;
>> +    g_autofree struct kvm_tdx_init_vm *init_vm = NULL;
>> +    Error *local_err = NULL;
>> +    int retry = 10000;
>> +    int r = 0;
>> +
>> +    QEMU_LOCK_GUARD(&tdx_guest->lock);
>> +    if (tdx_guest->initialized) {
>> +        return r;
>> +    }
>> +
>> +    init_vm = g_malloc0(sizeof(struct kvm_tdx_init_vm) +
>> +                        sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES);
>> +
>> +    r = setup_td_xfam(x86cpu, errp);
>> +    if (r) {
>> +        return r;
>> +    }
>> +
>> +    init_vm->cpuid.nent = kvm_x86_build_cpuid(env, init_vm->cpuid.entries, 0);
>> +    tdx_filter_cpuid(&init_vm->cpuid);
>> +
>> +    init_vm->attributes = tdx_guest->attributes;
>> +    init_vm->xfam = tdx_guest->xfam;
>> +
>> +    /*
>> +     * KVM_TDX_INIT_VM gets -EAGAIN when KVM side SEAMCALL(TDH_MNG_CREATE)
>> +     * gets TDX_RND_NO_ENTROPY due to Random number generation (e.g., RDRAND or
>> +     * RDSEED) is busy.
>> +     *
>> +     * Retry for the case.
>> +     */
>> +    do {
>> +        error_free(local_err);
>> +        local_err = NULL;
>> +        r = tdx_vm_ioctl(KVM_TDX_INIT_VM, 0, init_vm, &local_err);
>> +    } while (r == -EAGAIN && --retry);
>> +
>> +    if (r < 0) {
>> +        if (!retry) {
>> +            error_report("Hardware RNG (Random Number Generator) is busy "
>> +                         "occupied by someone (via RDRAND/RDSEED) maliciously, "
>> +                         "which leads to KVM_TDX_INIT_VM keeping failure "
>> +                         "due to lack of entropy.");
> 
> This needs to be
> 
>       error_append_hint(local_err, ....);
> 
> so that this message gets associated with the error object that
> is propagated, and the top level will print it all at once.

Good suggestion! Will change to it in the next version.

>> +        }
>> +        error_propagate(errp, local_err);
>> +        return r;
>> +    }
>> +
>> +    tdx_guest->initialized = true;
>> +
>> +    return 0;
>> +}
> 
> With regards,
> Daniel

Re: [PATCH v8 08/55] i386/tdx: Initialize TDX before creating TD vcpus

[Zhao Liu]

> > > +int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
> > > +{
> > > +    X86CPU *x86cpu = X86_CPU(cpu);
> > > +    CPUX86State *env = &x86cpu->env;
> > > +    g_autofree struct kvm_tdx_init_vm *init_vm = NULL;
> > > +    Error *local_err = NULL;
> > > +    int retry = 10000;
> > > +    int r = 0;
> > > +
> > > +    QEMU_LOCK_GUARD(&tdx_guest->lock);
> > > +    if (tdx_guest->initialized) {
> > > +        return r;
> > > +    }
> > > +
> > > +    init_vm = g_malloc0(sizeof(struct kvm_tdx_init_vm) +
> > > +                        sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES);
> > > +
> > > +    r = setup_td_xfam(x86cpu, errp);
> > > +    if (r) {
> > > +        return r;
> > > +    }
> > > +
> > > +    init_vm->cpuid.nent = kvm_x86_build_cpuid(env, init_vm->cpuid.entries, 0);
> > > +    tdx_filter_cpuid(&init_vm->cpuid);
> > > +
> > > +    init_vm->attributes = tdx_guest->attributes;
> > > +    init_vm->xfam = tdx_guest->xfam;
> > > +
> > > +    /*
> > > +     * KVM_TDX_INIT_VM gets -EAGAIN when KVM side SEAMCALL(TDH_MNG_CREATE)
> > > +     * gets TDX_RND_NO_ENTROPY due to Random number generation (e.g., RDRAND or
> > > +     * RDSEED) is busy.
> > > +     *
> > > +     * Retry for the case.
> > > +     */
> > > +    do {
> > > +        error_free(local_err);
> > > +        local_err = NULL;
> > > +        r = tdx_vm_ioctl(KVM_TDX_INIT_VM, 0, init_vm, &local_err);
> > > +    } while (r == -EAGAIN && --retry);
> > > +
> > > +    if (r < 0) {
> > > +        if (!retry) {
> > > +            error_report("Hardware RNG (Random Number Generator) is busy "
> > > +                         "occupied by someone (via RDRAND/RDSEED) maliciously, "
> > > +                         "which leads to KVM_TDX_INIT_VM keeping failure "
> > > +                         "due to lack of entropy.");
> > 
> > This needs to be
> > 
> >       error_append_hint(local_err, ....);
> > 
> > so that this message gets associated with the error object that
> > is propagated, and the top level will print it all at once.
> 
> Good suggestion! Will change to it in the next version.

A little suggestion:

With error_append_hint(local_err, ...), you can add "ERRP_GUARD()" at
the beginning of tdx_pre_create_vcpu(), just like the commit 95e9053a34ca.

Re: [PATCH v8 08/55] i386/tdx: Initialize TDX before creating TD vcpus

[Xiaoyao Li]

On 4/22/2025 11:34 PM, Zhao Liu wrote:
>>>> +int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
>>>> +{
>>>> +    X86CPU *x86cpu = X86_CPU(cpu);
>>>> +    CPUX86State *env = &x86cpu->env;
>>>> +    g_autofree struct kvm_tdx_init_vm *init_vm = NULL;
>>>> +    Error *local_err = NULL;
>>>> +    int retry = 10000;
>>>> +    int r = 0;
>>>> +
>>>> +    QEMU_LOCK_GUARD(&tdx_guest->lock);
>>>> +    if (tdx_guest->initialized) {
>>>> +        return r;
>>>> +    }
>>>> +
>>>> +    init_vm = g_malloc0(sizeof(struct kvm_tdx_init_vm) +
>>>> +                        sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES);
>>>> +
>>>> +    r = setup_td_xfam(x86cpu, errp);
>>>> +    if (r) {
>>>> +        return r;
>>>> +    }
>>>> +
>>>> +    init_vm->cpuid.nent = kvm_x86_build_cpuid(env, init_vm->cpuid.entries, 0);
>>>> +    tdx_filter_cpuid(&init_vm->cpuid);
>>>> +
>>>> +    init_vm->attributes = tdx_guest->attributes;
>>>> +    init_vm->xfam = tdx_guest->xfam;
>>>> +
>>>> +    /*
>>>> +     * KVM_TDX_INIT_VM gets -EAGAIN when KVM side SEAMCALL(TDH_MNG_CREATE)
>>>> +     * gets TDX_RND_NO_ENTROPY due to Random number generation (e.g., RDRAND or
>>>> +     * RDSEED) is busy.
>>>> +     *
>>>> +     * Retry for the case.
>>>> +     */
>>>> +    do {
>>>> +        error_free(local_err);
>>>> +        local_err = NULL;
>>>> +        r = tdx_vm_ioctl(KVM_TDX_INIT_VM, 0, init_vm, &local_err);
>>>> +    } while (r == -EAGAIN && --retry);
>>>> +
>>>> +    if (r < 0) {
>>>> +        if (!retry) {
>>>> +            error_report("Hardware RNG (Random Number Generator) is busy "
>>>> +                         "occupied by someone (via RDRAND/RDSEED) maliciously, "
>>>> +                         "which leads to KVM_TDX_INIT_VM keeping failure "
>>>> +                         "due to lack of entropy.");
>>>
>>> This needs to be
>>>
>>>        error_append_hint(local_err, ....);
>>>
>>> so that this message gets associated with the error object that
>>> is propagated, and the top level will print it all at once.
>>
>> Good suggestion! Will change to it in the next version.
> 
> A little suggestion:
> 
> With error_append_hint(local_err, ...), you can add "ERRP_GUARD()" at
> the beginning of tdx_pre_create_vcpu(), just like the commit 95e9053a34ca.

I don't think ERRP_GUARD() is needed.

ERRP_GUARD() is used to guard @errp, while here error_append_hint() is 
used on @local_err.

Re: [PATCH v8 08/55] i386/tdx: Initialize TDX before creating TD vcpus

[Zhao Liu]

> > A little suggestion:
> > 
> > With error_append_hint(local_err, ...), you can add "ERRP_GUARD()" at
> > the beginning of tdx_pre_create_vcpu(), just like the commit 95e9053a34ca.
> 
> I don't think ERRP_GUARD() is needed.
> 
> ERRP_GUARD() is used to guard @errp, while here error_append_hint() is used
> on @local_err.

Yes, you're right. My bad and thanks for correction!

Re: [PATCH v8 08/55] i386/tdx: Initialize TDX before creating TD vcpus

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:18AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:18 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 08/55] i386/tdx: Initialize TDX before creating TD vcpus
> X-Mailer: git-send-email 2.34.1
> 
> Invoke KVM_TDX_INIT_VM in kvm_arch_pre_create_vcpu() that
> KVM_TDX_INIT_VM configures global TD configurations, e.g. the canonical
> CPUID config, and must be executed prior to creating vCPUs.
> 
> Use kvm_x86_arch_cpuid() to setup the CPUID settings for TDX VM.
> 
> Note, this doesn't address the fact that QEMU may change the CPUID
> configuration when creating vCPUs, i.e. punts on refactoring QEMU to
> provide a stable CPUID config prior to kvm_arch_init().
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> Acked-by: Markus Armbruster <armbru@redhat.com>
> ---
> Changes in v8:
> - Drop the code that initializes cpu->kvm_state before
>   kvm_arch_pre_create_vcpu() because it's not needed anymore.
> 
> Changes in v7:
> - Add comments to explain why KVM_TDX_INIT_VM should retry on -EAGAIN;
> - Add retry limit of 10000 times for -EAGAIN on KVM_TDX_INIT_VM;
> 
> Changes in v6:
> - setup xfam explicitly to fit with new uapi;
> - use tdx_caps->cpuid to filter the input of cpuids because now KVM only
>   allows the leafs that reported via KVM_TDX_GET_CAPABILITIES;
> 
> Changes in v4:
> - mark init_vm with g_autofree() and use QEMU_LOCK_GUARD() to eliminate
>   the goto labels; (Daniel)
> Changes in v3:
> - Pass @errp in tdx_pre_create_vcpu() and pass error info to it. (Daniel)
> ---
>  target/i386/kvm/kvm.c       |  16 +++---
>  target/i386/kvm/kvm_i386.h  |   5 ++
>  target/i386/kvm/meson.build |   2 +-
>  target/i386/kvm/tdx-stub.c  |  10 ++++
>  target/i386/kvm/tdx.c       | 105 ++++++++++++++++++++++++++++++++++++
>  target/i386/kvm/tdx.h       |   6 +++
>  6 files changed, 137 insertions(+), 7 deletions(-)
>  create mode 100644 target/i386/kvm/tdx-stub.c

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 09/55] i386/tdx: Add property sept-ve-disable for tdx-guest object

[Xiaoyao Li]

Bit 28 of TD attribute, named SEPT_VE_DISABLE. When set to 1, it disables
EPT violation conversion to #VE on guest TD access of PENDING pages.

Some guest OS (e.g., Linux TD guest) may require this bit as 1.
Otherwise refuse to boot.

Add sept-ve-disable property for tdx-guest object, for user to configure
this bit.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
---
Changes in v4:
- collect Acked-by from Markus

Changes in v3:
- update the comment of property @sept-ve-disable to make it more
  descriptive and use new format. (Daniel and Markus)
---
 qapi/qom.json         |  8 +++++++-
 target/i386/kvm/tdx.c | 23 +++++++++++++++++++++++
 2 files changed, 30 insertions(+), 1 deletion(-)

diff --git a/qapi/qom.json b/qapi/qom.json
index c0b61df964ef..f229bb07aaec 100644
--- a/qapi/qom.json
+++ b/qapi/qom.json
@@ -1055,10 +1055,16 @@
 # @attributes: The 'attributes' of a TD guest that is passed to
 #     KVM_TDX_INIT_VM
 #
+# @sept-ve-disable: toggle bit 28 of TD attributes to control disabling
+#     of EPT violation conversion to #VE on guest TD access of PENDING
+#     pages.  Some guest OS (e.g., Linux TD guest) may require this to
+#     be set, otherwise they refuse to boot.
+#
 # Since: 10.1
 ##
 { 'struct': 'TdxGuestProperties',
-  'data': { '*attributes': 'uint64' } }
+  'data': { '*attributes': 'uint64',
+            '*sept-ve-disable': 'bool' } }
 
 ##
 # @ThreadContextProperties:
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 0afaf739c09f..081094ae29db 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -18,6 +18,8 @@
 #include "kvm_i386.h"
 #include "tdx.h"
 
+#define TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE   BIT_ULL(28)
+
 static TdxGuest *tdx_guest;
 
 static struct kvm_tdx_capabilities *tdx_caps;
@@ -252,6 +254,24 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
     return 0;
 }
 
+static bool tdx_guest_get_sept_ve_disable(Object *obj, Error **errp)
+{
+    TdxGuest *tdx = TDX_GUEST(obj);
+
+    return !!(tdx->attributes & TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE);
+}
+
+static void tdx_guest_set_sept_ve_disable(Object *obj, bool value, Error **errp)
+{
+    TdxGuest *tdx = TDX_GUEST(obj);
+
+    if (value) {
+        tdx->attributes |= TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE;
+    } else {
+        tdx->attributes &= ~TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE;
+    }
+}
+
 /* tdx guest */
 OBJECT_DEFINE_TYPE_WITH_INTERFACES(TdxGuest,
                                    tdx_guest,
@@ -272,6 +292,9 @@ static void tdx_guest_init(Object *obj)
 
     object_property_add_uint64_ptr(obj, "attributes", &tdx->attributes,
                                    OBJ_PROP_FLAG_READWRITE);
+    object_property_add_bool(obj, "sept-ve-disable",
+                             tdx_guest_get_sept_ve_disable,
+                             tdx_guest_set_sept_ve_disable);
 }
 
 static void tdx_guest_finalize(Object *obj)
-- 
2.34.1

Re: [PATCH v8 09/55] i386/tdx: Add property sept-ve-disable for tdx-guest object

[Daniel P. Berrangé]

On Tue, Apr 01, 2025 at 09:01:19AM -0400, Xiaoyao Li wrote:
> Bit 28 of TD attribute, named SEPT_VE_DISABLE. When set to 1, it disables
> EPT violation conversion to #VE on guest TD access of PENDING pages.
> 
> Some guest OS (e.g., Linux TD guest) may require this bit as 1.
> Otherwise refuse to boot.
> 
> Add sept-ve-disable property for tdx-guest object, for user to configure
> this bit.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> Acked-by: Markus Armbruster <armbru@redhat.com>
> ---
> Changes in v4:
> - collect Acked-by from Markus
> 
> Changes in v3:
> - update the comment of property @sept-ve-disable to make it more
>   descriptive and use new format. (Daniel and Markus)
> ---
>  qapi/qom.json         |  8 +++++++-
>  target/i386/kvm/tdx.c | 23 +++++++++++++++++++++++
>  2 files changed, 30 insertions(+), 1 deletion(-)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v8 09/55] i386/tdx: Add property sept-ve-disable for tdx-guest object

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:19AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:19 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 09/55] i386/tdx: Add property sept-ve-disable for
>  tdx-guest object
> X-Mailer: git-send-email 2.34.1
> 
> Bit 28 of TD attribute, named SEPT_VE_DISABLE. When set to 1, it disables
> EPT violation conversion to #VE on guest TD access of PENDING pages.
> 
> Some guest OS (e.g., Linux TD guest) may require this bit as 1.
> Otherwise refuse to boot.
> 
> Add sept-ve-disable property for tdx-guest object, for user to configure
> this bit.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> Acked-by: Markus Armbruster <armbru@redhat.com>
> ---
> Changes in v4:
> - collect Acked-by from Markus
> 
> Changes in v3:
> - update the comment of property @sept-ve-disable to make it more
>   descriptive and use new format. (Daniel and Markus)
> ---
>  qapi/qom.json         |  8 +++++++-
>  target/i386/kvm/tdx.c | 23 +++++++++++++++++++++++
>  2 files changed, 30 insertions(+), 1 deletion(-)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 10/55] i386/tdx: Make sept_ve_disable set by default

[Xiaoyao Li]

From: Isaku Yamahata <isaku.yamahata@intel.com>

For TDX KVM use case, Linux guest is the most major one.  It requires
sept_ve_disable set.  Make it default for the main use case.  For other use
case, it can be enabled/disabled via qemu command line.

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
 target/i386/kvm/tdx.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 081094ae29db..592fb9cbbf0b 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -288,7 +288,7 @@ static void tdx_guest_init(Object *obj)
     qemu_mutex_init(&tdx->lock);
 
     cgs->require_guest_memfd = true;
-    tdx->attributes = 0;
+    tdx->attributes = TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE;
 
     object_property_add_uint64_ptr(obj, "attributes", &tdx->attributes,
                                    OBJ_PROP_FLAG_READWRITE);
-- 
2.34.1

Re: [PATCH v8 10/55] i386/tdx: Make sept_ve_disable set by default

[Daniel P. Berrangé]

On Tue, Apr 01, 2025 at 09:01:20AM -0400, Xiaoyao Li wrote:
> From: Isaku Yamahata <isaku.yamahata@intel.com>
> 
> For TDX KVM use case, Linux guest is the most major one.  It requires
> sept_ve_disable set.  Make it default for the main use case.  For other use
> case, it can be enabled/disabled via qemu command line.
> 
> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
>  target/i386/kvm/tdx.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v8 10/55] i386/tdx: Make sept_ve_disable set by default

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:20AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:20 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 10/55] i386/tdx: Make sept_ve_disable set by default
> X-Mailer: git-send-email 2.34.1
> 
> From: Isaku Yamahata <isaku.yamahata@intel.com>
> 
> For TDX KVM use case, Linux guest is the most major one.  It requires
> sept_ve_disable set.  Make it default for the main use case.  For other use
> case, it can be enabled/disabled via qemu command line.
> 
> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
>  target/i386/kvm/tdx.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 11/55] i386/tdx: Wire CPU features up with attributes of TD guest

[Xiaoyao Li]

For QEMU VMs,
  - PKS is configured via CPUID_7_0_ECX_PKS, e.g., -cpu xxx,+pks  and
  - PMU is configured by x86cpu->enable_pmu, e.g., -cpu xxx,pmu=on

While the bit 30 (PKS) and bit 63 (PERFMON) of TD's attributes are also
used to configure the PKS and PERFMON/PMU of TD, reuse the existing
configuration interfaces of 'cpu' for TD's attributes.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
 target/i386/kvm/tdx.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 592fb9cbbf0b..1202b2111ba8 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -19,6 +19,8 @@
 #include "tdx.h"
 
 #define TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE   BIT_ULL(28)
+#define TDX_TD_ATTRIBUTES_PKS               BIT_ULL(30)
+#define TDX_TD_ATTRIBUTES_PERFMON           BIT_ULL(63)
 
 static TdxGuest *tdx_guest;
 
@@ -151,6 +153,15 @@ static int tdx_kvm_type(X86ConfidentialGuest *cg)
     return KVM_X86_TDX_VM;
 }
 
+static void setup_td_guest_attributes(X86CPU *x86cpu)
+{
+    CPUX86State *env = &x86cpu->env;
+
+    tdx_guest->attributes |= (env->features[FEAT_7_0_ECX] & CPUID_7_0_ECX_PKS) ?
+                             TDX_TD_ATTRIBUTES_PKS : 0;
+    tdx_guest->attributes |= x86cpu->enable_pmu ? TDX_TD_ATTRIBUTES_PERFMON : 0;
+}
+
 static int setup_td_xfam(X86CPU *x86cpu, Error **errp)
 {
     CPUX86State *env = &x86cpu->env;
@@ -214,6 +225,8 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
     init_vm = g_malloc0(sizeof(struct kvm_tdx_init_vm) +
                         sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES);
 
+    setup_td_guest_attributes(x86cpu);
+
     r = setup_td_xfam(x86cpu, errp);
     if (r) {
         return r;
-- 
2.34.1

Re: [PATCH v8 11/55] i386/tdx: Wire CPU features up with attributes of TD guest

[Daniel P. Berrangé]

On Tue, Apr 01, 2025 at 09:01:21AM -0400, Xiaoyao Li wrote:
> For QEMU VMs,
>   - PKS is configured via CPUID_7_0_ECX_PKS, e.g., -cpu xxx,+pks  and
>   - PMU is configured by x86cpu->enable_pmu, e.g., -cpu xxx,pmu=on
> 
> While the bit 30 (PKS) and bit 63 (PERFMON) of TD's attributes are also
> used to configure the PKS and PERFMON/PMU of TD, reuse the existing
> configuration interfaces of 'cpu' for TD's attributes.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
>  target/i386/kvm/tdx.c | 13 +++++++++++++
>  1 file changed, 13 insertions(+)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v8 11/55] i386/tdx: Wire CPU features up with attributes of TD guest

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:21AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:21 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 11/55] i386/tdx: Wire CPU features up with attributes of
>  TD guest
> X-Mailer: git-send-email 2.34.1
> 
> For QEMU VMs,
>   - PKS is configured via CPUID_7_0_ECX_PKS, e.g., -cpu xxx,+pks  and
>   - PMU is configured by x86cpu->enable_pmu, e.g., -cpu xxx,pmu=on
> 
> While the bit 30 (PKS) and bit 63 (PERFMON) of TD's attributes are also
> used to configure the PKS and PERFMON/PMU of TD, reuse the existing
> configuration interfaces of 'cpu' for TD's attributes.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
>  target/i386/kvm/tdx.c | 13 +++++++++++++
>  1 file changed, 13 insertions(+)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 12/55] i386/tdx: Validate TD attributes

[Xiaoyao Li]

Validate TD attributes with tdx_caps that only supported bits are
allowed by KVM.

Besides, sanity check the attribute bits that have not been supported by
QEMU yet. e.g., debug bit, it will be allowed in the future when debug
TD support lands in QEMU.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
Changes in v8:
- Split the mrconfigid/mrowner/mrownerconfig part into a seperate next
  patch;

Changes in v7:
- Define TDX_SUPPORTED_TD_ATTRS as QEMU supported mask, to validates
  user's request. (Rick)

Changes in v3:
- using error_setg() for error report; (Daniel)
---
 target/i386/kvm/tdx.c | 32 ++++++++++++++++++++++++++++++--
 1 file changed, 30 insertions(+), 2 deletions(-)

diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 1202b2111ba8..aa043acb1a88 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -18,10 +18,15 @@
 #include "kvm_i386.h"
 #include "tdx.h"
 
+#define TDX_TD_ATTRIBUTES_DEBUG             BIT_ULL(0)
 #define TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE   BIT_ULL(28)
 #define TDX_TD_ATTRIBUTES_PKS               BIT_ULL(30)
 #define TDX_TD_ATTRIBUTES_PERFMON           BIT_ULL(63)
 
+#define TDX_SUPPORTED_TD_ATTRS  (TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE |\
+                                 TDX_TD_ATTRIBUTES_PKS | \
+                                 TDX_TD_ATTRIBUTES_PERFMON)
+
 static TdxGuest *tdx_guest;
 
 static struct kvm_tdx_capabilities *tdx_caps;
@@ -153,13 +158,33 @@ static int tdx_kvm_type(X86ConfidentialGuest *cg)
     return KVM_X86_TDX_VM;
 }
 
-static void setup_td_guest_attributes(X86CPU *x86cpu)
+static int tdx_validate_attributes(TdxGuest *tdx, Error **errp)
+{
+    if ((tdx->attributes & ~tdx_caps->supported_attrs)) {
+        error_setg(errp, "Invalid attributes 0x%lx for TDX VM "
+                   "(KVM supported: 0x%llx)", tdx->attributes,
+                   tdx_caps->supported_attrs);
+        return -1;
+    }
+
+    if (tdx->attributes & ~TDX_SUPPORTED_TD_ATTRS) {
+        warn_report("Some QEMU unsupported TD attribute bits being requested:"
+                    "requested: 0x%lx QEMU supported: 0x%llx",
+                    tdx->attributes, TDX_SUPPORTED_TD_ATTRS);
+    }
+
+    return 0;
+}
+
+static int setup_td_guest_attributes(X86CPU *x86cpu, Error **errp)
 {
     CPUX86State *env = &x86cpu->env;
 
     tdx_guest->attributes |= (env->features[FEAT_7_0_ECX] & CPUID_7_0_ECX_PKS) ?
                              TDX_TD_ATTRIBUTES_PKS : 0;
     tdx_guest->attributes |= x86cpu->enable_pmu ? TDX_TD_ATTRIBUTES_PERFMON : 0;
+
+    return tdx_validate_attributes(tdx_guest, errp);
 }
 
 static int setup_td_xfam(X86CPU *x86cpu, Error **errp)
@@ -225,7 +250,10 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
     init_vm = g_malloc0(sizeof(struct kvm_tdx_init_vm) +
                         sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES);
 
-    setup_td_guest_attributes(x86cpu);
+    r = setup_td_guest_attributes(x86cpu, errp);
+    if (r) {
+        return r;
+    }
 
     r = setup_td_xfam(x86cpu, errp);
     if (r) {
-- 
2.34.1

Re: [PATCH v8 12/55] i386/tdx: Validate TD attributes

[Daniel P. Berrangé]

On Tue, Apr 01, 2025 at 09:01:22AM -0400, Xiaoyao Li wrote:
> Validate TD attributes with tdx_caps that only supported bits are
> allowed by KVM.
> 
> Besides, sanity check the attribute bits that have not been supported by
> QEMU yet. e.g., debug bit, it will be allowed in the future when debug
> TD support lands in QEMU.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
> Changes in v8:
> - Split the mrconfigid/mrowner/mrownerconfig part into a seperate next
>   patch;
> 
> Changes in v7:
> - Define TDX_SUPPORTED_TD_ATTRS as QEMU supported mask, to validates
>   user's request. (Rick)
> 
> Changes in v3:
> - using error_setg() for error report; (Daniel)
> ---
>  target/i386/kvm/tdx.c | 32 ++++++++++++++++++++++++++++++--
>  1 file changed, 30 insertions(+), 2 deletions(-)
> 
> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
> index 1202b2111ba8..aa043acb1a88 100644
> --- a/target/i386/kvm/tdx.c
> +++ b/target/i386/kvm/tdx.c
> @@ -18,10 +18,15 @@
>  #include "kvm_i386.h"
>  #include "tdx.h"
>  
> +#define TDX_TD_ATTRIBUTES_DEBUG             BIT_ULL(0)
>  #define TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE   BIT_ULL(28)
>  #define TDX_TD_ATTRIBUTES_PKS               BIT_ULL(30)
>  #define TDX_TD_ATTRIBUTES_PERFMON           BIT_ULL(63)
>  
> +#define TDX_SUPPORTED_TD_ATTRS  (TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE |\
> +                                 TDX_TD_ATTRIBUTES_PKS | \
> +                                 TDX_TD_ATTRIBUTES_PERFMON)
> +
>  static TdxGuest *tdx_guest;
>  
>  static struct kvm_tdx_capabilities *tdx_caps;
> @@ -153,13 +158,33 @@ static int tdx_kvm_type(X86ConfidentialGuest *cg)
>      return KVM_X86_TDX_VM;
>  }
>  
> -static void setup_td_guest_attributes(X86CPU *x86cpu)
> +static int tdx_validate_attributes(TdxGuest *tdx, Error **errp)
> +{
> +    if ((tdx->attributes & ~tdx_caps->supported_attrs)) {
> +        error_setg(errp, "Invalid attributes 0x%lx for TDX VM "
> +                   "(KVM supported: 0x%llx)", tdx->attributes,
> +                   tdx_caps->supported_attrs);
> +        return -1;
> +    }
> +
> +    if (tdx->attributes & ~TDX_SUPPORTED_TD_ATTRS) {
> +        warn_report("Some QEMU unsupported TD attribute bits being requested:"
> +                    "requested: 0x%lx QEMU supported: 0x%llx",
> +                    tdx->attributes, TDX_SUPPORTED_TD_ATTRS);

IIUC, this is an explicit user mis-configuration, and so we
ought to use  error_setg & return -1, not merely a warning.

> +    }
> +
> +    return 0;
> +}
> +
> +static int setup_td_guest_attributes(X86CPU *x86cpu, Error **errp)
>  {
>      CPUX86State *env = &x86cpu->env;
>  
>      tdx_guest->attributes |= (env->features[FEAT_7_0_ECX] & CPUID_7_0_ECX_PKS) ?
>                               TDX_TD_ATTRIBUTES_PKS : 0;
>      tdx_guest->attributes |= x86cpu->enable_pmu ? TDX_TD_ATTRIBUTES_PERFMON : 0;
> +
> +    return tdx_validate_attributes(tdx_guest, errp);
>  }
>  
>  static int setup_td_xfam(X86CPU *x86cpu, Error **errp)
> @@ -225,7 +250,10 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
>      init_vm = g_malloc0(sizeof(struct kvm_tdx_init_vm) +
>                          sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES);
>  
> -    setup_td_guest_attributes(x86cpu);
> +    r = setup_td_guest_attributes(x86cpu, errp);
> +    if (r) {
> +        return r;
> +    }
>  
>      r = setup_td_xfam(x86cpu, errp);
>      if (r) {
> -- 
> 2.34.1
> 

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v8 12/55] i386/tdx: Validate TD attributes

[Xiaoyao Li]

On 4/2/2025 7:47 PM, Daniel P. Berrangé wrote:
> On Tue, Apr 01, 2025 at 09:01:22AM -0400, Xiaoyao Li wrote:
>> Validate TD attributes with tdx_caps that only supported bits are
>> allowed by KVM.
>>
>> Besides, sanity check the attribute bits that have not been supported by
>> QEMU yet. e.g., debug bit, it will be allowed in the future when debug
>> TD support lands in QEMU.
>>
>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
>> ---
>> Changes in v8:
>> - Split the mrconfigid/mrowner/mrownerconfig part into a seperate next
>>    patch;
>>
>> Changes in v7:
>> - Define TDX_SUPPORTED_TD_ATTRS as QEMU supported mask, to validates
>>    user's request. (Rick)
>>
>> Changes in v3:
>> - using error_setg() for error report; (Daniel)
>> ---
>>   target/i386/kvm/tdx.c | 32 ++++++++++++++++++++++++++++++--
>>   1 file changed, 30 insertions(+), 2 deletions(-)
>>
>> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
>> index 1202b2111ba8..aa043acb1a88 100644
>> --- a/target/i386/kvm/tdx.c
>> +++ b/target/i386/kvm/tdx.c
>> @@ -18,10 +18,15 @@
>>   #include "kvm_i386.h"
>>   #include "tdx.h"
>>   
>> +#define TDX_TD_ATTRIBUTES_DEBUG             BIT_ULL(0)
>>   #define TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE   BIT_ULL(28)
>>   #define TDX_TD_ATTRIBUTES_PKS               BIT_ULL(30)
>>   #define TDX_TD_ATTRIBUTES_PERFMON           BIT_ULL(63)
>>   
>> +#define TDX_SUPPORTED_TD_ATTRS  (TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE |\
>> +                                 TDX_TD_ATTRIBUTES_PKS | \
>> +                                 TDX_TD_ATTRIBUTES_PERFMON)
>> +
>>   static TdxGuest *tdx_guest;
>>   
>>   static struct kvm_tdx_capabilities *tdx_caps;
>> @@ -153,13 +158,33 @@ static int tdx_kvm_type(X86ConfidentialGuest *cg)
>>       return KVM_X86_TDX_VM;
>>   }
>>   
>> -static void setup_td_guest_attributes(X86CPU *x86cpu)
>> +static int tdx_validate_attributes(TdxGuest *tdx, Error **errp)
>> +{
>> +    if ((tdx->attributes & ~tdx_caps->supported_attrs)) {
>> +        error_setg(errp, "Invalid attributes 0x%lx for TDX VM "
>> +                   "(KVM supported: 0x%llx)", tdx->attributes,
>> +                   tdx_caps->supported_attrs);
>> +        return -1;
>> +    }
>> +
>> +    if (tdx->attributes & ~TDX_SUPPORTED_TD_ATTRS) {
>> +        warn_report("Some QEMU unsupported TD attribute bits being requested:"
>> +                    "requested: 0x%lx QEMU supported: 0x%llx",
>> +                    tdx->attributes, TDX_SUPPORTED_TD_ATTRS);
> 
> IIUC, this is an explicit user mis-configuration, and so we
> ought to use  error_setg & return -1, not merely a warning.

My thought was to allow user to enable some QEMU unsupported but KVM 
supported attribute bits, for testing purpose.

But now I agree with you. It's not good for production QEMU.

>> +    }
>> +
>> +    return 0;
>> +}
>> +
>> +static int setup_td_guest_attributes(X86CPU *x86cpu, Error **errp)
>>   {
>>       CPUX86State *env = &x86cpu->env;
>>   
>>       tdx_guest->attributes |= (env->features[FEAT_7_0_ECX] & CPUID_7_0_ECX_PKS) ?
>>                                TDX_TD_ATTRIBUTES_PKS : 0;
>>       tdx_guest->attributes |= x86cpu->enable_pmu ? TDX_TD_ATTRIBUTES_PERFMON : 0;
>> +
>> +    return tdx_validate_attributes(tdx_guest, errp);
>>   }
>>   
>>   static int setup_td_xfam(X86CPU *x86cpu, Error **errp)
>> @@ -225,7 +250,10 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
>>       init_vm = g_malloc0(sizeof(struct kvm_tdx_init_vm) +
>>                           sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES);
>>   
>> -    setup_td_guest_attributes(x86cpu);
>> +    r = setup_td_guest_attributes(x86cpu, errp);
>> +    if (r) {
>> +        return r;
>> +    }
>>   
>>       r = setup_td_xfam(x86cpu, errp);
>>       if (r) {
>> -- 
>> 2.34.1
>>
> 
> With regards,
> Daniel

Re: [PATCH v8 12/55] i386/tdx: Validate TD attributes

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:22AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:22 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 12/55] i386/tdx: Validate TD attributes
> X-Mailer: git-send-email 2.34.1
> 
> Validate TD attributes with tdx_caps that only supported bits are
> allowed by KVM.
> 
> Besides, sanity check the attribute bits that have not been supported by
> QEMU yet. e.g., debug bit, it will be allowed in the future when debug
> TD support lands in QEMU.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
> Changes in v8:
> - Split the mrconfigid/mrowner/mrownerconfig part into a seperate next
>   patch;
> 
> Changes in v7:
> - Define TDX_SUPPORTED_TD_ATTRS as QEMU supported mask, to validates
>   user's request. (Rick)
> 
> Changes in v3:
> - using error_setg() for error report; (Daniel)
> ---
>  target/i386/kvm/tdx.c | 32 ++++++++++++++++++++++++++++++--
>  1 file changed, 30 insertions(+), 2 deletions(-)

Overall LGTM,

(with Daniel's comment fixed)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 13/55] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig

[Xiaoyao Li]

From: Isaku Yamahata <isaku.yamahata@intel.com>

Three sha384 hash values, mrconfigid, mrowner and mrownerconfig, of a TD
can be provided for TDX attestation. Detailed meaning of them can be
found: https://lore.kernel.org/qemu-devel/31d6dbc1-f453-4cef-ab08-4813f4e0ff92@intel.com/

Allow user to specify those values via property mrconfigid, mrowner and
mrownerconfig. They are all in base64 format.

example
-object tdx-guest, \
  mrconfigid=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
  mrowner=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
  mrownerconfig=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
Co-developed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
Changes in v8:
 - it gets squashed into previous patch in v7. So split it out in v8;

Changes in v6:
 - refine the doc comment of QAPI properties;

Changes in v5:
 - refine the description of QAPI properties and add description of
   default value when not specified;

Changes in v4:
 - describe more of there fields in qom.json
 - free the old value before set new value to avoid memory leak in
   _setter(); (Daniel)

Changes in v3:
 - use base64 encoding instread of hex-string;
---
 qapi/qom.json         | 16 +++++++-
 target/i386/kvm/tdx.c | 86 +++++++++++++++++++++++++++++++++++++++++++
 target/i386/kvm/tdx.h |  3 ++
 3 files changed, 104 insertions(+), 1 deletion(-)

diff --git a/qapi/qom.json b/qapi/qom.json
index f229bb07aaec..a8379bac1719 100644
--- a/qapi/qom.json
+++ b/qapi/qom.json
@@ -1060,11 +1060,25 @@
 #     pages.  Some guest OS (e.g., Linux TD guest) may require this to
 #     be set, otherwise they refuse to boot.
 #
+# @mrconfigid: ID for non-owner-defined configuration of the guest TD,
+#     e.g., run-time or OS configuration (base64 encoded SHA384 digest).
+#     Defaults to all zeros.
+#
+# @mrowner: ID for the guest TD’s owner (base64 encoded SHA384 digest).
+#     Defaults to all zeros.
+#
+# @mrownerconfig: ID for owner-defined configuration of the guest TD,
+#     e.g., specific to the workload rather than the run-time or OS
+#     (base64 encoded SHA384 digest).  Defaults to all zeros.
+#
 # Since: 10.1
 ##
 { 'struct': 'TdxGuestProperties',
   'data': { '*attributes': 'uint64',
-            '*sept-ve-disable': 'bool' } }
+            '*sept-ve-disable': 'bool',
+            '*mrconfigid': 'str',
+            '*mrowner': 'str',
+            '*mrownerconfig': 'str' } }
 
 ##
 # @ThreadContextProperties:
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index aa043acb1a88..77ddb2655c53 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -11,8 +11,10 @@
 
 #include "qemu/osdep.h"
 #include "qemu/error-report.h"
+#include "qemu/base64.h"
 #include "qapi/error.h"
 #include "qom/object_interfaces.h"
+#include "crypto/hash.h"
 
 #include "hw/i386/x86.h"
 #include "kvm_i386.h"
@@ -239,6 +241,7 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
     CPUX86State *env = &x86cpu->env;
     g_autofree struct kvm_tdx_init_vm *init_vm = NULL;
     Error *local_err = NULL;
+    size_t data_len;
     int retry = 10000;
     int r = 0;
 
@@ -250,6 +253,36 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
     init_vm = g_malloc0(sizeof(struct kvm_tdx_init_vm) +
                         sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES);
 
+    if (tdx_guest->mrconfigid) {
+        g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrconfigid,
+                              strlen(tdx_guest->mrconfigid), &data_len, errp);
+        if (!data || data_len != QCRYPTO_HASH_DIGEST_LEN_SHA384) {
+            error_setg(errp, "TDX: failed to decode mrconfigid");
+            return -1;
+        }
+        memcpy(init_vm->mrconfigid, data, data_len);
+    }
+
+    if (tdx_guest->mrowner) {
+        g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrowner,
+                              strlen(tdx_guest->mrowner), &data_len, errp);
+        if (!data || data_len != QCRYPTO_HASH_DIGEST_LEN_SHA384) {
+            error_setg(errp, "TDX: failed to decode mrowner");
+            return -1;
+        }
+        memcpy(init_vm->mrowner, data, data_len);
+    }
+
+    if (tdx_guest->mrownerconfig) {
+        g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrownerconfig,
+                            strlen(tdx_guest->mrownerconfig), &data_len, errp);
+        if (!data || data_len != QCRYPTO_HASH_DIGEST_LEN_SHA384) {
+            error_setg(errp, "TDX: failed to decode mrownerconfig");
+            return -1;
+        }
+        memcpy(init_vm->mrownerconfig, data, data_len);
+    }
+
     r = setup_td_guest_attributes(x86cpu, errp);
     if (r) {
         return r;
@@ -313,6 +346,51 @@ static void tdx_guest_set_sept_ve_disable(Object *obj, bool value, Error **errp)
     }
 }
 
+static char *tdx_guest_get_mrconfigid(Object *obj, Error **errp)
+{
+    TdxGuest *tdx = TDX_GUEST(obj);
+
+    return g_strdup(tdx->mrconfigid);
+}
+
+static void tdx_guest_set_mrconfigid(Object *obj, const char *value, Error **errp)
+{
+    TdxGuest *tdx = TDX_GUEST(obj);
+
+    g_free(tdx->mrconfigid);
+    tdx->mrconfigid = g_strdup(value);
+}
+
+static char *tdx_guest_get_mrowner(Object *obj, Error **errp)
+{
+    TdxGuest *tdx = TDX_GUEST(obj);
+
+    return g_strdup(tdx->mrowner);
+}
+
+static void tdx_guest_set_mrowner(Object *obj, const char *value, Error **errp)
+{
+    TdxGuest *tdx = TDX_GUEST(obj);
+
+    g_free(tdx->mrowner);
+    tdx->mrowner = g_strdup(value);
+}
+
+static char *tdx_guest_get_mrownerconfig(Object *obj, Error **errp)
+{
+    TdxGuest *tdx = TDX_GUEST(obj);
+
+    return g_strdup(tdx->mrownerconfig);
+}
+
+static void tdx_guest_set_mrownerconfig(Object *obj, const char *value, Error **errp)
+{
+    TdxGuest *tdx = TDX_GUEST(obj);
+
+    g_free(tdx->mrownerconfig);
+    tdx->mrownerconfig = g_strdup(value);
+}
+
 /* tdx guest */
 OBJECT_DEFINE_TYPE_WITH_INTERFACES(TdxGuest,
                                    tdx_guest,
@@ -336,6 +414,14 @@ static void tdx_guest_init(Object *obj)
     object_property_add_bool(obj, "sept-ve-disable",
                              tdx_guest_get_sept_ve_disable,
                              tdx_guest_set_sept_ve_disable);
+    object_property_add_str(obj, "mrconfigid",
+                            tdx_guest_get_mrconfigid,
+                            tdx_guest_set_mrconfigid);
+    object_property_add_str(obj, "mrowner",
+                            tdx_guest_get_mrowner, tdx_guest_set_mrowner);
+    object_property_add_str(obj, "mrownerconfig",
+                            tdx_guest_get_mrownerconfig,
+                            tdx_guest_set_mrownerconfig);
 }
 
 static void tdx_guest_finalize(Object *obj)
diff --git a/target/i386/kvm/tdx.h b/target/i386/kvm/tdx.h
index 4e2b5c61ff5b..e472b11fb0dd 100644
--- a/target/i386/kvm/tdx.h
+++ b/target/i386/kvm/tdx.h
@@ -24,6 +24,9 @@ typedef struct TdxGuest {
     bool initialized;
     uint64_t attributes;    /* TD attributes */
     uint64_t xfam;
+    char *mrconfigid;       /* base64 encoded sha348 digest */
+    char *mrowner;          /* base64 encoded sha348 digest */
+    char *mrownerconfig;    /* base64 encoded sha348 digest */
 } TdxGuest;
 
 #ifdef CONFIG_TDX
-- 
2.34.1

Re: [PATCH v8 13/55] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig

[Daniel P. Berrangé]

On Tue, Apr 01, 2025 at 09:01:23AM -0400, Xiaoyao Li wrote:
> From: Isaku Yamahata <isaku.yamahata@intel.com>
> 
> Three sha384 hash values, mrconfigid, mrowner and mrownerconfig, of a TD
> can be provided for TDX attestation. Detailed meaning of them can be
> found: https://lore.kernel.org/qemu-devel/31d6dbc1-f453-4cef-ab08-4813f4e0ff92@intel.com/
> 
> Allow user to specify those values via property mrconfigid, mrowner and
> mrownerconfig. They are all in base64 format.
> 
> example
> -object tdx-guest, \
>   mrconfigid=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
>   mrowner=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
>   mrownerconfig=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v
> 
> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
> Co-developed-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
> Changes in v8:
>  - it gets squashed into previous patch in v7. So split it out in v8;
> 
> Changes in v6:
>  - refine the doc comment of QAPI properties;
> 
> Changes in v5:
>  - refine the description of QAPI properties and add description of
>    default value when not specified;
> 
> Changes in v4:
>  - describe more of there fields in qom.json
>  - free the old value before set new value to avoid memory leak in
>    _setter(); (Daniel)
> 
> Changes in v3:
>  - use base64 encoding instread of hex-string;
> ---
>  qapi/qom.json         | 16 +++++++-
>  target/i386/kvm/tdx.c | 86 +++++++++++++++++++++++++++++++++++++++++++
>  target/i386/kvm/tdx.h |  3 ++
>  3 files changed, 104 insertions(+), 1 deletion(-)


> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
> index aa043acb1a88..77ddb2655c53 100644
> --- a/target/i386/kvm/tdx.c
> +++ b/target/i386/kvm/tdx.c
> @@ -11,8 +11,10 @@
>  
>  #include "qemu/osdep.h"
>  #include "qemu/error-report.h"
> +#include "qemu/base64.h"
>  #include "qapi/error.h"
>  #include "qom/object_interfaces.h"
> +#include "crypto/hash.h"
>  
>  #include "hw/i386/x86.h"
>  #include "kvm_i386.h"
> @@ -239,6 +241,7 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
>      CPUX86State *env = &x86cpu->env;
>      g_autofree struct kvm_tdx_init_vm *init_vm = NULL;
>      Error *local_err = NULL;
> +    size_t data_len;
>      int retry = 10000;
>      int r = 0;
>  
> @@ -250,6 +253,36 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
>      init_vm = g_malloc0(sizeof(struct kvm_tdx_init_vm) +
>                          sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES);
>  
> +    if (tdx_guest->mrconfigid) {
> +        g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrconfigid,
> +                              strlen(tdx_guest->mrconfigid), &data_len, errp);
> +        if (!data || data_len != QCRYPTO_HASH_DIGEST_LEN_SHA384) {
> +            error_setg(errp, "TDX: failed to decode mrconfigid");
> +            return -1;
> +        }

When '!data',  qbase64_decode will have already filled 'errp' with
details, so we must immediately 'return -1', as a repeated error_setg
call is an programming error.

The error_setg call should only be done in response to failing
the 'data_len' check and the message should specify the lengths
eg more like this

  if (!data) {
      return -1;
  }

  if (data_len != QCRYPTO_HASH_DIGEST_LEN_SHA384) {
      error_setg(errp, "TDX mrconfigid len %d must match SHA384 digest len %d",
                 data_len, QCRYPTO_HASH_DIGEST_LEN_SHA384)
      return -1;
  }


> +        memcpy(init_vm->mrconfigid, data, data_len);
> +    }
> +
> +    if (tdx_guest->mrowner) {
> +        g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrowner,
> +                              strlen(tdx_guest->mrowner), &data_len, errp);
> +        if (!data || data_len != QCRYPTO_HASH_DIGEST_LEN_SHA384) {
> +            error_setg(errp, "TDX: failed to decode mrowner");
> +            return -1;
> +        }
> +        memcpy(init_vm->mrowner, data, data_len);
> +    }
> +
> +    if (tdx_guest->mrownerconfig) {
> +        g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrownerconfig,
> +                            strlen(tdx_guest->mrownerconfig), &data_len, errp);
> +        if (!data || data_len != QCRYPTO_HASH_DIGEST_LEN_SHA384) {
> +            error_setg(errp, "TDX: failed to decode mrownerconfig");
> +            return -1;
> +        }
> +        memcpy(init_vm->mrownerconfig, data, data_len);
> +    }
> +
>      r = setup_td_guest_attributes(x86cpu, errp);
>      if (r) {
>          return r;

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v8 13/55] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig

[Xiaoyao Li]

On 4/2/2025 7:51 PM, Daniel P. Berrangé wrote:
> On Tue, Apr 01, 2025 at 09:01:23AM -0400, Xiaoyao Li wrote:
>> From: Isaku Yamahata <isaku.yamahata@intel.com>
>>
>> Three sha384 hash values, mrconfigid, mrowner and mrownerconfig, of a TD
>> can be provided for TDX attestation. Detailed meaning of them can be
>> found: https://lore.kernel.org/qemu-devel/31d6dbc1-f453-4cef-ab08-4813f4e0ff92@intel.com/
>>
>> Allow user to specify those values via property mrconfigid, mrowner and
>> mrownerconfig. They are all in base64 format.
>>
>> example
>> -object tdx-guest, \
>>    mrconfigid=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
>>    mrowner=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
>>    mrownerconfig=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v
>>
>> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
>> Co-developed-by: Xiaoyao Li <xiaoyao.li@intel.com>
>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>> ---
>> Changes in v8:
>>   - it gets squashed into previous patch in v7. So split it out in v8;
>>
>> Changes in v6:
>>   - refine the doc comment of QAPI properties;
>>
>> Changes in v5:
>>   - refine the description of QAPI properties and add description of
>>     default value when not specified;
>>
>> Changes in v4:
>>   - describe more of there fields in qom.json
>>   - free the old value before set new value to avoid memory leak in
>>     _setter(); (Daniel)
>>
>> Changes in v3:
>>   - use base64 encoding instread of hex-string;
>> ---
>>   qapi/qom.json         | 16 +++++++-
>>   target/i386/kvm/tdx.c | 86 +++++++++++++++++++++++++++++++++++++++++++
>>   target/i386/kvm/tdx.h |  3 ++
>>   3 files changed, 104 insertions(+), 1 deletion(-)
> 
> 
>> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
>> index aa043acb1a88..77ddb2655c53 100644
>> --- a/target/i386/kvm/tdx.c
>> +++ b/target/i386/kvm/tdx.c
>> @@ -11,8 +11,10 @@
>>   
>>   #include "qemu/osdep.h"
>>   #include "qemu/error-report.h"
>> +#include "qemu/base64.h"
>>   #include "qapi/error.h"
>>   #include "qom/object_interfaces.h"
>> +#include "crypto/hash.h"
>>   
>>   #include "hw/i386/x86.h"
>>   #include "kvm_i386.h"
>> @@ -239,6 +241,7 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
>>       CPUX86State *env = &x86cpu->env;
>>       g_autofree struct kvm_tdx_init_vm *init_vm = NULL;
>>       Error *local_err = NULL;
>> +    size_t data_len;
>>       int retry = 10000;
>>       int r = 0;
>>   
>> @@ -250,6 +253,36 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
>>       init_vm = g_malloc0(sizeof(struct kvm_tdx_init_vm) +
>>                           sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES);
>>   
>> +    if (tdx_guest->mrconfigid) {
>> +        g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrconfigid,
>> +                              strlen(tdx_guest->mrconfigid), &data_len, errp);
>> +        if (!data || data_len != QCRYPTO_HASH_DIGEST_LEN_SHA384) {
>> +            error_setg(errp, "TDX: failed to decode mrconfigid");
>> +            return -1;
>> +        }
> 
> When '!data',  qbase64_decode will have already filled 'errp' with
> details, so we must immediately 'return -1', as a repeated error_setg
> call is an programming error.
> 
> The error_setg call should only be done in response to failing
> the 'data_len' check and the message should specify the lengths
> eg more like this
> 
>    if (!data) {
>        return -1;
>    }
> 
>    if (data_len != QCRYPTO_HASH_DIGEST_LEN_SHA384) {
>        error_setg(errp, "TDX mrconfigid len %d must match SHA384 digest len %d",
>                   data_len, QCRYPTO_HASH_DIGEST_LEN_SHA384)
>        return -1;
>    }

Nice catch!

> 
>> +        memcpy(init_vm->mrconfigid, data, data_len);
>> +    }
>> +
>> +    if (tdx_guest->mrowner) {
>> +        g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrowner,
>> +                              strlen(tdx_guest->mrowner), &data_len, errp);
>> +        if (!data || data_len != QCRYPTO_HASH_DIGEST_LEN_SHA384) {
>> +            error_setg(errp, "TDX: failed to decode mrowner");
>> +            return -1;
>> +        }
>> +        memcpy(init_vm->mrowner, data, data_len);
>> +    }
>> +
>> +    if (tdx_guest->mrownerconfig) {
>> +        g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrownerconfig,
>> +                            strlen(tdx_guest->mrownerconfig), &data_len, errp);
>> +        if (!data || data_len != QCRYPTO_HASH_DIGEST_LEN_SHA384) {
>> +            error_setg(errp, "TDX: failed to decode mrownerconfig");
>> +            return -1;
>> +        }
>> +        memcpy(init_vm->mrownerconfig, data, data_len);
>> +    }
>> +
>>       r = setup_td_guest_attributes(x86cpu, errp);
>>       if (r) {
>>           return r;
> 
> With regards,
> Daniel

Re: [PATCH v8 13/55] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig

[Markus Armbruster]

Xiaoyao Li <xiaoyao.li@intel.com> writes:

> From: Isaku Yamahata <isaku.yamahata@intel.com>
>
> Three sha384 hash values, mrconfigid, mrowner and mrownerconfig, of a TD
> can be provided for TDX attestation. Detailed meaning of them can be
> found: https://lore.kernel.org/qemu-devel/31d6dbc1-f453-4cef-ab08-4813f4e0ff92@intel.com/
>
> Allow user to specify those values via property mrconfigid, mrowner and
> mrownerconfig. They are all in base64 format.
>
> example
> -object tdx-guest, \
>   mrconfigid=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
>   mrowner=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v,\
>   mrownerconfig=ASNFZ4mrze8BI0VniavN7wEjRWeJq83vASNFZ4mrze8BI0VniavN7wEjRWeJq83v
>
> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
> Co-developed-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>

[...]

> diff --git a/qapi/qom.json b/qapi/qom.json
> index f229bb07aaec..a8379bac1719 100644
> --- a/qapi/qom.json
> +++ b/qapi/qom.json
> @@ -1060,11 +1060,25 @@
>  #     pages.  Some guest OS (e.g., Linux TD guest) may require this to
>  #     be set, otherwise they refuse to boot.
>  #
> +# @mrconfigid: ID for non-owner-defined configuration of the guest TD,
> +#     e.g., run-time or OS configuration (base64 encoded SHA384 digest).
> +#     Defaults to all zeros.
> +#
> +# @mrowner: ID for the guest TD’s owner (base64 encoded SHA384 digest).
> +#     Defaults to all zeros.
> +#
> +# @mrownerconfig: ID for owner-defined configuration of the guest TD,
> +#     e.g., specific to the workload rather than the run-time or OS
> +#     (base64 encoded SHA384 digest).  Defaults to all zeros.
> +#
>  # Since: 10.1
>  ##
>  { 'struct': 'TdxGuestProperties',
>    'data': { '*attributes': 'uint64',
> -            '*sept-ve-disable': 'bool' } }
> +            '*sept-ve-disable': 'bool',
> +            '*mrconfigid': 'str',
> +            '*mrowner': 'str',
> +            '*mrownerconfig': 'str' } }
>  
>  ##

Acked-by: Markus Armbruster <armbru@redhat.com>

[...]

Re: [PATCH v8 13/55] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig

[Zhao Liu]

> diff --git a/qapi/qom.json b/qapi/qom.json
> index f229bb07aaec..a8379bac1719 100644
> --- a/qapi/qom.json
> +++ b/qapi/qom.json
> @@ -1060,11 +1060,25 @@
>  #     pages.  Some guest OS (e.g., Linux TD guest) may require this to
>  #     be set, otherwise they refuse to boot.
>  #
> +# @mrconfigid: ID for non-owner-defined configuration of the guest TD,
> +#     e.g., run-time or OS configuration (base64 encoded SHA384 digest).
> +#     Defaults to all zeros.

Maybe a typo? s/Defaults/Default/

> +#
> +# @mrowner: ID for the guest TD’s owner (base64 encoded SHA384 digest).
> +#     Defaults to all zeros.

Ditto.

> +#
> +# @mrownerconfig: ID for owner-defined configuration of the guest TD,
> +#     e.g., specific to the workload rather than the run-time or OS
> +#     (base64 encoded SHA384 digest).  Defaults to all zeros.
> +#
>  # Since: 10.1
>  ##
>  { 'struct': 'TdxGuestProperties',
>    'data': { '*attributes': 'uint64',
> -            '*sept-ve-disable': 'bool' } }
> +            '*sept-ve-disable': 'bool',
> +            '*mrconfigid': 'str',
> +            '*mrowner': 'str',
> +            '*mrownerconfig': 'str' } }
>  

Overall LGTM, (with Daniel's comment fixed,)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

Re: [PATCH v8 13/55] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig

[Xiaoyao Li]

On 4/22/2025 11:42 PM, Zhao Liu wrote:
>> diff --git a/qapi/qom.json b/qapi/qom.json
>> index f229bb07aaec..a8379bac1719 100644
>> --- a/qapi/qom.json
>> +++ b/qapi/qom.json
>> @@ -1060,11 +1060,25 @@
>>   #     pages.  Some guest OS (e.g., Linux TD guest) may require this to
>>   #     be set, otherwise they refuse to boot.
>>   #
>> +# @mrconfigid: ID for non-owner-defined configuration of the guest TD,
>> +#     e.g., run-time or OS configuration (base64 encoded SHA384 digest).
>> +#     Defaults to all zeros.
> 
> Maybe a typo? s/Defaults/Default/

(It) defaults to all zeros.

If you grep the "defaults to", you can get a lot of it.

>> +#
>> +# @mrowner: ID for the guest TD’s owner (base64 encoded SHA384 digest).
>> +#     Defaults to all zeros.
> 
> Ditto.
> 
>> +#
>> +# @mrownerconfig: ID for owner-defined configuration of the guest TD,
>> +#     e.g., specific to the workload rather than the run-time or OS
>> +#     (base64 encoded SHA384 digest).  Defaults to all zeros.
>> +#
>>   # Since: 10.1
>>   ##
>>   { 'struct': 'TdxGuestProperties',
>>     'data': { '*attributes': 'uint64',
>> -            '*sept-ve-disable': 'bool' } }
>> +            '*sept-ve-disable': 'bool',
>> +            '*mrconfigid': 'str',
>> +            '*mrowner': 'str',
>> +            '*mrownerconfig': 'str' } }
>>   
> 
> Overall LGTM, (with Daniel's comment fixed,)
> 
> Reviewed-by: Zhao Liu <zhao1.liu@intel.com>
> 
> 

Re: [PATCH v8 13/55] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig

[Zhao Liu]

On Wed, Apr 23, 2025 at 04:11:25PM +0800, Xiaoyao Li wrote:
> Date: Wed, 23 Apr 2025 16:11:25 +0800
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: Re: [PATCH v8 13/55] i386/tdx: Support user configurable
>  mrconfigid/mrowner/mrownerconfig
> 
> On 4/22/2025 11:42 PM, Zhao Liu wrote:
> > > diff --git a/qapi/qom.json b/qapi/qom.json
> > > index f229bb07aaec..a8379bac1719 100644
> > > --- a/qapi/qom.json
> > > +++ b/qapi/qom.json
> > > @@ -1060,11 +1060,25 @@
> > >   #     pages.  Some guest OS (e.g., Linux TD guest) may require this to
> > >   #     be set, otherwise they refuse to boot.
> > >   #
> > > +# @mrconfigid: ID for non-owner-defined configuration of the guest TD,
> > > +#     e.g., run-time or OS configuration (base64 encoded SHA384 digest).
> > > +#     Defaults to all zeros.
> > 
> > Maybe a typo? s/Defaults/Default/
> 
> (It) defaults to all zeros.
> 
> If you grep the "defaults to", you can get a lot of it.

The comment can be relaxed, but please try to be precise with the doc.
You can add the omitted "It".

Re: [PATCH v8 13/55] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig

[Xiaoyao Li]

On 4/23/2025 8:31 PM, Zhao Liu wrote:
> On Wed, Apr 23, 2025 at 04:11:25PM +0800, Xiaoyao Li wrote:
>> Date: Wed, 23 Apr 2025 16:11:25 +0800
>> From: Xiaoyao Li <xiaoyao.li@intel.com>
>> Subject: Re: [PATCH v8 13/55] i386/tdx: Support user configurable
>>   mrconfigid/mrowner/mrownerconfig
>>
>> On 4/22/2025 11:42 PM, Zhao Liu wrote:
>>>> diff --git a/qapi/qom.json b/qapi/qom.json
>>>> index f229bb07aaec..a8379bac1719 100644
>>>> --- a/qapi/qom.json
>>>> +++ b/qapi/qom.json
>>>> @@ -1060,11 +1060,25 @@
>>>>    #     pages.  Some guest OS (e.g., Linux TD guest) may require this to
>>>>    #     be set, otherwise they refuse to boot.
>>>>    #
>>>> +# @mrconfigid: ID for non-owner-defined configuration of the guest TD,
>>>> +#     e.g., run-time or OS configuration (base64 encoded SHA384 digest).
>>>> +#     Defaults to all zeros.
>>>
>>> Maybe a typo? s/Defaults/Default/
>>
>> (It) defaults to all zeros.
>>
>> If you grep the "defaults to", you can get a lot of it.
> 
> The comment can be relaxed, but please try to be precise with the doc.
> You can add the omitted "It".

Actually, it came from Markus[*], the QAPI maintainer.

So I would leave it to him to make the decision.

[*] https://lore.kernel.org/qemu-devel/87ttli87sw.fsf@pond.sub.org/

Re: [PATCH v8 13/55] i386/tdx: Support user configurable mrconfigid/mrowner/mrownerconfig

[Daniel P. Berrangé]

On Wed, Apr 23, 2025 at 04:11:25PM +0800, Xiaoyao Li wrote:
> On 4/22/2025 11:42 PM, Zhao Liu wrote:
> > > diff --git a/qapi/qom.json b/qapi/qom.json
> > > index f229bb07aaec..a8379bac1719 100644
> > > --- a/qapi/qom.json
> > > +++ b/qapi/qom.json
> > > @@ -1060,11 +1060,25 @@
> > >   #     pages.  Some guest OS (e.g., Linux TD guest) may require this to
> > >   #     be set, otherwise they refuse to boot.
> > >   #
> > > +# @mrconfigid: ID for non-owner-defined configuration of the guest TD,
> > > +#     e.g., run-time or OS configuration (base64 encoded SHA384 digest).
> > > +#     Defaults to all zeros.
> > 
> > Maybe a typo? s/Defaults/Default/
> 
> (It) defaults to all zeros.
> 
> If you grep the "defaults to", you can get a lot of it.

'Defaults to' is correct/acceptable terminology, IMHO, no need to change.


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

[PATCH v8 14/55] i386/tdx: Set APIC bus rate to match with what TDX module enforces

[Xiaoyao Li]

TDX advertises core crystal clock with cpuid[0x15] as 25MHz for TD
guests and it's unchangeable from VMM. As a result, TDX guest reads
the APIC timer as the same frequency, 25MHz.

While KVM's default emulated frequency for APIC bus is 1GHz, set the
APIC bus rate to match with TDX explicitly to ensure KVM provide correct
emulated APIC timer for TD guest.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
Changes in v6:
 - new patch;
---
 target/i386/kvm/tdx.c | 13 +++++++++++++
 target/i386/kvm/tdx.h |  3 +++
 2 files changed, 16 insertions(+)

diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 77ddb2655c53..1a99b677686e 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -253,6 +253,19 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
     init_vm = g_malloc0(sizeof(struct kvm_tdx_init_vm) +
                         sizeof(struct kvm_cpuid_entry2) * KVM_MAX_CPUID_ENTRIES);
 
+    if (!kvm_check_extension(kvm_state, KVM_CAP_X86_APIC_BUS_CYCLES_NS)) {
+        error_setg(errp, "KVM doesn't support KVM_CAP_X86_APIC_BUS_CYCLES_NS");
+        return -EOPNOTSUPP;
+    }
+
+    r = kvm_vm_enable_cap(kvm_state, KVM_CAP_X86_APIC_BUS_CYCLES_NS,
+                          0, TDX_APIC_BUS_CYCLES_NS);
+    if (r < 0) {
+        error_setg_errno(errp, -r,
+                         "Unable to set core crystal clock frequency to 25MHz");
+        return r;
+    }
+
     if (tdx_guest->mrconfigid) {
         g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrconfigid,
                               strlen(tdx_guest->mrconfigid), &data_len, errp);
diff --git a/target/i386/kvm/tdx.h b/target/i386/kvm/tdx.h
index e472b11fb0dd..d39e733d9fcc 100644
--- a/target/i386/kvm/tdx.h
+++ b/target/i386/kvm/tdx.h
@@ -16,6 +16,9 @@ typedef struct TdxGuestClass {
     X86ConfidentialGuestClass parent_class;
 } TdxGuestClass;
 
+/* TDX requires bus frequency 25MHz */
+#define TDX_APIC_BUS_CYCLES_NS 40
+
 typedef struct TdxGuest {
     X86ConfidentialGuest parent_obj;
 
-- 
2.34.1

Re: [PATCH v8 14/55] i386/tdx: Set APIC bus rate to match with what TDX module enforces

[Daniel P. Berrangé]

On Tue, Apr 01, 2025 at 09:01:24AM -0400, Xiaoyao Li wrote:
> TDX advertises core crystal clock with cpuid[0x15] as 25MHz for TD
> guests and it's unchangeable from VMM. As a result, TDX guest reads
> the APIC timer as the same frequency, 25MHz.

Did you mean    s/as/at/  ?

> While KVM's default emulated frequency for APIC bus is 1GHz, set the
> APIC bus rate to match with TDX explicitly to ensure KVM provide correct
> emulated APIC timer for TD guest.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
> Changes in v6:
>  - new patch;
> ---
>  target/i386/kvm/tdx.c | 13 +++++++++++++
>  target/i386/kvm/tdx.h |  3 +++
>  2 files changed, 16 insertions(+)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v8 14/55] i386/tdx: Set APIC bus rate to match with what TDX module enforces

[Xiaoyao Li]

On 4/2/2025 7:56 PM, Daniel P. Berrangé wrote:
> On Tue, Apr 01, 2025 at 09:01:24AM -0400, Xiaoyao Li wrote:
>> TDX advertises core crystal clock with cpuid[0x15] as 25MHz for TD
>> guests and it's unchangeable from VMM. As a result, TDX guest reads
>> the APIC timer as the same frequency, 25MHz.
> 
> Did you mean    s/as/at/  ?

yes, thanks for catching it!

>> While KVM's default emulated frequency for APIC bus is 1GHz, set the
>> APIC bus rate to match with TDX explicitly to ensure KVM provide correct
>> emulated APIC timer for TD guest.
>>
>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>> ---
>> Changes in v6:
>>   - new patch;
>> ---
>>   target/i386/kvm/tdx.c | 13 +++++++++++++
>>   target/i386/kvm/tdx.h |  3 +++
>>   2 files changed, 16 insertions(+)
> 
> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
> 
> 
> With regards,
> Daniel

Re: [PATCH v8 14/55] i386/tdx: Set APIC bus rate to match with what TDX module enforces

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:24AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:24 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 14/55] i386/tdx: Set APIC bus rate to match with what
>  TDX module enforces
> X-Mailer: git-send-email 2.34.1
> 
> TDX advertises core crystal clock with cpuid[0x15] as 25MHz for TD
> guests and it's unchangeable from VMM. As a result, TDX guest reads
> the APIC timer as the same frequency, 25MHz.
> 
> While KVM's default emulated frequency for APIC bus is 1GHz, set the
> APIC bus rate to match with TDX explicitly to ensure KVM provide correct
> emulated APIC timer for TD guest.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
> Changes in v6:
>  - new patch;
> ---
>  target/i386/kvm/tdx.c | 13 +++++++++++++
>  target/i386/kvm/tdx.h |  3 +++
>  2 files changed, 16 insertions(+)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 15/55] i386/tdx: Implement user specified tsc frequency

[Xiaoyao Li]

Reuse "-cpu,tsc-frequency=" to get user wanted tsc frequency and call VM
scope VM_SET_TSC_KHZ to set the tsc frequency of TD before KVM_TDX_INIT_VM.

Besides, sanity check the tsc frequency to be in the legal range and
legal granularity (required by TDX module).

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
Changes in v3:
- use @errp to report error info; (Daniel)

Changes in v1:
- Use VM scope VM_SET_TSC_KHZ to set the TSC frequency of TD since KVM
  side drop the @tsc_khz field in struct kvm_tdx_init_vm
---
 target/i386/kvm/kvm.c |  9 +++++++++
 target/i386/kvm/tdx.c | 25 +++++++++++++++++++++++++
 2 files changed, 34 insertions(+)

diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index a537699bb7df..7de5014051eb 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -869,6 +869,15 @@ static int kvm_arch_set_tsc_khz(CPUState *cs)
     int r, cur_freq;
     bool set_ioctl = false;
 
+    /*
+     * TSC of TD vcpu is immutable, it cannot be set/changed via vcpu scope
+     * VM_SET_TSC_KHZ, but only be initialized via VM scope VM_SET_TSC_KHZ
+     * before ioctl KVM_TDX_INIT_VM in tdx_pre_create_vcpu()
+     */
+    if (is_tdx_vm()) {
+        return 0;
+    }
+
     if (!env->tsc_khz) {
         return 0;
     }
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 1a99b677686e..d9e49b7444f8 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -20,6 +20,9 @@
 #include "kvm_i386.h"
 #include "tdx.h"
 
+#define TDX_MIN_TSC_FREQUENCY_KHZ   (100 * 1000)
+#define TDX_MAX_TSC_FREQUENCY_KHZ   (10 * 1000 * 1000)
+
 #define TDX_TD_ATTRIBUTES_DEBUG             BIT_ULL(0)
 #define TDX_TD_ATTRIBUTES_SEPT_VE_DISABLE   BIT_ULL(28)
 #define TDX_TD_ATTRIBUTES_PKS               BIT_ULL(30)
@@ -266,6 +269,28 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
         return r;
     }
 
+    if (env->tsc_khz && (env->tsc_khz < TDX_MIN_TSC_FREQUENCY_KHZ ||
+                         env->tsc_khz > TDX_MAX_TSC_FREQUENCY_KHZ)) {
+        error_setg(errp, "Invalid TSC %ld KHz, must specify cpu_frequency "
+                         "between [%d, %d] kHz", env->tsc_khz,
+                         TDX_MIN_TSC_FREQUENCY_KHZ, TDX_MAX_TSC_FREQUENCY_KHZ);
+       return -EINVAL;
+    }
+
+    if (env->tsc_khz % (25 * 1000)) {
+        error_setg(errp, "Invalid TSC %ld KHz, it must be multiple of 25MHz",
+                   env->tsc_khz);
+        return -EINVAL;
+    }
+
+    /* it's safe even env->tsc_khz is 0. KVM uses host's tsc_khz in this case */
+    r = kvm_vm_ioctl(kvm_state, KVM_SET_TSC_KHZ, env->tsc_khz);
+    if (r < 0) {
+        error_setg_errno(errp, -r, "Unable to set TSC frequency to %ld kHz",
+                         env->tsc_khz);
+        return r;
+    }
+
     if (tdx_guest->mrconfigid) {
         g_autofree uint8_t *data = qbase64_decode(tdx_guest->mrconfigid,
                               strlen(tdx_guest->mrconfigid), &data_len, errp);
-- 
2.34.1

Re: [PATCH v8 15/55] i386/tdx: Implement user specified tsc frequency

[Daniel P. Berrangé]

On Tue, Apr 01, 2025 at 09:01:25AM -0400, Xiaoyao Li wrote:
> Reuse "-cpu,tsc-frequency=" to get user wanted tsc frequency and call VM
> scope VM_SET_TSC_KHZ to set the tsc frequency of TD before KVM_TDX_INIT_VM.
> 
> Besides, sanity check the tsc frequency to be in the legal range and
> legal granularity (required by TDX module).
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
> Changes in v3:
> - use @errp to report error info; (Daniel)
> 
> Changes in v1:
> - Use VM scope VM_SET_TSC_KHZ to set the TSC frequency of TD since KVM
>   side drop the @tsc_khz field in struct kvm_tdx_init_vm
> ---
>  target/i386/kvm/kvm.c |  9 +++++++++
>  target/i386/kvm/tdx.c | 25 +++++++++++++++++++++++++
>  2 files changed, 34 insertions(+)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v8 15/55] i386/tdx: Implement user specified tsc frequency

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:25AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:25 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 15/55] i386/tdx: Implement user specified tsc frequency
> X-Mailer: git-send-email 2.34.1
> 
> Reuse "-cpu,tsc-frequency=" to get user wanted tsc frequency and call VM
> scope VM_SET_TSC_KHZ to set the tsc frequency of TD before KVM_TDX_INIT_VM.
> 
> Besides, sanity check the tsc frequency to be in the legal range and
> legal granularity (required by TDX module).
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
> Changes in v3:
> - use @errp to report error info; (Daniel)
> 
> Changes in v1:
> - Use VM scope VM_SET_TSC_KHZ to set the TSC frequency of TD since KVM
>   side drop the @tsc_khz field in struct kvm_tdx_init_vm
> ---
>  target/i386/kvm/kvm.c |  9 +++++++++
>  target/i386/kvm/tdx.c | 25 +++++++++++++++++++++++++
>  2 files changed, 34 insertions(+)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 16/55] i386/tdx: load TDVF for TD guest

[Xiaoyao Li]

From: Chao Peng <chao.p.peng@linux.intel.com>

TDVF(OVMF) needs to run at private memory for TD guest. TDX cannot
support pflash device since it doesn't support read-only private memory.
Thus load TDVF(OVMF) with -bios option for TDs.

Use memory_region_init_ram_guest_memfd() to allocate the MemoryRegion
for TDVF because it needs to be located at private memory.

Also store the MemoryRegion pointer of TDVF since the shared ramblock of
it can be discared after it gets copied to private ramblock.

Signed-off-by: Chao Peng <chao.p.peng@linux.intel.com>
Co-developed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
 hw/i386/x86-common.c  | 6 +++++-
 target/i386/kvm/tdx.c | 6 ++++++
 target/i386/kvm/tdx.h | 3 +++
 3 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/hw/i386/x86-common.c b/hw/i386/x86-common.c
index 1b0671c52397..b1b5f11e7396 100644
--- a/hw/i386/x86-common.c
+++ b/hw/i386/x86-common.c
@@ -44,6 +44,7 @@
 #include "standard-headers/asm-x86/bootparam.h"
 #include CONFIG_DEVICES
 #include "kvm/kvm_i386.h"
+#include "kvm/tdx.h"
 
 #ifdef CONFIG_XEN_EMU
 #include "hw/xen/xen.h"
@@ -1035,11 +1036,14 @@ void x86_bios_rom_init(X86MachineState *x86ms, const char *default_firmware,
     if (machine_require_guest_memfd(MACHINE(x86ms))) {
         memory_region_init_ram_guest_memfd(&x86ms->bios, NULL, "pc.bios",
                                            bios_size, &error_fatal);
+        if (is_tdx_vm()) {
+            tdx_set_tdvf_region(&x86ms->bios);
+        }
     } else {
         memory_region_init_ram(&x86ms->bios, NULL, "pc.bios",
                                bios_size, &error_fatal);
     }
-    if (sev_enabled()) {
+    if (sev_enabled() || is_tdx_vm()) {
         /*
          * The concept of a "reset" simply doesn't exist for
          * confidential computing guests, we have to destroy and
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index d9e49b7444f8..b7f6f3b708b0 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -137,6 +137,12 @@ static int get_tdx_capabilities(Error **errp)
     return 0;
 }
 
+void tdx_set_tdvf_region(MemoryRegion *tdvf_mr)
+{
+    assert(!tdx_guest->tdvf_mr);
+    tdx_guest->tdvf_mr = tdvf_mr;
+}
+
 static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
 {
     TdxGuest *tdx = TDX_GUEST(cgs);
diff --git a/target/i386/kvm/tdx.h b/target/i386/kvm/tdx.h
index d39e733d9fcc..b73461b8d8a3 100644
--- a/target/i386/kvm/tdx.h
+++ b/target/i386/kvm/tdx.h
@@ -30,6 +30,8 @@ typedef struct TdxGuest {
     char *mrconfigid;       /* base64 encoded sha348 digest */
     char *mrowner;          /* base64 encoded sha348 digest */
     char *mrownerconfig;    /* base64 encoded sha348 digest */
+
+    MemoryRegion *tdvf_mr;
 } TdxGuest;
 
 #ifdef CONFIG_TDX
@@ -39,5 +41,6 @@ bool is_tdx_vm(void);
 #endif /* CONFIG_TDX */
 
 int tdx_pre_create_vcpu(CPUState *cpu, Error **errp);
+void tdx_set_tdvf_region(MemoryRegion *tdvf_mr);
 
 #endif /* QEMU_I386_TDX_H */
-- 
2.34.1

Re: [PATCH v8 16/55] i386/tdx: load TDVF for TD guest

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:26AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:26 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 16/55] i386/tdx: load TDVF for TD guest
> X-Mailer: git-send-email 2.34.1
> 
> From: Chao Peng <chao.p.peng@linux.intel.com>
> 
> TDVF(OVMF) needs to run at private memory for TD guest. TDX cannot
> support pflash device since it doesn't support read-only private memory.
> Thus load TDVF(OVMF) with -bios option for TDs.
> 
> Use memory_region_init_ram_guest_memfd() to allocate the MemoryRegion
> for TDVF because it needs to be located at private memory.
> 
> Also store the MemoryRegion pointer of TDVF since the shared ramblock of
> it can be discared after it gets copied to private ramblock.
> 
> Signed-off-by: Chao Peng <chao.p.peng@linux.intel.com>
> Co-developed-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
>  hw/i386/x86-common.c  | 6 +++++-
>  target/i386/kvm/tdx.c | 6 ++++++
>  target/i386/kvm/tdx.h | 3 +++
>  3 files changed, 14 insertions(+), 1 deletion(-)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 17/55] i386/tdvf: Introduce function to parse TDVF metadata

[Xiaoyao Li]

From: Isaku Yamahata <isaku.yamahata@intel.com>

TDX VM needs to boot with its specialized firmware, Trusted Domain
Virtual Firmware (TDVF). QEMU needs to parse TDVF and map it in TD
guest memory prior to running the TDX VM.

A TDVF Metadata in TDVF image describes the structure of firmware.
QEMU refers to it to setup memory for TDVF. Introduce function
tdvf_parse_metadata() to parse the metadata from TDVF image and store
the info of each TDVF section.

TDX metadata is located by a TDX metadata offset block, which is a
GUID-ed structure. The data portion of the GUID structure contains
only an 4-byte field that is the offset of TDX metadata to the end
of firmware file.

Select X86_FW_OVMF when TDX is enable to leverage existing functions
to parse and search OVMF's GUID-ed structures.

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
Co-developed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
Changes in v8:
 - Drop the failure handling of memcpy() since it cannot fail;

Changes in v7:
 - Update license info to only use SPDX tag;
 - use g_autofree to avoid manually free;

Changes in v6:
 - Drop the the data endianness change for metadata->Length;

Changes in v1:
 - rename tdvf_parse_section_entry() to
   tdvf_parse_and_check_section_entry()

Changes in RFC v4:
 - rename TDX_METADATA_GUID to TDX_METADATA_OFFSET_GUID
---
 hw/i386/Kconfig        |   1 +
 hw/i386/meson.build    |   1 +
 hw/i386/tdvf.c         | 183 +++++++++++++++++++++++++++++++++++++++++
 include/hw/i386/tdvf.h |  38 +++++++++
 4 files changed, 223 insertions(+)
 create mode 100644 hw/i386/tdvf.c
 create mode 100644 include/hw/i386/tdvf.h

diff --git a/hw/i386/Kconfig b/hw/i386/Kconfig
index cce9521ba934..eb65bda6e071 100644
--- a/hw/i386/Kconfig
+++ b/hw/i386/Kconfig
@@ -12,6 +12,7 @@ config SGX
 
 config TDX
     bool
+    select X86_FW_OVMF
     depends on KVM
 
 config PC
diff --git a/hw/i386/meson.build b/hw/i386/meson.build
index 10bdfde27c69..3bc1da2b6eb4 100644
--- a/hw/i386/meson.build
+++ b/hw/i386/meson.build
@@ -32,6 +32,7 @@ i386_ss.add(when: 'CONFIG_PC', if_true: files(
   'port92.c'))
 i386_ss.add(when: 'CONFIG_X86_FW_OVMF', if_true: files('pc_sysfw_ovmf.c'),
                                         if_false: files('pc_sysfw_ovmf-stubs.c'))
+i386_ss.add(when: 'CONFIG_TDX', if_true: files('tdvf.c'))
 
 subdir('kvm')
 subdir('xen')
diff --git a/hw/i386/tdvf.c b/hw/i386/tdvf.c
new file mode 100644
index 000000000000..328d1b7ffdf8
--- /dev/null
+++ b/hw/i386/tdvf.c
@@ -0,0 +1,183 @@
+/*
+ * Copyright (c) 2025 Intel Corporation
+ * Author: Isaku Yamahata <isaku.yamahata at gmail.com>
+ *                        <isaku.yamahata at intel.com>
+ *         Xiaoyao Li <xiaoyao.li@intel.com>
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/error-report.h"
+
+#include "hw/i386/pc.h"
+#include "hw/i386/tdvf.h"
+#include "system/kvm.h"
+
+#define TDX_METADATA_OFFSET_GUID    "e47a6535-984a-4798-865e-4685a7bf8ec2"
+#define TDX_METADATA_VERSION        1
+#define TDVF_SIGNATURE              0x46564454 /* TDVF as little endian */
+
+typedef struct {
+    uint32_t DataOffset;
+    uint32_t RawDataSize;
+    uint64_t MemoryAddress;
+    uint64_t MemoryDataSize;
+    uint32_t Type;
+    uint32_t Attributes;
+} TdvfSectionEntry;
+
+typedef struct {
+    uint32_t Signature;
+    uint32_t Length;
+    uint32_t Version;
+    uint32_t NumberOfSectionEntries;
+    TdvfSectionEntry SectionEntries[];
+} TdvfMetadata;
+
+struct tdx_metadata_offset {
+    uint32_t offset;
+};
+
+static TdvfMetadata *tdvf_get_metadata(void *flash_ptr, int size)
+{
+    TdvfMetadata *metadata;
+    uint32_t offset = 0;
+    uint8_t *data;
+
+    if ((uint32_t) size != size) {
+        return NULL;
+    }
+
+    if (pc_system_ovmf_table_find(TDX_METADATA_OFFSET_GUID, &data, NULL)) {
+        offset = size - le32_to_cpu(((struct tdx_metadata_offset *)data)->offset);
+
+        if (offset + sizeof(*metadata) > size) {
+            return NULL;
+        }
+    } else {
+        error_report("Cannot find TDX_METADATA_OFFSET_GUID");
+        return NULL;
+    }
+
+    metadata = flash_ptr + offset;
+
+    /* Finally, verify the signature to determine if this is a TDVF image. */
+    metadata->Signature = le32_to_cpu(metadata->Signature);
+    if (metadata->Signature != TDVF_SIGNATURE) {
+        error_report("Invalid TDVF signature in metadata!");
+        return NULL;
+    }
+
+    /* Sanity check that the TDVF doesn't overlap its own metadata. */
+    metadata->Length = le32_to_cpu(metadata->Length);
+    if (offset + metadata->Length > size) {
+        return NULL;
+    }
+
+    /* Only version 1 is supported/defined. */
+    metadata->Version = le32_to_cpu(metadata->Version);
+    if (metadata->Version != TDX_METADATA_VERSION) {
+        return NULL;
+    }
+
+    return metadata;
+}
+
+static int tdvf_parse_and_check_section_entry(const TdvfSectionEntry *src,
+                                              TdxFirmwareEntry *entry)
+{
+    entry->data_offset = le32_to_cpu(src->DataOffset);
+    entry->data_len = le32_to_cpu(src->RawDataSize);
+    entry->address = le64_to_cpu(src->MemoryAddress);
+    entry->size = le64_to_cpu(src->MemoryDataSize);
+    entry->type = le32_to_cpu(src->Type);
+    entry->attributes = le32_to_cpu(src->Attributes);
+
+    /* sanity check */
+    if (entry->size < entry->data_len) {
+        error_report("Broken metadata RawDataSize 0x%x MemoryDataSize 0x%lx",
+                     entry->data_len, entry->size);
+        return -1;
+    }
+    if (!QEMU_IS_ALIGNED(entry->address, TARGET_PAGE_SIZE)) {
+        error_report("MemoryAddress 0x%lx not page aligned", entry->address);
+        return -1;
+    }
+    if (!QEMU_IS_ALIGNED(entry->size, TARGET_PAGE_SIZE)) {
+        error_report("MemoryDataSize 0x%lx not page aligned", entry->size);
+        return -1;
+    }
+
+    switch (entry->type) {
+    case TDVF_SECTION_TYPE_BFV:
+    case TDVF_SECTION_TYPE_CFV:
+        /* The sections that must be copied from firmware image to TD memory */
+        if (entry->data_len == 0) {
+            error_report("%d section with RawDataSize == 0", entry->type);
+            return -1;
+        }
+        break;
+    case TDVF_SECTION_TYPE_TD_HOB:
+    case TDVF_SECTION_TYPE_TEMP_MEM:
+        /* The sections that no need to be copied from firmware image */
+        if (entry->data_len != 0) {
+            error_report("%d section with RawDataSize 0x%x != 0",
+                         entry->type, entry->data_len);
+            return -1;
+        }
+        break;
+    default:
+        error_report("TDVF contains unsupported section type %d", entry->type);
+        return -1;
+    }
+
+    return 0;
+}
+
+int tdvf_parse_metadata(TdxFirmware *fw, void *flash_ptr, int size)
+{
+    g_autofree TdvfSectionEntry *sections = NULL;
+    TdvfMetadata *metadata;
+    ssize_t entries_size;
+    int i;
+
+    metadata = tdvf_get_metadata(flash_ptr, size);
+    if (!metadata) {
+        return -EINVAL;
+    }
+
+    /* load and parse metadata entries */
+    fw->nr_entries = le32_to_cpu(metadata->NumberOfSectionEntries);
+    if (fw->nr_entries < 2) {
+        error_report("Invalid number of fw entries (%u) in TDVF Metadata",
+                     fw->nr_entries);
+        return -EINVAL;
+    }
+
+    entries_size = fw->nr_entries * sizeof(TdvfSectionEntry);
+    if (metadata->Length != sizeof(*metadata) + entries_size) {
+        error_report("TDVF metadata len (0x%x) mismatch, expected (0x%x)",
+                     metadata->Length,
+                     (uint32_t)(sizeof(*metadata) + entries_size));
+        return -EINVAL;
+    }
+
+    fw->entries = g_new(TdxFirmwareEntry, fw->nr_entries);
+    sections = g_new(TdvfSectionEntry, fw->nr_entries);
+
+    memcpy(sections, (void *)metadata + sizeof(*metadata), entries_size);
+
+    for (i = 0; i < fw->nr_entries; i++) {
+        if (tdvf_parse_and_check_section_entry(&sections[i], &fw->entries[i])) {
+            goto err;
+        }
+    }
+
+    return 0;
+
+err:
+    fw->entries = 0;
+    g_free(fw->entries);
+    return -EINVAL;
+}
diff --git a/include/hw/i386/tdvf.h b/include/hw/i386/tdvf.h
new file mode 100644
index 000000000000..7ebcac42a36c
--- /dev/null
+++ b/include/hw/i386/tdvf.h
@@ -0,0 +1,38 @@
+/*
+ * Copyright (c) 2025 Intel Corporation
+ * Author: Isaku Yamahata <isaku.yamahata at gmail.com>
+ *                        <isaku.yamahata at intel.com>
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#ifndef HW_I386_TDVF_H
+#define HW_I386_TDVF_H
+
+#include "qemu/osdep.h"
+
+#define TDVF_SECTION_TYPE_BFV               0
+#define TDVF_SECTION_TYPE_CFV               1
+#define TDVF_SECTION_TYPE_TD_HOB            2
+#define TDVF_SECTION_TYPE_TEMP_MEM          3
+
+#define TDVF_SECTION_ATTRIBUTES_MR_EXTEND   (1U << 0)
+#define TDVF_SECTION_ATTRIBUTES_PAGE_AUG    (1U << 1)
+
+typedef struct TdxFirmwareEntry {
+    uint32_t data_offset;
+    uint32_t data_len;
+    uint64_t address;
+    uint64_t size;
+    uint32_t type;
+    uint32_t attributes;
+} TdxFirmwareEntry;
+
+typedef struct TdxFirmware {
+    uint32_t nr_entries;
+    TdxFirmwareEntry *entries;
+} TdxFirmware;
+
+int tdvf_parse_metadata(TdxFirmware *fw, void *flash_ptr, int size);
+
+#endif /* HW_I386_TDVF_H */
-- 
2.34.1

Re: [PATCH v8 17/55] i386/tdvf: Introduce function to parse TDVF metadata

[Daniel P. Berrangé]

On Tue, Apr 01, 2025 at 09:01:27AM -0400, Xiaoyao Li wrote:
> From: Isaku Yamahata <isaku.yamahata@intel.com>
> 
> TDX VM needs to boot with its specialized firmware, Trusted Domain
> Virtual Firmware (TDVF). QEMU needs to parse TDVF and map it in TD
> guest memory prior to running the TDX VM.
> 
> A TDVF Metadata in TDVF image describes the structure of firmware.
> QEMU refers to it to setup memory for TDVF. Introduce function
> tdvf_parse_metadata() to parse the metadata from TDVF image and store
> the info of each TDVF section.
> 
> TDX metadata is located by a TDX metadata offset block, which is a
> GUID-ed structure. The data portion of the GUID structure contains
> only an 4-byte field that is the offset of TDX metadata to the end
> of firmware file.
> 
> Select X86_FW_OVMF when TDX is enable to leverage existing functions
> to parse and search OVMF's GUID-ed structures.
> 
> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
> Co-developed-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
> Changes in v8:
>  - Drop the failure handling of memcpy() since it cannot fail;
> 
> Changes in v7:
>  - Update license info to only use SPDX tag;
>  - use g_autofree to avoid manually free;
> 
> Changes in v6:
>  - Drop the the data endianness change for metadata->Length;
> 
> Changes in v1:
>  - rename tdvf_parse_section_entry() to
>    tdvf_parse_and_check_section_entry()
> 
> Changes in RFC v4:
>  - rename TDX_METADATA_GUID to TDX_METADATA_OFFSET_GUID
> ---
>  hw/i386/Kconfig        |   1 +
>  hw/i386/meson.build    |   1 +
>  hw/i386/tdvf.c         | 183 +++++++++++++++++++++++++++++++++++++++++
>  include/hw/i386/tdvf.h |  38 +++++++++
>  4 files changed, 223 insertions(+)
>  create mode 100644 hw/i386/tdvf.c
>  create mode 100644 include/hw/i386/tdvf.h
> 
> diff --git a/hw/i386/Kconfig b/hw/i386/Kconfig
> index cce9521ba934..eb65bda6e071 100644
> --- a/hw/i386/Kconfig
> +++ b/hw/i386/Kconfig
> @@ -12,6 +12,7 @@ config SGX
>  
>  config TDX
>      bool
> +    select X86_FW_OVMF
>      depends on KVM
>  
>  config PC
> diff --git a/hw/i386/meson.build b/hw/i386/meson.build
> index 10bdfde27c69..3bc1da2b6eb4 100644
> --- a/hw/i386/meson.build
> +++ b/hw/i386/meson.build
> @@ -32,6 +32,7 @@ i386_ss.add(when: 'CONFIG_PC', if_true: files(
>    'port92.c'))
>  i386_ss.add(when: 'CONFIG_X86_FW_OVMF', if_true: files('pc_sysfw_ovmf.c'),
>                                          if_false: files('pc_sysfw_ovmf-stubs.c'))
> +i386_ss.add(when: 'CONFIG_TDX', if_true: files('tdvf.c'))
>  
>  subdir('kvm')
>  subdir('xen')
> diff --git a/hw/i386/tdvf.c b/hw/i386/tdvf.c
> new file mode 100644
> index 000000000000..328d1b7ffdf8
> --- /dev/null
> +++ b/hw/i386/tdvf.c
> @@ -0,0 +1,183 @@
> +/*
> + * Copyright (c) 2025 Intel Corporation
> + * Author: Isaku Yamahata <isaku.yamahata at gmail.com>
> + *                        <isaku.yamahata at intel.com>
> + *         Xiaoyao Li <xiaoyao.li@intel.com>
> + *
> + * SPDX-License-Identifier: GPL-2.0-or-later
> + */
> +
> +#include "qemu/osdep.h"
> +#include "qemu/error-report.h"
> +
> +#include "hw/i386/pc.h"
> +#include "hw/i386/tdvf.h"
> +#include "system/kvm.h"
> +
> +#define TDX_METADATA_OFFSET_GUID    "e47a6535-984a-4798-865e-4685a7bf8ec2"
> +#define TDX_METADATA_VERSION        1
> +#define TDVF_SIGNATURE              0x46564454 /* TDVF as little endian */
> +
> +typedef struct {
> +    uint32_t DataOffset;
> +    uint32_t RawDataSize;
> +    uint64_t MemoryAddress;
> +    uint64_t MemoryDataSize;
> +    uint32_t Type;
> +    uint32_t Attributes;
> +} TdvfSectionEntry;
> +
> +typedef struct {
> +    uint32_t Signature;
> +    uint32_t Length;
> +    uint32_t Version;
> +    uint32_t NumberOfSectionEntries;
> +    TdvfSectionEntry SectionEntries[];
> +} TdvfMetadata;

struct field names starting with an initial capital is
not the usual QEMU code style. Can this be all initial
lowercase, with capital just for word separation.


> +
> +struct tdx_metadata_offset {
> +    uint32_t offset;
> +};
> +

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v8 17/55] i386/tdvf: Introduce function to parse TDVF metadata

[Xiaoyao Li]

On 4/2/2025 8:08 PM, Daniel P. Berrangé wrote:
> On Tue, Apr 01, 2025 at 09:01:27AM -0400, Xiaoyao Li wrote:
>> From: Isaku Yamahata <isaku.yamahata@intel.com>
>>
>> TDX VM needs to boot with its specialized firmware, Trusted Domain
>> Virtual Firmware (TDVF). QEMU needs to parse TDVF and map it in TD
>> guest memory prior to running the TDX VM.
>>
>> A TDVF Metadata in TDVF image describes the structure of firmware.
>> QEMU refers to it to setup memory for TDVF. Introduce function
>> tdvf_parse_metadata() to parse the metadata from TDVF image and store
>> the info of each TDVF section.
>>
>> TDX metadata is located by a TDX metadata offset block, which is a
>> GUID-ed structure. The data portion of the GUID structure contains
>> only an 4-byte field that is the offset of TDX metadata to the end
>> of firmware file.
>>
>> Select X86_FW_OVMF when TDX is enable to leverage existing functions
>> to parse and search OVMF's GUID-ed structures.
>>
>> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
>> Co-developed-by: Xiaoyao Li <xiaoyao.li@intel.com>
>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
>> ---
>> Changes in v8:
>>   - Drop the failure handling of memcpy() since it cannot fail;
>>
>> Changes in v7:
>>   - Update license info to only use SPDX tag;
>>   - use g_autofree to avoid manually free;
>>
>> Changes in v6:
>>   - Drop the the data endianness change for metadata->Length;
>>
>> Changes in v1:
>>   - rename tdvf_parse_section_entry() to
>>     tdvf_parse_and_check_section_entry()
>>
>> Changes in RFC v4:
>>   - rename TDX_METADATA_GUID to TDX_METADATA_OFFSET_GUID
>> ---
>>   hw/i386/Kconfig        |   1 +
>>   hw/i386/meson.build    |   1 +
>>   hw/i386/tdvf.c         | 183 +++++++++++++++++++++++++++++++++++++++++
>>   include/hw/i386/tdvf.h |  38 +++++++++
>>   4 files changed, 223 insertions(+)
>>   create mode 100644 hw/i386/tdvf.c
>>   create mode 100644 include/hw/i386/tdvf.h
>>
>> diff --git a/hw/i386/Kconfig b/hw/i386/Kconfig
>> index cce9521ba934..eb65bda6e071 100644
>> --- a/hw/i386/Kconfig
>> +++ b/hw/i386/Kconfig
>> @@ -12,6 +12,7 @@ config SGX
>>   
>>   config TDX
>>       bool
>> +    select X86_FW_OVMF
>>       depends on KVM
>>   
>>   config PC
>> diff --git a/hw/i386/meson.build b/hw/i386/meson.build
>> index 10bdfde27c69..3bc1da2b6eb4 100644
>> --- a/hw/i386/meson.build
>> +++ b/hw/i386/meson.build
>> @@ -32,6 +32,7 @@ i386_ss.add(when: 'CONFIG_PC', if_true: files(
>>     'port92.c'))
>>   i386_ss.add(when: 'CONFIG_X86_FW_OVMF', if_true: files('pc_sysfw_ovmf.c'),
>>                                           if_false: files('pc_sysfw_ovmf-stubs.c'))
>> +i386_ss.add(when: 'CONFIG_TDX', if_true: files('tdvf.c'))
>>   
>>   subdir('kvm')
>>   subdir('xen')
>> diff --git a/hw/i386/tdvf.c b/hw/i386/tdvf.c
>> new file mode 100644
>> index 000000000000..328d1b7ffdf8
>> --- /dev/null
>> +++ b/hw/i386/tdvf.c
>> @@ -0,0 +1,183 @@
>> +/*
>> + * Copyright (c) 2025 Intel Corporation
>> + * Author: Isaku Yamahata <isaku.yamahata at gmail.com>
>> + *                        <isaku.yamahata at intel.com>
>> + *         Xiaoyao Li <xiaoyao.li@intel.com>
>> + *
>> + * SPDX-License-Identifier: GPL-2.0-or-later
>> + */
>> +
>> +#include "qemu/osdep.h"
>> +#include "qemu/error-report.h"
>> +
>> +#include "hw/i386/pc.h"
>> +#include "hw/i386/tdvf.h"
>> +#include "system/kvm.h"
>> +
>> +#define TDX_METADATA_OFFSET_GUID    "e47a6535-984a-4798-865e-4685a7bf8ec2"
>> +#define TDX_METADATA_VERSION        1
>> +#define TDVF_SIGNATURE              0x46564454 /* TDVF as little endian */
>> +
>> +typedef struct {
>> +    uint32_t DataOffset;
>> +    uint32_t RawDataSize;
>> +    uint64_t MemoryAddress;
>> +    uint64_t MemoryDataSize;
>> +    uint32_t Type;
>> +    uint32_t Attributes;
>> +} TdvfSectionEntry;
>> +
>> +typedef struct {
>> +    uint32_t Signature;
>> +    uint32_t Length;
>> +    uint32_t Version;
>> +    uint32_t NumberOfSectionEntries;
>> +    TdvfSectionEntry SectionEntries[];
>> +} TdvfMetadata;
> 
> struct field names starting with an initial capital is
> not the usual QEMU code style. Can this be all initial
> lowercase, with capital just for word separation.

I think Isaku used such names because they are defined in TDVF Design 
Guide spec[1].

QEMU's internal data structure for metadata are below, which follows 
QEMU coding style.

	typedef struct TdxFirmwareEntry {
	    uint32_t data_offset;
	    uint32_t data_len;
	    uint64_t address;
	    uint64_t size;
	    uint32_t type;
	    uint32_t attributes;
	} TdxFirmwareEntry;

	typedef struct TdxFirmware {
	    uint32_t nr_entries;
	    TdxFirmwareEntry *entries;
	} TdxFirmware;

So if no strong preference, I would keep it as-is that the raw struct 
read from TDVF keeps the name convention of TDVF Design Guide spec and 
internal data struct uses QEMU's convention.

[1] https://cdrdv2.intel.com/v1/dl/getContent/733585

> 
>> +
>> +struct tdx_metadata_offset {
>> +    uint32_t offset;
>> +};
>> +
> 
> With regards,
> Daniel

Re: [PATCH v8 17/55] i386/tdvf: Introduce function to parse TDVF metadata

[Zhao Liu]

> > struct field names starting with an initial capital is
> > not the usual QEMU code style. Can this be all initial
> > lowercase, with capital just for word separation.
> 
> I think Isaku used such names because they are defined in TDVF Design Guide
> spec[1].
> 
> QEMU's internal data structure for metadata are below, which follows QEMU
> coding style.
> 
> 	typedef struct TdxFirmwareEntry {
> 	    uint32_t data_offset;
> 	    uint32_t data_len;
> 	    uint64_t address;
> 	    uint64_t size;
> 	    uint32_t type;
> 	    uint32_t attributes;
> 	} TdxFirmwareEntry;
> 
> 	typedef struct TdxFirmware {
> 	    uint32_t nr_entries;
> 	    TdxFirmwareEntry *entries;
> 	} TdxFirmware;
> 
> So if no strong preference, I would keep it as-is that the raw struct read
> from TDVF keeps the name convention of TDVF Design Guide spec and internal
> data struct uses QEMU's convention.
> 
> [1] https://cdrdv2.intel.com/v1/dl/getContent/733585

Or you can add a simple comemnt about "the raw struct read from TDVF
keeps the name convention of TDVF Design Guide spec", to remind other
people that there is no need to clean up the style here. I have no
objections; up to you.

Re: [PATCH v8 17/55] i386/tdvf: Introduce function to parse TDVF metadata

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:27AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:27 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 17/55] i386/tdvf: Introduce function to parse TDVF
>  metadata
> X-Mailer: git-send-email 2.34.1
> 
> From: Isaku Yamahata <isaku.yamahata@intel.com>
> 
> TDX VM needs to boot with its specialized firmware, Trusted Domain
> Virtual Firmware (TDVF). QEMU needs to parse TDVF and map it in TD
> guest memory prior to running the TDX VM.
> 
> A TDVF Metadata in TDVF image describes the structure of firmware.
> QEMU refers to it to setup memory for TDVF. Introduce function
> tdvf_parse_metadata() to parse the metadata from TDVF image and store
> the info of each TDVF section.
> 
> TDX metadata is located by a TDX metadata offset block, which is a
> GUID-ed structure. The data portion of the GUID structure contains
> only an 4-byte field that is the offset of TDX metadata to the end
> of firmware file.
> 
> Select X86_FW_OVMF when TDX is enable to leverage existing functions
> to parse and search OVMF's GUID-ed structures.
> 
> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
> Co-developed-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
> Changes in v8:
>  - Drop the failure handling of memcpy() since it cannot fail;
> 
> Changes in v7:
>  - Update license info to only use SPDX tag;
>  - use g_autofree to avoid manually free;
> 
> Changes in v6:
>  - Drop the the data endianness change for metadata->Length;
> 
> Changes in v1:
>  - rename tdvf_parse_section_entry() to
>    tdvf_parse_and_check_section_entry()
> 
> Changes in RFC v4:
>  - rename TDX_METADATA_GUID to TDX_METADATA_OFFSET_GUID
> ---
>  hw/i386/Kconfig        |   1 +
>  hw/i386/meson.build    |   1 +
>  hw/i386/tdvf.c         | 183 +++++++++++++++++++++++++++++++++++++++++
>  include/hw/i386/tdvf.h |  38 +++++++++
>  4 files changed, 223 insertions(+)
>  create mode 100644 hw/i386/tdvf.c
>  create mode 100644 include/hw/i386/tdvf.h

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 18/55] i386/tdx: Parse TDVF metadata for TDX VM

[Xiaoyao Li]

After TDVF is loaded to bios MemoryRegion, it needs parse TDVF metadata.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
 hw/i386/pc_sysfw.c         | 7 +++++++
 target/i386/kvm/tdx-stub.c | 5 +++++
 target/i386/kvm/tdx.c      | 5 +++++
 target/i386/kvm/tdx.h      | 3 +++
 4 files changed, 20 insertions(+)

diff --git a/hw/i386/pc_sysfw.c b/hw/i386/pc_sysfw.c
index 1eeb58ab37f9..821396c16e91 100644
--- a/hw/i386/pc_sysfw.c
+++ b/hw/i386/pc_sysfw.c
@@ -37,6 +37,7 @@
 #include "hw/block/flash.h"
 #include "system/kvm.h"
 #include "target/i386/sev.h"
+#include "kvm/tdx.h"
 
 #define FLASH_SECTOR_SIZE 4096
 
@@ -280,5 +281,11 @@ void x86_firmware_configure(hwaddr gpa, void *ptr, int size)
         }
 
         sev_encrypt_flash(gpa, ptr, size, &error_fatal);
+    } else if (is_tdx_vm()) {
+        ret = tdx_parse_tdvf(ptr, size);
+        if (ret) {
+            error_report("failed to parse TDVF for TDX VM");
+            exit(1);
+        }
     }
 }
diff --git a/target/i386/kvm/tdx-stub.c b/target/i386/kvm/tdx-stub.c
index 2344433594ea..7748b6d0a446 100644
--- a/target/i386/kvm/tdx-stub.c
+++ b/target/i386/kvm/tdx-stub.c
@@ -8,3 +8,8 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
 {
     return -EINVAL;
 }
+
+int tdx_parse_tdvf(void *flash_ptr, int size)
+{
+    return -EINVAL;
+}
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index b7f6f3b708b0..8d3475c6752c 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -372,6 +372,11 @@ int tdx_pre_create_vcpu(CPUState *cpu, Error **errp)
     return 0;
 }
 
+int tdx_parse_tdvf(void *flash_ptr, int size)
+{
+    return tdvf_parse_metadata(&tdx_guest->tdvf, flash_ptr, size);
+}
+
 static bool tdx_guest_get_sept_ve_disable(Object *obj, Error **errp)
 {
     TdxGuest *tdx = TDX_GUEST(obj);
diff --git a/target/i386/kvm/tdx.h b/target/i386/kvm/tdx.h
index b73461b8d8a3..28a03c2a7b82 100644
--- a/target/i386/kvm/tdx.h
+++ b/target/i386/kvm/tdx.h
@@ -8,6 +8,7 @@
 #endif
 
 #include "confidential-guest.h"
+#include "hw/i386/tdvf.h"
 
 #define TYPE_TDX_GUEST "tdx-guest"
 #define TDX_GUEST(obj)  OBJECT_CHECK(TdxGuest, (obj), TYPE_TDX_GUEST)
@@ -32,6 +33,7 @@ typedef struct TdxGuest {
     char *mrownerconfig;    /* base64 encoded sha348 digest */
 
     MemoryRegion *tdvf_mr;
+    TdxFirmware tdvf;
 } TdxGuest;
 
 #ifdef CONFIG_TDX
@@ -42,5 +44,6 @@ bool is_tdx_vm(void);
 
 int tdx_pre_create_vcpu(CPUState *cpu, Error **errp);
 void tdx_set_tdvf_region(MemoryRegion *tdvf_mr);
+int tdx_parse_tdvf(void *flash_ptr, int size);
 
 #endif /* QEMU_I386_TDX_H */
-- 
2.34.1

Re: [PATCH v8 18/55] i386/tdx: Parse TDVF metadata for TDX VM

[Daniel P. Berrangé]

On Tue, Apr 01, 2025 at 09:01:28AM -0400, Xiaoyao Li wrote:
> After TDVF is loaded to bios MemoryRegion, it needs parse TDVF metadata.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
>  hw/i386/pc_sysfw.c         | 7 +++++++
>  target/i386/kvm/tdx-stub.c | 5 +++++
>  target/i386/kvm/tdx.c      | 5 +++++
>  target/i386/kvm/tdx.h      | 3 +++
>  4 files changed, 20 insertions(+)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v8 18/55] i386/tdx: Parse TDVF metadata for TDX VM

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:28AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:28 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 18/55] i386/tdx: Parse TDVF metadata for TDX VM
> X-Mailer: git-send-email 2.34.1
> 
> After TDVF is loaded to bios MemoryRegion, it needs parse TDVF metadata.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
>  hw/i386/pc_sysfw.c         | 7 +++++++
>  target/i386/kvm/tdx-stub.c | 5 +++++
>  target/i386/kvm/tdx.c      | 5 +++++
>  target/i386/kvm/tdx.h      | 3 +++
>  4 files changed, 20 insertions(+)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 19/55] i386/tdx: Don't initialize pc.rom for TDX VMs

[Xiaoyao Li]

For TDX, the address below 1MB are entirely general RAM. No need to
initialize pc.rom memory region for TDs.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
This is more as a workaround of the issue that for q35 machine type, the
real memslot update (which requires memslot deletion )for pc.rom happens
after tdx_init_memory_region. It leads to the private memory ADD'ed
before get lost. I haven't work out a good solution to resolve the
order issue. So just skip the pc.rom setup to avoid memslot deletion.
---
 hw/i386/pc.c | 29 ++++++++++++++++-------------
 1 file changed, 16 insertions(+), 13 deletions(-)

diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 01d0581f62a3..bcbbea235645 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -43,6 +43,7 @@
 #include "system/xen.h"
 #include "system/reset.h"
 #include "kvm/kvm_i386.h"
+#include "kvm/tdx.h"
 #include "hw/xen/xen.h"
 #include "qobject/qlist.h"
 #include "qemu/error-report.h"
@@ -972,21 +973,23 @@ void pc_memory_init(PCMachineState *pcms,
     /* Initialize PC system firmware */
     pc_system_firmware_init(pcms, rom_memory);
 
-    option_rom_mr = g_malloc(sizeof(*option_rom_mr));
-    if (machine_require_guest_memfd(machine)) {
-        memory_region_init_ram_guest_memfd(option_rom_mr, NULL, "pc.rom",
-                                           PC_ROM_SIZE, &error_fatal);
-    } else {
-        memory_region_init_ram(option_rom_mr, NULL, "pc.rom", PC_ROM_SIZE,
-                               &error_fatal);
-        if (pcmc->pci_enabled) {
-            memory_region_set_readonly(option_rom_mr, true);
+    if (!is_tdx_vm()) {
+        option_rom_mr = g_malloc(sizeof(*option_rom_mr));
+        if (machine_require_guest_memfd(machine)) {
+            memory_region_init_ram_guest_memfd(option_rom_mr, NULL, "pc.rom",
+                                            PC_ROM_SIZE, &error_fatal);
+        } else {
+            memory_region_init_ram(option_rom_mr, NULL, "pc.rom", PC_ROM_SIZE,
+                                &error_fatal);
+            if (pcmc->pci_enabled) {
+                memory_region_set_readonly(option_rom_mr, true);
+            }
         }
+        memory_region_add_subregion_overlap(rom_memory,
+                                            PC_ROM_MIN_VGA,
+                                            option_rom_mr,
+                                            1);
     }
-    memory_region_add_subregion_overlap(rom_memory,
-                                        PC_ROM_MIN_VGA,
-                                        option_rom_mr,
-                                        1);
 
     fw_cfg = fw_cfg_arch_create(machine,
                                 x86ms->boot_cpus, x86ms->apic_id_limit);
-- 
2.34.1

Re: [PATCH v8 19/55] i386/tdx: Don't initialize pc.rom for TDX VMs

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:29AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:29 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 19/55] i386/tdx: Don't initialize pc.rom for TDX VMs
> X-Mailer: git-send-email 2.34.1
> 
> For TDX, the address below 1MB are entirely general RAM. No need to
> initialize pc.rom memory region for TDs.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
> This is more as a workaround of the issue that for q35 machine type, the
> real memslot update (which requires memslot deletion )for pc.rom happens
> after tdx_init_memory_region. It leads to the private memory ADD'ed
> before get lost. I haven't work out a good solution to resolve the
> order issue. So just skip the pc.rom setup to avoid memslot deletion.
> ---
>  hw/i386/pc.c | 29 ++++++++++++++++-------------
>  1 file changed, 16 insertions(+), 13 deletions(-)

Fine for me,

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 20/55] i386/tdx: Track mem_ptr for each firmware entry of TDVF

[Xiaoyao Li]

For each TDVF sections, QEMU needs to copy the content to guest
private memory via KVM API (KVM_TDX_INIT_MEM_REGION).

Introduce a field @mem_ptr for TdxFirmwareEntry to track the memory
pointer of each TDVF sections. So that QEMU can add/copy them to guest
private memory later.

TDVF sections can be classified into two groups:
 - Firmware itself, e.g., TDVF BFV and CFV, that located separately from
   guest RAM. Its memory pointer is the bios pointer.

 - Sections located at guest RAM, e.g., TEMP_MEM and TD_HOB.
   mmap a new memory range for them.

Register a machine_init_done callback to do the stuff.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
Changes in v8:
- Remove the duplicated header include;
- Add error handling for qemu_ram_mmap() failure;
---
 hw/i386/tdvf.c         |  1 +
 include/hw/i386/tdvf.h |  7 +++++++
 target/i386/kvm/tdx.c  | 37 +++++++++++++++++++++++++++++++++++++
 3 files changed, 45 insertions(+)

diff --git a/hw/i386/tdvf.c b/hw/i386/tdvf.c
index 328d1b7ffdf8..840c392c44ab 100644
--- a/hw/i386/tdvf.c
+++ b/hw/i386/tdvf.c
@@ -174,6 +174,7 @@ int tdvf_parse_metadata(TdxFirmware *fw, void *flash_ptr, int size)
         }
     }
 
+    fw->mem_ptr = flash_ptr;
     return 0;
 
 err:
diff --git a/include/hw/i386/tdvf.h b/include/hw/i386/tdvf.h
index 7ebcac42a36c..e75c8d1acc68 100644
--- a/include/hw/i386/tdvf.h
+++ b/include/hw/i386/tdvf.h
@@ -26,13 +26,20 @@ typedef struct TdxFirmwareEntry {
     uint64_t size;
     uint32_t type;
     uint32_t attributes;
+
+    void *mem_ptr;
 } TdxFirmwareEntry;
 
 typedef struct TdxFirmware {
+    void *mem_ptr;
+
     uint32_t nr_entries;
     TdxFirmwareEntry *entries;
 } TdxFirmware;
 
+#define for_each_tdx_fw_entry(fw, e)    \
+    for (e = (fw)->entries; e != (fw)->entries + (fw)->nr_entries; e++)
+
 int tdvf_parse_metadata(TdxFirmware *fw, void *flash_ptr, int size);
 
 #endif /* HW_I386_TDVF_H */
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 8d3475c6752c..0885b99cfb88 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -12,10 +12,13 @@
 #include "qemu/osdep.h"
 #include "qemu/error-report.h"
 #include "qemu/base64.h"
+#include "qemu/mmap-alloc.h"
 #include "qapi/error.h"
 #include "qom/object_interfaces.h"
 #include "crypto/hash.h"
+#include "system/system.h"
 
+#include "hw/i386/tdvf.h"
 #include "hw/i386/x86.h"
 #include "kvm_i386.h"
 #include "tdx.h"
@@ -143,6 +146,38 @@ void tdx_set_tdvf_region(MemoryRegion *tdvf_mr)
     tdx_guest->tdvf_mr = tdvf_mr;
 }
 
+static void tdx_finalize_vm(Notifier *notifier, void *unused)
+{
+    TdxFirmware *tdvf = &tdx_guest->tdvf;
+    TdxFirmwareEntry *entry;
+
+    for_each_tdx_fw_entry(tdvf, entry) {
+        switch (entry->type) {
+        case TDVF_SECTION_TYPE_BFV:
+        case TDVF_SECTION_TYPE_CFV:
+            entry->mem_ptr = tdvf->mem_ptr + entry->data_offset;
+            break;
+        case TDVF_SECTION_TYPE_TD_HOB:
+        case TDVF_SECTION_TYPE_TEMP_MEM:
+            entry->mem_ptr = qemu_ram_mmap(-1, entry->size,
+                                           qemu_real_host_page_size(), 0, 0);
+            if (entry->mem_ptr == MAP_FAILED) {
+                error_report("Failed to mmap memory for TDVF section %d",
+                             entry->type);
+                exit(1);
+            }
+            break;
+        default:
+            error_report("Unsupported TDVF section %d", entry->type);
+            exit(1);
+        }
+    }
+}
+
+static Notifier tdx_machine_done_notify = {
+    .notify = tdx_finalize_vm,
+};
+
 static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
 {
     TdxGuest *tdx = TDX_GUEST(cgs);
@@ -157,6 +192,8 @@ static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
         }
     }
 
+    qemu_add_machine_init_done_notifier(&tdx_machine_done_notify);
+
     tdx_guest = tdx;
     return 0;
 }
-- 
2.34.1

Re: [PATCH v8 20/55] i386/tdx: Track mem_ptr for each firmware entry of TDVF

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:30AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:30 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 20/55] i386/tdx: Track mem_ptr for each firmware entry
>  of TDVF
> X-Mailer: git-send-email 2.34.1
> 
> For each TDVF sections, QEMU needs to copy the content to guest
> private memory via KVM API (KVM_TDX_INIT_MEM_REGION).
> 
> Introduce a field @mem_ptr for TdxFirmwareEntry to track the memory
> pointer of each TDVF sections. So that QEMU can add/copy them to guest
> private memory later.
> 
> TDVF sections can be classified into two groups:
>  - Firmware itself, e.g., TDVF BFV and CFV, that located separately from
>    guest RAM. Its memory pointer is the bios pointer.
> 
>  - Sections located at guest RAM, e.g., TEMP_MEM and TD_HOB.
>    mmap a new memory range for them.
> 
> Register a machine_init_done callback to do the stuff.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
> Changes in v8:
> - Remove the duplicated header include;
> - Add error handling for qemu_ram_mmap() failure;
> ---
>  hw/i386/tdvf.c         |  1 +
>  include/hw/i386/tdvf.h |  7 +++++++
>  target/i386/kvm/tdx.c  | 37 +++++++++++++++++++++++++++++++++++++
>  3 files changed, 45 insertions(+)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 21/55] i386/tdx: Track RAM entries for TDX VM

[Xiaoyao Li]

The RAM of TDX VM can be classified into two types:

 - TDX_RAM_UNACCEPTED: default type of TDX memory, which needs to be
   accepted by TDX guest before it can be used and will be all-zeros
   after being accepted.

 - TDX_RAM_ADDED: the RAM that is ADD'ed to TD guest before running, and
   can be used directly. E.g., TD HOB and TEMP MEM that needed by TDVF.

Maintain TdxRamEntries[] which grabs the initial RAM info from e820 table
and mark each RAM range as default type TDX_RAM_UNACCEPTED.

Then turn the range of TD HOB and TEMP MEM to TDX_RAM_ADDED since these
ranges will be ADD'ed before TD runs and no need to be accepted runtime.

The TdxRamEntries[] are later used to setup the memory TD resource HOB
that passes memory info from QEMU to TDVF.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
Changes in v3:
- use enum TdxRamType in struct TdxRamEntry; (Isaku)
- Fix the indention; (Daniel)

Changes in v1:
  - simplify the algorithm of tdx_accept_ram_range() (Suggested-by: Gerd Hoffman)
    (1) Change the existing entry to cover the accepted ram range.
    (2) If there is room before the accepted ram range add a
	TDX_RAM_UNACCEPTED entry for that.
    (3) If there is room after the accepted ram range add a
	TDX_RAM_UNACCEPTED entry for that.
---
 target/i386/kvm/tdx.c | 111 ++++++++++++++++++++++++++++++++++++++++++
 target/i386/kvm/tdx.h |  14 ++++++
 2 files changed, 125 insertions(+)

diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 0885b99cfb88..072ffcdfb41d 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -18,6 +18,7 @@
 #include "crypto/hash.h"
 #include "system/system.h"
 
+#include "hw/i386/e820_memory_layout.h"
 #include "hw/i386/tdvf.h"
 #include "hw/i386/x86.h"
 #include "kvm_i386.h"
@@ -146,11 +147,117 @@ void tdx_set_tdvf_region(MemoryRegion *tdvf_mr)
     tdx_guest->tdvf_mr = tdvf_mr;
 }
 
+static void tdx_add_ram_entry(uint64_t address, uint64_t length,
+                              enum TdxRamType type)
+{
+    uint32_t nr_entries = tdx_guest->nr_ram_entries;
+    tdx_guest->ram_entries = g_renew(TdxRamEntry, tdx_guest->ram_entries,
+                                     nr_entries + 1);
+
+    tdx_guest->ram_entries[nr_entries].address = address;
+    tdx_guest->ram_entries[nr_entries].length = length;
+    tdx_guest->ram_entries[nr_entries].type = type;
+    tdx_guest->nr_ram_entries++;
+}
+
+static int tdx_accept_ram_range(uint64_t address, uint64_t length)
+{
+    uint64_t head_start, tail_start, head_length, tail_length;
+    uint64_t tmp_address, tmp_length;
+    TdxRamEntry *e;
+    int i;
+
+    for (i = 0; i < tdx_guest->nr_ram_entries; i++) {
+        e = &tdx_guest->ram_entries[i];
+
+        if (address + length <= e->address ||
+            e->address + e->length <= address) {
+            continue;
+        }
+
+        /*
+         * The to-be-accepted ram range must be fully contained by one
+         * RAM entry.
+         */
+        if (e->address > address ||
+            e->address + e->length < address + length) {
+            return -EINVAL;
+        }
+
+        if (e->type == TDX_RAM_ADDED) {
+            return -EINVAL;
+        }
+
+        break;
+    }
+
+    if (i == tdx_guest->nr_ram_entries) {
+        return -1;
+    }
+
+    tmp_address = e->address;
+    tmp_length = e->length;
+
+    e->address = address;
+    e->length = length;
+    e->type = TDX_RAM_ADDED;
+
+    head_length = address - tmp_address;
+    if (head_length > 0) {
+        head_start = tmp_address;
+        tdx_add_ram_entry(head_start, head_length, TDX_RAM_UNACCEPTED);
+    }
+
+    tail_start = address + length;
+    if (tail_start < tmp_address + tmp_length) {
+        tail_length = tmp_address + tmp_length - tail_start;
+        tdx_add_ram_entry(tail_start, tail_length, TDX_RAM_UNACCEPTED);
+    }
+
+    return 0;
+}
+
+static int tdx_ram_entry_compare(const void *lhs_, const void* rhs_)
+{
+    const TdxRamEntry *lhs = lhs_;
+    const TdxRamEntry *rhs = rhs_;
+
+    if (lhs->address == rhs->address) {
+        return 0;
+    }
+    if (le64_to_cpu(lhs->address) > le64_to_cpu(rhs->address)) {
+        return 1;
+    }
+    return -1;
+}
+
+static void tdx_init_ram_entries(void)
+{
+    unsigned i, j, nr_e820_entries;
+
+    nr_e820_entries = e820_get_table(NULL);
+    tdx_guest->ram_entries = g_new(TdxRamEntry, nr_e820_entries);
+
+    for (i = 0, j = 0; i < nr_e820_entries; i++) {
+        uint64_t addr, len;
+
+        if (e820_get_entry(i, E820_RAM, &addr, &len)) {
+            tdx_guest->ram_entries[j].address = addr;
+            tdx_guest->ram_entries[j].length = len;
+            tdx_guest->ram_entries[j].type = TDX_RAM_UNACCEPTED;
+            j++;
+        }
+    }
+    tdx_guest->nr_ram_entries = j;
+}
+
 static void tdx_finalize_vm(Notifier *notifier, void *unused)
 {
     TdxFirmware *tdvf = &tdx_guest->tdvf;
     TdxFirmwareEntry *entry;
 
+    tdx_init_ram_entries();
+
     for_each_tdx_fw_entry(tdvf, entry) {
         switch (entry->type) {
         case TDVF_SECTION_TYPE_BFV:
@@ -166,12 +273,16 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused)
                              entry->type);
                 exit(1);
             }
+            tdx_accept_ram_range(entry->address, entry->size);
             break;
         default:
             error_report("Unsupported TDVF section %d", entry->type);
             exit(1);
         }
     }
+
+    qsort(tdx_guest->ram_entries, tdx_guest->nr_ram_entries,
+          sizeof(TdxRamEntry), &tdx_ram_entry_compare);
 }
 
 static Notifier tdx_machine_done_notify = {
diff --git a/target/i386/kvm/tdx.h b/target/i386/kvm/tdx.h
index 28a03c2a7b82..36a7400e7480 100644
--- a/target/i386/kvm/tdx.h
+++ b/target/i386/kvm/tdx.h
@@ -20,6 +20,17 @@ typedef struct TdxGuestClass {
 /* TDX requires bus frequency 25MHz */
 #define TDX_APIC_BUS_CYCLES_NS 40
 
+enum TdxRamType {
+    TDX_RAM_UNACCEPTED,
+    TDX_RAM_ADDED,
+};
+
+typedef struct TdxRamEntry {
+    uint64_t address;
+    uint64_t length;
+    enum TdxRamType type;
+} TdxRamEntry;
+
 typedef struct TdxGuest {
     X86ConfidentialGuest parent_obj;
 
@@ -34,6 +45,9 @@ typedef struct TdxGuest {
 
     MemoryRegion *tdvf_mr;
     TdxFirmware tdvf;
+
+    uint32_t nr_ram_entries;
+    TdxRamEntry *ram_entries;
 } TdxGuest;
 
 #ifdef CONFIG_TDX
-- 
2.34.1

Re: [PATCH v8 21/55] i386/tdx: Track RAM entries for TDX VM

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:31AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:31 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 21/55] i386/tdx: Track RAM entries for TDX VM
> X-Mailer: git-send-email 2.34.1
> 
> The RAM of TDX VM can be classified into two types:
> 
>  - TDX_RAM_UNACCEPTED: default type of TDX memory, which needs to be
>    accepted by TDX guest before it can be used and will be all-zeros
>    after being accepted.
> 
>  - TDX_RAM_ADDED: the RAM that is ADD'ed to TD guest before running, and
>    can be used directly. E.g., TD HOB and TEMP MEM that needed by TDVF.
> 
> Maintain TdxRamEntries[] which grabs the initial RAM info from e820 table
> and mark each RAM range as default type TDX_RAM_UNACCEPTED.
> 
> Then turn the range of TD HOB and TEMP MEM to TDX_RAM_ADDED since these
> ranges will be ADD'ed before TD runs and no need to be accepted runtime.
> 
> The TdxRamEntries[] are later used to setup the memory TD resource HOB
> that passes memory info from QEMU to TDVF.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> ---

[snip]

>  static void tdx_finalize_vm(Notifier *notifier, void *unused)
>  {
>      TdxFirmware *tdvf = &tdx_guest->tdvf;
>      TdxFirmwareEntry *entry;
>  
> +    tdx_init_ram_entries();
> +
>      for_each_tdx_fw_entry(tdvf, entry) {
>          switch (entry->type) {
>          case TDVF_SECTION_TYPE_BFV:
> @@ -166,12 +273,16 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused)
>                               entry->type);
>                  exit(1);
>              }
> +            tdx_accept_ram_range(entry->address, entry->size);

It's better to catch & report the failure case here.

>              break;

Others LGTM,

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 22/55] headers: Add definitions from UEFI spec for volumes, resources, etc...

[Xiaoyao Li]

Add UEFI definitions for literals, enums, structs, GUIDs, etc... that
will be used by TDX to build the UEFI Hand-Off Block (HOB) that is passed
to the Trusted Domain Virtual Firmware (TDVF).

All values come from the UEFI specification [1], PI spec [2] and TDVF
design guide[3].

[1] UEFI Specification v2.1.0 https://uefi.org/sites/default/files/resources/UEFI_Spec_2_10_Aug29.pdf
[2] UEFI PI spec v1.8 https://uefi.org/sites/default/files/resources/UEFI_PI_Spec_1_8_March3.pdf
[3] https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-virtual-firmware-design-guide-rev-1.pdf

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
Changes in v7:
 - use SPDX tag;
---
 include/standard-headers/uefi/uefi.h | 187 +++++++++++++++++++++++++++
 1 file changed, 187 insertions(+)
 create mode 100644 include/standard-headers/uefi/uefi.h

diff --git a/include/standard-headers/uefi/uefi.h b/include/standard-headers/uefi/uefi.h
new file mode 100644
index 000000000000..5256349ec0b6
--- /dev/null
+++ b/include/standard-headers/uefi/uefi.h
@@ -0,0 +1,187 @@
+/*
+ * Copyright (C) 2025 Intel Corporation
+ *
+ * Author: Isaku Yamahata <isaku.yamahata at gmail.com>
+ *                        <isaku.yamahata at intel.com>
+ *         Xiaoyao Li <xiaoyao.li@intel.com>
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#ifndef HW_I386_UEFI_H
+#define HW_I386_UEFI_H
+
+/***************************************************************************/
+/*
+ * basic EFI definitions
+ * supplemented with UEFI Specification Version 2.8 (Errata A)
+ * released February 2020
+ */
+/* UEFI integer is little endian */
+
+typedef struct {
+    uint32_t Data1;
+    uint16_t Data2;
+    uint16_t Data3;
+    uint8_t Data4[8];
+} EFI_GUID;
+
+typedef enum {
+    EfiReservedMemoryType,
+    EfiLoaderCode,
+    EfiLoaderData,
+    EfiBootServicesCode,
+    EfiBootServicesData,
+    EfiRuntimeServicesCode,
+    EfiRuntimeServicesData,
+    EfiConventionalMemory,
+    EfiUnusableMemory,
+    EfiACPIReclaimMemory,
+    EfiACPIMemoryNVS,
+    EfiMemoryMappedIO,
+    EfiMemoryMappedIOPortSpace,
+    EfiPalCode,
+    EfiPersistentMemory,
+    EfiUnacceptedMemoryType,
+    EfiMaxMemoryType
+} EFI_MEMORY_TYPE;
+
+#define EFI_HOB_HANDOFF_TABLE_VERSION 0x0009
+
+#define EFI_HOB_TYPE_HANDOFF              0x0001
+#define EFI_HOB_TYPE_MEMORY_ALLOCATION    0x0002
+#define EFI_HOB_TYPE_RESOURCE_DESCRIPTOR  0x0003
+#define EFI_HOB_TYPE_GUID_EXTENSION       0x0004
+#define EFI_HOB_TYPE_FV                   0x0005
+#define EFI_HOB_TYPE_CPU                  0x0006
+#define EFI_HOB_TYPE_MEMORY_POOL          0x0007
+#define EFI_HOB_TYPE_FV2                  0x0009
+#define EFI_HOB_TYPE_LOAD_PEIM_UNUSED     0x000A
+#define EFI_HOB_TYPE_UEFI_CAPSULE         0x000B
+#define EFI_HOB_TYPE_FV3                  0x000C
+#define EFI_HOB_TYPE_UNUSED               0xFFFE
+#define EFI_HOB_TYPE_END_OF_HOB_LIST      0xFFFF
+
+typedef struct {
+    uint16_t HobType;
+    uint16_t HobLength;
+    uint32_t Reserved;
+} EFI_HOB_GENERIC_HEADER;
+
+typedef uint64_t EFI_PHYSICAL_ADDRESS;
+typedef uint32_t EFI_BOOT_MODE;
+
+typedef struct {
+    EFI_HOB_GENERIC_HEADER Header;
+    uint32_t Version;
+    EFI_BOOT_MODE BootMode;
+    EFI_PHYSICAL_ADDRESS EfiMemoryTop;
+    EFI_PHYSICAL_ADDRESS EfiMemoryBottom;
+    EFI_PHYSICAL_ADDRESS EfiFreeMemoryTop;
+    EFI_PHYSICAL_ADDRESS EfiFreeMemoryBottom;
+    EFI_PHYSICAL_ADDRESS EfiEndOfHobList;
+} EFI_HOB_HANDOFF_INFO_TABLE;
+
+#define EFI_RESOURCE_SYSTEM_MEMORY          0x00000000
+#define EFI_RESOURCE_MEMORY_MAPPED_IO       0x00000001
+#define EFI_RESOURCE_IO                     0x00000002
+#define EFI_RESOURCE_FIRMWARE_DEVICE        0x00000003
+#define EFI_RESOURCE_MEMORY_MAPPED_IO_PORT  0x00000004
+#define EFI_RESOURCE_MEMORY_RESERVED        0x00000005
+#define EFI_RESOURCE_IO_RESERVED            0x00000006
+#define EFI_RESOURCE_MEMORY_UNACCEPTED      0x00000007
+#define EFI_RESOURCE_MAX_MEMORY_TYPE        0x00000008
+
+#define EFI_RESOURCE_ATTRIBUTE_PRESENT                  0x00000001
+#define EFI_RESOURCE_ATTRIBUTE_INITIALIZED              0x00000002
+#define EFI_RESOURCE_ATTRIBUTE_TESTED                   0x00000004
+#define EFI_RESOURCE_ATTRIBUTE_SINGLE_BIT_ECC           0x00000008
+#define EFI_RESOURCE_ATTRIBUTE_MULTIPLE_BIT_ECC         0x00000010
+#define EFI_RESOURCE_ATTRIBUTE_ECC_RESERVED_1           0x00000020
+#define EFI_RESOURCE_ATTRIBUTE_ECC_RESERVED_2           0x00000040
+#define EFI_RESOURCE_ATTRIBUTE_READ_PROTECTED           0x00000080
+#define EFI_RESOURCE_ATTRIBUTE_WRITE_PROTECTED          0x00000100
+#define EFI_RESOURCE_ATTRIBUTE_EXECUTION_PROTECTED      0x00000200
+#define EFI_RESOURCE_ATTRIBUTE_UNCACHEABLE              0x00000400
+#define EFI_RESOURCE_ATTRIBUTE_WRITE_COMBINEABLE        0x00000800
+#define EFI_RESOURCE_ATTRIBUTE_WRITE_THROUGH_CACHEABLE  0x00001000
+#define EFI_RESOURCE_ATTRIBUTE_WRITE_BACK_CACHEABLE     0x00002000
+#define EFI_RESOURCE_ATTRIBUTE_16_BIT_IO                0x00004000
+#define EFI_RESOURCE_ATTRIBUTE_32_BIT_IO                0x00008000
+#define EFI_RESOURCE_ATTRIBUTE_64_BIT_IO                0x00010000
+#define EFI_RESOURCE_ATTRIBUTE_UNCACHED_EXPORTED        0x00020000
+#define EFI_RESOURCE_ATTRIBUTE_READ_ONLY_PROTECTED      0x00040000
+#define EFI_RESOURCE_ATTRIBUTE_READ_ONLY_PROTECTABLE    0x00080000
+#define EFI_RESOURCE_ATTRIBUTE_READ_PROTECTABLE         0x00100000
+#define EFI_RESOURCE_ATTRIBUTE_WRITE_PROTECTABLE        0x00200000
+#define EFI_RESOURCE_ATTRIBUTE_EXECUTION_PROTECTABLE    0x00400000
+#define EFI_RESOURCE_ATTRIBUTE_PERSISTENT               0x00800000
+#define EFI_RESOURCE_ATTRIBUTE_PERSISTABLE              0x01000000
+#define EFI_RESOURCE_ATTRIBUTE_MORE_RELIABLE            0x02000000
+
+typedef uint32_t EFI_RESOURCE_TYPE;
+typedef uint32_t EFI_RESOURCE_ATTRIBUTE_TYPE;
+
+typedef struct {
+    EFI_HOB_GENERIC_HEADER Header;
+    EFI_GUID Owner;
+    EFI_RESOURCE_TYPE ResourceType;
+    EFI_RESOURCE_ATTRIBUTE_TYPE ResourceAttribute;
+    EFI_PHYSICAL_ADDRESS PhysicalStart;
+    uint64_t ResourceLength;
+} EFI_HOB_RESOURCE_DESCRIPTOR;
+
+typedef struct {
+    EFI_HOB_GENERIC_HEADER Header;
+    EFI_GUID Name;
+
+    /* guid specific data follows */
+} EFI_HOB_GUID_TYPE;
+
+typedef struct {
+    EFI_HOB_GENERIC_HEADER Header;
+    EFI_PHYSICAL_ADDRESS BaseAddress;
+    uint64_t Length;
+} EFI_HOB_FIRMWARE_VOLUME;
+
+typedef struct {
+    EFI_HOB_GENERIC_HEADER Header;
+    EFI_PHYSICAL_ADDRESS BaseAddress;
+    uint64_t Length;
+    EFI_GUID FvName;
+    EFI_GUID FileName;
+} EFI_HOB_FIRMWARE_VOLUME2;
+
+typedef struct {
+    EFI_HOB_GENERIC_HEADER Header;
+    EFI_PHYSICAL_ADDRESS BaseAddress;
+    uint64_t Length;
+    uint32_t AuthenticationStatus;
+    bool ExtractedFv;
+    EFI_GUID FvName;
+    EFI_GUID FileName;
+} EFI_HOB_FIRMWARE_VOLUME3;
+
+typedef struct {
+    EFI_HOB_GENERIC_HEADER Header;
+    uint8_t SizeOfMemorySpace;
+    uint8_t SizeOfIoSpace;
+    uint8_t Reserved[6];
+} EFI_HOB_CPU;
+
+typedef struct {
+    EFI_HOB_GENERIC_HEADER Header;
+} EFI_HOB_MEMORY_POOL;
+
+typedef struct {
+    EFI_HOB_GENERIC_HEADER Header;
+
+    EFI_PHYSICAL_ADDRESS BaseAddress;
+    uint64_t Length;
+} EFI_HOB_UEFI_CAPSULE;
+
+#define EFI_HOB_OWNER_ZERO                                      \
+    ((EFI_GUID){ 0x00000000, 0x0000, 0x0000,                    \
+        { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 } })
+
+#endif
-- 
2.34.1

Re: [PATCH v8 22/55] headers: Add definitions from UEFI spec for volumes, resources, etc...

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:32AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:32 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 22/55] headers: Add definitions from UEFI spec for
>  volumes, resources, etc...
> X-Mailer: git-send-email 2.34.1
> 
> Add UEFI definitions for literals, enums, structs, GUIDs, etc... that
> will be used by TDX to build the UEFI Hand-Off Block (HOB) that is passed
> to the Trusted Domain Virtual Firmware (TDVF).
> 
> All values come from the UEFI specification [1], PI spec [2] and TDVF
> design guide[3].
> 
> [1] UEFI Specification v2.1.0 https://uefi.org/sites/default/files/resources/UEFI_Spec_2_10_Aug29.pdf
> [2] UEFI PI spec v1.8 https://uefi.org/sites/default/files/resources/UEFI_PI_Spec_1_8_March3.pdf
> [3] https://software.intel.com/content/dam/develop/external/us/en/documents/tdx-virtual-firmware-design-guide-rev-1.pdf
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
> Changes in v7:
>  - use SPDX tag;
> ---
>  include/standard-headers/uefi/uefi.h | 187 +++++++++++++++++++++++++++
>  1 file changed, 187 insertions(+)
>  create mode 100644 include/standard-headers/uefi/uefi.h

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 23/55] i386/tdx: Setup the TD HOB list

[Xiaoyao Li]

The TD HOB list is used to pass the information from VMM to TDVF. The TD
HOB must include PHIT HOB and Resource Descriptor HOB. More details can
be found in TDVF specification and PI specification.

Build the TD HOB in TDX's machine_init_done callback.

Co-developed-by: Isaku Yamahata <isaku.yamahata@intel.com>
Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
Co-developed-by: Sean Christopherson <sean.j.christopherson@intel.com>
Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
Changes in v7:
 - use SPDX tag for license info;
 - clean up the included headers;

Changes in v1:
 - drop the code of adding mmio resources since OVMF prepares all the
   MMIO hob itself.
---
 hw/i386/meson.build   |   2 +-
 hw/i386/tdvf-hob.c    | 130 ++++++++++++++++++++++++++++++++++++++++++
 hw/i386/tdvf-hob.h    |  26 +++++++++
 target/i386/kvm/tdx.c |  16 ++++++
 4 files changed, 173 insertions(+), 1 deletion(-)
 create mode 100644 hw/i386/tdvf-hob.c
 create mode 100644 hw/i386/tdvf-hob.h

diff --git a/hw/i386/meson.build b/hw/i386/meson.build
index 3bc1da2b6eb4..7896f348cff8 100644
--- a/hw/i386/meson.build
+++ b/hw/i386/meson.build
@@ -32,7 +32,7 @@ i386_ss.add(when: 'CONFIG_PC', if_true: files(
   'port92.c'))
 i386_ss.add(when: 'CONFIG_X86_FW_OVMF', if_true: files('pc_sysfw_ovmf.c'),
                                         if_false: files('pc_sysfw_ovmf-stubs.c'))
-i386_ss.add(when: 'CONFIG_TDX', if_true: files('tdvf.c'))
+i386_ss.add(when: 'CONFIG_TDX', if_true: files('tdvf.c', 'tdvf-hob.c'))
 
 subdir('kvm')
 subdir('xen')
diff --git a/hw/i386/tdvf-hob.c b/hw/i386/tdvf-hob.c
new file mode 100644
index 000000000000..782b3d157879
--- /dev/null
+++ b/hw/i386/tdvf-hob.c
@@ -0,0 +1,130 @@
+/*
+ * Copyright (c) 2025 Intel Corporation
+ * Author: Isaku Yamahata <isaku.yamahata at gmail.com>
+ *                        <isaku.yamahata at intel.com>
+ *         Xiaoyao Li <xiaoyao.li@intel.com>
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/error-report.h"
+#include "standard-headers/uefi/uefi.h"
+#include "hw/pci/pcie_host.h"
+#include "tdvf-hob.h"
+
+typedef struct TdvfHob {
+    hwaddr hob_addr;
+    void *ptr;
+    int size;
+
+    /* working area */
+    void *current;
+    void *end;
+} TdvfHob;
+
+static uint64_t tdvf_current_guest_addr(const TdvfHob *hob)
+{
+    return hob->hob_addr + (hob->current - hob->ptr);
+}
+
+static void tdvf_align(TdvfHob *hob, size_t align)
+{
+    hob->current = QEMU_ALIGN_PTR_UP(hob->current, align);
+}
+
+static void *tdvf_get_area(TdvfHob *hob, uint64_t size)
+{
+    void *ret;
+
+    if (hob->current + size > hob->end) {
+        error_report("TD_HOB overrun, size = 0x%" PRIx64, size);
+        exit(1);
+    }
+
+    ret = hob->current;
+    hob->current += size;
+    tdvf_align(hob, 8);
+    return ret;
+}
+
+static void tdvf_hob_add_memory_resources(TdxGuest *tdx, TdvfHob *hob)
+{
+    EFI_HOB_RESOURCE_DESCRIPTOR *region;
+    EFI_RESOURCE_ATTRIBUTE_TYPE attr;
+    EFI_RESOURCE_TYPE resource_type;
+
+    TdxRamEntry *e;
+    int i;
+
+    for (i = 0; i < tdx->nr_ram_entries; i++) {
+        e = &tdx->ram_entries[i];
+
+        if (e->type == TDX_RAM_UNACCEPTED) {
+            resource_type = EFI_RESOURCE_MEMORY_UNACCEPTED;
+            attr = EFI_RESOURCE_ATTRIBUTE_TDVF_UNACCEPTED;
+        } else if (e->type == TDX_RAM_ADDED) {
+            resource_type = EFI_RESOURCE_SYSTEM_MEMORY;
+            attr = EFI_RESOURCE_ATTRIBUTE_TDVF_PRIVATE;
+        } else {
+            error_report("unknown TDX_RAM_ENTRY type %d", e->type);
+            exit(1);
+        }
+
+        region = tdvf_get_area(hob, sizeof(*region));
+        *region = (EFI_HOB_RESOURCE_DESCRIPTOR) {
+            .Header = {
+                .HobType = EFI_HOB_TYPE_RESOURCE_DESCRIPTOR,
+                .HobLength = cpu_to_le16(sizeof(*region)),
+                .Reserved = cpu_to_le32(0),
+            },
+            .Owner = EFI_HOB_OWNER_ZERO,
+            .ResourceType = cpu_to_le32(resource_type),
+            .ResourceAttribute = cpu_to_le32(attr),
+            .PhysicalStart = cpu_to_le64(e->address),
+            .ResourceLength = cpu_to_le64(e->length),
+        };
+    }
+}
+
+void tdvf_hob_create(TdxGuest *tdx, TdxFirmwareEntry *td_hob)
+{
+    TdvfHob hob = {
+        .hob_addr = td_hob->address,
+        .size = td_hob->size,
+        .ptr = td_hob->mem_ptr,
+
+        .current = td_hob->mem_ptr,
+        .end = td_hob->mem_ptr + td_hob->size,
+    };
+
+    EFI_HOB_GENERIC_HEADER *last_hob;
+    EFI_HOB_HANDOFF_INFO_TABLE *hit;
+
+    /* Note, Efi{Free}Memory{Bottom,Top} are ignored, leave 'em zeroed. */
+    hit = tdvf_get_area(&hob, sizeof(*hit));
+    *hit = (EFI_HOB_HANDOFF_INFO_TABLE) {
+        .Header = {
+            .HobType = EFI_HOB_TYPE_HANDOFF,
+            .HobLength = cpu_to_le16(sizeof(*hit)),
+            .Reserved = cpu_to_le32(0),
+        },
+        .Version = cpu_to_le32(EFI_HOB_HANDOFF_TABLE_VERSION),
+        .BootMode = cpu_to_le32(0),
+        .EfiMemoryTop = cpu_to_le64(0),
+        .EfiMemoryBottom = cpu_to_le64(0),
+        .EfiFreeMemoryTop = cpu_to_le64(0),
+        .EfiFreeMemoryBottom = cpu_to_le64(0),
+        .EfiEndOfHobList = cpu_to_le64(0), /* initialized later */
+    };
+
+    tdvf_hob_add_memory_resources(tdx, &hob);
+
+    last_hob = tdvf_get_area(&hob, sizeof(*last_hob));
+    *last_hob =  (EFI_HOB_GENERIC_HEADER) {
+        .HobType = EFI_HOB_TYPE_END_OF_HOB_LIST,
+        .HobLength = cpu_to_le16(sizeof(*last_hob)),
+        .Reserved = cpu_to_le32(0),
+    };
+    hit->EfiEndOfHobList = tdvf_current_guest_addr(&hob);
+}
diff --git a/hw/i386/tdvf-hob.h b/hw/i386/tdvf-hob.h
new file mode 100644
index 000000000000..4fc6a3740a57
--- /dev/null
+++ b/hw/i386/tdvf-hob.h
@@ -0,0 +1,26 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#ifndef HW_I386_TD_HOB_H
+#define HW_I386_TD_HOB_H
+
+#include "hw/i386/tdvf.h"
+#include "target/i386/kvm/tdx.h"
+
+void tdvf_hob_create(TdxGuest *tdx, TdxFirmwareEntry *td_hob);
+
+#define EFI_RESOURCE_ATTRIBUTE_TDVF_PRIVATE     \
+    (EFI_RESOURCE_ATTRIBUTE_PRESENT |           \
+     EFI_RESOURCE_ATTRIBUTE_INITIALIZED |       \
+     EFI_RESOURCE_ATTRIBUTE_TESTED)
+
+#define EFI_RESOURCE_ATTRIBUTE_TDVF_UNACCEPTED  \
+    (EFI_RESOURCE_ATTRIBUTE_PRESENT |           \
+     EFI_RESOURCE_ATTRIBUTE_INITIALIZED |       \
+     EFI_RESOURCE_ATTRIBUTE_TESTED)
+
+#define EFI_RESOURCE_ATTRIBUTE_TDVF_MMIO        \
+    (EFI_RESOURCE_ATTRIBUTE_PRESENT     |       \
+     EFI_RESOURCE_ATTRIBUTE_INITIALIZED |       \
+     EFI_RESOURCE_ATTRIBUTE_UNCACHEABLE)
+
+#endif
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 072ffcdfb41d..2e8d32285d9a 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -21,6 +21,7 @@
 #include "hw/i386/e820_memory_layout.h"
 #include "hw/i386/tdvf.h"
 #include "hw/i386/x86.h"
+#include "hw/i386/tdvf-hob.h"
 #include "kvm_i386.h"
 #include "tdx.h"
 
@@ -147,6 +148,19 @@ void tdx_set_tdvf_region(MemoryRegion *tdvf_mr)
     tdx_guest->tdvf_mr = tdvf_mr;
 }
 
+static TdxFirmwareEntry *tdx_get_hob_entry(TdxGuest *tdx)
+{
+    TdxFirmwareEntry *entry;
+
+    for_each_tdx_fw_entry(&tdx->tdvf, entry) {
+        if (entry->type == TDVF_SECTION_TYPE_TD_HOB) {
+            return entry;
+        }
+    }
+    error_report("TDVF metadata doesn't specify TD_HOB location.");
+    exit(1);
+}
+
 static void tdx_add_ram_entry(uint64_t address, uint64_t length,
                               enum TdxRamType type)
 {
@@ -283,6 +297,8 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused)
 
     qsort(tdx_guest->ram_entries, tdx_guest->nr_ram_entries,
           sizeof(TdxRamEntry), &tdx_ram_entry_compare);
+
+    tdvf_hob_create(tdx_guest, tdx_get_hob_entry(tdx_guest));
 }
 
 static Notifier tdx_machine_done_notify = {
-- 
2.34.1

Re: [PATCH v8 23/55] i386/tdx: Setup the TD HOB list

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:33AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:33 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 23/55] i386/tdx: Setup the TD HOB list
> X-Mailer: git-send-email 2.34.1
> 
> The TD HOB list is used to pass the information from VMM to TDVF. The TD
> HOB must include PHIT HOB and Resource Descriptor HOB. More details can
> be found in TDVF specification and PI specification.
> 
> Build the TD HOB in TDX's machine_init_done callback.
> 
> Co-developed-by: Isaku Yamahata <isaku.yamahata@intel.com>
> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
> Co-developed-by: Sean Christopherson <sean.j.christopherson@intel.com>
> Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
> Changes in v7:
>  - use SPDX tag for license info;
>  - clean up the included headers;
> 
> Changes in v1:
>  - drop the code of adding mmio resources since OVMF prepares all the
>    MMIO hob itself.
> ---
>  hw/i386/meson.build   |   2 +-
>  hw/i386/tdvf-hob.c    | 130 ++++++++++++++++++++++++++++++++++++++++++
>  hw/i386/tdvf-hob.h    |  26 +++++++++
>  target/i386/kvm/tdx.c |  16 ++++++
>  4 files changed, 173 insertions(+), 1 deletion(-)
>  create mode 100644 hw/i386/tdvf-hob.c
>  create mode 100644 hw/i386/tdvf-hob.h

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 24/55] i386/tdx: Add TDVF memory via KVM_TDX_INIT_MEM_REGION

[Xiaoyao Li]

From: Isaku Yamahata <isaku.yamahata@intel.com>

TDVF firmware (CODE and VARS) needs to be copied to TD's private
memory via KVM_TDX_INIT_MEM_REGION, as well as TD HOB and TEMP memory.

If the TDVF section has TDVF_SECTION_ATTRIBUTES_MR_EXTEND set in the
flag, calling KVM_TDX_EXTEND_MEMORY to extend the measurement.

After populating the TDVF memory, the original image located in shared
ramblock can be discarded.

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
Changes in v6:
 - switch back to use KVM_TDX_INIT_MEM_REGION according to KVM's change;
---
 target/i386/kvm/tdx.c | 42 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)

diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 2e8d32285d9a..4cf0929d22aa 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -17,6 +17,7 @@
 #include "qom/object_interfaces.h"
 #include "crypto/hash.h"
 #include "system/system.h"
+#include "exec/ramblock.h"
 
 #include "hw/i386/e820_memory_layout.h"
 #include "hw/i386/tdvf.h"
@@ -269,6 +270,9 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused)
 {
     TdxFirmware *tdvf = &tdx_guest->tdvf;
     TdxFirmwareEntry *entry;
+    RAMBlock *ram_block;
+    Error *local_err = NULL;
+    int r;
 
     tdx_init_ram_entries();
 
@@ -299,6 +303,44 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused)
           sizeof(TdxRamEntry), &tdx_ram_entry_compare);
 
     tdvf_hob_create(tdx_guest, tdx_get_hob_entry(tdx_guest));
+
+    for_each_tdx_fw_entry(tdvf, entry) {
+        struct kvm_tdx_init_mem_region region;
+        uint32_t flags;
+
+        region = (struct kvm_tdx_init_mem_region) {
+            .source_addr = (uint64_t)entry->mem_ptr,
+            .gpa = entry->address,
+            .nr_pages = entry->size >> 12,
+        };
+
+        flags = entry->attributes & TDVF_SECTION_ATTRIBUTES_MR_EXTEND ?
+                KVM_TDX_MEASURE_MEMORY_REGION : 0;
+
+        do {
+            error_free(local_err);
+            local_err = NULL;
+            r = tdx_vcpu_ioctl(first_cpu, KVM_TDX_INIT_MEM_REGION, flags,
+                               &region, &local_err);
+        } while (r == -EAGAIN || r == -EINTR);
+        if (r < 0) {
+            error_report_err(local_err);
+            exit(1);
+        }
+
+        if (entry->type == TDVF_SECTION_TYPE_TD_HOB ||
+            entry->type == TDVF_SECTION_TYPE_TEMP_MEM) {
+            qemu_ram_munmap(-1, entry->mem_ptr, entry->size);
+            entry->mem_ptr = NULL;
+        }
+    }
+
+    /*
+     * TDVF image has been copied into private region above via
+     * KVM_MEMORY_MAPPING. It becomes useless.
+     */
+    ram_block = tdx_guest->tdvf_mr->ram_block;
+    ram_block_discard_range(ram_block, 0, ram_block->max_length);
 }
 
 static Notifier tdx_machine_done_notify = {
-- 
2.34.1

Re: [PATCH v8 24/55] i386/tdx: Add TDVF memory via KVM_TDX_INIT_MEM_REGION

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:34AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:34 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 24/55] i386/tdx: Add TDVF memory via
>  KVM_TDX_INIT_MEM_REGION
> X-Mailer: git-send-email 2.34.1
> 
> From: Isaku Yamahata <isaku.yamahata@intel.com>
> 
> TDVF firmware (CODE and VARS) needs to be copied to TD's private
> memory via KVM_TDX_INIT_MEM_REGION, as well as TD HOB and TEMP memory.
> 
> If the TDVF section has TDVF_SECTION_ATTRIBUTES_MR_EXTEND set in the
> flag, calling KVM_TDX_EXTEND_MEMORY to extend the measurement.
> 
> After populating the TDVF memory, the original image located in shared
> ramblock can be discarded.
> 
> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
> Changes in v6:
>  - switch back to use KVM_TDX_INIT_MEM_REGION according to KVM's change;
> ---
>  target/i386/kvm/tdx.c | 42 ++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 42 insertions(+)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 25/55] i386/tdx: Call KVM_TDX_INIT_VCPU to initialize TDX vcpu

[Xiaoyao Li]

TDX vcpu needs to be initialized by SEAMCALL(TDH.VP.INIT) and KVM
provides vcpu level IOCTL KVM_TDX_INIT_VCPU for it.

KVM_TDX_INIT_VCPU needs the address of the HOB as input. Invoke it for
each vcpu after HOB list is created.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
 target/i386/kvm/tdx.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 4cf0929d22aa..281fcba9f031 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -266,6 +266,18 @@ static void tdx_init_ram_entries(void)
     tdx_guest->nr_ram_entries = j;
 }
 
+static void tdx_post_init_vcpus(void)
+{
+    TdxFirmwareEntry *hob;
+    CPUState *cpu;
+
+    hob = tdx_get_hob_entry(tdx_guest);
+    CPU_FOREACH(cpu) {
+        tdx_vcpu_ioctl(cpu, KVM_TDX_INIT_VCPU, 0, (void *)hob->address,
+                       &error_fatal);
+    }
+}
+
 static void tdx_finalize_vm(Notifier *notifier, void *unused)
 {
     TdxFirmware *tdvf = &tdx_guest->tdvf;
@@ -304,6 +316,8 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused)
 
     tdvf_hob_create(tdx_guest, tdx_get_hob_entry(tdx_guest));
 
+    tdx_post_init_vcpus();
+
     for_each_tdx_fw_entry(tdvf, entry) {
         struct kvm_tdx_init_mem_region region;
         uint32_t flags;
-- 
2.34.1

Re: [PATCH v8 25/55] i386/tdx: Call KVM_TDX_INIT_VCPU to initialize TDX vcpu

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:35AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:35 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 25/55] i386/tdx: Call KVM_TDX_INIT_VCPU to initialize
>  TDX vcpu
> X-Mailer: git-send-email 2.34.1
> 
> TDX vcpu needs to be initialized by SEAMCALL(TDH.VP.INIT) and KVM
> provides vcpu level IOCTL KVM_TDX_INIT_VCPU for it.
> 
> KVM_TDX_INIT_VCPU needs the address of the HOB as input. Invoke it for
> each vcpu after HOB list is created.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
>  target/i386/kvm/tdx.c | 14 ++++++++++++++
>  1 file changed, 14 insertions(+)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 26/55] i386/tdx: Finalize TDX VM

[Xiaoyao Li]

Invoke KVM_TDX_FINALIZE_VM to finalize the TD's measurement and make
the TD vCPUs runnable once machine initialization is complete.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
 target/i386/kvm/tdx.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 281fcba9f031..78764b1d4be4 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -355,6 +355,9 @@ static void tdx_finalize_vm(Notifier *notifier, void *unused)
      */
     ram_block = tdx_guest->tdvf_mr->ram_block;
     ram_block_discard_range(ram_block, 0, ram_block->max_length);
+
+    tdx_vm_ioctl(KVM_TDX_FINALIZE_VM, 0, NULL, &error_fatal);
+    CONFIDENTIAL_GUEST_SUPPORT(tdx_guest)->ready = true;
 }
 
 static Notifier tdx_machine_done_notify = {
-- 
2.34.1

Re: [PATCH v8 26/55] i386/tdx: Finalize TDX VM

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:36AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:36 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 26/55] i386/tdx: Finalize TDX VM
> X-Mailer: git-send-email 2.34.1
> 
> Invoke KVM_TDX_FINALIZE_VM to finalize the TD's measurement and make
> the TD vCPUs runnable once machine initialization is complete.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
>  target/i386/kvm/tdx.c | 3 +++
>  1 file changed, 3 insertions(+)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 27/55] i386/tdx: Enable user exit on KVM_HC_MAP_GPA_RANGE

[Xiaoyao Li]

KVM translates TDG.VP.VMCALL<MapGPA> to KVM_HC_MAP_GPA_RANGE, and QEMU
needs to enable user exit on KVM_HC_MAP_GPA_RANGE in order to handle the
memory conversion requested by TD guest.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
changes in v6:
 - new patch;
---
 target/i386/kvm/tdx.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 78764b1d4be4..f8953f598584 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -19,6 +19,8 @@
 #include "system/system.h"
 #include "exec/ramblock.h"
 
+#include <linux/kvm_para.h>
+
 #include "hw/i386/e820_memory_layout.h"
 #include "hw/i386/tdvf.h"
 #include "hw/i386/x86.h"
@@ -378,6 +380,11 @@ static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
         }
     }
 
+    /* TDX relies on KVM_HC_MAP_GPA_RANGE to handle TDG.VP.VMCALL<MapGPA> */
+    if (!kvm_enable_hypercall(BIT_ULL(KVM_HC_MAP_GPA_RANGE))) {
+        return -EOPNOTSUPP;
+    }
+
     qemu_add_machine_init_done_notifier(&tdx_machine_done_notify);
 
     tdx_guest = tdx;
-- 
2.34.1

Re: [PATCH v8 27/55] i386/tdx: Enable user exit on KVM_HC_MAP_GPA_RANGE

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:37AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:37 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 27/55] i386/tdx: Enable user exit on KVM_HC_MAP_GPA_RANGE
> X-Mailer: git-send-email 2.34.1
> 
> KVM translates TDG.VP.VMCALL<MapGPA> to KVM_HC_MAP_GPA_RANGE, and QEMU
> needs to enable user exit on KVM_HC_MAP_GPA_RANGE in order to handle the
> memory conversion requested by TD guest.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
> changes in v6:
>  - new patch;
> ---
>  target/i386/kvm/tdx.c | 7 +++++++
>  1 file changed, 7 insertions(+)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 28/55] i386/tdx: Handle KVM_SYSTEM_EVENT_TDX_FATAL

[Xiaoyao Li]

TD guest can use TDG.VP.VMCALL<REPORT_FATAL_ERROR> to request
termination. KVM translates such request into KVM_EXIT_SYSTEM_EVENT with
type of KVM_SYSTEM_EVENT_TDX_FATAL.

Add hanlder for such exit. Parse and print the error message, and
terminate the TD guest in the handler.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
Changes in v8:
 - update to the new data ABI of KVM_SYSTEM_EVENT_TDX_FATAL;

Changes in v6:
 - replace the patch " i386/tdx: Handle TDG.VP.VMCALL<REPORT_FATAL_ERROR>"
   in v5;
---
 target/i386/kvm/kvm.c      | 10 +++++++++
 target/i386/kvm/tdx-stub.c |  5 +++++
 target/i386/kvm/tdx.c      | 45 ++++++++++++++++++++++++++++++++++++++
 target/i386/kvm/tdx.h      |  2 ++
 4 files changed, 62 insertions(+)

diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index 7de5014051eb..a76f34537908 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -6128,6 +6128,16 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run)
     case KVM_EXIT_HYPERCALL:
         ret = kvm_handle_hypercall(run);
         break;
+    case KVM_EXIT_SYSTEM_EVENT:
+        switch (run->system_event.type) {
+        case KVM_SYSTEM_EVENT_TDX_FATAL:
+            ret = tdx_handle_report_fatal_error(cpu, run);
+            break;
+        default:
+            ret = -1;
+            break;
+        }
+        break;
     default:
         fprintf(stderr, "KVM: unknown exit reason %d\n", run->exit_reason);
         ret = -1;
diff --git a/target/i386/kvm/tdx-stub.c b/target/i386/kvm/tdx-stub.c
index 7748b6d0a446..720a4ff046ee 100644
--- a/target/i386/kvm/tdx-stub.c
+++ b/target/i386/kvm/tdx-stub.c
@@ -13,3 +13,8 @@ int tdx_parse_tdvf(void *flash_ptr, int size)
 {
     return -EINVAL;
 }
+
+int tdx_handle_report_fatal_error(X86CPU *cpu, struct kvm_run *run)
+{
+    return -EINVAL;
+}
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index f8953f598584..74b7e3ac85fe 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -607,6 +607,51 @@ int tdx_parse_tdvf(void *flash_ptr, int size)
     return tdvf_parse_metadata(&tdx_guest->tdvf, flash_ptr, size);
 }
 
+int tdx_handle_report_fatal_error(X86CPU *cpu, struct kvm_run *run)
+{
+    uint64_t error_code = run->system_event.data[R_R12];
+    uint64_t reg_mask = run->system_event.data[R_ECX];
+    char *message = NULL;
+    uint64_t *tmp;
+
+    if (error_code & 0xffff) {
+        error_report("TDX: REPORT_FATAL_ERROR: invalid error code: 0x%lx",
+                     error_code);
+        return -1;
+    }
+
+/*
+ * Only 8 registers can contain valid ASCII byte stream to form the fatal
+ * message, and their sequence is: R14, R15, RBX, RDI, RSI, R8, R9, RDX
+ */
+#define TDX_FATAL_MESSAGE_MAX        64
+    if (reg_mask) {
+        message = g_malloc0(TDX_FATAL_MESSAGE_MAX + 1);
+        tmp = (uint64_t *)message;
+
+#define COPY_REG(REG)                               \
+    do {                                            \
+        if (reg_mask & BIT_ULL(REG)) {              \
+            *(tmp++) = run->system_event.data[REG]; \
+        }                                           \
+    } while (0)
+
+        COPY_REG(R_R14);
+        COPY_REG(R_R15);
+        COPY_REG(R_EBX);
+        COPY_REG(R_EDI);
+        COPY_REG(R_ESI);
+        COPY_REG(R_R8);
+        COPY_REG(R_R9);
+        COPY_REG(R_EDX);
+        *((char *)tmp) = '\0';
+    }
+#undef COPY_REG
+
+    error_report("TD guest reports fatal error. %s", message ? : "");
+    return -1;
+}
+
 static bool tdx_guest_get_sept_ve_disable(Object *obj, Error **errp)
 {
     TdxGuest *tdx = TDX_GUEST(obj);
diff --git a/target/i386/kvm/tdx.h b/target/i386/kvm/tdx.h
index 36a7400e7480..04b5afe199f9 100644
--- a/target/i386/kvm/tdx.h
+++ b/target/i386/kvm/tdx.h
@@ -8,6 +8,7 @@
 #endif
 
 #include "confidential-guest.h"
+#include "cpu.h"
 #include "hw/i386/tdvf.h"
 
 #define TYPE_TDX_GUEST "tdx-guest"
@@ -59,5 +60,6 @@ bool is_tdx_vm(void);
 int tdx_pre_create_vcpu(CPUState *cpu, Error **errp);
 void tdx_set_tdvf_region(MemoryRegion *tdvf_mr);
 int tdx_parse_tdvf(void *flash_ptr, int size);
+int tdx_handle_report_fatal_error(X86CPU *cpu, struct kvm_run *run);
 
 #endif /* QEMU_I386_TDX_H */
-- 
2.34.1

Re: [PATCH v8 28/55] i386/tdx: Handle KVM_SYSTEM_EVENT_TDX_FATAL

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:38AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:38 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 28/55] i386/tdx: Handle KVM_SYSTEM_EVENT_TDX_FATAL
> X-Mailer: git-send-email 2.34.1
> 
> TD guest can use TDG.VP.VMCALL<REPORT_FATAL_ERROR> to request
> termination. KVM translates such request into KVM_EXIT_SYSTEM_EVENT with
> type of KVM_SYSTEM_EVENT_TDX_FATAL.
> 
> Add hanlder for such exit. Parse and print the error message, and
> terminate the TD guest in the handler.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
> Changes in v8:
>  - update to the new data ABI of KVM_SYSTEM_EVENT_TDX_FATAL;
> 
> Changes in v6:
>  - replace the patch " i386/tdx: Handle TDG.VP.VMCALL<REPORT_FATAL_ERROR>"
>    in v5;
> ---
>  target/i386/kvm/kvm.c      | 10 +++++++++
>  target/i386/kvm/tdx-stub.c |  5 +++++
>  target/i386/kvm/tdx.c      | 45 ++++++++++++++++++++++++++++++++++++++
>  target/i386/kvm/tdx.h      |  2 ++
>  4 files changed, 62 insertions(+)
> 
> diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
> index 7de5014051eb..a76f34537908 100644
> --- a/target/i386/kvm/kvm.c
> +++ b/target/i386/kvm/kvm.c
> @@ -6128,6 +6128,16 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run)
>      case KVM_EXIT_HYPERCALL:
>          ret = kvm_handle_hypercall(run);
>          break;
> +    case KVM_EXIT_SYSTEM_EVENT:
> +        switch (run->system_event.type) {
> +        case KVM_SYSTEM_EVENT_TDX_FATAL:
> +            ret = tdx_handle_report_fatal_error(cpu, run);
> +            break;
> +        default:
> +            ret = -1;
> +            break;
> +        }
> +        break;
>      default:
>          fprintf(stderr, "KVM: unknown exit reason %d\n", run->exit_reason);
>          ret = -1;
> diff --git a/target/i386/kvm/tdx-stub.c b/target/i386/kvm/tdx-stub.c
> index 7748b6d0a446..720a4ff046ee 100644
> --- a/target/i386/kvm/tdx-stub.c
> +++ b/target/i386/kvm/tdx-stub.c
> @@ -13,3 +13,8 @@ int tdx_parse_tdvf(void *flash_ptr, int size)
>  {
>      return -EINVAL;
>  }
> +
> +int tdx_handle_report_fatal_error(X86CPU *cpu, struct kvm_run *run)
> +{
> +    return -EINVAL;
> +}
> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
> index f8953f598584..74b7e3ac85fe 100644
> --- a/target/i386/kvm/tdx.c
> +++ b/target/i386/kvm/tdx.c
> @@ -607,6 +607,51 @@ int tdx_parse_tdvf(void *flash_ptr, int size)
>      return tdvf_parse_metadata(&tdx_guest->tdvf, flash_ptr, size);
>  }
>  
> +int tdx_handle_report_fatal_error(X86CPU *cpu, struct kvm_run *run)
> +{
> +    uint64_t error_code = run->system_event.data[R_R12];
> +    uint64_t reg_mask = run->system_event.data[R_ECX];
> +    char *message = NULL;
> +    uint64_t *tmp;
> +
> +    if (error_code & 0xffff) {
> +        error_report("TDX: REPORT_FATAL_ERROR: invalid error code: 0x%lx",
> +                     error_code);
> +        return -1;
> +    }
> +
> +/*
> + * Only 8 registers can contain valid ASCII byte stream to form the fatal
> + * message, and their sequence is: R14, R15, RBX, RDI, RSI, R8, R9, RDX
> + */
> +#define TDX_FATAL_MESSAGE_MAX        64

At least, for this macro, and TDX_REPORT_FATAL_ERROR_GPA_VALID in later
patch, could we move these simple macro definitions out of the function?

This could improve the readability for this one function.

> +    if (reg_mask) {
> +        message = g_malloc0(TDX_FATAL_MESSAGE_MAX + 1);
> +        tmp = (uint64_t *)message;
> +
> +#define COPY_REG(REG)                               \
> +    do {                                            \
> +        if (reg_mask & BIT_ULL(REG)) {              \
> +            *(tmp++) = run->system_event.data[REG]; \
> +        }                                           \
> +    } while (0)
> +
> +        COPY_REG(R_R14);
> +        COPY_REG(R_R15);
> +        COPY_REG(R_EBX);
> +        COPY_REG(R_EDI);
> +        COPY_REG(R_ESI);
> +        COPY_REG(R_R8);
> +        COPY_REG(R_R9);
> +        COPY_REG(R_EDX);
> +        *((char *)tmp) = '\0';
> +    }
> +#undef COPY_REG
> +
> +    error_report("TD guest reports fatal error. %s", message ? : "");
> +    return -1;
> +}
> +

Otherwise,

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

Re: [PATCH v8 28/55] i386/tdx: Handle KVM_SYSTEM_EVENT_TDX_FATAL

[Xiaoyao Li]

On 4/28/2025 11:00 PM, Zhao Liu wrote:
> On Tue, Apr 01, 2025 at 09:01:38AM -0400, Xiaoyao Li wrote:
>> Date: Tue,  1 Apr 2025 09:01:38 -0400
>> From: Xiaoyao Li <xiaoyao.li@intel.com>
>> Subject: [PATCH v8 28/55] i386/tdx: Handle KVM_SYSTEM_EVENT_TDX_FATAL
>> X-Mailer: git-send-email 2.34.1
>>
>> TD guest can use TDG.VP.VMCALL<REPORT_FATAL_ERROR> to request
>> termination. KVM translates such request into KVM_EXIT_SYSTEM_EVENT with
>> type of KVM_SYSTEM_EVENT_TDX_FATAL.
>>
>> Add hanlder for such exit. Parse and print the error message, and
>> terminate the TD guest in the handler.
>>
>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>> ---
>> Changes in v8:
>>   - update to the new data ABI of KVM_SYSTEM_EVENT_TDX_FATAL;
>>
>> Changes in v6:
>>   - replace the patch " i386/tdx: Handle TDG.VP.VMCALL<REPORT_FATAL_ERROR>"
>>     in v5;
>> ---
>>   target/i386/kvm/kvm.c      | 10 +++++++++
>>   target/i386/kvm/tdx-stub.c |  5 +++++
>>   target/i386/kvm/tdx.c      | 45 ++++++++++++++++++++++++++++++++++++++
>>   target/i386/kvm/tdx.h      |  2 ++
>>   4 files changed, 62 insertions(+)
>>
>> diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
>> index 7de5014051eb..a76f34537908 100644
>> --- a/target/i386/kvm/kvm.c
>> +++ b/target/i386/kvm/kvm.c
>> @@ -6128,6 +6128,16 @@ int kvm_arch_handle_exit(CPUState *cs, struct kvm_run *run)
>>       case KVM_EXIT_HYPERCALL:
>>           ret = kvm_handle_hypercall(run);
>>           break;
>> +    case KVM_EXIT_SYSTEM_EVENT:
>> +        switch (run->system_event.type) {
>> +        case KVM_SYSTEM_EVENT_TDX_FATAL:
>> +            ret = tdx_handle_report_fatal_error(cpu, run);
>> +            break;
>> +        default:
>> +            ret = -1;
>> +            break;
>> +        }
>> +        break;
>>       default:
>>           fprintf(stderr, "KVM: unknown exit reason %d\n", run->exit_reason);
>>           ret = -1;
>> diff --git a/target/i386/kvm/tdx-stub.c b/target/i386/kvm/tdx-stub.c
>> index 7748b6d0a446..720a4ff046ee 100644
>> --- a/target/i386/kvm/tdx-stub.c
>> +++ b/target/i386/kvm/tdx-stub.c
>> @@ -13,3 +13,8 @@ int tdx_parse_tdvf(void *flash_ptr, int size)
>>   {
>>       return -EINVAL;
>>   }
>> +
>> +int tdx_handle_report_fatal_error(X86CPU *cpu, struct kvm_run *run)
>> +{
>> +    return -EINVAL;
>> +}
>> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
>> index f8953f598584..74b7e3ac85fe 100644
>> --- a/target/i386/kvm/tdx.c
>> +++ b/target/i386/kvm/tdx.c
>> @@ -607,6 +607,51 @@ int tdx_parse_tdvf(void *flash_ptr, int size)
>>       return tdvf_parse_metadata(&tdx_guest->tdvf, flash_ptr, size);
>>   }
>>   
>> +int tdx_handle_report_fatal_error(X86CPU *cpu, struct kvm_run *run)
>> +{
>> +    uint64_t error_code = run->system_event.data[R_R12];
>> +    uint64_t reg_mask = run->system_event.data[R_ECX];
>> +    char *message = NULL;
>> +    uint64_t *tmp;
>> +
>> +    if (error_code & 0xffff) {
>> +        error_report("TDX: REPORT_FATAL_ERROR: invalid error code: 0x%lx",
>> +                     error_code);
>> +        return -1;
>> +    }
>> +
>> +/*
>> + * Only 8 registers can contain valid ASCII byte stream to form the fatal
>> + * message, and their sequence is: R14, R15, RBX, RDI, RSI, R8, R9, RDX
>> + */
>> +#define TDX_FATAL_MESSAGE_MAX        64
> 
> At least, for this macro, and TDX_REPORT_FATAL_ERROR_GPA_VALID in later
> patch, could we move these simple macro definitions out of the function?
> 
> This could improve the readability for this one function.

To me, it's common that put the one-off MACRO right before the line it 
is used.

Anyway, it's not a big deal to move them out of the function. So I will 
move them for your preference.

>> +    if (reg_mask) {
>> +        message = g_malloc0(TDX_FATAL_MESSAGE_MAX + 1);
>> +        tmp = (uint64_t *)message;
>> +
>> +#define COPY_REG(REG)                               \
>> +    do {                                            \
>> +        if (reg_mask & BIT_ULL(REG)) {              \
>> +            *(tmp++) = run->system_event.data[REG]; \
>> +        }                                           \
>> +    } while (0)
>> +
>> +        COPY_REG(R_R14);
>> +        COPY_REG(R_R15);
>> +        COPY_REG(R_EBX);
>> +        COPY_REG(R_EDI);
>> +        COPY_REG(R_ESI);
>> +        COPY_REG(R_R8);
>> +        COPY_REG(R_R9);
>> +        COPY_REG(R_EDX);
>> +        *((char *)tmp) = '\0';
>> +    }
>> +#undef COPY_REG
>> +
>> +    error_report("TD guest reports fatal error. %s", message ? : "");
>> +    return -1;
>> +}
>> +
> 
> Otherwise,
> 
> Reviewed-by: Zhao Liu <zhao1.liu@intel.com>
> 

[PATCH v8 29/55] i386/tdx: Wire TDX_REPORT_FATAL_ERROR with GuestPanic facility

[Xiaoyao Li]

Integrate TDX's TDX_REPORT_FATAL_ERROR into QEMU GuestPanic facility

Originated-from: Isaku Yamahata <isaku.yamahata@intel.com>
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
---
Changes in v8:
- use g_strdup() for copy string;
- use the new data ABI of KVM_SYSTEM_EVENT_TDX_FATAL to grab gpa info;

Changes in v6:
- change error_code of GuestPanicInformationTdx from uint64_t to
  uint32_t, to only contains the bit 31:0 returned in r12.

Changes in v5:
- mention additional error information in gpa when it presents;
- refine the documentation; (Markus)

Changes in v4:
- refine the documentation; (Markus)

Changes in v3:
- Add docmentation of new type and struct; (Daniel)
- refine the error message handling; (Daniel)
---
 qapi/run-state.json   | 31 +++++++++++++++++++--
 system/runstate.c     | 65 +++++++++++++++++++++++++++++++++++++++++++
 target/i386/kvm/tdx.c | 24 +++++++++++++++-
 3 files changed, 117 insertions(+), 3 deletions(-)

diff --git a/qapi/run-state.json b/qapi/run-state.json
index ce95cfa46b73..ee11adc50889 100644
--- a/qapi/run-state.json
+++ b/qapi/run-state.json
@@ -501,10 +501,12 @@
 #
 # @s390: s390 guest panic information type (Since: 2.12)
 #
+# @tdx: tdx guest panic information type (Since: 10.1)
+#
 # Since: 2.9
 ##
 { 'enum': 'GuestPanicInformationType',
-  'data': [ 'hyper-v', 's390' ] }
+  'data': [ 'hyper-v', 's390', 'tdx' ] }
 
 ##
 # @GuestPanicInformation:
@@ -519,7 +521,8 @@
  'base': {'type': 'GuestPanicInformationType'},
  'discriminator': 'type',
  'data': {'hyper-v': 'GuestPanicInformationHyperV',
-          's390': 'GuestPanicInformationS390'}}
+          's390': 'GuestPanicInformationS390',
+          'tdx' : 'GuestPanicInformationTdx'}}
 
 ##
 # @GuestPanicInformationHyperV:
@@ -598,6 +601,30 @@
           'psw-addr': 'uint64',
           'reason': 'S390CrashReason'}}
 
+##
+# @GuestPanicInformationTdx:
+#
+# TDX Guest panic information specific to TDX, as specified in the
+# "Guest-Hypervisor Communication Interface (GHCI) Specification",
+# section TDG.VP.VMCALL<ReportFatalError>.
+#
+# @error-code: TD-specific error code
+#
+# @message: Human-readable error message provided by the guest. Not
+#     to be trusted.
+#
+# @gpa: guest-physical address of a page that contains more verbose
+#     error information, as zero-terminated string.  Present when the
+#     "GPA valid" bit (bit 63) is set in @error-code.
+#
+#
+# Since: 10.1
+##
+{'struct': 'GuestPanicInformationTdx',
+ 'data': {'error-code': 'uint32',
+          'message': 'str',
+          '*gpa': 'uint64'}}
+
 ##
 # @MEMORY_FAILURE:
 #
diff --git a/system/runstate.c b/system/runstate.c
index 272801d30769..e9d5d7505b4a 100644
--- a/system/runstate.c
+++ b/system/runstate.c
@@ -565,6 +565,58 @@ static void qemu_system_wakeup(void)
     }
 }
 
+static char *tdx_parse_panic_message(char *message)
+{
+    bool printable = false;
+    char *buf = NULL;
+    int len = 0, i;
+
+    /*
+     * Although message is defined as a json string, we shouldn't
+     * unconditionally treat it as is because the guest generated it and
+     * it's not necessarily trustable.
+     */
+    if (message) {
+        /* The caller guarantees the NULL-terminated string. */
+        len = strlen(message);
+
+        printable = len > 0;
+        for (i = 0; i < len; i++) {
+            if (!(0x20 <= message[i] && message[i] <= 0x7e)) {
+                printable = false;
+                break;
+            }
+        }
+    }
+
+    if (len == 0) {
+        buf = g_malloc(1);
+        buf[0] = '\0';
+    } else {
+        if (!printable) {
+            /* 3 = length of "%02x " */
+            buf = g_malloc(len * 3);
+            for (i = 0; i < len; i++) {
+                if (message[i] == '\0') {
+                    break;
+                } else {
+                    sprintf(buf + 3 * i, "%02x ", message[i]);
+                }
+            }
+            if (i > 0) {
+                /* replace the last ' '(space) to NULL */
+                buf[i * 3 - 1] = '\0';
+            } else {
+                buf[0] = '\0';
+            }
+        } else {
+            buf = g_strdup(message);
+        }
+    }
+
+    return buf;
+}
+
 void qemu_system_guest_panicked(GuestPanicInformation *info)
 {
     qemu_log_mask(LOG_GUEST_ERROR, "Guest crashed");
@@ -606,7 +658,20 @@ void qemu_system_guest_panicked(GuestPanicInformation *info)
                           S390CrashReason_str(info->u.s390.reason),
                           info->u.s390.psw_mask,
                           info->u.s390.psw_addr);
+        } else if (info->type == GUEST_PANIC_INFORMATION_TYPE_TDX) {
+            char *message = tdx_parse_panic_message(info->u.tdx.message);
+            qemu_log_mask(LOG_GUEST_ERROR,
+                          "\nTDX guest reports fatal error."
+                          " error code: 0x%" PRIx32 " error message:\"%s\"\n",
+                          info->u.tdx.error_code, message);
+            g_free(message);
+            if (info->u.tdx.gpa != -1ull) {
+                qemu_log_mask(LOG_GUEST_ERROR, "Additional error information "
+                              "can be found at gpa page: 0x%" PRIx64 "\n",
+                              info->u.tdx.gpa);
+            }
         }
+
         qapi_free_GuestPanicInformation(info);
     }
 }
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 74b7e3ac85fe..282cdbd775c5 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -16,6 +16,7 @@
 #include "qapi/error.h"
 #include "qom/object_interfaces.h"
 #include "crypto/hash.h"
+#include "system/runstate.h"
 #include "system/system.h"
 #include "exec/ramblock.h"
 
@@ -607,12 +608,27 @@ int tdx_parse_tdvf(void *flash_ptr, int size)
     return tdvf_parse_metadata(&tdx_guest->tdvf, flash_ptr, size);
 }
 
+static void tdx_panicked_on_fatal_error(X86CPU *cpu, uint64_t error_code,
+                                        char *message, uint64_t gpa)
+{
+    GuestPanicInformation *panic_info;
+
+    panic_info = g_new0(GuestPanicInformation, 1);
+    panic_info->type = GUEST_PANIC_INFORMATION_TYPE_TDX;
+    panic_info->u.tdx.error_code = (uint32_t) error_code;
+    panic_info->u.tdx.message = message;
+    panic_info->u.tdx.gpa = gpa;
+
+    qemu_system_guest_panicked(panic_info);
+}
+
 int tdx_handle_report_fatal_error(X86CPU *cpu, struct kvm_run *run)
 {
     uint64_t error_code = run->system_event.data[R_R12];
     uint64_t reg_mask = run->system_event.data[R_ECX];
     char *message = NULL;
     uint64_t *tmp;
+    uint64_t gpa = -1ull;
 
     if (error_code & 0xffff) {
         error_report("TDX: REPORT_FATAL_ERROR: invalid error code: 0x%lx",
@@ -648,7 +664,13 @@ int tdx_handle_report_fatal_error(X86CPU *cpu, struct kvm_run *run)
     }
 #undef COPY_REG
 
-    error_report("TD guest reports fatal error. %s", message ? : "");
+#define TDX_REPORT_FATAL_ERROR_GPA_VALID    BIT_ULL(63)
+    if (error_code & TDX_REPORT_FATAL_ERROR_GPA_VALID) {
+        gpa = run->system_event.data[R_R13];
+    }
+
+    tdx_panicked_on_fatal_error(cpu, error_code, message, gpa);
+
     return -1;
 }
 
-- 
2.34.1

Re: [PATCH v8 29/55] i386/tdx: Wire TDX_REPORT_FATAL_ERROR with GuestPanic facility

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:39AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:39 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 29/55] i386/tdx: Wire TDX_REPORT_FATAL_ERROR with
>  GuestPanic facility
> X-Mailer: git-send-email 2.34.1
> 
> Integrate TDX's TDX_REPORT_FATAL_ERROR into QEMU GuestPanic facility
> 
> Originated-from: Isaku Yamahata <isaku.yamahata@intel.com>
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Markus Armbruster <armbru@redhat.com>
> ---
> Changes in v8:
> - use g_strdup() for copy string;
> - use the new data ABI of KVM_SYSTEM_EVENT_TDX_FATAL to grab gpa info;
> 
> Changes in v6:
> - change error_code of GuestPanicInformationTdx from uint64_t to
>   uint32_t, to only contains the bit 31:0 returned in r12.
> 
> Changes in v5:
> - mention additional error information in gpa when it presents;
> - refine the documentation; (Markus)
> 
> Changes in v4:
> - refine the documentation; (Markus)
> 
> Changes in v3:
> - Add docmentation of new type and struct; (Daniel)
> - refine the error message handling; (Daniel)
> ---
>  qapi/run-state.json   | 31 +++++++++++++++++++--
>  system/runstate.c     | 65 +++++++++++++++++++++++++++++++++++++++++++
>  target/i386/kvm/tdx.c | 24 +++++++++++++++-
>  3 files changed, 117 insertions(+), 3 deletions(-)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 30/55] kvm: Check KVM_CAP_MAX_VCPUS at vm level

[Xiaoyao Li]

KVM with TDX support starts to report different KVM_CAP_MAX_VCPUS per
different VM types. So switch to check the KVM_CAP_MAX_VCPUS at vm level.

KVM still returns the global KVM_CAP_MAX_VCPUS when the KVM is old that
doesn't report different value at vm level.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
 accel/kvm/kvm-all.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index df9840e53a35..5835d840f3ad 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -2425,7 +2425,7 @@ static int kvm_recommended_vcpus(KVMState *s)
 
 static int kvm_max_vcpus(KVMState *s)
 {
-    int ret = kvm_check_extension(s, KVM_CAP_MAX_VCPUS);
+    int ret = kvm_vm_check_extension(s, KVM_CAP_MAX_VCPUS);
     return (ret) ? ret : kvm_recommended_vcpus(s);
 }
 
-- 
2.34.1

Re: [PATCH v8 30/55] kvm: Check KVM_CAP_MAX_VCPUS at vm level

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:40AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:40 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 30/55] kvm: Check KVM_CAP_MAX_VCPUS at vm level
> X-Mailer: git-send-email 2.34.1
> 
> KVM with TDX support starts to report different KVM_CAP_MAX_VCPUS per
> different VM types. So switch to check the KVM_CAP_MAX_VCPUS at vm level.
> 
> KVM still returns the global KVM_CAP_MAX_VCPUS when the KVM is old that
> doesn't report different value at vm level.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
>  accel/kvm/kvm-all.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 31/55] i386/cpu: introduce x86_confidential_guest_cpu_instance_init()

[Xiaoyao Li]

To allow execute confidential guest specific cpu init operations.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
Changes in v6:
 - new patch;
---
 target/i386/confidential-guest.h | 11 +++++++++++
 target/i386/cpu.c                | 10 ++++++++++
 2 files changed, 21 insertions(+)

diff --git a/target/i386/confidential-guest.h b/target/i386/confidential-guest.h
index 164be7633a20..a86c42a47558 100644
--- a/target/i386/confidential-guest.h
+++ b/target/i386/confidential-guest.h
@@ -39,6 +39,7 @@ struct X86ConfidentialGuestClass {
 
     /* <public> */
     int (*kvm_type)(X86ConfidentialGuest *cg);
+    void (*cpu_instance_init)(X86ConfidentialGuest *cg, CPUState *cpu);
     uint32_t (*mask_cpuid_features)(X86ConfidentialGuest *cg, uint32_t feature, uint32_t index,
                                     int reg, uint32_t value);
 };
@@ -59,6 +60,16 @@ static inline int x86_confidential_guest_kvm_type(X86ConfidentialGuest *cg)
     }
 }
 
+static inline void x86_confidential_guest_cpu_instance_init(X86ConfidentialGuest *cg,
+                                                            CPUState *cpu)
+{
+    X86ConfidentialGuestClass *klass = X86_CONFIDENTIAL_GUEST_GET_CLASS(cg);
+
+    if (klass->cpu_instance_init) {
+        klass->cpu_instance_init(cg, cpu);
+    }
+}
+
 /**
  * x86_confidential_guest_mask_cpuid_features:
  *
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index add6430f7edd..5c69d1489365 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -36,6 +36,7 @@
 #include "hw/qdev-properties.h"
 #include "hw/i386/topology.h"
 #ifndef CONFIG_USER_ONLY
+#include "confidential-guest.h"
 #include "system/reset.h"
 #include "qapi/qapi-commands-machine-target.h"
 #include "exec/address-spaces.h"
@@ -8504,6 +8505,15 @@ static void x86_cpu_post_initfn(Object *obj)
     }
 
     accel_cpu_instance_init(CPU(obj));
+
+#ifndef CONFIG_USER_ONLY
+    MachineState *ms = MACHINE(object_dynamic_cast(qdev_get_machine(),
+                                                   TYPE_MACHINE));
+    if (ms && ms->cgs) {
+        x86_confidential_guest_cpu_instance_init(X86_CONFIDENTIAL_GUEST(ms->cgs),
+                                                 (CPU(obj)));
+    }
+#endif
 }
 
 static void x86_cpu_init_default_topo(X86CPU *cpu)
-- 
2.34.1

Re: [PATCH v8 31/55] i386/cpu: introduce x86_confidential_guest_cpu_instance_init()

[Xiaoyao Li]

Hi Paolo,

On 4/1/2025 9:01 PM, Xiaoyao Li wrote:
...
> diff --git a/target/i386/cpu.c b/target/i386/cpu.c
> index add6430f7edd..5c69d1489365 100644
> --- a/target/i386/cpu.c
> +++ b/target/i386/cpu.c
> @@ -36,6 +36,7 @@
>   #include "hw/qdev-properties.h"
>   #include "hw/i386/topology.h"
>   #ifndef CONFIG_USER_ONLY
> +#include "confidential-guest.h"
>   #include "system/reset.h"
>   #include "qapi/qapi-commands-machine-target.h"
>   #include "exec/address-spaces.h"
> @@ -8504,6 +8505,15 @@ static void x86_cpu_post_initfn(Object *obj)
>       }
>   
>       accel_cpu_instance_init(CPU(obj));
> +
> +#ifndef CONFIG_USER_ONLY
> +    MachineState *ms = MACHINE(object_dynamic_cast(qdev_get_machine(),
> +                                                   TYPE_MACHINE));

It leads to

   qemu-system-x86_64: ../hw/core/qdev.c:824: qdev_get_machine: 
Assertion `dev' failed.
   Aborted (core dumped)

for the case of "-cpu help" due to the assert(dev) in qdev_get_machine().

How do you want to resolve it? I can think of two:
1. remove the assert() in qdev_get_machine(). or
2. drop the callback introduce by this patch. Instead just do

    if (is_tdx_vm()) {
	tdx_cpu_instance_init();
    }

> +    if (ms && ms->cgs) {
> +        x86_confidential_guest_cpu_instance_init(X86_CONFIDENTIAL_GUEST(ms->cgs),
> +                                                 (CPU(obj)));
> +    }
> +#endif
>   }
>   
>   static void x86_cpu_init_default_topo(X86CPU *cpu)

Re: [PATCH v8 31/55] i386/cpu: introduce x86_confidential_guest_cpu_instance_init()

[Zhao Liu]

On Thu, Apr 24, 2025 at 01:51:55PM +0800, Xiaoyao Li wrote:
> Date: Thu, 24 Apr 2025 13:51:55 +0800
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: Re: [PATCH v8 31/55] i386/cpu: introduce
>  x86_confidential_guest_cpu_instance_init()
> 
> Hi Paolo,
> 
> On 4/1/2025 9:01 PM, Xiaoyao Li wrote:
> ...
> > diff --git a/target/i386/cpu.c b/target/i386/cpu.c
> > index add6430f7edd..5c69d1489365 100644
> > --- a/target/i386/cpu.c
> > +++ b/target/i386/cpu.c
> > @@ -36,6 +36,7 @@
> >   #include "hw/qdev-properties.h"
> >   #include "hw/i386/topology.h"
> >   #ifndef CONFIG_USER_ONLY
> > +#include "confidential-guest.h"
> >   #include "system/reset.h"
> >   #include "qapi/qapi-commands-machine-target.h"
> >   #include "exec/address-spaces.h"
> > @@ -8504,6 +8505,15 @@ static void x86_cpu_post_initfn(Object *obj)
> >       }
> >       accel_cpu_instance_init(CPU(obj));
> > +
> > +#ifndef CONFIG_USER_ONLY
> > +    MachineState *ms = MACHINE(object_dynamic_cast(qdev_get_machine(),
> > +                                                   TYPE_MACHINE));
> 
> It leads to
> 
>   qemu-system-x86_64: ../hw/core/qdev.c:824: qdev_get_machine: Assertion
> `dev' failed.
>   Aborted (core dumped)
> 
> for the case of "-cpu help" due to the assert(dev) in qdev_get_machine().
> 
> How do you want to resolve it? I can think of two:
> 1. remove the assert() in qdev_get_machine(). or
> 2. drop the callback introduce by this patch. Instead just do
> 
>    if (is_tdx_vm()) {
> 	tdx_cpu_instance_init();
>    }

Sorry I missed this mail when review this patch.

What about checking `current_machine`?

@@ -8541,10 +8541,8 @@ static void x86_cpu_post_initfn(Object *obj)
     accel_cpu_instance_init(CPU(obj));

 #ifndef CONFIG_USER_ONLY
-    MachineState *ms = MACHINE(object_dynamic_cast(qdev_get_machine(),
-                                                   TYPE_MACHINE));
-    if (ms && ms->cgs) {
-        x86_confidential_guest_cpu_instance_init(X86_CONFIDENTIAL_GUEST(ms->cgs),
+    if (current_machine && current_machine->cgs) {
+        x86_confidential_guest_cpu_instance_init(X86_CONFIDENTIAL_GUEST(current_machine->cgs),
                                                  (CPU(obj)));
     }
 #endif
---

"-cpu help" is processed before machine creation. The cpu-core
(cpu_core_instance_init) also checks current_machine to avoid similar
issue.

Regards,
Zhao

Re: [PATCH v8 31/55] i386/cpu: introduce x86_confidential_guest_cpu_instance_init()

[Xiaoyao Li]

On 4/29/2025 6:06 PM, Zhao Liu wrote:
> On Thu, Apr 24, 2025 at 01:51:55PM +0800, Xiaoyao Li wrote:
>> Date: Thu, 24 Apr 2025 13:51:55 +0800
>> From: Xiaoyao Li <xiaoyao.li@intel.com>
>> Subject: Re: [PATCH v8 31/55] i386/cpu: introduce
>>   x86_confidential_guest_cpu_instance_init()
>>
>> Hi Paolo,
>>
>> On 4/1/2025 9:01 PM, Xiaoyao Li wrote:
>> ...
>>> diff --git a/target/i386/cpu.c b/target/i386/cpu.c
>>> index add6430f7edd..5c69d1489365 100644
>>> --- a/target/i386/cpu.c
>>> +++ b/target/i386/cpu.c
>>> @@ -36,6 +36,7 @@
>>>    #include "hw/qdev-properties.h"
>>>    #include "hw/i386/topology.h"
>>>    #ifndef CONFIG_USER_ONLY
>>> +#include "confidential-guest.h"
>>>    #include "system/reset.h"
>>>    #include "qapi/qapi-commands-machine-target.h"
>>>    #include "exec/address-spaces.h"
>>> @@ -8504,6 +8505,15 @@ static void x86_cpu_post_initfn(Object *obj)
>>>        }
>>>        accel_cpu_instance_init(CPU(obj));
>>> +
>>> +#ifndef CONFIG_USER_ONLY
>>> +    MachineState *ms = MACHINE(object_dynamic_cast(qdev_get_machine(),
>>> +                                                   TYPE_MACHINE));
>>
>> It leads to
>>
>>    qemu-system-x86_64: ../hw/core/qdev.c:824: qdev_get_machine: Assertion
>> `dev' failed.
>>    Aborted (core dumped)
>>
>> for the case of "-cpu help" due to the assert(dev) in qdev_get_machine().
>>
>> How do you want to resolve it? I can think of two:
>> 1. remove the assert() in qdev_get_machine(). or
>> 2. drop the callback introduce by this patch. Instead just do
>>
>>     if (is_tdx_vm()) {
>> 	tdx_cpu_instance_init();
>>     }
> 
> Sorry I missed this mail when review this patch.
> 
> What about checking `current_machine`?
> 
> @@ -8541,10 +8541,8 @@ static void x86_cpu_post_initfn(Object *obj)
>       accel_cpu_instance_init(CPU(obj));
> 
>   #ifndef CONFIG_USER_ONLY
> -    MachineState *ms = MACHINE(object_dynamic_cast(qdev_get_machine(),
> -                                                   TYPE_MACHINE));
> -    if (ms && ms->cgs) {
> -        x86_confidential_guest_cpu_instance_init(X86_CONFIDENTIAL_GUEST(ms->cgs),
> +    if (current_machine && current_machine->cgs) {
> +        x86_confidential_guest_cpu_instance_init(X86_CONFIDENTIAL_GUEST(current_machine->cgs),
>                                                    (CPU(obj)));
>       }
>   #endif
> ---
> 
> "-cpu help" is processed before machine creation. The cpu-core
> (cpu_core_instance_init) also checks current_machine to avoid similar
> issue.

This is a really good suggestion! I'll take it.

Thanks!

> Regards,
> Zhao
> 
> 

Re: [PATCH v8 31/55] i386/cpu: introduce x86_confidential_guest_cpu_instance_init()

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:41AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:41 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 31/55] i386/cpu: introduce
>  x86_confidential_guest_cpu_instance_init()
> X-Mailer: git-send-email 2.34.1
> 
> To allow execute confidential guest specific cpu init operations.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
> Changes in v6:
>  - new patch;
> ---
>  target/i386/confidential-guest.h | 11 +++++++++++
>  target/i386/cpu.c                | 10 ++++++++++
>  2 files changed, 21 insertions(+)
> 
> diff --git a/target/i386/confidential-guest.h b/target/i386/confidential-guest.h
> index 164be7633a20..a86c42a47558 100644
> --- a/target/i386/confidential-guest.h
> +++ b/target/i386/confidential-guest.h
> @@ -39,6 +39,7 @@ struct X86ConfidentialGuestClass {
>  
>      /* <public> */
>      int (*kvm_type)(X86ConfidentialGuest *cg);
> +    void (*cpu_instance_init)(X86ConfidentialGuest *cg, CPUState *cpu);
>      uint32_t (*mask_cpuid_features)(X86ConfidentialGuest *cg, uint32_t feature, uint32_t index,
>                                      int reg, uint32_t value);
>  };
> @@ -59,6 +60,16 @@ static inline int x86_confidential_guest_kvm_type(X86ConfidentialGuest *cg)
>      }
>  }
>  
> +static inline void x86_confidential_guest_cpu_instance_init(X86ConfidentialGuest *cg,
> +                                                            CPUState *cpu)

Well, it's a so long function name.

Or maybe we can call it as "x86_confidential_guest_cpu_init" and rename
the hook as "cpu_init"?

> +{
> +    X86ConfidentialGuestClass *klass = X86_CONFIDENTIAL_GUEST_GET_CLASS(cg);
> +
> +    if (klass->cpu_instance_init) {
> +        klass->cpu_instance_init(cg, cpu);
> +    }
> +}
> +
>  /**
>   * x86_confidential_guest_mask_cpuid_features:
>   *
> diff --git a/target/i386/cpu.c b/target/i386/cpu.c
> index add6430f7edd..5c69d1489365 100644
> --- a/target/i386/cpu.c
> +++ b/target/i386/cpu.c
> @@ -36,6 +36,7 @@
>  #include "hw/qdev-properties.h"
>  #include "hw/i386/topology.h"
>  #ifndef CONFIG_USER_ONLY
> +#include "confidential-guest.h"
>  #include "system/reset.h"
>  #include "qapi/qapi-commands-machine-target.h"
>  #include "exec/address-spaces.h"
> @@ -8504,6 +8505,15 @@ static void x86_cpu_post_initfn(Object *obj)
>      }
>  
>      accel_cpu_instance_init(CPU(obj));
> +
> +#ifndef CONFIG_USER_ONLY
> +    MachineState *ms = MACHINE(object_dynamic_cast(qdev_get_machine(),
> +                                                   TYPE_MACHINE));

There's no need to use object_dynamic_cast(), to cast to the basic machine
type.

MACHINE(qdev_get_machine()) is enough.

> +    if (ms && ms->cgs) {
> +        x86_confidential_guest_cpu_instance_init(X86_CONFIDENTIAL_GUEST(ms->cgs),
> +                                                 (CPU(obj)));
> +    }
> +#endif
>  }
>

Others are fine for me,

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 32/55] i386/tdx: implement tdx_cpu_instance_init()

[Xiaoyao Li]

Currently, pmu is not supported for TDX by KVM.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
chanegs in v6:
 - new patch;
---
 target/i386/kvm/tdx.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 282cdbd775c5..49e748af3949 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -400,6 +400,11 @@ static int tdx_kvm_type(X86ConfidentialGuest *cg)
     return KVM_X86_TDX_VM;
 }
 
+static void tdx_cpu_instance_init(X86ConfidentialGuest *cg, CPUState *cpu)
+{
+    object_property_set_bool(OBJECT(cpu), "pmu", false, &error_abort);
+}
+
 static int tdx_validate_attributes(TdxGuest *tdx, Error **errp)
 {
     if ((tdx->attributes & ~tdx_caps->supported_attrs)) {
@@ -781,4 +786,5 @@ static void tdx_guest_class_init(ObjectClass *oc, void *data)
 
     klass->kvm_init = tdx_kvm_init;
     x86_klass->kvm_type = tdx_kvm_type;
+    x86_klass->cpu_instance_init = tdx_cpu_instance_init;
 }
-- 
2.34.1

Re: [PATCH v8 32/55] i386/tdx: implement tdx_cpu_instance_init()

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:42AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:42 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 32/55] i386/tdx: implement tdx_cpu_instance_init()
> X-Mailer: git-send-email 2.34.1
> 
> Currently, pmu is not supported for TDX by KVM.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
> chanegs in v6:
>  - new patch;
> ---
>  target/i386/kvm/tdx.c | 6 ++++++
>  1 file changed, 6 insertions(+)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 33/55] i386/cpu: Introduce enable_cpuid_0x1f to force exposing CPUID 0x1f

[Xiaoyao Li]

Currently, QEMU exposes CPUID 0x1f to guest only when necessary, i.e.,
when topology level that cannot be enumerated by leaf 0xB, e.g., die or
module level, are configured for the guest, e.g., -smp xx,dies=2.

However, TDX architecture forces to require CPUID 0x1f to configure CPU
topology.

Introduce a bool flag, enable_cpuid_0x1f, in CPU for the case that
requires CPUID leaf 0x1f to be exposed to guest.

Introduce a new function x86_has_cpuid_0x1f(), which is the warpper of
cpu->enable_cpuid_0x1f and x86_has_extended_topo() to check if it needs
to enable cpuid leaf 0x1f for the guest.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
 target/i386/cpu.c     | 4 ++--
 target/i386/cpu.h     | 9 +++++++++
 target/i386/kvm/kvm.c | 2 +-
 3 files changed, 12 insertions(+), 3 deletions(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 5c69d1489365..f219961b62cd 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -7006,7 +7006,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
         break;
     case 0x1F:
         /* V2 Extended Topology Enumeration Leaf */
-        if (!x86_has_extended_topo(env->avail_cpu_topo)) {
+        if (!x86_has_cpuid_0x1f(cpu)) {
             *eax = *ebx = *ecx = *edx = 0;
             break;
         }
@@ -7870,7 +7870,7 @@ void x86_cpu_expand_features(X86CPU *cpu, Error **errp)
          * cpu->vendor_cpuid_only has been unset for compatibility with older
          * machine types.
          */
-        if (x86_has_extended_topo(env->avail_cpu_topo) &&
+        if (x86_has_cpuid_0x1f(cpu) &&
             (IS_INTEL_CPU(env) || !cpu->vendor_cpuid_only)) {
             x86_cpu_adjust_level(cpu, &env->cpuid_min_level, 0x1F);
         }
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 76f24446a55d..3910b488f775 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -2251,6 +2251,9 @@ struct ArchCPU {
     /* Compatibility bits for old machine types: */
     bool enable_cpuid_0xb;
 
+    /* Force to enable cpuid 0x1f */
+    bool enable_cpuid_0x1f;
+
     /* Enable auto level-increase for all CPUID leaves */
     bool full_cpuid_auto_level;
 
@@ -2513,6 +2516,12 @@ void host_cpuid(uint32_t function, uint32_t count,
                 uint32_t *eax, uint32_t *ebx, uint32_t *ecx, uint32_t *edx);
 bool cpu_has_x2apic_feature(CPUX86State *env);
 
+static inline bool x86_has_cpuid_0x1f(X86CPU *cpu)
+{
+    return cpu->enable_cpuid_0x1f ||
+           x86_has_extended_topo(cpu->env.avail_cpu_topo);
+}
+
 /* helper.c */
 void x86_cpu_set_a20(X86CPU *cpu, int a20_state);
 void cpu_sync_avx_hflag(CPUX86State *env);
diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index a76f34537908..741b50181ed9 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -1871,7 +1871,7 @@ uint32_t kvm_x86_build_cpuid(CPUX86State *env, struct kvm_cpuid_entry2 *entries,
             break;
         }
         case 0x1f:
-            if (!x86_has_extended_topo(env->avail_cpu_topo)) {
+            if (!x86_has_cpuid_0x1f(env_archcpu(env))) {
                 cpuid_i--;
                 break;
             }
-- 
2.34.1

Re: [PATCH v8 33/55] i386/cpu: Introduce enable_cpuid_0x1f to force exposing CPUID 0x1f

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:43AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:43 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 33/55] i386/cpu: Introduce enable_cpuid_0x1f to force
>  exposing CPUID 0x1f
> X-Mailer: git-send-email 2.34.1
> 
> Currently, QEMU exposes CPUID 0x1f to guest only when necessary, i.e.,
> when topology level that cannot be enumerated by leaf 0xB, e.g., die or
> module level, are configured for the guest, e.g., -smp xx,dies=2.
> 
> However, TDX architecture forces to require CPUID 0x1f to configure CPU
> topology.
> 
> Introduce a bool flag, enable_cpuid_0x1f, in CPU for the case that
> requires CPUID leaf 0x1f to be exposed to guest.
> 
> Introduce a new function x86_has_cpuid_0x1f(), which is the warpper of

s/warpper/wrapper/

(found by "scripts/checkpatch.pl --codespell *" :-))

> cpu->enable_cpuid_0x1f and x86_has_extended_topo() to check if it needs
> to enable cpuid leaf 0x1f for the guest.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
>  target/i386/cpu.c     | 4 ++--
>  target/i386/cpu.h     | 9 +++++++++
>  target/i386/kvm/kvm.c | 2 +-
>  3 files changed, 12 insertions(+), 3 deletions(-)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 34/55] i386/tdx: Force exposing CPUID 0x1f

[Xiaoyao Li]

TDX uses CPUID 0x1f to configure TD guest's CPU topology. So set
enable_cpuid_0x1f for TDs.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
 target/i386/kvm/tdx.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 49e748af3949..b0616eb3d371 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -402,7 +402,11 @@ static int tdx_kvm_type(X86ConfidentialGuest *cg)
 
 static void tdx_cpu_instance_init(X86ConfidentialGuest *cg, CPUState *cpu)
 {
+    X86CPU *x86cpu = X86_CPU(cpu);
+
     object_property_set_bool(OBJECT(cpu), "pmu", false, &error_abort);
+
+    x86cpu->enable_cpuid_0x1f = true;
 }
 
 static int tdx_validate_attributes(TdxGuest *tdx, Error **errp)
-- 
2.34.1

Re: [PATCH v8 34/55] i386/tdx: Force exposing CPUID 0x1f

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:44AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:44 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 34/55] i386/tdx: Force exposing CPUID 0x1f
> X-Mailer: git-send-email 2.34.1
> 
> TDX uses CPUID 0x1f to configure TD guest's CPU topology. So set
> enable_cpuid_0x1f for TDs.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
>  target/i386/kvm/tdx.c | 4 ++++
>  1 file changed, 4 insertions(+)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 35/55] i386/tdx: Set kvm_readonly_mem_enabled to false for TDX VM

[Xiaoyao Li]

TDX only supports readonly for shared memory but not for private memory.

In the view of QEMU, it has no idea whether a memslot is used as shared
memory of private. Thus just mark kvm_readonly_mem_enabled to false to
TDX VM for simplicity.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
 target/i386/kvm/tdx.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index b0616eb3d371..a816f57043f6 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -386,6 +386,15 @@ static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
         return -EOPNOTSUPP;
     }
 
+    /*
+     * Set kvm_readonly_mem_allowed to false, because TDX only supports readonly
+     * memory for shared memory but not for private memory. Besides, whether a
+     * memslot is private or shared is not determined by QEMU.
+     *
+     * Thus, just mark readonly memory not supported for simplicity.
+     */
+    kvm_readonly_mem_allowed = false;
+
     qemu_add_machine_init_done_notifier(&tdx_machine_done_notify);
 
     tdx_guest = tdx;
-- 
2.34.1

Re: [PATCH v8 35/55] i386/tdx: Set kvm_readonly_mem_enabled to false for TDX VM

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:45AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:45 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 35/55] i386/tdx: Set kvm_readonly_mem_enabled to false
>  for TDX VM
> X-Mailer: git-send-email 2.34.1
> 
> TDX only supports readonly for shared memory but not for private memory.
> 
> In the view of QEMU, it has no idea whether a memslot is used as shared
> memory of private. Thus just mark kvm_readonly_mem_enabled to false to
> TDX VM for simplicity.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
>  target/i386/kvm/tdx.c | 9 +++++++++
>  1 file changed, 9 insertions(+)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 36/55] i386/tdx: Disable SMM for TDX VMs

[Xiaoyao Li]

TDX doesn't support SMM and VMM cannot emulate SMM for TDX VMs because
VMM cannot manipulate TDX VM's memory.

Disable SMM for TDX VMs and error out if user requests to enable SMM.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
 target/i386/kvm/tdx.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index a816f57043f6..0eefd058f7a2 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -369,11 +369,20 @@ static Notifier tdx_machine_done_notify = {
 
 static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
 {
+    MachineState *ms = MACHINE(qdev_get_machine());
+    X86MachineState *x86ms = X86_MACHINE(ms);
     TdxGuest *tdx = TDX_GUEST(cgs);
     int r = 0;
 
     kvm_mark_guest_state_protected();
 
+    if (x86ms->smm == ON_OFF_AUTO_AUTO) {
+        x86ms->smm = ON_OFF_AUTO_OFF;
+    } else if (x86ms->smm == ON_OFF_AUTO_ON) {
+        error_setg(errp, "TDX VM doesn't support SMM");
+        return -EINVAL;
+    }
+
     if (!tdx_caps) {
         r = get_tdx_capabilities(errp);
         if (r) {
-- 
2.34.1

Re: [PATCH v8 36/55] i386/tdx: Disable SMM for TDX VMs

[Daniel P. Berrangé]

On Tue, Apr 01, 2025 at 09:01:46AM -0400, Xiaoyao Li wrote:
> TDX doesn't support SMM and VMM cannot emulate SMM for TDX VMs because
> VMM cannot manipulate TDX VM's memory.
> 
> Disable SMM for TDX VMs and error out if user requests to enable SMM.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
>  target/i386/kvm/tdx.c | 9 +++++++++
>  1 file changed, 9 insertions(+)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v8 36/55] i386/tdx: Disable SMM for TDX VMs

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:46AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:46 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 36/55] i386/tdx: Disable SMM for TDX VMs
> X-Mailer: git-send-email 2.34.1
> 
> TDX doesn't support SMM and VMM cannot emulate SMM for TDX VMs because
> VMM cannot manipulate TDX VM's memory.
> 
> Disable SMM for TDX VMs and error out if user requests to enable SMM.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
>  target/i386/kvm/tdx.c | 9 +++++++++
>  1 file changed, 9 insertions(+)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 37/55] i386/tdx: Disable PIC for TDX VMs

[Xiaoyao Li]

Legacy PIC (8259) cannot be supported for TDX VMs since TDX module
doesn't allow directly interrupt injection.  Using posted interrupts
for the PIC is not a viable option as the guest BIOS/kernel will not
do EOI for PIC IRQs, i.e. will leave the vIRR bit set.

Hence disable PIC for TDX VMs and error out if user wants PIC.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
 target/i386/kvm/tdx.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 0eefd058f7a2..0d30506c2021 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -383,6 +383,13 @@ static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
         return -EINVAL;
     }
 
+    if (x86ms->pic == ON_OFF_AUTO_AUTO) {
+        x86ms->pic = ON_OFF_AUTO_OFF;
+    } else if (x86ms->pic == ON_OFF_AUTO_ON) {
+        error_setg(errp, "TDX VM doesn't support PIC");
+        return -EINVAL;
+    }
+
     if (!tdx_caps) {
         r = get_tdx_capabilities(errp);
         if (r) {
-- 
2.34.1

Re: [PATCH v8 37/55] i386/tdx: Disable PIC for TDX VMs

[Daniel P. Berrangé]

On Tue, Apr 01, 2025 at 09:01:47AM -0400, Xiaoyao Li wrote:
> Legacy PIC (8259) cannot be supported for TDX VMs since TDX module
> doesn't allow directly interrupt injection.  Using posted interrupts
> for the PIC is not a viable option as the guest BIOS/kernel will not
> do EOI for PIC IRQs, i.e. will leave the vIRR bit set.
> 
> Hence disable PIC for TDX VMs and error out if user wants PIC.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
>  target/i386/kvm/tdx.c | 7 +++++++
>  1 file changed, 7 insertions(+)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v8 37/55] i386/tdx: Disable PIC for TDX VMs

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:47AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:47 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 37/55] i386/tdx: Disable PIC for TDX VMs
> X-Mailer: git-send-email 2.34.1
> 
> Legacy PIC (8259) cannot be supported for TDX VMs since TDX module
> doesn't allow directly interrupt injection.  Using posted interrupts
> for the PIC is not a viable option as the guest BIOS/kernel will not
> do EOI for PIC IRQs, i.e. will leave the vIRR bit set.
> 
> Hence disable PIC for TDX VMs and error out if user wants PIC.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
>  target/i386/kvm/tdx.c | 7 +++++++
>  1 file changed, 7 insertions(+)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 38/55] i386/tdx: Set and check kernel_irqchip mode for TDX

[Xiaoyao Li]

KVM mandates kernel_irqchip to be split mode.

Set it to split mode automatically when users don't provide an explicit
value, otherwise check it to be the split mode.

Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
 target/i386/kvm/tdx.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 0d30506c2021..4128f27d6b64 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -16,6 +16,7 @@
 #include "qapi/error.h"
 #include "qom/object_interfaces.h"
 #include "crypto/hash.h"
+#include "system/kvm_int.h"
 #include "system/runstate.h"
 #include "system/system.h"
 #include "exec/ramblock.h"
@@ -390,6 +391,13 @@ static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
         return -EINVAL;
     }
 
+    if (kvm_state->kernel_irqchip_split == ON_OFF_AUTO_AUTO ) {
+        kvm_state->kernel_irqchip_split = ON_OFF_AUTO_ON;
+    } else if(kvm_state->kernel_irqchip_split != ON_OFF_AUTO_ON) {
+        error_setg(errp, "TDX VM requires kernel_irqchip to be split");
+        return -EINVAL;
+    }
+
     if (!tdx_caps) {
         r = get_tdx_capabilities(errp);
         if (r) {
-- 
2.34.1

Re: [PATCH v8 38/55] i386/tdx: Set and check kernel_irqchip mode for TDX

[Daniel P. Berrangé]

On Tue, Apr 01, 2025 at 09:01:48AM -0400, Xiaoyao Li wrote:
> KVM mandates kernel_irqchip to be split mode.
> 
> Set it to split mode automatically when users don't provide an explicit
> value, otherwise check it to be the split mode.
> 
> Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
>  target/i386/kvm/tdx.c | 8 ++++++++
>  1 file changed, 8 insertions(+)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
> index 0d30506c2021..4128f27d6b64 100644
> --- a/target/i386/kvm/tdx.c
> +++ b/target/i386/kvm/tdx.c
> @@ -16,6 +16,7 @@
>  #include "qapi/error.h"
>  #include "qom/object_interfaces.h"
>  #include "crypto/hash.h"
> +#include "system/kvm_int.h"
>  #include "system/runstate.h"
>  #include "system/system.h"
>  #include "exec/ramblock.h"
> @@ -390,6 +391,13 @@ static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
>          return -EINVAL;
>      }
>  
> +    if (kvm_state->kernel_irqchip_split == ON_OFF_AUTO_AUTO ) {

Nitpick, no need for a space before the ')' ...

> +        kvm_state->kernel_irqchip_split = ON_OFF_AUTO_ON;
> +    } else if(kvm_state->kernel_irqchip_split != ON_OFF_AUTO_ON) {

..and need a space between 'if' and '('

> +        error_setg(errp, "TDX VM requires kernel_irqchip to be split");
> +        return -EINVAL;
> +    }
> +

Note for self - 'tdx_kvm_init' is called by 'confidential_guest_kvm_init',
which is called by 'kvm_arch_init', which is called by 'kvm_init' *before*
it processes "kvm_state->kernel_irqchip_split == ON_OFF_AUTO_AUTO" to set
the default. So this change is correctly taking priority over the default
behaviour.

>      if (!tdx_caps) {
>          r = get_tdx_capabilities(errp);
>          if (r) {
> -- 
> 2.34.1
> 

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v8 38/55] i386/tdx: Set and check kernel_irqchip mode for TDX

[Xiaoyao Li]

On 4/2/2025 6:41 PM, Daniel P. Berrangé wrote:
> On Tue, Apr 01, 2025 at 09:01:48AM -0400, Xiaoyao Li wrote:
>> KVM mandates kernel_irqchip to be split mode.
>>
>> Set it to split mode automatically when users don't provide an explicit
>> value, otherwise check it to be the split mode.
>>
>> Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>> ---
>>   target/i386/kvm/tdx.c | 8 ++++++++
>>   1 file changed, 8 insertions(+)
> 
> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
> 
> 
>> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
>> index 0d30506c2021..4128f27d6b64 100644
>> --- a/target/i386/kvm/tdx.c
>> +++ b/target/i386/kvm/tdx.c
>> @@ -16,6 +16,7 @@
>>   #include "qapi/error.h"
>>   #include "qom/object_interfaces.h"
>>   #include "crypto/hash.h"
>> +#include "system/kvm_int.h"
>>   #include "system/runstate.h"
>>   #include "system/system.h"
>>   #include "exec/ramblock.h"
>> @@ -390,6 +391,13 @@ static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
>>           return -EINVAL;
>>       }
>>   
>> +    if (kvm_state->kernel_irqchip_split == ON_OFF_AUTO_AUTO ) {
> 
> Nitpick, no need for a space before the ')' ...
> 
>> +        kvm_state->kernel_irqchip_split = ON_OFF_AUTO_ON;
>> +    } else if(kvm_state->kernel_irqchip_split != ON_OFF_AUTO_ON) {
> 
> ..and need a space between 'if' and '('

Thanks for catching these coding style issue. Will fix them in the next 
version.

>> +        error_setg(errp, "TDX VM requires kernel_irqchip to be split");
>> +        return -EINVAL;
>> +    }
>> +
> 
> Note for self - 'tdx_kvm_init' is called by 'confidential_guest_kvm_init',
> which is called by 'kvm_arch_init', which is called by 'kvm_init' *before*
> it processes "kvm_state->kernel_irqchip_split == ON_OFF_AUTO_AUTO" to set
> the default. So this change is correctly taking priority over the default
> behaviour.
> 
>>       if (!tdx_caps) {
>>           r = get_tdx_capabilities(errp);
>>           if (r) {
>> -- 
>> 2.34.1
>>
> 
> With regards,
> Daniel

Re: [PATCH v8 38/55] i386/tdx: Set and check kernel_irqchip mode for TDX

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:48AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:48 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 38/55] i386/tdx: Set and check kernel_irqchip mode for
>  TDX
> X-Mailer: git-send-email 2.34.1
> 
> KVM mandates kernel_irqchip to be split mode.
> 
> Set it to split mode automatically when users don't provide an explicit
> value, otherwise check it to be the split mode.
> 
> Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
>  target/i386/kvm/tdx.c | 8 ++++++++
>  1 file changed, 8 insertions(+)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 39/55] i386/tdx: Don't synchronize guest tsc for TDs

[Xiaoyao Li]

From: Isaku Yamahata <isaku.yamahata@intel.com>

TSC of TDs is not accessible and KVM doesn't allow access of
MSR_IA32_TSC for TDs. To avoid the assert() in kvm_get_tsc, make
kvm_synchronize_all_tsc() noop for TDs,

Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
Reviewed-by: Connor Kuehl <ckuehl@redhat.com>
Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
 target/i386/kvm/kvm.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index 741b50181ed9..ead1d0263385 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -327,7 +327,7 @@ void kvm_synchronize_all_tsc(void)
 {
     CPUState *cpu;
 
-    if (kvm_enabled()) {
+    if (kvm_enabled() && !is_tdx_vm()) {
         CPU_FOREACH(cpu) {
             run_on_cpu(cpu, do_kvm_synchronize_tsc, RUN_ON_CPU_NULL);
         }
-- 
2.34.1

Re: [PATCH v8 39/55] i386/tdx: Don't synchronize guest tsc for TDs

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:49AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:49 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 39/55] i386/tdx: Don't synchronize guest tsc for TDs
> X-Mailer: git-send-email 2.34.1
> 
> From: Isaku Yamahata <isaku.yamahata@intel.com>
> 
> TSC of TDs is not accessible and KVM doesn't allow access of
> MSR_IA32_TSC for TDs. To avoid the assert() in kvm_get_tsc, make
> kvm_synchronize_all_tsc() noop for TDs,
> 
> Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
> Reviewed-by: Connor Kuehl <ckuehl@redhat.com>
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
>  target/i386/kvm/kvm.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 40/55] i386/tdx: Only configure MSR_IA32_UCODE_REV in kvm_init_msrs() for TDs

[Xiaoyao Li]

For TDs, only MSR_IA32_UCODE_REV in kvm_init_msrs() can be configured
by VMM, while the features enumerated/controlled by other MSRs except
MSR_IA32_UCODE_REV in kvm_init_msrs() are not under control of VMM.

Only configure MSR_IA32_UCODE_REV for TDs.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
---
 target/i386/kvm/kvm.c | 44 ++++++++++++++++++++++---------------------
 1 file changed, 23 insertions(+), 21 deletions(-)

diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index ead1d0263385..4078ba40473e 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -3863,32 +3863,34 @@ static void kvm_init_msrs(X86CPU *cpu)
     CPUX86State *env = &cpu->env;
 
     kvm_msr_buf_reset(cpu);
-    if (has_msr_arch_capabs) {
-        kvm_msr_entry_add(cpu, MSR_IA32_ARCH_CAPABILITIES,
-                          env->features[FEAT_ARCH_CAPABILITIES]);
-    }
-
-    if (has_msr_core_capabs) {
-        kvm_msr_entry_add(cpu, MSR_IA32_CORE_CAPABILITY,
-                          env->features[FEAT_CORE_CAPABILITY]);
-    }
-
-    if (has_msr_perf_capabs && cpu->enable_pmu) {
-        kvm_msr_entry_add_perf(cpu, env->features);
+
+    if (!is_tdx_vm()) {
+        if (has_msr_arch_capabs) {
+            kvm_msr_entry_add(cpu, MSR_IA32_ARCH_CAPABILITIES,
+                                env->features[FEAT_ARCH_CAPABILITIES]);
+        }
+
+        if (has_msr_core_capabs) {
+            kvm_msr_entry_add(cpu, MSR_IA32_CORE_CAPABILITY,
+                                env->features[FEAT_CORE_CAPABILITY]);
+        }
+
+        if (has_msr_perf_capabs && cpu->enable_pmu) {
+            kvm_msr_entry_add_perf(cpu, env->features);
+        }
+
+        /*
+         * Older kernels do not include VMX MSRs in KVM_GET_MSR_INDEX_LIST, but
+         * all kernels with MSR features should have them.
+         */
+        if (kvm_feature_msrs && cpu_has_vmx(env)) {
+            kvm_msr_entry_add_vmx(cpu, env->features);
+        }
     }
 
     if (has_msr_ucode_rev) {
         kvm_msr_entry_add(cpu, MSR_IA32_UCODE_REV, cpu->ucode_rev);
     }
-
-    /*
-     * Older kernels do not include VMX MSRs in KVM_GET_MSR_INDEX_LIST, but
-     * all kernels with MSR features should have them.
-     */
-    if (kvm_feature_msrs && cpu_has_vmx(env)) {
-        kvm_msr_entry_add_vmx(cpu, env->features);
-    }
-
     assert(kvm_buf_set_msrs(cpu) == 0);
 }
 
-- 
2.34.1

Re: [PATCH v8 40/55] i386/tdx: Only configure MSR_IA32_UCODE_REV in kvm_init_msrs() for TDs

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:50AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:50 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 40/55] i386/tdx: Only configure MSR_IA32_UCODE_REV in
>  kvm_init_msrs() for TDs
> X-Mailer: git-send-email 2.34.1
> 
> For TDs, only MSR_IA32_UCODE_REV in kvm_init_msrs() can be configured
> by VMM, while the features enumerated/controlled by other MSRs except
> MSR_IA32_UCODE_REV in kvm_init_msrs() are not under control of VMM.
> 
> Only configure MSR_IA32_UCODE_REV for TDs.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> Acked-by: Gerd Hoffmann <kraxel@redhat.com>
> ---
>  target/i386/kvm/kvm.c | 44 ++++++++++++++++++++++---------------------
>  1 file changed, 23 insertions(+), 21 deletions(-)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 41/55] i386/apic: Skip kvm_apic_put() for TDX

[Xiaoyao Li]

KVM neithers allow writing to MSR_IA32_APICBASE for TDs, nor allow for
KVM_SET_LAPIC[*].

Note, KVM_GET_LAPIC is also disallowed for TDX. It is called in the path

  do_kvm_cpu_synchronize_state()
  -> kvm_arch_get_registers()
     -> kvm_get_apic()

and it's already disllowed for confidential guest through
guest_state_protected.

[*] https://lore.kernel.org/all/Z3w4Ku4Jq0CrtXne@google.com/

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
Changes in v8:
- Fix the coding style; (Francesco)
---
 hw/i386/kvm/apic.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/hw/i386/kvm/apic.c b/hw/i386/kvm/apic.c
index 757510600098..cb65fca49586 100644
--- a/hw/i386/kvm/apic.c
+++ b/hw/i386/kvm/apic.c
@@ -17,6 +17,7 @@
 #include "system/hw_accel.h"
 #include "system/kvm.h"
 #include "kvm/kvm_i386.h"
+#include "kvm/tdx.h"
 
 static inline void kvm_apic_set_reg(struct kvm_lapic_state *kapic,
                                     int reg_id, uint32_t val)
@@ -141,6 +142,10 @@ static void kvm_apic_put(CPUState *cs, run_on_cpu_data data)
     struct kvm_lapic_state kapic;
     int ret;
 
+    if (is_tdx_vm()) {
+        return;
+    }
+
     kvm_put_apicbase(s->cpu, s->apicbase);
     kvm_put_apic_state(s, &kapic);
 
-- 
2.34.1

Re: [PATCH v8 41/55] i386/apic: Skip kvm_apic_put() for TDX

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:51AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:51 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 41/55] i386/apic: Skip kvm_apic_put() for TDX
> X-Mailer: git-send-email 2.34.1
> 
> KVM neithers allow writing to MSR_IA32_APICBASE for TDs, nor allow for
> KVM_SET_LAPIC[*].
> 
> Note, KVM_GET_LAPIC is also disallowed for TDX. It is called in the path
> 
>   do_kvm_cpu_synchronize_state()
>   -> kvm_arch_get_registers()
>      -> kvm_get_apic()
> 
> and it's already disllowed for confidential guest through
> guest_state_protected.
> 
> [*] https://lore.kernel.org/all/Z3w4Ku4Jq0CrtXne@google.com/
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
> Changes in v8:
> - Fix the coding style; (Francesco)
> ---
>  hw/i386/kvm/apic.c | 5 +++++
>  1 file changed, 5 insertions(+)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 42/55] cpu: Don't set vcpu_dirty when guest_state_protected

[Xiaoyao Li]

QEMU calls kvm_arch_put_registers() when vcpu_dirty is true in
kvm_vcpu_exec(). However, for confidential guest, like TDX, putting
registers is disallowed due to guest state is protected.

Only set vcpu_dirty to true with guest state is not protected when
creating the vcpu.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
Changes in v7:
 - new patch to replace "i386/tdx: Don't get/put guest state for TDX VMs"
   in v6;
---
 accel/kvm/kvm-all.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 5835d840f3ad..9862d8ff1d38 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -466,7 +466,9 @@ int kvm_create_vcpu(CPUState *cpu)
 
     cpu->kvm_fd = kvm_fd;
     cpu->kvm_state = s;
-    cpu->vcpu_dirty = true;
+    if (!s->guest_state_protected) {
+        cpu->vcpu_dirty = true;
+    }
     cpu->dirty_pages = 0;
     cpu->throttle_us_per_full = 0;
 
-- 
2.34.1

Re: [PATCH v8 42/55] cpu: Don't set vcpu_dirty when guest_state_protected

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:52AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:52 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 42/55] cpu: Don't set vcpu_dirty when
>  guest_state_protected
> X-Mailer: git-send-email 2.34.1
> 
> QEMU calls kvm_arch_put_registers() when vcpu_dirty is true in
> kvm_vcpu_exec(). However, for confidential guest, like TDX, putting
> registers is disallowed due to guest state is protected.
> 
> Only set vcpu_dirty to true with guest state is not protected when
> creating the vcpu.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
> Changes in v7:
>  - new patch to replace "i386/tdx: Don't get/put guest state for TDX VMs"
>    in v6;
> ---
>  accel/kvm/kvm-all.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 43/55] i386/cgs: Rename *mask_cpuid_features() to *adjust_cpuid_features()

[Xiaoyao Li]

Because for TDX case, there are also fixed-1 bits that enfored by TDX
module.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
 target/i386/confidential-guest.h | 20 ++++++++++----------
 target/i386/kvm/kvm.c            |  2 +-
 target/i386/sev.c                |  4 ++--
 3 files changed, 13 insertions(+), 13 deletions(-)

diff --git a/target/i386/confidential-guest.h b/target/i386/confidential-guest.h
index a86c42a47558..777d43cc9688 100644
--- a/target/i386/confidential-guest.h
+++ b/target/i386/confidential-guest.h
@@ -40,8 +40,8 @@ struct X86ConfidentialGuestClass {
     /* <public> */
     int (*kvm_type)(X86ConfidentialGuest *cg);
     void (*cpu_instance_init)(X86ConfidentialGuest *cg, CPUState *cpu);
-    uint32_t (*mask_cpuid_features)(X86ConfidentialGuest *cg, uint32_t feature, uint32_t index,
-                                    int reg, uint32_t value);
+    uint32_t (*adjust_cpuid_features)(X86ConfidentialGuest *cg, uint32_t feature,
+                                      uint32_t index, int reg, uint32_t value);
 };
 
 /**
@@ -71,21 +71,21 @@ static inline void x86_confidential_guest_cpu_instance_init(X86ConfidentialGuest
 }
 
 /**
- * x86_confidential_guest_mask_cpuid_features:
+ * x86_confidential_guest_adjust_cpuid_features:
  *
- * Removes unsupported features from a confidential guest's CPUID values, returns
- * the value with the bits removed.  The bits removed should be those that KVM
- * provides independent of host-supported CPUID features, but are not supported by
- * the confidential computing firmware.
+ * Adjust the supported features from a confidential guest's CPUID values,
+ * returns the adjusted value.  There are bits being removed that are not
+ * supported by the confidential computing firmware or bits being added that
+ * are forcibly exposed to guest by the confidential computing firmware.
  */
-static inline int x86_confidential_guest_mask_cpuid_features(X86ConfidentialGuest *cg,
+static inline int x86_confidential_guest_adjust_cpuid_features(X86ConfidentialGuest *cg,
                                                              uint32_t feature, uint32_t index,
                                                              int reg, uint32_t value)
 {
     X86ConfidentialGuestClass *klass = X86_CONFIDENTIAL_GUEST_GET_CLASS(cg);
 
-    if (klass->mask_cpuid_features) {
-        return klass->mask_cpuid_features(cg, feature, index, reg, value);
+    if (klass->adjust_cpuid_features) {
+        return klass->adjust_cpuid_features(cg, feature, index, reg, value);
     } else {
         return value;
     }
diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index 4078ba40473e..fa46edaeac8d 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -573,7 +573,7 @@ uint32_t kvm_arch_get_supported_cpuid(KVMState *s, uint32_t function,
     }
 
     if (current_machine->cgs) {
-        ret = x86_confidential_guest_mask_cpuid_features(
+        ret = x86_confidential_guest_adjust_cpuid_features(
             X86_CONFIDENTIAL_GUEST(current_machine->cgs),
             function, index, reg, ret);
     }
diff --git a/target/i386/sev.c b/target/i386/sev.c
index 0e1dbb6959ec..a6c0a697250b 100644
--- a/target/i386/sev.c
+++ b/target/i386/sev.c
@@ -946,7 +946,7 @@ out:
 }
 
 static uint32_t
-sev_snp_mask_cpuid_features(X86ConfidentialGuest *cg, uint32_t feature, uint32_t index,
+sev_snp_adjust_cpuid_features(X86ConfidentialGuest *cg, uint32_t feature, uint32_t index,
                             int reg, uint32_t value)
 {
     switch (feature) {
@@ -2404,7 +2404,7 @@ sev_snp_guest_class_init(ObjectClass *oc, void *data)
     klass->launch_finish = sev_snp_launch_finish;
     klass->launch_update_data = sev_snp_launch_update_data;
     klass->kvm_init = sev_snp_kvm_init;
-    x86_klass->mask_cpuid_features = sev_snp_mask_cpuid_features;
+    x86_klass->adjust_cpuid_features = sev_snp_adjust_cpuid_features;
     x86_klass->kvm_type = sev_snp_kvm_type;
 
     object_class_property_add(oc, "policy", "uint64",
-- 
2.34.1

Re: [PATCH v8 43/55] i386/cgs: Rename *mask_cpuid_features() to *adjust_cpuid_features()

[Daniel P. Berrangé]

On Tue, Apr 01, 2025 at 09:01:53AM -0400, Xiaoyao Li wrote:
> Because for TDX case, there are also fixed-1 bits that enfored by TDX
> module.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
>  target/i386/confidential-guest.h | 20 ++++++++++----------
>  target/i386/kvm/kvm.c            |  2 +-
>  target/i386/sev.c                |  4 ++--
>  3 files changed, 13 insertions(+), 13 deletions(-)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v8 43/55] i386/cgs: Rename *mask_cpuid_features() to *adjust_cpuid_features()

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:53AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:53 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 43/55] i386/cgs: Rename *mask_cpuid_features() to
>  *adjust_cpuid_features()
> X-Mailer: git-send-email 2.34.1
> 
> Because for TDX case, there are also fixed-1 bits that enfored by TDX
> module.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
>  target/i386/confidential-guest.h | 20 ++++++++++----------
>  target/i386/kvm/kvm.c            |  2 +-
>  target/i386/sev.c                |  4 ++--
>  3 files changed, 13 insertions(+), 13 deletions(-)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 44/55] i386/tdx: Implement adjust_cpuid_features() for TDX

[Xiaoyao Li]

Maintain a TDX specific supported CPUID set, and use it to mask the
common supported CPUID value of KVM. It can avoid newly added supported
features (reported via KVM_GET_SUPPORTED_CPUID) for common VMs being
falsely reported as supported for TDX.

As the first step, initialize the TDX supported CPUID set with all the
configurable CPUID bits. It's not complete because there are other CPUID
bits are supported for TDX but not reported as directly configurable.
E.g. the XFAM related bits, attribute related bits and fixed-1 bits.
They will be handled in the future.

Also, what matters are the CPUID bits related to QEMU's feature word.
Only mask the CPUID leafs which are feature word leaf.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
 target/i386/cpu.c          | 18 ++++++++++++++++++
 target/i386/cpu.h          |  1 +
 target/i386/kvm/kvm.c      |  2 +-
 target/i386/kvm/kvm_i386.h |  1 +
 target/i386/kvm/tdx.c      | 34 ++++++++++++++++++++++++++++++++++
 5 files changed, 55 insertions(+), 1 deletion(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index f219961b62cd..4061d38d3fbe 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -1655,6 +1655,24 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
     },
 };
 
+bool is_feature_word_cpuid(uint32_t feature, uint32_t index, int reg)
+{
+    FeatureWordInfo *wi;
+    FeatureWord w;
+
+    for (w = 0; w < FEATURE_WORDS; w++)
+    {
+        wi = &feature_word_info[w];
+        if (wi->type == CPUID_FEATURE_WORD && wi->cpuid.eax == feature &&
+            (!wi->cpuid.needs_ecx || wi->cpuid.ecx == index) &&
+            wi->cpuid.reg == reg)
+        {
+            return true;
+        }
+    }
+    return false;
+}
+
 typedef struct FeatureMask {
     FeatureWord index;
     uint64_t mask;
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 3910b488f775..42ef77789ded 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -2515,6 +2515,7 @@ void cpu_set_apic_feature(CPUX86State *env);
 void host_cpuid(uint32_t function, uint32_t count,
                 uint32_t *eax, uint32_t *ebx, uint32_t *ecx, uint32_t *edx);
 bool cpu_has_x2apic_feature(CPUX86State *env);
+bool is_feature_word_cpuid(uint32_t feature, uint32_t index, int reg);
 
 static inline bool x86_has_cpuid_0x1f(X86CPU *cpu)
 {
diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index fa46edaeac8d..17d7bf6ae9aa 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -393,7 +393,7 @@ static bool host_tsx_broken(void)
 
 /* Returns the value for a specific register on the cpuid entry
  */
-static uint32_t cpuid_entry_get_reg(struct kvm_cpuid_entry2 *entry, int reg)
+uint32_t cpuid_entry_get_reg(struct kvm_cpuid_entry2 *entry, int reg)
 {
     uint32_t ret = 0;
     switch (reg) {
diff --git a/target/i386/kvm/kvm_i386.h b/target/i386/kvm/kvm_i386.h
index dc696cb7238a..484a1de84d51 100644
--- a/target/i386/kvm/kvm_i386.h
+++ b/target/i386/kvm/kvm_i386.h
@@ -62,6 +62,7 @@ void kvm_update_msi_routes_all(void *private, bool global,
 struct kvm_cpuid_entry2 *cpuid_find_entry(struct kvm_cpuid2 *cpuid,
                                           uint32_t function,
                                           uint32_t index);
+uint32_t cpuid_entry_get_reg(struct kvm_cpuid_entry2 *entry, int reg);
 uint32_t kvm_x86_build_cpuid(CPUX86State *env, struct kvm_cpuid_entry2 *entries,
                              uint32_t cpuid_i);
 #endif /* CONFIG_KVM */
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 4128f27d6b64..b9a96c2e392d 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -45,6 +45,7 @@
 static TdxGuest *tdx_guest;
 
 static struct kvm_tdx_capabilities *tdx_caps;
+static struct kvm_cpuid2 *tdx_supported_cpuid;
 
 /* Valid after kvm_arch_init()->confidential_guest_kvm_init()->tdx_kvm_init() */
 bool is_tdx_vm(void)
@@ -368,6 +369,20 @@ static Notifier tdx_machine_done_notify = {
     .notify = tdx_finalize_vm,
 };
 
+static void tdx_setup_supported_cpuid(void)
+{
+    if (tdx_supported_cpuid) {
+        return;
+    }
+
+    tdx_supported_cpuid = g_malloc0(sizeof(*tdx_supported_cpuid) +
+                    KVM_MAX_CPUID_ENTRIES * sizeof(struct kvm_cpuid_entry2));
+
+    memcpy(tdx_supported_cpuid->entries, tdx_caps->cpuid.entries,
+           tdx_caps->cpuid.nent * sizeof(struct kvm_cpuid_entry2));
+    tdx_supported_cpuid->nent = tdx_caps->cpuid.nent;
+}
+
 static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
 {
     MachineState *ms = MACHINE(qdev_get_machine());
@@ -405,6 +420,8 @@ static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
         }
     }
 
+    tdx_setup_supported_cpuid();
+
     /* TDX relies on KVM_HC_MAP_GPA_RANGE to handle TDG.VP.VMCALL<MapGPA> */
     if (!kvm_enable_hypercall(BIT_ULL(KVM_HC_MAP_GPA_RANGE))) {
         return -EOPNOTSUPP;
@@ -442,6 +459,22 @@ static void tdx_cpu_instance_init(X86ConfidentialGuest *cg, CPUState *cpu)
     x86cpu->enable_cpuid_0x1f = true;
 }
 
+static uint32_t tdx_adjust_cpuid_features(X86ConfidentialGuest *cg,
+                                          uint32_t feature, uint32_t index,
+                                          int reg, uint32_t value)
+{
+    struct kvm_cpuid_entry2 *e;
+
+    if (is_feature_word_cpuid(feature, index, reg)) {
+        e = cpuid_find_entry(tdx_supported_cpuid, feature, index);
+        if (e) {
+            value &= cpuid_entry_get_reg(e, reg);
+        }
+    }
+
+    return value;
+}
+
 static int tdx_validate_attributes(TdxGuest *tdx, Error **errp)
 {
     if ((tdx->attributes & ~tdx_caps->supported_attrs)) {
@@ -824,4 +857,5 @@ static void tdx_guest_class_init(ObjectClass *oc, void *data)
     klass->kvm_init = tdx_kvm_init;
     x86_klass->kvm_type = tdx_kvm_type;
     x86_klass->cpu_instance_init = tdx_cpu_instance_init;
+    x86_klass->adjust_cpuid_features = tdx_adjust_cpuid_features;
 }
-- 
2.34.1

Re: [PATCH v8 44/55] i386/tdx: Implement adjust_cpuid_features() for TDX

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:54AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:54 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 44/55] i386/tdx: Implement adjust_cpuid_features() for
>  TDX
> X-Mailer: git-send-email 2.34.1
> 
> Maintain a TDX specific supported CPUID set, and use it to mask the
> common supported CPUID value of KVM. It can avoid newly added supported
> features (reported via KVM_GET_SUPPORTED_CPUID) for common VMs being
> falsely reported as supported for TDX.
> 
> As the first step, initialize the TDX supported CPUID set with all the
> configurable CPUID bits. It's not complete because there are other CPUID
> bits are supported for TDX but not reported as directly configurable.
> E.g. the XFAM related bits, attribute related bits and fixed-1 bits.
> They will be handled in the future.
> 
> Also, what matters are the CPUID bits related to QEMU's feature word.
> Only mask the CPUID leafs which are feature word leaf.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
>  target/i386/cpu.c          | 18 ++++++++++++++++++
>  target/i386/cpu.h          |  1 +
>  target/i386/kvm/kvm.c      |  2 +-
>  target/i386/kvm/kvm_i386.h |  1 +
>  target/i386/kvm/tdx.c      | 34 ++++++++++++++++++++++++++++++++++
>  5 files changed, 55 insertions(+), 1 deletion(-)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

> diff --git a/target/i386/cpu.c b/target/i386/cpu.c
> index f219961b62cd..4061d38d3fbe 100644
> --- a/target/i386/cpu.c
> +++ b/target/i386/cpu.c
> @@ -1655,6 +1655,24 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
>      },
>  };
>  
> +bool is_feature_word_cpuid(uint32_t feature, uint32_t index, int reg)
> +{
> +    FeatureWordInfo *wi;
> +    FeatureWord w;
> +
> +    for (w = 0; w < FEATURE_WORDS; w++)
> +    {

(style nit?) "{" should be at the end of the previous line?

> +        wi = &feature_word_info[w];
> +        if (wi->type == CPUID_FEATURE_WORD && wi->cpuid.eax == feature &&
> +            (!wi->cpuid.needs_ecx || wi->cpuid.ecx == index) &&
> +            wi->cpuid.reg == reg)
> +        {

ditto.

> +            return true;
> +        }
> +    }
> +    return false;
> +}
> +

[PATCH v8 45/55] i386/tdx: Add TDX fixed1 bits to supported CPUIDs

[Xiaoyao Li]

TDX architecture forcibly sets some CPUID bits for TD guest that VMM
cannot disable it. They are fixed1 bits.

Fixed1 bits are not covered by tdx_caps.cpuid (which only contians the
directly configurable bits), while fixed1 bits are supported for TD guest
obviously.

Add fixed1 bits to tdx_supported_cpuid. Besides, set all the fixed1
bits to the initial set of KVM's support since KVM might not report them
as supported.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
 target/i386/cpu.h          |   2 +
 target/i386/kvm/kvm_i386.h |   7 ++
 target/i386/kvm/tdx.c      | 132 +++++++++++++++++++++++++++++++++++++
 target/i386/sev.c          |   5 --
 4 files changed, 141 insertions(+), 5 deletions(-)

diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 42ef77789ded..115137279a1a 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -924,6 +924,8 @@ uint64_t x86_cpu_get_supported_feature_word(X86CPU *cpu, FeatureWord w);
 #define CPUID_7_0_EDX_FSRM              (1U << 4)
 /* AVX512 Vector Pair Intersection to a Pair of Mask Registers */
 #define CPUID_7_0_EDX_AVX512_VP2INTERSECT (1U << 8)
+ /* "md_clear" VERW clears CPU buffers */
+#define CPUID_7_0_EDX_MD_CLEAR          (1U << 10)
 /* SERIALIZE instruction */
 #define CPUID_7_0_EDX_SERIALIZE         (1U << 14)
 /* TSX Suspend Load Address Tracking instruction */
diff --git a/target/i386/kvm/kvm_i386.h b/target/i386/kvm/kvm_i386.h
index 484a1de84d51..c1bafcfc9b63 100644
--- a/target/i386/kvm/kvm_i386.h
+++ b/target/i386/kvm/kvm_i386.h
@@ -13,8 +13,15 @@
 
 #include "system/kvm.h"
 
+#include <linux/kvm.h>
+
 #define KVM_MAX_CPUID_ENTRIES  100
 
+typedef struct KvmCpuidInfo {
+    struct kvm_cpuid2 cpuid;
+    struct kvm_cpuid_entry2 entries[KVM_MAX_CPUID_ENTRIES];
+} KvmCpuidInfo;
+
 /* always false if !CONFIG_KVM */
 #define kvm_pit_in_kernel() \
     (kvm_irqchip_in_kernel() && !kvm_irqchip_is_split())
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index b9a96c2e392d..49a94d8ffe7d 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -369,6 +369,131 @@ static Notifier tdx_machine_done_notify = {
     .notify = tdx_finalize_vm,
 };
 
+/*
+ * Some CPUID bits change from fixed1 to configurable bits when TDX module
+ * supports TDX_FEATURES0.VE_REDUCTION. e.g., MCA/MCE/MTRR/CORE_CAPABILITY.
+ *
+ * To make QEMU work with all the versions of TDX module, keep the fixed1 bits
+ * here if they are ever fixed1 bits in any of the version though not fixed1 in
+ * the latest version. Otherwise, with the older version of TDX module, QEMU may
+ * treat the fixed1 bit as unsupported.
+ *
+ * For newer TDX module, it does no harm to keep them in tdx_fixed1_bits even
+ * though they changed to configurable bits. Because tdx_fixed1_bits is used to
+ * setup the supported bits.
+ */
+KvmCpuidInfo tdx_fixed1_bits = {
+    .cpuid.nent = 8,
+    .entries[0] = {
+        .function = 0x1,
+        .index = 0,
+        .ecx = CPUID_EXT_SSE3 | CPUID_EXT_PCLMULQDQ | CPUID_EXT_DTES64 |
+               CPUID_EXT_DSCPL | CPUID_EXT_SSSE3 | CPUID_EXT_CX16 |
+               CPUID_EXT_PDCM | CPUID_EXT_PCID | CPUID_EXT_SSE41 |
+               CPUID_EXT_SSE42 | CPUID_EXT_X2APIC | CPUID_EXT_MOVBE |
+               CPUID_EXT_POPCNT | CPUID_EXT_AES | CPUID_EXT_XSAVE |
+               CPUID_EXT_RDRAND | CPUID_EXT_HYPERVISOR,
+        .edx = CPUID_FP87 | CPUID_VME | CPUID_DE | CPUID_PSE | CPUID_TSC |
+               CPUID_MSR | CPUID_PAE | CPUID_MCE | CPUID_CX8 | CPUID_APIC |
+               CPUID_SEP | CPUID_MTRR | CPUID_PGE | CPUID_MCA | CPUID_CMOV |
+               CPUID_PAT | CPUID_CLFLUSH | CPUID_DTS | CPUID_MMX | CPUID_FXSR |
+               CPUID_SSE | CPUID_SSE2,
+    },
+    .entries[1] = {
+        .function = 0x6,
+        .index = 0,
+        .eax = CPUID_6_EAX_ARAT,
+    },
+    .entries[2] = {
+        .function = 0x7,
+        .index = 0,
+        .flags = KVM_CPUID_FLAG_SIGNIFCANT_INDEX,
+        .ebx = CPUID_7_0_EBX_FSGSBASE | CPUID_7_0_EBX_FDP_EXCPTN_ONLY |
+               CPUID_7_0_EBX_SMEP | CPUID_7_0_EBX_INVPCID |
+               CPUID_7_0_EBX_ZERO_FCS_FDS | CPUID_7_0_EBX_RDSEED |
+               CPUID_7_0_EBX_SMAP | CPUID_7_0_EBX_CLFLUSHOPT |
+               CPUID_7_0_EBX_CLWB | CPUID_7_0_EBX_SHA_NI,
+        .ecx = CPUID_7_0_ECX_BUS_LOCK_DETECT | CPUID_7_0_ECX_MOVDIRI |
+               CPUID_7_0_ECX_MOVDIR64B,
+        .edx = CPUID_7_0_EDX_MD_CLEAR | CPUID_7_0_EDX_SPEC_CTRL |
+               CPUID_7_0_EDX_STIBP | CPUID_7_0_EDX_FLUSH_L1D |
+               CPUID_7_0_EDX_ARCH_CAPABILITIES | CPUID_7_0_EDX_CORE_CAPABILITY |
+               CPUID_7_0_EDX_SPEC_CTRL_SSBD,
+    },
+    .entries[3] = {
+        .function = 0x7,
+        .index = 2,
+        .flags = KVM_CPUID_FLAG_SIGNIFCANT_INDEX,
+        .edx = (1U << 0) | (1U << 1) | (1U << 2) | (1U << 4),
+    },
+    .entries[4] = {
+        .function = 0xD,
+        .index = 0,
+        .flags = KVM_CPUID_FLAG_SIGNIFCANT_INDEX,
+        .eax = XSTATE_FP_MASK | XSTATE_SSE_MASK,
+    },
+    .entries[5] = {
+        .function = 0xD,
+        .index = 1,
+        .flags = KVM_CPUID_FLAG_SIGNIFCANT_INDEX,
+        .eax = CPUID_XSAVE_XSAVEOPT | CPUID_XSAVE_XSAVEC|
+               CPUID_XSAVE_XGETBV1 | CPUID_XSAVE_XSAVES,
+    },
+    .entries[6] = {
+        .function = 0x80000001,
+        .index = 0,
+        .ecx = CPUID_EXT3_LAHF_LM | CPUID_EXT3_ABM | CPUID_EXT3_3DNOWPREFETCH,
+        /* strictly speaking, SYSCALL is not fixed1 bit since it depends on
+         * the CPU to be in 64-bit mode. But here fixed1 is used to serve the
+         * purpose of supported bits for TDX. In this sense, SYACALL is always
+         * supported.
+         */
+        .edx = CPUID_EXT2_SYSCALL | CPUID_EXT2_NX | CPUID_EXT2_PDPE1GB |
+               CPUID_EXT2_RDTSCP | CPUID_EXT2_LM,
+    },
+    .entries[7] = {
+        .function = 0x80000007,
+        .index = 0,
+        .edx = CPUID_APM_INVTSC,
+    },
+};
+
+static struct kvm_cpuid_entry2 *find_in_supported_entry(uint32_t function,
+                                                        uint32_t index)
+{
+    struct kvm_cpuid_entry2 *e;
+
+    e = cpuid_find_entry(tdx_supported_cpuid, function, index);
+    if (!e) {
+        if (tdx_supported_cpuid->nent >= KVM_MAX_CPUID_ENTRIES) {
+            error_report("tdx_supported_cpuid requries more space than %d entries",
+                          KVM_MAX_CPUID_ENTRIES);
+            exit(1);
+        }
+        e = &tdx_supported_cpuid->entries[tdx_supported_cpuid->nent++];
+        e->function = function;
+        e->index = index;
+    }
+
+    return e;
+}
+
+static void tdx_add_supported_cpuid_by_fixed1_bits(void)
+{
+    struct kvm_cpuid_entry2 *e, *e1;
+    int i;
+
+    for (i = 0; i < tdx_fixed1_bits.cpuid.nent; i++) {
+        e = &tdx_fixed1_bits.entries[i];
+
+        e1 = find_in_supported_entry(e->function, e->index);
+        e1->eax |= e->eax;
+        e1->ebx |= e->ebx;
+        e1->ecx |= e->ecx;
+        e1->edx |= e->edx;
+    }
+}
+
 static void tdx_setup_supported_cpuid(void)
 {
     if (tdx_supported_cpuid) {
@@ -381,6 +506,8 @@ static void tdx_setup_supported_cpuid(void)
     memcpy(tdx_supported_cpuid->entries, tdx_caps->cpuid.entries,
            tdx_caps->cpuid.nent * sizeof(struct kvm_cpuid_entry2));
     tdx_supported_cpuid->nent = tdx_caps->cpuid.nent;
+
+    tdx_add_supported_cpuid_by_fixed1_bits();
 }
 
 static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
@@ -465,6 +592,11 @@ static uint32_t tdx_adjust_cpuid_features(X86ConfidentialGuest *cg,
 {
     struct kvm_cpuid_entry2 *e;
 
+    e = cpuid_find_entry(&tdx_fixed1_bits.cpuid, feature, index);
+    if (e) {
+        value |= cpuid_entry_get_reg(e, reg);
+    }
+
     if (is_feature_word_cpuid(feature, index, reg)) {
         e = cpuid_find_entry(tdx_supported_cpuid, feature, index);
         if (e) {
diff --git a/target/i386/sev.c b/target/i386/sev.c
index a6c0a697250b..217b19ad7bc6 100644
--- a/target/i386/sev.c
+++ b/target/i386/sev.c
@@ -214,11 +214,6 @@ static const char *const sev_fw_errlist[] = {
 /* <linux/kvm.h> doesn't expose this, so re-use the max from kvm.c */
 #define KVM_MAX_CPUID_ENTRIES 100
 
-typedef struct KvmCpuidInfo {
-    struct kvm_cpuid2 cpuid;
-    struct kvm_cpuid_entry2 entries[KVM_MAX_CPUID_ENTRIES];
-} KvmCpuidInfo;
-
 #define SNP_CPUID_FUNCTION_MAXCOUNT 64
 #define SNP_CPUID_FUNCTION_UNKNOWN 0xFFFFFFFF
 
-- 
2.34.1

Re: [PATCH v8 45/55] i386/tdx: Add TDX fixed1 bits to supported CPUIDs

[Daniel P. Berrangé]

On Tue, Apr 01, 2025 at 09:01:55AM -0400, Xiaoyao Li wrote:
> TDX architecture forcibly sets some CPUID bits for TD guest that VMM
> cannot disable it. They are fixed1 bits.
> 
> Fixed1 bits are not covered by tdx_caps.cpuid (which only contians the

Typo   s/contians/contains/

> directly configurable bits), while fixed1 bits are supported for TD guest
> obviously.
> 
> Add fixed1 bits to tdx_supported_cpuid. Besides, set all the fixed1
> bits to the initial set of KVM's support since KVM might not report them
> as supported.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
>  target/i386/cpu.h          |   2 +
>  target/i386/kvm/kvm_i386.h |   7 ++
>  target/i386/kvm/tdx.c      | 132 +++++++++++++++++++++++++++++++++++++
>  target/i386/sev.c          |   5 --
>  4 files changed, 141 insertions(+), 5 deletions(-)
> 
> diff --git a/target/i386/cpu.h b/target/i386/cpu.h
> index 42ef77789ded..115137279a1a 100644
> --- a/target/i386/cpu.h
> +++ b/target/i386/cpu.h
> @@ -924,6 +924,8 @@ uint64_t x86_cpu_get_supported_feature_word(X86CPU *cpu, FeatureWord w);
>  #define CPUID_7_0_EDX_FSRM              (1U << 4)
>  /* AVX512 Vector Pair Intersection to a Pair of Mask Registers */
>  #define CPUID_7_0_EDX_AVX512_VP2INTERSECT (1U << 8)
> + /* "md_clear" VERW clears CPU buffers */
> +#define CPUID_7_0_EDX_MD_CLEAR          (1U << 10)
>  /* SERIALIZE instruction */
>  #define CPUID_7_0_EDX_SERIALIZE         (1U << 14)
>  /* TSX Suspend Load Address Tracking instruction */
> diff --git a/target/i386/kvm/kvm_i386.h b/target/i386/kvm/kvm_i386.h
> index 484a1de84d51..c1bafcfc9b63 100644
> --- a/target/i386/kvm/kvm_i386.h
> +++ b/target/i386/kvm/kvm_i386.h
> @@ -13,8 +13,15 @@
>  
>  #include "system/kvm.h"
>  
> +#include <linux/kvm.h>
> +
>  #define KVM_MAX_CPUID_ENTRIES  100
>  
> +typedef struct KvmCpuidInfo {
> +    struct kvm_cpuid2 cpuid;
> +    struct kvm_cpuid_entry2 entries[KVM_MAX_CPUID_ENTRIES];
> +} KvmCpuidInfo;
> +
>  /* always false if !CONFIG_KVM */
>  #define kvm_pit_in_kernel() \
>      (kvm_irqchip_in_kernel() && !kvm_irqchip_is_split())
> diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
> index b9a96c2e392d..49a94d8ffe7d 100644
> --- a/target/i386/kvm/tdx.c
> +++ b/target/i386/kvm/tdx.c
> @@ -369,6 +369,131 @@ static Notifier tdx_machine_done_notify = {
>      .notify = tdx_finalize_vm,
>  };
>  
> +/*
> + * Some CPUID bits change from fixed1 to configurable bits when TDX module
> + * supports TDX_FEATURES0.VE_REDUCTION. e.g., MCA/MCE/MTRR/CORE_CAPABILITY.
> + *
> + * To make QEMU work with all the versions of TDX module, keep the fixed1 bits
> + * here if they are ever fixed1 bits in any of the version though not fixed1 in
> + * the latest version. Otherwise, with the older version of TDX module, QEMU may
> + * treat the fixed1 bit as unsupported.
> + *
> + * For newer TDX module, it does no harm to keep them in tdx_fixed1_bits even
> + * though they changed to configurable bits. Because tdx_fixed1_bits is used to
> + * setup the supported bits.
> + */
> +KvmCpuidInfo tdx_fixed1_bits = {
> +    .cpuid.nent = 8,
> +    .entries[0] = {
> +        .function = 0x1,
> +        .index = 0,
> +        .ecx = CPUID_EXT_SSE3 | CPUID_EXT_PCLMULQDQ | CPUID_EXT_DTES64 |
> +               CPUID_EXT_DSCPL | CPUID_EXT_SSSE3 | CPUID_EXT_CX16 |
> +               CPUID_EXT_PDCM | CPUID_EXT_PCID | CPUID_EXT_SSE41 |
> +               CPUID_EXT_SSE42 | CPUID_EXT_X2APIC | CPUID_EXT_MOVBE |
> +               CPUID_EXT_POPCNT | CPUID_EXT_AES | CPUID_EXT_XSAVE |
> +               CPUID_EXT_RDRAND | CPUID_EXT_HYPERVISOR,
> +        .edx = CPUID_FP87 | CPUID_VME | CPUID_DE | CPUID_PSE | CPUID_TSC |
> +               CPUID_MSR | CPUID_PAE | CPUID_MCE | CPUID_CX8 | CPUID_APIC |
> +               CPUID_SEP | CPUID_MTRR | CPUID_PGE | CPUID_MCA | CPUID_CMOV |
> +               CPUID_PAT | CPUID_CLFLUSH | CPUID_DTS | CPUID_MMX | CPUID_FXSR |
> +               CPUID_SSE | CPUID_SSE2,
> +    },
> +    .entries[1] = {
> +        .function = 0x6,
> +        .index = 0,
> +        .eax = CPUID_6_EAX_ARAT,
> +    },
> +    .entries[2] = {
> +        .function = 0x7,
> +        .index = 0,
> +        .flags = KVM_CPUID_FLAG_SIGNIFCANT_INDEX,
> +        .ebx = CPUID_7_0_EBX_FSGSBASE | CPUID_7_0_EBX_FDP_EXCPTN_ONLY |
> +               CPUID_7_0_EBX_SMEP | CPUID_7_0_EBX_INVPCID |
> +               CPUID_7_0_EBX_ZERO_FCS_FDS | CPUID_7_0_EBX_RDSEED |
> +               CPUID_7_0_EBX_SMAP | CPUID_7_0_EBX_CLFLUSHOPT |
> +               CPUID_7_0_EBX_CLWB | CPUID_7_0_EBX_SHA_NI,
> +        .ecx = CPUID_7_0_ECX_BUS_LOCK_DETECT | CPUID_7_0_ECX_MOVDIRI |
> +               CPUID_7_0_ECX_MOVDIR64B,
> +        .edx = CPUID_7_0_EDX_MD_CLEAR | CPUID_7_0_EDX_SPEC_CTRL |
> +               CPUID_7_0_EDX_STIBP | CPUID_7_0_EDX_FLUSH_L1D |
> +               CPUID_7_0_EDX_ARCH_CAPABILITIES | CPUID_7_0_EDX_CORE_CAPABILITY |
> +               CPUID_7_0_EDX_SPEC_CTRL_SSBD,
> +    },
> +    .entries[3] = {
> +        .function = 0x7,
> +        .index = 2,
> +        .flags = KVM_CPUID_FLAG_SIGNIFCANT_INDEX,
> +        .edx = (1U << 0) | (1U << 1) | (1U << 2) | (1U << 4),
> +    },
> +    .entries[4] = {
> +        .function = 0xD,
> +        .index = 0,
> +        .flags = KVM_CPUID_FLAG_SIGNIFCANT_INDEX,
> +        .eax = XSTATE_FP_MASK | XSTATE_SSE_MASK,
> +    },
> +    .entries[5] = {
> +        .function = 0xD,
> +        .index = 1,
> +        .flags = KVM_CPUID_FLAG_SIGNIFCANT_INDEX,
> +        .eax = CPUID_XSAVE_XSAVEOPT | CPUID_XSAVE_XSAVEC|
> +               CPUID_XSAVE_XGETBV1 | CPUID_XSAVE_XSAVES,
> +    },
> +    .entries[6] = {
> +        .function = 0x80000001,
> +        .index = 0,
> +        .ecx = CPUID_EXT3_LAHF_LM | CPUID_EXT3_ABM | CPUID_EXT3_3DNOWPREFETCH,
> +        /* strictly speaking, SYSCALL is not fixed1 bit since it depends on
> +         * the CPU to be in 64-bit mode. But here fixed1 is used to serve the
> +         * purpose of supported bits for TDX. In this sense, SYACALL is always
> +         * supported.
> +         */
> +        .edx = CPUID_EXT2_SYSCALL | CPUID_EXT2_NX | CPUID_EXT2_PDPE1GB |
> +               CPUID_EXT2_RDTSCP | CPUID_EXT2_LM,
> +    },
> +    .entries[7] = {
> +        .function = 0x80000007,
> +        .index = 0,
> +        .edx = CPUID_APM_INVTSC,
> +    },
> +};
> +
> +static struct kvm_cpuid_entry2 *find_in_supported_entry(uint32_t function,
> +                                                        uint32_t index)
> +{
> +    struct kvm_cpuid_entry2 *e;
> +
> +    e = cpuid_find_entry(tdx_supported_cpuid, function, index);
> +    if (!e) {
> +        if (tdx_supported_cpuid->nent >= KVM_MAX_CPUID_ENTRIES) {
> +            error_report("tdx_supported_cpuid requries more space than %d entries",
> +                          KVM_MAX_CPUID_ENTRIES);
> +            exit(1);
> +        }
> +        e = &tdx_supported_cpuid->entries[tdx_supported_cpuid->nent++];
> +        e->function = function;
> +        e->index = index;
> +    }
> +
> +    return e;
> +}
> +
> +static void tdx_add_supported_cpuid_by_fixed1_bits(void)
> +{
> +    struct kvm_cpuid_entry2 *e, *e1;
> +    int i;
> +
> +    for (i = 0; i < tdx_fixed1_bits.cpuid.nent; i++) {
> +        e = &tdx_fixed1_bits.entries[i];
> +
> +        e1 = find_in_supported_entry(e->function, e->index);
> +        e1->eax |= e->eax;
> +        e1->ebx |= e->ebx;
> +        e1->ecx |= e->ecx;
> +        e1->edx |= e->edx;
> +    }
> +}
> +
>  static void tdx_setup_supported_cpuid(void)
>  {
>      if (tdx_supported_cpuid) {
> @@ -381,6 +506,8 @@ static void tdx_setup_supported_cpuid(void)
>      memcpy(tdx_supported_cpuid->entries, tdx_caps->cpuid.entries,
>             tdx_caps->cpuid.nent * sizeof(struct kvm_cpuid_entry2));
>      tdx_supported_cpuid->nent = tdx_caps->cpuid.nent;
> +
> +    tdx_add_supported_cpuid_by_fixed1_bits();
>  }
>  
>  static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
> @@ -465,6 +592,11 @@ static uint32_t tdx_adjust_cpuid_features(X86ConfidentialGuest *cg,
>  {
>      struct kvm_cpuid_entry2 *e;
>  
> +    e = cpuid_find_entry(&tdx_fixed1_bits.cpuid, feature, index);
> +    if (e) {
> +        value |= cpuid_entry_get_reg(e, reg);
> +    }
> +
>      if (is_feature_word_cpuid(feature, index, reg)) {
>          e = cpuid_find_entry(tdx_supported_cpuid, feature, index);
>          if (e) {
> diff --git a/target/i386/sev.c b/target/i386/sev.c
> index a6c0a697250b..217b19ad7bc6 100644
> --- a/target/i386/sev.c
> +++ b/target/i386/sev.c
> @@ -214,11 +214,6 @@ static const char *const sev_fw_errlist[] = {
>  /* <linux/kvm.h> doesn't expose this, so re-use the max from kvm.c */
>  #define KVM_MAX_CPUID_ENTRIES 100
>  
> -typedef struct KvmCpuidInfo {
> -    struct kvm_cpuid2 cpuid;
> -    struct kvm_cpuid_entry2 entries[KVM_MAX_CPUID_ENTRIES];
> -} KvmCpuidInfo;
> -
>  #define SNP_CPUID_FUNCTION_MAXCOUNT 64
>  #define SNP_CPUID_FUNCTION_UNKNOWN 0xFFFFFFFF
>  
> -- 
> 2.34.1
> 

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v8 45/55] i386/tdx: Add TDX fixed1 bits to supported CPUIDs

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:55AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:55 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 45/55] i386/tdx: Add TDX fixed1 bits to supported CPUIDs
> X-Mailer: git-send-email 2.34.1
> 
> TDX architecture forcibly sets some CPUID bits for TD guest that VMM
> cannot disable it. They are fixed1 bits.
> 
> Fixed1 bits are not covered by tdx_caps.cpuid (which only contians the
> directly configurable bits), while fixed1 bits are supported for TD guest
> obviously.
> 
> Add fixed1 bits to tdx_supported_cpuid. Besides, set all the fixed1
> bits to the initial set of KVM's support since KVM might not report them
> as supported.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
>  target/i386/cpu.h          |   2 +
>  target/i386/kvm/kvm_i386.h |   7 ++
>  target/i386/kvm/tdx.c      | 132 +++++++++++++++++++++++++++++++++++++
>  target/i386/sev.c          |   5 --
>  4 files changed, 141 insertions(+), 5 deletions(-)
 
LGTM, (only some nits)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

> +    .entries[3] = {
> +        .function = 0x7,
> +        .index = 2,
> +        .flags = KVM_CPUID_FLAG_SIGNIFCANT_INDEX,
> +        .edx = (1U << 0) | (1U << 1) | (1U << 2) | (1U << 4),

Missed to use macro?

.edx = CPUID_7_2_EDX_PSFD | CPUID_7_2_EDX_IPRED_CTRL |
       CPUID_7_2_EDX_RRSBA_CTRL | CPUID_7_2_EDX_BHI_CTRL

> +    },
> +    .entries[4] = {
> +        .function = 0xD,
> +        .index = 0,
> +        .flags = KVM_CPUID_FLAG_SIGNIFCANT_INDEX,
> +        .eax = XSTATE_FP_MASK | XSTATE_SSE_MASK,
> +    },
> +    .entries[5] = {
> +        .function = 0xD,
> +        .index = 1,
> +        .flags = KVM_CPUID_FLAG_SIGNIFCANT_INDEX,
> +        .eax = CPUID_XSAVE_XSAVEOPT | CPUID_XSAVE_XSAVEC|
> +               CPUID_XSAVE_XGETBV1 | CPUID_XSAVE_XSAVES,
> +    },
> +    .entries[6] = {
> +        .function = 0x80000001,
> +        .index = 0,
> +        .ecx = CPUID_EXT3_LAHF_LM | CPUID_EXT3_ABM | CPUID_EXT3_3DNOWPREFETCH,
> +        /* strictly speaking, SYSCALL is not fixed1 bit since it depends on

style nit?

/*
 * strictly ...
 */

> +         * the CPU to be in 64-bit mode. But here fixed1 is used to serve the
> +         * purpose of supported bits for TDX. In this sense, SYACALL is always
> +         * supported.
> +         */
> +        .edx = CPUID_EXT2_SYSCALL | CPUID_EXT2_NX | CPUID_EXT2_PDPE1GB |
> +               CPUID_EXT2_RDTSCP | CPUID_EXT2_LM,
> +    },
> +    .entries[7] = {
> +        .function = 0x80000007,
> +        .index = 0,
> +        .edx = CPUID_APM_INVTSC,
> +    },
> +};
> +

...

> diff --git a/target/i386/sev.c b/target/i386/sev.c
> index a6c0a697250b..217b19ad7bc6 100644
> --- a/target/i386/sev.c
> +++ b/target/i386/sev.c
> @@ -214,11 +214,6 @@ static const char *const sev_fw_errlist[] = {
>  /* <linux/kvm.h> doesn't expose this, so re-use the max from kvm.c */
>  #define KVM_MAX_CPUID_ENTRIES 100

This macro can also be cleaned up in this patch or in your patch 5.

[PATCH v8 46/55] i386/tdx: Add supported CPUID bits related to TD Attributes

[Xiaoyao Li]

For TDX, some CPUID feature bit is configured via TD attributes. They
are not covered by tdx_caps.cpuid (which only contians the configurable
bits), but they are actually supported when the related attributre bit
is supported.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
 target/i386/cpu.h     |  4 +++
 target/i386/kvm/tdx.c | 59 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 63 insertions(+)

diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 115137279a1a..0e984ec42bb6 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -903,6 +903,8 @@ uint64_t x86_cpu_get_supported_feature_word(X86CPU *cpu, FeatureWord w);
 #define CPUID_7_0_ECX_LA57              (1U << 16)
 /* Read Processor ID */
 #define CPUID_7_0_ECX_RDPID             (1U << 22)
+/* KeyLocker */
+#define CPUID_7_0_ECX_KeyLocker         (1U << 23)
 /* Bus Lock Debug Exception */
 #define CPUID_7_0_ECX_BUS_LOCK_DETECT   (1U << 24)
 /* Cache Line Demote Instruction */
@@ -963,6 +965,8 @@ uint64_t x86_cpu_get_supported_feature_word(X86CPU *cpu, FeatureWord w);
 #define CPUID_7_1_EAX_AVX_VNNI          (1U << 4)
 /* AVX512 BFloat16 Instruction */
 #define CPUID_7_1_EAX_AVX512_BF16       (1U << 5)
+/* Linear address space separation */
+#define CPUID_7_1_EAX_LASS              (1U << 6)
 /* CMPCCXADD Instructions */
 #define CPUID_7_1_EAX_CMPCCXADD         (1U << 7)
 /* Fast Zero REP MOVS */
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 49a94d8ffe7d..5d72f1814606 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -458,6 +458,33 @@ KvmCpuidInfo tdx_fixed1_bits = {
     },
 };
 
+typedef struct TdxAttrsMap {
+    uint32_t attr_index;
+    uint32_t cpuid_leaf;
+    uint32_t cpuid_subleaf;
+    int cpuid_reg;
+    uint32_t feat_mask;
+} TdxAttrsMap;
+
+static TdxAttrsMap tdx_attrs_maps[] = {
+    {.attr_index = 27,
+     .cpuid_leaf = 7,
+     .cpuid_subleaf = 1,
+     .cpuid_reg = R_EAX,
+     .feat_mask = CPUID_7_1_EAX_LASS},
+    {.attr_index = 30,
+     .cpuid_leaf = 7,
+     .cpuid_subleaf = 0,
+     .cpuid_reg = R_ECX,
+     .feat_mask = CPUID_7_0_ECX_PKS,},
+    {.attr_index = 31,
+     .cpuid_leaf = 7,
+     .cpuid_subleaf = 0,
+     .cpuid_reg = R_ECX,
+     .feat_mask = CPUID_7_0_ECX_KeyLocker,
+    },
+};
+
 static struct kvm_cpuid_entry2 *find_in_supported_entry(uint32_t function,
                                                         uint32_t index)
 {
@@ -494,6 +521,37 @@ static void tdx_add_supported_cpuid_by_fixed1_bits(void)
     }
 }
 
+static void tdx_add_supported_cpuid_by_attrs(void)
+{
+    struct kvm_cpuid_entry2 *e;
+    TdxAttrsMap *map;
+    int i;
+
+    for (i = 0; i < ARRAY_SIZE(tdx_attrs_maps); i++) {
+        map = &tdx_attrs_maps[i];
+        if (!((1ULL << map->attr_index) & tdx_caps->supported_attrs)) {
+            continue;
+        }
+
+        e = find_in_supported_entry(map->cpuid_leaf, map->cpuid_subleaf);
+
+        switch(map->cpuid_reg) {
+        case R_EAX:
+            e->eax |= map->feat_mask;
+            break;
+        case R_EBX:
+            e->ebx |= map->feat_mask;
+            break;
+        case R_ECX:
+            e->ecx |= map->feat_mask;
+            break;
+        case R_EDX:
+            e->edx |= map->feat_mask;
+            break;
+        }
+    }
+}
+
 static void tdx_setup_supported_cpuid(void)
 {
     if (tdx_supported_cpuid) {
@@ -508,6 +566,7 @@ static void tdx_setup_supported_cpuid(void)
     tdx_supported_cpuid->nent = tdx_caps->cpuid.nent;
 
     tdx_add_supported_cpuid_by_fixed1_bits();
+    tdx_add_supported_cpuid_by_attrs();
 }
 
 static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
-- 
2.34.1

Re: [PATCH v8 46/55] i386/tdx: Add supported CPUID bits related to TD Attributes

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:56AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:56 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 46/55] i386/tdx: Add supported CPUID bits related to TD
>  Attributes
> X-Mailer: git-send-email 2.34.1
> 
> For TDX, some CPUID feature bit is configured via TD attributes. They
> are not covered by tdx_caps.cpuid (which only contians the configurable
> bits), but they are actually supported when the related attributre bit
> is supported.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
>  target/i386/cpu.h     |  4 +++
>  target/i386/kvm/tdx.c | 59 +++++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 63 insertions(+)
> 
> diff --git a/target/i386/cpu.h b/target/i386/cpu.h
> index 115137279a1a..0e984ec42bb6 100644
> --- a/target/i386/cpu.h
> +++ b/target/i386/cpu.h
> @@ -903,6 +903,8 @@ uint64_t x86_cpu_get_supported_feature_word(X86CPU *cpu, FeatureWord w);
>  #define CPUID_7_0_ECX_LA57              (1U << 16)
>  /* Read Processor ID */
>  #define CPUID_7_0_ECX_RDPID             (1U << 22)
> +/* KeyLocker */
> +#define CPUID_7_0_ECX_KeyLocker         (1U << 23)
>  /* Bus Lock Debug Exception */
>  #define CPUID_7_0_ECX_BUS_LOCK_DETECT   (1U << 24)
>  /* Cache Line Demote Instruction */
> @@ -963,6 +965,8 @@ uint64_t x86_cpu_get_supported_feature_word(X86CPU *cpu, FeatureWord w);
>  #define CPUID_7_1_EAX_AVX_VNNI          (1U << 4)
>  /* AVX512 BFloat16 Instruction */
>  #define CPUID_7_1_EAX_AVX512_BF16       (1U << 5)
> +/* Linear address space separation */
> +#define CPUID_7_1_EAX_LASS              (1U << 6)
>  /* CMPCCXADD Instructions */
>  #define CPUID_7_1_EAX_CMPCCXADD         (1U << 7)
>  /* Fast Zero REP MOVS */

The whole framework is very good for me. 

Since LASS/key locker are not yet supported, I think it's better to add
the note (in the commit message or code) that they are helpful for
future enabling. Of course, removing these two features would be best,
as they cannot be covered by current testing.

Regards,
Zhao

Re: [PATCH v8 46/55] i386/tdx: Add supported CPUID bits related to TD Attributes

[Xiaoyao Li]

On 5/6/2025 7:31 PM, Zhao Liu wrote:
> On Tue, Apr 01, 2025 at 09:01:56AM -0400, Xiaoyao Li wrote:
>> Date: Tue,  1 Apr 2025 09:01:56 -0400
>> From: Xiaoyao Li <xiaoyao.li@intel.com>
>> Subject: [PATCH v8 46/55] i386/tdx: Add supported CPUID bits related to TD
>>   Attributes
>> X-Mailer: git-send-email 2.34.1
>>
>> For TDX, some CPUID feature bit is configured via TD attributes. They
>> are not covered by tdx_caps.cpuid (which only contians the configurable
>> bits), but they are actually supported when the related attributre bit
>> is supported.
>>
>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>> ---
>>   target/i386/cpu.h     |  4 +++
>>   target/i386/kvm/tdx.c | 59 +++++++++++++++++++++++++++++++++++++++++++
>>   2 files changed, 63 insertions(+)
>>
>> diff --git a/target/i386/cpu.h b/target/i386/cpu.h
>> index 115137279a1a..0e984ec42bb6 100644
>> --- a/target/i386/cpu.h
>> +++ b/target/i386/cpu.h
>> @@ -903,6 +903,8 @@ uint64_t x86_cpu_get_supported_feature_word(X86CPU *cpu, FeatureWord w);
>>   #define CPUID_7_0_ECX_LA57              (1U << 16)
>>   /* Read Processor ID */
>>   #define CPUID_7_0_ECX_RDPID             (1U << 22)
>> +/* KeyLocker */
>> +#define CPUID_7_0_ECX_KeyLocker         (1U << 23)
>>   /* Bus Lock Debug Exception */
>>   #define CPUID_7_0_ECX_BUS_LOCK_DETECT   (1U << 24)
>>   /* Cache Line Demote Instruction */
>> @@ -963,6 +965,8 @@ uint64_t x86_cpu_get_supported_feature_word(X86CPU *cpu, FeatureWord w);
>>   #define CPUID_7_1_EAX_AVX_VNNI          (1U << 4)
>>   /* AVX512 BFloat16 Instruction */
>>   #define CPUID_7_1_EAX_AVX512_BF16       (1U << 5)
>> +/* Linear address space separation */
>> +#define CPUID_7_1_EAX_LASS              (1U << 6)
>>   /* CMPCCXADD Instructions */
>>   #define CPUID_7_1_EAX_CMPCCXADD         (1U << 7)
>>   /* Fast Zero REP MOVS */
> 
> The whole framework is very good for me.
> 
> Since LASS/key locker are not yet supported, I think it's better to add
> the note (in the commit message or code) that they are helpful for
> future enabling. Of course, removing these two features would be best,
> as they cannot be covered by current testing.

I will add note in the commit message and don't change the code.

Keep them in tdx_attrs_maps[] is helpful for the completeness of the 
existing attribute bits that are related to a CPUID feature.

> Regards,
> Zhao
> 
> 

[PATCH v8 47/55] i386/tdx: Add supported CPUID bits relates to XFAM

[Xiaoyao Li]

Some CPUID bits are controlled by XFAM. They are not covered by
tdx_caps.cpuid (which only contians the directly configurable bits), but
they are actually supported when the related XFAM bit is supported.

Add these XFAM controlled bits to TDX supported CPUID bits based on the
supported_xfam.

Besides, incorporate the supported_xfam into the supported CPUID leaf of
0xD.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
 target/i386/cpu.c     | 12 -------
 target/i386/cpu.h     | 16 ++++++++++
 target/i386/kvm/tdx.c | 73 +++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 89 insertions(+), 12 deletions(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 4061d38d3fbe..f09a1caac071 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -1673,15 +1673,6 @@ bool is_feature_word_cpuid(uint32_t feature, uint32_t index, int reg)
     return false;
 }
 
-typedef struct FeatureMask {
-    FeatureWord index;
-    uint64_t mask;
-} FeatureMask;
-
-typedef struct FeatureDep {
-    FeatureMask from, to;
-} FeatureDep;
-
 static FeatureDep feature_dependencies[] = {
     {
         .from = { FEAT_7_0_EDX,             CPUID_7_0_EDX_ARCH_CAPABILITIES },
@@ -1850,9 +1841,6 @@ static const X86RegisterInfo32 x86_reg_info_32[CPU_NB_REGS32] = {
 };
 #undef REGISTER
 
-/* CPUID feature bits available in XSS */
-#define CPUID_XSTATE_XSS_MASK    (XSTATE_ARCH_LBR_MASK)
-
 ExtSaveArea x86_ext_save_areas[XSAVE_STATE_AREA_COUNT] = {
     [XSTATE_FP_BIT] = {
         /* x87 FP state component is always enabled if XSAVE is supported */
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 0e984ec42bb6..132312d70a54 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -588,6 +588,7 @@ typedef enum X86Seg {
 #define XSTATE_OPMASK_BIT               5
 #define XSTATE_ZMM_Hi256_BIT            6
 #define XSTATE_Hi16_ZMM_BIT             7
+#define XSTATE_PT_BIT                   8
 #define XSTATE_PKRU_BIT                 9
 #define XSTATE_ARCH_LBR_BIT             15
 #define XSTATE_XTILE_CFG_BIT            17
@@ -601,6 +602,7 @@ typedef enum X86Seg {
 #define XSTATE_OPMASK_MASK              (1ULL << XSTATE_OPMASK_BIT)
 #define XSTATE_ZMM_Hi256_MASK           (1ULL << XSTATE_ZMM_Hi256_BIT)
 #define XSTATE_Hi16_ZMM_MASK            (1ULL << XSTATE_Hi16_ZMM_BIT)
+#define XSTATE_PT_MASK                  (1ULL << XSTATE_PT_BIT)
 #define XSTATE_PKRU_MASK                (1ULL << XSTATE_PKRU_BIT)
 #define XSTATE_ARCH_LBR_MASK            (1ULL << XSTATE_ARCH_LBR_BIT)
 #define XSTATE_XTILE_CFG_MASK           (1ULL << XSTATE_XTILE_CFG_BIT)
@@ -623,6 +625,11 @@ typedef enum X86Seg {
                                  XSTATE_Hi16_ZMM_MASK | XSTATE_PKRU_MASK | \
                                  XSTATE_XTILE_CFG_MASK | XSTATE_XTILE_DATA_MASK)
 
+/* CPUID feature bits available in XSS */
+#define CPUID_XSTATE_XSS_MASK    (XSTATE_ARCH_LBR_MASK)
+
+#define CPUID_XSTATE_MASK       (CPUID_XSTATE_XCR0_MASK | CPUID_XSTATE_XSS_MASK)
+
 /* CPUID feature words */
 typedef enum FeatureWord {
     FEAT_1_EDX,         /* CPUID[1].EDX */
@@ -671,6 +678,15 @@ typedef enum FeatureWord {
     FEATURE_WORDS,
 } FeatureWord;
 
+typedef struct FeatureMask {
+    FeatureWord index;
+    uint64_t mask;
+} FeatureMask;
+
+typedef struct FeatureDep {
+    FeatureMask from, to;
+} FeatureDep;
+
 typedef uint64_t FeatureWordArray[FEATURE_WORDS];
 uint64_t x86_cpu_get_supported_feature_word(X86CPU *cpu, FeatureWord w);
 
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 5d72f1814606..c7b4a098d12f 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -23,6 +23,8 @@
 
 #include <linux/kvm_para.h>
 
+#include "cpu.h"
+#include "cpu-internal.h"
 #include "hw/i386/e820_memory_layout.h"
 #include "hw/i386/tdvf.h"
 #include "hw/i386/x86.h"
@@ -485,6 +487,32 @@ static TdxAttrsMap tdx_attrs_maps[] = {
     },
 };
 
+typedef struct TdxXFAMDep {
+    int xfam_bit;
+    FeatureMask feat_mask;
+} TdxXFAMDep;
+
+/*
+ * Note, only the CPUID bits whose virtualization type are "XFAM & Native" are
+ * defiend here.
+ *
+ * For those whose virtualization type are "XFAM & Configured & Native", they
+ * are reported as configurable bits. And they are not supported if not in the
+ * configureable bits list from KVM even if the corresponding XFAM bit is
+ * supported.
+ */
+TdxXFAMDep tdx_xfam_deps[] = {
+    { XSTATE_YMM_BIT,       { FEAT_1_ECX, CPUID_EXT_FMA }},
+    { XSTATE_YMM_BIT,       { FEAT_7_0_EBX, CPUID_7_0_EBX_AVX2 }},
+    { XSTATE_OPMASK_BIT,    { FEAT_7_0_ECX, CPUID_7_0_ECX_AVX512_VBMI}},
+    { XSTATE_OPMASK_BIT,    { FEAT_7_0_EDX, CPUID_7_0_EDX_AVX512_FP16}},
+    { XSTATE_PT_BIT,        { FEAT_7_0_EBX, CPUID_7_0_EBX_INTEL_PT}},
+    { XSTATE_PKRU_BIT,      { FEAT_7_0_ECX, CPUID_7_0_ECX_PKU}},
+    { XSTATE_XTILE_CFG_BIT, { FEAT_7_0_EDX, CPUID_7_0_EDX_AMX_BF16 }},
+    { XSTATE_XTILE_CFG_BIT, { FEAT_7_0_EDX, CPUID_7_0_EDX_AMX_TILE }},
+    { XSTATE_XTILE_CFG_BIT, { FEAT_7_0_EDX, CPUID_7_0_EDX_AMX_INT8 }},
+};
+
 static struct kvm_cpuid_entry2 *find_in_supported_entry(uint32_t function,
                                                         uint32_t index)
 {
@@ -552,6 +580,50 @@ static void tdx_add_supported_cpuid_by_attrs(void)
     }
 }
 
+static void tdx_add_supported_cpuid_by_xfam(void)
+{
+    struct kvm_cpuid_entry2 *e;
+    int i;
+
+    const TdxXFAMDep *xfam_dep;
+    const FeatureWordInfo *f;
+    for (i = 0; i < ARRAY_SIZE(tdx_xfam_deps); i++) {
+        xfam_dep = &tdx_xfam_deps[i];
+        if (!((1ULL << xfam_dep->xfam_bit) & tdx_caps->supported_xfam)) {
+            continue;
+        }
+
+        f = &feature_word_info[xfam_dep->feat_mask.index];
+        if (f->type != CPUID_FEATURE_WORD) {
+            continue;
+        }
+
+        e = find_in_supported_entry(f->cpuid.eax, f->cpuid.ecx);
+        switch(f->cpuid.reg) {
+        case R_EAX:
+            e->eax |= xfam_dep->feat_mask.mask;
+            break;
+        case R_EBX:
+            e->ebx |= xfam_dep->feat_mask.mask;
+            break;
+        case R_ECX:
+            e->ecx |= xfam_dep->feat_mask.mask;
+            break;
+        case R_EDX:
+            e->edx |= xfam_dep->feat_mask.mask;
+            break;
+        }
+    }
+
+    e = find_in_supported_entry(0xd, 0);
+    e->eax |= (tdx_caps->supported_xfam & CPUID_XSTATE_XCR0_MASK);
+    e->edx |= (tdx_caps->supported_xfam & CPUID_XSTATE_XCR0_MASK) >> 32;
+
+    e = find_in_supported_entry(0xd, 1);
+    e->ecx |= (tdx_caps->supported_xfam & CPUID_XSTATE_XSS_MASK);
+    e->edx |= (tdx_caps->supported_xfam & CPUID_XSTATE_XSS_MASK) >> 32;
+}
+
 static void tdx_setup_supported_cpuid(void)
 {
     if (tdx_supported_cpuid) {
@@ -567,6 +639,7 @@ static void tdx_setup_supported_cpuid(void)
 
     tdx_add_supported_cpuid_by_fixed1_bits();
     tdx_add_supported_cpuid_by_attrs();
+    tdx_add_supported_cpuid_by_xfam();
 }
 
 static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
-- 
2.34.1

[PATCH v8 48/55] i386/tdx: Add XFD to supported bit of TDX

[Xiaoyao Li]

Just mark XFD as always supported for TDX. This simple solution relies
on the fact KVM will report XFD as 0 when it's not supported by the
hardware.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
 target/i386/cpu.h     | 1 +
 target/i386/kvm/tdx.c | 5 +++++
 2 files changed, 6 insertions(+)

diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 132312d70a54..a223e09a25c4 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -1126,6 +1126,7 @@ uint64_t x86_cpu_get_supported_feature_word(X86CPU *cpu, FeatureWord w);
 #define CPUID_XSAVE_XSAVEC     (1U << 1)
 #define CPUID_XSAVE_XGETBV1    (1U << 2)
 #define CPUID_XSAVE_XSAVES     (1U << 3)
+#define CPUID_XSAVE_XFD        (1U << 4)
 
 #define CPUID_6_EAX_ARAT       (1U << 2)
 
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index c7b4a098d12f..e07cd9a1ee15 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -620,6 +620,11 @@ static void tdx_add_supported_cpuid_by_xfam(void)
     e->edx |= (tdx_caps->supported_xfam & CPUID_XSTATE_XCR0_MASK) >> 32;
 
     e = find_in_supported_entry(0xd, 1);
+    /* Mark XFD always support for TDX, it will be cleared finally in
+     * tdx_adjust_cpuid_features() if XFD is unavailable on the hardware
+     * because in this case the original data has it as 0.
+     */
+    e->eax |= CPUID_XSAVE_XFD;
     e->ecx |= (tdx_caps->supported_xfam & CPUID_XSTATE_XSS_MASK);
     e->edx |= (tdx_caps->supported_xfam & CPUID_XSTATE_XSS_MASK) >> 32;
 }
-- 
2.34.1

Re: [PATCH v8 48/55] i386/tdx: Add XFD to supported bit of TDX

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:58AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:58 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 48/55] i386/tdx: Add XFD to supported bit of TDX
> X-Mailer: git-send-email 2.34.1
> 
> Just mark XFD as always supported for TDX. This simple solution relies
> on the fact KVM will report XFD as 0 when it's not supported by the
> hardware.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
>  target/i386/cpu.h     | 1 +
>  target/i386/kvm/tdx.c | 5 +++++
>  2 files changed, 6 insertions(+)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 49/55] i386/tdx: Define supported KVM features for TDX

[Xiaoyao Li]

For TDX, only limited KVM PV features are supported.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
 target/i386/kvm/tdx.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index e07cd9a1ee15..7382b53fcc51 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -32,6 +32,8 @@
 #include "kvm_i386.h"
 #include "tdx.h"
 
+#include "standard-headers/asm-x86/kvm_para.h"
+
 #define TDX_MIN_TSC_FREQUENCY_KHZ   (100 * 1000)
 #define TDX_MAX_TSC_FREQUENCY_KHZ   (10 * 1000 * 1000)
 
@@ -44,6 +46,14 @@
                                  TDX_TD_ATTRIBUTES_PKS | \
                                  TDX_TD_ATTRIBUTES_PERFMON)
 
+#define TDX_SUPPORTED_KVM_FEATURES  ((1U << KVM_FEATURE_NOP_IO_DELAY) | \
+                                     (1U << KVM_FEATURE_PV_UNHALT) | \
+                                     (1U << KVM_FEATURE_PV_TLB_FLUSH) | \
+                                     (1U << KVM_FEATURE_PV_SEND_IPI) | \
+                                     (1U << KVM_FEATURE_POLL_CONTROL) | \
+                                     (1U << KVM_FEATURE_PV_SCHED_YIELD) | \
+                                     (1U << KVM_FEATURE_MSI_EXT_DEST_ID))
+
 static TdxGuest *tdx_guest;
 
 static struct kvm_tdx_capabilities *tdx_caps;
@@ -629,6 +639,14 @@ static void tdx_add_supported_cpuid_by_xfam(void)
     e->edx |= (tdx_caps->supported_xfam & CPUID_XSTATE_XSS_MASK) >> 32;
 }
 
+static void tdx_add_supported_kvm_features(void)
+{
+    struct kvm_cpuid_entry2 *e;
+
+    e = find_in_supported_entry(0x40000001, 0);
+    e->eax = TDX_SUPPORTED_KVM_FEATURES;
+}
+
 static void tdx_setup_supported_cpuid(void)
 {
     if (tdx_supported_cpuid) {
@@ -645,6 +663,8 @@ static void tdx_setup_supported_cpuid(void)
     tdx_add_supported_cpuid_by_fixed1_bits();
     tdx_add_supported_cpuid_by_attrs();
     tdx_add_supported_cpuid_by_xfam();
+
+    tdx_add_supported_kvm_features();
 }
 
 static int tdx_kvm_init(ConfidentialGuestSupport *cgs, Error **errp)
-- 
2.34.1

Re: [PATCH v8 49/55] i386/tdx: Define supported KVM features for TDX

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:01:59AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:01:59 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 49/55] i386/tdx: Define supported KVM features for TDX
> X-Mailer: git-send-email 2.34.1
> 
> For TDX, only limited KVM PV features are supported.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
>  target/i386/kvm/tdx.c | 20 ++++++++++++++++++++
>  1 file changed, 20 insertions(+)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 50/55] i386/cgs: Introduce x86_confidential_guest_check_features()

[Xiaoyao Li]

To do cgs specific feature checking. Note the feature checking in
x86_cpu_filter_features() is valid for non-cgs VMs. For cgs VMs like
TDX, what features can be supported has more restrictions.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
 target/i386/confidential-guest.h | 13 +++++++++++++
 target/i386/kvm/kvm.c            |  8 ++++++++
 2 files changed, 21 insertions(+)

diff --git a/target/i386/confidential-guest.h b/target/i386/confidential-guest.h
index 777d43cc9688..48b88dbd3130 100644
--- a/target/i386/confidential-guest.h
+++ b/target/i386/confidential-guest.h
@@ -42,6 +42,7 @@ struct X86ConfidentialGuestClass {
     void (*cpu_instance_init)(X86ConfidentialGuest *cg, CPUState *cpu);
     uint32_t (*adjust_cpuid_features)(X86ConfidentialGuest *cg, uint32_t feature,
                                       uint32_t index, int reg, uint32_t value);
+    int (*check_features)(X86ConfidentialGuest *cg, CPUState *cs);
 };
 
 /**
@@ -91,4 +92,16 @@ static inline int x86_confidential_guest_adjust_cpuid_features(X86ConfidentialGu
     }
 }
 
+static inline int x86_confidential_guest_check_features(X86ConfidentialGuest *cg,
+                                                        CPUState *cs)
+{
+    X86ConfidentialGuestClass *klass = X86_CONFIDENTIAL_GUEST_GET_CLASS(cg);
+
+    if (klass->check_features) {
+        return klass->check_features(cg, cs);
+    }
+
+    return 0;
+}
+
 #endif
diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index 17d7bf6ae9aa..27b4a069d194 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -2092,6 +2092,14 @@ int kvm_arch_init_vcpu(CPUState *cs)
     int r;
     Error *local_err = NULL;
 
+    if (current_machine->cgs) {
+        r = x86_confidential_guest_check_features(
+                X86_CONFIDENTIAL_GUEST(current_machine->cgs), cs);
+        if (r < 0) {
+            return r;
+        }
+    }
+
     memset(&cpuid_data, 0, sizeof(cpuid_data));
 
     cpuid_i = 0;
-- 
2.34.1

Re: [PATCH v8 50/55] i386/cgs: Introduce x86_confidential_guest_check_features()

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:02:00AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:02:00 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 50/55] i386/cgs: Introduce
>  x86_confidential_guest_check_features()
> X-Mailer: git-send-email 2.34.1
> 
> To do cgs specific feature checking. Note the feature checking in
> x86_cpu_filter_features() is valid for non-cgs VMs. For cgs VMs like
> TDX, what features can be supported has more restrictions.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
>  target/i386/confidential-guest.h | 13 +++++++++++++
>  target/i386/kvm/kvm.c            |  8 ++++++++
>  2 files changed, 21 insertions(+)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 51/55] i386/tdx: Fetch and validate CPUID of TD guest

[Xiaoyao Li]

Use KVM_TDX_GET_CPUID to get the CPUIDs that are managed and enfored
by TDX module for TD guest. Check QEMU's configuration against the
fetched data.

Print wanring  message when 1. a feature is not supported but requested
by QEMU or 2. QEMU doesn't want to expose a feature while it is enforced
enabled.

- If cpu->enforced_cpuid is not set, prints the warning message of both
1) and 2) and tweak QEMU's configuration.

- If cpu->enforced_cpuid is set, quit if any case of 1) or 2).

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
 target/i386/cpu.c     | 33 ++++++++++++++-
 target/i386/cpu.h     |  7 +++
 target/i386/kvm/tdx.c | 99 +++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 137 insertions(+), 2 deletions(-)

diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index f09a1caac071..41407c8a7248 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -5772,8 +5772,8 @@ static bool x86_cpu_have_filtered_features(X86CPU *cpu)
     return false;
 }
 
-static void mark_unavailable_features(X86CPU *cpu, FeatureWord w, uint64_t mask,
-                                      const char *verbose_prefix)
+void mark_unavailable_features(X86CPU *cpu, FeatureWord w, uint64_t mask,
+                               const char *verbose_prefix)
 {
     CPUX86State *env = &cpu->env;
     FeatureWordInfo *f = &feature_word_info[w];
@@ -5800,6 +5800,35 @@ static void mark_unavailable_features(X86CPU *cpu, FeatureWord w, uint64_t mask,
     }
 }
 
+void mark_forced_on_features(X86CPU *cpu, FeatureWord w, uint64_t mask,
+                             const char *verbose_prefix)
+{
+    CPUX86State *env = &cpu->env;
+    FeatureWordInfo *f = &feature_word_info[w];
+    int i;
+
+    if (!cpu->force_features) {
+        env->features[w] |= mask;
+    }
+
+    cpu->forced_on_features[w] |= mask;
+
+    if (!verbose_prefix) {
+        return;
+    }
+
+    for (i = 0; i < 64; ++i) {
+        if ((1ULL << i) & mask) {
+            g_autofree char *feat_word_str = feature_word_description(f);
+            warn_report("%s: %s%s%s [bit %d]",
+                        verbose_prefix,
+                        feat_word_str,
+                        f->feat_names[i] ? "." : "",
+                        f->feat_names[i] ? f->feat_names[i] : "", i);
+        }
+    }
+}
+
 static void x86_cpuid_version_get_family(Object *obj, Visitor *v,
                                          const char *name, void *opaque,
                                          Error **errp)
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index a223e09a25c4..1600e826f372 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -2227,6 +2227,9 @@ struct ArchCPU {
     /* Features that were filtered out because of missing host capabilities */
     FeatureWordArray filtered_features;
 
+    /* Features that are forced enabled by underlying hypervisor, e.g., TDX */
+    FeatureWordArray forced_on_features;
+
     /* Enable PMU CPUID bits. This can't be enabled by default yet because
      * it doesn't have ABI stability guarantees, as it passes all PMU CPUID
      * bits returned by GET_SUPPORTED_CPUID (that depend on host CPU and kernel
@@ -2539,6 +2542,10 @@ void host_cpuid(uint32_t function, uint32_t count,
                 uint32_t *eax, uint32_t *ebx, uint32_t *ecx, uint32_t *edx);
 bool cpu_has_x2apic_feature(CPUX86State *env);
 bool is_feature_word_cpuid(uint32_t feature, uint32_t index, int reg);
+void mark_unavailable_features(X86CPU *cpu, FeatureWord w, uint64_t mask,
+                               const char *verbose_prefix);
+void mark_forced_on_features(X86CPU *cpu, FeatureWord w, uint64_t mask,
+                             const char *verbose_prefix);
 
 static inline bool x86_has_cpuid_0x1f(X86CPU *cpu)
 {
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 7382b53fcc51..58797470ba7e 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -764,6 +764,104 @@ static uint32_t tdx_adjust_cpuid_features(X86ConfidentialGuest *cg,
     return value;
 }
 
+static struct kvm_cpuid2 *tdx_fetch_cpuid(CPUState *cpu)
+{
+    struct kvm_cpuid2 *fetch_cpuid;
+    int size = KVM_MAX_CPUID_ENTRIES;
+    Error *local_err = NULL;
+    int r;
+
+    do {
+        error_free(local_err);
+        local_err = NULL;
+
+        fetch_cpuid = g_malloc0(sizeof(*fetch_cpuid) +
+                                sizeof(struct kvm_cpuid_entry2) * size);
+        fetch_cpuid->nent = size;
+        r = tdx_vcpu_ioctl(cpu, KVM_TDX_GET_CPUID, 0, fetch_cpuid, &local_err);
+        if (r == -E2BIG) {
+            g_free(fetch_cpuid);
+            size = fetch_cpuid->nent;
+        }
+    } while (r == -E2BIG);
+
+    if (r < 0) {
+        error_report_err(local_err);
+        return NULL;
+    }
+
+    return fetch_cpuid;
+}
+
+static int tdx_check_features(X86ConfidentialGuest *cg, CPUState *cs)
+{
+    uint64_t actual, requested, unavailable, forced_on;
+    g_autofree struct kvm_cpuid2 *fetch_cpuid;
+    const char *forced_on_prefix = NULL;
+    const char *unav_prefix = NULL;
+    struct kvm_cpuid_entry2 *entry;
+    X86CPU *cpu = X86_CPU(cs);
+    CPUX86State *env = &cpu->env;
+    FeatureWordInfo *wi;
+    FeatureWord w;
+    bool mismatch = false;
+
+    fetch_cpuid = tdx_fetch_cpuid(cs);
+    if (!fetch_cpuid) {
+        return -1;
+    }
+
+    if (cpu->check_cpuid || cpu->enforce_cpuid) {
+        unav_prefix = "TDX doesn't support requested feature";
+        forced_on_prefix = "TDX forcibly sets the feature";
+    }
+
+    for (w = 0; w < FEATURE_WORDS; w++) {
+        wi = &feature_word_info[w];
+        actual = 0;
+
+        switch (wi->type) {
+        case CPUID_FEATURE_WORD:
+            entry = cpuid_find_entry(fetch_cpuid, wi->cpuid.eax, wi->cpuid.ecx);
+            if (!entry) {
+                /*
+                 * If KVM doesn't report it means it's totally configurable
+                 * by QEMU
+                 */
+                continue;
+            }
+
+            actual = cpuid_entry_get_reg(entry, wi->cpuid.reg);
+            break;
+        case MSR_FEATURE_WORD:
+            /*
+             * TODO:
+             * validate MSR features when KVM has interface report them.
+             */
+            continue;
+        }
+
+        requested = env->features[w];
+        unavailable = requested & ~actual;
+        mark_unavailable_features(cpu, w, unavailable, unav_prefix);
+        if (unavailable) {
+            mismatch = true;
+        }
+
+        forced_on = actual & ~requested;
+        mark_forced_on_features(cpu, w, forced_on, forced_on_prefix);
+        if (forced_on) {
+            mismatch = true;
+        }
+    }
+
+    if (cpu->enforce_cpuid && mismatch) {
+        return -1;
+    }
+
+    return 0;
+}
+
 static int tdx_validate_attributes(TdxGuest *tdx, Error **errp)
 {
     if ((tdx->attributes & ~tdx_caps->supported_attrs)) {
@@ -1147,4 +1245,5 @@ static void tdx_guest_class_init(ObjectClass *oc, void *data)
     x86_klass->kvm_type = tdx_kvm_type;
     x86_klass->cpu_instance_init = tdx_cpu_instance_init;
     x86_klass->adjust_cpuid_features = tdx_adjust_cpuid_features;
+    x86_klass->check_features = tdx_check_features;
 }
-- 
2.34.1

[PATCH v8 52/55] i386/tdx: Don't treat SYSCALL as unavailable

[Xiaoyao Li]

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
Changes in v7:
 - fix CPUID_EXT2_SYSCALL by adding it to actual;
---
 target/i386/kvm/tdx.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 58797470ba7e..cd8e96d476b8 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -841,6 +841,19 @@ static int tdx_check_features(X86ConfidentialGuest *cg, CPUState *cs)
             continue;
         }
 
+        /* Fixup for special cases */
+        switch (w) {
+        case FEAT_8000_0001_EDX:
+            /*
+             * Intel enumerates SYSCALL bit as 1 only when processor in 64-bit
+             * mode and before vcpu running it's not in 64-bit mode.
+             */
+            actual |= CPUID_EXT2_SYSCALL;
+            break;
+        default:
+            break;
+        }
+
         requested = env->features[w];
         unavailable = requested & ~actual;
         mark_unavailable_features(cpu, w, unavailable, unav_prefix);
-- 
2.34.1

Re: [PATCH v8 52/55] i386/tdx: Don't treat SYSCALL as unavailable

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:02:02AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:02:02 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 52/55] i386/tdx: Don't treat SYSCALL as unavailable
> X-Mailer: git-send-email 2.34.1

It's better to add the commit message (for example, your additional
comment at v6 [1]).

[1]: https://lore.kernel.org/qemu-devel/3345330e-c111-4d08-9852-9dede46957d2@intel.com/

> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
> Changes in v7:
>  - fix CPUID_EXT2_SYSCALL by adding it to actual;
> ---
>  target/i386/kvm/tdx.c | 13 +++++++++++++
>  1 file changed, 13 insertions(+)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 53/55] i386/tdx: Make invtsc default on

[Xiaoyao Li]

Because it's fixed1 bit that enforced by TDX module.

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
 target/i386/kvm/tdx.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index cd8e96d476b8..8f075ba5a4de 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -740,6 +740,9 @@ static void tdx_cpu_instance_init(X86ConfidentialGuest *cg, CPUState *cpu)
 
     object_property_set_bool(OBJECT(cpu), "pmu", false, &error_abort);
 
+    /* invtsc is fixed1 for TD guest */
+    object_property_set_bool(OBJECT(cpu), "invtsc", true, &error_abort);
+
     x86cpu->enable_cpuid_0x1f = true;
 }
 
-- 
2.34.1

Re: [PATCH v8 53/55] i386/tdx: Make invtsc default on

[Zhao Liu]

On Tue, Apr 01, 2025 at 09:02:03AM -0400, Xiaoyao Li wrote:
> Date: Tue,  1 Apr 2025 09:02:03 -0400
> From: Xiaoyao Li <xiaoyao.li@intel.com>
> Subject: [PATCH v8 53/55] i386/tdx: Make invtsc default on
> X-Mailer: git-send-email 2.34.1
> 
> Because it's fixed1 bit that enforced by TDX module.
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
>  target/i386/kvm/tdx.c | 3 +++
>  1 file changed, 3 insertions(+)

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

[PATCH v8 54/55] i386/tdx: Validate phys_bits against host value

[Xiaoyao Li]

For TDX guest, the phys_bits is not configurable and can only be
host/native value.

Validate phys_bits inside tdx_check_features().

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
 target/i386/host-cpu.c | 2 +-
 target/i386/host-cpu.h | 1 +
 target/i386/kvm/tdx.c  | 8 ++++++++
 3 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/target/i386/host-cpu.c b/target/i386/host-cpu.c
index 3e4e85e729c8..8a15af458b05 100644
--- a/target/i386/host-cpu.c
+++ b/target/i386/host-cpu.c
@@ -15,7 +15,7 @@
 #include "system/system.h"
 
 /* Note: Only safe for use on x86(-64) hosts */
-static uint32_t host_cpu_phys_bits(void)
+uint32_t host_cpu_phys_bits(void)
 {
     uint32_t eax;
     uint32_t host_phys_bits;
diff --git a/target/i386/host-cpu.h b/target/i386/host-cpu.h
index 6a9bc918baa4..b97ec01c9bec 100644
--- a/target/i386/host-cpu.h
+++ b/target/i386/host-cpu.h
@@ -10,6 +10,7 @@
 #ifndef HOST_CPU_H
 #define HOST_CPU_H
 
+uint32_t host_cpu_phys_bits(void);
 void host_cpu_instance_init(X86CPU *cpu);
 void host_cpu_max_instance_init(X86CPU *cpu);
 bool host_cpu_realizefn(CPUState *cs, Error **errp);
diff --git a/target/i386/kvm/tdx.c b/target/i386/kvm/tdx.c
index 8f075ba5a4de..c8fffa94c7d2 100644
--- a/target/i386/kvm/tdx.c
+++ b/target/i386/kvm/tdx.c
@@ -25,6 +25,7 @@
 
 #include "cpu.h"
 #include "cpu-internal.h"
+#include "host-cpu.h"
 #include "hw/i386/e820_memory_layout.h"
 #include "hw/i386/tdvf.h"
 #include "hw/i386/x86.h"
@@ -875,6 +876,13 @@ static int tdx_check_features(X86ConfidentialGuest *cg, CPUState *cs)
         return -1;
     }
 
+    if (cpu->phys_bits != host_cpu_phys_bits()) {
+        error_report("TDX requires guest CPU physical bits (%u) "
+                     "to match host CPU physical bits (%u)",
+                     cpu->phys_bits, host_cpu_phys_bits());
+        exit(1);
+    }
+
     return 0;
 }
 
-- 
2.34.1

Re: [PATCH v8 54/55] i386/tdx: Validate phys_bits against host value

[Daniel P. Berrangé]

On Tue, Apr 01, 2025 at 09:02:04AM -0400, Xiaoyao Li wrote:
> For TDX guest, the phys_bits is not configurable and can only be
> host/native value.
> 
> Validate phys_bits inside tdx_check_features().
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
>  target/i386/host-cpu.c | 2 +-
>  target/i386/host-cpu.h | 1 +
>  target/i386/kvm/tdx.c  | 8 ++++++++
>  3 files changed, 10 insertions(+), 1 deletion(-)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v8 54/55] i386/tdx: Validate phys_bits against host value

[Zhao Liu]

> @e -875,6 +876,13 @@ static int tdx_check_features(X86ConfidentialGuest *cg, CPUState *cs)
>          return -1;

Here you has already used "return -1", so...

>      }
>  
> +    if (cpu->phys_bits != host_cpu_phys_bits()) {
> +        error_report("TDX requires guest CPU physical bits (%u) "
> +                     "to match host CPU physical bits (%u)",
> +                     cpu->phys_bits, host_cpu_phys_bits());
> +        exit(1);

...what about "return -1"?

kvm_init_vcpu() missed a "goto err", but it is not a big deal:

diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 9862d8ff1d38..05034f622f20 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -595,6 +595,7 @@ int kvm_init_vcpu(CPUState *cpu, Error **errp)
         error_setg_errno(errp, -ret,
                          "kvm_init_vcpu: kvm_arch_init_vcpu failed (%lu)",
                          kvm_arch_vcpu_id(cpu));
+        goto err;
     }
     cpu->kvm_vcpu_stats_fd = kvm_vcpu_ioctl(cpu, KVM_GET_STATS_FD, NULL);
---

Overall, LGTM,

Reviewed-by: Zhao Liu <zhao1.liu@intel.com>

> +    }
> +
>      return 0;
>  }
>  
> -- 
> 2.34.1
> 
> 

Re: [PATCH v8 54/55] i386/tdx: Validate phys_bits against host value

[Xiaoyao Li]

On 5/5/2025 11:29 PM, Zhao Liu wrote:
>> @e -875,6 +876,13 @@ static int tdx_check_features(X86ConfidentialGuest *cg, CPUState *cs)
>>           return -1;
> 
> Here you has already used "return -1", so...
> 
>>       }
>>   
>> +    if (cpu->phys_bits != host_cpu_phys_bits()) {
>> +        error_report("TDX requires guest CPU physical bits (%u) "
>> +                     "to match host CPU physical bits (%u)",
>> +                     cpu->phys_bits, host_cpu_phys_bits());
>> +        exit(1);
> 
> ...what about "return -1"?

I will change it to return -EINVAL and as well for the above. Since ...

> kvm_init_vcpu() missed a "goto err", but it is not a big deal:
> 
> diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
> index 9862d8ff1d38..05034f622f20 100644
> --- a/accel/kvm/kvm-all.c
> +++ b/accel/kvm/kvm-all.c
> @@ -595,6 +595,7 @@ int kvm_init_vcpu(CPUState *cpu, Error **errp)
>           error_setg_errno(errp, -ret,
>                            "kvm_init_vcpu: kvm_arch_init_vcpu failed (%lu)",
>                            kvm_arch_vcpu_id(cpu));

... the handling here expects the @ret is a errno.

> +        goto err;
>       }
>       cpu->kvm_vcpu_stats_fd = kvm_vcpu_ioctl(cpu, KVM_GET_STATS_FD, NULL);
> ---
> 
> Overall, LGTM,
> 
> Reviewed-by: Zhao Liu <zhao1.liu@intel.com>
> 
>> +    }
>> +
>>       return 0;
>>   }
>>   
>> -- 
>> 2.34.1
>>
>>

[PATCH v8 55/55] docs: Add TDX documentation

[Xiaoyao Li]

Add docs/system/i386/tdx.rst for TDX support, and add tdx in
confidential-guest-support.rst

Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
---
Changes in v6:
 - Add more information of "Feature configuration"
 - Mark TD Attestation as future work because KVM now drops the support
   of it.

Changes in v5:
 - Add TD attestation section and update the QEMU parameter;

Changes since v1:
 - Add prerequisite of private gmem;
 - update example command to launch TD;

Changes since RFC v4:
 - add the restriction that kernel-irqchip must be split
---
 docs/system/confidential-guest-support.rst |   1 +
 docs/system/i386/tdx.rst                   | 156 +++++++++++++++++++++
 docs/system/target-i386.rst                |   1 +
 3 files changed, 158 insertions(+)
 create mode 100644 docs/system/i386/tdx.rst

diff --git a/docs/system/confidential-guest-support.rst b/docs/system/confidential-guest-support.rst
index 0c490dbda2b7..66129fbab64c 100644
--- a/docs/system/confidential-guest-support.rst
+++ b/docs/system/confidential-guest-support.rst
@@ -38,6 +38,7 @@ Supported mechanisms
 Currently supported confidential guest mechanisms are:
 
 * AMD Secure Encrypted Virtualization (SEV) (see :doc:`i386/amd-memory-encryption`)
+* Intel Trust Domain Extension (TDX) (see :doc:`i386/tdx`)
 * POWER Protected Execution Facility (PEF) (see :ref:`power-papr-protected-execution-facility-pef`)
 * s390x Protected Virtualization (PV) (see :doc:`s390x/protvirt`)
 
diff --git a/docs/system/i386/tdx.rst b/docs/system/i386/tdx.rst
new file mode 100644
index 000000000000..ea2e601dde9a
--- /dev/null
+++ b/docs/system/i386/tdx.rst
@@ -0,0 +1,156 @@
+Intel Trusted Domain eXtension (TDX)
+====================================
+
+Intel Trusted Domain eXtensions (TDX) refers to an Intel technology that extends
+Virtual Machine Extensions (VMX) and Multi-Key Total Memory Encryption (MKTME)
+with a new kind of virtual machine guest called a Trust Domain (TD). A TD runs
+in a CPU mode that is designed to protect the confidentiality of its memory
+contents and its CPU state from any other software, including the hosting
+Virtual Machine Monitor (VMM), unless explicitly shared by the TD itself.
+
+Prerequisites
+-------------
+
+To run TD, the physical machine needs to have TDX module loaded and initialized
+while KVM hypervisor has TDX support and has TDX enabled. If those requirements
+are met, the ``KVM_CAP_VM_TYPES`` will report the support of ``KVM_X86_TDX_VM``.
+
+Trust Domain Virtual Firmware (TDVF)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Trust Domain Virtual Firmware (TDVF) is required to provide TD services to boot
+TD Guest OS. TDVF needs to be copied to guest private memory and measured before
+the TD boots.
+
+KVM vcpu ioctl ``KVM_TDX_INIT_MEM_REGION`` can be used to populate the TDVF
+content into its private memory.
+
+Since TDX doesn't support readonly memslot, TDVF cannot be mapped as pflash
+device and it actually works as RAM. "-bios" option is chosen to load TDVF.
+
+OVMF is the opensource firmware that implements the TDVF support. Thus the
+command line to specify and load TDVF is ``-bios OVMF.fd``
+
+Feature Configuration
+---------------------
+
+Unlike non-TDX VM, the CPU features (enumerated by CPU or MSR) of a TD are not
+under full control of VMM. VMM can only configure part of features of a TD on
+``KVM_TDX_INIT_VM`` command of VM scope ``MEMORY_ENCRYPT_OP`` ioctl.
+
+The configurable features have three types:
+
+- Attributes:
+  - PKS (bit 30) controls whether Supervisor Protection Keys is exposed to TD,
+  which determines related CPUID bit and CR4 bit;
+  - PERFMON (bit 63) controls whether PMU is exposed to TD.
+
+- XSAVE related features (XFAM):
+  XFAM is a 64b mask, which has the same format as XCR0 or IA32_XSS MSR. It
+  determines the set of extended features available for use by the guest TD.
+
+- CPUID features:
+  Only some bits of some CPUID leaves are directly configurable by VMM.
+
+What features can be configured is reported via TDX capabilities.
+
+TDX capabilities
+~~~~~~~~~~~~~~~~
+
+The VM scope ``MEMORY_ENCRYPT_OP`` ioctl provides command ``KVM_TDX_CAPABILITIES``
+to get the TDX capabilities from KVM. It returns a data structure of
+``struct kvm_tdx_capabilities``, which tells the supported configuration of
+attributes, XFAM and CPUIDs.
+
+TD attributes
+~~~~~~~~~~~~~
+
+QEMU supports configuring raw 64-bit TD attributes directly via "attributes"
+property of "tdx-guest" object. Note, it's users' responsibility to provide a
+valid value because some bits may not supported by current QEMU or KVM yet.
+
+QEMU also supports the configuration of individual attribute bits that are
+supported by it, via properties of "tdx-guest" object.
+E.g., "sept-ve-disable" (bit 28).
+
+MSR based features
+~~~~~~~~~~~~~~~~~~
+
+Current KVM doesn't support MSR based feature (e.g., MSR_IA32_ARCH_CAPABILITIES)
+configuration for TDX, and it's a future work to enable it in QEMU when KVM adds
+support of it.
+
+Feature check
+~~~~~~~~~~~~~
+
+QEMU checks if the final (CPU) features, determined by given cpu model and
+explicit feature adjustment of "+featureA/-featureB", can be supported or not.
+It can produce feature not supported warning like
+
+  "warning: host doesn't support requested feature: CPUID.07H:EBX.intel-pt [bit 25]"
+
+It can also produce warning like
+
+  "warning: TDX forcibly sets the feature: CPUID.80000007H:EDX.invtsc [bit 8]"
+
+if the fixed-1 feature is requested to be disabled explicitly. This is newly
+added to QEMU for TDX because TDX has fixed-1 features that are forcibly enabled
+by TDX module and VMM cannot disable them.
+
+Launching a TD (TDX VM)
+-----------------------
+
+To launch a TD, the necessary command line options are tdx-guest object and
+split kernel-irqchip, as below:
+
+.. parsed-literal::
+
+    |qemu_system_x86| \\
+        -object tdx-guest,id=tdx0 \\
+        -machine ...,kernel-irqchip=split,confidential-guest-support=tdx0 \\
+        -bios OVMF.fd \\
+
+Restrictions
+------------
+
+ - kernel-irqchip must be split;
+
+ - No readonly support for private memory;
+
+ - No SMM support: SMM support requires manipulating the guest register states
+   which is not allowed;
+
+Debugging
+---------
+
+Bit 0 of TD attributes, is DEBUG bit, which decides if the TD runs in off-TD
+debug mode. When in off-TD debug mode, TD's VCPU state and private memory are
+accessible via given SEAMCALLs. This requires KVM to expose APIs to invoke those
+SEAMCALLs and corresonponding QEMU change.
+
+It's targeted as future work.
+
+TD attestation
+--------------
+
+In TD guest, the attestation process is used to verify the TDX guest
+trustworthiness to other entities before provisioning secrets to the guest.
+
+TD attestation is initiated first by calling TDG.MR.REPORT inside TD to get the
+REPORT. Then the REPORT data needs to be converted into a remotely verifiable
+Quote by SGX Quoting Enclave (QE).
+
+It's a future work in QEMU to add support of TD attestation since it lacks
+support in current KVM.
+
+Live Migration
+--------------
+
+Future work.
+
+References
+----------
+
+- `TDX Homepage <https://www.intel.com/content/www/us/en/developer/articles/technical/intel-trust-domain-extensions.html>`__
+
+- `SGX QE <https://github.com/intel/SGXDataCenterAttestationPrimitives/tree/master/QuoteGeneration>`__
diff --git a/docs/system/target-i386.rst b/docs/system/target-i386.rst
index ab7af1a75d6e..43b09c79d6be 100644
--- a/docs/system/target-i386.rst
+++ b/docs/system/target-i386.rst
@@ -31,6 +31,7 @@ Architectural features
    i386/kvm-pv
    i386/sgx
    i386/amd-memory-encryption
+   i386/tdx
 
 OS requirements
 ~~~~~~~~~~~~~~~
-- 
2.34.1

Re: [PATCH v8 55/55] docs: Add TDX documentation

[Daniel P. Berrangé]

CC libvirt / Jiri, for confirmation about whether the CPUID restrictions
listed below will have any possible impact on libvirt CPUID handling...

On Tue, Apr 01, 2025 at 09:02:05AM -0400, Xiaoyao Li wrote:
> Add docs/system/i386/tdx.rst for TDX support, and add tdx in
> confidential-guest-support.rst
> 
> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
> ---
> Changes in v6:
>  - Add more information of "Feature configuration"
>  - Mark TD Attestation as future work because KVM now drops the support
>    of it.
> 
> Changes in v5:
>  - Add TD attestation section and update the QEMU parameter;
> 
> Changes since v1:
>  - Add prerequisite of private gmem;
>  - update example command to launch TD;
> 
> Changes since RFC v4:
>  - add the restriction that kernel-irqchip must be split
> ---
>  docs/system/confidential-guest-support.rst |   1 +
>  docs/system/i386/tdx.rst                   | 156 +++++++++++++++++++++
>  docs/system/target-i386.rst                |   1 +
>  3 files changed, 158 insertions(+)
>  create mode 100644 docs/system/i386/tdx.rst


> +Feature Configuration
> +---------------------
> +
> +Unlike non-TDX VM, the CPU features (enumerated by CPU or MSR) of a TD are not
> +under full control of VMM. VMM can only configure part of features of a TD on
> +``KVM_TDX_INIT_VM`` command of VM scope ``MEMORY_ENCRYPT_OP`` ioctl.
> +
> +The configurable features have three types:
> +
> +- Attributes:
> +  - PKS (bit 30) controls whether Supervisor Protection Keys is exposed to TD,
> +  which determines related CPUID bit and CR4 bit;
> +  - PERFMON (bit 63) controls whether PMU is exposed to TD.
> +
> +- XSAVE related features (XFAM):
> +  XFAM is a 64b mask, which has the same format as XCR0 or IA32_XSS MSR. It
> +  determines the set of extended features available for use by the guest TD.
> +
> +- CPUID features:
> +  Only some bits of some CPUID leaves are directly configurable by VMM.
> +
> +What features can be configured is reported via TDX capabilities.
> +
> +TDX capabilities
> +~~~~~~~~~~~~~~~~
> +
> +The VM scope ``MEMORY_ENCRYPT_OP`` ioctl provides command ``KVM_TDX_CAPABILITIES``
> +to get the TDX capabilities from KVM. It returns a data structure of
> +``struct kvm_tdx_capabilities``, which tells the supported configuration of
> +attributes, XFAM and CPUIDs.
> +
> +TD attributes
> +~~~~~~~~~~~~~
> +
> +QEMU supports configuring raw 64-bit TD attributes directly via "attributes"
> +property of "tdx-guest" object. Note, it's users' responsibility to provide a
> +valid value because some bits may not supported by current QEMU or KVM yet.
> +
> +QEMU also supports the configuration of individual attribute bits that are
> +supported by it, via properties of "tdx-guest" object.
> +E.g., "sept-ve-disable" (bit 28).
> +
> +MSR based features
> +~~~~~~~~~~~~~~~~~~
> +
> +Current KVM doesn't support MSR based feature (e.g., MSR_IA32_ARCH_CAPABILITIES)
> +configuration for TDX, and it's a future work to enable it in QEMU when KVM adds
> +support of it.
> +
> +Feature check
> +~~~~~~~~~~~~~
> +
> +QEMU checks if the final (CPU) features, determined by given cpu model and
> +explicit feature adjustment of "+featureA/-featureB", can be supported or not.
> +It can produce feature not supported warning like
> +
> +  "warning: host doesn't support requested feature: CPUID.07H:EBX.intel-pt [bit 25]"
> +
> +It can also produce warning like
> +
> +  "warning: TDX forcibly sets the feature: CPUID.80000007H:EDX.invtsc [bit 8]"
> +
> +if the fixed-1 feature is requested to be disabled explicitly. This is newly
> +added to QEMU for TDX because TDX has fixed-1 features that are forcibly enabled
> +by TDX module and VMM cannot disable them.

This is where I'm wondering if libvirt has anything to be concerned
about. Possibly when libvirt queries the actual CPUID after launching
the guest it will just "do the right thing" ? Wondering if there's any
need for libvirt to be aware of CPUID restrictions before that point
though ?


> +
> +Launching a TD (TDX VM)
> +-----------------------
> +
> +To launch a TD, the necessary command line options are tdx-guest object and
> +split kernel-irqchip, as below:
> +
> +.. parsed-literal::
> +
> +    |qemu_system_x86| \\
> +        -object tdx-guest,id=tdx0 \\
> +        -machine ...,kernel-irqchip=split,confidential-guest-support=tdx0 \\
> +        -bios OVMF.fd \\

I don't think we need to show 'kernel-irqchip=split' now that we "do the
right thing" by default


This surely also ought to include '-accel kvm', as IIUC there's no
TCG support for TDX.

And presumably '-cpu host', since QEMU's default 'qemu64' CPU model
is likely a terrible match for what TDX will force set.

> +
> +Restrictions
> +------------
> +
> + - kernel-irqchip must be split;

Can append

   "This is set by default for TDX guests if kernel-irqchip is left on
    its default 'auto' setting."

> +
> + - No readonly support for private memory;
> +
> + - No SMM support: SMM support requires manipulating the guest register states
> +   which is not allowed;
> +


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH v8 55/55] docs: Add TDX documentation

[Jiří Denemark]

On Wed, Apr 02, 2025 at 11:50:26 +0100, Daniel P. Berrangé wrote:
> CC libvirt / Jiri, for confirmation about whether the CPUID restrictions
> listed below will have any possible impact on libvirt CPUID handling...
..
> > +Feature check
> > +~~~~~~~~~~~~~
> > +
> > +QEMU checks if the final (CPU) features, determined by given cpu model and
> > +explicit feature adjustment of "+featureA/-featureB", can be supported or not.
> > +It can produce feature not supported warning like
> > +
> > +  "warning: host doesn't support requested feature: CPUID.07H:EBX.intel-pt [bit 25]"
> > +
> > +It can also produce warning like
> > +
> > +  "warning: TDX forcibly sets the feature: CPUID.80000007H:EDX.invtsc [bit 8]"
> > +
> > +if the fixed-1 feature is requested to be disabled explicitly. This is newly
> > +added to QEMU for TDX because TDX has fixed-1 features that are forcibly enabled
> > +by TDX module and VMM cannot disable them.
> 
> This is where I'm wondering if libvirt has anything to be concerned
> about. Possibly when libvirt queries the actual CPUID after launching
> the guest it will just "do the right thing" ? Wondering if there's any
> need for libvirt to be aware of CPUID restrictions before that point
> though ?

Yes, it will do the right thing.

Unless a user explicitly requests check='full'. Libvirt will use
check='full' on its own only during migration and at that point the CPU
definition is already modified according to what QEMU changed when the
domain was started. So this will just work.

For explicit check='full' to be usable we'd need a way to tell users
which features are required/forbidden with TDX. But to be honest I don't
think this is needed.

Jirka

Re: [PATCH v8 55/55] docs: Add TDX documentation

[Xiaoyao Li]

On 4/2/2025 6:50 PM, Daniel P. Berrangé wrote:
> CC libvirt / Jiri, for confirmation about whether the CPUID restrictions
> listed below will have any possible impact on libvirt CPUID handling...
> 
> On Tue, Apr 01, 2025 at 09:02:05AM -0400, Xiaoyao Li wrote:
>> Add docs/system/i386/tdx.rst for TDX support, and add tdx in
>> confidential-guest-support.rst
>>
>> Signed-off-by: Xiaoyao Li <xiaoyao.li@intel.com>
>> ---
>> Changes in v6:
>>   - Add more information of "Feature configuration"
>>   - Mark TD Attestation as future work because KVM now drops the support
>>     of it.
>>
>> Changes in v5:
>>   - Add TD attestation section and update the QEMU parameter;
>>
>> Changes since v1:
>>   - Add prerequisite of private gmem;
>>   - update example command to launch TD;
>>
>> Changes since RFC v4:
>>   - add the restriction that kernel-irqchip must be split
>> ---
>>   docs/system/confidential-guest-support.rst |   1 +
>>   docs/system/i386/tdx.rst                   | 156 +++++++++++++++++++++
>>   docs/system/target-i386.rst                |   1 +
>>   3 files changed, 158 insertions(+)
>>   create mode 100644 docs/system/i386/tdx.rst
> 
> 
>> +Feature Configuration
>> +---------------------
>> +
>> +Unlike non-TDX VM, the CPU features (enumerated by CPU or MSR) of a TD are not
>> +under full control of VMM. VMM can only configure part of features of a TD on
>> +``KVM_TDX_INIT_VM`` command of VM scope ``MEMORY_ENCRYPT_OP`` ioctl.
>> +
>> +The configurable features have three types:
>> +
>> +- Attributes:
>> +  - PKS (bit 30) controls whether Supervisor Protection Keys is exposed to TD,
>> +  which determines related CPUID bit and CR4 bit;
>> +  - PERFMON (bit 63) controls whether PMU is exposed to TD.
>> +
>> +- XSAVE related features (XFAM):
>> +  XFAM is a 64b mask, which has the same format as XCR0 or IA32_XSS MSR. It
>> +  determines the set of extended features available for use by the guest TD.
>> +
>> +- CPUID features:
>> +  Only some bits of some CPUID leaves are directly configurable by VMM.
>> +
>> +What features can be configured is reported via TDX capabilities.
>> +
>> +TDX capabilities
>> +~~~~~~~~~~~~~~~~
>> +
>> +The VM scope ``MEMORY_ENCRYPT_OP`` ioctl provides command ``KVM_TDX_CAPABILITIES``
>> +to get the TDX capabilities from KVM. It returns a data structure of
>> +``struct kvm_tdx_capabilities``, which tells the supported configuration of
>> +attributes, XFAM and CPUIDs.
>> +
>> +TD attributes
>> +~~~~~~~~~~~~~
>> +
>> +QEMU supports configuring raw 64-bit TD attributes directly via "attributes"
>> +property of "tdx-guest" object. Note, it's users' responsibility to provide a
>> +valid value because some bits may not supported by current QEMU or KVM yet.
>> +
>> +QEMU also supports the configuration of individual attribute bits that are
>> +supported by it, via properties of "tdx-guest" object.
>> +E.g., "sept-ve-disable" (bit 28).
>> +
>> +MSR based features
>> +~~~~~~~~~~~~~~~~~~
>> +
>> +Current KVM doesn't support MSR based feature (e.g., MSR_IA32_ARCH_CAPABILITIES)
>> +configuration for TDX, and it's a future work to enable it in QEMU when KVM adds
>> +support of it.
>> +
>> +Feature check
>> +~~~~~~~~~~~~~
>> +
>> +QEMU checks if the final (CPU) features, determined by given cpu model and
>> +explicit feature adjustment of "+featureA/-featureB", can be supported or not.
>> +It can produce feature not supported warning like
>> +
>> +  "warning: host doesn't support requested feature: CPUID.07H:EBX.intel-pt [bit 25]"
>> +
>> +It can also produce warning like
>> +
>> +  "warning: TDX forcibly sets the feature: CPUID.80000007H:EDX.invtsc [bit 8]"
>> +
>> +if the fixed-1 feature is requested to be disabled explicitly. This is newly
>> +added to QEMU for TDX because TDX has fixed-1 features that are forcibly enabled
>> +by TDX module and VMM cannot disable them.
> 
> This is where I'm wondering if libvirt has anything to be concerned
> about. Possibly when libvirt queries the actual CPUID after launching
> the guest it will just "do the right thing" ? Wondering if there's any
> need for libvirt to be aware of CPUID restrictions before that point
> though ?
> 
> 
>> +
>> +Launching a TD (TDX VM)
>> +-----------------------
>> +
>> +To launch a TD, the necessary command line options are tdx-guest object and
>> +split kernel-irqchip, as below:
>> +
>> +.. parsed-literal::
>> +
>> +    |qemu_system_x86| \\
>> +        -object tdx-guest,id=tdx0 \\
>> +        -machine ...,kernel-irqchip=split,confidential-guest-support=tdx0 \\
>> +        -bios OVMF.fd \\
> 
> I don't think we need to show 'kernel-irqchip=split' now that we "do the
> right thing" by default
> 
> 
> This surely also ought to include '-accel kvm', as IIUC there's no
> TCG support for TDX.
> 
> And presumably '-cpu host', since QEMU's default 'qemu64' CPU model
> is likely a terrible match for what TDX will force set.
> 
>> +
>> +Restrictions
>> +------------
>> +
>> + - kernel-irqchip must be split;
> 
> Can append
> 
>     "This is set by default for TDX guests if kernel-irqchip is left on
>      its default 'auto' setting."

This and above feedback are all good. Accept them all and will update in 
the next version.

Thanks!

>> +
>> + - No readonly support for private memory;
>> +
>> + - No SMM support: SMM support requires manipulating the guest register states
>> +   which is not allowed;
>> +
> 
> 
> With regards,
> Daniel