[RFC PATCH 0/4] TDX attestation support and GHCI fixup

[RFC PATCH 0/4] TDX attestation support and GHCI fixup

[Binbin Wu]

Hi,

This patch set includes TDX attestation support patch from [0] and some
fixups according to the proposed GHCI spec changes below. Tag the patch set
as RFC since it's based on the proposed GHCI spec changes. However, the
proposal and the patch set are expected to be discussed/finalized. The
attestation support and the fixups are hoped to be merged during the 6.16
merge window as part of the initial TDX support.

Paolo once suggested to implement all the TDVMCALLs defined in the current
GHCI 1.5 spec [1]. However, since there is a proposal to fix the GHCI spec
issues, this patch set doesn't implement the support for Service
SetupEventNotifyInterrupt, Instruction.WBINVD and Instruction.PCONFIG since
these TDVMCALLs have no real users for now.

Notable changes since attestation v2 [0]
========================================
- Use TDVMCALL_STATUS_SUBFUNC_UNSUPPORTED instead of
  TDVMCALL_STATUS_INVALID_OPERAND for unsupported TDVMCALLs.
- Document the definition of "The base GHCI TDVMCALLs".
- Forward GetTdVmCallInfo to userspace with the new exit reason
  KVM_EXIT_TDX_GET_TDVMCALL_INFO when leaf (r12) input is 1 to allow
  userspace to provide the information of TDVMCALLs supported in userspace.
- Move the check of userspace's opt-in of KVM exit on KVM_HC_MAP_GPA_RANGE
  to KVM_TDX_FINALIZE_VM, since MapGPA is one of the GHCI base TDVMCALLs
  according to the proposal below.
  This requires some changes to the TDX KVM selftests cases posted [2] to
  always opt-in KVM exit on KVM_HC_MAP_GPA_RANGE before
  KVM_TDX_FINALIZE_VM.
- Since there is no opt-in from userspace for GetTdVmCallInfo and GetQuote,
  userspace is required to handle the exit reasons
  KVM_EXIT_TDX_GET_TDVMCALL_INFO and KVM_EXIT_TDX_GET_QUOTE as the initial
  support for TDX. To simplify the implementation in userspace, userspace
  could return TDVMCALL_STATUS_SUBFUNC_UNSUPPORTED for GetQuote.

GHCI Change Proposal
====================
Current TDX Guest Host communication Interface (GHCI) spec[3] implies that
VMM should only return success for TDG.VP.VMCALL<GetTdVmCallInfo> if *all*
TDVMCALLs defined in the GHCI spec are supported. The spec is ambiguous on
the following perspectives:
- The description "all TDG.VP.VMCALLs defined in this specification" is not
  forward-compatible since more and more TDVMCALLs will be added when the
  GHCI spec evolves.
- It actually doesn't cover how to handle if the guest calls an unsupported
  TDVMCALL.
  Historically, KVM has returned TDVMCALL_STATUS_INVALID_OPERAND for any
  unknown TDVMCALL, as a reasonable interpretation of the ambiguous spec.
  However, TDX guests can't distinguish the error is due to the TDVMCALL is
  not supported or an invalid input of the TDVMCALL.
Also, enforce VMMs to implement the TDVMCALLs without real users is an
unnecessary burden.

To address the issues, the following are the proposed GHCI spec changes:
- Define "the GHCI base TDVMCALLs", which are: <GetTdVmCallInfo>, <MapGPA>,
  <ReportFatalError>, <Instruction.CPUID>, <#VE.RequestMMIO>,
  <Instruction.HLT>, <Instruction.IO>, <Instruction.RDMSR> and
  <Instruction.WRMSR>.
- Limit the scope of the TDG.VP.VMCALL<GetTdVmCallInfo> with leaf (R12) set
  to 0 to the GHCI base VMCALLs, so that the meaning is clear and
  unambiguous.
- Extend the TDG.VP.VMCALL<GetTdVmCallInfo> with leaf (R12) set to 1 to
  allow TDX guests to query the supported TDVMCALLs beyond the GHCI base
  TDVMCALLs.
  Use R11 - R14 to return the supported TDVMCALLs, which are defined as
  * R11
    bit 0: <GetQuote>
    bit 1: <SetupEventNotifyInterrupt>
    bit 2: <Service>
    bit 3: <MigTD>
    Other bits of R11 are reserved and must be 0.
  * R12
    bit 0: <Instruction.WBINVD>
    bit 1: <Instruction.PCONFIG>
    Other bits of R12 are reserved and must be 0.
  * R13 and R14 are reserved and must be 0.
- Add TDVMCALL_STATUS_SUBFUNC_UNSUPPORTED to the TDVMCALL status codes for
  the ones that beyond GHCI base TDVMCALLs to indicate the subfunction is
  not supported.
  For the back-compatibility analysis, please refer to the change log of
  "KVM: TDX: Add new TDVMCALL status code for unsupported subfuncs".

Testing
=======
This series is based on kvm/next with the commit:
- '61374cc145f4' ("Merge tag 'kvmarm-fixes-6.16-1' of
  https://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm into HEAD").

A matching QEMU is here:
https://github.com/intel-staging/qemu-tdx/commits/binbinwu/GetTdVmCallInfo_fixup

It requires TDX module 1.5.06.00.0744 [4], or later.
A working edk2 commit is 95d8a1c ("UnitTestFrameworkPkg: Use TianoCore
mirror of subhook submodule").

This patch series passed the TDX kvm-unit-tests, booting a Linux TD, and
TDX enhanced KVM selftests. It also passed the TDX related test cases
defined in the LKVS test suite as described in:
https://github.com/intel/lkvs/blob/main/KVM/docs/lkvs_on_avocado.md

KVM selftests patches based on the latest TDX KVM selftests patch
series [2], were used to test the flows of GetQuote, GetTdVmCallInfo with
leaf 1, unsupported TDVMCALL, and some modifications were made because the
opt-in of KVM exit on KVM_HC_MAP_GPA_RANGE should be done before
KVM_TDX_FINALIZE_VM.

[0] https://lore.kernel.org/kvm/20250416055433.2980510-1-binbin.wu@linux.intel.com
[1] https://lore.kernel.org/kvm/5e7e8cb7-27b2-416d-9262-e585034327be@redhat.com
[2] https://lore.kernel.org/kvm/20250414214801.2693294-1-sagis@google.com
[3] https://cdrdv2.intel.com/v1/dl/getContent/726792
[4] https://github.com/intel/tdx-module/releases/tag/TDX_1.5.06

Binbin Wu (4):
  KVM: TDX: Add new TDVMCALL status code for unsupported subfuncs
  KVM: TDX: Handle TDG.VP.VMCALL<GetQuote>
  KVM: TDX: Exit to userspace for GetTdVmCallInfo
  KVM: TDX: Check KVM exit on KVM_HC_MAP_GPA_RANGE when TD finalize

 Documentation/virt/kvm/api.rst           | 44 ++++++++++++
 Documentation/virt/kvm/x86/intel-tdx.rst | 12 ++++
 arch/x86/include/asm/shared/tdx.h        |  1 +
 arch/x86/kvm/vmx/tdx.c                   | 85 +++++++++++++++++++-----
 include/uapi/linux/kvm.h                 | 14 ++++
 5 files changed, 141 insertions(+), 15 deletions(-)


base-commit: 61374cc145f4a56377eaf87c7409a97ec7a34041
-- 
2.46.0

[RFC PATCH 1/4] KVM: TDX: Add new TDVMCALL status code for unsupported subfuncs

[Binbin Wu]

Add the new TDVMCALL status code TDVMCALL_STATUS_SUBFUNC_UNSUPPORTED and
return it for unimplemented TDVMCALL subfunctions.

Returning TDVMCALL_STATUS_INVALID_OPERAND when a subfunction is not
implemented is vague because TDX guests can't tell the error is due to
the subfunction is not supported or an invalid input of the subfunction.
New GHCI spec adds TDVMCALL_STATUS_SUBFUNC_UNSUPPORTED to avoid the
ambiguity. Use it instead of TDVMCALL_STATUS_INVALID_OPERAND.

Before the change, for common guest implementations, when a TDX guest
receives TDVMCALL_STATUS_INVALID_OPERAND, it has two cases:
1. Some operand is invalid. It could change the operand to another value
   retry.
2. The subfunction is not supported.
For case 1, an invalid operand usually means the guest implementation bug.
Since the TDX guest can't tell which case is, the best practice for
handling TDVMCALL_STATUS_INVALID_OPERAND is stopping calling such leaf.
Treat it as fatal if the TDVMCALL is essential or ignore it if the TDVMCALL
is optional.

After the change, TDVMCALL_STATUS_SUBFUNC_UNSUPPORTED is unknown to the
old TDX guest, the guest will make the same action as
TDVMCALL_STATUS_INVALID_OPERAND, unless the guest check the
TDVMCALL_STATUS_INVALID_OPERAND specifically.  Currently, no known
TDX guests do it, e.g., Linux TDX guests just check for success.

Signed-off-by: Binbin Wu <binbin.wu@linux.intel.com>
---
 arch/x86/include/asm/shared/tdx.h | 1 +
 arch/x86/kvm/vmx/tdx.c            | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/shared/tdx.h b/arch/x86/include/asm/shared/tdx.h
index fd9209e996e7..b109b947fadf 100644
--- a/arch/x86/include/asm/shared/tdx.h
+++ b/arch/x86/include/asm/shared/tdx.h
@@ -79,6 +79,7 @@
 #define TDVMCALL_STATUS_RETRY		0x0000000000000001ULL
 #define TDVMCALL_STATUS_INVALID_OPERAND	0x8000000000000000ULL
 #define TDVMCALL_STATUS_ALIGN_ERROR	0x8000000000000002ULL
+#define TDVMCALL_STATUS_SUBFUNC_UNSUPPORTED	0x8000000000000003ULL
 
 /*
  * Bitmasks of exposed registers (with VMM).
diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
index b952bc673271..8134d5805b03 100644
--- a/arch/x86/kvm/vmx/tdx.c
+++ b/arch/x86/kvm/vmx/tdx.c
@@ -1476,7 +1476,7 @@ static int handle_tdvmcall(struct kvm_vcpu *vcpu)
 		break;
 	}
 
-	tdvmcall_set_return_code(vcpu, TDVMCALL_STATUS_INVALID_OPERAND);
+	tdvmcall_set_return_code(vcpu, TDVMCALL_STATUS_SUBFUNC_UNSUPPORTED);
 	return 1;
 }
 
-- 
2.46.0

[RFC PATCH 2/4] KVM: TDX: Handle TDG.VP.VMCALL<GetQuote>

[Binbin Wu]

Handle TDVMCALL for GetQuote to generate a TD-Quote.

GetQuote is a doorbell-like interface used by TDX guests to request VMM
to generate a TD-Quote signed by a service hosting TD-Quoting Enclave
operating on the host.  A TDX guest passes a TD Report (TDREPORT_STRUCT) in
a shared-memory area as parameter.  Host VMM can access it and queue the
operation for a service hosting TD-Quoting enclave.  When completed, the
Quote is returned via the same shared-memory area.

KVM only checks the GPA from the TDX guest has the shared-bit set and drops
the shared-bit before exiting to userspace to avoid bleeding the shared-bit
into KVM's exit ABI.  KVM forwards the request to userspace VMM (e.g. QEMU)
and userspace VMM queues the operation asynchronously.  KVM sets the return
code according to the 'ret' field set by userspace to notify the TDX guest
whether the request has been queued successfully or not.  When the request
has been queued successfully, the TDX guest can poll the status field in
the shared-memory area to check whether the Quote generation is completed
or not.  When completed, the generated Quote is returned via the same
buffer.

Add KVM_EXIT_TDX_GET_QUOTE as a new exit reason to userspace. Userspace is
required to handle the KVM exit reason as the initial support for TDX,
however, userspace is allowed to set 'ret' filed to
TDVMCALL_STATUS_SUBFUNC_UNSUPPORTED.

Signed-off-by: Binbin Wu <binbin.wu@linux.intel.com>
Tested-by: Mikko Ylinen <mikko.ylinen@linux.intel.com>
Acked-by: Kai Huang <kai.huang@intel.com>
---
v2:
- Skip sanity checks except for the check of the shared-bit. (Rick, Sean)
- Drop the shared-bit of the GPA before exiting to userspace. (Sean)
- Improve the description of 'ret' field in the document. (Kai)
- Use number 40 for KVM_EXIT_TDX_GET_QUOTE. (Xiaoyao, Rick)
- Update the changelog and the description of KVM_EXIT_TDX_GET_QUOTE in the
  document according to the code changes.

RFC v1:
- Add "Acked-by" from Kai.
- State that KVM_EXIT_TDX_GET_QUOTE should be handled by userspace as the
  initial support for TDX.
- Remove a blank line.
---
 Documentation/virt/kvm/api.rst | 27 +++++++++++++++++++++++++++
 arch/x86/kvm/vmx/tdx.c         | 29 +++++++++++++++++++++++++++++
 include/uapi/linux/kvm.h       |  7 +++++++
 3 files changed, 63 insertions(+)

diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index 1bd2d42e6424..7c5bb6b5c2c2 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -7174,6 +7174,33 @@ The valid value for 'flags' is:
   - KVM_NOTIFY_CONTEXT_INVALID -- the VM context is corrupted and not valid
     in VMCS. It would run into unknown result if resume the target VM.
 
+::
+
+		/* KVM_EXIT_TDX_GET_QUOTE */
+		struct tdx_get_quote {
+			__u64 ret;
+			__u64 gpa;
+			__u64 size;
+		};
+
+If the exit reason is KVM_EXIT_TDX_GET_QUOTE, then it indicates that a TDX
+guest has requested to generate a TD-Quote signed by a service hosting
+TD-Quoting Enclave operating on the host. The 'gpa' field and 'size' specify
+the guest physical address and size of a shared-memory buffer, in which the
+TDX guest passes a TD Report. KVM checks the GPA from the TDX guest has the
+shared-bit set and drops the shared-bit in 'gpa' field before exiting to
+userspace. KVM doesn't do other sanity checks. The 'ret' field represents the
+return value of the GetQuote request. KVM only bridges the request to the
+userspace VMM, and the userspace VMM is responsible for setting up the return
+value since only userspace knows whether the request has been queued
+successfully or not. KVM sets the return code according to the 'ret' field
+before returning back to the TDX guest. When the request has been queued
+successfully, the TDX guest can poll the status field in the shared-memory area
+to check whether the Quote generation is completed or not. When completed, the
+generated Quote is returned via the same buffer. Userspace is required to handle
+the KVM exit reason as the initial support for TDX, however, userspace is
+allowed to set 'ret' filed to TDVMCALL_STATUS_SUBFUNC_UNSUPPORTED.
+
 ::
 
 		/* Fix the size of the union. */
diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
index 8134d5805b03..35428c6b5a67 100644
--- a/arch/x86/kvm/vmx/tdx.c
+++ b/arch/x86/kvm/vmx/tdx.c
@@ -1463,6 +1463,33 @@ static int tdx_get_td_vm_call_info(struct kvm_vcpu *vcpu)
 	return 1;
 }
 
+static int tdx_complete_get_quote(struct kvm_vcpu *vcpu)
+{
+	tdvmcall_set_return_code(vcpu, vcpu->run->tdx_get_quote.ret);
+	return 1;
+}
+
+static int tdx_get_quote(struct kvm_vcpu *vcpu)
+{
+	struct vcpu_tdx *tdx = to_tdx(vcpu);
+	u64 gpa = tdx->vp_enter_args.r12;
+	u64 size = tdx->vp_enter_args.r13;
+
+	/* The gpa of buffer must have shared bit set. */
+	if (vt_is_tdx_private_gpa(vcpu->kvm, gpa)) {
+		tdvmcall_set_return_code(vcpu, TDVMCALL_STATUS_INVALID_OPERAND);
+		return 1;
+	}
+
+	vcpu->run->exit_reason = KVM_EXIT_TDX_GET_QUOTE;
+	vcpu->run->tdx_get_quote.gpa = gpa & ~gfn_to_gpa(kvm_gfn_direct_bits(tdx->vcpu.kvm));
+	vcpu->run->tdx_get_quote.size = size;
+
+	vcpu->arch.complete_userspace_io = tdx_complete_get_quote;
+
+	return 0;
+}
+
 static int handle_tdvmcall(struct kvm_vcpu *vcpu)
 {
 	switch (tdvmcall_leaf(vcpu)) {
@@ -1472,6 +1499,8 @@ static int handle_tdvmcall(struct kvm_vcpu *vcpu)
 		return tdx_report_fatal_error(vcpu);
 	case TDVMCALL_GET_TD_VM_CALL_INFO:
 		return tdx_get_td_vm_call_info(vcpu);
+	case TDVMCALL_GET_QUOTE:
+		return tdx_get_quote(vcpu);
 	default:
 		break;
 	}
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index d00b85cb168c..e63e4df468b5 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -178,6 +178,7 @@ struct kvm_xen_exit {
 #define KVM_EXIT_NOTIFY           37
 #define KVM_EXIT_LOONGARCH_IOCSR  38
 #define KVM_EXIT_MEMORY_FAULT     39
+#define KVM_EXIT_TDX_GET_QUOTE    40
 
 /* For KVM_EXIT_INTERNAL_ERROR */
 /* Emulate instruction failed. */
@@ -447,6 +448,12 @@ struct kvm_run {
 			__u64 gpa;
 			__u64 size;
 		} memory_fault;
+		/* KVM_EXIT_TDX_GET_QUOTE */
+		struct {
+			__u64 ret;
+			__u64 gpa;
+			__u64 size;
+		} tdx_get_quote;
 		/* Fix the size of the union. */
 		char padding[256];
 	};
-- 
2.46.0

[RFC PATCH 3/4] KVM: TDX: Exit to userspace for GetTdVmCallInfo

[Binbin Wu]

Exit to userspace for TDG.VP.VMCALL<GetTdVmCallInfo> via a new KVM exit
reason to allow userspace to provide information about the support of
TDVMCALLs when r12 is 1 for the TDVMCALLs beyond the GHCI base API.

GHCI spec defines the GHCI base TDVMCALLs: <GetTdVmCallInfo>, <MapGPA>,
<ReportFatalError>, <Instruction.CPUID>, <#VE.RequestMMIO>,
<Instruction.HLT>, <Instruction.IO>, <Instruction.RDMSR> and
<Instruction.WRMSR>. They must be supported by VMM to support TDX guests.

For GetTdVmCallInfo
- When leaf (r12) to enumerate TDVMCALL functionality is set to 0,
  successful execution indicates all GHCI base TDVMCALLs listed above are
  supported.

  Update the KVM TDX document with the set of the GHCI base APIs.

- When leaf (r12) to enumerate TDVMCALL functionality is set to 1, it
  indicates the TDX guest is querying the supported TDVMCALLs beyond
  the GHCI base TDVMCALLs.
  Exit to userspace to let userspace set the TDVMCALL sub-function bit(s)
  accordingly to the leaf outputs.  KVM could set the TDVMCALL bit(s)
  supported by itself when the TDVMCALLs don't need support from userspace
  after returning from userspace and before entering guest. Currently, no
  such TDVMCALLs implemented, KVM just sets the values returned from
  userspace.

  A new KVM exit reason KVM_EXIT_TDX_GET_TDVMCALL_INFO and its structure
  are added. Userspace is required to handle the exit reason as the initial
  support for TDX.

Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Binbin Wu <binbin.wu@linux.intel.com>
---
 Documentation/virt/kvm/api.rst           | 17 +++++++++++
 Documentation/virt/kvm/x86/intel-tdx.rst |  9 ++++++
 arch/x86/kvm/vmx/tdx.c                   | 36 +++++++++++++++++++++---
 include/uapi/linux/kvm.h                 |  7 +++++
 4 files changed, 65 insertions(+), 4 deletions(-)

diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
index 7c5bb6b5c2c2..4a729841e000 100644
--- a/Documentation/virt/kvm/api.rst
+++ b/Documentation/virt/kvm/api.rst
@@ -7201,6 +7201,23 @@ generated Quote is returned via the same buffer. Userspace is required to handle
 the KVM exit reason as the initial support for TDX, however, userspace is
 allowed to set 'ret' filed to TDVMCALL_STATUS_SUBFUNC_UNSUPPORTED.
 
+::
+
+		/* KVM_EXIT_TDX_GET_TDVMCALL_INFO */
+		struct tdx_get_tdvmcall_info {
+			__u64 ret;
+			__u64 leaf;
+			__u64 leaf_output[4];
+		};
+
+If the exit reason is KVM_EXIT_TDX_GET_TDVMCALL_INFO, then it indicates that a
+TDX guest has requested to get the supporting status of TDVMCALLs. Currently,
+KVM only exits to userspace when the leaf value is 1, i.e, the TDX guest is
+querying the supporting status of TDVMCALLs beyond the GHCI base TDVMCALLs.
+Userspace is expected to set leaf outputs according to the layout defined in
+the GHCI spec if they are supported by userspace. Userspace is required to
+handle the exit reason as the initial support for TDX.
+
 ::
 
 		/* Fix the size of the union. */
diff --git a/Documentation/virt/kvm/x86/intel-tdx.rst b/Documentation/virt/kvm/x86/intel-tdx.rst
index 76bdd95334d6..69c64453e9ca 100644
--- a/Documentation/virt/kvm/x86/intel-tdx.rst
+++ b/Documentation/virt/kvm/x86/intel-tdx.rst
@@ -249,6 +249,15 @@ control flow is as follows:
 
 #. Run VCPU
 
+The GHCI base TDVMCALLs
+=======================
+The GHCI base TDVMCALLs are: <GetTdVmCallInfo>, <MapGPA>, <ReportFatalError>,
+<Instruction.CPUID>, <#VE.RequestMMIO>, <Instruction.HLT>, <Instruction.IO>,
+<Instruction.RDMSR> and <Instruction.WRMSR>. These base TDVMCALLs are mandatory
+for VMMs to support TDX guests.
+For the TDVMCALLs beyond the GHCI base TDVMCALLs, TDX guests can query the
+support status via GetTdVmCallInfo with leaf set to 1.
+
 References
 ==========
 
diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
index 35428c6b5a67..a3cd68f44a9c 100644
--- a/arch/x86/kvm/vmx/tdx.c
+++ b/arch/x86/kvm/vmx/tdx.c
@@ -1449,18 +1449,46 @@ static int tdx_emulate_mmio(struct kvm_vcpu *vcpu)
 	return 1;
 }
 
+static int tdx_complete_get_tdcall_info(struct kvm_vcpu *vcpu)
+{
+	struct vcpu_tdx *tdx = to_tdx(vcpu);
+
+	tdvmcall_set_return_code(vcpu, vcpu->run->tdx_get_tdvmcall_info.ret);
+
+	/*
+	 * For now, there is no TDVMCALL beyond GHCI base API supported by KVM
+	 * directly without the support from userspace, just set the value
+	 * returned from userspace.
+	 */
+	tdx->vp_enter_args.r11 = vcpu->run->tdx_get_tdvmcall_info.leaf_output[0];
+	tdx->vp_enter_args.r12 = vcpu->run->tdx_get_tdvmcall_info.leaf_output[1];
+	tdx->vp_enter_args.r13 = vcpu->run->tdx_get_tdvmcall_info.leaf_output[2];
+	tdx->vp_enter_args.r14 = vcpu->run->tdx_get_tdvmcall_info.leaf_output[3];
+
+	return 1;
+}
+
 static int tdx_get_td_vm_call_info(struct kvm_vcpu *vcpu)
 {
 	struct vcpu_tdx *tdx = to_tdx(vcpu);
 
-	if (tdx->vp_enter_args.r12)
-		tdvmcall_set_return_code(vcpu, TDVMCALL_STATUS_INVALID_OPERAND);
-	else {
+	switch (tdx->vp_enter_args.r12) {
+	case 0:
 		tdx->vp_enter_args.r11 = 0;
 		tdx->vp_enter_args.r13 = 0;
 		tdx->vp_enter_args.r14 = 0;
+		return 1;
+	case 1:
+		vcpu->run->tdx_get_tdvmcall_info.leaf = 1;
+		vcpu->run->exit_reason = KVM_EXIT_TDX_GET_TDVMCALL_INFO;
+		vcpu->arch.complete_userspace_io = tdx_complete_get_tdcall_info;
+		memset(vcpu->run->tdx_get_tdvmcall_info.leaf_output, 0,
+		       sizeof(vcpu->run->tdx_get_tdvmcall_info.leaf_output));
+		return 0;
+	default:
+		tdvmcall_set_return_code(vcpu, TDVMCALL_STATUS_INVALID_OPERAND);
+		return 1;
 	}
-	return 1;
 }
 
 static int tdx_complete_get_quote(struct kvm_vcpu *vcpu)
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index e63e4df468b5..0729b37ac911 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -179,6 +179,7 @@ struct kvm_xen_exit {
 #define KVM_EXIT_LOONGARCH_IOCSR  38
 #define KVM_EXIT_MEMORY_FAULT     39
 #define KVM_EXIT_TDX_GET_QUOTE    40
+#define KVM_EXIT_TDX_GET_TDVMCALL_INFO  41
 
 /* For KVM_EXIT_INTERNAL_ERROR */
 /* Emulate instruction failed. */
@@ -454,6 +455,12 @@ struct kvm_run {
 			__u64 gpa;
 			__u64 size;
 		} tdx_get_quote;
+		/* KVM_EXIT_TDX_GET_TDVMCALL_INFO */
+		struct {
+			__u64 ret;
+			__u64 leaf;
+			__u64 leaf_output[4];
+		} tdx_get_tdvmcall_info;
 		/* Fix the size of the union. */
 		char padding[256];
 	};
-- 
2.46.0

[RFC PATCH 4/4] KVM: TDX: Check KVM exit on KVM_HC_MAP_GPA_RANGE when TD finalize

[Binbin Wu]

Check userspace has enabled KVM exit on KVM_HC_MAP_GPA_RANGE during
KVM_TDX_FINALIZE_VM.

TDVMCALL_MAP_GPA is one of the GHCI base TDVMCALLs, so it must be
implemented by VMM to support TDX guests. KVM converts TDVMCALL_MAP_GPA
to KVM_HC_MAP_GPA_RANGE, which requires userspace to enable
KVM_CAP_EXIT_HYPERCALL with KVM_HC_MAP_GPA_RANGE bit set. Check it when
userspace requests KVM_TDX_FINALIZE_VM, so that there is no need to check
it during TDX guests running.

Signed-off-by: Binbin Wu <binbin.wu@linux.intel.com>
---
 Documentation/virt/kvm/x86/intel-tdx.rst |  3 +++
 arch/x86/kvm/vmx/tdx.c                   | 20 +++++++++-----------
 2 files changed, 12 insertions(+), 11 deletions(-)

diff --git a/Documentation/virt/kvm/x86/intel-tdx.rst b/Documentation/virt/kvm/x86/intel-tdx.rst
index 69c64453e9ca..41e38c38b034 100644
--- a/Documentation/virt/kvm/x86/intel-tdx.rst
+++ b/Documentation/virt/kvm/x86/intel-tdx.rst
@@ -246,6 +246,9 @@ control flow is as follows:
    * Prepare content of initial guest memory.
    * KVM_TDX_INIT_MEM_REGION: Add initial guest memory.
    * KVM_TDX_FINALIZE_VM: Finalize the measurement of the TDX guest.
+     Note: To support TDVMCALL_MAP_GPA, userspace must opt-in
+     KVM_CAP_EXIT_HYPERCALL with KVM_HC_MAP_GPA_RANGE before calling
+     KVM_TDX_FINALIZE_VM.
 
 #. Run VCPU
 
diff --git a/arch/x86/kvm/vmx/tdx.c b/arch/x86/kvm/vmx/tdx.c
index a3cd68f44a9c..7fc6e6b9c131 100644
--- a/arch/x86/kvm/vmx/tdx.c
+++ b/arch/x86/kvm/vmx/tdx.c
@@ -1209,17 +1209,6 @@ static int tdx_map_gpa(struct kvm_vcpu *vcpu)
 	u64 size = tdx->vp_enter_args.r13;
 	u64 ret;
 
-	/*
-	 * Converting TDVMCALL_MAP_GPA to KVM_HC_MAP_GPA_RANGE requires
-	 * userspace to enable KVM_CAP_EXIT_HYPERCALL with KVM_HC_MAP_GPA_RANGE
-	 * bit set.  If not, the error code is not defined in GHCI for TDX, use
-	 * TDVMCALL_STATUS_INVALID_OPERAND for this case.
-	 */
-	if (!user_exit_on_hypercall(vcpu->kvm, KVM_HC_MAP_GPA_RANGE)) {
-		ret = TDVMCALL_STATUS_INVALID_OPERAND;
-		goto error;
-	}
-
 	if (gpa + size <= gpa || !kvm_vcpu_is_legal_gpa(vcpu, gpa) ||
 	    !kvm_vcpu_is_legal_gpa(vcpu, gpa + size - 1) ||
 	    (vt_is_tdx_private_gpa(vcpu->kvm, gpa) !=
@@ -2821,6 +2810,15 @@ static int tdx_td_finalize(struct kvm *kvm, struct kvm_tdx_cmd *cmd)
 {
 	struct kvm_tdx *kvm_tdx = to_kvm_tdx(kvm);
 
+	/*
+	 * TDVMCALL_MAP_GPA is one of the GHCI base TDVMCALLs, so it must be
+	 * implemented by VMM to support TDX guests. KVM converts
+	 * TDVMCALL_MAP_GPA to KVM_HC_MAP_GPA_RANGE, which requires userspace to
+	 * enable KVM_CAP_EXIT_HYPERCALL with KVM_HC_MAP_GPA_RANGE bit set.
+	 */
+	if (!user_exit_on_hypercall(kvm, KVM_HC_MAP_GPA_RANGE))
+		return -EINVAL;
+
 	guard(mutex)(&kvm->slots_lock);
 
 	if (!is_hkid_assigned(kvm_tdx) || kvm_tdx->state == TD_STATE_RUNNABLE)
-- 
2.46.0

Re: [RFC PATCH 3/4] KVM: TDX: Exit to userspace for GetTdVmCallInfo

[Xiaoyao Li]

On 6/10/2025 10:14 AM, Binbin Wu wrote:
> Exit to userspace for TDG.VP.VMCALL<GetTdVmCallInfo> via a new KVM exit
> reason to allow userspace to provide information about the support of
> TDVMCALLs when r12 is 1 for the TDVMCALLs beyond the GHCI base API.
> 
> GHCI spec defines the GHCI base TDVMCALLs: <GetTdVmCallInfo>, <MapGPA>,
> <ReportFatalError>, <Instruction.CPUID>, <#VE.RequestMMIO>,
> <Instruction.HLT>, <Instruction.IO>, <Instruction.RDMSR> and
> <Instruction.WRMSR>. They must be supported by VMM to support TDX guests.
> 
> For GetTdVmCallInfo
> - When leaf (r12) to enumerate TDVMCALL functionality is set to 0,
>    successful execution indicates all GHCI base TDVMCALLs listed above are
>    supported.
> 
>    Update the KVM TDX document with the set of the GHCI base APIs.
> 
> - When leaf (r12) to enumerate TDVMCALL functionality is set to 1, it
>    indicates the TDX guest is querying the supported TDVMCALLs beyond
>    the GHCI base TDVMCALLs.
>    Exit to userspace to let userspace set the TDVMCALL sub-function bit(s)
>    accordingly to the leaf outputs.  KVM could set the TDVMCALL bit(s)
>    supported by itself when the TDVMCALLs don't need support from userspace
>    after returning from userspace and before entering guest. Currently, no
>    such TDVMCALLs implemented, KVM just sets the values returned from
>    userspace.
> 
>    A new KVM exit reason KVM_EXIT_TDX_GET_TDVMCALL_INFO and its structure
>    are added. Userspace is required to handle the exit reason as the initial
>    support for TDX.

It doesn't look like a good and correct design.

Consider the case that userspace supports SetupEventNotifyInterrupt and 
returns bit 1 of leaf_output[0] as 1 to KVM, and KVM returns it to TD 
guest for TDVMCALL_GET_TD_VM_CALL_INFO. So TD guest treats it as 
SetupEventNotifyInterrupt is support. But when TD guest issues this 
TDVMCALL, KVM doesn't support the exit to userspace for this specific 
leaf and userspace doesn't have chance to handle it.

I have a proposal:

Implement KVM_CAP_TDX_USER_EXIT_TDVMCALL

- on check of this cap, KVM returns the bitmap of TDVMCALL leafs that
   allows userspace to opt-in the user exit; e.g.,

#define TDX_USER_EXIT_TDVMCALL_GETQUOTE 		 BIT_ULL(0)
#define TDX_USER_EXIT_TDVMCALL_SetupEventNotifyInterrupt BIT_ULL(1)
...

- on enable of this cap, KVM allows userspace to opt-in which leaf will
   exit to usersapce for handling;

- KVM returns the result for leaf(r12)==1 of <GetTdVmCallInfo> based on
   the KVM_CAP_TDX_USER_EXIT_TDVMCALL enabled by userspace and its own
   capability;

If a non-base GHCI TDVMCALL is supported by KVM itself and no need exit 
to userspace, it's not reported in KVM_CAP_TDX_USER_EXIT_TDVMCALL so 
that usersapce cannot enable the user exit of this leaf. But KVM will 
still return the corresponding bit of this leaf as 1 for leaf(r12)==1 of 
<GetTdVmCallInfo> since it's supported by KVM itself.

If a non-base GHCI TDVMCALL is not supported by KVM itself but KVM 
allows/supports userspace to opt-in the user exit. e.g., <Getquote>. 
check of KVM_CAP_TDX_USER_EXIT_TDVMCALL will have the bit of this leaf set.

  - only when usersapce enable the corresponding bit by calling
    enable_cap(KVM_CAP_TDX_USER_EXIT_TDVMCALL), will KVM return the
    corresponding bit of this leaf as 1 for leaf(r12)==1 of
    <GetTdVmCallInfo> and exit to userspace for such leaf.

  - if userspace doesn't enable the corresponding bit with
    enable_cap(KVM_CAP_TDX_USER_EXIT_TDVMCALL). KVM will return 0 for
    the corresponding bit of this leaf for leaf(r12)==1 of
    <GetTdVmCallInfo> and return TDVMCALL_STATUS_SUBFUNC_UNSUPPORTED when
    TDX guest issues such leaf;

If a non-base GHCI TDVMCALL is not supported by KVM and KVM doesn't 
allow/support userspace to opt-in the user exit. e.g., 
<SetupEventNotifyInterrupt>. check of KVM_CAP_TDX_USER_EXIT_TDVMCALL 
will not have the bit of this leaf set, so that userspace cannot opt-in 
the user exit of such leaf. leaf(r12)==1 of <GetTdVmCallInfo> return the 
bit of such leaf as 0 and TDX guest request of such leaf gets 
TDVMCALL_STATUS_SUBFUNC_UNSUPPORTED.

Re: [RFC PATCH 3/4] KVM: TDX: Exit to userspace for GetTdVmCallInfo

[Edgecombe, Rick P]

On Tue, 2025-06-10 at 17:16 +0800, Xiaoyao Li wrote:
> >     A new KVM exit reason KVM_EXIT_TDX_GET_TDVMCALL_INFO and its structure
> >     are added. Userspace is required to handle the exit reason as the
> > initial
> >     support for TDX.
> 
> It doesn't look like a good and correct design.
> 
> Consider the case that userspace supports SetupEventNotifyInterrupt and 
> returns bit 1 of leaf_output[0] as 1 to KVM, and KVM returns it to TD 
> guest for TDVMCALL_GET_TD_VM_CALL_INFO. So TD guest treats it as 
> SetupEventNotifyInterrupt is support. But when TD guest issues this 
> TDVMCALL, KVM doesn't support the exit to userspace for this specific 
> leaf and userspace doesn't have chance to handle it.

Why do we need an opt-in interface instead of a way to expose which exit's are
supported by KVM? I would think the need for a TDVMCALL opt-in interface would
only come up if there was a bad guest that was making TDVMCALLs that it did not
see in GetTdVmCallInfo. So that we would actually require an opt-in is not
guaranteed. 

Another consideration could be how to handle GetQuote for an eventual TDVMCALL
opt-in interface, should it be needed. The problem would be GetQuote would be
opted in by default and make the interface weird. But we may not want to have a
TDVMCall specific opt-in interface. There could be other TDX behaviors that we
need to opt-in around. In which case the opt-in interface could be more generic,
and by implementing the TDVMCall opt-in interface ahead of time we would end up
with two opt-in interfaces instead of one.

So how about just adding a field to struct kvm_tdx_capabilities to describe the
KVM TDVMcalls? Or some other place? But don't invent an opt-in interface
until/if we need it.

Re: [RFC PATCH 3/4] KVM: TDX: Exit to userspace for GetTdVmCallInfo

[Edgecombe, Rick P]

On Tue, 2025-06-10 at 09:50 -0700, Rick Edgecombe wrote:
> Why do we need an opt-in interface instead of a way to expose which exit's are
> supported by KVM? I would think the need for a TDVMCALL opt-in interface would
> only come up if there was a bad guest that was making TDVMCALLs that it did not
> see in GetTdVmCallInfo. So that we would actually require an opt-in is not
> guaranteed. 
> 
> Another consideration could be how to handle GetQuote for an eventual TDVMCALL
> opt-in interface, should it be needed. The problem would be GetQuote would be
> opted in by default and make the interface weird. But we may not want to have a
> TDVMCall specific opt-in interface. There could be other TDX behaviors that we
> need to opt-in around. In which case the opt-in interface could be more generic,
> and by implementing the TDVMCall opt-in interface ahead of time we would end up
> with two opt-in interfaces instead of one.
> 
> So how about just adding a field to struct kvm_tdx_capabilities to describe the
> KVM TDVMcalls? Or some other place? But don't invent an opt-in interface
> until/if we need it.

Oh, and there already is a hypercall exit opt-in interface, so
KVM_CAP_TDX_USER_EXIT_TDVMCALL would overlap with it, right?

Re: [RFC PATCH 4/4] KVM: TDX: Check KVM exit on KVM_HC_MAP_GPA_RANGE when TD finalize

[Edgecombe, Rick P]

On Tue, 2025-06-10 at 10:14 +0800, Binbin Wu wrote:
> Check userspace has enabled KVM exit on KVM_HC_MAP_GPA_RANGE during
> KVM_TDX_FINALIZE_VM.
> 
> TDVMCALL_MAP_GPA is one of the GHCI base TDVMCALLs, so it must be
> implemented by VMM to support TDX guests. KVM converts TDVMCALL_MAP_GPA
> to KVM_HC_MAP_GPA_RANGE, which requires userspace to enable
> KVM_CAP_EXIT_HYPERCALL with KVM_HC_MAP_GPA_RANGE bit set. Check it when
> userspace requests KVM_TDX_FINALIZE_VM, so that there is no need to check
> it during TDX guests running.
> 
> Signed-off-by: Binbin Wu <binbin.wu@linux.intel.com>

Do we need this change? It seems reasonable, but I don't think we need KVM to
ensure that userspace creates a TD that meets the GHCI spec. So I'm not sure
about the justification.

It seems like the reasoning could be just to shrink the possible configurations
KVM has to think about, and that we only have the option to do this now before
the ABI becomes harder to change.

Did you need any QEMU changes as a result of this patch?

Wait, actually I think the patch is wrong, because KVM_CAP_EXIT_HYPERCALL could
be called again after KVM_TDX_FINALIZE_VM. In which case userspace could get an
exit unexpectedly. So should we drop this patch?

Re: [RFC PATCH 4/4] KVM: TDX: Check KVM exit on KVM_HC_MAP_GPA_RANGE when TD finalize

[Sean Christopherson]

On Tue, Jun 10, 2025, Rick P Edgecombe wrote:
> On Tue, 2025-06-10 at 10:14 +0800, Binbin Wu wrote:
> > Check userspace has enabled KVM exit on KVM_HC_MAP_GPA_RANGE during
> > KVM_TDX_FINALIZE_VM.
> > 
> > TDVMCALL_MAP_GPA is one of the GHCI base TDVMCALLs, so it must be
> > implemented by VMM to support TDX guests. KVM converts TDVMCALL_MAP_GPA
> > to KVM_HC_MAP_GPA_RANGE, which requires userspace to enable
> > KVM_CAP_EXIT_HYPERCALL with KVM_HC_MAP_GPA_RANGE bit set. Check it when
> > userspace requests KVM_TDX_FINALIZE_VM, so that there is no need to check
> > it during TDX guests running.
> > 
> > Signed-off-by: Binbin Wu <binbin.wu@linux.intel.com>
> 
> Do we need this change? It seems reasonable, but I don't think we need KVM to
> ensure that userspace creates a TD that meets the GHCI spec.

+1.  We do need to be careful about unintentionally creating ABI, but generally
speaking KVM shouldn't police userspace.

> So I'm not sure about the justification.
> 
> It seems like the reasoning could be just to shrink the possible configurations
> KVM has to think about, and that we only have the option to do this now before
> the ABI becomes harder to change.
> 
> Did you need any QEMU changes as a result of this patch?
> 
> Wait, actually I think the patch is wrong, because KVM_CAP_EXIT_HYPERCALL could
> be called again after KVM_TDX_FINALIZE_VM. In which case userspace could get an
> exit unexpectedly. So should we drop this patch?

Yes, drop it.

Re: [RFC PATCH 4/4] KVM: TDX: Check KVM exit on KVM_HC_MAP_GPA_RANGE when TD finalize

[Binbin Wu]

On 6/11/2025 3:58 AM, Sean Christopherson wrote:
> On Tue, Jun 10, 2025, Rick P Edgecombe wrote:
>> On Tue, 2025-06-10 at 10:14 +0800, Binbin Wu wrote:
>>> Check userspace has enabled KVM exit on KVM_HC_MAP_GPA_RANGE during
>>> KVM_TDX_FINALIZE_VM.
>>>
>>> TDVMCALL_MAP_GPA is one of the GHCI base TDVMCALLs, so it must be
>>> implemented by VMM to support TDX guests. KVM converts TDVMCALL_MAP_GPA
>>> to KVM_HC_MAP_GPA_RANGE, which requires userspace to enable
>>> KVM_CAP_EXIT_HYPERCALL with KVM_HC_MAP_GPA_RANGE bit set. Check it when
>>> userspace requests KVM_TDX_FINALIZE_VM, so that there is no need to check
>>> it during TDX guests running.
>>>
>>> Signed-off-by: Binbin Wu <binbin.wu@linux.intel.com>
>> Do we need this change? It seems reasonable, but I don't think we need KVM to
>> ensure that userspace creates a TD that meets the GHCI spec.
> +1.  We do need to be careful about unintentionally creating ABI, but generally
> speaking KVM shouldn't police userspace.

OK.

>> So I'm not sure about the justification.
>>
>> It seems like the reasoning could be just to shrink the possible configurations
>> KVM has to think about, and that we only have the option to do this now before
>> the ABI becomes harder to change.
>>
>> Did you need any QEMU changes as a result of this patch?
>>
>> Wait, actually I think the patch is wrong, because KVM_CAP_EXIT_HYPERCALL could
>> be called again after KVM_TDX_FINALIZE_VM. In which case userspace could get an
>> exit unexpectedly. So should we drop this patch?
> Yes, drop it.
>
So, when the TDX guest calls MapGPA and KVM finds userspace doesn't opt-in
KVM_HC_MAP_GPA_RANGE, just return error to userspace?

Re: [RFC PATCH 3/4] KVM: TDX: Exit to userspace for GetTdVmCallInfo

[Binbin Wu]

On 6/10/2025 5:16 PM, Xiaoyao Li wrote:
> On 6/10/2025 10:14 AM, Binbin Wu wrote:
>> Exit to userspace for TDG.VP.VMCALL<GetTdVmCallInfo> via a new KVM exit
>> reason to allow userspace to provide information about the support of
>> TDVMCALLs when r12 is 1 for the TDVMCALLs beyond the GHCI base API.
>>
>> GHCI spec defines the GHCI base TDVMCALLs: <GetTdVmCallInfo>, <MapGPA>,
>> <ReportFatalError>, <Instruction.CPUID>, <#VE.RequestMMIO>,
>> <Instruction.HLT>, <Instruction.IO>, <Instruction.RDMSR> and
>> <Instruction.WRMSR>. They must be supported by VMM to support TDX guests.
>>
>> For GetTdVmCallInfo
>> - When leaf (r12) to enumerate TDVMCALL functionality is set to 0,
>>    successful execution indicates all GHCI base TDVMCALLs listed above are
>>    supported.
>>
>>    Update the KVM TDX document with the set of the GHCI base APIs.
>>
>> - When leaf (r12) to enumerate TDVMCALL functionality is set to 1, it
>>    indicates the TDX guest is querying the supported TDVMCALLs beyond
>>    the GHCI base TDVMCALLs.
>>    Exit to userspace to let userspace set the TDVMCALL sub-function bit(s)
>>    accordingly to the leaf outputs.  KVM could set the TDVMCALL bit(s)
>>    supported by itself when the TDVMCALLs don't need support from userspace
>>    after returning from userspace and before entering guest. Currently, no
>>    such TDVMCALLs implemented, KVM just sets the values returned from
>>    userspace.
>>
>>    A new KVM exit reason KVM_EXIT_TDX_GET_TDVMCALL_INFO and its structure
>>    are added. Userspace is required to handle the exit reason as the initial
>>    support for TDX.
>
> It doesn't look like a good and correct design.
>
> Consider the case that userspace supports SetupEventNotifyInterrupt and returns bit 1 of leaf_output[0] as 1 to KVM, and KVM returns it to TD guest for TDVMCALL_GET_TD_VM_CALL_INFO. So TD guest treats it as SetupEventNotifyInterrupt is support. But when TD guest issues this TDVMCALL, KVM doesn't support the exit to userspace for this specific leaf and userspace doesn't have chance to handle it.
Previously, I also had the idea of setting the information based on userspace's
opt-ins.

But for simplicity, this patch set doesn't adopt the opt-in mechanism for KVM
exit reasons due to TDVMCALLs.

To resolve the issue you mentions that userspace could set a bit that KVM
doesn't support the exit to userspace, KVM could mask off the bit(s) not
supported by KVM before returning back to guest.

If we agree that it's the right time to have the opt-in, we could go the opt-in
way and KVM could set the information based on userspace's opt-ins without
exit to userspace for GetTdVmCallInfo.

Re: [RFC PATCH 3/4] KVM: TDX: Exit to userspace for GetTdVmCallInfo

[Binbin Wu]

On 6/11/2025 12:54 AM, Edgecombe, Rick P wrote:
> On Tue, 2025-06-10 at 09:50 -0700, Rick Edgecombe wrote:
>> Why do we need an opt-in interface instead of a way to expose which exit's are
>> supported by KVM? I would think the need for a TDVMCALL opt-in interface would
>> only come up if there was a bad guest that was making TDVMCALLs that it did not
>> see in GetTdVmCallInfo.

The opt-in interface can eliminate some requirements for userspace.
E.g, for GetQuote, this patch set enforces userspace to handle the exit reason
due to GetQuote as the initial support, because KVM doesn't know if userspace
is able to handle the exit reason or not without userspace's opt-in, unless
it's handled by default in userspace.

>>   So that we would actually require an opt-in is not
>> guaranteed.
>>
>> Another consideration could be how to handle GetQuote for an eventual TDVMCALL
>> opt-in interface, should it be needed. The problem would be GetQuote would be
>> opted in by default and make the interface weird. But we may not want to have a
>> TDVMCall specific opt-in interface. There could be other TDX behaviors that we
>> need to opt-in around. In which case the opt-in interface could be more generic,
>> and by implementing the TDVMCall opt-in interface ahead of time we would end up
>> with two opt-in interfaces instead of one.

Maybe we can use a TDX specific opt-in interface instead of TDVMCALL specific
interface.
But not sure we should add it now or later.

>>
>> So how about just adding a field to struct kvm_tdx_capabilities to describe the
>> KVM TDVMcalls? Or some other place? But don't invent an opt-in interface
>> until/if we need it.
> Oh, and there already is a hypercall exit opt-in interface, so
> KVM_CAP_TDX_USER_EXIT_TDVMCALL would overlap with it, right?
Not sure what does "overlap" mean here.

They have different namespaces, so they don't impact each other.

Or did you mean it's a duplication both having KVM_CAP_EXIT_HYPERCALL and
KVM_CAP_TDX_USER_EXIT_TDVMCALL?

Re: [RFC PATCH 3/4] KVM: TDX: Exit to userspace for GetTdVmCallInfo

[Xiaoyao Li]

On 6/11/2025 9:37 AM, Binbin Wu wrote:
> 
> 
> On 6/10/2025 5:16 PM, Xiaoyao Li wrote:
>> On 6/10/2025 10:14 AM, Binbin Wu wrote:
>>> Exit to userspace for TDG.VP.VMCALL<GetTdVmCallInfo> via a new KVM exit
>>> reason to allow userspace to provide information about the support of
>>> TDVMCALLs when r12 is 1 for the TDVMCALLs beyond the GHCI base API.
>>>
>>> GHCI spec defines the GHCI base TDVMCALLs: <GetTdVmCallInfo>, <MapGPA>,
>>> <ReportFatalError>, <Instruction.CPUID>, <#VE.RequestMMIO>,
>>> <Instruction.HLT>, <Instruction.IO>, <Instruction.RDMSR> and
>>> <Instruction.WRMSR>. They must be supported by VMM to support TDX 
>>> guests.
>>>
>>> For GetTdVmCallInfo
>>> - When leaf (r12) to enumerate TDVMCALL functionality is set to 0,
>>>    successful execution indicates all GHCI base TDVMCALLs listed 
>>> above are
>>>    supported.
>>>
>>>    Update the KVM TDX document with the set of the GHCI base APIs.
>>>
>>> - When leaf (r12) to enumerate TDVMCALL functionality is set to 1, it
>>>    indicates the TDX guest is querying the supported TDVMCALLs beyond
>>>    the GHCI base TDVMCALLs.
>>>    Exit to userspace to let userspace set the TDVMCALL sub-function 
>>> bit(s)
>>>    accordingly to the leaf outputs.  KVM could set the TDVMCALL bit(s)
>>>    supported by itself when the TDVMCALLs don't need support from 
>>> userspace
>>>    after returning from userspace and before entering guest. 
>>> Currently, no
>>>    such TDVMCALLs implemented, KVM just sets the values returned from
>>>    userspace.
>>>
>>>    A new KVM exit reason KVM_EXIT_TDX_GET_TDVMCALL_INFO and its 
>>> structure
>>>    are added. Userspace is required to handle the exit reason as the 
>>> initial
>>>    support for TDX.
>>
>> It doesn't look like a good and correct design.
>>
>> Consider the case that userspace supports SetupEventNotifyInterrupt 
>> and returns bit 1 of leaf_output[0] as 1 to KVM, and KVM returns it to 
>> TD guest for TDVMCALL_GET_TD_VM_CALL_INFO. So TD guest treats it as 
>> SetupEventNotifyInterrupt is support. But when TD guest issues this 
>> TDVMCALL, KVM doesn't support the exit to userspace for this specific 
>> leaf and userspace doesn't have chance to handle it.
> Previously, I also had the idea of setting the information based on 
> userspace's
> opt-ins.
> 
> But for simplicity, this patch set doesn't adopt the opt-in mechanism 
> for KVM
> exit reasons due to TDVMCALLs.
> 
> To resolve the issue you mentions that userspace could set a bit that KVM
> doesn't support the exit to userspace, KVM could mask off the bit(s) not
> supported by KVM before returning back to guest.

silently mask off the value provided from userspace is not a stable ABI 
from userspace perspective. A kvm cap to report what value is 
allowed/supported is still helpful.

> If we agree that it's the right time to have the opt-in, we could go the 
> opt-in
> way and KVM could set the information based on userspace's opt-ins without
> exit to userspace for GetTdVmCallInfo.

Let's see what Paolo and Sean will say.

Re: [RFC PATCH 3/4] KVM: TDX: Exit to userspace for GetTdVmCallInfo

[Xiaoyao Li]

On 6/11/2025 10:04 AM, Binbin Wu wrote:
> 
> 
> On 6/11/2025 12:54 AM, Edgecombe, Rick P wrote:
>> On Tue, 2025-06-10 at 09:50 -0700, Rick Edgecombe wrote:
>>> Why do we need an opt-in interface instead of a way to expose which 
>>> exit's are
>>> supported by KVM? I would think the need for a TDVMCALL opt-in 
>>> interface would
>>> only come up if there was a bad guest that was making TDVMCALLs that 
>>> it did not
>>> see in GetTdVmCallInfo.
> 
> The opt-in interface can eliminate some requirements for userspace.
> E.g, for GetQuote, this patch set enforces userspace to handle the exit 
> reason
> due to GetQuote as the initial support, because KVM doesn't know if 
> userspace
> is able to handle the exit reason or not without userspace's opt-in, unless
> it's handled by default in userspace.

Beside it.

opt-in is not needed if we only care about <getquote> for now since KVM 
makes <getquote> exit to userspace unconditionally.

But to support any new TDVMCALL leaf that needs to exit userspace, we 
will have to use opt-in. So to me, implement the opt-in at the first 
place as a common interface for all the optional tdvmcall leafs instead 
of making <getquote> specific is not a bad idea to me, especially the 
opt-in implementation doesn't look complicated.

>>>   So that we would actually require an opt-in is not
>>> guaranteed.
>>>
>>> Another consideration could be how to handle GetQuote for an eventual 
>>> TDVMCALL
>>> opt-in interface, should it be needed. The problem would be GetQuote 
>>> would be
>>> opted in by default and make the interface weird. But we may not want 
>>> to have a
>>> TDVMCall specific opt-in interface. There could be other TDX 
>>> behaviors that we
>>> need to opt-in around. In which case the opt-in interface could be 
>>> more generic,
>>> and by implementing the TDVMCall opt-in interface ahead of time we 
>>> would end up
>>> with two opt-in interfaces instead of one.
> 
> Maybe we can use a TDX specific opt-in interface instead of TDVMCALL 
> specific
> interface.
> But not sure we should add it now or later.

For simplicity, I prefer separate opt-in interfaces, it makes code simpler.

Re: [RFC PATCH 4/4] KVM: TDX: Check KVM exit on KVM_HC_MAP_GPA_RANGE when TD finalize

[Sean Christopherson]

On Wed, Jun 11, 2025, Binbin Wu wrote:
> On 6/11/2025 3:58 AM, Sean Christopherson wrote:
> > On Tue, Jun 10, 2025, Rick P Edgecombe wrote:
> > > It seems like the reasoning could be just to shrink the possible configurations
> > > KVM has to think about, and that we only have the option to do this now before
> > > the ABI becomes harder to change.
> > > 
> > > Did you need any QEMU changes as a result of this patch?
> > > 
> > > Wait, actually I think the patch is wrong, because KVM_CAP_EXIT_HYPERCALL could
> > > be called again after KVM_TDX_FINALIZE_VM. In which case userspace could get an
> > > exit unexpectedly. So should we drop this patch?
> > Yes, drop it.
> > 
> So, when the TDX guest calls MapGPA and KVM finds userspace doesn't opt-in
> KVM_HC_MAP_GPA_RANGE, just return error to userspace?

Why can't KVM just do what it already does, and return an error to the guest?

	if (!user_exit_on_hypercall(vcpu->kvm, KVM_HC_MAP_GPA_RANGE)) {
		ret = TDVMCALL_STATUS_INVALID_OPERAND;
		goto error;
	}

Re: [RFC PATCH 4/4] KVM: TDX: Check KVM exit on KVM_HC_MAP_GPA_RANGE when TD finalize

[Xiaoyao Li]

On 6/11/2025 9:36 PM, Sean Christopherson wrote:
> On Wed, Jun 11, 2025, Binbin Wu wrote:
>> On 6/11/2025 3:58 AM, Sean Christopherson wrote:
>>> On Tue, Jun 10, 2025, Rick P Edgecombe wrote:
>>>> It seems like the reasoning could be just to shrink the possible configurations
>>>> KVM has to think about, and that we only have the option to do this now before
>>>> the ABI becomes harder to change.
>>>>
>>>> Did you need any QEMU changes as a result of this patch?
>>>>
>>>> Wait, actually I think the patch is wrong, because KVM_CAP_EXIT_HYPERCALL could
>>>> be called again after KVM_TDX_FINALIZE_VM. In which case userspace could get an
>>>> exit unexpectedly. So should we drop this patch?
>>> Yes, drop it.
>>>
>> So, when the TDX guest calls MapGPA and KVM finds userspace doesn't opt-in
>> KVM_HC_MAP_GPA_RANGE, just return error to userspace?
> 
> Why can't KVM just do what it already does, and return an error to the guest?

Because GHCI requires it must be supported. No matter with the old GHCI 
that only allows <GetTdVmCallInfo> to succeed and the success of 
<GetTdVmCallInfo> means all the TDVMCALL leafs are support, or the 
proposed updated GHCI that defines <MapGpa> as one of the base API/leaf, 
and the base API must be supported by VMM.

Binbin wants to honor it.

> 	if (!user_exit_on_hypercall(vcpu->kvm, KVM_HC_MAP_GPA_RANGE)) {
> 		ret = TDVMCALL_STATUS_INVALID_OPERAND;
> 		goto error;
> 	}

Re: [RFC PATCH 4/4] KVM: TDX: Check KVM exit on KVM_HC_MAP_GPA_RANGE when TD finalize

[Edgecombe, Rick P]

On Wed, 2025-06-11 at 22:01 +0800, Xiaoyao Li wrote:
> > > So, when the TDX guest calls MapGPA and KVM finds userspace doesn't opt-in
> > > KVM_HC_MAP_GPA_RANGE, just return error to userspace?
> > 
> > Why can't KVM just do what it already does, and return an error to the
> > guest?
> 
> Because GHCI requires it must be supported. No matter with the old GHCI 
> that only allows <GetTdVmCallInfo> to succeed and the success of 
> <GetTdVmCallInfo> means all the TDVMCALL leafs are support, or the 
> proposed updated GHCI that defines <MapGpa> as one of the base API/leaf, 
> and the base API must be supported by VMM.
> 
> Binbin wants to honor it.

But KVM doesn't need to support all ways that userspace could meet the GHCI
spec. If userspace opts-in to the exit, they will meet the spec. If they
configure KVM differently then they wont, but this is their decision.

Re: [RFC PATCH 3/4] KVM: TDX: Exit to userspace for GetTdVmCallInfo

[Edgecombe, Rick P]

On Wed, 2025-06-11 at 10:37 +0800, Xiaoyao Li wrote:
> > Maybe we can use a TDX specific opt-in interface instead of TDVMCALL 
> > specific
> > interface.
> > But not sure we should add it now or later.
> 
> For simplicity, I prefer separate opt-in interfaces, it makes code simpler.

What is the problem with using the existing exit opt-in interface?

Re: [RFC PATCH 4/4] KVM: TDX: Check KVM exit on KVM_HC_MAP_GPA_RANGE when TD finalize

[Xiaoyao Li]

On 6/11/2025 10:04 PM, Edgecombe, Rick P wrote:
> On Wed, 2025-06-11 at 22:01 +0800, Xiaoyao Li wrote:
>>>> So, when the TDX guest calls MapGPA and KVM finds userspace doesn't opt-in
>>>> KVM_HC_MAP_GPA_RANGE, just return error to userspace?
>>>
>>> Why can't KVM just do what it already does, and return an error to the
>>> guest?
>>
>> Because GHCI requires it must be supported. No matter with the old GHCI
>> that only allows <GetTdVmCallInfo> to succeed and the success of
>> <GetTdVmCallInfo> means all the TDVMCALL leafs are support, or the
>> proposed updated GHCI that defines <MapGpa> as one of the base API/leaf,
>> and the base API must be supported by VMM.
>>
>> Binbin wants to honor it.
> 
> But KVM doesn't need to support all ways that userspace could meet the GHCI
> spec. If userspace opts-in to the exit, they will meet the spec. If they
> configure KVM differently then they wont, but this is their decision.

I agree with you and Sean. And I'm trying to answer Sean's question on 
behalf of Binbin.

Strictly speaking, KVM can be blamed for some reason. Because it is KVM 
that returns success for <GetTdVmCallInfo> unconditionally when r12 == 0 
  to report that all the (base) leafs are supported.

But I totally agree with KVM cannot guarantee userspace will behave 
correctly. Even with this patch that KVM mandates the userspace to 
enable user exit of KVM_HC_MAP_GPA_RANGE, it's still possible for a 
misbehaved userspace to error to TD guest on KVM_HC_MAP_GPA_RANGE and 
breaks the semantics of successful <GetTdVmCallInfo>.

So I'm with you and Sean.

Re: [RFC PATCH 3/4] KVM: TDX: Exit to userspace for GetTdVmCallInfo

[Xiaoyao Li]

On 6/11/2025 10:17 PM, Edgecombe, Rick P wrote:
> On Wed, 2025-06-11 at 10:37 +0800, Xiaoyao Li wrote:
>>> Maybe we can use a TDX specific opt-in interface instead of TDVMCALL
>>> specific
>>> interface.
>>> But not sure we should add it now or later.
>>
>> For simplicity, I prefer separate opt-in interfaces, it makes code simpler.
> 
> What is the problem with using the existing exit opt-in interface?

It mixes up common KVM defined hypercall leafs (KVM_HC_*) with TDX 
specific TDVMCALL leafs. Surely it can work but it just doesn't look 
clean to me.

Re: [RFC PATCH 3/4] KVM: TDX: Exit to userspace for GetTdVmCallInfo

[Edgecombe, Rick P]

On Wed, 2025-06-11 at 22:34 +0800, Xiaoyao Li wrote:
> > What is the problem with using the existing exit opt-in interface?
> 
> It mixes up common KVM defined hypercall leafs (KVM_HC_*) with TDX 
> specific TDVMCALL leafs. Surely it can work but it just doesn't look 
> clean to me.

 - TDVMCallInfo is used to expose which TDVMCalls are expose to the guest
(clean)
 - KVM HC opt-in is used to expose which exits KVM should fwd to userspace
(seems clean to me)

There is an asymmetry in that what is returned in TDVMCallInfo can't be blindly
dumped into a KVM opt-in interface. But the logic matching TDVMCallInfo leaf <->
KVM HC needs to be somewhere. The default place should be userspace.

Did I miss anything?

Re: [RFC PATCH 3/4] KVM: TDX: Exit to userspace for GetTdVmCallInfo

[Sean Christopherson]

On Wed, Jun 11, 2025, Xiaoyao Li wrote:
> On 6/11/2025 9:37 AM, Binbin Wu wrote:
> > On 6/10/2025 5:16 PM, Xiaoyao Li wrote:
> > > On 6/10/2025 10:14 AM, Binbin Wu wrote:
> > > > Exit to userspace for TDG.VP.VMCALL<GetTdVmCallInfo> via a new KVM exit
> > > > reason to allow userspace to provide information about the support of
> > > > TDVMCALLs when r12 is 1 for the TDVMCALLs beyond the GHCI base API.
> > > > 
> > > > GHCI spec defines the GHCI base TDVMCALLs: <GetTdVmCallInfo>, <MapGPA>,
> > > > <ReportFatalError>, <Instruction.CPUID>, <#VE.RequestMMIO>,
> > > > <Instruction.HLT>, <Instruction.IO>, <Instruction.RDMSR> and
> > > > <Instruction.WRMSR>. They must be supported by VMM to support
> > > > TDX guests.
> > > > 
> > > > For GetTdVmCallInfo
> > > > - When leaf (r12) to enumerate TDVMCALL functionality is set to 0,
> > > >    successful execution indicates all GHCI base TDVMCALLs listed
> > > > above are
> > > >    supported.
> > > > 
> > > >    Update the KVM TDX document with the set of the GHCI base APIs.
> > > > 
> > > > - When leaf (r12) to enumerate TDVMCALL functionality is set to 1, it
> > > >    indicates the TDX guest is querying the supported TDVMCALLs beyond
> > > >    the GHCI base TDVMCALLs.
> > > >    Exit to userspace to let userspace set the TDVMCALL
> > > > sub-function bit(s)
> > > >    accordingly to the leaf outputs.  KVM could set the TDVMCALL bit(s)
> > > >    supported by itself when the TDVMCALLs don't need support
> > > > from userspace
> > > >    after returning from userspace and before entering guest.
> > > > Currently, no
> > > >    such TDVMCALLs implemented, KVM just sets the values returned from
> > > >    userspace.
> > > > 
> > > >    A new KVM exit reason KVM_EXIT_TDX_GET_TDVMCALL_INFO and its
> > > > structure
> > > >    are added. Userspace is required to handle the exit reason as
> > > > the initial
> > > >    support for TDX.
> > > 
> > > It doesn't look like a good and correct design.
> > > 
> > > Consider the case that userspace supports SetupEventNotifyInterrupt
> > > and returns bit 1 of leaf_output[0] as 1 to KVM, and KVM returns it
> > > to TD guest for TDVMCALL_GET_TD_VM_CALL_INFO. So TD guest treats it
> > > as SetupEventNotifyInterrupt is support. But when TD guest issues
> > > this TDVMCALL, KVM doesn't support the exit to userspace for this
> > > specific leaf and userspace doesn't have chance to handle it.
> > Previously, I also had the idea of setting the information based on
> > userspace's opt-ins.
> > 
> > But for simplicity, this patch set doesn't adopt the opt-in mechanism for
> > KVM exit reasons due to TDVMCALLs.
> > 
> > To resolve the issue you mentions that userspace could set a bit that KVM
> > doesn't support the exit to userspace, KVM could mask off the bit(s) not
> > supported by KVM before returning back to guest.
> 
> silently mask off the value provided from userspace is not a stable ABI from
> userspace perspective. A kvm cap to report what value is allowed/supported
> is still helpful.
> 
> > If we agree that it's the right time to have the opt-in, we could go the
> > opt-in way and KVM could set the information based on userspace's opt-ins
> > without exit to userspace for GetTdVmCallInfo.
> 
> Let's see what Paolo and Sean will say.

Kicking this to userspace seems premature.  AIUI, no "optional" VMCALL features
are defined at this time, i.e. there's nothing to enumerate.  And there's no
guarantee that there will ever be capabilties that require enumeration from 
*userspace*.  E.g. if fancy feature XYZ requires enumeration, but that feature
requires explicit KVM support, then forcing userspace will be messy.

So I don't see why KVM should anything other than return '0' to the guest (or
whatever value says "there's nothing here").

Re: [RFC PATCH 3/4] KVM: TDX: Exit to userspace for GetTdVmCallInfo

[Edgecombe, Rick P]

On Wed, 2025-06-11 at 07:54 -0700, Sean Christopherson wrote:
> > Let's see what Paolo and Sean will say.
> 
> Kicking this to userspace seems premature.  AIUI, no "optional" VMCALL
> features
> are defined at this time, i.e. there's nothing to enumerate.  And there's no
> guarantee that there will ever be capabilties that require enumeration from 
> *userspace*.  E.g. if fancy feature XYZ requires enumeration, but that feature
> requires explicit KVM support, then forcing userspace will be messy.
> 
> So I don't see why KVM should anything other than return '0' to the guest (or
> whatever value says "there's nothing here").

GetQuote is not part of the "Base" TDVMCALLs and so has a bit in
GetTdVmCallInfo. We could move it to base?

Paolo seemed keen on GetTdVmCallInfo exiting to userspace, but this was before
the spec overhaul.

Re: [RFC PATCH 4/4] KVM: TDX: Check KVM exit on KVM_HC_MAP_GPA_RANGE when TD finalize

[Binbin Wu]

On 6/11/2025 9:36 PM, Sean Christopherson wrote:
> On Wed, Jun 11, 2025, Binbin Wu wrote:
>> On 6/11/2025 3:58 AM, Sean Christopherson wrote:
>>> On Tue, Jun 10, 2025, Rick P Edgecombe wrote:
>>>> It seems like the reasoning could be just to shrink the possible configurations
>>>> KVM has to think about, and that we only have the option to do this now before
>>>> the ABI becomes harder to change.
>>>>
>>>> Did you need any QEMU changes as a result of this patch?
>>>>
>>>> Wait, actually I think the patch is wrong, because KVM_CAP_EXIT_HYPERCALL could
>>>> be called again after KVM_TDX_FINALIZE_VM. In which case userspace could get an
>>>> exit unexpectedly. So should we drop this patch?
>>> Yes, drop it.
>>>
>> So, when the TDX guest calls MapGPA and KVM finds userspace doesn't opt-in
>> KVM_HC_MAP_GPA_RANGE, just return error to userspace?
> Why can't KVM just do what it already does, and return an error to the guest?
>
> 	if (!user_exit_on_hypercall(vcpu->kvm, KVM_HC_MAP_GPA_RANGE)) {
> 		ret = TDVMCALL_STATUS_INVALID_OPERAND;
> 		goto error;
> 	}
>
My previous thought was MapGpa is in base GHIC API.
Userspace is required to opt-in KVM_HC_MAP_GPA_RANGE.
If not, it's userspace's responsibility, so I thought exit to userspace with
error may be better.

If return an error code is preferred, now it has a new status code
TDVMCALL_STATUS_SUBFUNC_UNSUPPORTED to use.

Basically, if the MapGpa is not support, either choice will stop VM from
execution. But had a second thought, returning an error code to guest allows
guest to choose to continue or not if MapGpa failed, though I can't
imagine what case it is.

Re: [RFC PATCH 4/4] KVM: TDX: Check KVM exit on KVM_HC_MAP_GPA_RANGE when TD finalize

[Binbin Wu]

On 6/11/2025 10:26 PM, Xiaoyao Li wrote:
> On 6/11/2025 10:04 PM, Edgecombe, Rick P wrote:
>> On Wed, 2025-06-11 at 22:01 +0800, Xiaoyao Li wrote:
>>>>> So, when the TDX guest calls MapGPA and KVM finds userspace doesn't opt-in
>>>>> KVM_HC_MAP_GPA_RANGE, just return error to userspace?
>>>>
>>>> Why can't KVM just do what it already does, and return an error to the
>>>> guest?
>>>
>>> Because GHCI requires it must be supported. No matter with the old GHCI
>>> that only allows <GetTdVmCallInfo> to succeed and the success of
>>> <GetTdVmCallInfo> means all the TDVMCALL leafs are support, or the
>>> proposed updated GHCI that defines <MapGpa> as one of the base API/leaf,
>>> and the base API must be supported by VMM.
>>>
>>> Binbin wants to honor it.
>>
>> But KVM doesn't need to support all ways that userspace could meet the GHCI
>> spec. If userspace opts-in to the exit, they will meet the spec. If they
>> configure KVM differently then they wont, but this is their decision.
>
> I agree with you and Sean. And I'm trying to answer Sean's question on behalf of Binbin.
Yes, it was my thought.

>
> Strictly speaking, KVM can be blamed for some reason. Because it is KVM that returns success for <GetTdVmCallInfo> unconditionally when r12 == 0  to report that all the (base) leafs are supported.
>
> But I totally agree with KVM cannot guarantee userspace will behave correctly. Even with this patch that KVM mandates the userspace to enable user exit of KVM_HC_MAP_GPA_RANGE, it's still possible for a misbehaved userspace to error to TD guest on KVM_HC_MAP_GPA_RANGE and breaks the semantics of successful <GetTdVmCallInfo>.
>
> So I'm with you and Sean.
>
Also see my reply
https://lore.kernel.org/kvm/ba611f52-9817-46ff-b16b-a9ef7404a51d@linux.intel.com/

In the next version, how about combining this patch to patch 1 to use
TDVMCALL_STATUS_SUBFUNC_UNSUPPORTED if userspace doesn't opt-in
KVM_HC_MAP_GPA_RANGE?

Re: [RFC PATCH 3/4] KVM: TDX: Exit to userspace for GetTdVmCallInfo

[Sean Christopherson]

On Wed, Jun 11, 2025, Rick P Edgecombe wrote:
> On Wed, 2025-06-11 at 07:54 -0700, Sean Christopherson wrote:
> > > Let's see what Paolo and Sean will say.
> > 
> > Kicking this to userspace seems premature.  AIUI, no "optional" VMCALL
> > features
> > are defined at this time, i.e. there's nothing to enumerate.  And there's no
> > guarantee that there will ever be capabilties that require enumeration from 
> > *userspace*.  E.g. if fancy feature XYZ requires enumeration, but that feature
> > requires explicit KVM support, then forcing userspace will be messy.
> > 
> > So I don't see why KVM should anything other than return '0' to the guest (or
> > whatever value says "there's nothing here").
> 
> GetQuote is not part of the "Base" TDVMCALLs and so has a bit in
> GetTdVmCallInfo. We could move it to base?

Is GetQuote actually optional?  TDX without attestation seems rather pointless.

> Paolo seemed keen on GetTdVmCallInfo exiting to userspace, but this was before
> the spec overhaul.

If GetQuote is truly optional, then exiting to userspace makes sense.  But as
above, that seems odd to me.

Re: [RFC PATCH 3/4] KVM: TDX: Exit to userspace for GetTdVmCallInfo

[Edgecombe, Rick P]

On Wed, 2025-06-11 at 09:26 -0700, Sean Christopherson wrote:
> > GetQuote is not part of the "Base" TDVMCALLs and so has a bit in
> > GetTdVmCallInfo. We could move it to base?
> 
> Is GetQuote actually optional?  TDX without attestation seems rather
> pointless.

I don't know if that was a consideration for why it got added to the optional
category. The inputs were gathered from more than just Linux.

> 
> > Paolo seemed keen on GetTdVmCallInfo exiting to userspace, but this was
> > before
> > the spec overhaul.
> 
> If GetQuote is truly optional, then exiting to userspace makes sense.  But as
> above, that seems odd to me.

Let us check on adding it to the "base" TDVMCALLs. If there is some good reason,
we can post v2 with GetTdVmCallInfo exiting. This will probably be Friday due to
the required people being out. Does it work for the timelines?

Under the change to add GetQuote to the base TDVMCALLs, patch 1 and 2 would be
the only changes anyway. It starts to feel a little fast and loose, but another
option is just to take patches 1 and 2, and we can finalize GetTdVmCallInfo at
whatever pace is best. 

Xiaoyao, Binbin do you see any pitfalls to that? If we add GetTdVmCallInfo as a
new exit, and we could add the supported TDVMCALLs to struct
kvm_tdx_capabilities at that point. If it doesn't make it for 6.16, it gets an
opt-in.

Re: [RFC PATCH 3/4] KVM: TDX: Exit to userspace for GetTdVmCallInfo

[Sean Christopherson]

On Wed, Jun 11, 2025, Rick P Edgecombe wrote:
> On Wed, 2025-06-11 at 09:26 -0700, Sean Christopherson wrote:
> > > GetQuote is not part of the "Base" TDVMCALLs and so has a bit in
> > > GetTdVmCallInfo. We could move it to base?
> > 
> > Is GetQuote actually optional?  TDX without attestation seems rather
> > pointless.
> 
> I don't know if that was a consideration for why it got added to the optional
> category. The inputs were gathered from more than just Linux.

If there's an actual use case for TDX without attestation, then by all means,
make it optional.  I'm genuinely curious if there's a hypervisor that plans on
productizing TDX without supporting attestation.  It's entirely possible (likely?)
I'm missing or forgetting something.

Re: [RFC PATCH 3/4] KVM: TDX: Exit to userspace for GetTdVmCallInfo

[Edgecombe, Rick P]

On Wed, 2025-06-11 at 11:13 -0700, Sean Christopherson wrote:
> > I don't know if that was a consideration for why it got added to the
> > optional
> > category. The inputs were gathered from more than just Linux.
> 
> If there's an actual use case for TDX without attestation, then by all means,
> make it optional.  I'm genuinely curious if there's a hypervisor that plans on
> productizing TDX without supporting attestation.  It's entirely possible
> (likely?)
> I'm missing or forgetting something.

Ok, will check back in with the story.

The only things I could think of are:
1. TDX usage as a hardening thing, similar to unmapping guest memory for all
page tables in the host.
2. Some highly coupled guest/VMM has an alternate attestation scheme.

More likely it was to retroactively bring the initial KVM PR into spec. We got
some pretty specific direction from Paolo to explore GetTdVmCallInfo exiting, so
it didn't make much of a difference one way or the other until now.

Re: [RFC PATCH 3/4] KVM: TDX: Exit to userspace for GetTdVmCallInfo

[Huang, Kai]

On Wed, 2025-06-11 at 11:13 -0700, Sean Christopherson wrote:
> On Wed, Jun 11, 2025, Rick P Edgecombe wrote:
> > On Wed, 2025-06-11 at 09:26 -0700, Sean Christopherson wrote:
> > > > GetQuote is not part of the "Base" TDVMCALLs and so has a bit in
> > > > GetTdVmCallInfo. We could move it to base?
> > > 
> > > Is GetQuote actually optional?  TDX without attestation seems rather
> > > pointless.
> > 
> > I don't know if that was a consideration for why it got added to the optional
> > category. The inputs were gathered from more than just Linux.
> 
> If there's an actual use case for TDX without attestation, then by all means,
> make it optional.  I'm genuinely curious if there's a hypervisor that plans on
> productizing TDX without supporting attestation.  It's entirely possible (likely?)
> I'm missing or forgetting something.

With no intention to disrupt this discussion, but even w/o GetQuote TDX can
also support attestation, because TD can just get the TDREPORT and send to
remote Quoting Enclave to get it signed, via whatever communication channel
available (vsock, TCP/IP etc). :-)

It's just not all TDX guests have those communication channels available in
CSP's deployment, and GetQuote can fill up the hole as a last resort.

Of course now TD userspace may choose to only support GetQuote simply
because kernel supports "unified ABI" to return remotely verifiable blob
across vendors, but still ...

Re: [RFC PATCH 3/4] KVM: TDX: Exit to userspace for GetTdVmCallInfo

[Edgecombe, Rick P]

On Thu, 2025-06-12 at 08:27 +0000, Huang, Kai wrote:
> With no intention to disrupt this discussion, but even w/o GetQuote TDX can
> also support attestation, because TD can just get the TDREPORT and send to
> remote Quoting Enclave to get it signed, via whatever communication channel
> available (vsock, TCP/IP etc). :-)
> 
> It's just not all TDX guests have those communication channels available in
> CSP's deployment, and GetQuote can fill up the hole as a last resort.
> 
> Of course now TD userspace may choose to only support GetQuote simply
> because kernel supports "unified ABI" to return remotely verifiable blob
> across vendors, but still ...

This was what I was getting at by "Some highly coupled guest/VMM has an
alternate attestation scheme". If you don't care about running on any VMM, you
could invent your own communication channel. But the same is true for the other
TDVMCALLs as well.

I guess the way to look at it is what is the MVP for a TD that expects to run on
general VMMs. The others could also invent their own enumeration API and return
failure for GetTdVmCallInfo.

Re: [RFC PATCH 3/4] KVM: TDX: Exit to userspace for GetTdVmCallInfo

[Edgecombe, Rick P]

On Wed, 2025-06-11 at 11:13 -0700, Sean Christopherson wrote:
> On Wed, Jun 11, 2025, Rick P Edgecombe wrote:
> > On Wed, 2025-06-11 at 09:26 -0700, Sean Christopherson wrote:
> > > > GetQuote is not part of the "Base" TDVMCALLs and so has a bit in
> > > > GetTdVmCallInfo. We could move it to base?
> > > 
> > > Is GetQuote actually optional?  TDX without attestation seems rather
> > > pointless.
> > 
> > I don't know if that was a consideration for why it got added to the optional
> > category. The inputs were gathered from more than just Linux.
> 
> If there's an actual use case for TDX without attestation, then by all means,
> make it optional.  I'm genuinely curious if there's a hypervisor that plans on
> productizing TDX without supporting attestation.  It's entirely possible (likely?)
> I'm missing or forgetting something.

It turns out there is a VMM that has a proprietary interface that they use
instead of GetQuote.