(Proposal) New TDX Global Metadata To Report FIXED0 and FIXED1 CPUID Bits

[Xiaoyao Li <xiaoyao.li@intel.com>]

This is a proposal for a potential future TDX Module feature to assist 
QEMU/KVM in configuring CPUID leafs for TD guests. It is only in the 
idea stage and not currently being implemented. We are looking for 
comments on the suitability for QEMU/KVM.

# Background

To correctly virtualize CPUID for TD, the VMM needs to understand the 
behavior of CPUID configuration for each CPUID bit, including whether 
the bit can be configured by the VMM and what the allowed value is.

There is an interface to query the CPUID bit information after the TD 
has been configured. However, this interface does not work before the TD 
is configured. The TDX module, along with its release, provides a 
separate JSON format file, cpuid_virtualization.json, for CPUID 
virtualization information. This file can be used by the VMM even before 
the TD is configured. The TDX module also provides an interface to query 
some limited CPUID information, including:

  - The configurability of a subset of CPUIDs via global metadata 
CPUID_CONFIG_VALUES.

  - The 'fixed0' and 'fixed1' bits of ATTRIBUTES and XFAM via global 
metadata. The VMM can infer the 'configurable' bits related to 
ATTRIBUTES/XFAM indirectly (the bits that are neither 'fixed0' nor 
'fixed1' are 'configurable').

For the remaining CPUID bits not covered by the above two categories, no 
TDX module query interface exists.

# Problem

While the VMM can use the JSON format CPUID information and may embed or 
translate that information into the code, it may face several challenges:

  - The JSON file varies with each TDX module release, which can 
complicate the VMM code. Additionally, depending on its own needs, the 
VMM may require more information than what is provided in the JSON file.

  - The JSON format cannot be easily parsed with low-level programming 
languages like C, which is typically used to write VMMs.

There was objection from KVM community for parsing the JSON and requests 
for a more friendly interface to query CPUID information for each 
specific TDX module.[0][1]

# Analysis

There are many virtualization types defined for single bit or bitfields 
in JSON file, e.g., 12 types in TDX 1.5.06:

   - fixed
   - configured
   - configured & native
   - XFAM & native
   - XFAM & configured & native
   - attributes & native
   - attributes & configured & native
   - CPUID_enabled & native
   - attributes & CPUID_enabled & native
   - attributes & CPUID_enabled & configured & native
   - calculated
   - special

And more types are getting added as TDX evolves.

Though so many types defined, for a single bit, it can only be one of three:
   - fixed0
   - fixed1
   - configurable

For example:
1. For type "configured & native", the bit is “fixed0” bit if the native 
value is 0, and the “configurable” bit if native value is 1.

2. For type "XFAM & native",
    a) the CPUID is “fixed0” if the corresponding XFAM bit is reported 
in XFAM_FIXED0, or the native value is 0;

    b) the CPUID bit is ‘fixed1’ if the corresponding XFAM bit is set in 
XFAM_FIXED1;

    c) otherwise, the CPUID is ‘configurable’ (indirectly by TD_PRRAMS.XFAM)

# Proposal

Current TDX module provides interface to report the “configurable” bits 
via global metadata CPUID_CONFIG_VALUES directly or via global metadata 
ATTRIBUTES/XFAM_fixed0/1 indirectly. But it lacks the interface to 
report the “fixed0” and “fixed1” bits generally (it only reports the 
fixed bits for ATTRIBUTES and XFAM).

We propose to add two new global metadata fields, CPUID_FIXED0_BITS and 
CPUID_FIXED1_BITS, for “fixed0” and “fixed1” bits information respectively.

The encoding of the two fields uses the same format as TDCS field 
CPUID_VALUES:

   Field code is composed as follows:
     - Bits 31:17  Reserved, must be 0
     - Bit  16     Leaf number bit 31
     - Bits 15:9   Leaf number bit 6:0
     - Bit 8       Sub-leaf not applicable flag
     - Bits 7:1    Sub-leaf number bits 6:0
     - Bit 0       Element index within field

   The same for returned result:
     - Element 0[31:0]:   EAX
     - Element 0[63:32]:  EBX
     - Element 1[31:0]:   ECX
     - Element 1[63:32]:  EDX

For CPUID_FIXED0_BITS, any bit in E[A,B,C,D]X is 0, means the bit is fixed0.
For CPUID_FIXED1_BITS, any bit in E[A,B,C,D]X is 1, means the bit is fixed1.

# Interaction with TDX_FEATURES0.VE_REDUCTION

TDX introduces a new feature VE_REDUCTION[2]. From the perspective of 
host VMM, VE_REDUCTION turns several CPUID bits from fixed1 to 
configurable, e.g., MTRR, MCA, MCE, etc. However, from the perspective 
of TD guest, it’s an opt-in feature. The actual value seen by TD guest 
depends on multiple factors: 1). If TD guest enables REDUCE_VE in 
TDCS.TD_CTLS, 2) TDCS.FEATURE_PARAVIRT_CTRL, 3) CPUID value configured 
by host VMM via TD_PARAMS.CPUID_CONFIG[]. (Please refer to latest TDX 
1.5 spec for more details.)

Since host VMM has no idea on the setting of 1) and 2) when creating the 
TD. We make the design to treat them as configurable bits and the global 
metadata interface doesn’t report them as fixed1 bits for simplicity.

Host VMM must be aware itself that the value of these VE_REDUCTION 
related CPUID bits might not be what it configures. The actual value 
seen by TD guest also depends on the guest enabling and configuration of 
VE_REDUCTION.

# POC

We did a POC in QEMU to verify the fixed0/1 data by such an interface is 
enough for userspace to validate and generate a supported vcpu model for 
TD guest.[3]

It retrieves the “fixed” type in JSON file and hardcodes them into two 
arrays, tdx_fixed0_bits and tdx_fixed1_bits. Note, it doesn’t handle the 
other types than “fixed” because 1) just a few of them falls into fixed0 
or fixed1 and 2) turning them into fixed0 or fixed0 needs to check 
various condition which complicates the POC. And in the POC it uses 
value 1 in tdx_fixed0_bits for fixed0 bits, while the proposed metadata 
interface uses value 0 to indicate fixed0 bits.

With the hardcoded information, VMM can validate the TD configuration 
requested from user early by checking whether a feature requested from 
users is allowed to be enabled and is allowed to be disabled.

When TDX module provides fixed0 and fixed1 via global metadata, QEMU can 
change to requested them from KVM to replace the hardcoded one.

[0] https://lore.kernel.org/all/ZhVdh4afvTPq5ssx@google.com/
[1] https://lore.kernel.org/all/ZhVsHVqaff7AKagu@google.com/
[2] https://cdrdv2.intel.com/v1/dl/getContent/733575
[3] 
https://lore.kernel.org/qemu-devel/20241105062408.3533704-49-xiaoyao.li@intel.com/