Re: [PATCH RFC v1 00/20] KVM: x86: Support APX feature for guests

Paolo Bonzini
2025-11-11T18:14:47Z

On 11/10/25 19:01, Chang S. Bae wrote:
> Hi all,
>
> This series is intended to initiate the enablement of APX support in KVM.
> The goal is to start discussions and, ideally, reach consensus on key
> decisions for the enabling path.
>
> == Introduction ==
>
> Intel introduces Advanced Performance Extensions (APX), which add 16
> additional 64-bit general-purpose registers (EGPRs: R16–R31).
>
> APX also introduces new instruction prefixes that extend register indices
> to 5 bits, providing additional encoding space. The feature specification
> [1] describes these extensions in detail.
>
> The specification deliberately scopes out some areas. For example,
> Sections 3.1.4.4.2–7 note that initialization and reset behaviors follow
> existing XSTATE conventions.
>
> That said, there are still some essential elements to consider at first.
>
> == Ingredients to Consider ==
>
> With guest exposure, the new registers will affect how KVM handles VM
> exits and manages guest state. So the extended register indices need to
> be properly interpreted.
>
> There are three relevant contexts where KVM must handle the new 5-bit
> register index:
>
> * Instruction Information in VMCS
>
> VMX provides instruction information through a dedicated field in the
> VMCS. However, the legacy format supports only 4-bit register indices,
> so a new 64-bit field was introduced to support 5-bit indices for
> EGPRs (see Table 3.11 in the spec).
>
> * Exit Qualification in VMCS
>
> This field also carries register index information. Fortunately, the
> current layout is not fully packed, so it appears feasible to extend
> index fields to 5 bits without structural conflict.
>
> * Instruction Emulator
>
> When exits are handled through instruction emulation, the emulator
> must decode REX2-prefixed instructions, which carry the additional
> bits encoding the extended indices and other new modifiers.
>
> Once handlers can identify EGPR-related indices, the next question is how
> to access and maintain their state.
>
> * Extended GPR State Management
>
> Current KVM behavior for legacy GPRs can be summarized as follows:
> (a) All legacy GPRs are saved on every VM exit and written back on
> re-entry. Most access operations go through KVM register cache.
> (b) The instruction emulator also maintains a temporary GPR cache
> during emulation, backed by the same KVM-managed accessor
> interface.
>
> For the new EGPRs, there are a few important differences:
> (a) They are general-purpose registers, but their state is
> XSAVE-managed, which makes them distinct from legacy GPRs.
> (b) The kernel itself currently does not use them, so there is no
> requirement to save EGPRs on every VM exit -- they can remain in
> hardware registers.
>
> The mechanism to read and write EGPR state will therefore be commonly
> needed by both VMX handlers and the instruction emulator.
>
> == Approaches to Support ==
>
> Given the above aspects, the first step is to build out a generalized GPR
> accessor framework. This constitutes the first part (PART1: PATCH1–3) of
> the series, laying the foundational infrastructure.
>
> * New EGPR Accessors (PATCH3)
>
> Since EGPRs are not yet clobbered by the host, they can be accessed
> directly through hardware, using the existing helpers kvm_fpu_get()
> and kvm_fpu_put().
>
> This model follows the same approach used for legacy FP state
> handling.
>
> The design choice is based on the following considerations:
> (a) Caching EGPR state on every VM exit would be unnecessary cost.
> (b) If EGPRs are updated during VM exit handling or instruction
> emulation, synchronizing them with the guest fpstate would require
> new interfaces to interact with x86 core logic, adding complexity.
>
> * Common GPR Access Interface --Unifying Legacy and Extended Accessors
> (PATCH2)
>
> Although legacy GPRs and EGPRs differ in how their state is accessed,
> that distinction does not justify maintaining separate interfaces. A
> unified accessor abstraction allows both legacy and extended registers
> to be accessed uniformly, simplifying usage for both exit handlers and
> the emulator.
>
> Returning to the remaining key ingredients -- VMX handlers and the
> instruction emulator -- the necessary updates break down into the
> following.
>
> * Extended Instruction Information Extraction (PART2: PATCH4–8)
>
> Currently, instruction-related VMCS fields are stored as 32-bit raw
> data and interpreted on site. Adding separate variable in this manner
> would increase code complexity.
>
> Thus, refactoring this logic into a proper data structure with clear
> semantics appears to be a sensible direction.
>
> * Instruction Emulator (PART3: PATCH9–16)
>
> As noted in Paolo’s earlier feedback [2], support for REX2-prefixed
> instruction emulation is required.
>
> While REX2 largely mirrors legacy opcode behavior, a few exceptions
> and new instructions introduce additional decoding and handling
> requirements.
>
> Conceptually, the changes are straightforward, though details are better
> handled directly in the patches.
>
> Finally, actual feature exposure and XCR0 management form the last stage
> of enabling (PART4: PATCH17-20), relatively small but with a few key
> constraints:
>
> * XCR0 updates and CPUID feature exposure must occur together (PATCH18).
> * The current enabling scope applies only to VMX (PATCH17-18).
>
> == Patchset ==
>
> As mentioned earlier, while the number of patches is relatively large,
> the series is organized into four logical parts for clarity and
> reviewability:
>
> * PART1, PATCH 01–03: GPR accessor refactoring (foundational)
> * PART2, PATCH 04–08: VMX support for EGPR index handling
> * PART3, PATCH 09–16: Instruction emulator support for REX2
> * PART4, PATCH 17–20: APX exposure and minor selftest updates
>
> Each patch includes an RFC note to provide context for reviewers.
>
> This series is based on the next branch of the KVM x86 tree [3] and is
> available at:
>
> git://github.com/intel/apx.git apx-kvm_rfc-v1
>
> == Testing ==
>
> Testing so far has focused on unit and synthetic coverage of relevant
> code paths using KVM selftests, both on legacy and APX systems.
>
> For decoder-related changes, additional testing was performed by invoking
> x86_decode_insn() from a pseudo driver with some test inputs to exercise
> REX2 and legacy decoding logic.
>
> == References ==
>
> [1] https://cdrdv2.intel.com/v1/dl/getContent/784266
> [2] https://lore.kernel.org/kvm/CABgObfaHp9bH783Kdwm_tMBHZk5zWCxD7R+RroB_Q_o5NWBVZg@mail.gmail.com/
> [3] https://github.com/kvm-x86/linux
>
>
> I would also give thanks to Chao and Zhao for helping me out in this
> KVM series.

There are a few bugs, but most of my remarks are really just stylistic.
The next version, if you base it on the emulator AVX support, should be
pretty close.

Paolo