[PATCH v3 00/20] KVM: x86: Enable APX for guests

Chang S. Bae
2026-04-28T05:26:15Z

Hi all,

This revision tries to reflect the recent design discussions [1]. Notably
the series also depends on preparatory work [1,2], so should be on hold
until they are merged. But the intention here is to sort out next level
of details by collecting feedbacks.

Below is a summary of the recently established direction:

Since the V2 posting [4], Sean gave his feedbacks that led to shift the
approach. The initial approach was access physical EGPRs directly
similar to vector registers, but this introduces non-uniform access
patterns against directly accessing VCPU regs[].

To maintain uniformity, EGPRs need to be stored in regs[]. Sean's reg
accessor infra updates [1] makes this feasible. It was also realized
that saving EGPRs outside of the fastpath is not viable, as fastpath
handlers may access EGPRs as well.

Then, saving EGPRs on entry code appears to be the choice. This also
looks to provide some degree of robustness (but not completely yet
though [*]) for when the kernel clobbers those registers. The VCPU XCR0
can gate this path to avoid #UD from non-APX guests.

The conditional path should be also under speculation-safety, rather
than wild guest control. Paolo's rework on entry code [2] will
establish a SPEC_CTRL macro which allows a finer control.

For userspace interactions, the EGPR state in regs[] can be copied
directly to/from the userspace buffer at the boundary of ABI handlers.

With this design concept in mind, here are a couple of points to call out
while each patch has a note of changes:

* Entry changes

The entry code is first refactored to generalize (64-bit) register
save/restore via macros (part1-3). This simplifies integration of
EGPR handling (patch5). I also sorted to simplify the register saving
path by skipping VM-Fail check.

* XSAVE ABI

Similar to PKRU, the state is not managed by XSAVE internally but is
exposed via the XSAVE format. KVM fully owns both ends: storage and
ABI handling so KVM-side changes alone cover the support. There are
some subtlties (noted in patch6) in arrangement with the existing
copy function.

Finally, series layout and relevant test comments:

* Part1, PATCH 01-03: Prepare entry code by macrofying GPR handling
Most KVM test will capture any regression out of it. I also tested it
more explicitly with tweaking the posted patch [3].

* Part2, PATCH 04-06: Establish EGPR state management
These new state management flow is rather unique compared to other
XSTATEs. This fact leads to new test cases (patch18), in addition to
state_test (patch19).

* Part3, PATCH 07-11: Update VMX handlers for extended reg. indices
Previously I tweaked to test an exit case like LGDT with an extended
ID. But this part leans into VMX unless KVM is paranoid.

* Part4, PATCH 12-15: Add emulator support for REX2
KVM unit test patch [3] covers emulator changes.

* Part5, PATCH 16-19: Expose features and add selftests
XCR0 tests basically covers the exposition.

This series is currently based on an old commit (02896e0065ca) in x86 KVM
next branch and includes preparatory patches [1,2]. It can be also found
here:
git://github.com/intel/apx.git apx-kvm_v3

Thanks,
Chang

References:
[1] Sean's preparatory series:
https://lore.kernel.org/kvm/20260409224236.2021562-1-seanjc@google.com/
[2] Paolo's SPEC_CTRL refactoring:
https://lore.kernel.org/kvm/20260427105848.44865-1-pbonzini@redhat.com/
[3] KVM unit tests:
https://lore.kernel.org/kvm/20260420212355.507827-1-chang.seok.bae@intel.com/
[4] Previous version (V2):
https://lore.kernel.org/kvm/20260112235408.168200-1-chang.seok.bae@intel.com/
[5] APX specification:
https://cdrdv2.intel.com/v1/dl/getContent/784266
[*] E.g. NMIs at entry code could mess up with APX-clobbering handlers
when XCR0[APX]=0. VMX extention to afford XCR0 switching by hardware
itself could be an option to avoid the issue.

Chang S. Bae (20):
KVM: VMX: Macrofy 64-bit GPR swapping in __vmx_vcpu_run()
KVM: SVM: Macrofy 64-bit GPR swapping in __svm_vcpu_run()
KVM: SEV: Macrofy 64-bit GPR swapping in __svm_sev_es_vcpu_run()
KVM: x86: Extend VCPU registers for EGPRs
KVM: VMX: Save guest EGPRs in VCPU cache
KVM: x86: Support APX state for XSAVE ABI
KVM: VMX: Refactor VMX instruction information access
KVM: VMX: Refactor instruction information decoding
KVM: VMX: Refactor register index retrieval from exit qualification
KVM: VMX: Support instruction information extension
KVM: nVMX: Propagate the extended instruction info field
KVM: x86: Support EGPR accessing and tracking for emulator
KVM: x86: Handle EGPR index and REX2-incompatible opcodes
KVM: x86: Support REX2-prefixed opcode decode
KVM: x86: Reject EVEX-prefixed instructions
KVM: x86: Guard valid XCR0.APX settings
KVM: x86: Expose APX foundation feature to guests
KVM: x86: Expose APX sub-features to guests
KVM: x86: selftests: Add APX state and ABI test
KVM: x86: selftests: Add APX state handling and XCR0 sanity checks

arch/x86/Kconfig.assembler | 5 +
arch/x86/include/asm/kvm_host.h | 35 +++-
arch/x86/include/asm/kvm_vcpu_regs.h | 11 -
arch/x86/include/asm/vmx.h | 2 +
arch/x86/kvm/Kconfig | 4 +
arch/x86/kvm/cpuid.c | 28 ++-
arch/x86/kvm/cpuid.h | 2 +
arch/x86/kvm/emulate.c | 121 +++++++----
arch/x86/kvm/kvm_emulate.h | 13 +-
arch/x86/kvm/reverse_cpuid.h | 6 +
arch/x86/kvm/svm/svm.c | 8 +-
arch/x86/kvm/svm/vmenter.S | 51 +----
arch/x86/kvm/vmenter.h | 51 +++++
arch/x86/kvm/vmx/nested.c | 74 +++----
arch/x86/kvm/vmx/nested.h | 2 +-
arch/x86/kvm/vmx/vmcs12.c | 1 +
arch/x86/kvm/vmx/vmcs12.h | 3 +-
arch/x86/kvm/vmx/vmenter.S | 67 +++---
arch/x86/kvm/vmx/vmx.c | 26 ++-
arch/x86/kvm/vmx/vmx.h | 77 ++++++-
arch/x86/kvm/x86.c | 76 ++++++-
tools/testing/selftests/kvm/Makefile.kvm | 1 +
.../selftests/kvm/include/x86/processor.h | 120 +++++++++++
tools/testing/selftests/kvm/x86/apx_test.c | 192 ++++++++++++++++++
tools/testing/selftests/kvm/x86/state_test.c | 3 +
.../selftests/kvm/x86/xcr0_cpuid_test.c | 19 ++
26 files changed, 803 insertions(+), 195 deletions(-)
create mode 100644 tools/testing/selftests/kvm/x86/apx_test.c

base-commit: 109b8abcee6413717b09ba6b0bd4b3bc5aaa4608
--
2.51.0