[PATCH v6 00/64] KVM: arm64: ARMv8.3/8.4 Nested Virtualization support

Marc Zyngier
2022-01-28T12:19:42Z

Here the first drop of the KVM/arm64 NV support code for 2022. Nothing
to worry about, it certainly isn't going to be the last!

A number of changes since [1]:

- The exposure of the EL2 sysregs to userspace is now gated by NV
being enabled, as you'd expect. Which means we shouldn't break live
migration anymore (yay!).

- We now correctly detect and handle an Illegal Exception Return from
vEL2. Don't try this at home, kids!

- Non-nested exception injection when executing at EL0 has been fixed.

- We forbid NV+SVE for now. This is a change in ABI, and I hope to
remove this requirement at some point. I've pushed out a kvmtool
change that enforces this [2], and QEMU will need similar surgery
for now.

- nested_virt_in_use() is now renamed to vcpu_has_nv() (resp
vcpu_has_nv2() for the v8.4 support).

- A bunch of small nits being tidied up, thanks to our eagle eyed
reviewers.

Many thanks to Alexandru, Chase, Ganapatrao and Russell for spending a
lot of time reviewing this and actively spotting issues. There is a
pending issue that Ganapatrao mentioned when running L0 with 4kB and
L1 with 64kB, resulting in no forward progress. I haven't investigated
this yet, but I strongly suspect that something is amiss in the S2 PTW
at the point where we combine the L2 IPA with the L1 IPA.

As usual, blame me for any bug, and nobody else. It has been tested on
my usual zoo, and nothing caught fire. Which means nothing, of course.
Obviously, it isn't feature complete, and it is quite easy to write a
guest that will not behave as intended. The current goal is to make
sure that non-NV KVM is unaffected by the NV stuff.

It is massively painful to run on the FVP, but if you have a Neoverse
V1 or N2 system (or anything else with ARMv8.4-NV) that is collecting
dust, I have the right stuff to keep it busy.

M.

[1] https://lore.kernel.org/r/20211129200150.351436-1-maz@kernel.org
[2] https://git.kernel.org/pub/scm/linux/kernel/git/maz/kvmtool.git/log/?h=arm64/nv-5.16

Andre Przywara (1):
KVM: arm64: nv: vgic: Allow userland to set VGIC maintenance IRQ

Christoffer Dall (15):
KVM: arm64: nv: Introduce nested virtualization VCPU feature
KVM: arm64: nv: Reset VCPU to EL2 registers if VCPU nested virt is set
KVM: arm64: nv: Allow userspace to set PSR_MODE_EL2x
KVM: arm64: nv: Add nested virt VCPU primitives for vEL2 VCPU state
KVM: arm64: nv: Reset VMPIDR_EL2 and VPIDR_EL2 to sane values
KVM: arm64: nv: Handle trapped ERET from virtual EL2
KVM: arm64: nv: Emulate PSTATE.M for a guest hypervisor
KVM: arm64: nv: Trap EL1 VM register accesses in virtual EL2
KVM: arm64: nv: Only toggle cache for virtual EL2 when SCTLR_EL2
changes
KVM: arm64: nv: Implement nested Stage-2 page table walk logic
KVM: arm64: nv: Unmap/flush shadow stage 2 page tables
KVM: arm64: nv: arch_timer: Support hyp timer emulation
KVM: arm64: nv: vgic: Emulate the HW bit in software
KVM: arm64: nv: Add nested GICv3 tracepoints
KVM: arm64: nv: Sync nested timer state with ARMv8.4

Jintack Lim (18):
arm64: Add ARM64_HAS_NESTED_VIRT cpufeature
KVM: arm64: nv: Handle HCR_EL2.NV system register traps
KVM: arm64: nv: Support virtual EL2 exceptions
KVM: arm64: nv: Inject HVC exceptions to the virtual EL2
KVM: arm64: nv: Trap SPSR_EL1, ELR_EL1 and VBAR_EL1 from virtual EL2
KVM: arm64: nv: Trap CPACR_EL1 access in virtual EL2
KVM: arm64: nv: Handle PSCI call via smc from the guest
KVM: arm64: nv: Respect virtual HCR_EL2.TWX setting
KVM: arm64: nv: Respect virtual CPTR_EL2.{TFP,FPEN} settings
KVM: arm64: nv: Respect the virtual HCR_EL2.NV bit setting
KVM: arm64: nv: Respect virtual HCR_EL2.TVM and TRVM settings
KVM: arm64: nv: Respect the virtual HCR_EL2.NV1 bit setting
KVM: arm64: nv: Emulate EL12 register accesses from the virtual EL2
KVM: arm64: nv: Configure HCR_EL2 for nested virtualization
KVM: arm64: nv: Set a handler for the system instruction traps
KVM: arm64: nv: Trap and emulate AT instructions from virtual EL2
KVM: arm64: nv: Trap and emulate TLBI instructions from virtual EL2
KVM: arm64: nv: Nested GICv3 Support

Marc Zyngier (30):
KVM: arm64: nv: Add EL2 system registers to vcpu context
KVM: arm64: nv: Add non-VHE-EL2->EL1 translation helpers
KVM: arm64: nv: Handle virtual EL2 registers in
vcpu_read/write_sys_reg()
KVM: arm64: nv: Handle SPSR_EL2 specially
KVM: arm64: nv: Handle HCR_EL2.E2H specially
KVM: arm64: nv: Save/Restore vEL2 sysregs
KVM: arm64: nv: Allow a sysreg to be hidden from userspace only
KVM: arm64: nv: Forward debug traps to the nested guest
KVM: arm64: nv: Filter out unsupported features from ID regs
KVM: arm64: nv: Hide RAS from nested guests
KVM: arm64: nv: Support multiple nested Stage-2 mmu structures
KVM: arm64: nv: Handle shadow stage 2 page faults
KVM: arm64: nv: Restrict S2 RD/WR permissions to match the guest's
KVM: arm64: nv: Fold guest's HCR_EL2 configuration into the host's
KVM: arm64: nv: Add handling of EL2-specific timer registers
KVM: arm64: nv: Load timer before the GIC
KVM: arm64: nv: Don't load the GICv4 context on entering a nested
guest
KVM: arm64: nv: Implement maintenance interrupt forwarding
KVM: arm64: nv: Allow userspace to request KVM_ARM_VCPU_NESTED_VIRT
KVM: arm64: nv: Add handling of ARMv8.4-TTL TLB invalidation
KVM: arm64: nv: Invalidate TLBs based on shadow S2 TTL-like
information
KVM: arm64: nv: Tag shadow S2 entries with nested level
KVM: arm64: nv: Add include containing the VNCR_EL2 offsets
KVM: arm64: nv: Map VNCR-capable registers to a separate page
KVM: arm64: nv: Move nested vgic state into the sysreg file
KVM: arm64: Add ARMv8.4 Enhanced Nested Virt cpufeature
KVM: arm64: nv: Allocate VNCR page when required
KVM: arm64: nv: Enable ARMv8.4-NV support
KVM: arm64: nv: Fast-track 'InHost' exception returns
KVM: arm64: nv: Fast-track EL1 TLBIs for VHE guests

.../admin-guide/kernel-parameters.txt | 7 +-
.../virt/kvm/devices/arm-vgic-v3.rst | 12 +-
arch/arm64/include/asm/esr.h | 5 +
arch/arm64/include/asm/kvm_arm.h | 27 +-
arch/arm64/include/asm/kvm_asm.h | 4 +
arch/arm64/include/asm/kvm_emulate.h | 143 +-
arch/arm64/include/asm/kvm_host.h | 185 ++-
arch/arm64/include/asm/kvm_hyp.h | 2 +
arch/arm64/include/asm/kvm_mmu.h | 18 +-
arch/arm64/include/asm/kvm_nested.h | 156 ++
arch/arm64/include/asm/sysreg.h | 101 +-
arch/arm64/include/asm/vncr_mapping.h | 74 +
arch/arm64/include/uapi/asm/kvm.h | 2 +
arch/arm64/kernel/cpufeature.c | 34 +
arch/arm64/kvm/Makefile | 4 +-
arch/arm64/kvm/arch_timer.c | 202 ++-
arch/arm64/kvm/arm.c | 42 +-
arch/arm64/kvm/at.c | 219 +++
arch/arm64/kvm/emulate-nested.c | 211 +++
arch/arm64/kvm/guest.c | 6 +
arch/arm64/kvm/handle_exit.c | 81 +-
arch/arm64/kvm/hyp/exception.c | 49 +-
arch/arm64/kvm/hyp/include/hyp/switch.h | 12 +-
arch/arm64/kvm/hyp/include/hyp/sysreg-sr.h | 24 +-
arch/arm64/kvm/hyp/nvhe/switch.c | 2 +-
arch/arm64/kvm/hyp/nvhe/sysreg-sr.c | 2 +-
arch/arm64/kvm/hyp/vgic-v3-sr.c | 2 +-
arch/arm64/kvm/hyp/vhe/switch.c | 181 ++-
arch/arm64/kvm/hyp/vhe/sysreg-sr.c | 125 +-
arch/arm64/kvm/hyp/vhe/tlb.c | 83 ++
arch/arm64/kvm/inject_fault.c | 68 +-
arch/arm64/kvm/mmu.c | 206 ++-
arch/arm64/kvm/nested.c | 915 ++++++++++++
arch/arm64/kvm/reset.c | 30 +-
arch/arm64/kvm/sys_regs.c | 1252 ++++++++++++++++-
arch/arm64/kvm/sys_regs.h | 14 +-
arch/arm64/kvm/trace_arm.h | 65 +-
arch/arm64/kvm/vgic/vgic-init.c | 30 +
arch/arm64/kvm/vgic/vgic-kvm-device.c | 22 +
arch/arm64/kvm/vgic/vgic-nested-trace.h | 137 ++
arch/arm64/kvm/vgic/vgic-v3-nested.c | 242 ++++
arch/arm64/kvm/vgic/vgic-v3.c | 39 +-
arch/arm64/kvm/vgic/vgic.c | 44 +
arch/arm64/kvm/vgic/vgic.h | 10 +
arch/arm64/tools/cpucaps | 2 +
include/kvm/arm_arch_timer.h | 9 +-
include/kvm/arm_vgic.h | 16 +
include/uapi/linux/kvm.h | 1 +
tools/arch/arm/include/uapi/asm/kvm.h | 1 +
49 files changed, 4947 insertions(+), 171 deletions(-)
create mode 100644 arch/arm64/include/asm/kvm_nested.h
create mode 100644 arch/arm64/include/asm/vncr_mapping.h
create mode 100644 arch/arm64/kvm/at.c
create mode 100644 arch/arm64/kvm/emulate-nested.c
create mode 100644 arch/arm64/kvm/nested.c
create mode 100644 arch/arm64/kvm/vgic/vgic-nested-trace.h
create mode 100644 arch/arm64/kvm/vgic/vgic-v3-nested.c