[PATCH v2 0/8] KVM: arm64: VNCR TLB invalidation fixes

[Marc Zyngier <maz@kernel.org>]

Prompted by a patch [0] from Hyunwoo Kim which was addressing a pretty
annoying corner case, I spent some time looking at what was wrong in
our VNCR TLBI code. And there was a few things to say about it...

- The TLB tracking to get into the slow path is wrong. It tracks what
  is mapped, but not the established TLBs. Kind of annoying.

- Handling IPA invalidation when the TLB was established with the S1
  MMU disabled is failing for a number of reasons. This is the bug
  that Hyunwoo Kim found, but I decided to address it at its root
  rather than just fixing the symptoms.

- Deciding to run with a VNCR TLB doesn't take the state of
  SCTLR_EL2.M into account. Yes, it is a special sort of sport to run
  a hypervisor with its MMU disabled...

- TLB invalidation by VA targeting the last page/block of TTBR1_EL2 is
  dropped on the floor, because I cannot count. This was amusing. Not.

Additionally, Sashiko kindly provided some extra horrors to look at:

- A missing sign extension for range invalidation results in the wrong
  VA range being considered.

- TLBI doesn't participate in the general MMU invalidation retry
  machinery, meaning that VNCR faults and TLBIs race in an
  uncontrolled way.

- TLBI and vcpu_put() can race badly, leading to a TOCTOU pattern
  which results in either a BUG_ON() or a call to vncr_fixmap(-1).

I have fixes for each of these issues, all stable candidates.

On top of that, I have a patch reintroducing the tracking that the
first patch removes, this time in a way that is actually functional.
Or at least I think it is...

I'd like to thank Hyunwoo Kim for their initial patch and for
providing a reproducer that helped me finding these issues by running
it at multiple levels of nesting.

It is now too late for 7.2, so let's try to make this 7.3 material
(hence the early repost).

* From v1 [1]

  - Added sign extension to range invalidation (Sashiko)

  - Use mmu_invalidate_seq to resolve fault vs TLBI races (Sashiko)

  - Speculatively bump the TLB refcount on fault to make sure the TLBI
    takes the slow path when we are going for a S1 walk

  - Clarify the check for the TLB matching for the S1 MMU state

  - Extra sanitisation added on the S2 invalidation path

  - Applied RBs from Yao Yuan to the patches that didn't change, with
    thanks.
    
[0] https://lore.kernel.org/r/ameGoxbn2wzBq2kL@v4bel
[1] https://lore.kernel.org/r/20260801124818.366274-1-maz@kernel.org

Marc Zyngier (8):
  KVM: arm64: Remove VM-wide VNCR mapping counter
  KVM: arm64: Handle negative S1 walk levels in VNCR TLB size evaluation
  KVM: arm64: Consider SCTLR_EL2.M when mapping the L1 VNCR page
  KVM: arm64: Correctly handle end of VA space TLBI invalidation
  KVM: arm64: Handle VNCR TLB invalidation race with vcpu_put() VNCR
    unmapping
  KVM: arm64: Sign-extend VA for range-based TLBI invalidation
  KVM: arm64: Make VNCR invalidation participate in MMU invalidation
    retry
  KVM: arm64: Add VNCR TLB tracking again

 arch/arm64/include/asm/kvm_host.h   |   4 +-
 arch/arm64/include/asm/kvm_nested.h |  14 +++
 arch/arm64/kvm/at.c                 |   2 -
 arch/arm64/kvm/hyp/vhe/switch.c     |  10 +-
 arch/arm64/kvm/nested.c             | 173 +++++++++++++++++++---------
 arch/arm64/kvm/sys_regs.c           |  11 ++
 6 files changed, 155 insertions(+), 59 deletions(-)

-- 
2.47.3