[RFC PATCH v2 0/1] KVM: x86/mmu: Zap empty TDP leaf page tables

[Hao Zhang <hao_zhang_kdev@163.com>]

From: Hao Zhang <zhanghao1@kylinos.cn>

Hi,

v1 tried to avoid repeated walks of retained empty TDP leaf page
tables by adding a per-shadow-page hint.  The main v1 feedback was to
avoid adding more metadata to struct kvm_mmu_page and to look for a more
generic iterator-based approach.

Previous RFC:
https://lore.kernel.org/all/an7J9Gh2DTrtP5vT@192.168.1.215/

This RFC v2 is a metadata-free alternative.  It adds an opt-in TDP
iterator event that tells the caller when the walk has completed a
child page table.  tdp_mmu_zap_leafs() uses that event to unlink an
empty 4K leaf page table after a fully covered 2MiB range has been
walked.

The tradeoff is intentional:

  * v1/hint keeps the page-table page linked and remembers that it is
    empty across later invalidations.
  * v2/unlink avoids persistent per-MMU-page metadata by making the
    parent SPTE non-present, so later invalidations naturally stop
    descending into that empty 4K leaf page table.

That also means v2 can pay allocation/free/refault cost if the same
leaf page table is repeatedly emptied and repopulated.  To limit that
cost, v2 does not unlink a leaf page table if a leaf SPTE in that page
table was zapped in the current pass.

When a shadow page is unlinked, tdp_mmu_zap_leafs() also services
pending TLB invalidations with kvm_flush_remote_tlbs() before dropping
RCU protection.  The incoming @flush state can reflect earlier zaps in
the walk, not only the just-completed child range, so this RFC uses the
existing full flush primitive instead of trying to narrow the flush to
the child range.

The patch also avoids changing TDX/mirror external page-table lifetime:
mirror shadow pages are not unlinked by this optimization.  Please
confirm whether that is the preferred boundary, or whether unlinking
mirror leaf page tables is safe after the TDX maintainers audit the
external SPT lifetime.

The main RFC question is whether this metadata-free unlink tradeoff is
acceptable, or whether KVM should keep retaining empty 4K TDP leaf page
tables unless there is an explicit per-page-table hint.

Test setup
==========

All numbers below are from the same x86 host and the same 4-vCPU, 1GiB
guest workload.  Guest vCPUs continuously fault/write guest memory while
the host repeatedly invalidates guest memory with MADV_DONTNEED.  The
reported values are medians from five repetitions.

"sp-only" is the shadow-present-only iterator prototype that skips
non-present shadow-present SPTEs without unlinking empty page tables.

Measured paths:

  * tdp_mmu_zap_leafs() duration via paired perf probes.
  * vCPU fault-side mmu_lock wait time via queued_read_lock_slowpath()
    perf probes.

Scenarios:

  * same16/same4: repeatedly invalidate the same 16MiB or 4MiB range.
  * c16/c4/c2/c1: rolling 16MiB, 4MiB, 2MiB, and 1MiB chunks across
    the 1GiB guest.

zap_total_ms_med
================

  scenario   original   sp-only   v1 hint   v2 unlink
  --------   --------   -------   -------   ---------
  same16       330.3     221.8      35.3       61.9
  same4        341.3     234.2      54.4       52.4
  c16         1456.2    1696.0    1386.1     1549.9
  c4          1252.0    1290.6    1243.8     1575.7
  c2          1197.0    1173.9    1246.2     1501.3
  c1          1352.8    1198.9    1138.4     1304.2

Relative to the original kernel, v2 reduces repeated same-range zap
time by 81% for same16 and 85% for same4.  Rolling workloads are mixed:
c16 is +6%, c4 is +26%, c2 is +25%, and c1 is -4%.
Compared to the shadow-present-only iterator change, v2 is mixed on
rolling workloads: better on c16, worse on c4/c2/c1.  Same-range zap
time remains 3.6x faster for same16 and 4.5x faster for same4.

fwait_p999_us_med
=================

  scenario   original   sp-only   v1 hint   v2 unlink
  --------   --------   -------   -------   ---------
  same16          78       162        61          74
  same4           62       152        38          57
  c16             72       242        77         114
  c4             102       248       105         134
  c2             108       270       107         148
  c1              92       266        93          95

An additional no-unlink control on top of v2's iterator changes brought
same16/same4 back to the original kernel's zap time, while also bringing
rolling fwait back close to original/v1 levels.  That confirms the
same-range win and the rolling cost both come from unlinking, not from
the iterator event itself.

Changes since v1
================

  * Replace the per-kvm_mmu_page empty-child hint with a metadata-free
    unlink-based approach.
  * Add an opt-in TDP iterator event for callers that want notification
    after completing a child page table.
  * Do not unlink leaf page tables that had a leaf SPTE zapped in the
    current pass, to reduce refault churn in rolling invalidations.
  * Skip mirror shadow pages, so this optimization does not alter TDX
    external SPT lifetime.
  * Drop the unused skip-child iterator API from the v1/hint direction.
  * Move the event-valid fast gate to the caller.

Hao Zhang (1):
  KVM: x86/mmu: Zap empty TDP leaf page tables

 arch/x86/kvm/mmu/tdp_iter.c | 29 +++++++++++++--
 arch/x86/kvm/mmu/tdp_iter.h | 10 ++++++
 arch/x86/kvm/mmu/tdp_mmu.c  | 88 +++++++++++++++++++++++++++++++++++++--------
 3 files changed, 109 insertions(+), 18 deletions(-)

base-commit: 45c13f3f9e3bb15fd89ff2864c6f627a3b4b4229
-- 
2.15.0