[PATCH v1 0/2] Optimize S2 page splitting

[Leonardo Bras <leo.bras@arm.com>]

While playing with dirty-bit tracking, I decided to take a look on how page
splitting works. Found out all entries are walked, even though we can infer,
for instance that:
- If a level-3 entry is walked, it means the parent level-2 entry is split
- If a split just succeeded in an table entry, it means all children nodes
  are already split

This patches' idea is to introduce new walking flags to skip pagetable 
levels 0-3. 

The idea of skipping child nodes was also tested, but it was marginally 
slower than just skipping levels, so it was discarted. 

Optimization measured on two scenarios involving eager-splitting on a
VM with 1 memslot of 64GB:
- Scenario 1: No manual protect, whole memslot split at dirty-track enable
  (KVM_SET_USER_MEMORY_REGION2 ioctl with KVM_MEM_LOG_DIRTY_PAGES)
  - Split happens only once, whole region
  - Evalutes improved batch performance of splitting
- Scenario 2: Manual protect, split happens during every dirty-bit clean
  (KVM_CLEAR_DIRTY_LOG ioctl), average for 2 iterations.
  - Split called multiple times, for smaller 64-page sections.
  - Evaluate improved performance for multiple calls

Scenario 1, improvement on dirty-track enable ioctl for the memslot:
- Memory was already split (4k pages):  -35.47% runtime (stdev 5.63%)
- THP backed memory:                    -11.94% runtime (stdev 2.55%)
- 64x1GB hugetlb memory:                -14.46% runtime (stdev 2.68%)

Scenario 2, improvement on dirty-log clean ioctl for the memslot:
- Memory was already split (4k pages):  -26.36% runtime (stdev 3.32%)
- THP backed memory:                    -12.05% runtime (stdev 0.37%)
- 64x1GB hugetlb memory:                -13.87% runtime (stdev 0.86%)

For collecting above numbers, the following script was ran in both vanilla 
and patched kernels, with kernel parameter 'default_hugepagesz=1G', on an 
AmpereOne with 256GB RAM.

--- dirty_test.sh
#!/bin/bash
filename=$(uname -r |cut -d'-' -f 4-)

run_test(){
  uname -a
  cat /proc/cmdline

  #prepare
  sudo bash -c 'echo 64 > /proc/sys/vm/nr_hugepages'

  ./dirty_log_perf_test -g -b 64G
  ./dirty_log_perf_test -g -b 64G -s anonymous_thp
  ./dirty_log_perf_test -g -b 64G -s shared_hugetlb

  ./dirty_log_perf_test -b 64G
  ./dirty_log_perf_test -b 64G -s anonymous_thp
  ./dirty_log_perf_test -b 64G -s shared_hugetlb
}

run_test 2>&1 | tee ${filename}
---

Above dirty_log_perf_test command is the standard kvm selftest found in the 
kernel tree. It tested the following guest modes:
Testing guest mode: PA-bits:48,  VA-bits:48,  4K pages
Testing guest mode: PA-bits:48,  VA-bits:48, 16K pages
Testing guest mode: PA-bits:48,  VA-bits:48, 64K pages
Testing guest mode: PA-bits:40,  VA-bits:48,  4K pages
Testing guest mode: PA-bits:40,  VA-bits:48, 16K pages
Testing guest mode: PA-bits:40,  VA-bits:48, 64K pages

Performance numbers from above modes were used to calculate average and 
stdev showed in the optimization results.

Changes since v1:
- Changed approach from return value to walk flags (Will Deacon)
- Discarted skip_child approach (Oliver Upton)
- Measured in real hardware, and from userspace perspective (Marc Zyngier)
- Better explanation of what and how numbers were collected
v1 Link: https://lore.kernel.org/all/20260515195904.2466381-1-leo.bras@arm.com/

Thanks!
Leo

Leonardo Bras (2):
  KVM: arm64: Introduce KVM_PGTABLE_WALK_SKIP_LEVEL* walk flags
  KVM: arm64: Make stage2_split_walker() skip unnecessary walks

 arch/arm64/include/asm/kvm_pgtable.h | 13 +++++++++++++
 arch/arm64/kvm/hyp/pgtable.c         | 18 ++++++++++++++++--
 2 files changed, 29 insertions(+), 2 deletions(-)


base-commit: acb7500801e98639f6d8c2d796ed9f64cba83d3a
-- 
2.54.0