Re: [RFC PATCH 0/4] KVM: x86/mmu: don't zap secondary MMU on permission upgrade

["Carlos López" <clopez@suse.de>]

On 10/29/26 5:52 PM, Claudio Imbrenda wrote:
> On Thu, 29 Oct 2026 15:33:33 +0100
> Carlos López <clopez@suse.de> wrote:
> 
>> Prevent x86 KVM from over-zapping its secondary MMU on PTE permission
>> upgrades, which leave the PFNs unchanged and do not revoke access.
>> Permissions for SPTEs that KVM holds are updated on the next fault, so
>> dropping them early gains nothing.
>>
>> Do this by adding a new MMU notifier flag to signal that a
>> MMU_NOTIFY_PROTECTION_VMA strictly upgrades (loosens) permissions on a
>> particular range.
>>
>> See below for benchmark results.
>>
>> Background
>> ==========
>>
>> change_protection() pairs every protection change with an
>> MMU_NOTIFY_PROTECTION_VMA invalidation, and KVM zaps its SPTEs in
>> response, as it cannot tell whether permissions widened or narrowed.
>> This is costly e.g. during a QEMU background snapshot, which relies on
>> uffd-wp to write-protect guest RAM and then restore permissions as each
>> page is saved. Without this series, the permission restore causes the
>> guest to re-fault read-only memory needlessly.
>>
>> Note, the reverse situation is already handled for secondary MMUs
>> that reuse the CPU page tables (e.g. ARM SMMU, IOMMU/ATS). They cannot
>> refault, so they are notified of a permission upgrade on the CPU
>> TLB-flush path [1].
>>
>> KVM, however, owns separate page tables and faults permissions on demand,
>> so it needs no such notification (as Sean noted when that signalling
>> was first proposed on the fault path [2]). This series applies the same
>> principle to the change_protection() path, signalling the permission
>> upgrade so KVM can avoid the unnecessary zap.
>>
>> RFC Considerations
>> ==================
>>
>> 1. Having the zap skip in KVM's arch-independent code does not seem
>>    obviously correct, as s390 caches additional translation state
>>    that may not be rebuilt by refaults. Maybe an expert can correct
>>    me. The skip could be extended to arm64 and RISC-V, but I have not
>>    benchmarked the benefits and I'd like to get feedback first. Hence,
>>    x86-only for now.
>>
>> 2. change_protection() could simply not emit the invalidation at all
>>    for a pure permission upgrade (shared-page-table MMUs invalidate
>>    on the CPU TLB-flush path, as noted above, so they do not need it).
>>    But again, the s390 case could be broken, and even then, that would
>>    seem too much intrusion for KVM's benefit.
>>
>> 3. The flag reaches KVM as a bool on struct kvm_gfn_range.
>>    Alternatively, KVM could forward the notifier event, as Claudio
>>    Imbrenda suggested at LPC [3] and others have proposed in the past
>>    for different purposes [4] [5]. That, along with the event flags,
>>    could serve the same purpose. I'd be happy to take that approach if
>>    there is agreement that other consumers would want it.
>>
>>    (The LPC talk recording has some audio issues so I am not too sure
>>    on what the final agreement was)
> 
> AFAIR the consensus was:
> - adding the reason to the kvm kvm_gfn_range is fine.
> - having a return value from the notifier to pass information back is
>   _not_ fine; I should add a new callback instead.
> 
> regarding this series:
> 
> on x86 you don't care what the new permissions are, because you're
> updating when re-faulting (if I understood correctly).

Correct.

> for s390 it might be beneficial to know what the new permissions are;
> of course we could look up the vma and find out, but since you are
> already checking the permissions, maybe you can find a way to also
> convey the new permission bits?
> 
> that would make our life on s390 easier :)

Right, getting the new bits from vma->flags should be trivially done
from the MMU_NOTIFY_PROTECTION_VMA callsites. There'd be no users in
this series though. The rest of architectures still need the notifier to
tell them if the change is an upgrade, since they don't have an easy way
to grab the old RWX bits. Passing the old RWX bits from the MMU notifier
does not seem trivial to me either.

So what I can do is to plumb down the event and flags into KVM, and
leave it open for you to come back and forward the permission bits you
need via the range flags, if the MM maintainers are okay with it.

However, I tried to understand the s390 code better with AI, and it
seems to think that skipping the zap is also safe there (and an
improvement). It could be further optimized by passing down the new W
bit to eagerly fix the gmap when restoring write permissions.

>>
>> Benchmarks
>> ==========
>>
>> AMD EPYC 9654 96-Core Processor host running a recent Linux kernel
>> (6b4b543fa187), with a 64GB, 64-vCPU guest running stress-ng --vm while
>> the host takes a QEMU background snapshot. All results are aggregated
>> over 5 runs, and compared between the stock kernel and one with this
>> patch series. Guest is rebooted across each individual run, and NUMA
>> balancing is disabled.
>>
>> pf/s and tlb/s (per-second deltas of KVM's pf_taken and remote_tlb_flush
>> debugfs stats) clearly improve:
>>
>>   guest pattern          baseline                    patched
>>   -----------------------------------------------------------------------
>>   read64   (read)        pf/s=950,015   tlb/s=1,440  pf/s=2,503 tlb/s=0.1
>>   rand-set (read+write)  pf/s=1,004,353 tlb/s=862    pf/s=4,073 tlb/s=0.3
>>   write64  (write)       pf/s=968,018   tlb/s=94     pf/s=2,292 tlb/s=0.3
>>
>> Guest read latency was sampled with a paced probe thread in the
>> guest, reading one byte per page with an idle gap between reads, while
>> stress-ng ran with read64. Each figure is the median across 5 runs, with
>> the run-to-run min and max, in ns:
>>
>>   metric           baseline             patched
>>   ------------------------------------------------------
>>   p50 (min max)      340     (270 380)   260   (250 260)
>>   p99 (min max)    19789 (16885 20200)  5207 (4076 5638)
>>   p99.9 (min max)  23445 (22845 24036)  6850 (6259 7110)
>>
>> The median improves modestly and the refault cost shows in the tail,
>> where p99 drops ~3.8x.
>>
>> A separate stress-ng run with write64 and --verify showed no data
>> corruption and no livelocks.
>>
>> References
>> ==========
>>
>> [1] https://lore.kernel.org/linux-mm/cover.1eca029b8603ef4eebe5b41eae51facfc5920c41.1690292440.git-series.apopple@nvidia.com/
>> [2] https://lore.kernel.org/linux-mm/ZGu1vsbscdO48V6h@google.com/
>> [3] https://lpc.events/event/20/contributions/2552/
>> [4] https://lore.kernel.org/all/20240801183453.57199-1-seanjc@google.com
>> [5] https://lore.kernel.org/all/20231110003734.1014084-1-jackyli@google.com
>>
>> Carlos López (4):
>>   mm/mprotect: add MM_CP_PERM_UPGRADE
>>   mm/mmu_notifier: add MMU_NOTIFIER_RANGE_PERM_UPGRADE
>>   KVM: plumb MMU_NOTIFIER_RANGE_PERM_UPGRADE into kvm_gfn_range
>>   KVM: x86/mmu: don't zap the secondary MMU on permission upgrade
>>
>>  arch/x86/kvm/mmu/mmu.c       |  7 +++++++
>>  include/linux/kvm_host.h     |  1 +
>>  include/linux/mm.h           |  8 ++++++++
>>  include/linux/mmu_notifier.h |  5 +++++
>>  mm/hugetlb.c                 |  6 +++++-
>>  mm/mprotect.c                | 20 +++++++++++++++++++-
>>  virt/kvm/kvm_main.c          |  4 ++++
>>  7 files changed, 49 insertions(+), 2 deletions(-)
>>
>