Kernel KVM virtualization development
 help / color / mirror / Atom feed
* [PATCH 0/2] KVM: unbounded per-VM kernel memory growth via KVM_SET_MEMORY_ATTRIBUTES
@ 2026-09-11 18:48 David Ballesteros
  2026-09-11 18:48 ` [PATCH 1/2] KVM: Bound per-VM GFN materialization in KVM_SET_MEMORY_ATTRIBUTES David Ballesteros
                   ` (5 more replies)
  0 siblings, 6 replies; 16+ messages in thread
From: David Ballesteros @ 2026-09-11 18:48 UTC (permalink / raw)
  To: pbonzini; +Cc: kvm, security, David Ballesteros


Affected version/commit: v7.2-rc5 (commit 699594e8888d is not needed here;
verified on v7.2-rc5 as of 2026-09-09 and on v6.18.48; the code paths are
unchanged between them and present in mainline. The ioctl was introduced
in v6.8 with the per-page memory attributes series).

Problem
-------

kvm_vm_ioctl_set_mem_attributes() (virt/kvm/kvm_main.c) validates the
user-provided range only for representability (zero size, 64-bit wrap,
alignment, supported attributes) and not for magnitude.  With attributes
!= 0, kvm_vm_set_mem_attributes() then walks every GFN in the range:

        for (i = start; i < end; i++) {
                r = xa_reserve(&kvm->mem_attr_array, i, GFP_KERNEL_ACCOUNT);
                if (r)
                        goto out_unlock;
                cond_resched();
        }

xa_reserve() materializes the full root-to-leaf path of xa_nodes for each
GFN even though the GFNs need no backing memory and no memslots need to
exist.  Each xa_node is 576 bytes from radix_tree_node_cachep; densely
filled, the tree costs ~9.3 bytes per GFN.  On ENOMEM the loop bails out
without undoing prior reservations, and the only release path is
kvm_destroy_vm(); the memory is therefore retained across calls and lives
for as long as the VM fd.

Additionally, the intended memcg accounting does not take effect: the
nodes are allocated with GFP_NOWAIT in xas_alloc(), gaining __GFP_ACCOUNT
only when the xarray carries XA_FLAGS_ACCOUNT, which kvm does not set
(kvm_create_vm() uses plain xa_init()).  Measured: a process inside a
cgroup limited to 256 MiB grew host SReclaimable by ~450 MiB while its
memory.current stayed at ~488 KiB.

Triggering conditions
---------------------

- An unprivileged user with write access to /dev/kvm (sysfs default and
  the common udev default are 0666).
- A VM whose type sets private memory support: sw protected VM
  (CONFIG_KVM_SW_PROTECTED_VM, depends on EXPERT), TDX or SNP.
- No memslots, no guest memory and no guest CPUs are required.
- Verified outcomes on v6.18.48 (KASAN builds, isolated VMs):
  * ~9.29 bytes/GFN and ~75 MiB/s of sustained growth per VM; linear
    scaling with parallel VMs (4 VMs: 3.56x).
  * global OOM killing unrelated third-party processes while the
    attacker's own memcg stays uncharged.
  * when all surviving processes are OOM-disabled, select_bad_process()
    finds no candidate and the kernel panics ("System is deadlocked on
    memory", mm/oom_kill.c).  Reproduced twice.
  * one ioctl holds kvm->slots_lock for the whole reservation loop, so a
    large range blocks KVM_GET_DIRTY_LOG and memslot updates of that VM
    for the duration (measured GET_DIRTY_LOG latency 1us -> 7.7ms with a
    single concurrent call); KVM_RUN is unaffected.
  * replicated through a real container runtime (podman/crun) with
    --memory 512m: the container grew host slab by ~450 MiB while its
    memory.current stayed at ~488 KiB.

Reproducer status
-----------------

A reproducer is available on request (KVM_CREATE_VM + the ioctl in a
loop, plus slab/ accounting measurement), but following security-bugs.rst it is
not attached to this report.

Test environment
----------------

All dynamic results were collected in isolated inspection VMs on a
single AMD host with nested virtualization enabled
(kvm_amd nested=1): QEMU -cpu host, Debian 13 guests, guest kernels
built from the audited tree (v6.18.48) and from v7.2-rc5 with
CONFIG_KVM_SW_PROTECTED_VM=y / CONFIG_KVM_AMD=y / CONFIG_EXPERT=y and
KASAN (with CONFIG_KASAN_VMALLOC=y where noted); the PoC runs as root
inside the guest, host networking restricted (user-mode net,
restrict=on).  Nothing in the host kernels themselves was modified or
attacked.  The container replication used podman 5.4.2/crun inside the
guest with the host kernel having the gate enabled (a stock host kernel
without KVM_SW_PROTECTED_VM rejects KVM_CREATE_VM and the surface is
absent, as expected).

Proposed fix
------------

Attached as 0001-v3.patch (checkpatch clean, build warning-free on
v7.2-rc5, runtime-verified on v7.2-rc5): it bounds the total number of
materialized attribute entries per VM to KVM_MEM_ATTR_MAX_GFNS (2^25
GFNs = 128 GiB of GPA, ~300 MiB of xa_nodes), enforced under
kvm->slots_lock after the idempotency early-out.  A per-VM counter is
resynchronized from the xarray only when an ioctl mutates the array
(ENOMEM partials, successful sets and clears), so idempotent re-sets
and rejected requests stay O(1) — runtime-verified: a 4 KiB idempotent
re-set on an exhausted VM returns in ~4us; a fresh range on an
exhausted VM is rejected with -ENOSPC in ~17us; the bound cuts growth
at 2^25 GFNs.

Patch 2/2 restores the intended accounting with a one-liner; it
attributes the memory but does not bound it (see the analysis below):

-       xa_init(&kvm->mem_attr_array);
+       xa_init_flags(&kvm->mem_attr_array, XA_FLAGS_ACCOUNT);

Design notes for reviewers: the budget value (2^25 GFNs) is a trade-off
— it is well above any current consumer's usage but small next to the
GPA space TDX/SNP guests can have; a tunable or a symmetric decrement on
clear are alternatives if maintainers prefer.  The -ENOSPC errno
distinguishes the bound from the malformed-input -EINVAL paths.

Why a constant and not a derived bound?

We considered bounding materialization by what the VM has actually
declared instead of a constant, and rejected it because the declaration
itself is not proportional.  Two candidate signals:

- Memslot coverage ("reject ranges not covered by any memslot"):
  semantically attractive — attributes on unbacked GFNs are inert state
  with no consumer (kvm_handle_gfn_range only walks real memslots), and
  the only in-tree consumer (tools/testing/selftests/kvm, private_mem_
  conversions_test.c) sets attributes strictly inside its gmem memslot.
  But it is not a bound: a memslot requires only a MAP_NORESERVE userspace
  VMA (no committed pages); its kernel cost is the lpage_info metadata
  (4 bytes/entry, __vcalloc proportional to npages: ~16 MiB for a
  maximum 8 TiB memslot) while the same memslot's GPA span materializes
  ~19.9 GiB of xa_nodes — a ~1200:1 amplification of declared metadata.
  Three or four maximum-size NORESERVE memslots suffice to exhaust a
  64 GiB host.  As a semantic follow-up that eliminates the zero-setup
  variant of the attack it is worth considering on top of the bound.

- Sum of memslot pages as a dynamic budget: same flaw — the attacker
  raises his own budget by declaring free memslots; without a constant
  the ceiling becomes the physical address space.

Per-VM accounting to the caller's memcg (patch 2/2) attributes the
memory but bounds only cgroup-limited tenants; with the common
/dev/kvm 0666 default and unlimited cgroups it does not bound the host.
Hence the layering we propose: accounting (2/2) + hard per-VM bound
(1/2), with memslot-coverage rejection as an optional semantic layer
the maintainers may prefer on top.

Mitigations
-----------

- Restrict /dev/kvm (mode 0660 root:kvm) where the default 0666 is in
  place.
- Monitor SReclaimable and /sys/kernel/slab/radix_tree_node/objects, not
  SUnreclaim: the growth is reclaimable-accounted and never reclaimed.
- Per-tenant memcg limits do NOT mitigate the common path today (see
  above), which is why the counter bound is the effective fix.

Notes
-----

- Not verified: the TDX/SNP hardware paths end-to-end (no hardware);
  the software-protected path was verified end-to-end on both versions.

--
2.47.0



^ permalink raw reply	[flat|nested] 16+ messages in thread

end of thread, other threads:[~2026-09-12  8:48 UTC | newest]

Thread overview: 16+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-09-11 18:48 [PATCH 0/2] KVM: unbounded per-VM kernel memory growth via KVM_SET_MEMORY_ATTRIBUTES David Ballesteros
2026-09-11 18:48 ` [PATCH 1/2] KVM: Bound per-VM GFN materialization in KVM_SET_MEMORY_ATTRIBUTES David Ballesteros
2026-09-11 19:05   ` sashiko-bot
2026-09-11 18:48 ` [PATCH 2/2] KVM: Account mem_attr_array nodes to the caller's memcg David Ballesteros
2026-09-11 19:02   ` sashiko-bot
2026-09-11 20:32 ` [PATCH v2 0/2] KVM: unbounded per-VM kernel memory growth via KVM_SET_MEMORY_ATTRIBUTES David Ballesteros
2026-09-11 21:19   ` David Ballesteros
2026-09-11 22:13   ` [PATCH v3 " David Ballesteros
2026-09-11 22:13     ` [PATCH v3 1/2] KVM: Bound per-VM GFN materialization in KVM_SET_MEMORY_ATTRIBUTES David Ballesteros
2026-09-11 22:32       ` sashiko-bot
2026-09-11 22:13     ` [PATCH v2 2/2] KVM: Account mem_attr_array nodes to the caller's memcg David Ballesteros
2026-09-11 20:32 ` [PATCH v2 1/2] KVM: Bound per-VM GFN materialization in KVM_SET_MEMORY_ATTRIBUTES David Ballesteros
2026-09-11 20:45   ` sashiko-bot
2026-09-11 20:32 ` [PATCH v2 2/2] KVM: Account mem_attr_array nodes to the caller's memcg David Ballesteros
2026-09-11 20:44   ` sashiko-bot
2026-09-12  8:48 ` [syzbot ci] Re: KVM: unbounded per-VM kernel memory growth via KVM_SET_MEMORY_ATTRIBUTES syzbot ci

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox