From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7E931C53219 for ; Tue, 28 Jul 2026 12:11:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To:From: Subject:Message-ID:References:Mime-Version:In-Reply-To:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=V9fXM6lRs795AVXt0xgZm237lDVZT/43F4+i2ayf4wM=; b=SlEeXEGwtI1HwOZXe2MsJh3qJ5 lvZxahHXfRSubuaRZ+3jq/2Tl/6R6SqNLVzWG3rn2zjO/8xRIl6ddwgU7A1HNn+KL1Clu9+9XSiDo ELEdYhpidwvEo4tQWK9A7CJmYJvjai8RGnMBpcI3y93xLTRGWVzNa2ZJEzwlX8nWODUs61IB4GRo/ NCxwVTUKhm0/k/1xh75KlswqI/m+/9R5GLDQR1yozMjfDeLSnXL25elEZnLLGjfPZ4ZtjqJdPnS8v cB4RAmePtiwDCyHi9yknIq41+XHuJIrk7nJMNbnYEW7RlLzjraRc7kfMUq4/sIToKjP3q/RBSDAC5 Drcw84bw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.99.1 #2 (Red Hat Linux)) id 1wogf4-00000005Cm8-1spc; Tue, 28 Jul 2026 12:11:54 +0000 Received: from mail-ej1-x648.google.com ([2a00:1450:4864:20::648]) by bombadil.infradead.org with esmtps (Exim 4.99.1 #2 (Red Hat Linux)) id 1wogf1-00000005ChE-3xkv for kexec@lists.infradead.org; Tue, 28 Jul 2026 12:11:53 +0000 Received: by mail-ej1-x648.google.com with SMTP id a640c23a62f3a-c1670dd0f5dso347658066b.2 for ; Tue, 28 Jul 2026 05:11:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1785240710; x=1785845510; darn=lists.infradead.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=V9fXM6lRs795AVXt0xgZm237lDVZT/43F4+i2ayf4wM=; b=HJEVwVPf8LyQ5VbdHxnRSbK4Z/RBdw3kDPDkxf4q5iBXUJU6BQXpI2Aa9YEnKsiiIB DJhFQe9P/s6w4LHwX2XR/YcbX1t3v5sSAqZjD/pRA3P1f9N0n1e/MFCep6dj3Vxd4n52 x+5z9Lw2/3DXt2ni6Ed4TIjvn5pOBimJDsEnqIqNcEmlr4KZzciCHISk4B9+sdlga9rj Z2hua5c+Vb6rD45v6IPfRpTM1q3Wrr97XfBaRtOuW6Q1Iy2TbvumlnlpbFPHHp36Mbg5 vD7sKpOHtCFqyb8qnVYhWFBvm5a/chaaKE/8uupCgftGR+PpX6dYlHrp50FZWj2CKdqL SAJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785240710; x=1785845510; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=V9fXM6lRs795AVXt0xgZm237lDVZT/43F4+i2ayf4wM=; b=W222TdKWHEvL5bJzork1TrGUwl5exYEPr3S3w01FmP6bitlO4ta/xuXpIk3HOPoAKa KwmS67UM88E4s9ZqEe/l+dBc8DepzgAwgWa0z2U8HqCS1bZ6slMhtzqg05cLkvGoYvb0 uDJwL+XGh3lk0Goslo3Rmn2ZCIHP0u3JRWbmjmjMdjvA0h3zYASBSO6kgKwKTPNr1j2q EQHQnJQUZflxItyS8Siitf/gUhUPvK4TvDeLvZBrjluyb6WtemYVcll9UdQDtfaU8HZ8 UdvHh6g7Scyq8UmGWp6xJtRXFMM15qFDUkEHnk6Wr78VUEsMCur/7BkGXAdk514QFe7J 2GtQ== X-Forwarded-Encrypted: i=1; AHgh+RrrJkUAjAT+5IkIO3Ib1e+Fl9iV8VxIEAIm54XS+L7zxWq9PfQ3woIiUmj6aAEhXVjehi4p9A==@lists.infradead.org X-Gm-Message-State: AOJu0YyaAGQzwQ2GiDg3iVt12BIwcPFRUbMgAjhhS39G5oB6thW32qWb UrROK6YqBK83ws9qmODw2IAB9MLFtryyTs8rhoh5LTQNLu00VrGHvim8FtR1xMncblHZZJmOogf NIOGb9jWyWRNDzLMsTQ== X-Received: from ejdr26.prod.google.com ([2002:a17:906:38da:b0:c12:9b2c:ffb1]) (user=tarunsahu job=prod-delivery.src-stubby-dispatcher) by 2002:a17:907:c1e:b0:c19:45e3:2f57 with SMTP id a640c23a62f3a-c1f71d2cc6bmr102851666b.9.1785240709110; Tue, 28 Jul 2026 05:11:49 -0700 (PDT) Date: Tue, 28 Jul 2026 12:11:34 +0000 In-Reply-To: <20260728121138.1103610-1-tarunsahu@google.com> Mime-Version: 1.0 References: <20260728121138.1103610-1-tarunsahu@google.com> X-Mailer: git-send-email 2.55.0.229.g6434b31f56-goog Message-ID: <20260728121138.1103610-8-tarunsahu@google.com> Subject: [PATCH v4 07/11] KVM: guest_memfd: Add support for freezing mappings From: Tarun Sahu To: ackerleytng@google.com, fuad.tabba@linux.dev, Andrew Morton , seanjc@google.com, dmatlack@google.com, Shuah Khan , Jonathan Corbet , david@redhat.com, Tarun Sahu , Pasha Tatashin , Pratyush Yadav , sagis@google.com, Paolo Bonzini , Mike Rapoport , Alexander Graf Cc: linux-kselftest@vger.kernel.org, andre.przywara@arm.com, michael.roth@amd.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, will@kernel.org, vannapurve@google.com, maz@kernel.org, fvdl@google.com, kvm@vger.kernel.org, oliver.upton@linux.dev, kvmarm@lists.linux.dev, alexandru.elisei@arm.com, skhawaja@google.com, aneesh.kumar@kernel.org, linux-doc@vger.kernel.org, David Hildenbrand , yan.y.zhao@intel.com, kexec@lists.infradead.org, suzuki.poulose@arm.com Content-Type: text/plain; charset="UTF-8" X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.9.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20260728_051152_034276_1525E4F6 X-CRM114-Status: GOOD ( 23.84 ) X-BeenThere: kexec@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "kexec" Errors-To: kexec-bounces+kexec=archiver.kernel.org@lists.infradead.org Introduce kvm_gmem_freeze() to freeze a guest_memfd inode's mapping, which prevents fallocate() operations and new page fault allocations during preservation. Use a global SRCU (`kvm_gmem_freeze_srcu`) to synchronize freeze state checkers without incurring per-fault locking overhead or risking per-CPU memory exhaustion (as per-CPU structure not counted in cgroup) from per-inode SRCU structures by faulty/compromised VMM. Signed-off-by: Tarun Sahu --- virt/kvm/guest_memfd.c | 117 +++++++++++++++++++++++++++++++++++++---- virt/kvm/guest_memfd.h | 5 ++ 2 files changed, 111 insertions(+), 11 deletions(-) diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c index dd84bba8974b..d522cc34f476 100644 --- a/virt/kvm/guest_memfd.c +++ b/virt/kvm/guest_memfd.c @@ -7,11 +7,13 @@ #include #include #include +#include #include "guest_memfd.h" #include "kvm_mm.h" static struct vfsmount *kvm_gmem_mnt; +static struct srcu_struct kvm_gmem_freeze_srcu; #define kvm_gmem_for_each_file(f, inode) \ @@ -96,6 +98,7 @@ static struct folio *kvm_gmem_get_folio(struct inode *inode, pgoff_t index) /* TODO: Support huge pages. */ struct mempolicy *policy; struct folio *folio; + int idx; /* * Fast-path: See if folio is already present in mapping to avoid @@ -105,12 +108,20 @@ static struct folio *kvm_gmem_get_folio(struct inode *inode, pgoff_t index) if (!IS_ERR(folio)) return folio; + idx = srcu_read_lock(&kvm_gmem_freeze_srcu); + if (kvm_gmem_is_frozen(inode)) { + srcu_read_unlock(&kvm_gmem_freeze_srcu, idx); + return ERR_PTR(-EPERM); + } + policy = mpol_shared_policy_lookup(&GMEM_I(inode)->policy, index); folio = __filemap_get_folio_mpol(inode->i_mapping, index, FGP_LOCK | FGP_CREAT, mapping_gfp_mask(inode->i_mapping), policy); mpol_cond_put(policy); + srcu_read_unlock(&kvm_gmem_freeze_srcu, idx); + /* * External interfaces like kvm_gmem_get_pfn() support dealing * with hugepages to a degree, but internally, guest_memfd currently @@ -273,16 +284,30 @@ static long kvm_gmem_allocate(struct inode *inode, loff_t offset, loff_t len) static long kvm_gmem_fallocate(struct file *file, int mode, loff_t offset, loff_t len) { + struct inode *inode = file_inode(file); int ret; + int idx; - if (!(mode & FALLOC_FL_KEEP_SIZE)) - return -EOPNOTSUPP; + idx = srcu_read_lock(&kvm_gmem_freeze_srcu); + if (kvm_gmem_is_frozen(inode)) { + srcu_read_unlock(&kvm_gmem_freeze_srcu, idx); + return -EPERM; + } - if (mode & ~(FALLOC_FL_KEEP_SIZE | FALLOC_FL_PUNCH_HOLE)) - return -EOPNOTSUPP; + if (!(mode & FALLOC_FL_KEEP_SIZE)) { + ret = -EOPNOTSUPP; + goto out; + } - if (!PAGE_ALIGNED(offset) || !PAGE_ALIGNED(len)) - return -EINVAL; + if (mode & ~(FALLOC_FL_KEEP_SIZE | FALLOC_FL_PUNCH_HOLE)) { + ret = -EOPNOTSUPP; + goto out; + } + + if (!PAGE_ALIGNED(offset) || !PAGE_ALIGNED(len)) { + ret = -EINVAL; + goto out; + } if (mode & FALLOC_FL_PUNCH_HOLE) ret = kvm_gmem_punch_hole(file_inode(file), offset, len); @@ -291,6 +316,9 @@ static long kvm_gmem_fallocate(struct file *file, int mode, loff_t offset, if (!ret) file_modified(file); + +out: + srcu_read_unlock(&kvm_gmem_freeze_srcu, idx); return ret; } @@ -948,7 +976,9 @@ static void kvm_gmem_destroy_inode(struct inode *inode) static void kvm_gmem_free_inode(struct inode *inode) { - kmem_cache_free(kvm_gmem_inode_cachep, GMEM_I(inode)); + struct gmem_inode *gi = GMEM_I(inode); + + kmem_cache_free(kvm_gmem_inode_cachep, gi); } static const struct super_operations kvm_gmem_super_operations = { @@ -1003,12 +1033,21 @@ int kvm_gmem_init(struct module *module) if (!kvm_gmem_inode_cachep) return -ENOMEM; + ret = init_srcu_struct(&kvm_gmem_freeze_srcu); + if (ret) + goto err_cache; + ret = kvm_gmem_init_mount(); - if (ret) { - kmem_cache_destroy(kvm_gmem_inode_cachep); - return ret; - } + if (ret) + goto err_srcu; + return 0; + +err_srcu: + cleanup_srcu_struct(&kvm_gmem_freeze_srcu); +err_cache: + kmem_cache_destroy(kvm_gmem_inode_cachep); + return ret; } void kvm_gmem_exit(void) @@ -1016,5 +1055,61 @@ void kvm_gmem_exit(void) kern_unmount(kvm_gmem_mnt); kvm_gmem_mnt = NULL; rcu_barrier(); + cleanup_srcu_struct(&kvm_gmem_freeze_srcu); kmem_cache_destroy(kvm_gmem_inode_cachep); } + +/** + * kvm_gmem_freeze - Freeze or unfreeze a guest_memfd inode mapping. + * @inode: The guest_memfd inode. + * @freeze: True to freeze, false to unfreeze. + * + * This API is used strictly during the live update / preservation transition + * window to prevent host userspace and guest-side faults from making any + * mapping modifications (such as fallocate or page fault allocation) + * to the guest_memfd page cache. + * + * Synchronization Strategy (Sleepable RCU): + * To avoid high-contention VFS locks (like inode_lock or + * filemap_invalidate_lock) on the vCPU page fault hot paths, this subsystem + * implements a lightweight, system-wide Sleepable RCU (SRCU) mechanism + * (`kvm_gmem_freeze_srcu`): + * + * Global vs. Per-Inode SRCU + * ====================== + * A single system-wide global static `srcu_struct` is used instead of a + * per-inode SRCU structure to completely prevent unprivileged users from + * exhausting the host's per-CPU memory allocator. Because + * `init_srcu_struct()` allocates per-CPU memory via `alloc_percpu()`, which + * is not accounted by memory cgroups (memcg), + * a per-inode SRCU structure would allow a tenant to bypass cgroup limits and + * trigger a system-wide Out-of-Memory (OOM) crash simply by spawning a large + * number of guest_memfd file descriptors (bounded only by RLIMIT_NOFILE). + * + * Flag Modification Note: + * Since `GUEST_MEMFD_F_MAPPING_FROZEN` is the ONLY flag in + * `GMEM_I(inode)->flags` that is mutated dynamically at runtime (all other + * flags are creation-time flags which remain strictly read-only), there is + * no possibility of concurrent bit-modification races. Therefore, a standard + * `WRITE_ONCE` is fully safe and does not require complex `cmpxchg` + * synchronization loops. + */ +void kvm_gmem_freeze(struct inode *inode, bool freeze) +{ + u64 flags = READ_ONCE(GMEM_I(inode)->flags); + + if (freeze) + flags |= GUEST_MEMFD_F_MAPPING_FROZEN; + else + flags &= ~GUEST_MEMFD_F_MAPPING_FROZEN; + + WRITE_ONCE(GMEM_I(inode)->flags, flags); + + if (freeze) + synchronize_srcu(&kvm_gmem_freeze_srcu); +} + +bool kvm_gmem_is_frozen(struct inode *inode) +{ + return READ_ONCE(GMEM_I(inode)->flags) & GUEST_MEMFD_F_MAPPING_FROZEN; +} diff --git a/virt/kvm/guest_memfd.h b/virt/kvm/guest_memfd.h index c528b046dd69..028c348a1023 100644 --- a/virt/kvm/guest_memfd.h +++ b/virt/kvm/guest_memfd.h @@ -29,11 +29,16 @@ struct gmem_inode { u64 flags; }; +/* Internal kernel-only flags (must not overlap with UAPI flags) */ +#define GUEST_MEMFD_F_MAPPING_FROZEN (1ULL << 63) + static inline struct gmem_inode *GMEM_I(struct inode *inode) { return container_of(inode, struct gmem_inode, vfs_inode); } struct file *__kvm_gmem_create_file(struct kvm *kvm, loff_t size, u64 flags); +void kvm_gmem_freeze(struct inode *inode, bool freeze); +bool kvm_gmem_is_frozen(struct inode *inode); #endif /* __KVM_GUEST_MEMFD_H__ */ -- 2.55.0.229.g6434b31f56-goog