From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2F9E9CDB475 for ; Mon, 22 Jun 2026 18:49:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To:From: Subject:Message-ID:References:Mime-Version:In-Reply-To:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=rUJD/qDxfrMv3GVCezTN8Ddb/ezsPEksCOi1qF1OubI=; b=TJgt5xfTSDgpvJLi/JiWtjHhVe Y/lJXb3tDDSxEnwQHHTgluHF6uEgmf7mLT3G0xcyGKTmUXFMXMB0Aur9RD5nsCG3JM6d3fXoFZCh6 w8s9CBeCrf5mt8XFW6C4dDgGfMj5EscjIFiJQiXcr8l2gfhvNIvRaHcuEOLht7ihcdj1k9GkPVPcS CDQ4DwYyLkzbRxGxOZxL9XDoq/NFljkkNBGegvHWAyZz1wYblE1OziXMaCtA7iX3Nrc50AdxeE97v c20H7omAB5qkYLl8YyKI7I9R4djxoBwr+L9p9toAT67EmpwtPk2twg4IF1F9zZ9zUd8Tm7HV3ixpR N6kfToUw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.99.1 #2 (Red Hat Linux)) id 1wbjhp-00000005JMe-12BA; Mon, 22 Jun 2026 18:49:13 +0000 Received: from mail-ej1-x649.google.com ([2a00:1450:4864:20::649]) by bombadil.infradead.org with esmtps (Exim 4.99.1 #2 (Red Hat Linux)) id 1wbjhk-00000005JHP-33dC for kexec@lists.infradead.org; Mon, 22 Jun 2026 18:49:09 +0000 Received: by mail-ej1-x649.google.com with SMTP id a640c23a62f3a-c08306ae1c3so379669166b.1 for ; Mon, 22 Jun 2026 11:49:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1782154147; x=1782758947; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=rUJD/qDxfrMv3GVCezTN8Ddb/ezsPEksCOi1qF1OubI=; b=jjCYBH7GnSO8SE6B9Gp5/d5KuScjwWoPuStW9MGiGG8GGKfD39q9mcAWyjVZst3BGr Lnis63Rm7CI8QgJGirbPzqeetPSbJNg6oJMuGINhQ+vybZepMd+RMV7uBijP91mE9mQf EXq6GtFqUYn4PndlmhPMMOJ5R+M0Yp3shOpS2IAsxcoL/7mCJzONCEme/z8zJy1R6Ecf RQZnjmWsBUIu9viceiQYZeDKYNhTDdqywVh0J3h6RgNDkjo8m1FN0KcMRA74I9o0lJwK t8LjwDMECYpenYDNPrSQAB6PUbBGeLdfJKASLhXUIc4aQ9oYOyUuJb87GVUhzJjQmuM5 EkpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1782154147; x=1782758947; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=rUJD/qDxfrMv3GVCezTN8Ddb/ezsPEksCOi1qF1OubI=; b=ZXk32YE8sr8yUQQ2F3VdHOd7mz4PGANyppXbs7G1eLEfvri1Y94CeKs0cUIS2lkSTi +8oqsbXRzcFxwt3pBdexpqyTlzy728niPYHB/dj+xMlfQcUT6PaVMGVn/C/WsImRWxX2 unJdgGK6HFJmBccmD/NfDPc1RX9mN5R4o3HLIkUuNFb8hG6L1QPrtnHsGyiCSVuzsWqa +svTECPzDjCd073WGsGEA5tHFdXLDKRxK8C+2NoIGs1xeTRB/JbHztWcvpa3wQ7cUXP4 4pKEe4DZJk/2LbIl6BBI7enoFy4La+/NnBLpaSZUpeIcNavY/hoks5T6xPguIws/U/TZ Q1Zw== X-Forwarded-Encrypted: i=1; AFNElJ9+AyxkupX6uiYZJdmL6qZEkmmoUH3XhTAjfnQunqDIi7R0VKuB5S53I8fUxPwjKCVECSm85w==@lists.infradead.org X-Gm-Message-State: AOJu0Yyq8VerdpVuALWWoVqFNUPPKmXY5Q+u3EvxN9lUizKj1dJ0H5G4 LsCQ0Zt0ViGLNakK2ZaFGVdrgJrmgAXsP1pXUwED+IjSZnqzOL740byABoViEUsBpGOl/J7cUe7 5q5c0f6lEjX83uDkoCg== X-Received: from ejet16.prod.google.com ([2002:a17:906:1790:b0:c08:3cd3:d05f]) (user=tarunsahu job=prod-delivery.src-stubby-dispatcher) by 2002:a17:906:7951:b0:bf1:1df7:3e50 with SMTP id a640c23a62f3a-c0c63dba03amr559821466b.3.1782154146145; Mon, 22 Jun 2026 11:49:06 -0700 (PDT) Date: Mon, 22 Jun 2026 18:48:47 +0000 In-Reply-To: <20260622184851.2309827-1-tarunsahu@google.com> Mime-Version: 1.0 References: <20260622184851.2309827-1-tarunsahu@google.com> X-Mailer: git-send-email 2.55.0.rc0.786.g65d90a0328-goog Message-ID: <20260622184851.2309827-6-tarunsahu@google.com> Subject: [PATCH v3 5/9] kvm: guest_memfd: Add support for freezing and unfreezing mappings From: Tarun Sahu To: Jonathan Corbet , Mike Rapoport , Paolo Bonzini , Alexander Graf , Shuah Khan , Pratyush Yadav , Tarun Sahu , Pasha Tatashin Cc: kvm@vger.kernel.org, linux-mm@kvack.org, kexec@lists.infradead.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.9.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20260622_114908_812642_C9D14214 X-CRM114-Status: GOOD ( 25.12 ) X-BeenThere: kexec@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "kexec" Errors-To: kexec-bounces+kexec=archiver.kernel.org@lists.infradead.org This patch introduces the freeze on gmem_inode which prevents the fallocate call and any new page fault allocation. This will avoid gmem file modification when it is being preserved Used srcu lock to synchronise the freeze call, where write blocks until all the reads are free. And reads are re-entrant. Incase fault fails, It return -EPERM and VM_EXIT to userspace. userspace must handle this properly as every new fault will fail. Signed-off-by: Tarun Sahu --- virt/kvm/guest_memfd.c | 117 +++++++++++++++++++++++++++++++++++++---- virt/kvm/guest_memfd.h | 5 ++ 2 files changed, 111 insertions(+), 11 deletions(-) diff --git a/virt/kvm/guest_memfd.c b/virt/kvm/guest_memfd.c index fe1adc9b..a4d9d34 100644 --- a/virt/kvm/guest_memfd.c +++ b/virt/kvm/guest_memfd.c @@ -7,11 +7,13 @@ #include #include #include +#include #include "guest_memfd.h" #include "kvm_mm.h" static struct vfsmount *kvm_gmem_mnt; +static struct srcu_struct kvm_gmem_freeze_srcu; #define kvm_gmem_for_each_file(f, inode) \ @@ -96,6 +98,7 @@ static struct folio *kvm_gmem_get_folio(struct inode *inode, pgoff_t index) /* TODO: Support huge pages. */ struct mempolicy *policy; struct folio *folio; + int idx; /* * Fast-path: See if folio is already present in mapping to avoid @@ -105,12 +108,20 @@ static struct folio *kvm_gmem_get_folio(struct inode *inode, pgoff_t index) if (!IS_ERR(folio)) return folio; + idx = srcu_read_lock(&kvm_gmem_freeze_srcu); + if (kvm_gmem_is_frozen(inode)) { + srcu_read_unlock(&kvm_gmem_freeze_srcu, idx); + return ERR_PTR(-EPERM); + } + policy = mpol_shared_policy_lookup(&GMEM_I(inode)->policy, index); folio = __filemap_get_folio_mpol(inode->i_mapping, index, FGP_LOCK | FGP_CREAT, mapping_gfp_mask(inode->i_mapping), policy); mpol_cond_put(policy); + srcu_read_unlock(&kvm_gmem_freeze_srcu, idx); + /* * External interfaces like kvm_gmem_get_pfn() support dealing * with hugepages to a degree, but internally, guest_memfd currently @@ -273,16 +284,30 @@ static long kvm_gmem_allocate(struct inode *inode, loff_t offset, loff_t len) static long kvm_gmem_fallocate(struct file *file, int mode, loff_t offset, loff_t len) { + struct inode *inode = file_inode(file); int ret; + int idx; - if (!(mode & FALLOC_FL_KEEP_SIZE)) - return -EOPNOTSUPP; + idx = srcu_read_lock(&kvm_gmem_freeze_srcu); + if (kvm_gmem_is_frozen(inode)) { + srcu_read_unlock(&kvm_gmem_freeze_srcu, idx); + return -EPERM; + } - if (mode & ~(FALLOC_FL_KEEP_SIZE | FALLOC_FL_PUNCH_HOLE)) - return -EOPNOTSUPP; + if (!(mode & FALLOC_FL_KEEP_SIZE)) { + ret = -EOPNOTSUPP; + goto out; + } - if (!PAGE_ALIGNED(offset) || !PAGE_ALIGNED(len)) - return -EINVAL; + if (mode & ~(FALLOC_FL_KEEP_SIZE | FALLOC_FL_PUNCH_HOLE)) { + ret = -EOPNOTSUPP; + goto out; + } + + if (!PAGE_ALIGNED(offset) || !PAGE_ALIGNED(len)) { + ret = -EINVAL; + goto out; + } if (mode & FALLOC_FL_PUNCH_HOLE) ret = kvm_gmem_punch_hole(file_inode(file), offset, len); @@ -291,6 +316,9 @@ static long kvm_gmem_fallocate(struct file *file, int mode, loff_t offset, if (!ret) file_modified(file); + +out: + srcu_read_unlock(&kvm_gmem_freeze_srcu, idx); return ret; } @@ -948,7 +976,9 @@ static void kvm_gmem_destroy_inode(struct inode *inode) static void kvm_gmem_free_inode(struct inode *inode) { - kmem_cache_free(kvm_gmem_inode_cachep, GMEM_I(inode)); + struct gmem_inode *gi = GMEM_I(inode); + + kmem_cache_free(kvm_gmem_inode_cachep, gi); } static const struct super_operations kvm_gmem_super_operations = { @@ -1005,12 +1035,21 @@ int kvm_gmem_init(struct module *module) if (!kvm_gmem_inode_cachep) return -ENOMEM; + ret = init_srcu_struct(&kvm_gmem_freeze_srcu); + if (ret) + goto err_cache; + ret = kvm_gmem_init_mount(); - if (ret) { - kmem_cache_destroy(kvm_gmem_inode_cachep); - return ret; - } + if (ret) + goto err_srcu; + return 0; + +err_srcu: + cleanup_srcu_struct(&kvm_gmem_freeze_srcu); +err_cache: + kmem_cache_destroy(kvm_gmem_inode_cachep); + return ret; } void kvm_gmem_exit(void) @@ -1018,5 +1057,61 @@ void kvm_gmem_exit(void) kern_unmount(kvm_gmem_mnt); kvm_gmem_mnt = NULL; rcu_barrier(); + cleanup_srcu_struct(&kvm_gmem_freeze_srcu); kmem_cache_destroy(kvm_gmem_inode_cachep); } + +/** + * kvm_gmem_freeze - Freeze or unfreeze a guest_memfd inode mapping. + * @inode: The guest_memfd inode. + * @freeze: True to freeze, false to unfreeze. + * + * This API is used strictly during the live update / preservation transition + * window to prevent host userspace and guest-side faults from making any + * mapping modifications (such as fallocate or page fault allocation) + * to the guest_memfd page cache. + * + * Synchronization Strategy (Sleepable RCU): + * To avoid high-contention VFS locks (like inode_lock or + * filemap_invalidate_lock) on the vCPU page fault hot paths, this subsystem + * implements a lightweight, system-wide Sleepable RCU (SRCU) mechanism + * (`kvm_gmem_freeze_srcu`): + * + * Global vs. Per-Inode SRCU + * ====================== + * A single system-wide global static `srcu_struct` is used instead of a + * per-inode SRCU structure to completely prevent unprivileged users from + * exhausting the host's per-CPU memory allocator. Because + * `init_srcu_struct()` allocates per-CPU memory via `alloc_percpu()`, which + * is not accounted by memory cgroups (memcg), + * a per-inode SRCU structure would allow a tenant to bypass cgroup limits and + * trigger a system-wide Out-of-Memory (OOM) crash simply by spawning a large + * number of guest_memfd file descriptors (bounded only by RLIMIT_NOFILE). + * + * Flag Modification Note: + * Since `GUEST_MEMFD_F_MAPPING_FROZEN` is the ONLY flag in + * `GMEM_I(inode)->flags` that is mutated dynamically at runtime (all other + * flags are creation-time flags which remain strictly read-only), there is + * no possibility of concurrent bit-modification races. Therefore, a standard + * `WRITE_ONCE` is fully safe and does not require complex `cmpxchg` + * synchronization loops. + */ +void kvm_gmem_freeze(struct inode *inode, bool freeze) +{ + u64 flags = READ_ONCE(GMEM_I(inode)->flags); + + if (freeze) + flags |= GUEST_MEMFD_F_MAPPING_FROZEN; + else + flags &= ~GUEST_MEMFD_F_MAPPING_FROZEN; + + WRITE_ONCE(GMEM_I(inode)->flags, flags); + + if (freeze) + synchronize_srcu(&kvm_gmem_freeze_srcu); +} + +bool kvm_gmem_is_frozen(struct inode *inode) +{ + return READ_ONCE(GMEM_I(inode)->flags) & GUEST_MEMFD_F_MAPPING_FROZEN; +} diff --git a/virt/kvm/guest_memfd.h b/virt/kvm/guest_memfd.h index c528b04..028c348 100644 --- a/virt/kvm/guest_memfd.h +++ b/virt/kvm/guest_memfd.h @@ -29,11 +29,16 @@ struct gmem_inode { u64 flags; }; +/* Internal kernel-only flags (must not overlap with UAPI flags) */ +#define GUEST_MEMFD_F_MAPPING_FROZEN (1ULL << 63) + static inline struct gmem_inode *GMEM_I(struct inode *inode) { return container_of(inode, struct gmem_inode, vfs_inode); } struct file *__kvm_gmem_create_file(struct kvm *kvm, loff_t size, u64 flags); +void kvm_gmem_freeze(struct inode *inode, bool freeze); +bool kvm_gmem_is_frozen(struct inode *inode); #endif /* __KVM_GUEST_MEMFD_H__ */ -- 2.55.0.rc0.786.g65d90a0328-goog