From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C77BF39B95F for ; Mon, 30 Mar 2026 19:44:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774899842; cv=none; b=ixr5+ZFBlWhHYUVRng60awMCVYBiT4KSt4u4/cO+Nzg9lYg+lHH4rTrQRzOxyO0ARSZ8cTsWpvIdugvcqOsk9GAKyfLi6nw0iY5cGI6b0WBreScKs40vk+wAq+OrMlSDZ4P1ZMatDVoHFOrSE2swZZsniHQOYXYQGYpY3iv9HNg= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774899842; c=relaxed/simple; bh=Otg1owoxJwdLbz8NHDtmXF91JI4Ekv4VFQDAX69lVv4=; h=Date:To:From:Subject:Message-Id; b=fJ5ONVZ+GTSSQ5WGKdUPuApd7elU3QVJ2+wzoCND9T2fbRHupwgdPOq63dRl7CBuag/qemw2Gp3uy9/87x/vYg1sYVuw9v9j45ggbOLchFoiPmkrKIO8ky5Q6sGy9Cl4MTYzsFhaDPaZebuB/4iaPccD2D1pB45R9qqIaL7prZ4= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.b=QNww4I50; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.b="QNww4I50" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 98351C4CEF7; Mon, 30 Mar 2026 19:44:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1774899842; bh=Otg1owoxJwdLbz8NHDtmXF91JI4Ekv4VFQDAX69lVv4=; h=Date:To:From:Subject:From; b=QNww4I50Xa580vC6GbQ7J2trXG7ienUqtWd3TpEmaOthoQkMU2LBjJHCQYWgeJGCT 6KL9kudLY285PuWI4OMS8gB/PCmCFNZSZKSuXcFCca3z6AUvtr+fV6J3syq6p0PUI5 f+EGyC0bUgDWjGs8GDwr3lfSDp51mTL7ibiucgAc= Date: Mon, 30 Mar 2026 12:44:02 -0700 To: mm-commits@vger.kernel.org,rppt@kernel.org,akpm@linux-foundation.org From: Andrew Morton Subject: + userfaultfd-introduce-vm_uffd_ops-alloc_folio.patch added to mm-unstable branch Message-Id: <20260330194402.98351C4CEF7@smtp.kernel.org> Precedence: bulk X-Mailing-List: mm-commits@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: The patch titled Subject: userfaultfd: introduce vm_uffd_ops->alloc_folio() has been added to the -mm mm-unstable branch. Its filename is userfaultfd-introduce-vm_uffd_ops-alloc_folio.patch This patch will shortly appear at https://git.kernel.org/pub/scm/linux/kernel/git/akpm/25-new.git/tree/patches/userfaultfd-introduce-vm_uffd_ops-alloc_folio.patch This patch will later appear in the mm-unstable branch at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm Before you just go and hit "reply", please: a) Consider who else should be cc'ed b) Prefer to cc a suitable mailing list as well c) Ideally: find the original patch on the mailing list and do a reply-to-all to that, adding suitable additional cc's *** Remember to use Documentation/process/submit-checklist.rst when testing your code *** The -mm tree is included into linux-next via various branches at git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm and is updated there most days ------------------------------------------------------ From: "Mike Rapoport (Microsoft)" Subject: userfaultfd: introduce vm_uffd_ops->alloc_folio() Date: Mon, 30 Mar 2026 13:11:10 +0300 and use it to refactor mfill_atomic_pte_zeroed_folio() and mfill_atomic_pte_copy(). mfill_atomic_pte_zeroed_folio() and mfill_atomic_pte_copy() perform almost identical actions: * allocate a folio * update folio contents (either copy from userspace of fill with zeros) * update page tables with the new folio Split a __mfill_atomic_pte() helper that handles both cases and uses newly introduced vm_uffd_ops->alloc_folio() to allocate the folio. Pass the ops structure from the callers to __mfill_atomic_pte() to later allow using anon_uffd_ops for MAP_PRIVATE mappings of file-backed VMAs. Note, that the new ops method is called alloc_folio() rather than folio_alloc() to avoid clash with alloc_tag macro folio_alloc(). Link: https://lkml.kernel.org/r/20260330101116.1117699-10-rppt@kernel.org Signed-off-by: Mike Rapoport (Microsoft) Reviewed-by: James Houghton Cc: Andrea Arcangeli Cc: Andrei Vagin Cc: Axel Rasmussen Cc: Baolin Wang Cc: David Hildenbrand (Arm) Cc: Harry Yoo Cc: Hugh Dickins Cc: Liam Howlett Cc: Lorenzo Stoakes (Oracle) Cc: Matthew Wilcox (Oracle) Cc: Michal Hocko Cc: Muchun Song Cc: Nikita Kalyazin Cc: Oscar Salvador Cc: Paolo Bonzini Cc: Peter Xu Cc: Sean Christopherson Cc: Shuah Khan Cc: Suren Baghdasaryan Cc: Vlastimil Babka Signed-off-by: Andrew Morton --- include/linux/userfaultfd_k.h | 6 ++ mm/userfaultfd.c | 92 ++++++++++++++++---------------- 2 files changed, 54 insertions(+), 44 deletions(-) --- a/include/linux/userfaultfd_k.h~userfaultfd-introduce-vm_uffd_ops-alloc_folio +++ a/include/linux/userfaultfd_k.h @@ -94,6 +94,12 @@ struct vm_uffd_ops { * The returned folio is locked and with reference held. */ struct folio *(*get_folio_noalloc)(struct inode *inode, pgoff_t pgoff); + /* + * Called during resolution of UFFDIO_COPY request. + * Should allocate and return a folio or NULL if allocation fails. + */ + struct folio *(*alloc_folio)(struct vm_area_struct *vma, + unsigned long addr); }; /* A combined operation mode + behavior flags. */ --- a/mm/userfaultfd.c~userfaultfd-introduce-vm_uffd_ops-alloc_folio +++ a/mm/userfaultfd.c @@ -42,8 +42,26 @@ static bool anon_can_userfault(struct vm return true; } +static struct folio *anon_alloc_folio(struct vm_area_struct *vma, + unsigned long addr) +{ + struct folio *folio = vma_alloc_folio(GFP_HIGHUSER_MOVABLE, 0, vma, + addr); + + if (!folio) + return NULL; + + if (mem_cgroup_charge(folio, vma->vm_mm, GFP_KERNEL)) { + folio_put(folio); + return NULL; + } + + return folio; +} + static const struct vm_uffd_ops anon_uffd_ops = { .can_userfault = anon_can_userfault, + .alloc_folio = anon_alloc_folio, }; static const struct vm_uffd_ops *vma_uffd_ops(struct vm_area_struct *vma) @@ -456,7 +474,8 @@ static int mfill_copy_folio_retry(struct return 0; } -static int mfill_atomic_pte_copy(struct mfill_state *state) +static int __mfill_atomic_pte(struct mfill_state *state, + const struct vm_uffd_ops *ops) { unsigned long dst_addr = state->dst_addr; unsigned long src_addr = state->src_addr; @@ -464,16 +483,12 @@ static int mfill_atomic_pte_copy(struct struct folio *folio; int ret; - folio = vma_alloc_folio(GFP_HIGHUSER_MOVABLE, 0, state->vma, dst_addr); + folio = ops->alloc_folio(state->vma, state->dst_addr); if (!folio) return -ENOMEM; - ret = -ENOMEM; - if (mem_cgroup_charge(folio, state->vma->vm_mm, GFP_KERNEL)) - goto out_release; - - ret = mfill_copy_folio_locked(folio, src_addr); - if (unlikely(ret)) { + if (uffd_flags_mode_is(flags, MFILL_ATOMIC_COPY)) { + ret = mfill_copy_folio_locked(folio, src_addr); /* * Fallback to copy_from_user outside mmap_lock. * If retry is successful, mfill_copy_folio_locked() returns @@ -481,9 +496,15 @@ static int mfill_atomic_pte_copy(struct * If there was an error, we must mfill_put_vma() anyway and it * will take care of unlocking if needed. */ - ret = mfill_copy_folio_retry(state, folio); - if (ret) - goto out_release; + if (unlikely(ret)) { + ret = mfill_copy_folio_retry(state, folio); + if (ret) + goto err_folio_put; + } + } else if (uffd_flags_mode_is(flags, MFILL_ATOMIC_ZEROPAGE)) { + clear_user_highpage(&folio->page, state->dst_addr); + } else { + VM_WARN_ONCE(1, "Unknown UFFDIO operation, flags: %x", flags); } /* @@ -496,47 +517,30 @@ static int mfill_atomic_pte_copy(struct ret = mfill_atomic_install_pte(state->pmd, state->vma, dst_addr, &folio->page, true, flags); if (ret) - goto out_release; -out: - return ret; -out_release: + goto err_folio_put; + + return 0; + +err_folio_put: + folio_put(folio); /* Don't return -ENOENT so that our caller won't retry */ if (ret == -ENOENT) ret = -EFAULT; - folio_put(folio); - goto out; + return ret; } -static int mfill_atomic_pte_zeroed_folio(pmd_t *dst_pmd, - struct vm_area_struct *dst_vma, - unsigned long dst_addr) +static int mfill_atomic_pte_copy(struct mfill_state *state) { - struct folio *folio; - int ret = -ENOMEM; + const struct vm_uffd_ops *ops = vma_uffd_ops(state->vma); - folio = vma_alloc_zeroed_movable_folio(dst_vma, dst_addr); - if (!folio) - return ret; - - if (mem_cgroup_charge(folio, dst_vma->vm_mm, GFP_KERNEL)) - goto out_put; - - /* - * The memory barrier inside __folio_mark_uptodate makes sure that - * zeroing out the folio become visible before mapping the page - * using set_pte_at(). See do_anonymous_page(). - */ - __folio_mark_uptodate(folio); + return __mfill_atomic_pte(state, ops); +} - ret = mfill_atomic_install_pte(dst_pmd, dst_vma, dst_addr, - &folio->page, true, 0); - if (ret) - goto out_put; +static int mfill_atomic_pte_zeroed_folio(struct mfill_state *state) +{ + const struct vm_uffd_ops *ops = vma_uffd_ops(state->vma); - return 0; -out_put: - folio_put(folio); - return ret; + return __mfill_atomic_pte(state, ops); } static int mfill_atomic_pte_zeropage(struct mfill_state *state) @@ -549,7 +553,7 @@ static int mfill_atomic_pte_zeropage(str int ret; if (mm_forbids_zeropage(dst_vma->vm_mm)) - return mfill_atomic_pte_zeroed_folio(dst_pmd, dst_vma, dst_addr); + return mfill_atomic_pte_zeroed_folio(state); _dst_pte = pte_mkspecial(pfn_pte(zero_pfn(dst_addr), dst_vma->vm_page_prot)); _ Patches currently in -mm which might be from rppt@kernel.org are userfaultfd-introduce-mfill_copy_folio_locked-helper.patch userfaultfd-introduce-struct-mfill_state.patch userfaultfd-introduce-mfill_establish_pmd-helper.patch userfaultfd-introduce-mfill_get_vma-and-mfill_put_vma.patch userfaultfd-retry-copying-with-locks-dropped-in-mfill_atomic_pte_copy.patch userfaultfd-move-vma_can_userfault-out-of-line.patch userfaultfd-introduce-vm_uffd_ops.patch shmem-userfaultfd-use-a-vma-callback-to-handle-uffdio_continue.patch userfaultfd-introduce-vm_uffd_ops-alloc_folio.patch shmem-userfaultfd-implement-shmem-uffd-operations-using-vm_uffd_ops.patch userfaultfd-mfill_atomic-remove-retry-logic.patch