From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BF44E10FF for ; Thu, 22 Feb 2024 00:02:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708560175; cv=none; b=sc4jTArNVowgxm8inBDVxZE2JgX/+aPTOuLPlDvKv/pp+rFcJNSZ8U3PI+G0f4vRRdJJdQIyn4Kpl8xN5JPj4hcgW1YH75pFmRo/uoMzHqzgJw/o2AgmWtTV+vTFV6dw0911pjEe97aKAK8S7Kk7agEV6TtyVOfRw+23davzKPA= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708560175; c=relaxed/simple; bh=N0ogOAfRI6wWAbHiDYJ75gjpy1YAwZt2IXoxbAqhlkw=; h=Date:To:From:Subject:Message-Id; b=cJC4C6v+/xFKgOgzfPFP6URzlMfF4PgEYeUda8MpcVjUkPBLoES0cBgZiF828Krqk1RvjXix0KDBuB7XzLGHcbtOKcwRVuTzC+Y07ecPSjHMlKCrZGnjuIZhd4SjKVKccqwY+ybnZVooo0hEUsbaKM+PCUImo5AJZBRjXf6zVbQ= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.b=yKKjR2K/; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux-foundation.org header.i=@linux-foundation.org header.b="yKKjR2K/" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 89319C433F1; Thu, 22 Feb 2024 00:02:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1708560175; bh=N0ogOAfRI6wWAbHiDYJ75gjpy1YAwZt2IXoxbAqhlkw=; h=Date:To:From:Subject:From; b=yKKjR2K/ILoQmMNK1K77fbFvqq83fbxK6x0S66vwK0nVH4dcLbX1r8wohbR/Vd1qc kCVnWqnbtWaNH6kO/XWciHPXiTA3aYzfU0Zo98ylkHyzi8PGm9F0QfGsPzX/E96U0r nHcNYf4EjqY97iL/JEUdbn+IGRAzYMzl9TF7HqTM= Date: Wed, 21 Feb 2024 16:02:55 -0800 To: mm-commits@vger.kernel.org,willy@infradead.org,will@kernel.org,svens@linux.ibm.com,ryan.roberts@arm.com,rppt@kernel.org,paul.walmsley@sifive.com,palmer@dabbelt.com,npiggin@gmail.com,naveen.n.rao@linux.ibm.com,mpe@ellerman.id.au,linux@armlinux.org.uk,hca@linux.ibm.com,gor@linux.ibm.com,gerald.schaefer@linux.ibm.com,dinguyen@kernel.org,davem@davemloft.net,christophe.leroy@csgroup.eu,catalin.marinas@arm.com,borntraeger@linux.ibm.com,aou@eecs.berkeley.edu,aneesh.kumar@kernel.org,alexghiti@rivosinc.com,agordeev@linux.ibm.com,david@redhat.com,akpm@linux-foundation.org From: Andrew Morton Subject: [merged mm-stable] mm-memory-factor-out-copying-the-actual-pte-in-copy_present_pte.patch removed from -mm tree Message-Id: <20240222000255.89319C433F1@smtp.kernel.org> Precedence: bulk X-Mailing-List: mm-commits@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: The quilt patch titled Subject: mm/memory: factor out copying the actual PTE in copy_present_pte() has been removed from the -mm tree. Its filename was mm-memory-factor-out-copying-the-actual-pte-in-copy_present_pte.patch This patch was dropped because it was merged into the mm-stable branch of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm ------------------------------------------------------ From: David Hildenbrand Subject: mm/memory: factor out copying the actual PTE in copy_present_pte() Date: Mon, 29 Jan 2024 13:46:45 +0100 Let's prepare for further changes. Link: https://lkml.kernel.org/r/20240129124649.189745-12-david@redhat.com Signed-off-by: David Hildenbrand Reviewed-by: Ryan Roberts Reviewed-by: Mike Rapoport (IBM) Cc: Albert Ou Cc: Alexander Gordeev Cc: Alexandre Ghiti Cc: Aneesh Kumar K.V Cc: Catalin Marinas Cc: Christian Borntraeger Cc: Christophe Leroy Cc: David S. Miller Cc: Dinh Nguyen Cc: Gerald Schaefer Cc: Heiko Carstens Cc: Matthew Wilcox Cc: Michael Ellerman Cc: Naveen N. Rao Cc: Nicholas Piggin Cc: Palmer Dabbelt Cc: Paul Walmsley Cc: Russell King (Oracle) Cc: Sven Schnelle Cc: Vasily Gorbik Cc: Will Deacon Signed-off-by: Andrew Morton --- mm/memory.c | 63 ++++++++++++++++++++++++++------------------------ 1 file changed, 33 insertions(+), 30 deletions(-) --- a/mm/memory.c~mm-memory-factor-out-copying-the-actual-pte-in-copy_present_pte +++ a/mm/memory.c @@ -930,6 +930,29 @@ copy_present_page(struct vm_area_struct return 0; } +static inline void __copy_present_pte(struct vm_area_struct *dst_vma, + struct vm_area_struct *src_vma, pte_t *dst_pte, pte_t *src_pte, + pte_t pte, unsigned long addr) +{ + struct mm_struct *src_mm = src_vma->vm_mm; + + /* If it's a COW mapping, write protect it both processes. */ + if (is_cow_mapping(src_vma->vm_flags) && pte_write(pte)) { + ptep_set_wrprotect(src_mm, addr, src_pte); + pte = pte_wrprotect(pte); + } + + /* If it's a shared mapping, mark it clean in the child. */ + if (src_vma->vm_flags & VM_SHARED) + pte = pte_mkclean(pte); + pte = pte_mkold(pte); + + if (!userfaultfd_wp(dst_vma)) + pte = pte_clear_uffd_wp(pte); + + set_pte_at(dst_vma->vm_mm, addr, dst_pte, pte); +} + /* * Copy one pte. Returns 0 if succeeded, or -EAGAIN if one preallocated page * is required to copy this pte. @@ -939,23 +962,23 @@ copy_present_pte(struct vm_area_struct * pte_t *dst_pte, pte_t *src_pte, unsigned long addr, int *rss, struct folio **prealloc) { - struct mm_struct *src_mm = src_vma->vm_mm; - unsigned long vm_flags = src_vma->vm_flags; pte_t pte = ptep_get(src_pte); struct page *page; struct folio *folio; page = vm_normal_page(src_vma, addr, pte); - if (page) - folio = page_folio(page); - if (page && folio_test_anon(folio)) { + if (unlikely(!page)) + goto copy_pte; + + folio = page_folio(page); + folio_get(folio); + if (folio_test_anon(folio)) { /* * If this page may have been pinned by the parent process, * copy the page immediately for the child so that we'll always * guarantee the pinned page won't be randomly replaced in the * future. */ - folio_get(folio); if (unlikely(folio_try_dup_anon_rmap_pte(folio, page, src_vma))) { /* Page may be pinned, we have to copy. */ folio_put(folio); @@ -963,34 +986,14 @@ copy_present_pte(struct vm_area_struct * addr, rss, prealloc, page); } rss[MM_ANONPAGES]++; - } else if (page) { - folio_get(folio); + VM_WARN_ON_FOLIO(PageAnonExclusive(page), folio); + } else { folio_dup_file_rmap_pte(folio, page); rss[mm_counter_file(folio)]++; } - /* - * If it's a COW mapping, write protect it both - * in the parent and the child - */ - if (is_cow_mapping(vm_flags) && pte_write(pte)) { - ptep_set_wrprotect(src_mm, addr, src_pte); - pte = pte_wrprotect(pte); - } - VM_BUG_ON(page && folio_test_anon(folio) && PageAnonExclusive(page)); - - /* - * If it's a shared mapping, mark it clean in - * the child - */ - if (vm_flags & VM_SHARED) - pte = pte_mkclean(pte); - pte = pte_mkold(pte); - - if (!userfaultfd_wp(dst_vma)) - pte = pte_clear_uffd_wp(pte); - - set_pte_at(dst_vma->vm_mm, addr, dst_pte, pte); +copy_pte: + __copy_present_pte(dst_vma, src_vma, dst_pte, src_pte, pte, addr); return 0; } _ Patches currently in -mm which might be from david@redhat.com are mm-memory-factor-out-zapping-of-present-pte-into-zap_present_pte.patch mm-memory-handle-page-case-in-zap_present_pte-separately.patch mm-memory-further-separate-anon-and-pagecache-folio-handling-in-zap_present_pte.patch mm-memory-factor-out-zapping-folio-pte-into-zap_present_folio_pte.patch mm-mmu_gather-pass-delay_rmap-instead-of-encoded-page-to-__tlb_remove_page_size.patch mm-mmu_gather-define-encoded_page_flag_delay_rmap.patch mm-mmu_gather-add-tlb_remove_tlb_entries.patch mm-mmu_gather-add-__tlb_remove_folio_pages.patch mm-mmu_gather-improve-cond_resched-handling-with-large-folios-and-expensive-page-freeing.patch mm-memory-optimize-unmap-zap-with-pte-mapped-thp.patch