From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4ECC6C46467 for ; Mon, 16 Jan 2023 19:18:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BA2246B0072; Mon, 16 Jan 2023 14:18:08 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B526E6B0073; Mon, 16 Jan 2023 14:18:08 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9A4C76B0074; Mon, 16 Jan 2023 14:18:08 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 8B4656B0072 for ; Mon, 16 Jan 2023 14:18:08 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 68FF61A06E0 for ; Mon, 16 Jan 2023 19:18:08 +0000 (UTC) X-FDA: 80361622656.27.0CA2CCD Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf16.hostedemail.com (Postfix) with ESMTP id BC724180020 for ; Mon, 16 Jan 2023 19:18:06 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=XVjQcQS+; spf=none (imf16.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1673896687; a=rsa-sha256; cv=none; b=jaocd2+u7Az+0OVFgpSQyWXtqxZQ8299HZuIm/BlaALpS6iDwHu+o1SlmqulS8OGFgwkWg TaZygKP9sepRcyVglIZJUAt1v+R0xYVXEC7ILIz+ENl/R0gEv78NzGqdYBWYs2cLQI/O/v F+By/6CdRTw98nXZ2v++gItTaEBkzU4= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=XVjQcQS+; spf=none (imf16.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1673896687; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tkOiUS+n30+HsZTllFGts5Q9ie3dWyuhZBYnARcYVuM=; b=HwkcTBLClyXCrIiauwhYfEVQaVk/gHKvm5vrZehg4KnqmCcjpSPOGXT3m3+i32dS0rfw/0 r3SrlWeEktEFXd2ZjUXJZFedAhnTzr7NddqNwZmcfmcJUYmbLUdUrPqHAHIOEFLLiWyVvX 1xWu1/AfDupSPvcuYkqgNXkVCTOEngE= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=tkOiUS+n30+HsZTllFGts5Q9ie3dWyuhZBYnARcYVuM=; b=XVjQcQS+nmgIYecyQNPrJXUqFo 6Q+Q60PV1ZMu5D8kt5YHymP1GwWfrJkyA0AugZ99KTW2OTbLx31quIcIe5ZLxYwVkgGL3wOZfgu0+ ALg6VLovheomPGm1jeohcTXXvM2h7/xsGziv42JAlO3WsXuU/j5EWIEUhWKvSizXtVX/xjtO2LE+k /mXYD7y6l3S7NX7qYgW8z1vJ2gKoYHJ/utifGeUOtnoVGypIR99QzUHCSs8JxT5Hs2UlOgWAE30NL HFa2DAPcTZzCMNDkoKrHsSL82XpuRBdTvN5qBVy1AHV44Sn7egpG6HfvJ7y22IniSXDH5VfqmTFEO F5Rh+hHA==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1pHUzm-00904h-MU; Mon, 16 Jan 2023 19:18:14 +0000 From: "Matthew Wilcox (Oracle)" To: linux-mm@kvack.org, Andrew Morton Cc: "Matthew Wilcox (Oracle)" Subject: [PATCH 1/5] mm: Add vma_alloc_zeroed_movable_folio() Date: Mon, 16 Jan 2023 19:18:09 +0000 Message-Id: <20230116191813.2145215-2-willy@infradead.org> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20230116191813.2145215-1-willy@infradead.org> References: <20230116191813.2145215-1-willy@infradead.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Queue-Id: BC724180020 X-Rspamd-Server: rspam01 X-Stat-Signature: mu7iuts3ewobkbidkt47n88w93cyk7y8 X-HE-Tag: 1673896686-681857 X-HE-Meta: U2FsdGVkX1+hbu5kum9mbTPTprEUgRveMaAds9ZX0uNqRWuukDiMIgtwxBxzjephSUdUsONrJKJ2SJqwyqqrWtYnsmBfkcF8VyxP9lJC4tHKT97jsjSWVSnxLT6P+cQaeg7BAfGrqTx7qWfWHu8XM60/4WFT10yyZKzpJP7GjrrOp0svY65BZ308wAonZdeeFFBJyDSYCmq4bR4DGxHg/kWnBAarTH6BaOZ/DtWnuCOafUI5+pq9v5tBau7j+rJAvkaKsJ9JFdxGUm6iOUdpie1puaOCVf9NClPkaTOvFIIODNMNQ+FEzhsJ+mJrSOf5W6G7g+Wl+okF+gKPnyuijNfy14RVaVvCHPZszeMx1v8NoEUDPaUpMWBB7iiROt31015HwT/y4rCsPOM0kMmxuxEtuv9711CqdM4eVnc0OD56lBs4pBXsWtbLzZuhedKR3xo9aqH86Zuo+7Ko4TcTioiS63vs8M1xfI5dOc4OfKSYiQwdZ8sNkrp9Uvxd4qYiKOVZHOJWeOg+Ifa5s71OVIlZsoaz6VcKlw5LVP3X8YHBnCGKxR2rK1GtL1eGTDaNee3N5zdbosq+wPNUdNBW97X7OR3qLAMLyHzy8AsSBCPMcAoUXJDohZ6gzTx+i1J+MPtneZT3aIKCP6NzjumFxYpMNEvRY3pgBU7aZSFavvRrLOu4Wtedi0pyFKDrwCB8LrmZHGK8neea60fh4Q9cFo9rdE+y4eF/N9jrjPcDX0rpXMfU4U1itP8TI8inCJOXrb19Y9/yUQCyOmU2MihNuzGd7eCcAqjs7lCcyLoRTvzNjCHEHlHAq7xaKsyjSkzg6PIe8EvSg9T6lpszE+U+cquk6IAXVsAxiv5X7IL8uLdnZXfTOjsRLHxc5LYVBhN4u6txDDB+dmsr64z/+OzeiiE9I6RDxzzVonT5vL4C7pLIn260uAJCqA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Replace alloc_zeroed_user_highpage_movable(). The main difference is returning a folio containing a single page instead of returning the page, but take the opportunity to rename the function to match other allocation functions a little better and rewrite the documentation to place more emphasis on the zeroing rather than the highmem aspect. Signed-off-by: Matthew Wilcox (Oracle) --- arch/alpha/include/asm/page.h | 5 ++--- arch/arm64/include/asm/page.h | 4 ++-- arch/arm64/mm/fault.c | 4 ++-- arch/ia64/include/asm/page.h | 14 ++++++-------- arch/m68k/include/asm/page_no.h | 5 ++--- arch/s390/include/asm/page.h | 5 ++--- arch/x86/include/asm/page.h | 5 ++--- include/linux/highmem.h | 33 ++++++++++++++++----------------- mm/memory.c | 16 ++++++++++------ 9 files changed, 44 insertions(+), 47 deletions(-) diff --git a/arch/alpha/include/asm/page.h b/arch/alpha/include/asm/page.h index 8f3f5eecba28..bc5256fba8f0 100644 --- a/arch/alpha/include/asm/page.h +++ b/arch/alpha/include/asm/page.h @@ -17,9 +17,8 @@ extern void clear_page(void *page); #define clear_user_page(page, vaddr, pg) clear_page(page) -#define alloc_zeroed_user_highpage_movable(vma, vaddr) \ - alloc_page_vma(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, vma, vaddr) -#define __HAVE_ARCH_ALLOC_ZEROED_USER_HIGHPAGE_MOVABLE +#define vma_alloc_zeroed_movable_folio(vma, vaddr) \ + vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr, false) extern void copy_page(void * _to, void * _from); #define copy_user_page(to, from, vaddr, pg) copy_page(to, from) diff --git a/arch/arm64/include/asm/page.h b/arch/arm64/include/asm/page.h index 993a27ea6f54..2312e6ee595f 100644 --- a/arch/arm64/include/asm/page.h +++ b/arch/arm64/include/asm/page.h @@ -29,9 +29,9 @@ void copy_user_highpage(struct page *to, struct page *from, void copy_highpage(struct page *to, struct page *from); #define __HAVE_ARCH_COPY_HIGHPAGE -struct page *alloc_zeroed_user_highpage_movable(struct vm_area_struct *vma, +struct folio *vma_alloc_zeroed_movable_folio(struct vm_area_struct *vma, unsigned long vaddr); -#define __HAVE_ARCH_ALLOC_ZEROED_USER_HIGHPAGE_MOVABLE +#define vma_alloc_zeroed_movable_folio vma_alloc_zeroed_movable_folio void tag_clear_highpage(struct page *to); #define __HAVE_ARCH_TAG_CLEAR_HIGHPAGE diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 596f46dabe4e..f4cb0f85ccf4 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -925,7 +925,7 @@ NOKPROBE_SYMBOL(do_debug_exception); /* * Used during anonymous page fault handling. */ -struct page *alloc_zeroed_user_highpage_movable(struct vm_area_struct *vma, +struct folio *vma_alloc_zeroed_movable_folio(struct vm_area_struct *vma, unsigned long vaddr) { gfp_t flags = GFP_HIGHUSER_MOVABLE | __GFP_ZERO; @@ -938,7 +938,7 @@ struct page *alloc_zeroed_user_highpage_movable(struct vm_area_struct *vma, if (vma->vm_flags & VM_MTE) flags |= __GFP_ZEROTAGS; - return alloc_page_vma(flags, vma, vaddr); + return vma_alloc_folio(flags, 0, vma, vaddr, false); } void tag_clear_highpage(struct page *page) diff --git a/arch/ia64/include/asm/page.h b/arch/ia64/include/asm/page.h index 1b990466d540..ba0b365cf2b2 100644 --- a/arch/ia64/include/asm/page.h +++ b/arch/ia64/include/asm/page.h @@ -82,17 +82,15 @@ do { \ } while (0) -#define alloc_zeroed_user_highpage_movable(vma, vaddr) \ +#define vma_alloc_zeroed_movable_folio(vma, vaddr) \ ({ \ - struct page *page = alloc_page_vma( \ - GFP_HIGHUSER_MOVABLE | __GFP_ZERO, vma, vaddr); \ - if (page) \ - flush_dcache_page(page); \ - page; \ + struct folio *folio = vma_alloc_folio( \ + GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr, false); \ + if (folio) \ + flush_dcache_folio(folio); \ + folio; \ }) -#define __HAVE_ARCH_ALLOC_ZEROED_USER_HIGHPAGE_MOVABLE - #define virt_addr_valid(kaddr) pfn_valid(__pa(kaddr) >> PAGE_SHIFT) #include diff --git a/arch/m68k/include/asm/page_no.h b/arch/m68k/include/asm/page_no.h index c9d0d84158a4..abd2c3aeb015 100644 --- a/arch/m68k/include/asm/page_no.h +++ b/arch/m68k/include/asm/page_no.h @@ -13,9 +13,8 @@ extern unsigned long memory_end; #define clear_user_page(page, vaddr, pg) clear_page(page) #define copy_user_page(to, from, vaddr, pg) copy_page(to, from) -#define alloc_zeroed_user_highpage_movable(vma, vaddr) \ - alloc_page_vma(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, vma, vaddr) -#define __HAVE_ARCH_ALLOC_ZEROED_USER_HIGHPAGE_MOVABLE +#define vma_alloc_zeroed_movable_folio(vma, vaddr) \ + vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr, false) #define __pa(vaddr) ((unsigned long)(vaddr)) #define __va(paddr) ((void *)((unsigned long)(paddr))) diff --git a/arch/s390/include/asm/page.h b/arch/s390/include/asm/page.h index 61dea67bb9c7..8a2a3b5d1e29 100644 --- a/arch/s390/include/asm/page.h +++ b/arch/s390/include/asm/page.h @@ -73,9 +73,8 @@ static inline void copy_page(void *to, void *from) #define clear_user_page(page, vaddr, pg) clear_page(page) #define copy_user_page(to, from, vaddr, pg) copy_page(to, from) -#define alloc_zeroed_user_highpage_movable(vma, vaddr) \ - alloc_page_vma(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, vma, vaddr) -#define __HAVE_ARCH_ALLOC_ZEROED_USER_HIGHPAGE_MOVABLE +#define vma_alloc_zeroed_movable_folio(vma, vaddr) \ + vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr, false) /* * These are used to make use of C type-checking.. diff --git a/arch/x86/include/asm/page.h b/arch/x86/include/asm/page.h index 9cc82f305f4b..d18e5c332cb9 100644 --- a/arch/x86/include/asm/page.h +++ b/arch/x86/include/asm/page.h @@ -34,9 +34,8 @@ static inline void copy_user_page(void *to, void *from, unsigned long vaddr, copy_page(to, from); } -#define alloc_zeroed_user_highpage_movable(vma, vaddr) \ - alloc_page_vma(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, vma, vaddr) -#define __HAVE_ARCH_ALLOC_ZEROED_USER_HIGHPAGE_MOVABLE +#define vma_alloc_zeroed_movable_folio(vma, vaddr) \ + vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr, false) #ifndef __pa #define __pa(x) __phys_addr((unsigned long)(x)) diff --git a/include/linux/highmem.h b/include/linux/highmem.h index 56703082f803..9fa462561e05 100644 --- a/include/linux/highmem.h +++ b/include/linux/highmem.h @@ -208,31 +208,30 @@ static inline void clear_user_highpage(struct page *page, unsigned long vaddr) } #endif -#ifndef __HAVE_ARCH_ALLOC_ZEROED_USER_HIGHPAGE_MOVABLE +#ifndef vma_alloc_zeroed_movable_folio /** - * alloc_zeroed_user_highpage_movable - Allocate a zeroed HIGHMEM page for a VMA that the caller knows can move - * @vma: The VMA the page is to be allocated for - * @vaddr: The virtual address the page will be inserted into + * vma_alloc_zeroed_movable_folio - Allocate a zeroed page for a VMA. + * @vma: The VMA the page is to be allocated for. + * @vaddr: The virtual address the page will be inserted into. * - * Returns: The allocated and zeroed HIGHMEM page + * This function will allocate a page suitable for inserting into this + * VMA at this virtual address. It may be allocated from highmem or + * the movable zone. An architecture may provide its own implementation. * - * This function will allocate a page for a VMA that the caller knows will - * be able to migrate in the future using move_pages() or reclaimed - * - * An architecture may override this function by defining - * __HAVE_ARCH_ALLOC_ZEROED_USER_HIGHPAGE_MOVABLE and providing their own - * implementation. + * Return: A folio containing one allocated and zeroed page or NULL if + * we are out of memory. */ -static inline struct page * -alloc_zeroed_user_highpage_movable(struct vm_area_struct *vma, +static inline +struct folio *vma_alloc_zeroed_movable_folio(struct vm_area_struct *vma, unsigned long vaddr) { - struct page *page = alloc_page_vma(GFP_HIGHUSER_MOVABLE, vma, vaddr); + struct folio *folio; - if (page) - clear_user_highpage(page, vaddr); + folio = vma_alloc_folio(GFP_HIGHUSER_MOVABLE, 0, vma, vaddr, false); + if (folio) + clear_user_highpage(&folio->page, vaddr); - return page; + return folio; } #endif diff --git a/mm/memory.c b/mm/memory.c index f7613650efd4..4cb2cd809b18 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3056,10 +3056,12 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf) goto oom; if (is_zero_pfn(pte_pfn(vmf->orig_pte))) { - new_page = alloc_zeroed_user_highpage_movable(vma, - vmf->address); - if (!new_page) + struct folio *new_folio; + + new_folio = vma_alloc_zeroed_movable_folio(vma, vmf->address); + if (!new_folio) goto oom; + new_page = &new_folio->page; } else { new_page = alloc_page_vma(GFP_HIGHUSER_MOVABLE, vma, vmf->address); @@ -3998,6 +4000,7 @@ static vm_fault_t do_anonymous_page(struct vm_fault *vmf) { struct vm_area_struct *vma = vmf->vma; struct page *page; + struct folio *folio; vm_fault_t ret = 0; pte_t entry; @@ -4047,11 +4050,12 @@ static vm_fault_t do_anonymous_page(struct vm_fault *vmf) /* Allocate our own private page. */ if (unlikely(anon_vma_prepare(vma))) goto oom; - page = alloc_zeroed_user_highpage_movable(vma, vmf->address); - if (!page) + folio = vma_alloc_zeroed_movable_folio(vma, vmf->address); + if (!folio) goto oom; - if (mem_cgroup_charge(page_folio(page), vma->vm_mm, GFP_KERNEL)) + page = &folio->page; + if (mem_cgroup_charge(folio, vma->vm_mm, GFP_KERNEL)) goto oom_free_page; cgroup_throttle_swaprate(page, GFP_KERNEL); -- 2.35.1