From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 76466C83F01 for ; Wed, 30 Aug 2023 07:20:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EEE47280031; Wed, 30 Aug 2023 03:20:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E9D298E0009; Wed, 30 Aug 2023 03:20:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D65E3280031; Wed, 30 Aug 2023 03:20:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id C5B028E0009 for ; Wed, 30 Aug 2023 03:20:45 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 8AE3C802A1 for ; Wed, 30 Aug 2023 07:20:45 +0000 (UTC) X-FDA: 81179923650.05.DDE91A4 Received: from out-246.mta1.migadu.com (out-246.mta1.migadu.com [95.215.58.246]) by imf15.hostedemail.com (Postfix) with ESMTP id 99379A0020 for ; Wed, 30 Aug 2023 07:20:43 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=aZ+Z5jUW; spf=pass (imf15.hostedemail.com: domain of muchun.song@linux.dev designates 95.215.58.246 as permitted sender) smtp.mailfrom=muchun.song@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1693380043; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+KxXdpail2mUes3X2GO5ePhGNuXuJJMiMuAOXzmAzYc=; b=YgSJIH2an6xOYn7TDTrevXzShg0nRb5vRdVTKkWtvRCugkT99lXQs7KOzaKiUYuWpZBSCo sVdLKpucaJgVi1QSjhbvvHvAKEwUCJB+GjdJYpF7PknXRKeY1h2+WBAabscCXnugkcCPhG JjbFF90nRFk1ltSVPow79rWcIXmuZy8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1693380043; a=rsa-sha256; cv=none; b=6cE3lRR+XoWO/SeMI6vifLavqru1YmMOAvcdtnn0RIvngBq2Ewd9B4RvqJpZ+ED8I4swJK Shz8H2lUb2t2kRq7Nb/Txkvm+alMs+qrX8NtlPnrrawZgcrv7K2HqxdBBExs4sM2lfyaGw NnKsaUHBFh0OXWwMxfq+dvW+Zeo+JxQ= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=aZ+Z5jUW; spf=pass (imf15.hostedemail.com: domain of muchun.song@linux.dev designates 95.215.58.246 as permitted sender) smtp.mailfrom=muchun.song@linux.dev; dmarc=pass (policy=none) header.from=linux.dev Message-ID: <79f4d305-bddb-9042-d15f-a683fd6003d9@linux.dev> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1693380041; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+KxXdpail2mUes3X2GO5ePhGNuXuJJMiMuAOXzmAzYc=; b=aZ+Z5jUW0UiBQjucRr8I9dLYv+mLaEatzF+k1jv5iLH/PrYXqTWx/26G99LPjtNI9FY+Ax s5JOQ/LJQatTC6oM11424/ff+b4hvkwea6VEwQyXL8SsAgMKnuEhhJ5/YytE24Vb169Upx OJOD2oEOZ26U2+XtZIJqGJ0aubcZh1M= Date: Wed, 30 Aug 2023 15:20:27 +0800 MIME-Version: 1.0 Subject: Re: [PATCH 08/12] hugetlb: batch freeing of vmemmap pages To: Mike Kravetz Cc: Muchun Song , Joao Martins , Oscar Salvador , David Hildenbrand , Miaohe Lin , David Rientjes , Anshuman Khandual , Naoya Horiguchi , Barry Song , Michal Hocko , Matthew Wilcox , Xiongchun Duan , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org References: <20230825190436.55045-1-mike.kravetz@oracle.com> <20230825190436.55045-9-mike.kravetz@oracle.com> X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Muchun Song In-Reply-To: <20230825190436.55045-9-mike.kravetz@oracle.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Migadu-Flow: FLOW_OUT X-Rspamd-Queue-Id: 99379A0020 X-Rspam-User: X-Stat-Signature: aqhugm8cnzcbqyguaao3f9hdcgtjsrwq X-Rspamd-Server: rspam03 X-HE-Tag: 1693380043-683685 X-HE-Meta: U2FsdGVkX188HSNDzCAPffqNTJYW+aGqUfUDxMtzQB8BlXC/rJ+ja002MzPmemOAg2pUqsNF+bvmhozlmZNViFiyeQasl8Ta6ygEuUkxdhUeYRhSkrWDqCFi6DLoixwALz23DgqKOuJnJEUbdMAgCOJT/gzIxcRr74M6li4Te6nfqOYVkvCODNwv/7Ld6DIbwYDIrmgnH6AOGsQsYUu6YoXlFwQ58SVCQcdkvnk1AmZ49rla7skcb+ljfaTG7IOF0ZyEKBbMb8YarpIwwDtuzDKrRPE7AIQkQKnzpRB4yWfFo0h4/NSosBa/3KpI+gwIsVBfKyxn6tuRdnDY6to1P5d4J+2550Tcnx6PvvQoQbIb2KL5Sn0oQDK8Qr98qpNknoFifKCIkyJVQkN27dLdaXVXqDQxwIMlDznR97Rt1UYLctWA6JAd64M6PlL+FAj8K6klHiw4fkzjn3ShL3klH2dzwmy6RjlrQEN3KdkCh9uiM0RWYci93IiAwDLOIeIqXBy+s/Nc7DEey0aUyexQcV6dkWkeqqFw+wn1ithL38ggvz7B4vffvMjl5XDjmhKpCoE5saaAqwNv+T+Kjq2px76kKFBQivBogW5fJ5e7Xp7SXNQ+4Axqf4rP7/jYTvut7hSqGDCj02zHCNETmPPpdOGy+G7Oe9G+WrOM88RpRVYwxdyuzRc5Hv2Xs/DXfTLJD3cpWyyp1KbLEgjqxmC3IspibFXUkkloPQ9B0CjOtFHxnpMlipOhWvMNcdahiF6kyPNyIyhKXAYdCId23Uko++pNk/cXC244BuMFQDtEZE2fKW7/1I+11Yzx/Lgo4i/w1bMK8S4ELpHFJdoLx0l5Ewjti1t9SyDOMvk/KeYaDMsCI6EnmRK2Nkf0+hTWCb/gz1/jUCWAfhuiFM9XZTfy/bsnU+GsWNebUkU6gpm5bi7+71uNBEUM64S7Z9dF6SeyfeRRXcQPJhkEGpN7r71 IH6JuOwQ iMHeV4FQWJWIxXrCEhb6EzBlu2Jxfymc34bgiu31KkAfuSzBvjWZ5CkJIQjQ5lu+cdA/LWB2snpGe6ef1zfQWi7LZCFiodiAZK2xAqN+8h+M+zYy33pCcWOQGQg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 2023/8/26 03:04, Mike Kravetz wrote: > Now that batching of hugetlb vmemmap optimization processing is possible, > batch the freeing of vmemmap pages. When freeing vmemmap pages for a > hugetlb page, we add them to a list that is freed after the entire batch > has been processed. > > This enhances the ability to return contiguous ranges of memory to the > low level allocators. > > Signed-off-by: Mike Kravetz > --- > mm/hugetlb_vmemmap.c | 56 ++++++++++++++++++++++++++++++++------------ > 1 file changed, 41 insertions(+), 15 deletions(-) > > diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c > index d5e6b6c76dce..e390170c0887 100644 > --- a/mm/hugetlb_vmemmap.c > +++ b/mm/hugetlb_vmemmap.c > @@ -305,11 +305,14 @@ static void vmemmap_restore_pte(pte_t *pte, unsigned long addr, > * @end: end address of the vmemmap virtual address range that we want to > * remap. > * @reuse: reuse address. > + * @bulk_pages: list to deposit vmemmap pages to be freed in bulk operations > + * or NULL in non-bulk case; I'd like to rename bulk_pages to vmemmap_pages. Always add the vmemmap pages to this list and let the caller (hugetlb_vmemmap_optimize and hugetlb_vmemmap_optimize_folios) to help us to free them. It will be clear to me. > * > * Return: %0 on success, negative error code otherwise. > */ > static int vmemmap_remap_free(unsigned long start, unsigned long end, > - unsigned long reuse) > + unsigned long reuse, > + struct list_head *bulk_pages) > { > int ret; > LIST_HEAD(vmemmap_pages); > @@ -372,7 +375,14 @@ static int vmemmap_remap_free(unsigned long start, unsigned long end, > } > mmap_read_unlock(&init_mm); > > - free_vmemmap_page_list(&vmemmap_pages); > + /* > + * if performing bulk operation, do not free pages here. > + * rather add them to the bulk list > + */ > + if (!bulk_pages) > + free_vmemmap_page_list(&vmemmap_pages); > + else > + list_splice(&vmemmap_pages, bulk_pages); Here, always add vmemmap_pages to the list. > > return ret; > } > @@ -546,17 +556,9 @@ static bool vmemmap_should_optimize(const struct hstate *h, const struct page *h > return true; > } > > -/** > - * hugetlb_vmemmap_optimize - optimize @head page's vmemmap pages. > - * @h: struct hstate. > - * @head: the head page whose vmemmap pages will be optimized. > - * > - * This function only tries to optimize @head's vmemmap pages and does not > - * guarantee that the optimization will succeed after it returns. The caller > - * can use HPageVmemmapOptimized(@head) to detect if @head's vmemmap pages > - * have been optimized. > - */ > -void hugetlb_vmemmap_optimize(const struct hstate *h, struct page *head) > +static void __hugetlb_vmemmap_optimize(const struct hstate *h, > + struct page *head, > + struct list_head *bulk_pages) Also struct list_head *vmemmap_pages. > { > unsigned long vmemmap_start = (unsigned long)head, vmemmap_end; > unsigned long vmemmap_reuse; > @@ -575,18 +577,42 @@ void hugetlb_vmemmap_optimize(const struct hstate *h, struct page *head) > * to the page which @vmemmap_reuse is mapped to, then free the pages > * which the range [@vmemmap_start, @vmemmap_end] is mapped to. > */ > - if (vmemmap_remap_free(vmemmap_start, vmemmap_end, vmemmap_reuse)) > + if (vmemmap_remap_free(vmemmap_start, vmemmap_end, vmemmap_reuse, bulk_pages)) > static_branch_dec(&hugetlb_optimize_vmemmap_key); > else > SetHPageVmemmapOptimized(head); > } > > +/** > + * hugetlb_vmemmap_optimize - optimize @head page's vmemmap pages. > + * @h: struct hstate. > + * @head: the head page whose vmemmap pages will be optimized. > + * > + * This function only tries to optimize @head's vmemmap pages and does not > + * guarantee that the optimization will succeed after it returns. The caller > + * can use HPageVmemmapOptimized(@head) to detect if @head's vmemmap pages > + * have been optimized. > + */ > +void hugetlb_vmemmap_optimize(const struct hstate *h, struct page *head) > +{ > + __hugetlb_vmemmap_optimize(h, head, NULL); Use free_vmemmap_page_list to free vmemmap pages here. > +} > + > +void hugetlb_vmemmap_optimize_bulk(const struct hstate *h, struct page *head, > + struct list_head *bulk_pages) > +{ > + __hugetlb_vmemmap_optimize(h, head, bulk_pages); > +} > + > void hugetlb_vmemmap_optimize_folios(struct hstate *h, struct list_head *folio_list) > { > struct folio *folio; > + LIST_HEAD(vmemmap_pages); > > list_for_each_entry(folio, folio_list, lru) > - hugetlb_vmemmap_optimize(h, &folio->page); > + hugetlb_vmemmap_optimize_bulk(h, &folio->page, &vmemmap_pages); Directly use __hugetlb_vmemmap_optimize and delete hugetlb_vmemmap_optimize_bulk. In the future, we could rename hugetlb_vmemmap_optimize to hugetlb_vmemmap_optimize_folio, then, both function names are more consistent. E.g.   1) hugetlb_vmemmap_optimize_folio(): used to free one folio's vmemmap pages.   2) hugetlb_vmemmap_optimize_folios(): used to free multiple folio's vmemmap pages. Thanks. > + > + free_vmemmap_page_list(&vmemmap_pages); > } > > static struct ctl_table hugetlb_vmemmap_sysctls[] = {