From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 53018C02198 for ; Tue, 18 Feb 2025 09:22:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DD037280109; Tue, 18 Feb 2025 04:22:56 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D80E7280102; Tue, 18 Feb 2025 04:22:56 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C48B0280109; Tue, 18 Feb 2025 04:22:56 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id A6508280102 for ; Tue, 18 Feb 2025 04:22:56 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 2E243C0BB3 for ; Tue, 18 Feb 2025 09:22:56 +0000 (UTC) X-FDA: 83132525952.09.BEA330C Received: from m16.mail.126.com (m16.mail.126.com [117.135.210.6]) by imf12.hostedemail.com (Postfix) with ESMTP id 83CBA4000C for ; Tue, 18 Feb 2025 09:22:53 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=126.com header.s=s110527 header.b=TTGpfDfC; dmarc=pass (policy=none) header.from=126.com; spf=pass (imf12.hostedemail.com: domain of yangge1116@126.com designates 117.135.210.6 as permitted sender) smtp.mailfrom=yangge1116@126.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739870574; a=rsa-sha256; cv=none; b=QwGWULe72Op88pVd8v4f8cm/6I9AJ5DH7ez5nEWOhm3Rrj2ikxS3Q59qzr50TVBwW/3x/W YlBLo8NVOVnKHQCpPBhipVfWC6Dc5fIe1e0ndqXzLiKEXLJp0hXRT+gjH1/lTAOS6gZOFi pubF80jM1tsXmOTt4f3IbsEvMA07c/Y= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=126.com header.s=s110527 header.b=TTGpfDfC; dmarc=pass (policy=none) header.from=126.com; spf=pass (imf12.hostedemail.com: domain of yangge1116@126.com designates 117.135.210.6 as permitted sender) smtp.mailfrom=yangge1116@126.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739870574; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=q0FKelqD4MBXC07LZw1lv1trJGhq02WD8AUCXzMNScg=; b=SYYmpx5yoIyBZ68zpwtzs9j4lRuHZh43msqMLKlo2yoyjg5F96EVpWteXidYCPiWbhpCCk OFExWRLN2+l+yMzwjGDn4ahldwgI0VqjWuFpEt+6MVyAZX8AMjAN7jgZiDA9cXL0zgWDtL 6g4eHFKIAmN0+uG1QZGEeA1dzaHyDXU= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=126.com; s=s110527; h=Message-ID:Date:MIME-Version:Subject:From: Content-Type; bh=q0FKelqD4MBXC07LZw1lv1trJGhq02WD8AUCXzMNScg=; b=TTGpfDfC9SEr1KevxG+T+nBjD3R/STYOH9ZyeaAKJyA4BQ/t1cFHFCWCYOto0d Ss0Nl2zD01rF7zBdeIu4U0rz82bryM6P9goh6Y963VcgeSIPqxA2rGCFDQ7Kspqs KhpVKvNXphKYc1lWTk4fF94vA3Gn0hGmlad84Nodt0hAg= Received: from [172.19.20.199] (unknown []) by gzga-smtp-mtada-g0-0 (Coremail) with SMTP id _____wDnl6toUbRn6etmBA--.9439S2; Tue, 18 Feb 2025 17:22:48 +0800 (CST) Message-ID: <406c6713-356b-4acf-bcd0-e5a6c1e9adcf@126.com> Date: Tue, 18 Feb 2025 17:22:47 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH] mm/hugetlb: wait for hugepage folios to be freed To: David Hildenbrand , akpm@linux-foundation.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, stable@vger.kernel.org, 21cnbao@gmail.com, baolin.wang@linux.alibaba.com, muchun.song@linux.dev, osalvador@suse.de, liuzixing@hygon.cn References: <1739514729-21265-1-git-send-email-yangge1116@126.com> <37363b17-88b0-4ccc-a115-8c9f1d83a1b5@redhat.com> <2d0b01c5-a736-41d5-a0f7-db0da065d049@redhat.com> From: Ge Yang In-Reply-To: <2d0b01c5-a736-41d5-a0f7-db0da065d049@redhat.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-CM-TRANSID:_____wDnl6toUbRn6etmBA--.9439S2 X-Coremail-Antispam: 1Uf129KBjvJXoWxXw17Gry8tFWfXFWruFW3ZFb_yoW5Ww48pF W3Ca17G3yDJr9ayrnFqws09r10krWjqFWxWF1aqry3CFs8ArnrKF42ywn8uFW5Zr10ka10 qrWYvwnruF1UZa7anT9S1TB71UUUUU7qnTZGkaVYY2UrUUUUjbIjqfuFe4nvWSU5nxnvy2 9KBjDUYxBIdaVFxhVjvjDU0xZFpf9x07jUo7tUUUUU= X-Originating-IP: [112.64.138.194] X-CM-SenderInfo: 51dqwwjhrrila6rslhhfrp/1tbiOgD3G2e0Sbh-YAAAsr X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 83CBA4000C X-Stat-Signature: rjh5cen7j9hjs955p1ixbddmi6m8o4k7 X-Rspam-User: X-HE-Tag: 1739870573-216351 X-HE-Meta: U2FsdGVkX18+PyoHtoado8tfJBFXdYyv6o02NGO+m+m1lhwcFgfjZIbf/3E3cGkDhmIRGZ6P5uTCna1wceOB7WIxyB/MlFgVvfwFJ5hlPO454E20Xz0EnHgQDldfpbD9GQW5IDWFihJNuVDy8J/4VZv7KF7iACd7YT4Dlu1TVakkwSY15hQc/FcoMtzbT7nfOllq9XM6Ze5xCQjFVuGF5z5b8X9pFdGEmXaRdLf6swUYhR7ZUbIeRETCTKfoj9jNXD/mcA6WO11B17eEHa+9Tl2lrs99IPcFHPzVeRlJu/HrpxtuHdazlZStIvOlJ9/gczlLCEnobbbhnf+zLmLftzpeAOMq3djRsZGbZxy83Z/I8aPaJM1SfAxho4HaT+ClT7g+Ss03UzEg75EZMxUrlh7oasEPrNNblUTG62HWv47AjZLJurM/xK4p2Yyw4BHVc9VlP/0JxkOUfKbCoejs+eZu5F1Z12eAK8kcWX8PbF2ojh8DbOeDSL6QIjNal7M0jHWoD+PwPSqLytROF4ahxNUYZgPJwEh5DZo8rvX6utrU4Fg+1pbPmgaY4u8uoyA6G3FnkvNW0SMbeqIdMcsDoZZwCkQpnwqYYQM1ZCGL95qkXnk403eoazY/BMMnzNH4M4+I7xhNftDNGGSxafsyH2JZ+kTeiXXWlPRutN9k1KZJXBqhjCOuC5yNOhrrEGva0w+dQmhvdzaIoKTMCVrz0mQ+6Ydk35bTbJBISVwncYSOiHU0/SYYqVM/NdM6w4BYE4kMWXJskJBl08q+JlQ8NomvjxfNhZ76vMaJYWI18P5AZpyonuVui1FcvW6Aq0GDioQ2o3G6erlxo2W6BqbGW03gu6QMa4TtuNW2K6r1LpABS77Cm/rFP3TXNDevUyx6FSSWy/EJShuJso28SnYJsM75DlxzoYxKF4bcE85FsM9yLmaH13uKP/5b4DoU4OpletM8oLagVLvaGUZ0qiD dN9lY2o/ 8xVg2U7Y+VXL+D1SdAT7QGEyHwWbDiKbPSyoHAKQB+DvvpqwrdiBowdkRuyAm+iM9LVw5jEs7YL0pucX0JMv3CQ3SPMwKmqdsRGi0PYqYtvlx+VuNLU36/BxnMDtmB94OucINIYnVrPu6gmZkcSzx5SmN0Wm4L8BnCjAzCK3J+CChnR91ae5gyabUHXX/AFQnozMZ5J/lkgmELpy9WrTHiI6v3iRFypEyqHUm4xbQfy1GI03GE1rH5NsTYThBNYbo/RqBc8eLMY7XYV7dnW3AGW4VSs/++BtJg+MxAj0a+tFKJc1SVc8ac8Gw9BsmCRxkVISbt+RObG5DtkTKmrGvlGNw/8W+PI4lTTrChXsf/1H4nZUU6JwmkJv9B+SjLQPDxlLY3EjCSUYJmXE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.048435, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: 在 2025/2/18 16:55, David Hildenbrand 写道: > On 15.02.25 06:50, Ge Yang wrote: >> >> >> 在 2025/2/14 16:08, David Hildenbrand 写道: >>> On 14.02.25 07:32, yangge1116@126.com wrote: >>>> From: Ge Yang >>>> >>>> Since the introduction of commit b65d4adbc0f0 ("mm: hugetlb: defer >>>> freeing >>>> of HugeTLB pages"), which supports deferring the freeing of HugeTLB >>>> pages, >>>> the allocation of contiguous memory through cma_alloc() may fail >>>> probabilistically. >>>> >>>> In the CMA allocation process, if it is found that the CMA area is >>>> occupied >>>> by in-use hugepage folios, these in-use hugepage folios need to be >>>> migrated >>>> to another location. When there are no available hugepage folios in the >>>> free HugeTLB pool during the migration of in-use HugeTLB pages, new >>>> folios >>>> are allocated from the buddy system. A temporary state is set on the >>>> newly >>>> allocated folio. Upon completion of the hugepage folio migration, the >>>> temporary state is transferred from the new folios to the old folios. >>>> Normally, when the old folios with the temporary state are freed, it is >>>> directly released back to the buddy system. However, due to the >>>> deferred >>>> freeing of HugeTLB pages, the PageBuddy() check fails, ultimately >>>> leading >>>> to the failure of cma_alloc(). >>>> >>>> Here is a simplified call trace illustrating the process: >>>> cma_alloc() >>>>       ->__alloc_contig_migrate_range() // Migrate in-use hugepage >>>>           ->unmap_and_move_huge_page() >>>>               ->folio_putback_hugetlb() // Free old folios >>>>       ->test_pages_isolated() >>>>           ->__test_page_isolated_in_pageblock() >>>>                ->PageBuddy(page) // Check if the page is in buddy >>>> >>>> To resolve this issue, we have implemented a function named >>>> wait_for_hugepage_folios_freed(). This function ensures that the >>>> hugepage >>>> folios are properly released back to the buddy system after their >>>> migration >>>> is completed. By invoking wait_for_hugepage_folios_freed() following >>>> the >>>> migration process, we guarantee that when test_pages_isolated() is >>>> executed, it will successfully pass. >>> >>> Okay, so after every successful migration -> put of src, we wait for the >>> src to actually get freed. >>> >>> When migrating multiple hugetlb folios, we'd wait once per folio. >>> >>> It reminds me a bit about pcp caches, where folios are !buddy until the >>> pcp was drained. >>> >> It seems that we only track unmovable, reclaimable, and movable pages on >> the pcp lists. For specific details, please refer to the >> free_frozen_pages() function. > > It reminded me about PCP caches, because we effectively also have to > wait for some stuck folios to properly get freed to the buddy. > It seems that when an isolated page is freed, it won't be placed back into the PCP caches.