From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F0E7C83F05 for ; Fri, 4 Jul 2025 06:44:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1CEC26B8012; Fri, 4 Jul 2025 02:44:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 17F746B800A; Fri, 4 Jul 2025 02:44:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 046E36B8012; Fri, 4 Jul 2025 02:44:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id E28536B800A for ; Fri, 4 Jul 2025 02:44:03 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 87B00141F05 for ; Fri, 4 Jul 2025 06:44:03 +0000 (UTC) X-FDA: 83625642366.01.1C1BC0C Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf25.hostedemail.com (Postfix) with ESMTP id 051DEA000D for ; Fri, 4 Jul 2025 06:44:00 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=RWpK2Xit; dmarc=pass (policy=quarantine) header.from=redhat.com; spf=pass (imf25.hostedemail.com: domain of mpenttil@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=mpenttil@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1751611441; a=rsa-sha256; cv=none; b=oO/SmRGPlmXgkFhQTG3iZHEirIp4C0ibVb9tvRmEpJxyRVSmkKWcEIAqLTid7LCavEcdKZ m25YjVM+S4oT8FoybKMq5YGWE6A9LWb+r67HaF5Kix/ucEDskKvBBjPt0IXmuY+WV9874E fJles9Qu12pf5L9U0bxybWLUDfKvbFo= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=RWpK2Xit; dmarc=pass (policy=quarantine) header.from=redhat.com; spf=pass (imf25.hostedemail.com: domain of mpenttil@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=mpenttil@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1751611441; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=FwOdDxBLxTkdXVb361ZyrH0Sjt/XQOspSFQ+QVGV5NE=; b=pGQJ1kOgdojsUMtnX7Zck+QmFGvUXmOZAyz+sXWITJOgovNsrcJIKJPEASG7hpEJjon8s4 1MZwceWG4a/rhhzOLJ/DSusGPlkjkDe5GH+1lUV3ZKHp6udPIeq1WZFcXb4Yx0YPDwqAfE fmIjaxsB7JfKOqjTYtZFClZC2GeghyI= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1751611440; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FwOdDxBLxTkdXVb361ZyrH0Sjt/XQOspSFQ+QVGV5NE=; b=RWpK2Xit7b+wZzFjf85Az/MUGWFCHUchH98YefZFXMrUW490SfsMpCExrYYogl4EOtJaTy hxUveN4TbgqJ6uYq8Kuz1QDfTIUzTUhY5aUSM1oE3e4HFjcIJHde+2+e2n0rtxOzLdkOay cKZoSAZAU1Gz9WkwkZK5vR3rLzOc3kQ= Received: from mail-lj1-f198.google.com (mail-lj1-f198.google.com [209.85.208.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-660-wuxLMgs1OcWGt0YEG7Ph7A-1; Fri, 04 Jul 2025 02:43:58 -0400 X-MC-Unique: wuxLMgs1OcWGt0YEG7Ph7A-1 X-Mimecast-MFC-AGG-ID: wuxLMgs1OcWGt0YEG7Ph7A_1751611437 Received: by mail-lj1-f198.google.com with SMTP id 38308e7fff4ca-32cf580cacdso2735871fa.2 for ; Thu, 03 Jul 2025 23:43:58 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1751611437; x=1752216237; h=content-transfer-encoding:in-reply-to:content-language:references :cc:to:from:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=FwOdDxBLxTkdXVb361ZyrH0Sjt/XQOspSFQ+QVGV5NE=; b=PdXQD9h3cqGumwx2H3ttIq+Cp3bFFOVjtx3Cj3IetnG2Cy5WDQOSS8Nm5H9QI9PKef 2aPhZ3Z91Q2o7xEHjczv1MTFcvOc1LjBDK338vCB0isHf0fBSbvgJTWs/Joe7X5OYH3A ZuIxlXH5CIEokFVXzmcLWyHTGsZA4qpqHaD5jsEidYp8Sn3MlcnSFa5c9qxkQWEZJRVo ena2k++tcI/6OmOsVRoDXBbgROoeq2XwThGUPNoeGeBuAaxl4wHDgzoFMwzoTYjzMdya +FnlhgCrKGS0gNd7oglCmvSBJbBTtd4jzzohU57a1pJNmJQXPYXLvAMxN1ZAU3xTYW+b 47Kw== X-Forwarded-Encrypted: i=1; AJvYcCU8Jh7VTrTb5b9i5KVGyg4HkDOAf9NluvlKxG9msdJuqy/kvnar71aIq+N0wQftGZ9BGp4UVZJ9Mg==@kvack.org X-Gm-Message-State: AOJu0YzxjXpFiW1QZCla7hPb3kVw1651h8MNWa4AcuoMODoFtE4jai3c DBXN76V74WE6cTNDv3EGgXGJyr0aM37ZCRzELuotXNmQ79V8CU5CoEnqEaf8UJcYZhKJ3b29eaJ eTA/V7qBmT+zbJA0FDcnLaKlZfcdnmE1/+wNV/Ocag5YYmEMb9Ek= X-Gm-Gg: ASbGncvkEvi7RlC30/zTOPt5piPQVvurG83hkZKLG6skOOfMK2xCDG8diHT03ipi183 t/JD4PRScCCSyDLm4rC7hpqeLIzvRECYO5A9w1UI7I+Rm5jrU8Pzy2UjAhaQgZCOgGnPxRXmkSE V53xzlM/ipU8kBJQy8FBjNMo0eUU0P1H42EzL5kKDTXIuxbkvpD3ZqU48GEL2kjEDz1SkEkZ0Uz BYH3dUd6ujV+OLM1C3+BKDL1aLfwQS5fLrMPpRf3w09721LOq9c/ewn8RNywhmImAY7Xf8cWAgj CgQPFRVMlHo5aXWLLjFzMdMXJVoM1BYKFdVdYRW3QGffjTTo X-Received: by 2002:a05:651c:54a:b0:32a:91e6:1a26 with SMTP id 38308e7fff4ca-32f00c88f41mr2072621fa.3.1751611436686; Thu, 03 Jul 2025 23:43:56 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEdk2Gj3zw1YTSxh2xHL0CZ6nzXzE/Ymy4koaTE+MilL9ahfaFXFYUVtE4a3bLBSjapV9Ydhg== X-Received: by 2002:a05:651c:54a:b0:32a:91e6:1a26 with SMTP id 38308e7fff4ca-32f00c88f41mr2072521fa.3.1751611436164; Thu, 03 Jul 2025 23:43:56 -0700 (PDT) Received: from [192.168.1.86] (85-23-48-6.bb.dnainternet.fi. [85.23.48.6]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-32e1b1213dfsm1332731fa.73.2025.07.03.23.43.55 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 03 Jul 2025 23:43:55 -0700 (PDT) Message-ID: <715fc271-1af3-4061-b217-e3d6e32849c6@redhat.com> Date: Fri, 4 Jul 2025 09:43:54 +0300 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [v1 resend 08/12] mm/thp: add split during migration support From: =?UTF-8?Q?Mika_Penttil=C3=A4?= To: Balbir Singh , linux-mm@kvack.org Cc: akpm@linux-foundation.org, linux-kernel@vger.kernel.org, Karol Herbst , Lyude Paul , Danilo Krummrich , David Airlie , Simona Vetter , =?UTF-8?B?SsOpcsO0bWUgR2xpc3Nl?= , Shuah Khan , David Hildenbrand , Barry Song , Baolin Wang , Ryan Roberts , Matthew Wilcox , Peter Xu , Zi Yan , Kefeng Wang , Jane Chu , Alistair Popple , Donet Tom References: <20250703233511.2028395-1-balbirs@nvidia.com> <20250703233511.2028395-9-balbirs@nvidia.com> In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 0APIvgNRdWLK1zgQZw4ksFcvgzF-EceLMldwQCZ5ZAI_1751611437 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 051DEA000D X-Stat-Signature: g63bfi5xeeidcmdbjijxi7zqhqiskxc4 X-Rspam-User: X-HE-Tag: 1751611440-32588 X-HE-Meta: U2FsdGVkX1+iHjMttiGvFJ29StkSgF9fWDOpzXsXHCWBtnsMoGNMOKi3mEHhzteLgeYWjuKSUfWpmbZVvEjdwfY4kB+9jiRhXVQHfBzfCM5qnH6ul/UVZ2k9kQ7WEbgPrmcZWBou2jj2CTCOatBJH5F9bHsdppcG/Ch9g5swJjtfFlISA5XIKG6s2ZlOxZTj1J2WaDZ88d3e5/vEvT5yk7OFgUD6P4BG0Ho8rDEl96hSApW4j9J7QG1QAhyF2D3XYQ/BieAD+KAunq83Hg8Mv+nSr8PHqOwBjFbyrd2gFC05RS+NFS/L6JIaQAJqBlDlkKuQeXJ7C2Tn5cadb/ew6jK1xVUax6+/WIAz0Q/SHBZblxuKXwFwwoMjoesjiTyQseOuABYCZHnTIozAYpfokbjO9On3VVtrjOO1gI77TMBFYTPg2hNCPhUbJhphhUUwqsJGlrNnl4ubiTj3TjKyvSMjgcRdLaGFcKJZ3CHw19Lkiw68gnm2kgV7hh16uJaD43HJvn0gqnVhcgSGsoe20j+vISv/4msljmWXJFtpcSr3dNauCRd4U6ymUEK1hCLQ2obJf39EBO9XlVb2VeaueSiDOKPm4n9wYe6Il+xtEGkSOKGonoBfizXODlSb3wjz1v3sujBHrOnliVWE5CzSP7Ulmo+bhC3mU+cpedM1PIZ+yW+/mTwmUsAJ1dl5lj/U/wtS0yQx4/v+DdICEft2SGWezINdiX572Az4/T6gwt4VU4zq+kBfdzJMRuxV1AfkeC8IXRahWjxv/i7rTXTAJkQh6vWaMOf6sl8g6ttqOgrTIiDBRIsmevb17GqayAJ9NxY6Ukd00a9vDQ23LdlNVbYTgHB07Zdx3150JH+L66yX2nIz0XtE+3kSL0zmNZqGID0w+G1W+Fi/u3cEIhurpF/5Y+OuUiM5zkhXPGv4qmn2/WkanXHSjj2oQsDMvvw9vb/bo8U8lOk/yFoKuDL RcLEo+4l V4gY9hI5l0Zx8oyddDf987aDxttojiIQ/Ljg1yntP/cU2ljNBDpnanXBpPzGRuYqTcdffxvitjKRQuw8pPdzA4l9qWyczQAZC86RNoniyOYAllLRu4KjFLTDMd1xtzNWsSkPjL/VRPjfERrySETa8tZVfTUwmxcWBP9K7F7uYyv7MOAK7TUssDVXI/jAlO7xz0c+3hLrrJjFPdhY6dmgzHusITmo35Z1bxmApUjhE0PlQU/swLQyEFoxkmPv4Zn6Y7MdYR77AdWIQkssMuxM8QLBewTFeHoBek53bGkD04/DkiVrDkmyslh+vpxlrqlHBnmPRFcoyNmfmfajxK+92kiMkxSrz/+cHAb7elC1SZbhp85GZZvXlsG0pvqnjO9VMELfjsBKJFM8GB4c7vvg+IvKXhGDHrjz6rqo9UQnUZeijxxJitZy+1o2x7m76+UOhYHoFzylvN2KkpBtibKXRWyKGZbzUPoTi+VwU+CXHg6Vdaqn2zvFijwqTooyHlEfGtMq4qdV9tHb6N+gxQI6of4awIlfjkiTuNtPzNzc5mjrmEtI5LZppzvhjqTESChB/VGYRHeJiikMA5w7Kqguflqw2xKe8K5078uzhWi5whnTCYoxstazfrShQuFcDARGCUauFG47KKfp3YcumFfOGbVjGtd3Cv1a7LZgjVnBE+7hHvl0S77R3uAAK3vL71y5dbpz4Iah3lvH/gNsSAeCqk2mHFBcL2uz2gH2aK2hKz2q3yn4lFUj4fumpBKbAhL4Akb9AzR6yEGJAOYk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 7/4/25 08:17, Mika Penttilä wrote: > On 7/4/25 02:35, Balbir Singh wrote: >> Support splitting pages during THP zone device migration as needed. >> The common case that arises is that after setup, during migrate >> the destination might not be able to allocate MIGRATE_PFN_COMPOUND >> pages. >> >> Add a new routine migrate_vma_split_pages() to support the splitting >> of already isolated pages. The pages being migrated are already unmapped >> and marked for migration during setup (via unmap). folio_split() and >> __split_unmapped_folio() take additional isolated arguments, to avoid >> unmapping and remaping these pages and unlocking/putting the folio. >> >> Cc: Karol Herbst >> Cc: Lyude Paul >> Cc: Danilo Krummrich >> Cc: David Airlie >> Cc: Simona Vetter >> Cc: "Jérôme Glisse" >> Cc: Shuah Khan >> Cc: David Hildenbrand >> Cc: Barry Song >> Cc: Baolin Wang >> Cc: Ryan Roberts >> Cc: Matthew Wilcox >> Cc: Peter Xu >> Cc: Zi Yan >> Cc: Kefeng Wang >> Cc: Jane Chu >> Cc: Alistair Popple >> Cc: Donet Tom >> >> Signed-off-by: Balbir Singh >> --- >> include/linux/huge_mm.h | 11 ++++++-- >> mm/huge_memory.c | 54 ++++++++++++++++++++----------------- >> mm/migrate_device.c | 59 ++++++++++++++++++++++++++++++++--------- >> 3 files changed, 85 insertions(+), 39 deletions(-) >> >> diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h >> index 65a1bdf29bb9..5f55a754e57c 100644 >> --- a/include/linux/huge_mm.h >> +++ b/include/linux/huge_mm.h >> @@ -343,8 +343,8 @@ unsigned long thp_get_unmapped_area_vmflags(struct file *filp, unsigned long add >> vm_flags_t vm_flags); >> >> bool can_split_folio(struct folio *folio, int caller_pins, int *pextra_pins); >> -int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, >> - unsigned int new_order); >> +int __split_huge_page_to_list_to_order(struct page *page, struct list_head *list, >> + unsigned int new_order, bool isolated); >> int min_order_for_split(struct folio *folio); >> int split_folio_to_list(struct folio *folio, struct list_head *list); >> bool uniform_split_supported(struct folio *folio, unsigned int new_order, >> @@ -353,6 +353,13 @@ bool non_uniform_split_supported(struct folio *folio, unsigned int new_order, >> bool warns); >> int folio_split(struct folio *folio, unsigned int new_order, struct page *page, >> struct list_head *list); >> + >> +static inline int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, >> + unsigned int new_order) >> +{ >> + return __split_huge_page_to_list_to_order(page, list, new_order, false); >> +} >> + >> /* >> * try_folio_split - try to split a @folio at @page using non uniform split. >> * @folio: folio to be split >> diff --git a/mm/huge_memory.c b/mm/huge_memory.c >> index d55e36ae0c39..e00ddfed22fa 100644 >> --- a/mm/huge_memory.c >> +++ b/mm/huge_memory.c >> @@ -3424,15 +3424,6 @@ static void __split_folio_to_order(struct folio *folio, int old_order, >> new_folio->mapping = folio->mapping; >> new_folio->index = folio->index + i; >> >> - /* >> - * page->private should not be set in tail pages. Fix up and warn once >> - * if private is unexpectedly set. >> - */ >> - if (unlikely(new_folio->private)) { >> - VM_WARN_ON_ONCE_PAGE(true, new_head); >> - new_folio->private = NULL; >> - } >> - >> if (folio_test_swapcache(folio)) >> new_folio->swap.val = folio->swap.val + i; >> >> @@ -3519,7 +3510,7 @@ static int __split_unmapped_folio(struct folio *folio, int new_order, >> struct page *split_at, struct page *lock_at, >> struct list_head *list, pgoff_t end, >> struct xa_state *xas, struct address_space *mapping, >> - bool uniform_split) >> + bool uniform_split, bool isolated) >> { >> struct lruvec *lruvec; >> struct address_space *swap_cache = NULL; >> @@ -3643,8 +3634,9 @@ static int __split_unmapped_folio(struct folio *folio, int new_order, >> percpu_ref_get_many(&release->pgmap->ref, >> (1 << new_order) - 1); >> >> - lru_add_split_folio(origin_folio, release, lruvec, >> - list); >> + if (!isolated) >> + lru_add_split_folio(origin_folio, release, >> + lruvec, list); >> >> /* Some pages can be beyond EOF: drop them from cache */ >> if (release->index >= end) { >> @@ -3697,6 +3689,12 @@ static int __split_unmapped_folio(struct folio *folio, int new_order, >> if (nr_dropped) >> shmem_uncharge(mapping->host, nr_dropped); >> >> + /* >> + * Don't remap and unlock isolated folios >> + */ >> + if (isolated) >> + return ret; >> + >> remap_page(origin_folio, 1 << order, >> folio_test_anon(origin_folio) ? >> RMP_USE_SHARED_ZEROPAGE : 0); >> @@ -3790,6 +3788,7 @@ bool uniform_split_supported(struct folio *folio, unsigned int new_order, >> * @lock_at: a page within @folio to be left locked to caller >> * @list: after-split folios will be put on it if non NULL >> * @uniform_split: perform uniform split or not (non-uniform split) >> + * @isolated: The pages are already unmapped >> * >> * It calls __split_unmapped_folio() to perform uniform and non-uniform split. >> * It is in charge of checking whether the split is supported or not and >> @@ -3800,7 +3799,7 @@ bool uniform_split_supported(struct folio *folio, unsigned int new_order, >> */ >> static int __folio_split(struct folio *folio, unsigned int new_order, >> struct page *split_at, struct page *lock_at, >> - struct list_head *list, bool uniform_split) >> + struct list_head *list, bool uniform_split, bool isolated) >> { >> struct deferred_split *ds_queue = get_deferred_split_queue(folio); >> XA_STATE(xas, &folio->mapping->i_pages, folio->index); >> @@ -3846,14 +3845,16 @@ static int __folio_split(struct folio *folio, unsigned int new_order, >> * is taken to serialise against parallel split or collapse >> * operations. >> */ >> - anon_vma = folio_get_anon_vma(folio); >> - if (!anon_vma) { >> - ret = -EBUSY; >> - goto out; >> + if (!isolated) { >> + anon_vma = folio_get_anon_vma(folio); >> + if (!anon_vma) { >> + ret = -EBUSY; >> + goto out; >> + } >> + anon_vma_lock_write(anon_vma); >> } >> end = -1; >> mapping = NULL; >> - anon_vma_lock_write(anon_vma); >> } else { >> unsigned int min_order; >> gfp_t gfp; >> @@ -3920,7 +3921,8 @@ static int __folio_split(struct folio *folio, unsigned int new_order, >> goto out_unlock; >> } >> >> - unmap_folio(folio); >> + if (!isolated) >> + unmap_folio(folio); >> >> /* block interrupt reentry in xa_lock and spinlock */ >> local_irq_disable(); >> @@ -3973,14 +3975,15 @@ static int __folio_split(struct folio *folio, unsigned int new_order, >> >> ret = __split_unmapped_folio(folio, new_order, >> split_at, lock_at, list, end, &xas, mapping, >> - uniform_split); >> + uniform_split, isolated); >> } else { >> spin_unlock(&ds_queue->split_queue_lock); >> fail: >> if (mapping) >> xas_unlock(&xas); >> local_irq_enable(); >> - remap_page(folio, folio_nr_pages(folio), 0); >> + if (!isolated) >> + remap_page(folio, folio_nr_pages(folio), 0); >> ret = -EAGAIN; >> } >> >> @@ -4046,12 +4049,13 @@ static int __folio_split(struct folio *folio, unsigned int new_order, >> * Returns -EINVAL when trying to split to an order that is incompatible >> * with the folio. Splitting to order 0 is compatible with all folios. >> */ >> -int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, >> - unsigned int new_order) >> +int __split_huge_page_to_list_to_order(struct page *page, struct list_head *list, >> + unsigned int new_order, bool isolated) >> { >> struct folio *folio = page_folio(page); >> >> - return __folio_split(folio, new_order, &folio->page, page, list, true); >> + return __folio_split(folio, new_order, &folio->page, page, list, true, >> + isolated); >> } >> >> /* >> @@ -4080,7 +4084,7 @@ int folio_split(struct folio *folio, unsigned int new_order, >> struct page *split_at, struct list_head *list) >> { >> return __folio_split(folio, new_order, split_at, &folio->page, list, >> - false); >> + false, false); >> } >> >> int min_order_for_split(struct folio *folio) >> diff --git a/mm/migrate_device.c b/mm/migrate_device.c >> index 41d0bd787969..acd2f03b178d 100644 >> --- a/mm/migrate_device.c >> +++ b/mm/migrate_device.c >> @@ -813,6 +813,24 @@ static int migrate_vma_insert_huge_pmd_page(struct migrate_vma *migrate, >> src[i] &= ~MIGRATE_PFN_MIGRATE; >> return 0; >> } >> + >> +static void migrate_vma_split_pages(struct migrate_vma *migrate, >> + unsigned long idx, unsigned long addr, >> + struct folio *folio) >> +{ >> + unsigned long i; >> + unsigned long pfn; >> + unsigned long flags; >> + >> + folio_get(folio); >> + split_huge_pmd_address(migrate->vma, addr, true); >> + __split_huge_page_to_list_to_order(folio_page(folio, 0), NULL, 0, true); > We already have reference to folio, why is folio_get() needed ? > > Splitting the page splits pmd for anon folios, why is there split_huge_pmd_address() ? Oh I see + if (!isolated) + unmap_folio(folio); which explains the explicit split_huge_pmd_address(migrate->vma, addr, true); Still, why the folio_get(folio);? > >> + migrate->src[idx] &= ~MIGRATE_PFN_COMPOUND; >> + flags = migrate->src[idx] & ((1UL << MIGRATE_PFN_SHIFT) - 1); >> + pfn = migrate->src[idx] >> MIGRATE_PFN_SHIFT; >> + for (i = 1; i < HPAGE_PMD_NR; i++) >> + migrate->src[i+idx] = migrate_pfn(pfn + i) | flags; >> +} >> #else /* !CONFIG_ARCH_ENABLE_THP_MIGRATION */ >> static int migrate_vma_insert_huge_pmd_page(struct migrate_vma *migrate, >> unsigned long addr, >> @@ -822,6 +840,11 @@ static int migrate_vma_insert_huge_pmd_page(struct migrate_vma *migrate, >> { >> return 0; >> } >> + >> +static void migrate_vma_split_pages(struct migrate_vma *migrate, >> + unsigned long idx, unsigned long addr, >> + struct folio *folio) >> +{} >> #endif >> >> /* >> @@ -971,8 +994,9 @@ static void __migrate_device_pages(unsigned long *src_pfns, >> struct migrate_vma *migrate) >> { >> struct mmu_notifier_range range; >> - unsigned long i; >> + unsigned long i, j; >> bool notified = false; >> + unsigned long addr; >> >> for (i = 0; i < npages; ) { >> struct page *newpage = migrate_pfn_to_page(dst_pfns[i]); >> @@ -1014,12 +1038,16 @@ static void __migrate_device_pages(unsigned long *src_pfns, >> (!(dst_pfns[i] & MIGRATE_PFN_COMPOUND))) { >> nr = HPAGE_PMD_NR; >> src_pfns[i] &= ~MIGRATE_PFN_COMPOUND; >> - src_pfns[i] &= ~MIGRATE_PFN_MIGRATE; >> - goto next; >> + } else { >> + nr = 1; >> } >> >> - migrate_vma_insert_page(migrate, addr, &dst_pfns[i], >> - &src_pfns[i]); >> + for (j = 0; j < nr && i + j < npages; j++) { >> + src_pfns[i+j] |= MIGRATE_PFN_MIGRATE; >> + migrate_vma_insert_page(migrate, >> + addr + j * PAGE_SIZE, >> + &dst_pfns[i+j], &src_pfns[i+j]); >> + } >> goto next; >> } >> >> @@ -1041,7 +1069,9 @@ static void __migrate_device_pages(unsigned long *src_pfns, >> MIGRATE_PFN_COMPOUND); >> goto next; >> } >> - src_pfns[i] &= ~MIGRATE_PFN_MIGRATE; >> + nr = 1 << folio_order(folio); >> + addr = migrate->start + i * PAGE_SIZE; >> + migrate_vma_split_pages(migrate, i, addr, folio); >> } else if ((src_pfns[i] & MIGRATE_PFN_MIGRATE) && >> (dst_pfns[i] & MIGRATE_PFN_COMPOUND) && >> !(src_pfns[i] & MIGRATE_PFN_COMPOUND)) { >> @@ -1076,12 +1106,17 @@ static void __migrate_device_pages(unsigned long *src_pfns, >> BUG_ON(folio_test_writeback(folio)); >> >> if (migrate && migrate->fault_page == page) >> - extra_cnt = 1; >> - r = folio_migrate_mapping(mapping, newfolio, folio, extra_cnt); >> - if (r != MIGRATEPAGE_SUCCESS) >> - src_pfns[i] &= ~MIGRATE_PFN_MIGRATE; >> - else >> - folio_migrate_flags(newfolio, folio); >> + extra_cnt++; >> + for (j = 0; j < nr && i + j < npages; j++) { >> + folio = page_folio(migrate_pfn_to_page(src_pfns[i+j])); >> + newfolio = page_folio(migrate_pfn_to_page(dst_pfns[i+j])); >> + >> + r = folio_migrate_mapping(mapping, newfolio, folio, extra_cnt); >> + if (r != MIGRATEPAGE_SUCCESS) >> + src_pfns[i+j] &= ~MIGRATE_PFN_MIGRATE; >> + else >> + folio_migrate_flags(newfolio, folio); >> + } >> next: >> i += nr; >> }