Re: [v1 resend 08/12] mm/thp: add split during migration support

[Matthew Brost <matthew.brost@intel.com>]

On Fri, Jul 18, 2025 at 11:06:09AM -0400, Zi Yan wrote:
> On 17 Jul 2025, at 23:33, Matthew Brost wrote:
> 
> > On Thu, Jul 17, 2025 at 09:25:02PM -0400, Zi Yan wrote:
> >> On 17 Jul 2025, at 20:41, Matthew Brost wrote:
> >>
> >>> On Thu, Jul 17, 2025 at 07:04:48PM -0400, Zi Yan wrote:
> >>>> On 17 Jul 2025, at 18:24, Matthew Brost wrote:
> >>>>
> >>>>> On Thu, Jul 17, 2025 at 07:53:40AM +1000, Balbir Singh wrote:
> >>>>>> On 7/17/25 02:24, Matthew Brost wrote:
> >>>>>>> On Wed, Jul 16, 2025 at 07:19:10AM -0400, Zi Yan wrote:
> >>>>>>>> On 16 Jul 2025, at 1:34, Matthew Brost wrote:
> >>>>>>>>
> >>>>>>>>> On Sun, Jul 06, 2025 at 11:47:10AM +1000, Balbir Singh wrote:
> >>>>>>>>>> On 7/6/25 11:34, Zi Yan wrote:
> >>>>>>>>>>> On 5 Jul 2025, at 21:15, Balbir Singh wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> On 7/5/25 11:55, Zi Yan wrote:
> >>>>>>>>>>>>> On 4 Jul 2025, at 20:58, Balbir Singh wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> On 7/4/25 21:24, Zi Yan wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> s/pages/folio
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks, will make the changes
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Why name it isolated if the folio is unmapped? Isolated folios often mean
> >>>>>>>>>>>>>>> they are removed from LRU lists. isolated here causes confusion.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Ack, will change the name
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>   *
> >>>>>>>>>>>>>>>>   * It calls __split_unmapped_folio() to perform uniform and non-uniform split.
> >>>>>>>>>>>>>>>>   * It is in charge of checking whether the split is supported or not and
> >>>>>>>>>>>>>>>> @@ -3800,7 +3799,7 @@ bool uniform_split_supported(struct folio *folio, unsigned int new_order,
> >>>>>>>>>>>>>>>>   */
> >>>>>>>>>>>>>>>>  static int __folio_split(struct folio *folio, unsigned int new_order,
> >>>>>>>>>>>>>>>>  		struct page *split_at, struct page *lock_at,
> >>>>>>>>>>>>>>>> -		struct list_head *list, bool uniform_split)
> >>>>>>>>>>>>>>>> +		struct list_head *list, bool uniform_split, bool isolated)
> >>>>>>>>>>>>>>>>  {
> >>>>>>>>>>>>>>>>  	struct deferred_split *ds_queue = get_deferred_split_queue(folio);
> >>>>>>>>>>>>>>>>  	XA_STATE(xas, &folio->mapping->i_pages, folio->index);
> >>>>>>>>>>>>>>>> @@ -3846,14 +3845,16 @@ static int __folio_split(struct folio *folio, unsigned int new_order,
> >>>>>>>>>>>>>>>>  		 * is taken to serialise against parallel split or collapse
> >>>>>>>>>>>>>>>>  		 * operations.
> >>>>>>>>>>>>>>>>  		 */
> >>>>>>>>>>>>>>>> -		anon_vma = folio_get_anon_vma(folio);
> >>>>>>>>>>>>>>>> -		if (!anon_vma) {
> >>>>>>>>>>>>>>>> -			ret = -EBUSY;
> >>>>>>>>>>>>>>>> -			goto out;
> >>>>>>>>>>>>>>>> +		if (!isolated) {
> >>>>>>>>>>>>>>>> +			anon_vma = folio_get_anon_vma(folio);
> >>>>>>>>>>>>>>>> +			if (!anon_vma) {
> >>>>>>>>>>>>>>>> +				ret = -EBUSY;
> >>>>>>>>>>>>>>>> +				goto out;
> >>>>>>>>>>>>>>>> +			}
> >>>>>>>>>>>>>>>> +			anon_vma_lock_write(anon_vma);
> >>>>>>>>>>>>>>>>  		}
> >>>>>>>>>>>>>>>>  		end = -1;
> >>>>>>>>>>>>>>>>  		mapping = NULL;
> >>>>>>>>>>>>>>>> -		anon_vma_lock_write(anon_vma);
> >>>>>>>>>>>>>>>>  	} else {
> >>>>>>>>>>>>>>>>  		unsigned int min_order;
> >>>>>>>>>>>>>>>>  		gfp_t gfp;
> >>>>>>>>>>>>>>>> @@ -3920,7 +3921,8 @@ static int __folio_split(struct folio *folio, unsigned int new_order,
> >>>>>>>>>>>>>>>>  		goto out_unlock;
> >>>>>>>>>>>>>>>>  	}
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> -	unmap_folio(folio);
> >>>>>>>>>>>>>>>> +	if (!isolated)
> >>>>>>>>>>>>>>>> +		unmap_folio(folio);
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>  	/* block interrupt reentry in xa_lock and spinlock */
> >>>>>>>>>>>>>>>>  	local_irq_disable();
> >>>>>>>>>>>>>>>> @@ -3973,14 +3975,15 @@ static int __folio_split(struct folio *folio, unsigned int new_order,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>  		ret = __split_unmapped_folio(folio, new_order,
> >>>>>>>>>>>>>>>>  				split_at, lock_at, list, end, &xas, mapping,
> >>>>>>>>>>>>>>>> -				uniform_split);
> >>>>>>>>>>>>>>>> +				uniform_split, isolated);
> >>>>>>>>>>>>>>>>  	} else {
> >>>>>>>>>>>>>>>>  		spin_unlock(&ds_queue->split_queue_lock);
> >>>>>>>>>>>>>>>>  fail:
> >>>>>>>>>>>>>>>>  		if (mapping)
> >>>>>>>>>>>>>>>>  			xas_unlock(&xas);
> >>>>>>>>>>>>>>>>  		local_irq_enable();
> >>>>>>>>>>>>>>>> -		remap_page(folio, folio_nr_pages(folio), 0);
> >>>>>>>>>>>>>>>> +		if (!isolated)
> >>>>>>>>>>>>>>>> +			remap_page(folio, folio_nr_pages(folio), 0);
> >>>>>>>>>>>>>>>>  		ret = -EAGAIN;
> >>>>>>>>>>>>>>>>  	}
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> These "isolated" special handlings does not look good, I wonder if there
> >>>>>>>>>>>>>>> is a way of letting split code handle device private folios more gracefully.
> >>>>>>>>>>>>>>> It also causes confusions, since why does "isolated/unmapped" folios
> >>>>>>>>>>>>>>> not need to unmap_page(), remap_page(), or unlock?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> There are two reasons for going down the current code path
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> After thinking more, I think adding isolated/unmapped is not the right
> >>>>>>>>>>>>> way, since unmapped folio is a very generic concept. If you add it,
> >>>>>>>>>>>>> one can easily misuse the folio split code by first unmapping a folio
> >>>>>>>>>>>>> and trying to split it with unmapped = true. I do not think that is
> >>>>>>>>>>>>> supported and your patch does not prevent that from happening in the future.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> I don't understand the misuse case you mention, I assume you mean someone can
> >>>>>>>>>>>> get the usage wrong? The responsibility is on the caller to do the right thing
> >>>>>>>>>>>> if calling the API with unmapped
> >>>>>>>>>>>
> >>>>>>>>>>> Before your patch, there is no use case of splitting unmapped folios.
> >>>>>>>>>>> Your patch only adds support for device private page split, not any unmapped
> >>>>>>>>>>> folio split. So using a generic isolated/unmapped parameter is not OK.
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> There is a use for splitting unmapped folios (see below)
> >>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>> You should teach different parts of folio split code path to handle
> >>>>>>>>>>>>> device private folios properly. Details are below.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> 1. if the isolated check is not present, folio_get_anon_vma will fail and cause
> >>>>>>>>>>>>>>    the split routine to return with -EBUSY
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> You do something below instead.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> if (!anon_vma && !folio_is_device_private(folio)) {
> >>>>>>>>>>>>> 	ret = -EBUSY;
> >>>>>>>>>>>>> 	goto out;
> >>>>>>>>>>>>> } else if (anon_vma) {
> >>>>>>>>>>>>> 	anon_vma_lock_write(anon_vma);
> >>>>>>>>>>>>> }
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> folio_get_anon() cannot be called for unmapped folios. In our case the page has
> >>>>>>>>>>>> already been unmapped. Is there a reason why you mix anon_vma_lock_write with
> >>>>>>>>>>>> the check for device private folios?
> >>>>>>>>>>>
> >>>>>>>>>>> Oh, I did not notice that anon_vma = folio_get_anon_vma(folio) is also
> >>>>>>>>>>> in if (!isolated) branch. In that case, just do
> >>>>>>>>>>>
> >>>>>>>>>>> if (folio_is_device_private(folio) {
> >>>>>>>>>>> ...
> >>>>>>>>>>> } else if (is_anon) {
> >>>>>>>>>>> ...
> >>>>>>>>>>> } else {
> >>>>>>>>>>> ...
> >>>>>>>>>>> }
> >>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>> People can know device private folio split needs a special handling.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> BTW, why a device private folio can also be anonymous? Does it mean
> >>>>>>>>>>>>> if a page cache folio is migrated to device private, kernel also
> >>>>>>>>>>>>> sees it as both device private and file-backed?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> FYI: device private folios only work with anonymous private pages, hence
> >>>>>>>>>>>> the name device private.
> >>>>>>>>>>>
> >>>>>>>>>>> OK.
> >>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> 2. Going through unmap_page(), remap_page() causes a full page table walk, which
> >>>>>>>>>>>>>>    the migrate_device API has already just done as a part of the migration. The
> >>>>>>>>>>>>>>    entries under consideration are already migration entries in this case.
> >>>>>>>>>>>>>>    This is wasteful and in some case unexpected.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> unmap_folio() already adds TTU_SPLIT_HUGE_PMD to try to split
> >>>>>>>>>>>>> PMD mapping, which you did in migrate_vma_split_pages(). You probably
> >>>>>>>>>>>>> can teach either try_to_migrate() or try_to_unmap() to just split
> >>>>>>>>>>>>> device private PMD mapping. Or if that is not preferred,
> >>>>>>>>>>>>> you can simply call split_huge_pmd_address() when unmap_folio()
> >>>>>>>>>>>>> sees a device private folio.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> For remap_page(), you can simply return for device private folios
> >>>>>>>>>>>>> like it is currently doing for non anonymous folios.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> Doing a full rmap walk does not make sense with unmap_folio() and
> >>>>>>>>>>>> remap_folio(), because
> >>>>>>>>>>>>
> >>>>>>>>>>>> 1. We need to do a page table walk/rmap walk again
> >>>>>>>>>>>> 2. We'll need special handling of migration <-> migration entries
> >>>>>>>>>>>>    in the rmap handling (set/remove migration ptes)
> >>>>>>>>>>>> 3. In this context, the code is already in the middle of migration,
> >>>>>>>>>>>>    so trying to do that again does not make sense.
> >>>>>>>>>>>
> >>>>>>>>>>> Why doing split in the middle of migration? Existing split code
> >>>>>>>>>>> assumes to-be-split folios are mapped.
> >>>>>>>>>>>
> >>>>>>>>>>> What prevents doing split before migration?
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> The code does do a split prior to migration if THP selection fails
> >>>>>>>>>>
> >>>>>>>>>> Please see https://lore.kernel.org/lkml/20250703233511.2028395-5-balbirs@nvidia.com/
> >>>>>>>>>> and the fallback part which calls split_folio()
> >>>>>>>>>>
> >>>>>>>>>> But the case under consideration is special since the device needs to allocate
> >>>>>>>>>> corresponding pfn's as well. The changelog mentions it:
> >>>>>>>>>>
> >>>>>>>>>> "The common case that arises is that after setup, during migrate
> >>>>>>>>>> the destination might not be able to allocate MIGRATE_PFN_COMPOUND
> >>>>>>>>>> pages."
> >>>>>>>>>>
> >>>>>>>>>> I can expand on it, because migrate_vma() is a multi-phase operation
> >>>>>>>>>>
> >>>>>>>>>> 1. migrate_vma_setup()
> >>>>>>>>>> 2. migrate_vma_pages()
> >>>>>>>>>> 3. migrate_vma_finalize()
> >>>>>>>>>>
> >>>>>>>>>> It can so happen that when we get the destination pfn's allocated the destination
> >>>>>>>>>> might not be able to allocate a large page, so we do the split in migrate_vma_pages().
> >>>>>>>>>>
> >>>>>>>>>> The pages have been unmapped and collected in migrate_vma_setup()
> >>>>>>>>>>
> >>>>>>>>>> The next patch in the series 9/12 (https://lore.kernel.org/lkml/20250703233511.2028395-10-balbirs@nvidia.com/)
> >>>>>>>>>> tests the split and emulates a failure on the device side to allocate large pages
> >>>>>>>>>> and tests it in 10/12 (https://lore.kernel.org/lkml/20250703233511.2028395-11-balbirs@nvidia.com/)
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Another use case I’ve seen is when a previously allocated high-order
> >>>>>>>>> folio, now in the free memory pool, is reallocated as a lower-order
> >>>>>>>>> page. For example, a 2MB fault allocates a folio, the memory is later
> >>>>>>>>
> >>>>>>>> That is different. If the high-order folio is free, it should be split
> >>>>>>>> using split_page() from mm/page_alloc.c.
> >>>>>>>>
> >>>>>>>
> >>>>>>> Ah, ok. Let me see if that works - it would easier.
> >>>>>>>
> >>>>>
> >>>>> This suggestion quickly blows up as PageCompound is true and page_count
> >>>>> here is zero.
> >>>>
> >>>> OK, your folio has PageCompound set. Then you will need __split_unmapped_foio().
> >>>>
> >>>>>
> >>>>>>>>> freed, and then a 4KB fault reuses a page from that previously allocated
> >>>>>>>>> folio. This will be actually quite common in Xe / GPU SVM. In such
> >>>>>>>>> cases, the folio in an unmapped state needs to be split. I’d suggest a
> >>>>>>>>
> >>>>>>>> This folio is unused, so ->flags, ->mapping, and etc. are not set,
> >>>>>>>> __split_unmapped_folio() is not for it, unless you mean free folio
> >>>>>>>> differently.
> >>>>>>>>
> >>>>>>>
> >>>>>>> This is right, those fields should be clear.
> >>>>>>>
> >>>>>>> Thanks for the tip.
> >>>>>>>
> >>>>>> I was hoping to reuse __split_folio_to_order() at some point in the future
> >>>>>> to split the backing pages in the driver, but it is not an immediate priority
> >>>>>>
> >>>>>
> >>>>> I think we need something for the scenario I describe here. I was to
> >>>>> make __split_huge_page_to_list_to_order with a couple of hacks but it
> >>>>> almostly certainig not right as Zi pointed out.
> >>>>>
> >>>>> New to the MM stuff, but play around with this a bit and see if I can
> >>>>> come up with something that will work here.
> >>>>
> >>>> Can you try to write a new split_page function with __split_unmapped_folio()?
> >>>> Since based on your description, your folio is not mapped.
> >>>>
> >>>
> >>> Yes, page->mapping is NULL in this case - that was part of the hacks to
> >>> __split_huge_page_to_list_to_order (more specially __folio_split) I had
> >>> to make in order to get something working for this case.
> >>>
> >>> I can try out something based on __split_unmapped_folio and report back.
> >>
> >> mm-new tree has an updated __split_unmapped_folio() version, it moves
> >> all unmap irrelevant code out of __split_unmaped_folio(). You might find
> >> it easier to reuse.
> >>
> >> See: https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm.git/tree/mm/huge_memory.c?h=mm-new#n3430
> >>

I pulled in the new version and it to works for this case.

Matt

> >
> > Will take a look. It is possible some of the issues we are hitting are
> > due to working on drm-tip + pulling in core MM patches in this series on
> > top of that branch then missing some other patches in mm-new. I'll see
> > if ww can figure out a work flow to have the latest and greatest from
> > both drm-tip and the MM branches.
> >
> > Will these changes be in 6.17?
> 
> Hopefully yes. mm patches usually go from mm-new to mm-unstable
> to mm-stable to mainline. If not, we will figure it out. :)
> 
> >
> >> I am about to update the code with v4 patches. I will cc you, so that
> >> you can get the updated __split_unmaped_folio().
> >>
> >> Feel free to ask questions on folio split code.
> >>
> >
> > Thanks.
> >
> > Matt
> >
> >> Best Regards,
> >> Yan, Zi
> 
> 
> Best Regards,
> Yan, Zi