Re: [RFC PATCH 0/2] mm/memory_hotplug: bound offline retry loops with a configurable limit

[Aboorva Devarajan <aboorvad@linux.ibm.com>]

On Wed, 2026-07-22 at 14:29 +0200, David Hildenbrand (Arm) wrote:
> On 7/22/26 14:08, David Hildenbrand (Arm) wrote:
> > On 7/22/26 09:48, Aboorva Devarajan wrote:
> > > Memory offlining can loop forever when a memory block holds a page that
> > > can never be migrated or freed.  This series adds an opt-in retry limit
> > > that lets offline_pages() bail out with -EBUSY instead of retrying
> > > forever.  The default (0) preserves today's behaviour exactly.
> > > 
> > > Patch 1 is the fix; patch 2 is an in-tree selftest that reproduces the
> > > hang from an unsignalable kworker and verifies the limit recovers it.
> > > 
> > > 1. The problem
> > > ==============
> > > 
> > > offline_pages() repeats two nested steps until the whole range is
> > > isolated:
> > > 
> > >   1. Inner loop: find and migrate movable pages out of the range
> > >      (scan_movable_pages() + do_migrate_range()).
> > >   2. Outer loop: re-check isolation (test_pages_isolated()); if not
> > >      fully isolated, go back to step 1.
> > > 
> > > Neither loop has a termination condition.  When a page can never be
> > > migrated, scan_movable_pages() keeps returning the same pfn,
> > > do_migrate_range() keeps failing on it, and control never even reaches
> > > the outer isolation re-check - the inner loop spins forever.  The admin
> > > guide acknowledges this:
> > > 
> > >   "Further, memory offlining might retry for a long time (or even
> > >    forever), until aborted by the user."
> > > 
> > > The single escape in the loop is signal_pending(current):
> > > 
> > >     offline_pages(pfn, end_pfn):
> > >         do {                                    /* outer: isolation */
> > >             do {                                /* inner: migration  */
> > >                 if (signal_pending(current))    /* <-- ONLY escape   */
> > >                         goto failed_removal;
> > >                 pfn = scan_movable_pages(pfn, end_pfn);
> > >                 do_migrate_range(pfn, end_pfn); /* may never succeed */
> > >             } while (pfn < end_pfn);
> > >         } while (test_pages_isolated(...));
> > > 
> > > 
> > > 2. Why the kernel itself should be able to bail
> > > ===============================================
> > > 
> > > We keep hitting this during memory hot-remove operations: a single
> > > stuck page can block the whole operation.  This was also reported in
> > > [4] earlier; one example, where offlining kept failing to migrate a
> > > busy block-device page-cache page (aops:def_blk_aops) in a normal zone:
> > > 
> > >   [10880.889199] page dumped because: migration failure
> > >   [10880.889232] migrating pfn 2a87b failed ret:1
> > >   [10880.889235] page: refcount:3 mapcount:0 mapping:00000000718ec5a6 index:0x857 pfn:0x2a87b
> > >   [10880.889241] aops:def_blk_aops ino:800003 dentry name(?):""
> > >   [10880.889245] flags: 0x33ffffe00004104(referenced|active|private|node=3|zone=0|lastcpupid=0x1fffff)
> > >   ...
> > >   [10880.889291] migrating pfn 2a87b failed ret:1
> > 
> > Hi,
> > 

Hi David,

Thanks a lot for the review.

> > the text reads AI generated. If you did use AI, please disclose it properly.

yes, I did use LLM to help re-phrase this. I'll make sure to disclose the usage going forward.

> > 
> > Quick feedback:
> > 
> > 1) Using passes is not really what we want in many cases (e.g., ZONE_MOVABLE
> > where we might need a couple of seconds/minutes to complete offlining with a lot
> > of concurrent activity). An actual timeout is also problematic (see below).
> > 
> > 2) Having a toggle for all offline_pages() users is questionable. In particular,
> > user-triggered offlining or offlinig triggered on ZONE_MOVABLE etc should not
> > obey such timeouts.
> > 
> > I proposed a more restricted approach only for offline_and_remove_memory()
> > previously [1]
> > 
> > [1] https://lore.kernel.org/all/20230627112220.229240-1-david@redhat.com/
> > 
> > Michal back then commented "I really hate having timeouts back. They just proven
> > to be hard to get right and it is essentially a policy implemented in the
> > kernel. They simply do not belong to the kernel space IMHO."
> > 
> > And I agree.

Thanks for pointing to [1]. On our end, on powerpc (pseries),
drmgr (the userspace tool that triggers memory hotplug) already wraps memory hotplug
operations with a timeout for exactly this reason. However, since we've now seen this
issue occur a couple of times under system load in ZONE_NORMAL, and also as per my understanding 
offline_pages() can be triggered from other kworker path where it cannot be interrupted
and could end up retrying indefinitely, So I wanted to get a consensus on how this should
be handled.

> > 
> > I assume you run into such issues with DIMMs in VMs? virtio-mem handles that
> > much nicer nowadays, by essentially doing the hard part that should fail easily
> > through alloc_contig_range().
> > 
> > offline_pages() is just designed to retry forever.
> > 
> 

Yes, I do not hit this issue with virtio-mem path, looks like the unsafe unplug mode that
relied on offline_pages() for migration was removed in f504e15b94eb ("virtio-mem:
remove unsafe unplug in Big Block Mode (BBM)"), so now it rather goes through the
alloc_contig_range() path, which gives up with -EBUSY after a few migration attempts
on the same page, so offline_pages() never sees the stuck page.

We originally hit this on a pseries LPAR, during memory hot-remove. offline_pages()
gets stuck on a single page that won't move, with two specific cases as below,
both in ZONE_NORMAL:

  Case 1. a slab page that ends up in the isolated range:

	[83310.373767] page_type: f5(slab)
	[83310.373770] raw: 023ffffe00000000 c0000028e001fa00 
	[83310.373774] raw: 0000000000000000 0000000001e101e1
	[83310.373778] page dumped because: isolation failed
	[83310.373788] failed to isolate pfn 4dc68

  Case 2. a busy block-device page-cache page that keeps failing to migrate:

	[10880.889058] page: refcount:3 mapcount:0 mapping:00000000718ec5a6 index:0x857 pfn:0x2a87b
	[10880.889062] memcg:c00000006a67e000
	[10880.889064] aops:def_blk_aops ino:800003 dentry name(?):""
	[10880.889069] flags: 0x33ffffe00004104(referenced|active|private|node=3|zone=0|lastcpupid=0x1fffff)
	[10880.889073] raw: 033ffffe00004104 c0000000b8667960 c0000000b8667960 c0000000130f8dd8
	[10880.889077] raw: 0000000000000857 c000000172abe1e0 00000003ffffffff c00000006a67e000
	[10880.889081] page dumped because: migration failure
	[10880.889114] migrating pfn 2a87b failed ret:1

I can also trigger it in a VM by hot-adding a DIMM, pinning a page and removing the DIMM.
This is the part I wanted to highlight, the ACPI DIMM remove runs offline_pages() from the ACPI kworker,
and as far as I understand, nothing can abort it, there is no signal to deliver, so there is no way
to cancel the work, so it spins until reboot in this case as well? 

> One thing we can definitely do is to just fail faster if we find an unmovable
> page in !ZONE_MOVABLE.
> 
> Usually that happens when we race offlining (has_unmovable_pages()) with page
> allocation, and failing on unmovable pages is actually perfectly fine.
> 
> But for ZONE_MOVABLE we should keep retrying, because some pages might only look
> temporarily unmovable.
> 
> So that would be the low hanging fruit: on !ZONE_MOVABLE, fail faster.

This handles scenarios like Case 1. A page can still be allocated from the range after
isolation and become slab. When test_pages_isolated() fails, we can re-check that
page with page_is_unmovable() and return -EBUSY if it's unmovable. I'm currently testing this approach
will send a patch for review.

Case 2 is different. The page is on the LRU, so it is considered movable and the fast-fail condition abovewon't be triggered. Instead page migration keeps failing, causing
offline_pages() to retry indefinitely
and can hang forever. This could be handled with a timeout or a user
policy in the application that
initiated the hotplug operation (signals). However, for the kworker
paths that trigger memory
offlining (ACPI remove, etc.), should this instead be handled by the
driver rather than relying on
offline_pages(), similar to virtio-mem?

Please let me know your comments.

Thanks,
Aboorva