Re: [RFC, PATCH 00/12] userfaultfd: working set tracking for VM guest memory

[Kiryl Shutsemau <kas@kernel.org>]

On Fri, Apr 24, 2026 at 11:55:39AM -0400, Peter Xu wrote:
> On Fri, Apr 24, 2026 at 02:49:58PM +0100, Kiryl Shutsemau wrote:
> > On Fri, Apr 24, 2026 at 07:51:44AM -0400, Peter Xu wrote:
> > > On Fri, Apr 24, 2026 at 11:34:48AM +0100, Kiryl Shutsemau wrote:
> > > > Both page_idle and the LRUs (legacy or MGLRU) track accesses on physical
> > > > memory. We need visibility in the virtual address space domain.
> > > 
> > > Yes they are, but ACCESS bit isn't.
> > 
> > A-bit is not a reliable signal for userspace working-set tracking
> > because the kernel itself is a concurrent consumer. It is exactly why
> > page_idle needs PG_young on top of the A-bit: PG_young is the "kernel
> 
> I assume you meant PG_idle.  I actually don't know whether PG_young is
> still actively used anywhere in the current code base.
> 
> > ate the A-bit but the page was actually touched" escape hatch. And
> > bringing PG_young into the picture puts us right back into physical-side
> > tracking.
> > 
> > > For migration, see e.g. remove_migration_pte() has:
> > > 
> > > 		if (!softleaf_is_migration_young(entry))
> > > 			pte = pte_mkold(pte);
> > 
> > remove_migration_pte() only propagates young-at-unmap. It does not
> > cover the common case: A-bit cleared by reclaim before migration
> > started. The concurrent-consumer problem is what breaks the signal,
> > not the migration boundary.
> 
> IMHO it's a separate problem, and AFAIU it was well solved at least with
> old LRUs with PG_idle. It's just slightly unfortunate it doesn't yet work
> with MGLRU.  Also, when the extra bit is in folio->flags, it only works if
> both the consumers are reporting per-folio, not per-mm.
> 
> I'm actually curious whether there're numbers or solid proof showing that
> in your case the per-folio perf is too bad already to justify a new per-mm
> API, like RWP. 

Fair ask, and I don't have numbers I can point to right now. But I'd
flag that the case for RWP doesn't rest only on cost:

  - LRU-agnostic. Per-folio approaches are bound to the current
    reclaim backend (legacy, MGLRU, whatever is next);
  - Race-free against reclaim's A-bit consumption;
  - Deterministic preservation across swap and migration.

Numbers would strengthen the cost story but they don't change those
structural points.

> I want to explore if there's something that can still be generic and work
> for per-mm tracking.  I believe if we can have some bit in the ptes, then
> when mm reclaim code walks clearing ACCESS bit and sees some vma is being
> tracked, then instead of setting PG_idle, it can just move the access bit
> over to that special pte bit, and only to this vma this pte.  IIUC that'll
> benefit from both worlds: fast HW-accelerated access bit, and no minor
> faults.
> 
> Would something like that worth exploring?

This can be interesting. But a spare pte bit is high ask.

And when you start tracking, you need to clear A-bit. Where do you move
it? Activate folio?

-- 
  Kiryl Shutsemau / Kirill A. Shutemov