[RFC PATCH v6 0/5] mm: Hot page tracking and promotion infrastructure

[Bharata B Rao <bharata@amd.com>]

Hi,

This is v6 of pghot, a hot-page tracking and promotion subsystem. The
main changes in this version are to retain only hintfault as source
and a lot of cleanups, fixes and restructuring.

This patchset introduces a new subsystem for hot page tracking and
promotion (pghot) with the following goals:

- Unify hot page detection from multiple sources like hint faults,
  page table scans, hardware hints (AMD IBS).
- Decouple detection from migration.
- Centralize promotion logic via per-lower-tier-node kmigrated kernel
  thread.
- Move promotion rate‑limiting and related logic used by
  numa_balancing=2 (NUMAB2, the current NUMA balancing–based promotion)
  from the scheduler to pghot for broader reuse.
  
Currently, multiple kernel subsystems detect page accesses independently.
This patchset consolidates accesses from these mechanisms by providing:

- A common API for reporting page accesses.
- Shared infrastructure for tracking hotness at PFN granularity.
- Per-lower-tier-node kernel threads for promoting pages.

Here is a brief summary of how this subsystem works:

- Tracks frequency and last access time.
- Additionally, the accessing NUMA node ID (NID) for each recorded
  access is also tracked in the precision mode.
- These hotness parameters are maintained in a per-PFN hotness record
  within the existing mem_section data structure.
  - In default mode, one byte (u8) is used for hotness record. 5 bits are
    used to store time and bucketing scheme is used to represent a total
    access time up to 4s with HZ=1000. Default toptier NID (0) is used as
    the target for promotion which can be changed via debugfs tunable.
  - In precision mode, 4 bytes (u32) are used for each hotness record.
    14 bits are used to store time which can represent around 16s
    with HZ=1000.
- Classifies pages as hot based on configurable thresholds.
- Pages classified as hot are marked as ready for migration using the
  ready bit. Both modes use MSB of the hotness record as ready bit.
- Per-lower-tier-node kmigrated threads periodically scan the PFNs of
  lower-tier nodes, checking for the migration-ready bit to perform
  batched migrations. Interval between successive scans and batching
  value are configurable via debugfs tunables.

Memory overhead
---------------
Default mode: 1 byte per lower-tier PFN. For a 1TB lower-tier memory
this amounts to 256MB overhead (assuming 4K pages)

Precision mode: 4 bytes per lower-tier PFN. For a 1TB of lower memory
this amounts to 1G overhead.

Bit layout of hotness record
----------------------------
Default mode
- Bits 0-1: Frequency (2bits, 4 access samples)
- Bits 2-6: Bucketed time (5bits, up to 4s with HZ=1000)
- Bit 7: Migration ready bit

Precision mode
- Bits 0-9: Target NID (10 bits)
- Bits 10-12: Frequency (3bits, 8 access samples)
- Bits 13-26: Time (14bits, up to 16s with HZ=1000)
- Bits 27-30: Reserved
- Bit 31: Migration ready bit

Potential hotness sources
-------------------------
1. NUMA Balancing (NUMAB2, Tiering mode)
2. IBS - Instruction Based Sampling, hardware based sampling
   mechanism present on AMD CPUs.
3. klruscand - PTE‑A bit scanning built on MGLRU’s walk helpers.
4. folio_mark_accessed() - Page cache access tracking (unmapped
   page cache pages)

Changes in v6
=============
- While earlier versions included sample implementation for all
  the hotness sources listed above, I have retained only NUMAB2
  in this iteration and will include the hardened versions
  of them in subsequent iterations.
- Cleaned up NUMAB2 implementation by removing the unused code
  (like access time tracking code etc).
- Ensured that NUMAB1 mode works as before. (Earlier versions made
  NUMA hint fault handler work only for NUMAB2 version)
- NUMA Balancing tiering mode is moved to its own new config
  CONFIG_NUMA_BALANCING_TIERING to make code sharing between
  NUMA Balancing and pghot easier.
- A bunch of hot page promotion related stats now depend upon
  CONFIG_PGHOT as the promotion engine is part of pghot.
- Fixed kmigrated to take a reference on the folio when walking
  the PFNs checking for migrate-ready folios.
- Fixed speculative folio access issue reported by Chris Mason's
  review-prompts.
- Added per-memcg NUMA_PAGE_MIGRATE stats accounting for batch
  migration API too.
- Added support for initializing hot_maps in the newly added
  sections during memory hotplug. 
- Default hotness threshold window changed from 4s to 3s as the
  maximum time representable in default mode is 3.9s only.
- Lot of cleanups and code restructuring.

Results
=======
Posted as replies to this mail thread.

This v6 patchset applies on top of upstream commit a989fde763f4f and
can be fetched from:

https://github.com/AMDESE/linux-mm/tree/bharata/pghot-rfcv6

v5: https://lore.kernel.org/linux-mm/20260129144043.231636-1-bharata@amd.com/
v4: https://lore.kernel.org/linux-mm/20251206101423.5004-1-bharata@amd.com/
v3: https://lore.kernel.org/linux-mm/20251110052343.208768-1-bharata@amd.com/
v2: https://lore.kernel.org/linux-mm/20250910144653.212066-1-bharata@amd.com/
v1: https://lore.kernel.org/linux-mm/20250814134826.154003-1-bharata@amd.com/
v0: https://lore.kernel.org/linux-mm/20250306054532.221138-1-bharata@amd.com/

Bharata B Rao (4):
  mm: migrate: Allow misplaced migration without VMA
  mm: Hot page tracking and promotion - pghot
  mm: pghot: Precision mode for pghot
  mm: sched: move NUMA balancing tiering promotion to pghot

Gregory Price (1):
  mm: migrate: Add migrate_misplaced_folios_batch()

 Documentation/admin-guide/mm/pghot.txt |  80 ++++
 include/linux/migrate.h                |  10 +-
 include/linux/mm.h                     |  35 +-
 include/linux/mmzone.h                 |  24 +-
 include/linux/pghot.h                  | 113 +++++
 include/linux/vm_event_item.h          |   5 +
 init/Kconfig                           |  13 +
 kernel/sched/core.c                    |   7 +
 kernel/sched/debug.c                   |   1 -
 kernel/sched/fair.c                    | 177 +------
 kernel/sched/sched.h                   |   1 -
 mm/Kconfig                             |  25 +
 mm/Makefile                            |   6 +
 mm/huge_memory.c                       |  27 +-
 mm/memcontrol.c                        |   6 +-
 mm/memory-tiers.c                      |  15 +-
 mm/memory.c                            |  36 +-
 mm/mempolicy.c                         |   3 -
 mm/migrate.c                           |  88 +++-
 mm/mm_init.c                           |  10 +
 mm/pghot-default.c                     |  79 ++++
 mm/pghot-precise.c                     |  81 ++++
 mm/pghot-tunables.c                    | 182 ++++++++
 mm/pghot.c                             | 618 +++++++++++++++++++++++++
 mm/vmstat.c                            |   7 +-
 25 files changed, 1411 insertions(+), 238 deletions(-)
 create mode 100644 Documentation/admin-guide/mm/pghot.txt
 create mode 100644 include/linux/pghot.h
 create mode 100644 mm/pghot-default.c
 create mode 100644 mm/pghot-precise.c
 create mode 100644 mm/pghot-tunables.c
 create mode 100644 mm/pghot.c

base-commit: a989fde763f4f24209e4702f50a45be572340e68
-- 
2.34.1