[PATCH] mm: page_alloc: add trace_mm_page_alloc to bulk allocation path

[PATCH] mm: page_alloc: add trace_mm_page_alloc to bulk allocation path

[liuqiqi]

From: Qiqi Liu <liuqiqi@kylinos.cn>

The bulk allocation path in alloc_pages_bulk_noprof() does not fire
trace_mm_page_alloc for each allocated page, leaving bulk-allocated
pages invisible to ftrace, BPF, and perf. Only the fallback path via
__alloc_frozen_pages_noprof() is traced.

Add trace_mm_page_alloc() after set_page_refcounted() in the bulk loop
to match the standard allocation path. The tracepoint is gated by
static key, so there is no overhead when tracing is disabled.

Signed-off-by: Qiqi Liu <liuqiqi@kylinos.cn>
---
 mm/page_alloc.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 12fac9084c48..d17c21775cb2 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -5280,6 +5280,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid,
 
 		prep_new_page(page, 0, gfp, ALLOC_DEFAULT);
 		set_page_refcounted(page);
+		trace_mm_page_alloc(page, 0, gfp, ac.migratetype);
 		page_array[nr_populated++] = page;
 	}
 
-- 
2.25.1

Re: [PATCH] mm: page_alloc: add trace_mm_page_alloc to bulk allocation path

[Gregory Price]

On Mon, Sep 07, 2026 at 08:09:49PM +0800, liuqiqi@kylinos.cn wrote:
> From: Qiqi Liu <liuqiqi@kylinos.cn>
> 
> The bulk allocation path in alloc_pages_bulk_noprof() does not fire
> trace_mm_page_alloc for each allocated page, leaving bulk-allocated
> pages invisible to ftrace, BPF, and perf. Only the fallback path via
> __alloc_frozen_pages_noprof() is traced.
> 
> Add trace_mm_page_alloc() after set_page_refcounted() in the bulk loop
> to match the standard allocation path. The tracepoint is gated by
> static key, so there is no overhead when tracing is disabled.
> 
> Signed-off-by: Qiqi Liu <liuqiqi@kylinos.cn>
> ---
>  mm/page_alloc.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 12fac9084c48..d17c21775cb2 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -5280,6 +5280,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid,
>  
>  		prep_new_page(page, 0, gfp, ALLOC_DEFAULT);
>  		set_page_refcounted(page);
> +		trace_mm_page_alloc(page, 0, gfp, ac.migratetype);

Sashiko asked whether we're also missing kmsan hook here as well:

	kmsan_alloc_page(page, 0, gfp);

Might be worth a quick look and just adding both in one go.

~Gregory

[PATCH v2] mm: page_alloc: add missing hooks to bulk allocation path

[Qiqi Liu]

The bulk allocation path in alloc_pages_bulk_noprof() currently misses
trace_mm_page_alloc() and kmsan_alloc_page() calls, leaving bulk-allocated
pages invisible to ftrace/BPF/perf and leaving KMSAN shadow memory stale.

Add both calls after set_page_refcounted() in the bulk loop to match the
standard allocation path. Both are no-ops when their respective features
are disabled, so there is no overhead in production kernels.

Suggested-by: Gregory Price <gourry@gourry.net>
Signed-off-by: Qiqi Liu <liuqiqi@kylinos.cn>
---
 mm/page_alloc.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 12fac9084c48..02ba19f6dbb5 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -5280,6 +5280,8 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid,
 
 		prep_new_page(page, 0, gfp, ALLOC_DEFAULT);
 		set_page_refcounted(page);
+		trace_mm_page_alloc(page, 0, gfp, ac.migratetype);
+		kmsan_alloc_page(page, 0, gfp);
 		page_array[nr_populated++] = page;
 	}
 
-- 
2.25.1

[PATCH v3] mm: page_alloc: add missing hooks to bulk allocation path

[Qiqi Liu]

The bulk allocation path in alloc_pages_bulk_noprof() currently misses
trace_mm_page_alloc() and kmsan_alloc_page() calls, leaving bulk-allocated
pages invisible to ftrace/BPF/perf and leaving KMSAN shadow memory stale.

Add both calls after set_page_refcounted() in the bulk loop to match
the standard allocation path. The gfp mask passed to kmsan_alloc_page()
is stripped of __GFP_RECLAIM because the bulk loop runs under the PCP
spinlock, and KMSAN's stack depot allocation must not sleep.

Both are no-ops when their respective features are disabled, so there
is no overhead in production kernels.

Suggested-by: Gregory Price <gourry@gourry.net>
Signed-off-by: Qiqi Liu <liuqiqi@kylinos.cn>
---
 mm/page_alloc.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 12fac9084c48..73c73499a051 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -5280,6 +5280,8 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid,
 
 		prep_new_page(page, 0, gfp, ALLOC_DEFAULT);
 		set_page_refcounted(page);
+		trace_mm_page_alloc(page, 0, gfp, ac.migratetype);
+		kmsan_alloc_page(page, 0, gfp & ~__GFP_RECLAIM);
 		page_array[nr_populated++] = page;
 	}
 
-- 
2.25.1

Re: [PATCH v3] mm: page_alloc: add missing hooks to bulk allocation path

[Vlastimil Babka (SUSE)]

On 9/8/26 08:50, Qiqi Liu wrote:
> The bulk allocation path in alloc_pages_bulk_noprof() currently misses
> trace_mm_page_alloc() and kmsan_alloc_page() calls, leaving bulk-allocated
> pages invisible to ftrace/BPF/perf and leaving KMSAN shadow memory stale.
> 
> Add both calls after set_page_refcounted() in the bulk loop to match
> the standard allocation path. The gfp mask passed to kmsan_alloc_page()
> is stripped of __GFP_RECLAIM because the bulk loop runs under the PCP
> spinlock, and KMSAN's stack depot allocation must not sleep.
> 
> Both are no-ops when their respective features are disabled, so there
> is no overhead in production kernels.
> 
> Suggested-by: Gregory Price <gourry@gourry.net>
> Signed-off-by: Qiqi Liu <liuqiqi@kylinos.cn>
> ---
>  mm/page_alloc.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 12fac9084c48..73c73499a051 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -5280,6 +5280,8 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid,
>  
>  		prep_new_page(page, 0, gfp, ALLOC_DEFAULT);
>  		set_page_refcounted(page);
> +		trace_mm_page_alloc(page, 0, gfp, ac.migratetype);
> +		kmsan_alloc_page(page, 0, gfp & ~__GFP_RECLAIM);

With normal page allocation the set_page_refcounted() happens after
trace+kmsan. While it currently shouldn't matter, it could be more future
proof to keep the same order.

>  		page_array[nr_populated++] = page;
>  	}
>  

Re: [PATCH] mm: page_alloc: add trace_mm_page_alloc to bulk allocation path

[Qiqi Liu]

On Mon, Sep 07, 2026 at 10:53:19AM -0400, Gregory Price wrote:
> Sashiko asked whether we're also missing kmsan hook here as well:
> 
> 	kmsan_alloc_page(page, 0, gfp);
> 
> Might be worth a quick look and just adding both in one go.
> 
> ~Gregory

Hi Gregory,

Thanks for the suggestion and for taking the time to review this patch.

I've added both trace_mm_page_alloc() and kmsan_alloc_page() in v3 to
ensure the bulk path matches the standard allocation path.

Best regards,
Qiqi Liu

Re: [PATCH v3] mm: page_alloc: add missing hooks to bulk allocation path

[Qiqi Liu]

On Tue, Sep 08, 2026 at 10:38:46AM +0200, Vlastimil Babka wrote:
> With normal page allocation the set_page_refcounted() happens after
> trace+kmsan. While it currently shouldn't matter, it could be more future
> proof to keep the same order.

Hi Vlastimil,

Thanks for the review. You're right — I'll swap the order in v4 so that
trace_mm_page_alloc() and kmsan_alloc_page() come before set_page_refcounted()
to match the normal allocation path.

Best regards,
Qiqi Liu

[PATCH v4] mm: page_alloc: add missing hooks to bulk allocation path

[Qiqi Liu]

The bulk allocation path in alloc_pages_bulk_noprof() currently misses
trace_mm_page_alloc() and kmsan_alloc_page() calls, leaving bulk-allocated
pages invisible to ftrace/BPF/perf and leaving KMSAN shadow memory stale.

Add both calls in the bulk loop to match the standard allocation path,
placing them before set_page_refcounted() for consistency. The gfp mask
passed to kmsan_alloc_page() is stripped of __GFP_RECLAIM because the
bulk loop runs under the PCP spinlock, and KMSAN's stack depot allocation
must not sleep.

Both are no-ops when their respective features are disabled, so there
is no overhead in production kernels.

Suggested-by: Gregory Price <gourry@gourry.net>
Signed-off-by: Qiqi Liu <liuqiqi@kylinos.cn>
---
 mm/page_alloc.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 12fac9084c48..eac419eef697 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -5279,6 +5279,8 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid,
 		nr_account++;
 
 		prep_new_page(page, 0, gfp, ALLOC_DEFAULT);
+		trace_mm_page_alloc(page, 0, gfp, ac.migratetype);
+		kmsan_alloc_page(page, 0, gfp & ~__GFP_RECLAIM);
 		set_page_refcounted(page);
 		page_array[nr_populated++] = page;
 	}
-- 
2.25.1

Re: [PATCH v4] mm: page_alloc: add missing hooks to bulk allocation path

[Vlastimil Babka (SUSE)]

On 9/8/26 12:23, Qiqi Liu wrote:
> The bulk allocation path in alloc_pages_bulk_noprof() currently misses
> trace_mm_page_alloc() and kmsan_alloc_page() calls, leaving bulk-allocated
> pages invisible to ftrace/BPF/perf and leaving KMSAN shadow memory stale.
> 
> Add both calls in the bulk loop to match the standard allocation path,
> placing them before set_page_refcounted() for consistency. The gfp mask
> passed to kmsan_alloc_page() is stripped of __GFP_RECLAIM because the
> bulk loop runs under the PCP spinlock, and KMSAN's stack depot allocation
> must not sleep.
> 
> Both are no-ops when their respective features are disabled, so there
> is no overhead in production kernels.
> 
> Suggested-by: Gregory Price <gourry@gourry.net>
> Signed-off-by: Qiqi Liu <liuqiqi@kylinos.cn>

Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>

> ---
>  mm/page_alloc.c | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 12fac9084c48..eac419eef697 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -5279,6 +5279,8 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid,
>  		nr_account++;
>  
>  		prep_new_page(page, 0, gfp, ALLOC_DEFAULT);
> +		trace_mm_page_alloc(page, 0, gfp, ac.migratetype);
> +		kmsan_alloc_page(page, 0, gfp & ~__GFP_RECLAIM);
>  		set_page_refcounted(page);
>  		page_array[nr_populated++] = page;
>  	}

Re: [PATCH v4] mm: page_alloc: add missing hooks to bulk allocation path

[Gregory Price]

On Tue, Sep 08, 2026 at 06:23:56PM +0800, Qiqi Liu wrote:
> The bulk allocation path in alloc_pages_bulk_noprof() currently misses
> trace_mm_page_alloc() and kmsan_alloc_page() calls, leaving bulk-allocated
> pages invisible to ftrace/BPF/perf and leaving KMSAN shadow memory stale.
> 
> Add both calls in the bulk loop to match the standard allocation path,
> placing them before set_page_refcounted() for consistency. The gfp mask
> passed to kmsan_alloc_page() is stripped of __GFP_RECLAIM because the
> bulk loop runs under the PCP spinlock, and KMSAN's stack depot allocation
> must not sleep.
> 
> Both are no-ops when their respective features are disabled, so there
> is no overhead in production kernels.
> 
> Suggested-by: Gregory Price <gourry@gourry.net>
> Signed-off-by: Qiqi Liu <liuqiqi@kylinos.cn>

Thanks!

Reviewed-by: Gregory Price (Meta) <gourry@gourry.net>

Re: [PATCH v4] mm: page_alloc: add missing hooks to bulk allocation path

[Zi Yan]

On Tue Sep 8, 2026 at 6:23 AM EDT, Qiqi Liu wrote:
> The bulk allocation path in alloc_pages_bulk_noprof() currently misses
> trace_mm_page_alloc() and kmsan_alloc_page() calls, leaving bulk-allocated
> pages invisible to ftrace/BPF/perf and leaving KMSAN shadow memory stale.
>
> Add both calls in the bulk loop to match the standard allocation path,
> placing them before set_page_refcounted() for consistency. The gfp mask
> passed to kmsan_alloc_page() is stripped of __GFP_RECLAIM because the
> bulk loop runs under the PCP spinlock, and KMSAN's stack depot allocation
> must not sleep.
>
> Both are no-ops when their respective features are disabled, so there
> is no overhead in production kernels.
>
> Suggested-by: Gregory Price <gourry@gourry.net>
> Signed-off-by: Qiqi Liu <liuqiqi@kylinos.cn>
> ---
>  mm/page_alloc.c | 2 ++
>  1 file changed, 2 insertions(+)
>

LGTM.

Reviewed-by: Zi Yan <ziy@nvidia.com>

BTW, please send new versions in a new thread instead of replying to the
old one. Thanks.

-- 
Best Regards,
Yan, Zi