linux-trace-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH net-next v2] page_pool: Add page_pool_release_stalled tracepoint
@ 2026-01-02  6:17 Leon Hwang
  2026-01-02 15:45 ` Steven Rostedt
  0 siblings, 1 reply; 4+ messages in thread
From: Leon Hwang @ 2026-01-02  6:17 UTC (permalink / raw)
  To: netdev
  Cc: Jesper Dangaard Brouer, Ilias Apalodimas, Steven Rostedt,
	Masami Hiramatsu, Mathieu Desnoyers, David S . Miller,
	Eric Dumazet, Jakub Kicinski, Paolo Abeni, Simon Horman,
	kerneljasonxing, lance.yang, jiayuan.chen, linux-kernel,
	linux-trace-kernel, Leon Hwang, Leon Huang Fu

Introduce a new tracepoint to track stalled page pool releases,
providing better observability for page pool lifecycle issues.

Problem:
Currently, when a page pool shutdown is stalled due to inflight pages,
the kernel only logs a warning message via pr_warn(). This has several
limitations:

1. The warning floods the kernel log after the initial DEFER_WARN_INTERVAL,
   making it difficult to track the progression of stalled releases
2. There's no structured way to monitor or analyze these events
3. Debugging tools cannot easily capture and correlate stalled pool
   events with other network activity

Solution:
Add a new tracepoint, page_pool_release_stalled, that fires when a page
pool shutdown is stalled. The tracepoint captures:
- pool: pointer to the stalled page_pool
- inflight: number of pages still in flight
- sec: seconds since the release was deferred

The implementation also modifies the logging behavior:
- pr_warn() is only emitted during the first warning interval
  (DEFER_WARN_INTERVAL to DEFER_WARN_INTERVAL*2)
- The tracepoint is fired always, reducing log noise while still
  allowing monitoring tools to track the issue

This allows developers and system administrators to:
- Use tools like perf, ftrace, or eBPF to monitor stalled releases
- Correlate page pool issues with network driver behavior
- Analyze patterns without parsing kernel logs
- Track the progression of inflight page counts over time

Signed-off-by: Leon Huang Fu <leon.huangfu@shopee.com>
Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
---
v1 -> v2:
 - Drop RFC.
 - Store 'pool->user.id' to '__entry->id' (per Steven Rostedt).
 - https://lore.kernel.org/netdev/20251125082207.356075-1-leon.hwang@linux.dev/
---
 include/trace/events/page_pool.h | 24 ++++++++++++++++++++++++
 net/core/page_pool.c             |  6 ++++--
 2 files changed, 28 insertions(+), 2 deletions(-)

diff --git a/include/trace/events/page_pool.h b/include/trace/events/page_pool.h
index 31825ed30032..c34de6a5ae80 100644
--- a/include/trace/events/page_pool.h
+++ b/include/trace/events/page_pool.h
@@ -113,6 +113,30 @@ TRACE_EVENT(page_pool_update_nid,
 		  __entry->pool, __entry->pool_nid, __entry->new_nid)
 );
 
+TRACE_EVENT(page_pool_release_stalled,
+
+	TP_PROTO(const struct page_pool *pool, int inflight, int sec),
+
+	TP_ARGS(pool, inflight, sec),
+
+	TP_STRUCT__entry(
+		__field(const struct page_pool *, pool)
+		__field(u32,			  id)
+		__field(int,			  inflight)
+		__field(int,			  sec)
+	),
+
+	TP_fast_assign(
+		__entry->pool		= pool;
+		__entry->id		= pool->user.id;
+		__entry->inflight	= inflight;
+		__entry->sec		= sec;
+	),
+
+	TP_printk("page_pool=%p id=%d inflight=%d sec=%d",
+		  __entry->pool, __entry->id, __entry->inflight, __entry->sec)
+);
+
 #endif /* _TRACE_PAGE_POOL_H */
 
 /* This part must be outside protection */
diff --git a/net/core/page_pool.c b/net/core/page_pool.c
index 265a729431bb..01564aa84c89 100644
--- a/net/core/page_pool.c
+++ b/net/core/page_pool.c
@@ -1222,8 +1222,10 @@ static void page_pool_release_retry(struct work_struct *wq)
 	    (!netdev || netdev == NET_PTR_POISON)) {
 		int sec = (s32)((u32)jiffies - (u32)pool->defer_start) / HZ;
 
-		pr_warn("%s() stalled pool shutdown: id %u, %d inflight %d sec\n",
-			__func__, pool->user.id, inflight, sec);
+		if (sec >= DEFER_WARN_INTERVAL / HZ && sec < DEFER_WARN_INTERVAL * 2 / HZ)
+			pr_warn("%s() stalled pool shutdown: id %u, %d inflight %d sec\n",
+				__func__, pool->user.id, inflight, sec);
+		trace_page_pool_release_stalled(pool, inflight, sec);
 		pool->defer_warn = jiffies + DEFER_WARN_INTERVAL;
 	}
 
-- 
2.52.0


^ permalink raw reply related	[flat|nested] 4+ messages in thread

* Re: [PATCH net-next v2] page_pool: Add page_pool_release_stalled tracepoint
  2026-01-02  6:17 [PATCH net-next v2] page_pool: Add page_pool_release_stalled tracepoint Leon Hwang
@ 2026-01-02 15:45 ` Steven Rostedt
  2026-01-02 15:54   ` Leon Hwang
  0 siblings, 1 reply; 4+ messages in thread
From: Steven Rostedt @ 2026-01-02 15:45 UTC (permalink / raw)
  To: Leon Hwang
  Cc: netdev, Jesper Dangaard Brouer, Ilias Apalodimas,
	Masami Hiramatsu, Mathieu Desnoyers, David S . Miller,
	Eric Dumazet, Jakub Kicinski, Paolo Abeni, Simon Horman,
	kerneljasonxing, lance.yang, jiayuan.chen, linux-kernel,
	linux-trace-kernel, Leon Huang Fu

On Fri,  2 Jan 2026 14:17:18 +0800
Leon Hwang <leon.hwang@linux.dev> wrote:

> diff --git a/include/trace/events/page_pool.h b/include/trace/events/page_pool.h
> index 31825ed30032..c34de6a5ae80 100644
> --- a/include/trace/events/page_pool.h
> +++ b/include/trace/events/page_pool.h
> @@ -113,6 +113,30 @@ TRACE_EVENT(page_pool_update_nid,
>  		  __entry->pool, __entry->pool_nid, __entry->new_nid)
>  );
>  
> +TRACE_EVENT(page_pool_release_stalled,
> +
> +	TP_PROTO(const struct page_pool *pool, int inflight, int sec),
> +
> +	TP_ARGS(pool, inflight, sec),
> +
> +	TP_STRUCT__entry(
> +		__field(const struct page_pool *, pool)
> +		__field(u32,			  id)
> +		__field(int,			  inflight)
> +		__field(int,			  sec)
> +	),
> +
> +	TP_fast_assign(
> +		__entry->pool		= pool;
> +		__entry->id		= pool->user.id;
> +		__entry->inflight	= inflight;
> +		__entry->sec		= sec;
> +	),
> +
> +	TP_printk("page_pool=%p id=%d inflight=%d sec=%d",
> +		  __entry->pool, __entry->id, __entry->inflight, __entry->sec)
> +);
> +
>  #endif /* _TRACE_PAGE_POOL_H */

From a tracing POV, I see nothing wrong with this.

Reviewed-by: Steven Rostedt (Google) <rostedt@goodmis.org>

-- Steve

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [PATCH net-next v2] page_pool: Add page_pool_release_stalled tracepoint
  2026-01-02 15:45 ` Steven Rostedt
@ 2026-01-02 15:54   ` Leon Hwang
  2026-01-02 18:00     ` Steven Rostedt
  0 siblings, 1 reply; 4+ messages in thread
From: Leon Hwang @ 2026-01-02 15:54 UTC (permalink / raw)
  To: Steven Rostedt
  Cc: netdev, Jesper Dangaard Brouer, Ilias Apalodimas,
	Masami Hiramatsu, Mathieu Desnoyers, David S . Miller,
	Eric Dumazet, Jakub Kicinski, Paolo Abeni, Simon Horman,
	kerneljasonxing, lance.yang, jiayuan.chen, linux-kernel,
	linux-trace-kernel, Leon Huang Fu



On 2026/1/2 23:45, Steven Rostedt wrote:
> On Fri,  2 Jan 2026 14:17:18 +0800
> Leon Hwang <leon.hwang@linux.dev> wrote:
> 
>> diff --git a/include/trace/events/page_pool.h b/include/trace/events/page_pool.h
>> index 31825ed30032..c34de6a5ae80 100644
>> --- a/include/trace/events/page_pool.h
>> +++ b/include/trace/events/page_pool.h
>> @@ -113,6 +113,30 @@ TRACE_EVENT(page_pool_update_nid,
>>  		  __entry->pool, __entry->pool_nid, __entry->new_nid)
>>  );
>>  
>> +TRACE_EVENT(page_pool_release_stalled,
>> +
>> +	TP_PROTO(const struct page_pool *pool, int inflight, int sec),
>> +
>> +	TP_ARGS(pool, inflight, sec),
>> +
>> +	TP_STRUCT__entry(
>> +		__field(const struct page_pool *, pool)
>> +		__field(u32,			  id)
>> +		__field(int,			  inflight)
>> +		__field(int,			  sec)
>> +	),
>> +
>> +	TP_fast_assign(
>> +		__entry->pool		= pool;
>> +		__entry->id		= pool->user.id;
>> +		__entry->inflight	= inflight;
>> +		__entry->sec		= sec;
>> +	),
>> +
>> +	TP_printk("page_pool=%p id=%d inflight=%d sec=%d",
>> +		  __entry->pool, __entry->id, __entry->inflight, __entry->sec)
>> +);
>> +
>>  #endif /* _TRACE_PAGE_POOL_H */
> 
> From a tracing POV, I see nothing wrong with this.
> 
> Reviewed-by: Steven Rostedt (Google) <rostedt@goodmis.org>
> 
> -- Steve

Hi Steve,

Thanks for the review!

I realized the id should be printed with '%u', so I've sent out v3 [1]
with that adjustment.

Links:
[1]
https://lore.kernel.org/netdev/20260102071745.291969-1-leon.hwang@linux.dev/

Thanks,
Leon


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [PATCH net-next v2] page_pool: Add page_pool_release_stalled tracepoint
  2026-01-02 15:54   ` Leon Hwang
@ 2026-01-02 18:00     ` Steven Rostedt
  0 siblings, 0 replies; 4+ messages in thread
From: Steven Rostedt @ 2026-01-02 18:00 UTC (permalink / raw)
  To: Leon Hwang
  Cc: netdev, Jesper Dangaard Brouer, Ilias Apalodimas,
	Masami Hiramatsu, Mathieu Desnoyers, David S . Miller,
	Eric Dumazet, Jakub Kicinski, Paolo Abeni, Simon Horman,
	kerneljasonxing, lance.yang, jiayuan.chen, linux-kernel,
	linux-trace-kernel, Leon Huang Fu

On Fri, 2 Jan 2026 23:54:20 +0800
Leon Hwang <leon.hwang@linux.dev> wrote:

> Thanks for the review!
> 
> I realized the id should be printed with '%u', so I've sent out v3 [1]
> with that adjustment.
> 
> Links:
> [1]
> https://lore.kernel.org/netdev/20260102071745.291969-1-leon.hwang@linux.dev/

Feel free to add my r-b tag to that one as well.

-- Steve

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2026-01-02 18:00 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-01-02  6:17 [PATCH net-next v2] page_pool: Add page_pool_release_stalled tracepoint Leon Hwang
2026-01-02 15:45 ` Steven Rostedt
2026-01-02 15:54   ` Leon Hwang
2026-01-02 18:00     ` Steven Rostedt

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).