linux-trace-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Leon Hwang <leon.hwang@linux.dev>
To: Jesper Dangaard Brouer <hawk@kernel.org>, netdev@vger.kernel.org
Cc: Ilias Apalodimas <ilias.apalodimas@linaro.org>,
	Steven Rostedt <rostedt@goodmis.org>,
	Masami Hiramatsu <mhiramat@kernel.org>,
	Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
	"David S . Miller" <davem@davemloft.net>,
	Eric Dumazet <edumazet@google.com>,
	Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>,
	Simon Horman <horms@kernel.org>,
	kerneljasonxing@gmail.com, lance.yang@linux.dev,
	jiayuan.chen@linux.dev, linux-kernel@vger.kernel.org,
	linux-trace-kernel@vger.kernel.org,
	Leon Huang Fu <leon.huangfu@shopee.com>,
	Dragos Tatulea <dtatulea@nvidia.com>,
	kernel-team <kernel-team@cloudflare.com>,
	Yan Zhai <yan@cloudflare.com>
Subject: Re: [PATCH net-next v3] page_pool: Add page_pool_release_stalled tracepoint
Date: Fri, 2 Jan 2026 21:54:16 +0800	[thread overview]
Message-ID: <3461bd2d-2ee6-4592-9069-2c7a4b8a0b4b@linux.dev> (raw)
In-Reply-To: <011ca15e-107b-4679-8203-f5f821f27900@kernel.org>



On 2026/1/2 19:43, Jesper Dangaard Brouer wrote:
> 
> 
> On 02/01/2026 08.17, Leon Hwang wrote:
>> Introduce a new tracepoint to track stalled page pool releases,
>> providing better observability for page pool lifecycle issues.
>>
> 
> In general I like/support adding this tracepoint for "debugability" of
> page pool lifecycle issues.
> 
> For "observability" @Kuba added a netlink scheme[1][2] for page_pool[3],
> which gives us the ability to get events and list page_pools from
> userspace.
> I've not used this myself (yet) so I need input from others if this is
> something that others have been using for page pool lifecycle issues?
> 
> Need input from @Kuba/others as the "page-pool-get"[4] state that "Only
> Page Pools associated with a net_device can be listed".  Don't we want
> the ability to list "invisible" page_pool's to allow debugging issues?
> 
>  [1] https://docs.kernel.org/userspace-api/netlink/intro-specs.html
>  [2] https://docs.kernel.org/userspace-api/netlink/index.html
>  [3] https://docs.kernel.org/netlink/specs/netdev.html
>  [4] https://docs.kernel.org/netlink/specs/netdev.html#page-pool-get
> 
> Looking at the code, I see that NETDEV_CMD_PAGE_POOL_CHANGE_NTF netlink
> notification is only generated once (in page_pool_destroy) and not when
> we retry in page_pool_release_retry (like this patch).  In that sense,
> this patch/tracepoint is catching something more than netlink provides.
> First I though we could add a netlink notification, but I can imagine
> cases this could generate too many netlink messages e.g. a netdev with
> 128 RX queues generating these every second for every RX queue.
> 
> Guess, I've talked myself into liking this change, what do other
> maintainers think?  (e.g. netlink scheme and debugging balance)
> 

Hi Jesper,

Thanks for the thoughtful review and for sharing the context around the
existing netlink-based observability.

I ran into a real-world issue where stalled pages were still referenced
by dangling TCP sockets. I wrote up the investigation in more detail in
my blog post “let page inflight” [1] (unfortunately only available in
Chinese at the moment).

In practice, the hardest part was identifying *who* was still holding
references to the inflight pages. With the current tooling, it is very
difficult to introspect the active users of a page once it becomes stalled.

If we can expose more information about current page users—such as the
user type and a user pointer, it becomes much easier to debug these
issues using BPF-based tools. For example, by tracing
page_pool_state_hold and page_pool_state_release, tools like bpftrace
[2] or bpfsnoop [3] (which I implemented) can correlate inflight page
pointers with their active users. This significantly lowers the barrier
to diagnosing page pool lifecycle problems.

As you noted, the existing netlink notifications are generated only at
page_pool_destroy, and not during retries in page_pool_release_retry. In
that sense, the proposed tracepoint captures a class of issues that
netlink does not currently cover, and does so without the risk of
generating excessive userspace events.

Thanks again for the feedback, and I’m happy to refine the approach
based on further input from you, Kuba, or other maintainers.

Links:
[1] https://blog.leonhw.com/post/linux-networking-6-inflight-page/
[2] https://github.com/bpftrace/bpftrace/
[3] https://github.com/bpfsnoop/bpfsnoop/

Thanks,
Leon

[...]


      reply	other threads:[~2026-01-02 13:54 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-01-02  7:17 [PATCH net-next v3] page_pool: Add page_pool_release_stalled tracepoint Leon Hwang
2026-01-02 11:43 ` Jesper Dangaard Brouer
2026-01-02 13:54   ` Leon Hwang [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=3461bd2d-2ee6-4592-9069-2c7a4b8a0b4b@linux.dev \
    --to=leon.hwang@linux.dev \
    --cc=davem@davemloft.net \
    --cc=dtatulea@nvidia.com \
    --cc=edumazet@google.com \
    --cc=hawk@kernel.org \
    --cc=horms@kernel.org \
    --cc=ilias.apalodimas@linaro.org \
    --cc=jiayuan.chen@linux.dev \
    --cc=kernel-team@cloudflare.com \
    --cc=kerneljasonxing@gmail.com \
    --cc=kuba@kernel.org \
    --cc=lance.yang@linux.dev \
    --cc=leon.huangfu@shopee.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-trace-kernel@vger.kernel.org \
    --cc=mathieu.desnoyers@efficios.com \
    --cc=mhiramat@kernel.org \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    --cc=rostedt@goodmis.org \
    --cc=yan@cloudflare.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).