public inbox for netdev@vger.kernel.org
 help / color / mirror / Atom feed
From: Leon Hwang <leon.hwang@linux.dev>
To: Jakub Kicinski <kuba@kernel.org>
Cc: netdev@vger.kernel.org, Jesper Dangaard Brouer <hawk@kernel.org>,
	Ilias Apalodimas <ilias.apalodimas@linaro.org>,
	Steven Rostedt <rostedt@goodmis.org>,
	Masami Hiramatsu <mhiramat@kernel.org>,
	Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
	"David S . Miller" <davem@davemloft.net>,
	Eric Dumazet <edumazet@google.com>,
	Paolo Abeni <pabeni@redhat.com>, Simon Horman <horms@kernel.org>,
	kerneljasonxing@gmail.com, lance.yang@linux.dev,
	jiayuan.chen@linux.dev, linux-kernel@vger.kernel.org,
	linux-trace-kernel@vger.kernel.org,
	Leon Huang Fu <leon.huangfu@shopee.com>
Subject: Re: [PATCH net-next v4] page_pool: Add page_pool_release_stalled tracepoint
Date: Wed, 21 Jan 2026 10:17:51 +0800	[thread overview]
Message-ID: <15c40e1a-4207-4de5-9a22-e991368310f3@linux.dev> (raw)
In-Reply-To: <20260120152934.2eb16a11@kernel.org>



On 21/1/26 07:29, Jakub Kicinski wrote:
> On Tue, 20 Jan 2026 11:16:20 +0800 Leon Hwang wrote:
>> I encountered the 'pr_warn()' messages during Mellanox NIC flapping on a
>> system using the 'mlx5_core' driver (kernel 6.6). The root cause turned
>> out to be an application-level issue: the IBM/sarama “Client SeekBroker
>> Connection Leak” [1].
> 
> The scenario you are describing matches the situations we run into 
> at Meta. With the upstream kernel you can find that the pages are
> leaking based on stats, and if you care use drgn to locate them
> (in the recv queue).
> 

Thanks, that makes sense.

drgn indeed sounds helpful for locating the pages once it is confirmed
that the inflight pages are being held by the socket receive queue.

Before reaching that point, however, it was quite difficult to pinpoint
where those inflight pages were stuck. I was wondering whether there is
any other handy tool or method to help locate them earlier.

> The 6.6 kernel did not have page pool stats. I feel quite odd about
> adding more uAPI because someone is running a 2+ years old kernel 
> and doesn't have access to the already existing facilities.

After checking the code again, I realized that the 6.6 kernel does
have page pool stats support.

Unfortunately, CONFIG_PAGE_POOL_STATS was not enabled in our Shopee
deployment, which is why those facilities were not available to us.

In any case, I understand your concern. I won’t pursue adding this
tracepoint further if it’s not something you’d like to see upstream.

Thanks,
Leon


      reply	other threads:[~2026-01-21  2:18 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-01-19 10:21 [PATCH net-next v4] page_pool: Add page_pool_release_stalled tracepoint Leon Hwang
2026-01-19 16:37 ` Jakub Kicinski
2026-01-19 16:43   ` Jesper Dangaard Brouer
2026-01-19 17:23     ` Jakub Kicinski
2026-01-20  3:16   ` Leon Hwang
2026-01-20 23:29     ` Jakub Kicinski
2026-01-21  2:17       ` Leon Hwang [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=15c40e1a-4207-4de5-9a22-e991368310f3@linux.dev \
    --to=leon.hwang@linux.dev \
    --cc=davem@davemloft.net \
    --cc=edumazet@google.com \
    --cc=hawk@kernel.org \
    --cc=horms@kernel.org \
    --cc=ilias.apalodimas@linaro.org \
    --cc=jiayuan.chen@linux.dev \
    --cc=kerneljasonxing@gmail.com \
    --cc=kuba@kernel.org \
    --cc=lance.yang@linux.dev \
    --cc=leon.huangfu@shopee.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-trace-kernel@vger.kernel.org \
    --cc=mathieu.desnoyers@efficios.com \
    --cc=mhiramat@kernel.org \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    --cc=rostedt@goodmis.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox