netdev.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Jakub Kicinski <kuba@kernel.org>
To: Yonglong Liu <liuyonglong@huawei.com>
Cc: Yunsheng Lin <linyunsheng@huawei.com>, <netdev@vger.kernel.org>,
	<davem@davemloft.net>, <edumazet@google.com>, <pabeni@redhat.com>,
	<ilias.apalodimas@linaro.org>,
	Jesper Dangaard Brouer <hawk@kernel.org>,
	Alexander Duyck <alexander.duyck@gmail.com>
Subject: Re: [RFC net] net: make page pool stall netdev unregistration to avoid IOMMU crashes
Date: Wed, 14 Aug 2024 07:56:03 -0700	[thread overview]
Message-ID: <20240814075603.05f8b0f5@kernel.org> (raw)
In-Reply-To: <cec044dc-504c-47e6-8ffa-58e8c9b42713@huawei.com>

On Wed, 14 Aug 2024 18:09:59 +0800 Yonglong Liu wrote:
> On 2024/8/10 11:57, Jakub Kicinski wrote:
> > On Fri, 9 Aug 2024 14:06:02 +0800 Yonglong Liu wrote:  
> >> [ 7724.272853] hns3 0000:7d:01.0: page_pool_release_retry(): eno1v0
> >> stalled pool shutdown: id 553, 82 inflight 6706 sec (hold netdev: 1855491)  
> > Alright :( You gotta look around for those 82 pages somehow with drgn.
> > bpftrace+kfunc the work that does the periodic print to get the address
> > of the page pool struct and then look around for pages from that pp.. :(  
> 
> I spent some time to learn how to use the drgn, and found those page, 
> but I think those page
> 
> is allocated by the hns3 driver, how to find out who own those page now?

Scan the entire system memory looking for the pointer to this page.
Dump the memory around location which hold that pointer. If you're
lucky the page will be held by an skb, and the memory around it will
look like struct skb_shared_info. If you're less lucky the page is used
by sk_buff for the head and address will not be exact. If you're less
lucky still the page will be directly leaked by the driver, and not
pointed to by anything...

I think the last case is most likely, FWIW.

  reply	other threads:[~2024-08-14 14:56 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-08-06 15:16 [RFC net] net: make page pool stall netdev unregistration to avoid IOMMU crashes Jakub Kicinski
2024-08-07  7:03 ` Yonglong Liu
2024-08-07 14:27   ` Jakub Kicinski
2024-08-08  1:11     ` Yonglong Liu
2024-08-07 11:00 ` Yunsheng Lin
2024-08-07 14:29   ` Jakub Kicinski
2024-08-08 12:52     ` Yonglong Liu
2024-08-08 14:05       ` Jakub Kicinski
2024-08-09  6:06         ` Yonglong Liu
2024-08-10  3:57           ` Jakub Kicinski
2024-08-14 10:09             ` Yonglong Liu
2024-08-14 14:56               ` Jakub Kicinski [this message]
2024-08-08 11:12 ` Ilias Apalodimas
2024-08-08 13:52   ` Jakub Kicinski
2024-08-08 14:30     ` Ilias Apalodimas
2024-08-08 14:51       ` Jakub Kicinski
2024-09-05 10:47 ` Yunsheng Lin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20240814075603.05f8b0f5@kernel.org \
    --to=kuba@kernel.org \
    --cc=alexander.duyck@gmail.com \
    --cc=davem@davemloft.net \
    --cc=edumazet@google.com \
    --cc=hawk@kernel.org \
    --cc=ilias.apalodimas@linaro.org \
    --cc=linyunsheng@huawei.com \
    --cc=liuyonglong@huawei.com \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).