From: "Yajun Deng" <yajun.deng@linux.dev>
To: "Jakub Kicinski" <kuba@kernel.org>
Cc: jhs@mojatatu.com, xiyou.wangcong@gmail.com, jiri@resnulli.us,
davem@davemloft.net, edumazet@google.com, pabeni@redhat.com,
netdev@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH] net: sched: print jiffies when transmit queue time out
Date: Thu, 20 Apr 2023 02:17:21 +0000 [thread overview]
Message-ID: <acb95e0283e0720979f67f8321c3cdbe@linux.dev> (raw)
In-Reply-To: <20230419182713.2cd1f81b@kernel.org>
April 20, 2023 9:27 AM, "Jakub Kicinski" <kuba@kernel.org> wrote:
> On Wed, 19 Apr 2023 19:56:32 +0800 Yajun Deng wrote:
>
>> Although there is watchdog_timeo to let users know when the transmit queue
>> begin stall, but dev_watchdog() is called with an interval. The jiffies
>> will always be greater than watchdog_timeo.
>>
>> To let users know the exact time the stall started, print jiffies when
>> the transmit queue time out.
>
> Please add an explanation of how this information is useful in practice.
We found some cases with several warnings. We want to confirm which happened first.
First warning:
16:37:57 kernel: [ 7100.097547] ------------[ cut here ]------------
16:37:57 kernel: [ 7100.097550] NETDEV WATCHDOG: eno2 (i40e): transmit queue 8 timed out
16:37:57 kernel: [ 7100.097571] WARNING: CPU: 8 PID: 0 at net/sched/sch_generic.c:467 dev_watchdog+0x260/0x270
...
Second warning:
16:38:44 kernel: [ 7147.756952] rcu: INFO: rcu_preempt self-detected stall on CPU
16:38:44 kernel: [ 7147.756958] rcu: 24-....: (59999 ticks this GP) idle=546/1/0x4000000000000000 softirq=367 3137/3673146 fqs=13844
16:38:44 kernel: [ 7147.756960] (t=60001 jiffies g=4322709 q=133381)
16:38:44 kernel: [ 7147.756962] NMI backtrace for cpu 24
...
As we can see, the transmit queue start stall should be before 16:37:52, the rcu start stall is 16:37:44.
These two times are closer, we want to confirm which happened first.
prev parent reply other threads:[~2023-04-20 2:26 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-04-19 11:56 [PATCH] net: sched: print jiffies when transmit queue time out Yajun Deng
2023-04-19 12:02 ` Eric Dumazet
2023-04-20 2:49 ` Yajun Deng
2023-04-20 1:27 ` Jakub Kicinski
2023-04-20 2:17 ` Yajun Deng [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=acb95e0283e0720979f67f8321c3cdbe@linux.dev \
--to=yajun.deng@linux.dev \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=jhs@mojatatu.com \
--cc=jiri@resnulli.us \
--cc=kuba@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=xiyou.wangcong@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).