From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: bigeasy@linutronix.de, fweisbec@gmail.com,
linux-kernel@vger.kernel.org, hpa@zytor.com,
peterz@infradead.org, mingo@kernel.org, tglx@linutronix.de,
anna-maria@linutronix.de
Cc: linux-tip-commits@vger.kernel.org
Subject: Re: [tip:timers/urgent] timers: Unconditionally check deferrable base
Date: Mon, 15 Jan 2018 15:57:37 -0800 [thread overview]
Message-ID: <20180115235737.GA31127@linux.vnet.ibm.com> (raw)
In-Reply-To: <20180114224415.GX9671@linux.vnet.ibm.com>
On Sun, Jan 14, 2018 at 02:44:15PM -0800, Paul E. McKenney wrote:
> On Sun, Jan 14, 2018 at 02:31:07PM -0800, tip-bot for Thomas Gleixner wrote:
> > Commit-ID: ed4bbf7910b28ce3c691aef28d245585eaabda06
> > Gitweb: https://git.kernel.org/tip/ed4bbf7910b28ce3c691aef28d245585eaabda06
> > Author: Thomas Gleixner <tglx@linutronix.de>
> > AuthorDate: Sun, 14 Jan 2018 23:19:49 +0100
> > Committer: Thomas Gleixner <tglx@linutronix.de>
> > CommitDate: Sun, 14 Jan 2018 23:25:33 +0100
> >
> > timers: Unconditionally check deferrable base
> >
> > When the timer base is checked for expired timers then the deferrable base
> > must be checked as well. This was missed when making the deferrable base
> > independent of base::nohz_active.
>
> Very cool, thank you! I have fired up some of the dreaded TREE01 tests.
And (tentatively) a change in behavior!
In one case, I did not see stall warnings (at least not until the ftrace
dump at test end), but just the occasional "Waylayed timer" message:
[ 1316.628105] schedule_timeout: Waylayed timer base->clk: 0x1000eff23 jiffies: 0x1000f8335 base->next_expiry: 0x1000eff23 timer->flags: 0x3c800006 timer->expires 0x1000f62ca idx: f2 idx_now: 139 base->pending_map 000000080000000000000000000000000000000000000000100400002000000000000000000000000000010000000000000000000000000000000000000000000000000000000000
[ 1316.638372] Torture onoff task state:
[ 1316.639774] rcu_preempt I14984 8 2 0x80000000
[ 1316.641766] Call Trace:
[ 1316.642757] ? __schedule+0x33c/0x6f0
[ 1316.644109] ? trace_event_raw_event_timer_start+0x6d/0xb0
[ 1316.646042] schedule+0x37/0x90
[ 1316.647219] schedule_timeout+0x159/0x470
[ 1316.648825] ? __next_timer_interrupt+0xc0/0xc0
[ 1316.650459] rcu_gp_kthread+0x9ee/0x1090
[ 1316.652147] kthread+0xf0/0x130
[ 1316.653415] ? rcu_oom_notify+0xd0/0xd0
[ 1316.654790] ? kthread_destroy_worker+0x40/0x40
[ 1316.656470] ret_from_fork+0x1f/0x30
If I am reading the waylay message correctly, the timer was delayed by
about 8.3 seconds instead of the 21 seconds required to cause an RCU
CPU stall warning. I am going to rearrange the ftrace-dump code a bit
and run more tests.
The next one got the usual stall warning and:
[ 633.772036] schedule_timeout: Waylayed timer base->clk: 0x10004c5cb jiffies: 0x10005181e base->next_expiry: 0x10004c5cb timer->flags: 0x2c00006 timer->expires 0x10004c5ca idx: b idx_now: cd base->pending_map 000000000000080000000000000000000000020000000000000020000000000000000000000000000000000000080000000000000000000000000000000000000000000000000000
[ 633.855300] Torture onoff task state:
[ 633.855300] rcu_preempt I14984 8 2 0x80000000
[ 633.855300] Call Trace:
[ 633.855300] ? __schedule+0x33c/0x6f0
[ 633.855300] ? _raw_spin_lock_irqsave+0x12/0x40
[ 633.855300] schedule+0x37/0x90
[ 633.855300] rcu_gp_kthread+0xad/0x1090
[ 633.855300] kthread+0xf0/0x130
[ 633.855300] ? rcu_oom_notify+0xd0/0xd0
[ 633.855300] ? kthread_destroy_worker+0x40/0x40
[ 633.855300] ret_from_fork+0x1f/0x30
And the third one also got the usual stall warning and:
[ 382.482240] schedule_timeout: Waylayed timer base->clk: 0x10000322a jiffies: 0x100014232 base->next_expiry: 0x10000322a timer->flags: 0x44000001 timer->expires 0x10000f00f idx: 110 idx_now: 115 base->pending_map 000004000000000000000000000000000000000000000000000000000100000000000000000140000000000000000400000000000000000000000000000000000000000000000000
[ 382.482245] Torture onoff task state:
[ 382.482246] rcu_preempt I14984 8 2 0x80000000
[ 382.482273] Call Trace:
[ 382.482280] ? __schedule+0x33c/0x6f0
[ 382.482283] ? trace_event_raw_event_timer_start+0x6d/0xb0
[ 382.482285] schedule+0x37/0x90
[ 382.482287] schedule_timeout+0x159/0x470
[ 382.482289] ? __next_timer_interrupt+0xc0/0xc0
[ 382.482292] rcu_gp_kthread+0x9ee/0x1090
[ 382.482297] kthread+0xf0/0x130
[ 382.482299] ? rcu_oom_notify+0xd0/0xd0
[ 382.482301] ? kthread_destroy_worker+0x40/0x40
[ 382.482303] ret_from_fork+0x1f/0x30
I don't have enough runs to be sure, but my current (sad) guess is that
the error rate is unchanged.
Thanx, Paul
next prev parent reply other threads:[~2018-01-15 23:57 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <tip-ed4bbf7910b28ce3c691aef28d245585eaabda06@git.kernel.org>
2018-01-14 22:44 ` [tip:timers/urgent] timers: Unconditionally check deferrable base Paul E. McKenney
2018-01-15 23:57 ` Paul E. McKenney [this message]
2018-01-16 10:27 ` Thomas Gleixner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180115235737.GA31127@linux.vnet.ibm.com \
--to=paulmck@linux.vnet.ibm.com \
--cc=anna-maria@linutronix.de \
--cc=bigeasy@linutronix.de \
--cc=fweisbec@gmail.com \
--cc=hpa@zytor.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-tip-commits@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=peterz@infradead.org \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox