* [BUG] NOHZ tick-stop error: local softirq work is pending @ 2023-09-10 2:28 Joel Fernandes 2023-09-10 10:21 ` Paul E. McKenney 0 siblings, 1 reply; 6+ messages in thread From: Joel Fernandes @ 2023-09-10 2:28 UTC (permalink / raw) To: Frederic Weisbecker, Paul E. McKenney; +Cc: rcu Hi Paul, Frederic, ever since this recent patch got merged into all recent stables: https://lore.kernel.org/all/20230818200757.1808398-1-paul.gortmaker@windriver.com/ .. .. I am seeing this happen a lot now causing rcutorture failures: "NOHZ tick-stop error: local softirq work is pending" It looks like the "tick-stop error" as local_bh_blocked always returns false on !PREEMPT_RT, so that patch itself seems correct. However, it appears the string in console-badness.sh is different: console-badness.sh:grep -v 'NOHZ tick-stop error: Non-RCU local softirq work is pending, handler' So the string in the script needs an update? Also, considering that it is being grep -v'ed out, any background on why this message is not bad and any history? Why is softirq work pending when the tick is stopped? thanks, - Joel ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [BUG] NOHZ tick-stop error: local softirq work is pending 2023-09-10 2:28 [BUG] NOHZ tick-stop error: local softirq work is pending Joel Fernandes @ 2023-09-10 10:21 ` Paul E. McKenney 2023-09-10 20:25 ` Frederic Weisbecker 0 siblings, 1 reply; 6+ messages in thread From: Paul E. McKenney @ 2023-09-10 10:21 UTC (permalink / raw) To: Joel Fernandes; +Cc: Frederic Weisbecker, rcu On Sat, Sep 09, 2023 at 10:28:36PM -0400, Joel Fernandes wrote: > Hi Paul, Frederic, ever since this recent patch got merged into all > recent stables: > https://lore.kernel.org/all/20230818200757.1808398-1-paul.gortmaker@windriver.com/ > .. > > .. I am seeing this happen a lot now causing rcutorture failures: > "NOHZ tick-stop error: local softirq work is pending" > > It looks like the "tick-stop error" as local_bh_blocked always returns > false on !PREEMPT_RT, so that patch itself seems correct. > > However, it appears the string in console-badness.sh is different: > > console-badness.sh:grep -v 'NOHZ tick-stop error: Non-RCU local > softirq work is pending, handler' > > So the string in the script needs an update? Looks like it, though I believe that there is a fix in the offing, at which point that "grep -v" should just be removed. Depending on timing, of course. > Also, considering that it is being grep -v'ed out, any background on > why this message is not bad and any history? Why is softirq work > pending when the tick is stopped? That was so long back that it is pretty hazy. I vaguely recall that this was considered to be in need of fixing, but a false positive from a torture-test perspective. Thanx, Paul ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [BUG] NOHZ tick-stop error: local softirq work is pending 2023-09-10 10:21 ` Paul E. McKenney @ 2023-09-10 20:25 ` Frederic Weisbecker 2023-09-10 21:48 ` Paul E. McKenney 2023-09-10 23:02 ` Joel Fernandes 0 siblings, 2 replies; 6+ messages in thread From: Frederic Weisbecker @ 2023-09-10 20:25 UTC (permalink / raw) To: Paul E. McKenney; +Cc: Joel Fernandes, rcu Le Sun, Sep 10, 2023 at 03:21:33AM -0700, Paul E. McKenney a écrit : > On Sat, Sep 09, 2023 at 10:28:36PM -0400, Joel Fernandes wrote: > > Hi Paul, Frederic, ever since this recent patch got merged into all > > recent stables: > > https://lore.kernel.org/all/20230818200757.1808398-1-paul.gortmaker@windriver.com/ > > .. > > > > .. I am seeing this happen a lot now causing rcutorture failures: > > "NOHZ tick-stop error: local softirq work is pending" > > > > It looks like the "tick-stop error" as local_bh_blocked always returns > > false on !PREEMPT_RT, so that patch itself seems correct. > > > > However, it appears the string in console-badness.sh is different: > > > > console-badness.sh:grep -v 'NOHZ tick-stop error: Non-RCU local > > softirq work is pending, handler' > > > > So the string in the script needs an update? > > Looks like it, though I believe that there is a fix in the offing, > at which point that "grep -v" should just be removed. Depending on > timing, of course. Yep I have a pending fix for that, will post soonish. The grep will still make sense afterward though. Thanks. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [BUG] NOHZ tick-stop error: local softirq work is pending 2023-09-10 20:25 ` Frederic Weisbecker @ 2023-09-10 21:48 ` Paul E. McKenney 2023-09-10 23:02 ` Joel Fernandes 1 sibling, 0 replies; 6+ messages in thread From: Paul E. McKenney @ 2023-09-10 21:48 UTC (permalink / raw) To: Frederic Weisbecker; +Cc: Joel Fernandes, rcu On Sun, Sep 10, 2023 at 10:25:43PM +0200, Frederic Weisbecker wrote: > Le Sun, Sep 10, 2023 at 03:21:33AM -0700, Paul E. McKenney a écrit : > > On Sat, Sep 09, 2023 at 10:28:36PM -0400, Joel Fernandes wrote: > > > Hi Paul, Frederic, ever since this recent patch got merged into all > > > recent stables: > > > https://lore.kernel.org/all/20230818200757.1808398-1-paul.gortmaker@windriver.com/ > > > .. > > > > > > .. I am seeing this happen a lot now causing rcutorture failures: > > > "NOHZ tick-stop error: local softirq work is pending" > > > > > > It looks like the "tick-stop error" as local_bh_blocked always returns > > > false on !PREEMPT_RT, so that patch itself seems correct. > > > > > > However, it appears the string in console-badness.sh is different: > > > > > > console-badness.sh:grep -v 'NOHZ tick-stop error: Non-RCU local > > > softirq work is pending, handler' > > > > > > So the string in the script needs an update? > > > > Looks like it, though I believe that there is a fix in the offing, > > at which point that "grep -v" should just be removed. Depending on > > timing, of course. > > Yep I have a pending fix for that, will post soonish. The grep will still > make sense afterward though. So I do nothing? I can do that! ;-) Thanx, Paul ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [BUG] NOHZ tick-stop error: local softirq work is pending 2023-09-10 20:25 ` Frederic Weisbecker 2023-09-10 21:48 ` Paul E. McKenney @ 2023-09-10 23:02 ` Joel Fernandes 2023-09-11 21:23 ` Frederic Weisbecker 1 sibling, 1 reply; 6+ messages in thread From: Joel Fernandes @ 2023-09-10 23:02 UTC (permalink / raw) To: Frederic Weisbecker; +Cc: Paul E. McKenney, rcu On Sun, Sep 10, 2023 at 10:25:43PM +0200, Frederic Weisbecker wrote: > Le Sun, Sep 10, 2023 at 03:21:33AM -0700, Paul E. McKenney a écrit : > > On Sat, Sep 09, 2023 at 10:28:36PM -0400, Joel Fernandes wrote: > > > Hi Paul, Frederic, ever since this recent patch got merged into all > > > recent stables: > > > https://lore.kernel.org/all/20230818200757.1808398-1-paul.gortmaker@windriver.com/ > > > .. > > > > > > .. I am seeing this happen a lot now causing rcutorture failures: > > > "NOHZ tick-stop error: local softirq work is pending" > > > > > > It looks like the "tick-stop error" as local_bh_blocked always returns > > > false on !PREEMPT_RT, so that patch itself seems correct. > > > > > > However, it appears the string in console-badness.sh is different: > > > > > > console-badness.sh:grep -v 'NOHZ tick-stop error: Non-RCU local > > > softirq work is pending, handler' > > > > > > So the string in the script needs an update? > > > > Looks like it, though I believe that there is a fix in the offing, > > at which point that "grep -v" should just be removed. Depending on > > timing, of course. > > Yep I have a pending fix for that, will post soonish. The grep will still > make sense afterward though. I was wondering why the grep -v will still make sense if the pending fix works. Or is there another legitimate reason for the string to appear? Either way, looking forward to the patch! thanks, - Joel ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [BUG] NOHZ tick-stop error: local softirq work is pending 2023-09-10 23:02 ` Joel Fernandes @ 2023-09-11 21:23 ` Frederic Weisbecker 0 siblings, 0 replies; 6+ messages in thread From: Frederic Weisbecker @ 2023-09-11 21:23 UTC (permalink / raw) To: Joel Fernandes; +Cc: Paul E. McKenney, rcu Le Sun, Sep 10, 2023 at 11:02:09PM +0000, Joel Fernandes a écrit : > On Sun, Sep 10, 2023 at 10:25:43PM +0200, Frederic Weisbecker wrote: > > Le Sun, Sep 10, 2023 at 03:21:33AM -0700, Paul E. McKenney a écrit : > > > On Sat, Sep 09, 2023 at 10:28:36PM -0400, Joel Fernandes wrote: > > > > Hi Paul, Frederic, ever since this recent patch got merged into all > > > > recent stables: > > > > https://lore.kernel.org/all/20230818200757.1808398-1-paul.gortmaker@windriver.com/ > > > > .. > > > > > > > > .. I am seeing this happen a lot now causing rcutorture failures: > > > > "NOHZ tick-stop error: local softirq work is pending" > > > > > > > > It looks like the "tick-stop error" as local_bh_blocked always returns > > > > false on !PREEMPT_RT, so that patch itself seems correct. > > > > > > > > However, it appears the string in console-badness.sh is different: > > > > > > > > console-badness.sh:grep -v 'NOHZ tick-stop error: Non-RCU local > > > > softirq work is pending, handler' > > > > > > > > So the string in the script needs an update? > > > > > > Looks like it, though I believe that there is a fix in the offing, > > > at which point that "grep -v" should just be removed. Depending on > > > timing, of course. > > > > Yep I have a pending fix for that, will post soonish. The grep will still > > make sense afterward though. > > I was wondering why the grep -v will still make sense if the pending fix > works. Or is there another legitimate reason for the string to appear? Because my fix will only take out (hr)timer softirqs out of the report. The others will still be considered buggy if pending while the CPU goes down. Anyway, cooking the timer patchset to post. Thanks. ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2023-09-12 4:17 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2023-09-10 2:28 [BUG] NOHZ tick-stop error: local softirq work is pending Joel Fernandes 2023-09-10 10:21 ` Paul E. McKenney 2023-09-10 20:25 ` Frederic Weisbecker 2023-09-10 21:48 ` Paul E. McKenney 2023-09-10 23:02 ` Joel Fernandes 2023-09-11 21:23 ` Frederic Weisbecker
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.