public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed
From: Daniel Lezcano <daniel.lezcano@linaro.org>
To: Paul McKenney <Paul.McKenney@us.ibm.com>, john.stultz@linaro.org
Cc: LKML <linux-kernel@vger.kernel.org>
Subject: RCU stall when using function_graph
Date: Tue, 1 Aug 2017 23:07:09 +0200	[thread overview]
Message-ID: <20170801210709.GC1919@mai> (raw)

Hi Paul,

I have been trying to set the function_graph tracer for ftrace and each time I
get a CPU stall.

How to reproduce:
-----------------

	echo function_graph > /sys/kernel/debug/tracing/current_tracer

This error appears with v4.13-rc3 and v4.12-rc6.

Is it something already reported ?

[ ... ]

[  472.934986] INFO: rcu_preempt detected stalls on CPUs/tasks:
[  472.941302]  0-...: (3 GPs behind) idle=d1a/2/0 softirq=2556/2557 fqs=3195
[  472.948982]  (detected by 3, t=6502 jiffies, g=549, c=548, q=10)
[  472.955180] Task dump for CPU 0:
[  472.958513] swapper/0       R  running task        0     0      0 0x00000002
[  550.954786] INFO: rcu_preempt detected stalls on CPUs/tasks:
[  550.960657]  0-...: (3 GPs behind) idle=d1a/2/0 softirq=2556/2557 fqs=12926
[  550.967897]  (detected by 3, t=26007 jiffies, g=549, c=548, q=27)
[  550.974181] Task dump for CPU 0:
[  550.977513] swapper/0       R  running task        0     0      0 0x00000002
[  628.974788] INFO: rcu_preempt detected stalls on CPUs/tasks:
[  628.980659]  0-...: (3 GPs behind) idle=d1a/2/0 softirq=2556/2557 fqs=22666
[  628.987900]  (detected by 3, t=45512 jiffies, g=549, c=548, q=43)
[  628.994184] Task dump for CPU 0:
[  628.997518] swapper/0       R  running task        0     0      0 0x00000002
[  706.994787] INFO: rcu_preempt detected stalls on CPUs/tasks:
[  707.000658]  0-...: (3 GPs behind) idle=d1a/2/0 softirq=2556/2557 fqs=31785
[  707.007899]  (detected by 3, t=65017 jiffies, g=549, c=548, q=74)
[  707.014183] Task dump for CPU 0:
[  707.017516] swapper/0       R  running task        0     0      0 0x00000002
[  707.024902] rcu_preempt kthread starved for 1261 jiffies! g549 c548 f0x0 RCU_GP_WAIT_FQS(3) ->state=0x1
[  707.034548] rcu_preempt     S    0     8      2 0x00000000
[  726.049583] INFO: task jbd2/mmcblk1p2-:1120 blocked for more than 120 seconds.
[  726.061772]       Not tainted 4.13.0-rc3-ktest-linaro-hikey #202
[  726.074294] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  726.089027] jbd2/mmcblk1p2- D    0  1120      2 0x00000000
[  726.126737] INFO: task rs:main Q:Reg:2272 blocked for more than 120 seconds.
[  726.140484]       Not tainted 4.13.0-rc3-ktest-linaro-hikey #202
[  726.154515] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  726.170140] rs:main Q:Reg   D    0  2272      1 0x00000000
[  726.205626]
[  726.205626] Showing all locks held in the system:
[  726.272952]
[  726.278194] =============================================
[  726.278194]
[  785.014779] INFO: rcu_preempt detected stalls on CPUs/tasks:
[  785.020658]  0-...: (3 GPs behind) idle=d1a/2/0 softirq=2556/2557 fqs=41387
[  785.027905]  (detected by 3, t=84524 jiffies, g=549, c=548, q=94)
[  785.034194] Task dump for CPU 0:
[  785.037532] swapper/0       R  running task        0     0      0 0x00000002
[  846.879753] INFO: task jbd2/mmcblk1p2-:1120 blocked for more than 120 seconds.
[  846.892966]       Not tainted 4.13.0-rc3-ktest-linaro-hikey #202
[  846.909597] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  846.926726] jbd2/mmcblk1p2- D    0  1120      2 0x00000000
[  846.981356] INFO: task rs:main Q:Reg:2272 blocked for more than 120 seconds.
[  847.001593]       Not tainted 4.13.0-rc3-ktest-linaro-hikey #202
[  847.017642] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  847.034135] rs:main Q:Reg   D    0  2272      1 0x00000000
[  847.077209]
[  847.077209] Showing all locks held in the system:
[  847.180736]
[  847.186753] =============================================
[  847.186753]
[  863.034778] INFO: rcu_preempt detected stalls on CPUs/tasks:
[  863.040660]  0-...: (3 GPs behind) idle=d1a/2/0 softirq=2556/2557 fqs=50881
[  863.047907]  (detected by 4, t=104030 jiffies, g=549, c=548, q=114)
[  863.054373] Task dump for CPU 0:
[  863.057711] swapper/0       R  running task        0     0      0 0x00000002
[  941.054780] INFO: rcu_preempt detected stalls on CPUs/tasks:
[  941.060659]  0-...: (3 GPs behind) idle=d1a/2/0 softirq=2556/2557 fqs=59726
[  941.067903]  (detected by 7, t=123535 jiffies, g=549, c=548, q=303)
[  941.074370] Task dump for CPU 0:
[  941.077708] swapper/0       R  running task        0     0      0 0x00000002
[  967.711682] INFO: task jbd2/mmcblk1p2-:1120 blocked for more than 120 seconds.
[  967.725731]       Not tainted 4.13.0-rc3-ktest-linaro-hikey #202
[  967.743347] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  967.765658] jbd2/mmcblk1p2- D    0  1120      2 0x00000000
[  967.810595] INFO: task rs:main Q:Reg:2272 blocked for more than 120 seconds.
[  967.825360]       Not tainted 4.13.0-rc3-ktest-linaro-hikey #202
[  967.838595] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  967.854460] rs:main Q:Reg   D    0  2272      1 0x00000000
[  967.897804]
[  967.897804] Showing all locks held in the system:
[  967.997067]
[  968.002570] =============================================


[ ... ]

etc ...


-- 

 <http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs

Follow Linaro:  <http://www.facebook.com/pages/Linaro> Facebook |
<http://twitter.com/#!/linaroorg> Twitter |
<http://www.linaro.org/linaro-blog/> Blog

             reply	other threads:[~2017-08-01 21:07 UTC|newest]

Thread overview: 29+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-08-01 21:07 Daniel Lezcano [this message]
  -- strict thread matches above, loose matches on Subject: below --
2017-08-01 22:04 RCU stall when using function_graph Paul E. McKenney
2017-08-01 22:15 ` Daniel Lezcano
2017-08-02  0:12   ` Steven Rostedt
2017-08-02 12:42     ` Daniel Lezcano
2017-08-02 13:07       ` Steven Rostedt
2017-08-03  2:40         ` Paul E. McKenney
2017-08-03 11:41         ` Daniel Lezcano
2017-08-03 12:44           ` Paul E. McKenney
2017-08-03 14:38             ` Daniel Lezcano
     [not found]               ` <CAOoBcBXo-=VYy2+TYEp=8+WSkOpDBr1x6uY=-r_GnTFKctXndQ@mail.gmail.com>
     [not found]                 ` <CAOoBcBVKpQkAVXji5qQu8r8GErqxpy9Ae9N97NhGpOQPgXudZg@mail.gmail.com>
     [not found]                   ` <CAOoBcBU00VRXmrNNEOjJHgXf9BimxKYOorJC0d3766mNdda=Bg@mail.gmail.com>
2017-08-06 17:02                     ` Paul E. McKenney
2017-08-09  9:13                       ` Pratyush Anand
2017-08-09 12:58                         ` Paul E. McKenney
2017-08-09 13:28                           ` Daniel Lezcano
2017-08-09 14:40                             ` Paul E. McKenney
2017-08-09 15:51                               ` Daniel Lezcano
2017-08-09 17:22                                 ` Paul E. McKenney
2017-08-10  9:45                                   ` Daniel Lezcano
2017-08-10 21:39                                     ` Paul E. McKenney
2017-08-11  9:38                                       ` Daniel Lezcano
2017-08-15 13:29                                 ` Steven Rostedt
2017-08-16  8:42                                   ` Daniel Lezcano
2017-08-16 14:04                                     ` Steven Rostedt
2017-08-16 16:32                                       ` Paul E. McKenney
2017-08-16 16:41                                         ` Steven Rostedt
2017-08-16 17:58                                           ` Paul E. McKenney
2017-08-30 22:07                                             ` Paul E. McKenney
2017-08-02 16:51       ` Paul E. McKenney
2017-08-02 12:49 ` Paul E. McKenney

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170801210709.GC1919@mai \
    --to=daniel.lezcano@linaro.org \
    --cc=Paul.McKenney@us.ibm.com \
    --cc=john.stultz@linaro.org \
    --cc=linux-kernel@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox