All of lore.kernel.org
 help / color / mirror / Atom feed
From: Daniel Lezcano <daniel.lezcano@linaro.org>
To: Paul McKenney <Paul.McKenney@us.ibm.com>, john.stultz@linaro.org
Cc: LKML <linux-kernel@vger.kernel.org>
Subject: RCU stall when using function_graph
Date: Tue, 1 Aug 2017 23:07:09 +0200	[thread overview]
Message-ID: <20170801210709.GC1919@mai> (raw)

Hi Paul,

I have been trying to set the function_graph tracer for ftrace and each time I
get a CPU stall.

How to reproduce:
-----------------

	echo function_graph > /sys/kernel/debug/tracing/current_tracer

This error appears with v4.13-rc3 and v4.12-rc6.

Is it something already reported ?

[ ... ]

[  472.934986] INFO: rcu_preempt detected stalls on CPUs/tasks:
[  472.941302]  0-...: (3 GPs behind) idle=d1a/2/0 softirq=2556/2557 fqs=3195
[  472.948982]  (detected by 3, t=6502 jiffies, g=549, c=548, q=10)
[  472.955180] Task dump for CPU 0:
[  472.958513] swapper/0       R  running task        0     0      0 0x00000002
[  550.954786] INFO: rcu_preempt detected stalls on CPUs/tasks:
[  550.960657]  0-...: (3 GPs behind) idle=d1a/2/0 softirq=2556/2557 fqs=12926
[  550.967897]  (detected by 3, t=26007 jiffies, g=549, c=548, q=27)
[  550.974181] Task dump for CPU 0:
[  550.977513] swapper/0       R  running task        0     0      0 0x00000002
[  628.974788] INFO: rcu_preempt detected stalls on CPUs/tasks:
[  628.980659]  0-...: (3 GPs behind) idle=d1a/2/0 softirq=2556/2557 fqs=22666
[  628.987900]  (detected by 3, t=45512 jiffies, g=549, c=548, q=43)
[  628.994184] Task dump for CPU 0:
[  628.997518] swapper/0       R  running task        0     0      0 0x00000002
[  706.994787] INFO: rcu_preempt detected stalls on CPUs/tasks:
[  707.000658]  0-...: (3 GPs behind) idle=d1a/2/0 softirq=2556/2557 fqs=31785
[  707.007899]  (detected by 3, t=65017 jiffies, g=549, c=548, q=74)
[  707.014183] Task dump for CPU 0:
[  707.017516] swapper/0       R  running task        0     0      0 0x00000002
[  707.024902] rcu_preempt kthread starved for 1261 jiffies! g549 c548 f0x0 RCU_GP_WAIT_FQS(3) ->state=0x1
[  707.034548] rcu_preempt     S    0     8      2 0x00000000
[  726.049583] INFO: task jbd2/mmcblk1p2-:1120 blocked for more than 120 seconds.
[  726.061772]       Not tainted 4.13.0-rc3-ktest-linaro-hikey #202
[  726.074294] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  726.089027] jbd2/mmcblk1p2- D    0  1120      2 0x00000000
[  726.126737] INFO: task rs:main Q:Reg:2272 blocked for more than 120 seconds.
[  726.140484]       Not tainted 4.13.0-rc3-ktest-linaro-hikey #202
[  726.154515] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  726.170140] rs:main Q:Reg   D    0  2272      1 0x00000000
[  726.205626]
[  726.205626] Showing all locks held in the system:
[  726.272952]
[  726.278194] =============================================
[  726.278194]
[  785.014779] INFO: rcu_preempt detected stalls on CPUs/tasks:
[  785.020658]  0-...: (3 GPs behind) idle=d1a/2/0 softirq=2556/2557 fqs=41387
[  785.027905]  (detected by 3, t=84524 jiffies, g=549, c=548, q=94)
[  785.034194] Task dump for CPU 0:
[  785.037532] swapper/0       R  running task        0     0      0 0x00000002
[  846.879753] INFO: task jbd2/mmcblk1p2-:1120 blocked for more than 120 seconds.
[  846.892966]       Not tainted 4.13.0-rc3-ktest-linaro-hikey #202
[  846.909597] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  846.926726] jbd2/mmcblk1p2- D    0  1120      2 0x00000000
[  846.981356] INFO: task rs:main Q:Reg:2272 blocked for more than 120 seconds.
[  847.001593]       Not tainted 4.13.0-rc3-ktest-linaro-hikey #202
[  847.017642] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  847.034135] rs:main Q:Reg   D    0  2272      1 0x00000000
[  847.077209]
[  847.077209] Showing all locks held in the system:
[  847.180736]
[  847.186753] =============================================
[  847.186753]
[  863.034778] INFO: rcu_preempt detected stalls on CPUs/tasks:
[  863.040660]  0-...: (3 GPs behind) idle=d1a/2/0 softirq=2556/2557 fqs=50881
[  863.047907]  (detected by 4, t=104030 jiffies, g=549, c=548, q=114)
[  863.054373] Task dump for CPU 0:
[  863.057711] swapper/0       R  running task        0     0      0 0x00000002
[  941.054780] INFO: rcu_preempt detected stalls on CPUs/tasks:
[  941.060659]  0-...: (3 GPs behind) idle=d1a/2/0 softirq=2556/2557 fqs=59726
[  941.067903]  (detected by 7, t=123535 jiffies, g=549, c=548, q=303)
[  941.074370] Task dump for CPU 0:
[  941.077708] swapper/0       R  running task        0     0      0 0x00000002
[  967.711682] INFO: task jbd2/mmcblk1p2-:1120 blocked for more than 120 seconds.
[  967.725731]       Not tainted 4.13.0-rc3-ktest-linaro-hikey #202
[  967.743347] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  967.765658] jbd2/mmcblk1p2- D    0  1120      2 0x00000000
[  967.810595] INFO: task rs:main Q:Reg:2272 blocked for more than 120 seconds.
[  967.825360]       Not tainted 4.13.0-rc3-ktest-linaro-hikey #202
[  967.838595] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  967.854460] rs:main Q:Reg   D    0  2272      1 0x00000000
[  967.897804]
[  967.897804] Showing all locks held in the system:
[  967.997067]
[  968.002570] =============================================


[ ... ]

etc ...


-- 

 <http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs

Follow Linaro:  <http://www.facebook.com/pages/Linaro> Facebook |
<http://twitter.com/#!/linaroorg> Twitter |
<http://www.linaro.org/linaro-blog/> Blog

             reply	other threads:[~2017-08-01 21:07 UTC|newest]

Thread overview: 29+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-08-01 21:07 Daniel Lezcano [this message]
  -- strict thread matches above, loose matches on Subject: below --
2017-08-01 22:04 RCU stall when using function_graph Paul E. McKenney
2017-08-01 22:15 ` Daniel Lezcano
2017-08-02  0:12   ` Steven Rostedt
2017-08-02 12:42     ` Daniel Lezcano
2017-08-02 13:07       ` Steven Rostedt
2017-08-03  2:40         ` Paul E. McKenney
2017-08-03 11:41         ` Daniel Lezcano
2017-08-03 12:44           ` Paul E. McKenney
2017-08-03 14:38             ` Daniel Lezcano
     [not found]               ` <CAOoBcBXo-=VYy2+TYEp=8+WSkOpDBr1x6uY=-r_GnTFKctXndQ@mail.gmail.com>
     [not found]                 ` <CAOoBcBVKpQkAVXji5qQu8r8GErqxpy9Ae9N97NhGpOQPgXudZg@mail.gmail.com>
     [not found]                   ` <CAOoBcBU00VRXmrNNEOjJHgXf9BimxKYOorJC0d3766mNdda=Bg@mail.gmail.com>
2017-08-06 17:02                     ` Paul E. McKenney
2017-08-09  9:13                       ` Pratyush Anand
2017-08-09 12:58                         ` Paul E. McKenney
2017-08-09 13:28                           ` Daniel Lezcano
2017-08-09 14:40                             ` Paul E. McKenney
2017-08-09 15:51                               ` Daniel Lezcano
2017-08-09 17:22                                 ` Paul E. McKenney
2017-08-10  9:45                                   ` Daniel Lezcano
2017-08-10 21:39                                     ` Paul E. McKenney
2017-08-11  9:38                                       ` Daniel Lezcano
2017-08-15 13:29                                 ` Steven Rostedt
2017-08-16  8:42                                   ` Daniel Lezcano
2017-08-16 14:04                                     ` Steven Rostedt
2017-08-16 16:32                                       ` Paul E. McKenney
2017-08-16 16:41                                         ` Steven Rostedt
2017-08-16 17:58                                           ` Paul E. McKenney
2017-08-30 22:07                                             ` Paul E. McKenney
2017-08-02 16:51       ` Paul E. McKenney
2017-08-02 12:49 ` Paul E. McKenney

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170801210709.GC1919@mai \
    --to=daniel.lezcano@linaro.org \
    --cc=Paul.McKenney@us.ibm.com \
    --cc=john.stultz@linaro.org \
    --cc=linux-kernel@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.