* rcu_preempt running flat out on idle desktop. @ 2013-05-08 20:30 Dave Jones 2013-05-08 20:52 ` Paul E. McKenney 2013-05-09 21:02 ` Frederic Weisbecker 0 siblings, 2 replies; 18+ messages in thread From: Dave Jones @ 2013-05-08 20:30 UTC (permalink / raw) To: paulmck; +Cc: Linux Kernel On what should be an idle desktop, I'm seeing really strange things. The XFCE CPU meter shows both cores are running with no idle time. here's the top of the top(1) output.. PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 10 root 20 0 0 0 0 S 200.0 0.0 185301:36 rcu_preempt 553 root 20 0 268m 76m 6764 S 200.0 2.6 144579:53 Xorg 1199 root 20 0 0 0 0 S 200.0 0.0 306:17.85 kworker/1:0 501 root 20 0 0 0 0 S 200.0 0.0 4471:03 kworker/0:2 12 root 20 0 0 0 0 S 200.0 0.0 67277:16 rcuop/1 1237 davej 20 0 535m 15m 8484 S 200.0 0.5 3645:16 Terminal 859 davej 20 0 117m 3036 1336 S 200.0 0.1 1986:55 htop There are a lot of processes allegedly using "200%" of CPU time, a handful in the "196%" range, and then a bunch at 0. rcu_preempt is staying at the top of the list while everything else bubbles up the list. And what's up with those TIME entries? Box has only been up a few hours, so those numbers look really crazy. RCU options for this build are.. CONFIG_TREE_PREEMPT_RCU=y CONFIG_PREEMPT_RCU=y CONFIG_RCU_STALL_COMMON=y CONFIG_RCU_USER_QS=y CONFIG_RCU_FANOUT=64 CONFIG_RCU_FANOUT_LEAF=16 # CONFIG_RCU_FANOUT_EXACT is not set # CONFIG_RCU_FAST_NO_HZ is not set CONFIG_TREE_RCU_TRACE=y CONFIG_RCU_BOOST=y CONFIG_RCU_BOOST_PRIO=1 CONFIG_RCU_BOOST_DELAY=500 CONFIG_RCU_NOCB_CPU=y CONFIG_RCU_NOCB_CPU_ALL=y CONFIG_DEBUG_OBJECTS_RCU_HEAD=y CONFIG_PROVE_RCU=y # CONFIG_PROVE_RCU_REPEATEDLY is not set CONFIG_PROVE_RCU_DELAY=y CONFIG_SPARSE_RCU_POINTER=y # CONFIG_RCU_TORTURE_TEST is not set CONFIG_RCU_CPU_STALL_TIMEOUT=60 CONFIG_RCU_CPU_STALL_VERBOSE=y # CONFIG_RCU_CPU_STALL_INFO is not set CONFIG_RCU_TRACE=y Dave ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: rcu_preempt running flat out on idle desktop. 2013-05-08 20:30 rcu_preempt running flat out on idle desktop Dave Jones @ 2013-05-08 20:52 ` Paul E. McKenney 2013-05-08 20:56 ` Dave Jones 2013-05-09 21:02 ` Frederic Weisbecker 1 sibling, 1 reply; 18+ messages in thread From: Paul E. McKenney @ 2013-05-08 20:52 UTC (permalink / raw) To: Dave Jones, Linux Kernel On Wed, May 08, 2013 at 04:30:42PM -0400, Dave Jones wrote: > On what should be an idle desktop, I'm seeing really strange things. > > The XFCE CPU meter shows both cores are running with no idle time. > > here's the top of the top(1) output.. > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > 10 root 20 0 0 0 0 S 200.0 0.0 185301:36 rcu_preempt > 553 root 20 0 268m 76m 6764 S 200.0 2.6 144579:53 Xorg > 1199 root 20 0 0 0 0 S 200.0 0.0 306:17.85 kworker/1:0 > 501 root 20 0 0 0 0 S 200.0 0.0 4471:03 kworker/0:2 > 12 root 20 0 0 0 0 S 200.0 0.0 67277:16 rcuop/1 > 1237 davej 20 0 535m 15m 8484 S 200.0 0.5 3645:16 Terminal > 859 davej 20 0 117m 3036 1336 S 200.0 0.1 1986:55 htop > > There are a lot of processes allegedly using "200%" of CPU time, a handful > in the "196%" range, and then a bunch at 0. Yow!!! 185301 minutes is 128 days, which is a truly impressive amount of CPU time to accumulate in a few short hours. > rcu_preempt is staying at the top of the list while everything else bubbles > up the list. > > And what's up with those TIME entries? Box has only been up a few hours, > so those numbers look really crazy. > > RCU options for this build are.. > > CONFIG_TREE_PREEMPT_RCU=y > CONFIG_PREEMPT_RCU=y > CONFIG_RCU_STALL_COMMON=y > CONFIG_RCU_USER_QS=y > CONFIG_RCU_FANOUT=64 > CONFIG_RCU_FANOUT_LEAF=16 > # CONFIG_RCU_FANOUT_EXACT is not set > # CONFIG_RCU_FAST_NO_HZ is not set > CONFIG_TREE_RCU_TRACE=y > CONFIG_RCU_BOOST=y > CONFIG_RCU_BOOST_PRIO=1 > CONFIG_RCU_BOOST_DELAY=500 > CONFIG_RCU_NOCB_CPU=y > CONFIG_RCU_NOCB_CPU_ALL=y > CONFIG_DEBUG_OBJECTS_RCU_HEAD=y > CONFIG_PROVE_RCU=y > # CONFIG_PROVE_RCU_REPEATEDLY is not set > CONFIG_PROVE_RCU_DELAY=y > CONFIG_SPARSE_RCU_POINTER=y > # CONFIG_RCU_TORTURE_TEST is not set > CONFIG_RCU_CPU_STALL_TIMEOUT=60 > CONFIG_RCU_CPU_STALL_VERBOSE=y > # CONFIG_RCU_CPU_STALL_INFO is not set > CONFIG_RCU_TRACE=y This is 3.9, or Linus's current tree? I am guessing the latter, but figured I should ask. Thanx, Paul ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: rcu_preempt running flat out on idle desktop. 2013-05-08 20:52 ` Paul E. McKenney @ 2013-05-08 20:56 ` Dave Jones 2013-05-08 21:16 ` Paul E. McKenney 2013-05-09 9:04 ` Frederic Weisbecker 0 siblings, 2 replies; 18+ messages in thread From: Dave Jones @ 2013-05-08 20:56 UTC (permalink / raw) To: Paul E. McKenney; +Cc: Linux Kernel On Wed, May 08, 2013 at 01:52:58PM -0700, Paul E. McKenney wrote: > On Wed, May 08, 2013 at 04:30:42PM -0400, Dave Jones wrote: > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > > 10 root 20 0 0 0 0 S 200.0 0.0 185301:36 rcu_preempt > > 553 root 20 0 268m 76m 6764 S 200.0 2.6 144579:53 Xorg > > 1199 root 20 0 0 0 0 S 200.0 0.0 306:17.85 kworker/1:0 > > 501 root 20 0 0 0 0 S 200.0 0.0 4471:03 kworker/0:2 > > 12 root 20 0 0 0 0 S 200.0 0.0 67277:16 rcuop/1 > > 1237 davej 20 0 535m 15m 8484 S 200.0 0.5 3645:16 Terminal > > 859 davej 20 0 117m 3036 1336 S 200.0 0.1 1986:55 htop > > > > There are a lot of processes allegedly using "200%" of CPU time, a handful > > in the "196%" range, and then a bunch at 0. > > Yow!!! 185301 minutes is 128 days, which is a truly impressive amount of > CPU time to accumulate in a few short hours. perf top doesn't show anything out of the ordinary. Just as an idle desktop should behave, it's spending a bunch of time in delay_tsc. > This is 3.9, or Linus's current tree? I am guessing the latter, but > figured I should ask. Yeah, the latter. (v3.9-11572-g5af43c2) Dave ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: rcu_preempt running flat out on idle desktop. 2013-05-08 20:56 ` Dave Jones @ 2013-05-08 21:16 ` Paul E. McKenney 2013-05-08 21:17 ` Dave Jones 2013-05-09 9:04 ` Frederic Weisbecker 1 sibling, 1 reply; 18+ messages in thread From: Paul E. McKenney @ 2013-05-08 21:16 UTC (permalink / raw) To: Dave Jones, Linux Kernel On Wed, May 08, 2013 at 04:56:33PM -0400, Dave Jones wrote: > On Wed, May 08, 2013 at 01:52:58PM -0700, Paul E. McKenney wrote: > > On Wed, May 08, 2013 at 04:30:42PM -0400, Dave Jones wrote: > > > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > > > 10 root 20 0 0 0 0 S 200.0 0.0 185301:36 rcu_preempt > > > 553 root 20 0 268m 76m 6764 S 200.0 2.6 144579:53 Xorg > > > 1199 root 20 0 0 0 0 S 200.0 0.0 306:17.85 kworker/1:0 > > > 501 root 20 0 0 0 0 S 200.0 0.0 4471:03 kworker/0:2 > > > 12 root 20 0 0 0 0 S 200.0 0.0 67277:16 rcuop/1 > > > 1237 davej 20 0 535m 15m 8484 S 200.0 0.5 3645:16 Terminal > > > 859 davej 20 0 117m 3036 1336 S 200.0 0.1 1986:55 htop > > > > > > There are a lot of processes allegedly using "200%" of CPU time, a handful > > > in the "196%" range, and then a bunch at 0. > > > > Yow!!! 185301 minutes is 128 days, which is a truly impressive amount of > > CPU time to accumulate in a few short hours. > > perf top doesn't show anything out of the ordinary. Just as an idle desktop > should behave, it's spending a bunch of time in delay_tsc. > > > This is 3.9, or Linus's current tree? I am guessing the latter, but > > figured I should ask. > > Yeah, the latter. (v3.9-11572-g5af43c2) Do you have CONFIG_NO_HZ_FULL=y? We just might have a bug in tickless CPU-time accounting... Thanx, Paul ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: rcu_preempt running flat out on idle desktop. 2013-05-08 21:16 ` Paul E. McKenney @ 2013-05-08 21:17 ` Dave Jones 2013-05-08 22:20 ` Paul E. McKenney 0 siblings, 1 reply; 18+ messages in thread From: Dave Jones @ 2013-05-08 21:17 UTC (permalink / raw) To: Paul E. McKenney; +Cc: Linux Kernel On Wed, May 08, 2013 at 02:16:35PM -0700, Paul E. McKenney wrote: > On Wed, May 08, 2013 at 04:56:33PM -0400, Dave Jones wrote: > > On Wed, May 08, 2013 at 01:52:58PM -0700, Paul E. McKenney wrote: > > > On Wed, May 08, 2013 at 04:30:42PM -0400, Dave Jones wrote: > > > > > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > > > > 10 root 20 0 0 0 0 S 200.0 0.0 185301:36 rcu_preempt > > > > 553 root 20 0 268m 76m 6764 S 200.0 2.6 144579:53 Xorg > > > > 1199 root 20 0 0 0 0 S 200.0 0.0 306:17.85 kworker/1:0 > > > > 501 root 20 0 0 0 0 S 200.0 0.0 4471:03 kworker/0:2 > > > > 12 root 20 0 0 0 0 S 200.0 0.0 67277:16 rcuop/1 > > > > 1237 davej 20 0 535m 15m 8484 S 200.0 0.5 3645:16 Terminal > > > > 859 davej 20 0 117m 3036 1336 S 200.0 0.1 1986:55 htop > > > > > > > > There are a lot of processes allegedly using "200%" of CPU time, a handful > > > > in the "196%" range, and then a bunch at 0. > > > > > > Yow!!! 185301 minutes is 128 days, which is a truly impressive amount of > > > CPU time to accumulate in a few short hours. > > > > perf top doesn't show anything out of the ordinary. Just as an idle desktop > > should behave, it's spending a bunch of time in delay_tsc. > > > > > This is 3.9, or Linus's current tree? I am guessing the latter, but > > > figured I should ask. > > > > Yeah, the latter. (v3.9-11572-g5af43c2) > > Do you have CONFIG_NO_HZ_FULL=y? We just might have a bug in tickless > CPU-time accounting... I never saw a new config option I didn't like. CONFIG_NO_HZ_FULL=y CONFIG_NO_HZ_FULL_ALL=y Dave ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: rcu_preempt running flat out on idle desktop. 2013-05-08 21:17 ` Dave Jones @ 2013-05-08 22:20 ` Paul E. McKenney 0 siblings, 0 replies; 18+ messages in thread From: Paul E. McKenney @ 2013-05-08 22:20 UTC (permalink / raw) To: Dave Jones, Linux Kernel; +Cc: fweisbec On Wed, May 08, 2013 at 05:17:59PM -0400, Dave Jones wrote: > On Wed, May 08, 2013 at 02:16:35PM -0700, Paul E. McKenney wrote: > > On Wed, May 08, 2013 at 04:56:33PM -0400, Dave Jones wrote: > > > On Wed, May 08, 2013 at 01:52:58PM -0700, Paul E. McKenney wrote: > > > > On Wed, May 08, 2013 at 04:30:42PM -0400, Dave Jones wrote: > > > > > > > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > > > > > 10 root 20 0 0 0 0 S 200.0 0.0 185301:36 rcu_preempt > > > > > 553 root 20 0 268m 76m 6764 S 200.0 2.6 144579:53 Xorg > > > > > 1199 root 20 0 0 0 0 S 200.0 0.0 306:17.85 kworker/1:0 > > > > > 501 root 20 0 0 0 0 S 200.0 0.0 4471:03 kworker/0:2 > > > > > 12 root 20 0 0 0 0 S 200.0 0.0 67277:16 rcuop/1 > > > > > 1237 davej 20 0 535m 15m 8484 S 200.0 0.5 3645:16 Terminal > > > > > 859 davej 20 0 117m 3036 1336 S 200.0 0.1 1986:55 htop > > > > > > > > > > There are a lot of processes allegedly using "200%" of CPU time, a handful > > > > > in the "196%" range, and then a bunch at 0. > > > > > > > > Yow!!! 185301 minutes is 128 days, which is a truly impressive amount of > > > > CPU time to accumulate in a few short hours. > > > > > > perf top doesn't show anything out of the ordinary. Just as an idle desktop > > > should behave, it's spending a bunch of time in delay_tsc. > > > > > > > This is 3.9, or Linus's current tree? I am guessing the latter, but > > > > figured I should ask. > > > > > > Yeah, the latter. (v3.9-11572-g5af43c2) > > > > Do you have CONFIG_NO_HZ_FULL=y? We just might have a bug in tickless > > CPU-time accounting... > > I never saw a new config option I didn't like. ;-) ;-) ;-) > CONFIG_NO_HZ_FULL=y > CONFIG_NO_HZ_FULL_ALL=y Adding Frederic on CC... Thanx, Paul ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: rcu_preempt running flat out on idle desktop. 2013-05-08 20:56 ` Dave Jones 2013-05-08 21:16 ` Paul E. McKenney @ 2013-05-09 9:04 ` Frederic Weisbecker 2013-05-09 9:20 ` Ingo Molnar 1 sibling, 1 reply; 18+ messages in thread From: Frederic Weisbecker @ 2013-05-09 9:04 UTC (permalink / raw) To: Dave Jones, Paul E. McKenney, Linux Kernel, Ingo Molnar 2013/5/8 Dave Jones <davej@redhat.com>: > On Wed, May 08, 2013 at 01:52:58PM -0700, Paul E. McKenney wrote: > > On Wed, May 08, 2013 at 04:30:42PM -0400, Dave Jones wrote: > > > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > > > 10 root 20 0 0 0 0 S 200.0 0.0 185301:36 rcu_preempt > > > 553 root 20 0 268m 76m 6764 S 200.0 2.6 144579:53 Xorg > > > 1199 root 20 0 0 0 0 S 200.0 0.0 306:17.85 kworker/1:0 > > > 501 root 20 0 0 0 0 S 200.0 0.0 4471:03 kworker/0:2 > > > 12 root 20 0 0 0 0 S 200.0 0.0 67277:16 rcuop/1 > > > 1237 davej 20 0 535m 15m 8484 S 200.0 0.5 3645:16 Terminal > > > 859 davej 20 0 117m 3036 1336 S 200.0 0.1 1986:55 htop > > > > > > There are a lot of processes allegedly using "200%" of CPU time, a handful > > > in the "196%" range, and then a bunch at 0. > > > > Yow!!! 185301 minutes is 128 days, which is a truly impressive amount of > > CPU time to accumulate in a few short hours. Right, Ingo has reported something about strange high numbers in top too. I need to investigate that. Thanks. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: rcu_preempt running flat out on idle desktop. 2013-05-09 9:04 ` Frederic Weisbecker @ 2013-05-09 9:20 ` Ingo Molnar 0 siblings, 0 replies; 18+ messages in thread From: Ingo Molnar @ 2013-05-09 9:20 UTC (permalink / raw) To: Frederic Weisbecker; +Cc: Dave Jones, Paul E. McKenney, Linux Kernel * Frederic Weisbecker <fweisbec@gmail.com> wrote: > 2013/5/8 Dave Jones <davej@redhat.com>: > > On Wed, May 08, 2013 at 01:52:58PM -0700, Paul E. McKenney wrote: > > > On Wed, May 08, 2013 at 04:30:42PM -0400, Dave Jones wrote: > > > > > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > > > > 10 root 20 0 0 0 0 S 200.0 0.0 185301:36 rcu_preempt > > > > 553 root 20 0 268m 76m 6764 S 200.0 2.6 144579:53 Xorg > > > > 1199 root 20 0 0 0 0 S 200.0 0.0 306:17.85 kworker/1:0 > > > > 501 root 20 0 0 0 0 S 200.0 0.0 4471:03 kworker/0:2 > > > > 12 root 20 0 0 0 0 S 200.0 0.0 67277:16 rcuop/1 > > > > 1237 davej 20 0 535m 15m 8484 S 200.0 0.5 3645:16 Terminal > > > > 859 davej 20 0 117m 3036 1336 S 200.0 0.1 1986:55 htop > > > > > > > > There are a lot of processes allegedly using "200%" of CPU time, a handful > > > > in the "196%" range, and then a bunch at 0. > > > > > > Yow!!! 185301 minutes is 128 days, which is a truly impressive amount of > > > CPU time to accumulate in a few short hours. > > Right, Ingo has reported something about strange high numbers in top > too. I need to investigate that. For CPU-intense loops I have seen a small anomaly with "slightly higher CPU usage numbers" - sometimes going to 101% or so but still converging to around 99.9% in the long run. The 200% bug pasted by Dave here seems to be very big and seems to affect all tasks not just true CPU-loopers. So I'd suggest reproducing and fixing Dave's bug first - that might fix the small anomaly I've seen. Thanks, Ingo ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: rcu_preempt running flat out on idle desktop. 2013-05-08 20:30 rcu_preempt running flat out on idle desktop Dave Jones 2013-05-08 20:52 ` Paul E. McKenney @ 2013-05-09 21:02 ` Frederic Weisbecker 2013-05-09 21:10 ` Dave Jones 1 sibling, 1 reply; 18+ messages in thread From: Frederic Weisbecker @ 2013-05-09 21:02 UTC (permalink / raw) To: Dave Jones, paulmck, Linux Kernel; +Cc: Ingo Molnar 2013/5/8 Dave Jones <davej@redhat.com>: > On what should be an idle desktop, I'm seeing really strange things. > > The XFCE CPU meter shows both cores are running with no idle time. > > here's the top of the top(1) output.. > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > 10 root 20 0 0 0 0 S 200.0 0.0 185301:36 rcu_preempt > 553 root 20 0 268m 76m 6764 S 200.0 2.6 144579:53 Xorg > 1199 root 20 0 0 0 0 S 200.0 0.0 306:17.85 kworker/1:0 > 501 root 20 0 0 0 0 S 200.0 0.0 4471:03 kworker/0:2 > 12 root 20 0 0 0 0 S 200.0 0.0 67277:16 rcuop/1 > 1237 davej 20 0 535m 15m 8484 S 200.0 0.5 3645:16 Terminal > 859 davej 20 0 117m 3036 1336 S 200.0 0.1 1986:55 htop > > There are a lot of processes allegedly using "200%" of CPU time, a handful > in the "196%" range, and then a bunch at 0. > > rcu_preempt is staying at the top of the list while everything else bubbles > up the list. > > And what's up with those TIME entries? Box has only been up a few hours, > so those numbers look really crazy. > > RCU options for this build are.. > > CONFIG_TREE_PREEMPT_RCU=y > CONFIG_PREEMPT_RCU=y > CONFIG_RCU_STALL_COMMON=y > CONFIG_RCU_USER_QS=y > CONFIG_RCU_FANOUT=64 > CONFIG_RCU_FANOUT_LEAF=16 > # CONFIG_RCU_FANOUT_EXACT is not set > # CONFIG_RCU_FAST_NO_HZ is not set > CONFIG_TREE_RCU_TRACE=y > CONFIG_RCU_BOOST=y > CONFIG_RCU_BOOST_PRIO=1 > CONFIG_RCU_BOOST_DELAY=500 > CONFIG_RCU_NOCB_CPU=y > CONFIG_RCU_NOCB_CPU_ALL=y > CONFIG_DEBUG_OBJECTS_RCU_HEAD=y > CONFIG_PROVE_RCU=y > # CONFIG_PROVE_RCU_REPEATEDLY is not set > CONFIG_PROVE_RCU_DELAY=y > CONFIG_SPARSE_RCU_POINTER=y > # CONFIG_RCU_TORTURE_TEST is not set > CONFIG_RCU_CPU_STALL_TIMEOUT=60 > CONFIG_RCU_CPU_STALL_VERBOSE=y > # CONFIG_RCU_CPU_STALL_INFO is not set > CONFIG_RCU_TRACE=y I can't reproduce that issue even with the above setting. Could you please send me your whole config? Thanks. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: rcu_preempt running flat out on idle desktop. 2013-05-09 21:02 ` Frederic Weisbecker @ 2013-05-09 21:10 ` Dave Jones 2013-05-14 1:21 ` Frederic Weisbecker 0 siblings, 1 reply; 18+ messages in thread From: Dave Jones @ 2013-05-09 21:10 UTC (permalink / raw) To: Frederic Weisbecker; +Cc: paulmck, Linux Kernel, Ingo Molnar On Thu, May 09, 2013 at 11:02:08PM +0200, Frederic Weisbecker wrote: > > RCU options for this build are.. > > > I can't reproduce that issue even with the above setting. Could you > please send me your whole config? http://paste.fedoraproject.org/11363/36813375/raw/ Dave ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: rcu_preempt running flat out on idle desktop. 2013-05-09 21:10 ` Dave Jones @ 2013-05-14 1:21 ` Frederic Weisbecker 2013-06-06 15:31 ` Dave Jones 0 siblings, 1 reply; 18+ messages in thread From: Frederic Weisbecker @ 2013-05-14 1:21 UTC (permalink / raw) To: Dave Jones, paulmck, Linux Kernel, Ingo Molnar On Thu, May 09, 2013 at 05:10:26PM -0400, Dave Jones wrote: > On Thu, May 09, 2013 at 11:02:08PM +0200, Frederic Weisbecker wrote: > > > > RCU options for this build are.. > > > > > I can't reproduce that issue even with the above setting. Could you > > please send me your whole config? > > http://paste.fedoraproject.org/11363/36813375/raw/ Sorry it took me some time to get that config booting, I ran into other unrelated crashes. So I'm observing the exact opposite than you: the processes appear to have no load nor to consume cputime. Even if I use some dumb user loop. Top don't even show it. I'm digging further and will tell you when I get more info. Thanks. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: rcu_preempt running flat out on idle desktop. 2013-05-14 1:21 ` Frederic Weisbecker @ 2013-06-06 15:31 ` Dave Jones 2013-06-06 15:43 ` Frederic Weisbecker 0 siblings, 1 reply; 18+ messages in thread From: Dave Jones @ 2013-06-06 15:31 UTC (permalink / raw) To: Frederic Weisbecker; +Cc: paulmck, Linux Kernel, Ingo Molnar On Tue, May 14, 2013 at 03:21:07AM +0200, Frederic Weisbecker wrote: > On Thu, May 09, 2013 at 05:10:26PM -0400, Dave Jones wrote: > > On Thu, May 09, 2013 at 11:02:08PM +0200, Frederic Weisbecker wrote: > > > > > > RCU options for this build are.. > > > > > > > I can't reproduce that issue even with the above setting. Could you > > > please send me your whole config? > > > > http://paste.fedoraproject.org/11363/36813375/raw/ > > Sorry it took me some time to get that config booting, I ran into > other unrelated crashes. > > So I'm observing the exact opposite than you: the processes appear to > have no load nor to consume cputime. Even if I use some dumb user loop. > Top don't even show it. > > I'm digging further and will tell you when I get more info. So I'm still seeing this on rc4. Here's my 'idle' firewall, just routing packets: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 4405 root 20 0 0 0 0 S 400.0 0.0 52336:27 kworker/2:0 20968 davej 20 0 115m 1840 1092 R 400.0 0.1 2297:48 top 964 named 20 0 383m 32m 2896 S 400.0 1.7 81384:54 named 10 root 20 0 0 0 0 S 400.0 0.0 195403:05 rcu_preempt 20830 davej 20 0 125m 2172 1024 S 400.0 0.1 939:36.10 sshd 11 root 20 0 0 0 0 S 400.0 0.0 42644:33 rcuop/0 13 root 20 0 0 0 0 S 400.0 0.0 34729:04 rcuop/2 754 lighttpd 20 0 54640 2692 1732 S 400.0 0.1 35286:11 lighttpd 756 root 20 0 216m 10m 6116 S 400.0 0.5 34143:41 snmpd 35 root 20 0 0 0 0 S 400.0 0.0 12790:56 ksoftirqd/2 19893 root 20 0 0 0 0 S 400.0 0.0 12570:49 kworker/0:1 20059 root 20 0 0 0 0 S 400.0 0.0 4666:30 kworker/1:1 32 root rt 0 0 0 0 S 400.0 0.0 8780:22 watchdog/2 792 root 20 0 92220 2516 632 S 400.0 0.1 7821:28 sendmail 25 root rt 0 0 0 0 S 400.0 0.0 8780:21 watchdog/0 26 root rt 0 0 0 0 S 400.0 0.0 8780:22 watchdog/1 38 root rt 0 0 0 0 S 400.0 0.0 8780:22 watchdog/3 19167 root 20 0 0 0 0 S 400.0 0.0 7804:04 kworker/3:2 Every process is either 400%, or 0%. Here's my desktop box, just writing this email: PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 10 root 20 0 0 0 0 S 200.0 0.0 35134:00 rcu_preempt 12 root 20 0 0 0 0 S 200.0 0.0 14059:47 rcuop/1 12528 root 20 0 0 0 0 S 200.0 0.0 1012:56 kworker/0:1 12999 davej 20 0 115m 1700 956 R 200.0 0.1 1:09.10 top 4799 root 20 0 0 0 0 S 200.0 0.0 3510:40 kworker/1:1 1 root 20 0 50760 5092 1648 S 0.0 0.2 3:14.96 systemd 2 root 20 0 0 0 0 S 0.0 0.0 1:40.55 kthreadd Every process 200% or 0%. Dave ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: rcu_preempt running flat out on idle desktop. 2013-06-06 15:31 ` Dave Jones @ 2013-06-06 15:43 ` Frederic Weisbecker 2013-06-06 23:10 ` Dave Jones 2013-06-12 15:34 ` Dave Jones 0 siblings, 2 replies; 18+ messages in thread From: Frederic Weisbecker @ 2013-06-06 15:43 UTC (permalink / raw) To: Dave Jones, paulmck, Linux Kernel, Ingo Molnar On Thu, Jun 06, 2013 at 11:31:36AM -0400, Dave Jones wrote: > On Tue, May 14, 2013 at 03:21:07AM +0200, Frederic Weisbecker wrote: > > On Thu, May 09, 2013 at 05:10:26PM -0400, Dave Jones wrote: > > > On Thu, May 09, 2013 at 11:02:08PM +0200, Frederic Weisbecker wrote: > > > > > > > > RCU options for this build are.. > > > > > > > > > I can't reproduce that issue even with the above setting. Could you > > > > please send me your whole config? > > > > > > http://paste.fedoraproject.org/11363/36813375/raw/ > > > > Sorry it took me some time to get that config booting, I ran into > > other unrelated crashes. > > > > So I'm observing the exact opposite than you: the processes appear to > > have no load nor to consume cputime. Even if I use some dumb user loop. > > Top don't even show it. > > > > I'm digging further and will tell you when I get more info. > > So I'm still seeing this on rc4. > > Here's my 'idle' firewall, just routing packets: > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > 4405 root 20 0 0 0 0 S 400.0 0.0 52336:27 kworker/2:0 > 20968 davej 20 0 115m 1840 1092 R 400.0 0.1 2297:48 top > 964 named 20 0 383m 32m 2896 S 400.0 1.7 81384:54 named > 10 root 20 0 0 0 0 S 400.0 0.0 195403:05 rcu_preempt > 20830 davej 20 0 125m 2172 1024 S 400.0 0.1 939:36.10 sshd > 11 root 20 0 0 0 0 S 400.0 0.0 42644:33 rcuop/0 > 13 root 20 0 0 0 0 S 400.0 0.0 34729:04 rcuop/2 > 754 lighttpd 20 0 54640 2692 1732 S 400.0 0.1 35286:11 lighttpd > 756 root 20 0 216m 10m 6116 S 400.0 0.5 34143:41 snmpd > 35 root 20 0 0 0 0 S 400.0 0.0 12790:56 ksoftirqd/2 > 19893 root 20 0 0 0 0 S 400.0 0.0 12570:49 kworker/0:1 > 20059 root 20 0 0 0 0 S 400.0 0.0 4666:30 kworker/1:1 > 32 root rt 0 0 0 0 S 400.0 0.0 8780:22 watchdog/2 > 792 root 20 0 92220 2516 632 S 400.0 0.1 7821:28 sendmail > 25 root rt 0 0 0 0 S 400.0 0.0 8780:21 watchdog/0 > 26 root rt 0 0 0 0 S 400.0 0.0 8780:22 watchdog/1 > 38 root rt 0 0 0 0 S 400.0 0.0 8780:22 watchdog/3 > 19167 root 20 0 0 0 0 S 400.0 0.0 7804:04 kworker/3:2 > > > Every process is either 400%, or 0%. > > Here's my desktop box, just writing this email: > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > 10 root 20 0 0 0 0 S 200.0 0.0 35134:00 rcu_preempt > 12 root 20 0 0 0 0 S 200.0 0.0 14059:47 rcuop/1 > 12528 root 20 0 0 0 0 S 200.0 0.0 1012:56 kworker/0:1 > 12999 davej 20 0 115m 1700 956 R 200.0 0.1 1:09.10 top > 4799 root 20 0 0 0 0 S 200.0 0.0 3510:40 kworker/1:1 > 1 root 20 0 50760 5092 1648 S 0.0 0.2 3:14.96 systemd > 2 root 20 0 0 0 0 S 0.0 0.0 1:40.55 kthreadd > > Every process 200% or 0%. I see, would you mind testing this branch? git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git timers/urgent It might help, I specially think about 45eacc692771bd2b1ea3d384e6345cab3da10861 ("vtime: Use consistent clocks among nohz accounting") which fixes an issue with cputime accounting that resulted in similar symptoms. Thanks. ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: rcu_preempt running flat out on idle desktop. 2013-06-06 15:43 ` Frederic Weisbecker @ 2013-06-06 23:10 ` Dave Jones 2013-06-12 15:34 ` Dave Jones 1 sibling, 0 replies; 18+ messages in thread From: Dave Jones @ 2013-06-06 23:10 UTC (permalink / raw) To: Frederic Weisbecker; +Cc: paulmck, Linux Kernel, Ingo Molnar On Thu, Jun 06, 2013 at 05:43:13PM +0200, Frederic Weisbecker wrote: > > Every process 200% or 0%. > > I see, would you mind testing this branch? > > git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git > timers/urgent > > It might help, I specially think about 45eacc692771bd2b1ea3d384e6345cab3da10861 > ("vtime: Use consistent clocks among nohz accounting") which fixes an issue > with cputime accounting that resulted in similar symptoms. > > Thanks. 5 hours of uptime so far, without anything looking weird. I'll keep an eye on it, but this is looking a lot better so far. Dave ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: rcu_preempt running flat out on idle desktop. 2013-06-06 15:43 ` Frederic Weisbecker 2013-06-06 23:10 ` Dave Jones @ 2013-06-12 15:34 ` Dave Jones 2013-06-20 19:10 ` Dave Jones 1 sibling, 1 reply; 18+ messages in thread From: Dave Jones @ 2013-06-12 15:34 UTC (permalink / raw) To: Frederic Weisbecker; +Cc: paulmck, Linux Kernel, Ingo Molnar On Thu, Jun 06, 2013 at 05:43:13PM +0200, Frederic Weisbecker wrote: > > Every process 200% or 0%. > > I see, would you mind testing this branch? > > git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git > timers/urgent > > It might help, I specially think about 45eacc692771bd2b1ea3d384e6345cab3da10861 > ("vtime: Use consistent clocks among nohz accounting") which fixes an issue > with cputime accounting that resulted in similar symptoms. So that patch has been in 'urgent' for a month. Can someone move it to really-urgent or whatever is necessary to get it into Linus' tree ? DAve ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: rcu_preempt running flat out on idle desktop. 2013-06-12 15:34 ` Dave Jones @ 2013-06-20 19:10 ` Dave Jones 2013-06-23 10:58 ` Ingo Molnar 0 siblings, 1 reply; 18+ messages in thread From: Dave Jones @ 2013-06-20 19:10 UTC (permalink / raw) To: Frederic Weisbecker, paulmck, Linux Kernel, Ingo Molnar, Linus Torvalds On Wed, Jun 12, 2013 at 11:34:07AM -0400, Dave Jones wrote: > On Thu, Jun 06, 2013 at 05:43:13PM +0200, Frederic Weisbecker wrote: > > > > Every process 200% or 0%. > > > > I see, would you mind testing this branch? > > > > git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git > > timers/urgent > > > > It might help, I specially think about 45eacc692771bd2b1ea3d384e6345cab3da10861 > > ("vtime: Use consistent clocks among nohz accounting") which fixes an issue > > with cputime accounting that resulted in similar symptoms. > > So that patch has been in 'urgent' for a month. Can someone move it > to really-urgent or whatever is necessary to get it into Linus' tree ? Another week passes... This is kind of ridiculous. Dave ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: rcu_preempt running flat out on idle desktop. 2013-06-20 19:10 ` Dave Jones @ 2013-06-23 10:58 ` Ingo Molnar 2013-07-01 12:10 ` Frederic Weisbecker 0 siblings, 1 reply; 18+ messages in thread From: Ingo Molnar @ 2013-06-23 10:58 UTC (permalink / raw) To: Dave Jones, Frederic Weisbecker, paulmck, Linux Kernel, Linus Torvalds * Dave Jones <davej@redhat.com> wrote: > On Wed, Jun 12, 2013 at 11:34:07AM -0400, Dave Jones wrote: > > On Thu, Jun 06, 2013 at 05:43:13PM +0200, Frederic Weisbecker wrote: > > > > > > Every process 200% or 0%. > > > > > > I see, would you mind testing this branch? > > > > > > git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git > > > timers/urgent > > > > > > It might help, I specially think about 45eacc692771bd2b1ea3d384e6345cab3da10861 > > > ("vtime: Use consistent clocks among nohz accounting") which fixes an issue > > > with cputime accounting that resulted in similar symptoms. > > > > So that patch has been in 'urgent' for a month. Can someone move it > > to really-urgent or whatever is necessary to get it into Linus' tree ? > > Another week passes... It was sent to Linus two days before your mail and Linus merged it at around the time you wrote your mail. So it should be all in -rc7, unless I'm missing something? Thanks, Ingo ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: rcu_preempt running flat out on idle desktop. 2013-06-23 10:58 ` Ingo Molnar @ 2013-07-01 12:10 ` Frederic Weisbecker 0 siblings, 0 replies; 18+ messages in thread From: Frederic Weisbecker @ 2013-07-01 12:10 UTC (permalink / raw) To: Ingo Molnar; +Cc: Dave Jones, paulmck, Linux Kernel, Linus Torvalds On Sun, Jun 23, 2013 at 12:58:39PM +0200, Ingo Molnar wrote: > > * Dave Jones <davej@redhat.com> wrote: > > > On Wed, Jun 12, 2013 at 11:34:07AM -0400, Dave Jones wrote: > > > On Thu, Jun 06, 2013 at 05:43:13PM +0200, Frederic Weisbecker wrote: > > > > > > > > Every process 200% or 0%. > > > > > > > > I see, would you mind testing this branch? > > > > > > > > git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git > > > > timers/urgent > > > > > > > > It might help, I specially think about 45eacc692771bd2b1ea3d384e6345cab3da10861 > > > > ("vtime: Use consistent clocks among nohz accounting") which fixes an issue > > > > with cputime accounting that resulted in similar symptoms. > > > > > > So that patch has been in 'urgent' for a month. Can someone move it > > > to really-urgent or whatever is necessary to get it into Linus' tree ? > > > > Another week passes... > > It was sent to Linus two days before your mail and Linus merged it at > around the time you wrote your mail. > > So it should be all in -rc7, unless I'm missing something? Ah sorry I missed that merge. It's all fine indeed. Thanks! ^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2013-07-01 12:11 UTC | newest] Thread overview: 18+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2013-05-08 20:30 rcu_preempt running flat out on idle desktop Dave Jones 2013-05-08 20:52 ` Paul E. McKenney 2013-05-08 20:56 ` Dave Jones 2013-05-08 21:16 ` Paul E. McKenney 2013-05-08 21:17 ` Dave Jones 2013-05-08 22:20 ` Paul E. McKenney 2013-05-09 9:04 ` Frederic Weisbecker 2013-05-09 9:20 ` Ingo Molnar 2013-05-09 21:02 ` Frederic Weisbecker 2013-05-09 21:10 ` Dave Jones 2013-05-14 1:21 ` Frederic Weisbecker 2013-06-06 15:31 ` Dave Jones 2013-06-06 15:43 ` Frederic Weisbecker 2013-06-06 23:10 ` Dave Jones 2013-06-12 15:34 ` Dave Jones 2013-06-20 19:10 ` Dave Jones 2013-06-23 10:58 ` Ingo Molnar 2013-07-01 12:10 ` Frederic Weisbecker
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox