linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [sched/balance] 7511dd0a7: +2.1e+05% context switches
@ 2014-02-19 13:00 Fengguang Wu
  2014-02-20  7:21 ` Alex Shi
  0 siblings, 1 reply; 4+ messages in thread
From: Fengguang Wu @ 2014-02-19 13:00 UTC (permalink / raw)
  To: Alex Shi; +Cc: LKML

[-- Attachment #1: Type: text/plain, Size: 11429 bytes --]

Alex,

We noticed the below changes for commit

commit 7511dd0a73aaf2ca4bcd829f991dcd2c839c5417
Author:     Alex Shi <alex.shi@linaro.org>
AuthorDate: Fri Jan 24 15:17:02 2014 +0800
Commit:     Alex Shi <alex.shi@linaro.org>
CommitDate: Fri Feb 14 15:09:01 2014 +0800

    sched: only do load balance on tick_do_timer_cpu
    
    that is a central balance.
    
    Signed-off-by: Alex Shi <alex.shi@linaro.org>

bc575710efe937e  7511dd0a73aaf2ca4bcd829f9
---------------  -------------------------
     83892 ~ 5%     -96.1%       3245 ~ 7%  lkp-snb01/micro/will-it-scale/pthread_mutex2
     86509 ~ 5%     -96.2%       3271 ~ 6%  lkp-snb01/micro/will-it-scale/sched_yield
    170401 ~ 5%     -96.2%       6517 ~ 7%  TOTAL softirqs.SCHED

bc575710efe937e  7511dd0a73aaf2ca4bcd829f9
---------------  -------------------------
     60321 ~ 0%     -78.5%      12962 ~ 3%  lkp-snb01/micro/will-it-scale/pthread_mutex2
     60358 ~ 0%     -80.5%      11797 ~ 4%  lkp-snb01/micro/will-it-scale/sched_yield
    120679 ~ 0%     -79.5%      24759 ~ 4%  TOTAL interrupts.RES

bc575710efe937e  7511dd0a73aaf2ca4bcd829f9
---------------  -------------------------
    105019 ~ 0%     -41.8%      61097 ~ 3%  lkp-snb01/micro/will-it-scale/pthread_mutex2
    103938 ~ 1%     -35.4%      67183 ~ 1%  lkp-snb01/micro/will-it-scale/sched_yield
    208958 ~ 0%     -38.6%     128281 ~ 2%  TOTAL cpuidle.C7-SNB.usage

bc575710efe937e  7511dd0a73aaf2ca4bcd829f9
---------------  -------------------------
      2492 ~23%     +39.8%       3483 ~14%  lkp-snb01/micro/will-it-scale/sched_yield
      2492 ~23%     +39.8%       3483 ~14%  TOTAL cpuidle.C1E-SNB.usage

bc575710efe937e  7511dd0a73aaf2ca4bcd829f9
---------------  -------------------------
       543 ~10%     +30.8%        710 ~11%  lkp-snb01/micro/will-it-scale/sched_yield
       543 ~10%     +30.8%        710 ~11%  TOTAL cpuidle.C3-SNB.usage

bc575710efe937e  7511dd0a73aaf2ca4bcd829f9
---------------  -------------------------
     20.65 ~ 0%     +24.1%      25.63 ~ 2%  lkp-snb01/micro/will-it-scale/sched_yield
     20.65 ~ 0%     +24.1%      25.63 ~ 2%  TOTAL turbostat.%c1

bc575710efe937e  7511dd0a73aaf2ca4bcd829f9
---------------  -------------------------
      1268 ~13%     +17.0%       1483 ~ 5%  lkp-snb01/micro/will-it-scale/sched_yield
      1268 ~13%     +17.0%       1483 ~ 5%  TOTAL numa-vmstat.node0.nr_alloc_batch

bc575710efe937e  7511dd0a73aaf2ca4bcd829f9
---------------  -------------------------
      7255 ~ 2%     +14.6%       8317 ~ 2%  lkp-snb01/micro/will-it-scale/pthread_mutex2
      7255 ~ 2%     +14.6%       8317 ~ 2%  TOTAL slabinfo.kmalloc-192.num_objs

bc575710efe937e  7511dd0a73aaf2ca4bcd829f9
---------------  -------------------------
      7255 ~ 2%     +14.6%       8317 ~ 2%  lkp-snb01/micro/will-it-scale/pthread_mutex2
      7255 ~ 2%     +14.6%       8317 ~ 2%  TOTAL slabinfo.kmalloc-192.active_objs

bc575710efe937e  7511dd0a73aaf2ca4bcd829f9
---------------  -------------------------
    683581 ~ 0%      -9.3%     620114 ~ 1%  lkp-snb01/micro/will-it-scale/sched_yield
    683581 ~ 0%      -9.3%     620114 ~ 1%  TOTAL softirqs.TIMER

bc575710efe937e  7511dd0a73aaf2ca4bcd829f9
---------------  -------------------------
    204033 ~ 1%      -8.6%     186557 ~ 2%  lkp-snb01/micro/will-it-scale/sched_yield
    204033 ~ 1%      -8.6%     186557 ~ 2%  TOTAL softirqs.RCU

bc575710efe937e  7511dd0a73aaf2ca4bcd829f9
---------------  -------------------------
 4.902e+09 ~ 0%      +9.8%  5.385e+09 ~ 1%  lkp-snb01/micro/will-it-scale/sched_yield
 4.902e+09 ~ 0%      +9.8%  5.385e+09 ~ 1%  TOTAL cpuidle.C7-SNB.time

bc575710efe937e  7511dd0a73aaf2ca4bcd829f9
---------------  -------------------------
      8204 ~ 2%     +11.3%       9134 ~ 4%  lkp-snb01/micro/will-it-scale/sched_yield
      8204 ~ 2%     +11.3%       9134 ~ 4%  TOTAL slabinfo.kmalloc-256.num_objs

bc575710efe937e  7511dd0a73aaf2ca4bcd829f9
---------------  -------------------------
      2405 ~25%  +2.1e+05%    5058323 ~13%  lkp-snb01/micro/will-it-scale/sched_yield
      2405 ~25%  +2.1e+05%    5058323 ~13%  TOTAL vmstat.system.cs

bc575710efe937e  7511dd0a73aaf2ca4bcd829f9
---------------  -------------------------
      2029 ~ 0%    +222.9%       6551 ~17%  lkp-snb01/micro/will-it-scale/pthread_mutex2
    143678 ~42%  +4.8e+05%  6.927e+08 ~ 0%  lkp-snb01/micro/will-it-scale/sched_yield
    145708 ~42%  +4.8e+05%  6.927e+08 ~ 0%  TOTAL time.involuntary_context_switches

bc575710efe937e  7511dd0a73aaf2ca4bcd829f9
---------------  -------------------------
      1267 ~ 0%     -22.3%        985 ~ 1%  lkp-snb01/micro/will-it-scale/sched_yield
      1267 ~ 0%     -22.3%        985 ~ 1%  TOTAL time.system_time

bc575710efe937e  7511dd0a73aaf2ca4bcd829f9
---------------  -------------------------
      1518 ~ 0%      -4.9%       1444 ~ 1%  lkp-snb01/micro/will-it-scale/pthread_mutex2
       250 ~ 4%     +26.3%        316 ~ 5%  lkp-snb01/micro/will-it-scale/sched_yield
      1769 ~ 0%      -0.5%       1761 ~ 2%  TOTAL time.user_time

bc575710efe937e  7511dd0a73aaf2ca4bcd829f9
---------------  -------------------------
       489 ~ 0%      -4.9%        464 ~ 1%  lkp-snb01/micro/will-it-scale/pthread_mutex2
       489 ~ 0%     -14.3%        419 ~ 0%  lkp-snb01/micro/will-it-scale/sched_yield
       978 ~ 0%      -9.6%        883 ~ 0%  TOTAL time.percent_of_cpu_this_job_got

bc575710efe937e  7511dd0a73aaf2ca4bcd829f9
---------------  -------------------------
     50.58 ~ 0%      -2.0%      49.56 ~ 0%  lkp-snb01/micro/will-it-scale/pthread_mutex2
     50.62 ~ 0%      -9.9%      45.62 ~ 1%  lkp-snb01/micro/will-it-scale/sched_yield
    101.20 ~ 0%      -5.9%      95.19 ~ 0%  TOTAL turbostat.%c0

bc575710efe937e  7511dd0a73aaf2ca4bcd829f9
---------------  -------------------------
      4386 ~ 0%      -5.2%       4158 ~ 0%  lkp-snb01/micro/will-it-scale/pthread_mutex2
      4386 ~ 0%      -5.2%       4158 ~ 0%  TOTAL vmstat.system.in

bc575710efe937e  7511dd0a73aaf2ca4bcd829f9
---------------  -------------------------
       148 ~ 0%      -1.9%        145 ~ 0%  lkp-snb01/micro/will-it-scale/sched_yield
       148 ~ 0%      -1.9%        145 ~ 0%  TOTAL turbostat.Pkg_W

bc575710efe937e  7511dd0a73aaf2ca4bcd829f9
---------------  -------------------------
       109 ~ 0%      -0.1%        109 ~ 0%  lkp-snb01/micro/will-it-scale/pthread_mutex2
       121 ~ 0%      -2.3%        119 ~ 0%  lkp-snb01/micro/will-it-scale/sched_yield
       231 ~ 0%      -1.3%        228 ~ 0%  TOTAL turbostat.Cor_W

                         time.percent_of_cpu_this_job_got

   490 *+-----*---*---*--*---*--*---*---*--*---*---*--*---*--*---*---*--*---*
       | ..  +                                                              |
   480 ++   +                                                               |
   470 ++  *                                                                |
       |                                                                    |
   460 ++                                                                   |
       |                                                                    |
   450 ++                                                                   |
       |                                                                    |
   440 ++                                                                   |
   430 ++                                                                   |
       |                                O                                   |
   420 O+  O  O   O   O  O   O  O   O      O   O   O  O   O  O   O   O  O   O
       |                                                                    |
   410 ++-------------------------------------------------------------------+


                          time.involuntary_context_switches

   8e+08 ++-----------------------------------------------------------------+
         |                                                                  |
   7e+08 O+  O  O   O  O   O  O   O  O      O   O  O   O  O   O  O   O  O   O
   6e+08 ++                              O                                  |
         |                                                                  |
   5e+08 ++                                                                 |
         |                                                                  |
   4e+08 ++                                                                 |
         |                                                                  |
   3e+08 ++                                                                 |
   2e+08 ++                                                                 |
         |                                                                  |
   1e+08 ++                                                                 |
         |                                                                  |
       0 *+--*--*---*--*---*--*---*--*---*--*---*--*---*--*---*--*---*--*---*


                                   turbostat.%c0

   51 *+--------------------------------------------------------------------+
      |  .  .*...*...*..*...*...*..*...*...*...*..*...*...*..*...*...*..*...*
   50 ++  *.                                                                |
      |                                                                     |
   49 ++                                                                    |
      |                                                                     |
   48 ++                                                                    |
      |                                                                     |
   47 ++                                                                    |
      |                                                                     O
   46 ++     O                                    O       O          O      |
      O   O      O   O  O   O   O  O   O   O   O      O      O          O   |
   45 ++                                                                    |
      |                                                          O          |
   44 ++--------------------------------------------------------------------+


                                  vmstat.system.cs

   6e+06 ++------------------------------------------------------O----------+
         |                    O                               O         O   |
   5e+06 O+  O      O  O   O      O  O   O  O   O      O                    |
         |      O                                  O      O                 |
         |                                                           O      O
   4e+06 ++                                                                 |
         |                                                                  |
   3e+06 ++                                                                 |
         |                                                                  |
   2e+06 ++                                                                 |
         |                                                                  |
         |                                                                  |
   1e+06 ++                                                                 |
         |                                                                  |
       0 *+--*--*---*--*---*--*---*--*---*--*---*--*---*--*---*--*---*--*---*

Thanks,
Fengguang

[-- Attachment #2: 0001-sched-only-do-load-balance-on-tick_do_timer_cpu.patch --]
[-- Type: text/x-diff, Size: 1136 bytes --]

>From 7511dd0a73aaf2ca4bcd829f991dcd2c839c5417 Mon Sep 17 00:00:00 2001
From: Alex Shi <alex.shi@linaro.org>
Date: Fri, 24 Jan 2014 15:17:02 +0800
Subject: [PATCH] sched: only do load balance on tick_do_timer_cpu

that is a central balance.

Signed-off-by: Alex Shi <alex.shi@linaro.org>
---
 kernel/sched/core.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 7368964..1be85a7 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -2425,6 +2425,7 @@ unsigned long long task_sched_runtime(struct task_struct *p)
 	return ns;
 }
 
+extern int tick_do_timer_cpu __read_mostly;
 /*
  * This function gets called by the timer code, with HZ frequency.
  * We call it with interrupts disabled.
@@ -2446,8 +2447,11 @@ void scheduler_tick(void)
 	perf_event_task_tick();
 
 #ifdef CONFIG_SMP
-	rq->idle_balance = idle_cpu(cpu);
-	trigger_load_balance(rq);
+	/* only do load balance on tick_do_timer_cpu */
+	if (cpu == tick_do_timer_cpu) {
+		rq->idle_balance = idle_cpu(cpu);
+		trigger_load_balance(rq);
+	}
 #endif
 	rq_last_tick_reset(rq);
 }
-- 
1.8.5.2


[-- Attachment #3: 2014-02-16_10:06:49.png --]
[-- Type: image/png, Size: 26554 bytes --]

[-- Attachment #4: current --]
[-- Type: text/plain, Size: 25582 bytes --]

2014-02-16 04:18:41  /lkp/lkp/src/sbin/perf-bisect /lkp/.bisect-descs/bisect-desc.vmstat.system.cs.2014-02-16.03:54:18
---
stat: vmstat.system.cs
interval: ! '[ 5.1883e+06 - 5.1883e+06 ] -- [ 2474       - 4961       ]'
a:
- 5188285
b:
- 2474
- 2919
- 3479
- 3594
- 3669
- 3802
- 4124
- 4194
- 4961
ttl: 2014-02-15 17:33:53.961378385 +08:00
ratio: 5
delta: 5184616
mean_a: 5188285
mean_b: 3669
x: 0
y: 5183324
z: 2487
min_a: 5188285
max_a: 5188285
min_b: 2474
max_b: 4961
max: 5188285
nr_run: 1469
good_commit: b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed
tbox_group: lkp-snb01
run_time: 311
result_root: /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/5abff587f6ba8de0adf41a19c4add6a3a4b37b64/0
_result_root: /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/5abff587f6ba8de0adf41a19c4add6a3a4b37b64
head_commit: 5abff587f6ba8de0adf41a19c4add6a3a4b37b64
base_commit: b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed
commit: 5abff587f6ba8de0adf41a19c4add6a3a4b37b64
branch: linux-devel/devel-hourly-2014021515
kconfig: x86_64-lkp
testbox: lkp-snb01
score_perf_change: 0
change_percent: 0.999292829904294
variation: 0.00047957783266686733
avg_run_time: 312
score_change: 99.96463524188411
score_var: 9.985619563763917
score_time: 90
score_metric: 4
prio: 203.9503
2014-02-16 04:18:42 [perf-bisect 98512] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed -q reconfirm -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
Enqueue job 3 times: /lkp/jobs/reconfirm/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed-2.yaml
2014-02-16 04:24:23 [perf-bisect 98512] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:5abff587f6ba8de0adf41a19c4add6a3a4b37b64 -q reconfirm -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
Enqueue job 3 times: /lkp/jobs/reconfirm/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-5abff587f6ba8de0adf41a19c4add6a3a4b37b64-2.yaml
2014-02-16 04:44:24 [perf-bisect 98512] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed -q reconfirm -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
Enqueue job 5 times: /lkp/jobs/reconfirm/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed-4.yaml
2014-02-16 04:56:44 [perf-bisect 98512] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:5abff587f6ba8de0adf41a19c4add6a3a4b37b64 -q reconfirm -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
Enqueue job 5 times: /lkp/jobs/reconfirm/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-5abff587f6ba8de0adf41a19c4add6a3a4b37b64-4.yaml
git checkout b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed
2014-02-16 05:15:06 [perf-bisect-test 73445] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed -q bisect -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
Enqueue job 2 times: /lkp/jobs/bisect/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed-1.yaml
2014-02-16 05:22:06 [perf-bisect-test 73445] /lkp/lkp/src/sbin/stats-good-or-bad vmstat.system.cs /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/5abff587f6ba8de0adf41a19c4add6a3a4b37b64/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed/matrix.json
bisect GOOD: [2734, 3802, 7773]  [2734, 3802, 7773] 1098637.2831603948 [4535025, 5179772, 5188285]

bisect: good commit b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed
git bisect start 5abff587f6ba8de0adf41a19c4add6a3a4b37b64 b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed --
/c/kernel-tests/linear-bisect: ["-b", "5abff587f6ba8de0adf41a19c4add6a3a4b37b64", "-g", "b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed", "/lkp/lkp/src/sbin/perf-bisect-test", "/cc/perf-bisect-obj/lkp/x86_64-lkp"]
Bisecting: 1944 revisions left to test after this (roughly 11 steps)
[7aaf3caaab7f5ddc8da4359a93a8a6be86946169] Merge 'at91/at91-3.15-cleanup' into devel-hourly-2014021515
running /lkp/lkp/src/sbin/perf-bisect-test /cc/perf-bisect-obj/lkp/x86_64-lkp
2014-02-16 05:22:07 [perf-bisect-test 10038] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:7aaf3caaab7f5ddc8da4359a93a8a6be86946169 -q bisect -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
Enqueue job 2 times: /lkp/jobs/bisect/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-7aaf3caaab7f5ddc8da4359a93a8a6be86946169-1.yaml
2014-02-16 05:40:58 [perf-bisect-test 10038] /lkp/lkp/src/sbin/stats-good-or-bad vmstat.system.cs /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/5abff587f6ba8de0adf41a19c4add6a3a4b37b64/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/7aaf3caaab7f5ddc8da4359a93a8a6be86946169/matrix.json
bisect BAD: [5152320, 5470320, 5470320]  [2734, 3802, 7773] 1098637.2831603948 [4535025, 5179772, 5188285]

Bisecting: 1076 revisions left to test after this (roughly 11 steps)
[f6afc6d8b9d433388365d1567babe4b1738400bb] Merge 'pci/pci/host-mvebu' into devel-hourly-2014021515
running /lkp/lkp/src/sbin/perf-bisect-test /cc/perf-bisect-obj/lkp/x86_64-lkp
2014-02-16 05:40:58 [perf-bisect-test 40970] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:f6afc6d8b9d433388365d1567babe4b1738400bb -q bisect -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
/lkp/lkp/src/sbin/kbuild-queue -c x86_64-lkp -b linux-devel/devel-hourly-2014021515 -k f6afc6d8b9d433388365d1567babe4b1738400bb -w
Queued build task to /kernel-tests/build-queue/x86_64-lkp-f6afc6d8b9d433388365d1567babe4b1738400bb
Check for kernel in /kernel/x86_64-lkp/f6afc6d8b9d433388365d1567babe4b1738400bb
waiting for completion of /kernel-tests/build-queue/x86_64-lkp-f6afc6d8b9d433388365d1567babe4b1738400bb
waiting for completion of /kernel-tests/build-queue/.x86_64-lkp-f6afc6d8b9d433388365d1567babe4b1738400bb
kernel: /kernel/x86_64-lkp/f6afc6d8b9d433388365d1567babe4b1738400bb/vmlinuz-3.14.0-rc2-wl-ath-00835-gf6afc6d
Enqueue job 2 times: /lkp/jobs/bisect/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-f6afc6d8b9d433388365d1567babe4b1738400bb-1.yaml
2014-02-16 06:00:19 [perf-bisect-test 40970] /lkp/lkp/src/sbin/stats-good-or-bad vmstat.system.cs /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/5abff587f6ba8de0adf41a19c4add6a3a4b37b64/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/f6afc6d8b9d433388365d1567babe4b1738400bb/matrix.json
bisect GOOD: [2868, 3166, 3166]  [2734, 3802, 7773] 1098637.2831603948 [4535025, 5179772, 5188285]

Bisecting: 241 revisions left to test after this (roughly 8 steps)
[d67497e37737967e5edb30ff331e7479e19a13f3] Merge 'char-misc/char-misc-linus' into devel-hourly-2014021515
running /lkp/lkp/src/sbin/perf-bisect-test /cc/perf-bisect-obj/lkp/x86_64-lkp
2014-02-16 06:00:19 [perf-bisect-test 797] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:d67497e37737967e5edb30ff331e7479e19a13f3 -q bisect -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
/lkp/lkp/src/sbin/kbuild-queue -c x86_64-lkp -b linux-devel/devel-hourly-2014021515 -k d67497e37737967e5edb30ff331e7479e19a13f3 -w
Queued build task to /kernel-tests/build-queue/x86_64-lkp-d67497e37737967e5edb30ff331e7479e19a13f3
Check for kernel in /kernel/x86_64-lkp/d67497e37737967e5edb30ff331e7479e19a13f3
waiting for completion of /kernel-tests/build-queue/x86_64-lkp-d67497e37737967e5edb30ff331e7479e19a13f3
waiting for completion of /kernel-tests/build-queue/.x86_64-lkp-d67497e37737967e5edb30ff331e7479e19a13f3
kernel: /kernel/x86_64-lkp/d67497e37737967e5edb30ff331e7479e19a13f3/vmlinuz-3.14.0-rc2-wl-ath-00912-gd67497e
Enqueue job 2 times: /lkp/jobs/bisect/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-d67497e37737967e5edb30ff331e7479e19a13f3-1.yaml
2014-02-16 06:18:30 [perf-bisect-test 797] /lkp/lkp/src/sbin/stats-good-or-bad vmstat.system.cs /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/5abff587f6ba8de0adf41a19c4add6a3a4b37b64/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/d67497e37737967e5edb30ff331e7479e19a13f3/matrix.json
bisect BAD: [5209415, 5211906, 5211906]  [2734, 3802, 7773] 1098637.2831603948 [4535025, 5179772, 5188285]

Bisecting: 77 revisions left to test after this (roughly 7 steps)
[933530ed7e473b3c0efa0a0b39c09c65c04beb37] Merge 'arm-perf/for-next/perf' into devel-hourly-2014021515
running /lkp/lkp/src/sbin/perf-bisect-test /cc/perf-bisect-obj/lkp/x86_64-lkp
2014-02-16 06:18:31 [perf-bisect-test 41141] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:933530ed7e473b3c0efa0a0b39c09c65c04beb37 -q bisect -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
/lkp/lkp/src/sbin/kbuild-queue -c x86_64-lkp -b linux-devel/devel-hourly-2014021515 -k 933530ed7e473b3c0efa0a0b39c09c65c04beb37 -w
Queued build task to /kernel-tests/build-queue/x86_64-lkp-933530ed7e473b3c0efa0a0b39c09c65c04beb37
Check for kernel in /kernel/x86_64-lkp/933530ed7e473b3c0efa0a0b39c09c65c04beb37
waiting for completion of /kernel-tests/build-queue/x86_64-lkp-933530ed7e473b3c0efa0a0b39c09c65c04beb37
waiting for completion of /kernel-tests/build-queue/.x86_64-lkp-933530ed7e473b3c0efa0a0b39c09c65c04beb37
kernel: /kernel/x86_64-lkp/933530ed7e473b3c0efa0a0b39c09c65c04beb37/vmlinuz-3.14.0-rc2-wl-ath-00868-g933530e
Enqueue job 2 times: /lkp/jobs/bisect/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-933530ed7e473b3c0efa0a0b39c09c65c04beb37-1.yaml
2014-02-16 06:36:23 [perf-bisect-test 41141] /lkp/lkp/src/sbin/stats-good-or-bad vmstat.system.cs /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/5abff587f6ba8de0adf41a19c4add6a3a4b37b64/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/933530ed7e473b3c0efa0a0b39c09c65c04beb37/matrix.json
bisect GOOD: [2499, 2714, 2714]  [2734, 3802, 7773] 1098637.2831603948 [4535025, 5179772, 5188285]

Bisecting: 44 revisions left to test after this (roughly 6 steps)
[9fd1bb34f21472d352fbf49bbdf1f07a320393ca] Merge 'kvm/master' into devel-hourly-2014021515
running /lkp/lkp/src/sbin/perf-bisect-test /cc/perf-bisect-obj/lkp/x86_64-lkp
2014-02-16 06:36:23 [perf-bisect-test 69581] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:9fd1bb34f21472d352fbf49bbdf1f07a320393ca -q bisect -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
/lkp/lkp/src/sbin/kbuild-queue -c x86_64-lkp -b linux-devel/devel-hourly-2014021515 -k 9fd1bb34f21472d352fbf49bbdf1f07a320393ca -w
Queued build task to /kernel-tests/build-queue/x86_64-lkp-9fd1bb34f21472d352fbf49bbdf1f07a320393ca
Check for kernel in /kernel/x86_64-lkp/9fd1bb34f21472d352fbf49bbdf1f07a320393ca
waiting for completion of /kernel-tests/build-queue/x86_64-lkp-9fd1bb34f21472d352fbf49bbdf1f07a320393ca
waiting for completion of /kernel-tests/build-queue/.x86_64-lkp-9fd1bb34f21472d352fbf49bbdf1f07a320393ca
kernel: /kernel/x86_64-lkp/9fd1bb34f21472d352fbf49bbdf1f07a320393ca/vmlinuz-3.14.0-rc2-wl-ath-00876-g9fd1bb3
Enqueue job 2 times: /lkp/jobs/bisect/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-9fd1bb34f21472d352fbf49bbdf1f07a320393ca-1.yaml
2014-02-16 06:55:04 [perf-bisect-test 69581] /lkp/lkp/src/sbin/stats-good-or-bad vmstat.system.cs /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/5abff587f6ba8de0adf41a19c4add6a3a4b37b64/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/9fd1bb34f21472d352fbf49bbdf1f07a320393ca/matrix.json
bisect GOOD: [3523, 7441, 7441]  [2734, 3802, 7773] 1098637.2831603948 [4535025, 5179772, 5188285]

Bisecting: 36 revisions left to test after this (roughly 6 steps)
[1a9a97cd15230efd44ff9957326dd245f89bdbd6] Merge 'alexshi/topdown' into devel-hourly-2014021515
running /lkp/lkp/src/sbin/perf-bisect-test /cc/perf-bisect-obj/lkp/x86_64-lkp
2014-02-16 06:55:05 [perf-bisect-test 101819] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:1a9a97cd15230efd44ff9957326dd245f89bdbd6 -q bisect -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
/lkp/lkp/src/sbin/kbuild-queue -c x86_64-lkp -b linux-devel/devel-hourly-2014021515 -k 1a9a97cd15230efd44ff9957326dd245f89bdbd6 -w
Queued build task to /kernel-tests/build-queue/x86_64-lkp-1a9a97cd15230efd44ff9957326dd245f89bdbd6
Check for kernel in /kernel/x86_64-lkp/1a9a97cd15230efd44ff9957326dd245f89bdbd6
waiting for completion of /kernel-tests/build-queue/x86_64-lkp-1a9a97cd15230efd44ff9957326dd245f89bdbd6
waiting for completion of /kernel-tests/build-queue/.x86_64-lkp-1a9a97cd15230efd44ff9957326dd245f89bdbd6
kernel: /kernel/x86_64-lkp/1a9a97cd15230efd44ff9957326dd245f89bdbd6/vmlinuz-3.14.0-rc2-wl-ath-00895-g1a9a97c
Enqueue job 2 times: /lkp/jobs/bisect/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-1a9a97cd15230efd44ff9957326dd245f89bdbd6-1.yaml
2014-02-16 07:21:36 [perf-bisect-test 101819] /lkp/lkp/src/sbin/stats-good-or-bad vmstat.system.cs /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/5abff587f6ba8de0adf41a19c4add6a3a4b37b64/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/1a9a97cd15230efd44ff9957326dd245f89bdbd6/matrix.json
bisect BAD: [5211562, 5215236, 5215236]  [2734, 3802, 7773] 1098637.2831603948 [4535025, 5179772, 5188285]

linear-bisect: bad branch may be alexshi/topdown
[v3.14-rc2] Linux 3.14-rc2
running /lkp/lkp/src/sbin/perf-bisect-test /cc/perf-bisect-obj/lkp/x86_64-lkp
2014-02-16 07:21:39 [perf-bisect-test 43744] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed -q bisect -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
Enqueue job 2 times: /lkp/jobs/bisect/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed-1.yaml
2014-02-16 07:25:30 [perf-bisect-test 43744] /lkp/lkp/src/sbin/stats-good-or-bad vmstat.system.cs /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/5abff587f6ba8de0adf41a19c4add6a3a4b37b64/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed/matrix.json
bisect GOOD: [2734, 3802, 7773]  [2734, 3802, 7773] 1098637.2831603948 [4535025, 5179772, 5188285]

[d2f8e017fdd913fc150160508d2e1161cab5fef6] sched: remove un-used functions
running /lkp/lkp/src/sbin/perf-bisect-test /cc/perf-bisect-obj/lkp/x86_64-lkp
2014-02-16 07:25:30 [perf-bisect-test 115104] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:d2f8e017fdd913fc150160508d2e1161cab5fef6 -q bisect -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
Enqueue job 2 times: /lkp/jobs/bisect/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-d2f8e017fdd913fc150160508d2e1161cab5fef6-1.yaml
2014-02-16 07:45:41 [perf-bisect-test 115104] /lkp/lkp/src/sbin/stats-good-or-bad vmstat.system.cs /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/5abff587f6ba8de0adf41a19c4add6a3a4b37b64/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/d2f8e017fdd913fc150160508d2e1161cab5fef6/matrix.json
bisect BAD: [4736446, 5001400, 5001400]  [2734, 3802, 7773] 1098637.2831603948 [4535025, 5179772, 5188285]

linear-bisect: recursive descent into merged branch
/c/kernel-tests/linear-bisect: ["-g", "v3.14-rc2", "-b", "d2f8e017fdd913fc150160508d2e1161cab5fef6", "/lkp/lkp/src/sbin/perf-bisect-test", "/cc/perf-bisect-obj/lkp/x86_64-lkp"]
Bisecting: 18 revisions left to test after this (roughly 5 steps)
[f6ab232915356bd6649052c1672c10424feab995] sched: remove rq->cpu_load and rq->nr_load_updates
running /lkp/lkp/src/sbin/perf-bisect-test /cc/perf-bisect-obj/lkp/x86_64-lkp
2014-02-16 07:45:42 [perf-bisect-test 74444] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:f6ab232915356bd6649052c1672c10424feab995 -q bisect -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
Enqueue job 2 times: /lkp/jobs/bisect/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-f6ab232915356bd6649052c1672c10424feab995-1.yaml
2014-02-16 08:04:02 [perf-bisect-test 74444] /lkp/lkp/src/sbin/stats-good-or-bad vmstat.system.cs /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/5abff587f6ba8de0adf41a19c4add6a3a4b37b64/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/f6ab232915356bd6649052c1672c10424feab995/matrix.json
bisect GOOD: [2329, 4551, 4551]  [2734, 3802, 7773] 1098637.2831603948 [4535025, 5179772, 5188285]

Bisecting: 9 revisions left to test after this (roughly 4 steps)
[7511dd0a73aaf2ca4bcd829f991dcd2c839c5417] sched: only do load balance on tick_do_timer_cpu
running /lkp/lkp/src/sbin/perf-bisect-test /cc/perf-bisect-obj/lkp/x86_64-lkp
2014-02-16 08:04:03 [perf-bisect-test 53445] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:7511dd0a73aaf2ca4bcd829f991dcd2c839c5417 -q bisect -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
Enqueue job 2 times: /lkp/jobs/bisect/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-7511dd0a73aaf2ca4bcd829f991dcd2c839c5417-1.yaml
2014-02-16 08:21:33 [perf-bisect-test 53445] /lkp/lkp/src/sbin/stats-good-or-bad vmstat.system.cs /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/5abff587f6ba8de0adf41a19c4add6a3a4b37b64/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/7511dd0a73aaf2ca4bcd829f991dcd2c839c5417/matrix.json
bisect BAD: [5373928, 5996496, 5996496]  [2734, 3802, 7773] 1098637.2831603948 [4535025, 5179772, 5188285]

Bisecting: 4 revisions left to test after this (roughly 3 steps)
[6de9950778921604a8bd0fbab09911df5d98bbeb] sched: add statistic for newidle load balance cost
running /lkp/lkp/src/sbin/perf-bisect-test /cc/perf-bisect-obj/lkp/x86_64-lkp
2014-02-16 08:21:34 [perf-bisect-test 112510] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:6de9950778921604a8bd0fbab09911df5d98bbeb -q bisect -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
/lkp/lkp/src/sbin/kbuild-queue -c x86_64-lkp -b linux-devel/devel-hourly-2014021515 -k 6de9950778921604a8bd0fbab09911df5d98bbeb -w
Queued build task to /kernel-tests/build-queue/x86_64-lkp-6de9950778921604a8bd0fbab09911df5d98bbeb
Check for kernel in /kernel/x86_64-lkp/6de9950778921604a8bd0fbab09911df5d98bbeb
waiting for completion of /kernel-tests/build-queue/x86_64-lkp-6de9950778921604a8bd0fbab09911df5d98bbeb
waiting for completion of /kernel-tests/build-queue/.x86_64-lkp-6de9950778921604a8bd0fbab09911df5d98bbeb
kernel: /kernel/x86_64-lkp/6de9950778921604a8bd0fbab09911df5d98bbeb/vmlinuz-3.14.0-rc2-00011-g6de9950
Enqueue job 2 times: /lkp/jobs/bisect/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-6de9950778921604a8bd0fbab09911df5d98bbeb-1.yaml
2014-02-16 08:46:04 [perf-bisect-test 112510] /lkp/lkp/src/sbin/stats-good-or-bad vmstat.system.cs /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/5abff587f6ba8de0adf41a19c4add6a3a4b37b64/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/6de9950778921604a8bd0fbab09911df5d98bbeb/matrix.json
bisect GOOD: [2622, 2906, 2906]  [2734, 3802, 7773] 1098637.2831603948 [4535025, 5179772, 5188285]

Bisecting: 2 revisions left to test after this (roughly 2 steps)
[bc575710efe937e26d2bd541d92362b240ef0559] sched: clean up task_hot function
running /lkp/lkp/src/sbin/perf-bisect-test /cc/perf-bisect-obj/lkp/x86_64-lkp
2014-02-16 08:46:05 [perf-bisect-test 5884] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:bc575710efe937e26d2bd541d92362b240ef0559 -q bisect -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
/lkp/lkp/src/sbin/kbuild-queue -c x86_64-lkp -b linux-devel/devel-hourly-2014021515 -k bc575710efe937e26d2bd541d92362b240ef0559 -w
Queued build task to /kernel-tests/build-queue/x86_64-lkp-bc575710efe937e26d2bd541d92362b240ef0559
Check for kernel in /kernel/x86_64-lkp/bc575710efe937e26d2bd541d92362b240ef0559
waiting for completion of /kernel-tests/build-queue/x86_64-lkp-bc575710efe937e26d2bd541d92362b240ef0559
waiting for completion of /kernel-tests/build-queue/.x86_64-lkp-bc575710efe937e26d2bd541d92362b240ef0559
kernel: /kernel/x86_64-lkp/bc575710efe937e26d2bd541d92362b240ef0559/vmlinuz-3.14.0-rc2-00012-gbc57571
Enqueue job 2 times: /lkp/jobs/bisect/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-bc575710efe937e26d2bd541d92362b240ef0559-1.yaml
2014-02-16 09:17:05 [perf-bisect-test 5884] /lkp/lkp/src/sbin/stats-good-or-bad vmstat.system.cs /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/5abff587f6ba8de0adf41a19c4add6a3a4b37b64/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/b28a960c42fcd9cfc987441fa6d1c1a471f0f9ed/matrix.json /lkp/result/lkp-snb01/micro/will-it-scale/sched_yield/x86_64-lkp/bc575710efe937e26d2bd541d92362b240ef0559/matrix.json
bisect GOOD: [1794, 2328, 2328]  [2734, 3802, 7773] 1098637.2831603948 [4535025, 5179772, 5188285]

7511dd0a73aaf2ca4bcd829f991dcd2c839c5417 is the first bad commit
commit 7511dd0a73aaf2ca4bcd829f991dcd2c839c5417
Author:     Alex Shi <alex.shi@linaro.org>
AuthorDate: Fri Jan 24 15:17:02 2014 +0800
Commit:     Alex Shi <alex.shi@linaro.org>
CommitDate: Fri Feb 14 15:09:01 2014 +0800

    sched: only do load balance on tick_do_timer_cpu
    
    that is a central balance.
    
    Signed-off-by: Alex Shi <alex.shi@linaro.org>

 kernel/sched/core.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
exit
/c/kernel-tests/linear-bisect:274:in `exit'
/c/kernel-tests/linear-bisect:274:in `linear_bisect'
/c/kernel-tests/linear-bisect:294:in `<main>'
exit
/c/kernel-tests/linear-bisect:181:in `exit'
/c/kernel-tests/linear-bisect:181:in `recursive_bisect_into'
/c/kernel-tests/linear-bisect:210:in `perf_test_merged_branch'
/c/kernel-tests/linear-bisect:286:in `block in linear_bisect'
/c/kernel-tests/linear-bisect:286:in `each'
/c/kernel-tests/linear-bisect:286:in `linear_bisect'
/c/kernel-tests/linear-bisect:294:in `<main>'
2014-02-16 09:17:06 [perf-bisect 98512] validating first bad commit 7511dd0a73aaf2ca4bcd829f991dcd2c839c5417 ...
2014-02-16 09:17:06 [perf-bisect 98512] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:7511dd0a73aaf2ca4bcd829f991dcd2c839c5417 -q validate -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
Enqueue job 5 times: /lkp/jobs/validate/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-7511dd0a73aaf2ca4bcd829f991dcd2c839c5417-4.yaml
bisect BAD: [4110714, 5373928, 5996496]  [2734, 3802, 7773] 1098637.2831603948 [4535025, 5179772, 5188285]

2014-02-16 09:41:56 [perf-bisect 98512] validating parent commit bc575710efe937e26d2bd541d92362b240ef0559 ...
2014-02-16 09:41:56 [perf-bisect 98512] /lkp/lkp/src/sbin/queue -k linux-devel/devel-hourly-2014021515:bc575710efe937e26d2bd541d92362b240ef0559 -q validate -w 36000 -R0 /lkp/lkp/linux5/.git/job.yaml
run: /lkp/lkp/src/filters/iostat
run: /lkp/lkp/src/filters/kconfig
Enqueue job 5 times: /lkp/jobs/validate/lkp-snb01/will-it-scale-sched_yield-x86_64-lkp-bc575710efe937e26d2bd541d92362b240ef0559-4.yaml
bisect GOOD: [1794, 2316, 3578]  [2734, 3802, 7773] 1098637.2831603948 [4535025, 5179772, 5188285]

2014-02-16 10:06:47 [perf-bisect 98512] first bad commit validated successfully
==================== PARENT FIRST_BAD KCONFIGS ====================
stats changes plotted at /lkp/bisect-plot/vmstat.system.cs/2014-02-16_10:06:49.png

[-- Attachment #5: Type: text/plain, Size: 85 bytes --]

_______________________________________________
LKP mailing list
LKP@linux.intel.com

^ permalink raw reply related	[flat|nested] 4+ messages in thread

* Re: [sched/balance] 7511dd0a7: +2.1e+05% context switches
  2014-02-19 13:00 [sched/balance] 7511dd0a7: +2.1e+05% context switches Fengguang Wu
@ 2014-02-20  7:21 ` Alex Shi
  2014-02-25  1:33   ` Fengguang Wu
  0 siblings, 1 reply; 4+ messages in thread
From: Alex Shi @ 2014-02-20  7:21 UTC (permalink / raw)
  To: Fengguang Wu; +Cc: LKML

On 02/19/2014 09:00 PM, Fengguang Wu wrote:
> bc575710efe937e  7511dd0a73aaf2ca4bcd829f9
> ---------------  -------------------------
>       2029 ~ 0%    +222.9%       6551 ~17%  lkp-snb01/micro/will-it-scale/pthread_mutex2
>     143678 ~42%  +4.8e+05%  6.927e+08 ~ 0%  lkp-snb01/micro/will-it-scale/sched_yield
>     145708 ~42%  +4.8e+05%  6.927e+08 ~ 0%  TOTAL time.involuntary_context_switches


Thanks for testing, Fengguang!

Does the context switch increasing happen on whole patchset? or just
happens on this patch?

-- 
Thanks
    Alex

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [sched/balance] 7511dd0a7: +2.1e+05% context switches
  2014-02-20  7:21 ` Alex Shi
@ 2014-02-25  1:33   ` Fengguang Wu
  2014-02-25  3:10     ` Alex Shi
  0 siblings, 1 reply; 4+ messages in thread
From: Fengguang Wu @ 2014-02-25  1:33 UTC (permalink / raw)
  To: Alex Shi; +Cc: LKML

On Thu, Feb 20, 2014 at 03:21:29PM +0800, Alex Shi wrote:
> On 02/19/2014 09:00 PM, Fengguang Wu wrote:
> > bc575710efe937e  7511dd0a73aaf2ca4bcd829f9
> > ---------------  -------------------------
> >       2029 ~ 0%    +222.9%       6551 ~17%  lkp-snb01/micro/will-it-scale/pthread_mutex2
> >     143678 ~42%  +4.8e+05%  6.927e+08 ~ 0%  lkp-snb01/micro/will-it-scale/sched_yield
> >     145708 ~42%  +4.8e+05%  6.927e+08 ~ 0%  TOTAL time.involuntary_context_switches
> 
> 
> Thanks for testing, Fengguang!
> 
> Does the context switch increasing happen on whole patchset? or just
> happens on this patch?

Only this patch. Some other patches actually reduce the context switches.
This is all the changes for branch alexshi/topdown:

      v3.14-rc2  d2f8e017fdd913fc150160508  
---------------  -------------------------  
    111372 ~ 1%     +60.1%     178267 ~ 1%  lkp-ws02/micro/hackbench/1600%-threads-pipe
    111372 ~ 1%     +60.1%     178267 ~ 1%  TOTAL hackbench.throughput

      v3.14-rc2  d2f8e017fdd913fc150160508  
---------------  -------------------------  
  11862017 ~10%    +769.1%  1.031e+08 ~ 9%  lkp-ws02/micro/hackbench/1600%-threads-pipe
  11862017 ~10%    +769.1%  1.031e+08 ~ 9%  TOTAL numa-numastat.node0.numa_hit

      v3.14-rc2  d2f8e017fdd913fc150160508  
---------------  -------------------------  
  11862016 ~10%    +769.1%  1.031e+08 ~ 9%  lkp-ws02/micro/hackbench/1600%-threads-pipe
  11862016 ~10%    +769.1%  1.031e+08 ~ 9%  TOTAL numa-numastat.node0.local_node

      v3.14-rc2  d2f8e017fdd913fc150160508  
---------------  -------------------------  
   4629280 ~ 9%    +714.0%   37683303 ~ 9%  lkp-ws02/micro/hackbench/1600%-threads-pipe
   4629280 ~ 9%    +714.0%   37683303 ~ 9%  TOTAL proc-vmstat.pgalloc_dma32

      v3.14-rc2  d2f8e017fdd913fc150160508  
---------------  -------------------------  
   6476588 ~11%    +751.1%   55123483 ~ 7%  lkp-ws02/micro/hackbench/1600%-threads-pipe
   6476588 ~11%    +751.1%   55123483 ~ 7%  TOTAL numa-vmstat.node0.numa_hit

      v3.14-rc2  d2f8e017fdd913fc150160508  
---------------  -------------------------  
   6380486 ~11%    +761.9%   54995307 ~ 7%  lkp-ws02/micro/hackbench/1600%-threads-pipe
   6380486 ~11%    +761.9%   54995307 ~ 7%  TOTAL numa-vmstat.node0.numa_local

      v3.14-rc2  d2f8e017fdd913fc150160508  
---------------  -------------------------  
     24214 ~ 1%     -71.8%       6824 ~ 7%  lkp-a03/micro/will-it-scale/sched_yield
     55173 ~ 0%     -76.9%      12720 ~ 0%  lkp-snb01/micro/will-it-scale/pthread_mutex2
     55292 ~ 0%     -78.9%      11653 ~ 3%  lkp-snb01/micro/will-it-scale/sched_yield
    134680 ~ 0%     -76.8%      31198 ~ 3%  TOTAL interrupts.RES

      v3.14-rc2  d2f8e017fdd913fc150160508  
---------------  -------------------------  
     25891 ~ 1%     +99.5%      51664 ~ 0%  lkp-a03/micro/will-it-scale/sched_yield
     25891 ~ 1%     +99.5%      51664 ~ 0%  TOTAL softirqs.SCHED

      v3.14-rc2  d2f8e017fdd913fc150160508  
---------------  -------------------------  
   4282244 ~ 1%     -46.6%    2284599 ~ 8%  lkp-ws02/micro/hackbench/1600%-threads-pipe
   4282244 ~ 1%     -46.6%    2284599 ~ 8%  TOTAL proc-vmstat.pgfault

      v3.14-rc2  d2f8e017fdd913fc150160508  
---------------  -------------------------  
    105276 ~ 0%     -43.3%      59659 ~ 0%  lkp-snb01/micro/will-it-scale/pthread_mutex2
    105799 ~ 1%     -38.0%      65577 ~ 1%  lkp-snb01/micro/will-it-scale/sched_yield
    211075 ~ 0%     -40.7%     125236 ~ 0%  TOTAL cpuidle.C7-SNB.usage

      v3.14-rc2  d2f8e017fdd913fc150160508  
---------------  -------------------------  
 4.174e+08 ~ 0%     +31.2%  5.478e+08 ~ 0%  lkp-a03/micro/will-it-scale/sched_yield
 4.174e+08 ~ 0%     +31.2%  5.478e+08 ~ 0%  TOTAL cpuidle.C4.time

      v3.14-rc2  d2f8e017fdd913fc150160508  
---------------  -------------------------  
     20.64 ~ 0%     +23.9%      25.57 ~ 0%  lkp-snb01/micro/will-it-scale/sched_yield
     20.64 ~ 0%     +23.9%      25.57 ~ 0%  TOTAL turbostat.%c1

      v3.14-rc2  d2f8e017fdd913fc150160508  
---------------  -------------------------  
    214235 ~ 0%     -13.4%     185581 ~ 0%  lkp-a03/micro/will-it-scale/sched_yield
    214235 ~ 0%     -13.4%     185581 ~ 0%  TOTAL interrupts.LOC

      v3.14-rc2  d2f8e017fdd913fc150160508  
---------------  -------------------------  
    133373 ~ 0%     -10.8%     119020 ~ 0%  lkp-a03/micro/will-it-scale/sched_yield
    133373 ~ 0%     -10.8%     119020 ~ 0%  TOTAL softirqs.TIMER

      v3.14-rc2  d2f8e017fdd913fc150160508  
---------------  -------------------------  
      1109 ~ 8%   +9292.8%     104250 ~ 1%  lkp-a03/micro/will-it-scale/sched_yield
      4060 ~38%  +1.2e+05%    4868923 ~ 2%  lkp-snb01/micro/will-it-scale/sched_yield
      5169 ~32%  +96094.8%    4973173 ~ 2%  TOTAL vmstat.system.cs

      v3.14-rc2  d2f8e017fdd913fc150160508  
---------------  -------------------------  
      2045 ~ 1%    +221.7%       6578 ~ 2%  lkp-snb01/micro/will-it-scale/pthread_mutex2
      2045 ~ 1%    +221.7%       6578 ~ 2%  TOTAL time.involuntary_context_switches

      v3.14-rc2  d2f8e017fdd913fc150160508  
---------------  -------------------------  
       800 ~ 0%     -18.8%        650 ~ 0%  lkp-a03/micro/will-it-scale/sched_yield
       800 ~ 0%     -18.8%        650 ~ 0%  TOTAL vmstat.system.in

      v3.14-rc2  d2f8e017fdd913fc150160508  
---------------  -------------------------  
     50.56 ~ 0%      -1.8%      49.64 ~ 0%  lkp-snb01/micro/will-it-scale/pthread_mutex2
     50.55 ~ 0%      -9.4%      45.80 ~ 0%  lkp-snb01/micro/will-it-scale/sched_yield
    101.11 ~ 0%      -5.6%      95.44 ~ 0%  TOTAL turbostat.%c0

      v3.14-rc2  d2f8e017fdd913fc150160508  
---------------  -------------------------  
      1518 ~ 0%      -4.9%       1443 ~ 0%  lkp-snb01/micro/will-it-scale/pthread_mutex2
      1518 ~ 0%      -4.9%       1443 ~ 0%  TOTAL time.user_time

      v3.14-rc2  d2f8e017fdd913fc150160508  
---------------  -------------------------  
       488 ~ 0%      -4.8%        465 ~ 0%  lkp-snb01/micro/will-it-scale/pthread_mutex2
       488 ~ 0%      -4.8%        465 ~ 0%  TOTAL time.percent_of_cpu_this_job_got

Thanks,
Fengguang

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [sched/balance] 7511dd0a7: +2.1e+05% context switches
  2014-02-25  1:33   ` Fengguang Wu
@ 2014-02-25  3:10     ` Alex Shi
  0 siblings, 0 replies; 4+ messages in thread
From: Alex Shi @ 2014-02-25  3:10 UTC (permalink / raw)
  To: Fengguang Wu; +Cc: LKML

On 02/25/2014 09:33 AM, Fengguang Wu wrote:
> On Thu, Feb 20, 2014 at 03:21:29PM +0800, Alex Shi wrote:
>> On 02/19/2014 09:00 PM, Fengguang Wu wrote:
>>> bc575710efe937e  7511dd0a73aaf2ca4bcd829f9
>>> ---------------  -------------------------
>>>       2029 ~ 0%    +222.9%       6551 ~17%  lkp-snb01/micro/will-it-scale/pthread_mutex2
>>>     143678 ~42%  +4.8e+05%  6.927e+08 ~ 0%  lkp-snb01/micro/will-it-scale/sched_yield
>>>     145708 ~42%  +4.8e+05%  6.927e+08 ~ 0%  TOTAL time.involuntary_context_switches
>>
>>
>> Thanks for testing, Fengguang!
>>
>> Does the context switch increasing happen on whole patchset? or just
>> happens on this patch?
> 
> Only this patch. Some other patches actually reduce the context switches.
> This is all the changes for branch alexshi/topdown:

Thanks a lot for great testing!

Would you like sharing the every benchmarks run on each of testing
machine, then we not only know the benchmark change on the machine, but
also know what benchmark was not effected on the machine?

And it will be perfect if you like to give a URL for benchmarks which
you are using. :)
> 
>       v3.14-rc2  d2f8e017fdd913fc150160508  
> ---------------  -------------------------  
>     111372 ~ 1%     +60.1%     178267 ~ 1%  lkp-ws02/micro/hackbench/1600%-threads-pipe
>     111372 ~ 1%     +60.1%     178267 ~ 1%  TOTAL hackbench.throughput
> 
>       v3.14-rc2  d2f8e017fdd913fc150160508  
> ---------------  -------------------------  
>   11862017 ~10%    +769.1%  1.031e+08 ~ 9%  lkp-ws02/micro/hackbench/1600%-threads-pipe
>   11862017 ~10%    +769.1%  1.031e+08 ~ 9%  TOTAL numa-numastat.node0.numa_hit
> 
>       v3.14-rc2  d2f8e017fdd913fc150160508  
> ---------------  -------------------------  
>   11862016 ~10%    +769.1%  1.031e+08 ~ 9%  lkp-ws02/micro/hackbench/1600%-threads-pipe
>   11862016 ~10%    +769.1%  1.031e+08 ~ 9%  TOTAL numa-numastat.node0.local_node
> 
>       v3.14-rc2  d2f8e017fdd913fc150160508  
> ---------------  -------------------------  
>    4629280 ~ 9%    +714.0%   37683303 ~ 9%  lkp-ws02/micro/hackbench/1600%-threads-pipe
>    4629280 ~ 9%    +714.0%   37683303 ~ 9%  TOTAL proc-vmstat.pgalloc_dma32
> 
>       v3.14-rc2  d2f8e017fdd913fc150160508  
> ---------------  -------------------------  
>    6476588 ~11%    +751.1%   55123483 ~ 7%  lkp-ws02/micro/hackbench/1600%-threads-pipe
>    6476588 ~11%    +751.1%   55123483 ~ 7%  TOTAL numa-vmstat.node0.numa_hit
> 
>       v3.14-rc2  d2f8e017fdd913fc150160508  
> ---------------  -------------------------  
>    6380486 ~11%    +761.9%   54995307 ~ 7%  lkp-ws02/micro/hackbench/1600%-threads-pipe
>    6380486 ~11%    +761.9%   54995307 ~ 7%  TOTAL numa-vmstat.node0.numa_local
> 
>       v3.14-rc2  d2f8e017fdd913fc150160508  
> ---------------  -------------------------  
>      24214 ~ 1%     -71.8%       6824 ~ 7%  lkp-a03/micro/will-it-scale/sched_yield
>      55173 ~ 0%     -76.9%      12720 ~ 0%  lkp-snb01/micro/will-it-scale/pthread_mutex2
>      55292 ~ 0%     -78.9%      11653 ~ 3%  lkp-snb01/micro/will-it-scale/sched_yield
>     134680 ~ 0%     -76.8%      31198 ~ 3%  TOTAL interrupts.RES
> 
>       v3.14-rc2  d2f8e017fdd913fc150160508  
> ---------------  -------------------------  
>      25891 ~ 1%     +99.5%      51664 ~ 0%  lkp-a03/micro/will-it-scale/sched_yield
>      25891 ~ 1%     +99.5%      51664 ~ 0%  TOTAL softirqs.SCHED
> 
>       v3.14-rc2  d2f8e017fdd913fc150160508  
> ---------------  -------------------------  
>    4282244 ~ 1%     -46.6%    2284599 ~ 8%  lkp-ws02/micro/hackbench/1600%-threads-pipe
>    4282244 ~ 1%     -46.6%    2284599 ~ 8%  TOTAL proc-vmstat.pgfault
> 
>       v3.14-rc2  d2f8e017fdd913fc150160508  
> ---------------  -------------------------  
>     105276 ~ 0%     -43.3%      59659 ~ 0%  lkp-snb01/micro/will-it-scale/pthread_mutex2
>     105799 ~ 1%     -38.0%      65577 ~ 1%  lkp-snb01/micro/will-it-scale/sched_yield
>     211075 ~ 0%     -40.7%     125236 ~ 0%  TOTAL cpuidle.C7-SNB.usage
> 
>       v3.14-rc2  d2f8e017fdd913fc150160508  
> ---------------  -------------------------  
>  4.174e+08 ~ 0%     +31.2%  5.478e+08 ~ 0%  lkp-a03/micro/will-it-scale/sched_yield
>  4.174e+08 ~ 0%     +31.2%  5.478e+08 ~ 0%  TOTAL cpuidle.C4.time
> 
>       v3.14-rc2  d2f8e017fdd913fc150160508  
> ---------------  -------------------------  
>      20.64 ~ 0%     +23.9%      25.57 ~ 0%  lkp-snb01/micro/will-it-scale/sched_yield
>      20.64 ~ 0%     +23.9%      25.57 ~ 0%  TOTAL turbostat.%c1
> 
>       v3.14-rc2  d2f8e017fdd913fc150160508  
> ---------------  -------------------------  
>     214235 ~ 0%     -13.4%     185581 ~ 0%  lkp-a03/micro/will-it-scale/sched_yield
>     214235 ~ 0%     -13.4%     185581 ~ 0%  TOTAL interrupts.LOC
> 
>       v3.14-rc2  d2f8e017fdd913fc150160508  
> ---------------  -------------------------  
>     133373 ~ 0%     -10.8%     119020 ~ 0%  lkp-a03/micro/will-it-scale/sched_yield
>     133373 ~ 0%     -10.8%     119020 ~ 0%  TOTAL softirqs.TIMER
> 
>       v3.14-rc2  d2f8e017fdd913fc150160508  
> ---------------  -------------------------  
>       1109 ~ 8%   +9292.8%     104250 ~ 1%  lkp-a03/micro/will-it-scale/sched_yield
>       4060 ~38%  +1.2e+05%    4868923 ~ 2%  lkp-snb01/micro/will-it-scale/sched_yield
>       5169 ~32%  +96094.8%    4973173 ~ 2%  TOTAL vmstat.system.cs
> 
>       v3.14-rc2  d2f8e017fdd913fc150160508  
> ---------------  -------------------------  
>       2045 ~ 1%    +221.7%       6578 ~ 2%  lkp-snb01/micro/will-it-scale/pthread_mutex2
>       2045 ~ 1%    +221.7%       6578 ~ 2%  TOTAL time.involuntary_context_switches
> 
>       v3.14-rc2  d2f8e017fdd913fc150160508  
> ---------------  -------------------------  
>        800 ~ 0%     -18.8%        650 ~ 0%  lkp-a03/micro/will-it-scale/sched_yield
>        800 ~ 0%     -18.8%        650 ~ 0%  TOTAL vmstat.system.in
> 
>       v3.14-rc2  d2f8e017fdd913fc150160508  
> ---------------  -------------------------  
>      50.56 ~ 0%      -1.8%      49.64 ~ 0%  lkp-snb01/micro/will-it-scale/pthread_mutex2
>      50.55 ~ 0%      -9.4%      45.80 ~ 0%  lkp-snb01/micro/will-it-scale/sched_yield
>     101.11 ~ 0%      -5.6%      95.44 ~ 0%  TOTAL turbostat.%c0
> 
>       v3.14-rc2  d2f8e017fdd913fc150160508  
> ---------------  -------------------------  
>       1518 ~ 0%      -4.9%       1443 ~ 0%  lkp-snb01/micro/will-it-scale/pthread_mutex2
>       1518 ~ 0%      -4.9%       1443 ~ 0%  TOTAL time.user_time
> 
>       v3.14-rc2  d2f8e017fdd913fc150160508  
> ---------------  -------------------------  
>        488 ~ 0%      -4.8%        465 ~ 0%  lkp-snb01/micro/will-it-scale/pthread_mutex2
>        488 ~ 0%      -4.8%        465 ~ 0%  TOTAL time.percent_of_cpu_this_job_got
> 
> Thanks,
> Fengguang
> 


-- 
Thanks
    Alex

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2014-02-25  3:10 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-02-19 13:00 [sched/balance] 7511dd0a7: +2.1e+05% context switches Fengguang Wu
2014-02-20  7:21 ` Alex Shi
2014-02-25  1:33   ` Fengguang Wu
2014-02-25  3:10     ` Alex Shi

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).