From: Alex Shi <alex.shi@intel.com>
To: Paul Turner <pjt@google.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>,
Ingo Molnar <mingo@redhat.com>,
Thomas Gleixner <tglx@linutronix.de>,
Andrew Morton <akpm@linux-foundation.org>,
Borislav Petkov <bp@alien8.de>,
Namhyung Kim <namhyung@kernel.org>,
Mike Galbraith <efault@gmx.de>,
Morten Rasmussen <morten.rasmussen@arm.com>,
Vincent Guittot <vincent.guittot@linaro.org>,
Preeti U Murthy <preeti@linux.vnet.ibm.com>,
Viresh Kumar <viresh.kumar@linaro.org>,
LKML <linux-kernel@vger.kernel.org>, Mel Gorman <mgorman@suse.de>,
Rik van Riel <riel@redhat.com>,
Michael Wang <wangyun@linux.vnet.ibm.com>
Subject: Re: [PATCH v5 6/7] sched: consider runnable load average in move_tasks
Date: Thu, 09 May 2013 13:29:16 +0800 [thread overview]
Message-ID: <518B342C.4030808@intel.com> (raw)
In-Reply-To: <5189ACD2.1080105@intel.com>
On 05/08/2013 09:39 AM, Alex Shi wrote:
> On 05/07/2013 01:17 PM, Alex Shi wrote:
>>>> >> > Sorry, what I meant to say here is:
>>>> >> > If we're going to be using a runnable average based load here the
>>>> >> > fraction we take (currently instantaneous) in tg_load_down should be
>>>> >> > consistent.
>> > yes. I think so.
>> >
>> > So, here is the patch, could you like take a look?
> The new patchset till to this patch has a bad results on kbuild. So,
> will do testing and drop some of them or all.
After added cfs_rq->blocked_load_avg consideration in balance:
kbuild decrease 6% on my all machines, core2, nhm, snb 2P/4P boxes.
aim7 dropped 2~10% on 4P boxes.
oltp also drop much on 4P box, but it is not very stable.
hackbench dropped 20% on SNB, but increase about 15% on NHM EX box.
Kbuild vmstat show without blocked_load_avg, system has much less CS.
vmstat average values on SNB 4P machine,the box has 4P * 8cores * HT.
without bloacked_load_avg
r b swpd free buff cache si so bi bo in cs us sy id wa st
52 0 0 60820216 30578 2708963 0 0 2404 27841 49176 33642 61 8 29 0 0
with blocked_load_avg:
r b swpd free buff cache si so bi bo in cs us sy id wa st
52 0 0 61045626 32428 2714190 0 0 2272 26943 48923 49661 50 6 42 0 0
aim7 with default workfile, also show less CS.
alexs@debian-alexs:~/tiptest$ cat aim7.vmstat.good
0 0 0 64971976 11316 61352 0 0 0 9 285 589 0 0 100 0 0
0 0 0 64921204 11344 61452 0 0 0 16 36438 97399 15 7 78 0 0
1 0 0 64824712 11368 65452 0 0 0 21 71543 190097 31 15 54 0 0
461 0 0 64569296 11384 77620 0 0 0 2 4164 2113 4 1 94 0 0
1 0 0 64724120 11400 66872 0 0 0 28 107881 296017 42 22 35 0 0
124 0 0 64437332 11416 84248 0 0 0 2 9784 6339 10 4 86 0 0
87 0 0 64224904 11424 63868 0 0 0 1 148456 462921 41 28 31 0 0
1 0 0 64706668 11448 62100 0 0 0 59 33134 41977 30 10 60 0 0
32 0 0 64104320 13064 65836 0 0 324 14 75769 217348 25 16 59 0 0
88 0 0 64444028 13080 66648 0 0 0 4 121466 338645 50 27 22 0 0
2 0 0 64664168 13096 64384 0 0 0 79 20383 22746 20 6 75 0 0
40 0 0 63940308 13104 65020 0 0 0 1 103459 307360 31 20 49 0 0
58 0 0 64197384 13124 67316 0 0 1 2 121445 317690 52 28 20 0 0
average value:
r b swpd free buff cache si so bi bo in cs us sy id wa st
68 0 0 64517724 12043 67089 0 0 25 18 65708 177018 27 14 58 0 0
alexs@debian-alexs:~/tiptest$ cat aim7.vmstat.bad
193 1 0 64701572 8776 67604 0 0 0 2 42509 157649 11 8 81 0 0
0 0 0 64897084 8796 62056 0 0 0 17 15819 21496 11 3 85 0 0
316 0 0 64451460 8812 68488 0 0 0 3 86321 292694 27 17 56 0 0
0 0 0 64853132 8828 61880 0 0 0 32 28989 44443 20 6 73 0 0
82 0 0 64394268 9020 63984 0 0 174 14 74398 280771 18 14 67 0 0
0 0 0 64776500 9036 63752 0 0 0 47 69966 153509 39 16 45 0 0
292 0 0 64347432 9052 74428 0 0 0 2 16542 25876 11 4 85 0 0
340 0 0 64054336 9068 72020 0 0 0 2 132096 524224 28 26 46 0 0
1 0 0 64715984 9084 64440 0 0 0 62 47487 51573 41 13 46 0 0
156 0 0 64124992 9100 73888 0 0 0 2 27755 38801 19 8 73 0 0
326 0 0 63795768 9116 74624 0 0 0 2 138341 560004 25 26 49 0 0
0 0 0 64661592 9140 68796 0 0 0 96 77724 113395 58 20 22 0 0
1951 2 0 64605544 9148 71664 0 0 0 1 1530 2094 1 0 99 0 0
188 0 0 63856212 9164 68536 0 0 0 2 106011 361647 33 23 44 0 0
393 0 0 63941972 9180 76520 0 0 0 3 115553 360168 41 25 34 0 0
average value:
r b swpd free buff cache si so bi bo in cs us sy id wa st
282 0 0 64411856 9021 68845 0 0 11 19 65402 199222 25 13 60 0 0
I reviewed the cfs_rq->blocked_load_avg code path, no clear abnormal found.
Seems the blocked load avg is fit current balance rules.
Sometime the blocked load far bigger than runnable load. The blocked_load_avg
has a long time effect(more than half weight in 32ms), that drives wakeup task to other
cpus not locate, and give unnecessary load in periodic balance, isn't it?
--
Thanks
Alex
next prev parent reply other threads:[~2013-05-09 5:29 UTC|newest]
Thread overview: 87+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-05-06 1:45 [PATCH v5 0/7] use runnable load avg in load balance Alex Shi
2013-05-06 1:45 ` [PATCH v5 1/7] Revert "sched: Introduce temporary FAIR_GROUP_SCHED dependency for load-tracking" Alex Shi
2013-05-06 8:24 ` Paul Turner
2013-05-06 8:49 ` Alex Shi
2013-05-06 8:55 ` Paul Turner
2013-05-06 8:58 ` Alex Shi
2013-05-07 5:05 ` Alex Shi
2013-05-06 1:45 ` [PATCH v5 2/7] sched: remove SMP cover for runnable variables in cfs_rq Alex Shi
2013-05-06 4:11 ` Preeti U Murthy
2013-05-06 7:18 ` Alex Shi
2013-05-06 8:01 ` Paul Turner
2013-05-06 8:57 ` Alex Shi
2013-05-06 9:08 ` Paul Turner
2013-05-06 10:47 ` Preeti U Murthy
2013-05-06 15:02 ` Alex Shi
2013-05-07 5:07 ` Alex Shi
2013-05-06 1:45 ` [PATCH v5 3/7] sched: set initial value of runnable avg for new forked task Alex Shi
2013-05-06 8:19 ` Paul Turner
2013-05-06 9:21 ` Alex Shi
2013-05-06 10:17 ` Paul Turner
2013-05-07 2:18 ` Alex Shi
2013-05-07 3:06 ` Paul Turner
2013-05-07 3:24 ` Alex Shi
2013-05-07 5:03 ` Alex Shi
2013-05-09 8:31 ` Alex Shi
2013-05-09 9:30 ` Paul Turner
2013-05-09 14:23 ` Alex Shi
2013-05-08 11:15 ` Peter Zijlstra
2013-05-09 9:34 ` Paul Turner
2013-05-07 9:57 ` Morten Rasmussen
2013-05-07 11:05 ` Alex Shi
2013-05-07 11:20 ` Paul Turner
2013-05-08 11:34 ` Peter Zijlstra
2013-05-08 12:00 ` Paul Turner
2013-05-09 10:55 ` Morten Rasmussen
2013-05-09 8:22 ` Alex Shi
2013-05-09 9:24 ` Paul Turner
2013-05-09 13:13 ` Alex Shi
2013-05-06 10:22 ` Paul Turner
2013-05-06 15:26 ` Alex Shi
2013-05-06 15:28 ` Peter Zijlstra
2013-05-07 2:19 ` Alex Shi
2013-05-06 1:45 ` [PATCH v5 4/7] sched: update cpu load after task_tick Alex Shi
2013-05-06 1:45 ` [PATCH v5 5/7] sched: compute runnable load avg in cpu_load and cpu_avg_load_per_task Alex Shi
2013-05-06 8:46 ` Paul Turner
2013-05-06 10:19 ` Peter Zijlstra
2013-05-06 10:33 ` Paul Turner
2013-05-06 11:10 ` Peter Zijlstra
2013-05-07 6:17 ` Alex Shi
2013-06-04 1:45 ` Alex Shi
2013-06-04 1:51 ` [DISCUSSION] removing variety rq->cpu_load ? Alex Shi
2013-06-04 2:33 ` Michael Wang
2013-06-04 2:44 ` Alex Shi
2013-06-04 3:09 ` Michael Wang
2013-06-04 4:55 ` Alex Shi
2013-05-06 15:00 ` [PATCH v5 5/7] sched: compute runnable load avg in cpu_load and cpu_avg_load_per_task Alex Shi
2013-05-06 18:34 ` Paul Turner
2013-05-07 0:24 ` Alex Shi
2013-05-07 5:12 ` Alex Shi
2013-05-06 1:45 ` [PATCH v5 6/7] sched: consider runnable load average in move_tasks Alex Shi
2013-05-06 8:53 ` Paul Turner
2013-05-06 15:04 ` Peter Zijlstra
2013-05-06 20:59 ` Paul Turner
2013-05-07 5:17 ` Alex Shi
2013-05-08 1:39 ` Alex Shi
2013-05-09 1:24 ` Alex Shi
2013-05-10 13:58 ` Alex Shi
2013-05-09 5:29 ` Alex Shi [this message]
2013-05-10 14:03 ` Alex Shi
2013-05-06 15:07 ` Alex Shi
2013-05-06 1:45 ` [PATCH v5 7/7] sched: consider runnable load average in effective_load Alex Shi
2013-05-06 3:34 ` Michael Wang
2013-05-06 5:39 ` Alex Shi
2013-05-06 6:11 ` Michael Wang
2013-05-06 9:39 ` Alex Shi
2013-05-06 7:49 ` Michael Wang
2013-05-06 8:02 ` Alex Shi
2013-05-06 8:34 ` Michael Wang
2013-05-06 9:06 ` Paul Turner
2013-05-06 9:35 ` Alex Shi
2013-05-06 9:59 ` Preeti U Murthy
2013-05-07 2:43 ` Michael Wang
2013-05-07 5:43 ` Alex Shi
2013-05-08 1:33 ` Alex Shi
2013-05-06 10:00 ` Paul Turner
2013-05-06 7:10 ` Preeti U Murthy
2013-05-06 7:20 ` Michael Wang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=518B342C.4030808@intel.com \
--to=alex.shi@intel.com \
--cc=a.p.zijlstra@chello.nl \
--cc=akpm@linux-foundation.org \
--cc=bp@alien8.de \
--cc=efault@gmx.de \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@suse.de \
--cc=mingo@redhat.com \
--cc=morten.rasmussen@arm.com \
--cc=namhyung@kernel.org \
--cc=pjt@google.com \
--cc=preeti@linux.vnet.ibm.com \
--cc=riel@redhat.com \
--cc=tglx@linutronix.de \
--cc=vincent.guittot@linaro.org \
--cc=viresh.kumar@linaro.org \
--cc=wangyun@linux.vnet.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.