From: Mike Galbraith <bitbucket@online.de>
To: Len Brown <lenb@kernel.org>
Cc: Borislav Petkov <bp@alien8.de>, Alex Shi <alex.shi@intel.com>,
mingo@redhat.com, peterz@infradead.org, tglx@linutronix.de,
akpm@linux-foundation.org, arjan@linux.intel.com, pjt@google.com,
namhyung@kernel.org, morten.rasmussen@arm.com,
vincent.guittot@linaro.org, gregkh@linuxfoundation.org,
preeti@linux.vnet.ibm.com, viresh.kumar@linaro.org,
linux-kernel@vger.kernel.org, len.brown@intel.com,
rafael.j.wysocki@intel.com, jkosina@suse.cz,
clark.williams@gmail.com, tony.luck@intel.com,
keescook@chromium.org, mgorman@suse.de, riel@redhat.com,
Linux PM list <linux-pm@vger.kernel.org>
Subject: Re: [patch v7 0/21] sched: power aware scheduling
Date: Tue, 30 Apr 2013 07:16:12 +0200 [thread overview]
Message-ID: <1367298972.4616.41.camel@marge.simpson.net> (raw)
In-Reply-To: <1366989087.30242.11.camel@marge.simpson.net>
On Fri, 2013-04-26 at 17:11 +0200, Mike Galbraith wrote:
> On Wed, 2013-04-17 at 17:53 -0400, Len Brown wrote:
> > On 04/12/2013 12:48 PM, Mike Galbraith wrote:
> > > On Fri, 2013-04-12 at 18:23 +0200, Borislav Petkov wrote:
> > >> On Fri, Apr 12, 2013 at 04:46:50PM +0800, Alex Shi wrote:
> > >>> Thanks a lot for comments, Len!
> > >>
> > >> AFAICT, you kinda forgot to answer his most important question:
> > >>
> > >>> These numbers suggest that this patch series simultaneously
> > >>> has a negative impact on performance and energy required
> > >>> to retire the workload. Why do it?
> > >
> > > Hm. When I tested AIM7 compute on a NUMA box, there was a marked
> > > throughput increase at the low to moderate load end of the test spectrum
> > > IIRC. Fully repeatable. There were also other benefits unrelated to
> > > power, ie mitigation of the evil face of select_idle_sibling(). I
> > > rather liked what I saw during ~big box test-drive.
> > >
> > > (just saying there are other aspects besides joules in there)
> >
> > Mike,
> >
> > Can you re-run your AIM7 measurement with turbo-mode and HT-mode disabled,
> > and then independently re-enable them?
> >
> > If you still see the performance benefit, then that proves
> > that the scheduler hacks are not about tricking into
> > turbo mode, but something else.
>
> I did that today, neither turbo nor HT affected the performance gain. I
> used the same box and patch set as tested before (v4), but plugged into
> linus HEAD. "powersaving" AIM7 numbers are ~identical to those I posted
> before, "performance" is lower at the low end of AIM7 test spectrum, but
> as before, delta goes away once the load becomes hefty.
Well now, that's not exactly what I expected to see for AIM7 compute.
Filesystem is munching cycles otherwise used for compute when load is
spread across the whole box vs consolidated.
performance
PerfTop: 35 irqs/sec kernel:94.3% exact: 0.0% [1000Hz cycles], (all, 80 CPUs)
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
samples pcnt function DSO
_______ _____ ______________________________ ________________________________________
9367.00 15.5% jbd2_journal_put_journal_head /lib/modules/3.9.0-default/build/vmlinux
7658.00 12.7% jbd2_journal_add_journal_head /lib/modules/3.9.0-default/build/vmlinux
7042.00 11.7% jbd2_journal_grab_journal_head /lib/modules/3.9.0-default/build/vmlinux
4433.00 7.4% sieve /abuild/mike/aim7/multitask
3248.00 5.4% jbd_lock_bh_state /lib/modules/3.9.0-default/build/vmlinux
3034.00 5.0% do_get_write_access /lib/modules/3.9.0-default/build/vmlinux
2058.00 3.4% mul_double /abuild/mike/aim7/multitask
2038.00 3.4% add_double /abuild/mike/aim7/multitask
1365.00 2.3% native_write_msr_safe /lib/modules/3.9.0-default/build/vmlinux
1333.00 2.2% __find_get_block /lib/modules/3.9.0-default/build/vmlinux
1213.00 2.0% add_long /abuild/mike/aim7/multitask
1208.00 2.0% add_int /abuild/mike/aim7/multitask
1084.00 1.8% __wait_on_bit_lock /lib/modules/3.9.0-default/build/vmlinux
1065.00 1.8% div_double /abuild/mike/aim7/multitask
901.00 1.5% intel_idle /lib/modules/3.9.0-default/build/vmlinux
812.00 1.3% _raw_spin_lock_irqsave /lib/modules/3.9.0-default/build/vmlinux
559.00 0.9% jbd2_journal_dirty_metadata /lib/modules/3.9.0-default/build/vmlinux
464.00 0.8% copy_user_generic_string /lib/modules/3.9.0-default/build/vmlinux
455.00 0.8% div_int /abuild/mike/aim7/multitask
430.00 0.7% string_rtns_1 /abuild/mike/aim7/multitask
419.00 0.7% strncat /lib64/libc-2.11.3.so
412.00 0.7% wake_bit_function /lib/modules/3.9.0-default/build/vmlinux
347.00 0.6% jbd2_journal_cancel_revoke /lib/modules/3.9.0-default/build/vmlinux
346.00 0.6% ext4_mark_iloc_dirty /lib/modules/3.9.0-default/build/vmlinux
306.00 0.5% __brelse /lib/modules/3.9.0-default/build/vmlinux
powersaving
PerfTop: 59 irqs/sec kernel:78.0% exact: 0.0% [1000Hz cycles], (all, 80 CPUs)
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
samples pcnt function DSO
_______ _____ ______________________________ ________________________________________
6383.00 22.5% sieve /abuild/mike/aim7/multitask
2380.00 8.4% mul_double /abuild/mike/aim7/multitask
2375.00 8.4% add_double /abuild/mike/aim7/multitask
1678.00 5.9% add_long /abuild/mike/aim7/multitask
1633.00 5.8% add_int /abuild/mike/aim7/multitask
1338.00 4.7% div_double /abuild/mike/aim7/multitask
770.00 2.7% strncat /lib64/libc-2.11.3.so
698.00 2.5% string_rtns_1 /abuild/mike/aim7/multitask
678.00 2.4% copy_user_generic_string /lib/modules/3.9.0-default/build/vmlinux
569.00 2.0% div_int /abuild/mike/aim7/multitask
329.00 1.2% jbd2_journal_put_journal_head /lib/modules/3.9.0-default/build/vmlinux
306.00 1.1% array_rtns /abuild/mike/aim7/multitask
298.00 1.1% do_get_write_access /lib/modules/3.9.0-default/build/vmlinux
270.00 1.0% jbd2_journal_add_journal_head /lib/modules/3.9.0-default/build/vmlinux
258.00 0.9% _int_malloc /lib64/libc-2.11.3.so
251.00 0.9% __find_get_block /lib/modules/3.9.0-default/build/vmlinux
236.00 0.8% __memset /lib/modules/3.9.0-default/build/vmlinux
224.00 0.8% jbd2_journal_grab_journal_head /lib/modules/3.9.0-default/build/vmlinux
221.00 0.8% intel_idle /lib/modules/3.9.0-default/build/vmlinux
161.00 0.6% jbd_lock_bh_state /lib/modules/3.9.0-default/build/vmlinux
161.00 0.6% start_this_handle /lib/modules/3.9.0-default/build/vmlinux
153.00 0.5% __GI_memset /lib64/libc-2.11.3.so
147.00 0.5% ext4_do_update_inode /lib/modules/3.9.0-default/build/vmlinux
135.00 0.5% jbd2_journal_stop /lib/modules/3.9.0-default/build/vmlinux
123.00 0.4% jbd2_journal_dirty_metadata /lib/modules/3.9.0-default/build/vmlinux
performance
procs -----------memory---------- ---swap-- -----io---- -system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
14 7 0 47716456 255124 674808 0 0 0 0 6183 93733 1 3 95 1 0
0 0 0 47791912 255152 602068 0 0 0 2671 14526 49606 2 2 94 1 0
1 0 0 47794384 255152 603796 0 0 0 0 68 111 0 0 100 0 0
8 6 0 47672340 255156 730040 0 0 0 0 36249 103961 2 8 86 4 0
0 0 0 47793976 255216 604616 0 0 0 2686 5322 6379 2 1 97 0 0
0 0 0 47799128 255216 603108 0 0 0 0 62 106 0 0 100 0 0
3 0 0 47795972 255300 603136 0 0 0 2626 39115 146228 3 5 88 3 0
0 0 0 47797176 255300 603284 0 0 0 43 128 216 0 0 100 0 0
0 0 0 47803244 255300 602580 0 0 0 0 78 124 0 0 100 0 0
0 0 0 47789120 255336 603940 0 0 0 2676 14085 85798 3 3 92 1 0
powersaving
0 0 0 47820780 255516 590292 0 0 0 31 81 126 0 0 100 0 0
0 0 0 47823712 255516 589376 0 0 0 0 107 190 0 0 100 0 0
0 0 0 47826608 255516 588060 0 0 0 0 76 130 0 0 100 0 0
0 0 0 47811260 255632 602080 0 0 0 2678 106 200 0 0 100 0 0
0 0 0 47812548 255632 601892 0 0 0 0 69 110 0 0 100 0 0
0 0 0 47808284 255680 604400 0 0 0 2668 1588 3451 4 2 94 0 0
0 0 0 47810300 255680 603624 0 0 0 0 77 124 0 0 100 0 0
20 3 0 47760764 255720 643744 0 0 1 0 948 2817 2 1 97 0 0
0 0 0 47817828 255756 602400 0 0 1 2703 984 797 2 0 98 0 0
0 0 0 47819548 255756 602532 0 0 0 0 93 158 0 0 100 0 0
1 0 0 47819312 255792 603080 0 0 0 2661 1774 3348 4 2 94 0 0
0 0 0 47821912 255800 602608 0 0 0 2 66 107 0 0 100 0 0
Invisible ink is pretty expensive stuff.
-Mike
next prev parent reply other threads:[~2013-04-30 5:17 UTC|newest]
Thread overview: 64+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-04-04 2:00 [patch v7 0/21] sched: power aware scheduling Alex Shi
2013-04-04 2:00 ` [patch v7 01/21] Revert "sched: Introduce temporary FAIR_GROUP_SCHED dependency for load-tracking" Alex Shi
2013-04-04 2:00 ` [patch v7 02/21] sched: set initial value of runnable avg for new forked task Alex Shi
2013-05-06 3:24 ` Preeti U Murthy
2013-04-04 2:00 ` [patch v7 03/21] sched: add sched balance policies in kernel Alex Shi
2013-04-04 2:00 ` [patch v7 04/21] sched: add sysfs interface for sched_balance_policy selection Alex Shi
2013-04-04 2:00 ` [patch v7 05/21] sched: log the cpu utilization at rq Alex Shi
2013-05-06 3:26 ` Preeti U Murthy
2013-05-06 5:22 ` Alex Shi
2013-05-06 12:03 ` Phil Carmody
2013-05-06 12:35 ` Alex Shi
2013-05-06 21:19 ` Paul Turner
2013-04-04 2:00 ` [patch v7 06/21] sched: add new sg/sd_lb_stats fields for incoming fork/exec/wake balancing Alex Shi
2013-04-04 2:00 ` [patch v7 07/21] sched: move sg/sd_lb_stats struct ahead Alex Shi
2013-04-04 2:00 ` [patch v7 08/21] sched: scale_rt_power rename and meaning change Alex Shi
2013-04-04 2:00 ` [patch v7 09/21] sched: get rq potential maximum utilization Alex Shi
2013-04-04 2:00 ` [patch v7 10/21] sched: add power aware scheduling in fork/exec/wake Alex Shi
2013-04-04 2:00 ` [patch v7 11/21] sched: add sched_burst_threshold_ns as wakeup burst indicator Alex Shi
2013-04-04 2:00 ` [patch v7 12/21] sched: using avg_idle to detect bursty wakeup Alex Shi
2013-04-04 2:00 ` [patch v7 13/21] sched: packing transitory tasks in wakeup power balancing Alex Shi
2013-04-04 2:00 ` [patch v7 14/21] sched: add power/performance balance allow flag Alex Shi
2013-04-04 2:00 ` [patch v7 15/21] sched: pull all tasks from source group Alex Shi
2013-04-04 5:59 ` Namhyung Kim
2013-04-06 11:49 ` Alex Shi
2013-04-04 2:00 ` [patch v7 16/21] sched: no balance for prefer_sibling in power scheduling Alex Shi
2013-05-06 3:26 ` Preeti U Murthy
2013-04-04 2:00 ` [patch v7 17/21] sched: add new members of sd_lb_stats Alex Shi
2013-04-04 2:00 ` [patch v7 18/21] sched: power aware load balance Alex Shi
2013-04-04 2:01 ` [patch v7 19/21] sched: lazy power balance Alex Shi
2013-04-04 2:01 ` [patch v7 20/21] sched: don't do power balance on share cpu power domain Alex Shi
2013-04-08 3:17 ` Preeti U Murthy
2013-04-08 3:25 ` Preeti U Murthy
2013-04-08 4:19 ` Alex Shi
2013-04-04 2:01 ` [patch v7 21/21] sched: make sure select_tas_rq_fair get a cpu Alex Shi
2013-04-11 21:02 ` [patch v7 0/21] sched: power aware scheduling Len Brown
2013-04-12 8:46 ` Alex Shi
2013-04-12 16:23 ` Borislav Petkov
2013-04-12 16:48 ` Mike Galbraith
2013-04-12 17:12 ` Borislav Petkov
2013-04-14 1:36 ` Alex Shi
2013-04-17 21:53 ` Len Brown
2013-04-18 1:51 ` Mike Galbraith
2013-04-26 15:11 ` Mike Galbraith
2013-04-30 5:16 ` Mike Galbraith [this message]
2013-04-30 8:30 ` Mike Galbraith
2013-04-30 8:41 ` Ingo Molnar
2013-04-30 9:35 ` Mike Galbraith
2013-04-30 9:49 ` Mike Galbraith
2013-04-30 9:56 ` Mike Galbraith
2013-05-17 8:06 ` Preeti U Murthy
2013-05-20 1:01 ` Alex Shi
2013-05-20 2:30 ` Preeti U Murthy
2013-04-14 1:28 ` Alex Shi
2013-04-14 5:10 ` Alex Shi
2013-04-14 15:59 ` Borislav Petkov
2013-04-15 6:04 ` Alex Shi
2013-04-15 6:16 ` Alex Shi
2013-04-15 9:52 ` Borislav Petkov
2013-04-15 13:50 ` Alex Shi
2013-04-15 23:12 ` Borislav Petkov
2013-04-16 0:22 ` Alex Shi
2013-04-16 10:24 ` Borislav Petkov
2013-04-17 1:18 ` Alex Shi
2013-04-17 7:38 ` Borislav Petkov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1367298972.4616.41.camel@marge.simpson.net \
--to=bitbucket@online.de \
--cc=akpm@linux-foundation.org \
--cc=alex.shi@intel.com \
--cc=arjan@linux.intel.com \
--cc=bp@alien8.de \
--cc=clark.williams@gmail.com \
--cc=gregkh@linuxfoundation.org \
--cc=jkosina@suse.cz \
--cc=keescook@chromium.org \
--cc=len.brown@intel.com \
--cc=lenb@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pm@vger.kernel.org \
--cc=mgorman@suse.de \
--cc=mingo@redhat.com \
--cc=morten.rasmussen@arm.com \
--cc=namhyung@kernel.org \
--cc=peterz@infradead.org \
--cc=pjt@google.com \
--cc=preeti@linux.vnet.ibm.com \
--cc=rafael.j.wysocki@intel.com \
--cc=riel@redhat.com \
--cc=tglx@linutronix.de \
--cc=tony.luck@intel.com \
--cc=vincent.guittot@linaro.org \
--cc=viresh.kumar@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.