From: "Li, Aubrey" <aubrey.li@linux.intel.com>
To: Vineeth Remanan Pillai <vpillai@digitalocean.com>,
Nishanth Aravamudan <naravamudan@digitalocean.com>,
Julien Desfossez <jdesfossez@digitalocean.com>,
Peter Zijlstra <peterz@infradead.org>,
Tim Chen <tim.c.chen@linux.intel.com>,
mingo@kernel.org, tglx@linutronix.de, pjt@google.com,
torvalds@linux-foundation.org
Cc: linux-kernel@vger.kernel.org, Dario Faggioli <dfaggioli@suse.com>,
fweisbec@gmail.com, keescook@chromium.org, kerrnel@google.com,
Phil Auld <pauld@redhat.com>, Aaron Lu <aaron.lwe@gmail.com>,
Aubrey Li <aubrey.intel@gmail.com>,
Valentin Schneider <valentin.schneider@arm.com>,
Mel Gorman <mgorman@techsingularity.net>,
Pawan Gupta <pawan.kumar.gupta@linux.intel.com>,
Paolo Bonzini <pbonzini@redhat.com>
Subject: Re: [RFC PATCH v4 00/19] Core scheduling v4
Date: Thu, 31 Oct 2019 19:42:53 +0800 [thread overview]
Message-ID: <ac746638-9d6e-dde6-59ef-dc6f4e19caa0@linux.intel.com> (raw)
In-Reply-To: <cover.1572437285.git.vpillai@digitalocean.com>
On 2019/10/31 2:33, Vineeth Remanan Pillai wrote:
> Fourth iteration of the Core-Scheduling feature.
>
> This version was aiming mostly at addressing the vruntime comparison
> issues with v3. The main issue seen in v3 was the starvation of
> interactive tasks when competing with cpu intensive tasks. This issue
> is mitigated to a large extent.
>
> We have tested and verified that incompatible processes are not
> selected during schedule. In terms of performance, the impact
> depends on the workload:
> - on CPU intensive applications that use all the logical CPUs with
> SMT enabled, enabling core scheduling performs better than nosmt.
> - on mixed workloads with considerable io compared to cpu usage,
> nosmt seems to perform better than core scheduling.
>
> v4 is rebased on top of 5.3.5(dc073f193b70):
> https://github.com/digitalocean/linux-coresched/tree/coresched/v4-v5.3.5
Thanks to post V4 out. Refresh the data on my side. Since we have played
with Aaron's core vruntime patch for a while, no surprise in the result.
Thanks,
-Aubrey
Environment setup
--------------------------
Skylake 8170 server, 2 NUMA nodes, 52 cores, 104 CPUs (HT on)
Case 1:
-------
cgroup1 workload, sysbench CPU mode (non AVX workload)
cgroup2 workload, gemmbench (AVX512 workload)
sysbench throughput result:
.--------------------------------------------------------------------------------------------------------------------------------------.
|NA/AVX vanilla-SMT [std% / sem%] cpu% |coresched-SMT [std% / sem%] +/- cpu% | no-SMT [std% / sem%] +/- cpu% |
|--------------------------------------------------------------------------------------------------------------------------------------|
| 1/1 1269.1 [ 0.1%/ 0.0%] 1.9% | 1272.4 [ 0.1%/ 0.0%] 0.3% 1.9% | 1272.0 [ 0.1%/ 0.0%] 0.2% 3.9% |
| 2/2 2466.9 [ 0.6%/ 0.1%] 3.9% | 2534.2 [ 0.6%/ 0.1%] 2.7% 3.8% | 2511.9 [ 0.2%/ 0.0%] 1.8% 7.7% |
| 4/4 4725.2 [ 0.3%/ 0.0%] 7.7% | 4806.3 [ 0.2%/ 0.0%] 1.7% 7.7% | 4786.7 [ 0.9%/ 0.1%] 1.3% 14.6% |
| 8/8 9353.4 [ 0.1%/ 0.0%] 14.6% | 9357.4 [ 0.1%/ 0.0%] 0.0% 14.6% | 9352.3 [ 0.1%/ 0.0%] -0.0% 30.0% |
| 16/16 17543.1 [ 1.0%/ 0.1%] 30.1% | 18120.7 [ 0.2%/ 0.0%] 3.3% 30.1% | 17864.8 [ 1.2%/ 0.1%] 1.8% 60.1% |
| 32/32 26968.8 [ 3.9%/ 0.4%] 60.1% | 29448.9 [ 3.5%/ 0.3%] 9.2% 59.9% | 25308.1 [10.7%/ 0.9%] -6.2% 97.7% |
| 48/48 30466.2 [10.4%/ 1.0%] 89.3% | 38624.4 [ 4.2%/ 0.4%] 26.8% 89.1% | 26891.2 [14.8%/ 1.0%] -11.7% 99.5% |
| 64/64 37909.3 [11.1%/ 1.1%] 97.7% | 41671.7 [ 8.7%/ 0.9%] 9.9% 97.6% | 25898.3 [16.2%/ 1.0%] -31.7% 100.0% |
|128/128 39479.4 [24.6%/ 2.5%] 100.0% | 42119.6 [ 6.3%/ 0.6%] 6.7% 99.5% | 26830.1 [16.5%/ 1.1%] -32.0% 100.0% |
|256/256 42602.1 [16.4%/ 1.6%] 100.0% | 40041.3 [ 7.0%/ 0.7%] -6.0% 99.7% | 27634.7 [15.4%/ 1.1%] -35.1% 100.0% |
'--------------------------------------------------------------------------------------------------------------------------------------'
Case 2
------
cgroup1 workload, sysbench MySQL (non AVX workload)
cgroup2 workload, gemmbench (AVX512 workload)
sysbench throughput result:
.--------------------------------------------------------------------------------------------------------------------------------------.
|NA/AVX vanilla-SMT [std% / sem%] cpu% |coresched-SMT [std% / sem%] +/- cpu% | no-SMT [std% / sem%] +/- cpu% |
|--------------------------------------------------------------------------------------------------------------------------------------|
| 1/1 1018.2 [ 1.0%/ 0.1%] 1.9% | 915.8 [ 0.9%/ 0.1%] -10.1% 1.9% | 994.0 [ 1.4%/ 0.2%] -2.4% 3.9% |
| 2/2 1941.2 [ 0.7%/ 0.1%] 3.9% | 1746.0 [ 0.5%/ 0.1%] -10.1% 3.9% | 1946.2 [ 0.8%/ 0.1%] 0.3% 7.8% |
| 4/4 3763.9 [ 0.5%/ 0.0%] 7.8% | 3426.0 [ 1.5%/ 0.2%] -9.0% 7.8% | 3745.1 [ 1.1%/ 0.1%] -0.5% 15.6% |
| 8/8 7375.5 [ 1.3%/ 0.1%] 15.5% | 6647.1 [ 1.1%/ 0.1%] -9.9% 16.1% | 7368.4 [ 0.8%/ 0.1%] -0.1% 31.1% |
| 16/16 12990.3 [ 0.6%/ 0.1%] 31.1% | 10903.7 [ 1.9%/ 0.2%] -16.1% 32.0% | 12082.6 [ 4.7%/ 0.5%] -7.0% 62.9% |
| 32/32 18238.1 [ 6.1%/ 0.6%] 62.1% | 16580.8 [ 3.0%/ 0.3%] -9.1% 62.8% | 21193.6 [ 4.9%/ 0.6%] 16.2% 97.8% |
| 48/48 21708.6 [ 8.3%/ 0.8%] 90.3% | 17064.1 [ 9.5%/ 0.9%] -21.4% 90.4% | 18531.4 [16.6%/ 1.8%] -14.6% 99.5% |
| 64/64 18636.9 [13.1%/ 1.3%] 97.9% | 12376.1 [20.9%/ 2.1%] -33.6% 96.8% | 20025.8 [14.9%/ 2.4%] 7.5% 100.0% |
|128/128 16204.2 [16.8%/ 1.7%] 99.4% | 3776.1 [88.7%/ 8.9%] -76.7% 97.6% | 20263.5 [12.7%/ 6.8%] 25.1% 100.0% |
|256/256 16730.5 [17.9%/ 1.8%] 98.9% | 1499.7 [210.3%/21.0%] -91.0% 98.4% | 17633.1 [ 7.5%/ 8.9%] 5.4% 100.0% |
'--------------------------------------------------------------------------------------------------------------------------------------'
And for this case, we care about sysbench MySQL latency(ms):
.--------------------------------------------------------------------------------------------------------------------------------------.
|NA/AVX vanilla-SMT [std% / sem%] cpu% |coresched-SMT [std% / sem%] +/- cpu% | no-SMT [std% / sem%] +/- cpu% |
|--------------------------------------------------------------------------------------------------------------------------------------|
| 1/1 1.1 [ 3.7%/ 0.4%] 1.9% | 1.1 [ 1.0%/ 0.1%] -8.9% 1.9% | 1.1 [ 4.1%/ 0.4%] -2.0% 3.9% |
| 2/2 1.1 [ 0.7%/ 0.1%] 3.9% | 1.2 [ 0.8%/ 0.1%] -10.8% 3.9% | 1.1 [ 0.8%/ 0.1%] 0.2% 7.8% |
| 4/4 1.1 [ 0.7%/ 0.1%] 7.8% | 1.3 [ 3.8%/ 0.4%] -11.8% 7.8% | 1.2 [ 2.2%/ 0.2%] -1.1% 15.6% |
| 8/8 1.2 [ 2.2%/ 0.2%] 15.5% | 1.3 [ 3.0%/ 0.3%] -11.7% 16.1% | 1.2 [ 1.8%/ 0.2%] 0.4% 31.1% |
| 16/16 1.4 [ 1.5%/ 0.1%] 31.1% | 2.0 [ 8.2%/ 0.8%] -45.8% 32.0% | 1.9 [18.2%/ 1.7%] -33.2% 62.9% |
| 32/32 2.4 [ 6.6%/ 0.7%] 62.1% | 2.6 [ 3.1%/ 0.3%] -6.2% 62.8% | 2.2 [23.5%/ 2.0%] 8.5% 97.8% |
| 48/48 2.7 [ 5.3%/ 0.5%] 90.3% | 3.4 [ 3.5%/ 0.4%] -26.1% 90.4% | 6.2 [19.3%/ 3.5%] -128.0% 99.5% |
| 64/64 5.9 [13.0%/ 1.3%] 97.9% | 8.3 [ 9.8%/ 1.0%] -40.1% 96.8% | 7.4 [16.6%/ 1.5%] -25.1% 100.0% |
|128/128 17.4 [46.8%/ 4.7%] 99.4% | 248.0 [146.9%/14.7%] -1327.8% 97.6% | 11.0 [10.5%/ 0.0%] 36.7% 100.0% |
|256/256 33.5 [67.1%/ 6.7%] 98.9% | 1279.5 [245.6%/24.6%] -3716.6% 98.4% | 21.5 [21.5%/ 0.0%] 36.0% 100.0% |
'--------------------------------------------------------------------------------------------------------------------------------------'
Note:
----
64/64: 64 sysbench threads(in one cgroup) and 64 gemmbench threads(in other cgroup) run simultaneously.
Vanilla-SMT: baseline with HT on
coresched-SMT: core scheduling enabled
no-SMT: HT off thru /sys/devices/system/cpu/smt/control
std%: standard deviation
sem%: standard error of the mean
±: improvement/regression against baseline
cpu%: derived by vmstat.idle and vmstat.iowait
next prev parent reply other threads:[~2019-10-31 11:42 UTC|newest]
Thread overview: 84+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-10-30 18:33 [RFC PATCH v4 00/19] Core scheduling v4 Vineeth Remanan Pillai
2019-10-30 18:33 ` [RFC PATCH v4 01/19] stop_machine: Fix stop_cpus_in_progress ordering Vineeth Remanan Pillai
2019-10-30 18:33 ` [RFC PATCH v4 02/19] sched: Fix kerneldoc comment for ia64_set_curr_task Vineeth Remanan Pillai
2019-10-30 18:33 ` [RFC PATCH v4 03/19] sched: Wrap rq::lock access Vineeth Remanan Pillai
2019-10-30 18:33 ` [RFC PATCH v4 04/19] sched/{rt,deadline}: Fix set_next_task vs pick_next_task Vineeth Remanan Pillai
2019-10-30 18:33 ` [RFC PATCH v4 05/19] sched: Add task_struct pointer to sched_class::set_curr_task Vineeth Remanan Pillai
2019-10-30 18:33 ` [RFC PATCH v4 06/19] sched/fair: Export newidle_balance() Vineeth Remanan Pillai
2019-10-30 18:33 ` [RFC PATCH v4 07/19] sched: Allow put_prev_task() to drop rq->lock Vineeth Remanan Pillai
2019-10-30 18:33 ` [RFC PATCH v4 08/19] sched: Rework pick_next_task() slow-path Vineeth Remanan Pillai
2019-10-30 18:33 ` [RFC PATCH v4 09/19] sched: Introduce sched_class::pick_task() Vineeth Remanan Pillai
2019-10-30 18:33 ` [RFC PATCH v4 10/19] sched: Core-wide rq->lock Vineeth Remanan Pillai
2019-10-30 18:33 ` [RFC PATCH v4 11/19] sched: Basic tracking of matching tasks Vineeth Remanan Pillai
2019-10-30 18:33 ` [RFC PATCH v4 12/19] sched: A quick and dirty cgroup tagging interface Vineeth Remanan Pillai
2019-10-30 18:33 ` [RFC PATCH v4 13/19] sched: Add core wide task selection and scheduling Vineeth Remanan Pillai
2019-10-30 18:33 ` [RFC PATCH v4 14/19] sched/fair: Add a few assertions Vineeth Remanan Pillai
2019-10-30 18:33 ` [RFC PATCH v4 15/19] sched: Trivial forced-newidle balancer Vineeth Remanan Pillai
2019-10-30 18:33 ` [RFC PATCH v4 16/19] sched: Debug bits Vineeth Remanan Pillai
2019-10-30 18:33 ` [RFC PATCH v4 17/19] sched/fair: wrapper for cfs_rq->min_vruntime Vineeth Remanan Pillai
2019-10-30 18:33 ` [RFC PATCH v4 18/19] sched/fair: core wide vruntime comparison Vineeth Remanan Pillai
2019-10-30 18:33 ` [RFC PATCH v4 19/19] sched/fair : Wake up forced idle siblings if needed Vineeth Remanan Pillai
2019-10-31 11:42 ` Li, Aubrey [this message]
2019-11-01 11:33 ` [RFC PATCH v4 00/19] Core scheduling v4 Li, Aubrey
2019-11-08 3:20 ` Li, Aubrey
2019-10-31 18:42 ` Phil Auld
2019-11-01 14:03 ` Vineeth Remanan Pillai
2019-11-01 16:35 ` Greg Kerr
2019-11-01 18:07 ` Dario Faggioli
2019-11-12 1:45 ` Dario Faggioli
2019-11-13 17:16 ` Tim Chen
2020-01-02 2:28 ` Aubrey Li
2020-01-10 23:19 ` Tim Chen
2019-11-11 19:10 ` Tim Chen
2020-01-14 1:12 ` Tim Chen
2020-01-14 15:40 ` Vineeth Remanan Pillai
2020-01-15 3:43 ` Li, Aubrey
2020-01-15 19:33 ` Tim Chen
2020-01-16 1:45 ` Aubrey Li
2020-01-17 16:00 ` Vineeth Remanan Pillai
2020-01-22 18:04 ` Gruza, Agata
2020-01-28 2:40 ` Dario Faggioli
[not found] ` <CANaguZDDpzrzdTmvjXvCmV2c+wBt6mXWSz4Vn-LJ-onc_Oj=yw@mail.gmail.com>
2020-02-01 15:31 ` Dario Faggioli
2020-02-06 0:28 ` Tim Chen
2020-02-06 22:37 ` Julien Desfossez
2020-02-12 23:07 ` Julien Desfossez
2020-02-13 18:37 ` Tim Chen
2020-02-14 6:10 ` Aubrey Li
[not found] ` <CANaguZC40mDHfL1H_9AA7H8cyd028t9PQVRqQ3kB4ha8R7hhqg@mail.gmail.com>
2020-02-15 6:01 ` Aubrey Li
[not found] ` <CANaguZBj_x_2+9KwbHCQScsmraC_mHdQB6uRqMTYMmvhBYfv2Q@mail.gmail.com>
2020-02-21 23:20 ` Julien Desfossez
2020-03-17 0:55 ` Joel Fernandes
2020-03-17 19:07 ` Tim Chen
2020-03-17 20:18 ` Tim Chen
2020-03-18 1:10 ` Joel Fernandes
2020-03-17 21:17 ` Thomas Gleixner
2020-03-17 21:58 ` Tim Chen
2020-03-18 1:03 ` Joel Fernandes
2020-03-18 2:30 ` Joel Fernandes
2020-03-18 0:52 ` Joel Fernandes
2020-03-18 11:53 ` Thomas Gleixner
2020-03-19 1:54 ` Joel Fernandes
2020-02-25 3:44 ` Aaron Lu
2020-02-25 5:32 ` Aubrey Li
2020-02-25 7:34 ` Aaron Lu
2020-02-25 10:40 ` Aubrey Li
2020-02-25 11:21 ` Aaron Lu
2020-02-25 13:41 ` Aubrey Li
[not found] ` <CANaguZD205ccu1V_2W-QuMRrJA9SjJ5ng1do4NCdLy8NDKKrbA@mail.gmail.com>
2020-02-26 3:13 ` Aaron Lu
2020-02-26 7:21 ` Aubrey Li
[not found] ` <CANaguZDQZg-Z6aNpeLcjQ-cGm3X8CQOkZ_hnJNUyqDRM=yVDFQ@mail.gmail.com>
2020-02-27 4:45 ` Aubrey Li
2020-02-28 23:55 ` Tim Chen
2020-03-03 14:59 ` Li, Aubrey
2020-03-03 23:54 ` Li, Aubrey
2020-03-05 4:33 ` Aaron Lu
2020-03-05 6:10 ` Li, Aubrey
2020-03-05 8:52 ` Aaron Lu
2020-02-27 2:04 ` Aaron Lu
2020-02-27 14:10 ` Phil Auld
2020-02-27 14:37 ` Aubrey Li
2020-02-28 2:54 ` Aaron Lu
2020-03-05 13:45 ` Aubrey Li
2020-03-06 2:41 ` Aaron Lu
2020-03-06 18:06 ` Tim Chen
2020-03-06 18:33 ` Phil Auld
2020-03-06 21:44 ` Tim Chen
2020-03-07 3:13 ` Aaron Lu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ac746638-9d6e-dde6-59ef-dc6f4e19caa0@linux.intel.com \
--to=aubrey.li@linux.intel.com \
--cc=aaron.lwe@gmail.com \
--cc=aubrey.intel@gmail.com \
--cc=dfaggioli@suse.com \
--cc=fweisbec@gmail.com \
--cc=jdesfossez@digitalocean.com \
--cc=keescook@chromium.org \
--cc=kerrnel@google.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mgorman@techsingularity.net \
--cc=mingo@kernel.org \
--cc=naravamudan@digitalocean.com \
--cc=pauld@redhat.com \
--cc=pawan.kumar.gupta@linux.intel.com \
--cc=pbonzini@redhat.com \
--cc=peterz@infradead.org \
--cc=pjt@google.com \
--cc=tglx@linutronix.de \
--cc=tim.c.chen@linux.intel.com \
--cc=torvalds@linux-foundation.org \
--cc=valentin.schneider@arm.com \
--cc=vpillai@digitalocean.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).