From: Ingo Molnar <mingo@elte.hu>
To: Srivatsa Vaddagiri <vatsa@linux.vnet.ibm.com>
Cc: Mike Galbraith <efault@gmx.de>,
linux-kernel@vger.kernel.org,
Peter Zijlstra <a.p.zijlstra@chello.nl>,
Dhaval Giani <dhaval@linux.vnet.ibm.com>,
Dmitry Adamushko <dmitry.adamushko@gmail.com>,
Andrew Morton <akpm@linux-foundation.org>
Subject: Re: [git] CFS-devel, latest code
Date: Tue, 25 Sep 2007 12:36:17 +0200 [thread overview]
Message-ID: <20070925103617.GA3426@elte.hu> (raw)
In-Reply-To: <20070925102855.GL26289@linux.vnet.ibm.com>
* Srivatsa Vaddagiri <vatsa@linux.vnet.ibm.com> wrote:
> On Tue, Sep 25, 2007 at 12:10:44PM +0200, Ingo Molnar wrote:
> > So the patch below just removes the is_same_group() condition. But i can
> > still see bad (and obvious) latencies with Mike's 2-hogs test:
> >
> > taskset 01 perl -e 'while (1) {}' &
> > nice -19 taskset 02 perl -e 'while (1) {}' &
> >
> > So something's amiss.
>
> While I try recreating this myself, I wonder if this patch helps?
you should be able to recreate this easily by booting with maxcpus=1 and
the commands above - then run a few instances of chew-max (without them
being bound to any particular CPUs) and the latencies should show up.
i have tried your patch and it does not solve the problem - i think
there's a more fundamental bug lurking, besides the wakeup latency
problem.
Find below a /proc/sched_debug output of a really large latency. The
latency is caused by the _huge_ (~450 seconds!) vruntime offset that
'loop_silent' and 'sshd' has:
task PID tree-key switches prio exec-runtime
-------------------------------------------------------------------
loop_silent 2391 55344.211189 203 120 55344.211189
sshd 2440 513334.978030 4 120 513334.978030
R cat 2496 513672.558835 4 120 513672.558835
hm. perhaps this fixup in kernel/sched.c:set_task_cpu():
p->se.vruntime -= old_rq->cfs.min_vruntime - new_rq->cfs.min_vruntime;
needs to become properly group-hierarchy aware?
Ingo
------------------>
Sched Debug Version: v0.05-v20, 2.6.23-rc7 #89
now at 95878.065440 msecs
.sysctl_sched_latency : 20.000000
.sysctl_sched_min_granularity : 2.000000
.sysctl_sched_wakeup_granularity : 2.000000
.sysctl_sched_batch_wakeup_granularity : 25.000000
.sysctl_sched_child_runs_first : 0.000001
.sysctl_sched_features : 3
cpu#0, 1828.868 MHz
.nr_running : 3
.load : 3072
.nr_switches : 32032
.nr_load_updates : 95906
.nr_uninterruptible : 4294967238
.jiffies : 4294763202
.next_balance : 4294.763420
.curr->pid : 2496
.clock : 95893.484495
.idle_clock : 55385.089335
.prev_clock_raw : 84753.749367
.clock_warps : 0
.clock_overflows : 1737
.clock_deep_idle_events : 71815
.clock_max_delta : 0.999843
.cpu_load[0] : 3072
.cpu_load[1] : 2560
.cpu_load[2] : 2304
.cpu_load[3] : 2176
.cpu_load[4] : 2119
cfs_rq
.exec_clock : 38202.223241
.MIN_vruntime : 36334.281860
.min_vruntime : 36334.279140
.max_vruntime : 36334.281860
.spread : 0.000000
.spread0 : 0.000000
.nr_running : 2
.load : 3072
.bkl_cnt : 3934
.nr_spread_over : 37
cfs_rq
.exec_clock : 34769.316246
.MIN_vruntime : 55344.211189
.min_vruntime : 36334.279140
.max_vruntime : 513334.978030
.spread : 457990.766841
.spread0 : 0.000000
.nr_running : 2
.load : 2048
.bkl_cnt : 3934
.nr_spread_over : 10
cfs_rq
.exec_clock : 36.982394
.MIN_vruntime : 0.000001
.min_vruntime : 36334.279140
.max_vruntime : 0.000001
.spread : 0.000000
.spread0 : 0.000000
.nr_running : 0
.load : 0
.bkl_cnt : 3934
.nr_spread_over : 1
cfs_rq
.exec_clock : 20.244893
.MIN_vruntime : 0.000001
.min_vruntime : 36334.279140
.max_vruntime : 0.000001
.spread : 0.000000
.spread0 : 0.000000
.nr_running : 0
.load : 0
.bkl_cnt : 3934
.nr_spread_over : 0
cfs_rq
.exec_clock : 3305.155973
.MIN_vruntime : 0.000001
.min_vruntime : 36334.279140
.max_vruntime : 0.000001
.spread : 0.000000
.spread0 : 0.000000
.nr_running : 1
.load : 1024
.bkl_cnt : 3934
.nr_spread_over : 13
runnable tasks:
task PID tree-key switches prio exec-runtime sum-exec sum-sleep
----------------------------------------------------------------------------------------------------------
loop_silent 2391 55344.211189 203 120 55344.211189 34689.036169 42.855690
sshd 2440 513334.978030 4 120 513334.978030 0.726092 0.000000
R cat 2496 513672.558835 4 120 513672.558835 0.656690 26.294621
cpu#1, 1828.868 MHz
.nr_running : 3
.load : 2063
.nr_switches : 22792
.nr_load_updates : 95625
.nr_uninterruptible : 58
.jiffies : 4294763202
.next_balance : 4294.763333
.curr->pid : 2427
.clock : 95643.855219
.idle_clock : 54735.436719
.prev_clock_raw : 84754.067800
.clock_warps : 0
.clock_overflows : 2521
.clock_deep_idle_events : 120633
.clock_max_delta : 0.999843
.cpu_load[0] : 2063
.cpu_load[1] : 2063
.cpu_load[2] : 2063
.cpu_load[3] : 2063
.cpu_load[4] : 2063
cfs_rq
.exec_clock : 38457.557282
.MIN_vruntime : 0.000001
.min_vruntime : 35360.227495
.max_vruntime : 0.000001
.spread : 0.000000
.spread0 : -974.051645
.nr_running : 1
.load : 1024
.bkl_cnt : 456
.nr_spread_over : 18
cfs_rq
.exec_clock : 32536.311766
.MIN_vruntime : 610653.297636
.min_vruntime : 35360.227495
.max_vruntime : 610702.945490
.spread : 49.647854
.spread0 : -974.051645
.nr_running : 3
.load : 2063
.bkl_cnt : 456
.nr_spread_over : 202
cfs_rq
.exec_clock : 17.392835
.MIN_vruntime : 0.000001
.min_vruntime : 35360.227495
.max_vruntime : 0.000001
.spread : 0.000000
.spread0 : -974.051645
.nr_running : 0
.load : 0
.bkl_cnt : 456
.nr_spread_over : 1
cfs_rq
.exec_clock : 0.428251
.MIN_vruntime : 0.000001
.min_vruntime : 35360.227495
.max_vruntime : 0.000001
.spread : 0.000000
.spread0 : -974.051645
.nr_running : 0
.load : 0
.bkl_cnt : 456
.nr_spread_over : 1
cfs_rq
.exec_clock : 5812.752391
.MIN_vruntime : 0.000001
.min_vruntime : 35360.227495
.max_vruntime : 0.000001
.spread : 0.000000
.spread0 : -974.051645
.nr_running : 0
.load : 0
.bkl_cnt : 456
.nr_spread_over : 11
runnable tasks:
task PID tree-key switches prio exec-runtime sum-exec sum-sleep
----------------------------------------------------------------------------------------------------------
loop_silent 2400 610702.945490 859 139 610702.945490 8622.783425 13.951214
R chew-max 2427 610644.849769 2057 120 610644.849769 13737.197972 10.079232
chew-max 2433 610653.297636 1969 120 610653.297636 9679.778096 10.750704
next prev parent reply other threads:[~2007-09-25 10:36 UTC|newest]
Thread overview: 69+ messages / expand[flat|nested] mbox.gz Atom feed top
2007-09-24 21:45 [git] CFS-devel, latest code Ingo Molnar
2007-09-24 21:55 ` Andrew Morton
2007-09-24 21:59 ` Ingo Molnar
2007-09-25 0:08 ` Daniel Walker
2007-09-25 6:45 ` Ingo Molnar
2007-09-25 15:17 ` Daniel Walker
2007-09-25 6:10 ` Mike Galbraith
2007-09-25 7:35 ` Mike Galbraith
2007-09-25 8:33 ` Mike Galbraith
2007-09-25 8:53 ` Srivatsa Vaddagiri
2007-09-25 9:11 ` Srivatsa Vaddagiri
2007-09-25 9:15 ` Mike Galbraith
2007-09-25 9:12 ` Mike Galbraith
2007-09-25 9:13 ` Ingo Molnar
2007-09-25 9:17 ` Mike Galbraith
2007-09-25 9:47 ` Ingo Molnar
2007-09-25 10:02 ` Mike Galbraith
2007-09-26 8:04 ` Mike Galbraith
2007-09-28 21:46 ` Bill Davidsen
2007-09-25 9:44 ` Srivatsa Vaddagiri
2007-09-25 9:40 ` Ingo Molnar
2007-09-25 10:10 ` Ingo Molnar
2007-09-25 10:28 ` Srivatsa Vaddagiri
2007-09-25 10:36 ` Ingo Molnar [this message]
2007-09-25 11:33 ` Ingo Molnar
2007-09-25 14:48 ` Srivatsa Vaddagiri
2007-09-25 12:51 ` Srivatsa Vaddagiri
2007-09-25 13:35 ` Mike Galbraith
2007-09-25 14:07 ` Srivatsa Vaddagiri
2007-09-25 12:28 ` Mike Galbraith
2007-09-25 12:54 ` Mike Galbraith
[not found] ` <20070925131717.GM26289@linux.vnet.ibm.com>
[not found] ` <1190725693.13716.10.camel@Homer.simpson.net>
[not found] ` <20070925132528.GN26289@linux.vnet.ibm.com>
[not found] ` <1190726682.11260.1.camel@Homer.simpson.net>
[not found] ` <20070925140559.GB26310@linux.vnet.ibm.com>
[not found] ` <20070925143755.GA15594@elte.hu>
[not found] ` <20070926210737.GA8663@elte.hu>
2007-10-01 14:04 ` [RFC/PATCH] Add sysfs control to modify a user's cpu share Dhaval Giani
2007-10-01 14:44 ` Ingo Molnar
2007-10-01 15:32 ` Srivatsa Vaddagiri
2007-10-02 22:12 ` Eric St-Laurent
2007-10-03 4:09 ` Srivatsa Vaddagiri
2007-10-03 17:10 ` [RFC/PATCH -v2] " Dhaval Giani
2007-10-04 7:57 ` Ingo Molnar
2007-10-04 8:54 ` Heiko Carstens
2007-10-04 16:02 ` Bill Davidsen
2007-10-04 17:20 ` Srivatsa Vaddagiri
2007-10-04 21:32 ` Valdis.Kletnieks
2007-10-05 7:01 ` Srivatsa Vaddagiri
2007-10-09 15:12 ` [PATCH sched-devel] Generate uevents for user creation/destruction Srivatsa Vaddagiri
2007-10-10 7:42 ` Ingo Molnar
2007-10-01 16:12 ` [RFC/PATCH] Add sysfs control to modify a user's cpu share Dave Jones
2007-10-01 16:37 ` Srivatsa Vaddagiri
2007-09-25 6:50 ` [git] CFS-devel, latest code S.Çağlar Onur
2007-09-25 9:17 ` Ingo Molnar
2007-09-25 7:41 ` Andrew Morton
2007-09-25 8:43 ` Srivatsa Vaddagiri
2007-09-25 8:48 ` Andrew Morton
2007-09-25 11:00 ` Ingo Molnar
-- strict thread matches above, loose matches on Subject: below --
2007-09-25 14:44 Ingo Molnar
2007-09-25 16:04 ` Srivatsa Vaddagiri
2007-09-25 16:08 ` Srivatsa Vaddagiri
2007-09-25 21:35 Dmitry Adamushko
2007-09-27 7:56 ` Ingo Molnar
2007-09-30 19:13 Dmitry Adamushko
2007-10-01 6:11 ` Ingo Molnar
2007-09-30 19:15 Dmitry Adamushko
2007-10-01 5:53 ` Mike Galbraith
2007-10-01 5:55 ` Ingo Molnar
2007-09-30 19:18 Dmitry Adamushko
2007-10-02 19:49 Dmitry Adamushko
2007-10-02 19:59 ` Dmitry Adamushko
2007-10-03 4:15 ` Srivatsa Vaddagiri
2007-10-04 7:40 ` Ingo Molnar
2007-10-04 7:41 ` Ingo Molnar
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20070925103617.GA3426@elte.hu \
--to=mingo@elte.hu \
--cc=a.p.zijlstra@chello.nl \
--cc=akpm@linux-foundation.org \
--cc=dhaval@linux.vnet.ibm.com \
--cc=dmitry.adamushko@gmail.com \
--cc=efault@gmx.de \
--cc=linux-kernel@vger.kernel.org \
--cc=vatsa@linux.vnet.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox