From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756108Ab2LNEqP (ORCPT ); Thu, 13 Dec 2012 23:46:15 -0500 Received: from moutng.kundenserver.de ([212.227.126.186]:63677 "EHLO moutng.kundenserver.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756036Ab2LNEqN (ORCPT ); Thu, 13 Dec 2012 23:46:13 -0500 Message-ID: <1355460356.5777.12.camel@marge.simpson.net> Subject: Re: [RFC PATCH v2 3/6] sched: pack small tasks From: Mike Galbraith To: Alex Shi Cc: Vincent Guittot , linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linaro-dev@lists.linaro.org, peterz@infradead.org, mingo@kernel.org, linux@arm.linux.org.uk, pjt@google.com, santosh.shilimkar@ti.com, Morten.Rasmussen@arm.com, chander.kashyap@linaro.org, cmetcalf@tilera.com, tony.luck@intel.com, preeti@linux.vnet.ibm.com, paulmck@linux.vnet.ibm.com, tglx@linutronix.de, len.brown@intel.com, arjan@linux.intel.com, amit.kucheria@linaro.org, viresh.kumar@linaro.org Date: Fri, 14 Dec 2012 05:45:56 +0100 In-Reply-To: <50C9E552.1010600@intel.com> References: <1355319092-30980-1-git-send-email-vincent.guittot@linaro.org> <1355319092-30980-4-git-send-email-vincent.guittot@linaro.org> <50C93AC1.1060202@intel.com> <50C9E552.1010600@intel.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.2.3 Content-Transfer-Encoding: 7bit Mime-Version: 1.0 X-Provags-ID: V02:K0:35/a4KmgaDRVWeuF4WHLC0el11mr12F+0jZsOdEA1aH +On4rpyZaXNJaqcs5QPsxONEmXar7xsuWDidTKKnG07voaYfet q/gh+0UZVWs65+0k83QGv+tdia/uatxoxkW3j+UsICkW4FCo9Z cyzpC40bGlALGs2zHCR6tuU8UFPxyjlwZz0KPP+H5wv5yHgvaN G11/u0D5Gz+9fbE7IJ0JfXNUgLwCn52I/W7gtjdo2y/5j3S5tv coYVvIClwuuEaAr4DdzoOju3D3VvYIf1FdYFg1p4eSAeMpa3C3 BTfGqNCdBu9CQio2Dnv+Zf11V+DXZS4z+bN0QeKQ7KRXXclzOh 26ZUp7YlGQ+XRUntoL516cqP4WPfbr1UPJTRLkTcII+vfQ1tnn uMLPhx8yjSTsA== Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, 2012-12-13 at 22:25 +0800, Alex Shi wrote: > On 12/13/2012 06:11 PM, Vincent Guittot wrote: > > On 13 December 2012 03:17, Alex Shi wrote: > >> On 12/12/2012 09:31 PM, Vincent Guittot wrote: > >>> During the creation of sched_domain, we define a pack buddy CPU for each CPU > >>> when one is available. We want to pack at all levels where a group of CPU can > >>> be power gated independently from others. > >>> On a system that can't power gate a group of CPUs independently, the flag is > >>> set at all sched_domain level and the buddy is set to -1. This is the default > >>> behavior. > >>> On a dual clusters / dual cores system which can power gate each core and > >>> cluster independently, the buddy configuration will be : > >>> > >>> | Cluster 0 | Cluster 1 | > >>> | CPU0 | CPU1 | CPU2 | CPU3 | > >>> ----------------------------------- > >>> buddy | CPU0 | CPU0 | CPU0 | CPU2 | > >>> > >>> Small tasks tend to slip out of the periodic load balance so the best place > >>> to choose to migrate them is during their wake up. The decision is in O(1) as > >>> we only check again one buddy CPU > >> > >> Just have a little worry about the scalability on a big machine, like on > >> a 4 sockets NUMA machine * 8 cores * HT machine, the buddy cpu in whole > >> system need care 64 LCPUs. and in your case cpu0 just care 4 LCPU. That > >> is different on task distribution decision. > > > > The buddy CPU should probably not be the same for all 64 LCPU it > > depends on where it's worth packing small tasks > > Do you have further ideas for buddy cpu on such example? > > > > Which kind of sched_domain configuration have you for such system ? > > and how many sched_domain level have you ? > > it is general X86 domain configuration. with 4 levels, > sibling/core/cpu/numa. CPU is a bug that slipped into domain degeneration. You should have SIBLING/MC/NUMA (chasing that down is on todo). -Mike