From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ralf Baechle Date: Wed, 26 May 2010 17:38:15 +0000 Subject: Re: Affinity Automation (was Re: [PATCH 3/17] arch/mips/kernel: Add Message-Id: <20100526173815.GB17043@linux-mips.org> List-Id: References: <20100526163227.GA17043@linux-mips.org> <4BFD5A19.5030602@paralogos.com> In-Reply-To: <4BFD5A19.5030602@paralogos.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: "Kevin D. Kissell" Cc: Julia Lawall , linux-mips@linux-mips.org, linux-kernel@vger.kernel.org, kernel-janitors@vger.kernel.org, Ingo Molnar , Peter Zijlstra On Wed, May 26, 2010 at 10:27:53AM -0700, Kevin D. Kissell wrote: > Date: Wed, 26 May 2010 10:27:53 -0700 > From: "Kevin D. Kissell" > To: Ralf Baechle > CC: Julia Lawall , linux-mips@linux-mips.org, > linux-kernel@vger.kernel.org, kernel-janitors@vger.kernel.org > Subject: Affinity Automation (was Re: [PATCH 3/17] arch/mips/kernel: Add > missing > read_unlock) > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > Ralf Baechle wrote: > >Your patch appears correct - and mipsmt_sys_sched_setaffinity() even > >more broken than you thought. It duplicates some code from kernel/sched.c > >and has gotten out of sync. > Yeah, that was inevitable. Since the distribution of the previous > message seems to cover concerned developers outside the MIPS > community, let me make one final(?) plea to actually do this right. > > The MIPS SMTC support for managing a single FPU context on a > processor with multiple integer TC contexts involves having the > system make automous, real time, decisions about scheduling > affinity. It may be a first, but it can't possibly be the only > case, especially as we've started seeing more and more mainstream > multi-core, multi-thread designs. System and chip resources are > going to be "closer" to one processor or another. The current Linux > paradigm is that it's the responsibility of programs, or users, to > know what the optimal placement of processes should be for a given > system platform, and while it's absolutely appropriate to provide > that level of control for the cases where the user really does know > best, it's mildly insane to make that the only way that thread > placement can be optimized. It's really the OS's job to match > demand to resources. > > My contention years ago was, and remains, that it would be a bad > idea to burden the main scheduler loop with checks for two different > levels of affinity, system-automatic and user-specified. It would > add non-trivially to the cache footprint and execution overhead of > thread dispatch, and there's no logical need for it. So the model I > proposed, and implemented in the cloned affinity system calls for > SMTC, was that a *single* affinity mask continue to be used by the > scheduler, but that the per-thread data structures carry two: The > one requested explicitly by the user, and the one actually used by > the scheduler. The idea is that normally those two are the same, > but that the system - in the MIPS SMTC case, the FPU emulator - can > overlay its constraints with the user's constraints to come up with > an intersection-of-sets constraint that satisfies both (there was a > clause which prevents system affinity heuristics from restricting > the affinity mask to a null set of CPUs, though of course the user > can do that if he really wants to). > > There's nothing particularly MIPS-specific about the problem or the > solution. Most of the mechanisms should really be in > platform-independent code, so we don't get the drift of cloned > components. > > I don't recall who owned the scheduler at the time, but whoever it > was was too busy dealing with mainstream processor problems to even > engage in a dialogue about this. Is it time to raise the question > again? That was Ingo Molnar. The scheduler is owned by these two guys: SCHEDULER M: Ingo Molnar M: Peter Zijlstra S: Maintained F: kernel/sched* F: include/linux/sched.h I've added them to cc. Ralf From mboxrd@z Thu Jan 1 00:00:00 1970 Received: with ECARTIS (v1.0.0; list linux-mips); Wed, 26 May 2010 19:38:57 +0200 (CEST) Received: from h5.dl5rb.org.uk ([81.2.74.5]:58771 "EHLO h5.dl5rb.org.uk" rhost-flags-OK-OK-OK-OK) by eddie.linux-mips.org with ESMTP id S1492219Ab0EZRix (ORCPT ); Wed, 26 May 2010 19:38:53 +0200 Received: from h5.dl5rb.org.uk (localhost.localdomain [127.0.0.1]) by h5.dl5rb.org.uk (8.14.3/8.14.3) with ESMTP id o4QHcImL025211; Wed, 26 May 2010 18:38:18 +0100 Received: (from ralf@localhost) by h5.dl5rb.org.uk (8.14.3/8.14.3/Submit) id o4QHcGmr025209; Wed, 26 May 2010 18:38:16 +0100 Date: Wed, 26 May 2010 18:38:15 +0100 From: Ralf Baechle To: "Kevin D. Kissell" Cc: Julia Lawall , linux-mips@linux-mips.org, linux-kernel@vger.kernel.org, kernel-janitors@vger.kernel.org, Ingo Molnar , Peter Zijlstra Subject: Re: Affinity Automation (was Re: [PATCH 3/17] arch/mips/kernel: Add missing read_unlock) Message-ID: <20100526173815.GB17043@linux-mips.org> References: <20100526163227.GA17043@linux-mips.org> <4BFD5A19.5030602@paralogos.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4BFD5A19.5030602@paralogos.com> User-Agent: Mutt/1.5.20 (2009-08-17) Return-Path: X-Envelope-To: <"|/home/ecartis/ecartis -s linux-mips"> (uid 0) X-Orcpt: rfc822;linux-mips@linux-mips.org Original-Recipient: rfc822;linux-mips@linux-mips.org X-archive-position: 26857 X-ecartis-version: Ecartis v1.0.0 Sender: linux-mips-bounce@linux-mips.org Errors-to: linux-mips-bounce@linux-mips.org X-original-sender: ralf@linux-mips.org Precedence: bulk X-list: linux-mips On Wed, May 26, 2010 at 10:27:53AM -0700, Kevin D. Kissell wrote: > Date: Wed, 26 May 2010 10:27:53 -0700 > From: "Kevin D. Kissell" > To: Ralf Baechle > CC: Julia Lawall , linux-mips@linux-mips.org, > linux-kernel@vger.kernel.org, kernel-janitors@vger.kernel.org > Subject: Affinity Automation (was Re: [PATCH 3/17] arch/mips/kernel: Add > missing > read_unlock) > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > Ralf Baechle wrote: > >Your patch appears correct - and mipsmt_sys_sched_setaffinity() even > >more broken than you thought. It duplicates some code from kernel/sched.c > >and has gotten out of sync. > Yeah, that was inevitable. Since the distribution of the previous > message seems to cover concerned developers outside the MIPS > community, let me make one final(?) plea to actually do this right. > > The MIPS SMTC support for managing a single FPU context on a > processor with multiple integer TC contexts involves having the > system make automous, real time, decisions about scheduling > affinity. It may be a first, but it can't possibly be the only > case, especially as we've started seeing more and more mainstream > multi-core, multi-thread designs. System and chip resources are > going to be "closer" to one processor or another. The current Linux > paradigm is that it's the responsibility of programs, or users, to > know what the optimal placement of processes should be for a given > system platform, and while it's absolutely appropriate to provide > that level of control for the cases where the user really does know > best, it's mildly insane to make that the only way that thread > placement can be optimized. It's really the OS's job to match > demand to resources. > > My contention years ago was, and remains, that it would be a bad > idea to burden the main scheduler loop with checks for two different > levels of affinity, system-automatic and user-specified. It would > add non-trivially to the cache footprint and execution overhead of > thread dispatch, and there's no logical need for it. So the model I > proposed, and implemented in the cloned affinity system calls for > SMTC, was that a *single* affinity mask continue to be used by the > scheduler, but that the per-thread data structures carry two: The > one requested explicitly by the user, and the one actually used by > the scheduler. The idea is that normally those two are the same, > but that the system - in the MIPS SMTC case, the FPU emulator - can > overlay its constraints with the user's constraints to come up with > an intersection-of-sets constraint that satisfies both (there was a > clause which prevents system affinity heuristics from restricting > the affinity mask to a null set of CPUs, though of course the user > can do that if he really wants to). > > There's nothing particularly MIPS-specific about the problem or the > solution. Most of the mechanisms should really be in > platform-independent code, so we don't get the drift of cloned > components. > > I don't recall who owned the scheduler at the time, but whoever it > was was too busy dealing with mainstream processor problems to even > engage in a dialogue about this. Is it time to raise the question > again? That was Ingo Molnar. The scheduler is owned by these two guys: SCHEDULER M: Ingo Molnar M: Peter Zijlstra S: Maintained F: kernel/sched* F: include/linux/sched.h I've added them to cc. Ralf