From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1759598Ab2CTKnF (ORCPT ); Tue, 20 Mar 2012 06:43:05 -0400 Received: from merlin.infradead.org ([205.233.59.134]:52565 "EHLO merlin.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1759537Ab2CTKnD convert rfc822-to-8bit (ORCPT ); Tue, 20 Mar 2012 06:43:03 -0400 Message-ID: <1332240151.18960.401.camel@twins> Subject: Re: CPU Hotplug rework From: Peter Zijlstra To: Rusty Russell Cc: "Srivatsa S. Bhat" , "Paul E. McKenney" , Arjan van de Ven , Steven Rostedt , "Rafael J. Wysocki" , Srivatsa Vaddagiri , "akpm@linux-foundation.org" , Paul Gortmaker , Milton Miller , "mingo@elte.hu" , Tejun Heo , KOSAKI Motohiro , linux-kernel , Linux PM mailing list Date: Tue, 20 Mar 2012 11:42:31 +0100 In-Reply-To: <87aa3cqht9.fsf@rustcorp.com.au> References: <4F674649.2000300@linux.vnet.ibm.com> <87aa3cqht9.fsf@rustcorp.com.au> Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7BIT X-Mailer: Evolution 3.2.2- Mime-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, 2012-03-20 at 10:12 +1030, Rusty Russell wrote: > On Mon, 19 Mar 2012 20:14:25 +0530, "Srivatsa S. Bhat" wrote: > > Hi, > > > > There had been some discussion on CPU Hotplug redesign/rework > > some time ago, but it was buried under a thread with a different > > subject. > > (http://thread.gmane.org/gmane.linux.kernel/1246208/focus=1246404) > > > > So I am opening a new thread with an appropriate subject to discuss > > what needs to be done and how to go about it, as part of the rework. > > > > Peter Zijlstra and Paul McKenney had come up with TODO lists for the > > rework, and here are their extracts from the previous discussion: > > This is possible, but quite a lot of tricky auditing work. There's an > underlying assumption that stop_machine is the slow part, since it feels > so heavy. Depends on the machine and the needs. For the regular desktop with a regular kernel, the stop_machine in hotplug isn't really a problem. For _BIG_ machines stop_machine is a problem, for -RT stop_machine is a problem. So if we're going to re-architect hotplug anyway, it would be very good to get rid of it, because I really don't see any hard reasons why we would need it. > Unfortunately, this doesn't seem to be the case in my testing. The time > for hotplug seems to be moving all the threads around. So how about: Agreed, the thread creation on online is the most expensive operation. > (1) Let's not shutdown per-cpu kthreads, just leave them there to run > if the CPU comes back. Wasn't as easy as it sounds, but should be doable. > (2) Do something more efficient with userspace threads than migrating > them one at a time. Sadly that can't really be done. We need to pick up every task (userspace, but also running kernel threads) and update their state. > Otherwise, we risk doing a great deal of work and gaining nothing > (cleanups aside, of course). I don't really think its possible to spend too much time cleaning up hotplug at this point :-)