From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
	id S933908AbbJHPEg (ORCPT <rfc822;w@1wt.eu>);
	Thu, 8 Oct 2015 11:04:36 -0400
Received: from casper.infradead.org ([85.118.1.10]:59845 "EHLO
	casper.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
	with ESMTP id S933426AbbJHPEf (ORCPT
	<rfc822;linux-kernel@vger.kernel.org>);
	Thu, 8 Oct 2015 11:04:35 -0400
Date: Thu, 8 Oct 2015 17:04:31 +0200
From: Peter Zijlstra <peterz@infradead.org>
To: Oleg Nesterov <oleg@redhat.com>
Cc: heiko.carstens@de.ibm.com, Tejun Heo <tj@kernel.org>,
        Ingo Molnar <mingo@kernel.org>, Rik van Riel <riel@redhat.com>,
        Thomas Gleixner <tglx@linutronix.de>, linux-kernel@vger.kernel.org
Subject: Re: [PATCH 3/3] stop_machine: change cpu_stop_queue_two_works() to
 rely on stopper->enabled
Message-ID: <20151008150431.GO3816@twins.programming.kicks-ass.net>
References: <20151008145059.GA17916@redhat.com>
 <20151008145136.GA18149@redhat.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20151008145136.GA18149@redhat.com>
User-Agent: Mutt/1.5.21 (2012-12-30)
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Thu, Oct 08, 2015 at 04:51:36PM +0200, Oleg Nesterov wrote:
> @@ -261,12 +276,8 @@ int stop_two_cpus(unsigned int cpu1, unsigned int cpu2, cpu_stop_fn_t fn, void *
>  	set_state(&msdata, MULTI_STOP_PREPARE);
>  
>  	/*
> +	 * We do not want to migrate to inactive CPU. FIXME: move this
> +	 * into the caller.
>  	 */
>  	if (!cpu_active(cpu1) || !cpu_active(cpu2)) {
>  		preempt_enable();

So we cannot move that into the caller.. because this function sleeps
with wait_for_completion().

Or rather, it would force the caller to use get_online_cpus(), which we
worked really hard to avoid.

Also, I think we still want the patch I proposed which ensures the
stopper thread is active 'early', because the load balancer pretty much
assumes that its available. And when 'online && active' the
load-balancer is fully available.

Not only the numa balancing stop_two_cpus() caller relies on it, but
also the self migration stuff does, and at CPU_ONLINE time the cpu
really is 'free' to run anything.