From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
	id S1754373AbbKQXzW (ORCPT <rfc822;w@1wt.eu>);
	Tue, 17 Nov 2015 18:55:22 -0500
Received: from bombadil.infradead.org ([198.137.202.9]:40526 "EHLO
	bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
	with ESMTP id S1753314AbbKQXzV (ORCPT
	<rfc822;linux-kernel@vger.kernel.org>);
	Tue, 17 Nov 2015 18:55:21 -0500
Date: Wed, 18 Nov 2015 00:55:10 +0100
From: Peter Zijlstra <peterz@infradead.org>
To: Byungchul Park <byungchul.park@lge.com>
Cc: mingo@kernel.org, linux-kernel@vger.kernel.org, yuyang.du@intel.com,
        pjt@google.com, efault@gmx.de, tglx@linutronix.de
Subject: Re: [PATCH v4 3/3] sched: optimize migration by forcing rmb() and
 updating to be called once
Message-ID: <20151117235510.GC3816@twins.programming.kicks-ass.net>
References: <1445616981-29904-1-git-send-email-byungchul.park@lge.com>
 <1445616981-29904-4-git-send-email-byungchul.park@lge.com>
 <20151109132914.GK17308@twins.programming.kicks-ass.net>
 <20151110010905.GD4164@byungchulpark-X58A-UD3R>
 <20151110121647.GZ17308@twins.programming.kicks-ass.net>
 <20151110235147.GE4164@byungchulpark-X58A-UD3R>
 <20151116125351.GT17308@twins.programming.kicks-ass.net>
 <20151117004416.GC18234@byungchulpark-X58A-UD3R>
 <20151117112149.GV3816@twins.programming.kicks-ass.net>
 <20151117233700.GD18234@byungchulpark-X58A-UD3R>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20151117233700.GD18234@byungchulpark-X58A-UD3R>
User-Agent: Mutt/1.5.21 (2012-12-30)
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Wed, Nov 18, 2015 at 08:37:00AM +0900, Byungchul Park wrote:

> Which one do you think to be fixed? The one above migrate_task_rq_fair()?
> I wonder if it would be ok even it does not hold pi_lock in
> migrate_task_rq_fair(). If you say *no problem*, I will try to fix the
> comment.

The one above migrate_task_rq_fair() is obviously broken, as
demonstrated by the move_queued_task() case.

Also, pretty much all runnable task migration code will not take
pi_lock, see also {pull,push}_{rt,dl}_task().

Note that this is done very much by design, task_rq_lock() is the thing
that fully serializes a task's scheduler state. Runnable tasks use
rq->lock, waking tasks use pi_lock.

> > I meant, if you call __set_task_cpu() before
> > sched_class::migrate_task_rq(), in that case task_rq_lock() will no
> > longer fully serialize against set_task_cpu().
> > 
> > Because once you've called __set_task_cpu(), task_rq_lock() will acquire
> > the _other_ rq->lock. And we cannot rely on our rq->lock to serialize
> > things.
> 
> I agree with you if migtrate_task_rq() can be serialized by rq->lock
> without holding pi_lock. (even though I am still wondering..)

move_queued_task() illustrates this.

> But I thought it was no problem if migrate_task_rq() was serialized only
> by pi_lock as the comment above the migrate_task_rq() describes, because
> breaking rq->lock does not affect the sericalization by pi_lock.

Right, but per the above, we cannot assume pi_lock is in fact held over
this.