From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S964794AbbI2Kdp (ORCPT ); Tue, 29 Sep 2015 06:33:45 -0400 Received: from terminus.zytor.com ([198.137.202.10]:39349 "EHLO terminus.zytor.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S934682AbbI2Kda (ORCPT ); Tue, 29 Sep 2015 06:33:30 -0400 Date: Tue, 29 Sep 2015 03:30:58 -0700 From: tip-bot for Peter Zijlstra Message-ID: Cc: hpa@zytor.com, torvalds@linux-foundation.org, linux-kernel@vger.kernel.org, peterz@infradead.org, mingo@kernel.org, efault@gmx.de, tglx@linutronix.de Reply-To: hpa@zytor.com, linux-kernel@vger.kernel.org, torvalds@linux-foundation.org, peterz@infradead.org, tglx@linutronix.de, efault@gmx.de, mingo@kernel.org In-Reply-To: <20150929093519.817299442@infradead.org> References: <20150929093519.817299442@infradead.org> To: linux-tip-commits@vger.kernel.org Subject: [tip:sched/core] sched/core: Create preempt_count invariant Git-Commit-ID: e26d555f695ac8a3aa38055dd04bd23c1334723b X-Mailer: tip-git-log-daemon Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8 Content-Disposition: inline Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Commit-ID: e26d555f695ac8a3aa38055dd04bd23c1334723b Gitweb: http://git.kernel.org/tip/e26d555f695ac8a3aa38055dd04bd23c1334723b Author: Peter Zijlstra AuthorDate: Tue, 29 Sep 2015 11:28:27 +0200 Committer: Ingo Molnar CommitDate: Tue, 29 Sep 2015 12:27:40 +0200 sched/core: Create preempt_count invariant Ensure that upon scheduling preempt_count == 2; although currently an additional PREEMPT_ACTIVE is still possible. Signed-off-by: Peter Zijlstra (Intel) Cc: Linus Torvalds Cc: Mike Galbraith Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: fweisbec@gmail.com Cc: linux-kernel@vger.kernel.org Cc: oleg@redhat.com Cc: rostedt@goodmis.org Cc: umgwanakikbuti@gmail.com Link: http://lkml.kernel.org/r/20150929093519.817299442@infradead.org Signed-off-by: Ingo Molnar --- arch/x86/include/asm/preempt.h | 3 ++- include/asm-generic/preempt.h | 2 +- kernel/sched/core.c | 14 ++++++++++---- 3 files changed, 13 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/preempt.h b/arch/x86/include/asm/preempt.h index b12f810..183d95c6 100644 --- a/arch/x86/include/asm/preempt.h +++ b/arch/x86/include/asm/preempt.h @@ -31,7 +31,8 @@ static __always_inline void preempt_count_set(int pc) * must be macros to avoid header recursion hell */ #define init_task_preempt_count(p) do { \ - task_thread_info(p)->saved_preempt_count = PREEMPT_DISABLED; \ + task_thread_info(p)->saved_preempt_count = \ + 2*PREEMPT_DISABLE_OFFSET + PREEMPT_NEED_RESCHED; \ } while (0) #define init_idle_preempt_count(p, cpu) do { \ diff --git a/include/asm-generic/preempt.h b/include/asm-generic/preempt.h index 0bec580..1d6f104 100644 --- a/include/asm-generic/preempt.h +++ b/include/asm-generic/preempt.h @@ -24,7 +24,7 @@ static __always_inline void preempt_count_set(int pc) * must be macros to avoid header recursion hell */ #define init_task_preempt_count(p) do { \ - task_thread_info(p)->preempt_count = PREEMPT_DISABLED; \ + task_thread_info(p)->preempt_count = 2*PREEMPT_DISABLED; \ } while (0) #define init_idle_preempt_count(p, cpu) do { \ diff --git a/kernel/sched/core.c b/kernel/sched/core.c index a91df61..ecd585c 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -2588,11 +2588,17 @@ asmlinkage __visible void schedule_tail(struct task_struct *prev) { struct rq *rq; - /* finish_task_switch() drops rq->lock and enables preemtion */ - preempt_disable(); - rq = finish_task_switch(prev); + /* + * Still have preempt_count() == 2, from: + * + * schedule() + * preempt_disable(); // 1 + * __schedule() + * raw_spin_lock_irq(&rq->lock) // 2 + */ + rq = finish_task_switch(prev); /* drops rq->lock, preempt_count() == 1 */ balance_callback(rq); - preempt_enable(); + preempt_enable(); /* preempt_count() == 0 */ if (current->set_child_tid) put_user(task_pid_vnr(current), current->set_child_tid);