From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934128AbbBDOhM (ORCPT ); Wed, 4 Feb 2015 09:37:12 -0500 Received: from terminus.zytor.com ([198.137.202.10]:38760 "EHLO terminus.zytor.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S966166AbbBDOhG (ORCPT ); Wed, 4 Feb 2015 09:37:06 -0500 Date: Wed, 4 Feb 2015 06:36:30 -0800 From: tip-bot for Frederic Weisbecker Message-ID: Cc: fweisbec@gmail.com, rostedt@goodmis.org, peterz@infradead.org, torvalds@linux-foundation.org, tglx@linutronix.de, linux-kernel@vger.kernel.org, mingo@kernel.org, hpa@zytor.com Reply-To: rostedt@goodmis.org, fweisbec@gmail.com, peterz@infradead.org, torvalds@linux-foundation.org, mingo@kernel.org, hpa@zytor.com, linux-kernel@vger.kernel.org, tglx@linutronix.de In-Reply-To: <1422404652-29067-2-git-send-email-fweisbec@gmail.com> References: <1422404652-29067-2-git-send-email-fweisbec@gmail.com> To: linux-tip-commits@vger.kernel.org Subject: [tip:sched/core] sched: Pull resched loop to __schedule() callers Git-Commit-ID: bfd9b2b5f80e7289fdd50210afe4d9ca5952a865 X-Mailer: tip-git-log-daemon Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8 Content-Disposition: inline Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Commit-ID: bfd9b2b5f80e7289fdd50210afe4d9ca5952a865 Gitweb: http://git.kernel.org/tip/bfd9b2b5f80e7289fdd50210afe4d9ca5952a865 Author: Frederic Weisbecker AuthorDate: Wed, 28 Jan 2015 01:24:09 +0100 Committer: Ingo Molnar CommitDate: Wed, 4 Feb 2015 07:52:30 +0100 sched: Pull resched loop to __schedule() callers __schedule() disables preemption during its job and re-enables it afterward without doing a preemption check to avoid recursion. But if an event happens after the context switch which requires rescheduling, we need to check again if a task of a higher priority needs the CPU. A preempt irq can raise such a situation. To handle that, __schedule() loops on need_resched(). But preempt_schedule_*() functions, which call __schedule(), also loop on need_resched() to handle missed preempt irqs. Hence we end up with the same loop happening twice. Lets simplify that by attributing the need_resched() loop responsibility to all __schedule() callers. There is a risk that the outer loop now handles reschedules that used to be handled by the inner loop with the added overhead of caller details (inc/dec of PREEMPT_ACTIVE, irq save/restore) but assuming those inner rescheduling loop weren't too frequent, this shouldn't matter. Especially since the whole preemption path is now losing one loop in any case. Suggested-by: Linus Torvalds Signed-off-by: Frederic Weisbecker Signed-off-by: Peter Zijlstra (Intel) Cc: Steven Rostedt Link: http://lkml.kernel.org/r/1422404652-29067-2-git-send-email-fweisbec@gmail.com Signed-off-by: Ingo Molnar --- kernel/sched/core.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 5091fd4..b269e8a 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -2765,6 +2765,10 @@ again: * - explicit schedule() call * - return from syscall or exception to user-space * - return from interrupt-handler to user-space + * + * WARNING: all callers must re-check need_resched() afterward and reschedule + * accordingly in case an event triggered the need for rescheduling (such as + * an interrupt waking up a task) while preemption was disabled in __schedule(). */ static void __sched __schedule(void) { @@ -2773,7 +2777,6 @@ static void __sched __schedule(void) struct rq *rq; int cpu; -need_resched: preempt_disable(); cpu = smp_processor_id(); rq = cpu_rq(cpu); @@ -2840,8 +2843,6 @@ need_resched: post_schedule(rq); sched_preempt_enable_no_resched(); - if (need_resched()) - goto need_resched; } static inline void sched_submit_work(struct task_struct *tsk) @@ -2861,7 +2862,9 @@ asmlinkage __visible void __sched schedule(void) struct task_struct *tsk = current; sched_submit_work(tsk); - __schedule(); + do { + __schedule(); + } while (need_resched()); } EXPORT_SYMBOL(schedule);