From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753803AbcEWJTU (ORCPT ); Mon, 23 May 2016 05:19:20 -0400 Received: from bombadil.infradead.org ([198.137.202.9]:58251 "EHLO bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752068AbcEWJTS (ORCPT ); Mon, 23 May 2016 05:19:18 -0400 Date: Mon, 23 May 2016 11:19:07 +0200 From: Peter Zijlstra To: Mike Galbraith Cc: mingo@kernel.org, linux-kernel@vger.kernel.org, Pavan Kondeti , Ben Segall , Matt Fleming , Morten Rasmussen , Paul Turner , Thomas Gleixner , byungchul.park@lge.com, Andrew Hunter Subject: Re: [patch] sched/fair: Move se->vruntime normalization state into struct sched_entity Message-ID: <20160523091907.GD15728@worktop.ger.corp.intel.com> References: <20160510174314.355953085@infradead.org> <20160510174613.902178264@infradead.org> <1463839488.24578.45.camel@suse.de> <1463857236.10353.5.camel@gmail.com> <1463900401.30072.6.camel@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1463900401.30072.6.camel@gmail.com> User-Agent: Mutt/1.5.22.1 (2013-10-16) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Sun, May 22, 2016 at 09:00:01AM +0200, Mike Galbraith wrote: > On Sat, 2016-05-21 at 21:00 +0200, Mike Galbraith wrote: > > On Sat, 2016-05-21 at 16:04 +0200, Mike Galbraith wrote: > > > > > Wakees that were not migrated/normalized eat an unwanted min_vruntime, > > > and likely take a size XXL latency hit. Big box running master bled > > > profusely under heavy load until I turned TTWU_QUEUE off. > > May as well make it official and against master.today. Fly or die > little patchlet. > > sched/fair: Move se->vruntime normalization state into struct sched_entity Does this work? --- include/linux/sched.h | 1 + kernel/sched/core.c | 18 +++++++++++------- 2 files changed, 12 insertions(+), 7 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index 1b43b45a22b9..a2001e01b3df 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1534,6 +1534,7 @@ struct task_struct { unsigned sched_reset_on_fork:1; unsigned sched_contributes_to_load:1; unsigned sched_migrated:1; + unsigned sched_remote_wakeup:1; unsigned :0; /* force alignment to the next boundary */ /* unserialized, strictly 'current' */ diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 404c0784b1fc..7f2cae4620c7 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -1768,13 +1768,15 @@ void sched_ttwu_pending(void) cookie = lockdep_pin_lock(&rq->lock); while (llist) { + int wake_flags = 0; + p = llist_entry(llist, struct task_struct, wake_entry); llist = llist_next(llist); - /* - * See ttwu_queue(); we only call ttwu_queue_remote() when - * its a x-cpu wakeup. - */ - ttwu_do_activate(rq, p, WF_MIGRATED, cookie); + + if (p->sched_remote_wakeup) + wake_flags = WF_MIGRATED; + + ttwu_do_activate(rq, p, wake_flags, cookie); } lockdep_unpin_lock(&rq->lock, cookie); @@ -1819,10 +1821,12 @@ void scheduler_ipi(void) irq_exit(); } -static void ttwu_queue_remote(struct task_struct *p, int cpu) +static void ttwu_queue_remote(struct task_struct *p, int cpu, int wake_flags) { struct rq *rq = cpu_rq(cpu); + p->sched_remote_wakeup = !!(wake_flags & WF_MIGRATED); + if (llist_add(&p->wake_entry, &cpu_rq(cpu)->wake_list)) { if (!set_nr_if_polling(rq->idle)) smp_send_reschedule(cpu); @@ -1869,7 +1873,7 @@ static void ttwu_queue(struct task_struct *p, int cpu, int wake_flags) #if defined(CONFIG_SMP) if (sched_feat(TTWU_QUEUE) && !cpus_share_cache(smp_processor_id(), cpu)) { sched_clock_cpu(cpu); /* sync clocks x-cpu */ - ttwu_queue_remote(p, cpu); + ttwu_queue_remote(p, cpu, wake_flags); return; } #endif