From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934149AbcATOy1 (ORCPT ); Wed, 20 Jan 2016 09:54:27 -0500 Received: from mail-wm0-f66.google.com ([74.125.82.66]:35960 "EHLO mail-wm0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751892AbcATOyZ (ORCPT ); Wed, 20 Jan 2016 09:54:25 -0500 Date: Wed, 20 Jan 2016 15:54:19 +0100 From: Frederic Weisbecker To: Peter Zijlstra Cc: LKML , Byungchul Park , Chris Metcalf , Thomas Gleixner , Luiz Capitulino , Christoph Lameter , "Paul E . McKenney" , Mike Galbraith , Rik van Riel Subject: Re: [RFC PATCH 4/4] sched: Upload nohz full CPU load on task enqueue/dequeue Message-ID: <20160120145416.GB22723@lerouge> References: <1452700891-21807-1-git-send-email-fweisbec@gmail.com> <1452700891-21807-5-git-send-email-fweisbec@gmail.com> <20160119131708.GF6344@twins.programming.kicks-ass.net> <20160119170317.GC5317@lerouge> <20160120090906.GG6357@twins.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160120090906.GG6357@twins.programming.kicks-ass.net> User-Agent: Mutt/1.5.24 (2015-08-30) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Jan 20, 2016 at 10:09:06AM +0100, Peter Zijlstra wrote: > On Tue, Jan 19, 2016 at 06:03:19PM +0100, Frederic Weisbecker wrote: > > On Tue, Jan 19, 2016 at 02:17:08PM +0100, Peter Zijlstra wrote: > > > On Wed, Jan 13, 2016 at 05:01:31PM +0100, Frederic Weisbecker wrote: > > > > The full nohz CPU load is currently accounted on tick restart only. > > > > But there are a few issues with this model: > > > > > > > > _ On tick restart, if cpu_load[0] doesn't contain the load of the actual > > > > tickless load that just ran, we are going to account a wrong value. > > > > And it is very likely to be so given that cpu_load[0] doesn't have > > > > an opportunity to be updated between tick stop and tick restart. > > > > > > > > _ If the runqueue had updates that didn't trigger a tick restart, we > > > > are going to miss those CPU load changes. > > > > > > > > A solution to fix this is to update the CPU load everytime we enqueue > > > > or dequeue a task in the fair runqueue and more than a jiffy occured > > > > since the last update. > > > > > > Would not a much better solution be to do this remotely instead of from > > > one of the hottest functions in the scheduler? > > > > The problem with doing this remotely is that we can miss past cpu loads if > > there was several enqueue/dequeue operations happening while tickless. > > Its a timer based sample, it _always_ and per definition misses > intermediate state. Sure, but the problem is when these intermediate states are long enough. > > You can simply do: > > for_each_nohzfull_cpu(cpu) { > struct rq *rq = rq_of(cpu); > > raw_spin_lock(&rq->lock); > update_cpu_load_active(rq); > raw_spin_unlock(&rq->lock); > } But from where should we do that? Maybe we can do it before we call source/target_load(), on the selected targets needed by the caller? The problem is that if we do that right after a task got enqueued on the nohz runqueue, we may accidentally account it as the whole dynticks frame (I mean, if we get rid of that enqueue/dequeue accounting). > > Also, since when can we have enqueues/dequeues while NOHZ_FULL ? I > thought that was the 1 task 100% cpu case, there are no > enqueues/dequeues there. That's the most optimized case but we can definetly have small moments with more than one task running. For example if we have a workqueue, or such short and quick tasks. If the user makes use of full dynticks for soft isolation (for performance, can live with a few interrupts...), there can be short moments of multitasking.