From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4CDD8C4332F for ; Mon, 21 Nov 2022 11:58:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231137AbiKUL6t (ORCPT ); Mon, 21 Nov 2022 06:58:49 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57794 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229677AbiKUL6q (ORCPT ); Mon, 21 Nov 2022 06:58:46 -0500 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9E4B625DE for ; Mon, 21 Nov 2022 03:58:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=o0EKc3Pe6P0zEDho1WqfbGNf67EYwO93LZKKivn25Io=; b=StZ2cJN8J1/aPZmmf1J5LCMRiY cKQCbW2OlifgO0Bx2Ah+WEaZD5/chHOyAjw7CdKxiblAmlTeiGVlAFWmp0q3hQCU501u/Q/x/Pj9Q kAIEQNWmCGeV9qf1zk/Y6+0T0sdrzFg4nhsg/a5dfjQLr0sbe0g9eEGtotmGzYt+tZopiOiXNFQ+J eE+qVpUbpl11ZdKdXtSt5Tio24tzZsa5Vz/VIEAjiRAjHX8p9ZnNNRbA8RvuZpMn54ho8fOYq3XLA Govorcb1jlq841qi7ug2LpljIxCsaWVS17yb7Iy9shBov8aJejpq/e3QlBOYPK7ynLagyArDUvlWI YKX3CpMQ==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1ox5Rf-005BNo-OT; Mon, 21 Nov 2022 11:58:39 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 2CCF0300322; Mon, 21 Nov 2022 12:58:32 +0100 (CET) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 1000) id 0E5EB2BE3B106; Mon, 21 Nov 2022 12:58:32 +0100 (CET) Date: Mon, 21 Nov 2022 12:58:31 +0100 From: Peter Zijlstra To: Chengming Zhou Cc: Josh Don , Ingo Molnar , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Daniel Bristot de Oliveira , Valentin Schneider , linux-kernel@vger.kernel.org, Tejun Heo , Michal =?iso-8859-1?Q?Koutn=FD?= , Christian Brauner , Zefan Li Subject: Re: [PATCH v3] sched: async unthrottling for cfs bandwidth Message-ID: References: <20221117005418.3499691-1-joshdon@google.com> <094299a3-f039-04c1-d749-2bea0bc14246@linux.dev> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <094299a3-f039-04c1-d749-2bea0bc14246@linux.dev> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Sun, Nov 20, 2022 at 10:22:40AM +0800, Chengming Zhou wrote: > > + if (cfs_rq->runtime_remaining > 0) { > > + if (cpu_of(rq) != this_cpu || > > + SCHED_WARN_ON(local_unthrottle)) { > > + unthrottle_cfs_rq_async(cfs_rq); > > + } else { > > + local_unthrottle = cfs_rq; > > + } > > + } else { > > + throttled = true; > > + } > > Hello, > > I don't get the point why local unthrottle is put after all the remote cpus, > since this list is FIFO? (earliest throttled cfs_rq is at the head) Let the local completion time for a CPU be W. Then if we queue a remote work after the local synchronous work, the lower bound for total completion is at least 2W. OTOH, if we first queue all remote work and then process the local synchronous work, the lower bound for total completion is W. The practical difference is that all relevant CPUs get unthrottled rougly at the same point in time, unlike with the original case, where some CPUs have the opportunity to consume W runtime while another is still throttled.