From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 733BCEE7FEE for ; Fri, 8 Sep 2023 12:46:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243374AbjIHMqp (ORCPT ); Fri, 8 Sep 2023 08:46:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48850 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243317AbjIHMqo (ORCPT ); Fri, 8 Sep 2023 08:46:44 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CCFA41FE2 for ; Fri, 8 Sep 2023 05:45:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1694177108; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=1Y+Afm1AbyOOay2suIW9GuS3A+26h6E5SR8gkW6kff4=; b=M7ZJt3iUveBJp9DC3YFinhLNehpTt4gruByPlaqqiIgw63aWEburSKJTsJaejXu95Z0tor Ro2zDo6v2MBWV9WesoYcimNnOcJq8umRmR+bnbJP0VU8743ppFwmC8j8F+sTUCB/KnKn6w c2XpW7TGKIoJGgCzHlGG27k0Eo32rwo= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-659-XJL12QFxN5WlgOC_lK5NiQ-1; Fri, 08 Sep 2023 08:45:05 -0400 X-MC-Unique: XJL12QFxN5WlgOC_lK5NiQ-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 6A9BA1C09CF4; Fri, 8 Sep 2023 12:45:04 +0000 (UTC) Received: from lorien.usersys.redhat.com (unknown [10.22.16.148]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9F57CC03295; Fri, 8 Sep 2023 12:45:03 +0000 (UTC) Date: Fri, 8 Sep 2023 08:45:01 -0400 From: Phil Auld To: Hao Jia Cc: mingo@redhat.com, peterz@infradead.org, mingo@kernel.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, linux-kernel@vger.kernel.org Subject: Re: [External] Re: [PATCH 0/2] Fix nohz_full vs rt bandwidth Message-ID: <20230908124501.GA471894@lorien.usersys.redhat.com> References: <20230821094927.51079-1-jiahao.os@bytedance.com> <1823b6fd-037f-38dc-2d33-0879d77768c3@bytedance.com> <20230907141703.GA441901@lorien.usersys.redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Sep 08, 2023 at 10:57:26AM +0800 Hao Jia wrote: > > > On 2023/9/7 Phil Auld wrote: > > Hi Hao, > > > > On Wed, Sep 06, 2023 at 02:45:39PM +0800 Hao Jia wrote: > > > > > > Friendly ping... > > > > > > On 2023/8/21 Hao Jia wrote: > > > > Since the commit 88c56cfeaec4 ("sched/fair: Block nohz tick_stop > > > > when cfs bandwidth in use") was merged, it handles conflicts between > > > > NOHZ full and cfs_bandwidth well, and the scheduler feature HZ_BW > > > > allows us to choose which one to prefer. > > > > > > > > This conflict also exists between NOHZ full and rt_bandwidth, > > > > these two patches try to handle it in a similar way. > > > > > > > > Are you actually hitting this in the real world? > > > > We, for example, no longer enable RT_GROUP_SCHED so this is a non-issue > > for our use cases. I'd recommend considering that. (Does it even > > work with cgroup2?) > > > > Yes, it has always been there. Regardless of whether RT_GROUP_SCHED is > enabled or not, rt bandwidth is always enabled. If RT_GROUP_SCHED is not > enabled, all rt tasks in the system are a group, and rt_runtime is 950000, > and rt_period is 1000000.So rt bandwidth is always enabled by default. Sure, there is that. But I think Daniel is actively trying to remove it. Also I'm not sure you answered my question. Are you actually hitting this in the real world? I'd be tempted to think this is a mis-configuration or mis-use of RT. Plus you can disable that throttling and use stalld to catch cases where the rt task goes out of control. I'm not totally against doing this (for what my vote counts...), I just wonder if it's really needed. It seem it may be over-engineering something that is soon to be a non-problem. Cheers, Phil > > > Thanks, > Hao > > > In some ways what you have is a simplification of code, but it also > > obfuscates the stop_tick conditions by hiding them all in the class > > specific functions. It was easier to see why the tick didn't stop > > looking at the original code. > > > > It would be better to do this only if it is really needed, in my opinion. > > > > > > Cheers, > > Phil > > > > > > patch1: Extracts a can_stop_tick() callback function for each > > > > sched_class from sched_can_stop_tick(), it will make things clearer > > > > and also convenient to handle the conflict between NOHZ full > > > > and rt_bandwidth. > > > > > > > > patch2: If the HZ_BW scheduler feature is enabled, and the RT task > > > > to be run is constrained by rt_bandwidth runtime. Then it will > > > > prevent NO_HZ full from stopping tick. > > > > > > > > Hao Jia (2): > > > > sched/core: Introduce sched_class::can_stop_tick() > > > > sched/rt: Block nohz tick_stop when rt bandwidth in use > > > > > > > > kernel/sched/core.c | 67 +++++-------------------------- > > > > kernel/sched/deadline.c | 16 ++++++++ > > > > kernel/sched/fair.c | 56 +++++++++++++++++++++++--- > > > > kernel/sched/rt.c | 89 ++++++++++++++++++++++++++++++++++++++++- > > > > kernel/sched/sched.h | 5 ++- > > > > 5 files changed, 168 insertions(+), 65 deletions(-) > > > > > > > > > > --