From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 671B7C433FE for ; Fri, 11 Dec 2020 10:25:28 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0923523D50 for ; Fri, 11 Dec 2020 10:25:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0923523D50 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=techsingularity.net Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:Message-ID: Subject:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=NOc/8q91RIfDnkJ8vhh181bXYW5CHwoScj5umNIefB0=; b=pxP9hFR7r/MnBpWBVx/xx6Dg1 ibyQAT4kfjZoYFa6RIZkbP98wcmVnl/DA75YGe80meAsfwGMwyiMlHZsQWtQn+tNDmWXGDkIN/WG6 v+7gOOmQpRu9+0CNkxrV42PajOc6P0Hp3BXcKXNuh6b6C/zZKiVU7/pxQpQiD1OCzYssLeg7BpEou 9yDo2xTPbzdJJTKmi3aenvYJ63PsvScFZwcvV5GHQtxS6BuwoLNhxSX0oqJQBU+ok3uXuMj0k5jFN uVGZbzGN7s1tJF6zCjA7g/qqLd8Vgs/zk7Dol7aSayM0iW8PuDkMJQ+chM9l7cbv8brIeL5q5eR21 1fi4c03Kg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1knfap-0005Oq-8y; Fri, 11 Dec 2020 10:24:07 +0000 Received: from outbound-smtp44.blacknight.com ([46.22.136.52]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1knfam-0005OE-JC for linux-arm-kernel@lists.infradead.org; Fri, 11 Dec 2020 10:24:06 +0000 Received: from mail.blacknight.com (pemlinmail06.blacknight.ie [81.17.255.152]) by outbound-smtp44.blacknight.com (Postfix) with ESMTPS id 8E809F83FA for ; Fri, 11 Dec 2020 10:23:59 +0000 (GMT) Received: (qmail 11456 invoked from network); 11 Dec 2020 10:23:59 -0000 Received: from unknown (HELO techsingularity.net) (mgorman@techsingularity.net@[84.203.22.4]) by 81.17.254.9 with ESMTPSA (AES256-SHA encrypted, authenticated); 11 Dec 2020 10:23:59 -0000 Date: Fri, 11 Dec 2020 10:23:57 +0000 From: Mel Gorman To: Vincent Guittot Subject: Re: [PATCH 0/4] Reduce scanning of runqueues in select_idle_sibling Message-ID: <20201211102357.GW3371@techsingularity.net> References: <20201208153501.1467-1-mgorman@techsingularity.net> <20201209143748.GP3371@techsingularity.net> <20201210110424.GR3371@techsingularity.net> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201211_052404_765822_4EFB0D7D X-CRM114-Status: GOOD ( 32.52 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juri Lelli , Barry Song , Peter Ziljstra , Aubrey Li , LKML , Ingo Molnar , Valentin Schneider , Linux-ARM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, Dec 11, 2020 at 10:51:17AM +0100, Vincent Guittot wrote: > On Thu, 10 Dec 2020 at 12:04, Mel Gorman wrote: > > > > On Thu, Dec 10, 2020 at 10:38:37AM +0100, Vincent Guittot wrote: > > > > while testing your patchset and Aubrey one on top of tip, I'm facing > > > > some perf regression on my arm64 numa system on hackbench and reaim. > > > > The regression seems to comes from your patchset but i don't know > > > > which patch in particular yet > > > > > > > > hackbench -l 256000 -g 1 > > > > > > > > v5.10-rc7 + tip/sched/core 13,255(+/- 3.22%) > > > > with your patchset 15.368(+/- 2.74) -15.9% > > > > > > > > I'm also seeing perf regression on reaim but this one needs more > > > > investigation before confirming > > > > > > > > TBH, I was not expecting regressions. I'm running more test to find > > > > which patch is the culprit > > > > > > The regression comes from patch 3: sched/fair: Do not replace > > > recent_used_cpu with the new target > > > > > > > That's not entirely surprising. The intent of the patch is to increase the > > hit rate of p->recent_used_cpu but it's not a guaranteed win due to two > > corner cases. If multiple tasks have the same p->recent_used_cpu, they can > > race to use that CPU and stack as a result instead of searching the domain. > > If SMT is enabled then p->recent_used_cpu can point to an idle CPU that has > > a busy sibling which the search would have avoided in select_idle_core(). > > > > I think you are using processes and sockets for hackbench but as you'll > > see later, hackbench can be used both to show losses and gains. > > I run more hackbench tests with pipe and socket and both show > regression with patch 3 whereas this is significant improvement with > other patches and Aubrey's one > Is SMT enabled on your test machine? If not, then patch 4 should make no difference but if SMT is enabled, I wonder how this untested version of patch 3 behaves for you. The main difference is that the recent used cpu is used as a search target so that it would still check if it's an idle core and if not, fall through so it's used as an idle CPU after checking it's allowed by p->cpus_ptr. diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 5c41875aec23..63980bcf6e70 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -6275,21 +6275,14 @@ static int select_idle_sibling(struct task_struct *p, int prev, int target) return prev; } - /* Check a recently used CPU as a potential idle candidate: */ + /* Check a recently used CPU as a search target: */ recent_used_cpu = p->recent_used_cpu; + p->recent_used_cpu = prev; if (recent_used_cpu != prev && recent_used_cpu != target && cpus_share_cache(recent_used_cpu, target) && - (available_idle_cpu(recent_used_cpu) || sched_idle_cpu(recent_used_cpu)) && - cpumask_test_cpu(p->recent_used_cpu, p->cpus_ptr) && - asym_fits_capacity(task_util, recent_used_cpu)) { - /* - * Replace recent_used_cpu with prev as it is a potential - * candidate for the next wake: - */ - p->recent_used_cpu = prev; - return recent_used_cpu; - } + (available_idle_cpu(recent_used_cpu) || sched_idle_cpu(recent_used_cpu))) + target = recent_used_cpu; /* * For asymmetric CPU capacity systems, our domain of interest is @@ -6768,9 +6761,6 @@ select_task_rq_fair(struct task_struct *p, int prev_cpu, int wake_flags) } else if (wake_flags & WF_TTWU) { /* XXX always ? */ /* Fast path */ new_cpu = select_idle_sibling(p, prev_cpu, new_cpu); - - if (want_affine) - current->recent_used_cpu = cpu; } rcu_read_unlock(); -- Mel Gorman SUSE Labs _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel