From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 02EE1C433F5 for ; Fri, 20 May 2022 12:34:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:From:References:Cc:To: Subject:MIME-Version:Date:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=F4mnofF5DUVMI2XXO4Xxg2N+MLTVE19ULFMpJczigR8=; b=sPaS30Vjdk/awm R52+lsgr930MW9hD8M2JVvDmNHvq4wrA//ikWzO+MWR189wh5QvNQQJmTBFUzx+13JAM1fxbXY8jJ obnQPctj7Ra7xHyFhbx2JIx+5nFVPiOdUErJG0NXDw5qrGN6qVkAERWMPbLwG5UL7kFEiSPv1dzvJ 3uSXVAyT+bUOuqo01+GAo7XmF5+y7XDHrlCKAEttIaFx7CYr5jKJ7KdmJ01djyV2gkHjuiCXRyPuC 8T1wJHj65TSzi7JFW41dUs0vVHcvPGjMV6Zm5CyxpVPjdJsX9/CL+i7VgTZc9wmWhm30X19yZWZqM Pgxy4qddlbYR0TQUZ3Iw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ns1p0-00CLe3-MF; Fri, 20 May 2022 12:33:34 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ns1ox-00CLc4-Sm; Fri, 20 May 2022 12:33:33 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 3916A1477; Fri, 20 May 2022 05:33:31 -0700 (PDT) Received: from [172.29.1.145] (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 629363F718; Fri, 20 May 2022 05:33:28 -0700 (PDT) Message-ID: <26f39a9d-1a02-b77d-5c89-88a1fb0e4eac@arm.com> Date: Fri, 20 May 2022 14:33:19 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.8.1 Subject: Re: [PATCH v2 3/8] arch_topology: Set cluster identifier in each core/thread from /cpu-map Content-Language: en-US To: Ionela Voinescu , Sudeep Holla Cc: Atish Patra , linux-kernel@vger.kernel.org, Atish Patra , Vincent Guittot , Morten Rasmussen , Qing Wang , linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, Rob Herring References: <20220518093325.2070336-1-sudeep.holla@arm.com> <20220518093325.2070336-4-sudeep.holla@arm.com> From: Dietmar Eggemann In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220520_053332_048101_9D09E857 X-CRM114-Status: GOOD ( 24.24 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 19/05/2022 18:55, Ionela Voinescu wrote: > Hi, > > As said before, this creates trouble for CONFIG_SCHED_CLUSTER=y. > The output below is obtained from Juno. > > When cluster_id is populated, a new CLS level is created by the scheduler > topology code. In this case the clusters in DT determine that the cluster > siblings and llc siblings are the same so the MC scheduler domain will > be removed and, for Juno, only CLS and DIE will be kept. [...] > To be noted that we also get a new flag SD_PREFER_SIBLING for the CLS > level that is not appropriate. We usually remove it for the child of a > SD_ASYM_CPUCAPACITY domain, but we don't currently redo this after some > levels are degenerated. This is a fixable issue. > > But looking at the bigger picture, a good question is what is the best > thing to do when cluster domains and llc domains span the same CPUs? > > Possibly it would be best to restrict clusters (which are almost an > arbitrary concept) to always span a subset of CPUs of the llc domain, > if llc siblings can be obtained? If those clusters are not properly set > up in DT to respect this condition, cluster_siblings would need to be > cleared (or set to the current CPU) so the CLS domain is not created at > all. > > Additionally, should we use cluster information from DT (cluster_id) to > create an MC level if we don't have llc information, even if > CONFIG_SCHED_CLUSTER=n? > > I currently don't have a very clear picture of how cluster domains and > llc domains would "live" together in a variety of topologies. I'll try > other DT topologies to see if there are others that can lead to trouble. This would be an issue. Depending on CONFIG_SCHED_CLUSTER we would get two different systems from the viewpoint of the scheduler. To me `cluster_id/_sibling` don't describe a certain level of CPU grouping (e.g. one level above core or one level below package). They were introduced to describe one level below LLC (e.g. Kunpeng920 L3 (24 CPUs LLC) -> L3 tag (4 CPUs) or x86 Jacobsville L3 -> L2), (Commit ^^^^^^ ^^ c5e22feffdd7 ("topology: Represent clusters of CPUs within a die")). The Ampere Altra issue already gave us a taste of the possible issues of this definition, commit db1e59483dfd ("topology: make core_mask include at least cluster_siblings"). If we link `cluster_id/_sibling` against (1. level) cpu-map cluster nodes plus using llc and `cluster_sibling >= llc_sibling` we will run into these issues. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel