From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2BC65C64E90 for ; Tue, 1 Dec 2020 16:05:30 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A925F2223F for ; Tue, 1 Dec 2020 16:05:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="vbHvaz+5" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A925F2223F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:Message-ID:Date:In-reply-to:Subject:To: From:References:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=SHzbiKueyxI8TFpDrmXP1br8DbwVU1Gr3qJQCwK1cuI=; b=vbHvaz+5KqyphljMQ5uM+djNS JsEI2l8SmzkffXWWfQ5MtqVto6d5LgvmexfylQaxjp0zi1MWZxHqbtvV2numCpKSjYIkerZUcXT8H +FAG4/D6TW2yax6u3hIjksoY+zgv1T6iy8kNfV8yNR7RG1MAaEyC6jwWRI+4S2ONUvFOVGLb8gp3T 4+s0dqEbee4j6ELSlOPF/19kFO0hZLzQ78OqwGYEuUbJ+vvIIpuc6taOfql1sffyqFPlfymLtNEik l35EyB3THaSa01cq6zmXgJ8+VX14vNThz9J9VsEoy8hXDXeH+8D6VO7vru3Pc+m1JWH/deJljoVIu v5UgY+Gfg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kk88Y-0007Q5-Tb; Tue, 01 Dec 2020 16:04:18 +0000 Received: from foss.arm.com ([217.140.110.172]) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kk88V-0007Pf-OD for linux-arm-kernel@lists.infradead.org; Tue, 01 Dec 2020 16:04:16 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1C0311042; Tue, 1 Dec 2020 08:04:09 -0800 (PST) Received: from e113632-lin (e113632-lin.cambridge.arm.com [10.1.194.46]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 713343F575; Tue, 1 Dec 2020 08:04:06 -0800 (PST) References: <20201201025944.18260-1-song.bao.hua@hisilicon.com> <20201201025944.18260-3-song.bao.hua@hisilicon.com> User-agent: mu4e 0.9.17; emacs 26.3 From: Valentin Schneider To: Barry Song Subject: Re: [RFC PATCH v2 2/2] scheduler: add scheduler level for clusters In-reply-to: <20201201025944.18260-3-song.bao.hua@hisilicon.com> Date: Tue, 01 Dec 2020 16:04:04 +0000 Message-ID: MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201201_110415_909620_88A1C8DF X-CRM114-Status: GOOD ( 18.60 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: juri.lelli@redhat.com, mark.rutland@arm.com, vincent.guittot@linaro.org, prime.zeng@hisilicon.com, peterz@infradead.org, catalin.marinas@arm.com, Jonathan.Cameron@huawei.com, rjw@rjwysocki.net, linux-kernel@vger.kernel.org, rostedt@goodmis.org, linuxarm@huawei.com, bsegall@google.com, linux-acpi@vger.kernel.org, mingo@redhat.com, mgorman@suse.de, xuwei5@huawei.com, gregkh@linuxfoundation.org, will@kernel.org, dietmar.eggemann@arm.com, linux-arm-kernel@lists.infradead.org, lenb@kernel.org Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 01/12/20 02:59, Barry Song wrote: > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c > index 1a68a05..ae8ec910 100644 > --- a/kernel/sched/fair.c > +++ b/kernel/sched/fair.c > @@ -6106,6 +6106,37 @@ static inline int select_idle_smt(struct task_struct *p, int target) > > #endif /* CONFIG_SCHED_SMT */ > > +#ifdef CONFIG_SCHED_CLUSTER > +/* > + * Scan the local CLUSTER mask for idle CPUs. > + */ > +static int select_idle_cluster(struct task_struct *p, int target) > +{ > + int cpu; > + > + /* right now, no hardware with both cluster and smt to run */ > + if (sched_smt_active()) > + return -1; > + > + for_each_cpu_wrap(cpu, cpu_cluster_mask(target), target) { Gating this behind this new config only leveraged by arm64 doesn't make it very generic. Note that powerpc also has this newish "CACHE" level which seems to overlap in function with your "CLUSTER" one (both are arch specific, though). I think what you are after here is an SD_SHARE_PKG_RESOURCES domain walk, i.e. scan CPUs by increasing cache "distance". We already have it in some form, as we scan SMT & LLC domains; AFAICT LLC always maps to MC, except for said powerpc's CACHE thingie. *If* we are to generally support more levels with SD_SHARE_PKG_RESOURCES, we could say frob something into select_idle_cpu(). I'm thinking of something like the incomplete, untested below: --- diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index ae7ceba8fd4f..70692888db00 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -6120,7 +6120,7 @@ static inline int select_idle_smt(struct task_struct *p, struct sched_domain *sd static int select_idle_cpu(struct task_struct *p, struct sched_domain *sd, int target) { struct cpumask *cpus = this_cpu_cpumask_var_ptr(select_idle_mask); - struct sched_domain *this_sd; + struct sched_domain *this_sd, *child = NULL; u64 avg_cost, avg_idle; u64 time; int this = smp_processor_id(); @@ -6150,14 +6150,22 @@ static int select_idle_cpu(struct task_struct *p, struct sched_domain *sd, int t time = cpu_clock(this); - cpumask_and(cpus, sched_domain_span(sd), p->cpus_ptr); + do { + /* XXX: sd should start as SMT's parent */ + cpumask_and(cpus, sched_domain_span(sd), p->cpus_ptr); + if (child) + cpumask_andnot(cpus, cpus, sched_domain_span(child)); + + for_each_cpu_wrap(cpu, cpus, target) { + if (!--nr) + return -1; + if (available_idle_cpu(cpu) || sched_idle_cpu(cpu)) + break; + } - for_each_cpu_wrap(cpu, cpus, target) { - if (!--nr) - return -1; - if (available_idle_cpu(cpu) || sched_idle_cpu(cpu)) - break; - } + child = sd; + sd = sd->parent; + } while (sd && sd->flags & SD_SHARE_PKG_RESOURCES); time = cpu_clock(this) - time; update_avg(&this_sd->avg_scan_cost, time); _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel