From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7D54034A76A for ; Wed, 4 Feb 2026 21:42:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.18 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770241370; cv=none; b=n1pqE5djG+UDsyApxrHInk/ho7f4NkYCUg167d/pA/yIkFFZvswN3GW1y+znMVcLRqqkBtcyeDOWa4aP8QKbs6kjrTCf4n25qHV+DoLVshqheFBCHGo47WSus2bOXkRWBN8Wh3AgajGu5Q/tXPHxMMVU6edBRArxoUMpezCAOPY= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770241370; c=relaxed/simple; bh=9tMu9vFGEGDbGaP0paqFttycZSqt2yIUATDmFnEhgQI=; h=Message-ID:Subject:From:To:Cc:Date:In-Reply-To:References: Content-Type:MIME-Version; b=VjorLaM1aUfAMl9HBQLJWRIS7ksf/xKXDutRucTp/9H/1nE2cvRLBKuVWEwqYfI7/+zLirUAfn6Cc8JLob0IRTHm+mqVBWlZFJWh5u8GP6hN55xYetJjrKfSinZizoNe0mPuYlnGIGKtHHXGRDpKCGP2eKcnNpZjbBYMtDDK6Rk= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=pass smtp.mailfrom=linux.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=LtqEFgQp; arc=none smtp.client-ip=192.198.163.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="LtqEFgQp" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1770241370; x=1801777370; h=message-id:subject:from:to:cc:date:in-reply-to: references:content-transfer-encoding:mime-version; bh=9tMu9vFGEGDbGaP0paqFttycZSqt2yIUATDmFnEhgQI=; b=LtqEFgQpsdZ1XVuxnNgOYTBHZpR3YgPgSK7fbKvNu8ApDNbfIVPnxYHQ OIf9SGRzceOEK+8edVLIZp+iDDbdif1z001kp2vnYSJWmaOl6HP2UX48e x40J5rAe0LtAGb852y35tVIPLq4Hb0/3QsqdDHW3MvUwKjzMoW6sh1E5d Dwx7LGIfTAh7/Jgbw8d+OKNM2H99fvTqW/mOVWbNPm2zIQjHNWR3AtwRF 7WqLh0/ira24mtwNazzGoUfaxa7AtPD1mVDe+j/d8gjhfDnR8/bylGKn8 wAlqBargTHqXvmyibZ+R8q8f4q+DcpjattcGaui8wGQNWUCy/BYbr/nDZ g==; X-CSE-ConnectionGUID: vHgqBC99QWuvZ3gFvbYjqA== X-CSE-MsgGUID: mZ2r2rpBT0+B6lU0p7FFxA== X-IronPort-AV: E=McAfee;i="6800,10657,11691"; a="70634113" X-IronPort-AV: E=Sophos;i="6.21,273,1763452800"; d="scan'208";a="70634113" Received: from fmviesa005.fm.intel.com ([10.60.135.145]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2026 13:42:49 -0800 X-CSE-ConnectionGUID: mf/3vfn1Rpeq0JrvFsWxAg== X-CSE-MsgGUID: FVar8lD1SBCw2Ot5Hq75RA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.21,273,1763452800"; d="scan'208";a="214778225" Received: from unknown (HELO [10.241.243.83]) ([10.241.243.83]) by fmviesa005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2026 13:42:49 -0800 Message-ID: Subject: Re: [PATCH] sched/topology: Check average distances to remote packages From: Tim Chen To: Kyle Meyer , bp@alien8.de, dave.hansen@linux.intel.com, mingo@redhat.com, peterz@infradead.org, tglx@kernel.org, vinicius.gomes@intel.com Cc: brgerst@gmail.com, hpa@zytor.com, kprateek.nayak@amd.com, linux-kernel@vger.kernel.org, patryk.wlazlyn@linux.intel.com, rafael.j.wysocki@intel.com, russ.anderson@hpe.com, x86@kernel.org, yu.c.chen@intel.com, zhao1.liu@intel.com Date: Wed, 04 Feb 2026 13:42:49 -0800 In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.58.1 (3.58.1-1.fc43) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 On Tue, 2026-01-27 at 11:02 -0600, Kyle Meyer wrote: > Granite Rapids (GNR) and Clearwater Forest (CWF) average distances to > remote packages to fix scheduler domains, see [1] for more information. >=20 > A warning and backtrace are printed when sub-NUMA clustering (SNC) is > enabled and there are more than 2 packages because the average distances > to remote packages could be different, skewing the single average remote > distance. >=20 > This is unnecessary when the average distances to remote packages are > the same. >=20 > Support single average remote distance on systems with more than 2 > packages, preventing unnecessary warnings and backtraces by checking if > average distances to remote packages are the same. Yes, I agree that if there are more than 2 packages but the packages are th= e same average distance with each other (symmetric package topology), that should be okay and no warnings is necessary. Reviewed-by: Tim Chen >=20 > [1] commit 4d6dd05d07d0 ("sched/topology: Fix sched domain build error fo= r GNR, CWF in SNC-3 mode"). >=20 > Signed-off-by: Kyle Meyer > --- >=20 > The warning and backtrace were noticed on a 16 socket GNR system with SNC= -2 enabled. >=20 > --- > arch/x86/kernel/smpboot.c | 70 ++++++++++++++++++++++++++++----------- > 1 file changed, 51 insertions(+), 19 deletions(-) >=20 > diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c > index 5cd6950ab672..4467716f4054 100644 > --- a/arch/x86/kernel/smpboot.c > +++ b/arch/x86/kernel/smpboot.c > @@ -518,27 +518,70 @@ static int avg_remote_numa_distance(void) > { > int i, j; > int distance, nr_remote, total_distance; > + int max_pkgs =3D topology_max_packages(); > + int cpu, pkg, pkg_avg_distance; > + int *pkg_total_distance; > + int *pkg_nr_remote; > =20 > if (sched_avg_remote_distance > 0) > return sched_avg_remote_distance; > =20 > + sched_avg_remote_distance =3D REMOTE_DISTANCE; > + > nr_remote =3D 0; > total_distance =3D 0; > + > + pkg_total_distance =3D kcalloc(max_pkgs, sizeof(int), GFP_KERNEL); > + if (!pkg_total_distance) > + goto cleanup; > + > + pkg_nr_remote =3D kcalloc(max_pkgs, sizeof(int), GFP_KERNEL); > + if (!pkg_nr_remote) > + goto cleanup; > + > for_each_node_state(i, N_CPU) { > for_each_node_state(j, N_CPU) { > distance =3D node_distance(i, j); > =20 > - if (distance >=3D REMOTE_DISTANCE) { > - nr_remote++; > - total_distance +=3D distance; > - } > + if (distance < REMOTE_DISTANCE) > + continue; > + > + nr_remote++; > + total_distance +=3D distance; > + > + cpu =3D cpumask_first(cpumask_of_node(j)); > + if (cpu >=3D nr_cpu_ids) > + continue; > + > + pkg =3D topology_physical_package_id(cpu); > + pkg_total_distance[pkg] +=3D distance; > + pkg_nr_remote[pkg]++; > } > } > - if (nr_remote) > - sched_avg_remote_distance =3D total_distance / nr_remote; > - else > - sched_avg_remote_distance =3D REMOTE_DISTANCE; > =20 > + if (!nr_remote) > + goto cleanup; > + > + sched_avg_remote_distance =3D total_distance / nr_remote; > + > + /* > + * Single average remote distance won't be appropriate if different > + * packages have different distances to remote packages. > + */ > + for (i =3D 0; i < max_pkgs; i++) { > + if (!pkg_nr_remote[i]) > + continue; > + > + pkg_avg_distance =3D pkg_total_distance[i] / pkg_nr_remote[i]; > + > + pr_debug("sched: Avg. distance to remote package %d: %d\n", i, pkg_avg= _distance); > + > + if (pkg_avg_distance !=3D sched_avg_remote_distance) > + WARN_ONCE(1, "sched: Avg. distances to remote packages are different\= n"); > + } > +cleanup: > + kfree(pkg_nr_remote); > + kfree(pkg_total_distance); > return sched_avg_remote_distance; > } > =20 > @@ -564,18 +607,7 @@ int arch_sched_node_distance(int from, int to) > * in the remote package in the same sched group. > * Simplify NUMA domains and avoid extra NUMA levels including > * different remote NUMA nodes and local nodes. > - * > - * GNR and CWF don't expect systems with more than 2 packages > - * and more than 2 hops between packages. Single average remote > - * distance won't be appropriate if there are more than 2 > - * packages as average distance to different remote packages > - * could be different. > */ > - WARN_ONCE(topology_max_packages() > 2, > - "sched: Expect only up to 2 packages for GNR or CWF, " > - "but saw %d packages when building sched domains.", > - topology_max_packages()); > - > d =3D avg_remote_numa_distance(); > } > return d;