From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751935AbdFHUUr (ORCPT ); Thu, 8 Jun 2017 16:20:47 -0400 Received: from merlin.infradead.org ([205.233.59.134]:52036 "EHLO merlin.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751836AbdFHUUq (ORCPT ); Thu, 8 Jun 2017 16:20:46 -0400 Date: Thu, 8 Jun 2017 22:20:41 +0200 From: Peter Zijlstra To: "Luck, Tony" Cc: Dave Hansen , "linux-kernel@vger.kernel.org" , "tim.c.chen@linux.intel.com" , "bp@alien8.de" , "rientjes@google.com" , "imammedo@redhat.com" , "torvalds@linux-foundation.org" , "prarit@redhat.com" , "toshi.kani@hp.com" , "brice.goglin@gmail.com" , "hpa@linux.intel.com" , "mingo@kernel.org" Subject: Re: [PATCH] x86, sched: allow topolgies where NUMA nodes share an LLC Message-ID: <20170608202041.GE8337@worktop.programming.kicks-ass.net> References: <20170608193928.A3B5497F@viggo.jf.intel.com> <20170608200041.GB8337@worktop.programming.kicks-ass.net> <3908561D78D1C84285E8C5FCA982C28F612CC19A@ORSMSX114.amr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3908561D78D1C84285E8C5FCA982C28F612CC19A@ORSMSX114.amr.corp.intel.com> User-Agent: Mutt/1.5.22.1 (2013-10-16) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Jun 08, 2017 at 08:08:31PM +0000, Luck, Tony wrote: > > What does? That does sound broken. How can a cache domain sanely span > > memory controllers? > > Think "cluster on die" with cores on the socket split into two clusters, but still sharing LLC. The thing is, cluster-on-die works with the current code, and therefore seems to modify the SRAT an CPUID information in a consistent manner. Which in turn seems to suggest the LLC really is split for cluster-on-die. This is something new, and the Changelog is absolute crap for not explaining _anything_. So while SRAT seems to invent new nodes, the CPUID topology bits still describes the full LLC, now shared across nodes. Is this accurate?, do these nodes, as described by SRAT, actually have a memory controller each? And is the LLC still fully integrated across the nodes? If so, we need to go fix the scheduler domain topology to put a cache domain across nodes (which is going to be painful). Just making the warning go away and not explaining things sucks.