Re: [PATCH 15/15] x86: Fix cpu_coregroup_mask to return correct cpumask on multi-node processors

[Andreas Herrmann <andreas.herrmann3@amd.com>]

On Fri, Aug 28, 2009 at 12:39:44PM +0200, Peter Zijlstra wrote:
> On Thu, 2009-08-27 at 17:25 +0200, Andreas Herrmann wrote:
> > On Tue, Aug 25, 2009 at 11:55:43AM +0200, Peter Zijlstra wrote:
> > > On Tue, 2009-08-25 at 11:31 +0200, Andreas Herrmann wrote:
> > > > On Mon, Aug 24, 2009 at 05:36:16PM +0200, Peter Zijlstra wrote:
> > > > > On Thu, 2009-08-20 at 15:46 +0200, Andreas Herrmann wrote:
> > > > > > The correct mask that describes core-siblings of an processor
> > > > > > is topology_core_cpumask. See topology adapation patches, especially
> > > > > > http://marc.info/?l=linux-kernel&m=124964999608179
> > > > > 
> > > > > 
> > > > > argh, violence, murder kill.. this is the worst possible hack and you're
> > > > > extending it :/
> > > > 
> > > > So this is the third code area
> > > > (besides sched_*_power_savings sysfs interface, and the __cpu_power fiddling)
> > > > that is crap, mess, a hack.
> > > > 
> > > > Didn't know that I'd enter such a minefield when touching this code. ;-(
> > > 
> > > Yeah, you're lucky that way ;-) Its been creaking for a while, and I've
> > > been making noises to the IBM people (who so far have been the main
> > > source of power saving patches) to clean this up, but now you trod onto
> > > all of it at once..
> > > 
> > > > What would be your perferred solution for the
> > > > core_cpumask/llc_shared_map stuff?  Another domain level to get rid of
> > > > this function?
> > > 
> > > Right, I'd like to see everything exposed as domain levels.
> > > 
> > > 
> > > numa-cluster
> > > numa
> > > socket
> > > in-socket-numa
> > > multi-core
> > > shared-cache
> > > core
> > > threads
> > 
> > Out of curiosity, when does cpu_core_mask differ from llc_shared_map
> > on Intel? Only in case of MCM (e.g. Core2 Quad)?
> 
> Yes, I think both c2q and some dual-core opteron have multiple cache
> domains per socket.
> 
> > If yes, the hackery of cpu_coregroup_mask() could be replace by
> > the domain that I'd like to introduce for Magny-Cours:
> > 
> >   MC domain span would represent one die.
> >   The new domain would span all dies in an MCM.
> > 
> > Bad idea?
> 
> No, I think all the mentioned chips have the multi-die thing in common,
> the intel c2q has 2 dual-core dies,

> the opteron I have seems to be two
> single cores

Really? I am not aware of such a thing.

Can you check how many sets of northbridge functions do you have?  If
you would have two dies in one package then you should see one set of
PCI functions at bus 0 device 24 and a second set of PCI functions at
bus 0 device 25, e.g.

  # lspci -d 1022:
  ...
  00:18.0 Host bridge [0600]: Advanced Micro Devices [AMD] Family 10h
          [Opteron, Athlon64, Sempron] HyperTransport Configuration [1022:1200]
  ...
  00:19.0 Host bridge [0600]: Advanced Micro Devices [AMD] Family 10h
          [Opteron, Athlon64, Sempron] HyperTransport Configuration [1022:1200]
  ... 

> and this magny thing has 2 many cores -- teh pun, sides
> aching :-)
> 
> So the generalization to dies per socket seems sensible.

Yup


Andreas

-- 
Operating | Advanced Micro Devices GmbH
  System  | Karl-Hammerschmidt-Str. 34, 85609 Dornach b. München, Germany
 Research | Geschäftsführer: Andrew Bowd, Thomas M. McCoy, Giuliano Meroni
  Center  | Sitz: Dornach, Gemeinde Aschheim, Landkreis München
  (OSRC)  | Registergericht München, HRB Nr. 43632