Re: Hwloc with Xen host topology

[Andrew Cooper <andrew.cooper3@citrix.com>]

On 02/01/14 21:55, Samuel Thibault wrote:
> Andrew Cooper, le Thu 02 Jan 2014 21:50:06 +0000, a écrit :
>> On 02/01/14 21:24, Samuel Thibault wrote:
>>> Andrew Cooper, le Thu 02 Jan 2014 20:26:49 +0000, a écrit :
>>>> Cores are numbered per-socket in Xen, while sockets,
>>>> numa nodes and cpus are numbered on an absolute scale.  There is
>>>> currently a gross hack in my hwloc code which adds (socket_id *
>>>> cores_per_socket * threads_per_core) onto each core id to make them
>>>> similarly numbered on an absolute scale.  This is fine for a homogeneous
>>>> system, but not for a hetrogeneous system.
>>> BTW, hwloc does not need these physical ids to be unique, it can cope
>>> with duplication and whatnot.  That said, having a coherent interface at
>>> the Xen layer would be a good thing, indeed :)
>> If I take out the described hack, I am presented with
>>
>> ****************************************************************************
>> * hwloc has encountered what looks like an error from the operating system.
>> *
>> * object (Core P#0 cpuset 0x30000003) intersection without inclusion!
>> * Error occurred in topology.c line 853
>> *
>> * Please report this error message to the hwloc user's mailing list,
>> * along with the output from the hwloc-gather-topology.sh script.
>> ****************************************************************************
>>
>> Which I took to mean "I have done something stupid".  I looked and saw
>> that I was attempting to insert a second Core P#0 object with a
>> different cpuset and decided to renumber the cores so they didn't
>> overlap in physical ids.
>>
>> If you believe that this should indeed work, then I guess I need to
>> raise a bug...
> Well, logical processor physical ids, i.e. what is used for indexing
> physical cpusets, have to be unique. The core/socket/node IDs don't have
> to.
>
> Samuel

Then a bug needs raising.  My hack only changes the Core physical ID as
far as hwloc is concerned.  The PU physical IDs are unchanged by the
hack, and already unique as presented by Xen.

~Andrew