Re: [patch V4 24/41] x86/cpu: Provide cpu_init/parse_topology()

K Prateek Nayak
2023-09-14T09:21:06Z

Hello Thomas,

On 8/28/2023 7:58 PM, K Prateek Nayak wrote:
> Hello Thomas,
>
> On 8/28/2023 3:35 PM, Thomas Gleixner wrote:
>> Prateek!
>>
>> On Mon, Aug 28 2023 at 11:37, K. Prateek Nayak wrote:
>>> On 8/14/2023 2:24 PM, Thomas Gleixner wrote:
>>>
>>> Since these enums come from the description of level type of CPUID leaf
>>> 0x1f, can we have a short description clarifying what each signify. This
>>> will also help clarify the mappings for AMD's extended CPUID leaf
>>> 0x80000026 (specifically for CCX and CCD level types). I had following
>>> in my mind:
>>
>> Makes sense.
>>
>>> TOPO_MODULE_DOMAIN,
>>> + /*
>>> + * If exists, represents a group of tiles within
>>> + * an instance of the next domain
>>> + *
>>> + * On Intel: This level contains a group of Tile
>>> + * type as described by CPUID leaf 0x1f
>>> + *
>>> + * On AMD: This is the group of "Complex" type
>>> + * instances as described by CPUID leaf
>>> + * 0x8000_0026
>>> + */
>>> TOPO_TILE_DOMAIN,
>>> + /*
>>> + * If exists, represents a group of dies within an
>>> + * instance of the next domain
>>> + *
>>> + * On Intel: This level contains group of Die
>>> + * type as described by CPUID leaf 0x1f
>>> + *
>>> + * On AMD: This is the group of "CCD (Die)"
>>> + * type instances as described by CPUID leaf
>>> + * 0x8000_0026
>>> + */
>>> TOPO_DIE_DOMAIN,
>>> + /*
>>> + * If exists, represents a group of packages
>>> + * within the root domain
>>> + */
>>> TOPO_PKG_DOMAIN,
>>> + /* Topmost domain with a singular instance */
>>> TOPO_ROOT_DOMAIN,
>>> TOPO_MAX_DOMAIN,
>>> };
>>
>> Now this begs the obvious question what the actual meaning of these
>> domains is and what's their relevance for the kernel.
>>
>> It's probably undisputed what SMT/CORE mean and what their relevance is.
>> The PKG/DIE domains are pretty clear too.
>
> Yup! Those seem to be clear.
>
>>
>> Now we have:
>>
>> MODULE (Intel only)
>>
>> TILE Intel, AMD names it "Complex"
>
> So here is my interpretation of 0x1f since I could not find much in the
> SDM for these level types.

Turns out, there is indeed more information about these types in SDM.
Intel SDM Vol. 3A, in Sec. 9.9.1 "Hierarchical Mapping of Shared
Resources" contains description of these level. I'll quote it here for
ease of readers:

"""
The decomposition of an APIC_ID may consist of several sub fields
representing the topology within a physical processor package, the
higher-order bits of an APIC ID may also be used by cluster vendors to
represent the topology of cluster nodes of each coherent multiprocessor
systems:

o Cluster : Some multi-threading environments consists of multiple
clusters of multi-processor systems. The CLUSTER_ID
sub-field is usually supported by vendor firmware to
distinguish different clusters. For nonclustered systems,
CLUSTER_ID is usually 0 and system topology is reduced.

o Package : A physical processor package mates with a socket. A package
may contain one or more software visible die. The PACKAGE_ID
sub-field distinguishes different physical packages within a
cluster.

o Die : A software-visible chip inside a package. The DIE_ID
sub-field distinguishes different die within a package. If
there are no software visible die, the width of this bit
field is 0.

o DieGrp : A group of die that share certain resources.

o Tile : A set of cores that share certain resources. The TILE_ID
sub-field distinguishes different tiles. If there are no
software visible tiles, the width of this bit field is 0.

o Module : A set of cores that share certain resources. The MODULE_ID
sub-field distinguishes different modules. If there are no
software visible modules, the width of this bit field is 0.

o Core : Processor cores may be contained within modules, within
tiles, on software-visible die, or appear directly at the
package domain. The CORE_ID sub-field distinguishes
processor cores. For a single-core processor, the width of
this bit field is 0.

o Logical Processor : A processor core provides one or more logical
processors sharing execution resources. The
LOGICAL_PROCESSOR_ID sub-field distinguishes logical
processors in a core. The width of this bit field is
non-zero if a processor core provides more than one logical
processors.
"""

So some questions to Intel folks to determine whether mapping AMD's
Complex to Tile makes sense or not:

- What are the "certain resources" a group of module / tile share?

- Module and Tile both use the phrase "set of cores" in their
description. Does this mean their existence is mutually exclusive?

AMD's Complex (CCX) marks the L3 cache boundary. If either of the
"certain resources" refer to L3 cache, then Complex can be safely mapped
to the respective level without any fear of misinterpretation of the
characteristics.

Also, I do not see a "DieGrp" domain in the "x86_topology_domains". Is
this because of the lack of "software visible" aspect of it despite its
possible existence?

> [..snip..]
--
Thanks and Regards,
Prateek