linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v5 0/4]  x86/cpu/topology: Fix the preferred order of initial APIC ID parsing on AMD/Hygon
@ 2025-09-01 17:04 K Prateek Nayak
  2025-09-01 17:04 ` [PATCH v5 1/4] x86/cpu/topology: Always try cpu_parse_topology_ext() " K Prateek Nayak
                   ` (3 more replies)
  0 siblings, 4 replies; 5+ messages in thread
From: K Prateek Nayak @ 2025-09-01 17:04 UTC (permalink / raw)
  To: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
	Sean Christopherson, Paolo Bonzini, Jonathan Corbet, x86
  Cc: Naveen rao, Sairaj Kodilkar, H. Peter Anvin,
	Peter Zijlstra (Intel), Xin Li (Intel), Pawan Gupta, linux-kernel,
	kvm, Mario Limonciello, Gautham R. Shenoy, Babu Moger,
	Suravee Suthikulpanit, K Prateek Nayak

When running an AMD guest on QEMU with > 255 cores, the following FW_BUG
was noticed with recent kernels when topoext feature wasn't explicitly
enabled:

    [Firmware Bug]: CPU 512: APIC ID mismatch. CPUID: 0x0000 APIC: 0x0200

QEMU provides the extended topology leaf 0xb for these guests but in an
effort to keep all the topology parsing bits together during the
enablement of the 0xb leaf for AMD, a pseudo dependency on
X86_FETURE_TOPOEXT was created which prevents these guests from parsing
the topology from the 0xb leaf.

The support for CPUID leaf 0xb is independent of the TOPOEXT feature and
is rather linked to the x2APIC enablement. The support for the extended
topology leaves is expected to be confirmed by ensuring:

1. "leaf <= {extended_}cpuid_level" and then
2. Parsing the level 0 of the respective leaf to confirm EBX[15:0]
   (LogProcAtThisLevel) is non-zero

as stated in the definition of "CPUID_Fn0000000B_EAX_x00 [Extended
Topology Enumeration] (Core::X86::Cpuid::ExtTopEnumEax0)" in Processor
Programming Reference (PPR) for AMD Family 19h Model 01h Rev B1 Vol1 [1]
Sec. 2.1.15.1 "CPUID Instruction Functions".

On baremetal, this has not been a problem since TOPOEXT support (Fam
0x15 and above) predates the support for CPUID leaf 0xb (Fam 0x17[Zen2]
and above) however, in virtualized environment, the support for x2APIC
can be enabled independent of topoext where QEMU expects the guest to
parse the topology and the APICID from CPUID leaf 0xb.

Boris asked why QEMU doesn't force enable TOPOEXT feature with x2APIC
[2] and Naveen discovered there were historic reasons to not enable
TOPOEXT by default when using "-cpu host" on AMD systems [3].

The same behavior continues unless an EPYC cpu model is explicitly
passed to QEMU. More details are enclosed in the commit logs.

Ideally, these changes should not affect baremetal AMD/Hygon platforms
as they have supported TOPOEXT long before the support for CPUID leaf
0xb and the extended CPUID leaf 0x80000026 (famous last words).

Patch 2 and 3 are yak shaving to explicitly define a raw MSR value used
in the topology parsing bits and simplify the flow around "has_topoext"
when the same can be discovered using X86_FEATURE_XTOPOLOGY.

Patch 4 is the documentation patch that outlines the preferred parsing
order of CPUID leaves during topology enumeration on x86 platforms.

Previous version of this series has been tested on baremetal Zen1
(contains topoext but not 0xb leaf), Zen3 (contains both topoext and 0xb
leaf), and Zen4 (contains topoext, 0xb leaf, and 0x80000026 leaf)
servers with no changes observed in "/sys/kernel/debug/x86/topo/"
directory.

The series was also tested on 255 and 512 vCPU (each vCPU is an
individual core from QEMU topology being passed) EPYC-Genoa guest with
and without x2apic and topoext enabled and this series solves the FW_BUG
seen on guest with > 255 VCPUs. No changes observed in
"/sys/kernel/debug/x86/topo/" for all other cases without warning.
0xb leaf is provided unconditionally on these guests (with or without
topoext, even with x2apic disabled on guests with <= 255 vCPU).

In all the cases initial_apicid matched the apicid in
"/sys/kernel/debug/x86/topo/" after applying this series.

Relevant bits of QEMU cmdline used during testing are as follows:

    qemu-system-x86_64 \
    -enable-kvm -m 32G -smp cpus=512,cores=512 \
    -cpu EPYC-Genoa,x2apic=on,kvm-msi-ext-dest-id=on,+kvm-pv-unhalt,kvm-pv-tlb-flush,kvm-pv-ipi,kvm-pv-sched-yield,[-topoext]  \
    -machine q35,kernel_irqchip=split \
    -global kvm-pit.lost_tick_policy=discard
    ...

References:

[1] https://bugzilla.kernel.org/show_bug.cgi?id=206537
[2] https://lore.kernel.org/lkml/20250819113447.GJaKRhVx6lBPUc6NMz@fat_crate.local/
[3] https://lore.kernel.org/qemu-devel/20180809221852.15285-1-ehabkost@redhat.com/

Series is based on tip:master at commit 4f0d2af9e565 ("Merge branch into
tip/master: 'x86/tdx'")

---
Changelog v4..v5:

o Dropped the patch that was merged.

o Addressed review comments by Boris on Patch 1.

o Included the documentation patch formally.

v4: https://lore.kernel.org/lkml/20250825075732.10694-1-kprateek.nayak@amd.com/

Changelog v3..v4:

o Renamed the series title to better capture the purpose. Based on the
  readout of the APM and PPR, this problem was only exposed by QEMU
  and QEMU is not doing anything wrong considering the spec.

o Fixed references to X86_FEATURE_XTOPOLOGY (XTOPOLOGY) which was
  mistakenly referred to as XTOPOEXT. (Boris)

o Reordered the patches to have the fixes before cleanups. (Thomas)

o Refreshed the diff of Patch 1 with the one Thomas suggested in
  https://lore.kernel.org/lkml/87ms7o3kn6.ffs@tglx/. (Thomas)

o Quoted the relevant sections of the APM and the PPR to support the
  changes. (Mentioned on v3 by Naveen and Boris)

Note: The debate on "CoreId" from CPUID 0x8000001e EBX has not been
addressed yet. I'll check internally and follow up on the QEMU bits once
H/W folks confirm what their strategy is with the 8-bit field in future
processors.

The updates in this series ensures the usage of the topology information
from the XTOPOLOGY leaves (0x80000026 / 0xb)  when they are present and
systems that support more than 256 CPUs need x2APIC enabled to address
all the CPUs present thus removing the dependency on CPUID leaf
0x8000001e for Core ID.

v3: https://lore.kernel.org/lkml/20250818060435.2452-1-kprateek.nayak@amd.com/

Changelog v2..v3:

o Patch 1 was added to the series.
o Use cpu_feature_enabled() in Patch 3.
o Rebased on top of tip:x86/cpu.

v2: https://lore.kernel.org/lkml/20250725110622.59743-1-kprateek.nayak@amd.com/

Changelog v1..v2:

o Collected tags from Naveen. (Thank you for testing!)
o Rebased the series on tip:x86/cpu.
o Swapped Patch 1 and Patch 2 from v1.
o Merged the body of two if blocks in Patch 1 to allow for cleanup in
  Patch 3.

v1: https://lore.kernel.org/lkml/20250612072921.15107-1-kprateek.nayak@amd.com/
---
K Prateek Nayak (4):
  x86/cpu/topology: Always try cpu_parse_topology_ext() on AMD/Hygon
  x86/cpu/topology: Check for X86_FEATURE_XTOPOLOGY instead of passing
    has_xtopology
  x86/msr-index: Define AMD64_CPUID_FN_EXT MSR
  Documentation/x86/topology: Detail CPUID leaves used for topology
    enumeration

 Documentation/arch/x86/topology.rst | 198 ++++++++++++++++++++++++++++
 arch/x86/include/asm/msr-index.h    |   5 +
 arch/x86/kernel/cpu/topology_amd.c  |  39 +++---
 3 files changed, 223 insertions(+), 19 deletions(-)


base-commit: 4f0d2af9e56558e125b321b176b25cd6ad5fdac7
-- 
2.34.1


^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH v5 1/4] x86/cpu/topology: Always try cpu_parse_topology_ext() on AMD/Hygon
  2025-09-01 17:04 [PATCH v5 0/4] x86/cpu/topology: Fix the preferred order of initial APIC ID parsing on AMD/Hygon K Prateek Nayak
@ 2025-09-01 17:04 ` K Prateek Nayak
  2025-09-01 17:04 ` [PATCH v5 2/4] x86/cpu/topology: Check for X86_FEATURE_XTOPOLOGY instead of passing has_xtopology K Prateek Nayak
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 5+ messages in thread
From: K Prateek Nayak @ 2025-09-01 17:04 UTC (permalink / raw)
  To: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
	Sean Christopherson, Paolo Bonzini, Jonathan Corbet, x86
  Cc: Naveen rao, Sairaj Kodilkar, H. Peter Anvin,
	Peter Zijlstra (Intel), Xin Li (Intel), Pawan Gupta, linux-kernel,
	kvm, Mario Limonciello, Gautham R. Shenoy, Babu Moger,
	Suravee Suthikulpanit, K Prateek Nayak, stable, Naveen N Rao

Support for parsing the topology on AMD/Hygon processors using CPUID
leaf 0xb was added in commit 3986a0a805e6 ("x86/CPU/AMD: Derive CPU
topology from CPUID function 0xB when available"). In an effort to keep
all the topology parsing bits in one place, this commit also introduced
a pseudo dependency on the TOPOEXT feature to parse the CPUID leaf 0xb.

TOPOEXT feature (CPUID 0x80000001 ECX[22]) advertises the support for
Cache Properties leaf 0x8000001d and the CPUID leaf 0x8000001e EAX for
"Extended APIC ID" however support for 0xb was introduced alongside the
x2APIC support not only on AMD [1], but also historically on x86 [2].

Similar to 0xb, the support for extended CPU topology leaf 0x80000026
too does not depend on the TOPOEXT feature.

The support for these leaves is expected to be confirmed by ensuring
"leaf <= {extended_}cpuid_level" and then parsing the level 0 of the
respective leaf to confirm EBX[15:0] (LogProcAtThisLevel) is non-zero as
stated in the definition of "CPUID_Fn0000000B_EAX_x00 [Extended Topology
Enumeration] (Core::X86::Cpuid::ExtTopEnumEax0)" in Processor
Programming Reference (PPR) for AMD Family 19h Model 01h Rev B1 Vol1 [3]
Sec. 2.1.15.1 "CPUID Instruction Functions".

This has not been a problem on baremetal platforms since support for
TOPOEXT (Fam 0x15 and later) predates the support for CPUID leaf 0xb
(Fam 0x17[Zen2] and later), however, for AMD guests on QEMU, "x2apic"
feature can be enabled independent of the "topoext" feature where QEMU
expects topology and the initial APICID to be parsed using the CPUID
leaf 0xb (especially when number of cores > 255) which is populated
independent of the "topoext" feature flag.

Unconditionally call cpu_parse_topology_ext() on AMD and Hygon
processors to first parse the topology using the XTOPOLOGY leaves
(0x80000026 / 0xb) before using the TOPOEXT leaf (0x8000001e).

While at it, break down the single large comment in parse_topology_amd()
to better highlight the purpose of each CPUID leaf.

Cc: stable@vger.kernel.org # Only v6.9 and above; Depends on x86 topology rewrite
Link: https://lore.kernel.org/lkml/1529686927-7665-1-git-send-email-suravee.suthikulpanit@amd.com/ [1]
Link: https://lore.kernel.org/lkml/20080818181435.523309000@linux-os.sc.intel.com/ [2]
Link: https://bugzilla.kernel.org/show_bug.cgi?id=206537 [3]
Suggested-by: Naveen N Rao (AMD) <naveen@kernel.org>
Fixes: 3986a0a805e6 ("x86/CPU/AMD: Derive CPU topology from CPUID function 0xB when available")
Signed-off-by: K Prateek Nayak <kprateek.nayak@amd.com>
---
Changelog v4..v5:

o Made a note on only targeting versions >= v6.9 for stable backports
  since the fix depends on the x86 topology rewrite. (Boris)

o Renamed "has_topoext" to "has_xtopology". (Boris)

o Broke down the large comment in parse_topology_amd() to better
  highlight the purpose of each leaf and the overall parsing flow.
  (Boris)
---
 arch/x86/kernel/cpu/topology_amd.c | 25 ++++++++++++++-----------
 1 file changed, 14 insertions(+), 11 deletions(-)

diff --git a/arch/x86/kernel/cpu/topology_amd.c b/arch/x86/kernel/cpu/topology_amd.c
index 827dd0dbb6e9..c79ebbb639cb 100644
--- a/arch/x86/kernel/cpu/topology_amd.c
+++ b/arch/x86/kernel/cpu/topology_amd.c
@@ -175,27 +175,30 @@ static void topoext_fixup(struct topo_scan *tscan)
 
 static void parse_topology_amd(struct topo_scan *tscan)
 {
-	bool has_topoext = false;
-
 	/*
-	 * If the extended topology leaf 0x8000_001e is available
-	 * try to get SMT, CORE, TILE, and DIE shifts from extended
+	 * Try to get SMT, CORE, TILE, and DIE shifts from extended
 	 * CPUID leaf 0x8000_0026 on supported processors first. If
 	 * extended CPUID leaf 0x8000_0026 is not supported, try to
-	 * get SMT and CORE shift from leaf 0xb first, then try to
-	 * get the CORE shift from leaf 0x8000_0008.
+	 * get SMT and CORE shift from leaf 0xb. If either leaf is
+	 * available, cpu_parse_topology_ext() will return true.
 	 */
-	if (cpu_feature_enabled(X86_FEATURE_TOPOEXT))
-		has_topoext = cpu_parse_topology_ext(tscan);
+	bool has_xtopology = cpu_parse_topology_ext(tscan);
 
 	if (cpu_feature_enabled(X86_FEATURE_AMD_HTR_CORES))
 		tscan->c->topo.cpu_type = cpuid_ebx(0x80000026);
 
-	if (!has_topoext && !parse_8000_0008(tscan))
+	/*
+	 * If XTOPOLOGY leaves (0x26/0xb) are not available, try to
+	 * get the CORE shift from leaf 0x8000_0008 first.
+	 */
+	if (!has_xtopology && !parse_8000_0008(tscan))
 		return;
 
-	/* Prefer leaf 0x8000001e if available */
-	if (parse_8000_001e(tscan, has_topoext))
+	/*
+	 * Prefer leaf 0x8000001e if available to get the SMT shift and
+	 * the initial APIC ID if XTOPOLOGY leaves are not available.
+	 */
+	if (parse_8000_001e(tscan, has_xtopology))
 		return;
 
 	/* Try the NODEID MSR */
-- 
2.34.1


^ permalink raw reply related	[flat|nested] 5+ messages in thread

* [PATCH v5 2/4] x86/cpu/topology: Check for X86_FEATURE_XTOPOLOGY instead of passing has_xtopology
  2025-09-01 17:04 [PATCH v5 0/4] x86/cpu/topology: Fix the preferred order of initial APIC ID parsing on AMD/Hygon K Prateek Nayak
  2025-09-01 17:04 ` [PATCH v5 1/4] x86/cpu/topology: Always try cpu_parse_topology_ext() " K Prateek Nayak
@ 2025-09-01 17:04 ` K Prateek Nayak
  2025-09-01 17:04 ` [PATCH v5 3/4] x86/msr-index: Define AMD64_CPUID_FN_EXT MSR K Prateek Nayak
  2025-09-01 17:04 ` [RFC PATCH v5 4/4] Documentation/x86/topology: Detail CPUID leaves used for topology enumeration K Prateek Nayak
  3 siblings, 0 replies; 5+ messages in thread
From: K Prateek Nayak @ 2025-09-01 17:04 UTC (permalink / raw)
  To: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
	Sean Christopherson, Paolo Bonzini, Jonathan Corbet, x86
  Cc: Naveen rao, Sairaj Kodilkar, H. Peter Anvin,
	Peter Zijlstra (Intel), Xin Li (Intel), Pawan Gupta, linux-kernel,
	kvm, Mario Limonciello, Gautham R. Shenoy, Babu Moger,
	Suravee Suthikulpanit, K Prateek Nayak

cpu_parse_topology_ext() sets X86_FEATURE_XTOPOLOGY before returning
true if any of the XTOPOLOGY leaf (0x80000026 / 0xb) could be parsed
successfully.

Instead of storing and passing around this return value using
"has_xtopology" in parse_topology_amd(), check for X86_FEATURE_XTOPOLOGY
directly in parse_8000_001e() to simplify the flow.

No functional changes intended.

Signed-off-by: K Prateek Nayak <kprateek.nayak@amd.com>
---
Changelog v4..v5:

o No functional changes. The diff is slightly altered as a result of
  modifying the comment in parse_topology_amd() in Patch 1.
---
 arch/x86/kernel/cpu/topology_amd.c | 19 ++++++++-----------
 1 file changed, 8 insertions(+), 11 deletions(-)

diff --git a/arch/x86/kernel/cpu/topology_amd.c b/arch/x86/kernel/cpu/topology_amd.c
index c79ebbb639cb..7ebd4a15c561 100644
--- a/arch/x86/kernel/cpu/topology_amd.c
+++ b/arch/x86/kernel/cpu/topology_amd.c
@@ -59,7 +59,7 @@ static void store_node(struct topo_scan *tscan, u16 nr_nodes, u16 node_id)
 	tscan->amd_node_id = node_id;
 }
 
-static bool parse_8000_001e(struct topo_scan *tscan, bool has_topoext)
+static bool parse_8000_001e(struct topo_scan *tscan)
 {
 	struct {
 		// eax
@@ -85,7 +85,7 @@ static bool parse_8000_001e(struct topo_scan *tscan, bool has_topoext)
 	 * If leaf 0xb/0x26 is available, then the APIC ID and the domain
 	 * shifts are set already.
 	 */
-	if (!has_topoext) {
+	if (!cpu_feature_enabled(X86_FEATURE_XTOPOLOGY)) {
 		tscan->c->topo.initial_apicid = leaf.ext_apic_id;
 
 		/*
@@ -175,30 +175,27 @@ static void topoext_fixup(struct topo_scan *tscan)
 
 static void parse_topology_amd(struct topo_scan *tscan)
 {
+	if (cpu_feature_enabled(X86_FEATURE_AMD_HTR_CORES))
+		tscan->c->topo.cpu_type = cpuid_ebx(0x80000026);
+
 	/*
 	 * Try to get SMT, CORE, TILE, and DIE shifts from extended
 	 * CPUID leaf 0x8000_0026 on supported processors first. If
 	 * extended CPUID leaf 0x8000_0026 is not supported, try to
 	 * get SMT and CORE shift from leaf 0xb. If either leaf is
 	 * available, cpu_parse_topology_ext() will return true.
-	 */
-	bool has_xtopology = cpu_parse_topology_ext(tscan);
-
-	if (cpu_feature_enabled(X86_FEATURE_AMD_HTR_CORES))
-		tscan->c->topo.cpu_type = cpuid_ebx(0x80000026);
-
-	/*
+	 *
 	 * If XTOPOLOGY leaves (0x26/0xb) are not available, try to
 	 * get the CORE shift from leaf 0x8000_0008 first.
 	 */
-	if (!has_xtopology && !parse_8000_0008(tscan))
+	if (!cpu_parse_topology_ext(tscan) && !parse_8000_0008(tscan))
 		return;
 
 	/*
 	 * Prefer leaf 0x8000001e if available to get the SMT shift and
 	 * the initial APIC ID if XTOPOLOGY leaves are not available.
 	 */
-	if (parse_8000_001e(tscan, has_xtopology))
+	if (parse_8000_001e(tscan))
 		return;
 
 	/* Try the NODEID MSR */
-- 
2.34.1


^ permalink raw reply related	[flat|nested] 5+ messages in thread

* [PATCH v5 3/4] x86/msr-index: Define AMD64_CPUID_FN_EXT MSR
  2025-09-01 17:04 [PATCH v5 0/4] x86/cpu/topology: Fix the preferred order of initial APIC ID parsing on AMD/Hygon K Prateek Nayak
  2025-09-01 17:04 ` [PATCH v5 1/4] x86/cpu/topology: Always try cpu_parse_topology_ext() " K Prateek Nayak
  2025-09-01 17:04 ` [PATCH v5 2/4] x86/cpu/topology: Check for X86_FEATURE_XTOPOLOGY instead of passing has_xtopology K Prateek Nayak
@ 2025-09-01 17:04 ` K Prateek Nayak
  2025-09-01 17:04 ` [RFC PATCH v5 4/4] Documentation/x86/topology: Detail CPUID leaves used for topology enumeration K Prateek Nayak
  3 siblings, 0 replies; 5+ messages in thread
From: K Prateek Nayak @ 2025-09-01 17:04 UTC (permalink / raw)
  To: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
	Sean Christopherson, Paolo Bonzini, Jonathan Corbet, x86
  Cc: Naveen rao, Sairaj Kodilkar, H. Peter Anvin,
	Peter Zijlstra (Intel), Xin Li (Intel), Pawan Gupta, linux-kernel,
	kvm, Mario Limonciello, Gautham R. Shenoy, Babu Moger,
	Suravee Suthikulpanit, K Prateek Nayak

Explicitly define the AMD64_CPUID_FN_EXT MSR used to toggle the extended
features. Also define and use the bits necessary for an old TOPOEXT
fixup on AMD Family 0x15 processors.

No functional changes intended.

Signed-off-by: K Prateek Nayak <kprateek.nayak@amd.com>
---
Changelog v4..v5:

o No changes.
---
 arch/x86/include/asm/msr-index.h   | 5 +++++
 arch/x86/kernel/cpu/topology_amd.c | 7 ++++---
 2 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index f627196eb796..176ca7040139 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -633,6 +633,11 @@
 #define MSR_AMD_PPIN			0xc00102f1
 #define MSR_AMD64_CPUID_FN_7		0xc0011002
 #define MSR_AMD64_CPUID_FN_1		0xc0011004
+
+#define MSR_AMD64_CPUID_FN_EXT				0xc0011005
+#define MSR_AMD64_CPUID_FN_EXT_TOPOEXT_ENABLED_BIT	54
+#define MSR_AMD64_CPUID_FN_EXT_TOPOEXT_ENABLED		BIT_ULL(MSR_AMD64_CPUID_FN_EXT_TOPOEXT_ENABLED_BIT)
+
 #define MSR_AMD64_LS_CFG		0xc0011020
 #define MSR_AMD64_DC_CFG		0xc0011022
 #define MSR_AMD64_TW_CFG		0xc0011023
diff --git a/arch/x86/kernel/cpu/topology_amd.c b/arch/x86/kernel/cpu/topology_amd.c
index 7ebd4a15c561..07510647a378 100644
--- a/arch/x86/kernel/cpu/topology_amd.c
+++ b/arch/x86/kernel/cpu/topology_amd.c
@@ -163,11 +163,12 @@ static void topoext_fixup(struct topo_scan *tscan)
 	    c->x86 != 0x15 || c->x86_model < 0x10 || c->x86_model > 0x6f)
 		return;
 
-	if (msr_set_bit(0xc0011005, 54) <= 0)
+	if (msr_set_bit(MSR_AMD64_CPUID_FN_EXT,
+			MSR_AMD64_CPUID_FN_EXT_TOPOEXT_ENABLED_BIT) <= 0)
 		return;
 
-	rdmsrq(0xc0011005, msrval);
-	if (msrval & BIT_64(54)) {
+	rdmsrq(MSR_AMD64_CPUID_FN_EXT, msrval);
+	if (msrval & MSR_AMD64_CPUID_FN_EXT_TOPOEXT_ENABLED) {
 		set_cpu_cap(c, X86_FEATURE_TOPOEXT);
 		pr_info_once(FW_INFO "CPU: Re-enabling disabled Topology Extensions Support.\n");
 	}
-- 
2.34.1


^ permalink raw reply related	[flat|nested] 5+ messages in thread

* [RFC PATCH v5 4/4] Documentation/x86/topology: Detail CPUID leaves used for topology enumeration
  2025-09-01 17:04 [PATCH v5 0/4] x86/cpu/topology: Fix the preferred order of initial APIC ID parsing on AMD/Hygon K Prateek Nayak
                   ` (2 preceding siblings ...)
  2025-09-01 17:04 ` [PATCH v5 3/4] x86/msr-index: Define AMD64_CPUID_FN_EXT MSR K Prateek Nayak
@ 2025-09-01 17:04 ` K Prateek Nayak
  3 siblings, 0 replies; 5+ messages in thread
From: K Prateek Nayak @ 2025-09-01 17:04 UTC (permalink / raw)
  To: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
	Sean Christopherson, Paolo Bonzini, Jonathan Corbet, x86
  Cc: Naveen rao, Sairaj Kodilkar, H. Peter Anvin,
	Peter Zijlstra (Intel), Xin Li (Intel), Pawan Gupta, linux-kernel,
	kvm, Mario Limonciello, Gautham R. Shenoy, Babu Moger,
	Suravee Suthikulpanit, K Prateek Nayak

Add a new section describing the different CPUID leaves and fields used
to parse topology on x86 systems.

Suggested-by: Borislav Petkov <bp@alien8.de>
Signed-off-by: K Prateek Nayak <kprateek.nayak@amd.com>
---
Changelog v4..v5:

o Added a nte about the NODE_ID_MSR on AMD platforms.
---
 Documentation/arch/x86/topology.rst | 198 ++++++++++++++++++++++++++++
 1 file changed, 198 insertions(+)

diff --git a/Documentation/arch/x86/topology.rst b/Documentation/arch/x86/topology.rst
index c12837e61bda..4227eba65957 100644
--- a/Documentation/arch/x86/topology.rst
+++ b/Documentation/arch/x86/topology.rst
@@ -141,6 +141,204 @@ Thread-related topology information in the kernel:
 
 
 
+System topology enumeration
+===========================
+The topology on x86 systems can be discovered using a combination of vendor
+specific CPUID leaves introduced specifically to enumerate the processor
+topology and the cache hierarchy.
+
+The CPUID leaves in their preferred order of parsing for each x86 vendor is as
+follows:
+
+1) AMD and Hygon
+
+   On AMD and Hygon platforms, the CPUID leaves that enumerate the processor
+   topology are as follows:
+
+   1) CPUID leaf 0x80000026 [Extended CPU Topology] (Core::X86::Cpuid::ExCpuTopology)
+
+      The extended CPUID leaf 0x80000026 is the extension of the CPUID leaf 0xB
+      and provides the topology information of Core, Complex, CCD(Die), and
+      Socket in each level.
+
+      The support for the leaf is expected to be discovered by checking if the
+      supported extended CPUID level is >= 0x80000026 and then checking if
+      `LogProcAtThisLevel` in `EBX[15:0]` at a particular level (starting from
+      0) is non-zero.
+
+      The `LevelType` in `ECX[15:8]` at the level provides the detail of the
+      topology domain that the level describes - Core, Complex, CCD(Die), or
+      the Socket.
+
+      The kernel uses the `CoreMaskWidth` from `EAX[4:0]` to discover the
+      number of bits that need to be right shifted from the
+      `ExtendedLocalApicId` in `EDX[31:0]` to get a unique Topology ID for
+      the topology level. CPUs with the same Topology ID share the resources
+      at that level.
+
+      CPUID leaf 0x80000026 also provides more information regarding the
+      power and efficiency rankings, and about the core type on AMD
+      processors with heterogeneous characteristics.
+
+      If CPUID leaf 0x80000026 is supported, further parsing is not required.
+
+
+   2) CPUID leaf 0x0000000B [Extended Topology Enumeration] (Core::X86::Cpuid::ExtTopEnum)
+
+      The extended CPUID leaf 0x0000000B is the predecessor on the extended
+      CPUID leaf 0x80000026 and only describes the core, and the socket domains
+      of the processor topology.
+
+      The support for the leaf is expected to be discovered by checking if the
+      supported CPUID level is >= 0xB and then checking if `EBX[31:0]` at a
+      particular level (starting from 0) is non-zero.
+
+      The `LevelType` in `ECX[15:8]` at the level provides the detail of the
+      topology domain that the level describes - Thread, or Processor (Socket).
+
+      The kernel uses the `CoreMaskWidth` from `EAX[4:0]` to discover the
+      number of bits that need to be right shifted from the
+      `ExtendedLocalApicId` in `EDX[31:0]` to get a unique Topology ID for
+      that topology level. CPUs sharing the Topology ID share the resources
+      at that level.
+
+      If CPUID leaf 0xB is supported, further parsing is not required.
+
+
+   3) CPUID leaf 0x80000008 ECX [Size Identifiers] (Core::X86::Cpuid::SizeId)
+
+      If neither the CPUID leaf 0x80000026 or CPUID leaf 0xB is supported, the
+      number of CPUs on the package is detected using the Size Identifier leaf
+      0x80000008 ECX.
+
+      The support for the leaf is expected to be discovered by checking if the
+      supported extended CPUID level is >= 0x80000008.
+
+      The shifts from the APIC ID for the Socket ID is calculated from the
+      `ApicIdSize` field in `ECX[15:12]` if it is non-zero.
+
+      If `ApicIdSize` is reported to be zero, the shift is calculated as the
+      order of the `number of threads` calculated from `NC` field in
+      `ECX[7:0]` which describes the `number of threads - 1` on the package.
+
+      Unless Extended APIC ID is supported, the APIC ID used to find the
+      Socket ID is from the `LocalApicId` field of CPUID leaf 0x00000001
+      `EBX[31:24]`.
+
+      The topology parsing continues to detect if Extended APIC ID is
+      supported or not.
+
+
+   4) CPUID leaf 0x8000001E [Extended APIC ID, Core Identifiers, Node Identifiers]
+      (Core::X86::Cpuid::{ExtApicId,CoreId,NodeId})
+
+      The support for Extended APIC ID can be detected by checking for the
+      presence of `TopologyExtensions` in `ECX[22]` of CPUID leaf 0x80000001
+      [Feature Identifiers] (Core::X86::Cpuid::FeatureExtIdEcx).
+
+      If Topology Extensions is supported, the APIC ID from `ExtendedApicId`
+      from CPUID leaf 0x8000001E `EAX[31:0]` should be preferred over that from
+      `LocalApicId` field of CPUID leaf 0x00000001 `EBX[31:24]` for topology
+      enumeration.
+
+      On processors of Family 0x17 and above that do not support CPUID leaf
+      0x80000026 or CPUID leaf 0xB, the shifts from the APIC ID for the Core
+      ID is calculated using the order of `number of threads per core`
+      calculated using the `ThreadsPerCore` field in `EBX[15:8]` which
+      describes `number of threads per core - 1`.
+
+      On Processors of Family 0x15, the Core ID from `EBX[7:0]` is used as the
+      `cu_id` (Compute Unit ID) to detect CPUs that share the compute units.
+
+
+   All AMD and Hygon processors that support the `TopologyExtensions` feature
+   stores the `NodeId` from the `ECX[7:0]` of CPUID leaf 0x8000001E
+   (Core::X86::Cpuid::NodeId) as the per-CPU `node_id`. On older processors,
+   the `node_id` was discovered using MSR_FAM10H_NODE_ID MSR (MSR
+   0x0xc001_100c). The presence of the NODE_ID MSR was detected by checking
+   `ECX[19]` of CPUID leaf 0x80000001 [Feature Identifiers]
+   (Core::X86::Cpuid::FeatureExtIdEcx).
+
+
+2) Intel
+
+   On Intel platforms, the CPUID leaves that enumerate the processor
+   topology are as follows:
+
+   1) CPUID leaf 0x1F (V2 Extended Topology Enumeration Leaf)
+
+      The CPUID leaf 0x1F is the extension of the CPUID leaf 0xB and provides
+      the topology information of Core, Module, Tile, Die, DieGrp, and Socket
+      in each level.
+
+      The support for the leaf is expected to be discovered by checking if
+      the supported CPUID level is >= 0x1F and then `EBX[31:0]` at a
+      particular level (starting from 0) is non-zero.
+
+      The `Domain Type` in `ECX[15:8]` of the sub-leaf provides the detail of
+      the topology domain that the level describes - Core, Module, Tile, Die,
+      DieGrp, and Socket.
+
+      The kernel uses the value from `EAX[4:0]` to discover the number of
+      bits that need to be right shifted from the `x2APIC ID` in `EDX[31:0]`
+      to get a unique Topology ID for the topology level. CPUs with the same
+      Topology ID share the resources at that level.
+
+      If CPUID leaf 0x1F is supported, further parsing is not required.
+
+
+   2) CPUID leaf 0x0000000B (Extended Topology Enumeration Leaf)
+
+      The extended CPUID leaf 0x0000000B is the predecessor of the V2 Extended
+      Topology Enumeration Leaf 0x1F and only describes the core, and the
+      socket domains of the processor topology.
+
+      The support for the leaf is expected to be discovered by checking if the
+      supported CPUID level is >= 0xB and then checking if `EBX[31:0]` at a
+      particular level (starting from 0) is non-zero.
+
+      CPUID leaf 0x0000000B shares the same layout as CPUID leaf 0x1F and
+      should be enumerated in a similar manner.
+
+      If CPUID leaf 0xB is supported, further parsing is not required.
+
+
+   3) CPUID leaf 0x00000004 (Deterministic Cache Parameters Leaf)
+
+      On Intel processors that support neither CPUID leaf 0x1F, nor CPUID leaf
+      0xB, the shifts for the SMT domains is calculated using the number of
+      CPUs sharing the L1 cache.
+
+      Processors that feature Hyper-Threading is detected using `EDX[28]` of
+      CPUID leaf 0x1 (Basic CPUID Information).
+
+      The order of `Maximum number of addressable IDs for logical processors
+      sharing this cache` from `EAX[25:14]` of level-0 of CPUID 0x4 provides
+      the shifts from the APIC ID required to compute the Core ID.
+
+      The APIC ID and Package information is computed using the data from
+      CPUID leaf 0x1.
+
+
+   4) CPUID leaf 0x00000001 (Basic CPUID Information)
+
+      The mask and shifts to derive the Physical Package (socket) ID is
+      computed using the `Maximum number of addressable IDs for logical
+      processors in this physical package` from `EBX[23:16]` of CPUID leaf
+      0x1.
+
+     The APIC ID on the legacy platforms is derived from the `Initial APIC
+     ID` field from `EBX[31:24]` of CPUID leaf 0x1.
+
+
+3) Centaur and Zhaoxin
+
+   Similar to Intel, Centaur and Zhaoxin use a combination of CPUID leaf
+   0x00000004 (Deterministic Cache Parameters Leaf) and CPUID leaf 0x00000001
+   (Basic CPUID Information) to derive the topology information.
+
+
+
 System topology examples
 ========================
 
-- 
2.34.1


^ permalink raw reply related	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2025-09-01 17:06 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-09-01 17:04 [PATCH v5 0/4] x86/cpu/topology: Fix the preferred order of initial APIC ID parsing on AMD/Hygon K Prateek Nayak
2025-09-01 17:04 ` [PATCH v5 1/4] x86/cpu/topology: Always try cpu_parse_topology_ext() " K Prateek Nayak
2025-09-01 17:04 ` [PATCH v5 2/4] x86/cpu/topology: Check for X86_FEATURE_XTOPOLOGY instead of passing has_xtopology K Prateek Nayak
2025-09-01 17:04 ` [PATCH v5 3/4] x86/msr-index: Define AMD64_CPUID_FN_EXT MSR K Prateek Nayak
2025-09-01 17:04 ` [RFC PATCH v5 4/4] Documentation/x86/topology: Detail CPUID leaves used for topology enumeration K Prateek Nayak

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).