[PATCH 0/3] Increase MCA bank count

[PATCH 0/3] Increase MCA bank count

[Yazen Ghannam]

Hi all,

This set increases the max MCA bank count (MAX_NR_BANKS) to the
architectural limit.

Patch 1 is a bug fix that came out of code review.

Patch 2 is a pre-patch to avoid a bug for systems with more than 64 MCA
banks per thread.

Patch 3 is where MAX_NR_BANKS is updated. The commit message includes
some memory stats like the last time we bumped up MAX_NR_BANKS.

Originally, I intended to include additional patches to consolidate and
dynamically allocate many of the per-CPU and per-bank structures. These
still need more work. So I thought to send just this small set to start.

Also, I figure a smaller set would be easier for backporting for those
who want the feature support but don't want the additional rework.

Thanks,
Yazen

Yazen Ghannam (3):
  x86/mce/amd: Fix bank lookup in amd_mce_usable_address()
  x86/mce/amd: Convert bank_map to a bitmap
  x86/mce: Increase MAX_NR_BANKS to 255

 arch/x86/include/asm/mce.h      | 2 +-
 arch/x86/kernel/cpu/mce/amd.c   | 9 +++++----
 arch/x86/kernel/cpu/mce/core.c  | 6 ------
 arch/x86/kernel/cpu/mce/intel.c | 2 +-
 4 files changed, 7 insertions(+), 12 deletions(-)


base-commit: cee9395acd8043be0644b25c34bfa86623f2b935
-- 
2.43.0

[PATCH 1/3] x86/mce/amd: Fix bank lookup in amd_mce_usable_address()

[Yazen Ghannam]

amd_mce_usable_address() reads the per-CPU smca_banks array at index
m->bank on the running CPU. The index is not checked, and the bank
belongs to m->extcpu, not to whichever CPU is decoding.

m->bank is unbounded here. apei_mce_report_mem_error() sets it to -1,
which is 255 in the u8 field, and apei_smca_report_x86_error() takes it
from a firmware BERT record. Both reach this function through the
decoder chain, which runs from a workqueue on any CPU. Bank counts
differ per CPU on SMCA systems. So the read can land past the end of
the array, or on a CPU where that bank means something else.

Use m->extcpu for both the bound and the lookup.

Fixes: 821f5fe4dbcb ("x86/mce: Add support for physical address valid bit")
Cc: stable@vger.kernel.org
Assisted-by: Claude:claude-opus-5
Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
 arch/x86/kernel/cpu/mce/amd.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kernel/cpu/mce/amd.c b/arch/x86/kernel/cpu/mce/amd.c
index f916fb4c5d13..e6542e00dc5a 100644
--- a/arch/x86/kernel/cpu/mce/amd.c
+++ b/arch/x86/kernel/cpu/mce/amd.c
@@ -837,7 +837,8 @@ bool amd_mce_usable_address(struct mce *m)
 			return false;
 	}
 
-	if (this_cpu_ptr(smca_banks)[m->bank].paddrv)
+	if (m->bank < per_cpu(mce_num_banks, m->extcpu) &&
+	    per_cpu(smca_banks, m->extcpu)[m->bank].paddrv)
 		return m->status & MCI_STATUS_PADDRV;
 
 	/* Check poison bit for all other bank types. */
-- 
2.43.0

[PATCH 2/3] x86/mce/amd: Convert bank_map to a bitmap

[Yazen Ghannam]

The per-CPU bank_map records which banks had their block 0 prepared
during mce_amd_feature_init(). It is a u64, so BIT_ULL(bank) is
undefined once the bank number reaches 64. That caps the usable bank
count at 64 regardless of MAX_NR_BANKS.

mce_banks_t is the existing type for per-bank bitmaps. It is declared
with MAX_NR_BANKS bits, so it follows the macro instead of fixing a
width of its own. thr_intr_banks and dfr_intr_banks in this file
already use it.

Convert bank_map to mce_banks_t and use the bitmap helpers. This is
preparation for raising MAX_NR_BANKS.

Assisted-by: Claude:claude-opus-5
Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
 arch/x86/kernel/cpu/mce/amd.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kernel/cpu/mce/amd.c b/arch/x86/kernel/cpu/mce/amd.c
index e6542e00dc5a..598b71d54f41 100644
--- a/arch/x86/kernel/cpu/mce/amd.c
+++ b/arch/x86/kernel/cpu/mce/amd.c
@@ -263,7 +263,7 @@ static DEFINE_PER_CPU(struct threshold_bank **, threshold_banks);
  * A list of the banks enabled on each logical CPU. Controls which respective
  * descriptors to initialize later in mce_threshold_create_device().
  */
-static DEFINE_PER_CPU(u64, bank_map);
+static DEFINE_PER_CPU(mce_banks_t, bank_map);
 
 static void amd_threshold_interrupt(void);
 static void amd_deferred_error_interrupt(void);
@@ -572,7 +572,7 @@ static int prepare_threshold_block(unsigned int bank, unsigned int block, u32 ad
 	int new;
 
 	if (!block)
-		per_cpu(bank_map, cpu) |= BIT_ULL(bank);
+		__set_bit(bank, per_cpu(bank_map, cpu));
 
 	memset(&b, 0, sizeof(b));
 	b.cpu			= cpu;
@@ -1272,7 +1272,7 @@ void mce_threshold_create_device(unsigned int cpu)
 		return;
 
 	for (bank = 0; bank < numbanks; ++bank) {
-		if (!(this_cpu_read(bank_map) & BIT_ULL(bank)))
+		if (!test_bit(bank, this_cpu_ptr(bank_map)))
 			continue;
 		if (threshold_create_bank(bp, cpu, bank)) {
 			__threshold_remove_device(bp);
-- 
2.43.0

[PATCH 3/3] x86/mce: Increase MAX_NR_BANKS to 255

[Yazen Ghannam]

Machine Check Architecture (MCA) allows up to 255 MCA banks per CPU
thread. This is based on the size of the "Count" field of the MCG_CAP
registers. The "Count" field is 8 bits and holds the number of MCA banks
per CPU thread. So the value range is 0-255.

Change the MAX_NR_BANKS value to match the architectural definition.

Drop the clamps in __mcheck_cpu_cap_init() and cmci_supported().
MCG_CAP[Count] can no longer exceed MAX_NR_BANKS, so neither is
reachable.

MAX_NR_BANKS sizes a number of arrays and bitmaps. Growing it by 191
banks costs the following, measured with size -A on the MCA objects
built from x86_64 defconfig.

Global bitmaps:
- core.c / mce_banks_ce_disabled
- Total: 191 new bits = 24 new bytes

Per-CPU bitmaps:
- core.c / mce_poll_banks
- intel.c / mce_banks_owned
- amd.c / bank_map
- amd.c / mce_amd_data: thr_intr_banks and dfr_intr_banks
- Total: 191 new bits * 5 bitmaps = 120 new bytes

Global structs:
- core.c / struct mce_bank_dev mce_bank_devs[]: 56 bytes per bank
- intel.c / u16 cmci_threshold[]: 2 bytes per bank
- Total: 191 new banks * 58 bytes = 11078 new bytes

Per-CPU structs:
- core.c / struct mce_bank mce_banks_array[]: 16 bytes per bank
- amd.c / struct smca_bank smca_banks[]: 24 bytes per bank
- threshold.c / struct mca_storm_desc storm_desc: 24 bytes per bank
- Total: 191 new banks * 64 bytes = 12224 new bytes

Section totals, which include alignment padding:

Total global size increase: 11152 bytes of .bss
Total per-CPU size increase: 12368 bytes

The per-CPU total is replicated for every possible CPU. In the linked
image .data..percpu grows by 12352 bytes. The first per-CPU chunk is
page aligned, so each CPU retains three more pages, 229376 bytes up
from 217088. That is 5.7% more per-CPU memory, or 6 MiB on a 512
thread system.

Assisted-by: Claude:claude-opus-5
Signed-off-by: Yazen Ghannam <yazen.ghannam@amd.com>
---
 arch/x86/include/asm/mce.h      | 2 +-
 arch/x86/kernel/cpu/mce/core.c  | 6 ------
 arch/x86/kernel/cpu/mce/intel.c | 2 +-
 3 files changed, 2 insertions(+), 8 deletions(-)

diff --git a/arch/x86/include/asm/mce.h b/arch/x86/include/asm/mce.h
index e575b702063d..877641d76efb 100644
--- a/arch/x86/include/asm/mce.h
+++ b/arch/x86/include/asm/mce.h
@@ -268,7 +268,7 @@ void mce_log(struct mce_hw_err *err);
 DECLARE_PER_CPU(struct device *, mce_device);
 
 /* Maximum number of MCA banks per CPU. */
-#define MAX_NR_BANKS 64
+#define MAX_NR_BANKS 255
 
 #ifdef CONFIG_X86_MCE_INTEL
 void mce_intel_feature_init(struct cpuinfo_x86 *c);
diff --git a/arch/x86/kernel/cpu/mce/core.c b/arch/x86/kernel/cpu/mce/core.c
index ab469605fc89..66bc30f2b106 100644
--- a/arch/x86/kernel/cpu/mce/core.c
+++ b/arch/x86/kernel/cpu/mce/core.c
@@ -1849,12 +1849,6 @@ static void __mcheck_cpu_cap_init(void)
 
 	b = cap & MCG_BANKCNT_MASK;
 
-	if (b > MAX_NR_BANKS) {
-		pr_warn("CPU%d: Using only %u machine check banks out of %u\n",
-			smp_processor_id(), MAX_NR_BANKS, b);
-		b = MAX_NR_BANKS;
-	}
-
 	this_cpu_write(mce_num_banks, b);
 
 	__mcheck_cpu_mce_banks_init();
diff --git a/arch/x86/kernel/cpu/mce/intel.c b/arch/x86/kernel/cpu/mce/intel.c
index 4655223ba560..fa63f864e4ad 100644
--- a/arch/x86/kernel/cpu/mce/intel.c
+++ b/arch/x86/kernel/cpu/mce/intel.c
@@ -95,7 +95,7 @@ static bool cmci_supported(int *banks)
 		return false;
 
 	rdmsrq(MSR_IA32_MCG_CAP, cap);
-	*banks = min_t(unsigned, MAX_NR_BANKS, cap & MCG_BANKCNT_MASK);
+	*banks = this_cpu_read(mce_num_banks);
 	return !!(cap & MCG_CMCI_P);
 }
 
-- 
2.43.0