[PATCH v2] cxl/mce: Avoid alias page retirement for corrected errors

[PATCH v2] cxl/mce: Avoid alias page retirement for corrected errors

[Shaikh Kamaluddin]

cxl_handle_mce() offlines the aliased page of an Extended Linear Cache
(ELC) region for any MCE with a usable address in the region. This
includes corrected errors and non-memory errors, needlessly reducing
usable memory.

Restrict ELC MCE handling to uncorrected memory errors. Use
mce_is_correctable() so that MCE_AR_SEVERITY errors continue to retire
the alias when the reported page is handled elsewhere.

Fixes: 516e5bd0b6bf ("cxl: Add mce notifier to emit aliased address for extended linear cache")
Signed-off-by: Shaikh Kamaluddin <shaikhkamal2012@gmail.com>
---
Changelog:

v1 -> v2:
  - Reworked commit message per review feedback
  - Dropped redundant comment from the code
 
Reproduced under QEMU/vng. QEMU's cxl-type3 does not emulate the HMAT
extended-linear address_mode bit, so ELC was forced locally for testing
via a one-line debug hack in cxl_region_probe() (not part of this patch):

	if (!p->cache_size && p->res)
		p->cache_size = resource_size(p->res) / 2;

Steps:
1. Boot with an Intel CPU model under TCG (KVM host-passthrough will
    otherwise leak the host's real vendor ID, and AMD/SMCA takes a
    different mce_usable_address() path than the one under test):
vng -v -r ./arch/x86/boot/bzImage --disable-kvm --qemu-opts='-cpu Skylake-Server-v4,+mce,+mca -m 4G -machine q35,cxl=on -object memory-backend-ram,id=cxl-mem0,size=512M -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.0 -device cxl-rp,port=0,bus=cxl.0,id=root_port0,chassis=0,slot=0 -device cxl-type3,bus=root_port0,volatile-memdev=cxl-mem0,id=cxl-mem-device0 -M cxl-fmw.0.targets.0=cxl.0,cxl-fmw.0.size=512M'

2. modprobe mce-inject
$cxl list -M
$cxl list -D
$cxl create-region -m mem0 -d decoder0.0 -w 1 -g 256 -t ram
$dmesg | grep "DEBUG: forced cache_size"
$modprobe device_dax
$modprobe kmem
$ls /sys/bus/dax/devices/
$daxctl reconfigure-device dax0.0 --mode=system-ram
$lsmem
it will show as below:
dax0.0
[   42.857642] Fallback order for Node 0: 0 
[   42.857828] Built 1 zonelists, mobility grouping on.  Total pages: 1010805
[   42.858446] Policy zone: Normal
[
  {
    "chardev":"dax0.0",
    "size":536870912,
    "target_node":0,
    "align":2097152,
    "mode":"system-ram",
    "online_memblocks":4,
    "total_memblocks":4,
    "movable":true
  }
]
reconfigured 1 device
RANGE                                  SIZE  STATE REMOVABLE BLOCK
0x0000000000000000-0x000000007fffffff    2G online       yes  0-15
0x0000000100000000-0x000000017fffffff    2G online       yes 32-47
0x0000000190000000-0x00000001afffffff  512M online       yes 50-53

Memory block size:       128M
Total online memory:     4.5G
Total offline memory:      0B

3. Load the injector if not loaded earlier and derive the two MCi_STATUS values.

    modprobe mce-inject

    MCi_STATUS bit layout used here (arch/x86/include/asm/mce.h):
      bit 63  VAL     - record valid
      bit 61  UC      - uncorrected (0 = corrected error under test)
      bit 60  EN      - error reporting enabled
      bit 59  MISCV   - MCi_MISC valid
      bit 58  ADDRV   - MCi_ADDR valid
      bits[15:0] MCACOD - bit 7 set = memory-error signature, per
                          mce_is_memory_error()'s Intel branch

    python3 -c "
    VAL, UC, EN, MISCV, ADDRV = 1<<63, 1<<61, 1<<60, 1<<59, 1<<58
    MCACOD_MEM = 1<<7                      # memory error signature
    ce = VAL | EN | MISCV | ADDRV | MCACOD_MEM
    uc = ce | UC
    print(f'CE status = {hex(ce)}')
    print(f'UC status = {hex(uc)}')"
    # CE status = 0x9c00000000000080
    # UC status = 0xbc00000000000080

    MCi_MISC: address-mode field, bits[8:6], must be 2 (physical):

    python3 -c "print(hex(2 << 6))"
    # misc = 0x80

4.  $ cd /sys/kernel/debug/mce-inject
$echo sw > flags
$echo 0x9C00000000000080 > status
$echo 0x80 > misc
$echo 0x190010000 > addr
$echo 9 > bank
[  221.813395] mce: [Hardware Error]: Machine check events logged
[  221.814716] cxl_mce_debug: entered status=0x9c00000000000080 addr=0x190010000 cache_size=0x10000000 res=[mem 0x190000000-0x1afffffff flags 0x200] usable=1
[  221.816200] cxl_mce_debug: spa=0x190010000 contains=1
[  221.817089] cxl_mce_debug: spa_alias=0x1a0010000 pfn=0x1a0010 pfn_valid=1
[  221.817655] cxl_region region0: Offlining aliased SPA address0: 0x1a0010000
[  221.821504] Memory failure: 0x1a0010: recovery action for free buddy page: Recovered
[  221.823903] mce: [Hardware Error]: CPU 0: Machine Check: 0 Bank 9: 9c00000000000080
[  221.824617] mce: [Hardware Error]: TSC a605ccda40 ADDR 190010000 MISC 80 
[  221.824997] mce: [Hardware Error]: PROCESSOR 0:50654 TIME 1786375908 SOCKET 0 APIC 0 microcode 1
root@virtme-ng:/sys/kernel/debug/mce-inject# 
root@virtme-ng:/sys/kernel/debug/mce-inject# grep HardwareCorrupted /proc/meminfo
HardwareCorrupted:     4 kB

CE without this patch:
  cxl_region region0: Offlining aliased SPA address0: 0x1a0010000
  Memory failure: 0x1a0010: recovery action for free buddy page: Recovered
  HardwareCorrupted: 4 kB
------------------------------------- 
CE with this patch:
  (no "Offlining aliased SPA" message logged)
  HardwareCorrupted: 0 kB

$cd /sys/kernel/debug/mce-inject
$echo sw > flags
$echo 0x9C00000000000080 > status
$echo 0x80 > misc
$echo 0x190010000 > addr
$echo 9 > bank
addr  bank  cpu  flags	ipid  misc  README  status  synd
[   61.681659] mce: [Hardware Error]: Machine check events logged
[   61.683442] cxl_mce_debug: entered status=0x9c00000000000080 addr=0x190010000 cache_size=0x10000000 res=[mem 0x190000000-0x1afffffff flags 0x200] usable=1
[   61.684528] mce: [Hardware Error]: CPU 0: Machine Check: 0 Bank 9: 9c00000000000080
[   61.684974] mce: [Hardware Error]: TSC 2eeae32fe0 ADDR 190010000 MISC 80 
[   61.688880] mce: [Hardware Error]: PROCESSOR 0:50654 TIME 1786377156 SOCKET 0 APIC 0 microcode 1
[   86.504456] clocksource: Watchdog remote CPU 11 read timed out

$grep HardwareCorrupted /proc/meminfo
HardwareCorrupted:     0 kB
-------------------------
For UC, same steps only status bit information will change :
$cd /sys/kernel/debug/mce-inject
$echo sw > flags
$echo 0xbc00000000000080 > status
$echo 0x80 > misc
$echo 0x190010000 > addr
$echo 9 > bank
$dmesg | grep cxl_mce_debug
$grep HardwareCorrupted /proc/meminfo
[  313.398747] mce: [Hardware Error]: Machine check events logged
[  313.400518] cxl_mce_debug: entered status=0xbc00000000000080 addr=0x190010000 cache_size=0x10000000 res=[mem 0x190000000-0x1afffffff flags 0x200] usable=1
[  313.401727] cxl_mce_debug: spa=0x190010000 contains=1
[  313.402157] cxl_mce_debug: spa_alias=0x1a0010000 pfn=0x1a0010 pfn_valid=1
[  313.402663] cxl_region region0: Offlining aliased SPA address0: 0x1a0010000
[  313.405999] Memory failure: 0x1a0010: recovery action for free buddy page: Recovered
[  313.408375] mce: [Hardware Error]: CPU 0: Machine Check: 0 Bank 9: bc00000000000080
[  313.408999] mce: [Hardware Error]: TSC e9dd2869e0 ADDR 190010000 MISC 80 
[  313.409367] mce: [Hardware Error]: PROCESSOR 0:50654 TIME 1786377750 SOCKET 0 APIC 0 microcode 1

Patched, UC: alias still offlined, confirming uncorrected handling is unchanged by this patch.
    cxl_region region0: Offlining aliased SPA address0: 0x1a0010000
    Memory failure: 0x1a0010: recovery action for free buddy page: Recovered
    HardwareCorrupted: 4 kB

 drivers/cxl/core/mce.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/drivers/cxl/core/mce.c b/drivers/cxl/core/mce.c
index 65fed913b221..e3c561489a00 100644
--- a/drivers/cxl/core/mce.c
+++ b/drivers/cxl/core/mce.c
@@ -18,7 +18,13 @@ static int cxl_handle_mce(struct notifier_block *nb, unsigned long val,
 	u64 spa, spa_alias;
 	unsigned long pfn;
 
-	if (!mce || !mce_usable_address(mce))
+	if (!mce)
+		return NOTIFY_DONE;
+
+	if (!mce_is_memory_error(mce) || mce_is_correctable(mce))
+		return NOTIFY_DONE;
+
+	if (!mce_usable_address(mce))
 		return NOTIFY_DONE;
 
 	spa = mce->addr & MCI_ADDR_PHYSADDR;

base-commit: 7098e9cd98a05c0c5de2fae0c2465f9d966fdd07
-- 
2.43.0

Re: [PATCH v2] cxl/mce: Avoid alias page retirement for corrected errors

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Bypassing CXL MCE handling for poison consumption errors on AMD architectures.
--

--- Patch [1]: [PATCH v2] cxl/mce: Avoid alias page retirement for corrected errors ---
commit efcb1cbc0f2e51e970d17decc010e02b84097870
Author: Shaikh Kamaluddin <shaikhkamal2012@gmail.com>

cxl/mce: Avoid alias page retirement for corrected errors

This commit restricts Extended Linear Cache (ELC) MCE handling to uncorrected
memory errors. It aims to prevent needlessly reducing usable memory on
corrected or non-memory errors.

> diff --git a/drivers/cxl/core/mce.c b/drivers/cxl/core/mce.c
> index 65fed913b2217..e3c561489a003 100644
> --- a/drivers/cxl/core/mce.c
> +++ b/drivers/cxl/core/mce.c
> @@ -18,7 +18,13 @@ static int cxl_handle_mce(struct notifier_block *nb, unsigned long val,
>  	u64 spa, spa_alias;
>  	unsigned long pfn;
>  
> -	if (!mce || !mce_usable_address(mce))
> +	if (!mce)
> +		return NOTIFY_DONE;
> +
> +	if (!mce_is_memory_error(mce) || mce_is_correctable(mce))
> +		return NOTIFY_DONE;

[Severity: High]
Does this new check bypass CXL MCE handling for poison consumption errors
on AMD architectures?

On AMD, mce_is_memory_error() maps to amd_mce_is_memory_error(), which
evaluates to true strictly for SMCA Unified Memory Controller (UMC) banks or
legacy northbridge banks.

When a CXL poison consumption error occurs, it is logged in the consuming core
unit (e.g., Load/Store or Data Fabric) with the MCI_STATUS_POISON bit set,
which is not a UMC bank. This means mce_is_memory_error() will evaluate to
false, causing this handler to return NOTIFY_DONE and silently skip offlining
the aliased page.

Previously, relying on mce_usable_address() correctly identified these errors
since amd_mce_usable_address() checks for the MCI_STATUS_POISON bit on all
non-northbridge banks. Could this lead to data corruption or repeated machine
checks when the corrupted memory is re-accessed?

> +
> +	if (!mce_usable_address(mce))
>  		return NOTIFY_DONE;
>  
>  	spa = mce->addr & MCI_ADDR_PHYSADDR;

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260812151759.14390-1-shaikhkamal2012@gmail.com?part=1