[RFC PATCH] x86/mce: Avoid arming periodic polling timer on isolated CPUs

[RFC PATCH] x86/mce: Avoid arming periodic polling timer on isolated CPUs

[Aaron Tomlin]

Latency-sensitive workloads rely on CPU isolation (i.e., configured via
nohz_full= or isolcpus=nohz) to guarantee deterministic execution
without interruption from background kernel activity.

On x86 platforms, the Machine Check Architecture (MCA) subsystem arms a
per-CPU, pinned standard timer (mce_timer) to periodically poll hardware
banks for "silent" corrected machine check errors. Because mce_timer is
pinned to the local CPU via TIMER_PINNED, the timer core cannot migrate
its expiration to a housekeeping CPU. Consequently, every check_interval
(defaulting to five minutes, or as frequently as every 10 ms during
error decay), a timer tick interrupts the isolated CPU to execute
mce_timer_fn() and machine_check_poll(), introducing unavoidable latency
jitter.

Presently, the only mechanisms available to suppress this timer (i.e.,
the mce=ignore_ce boot parameter or setting check_interval=0 via sysfs)
are system-wide globals. Setting these attributes disables polling
across the entire platform, needlessly blinding non-isolated
housekeeping CPUs from monitoring shared memory controllers and uncore
error telemetry.

Amend should_enable_timer() to query housekeeping_cpu() for
HK_TYPE_TIMER (HK_TYPE_KERNEL_NOISE). When a CPU is designated as
isolated, the timer is neither armed at CPU online nor re-armed upon
timer expiration. Housekeeping CPUs continue to run their polling timers
unaltered, preserving routine monitoring of shared package and memory
controller banks.

On isolated CPUs, critical synchronous exceptions (#MC) remain fully
functional for fatal and recoverable uncorrected errors, and CMCI
interrupts continue to deliver asynchronous notifications where
supported. When CPU isolation is not configured, static branch
optimisations ensure zero runtime overhead.

Signed-off-by: Aaron Tomlin <atomlin@atomlin.com>
---
 arch/x86/kernel/cpu/mce/core.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/arch/x86/kernel/cpu/mce/core.c b/arch/x86/kernel/cpu/mce/core.c
index ab469605fc89..fd047968ad17 100644
--- a/arch/x86/kernel/cpu/mce/core.c
+++ b/arch/x86/kernel/cpu/mce/core.c
@@ -25,6 +25,7 @@
 #include <linux/delay.h>
 #include <linux/ctype.h>
 #include <linux/sched.h>
+#include <linux/sched/isolation.h>
 #include <linux/sysfs.h>
 #include <linux/types.h>
 #include <linux/slab.h>
@@ -1759,6 +1760,9 @@ void (*mc_poll_banks)(void) = mc_poll_banks_default;
 
 static bool should_enable_timer(unsigned long iv)
 {
+	if (!housekeeping_cpu(smp_processor_id(), HK_TYPE_TIMER))
+		return false;
+
 	return !mca_cfg.ignore_ce && iv;
 }
 
-- 
2.55.0

Re: [RFC PATCH] x86/mce: Avoid arming periodic polling timer on isolated CPUs

[Luck, Tony]

On Tue, Sep 01, 2026 at 11:11:38AM -0400, Aaron Tomlin wrote:
> Latency-sensitive workloads rely on CPU isolation (i.e., configured via
> nohz_full= or isolcpus=nohz) to guarantee deterministic execution
> without interruption from background kernel activity.
> 
> On x86 platforms, the Machine Check Architecture (MCA) subsystem arms a
> per-CPU, pinned standard timer (mce_timer) to periodically poll hardware
> banks for "silent" corrected machine check errors. Because mce_timer is
> pinned to the local CPU via TIMER_PINNED, the timer core cannot migrate
> its expiration to a housekeeping CPU. Consequently, every check_interval
> (defaulting to five minutes, or as frequently as every 10 ms during
> error decay), a timer tick interrupts the isolated CPU to execute
> mce_timer_fn() and machine_check_poll(), introducing unavoidable latency
> jitter.
> 
> Presently, the only mechanisms available to suppress this timer (i.e.,
> the mce=ignore_ce boot parameter or setting check_interval=0 via sysfs)
> are system-wide globals. Setting these attributes disables polling
> across the entire platform, needlessly blinding non-isolated
> housekeeping CPUs from monitoring shared memory controllers and uncore
> error telemetry.
> 
> Amend should_enable_timer() to query housekeeping_cpu() for
> HK_TYPE_TIMER (HK_TYPE_KERNEL_NOISE). When a CPU is designated as
> isolated, the timer is neither armed at CPU online nor re-armed upon
> timer expiration. Housekeeping CPUs continue to run their polling timers
> unaltered, preserving routine monitoring of shared package and memory
> controller banks.
> 
> On isolated CPUs, critical synchronous exceptions (#MC) remain fully
> functional for fatal and recoverable uncorrected errors, and CMCI
> interrupts continue to deliver asynchronous notifications where
> supported. When CPU isolation is not configured, static branch
> optimisations ensure zero runtime overhead.

There's a better approach which hasn't managed to get to the top of my
priority queue.

If you look at the mce_poll_banks bitmask, you'll likely see that the
only bit set in the range of supported banks is for bank 4 or bank 6.
That's the "PCU" (Power Control Unit?) bank. All the other bits are
zeroed because Linux found that CMCI was supported, so no polling is
needed (assuming CMCI hasn't been disabled).

There was a misunderstanding about the full meaning of the CMCI_EN
bit in the IA32_MCi_CTL2 MSR. Older versions of the Intel Software
Developer's Manual left Linux coders with the impression that if
CMCI_EN was not supported for a bank, then software would have to
poll to see any corrected errors.

Newer editions update volume 3B section 18.5 "CORRECTED MACHINE CHECK ERROR INTERRUPT":

  To detect the existence of thresholding for a given bank, software writes only
  bits 14:0 with the threshold value. If the bits persist, then thresholding
  is available (and CMCI is available). If the bits are all 0's, then no
  thresholding exists. To detect that CMCI signaling exists, software writes
  a 1 to bit 30 of the MCi_CTL2 register. Upon subsequent read, if bit 30 =
  0, no CMCI is available for this bank and no corrected or UCNA errors will
  be reported on this bank.  If bit 30 = 1, then CMCI is available and enabled.

The addition being the "and no corrected or UCNA errors will be reported on this bank"

The PCU bank behaves this way. It doesn't support CMCI because:

  *NO CORRECTED ERRORS WILL BE REPORTED IN THE PCU BANK*

so a waste of cycles to poll it.

So Linux needs:

1) Properly handle systems that support CMCI, but have bank(s) that don't.
   The bit for such banks should be cleared in mce_poll_banks.
2) Add a check to see if mce_poll_banks is empty, don't start the timer.

> 
> Signed-off-by: Aaron Tomlin <atomlin@atomlin.com>
> ---
>  arch/x86/kernel/cpu/mce/core.c | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/arch/x86/kernel/cpu/mce/core.c b/arch/x86/kernel/cpu/mce/core.c
> index ab469605fc89..fd047968ad17 100644
> --- a/arch/x86/kernel/cpu/mce/core.c
> +++ b/arch/x86/kernel/cpu/mce/core.c
> @@ -25,6 +25,7 @@
>  #include <linux/delay.h>
>  #include <linux/ctype.h>
>  #include <linux/sched.h>
> +#include <linux/sched/isolation.h>
>  #include <linux/sysfs.h>
>  #include <linux/types.h>
>  #include <linux/slab.h>
> @@ -1759,6 +1760,9 @@ void (*mc_poll_banks)(void) = mc_poll_banks_default;
>  
>  static bool should_enable_timer(unsigned long iv)
>  {
> +	if (!housekeeping_cpu(smp_processor_id(), HK_TYPE_TIMER))
> +		return false;
> +
>  	return !mca_cfg.ignore_ce && iv;
>  }
>  
> -- 
> 2.55.0
> 

-Tony

Re: [RFC PATCH] x86/mce: Avoid arming periodic polling timer on isolated CPUs

[Aaron Tomlin]

On Tue, Sep 01, 2026 at 10:58:52AM -0700, Luck, Tony wrote:
> On Tue, Sep 01, 2026 at 11:11:38AM -0400, Aaron Tomlin wrote:
> > Latency-sensitive workloads rely on CPU isolation (i.e., configured via
> > nohz_full= or isolcpus=nohz) to guarantee deterministic execution
> > without interruption from background kernel activity.
> > 
> > On x86 platforms, the Machine Check Architecture (MCA) subsystem arms a
> > per-CPU, pinned standard timer (mce_timer) to periodically poll hardware
> > banks for "silent" corrected machine check errors. Because mce_timer is
> > pinned to the local CPU via TIMER_PINNED, the timer core cannot migrate
> > its expiration to a housekeeping CPU. Consequently, every check_interval
> > (defaulting to five minutes, or as frequently as every 10 ms during
> > error decay), a timer tick interrupts the isolated CPU to execute
> > mce_timer_fn() and machine_check_poll(), introducing unavoidable latency
> > jitter.
> > 
> > Presently, the only mechanisms available to suppress this timer (i.e.,
> > the mce=ignore_ce boot parameter or setting check_interval=0 via sysfs)
> > are system-wide globals. Setting these attributes disables polling
> > across the entire platform, needlessly blinding non-isolated
> > housekeeping CPUs from monitoring shared memory controllers and uncore
> > error telemetry.
> > 
> > Amend should_enable_timer() to query housekeeping_cpu() for
> > HK_TYPE_TIMER (HK_TYPE_KERNEL_NOISE). When a CPU is designated as
> > isolated, the timer is neither armed at CPU online nor re-armed upon
> > timer expiration. Housekeeping CPUs continue to run their polling timers
> > unaltered, preserving routine monitoring of shared package and memory
> > controller banks.
> > 
> > On isolated CPUs, critical synchronous exceptions (#MC) remain fully
> > functional for fatal and recoverable uncorrected errors, and CMCI
> > interrupts continue to deliver asynchronous notifications where
> > supported. When CPU isolation is not configured, static branch
> > optimisations ensure zero runtime overhead.
> 
> There's a better approach which hasn't managed to get to the top of my
> priority queue.
> 
> If you look at the mce_poll_banks bitmask, you'll likely see that the
> only bit set in the range of supported banks is for bank 4 or bank 6.
> That's the "PCU" (Power Control Unit?) bank. All the other bits are
> zeroed because Linux found that CMCI was supported, so no polling is
> needed (assuming CMCI hasn't been disabled).
> 
> There was a misunderstanding about the full meaning of the CMCI_EN
> bit in the IA32_MCi_CTL2 MSR. Older versions of the Intel Software
> Developer's Manual left Linux coders with the impression that if
> CMCI_EN was not supported for a bank, then software would have to
> poll to see any corrected errors.
> 
> Newer editions update volume 3B section 18.5 "CORRECTED MACHINE CHECK ERROR INTERRUPT":
> 
>   To detect the existence of thresholding for a given bank, software writes only
>   bits 14:0 with the threshold value. If the bits persist, then thresholding
>   is available (and CMCI is available). If the bits are all 0's, then no
>   thresholding exists. To detect that CMCI signaling exists, software writes
>   a 1 to bit 30 of the MCi_CTL2 register. Upon subsequent read, if bit 30 =
>   0, no CMCI is available for this bank and no corrected or UCNA errors will
>   be reported on this bank.  If bit 30 = 1, then CMCI is available and enabled.
> 
> The addition being the "and no corrected or UCNA errors will be reported on this bank"
> 
> The PCU bank behaves this way. It doesn't support CMCI because:
> 
>   *NO CORRECTED ERRORS WILL BE REPORTED IN THE PCU BANK*
> 
> so a waste of cycles to poll it.
> 
> So Linux needs:
> 
> 1) Properly handle systems that support CMCI, but have bank(s) that don't.
>    The bit for such banks should be cleared in mce_poll_banks.
> 2) Add a check to see if mce_poll_banks is empty, don't start the timer.

Hi Tony,

Thank you for your feedback.

I see. Addressing the root cause by not polling banks that will never log
corrected/Uncorrected No Action Required (UCNA) errors, and subsequently
eliding mce_timer whenever mce_poll_banks is empty, is indeed a far cleaner
and more elegant solution. It eliminates the polling overhead across the
entire system rather than merely masking it on isolated cores.

Based on your suggestions, the implementation would look like:

    1.  When CMCI is supported on the CPU but a given bank does not support
        setting MCI_CTL2_CMCI_EN (i.e., the PCU bank), clear that bank's
        bit from mce_poll_banks instead of leaving it set and flagging it
        as poll_only.

    2.  Add a check (e.g., bitmap_empty(this_cpu_ptr(mce_poll_banks),
        MAX_NR_BANKS)) so that the timer is never armed or re-armed when no
        banks require polling.

This also integrates cleanly with the existing storm tracking logic. When a
storm begins, cmci_storm_begin() sets the bank's bit in mce_poll_banks and
kicks the timer; once the storm concludes, cmci_storm_end() clears the bit
and the timer ceases re-arming. Legacy systems lacking CMCI (or systems
booted with mce=no_cmci) will continue to retain bits in mce_poll_banks and
poll as expected.

I will prepare a v2 patch implementing this approach and post it shortly
for review.


Kind regards,
-- 
Aaron Tomlin