[PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[lirongqing]

From: Li RongQing <lirongqing@baidu.com>

Since commit 011d82611172 ("RAS: Add a Corrected Errors Collector"),
mce_notify_irq() in should_enable_timer() always returns false even
when an MCE event is logged, because bit 0 of mce_need_notify is not
set in mce_log. This prevents the timer interval from being properly
adjusted.

Fix this by modifying machine_check_poll() to return a boolean indicating
whether an MCE was logged, and update mc_poll_banks() to propagate this
return value. The timer interval logic in mce_timer_fn() now uses this
return value directly instead of relying on mce_notify_irq().

Fixes: 011d82611172 ("RAS: Add a Corrected Errors Collector")
Signed-off-by: Li RongQing <lirongqing@baidu.com>
---
 arch/x86/include/asm/mce.h         |  2 +-
 arch/x86/kernel/cpu/mce/core.c     | 17 +++++++++++------
 arch/x86/kernel/cpu/mce/intel.c    |  8 ++++++--
 arch/x86/kernel/cpu/mce/internal.h |  2 +-
 4 files changed, 19 insertions(+), 10 deletions(-)

diff --git a/arch/x86/include/asm/mce.h b/arch/x86/include/asm/mce.h
index 2d98886..fb9eab4 100644
--- a/arch/x86/include/asm/mce.h
+++ b/arch/x86/include/asm/mce.h
@@ -303,7 +303,7 @@ enum mcp_flags {
 	MCP_QUEUE_LOG	= BIT(2),	/* only queue to genpool */
 };
 
-void machine_check_poll(enum mcp_flags flags, mce_banks_t *b);
+bool machine_check_poll(enum mcp_flags flags, mce_banks_t *b);
 
 DECLARE_PER_CPU(struct mce, injectm);
 
diff --git a/arch/x86/kernel/cpu/mce/core.c b/arch/x86/kernel/cpu/mce/core.c
index 3444002..8d42691 100644
--- a/arch/x86/kernel/cpu/mce/core.c
+++ b/arch/x86/kernel/cpu/mce/core.c
@@ -813,10 +813,11 @@ static void clear_bank(struct mce *m)
  * is already totally * confused. In this case it's likely it will
  * not fully execute the machine check handler either.
  */
-void machine_check_poll(enum mcp_flags flags, mce_banks_t *b)
+bool machine_check_poll(enum mcp_flags flags, mce_banks_t *b)
 {
 	struct mce_bank *mce_banks = this_cpu_ptr(mce_banks_array);
 	struct mce_hw_err err;
+	bool logged = false;
 	struct mce *m;
 	int i;
 
@@ -868,6 +869,7 @@ void machine_check_poll(enum mcp_flags flags, mce_banks_t *b)
 		else
 			mce_log(&err);
 
+		logged = true;
 clear_it:
 		clear_bank(m);
 	}
@@ -878,6 +880,8 @@ void machine_check_poll(enum mcp_flags flags, mce_banks_t *b)
 	 */
 
 	sync_core();
+
+	return logged;
 }
 EXPORT_SYMBOL_GPL(machine_check_poll);
 
@@ -1776,12 +1780,12 @@ static void __start_timer(struct timer_list *t, unsigned long interval)
 	local_irq_restore(flags);
 }
 
-static void mc_poll_banks_default(void)
+static bool mc_poll_banks_default(void)
 {
-	machine_check_poll(0, this_cpu_ptr(&mce_poll_banks));
+	return machine_check_poll(0, this_cpu_ptr(&mce_poll_banks));
 }
 
-void (*mc_poll_banks)(void) = mc_poll_banks_default;
+bool (*mc_poll_banks)(void) = mc_poll_banks_default;
 
 static bool should_enable_timer(unsigned long iv)
 {
@@ -1792,19 +1796,20 @@ static void mce_timer_fn(struct timer_list *t)
 {
 	struct timer_list *cpu_t = this_cpu_ptr(&mce_timer);
 	unsigned long iv;
+	bool logged = false;
 
 	WARN_ON(cpu_t != t);
 
 	iv = __this_cpu_read(mce_next_interval);
 
 	if (mce_available(this_cpu_ptr(&cpu_info)))
-		mc_poll_banks();
+		logged = mc_poll_banks();
 
 	/*
 	 * Alert userspace if needed. If we logged an MCE, reduce the polling
 	 * interval, otherwise increase the polling interval.
 	 */
-	if (mce_notify_irq())
+	if (logged)
 		iv = max(iv / 2, (unsigned long) HZ/100);
 	else
 		iv = min(iv * 2, round_jiffies_relative(check_interval * HZ));
diff --git a/arch/x86/kernel/cpu/mce/intel.c b/arch/x86/kernel/cpu/mce/intel.c
index 4655223..a3d2730 100644
--- a/arch/x86/kernel/cpu/mce/intel.c
+++ b/arch/x86/kernel/cpu/mce/intel.c
@@ -395,11 +395,15 @@ void cmci_disable_bank(int bank)
 }
 
 /* Bank polling function when CMCI is disabled. */
-static void cmci_mc_poll_banks(void)
+static bool cmci_mc_poll_banks(void)
 {
+	bool logged;
+
 	spin_lock(&cmci_poll_lock);
-	machine_check_poll(0, this_cpu_ptr(&mce_poll_banks));
+	logged = machine_check_poll(0, this_cpu_ptr(&mce_poll_banks));
 	spin_unlock(&cmci_poll_lock);
+
+	return logged;
 }
 
 void intel_init_cmci(void)
diff --git a/arch/x86/kernel/cpu/mce/internal.h b/arch/x86/kernel/cpu/mce/internal.h
index a31cf98..7bf2360 100644
--- a/arch/x86/kernel/cpu/mce/internal.h
+++ b/arch/x86/kernel/cpu/mce/internal.h
@@ -348,5 +348,5 @@ static __always_inline u32 mca_msr_reg(int bank, enum mca_msr reg)
 	return 0;
 }
 
-extern void (*mc_poll_banks)(void);
+extern bool (*mc_poll_banks)(void);
 #endif /* __X86_MCE_INTERNAL_H__ */
-- 
2.9.4

Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Nikolay Borisov]

On 12.01.26 г. 10:27 ч., lirongqing wrote:
> From: Li RongQing <lirongqing@baidu.com>
> 
> Since commit 011d82611172 ("RAS: Add a Corrected Errors Collector"),
> mce_notify_irq() in should_enable_timer() always returns false even

should_enable_timer doesn't call mce_notify_irq

> when an MCE event is logged, because bit 0 of mce_need_notify is not
> set in mce_log. This prevents the timer interval from being properly
> adjusted.
> 
> Fix this by modifying machine_check_poll() to return a boolean indicating
> whether an MCE was logged, and update mc_poll_banks() to propagate this
> return value. The timer interval logic in mce_timer_fn() now uses this
> return value directly instead of relying on mce_notify_irq().

This warrants a bit more explanation why it's correct. Because 
mce_notify_irq is really a misnomer, it will ideally be named 
mce_notify_user(). That function is called from 2 places:

1. Early notifier block, but there it's guaranteed to do the right thing 
as mce_need_notify is explicitly set.

2. From the timer function, where as you have pointed out 
mce_need_notify is never set by the polling code, hence the function is 
a noop. But actually calling mce_log() processes all logged errors in 
the gen pool and that is processed by calling the x86_mce_decoder_chain 
which will DTRT w.r.t to mce_notify_irq since the early notifier will be 
called from there.

<snip>

答复: [外部邮件] Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Li,Rongqing]

> 
> On 12.01.26 г. 10:27 ч., lirongqing wrote:
> > From: Li RongQing <lirongqing@baidu.com>
> >
> > Since commit 011d82611172 ("RAS: Add a Corrected Errors Collector"),
> > mce_notify_irq() in should_enable_timer() always returns false even
> 
> should_enable_timer doesn't call mce_notify_irq
> 
> > when an MCE event is logged, because bit 0 of mce_need_notify is not
> > set in mce_log. This prevents the timer interval from being properly
> > adjusted.
> >
> > Fix this by modifying machine_check_poll() to return a boolean
> > indicating whether an MCE was logged, and update mc_poll_banks() to
> > propagate this return value. The timer interval logic in
> > mce_timer_fn() now uses this return value directly instead of relying on
> mce_notify_irq().
> 
> This warrants a bit more explanation why it's correct. Because mce_notify_irq is
> really a misnomer, it will ideally be named mce_notify_user(). That function is
> called from 2 places:
> 
> 1. Early notifier block, but there it's guaranteed to do the right thing as
> mce_need_notify is explicitly set.
> 
> 2. From the timer function, where as you have pointed out mce_need_notify is
> never set by the polling code, hence the function is a noop. But actually calling
> mce_log() processes all logged errors in the gen pool and that is processed by
> calling the x86_mce_decoder_chain which will DTRT w.r.t to mce_notify_irq
> since the early notifier will be called from there.
> 
> <snip>

Ok, I will add more explanation, and rename mce_notify_irq() as mce_notify_user();

Thanks

-Li

Re: 答复: [外部邮件] Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Borislav Petkov]

On Mon, Jan 12, 2026 at 09:36:21AM +0000, Li,Rongqing wrote:
> Ok, I will add more explanation, and rename mce_notify_irq() as
> mce_notify_user();

No, first you should explain what you're fixing here and why.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

答复: 答复: [外部邮件] Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Li,Rongqing]

> On Mon, Jan 12, 2026 at 09:36:21AM +0000, Li,Rongqing wrote:
> > Ok, I will add more explanation, and rename mce_notify_irq() as
> > mce_notify_user();
> 
> No, first you should explain what you're fixing here and why.
> 
> --
> Regards/Gruss,
>     Boris.
> 

How about modifying the changelog as follows


    x86/mce: Fix timer interval adjustment after logging a MCE event

    Since commit 011d82611172 ("RAS: Add a Corrected Errors Collector"),
    mce_timer_fn() has incorrectly determined whether to adjust the
    timer interval. The issue arises because mce_notify_irq() now always
    returns false when called from the timer path, since the polling code
    never sets bit 0 of mce_need_notify. This prevents proper adjustment of
    the timer interval based on whether MCE events were logged.

    The mce_notify_irq() is called from two contexts:
    1. Early notifier block - correctly sets mce_need_notify
    2. Timer function - never sets mce_need_notify, making it a noop
       (though logged errors are still processed through mce_log()->
        x86_mce_decoder_chain -> early notifier).

    Fix this by modifying machine_check_poll() to return a boolean indicating
    whether any MCE was logged, and updating mc_poll_banks() and related
    functions to propagate this return value. Then, mce_timer_fn() can use
    this direct return value instead of relying on mce_notify_irq() for
    timer interval decisions.

    This ensures the timer interval is correctly reduced when MCE events are
    logged and increased when no events occur.

    Fixes: 011d82611172 ("RAS: Add a Corrected Errors Collector")
> https://people.kernel.org/tglx/notes-about-netiquette

Re: 答复: 答复: [外部邮件] Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Borislav Petkov]

On Mon, Jan 12, 2026 at 10:24:11AM +0000, Li,Rongqing wrote:
>     Since commit 011d82611172 ("RAS: Add a Corrected Errors Collector"),
>     mce_timer_fn() has incorrectly determined whether to adjust the
>     timer interval. The issue arises because mce_notify_irq() now always
>     returns false when called from the timer path, since the polling code
>     never sets bit 0 of mce_need_notify. This prevents proper adjustment of
>     the timer interval based on whether MCE events were logged.

That's because you missed the main point of the error collector:

    "The error decoding is done with the decoding chain now and
    mce_first_notifier() gets to see the error first and the CEC decides
    whether to log it and then the rest of the chain doesn't hear about it -
    						     ^^^^^^^^^^^^^^^^^^^^^^

    basically the main reason for the CE collector - or to continue running
    the notifiers."

So lemme ask you again: what are you really fixing?!

What is the actual problem you're trying to fix?

And do not send me another revised commit message.

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: 答复: 答复: 答复: [外部邮件] Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Borislav Petkov]

On Tue, Jan 13, 2026 at 12:37:02PM +0000, Li,Rongqing wrote:
> The comment in mce_timer_fn says to adjust the polling interval, but
> I notice the kernel log always shows an MCE log every 5 minutes. Is this
> normal?

Use git annotate to figure out which patch added this comment and in context
of what and that'll tell you why.

As to the 5 minutes, look at how the check interval gets established.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

RE: 答复: 答复: 答复: [外部邮件] Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Luck, Tony]

> > The comment in mce_timer_fn says to adjust the polling interval, but
> > I notice the kernel log always shows an MCE log every 5 minutes. Is this
> > normal?
>
> Use git annotate to figure out which patch added this comment and in context
> of what and that'll tell you why.
>
> As to the 5 minutes, look at how the check interval gets established.

Once upon a time the polling interval started out at 5 minutes, but the
interval was halved each time an error was found (so interval went
150s, 75s, 37s, ... down to 1s). If no error was found, then the interval
was doubled (going back up to 300s).

This is described in the comment:

        /*
         * Alert userspace if needed. If we logged an MCE, reduce the polling
         * interval, otherwise increase the polling interval.
         */

It seems that the kernel isn't doing that today. Polling at a fixed 300 seconds
event though errors are being found and logged. Interesting that the timestamps
are 327.68 seconds apart, rather than 300 and change. So there is some strange
stuff going on.

I can reproduce here on an Icelake system. Booted with mce=no_cmci to force polling
(and turned of BIOS firmware first mode).  Injecting an error every 30 seconds I also see
constant 327 seconds between logs (multiple logs show up because my injection picks memory
channel "randomly", so there can be several banks with errors when polling happens).

$ dmesg | grep 'Machine Check Event:'
[  662.632988] EDAC skx MC4: CPU 40: Machine Check Event: 0x0 Bank 13: 0x8c00014200800090
[  662.727377] EDAC skx MC6: CPU 40: Machine Check Event: 0x0 Bank 21: 0x8c0000c200800090
[  990.283484] EDAC skx MC4: CPU 121: Machine Check Event: 0x0 Bank 13: 0x8c00010200800090
[  990.378233] EDAC skx MC6: CPU 121: Machine Check Event: 0x0 Bank 21: 0x8c00014200800090
[  990.467199] EDAC skx MC0: CPU 3: Machine Check Event: 0x0 Bank 13: 0x8c00004200800090
[ 1317.939260] EDAC skx MC4: CPU 122: Machine Check Event: 0x0 Bank 13: 0x8c00010200800090
[ 1318.033721] EDAC skx MC6: CPU 122: Machine Check Event: 0x0 Bank 21: 0x8c00010200800090
[ 1318.122612] EDAC skx MC0: CPU 14: Machine Check Event: 0x0 Bank 13: 0x8c00004200800090
[ 1318.211507] EDAC skx MC2: CPU 14: Machine Check Event: 0x0 Bank 21: 0x8c00004200800090
[ 1645.590773] EDAC skx MC4: CPU 129: Machine Check Event: 0x0 Bank 13: 0x8c00010200800090
[ 1645.685153] EDAC skx MC6: CPU 129: Machine Check Event: 0x0 Bank 21: 0x8c00018200800090
[ 1645.773744] EDAC skx MC0: CPU 100: Machine Check Event: 0x0 Bank 13: 0x8c00004200800090

-Tony

Re: 答复: 答复: 答复: [外部邮件] Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Nikolay Borisov]

On 13.01.26 г. 20:53 ч., Luck, Tony wrote:
>>> The comment in mce_timer_fn says to adjust the polling interval, but
>>> I notice the kernel log always shows an MCE log every 5 minutes. Is this
>>> normal?
>>
>> Use git annotate to figure out which patch added this comment and in context
>> of what and that'll tell you why.
>>
>> As to the 5 minutes, look at how the check interval gets established.
> 
> Once upon a time the polling interval started out at 5 minutes, but the
> interval was halved each time an error was found (so interval went
> 150s, 75s, 37s, ... down to 1s). If no error was found, then the interval
> was doubled (going back up to 300s).
> 
> This is described in the comment:
> 
>          /*
>           * Alert userspace if needed. If we logged an MCE, reduce the polling
>           * interval, otherwise increase the polling interval.
>           */
> 
> It seems that the kernel isn't doing that today. Polling at a fixed 300 seconds
> event though errors are being found and logged. Interesting that the timestamps
> are 327.68 seconds apart, rather than 300 and change. So there is some strange
> stuff going on.
> 
> I can reproduce here on an Icelake system. Booted with mce=no_cmci to force polling
> (and turned of BIOS firmware first mode).  Injecting an error every 30 seconds I also see
> constant 327 seconds between logs (multiple logs show up because my injection picks memory
> channel "randomly", so there can be several banks with errors when polling happens).
> 
> $ dmesg | grep 'Machine Check Event:'
> [  662.632988] EDAC skx MC4: CPU 40: Machine Check Event: 0x0 Bank 13: 0x8c00014200800090
> [  662.727377] EDAC skx MC6: CPU 40: Machine Check Event: 0x0 Bank 21: 0x8c0000c200800090
> [  990.283484] EDAC skx MC4: CPU 121: Machine Check Event: 0x0 Bank 13: 0x8c00010200800090
> [  990.378233] EDAC skx MC6: CPU 121: Machine Check Event: 0x0 Bank 21: 0x8c00014200800090
> [  990.467199] EDAC skx MC0: CPU 3: Machine Check Event: 0x0 Bank 13: 0x8c00004200800090
> [ 1317.939260] EDAC skx MC4: CPU 122: Machine Check Event: 0x0 Bank 13: 0x8c00010200800090
> [ 1318.033721] EDAC skx MC6: CPU 122: Machine Check Event: 0x0 Bank 21: 0x8c00010200800090
> [ 1318.122612] EDAC skx MC0: CPU 14: Machine Check Event: 0x0 Bank 13: 0x8c00004200800090
> [ 1318.211507] EDAC skx MC2: CPU 14: Machine Check Event: 0x0 Bank 21: 0x8c00004200800090
> [ 1645.590773] EDAC skx MC4: CPU 129: Machine Check Event: 0x0 Bank 13: 0x8c00010200800090
> [ 1645.685153] EDAC skx MC6: CPU 129: Machine Check Event: 0x0 Bank 21: 0x8c00018200800090
> [ 1645.773744] EDAC skx MC0: CPU 100: Machine Check Event: 0x0 Bank 13: 0x8c00004200800090
> 
> -Tony
> 

At this stage I think lirongqi's patch is ok, but in the long run (i.e 
tomorrow) I will send a patch that simply eliminates mce_notify_irq's 
call in mce_timer_fn. I.e that function should be called only from the 
early notifier.

Re: 答复: 答复: 答复: [外部邮件] Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Borislav Petkov]

On Tue, Jan 13, 2026 at 06:53:13PM +0000, Luck, Tony wrote:
> Once upon a time the polling interval started out at 5 minutes, but the
> interval was halved each time an error was found (so interval went
> 150s, 75s, 37s, ... down to 1s). If no error was found, then the interval
> was doubled (going back up to 300s).
> 
> This is described in the comment:
> 
>         /*
>          * Alert userspace if needed. If we logged an MCE, reduce the polling
>          * interval, otherwise increase the polling interval.
>          */
> 
> It seems that the kernel isn't doing that today.

So the halving is not happening anymore because mce_notify_irq() is not
returning true.

I guess we botched that somewhere along the way...

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: 答复: 答复: 答复: [外部邮件] Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Borislav Petkov]

On Tue, Jan 13, 2026 at 08:55:08PM +0200, Nikolay Borisov wrote:
> tomorrow) I will send a patch that simply eliminates mce_notify_irq's call
> in mce_timer_fn. I.e that function should be called only from the early
> notifier.

You still need to know whether to halve the timeout or not. And that's that
mce_need_notify thing. And that gets called in the early notifier so it should
work.

However, it would be good to be able to bisect this and find a minimal fix to
backport...

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: 答复: 答复: 答复: [外部邮件] Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Nikolay Borisov]

On 13.01.26 г. 21:13 ч., Borislav Petkov wrote:
> On Tue, Jan 13, 2026 at 08:55:08PM +0200, Nikolay Borisov wrote:
>> tomorrow) I will send a patch that simply eliminates mce_notify_irq's call
>> in mce_timer_fn. I.e that function should be called only from the early
>> notifier.
> 
> You still need to know whether to halve the timeout or not. And that's that
> mce_need_notify thing. And that gets called in the early notifier so it should
> work.
> 
> However, it would be good to be able to bisect this and find a minimal fix to
> backport...
> 

mce_need_notify even now works just in the early notifier because it's 
set in mce_early_notifier() and subsequently reset in mce_notify_irq() 
which is called immediately after the set.

Re: 答复: 答复: 答复: [外部邮件] Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Nikolay Borisov]

On 13.01.26 г. 20:53 ч., Luck, Tony wrote:
>>> The comment in mce_timer_fn says to adjust the polling interval, but
>>> I notice the kernel log always shows an MCE log every 5 minutes. Is this
>>> normal?
>>
>> Use git annotate to figure out which patch added this comment and in context
>> of what and that'll tell you why.
>>
>> As to the 5 minutes, look at how the check interval gets established.
> 
> Once upon a time the polling interval started out at 5 minutes, but the
> interval was halved each time an error was found (so interval went
> 150s, 75s, 37s, ... down to 1s). If no error was found, then the interval
> was doubled (going back up to 300s).
> 
> This is described in the comment:
> 
>          /*
>           * Alert userspace if needed. If we logged an MCE, reduce the polling
>           * interval, otherwise increase the polling interval.
>           */
> 
> It seems that the kernel isn't doing that today. Polling at a fixed 300 seconds
> event though errors are being found and logged. Interesting that the timestamps
> are 327.68 seconds apart, rather than 300 and change. So there is some strange
> stuff going on.

I think Li Rongqing patch does exactly that, since it predicates the 
halving/doubling of the interval based on whether an error was found and 
not whether it was reported to user space (what mce_notify_irq() ) does. 
Both concepts seems to be independent and the former being the core one 
we care about w.r.t to the decision how to adjust the interval, no ?

<snip>

Re: 答复: 答复: 答复: [外部邮件] Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Borislav Petkov]

On Tue, Jan 13, 2026 at 09:25:19PM +0200, Nikolay Borisov wrote:
> mce_need_notify even now works just in the early notifier because it's set
> in mce_early_notifier() and subsequently reset in mce_notify_irq() which is
> called immediately after the set.

You still need to call something in mce_timer_fn() to know whether to halve
the interval or not.

I wonder if it would be better/cleaner if mce_early_notifier() halves the
interval and mce_timer_fn() just reads only the current interval value...
maybe that'll be less logic spread around...

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: 答复: 答复: 答复: [外部邮件] Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Nikolay Borisov]

On 13.01.26 г. 21:33 ч., Borislav Petkov wrote:
> On Tue, Jan 13, 2026 at 09:25:19PM +0200, Nikolay Borisov wrote:
>> mce_need_notify even now works just in the early notifier because it's set
>> in mce_early_notifier() and subsequently reset in mce_notify_irq() which is
>> called immediately after the set.
> 
> You still need to call something in mce_timer_fn() to know whether to halve
> the interval or not.

Yes, and why is Li's approach not working, i.e if mc_poll_banks() 
returns a value signalling "i found an MCE" we halve, otherwise we 
double it?


> 
> I wonder if it would be better/cleaner if mce_early_notifier() halves the
> interval and mce_timer_fn() just reads only the current interval value...
> maybe that'll be less logic spread around...
> 

That's certainly doable, but why should the interval setting be coupled 
to the initial notifier and not to mc_poll_banks?

Re: 答复: 答复: 答复: [外部邮件] Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Borislav Petkov]

On Tue, Jan 13, 2026 at 09:37:13PM +0200, Nikolay Borisov wrote:
> Yes, and why is Li's approach not working, i.e if mc_poll_banks() returns a
> value signalling "i found an MCE" we halve, otherwise we double it?

It might be "working" but I don't like that "bool logged" thing and
marshalling it back'n'forth. This should be waaay simpler.

> That's certainly doable, but why should the interval setting be coupled to
> the initial notifier and not to mc_poll_banks?

What's the difference who notifies the timer? The banks polling will call the
notifier if it finds an error to log.

But before we do any of that, we need to figure out what commit broke this so
that we fix stable.

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: 答复: 答复: 答复: [外部邮件] Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Nikolay Borisov]

On 13.01.26 г. 21:44 ч., Borislav Petkov wrote:
> On Tue, Jan 13, 2026 at 09:37:13PM +0200, Nikolay Borisov wrote:
>> Yes, and why is Li's approach not working, i.e if mc_poll_banks() returns a
>> value signalling "i found an MCE" we halve, otherwise we double it?
> 
> It might be "working" but I don't like that "bool logged" thing and
> marshalling it back'n'forth. This should be waaay simpler.
> 
>> That's certainly doable, but why should the interval setting be coupled to
>> the initial notifier and not to mc_poll_banks?
> 
> What's the difference who notifies the timer? The banks polling will call the
> notifier if it finds an error to log.
> 
> But before we do any of that, we need to figure out what commit broke this so
> that we fix stable.
> 
> Thx.
> 


grepping around points to your 011d82611172 ("RAS: Add a Corrected 
Errors Collector"). Because looking at the commit log of my 6447828875b7 
("x86/mce/inject: Remove call to mce_notify_irq()") I mention that 
mce_need_notify got introduced in your commit and it was called 
"notify_user" before that. Before your commit in mce_log there was an 
explicit set_bit(mce_need_notify) call .

RE: 答复: 答复: 答复: [外部邮件] Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Thomas Gleixner]

On Tue, Jan 13 2026 at 18:53, Tony Luck wrote:
>> > The comment in mce_timer_fn says to adjust the polling interval, but
>> > I notice the kernel log always shows an MCE log every 5 minutes. Is this
>> > normal?
>>
>> Use git annotate to figure out which patch added this comment and in context
>> of what and that'll tell you why.
>>
>> As to the 5 minutes, look at how the check interval gets established.
>
> Once upon a time the polling interval started out at 5 minutes, but the
> interval was halved each time an error was found (so interval went
> 150s, 75s, 37s, ... down to 1s). If no error was found, then the interval
> was doubled (going back up to 300s).
>
> This is described in the comment:
>
>         /*
>          * Alert userspace if needed. If we logged an MCE, reduce the polling
>          * interval, otherwise increase the polling interval.
>          */
>
> It seems that the kernel isn't doing that today. Polling at a fixed 300 seconds
> event though errors are being found and logged.

How did we lose that?

> Interesting that the timestamps are 327.68 seconds apart, rather than
> 300 and change. So there is some strange stuff going on.

Nothing strange. That's the batching inaccuracy, aka. granularity of the
timer wheel. See the big fat comment on top of kernel/time/timer.c

So looking at that table, I'm sure you have HZ=250. But that granularity
does not explain why that interval magic is not longer working....

Thanks,

        tglx

Re: 答复: 答复: 答复: [外部邮件] Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Borislav Petkov]

On Tue, Jan 13, 2026 at 09:51:48PM +0200, Nikolay Borisov wrote:
> grepping around points to your 011d82611172 ("RAS: Add a Corrected Errors
> Collector"). Because looking at the commit log of my 6447828875b7
> ("x86/mce/inject: Remove call to mce_notify_irq()") I mention that
> mce_need_notify got introduced in your commit and it was called
> "notify_user" before that. Before your commit in mce_log there was an
> explicit set_bit(mce_need_notify) call .

It was removed there so that it doesn't issue the

	"Machine check events logged\n"

thing because that's the main point of the error collector - to collect errors
in the background and not upset users.

Also, there's an explicit set_bit(mce_need_notify) call in the early notifier.
So it'll get set a bit later, when the notifier chain goes.

But that same notifier does:

        set_bit(0, &mce_need_notify);
	mce_notify_irq();

so it'll clear that bit immediately.

Which means, the timer would probably never see the need_notify thing and thus
won't halve.

Which begs the question: is the timer even supposed to halve when you have the
error collector?

I guess so - we need it to log errors faster because the assumption is that
you'll be getting more errors so you need to read them out faster.

I need to think about it tomorrow on a clear head...

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: 答复: 答复: 答复: [外部邮件] Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Borislav Petkov]

On Tue, Jan 13, 2026 at 06:53:13PM +0000, Luck, Tony wrote:
> I can reproduce here on an Icelake system. Booted with mce=no_cmci to force polling
> (and turned of BIOS firmware first mode).  Injecting an error every 30 seconds I also see
> constant 327 seconds between logs (multiple logs show up because my injection picks memory
> channel "randomly", so there can be several banks with errors when polling happens).

I'm assuming you also disabled the CEC? I.e., ras=cec_disable?

> $ dmesg | grep 'Machine Check Event:'

Did you see the "Machine check events logged\n" print from mce_notify_irq() in
dmesg too?

I'll try to repro here tomorrow.

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

RE: 答复: 答复: 答复: [外部邮件] Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Luck, Tony]

>> $ dmesg | grep 'Machine Check Event:'
>
> Did you see the "Machine check events logged\n" print from mce_notify_irq() in
> dmesg too?

Yes.  I used the other grep pattern to see detail of which CPU/bank logged the error.
Same pattern of timestamps shows up with this grep too.

$ dmesg | grep 'Machine check events logged'
[  662.574376] mce: [Hardware Error]: Machine check events logged
[  662.669274] mce: [Hardware Error]: Machine check events logged
[  990.224918] mce: [Hardware Error]: Machine check events logged
[  990.319788] mce: [Hardware Error]: Machine check events logged
[ 1317.875993] mce: [Hardware Error]: Machine check events logged
[ 1317.975408] mce: [Hardware Error]: Machine check events logged
[ 1645.527190] mce: [Hardware Error]: Machine check events logged
[ 1645.627058] mce: [Hardware Error]: Machine check events logged
[ 1973.178560] mce: [Hardware Error]: Machine check events logged
[ 1973.277908] mce: [Hardware Error]: Machine check events logged
[ 7870.902444] mce: [Hardware Error]: Machine check events logged
[ 8198.553364] mce: [Hardware Error]: Machine check events logged

-Tony

Re: 答复: 答复: 答复: [外部邮件] Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Borislav Petkov]

On Tue, Jan 13, 2026 at 09:05:01PM +0000, Luck, Tony wrote:
> >> $ dmesg | grep 'Machine Check Event:'
> >
> > Did you see the "Machine check events logged\n" print from mce_notify_irq() in
> > dmesg too?
> 
> Yes.  I used the other grep pattern to see detail of which CPU/bank logged the error.
> Same pattern of timestamps shows up with this grep too.

Yah, this confirms the flow:

mce_timer_fn()-> ... -> machine_check_poll -> mce_log which will queue the
work and return.

Now, back in mce_timer_fn:

        /*
         * Alert userspace if needed. If we logged an MCE, reduce the polling
         * interval, otherwise increase the polling interval.
         */
        if (mce_notify_irq())


<--- we haven't ran the notifier chain yet so mce_need_notify is not set yet
so this won't hit and we won't halve the interval. I need to verify that
empirically.

                iv = max(iv / 2, (unsigned long) HZ/100);
        else
                iv = min(iv * 2, round_jiffies_relative(check_interval * HZ));

And now the notifier chain runs. mce_early_notifier() sets the bit, does
mce_notify_irq(), that clears the bit and then the notifier chain a little
later (skx_edac) logs the error.

So this looks like a silly timing issue...

We could set mce_need_notify in mce_log(), zap this thing:

                if (__ratelimit(&ratelimit))
                        pr_info(HW_ERR "Machine check events logged\n");

in mce_notify_irq() or at least predicate it on the CEC being enabled and then
not call mce_notify_irq() in the notifier but leave it be called in the timer
function...

Ufff, how silly and overengineered we've made it. I need to think about
a cleaner solution tomorrow...

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Borislav Petkov]

On Tue, Jan 13, 2026 at 10:31:58PM +0100, Borislav Petkov wrote:
> Ufff, how silly and overengineered we've made it. I need to think about
> a cleaner solution tomorrow...

One idea I just thought of:

those two steps:

1. setting mce_need_notify, and

2.  testing and clearing it

are basically logically a one query: do we have logged errors, or, IOW, is the
gen pool not empty?

IOW, this below.

That'll get rid of that gunk of when should we test and when set and will
JustWork(tm) for what we want it to do.

But let's see how it all really works tomorrow...

diff --git a/arch/x86/kernel/cpu/mce/core.c b/arch/x86/kernel/cpu/mce/core.c
index 34440021e8cf..c378d4d450b6 100644
--- a/arch/x86/kernel/cpu/mce/core.c
+++ b/arch/x86/kernel/cpu/mce/core.c
@@ -595,7 +595,7 @@ static bool mce_notify_irq(void)
 	/* Not more than two messages every minute */
 	static DEFINE_RATELIMIT_STATE(ratelimit, 60*HZ, 2);
 
-	if (test_and_clear_bit(0, &mce_need_notify)) {
+	if (!mce_gen_pool_empty()) {
 		mce_work_trigger();
 
 		if (__ratelimit(&ratelimit))
@@ -618,10 +618,6 @@ static int mce_early_notifier(struct notifier_block *nb, unsigned long val,
 	/* Emit the trace record: */
 	trace_mce_record(err);
 
-	set_bit(0, &mce_need_notify);
-
-	mce_notify_irq();
-
 	return NOTIFY_DONE;
 }
 
-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Luck, Tony]

On Tue, Jan 13, 2026 at 11:41:52PM +0100, Borislav Petkov wrote:
> On Tue, Jan 13, 2026 at 10:31:58PM +0100, Borislav Petkov wrote:
> > Ufff, how silly and overengineered we've made it. I need to think about
> > a cleaner solution tomorrow...
> 
> One idea I just thought of:
> 
> those two steps:
> 
> 1. setting mce_need_notify, and
> 
> 2.  testing and clearing it
> 
> are basically logically a one query: do we have logged errors, or, IOW, is the
> gen pool not empty?
> 
> IOW, this below.
> 
> That'll get rid of that gunk of when should we test and when set and will
> JustWork(tm) for what we want it to do.
> 
> But let's see how it all really works tomorrow...
> 
> diff --git a/arch/x86/kernel/cpu/mce/core.c b/arch/x86/kernel/cpu/mce/core.c
> index 34440021e8cf..c378d4d450b6 100644
> --- a/arch/x86/kernel/cpu/mce/core.c
> +++ b/arch/x86/kernel/cpu/mce/core.c
> @@ -595,7 +595,7 @@ static bool mce_notify_irq(void)
>  	/* Not more than two messages every minute */
>  	static DEFINE_RATELIMIT_STATE(ratelimit, 60*HZ, 2);
>  
> -	if (test_and_clear_bit(0, &mce_need_notify)) {
> +	if (!mce_gen_pool_empty()) {
>  		mce_work_trigger();
>  
>  		if (__ratelimit(&ratelimit))
> @@ -618,10 +618,6 @@ static int mce_early_notifier(struct notifier_block *nb, unsigned long val,
>  	/* Emit the trace record: */
>  	trace_mce_record(err);
>  
> -	set_bit(0, &mce_need_notify);
> -
> -	mce_notify_irq();
> -
>  	return NOTIFY_DONE;
>  }
>  

Seems to work (though you've deleted all the places where mce_need_notify
is used, so you can also delete the declaration.

@@ -90,7 +90,6 @@ struct mca_config mca_cfg __read_mostly = {
 };

 static DEFINE_PER_CPU(struct mce_hw_err, hw_errs_seen);
-static unsigned long mce_need_notify;

 /*
  * MCA banks polled by the period polling timer for corrected events.


I see time delta between logs reducing while I'm injecting errors.

When I pause injection for several minutes, and then restart I see the
interval went back up again.

-Tony

Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Nikolay Borisov]

On 14.01.26 г. 0:41 ч., Borislav Petkov wrote:
> On Tue, Jan 13, 2026 at 10:31:58PM +0100, Borislav Petkov wrote:
>> Ufff, how silly and overengineered we've made it. I need to think about
>> a cleaner solution tomorrow...
> 
> One idea I just thought of:
> 
> those two steps:
> 
> 1. setting mce_need_notify, and
> 
> 2.  testing and clearing it
> 
> are basically logically a one query: do we have logged errors, or, IOW, is the
> gen pool not empty?
> 
> IOW, this below.
> 
> That'll get rid of that gunk of when should we test and when set and will
> JustWork(tm) for what we want it to do.
> 
> But let's see how it all really works tomorrow...
> 
> diff --git a/arch/x86/kernel/cpu/mce/core.c b/arch/x86/kernel/cpu/mce/core.c
> index 34440021e8cf..c378d4d450b6 100644
> --- a/arch/x86/kernel/cpu/mce/core.c
> +++ b/arch/x86/kernel/cpu/mce/core.c
> @@ -595,7 +595,7 @@ static bool mce_notify_irq(void)
>   	/* Not more than two messages every minute */
>   	static DEFINE_RATELIMIT_STATE(ratelimit, 60*HZ, 2);
>   
> -	if (test_and_clear_bit(0, &mce_need_notify)) {
> +	if (!mce_gen_pool_empty()) {
>   		mce_work_trigger();
>   
>   		if (__ratelimit(&ratelimit))
> @@ -618,10 +618,6 @@ static int mce_early_notifier(struct notifier_block *nb, unsigned long val,
>   	/* Emit the trace record: */
>   	trace_mce_record(err);
>   
> -	set_bit(0, &mce_need_notify);
> -
> -	mce_notify_irq();
> -

so now mce_notify_irq() is not being called from the early notifier, i.e 
the callchain mc_poll_banks -> machine_check_poll -> mce_log no longer 
call mce_notify_irq() in the early notifier chain.

>   	return NOTIFY_DONE;
>   }
>   

Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Borislav Petkov]

On Tue, Jan 13, 2026 at 04:30:08PM -0800, Luck, Tony wrote:
> Seems to work (though you've deleted all the places where mce_need_notify
> is used, so you can also delete the declaration.

Right.

> I see time delta between logs reducing while I'm injecting errors.
> 
> When I pause injection for several minutes, and then restart I see the
> interval went back up again.

Thanks Tony, I'll play with this too and ponder over what would be the proper
fix which to take to stable too.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Borislav Petkov]

On Wed, Jan 14, 2026 at 08:17:50AM +0200, Nikolay Borisov wrote:
> so now mce_notify_irq() is not being called from the early notifier, i.e the
> callchain mc_poll_banks -> machine_check_poll -> mce_log no longer call
> mce_notify_irq() in the early notifier chain.

... which was the result of us moving this back'n'forth and refactoring it as
part of normal development. The thing is, that notification mechanism is
perhaps not needed there anymore, in favor of simply querying the gen pool.

But we'll see - needs more staring and injection.

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Borislav Petkov]

Ok,

I can repro in a guest here too:

[  316.791005] mce: [Hardware Error]: Machine check events logged
[  316.793836] mce: [Hardware Error]: CPU 0: Machine Check: 0 Bank 4: 9c2041000000011b
[  316.797398] mce: [Hardware Error]: TSC 0 ADDR 6d3d483b 
[  316.800013] mce: [Hardware Error]: PROCESSOR 2:800f82 TIME 1768404431 SOCKET 0 APIC 0 microcode 800820d
[  628.086857] mce: [Hardware Error]: Machine check events logged
[  628.089625] mce: [Hardware Error]: CPU 0: Machine Check: 0 Bank 4: 9c2041000000011b
[  628.093156] mce: [Hardware Error]: TSC 0 ADDR 6d3d483b 
[  628.095739] mce: [Hardware Error]: PROCESSOR 2:800f82 TIME 1768404743 SOCKET 0 APIC 0 microcode 800820d
[  939.382727] mce: [Hardware Error]: Machine check events logged
[  939.385593] mce: [Hardware Error]: CPU 0: Machine Check: 0 Bank 4: 9c2041000000011b
[  939.389194] mce: [Hardware Error]: TSC 0 ADDR 6d3d483b 
[  939.391690] mce: [Hardware Error]: PROCESSOR 2:800f82 TIME 1768405054 SOCKET 0 APIC 0 microcode 800820d
[ 1250.678672] mce: [Hardware Error]: Machine check events logged
[ 1250.681584] mce: [Hardware Error]: CPU 0: Machine Check: 0 Bank 4: 9c2041000000011b
[ 1250.685221] mce: [Hardware Error]: TSC 0 ADDR 6d3d483b 
[ 1250.687786] mce: [Hardware Error]: PROCESSOR 2:800f82 TIME 1768405365 SOCKET 0 APIC 0 microcode 800820d

Timestamps are consistent with the observation.

Now on to find what causes this. Even if we can't find the proper commit,
I guess testing 6.18 and 6.12 - the LTS kernels - should be good enough as to
backport a fix there.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Borislav Petkov]

On Wed, Jan 14, 2026 at 03:48:13PM +0100, Borislav Petkov wrote:
> Now on to find what causes this. Even if we can't find the proper commit,
> I guess testing 6.18 and 6.12 - the LTS kernels - should be good enough as to
> backport a fix there.

Ok, finally back to staring at this.

Looks like adding this:

diff --git a/arch/x86/kernel/cpu/mce/core.c b/arch/x86/kernel/cpu/mce/core.c
index 34440021e8cf..b94efe5950c4 100644
--- a/arch/x86/kernel/cpu/mce/core.c
+++ b/arch/x86/kernel/cpu/mce/core.c
@@ -154,6 +154,8 @@ void mce_log(struct mce_hw_err *err)
 {
        if (mce_gen_pool_add(err))
                irq_work_queue(&mce_irq_work);
+
+       set_bit(0, &mce_need_notify);
 }
 EXPORT_SYMBOL_GPL(mce_log);
 
makes the interval halve again, see below for the timestamps.

I guess I'll do a proper patch from the hunk here:

https://lore.kernel.org/r/20260113224152.GVaWbKMMzManQ5WwlT@fat_crate.local

along with 6.12 and 6.18 backports and see whether that's a good enough as
a stable fix too.

Thx.

[  316.795248] mce: [Hardware Error]: Machine check events logged
[  316.795262] mce: [Hardware Error]: Machine check events logged
[  316.798331] mce: [Hardware Error]: CPU 0: Machine Check: 0 Bank 4: 9c2041000000011b
[  316.800104] mce: [Hardware Error]: TSC 0 ADDR 6d3d483b 
[  316.801442] mce: [Hardware Error]: PROCESSOR 2:800f82 TIME 1770040950 SOCKET 0 APIC 0 microcode 800820d
[  628.091492] mce: [Hardware Error]: Machine check events logged
[  628.091515] mce: [Hardware Error]: Machine check events logged
[  628.097216] mce: [Hardware Error]: CPU 0: Machine Check: 0 Bank 4: 9c2041000000011b
[  628.101393] mce: [Hardware Error]: TSC 0 ADDR 6d3d483b 
[  628.103992] mce: [Hardware Error]: PROCESSOR 2:800f82 TIME 1770041262 SOCKET 0 APIC 0 microcode 800820d

<--- it starts decreasing the interval here.

[  825.581354] hrtimer: interrupt took 18820 ns
[  939.387367] mce: [Hardware Error]: Machine check events logged
[  939.390185] mce: [Hardware Error]: Machine check events logged
[  939.392936] mce: [Hardware Error]: CPU 0: Machine Check: 0 Bank 4: 9c2041000000011b
[  939.396465] mce: [Hardware Error]: TSC 0 ADDR 6d3d483b 
[  939.399042] mce: [Hardware Error]: PROCESSOR 2:800f82 TIME 1770041573 SOCKET 0 APIC 0 microcode 800820d
[ 1103.227402] mce: [Hardware Error]: Machine check events logged
[ 1103.230267] mce: [Hardware Error]: Machine check events logged
[ 1103.233018] mce: [Hardware Error]: CPU 0: Machine Check: 0 Bank 4: 9c2041000000011b
[ 1103.236565] mce: [Hardware Error]: TSC 0 ADDR 6d3d483b 
[ 1103.239146] mce: [Hardware Error]: PROCESSOR 2:800f82 TIME 1770041737 SOCKET 0 APIC 0 microcode 800820d
[ 1179.003479] mce: [Hardware Error]: Machine check events logged
[ 1179.006452] mce: [Hardware Error]: Machine check events logged
[ 1179.009144] mce: [Hardware Error]: CPU 0: Machine Check: 0 Bank 4: 9c2041000000011b
[ 1179.012757] mce: [Hardware Error]: TSC 0 ADDR 6d3d483b 
[ 1179.015338] mce: [Hardware Error]: PROCESSOR 2:800f82 TIME 1770041813 SOCKET 0 APIC 0 microcode 800820d
[ 1217.915386] mce: [Hardware Error]: CPU 0: Machine Check: 0 Bank 4: 9c2041000000011b
[ 1217.919088] mce: [Hardware Error]: TSC 0 ADDR 6d3d483b 
[ 1217.921662] mce: [Hardware Error]: PROCESSOR 2:800f82 TIME 1770041852 SOCKET 0 APIC 0 microcode 800820d
[ 1238.395440] mce: [Hardware Error]: CPU 0: Machine Check: 0 Bank 4: 9c2041000000011b
[ 1238.399041] mce: [Hardware Error]: TSC 0 ADDR 6d3d483b 
[ 1238.401619] mce: [Hardware Error]: PROCESSOR 2:800f82 TIME 1770041872 SOCKET 0 APIC 0 microcode 800820d
[ 1269.115368] mce_notify_irq: 4 callbacks suppressed
[ 1269.117829] mce: [Hardware Error]: Machine check events logged
[ 1269.120586] mce: [Hardware Error]: Machine check events logged
[ 1269.123412] mce: [Hardware Error]: CPU 0: Machine Check: 0 Bank 4: 9c2041000000011b
[ 1269.126950] mce: [Hardware Error]: TSC 0 ADDR 6d3d483b 
[ 1269.129511] mce: [Hardware Error]: PROCESSOR 2:800f82 TIME 1770041903 SOCKET 0 APIC 0 microcode 800820d

and then it started enlarging it again when I changed the injection interval
to 300s.

[ 1578.363408] mce: [Hardware Error]: Machine check events logged
[ 1578.366346] mce: [Hardware Error]: Machine check events logged
[ 1578.369174] mce: [Hardware Error]: CPU 0: Machine Check: 0 Bank 4: 9c2041000000011b
[ 1578.372742] mce: [Hardware Error]: TSC 0 ADDR 6d3d483b 
[ 1578.375226] mce: [Hardware Error]: PROCESSOR 2:800f82 TIME 1770042212 SOCKET 0 APIC 0 microcode 800820d
[ 2119.035460] mce: [Hardware Error]: Machine check events logged
[ 2119.038432] mce: [Hardware Error]: Machine check events logged
[ 2119.041236] mce: [Hardware Error]: CPU 0: Machine Check: 0 Bank 4: 9c2041000000011b
[ 2119.044846] mce: [Hardware Error]: TSC 0 ADDR 6d3d483b 
[ 2119.047340] mce: [Hardware Error]: PROCESSOR 2:800f82 TIME 1770042753 SOCKET 0 APIC 0 microcode 800820d
[ 2282.875491] mce: [Hardware Error]: Machine check events logged
[ 2282.878409] mce: [Hardware Error]: Machine check events logged
[ 2282.881277] mce: [Hardware Error]: CPU 0: Machine Check: 0 Bank 4: 9c2041000000011b
[ 2282.884978] mce: [Hardware Error]: TSC 0 ADDR 6d3d483b 
[ 2282.887482] mce: [Hardware Error]: PROCESSOR 2:800f82 TIME 1770042917 SOCKET 0 APIC 0 microcode 800820d
[ 2512.251516] mce: [Hardware Error]: Machine check events logged
[ 2512.254371] mce: [Hardware Error]: Machine check events logged
[ 2512.257261] mce: [Hardware Error]: CPU 0: Machine Check: 0 Bank 4: 9c2041000000011b
[ 2512.260841] mce: [Hardware Error]: TSC 0 ADDR 6d3d483b 
[ 2512.263406] mce: [Hardware Error]: PROCESSOR 2:800f82 TIME 1770043146 SOCKET 0 APIC 0 microcode 800820d

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

答复: [外部邮件] Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Li,Rongqing]

> On Wed, Jan 14, 2026 at 03:48:13PM +0100, Borislav Petkov wrote:
> > Now on to find what causes this. Even if we can't find the proper
> > commit, I guess testing 6.18 and 6.12 - the LTS kernels - should be
> > good enough as to backport a fix there.
> 
> Ok, finally back to staring at this.
> 
> Looks like adding this:
> 
> diff --git a/arch/x86/kernel/cpu/mce/core.c b/arch/x86/kernel/cpu/mce/core.c
> index 34440021e8cf..b94efe5950c4 100644
> --- a/arch/x86/kernel/cpu/mce/core.c
> +++ b/arch/x86/kernel/cpu/mce/core.c
> @@ -154,6 +154,8 @@ void mce_log(struct mce_hw_err *err)  {
>         if (mce_gen_pool_add(err))
>                 irq_work_queue(&mce_irq_work);
> +
> +       set_bit(0, &mce_need_notify);
>  }
>  EXPORT_SYMBOL_GPL(mce_log);
> 
> makes the interval halve again, see below for the timestamps.
> 
> I guess I'll do a proper patch from the hunk here:
> 
> https://lore.kernel.org/r/20260113224152.GVaWbKMMzManQ5WwlT@fat_cr
> ate.local
> 

Is it possible where CPU0 sets mce_need_notify, but CPU1 concurrently calls mce_notify_irq in mce_timer_fn, and then CPU1 sets its own timer to 1/2 instead of CPU0's

[Li,Rongqing] 



> along with 6.12 and 6.18 backports and see whether that's a good enough as a
> stable fix too.
> 
> Thx.
> 
> [  316.795248] mce: [Hardware Error]: Machine check events logged
> [  316.795262] mce: [Hardware Error]: Machine check events logged
> [  316.798331] mce: [Hardware Error]: CPU 0: Machine Check: 0 Bank 4:
> 9c2041000000011b [  316.800104] mce: [Hardware Error]: TSC 0 ADDR
> 6d3d483b [  316.801442] mce: [Hardware Error]: PROCESSOR 2:800f82 TIME
> 1770040950 SOCKET 0 APIC 0 microcode 800820d [  628.091492] mce:
> [Hardware Error]: Machine check events logged [  628.091515] mce:
> [Hardware Error]: Machine check events logged [  628.097216] mce:
> [Hardware Error]: CPU 0: Machine Check: 0 Bank 4: 9c2041000000011b
> [  628.101393] mce: [Hardware Error]: TSC 0 ADDR 6d3d483b [  628.103992]
> mce: [Hardware Error]: PROCESSOR 2:800f82 TIME 1770041262 SOCKET 0
> APIC 0 microcode 800820d
> 
> <--- it starts decreasing the interval here.
> 
> [  825.581354] hrtimer: interrupt took 18820 ns [  939.387367] mce:
> [Hardware Error]: Machine check events logged [  939.390185] mce:
> [Hardware Error]: Machine check events logged [  939.392936] mce:
> [Hardware Error]: CPU 0: Machine Check: 0 Bank 4: 9c2041000000011b
> [  939.396465] mce: [Hardware Error]: TSC 0 ADDR 6d3d483b [  939.399042]
> mce: [Hardware Error]: PROCESSOR 2:800f82 TIME 1770041573 SOCKET 0
> APIC 0 microcode 800820d [ 1103.227402] mce: [Hardware Error]: Machine
> check events logged [ 1103.230267] mce: [Hardware Error]: Machine check
> events logged [ 1103.233018] mce: [Hardware Error]: CPU 0: Machine Check: 0
> Bank 4: 9c2041000000011b [ 1103.236565] mce: [Hardware Error]: TSC 0
> ADDR 6d3d483b [ 1103.239146] mce: [Hardware Error]: PROCESSOR 2:800f82
> TIME 1770041737 SOCKET 0 APIC 0 microcode 800820d [ 1179.003479] mce:
> [Hardware Error]: Machine check events logged [ 1179.006452] mce:
> [Hardware Error]: Machine check events logged [ 1179.009144] mce:
> [Hardware Error]: CPU 0: Machine Check: 0 Bank 4: 9c2041000000011b
> [ 1179.012757] mce: [Hardware Error]: TSC 0 ADDR 6d3d483b [ 1179.015338]
> mce: [Hardware Error]: PROCESSOR 2:800f82 TIME 1770041813 SOCKET 0
> APIC 0 microcode 800820d [ 1217.915386] mce: [Hardware Error]: CPU 0:
> Machine Check: 0 Bank 4: 9c2041000000011b [ 1217.919088] mce:
> [Hardware Error]: TSC 0 ADDR 6d3d483b [ 1217.921662] mce: [Hardware
> Error]: PROCESSOR 2:800f82 TIME 1770041852 SOCKET 0 APIC 0 microcode
> 800820d [ 1238.395440] mce: [Hardware Error]: CPU 0: Machine Check: 0
> Bank 4: 9c2041000000011b [ 1238.399041] mce: [Hardware Error]: TSC 0
> ADDR 6d3d483b [ 1238.401619] mce: [Hardware Error]: PROCESSOR 2:800f82
> TIME 1770041872 SOCKET 0 APIC 0 microcode 800820d [ 1269.115368]
> mce_notify_irq: 4 callbacks suppressed [ 1269.117829] mce: [Hardware Error]:
> Machine check events logged [ 1269.120586] mce: [Hardware Error]: Machine
> check events logged [ 1269.123412] mce: [Hardware Error]: CPU 0: Machine
> Check: 0 Bank 4: 9c2041000000011b [ 1269.126950] mce: [Hardware Error]:
> TSC 0 ADDR 6d3d483b [ 1269.129511] mce: [Hardware Error]: PROCESSOR
> 2:800f82 TIME 1770041903 SOCKET 0 APIC 0 microcode 800820d
> 
> and then it started enlarging it again when I changed the injection interval to
> 300s.
> 
> [ 1578.363408] mce: [Hardware Error]: Machine check events logged
> [ 1578.366346] mce: [Hardware Error]: Machine check events logged
> [ 1578.369174] mce: [Hardware Error]: CPU 0: Machine Check: 0 Bank 4:
> 9c2041000000011b [ 1578.372742] mce: [Hardware Error]: TSC 0 ADDR
> 6d3d483b [ 1578.375226] mce: [Hardware Error]: PROCESSOR 2:800f82 TIME
> 1770042212 SOCKET 0 APIC 0 microcode 800820d [ 2119.035460] mce:
> [Hardware Error]: Machine check events logged [ 2119.038432] mce:
> [Hardware Error]: Machine check events logged [ 2119.041236] mce:
> [Hardware Error]: CPU 0: Machine Check: 0 Bank 4: 9c2041000000011b
> [ 2119.044846] mce: [Hardware Error]: TSC 0 ADDR 6d3d483b [ 2119.047340]
> mce: [Hardware Error]: PROCESSOR 2:800f82 TIME 1770042753 SOCKET 0
> APIC 0 microcode 800820d [ 2282.875491] mce: [Hardware Error]: Machine
> check events logged [ 2282.878409] mce: [Hardware Error]: Machine check
> events logged [ 2282.881277] mce: [Hardware Error]: CPU 0: Machine Check: 0
> Bank 4: 9c2041000000011b [ 2282.884978] mce: [Hardware Error]: TSC 0
> ADDR 6d3d483b [ 2282.887482] mce: [Hardware Error]: PROCESSOR 2:800f82
> TIME 1770042917 SOCKET 0 APIC 0 microcode 800820d [ 2512.251516] mce:
> [Hardware Error]: Machine check events logged [ 2512.254371] mce:
> [Hardware Error]: Machine check events logged [ 2512.257261] mce:
> [Hardware Error]: CPU 0: Machine Check: 0 Bank 4: 9c2041000000011b
> [ 2512.260841] mce: [Hardware Error]: TSC 0 ADDR 6d3d483b [ 2512.263406]
> mce: [Hardware Error]: PROCESSOR 2:800f82 TIME 1770043146 SOCKET 0
> APIC 0 microcode 800820d
> 
> --
> Regards/Gruss,
>     Boris.
> 
> https://people.kernel.org/tglx/notes-about-netiquette

Re: 答复: [外部邮件] Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Borislav Petkov]

On Mon, Feb 02, 2026 at 11:49:40PM +0000, Li,Rongqing wrote:
> Is it possible where CPU0 sets mce_need_notify, but CPU1 concurrently calls
> mce_notify_irq in mce_timer_fn, and then CPU1 sets its own timer to 1/2
> instead of CPU0's

I don't see why not.

But this is no different than what it was before AFAICT.

And if the timer expiry interval on the wrong CPU gets halved, then it'll see
faster that there's no error for it to log and it'll enlarge the interval
again.

So I don't see a problem with that particular scenario.

Or?

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Borislav Petkov]

On Wed, Jan 14, 2026 at 02:50:34PM +0100, Borislav Petkov wrote:
> On Tue, Jan 13, 2026 at 04:30:08PM -0800, Luck, Tony wrote:
> > Seems to work (though you've deleted all the places where mce_need_notify
> > is used, so you can also delete the declaration.
> 
> Right.
> 
> > I see time delta between logs reducing while I'm injecting errors.
> > 
> > When I pause injection for several minutes, and then restart I see the
> > interval went back up again.
> 
> Thanks Tony, I'll play with this too and ponder over what would be the proper
> fix which to take to stable too.

Hmm, so looking at this more while it is all peaceful and I can actually hear
the thoughts in my head... :-)

The whole dance here on the MCE logging path:

mce_log -> ... mce_irq_work -> ... mce_work -> mce_gen_pool_process

can happen in between two mce_timer_fn() function firings - just think of
the default timer running once every 5 mins.

So in-between those runs with 5 min timeout, errors can get logged and when
mce_notify_irq() runs, it won't see either that the genpool is not empty
- it will be empty - and mce_need_notify will be 0 too because we would've
set and cleared it. 

So basically, the timer fires, we log errors without it noticing anything, and
it won't halve.

The only way it would halve is if it manages to notice an error being
in-flight to being logged and it fires right then and there. Then its interval
would get halved.

And this sounds kinda weird and not what we want perhaps.

So fixing that would mean, we'd have to write down the fact that in-between
two timer invocations, we have logged errors. Maybe a per-CPU counter
somewhere which says "this CPU logged so many errors after the timer ran the
last time".

The timer would fire, check that counter for != 0, and if so, decrease
interval and clear it.

And it doesn't even have to be a counter - it suffices to be a single bit
which gets set.

A scheme like that would solve this accurately I'd say.

But the real question actually is, do we really care?

I mean, this thing went unnoticed for so long and frankly, people should run
the CEC anyway which has a better MCE-has-been-logged stifling capability so
that I wanna say, let's do the simplest thing and be done with it.

Or?

Do we care about some real use case here...?

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Luck, Tony]

On Sat, Feb 07, 2026 at 12:51:42PM +0100, Borislav Petkov wrote:
> On Wed, Jan 14, 2026 at 02:50:34PM +0100, Borislav Petkov wrote:
> > On Tue, Jan 13, 2026 at 04:30:08PM -0800, Luck, Tony wrote:
> > > Seems to work (though you've deleted all the places where mce_need_notify
> > > is used, so you can also delete the declaration.
> > 
> > Right.
> > 
> > > I see time delta between logs reducing while I'm injecting errors.
> > > 
> > > When I pause injection for several minutes, and then restart I see the
> > > interval went back up again.
> > 
> > Thanks Tony, I'll play with this too and ponder over what would be the proper
> > fix which to take to stable too.
> 
> Hmm, so looking at this more while it is all peaceful and I can actually hear
> the thoughts in my head... :-)
> 
> The whole dance here on the MCE logging path:
> 
> mce_log -> ... mce_irq_work -> ... mce_work -> mce_gen_pool_process
> 
> can happen in between two mce_timer_fn() function firings - just think of
> the default timer running once every 5 mins.
> 
> So in-between those runs with 5 min timeout, errors can get logged and when
> mce_notify_irq() runs, it won't see either that the genpool is not empty
> - it will be empty - and mce_need_notify will be 0 too because we would've
> set and cleared it. 

The algorithm to halve the interval when errors are found, and double it
when they are not found was orginally for a "poll-only" configuration.
So there wasn't an option for an error to be logged between timer
invocations. This all dates back to before #MC was recoverable.

If the system is now running in some mixed mode of polling and
interrupts, then it is unclear what should be done in various
new cases.

> 
> So basically, the timer fires, we log errors without it noticing anything, and
> it won't halve.
> 
> The only way it would halve is if it manages to notice an error being
> in-flight to being logged and it fires right then and there. Then its interval
> would get halved.
> 
> And this sounds kinda weird and not what we want perhaps.
> 
> So fixing that would mean, we'd have to write down the fact that in-between
> two timer invocations, we have logged errors. Maybe a per-CPU counter
> somewhere which says "this CPU logged so many errors after the timer ran the
> last time".
> 
> The timer would fire, check that counter for != 0, and if so, decrease
> interval and clear it.
> 
> And it doesn't even have to be a counter - it suffices to be a single bit
> which gets set.
> 
> A scheme like that would solve this accurately I'd say.
> 
> But the real question actually is, do we really care?

I don't think we care. If we miss out halving the interval becuause an
error was logged between timer based polling, nothing really bad will
happen. The interval might get sorted out on the next interval.

> I mean, this thing went unnoticed for so long and frankly, people should run
> the CEC anyway which has a better MCE-has-been-logged stifling capability so
> that I wanna say, let's do the simplest thing and be done with it.
> 
> Or?
> 
> Do we care about some real use case here...?
> 

Unless someone has a real world case where something is going badly
wrong, then I don't think any changes are needed to cover this race.

-Tony

Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Borislav Petkov]

On Mon, Feb 09, 2026 at 09:37:35AM -0800, Luck, Tony wrote:
> If the system is now running in some mixed mode of polling and
> interrupts, then it is unclear what should be done in various
> new cases.

Well, what about CMCI? AMD has similar interrupt types. Those handlers end up
in the same path of machine_check_poll() and the above scenario can happen,
AFAICT.

> I don't think we care. If we miss out halving the interval becuause an
> error was logged between timer based polling, nothing really bad will
> happen. The interval might get sorted out on the next interval.

Right, that's what I'm thinking too.

> Unless someone has a real world case where something is going badly
> wrong, then I don't think any changes are needed to cover this race.

Ok, let's try the simple thing first.

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

答复: [外部邮件] Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Li,Rongqing(ACG CCN)]

> Well, what about CMCI? AMD has similar interrupt types. Those handlers end up
> in the same path of machine_check_poll() and the above scenario can happen,
> AFAICT.
> 
> > I don't think we care. If we miss out halving the interval becuause an
> > error was logged between timer based polling, nothing really bad will
> > happen. The interval might get sorted out on the next interval.
> 
> Right, that's what I'm thinking too.
> 
> > Unless someone has a real world case where something is going badly
> > wrong, then I don't think any changes are needed to cover this race.
> 
> Ok, let's try the simple thing first.
> 
> Thx.

Is there any progress on this?

Thx

[Li,Rongqing] 


> 
> --
> Regards/Gruss,
>     Boris.
> 
> https://people.kernel.org/tglx/notes-about-netiquette

Re: 答复: [外部邮件] Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Borislav Petkov]

On March 6, 2026 7:37:01 AM UTC, "Li,Rongqing(ACG CCN)" <lirongqing@baidu.com> wrote:
>Is there any progress on this?

Not yet.

May I ask what is your usecase for this? 

-- 
Small device. Typos and formatting crap

答复: 答复: [外部邮件] Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Li,Rongqing(ACG CCN)]

> -----邮件原件-----
> 发件人: Borislav Petkov <bp@alien8.de>
> 发送时间: 2026年3月6日 22:00
> 收件人: Li,Rongqing(ACG CCN) <lirongqing@baidu.com>; Luck, Tony
> <tony.luck@intel.com>
> 抄送: Nikolay Borisov <nik.borisov@suse.com>; Thomas Gleixner
> <tglx@kernel.org>; Ingo Molnar <mingo@redhat.com>; Dave Hansen
> <dave.hansen@linux.intel.com>; x86@kernel.org; H . Peter Anvin
> <hpa@zytor.com>; Yazen Ghannam <yazen.ghannam@amd.com>; Zhuo, Qiuxu
> <qiuxu.zhuo@intel.com>; Avadhut Naik <avadhut.naik@amd.com>;
> linux-kernel@vger.kernel.org; linux-edac@vger.kernel.org
> 主题: Re: 答复: [外部邮件] Re: [PATCH] x86/mce: Fix timer interval
> adjustment after logging a MCE event
> 
> On March 6, 2026 7:37:01 AM UTC, "Li,Rongqing(ACG CCN)"
> <lirongqing@baidu.com> wrote:
> >Is there any progress on this?
> 
> Not yet.
> 
> May I ask what is your usecase for this? 


We anticipate potential UE issues by analyzing the volume and frequency of collected CE reports, enabling us to perform proactive task offloading and machine maintenance. However, inaccuracies in the collected data are currently undermining this approach, making it difficult to reliably predict UE incidents.

[Li,Rongqing] 
> 
> --
> Small device. Typos and formatting crap

Re: 答复: 答复: [外部邮件] Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Borislav Petkov]

On Fri, Mar 06, 2026 at 02:38:29PM +0000, Li,Rongqing(ACG CCN) wrote:
> We anticipate potential UE issues by analyzing the volume and frequency of
> collected CE reports, enabling us to perform proactive task offloading and
> machine maintenance. However, inaccuracies in the collected data are
> currently undermining this approach, making it difficult to reliably predict
> UE incidents.

This looks like a canned AI reply to me.

I think you wanna say, you want to get *every* single error logged. Yes?

So you want to be able to decrease the polling interval if necessary?

Do you also disable the RAS CEC?

Thx.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

答复: 答复: 答复: [外部邮件] Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Li,Rongqing(ACG CCN)]

> -----邮件原件-----
> 发件人: Borislav Petkov <bp@alien8.de>
> 发送时间: 2026年3月6日 23:29
> 收件人: Li,Rongqing(ACG CCN) <lirongqing@baidu.com>
> 抄送: Luck, Tony <tony.luck@intel.com>; Nikolay Borisov
> <nik.borisov@suse.com>; Thomas Gleixner <tglx@kernel.org>; Ingo Molnar
> <mingo@redhat.com>; Dave Hansen <dave.hansen@linux.intel.com>;
> x86@kernel.org; H . Peter Anvin <hpa@zytor.com>; Yazen Ghannam
> <yazen.ghannam@amd.com>; Zhuo, Qiuxu <qiuxu.zhuo@intel.com>; Avadhut
> Naik <avadhut.naik@amd.com>; linux-kernel@vger.kernel.org;
> linux-edac@vger.kernel.org
> 主题: Re: 答复: 答复: [外部邮件] Re: [PATCH] x86/mce: Fix timer interval
> adjustment after logging a MCE event
> 
> On Fri, Mar 06, 2026 at 02:38:29PM +0000, Li,Rongqing(ACG CCN) wrote:
> > We anticipate potential UE issues by analyzing the volume and
> > frequency of collected CE reports, enabling us to perform proactive
> > task offloading and machine maintenance. However, inaccuracies in the
> > collected data are currently undermining this approach, making it
> > difficult to reliably predict UE incidents.
> 
> This looks like a canned AI reply to me.
> 
> I think you wanna say, you want to get *every* single error logged. Yes?
> 
> So you want to be able to decrease the polling interval if necessary?
> 
> Do you also disable the RAS CEC?
> 

CEC may not work in some cases. For example, when QEMU uses vDPA devices, all of QEMU's memory is pinned and cannot be offlined. hugetlbfs has only recently gained support for offline operations; offlining hugetlbfs can cause issues. The kernel provides an interface to disable offline, as referenced in the patch "mm/memory-failure: userspace controls soft-offlining pages." 
And evaluating Correctable Errors (CE) in userspace, more parameters can be considered, such as memory manufacturer, batch, and frequency, to improve prediction accuracy 


[Li,Rongqing] 


> Thx.
> 
> --
> Regards/Gruss,
>     Boris.
> 
> https://people.kernel.org/tglx/notes-about-netiquette

Re: 答复: 答复: 答复: [外部邮件] Re: [PATCH] x86/mce: Fix timer interval adjustment after logging a MCE event

[Borislav Petkov]

On Sat, Mar 07, 2026 at 01:18:50AM +0000, Li,Rongqing(ACG CCN) wrote:
> CEC may not work in some cases.

I actually am saying that if you wanna look at every CE, you *want* to disable
the CEC because it consumes them. Look at

drivers/ras/cec.c

for more info.

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette