[PATCH RESEND 0/2] tick/nohz: CPU cannot enter NOHZ idle balance state

[PATCH RESEND 0/2] tick/nohz: CPU cannot enter NOHZ idle balance state

[Adam Li]

Valentin Schneider suggested to resend this patch and copy to
scheduler reviewers [1].

When running llama on arm64 server, some CPUs *keep* idle while others
are 100% busy. All CPUs are in 'nohz_full=' cpu list, and CONFIG_NO_HZ_FULL
is set. The server has 192 CPUs, with kernel option 'nohz_full=0-191'.

The problem is caused by two issues:
1) Some idle CPUs cannot be added to 'nohz.idle_cpus_mask'. This bug
is fixed by the first patch in this serial:
"tick/nohz: Fix wrong NOHZ idle CPU state".

2) Even if the idle CPUs are in 'nohz.idle_cpus_mask', no CPU can be
selected to do NOHZ idle load balancing because conditions in
find_new_ilb() is too strict. This issue is fixed by patch in [2].

We can see that the idle CPUs are not in nohz.idle_cpus_mask. The NOHZ
idle load balancing only considers CPUs in nohz.idle_cpus_mask. The ticks
on the idle CPUs are stopped and therefore period load balancing
is not triggered. Therefore the CPUs are not used and the
imbalance persists.

A CPU is added to nohz.idle_cpus_mask in:
do_idle()
   -> tick_nohz_idle_stop_tick()
      -> nohz_balance_enter_idle()

nohz_balance_enter_idle() depends on '!was_stopped' condition.
It looks 'was_stopped' is used to avoid duplicated calling
nohz_balance_enter_idle() and duplicated setting 'ts->idle_jiffies'.

When the CPU is in nohz_full mode, 'was_stopped' may alwasy be true.
The call path might be:

tick_nohz_full_stop_tick() /* stop tick and set TS_FLAG_STOPPED */
... ...
do_idle()
    -> tick_nohz_idle_stop_tick() /* was_stoppped == 1 */

The first patch "Fix wrong NOHZ idle CPU state" makes
nohz_balance_enter_idle() independent of '!was_stopped'. It is safe
since in nohz_balance_enter_idle(), there exists a condition check
'rq->nohz_tick_stopped' to avoid duplicated nohz.idle_cpus_mask setting.

The second patch "Trigger warning when CPU in wrong NOHZ idle state"
is for debug only. It is not intended to be merged. The patch can help
to reproduce the bug.

Warning is triggerred when CPU is in this 'wrong' state:
1) tick was already stopped before tick_nohz_idle_stop_tick()
   stops the tick
2) and CPU is not in nohz.idle_cpus_mask
3) and CPU is idle
4) and tick is stopped

When kernel booting on my system there is warning:
[   15.536604] WARNING: CPU: 1 PID: 0 at kernel/time/tick-sched.c:1230 tick_nohz_idle_stop_tick+0x148/0x160
[   15.550687] Modules linked in:
[   15.553731] CPU: 1 UID: 0 PID: 0 Comm: swapper/1 Not tainted 6.17.0-rc1-cls-00002-g39cde4c0206e-dirty #109 VOLUNTARY
[   15.580390] pstate: 614000c9 (nZCv daIF +PAN -UAO -TCO +DIT -SSBS BTYPE=--)
<snip>
[   15.703028] Call trace:
[   15.705462]  tick_nohz_idle_stop_tick+0x148/0x160 (P)
[   15.710502]  cpuidle_idle_call+0x118/0x1d0
[   15.714588]  do_idle+0xf4/0x100
[   15.717717]  cpu_startup_entry+0x40/0x50
[   15.721627]  secondary_start_kernel+0xe4/0x128
[   15.732745]  __secondary_switched+0xc0/0xc8

After the first patch, CPU is added to nohz.idle_cpus_mask.
NOHZ idle balancing can move task to this CPU.

Adam Li (2):
  tick/nohz: Fix wrong NOHZ idle CPU state
  tick/nohz: Trigger warning when CPU in wrong NOHZ idle state

Links
[1]: https://lore.kernel.org/all/xhsmho6sagz7p.mognet@vschneid-thinkpadt14sgen2i.remote.csb/
[2]: https://lore.kernel.org/all/20250819025720.14794-1-adamli@os.amperecomputing.com/

 include/linux/sched/nohz.h | 2 ++
 kernel/sched/fair.c        | 5 +++++
 kernel/time/tick-sched.c   | 8 +++++---
 3 files changed, 12 insertions(+), 3 deletions(-)

-- 
2.34.1

[PATCH RESEND 1/2] tick/nohz: Fix wrong NOHZ idle CPU state

[Adam Li]

NOHZ idle load balance is done among CPUs in nohz.idle_cpus_mask.
A CPU is added to nohz.idle_cpus_mask in:
do_idle()
  -> tick_nohz_idle_stop_tick()
     -> nohz_balance_enter_idle()

nohz_balance_enter_idle() is called if:
1) tick is stopped (TS_FLAG_STOPPED is set)
2) and tick was not already stopped before tick_nohz_idle_stop_tick()
   stops the tick (!was_stopped)

When CONFIG_NO_HZ_FULL is set and the CPU is in the nohz_full list
then 'was_stopped' may always be true.
The flag 'TS_FLAG_STOPPED' may be already set in
tick_nohz_full_stop_tick(). So nohz_balance_enter_idle() has no chance
to be called.

As a result, CPU will stay in a 'wrong' state:
1) tick is stopped (TS_FLAG_STOPPED is set)
2) and CPU is not in nohz.idle_cpus_mask
3) and CPU stays idle

Neither the periodic nor the NOHZ idle load balancing can move task
to this CPU. Some CPUs keep idle while others busy.

In nohz_balance_enter_idle(), 'rq->nohz_tick_stopped' is checked to avoid
duplicated nohz.idle_cpus_mask setting. So for nohz_balance_enter_idle()
there is no need to check the '!was_stopped' condition.

This patch will add the CPU to nohz.idle_cpus_mask as expected.

Signed-off-by: Adam Li <adamli@os.amperecomputing.com>
Reviewed-by: Christoph Lameter (Ampere) <cl@gentwo.org>
---
 kernel/time/tick-sched.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index c527b421c865..b900a120ab54 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -1229,8 +1229,9 @@ void tick_nohz_idle_stop_tick(void)
 		ts->idle_sleeps++;
 		ts->idle_expires = expires;
 
-		if (!was_stopped && tick_sched_flag_test(ts, TS_FLAG_STOPPED)) {
-			ts->idle_jiffies = ts->last_jiffies;
+		if (tick_sched_flag_test(ts, TS_FLAG_STOPPED)) {
+			if (!was_stopped)
+				ts->idle_jiffies = ts->last_jiffies;
 			nohz_balance_enter_idle(cpu);
 		}
 	} else {
-- 
2.34.1

Re: [PATCH RESEND 1/2] tick/nohz: Fix wrong NOHZ idle CPU state

[Frederic Weisbecker]

Le Thu, Aug 21, 2025 at 04:27:06AM +0000, Adam Li a écrit :
> NOHZ idle load balance is done among CPUs in nohz.idle_cpus_mask.
> A CPU is added to nohz.idle_cpus_mask in:
> do_idle()
>   -> tick_nohz_idle_stop_tick()
>      -> nohz_balance_enter_idle()
> 
> nohz_balance_enter_idle() is called if:
> 1) tick is stopped (TS_FLAG_STOPPED is set)
> 2) and tick was not already stopped before tick_nohz_idle_stop_tick()
>    stops the tick (!was_stopped)
> 
> When CONFIG_NO_HZ_FULL is set and the CPU is in the nohz_full list
> then 'was_stopped' may always be true.
> The flag 'TS_FLAG_STOPPED' may be already set in
> tick_nohz_full_stop_tick(). So nohz_balance_enter_idle() has no chance
> to be called.
> 
> As a result, CPU will stay in a 'wrong' state:
> 1) tick is stopped (TS_FLAG_STOPPED is set)
> 2) and CPU is not in nohz.idle_cpus_mask
> 3) and CPU stays idle
> 
> Neither the periodic nor the NOHZ idle load balancing can move task
> to this CPU. Some CPUs keep idle while others busy.
> 
> In nohz_balance_enter_idle(), 'rq->nohz_tick_stopped' is checked to avoid
> duplicated nohz.idle_cpus_mask setting. So for nohz_balance_enter_idle()
> there is no need to check the '!was_stopped' condition.
> 
> This patch will add the CPU to nohz.idle_cpus_mask as expected.
> 
> Signed-off-by: Adam Li <adamli@os.amperecomputing.com>
> Reviewed-by: Christoph Lameter (Ampere) <cl@gentwo.org>
> ---
>  kernel/time/tick-sched.c | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
> index c527b421c865..b900a120ab54 100644
> --- a/kernel/time/tick-sched.c
> +++ b/kernel/time/tick-sched.c
> @@ -1229,8 +1229,9 @@ void tick_nohz_idle_stop_tick(void)
>  		ts->idle_sleeps++;
>  		ts->idle_expires = expires;
>  
> -		if (!was_stopped && tick_sched_flag_test(ts, TS_FLAG_STOPPED)) {
> -			ts->idle_jiffies = ts->last_jiffies;
> +		if (tick_sched_flag_test(ts, TS_FLAG_STOPPED)) {
> +			if (!was_stopped)
> +				ts->idle_jiffies = ts->last_jiffies;
>  			nohz_balance_enter_idle(cpu);

The current state is indeed broken and some people have already tried to fix it.
The thing is nohz_full don't want dynamic isolation because it is deemed to run a
single task. Therefore those tasks must be placed manually in order not to break
isolation guarantees by accident.

In fact nohz_full doesn't make much sense without isolcpus (or isolated cpuset
v2 partitions) and I even intend to make nohz_full depend on domain isolation
in the long term.

Thanks.

>  		}
>  	} else {
> -- 
> 2.34.1
> 

-- 
Frederic Weisbecker
SUSE Labs

Re: [PATCH RESEND 1/2] tick/nohz: Fix wrong NOHZ idle CPU state

[Christoph Lameter (Ampere)]

On Thu, 4 Sep 2025, Frederic Weisbecker wrote:

> The current state is indeed broken and some people have already tried to fix it.
> The thing is nohz_full don't want dynamic isolation because it is deemed to run a
> single task. Therefore those tasks must be placed manually in order not to break
> isolation guarantees by accident.
>
> In fact nohz_full doesn't make much sense without isolcpus (or isolated cpuset
> v2 partitions) and I even intend to make nohz_full depend on domain isolation
> in the long term.

I have never used isolcpus with nohz_full. AFAICT isolcpus is depreciated
and cpusets are unnecessary complex overhead.

Re: [PATCH RESEND 1/2] tick/nohz: Fix wrong NOHZ idle CPU state

[Frederic Weisbecker]

Le Thu, Sep 04, 2025 at 09:10:29AM -0700, Christoph Lameter (Ampere) a écrit :
> On Thu, 4 Sep 2025, Frederic Weisbecker wrote:
> 
> > The current state is indeed broken and some people have already tried to fix it.
> > The thing is nohz_full don't want dynamic isolation because it is deemed to run a
> > single task. Therefore those tasks must be placed manually in order not to break
> > isolation guarantees by accident.
> >
> > In fact nohz_full doesn't make much sense without isolcpus (or isolated cpuset
> > v2 partitions) and I even intend to make nohz_full depend on domain isolation
> > in the long term.
> 
> I have never used isolcpus with nohz_full. AFAICT isolcpus is depreciated
> and cpusets are unnecessary complex overhead.

isolcpus for domain isolation is indeed in the way for long term deprecation
and the only replacement possible is cpuset, which overhead is only visible
on partition creation and update.

We could argue on the interface, the point is that nohz_full doesn't make sense
without domain isolation.

Thanks.

-- 
Frederic Weisbecker
SUSE Labs

Re: [PATCH RESEND 1/2] tick/nohz: Fix wrong NOHZ idle CPU state

[Christoph Lameter (Ampere)]

On Fri, 5 Sep 2025, Frederic Weisbecker wrote:

> isolcpus for domain isolation is indeed in the way for long term deprecation
> and the only replacement possible is cpuset, which overhead is only visible
> on partition creation and update.
>
> We could argue on the interface, the point is that nohz_full doesn't make sense
> without domain isolation.

Most use cases I see use nohz_full on all cpus and rely on the OS to
exempt the sheperd cpu.

Re: [PATCH RESEND 1/2] tick/nohz: Fix wrong NOHZ idle CPU state

[Shubhang Kaushik]

[-- Attachment #1: Type: text/plain, Size: 2439 bytes --]

Hi Frederic,
On Thu, 4 Sep 2025, Frederic Weisbecker wrote:

>> diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
>> index c527b421c865..b900a120ab54 100644
>> --- a/kernel/time/tick-sched.c
>> +++ b/kernel/time/tick-sched.c
>> @@ -1229,8 +1229,9 @@ void tick_nohz_idle_stop_tick(void)
>>  		ts->idle_sleeps++;
>>  		ts->idle_expires = expires;
>>
>> -		if (!was_stopped && tick_sched_flag_test(ts, TS_FLAG_STOPPED)) {
>> -			ts->idle_jiffies = ts->last_jiffies;
>> +		if (tick_sched_flag_test(ts, TS_FLAG_STOPPED)) {
>> +			if (!was_stopped)
>> +				ts->idle_jiffies = ts->last_jiffies;
>>  			nohz_balance_enter_idle(cpu);
>
> The current state is indeed broken and some people have already tried to fix it.
> The thing is nohz_full don't want dynamic isolation because it is deemed to run a
> single task. Therefore those tasks must be placed manually in order not to break
> isolation guarantees by accident.
>
> In fact nohz_full doesn't make much sense without isolcpus (or isolated cpuset
> v2 partitions) and I even intend to make nohz_full depend on domain isolation
> in the long term.
>
> Thanks.

Following up on the isolation concerns raised previously, I’ve posted an 
updated patch [1] that provides a clearer justification and performance 
data from Ampere Altra.

The core issue identified is that on high core count systems, 
nohz_full CPUs often become stranded idle because they are missing from 
nohz.idle_cpus_mask. While I understand the intent for manual isolation, 
our testing shows that this current behavior leads to significant under 
utilization.

- LLM Workloads: ~14% throughput improvement in llama-batched-bench.
- Scheduler Jitter: ~26% improvement in hackbench multi-process tests.

The patch decouples the tick-stop accounting (which should only happen 
once) from the balancer registration. Because nohz_balance_enter_idle() is 
idempotent, an idle CPU would be visible to the balancer without breaking 
the isolation of other cores that are actually running 
tasks.

If a CPU has entered do_idle(), it is no longer running an isolated 
workload. It is invisible to the balancer at that point leading to 
performance regression rather than an isolation guarantee.

You can find the updated patch and full performance breakdown here:
[1] https://lkml.org/lkml/2026/2/3/2119

Regards,
Shubhang Kaushik
>
>>  		}
>>  	} else {
>> --
>> 2.34.1
>>
>
> -- 
> Frederic Weisbecker
> SUSE Labs
>

[PATCH RESEND 2/2] tick/nohz: Trigger warning when CPU in wrong NOHZ idle state

[Adam Li]

This patch is for debug only.

Warning is triggerred when CPU is in this state:
1) tick was already stopped before tick_nohz_idle_stop_tick()
   stops the tick
2) and CPU is not in nohz.idle_cpus_mask
3) and CPU is idle
4) and tick is stopped

CPU will stay idle in this state, since neither the periodic nor
the NOHZ idle load balancing can move task to this CPU.

Signed-off-by: Adam Li <adamli@os.amperecomputing.com>
---
 include/linux/sched/nohz.h | 2 ++
 kernel/sched/fair.c        | 5 +++++
 kernel/time/tick-sched.c   | 3 ++-
 3 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/include/linux/sched/nohz.h b/include/linux/sched/nohz.h
index 0db7f67935fe..ea6e07777395 100644
--- a/include/linux/sched/nohz.h
+++ b/include/linux/sched/nohz.h
@@ -9,8 +9,10 @@
 #ifdef CONFIG_NO_HZ_COMMON
 extern void nohz_balance_enter_idle(int cpu);
 extern int get_nohz_timer_target(void);
+extern bool nohz_balance_idle_cpu(int cpu);
 #else
 static inline void nohz_balance_enter_idle(int cpu) { }
+static inline bool nohz_balance_idle_cpu(int cpu) { return false; }
 #endif
 
 #ifdef CONFIG_NO_HZ_COMMON
diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index b173a059315c..cd1c17368e05 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -7109,6 +7109,11 @@ static struct {
 	unsigned long next_blocked;	/* Next update of blocked load in jiffies */
 } nohz ____cacheline_aligned;
 
+inline bool nohz_balance_idle_cpu(int cpu)
+{
+	return cpumask_test_cpu(cpu, nohz.idle_cpus_mask);
+}
+
 #endif /* CONFIG_NO_HZ_COMMON */
 
 static unsigned long cpu_load(struct rq *rq)
diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index b900a120ab54..8241b14842f3 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -1228,7 +1228,8 @@ void tick_nohz_idle_stop_tick(void)
 
 		ts->idle_sleeps++;
 		ts->idle_expires = expires;
-
+		WARN_ON_ONCE(was_stopped && !nohz_balance_idle_cpu(cpu) &&
+			idle_cpu(cpu) && tick_nohz_tick_stopped_cpu(cpu));
 		if (tick_sched_flag_test(ts, TS_FLAG_STOPPED)) {
 			if (!was_stopped)
 				ts->idle_jiffies = ts->last_jiffies;
-- 
2.34.1

Re: [PATCH RESEND 2/2] tick/nohz: Trigger warning when CPU in wrong NOHZ idle state

[kernel test robot]

Hello,


we are not sure the purpose of this debug patch so not sure if below report
could supply any useful information. just report FYI. if low value, please
just kindly ignore. thanks


kernel test robot noticed "WARNING:at_kernel/time/tick-sched.c:#tick_nohz_idle_stop_tick" on:

commit: d112a298a9368568686e1a399cc5073a02f60c2f ("[PATCH RESEND 2/2] tick/nohz: Trigger warning when CPU in wrong NOHZ idle state")
url: https://github.com/intel-lab-lkp/linux/commits/Adam-Li/tick-nohz-Fix-wrong-NOHZ-idle-CPU-state/20250821-122906
base: https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git 1b5f1454091e9e9fb5c944b3161acf4ec0894d0d
patch link: https://lore.kernel.org/all/20250821042707.62993-3-adamli@os.amperecomputing.com/
patch subject: [PATCH RESEND 2/2] tick/nohz: Trigger warning when CPU in wrong NOHZ idle state

in testcase: cpu-hotplug
version: 
with following parameters:


config: x86_64-rhel-9.4-func
compiler: gcc-12
test machine: 8 threads 1 sockets Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz (Kaby Lake) with 32G memory

(please refer to attached dmesg/kmsg for entire log/backtrace)



If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <oliver.sang@intel.com>
| Closes: https://lore.kernel.org/oe-lkp/202509031529.464ff656-lkp@intel.com


[   96.913433][    T0] ------------[ cut here ]------------
[   43.020727][  T307]
[   87.665815][    T0] Masked ExtINT on CPU#3
[   43.030461][  T307] (Reading database ... 16595 files and directories currently installed.)
[   89.314224][  T252] smpboot: CPU 1 is now offline
[ 96.918719][ T0] WARNING: CPU: 0 PID: 0 at kernel/time/tick-sched.c:1231 tick_nohz_idle_stop_tick (kernel/time/tick-sched.c:1231) 
[   43.030468][  T307]
[   87.817541][  T649] smpboot: Booting Node 0 Processor 4 APIC 0x1
[   43.042435][  T307] Preparing to unpack .../ntpdate_1%3a4.2.8p15+dfsg-2~1.2.2+dfsg1-1+deb12u1_all.deb ...
[   90.352309][  T252] smpboot: CPU 2 is now offline
[   43.042441][  T307]
[   96.928830][    T0] Modules linked in: snd_hda_codec_hdmi snd_ctl_led snd_hda_codec_realtek snd_hda_codec_generic snd_hda_scodec_component btrfs intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common blake2b_generic xor zstd_compress snd_soc_avs x86_pkg_temp_thermal snd_soc_hda_codec snd_hda_ext_core intel_powerclamp raid6_pq snd_soc_core coretemp snd_compress i915 sd_mod snd_hda_intel sg snd_intel_dspcfg kvm_intel intel_gtt ipmi_devintf snd_intel_sdw_acpi ipmi_msghandler cec snd_hda_codec drm_buddy kvm snd_hda_core ttm mei_wdt irqbypass snd_hwdep i2c_designware_platform ghash_clmulni_intel snd_pcm dell_wmi platform_profile ahci rapl drm_display_helper i2c_designware_core mei_me dcdbas libahci dell_wmi_aio snd_timer dell_smbios intel_cstate drm_client_lib dell_smm_hwmon wmi_bmof dell_wmi_descriptor sparse_keymap intel_lpss_pci snd drm_kms_helper i2c_i801 intel_lpss libata pcspkr intel_uncore mei idma64 soundcore i2c_smbus video intel_pmc_core pmt_telemetry pmt_class wmi pinctrl_sunrisepoint
[   87.823787][    T0] Masked ExtINT on CPU#4
[   43.055242][  T307] Unpacking ntpdate (1:4.2.8p15+dfsg-2~1.2.2+dfsg1-1+deb12u1) ...
[   91.391058][  T252] smpboot: CPU 3 is now offline
[   96.928957][    T0]  intel_pmc_ssram_telemetry intel_vsec acpi_pad binfmt_misc drm fuse loop dm_mod ip_tables
[   43.055248][  T307]
[   87.976451][  T649] smpboot: Booting Node 0 Processor 5 APIC 0x3
[   43.065976][  T307] Selecting previously unselected package ntpsec-ntpdate.
[   92.424836][  T252] smpboot: CPU 4 is now offline
[   97.028568][    T0] CPU: 0 UID: 0 PID: 0 Comm: swapper/0 Tainted: G          I         6.16.0-rc6-00089-gd112a298a936 #1 PREEMPT(voluntary)
[   43.065982][  T307]
[   87.982768][    T0] Masked ExtINT on CPU#5
[   43.076378][  T307] Preparing to unpack .../ntpsec-ntpdate_1.2.2+dfsg1-1+deb12u1_amd64.deb ...
[   97.041089][    T0] Tainted: [I]=FIRMWARE_WORKAROUND
[   93.454857][  T252] smpboot: CPU 5 is now offline
[   43.076384][  T307]
[   88.118542][  T649] smpboot: Booting Node 0 Processor 6 APIC 0x5
[   43.087987][  T307] Unpacking ntpsec-ntpdate (1.2.2+dfsg1-1+deb12u1) ...
[   94.507924][  T252] smpboot: CPU 6 is now offline
[   97.046019][    T0] Hardware name: Dell Inc. OptiPlex 7050/062KRH, BIOS 1.2.0 12/22/2016
[   43.087993][  T307]
[   88.124813][    T0] Masked ExtINT on CPU#6
[   43.097723][  T307] Selecting previously unselected package python3-ntp.
[   95.537790][  T252] smpboot: CPU 7 is now offline
[ 97.054053][ T0] RIP: 0010:tick_nohz_idle_stop_tick (kernel/time/tick-sched.c:1231) 
[   43.097729][  T307]
[   88.268535][  T649] smpboot: Booting Node 0 Processor 7 APIC 0x7
[   43.107761][  T307] Preparing to unpack .../python3-ntp_1.2.2+dfsg1-1+deb12u1_amd64.deb ...
[ 97.060194][ T0] Code: e0 5d f6 84 48 b8 00 00 00 00 00 fc ff df 4c 89 e2 48 c1 ea 03 80 3c 02 00 0f 85 b7 00 00 00 41 f6 04 24 02 0f 84 63 ff ff ff <0f> 0b e9 5c ff ff ff 48 b8 00 00 00 00 00 fc ff df 48 89 da 48 c1
All code
========
   0:	e0 5d                	loopne 0x5f
   2:	f6 84 48 b8 00 00 00 	testb  $0x0,0xb8(%rax,%rcx,2)
   9:	00 
   a:	00 fc                	add    %bh,%ah
   c:	ff                   	(bad)
   d:	df 4c 89 e2          	fisttps -0x1e(%rcx,%rcx,4)
  11:	48 c1 ea 03          	shr    $0x3,%rdx
  15:	80 3c 02 00          	cmpb   $0x0,(%rdx,%rax,1)
  19:	0f 85 b7 00 00 00    	jne    0xd6
  1f:	41 f6 04 24 02       	testb  $0x2,(%r12)
  24:	0f 84 63 ff ff ff    	je     0xffffffffffffff8d
  2a:*	0f 0b                	ud2		<-- trapping instruction
  2c:	e9 5c ff ff ff       	jmp    0xffffffffffffff8d
  31:	48 b8 00 00 00 00 00 	movabs $0xdffffc0000000000,%rax
  38:	fc ff df 
  3b:	48 89 da             	mov    %rbx,%rdx
  3e:	48                   	rex.W
  3f:	c1                   	.byte 0xc1

Code starting with the faulting instruction
===========================================
   0:	0f 0b                	ud2
   2:	e9 5c ff ff ff       	jmp    0xffffffffffffff63
   7:	48 b8 00 00 00 00 00 	movabs $0xdffffc0000000000,%rax
   e:	fc ff df 
  11:	48 89 da             	mov    %rbx,%rdx
  14:	48                   	rex.W
  15:	c1                   	.byte 0xc1
[   95.560557][  T649] smpboot: Booting Node 0 Processor 1 APIC 0x2
[   43.107767][  T307]
[   88.274805][    T0] Masked ExtINT on CPU#7
[   43.119061][  T307] Unpacking python3-ntp (1.2.2+dfsg1-1+deb12u1) ...
[   95.566801][    T0] Masked ExtINT on CPU#1
[   97.079531][    T0] RSP: 0018:ffffffff85207df0 EFLAGS: 00010002
[   43.119067][  T307]
[   89.314224][  T252] smpboot: CPU 1 is now offline
[   43.128509][  T307] Setting up python3-ntp (1.2.2+dfsg1-1+deb12u1) ...
[   95.591432][  T649] smpboot: Booting Node 0 Processor 2 APIC 0x4
[   97.085419][    T0] RAX: dffffc0000000000 RBX: ffff888796430c40 RCX: 0000000000000000
[   43.128515][  T307]
[   90.352309][  T252] smpboot: CPU 2 is now offline
[   43.138294][  T307] Setting up ntpdate (1:4.2.8p15+dfsg-2~1.2.2+dfsg1-1+deb12u1) ...
[   95.597695][    T0] Masked ExtINT on CPU#2
[   97.093208][    T0] RDX: 1ffff110f2c86188 RSI: 0000000000000008 RDI: ffffffff84f65de0
[   43.138300][  T307]
[   91.391058][  T252] smpboot: CPU 3 is now offline
[   43.149075][  T307] Setting up ntpsec-ntpdate (1.2.2+dfsg1-1+deb12u1) ...
[   95.625417][  T649] smpboot: Booting Node 0 Processor 3 APIC 0x6
[   97.100997][    T0] RBP: ffff88880f074000 R08: 0000000000000000 R09: ffffed10219177b8
[   43.149081][  T307]
[   92.424836][  T252] smpboot: CPU 4 is now offline
[   43.158251][  T307] NO_NETWORK=
[   97.108775][    T0] R10: ffff88810c8bbdc7 R11: ffff888796430c70 R12: ffff888796430c40
[   95.631679][    T0] Masked ExtINT on CPU#3
[   43.158257][  T307]
[   93.454857][  T252] smpboot: CPU 5 is now offline
[   43.164205][  T307] CLOCK: time stepped by 32064468.825994
[   95.675523][  T649] smpboot: Booting Node 0 Processor 4 APIC 0x1
[   97.116551][    T0] R13: 0000000000000000 R14: 0000000000000000 R15: 000000081dff1000
[   43.164212][  T307]
[   94.507924][  T252] smpboot: CPU 6 is now offline
[   43.172683][  T307] CLOCK: time changed from 2024-08-25 to 2025-08-31
[   97.124328][    T0] FS:  0000000000000000(0000) GS:ffff88880f074000(0000) knlGS:0000000000000000
[   95.681842][    T0] Masked ExtINT on CPU#4
[   43.172690][  T307]
[   95.537790][  T252] smpboot: CPU 7 is now offline
[   43.183164][  T307] 2025-08-31 20:23:30.162487 (+0000) +32064468.825994 +/- 0.000310 internal-lkp-server 192.168.1.200 s5 no-leap
[   95.721433][  T649] smpboot: Booting Node 0 Processor 5 APIC 0x3
[   97.133051][    T0] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   43.183170][  T307]
[   95.560557][  T649] smpboot: Booting Node 0 Processor 1 APIC 0x2
[   43.221647][   T10] smpboot: CPU 6 is now offline
[   97.139448][    T0] CR2: 00007fe2bc1c1e90 CR3: 000000081aa72006 CR4: 00000000003726f0
[   95.727679][    T0] Masked ExtINT on CPU#5
[   44.252774][   T10] smpboot: CPU 7 is now offline
[   95.566801][    T0] Masked ExtINT on CPU#1
[   44.289262][  T649] smpboot: Booting Node 0 Processor 1 APIC 0x2
[   97.147225][    T0] Call Trace:
[   95.761430][  T649] smpboot: Booting Node 0 Processor 6 APIC 0x5
[   44.301168][    T0] Masked ExtINT on CPU#1
[   95.591432][  T649] smpboot: Booting Node 0 Processor 2 APIC 0x4
[   44.323288][  T649] smpboot: Booting Node 0 Processor 2 APIC 0x4
[   97.150345][    T0]  <TASK>
[   44.329546][    T0] Masked ExtINT on CPU#2
[   95.767675][    T0] Masked ExtINT on CPU#6
[   95.597695][    T0] Masked ExtINT on CPU#2
[   44.357295][  T649] smpboot: Booting Node 0 Processor 3 APIC 0x6
[ 97.153121][ T0] cpuidle_idle_call (arch/x86/include/asm/current.h:25 include/linux/sched/idle.h:33 include/linux/sched/idle.h:67 kernel/sched/idle.c:149 kernel/sched/idle.c:235) 
[   95.808544][  T649] smpboot: Booting Node 0 Processor 7 APIC 0x7
[   44.363568][    T0] Masked ExtINT on CPU#3
[   95.625417][  T649] smpboot: Booting Node 0 Processor 3 APIC 0x6
[   44.402300][  T649] smpboot: Booting Node 0 Processor 4 APIC 0x1
[ 97.157882][ T0] ? __pfx_cpuidle_idle_call (kernel/sched/idle.c:173) 
[   95.814793][    T0] Masked ExtINT on CPU#7
[   44.408556][    T0] Masked ExtINT on CPU#4
[   95.631679][    T0] Masked ExtINT on CPU#3
[   44.445282][  T649] smpboot: Booting Node 0 Processor 5 APIC 0x3
[ 97.163158][ T0] ? tsc_verify_tsc_adjust (arch/x86/kernel/tsc_sync.c:60) 
[   95.965121][  T252] smpboot: CPU 1 is now offline
[   44.451534][    T0] Masked ExtINT on CPU#5
[   95.675523][  T649] smpboot: Booting Node 0 Processor 4 APIC 0x1
[   44.491303][  T649] smpboot: Booting Node 0 Processor 6 APIC 0x5
[ 97.168350][ T0] do_idle (kernel/sched/idle.c:330) 
[   96.109316][  T252] smpboot: CPU 2 is now offline
[   44.497553][    T0] Masked ExtINT on CPU#6
[   95.681842][    T0] Masked ExtINT on CPU#4
[   44.536391][  T649] smpboot: Booting Node 0 Processor 7 APIC 0x7
[ 97.172160][ T0] cpu_startup_entry (kernel/sched/idle.c:427 (discriminator 1)) 
[   96.244243][  T252] smpboot: CPU 3 is now offline
[   44.542639][    T0] Masked ExtINT on CPU#7
[   95.721433][  T649] smpboot: Booting Node 0 Processor 5 APIC 0x3
[   44.699078][   T10] smpboot: CPU 1 is now offline
[ 97.176746][ T0] rest_init (init/main.c:718) 
[   96.397668][  T252] smpboot: CPU 4 is now offline


The kernel config and materials to reproduce are available at:
https://download.01.org/0day-ci/archive/20250903/202509031529.464ff656-lkp@intel.com



-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki