From: "John B. Wyatt IV" <jwyatt@redhat.com>
To: John Ogness <john.ogness@linutronix.de>
Cc: Petr Mladek <pmladek@suse.com>,
Clark Williams <williams@redhat.com>,
jlelli@redhat.com, Derek Barbosa <debarbos@redhat.com>,
"John B. Wyatt IV" <sageofredondo@gmail.com>,
LKML <linux-kernel@vger.kernel.org>,
linux-rt-users <linux-rt-users@vger.kernel.org>
Subject: Re: NMI Reported with console_blast.sh
Date: Thu, 22 Feb 2024 00:21:19 -0500 [thread overview]
Message-ID: <ZdbZz-v_lug9_t6C@thinkpad2021> (raw)
In-Reply-To: <ZcqMOKtHsZ9qnxNg@thinkpad2021>
On Mon, Feb 12, 2024 at 04:23:04PM -0500, John B. Wyatt IV wrote:
>
> Red Hat's conservers are having an issue with the machine I was testing
> on. It may take me a while to get back to you with more test results.
>
Found a work-around with conserver. I was able to follow up on the printk-caller
info you requested.
I found 2 additional NMIs for a total of 3. Number 2 is very
large-please feel free to let me know what specific information you
wanted if it was unnecessary.
This first one (the original) is with the real-time profile (no preemption).
The next two are with the performance-througput (no preemption).
This is what real-time NMI reports with the caller information enabled. It
looks like it is lacking some information compared to the other two further below.
[ T2481] Call Trace:
[ T2477] Kernel panic - not syncing: sysrq triggered crash
[ C0] NMI backtrace for cpu 0
[ C0] Hardware name: Intel Corporation D50DNP1SBB/D50DNP1SBB, BIOS SE5C7411.86B.9409.D04.2212261349 12/26/2022
[ C0] RIP: 0010:io_serial_in (arch/x86/kernel/early_printk.c:100)
[ C0] Code: cc 0f 1f 40 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 0f 1f 44 00 00 0f b6 8f c1 00 00 00 0f b7 57 08 d3 e6 01 f2 ec <0f> b6 c0 c3 cc cc cc cc 0f 1f 00 90 90 90 90 90 90 90 90 90 90 90
All code
========
0: cc int3
1: 0f 1f 40 00 nopl 0x0(%rax)
5: 90 nop
6: 90 nop
7: 90 nop
8: 90 nop
9: 90 nop
a: 90 nop
b: 90 nop
c: 90 nop
d: 90 nop
e: 90 nop
f: 90 nop
10: 90 nop
11: 90 nop
12: 90 nop
13: 90 nop
14: 90 nop
15: 0f 1f 44 00 00 nopl 0x0(%rax,%rax,1)
1a: 0f b6 8f c1 00 00 00 movzbl 0xc1(%rdi),%ecx
21: 0f b7 57 08 movzwl 0x8(%rdi),%edx
25: d3 e6 shl %cl,%esi
27: 01 f2 add %esi,%edx
29: ec in (%dx),%al
2a:* 0f b6 c0 movzbl %al,%eax <-- trapping instruction
2d: c3 ret
2e: cc int3
2f: cc int3
30: cc int3
31: cc int3
32: 0f 1f 00 nopl (%rax)
35: 90 nop
36: 90 nop
37: 90 nop
38: 90 nop
39: 90 nop
3a: 90 nop
3b: 90 nop
3c: 90 nop
3d: 90 nop
3e: 90 nop
3f: 90 nop
Code starting with the faulting instruction
===========================================
0: 0f b6 c0 movzbl %al,%eax
3: c3 ret
4: cc int3
5: cc int3
6: cc int3
7: cc int3
8: 0f 1f 00 nopl (%rax)
b: 90 nop
c: 90 nop
d: 90 nop
e: 90 nop
f: 90 nop
10: 90 nop
11: 90 nop
12: 90 nop
13: 90 nop
14: 90 nop
15: 90 nop
[ C0] RSP: 0018:ff4507d95ec1fac0 EFLAGS: 00000002
[ C0] RAX: ffffffffa13d6f00 RBX: 0000000000000000 RCX: 0000000000000000
[ C0] RDX: 00000000000003fd RSI: 0000000000000005 RDI: ffffffffa3ad49c0
[ C0] RBP: 000000000000270f R08: ffffffffa39c73b3 R09: ff4507d95ec1fa1f
[ C0] R10: 0000000031383432 R11: 0000000038343254 R12: 0000000000000020
[ C0] R13: ffffffffa3ad49c0 R14: 0000000000000000 R15: 0000000000000000
[ C0] FS: 00007fd6cce81740(0000) GS:ff1ef00bbf800000(0000) knlGS:0000000000000000
[ C0] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ C0] CR2: 00007fd6cd05d650 CR3: 000000208e5bc004 CR4: 0000000000771ef0
[ C0] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ C0] DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400
[ C0] PKRU: 55555554
[ C0] Call Trace:
[ C0] <NMI>
[ C0] ? nmi_cpu_backtrace (lib/nmi_backtrace.c:115)
[ C0] ? nmi_cpu_backtrace_handler (arch/x86/kernel/apic/hw_nmi.c:47 (discriminator 1))
[ C0] ? nmi_handle (arch/x86/kernel/nmi.c:150)
[ C0] ? io_serial_in (arch/x86/kernel/early_printk.c:100)
[ C0] ? default_do_nmi (arch/x86/kernel/nmi.c:351)
[ C0] ? exc_nmi (arch/x86/kernel/nmi.c:545)
[ C0] ? end_repeat_nmi (arch/x86/entry/entry_64.S:1394)
[ C0] ? mem_serial_out (drivers/tty/serial/8250/8250_port.c:368)
[ C0] ? io_serial_in (arch/x86/kernel/early_printk.c:100)
[ C0] ? io_serial_in (arch/x86/kernel/early_printk.c:100)
[ C0] ? io_serial_in (arch/x86/kernel/early_printk.c:100)
[ C0] </NMI>
[ C0] <TASK>
------------------------------------
Compared to the two NMIs with throughput-performance (no preemption)
<NMI>
cpus=0
.runnable_avg : 3072
kthread (kernel/kthread.c:388)
.util_est_enqueued : 0
stack:0 pid:1733 tgid:1733 ppid:2 flags:0x00004000
.min_vruntime : 2084315.290254
.removed.load_avg : 0
.avg_vruntime : 2084315.290254
console_blast.s 3497 34770.405603 N 34773.405603 3.000000 34764.898340 4 120
.util_avg : 1024
.util_avg : 1024
.util_est_enqueued : 0
.left_vruntime : 0.000001
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
.removed.util_avg : 0
worker_thread (kernel/workqueue.c:2734)
0.000000 34764.898340 0.000000 0.000000
? __pfx_kthread (kernel/kthread.c:341)
.min_vruntime : 3166003.080642
.se->vruntime : 4421649.079833
Call Trace:
.spread : 0.000000
RDX: 00007fffb4d634a0 RSI: 0000000000000000 RDI: 0000000000000000
.removed.load_avg : 0
<TASK>
smpboot_thread_fn (kernel/smpboot.c:160)
.right_vruntime : 0.000001
.removed.runnable_avg : 0
.avg_vruntime : 3166003.080642
ret_from_fork (arch/x86/kernel/process.c:147)
.util_est_enqueued : 1576
</TASK>
.se->sum_exec_runtime : 34772.922820
.right_vruntime : 0.000001
.curr->pid : 0
.removed.load_avg : 0
.removed.util_avg : 0
.removed.load_avg : 0
.tg_load_avg_contrib : 2042
/user.slice/user-0.slice/session-1.scope
.spread : 0.000000
.removed.runnable_avg : 0
.removed.util_avg : 0
.se->load.weight : 8798
ret_from_fork (arch/x86/kernel/process.c:147)
.se->avg.load_avg : 7
task:kworker/162:1 state:I
I
.removed.runnable_avg : 0
.tg_load_avg_contrib : 8
? preempt_count_add (./include/linux/ftrace.h:977 kernel/sched/core.c:5839 kernel/sched/core.c:5836 kernel/sched/core.c:5864)
ret_from_fork (arch/x86/kernel/process.c:147)
.spread : 0.000000
.tg_load_avg_contrib : 4
? __pfx_kthread (kernel/kthread.c:341)
.removed.util_avg : 0
kworker/u453:14 1537 8085.292524 E 8088.278508 3.000000 48.419745 146 120
<TASK>
.tg_load_avg : 178559
.nr_spread_over : 0
S
.tg_load_avg : 1488
.tg_load_avg : 1488
console_blast.s 3717 6.183406 E 8.771080 3.000000 3.211160 4 120
.nr_spread_over : 0
.throttled : 0
.removed.runnable_avg : 0
.nr_running : 1
.throttled : 0
.nr_running : 1
.throttled : 0
.se->avg.util_avg : 1024
.throttle_count : 0
.throttle_count : 0
.h_nr_running : 3
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
0.000000 3.211160 0.000000 0.000000
.tg_load_avg_contrib : 0
.idle_nr_running : 0
RBP: 00007fffb4d63450 R08: 0000000000000000 R09: 0000000000000000
worker_thread (kernel/workqueue.c:2734)
.throttle_count : 0
/user.slice/user-0.slice/session-1.scope
.se->avg.runnable_avg : 3072
.se->exec_start : 208059.574049
R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000001
.h_nr_running : 2
.se->exec_start : 208050.612936
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
.tg_load_avg : 0
.se->exec_start : 208020.052027
.idle_h_nr_running : 0
.throttled : 0
.idle_nr_running : 0
.se->vruntime : 11608324.604177
.throttle_count : 0
.load : 18045
.se->vruntime : 7056777.768657
? __pfx_kthread (kernel/kthread.c:341)
.idle_h_nr_running : 0
.se->sum_exec_runtime : 34774.514688
S
cfs_rq[25]:/user.slice
? __pfx_kthread (kernel/kthread.c:341)
0.000000 48.419745 0.000000 0.000000
.se->sum_exec_runtime : 34430.603888
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
ret_from_fork (arch/x86/kernel/process.c:147)
.se->vruntime : 3166003.080642
? __pfx_kthread (kernel/kthread.c:341)
.load : 12026
? __pfx_smpboot_thread_fn (kernel/smpboot.c:107)
? __pfx_kthread (kernel/kthread.c:341)
.exec_clock : 0.000000
.se->load.weight : 5637
.se->load.weight : 3523
console_blast.s 3909 7.636665 E 10.235762 3.000000 3.107414 4 120
.load_avg : 11
kthread (kernel/kthread.c:388)
.load_avg : 17
(mm_percpu_wq)
.left_vruntime : 0.000001
.se->sum_exec_runtime : 34424.300264
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
rt_rq[27]:
0.000000 0.005913 0.000000 0.000000
.runnable_avg : 3073
.min_vruntime : 4421649.079833
.rt_nr_running : 0
.se->avg.load_avg : 2
.clock : 208093.933560
node=0
0.000000 3.107414 0.000000 0.000000
.runnable_avg : 2047
.se->avg.load_avg : 4
? __pfx_smpboot_thread_fn (kernel/smpboot.c:107)
.se->avg.util_avg : 1023
.util_avg : 1023
.rt_throttled : 0
.avg_vruntime : 4421649.079833
.nr_spread_over : 0
ret_from_fork (arch/x86/kernel/process.c:147)
? __pfx_kthread (kernel/kthread.c:341)
? __pfx_worker_thread (kernel/workqueue.c:2727)
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
.util_avg : 1024
.se->load.weight : 12026
.se->avg.runnable_avg : 2047
? _raw_spin_unlock (./arch/x86/include/asm/preempt.h:104 ./include/linux/spinlock_api_smp.h:143 kernel/locking/spinlock.c:186)
/user.slice/user-0.slice/session-1.scope
.rt_time : 0.000000
ret_from_fork (arch/x86/kernel/process.c:147)
.util_est_enqueued : 0
.right_vruntime : 0.000001
? __pfx_kthread (kernel/kthread.c:341)
S
.util_est_enqueued : 0
.rt_runtime : 950.000000
.se->avg.load_avg : 10
.clock_task : 208036.282491
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
.se->avg.util_avg : 1024
console_blast.s 4010 4.594964 E 7.189746 3.000000 2.839766 4 120
.removed.load_avg : 0
.spread : 0.000000
.se->avg.util_avg : 1023
.removed.load_avg : 0
.removed.util_avg : 0
? preempt_count_add (./include/linux/ftrace.h:977 kernel/sched/core.c:5839 kernel/sched/core.c:5836 kernel/sched/core.c:5864)
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
? __pfx_kthread (kernel/kthread.c:341)
.removed.runnable_avg : 0
.nr_spread_over : 0
smpboot_thread_fn (kernel/smpboot.c:160)
.tg_load_avg_contrib : 17
Workqueue: 0x0
.se->avg.runnable_avg : 3072
cfs_rq[50]:/
task:kworker/u453:19 state:I
dl_rq[27]:
.tg_load_avg : 2026
.removed.util_avg : 0
flags=0x0 nice=0
/
.dl_nr_running : 0
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
0.000000 2.839766 0.000000 0.000000
.dl_bw->bw : 996147
</TASK>
.throttled : 0
.removed.runnable_avg : 0
? __pfx_smpboot_thread_fn (kernel/smpboot.c:107)
.exec_clock : 0.000000
.se->avg.runnable_avg : 2047
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
.nr_running : 1
cfs_rq[16]:/
/user.slice/user-0.slice/session-1.scope
R10: 00007fffb4d63490 R11: 0000000000000202 R12: 00007f4a03e696c8
.tg_load_avg_contrib : 11
ret_from_fork (arch/x86/kernel/process.c:147)
.h_nr_running : 3
.exec_clock : 0.000000
smpboot_thread_fn (kernel/smpboot.c:160)
.throttle_count : 0
.left_vruntime : 268170.491452
RSP: 002b:00007ffd3c9630d8 EFLAGS: 00000202
stack:0 pid:1563 tgid:1563 ppid:2 flags:0x00004000
.left_vruntime : 64498.686713
.tg_load_avg : 2026
.se->exec_start : 208007.448274
Call Trace:
kthread (kernel/kthread.c:388)
.min_vruntime : 268170.906344
.se->vruntime : 4438283.868821
.nr_running : 1
.throttled : 0
.idle_nr_running : 0
.dl_bw->total_bw : 0
active=1/256 refcnt=2
.min_vruntime : 64498.686713
.throttle_count : 0
kthread (kernel/kthread.c:388)
RIP: 0033:0x7fcfe00a4d07
? __pfx_kthread (kernel/kthread.c:341)
? __pfx_kthread (kernel/kthread.c:341)
.avg_vruntime : 289098.917351
.avg_idle : 4953240
S
.se->sum_exec_runtime : 34382.705140
do_syscall_64 (arch/x86/entry/common.c:52 arch/x86/entry/common.c:83)
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
I
runnable tasks:
cfs_rq[53]:/user.slice/user-0.slice
.se->exec_start : 208059.574049
.se->load.weight : 8798
console_blast.s 4084 5.771080 E 8.366011 3.000000 2.424395 3 120
RSP: 002b:00007ffec60d77c8 EFLAGS: 00000202
pending:
.se->avg.load_avg : 7
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
? __pfx_kthread (kernel/kthread.c:341)
RSP: 002b:00007ffe0b913168 EFLAGS: 00000202
.avg_vruntime : 82619.613072
.exec_clock : 0.000000
vmstat_update
0.000000 2.424395 0.000000 0.000000
ORIG_RAX: 00000000000000e6
.idle_h_nr_running : 0
.left_vruntime : 0.000001
kthread (kernel/kthread.c:388)
/user.slice/user-0.slice/session-1.scope
? __pfx_kthread (kernel/kthread.c:341)
.right_vruntime : 66687.032812
.se->vruntime : 6969082.950841
.se->avg.util_avg : 1024
.right_vruntime : 276865.549246
</TASK>
.spread : 2188.346099
task:kcompactd2 state:S
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
Workqueue: 0x0
.nr_spread_over : 0
R13: 00007fffb4d63490 R14: 0000000000000002 R15: 00007fffb4d63490
.se->sum_exec_runtime : 34424.300264
<TASK>
ret_from_fork (arch/x86/kernel/process.c:147)
.spread : 8695.057794
.min_vruntime : 3166003.080642
kworker/169:0H 1036 184.779405 E 184.813648 3.000000 0.051581 4 100
.load : 8798
.se->load.weight : 5693
? _raw_spin_lock_irqsave (./arch/x86/include/asm/atomic.h:115 (discriminator 4) ./include/linux/atomic/atomic-arch-fallback.h:2164 (discriminator 4) ./include/linux/atomic/atomic-instrumented.h:1296 (discriminator 4) ./include/asm-generic/qspinlock.h:111 (discriminator 4) ./include/linux/spinlock.h:187 (discriminator 4) ./include/linux/spinlock_api_smp.h:111 (discriminator 4) kernel/locking/spinlock.c:162 (discriminator 4))
.avg_vruntime : 3166003.080642
.load_avg : 8
? __pfx_kthread (kernel/kthread.c:341)
(events_unbound)
.nr_running : 3
.h_nr_running : 3
R
task:kworker/27:1 state:R
</TASK>
? preempt_count_add (./include/linux/ftrace.h:977 kernel/sched/core.c:5839 kernel/sched/core.c:5836 kernel/sched/core.c:5864)
? __pfx_worker_thread (kernel/workqueue.c:2727)
.right_vruntime : 0.000001
.max_idle_balance_cost : 2418910
</TASK>
? __pfx_kthread (kernel/kthread.c:341)
smpboot_thread_fn (kernel/smpboot.c:160)
.se->avg.runnable_avg : 3073
.h_nr_running : 5
sleep 4214 767.966687 E 770.966687 3.000000 1.092834 1 120
S task PID tree-key switches prio wait-time sum-exec sum-sleep
ORIG_RAX: 000000000000003d
.se->avg.load_avg : 4
.nr_spread_over : 0
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
.se->avg.util_avg : 1023
.runnable_avg : 3072
.spread : 0.000000
-------------------------------------------------------------------------------------------------------------
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
ret_from_fork (arch/x86/kernel/process.c:147)
task:idle_inject/127 state:S
.util_avg : 1024
0.000000 1.092834 0.000000 0.000000
.idle_nr_running : 0
.nr_spread_over : 0
cfs_rq[52]:/user.slice
task:ksoftirqd/10 state:S
.se->avg.runnable_avg : 2047
.idle_nr_running : 0
/user.slice/user-0.slice/session-1.scope
.util_est_enqueued : 0
.nr_running : 3
.exec_clock : 0.000000
.removed.load_avg : 0
.h_nr_running : 4
.idle_h_nr_running : 0
S
.idle_nr_running : 0
cfs_rq[53]:/user.slice
R
.exec_clock : 0.000000
cpuhp/27 179 492.490190 E 495.485947 3.000000 1.582983 35 120
.nr_running : 1
.load : 2102789
.removed.util_avg : 0
.idle_h_nr_running : 0
sleep 4276 1145.889782 E 1148.889782 3.000000 0.995931 2 120
.removed.runnable_avg : 0
.left_vruntime : 0.000001
.load_avg : 2053
.left_vruntime : 0.000001
.runnable_avg : 5120
.util_avg : 1024
0.000000 1.582983 0.000000 0.000000
.h_nr_running : 2
</TASK>
? handle_mm_fault (mm/memory.c:5222 mm/memory.c:5361)
RAX: ffffffffffffffda RBX: 00007ffec60d7820 RCX: 00007f008c94c167
0.000000 0.995931 0.000000 0.000000
.min_vruntime : 6969082.950841
.idle_nr_running : 0
.min_vruntime : 4438565.660308
.util_est_enqueued : 1342
.idle_h_nr_running : 0
.tg_load_avg_contrib : 8
/user.slice/user-0.slice/session-1.scope
.avg_vruntime : 4438565.660308
/
.load : 2100675
.removed.load_avg : 0
ret_from_fork (arch/x86/kernel/process.c:147)
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
.right_vruntime : 0.000001
running task
.removed.util_avg : 0
.avg_vruntime : 6969082.950841
worker_thread (kernel/workqueue.c:2734)
stack:0 pid:80 tgid:80 ppid:2 flags:0x00004000
S
.right_vruntime : 0.000001
.removed.runnable_avg : 0
.tg_load_avg : 1488
idle_inject/27 180 -1.045772 E 1.951454 3.000000 0.007012 3 49
.load_avg : 2050
RDX: 00007ffec60d7820 RSI: 0000000000000000 RDI: 0000000000000000
? __pfx_kthread (kernel/kthread.c:341)
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
task:kworker/38:0H state:I
.spread : 0.000000
.spread : 0.000000
.tg_load_avg_contrib : 0
? __pfx_rescuer_thread (kernel/workqueue.c:2822)
0.000000 0.007012 0.000000 0.000000
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
(mm_percpu_wq)
.idle_h_nr_running : 0
.load : 12026
? __pfx_kthread (kernel/kthread.c:341)
Workqueue: 0x0
/
stack:0 pid:780 tgid:780 ppid:2 flags:0x00004000
? __pfx_kthread (kernel/kthread.c:341)
.throttled : 0
? __pfx_kthread (kernel/kthread.c:341)
.tg_load_avg : 0
.load_avg : 11
.nr_spread_over : 0
.throttle_count : 0
cpu#20, 1900.000 MHz
? preempt_count_add (./include/linux/ftrace.h:977 kernel/sched/core.c:5839 kernel/sched/core.c:5836 kernel/sched/core.c:5864)
</TASK>
.throttled : 0
.throttle_count : 0
.runnable_avg : 4095
? __pfx_kthread (kernel/kthread.c:341)
ret_from_fork (arch/x86/kernel/process.c:147)
? preempt_count_add (./include/linux/ftrace.h:974 (discriminator 1) kernel/sched/core.c:5839 (discriminator 1) kernel/sched/core.c:5836 (discriminator 1) kernel/sched/core.c:5864 (discriminator 1))
.nr_running : 1
.util_avg : 1023
.runnable_avg : 2047
stack:0 pid:1774 tgid:1774 ppid:2 flags:0x00004000
.se->exec_start : 208020.176498
.nr_running : 6
R
.h_nr_running : 3
RAX: ffffffffffffffda RBX: 00000000000010bb RCX: 00007fcfe00a4d07
.util_avg : 1023
migration/27 181 42.523932 E 43.273173 0.750000 10240.439543 46 0
.nr_switches : 502
.nr_spread_over : 0
rt_rq[16]:
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
0.000000 10240.439543 0.000000 0.000000
Call Trace:
.se->vruntime : 7029113.257567
? _raw_spin_lock_irqsave (./arch/x86/include/asm/atomic.h:115 (discriminator 4) ./include/linux/atomic/atomic-arch-fallback.h:2164 (discriminator 4) ./include/linux/atomic/atomic-instrumented.h:1296 (discriminator 4) ./include/asm-generic/qspinlock.h:111 (discriminator 4) ./include/linux/spinlock.h:187 (discriminator 4) ./include/linux/spinlock_api_smp.h:111 (discriminator 4) kernel/locking/spinlock.c:162 (discriminator 4))
.idle_nr_running : 0
.se->sum_exec_runtime : 34775.606149
.util_est_enqueued : 0
.idle_h_nr_running : 0
.removed.load_avg : 0
.rt_nr_running : 0
.util_est_enqueued : 755
.removed.util_avg : 0
ret_from_fork (arch/x86/kernel/process.c:147)
/
Call Trace:
/
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
stack:0 pid:1641 tgid:1641 ppid:2 flags:0x00004000
.se->load.weight : 5637
.removed.runnable_avg : 0
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
.se->avg.load_avg : 4
? __pfx_kthread (kernel/kthread.c:341)
stack:0 pid:250 tgid:250 ppid:2 flags:0x00004000
Call Trace:
R
.nr_uninterruptible : 20
.rt_throttled : 0
.nr_running : 1
<TASK>
.load : 8798
.tg_load_avg_contrib : 11
.removed.load_avg : 0
rt_rq[162]:
.rt_time : 0.000000
</TASK>
.load_avg : 8
</TASK>
0.000000 0.051581 0.000000 0.000000
.tg_load_avg : 2026
.rt_runtime : 950.000000
.throttled : 0
.load : 18039
stack:0 pid:1603 tgid:1603 ppid:2 flags:0x00004000
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
ksoftirqd/27 182 31046.484334 E 31049.484334 3.000000 0.009125 9 120
.removed.util_avg : 0
task:kworker/18:0 state:I
.h_nr_running : 2
ORIG_RAX: 00000000000000e6
.removed.runnable_avg : 0
0.000000 0.009125 0.000000 0.000000
.next_balance : 4294.870336
dl_rq[16]:
.throttle_count : 0
</TASK>
.se->avg.util_avg : 1024
.runnable_avg : 3073
.dl_nr_running : 0
.curr->pid : 3500
.util_avg : 1024
.se->avg.runnable_avg : 3072
/
Workqueue: 0x0
.rt_nr_running : 0
.load_avg : 17
.util_est_enqueued : 0
.clock : 208129.798673
stack:0 pid:129 tgid:129 ppid:2 flags:0x00004000
.tg_load_avg_contrib : 0
.clock_task : 208003.714540
.tg_load_avg : 0
.removed.load_avg : 0
.dl_bw->bw : 996147
worker_thread (kernel/workqueue.c:2734)
.removed.util_avg : 0
Call Trace:
.throttled : 0
.se->exec_start : 208059.574049
kthread (kernel/kthread.c:388)
.dl_bw->total_bw : 0
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
? preempt_count_add (./include/linux/ftrace.h:977 kernel/sched/core.c:5839 kernel/sched/core.c:5836 kernel/sched/core.c:5864)
cfs_rq[25]:/
.avg_idle : 1000000
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
RAX: ffffffffffffffda RBX: 00007ffd3c963130 RCX: 00007f1e54fc1167
(events_highpri)
.max_idle_balance_cost : 500000
Workqueue: 0x0
/
.idle_nr_running : 0
<TASK>
I
I
.rt_throttled : 0
.exec_clock : 0.000000
kworker/27:0 183 48.316899 E 51.309998 3.000000 0.035706 13 120
Call Trace:
workqueue pm: flags=0x4
runnable tasks:
kthread (kernel/kthread.c:388)
task:ksoftirqd/11 state:R
? __pfx_kthread (kernel/kthread.c:341)
kworker/u453:18 1557 31.410494 E 33.875313 3.000000 32.452367 3 120
? __pfx_smpboot_thread_fn (kernel/smpboot.c:107)
.se->vruntime : 6969082.950841
Call Trace:
RDX: 0000000000000000 RSI: 00007ffe0b9131a0 RDI: 00000000ffffffff
task:sleep state:R
S task PID tree-key switches prio wait-time sum-exec sum-sleep
</TASK>
cfs_rq[20]:/user.slice/user-0.slice/session-1.scope
.removed.runnable_avg : 0
.idle_h_nr_running : 0
</TASK>
task:kworker/u452:22 state:I
task:cpuhp/97 state:S
-------------------------------------------------------------------------------------------------------------
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
.load : 5693
RDX: 00007ffd3c963130 RSI: 0000000000000000 RDI: 0000000000000000
0.000000 0.035706 0.000000 0.000000
.se->sum_exec_runtime : 34424.300264
.throttle_count : 0
.exec_clock : 0.000000
? __pfx_kthread (kernel/kthread.c:341)
.left_vruntime : 235690.131483
<TASK>
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
smpboot_thread_fn (kernel/smpboot.c:160)
task:migration/106 state:S
.tg_load_avg_contrib : 8
.se->load.weight : 5693
.left_vruntime : 786.779707
/
.min_vruntime : 786.779707
? _raw_spin_lock_irqsave (./arch/x86/include/asm/atomic.h:115 (discriminator 4) ./include/linux/atomic/atomic-arch-fallback.h:2164 (discriminator 4) ./include/linux/atomic/atomic-instrumented.h:1296 (discriminator 4) ./include/asm-generic/qspinlock.h:111 (discriminator 4) ./include/linux/spinlock.h:187 (discriminator 4) ./include/linux/spinlock_api_smp.h:111 (discriminator 4) kernel/locking/spinlock.c:162 (discriminator 4))
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
.load_avg : 4
.min_vruntime : 235691.830156
</TASK>
.rt_time : 0.000000
kthread (kernel/kthread.c:388)
.se->avg.load_avg : 4
S
pwq 8:
.runnable_avg : 2047
<TASK>
.avg_vruntime : 255911.103082
.right_vruntime : 243059.798538
.util_avg : 1023
task:kworker/u454:24 state:I
task:kworker/u451:12 state:I
cpus=4
rt_rq[50]:
cpuhp/16 113 7939.980674 E 7942.976661 3.000000 1.507300 35 120
.util_est_enqueued : 0
<TASK>
stack:0 pid:599 tgid:599 ppid:2 flags:0x00004000
Workqueue: 0x0
.spread : 7369.667055
? __pfx_kthread (kernel/kthread.c:341)
.se->avg.util_avg : 1023
.rt_nr_running : 0
(events_unbound)
running task
I
.avg_vruntime : 12248.530216
.rt_throttled : 0
.tg_load_avg : 1488
Workqueue: 0x0
kworker/27:0H 184 53.122947 E 53.157084 3.000000 0.048039 4 100
</TASK>
stack:0 pid:86 tgid:86 ppid:2 flags:0x00004000
0.000000 1.507300 0.000000 0.000000
? __pfx_kthread (kernel/kthread.c:341)
0.000000 0.048039 0.000000 0.000000
<TASK>
.se->avg.runnable_avg : 2047
.throttled : 0
/
.nr_spread_over : 0
.rt_time : 0.000000
/
RBP: 00007ffd3c9630e0 R08: 0000000000000000 R09: 0000000000000000
(mm_percpu_wq)
node=0
.right_vruntime : 1164.357159
.nr_running : 3
? __pfx_worker_thread (kernel/workqueue.c:2727)
.h_nr_running : 5
ret_from_fork (arch/x86/kernel/process.c:147)
S
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
.throttle_count : 0
RBP: 00007ffec60d77d0 R08: 0000000000000000 R09: 0000000000000000
running task
flags=0x0 nice=0
cfs_rq[53]:/user.slice
stack:0 pid:1467 tgid:1467 ppid:2 flags:0x00004000
ret_from_fork (arch/x86/kernel/process.c:147)
<TASK>
.removed.load_avg : 0
.idle_nr_running : 0
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
.exec_clock : 0.000000
rescuer_thread (kernel/workqueue.c:2932)
stack:0 pid:4247 tgid:4247 ppid:3813 flags:0x00000002
.rt_runtime : 950.000000
.removed.util_avg : 0
.left_vruntime : 0.000001
stack:0 pid:655 tgid:655 ppid:2 flags:0x00004000
.se->exec_start : 208012.340322
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
stack:0 pid:1509 tgid:1509 ppid:2 flags:0x00004000
.removed.runnable_avg : 0
idle_inject/16 114 -1.031481 E 1.966572 3.000000 0.006806 3 49
.min_vruntime : 6969082.950841
.spread : 377.577452
.se->vruntime : 7059421.129015
.nr_spread_over : 0
0.000000 0.006806 0.000000 0.000000
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
? __pfx_kthread (kernel/kthread.c:341)
? __pfx_kthread (kernel/kthread.c:341)
active=1/256 refcnt=2
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
Call Trace:
.avg_vruntime : 6969082.950841
ret_from_fork (arch/x86/kernel/process.c:147)
.nr_running : 3
task:kworker/u455:21 state:I
/
? __pfx_smpboot_thread_fn (kernel/smpboot.c:107)
0.000000 32.452367 0.000000 0.000000
dl_rq[50]:
.se->sum_exec_runtime : 34387.597188
Stopper: 0x0 <- 0x0
.right_vruntime : 0.000001
task:taskset state:S
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
RBP: 00007ffe0b9131e0 R08: 0000555adf8191d0 R09: 0000000000000001
.spread : 0.000000
(mm_percpu_wq)
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
.idle_h_nr_running : 0
.dl_nr_running : 0
.nr_spread_over : 0
I
? up_read (./arch/x86/include/asm/preempt.h:104 kernel/locking/rwsem.c:1354 kernel/locking/rwsem.c:1622)
R10: 00007ffd3c963120 R11: 0000000000000202 R12: 00007f1e54ee56c8
.tg_load_avg_contrib : 4
.dl_bw->bw : 996147
R10: 00007ffec60d7810 R11: 0000000000000202 R12: 00007f008c8706c8
.rt_runtime : 950.000000
.tg_load_avg : 1488
kworker/u453:4 1401 48.633974 E 49.637603 3.000000 49.674597 3 120
.nr_running : 1
I
.throttled : 0
? _raw_spin_lock_irqsave (./arch/x86/include/asm/atomic.h:115 (discriminator 4) ./include/linux/atomic/atomic-arch-fallback.h:2164 (discriminator 4) ./include/linux/atomic/atomic-instrumented.h:1296 (discriminator 4) ./include/asm-generic/qspinlock.h:111 (discriminator 4) ./include/linux/spinlock.h:187 (discriminator 4) ./include/linux/spinlock_api_smp.h:111 (discriminator 4) kernel/locking/spinlock.c:162 (discriminator 4))
.load : 2102789
.se->load.weight : 5637
.dl_bw->total_bw : 0
.h_nr_running : 3
.h_nr_running : 2
.se->avg.load_avg : 4
do_nanosleep (kernel/time/hrtimer.c:1446 kernel/time/hrtimer.c:2049)
.throttle_count : 0
R
R13: 00007ffec60d7810 R14: 0000000000000002 R15: 00007ffec60d7810
kworker/u450:14 1447 40.793566 E 41.382099 3.000000 41.838568 3 120
.idle_nr_running : 0
runnable tasks:
? _raw_spin_lock_irqsave (./arch/x86/include/asm/atomic.h:115 (discriminator 4) ./include/linux/atomic/atomic-arch-fallback.h:2164 (discriminator 4) ./include/linux/atomic/atomic-instrumented.h:1296 (discriminator 4) ./include/asm-generic/qspinlock.h:111 (discriminator 4) ./include/linux/spinlock.h:187 (discriminator 4) ./include/linux/spinlock_api_smp.h:111 (discriminator 4) kernel/locking/spinlock.c:162 (discriminator 4))
Call Trace:
migration/16 115 39.390982 E 40.140140 0.750000 10230.050371 46 0
? do_user_addr_fault (arch/x86/mm/fault.c:1445 (discriminator 1))
0.000000 41.838568 0.000000 0.000000
.load_avg : 2053
<TASK>
0.000000 49.674597 0.000000 0.000000
.se->exec_start : 208059.574049
.idle_nr_running : 0
task:kworker/119:1 state:I
.idle_h_nr_running : 0
.runnable_avg : 5120
0.000000 10230.050371 0.000000 0.000000
.idle_h_nr_running : 0
.se->vruntime : 11601569.351706
.load : 5693
.runnable_avg : 3072
/
.se->avg.util_avg : 1024
/
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
</TASK>
? _raw_spin_lock (./arch/x86/include/asm/atomic.h:115 (discriminator 4) ./include/linux/atomic/atomic-arch-fallback.h:2164 (discriminator 4) ./include/linux/atomic/atomic-instrumented.h:1296 (discriminator 4) ./include/asm-generic/qspinlock.h:111 (discriminator 4) ./include/linux/spinlock.h:187 (discriminator 4) ./include/linux/spinlock_api_smp.h:134 (discriminator 4) kernel/locking/spinlock.c:154 (discriminator 4))
.se->sum_exec_runtime : 34424.300264
R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000001
Call Trace:
? do_pte_missing (mm/memory.c:4763 mm/memory.c:3731)
.util_avg : 1024
.se->avg.runnable_avg : 3073
.load : 3145728
.util_est_enqueued : 1346
smpboot_thread_fn (kernel/smpboot.c:160)
<TASK>
/
.removed.load_avg : 0
S task PID tree-key switches prio wait-time sum-exec sum-sleep
</TASK>
.se->load.weight : 3521
<TASK>
R
.util_avg : 1024
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
hrtimer_nanosleep (kernel/time/hrtimer.c:2101)
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
.se->avg.load_avg : 2
.load_avg : 3072
cfs_rq[52]:/
pending:
kthread (kernel/kthread.c:388)
task:sleep state:R
.removed.util_avg : 0
-------------------------------------------------------------------------------------------------------------
.load_avg : 4
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
stack:0 pid:4055 tgid:4055 ppid:4018 flags:0x00000002
stack:0 pid:1731 tgid:1731 ppid:2 flags:0x00004000
.removed.runnable_avg : 0
? fill_page_cache_func (kernel/rcu/tree.c:3261 (discriminator 3))
.runnable_avg : 3072
.runnable_avg : 2047
.se->avg.util_avg : 1023
.util_avg : 1024
R
.exec_clock : 0.000000
.util_avg : 1023
ksoftirqd/16 116 66687.032812 E 66690.032812 3.000000 0.161350 8 120
.se->avg.runnable_avg : 2047
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
.util_est_enqueued : 0
0.000000 0.161350 0.000000 0.000000
Call Trace:
.util_est_enqueued : 0
.left_vruntime : 253596.183443
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
.removed.load_avg : 0
Call Trace:
S
? __pfx_smpboot_thread_fn (kernel/smpboot.c:107)
.removed.util_avg : 0
.removed.load_avg : 0
/
cpuhp/50 317 7794.497451 E 7797.492040 3.000000 1.639626 35 120
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
kworker/27:1 1641 26062.664491 E 26065.664491 3.000000 0.314750 24 120
/
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
.removed.runnable_avg : 0
cfs_rq[53]:/
0.000000 1.639626 0.000000 0.000000
worker_thread (kernel/workqueue.c:2734)
? __pfx_hrtimer_wakeup (kernel/time/hrtimer.c:1935)
</TASK>
? try_to_wake_up (./arch/x86/include/asm/preempt.h:104 ./include/linux/preempt.h:484 ./include/linux/preempt.h:484 kernel/sched/core.c:4217)
.util_est_enqueued : 0
Workqueue: 0x0
kthread (kernel/kthread.c:388)
.min_vruntime : 253598.301742
.tg_load_avg_contrib : 3042
.exec_clock : 0.000000
0.000000 0.314750 0.000000 0.000000
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
I
.removed.load_avg : 0
.avg_vruntime : 286666.275501
Workqueue: 0x0
pm_runtime_work
/
.tg_load_avg : 178559
worker_thread (kernel/workqueue.c:2734)
Call Trace:
.tg_load_avg_contrib : 0
<TASK>
R13: 00007ffd3c963120 R14: 0000000000000002 R15: 00007ffd3c963120
.left_vruntime : 265668.598558
/
.tg_load_avg : 0
kworker/16:0 117 41.657500 E 44.657500 3.000000 0.001594 4 120
.min_vruntime : 265685.378461
S
.throttled : 0
.right_vruntime : 253596.183443
? exc_page_fault (./arch/x86/include/asm/irqflags.h:37 ./arch/x86/include/asm/irqflags.h:72 arch/x86/mm/fault.c:1515 arch/x86/mm/fault.c:1563)
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
idle_inject/50 318 -1.045356 E 1.952059 3.000000 0.007742 3 49
task:migration/127 state:S
I
? __pfx_smpboot_thread_fn (kernel/smpboot.c:107)
.spread : 0.000000
0.000000 0.001594 0.000000 0.000000
.removed.util_avg : 0
worker_thread (kernel/workqueue.c:2734)
0.000000 0.007742 0.000000 0.000000
.throttled : 0
.throttle_count : 0
.avg_vruntime : 286627.043269
.nr_spread_over : 0
.nr_running : 2
.throttle_count : 0
.right_vruntime : 274436.242937
kworker/27:2 1859 48.309998 E 51.299781 3.000000 7.118016 9 120
/
<TASK>
running task
.removed.runnable_avg : 0
.spread : 8767.644379
/
.tg_load_avg_contrib : 4
.tg_load_avg : 1488
I
.h_nr_running : 4
? __pfx_worker_thread (kernel/workqueue.c:2727)
0.000000 7.118016 0.000000 0.000000
stack:0 pid:781 tgid:781 ppid:2 flags:0x00004000
? __pfx_kthread (kernel/kthread.c:341)
.idle_nr_running : 0
.nr_spread_over : 0
kworker/16:0H 118 51.029924 E 51.064418 3.000000 0.017708 4 100
stack:0 pid:4288 tgid:4288 ppid:3942 flags:0x00000002
</TASK>
.se->exec_start : 208005.280425
0.000000 0.017708 0.000000 0.000000
(mm_percpu_wq)
ret_from_fork (arch/x86/kernel/process.c:147)
/
.nr_running : 3
.idle_h_nr_running : 0
.throttled : 0
rt_rq[25]:
.load : 1054213
/
Call Trace:
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
? __pfx_worker_thread (kernel/workqueue.c:2727)
.h_nr_running : 4
.rt_nr_running : 0
I
.se->vruntime : 2070475.190185
R
.rt_throttled : 0
kworker/27:3 1860 41.217299 E 44.214125 3.000000 0.010947 2 120
dl_rq[162]:
.se->sum_exec_runtime : 34797.213562
Stopper: 0x0 <- 0x0
kthread (kernel/kthread.c:388)
(rcu_par_gp)
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
? __pfx_kthread (kernel/kthread.c:341)
.rt_time : 0.000000
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
.load_avg : 1029
.removed.util_avg : 0
.idle_nr_running : 0
.throttle_count : 0
.rt_runtime : 950.000000
.idle_h_nr_running : 0
.se->load.weight : 18037
0.000000 0.010947 0.000000 0.000000
.runnable_avg : 4097
migration/50 319 49.130632 E 49.879409 0.750000 10261.103701 47 0
.se->avg.load_avg : 16
.util_avg : 1024
Workqueue: 0x0
kthread (kernel/kthread.c:388)
Call Trace:
I
Call Trace:
.removed.runnable_avg : 0
/
.load : 2100673
0.000000 10261.103701 0.000000 0.000000
dl_rq[25]:
? __pfx_kthread (kernel/kthread.c:341)
kworker/u450:7 1421 41.657500 E 41.876554 3.000000 42.929731 3 120
.util_est_enqueued : 1266
.load_avg : 2050
smpboot_thread_fn (kernel/smpboot.c:160)
.se->exec_start : 208059.574049
.dl_nr_running : 0
</TASK>
.runnable_avg : 4095
.removed.load_avg : 0
.se->vruntime : 11601569.351706
.se->avg.util_avg : 1024
0.000000 42.929731 0.000000 0.000000
<TASK>
I
.util_avg : 1023
.dl_nr_running : 0
kworker/27:1H 2091 53.123206 E 53.157785 3.000000 0.020736 2 100
.removed.util_avg : 0
? __pfx_worker_thread (kernel/workqueue.c:2727)
.se->avg.runnable_avg : 3072
.util_est_enqueued : 930
0.000000 0.020736 0.000000 0.000000
.removed.runnable_avg : 0
/
.dl_bw->bw : 996147
.tg_load_avg_contrib : 0
.se->sum_exec_runtime : 34424.300264
? __pfx_smpboot_thread_fn (kernel/smpboot.c:107)
/
/
R
I
<TASK>
.tg_load_avg : 0
.removed.load_avg : 0
.se->load.weight : 3521
.removed.util_avg : 0
ret_from_fork (arch/x86/kernel/process.c:147)
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
.tg_load_avg_contrib : 17
? __pfx_kthread (kernel/kthread.c:341)
</TASK>
.removed.runnable_avg : 0
.dl_bw->total_bw : 0
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
.throttled : 0
>R
kworker/16:1 1630 64498.686713 E 64501.686713 3.000000 0.906720 19 120
.tg_load_avg_contrib : 0
R
.se->avg.load_avg : 2
? watchdog_hardlockup_check (kernel/watchdog.c:175)
.se->avg.util_avg : 1023
kthread (kernel/kthread.c:388)
.tg_load_avg : 0
0.000000 0.906720 0.000000 0.000000
runnable tasks:
console_blast.s 3530 34414.775057 N 34415.156603 3.000000 34401.655334 4 120
S task PID tree-key switches prio wait-time sum-exec sum-sleep
ksoftirqd/50 320 276865.549246 E 276868.549246 3.000000 0.032880 7 120
.se->avg.runnable_avg : 2047
.throttled : 0
-------------------------------------------------------------------------------------------------------------
kthread (kernel/kthread.c:388)
cfs_rq[20]:/user.slice/user-0.slice
</TASK>
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
<TASK>
/
? __pfx_kthread (kernel/kthread.c:341)
.throttle_count : 0
kworker/u453:23 1577 314.638416 E 317.188142 3.000000 22.628685 37 120
smpboot_thread_fn (kernel/smpboot.c:160)
.throttle_count : 0
ret_from_fork (arch/x86/kernel/process.c:147)
.exec_clock : 0.000000
Workqueue: 0x0
kthread (kernel/kthread.c:388)
? preempt_count_add (./include/linux/ftrace.h:977 kernel/sched/core.c:5839 kernel/sched/core.c:5836 kernel/sched/core.c:5864)
cfs_rq[53]:/
schedule_preempt_disabled (kernel/sched/core.c:6856)
rt_rq[53]:
0.000000 34401.655334 0.000000 0.000000
? __pfx_worker_thread (kernel/workqueue.c:2727)
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
.left_vruntime : 0.000001
S
rt_rq[52]:
.rt_nr_running : 0
.min_vruntime : 2070598.189932
workqueue kacpid: flags=0x0
0.000000 0.032880 0.000000 0.000000
cpuhp/25 167 7034.347940 E 7037.344060 3.000000 1.355944 35 120
? __pfx_kthread (kernel/kthread.c:341)
? __pfx_kthread (kernel/kthread.c:341)
.avg_vruntime : 2070598.189932
.rt_throttled : 0
</TASK>
.right_vruntime : 0.000001
I
.rt_time : 0.000000
I
.dl_bw->bw : 996147
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
? __pfx_kthread (kernel/kthread.c:341)
.rt_nr_running : 0
.exec_clock : 0.000000
/
.left_vruntime : 265668.598558
.spread : 0.000000
task:kworker/10:0 state:I
? __pfx_worker_thread (kernel/workqueue.c:2727)
.rt_runtime : 950.000000
kworker/16:1H 2287 1318.056182 E 1318.090734 3.000000 0.017695 3 100
.nr_spread_over : 0
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
/user.slice/user-0.slice/session-1.scope
.rt_throttled : 0
.min_vruntime : 265685.378461
task:sleep state:R
0.000000 1.355944 0.000000 0.000000
S
.nr_running : 1
</TASK>
0.000000 0.017695 0.000000 0.000000 /
? __pfx_kthread (kernel/kthread.c:341)
.avg_vruntime : 286627.043269
ret_from_fork (arch/x86/kernel/process.c:147)
.dl_bw->total_bw : 0
(events_unbound)
S
pwq 0:
? __pfx_smpboot_thread_fn (kernel/smpboot.c:107)
.h_nr_running : 3
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
task:console_blast.s state:S
</TASK>
.right_vruntime : 274436.242937
.idle_nr_running : 0
irq/338-idxd-mi 3209 7826.080206 E 7829.075933 3.000000 0.027938 3 49
console_blast.s 3740 13.308723 E 15.812479 3.000000 3.672988 5 120
ret_from_fork (arch/x86/kernel/process.c:147)
stack:0 pid:81 tgid:81 ppid:2 flags:0x00004000
dl_rq[53]:
kthread (kernel/kthread.c:388)
task:kworker/170:1 state:I
0.000000 0.027938 0.000000 0.000000
cpus=0
/
(events_unbound)
kworker/177:1 1785 12730.838592 E 12733.829955 3.000000 0.514560 40 120
I
.idle_h_nr_running : 0
.rt_time : 0.000000
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
task:cpuhp/39 state:S
task:kworker/18:0H state:I
entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:129)
node=0
/
.spread : 8767.644379
.rt_runtime : 950.000000
do_nanosleep (kernel/time/hrtimer.c:1446 kernel/time/hrtimer.c:2049)
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
stack:0 pid:251 tgid:251 ppid:2 flags:0x00004000
0.000000 22.628685 0.000000 0.000000
.load : 18037
? _raw_spin_lock (./arch/x86/include/asm/atomic.h:115 (discriminator 4) ./include/linux/atomic/atomic-arch-fallback.h:2164 (discriminator 4) ./include/linux/atomic/atomic-instrumented.h:1296 (discriminator 4) ./include/asm-generic/qspinlock.h:111 (discriminator 4) ./include/linux/spinlock.h:187 (discriminator 4) ./include/linux/spinlock_api_smp.h:134 (discriminator 4) kernel/locking/spinlock.c:154 (discriminator 4))
kworker/50:0 321 7788.065883 E 7790.887220 3.000000 0.187322 6 120
>R
.dl_nr_running : 0
.load_avg : 17
0.000000 3.672988 0.000000 0.000000
.runnable_avg : 3072
console_blast.s 3487 34778.915212 N 34781.389204 3.000000 34773.480133 6 120
S
Call Trace:
dl_rq[52]:
Workqueue: 0x0
.dl_bw->bw : 996147
0.000000 0.514560 0.000000 0.000000
.dl_nr_running : 0
0.000000 34773.480133 0.000000 0.000000
.dl_bw->total_bw : 0
/user.slice/user-0.slice/session-1.scope
.util_avg : 1024
? __pfx_kthread (kernel/kthread.c:341)
0.000000 0.187322 0.000000 0.000000
idle_inject/25 168 -1.048608 E 1.951392 3.000000 0.001548 3 49
.util_est_enqueued : 0
.dl_bw->bw : 996147
.nr_spread_over : 0
.removed.load_avg : 0
S
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
console_blast.s 3932 15.315284 E 17.825060 3.000000 3.120333 5 120
.dl_bw->total_bw : 0
.nr_running : 3
/
stack:0 pid:3941 tgid:3941 ppid:3454 flags:0x00000002
stack:0 pid:130 tgid:130 ppid:2 flags:0x00004000
.h_nr_running : 4
/user.slice/user-0.slice/session-1.scope
(events)
? __pfx_rescuer_thread (kernel/workqueue.c:2822)
I
0.000000 3.120333 0.000000 0.000000
runnable tasks:
kworker/50:0H 322 131.469806 E 131.503793 3.000000 0.062893 4 100
0.000000 0.001548 0.000000 0.000000
? __pfx_kthread (kernel/kthread.c:341)
RIP: 0033:0x7f1f7f5c9c74
.removed.util_avg : 0
Call Trace:
.removed.runnable_avg : 0
/user.slice/user-0.slice/session-1.scope
stack:0 pid:1572 tgid:1572 ppid:2 flags:0x00004000
.tg_load_avg_contrib : 17
flags=0x0 nice=0
S task PID tree-key switches prio wait-time sum-exec sum-sleep
<TASK>
Workqueue: 0x0
.tg_load_avg : 2026
.idle_nr_running : 0
/
(mm_percpu_wq)
S
? preempt_count_add (./include/linux/ftrace.h:977 kernel/sched/core.c:5839 kernel/sched/core.c:5836 kernel/sched/core.c:5864)
task:migration/14 state:R
runnable tasks:
.idle_h_nr_running : 0
active=1/1 refcnt=2
console_blast.s 3706 6.196533 E 8.708289 3.000000 3.575946 5 120
/
.throttled : 0
.load : 2100673
0.000000 0.062893 0.000000 0.000000
smpboot_thread_fn (kernel/smpboot.c:160)
kthread (kernel/kthread.c:388)
kthread (kernel/kthread.c:388)
S task PID tree-key switches prio wait-time sum-exec sum-sleep
0.000000 3.575946 0.000000 0.000000
-------------------------------------------------------------------------------------------------------------
.load_avg : 2050
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
stack:0 pid:1570 tgid:1570 ppid:2 flags:0x00004000
/
<TASK>
(events_highpri)
</TASK>
/
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
-------------------------------------------------------------------------------------------------------------
common_nsleep (kernel/time/posix-timers.c:1353)
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
stack:0 pid:1782 tgid:1782 ppid:2 flags:0x00004000
task:kworker/103:1 state:I
Workqueue: 0x0
<TASK>
/user.slice/user-0.slice/session-1.scope
.throttle_count : 0
stack:0 pid:1561 tgid:1561 ppid:2 flags:0x00004000
? try_to_wake_up (./arch/x86/include/asm/preempt.h:104 ./include/linux/preempt.h:484 ./include/linux/preempt.h:484 kernel/sched/core.c:4217)
.se->exec_start : 208009.254121
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
.tg_load_avg : 2026
R
Call Trace:
(events_unbound)
__mutex_lock.constprop.0 (kernel/locking/mutex.c:197 kernel/locking/mutex.c:681 kernel/locking/mutex.c:747)
.runnable_avg : 4095
S
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
.se->vruntime : 4401788.968903
R
cpuhp/52 329 1316.328938 E 1319.324334 3.000000 1.364705 35 120
I
S
sleep 4222 775.938814 E 778.938814 3.000000 1.058992 1 120
Workqueue: 0x0
.util_avg : 1023
worker_thread (kernel/workqueue.c:2734)
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
.se->sum_exec_runtime : 34801.187258
kworker/u452:16 1463 43.049371 E 45.782723 3.000000 44.094070 3 120
migration/25 169 41.974646 E 42.723865 0.750000 10239.047864 47 0
.util_est_enqueued : 930
0.000000 1.058992 0.000000 0.000000
.se->load.weight : 8798
stack:0 pid:1715 tgid:1715 ppid:2 flags:0x00004000
S
? __perf_event_overflow (kernel/events/core.c:9612 (discriminator 2))
.removed.load_avg : 0
.removed.util_avg : 0
hrtimer_nanosleep (kernel/time/hrtimer.c:2101)
0.000000 44.094070 0.000000 0.000000
.removed.runnable_avg : 0
? __pfx_kthread (kernel/kthread.c:341)
? __pfx_worker_thread (kernel/workqueue.c:2727)
cpuhp/53 335 321.131496 E 324.127291 3.000000 2.330872 35 120
0.000000 10239.047864 0.000000 0.000000
.tg_load_avg_contrib : 0
.se->avg.load_avg : 7
console_blast.s 3900 7.665190 E 10.222764 3.000000 3.035136 4 120
/
/user.slice/user-0.slice/session-1.scope
/
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
(btrfs-endio)
.tg_load_avg : 0
kthread (kernel/kthread.c:388)
do_nanosleep (kernel/time/hrtimer.c:1446 kernel/time/hrtimer.c:2049)
I
Workqueue: 0x0
.se->avg.util_avg : 1024
kworker/50:1 1664 7794.214185 E 7797.208466 3.000000 0.318829 34 120
0.000000 1.364705 0.000000 0.000000
0.000000 3.035136 0.000000 0.000000
Call Trace:
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
? __pfx_kthread (kernel/kthread.c:341)
R
/
? __pfx_kthread (kernel/kthread.c:341)
0.000000 2.330872 0.000000 0.000000
</TASK>
/user.slice/user-0.slice/session-1.scope
? __pfx_kthread (kernel/kthread.c:341)
.throttled : 0
running task
.se->avg.runnable_avg : 3072
sleep 4284 1153.243484 E 1156.243484 3.000000 1.126132 1 120
S
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
idle_inject/52 330 -1.048464 E 1.951536 3.000000 0.001833 3 49
/
I
<TASK>
? __pfx_hrtimer_wakeup (kernel/time/hrtimer.c:1935)
ret_from_fork (arch/x86/kernel/process.c:147)
cfs_rq[20]:/user.slice
pending:
task:kworker/141:1 state:I
? __pfx_smpboot_thread_fn (kernel/smpboot.c:107)
0.000000 0.318829 0.000000 0.000000
.throttle_count : 0
0.000000 0.001833 0.000000 0.000000
/
Call Trace:
</TASK>
R
I
.exec_clock : 0.000000
stack:0 pid:103 tgid:103 ppid:2 flags:0x00004000
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
acpi_os_execute_deferred
sleep 4211 767.549102 E 770.549102 3.000000 1.074312 1 120
/
S
0.000000 1.126132 0.000000 0.000000
R
R
ksoftirqd/25 170 243059.798538 E 243062.798538 3.000000 0.809738 3 120
kworker/177:1H 2568 2095.469601 E 2095.504180 3.000000 0.135817 5 100
0.000000 1.074312 0.000000 0.000000
I
0.000000 0.809738 0.000000 0.000000
.left_vruntime : 0.000001
common_nsleep (kernel/time/posix-timers.c:1353)
/user.slice/user-0.slice/session-1.scope
rt_rq[53]:
/user.slice/user-0.slice/session-1.scope
.rt_nr_running : 0
.min_vruntime : 4401788.968903
smpboot_thread_fn (kernel/smpboot.c:160)
kworker/50:1H 2731 131.471556 E 131.506147 3.000000 0.011530 2 100
worker_thread (kernel/workqueue.c:2734)
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
idle_inject/53 336 -1.048416 E 1.951584 3.000000 0.002267 3 49
0.000000 0.011530 0.000000 0.000000
? __pfx_kthread (kernel/kthread.c:341)
.rt_throttled : 0
migration/52 331 49.834186 E 50.583186 0.750000 10262.181147 46 0
.avg_vruntime : 4401788.968903
.rt_time : 0.000000
/
0.000000 0.002267 0.000000 0.000000
/
.right_vruntime : 0.000001
.spread : 0.000000
.rt_runtime : 950.000000
kthread (kernel/kthread.c:388)
? _raw_spin_lock_irqsave (./arch/x86/include/asm/atomic.h:115 (discriminator 4) ./include/linux/atomic/atomic-arch-fallback.h:2164 (discriminator 4) ./include/linux/atomic/atomic-instrumented.h:1296 (discriminator 4) ./include/asm-generic/qspinlock.h:111 (discriminator 4) ./include/linux/spinlock.h:187 (discriminator 4) ./include/linux/spinlock_api_smp.h:111 (discriminator 4) kernel/locking/spinlock.c:162 (discriminator 4))
I
I
cpu#28, 1900.000 MHz
smpboot_thread_fn (kernel/smpboot.c:160)
task:idle_inject/97 state:S
R
0.000000 10262.181147 0.000000 0.000000
? __pfx_kthread (kernel/kthread.c:341)
__x64_sys_clock_nanosleep (kernel/time/posix-timers.c:1396 kernel/time/posix-timers.c:1373 kernel/time/posix-timers.c:1373)
.nr_running : 5
.nr_spread_over : 0
kworker/169:1 1780 17062.233443 E 17065.230308 3.000000 0.619638 34 120
.throttled : 0
</TASK>
? __pfx_kthread (kernel/kthread.c:341)
.nr_running : 1
kworker/25:0 171 49.221004 E 52.221004 3.000000 0.001349 4 120
/
.nr_switches : 842
sleep 4274 1144.530391 E 1147.530391 3.000000 1.231738 1 120
task:kworker/11:0 state:I
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
0.000000 1.231738 0.000000 0.000000
dl_rq[53]:
.h_nr_running : 3
/user.slice/user-0.slice/session-1.scope
0.000000 0.001349 0.000000 0.000000
.idle_nr_running : 0
.idle_h_nr_running : 0
kworker/R-0000: 3236 7563.854930 E 7563.889524 3.000000 0.014703 2 100
Workqueue: 0x0
<TASK>
ret_from_fork (arch/x86/kernel/process.c:147)
.load : 8798
.nr_uninterruptible : -9
S
ret_from_fork (arch/x86/kernel/process.c:147)
.load_avg : 8
0.000000 0.014703 0.000000 0.000000
/
.runnable_avg : 3072
/
.next_balance : 4294.870318
stack:0 pid:600 tgid:600 ppid:2 flags:0x00004000
R
runnable tasks:
</TASK>
0.000000 0.619638 0.000000 0.000000
ksoftirqd/52 332 0.266993 E 1.950999 3.000000 0.784975 3 120
hrtimer_nanosleep (kernel/time/hrtimer.c:2101)
? __pfx_kthread (kernel/kthread.c:341)
do_syscall_64 (arch/x86/entry/common.c:52 arch/x86/entry/common.c:83)
cpu#17, 1900.000 MHz
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
kworker/50:2 3290 268170.491452 E 268173.491452 3.000000 0.247312 25 120
.util_avg : 1024
ret_from_fork (arch/x86/kernel/process.c:147)
Call Trace:
stack:0 pid:87 tgid:87 ppid:2 flags:0x00004000
? fill_page_cache_func (kernel/rcu/tree.c:3261 (discriminator 3))
.util_est_enqueued : 0
.curr->pid : 3533
.removed.load_avg : 0
Workqueue: 0x0
.removed.util_avg : 0
/
kthread (kernel/kthread.c:388)
0.000000 0.247312 0.000000 0.000000
.dl_nr_running : 0
I
.nr_running : 6
/
.removed.runnable_avg : 0
.nr_switches : 355
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
.clock : 208136.821159
(events_unbound)
? __pfx_kthread (kernel/kthread.c:341)
.tg_load_avg_contrib : 8
kworker/25:0H 172 107.523453 E 107.557997 3.000000 0.015087 4 100
.nr_uninterruptible : 2
>R
.dl_bw->bw : 996147
task:ksoftirqd/106 state:S
0.000000 0.015087 0.000000 0.000000
ret_from_fork (arch/x86/kernel/process.c:147)
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
.tg_load_avg : 1488
R
? avc_has_perm_noaudit (security/selinux/avc.c:1167)
console_blast.s 3613 34429.493690 N 34429.528981 3.000000 34430.296732 6 120
0.000000 0.784975 0.000000 0.000000
.next_balance : 4294.870320
.clock_task : 207962.507992
0.000000 34430.296732 0.000000 0.000000
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
/
<TASK>
running task
__x64_sys_clock_nanosleep (kernel/time/posix-timers.c:1396 kernel/time/posix-timers.c:1373 kernel/time/posix-timers.c:1373)
.avg_idle : 1000000
/
.throttled : 0
.curr->pid : 3490
? handle_pmi_common (arch/x86/events/intel/core.c:3052 (discriminator 1))
/user.slice/user-0.slice/session-1.scope
.clock : 208136.715991
.dl_bw->total_bw : 0
? exc_page_fault (./arch/x86/include/asm/irqflags.h:37 ./arch/x86/include/asm/irqflags.h:72 arch/x86/mm/fault.c:1515 arch/x86/mm/fault.c:1563)
Workqueue: 0x0
I
0.000000 0.135817 0.000000 0.000000
.throttle_count : 0
migration/53 337 50.226335 E 50.975278 0.750000 10263.028501 46 0
S task PID tree-key switches prio wait-time sum-exec sum-sleep
worker_thread (kernel/workqueue.c:2734)
.clock_task : 208029.184588
kworker/52:0 333 34.436812 E 37.431970 3.000000 0.008109 4 120
? __pfx_worker_thread (kernel/workqueue.c:2727)
? __pfx_kthread (kernel/kthread.c:341)
.se->exec_start : 208009.254121
ret_from_fork (arch/x86/kernel/process.c:147)
Call Trace:
.avg_idle : 1000000
worker_thread (kernel/workqueue.c:2734)
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
.max_idle_balance_cost : 500000
</TASK>
S
Call Trace:
stack:0 pid:656 tgid:656 ppid:2 flags:0x00004000
0.000000 10263.028501 0.000000 0.000000
I
(mm_percpu_wq)
0.000000 0.008109 0.000000 0.000000
.max_idle_balance_cost : 500000
/
stack:0 pid:4230 tgid:4230 ppid:3764 flags:0x00000002
? __pfx_kthread (kernel/kthread.c:341)
entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:129)
runnable tasks:
task:idle_inject/39 state:S
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
.se->vruntime : 7059428.096095
/
cfs_rq[28]:/user.slice/user-0.slice/session-1.scope
? set_pte_vaddr_p4d (arch/x86/mm/init_64.c:307 arch/x86/mm/init_64.c:315)
worker_thread (kernel/workqueue.c:2734)
cfs_rq[17]:/user.slice/user-0.slice/session-1.scope
/
<TASK>
.exec_clock : 0.000000
? __pfx_smpboot_thread_fn (kernel/smpboot.c:107)
.exec_clock : 0.000000
R
Call Trace:
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
I
? __pfx_hrtimer_wakeup (kernel/time/hrtimer.c:1935)
RSP: 002b:00007ffec4639108 EFLAGS: 00000202
Call Trace:
Call Trace:
.se->sum_exec_runtime : 34801.187258
kworker/52:0H 334 73.615530 E 73.649793 3.000000 0.042464 4 100
.left_vruntime : 767.543871
Workqueue: 0x0
.se->load.weight : 5637
kworker/u456:0 1368 49.221004 E 49.775388 3.000000 50.269580 4 120
console_blast.s 3805 0.814064 E 3.344904 3.000000 3.277275 4 120
? flush_tlb_one_kernel (./arch/x86/include/asm/paravirt.h:81 arch/x86/mm/tlb.c:1171 arch/x86/mm/tlb.c:1126)
.left_vruntime : 1145.560279
ksoftirqd/53 338 274436.242937 E 274439.242937 3.000000 0.002555 3 120
/
? preempt_count_add (./include/linux/ftrace.h:977 kernel/sched/core.c:5839 kernel/sched/core.c:5836 kernel/sched/core.c:5864)
stack:0 pid:252 tgid:252 ppid:2 flags:0x00004000
do_wait (kernel/exit.c:1625 (discriminator 4))
? __pfx_kthread (kernel/kthread.c:341)
.se->avg.load_avg : 4
0.000000 50.269580 0.000000 0.000000
<TASK>
Call Trace:
kthread (kernel/kthread.c:388)
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
(mm_percpu_wq)
0.000000 0.002555 0.000000 0.000000
.min_vruntime : 767.543871
? __pfx_kthread (kernel/kthread.c:341)
kthread (kernel/kthread.c:388)
0.000000 0.042464 0.000000 0.000000
S task PID tree-key switches prio wait-time sum-exec sum-sleep
.avg_vruntime : 12231.489035
.min_vruntime : 1145.560279
/
<TASK>
.throttle_count : 0
0.000000 3.277275 0.000000 0.000000
ret_from_fork (arch/x86/kernel/process.c:147)
.right_vruntime : 1146.188899
/
-------------------------------------------------------------------------------------------------------------
kthread (kernel/kthread.c:388)
I
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
do_syscall_64 (arch/x86/entry/common.c:52 arch/x86/entry/common.c:83)
<TASK>
Call Trace:
R
.se->avg.util_avg : 1024
</TASK>
iterate_tty_write (drivers/tty/tty_io.c:957 (discriminator 1) drivers/tty/tty_io.c:973 (discriminator 1))
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
.se->avg.runnable_avg : 3072
kworker/25:1 1644 235690.131483 E 235693.131483 3.000000 0.129072 15 120
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
<TASK>
? exc_page_fault (./arch/x86/include/asm/irqflags.h:37 ./arch/x86/include/asm/irqflags.h:72 arch/x86/mm/fault.c:1515 arch/x86/mm/fault.c:1563)
.spread : 378.645028
/user.slice/user-0.slice/session-1.scope
.avg_vruntime : 12462.171482
/
.se->exec_start : 207963.261501
S
.nr_spread_over : 0
cfs_rq[20]:/
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
.exec_clock : 0.000000
cpuhp/53 335 321.131496 E 324.127291 3.000000 2.330872 35 120
smpboot_thread_fn (kernel/smpboot.c:160)
.nr_running : 3
I
? __pfx_smpboot_thread_fn (kernel/smpboot.c:107)
? __schedule (kernel/sched/core.c:6714)
entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:129)
.right_vruntime : 1522.809229
0.000000 0.129072 0.000000 0.000000
Call Trace:
0.000000 2.330872 0.000000 0.000000
kworker/53:0 339 36.959343 E 39.956448 3.000000 0.004667 4 120
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
/
ORIG_RAX: 0000000000000001
I
/
.h_nr_running : 3
? native_set_fixmap (arch/x86/mm/pgtable.c:679 arch/x86/mm/pgtable.c:688)
R
Call Trace:
.idle_nr_running : 0
.spread : 377.248950
? __pfx_worker_thread (kernel/workqueue.c:2727)
kworker/169:1H 2565 184.779943 E 184.814527 3.000000 0.010232 2 100
? _raw_spin_lock (./arch/x86/include/asm/atomic.h:115 (discriminator 4) ./include/linux/atomic/atomic-arch-fallback.h:2164 (discriminator 4) ./include/linux/atomic/atomic-instrumented.h:1296 (discriminator 4) ./include/asm-generic/qspinlock.h:111 (discriminator 4) ./include/linux/spinlock.h:187 (discriminator 4) ./include/linux/spinlock_api_smp.h:134 (discriminator 4) kernel/locking/spinlock.c:154 (discriminator 4))
sleep 4246 790.110901 E 793.110901 3.000000 1.203562 1 120
RIP: 0033:0x7fb29f2e1167
kworker/u452:22 1509 34.431970 E 37.431970 3.000000 35.918421 3 120
.idle_h_nr_running : 0
I
0.000000 1.203562 0.000000 0.000000
0.000000 0.004667 0.000000 0.000000
<TASK>
? __pfx_kthread (kernel/kthread.c:341)
task:kworker/193:1 state:I
.nr_spread_over : 0
0.000000 0.010232 0.000000 0.000000
.left_vruntime : 65513.355468
.load : 3145728
common_nsleep (kernel/time/posix-timers.c:1353)
.nr_running : 3
RIP: 0033:0x7f5eb8fa6167
/user.slice/user-0.slice/session-1.scope
0.000000 35.918421 0.000000 0.000000
/
.h_nr_running : 3
kworker/25:1H 2638 107.523439 E 107.558031 3.000000 0.014532 2 100
.load_avg : 3072
.se->vruntime : 4393937.012123
S
.runnable_avg : 3072
idle_inject/53 336 -1.048416 E 1.951584 3.000000 0.002267 3 49
? ghes_copy_tofrom_phys (drivers/acpi/apei/ghes.c:330)
.idle_nr_running : 0
.min_vruntime : 65522.307768
? __pfx_kthread (kernel/kthread.c:341)
0.000000 0.002267 0.000000 0.000000
? preempt_count_add (./include/linux/ftrace.h:977 kernel/sched/core.c:5839 kernel/sched/core.c:5836 kernel/sched/core.c:5864)
.idle_h_nr_running : 0
0.000000 0.014532 0.000000 0.000000
.util_avg : 1024
/
.load : 3145728
I
? __pfx_smpboot_thread_fn (kernel/smpboot.c:107)
/
cpu#51, 1900.000 MHz
.util_est_enqueued : 0
.load_avg : 3072
.avg_vruntime : 98383.785872
/
kworker/53:0H 340 37.899491 E 37.933990 3.000000 0.023389 4 100
.removed.load_avg : 0
RSP: 002b:00007fffe8a591e8 EFLAGS: 00000202
.right_vruntime : 97262.314120
.nr_running : 4
.spread : 31748.958652
__x64_sys_clock_nanosleep (kernel/time/posix-timers.c:1396 kernel/time/posix-timers.c:1373 kernel/time/posix-timers.c:1373)
.nr_switches : 391
.removed.util_avg : 0
0.000000 0.023389 0.000000 0.000000
.nr_spread_over : 0
.removed.runnable_avg : 0
/
R
.runnable_avg : 3072
migration/53 337 50.226335 E 50.975278 0.750000 10263.028501 46 0
.nr_uninterruptible : -2
.nr_running : 3
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
.tg_load_avg_contrib : 3041
R
? file_has_perm (security/selinux/hooks.c:1759)
/
0.000000 10263.028501 0.000000 0.000000
.util_avg : 1024
Call Trace:
kworker/52:1 1667 253596.183443 E 253599.183443 3.000000 0.110518 14 120
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
/
0.000000 0.110518 0.000000 0.000000
.next_balance : 4294.870340
<TASK>
-------------------------------------------------------------------------------------------------------------
Call Trace:
kernel_wait4 (kernel/exit.c:1786)
.h_nr_running : 5
.curr->pid : 3810
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
.util_est_enqueued : 0
.tg_load_avg : 178559
.idle_nr_running : 0
Call Trace:
.idle_h_nr_running : 0
smpboot_thread_fn (kernel/smpboot.c:160)
.removed.load_avg : 0
</TASK>
<TASK>
.clock : 208137.463513
.throttled : 0
kthread (kernel/kthread.c:388)
<TASK>
>R
kthread (kernel/kthread.c:388)
.removed.util_avg : 0
.clock_task : 208024.838205
.throttle_count : 0
RAX: ffffffffffffffda RBX: 000000000000005b RCX: 00007f1f7f5c9c74
R
/
Call Trace:
ksoftirqd/53 338 274436.242937 E 274439.242937 3.000000 0.002555 3 120
.se->exec_start : 208030.360708
I
.avg_idle : 1000000
? _raw_spin_lock_irqsave (./arch/x86/include/asm/atomic.h:115 (discriminator 4) ./include/linux/atomic/atomic-arch-fallback.h:2164 (discriminator 4) ./include/linux/atomic/atomic-instrumented.h:1296 (discriminator 4) ./include/asm-generic/qspinlock.h:111 (discriminator 4) ./include/linux/spinlock.h:187 (discriminator 4) ./include/linux/spinlock_api_smp.h:111 (discriminator 4) kernel/locking/spinlock.c:162 (discriminator 4))
.se->vruntime : 2070052.454146
kworker/u452:18 1477 36.956448 E 37.228265 3.000000 38.004859 3 120
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
.removed.runnable_avg : 0
.load : 2102789
RDX: 000000000000005b RSI: 0000558a87b6e2c0 RDI: 0000000000000001
<TASK>
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
console_blast.s 3519 34781.602244 N 34783.276571 3.000000 34775.680993 5 120
stack:0 pid:1750 tgid:1750 ppid:2 flags:0x00004000
? __pfx_kthread (kernel/kthread.c:341)
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
0.000000 0.002555 0.000000 0.000000
I
.max_idle_balance_cost : 500000
/
0.000000 34775.680993 0.000000 0.000000
.load_avg : 2053
<TASK>
stack:0 pid:1801 tgid:1801 ppid:2 flags:0x00004000
kworker/52:1H 2967 73.615840 E 73.650417 3.000000 0.012694 2 100
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
0.000000 38.004859 0.000000 0.000000
cfs_rq[51]:/user.slice/user-0.slice/session-1.scope
.tg_load_avg_contrib : 3046
/
I
Stopper: 0x0 <- 0x0
.runnable_avg : 5121
/user.slice/user-0.slice/session-1.scope
? intel_pmu_handle_irq (./arch/x86/include/asm/msr.h:84 ./arch/x86/include/asm/msr.h:118 arch/x86/events/intel/core.c:2427 arch/x86/events/intel/core.c:3118)
? __pfx_kthread (kernel/kthread.c:341)
.exec_clock : 0.000000
.tg_load_avg : 178559
.se->sum_exec_runtime : 34784.378588
.util_avg : 1024
kworker/53:0 339 36.959343 E 39.956448 3.000000 0.004667 4 120
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
file_tty_write.isra.0 (drivers/tty/tty_io.c:1092)
.se->load.weight : 18036
kthread (kernel/kthread.c:388)
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
? _raw_spin_lock (./arch/x86/include/asm/atomic.h:115 (discriminator 4) ./include/linux/atomic/atomic-arch-fallback.h:2164 (discriminator 4) ./include/linux/atomic/atomic-instrumented.h:1296 (discriminator 4) ./include/asm-generic/qspinlock.h:111 (discriminator 4) ./include/linux/spinlock.h:187 (discriminator 4) ./include/linux/spinlock_api_smp.h:134 (discriminator 4) kernel/locking/spinlock.c:154 (discriminator 4))
.left_vruntime : 29882.540004
.util_est_enqueued : 1320
0.000000 0.012694 0.000000 0.000000
.se->avg.load_avg : 16
.min_vruntime : 19826.431616
S
.se->avg.util_avg : 1024
Workqueue: 0x0
/
? __pfx_kthread (kernel/kthread.c:341)
0.000000 0.004667 0.000000 0.000000 /
R
.removed.load_avg : 0
console_blast.s 3734 6.316129 E 8.909189 3.000000 3.334433 3 120
.avg_vruntime : 24854.485810
.se->avg.runnable_avg : 3072
</TASK>
.throttled : 0
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
kworker/53:1 1668 265668.598558 E 265671.598558 3.000000 0.244240 15 120
? __pfx_child_wait_callback (kernel/exit.c:1510)
.throttle_count : 0
RSP: 002b:00007ffc0efe2038 EFLAGS: 00000202
.se->exec_start : 207964.196942
workqueue i40e: flags=0x8
<TASK>
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
.right_vruntime : 29882.540004
ret_from_fork (arch/x86/kernel/process.c:147)
.removed.util_avg : 0
ORIG_RAX: 00000000000000e6
cfs_rq[17]:/user.slice/user-0.slice
0.000000 3.334433 0.000000 0.000000
I
ret_from_fork (arch/x86/kernel/process.c:147)
>R
.spread : 0.000000
.se->vruntime : 2092359.339934
.removed.runnable_avg : 0
/user.slice/user-0.slice/session-1.scope
console_blast.s 3618 34394.344179 N 34395.149488 3.000000 34393.469514 4 120
0.000000 0.244240 0.000000 0.000000
.exec_clock : 0.000000
.nr_spread_over : 0
ret_from_fork (arch/x86/kernel/process.c:147)
kworker/53:0H 340 37.899491 E 37.933990 3.000000 0.023389 4 100
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
.tg_load_avg_contrib : 0
<TASK>
0.000000 34393.469514 0.000000 0.000000
S
.tg_load_avg : 0
.nr_running : 2
.left_vruntime : 0.000001
S
.se->sum_exec_runtime : 34720.968034
.h_nr_running : 2
/user.slice/user-0.slice/session-1.scope
.min_vruntime : 2070052.454146
? __pfx_kthread (kernel/kthread.c:341)
? __pfx_smpboot_thread_fn (kernel/smpboot.c:107)
worker_thread (kernel/workqueue.c:2734)
.idle_nr_running : 0
/
.avg_vruntime : 2070052.454146
0.000000 0.023389 0.000000 0.000000
.se->load.weight : 18037
/
vfs_write (./include/linux/fs.h:2020 fs/read_write.c:491 fs/read_write.c:584)
S
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
.idle_h_nr_running : 0
.right_vruntime : 0.000001
.throttled : 0
console_blast.s 3813 0.833437 E 3.329484 3.000000 3.661653 5 120
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
.spread : 0.000000
.load : 2097152
.throttle_count : 0
console_blast.s 3928 7.822891 E 10.346808 3.000000 3.246692 4 120
.load_avg : 2048
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
? _raw_spin_lock_irqsave (./arch/x86/include/asm/atomic.h:115 (discriminator 4) ./include/linux/atomic/atomic-arch-fallback.h:2164 (discriminator 4) ./include/linux/atomic/atomic-instrumented.h:1296 (discriminator 4) ./include/asm-generic/qspinlock.h:111 (discriminator 4) ./include/linux/spinlock.h:187 (discriminator 4) ./include/linux/spinlock_api_smp.h:111 (discriminator 4) kernel/locking/spinlock.c:162 (discriminator 4))
.runnable_avg : 2048
ret_from_fork (arch/x86/kernel/process.c:147)
I
taskset 4158 6.319979 E 8.210525 3.000000 1.109454 1 120
.nr_spread_over : 0
.se->avg.load_avg : 16
Call Trace:
.se->sum_exec_runtime : 34763.444574
0.000000 3.246692 0.000000 0.000000
task:ksoftirqd/127 state:S
do_nanosleep (kernel/time/hrtimer.c:1446 kernel/time/hrtimer.c:2049)
.se->avg.util_avg : 1024
.util_avg : 1024
rt_rq[20]:
/user.slice/user-0.slice/session-1.scope
do_syscall_64 (arch/x86/entry/common.c:52 arch/x86/entry/common.c:83)
? __pfx_kthread (kernel/kthread.c:341)
kworker/53:1H 2831 82.279389 E 82.313955 3.000000 0.033790 8 100
0.000000 3.661653 0.000000 0.000000
.util_est_enqueued : 0
I
? __pfx_kthread (kernel/kthread.c:341)
.removed.load_avg : 0
.rt_nr_running : 0
.nr_running : 1
? __pfx_worker_thread (kernel/workqueue.c:2727)
S
.h_nr_running : 3
.removed.util_avg : 0
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
.se->avg.runnable_avg : 3072
? _raw_spin_lock (./arch/x86/include/asm/atomic.h:115 (discriminator 4) ./include/linux/atomic/atomic-arch-fallback.h:2164 (discriminator 4) ./include/linux/atomic/atomic-instrumented.h:1296 (discriminator 4) ./include/asm-generic/qspinlock.h:111 (discriminator 4) ./include/linux/spinlock.h:187 (discriminator 4) ./include/linux/spinlock_api_smp.h:134 (discriminator 4) kernel/locking/spinlock.c:154 (discriminator 4))
/user.slice/user-0.slice/session-1.scope
? _raw_spin_lock (./arch/x86/include/asm/atomic.h:115 (discriminator 4) ./include/linux/atomic/atomic-arch-fallback.h:2164 (discriminator 4) ./include/linux/atomic/atomic-instrumented.h:1296 (discriminator 4) ./include/asm-generic/qspinlock.h:111 (discriminator 4) ./include/linux/spinlock.h:187 (discriminator 4) ./include/linux/spinlock_api_smp.h:134 (discriminator 4) kernel/locking/spinlock.c:154 (discriminator 4))
0.000000 1.109454 0.000000 0.000000
.idle_nr_running : 0
smpboot_thread_fn (kernel/smpboot.c:160)
ret_from_fork (arch/x86/kernel/process.c:147)
console_blast.s 4047 4.139249 E 6.827856 3.000000 2.340944 3 120
.removed.runnable_avg : 0
0.000000 0.033790 0.000000 0.000000
kworker/u452:18 1477 36.956448 E 37.228265 3.000000 38.004859 3 120
pwq 0:
<TASK>
cpus=0
.tg_load_avg_contrib : 2020
? perf_event_nmi_handler (arch/x86/events/core.c:1743 arch/x86/events/core.c:1729)
.se->load.weight : 8798
cfs_rq[28]:/user.slice/user-0.slice
0.000000 2.340944 0.000000 0.000000
? __pfx_kthread (kernel/kthread.c:341)
<TASK>
.idle_h_nr_running : 0
RBP: 00007ffec4639130 R08: 0000000000000000 R09: 0000000000000001
kthread (kernel/kthread.c:388)
0.000000 38.004859 0.000000 0.000000
S
.tg_load_avg : 178559
.load : 18036
.throttled : 0
/
/user.slice/user-0.slice/session-1.scope
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
/
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
? __pfx_kthread (kernel/kthread.c:341)
.exec_clock : 0.000000
stack:0 pid:782 tgid:782 ppid:2 flags:0x00004000
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
kthread (kernel/kthread.c:388)
.rt_throttled : 0
R
.load_avg : 17
? preempt_count_add (./include/linux/ftrace.h:974 (discriminator 1) kernel/sched/core.c:5839 (discriminator 1) kernel/sched/core.c:5836 (discriminator 1) kernel/sched/core.c:5864 (discriminator 1))
</TASK>
.throttle_count : 0
.left_vruntime : 0.000001
R10: 0000000000000004 R11: 0000000000000202 R12: 000000000000005b
.min_vruntime : 2092444.218209
.rt_time : 0.000000
cpu#178, 1900.000 MHz
Call Trace:
worker_thread (kernel/workqueue.c:2734)
S
? __pfx_kthread (kernel/kthread.c:341)
.se->exec_start : 208026.395470
S
>R
.avg_vruntime : 2092472.856055
? __pfx_worker_thread (kernel/workqueue.c:2727)
kworker/53:1 1668 265668.598558 E 265671.598558 3.000000 0.244240 15 120
? up_read (./arch/x86/include/asm/preempt.h:104 kernel/locking/rwsem.c:1354 kernel/locking/rwsem.c:1622)
.runnable_avg : 3072
.rt_runtime : 950.000000
? __pfx_kthread (kernel/kthread.c:341)
.right_vruntime : 0.000001
/user.slice/user-0.slice/session-1.scope
.util_avg : 1024
.spread : 0.000000
? __pfx_kthread (kernel/kthread.c:341)
? native_queued_spin_lock_slowpath (./arch/x86/include/asm/vdso/processor.h:13 ./arch/x86/include/asm/vdso/processor.h:18 kernel/locking/qspinlock.c:383)
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
0.000000 0.244240 0.000000 0.000000
? __pfx_worker_thread (kernel/workqueue.c:2727)
console_blast.s 3622 34452.049073 N 34455.049073 3.000000 34452.470317 6 120
console_blast.s 4006 2.490438 E 4.947241 3.000000 3.002378 5 120
.nr_spread_over : 0
/
Workqueue: 0x0
.util_est_enqueued : 0
.removed.load_avg : 0
dl_rq[20]:
node=0
.dl_nr_running : 0
? do_user_addr_fault (arch/x86/mm/fault.c:1445 (discriminator 1))
kthread (kernel/kthread.c:388)
.removed.util_avg : 0
.se->vruntime : 3164381.501305
0.000000 34452.470317 0.000000 0.000000
ksys_write (fs/read_write.c:637)
taskset 4102 5.271925 E 7.139249 3.000000 1.132676 1 120
.nr_running : 1
.nr_running : 0
.dl_bw->bw : 996147
.se->sum_exec_runtime : 34391.758616
ret_from_fork (arch/x86/kernel/process.c:147)
do_syscall_64 (arch/x86/entry/common.c:52 arch/x86/entry/common.c:83)
I
cpuhp/162 989 802.473507 E 805.468289 3.000000 1.904840 35 120
.dl_bw->total_bw : 0
.removed.runnable_avg : 0
kworker/53:1H 2831 82.279389 E 82.313955 3.000000 0.033790 8 100
.h_nr_running : 3
.se->load.weight : 12024
.tg_load_avg_contrib : 17
</TASK>
0.000000 3.002378 0.000000 0.000000
runnable tasks:
.tg_load_avg : 2026
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
S task PID tree-key switches prio wait-time sum-exec sum-sleep
(events)
? nmi_handle (arch/x86/kernel/nmi.c:150)
? __pfx_kthread (kernel/kthread.c:341)
/user.slice/user-0.slice/session-1.scope
.se->avg.load_avg : 10
(mm_percpu_wq)
task:kworker/u450:26 state:I
worker_thread (kernel/workqueue.c:2734)
<TASK>
task:kworker/106:0 state:I
.se->avg.util_avg : 1024
task:migration/97 state:S
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
? __x64_sys_rt_sigaction (kernel/signal.c:4477 (discriminator 1) kernel/signal.c:4458 (discriminator 1) kernel/signal.c:4458 (discriminator 1))
flags=0x0 nice=0
ret_from_fork (arch/x86/kernel/process.c:147)
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
0.000000 1.132676 0.000000 0.000000
.idle_nr_running : 0
0.000000 1.904840 0.000000 0.000000
.throttled : 0
-------------------------------------------------------------------------------------------------------------
/user.slice/user-0.slice/session-1.scope
stack:0 pid:601 tgid:601 ppid:2 flags:0x00004000
? __schedule (kernel/sched/core.c:6714)
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
.se->avg.runnable_avg : 2048
.idle_h_nr_running : 0
S
.se->avg.load_avg : 7
.load : 18037
stack:0 pid:657 tgid:657 ppid:2 flags:0x00004000
.throttle_count : 0
S
.load_avg : 17
.se->exec_start : 208033.491020
/user.slice/user-0.slice/session-1.scope
ORIG_RAX: 00000000000000e6
cfs_rq[51]:/user.slice/user-0.slice
0.000000 0.033790 0.000000 0.000000
do_syscall_64 (arch/x86/entry/common.c:52 arch/x86/entry/common.c:83)
.exec_clock : 0.000000
.se->vruntime : 4398353.428294
? __pfx_smpboot_thread_fn (kernel/smpboot.c:107)
? __pfx_worker_thread (kernel/workqueue.c:2727)
.nr_switches : 913
worker_thread (kernel/workqueue.c:2734)
? __pfx_kthread (kernel/kthread.c:341)
.left_vruntime : 0.000001
? syscall_exit_to_user_mode (kernel/entry/common.c:299)
(mm_percpu_wq)
R13: 0000558a87b6e2c0 R14: 00007f1f7f69a5c0 R15: 00007f1f7f697f20
.se->sum_exec_runtime : 34787.508900
? __pfx_kthread (kernel/kthread.c:341)
? __pfx_worker_thread (kernel/workqueue.c:2727)
.min_vruntime : 3164381.501305
Call Trace:
? __pfx_worker_thread (kernel/workqueue.c:2727)
/
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
cpuhp/20 137 1251.357580 E 1254.354032 3.000000 2.107902 35 120
active=1/256 refcnt=2
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
.runnable_avg : 3072
.se->avg.util_avg : 1024
/
.avg_vruntime : 3164381.501305
</TASK>
cpu#170, 1900.000 MHz
Workqueue: 0x0
Stopper: 0x0 <- 0x0
? do_syscall_64 (arch/x86/entry/common.c:102)
.se->load.weight : 8798
.util_avg : 1024
</TASK>
.util_est_enqueued : 0
.right_vruntime : 0.000001
0.000000 2.107902 0.000000 0.000000
R
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
kthread (kernel/kthread.c:388)
S
RAX: ffffffffffffffda RBX: 00007fffe8a59240 RCX: 00007f5eb8fa6167
console_blast.s 3816 0.833077 E 3.313971 3.000000 3.252947 4 120
>R
.spread : 0.000000
.nr_uninterruptible : -3
sleep 4247 760.991109 E 763.991109 3.000000 1.168133 1 120
RAX: ffffffffffffffda RBX: 00007ffc0efe2090 RCX: 00007fb29f2e1167
.removed.load_avg : 0
S
Call Trace:
<TASK>
.nr_spread_over : 0
worker_thread (kernel/workqueue.c:2734)
/
0.000000 1.168133 0.000000 0.000000
(mm_percpu_wq)
? do_syscall_64 (arch/x86/entry/common.c:102)
</TASK>
.next_balance : 4294.870313
/user.slice/user-0.slice/session-1.scope
console_blast.s 4116 5.673854 E 8.299992 3.000000 2.476680 4 120
console_blast.s 3622 34452.049073 N 34455.049073 3.000000 34452.470317 6 120
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
<TASK>
.removed.util_avg : 0
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
S
.nr_running : 1
0.000000 34452.470317 0.000000 0.000000
.se->avg.load_avg : 7
0.000000 2.476680 0.000000 0.000000
0.000000 3.252947 0.000000 0.000000
task:kworker/147:1 state:I
.se->avg.util_avg : 1024
idle_inject/20 138 -1.048470 E 1.951530 3.000000 0.001521 3 49
/user.slice/user-0.slice/session-1.scope
R
.se->avg.runnable_avg : 3072
.removed.runnable_avg : 0
.h_nr_running : 2
/user.slice/user-0.slice/session-1.scope
R
.idle_nr_running : 0
.nr_running : 0
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
kthread (kernel/kthread.c:388)
schedule_timeout (kernel/time/timer.c:1635 ./include/linux/timer.h:199 kernel/time/timer.c:2175)
sleep 4219 766.197110 E 769.197110 3.000000 1.060727 1 120
.tg_load_avg_contrib : 17
/user.slice/user-0.slice/session-1.scope
? __pfx_smpboot_thread_fn (kernel/smpboot.c:107)
0.000000 0.001521 0.000000 0.000000
? preempt_count_add (./include/linux/ftrace.h:977 kernel/sched/core.c:5839 kernel/sched/core.c:5836 kernel/sched/core.c:5864)
RDX: 00007fffe8a59240 RSI: 0000000000000000 RDI: 0000000000000000
cfs_rq[17]:/user.slice
.tg_load_avg : 2026
? exc_page_fault (./arch/x86/include/asm/irqflags.h:37 ./arch/x86/include/asm/irqflags.h:72 arch/x86/mm/fault.c:1515 arch/x86/mm/fault.c:1563)
sleep 4306 1139.575216 E 1142.575216 3.000000 1.191652 1 120
.idle_h_nr_running : 0
/
0.000000 1.060727 0.000000 0.000000
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
R
S
.exec_clock : 0.000000
? __pfx_kthread (kernel/kthread.c:341)
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
migration/20 139 40.541546 E 41.290758 0.750000 10235.071352 45 0
.left_vruntime : 0.000001
.throttled : 0
/user.slice/user-0.slice/session-1.scope
.min_vruntime : 4398353.428294
pending:
? syscall_exit_to_user_mode (kernel/entry/common.c:299)
0.000000 1.191652 0.000000 0.000000
.nr_switches : 1317
? exc_page_fault (./arch/x86/include/asm/irqflags.h:37 ./arch/x86/include/asm/irqflags.h:72 arch/x86/mm/fault.c:1515 arch/x86/mm/fault.c:1563)
R
0.000000 10235.071352 0.000000 0.000000
.curr->pid : 0
? _raw_spin_lock (./arch/x86/include/asm/atomic.h:115 (discriminator 4) ./include/linux/atomic/atomic-arch-fallback.h:2164 (discriminator 4) ./include/linux/atomic/atomic-instrumented.h:1296 (discriminator 4) ./include/asm-generic/qspinlock.h:111 (discriminator 4) ./include/linux/spinlock.h:187 (discriminator 4) ./include/linux/spinlock_api_smp.h:134 (discriminator 4) kernel/locking/spinlock.c:154 (discriminator 4))
console_blast.s 3816 0.833077 E 3.313971 3.000000 3.252947 4 120
worker_thread (kernel/workqueue.c:2734)
R
smpboot_thread_fn (kernel/smpboot.c:160)
idle_inject/162 990 -1.042195 E 1.954926 3.000000 0.008217 3 49
.throttle_count : 0
RDX: 00007ffc0efe2090 RSI: 0000000000000000 RDI: 0000000000000000
.avg_vruntime : 4398353.428294
/user.slice/user-0.slice/session-1.scope
.load : 12024
/
</TASK>
.nr_uninterruptible : -2
task:taskset state:S
? _raw_spin_lock_irqsave (./arch/x86/include/asm/atomic.h:115 (discriminator 4) ./include/linux/atomic/atomic-arch-fallback.h:2164 (discriminator 4) ./include/linux/atomic/atomic-instrumented.h:1296 (discriminator 4) ./include/asm-generic/qspinlock.h:111 (discriminator 4) ./include/linux/spinlock.h:187 (discriminator 4) ./include/linux/spinlock_api_smp.h:111 (discriminator 4) kernel/locking/spinlock.c:162 (discriminator 4))
sleep 4248 790.763772 E 793.763772 3.000000 1.142544 1 120
R
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
sleep 4280 1331.430690 E 1334.430690 3.000000 1.124666 2 120
ksoftirqd/20 140 97262.314120 E 97265.314120 3.000000 0.000881 3 120
.load_avg : 11
stack:0 pid:1759 tgid:1759 ppid:2 flags:0x00004000
? __pfx_smpboot_thread_fn (kernel/smpboot.c:107)
stack:0 pid:4098 tgid:4098 ppid:4060 flags:0x00000002
RBP: 00007ffc0efe2040 R08: 0000000000000000 R09: 0000000000000000
? __pfx_process_timeout (kernel/time/timer.c:2098)
? preempt_count_add (./include/linux/ftrace.h:977 kernel/sched/core.c:5839 kernel/sched/core.c:5836 kernel/sched/core.c:5864)
.right_vruntime : 0.000001
cpu#53, 1900.000 MHz
.clock : 208129.555375
0.000000 1.124666 0.000000 0.000000
? __pfx_worker_thread (kernel/workqueue.c:2727)
.se->exec_start : 207967.102214
/user.slice/user-0.slice/session-1.scope
.runnable_avg : 2048
.nr_running : 5
kthread (kernel/kthread.c:388)
Call Trace:
0.000000 1.142544 0.000000 0.000000
.spread : 0.000000
? _raw_spin_lock (./arch/x86/include/asm/atomic.h:115 (discriminator 4) ./include/linux/atomic/atomic-arch-fallback.h:2164 (discriminator 4) ./include/linux/atomic/atomic-instrumented.h:1296 (discriminator 4) ./include/asm-generic/qspinlock.h:111 (discriminator 4) ./include/linux/spinlock.h:187 (discriminator 4) ./include/linux/spinlock_api_smp.h:134 (discriminator 4) kernel/locking/spinlock.c:154 (discriminator 4))
.next_balance : 4294.870413
.util_avg : 1024
.se->vruntime : 4456597.212089
0.000000 0.000881 0.000000 0.000000
.se->avg.runnable_avg : 3072
.nr_switches : 269
/
.nr_spread_over : 0
.se->sum_exec_runtime : 34723.873306
0.000000 3.252947 0.000000 0.000000
.util_est_enqueued : 0
worker_thread (kernel/workqueue.c:2734)
/user.slice/user-0.slice/session-1.scope
/user.slice/user-0.slice/session-1.scope
</TASK>
I
? native_queued_spin_lock_slowpath (./arch/x86/include/asm/vdso/processor.h:13 ./arch/x86/include/asm/vdso/processor.h:18 kernel/locking/qspinlock.c:383)
.nr_running : 1
.nr_uninterruptible : -4
entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:129)
task:cpuhp/19 state:S
.clock_task : 208090.886111
.next_balance : 4294.870351
.removed.load_avg : 0
.se->load.weight : 8798
kworker/20:0 141 38.956058 E 41.956058 3.000000 0.001667 4 120
? preempt_count_add (./include/linux/ftrace.h:977 kernel/sched/core.c:5839 kernel/sched/core.c:5836 kernel/sched/core.c:5864)
</TASK>
.removed.util_avg : 0
.h_nr_running : 3
.curr->pid : 3622
.se->avg.load_avg : 7
Call Trace:
cpu#54, 1900.000 MHz
0.000000 0.001667 0.000000 0.000000
R
.nr_running : 2
.idle_nr_running : 0
? __pfx_kthread (kernel/kthread.c:341)
.se->avg.util_avg : 1024
sleep 4248 790.763772 E 793.763772 3.000000 1.142544 1 120
? do_syscall_64 (arch/x86/entry/common.c:102)
.se->avg.runnable_avg : 3072
task:migration/39 state:R
.removed.runnable_avg : 0
.idle_h_nr_running : 0
.nr_switches : 301
/
R10: 00007ffc0efe2080 R11: 0000000000000202 R12: 00007fb29f2056c8
.nr_uninterruptible : -3
.load : 8798
.tg_load_avg_contrib : 11
cpu#26, 1900.000 MHz
.load_avg : 8
Call Trace:
.clock : 208138.007006
0.000000 1.142544 0.000000 0.000000
cfs_rq[28]:/user.slice
.nr_running : 5
I
.clock_task : 208088.464512
? _raw_spin_lock_irqsave (./arch/x86/include/asm/atomic.h:115 (discriminator 4) ./include/linux/atomic/atomic-arch-fallback.h:2164 (discriminator 4) ./include/linux/atomic/atomic-instrumented.h:1296 (discriminator 4) ./include/asm-generic/qspinlock.h:111 (discriminator 4) ./include/linux/spinlock.h:187 (discriminator 4) ./include/linux/spinlock_api_smp.h:111 (discriminator 4) kernel/locking/spinlock.c:162 (discriminator 4))
.tg_load_avg : 2026
kworker/20:0H 142 181.803384 E 181.837903 3.000000 0.015103 4 100
.nr_switches : 838
.runnable_avg : 3072
/user.slice/user-0.slice/session-1.scope
.avg_idle : 1000000
<TASK>
i40e_service_task [i40e]
.util_avg : 1024
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
0.000000 0.015103 0.000000 0.000000
.max_idle_balance_cost : 500000
.nr_uninterruptible : 5
.next_balance : 4294.870387
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
.throttled : 0
.exec_clock : 0.000000
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
.curr->pid : 3820
Workqueue: 0x0
/
kthread (kernel/kthread.c:388)
.util_est_enqueued : 0
.next_balance : 4294.870340
.curr->pid : 3524
RIP: 0033:0x7ff079799c74
.left_vruntime : 0.000001
</TASK>
running task
.throttle_count : 0
.clock : 208142.896875
.min_vruntime : 4456735.892024
.clock_task : 208037.638922
cpu#54, 1900.000 MHz
.se->exec_start : 208028.447682
cfs_rq[53]:/user.slice/user-0.slice/session-1.scope
.clock : 208141.406132
.exec_clock : 0.000000
.nr_running : 2
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
.avg_vruntime : 4456735.892024
(events_unbound)
.clock_task : 208006.423060
RBP: 00007fffe8a591f0 R08: 0000000000000000 R09: 0000000000000000
.avg_idle : 1000000
.left_vruntime : 790.763772
? __pfx_kthread (kernel/kthread.c:341)
.avg_idle : 1000000
entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:129)
I
.max_idle_balance_cost : 500000
.removed.load_avg : 0
.nr_switches : 301
.right_vruntime : 0.000001
RSP: 002b:00007fff94510368 EFLAGS: 00000202
? preempt_count_add (./include/linux/ftrace.h:977 kernel/sched/core.c:5839 kernel/sched/core.c:5836 kernel/sched/core.c:5864)
kworker/u450:16 1454 38.956058 E 41.818696 3.000000 40.004528 3 120
.se->vruntime : 6964116.547114
.spread : 0.000000
R10: 00007fffe8a59230 R11: 0000000000000202 R12: 00007f5eb8eca6c8
.max_idle_balance_cost : 500000
.min_vruntime : 790.763772
stack:0 pid:253 tgid:253 ppid:2 flags:0x00004000
.nr_spread_over : 0
kthread (kernel/kthread.c:388)
.se->sum_exec_runtime : 34393.810828
? _raw_spin_lock_irqsave (./arch/x86/include/asm/atomic.h:115 (discriminator 4) ./include/linux/atomic/atomic-arch-fallback.h:2164 (discriminator 4) ./include/linux/atomic/atomic-instrumented.h:1296 (discriminator 4) ./include/asm-generic/qspinlock.h:111 (discriminator 4) ./include/linux/spinlock.h:187 (discriminator 4) ./include/linux/spinlock_api_smp.h:111 (discriminator 4) kernel/locking/spinlock.c:162 (discriminator 4))
.avg_vruntime : 17621.406422
stack:0 pid:131 tgid:131 ppid:2 flags:0x00004000
.avg_idle : 1114779
.right_vruntime : 790.763772
<TASK>
cfs_rq[54]:/user.slice/user-0.slice/session-1.scope
.spread : 0.000000
? __handle_mm_fault (mm/memory.c:5180)
.se->load.weight : 5693
.removed.util_avg : 0
.se->avg.load_avg : 4
.nr_running : 1
cfs_rq[26]:/user.slice/user-0.slice/session-1.scope
.exec_clock : 0.000000
ret_from_fork (arch/x86/kernel/process.c:147)
0.000000 40.004528 0.000000 0.000000
.se->avg.util_avg : 1024
task:kworker/11:0H state:I
.nr_uninterruptible : -3
.removed.runnable_avg : 0
/
smpboot_thread_fn (kernel/smpboot.c:160)
.left_vruntime : 33777.074030
ret_from_fork (arch/x86/kernel/process.c:147)
Call Trace:
.se->avg.runnable_avg : 2048
0.000000 0.008217 0.000000 0.000000
.min_vruntime : 17883.838663
.exec_clock : 0.000000
.nr_spread_over : 0
.next_balance : 4294.870387
.h_nr_running : 3
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
cfs_rq[51]:/user.slice
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
<TASK>
R13: 00007fffe8a59230 R14: 0000000000000002 R15: 00007fffe8a59230
? __pfx_kthread (kernel/kthread.c:341)
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
Call Trace:
R
.curr->pid : 0
<TASK>
.avg_vruntime : 25830.673055
.exec_clock : 0.000000
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
.left_vruntime : 767.733831
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
? __pfx_kthread (kernel/kthread.c:341)
.clock : 208010.267801
.right_vruntime : 33777.074030
.max_idle_balance_cost : 615743
? __pfx_worker_thread (kernel/workqueue.c:2727)
schedule_preempt_disabled (kernel/sched/core.c:6856)
.tg_load_avg_contrib : 8
.min_vruntime : 767.733831
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
.left_vruntime : 0.000001
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
(events_highpri)
.tg_load_avg : 1488
.clock_task : 207961.599389
.nr_running : 2
R13: 00007ffc0efe2080 R14: 0000000000000002 R15: 00007ffc0efe2080
</TASK>
worker_thread (kernel/workqueue.c:2734)
.spread : 0.000000
Stopper: 0x0 <- 0x0
.h_nr_running : 2
.throttled : 0
.avg_vruntime : 12296.053301
kthread (kernel/kthread.c:388)
</TASK>
.throttle_count : 0
.idle_nr_running : 0
.nr_spread_over : 0
.min_vruntime : 6964338.572227
.curr->pid : 3820
.se->exec_start : 208034.879130
kworker/20:1 1635 65513.355468 E 65516.355468 3.000000 0.113279 14 120
? __pfx_kthread (kernel/kthread.c:341)
.se->vruntime : 7051119.625257
/
.nr_running : 2
.idle_nr_running : 0
<TASK>
.idle_h_nr_running : 0
0.000000 0.113279 0.000000 0.000000
worker_thread (kernel/workqueue.c:2734)
(mm_percpu_wq)
.avg_vruntime : 6964338.572227
.idle_h_nr_running : 0
.se->sum_exec_runtime : 34788.897010
.load : 2097152
.right_vruntime : 1332.724096
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
.load_avg : 2047
? try_to_wake_up (./arch/x86/include/asm/preempt.h:104 ./include/linux/preempt.h:484 ./include/linux/preempt.h:484 kernel/sched/core.c:4217)
.avg_idle : 2121695
.right_vruntime : 0.000001
</TASK>
.clock : 208143.547951
.se->load.weight : 5637
/
.load : 8798
.clock_task : 208006.856478
.runnable_avg : 2047
.se->avg.load_avg : 4
.spread : 0.000000
.h_nr_running : 2
.avg_idle : 1000000
stack:0 pid:88 tgid:88 ppid:2 flags:0x00004000
cfs_rq[178]:/
.se->avg.util_avg : 1024
smpboot_thread_fn (kernel/smpboot.c:160)
? exc_page_fault (./arch/x86/include/asm/irqflags.h:37 ./arch/x86/include/asm/irqflags.h:72 arch/x86/mm/fault.c:1515 arch/x86/mm/fault.c:1563)
? __pfx_worker_thread (kernel/workqueue.c:2727)
? __pfx_worker_thread (kernel/workqueue.c:2727)
.nr_spread_over : 0
.se->avg.runnable_avg : 3072
.idle_nr_running : 0
ret_from_fork (arch/x86/kernel/process.c:147)
.load_avg : 8
.util_avg : 1023
Call Trace:
.max_idle_balance_cost : 500000
.max_idle_balance_cost : 1130474
Workqueue: 0x0
? __pfx_worker_thread (kernel/workqueue.c:2727)
task:sleep state:R
.spread : 564.990265
.nr_running : 1
I
.idle_h_nr_running : 0
.h_nr_running : 2
.nr_spread_over : 0
.load : 2097152
.runnable_avg : 3072
kthread (kernel/kthread.c:388)
cfs_rq[54]:/user.slice/user-0.slice/session-1.scope
.nr_running : 3
.idle_nr_running : 0
.util_est_enqueued : 0
__mutex_lock.constprop.0 (kernel/locking/mutex.c:197 kernel/locking/mutex.c:681 kernel/locking/mutex.c:747)
.idle_h_nr_running : 0
.util_avg : 1024
<TASK>
.load_avg : 2048
.load : 5693
worker_thread (kernel/workqueue.c:2734)
cfs_rq[17]:/
kworker/20:1H 2823 181.803656 E 181.838242 3.000000 0.009815 2 100
? __pfx_kthread (kernel/kthread.c:341)
entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:129)
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
.util_est_enqueued : 0
.removed.load_avg : 0
.load_avg : 5
(events_highpri)
.exec_clock : 0.000000
</TASK>
.h_nr_running : 3
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
.removed.load_avg : 0
.idle_nr_running : 0
.removed.util_avg : 0
.removed.util_avg : 0
.idle_h_nr_running : 0
.removed.runnable_avg : 0
.left_vruntime : 33777.074030
.runnable_avg : 2048
task:kworker/10:0H state:I
.runnable_avg : 2048
worker_thread (kernel/workqueue.c:2734)
running task
.exec_clock : 0.000000
.util_avg : 1024
? __pfx_kthread (kernel/kthread.c:341)
.util_avg : 1024
.left_vruntime : 55735.034269
? __pfx_kthread (kernel/kthread.c:341)
.load : 3145728
stack:0 pid:82 tgid:82 ppid:2 flags:0x00004000
.removed.runnable_avg : 0
.min_vruntime : 17885.102426
.min_vruntime : 55735.254269
.exec_clock : 0.000000
task:kworker/127:0 state:I
kthread (kernel/kthread.c:388)
.avg_vruntime : 87308.720187
.util_est_enqueued : 0
.load_avg : 3072
.right_vruntime : 84879.423171
ret_from_fork (arch/x86/kernel/process.c:147)
.util_est_enqueued : 0
.tg_load_avg_contrib : 2042
.avg_vruntime : 25831.088228
S
stack:0 pid:1521 tgid:1521 ppid:2 flags:0x00004000
task:kworker/162:1 state:I
.runnable_avg : 3072
? __pfx_smpboot_thread_fn (kernel/smpboot.c:107)
ORIG_RAX: 0000000000000001
? __pfx_smpboot_thread_fn (kernel/smpboot.c:107)
task:kworker/u450:20 state:I
Workqueue: 0x0
.util_avg : 1024
.removed.load_avg : 0
.right_vruntime : 33777.074030
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
0.000000 0.009815 0.000000 0.000000
ret_from_fork (arch/x86/kernel/process.c:147)
.removed.util_avg : 0
.util_est_enqueued : 0
.removed.load_avg : 0
RIP: 0033:0x7f6d0c61cd07
pool 234:
/
.removed.util_avg : 0
.spread : 0.000000
ret_from_fork (arch/x86/kernel/process.c:147)
.removed.load_avg : 0
ret_from_fork (arch/x86/kernel/process.c:147)
.removed.runnable_avg : 0
task:kworker/198:1 state:I
Call Trace:
.tg_load_avg_contrib : 5
.nr_spread_over : 0
.tg_load_avg : 1488
stack:0 pid:4274 tgid:4274 ppid:3900 flags:0x00000002
RSP: 002b:00007ffef1f14b58 EFLAGS: 00000202
? __pfx_kthread (kernel/kthread.c:341)
smpboot_thread_fn (kernel/smpboot.c:160)
.removed.util_avg : 0
.throttled : 0
.tg_load_avg_contrib : 8
.spread : 29144.388902
.tg_load_avg : 178559
.removed.runnable_avg : 0
.tg_load_avg : 1488
migration/162 991 88.612763 E 91.611703 3.000000 10459.643851 55 0
.throttled : 0
kthread (kernel/kthread.c:388)
>R
.throttle_count : 0
console_blast.s 3500 34809.276853 N 34812.276853 3.000000 34786.282495 5 120
.nr_spread_over : 0
.se->exec_start : 208030.441781
? __pfx_kthread (kernel/kthread.c:341)
.nr_running : 2
.removed.runnable_avg : 0
.throttle_count : 0
0.000000 34786.282495 0.000000 0.000000
stack:0 pid:783 tgid:783 ppid:2 flags:0x00004000
.h_nr_running : 2
? __pfx_kthread (kernel/kthread.c:341)
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
kthread (kernel/kthread.c:388)
.throttled : 0
? avc_has_perm_noaudit (security/selinux/avc.c:1167)
0.000000 10459.643851 0.000000 0.000000
.tg_load_avg_contrib : 2024
ret_from_fork (arch/x86/kernel/process.c:147)
<TASK>
.se->exec_start : 208088.464512
.idle_nr_running : 0
.throttle_count : 0
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
cfs_rq[4]:/user.slice
.se->vruntime : 11718824.508045
.idle_h_nr_running : 0
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
.se->vruntime : 3168692.520104
.se->exec_start : 207969.491837
.tg_load_avg : 178559
stack:0 pid:1473 tgid:1473 ppid:2 flags:0x00004000
? __pfx_kthread (kernel/kthread.c:341)
.se->sum_exec_runtime : 34453.190727
.nr_running : 3
.se->sum_exec_runtime : 34395.804927
/user.slice/user-0.slice/session-1.scope
.h_nr_running : 5
ORIG_RAX: 000000000000003d
? __pfx_kthread (kernel/kthread.c:341)
cfs_rq[170]:/
.throttled : 0
smpboot_thread_fn (kernel/smpboot.c:160)
</TASK>
.load : 2097152
.se->vruntime : 7040612.160458
.se->load.weight : 12026
.tg_load_avg_contrib : 3047
.idle_nr_running : 0
Workqueue: 0x0
.se->sum_exec_runtime : 34727.705227
.se->avg.load_avg : 10
.throttle_count : 0
.se->load.weight : 3523
.se->exec_start : 208007.686823
.se->load.weight : 5637
.load_avg : 2048
(events_highpri)
.se->avg.load_avg : 2
iterate_tty_write (drivers/tty/tty_io.c:957 (discriminator 1) drivers/tty/tty_io.c:973 (discriminator 1))
stack:0 pid:1774 tgid:1774 ppid:2 flags:0x00004000
.runnable_avg : 2048
</TASK>
? fill_page_cache_func (kernel/rcu/tree.c:3261 (discriminator 3))
.se->avg.load_avg : 4
.se->avg.util_avg : 1024
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
kthread (kernel/kthread.c:388)
cpus=117
.util_avg : 1024
RAX: ffffffffffffffda RBX: 00000000000010bf RCX: 00007f6d0c61cd07
.tg_load_avg : 178559
.idle_h_nr_running : 0
.se->avg.util_avg : 1024
.exec_clock : 0.000000
? file_has_perm (security/selinux/hooks.c:1759)
.se->avg.runnable_avg : 2048
S
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
console_blast.s 3720 22.687922 E 25.266887 3.000000 3.494354 4 120
.load : 2102789
.se->vruntime : 3164876.002386
task:sleep state:R
.load_avg : 2053
.se->avg.runnable_avg : 3072
cfs_rq[51]:/
0.000000 3.494354 0.000000 0.000000
.util_est_enqueued : 0
.se->avg.util_avg : 1023
hrtimer_nanosleep (kernel/time/hrtimer.c:2101)
.throttled : 0
(mm_percpu_wq)
cfs_rq[28]:/
.exec_clock : 0.000000
/user.slice/user-0.slice/session-1.scope
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
kthread (kernel/kthread.c:388)
.exec_clock : 0.000000
.se->avg.runnable_avg : 2047
? __pfx_kthread (kernel/kthread.c:341)
.se->sum_exec_runtime : 34397.506925
ret_from_fork (arch/x86/kernel/process.c:147)
.left_vruntime : 10547918.653827
.left_vruntime : 59318.094212
.throttle_count : 0
</TASK>
.se->load.weight : 12025
.se->exec_start : 208038.586015
.removed.load_avg : 0
task:ksoftirqd/97 state:S
.left_vruntime : 0.000001
.runnable_avg : 5120
cfs_rq[53]:/user.slice/user-0.slice
.min_vruntime : 59318.094212
.removed.util_avg : 0
S
task:ksoftirqd/39 state:R
.se->avg.load_avg : 10
.min_vruntime : 10549972.653827
stack:0 pid:1814 tgid:1814 ppid:2 flags:0x00004000
.left_vruntime : 0.000001
.se->vruntime : 2079640.954164
stack:0 pid:602 tgid:602 ppid:2 flags:0x00004000
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
.removed.runnable_avg : 0
.exec_clock : 0.000000
? __pfx_kthread (kernel/kthread.c:341)
console_blast.s 3912 25.143113 E 27.566505 3.000000 3.290408 5 120
.avg_vruntime : 93242.568992
? __pfx_kthread (kernel/kthread.c:341)
.util_avg : 1024
.left_vruntime : 0.000001
.avg_vruntime : 10551339.902245
.se->sum_exec_runtime : 34790.825437
.se->avg.util_avg : 1024
kcompactd (mm/compaction.c:3086 (discriminator 13))
node=0
.min_vruntime : 3168692.520104
RAX: ffffffffffffffda RBX: 000000000000005b RCX: 00007ff079799c74
.right_vruntime : 59318.094212
.min_vruntime : 4395792.774550
.tg_load_avg_contrib : 2024
RIP: 0033:0x7f4002dba167
.se->load.weight : 18037
.se->avg.runnable_avg : 2048
file_tty_write.isra.0 (drivers/tty/tty_io.c:1092)
.right_vruntime : 10547918.653827
0.000000 3.290408 0.000000 0.000000
.se->avg.load_avg : 16
</TASK>
ret_from_fork (arch/x86/kernel/process.c:147)
.spread : 0.000000
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
RDX: 0000000000000000 RSI: 00007ffef1f14b90 RDI: 00000000ffffffff
Call Trace:
.tg_load_avg : 178559
worker_thread (kernel/workqueue.c:2734)
.spread : 0.000000
.avg_vruntime : 3168692.520104
flags=0x0 nice=0
.se->avg.util_avg : 1024
running task
Call Trace:
Call Trace:
.right_vruntime : 0.000001
.util_est_enqueued : 1356
.spread : 0.000000
vfs_write (./include/linux/fs.h:2020 fs/read_write.c:491 fs/read_write.c:584)
.throttled : 0
/user.slice/user-0.slice/session-1.scope
/
.se->avg.runnable_avg : 3072
cfs_rq[54]:/user.slice/user-0.slice
.nr_spread_over : 0
running task
.nr_spread_over : 0
.nr_spread_over : 0
.throttle_count : 0
.exec_clock : 0.000000
cfs_rq[26]:/user.slice/user-0.slice
.nr_running : 2
Workqueue: 0x0
.exec_clock : 0.000000
.nr_running : 2
.removed.load_avg : 0
.se->exec_start : 208008.835791
S
.removed.util_avg : 0
<TASK>
.left_vruntime : 0.000001
.h_nr_running : 4
.nr_running : 1
.removed.runnable_avg : 0
hung=19s workers=4
.left_vruntime : 0.000001
? __pfx_worker_thread (kernel/workqueue.c:2727)
? __pfx_worker_thread (kernel/workqueue.c:2727)
.min_vruntime : 16272.963166
.tg_load_avg_contrib : 0
ksys_write (fs/read_write.c:637)
.min_vruntime : 2079767.836377
.h_nr_running : 2
? __pfx_kthread (kernel/kthread.c:341)
.avg_vruntime : 4395792.774550
</TASK>
? __pfx_hrtimer_wakeup (kernel/time/hrtimer.c:1935)
.min_vruntime : 3164982.960861
.idle_nr_running : 0
.se->vruntime : 3164982.960861
.h_nr_running : 3
console_blast.s 4091 22.266887 E 24.855117 3.000000 2.499660 3 120
RDX: 000000000000005b RSI: 0000560f80c8d2c0 RDI: 0000000000000001
ret_from_fork (arch/x86/kernel/process.c:147)
.idle_nr_running : 0
idle: 1726
.avg_vruntime : 3164982.960861
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
.tg_load_avg : 0
.idle_h_nr_running : 0
.avg_vruntime : 2079767.836377
.throttled : 0
.idle_h_nr_running : 0
.idle_nr_running : 0
do_syscall_64 (arch/x86/entry/common.c:52 arch/x86/entry/common.c:83)
task:kworker/106:0H state:I
.right_vruntime : 0.000001
.throttle_count : 0
.avg_vruntime : 16272.963166
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
3301
common_nsleep (kernel/time/posix-timers.c:1353)
? __schedule (kernel/sched/core.c:6714)
.load : 1054213
<TASK>
.se->sum_exec_runtime : 34397.506925
.right_vruntime : 0.000001
0.000000 2.499660 0.000000 0.000000
.spread : 0.000000
kthread (kernel/kthread.c:388)
.se->load.weight : 12025
<TASK>
.idle_h_nr_running : 0
.se->avg.load_avg : 10
stack:0 pid:254 tgid:254 ppid:2 flags:0x00004000
.load : 1052099
stack:0 pid:658 tgid:658 ppid:2 flags:0x00004000
? preempt_count_add (./include/linux/ftrace.h:977 kernel/sched/core.c:5839 kernel/sched/core.c:5836 kernel/sched/core.c:5864)
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
rt_rq[17]:
/user.slice/user-0.slice/session-1.scope
.load_avg : 1029
.load_avg : 1027
.load : 12026
.rt_nr_running : 0
.nr_spread_over : 0
.right_vruntime : 0.000001
.load_avg : 11
? __pfx_kthread (kernel/kthread.c:341)
.runnable_avg : 4097
? handle_mm_fault (mm/memory.c:5222 mm/memory.c:5361)
.runnable_avg : 3073
.nr_running : 1
? __pfx_autoremove_wake_function (kernel/sched/wait.c:383)
.rt_throttled : 0
.se->avg.util_avg : 1024
Workqueue: 0x0
? __pfx_smpboot_thread_fn (kernel/smpboot.c:107)
.spread : 0.000000
.util_avg : 1024
RBP: 00007ffef1f14bd0 R08: 0000558cba56b1d0 R09: 0000000000000001
? preempt_count_add (./include/linux/ftrace.h:974 (discriminator 1) kernel/sched/core.c:5839 (discriminator 1) kernel/sched/core.c:5836 (discriminator 1) kernel/sched/core.c:5864 (discriminator 1))
.right_vruntime : 0.000001
.nr_spread_over : 0
? __pfx_kthread (kernel/kthread.c:341)
.util_est_enqueued : 1588
task:ksoftirqd/14 state:R
.util_avg : 1024
.runnable_avg : 2047
S
.se->avg.runnable_avg : 2048
.nr_running : 1
.util_est_enqueued : 511
.util_avg : 1023
R sleep 4215 786.779707 E 789.779707 3.000000 1.076314 1 120
worker_thread (kernel/workqueue.c:2734)
RSP: 002b:00007ffdba3de878 EFLAGS: 00000202
.h_nr_running : 2
.spread : 0.000000
0.000000 1.076314 0.000000 0.000000
.removed.load_avg : 0
.h_nr_running : 3
.removed.load_avg : 0
ret_from_fork (arch/x86/kernel/process.c:147)
RBP: 00007fff94510390 R08: 0000000000000000 R09: 0000000000000001
/user.slice/user-0.slice/session-1.scope
.removed.util_avg : 0
ksoftirqd/162 992 -1.041676 E 1.956071 3.000000 0.007274 3 120
.idle_nr_running : 0
R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000001
.idle_nr_running : 0
.removed.util_avg : 0
.util_est_enqueued : 0
</TASK>
.removed.runnable_avg : 0
.idle_h_nr_running : 0
.rt_time : 0.000000
.tg_load_avg_contrib : 0
.exec_clock : 0.000000
? __pfx_kcompactd (mm/compaction.c:3060)
ORIG_RAX: 00000000000000e6
.idle_h_nr_running : 0
cfs_rq[54]:/user.slice/user-0.slice
(events_highpri)
.exec_clock : 0.000000
.tg_load_avg : 0
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
.load : 12025
.load : 18037
.removed.runnable_avg : 0
R
.load_avg : 11
kthread (kernel/kthread.c:388)
.rt_runtime : 950.000000
Workqueue: 0x0
Workqueue: 0x0
Call Trace:
.removed.load_avg : 0
sleep 4277 1164.357159 E 1167.357159 3.000000 0.951373 1 120
.throttled : 0
.tg_load_avg_contrib : 0
.load_avg : 17
.left_vruntime : 0.000001
.tg_load_avg : 0
.runnable_avg : 2048
0.000000 0.951373 0.000000 0.000000
.runnable_avg : 3072
? up_read (./arch/x86/include/asm/preempt.h:104 kernel/locking/rwsem.c:1354 kernel/locking/rwsem.c:1622)
.util_avg : 1024
? __pfx_worker_thread (kernel/workqueue.c:2727)
.throttled : 0
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
.nr_spread_over : 0
.util_est_enqueued : 0
.throttle_count : 0
/user.slice/user-0.slice/session-1.scope
? do_user_addr_fault (arch/x86/mm/fault.c:1445 (discriminator 1))
R10: 0000000000000004 R11: 0000000000000202 R12: 000000000000005b
.util_avg : 1024
RAX: ffffffffffffffda RBX: 00007ffdba3de8d0 RCX: 00007f4002dba167
task:kcompactd0 state:S
.removed.load_avg : 0
(cgroup_destroy)
stack:0 pid:4248 tgid:4248 ppid:3816 flags:0x00000002
.util_est_enqueued : 0
? __pfx_worker_thread (kernel/workqueue.c:2727)
.min_vruntime : 3165099.171269
.removed.util_avg : 0
dl_rq[17]:
Call Trace:
.removed.util_avg : 0
.throttle_count : 0
rt_rq[51]:
kthread (kernel/kthread.c:388)
.nr_running : 0
.removed.runnable_avg : 0
.rt_nr_running : 0
.removed.runnable_avg : 0
.removed.load_avg : 0
task:idle_inject/19 state:S
.dl_nr_running : 0
0.000000 0.007274 0.000000 0.000000
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
.avg_vruntime : 3165169.892991
.tg_load_avg_contrib : 11
.rt_throttled : 0
cpu#21, 1900.000 MHz
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
.right_vruntime : 0.000001
Call Trace:
? default_do_nmi (arch/x86/kernel/nmi.c:351)
? __pfx_kthread (kernel/kthread.c:341)
R13: 0000560f80c8d2c0 R14: 00007ff07986a5c0 R15: 00007ff079867f20
.removed.util_avg : 0
smpboot_thread_fn (kernel/smpboot.c:160)
? _raw_spin_lock_irqsave (./arch/x86/include/asm/atomic.h:115 (discriminator 4) ./include/linux/atomic/atomic-arch-fallback.h:2164 (discriminator 4) ./include/linux/atomic/atomic-instrumented.h:1296 (discriminator 4) ./include/asm-generic/qspinlock.h:111 (discriminator 4) ./include/linux/spinlock.h:187 (discriminator 4) ./include/linux/spinlock_api_smp.h:111 (discriminator 4) kernel/locking/spinlock.c:162 (discriminator 4))
.spread : 0.000000
? __pfx_kthread (kernel/kthread.c:341)
worker_thread (kernel/workqueue.c:2734)
.tg_load_avg_contrib : 11
.rt_time : 0.000000
rt_rq[28]:
? __pfx_kthread (kernel/kthread.c:341)
.left_vruntime : 0.000001
? exc_page_fault (./arch/x86/include/asm/irqflags.h:37 ./arch/x86/include/asm/irqflags.h:72 arch/x86/mm/fault.c:1515 arch/x86/mm/fault.c:1563)
<TASK>
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
? __pfx_worker_thread (kernel/workqueue.c:2727)
? exc_nmi (arch/x86/kernel/nmi.c:545)
kthread (kernel/kthread.c:388)
? __pfx_kthread (kernel/kthread.c:341)
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
.rt_runtime : 950.000000
worker_thread (kernel/workqueue.c:2734)
.nr_running : 5
.dl_bw->bw : 996147
</TASK>
.h_nr_running : 0
.spread : 0.000000
.rt_nr_running : 0
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
kthread (kernel/kthread.c:388)
.nr_spread_over : 0
? _raw_spin_lock_irqsave (./arch/x86/include/asm/atomic.h:115 (discriminator 4) ./include/linux/atomic/atomic-arch-fallback.h:2164 (discriminator 4) ./include/linux/atomic/atomic-instrumented.h:1296 (discriminator 4) ./include/asm-generic/qspinlock.h:111 (discriminator 4) ./include/linux/spinlock.h:187 (discriminator 4) ./include/linux/spinlock_api_smp.h:111 (discriminator 4) kernel/locking/spinlock.c:162 (discriminator 4))
.tg_load_avg : 2026
</TASK>
.dl_bw->total_bw : 0
<TASK>
running task
.nr_running : 1
? __pfx_kthread (kernel/kthread.c:341)
.removed.runnable_avg : 0
entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:129)
Call Trace:
.nr_switches : 571
.tg_load_avg_contrib : 17
__x64_sys_clock_nanosleep (kernel/time/posix-timers.c:1396 kernel/time/posix-timers.c:1373 kernel/time/posix-timers.c:1373)
dl_rq[51]:
.throttled : 0
runnable tasks:
.nr_uninterruptible : -2
.dl_nr_running : 0
ret_from_fork (arch/x86/kernel/process.c:147)
.rt_throttled : 0
worker_thread (kernel/workqueue.c:2734)
.next_balance : 4294.870340
.tg_load_avg : 2026
.h_nr_running : 2
.tg_load_avg : 2026
.throttle_count : 0
.dl_bw->bw : 996147
? __pfx_kthread (kernel/kthread.c:341)
.rt_time : 0.000000
RDX: 00007ffdba3de8d0 RSI: 0000000000000000 RDI: 0000000000000000
.curr->pid : 3501
S task PID tree-key switches prio wait-time sum-exec sum-sleep
? __pfx_kthread (kernel/kthread.c:341)
ret_from_fork (arch/x86/kernel/process.c:147)
.se->exec_start : 208088.464512
.dl_bw->total_bw : 0
.clock : 208147.499540
.rt_runtime : 950.000000
.throttled : 0
Call Trace:
.idle_nr_running : 0
? __pfx_kthread (kernel/kthread.c:341)
? __pfx_kthread (kernel/kthread.c:341)
worker_thread (kernel/workqueue.c:2734)
.throttle_count : 0
-------------------------------------------------------------------------------------------------------------
.clock_task : 208037.119313
dl_rq[28]:
.se->vruntime : 6974999.717662
.idle_h_nr_running : 0
.avg_idle : 975453
<TASK>
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
Call Trace:
? preempt_count_add (./include/linux/ftrace.h:977 kernel/sched/core.c:5839 kernel/sched/core.c:5836 kernel/sched/core.c:5864)
.throttled : 0
Workqueue: 0x0
.se->sum_exec_runtime : 34453.190727
ret_from_fork (arch/x86/kernel/process.c:147)
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
runnable tasks:
.load : 12025
.se->exec_start : 208010.843851
.max_idle_balance_cost : 500000
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
.dl_nr_running : 0
.se->vruntime : 6965484.659526
S
.throttle_count : 0
.se->load.weight : 5693
.dl_bw->bw : 996147
.se->sum_exec_runtime : 34399.514985
stack:0 pid:104 tgid:104 ppid:2 flags:0x00004000
.se->load.weight : 5693
RIP: 0033:0x7f1042c5fc74
? __pfx_kthread (kernel/kthread.c:341)
? __pfx_kthread (kernel/kthread.c:341)
</TASK>
Call Trace:
.nr_spread_over : 0
<TASK>
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
S task PID tree-key switches prio wait-time sum-exec sum-sleep
cpuhp/17 119 422.431808 E 425.427449 3.000000 1.531719 35 120
.load_avg : 11
</TASK>
.se->exec_start : 208043.148735
.dl_bw->total_bw : 0
/
-------------------------------------------------------------------------------------------------------------
.runnable_avg : 2048
.se->avg.load_avg : 4
0.000000 1.531719 0.000000 0.000000
(mm_percpu_wq)
.se->avg.load_avg : 4
task:kworker/168:1 state:I
Call Trace:
.se->avg.util_avg : 1024
cfs_rq[21]:/user.slice/user-0.slice/session-1.scope
.se->vruntime : 4425426.460114
.util_avg : 1024
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
.se->sum_exec_runtime : 34795.388157
</TASK>
? __pfx_kthread (kernel/kthread.c:341)
.se->avg.runnable_avg : 2048
.exec_clock : 0.000000
/
.se->avg.util_avg : 1023
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
task:console_blast.s state:S
? __pfx_worker_thread (kernel/workqueue.c:2727)
runnable tasks:
S
.min_vruntime : 21076.391649
<TASK>
.se->load.weight : 8798
S task PID tree-key switches prio wait-time sum-exec sum-sleep
idle_inject/17 120 -1.045752 E 1.951434 3.000000 0.006978 3 49
.left_vruntime : 765.923807
.util_est_enqueued : 0
(events_unbound)
.se->avg.load_avg : 7
do_nanosleep (kernel/time/hrtimer.c:1446 kernel/time/hrtimer.c:2049)
</TASK>
0.000000 0.006978 0.000000 0.000000
.se->avg.runnable_avg : 2047
/
.se->avg.util_avg : 1024
cfs_rq[54]:/user.slice
? fill_page_cache_func (kernel/rcu/tree.c:3261 (discriminator 3))
.removed.load_avg : 0
cfs_rq[53]:/user.slice
.removed.util_avg : 0
.se->avg.runnable_avg : 3072
</TASK>
R
RBP: 00007ffdba3de880 R08: 0000000000000000 R09: 0000000000000000
.exec_clock : 0.000000
migration/17 121 39.697241 E 40.446470 0.750000 10233.033387 46 0
task:kworker/171:1 state:I
.removed.runnable_avg : 0
ret_from_fork (arch/x86/kernel/process.c:147)
.left_vruntime : 0.000001
.tg_load_avg_contrib : 11
.min_vruntime : 6974999.717662
-------------------------------------------------------------------------------------------------------------
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
.avg_vruntime : 6974999.717662
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
.tg_load_avg : 2026
.exec_clock : 0.000000
S
stack:0 pid:3942 tgid:3942 ppid:3454 flags:0x00000002
cfs_rq[26]:/user.slice
.min_vruntime : 765.923807
.avg_vruntime : 21076.391649
Call Trace:
.throttled : 0
S
.right_vruntime : 0.000001
cpuhp/51 323 2096.880431 E 2099.874038 3.000000 1.579524 35 120
cpuhp/28 185 300.474850 E 303.470940 3.000000 1.430283 35 120
0.000000 10233.033387 0.000000 0.000000
.left_vruntime : 0.000001
.avg_vruntime : 12296.313565
0.000000 1.579524 0.000000 0.000000
0.000000 1.430283 0.000000 0.000000
.exec_clock : 0.000000
.min_vruntime : 6965694.959634
/
.right_vruntime : 1331.022673
.spread : 0.000000
S
RSP: 002b:00007ffc5ac123d8 EFLAGS: 00000202
/
.spread : 565.098866
.throttle_count : 0
.nr_spread_over : 0
idle_inject/51 324 -1.046441 E 1.951574 3.000000 0.007516 3 49
stack:0 pid:1601 tgid:1601 ppid:2 flags:0x00004000
R
.nr_spread_over : 0
.nr_running : 1
.avg_vruntime : 6965694.959634
stack:0 pid:132 tgid:132 ppid:2 flags:0x00004000
.nr_running : 3
</TASK>
.left_vruntime : 0.000001
.se->exec_start : 208011.870707
/
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
ksoftirqd/17 122 84879.423171 E 84882.423171 3.000000 0.024133 5 120
do_syscall_64 (arch/x86/entry/common.c:52 arch/x86/entry/common.c:83)
.se->vruntime : 6965694.959634
0.000000 0.024133 0.000000 0.000000
ORIG_RAX: 0000000000000001
? __schedule (kernel/sched/core.c:6714)
S
Call Trace:
.nr_running : 1
.min_vruntime : 4425426.460114
kthread (kernel/kthread.c:388)
.h_nr_running : 3
? fill_page_cache_func (kernel/rcu/tree.c:3261 (discriminator 3))
idle_inject/28 186 0.858638 E 1.951437 3.000000 0.193871 3 49
<TASK>
.se->sum_exec_runtime : 34400.541841
.avg_vruntime : 4425426.460114
0.000000 0.007516 0.000000 0.000000
.idle_nr_running : 0
/
.right_vruntime : 0.000001
? end_repeat_nmi (arch/x86/entry/entry_64.S:1394)
.right_vruntime : 0.000001
.h_nr_running : 2
? preempt_count_add (./include/linux/ftrace.h:977 kernel/sched/core.c:5839 kernel/sched/core.c:5836 kernel/sched/core.c:5864)
? _raw_spin_lock_irqsave (./arch/x86/include/asm/atomic.h:115 (discriminator 4) ./include/linux/atomic/atomic-arch-fallback.h:2164 (discriminator 4) ./include/linux/atomic/atomic-instrumented.h:1296 (discriminator 4) ./include/asm-generic/qspinlock.h:111 (discriminator 4) ./include/linux/spinlock.h:187 (discriminator 4) ./include/linux/spinlock_api_smp.h:111 (discriminator 4) kernel/locking/spinlock.c:162 (discriminator 4))
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
.spread : 0.000000
.idle_nr_running : 0
? exc_page_fault (./arch/x86/include/asm/irqflags.h:37 ./arch/x86/include/asm/irqflags.h:72 arch/x86/mm/fault.c:1515 arch/x86/mm/fault.c:1563)
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
.idle_nr_running : 0
stack:0 pid:1778 tgid:1778 ppid:2 flags:0x00004000
Call Trace:
? native_queued_spin_lock_slowpath (./arch/x86/include/asm/vdso/processor.h:13 ./arch/x86/include/asm/vdso/processor.h:18 kernel/locking/qspinlock.c:383)
.spread : 0.000000
kthread (kernel/kthread.c:388)
? __pfx_smpboot_thread_fn (kernel/smpboot.c:107)
.idle_h_nr_running : 0
0.000000 0.193871 0.000000 0.000000
/
I
task:kworker/97:0 state:I
.nr_spread_over : 0
.nr_spread_over : 0
.se->load.weight : 5693
worker_thread (kernel/workqueue.c:2734)
? __pfx_kthread (kernel/kthread.c:341)
? __pfx_worker_thread (kernel/workqueue.c:2727)
task:kworker/142:1 state:I
<TASK>
/
.se->avg.load_avg : 4
R
ret_from_fork (arch/x86/kernel/process.c:147)
kworker/17:0 123 41.440664 E 44.440664 3.000000 0.001570 4 120
? __pfx_kthread (kernel/kthread.c:341)
<TASK>
.nr_running : 1
kthread (kernel/kthread.c:388)
.idle_h_nr_running : 0
stack:0 pid:1783 tgid:1783 ppid:2 flags:0x00004000
(btrfs-endio-meta)
0.000000 0.001570 0.000000 0.000000
hrtimer_nanosleep (kernel/time/hrtimer.c:2101)
.nr_running : 1
migration/51 325 49.479633 E 50.229058 0.750000 10262.170012 47 0
entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:129)
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
.load : 3145728
.idle_h_nr_running : 0
.h_nr_running : 3
stack:0 pid:603 tgid:603 ppid:2 flags:0x00004000
kthread (kernel/kthread.c:388)
.h_nr_running : 3
? __pfx_worker_thread (kernel/workqueue.c:2727)
Call Trace:
? __pfx_kthread (kernel/kthread.c:341)
.se->avg.util_avg : 1024
RAX: ffffffffffffffda RBX: 000000000000005b RCX: 00007f1042c5fc74
</TASK>
.se->avg.runnable_avg : 2048
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
.load : 0
.h_nr_running : 2
.load_avg : 3072
.idle_nr_running : 0
.idle_nr_running : 0
ret_from_fork (arch/x86/kernel/process.c:147)
/
RIP: 0033:0x7fc3970ac167
RDX: 000000000000005b RSI: 00005625ccb682c0 RDI: 0000000000000001
.idle_h_nr_running : 0
do_nanosleep (kernel/time/hrtimer.c:1446 kernel/time/hrtimer.c:2049)
.load : 5693
worker_thread (kernel/workqueue.c:2734)
0.000000 10262.170012 0.000000 0.000000
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
.right_vruntime : 0.000001
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
I
task:taskset state:S
? __pfx_worker_thread (kernel/workqueue.c:2727)
.load_avg : 0
kworker/17:0H 124 48.219887 E 48.254073 3.000000 0.043475 4 100
.spread : 0.000000
.load_avg : 4
/
.load : 8798
? native_queued_spin_lock_slowpath (./arch/x86/include/asm/vdso/processor.h:13 ./arch/x86/include/asm/vdso/processor.h:18 kernel/locking/qspinlock.c:383)
.runnable_avg : 2047
ret_from_fork (arch/x86/kernel/process.c:147)
<TASK>
R
Workqueue: 0x0
? __pfx_hrtimer_wakeup (kernel/time/hrtimer.c:1935)
task:kworker/122:1 state:I
stack:0 pid:1753 tgid:1753 ppid:2 flags:0x00004000
smpboot_thread_fn (kernel/smpboot.c:160)
.util_avg : 1023
.runnable_avg : 3072
S
.util_avg : 1024
? __pfx_kthread (kernel/kthread.c:341)
.idle_h_nr_running : 0
.runnable_avg : 0
.util_est_enqueued : 0
? __pfx_kthread (kernel/kthread.c:341)
R10: 00007ffdba3de8c0 R11: 0000000000000202 R12: 00007f4002cde6c8
ksoftirqd/51 326 2854.180743 E 2857.165365 3.000000 0.020810 4 120
(mm_percpu_wq)
? __pfx_kthread (kernel/kthread.c:341)
common_nsleep (kernel/time/posix-timers.c:1353)
RBP: 00007ffc5ac12400 R08: 0000000000000000 R09: 0000000000000001
migration/28 187 42.813203 E 43.562459 0.750000 10240.580888 47 0
<TASK>
.load : 5693
.removed.load_avg : 0
.load_avg : 8
.load_avg : 5
.nr_spread_over : 0
R13: 0000000000000000 R14: 0000000000000000 R15: 00007f4002ebafc0
cfs_rq[54]:/user.slice
.runnable_avg : 3072
0.000000 0.043475 0.000000 0.000000
.removed.util_avg : 0
.util_avg : 1024
/
.exec_clock : 0.000000
0.000000 10240.580888 0.000000 0.000000
.left_vruntime : 0.000001
Call Trace:
0.000000 0.020810 0.000000 0.000000
.util_est_enqueued : 0
.removed.runnable_avg : 0
.util_est_enqueued : 0
/
.idle_nr_running : 0
.removed.load_avg : 0
stack:0 pid:4057 tgid:4057 ppid:4016 flags:0x00000002
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
.tg_load_avg_contrib : 4
ret_from_fork (arch/x86/kernel/process.c:147)
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
/
.removed.util_avg : 0
.runnable_avg : 2048
.removed.load_avg : 0
.removed.util_avg : 0
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
? _raw_spin_lock_irqsave (./arch/x86/include/asm/atomic.h:115 (discriminator 4) ./include/linux/atomic/atomic-arch-fallback.h:2164 (discriminator 4) ./include/linux/atomic/atomic-instrumented.h:1296 (discriminator 4) ./include/asm-generic/qspinlock.h:111 (discriminator 4) ./include/linux/spinlock.h:187 (discriminator 4) ./include/linux/spinlock_api_smp.h:111 (discriminator 4) kernel/locking/spinlock.c:162 (discriminator 4))
.tg_load_avg : 1488
.nr_running : 0
task:cpuhp/12 state:S
I
.throttled : 0
I
S
.min_vruntime : 6965694.959634
RSP: 002b:00007ffcccdb45d8 EFLAGS: 00000202
.removed.runnable_avg : 0
? __pfx_kthread (kernel/kthread.c:341)
kworker/51:0 327 40.836423 E 43.828130 3.000000 0.010866 4 120
</TASK>
I
kworker/u450:8 1427 41.440664 E 41.804848 3.000000 42.486212 3 120
.avg_vruntime : 6965694.959634
ksoftirqd/28 188 -1.045149 E 1.952472 3.000000 0.003857 3 120
Workqueue: 0x0
.util_avg : 0
.removed.runnable_avg : 0
.util_avg : 1024
.idle_h_nr_running : 0
hrtimer_nanosleep (kernel/time/hrtimer.c:2101)
.right_vruntime : 0.000001
0.000000 42.486212 0.000000 0.000000
.tg_load_avg_contrib : 8
.spread : 0.000000
ORIG_RAX: 00000000000000e6
0.000000 0.003857 0.000000 0.000000
<TASK>
.util_est_enqueued : 0
task:kworker/127:0H state:I
stack:0 pid:89 tgid:89 ppid:2 flags:0x00004000
stack:0 pid:1734 tgid:1734 ppid:2 flags:0x00004000
0.000000 0.010866 0.000000 0.000000
/
.load : 8798
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
kthread (kernel/kthread.c:388)
? __pfx_kthread (kernel/kthread.c:341)
.removed.load_avg : 0
stack:0 pid:784 tgid:784 ppid:2 flags:0x00004000
__x64_sys_clock_nanosleep (kernel/time/posix-timers.c:1396 kernel/time/posix-timers.c:1373 kernel/time/posix-timers.c:1373)
.throttle_count : 0
.nr_spread_over : 0
Call Trace:
.removed.util_avg : 0
kthread (kernel/kthread.c:388)
kworker/162:0 993 -1.039343 E 1.958324 3.000000 0.006918 4 120
.tg_load_avg_contrib : 3055
.se->exec_start : 208088.464512
/
/
.tg_load_avg : 1488
.tg_load_avg : 177599
.throttled : 0
.nr_running : 1
I
.removed.runnable_avg : 0
.throttle_count : 0
Call Trace:
.h_nr_running : 2
Call Trace:
? __pfx_kthread (kernel/kthread.c:341)
? __pfx_worker_thread (kernel/workqueue.c:2727)
</TASK>
Workqueue: 0x0
<TASK>
kworker/51:0H 328 114.234904 E 114.269398 3.000000 0.021361 4 100
? __pfx_smpboot_thread_fn (kernel/smpboot.c:107)
.throttled : 0
.se->exec_start : 208044.417476
ret_from_fork (arch/x86/kernel/process.c:147)
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
I
.idle_nr_running : 0
0.000000 0.021361 0.000000 0.000000
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
kthread (kernel/kthread.c:388)
.util_est_enqueued : 0
R10: 0000000000000004 R11: 0000000000000202 R12: 000000000000005b
.tg_load_avg_contrib : 5
R
.se->vruntime : 11611430.629741
.se->vruntime : 7038437.899121
/
.throttle_count : 0
kworker/17:1 1631 55735.034269 E 55738.034269 3.000000 0.253657 19 120
kworker/28:0 189 37.897547 E 40.897547 3.000000 0.001321 4 120
.idle_h_nr_running : 0
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
? __pfx_kthread (kernel/kthread.c:341)
.se->sum_exec_runtime : 34453.190727
.se->sum_exec_runtime : 34796.656898
<TASK>
.tg_load_avg : 1488
.se->exec_start : 208040.777009
.throttled : 0
.load : 5693
0.000000 0.253657 0.000000 0.000000
.se->load.weight : 5637
? try_to_wake_up (./arch/x86/include/asm/preempt.h:104 ./include/linux/preempt.h:484 ./include/linux/preempt.h:484 kernel/sched/core.c:4217)
/
task:sleep state:R
.se->vruntime : 2079949.534792
.load_avg : 5
<TASK>
.se->avg.load_avg : 4
I
.se->sum_exec_runtime : 34795.774033
RAX: ffffffffffffffda RBX: 00007ffcccdb4630 RCX: 00007fc3970ac167
? __pfx_smpboot_thread_fn (kernel/smpboot.c:107)
.se->avg.util_avg : 1024
.load_avg : 8
? __pfx_kthread (kernel/kthread.c:341)
0.000000 0.001321 0.000000 0.000000
.se->load.weight : 3521
kworker/u452:17 1469 37.199095 E 37.631311 3.000000 38.245536 3 120
<TASK>
.se->avg.runnable_avg : 3072
Workqueue: 0x0
0.000000 38.245536 0.000000 0.000000
.throttle_count : 0
.se->avg.load_avg : 2
.se->load.weight : 18135
.se->exec_start : 208014.674547
.h_nr_running : 0
kthread (kernel/kthread.c:388)
? __pfx_worker_thread (kernel/workqueue.c:2727)
(events_highpri)
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
</TASK>
.runnable_avg : 2048
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
Workqueue: 0x0
task:kworker/28:1 state:R
.se->avg.util_avg : 1023
I
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
smpboot_thread_fn (kernel/smpboot.c:160)
/
.se->avg.load_avg : 16
kworker/17:1H 2489 48.220159 E 48.254739 3.000000 0.020348 2 100
cfs_rq[26]:/
0.000000 0.020348 0.000000 0.000000
.se->avg.runnable_avg : 2047
.util_avg : 1024
.se->vruntime : 11640656.065238
.se->avg.util_avg : 1024
.exec_clock : 0.000000
I
.util_est_enqueued : 0
worker_thread (kernel/workqueue.c:2734)
ret_from_fork (arch/x86/kernel/process.c:147)
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
? __pfx_kthread (kernel/kthread.c:341)
.removed.load_avg : 0
kworker/28:0H 190 93.435844 E 93.470287 3.000000 0.021741 4 100
/
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
ret_from_fork (arch/x86/kernel/process.c:147)
.removed.load_avg : 0
RDX: 00007ffcccdb4630 RSI: 0000000000000000 RDI: 0000000000000000
.left_vruntime : 240480.766952
? __pfx_worker_thread (kernel/workqueue.c:2727)
? __schedule (kernel/sched/core.c:6714)
smpboot_thread_fn (kernel/smpboot.c:160)
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
.se->sum_exec_runtime : 34403.345681
<TASK>
.idle_nr_running : 0
smpboot_thread_fn (kernel/smpboot.c:160)
(mm_percpu_wq)
.removed.util_avg : 0
kthread (kernel/kthread.c:388)
/
>R
.se->avg.runnable_avg : 3072
.min_vruntime : 240496.180960
? __pfx_kthread (kernel/kthread.c:341)
R13: 00005625ccb682c0 R14: 00007f1042d305c0 R15: 00007f1042d2df20
0.000000 0.006918 0.000000 0.000000
kthread (kernel/kthread.c:388)
do_syscall_64 (arch/x86/entry/common.c:52 arch/x86/entry/common.c:83)
kthread (kernel/kthread.c:388)
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
.removed.runnable_avg : 0
? native_queued_spin_lock_slowpath (./arch/x86/include/asm/vdso/processor.h:13 ./arch/x86/include/asm/vdso/processor.h:18 kernel/locking/qspinlock.c:383)
worker_thread (kernel/workqueue.c:2734)
I
Call Trace:
cfs_rq[53]:/
console_blast.s 3490 34793.174413 N 34794.741364 3.000000 34789.492844 5 120
0.000000 0.021741 0.000000 0.000000
kworker/u451:25 1582 40.828130 E 43.607466 3.000000 11.615802 5 120
ret_from_fork (arch/x86/kernel/process.c:147)
.exec_clock : 0.000000
cfs_rq[21]:/user.slice/user-0.slice
Call Trace:
.se->load.weight : 3523
.avg_vruntime : 273514.135274
/
</TASK>
</TASK>
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
.left_vruntime : 265668.598558
.tg_load_avg_contrib : 5
? __pfx_kthread (kernel/kthread.c:341)
ret_from_fork (arch/x86/kernel/process.c:147)
.min_vruntime : 265685.378461
.right_vruntime : 240480.766952
.removed.util_avg : 0
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
.se->avg.load_avg : 2
.avg_vruntime : 286641.467372
.tg_load_avg : 1488
.exec_clock : 0.000000
.right_vruntime : 274436.242937
.throttled : 0
0.000000 34789.492844 0.000000 0.000000
.spread : 8767.644379
0.000000 11.615802 0.000000 0.000000
.runnable_avg : 3072
.spread : 0.000000
/
.se->avg.util_avg : 1024
RBP: 00007ffcccdb45e0 R08: 0000000000000000 R09: 0000000000000000
.se->avg.runnable_avg : 2048
? __pfx_smpboot_thread_fn (kernel/smpboot.c:107)
worker_thread (kernel/workqueue.c:2734)
</TASK>
/user.slice/user-0.slice/session-1.scope
I
.nr_spread_over : 0
.nr_spread_over : 0
.throttle_count : 0
kthread (kernel/kthread.c:388)
.nr_running : 2
cfs_rq[54]:/
kworker/u451:7 1407 235.252529 E 238.174464 3.000000 49.705933 35 120
.h_nr_running : 4
S
.left_vruntime : 0.000001
.idle_nr_running : 0
.se->exec_start : 208014.674547
task:kworker/39:0 state:I
? __pfx_kthread (kernel/kthread.c:341)
.removed.runnable_avg : 0
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
.util_avg : 1024
.se->vruntime : 11640987.856939
Call Trace:
0.000000 49.705933 0.000000 0.000000
ret_from_fork (arch/x86/kernel/process.c:147)
? __count_memcg_events (mm/memcontrol.c:631 (discriminator 4) mm/memcontrol.c:936 (discriminator 4))
.idle_h_nr_running : 0
.nr_running : 3
(mm_percpu_wq)
.min_vruntime : 2079949.534792
.load : 1054213
R
.idle_h_nr_running : 0
.exec_clock : 0.000000
.se->sum_exec_runtime : 34404.317727
console_blast.s 3708 4.545572 E 7.100117 3.000000 3.332835 4 120
/
kthread (kernel/kthread.c:388)
.avg_vruntime : 2080018.807367
.load_avg : 1029
.h_nr_running : 4
? __pfx_kthread (kernel/kthread.c:341)
.runnable_avg : 4097
kworker/51:1 1666 10547918.653827 E 10547921.653827 3.000000 0.503578 15 120
.se->load.weight : 3523
.left_vruntime : 0.000001
.util_avg : 1024
0.000000 3.332835 0.000000 0.000000
R
.min_vruntime : 11640987.856939
/user.slice/user-0.slice/session-1.scope
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
.idle_nr_running : 0
kworker/28:1 1642 59318.094212 E 59321.094212 3.000000 2.414195 253 120
.avg_vruntime : 11640987.856939
.tg_load_avg_contrib : 0
.right_vruntime : 0.000001
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
0.000000 0.503578 0.000000 0.000000
.util_est_enqueued : 1328
.idle_h_nr_running : 0
.se->avg.load_avg : 2
0.000000 2.414195 0.000000 0.000000
/
.spread : 0.000000
.right_vruntime : 0.000001
</TASK>
.nr_spread_over : 0
/
.se->avg.util_avg : 1024
? __pfx_hrtimer_wakeup (kernel/time/hrtimer.c:1935)
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
.load : 2100673
? __pfx_kthread (kernel/kthread.c:341)
task:taskset state:S
.removed.load_avg : 0
.nr_running : 1
? __pfx_kthread (kernel/kthread.c:341)
.h_nr_running : 3
S
.load_avg : 2050
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
.spread : 0.000000
.se->avg.runnable_avg : 2048
.idle_nr_running : 0
.removed.util_avg : 0
.util_est_enqueued : 0
<TASK>
do_wait (kernel/exit.c:1625 (discriminator 4))
console_blast.s 3901 5.990935 E 8.526463 3.000000 3.076880 4 120
? __pfx_kthread (kernel/kthread.c:341)
.nr_spread_over : 0
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
? __pfx_kthread (kernel/kthread.c:341)
.runnable_avg : 4095
.idle_h_nr_running : 0
stack:0 pid:4100 tgid:4100 ppid:4061 flags:0x00000002
schedule_preempt_disabled (kernel/sched/core.c:6856)
/
cfs_rq[54]:/
.nr_running : 1
.util_avg : 1023
.exec_clock : 0.000000
.load : 18237
0.000000 3.076880 0.000000 0.000000
.util_est_enqueued : 930
.load_avg : 17
.left_vruntime : 0.000001
I
/user.slice/user-0.slice/session-1.scope
.h_nr_running : 2
S
.min_vruntime : 11640987.856939
.idle_nr_running : 0
kswapd2 1845 38.002790 E 40.963900 3.000000 0.055285 4 120
Workqueue: 0x0
running task
.runnable_avg : 3072
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
? preempt_count_add (./include/linux/ftrace.h:977 kernel/sched/core.c:5839 kernel/sched/core.c:5836 kernel/sched/core.c:5864)
.removed.runnable_avg : 0
sysrq: Show State
.util_avg : 1024
.avg_vruntime : 11640987.856939
</TASK>
.right_vruntime : 0.000001
task:cpuhp/11 state:S
kworker/51:1H 2822 1892.062876 E 1892.097374 3.000000 0.080855 6 100
? __pfx_kthread (kernel/kthread.c:341)
.spread : 0.000000
smpboot_thread_fn (kernel/smpboot.c:160)
? handle_mm_fault (mm/memory.c:5222 mm/memory.c:5361)
.removed.load_avg : 0
R
.removed.load_avg : 0
.nr_spread_over : 0
.tg_load_avg_contrib : 0
? fill_page_cache_func (kernel/rcu/tree.c:3261 (discriminator 3))
.util_est_enqueued : 0
kernel_wait4 (kernel/exit.c:1786)
.nr_running : 1
running task
ret_from_fork (arch/x86/kernel/process.c:147)
ret_from_fork (arch/x86/kernel/process.c:147)
sleep 4210 767.543871 E 770.543871 3.000000 1.021893 1 120
<TASK>
.idle_h_nr_running : 0
.removed.load_avg : 0
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
0.000000 0.080855 0.000000 0.000000
? __pfx_child_wait_callback (kernel/exit.c:1510)
stack:0 pid:255 tgid:255 ppid:2 flags:0x00004000
0.000000 0.055285 0.000000 0.000000
.load : 0
0.000000 1.021893 0.000000 0.000000
.removed.util_avg : 0
.tg_load_avg : 0
.h_nr_running : 2
/
.tg_load_avg : 0
/
? __pfx_kthread (kernel/kthread.c:341)
kthread (kernel/kthread.c:388)
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
do_syscall_64 (arch/x86/entry/common.c:52 arch/x86/entry/common.c:83)
(mm_percpu_wq)
ret_from_fork (arch/x86/kernel/process.c:147)
? __pfx_kthread (kernel/kthread.c:341)
.throttled : 0
task:kworker/97:0H state:I
.idle_nr_running : 0
? preempt_count_add (./include/linux/ftrace.h:974 (discriminator 1) kernel/sched/core.c:5839 (discriminator 1) kernel/sched/core.c:5836 (discriminator 1) kernel/sched/core.c:5864 (discriminator 1))
.load : 3523
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
ret_from_fork (arch/x86/kernel/process.c:147)
ret_from_fork (arch/x86/kernel/process.c:147)
worker_thread (kernel/workqueue.c:2734)
.throttle_count : 0
<TASK>
S
</TASK>
.removed.runnable_avg : 0
.idle_h_nr_running : 0
R10: 00007ffcccdb4620 R11: 0000000000000202 R12: 00007fc396fd06c8
.tg_load_avg_contrib : 17
.load_avg : 3
common_nsleep (kernel/time/posix-timers.c:1353)
.load : 3523
rt_rq[26]:
.removed.util_avg : 0
? __pfx_kthread (kernel/kthread.c:341)
irq/49-aerdrv 1875 39.642784 E 42.642719 3.000000 0.015890 3 49
? __pfx_kthread (kernel/kthread.c:341)
? __pfx_worker_thread (kernel/workqueue.c:2727)
.tg_load_avg : 2026
/user.slice/user-0.slice/session-1.scope
R
.removed.runnable_avg : 0
.rt_nr_running : 0
.runnable_avg : 2048
console_blast.s 3615 29882.540004 N 29885.540004 3.000000 29883.915066 5 120
.throttled : 0
.load_avg : 3
? __pfx_kthread (kernel/kthread.c:341)
? __pfx_kthread (kernel/kthread.c:341)
.util_avg : 1024
Workqueue: 0x0
.tg_load_avg_contrib : 0
0.000000 0.015890 0.000000 0.000000
? __pfx_kthread (kernel/kthread.c:341)
? __pfx_kthread (kernel/kthread.c:341)
stack:0 pid:604 tgid:604 ppid:2 flags:0x00004000
.runnable_avg : 2048
.tg_load_avg : 0
.throttle_count : 0
0.000000 29883.915066 0.000000 0.000000
.load_avg : 0
.se->exec_start : 208042.810223
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
__x64_sys_clock_nanosleep (kernel/time/posix-timers.c:1396 kernel/time/posix-timers.c:1373 kernel/time/posix-timers.c:1373)
.util_est_enqueued : 387
.throttled : 0
.util_avg : 1024
? up_read (./arch/x86/include/asm/preempt.h:104 kernel/locking/rwsem.c:1354 kernel/locking/rwsem.c:1622)
/
task:systemd state:S
/user.slice/user-0.slice/session-1.scope
schedule (kernel/sched/core.c:6784 kernel/sched/core.c:6798)
(mm_percpu_wq)
.rt_throttled : 0
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
? syscall_exit_to_user_mode (kernel/entry/common.c:299)
.se->vruntime : 4424947.833142
stack:0 pid:83 tgid:83 ppid:2 flags:0x00004000
.rt_time : 0.000000
? __pfx_worker_thread (kernel/workqueue.c:2727)
(mm_percpu_wq)
ret_from_fork_asm (arch/x86/entry/entry_64.S:250)
R
__schedule (kernel/sched/core.c:5396 kernel/sched/core.c:6708)
ret_from_fork (arch/x86/kernel/process.c:147)
schedule_timeout (kernel/time/timer.c:1635 ./include/linux/timer.h:199 kernel/time/timer.c:2175)
>R
.removed.load_avg : 0
</TASK>
.se->sum_exec_runtime : 34797.807247
R13: 00007ffcccdb4620 R14: 0000000000000002 R15: 00007ffcccdb4620
.throttle_count : 0
console_blast.s 3810 19840.925913 E 19843.377808 3.000000 4523.842729 4 120
.rt_runtime : 950.000000
.removed.util_avg : 0
do_syscall_64 (arch/x86/entry/common.c:52 arch/x86/entry/common.c:83)
? __pfx_kthread (kernel/kthread.c:341)
I
? __pfx_kthread (kernel/kthread.c:341)
rt_rq[53]:
? do_user_addr_fault (arch/x86/mm/fault.c:1445 (discriminator 1))
dl_rq[26]:
stack:0 pid:1642 tgid:1642 ppid:2 flags:0x00004000
.se->load.weight : 8798
.rt_nr_running : 0
0.000000 4523.842729 0.000000 0.000000
task:cpuhp/107 state:S
.dl_nr_running : 0
.removed.runnable_avg : 0
sleep 4272 1146.188899 E 1149.188899 3.000000 1.275331 1 120
.se->avg.load_avg : 7
.rt_throttled : 0
? do_syscall_64 (arch/x86/entry/common.c:102)
</NMI>
---------------------------------
<NMI>
dump_stack_lvl (lib/dump_stack.c:107)
panic (kernel/panic.c:344)
nmi_panic (kernel/panic.c:203)
watchdog_hardlockup_check (kernel/watchdog.c:180)
__perf_event_overflow (kernel/events/core.c:9612 (discriminator 2))
handle_pmi_common (arch/x86/events/intel/core.c:3052 (discriminator 1))
? set_pte_vaddr_p4d (arch/x86/mm/init_64.c:307 arch/x86/mm/init_64.c:315)
? flush_tlb_one_kernel (./arch/x86/include/asm/paravirt.h:81 arch/x86/mm/tlb.c:1171 arch/x86/mm/tlb.c:1126)
? native_set_fixmap (arch/x86/mm/pgtable.c:679 arch/x86/mm/pgtable.c:688)
? ghes_copy_tofrom_phys (drivers/acpi/apei/ghes.c:330)
intel_pmu_handle_irq (./arch/x86/include/asm/msr.h:84 ./arch/x86/include/asm/msr.h:118 arch/x86/events/intel/core.c:2427 arch/x86/events/intel/core.c:3118)
perf_event_nmi_handler (arch/x86/events/core.c:1743 arch/x86/events/core.c:1729)
? native_queued_spin_lock_slowpath (./arch/x86/include/asm/vdso/processor.h:13 ./arch/x86/include/asm/vdso/processor.h:18 kernel/locking/qspinlock.c:383)
nmi_handle (arch/x86/kernel/nmi.c:150)
? native_queued_spin_lock_slowpath (./arch/x86/include/asm/vdso/processor.h:13 ./arch/x86/include/asm/vdso/processor.h:18 kernel/locking/qspinlock.c:383)
default_do_nmi (arch/x86/kernel/nmi.c:351)
exc_nmi (arch/x86/kernel/nmi.c:545)
end_repeat_nmi (arch/x86/entry/entry_64.S:1394)
RIP: 0010:native_queued_spin_lock_slowpath (./arch/x86/include/asm/vdso/processor.h:13 ./arch/x86/include/asm/vdso/processor.h:18 kernel/locking/qspinlock.c:383)
Code: 00 00 77 77 f0 0f ba 2b 08 0f 92 c2 8b 03 0f b6 d2 c1 e2 08 30 e4 09 d0 3d ff 00 00 00 77 53 85 c0 74 10 0f b6 03 84 c0 74 09 <f3> 90 0f b6 03 84 c0 75 f7 b8 01 00 00 00 66 89 03 5b 5d 41 5c 41
All code
========
0: 00 00 add %al,(%rax)
2: 77 77 ja 0x7b
4: f0 0f ba 2b 08 lock btsl $0x8,(%rbx)
9: 0f 92 c2 setb %dl
c: 8b 03 mov (%rbx),%eax
e: 0f b6 d2 movzbl %dl,%edx
11: c1 e2 08 shl $0x8,%edx
14: 30 e4 xor %ah,%ah
16: 09 d0 or %edx,%eax
18: 3d ff 00 00 00 cmp $0xff,%eax
1d: 77 53 ja 0x72
1f: 85 c0 test %eax,%eax
21: 74 10 je 0x33
23: 0f b6 03 movzbl (%rbx),%eax
26: 84 c0 test %al,%al
28: 74 09 je 0x33
2a:* f3 90 pause <-- trapping instruction
2c: 0f b6 03 movzbl (%rbx),%eax
2f: 84 c0 test %al,%al
31: 75 f7 jne 0x2a
33: b8 01 00 00 00 mov $0x1,%eax
38: 66 89 03 mov %ax,(%rbx)
3b: 5b pop %rbx
3c: 5d pop %rbp
3d: 41 5c pop %r12
3f: 41 rex.B
Code starting with the faulting instruction
===========================================
0: f3 90 pause
2: 0f b6 03 movzbl (%rbx),%eax
5: 84 c0 test %al,%al
7: 75 f7 jne 0x0
9: b8 01 00 00 00 mov $0x1,%eax
e: 66 89 03 mov %ax,(%rbx)
11: 5b pop %rbx
12: 5d pop %rbp
13: 41 5c pop %r12
15: 41 rex.B
RSP: 0018:ff26217019714e98 EFLAGS: 00000002
RAX: 0000000000000001 RBX: ffffffff8e8d49c0 RCX: 0000000000000000
RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffffffff8e8d49c0
RBP: 0000000000000046 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: ff26217019714ff8 R12: 0000000000000000
R13: ff15126c46b88000 R14: 00000000000000c2 R15: ff15130c44681254
? native_queued_spin_lock_slowpath (./arch/x86/include/asm/vdso/processor.h:13 ./arch/x86/include/asm/vdso/processor.h:18 kernel/locking/qspinlock.c:383)
? native_queued_spin_lock_slowpath (./arch/x86/include/asm/vdso/processor.h:13 ./arch/x86/include/asm/vdso/processor.h:18 kernel/locking/qspinlock.c:383)
</NMI>
--
Sincerly,
John Wyatt
Software Engineer, Core Kernel
next prev parent reply other threads:[~2024-02-22 5:21 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-02-08 0:43 NMI Reported with console_blast.sh John B. Wyatt IV
2024-02-09 9:11 ` John Ogness
2024-02-09 20:50 ` John B. Wyatt IV
2024-02-10 21:27 ` John Ogness
2024-02-12 21:23 ` John B. Wyatt IV
2024-02-22 5:21 ` John B. Wyatt IV [this message]
2024-02-29 11:15 ` Petr Mladek
2024-02-29 11:19 ` Petr Mladek
2024-03-01 10:11 ` Sebastian Andrzej Siewior
2024-03-01 16:53 ` John B. Wyatt IV
2024-02-20 22:24 ` John B. Wyatt IV
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ZdbZz-v_lug9_t6C@thinkpad2021 \
--to=jwyatt@redhat.com \
--cc=debarbos@redhat.com \
--cc=jlelli@redhat.com \
--cc=john.ogness@linutronix.de \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rt-users@vger.kernel.org \
--cc=pmladek@suse.com \
--cc=sageofredondo@gmail.com \
--cc=williams@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.