From: Steven Rostedt <rostedt@goodmis.org>
To: Naresh Kamboju <naresh.kamboju@linaro.org>
Cc: open list <linux-kernel@vger.kernel.org>,
Linux trace kernel <linux-trace-kernel@vger.kernel.org>,
lkft-triage@lists.linaro.org,
Stephen Rothwell <sfr@canb.auug.org.au>,
Masami Hiramatsu <mhiramat@kernel.org>,
Arnd Bergmann <arnd@arndb.de>,
Dan Carpenter <dan.carpenter@linaro.org>,
Anders Roxell <anders.roxell@linaro.org>,
Peter Zijlstra <peterz@infradead.org>,
Ingo Molnar <mingo@kernel.org>,
Thomas Gleixner <tglx@linutronix.de>,
Borislav Petkov <bp@alien8.de>,
Dave Hansen <dave.hansen@linux.intel.com>,
x86@kernel.org
Subject: Re: next-20250605: Test regression: qemu-x86_64-compat mode ltp tracing Oops int3 kernel panic
Date: Mon, 9 Jun 2025 20:29:11 -0400 [thread overview]
Message-ID: <20250609202911.2e25f528@gandalf.local.home> (raw)
In-Reply-To: <CA+G9fYsLu0roY3DV=tKyqP7FEKbOEETRvTDhnpPxJGbA=Cg+4w@mail.gmail.com>
[ Adding x86 and text_poke folks ]
On Thu, 5 Jun 2025 17:12:10 +0530
Naresh Kamboju <naresh.kamboju@linaro.org> wrote:
> Regressions found on qemu-x86_64 with compat mode (64-bit kernel
> running on 32-bit userspace) while running LTP tracing test suite
> on Linux next-20250605 tag kernel.
>
> Regressions found on
> - LTP tracing
>
> Regression Analysis:
> - New regression? Yes
> - Reproducible? Intermittent
>
> Test regression: qemu-x86_64-compat mode ltp tracing Oops int3 kernel panic
>
> Reported-by: Linux Kernel Functional Testing <lkft@linaro.org>
>
> ## Test log
> ftrace-stress-test: <12>[ 21.971153] /usr/local/bin/kirk[277]:
> starting test ftrace-stress-test (ftrace_stress_test.sh 90)
> <4>[ 58.997439] Oops: int3: 0000 [#1] SMP PTI
Did anything change with text_poke? Ftrace just happens to stress text_poke
more than anything else, as it updates tens of thousands of locations at a time.
The ftrace code hasn't changed in a while, but I think there's been updates
to text_poke.
The modifying of code and adding and removing the int3 handler needs to be
synchronized correctly or something like this bug can happen.
-- Steve
> <4>[ 58.998089] CPU: 0 UID: 0 PID: 323 Comm: sh Not tainted
> 6.15.0-next-20250605 #1 PREEMPT(voluntary)
> <4>[ 58.998152] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009),
> BIOS 1.16.3-debian-1.16.3-2 04/01/2014
> <4>[ 58.998260] RIP: 0010:_raw_spin_lock+0x5/0x50
> <4>[ 58.998563] Code: 5d e9 ff 12 00 00 66 66 2e 0f 1f 84 00 00 00
> 00 00 0f 1f 40 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3
> 0f 1e fa 0f <1f> 44 00 00 55 48 89 e5 53 48 89 fb bf 01 00 00 00 e8 15
> 12 e4 fe
> <4>[ 58.998610] RSP: 0018:ffff9494007bbe98 EFLAGS: 00000246
> <4>[ 58.998715] RAX: ffff912a042edd00 RBX: 000000000000000b RCX:
> 0000000000000000
> <4>[ 58.998727] RDX: 0000000000000000 RSI: 0000000000000006 RDI:
> ffff912a00f2c8c0
> <4>[ 58.998737] RBP: ffff9494007bbeb8 R08: 0000000000000000 R09:
> 0000000000000000
> <4>[ 58.998748] R10: 0000000000000000 R11: 0000000000000000 R12:
> ffff912a00f2c8c0
> <4>[ 58.998759] R13: ffff912a00f2c840 R14: 0000000000000006 R15:
> 0000000000000000
> <4>[ 58.998804] FS: 0000000000000000(0000)
> GS:ffff912ad7cbf000(0063) knlGS:00000000f7f05580
> <4>[ 58.998821] CS: 0010 DS: 002b ES: 002b CR0: 0000000080050033
> <4>[ 58.998832] CR2: 00000000f7d8f890 CR3: 000000010124e000 CR4:
> 00000000000006f0
> <4>[ 58.998915] Call Trace:
> <4>[ 58.999010] <TASK>
> <4>[ 58.999077] ? file_close_fd+0x32/0x60
> <4>[ 58.999147] __ia32_sys_close+0x18/0x90
> <4>[ 58.999172] ia32_sys_call+0x1c3c/0x27e0
> <4>[ 58.999183] __do_fast_syscall_32+0x79/0x1e0
> <4>[ 58.999194] do_fast_syscall_32+0x37/0x80
> <4>[ 58.999203] do_SYSENTER_32+0x23/0x30
> <4>[ 58.999211] entry_SYSENTER_compat_after_hwframe+0x84/0x8e
> <4>[ 58.999254] RIP: 0023:0xf7f0c579
> <4>[ 58.999459] Code: b8 01 10 06 03 74 b4 01 10 07 03 74 b0 01 10
> 08 03 74 d8 01 00 00 00 00 00 00 00 00 00 00 00 00 00 51 52 55 89 e5
> 0f 34 cd 80 <5d> 5a 59 c3 90 90 90 90 2e 8d b4 26 00 00 00 00 8d b4 26
> 00 00 00
> <4>[ 58.999466] RSP: 002b:00000000fff98500 EFLAGS: 00000206
> ORIG_RAX: 0000000000000006
> <4>[ 58.999479] RAX: ffffffffffffffda RBX: 000000000000000b RCX:
> 0000000000000000
> <4>[ 58.999484] RDX: 0000000000000000 RSI: 0000000000000000 RDI:
> 0000000000000000
> <4>[ 58.999488] RBP: 0000000000000000 R08: 0000000000000000 R09:
> 0000000000000000
> <4>[ 58.999492] R10: 0000000000000000 R11: 0000000000000206 R12:
> 0000000000000000
> <4>[ 58.999497] R13: 0000000000000000 R14: 0000000000000000 R15:
> 0000000000000000
> <4>[ 58.999534] </TASK>
> <4>[ 58.999579] Modules linked in:
> <4>[ 58.999895] ---[ end trace 0000000000000000 ]---
> <4>[ 58.999892] Oops: int3: 0000 [#2] SMP PTI
> <4>[ 58.999997] RIP: 0010:_raw_spin_lock+0x5/0x50
> <4>[ 59.000008] Code: 5d e9 ff 12 00 00 66 66 2e 0f 1f 84 00 00 00
> 00 00 0f 1f 40 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3
> 0f 1e fa 0f <1f> 44 00 00 55 48 89 e5 53 48 89 fb bf 01 00 00 00 e8 15
> 12 e4 fe
> <4>[ 59.000010] CPU: 1 UID: 0 PID: 339 Comm: sh Tainted: G D
> 6.15.0-next-20250605 #1 PREEMPT(voluntary)
> <4>[ 59.000014] RSP: 0018:ffff9494007bbe98 EFLAGS: 00000246
> <4>[ 59.000021] RAX: ffff912a042edd00 RBX: 000000000000000b RCX:
> 0000000000000000
> <4>[ 59.000026] RDX: 0000000000000000 RSI: 0000000000000006 RDI:
> ffff912a00f2c8c0
> <4>[ 59.000030] RBP: ffff9494007bbeb8 R08: 0000000000000000 R09:
> 0000000000000000
> <4>[ 59.000040] R10: 0000000000000000 R11: 0000000000000000 R12:
> ffff912a00f2c8c0
> <4>[ 59.000044] R13: ffff912a00f2c840 R14: 0000000000000006 R15:
> 0000000000000000
> <4>[ 59.000049] FS: 0000000000000000(0000)
> GS:ffff912ad7cbf000(0063) knlGS:00000000f7f05580
> <4>[ 59.000054] CS: 0010 DS: 002b ES: 002b CR0: 0000000080050033
> <4>[ 59.000059] CR2: 00000000f7d8f890 CR3: 000000010124e000 CR4:
> 00000000000006f0
> <4>[ 59.000070] Tainted: [D]=DIE
> <4>[ 59.000080] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009),
> BIOS 1.16.3-debian-1.16.3-2 04/01/2014
> <4>[ 59.000085] RIP: 0010:_raw_spin_lock+0x5/0x50
> <4>[ 59.000101] Code: 5d e9 ff 12 00 00 66 66 2e 0f 1f 84 00 00 00
> 00 00 0f 1f 40 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 f3
> 0f 1e fa 0f <1f> 44 00 00 55 48 89 e5 53 48 89 fb bf 01 00 00 00 e8 15
> 12 e4 fe
> <4>[ 59.000108] RSP: 0018:ffff9494000e0e88 EFLAGS: 00000097
> <4>[ 59.000117] RAX: 0000000000010002 RBX: ffff912a7bd29500 RCX:
> ffff912a7bd2a400
> <0>[ 59.000179] Kernel panic - not syncing: Fatal exception in interrupt
> <0>[ 60.592321] Shutting down cpus with NMI
> <0>[ 60.593242] Kernel Offset: 0x20800000 from 0xffffffff81000000
> (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
> <0>[ 60.618536] ---[ end Kernel panic - not syncing: Fatal exception
> in interrupt ]---
>
> ## Source
> * Kernel version: 6.15.0-next-20250605
> * Git tree: https://kernel.googlesource.com/pub/scm/linux/kernel/git/next/linux-next.git
> * Git sha: 4f27f06ec12190c7c62c722e99ab6243dea81a94
>
> ## Build
> * Test log: https://qa-reports.linaro.org/api/testruns/28675335/log_file/
> * Build link: https://storage.tuxsuite.com/public/linaro/lkft/builds/2y4whKazVqJKOUFD08taHC8XHRq/
> * Kernel config:
> https://storage.tuxsuite.com/public/linaro/lkft/builds/2y4whKazVqJKOUFD08taHC8XHRq/config
>
>
> --
> Linaro LKFT
> https://lkft.linaro.org
prev parent reply other threads:[~2025-06-10 0:27 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-06-05 11:42 next-20250605: Test regression: qemu-x86_64-compat mode ltp tracing Oops int3 kernel panic Naresh Kamboju
2025-06-09 13:09 ` Masami Hiramatsu
2025-06-10 8:41 ` Masami Hiramatsu
2025-06-10 13:25 ` Steven Rostedt
2025-06-10 13:20 ` Naresh Kamboju
2025-06-10 14:43 ` Masami Hiramatsu
2025-06-10 14:47 ` [RFC PATCH 1/2] x86: Retry with new instruction if INT3 is disappaered Masami Hiramatsu (Google)
2025-06-10 14:47 ` [RFC PATCH 2/2] x86: alternative: Invalidate the cache for updated instructions Masami Hiramatsu (Google)
2025-06-10 15:50 ` Steven Rostedt
2025-06-11 0:21 ` Masami Hiramatsu
2025-06-11 10:26 ` Masami Hiramatsu
2025-06-11 14:20 ` Steven Rostedt
2025-06-11 15:42 ` Steven Rostedt
2025-06-12 0:04 ` Masami Hiramatsu
2025-06-11 11:30 ` Peter Zijlstra
2025-06-12 0:17 ` Masami Hiramatsu
2025-06-12 16:24 ` Naresh Kamboju
2025-06-13 3:09 ` Masami Hiramatsu
2025-06-10 14:53 ` next-20250605: Test regression: qemu-x86_64-compat mode ltp tracing Oops int3 kernel panic Steven Rostedt
2025-06-12 13:09 ` Naresh Kamboju
2025-06-13 8:27 ` Masami Hiramatsu
2025-06-13 12:01 ` Masami Hiramatsu
2025-06-16 7:36 ` Masami Hiramatsu
2025-06-17 10:41 ` Masami Hiramatsu
2025-06-17 12:10 ` Naresh Kamboju
2025-06-17 12:25 ` Steven Rostedt
2025-06-17 12:31 ` Naresh Kamboju
2025-06-17 14:29 ` Steven Rostedt
2025-06-17 23:40 ` Masami Hiramatsu
2025-06-19 14:00 ` Steven Rostedt
2025-06-17 16:45 ` Naresh Kamboju
2025-06-17 23:05 ` Masami Hiramatsu
2025-06-17 23:32 ` Steven Rostedt
2025-06-10 0:29 ` Steven Rostedt [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20250609202911.2e25f528@gandalf.local.home \
--to=rostedt@goodmis.org \
--cc=anders.roxell@linaro.org \
--cc=arnd@arndb.de \
--cc=bp@alien8.de \
--cc=dan.carpenter@linaro.org \
--cc=dave.hansen@linux.intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-trace-kernel@vger.kernel.org \
--cc=lkft-triage@lists.linaro.org \
--cc=mhiramat@kernel.org \
--cc=mingo@kernel.org \
--cc=naresh.kamboju@linaro.org \
--cc=peterz@infradead.org \
--cc=sfr@canb.auug.org.au \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).