* [bug report] BUG: kernel NULL pointer dereference, address: 0000000000000001
@ 2025-06-23 2:58 Changhui Zhong
2025-06-23 4:02 ` Ming Lei
0 siblings, 1 reply; 6+ messages in thread
From: Changhui Zhong @ 2025-06-23 2:58 UTC (permalink / raw)
To: Linux Block Devices; +Cc: Ming Lei
Hello,
the following kernel panic was triggered by ubdsrv generic/002,
please help check and let me know if you need any info/test, thanks.
commit HEAD:
commit 2589cd05008205ee29f5f66f24a684732ee2e3a3
Merge: 98d0347fe8fb e1c75831f682
Author: Jens Axboe <axboe@kernel.dk>
Date: Wed Jun 18 05:11:50 2025 -0600
Merge branch 'io_uring-6.16' into for-next
* io_uring-6.16:
io_uring: fix potential page leak in io_sqe_buffer_register()
io_uring/sqpoll: don't put task_struct on tctx setup failure
io_uring: remove duplicate io_uring_alloc_task_context() definition
dmesg log:
[ 7016.058777] running generic/002
[ 7018.902645] I/O error, dev ublkb0, sector 164120 op 0x1:(WRITE)
flags 0x8800 phys_seg 1 prio class 0
[ 7018.911876] I/O error, dev ublkb0, sector 164072 op 0x0:(READ)
flags 0x0 phys_seg 2 prio class 0
[ 7018.920776] I/O error, dev ublkb0, sector 164128 op 0x1:(WRITE)
flags 0x8800 phys_seg 3 prio class 0
[ 7018.930012] I/O error, dev ublkb0, sector 164088 op 0x0:(READ)
flags 0x0 phys_seg 2 prio class 0
[ 7018.938885] I/O error, dev ublkb0, sector 164152 op 0x1:(WRITE)
flags 0x8800 phys_seg 1 prio class 0
[ 7018.948100] I/O error, dev ublkb0, sector 164104 op 0x0:(READ)
flags 0x0 phys_seg 3 prio class 0
[ 7018.956985] I/O error, dev ublkb0, sector 152112 op 0x0:(READ)
flags 0x0 phys_seg 3 prio class 0
[ 7018.965884] I/O error, dev ublkb0, sector 153040 op 0x1:(WRITE)
flags 0x8800 phys_seg 1 prio class 0
[ 7018.966317] I/O error, dev ublkb0, sector 164160 op 0x1:(WRITE)
flags 0x8800 phys_seg 2 prio class 0
[ 7018.966398] I/O error, dev ublkb0, sector 153056 op 0x1:(WRITE)
flags 0x8800 phys_seg 2 prio class 0
[ 7019.030649] Buffer I/O error on dev ublkb0, logical block 0, async page read
[ 7019.037860] Buffer I/O error on dev ublkb0, logical block 0, async page read
[ 7019.045042] Buffer I/O error on dev ublkb0, logical block 0, async page read
[ 7019.052207] Buffer I/O error on dev ublkb0, logical block 0, async page read
[ 7019.059367] Buffer I/O error on dev ublkb0, logical block 0, async page read
[ 7019.066568] Buffer I/O error on dev ublkb0, logical block 0, async page read
[ 7023.086712] Buffer I/O error on dev ublkb0, logical block 0, async page read
[ 7023.093983] Buffer I/O error on dev ublkb0, logical block 0, async page read
[ 7023.101162] Buffer I/O error on dev ublkb0, logical block 0, async page read
[ 7023.108331] Buffer I/O error on dev ublkb0, logical block 0, async page read
[ 7027.271895] blk_print_req_error: 524 callbacks suppressed
[ 7027.271948] I/O error, dev ublkb0, sector 78016 op 0x0:(READ) flags
0x0 phys_seg 1 prio class 0
[ 7027.272241] I/O error, dev ublkb0, sector 174712 op 0x1:(WRITE)
flags 0x8800 phys_seg 5 prio class 0
[ 7027.277500] I/O error, dev ublkb0, sector 78024 op 0x1:(WRITE)
flags 0x8800 phys_seg 2 prio class 0
[ 7027.286245] I/O error, dev ublkb0, sector 174472 op 0x0:(READ)
flags 0x0 phys_seg 1 prio class 0
[ 7027.295391] I/O error, dev ublkb0, sector 78008 op 0x0:(READ) flags
0x0 phys_seg 1 prio class 0
[ 7027.304482] I/O error, dev ublkb0, sector 174752 op 0x1:(WRITE)
flags 0x8800 phys_seg 3 prio class 0
[ 7027.313313] I/O error, dev ublkb0, sector 70808 op 0x0:(READ) flags
0x0 phys_seg 1 prio class 0
[ 7027.322074] I/O error, dev ublkb0, sector 174480 op 0x0:(READ)
flags 0x0 phys_seg 1 prio class 0
[ 7027.331237] I/O error, dev ublkb0, sector 70888 op 0x1:(WRITE)
flags 0x8800 phys_seg 2 prio class 0
[ 7027.339972] I/O error, dev ublkb0, sector 174776 op 0x1:(WRITE)
flags 0x8800 phys_seg 2 prio class 0
[ 7027.344409] buffer_io_error: 2 callbacks suppressed
[ 7027.344423] Buffer I/O error on dev ublkb0, logical block 0, async page read
[ 7027.379136] Buffer I/O error on dev ublkb0, logical block 0, async page read
[ 7027.386297] Buffer I/O error on dev ublkb0, logical block 0, async page read
[ 7027.393505] Buffer I/O error on dev ublkb0, logical block 0, async page read
[ 7027.400652] Buffer I/O error on dev ublkb0, logical block 0, async page read
[ 7027.407828] Buffer I/O error on dev ublkb0, logical block 0, async page read
[ 7027.441199] Buffer I/O error on dev ublkb0, logical block 65535984,
async page read
[ 7032.010124] restraintd[1486]: *** Current Time: Fri Jun 20 13:33:51
2025 Localwatchdog at: Fri Jun 20 17:14:51 2025
[ 7031.507630] Buffer I/O error on dev ublkb0, logical block 0, async page read
[ 7031.514883] Buffer I/O error on dev ublkb0, logical block 0, async page read
[ 7031.522069] Buffer I/O error on dev ublkb0, logical block 0, async page read
[ 7035.677852] blk_print_req_error: 506 callbacks suppressed
[ 7035.677867] I/O error, dev ublkb0, sector 113224 op 0x0:(READ)
flags 0x0 phys_seg 1 prio class 0
[ 7035.678946] I/O error, dev ublkb0, sector 205808 op 0x1:(WRITE)
flags 0x8800 phys_seg 3 prio class 0
[ 7035.683413] I/O error, dev ublkb0, sector 113168 op 0x0:(READ)
flags 0x0 phys_seg 2 prio class 0
[ 7035.692216] I/O error, dev ublkb0, sector 207248 op 0x0:(READ)
flags 0x0 phys_seg 4 prio class 0
[ 7035.719077] I/O error, dev ublkb0, sector 205832 op 0x1:(WRITE)
flags 0x8800 phys_seg 1 prio class 0
[ 7035.728292] I/O error, dev ublkb0, sector 207280 op 0x0:(READ)
flags 0x0 phys_seg 1 prio class 0
[ 7035.737160] I/O error, dev ublkb0, sector 205840 op 0x1:(WRITE)
flags 0x8800 phys_seg 1 prio class 0
[ 7035.746416] I/O error, dev ublkb0, sector 207288 op 0x0:(READ)
flags 0x0 phys_seg 3 prio class 0
[ 7035.755296] I/O error, dev ublkb0, sector 205848 op 0x1:(WRITE)
flags 0x8800 phys_seg 2 prio class 0
[ 7035.764516] I/O error, dev ublkb0, sector 207312 op 0x0:(READ)
flags 0x0 phys_seg 1 prio class 0
[ 7035.778303] buffer_io_error: 3 callbacks suppressed
[ 7035.778317] Buffer I/O error on dev ublkb0, logical block 0, async page read
[ 7035.790565] Buffer I/O error on dev ublkb0, logical block 0, async page read
[ 7035.797745] Buffer I/O error on dev ublkb0, logical block 0, async page read
[ 7035.804930] Buffer I/O error on dev ublkb0, logical block 0, async page read
[ 7035.812112] Buffer I/O error on dev ublkb0, logical block 0, async page read
[ 7035.819299] Buffer I/O error on dev ublkb0, logical block 0, async page read
[ 7035.862857] Buffer I/O error on dev ublkb0, logical block 65535984,
async page read
[ 7039.928599] Buffer I/O error on dev ublkb0, logical block 0, async page read
[ 7039.935813] Buffer I/O error on dev ublkb0, logical block 0, async page read
[ 7039.942990] Buffer I/O error on dev ublkb0, logical block 0, async page read
[ 7044.064528] BUG: kernel NULL pointer dereference, address: 0000000000000001
[ 7044.071507] #PF: supervisor read access in kernel mode
[ 7044.076653] #PF: error_code(0x0000) - not-present page
[ 7044.081801] PGD 462c42067 P4D 462c42067 PUD 462c43067 PMD 0
[ 7044.087488] Oops: Oops: 0000 [#1] SMP NOPTI
[ 7044.091685] CPU: 13 UID: 0 PID: 367 Comm: kworker/13:1H Not tainted
6.16.0-rc2+ #1 PREEMPT(voluntary)
[ 7044.100991] Hardware name: Dell Inc. PowerEdge R640/0X45NX, BIOS
2.22.2 09/12/2024
[ 7044.108565] Workqueue: kblockd blk_mq_requeue_work
[ 7044.113374] RIP: 0010:__io_req_task_work_add+0x18/0x1f0
[ 7044.118608] Code: 40 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90
90 90 66 0f 1f 00 e8 27 5a ac 0d 41 56 41 55 41 54 55 53 48 8b 6f 60
48 89 fb <f6> 45 01 20 0f 84 8e 00 00 00 31 c0 f6 47 48 0c 0f 94 c0 21
c6 41
[ 7044.137362] RSP: 0018:ffffcf6ec3d63c50 EFLAGS: 00010292
[ 7044.142598] RAX: ffffffffc136e3b0 RBX: ffff8ecf44fb3e80 RCX: 0000000000000000
[ 7044.149740] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff8ecf44fb3e80
[ 7044.156882] RBP: 0000000000000000 R08: 00000000248436de R09: ffff8ecf4409a100
[ 7044.164021] R10: 000000000000003c R11: ffff8ed2812f1180 R12: ffff8ecf44fb3e80
[ 7044.171163] R13: ffffcf6ec3d63cc8 R14: 0000000000000000 R15: ffff8ece97859310
[ 7044.178304] FS: 0000000000000000(0000) GS:ffff8ed379e45000(0000)
knlGS:0000000000000000
[ 7044.186399] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 7044.192155] CR2: 0000000000000001 CR3: 000000036eb7c004 CR4: 00000000007726f0
[ 7044.199295] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 7044.206437] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 7044.213578] PKRU: 55555554
[ 7044.216299] Call Trace:
[ 7044.218762] <TASK>
[ 7044.220881] ? ftrace_stub_direct_tramp+0x10/0x10
[ 7044.225599] ublk_queue_rq+0x50/0x90 [ublk_drv]
[ 7044.230155] ? ftrace_stub_direct_tramp+0x10/0x10
[ 7044.234873] blk_mq_dispatch_rq_list+0x13c/0x510
[ 7044.239520] ? ftrace_stub_direct_tramp+0x10/0x10
[ 7044.244230] __blk_mq_sched_dispatch_requests+0x118/0x1a0
[ 7044.249652] ? ftrace_stub_direct_tramp+0x10/0x10
[ 7044.254363] blk_mq_sched_dispatch_requests+0x2d/0x70
[ 7044.259426] ? ftrace_stub_direct_tramp+0x10/0x10
[ 7044.264140] blk_mq_run_hw_queue+0x26a/0x2e0
[ 7044.268430] ? ftrace_stub_direct_tramp+0x10/0x10
[ 7044.273144] blk_mq_run_hw_queues+0x7f/0x140
[ 7044.277436] ? ftrace_stub_direct_tramp+0x10/0x10
[ 7044.282150] blk_mq_requeue_work+0x19f/0x1e0
[ 7044.286445] ? ftrace_stub_direct_tramp+0x10/0x10
[ 7044.291160] process_one_work+0x188/0x340
[ 7044.295194] ? ftrace_stub_direct_tramp+0x10/0x10
[ 7044.299906] worker_thread+0x257/0x3a0
[ 7044.303677] ? __pfx_worker_thread+0x10/0x10
[ 7044.307959] kthread+0xfc/0x240
[ 7044.311115] ? __pfx_kthread+0x10/0x10
[ 7044.314875] ? __pfx_kthread+0x10/0x10
[ 7044.318643] ret_from_fork+0xed/0x110
[ 7044.322319] ? __pfx_kthread+0x10/0x10
[ 7044.326083] ret_from_fork_asm+0x1a/0x30
[ 7044.330046] </TASK>
[ 7044.332243] Modules linked in: ublk_drv raid1 ext4 crc16 mbcache
jbd2 rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace
nfs_localio netfs rfkill intel_rapl_msr intel_rapl_common sunrpc
intel_uncore_frequency intel_uncore_frequency_common skx_edac
skx_edac_common nfit libnvdimm x86_pkg_temp_thermal intel_powerclamp
coretemp kvm_intel vfat fat kvm iTCO_wdt iTCO_vendor_support irqbypass
rapl intel_cstate dell_smbios ipmi_ssif platform_profile bnxt_en
mgag200 intel_uncore dcdbas tg3 dell_wmi_descriptor wmi_bmof mei_me
i2c_i801 pcspkr i2c_algo_bit mei acpi_power_meter i2c_smbus lpc_ich
intel_pch_thermal ipmi_si acpi_ipmi ipmi_devintf ipmi_msghandler sg
fuse loop dm_multipath nfnetlink xfs sd_mod nvme ahci nvme_core
libahci ghash_clmulni_intel libata nvme_keyring megaraid_sas nvme_auth
wmi dm_mirror dm_region_hash dm_log dm_mod [last unloaded: null_blk]
[ 7044.408751] CR2: 0000000000000001
[ 7044.412082] ---[ end trace 0000000000000000 ]---
[ 7044.427300] RIP: 0010:__io_req_task_work_add+0x18/0x1f0
[ 7044.432549] Code: 40 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90
90 90 66 0f 1f 00 e8 27 5a ac 0d 41 56 41 55 41 54 55 53 48 8b 6f 60
48 89 fb <f6> 45 01 20 0f 84 8e 00 00 00 31 c0 f6 47 48 0c 0f 94 c0 21
c6 41
[ 7044.451303] RSP: 0018:ffffcf6ec3d63c50 EFLAGS: 00010292
[ 7044.456539] RAX: ffffffffc136e3b0 RBX: ffff8ecf44fb3e80 RCX: 0000000000000000
[ 7044.463681] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff8ecf44fb3e80
[ 7044.470822] RBP: 0000000000000000 R08: 00000000248436de R09: ffff8ecf4409a100
[ 7044.477964] R10: 000000000000003c R11: ffff8ed2812f1180 R12: ffff8ecf44fb3e80
[ 7044.485102] R13: ffffcf6ec3d63cc8 R14: 0000000000000000 R15: ffff8ece97859310
[ 7044.492246] FS: 0000000000000000(0000) GS:ffff8ed379e45000(0000)
knlGS:0000000000000000
[ 7044.500339] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 7044.506095] CR2: 0000000000000001 CR3: 000000036eb7c004 CR4: 00000000007726f0
[ 7044.513235] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 7044.520377] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 7044.527518] PKRU: 55555554
[ 7044.530241] Kernel panic - not syncing: Fatal exception
[ 7044.535536] Kernel Offset: 0x32200000 from 0xffffffff81000000
(relocation range: 0xffffffff80000000-0xffffffffbfffffff)
[ 7044.557117] ---[ end Kernel panic - not syncing: Fatal exception ]---
Best Regards,
Changhui
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [bug report] BUG: kernel NULL pointer dereference, address: 0000000000000001
2025-06-23 2:58 [bug report] BUG: kernel NULL pointer dereference, address: 0000000000000001 Changhui Zhong
@ 2025-06-23 4:02 ` Ming Lei
[not found] ` <CAGVVp+UEQ2XWrNpAz4-+SuyoHybrQ3-Uv5hE-SLQAdSpVw-kgQ@mail.gmail.com>
2025-06-23 9:12 ` Changhui Zhong
0 siblings, 2 replies; 6+ messages in thread
From: Ming Lei @ 2025-06-23 4:02 UTC (permalink / raw)
To: Changhui Zhong; +Cc: Linux Block Devices
Hi Changhui,
On Mon, Jun 23, 2025 at 10:58:24AM +0800, Changhui Zhong wrote:
> Hello,
>
> the following kernel panic was triggered by ubdsrv generic/002,
> please help check and let me know if you need any info/test, thanks.
>
> commit HEAD:
>
> commit 2589cd05008205ee29f5f66f24a684732ee2e3a3
> Merge: 98d0347fe8fb e1c75831f682
> Author: Jens Axboe <axboe@kernel.dk>
> Date: Wed Jun 18 05:11:50 2025 -0600
>
> Merge branch 'io_uring-6.16' into for-next
>
> * io_uring-6.16:
> io_uring: fix potential page leak in io_sqe_buffer_register()
> io_uring/sqpoll: don't put task_struct on tctx setup failure
> io_uring: remove duplicate io_uring_alloc_task_context() definition
The above branch has been merged to v6.16-rc3, can you reproduce it with -rc3?
I tried to duplicate in my test VM, not succeed with -rc3.
...
> [ 7044.064528] BUG: kernel NULL pointer dereference, address: 0000000000000001
> [ 7044.071507] #PF: supervisor read access in kernel mode
> [ 7044.076653] #PF: error_code(0x0000) - not-present page
> [ 7044.081801] PGD 462c42067 P4D 462c42067 PUD 462c43067 PMD 0
> [ 7044.087488] Oops: Oops: 0000 [#1] SMP NOPTI
> [ 7044.091685] CPU: 13 UID: 0 PID: 367 Comm: kworker/13:1H Not tainted
> 6.16.0-rc2+ #1 PREEMPT(voluntary)
> [ 7044.100991] Hardware name: Dell Inc. PowerEdge R640/0X45NX, BIOS
> 2.22.2 09/12/2024
> [ 7044.108565] Workqueue: kblockd blk_mq_requeue_work
> [ 7044.113374] RIP: 0010:__io_req_task_work_add+0x18/0x1f0
Can you share where the above line points to source line if it can be
reproduced in -rc3?
gdb> l *(__io_req_task_work_add+0x18)
Thanks,
Ming
^ permalink raw reply [flat|nested] 6+ messages in thread
* Fwd: [bug report] BUG: kernel NULL pointer dereference, address: 0000000000000001
[not found] ` <CAGVVp+UEQ2XWrNpAz4-+SuyoHybrQ3-Uv5hE-SLQAdSpVw-kgQ@mail.gmail.com>
@ 2025-06-23 8:54 ` Changhui Zhong
0 siblings, 0 replies; 6+ messages in thread
From: Changhui Zhong @ 2025-06-23 8:54 UTC (permalink / raw)
To: Linux Block Devices; +Cc: Ming Lei
---------- Forwarded message ---------
From: Changhui Zhong <czhong@redhat.com>
Date: Mon, Jun 23, 2025 at 4:48 PM
Subject: Re: [bug report] BUG: kernel NULL pointer dereference,
address: 0000000000000001
To: Ming Lei <ming.lei@redhat.com>
On Mon, Jun 23, 2025 at 12:02 PM Ming Lei <ming.lei@redhat.com> wrote:
>
> Hi Changhui,
>
> On Mon, Jun 23, 2025 at 10:58:24AM +0800, Changhui Zhong wrote:
> > Hello,
> >
> > the following kernel panic was triggered by ubdsrv generic/002,
> > please help check and let me know if you need any info/test, thanks.
> >
> > commit HEAD:
> >
> > commit 2589cd05008205ee29f5f66f24a684732ee2e3a3
> > Merge: 98d0347fe8fb e1c75831f682
> > Author: Jens Axboe <axboe@kernel.dk>
> > Date: Wed Jun 18 05:11:50 2025 -0600
> >
> > Merge branch 'io_uring-6.16' into for-next
> >
> > * io_uring-6.16:
> > io_uring: fix potential page leak in io_sqe_buffer_register()
> > io_uring/sqpoll: don't put task_struct on tctx setup failure
> > io_uring: remove duplicate io_uring_alloc_task_context() definition
>
> The above branch has been merged to v6.16-rc3, can you reproduce it with -rc3?
>
> I tried to duplicate in my test VM, not succeed with -rc3.
>
Hi，Ming
I hit this issue with repo
https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git/log/?h=for-next.
it is not a 100% reproducible issue. I triggered it in the run
T=generic as loop, ‘for i in {0..10};do make test T=generic; done’
and later I tried it with repo https://github.com/torvalds/linux,
branch v6.16-rc3, but have not been able to reproduce it so far.
> ...
>
> > [ 7044.064528] BUG: kernel NULL pointer dereference, address: 0000000000000001
> > [ 7044.071507] #PF: supervisor read access in kernel mode
> > [ 7044.076653] #PF: error_code(0x0000) - not-present page
> > [ 7044.081801] PGD 462c42067 P4D 462c42067 PUD 462c43067 PMD 0
> > [ 7044.087488] Oops: Oops: 0000 [#1] SMP NOPTI
> > [ 7044.091685] CPU: 13 UID: 0 PID: 367 Comm: kworker/13:1H Not tainted
> > 6.16.0-rc2+ #1 PREEMPT(voluntary)
> > [ 7044.100991] Hardware name: Dell Inc. PowerEdge R640/0X45NX, BIOS
> > 2.22.2 09/12/2024
> > [ 7044.108565] Workqueue: kblockd blk_mq_requeue_work
> > [ 7044.113374] RIP: 0010:__io_req_task_work_add+0x18/0x1f0
>
> Can you share where the above line points to source line if it can be
> reproduced in -rc3?
>
> gdb> l *(__io_req_task_work_add+0x18)
>
>
vmlinux is compiled by repo
https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git
，branch for-next，
(gdb) l *(__io_req_task_work_add+0x18)
0xffffffff819075e8 is in __io_req_task_work_add (io_uring/io_uring.c:1251).
1246 io_fallback_tw(tctx, false);
1247 }
1248
1249 void __io_req_task_work_add(struct io_kiocb *req, unsigned flags)
1250 {
1251 if (req->ctx->flags & IORING_SETUP_DEFER_TASKRUN)
1252 io_req_local_work_add(req, flags);
1253 else
1254 io_req_normal_work_add(req);
1255 }
(gdb)
> Thanks,
> Ming
>
Thanks，
Changhui
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [bug report] BUG: kernel NULL pointer dereference, address: 0000000000000001
2025-06-23 4:02 ` Ming Lei
[not found] ` <CAGVVp+UEQ2XWrNpAz4-+SuyoHybrQ3-Uv5hE-SLQAdSpVw-kgQ@mail.gmail.com>
@ 2025-06-23 9:12 ` Changhui Zhong
2025-06-23 20:33 ` Caleb Sander Mateos
1 sibling, 1 reply; 6+ messages in thread
From: Changhui Zhong @ 2025-06-23 9:12 UTC (permalink / raw)
To: Ming Lei; +Cc: Linux Block Devices
On Mon, Jun 23, 2025 at 12:02 PM Ming Lei <ming.lei@redhat.com> wrote:
>
> Hi Changhui,
>
> On Mon, Jun 23, 2025 at 10:58:24AM +0800, Changhui Zhong wrote:
> > Hello,
> >
> > the following kernel panic was triggered by ubdsrv generic/002,
> > please help check and let me know if you need any info/test, thanks.
> >
> > commit HEAD:
> >
> > commit 2589cd05008205ee29f5f66f24a684732ee2e3a3
> > Merge: 98d0347fe8fb e1c75831f682
> > Author: Jens Axboe <axboe@kernel.dk>
> > Date: Wed Jun 18 05:11:50 2025 -0600
> >
> > Merge branch 'io_uring-6.16' into for-next
> >
> > * io_uring-6.16:
> > io_uring: fix potential page leak in io_sqe_buffer_register()
> > io_uring/sqpoll: don't put task_struct on tctx setup failure
> > io_uring: remove duplicate io_uring_alloc_task_context() definition
>
> The above branch has been merged to v6.16-rc3, can you reproduce it with -rc3?
>
> I tried to duplicate in my test VM, not succeed with -rc3.
>
> ...
>
> > [ 7044.064528] BUG: kernel NULL pointer dereference, address: 0000000000000001
> > [ 7044.071507] #PF: supervisor read access in kernel mode
> > [ 7044.076653] #PF: error_code(0x0000) - not-present page
> > [ 7044.081801] PGD 462c42067 P4D 462c42067 PUD 462c43067 PMD 0
> > [ 7044.087488] Oops: Oops: 0000 [#1] SMP NOPTI
> > [ 7044.091685] CPU: 13 UID: 0 PID: 367 Comm: kworker/13:1H Not tainted
> > 6.16.0-rc2+ #1 PREEMPT(voluntary)
> > [ 7044.100991] Hardware name: Dell Inc. PowerEdge R640/0X45NX, BIOS
> > 2.22.2 09/12/2024
> > [ 7044.108565] Workqueue: kblockd blk_mq_requeue_work
> > [ 7044.113374] RIP: 0010:__io_req_task_work_add+0x18/0x1f0
>
> Can you share where the above line points to source line if it can be
> reproduced in -rc3?
>
> gdb> l *(__io_req_task_work_add+0x18)
>
>
> Thanks,
> Ming
>
now successfully reproduced on v6.16-rc3, more loop tests are needed
to trigger this issue，
[ 8898.102836] BUG: kernel NULL pointer dereference, address: 0000000000000001
[ 8898.109848] #PF: supervisor read access in kernel mode
[ 8898.115011] #PF: error_code(0x0000) - not-present page
[ 8898.120161] PGD 80000001bcd7b067 P4D 80000001bcd7b067 PUD 1ee49f067 PMD 0
[ 8898.127043] Oops: Oops: 0000 [#1] SMP PTI
[ 8898.131065] CPU: 2 UID: 0 PID: 47056 Comm: kworker/2:2H Not tainted
6.16.0-rc3 #1 PREEMPT(voluntary)
[ 8898.140283] Hardware name: Dell Inc. PowerEdge R340/045M96, BIOS
2.17.3 09/12/2024
[ 8898.147860] Workqueue: kblockd blk_mq_requeue_work
[ 8898.152658] RIP: 0010:__io_req_task_work_add+0x18/0x1f0
[ 8898.157895] Code: 40 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90
90 90 66 0f 1f 00 0f 1f 44 00 00 41 56 41 55 41 54 55 53 48 8b 6f 60
48 89 fb <f6> 45 01 20 0f 84 8e 00 00 00 31 c0 f6 47 48 0c 0f 94 c0 21
c6 41
[ 8898.176650] RSP: 0018:ffffd28e08d03c50 EFLAGS: 00010206
[ 8898.181882] RAX: ffffffffc0dc73d0 RBX: ffff8d64218c35c0 RCX: ffff8d676ee1e828
[ 8898.189025] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff8d64218c35c0
[ 8898.196165] RBP: 0000000000000000 R08: 0000000000010000 R09: ffff8d6402d42600
[ 8898.203308] R10: ffff8d6400c1d8c0 R11: fefefefefefefeff R12: ffff8d64218c35c0
[ 8898.210448] R13: ffffd28e08d03cc8 R14: 0000000000000000 R15: ffff8d6420901310
[ 8898.217592] FS: 0000000000000000(0000) GS:ffff8d67cd7c5000(0000)
knlGS:0000000000000000
[ 8898.225685] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8898.231441] CR2: 0000000000000001 CR3: 00000001951b8003 CR4: 00000000003726f0
[ 8898.238581] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 8898.245720] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 8898.252876] Call Trace:
[ 8898.255335] <TASK>
[ 8898.257450] ublk_queue_rq+0x50/0x90 [ublk_drv]
[ 8898.261989] blk_mq_dispatch_rq_list+0x13c/0x510
[ 8898.266620] __blk_mq_sched_dispatch_requests+0x118/0x1a0
[ 8898.272027] ? xa_find_after+0xfc/0x190
[ 8898.275876] blk_mq_sched_dispatch_requests+0x2d/0x70
[ 8898.280937] blk_mq_run_hw_queue+0x26a/0x2e0
[ 8898.285216] blk_mq_run_hw_queues+0x7f/0x140
[ 8898.289498] blk_mq_requeue_work+0x19f/0x1e0
[ 8898.293782] process_one_work+0x188/0x340
[ 8898.297820] worker_thread+0x257/0x3a0
[ 8898.301578] ? __pfx_worker_thread+0x10/0x10
[ 8898.305871] kthread+0xf9/0x240
[ 8898.309022] ? __pfx_kthread+0x10/0x10
[ 8898.312785] ? __pfx_kthread+0x10/0x10
[ 8898.316549] ret_from_fork+0xed/0x110
[ 8898.320220] ? __pfx_kthread+0x10/0x10
[ 8898.323981] ret_from_fork_asm+0x1a/0x30
[ 8898.327919] </TASK>
[ 8898.330118] Modules linked in: ublk_drv rpcsec_gss_krb5 auth_rpcgss
nfsv4 dns_resolver nfs lockd grace nfs_localio netfs sunrpc ipmi_ssif
intel_rapl_msr intel_rapl_common intel_uncore_frequency
intel_uncore_frequency_common intel_pmc_core_pltdrv intel_pmc_core
pmt_telemetry pmt_class intel_pmc_ssram_telemetry intel_vsec
intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp
kvm_intel kvm platform_profile dell_wmi dell_smbios iTCO_wdt irqbypass
dell_wmi_descriptor iTCO_vendor_support rapl sparse_keymap rfkill
intel_cstate mgag200 tg3 mei_me dcdbas intel_uncore i2c_algo_bit
pcspkr mei i2c_i801 idma64 i2c_smbus ie31200_edac acpi_power_meter
intel_pch_thermal ipmi_si acpi_ipmi ipmi_devintf ipmi_msghandler sg
fuse loop dm_multipath nfnetlink xfs sd_mod ahci libahci megaraid_sas
libata ghash_clmulni_intel video pinctrl_cannonlake wmi dm_mirror
dm_region_hash dm_log dm_mod [last unloaded: ublk_drv]
[ 8898.409843] CR2: 0000000000000001
[ 8898.413172] ---[ end trace 0000000000000000 ]---
[ 8898.510831] pstore: backend (erst) writing error (-19)
[ 8898.515985] RIP: 0010:__io_req_task_work_add+0x18/0x1f0
[ 8898.521221] Code: 40 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90
90 90 66 0f 1f 00 0f 1f 44 00 00 41 56 41 55 41 54 55 53 48 8b 6f 60
48 89 fb <f6> 45 01 20 0f 84 8e 00 00 00 31 c0 f6 47 48 0c 0f 94 c0 21
c6 41
[ 8898.539975] RSP: 0018:ffffd28e08d03c50 EFLAGS: 00010206
[ 8898.545208] RAX: ffffffffc0dc73d0 RBX: ffff8d64218c35c0 RCX: ffff8d676ee1e828
[ 8898.552348] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff8d64218c35c0
[ 8898.559492] RBP: 0000000000000000 R08: 0000000000010000 R09: ffff8d6402d42600
[ 8898.566631] R10: ffff8d6400c1d8c0 R11: fefefefefefefeff R12: ffff8d64218c35c0
[ 8898.573775] R13: ffffd28e08d03cc8 R14: 0000000000000000 R15: ffff8d6420901310
[ 8898.580913] FS: 0000000000000000(0000) GS:ffff8d67cd7c5000(0000)
knlGS:0000000000000000
[ 8898.589011] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 8898.594763] CR2: 0000000000000001 CR3: 00000001951b8003 CR4: 00000000003726f0
[ 8898.601906] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 8898.609047] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 8898.616191] Kernel panic - not syncing: Fatal exception
[ 8898.621466] Kernel Offset: 0x1dc00000 from 0xffffffff81000000
(relocation range: 0xffffffff80000000-0xffffffffbfffffff)
[ 8898.646077] ---[ end Kernel panic - not syncing: Fatal exception ]---
(gdb) l *(__io_req_task_work_add+0x18)
0xffffffff81907668 is in __io_req_task_work_add (io_uring/io_uring.c:1251).
1246 io_fallback_tw(tctx, false);
1247 }
1248
1249 void __io_req_task_work_add(struct io_kiocb *req, unsigned flags)
1250 {
1251 if (req->ctx->flags & IORING_SETUP_DEFER_TASKRUN)
1252 io_req_local_work_add(req, flags);
1253 else
1254 io_req_normal_work_add(req);
1255 }
(gdb)
Thanks,
Changhui
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [bug report] BUG: kernel NULL pointer dereference, address: 0000000000000001
2025-06-23 9:12 ` Changhui Zhong
@ 2025-06-23 20:33 ` Caleb Sander Mateos
2025-06-24 1:26 ` Ming Lei
0 siblings, 1 reply; 6+ messages in thread
From: Caleb Sander Mateos @ 2025-06-23 20:33 UTC (permalink / raw)
To: Changhui Zhong; +Cc: Ming Lei, Linux Block Devices
On Mon, Jun 23, 2025 at 2:13 AM Changhui Zhong <czhong@redhat.com> wrote:
>
> On Mon, Jun 23, 2025 at 12:02 PM Ming Lei <ming.lei@redhat.com> wrote:
> >
> > Hi Changhui,
> >
> > On Mon, Jun 23, 2025 at 10:58:24AM +0800, Changhui Zhong wrote:
> > > Hello,
> > >
> > > the following kernel panic was triggered by ubdsrv generic/002,
> > > please help check and let me know if you need any info/test, thanks.
> > >
> > > commit HEAD:
> > >
> > > commit 2589cd05008205ee29f5f66f24a684732ee2e3a3
> > > Merge: 98d0347fe8fb e1c75831f682
> > > Author: Jens Axboe <axboe@kernel.dk>
> > > Date: Wed Jun 18 05:11:50 2025 -0600
> > >
> > > Merge branch 'io_uring-6.16' into for-next
> > >
> > > * io_uring-6.16:
> > > io_uring: fix potential page leak in io_sqe_buffer_register()
> > > io_uring/sqpoll: don't put task_struct on tctx setup failure
> > > io_uring: remove duplicate io_uring_alloc_task_context() definition
> >
> > The above branch has been merged to v6.16-rc3, can you reproduce it with -rc3?
> >
> > I tried to duplicate in my test VM, not succeed with -rc3.
> >
> > ...
> >
> > > [ 7044.064528] BUG: kernel NULL pointer dereference, address: 0000000000000001
> > > [ 7044.071507] #PF: supervisor read access in kernel mode
> > > [ 7044.076653] #PF: error_code(0x0000) - not-present page
> > > [ 7044.081801] PGD 462c42067 P4D 462c42067 PUD 462c43067 PMD 0
> > > [ 7044.087488] Oops: Oops: 0000 [#1] SMP NOPTI
> > > [ 7044.091685] CPU: 13 UID: 0 PID: 367 Comm: kworker/13:1H Not tainted
> > > 6.16.0-rc2+ #1 PREEMPT(voluntary)
> > > [ 7044.100991] Hardware name: Dell Inc. PowerEdge R640/0X45NX, BIOS
> > > 2.22.2 09/12/2024
> > > [ 7044.108565] Workqueue: kblockd blk_mq_requeue_work
> > > [ 7044.113374] RIP: 0010:__io_req_task_work_add+0x18/0x1f0
> >
> > Can you share where the above line points to source line if it can be
> > reproduced in -rc3?
> >
> > gdb> l *(__io_req_task_work_add+0x18)
> >
> >
> > Thanks,
> > Ming
> >
>
> now successfully reproduced on v6.16-rc3, more loop tests are needed
> to trigger this issue，
>
> [ 8898.102836] BUG: kernel NULL pointer dereference, address: 0000000000000001
> [ 8898.109848] #PF: supervisor read access in kernel mode
> [ 8898.115011] #PF: error_code(0x0000) - not-present page
> [ 8898.120161] PGD 80000001bcd7b067 P4D 80000001bcd7b067 PUD 1ee49f067 PMD 0
> [ 8898.127043] Oops: Oops: 0000 [#1] SMP PTI
> [ 8898.131065] CPU: 2 UID: 0 PID: 47056 Comm: kworker/2:2H Not tainted
> 6.16.0-rc3 #1 PREEMPT(voluntary)
> [ 8898.140283] Hardware name: Dell Inc. PowerEdge R340/045M96, BIOS
> 2.17.3 09/12/2024
> [ 8898.147860] Workqueue: kblockd blk_mq_requeue_work
> [ 8898.152658] RIP: 0010:__io_req_task_work_add+0x18/0x1f0
> [ 8898.157895] Code: 40 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90
> 90 90 66 0f 1f 00 0f 1f 44 00 00 41 56 41 55 41 54 55 53 48 8b 6f 60
> 48 89 fb <f6> 45 01 20 0f 84 8e 00 00 00 31 c0 f6 47 48 0c 0f 94 c0 21
> c6 41
Disassembling this:
0: 41 56 push r14
2: 41 55 push r13
4: 41 54 push r12
6: 55 push rbp
7: 53 push rbx
8: 48 8b 6f 60 mov rbp,QWORD PTR [rdi+0x60]
c: 48 89 fb mov rbx,rdi
f: f6 45 01 20 test BYTE PTR [rbp+0x1],0x20 <--here
13: 0f 84 8e 00 00 00 je 0xa7
19: 31 c0 xor eax,eax
1b: f6 47 48 0c test BYTE PTR [rdi+0x48],0xc
1f: 0f 94 c0 sete al
22: 21 c6 and esi,eax
So we look to be at the start of __io_req_task_work_add(). rdi stores
req, rbp stores req->ctx, and so the test instruction that's faulting
is loading (the second byte of) req->ctx->flags for the
req->ctx->flags & IORING_SETUP_DEFER_TASKRUN check. This means
req->ctx is NULL. Is it possible the req has already been completed or
cancelled? The stacktrace shows that this is coming from
blk_mq_requeue_work, which is definitely interesting.
Best,
Caleb
> [ 8898.176650] RSP: 0018:ffffd28e08d03c50 EFLAGS: 00010206
> [ 8898.181882] RAX: ffffffffc0dc73d0 RBX: ffff8d64218c35c0 RCX: ffff8d676ee1e828
> [ 8898.189025] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff8d64218c35c0
> [ 8898.196165] RBP: 0000000000000000 R08: 0000000000010000 R09: ffff8d6402d42600
> [ 8898.203308] R10: ffff8d6400c1d8c0 R11: fefefefefefefeff R12: ffff8d64218c35c0
> [ 8898.210448] R13: ffffd28e08d03cc8 R14: 0000000000000000 R15: ffff8d6420901310
> [ 8898.217592] FS: 0000000000000000(0000) GS:ffff8d67cd7c5000(0000)
> knlGS:0000000000000000
> [ 8898.225685] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 8898.231441] CR2: 0000000000000001 CR3: 00000001951b8003 CR4: 00000000003726f0
> [ 8898.238581] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 8898.245720] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> [ 8898.252876] Call Trace:
> [ 8898.255335] <TASK>
> [ 8898.257450] ublk_queue_rq+0x50/0x90 [ublk_drv]
> [ 8898.261989] blk_mq_dispatch_rq_list+0x13c/0x510
> [ 8898.266620] __blk_mq_sched_dispatch_requests+0x118/0x1a0
> [ 8898.272027] ? xa_find_after+0xfc/0x190
> [ 8898.275876] blk_mq_sched_dispatch_requests+0x2d/0x70
> [ 8898.280937] blk_mq_run_hw_queue+0x26a/0x2e0
> [ 8898.285216] blk_mq_run_hw_queues+0x7f/0x140
> [ 8898.289498] blk_mq_requeue_work+0x19f/0x1e0
> [ 8898.293782] process_one_work+0x188/0x340
> [ 8898.297820] worker_thread+0x257/0x3a0
> [ 8898.301578] ? __pfx_worker_thread+0x10/0x10
> [ 8898.305871] kthread+0xf9/0x240
> [ 8898.309022] ? __pfx_kthread+0x10/0x10
> [ 8898.312785] ? __pfx_kthread+0x10/0x10
> [ 8898.316549] ret_from_fork+0xed/0x110
> [ 8898.320220] ? __pfx_kthread+0x10/0x10
> [ 8898.323981] ret_from_fork_asm+0x1a/0x30
> [ 8898.327919] </TASK>
> [ 8898.330118] Modules linked in: ublk_drv rpcsec_gss_krb5 auth_rpcgss
> nfsv4 dns_resolver nfs lockd grace nfs_localio netfs sunrpc ipmi_ssif
> intel_rapl_msr intel_rapl_common intel_uncore_frequency
> intel_uncore_frequency_common intel_pmc_core_pltdrv intel_pmc_core
> pmt_telemetry pmt_class intel_pmc_ssram_telemetry intel_vsec
> intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp
> kvm_intel kvm platform_profile dell_wmi dell_smbios iTCO_wdt irqbypass
> dell_wmi_descriptor iTCO_vendor_support rapl sparse_keymap rfkill
> intel_cstate mgag200 tg3 mei_me dcdbas intel_uncore i2c_algo_bit
> pcspkr mei i2c_i801 idma64 i2c_smbus ie31200_edac acpi_power_meter
> intel_pch_thermal ipmi_si acpi_ipmi ipmi_devintf ipmi_msghandler sg
> fuse loop dm_multipath nfnetlink xfs sd_mod ahci libahci megaraid_sas
> libata ghash_clmulni_intel video pinctrl_cannonlake wmi dm_mirror
> dm_region_hash dm_log dm_mod [last unloaded: ublk_drv]
> [ 8898.409843] CR2: 0000000000000001
> [ 8898.413172] ---[ end trace 0000000000000000 ]---
> [ 8898.510831] pstore: backend (erst) writing error (-19)
> [ 8898.515985] RIP: 0010:__io_req_task_work_add+0x18/0x1f0
> [ 8898.521221] Code: 40 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90
> 90 90 66 0f 1f 00 0f 1f 44 00 00 41 56 41 55 41 54 55 53 48 8b 6f 60
> 48 89 fb <f6> 45 01 20 0f 84 8e 00 00 00 31 c0 f6 47 48 0c 0f 94 c0 21
> c6 41
> [ 8898.539975] RSP: 0018:ffffd28e08d03c50 EFLAGS: 00010206
> [ 8898.545208] RAX: ffffffffc0dc73d0 RBX: ffff8d64218c35c0 RCX: ffff8d676ee1e828
> [ 8898.552348] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff8d64218c35c0
> [ 8898.559492] RBP: 0000000000000000 R08: 0000000000010000 R09: ffff8d6402d42600
> [ 8898.566631] R10: ffff8d6400c1d8c0 R11: fefefefefefefeff R12: ffff8d64218c35c0
> [ 8898.573775] R13: ffffd28e08d03cc8 R14: 0000000000000000 R15: ffff8d6420901310
> [ 8898.580913] FS: 0000000000000000(0000) GS:ffff8d67cd7c5000(0000)
> knlGS:0000000000000000
> [ 8898.589011] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 8898.594763] CR2: 0000000000000001 CR3: 00000001951b8003 CR4: 00000000003726f0
> [ 8898.601906] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 8898.609047] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> [ 8898.616191] Kernel panic - not syncing: Fatal exception
> [ 8898.621466] Kernel Offset: 0x1dc00000 from 0xffffffff81000000
> (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
> [ 8898.646077] ---[ end Kernel panic - not syncing: Fatal exception ]---
>
>
> (gdb) l *(__io_req_task_work_add+0x18)
> 0xffffffff81907668 is in __io_req_task_work_add (io_uring/io_uring.c:1251).
> 1246 io_fallback_tw(tctx, false);
> 1247 }
> 1248
> 1249 void __io_req_task_work_add(struct io_kiocb *req, unsigned flags)
> 1250 {
> 1251 if (req->ctx->flags & IORING_SETUP_DEFER_TASKRUN)
> 1252 io_req_local_work_add(req, flags);
> 1253 else
> 1254 io_req_normal_work_add(req);
> 1255 }
> (gdb)
>
>
> Thanks,
> Changhui
>
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [bug report] BUG: kernel NULL pointer dereference, address: 0000000000000001
2025-06-23 20:33 ` Caleb Sander Mateos
@ 2025-06-24 1:26 ` Ming Lei
0 siblings, 0 replies; 6+ messages in thread
From: Ming Lei @ 2025-06-24 1:26 UTC (permalink / raw)
To: Caleb Sander Mateos; +Cc: Changhui Zhong, Linux Block Devices
On Mon, Jun 23, 2025 at 01:33:44PM -0700, Caleb Sander Mateos wrote:
> On Mon, Jun 23, 2025 at 2:13 AM Changhui Zhong <czhong@redhat.com> wrote:
> >
> > On Mon, Jun 23, 2025 at 12:02 PM Ming Lei <ming.lei@redhat.com> wrote:
> > >
> > > Hi Changhui,
> > >
> > > On Mon, Jun 23, 2025 at 10:58:24AM +0800, Changhui Zhong wrote:
> > > > Hello,
> > > >
> > > > the following kernel panic was triggered by ubdsrv generic/002,
> > > > please help check and let me know if you need any info/test, thanks.
> > > >
> > > > commit HEAD:
> > > >
> > > > commit 2589cd05008205ee29f5f66f24a684732ee2e3a3
> > > > Merge: 98d0347fe8fb e1c75831f682
> > > > Author: Jens Axboe <axboe@kernel.dk>
> > > > Date: Wed Jun 18 05:11:50 2025 -0600
> > > >
> > > > Merge branch 'io_uring-6.16' into for-next
> > > >
> > > > * io_uring-6.16:
> > > > io_uring: fix potential page leak in io_sqe_buffer_register()
> > > > io_uring/sqpoll: don't put task_struct on tctx setup failure
> > > > io_uring: remove duplicate io_uring_alloc_task_context() definition
> > >
> > > The above branch has been merged to v6.16-rc3, can you reproduce it with -rc3?
> > >
> > > I tried to duplicate in my test VM, not succeed with -rc3.
> > >
> > > ...
> > >
> > > > [ 7044.064528] BUG: kernel NULL pointer dereference, address: 0000000000000001
> > > > [ 7044.071507] #PF: supervisor read access in kernel mode
> > > > [ 7044.076653] #PF: error_code(0x0000) - not-present page
> > > > [ 7044.081801] PGD 462c42067 P4D 462c42067 PUD 462c43067 PMD 0
> > > > [ 7044.087488] Oops: Oops: 0000 [#1] SMP NOPTI
> > > > [ 7044.091685] CPU: 13 UID: 0 PID: 367 Comm: kworker/13:1H Not tainted
> > > > 6.16.0-rc2+ #1 PREEMPT(voluntary)
> > > > [ 7044.100991] Hardware name: Dell Inc. PowerEdge R640/0X45NX, BIOS
> > > > 2.22.2 09/12/2024
> > > > [ 7044.108565] Workqueue: kblockd blk_mq_requeue_work
> > > > [ 7044.113374] RIP: 0010:__io_req_task_work_add+0x18/0x1f0
> > >
> > > Can you share where the above line points to source line if it can be
> > > reproduced in -rc3?
> > >
> > > gdb> l *(__io_req_task_work_add+0x18)
> > >
> > >
> > > Thanks,
> > > Ming
> > >
> >
> > now successfully reproduced on v6.16-rc3, more loop tests are needed
> > to trigger this issue，
> >
> > [ 8898.102836] BUG: kernel NULL pointer dereference, address: 0000000000000001
> > [ 8898.109848] #PF: supervisor read access in kernel mode
> > [ 8898.115011] #PF: error_code(0x0000) - not-present page
> > [ 8898.120161] PGD 80000001bcd7b067 P4D 80000001bcd7b067 PUD 1ee49f067 PMD 0
> > [ 8898.127043] Oops: Oops: 0000 [#1] SMP PTI
> > [ 8898.131065] CPU: 2 UID: 0 PID: 47056 Comm: kworker/2:2H Not tainted
> > 6.16.0-rc3 #1 PREEMPT(voluntary)
> > [ 8898.140283] Hardware name: Dell Inc. PowerEdge R340/045M96, BIOS
> > 2.17.3 09/12/2024
> > [ 8898.147860] Workqueue: kblockd blk_mq_requeue_work
> > [ 8898.152658] RIP: 0010:__io_req_task_work_add+0x18/0x1f0
> > [ 8898.157895] Code: 40 00 90 90 90 90 90 90 90 90 90 90 90 90 90 90
> > 90 90 66 0f 1f 00 0f 1f 44 00 00 41 56 41 55 41 54 55 53 48 8b 6f 60
> > 48 89 fb <f6> 45 01 20 0f 84 8e 00 00 00 31 c0 f6 47 48 0c 0f 94 c0 21
> > c6 41
>
> Disassembling this:
> 0: 41 56 push r14
> 2: 41 55 push r13
> 4: 41 54 push r12
> 6: 55 push rbp
> 7: 53 push rbx
> 8: 48 8b 6f 60 mov rbp,QWORD PTR [rdi+0x60]
> c: 48 89 fb mov rbx,rdi
> f: f6 45 01 20 test BYTE PTR [rbp+0x1],0x20 <--here
> 13: 0f 84 8e 00 00 00 je 0xa7
> 19: 31 c0 xor eax,eax
> 1b: f6 47 48 0c test BYTE PTR [rdi+0x48],0xc
> 1f: 0f 94 c0 sete al
> 22: 21 c6 and esi,eax
>
> So we look to be at the start of __io_req_task_work_add(). rdi stores
> req, rbp stores req->ctx, and so the test instruction that's faulting
> is loading (the second byte of) req->ctx->flags for the
> req->ctx->flags & IORING_SETUP_DEFER_TASKRUN check. This means
> req->ctx is NULL. Is it possible the req has already been completed or
> cancelled? The stacktrace shows that this is coming from
> blk_mq_requeue_work, which is definitely interesting.
>
The issue should be in handling UBLK_IO_NEED_GET_DATA, -EIOCBQUEUED is
returned without setting io->cmd.
I will send a fix soon.
Thanks
Ming
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2025-06-24 1:26 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-06-23 2:58 [bug report] BUG: kernel NULL pointer dereference, address: 0000000000000001 Changhui Zhong
2025-06-23 4:02 ` Ming Lei
[not found] ` <CAGVVp+UEQ2XWrNpAz4-+SuyoHybrQ3-Uv5hE-SLQAdSpVw-kgQ@mail.gmail.com>
2025-06-23 8:54 ` Fwd: " Changhui Zhong
2025-06-23 9:12 ` Changhui Zhong
2025-06-23 20:33 ` Caleb Sander Mateos
2025-06-24 1:26 ` Ming Lei
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox