* Re: [syzbot] [netfs?] kernel BUG in iov_iter_revert (2)
2024-10-31 13:07 [syzbot] [netfs?] kernel BUG in iov_iter_revert (2) syzbot
@ 2024-11-30 14:44 ` syzbot
2024-12-04 13:59 ` David Howells
2024-12-04 14:11 ` David Howells
2 siblings, 0 replies; 9+ messages in thread
From: syzbot @ 2024-11-30 14:44 UTC (permalink / raw)
To: dhowells, dmantipov, jlayton, joannelkoong, josef, linux-fsdevel,
linux-kernel, lvc-project, miklos, mszeredi, netfs,
syzkaller-bugs
syzbot has bisected this issue to:
commit 3b97c3652d9128ab7f8c9b8adec6108611fdb153
Author: Joanne Koong <joannelkoong@gmail.com>
Date: Thu Oct 24 17:18:08 2024 +0000
fuse: convert direct io to use folios
bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=13e29f78580000
start commit: f486c8aa16b8 Add linux-next specific files for 20241128
git tree: linux-next
final oops: https://syzkaller.appspot.com/x/report.txt?x=10129f78580000
console output: https://syzkaller.appspot.com/x/log.txt?x=17e29f78580000
kernel config: https://syzkaller.appspot.com/x/.config?x=e348a4873516af92
dashboard link: https://syzkaller.appspot.com/bug?extid=404b4b745080b6210c6c
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=131f71e8580000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=12bc200f980000
Reported-by: syzbot+404b4b745080b6210c6c@syzkaller.appspotmail.com
Fixes: 3b97c3652d91 ("fuse: convert direct io to use folios")
For information about bisection process see: https://goo.gl/tpsmEJ#bisection
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [syzbot] [netfs?] kernel BUG in iov_iter_revert (2)
2024-10-31 13:07 [syzbot] [netfs?] kernel BUG in iov_iter_revert (2) syzbot
2024-11-30 14:44 ` syzbot
@ 2024-12-04 13:59 ` David Howells
2024-12-04 14:20 ` syzbot
2024-12-04 14:11 ` David Howells
2 siblings, 1 reply; 9+ messages in thread
From: David Howells @ 2024-12-04 13:59 UTC (permalink / raw)
To: syzbot
Cc: dhowells, jlayton, linux-fsdevel, linux-kernel, netfs,
syzkaller-bugs
#syz test: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git v6.13-rc1
netfs: Fix enomem handling in buffered reads
If netfs_read_to_pagecache() gets an error from either ->prepare_read() or
from netfs_prepare_read_iterator(), it needs to decrement ->nr_outstanding,
cancel the subrequest and break out of the issuing loop. Currently, it
only does this for two of the cases, but there are two more that aren't
handled.
Fix this by moving the handling to a common place and jumping to it from
all four places. This is in preference to inserting a wrapper around
netfs_prepare_read_iterator() as proposed by Dmitry Antipov[1].
Fixes: ee4cdf7ba857 ("netfs: Speed up buffered reading")
Reported-by: syzbot+404b4b745080b6210c6c@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=404b4b745080b6210c6c
Signed-off-by: David Howells <dhowells@redhat.com>
cc: Dmitry Antipov <dmantipov@yandex.ru>
cc: Jeff Layton <jlayton@kernel.org>
cc: netfs@lists.linux.dev
cc: linux-fsdevel@vger.kernel.org
Link: https://lore.kernel.org/r/20241202093943.227786-1-dmantipov@yandex.ru/ [1]
---
fs/netfs/buffered_read.c | 27 +++++++++++++++------------
1 file changed, 15 insertions(+), 12 deletions(-)
diff --git a/fs/netfs/buffered_read.c b/fs/netfs/buffered_read.c
index 7ac34550c403..e5c7dd5a4c90 100644
--- a/fs/netfs/buffered_read.c
+++ b/fs/netfs/buffered_read.c
@@ -275,22 +275,14 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
netfs_stat(&netfs_n_rh_download);
if (rreq->netfs_ops->prepare_read) {
ret = rreq->netfs_ops->prepare_read(subreq);
- if (ret < 0) {
- atomic_dec(&rreq->nr_outstanding);
- netfs_put_subrequest(subreq, false,
- netfs_sreq_trace_put_cancel);
- break;
- }
+ if (ret < 0)
+ goto prep_failed;
trace_netfs_sreq(subreq, netfs_sreq_trace_prepare);
}
slice = netfs_prepare_read_iterator(subreq);
- if (slice < 0) {
- atomic_dec(&rreq->nr_outstanding);
- netfs_put_subrequest(subreq, false, netfs_sreq_trace_put_cancel);
- ret = slice;
- break;
- }
+ if (slice < 0)
+ goto prep_failed;
rreq->netfs_ops->issue_read(subreq);
goto done;
@@ -302,6 +294,8 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
netfs_stat(&netfs_n_rh_zero);
slice = netfs_prepare_read_iterator(subreq);
+ if (slice < 0)
+ goto prep_failed;
__set_bit(NETFS_SREQ_CLEAR_TAIL, &subreq->flags);
netfs_read_subreq_terminated(subreq, 0, false);
goto done;
@@ -310,6 +304,8 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
if (source == NETFS_READ_FROM_CACHE) {
trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
slice = netfs_prepare_read_iterator(subreq);
+ if (slice < 0)
+ goto prep_failed;
netfs_read_cache_to_pagecache(rreq, subreq);
goto done;
}
@@ -318,6 +314,13 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
WARN_ON_ONCE(1);
break;
+ prep_failed:
+ ret = slice;
+ subreq->error = ret;
+ atomic_dec(&rreq->nr_outstanding);
+ netfs_put_subrequest(subreq, false, netfs_sreq_trace_put_cancel);
+ break;
+
done:
size -= slice;
start += slice;
^ permalink raw reply related [flat|nested] 9+ messages in thread* Re: [syzbot] [netfs?] kernel BUG in iov_iter_revert (2)
2024-10-31 13:07 [syzbot] [netfs?] kernel BUG in iov_iter_revert (2) syzbot
2024-11-30 14:44 ` syzbot
2024-12-04 13:59 ` David Howells
@ 2024-12-04 14:11 ` David Howells
2024-12-04 14:39 ` syzbot
2 siblings, 1 reply; 9+ messages in thread
From: David Howells @ 2024-12-04 14:11 UTC (permalink / raw)
To: syzbot
Cc: dhowells, jlayton, linux-fsdevel, linux-kernel, netfs,
syzkaller-bugs
#syz test: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git v6.13-rc1
netfs: Fix enomem handling in buffered reads
If netfs_read_to_pagecache() gets an error from either ->prepare_read() or
from netfs_prepare_read_iterator(), it needs to decrement ->nr_outstanding,
cancel the subrequest and break out of the issuing loop. Currently, it
only does this for two of the cases, but there are two more that aren't
handled.
Fix this by moving the handling to a common place and jumping to it from
all four places. This is in preference to inserting a wrapper around
netfs_prepare_read_iterator() as proposed by Dmitry Antipov[1].
Fixes: ee4cdf7ba857 ("netfs: Speed up buffered reading")
Reported-by: syzbot+404b4b745080b6210c6c@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=404b4b745080b6210c6c
Signed-off-by: David Howells <dhowells@redhat.com>
cc: Dmitry Antipov <dmantipov@yandex.ru>
cc: Jeff Layton <jlayton@kernel.org>
cc: netfs@lists.linux.dev
cc: linux-fsdevel@vger.kernel.org
Link: https://lore.kernel.org/r/20241202093943.227786-1-dmantipov@yandex.ru/ [1]
---
fs/netfs/buffered_read.c | 28 ++++++++++++++++------------
1 file changed, 16 insertions(+), 12 deletions(-)
diff --git a/fs/netfs/buffered_read.c b/fs/netfs/buffered_read.c
index 7ac34550c403..4dc9b8286355 100644
--- a/fs/netfs/buffered_read.c
+++ b/fs/netfs/buffered_read.c
@@ -275,22 +275,14 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
netfs_stat(&netfs_n_rh_download);
if (rreq->netfs_ops->prepare_read) {
ret = rreq->netfs_ops->prepare_read(subreq);
- if (ret < 0) {
- atomic_dec(&rreq->nr_outstanding);
- netfs_put_subrequest(subreq, false,
- netfs_sreq_trace_put_cancel);
- break;
- }
+ if (ret < 0)
+ goto prep_failed;
trace_netfs_sreq(subreq, netfs_sreq_trace_prepare);
}
slice = netfs_prepare_read_iterator(subreq);
- if (slice < 0) {
- atomic_dec(&rreq->nr_outstanding);
- netfs_put_subrequest(subreq, false, netfs_sreq_trace_put_cancel);
- ret = slice;
- break;
- }
+ if (slice < 0)
+ goto prep_iter_failed;
rreq->netfs_ops->issue_read(subreq);
goto done;
@@ -302,6 +294,8 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
netfs_stat(&netfs_n_rh_zero);
slice = netfs_prepare_read_iterator(subreq);
+ if (slice < 0)
+ goto prep_iter_failed;
__set_bit(NETFS_SREQ_CLEAR_TAIL, &subreq->flags);
netfs_read_subreq_terminated(subreq, 0, false);
goto done;
@@ -310,6 +304,8 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
if (source == NETFS_READ_FROM_CACHE) {
trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
slice = netfs_prepare_read_iterator(subreq);
+ if (slice < 0)
+ goto prep_iter_failed;
netfs_read_cache_to_pagecache(rreq, subreq);
goto done;
}
@@ -318,6 +314,14 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
WARN_ON_ONCE(1);
break;
+ prep_iter_failed:
+ ret = slice;
+ prep_failed:
+ subreq->error = ret;
+ atomic_dec(&rreq->nr_outstanding);
+ netfs_put_subrequest(subreq, false, netfs_sreq_trace_put_cancel);
+ break;
+
done:
size -= slice;
start += slice;
^ permalink raw reply related [flat|nested] 9+ messages in thread* Re: [syzbot] [netfs?] kernel BUG in iov_iter_revert (2)
2024-12-04 14:11 ` David Howells
@ 2024-12-04 14:39 ` syzbot
2024-12-04 14:43 ` David Howells
2024-12-05 9:38 ` David Howells
0 siblings, 2 replies; 9+ messages in thread
From: syzbot @ 2024-12-04 14:39 UTC (permalink / raw)
To: dhowells, jlayton, linux-fsdevel, linux-kernel, netfs,
syzkaller-bugs
Hello,
syzbot has tested the proposed patch but the reproducer is still triggering an issue:
possible deadlock in __submit_bio
======================================================
WARNING: possible circular locking dependency detected
6.13.0-rc1-syzkaller-dirty #0 Not tainted
------------------------------------------------------
kswapd0/75 is trying to acquire lock:
ffff888034c41438 (&q->q_usage_counter(io)#37){++++}-{0:0}, at: __submit_bio+0x2c6/0x560 block/blk-core.c:629
but task is already holding lock:
ffffffff8ea35b00 (fs_reclaim){+.+.}-{0:0}, at: balance_pgdat mm/vmscan.c:6864 [inline]
ffffffff8ea35b00 (fs_reclaim){+.+.}-{0:0}, at: kswapd+0xbf1/0x36f0 mm/vmscan.c:7246
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
-> #1 (fs_reclaim){+.+.}-{0:0}:
lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5849
__fs_reclaim_acquire mm/page_alloc.c:3851 [inline]
fs_reclaim_acquire+0x88/0x130 mm/page_alloc.c:3865
might_alloc include/linux/sched/mm.h:318 [inline]
slab_pre_alloc_hook mm/slub.c:4055 [inline]
slab_alloc_node mm/slub.c:4133 [inline]
__do_kmalloc_node mm/slub.c:4282 [inline]
__kmalloc_node_noprof+0xb2/0x4d0 mm/slub.c:4289
__kvmalloc_node_noprof+0x72/0x190 mm/util.c:650
sbitmap_init_node+0x2d4/0x670 lib/sbitmap.c:132
scsi_realloc_sdev_budget_map+0x2a7/0x460 drivers/scsi/scsi_scan.c:246
scsi_add_lun drivers/scsi/scsi_scan.c:1106 [inline]
scsi_probe_and_add_lun+0x3173/0x4bd0 drivers/scsi/scsi_scan.c:1287
__scsi_add_device+0x228/0x2f0 drivers/scsi/scsi_scan.c:1622
ata_scsi_scan_host+0x236/0x740 drivers/ata/libata-scsi.c:4575
async_run_entry_fn+0xa8/0x420 kernel/async.c:129
process_one_work kernel/workqueue.c:3229 [inline]
process_scheduled_works+0xa66/0x1840 kernel/workqueue.c:3310
worker_thread+0x870/0xd30 kernel/workqueue.c:3391
kthread+0x2f0/0x390 kernel/kthread.c:389
ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
-> #0 (&q->q_usage_counter(io)#37){++++}-{0:0}:
check_prev_add kernel/locking/lockdep.c:3161 [inline]
check_prevs_add kernel/locking/lockdep.c:3280 [inline]
validate_chain+0x18ef/0x5920 kernel/locking/lockdep.c:3904
__lock_acquire+0x1397/0x2100 kernel/locking/lockdep.c:5226
lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5849
bio_queue_enter block/blk.h:75 [inline]
blk_mq_submit_bio+0x1536/0x2390 block/blk-mq.c:3091
__submit_bio+0x2c6/0x560 block/blk-core.c:629
__submit_bio_noacct_mq block/blk-core.c:710 [inline]
submit_bio_noacct_nocheck+0x4d3/0xe30 block/blk-core.c:739
swap_writepage_bdev_async mm/page_io.c:451 [inline]
__swap_writepage+0x5fc/0x1400 mm/page_io.c:474
swap_writepage+0x8f4/0x1170 mm/page_io.c:289
pageout mm/vmscan.c:689 [inline]
shrink_folio_list+0x3c0e/0x8cb0 mm/vmscan.c:1367
evict_folios+0x5568/0x7be0 mm/vmscan.c:4593
try_to_shrink_lruvec+0x9a6/0xc70 mm/vmscan.c:4789
shrink_one+0x3b9/0x850 mm/vmscan.c:4834
shrink_many mm/vmscan.c:4897 [inline]
lru_gen_shrink_node mm/vmscan.c:4975 [inline]
shrink_node+0x37c5/0x3e50 mm/vmscan.c:5956
kswapd_shrink_node mm/vmscan.c:6785 [inline]
balance_pgdat mm/vmscan.c:6977 [inline]
kswapd+0x1ca9/0x36f0 mm/vmscan.c:7246
kthread+0x2f0/0x390 kernel/kthread.c:389
ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
other info that might help us debug this:
Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock(fs_reclaim);
lock(&q->q_usage_counter(io)#37);
lock(fs_reclaim);
rlock(&q->q_usage_counter(io)#37);
*** DEADLOCK ***
1 lock held by kswapd0/75:
#0: ffffffff8ea35b00 (fs_reclaim){+.+.}-{0:0}, at: balance_pgdat mm/vmscan.c:6864 [inline]
#0: ffffffff8ea35b00 (fs_reclaim){+.+.}-{0:0}, at: kswapd+0xbf1/0x36f0 mm/vmscan.c:7246
stack backtrace:
CPU: 0 UID: 0 PID: 75 Comm: kswapd0 Not tainted 6.13.0-rc1-syzkaller-dirty #0
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-1.16.3-2~bpo12+1 04/01/2014
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:94 [inline]
dump_stack_lvl+0x241/0x360 lib/dump_stack.c:120
print_circular_bug+0x13a/0x1b0 kernel/locking/lockdep.c:2074
check_noncircular+0x36a/0x4a0 kernel/locking/lockdep.c:2206
check_prev_add kernel/locking/lockdep.c:3161 [inline]
check_prevs_add kernel/locking/lockdep.c:3280 [inline]
validate_chain+0x18ef/0x5920 kernel/locking/lockdep.c:3904
__lock_acquire+0x1397/0x2100 kernel/locking/lockdep.c:5226
lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5849
bio_queue_enter block/blk.h:75 [inline]
blk_mq_submit_bio+0x1536/0x2390 block/blk-mq.c:3091
__submit_bio+0x2c6/0x560 block/blk-core.c:629
__submit_bio_noacct_mq block/blk-core.c:710 [inline]
submit_bio_noacct_nocheck+0x4d3/0xe30 block/blk-core.c:739
swap_writepage_bdev_async mm/page_io.c:451 [inline]
__swap_writepage+0x5fc/0x1400 mm/page_io.c:474
swap_writepage+0x8f4/0x1170 mm/page_io.c:289
pageout mm/vmscan.c:689 [inline]
shrink_folio_list+0x3c0e/0x8cb0 mm/vmscan.c:1367
evict_folios+0x5568/0x7be0 mm/vmscan.c:4593
try_to_shrink_lruvec+0x9a6/0xc70 mm/vmscan.c:4789
shrink_one+0x3b9/0x850 mm/vmscan.c:4834
shrink_many mm/vmscan.c:4897 [inline]
lru_gen_shrink_node mm/vmscan.c:4975 [inline]
shrink_node+0x37c5/0x3e50 mm/vmscan.c:5956
kswapd_shrink_node mm/vmscan.c:6785 [inline]
balance_pgdat mm/vmscan.c:6977 [inline]
kswapd+0x1ca9/0x36f0 mm/vmscan.c:7246
kthread+0x2f0/0x390 kernel/kthread.c:389
ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
</TASK>
Tested on:
commit: 40384c84 Linux 6.13-rc1
git tree: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git v6.13-rc1
console output: https://syzkaller.appspot.com/x/log.txt?x=101560f8580000
kernel config: https://syzkaller.appspot.com/x/.config?x=58639d2215ba9a07
dashboard link: https://syzkaller.appspot.com/bug?extid=404b4b745080b6210c6c
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
patch: https://syzkaller.appspot.com/x/patch.diff?x=138c4de8580000
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [syzbot] [netfs?] kernel BUG in iov_iter_revert (2)
2024-12-04 14:39 ` syzbot
@ 2024-12-04 14:43 ` David Howells
2024-12-05 9:38 ` David Howells
1 sibling, 0 replies; 9+ messages in thread
From: David Howells @ 2024-12-04 14:43 UTC (permalink / raw)
To: syzbot
Cc: dhowells, jlayton, linux-fsdevel, linux-kernel, netfs,
syzkaller-bugs
This looks like it's probably a separate bug.
David
syzbot <syzbot+404b4b745080b6210c6c@syzkaller.appspotmail.com> wrote:
> syzbot has tested the proposed patch but the reproducer is still triggering an issue:
> possible deadlock in __submit_bio
>
> ======================================================
> WARNING: possible circular locking dependency detected
> 6.13.0-rc1-syzkaller-dirty #0 Not tainted
> ------------------------------------------------------
> kswapd0/75 is trying to acquire lock:
> ffff888034c41438 (&q->q_usage_counter(io)#37){++++}-{0:0}, at: __submit_bio+0x2c6/0x560 block/blk-core.c:629
>
> but task is already holding lock:
> ffffffff8ea35b00 (fs_reclaim){+.+.}-{0:0}, at: balance_pgdat mm/vmscan.c:6864 [inline]
> ffffffff8ea35b00 (fs_reclaim){+.+.}-{0:0}, at: kswapd+0xbf1/0x36f0 mm/vmscan.c:7246
>
> which lock already depends on the new lock.
>
>
> the existing dependency chain (in reverse order) is:
>
> -> #1 (fs_reclaim){+.+.}-{0:0}:
> lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5849
> __fs_reclaim_acquire mm/page_alloc.c:3851 [inline]
> fs_reclaim_acquire+0x88/0x130 mm/page_alloc.c:3865
> might_alloc include/linux/sched/mm.h:318 [inline]
> slab_pre_alloc_hook mm/slub.c:4055 [inline]
> slab_alloc_node mm/slub.c:4133 [inline]
> __do_kmalloc_node mm/slub.c:4282 [inline]
> __kmalloc_node_noprof+0xb2/0x4d0 mm/slub.c:4289
> __kvmalloc_node_noprof+0x72/0x190 mm/util.c:650
> sbitmap_init_node+0x2d4/0x670 lib/sbitmap.c:132
> scsi_realloc_sdev_budget_map+0x2a7/0x460 drivers/scsi/scsi_scan.c:246
> scsi_add_lun drivers/scsi/scsi_scan.c:1106 [inline]
> scsi_probe_and_add_lun+0x3173/0x4bd0 drivers/scsi/scsi_scan.c:1287
> __scsi_add_device+0x228/0x2f0 drivers/scsi/scsi_scan.c:1622
> ata_scsi_scan_host+0x236/0x740 drivers/ata/libata-scsi.c:4575
> async_run_entry_fn+0xa8/0x420 kernel/async.c:129
> process_one_work kernel/workqueue.c:3229 [inline]
> process_scheduled_works+0xa66/0x1840 kernel/workqueue.c:3310
> worker_thread+0x870/0xd30 kernel/workqueue.c:3391
> kthread+0x2f0/0x390 kernel/kthread.c:389
> ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
> ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
>
> -> #0 (&q->q_usage_counter(io)#37){++++}-{0:0}:
> check_prev_add kernel/locking/lockdep.c:3161 [inline]
> check_prevs_add kernel/locking/lockdep.c:3280 [inline]
> validate_chain+0x18ef/0x5920 kernel/locking/lockdep.c:3904
> __lock_acquire+0x1397/0x2100 kernel/locking/lockdep.c:5226
> lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5849
> bio_queue_enter block/blk.h:75 [inline]
> blk_mq_submit_bio+0x1536/0x2390 block/blk-mq.c:3091
> __submit_bio+0x2c6/0x560 block/blk-core.c:629
> __submit_bio_noacct_mq block/blk-core.c:710 [inline]
> submit_bio_noacct_nocheck+0x4d3/0xe30 block/blk-core.c:739
> swap_writepage_bdev_async mm/page_io.c:451 [inline]
> __swap_writepage+0x5fc/0x1400 mm/page_io.c:474
> swap_writepage+0x8f4/0x1170 mm/page_io.c:289
> pageout mm/vmscan.c:689 [inline]
> shrink_folio_list+0x3c0e/0x8cb0 mm/vmscan.c:1367
> evict_folios+0x5568/0x7be0 mm/vmscan.c:4593
> try_to_shrink_lruvec+0x9a6/0xc70 mm/vmscan.c:4789
> shrink_one+0x3b9/0x850 mm/vmscan.c:4834
> shrink_many mm/vmscan.c:4897 [inline]
> lru_gen_shrink_node mm/vmscan.c:4975 [inline]
> shrink_node+0x37c5/0x3e50 mm/vmscan.c:5956
> kswapd_shrink_node mm/vmscan.c:6785 [inline]
> balance_pgdat mm/vmscan.c:6977 [inline]
> kswapd+0x1ca9/0x36f0 mm/vmscan.c:7246
> kthread+0x2f0/0x390 kernel/kthread.c:389
> ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
> ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
>
> other info that might help us debug this:
>
> Possible unsafe locking scenario:
>
> CPU0 CPU1
> ---- ----
> lock(fs_reclaim);
> lock(&q->q_usage_counter(io)#37);
> lock(fs_reclaim);
> rlock(&q->q_usage_counter(io)#37);
>
> *** DEADLOCK ***
>
> 1 lock held by kswapd0/75:
> #0: ffffffff8ea35b00 (fs_reclaim){+.+.}-{0:0}, at: balance_pgdat mm/vmscan.c:6864 [inline]
> #0: ffffffff8ea35b00 (fs_reclaim){+.+.}-{0:0}, at: kswapd+0xbf1/0x36f0 mm/vmscan.c:7246
>
> stack backtrace:
> CPU: 0 UID: 0 PID: 75 Comm: kswapd0 Not tainted 6.13.0-rc1-syzkaller-dirty #0
> Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-1.16.3-2~bpo12+1 04/01/2014
> Call Trace:
> <TASK>
> __dump_stack lib/dump_stack.c:94 [inline]
> dump_stack_lvl+0x241/0x360 lib/dump_stack.c:120
> print_circular_bug+0x13a/0x1b0 kernel/locking/lockdep.c:2074
> check_noncircular+0x36a/0x4a0 kernel/locking/lockdep.c:2206
> check_prev_add kernel/locking/lockdep.c:3161 [inline]
> check_prevs_add kernel/locking/lockdep.c:3280 [inline]
> validate_chain+0x18ef/0x5920 kernel/locking/lockdep.c:3904
> __lock_acquire+0x1397/0x2100 kernel/locking/lockdep.c:5226
> lock_acquire+0x1ed/0x550 kernel/locking/lockdep.c:5849
> bio_queue_enter block/blk.h:75 [inline]
> blk_mq_submit_bio+0x1536/0x2390 block/blk-mq.c:3091
> __submit_bio+0x2c6/0x560 block/blk-core.c:629
> __submit_bio_noacct_mq block/blk-core.c:710 [inline]
> submit_bio_noacct_nocheck+0x4d3/0xe30 block/blk-core.c:739
> swap_writepage_bdev_async mm/page_io.c:451 [inline]
> __swap_writepage+0x5fc/0x1400 mm/page_io.c:474
> swap_writepage+0x8f4/0x1170 mm/page_io.c:289
> pageout mm/vmscan.c:689 [inline]
> shrink_folio_list+0x3c0e/0x8cb0 mm/vmscan.c:1367
> evict_folios+0x5568/0x7be0 mm/vmscan.c:4593
> try_to_shrink_lruvec+0x9a6/0xc70 mm/vmscan.c:4789
> shrink_one+0x3b9/0x850 mm/vmscan.c:4834
> shrink_many mm/vmscan.c:4897 [inline]
> lru_gen_shrink_node mm/vmscan.c:4975 [inline]
> shrink_node+0x37c5/0x3e50 mm/vmscan.c:5956
> kswapd_shrink_node mm/vmscan.c:6785 [inline]
> balance_pgdat mm/vmscan.c:6977 [inline]
> kswapd+0x1ca9/0x36f0 mm/vmscan.c:7246
> kthread+0x2f0/0x390 kernel/kthread.c:389
> ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147
> ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244
> </TASK>
^ permalink raw reply [flat|nested] 9+ messages in thread* Re: [syzbot] [netfs?] kernel BUG in iov_iter_revert (2)
2024-12-04 14:39 ` syzbot
2024-12-04 14:43 ` David Howells
@ 2024-12-05 9:38 ` David Howells
2024-12-05 9:59 ` syzbot
1 sibling, 1 reply; 9+ messages in thread
From: David Howells @ 2024-12-05 9:38 UTC (permalink / raw)
To: syzbot
Cc: dhowells, jlayton, Ming Lei, Jens Axboe, netfs, linux-block,
linux-fsdevel, linux-kernel, syzkaller-bugs
#syz test: https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git for-6.14/block
netfs: Fix enomem handling in buffered reads
If netfs_read_to_pagecache() gets an error from either ->prepare_read() or
from netfs_prepare_read_iterator(), it needs to decrement ->nr_outstanding,
cancel the subrequest and break out of the issuing loop. Currently, it
only does this for two of the cases, but there are two more that aren't
handled.
Fix this by moving the handling to a common place and jumping to it from
all four places. This is in preference to inserting a wrapper around
netfs_prepare_read_iterator() as proposed by Dmitry Antipov[1].
Fixes: ee4cdf7ba857 ("netfs: Speed up buffered reading")
Reported-by: syzbot+404b4b745080b6210c6c@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=404b4b745080b6210c6c
Signed-off-by: David Howells <dhowells@redhat.com>
cc: Dmitry Antipov <dmantipov@yandex.ru>
cc: Jeff Layton <jlayton@kernel.org>
cc: netfs@lists.linux.dev
cc: linux-fsdevel@vger.kernel.org
Link: https://lore.kernel.org/r/20241202093943.227786-1-dmantipov@yandex.ru/ [1]
---
fs/netfs/buffered_read.c | 28 ++++++++++++++++------------
1 file changed, 16 insertions(+), 12 deletions(-)
diff --git a/fs/netfs/buffered_read.c b/fs/netfs/buffered_read.c
index 7ac34550c403..4dc9b8286355 100644
--- a/fs/netfs/buffered_read.c
+++ b/fs/netfs/buffered_read.c
@@ -275,22 +275,14 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
netfs_stat(&netfs_n_rh_download);
if (rreq->netfs_ops->prepare_read) {
ret = rreq->netfs_ops->prepare_read(subreq);
- if (ret < 0) {
- atomic_dec(&rreq->nr_outstanding);
- netfs_put_subrequest(subreq, false,
- netfs_sreq_trace_put_cancel);
- break;
- }
+ if (ret < 0)
+ goto prep_failed;
trace_netfs_sreq(subreq, netfs_sreq_trace_prepare);
}
slice = netfs_prepare_read_iterator(subreq);
- if (slice < 0) {
- atomic_dec(&rreq->nr_outstanding);
- netfs_put_subrequest(subreq, false, netfs_sreq_trace_put_cancel);
- ret = slice;
- break;
- }
+ if (slice < 0)
+ goto prep_iter_failed;
rreq->netfs_ops->issue_read(subreq);
goto done;
@@ -302,6 +294,8 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
netfs_stat(&netfs_n_rh_zero);
slice = netfs_prepare_read_iterator(subreq);
+ if (slice < 0)
+ goto prep_iter_failed;
__set_bit(NETFS_SREQ_CLEAR_TAIL, &subreq->flags);
netfs_read_subreq_terminated(subreq, 0, false);
goto done;
@@ -310,6 +304,8 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
if (source == NETFS_READ_FROM_CACHE) {
trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
slice = netfs_prepare_read_iterator(subreq);
+ if (slice < 0)
+ goto prep_iter_failed;
netfs_read_cache_to_pagecache(rreq, subreq);
goto done;
}
@@ -318,6 +314,14 @@ static void netfs_read_to_pagecache(struct netfs_io_request *rreq)
WARN_ON_ONCE(1);
break;
+ prep_iter_failed:
+ ret = slice;
+ prep_failed:
+ subreq->error = ret;
+ atomic_dec(&rreq->nr_outstanding);
+ netfs_put_subrequest(subreq, false, netfs_sreq_trace_put_cancel);
+ break;
+
done:
size -= slice;
start += slice;
^ permalink raw reply related [flat|nested] 9+ messages in thread* Re: [syzbot] [netfs?] kernel BUG in iov_iter_revert (2)
2024-12-05 9:38 ` David Howells
@ 2024-12-05 9:59 ` syzbot
0 siblings, 0 replies; 9+ messages in thread
From: syzbot @ 2024-12-05 9:59 UTC (permalink / raw)
To: axboe, dhowells, jlayton, linux-block, linux-fsdevel,
linux-kernel, ming.lei, netfs, syzkaller-bugs
Hello,
syzbot has tested the proposed patch and the reproducer did not trigger any issue:
Reported-by: syzbot+404b4b745080b6210c6c@syzkaller.appspotmail.com
Tested-by: syzbot+404b4b745080b6210c6c@syzkaller.appspotmail.com
Tested on:
commit: c018ec9d block: rnull: Initialize the module in place
git tree: https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git for-6.14/block
console output: https://syzkaller.appspot.com/x/log.txt?x=159ca8df980000
kernel config: https://syzkaller.appspot.com/x/.config?x=58639d2215ba9a07
dashboard link: https://syzkaller.appspot.com/bug?extid=404b4b745080b6210c6c
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
patch: https://syzkaller.appspot.com/x/patch.diff?x=14b910f8580000
Note: testing is done by a robot and is best-effort only.
^ permalink raw reply [flat|nested] 9+ messages in thread