* [PATCH v2] erofs: fix use-after-free of on-stack io[]
@ 2022-04-01 11:55 Hongyu Jin
2022-04-02 2:53 ` Gao Xiang
2022-04-15 12:52 ` Chao Yu
0 siblings, 2 replies; 3+ messages in thread
From: Hongyu Jin @ 2022-04-01 11:55 UTC (permalink / raw)
To: linux-erofs; +Cc: hongyu.jin.cn
From: Hongyu Jin <hongyu.jin@unisoc.com>
The root cause is the race as follows:
Thread #1 Thread #2(irq ctx)
z_erofs_runqueue()
struct z_erofs_decompressqueue io_A[];
submit bio A
z_erofs_decompress_kickoff(,,1)
z_erofs_decompressqueue_endio(bio A)
z_erofs_decompress_kickoff(,,-1)
spin_lock_irqsave()
atomic_add_return()
io_wait_event()
[end of function]
z_erofs_runqueue() // bio B
wake_up_locked(io_A[]) // crash
struct z_erofs_decompressqueue io_B[];
submit bio B
z_erofs_decompress_kickoff(,,1)
Backtrace in kernel5.4:
[ 10.129413] 8<--- cut here ---
[ 10.129422] Unable to handle kernel paging request at virtual address eb0454a4
[ 10.364157] CPU: 0 PID: 709 Comm: getprop Tainted: G WC O 5.4.147-ab09225 #1
[ 11.556325] [<c01b33b8>] (__wake_up_common) from [<c01b3300>] (__wake_up_locked+0x40/0x48)
[ 11.565487] [<c01b3300>] (__wake_up_locked) from [<c044c8d0>] (z_erofs_vle_unzip_kickoff+0x6c/0xc0)
[ 11.575438] [<c044c8d0>] (z_erofs_vle_unzip_kickoff) from [<c044c854>] (z_erofs_vle_read_endio+0x16c/0x17c)
[ 11.586082] [<c044c854>] (z_erofs_vle_read_endio) from [<c06a80e8>] (clone_endio+0xb4/0x1d0)
[ 11.595428] [<c06a80e8>] (clone_endio) from [<c04a1280>] (blk_update_request+0x150/0x4dc)
[ 11.604516] [<c04a1280>] (blk_update_request) from [<c06dea28>] (mmc_blk_cqe_complete_rq+0x144/0x15c)
[ 11.614640] [<c06dea28>] (mmc_blk_cqe_complete_rq) from [<c04a5d90>] (blk_done_softirq+0xb0/0xcc)
[ 11.624419] [<c04a5d90>] (blk_done_softirq) from [<c010242c>] (__do_softirq+0x184/0x56c)
[ 11.633419] [<c010242c>] (__do_softirq) from [<c01051e8>] (irq_exit+0xd4/0x138)
[ 11.641640] [<c01051e8>] (irq_exit) from [<c010c314>] (__handle_domain_irq+0x94/0xd0)
[ 11.650381] [<c010c314>] (__handle_domain_irq) from [<c04fde70>] (gic_handle_irq+0x50/0xd4)
[ 11.659641] [<c04fde70>] (gic_handle_irq) from [<c0101b70>] (__irq_svc+0x70/0xb0)
Signed-off-by: Hongyu Jin <hongyu.jin@unisoc.com>
---
Changes in v2:
- Turn the race function names in commit message to the current
fs/erofs/zdata.c | 12 ++++--------
fs/erofs/zdata.h | 2 +-
2 files changed, 5 insertions(+), 9 deletions(-)
diff --git a/fs/erofs/zdata.c b/fs/erofs/zdata.c
index 0ed880f42525..e6dea6dfca16 100644
--- a/fs/erofs/zdata.c
+++ b/fs/erofs/zdata.c
@@ -1066,12 +1066,9 @@ static void z_erofs_decompress_kickoff(struct z_erofs_decompressqueue *io,
/* wake up the caller thread for sync decompression */
if (sync) {
- unsigned long flags;
-
- spin_lock_irqsave(&io->u.wait.lock, flags);
if (!atomic_add_return(bios, &io->pending_bios))
- wake_up_locked(&io->u.wait);
- spin_unlock_irqrestore(&io->u.wait.lock, flags);
+ complete(&io->u.done);
+
return;
}
@@ -1217,7 +1214,7 @@ jobqueue_init(struct super_block *sb,
} else {
fg_out:
q = fgq;
- init_waitqueue_head(&fgq->u.wait);
+ init_completion(&fgq->u.done);
atomic_set(&fgq->pending_bios, 0);
}
q->sb = sb;
@@ -1419,8 +1416,7 @@ static void z_erofs_runqueue(struct super_block *sb,
return;
/* wait until all bios are completed */
- io_wait_event(io[JQ_SUBMIT].u.wait,
- !atomic_read(&io[JQ_SUBMIT].pending_bios));
+ wait_for_completion_io(&io[JQ_SUBMIT].u.done);
/* handle synchronous decompress queue in the caller context */
z_erofs_decompress_queue(&io[JQ_SUBMIT], pagepool);
diff --git a/fs/erofs/zdata.h b/fs/erofs/zdata.h
index e043216b545f..800b11c53f57 100644
--- a/fs/erofs/zdata.h
+++ b/fs/erofs/zdata.h
@@ -97,7 +97,7 @@ struct z_erofs_decompressqueue {
z_erofs_next_pcluster_t head;
union {
- wait_queue_head_t wait;
+ struct completion done;
struct work_struct work;
} u;
};
--
2.25.1
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH v2] erofs: fix use-after-free of on-stack io[]
2022-04-01 11:55 [PATCH v2] erofs: fix use-after-free of on-stack io[] Hongyu Jin
@ 2022-04-02 2:53 ` Gao Xiang
2022-04-15 12:52 ` Chao Yu
1 sibling, 0 replies; 3+ messages in thread
From: Gao Xiang @ 2022-04-02 2:53 UTC (permalink / raw)
To: Hongyu Jin; +Cc: linux-erofs
On Fri, Apr 01, 2022 at 07:55:27PM +0800, Hongyu Jin wrote:
> From: Hongyu Jin <hongyu.jin@unisoc.com>
>
> The root cause is the race as follows:
> Thread #1 Thread #2(irq ctx)
>
> z_erofs_runqueue()
> struct z_erofs_decompressqueue io_A[];
> submit bio A
> z_erofs_decompress_kickoff(,,1)
> z_erofs_decompressqueue_endio(bio A)
> z_erofs_decompress_kickoff(,,-1)
> spin_lock_irqsave()
> atomic_add_return()
> io_wait_event() -> pending_bios is already 0
> [end of function]
> wake_up_locked(io_A[]) // crash
Actually I think it can be written like this to simplify the case a
bit.
>
> Backtrace in kernel5.4:
> [ 10.129413] 8<--- cut here ---
> [ 10.129422] Unable to handle kernel paging request at virtual address eb0454a4
> [ 10.364157] CPU: 0 PID: 709 Comm: getprop Tainted: G WC O 5.4.147-ab09225 #1
> [ 11.556325] [<c01b33b8>] (__wake_up_common) from [<c01b3300>] (__wake_up_locked+0x40/0x48)
> [ 11.565487] [<c01b3300>] (__wake_up_locked) from [<c044c8d0>] (z_erofs_vle_unzip_kickoff+0x6c/0xc0)
> [ 11.575438] [<c044c8d0>] (z_erofs_vle_unzip_kickoff) from [<c044c854>] (z_erofs_vle_read_endio+0x16c/0x17c)
> [ 11.586082] [<c044c854>] (z_erofs_vle_read_endio) from [<c06a80e8>] (clone_endio+0xb4/0x1d0)
> [ 11.595428] [<c06a80e8>] (clone_endio) from [<c04a1280>] (blk_update_request+0x150/0x4dc)
> [ 11.604516] [<c04a1280>] (blk_update_request) from [<c06dea28>] (mmc_blk_cqe_complete_rq+0x144/0x15c)
> [ 11.614640] [<c06dea28>] (mmc_blk_cqe_complete_rq) from [<c04a5d90>] (blk_done_softirq+0xb0/0xcc)
> [ 11.624419] [<c04a5d90>] (blk_done_softirq) from [<c010242c>] (__do_softirq+0x184/0x56c)
> [ 11.633419] [<c010242c>] (__do_softirq) from [<c01051e8>] (irq_exit+0xd4/0x138)
> [ 11.641640] [<c01051e8>] (irq_exit) from [<c010c314>] (__handle_domain_irq+0x94/0xd0)
> [ 11.650381] [<c010c314>] (__handle_domain_irq) from [<c04fde70>] (gic_handle_irq+0x50/0xd4)
> [ 11.659641] [<c04fde70>] (gic_handle_irq) from [<c0101b70>] (__irq_svc+0x70/0xb0)
>
> Signed-off-by: Hongyu Jin <hongyu.jin@unisoc.com>
Looks good to me!
Reviewed-by: Gao Xiang <hsiangkao@linux.alibaba.com>
Let me test it this weekend and upstream it this cycle.
Thanks,
Gao Xiang
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH v2] erofs: fix use-after-free of on-stack io[]
2022-04-01 11:55 [PATCH v2] erofs: fix use-after-free of on-stack io[] Hongyu Jin
2022-04-02 2:53 ` Gao Xiang
@ 2022-04-15 12:52 ` Chao Yu
1 sibling, 0 replies; 3+ messages in thread
From: Chao Yu @ 2022-04-15 12:52 UTC (permalink / raw)
To: Hongyu Jin, linux-erofs
On 2022/4/1 19:55, Hongyu Jin wrote:
> From: Hongyu Jin <hongyu.jin@unisoc.com>
>
> The root cause is the race as follows:
> Thread #1 Thread #2(irq ctx)
>
> z_erofs_runqueue()
> struct z_erofs_decompressqueue io_A[];
> submit bio A
> z_erofs_decompress_kickoff(,,1)
> z_erofs_decompressqueue_endio(bio A)
> z_erofs_decompress_kickoff(,,-1)
> spin_lock_irqsave()
> atomic_add_return()
> io_wait_event()
> [end of function]
> z_erofs_runqueue() // bio B
> wake_up_locked(io_A[]) // crash
> struct z_erofs_decompressqueue io_B[];
> submit bio B
> z_erofs_decompress_kickoff(,,1)
>
> Backtrace in kernel5.4:
> [ 10.129413] 8<--- cut here ---
> [ 10.129422] Unable to handle kernel paging request at virtual address eb0454a4
> [ 10.364157] CPU: 0 PID: 709 Comm: getprop Tainted: G WC O 5.4.147-ab09225 #1
> [ 11.556325] [<c01b33b8>] (__wake_up_common) from [<c01b3300>] (__wake_up_locked+0x40/0x48)
> [ 11.565487] [<c01b3300>] (__wake_up_locked) from [<c044c8d0>] (z_erofs_vle_unzip_kickoff+0x6c/0xc0)
> [ 11.575438] [<c044c8d0>] (z_erofs_vle_unzip_kickoff) from [<c044c854>] (z_erofs_vle_read_endio+0x16c/0x17c)
> [ 11.586082] [<c044c854>] (z_erofs_vle_read_endio) from [<c06a80e8>] (clone_endio+0xb4/0x1d0)
> [ 11.595428] [<c06a80e8>] (clone_endio) from [<c04a1280>] (blk_update_request+0x150/0x4dc)
> [ 11.604516] [<c04a1280>] (blk_update_request) from [<c06dea28>] (mmc_blk_cqe_complete_rq+0x144/0x15c)
> [ 11.614640] [<c06dea28>] (mmc_blk_cqe_complete_rq) from [<c04a5d90>] (blk_done_softirq+0xb0/0xcc)
> [ 11.624419] [<c04a5d90>] (blk_done_softirq) from [<c010242c>] (__do_softirq+0x184/0x56c)
> [ 11.633419] [<c010242c>] (__do_softirq) from [<c01051e8>] (irq_exit+0xd4/0x138)
> [ 11.641640] [<c01051e8>] (irq_exit) from [<c010c314>] (__handle_domain_irq+0x94/0xd0)
> [ 11.650381] [<c010c314>] (__handle_domain_irq) from [<c04fde70>] (gic_handle_irq+0x50/0xd4)
> [ 11.659641] [<c04fde70>] (gic_handle_irq) from [<c0101b70>] (__irq_svc+0x70/0xb0)
>
> Signed-off-by: Hongyu Jin <hongyu.jin@unisoc.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Thanks,
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2022-04-15 12:52 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-04-01 11:55 [PATCH v2] erofs: fix use-after-free of on-stack io[] Hongyu Jin
2022-04-02 2:53 ` Gao Xiang
2022-04-15 12:52 ` Chao Yu
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox