[PATCH 0/2] mtd: ubi: fix direct reclaim deadlock with synchronous block devices

[PATCH 0/2] mtd: ubi: fix direct reclaim deadlock with synchronous block devices

[Yelin Fei]

From: Ye Linfei <yelinfei@ruijie.com.cn>

Two-part fix for ubiblock swap deadlock: forbid direct reclaim I/O in the synchronous dispatch path and switch the ltree entry allocation to GFP_NOIO.

Ye Linfei (2):
  mtd: ubi: block: forbid direct reclaim I/O during synchronous read
  mtd: ubi: use GFP_NOIO for lock tree entry allocation

 drivers/mtd/ubi/block.c | 15 +++++++++++++++
 drivers/mtd/ubi/eba.c   |  2 +-
 2 files changed, 16 insertions(+), 1 deletion(-)

-- 
2.25.1


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

[PATCH 1/2] mtd: ubi: block: forbid direct reclaim I/O during synchronous read

[Yelin Fei]

From: Ye Linfei <yelinfei@ruijie.com.cn>

ubiblock implements ->queue_rq() synchronously: the task submitting the
bio executes the whole UBI/MTD read itself, from inside the block layer
submission loop (current->bio_list is non-empty) and while holding the
tag_set SRCU read lock (blk_mq_run_dispatch_ops(), taken for
BLK_MQ_F_BLOCKING drivers).

An allocation in that context may trigger direct reclaim, which with
GFP_NOFS - used by all UBI allocations - may synchronously write dirty
pages to swap.  On systems where swap is backed by a device with
BLK_FEAT_SYNCHRONOUS (zram), that ends in swap_writepage_bdev_sync()
-> submit_bio_wait().  The swap bio is only queued onto
current->bio_list, never dispatched: per submit_bio_noacct_nocheck(),
bios submitted while a submission loop is active are only collected on
current->bio_list, and their sole dispatcher is the task currently
executing ->queue_rq() - which is now sleeping waiting for the very
same bio's completion.  The task hangs forever, the pending SRCU read
lock freezes grace periods (blk_mq_timeout_work() -> synchronize_srcu()
never returns), and the unfinished request blocks all other I/O to the
device.

This was observed in the field on a 1 GiB RISC-V networking product
running squashfs-on-ubiblock with zram swap: 47 tasks stuck in disk
sleep for 600+ seconds, ending in a hung task panic.  The allocator
never reached the OOM decision point, because the task died inside
try_to_free_pages().

Mark the whole read with memalloc_noio_save()/memalloc_noio_restore()
so direct reclaim never issues block I/O while this driver executes:
reclaim is restricted to dropping clean page cache and shrinking slabs,
page (swap) reclaim is performed asynchronously by kswapd, and the
worst case is -ENOMEM, which propagates to the caller as an ordinary
recoverable I/O error - a recoverable read failure instead of a total
system hang.

This covers every allocation performed below ubi_read_sg(), including
lock-tree entries, VID buffers and I/O helpers, regardless of the gfp
flags they use internally.

Fixes: 9d54c8a33eec ("UBI: R/O block driver on top of UBI volumes")
Cc: stable@vger.kernel.org
Signed-off-by: Ye Linfei <yelinfei@ruijie.com.cn>
---
 drivers/mtd/ubi/block.c | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/drivers/mtd/ubi/block.c b/drivers/mtd/ubi/block.c
index 29c0d6941a81..c9d42181bf64 100644
--- a/drivers/mtd/ubi/block.c
+++ b/drivers/mtd/ubi/block.c
@@ -38,6 +38,7 @@
 #include <linux/blkdev.h>
 #include <linux/blk-mq.h>
 #include <linux/hdreg.h>
+#include <linux/sched/mm.h>
 #include <linux/scatterlist.h>
 #include <linux/idr.h>
 #include <asm/div64.h>
@@ -189,6 +190,7 @@ static blk_status_t ubiblock_read(struct request *req)
 	int leb = pos;
 	struct req_iterator iter;
 	struct bio_vec bvec;
+	unsigned int noio_flags;
 	int ret;
 
 	blk_mq_start_request(req);
@@ -201,6 +203,18 @@ static blk_status_t ubiblock_read(struct request *req)
 	ubi_sgl_init(&pdu->usgl);
 	blk_rq_map_sg(req, pdu->usgl.sg);
 
+	/*
+	 * This entire callback runs in the synchronous dispatch path of
+	 * blk-mq: submit_bio_noacct_nocheck() only queues a bio submitted
+	 * from here onto current->bio_list, and the sole dispatcher of
+	 * that list is this very task, busy inside ->queue_rq().  Any
+	 * block I/O issued from below would therefore deadlock against
+	 * itself (e.g. synchronous swap writeback reached via direct
+	 * reclaim).  Forbid it for the whole read instead of relying on
+	 * every callee using safe allocation flags.
+	 */
+	noio_flags = memalloc_noio_save();
+
 	while (bytes_left) {
 		/*
 		 * We can only read one LEB at a time. Therefore if the read
@@ -221,6 +235,7 @@ static blk_status_t ubiblock_read(struct request *req)
 
 	rq_for_each_segment(bvec, req, iter)
 		flush_dcache_page(bvec.bv_page);
+	memalloc_noio_restore(noio_flags);
 
 	blk_mq_end_request(req, errno_to_blk_status(ret));
 
-- 
2.25.1


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

[PATCH 2/2] mtd: ubi: use GFP_NOIO for lock tree entry allocation

[Yelin Fei]

From: Ye Linfei <yelinfei@ruijie.com.cn>

The per-LEB lock tree entry can be allocated while UBI is executing on
behalf of a synchronous block-layer consumer (ubiblock), where issuing
further block I/O from direct reclaim would deadlock; see the
preceding commit "mtd: ubi: block: forbid direct reclaim I/O during
synchronous read" for the full analysis.

Make the allocation self-contained: GFP_NOIO restricts direct reclaim
to dropping clean page cache and shrinking slabs, page (swap) reclaim
is left to kswapd, and the worst case is -ENOMEM, which callers
already propagate as an ordinary recoverable I/O error.

memalloc_noio_save()/memalloc_noio_restore() in ubiblock_read()
already enforce this for the whole read scope; keeping the allocation
here self-contained as well means the safety property does not depend
on every future caller remembering to set up that scope.

Signed-off-by: Ye Linfei <yelinfei@ruijie.com.cn>
---
 drivers/mtd/ubi/eba.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/mtd/ubi/eba.c b/drivers/mtd/ubi/eba.c
index f0c549e62c55..8fe68e7fff58 100644
--- a/drivers/mtd/ubi/eba.c
+++ b/drivers/mtd/ubi/eba.c
@@ -247,7 +247,7 @@ static struct ubi_ltree_entry *ltree_add_entry(struct ubi_device *ubi,
 {
 	struct ubi_ltree_entry *le, *le1, *le_free;
 
-	le = kmalloc_obj(struct ubi_ltree_entry, GFP_NOFS);
+	le = kmalloc_obj(struct ubi_ltree_entry, GFP_NOIO); /* see ubiblock_read() */
 	if (!le)
 		return ERR_PTR(-ENOMEM);
 
-- 
2.25.1


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: [PATCH 1/2] mtd: ubi: block: forbid direct reclaim I/O during synchronous read

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] mtd: block: uninitialized use of 'ret' on 0-byte read
--

commit d98a6ab0cd0bd3200e8fcacff751a85e2f9ee372
Author: Ye Linfei <yelinfei@ruijie.com.cn>

mtd: ubi: block: forbid direct reclaim I/O during synchronous read

This commit marks the entire ubiblock_read operation with
memalloc_noio_save() and memalloc_noio_restore() to forbid direct reclaim
I/O while the driver executes. This prevents a deadlock where synchronous
swap writeback could be triggered via direct reclaim while the task is
already holding the SRCU read lock in the blk-mq synchronous dispatch path.

Cc: stable@vger.kernel.org

> diff --git a/drivers/mtd/ubi/block.c b/drivers/mtd/ubi/block.c
> index 29c0d6941a818..c9d42181bf648 100644
> --- a/drivers/mtd/ubi/block.c
> +++ b/drivers/mtd/ubi/block.c

[ ... ]

> @@ -189,6 +190,7 @@ static blk_status_t ubiblock_read(struct request *req)
>  	int leb = pos;
>  	struct req_iterator iter;
>  	struct bio_vec bvec;
> +	unsigned int noio_flags;
>  	int ret;
>  
>  	blk_mq_start_request(req);
> @@ -201,6 +203,18 @@ static blk_status_t ubiblock_read(struct request *req)
>  	ubi_sgl_init(&pdu->usgl);
>  	blk_rq_map_sg(req, pdu->usgl.sg);
>  
> +	/*
> +	 * This entire callback runs in the synchronous dispatch path of
> +	 * blk-mq: submit_bio_noacct_nocheck() only queues a bio submitted
> +	 * from here onto current->bio_list, and the sole dispatcher of
> +	 * that list is this very task, busy inside ->queue_rq().  Any
> +	 * block I/O issued from below would therefore deadlock against
> +	 * itself (e.g. synchronous swap writeback reached via direct
> +	 * reclaim).  Forbid it for the whole read instead of relying on
> +	 * every callee using safe allocation flags.
> +	 */
> +	noio_flags = memalloc_noio_save();
> +
>  	while (bytes_left) {
>  		/*
>  		 * We can only read one LEB at a time. Therefore if the read
> @@ -221,6 +235,7 @@ static blk_status_t ubiblock_read(struct request *req)
>  
>  	rq_for_each_segment(bvec, req, iter)
>  		flush_dcache_page(bvec.bv_page);
> +	memalloc_noio_restore(noio_flags);
>  
>  	blk_mq_end_request(req, errno_to_blk_status(ret));

[Severity: Low]
Does ubiblock_read() in drivers/mtd/ubi/block.c leave ret uninitialized if a
0-byte read request is submitted?

If blk_rq_bytes(req) evaluates to 0, bytes_left is 0 and the loop is bypassed:

    int to_read = blk_rq_bytes(req);
    int bytes_left = to_read;
    ...
    int ret;
    ...
    while (bytes_left) { // bypassed completely
        ...
        ret = ubi_read_sg(...);
    }

This leaves the uninitialized ret value on the stack to be consumed here by
errno_to_blk_status(), returning an unpredictable block completion status.

While 0-byte reads might be rare in practice, the block layer does not
structurally filter out dataless REQ_OP_READ bios.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/cover.1792046946.git.yelinfei@ruijie.com.cn?part=1

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: [PATCH 1/2] mtd: ubi: block: forbid direct reclaim I/O during synchronous read

[Zhihao Cheng]

在 2026/10/15 15:33, Yelin Fei 写道:
> From: Ye Linfei <yelinfei@ruijie.com.cn>
> 
> ubiblock implements ->queue_rq() synchronously: the task submitting the
> bio executes the whole UBI/MTD read itself, from inside the block layer
> submission loop (current->bio_list is non-empty) and while holding the
> tag_set SRCU read lock (blk_mq_run_dispatch_ops(), taken for
> BLK_MQ_F_BLOCKING drivers).
> 
> An allocation in that context may trigger direct reclaim, which with
> GFP_NOFS - used by all UBI allocations - may synchronously write dirty
> pages to swap.  On systems where swap is backed by a device with
> BLK_FEAT_SYNCHRONOUS (zram), that ends in swap_writepage_bdev_sync()
> -> submit_bio_wait().  The swap bio is only queued onto
> current->bio_list, never dispatched: per submit_bio_noacct_nocheck(),
> bios submitted while a submission loop is active are only collected on
> current->bio_list, and their sole dispatcher is the task currently
> executing ->queue_rq() - which is now sleeping waiting for the very
> same bio's completion.  The task hangs forever, the pending SRCU read
> lock freezes grace periods (blk_mq_timeout_work() -> synchronize_srcu()
> never returns), and the unfinished request blocks all other I/O to the
> device.
> 
> This was observed in the field on a 1 GiB RISC-V networking product
> running squashfs-on-ubiblock with zram swap: 47 tasks stuck in disk
> sleep for 600+ seconds, ending in a hung task panic.  The allocator
> never reached the OOM decision point, because the task died inside
> try_to_free_pages().
> 
> Mark the whole read with memalloc_noio_save()/memalloc_noio_restore()
> so direct reclaim never issues block I/O while this driver executes:
> reclaim is restricted to dropping clean page cache and shrinking slabs,
> page (swap) reclaim is performed asynchronously by kswapd, and the
> worst case is -ENOMEM, which propagates to the caller as an ordinary
> recoverable I/O error - a recoverable read failure instead of a total
> system hang.
> 
> This covers every allocation performed below ubi_read_sg(), including
> lock-tree entries, VID buffers and I/O helpers, regardless of the gfp
> flags they use internally.
> 
> Fixes: 9d54c8a33eec ("UBI: R/O block driver on top of UBI volumes")
> Cc: stable@vger.kernel.org
> Signed-off-by: Ye Linfei <yelinfei@ruijie.com.cn>
> ---
>   drivers/mtd/ubi/block.c | 15 +++++++++++++++
>   1 file changed, 15 insertions(+)

Looks like other block drivers(eg. nvme/nbd/rbd..)  won't trigger direct 
mem reclaim by using flags 
GFP_ATOMIC(nvme_pci_setup_meta_iter)/GFP_NOIO(nbd 
sock_xmit)/GFP_NOWAIT(loop).

Reviewed-by: Zhihao Cheng <chengzhihao1@huawei.com>
> 
> diff --git a/drivers/mtd/ubi/block.c b/drivers/mtd/ubi/block.c
> index 29c0d6941a81..c9d42181bf64 100644
> --- a/drivers/mtd/ubi/block.c
> +++ b/drivers/mtd/ubi/block.c
> @@ -38,6 +38,7 @@
>   #include <linux/blkdev.h>
>   #include <linux/blk-mq.h>
>   #include <linux/hdreg.h>
> +#include <linux/sched/mm.h>
>   #include <linux/scatterlist.h>
>   #include <linux/idr.h>
>   #include <asm/div64.h>
> @@ -189,6 +190,7 @@ static blk_status_t ubiblock_read(struct request *req)
>   	int leb = pos;
>   	struct req_iterator iter;
>   	struct bio_vec bvec;
> +	unsigned int noio_flags;
>   	int ret;
>   
>   	blk_mq_start_request(req);
> @@ -201,6 +203,18 @@ static blk_status_t ubiblock_read(struct request *req)
>   	ubi_sgl_init(&pdu->usgl);
>   	blk_rq_map_sg(req, pdu->usgl.sg);
>   
> +	/*
> +	 * This entire callback runs in the synchronous dispatch path of
> +	 * blk-mq: submit_bio_noacct_nocheck() only queues a bio submitted
> +	 * from here onto current->bio_list, and the sole dispatcher of
> +	 * that list is this very task, busy inside ->queue_rq().  Any
> +	 * block I/O issued from below would therefore deadlock against
> +	 * itself (e.g. synchronous swap writeback reached via direct
> +	 * reclaim).  Forbid it for the whole read instead of relying on
> +	 * every callee using safe allocation flags.
> +	 */
> +	noio_flags = memalloc_noio_save();
> +
>   	while (bytes_left) {
>   		/*
>   		 * We can only read one LEB at a time. Therefore if the read
> @@ -221,6 +235,7 @@ static blk_status_t ubiblock_read(struct request *req)
>   
>   	rq_for_each_segment(bvec, req, iter)
>   		flush_dcache_page(bvec.bv_page);
> +	memalloc_noio_restore(noio_flags);
>   
>   	blk_mq_end_request(req, errno_to_blk_status(ret));
>   
> 


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/