* Re: [PATCH 0/1] drbd: fix false positive resync throttling (in-tree 8.4)
From: Ionut Nechita (Wind River) @ 2026-05-15 12:30 UTC (permalink / raw)
To: drbd-dev, linux-block
Cc: Philipp Reisner, Christoph Boehmwalder, Lars Ellenberg,
Jens Axboe, linux-kernel, Ionut Nechita
In-Reply-To: <20260513163905.562722-1-ionut.nechita@windriver.com>
From: Ionut Nechita <ionut.nechita@windriver.com>
Hi Philipp, Christoph,
Friendly ping on this patch. We are planning to integrate it into our
next release and our engineering team asked whether you could provide
a brief risk assessment or a Reviewed-by tag for the in-tree 8.4
backport.
To summarize the risk profile from our side:
- The change is confined to drbd_rs_c_min_rate_throttle() — only the
throttle decision, no data path or replication protocol changes.
- Worst-case failure mode (ap_bio_cnt not reflecting reality) is
equivalent to c-min-rate=0, which is already a supported config.
- The approach is identical to what drbd 9.x already uses.
Could you confirm the patch is correct as-is, or provide a Reviewed-by
/ Acked-by if you're comfortable with it? If there are any concerns
or suggested changes, happy to spin a v3.
Thanks,
Ionut
^ permalink raw reply
* Re: improve the swap_activate interface
From: Christoph Hellwig @ 2026-05-15 11:33 UTC (permalink / raw)
To: Steve French
Cc: Christoph Hellwig, Andrew Morton, Chris Li, Kairui Song,
Christian Brauner, Darrick J . Wong, Jens Axboe, David Sterba,
Theodore Ts'o, Jaegeuk Kim, Chao Yu, Trond Myklebust,
Anna Schumaker, Namjae Jeon, Hyunchul Lee, Steve French,
Paulo Alcantara, Carlos Maiolino, Damien Le Moal, Naohiro Aota,
linux-xfs, linux-fsdevel, linux-doc, linux-mm, linux-block,
linux-btrfs, linux-ext4, linux-f2fs-devel, linux-nfs, linux-cifs
In-Reply-To: <CAH2r5msnYVb3hhXHwqDVHGGC1h4E6mLCRS4ktCrQoD9zdUW81g@mail.gmail.com>
On Wed, May 13, 2026 at 03:34:03PM -0500, Steve French wrote:
> I just tried this on 7.1-rc3 with the swap patches (full kernel build,
> on Ubuntu 25,10) and boot failed with out of memory which I had never
> seen before. Any idea how to workaround this with the swap patch
> series, or is there a fix for this in the swap series already?
Is that a failure with the patches or also with the baseline?
^ permalink raw reply
* Re: [PATCH] blk-cgroup: Fix UAF in blkcg_activate_policy() by using blkg_tryget()
From: Zizhi Wo @ 2026-05-15 10:16 UTC (permalink / raw)
To: Zizhi Wo, axboe, tj, josef, yukuai, linux-block
Cc: cgroups, yangerkun, chengzhihao1
In-Reply-To: <20260515061516.3461291-1-wozizhi@huaweicloud.com>
I just realized that the fix has already been included in this patchset:
https://lore.kernel.org/all/20260304073809.3438679-5-yukuai@fnnas.com/
Please disregard this patch.
在 2026/5/15 14:15, Zizhi Wo 写道:
> [BUG]
> Our fuzz testing triggered a blkg use-after-free issue:
>
> BUG: KASAN: slab-use-after-free in percpu_ref_put_many.constprop.0+0xbe/0xe0
> Call Trace:
> ...
> blkcg_activate_policy+0x347/0xfa0
> bfq_create_group_hierarchy+0x5b/0x140
> bfq_init_queue+0xc1b/0x1470
> ? mutex_init_generic+0x9f/0x100
> ? elevator_alloc+0x166/0x2b0
> blk_mq_init_sched+0x2b0/0x730
> elevator_switch+0x188/0x450
> elevator_change+0x290/0x470
> elv_iosched_store+0x30a/0x3a0
> ...
>
> [CAUSE]
> process1 process2
> cgroup_rmdir
> ...
> blkcg_destroy_blkgs
> spin_trylock(&q->queue_lock)
> blkg_destroy
> percpu_ref_kill(&blkg->refcnt)
> ...
> blkg_free
> INIT_WORK(xxx, blkg_free_workfn)
> schedule_work
> spin_unlock(&q->queue_lock)
> ====================================schedule_work
> blkg_free_workfn
> elevator_change
> ...
> bfq_create_group_hierarchy
> blkcg_activate_policy
> spin_lock_irq(&q->queue_lock)
> blkg_get // get dead ref !!
> pinned_blkg = blkg
> spin_unlock_irq(&q->queue_lock)
> spin_lock_irq(&q->queue_lock)
> list_del_init(&blkg->q_node)
> spin_unlock_irq(&q->queue_lock)
> kfree(blkg)
> blkg_put(pinned_blkg) // UAF !!
>
> A blkg killed by blkg_destroy() stays on q->blkg_list until
> blkg_free_workfn() grabs queue_lock and unlinks it. blkg_get() on a dead
> percpu_ref does not resurrect the blkg, so the later blkg_put() hits freed
> memory and triggers this issue.
>
> [Fix]
> Replace blkg_get() with blkg_tryget(), which fails on a dead ref and lets
> the loop skip dying blkgs.
>
> Also hoist the ref acquisition to the top of the loop so dying blkgs are
> filtered out before a pd is allocated and attached. Otherwise a pd attached
> to an already-destroyed blkg would never called pd_offline_fn().
>
> Fixes: 9d179b865449 ("blkcg: Fix multiple bugs in blkcg_activate_policy()")
> Signed-off-by: Zizhi Wo <wozizhi@huaweicloud.com>
> ---
> block/blk-cgroup.c | 7 ++++++-
> 1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c
> index 554c87bb4a86..03b6ce934848 100644
> --- a/block/blk-cgroup.c
> +++ b/block/blk-cgroup.c
> @@ -1621,6 +1621,10 @@ int blkcg_activate_policy(struct gendisk *disk, const struct blkcg_policy *pol)
> if (blkg->pd[pol->plid])
> continue;
>
> + /* a destroyed blkg may still be on q->blkg_list; skip it via tryget */
> + if (!blkg_tryget(blkg))
> + continue;
> +
> /* If prealloc matches, use it; otherwise try GFP_NOWAIT */
> if (blkg == pinned_blkg) {
> pd = pd_prealloc;
> @@ -1637,7 +1641,6 @@ int blkcg_activate_policy(struct gendisk *disk, const struct blkcg_policy *pol)
> */
> if (pinned_blkg)
> blkg_put(pinned_blkg);
> - blkg_get(blkg);
> pinned_blkg = blkg;
>
> spin_unlock_irq(&q->queue_lock);
> @@ -1666,6 +1669,8 @@ int blkcg_activate_policy(struct gendisk *disk, const struct blkcg_policy *pol)
> pd->online = true;
>
> spin_unlock(&blkg->blkcg->lock);
> +
> + blkg_put(blkg);
> }
>
> __set_bit(pol->plid, q->blkcg_pols);
^ permalink raw reply
* Re: [PATCH blktests v3] nvme/068: add a test for multipath delayed removal
From: Shin'ichiro Kawasaki @ 2026-05-15 6:38 UTC (permalink / raw)
To: John Garry; +Cc: linux-block, linux-nvme, nilay, dwagner, Chaitanya Kulkarni
In-Reply-To: <20260430084635.2438048-1-john.g.garry@oracle.com>
On Apr 30, 2026 / 08:46, John Garry wrote:
> For NVMe multipath, the delayed removal feature allows the multipath
> gendisk to remain present when all available paths are gone. The purpose of
> this feature is to ensure that we keep the gendisk for intermittent path
> failures.
>
> The delayed removal works on a timer - when all paths are gone, a timer is
> kicked off; once the timer expires and no paths have returned, the gendisk
> is removed.
>
> When all paths are gone and the gendisk is still present, all reads and
> writes to the disk are queued. If a path returns before the timer
> expiration, the timer canceled and the queued IO is submitted;
> otherwise they fail when the timer expires.
>
> This testcase covers two scenarios in separate parts:
> a. test that IOs submitted after all paths are removed (and do not return)
> fail
> b. test that IOs submitted between all paths removed and a path
> returning succeed
>
> During the period of the timer being active, it must be ensured that the
> nvme-core module is not removed. Otherwise the driver may not be present
> to handle the timeout expiry. The kernel ensures this by taking a
> reference to the module. Ideally, we would try to remove the module during
> this test to prove that this is not possible (and the kernel behaves as
> expected), but that module will probably not be removable anyway due to
> many references. To test this feature, check that the refcount of the
> nvme-core module is incremented when the delayed timer is active.
>
> Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>
> Reviewed-by: Nilay Shroff <nilay@linux.ibm.com>
> Signed-off-by: John Garry <john.g.garry@oracle.com>
I applied this patch. Thanks!
^ permalink raw reply
* [PATCH] blk-cgroup: Fix UAF in blkcg_activate_policy() by using blkg_tryget()
From: Zizhi Wo @ 2026-05-15 6:15 UTC (permalink / raw)
To: axboe, tj, josef, yukuai, linux-block
Cc: cgroups, yangerkun, chengzhihao1, wozizhi
[BUG]
Our fuzz testing triggered a blkg use-after-free issue:
BUG: KASAN: slab-use-after-free in percpu_ref_put_many.constprop.0+0xbe/0xe0
Call Trace:
...
blkcg_activate_policy+0x347/0xfa0
bfq_create_group_hierarchy+0x5b/0x140
bfq_init_queue+0xc1b/0x1470
? mutex_init_generic+0x9f/0x100
? elevator_alloc+0x166/0x2b0
blk_mq_init_sched+0x2b0/0x730
elevator_switch+0x188/0x450
elevator_change+0x290/0x470
elv_iosched_store+0x30a/0x3a0
...
[CAUSE]
process1 process2
cgroup_rmdir
...
blkcg_destroy_blkgs
spin_trylock(&q->queue_lock)
blkg_destroy
percpu_ref_kill(&blkg->refcnt)
...
blkg_free
INIT_WORK(xxx, blkg_free_workfn)
schedule_work
spin_unlock(&q->queue_lock)
====================================schedule_work
blkg_free_workfn
elevator_change
...
bfq_create_group_hierarchy
blkcg_activate_policy
spin_lock_irq(&q->queue_lock)
blkg_get // get dead ref !!
pinned_blkg = blkg
spin_unlock_irq(&q->queue_lock)
spin_lock_irq(&q->queue_lock)
list_del_init(&blkg->q_node)
spin_unlock_irq(&q->queue_lock)
kfree(blkg)
blkg_put(pinned_blkg) // UAF !!
A blkg killed by blkg_destroy() stays on q->blkg_list until
blkg_free_workfn() grabs queue_lock and unlinks it. blkg_get() on a dead
percpu_ref does not resurrect the blkg, so the later blkg_put() hits freed
memory and triggers this issue.
[Fix]
Replace blkg_get() with blkg_tryget(), which fails on a dead ref and lets
the loop skip dying blkgs.
Also hoist the ref acquisition to the top of the loop so dying blkgs are
filtered out before a pd is allocated and attached. Otherwise a pd attached
to an already-destroyed blkg would never called pd_offline_fn().
Fixes: 9d179b865449 ("blkcg: Fix multiple bugs in blkcg_activate_policy()")
Signed-off-by: Zizhi Wo <wozizhi@huaweicloud.com>
---
block/blk-cgroup.c | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c
index 554c87bb4a86..03b6ce934848 100644
--- a/block/blk-cgroup.c
+++ b/block/blk-cgroup.c
@@ -1621,6 +1621,10 @@ int blkcg_activate_policy(struct gendisk *disk, const struct blkcg_policy *pol)
if (blkg->pd[pol->plid])
continue;
+ /* a destroyed blkg may still be on q->blkg_list; skip it via tryget */
+ if (!blkg_tryget(blkg))
+ continue;
+
/* If prealloc matches, use it; otherwise try GFP_NOWAIT */
if (blkg == pinned_blkg) {
pd = pd_prealloc;
@@ -1637,7 +1641,6 @@ int blkcg_activate_policy(struct gendisk *disk, const struct blkcg_policy *pol)
*/
if (pinned_blkg)
blkg_put(pinned_blkg);
- blkg_get(blkg);
pinned_blkg = blkg;
spin_unlock_irq(&q->queue_lock);
@@ -1666,6 +1669,8 @@ int blkcg_activate_policy(struct gendisk *disk, const struct blkcg_policy *pol)
pd->online = true;
spin_unlock(&blkg->blkcg->lock);
+
+ blkg_put(blkg);
}
__set_bit(pol->plid, q->blkcg_pols);
--
2.52.0
^ permalink raw reply related
* Re: [PATCH] sched: flush plug in schedule_preempt_disabled() to prevent deadlock
From: Xiaosen @ 2026-05-15 6:18 UTC (permalink / raw)
To: Ming Lei, Peter Zijlstra
Cc: Tejun Heo, Jens Axboe, linux-block, linux-kernel, Ingo Molnar,
Juri Lelli, Vincent Guittot, Michael Wu, Thomas Gleixner
In-Reply-To: <agQxg-pBzF8rp1Ii@fedora>
https://lore.kernel.org/lkml/20260427183848.698551-2-jstultz@google.com/
The above change can resolve the deadlock I reported before by setting
task's state to TASK_RUNNING before switching context.
There is the likely alternative fix below.
https://lore.kernel.org/lkml/20260512025635.2840817-1-jstultz@google.com/
Regards,
Xiaosen
On 5/13/2026 4:08 PM, Ming Lei wrote:
> On Wed, May 13, 2026 at 09:30:39AM +0200, Peter Zijlstra wrote:
>> On Wed, May 13, 2026 at 10:07:03AM +0800, Ming Lei wrote:
>>> On Tue, May 12, 2026 at 07:16:36AM -1000, Tejun Heo wrote:
>>>> Hello, Ming.
>>>>
>>>> On Tue, May 12, 2026 at 11:45:14PM +0800, Ming Lei wrote:
>>>>> On Tue, May 12, 2026 at 02:40:21PM +0200, Peter Zijlstra wrote:
>>>>>> On Tue, May 12, 2026 at 02:04:32PM +0200, Peter Zijlstra wrote:
>>>>>>> On Tue, May 12, 2026 at 04:59:39PM +0800, Ming Lei wrote:
>>>>>>>> On preemptible kernels, a deadlock can occur when a task with plugged IO
>>>>>>>> calls schedule_preempt_disabled():
>>>>>>>>
>>>>>>>> schedule_preempt_disabled()
>>>>>>>> sched_preempt_enable_no_resched() // preemption now enabled
>>>>>>>> schedule() // <-- preemption can happen here
>>>>>>>> sched_submit_work()
>>>>>>>> blk_flush_plug()
>>>>>>>>
>>>>>>>> After sched_preempt_enable_no_resched() re-enables preemption, the task
>>>>>>>> can be preempted (e.g., by a higher-priority RT task) before reaching
>>>>>>>> blk_flush_plug() in sched_submit_work(). Since the task's state is
>>>>>>>> already TASK_UNINTERRUPTIBLE (set by the mutex/rwsem slowpath caller),
>>>>>>>> requests in current->plug remain unflushed for an unbounded time.
>>>>>>>>
>>>>>>>> If another task depends on those plugged requests to make progress (e.g.,
>>>>>>>> to release a lock the sleeping task needs), a deadlock results:
>>>>>>>>
>>>>>>>> - Task A (writeback worker): holds plugged IO, preempted before
>>>>>>>> flushing, stuck on run queue behind higher-priority work
>>>>>>>> - Task B: waiting for IO completion from Task A's plug, holds a lock
>>>>>>>> that Task A needs to be woken up
>>>>
>>>> My memory is hazy around io_schedule but the above reads really weird to me.
>>>> A task, regardless of its current state stays on the runqueue when
>>>> preempted, so the condition is temporary. As soon as the preempted task can
>>>> get CPU, it should unwind the situation. That's not a deadlock. Is the
>>>> problem that there can be preemption-induced delay in flushing the plugs?
>>>
>>> IMO, preempting a `!TASK_RUNNING` task can be thought as effective sleep,
>>
>> No it cannot be. Preemption ignores task state.
>
> Yeah, I get similar conclusion too with AI's assistance.
>
> But both two reports show that the preempted task aren't switched back for
> long enough time, can you share any idea for Michael & Xiaosen to investigate
> further from scheduler side?
>
> https://lore.kernel.org/linux-block/20260417082744.30124-1-michael@allwinnertech.com/
>
> https://lore.kernel.org/linux-block/5660795d-87de-46f5-add4-7729a02225ef@oss.qualcomm.com/
>
>
> Thanks,
> Ming
^ permalink raw reply
* [PATCH blktests] nbd/rc: redirect nbd-server stderr
From: Shin'ichiro Kawasaki @ 2026-05-15 5:30 UTC (permalink / raw)
To: linux-block, nbd; +Cc: Shin'ichiro Kawasaki
Recent nbd commit da5e07c057ab ("Reimplement daemonize() without using
daemon()") changed how nbd-server daemonizes itself. After this commit,
nbd-server prints messages to stderr after daemonization. This caused
nbd test cases to fail due to unexpected stderr output.
nbd/001 (resize a connected nbd device) [failed]
runtime 0.865s ... 0.883s
--- tests/nbd/001.out 2025-04-22 13:13:27.727873155 +0900
+++ /home/shin/Blktests/blktests/results/nodev/nbd/001.out.bad 2026-05-15 11:44:46.269000000 +0900
@@ -1,4 +1,5 @@
Running nbd/001
+Error: Session terminated by client
Disk /dev/nbd0: 10.7GB
nbd0 43:0 0 10G 0 disk
Setting size to 1gib
The commit is in nbd version v3.27, which is included in Fedora 44.
To avoid the failures, redirect nbd-server stderr to .full files.
Link: https://github.com/NetworkBlockDevice/nbd/commit/da5e07c057ab
Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
---
tests/nbd/rc | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tests/nbd/rc b/tests/nbd/rc
index e200ba6..6205c49 100644
--- a/tests/nbd/rc
+++ b/tests/nbd/rc
@@ -72,7 +72,7 @@ allowlist=true
[export]
exportname=${TMPDIR}/export
EOF
- nbd-server -p "${TMPDIR}/nbd.pid" -C "${TMPDIR}/nbd.conf"
+ nbd-server -p "${TMPDIR}/nbd.pid" -C "${TMPDIR}/nbd.conf" 2> "$FULL"
# Wait for nbd-server start listening the port
for ((i = 0; i < 100; i++)); do
--
2.54.0
^ permalink raw reply related
* [PATCH 5/5] block: unexport bio_{set,check}_pages_dirty
From: Christoph Hellwig @ 2026-05-15 4:55 UTC (permalink / raw)
To: Jens Axboe; +Cc: linux-block
In-Reply-To: <20260515045547.3790129-1-hch@lst.de>
Only used in built-in code.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
block/bio.c | 2 --
1 file changed, 2 deletions(-)
diff --git a/block/bio.c b/block/bio.c
index 57d5a87b3e2f..2d880d1255fe 100644
--- a/block/bio.c
+++ b/block/bio.c
@@ -1650,7 +1650,6 @@ void bio_set_pages_dirty(struct bio *bio)
folio_unlock(fi.folio);
}
}
-EXPORT_SYMBOL_GPL(bio_set_pages_dirty);
/*
* bio_check_pages_dirty() will check that all the BIO's pages are still dirty.
@@ -1709,7 +1708,6 @@ void bio_check_pages_dirty(struct bio *bio)
spin_unlock_irqrestore(&bio_dirty_lock, flags);
schedule_work(&bio_dirty_work);
}
-EXPORT_SYMBOL_GPL(bio_check_pages_dirty);
static inline bool bio_remaining_done(struct bio *bio)
{
--
2.53.0
^ permalink raw reply related
* [PATCH 4/5] block: unexport blk_status_to_str
From: Christoph Hellwig @ 2026-05-15 4:55 UTC (permalink / raw)
To: Jens Axboe; +Cc: linux-block
In-Reply-To: <20260515045547.3790129-1-hch@lst.de>
Only used in core block code, so unexport and move the prototype to
blk.h.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
block/blk-core.c | 1 -
block/blk.h | 2 ++
include/linux/blkdev.h | 1 -
3 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/block/blk-core.c b/block/blk-core.c
index d7de87e86994..22af5dec112b 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -197,7 +197,6 @@ const char *blk_status_to_str(blk_status_t status)
return "<null>";
return blk_errors[idx].name;
}
-EXPORT_SYMBOL_GPL(blk_status_to_str);
/**
* blk_sync_queue - cancel any pending callbacks on a queue
diff --git a/block/blk.h b/block/blk.h
index b998a7761faf..bf1a80493ff1 100644
--- a/block/blk.h
+++ b/block/blk.h
@@ -49,6 +49,8 @@ struct blk_flush_queue *blk_alloc_flush_queue(int node, int cmd_size,
gfp_t flags);
void blk_free_flush_queue(struct blk_flush_queue *q);
+const char *blk_status_to_str(blk_status_t status);
+
bool __blk_mq_unfreeze_queue(struct request_queue *q, bool force_atomic);
bool blk_queue_start_drain(struct request_queue *q);
bool __blk_freeze_queue_start(struct request_queue *q,
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index 890128cdea1c..17270a28c66d 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -1040,7 +1040,6 @@ extern const char *blk_op_str(enum req_op op);
int blk_status_to_errno(blk_status_t status);
blk_status_t errno_to_blk_status(int errno);
-const char *blk_status_to_str(blk_status_t status);
/* only poll the hardware once, don't continue until a completion was found */
#define BLK_POLL_ONESHOT (1 << 0)
--
2.53.0
^ permalink raw reply related
* [PATCH 3/5] block: unexport blk_io_schedule
From: Christoph Hellwig @ 2026-05-15 4:55 UTC (permalink / raw)
To: Jens Axboe; +Cc: linux-block
In-Reply-To: <20260515045547.3790129-1-hch@lst.de>
Only used in built-in code.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
block/blk-core.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/block/blk-core.c b/block/blk-core.c
index 17450058ea6d..d7de87e86994 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -1270,7 +1270,6 @@ void blk_io_schedule(void)
else
io_schedule();
}
-EXPORT_SYMBOL_GPL(blk_io_schedule);
int __init blk_dev_init(void)
{
--
2.53.0
^ permalink raw reply related
* [PATCH 2/5] block: remove bio_copy_data_iter
From: Christoph Hellwig @ 2026-05-15 4:55 UTC (permalink / raw)
To: Jens Axboe; +Cc: linux-block
In-Reply-To: <20260515045547.3790129-1-hch@lst.de>
Only used by bio_copy_data, so implement that directly.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
block/bio.c | 36 +++++++++++++++---------------------
include/linux/bio.h | 2 --
2 files changed, 15 insertions(+), 23 deletions(-)
diff --git a/block/bio.c b/block/bio.c
index b990c453d72f..57d5a87b3e2f 100644
--- a/block/bio.c
+++ b/block/bio.c
@@ -1575,26 +1575,6 @@ void __bio_advance(struct bio *bio, unsigned bytes)
}
EXPORT_SYMBOL(__bio_advance);
-void bio_copy_data_iter(struct bio *dst, struct bvec_iter *dst_iter,
- struct bio *src, struct bvec_iter *src_iter)
-{
- while (src_iter->bi_size && dst_iter->bi_size) {
- struct bio_vec src_bv = bio_iter_iovec(src, *src_iter);
- struct bio_vec dst_bv = bio_iter_iovec(dst, *dst_iter);
- unsigned int bytes = min(src_bv.bv_len, dst_bv.bv_len);
- void *src_buf = bvec_kmap_local(&src_bv);
- void *dst_buf = bvec_kmap_local(&dst_bv);
-
- memcpy(dst_buf, src_buf, bytes);
-
- kunmap_local(dst_buf);
- kunmap_local(src_buf);
-
- bio_advance_iter_single(src, src_iter, bytes);
- bio_advance_iter_single(dst, dst_iter, bytes);
- }
-}
-EXPORT_SYMBOL(bio_copy_data_iter);
/**
* bio_copy_data - copy contents of data buffers from one bio to another
@@ -1609,7 +1589,21 @@ void bio_copy_data(struct bio *dst, struct bio *src)
struct bvec_iter src_iter = src->bi_iter;
struct bvec_iter dst_iter = dst->bi_iter;
- bio_copy_data_iter(dst, &dst_iter, src, &src_iter);
+ while (src_iter.bi_size && dst_iter.bi_size) {
+ struct bio_vec src_bv = bio_iter_iovec(src, src_iter);
+ struct bio_vec dst_bv = bio_iter_iovec(dst, dst_iter);
+ unsigned int bytes = min(src_bv.bv_len, dst_bv.bv_len);
+ void *src_buf = bvec_kmap_local(&src_bv);
+ void *dst_buf = bvec_kmap_local(&dst_bv);
+
+ memcpy(dst_buf, src_buf, bytes);
+
+ kunmap_local(dst_buf);
+ kunmap_local(src_buf);
+
+ bio_advance_iter_single(src, &src_iter, bytes);
+ bio_advance_iter_single(dst, &dst_iter, bytes);
+ }
}
EXPORT_SYMBOL(bio_copy_data);
diff --git a/include/linux/bio.h b/include/linux/bio.h
index 84643fc0fb08..85463981d0f5 100644
--- a/include/linux/bio.h
+++ b/include/linux/bio.h
@@ -478,8 +478,6 @@ extern void bio_check_pages_dirty(struct bio *bio);
int bio_iov_iter_bounce(struct bio *bio, struct iov_iter *iter, size_t maxlen);
void bio_iov_iter_unbounce(struct bio *bio, bool is_error, bool mark_dirty);
-extern void bio_copy_data_iter(struct bio *dst, struct bvec_iter *dst_iter,
- struct bio *src, struct bvec_iter *src_iter);
extern void bio_copy_data(struct bio *dst, struct bio *src);
extern void bio_free_pages(struct bio *bio);
void zero_fill_bio(struct bio *bio);
--
2.53.0
^ permalink raw reply related
* [PATCH 1/5] block: remove zero_fill_bio_iter
From: Christoph Hellwig @ 2026-05-15 4:55 UTC (permalink / raw)
To: Jens Axboe; +Cc: linux-block
In-Reply-To: <20260515045547.3790129-1-hch@lst.de>
Only used to implement zero_fill_bio, so directly implement that.
Signed-off-by: Christoph Hellwig <hch@lst.de>
---
block/bio.c | 6 +++---
include/linux/bio.h | 7 +------
2 files changed, 4 insertions(+), 9 deletions(-)
diff --git a/block/bio.c b/block/bio.c
index b8972dba68a0..b990c453d72f 100644
--- a/block/bio.c
+++ b/block/bio.c
@@ -635,15 +635,15 @@ struct bio *bio_kmalloc(unsigned short nr_vecs, gfp_t gfp_mask)
}
EXPORT_SYMBOL(bio_kmalloc);
-void zero_fill_bio_iter(struct bio *bio, struct bvec_iter start)
+void zero_fill_bio(struct bio *bio)
{
struct bio_vec bv;
struct bvec_iter iter;
- __bio_for_each_segment(bv, bio, iter, start)
+ bio_for_each_segment(bv, bio, iter)
memzero_bvec(&bv);
}
-EXPORT_SYMBOL(zero_fill_bio_iter);
+EXPORT_SYMBOL(zero_fill_bio);
/**
* bio_truncate - truncate the bio to small size of @new_size
diff --git a/include/linux/bio.h b/include/linux/bio.h
index 97d747320b35..84643fc0fb08 100644
--- a/include/linux/bio.h
+++ b/include/linux/bio.h
@@ -482,13 +482,8 @@ extern void bio_copy_data_iter(struct bio *dst, struct bvec_iter *dst_iter,
struct bio *src, struct bvec_iter *src_iter);
extern void bio_copy_data(struct bio *dst, struct bio *src);
extern void bio_free_pages(struct bio *bio);
+void zero_fill_bio(struct bio *bio);
void guard_bio_eod(struct bio *bio);
-void zero_fill_bio_iter(struct bio *bio, struct bvec_iter iter);
-
-static inline void zero_fill_bio(struct bio *bio)
-{
- zero_fill_bio_iter(bio, bio->bi_iter);
-}
static inline void bio_release_pages(struct bio *bio, bool mark_dirty)
{
--
2.53.0
^ permalink raw reply related
* remove dead code and export
From: Christoph Hellwig @ 2026-05-15 4:55 UTC (permalink / raw)
To: Jens Axboe; +Cc: linux-block
Hi Jens,
this series removes some minor dead code and dead exports in the block code.
Diffstat:
block/bio.c | 44 ++++++++++++++++++--------------------------
block/blk-core.c | 2 --
block/blk.h | 2 ++
include/linux/bio.h | 9 +--------
include/linux/blkdev.h | 1 -
5 files changed, 21 insertions(+), 37 deletions(-)
^ permalink raw reply
* Re: [PATCH V4 0/3] md/nvme: Enable PCI P2PDMA support for RAID0 and NVMe Multipath
From: Christoph Hellwig @ 2026-05-15 4:35 UTC (permalink / raw)
To: Chaitanya Kulkarni
Cc: song, yukuai, linan122, kbusch, axboe, sagi, linux-block,
linux-raid, linux-nvme, kmodukuri
In-Reply-To: <20260513185153.95552-1-kch@nvidia.com>
Still looks good to me as per the reviews.
^ permalink raw reply
* Re: fix block layer bounce buffering for block size > PAGE_SIZE v2
From: Christoph Hellwig @ 2026-05-15 4:28 UTC (permalink / raw)
To: Jens Axboe
Cc: Christoph Hellwig, Christian Brauner, Darrick J. Wong,
Pankaj Raghav, linux-block, linux-xfs, linux-fsdevel
In-Reply-To: <30acedd3-e994-4810-8952-613b7d0bd9ca@kernel.dk>
On Wed, May 13, 2026 at 01:56:09PM -0600, Jens Axboe wrote:
> > Can we get some Fixes tags on these, please?
>
> Looked myself to not delay this any further, both marked with:
>
> Fixes: 8dd5e7c75d7b ("block: add helpers to bounce buffer an iov_iter into bios")
Sorry. The tag looks correct.
^ permalink raw reply
* Re: [PATCH v6 1/4] block: add task-context bio completion infrastructure
From: Hillf Danton @ 2026-05-15 2:38 UTC (permalink / raw)
To: Tal Zussman
Cc: Matthew Wilcox (Oracle), Christoph Hellwig, linux-block,
linux-kernel
In-Reply-To: <20260514-blk-dontcache-v6-1-782e2fa7477b@columbia.edu>
On Thu, 14 May 2026 17:51:14 -0400 Tal Zussman wrote:
> +
> +static void bio_complete_work_fn(struct work_struct *w)
> +{
> + struct delayed_work *dw = to_delayed_work(w);
> + struct bio_complete_batch *batch =
> + container_of(dw, struct bio_complete_batch, work);
> +
> + while (1) {
> + struct bio_list list;
> + struct bio *bio;
> +
> + local_lock_irq(&bio_complete_batch.lock);
> + list = batch->list;
> + bio_list_init(&batch->list);
> + local_unlock_irq(&bio_complete_batch.lock);
> +
> + if (bio_list_empty(&list))
> + break;
> +
> + while ((bio = bio_list_pop(&list)))
> + bio->bi_end_io(bio);
> +
> + if (need_resched()) {
> + bool is_empty;
> +
Checking resched is not needed as workqueue worker can be preempted
while processing bios．Given batch and delayed work, I suspect completing
more than batch, the bios accumulated within a jiff, makes sense.
> + local_lock_irq(&bio_complete_batch.lock);
> + is_empty = bio_list_empty(&batch->list);
> + local_unlock_irq(&bio_complete_batch.lock);
> + if (!is_empty)
> + mod_delayed_work_on(batch->cpu,
> + bio_complete_wq,
> + &batch->work, 0);
> + break;
> + }
> + }
> +}
> +
^ permalink raw reply
* [PATCH v2] loop: Fix NULL pointer dereference by synchronizing lo_release and loop_queue_rq
From: Tetsuo Handa @ 2026-05-15 1:38 UTC (permalink / raw)
To: Bart Van Assche, Jens Axboe, Christoph Hellwig, Damien Le Moal
Cc: linux-block, LKML, Andrew Morton
In-Reply-To: <af2205e3-205e-4743-a767-8593c2c4747a@I-love.SAKURA.ne.jp>
The loop driver relies on lo_release() to automatically clear the loop
device via __loop_clr_fd() when the last file descriptor is closed
(LO_FLAGS_AUTOCLEAR). Although the backing file structure itself remains
allocated in memory thanks to proper file reference counting (f_count is
not zero), a severe race condition exists regarding the visibility of
the lo->lo_backing_file pointer.
This race window was exposed by commit 65565ca5f99b ("block: unify
the synchronous bi_end_io callbacks"). By unifying and optimizing
the synchronous I/O completion path, the timing and scheduling behavior of
the block layer altered significantly.
As a result, a highly-concurrent execution pipeline emerged where
lo_release() can progress to __loop_clr_fd() and nullify
lo->lo_backing_file while an already-scheduled asynchronous I/O work
(lo_rw_aio) is just about to be executed by a kworker thread.
Since the kworker enters lo_rw_aio() after lo->lo_backing_file has been
cleared, it attempts to dereference the now-NULL pointer when initializing
the kiocb, leading to the reported NULL pointer dereference bug.
To close this race safely without introducing heavy fast-path checks,
we must ensure that any running or scheduled dispatch threads have
completed before we nullify the pointer. Since loop_queue_rq() operates
within the block layer's RCU read-side critical section, invoke
synchronize_rcu() and drain_workqueue() in __loop_clr_fd() prior to
clearing lo->lo_backing_file.
Reported-by: syzbot+cd8a9a308e879a4e2c28@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=cd8a9a308e879a4e2c28
Reported-by: syzbot+bc273027d5643e48e5b3@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=bc273027d5643e48e5b3
Analyzed-by: AI Mode in Google Search (no mail address)
Signed-off-by: Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
---
drivers/block/loop.c | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/drivers/block/loop.c b/drivers/block/loop.c
index 0000913f7efc..ff117f340b2f 100644
--- a/drivers/block/loop.c
+++ b/drivers/block/loop.c
@@ -1118,6 +1118,17 @@ static void __loop_clr_fd(struct loop_device *lo)
struct file *filp;
gfp_t gfp = lo->old_gfp_mask;
+ /*
+ * Now that loop_queue_rq() sees lo->lo_state != Lo_bound,
+ * wait for already started loop_queue_rq() to complete.
+ */
+ synchronize_rcu();
+ /*
+ * Now that no more works are scheduled by loop_queue_rq(),
+ * wait for already scheduled works to complete.
+ */
+ drain_workqueue(lo->workqueue);
+
spin_lock_irq(&lo->lo_lock);
filp = lo->lo_backing_file;
lo->lo_backing_file = NULL;
--
2.54.0
^ permalink raw reply related
* Re: [PATCH] Re:[PATCH v3] zram: fix use-after-free in zram_writeback_endio
From: Minchan Kim @ 2026-05-14 22:02 UTC (permalink / raw)
To: wang wei
Cc: richardycc, akpm, axboe, bgeffon, linux-block, linux-kernel,
linux-mm, liumartin, senozhatsky, stable
In-Reply-To: <20260513140218.7425-1-a929244872@163.com>
On Wed, May 13, 2026 at 10:02:18PM +0800, wang wei wrote:
> >@@ -847,7 +849,7 @@ static void release_wb_ctl(struct zram_wb_ctl *wb_ctl)
> > release_wb_req(req);
> > }
> >
> >- kfree(wb_ctl);
> >+ kfree_rcu(wb_ctl, rcu);
> > }
>
> Do we need to add a 'rcu_assign_pointer(wb_ctl, NULL);' before 'kfree_rcu(wb_ctl, rcu)'?
>
> Signed-off-by: wang wei <a929244872@163.com>
Why do we need it?
My understanding is rcu_assign_pointer() is typically used to publish NULL to
a shared pointer variable so that future RCU readers (using rcu_dereference)
won't access the object before kfree_rcu().
However, in our case, wb_ctl is not stored in any shared pointer variable.
It is a local variable in writeback_store() and RCU readers (zram_writeback_endio)
do not look up wb_ctl from a shared pointer. They obtain it directly from
bio->bi_private of the specific bio they are completing.
Please let me know if I missed anything.
^ permalink raw reply
* [PATCH v6 4/4] block: enable RWF_DONTCACHE for block devices
From: Tal Zussman @ 2026-05-14 21:51 UTC (permalink / raw)
To: Jens Axboe, Matthew Wilcox (Oracle), Christian Brauner,
Darrick J. Wong, Carlos Maiolino, Alexander Viro, Jan Kara,
Christoph Hellwig
Cc: Dave Chinner, Bart Van Assche, linux-block, linux-kernel,
linux-xfs, linux-fsdevel, linux-mm, Gao Xiang, Tal Zussman
In-Reply-To: <20260514-blk-dontcache-v6-0-782e2fa7477b@columbia.edu>
Block device buffered reads and writes already pass through
filemap_read() and iomap_file_buffered_write() respectively, both of
which handle IOCB_DONTCACHE. Enable RWF_DONTCACHE for block device files
by setting FOP_DONTCACHE in def_blk_fops.
For CONFIG_BUFFER_HEAD=y paths, use block_write_begin_iocb() in
blkdev_write_begin() to thread the kiocb through so that buffer_head
writeback gets dropbehind support.
CONFIG_BUFFER_HEAD=n paths are handled by the previously added iomap
BIO_COMPLETE_IN_TASK support.
This support is useful for databases that operate on raw block devices,
among other userspace applications.
Signed-off-by: Tal Zussman <tz2294@columbia.edu>
---
block/fops.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/block/fops.c b/block/fops.c
index bb6642b45937..31b073181d87 100644
--- a/block/fops.c
+++ b/block/fops.c
@@ -504,7 +504,8 @@ static int blkdev_write_begin(const struct kiocb *iocb,
unsigned len, struct folio **foliop,
void **fsdata)
{
- return block_write_begin(mapping, pos, len, foliop, blkdev_get_block);
+ return block_write_begin_iocb(iocb, mapping, pos, len, foliop,
+ blkdev_get_block);
}
static int blkdev_write_end(const struct kiocb *iocb,
@@ -966,7 +967,7 @@ const struct file_operations def_blk_fops = {
.splice_write = iter_file_splice_write,
.fallocate = blkdev_fallocate,
.uring_cmd = blkdev_uring_cmd,
- .fop_flags = FOP_BUFFER_RASYNC,
+ .fop_flags = FOP_BUFFER_RASYNC | FOP_DONTCACHE,
};
static __init int blkdev_init(void)
--
2.39.5
^ permalink raw reply related
* [PATCH v6 3/4] buffer: add dropbehind writeback support
From: Tal Zussman @ 2026-05-14 21:51 UTC (permalink / raw)
To: Jens Axboe, Matthew Wilcox (Oracle), Christian Brauner,
Darrick J. Wong, Carlos Maiolino, Alexander Viro, Jan Kara,
Christoph Hellwig
Cc: Dave Chinner, Bart Van Assche, linux-block, linux-kernel,
linux-xfs, linux-fsdevel, linux-mm, Gao Xiang, Tal Zussman
In-Reply-To: <20260514-blk-dontcache-v6-0-782e2fa7477b@columbia.edu>
Add block_write_begin_iocb() which threads the kiocb through to
__filemap_get_folio() so that buffer_head-based I/O can use DONTCACHE
behavior. When the iocb has IOCB_DONTCACHE set, FGP_DONTCACHE is
passed to mark the folio for dropbehind. The existing
block_write_begin() is preserved as a wrapper that passes a NULL iocb.
Set BIO_COMPLETE_IN_TASK in submit_bh_wbc() when the folio has
dropbehind set, so that buffer_head writeback completions get deferred
to task context.
Signed-off-by: Tal Zussman <tz2294@columbia.edu>
---
fs/buffer.c | 19 +++++++++++++++++--
include/linux/buffer_head.h | 3 +++
2 files changed, 20 insertions(+), 2 deletions(-)
diff --git a/fs/buffer.c b/fs/buffer.c
index b0b3792b1496..d0abaf44d782 100644
--- a/fs/buffer.c
+++ b/fs/buffer.c
@@ -2138,14 +2138,19 @@ EXPORT_SYMBOL(block_commit_write);
*
* The filesystem needs to handle block truncation upon failure.
*/
-int block_write_begin(struct address_space *mapping, loff_t pos, unsigned len,
+int block_write_begin_iocb(const struct kiocb *iocb,
+ struct address_space *mapping, loff_t pos, unsigned len,
struct folio **foliop, get_block_t *get_block)
{
pgoff_t index = pos >> PAGE_SHIFT;
+ fgf_t fgp_flags = FGP_WRITEBEGIN;
struct folio *folio;
int status;
- folio = __filemap_get_folio(mapping, index, FGP_WRITEBEGIN,
+ if (iocb && iocb->ki_flags & IOCB_DONTCACHE)
+ fgp_flags |= FGP_DONTCACHE;
+
+ folio = __filemap_get_folio(mapping, index, fgp_flags,
mapping_gfp_mask(mapping));
if (IS_ERR(folio))
return PTR_ERR(folio);
@@ -2160,6 +2165,13 @@ int block_write_begin(struct address_space *mapping, loff_t pos, unsigned len,
*foliop = folio;
return status;
}
+
+int block_write_begin(struct address_space *mapping, loff_t pos, unsigned len,
+ struct folio **foliop, get_block_t *get_block)
+{
+ return block_write_begin_iocb(NULL, mapping, pos, len, foliop,
+ get_block);
+}
EXPORT_SYMBOL(block_write_begin);
int block_write_end(loff_t pos, unsigned len, unsigned copied,
@@ -2715,6 +2727,9 @@ static void submit_bh_wbc(blk_opf_t opf, struct buffer_head *bh,
bio = bio_alloc(bh->b_bdev, 1, opf, GFP_NOIO);
+ if (folio_test_dropbehind(bh->b_folio))
+ bio_set_flag(bio, BIO_COMPLETE_IN_TASK);
+
if (IS_ENABLED(CONFIG_FS_ENCRYPTION))
buffer_set_crypto_ctx(bio, bh, GFP_NOIO);
diff --git a/include/linux/buffer_head.h b/include/linux/buffer_head.h
index e4939e33b4b5..4ce50882d621 100644
--- a/include/linux/buffer_head.h
+++ b/include/linux/buffer_head.h
@@ -260,6 +260,9 @@ int block_read_full_folio(struct folio *, get_block_t *);
bool block_is_partially_uptodate(struct folio *, size_t from, size_t count);
int block_write_begin(struct address_space *mapping, loff_t pos, unsigned len,
struct folio **foliop, get_block_t *get_block);
+int block_write_begin_iocb(const struct kiocb *iocb,
+ struct address_space *mapping, loff_t pos, unsigned len,
+ struct folio **foliop, get_block_t *get_block);
int __block_write_begin(struct folio *folio, loff_t pos, unsigned len,
get_block_t *get_block);
int block_write_end(loff_t pos, unsigned len, unsigned copied, struct folio *);
--
2.39.5
^ permalink raw reply related
* [PATCH v6 2/4] iomap: use BIO_COMPLETE_IN_TASK for dropbehind writeback
From: Tal Zussman @ 2026-05-14 21:51 UTC (permalink / raw)
To: Jens Axboe, Matthew Wilcox (Oracle), Christian Brauner,
Darrick J. Wong, Carlos Maiolino, Alexander Viro, Jan Kara,
Christoph Hellwig
Cc: Dave Chinner, Bart Van Assche, linux-block, linux-kernel,
linux-xfs, linux-fsdevel, linux-mm, Gao Xiang, Tal Zussman
In-Reply-To: <20260514-blk-dontcache-v6-0-782e2fa7477b@columbia.edu>
Set BIO_COMPLETE_IN_TASK on iomap writeback bios when a dropbehind folio
is added. This ensures that bi_end_io runs in task context, where
folio_end_dropbehind() can safely invalidate folios.
With the bio layer now handling task-context deferral generically,
IOMAP_IOEND_DONTCACHE is no longer needed, as XFS no longer needs to
route DONTCACHE ioends through its completion workqueue. Remove the flag
and its NOMERGE entry.
Without the NOMERGE, regular I/Os that get merged with a dropbehind
folio will also have their completion deferred to task context.
Signed-off-by: Tal Zussman <tz2294@columbia.edu>
---
fs/iomap/ioend.c | 5 +++--
fs/xfs/xfs_aops.c | 4 ----
include/linux/iomap.h | 5 +----
3 files changed, 4 insertions(+), 10 deletions(-)
diff --git a/fs/iomap/ioend.c b/fs/iomap/ioend.c
index acf3cf98b23a..892dbfc77ae9 100644
--- a/fs/iomap/ioend.c
+++ b/fs/iomap/ioend.c
@@ -237,8 +237,6 @@ ssize_t iomap_add_to_ioend(struct iomap_writepage_ctx *wpc, struct folio *folio,
if (wpc->iomap.flags & IOMAP_F_SHARED)
ioend_flags |= IOMAP_IOEND_SHARED;
- if (folio_test_dropbehind(folio))
- ioend_flags |= IOMAP_IOEND_DONTCACHE;
if (pos == wpc->iomap.offset && (wpc->iomap.flags & IOMAP_F_BOUNDARY))
ioend_flags |= IOMAP_IOEND_BOUNDARY;
@@ -255,6 +253,9 @@ ssize_t iomap_add_to_ioend(struct iomap_writepage_ctx *wpc, struct folio *folio,
if (!bio_add_folio(&ioend->io_bio, folio, map_len, poff))
goto new_ioend;
+ if (folio_test_dropbehind(folio))
+ bio_set_flag(&ioend->io_bio, BIO_COMPLETE_IN_TASK);
+
/*
* Clamp io_offset and io_size to the incore EOF so that ondisk
* file size updates in the ioend completion are byte-accurate.
diff --git a/fs/xfs/xfs_aops.c b/fs/xfs/xfs_aops.c
index f279055fcea0..0dcf78beae8a 100644
--- a/fs/xfs/xfs_aops.c
+++ b/fs/xfs/xfs_aops.c
@@ -511,10 +511,6 @@ xfs_ioend_needs_wq_completion(
if (ioend->io_flags & (IOMAP_IOEND_UNWRITTEN | IOMAP_IOEND_SHARED))
return true;
- /* Page cache invalidation cannot be done in irq context. */
- if (ioend->io_flags & IOMAP_IOEND_DONTCACHE)
- return true;
-
return false;
}
diff --git a/include/linux/iomap.h b/include/linux/iomap.h
index 2c5685adf3a9..fef04e01116f 100644
--- a/include/linux/iomap.h
+++ b/include/linux/iomap.h
@@ -399,16 +399,13 @@ sector_t iomap_bmap(struct address_space *mapping, sector_t bno,
#define IOMAP_IOEND_BOUNDARY (1U << 2)
/* is direct I/O */
#define IOMAP_IOEND_DIRECT (1U << 3)
-/* is DONTCACHE I/O */
-#define IOMAP_IOEND_DONTCACHE (1U << 4)
/*
* Flags that if set on either ioend prevent the merge of two ioends.
* (IOMAP_IOEND_BOUNDARY also prevents merges, but only one-way)
*/
#define IOMAP_IOEND_NOMERGE_FLAGS \
- (IOMAP_IOEND_SHARED | IOMAP_IOEND_UNWRITTEN | IOMAP_IOEND_DIRECT | \
- IOMAP_IOEND_DONTCACHE)
+ (IOMAP_IOEND_SHARED | IOMAP_IOEND_UNWRITTEN | IOMAP_IOEND_DIRECT)
/*
* Structure for writeback I/O completions.
--
2.39.5
^ permalink raw reply related
* [PATCH v6 0/4] block: enable RWF_DONTCACHE for block devices
From: Tal Zussman @ 2026-05-14 21:51 UTC (permalink / raw)
To: Jens Axboe, Matthew Wilcox (Oracle), Christian Brauner,
Darrick J. Wong, Carlos Maiolino, Alexander Viro, Jan Kara,
Christoph Hellwig
Cc: Dave Chinner, Bart Van Assche, linux-block, linux-kernel,
linux-xfs, linux-fsdevel, linux-mm, Gao Xiang, Tal Zussman
Add support for using RWF_DONTCACHE with block devices.
Dropbehind pruning needs to be done in non-IRQ context, but block
devices complete writeback in IRQ context. To fix this, we defer
dropbehind invalidation to task context. Add infrastructure that lets
bi_end_io callbacks run from a worker, in two forms:
1. BIO_COMPLETE_IN_TASK, a bio flag the submitter sets when it knows
upfront that the callback needs task context, as in the dropbehind
writeback paths.
2. bio_complete_in_task(), a helper that callbacks can invoke from
bi_end_io() when the decision to defer is dynamic, as in iomap
fserror reporting.
These queue the bio to a per-CPU batch and schedule a delayed work item
to do bio completion.
Patch 1 adds the block layer task-context completion infrastructure,
with both the flag and the procedural helper. This builds on top of
suggestions by Matthew and Christoph: the procedural helper and
bio_in_atomic() come from Christoph's "bio completion in task
enhancements / experiments" series [1].
[Christoph, I put you down as Suggested-by for this patch. Let me know
if you'd like it to be Co-authored-by with your sign-off.]
Patch 2 wires BIO_COMPLETE_IN_TASK into iomap writeback for dropbehind
folios, removes IOMAP_IOEND_DONTCACHE, and removes the DONTCACHE
workqueue deferral from XFS.
Patch 3 sets up DONTCACHE support for buffer-head-based I/O by setting
BIO_COMPLETE_IN_TASK in submit_bh_wbc() for the CONFIG_BUFFER_HEAD
path.
Patch 4 enables RWF_DONTCACHE for block devices based on the previous
support. This support is useful for databases that operate on raw block
devices, among other userspace applications.
I tested this (with CONFIG_BUFFER_HEAD=y) for reads and writes on a
single block device on a VM, so results may be noisy.
Reads were tested on the root partition with a 45GB range (~2x RAM).
Writes were tested on a disabled swap parition (~1GB) in a memcg of size
244MB to force reclaim pressure.
Results:
===== READS (/dev/nvme0n1p2) =====
sec normal MB/s dontcache MB/s
---- ------------ --------------
1 1098.6 1609.0
2 1270.3 1506.6
3 1093.3 1576.5
4 1141.8 2393.9
5 1365.3 2793.8
6 1324.6 2065.9
7 879.6 1920.7
8 1434.1 1662.4
9 1184.9 1857.9
10 1166.4 1702.8
11 1161.4 1653.4
12 1086.9 1555.4
13 1198.5 1718.9
14 1111.9 1752.2
---- ------------ --------------
avg 1173.7 1828.8 (+56%)
==== WRITES (/dev/nvme0n1p3) =====
sec normal MB/s dontcache MB/s
---- ------------ --------------
1 692.4 9297.7
2 4810.8 9342.8
3 5221.7 2955.2
4 396.7 8488.3
5 7249.2 9249.3
6 6695.4 1376.2
7 122.9 9125.8
8 5486.5 9414.7
9 6921.5 8743.5
10 27.9 8997.8
---- ------------ --------------
avg 3762.5 7699.1 (+105%)
[1]: https://lore.kernel.org/all/20260409160243.1008358-1-hch@lst.de/
---
Changes in v6:
- Remove RFC tag
- Rebase on v7.1-rc3.
- 1/4: Revert to using a bio_list, per Jens.
- 1/4: Restructure and simplify work function loop.
- 1/4: Expose both the flag and procedural version, in order to allow
static and dynamic deferral decisions, per conversation with Matthew
and Christoph at LSFMM.
- 1/4: Use bio_in_atomic() predicate, per Christoph.
- 1/4: Use the CPU hot-unplug protocol from mm/vmstat.c, to take into
account use of delayed_work.
- 1/4: Mark the workqueue WQ_PERCPU.
- 1/4: Add comments.
- 3/4 and 4/4: Split into two patches, per Christoph.
- 3/4: Drop the cont_write_begin() change. Block devices don't go
through cont_write_begin(), so it was out of scope and was left over
from v1.
- Link to v5: https://lore.kernel.org/r/20260408-blk-dontcache-v5-0-0f080c20a96f@columbia.edu
Changes in v5:
- 1/3: Replace local_lock + bio_list with struct llist, per Dave.
- 1/3: Use delayed_work with 1-jiffie delay, per Dave.
- 1/3: Add dedicated workqueue to avoid deadlocks, per Christoph.
- 1/3: Restructure work function as do/while loop and only schedule work
originally when the list was previously empty, per Jens.
- 2/3: Delete IOMAP_IOEND_DONTCACHE and its NOMERGE entry, per Matthew
and Christoph.
- Link to v4: https://lore.kernel.org/r/20260325-blk-dontcache-v4-0-c4b56db43f64@columbia.edu
Changes in v4:
- 1/3: Move dropbehind deferral from folio-level to bio-level using
BIO_COMPLETE_IN_TASK, per Matthew and Jan.
- 1/3: Work function yields on need_resched() to avoid hogging the CPU,
per Jan.
- 2/3: New patch. Set BIO_COMPLETE_IN_TASK on iomap writeback bios for
DONTCACHE folios, removing the need for XFS-specific workqueue
deferral.
- 3/3: Set BIO_COMPLETE_IN_TASK in submit_bh_wbc() for buffer_head
path.
- 3/3: Update commit message to mention CONFIG_BUFFER_HEAD=n path.
- Link to v3: https://lore.kernel.org/r/20260227-blk-dontcache-v3-0-cd309ccd5868@columbia.edu
Changes in v3:
- 1/2: Convert dropbehind deferral to per-CPU folio_batches protected by
local_lock using per-CPU work items, to reduce contention, per Jens.
- 1/2: Call folio_end_dropbehind_irq() directly from
folio_end_writeback(), per Jens.
- 1/2: Add CPU hotplug dead callback to drain the departing CPU's folio
batch.
- 2/2: Introduce block_write_begin_iocb(), per Christoph.
- 2/2: Dropped R-b due to changes.
- Link to v2: https://lore.kernel.org/r/20260225-blk-dontcache-v2-0-70e7ac4f7108@columbia.edu
Changes in v2:
- Add R-b from Jan Kara for 2/2.
- Add patch to defer dropbehind completion from IRQ context via a work
item (1/2).
- Add initial performance numbers to cover letter.
- Link to v1: https://lore.kernel.org/r/20260218-blk-dontcache-v1-1-fad6675ef71f@columbia.edu
---
Tal Zussman (4):
block: add task-context bio completion infrastructure
iomap: use BIO_COMPLETE_IN_TASK for dropbehind writeback
buffer: add dropbehind writeback support
block: enable RWF_DONTCACHE for block devices
block/bio.c | 147 +++++++++++++++++++++++++++++++++++++++++++-
block/fops.c | 5 +-
fs/buffer.c | 19 +++++-
fs/iomap/ioend.c | 5 +-
fs/xfs/xfs_aops.c | 4 --
include/linux/bio.h | 32 ++++++++++
include/linux/blk_types.h | 1 +
include/linux/buffer_head.h | 3 +
include/linux/iomap.h | 5 +-
9 files changed, 206 insertions(+), 15 deletions(-)
---
base-commit: 695fee9be55747935d0a7b58f3d1fb83397a8b4f
change-id: 20260218-blk-dontcache-338133dd045e
Best regards,
--
Tal Zussman <tz2294@columbia.edu>
^ permalink raw reply
* [PATCH v6 1/4] block: add task-context bio completion infrastructure
From: Tal Zussman @ 2026-05-14 21:51 UTC (permalink / raw)
To: Jens Axboe, Matthew Wilcox (Oracle), Christian Brauner,
Darrick J. Wong, Carlos Maiolino, Alexander Viro, Jan Kara,
Christoph Hellwig
Cc: Dave Chinner, Bart Van Assche, linux-block, linux-kernel,
linux-xfs, linux-fsdevel, linux-mm, Gao Xiang, Tal Zussman
In-Reply-To: <20260514-blk-dontcache-v6-0-782e2fa7477b@columbia.edu>
Some bio completion handlers need to run from preemptible task context,
but bio_endio() may be called from IRQ context (e.g., buffer_head
writeback). Callers need a way to ensure their callback eventually runs
from a sleepable context. Add infrastructure for that, in two forms:
1. BIO_COMPLETE_IN_TASK, a bio flag the submitter sets when it knows
in advance that its callback needs task context (e.g., dropbehind
writeback). bio_endio() sees the flag and offloads completion to a
worker automatically.
2. bio_complete_in_task(), a helper that completion callbacks can
invoke from within bi_end_io() when the deferral decision is
dynamic (e.g., fserror reporting).
Both share a per-CPU batch list drained by a delayed work item on a
WQ_PERCPU workqueue. Producers push the bio onto the local CPU's batch
and schedule the work item, which then dispatches each bio's bi_end_io()
from task context. The delayed work item uses a 1-jiffie delay to allow
batches of completions to accumulate before processing.
Both methods are gated on bio_in_atomic(), which returns true in any
context where a sleeping bi_end_io() is unsafe, including
non-preemptible task context. This logic is copied from commit
c99fab6e80b7 ("erofs: fix atomic context detection when
!CONFIG_DEBUG_LOCK_ALLOC").
Two CPU hotplug callbacks are used to drain remaining bios from the
departing CPU's batch, while maintaining the per-CPU behavior. The
CPUHP_AP_ONLINE_DYN callback disables the per-CPU delayed work while the
CPU is still online, preventing it from running on an unbound worker
later. CPUHP_BP_PREPARE_DYN then drains any bios added between disabling
the work item and CPU offline.
Link: https://lore.kernel.org/all/20260409160243.1008358-1-hch@lst.de/
Suggested-by: Matthew Wilcox <willy@infradead.org>
Suggested-by: Christoph Hellwig <hch@infradead.org>
Signed-off-by: Tal Zussman <tz2294@columbia.edu>
---
block/bio.c | 147 +++++++++++++++++++++++++++++++++++++++++++++-
include/linux/bio.h | 32 ++++++++++
include/linux/blk_types.h | 1 +
3 files changed, 179 insertions(+), 1 deletion(-)
diff --git a/block/bio.c b/block/bio.c
index b8972dba68a0..6864ee737400 100644
--- a/block/bio.c
+++ b/block/bio.c
@@ -19,6 +19,7 @@
#include <linux/blk-crypto.h>
#include <linux/xarray.h>
#include <linux/kmemleak.h>
+#include <linux/local_lock.h>
#include <trace/events/block.h>
#include "blk.h"
@@ -1717,6 +1718,79 @@ void bio_check_pages_dirty(struct bio *bio)
}
EXPORT_SYMBOL_GPL(bio_check_pages_dirty);
+/*
+ * Infrastructure for deferring bio completions to task-context via a per-CPU
+ * workqueue. Triggered either by the BIO_COMPLETE_IN_TASK bio flag (static
+ * decision at submit time) or by calling bio_complete_in_task() from
+ * bi_end_io() (dynamic decision at completion time).
+ */
+
+struct bio_complete_batch {
+ local_lock_t lock;
+ struct bio_list list;
+ struct delayed_work work;
+ int cpu;
+};
+
+static DEFINE_PER_CPU(struct bio_complete_batch, bio_complete_batch) = {
+ .lock = INIT_LOCAL_LOCK(lock),
+};
+static struct workqueue_struct *bio_complete_wq;
+
+static void bio_complete_work_fn(struct work_struct *w)
+{
+ struct delayed_work *dw = to_delayed_work(w);
+ struct bio_complete_batch *batch =
+ container_of(dw, struct bio_complete_batch, work);
+
+ while (1) {
+ struct bio_list list;
+ struct bio *bio;
+
+ local_lock_irq(&bio_complete_batch.lock);
+ list = batch->list;
+ bio_list_init(&batch->list);
+ local_unlock_irq(&bio_complete_batch.lock);
+
+ if (bio_list_empty(&list))
+ break;
+
+ while ((bio = bio_list_pop(&list)))
+ bio->bi_end_io(bio);
+
+ if (need_resched()) {
+ bool is_empty;
+
+ local_lock_irq(&bio_complete_batch.lock);
+ is_empty = bio_list_empty(&batch->list);
+ local_unlock_irq(&bio_complete_batch.lock);
+ if (!is_empty)
+ mod_delayed_work_on(batch->cpu,
+ bio_complete_wq,
+ &batch->work, 0);
+ break;
+ }
+ }
+}
+
+void __bio_complete_in_task(struct bio *bio)
+{
+ struct bio_complete_batch *batch;
+ unsigned long flags;
+ bool was_empty;
+
+ local_lock_irqsave(&bio_complete_batch.lock, flags);
+ batch = this_cpu_ptr(&bio_complete_batch);
+ was_empty = bio_list_empty(&batch->list);
+ bio_list_add(&batch->list, bio);
+ local_unlock_irqrestore(&bio_complete_batch.lock, flags);
+
+ if (was_empty)
+ mod_delayed_work_on(batch->cpu, bio_complete_wq,
+ &batch->work, 1);
+}
+EXPORT_SYMBOL_GPL(__bio_complete_in_task);
+
static inline bool bio_remaining_done(struct bio *bio)
{
/*
@@ -1791,7 +1865,9 @@ void bio_endio(struct bio *bio)
}
#endif
- if (bio->bi_end_io)
+ if (bio_flagged(bio, BIO_COMPLETE_IN_TASK) && bio_in_atomic())
+ __bio_complete_in_task(bio);
+ else if (bio->bi_end_io)
bio->bi_end_io(bio);
}
EXPORT_SYMBOL(bio_endio);
@@ -1977,6 +2053,51 @@ int bioset_init(struct bio_set *bs,
}
EXPORT_SYMBOL(bioset_init);
+static int bio_complete_batch_cpu_online(unsigned int cpu)
+{
+ enable_delayed_work(&per_cpu(bio_complete_batch, cpu).work);
+ return 0;
+}
+
+/*
+ * Disable this CPU's delayed work so that it cannot run on an unbound worker
+ * after the CPU is offlined.
+ */
+static int bio_complete_batch_cpu_down_prep(unsigned int cpu)
+{
+ disable_delayed_work_sync(&per_cpu(bio_complete_batch, cpu).work);
+ return 0;
+}
+
+/*
+ * Drain a dead CPU's deferred bio completions. The CPU is dead and the worker
+ * is canceled so no locking is needed.
+ */
+static int bio_complete_batch_cpu_dead(unsigned int cpu)
+{
+ struct bio_complete_batch *batch =
+ per_cpu_ptr(&bio_complete_batch, cpu);
+ struct bio *bio;
+
+ while ((bio = bio_list_pop(&batch->list)))
+ bio->bi_end_io(bio);
+
+ return 0;
+}
+
+static void __init bio_complete_batch_init(int cpu)
+{
+ struct bio_complete_batch *batch =
+ per_cpu_ptr(&bio_complete_batch, cpu);
+
+ bio_list_init(&batch->list);
+ INIT_DELAYED_WORK(&batch->work, bio_complete_work_fn);
+ batch->cpu = cpu;
+
+ if (!cpu_online(cpu))
+ disable_delayed_work_sync(&batch->work);
+}
+
static int __init init_bio(void)
{
int i;
@@ -1991,6 +2112,30 @@ static int __init init_bio(void)
SLAB_HWCACHE_ALIGN | SLAB_PANIC, NULL);
}
+ for_each_possible_cpu(i)
+ bio_complete_batch_init(i);
+
+ bio_complete_wq = alloc_workqueue("bio_complete",
+ WQ_MEM_RECLAIM | WQ_PERCPU, 0);
+ if (!bio_complete_wq)
+ panic("bio: can't allocate bio_complete workqueue\n");
+
+ /*
+ * bio task-context completion draining on hot-unplugged CPUs:
+ *
+ * 1. Stop the per-CPU delayed work while the CPU is still online, so
+ * that it cannot run on an unbound worker later.
+ * 2. Drain leftover bios added between worker disabling and CPU
+ * offlining.
+ */
+ cpuhp_setup_state_nocalls(CPUHP_AP_ONLINE_DYN,
+ "block/bio:complete:online",
+ bio_complete_batch_cpu_online,
+ bio_complete_batch_cpu_down_prep);
+ cpuhp_setup_state_nocalls(CPUHP_BP_PREPARE_DYN,
+ "block/bio:complete:dead",
+ NULL, bio_complete_batch_cpu_dead);
+
cpuhp_setup_state_multi(CPUHP_BIO_DEAD, "block/bio:dead", NULL,
bio_cpu_dead);
diff --git a/include/linux/bio.h b/include/linux/bio.h
index 97d747320b35..c0214d6c28d6 100644
--- a/include/linux/bio.h
+++ b/include/linux/bio.h
@@ -369,6 +369,38 @@ static inline struct bio *bio_alloc(struct block_device *bdev,
void submit_bio(struct bio *bio);
+/**
+ * bio_in_atomic - check if the current context is unsafe for bio completion
+ *
+ * Return: %true in atomic contexts (e.g. hard/soft IRQ, preempt-disabled);
+ * %false when a bio can be safely completed in the current context.
+ */
+static inline bool bio_in_atomic(void)
+{
+ if (IS_ENABLED(CONFIG_PREEMPTION) && rcu_preempt_depth())
+ return true;
+ if (!IS_ENABLED(CONFIG_PREEMPT_COUNT))
+ return true;
+ return !preemptible();
+}
+
+void __bio_complete_in_task(struct bio *bio);
+
+/**
+ * bio_complete_in_task - ensure a bio is completed in preemptible task context
+ * @bio: bio to complete
+ *
+ * If called from non-task context, offload the bio completion to a worker
+ * thread and return %true. Else return %false and do nothing.
+ */
+static inline bool bio_complete_in_task(struct bio *bio)
+{
+ if (!bio_in_atomic())
+ return false;
+ __bio_complete_in_task(bio);
+ return true;
+}
+
extern void bio_endio(struct bio *);
static inline void bio_io_error(struct bio *bio)
diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h
index 8808ee76e73c..d49d97a050d0 100644
--- a/include/linux/blk_types.h
+++ b/include/linux/blk_types.h
@@ -322,6 +322,7 @@ enum {
BIO_REMAPPED,
BIO_ZONE_WRITE_PLUGGING, /* bio handled through zone write plugging */
BIO_EMULATES_ZONE_APPEND, /* bio emulates a zone append operation */
+ BIO_COMPLETE_IN_TASK, /* complete bi_end_io() in task context */
BIO_FLAG_LAST
};
--
2.39.5
^ permalink raw reply related
* Re: [PATCH 6/8] iomap: use bio_complete_in_task for buffered read errors
From: Tal Zussman @ 2026-05-14 18:30 UTC (permalink / raw)
To: Christoph Hellwig, Jens Axboe, Matthew Wilcox (Oracle),
Christian Brauner, Darrick J. Wong, Carlos Maiolino, Al Viro,
Jan Kara
Cc: Dave Chinner, Bart Van Assche, Gao Xiang, linux-block,
linux-kernel, linux-xfs, linux-fsdevel, linux-mm
In-Reply-To: <20260409160243.1008358-7-hch@lst.de>
On 4/9/26 12:02 PM, Christoph Hellwig wrote:
> Replace out own hand-crafted complete in task context scheme with the
> generic block code.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> fs/iomap/bio.c | 44 +-------------------------------------------
> 1 file changed, 1 insertion(+), 43 deletions(-)
>
> diff --git a/fs/iomap/bio.c b/fs/iomap/bio.c
> index 4504f4633f17..5b9b91198ec8 100644
> --- a/fs/iomap/bio.c
> +++ b/fs/iomap/bio.c
> @@ -9,9 +9,6 @@
> #include "internal.h"
> #include "trace.h"
>
> -static DEFINE_SPINLOCK(failed_read_lock);
> -static struct bio_list failed_read_list = BIO_EMPTY_LIST;
> -
> static u32 __iomap_read_end_io(struct bio *bio, int error)
> {
> struct folio_iter fi;
> @@ -27,49 +24,10 @@ static u32 __iomap_read_end_io(struct bio *bio, int error)
> return folio_count;
> }
>
> -static void
> -iomap_fail_reads(
> - struct work_struct *work)
> -{
> - struct bio *bio;
> - struct bio_list tmp = BIO_EMPTY_LIST;
> - unsigned long flags;
> -
> - spin_lock_irqsave(&failed_read_lock, flags);
> - bio_list_merge_init(&tmp, &failed_read_list);
> - spin_unlock_irqrestore(&failed_read_lock, flags);
> -
> - while ((bio = bio_list_pop(&tmp)) != NULL) {
> - __iomap_read_end_io(bio, blk_status_to_errno(bio->bi_status));
> - cond_resched();
> - }
> -}
> -
> -static DECLARE_WORK(failed_read_work, iomap_fail_reads);
> -
> -static void iomap_fail_buffered_read(struct bio *bio)
> -{
> - unsigned long flags;
> -
> - /*
> - * Bounce I/O errors to a workqueue to avoid nested i_lock acquisitions
> - * in the fserror code. The caller no longer owns the bio reference
> - * after the spinlock drops.
> - */
> - spin_lock_irqsave(&failed_read_lock, flags);
> - if (bio_list_empty(&failed_read_list))
> - WARN_ON_ONCE(!schedule_work(&failed_read_work));
> - bio_list_add(&failed_read_list, bio);
> - spin_unlock_irqrestore(&failed_read_lock, flags);
> -}
> -
> static void iomap_read_end_io(struct bio *bio)
> {
> - if (bio->bi_status) {
> - iomap_fail_buffered_read(bio);
> + if (bio->bi_status && bio_complete_in_task(bio))
> return;
> - }
> -
> __iomap_read_end_io(bio, 0);
I think this should be something like:
__iomap_read_end_io(bio, blk_status_to_errno(bio->bi_status));
Otherwise the error is never propagated when iomap_read_end_io() is later
called from task context. Maybe worth rearranging slightly to avoid the
blk_status_to_errno() call for non-error I/Os?
I was planning on carrying this and 7/8 in the next version of my series
so that the procedural version has users, but I'll send without for now.
> }
>
> --
> 2.47.3
>
^ permalink raw reply
* Re: [RFC PATCH v2] virtio_blk: add fua write support
From: Stefan Hajnoczi @ 2026-05-14 18:22 UTC (permalink / raw)
To: Sana Sharma; +Cc: afaria, linux-block
In-Reply-To: <20260512191808.45334-2-sansshar@redhat.com>
[-- Attachment #1: Type: text/plain, Size: 1444 bytes --]
On Tue, May 12, 2026 at 03:18:09PM -0400, Sana Sharma wrote:
> Continuing the work of Alberto Faria <afaria@redhat.com>
> this patch addresses the remaining feedback regarding removing
> VIRTIO_BLK_F_OUT_FUA from features_legacy[]
>
> Link: https://lore.kernel.org/linux-block/20250508001951.421467-1-afaria@redhat.com/
> Originally-by: Alberto Faria <afaria@redhat.com>
> Signed-off-by: Alberto Faria <afaria@redhat.com>
> Signed-off-by: Sana Sharma <sansshar@redhat.com>
>
>
> ---
> drivers/block/virtio_blk.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
Thanks, Sana!
Please squash your change into Alberto's commit, rebase against the
latest linux.git/master, test against a patched QEMU, and resend the
entire series once the VIRTIO spec change has been merged (the vote is
currently ongoing).
>
> diff --git a/drivers/block/virtio_blk.c b/drivers/block/virtio_blk.c
> index 5a6aa3dfb18a..aee2d503d6ef 100644
> --- a/drivers/block/virtio_blk.c
> +++ b/drivers/block/virtio_blk.c
> @@ -1665,7 +1665,7 @@ static unsigned int features_legacy[] = {
> VIRTIO_BLK_F_RO, VIRTIO_BLK_F_BLK_SIZE,
> VIRTIO_BLK_F_FLUSH, VIRTIO_BLK_F_TOPOLOGY, VIRTIO_BLK_F_CONFIG_WCE,
> VIRTIO_BLK_F_MQ, VIRTIO_BLK_F_DISCARD, VIRTIO_BLK_F_WRITE_ZEROES,
> - VIRTIO_BLK_F_SECURE_ERASE, VIRTIO_BLK_F_OUT_FUA,
> + VIRTIO_BLK_F_SECURE_ERASE,
> }
> ;
> static unsigned int features[] = {
> --
> 2.54.0
>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox