* Re: [PATCH 1/1] block: partitions: bound sysv68 slice table count
From: Philippe De Muyter @ 2026-06-16 10:44 UTC (permalink / raw)
To: Ren Wei
Cc: linux-block, kees, axboe, objecting, akpm, yuantan098, zcliangcn,
bird, zzhan461
In-Reply-To: <f16321f7378d613d81af13f288de82217fc7d934.1781036698.git.zzhan461@ucr.edu>
Hi Ren Wei,
On Thu, Jun 11, 2026 at 12:58:13AM +0800, Ren Wei wrote:
> From: Zhao Zhang <zzhan461@ucr.edu>
>
> sysv68_partition() reads a single sector for the slice table, but it
> trusts ios_slccnt from disk and walks that many entries after skipping
> the synthetic whole-disk slice. A crafted image can set ios_slccnt
> larger than the 64 struct slice records that fit in one sector and
> trigger an out-of-bounds read while scanning partitions.
>
> Limit the slice count to the number of records that fit in the sector
> returned by read_part_sector(), then drop the whole-disk entry only
> when the bounded count is non-zero.
>
> Fixes: 19d0e8ce856a ("partition: add support for sysv68 partitions")
> Cc: stable@vger.kernel.org
> Reported-by: Yuan Tan <yuantan098@gmail.com>
> Reported-by: Zhengchuan Liang <zcliangcn@gmail.com>
> Reported-by: Xin Liu <bird@lzu.edu.cn>
> Assisted-by: Codex:GPT-5.4
> Signed-off-by: Zhao Zhang <zzhan461@ucr.edu>
> Signed-off-by: Ren Wei <n05ec@lzu.edu.cn>
> ---
> block/partitions/sysv68.c | 11 +++++++----
> 1 file changed, 7 insertions(+), 4 deletions(-)
>
> diff --git a/block/partitions/sysv68.c b/block/partitions/sysv68.c
> index 470e0f9de7be..5110ed83c541 100644
> --- a/block/partitions/sysv68.c
> +++ b/block/partitions/sysv68.c
> @@ -48,7 +48,8 @@ struct slice {
>
> int sysv68_partition(struct parsed_partitions *state)
> {
> - int i, slices;
> + sector_t slice_sector;
> + unsigned int i, slices;
> int slot = 1;
> Sector sect;
> unsigned char *data;
> @@ -65,14 +66,16 @@ int sysv68_partition(struct parsed_partitions *state)
> return 0;
> }
> slices = be16_to_cpu(b->dk_ios.ios_slccnt);
> - i = be32_to_cpu(b->dk_ios.ios_slcblk);
> + slice_sector = be32_to_cpu(b->dk_ios.ios_slcblk);
> put_dev_sector(sect);
>
> - data = read_part_sector(state, i, §);
> + data = read_part_sector(state, slice_sector, §);
> if (!data)
> return -1;
>
> - slices -= 1; /* last slice is the whole disk */
> + slices = min_t(unsigned int, slices, SECTOR_SIZE / sizeof(*slice));
> + if (slices)
> + slices -= 1; /* last slice is the whole disk */
> seq_buf_printf(&state->pp_buf, "sysV68: %s(s%u)", state->name, slices);
> slice = (struct slice *)data;
> for (i = 0; i < slices; i++, slice++) {
> --
> 2.47.3
That does the job. IIRC 'last slice' had number 7, so ios_slccnt had to be
8. I do not have such a partition handy at the moment, so
Reviewed-by: Philippe De Muyter <phdm@macqel.be>
Best regards
Philippe
^ permalink raw reply
* [PATCH v4 15/30] iov_iter: Add a segmented queue of bio_vec[]
From: David Howells @ 2026-06-16 10:08 UTC (permalink / raw)
To: Christian Brauner, Matthew Wilcox, Christoph Hellwig
Cc: David Howells, Paulo Alcantara, Jens Axboe, Leon Romanovsky,
Steve French, ChenXiaoSong, Marc Dionne, Eric Van Hensbergen,
Dominique Martinet, Ilya Dryomov, netfs, linux-afs, linux-cifs,
linux-nfs, ceph-devel, v9fs, linux-erofs, linux-fsdevel,
linux-kernel, linux-block
In-Reply-To: <20260616100821.2062304-1-dhowells@redhat.com>
Add the concept of a segmented queue of bio_vec[] arrays. This allows an
indefinite quantity of elements to be handled and allows things like
network filesystems and crypto drivers to glue bits on the ends without
having to reallocate the array.
The bvecq struct that defines each segment also carries capacity/usage
information along with flags indicating whether the constituent memory
regions need freeing or unpinning and the file position of the first
element in a segment. The bvecq structs are refcounted to allow a queue to
be extracted in batches and split between a number of subrequests.
The bvecq can have the bio_vec[] it manages allocated in with it, but this
is not required. A flag is provided for if this is the case as comparing
->bv to ->__bv is not sufficient to detect this case.
Add an iterator type ITER_BVECQ for it. This is intended to replace
ITER_FOLIOQ (and ITER_XARRAY).
Note that the prev pointer is only really needed for iov_iter_revert() and
could be dispensed with if struct iov_iter contained the head information
as well as the current point.
Signed-off-by: David Howells <dhowells@redhat.com>
Reviewed-by: Paulo Alcantara (Red Hat) <pc@manguebit.org>
cc: Matthew Wilcox <willy@infradead.org>
cc: Christoph Hellwig <hch@infradead.org>
cc: Jens Axboe <axboe@kernel.dk>
cc: linux-block@vger.kernel.org
cc: netfs@lists.linux.dev
cc: linux-fsdevel@vger.kernel.org
---
include/linux/bvecq.h | 56 +++++++
include/linux/iov_iter.h | 74 ++++++++-
include/linux/uio.h | 11 ++
lib/iov_iter.c | 324 ++++++++++++++++++++++++++++++++++++-
lib/scatterlist.c | 67 +++++++-
lib/tests/kunit_iov_iter.c | 262 ++++++++++++++++++++++++++++++
6 files changed, 788 insertions(+), 6 deletions(-)
create mode 100644 include/linux/bvecq.h
diff --git a/include/linux/bvecq.h b/include/linux/bvecq.h
new file mode 100644
index 000000000000..15f16f905877
--- /dev/null
+++ b/include/linux/bvecq.h
@@ -0,0 +1,56 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Implementation of a segmented queue of bio_vec[].
+ *
+ * Copyright (C) 2026 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells@redhat.com)
+ */
+
+#ifndef _LINUX_BVECQ_H
+#define _LINUX_BVECQ_H
+
+#include <linux/bvec.h>
+
+/*
+ * The type of memory retention used by the elements in bvecq->bv[] and how to
+ * clean it up.
+ */
+enum bvecq_mem {
+ BVECQ_MEM_EXTERNAL, /* Externally retained memory - no freeing */
+ BVECQ_MEM_PAGECACHE, /* Ref'd pagecache pages - must put */
+ BVECQ_MEM_GUP, /* Pinned memory from get_user_pages() - unpin */
+ BVECQ_MEM_ALLOCED, /* Memory alloc'd by bvecq - can be freed/pooled */
+} __mode(byte);
+
+/*
+ * Segmented bio_vec queue.
+ *
+ * These can be linked together to form messages of indefinite length and
+ * iterated over with an ITER_BVECQ iterator. The list is non-circular; next
+ * and prev are NULL at the ends.
+ *
+ * The bv pointer points to the bio_vec array; this may be __bv if allocated
+ * together. The caller is responsible for determining whether or not this is
+ * the case as the array pointed to by bv may be follow on directly from the
+ * bvecq by accident of allocation (ie. ->bv == ->__bv is *not* sufficient to
+ * determine this).
+ *
+ * The file position and discontiguity flag allow non-contiguous data sets to
+ * be chained together, but still teased apart without the need to convert the
+ * info in the bio_vec back into a folio pointer.
+ */
+struct bvecq {
+ struct bvecq *next; /* Next bvec in the list or NULL */
+ struct bvecq *prev; /* Prev bvec in the list or NULL */
+ unsigned long long fpos; /* File position */
+ refcount_t ref;
+ u32 priv; /* Private data */
+ u16 nr_slots; /* Number of elements in bv[] used */
+ u16 max_slots; /* Number of elements allocated in bv[] */
+ enum bvecq_mem mem_type:3; /* What sort of memory and how to free it */
+ bool inline_bv:1; /* T if __bv[] is being used */
+ bool discontig:1; /* T if not contiguous with previous bvecq */
+ struct bio_vec *bv; /* Pointer to array of page fragments */
+ struct bio_vec __bv[]; /* Default array (if ->inline_bv) */
+};
+
+#endif /* _LINUX_BVECQ_H */
diff --git a/include/linux/iov_iter.h b/include/linux/iov_iter.h
index f9a17fbbd398..04b8a6d943fa 100644
--- a/include/linux/iov_iter.h
+++ b/include/linux/iov_iter.h
@@ -10,6 +10,7 @@
#include <linux/uio.h>
#include <linux/bvec.h>
+#include <linux/bvecq.h>
#include <linux/folio_queue.h>
typedef size_t (*iov_step_f)(void *iter_base, size_t progress, size_t len,
@@ -141,6 +142,71 @@ size_t iterate_bvec(struct iov_iter *iter, size_t len, void *priv, void *priv2,
return progress;
}
+/*
+ * Handle ITER_BVECQ.
+ */
+static __always_inline
+size_t iterate_bvecq(struct iov_iter *iter, size_t len, void *priv, void *priv2,
+ iov_step_f step)
+{
+ const struct bvecq *bq = iter->bvecq;
+ unsigned int slot = iter->bvecq_slot;
+ size_t progress = 0, skip = iter->iov_offset;
+
+ do {
+ const struct bio_vec *bvec;
+ struct page *page;
+ size_t poff, plen;
+ void *base;
+
+ if (slot >= bq->nr_slots) {
+ if (!bq->next)
+ break;
+ bq = bq->next;
+ slot = 0;
+ continue;
+ }
+
+ bvec = &bq->bv[slot];
+ /*
+ * The caller must ensure that a slot with bv_len>0 has a valid
+ * bv_page.
+ */
+ page = bvec->bv_page + (bvec->bv_offset + skip) / PAGE_SIZE;
+ poff = (bvec->bv_offset + skip) % PAGE_SIZE;
+ plen = umin(bvec->bv_len - skip, len);
+
+ while (plen > 0) {
+ size_t part, remain, consumed;
+
+ part = umin(plen, PAGE_SIZE - poff);
+ base = kmap_local_page(page) + poff;
+ remain = step(base, progress, part, priv, priv2);
+ kunmap_local(base);
+
+ consumed = part - remain;
+ progress += consumed;
+ skip += consumed;
+ len -= consumed;
+ if (!len || remain)
+ goto stop;
+ page++;
+ poff = 0;
+ plen -= consumed;
+ }
+
+ skip = 0;
+ slot++;
+ } while (len);
+
+stop:
+ iter->bvecq_slot = slot;
+ iter->bvecq = bq;
+ iter->iov_offset = skip;
+ iter->count -= progress;
+ return progress;
+}
+
/*
* Handle ITER_FOLIOQ.
*/
@@ -306,6 +372,8 @@ size_t iterate_and_advance2(struct iov_iter *iter, size_t len, void *priv,
return iterate_bvec(iter, len, priv, priv2, step);
if (iov_iter_is_kvec(iter))
return iterate_kvec(iter, len, priv, priv2, step);
+ if (iov_iter_is_bvecq(iter))
+ return iterate_bvecq(iter, len, priv, priv2, step);
if (iov_iter_is_folioq(iter))
return iterate_folioq(iter, len, priv, priv2, step);
if (iov_iter_is_xarray(iter))
@@ -342,8 +410,8 @@ size_t iterate_and_advance(struct iov_iter *iter, size_t len, void *priv,
* buffer is presented in segments, which for kernel iteration are broken up by
* physical pages and mapped, with the mapped address being presented.
*
- * [!] Note This will only handle BVEC, KVEC, FOLIOQ, XARRAY and DISCARD-type
- * iterators; it will not handle UBUF or IOVEC-type iterators.
+ * [!] Note This will only handle BVEC, KVEC, BVECQ, FOLIOQ, XARRAY and
+ * DISCARD-type iterators; it will not handle UBUF or IOVEC-type iterators.
*
* A step functions, @step, must be provided, one for handling mapped kernel
* addresses and the other is given user addresses which have the potential to
@@ -370,6 +438,8 @@ size_t iterate_and_advance_kernel(struct iov_iter *iter, size_t len, void *priv,
return iterate_bvec(iter, len, priv, priv2, step);
if (iov_iter_is_kvec(iter))
return iterate_kvec(iter, len, priv, priv2, step);
+ if (iov_iter_is_bvecq(iter))
+ return iterate_bvecq(iter, len, priv, priv2, step);
if (iov_iter_is_folioq(iter))
return iterate_folioq(iter, len, priv, priv2, step);
if (iov_iter_is_xarray(iter))
diff --git a/include/linux/uio.h b/include/linux/uio.h
index a9bc5b3067e3..f7cfa6ea8213 100644
--- a/include/linux/uio.h
+++ b/include/linux/uio.h
@@ -26,6 +26,7 @@ enum iter_type {
ITER_IOVEC,
ITER_BVEC,
ITER_KVEC,
+ ITER_BVECQ,
ITER_FOLIOQ,
ITER_XARRAY,
ITER_DISCARD,
@@ -68,6 +69,7 @@ struct iov_iter {
const struct iovec *__iov;
const struct kvec *kvec;
const struct bio_vec *bvec;
+ const struct bvecq *bvecq;
const struct folio_queue *folioq;
struct xarray *xarray;
void __user *ubuf;
@@ -77,6 +79,7 @@ struct iov_iter {
};
union {
unsigned long nr_segs;
+ u16 bvecq_slot;
u8 folioq_slot;
loff_t xarray_start;
};
@@ -145,6 +148,11 @@ static inline bool iov_iter_is_discard(const struct iov_iter *i)
return iov_iter_type(i) == ITER_DISCARD;
}
+static inline bool iov_iter_is_bvecq(const struct iov_iter *i)
+{
+ return iov_iter_type(i) == ITER_BVECQ;
+}
+
static inline bool iov_iter_is_folioq(const struct iov_iter *i)
{
return iov_iter_type(i) == ITER_FOLIOQ;
@@ -295,6 +303,9 @@ void iov_iter_kvec(struct iov_iter *i, unsigned int direction, const struct kvec
void iov_iter_bvec(struct iov_iter *i, unsigned int direction, const struct bio_vec *bvec,
unsigned long nr_segs, size_t count);
void iov_iter_discard(struct iov_iter *i, unsigned int direction, size_t count);
+void iov_iter_bvec_queue(struct iov_iter *i, unsigned int direction,
+ const struct bvecq *bvecq,
+ unsigned int first_slot, unsigned int offset, size_t count);
void iov_iter_folio_queue(struct iov_iter *i, unsigned int direction,
const struct folio_queue *folioq,
unsigned int first_slot, unsigned int offset, size_t count);
diff --git a/lib/iov_iter.c b/lib/iov_iter.c
index 49bca2c2f019..205d0da47b12 100644
--- a/lib/iov_iter.c
+++ b/lib/iov_iter.c
@@ -538,6 +538,40 @@ static void iov_iter_iovec_advance(struct iov_iter *i, size_t size)
i->__iov = iov;
}
+static void iov_iter_bvecq_advance(struct iov_iter *i, size_t by)
+{
+ const struct bvecq *bq = i->bvecq;
+ unsigned int slot = i->bvecq_slot;
+
+ if (!i->count)
+ return;
+ i->count -= by;
+
+ by += i->iov_offset; /* From beginning of current segment. */
+ do {
+ size_t len;
+
+ if (slot >= bq->nr_slots) {
+ if (!bq->next)
+ break;
+ bq = bq->next;
+ slot = 0;
+ continue;
+ }
+
+ len = bq->bv[slot].bv_len;
+
+ if (likely(by < len))
+ break;
+ by -= len;
+ slot++;
+ } while (by);
+
+ i->iov_offset = by;
+ i->bvecq_slot = slot;
+ i->bvecq = bq;
+}
+
static void iov_iter_folioq_advance(struct iov_iter *i, size_t size)
{
const struct folio_queue *folioq = i->folioq;
@@ -583,6 +617,8 @@ void iov_iter_advance(struct iov_iter *i, size_t size)
iov_iter_iovec_advance(i, size);
} else if (iov_iter_is_bvec(i)) {
iov_iter_bvec_advance(i, size);
+ } else if (iov_iter_is_bvecq(i)) {
+ iov_iter_bvecq_advance(i, size);
} else if (iov_iter_is_folioq(i)) {
iov_iter_folioq_advance(i, size);
} else if (iov_iter_is_discard(i)) {
@@ -591,6 +627,33 @@ void iov_iter_advance(struct iov_iter *i, size_t size)
}
EXPORT_SYMBOL(iov_iter_advance);
+static void iov_iter_bvecq_revert(struct iov_iter *i, size_t unroll)
+{
+ const struct bvecq *bq = i->bvecq;
+ unsigned int slot = i->bvecq_slot;
+
+ for (;;) {
+ size_t len;
+
+ if (slot == 0) {
+ bq = bq->prev;
+ slot = bq->nr_slots;
+ continue;
+ }
+ slot--;
+
+ len = bq->bv[slot].bv_len;
+ if (unroll <= len) {
+ i->iov_offset = len - unroll;
+ break;
+ }
+ unroll -= len;
+ }
+
+ i->bvecq_slot = slot;
+ i->bvecq = bq;
+}
+
static void iov_iter_folioq_revert(struct iov_iter *i, size_t unroll)
{
const struct folio_queue *folioq = i->folioq;
@@ -648,6 +711,9 @@ void iov_iter_revert(struct iov_iter *i, size_t unroll)
}
unroll -= n;
}
+ } else if (iov_iter_is_bvecq(i)) {
+ i->iov_offset = 0;
+ iov_iter_bvecq_revert(i, unroll);
} else if (iov_iter_is_folioq(i)) {
i->iov_offset = 0;
iov_iter_folioq_revert(i, unroll);
@@ -678,9 +744,24 @@ size_t iov_iter_single_seg_count(const struct iov_iter *i)
if (iov_iter_is_bvec(i))
return min(i->count, i->bvec->bv_len - i->iov_offset);
}
+ if (!i->count)
+ return 0;
+ if (unlikely(iov_iter_is_bvecq(i))) {
+ const struct bvecq *bq = i->bvecq;
+ unsigned int slot = i->bvecq_slot;
+ size_t offset = i->iov_offset;
+
+ while (slot >= bq->nr_slots) {
+ bq = bq->next;
+ if (!bq)
+ return 0;
+ slot = 0;
+ offset = 0;
+ }
+ return umin(i->count, bq->bv[slot].bv_len - offset);
+ }
if (unlikely(iov_iter_is_folioq(i)))
- return !i->count ? 0 :
- umin(folioq_folio_size(i->folioq, i->folioq_slot), i->count);
+ return umin(folioq_folio_size(i->folioq, i->folioq_slot), i->count);
return i->count;
}
EXPORT_SYMBOL(iov_iter_single_seg_count);
@@ -717,6 +798,35 @@ void iov_iter_bvec(struct iov_iter *i, unsigned int direction,
}
EXPORT_SYMBOL(iov_iter_bvec);
+/**
+ * iov_iter_bvec_queue - Initialise an I/O iterator to use a segmented bvec queue
+ * @i: The iterator to initialise.
+ * @direction: The direction of the transfer.
+ * @bvecq: The starting point in the bvec queue.
+ * @first_slot: The first slot in the bvec queue to use
+ * @offset: The offset into the bvec in the first slot to start at
+ * @count: The size of the I/O buffer in bytes.
+ *
+ * Set up an I/O iterator to either draw data out of the buffers attached to an
+ * inode or to inject data into those buffers. The pages *must* be prevented
+ * from evaporation, either by the caller.
+ */
+void iov_iter_bvec_queue(struct iov_iter *i, unsigned int direction,
+ const struct bvecq *bvecq, unsigned int first_slot,
+ unsigned int offset, size_t count)
+{
+ WARN_ON(direction & ~(READ | WRITE));
+ *i = (struct iov_iter) {
+ .iter_type = ITER_BVECQ,
+ .data_source = direction,
+ .bvecq = bvecq,
+ .bvecq_slot = first_slot,
+ .count = count,
+ .iov_offset = offset,
+ };
+}
+EXPORT_SYMBOL(iov_iter_bvec_queue);
+
/**
* iov_iter_folio_queue - Initialise an I/O iterator to use the folios in a folio queue
* @i: The iterator to initialise.
@@ -839,6 +949,37 @@ static unsigned long iov_iter_alignment_bvec(const struct iov_iter *i)
return res;
}
+static unsigned long iov_iter_alignment_bvecq(const struct iov_iter *iter)
+{
+ const struct bvecq *bq;
+ unsigned long res = 0;
+ unsigned int slot = iter->bvecq_slot;
+ size_t skip = iter->iov_offset;
+ size_t size = iter->count;
+
+ if (!size)
+ return res;
+
+ for (bq = iter->bvecq; bq; bq = bq->next) {
+ for (; slot < bq->nr_slots; slot++) {
+ const struct bio_vec *bvec = &bq->bv[slot];
+ size_t part = umin(bvec->bv_len - skip, size);
+
+ res |= bvec->bv_offset + skip;
+ res |= part;
+
+ size -= part;
+ if (size == 0)
+ return res;
+ skip = 0;
+ }
+
+ slot = 0;
+ }
+
+ return res;
+}
+
unsigned long iov_iter_alignment(const struct iov_iter *i)
{
if (likely(iter_is_ubuf(i))) {
@@ -854,6 +995,8 @@ unsigned long iov_iter_alignment(const struct iov_iter *i)
if (iov_iter_is_bvec(i))
return iov_iter_alignment_bvec(i);
+ if (iov_iter_is_bvecq(i))
+ return iov_iter_alignment_bvecq(i);
/* With both xarray and folioq types, we're dealing with whole folios. */
if (iov_iter_is_folioq(i))
@@ -1066,6 +1209,36 @@ static int bvec_npages(const struct iov_iter *i, int maxpages)
return npages;
}
+static size_t iov_npages_bvecq(const struct iov_iter *iter, size_t maxpages)
+{
+ const struct bvecq *bq;
+ unsigned int slot = iter->bvecq_slot;
+ size_t npages = 0;
+ size_t skip = iter->iov_offset;
+ size_t size = iter->count;
+
+ for (bq = iter->bvecq; bq; bq = bq->next) {
+ for (; slot < bq->nr_slots; slot++) {
+ const struct bio_vec *bvec = &bq->bv[slot];
+ size_t offs = (bvec->bv_offset + skip) % PAGE_SIZE;
+ size_t part = umin(bvec->bv_len - skip, size);
+
+ npages += DIV_ROUND_UP(offs + part, PAGE_SIZE);
+ if (npages >= maxpages)
+ goto out;
+
+ size -= part;
+ if (!size)
+ goto out;
+ skip = 0;
+ }
+
+ slot = 0;
+ }
+out:
+ return umin(npages, maxpages);
+}
+
int iov_iter_npages(const struct iov_iter *i, int maxpages)
{
if (unlikely(!i->count))
@@ -1080,6 +1253,8 @@ int iov_iter_npages(const struct iov_iter *i, int maxpages)
return iov_npages(i, maxpages);
if (iov_iter_is_bvec(i))
return bvec_npages(i, maxpages);
+ if (iov_iter_is_bvecq(i))
+ return iov_npages_bvecq(i, maxpages);
if (iov_iter_is_folioq(i)) {
unsigned offset = i->iov_offset % PAGE_SIZE;
int npages = DIV_ROUND_UP(offset + i->count, PAGE_SIZE);
@@ -1366,6 +1541,147 @@ void iov_iter_restore(struct iov_iter *i, struct iov_iter_state *state)
i->nr_segs = state->nr_segs;
}
+/*
+ * Count the number of virtually contiguous pages coming up next in an
+ * ITER_BVECQ iterator, up to the specified maxima.
+ */
+static unsigned int iter_count_bvecq_pages(const struct iov_iter *iter,
+ size_t maxsize,
+ unsigned int maxpages)
+{
+ const struct bvecq *bvecq = iter->bvecq;
+ unsigned int slot = iter->bvecq_slot;
+ ssize_t remain = umin(maxsize, iter->count);
+ size_t count = 0, offset = iter->iov_offset;
+
+ do {
+ const struct bio_vec *bv;
+ size_t boff, blen;
+
+ if (slot >= bvecq->nr_slots) {
+ if (!bvecq->next) {
+ WARN_ON_ONCE(remain > 0);
+ break;
+ }
+ bvecq = bvecq->next;
+ slot = 0;
+ offset = 0;
+ continue;
+ }
+
+ bv = &bvecq->bv[slot++];
+ boff = bv->bv_offset;
+ blen = bv->bv_len;
+
+ if (unlikely(!bv->bv_page)) {
+ if (blen && count > 0)
+ break;
+ continue;
+ }
+ if (!PAGE_ALIGNED(boff) && count > 0)
+ break;
+
+ boff += offset;
+ blen -= offset;
+ offset = 0;
+ if (!blen)
+ continue;
+
+ blen = umin(blen, remain);
+ remain -= blen;
+ blen += offset_in_page(boff);
+ count += DIV_ROUND_UP(blen, PAGE_SIZE);
+
+ if (!PAGE_ALIGNED(blen))
+ break;
+ } while (remain > 0 && count < maxpages);
+
+ return umin(count, maxpages);
+}
+
+/*
+ * Extract a list of virtually contiguous pages from an ITER_BVECQ iterator.
+ * This does not get references on the pages, nor does it get a pin on them.
+ */
+static ssize_t iov_iter_extract_bvecq_pages(struct iov_iter *iter,
+ struct page ***pages, size_t maxsize,
+ unsigned int maxpages,
+ iov_iter_extraction_t extraction_flags,
+ size_t *offset0)
+{
+ const struct bvecq *bvecq;
+ struct page **p;
+ unsigned int slot, nr = 0;
+ size_t extracted = 0, offset;
+
+ /* Count the next run of virtually contiguous pages. */
+ maxpages = iter_count_bvecq_pages(iter, maxsize, maxpages);
+
+ if (!*pages) {
+ *pages = kvmalloc_array(maxpages, sizeof(struct page *), GFP_KERNEL);
+ if (!*pages)
+ return -ENOMEM;
+ }
+
+ p = *pages;
+
+ /* Now transcribe the page pointers. */
+ extracted = 0;
+ bvecq = iter->bvecq;
+ offset = iter->iov_offset;
+ slot = iter->bvecq_slot;
+
+ do {
+ const struct bio_vec *bv;
+ size_t boff, blen;
+
+ if (slot >= bvecq->nr_slots) {
+ if (!bvecq->next) {
+ WARN_ON_ONCE(extracted < iter->count);
+ break;
+ }
+ bvecq = bvecq->next;
+ slot = 0;
+ offset = 0;
+ continue;
+ }
+
+ bv = &bvecq->bv[slot];
+ boff = bv->bv_offset;
+ blen = bv->bv_len;
+
+ if (!bv->bv_page)
+ blen = 0;
+
+ if (offset < blen) {
+ size_t part = umin(maxsize - extracted, blen - offset);
+ size_t poff = (boff + offset) % PAGE_SIZE;
+ size_t pix = (boff + offset) / PAGE_SIZE;
+
+ if (poff + part > PAGE_SIZE)
+ part = PAGE_SIZE - poff;
+
+ if (!extracted)
+ *offset0 = poff;
+
+ p[nr++] = bv->bv_page + pix;
+ offset += part;
+ extracted += part;
+ }
+
+ if (offset >= blen) {
+ offset = 0;
+ slot++;
+ }
+ } while (nr < maxpages && extracted < maxsize);
+
+ iter->bvecq = bvecq;
+ iter->bvecq_slot = slot;
+ iter->iov_offset = offset;
+ iter->count -= extracted;
+ return extracted;
+}
+
/*
* Extract a list of contiguous pages from an ITER_FOLIOQ iterator. This does
* not get references on the pages, nor does it get a pin on them.
@@ -1713,6 +2029,10 @@ ssize_t iov_iter_extract_pages(struct iov_iter *i,
return iov_iter_extract_bvec_pages(i, pages, maxsize,
maxpages, extraction_flags,
offset0);
+ if (iov_iter_is_bvecq(i))
+ return iov_iter_extract_bvecq_pages(i, pages, maxsize,
+ maxpages, extraction_flags,
+ offset0);
if (iov_iter_is_folioq(i))
return iov_iter_extract_folioq_pages(i, pages, maxsize,
maxpages, extraction_flags,
diff --git a/lib/scatterlist.c b/lib/scatterlist.c
index 6ea40d2e6247..23e5a180103b 100644
--- a/lib/scatterlist.c
+++ b/lib/scatterlist.c
@@ -10,6 +10,7 @@
#include <linux/highmem.h>
#include <linux/kmemleak.h>
#include <linux/bvec.h>
+#include <linux/bvecq.h>
#include <linux/uio.h>
#include <linux/folio_queue.h>
@@ -1267,6 +1268,65 @@ static ssize_t extract_kvec_to_sg(struct iov_iter *iter,
return ret;
}
+/*
+ * Extract up to sg_max folios from an BVECQ-type iterator and add them to
+ * the scatterlist. The pages are not pinned.
+ */
+static ssize_t extract_bvecq_to_sg(struct iov_iter *iter,
+ ssize_t maxsize,
+ struct sg_table *sgtable,
+ unsigned int sg_max,
+ iov_iter_extraction_t extraction_flags)
+{
+ const struct bvecq *bvecq = iter->bvecq;
+ struct scatterlist *sg = sgtable->sgl + sgtable->nents;
+ unsigned int slot = iter->bvecq_slot;
+ ssize_t ret = 0;
+ size_t offset = iter->iov_offset;
+
+ maxsize = umin(maxsize, iov_iter_count(iter));
+
+ while (sg_max > 0 && ret < maxsize) {
+ const struct bio_vec *bv;
+ size_t blen, part;
+
+ if (slot >= bvecq->nr_slots) {
+ if (!bvecq->next) {
+ WARN_ON_ONCE(ret < iter->count);
+ break;
+ }
+ bvecq = bvecq->next;
+ slot = 0;
+ offset = 0;
+ continue;
+ }
+
+ bv = &bvecq->bv[slot];
+ blen = bv->bv_len;
+
+ if (offset >= blen) {
+ offset = 0;
+ slot++;
+ continue;
+ }
+
+ part = umin(maxsize - ret, blen - offset);
+
+ sg_set_page(sg, bv->bv_page, part, bv->bv_offset + offset);
+ sgtable->nents++;
+ sg++;
+ sg_max--;
+ offset += part;
+ ret += part;
+ }
+
+ iter->bvecq = bvecq;
+ iter->bvecq_slot = slot;
+ iter->iov_offset = offset;
+ iter->count -= ret;
+ return ret;
+}
+
/*
* Extract up to sg_max folios from an FOLIOQ-type iterator and add them to
* the scatterlist. The pages are not pinned.
@@ -1391,8 +1451,8 @@ static ssize_t extract_xarray_to_sg(struct iov_iter *iter,
* addition of @sg_max elements.
*
* The pages referred to by UBUF- and IOVEC-type iterators are extracted and
- * pinned; BVEC-, KVEC-, FOLIOQ- and XARRAY-type are extracted but aren't
- * pinned; DISCARD-type is not supported.
+ * pinned; BVEC-, BVECQ-, KVEC-, FOLIOQ- and XARRAY-type are extracted but
+ * aren't pinned; DISCARD-type is not supported.
*
* No end mark is placed on the scatterlist; that's left to the caller.
*
@@ -1424,6 +1484,9 @@ ssize_t extract_iter_to_sg(struct iov_iter *iter, size_t maxsize,
case ITER_KVEC:
return extract_kvec_to_sg(iter, maxsize, sgtable, sg_max,
extraction_flags);
+ case ITER_BVECQ:
+ return extract_bvecq_to_sg(iter, maxsize, sgtable, sg_max,
+ extraction_flags);
case ITER_FOLIOQ:
return extract_folioq_to_sg(iter, maxsize, sgtable, sg_max,
extraction_flags);
diff --git a/lib/tests/kunit_iov_iter.c b/lib/tests/kunit_iov_iter.c
index e7e154f94f66..df3e78e7c7aa 100644
--- a/lib/tests/kunit_iov_iter.c
+++ b/lib/tests/kunit_iov_iter.c
@@ -12,6 +12,7 @@
#include <linux/mm.h>
#include <linux/uio.h>
#include <linux/bvec.h>
+#include <linux/bvecq.h>
#include <linux/folio_queue.h>
#include <linux/scatterlist.h>
#include <linux/minmax.h>
@@ -544,6 +545,185 @@ static void __init iov_kunit_copy_from_folioq(struct kunit *test)
KUNIT_SUCCEED(test);
}
+static void iov_kunit_destroy_bvecq(void *data)
+{
+ struct bvecq *bq, *next;
+
+ for (bq = data; bq; bq = next) {
+ next = bq->next;
+ for (int i = 0; i < bq->nr_slots; i++)
+ if (bq->bv[i].bv_page)
+ put_page(bq->bv[i].bv_page);
+ kfree(bq);
+ }
+}
+
+static struct bvecq *iov_kunit_alloc_bvecq(struct kunit *test, unsigned int max_slots)
+{
+ struct bvecq *bq;
+
+ bq = kzalloc(struct_size(bq, __bv, max_slots), GFP_KERNEL);
+ KUNIT_ASSERT_NOT_ERR_OR_NULL(test, bq);
+ bq->max_slots = max_slots;
+ bq->bv = bq->__bv;
+ bq->inline_bv = true;
+ return bq;
+}
+
+static struct bvecq *iov_kunit_create_bvecq(struct kunit *test, unsigned int max_slots)
+{
+ struct bvecq *bq;
+
+ bq = iov_kunit_alloc_bvecq(test, max_slots);
+ kunit_add_action_or_reset(test, iov_kunit_destroy_bvecq, bq);
+ return bq;
+}
+
+static void __init iov_kunit_load_bvecq(struct kunit *test,
+ struct iov_iter *iter, int dir,
+ struct bvecq *bq_head,
+ struct page **pages, size_t npages)
+{
+ struct bvecq *bq = bq_head;
+ size_t size = 0;
+
+ for (int i = 0; i < npages; i++) {
+ if (bq->nr_slots >= bq->max_slots) {
+ bq->next = iov_kunit_alloc_bvecq(test, 13);
+ bq->next->prev = bq;
+ bq = bq->next;
+ }
+ bvec_set_page(&bq->bv[bq->nr_slots], pages[i], PAGE_SIZE, 0);
+ bq->nr_slots++;
+ size += PAGE_SIZE;
+ }
+ iov_iter_bvec_queue(iter, dir, bq_head, 0, 0, size);
+}
+
+/*
+ * Test copying to a ITER_BVECQ-type iterator.
+ */
+static void __init iov_kunit_copy_to_bvecq(struct kunit *test)
+{
+ const struct kvec_test_range *pr;
+ struct iov_iter iter;
+ struct bvecq *bq;
+ struct page **spages, **bpages;
+ u8 *scratch, *buffer;
+ size_t bufsize, npages, size, copied;
+ int i, patt;
+
+ bufsize = 0x100000;
+ npages = bufsize / PAGE_SIZE;
+
+ bq = iov_kunit_create_bvecq(test, 13);
+
+ scratch = iov_kunit_create_buffer(test, &spages, npages);
+ for (i = 0; i < bufsize; i++)
+ scratch[i] = pattern(i);
+
+ buffer = iov_kunit_create_buffer(test, &bpages, npages);
+ memset(buffer, 0, bufsize);
+
+ iov_kunit_load_bvecq(test, &iter, READ, bq, bpages, npages);
+
+ i = 0;
+ for (pr = kvec_test_ranges; pr->from >= 0; pr++) {
+ size = pr->to - pr->from;
+ KUNIT_ASSERT_LE(test, pr->to, bufsize);
+
+ iov_iter_bvec_queue(&iter, READ, bq, 0, 0, pr->to);
+ iov_iter_advance(&iter, pr->from);
+ copied = copy_to_iter(scratch + i, size, &iter);
+
+ KUNIT_EXPECT_EQ(test, copied, size);
+ KUNIT_EXPECT_EQ(test, iter.count, 0);
+ i += size;
+ if (test->status == KUNIT_FAILURE)
+ goto stop;
+ }
+
+ /* Build the expected image in the scratch buffer. */
+ patt = 0;
+ memset(scratch, 0, bufsize);
+ for (pr = kvec_test_ranges; pr->from >= 0; pr++)
+ for (i = pr->from; i < pr->to; i++)
+ scratch[i] = pattern(patt++);
+
+ /* Compare the images */
+ for (i = 0; i < bufsize; i++) {
+ KUNIT_EXPECT_EQ_MSG(test, buffer[i], scratch[i], "at i=%x", i);
+ if (buffer[i] != scratch[i])
+ return;
+ }
+
+stop:
+ KUNIT_SUCCEED(test);
+}
+
+/*
+ * Test copying from a ITER_BVECQ-type iterator.
+ */
+static void __init iov_kunit_copy_from_bvecq(struct kunit *test)
+{
+ const struct kvec_test_range *pr;
+ struct iov_iter iter;
+ struct bvecq *bq;
+ struct page **spages, **bpages;
+ u8 *scratch, *buffer;
+ size_t bufsize, npages, size, copied;
+ int i, j;
+
+ bufsize = 0x100000;
+ npages = bufsize / PAGE_SIZE;
+
+ bq = iov_kunit_create_bvecq(test, 13);
+
+ buffer = iov_kunit_create_buffer(test, &bpages, npages);
+ for (i = 0; i < bufsize; i++)
+ buffer[i] = pattern(i);
+
+ scratch = iov_kunit_create_buffer(test, &spages, npages);
+ memset(scratch, 0, bufsize);
+
+ iov_kunit_load_bvecq(test, &iter, READ, bq, bpages, npages);
+
+ i = 0;
+ for (pr = kvec_test_ranges; pr->from >= 0; pr++) {
+ size = pr->to - pr->from;
+ KUNIT_ASSERT_LE(test, pr->to, bufsize);
+
+ iov_iter_bvec_queue(&iter, WRITE, bq, 0, 0, pr->to);
+ iov_iter_advance(&iter, pr->from);
+ copied = copy_from_iter(scratch + i, size, &iter);
+
+ KUNIT_EXPECT_EQ(test, copied, size);
+ KUNIT_EXPECT_EQ(test, iter.count, 0);
+ i += size;
+ }
+
+ /* Build the expected image in the main buffer. */
+ i = 0;
+ memset(buffer, 0, bufsize);
+ for (pr = kvec_test_ranges; pr->from >= 0; pr++) {
+ for (j = pr->from; j < pr->to; j++) {
+ buffer[i++] = pattern(j);
+ if (i >= bufsize)
+ goto stop;
+ }
+ }
+stop:
+
+ /* Compare the images */
+ for (i = 0; i < bufsize; i++) {
+ KUNIT_EXPECT_EQ_MSG(test, scratch[i], buffer[i], "at i=%x", i);
+ if (scratch[i] != buffer[i])
+ return;
+ }
+
+ KUNIT_SUCCEED(test);
+}
+
static void iov_kunit_destroy_xarray(void *data)
{
struct xarray *xarray = data;
@@ -859,6 +1039,85 @@ static void __init iov_kunit_extract_pages_bvec(struct kunit *test)
KUNIT_SUCCEED(test);
}
+/*
+ * Test the extraction of ITER_BVECQ-type iterators.
+ */
+static void __init iov_kunit_extract_pages_bvecq(struct kunit *test)
+{
+ const struct kvec_test_range *pr;
+ struct iov_iter iter;
+ struct bvecq *bq;
+ struct page **bpages, *pagelist[8], **pages = pagelist;
+ ssize_t len;
+ size_t bufsize, size = 0, npages;
+ int i, from;
+
+ bufsize = 0x100000;
+ npages = bufsize / PAGE_SIZE;
+
+ bq = iov_kunit_create_bvecq(test, 13);
+
+ iov_kunit_create_buffer(test, &bpages, npages);
+ iov_kunit_load_bvecq(test, &iter, READ, bq, bpages, npages);
+
+ for (pr = kvec_test_ranges; pr->from >= 0; pr++) {
+ from = pr->from;
+ size = pr->to - from;
+ KUNIT_ASSERT_LE(test, pr->to, bufsize);
+
+ iov_iter_bvec_queue(&iter, WRITE, bq, 0, 0, pr->to);
+ iov_iter_advance(&iter, from);
+
+ do {
+ size_t offset0 = LONG_MAX;
+
+ for (i = 0; i < ARRAY_SIZE(pagelist); i++)
+ pagelist[i] = (void *)(unsigned long)0xaa55aa55aa55aa55ULL;
+
+ len = iov_iter_extract_pages(&iter, &pages, 100 * 1024,
+ ARRAY_SIZE(pagelist), 0, &offset0);
+ KUNIT_EXPECT_GE(test, len, 0);
+ if (len < 0)
+ break;
+ KUNIT_EXPECT_LE(test, len, size);
+ KUNIT_EXPECT_EQ(test, iter.count, size - len);
+ if (len == 0)
+ break;
+ size -= len;
+ KUNIT_EXPECT_GE(test, (ssize_t)offset0, 0);
+ KUNIT_EXPECT_LT(test, offset0, PAGE_SIZE);
+
+ for (i = 0; i < ARRAY_SIZE(pagelist); i++) {
+ struct page *p;
+ ssize_t part = min_t(ssize_t, len, PAGE_SIZE - offset0);
+ int ix;
+
+ KUNIT_ASSERT_GE(test, part, 0);
+ ix = from / PAGE_SIZE;
+ KUNIT_ASSERT_LT(test, ix, npages);
+ p = bpages[ix];
+ KUNIT_EXPECT_PTR_EQ(test, pagelist[i], p);
+ KUNIT_EXPECT_EQ(test, offset0, from % PAGE_SIZE);
+ from += part;
+ len -= part;
+ KUNIT_ASSERT_GE(test, len, 0);
+ if (len == 0)
+ break;
+ offset0 = 0;
+ }
+
+ if (test->status == KUNIT_FAILURE)
+ goto stop;
+ } while (iov_iter_count(&iter) > 0);
+
+ KUNIT_EXPECT_EQ(test, size, 0);
+ KUNIT_EXPECT_EQ(test, iter.count, 0);
+ }
+
+stop:
+ KUNIT_SUCCEED(test);
+}
+
/*
* Test the extraction of ITER_FOLIOQ-type iterators.
*/
@@ -1218,12 +1477,15 @@ static struct kunit_case __refdata iov_kunit_cases[] = {
KUNIT_CASE(iov_kunit_copy_from_kvec),
KUNIT_CASE(iov_kunit_copy_to_bvec),
KUNIT_CASE(iov_kunit_copy_from_bvec),
+ KUNIT_CASE(iov_kunit_copy_to_bvecq),
+ KUNIT_CASE(iov_kunit_copy_from_bvecq),
KUNIT_CASE(iov_kunit_copy_to_folioq),
KUNIT_CASE(iov_kunit_copy_from_folioq),
KUNIT_CASE(iov_kunit_copy_to_xarray),
KUNIT_CASE(iov_kunit_copy_from_xarray),
KUNIT_CASE(iov_kunit_extract_pages_kvec),
KUNIT_CASE(iov_kunit_extract_pages_bvec),
+ KUNIT_CASE(iov_kunit_extract_pages_bvecq),
KUNIT_CASE(iov_kunit_extract_pages_folioq),
KUNIT_CASE(iov_kunit_extract_pages_xarray),
KUNIT_CASE(iov_kunit_iter_to_sg_kvec),
^ permalink raw reply related
* [PATCH v4 14/30] iov_iter: Make iov_iter_get_pages*() wrap iov_iter_extract_pages()
From: David Howells @ 2026-06-16 10:08 UTC (permalink / raw)
To: Christian Brauner, Matthew Wilcox, Christoph Hellwig
Cc: David Howells, Paulo Alcantara, Jens Axboe, Leon Romanovsky,
Steve French, ChenXiaoSong, Marc Dionne, Eric Van Hensbergen,
Dominique Martinet, Ilya Dryomov, netfs, linux-afs, linux-cifs,
linux-nfs, ceph-devel, v9fs, linux-erofs, linux-fsdevel,
linux-kernel, linux-block
In-Reply-To: <20260616100821.2062304-1-dhowells@redhat.com>
Make iov_iter_get_pages*() wrap iov_iter_extract_pages() for kernel
iterator types (e.g. ITER_BVEC, ITER_FOLIOQ, ITER_XARRAY). The pages
obtained have their refcounts incremented afterwards if they're not slab
pages. ITER_KVEC is left returning -EFAULT.
Signed-off-by: David Howells <dhowells@redhat.com>
Reviewed-by: Paulo Alcantara (Red Hat) <pc@manguebit.org>
cc: Matthew Wilcox <willy@infradead.org>
cc: Christoph Hellwig <hch@infradead.org>
cc: Jens Axboe <axboe@kernel.dk>
cc: linux-block@vger.kernel.org
cc: netfs@lists.linux.dev
cc: linux-fsdevel@vger.kernel.org
---
lib/iov_iter.c | 164 ++++++-------------------------------------------
1 file changed, 19 insertions(+), 145 deletions(-)
diff --git a/lib/iov_iter.c b/lib/iov_iter.c
index 6386ae4ef491..49bca2c2f019 100644
--- a/lib/iov_iter.c
+++ b/lib/iov_iter.c
@@ -910,118 +910,34 @@ static int want_pages_array(struct page ***res, size_t size,
return count;
}
-static ssize_t iter_folioq_get_pages(struct iov_iter *iter,
+/*
+ * Wrap iov_iter_extract_pages() and then pin the non-slab pages we got back.
+ * This only works for non-user iterator types as get_pages uses get_user_pages
+ * not pin_user_pages.
+ */
+static ssize_t iter_get_kernel_pages(struct iov_iter *iter,
struct page ***ppages, size_t maxsize,
unsigned maxpages, size_t *_start_offset)
{
- const struct folio_queue *folioq = iter->folioq;
struct page **pages;
- unsigned int slot = iter->folioq_slot;
- size_t extracted = 0, count = iter->count, iov_offset = iter->iov_offset;
+ ssize_t ret, done;
- if (slot >= folioq_nr_slots(folioq)) {
- folioq = folioq->next;
- slot = 0;
- if (WARN_ON(iov_offset != 0))
- return -EIO;
- }
+ ret = iov_iter_extract_pages(iter, ppages, maxsize, maxpages,
+ 0, _start_offset);
+ if (ret <= 0)
+ return ret;
- maxpages = want_pages_array(ppages, maxsize, iov_offset & ~PAGE_MASK, maxpages);
- if (!maxpages)
- return -ENOMEM;
- *_start_offset = iov_offset & ~PAGE_MASK;
pages = *ppages;
+ for (done = ret + *_start_offset; done > 0; done -= PAGE_SIZE) {
+ struct folio *folio = page_folio(*pages);
- for (;;) {
- struct folio *folio = folioq_folio(folioq, slot);
- size_t offset = iov_offset, fsize = folioq_folio_size(folioq, slot);
- size_t part = PAGE_SIZE - offset % PAGE_SIZE;
-
- if (offset < fsize) {
- part = umin(part, umin(maxsize - extracted, fsize - offset));
- count -= part;
- iov_offset += part;
- extracted += part;
-
- *pages = folio_page(folio, offset / PAGE_SIZE);
- get_page(*pages);
- pages++;
- maxpages--;
- }
-
- if (maxpages == 0 || extracted >= maxsize)
- break;
-
- if (iov_offset >= fsize) {
- iov_offset = 0;
- slot++;
- if (slot == folioq_nr_slots(folioq) && folioq->next) {
- folioq = folioq->next;
- slot = 0;
- }
- }
+ if (!folio_test_slab(folio))
+ folio_get(folio);
+ pages++;
}
-
- iter->count = count;
- iter->iov_offset = iov_offset;
- iter->folioq = folioq;
- iter->folioq_slot = slot;
- return extracted;
-}
-
-static ssize_t iter_xarray_populate_pages(struct page **pages, struct xarray *xa,
- pgoff_t index, unsigned int nr_pages)
-{
- XA_STATE(xas, xa, index);
- struct folio *folio;
- unsigned int ret = 0;
-
- rcu_read_lock();
- for (folio = xas_load(&xas); folio; folio = xas_next(&xas)) {
- if (xas_retry(&xas, folio))
- continue;
-
- /* Has the folio moved or been split? */
- if (unlikely(folio != xas_reload(&xas))) {
- xas_reset(&xas);
- continue;
- }
-
- pages[ret] = folio_file_page(folio, xas.xa_index);
- folio_get(folio);
- if (++ret == nr_pages)
- break;
- }
- rcu_read_unlock();
return ret;
}
-static ssize_t iter_xarray_get_pages(struct iov_iter *i,
- struct page ***pages, size_t maxsize,
- unsigned maxpages, size_t *_start_offset)
-{
- unsigned nr, offset, count;
- pgoff_t index;
- loff_t pos;
-
- pos = i->xarray_start + i->iov_offset;
- index = pos >> PAGE_SHIFT;
- offset = pos & ~PAGE_MASK;
- *_start_offset = offset;
-
- count = want_pages_array(pages, maxsize, offset, maxpages);
- if (!count)
- return -ENOMEM;
- nr = iter_xarray_populate_pages(*pages, i->xarray, index, count);
- if (nr == 0)
- return 0;
-
- maxsize = min_t(size_t, nr * PAGE_SIZE - offset, maxsize);
- i->iov_offset += maxsize;
- i->count -= maxsize;
- return maxsize;
-}
-
/* must be done on non-empty ITER_UBUF or ITER_IOVEC one */
static unsigned long first_iovec_segment(const struct iov_iter *i, size_t *size)
{
@@ -1044,22 +960,6 @@ static unsigned long first_iovec_segment(const struct iov_iter *i, size_t *size)
BUG(); // if it had been empty, we wouldn't get called
}
-/* must be done on non-empty ITER_BVEC one */
-static struct page *first_bvec_segment(const struct iov_iter *i,
- size_t *size, size_t *start)
-{
- struct page *page;
- size_t skip = i->iov_offset, len;
-
- len = i->bvec->bv_len - skip;
- if (*size > len)
- *size = len;
- skip += i->bvec->bv_offset;
- page = i->bvec->bv_page + skip / PAGE_SIZE;
- *start = skip % PAGE_SIZE;
- return page;
-}
-
static ssize_t __iov_iter_get_pages_alloc(struct iov_iter *i,
struct page ***pages, size_t maxsize,
unsigned int maxpages, size_t *start)
@@ -1095,36 +995,10 @@ static ssize_t __iov_iter_get_pages_alloc(struct iov_iter *i,
iov_iter_advance(i, maxsize);
return maxsize;
}
- if (iov_iter_is_bvec(i)) {
- struct page **p;
- struct page *page;
- page = first_bvec_segment(i, &maxsize, start);
- n = want_pages_array(pages, maxsize, *start, maxpages);
- if (!n)
- return -ENOMEM;
- p = *pages;
- for (int k = 0; k < n; k++) {
- struct folio *folio = page_folio(page + k);
- p[k] = page + k;
- if (!folio_test_slab(folio))
- folio_get(folio);
- }
- maxsize = min_t(size_t, maxsize, n * PAGE_SIZE - *start);
- i->count -= maxsize;
- i->iov_offset += maxsize;
- if (i->iov_offset == i->bvec->bv_len) {
- i->iov_offset = 0;
- i->bvec++;
- i->nr_segs--;
- }
- return maxsize;
- }
- if (iov_iter_is_folioq(i))
- return iter_folioq_get_pages(i, pages, maxsize, maxpages, start);
- if (iov_iter_is_xarray(i))
- return iter_xarray_get_pages(i, pages, maxsize, maxpages, start);
- return -EFAULT;
+ if (iov_iter_is_kvec(i))
+ return -EFAULT;
+ return iter_get_kernel_pages(i, pages, maxsize, maxpages, start);
}
ssize_t iov_iter_get_pages2(struct iov_iter *i, struct page **pages,
^ permalink raw reply related
* [PATCH v4 13/30] Add a function to kmap one page of a multipage bio_vec
From: David Howells @ 2026-06-16 10:08 UTC (permalink / raw)
To: Christian Brauner, Matthew Wilcox, Christoph Hellwig
Cc: David Howells, Paulo Alcantara, Jens Axboe, Leon Romanovsky,
Steve French, ChenXiaoSong, Marc Dionne, Eric Van Hensbergen,
Dominique Martinet, Ilya Dryomov, netfs, linux-afs, linux-cifs,
linux-nfs, ceph-devel, v9fs, linux-erofs, linux-fsdevel,
linux-kernel, linux-block
In-Reply-To: <20260616100821.2062304-1-dhowells@redhat.com>
Add a function to kmap one page of a multipage bio_vec by offset (which is
added to the offset in the bio_vec internally). The caller is responsible
for calculating how much of the page is then available.
Signed-off-by: David Howells <dhowells@redhat.com>
Acked-by: Paulo Alcantara (Red Hat) <pc@manguebit.org>
cc: Matthew Wilcox <willy@infradead.org>
cc: Christoph Hellwig <hch@infradead.org>
cc: Jens Axboe <axboe@kernel.dk>
cc: linux-block@vger.kernel.org
cc: netfs@lists.linux.dev
cc: linux-fsdevel@vger.kernel.org
---
include/linux/bvec.h | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/include/linux/bvec.h b/include/linux/bvec.h
index d36dd476feda..f834a862224e 100644
--- a/include/linux/bvec.h
+++ b/include/linux/bvec.h
@@ -299,4 +299,22 @@ static inline phys_addr_t bvec_phys(const struct bio_vec *bvec)
return page_to_phys(bvec->bv_page) + bvec->bv_offset;
}
+/**
+ * bvec_kmap_partial - Map part of a bvec into the kernel virtual address space
+ * @bvec: bvec to map
+ * @offset: Offset into bvec
+ *
+ * Map the page containing the byte at @offset into the kernel virtual address
+ * space. The caller is responsible for making sure this doesn't overrun.
+ *
+ * Call kunmap_local on the returned address to unmap.
+ */
+static inline void *bvec_kmap_partial(struct bio_vec *bvec, size_t offset)
+{
+ offset += bvec->bv_offset;
+
+ return kmap_local_page(bvec->bv_page + (offset >> PAGE_SHIFT)) +
+ (offset & ~PAGE_MASK);
+}
+
#endif /* __LINUX_BVEC_H */
^ permalink raw reply related
* [PATCH v4 2/3] block: respect iov_iter::nofault flag in bio_iov_iter_bounce_write()
From: Qu Wenruo @ 2026-06-16 8:12 UTC (permalink / raw)
To: linux-btrfs, linux-block, linux-fsdevel, linux-xfs
In-Reply-To: <cover.1781597506.git.wqu@suse.com>
For the incoming usage of IOMAP_DIO_BOUNCE in btrfs, btrfs has set
iov_iter::nofault to prevent deadlock when a page fault is needed to
read out the buffer.
However bio_iov_iter_bounce_write() doesn't respect iov_iter::nofault
flag, and just call a plain copy_from_iter() so it can still trigger
page fault and cause deadlock in btrfs.
Fix it by utilizing copy_folio_from_iter_atomic() if nofault flag is
set, otherwise use copy_folio_from_iter().
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
block/bio.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/block/bio.c b/block/bio.c
index d2fdfcb016c1..7fd7c35f9a28 100644
--- a/block/bio.c
+++ b/block/bio.c
@@ -1335,7 +1335,11 @@ static int bio_iov_iter_bounce_write(struct bio *bio, struct iov_iter *iter,
break;
bio_add_folio_nofail(bio, folio, this_len, 0);
- copied = copy_from_iter(folio_address(folio), this_len, iter);
+ if (iter->nofault)
+ copied = copy_folio_from_iter_atomic(folio, 0, this_len,
+ iter);
+ else
+ copied = copy_folio_from_iter(folio, 0, this_len, iter);
if (copied < this_len) {
/*
* Need to revert the iov iter for all bytes we have
--
2.54.0
^ permalink raw reply related
* [PATCH v4 3/3] btrfs: use IOMAP_DIO_BOUNCE flag instead of falling back to buffered IO
From: Qu Wenruo @ 2026-06-16 8:12 UTC (permalink / raw)
To: linux-btrfs, linux-block, linux-fsdevel, linux-xfs
In-Reply-To: <cover.1781597506.git.wqu@suse.com>
Previously btrfs forces direct writes to fall back to buffered ones if the
inode has data checksum or the profile has duplication.
That fallback is to avoid the content being modified that the final
content may mismatch with the checksum or the other mirrors.
That brings a pretty huge performance cost, which already caused some
concern at that time.
But later upstream commit c9d114846b38 ("iomap: add a flag to bounce
buffer direct I/O") introduced a new method by copying the content into
new pages, and do all the operations based on the newly allocated pages.
So let btrfs to utilize the new flag for direct writes if we require
stable folios.
There is a quick benchmark, using the following fio setup:
fio --name=randwrite --filename $mnt/foobar --ioengine=libaio --size=4G \
--rw=randwrite --iodepth=64 --runtime=60 --time_based --direct=1 \
--bs=$blocksize
Unit is MiB/s.
Blocksize | Zero-copy (*) | Buffered | Bounce
-----------+---------------+----------+-----------
4K | 35.1 | 17.1 | 33.8
64K | 522 | 251 | 492
*: This is done by reverting the commit 968f19c5b1b7 ("btrfs: always
fallback to buffered write if the inode requires checksum")
Although with page bouncing the performance is only around 95% of
true-zero copy, it's still almost double the performance of buffered
fallback.
There will be a small change in behavior, since we're using
IOMAP_DIO_BOUNCE flag to allocate new folios, NOWAIT flag will
immediately fail.
So for true NOWAIT direct IOs, NODATASUM and RAID0/SINGLE profiles are
still required.
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
fs/btrfs/direct-io.c | 58 ++++++++++++++++++++++----------------------
1 file changed, 29 insertions(+), 29 deletions(-)
diff --git a/fs/btrfs/direct-io.c b/fs/btrfs/direct-io.c
index e566a60b0ce5..bb457649902e 100644
--- a/fs/btrfs/direct-io.c
+++ b/fs/btrfs/direct-io.c
@@ -818,13 +818,41 @@ static ssize_t btrfs_dio_read(struct kiocb *iocb, struct iov_iter *iter,
IOMAP_DIO_PARTIAL | IOMAP_DIO_FSBLOCK_ALIGNED, &data, done_before);
}
+static bool need_stable_write(struct btrfs_inode *inode)
+{
+ const u64 data_profile = btrfs_data_alloc_profile(inode->root->fs_info) &
+ BTRFS_BLOCK_GROUP_PROFILE_MASK;
+
+ /* Data checksum requires stable buffer. */
+ if (!(inode->flags & BTRFS_INODE_NODATASUM))
+ return true;
+ /*
+ * Any profile with mirror/parity will require stable buffer.
+ * Otherwise the mirror may differ from each other.
+ *
+ * Thus only SINGLE and RAID0 doesn't require stable buffer.
+ */
+ if (data_profile != 0 && data_profile != BTRFS_BLOCK_GROUP_RAID0)
+ return true;
+ return false;
+}
+
static struct iomap_dio *btrfs_dio_write(struct kiocb *iocb, struct iov_iter *iter,
size_t done_before)
{
struct btrfs_dio_data data = { 0 };
+ unsigned int dio_flags = IOMAP_DIO_PARTIAL | IOMAP_DIO_FSBLOCK_ALIGNED;
+
+ if (need_stable_write(BTRFS_I(file_inode(iocb->ki_filp)))) {
+ /* For now no support for BOUNCE and NOWAIT direct write. */
+ if (iocb->ki_flags & IOCB_NOWAIT)
+ return ERR_PTR(-EAGAIN);
+
+ dio_flags |= IOMAP_DIO_BOUNCE;
+ }
return __iomap_dio_rw(iocb, iter, &btrfs_dio_iomap_ops, &btrfs_dio_ops,
- IOMAP_DIO_PARTIAL | IOMAP_DIO_FSBLOCK_ALIGNED, &data, done_before);
+ dio_flags, &data, done_before);
}
static ssize_t check_direct_IO(struct btrfs_fs_info *fs_info,
@@ -853,8 +881,6 @@ ssize_t btrfs_direct_write(struct kiocb *iocb, struct iov_iter *from)
ssize_t ret;
unsigned int ilock_flags = 0;
struct iomap_dio *dio;
- const u64 data_profile = btrfs_data_alloc_profile(fs_info) &
- BTRFS_BLOCK_GROUP_PROFILE_MASK;
if (iocb->ki_flags & IOCB_NOWAIT)
ilock_flags |= BTRFS_ILOCK_TRY;
@@ -868,16 +894,6 @@ ssize_t btrfs_direct_write(struct kiocb *iocb, struct iov_iter *from)
if (iocb->ki_pos + iov_iter_count(from) <= i_size_read(inode) && IS_NOSEC(inode))
ilock_flags |= BTRFS_ILOCK_SHARED;
- /*
- * If our data profile has duplication (either extra mirrors or RAID56),
- * we can not trust the direct IO buffer, the content may change during
- * writeback and cause different contents written to different mirrors.
- *
- * Thus only RAID0 and SINGLE can go true zero-copy direct IO.
- */
- if (data_profile != BTRFS_BLOCK_GROUP_RAID0 && data_profile != 0)
- goto buffered;
-
relock:
ret = btrfs_inode_lock(BTRFS_I(inode), ilock_flags);
if (ret < 0)
@@ -918,22 +934,6 @@ ssize_t btrfs_direct_write(struct kiocb *iocb, struct iov_iter *from)
btrfs_inode_unlock(BTRFS_I(inode), ilock_flags);
goto buffered;
}
- /*
- * We can't control the folios being passed in, applications can write
- * to them while a direct IO write is in progress. This means the
- * content might change after we calculated the data checksum.
- * Therefore we can end up storing a checksum that doesn't match the
- * persisted data.
- *
- * To be extra safe and avoid false data checksum mismatch, if the
- * inode requires data checksum, just fallback to buffered IO.
- * For buffered IO we have full control of page cache and can ensure
- * no one is modifying the content during writeback.
- */
- if (!(BTRFS_I(inode)->flags & BTRFS_INODE_NODATASUM)) {
- btrfs_inode_unlock(BTRFS_I(inode), ilock_flags);
- goto buffered;
- }
/*
* The iov_iter can be mapped to the same file range we are writing to.
--
2.54.0
^ permalink raw reply related
* [PATCH v4 1/3] block: revert the iov_iter after a short copy in bio_iov_iter_bounce_write()
From: Qu Wenruo @ 2026-06-16 8:12 UTC (permalink / raw)
To: linux-btrfs, linux-block, linux-fsdevel, linux-xfs
In-Reply-To: <cover.1781597506.git.wqu@suse.com>
For the incoming IOMAP_DIO_BOUNCE flag usage inside btrfs, it's pretty
easy to hit short copy inside bio_iov_iter_bounce_write().
This is because btrfs has disabled page fault to avoid certain deadlock
during direct writes, and instead btrfs manually fault in the pages then
retry.
And inside bio_iov_iter_bounce_write(), if we hit a short write, we
didn't revert the iov_iter, which can cause problems like unexpected
garbage for the next retry.
Revert the iov_iter after a short copy.
One thing to note is that, the folio is allocated then immediately
queued into the bio, so the proper revert size should be
(bi_size - this_len + copied).
Fixes: 8dd5e7c75d7b ("block: add helpers to bounce buffer an iov_iter into bios")
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
block/bio.c | 17 ++++++++++++++---
1 file changed, 14 insertions(+), 3 deletions(-)
diff --git a/block/bio.c b/block/bio.c
index 5f10900b3f42..d2fdfcb016c1 100644
--- a/block/bio.c
+++ b/block/bio.c
@@ -1321,6 +1321,7 @@ static int bio_iov_iter_bounce_write(struct bio *bio, struct iov_iter *iter,
do {
size_t this_len = min(total_len, SZ_1M);
+ size_t copied;
struct folio *folio;
if (this_len > minsize * 2)
@@ -1334,12 +1335,22 @@ static int bio_iov_iter_bounce_write(struct bio *bio, struct iov_iter *iter,
break;
bio_add_folio_nofail(bio, folio, this_len, 0);
- if (copy_from_iter(folio_address(folio), this_len, iter) !=
- this_len) {
+ copied = copy_from_iter(folio_address(folio), this_len, iter);
+ if (copied < this_len) {
+ /*
+ * Need to revert the iov iter for all bytes we have
+ * copied.
+ *
+ * However the bio size differs from the real copied
+ * bytes as @this_len is queued but only advanced
+ * less than that.
+ * Need to compensate that for the revert.
+ */
+ iov_iter_revert(iter, bio->bi_iter.bi_size - this_len +
+ copied);
bio_free_folios(bio);
return -EFAULT;
}
-
total_len -= this_len;
} while (total_len && bio->bi_vcnt < bio->bi_max_vecs);
--
2.54.0
^ permalink raw reply related
* [PATCH v4 0/3] btrfs: use IOMAP_DIO_BOUNCE flag instead of falling back to buffered IO
From: Qu Wenruo @ 2026-06-16 8:12 UTC (permalink / raw)
To: linux-btrfs, linux-block, linux-fsdevel, linux-xfs
[CHANGELOG]
v4:
- Follow iomap/block layer code style to avoid lines over 80 chars
- Reject NOWAIT BOUNCE direct writes inside btrfs
The iomap code still allocates memory with GFP_KERNEL in other
locations.
For now just disable NOWAIT BOUNCE direct writes and let the caller
fall back to blocking mode.
v3:
- Fix a bug in error handling of bio_iov_iter_bounce_write()
Which can lead to generic/708 failure on btrfs.
- Respect nofault flag in bio_iov_iter_bounce_write()
To avoid btrfs specific deadlocks.
- Reject NOWAIT and BOUNCE direct IOs
Since BOUNCE always allocate pages using GFP_KERNEL, which can sleep
and break NOWAIT requirement, has to reject such combination.
v2:
- Rework the comment in btrfs_dio_write()
Commit 968f19c5b1b7 ("btrfs: always fallback to buffered write if the
inode requires checksum") solved the csum mismatch caused by unstable
direct IO buffers, it has a pretty hefty performance penalty.
Meanwhile upstream iomap has introduce IOMAP_DIO_BOUNCE flag to get
stable buffers meanwhile without falling back to buffered IOs.
Using that flag btrfs can reach 95% of the original zero-copy direct IO
performance, almost 2x the current buffered fallback performance.
However during my tests, there are several bugs related to iomap that
can lead to direct IO test case failures:
- generic/708
Results garbage in the end of the writes, is a bug in the error
handling of a short copy.
Fixed in the first patch.
- Deadlock if using the page cache as direct IO buffer
This is because bio_iov_iter_bounce_write() doesn't respect
iov_iter::nofault flag.
Fixed in the second patch.
- Possible NOWAIT and BOUNCE conflicts
BOUNCE flag for both reads and writes will allocate new folios using
GFP_KERNEL, which can sleep and break NOWAIT requirement.
Reject such combination in btrfs when enabling IOMAP_DIO_BOUNCE
support.
And the final one will enable btrfs to use IOMAP_DIO_BOUNCE flag, so
that even with data checksum we do not need to fallback to buffered IO
and reclaim most of the dropped direct IO performance.
Qu Wenruo (3):
block: revert the iov_iter after a short copy in
bio_iov_iter_bounce_write()
block: respect iov_iter::nofault flag in bio_iov_iter_bounce_write()
btrfs: use IOMAP_DIO_BOUNCE flag instead of falling back to buffered
IO
block/bio.c | 21 +++++++++++++---
fs/btrfs/direct-io.c | 58 ++++++++++++++++++++++----------------------
2 files changed, 47 insertions(+), 32 deletions(-)
--
2.54.0
^ permalink raw reply
* Re: [GIT PULL] Block updates for 7.2
From: pr-tracker-bot @ 2026-06-16 7:54 UTC (permalink / raw)
To: Jens Axboe; +Cc: Linus Torvalds, linux-block@vger.kernel.org
In-Reply-To: <28f00610-4a45-47f5-9e08-468c31736090@kernel.dk>
The pull request you sent on Mon, 15 Jun 2026 09:24:22 -0600:
> https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux.git tags/for-7.2/block-20260615
has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/ba9c792c824fff732df85119011d399d9b6d9155
Thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/prtracker.html
^ permalink raw reply
* Re: [PATCH v3 3/4] iomap: reject NOWAIT and BOUNCE direct IOs
From: Christoph Hellwig @ 2026-06-16 5:19 UTC (permalink / raw)
To: Qu Wenruo
Cc: Christoph Hellwig, Qu Wenruo, linux-btrfs, linux-block,
linux-fsdevel, linux-xfs
In-Reply-To: <8a0e1881-fe73-4912-95f9-8eac998840d5@gmx.com>
On Tue, Jun 16, 2026 at 10:30:27AM +0930, Qu Wenruo wrote:
> After scanning the code for related memory allocation, there are some other
> locations doing memory allocation, including but not limited to:
>
> - iomap_dio_alloc_bio()
> This one is a little tricky, if we pass GFP_NOWAIT, we can break the
> old assumption that the function will always return a bio.
>
> - fscrypt_set_bio_crypt_ctx()
> This one is fine so far, as neither XFS nor btrfs support fscrypt yet.
>
> And any memory allocation failure in NOWAIT mode should return -EAGAIN, not
> -ENOMEM so that the caller can retry in blocking mode as a fallback.
>
> To me, considering NOWAIT itself is only an optimistic flag, and caller
> should always have a blocking mode as fallback, I'd prefer to reject NOWAIT
> + BOUNCE direct writes completely inside btrfs for now.
If you want to do that in btrfs please do it there.
> And leave all the missing NOWAIT handling in iomap in a dedicated series.
This might be worth looking into.
^ permalink raw reply
* Re: [PATCH v3 3/4] iomap: reject NOWAIT and BOUNCE direct IOs
From: Christoph Hellwig @ 2026-06-16 5:18 UTC (permalink / raw)
To: Qu Wenruo
Cc: Christoph Hellwig, Qu Wenruo, linux-btrfs, linux-block,
linux-fsdevel, linux-xfs
In-Reply-To: <efa3f1f9-20c2-44c0-914c-5e34a40235a1@gmx.com>
On Tue, Jun 16, 2026 at 08:13:27AM +0930, Qu Wenruo wrote:
> It looks like NORETRY can still sleep, thus again breaking NOWAIT
> requirement.
>
> I think you're talking about GFP_NOWAIT?
Yes, GFP_NOWAIT would be a better fit.
^ permalink raw reply
* Re: [PATCH v4 0/3] crypto: skcipher - per-request multi-data-unit batching
From: Herbert Xu @ 2026-06-16 4:53 UTC (permalink / raw)
To: Eric Biggers
Cc: Leonid Ravich, Alasdair Kergon, Ard Biesheuvel, Jens Axboe,
Horia Geanta, Gilad Ben-Yossef, linux-crypto, dm-devel,
linux-block
In-Reply-To: <20260616045023.GA113934@sol>
On Mon, Jun 15, 2026 at 09:50:23PM -0700, Eric Biggers wrote:
>
> Have you checked the code? This patchset adds overhead in multiple
> places. Dynamically allocating multiple scatterlists and then parsing
> them, adding a new field to skcipher_request for everyone, new checks in
> crypto_skcipher_en/decrypt for everyone, new checks to validate the data
> unit size that the caller knew was valid in the first place, etc.
No I have not :)
I'm just stating the general principle.
Of course I will not apply the patch-set until I have reviewed it
properly.
Cheers,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [PATCH v4 0/3] crypto: skcipher - per-request multi-data-unit batching
From: Eric Biggers @ 2026-06-16 4:50 UTC (permalink / raw)
To: Herbert Xu
Cc: Leonid Ravich, Alasdair Kergon, Ard Biesheuvel, Jens Axboe,
Horia Geanta, Gilad Ben-Yossef, linux-crypto, dm-devel,
linux-block
In-Reply-To: <ajDNT5jVGgRtiNH6@gondor.apana.org.au>
On Tue, Jun 16, 2026 at 12:13:03PM +0800, Herbert Xu wrote:
> On Mon, Jun 15, 2026 at 03:53:17PM -0700, Eric Biggers wrote:
> >
> > So in other words, this series slows down dm-crypt and crypto_skcipher
> > for everyone to optimize for an out-of-tree driver. And there's also no
> > benchmark showing that your driver is even worth it over just using the
> > CPU.
>
> There is no reason why the software fallback should be slower
> than the status quo. Existing callers of the Crypto API will
> be issuing one indirect function call per data unit. With the
> new scheme, the indirect calls per unit moves from from the caller
> into the Crypto API.
Have you checked the code? This patchset adds overhead in multiple
places. Dynamically allocating multiple scatterlists and then parsing
them, adding a new field to skcipher_request for everyone, new checks in
crypto_skcipher_en/decrypt for everyone, new checks to validate the data
unit size that the caller knew was valid in the first place, etc.
> In fact, we could move it down further and improve upon the
> status quo by splitting the data in each algorithm implemntation
> so that the calls per unit become direct function calls and only
> the overall call into the Crypto API remains indirect.
That's not what this patchset does. But also, as we know, a better way
to eliminate "Crypto API" overhead is to call the algorithms directly.
- Eric
^ permalink raw reply
* Re: [PATCH v4 0/3] crypto: skcipher - per-request multi-data-unit batching
From: Herbert Xu @ 2026-06-16 4:13 UTC (permalink / raw)
To: Eric Biggers
Cc: Leonid Ravich, Alasdair Kergon, Ard Biesheuvel, Jens Axboe,
Horia Geanta, Gilad Ben-Yossef, linux-crypto, dm-devel,
linux-block
In-Reply-To: <20260615225317.GB28589@quark>
On Mon, Jun 15, 2026 at 03:53:17PM -0700, Eric Biggers wrote:
>
> So in other words, this series slows down dm-crypt and crypto_skcipher
> for everyone to optimize for an out-of-tree driver. And there's also no
> benchmark showing that your driver is even worth it over just using the
> CPU.
There is no reason why the software fallback should be slower
than the status quo. Existing callers of the Crypto API will
be issuing one indirect function call per data unit. With the
new scheme, the indirect calls per unit moves from from the caller
into the Crypto API.
In fact, we could move it down further and improve upon the
status quo by splitting the data in each algorithm implemntation
so that the calls per unit become direct function calls and only
the overall call into the Crypto API remains indirect.
But yes it would be nice to provide numbers for the fallback
path to verify that we didn't get this case terribly wrong.
Cheers,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: Repeatable, raid1+O_DIRECT, hang/warn
From: Vjaceslavs Klimovs @ 2026-06-16 1:25 UTC (permalink / raw)
To: Keith Busch
Cc: Dr. David Alan Gilbert, Thorsten Leemhuis, trnka, linux-block,
dm-devel, Linux kernel regressions list
In-Reply-To: <ajCTaUaACV9eNmWo@kbusch-mbp>
Hi Keith,
Thanks. I tested both patches on current mainline
(v7.1-rc7-271-g424280953322) with my QEMU + LVM "--type mirror"
reproducer (virtio-blk, cache=none, aio=native).
With only the "block: check bio split for unaligned bvec" patch, the
hang still reproduces. The WARN fires from a kmirrord worker:
WARNING: block/bio.c:1044 at bio_add_page+0x108/0x200
Workqueue: kmirrord do_mirror
Call Trace:
bio_add_page+0x108/0x200
do_region+0x21d/0x270
dispatch_io+0xf1/0x150
dm_io+0x136/0x240
do_reads+0x13e/0x210
do_mirror+0x117/0x2b0
and the VM then wedges.
With the dm-io.c clone patch applied on top, the WARN and the hang are
both gone. dm-mirror just fails the read instead:
device-mapper: raid1: Mirror read failed from 252:0. Trying
alternative device.
device-mapper: raid1: All sides of mirror have failed.
device-mapper: raid1: Read failure on mirror device 252:1. Failing I/O.
The guest still gets an I/O error, as you expected, but the host stays
up: no splat, no stuck task. For comparison, on the same kernel the
"--type raid1" case boots the guest and reads fine, and the 128 MB
mirror seed write goes through the clone path without trouble, so
normal I/O looks unaffected.
Thanks,
Vjaceslavs
On Mon, Jun 15, 2026 at 5:06 PM Keith Busch <kbusch@kernel.org> wrote:
>
> On Mon, Jun 15, 2026 at 04:16:12PM -0700, Vjaceslavs Klimovs wrote:
> > Your trace looks like what the two earlier reports hit: a read reaching
> > a leaf device with sectors > 0 but phys_seg 0 (an empty bio). One aside
> > that may help read the trace: blk_io_trace.error is a __u16, so the
> > bracketed values on your C lines are errnos as u16 (65514 = -EINVAL,
> > 65531 = -EIO).
> >
> > The WARN itself is new, the bad bio isn't. bio_add_page() only started
> > rejecting len == 0 in 643893647cac ("block: reject zero length in
> > bio_add_page()", v7.1-rc1); on 7.0.8 the same empty bio tripped
> > scsi_alloc_sgtables()'s !nr_segs instead, which matches what you saw.
> > That fits your "not a recent regression": the condition is older, v7.1
> > just made it loud.
> >
> > For Tomas's and my reports (QEMU O_DIRECT to the LV block device) the
> > origin looks like 5ff3f74e145a ("block: simplify direct io validity
> > check", v6.18): blkdev_dio_invalid() now checks only aggregate
> > ki_pos | count alignment and dropped the per-segment
> > bdev_iter_is_aligned() walk, so a degenerate or misaligned O_DIRECT no
> > longer gets -EINVAL at the fops boundary. But your reproducer reads a
> > file, which goes through the filesystem O_DIRECT path and never calls
> > blkdev_dio_invalid(), and still makes the empty bio. So it isn't only
> > that one entry point.
> >
> > dm-mirror then hangs because Keith's f7b24c7b41f2 only covers md
> > raid1/raid10; legacy dm-mirror (dm-raid1.c) has no equivalent and
> > rebuilds the empty read onto the other leg. Note the leg's status isn't
> > even consistent (your SATA path returns BLK_STS_IOERR, not
> > BLK_STS_INVAL), so copying that status check into dm-mirror probably
> > wouldn't catch every case.
> >
> > For what it's worth, that points me toward rejecting the empty or
> > misaligned bio once, at submission, with -EINVAL, rather than teaching
> > each consumer to tolerate it. But you'll know the tradeoffs far better
> > than I do.
> >
> > I have a small QEMU + LVM raid1/mirror setup that reproduces the
> > block-device variant and bisects to 5ff3f74e. Happy to run your file
> > reproducer with some instrumentation at the dm-mirror read entry
> > (bi_size vs bio_sectors vs bvec lengths) to see whether the bio is
> > already empty on arrival or built that way on the retry, and to test
> > any patch.
>
> Thanks for following up here. I didn't initially see your follow-up
> until Thorsten linked it. I apologize for missing that, this feature is
> important so I don't want to see anything regress for it.
>
> There is a known bug fix I think future tests should include:
>
> https://lore.kernel.org/linux-block/20260612223205.465913-1-kbusch@meta.com/
>
> This likely isn't the fix you're looking for, but including it rules out
> conditions that are not important here.
>
> After that, can we try this suggestion and see if the hang goes away?
>
> https://lore.kernel.org/linux-block/ajBb8tK-0aJBpIgF@kbusch-mbp/
>
> I expect the original test case to still return an error (and I think it
> was designed to), but it shouldn't produce the warn or bug splats with a
> stuck uninterruptable task.
^ permalink raw reply
* [PATCH V3] blk-cgroup: defer blkcg css_put until blkg is unlinked from queue
From: Zizhi Wo @ 2026-06-16 1:17 UTC (permalink / raw)
To: axboe, tj, josef, linux-block
Cc: cgroups, yangerkun, chengzhihao1, houtao1, yukuai, wozizhi
From: Zizhi Wo <wozizhi@huawei.com>
[BUG]
Our fuzz testing triggered a blkcg use-after-free issue:
BUG: KASAN: slab-use-after-free in _raw_spin_lock+0x75/0xe0
Call Trace:
...
blkcg_deactivate_policy+0x244/0x4d0
ioc_rqos_exit+0x44/0xe0
rq_qos_exit+0xba/0x120
__del_gendisk+0x50b/0x800
del_gendisk+0xff/0x190
...
[CAUSE]
process1 process2
cgroup_rmdir
...
css_killed_work_fn
offline_css
...
blkcg_destroy_blkgs
...
__blkg_release
css_put(&blkg->blkcg->css)
blkg_free
INIT_WORK(xxx, blkg_free_workfn)
schedule_work
css_put
...
blkcg_css_free
kfree(blkcg)--------blkcg has been freed!!!
====================================schedule_work
blkg_free_workfn
__del_gendisk
rq_qos_exit
ioc_rqos_exit
blkcg_deactivate_policy
mutex_lock(&q->blkcg_mutex)
spin_lock_irq(&q->queue_lock)
list_for_each_entry(blkg, xxx)
blkcg = blkg->blkcg
spin_lock(&blkcg->lock)-------UAF!!!
mutex_lock(&q->blkcg_mutex)
spin_lock_irq(&q->queue_lock)
/* Only then is the blkg removed from the list */
list_del_init(&blkg->q_node)
As a result, a blkg can still be reachable through q->blkg_list while
its ->blkcg has already been freed.
[Fix]
Fix this by deferring the blkcg css_put() until after the blkg has been
unlinked from q->blkg_list in blkg_free_workfn(). This ensures that the
blkcg outlives every blkg still reachable through q->blkg_list, so any
iterator holding q->queue_lock is guaranteed to observe a valid
blkg->blkcg.
While at it, move css_tryget_online() from blkg_create() into blkg_alloc()
so that the css reference is owned by the alloc/free pair rather than
straddling layers:
blkg_alloc() <-> blkg_free()
blkg_create() <-> blkg_destroy()
Fixes: f1c006f1c685 ("blk-cgroup: synchronize pd_free_fn() from blkg_free_workfn() and blkcg_deactivate_policy()")
Suggested-by: Hou Tao <houtao1@huawei.com>
Signed-off-by: Zizhi Wo <wozizhi@huawei.com>
Reviewed-by: Yu Kuai <yukuai@fygo.io>
---
v3:
- move css_put() after mutex_unlock() in blkg_free_workfn().
v2:
- Move css_tryget_online() from blkg_create() into blkg_alloc() so the
css reference follows the blkg's own lifetime, making the put in
blkg_free_workfn() symmetric with the get in blkg_alloc().
v1: https://lore.kernel.org/all/20260518010932.633707-1-wozizhi@huaweicloud.com/
block/blk-cgroup.c | 24 ++++++++++++------------
1 file changed, 12 insertions(+), 12 deletions(-)
diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c
index bc63bd220865..3ac41f766caf 100644
--- a/block/blk-cgroup.c
+++ b/block/blk-cgroup.c
@@ -136,6 +136,11 @@ static void blkg_free_workfn(struct work_struct *work)
spin_unlock_irq(&q->queue_lock);
mutex_unlock(&q->blkcg_mutex);
+ /*
+ * Release blkcg css ref only after blkg is removed from q->blkg_list,
+ * so concurrent iterators won't see a blkg with a freed blkcg.
+ */
+ css_put(&blkg->blkcg->css);
blk_put_queue(q);
free_percpu(blkg->iostat_cpu);
percpu_ref_exit(&blkg->refcnt);
@@ -179,8 +184,6 @@ static void __blkg_release(struct rcu_head *rcu)
for_each_possible_cpu(cpu)
__blkcg_rstat_flush(blkcg, cpu);
- /* release the blkcg and parent blkg refs this blkg has been holding */
- css_put(&blkg->blkcg->css);
blkg_free(blkg);
}
@@ -313,6 +316,9 @@ static struct blkcg_gq *blkg_alloc(struct blkcg *blkcg, struct gendisk *disk,
goto out_exit_refcnt;
if (!blk_get_queue(disk->queue))
goto out_free_iostat;
+ /* blkg holds a reference to blkcg */
+ if (!css_tryget_online(&blkcg->css))
+ goto out_put_queue;
blkg->q = disk->queue;
INIT_LIST_HEAD(&blkg->q_node);
@@ -353,6 +359,8 @@ static struct blkcg_gq *blkg_alloc(struct blkcg *blkcg, struct gendisk *disk,
while (--i >= 0)
if (blkg->pd[i])
blkcg_policy[i]->pd_free_fn(blkg->pd[i]);
+ css_put(&blkcg->css);
+out_put_queue:
blk_put_queue(disk->queue);
out_free_iostat:
free_percpu(blkg->iostat_cpu);
@@ -381,18 +389,12 @@ static struct blkcg_gq *blkg_create(struct blkcg *blkcg, struct gendisk *disk,
goto err_free_blkg;
}
- /* blkg holds a reference to blkcg */
- if (!css_tryget_online(&blkcg->css)) {
- ret = -ENODEV;
- goto err_free_blkg;
- }
-
/* allocate */
if (!new_blkg) {
new_blkg = blkg_alloc(blkcg, disk, GFP_NOWAIT);
if (unlikely(!new_blkg)) {
ret = -ENOMEM;
- goto err_put_css;
+ goto err_free_blkg;
}
}
blkg = new_blkg;
@@ -402,7 +404,7 @@ static struct blkcg_gq *blkg_create(struct blkcg *blkcg, struct gendisk *disk,
blkg->parent = blkg_lookup(blkcg_parent(blkcg), disk->queue);
if (WARN_ON_ONCE(!blkg->parent)) {
ret = -ENODEV;
- goto err_put_css;
+ goto err_free_blkg;
}
blkg_get(blkg->parent);
}
@@ -442,8 +444,6 @@ static struct blkcg_gq *blkg_create(struct blkcg *blkcg, struct gendisk *disk,
blkg_put(blkg);
return ERR_PTR(ret);
-err_put_css:
- css_put(&blkcg->css);
err_free_blkg:
if (new_blkg)
blkg_free(new_blkg);
--
2.52.0
^ permalink raw reply related
* Re: [PATCH V2] blk-cgroup: defer blkcg css_put until blkg is unlinked from queue
From: Hou Tao @ 2026-06-16 1:23 UTC (permalink / raw)
To: yukuai, Zizhi Wo, axboe, tj, josef, linux-block
Cc: cgroups, yangerkun, chengzhihao1
In-Reply-To: <70642ddf-9ed9-45cb-bf40-891a07247c97@fnnas.com>
Hi,
On 6/16/2026 12:16 AM, Yu Kuai wrote:
> Hi，
>
> 在 2026/6/15 19:55, Zizhi Wo 写道:
>> From: Zizhi Wo <wozizhi@huawei.com>
>>
>> [BUG]
>> Our fuzz testing triggered a blkcg use-after-free issue:
>>
>> BUG: KASAN: slab-use-after-free in _raw_spin_lock+0x75/0xe0
>> Call Trace:
>> ...
>> blkcg_deactivate_policy+0x244/0x4d0
>> ioc_rqos_exit+0x44/0xe0
>> rq_qos_exit+0xba/0x120
>> __del_gendisk+0x50b/0x800
>> del_gendisk+0xff/0x190
>> ...
>>
>> [CAUSE]
>> process1 process2
>> cgroup_rmdir
>> ...
>> css_killed_work_fn
>> offline_css
>> ...
>> blkcg_destroy_blkgs
>> ...
>> __blkg_release
>> css_put(&blkg->blkcg->css)
>> blkg_free
>> INIT_WORK(xxx, blkg_free_workfn)
>> schedule_work
>> css_put
>> ...
>> blkcg_css_free
>> kfree(blkcg)--------blkcg has been freed!!!
>> ====================================schedule_work
>> blkg_free_workfn
>> __del_gendisk
>> rq_qos_exit
>> ioc_rqos_exit
>> blkcg_deactivate_policy
>> mutex_lock(&q->blkcg_mutex)
>> spin_lock_irq(&q->queue_lock)
>> list_for_each_entry(blkg, xxx)
>> blkcg = blkg->blkcg
>> spin_lock(&blkcg->lock)-------UAF!!!
>> mutex_lock(&q->blkcg_mutex)
>> spin_lock_irq(&q->queue_lock)
>> /* Only then is the blkg removed from the list */
>> list_del_init(&blkg->q_node)
>>
>> As a result, a blkg can still be reachable through q->blkg_list while
>> its ->blkcg has already been freed.
>>
>> [Fix]
>> Fix this by deferring the blkcg css_put() until after the blkg has been
>> unlinked from q->blkg_list in blkg_free_workfn(). This ensures that the
>> blkcg outlives every blkg still reachable through q->blkg_list, so any
>> iterator holding q->queue_lock is guaranteed to observe a valid
>> blkg->blkcg.
>>
>> While at it, move css_tryget_online() from blkg_create() into blkg_alloc()
>> so that the css reference is owned by the alloc/free pair rather than
>> straddling layers:
>> blkg_alloc() <-> blkg_free()
>> blkg_create() <-> blkg_destroy()
>>
>> Fixes: f1c006f1c685 ("blk-cgroup: synchronize pd_free_fn() from blkg_free_workfn() and blkcg_deactivate_policy()")
>> Suggested-by: Hou Tao <houtao1@huawei.com>
>> Signed-off-by: Zizhi Wo <wozizhi@huawei.com>
>> ---
>> v2:
>> - Move css_tryget_online() from blkg_create() into blkg_alloc() so the
>> css reference follows the blkg's own lifetime, making the put in
>> blkg_free_workfn() symmetric with the get in blkg_alloc().
>>
>> v1: https://lore.kernel.org/all/20260518010932.633707-1-wozizhi@huaweicloud.com/
>>
>> block/blk-cgroup.c | 24 ++++++++++++------------
>> 1 file changed, 12 insertions(+), 12 deletions(-)
>>
>> diff --git a/block/blk-cgroup.c b/block/blk-cgroup.c
>> index bc63bd220865..27414c291e49 100644
>> --- a/block/blk-cgroup.c
>> +++ b/block/blk-cgroup.c
>> @@ -132,10 +132,15 @@ static void blkg_free_workfn(struct work_struct *work)
>> if (blkg->parent)
>> blkg_put(blkg->parent);
>> spin_lock_irq(&q->queue_lock);
>> list_del_init(&blkg->q_node);
>> spin_unlock_irq(&q->queue_lock);
>> + /*
>> + * Release blkcg css ref only after blkg is removed from q->blkg_list,
>> + * so concurrent iterators won't see a blkg with a freed blkcg.
>> + */
>> + css_put(&blkg->blkcg->css);
>> mutex_unlock(&q->blkcg_mutex);
> Please move css_put after mutex_unlock, unless there is a strong reason.
I think blkcg_mutex is used here to serialize the access of blkg->q_node
and blkg->blkcg. We could move the css_put after the mutex_unlock(),
however it stills depends on the mutex_lock and mutex_unlock pair on
blkcg_mutex implicitly. Instead of such implicit dependency, we move the
css_put inside the lock to make it be explicit.
>
> With above change, feel free to add:
>
> Reviewed-by: Yu Kuai <yukuai@fygo.io>
>
>>
>> blk_put_queue(q);
>> free_percpu(blkg->iostat_cpu);
>> percpu_ref_exit(&blkg->refcnt);
>> @@ -177,12 +182,10 @@ static void __blkg_release(struct rcu_head *rcu)
>> * blkg_stat_lock is for serializing blkg stat update
>> */
>> for_each_possible_cpu(cpu)
>> __blkcg_rstat_flush(blkcg, cpu);
>>
>> - /* release the blkcg and parent blkg refs this blkg has been holding */
>> - css_put(&blkg->blkcg->css);
>> blkg_free(blkg);
>> }
>>
>> /*
>> * A group is RCU protected, but having an rcu lock does not mean that one
>> @@ -311,10 +314,13 @@ static struct blkcg_gq *blkg_alloc(struct blkcg *blkcg, struct gendisk *disk,
>> blkg->iostat_cpu = alloc_percpu_gfp(struct blkg_iostat_set, gfp_mask);
>> if (!blkg->iostat_cpu)
>> goto out_exit_refcnt;
>> if (!blk_get_queue(disk->queue))
>> goto out_free_iostat;
>> + /* blkg holds a reference to blkcg */
>> + if (!css_tryget_online(&blkcg->css))
>> + goto out_put_queue;
>>
>> blkg->q = disk->queue;
>> INIT_LIST_HEAD(&blkg->q_node);
>> blkg->blkcg = blkcg;
>> blkg->iostat.blkg = blkg;
>> @@ -351,10 +357,12 @@ static struct blkcg_gq *blkg_alloc(struct blkcg *blkcg, struct gendisk *disk,
>>
>> out_free_pds:
>> while (--i >= 0)
>> if (blkg->pd[i])
>> blkcg_policy[i]->pd_free_fn(blkg->pd[i]);
>> + css_put(&blkcg->css);
>> +out_put_queue:
>> blk_put_queue(disk->queue);
>> out_free_iostat:
>> free_percpu(blkg->iostat_cpu);
>> out_exit_refcnt:
>> percpu_ref_exit(&blkg->refcnt);
>> @@ -379,32 +387,26 @@ static struct blkcg_gq *blkg_create(struct blkcg *blkcg, struct gendisk *disk,
>> if (blk_queue_dying(disk->queue)) {
>> ret = -ENODEV;
>> goto err_free_blkg;
>> }
>>
>> - /* blkg holds a reference to blkcg */
>> - if (!css_tryget_online(&blkcg->css)) {
>> - ret = -ENODEV;
>> - goto err_free_blkg;
>> - }
>> -
>> /* allocate */
>> if (!new_blkg) {
>> new_blkg = blkg_alloc(blkcg, disk, GFP_NOWAIT);
>> if (unlikely(!new_blkg)) {
>> ret = -ENOMEM;
>> - goto err_put_css;
>> + goto err_free_blkg;
>> }
>> }
>> blkg = new_blkg;
>>
>> /* link parent */
>> if (blkcg_parent(blkcg)) {
>> blkg->parent = blkg_lookup(blkcg_parent(blkcg), disk->queue);
>> if (WARN_ON_ONCE(!blkg->parent)) {
>> ret = -ENODEV;
>> - goto err_put_css;
>> + goto err_free_blkg;
>> }
>> blkg_get(blkg->parent);
>> }
>>
>> /* invoke per-policy init */
>> @@ -440,12 +442,10 @@ static struct blkcg_gq *blkg_create(struct blkcg *blkcg, struct gendisk *disk,
>>
>> /* @blkg failed fully initialized, use the usual release path */
>> blkg_put(blkg);
>> return ERR_PTR(ret);
>>
>> -err_put_css:
>> - css_put(&blkcg->css);
>> err_free_blkg:
>> if (new_blkg)
>> blkg_free(new_blkg);
>> return ERR_PTR(ret);
>> }
^ permalink raw reply
* Re: [PATCH RFC 0/1] block: fix concurrent elevator change failure
From: Shin'ichiro Kawasaki @ 2026-06-16 1:20 UTC (permalink / raw)
To: Nilay Shroff; +Cc: Ming Lei, linux-block, Jens Axboe
In-Reply-To: <3db036fe-747f-44eb-93c3-595350278297@linux.ibm.com>
On Jun 12, 2026 / 17:15, Nilay Shroff wrote:
> On 6/12/26 4:36 PM, Ming Lei wrote:
> > On Fri, Jun 12, 2026 at 06:47:50PM +0900, Shin'ichiro Kawasaki wrote:
> > > On Jun 11, 2026 / 06:22, Ming Lei wrote:
> > > > Hi Shin'ichiro,
> > >
> > > Hi Ming, thanks for the comments.
> > >
> > > >
> > > > On Thu, Jun 11, 2026 at 04:41:59PM +0900, Shin'ichiro Kawasaki wrote:
> > > > > I observed that the blktests test case block/005 hangs on a specific
> > > > > server hardware using a specific HDD as a block device. During the test
> > > > > case run, the kernel reported a KASAN null-ptr-deref (and other memory
> > > > > corruption symptoms) [2]. This failure looked sporadic and hardware-
> > > > > dependent.
> > > > >
> > > > > From the kernel message, I noticed that udev-worker wrote to the
> > > > > queue/scheduler sysfs attribute to change the IO scheduler, or elevator.
> > > > > The test case block/005 also wrote to the same sysfs attribute, which
> > > >
> > > > sysfs write is supposed to be serialized...
> > >
> > > I checked the sysfs write handler elv_iosched_store() in block/elevator.c.
> > > I found elevator_change() call is guarded with the rw_semaphore
> > > "set->update_nr_hwq_lock", but the guard is not the writer lock but the reader
> > > lock. This does not serialize the sysfs writes.
> >
> > Please see kernfs_fop_write_iter(), in which mutex is held before calling
> > ->write().
> >
> I think you're referring to @of->mutex here; however of->mutex is per struct
> kernfs_open_file, which is associated with an open instance of the sysfs file.
> The important point is that two separate opens can have different kernfs_open_file
> instances and therefore different mutexes. Thus, concurrent write to same sysfs
> attribute from two different processes may still be possible.
Thanks Nilay, I added debug prints to print @of->mutex address, and it observed
the address is different for each process and each file open. So, I don't think
sysfs write is serialized.
>
>
> > >
> > > I tried the patch below to replace the reader lock with the writer lock. With
> > > a quick trial, it looks working. The kernel message is no longer observed and
> > > the new test case does not cause hangs. I will do further testing to confirm
> > > that this change does not trigger other new lockdep WARNs. Assuming it does not
> > > have such side effects, I hope this fix approach is acceptable. It doesn't add
> > > the new lock, so I think it's the better.
> > >
> > > diff --git a/block/elevator.c b/block/elevator.c
> > > index 3bcd37c2aa34..b03185a217ff 100644
> > > --- a/block/elevator.c
> > > +++ b/block/elevator.c
> > > @@ -813,7 +813,7 @@ ssize_t elv_iosched_store(struct gendisk *disk, const char *buf,
> > > * update_nr_hwq_lock -> kn->active (via del_gendisk -> kobject_del)
> > > * kn->active -> update_nr_hwq_lock (via this sysfs write path)
> > > */
> > > - if (!down_read_trylock(&set->update_nr_hwq_lock)) {
> > > + if (!down_write_trylock(&set->update_nr_hwq_lock)) {
> > > ret = -EBUSY;
> > > goto out;
> > > }
> > > @@ -824,7 +824,7 @@ ssize_t elv_iosched_store(struct gendisk *disk, const char *buf,
> > > } else {
> > > ret = -ENOENT;
> > > }
> > > - up_read(&set->update_nr_hwq_lock);
> > > + up_write(&set->update_nr_hwq_lock);
> > > out:
> > > if (ctx.type)
> > >
> > > [...]
> > >
> > > > blk_mq_sched_reg_debugfs already includes debugfs lock, so I feel the proper
> > > > fix could be check & avoid the null-ptr-deref.
> > >
> > > Actually, null-ptr-deref is one of the failure symptoms. KASAN slab-user-after
> > > free is also observed [3]. Then I'm guessing adding null checks may not be
> > > enough.
> > >
> > > > Adding new lock should be the last straw usually, especially this one is
> > > > depended by queue freeze.
> > >
> > > Got it, thanks.
> > >
> > >
> > > [3] KASAN slab-use-after-free
> >
> > Then you need to figure out the exact slab type and check if the pointer is cleared
> > during free.
> >
> > Anyway, there is guard already, not see reason to add new lock for covering
> > it.
> >
> Regarding the observed failure, my understanding is that blk_mq_debugfs_register_sched()
> and blk_mq_debugfs_register_sched_hctx() access q->elevator without holding q->elevator_lock.
> If multiple scheduler update paths run concurrently, one path can replace and free the
> elevator while another path is still using it, which would explain the observed KASAN
> use-after-free and NULL pointer dereference reports.
I have the same view. I think the use-after-free and the null-ptr-deref indicate
that elevator_queue object address in q->elevator is the problem. The references
of the object is also kept in the struct elv_change_ctx as ctx->old and
ctx->new. These multiple references are used concurrently, then I'm not sure if
adding pointer clears and null checks would fix the problem.
>
> With the proposed change, upgrading update_nr_hwq_lock from a reader lock to a writer
> lock in elv_iosched_store() would serialize concurrent scheduler updates and therefore
> prevent multiple elevator switch operations from running at the same time.
>
> The another way to fix this might be to acquire q->elevator_lock in blk_mq_sched_reg_debugfs()
> and thus serialize access to q->elevator in blk_mq_debugfs_register_sched() and
> blk_mq_debugfs_register_sched_hctx().
Thanks for the idea. I tried the patch below [X], but it triggered WARN in
debugfs_create_files() in block/blk-mq-debufs.c [Y]. Then I'm afraid, this
approach does not look working.
At this moment, the writer lock in elv_iosched_store() looks like the solution
to me, but further comments on other solution possibility will be welcomed.
[X]
diff --git a/block/blk-mq-sched.c b/block/blk-mq-sched.c
index 0a00f5a76f5a..12c582b6c713 100644
--- a/block/blk-mq-sched.c
+++ b/block/blk-mq-sched.c
@@ -394,9 +394,11 @@ void blk_mq_sched_reg_debugfs(struct request_queue *q)
unsigned long i;
memflags = blk_debugfs_lock(q);
+ mutex_lock(&q->elevator_lock);
blk_mq_debugfs_register_sched(q);
queue_for_each_hw_ctx(q, hctx, i)
blk_mq_debugfs_register_sched_hctx(q, hctx);
+ mutex_unlock(&q->elevator_lock);
blk_debugfs_unlock(q, memflags);
}
[Y]
612 static void debugfs_create_files(struct request_queue *q, struct dentry *parent,|
613 void *data, |
614 const struct blk_mq_debugfs_attr *attr) |
615 { |
616 lockdep_assert_held(&q->debugfs_mutex); |
617 /* |
618 * debugfs_mutex should not be nested under other locks that can be |
619 * grabbed while queue is frozen. |
620 */ |
621 lockdep_assert_not_held(&q->elevator_lock); | <----
622 lockdep_assert_not_held(&q->rq_qos_mutex); |
623 |
^ permalink raw reply related
* Re: [PATCH v3 3/4] iomap: reject NOWAIT and BOUNCE direct IOs
From: Qu Wenruo @ 2026-06-16 1:00 UTC (permalink / raw)
To: Christoph Hellwig, Qu Wenruo
Cc: linux-btrfs, linux-block, linux-fsdevel, linux-xfs
In-Reply-To: <efa3f1f9-20c2-44c0-914c-5e34a40235a1@gmx.com>
在 2026/6/16 08:13, Qu Wenruo 写道:
>
>
> 在 2026/6/16 00:35, Christoph Hellwig 写道:
>> On Fri, Jun 12, 2026 at 07:21:14PM +0930, Qu Wenruo wrote:
>>> If a direct IO requires bounced pages for stable buffer, it will always
>>> allocate memory, and both bio_iov_iter_bounce_write() and
>>> bio_iov_iter_bounce_read() are allocating pages using GFP_KERNEL, which
>>> can sleep and break NOWAIT requirement.
>>>
>>> So we need to reject such NOWAIT and BOUNCE direct IO in
>>> iomap_dio_bio_iter().
>>
>> That's a bit heavy handed. Just do a noretry allocation.
>
> From the comment of __GFP_NORETRY:
>
> * %__GFP_NORETRY: The VM implementation will try only very lightweight
> * memory direct reclaim to get some memory under memory pressure (thus
> * it can sleep).
>
> It looks like NORETRY can still sleep, thus again breaking NOWAIT
> requirement.
>
> I think you're talking about GFP_NOWAIT?
>
After scanning the code for related memory allocation, there are some
other locations doing memory allocation, including but not limited to:
- iomap_dio_alloc_bio()
This one is a little tricky, if we pass GFP_NOWAIT, we can break the
old assumption that the function will always return a bio.
- fscrypt_set_bio_crypt_ctx()
This one is fine so far, as neither XFS nor btrfs support fscrypt yet.
And any memory allocation failure in NOWAIT mode should return -EAGAIN,
not -ENOMEM so that the caller can retry in blocking mode as a fallback.
To me, considering NOWAIT itself is only an optimistic flag, and caller
should always have a blocking mode as fallback, I'd prefer to reject
NOWAIT + BOUNCE direct writes completely inside btrfs for now.
And leave all the missing NOWAIT handling in iomap in a dedicated series.
Would that be acceptable to you?
Thanks,
Qu
^ permalink raw reply
* Re: [PATCH 12/19] x86: define DPS root partition type UUIDs
From: Dave Hansen @ 2026-06-16 0:09 UTC (permalink / raw)
To: Vincent Mailhol, Jens Axboe, Davidlohr Bueso, Alexander Viro,
Christian Brauner, Jan Kara
Cc: linux-kernel, linux-block, linux-efi, linux-fsdevel,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen, x86
In-Reply-To: <ea71a433-fdcc-429d-adce-14e9fb726957@kernel.org>
On 6/15/26 13:19, Vincent Mailhol wrote:
...
> That said, your points make sense to me, and I would be supportive to
> allow a search for a secondary UUID as a kernel extension. If we do
> so, I think the only constraint should be to make sure that we check
> for the exact match first (e.g. check x86_64 type before x86_32 type).
>
> Would that make sense?
Yep, that makes sense to me.
>> 2. Should the UUIDs be defined in arch code or generic code?
>
> I think that you convinced me to put it in generic code.
>
>> 3. Kconfig or #ifdefs?
>
> I would say Kconfig. If we go for the exact match only, that would be:
>
> CONFIG_DPS_ROOT_PARTITION_TYPE_UUID
>
> If we allow more as an extension, that would become:
>
> - CONFIG_DPS_ROOT_PARTITION_TYPE_UUID for the exact match
> - CONFIG_DPS_ROOT_PARTITION_TYPE_UUID_SECONDARY for the compatible
> one.
>
> The drawback is that some entries will be in both:
>
> config DPS_ROOT_PARTITION_TYPE_UUID
> string
> default "4f68bce3-e8cd-4db1-96e7-fbcaf984b709" if X86_64
> default "44479540-f297-41b2-9af7-d131d5f0458a" if X86
>
> config DPS_ROOT_PARTITION_TYPE_UUID_SECONDARY
> string
> default "44479540-f297-41b2-9af7-d131d5f0458a" if X86_64 && COMPAT_32
>
> And I don't think we need more than two.
That's not ideal, but it's also a completely static thing that will get
written very, very rarely.
> A bonus question: should those Kconfig entries be hidden? I prefer the
> hidden option because it doesn't add that much code and I thought this
> was not worth bothering the user with one more menuconfig question.
> But I would be happy to change if people this this is worth an
> menuconfig entry.
Yeah, it should be hidden. Anybody that wants to change it for whatever
reason can edit the .config file or hack Kconfig.
^ permalink raw reply
* Re: Repeatable, raid1+O_DIRECT, hang/warn
From: Keith Busch @ 2026-06-16 0:06 UTC (permalink / raw)
To: Vjaceslavs Klimovs
Cc: Dr. David Alan Gilbert, Thorsten Leemhuis, trnka, linux-block,
dm-devel, Linux kernel regressions list
In-Reply-To: <CAC_j7i0eDccVWzPeRafM50mZEOFHPz2cwd=RZqqx6TK2EVRFvw@mail.gmail.com>
On Mon, Jun 15, 2026 at 04:16:12PM -0700, Vjaceslavs Klimovs wrote:
> Your trace looks like what the two earlier reports hit: a read reaching
> a leaf device with sectors > 0 but phys_seg 0 (an empty bio). One aside
> that may help read the trace: blk_io_trace.error is a __u16, so the
> bracketed values on your C lines are errnos as u16 (65514 = -EINVAL,
> 65531 = -EIO).
>
> The WARN itself is new, the bad bio isn't. bio_add_page() only started
> rejecting len == 0 in 643893647cac ("block: reject zero length in
> bio_add_page()", v7.1-rc1); on 7.0.8 the same empty bio tripped
> scsi_alloc_sgtables()'s !nr_segs instead, which matches what you saw.
> That fits your "not a recent regression": the condition is older, v7.1
> just made it loud.
>
> For Tomas's and my reports (QEMU O_DIRECT to the LV block device) the
> origin looks like 5ff3f74e145a ("block: simplify direct io validity
> check", v6.18): blkdev_dio_invalid() now checks only aggregate
> ki_pos | count alignment and dropped the per-segment
> bdev_iter_is_aligned() walk, so a degenerate or misaligned O_DIRECT no
> longer gets -EINVAL at the fops boundary. But your reproducer reads a
> file, which goes through the filesystem O_DIRECT path and never calls
> blkdev_dio_invalid(), and still makes the empty bio. So it isn't only
> that one entry point.
>
> dm-mirror then hangs because Keith's f7b24c7b41f2 only covers md
> raid1/raid10; legacy dm-mirror (dm-raid1.c) has no equivalent and
> rebuilds the empty read onto the other leg. Note the leg's status isn't
> even consistent (your SATA path returns BLK_STS_IOERR, not
> BLK_STS_INVAL), so copying that status check into dm-mirror probably
> wouldn't catch every case.
>
> For what it's worth, that points me toward rejecting the empty or
> misaligned bio once, at submission, with -EINVAL, rather than teaching
> each consumer to tolerate it. But you'll know the tradeoffs far better
> than I do.
>
> I have a small QEMU + LVM raid1/mirror setup that reproduces the
> block-device variant and bisects to 5ff3f74e. Happy to run your file
> reproducer with some instrumentation at the dm-mirror read entry
> (bi_size vs bio_sectors vs bvec lengths) to see whether the bio is
> already empty on arrival or built that way on the retry, and to test
> any patch.
Thanks for following up here. I didn't initially see your follow-up
until Thorsten linked it. I apologize for missing that, this feature is
important so I don't want to see anything regress for it.
There is a known bug fix I think future tests should include:
https://lore.kernel.org/linux-block/20260612223205.465913-1-kbusch@meta.com/
This likely isn't the fix you're looking for, but including it rules out
conditions that are not important here.
After that, can we try this suggestion and see if the hang goes away?
https://lore.kernel.org/linux-block/ajBb8tK-0aJBpIgF@kbusch-mbp/
I expect the original test case to still return an error (and I think it
was designed to), but it shouldn't produce the warn or bug splats with a
stuck uninterruptable task.
^ permalink raw reply
* Re: Repeatable, raid1+O_DIRECT, hang/warn
From: Vjaceslavs Klimovs @ 2026-06-15 23:16 UTC (permalink / raw)
To: Dr. David Alan Gilbert
Cc: Thorsten Leemhuis, kbusch, trnka, linux-block, dm-devel,
Linux kernel regressions list
In-Reply-To: <ai_1LYtofh1fwD-N@gallifrey>
Hi Dave, all,
I'm one of the original reporters and very much a user, not a block/dm
developer, so please sanity-check all of this.
Your trace looks like what the two earlier reports hit: a read reaching
a leaf device with sectors > 0 but phys_seg 0 (an empty bio). One aside
that may help read the trace: blk_io_trace.error is a __u16, so the
bracketed values on your C lines are errnos as u16 (65514 = -EINVAL,
65531 = -EIO).
The WARN itself is new, the bad bio isn't. bio_add_page() only started
rejecting len == 0 in 643893647cac ("block: reject zero length in
bio_add_page()", v7.1-rc1); on 7.0.8 the same empty bio tripped
scsi_alloc_sgtables()'s !nr_segs instead, which matches what you saw.
That fits your "not a recent regression": the condition is older, v7.1
just made it loud.
For Tomas's and my reports (QEMU O_DIRECT to the LV block device) the
origin looks like 5ff3f74e145a ("block: simplify direct io validity
check", v6.18): blkdev_dio_invalid() now checks only aggregate
ki_pos | count alignment and dropped the per-segment
bdev_iter_is_aligned() walk, so a degenerate or misaligned O_DIRECT no
longer gets -EINVAL at the fops boundary. But your reproducer reads a
file, which goes through the filesystem O_DIRECT path and never calls
blkdev_dio_invalid(), and still makes the empty bio. So it isn't only
that one entry point.
dm-mirror then hangs because Keith's f7b24c7b41f2 only covers md
raid1/raid10; legacy dm-mirror (dm-raid1.c) has no equivalent and
rebuilds the empty read onto the other leg. Note the leg's status isn't
even consistent (your SATA path returns BLK_STS_IOERR, not
BLK_STS_INVAL), so copying that status check into dm-mirror probably
wouldn't catch every case.
For what it's worth, that points me toward rejecting the empty or
misaligned bio once, at submission, with -EINVAL, rather than teaching
each consumer to tolerate it. But you'll know the tradeoffs far better
than I do.
I have a small QEMU + LVM raid1/mirror setup that reproduces the
block-device variant and bisects to 5ff3f74e. Happy to run your file
reproducer with some instrumentation at the dm-mirror read entry
(bi_size vs bio_sectors vs bvec lengths) to see whether the bio is
already empty on arrival or built that way on the retry, and to test
any patch.
Thanks,
Vjaceslavs
On Mon, Jun 15, 2026 at 5:50 AM Dr. David Alan Gilbert
<linux@treblig.org> wrote:
>
> * Thorsten Leemhuis (regressions@leemhuis.info) wrote:
> > On 6/14/26 19:57, Dr. David Alan Gilbert wrote:
> > >
> > > I've got a repeatable raid hang/warn and would appreciate some pointers
> > > as where to debug.
> > > (I've been logging stuff on https://bugzilla.kernel.org/show_bug.cgi?id=221535 )
> >
> > Note: not my area of expertise, so I might be sending you totally
> > off-track with this comment. Feel free to ignore it. But FWIW:
>
> Hi Thorsten,
> Thanks for the reply - these do seem to be related!
> (So copying in Keith, Vjaceslavs, and Tomáš )
> (Not my area either).
>
> > Have you seen these reports?
> > https://lore.kernel.org/all/2982107.4sosBPzcNG@electra/
> > https://lore.kernel.org/all/CAC_j7i1R7oy+nRhxEjCTba=DUgn02w9X+p94DCu0aHv5+5tKnQ@mail.gmail.com/
>
> I hadn't! Those are both the problem I originally was trying to debug
> and stumbled into the WARN/BUG/hang with my test program.
>
> > The former lead to a fix in the mdraid code that should be in the kernel
> > version you are using. But in a reply to the latter report the repoter
> > claimed that that fix is not enough (claiming "this was obvious" and
> > also using dm), but things then stalled there.
>
> Yeh I see my world has Keith's f7b24c7b41f23
>
> I think the problem I'm seeing is zero length requests coming from somewhere.
>
> The WARN I'm seeing in 7.1.0-rc7+ is:
>
> [ 2681.597042] device-mapper: raid1: Mirror read failed from 252:25. Trying alternative device.
> [ 2681.631933] ------------[ cut here ]------------
> [ 2681.631939] WARNING: block/bio.c:1044 at bio_add_page+0x18b/0x250, CPU#22: kworker/22:0/18929
>
> 1039 int bio_add_page(struct bio *bio, struct page *page,
> 1040 unsigned int len, unsigned int offset)
> 1041 {
> 1042 if (WARN_ON_ONCE(bio_flagged(bio, BIO_CLONED)))
> 1043 return 0;
> 1044 if (WARN_ON_ONCE(len == 0))
> 1045 return 0;
>
> So it's the ' if (WARN_ON_ONCE(len == 0))'
>
> and the warn I got on the older 7.0.8 was:
> [Sun May 17 17:22:52 2026] WARNING: drivers/scsi/scsi_lib.c:1140 at scsi_alloc_sgtables+0x38a/0x400, CPU#28: kworker/28:1H/3943
>
> which I *think* corresponds to:
> 1164 if (WARN_ON_ONCE(!nr_segs))
> 1165 return BLK_STS_IOERR;
>
> so it sounds like we need to find where zero length requests are coming from??
>
> Thanks again,
>
> Dave
>
> > Ciao, Thorsten
> >
> > > This started off as debugging a case where I'd get my RAID1 (on the host)
> > > getting a reliable 'rescheduling sector'/disk failure while running the qemu block test suite
> > > during a qemu build, but then I tried to build a smaller discrete
> > > test, and now I've got a simply triggerable warn and test hang.
> > > There's no errors from the underlying SATA layer on the storage,
> > > everything resyncs just fine.
> > >
> > > I've got an existing LVM vg ('main') with two mirrors on sda2, and sdb2
> > > which are SATA disks.
> > >
> > > # lvcreate --type mirror --mirrors 1 -L 1G main /dev/sda2 /dev/sdb2
> > > # mkfs.ext4 /dev/mapper/main-lvol0
> > > # mount /dev/mapper/main-lvol0 /mnt/tmp/
> > > # chmod a+rwx /mnt/tmp
> > >
> > > $ dd if=/dev/zero of=/mnt/tmp/testfile bs=1024k count=1
> > >
> > > (I then wait for the IO to stop)
> > >
> > > then we've got this little test program:
> > >
> > > <--><--><--><--><--><--><--><--><--><--><--><--><--><--><--><--><--><-->
> > > #include <errno.h>
> > > #include <fcntl.h>
> > > #include <asm-generic/fcntl.h>
> > > #include <stdio.h>
> > > #include <unistd.h>
> > >
> > >
> > > const char* path="/mnt/tmp/testfile";
> > > static char buf[8192];
> > >
> > > int main()
> > > {
> > > int fd=open(path, O_RDWR|O_DIRECT|O_CLOEXEC);
> > >
> > > errno=0;
> > > int res3=pread(fd, buf, 4096, 0);
> > > printf("pread of 4096 said: %d (%m)\n", res3);
> > >
> > > }
> > > <--><--><--><--><--><--><--><--><--><--><--><--><--><--><--><--><--><-->
> > >
> > > running that, either hangs or gets a 'pread of 4096 said: -1 (Input/output error)'
> > > when it hangs it's unkillable.
> > >
> > > at the moment (on 7.1.0-rc7) this is giving:
> > > Jun 14 18:08:32 dalek kernel: device-mapper: raid1: Mirror read failed from 252:24. Trying alternative device.
> > > Jun 14 18:08:32 dalek kernel: ------------[ cut here ]------------
> > > Jun 14 18:08:32 dalek dmeventd[1010]: Primary mirror device 252:24 read failed.
> > > Jun 14 18:08:32 dalek kernel: WARNING: block/bio.c:1044 at bio_add_page+0x18b/0x250, CPU#15: kworker/15:1/369
> > >
> > > (full backtrace below)
> > > (Note there is a moan in there about sdb IO error - repeated a lot - but
> > > again, there's no SATA level errors, and the drive is fine on smart, and
> > > I can read the whole of the underlying lvm mirrors, so I don't think it's
> > > physically there).
> > >
> > > I did a blktrace, although that gives me a 23G blkparse output, hmm
> > > (I see each event repeated a lot - maybe per thread?)
> > >
> > > 252,26 15 1 0.000000000 3435 Q RS 264192 + 8 [dbf]
> > > 252,26 is /dev/mapper/main-lvol0
> > > 252,24 15 1 0.000005501 3435 A RS 264192 + 8 <- (252,26) 264192
> > > 252,24 is main-lvol0_mimage_0
> > > 252,24 15 2 0.000005761 3435 Q RS 264192 + 8 [dbf]
> > > 8,0 15 1 0.000008646 3435 A RS 71634944 + 8 <- (252,24) 264192
> > > so that's sda
> > > 8,0 15 2 0.000008787 3435 A RS 73734144 + 8 <- (8,2) 71634944
> > > I guess mapping down from sda2 to sda
> > > 8,0 15 3 0.000009037 3435 Q RS 73734144 + 8 [dbf]
> > > 8,0 15 4 0.000009809 3435 C RS 73734144 + 8 [65514]
> > > ??? Hmm what's the 65514 there?
> > > 252,24 15 3 0.000010320 3435 C RS 264192 + 8 [65514]
> > > 252,25 15 1 0.000290384 369 Q R 264192 + 8 [kworker/15:1]
> > > 252,25 is main-lvol0_mimage_1
> > >
> > > and at this point I'm a bit lost as to what I'm looking for.
> > >
> > > Hints appreciated!
> > >
> > > (I don't believe this is a regression - or at least not recent)
> > >
> > > Dave
> > >
> > >
> > >
> > >
> > > Jun 14 18:08:32 dalek kernel: device-mapper: raid1: Mirror read failed from 252:24. Trying alternative device.
> > > Jun 14 18:08:32 dalek kernel: ------------[ cut here ]------------
> > > Jun 14 18:08:32 dalek dmeventd[1010]: Primary mirror device 252:24 read failed.
> > > Jun 14 18:08:32 dalek kernel: WARNING: block/bio.c:1044 at bio_add_page+0x18b/0x250, CPU#15: kworker/15:1/369
> > > Jun 14 18:08:32 dalek dmeventd[1010]: main-lvol0 is now in-sync.
> > > Jun 14 18:08:32 dalek kernel: Modules linked in: nft_masq nft_reject_ipv4 act_csum cls_u32 sch_htb nf_nat_tftp nf_conntrack_tftp bridge stp llc rfkill nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reje>
> > > Jun 14 18:08:32 dalek kernel: drm_panel_backlight_quirks gpu_sched drm_suballoc_helper video nvme drm_display_helper nvme_core cec nvme_keyring sp5100_tco nvme_auth wmi serio_raw fuse scsi_dh_alua i2c_dev scsi_dh_rdac scsi_dh_emc
> > > Jun 14 18:08:32 dalek kernel: CPU: 15 UID: 0 PID: 369 Comm: kworker/15:1 Not tainted 7.1.0-rc7+ #786 PREEMPT(lazy)
> > > Jun 14 18:08:32 dalek kernel: Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./X570 Pro4, BIOS P3.10 07/13/2020
> > > Jun 14 18:08:32 dalek kernel: Workqueue: kmirrord do_mirror
> > > Jun 14 18:08:32 dalek kernel: RIP: 0010:bio_add_page+0x18b/0x250
> > > Jun 14 18:08:32 dalek kernel: Code: 24 10 4c 8b 04 24 84 c0 0f 85 c9 00 00 00 41 0f b7 40 78 48 8b 74 24 08 8b 4c 24 14 e9 b4 fe ff ff 0f 0b 31 c0 e9 55 d1 af 00 <0f> 0b eb f5 48 8b 7f 08 83 7f 60 05 0f 85 00 ff ff ff 49 8b 3b 4c
> > > Jun 14 18:08:32 dalek kernel: RSP: 0018:ffffd1fb8176fc10 EFLAGS: 00010246
> > > Jun 14 18:08:32 dalek kernel: RAX: 0000000000000000 RBX: ffffd1fb8176fd18 RCX: 0000000000000000
> > > Jun 14 18:08:32 dalek kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff8d1a8eb28b00
> > > Jun 14 18:08:32 dalek kernel: RBP: 0000000000000000 R08: ffffd1fb8176fc38 R09: ffffd1fb8176fc40
> > > Jun 14 18:08:32 dalek kernel: R10: ffffd1fb8176fc34 R11: 0000000000000000 R12: 0000000000000000
> > > Jun 14 18:08:32 dalek kernel: R13: ffffd1fb8176fd90 R14: 0000000000000001 R15: ffff8d1a8eb28b00
> > > Jun 14 18:08:32 dalek kernel: FS: 0000000000000000(0000) GS:ffff8d29d161f000(0000) knlGS:0000000000000000
> > > Jun 14 18:08:32 dalek kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > > Jun 14 18:08:32 dalek kernel: CR2: 00007f0ddcd7b9d0 CR3: 000000023dcbf000 CR4: 0000000000350ef0
> > > Jun 14 18:08:32 dalek kernel: Call Trace:
> > > Jun 14 18:08:32 dalek kernel: <TASK>
> > > Jun 14 18:08:32 dalek kernel: do_region+0x227/0x2a0
> > > Jun 14 18:08:32 dalek kernel: dispatch_io+0xf1/0x150
> > > Jun 14 18:08:32 dalek kernel: ? __pfx_bio_get_page+0x10/0x10
> > > Jun 14 18:08:32 dalek kernel: ? __pfx_bio_next_page+0x10/0x10
> > > Jun 14 18:08:32 dalek kernel: ? __pfx_read_callback+0x10/0x10
> > > Jun 14 18:08:32 dalek kernel: dm_io+0x169/0x2d0
> > > Jun 14 18:08:32 dalek kernel: ? __pfx_bio_get_page+0x10/0x10
> > > Jun 14 18:08:32 dalek kernel: ? __pfx_bio_next_page+0x10/0x10
> > > Jun 14 18:08:32 dalek kernel: do_reads+0x149/0x230
> > > Jun 14 18:08:32 dalek kernel: ? __pfx_read_callback+0x10/0x10
> > > Jun 14 18:08:32 dalek kernel: do_mirror+0x11a/0x2b0
> > > Jun 14 18:08:32 dalek kernel: process_one_work+0x19e/0x390
> > > Jun 14 18:08:32 dalek kernel: worker_thread+0x1a6/0x310
> > > Jun 14 18:08:32 dalek kernel: ? __pfx_worker_thread+0x10/0x10
> > > Jun 14 18:08:32 dalek kernel: kthread+0xe4/0x120
> > > Jun 14 18:08:32 dalek kernel: ? __pfx_kthread+0x10/0x10
> > > Jun 14 18:08:32 dalek kernel: ret_from_fork+0x1a1/0x270
> > > Jun 14 18:08:32 dalek kernel: ? __pfx_kthread+0x10/0x10
> > > Jun 14 18:08:32 dalek kernel: ret_from_fork_asm+0x1a/0x30
> > > Jun 14 18:08:32 dalek kernel: </TASK>
> > > Jun 14 18:08:32 dalek kernel: ---[ end trace 0000000000000000 ]---
> > > Jun 14 18:08:32 dalek kernel: ------------[ cut here ]------------
> > > Jun 14 18:08:32 dalek kernel: WARNING: drivers/scsi/scsi_lib.c:1164 at scsi_alloc_sgtables+0x38a/0x400, CPU#15: kworker/15:1/369
> > > Jun 14 18:08:32 dalek kernel: Modules linked in: nft_masq nft_reject_ipv4 act_csum cls_u32 sch_htb nf_nat_tftp nf_conntrack_tftp bridge stp llc rfkill nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reje>
> > > Jun 14 18:08:32 dalek kernel: drm_panel_backlight_quirks gpu_sched drm_suballoc_helper video nvme drm_display_helper nvme_core cec nvme_keyring sp5100_tco nvme_auth wmi serio_raw fuse scsi_dh_alua i2c_dev scsi_dh_rdac scsi_dh_emc
> > > Jun 14 18:08:32 dalek kernel: CPU: 15 UID: 0 PID: 369 Comm: kworker/15:1 Tainted: G W 7.1.0-rc7+ #786 PREEMPT(lazy)
> > > Jun 14 18:08:32 dalek kernel: Tainted: [W]=WARN
> > > Jun 14 18:08:32 dalek kernel: Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./X570 Pro4, BIOS P3.10 07/13/2020
> > > Jun 14 18:08:32 dalek kernel: Workqueue: kmirrord do_mirror
> > > Jun 14 18:08:32 dalek kernel: RIP: 0010:scsi_alloc_sgtables+0x38a/0x400
> > > Jun 14 18:08:32 dalek kernel: Code: 8b 3d ba 2d a9 01 e9 d1 fd ff ff 48 8b 75 00 48 8d bb f0 fe ff ff e8 15 b7 b0 ff 48 89 ab e0 00 00 00 89 45 08 e9 30 ff ff ff <0f> 0b 4c 8b 6c 24 30 b8 0a 00 00 00 e9 21 ff ff ff b8 09 00 00 00
> > > Jun 14 18:08:32 dalek kernel: RSP: 0018:ffffd1fb8176f7f0 EFLAGS: 00010246
> > > Jun 14 18:08:32 dalek kernel: RAX: 0000000000000000 RBX: ffff8d1aedad0110 RCX: 0000000000000009
> > > Jun 14 18:08:32 dalek kernel: RDX: 0000000000000000 RSI: ffffffff99c15960 RDI: ffff8d1aedad0110
> > > Jun 14 18:08:32 dalek kernel: RBP: ffff8d1a93d17000 R08: ffff8d1aedad0110 R09: ffff8d1a818fa800
> > > Jun 14 18:08:32 dalek kernel: R10: 7020676e69736961 R11: 0000000000000000 R12: 0000000000000000
> > > Jun 14 18:08:32 dalek kernel: R13: 0000000000000000 R14: ffff8d1a93394000 R15: ffff8d1a93d17000
> > > Jun 14 18:08:32 dalek kernel: FS: 0000000000000000(0000) GS:ffff8d29d161f000(0000) knlGS:0000000000000000
> > > Jun 14 18:08:32 dalek kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > > Jun 14 18:08:32 dalek kernel: CR2: 00007f0ddcd7b9d0 CR3: 000000023dcbf000 CR4: 0000000000350ef0
> > > Jun 14 18:08:32 dalek kernel: Call Trace:
> > > Jun 14 18:08:32 dalek kernel: <TASK>
> > > Jun 14 18:08:32 dalek kernel: ? srso_return_thunk+0x5/0x5f
> > > Jun 14 18:08:32 dalek kernel: sd_setup_read_write_cmnd+0x9d/0x740
> > > Jun 14 18:08:32 dalek kernel: ? srso_return_thunk+0x5/0x5f
> > > Jun 14 18:08:32 dalek kernel: scsi_queue_rq+0x4d2/0x890
> > > Jun 14 18:08:32 dalek kernel: blk_mq_dispatch_rq_list+0x241/0x530
> > > Jun 14 18:08:32 dalek kernel: ? srso_return_thunk+0x5/0x5f
> > > Jun 14 18:08:32 dalek kernel: ? sbitmap_get+0x61/0x100
> > > Jun 14 18:08:32 dalek kernel: __blk_mq_do_dispatch_sched+0x330/0x340
> > > Jun 14 18:08:32 dalek kernel: __blk_mq_sched_dispatch_requests+0x143/0x180
> > > Jun 14 18:08:32 dalek kernel: blk_mq_sched_dispatch_requests+0x2d/0x70
> > > Jun 14 18:08:32 dalek kernel: blk_mq_run_hw_queue+0x2bf/0x350
> > > Jun 14 18:08:32 dalek kernel: ? srso_return_thunk+0x5/0x5f
> > > Jun 14 18:08:32 dalek kernel: blk_mq_dispatch_list+0x172/0x350
> > > Jun 14 18:08:32 dalek kernel: blk_mq_flush_plug_list+0x51/0x1a0
> > > Jun 14 18:08:32 dalek kernel: ? blk_mq_submit_bio+0x71c/0x9f0
> > > Jun 14 18:08:32 dalek kernel: __blk_flush_plug+0x112/0x180
> > > Jun 14 18:08:32 dalek kernel: ? srso_return_thunk+0x5/0x5f
> > > Jun 14 18:08:32 dalek kernel: __submit_bio+0x19c/0x260
> > > Jun 14 18:08:32 dalek kernel: __submit_bio_noacct+0x8e/0x210
> > > Jun 14 18:08:32 dalek kernel: do_region+0x14c/0x2a0
> > > Jun 14 18:08:32 dalek kernel: dispatch_io+0xf1/0x150
> > > Jun 14 18:08:32 dalek kernel: ? __pfx_bio_get_page+0x10/0x10
> > > Jun 14 18:08:32 dalek kernel: ? __pfx_bio_next_page+0x10/0x10
> > > Jun 14 18:08:32 dalek kernel: ? __pfx_read_callback+0x10/0x10
> > > Jun 14 18:08:32 dalek kernel: dm_io+0x169/0x2d0
> > > Jun 14 18:08:32 dalek kernel: ? __pfx_bio_get_page+0x10/0x10
> > > Jun 14 18:08:32 dalek kernel: ? __pfx_bio_next_page+0x10/0x10
> > > Jun 14 18:08:32 dalek kernel: do_reads+0x149/0x230
> > > Jun 14 18:08:32 dalek kernel: ? __pfx_read_callback+0x10/0x10
> > > Jun 14 18:08:32 dalek kernel: do_mirror+0x11a/0x2b0
> > > Jun 14 18:08:32 dalek kernel: process_one_work+0x19e/0x390
> > > Jun 14 18:08:32 dalek kernel: worker_thread+0x1a6/0x310
> > > Jun 14 18:08:32 dalek kernel: ? __pfx_worker_thread+0x10/0x10
> > > Jun 14 18:08:32 dalek kernel: kthread+0xe4/0x120
> > > Jun 14 18:08:32 dalek kernel: ? __pfx_kthread+0x10/0x10
> > > Jun 14 18:08:32 dalek kernel: ret_from_fork+0x1a1/0x270
> > > Jun 14 18:08:32 dalek kernel: ? __pfx_kthread+0x10/0x10
> > > Jun 14 18:08:32 dalek kernel: ret_from_fork_asm+0x1a/0x30
> > > Jun 14 18:08:32 dalek kernel: </TASK>
> > > Jun 14 18:08:32 dalek kernel: ---[ end trace 0000000000000000 ]---
> > > Jun 14 18:08:32 dalek kernel: I/O error, dev sdb, sector 50606087 op 0x0:(READ) flags 0x0 phys_seg 0 prio class 2
> > > Jun 14 18:08:32 dalek kernel: I/O error, dev sdb, sector 50606087 op 0x0:(READ) flags 0x0 phys_seg 0 prio class 2
> > > Jun 14 18:08:32 dalek kernel: I/O error, dev sdb, sector 50606087 op 0x0:(READ) flags 0x0 phys_seg 0 prio class 2
> > > Jun 14 18:08:32 dalek kernel: I/O error, dev sdb, sector 50606087 op 0x0:(READ) flags 0x0 phys_seg 0 prio class 2
> > > Jun 14 18:08:32 dalek kernel: I/O error, dev sdb, sector 50606087 op 0x0:(READ) flags 0x0 phys_seg 0 prio class 2
> > > Jun 14 18:08:32 dalek kernel: I/O error, dev sdb, sector 50606087 op 0x0:(READ) flags 0x0 phys_seg 0 prio class 2
> > > Jun 14 18:08:32 dalek kernel: I/O error, dev sdb, sector 50606087 op 0x0:(READ) flags 0x0 phys_seg 0 prio class 2
> > > Jun 14 18:08:32 dalek kernel: I/O error, dev sdb, sector 50606087 op 0x0:(READ) flags 0x0 phys_seg 0 prio class 2
> > > Jun 14 18:08:32 dalek kernel: I/O error, dev sdb, sector 50606087 op 0x0:(READ) flags 0x0 phys_seg 0 prio class 2
> > > Jun 14 18:08:32 dalek kernel: I/O error, dev sdb, sector 50606087 op 0x0:(READ) flags 0x0 phys_seg 0 prio class 2
> > > Jun 14 18:08:37 dalek kernel: blk_print_req_error: 241000 callbacks suppressed
> > > Jun 14 18:08:37 dalek kernel: I/O error, dev sdb, sector 50606087 op 0x0:(READ) flags 0x0 phys_seg 0 prio class 2
> > >
> > >
> >
> --
> -----Open up your eyes, open up your mind, open up your code -------
> / Dr. David Alan Gilbert | Running GNU/Linux | Happy \
> \ dave @ treblig.org | | In Hex /
> \ _________________________|_____ http://www.treblig.org |_______/
^ permalink raw reply
* Re: [PATCH v4 0/3] crypto: skcipher - per-request multi-data-unit batching
From: Eric Biggers @ 2026-06-15 22:53 UTC (permalink / raw)
To: Leonid Ravich
Cc: Herbert Xu, Alasdair Kergon, Ard Biesheuvel, Jens Axboe,
Horia Geanta, Gilad Ben-Yossef, linux-crypto, dm-devel,
linux-block
In-Reply-To: <20260615111459.9452-1-lravich@amazon.com>
On Mon, Jun 15, 2026 at 11:14:56AM +0000, Leonid Ravich wrote:
> The series adds a per-request "data unit size" to the skcipher API
> so a caller can submit several data units (typically 512..4096-byte
> sectors) sharing one starting IV in a single request. Algorithms
> derive each data unit's IV from the caller-supplied IV by treating
> it as a 128-bit little-endian counter and adding the data-unit
> index, matching the layout produced by dm-crypt's plain64 IV mode
> and by typical inline-encryption hardware.
>
> This mirrors the data_unit_size concept already exposed by
> struct blk_crypto_config for inline encryption.
>
> The first user is dm-crypt, which today issues one skcipher request
> per sector and so pays a per-sector cost in request allocation,
> callback dispatch, completion handling, and scatterlist setup.
>
> Proof-of-concept performance numbers from the RFC reply [1]: +19%
> throughput / -40% CPU on a single-core arm64 system with a hardware
> XTS-AES-256 accelerator running fio 4 KiB sequential writes through
> dm-crypt, when an out-of-tree arm64 xts driver advertises
> CRYPTO_ALG_SKCIPHER_NATIVE_MULTI_DU. This series itself does not
> include arch enablement; the fast path is opt-in per driver, the
> slow path is universal via the auto-splitter.
>
> The native fast path amortises both per-sector dispatch and per-sector
> crypto setup across a bio - the measured win above, on an engine that
> offloads the AES compute. The auto-splitter is for correctness and
> reach: any consumer can set data_unit_size and get correct output with
> the per-request allocation/callback/completion cost removed, but it
> still issues one alg->encrypt per data unit, so on a software cipher it
> saves only dispatch overhead (no throughput figure claimed - that is
> hardware- and workload-dependent). What it guarantees unconditionally
> is byte-identical output (Verification below) at O(entries + units),
> walking the scatterlists with a pair of struct scatter_walk cursors
> rather than rescanning from the head per unit.
So in other words, this series slows down dm-crypt and crypto_skcipher
for everyone to optimize for an out-of-tree driver. And there's also no
benchmark showing that your driver is even worth it over just using the
CPU.
- Eric
^ permalink raw reply
* Re: [PATCH v3 3/4] iomap: reject NOWAIT and BOUNCE direct IOs
From: Qu Wenruo @ 2026-06-15 22:43 UTC (permalink / raw)
To: Christoph Hellwig, Qu Wenruo
Cc: linux-btrfs, linux-block, linux-fsdevel, linux-xfs
In-Reply-To: <ajAU1yLd32BCiCNj@infradead.org>
在 2026/6/16 00:35, Christoph Hellwig 写道:
> On Fri, Jun 12, 2026 at 07:21:14PM +0930, Qu Wenruo wrote:
>> If a direct IO requires bounced pages for stable buffer, it will always
>> allocate memory, and both bio_iov_iter_bounce_write() and
>> bio_iov_iter_bounce_read() are allocating pages using GFP_KERNEL, which
>> can sleep and break NOWAIT requirement.
>>
>> So we need to reject such NOWAIT and BOUNCE direct IO in
>> iomap_dio_bio_iter().
>
> That's a bit heavy handed. Just do a noretry allocation.
From the comment of __GFP_NORETRY:
* %__GFP_NORETRY: The VM implementation will try only very lightweight
* memory direct reclaim to get some memory under memory pressure (thus
* it can sleep).
It looks like NORETRY can still sleep, thus again breaking NOWAIT
requirement.
I think you're talking about GFP_NOWAIT?
^ permalink raw reply
* Re: [PATCH] sunvdc: fix -EIO issue due to lack of retries
From: John Paul Adrian Glaubitz @ 2026-06-15 22:09 UTC (permalink / raw)
To: Jens Axboe, linux-block@vger.kernel.org
In-Reply-To: <418310b3-2b77-4534-b2fd-27dcc11e333c@kernel.dk>
Hi,
On Mon, 2025-10-06 at 08:59 -0600, Jens Axboe wrote:
> John reports that since commit:
>
> a11f6ca9aef9 ("sunvdc: Do not spin in an infinite loop when vio_ldc_send() returns EAGAIN")
>
> users of Linux inside Solaris ldom see occasional -EIO errors because
> the request send loop now times out. The current loop does 10 retries,
> and inside vio_ldc_send() a further 1000 1usec retries are done as well.
> Even with 10.5 msec of busy loop retries that's apparently not enough to
> always succeed.
>
> Rather than introduce continued busy looping, requeue the request and
> have the delayed queue kicking retry the request after another 10ms.
> This obviously isn't ideal, but there's seemingly no way to wait for
> this type of event. And if 10ms of busy looping was not enough to make
> progress, then presumably this is an edge condition and we just need to
> guarantee to make forward progress at some later point in time. That's
> more suitably done through letting the CPU tend to other work, rather
> than sitting in a tight loop retrying.
>
> Reported-by: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de>
> Link: https://lore.kernel.org/all/20251006100226.4246-2-glaubitz@physik.fu-berlin.de/
> Signed-off-by: Jens Axboe <axboe@kernel.dk>
>
> ---
>
> Caveat: 100% untested, not even compiled. Sending out on John's behest.
>
> diff --git a/drivers/block/sunvdc.c b/drivers/block/sunvdc.c
> index db1fe9772a4d..aa49dffb1b53 100644
> --- a/drivers/block/sunvdc.c
> +++ b/drivers/block/sunvdc.c
> @@ -539,6 +539,7 @@ static blk_status_t vdc_queue_rq(struct blk_mq_hw_ctx *hctx,
> struct vdc_port *port = hctx->queue->queuedata;
> struct vio_dring_state *dr;
> unsigned long flags;
> + int ret;
>
> dr = &port->vio.drings[VIO_DRIVER_TX_RING];
>
> @@ -560,7 +561,13 @@ static blk_status_t vdc_queue_rq(struct blk_mq_hw_ctx *hctx,
> return BLK_STS_DEV_RESOURCE;
> }
>
> - if (__send_request(bd->rq) < 0) {
> + ret = __send_request(bd->rq);
> + if (ret == -EAGAIN) {
> + spin_unlock_irqrestore(&port->vio.lock, flags);
> + /* already spun for 10msec, defer 10msec and retry */
> + blk_mq_delay_kick_requeue_list(hctx->queue, 10);
> + return BLK_STS_DEV_RESOURCE;
> + } else if (ret < 0) {
> spin_unlock_irqrestore(&port->vio.lock, flags);
> return BLK_STS_IOERR;
> }
I will give this patch a try this week as I finally want to get this fixed.
Adrian
--
.''`. John Paul Adrian Glaubitz
: :' : Debian Developer
`. `' Physicist
`- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox