* [PATCH v5 03/21] Add a function to kmap one page of a multipage bio_vec
From: David Howells @ 2026-07-06 15:33 UTC (permalink / raw)
To: Christian Brauner, Matthew Wilcox, Christoph Hellwig
Cc: David Howells, Paulo Alcantara, Jens Axboe, Leon Romanovsky,
Steve French, ChenXiaoSong, Marc Dionne, Stefan Metzmacher,
Eric Van Hensbergen, Dominique Martinet, Ilya Dryomov, netfs,
linux-afs, linux-cifs, linux-nfs, ceph-devel, v9fs, linux-erofs,
linux-fsdevel, linux-kernel, linux-block
In-Reply-To: <20260706153408.1231650-1-dhowells@redhat.com>
Add a function to kmap one page of a multipage bio_vec by offset (which is
added to the offset in the bio_vec internally). The caller is responsible
for calculating how much of the page is then available.
Signed-off-by: David Howells <dhowells@redhat.com>
Acked-by: Paulo Alcantara (Red Hat) <pc@manguebit.org>
cc: Matthew Wilcox <willy@infradead.org>
cc: Christoph Hellwig <hch@infradead.org>
cc: Jens Axboe <axboe@kernel.dk>
cc: linux-block@vger.kernel.org
cc: netfs@lists.linux.dev
cc: linux-fsdevel@vger.kernel.org
---
include/linux/bvec.h | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/include/linux/bvec.h b/include/linux/bvec.h
index 92837e2743f1..53cf36e73967 100644
--- a/include/linux/bvec.h
+++ b/include/linux/bvec.h
@@ -343,4 +343,22 @@ static inline phys_addr_t bvec_phys(const struct bio_vec *bvec)
return page_to_phys(bvec->bv_page) + bvec->bv_offset;
}
+/**
+ * bvec_kmap_partial - Map part of a bvec into the kernel virtual address space
+ * @bvec: bvec to map
+ * @offset: Offset into bvec
+ *
+ * Map the page containing the byte at @offset into the kernel virtual address
+ * space. The caller is responsible for making sure this doesn't overrun.
+ *
+ * Call kunmap_local on the returned address to unmap.
+ */
+static inline void *bvec_kmap_partial(struct bio_vec *bvec, size_t offset)
+{
+ offset += bvec->bv_offset;
+
+ return kmap_local_page(bvec->bv_page + (offset >> PAGE_SHIFT)) +
+ (offset & ~PAGE_MASK);
+}
+
#endif /* __LINUX_BVEC_H */
^ permalink raw reply related
* [PATCH v5 05/21] iov_iter: Add a segmented queue of bio_vec[]
From: David Howells @ 2026-07-06 15:27 UTC (permalink / raw)
To: Christian Brauner, Matthew Wilcox, Christoph Hellwig
Cc: David Howells, Paulo Alcantara, Jens Axboe, Leon Romanovsky,
Steve French, ChenXiaoSong, Marc Dionne, Stefan Metzmacher,
Eric Van Hensbergen, Dominique Martinet, Ilya Dryomov, netfs,
linux-afs, linux-cifs, linux-nfs, ceph-devel, v9fs, linux-erofs,
linux-fsdevel, linux-kernel, linux-block
In-Reply-To: <20260706152737.1231312-1-dhowells@redhat.com>
Add the concept of a segmented queue of bio_vec[] arrays. This allows an
indefinite quantity of elements to be handled and allows things like
network filesystems and crypto drivers to glue bits on the ends without
having to reallocate the array.
The bvecq struct that defines each segment also carries capacity/usage
information along with flags indicating whether the constituent memory
regions need freeing or unpinning and the file position of the first
element in a segment. The bvecq structs are refcounted to allow a queue to
be extracted in batches and split between a number of subrequests.
The bvecq can have the bio_vec[] it manages allocated in with it, but this
is not required. A flag is provided for if this is the case as comparing
->bv to ->__bv is not sufficient to detect this case.
Add an iterator type ITER_BVECQ for it. This is intended to replace
ITER_FOLIOQ (and ITER_XARRAY).
Note that the prev pointer is only really needed for iov_iter_revert() and
could be dispensed with if struct iov_iter contained the head information
as well as the current point.
Signed-off-by: David Howells <dhowells@redhat.com>
Reviewed-by: Paulo Alcantara (Red Hat) <pc@manguebit.org>
cc: Matthew Wilcox <willy@infradead.org>
cc: Christoph Hellwig <hch@infradead.org>
cc: Jens Axboe <axboe@kernel.dk>
cc: linux-block@vger.kernel.org
cc: netfs@lists.linux.dev
cc: linux-fsdevel@vger.kernel.org
---
include/linux/bvecq.h | 56 +++++++
include/linux/iov_iter.h | 74 ++++++++-
include/linux/uio.h | 11 ++
lib/iov_iter.c | 324 ++++++++++++++++++++++++++++++++++++-
lib/scatterlist.c | 67 +++++++-
lib/tests/kunit_iov_iter.c | 262 ++++++++++++++++++++++++++++++
6 files changed, 788 insertions(+), 6 deletions(-)
create mode 100644 include/linux/bvecq.h
diff --git a/include/linux/bvecq.h b/include/linux/bvecq.h
new file mode 100644
index 000000000000..15f16f905877
--- /dev/null
+++ b/include/linux/bvecq.h
@@ -0,0 +1,56 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Implementation of a segmented queue of bio_vec[].
+ *
+ * Copyright (C) 2026 Red Hat, Inc. All Rights Reserved.
+ * Written by David Howells (dhowells@redhat.com)
+ */
+
+#ifndef _LINUX_BVECQ_H
+#define _LINUX_BVECQ_H
+
+#include <linux/bvec.h>
+
+/*
+ * The type of memory retention used by the elements in bvecq->bv[] and how to
+ * clean it up.
+ */
+enum bvecq_mem {
+ BVECQ_MEM_EXTERNAL, /* Externally retained memory - no freeing */
+ BVECQ_MEM_PAGECACHE, /* Ref'd pagecache pages - must put */
+ BVECQ_MEM_GUP, /* Pinned memory from get_user_pages() - unpin */
+ BVECQ_MEM_ALLOCED, /* Memory alloc'd by bvecq - can be freed/pooled */
+} __mode(byte);
+
+/*
+ * Segmented bio_vec queue.
+ *
+ * These can be linked together to form messages of indefinite length and
+ * iterated over with an ITER_BVECQ iterator. The list is non-circular; next
+ * and prev are NULL at the ends.
+ *
+ * The bv pointer points to the bio_vec array; this may be __bv if allocated
+ * together. The caller is responsible for determining whether or not this is
+ * the case as the array pointed to by bv may be follow on directly from the
+ * bvecq by accident of allocation (ie. ->bv == ->__bv is *not* sufficient to
+ * determine this).
+ *
+ * The file position and discontiguity flag allow non-contiguous data sets to
+ * be chained together, but still teased apart without the need to convert the
+ * info in the bio_vec back into a folio pointer.
+ */
+struct bvecq {
+ struct bvecq *next; /* Next bvec in the list or NULL */
+ struct bvecq *prev; /* Prev bvec in the list or NULL */
+ unsigned long long fpos; /* File position */
+ refcount_t ref;
+ u32 priv; /* Private data */
+ u16 nr_slots; /* Number of elements in bv[] used */
+ u16 max_slots; /* Number of elements allocated in bv[] */
+ enum bvecq_mem mem_type:3; /* What sort of memory and how to free it */
+ bool inline_bv:1; /* T if __bv[] is being used */
+ bool discontig:1; /* T if not contiguous with previous bvecq */
+ struct bio_vec *bv; /* Pointer to array of page fragments */
+ struct bio_vec __bv[]; /* Default array (if ->inline_bv) */
+};
+
+#endif /* _LINUX_BVECQ_H */
diff --git a/include/linux/iov_iter.h b/include/linux/iov_iter.h
index f9a17fbbd398..04b8a6d943fa 100644
--- a/include/linux/iov_iter.h
+++ b/include/linux/iov_iter.h
@@ -10,6 +10,7 @@
#include <linux/uio.h>
#include <linux/bvec.h>
+#include <linux/bvecq.h>
#include <linux/folio_queue.h>
typedef size_t (*iov_step_f)(void *iter_base, size_t progress, size_t len,
@@ -141,6 +142,71 @@ size_t iterate_bvec(struct iov_iter *iter, size_t len, void *priv, void *priv2,
return progress;
}
+/*
+ * Handle ITER_BVECQ.
+ */
+static __always_inline
+size_t iterate_bvecq(struct iov_iter *iter, size_t len, void *priv, void *priv2,
+ iov_step_f step)
+{
+ const struct bvecq *bq = iter->bvecq;
+ unsigned int slot = iter->bvecq_slot;
+ size_t progress = 0, skip = iter->iov_offset;
+
+ do {
+ const struct bio_vec *bvec;
+ struct page *page;
+ size_t poff, plen;
+ void *base;
+
+ if (slot >= bq->nr_slots) {
+ if (!bq->next)
+ break;
+ bq = bq->next;
+ slot = 0;
+ continue;
+ }
+
+ bvec = &bq->bv[slot];
+ /*
+ * The caller must ensure that a slot with bv_len>0 has a valid
+ * bv_page.
+ */
+ page = bvec->bv_page + (bvec->bv_offset + skip) / PAGE_SIZE;
+ poff = (bvec->bv_offset + skip) % PAGE_SIZE;
+ plen = umin(bvec->bv_len - skip, len);
+
+ while (plen > 0) {
+ size_t part, remain, consumed;
+
+ part = umin(plen, PAGE_SIZE - poff);
+ base = kmap_local_page(page) + poff;
+ remain = step(base, progress, part, priv, priv2);
+ kunmap_local(base);
+
+ consumed = part - remain;
+ progress += consumed;
+ skip += consumed;
+ len -= consumed;
+ if (!len || remain)
+ goto stop;
+ page++;
+ poff = 0;
+ plen -= consumed;
+ }
+
+ skip = 0;
+ slot++;
+ } while (len);
+
+stop:
+ iter->bvecq_slot = slot;
+ iter->bvecq = bq;
+ iter->iov_offset = skip;
+ iter->count -= progress;
+ return progress;
+}
+
/*
* Handle ITER_FOLIOQ.
*/
@@ -306,6 +372,8 @@ size_t iterate_and_advance2(struct iov_iter *iter, size_t len, void *priv,
return iterate_bvec(iter, len, priv, priv2, step);
if (iov_iter_is_kvec(iter))
return iterate_kvec(iter, len, priv, priv2, step);
+ if (iov_iter_is_bvecq(iter))
+ return iterate_bvecq(iter, len, priv, priv2, step);
if (iov_iter_is_folioq(iter))
return iterate_folioq(iter, len, priv, priv2, step);
if (iov_iter_is_xarray(iter))
@@ -342,8 +410,8 @@ size_t iterate_and_advance(struct iov_iter *iter, size_t len, void *priv,
* buffer is presented in segments, which for kernel iteration are broken up by
* physical pages and mapped, with the mapped address being presented.
*
- * [!] Note This will only handle BVEC, KVEC, FOLIOQ, XARRAY and DISCARD-type
- * iterators; it will not handle UBUF or IOVEC-type iterators.
+ * [!] Note This will only handle BVEC, KVEC, BVECQ, FOLIOQ, XARRAY and
+ * DISCARD-type iterators; it will not handle UBUF or IOVEC-type iterators.
*
* A step functions, @step, must be provided, one for handling mapped kernel
* addresses and the other is given user addresses which have the potential to
@@ -370,6 +438,8 @@ size_t iterate_and_advance_kernel(struct iov_iter *iter, size_t len, void *priv,
return iterate_bvec(iter, len, priv, priv2, step);
if (iov_iter_is_kvec(iter))
return iterate_kvec(iter, len, priv, priv2, step);
+ if (iov_iter_is_bvecq(iter))
+ return iterate_bvecq(iter, len, priv, priv2, step);
if (iov_iter_is_folioq(iter))
return iterate_folioq(iter, len, priv, priv2, step);
if (iov_iter_is_xarray(iter))
diff --git a/include/linux/uio.h b/include/linux/uio.h
index a9bc5b3067e3..f7cfa6ea8213 100644
--- a/include/linux/uio.h
+++ b/include/linux/uio.h
@@ -26,6 +26,7 @@ enum iter_type {
ITER_IOVEC,
ITER_BVEC,
ITER_KVEC,
+ ITER_BVECQ,
ITER_FOLIOQ,
ITER_XARRAY,
ITER_DISCARD,
@@ -68,6 +69,7 @@ struct iov_iter {
const struct iovec *__iov;
const struct kvec *kvec;
const struct bio_vec *bvec;
+ const struct bvecq *bvecq;
const struct folio_queue *folioq;
struct xarray *xarray;
void __user *ubuf;
@@ -77,6 +79,7 @@ struct iov_iter {
};
union {
unsigned long nr_segs;
+ u16 bvecq_slot;
u8 folioq_slot;
loff_t xarray_start;
};
@@ -145,6 +148,11 @@ static inline bool iov_iter_is_discard(const struct iov_iter *i)
return iov_iter_type(i) == ITER_DISCARD;
}
+static inline bool iov_iter_is_bvecq(const struct iov_iter *i)
+{
+ return iov_iter_type(i) == ITER_BVECQ;
+}
+
static inline bool iov_iter_is_folioq(const struct iov_iter *i)
{
return iov_iter_type(i) == ITER_FOLIOQ;
@@ -295,6 +303,9 @@ void iov_iter_kvec(struct iov_iter *i, unsigned int direction, const struct kvec
void iov_iter_bvec(struct iov_iter *i, unsigned int direction, const struct bio_vec *bvec,
unsigned long nr_segs, size_t count);
void iov_iter_discard(struct iov_iter *i, unsigned int direction, size_t count);
+void iov_iter_bvec_queue(struct iov_iter *i, unsigned int direction,
+ const struct bvecq *bvecq,
+ unsigned int first_slot, unsigned int offset, size_t count);
void iov_iter_folio_queue(struct iov_iter *i, unsigned int direction,
const struct folio_queue *folioq,
unsigned int first_slot, unsigned int offset, size_t count);
diff --git a/lib/iov_iter.c b/lib/iov_iter.c
index 31afc9687eb6..29ec77a0d2b4 100644
--- a/lib/iov_iter.c
+++ b/lib/iov_iter.c
@@ -538,6 +538,40 @@ static void iov_iter_iovec_advance(struct iov_iter *i, size_t size)
i->__iov = iov;
}
+static void iov_iter_bvecq_advance(struct iov_iter *i, size_t by)
+{
+ const struct bvecq *bq = i->bvecq;
+ unsigned int slot = i->bvecq_slot;
+
+ if (!i->count)
+ return;
+ i->count -= by;
+
+ by += i->iov_offset; /* From beginning of current segment. */
+ do {
+ size_t len;
+
+ if (slot >= bq->nr_slots) {
+ if (!bq->next)
+ break;
+ bq = bq->next;
+ slot = 0;
+ continue;
+ }
+
+ len = bq->bv[slot].bv_len;
+
+ if (likely(by < len))
+ break;
+ by -= len;
+ slot++;
+ } while (by);
+
+ i->iov_offset = by;
+ i->bvecq_slot = slot;
+ i->bvecq = bq;
+}
+
static void iov_iter_folioq_advance(struct iov_iter *i, size_t size)
{
const struct folio_queue *folioq = i->folioq;
@@ -583,6 +617,8 @@ void iov_iter_advance(struct iov_iter *i, size_t size)
iov_iter_iovec_advance(i, size);
} else if (iov_iter_is_bvec(i)) {
iov_iter_bvec_advance(i, size);
+ } else if (iov_iter_is_bvecq(i)) {
+ iov_iter_bvecq_advance(i, size);
} else if (iov_iter_is_folioq(i)) {
iov_iter_folioq_advance(i, size);
} else if (iov_iter_is_discard(i)) {
@@ -591,6 +627,33 @@ void iov_iter_advance(struct iov_iter *i, size_t size)
}
EXPORT_SYMBOL(iov_iter_advance);
+static void iov_iter_bvecq_revert(struct iov_iter *i, size_t unroll)
+{
+ const struct bvecq *bq = i->bvecq;
+ unsigned int slot = i->bvecq_slot;
+
+ for (;;) {
+ size_t len;
+
+ if (slot == 0) {
+ bq = bq->prev;
+ slot = bq->nr_slots;
+ continue;
+ }
+ slot--;
+
+ len = bq->bv[slot].bv_len;
+ if (unroll <= len) {
+ i->iov_offset = len - unroll;
+ break;
+ }
+ unroll -= len;
+ }
+
+ i->bvecq_slot = slot;
+ i->bvecq = bq;
+}
+
static void iov_iter_folioq_revert(struct iov_iter *i, size_t unroll)
{
const struct folio_queue *folioq = i->folioq;
@@ -648,6 +711,9 @@ void iov_iter_revert(struct iov_iter *i, size_t unroll)
}
unroll -= n;
}
+ } else if (iov_iter_is_bvecq(i)) {
+ i->iov_offset = 0;
+ iov_iter_bvecq_revert(i, unroll);
} else if (iov_iter_is_folioq(i)) {
i->iov_offset = 0;
iov_iter_folioq_revert(i, unroll);
@@ -678,9 +744,24 @@ size_t iov_iter_single_seg_count(const struct iov_iter *i)
if (iov_iter_is_bvec(i))
return min(i->count, i->bvec->bv_len - i->iov_offset);
}
+ if (!i->count)
+ return 0;
+ if (unlikely(iov_iter_is_bvecq(i))) {
+ const struct bvecq *bq = i->bvecq;
+ unsigned int slot = i->bvecq_slot;
+ size_t offset = i->iov_offset;
+
+ while (slot >= bq->nr_slots) {
+ bq = bq->next;
+ if (!bq)
+ return 0;
+ slot = 0;
+ offset = 0;
+ }
+ return umin(i->count, bq->bv[slot].bv_len - offset);
+ }
if (unlikely(iov_iter_is_folioq(i)))
- return !i->count ? 0 :
- umin(folioq_folio_size(i->folioq, i->folioq_slot), i->count);
+ return umin(folioq_folio_size(i->folioq, i->folioq_slot), i->count);
return i->count;
}
EXPORT_SYMBOL(iov_iter_single_seg_count);
@@ -717,6 +798,35 @@ void iov_iter_bvec(struct iov_iter *i, unsigned int direction,
}
EXPORT_SYMBOL(iov_iter_bvec);
+/**
+ * iov_iter_bvec_queue - Initialise an I/O iterator to use a segmented bvec queue
+ * @i: The iterator to initialise.
+ * @direction: The direction of the transfer.
+ * @bvecq: The starting point in the bvec queue.
+ * @first_slot: The first slot in the bvec queue to use
+ * @offset: The offset into the bvec in the first slot to start at
+ * @count: The size of the I/O buffer in bytes.
+ *
+ * Set up an I/O iterator to either draw data out of the buffers attached to an
+ * inode or to inject data into those buffers. The pages *must* be prevented
+ * from evaporation, either by the caller.
+ */
+void iov_iter_bvec_queue(struct iov_iter *i, unsigned int direction,
+ const struct bvecq *bvecq, unsigned int first_slot,
+ unsigned int offset, size_t count)
+{
+ WARN_ON(direction & ~(READ | WRITE));
+ *i = (struct iov_iter) {
+ .iter_type = ITER_BVECQ,
+ .data_source = direction,
+ .bvecq = bvecq,
+ .bvecq_slot = first_slot,
+ .count = count,
+ .iov_offset = offset,
+ };
+}
+EXPORT_SYMBOL(iov_iter_bvec_queue);
+
/**
* iov_iter_folio_queue - Initialise an I/O iterator to use the folios in a folio queue
* @i: The iterator to initialise.
@@ -839,6 +949,37 @@ static unsigned long iov_iter_alignment_bvec(const struct iov_iter *i)
return res;
}
+static unsigned long iov_iter_alignment_bvecq(const struct iov_iter *iter)
+{
+ const struct bvecq *bq;
+ unsigned long res = 0;
+ unsigned int slot = iter->bvecq_slot;
+ size_t skip = iter->iov_offset;
+ size_t size = iter->count;
+
+ if (!size)
+ return res;
+
+ for (bq = iter->bvecq; bq; bq = bq->next) {
+ for (; slot < bq->nr_slots; slot++) {
+ const struct bio_vec *bvec = &bq->bv[slot];
+ size_t part = umin(bvec->bv_len - skip, size);
+
+ res |= bvec->bv_offset + skip;
+ res |= part;
+
+ size -= part;
+ if (size == 0)
+ return res;
+ skip = 0;
+ }
+
+ slot = 0;
+ }
+
+ return res;
+}
+
unsigned long iov_iter_alignment(const struct iov_iter *i)
{
if (likely(iter_is_ubuf(i))) {
@@ -854,6 +995,8 @@ unsigned long iov_iter_alignment(const struct iov_iter *i)
if (iov_iter_is_bvec(i))
return iov_iter_alignment_bvec(i);
+ if (iov_iter_is_bvecq(i))
+ return iov_iter_alignment_bvecq(i);
/* With both xarray and folioq types, we're dealing with whole folios. */
if (iov_iter_is_folioq(i))
@@ -1066,6 +1209,36 @@ static int bvec_npages(const struct iov_iter *i, int maxpages)
return npages;
}
+static size_t iov_npages_bvecq(const struct iov_iter *iter, size_t maxpages)
+{
+ const struct bvecq *bq;
+ unsigned int slot = iter->bvecq_slot;
+ size_t npages = 0;
+ size_t skip = iter->iov_offset;
+ size_t size = iter->count;
+
+ for (bq = iter->bvecq; bq; bq = bq->next) {
+ for (; slot < bq->nr_slots; slot++) {
+ const struct bio_vec *bvec = &bq->bv[slot];
+ size_t offs = (bvec->bv_offset + skip) % PAGE_SIZE;
+ size_t part = umin(bvec->bv_len - skip, size);
+
+ npages += DIV_ROUND_UP(offs + part, PAGE_SIZE);
+ if (npages >= maxpages)
+ goto out;
+
+ size -= part;
+ if (!size)
+ goto out;
+ skip = 0;
+ }
+
+ slot = 0;
+ }
+out:
+ return umin(npages, maxpages);
+}
+
int iov_iter_npages(const struct iov_iter *i, int maxpages)
{
if (unlikely(!i->count))
@@ -1080,6 +1253,8 @@ int iov_iter_npages(const struct iov_iter *i, int maxpages)
return iov_npages(i, maxpages);
if (iov_iter_is_bvec(i))
return bvec_npages(i, maxpages);
+ if (iov_iter_is_bvecq(i))
+ return iov_npages_bvecq(i, maxpages);
if (iov_iter_is_folioq(i)) {
unsigned offset = i->iov_offset % PAGE_SIZE;
int npages = DIV_ROUND_UP(offset + i->count, PAGE_SIZE);
@@ -1366,6 +1541,147 @@ void iov_iter_restore(struct iov_iter *i, struct iov_iter_state *state)
i->nr_segs = state->nr_segs;
}
+/*
+ * Count the number of virtually contiguous pages coming up next in an
+ * ITER_BVECQ iterator, up to the specified maxima.
+ */
+static unsigned int iter_count_bvecq_pages(const struct iov_iter *iter,
+ size_t maxsize,
+ unsigned int maxpages)
+{
+ const struct bvecq *bvecq = iter->bvecq;
+ unsigned int slot = iter->bvecq_slot;
+ ssize_t remain = umin(maxsize, iter->count);
+ size_t count = 0, offset = iter->iov_offset;
+
+ do {
+ const struct bio_vec *bv;
+ size_t boff, blen;
+
+ if (slot >= bvecq->nr_slots) {
+ if (!bvecq->next) {
+ WARN_ON_ONCE(remain > 0);
+ break;
+ }
+ bvecq = bvecq->next;
+ slot = 0;
+ offset = 0;
+ continue;
+ }
+
+ bv = &bvecq->bv[slot++];
+ boff = bv->bv_offset;
+ blen = bv->bv_len;
+
+ if (unlikely(!bv->bv_page)) {
+ if (blen && count > 0)
+ break;
+ continue;
+ }
+ if (!PAGE_ALIGNED(boff) && count > 0)
+ break;
+
+ boff += offset;
+ blen -= offset;
+ offset = 0;
+ if (!blen)
+ continue;
+
+ blen = umin(blen, remain);
+ remain -= blen;
+ blen += offset_in_page(boff);
+ count += DIV_ROUND_UP(blen, PAGE_SIZE);
+
+ if (!PAGE_ALIGNED(blen))
+ break;
+ } while (remain > 0 && count < maxpages);
+
+ return umin(count, maxpages);
+}
+
+/*
+ * Extract a list of virtually contiguous pages from an ITER_BVECQ iterator.
+ * This does not get references on the pages, nor does it get a pin on them.
+ */
+static ssize_t iov_iter_extract_bvecq_pages(struct iov_iter *iter,
+ struct page ***pages, size_t maxsize,
+ unsigned int maxpages,
+ iov_iter_extraction_t extraction_flags,
+ size_t *offset0)
+{
+ const struct bvecq *bvecq;
+ struct page **p;
+ unsigned int slot, nr = 0;
+ size_t extracted = 0, offset;
+
+ /* Count the next run of virtually contiguous pages. */
+ maxpages = iter_count_bvecq_pages(iter, maxsize, maxpages);
+
+ if (!*pages) {
+ *pages = kvmalloc_array(maxpages, sizeof(struct page *), GFP_KERNEL);
+ if (!*pages)
+ return -ENOMEM;
+ }
+
+ p = *pages;
+
+ /* Now transcribe the page pointers. */
+ extracted = 0;
+ bvecq = iter->bvecq;
+ offset = iter->iov_offset;
+ slot = iter->bvecq_slot;
+
+ do {
+ const struct bio_vec *bv;
+ size_t boff, blen;
+
+ if (slot >= bvecq->nr_slots) {
+ if (!bvecq->next) {
+ WARN_ON_ONCE(extracted < iter->count);
+ break;
+ }
+ bvecq = bvecq->next;
+ slot = 0;
+ offset = 0;
+ continue;
+ }
+
+ bv = &bvecq->bv[slot];
+ boff = bv->bv_offset;
+ blen = bv->bv_len;
+
+ if (!bv->bv_page)
+ blen = 0;
+
+ if (offset < blen) {
+ size_t part = umin(maxsize - extracted, blen - offset);
+ size_t poff = (boff + offset) % PAGE_SIZE;
+ size_t pix = (boff + offset) / PAGE_SIZE;
+
+ if (poff + part > PAGE_SIZE)
+ part = PAGE_SIZE - poff;
+
+ if (!extracted)
+ *offset0 = poff;
+
+ p[nr++] = bv->bv_page + pix;
+ offset += part;
+ extracted += part;
+ }
+
+ if (offset >= blen) {
+ offset = 0;
+ slot++;
+ }
+ } while (nr < maxpages && extracted < maxsize);
+
+ iter->bvecq = bvecq;
+ iter->bvecq_slot = slot;
+ iter->iov_offset = offset;
+ iter->count -= extracted;
+ return extracted;
+}
+
/*
* Extract a list of contiguous pages from an ITER_FOLIOQ iterator. This does
* not get references on the pages, nor does it get a pin on them.
@@ -1726,6 +2042,10 @@ ssize_t iov_iter_extract_pages(struct iov_iter *i,
return iov_iter_extract_bvec_pages(i, pages, maxsize,
maxpages, extraction_flags,
offset0);
+ if (iov_iter_is_bvecq(i))
+ return iov_iter_extract_bvecq_pages(i, pages, maxsize,
+ maxpages, extraction_flags,
+ offset0);
if (iov_iter_is_folioq(i))
return iov_iter_extract_folioq_pages(i, pages, maxsize,
maxpages, extraction_flags,
diff --git a/lib/scatterlist.c b/lib/scatterlist.c
index 6ea40d2e6247..23e5a180103b 100644
--- a/lib/scatterlist.c
+++ b/lib/scatterlist.c
@@ -10,6 +10,7 @@
#include <linux/highmem.h>
#include <linux/kmemleak.h>
#include <linux/bvec.h>
+#include <linux/bvecq.h>
#include <linux/uio.h>
#include <linux/folio_queue.h>
@@ -1267,6 +1268,65 @@ static ssize_t extract_kvec_to_sg(struct iov_iter *iter,
return ret;
}
+/*
+ * Extract up to sg_max folios from an BVECQ-type iterator and add them to
+ * the scatterlist. The pages are not pinned.
+ */
+static ssize_t extract_bvecq_to_sg(struct iov_iter *iter,
+ ssize_t maxsize,
+ struct sg_table *sgtable,
+ unsigned int sg_max,
+ iov_iter_extraction_t extraction_flags)
+{
+ const struct bvecq *bvecq = iter->bvecq;
+ struct scatterlist *sg = sgtable->sgl + sgtable->nents;
+ unsigned int slot = iter->bvecq_slot;
+ ssize_t ret = 0;
+ size_t offset = iter->iov_offset;
+
+ maxsize = umin(maxsize, iov_iter_count(iter));
+
+ while (sg_max > 0 && ret < maxsize) {
+ const struct bio_vec *bv;
+ size_t blen, part;
+
+ if (slot >= bvecq->nr_slots) {
+ if (!bvecq->next) {
+ WARN_ON_ONCE(ret < iter->count);
+ break;
+ }
+ bvecq = bvecq->next;
+ slot = 0;
+ offset = 0;
+ continue;
+ }
+
+ bv = &bvecq->bv[slot];
+ blen = bv->bv_len;
+
+ if (offset >= blen) {
+ offset = 0;
+ slot++;
+ continue;
+ }
+
+ part = umin(maxsize - ret, blen - offset);
+
+ sg_set_page(sg, bv->bv_page, part, bv->bv_offset + offset);
+ sgtable->nents++;
+ sg++;
+ sg_max--;
+ offset += part;
+ ret += part;
+ }
+
+ iter->bvecq = bvecq;
+ iter->bvecq_slot = slot;
+ iter->iov_offset = offset;
+ iter->count -= ret;
+ return ret;
+}
+
/*
* Extract up to sg_max folios from an FOLIOQ-type iterator and add them to
* the scatterlist. The pages are not pinned.
@@ -1391,8 +1451,8 @@ static ssize_t extract_xarray_to_sg(struct iov_iter *iter,
* addition of @sg_max elements.
*
* The pages referred to by UBUF- and IOVEC-type iterators are extracted and
- * pinned; BVEC-, KVEC-, FOLIOQ- and XARRAY-type are extracted but aren't
- * pinned; DISCARD-type is not supported.
+ * pinned; BVEC-, BVECQ-, KVEC-, FOLIOQ- and XARRAY-type are extracted but
+ * aren't pinned; DISCARD-type is not supported.
*
* No end mark is placed on the scatterlist; that's left to the caller.
*
@@ -1424,6 +1484,9 @@ ssize_t extract_iter_to_sg(struct iov_iter *iter, size_t maxsize,
case ITER_KVEC:
return extract_kvec_to_sg(iter, maxsize, sgtable, sg_max,
extraction_flags);
+ case ITER_BVECQ:
+ return extract_bvecq_to_sg(iter, maxsize, sgtable, sg_max,
+ extraction_flags);
case ITER_FOLIOQ:
return extract_folioq_to_sg(iter, maxsize, sgtable, sg_max,
extraction_flags);
diff --git a/lib/tests/kunit_iov_iter.c b/lib/tests/kunit_iov_iter.c
index d9690ba1db88..8426c33e48a4 100644
--- a/lib/tests/kunit_iov_iter.c
+++ b/lib/tests/kunit_iov_iter.c
@@ -12,6 +12,7 @@
#include <linux/mm.h>
#include <linux/uio.h>
#include <linux/bvec.h>
+#include <linux/bvecq.h>
#include <linux/folio_queue.h>
#include <linux/scatterlist.h>
#include <linux/minmax.h>
@@ -552,6 +553,185 @@ static void __init iov_kunit_copy_from_folioq(struct kunit *test)
KUNIT_SUCCEED(test);
}
+static void iov_kunit_destroy_bvecq(void *data)
+{
+ struct bvecq *bq, *next;
+
+ for (bq = data; bq; bq = next) {
+ next = bq->next;
+ for (int i = 0; i < bq->nr_slots; i++)
+ if (bq->bv[i].bv_page)
+ put_page(bq->bv[i].bv_page);
+ kfree(bq);
+ }
+}
+
+static struct bvecq *iov_kunit_alloc_bvecq(struct kunit *test, unsigned int max_slots)
+{
+ struct bvecq *bq;
+
+ bq = kzalloc(struct_size(bq, __bv, max_slots), GFP_KERNEL);
+ KUNIT_ASSERT_NOT_ERR_OR_NULL(test, bq);
+ bq->max_slots = max_slots;
+ bq->bv = bq->__bv;
+ bq->inline_bv = true;
+ return bq;
+}
+
+static struct bvecq *iov_kunit_create_bvecq(struct kunit *test, unsigned int max_slots)
+{
+ struct bvecq *bq;
+
+ bq = iov_kunit_alloc_bvecq(test, max_slots);
+ kunit_add_action_or_reset(test, iov_kunit_destroy_bvecq, bq);
+ return bq;
+}
+
+static void __init iov_kunit_load_bvecq(struct kunit *test,
+ struct iov_iter *iter, int dir,
+ struct bvecq *bq_head,
+ struct page **pages, size_t npages)
+{
+ struct bvecq *bq = bq_head;
+ size_t size = 0;
+
+ for (int i = 0; i < npages; i++) {
+ if (bq->nr_slots >= bq->max_slots) {
+ bq->next = iov_kunit_alloc_bvecq(test, 13);
+ bq->next->prev = bq;
+ bq = bq->next;
+ }
+ bvec_set_page(&bq->bv[bq->nr_slots], pages[i], PAGE_SIZE, 0);
+ bq->nr_slots++;
+ size += PAGE_SIZE;
+ }
+ iov_iter_bvec_queue(iter, dir, bq_head, 0, 0, size);
+}
+
+/*
+ * Test copying to a ITER_BVECQ-type iterator.
+ */
+static void __init iov_kunit_copy_to_bvecq(struct kunit *test)
+{
+ const struct kvec_test_range *pr;
+ struct iov_iter iter;
+ struct bvecq *bq;
+ struct page **spages, **bpages;
+ u8 *scratch, *buffer;
+ size_t bufsize, npages, size, copied;
+ int i, patt;
+
+ bufsize = 0x100000;
+ npages = bufsize / PAGE_SIZE;
+
+ bq = iov_kunit_create_bvecq(test, 13);
+
+ scratch = iov_kunit_create_buffer(test, &spages, npages);
+ for (i = 0; i < bufsize; i++)
+ scratch[i] = pattern(i);
+
+ buffer = iov_kunit_create_buffer(test, &bpages, npages);
+ memset(buffer, 0, bufsize);
+
+ iov_kunit_load_bvecq(test, &iter, READ, bq, bpages, npages);
+
+ i = 0;
+ for (pr = kvec_test_ranges; pr->from >= 0; pr++) {
+ size = pr->to - pr->from;
+ KUNIT_ASSERT_LE(test, pr->to, bufsize);
+
+ iov_iter_bvec_queue(&iter, READ, bq, 0, 0, pr->to);
+ iov_iter_advance(&iter, pr->from);
+ copied = copy_to_iter(scratch + i, size, &iter);
+
+ KUNIT_EXPECT_EQ(test, copied, size);
+ KUNIT_EXPECT_EQ(test, iter.count, 0);
+ i += size;
+ if (test->status == KUNIT_FAILURE)
+ goto stop;
+ }
+
+ /* Build the expected image in the scratch buffer. */
+ patt = 0;
+ memset(scratch, 0, bufsize);
+ for (pr = kvec_test_ranges; pr->from >= 0; pr++)
+ for (i = pr->from; i < pr->to; i++)
+ scratch[i] = pattern(patt++);
+
+ /* Compare the images */
+ for (i = 0; i < bufsize; i++) {
+ KUNIT_EXPECT_EQ_MSG(test, buffer[i], scratch[i], "at i=%x", i);
+ if (buffer[i] != scratch[i])
+ return;
+ }
+
+stop:
+ KUNIT_SUCCEED(test);
+}
+
+/*
+ * Test copying from a ITER_BVECQ-type iterator.
+ */
+static void __init iov_kunit_copy_from_bvecq(struct kunit *test)
+{
+ const struct kvec_test_range *pr;
+ struct iov_iter iter;
+ struct bvecq *bq;
+ struct page **spages, **bpages;
+ u8 *scratch, *buffer;
+ size_t bufsize, npages, size, copied;
+ int i, j;
+
+ bufsize = 0x100000;
+ npages = bufsize / PAGE_SIZE;
+
+ bq = iov_kunit_create_bvecq(test, 13);
+
+ buffer = iov_kunit_create_buffer(test, &bpages, npages);
+ for (i = 0; i < bufsize; i++)
+ buffer[i] = pattern(i);
+
+ scratch = iov_kunit_create_buffer(test, &spages, npages);
+ memset(scratch, 0, bufsize);
+
+ iov_kunit_load_bvecq(test, &iter, READ, bq, bpages, npages);
+
+ i = 0;
+ for (pr = kvec_test_ranges; pr->from >= 0; pr++) {
+ size = pr->to - pr->from;
+ KUNIT_ASSERT_LE(test, pr->to, bufsize);
+
+ iov_iter_bvec_queue(&iter, WRITE, bq, 0, 0, pr->to);
+ iov_iter_advance(&iter, pr->from);
+ copied = copy_from_iter(scratch + i, size, &iter);
+
+ KUNIT_EXPECT_EQ(test, copied, size);
+ KUNIT_EXPECT_EQ(test, iter.count, 0);
+ i += size;
+ }
+
+ /* Build the expected image in the main buffer. */
+ i = 0;
+ memset(buffer, 0, bufsize);
+ for (pr = kvec_test_ranges; pr->from >= 0; pr++) {
+ for (j = pr->from; j < pr->to; j++) {
+ buffer[i++] = pattern(j);
+ if (i >= bufsize)
+ goto stop;
+ }
+ }
+stop:
+
+ /* Compare the images */
+ for (i = 0; i < bufsize; i++) {
+ KUNIT_EXPECT_EQ_MSG(test, scratch[i], buffer[i], "at i=%x", i);
+ if (scratch[i] != buffer[i])
+ return;
+ }
+
+ KUNIT_SUCCEED(test);
+}
+
static void iov_kunit_destroy_xarray(void *data)
{
struct xarray *xarray = data;
@@ -867,6 +1047,85 @@ static void __init iov_kunit_extract_pages_bvec(struct kunit *test)
KUNIT_SUCCEED(test);
}
+/*
+ * Test the extraction of ITER_BVECQ-type iterators.
+ */
+static void __init iov_kunit_extract_pages_bvecq(struct kunit *test)
+{
+ const struct kvec_test_range *pr;
+ struct iov_iter iter;
+ struct bvecq *bq;
+ struct page **bpages, *pagelist[8], **pages = pagelist;
+ ssize_t len;
+ size_t bufsize, size = 0, npages;
+ int i, from;
+
+ bufsize = 0x100000;
+ npages = bufsize / PAGE_SIZE;
+
+ bq = iov_kunit_create_bvecq(test, 13);
+
+ iov_kunit_create_buffer(test, &bpages, npages);
+ iov_kunit_load_bvecq(test, &iter, READ, bq, bpages, npages);
+
+ for (pr = kvec_test_ranges; pr->from >= 0; pr++) {
+ from = pr->from;
+ size = pr->to - from;
+ KUNIT_ASSERT_LE(test, pr->to, bufsize);
+
+ iov_iter_bvec_queue(&iter, WRITE, bq, 0, 0, pr->to);
+ iov_iter_advance(&iter, from);
+
+ do {
+ size_t offset0 = LONG_MAX;
+
+ for (i = 0; i < ARRAY_SIZE(pagelist); i++)
+ pagelist[i] = (void *)(unsigned long)0xaa55aa55aa55aa55ULL;
+
+ len = iov_iter_extract_pages(&iter, &pages, 100 * 1024,
+ ARRAY_SIZE(pagelist), 0, &offset0);
+ KUNIT_EXPECT_GE(test, len, 0);
+ if (len < 0)
+ break;
+ KUNIT_EXPECT_LE(test, len, size);
+ KUNIT_EXPECT_EQ(test, iter.count, size - len);
+ if (len == 0)
+ break;
+ size -= len;
+ KUNIT_EXPECT_GE(test, (ssize_t)offset0, 0);
+ KUNIT_EXPECT_LT(test, offset0, PAGE_SIZE);
+
+ for (i = 0; i < ARRAY_SIZE(pagelist); i++) {
+ struct page *p;
+ ssize_t part = min_t(ssize_t, len, PAGE_SIZE - offset0);
+ int ix;
+
+ KUNIT_ASSERT_GE(test, part, 0);
+ ix = from / PAGE_SIZE;
+ KUNIT_ASSERT_LT(test, ix, npages);
+ p = bpages[ix];
+ KUNIT_EXPECT_PTR_EQ(test, pagelist[i], p);
+ KUNIT_EXPECT_EQ(test, offset0, from % PAGE_SIZE);
+ from += part;
+ len -= part;
+ KUNIT_ASSERT_GE(test, len, 0);
+ if (len == 0)
+ break;
+ offset0 = 0;
+ }
+
+ if (test->status == KUNIT_FAILURE)
+ goto stop;
+ } while (iov_iter_count(&iter) > 0);
+
+ KUNIT_EXPECT_EQ(test, size, 0);
+ KUNIT_EXPECT_EQ(test, iter.count, 0);
+ }
+
+stop:
+ KUNIT_SUCCEED(test);
+}
+
/*
* Test the extraction of ITER_FOLIOQ-type iterators.
*/
@@ -1226,12 +1485,15 @@ static struct kunit_case __refdata iov_kunit_cases[] = {
KUNIT_CASE(iov_kunit_copy_from_kvec),
KUNIT_CASE(iov_kunit_copy_to_bvec),
KUNIT_CASE(iov_kunit_copy_from_bvec),
+ KUNIT_CASE(iov_kunit_copy_to_bvecq),
+ KUNIT_CASE(iov_kunit_copy_from_bvecq),
KUNIT_CASE(iov_kunit_copy_to_folioq),
KUNIT_CASE(iov_kunit_copy_from_folioq),
KUNIT_CASE(iov_kunit_copy_to_xarray),
KUNIT_CASE(iov_kunit_copy_from_xarray),
KUNIT_CASE(iov_kunit_extract_pages_kvec),
KUNIT_CASE(iov_kunit_extract_pages_bvec),
+ KUNIT_CASE(iov_kunit_extract_pages_bvecq),
KUNIT_CASE(iov_kunit_extract_pages_folioq),
KUNIT_CASE(iov_kunit_extract_pages_xarray),
KUNIT_CASE(iov_kunit_iter_to_sg_kvec),
^ permalink raw reply related
* [PATCH v5 04/21] iov_iter: Make iov_iter_get_pages*() wrap iov_iter_extract_pages()
From: David Howells @ 2026-07-06 15:27 UTC (permalink / raw)
To: Christian Brauner, Matthew Wilcox, Christoph Hellwig
Cc: David Howells, Paulo Alcantara, Jens Axboe, Leon Romanovsky,
Steve French, ChenXiaoSong, Marc Dionne, Stefan Metzmacher,
Eric Van Hensbergen, Dominique Martinet, Ilya Dryomov, netfs,
linux-afs, linux-cifs, linux-nfs, ceph-devel, v9fs, linux-erofs,
linux-fsdevel, linux-kernel, linux-block
In-Reply-To: <20260706152737.1231312-1-dhowells@redhat.com>
Make iov_iter_get_pages*() wrap iov_iter_extract_pages() for kernel
iterator types (e.g. ITER_BVEC, ITER_FOLIOQ, ITER_XARRAY). The pages
obtained have their refcounts incremented afterwards if they're not slab
pages. ITER_KVEC is left returning -EFAULT.
Signed-off-by: David Howells <dhowells@redhat.com>
Reviewed-by: Paulo Alcantara (Red Hat) <pc@manguebit.org>
cc: Matthew Wilcox <willy@infradead.org>
cc: Christoph Hellwig <hch@infradead.org>
cc: Jens Axboe <axboe@kernel.dk>
cc: linux-block@vger.kernel.org
cc: netfs@lists.linux.dev
cc: linux-fsdevel@vger.kernel.org
---
lib/iov_iter.c | 164 ++++++-------------------------------------------
1 file changed, 19 insertions(+), 145 deletions(-)
diff --git a/lib/iov_iter.c b/lib/iov_iter.c
index c2484551a4e8..31afc9687eb6 100644
--- a/lib/iov_iter.c
+++ b/lib/iov_iter.c
@@ -910,118 +910,34 @@ static int want_pages_array(struct page ***res, size_t size,
return count;
}
-static ssize_t iter_folioq_get_pages(struct iov_iter *iter,
+/*
+ * Wrap iov_iter_extract_pages() and then pin the non-slab pages we got back.
+ * This only works for non-user iterator types as get_pages uses get_user_pages
+ * not pin_user_pages.
+ */
+static ssize_t iter_get_kernel_pages(struct iov_iter *iter,
struct page ***ppages, size_t maxsize,
unsigned maxpages, size_t *_start_offset)
{
- const struct folio_queue *folioq = iter->folioq;
struct page **pages;
- unsigned int slot = iter->folioq_slot;
- size_t extracted = 0, count = iter->count, iov_offset = iter->iov_offset;
+ ssize_t ret, done;
- if (slot >= folioq_nr_slots(folioq)) {
- folioq = folioq->next;
- slot = 0;
- if (WARN_ON(iov_offset != 0))
- return -EIO;
- }
+ ret = iov_iter_extract_pages(iter, ppages, maxsize, maxpages,
+ 0, _start_offset);
+ if (ret <= 0)
+ return ret;
- maxpages = want_pages_array(ppages, maxsize, iov_offset & ~PAGE_MASK, maxpages);
- if (!maxpages)
- return -ENOMEM;
- *_start_offset = iov_offset & ~PAGE_MASK;
pages = *ppages;
+ for (done = ret + *_start_offset; done > 0; done -= PAGE_SIZE) {
+ struct folio *folio = page_folio(*pages);
- for (;;) {
- struct folio *folio = folioq_folio(folioq, slot);
- size_t offset = iov_offset, fsize = folioq_folio_size(folioq, slot);
- size_t part = PAGE_SIZE - offset % PAGE_SIZE;
-
- if (offset < fsize) {
- part = umin(part, umin(maxsize - extracted, fsize - offset));
- count -= part;
- iov_offset += part;
- extracted += part;
-
- *pages = folio_page(folio, offset / PAGE_SIZE);
- get_page(*pages);
- pages++;
- maxpages--;
- }
-
- if (maxpages == 0 || extracted >= maxsize)
- break;
-
- if (iov_offset >= fsize) {
- iov_offset = 0;
- slot++;
- if (slot == folioq_nr_slots(folioq) && folioq->next) {
- folioq = folioq->next;
- slot = 0;
- }
- }
- }
-
- iter->count = count;
- iter->iov_offset = iov_offset;
- iter->folioq = folioq;
- iter->folioq_slot = slot;
- return extracted;
-}
-
-static ssize_t iter_xarray_populate_pages(struct page **pages, struct xarray *xa,
- pgoff_t index, unsigned int nr_pages)
-{
- XA_STATE(xas, xa, index);
- struct folio *folio;
- unsigned int ret = 0;
-
- rcu_read_lock();
- for (folio = xas_load(&xas); folio; folio = xas_next(&xas)) {
- if (xas_retry(&xas, folio))
- continue;
-
- /* Has the folio moved or been split? */
- if (unlikely(folio != xas_reload(&xas))) {
- xas_reset(&xas);
- continue;
- }
-
- pages[ret] = folio_file_page(folio, xas.xa_index);
- folio_get(folio);
- if (++ret == nr_pages)
- break;
+ if (!folio_test_slab(folio))
+ folio_get(folio);
+ pages++;
}
- rcu_read_unlock();
return ret;
}
-static ssize_t iter_xarray_get_pages(struct iov_iter *i,
- struct page ***pages, size_t maxsize,
- unsigned maxpages, size_t *_start_offset)
-{
- unsigned nr, offset, count;
- pgoff_t index;
- loff_t pos;
-
- pos = i->xarray_start + i->iov_offset;
- index = pos >> PAGE_SHIFT;
- offset = pos & ~PAGE_MASK;
- *_start_offset = offset;
-
- count = want_pages_array(pages, maxsize, offset, maxpages);
- if (!count)
- return -ENOMEM;
- nr = iter_xarray_populate_pages(*pages, i->xarray, index, count);
- if (nr == 0)
- return 0;
-
- maxsize = min_t(size_t, nr * PAGE_SIZE - offset, maxsize);
- i->iov_offset += maxsize;
- i->count -= maxsize;
- return maxsize;
-}
-
/* must be done on non-empty ITER_UBUF or ITER_IOVEC one */
static unsigned long first_iovec_segment(const struct iov_iter *i, size_t *size)
{
@@ -1044,22 +960,6 @@ static unsigned long first_iovec_segment(const struct iov_iter *i, size_t *size)
BUG(); // if it had been empty, we wouldn't get called
}
-/* must be done on non-empty ITER_BVEC one */
-static struct page *first_bvec_segment(const struct iov_iter *i,
- size_t *size, size_t *start)
-{
- struct page *page;
- size_t skip = i->iov_offset, len;
-
- len = i->bvec->bv_len - skip;
- if (*size > len)
- *size = len;
- skip += i->bvec->bv_offset;
- page = i->bvec->bv_page + skip / PAGE_SIZE;
- *start = skip % PAGE_SIZE;
- return page;
-}
-
static ssize_t __iov_iter_get_pages_alloc(struct iov_iter *i,
struct page ***pages, size_t maxsize,
unsigned int maxpages, size_t *start)
@@ -1095,36 +995,10 @@ static ssize_t __iov_iter_get_pages_alloc(struct iov_iter *i,
iov_iter_advance(i, maxsize);
return maxsize;
}
- if (iov_iter_is_bvec(i)) {
- struct page **p;
- struct page *page;
- page = first_bvec_segment(i, &maxsize, start);
- n = want_pages_array(pages, maxsize, *start, maxpages);
- if (!n)
- return -ENOMEM;
- p = *pages;
- for (int k = 0; k < n; k++) {
- struct folio *folio = page_folio(page + k);
- p[k] = page + k;
- if (!folio_test_slab(folio))
- folio_get(folio);
- }
- maxsize = min_t(size_t, maxsize, n * PAGE_SIZE - *start);
- i->count -= maxsize;
- i->iov_offset += maxsize;
- if (i->iov_offset == i->bvec->bv_len) {
- i->iov_offset = 0;
- i->bvec++;
- i->nr_segs--;
- }
- return maxsize;
- }
- if (iov_iter_is_folioq(i))
- return iter_folioq_get_pages(i, pages, maxsize, maxpages, start);
- if (iov_iter_is_xarray(i))
- return iter_xarray_get_pages(i, pages, maxsize, maxpages, start);
- return -EFAULT;
+ if (iov_iter_is_kvec(i))
+ return -EFAULT;
+ return iter_get_kernel_pages(i, pages, maxsize, maxpages, start);
}
ssize_t iov_iter_get_pages2(struct iov_iter *i, struct page **pages,
^ permalink raw reply related
* [PATCH v5 03/21] Add a function to kmap one page of a multipage bio_vec
From: David Howells @ 2026-07-06 15:27 UTC (permalink / raw)
To: Christian Brauner, Matthew Wilcox, Christoph Hellwig
Cc: David Howells, Paulo Alcantara, Jens Axboe, Leon Romanovsky,
Steve French, ChenXiaoSong, Marc Dionne, Stefan Metzmacher,
Eric Van Hensbergen, Dominique Martinet, Ilya Dryomov, netfs,
linux-afs, linux-cifs, linux-nfs, ceph-devel, v9fs, linux-erofs,
linux-fsdevel, linux-kernel, linux-block
In-Reply-To: <20260706152737.1231312-1-dhowells@redhat.com>
Add a function to kmap one page of a multipage bio_vec by offset (which is
added to the offset in the bio_vec internally). The caller is responsible
for calculating how much of the page is then available.
Signed-off-by: David Howells <dhowells@redhat.com>
Acked-by: Paulo Alcantara (Red Hat) <pc@manguebit.org>
cc: Matthew Wilcox <willy@infradead.org>
cc: Christoph Hellwig <hch@infradead.org>
cc: Jens Axboe <axboe@kernel.dk>
cc: linux-block@vger.kernel.org
cc: netfs@lists.linux.dev
cc: linux-fsdevel@vger.kernel.org
---
include/linux/bvec.h | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/include/linux/bvec.h b/include/linux/bvec.h
index 92837e2743f1..53cf36e73967 100644
--- a/include/linux/bvec.h
+++ b/include/linux/bvec.h
@@ -343,4 +343,22 @@ static inline phys_addr_t bvec_phys(const struct bio_vec *bvec)
return page_to_phys(bvec->bv_page) + bvec->bv_offset;
}
+/**
+ * bvec_kmap_partial - Map part of a bvec into the kernel virtual address space
+ * @bvec: bvec to map
+ * @offset: Offset into bvec
+ *
+ * Map the page containing the byte at @offset into the kernel virtual address
+ * space. The caller is responsible for making sure this doesn't overrun.
+ *
+ * Call kunmap_local on the returned address to unmap.
+ */
+static inline void *bvec_kmap_partial(struct bio_vec *bvec, size_t offset)
+{
+ offset += bvec->bv_offset;
+
+ return kmap_local_page(bvec->bv_page + (offset >> PAGE_SHIFT)) +
+ (offset & ~PAGE_MASK);
+}
+
#endif /* __LINUX_BVEC_H */
^ permalink raw reply related
* Re: [PATCH 2/3] null_blk: give the file-scope mutex a descriptive name
From: Bart Van Assche @ 2026-07-06 14:45 UTC (permalink / raw)
To: Zizhi Wo, axboe, dlemoal, nilay, linux-block, kch,
johannes.thumshirn, kbusch
Cc: linux-kernel, yangerkun, chengzhihao1, wozizhi
In-Reply-To: <5f99b2f3-05c1-4754-9827-d55528861ecc@huaweicloud.com>
On 7/6/26 6:21 AM, Zizhi Wo wrote:
> Thanks for the reminder! Agreed that "nullb_list_lock" is too narrow
> since it also serializes nullb_device_power_store() and
> nullb_update_nr_hw_queues().
>
> However, I'm a bit worried that "nullb_lock" is easy to confuse with the
> per-device spinlock "nullb->lock"... They are only one "->" apart?
>
> Would "nullb_global_lock" (or "nullb_config_lock") work better? It
> reflects the broader purpose and avoids clashing with nullb->lock.
Not sure what others prefer but nullb_global_lock sounds good to me.
Thanks,
Bart.
^ permalink raw reply
* Re: [PATCH 3/3] null_blk: register configfs subsystem after creating default devices
From: Zizhi Wo @ 2026-07-06 13:34 UTC (permalink / raw)
To: Bart Van Assche, axboe, dlemoal, nilay, linux-block, kch,
johannes.thumshirn, kbusch
Cc: linux-kernel, yangerkun, chengzhihao1, wozizhi
In-Reply-To: <490008e2-2345-48ea-bd8e-c078d828a43f@acm.org>
在 2026/7/6 21:17, Bart Van Assche 写道:
> On 7/6/26 5:35 AM, Zizhi Wo wrote:
>> @@ -2187,8 +2185,6 @@ static int __init null_init(void)
>> null_destroy_dev(nullb);
>> }
>> unregister_blkdev(null_major, "nullb");
>> -err_conf:
>> - configfs_unregister_subsystem(&nullb_subsys);
>> return ret;
>> }
>
> Please make null_exit() consistent with the cleanup path in null_init().
> null_exit() calls unregister_blkdev() before it destroys the null_blk
> instances. That doesn't look right to me.
>
> Thanks,
>
> Bart.
>
>
Thanks for pointing this out, I hadn't noticed that. I'll add this
change in the next version.
Thanks,
Zizhi Wo
^ permalink raw reply
* Re: [PATCH 2/3] null_blk: give the file-scope mutex a descriptive name
From: Zizhi Wo @ 2026-07-06 13:21 UTC (permalink / raw)
To: Bart Van Assche, axboe, dlemoal, nilay, linux-block, kch,
johannes.thumshirn, kbusch
Cc: linux-kernel, yangerkun, chengzhihao1, wozizhi
In-Reply-To: <9004743e-f522-424e-a1ab-8779ecc6da02@acm.org>
在 2026/7/6 21:12, Bart Van Assche 写道:
> On 7/6/26 5:35 AM, Zizhi Wo wrote:
>> The file-scope lock mutex mainly serializes access to the global
>> nullb_list
>> and related device setup. Rename it to "nullb_list_lock" to make its
>> purpose clear. No functional change.
>
> From commit a2db328b0839 ("null_blk: fix null-ptr-dereference while
> configuring 'power' and 'submit_queues'"):
>
> Writing 'power' and 'submit_queues' concurrently will trigger kernel
> panic:
> [ ... ]
> Fix this problem by resuing the global mutex to protect
> nullb_device_power_store() and nullb_update_nr_hw_queues() from
> configfs.
>
> This makes it clear that the "lock" mutex has a broader purpose than
> only protecting nullb_list. Hence, the name nullb_list_lock is
> confusing. How about using the name "nullb_lock"?
>
> Thanks,
>
> Bart.
Thanks for the reminder! Agreed that "nullb_list_lock" is too narrow
since it also serializes nullb_device_power_store() and
nullb_update_nr_hw_queues().
However, I'm a bit worried that "nullb_lock" is easy to confuse with the
per-device spinlock "nullb->lock"... They are only one "->" apart?
Would "nullb_global_lock" (or "nullb_config_lock") work better? It
reflects the broader purpose and avoids clashing with nullb->lock.
Thanks,
Zizhi Wo
^ permalink raw reply
* Re: [PATCH 3/3] null_blk: register configfs subsystem after creating default devices
From: Bart Van Assche @ 2026-07-06 13:17 UTC (permalink / raw)
To: Zizhi Wo, axboe, dlemoal, nilay, linux-block, kch,
johannes.thumshirn, kbusch
Cc: linux-kernel, yangerkun, chengzhihao1, wozizhi
In-Reply-To: <20260706123507.3809871-4-wozizhi@huaweicloud.com>
On 7/6/26 5:35 AM, Zizhi Wo wrote:
> @@ -2187,8 +2185,6 @@ static int __init null_init(void)
> null_destroy_dev(nullb);
> }
> unregister_blkdev(null_major, "nullb");
> -err_conf:
> - configfs_unregister_subsystem(&nullb_subsys);
> return ret;
> }
Please make null_exit() consistent with the cleanup path in null_init().
null_exit() calls unregister_blkdev() before it destroys the null_blk
instances. That doesn't look right to me.
Thanks,
Bart.
^ permalink raw reply
* Re: [PATCH 1/3] null_blk: use DEFINE_MUTEX for the file-scope mutex
From: Bart Van Assche @ 2026-07-06 13:13 UTC (permalink / raw)
To: Zizhi Wo, axboe, dlemoal, nilay, linux-block, kch,
johannes.thumshirn, kbusch
Cc: linux-kernel, yangerkun, chengzhihao1, wozizhi
In-Reply-To: <20260706123507.3809871-2-wozizhi@huaweicloud.com>
On 7/6/26 5:35 AM, Zizhi Wo wrote:
> [ ... ]
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
^ permalink raw reply
* Re: [PATCH 2/3] null_blk: give the file-scope mutex a descriptive name
From: Bart Van Assche @ 2026-07-06 13:12 UTC (permalink / raw)
To: Zizhi Wo, axboe, dlemoal, nilay, linux-block, kch,
johannes.thumshirn, kbusch
Cc: linux-kernel, yangerkun, chengzhihao1, wozizhi
In-Reply-To: <20260706123507.3809871-3-wozizhi@huaweicloud.com>
On 7/6/26 5:35 AM, Zizhi Wo wrote:
> The file-scope lock mutex mainly serializes access to the global nullb_list
> and related device setup. Rename it to "nullb_list_lock" to make its
> purpose clear. No functional change.
From commit a2db328b0839 ("null_blk: fix null-ptr-dereference while
configuring 'power' and 'submit_queues'"):
Writing 'power' and 'submit_queues' concurrently will trigger kernel
panic:
[ ... ]
Fix this problem by resuing the global mutex to protect
nullb_device_power_store() and nullb_update_nr_hw_queues() from
configfs.
This makes it clear that the "lock" mutex has a broader purpose than
only protecting nullb_list. Hence, the name nullb_list_lock is
confusing. How about using the name "nullb_lock"?
Thanks,
Bart.
^ permalink raw reply
* Re: [PATCH] block: split out a new blk_plug.h helper
From: Christian Brauner @ 2026-07-06 13:09 UTC (permalink / raw)
To: Christoph Hellwig
Cc: axboe, linux-block, linux-fsdevel, linux-aio, linux-kernel,
io-uring, linux-mm
In-Reply-To: <20260706041125.642097-1-hch@lst.de>
> blkdev.h gets included in various places outside the block layer just
> for struct blk_plug and related plugging functions.
>
> Split blk_plug into a separate helper to reduce the amount of code
> that needs to get rebuilt when blkdev.h changes and to slightly
> reduce compile times.
>
> In io_uring this requires pulling in a few other headers explicitly that
> previously were implicitly included through blkdev.h.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Christian Brauner (Amutable) <brauner@kernel.org>
--
Christian Brauner <brauner@kernel.org>
^ permalink raw reply
* [PATCH 3/3] null_blk: register configfs subsystem after creating default devices
From: Zizhi Wo @ 2026-07-06 12:35 UTC (permalink / raw)
To: axboe, dlemoal, nilay, linux-block, kch, johannes.thumshirn,
kbusch, bvanassche
Cc: linux-kernel, yangerkun, chengzhihao1, wozizhi
In-Reply-To: <20260706123507.3809871-1-wozizhi@huaweicloud.com>
From: Zizhi Wo <wozizhi@huawei.com>
In null_init(), configfs_register_subsystem() currently runs before
register_blkdev(), so when null_blk is built as a module, a racing mkdir()
+ poweron from userspace can reach null_add_dev() while null_major is still
0. __add_disk() then hits WARN_ON(disk->minors) (major=0 with minors!=0)
and fails:
[root@fedora ~]# [ 2366.521436] WARNING: block/genhd.c:476 at __add_disk+0x8a7/0xde0,
[ 2366.523552] Modules linked in: null_blk(+) nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib
[ 2366.529081] CPU: 26 UID: 0 PID: 1600 Comm: sh Not tainted 7.2.0-rc1+ #66 PREEMPT(full)
......
[ 2366.547251] Call Trace:
[ 2366.547575] <TASK>
[ 2366.547831] ? _raw_spin_lock+0x84/0xe0
[ 2366.548260] add_disk_fwnode+0x114/0x560
[ 2366.548739] null_add_dev+0x102d/0x1b80 [null_blk]
[ 2366.549310] ? __pfx_null_add_dev+0x10/0x10 [null_blk]
[ 2366.549906] ? mutex_lock+0xde/0x1c0
[ 2366.550361] ? __pfx_mutex_lock+0x10/0x10
[ 2366.550827] nullb_device_power_store+0x1e7/0x280 [null_blk]
[ 2366.551499] ? __pfx_nullb_device_power_store+0x10/0x10 [null_blk]
[ 2366.552177] ? __kmalloc_cache_noprof+0x1f5/0x470
[ 2366.552748] ? configfs_write_iter+0x35c/0x4e0
[ 2366.553242] configfs_write_iter+0x286/0x4e0
[ 2366.553787] vfs_write+0x52d/0xd00
[ 2366.554169] ? __pfx_vfs_write+0x10/0x10
[ 2366.554679] ? __pfx___css_rstat_updated+0x10/0x10
[ 2366.555196] ? fdget_pos+0x1cf/0x4c0
[ 2366.555649] ksys_write+0xfc/0x1d0
......
Additionally, the err_dev path destroys all devices on nullb_list while
configfs is still registered. If a racing mkdir() + poweron puts a user
device on the list, null_destroy_dev()->null_free_dev() kfrees the user
device's nullb_device but /sys/kernel/config/nullb/<name> is still
reachable. Any userspace access to the item will trigger a UAF.
For simplicity, move configfs_register_subsystem() to the end to solve
the problems above. This also mirrors null_exit().
Fixes: 3bf2bd20734e ("nullb: add configfs interface")
Signed-off-by: Zizhi Wo <wozizhi@huawei.com>
---
drivers/block/null_blk/main.c | 16 ++++++----------
1 file changed, 6 insertions(+), 10 deletions(-)
diff --git a/drivers/block/null_blk/main.c b/drivers/block/null_blk/main.c
index a775e70e16fc..9800049b3c3f 100644
--- a/drivers/block/null_blk/main.c
+++ b/drivers/block/null_blk/main.c
@@ -2162,15 +2162,9 @@ static int __init null_init(void)
config_group_init(&nullb_subsys.su_group);
mutex_init(&nullb_subsys.su_mutex);
- ret = configfs_register_subsystem(&nullb_subsys);
- if (ret)
- return ret;
-
null_major = register_blkdev(0, "nullb");
- if (null_major < 0) {
- ret = null_major;
- goto err_conf;
- }
+ if (null_major < 0)
+ return null_major;
for (i = 0; i < nr_devices; i++) {
ret = null_create_dev();
@@ -2178,6 +2172,10 @@ static int __init null_init(void)
goto err_dev;
}
+ ret = configfs_register_subsystem(&nullb_subsys);
+ if (ret)
+ goto err_dev;
+
pr_info("module loaded\n");
return 0;
@@ -2187,8 +2185,6 @@ static int __init null_init(void)
null_destroy_dev(nullb);
}
unregister_blkdev(null_major, "nullb");
-err_conf:
- configfs_unregister_subsystem(&nullb_subsys);
return ret;
}
--
2.52.0
^ permalink raw reply related
* [PATCH 1/3] null_blk: use DEFINE_MUTEX for the file-scope mutex
From: Zizhi Wo @ 2026-07-06 12:35 UTC (permalink / raw)
To: axboe, dlemoal, nilay, linux-block, kch, johannes.thumshirn,
kbusch, bvanassche
Cc: linux-kernel, yangerkun, chengzhihao1, wozizhi
In-Reply-To: <20260706123507.3809871-1-wozizhi@huaweicloud.com>
From: Zizhi Wo <wozizhi@huawei.com>
In null_init(), mutex_init(&lock) currently happens after
configfs_register_subsystem(), which exposes the nullb subsystem to
userspace. A racing mkdir() into /sys/kernel/config/nullb/ can reach
null_find_dev_by_name() -> mutex_lock(&lock) before the mutex is
initialized, trigger warning:
[ 123.137788] DEBUG_LOCKS_WARN_ON(lock->magic != lock)
[ 123.137796] WARNING: kernel/locking/mutex.c:159 at mutex_lock+0x171/0x1c0, CPU#13: mkdir/1301
[ 123.140090] Modules linked in: null_blk(+) nft_fib_inet nft_fib_ipv4
......
[ 123.154926] Call Trace:
[ 123.155172] <TASK>
[ 123.155419] ? __pfx_mutex_lock+0x10/0x10
[ 123.156181] ? __pfx__raw_spin_lock+0x10/0x10
[ 123.156571] nullb_group_make_group+0x20/0x100 [null_blk]
[ 123.157011] configfs_mkdir+0x47b/0xc70
[ 123.157337] ? __pfx_configfs_mkdir+0x10/0x10
[ 123.157719] ? may_create_dentry+0x242/0x2e0
[ 123.158061] vfs_mkdir+0x2a9/0x6c0
[ 123.158352] filename_mkdirat+0x3dc/0x500
[ 123.158710] ? __pfx_filename_mkdirat+0x10/0x10
[ 123.159070] ? strncpy_from_user+0x3a/0x1d0
[ 123.159413] __x64_sys_mkdir+0x6b/0x90
[ 123.159760] do_syscall_64+0xea/0x600
Replace the runtime mutex_init(&lock) with a static DEFINE_MUTEX(lock)
declaration to fix this issue.
Fixes: 49c3b9266a71 ("block: null_blk: Improve device creation with configfs")
Suggested-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Zizhi Wo <wozizhi@huawei.com>
---
drivers/block/null_blk/main.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/drivers/block/null_blk/main.c b/drivers/block/null_blk/main.c
index f8c0fd57e041..eba204b27785 100644
--- a/drivers/block/null_blk/main.c
+++ b/drivers/block/null_blk/main.c
@@ -66,7 +66,7 @@ struct nullb_page {
#define NULLB_PAGE_FREE (MAP_SZ - 2)
static LIST_HEAD(nullb_list);
-static struct mutex lock;
+static DEFINE_MUTEX(lock);
static int null_major;
static DEFINE_IDA(nullb_indexes);
static struct blk_mq_tag_set tag_set;
@@ -2166,8 +2166,6 @@ static int __init null_init(void)
if (ret)
return ret;
- mutex_init(&lock);
-
null_major = register_blkdev(0, "nullb");
if (null_major < 0) {
ret = null_major;
--
2.52.0
^ permalink raw reply related
* [PATCH 0/3] null_blk: fix mutex initialization and configfs teardown race
From: Zizhi Wo @ 2026-07-06 12:35 UTC (permalink / raw)
To: axboe, dlemoal, nilay, linux-block, kch, johannes.thumshirn,
kbusch, bvanassche
Cc: linux-kernel, yangerkun, chengzhihao1, wozizhi
This series fixes a few simple issues in null_blk.
Patches 1-2 fix a problem where the null_blk mutex was left uninitialized.
The fix for the first issue was originally sent as a separate patch[1].
Following Bart's suggestion, it now use DEFINE_MUTEX() and the lock has
been renamed to a more descriptive name.
Patch 3 fixes concurrency issues around configfs registration.
See the individual patch descriptions for details.
[1] https://lore.kernel.org/all/20260704092323.748772-1-wozizhi@huaweicloud.com/
Zizhi Wo (3):
null_blk: use DEFINE_MUTEX for the file-scope mutex
null_blk: give the file-scope mutex a descriptive name
null_blk: register configfs subsystem after creating default devices
drivers/block/null_blk/main.c | 50 +++++++++++++++--------------------
1 file changed, 22 insertions(+), 28 deletions(-)
--
2.52.0
^ permalink raw reply
* [PATCH 2/3] null_blk: give the file-scope mutex a descriptive name
From: Zizhi Wo @ 2026-07-06 12:35 UTC (permalink / raw)
To: axboe, dlemoal, nilay, linux-block, kch, johannes.thumshirn,
kbusch, bvanassche
Cc: linux-kernel, yangerkun, chengzhihao1, wozizhi
In-Reply-To: <20260706123507.3809871-1-wozizhi@huaweicloud.com>
From: Zizhi Wo <wozizhi@huawei.com>
The file-scope lock mutex mainly serializes access to the global nullb_list
and related device setup. Rename it to "nullb_list_lock" to make its
purpose clear. No functional change.
Suggested-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Zizhi Wo <wozizhi@huawei.com>
---
drivers/block/null_blk/main.c | 32 ++++++++++++++++----------------
1 file changed, 16 insertions(+), 16 deletions(-)
diff --git a/drivers/block/null_blk/main.c b/drivers/block/null_blk/main.c
index eba204b27785..a775e70e16fc 100644
--- a/drivers/block/null_blk/main.c
+++ b/drivers/block/null_blk/main.c
@@ -66,7 +66,7 @@ struct nullb_page {
#define NULLB_PAGE_FREE (MAP_SZ - 2)
static LIST_HEAD(nullb_list);
-static DEFINE_MUTEX(lock);
+static DEFINE_MUTEX(nullb_list_lock);
static int null_major;
static DEFINE_IDA(nullb_indexes);
static struct blk_mq_tag_set tag_set;
@@ -423,9 +423,9 @@ static int nullb_apply_submit_queues(struct nullb_device *dev,
{
int ret;
- mutex_lock(&lock);
+ mutex_lock(&nullb_list_lock);
ret = nullb_update_nr_hw_queues(dev, submit_queues, dev->poll_queues);
- mutex_unlock(&lock);
+ mutex_unlock(&nullb_list_lock);
return ret;
}
@@ -435,9 +435,9 @@ static int nullb_apply_poll_queues(struct nullb_device *dev,
{
int ret;
- mutex_lock(&lock);
+ mutex_lock(&nullb_list_lock);
ret = nullb_update_nr_hw_queues(dev, dev->submit_queues, poll_queues);
- mutex_unlock(&lock);
+ mutex_unlock(&nullb_list_lock);
return ret;
}
@@ -493,7 +493,7 @@ static ssize_t nullb_device_power_store(struct config_item *item,
return ret;
ret = count;
- mutex_lock(&lock);
+ mutex_lock(&nullb_list_lock);
if (!dev->power && newp) {
if (test_and_set_bit(NULLB_DEV_FL_UP, &dev->flags))
goto out;
@@ -516,7 +516,7 @@ static ssize_t nullb_device_power_store(struct config_item *item,
}
out:
- mutex_unlock(&lock);
+ mutex_unlock(&nullb_list_lock);
return ret;
}
@@ -707,10 +707,10 @@ nullb_group_drop_item(struct config_group *group, struct config_item *item)
struct nullb_device *dev = to_nullb_device(item);
if (test_and_clear_bit(NULLB_DEV_FL_UP, &dev->flags)) {
- mutex_lock(&lock);
+ mutex_lock(&nullb_list_lock);
dev->power = false;
null_del_dev(dev->nullb);
- mutex_unlock(&lock);
+ mutex_unlock(&nullb_list_lock);
}
nullb_del_fault_config(dev);
config_item_put(item);
@@ -2081,14 +2081,14 @@ static struct nullb *null_find_dev_by_name(const char *name)
{
struct nullb *nullb = NULL, *nb;
- mutex_lock(&lock);
+ mutex_lock(&nullb_list_lock);
list_for_each_entry(nb, &nullb_list, list) {
if (strcmp(nb->disk_name, name) == 0) {
nullb = nb;
break;
}
}
- mutex_unlock(&lock);
+ mutex_unlock(&nullb_list_lock);
return nullb;
}
@@ -2102,9 +2102,9 @@ static int null_create_dev(void)
if (!dev)
return -ENOMEM;
- mutex_lock(&lock);
+ mutex_lock(&nullb_list_lock);
ret = null_add_dev(dev);
- mutex_unlock(&lock);
+ mutex_unlock(&nullb_list_lock);
if (ret) {
null_free_dev(dev);
return ret;
@@ -2200,17 +2200,17 @@ static void __exit null_exit(void)
unregister_blkdev(null_major, "nullb");
- mutex_lock(&lock);
+ mutex_lock(&nullb_list_lock);
while (!list_empty(&nullb_list)) {
nullb = list_entry(nullb_list.next, struct nullb, list);
null_destroy_dev(nullb);
}
- mutex_unlock(&lock);
+ mutex_unlock(&nullb_list_lock);
if (tag_set.ops)
blk_mq_free_tag_set(&tag_set);
- mutex_destroy(&lock);
+ mutex_destroy(&nullb_list_lock);
}
module_init(null_init);
--
2.52.0
^ permalink raw reply related
* Re: [PATCH] block: split out a new blk_plug.h helper
From: Christoph Hellwig @ 2026-07-06 9:54 UTC (permalink / raw)
To: Damien Le Moal
Cc: Christoph Hellwig, axboe, linux-block, linux-fsdevel, linux-aio,
linux-kernel, io-uring, linux-mm
In-Reply-To: <f389a101-2af1-4b18-a74b-031e6c1899bd@kernel.org>
On Mon, Jul 06, 2026 at 04:04:48PM +0900, Damien Le Moal wrote:
> > +#include <linux/sched.h>
>
> struct blk_plug_cb has a list_head. So maybe also add
>
> #include <linux/types.h>
>
> ?
That already gets pulled in.
> > +
> > +struct blk_plug_cb;
>
> Maybe add "struct request;" here too ?
No need for type forward declarations when the types are only used
for pointers in structs (or as pointer variables). They are needed
only when pointers to the type are used in prototypes.
^ permalink raw reply
* Re: [PATCH] block: split out a new blk_plug.h helper
From: Damien Le Moal @ 2026-07-06 7:04 UTC (permalink / raw)
To: Christoph Hellwig, axboe
Cc: linux-block, linux-fsdevel, linux-aio, linux-kernel, io-uring,
linux-mm
In-Reply-To: <20260706041125.642097-1-hch@lst.de>
On 7/6/26 1:11 PM, Christoph Hellwig wrote:
> blkdev.h gets included in various places outside the block layer just
> for struct blk_plug and related plugging functions.
>
> Split blk_plug into a separate helper to reduce the amount of code
> that needs to get rebuilt when blkdev.h changes and to slightly
> reduce compile times.
>
> In io_uring this requires pulling in a few other headers explicitly that
> previously were implicitly included through blkdev.h.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
A couple of nits below (not entirely sure if they make sense).
But otherwise looks OK to me.
Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
> diff --git a/include/linux/blk_plug.h b/include/linux/blk_plug.h
> new file mode 100644
> index 000000000000..2ac1265662ad
> --- /dev/null
> +++ b/include/linux/blk_plug.h
> @@ -0,0 +1,95 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef _LINUX_BLK_PLUG_H
> +#define _LINUX_BLK_PLUG_H
> +
> +#include <linux/sched.h>
struct blk_plug_cb has a list_head. So maybe also add
#include <linux/types.h>
?
> +
> +struct blk_plug_cb;
Maybe add "struct request;" here too ?
> +typedef void (*blk_plug_cb_fn)(struct blk_plug_cb *cb, bool from_schedule);
> +
> +struct rq_list {
> + struct request *head;
> + struct request *tail;
> +};
--
Damien Le Moal
Western Digital Research
^ permalink raw reply
* Re: [PATCH] block: split out a new blk_plug.h helper
From: Johannes Thumshirn @ 2026-07-06 6:47 UTC (permalink / raw)
To: Christoph Hellwig
Cc: axboe, linux-block, linux-fsdevel, linux-aio, linux-kernel,
io-uring, linux-mm
In-Reply-To: <20260706064547.GA25268@lst.de>
On 7/6/26 8:45 AM, Christoph Hellwig wrote:
> On Mon, Jul 06, 2026 at 08:38:12AM +0200, Johannes Thumshirn wrote:
>> On 7/6/26 6:11 AM, Christoph Hellwig wrote:
>>> diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
>>> index 9213a5716f95..20cb8ed7d987 100644
>>> --- a/include/linux/blkdev.h
>>> +++ b/include/linux/blkdev.h
>>> @@ -7,6 +7,7 @@
>>> #include <linux/types.h>
>>> #include <linux/blk_types.h>
>>> +#include <linux/blk_plug.h>
>>> #include <linux/device.h>
>>> #include <linux/list.h>
>>> #include <linux/llist.h>
>>>
>> I know it's a lot of cross subsystem churn, but wouldn't it be cleaner to
>> not include blk_plug.h in blkdev.h, but patch the update the consumers? A
>> quick grep shows 68 files that would need updating and some you already
>> have updated.
> Right now blkdev.h needs the rq_list from it. So we'd need to move
> that to linux/types.h or something first, which feels a bit iffy.
>
> And no, including blk_types.h in blk_plug.h is not a solution,
> as that is still touched far too often.
Ah sh*t I did miss rq_list. Fair enough.
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
^ permalink raw reply
* Re: [PATCH] block: split out a new blk_plug.h helper
From: Christoph Hellwig @ 2026-07-06 6:45 UTC (permalink / raw)
To: Johannes Thumshirn
Cc: Christoph Hellwig, axboe, linux-block, linux-fsdevel, linux-aio,
linux-kernel, io-uring, linux-mm
In-Reply-To: <8583b332-0d24-4f6f-8831-69e3aad936fd@wdc.com>
On Mon, Jul 06, 2026 at 08:38:12AM +0200, Johannes Thumshirn wrote:
> On 7/6/26 6:11 AM, Christoph Hellwig wrote:
>> diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
>> index 9213a5716f95..20cb8ed7d987 100644
>> --- a/include/linux/blkdev.h
>> +++ b/include/linux/blkdev.h
>> @@ -7,6 +7,7 @@
>> #include <linux/types.h>
>> #include <linux/blk_types.h>
>> +#include <linux/blk_plug.h>
>> #include <linux/device.h>
>> #include <linux/list.h>
>> #include <linux/llist.h>
>>
> I know it's a lot of cross subsystem churn, but wouldn't it be cleaner to
> not include blk_plug.h in blkdev.h, but patch the update the consumers? A
> quick grep shows 68 files that would need updating and some you already
> have updated.
Right now blkdev.h needs the rq_list from it. So we'd need to move
that to linux/types.h or something first, which feels a bit iffy.
And no, including blk_types.h in blk_plug.h is not a solution,
as that is still touched far too often.
^ permalink raw reply
* Re: [PATCH] block: split out a new blk_plug.h helper
From: Johannes Thumshirn @ 2026-07-06 6:38 UTC (permalink / raw)
To: Christoph Hellwig, axboe
Cc: linux-block, linux-fsdevel, linux-aio, linux-kernel, io-uring,
linux-mm
In-Reply-To: <20260706041125.642097-1-hch@lst.de>
On 7/6/26 6:11 AM, Christoph Hellwig wrote:
> diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
> index 9213a5716f95..20cb8ed7d987 100644
> --- a/include/linux/blkdev.h
> +++ b/include/linux/blkdev.h
> @@ -7,6 +7,7 @@
>
> #include <linux/types.h>
> #include <linux/blk_types.h>
> +#include <linux/blk_plug.h>
> #include <linux/device.h>
> #include <linux/list.h>
> #include <linux/llist.h>
>
I know it's a lot of cross subsystem churn, but wouldn't it be cleaner
to not include blk_plug.h in blkdev.h, but patch the update the
consumers? A quick grep shows 68 files that would need updating and some
you already have updated.
^ permalink raw reply
* Re: [PATCH] iomap: Remove FGP_NOFS from iomap_get_folio()
From: Shin'ichiro Kawasaki @ 2026-07-06 6:29 UTC (permalink / raw)
To: Matthew Wilcox (Oracle)
Cc: Christian Brauner, Darrick J. Wong, Jens Axboe, Namjae Jeon,
Sungjong Seo, Yuezhang Mo, Miklos Szeredi, Andreas Gruenbacher,
Hyunchul Lee, Konstantin Komarov, Carlos Maiolino, Damien Le Moal,
Naohiro Aota, Johannes Thumshirn, linux-xfs, linux-fsdevel,
linux-block, fuse-devel, gfs2, ntfs3
In-Reply-To: <20260624174228.2015893-1-willy@infradead.org>
On Jun 24, 2026 / 18:42, Matthew Wilcox (Oracle) wrote:
> FGP_NOFS is legacy; filesystems should be using memalloc_nofs_save/restore
> instead. We have it here in iomap because it was buried in
> grab_cache_page_write_begin() and we didn't want to change this behaviour
> as part of the folio transition.
>
> I have tested this with XFS and see no issues. Other filesystems (cc'd)
> may need to make adjustments. Please test with lockdep enabled.
I applied this patch on top of v7.2-rc1 kernel, and ran my zonefs test set.
I observed no failure. Good from zonefs point of view:
Tested-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
^ permalink raw reply
* [PATCH v19 40/40] dept: implement a basic unit test for dept
From: Byungchul Park @ 2026-07-06 6:19 UTC (permalink / raw)
To: linux-kernel
Cc: max.byungchul.park, kernel_team, torvalds, damien.lemoal,
linux-ide, adilger.kernel, linux-ext4, mingo, peterz, will, tglx,
rostedt, joel, sashal, daniel.vetter, duyuyang, johannes.berg, tj,
tytso, willy, david, amir73il, gregkh, kernel-team, linux-mm,
akpm, mhocko, minchan, hannes, vdavydov.dev, sj, jglisse, dennis,
cl, penberg, rientjes, vbabka, ngupta, linux-block, josef,
linux-fsdevel, jack, jlayton, dan.j.williams, hch, djwong,
dri-devel, rodrigosiqueiramelo, melissa.srw, hamohammed.sa,
harry.yoo, chris.p.wilson, gwan-gyeong.mun, boqun.feng, longman,
yunseong.kim, ysk, yeoreum.yun, netdev, matthew.brost, her0gyugyu,
corbet, catalin.marinas, bp, x86, hpa, luto, sumit.semwal,
gustavo, christian.koenig, andi.shyti, arnd, lorenzo.stoakes,
Liam.Howlett, rppt, surenb, mcgrof, petr.pavlu, da.gomez,
samitolvanen, paulmck, frederic, neeraj.upadhyay, joelagnelf,
josh, urezki, mathieu.desnoyers, jiangshanlai, qiang.zhang,
juri.lelli, vincent.guittot, dietmar.eggemann, bsegall, mgorman,
vschneid, chuck.lever, neil, okorniev, Dai.Ngo, tom, trondmy,
anna, kees, bigeasy, clrkwllms, mark.rutland, ada.coupriediaz,
kristina.martsenko, wangkefeng.wang, broonie, kevin.brodsky, dwmw,
shakeel.butt, ast, ziy, yuzhao, baolin.wang, usamaarif642,
joel.granados, richard.weiyang, geert+renesas, tim.c.chen, linux,
alexander.shishkin, lillian, chenhuacai, francesco,
guoweikang.kernel, link, jpoimboe, masahiroy, brauner,
thomas.weissschuh, oleg, mjguzik, andrii, wangfushuai, linux-doc,
linux-arm-kernel, linux-media, linaro-mm-sig, linux-i2c,
linux-arch, linux-modules, rcu, linux-nfs, linux-rt-devel,
2407018371, dakr, miguel.ojeda.sandonis, neilb, bagasdotme,
wsa+renesas, dave.hansen, geert, ojeda, alex.gaynor, gary,
bjorn3_gh, lossin, a.hindborg, aliceryhl, tmgross, rust-for-linux
In-Reply-To: <20260706061928.66713-1-byungchul@sk.com>
Implement CONFIG_DEPT_UNIT_TEST introducing a kernel module that runs
basic unit test for dept.
Signed-off-by: Byungchul Park <byungchul@sk.com>
---
include/linux/dept_unit_test.h | 61 ++++++++++++
kernel/dependency/Makefile | 1 +
kernel/dependency/dept.c | 10 ++
kernel/dependency/dept_unit_test.c | 149 +++++++++++++++++++++++++++++
lib/Kconfig.debug | 12 +++
5 files changed, 233 insertions(+)
create mode 100644 include/linux/dept_unit_test.h
create mode 100644 kernel/dependency/dept_unit_test.c
diff --git a/include/linux/dept_unit_test.h b/include/linux/dept_unit_test.h
new file mode 100644
index 000000000000..753ac9ac727c
--- /dev/null
+++ b/include/linux/dept_unit_test.h
@@ -0,0 +1,61 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * DEPT unit test
+ *
+ * Started by Byungchul Park <max.byungchul.park@gmail.com>:
+ *
+ * Copyright (c) 2025 SK hynix, Inc., Byungchul Park
+ */
+
+#ifndef __LINUX_DEPT_UNIT_TEST_H
+#define __LINUX_DEPT_UNIT_TEST_H
+
+#if defined(CONFIG_DEPT_UNIT_TEST) || defined(CONFIG_DEPT_UNIT_TEST_MODULE)
+struct dept_ut {
+ bool circle_detected;
+
+ int ecxt_stack_total_cnt;
+ int wait_stack_total_cnt;
+ int evnt_stack_total_cnt;
+ int ecxt_stack_valid_cnt;
+ int wait_stack_valid_cnt;
+ int evnt_stack_valid_cnt;
+};
+
+extern struct dept_ut dept_ut_results;
+
+static inline void dept_ut_circle_detect(void)
+{
+ dept_ut_results.circle_detected = true;
+}
+static inline void dept_ut_ecxt_stack_account(bool valid)
+{
+ dept_ut_results.ecxt_stack_total_cnt++;
+
+ if (valid)
+ dept_ut_results.ecxt_stack_valid_cnt++;
+}
+static inline void dept_ut_wait_stack_account(bool valid)
+{
+ dept_ut_results.wait_stack_total_cnt++;
+
+ if (valid)
+ dept_ut_results.wait_stack_valid_cnt++;
+}
+static inline void dept_ut_evnt_stack_account(bool valid)
+{
+ dept_ut_results.evnt_stack_total_cnt++;
+
+ if (valid)
+ dept_ut_results.evnt_stack_valid_cnt++;
+}
+#else
+struct dept_ut {};
+
+#define dept_ut_circle_detect() do { } while (0)
+#define dept_ut_ecxt_stack_account(v) do { } while (0)
+#define dept_ut_wait_stack_account(v) do { } while (0)
+#define dept_ut_evnt_stack_account(v) do { } while (0)
+
+#endif
+#endif /* __LINUX_DEPT_UNIT_TEST_H */
diff --git a/kernel/dependency/Makefile b/kernel/dependency/Makefile
index 92f165400187..fc584ca87124 100644
--- a/kernel/dependency/Makefile
+++ b/kernel/dependency/Makefile
@@ -2,3 +2,4 @@
obj-$(CONFIG_DEPT) += dept.o
obj-$(CONFIG_DEPT) += dept_proc.o
+obj-$(CONFIG_DEPT_UNIT_TEST) += dept_unit_test.o
diff --git a/kernel/dependency/dept.c b/kernel/dependency/dept.c
index 35a3667ac8b3..bcff14f20046 100644
--- a/kernel/dependency/dept.c
+++ b/kernel/dependency/dept.c
@@ -78,8 +78,12 @@
#include <linux/workqueue.h>
#include <linux/irq_work.h>
#include <linux/vmalloc.h>
+#include <linux/dept_unit_test.h>
#include "dept_internal.h"
+struct dept_ut dept_ut_results;
+EXPORT_SYMBOL_GPL(dept_ut_results);
+
static int dept_stop;
static int dept_per_cpu_ready;
@@ -826,6 +830,10 @@ static void print_dep(struct dept_dep *d)
pr_warn("(wait to wake up)\n");
print_ip_stack(0, e->ewait_stack);
}
+
+ dept_ut_ecxt_stack_account(valid_stack(e->ecxt_stack));
+ dept_ut_wait_stack_account(valid_stack(w->wait_stack));
+ dept_ut_evnt_stack_account(valid_stack(e->event_stack));
}
}
@@ -920,6 +928,8 @@ static void print_circle(struct dept_class *c)
dump_stack();
dept_outworld_exit();
+
+ dept_ut_circle_detect();
}
/*
diff --git a/kernel/dependency/dept_unit_test.c b/kernel/dependency/dept_unit_test.c
new file mode 100644
index 000000000000..e8dada2e3dfb
--- /dev/null
+++ b/kernel/dependency/dept_unit_test.c
@@ -0,0 +1,149 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * DEPT unit test
+ *
+ * Started by Byungchul Park <max.byungchul.park@gmail.com>:
+ *
+ * Copyright (c) 2025 SK hynix, Inc., Byungchul Park
+ */
+
+#include <linux/module.h>
+#include <linux/spinlock.h>
+#include <linux/mutex.h>
+#include <linux/dept.h>
+#include <linux/dept_unit_test.h>
+
+MODULE_DESCRIPTION("DEPT unit test");
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Byungchul Park <max.byungchul.park@sk.com>");
+
+struct unit {
+ const char *name;
+ bool (*func)(void);
+ bool result;
+};
+
+static DEFINE_SPINLOCK(s1);
+static DEFINE_SPINLOCK(s2);
+static bool test_spin_lock_deadlock(void)
+{
+ dept_ut_results.circle_detected = false;
+
+ spin_lock(&s1);
+ spin_lock(&s2);
+ spin_unlock(&s2);
+ spin_unlock(&s1);
+
+ spin_lock(&s2);
+ spin_lock(&s1);
+ spin_unlock(&s1);
+ spin_unlock(&s2);
+
+ return dept_ut_results.circle_detected;
+}
+
+static DEFINE_MUTEX(m1);
+static DEFINE_MUTEX(m2);
+static bool test_mutex_lock_deadlock(void)
+{
+ dept_ut_results.circle_detected = false;
+
+ mutex_lock(&m1);
+ mutex_lock(&m2);
+ mutex_unlock(&m2);
+ mutex_unlock(&m1);
+
+ mutex_lock(&m2);
+ mutex_lock(&m1);
+ mutex_unlock(&m1);
+ mutex_unlock(&m2);
+
+ return dept_ut_results.circle_detected;
+}
+
+static bool test_wait_event_deadlock(void)
+{
+ struct dept_map dmap1;
+ struct dept_map dmap2;
+
+ sdt_map_init(&dmap1);
+ sdt_map_init(&dmap2);
+
+ dept_ut_results.circle_detected = false;
+
+ sdt_request_event(&dmap1); /* [S] */
+ sdt_wait(&dmap2); /* [W] */
+ sdt_event(&dmap1); /* [E] */
+
+ sdt_request_event(&dmap2); /* [S] */
+ sdt_wait(&dmap1); /* [W] */
+ sdt_event(&dmap2); /* [E] */
+
+ return dept_ut_results.circle_detected;
+}
+
+static struct unit units[] = {
+ {
+ .name = "spin lock deadlock test",
+ .func = test_spin_lock_deadlock,
+ },
+ {
+ .name = "mutex lock deadlock test",
+ .func = test_mutex_lock_deadlock,
+ },
+ {
+ .name = "wait event deadlock test",
+ .func = test_wait_event_deadlock,
+ },
+};
+
+static int __init dept_ut_init(void)
+{
+ int i;
+
+ lockdep_off();
+
+ dept_ut_results.ecxt_stack_valid_cnt = 0;
+ dept_ut_results.ecxt_stack_total_cnt = 0;
+ dept_ut_results.wait_stack_valid_cnt = 0;
+ dept_ut_results.wait_stack_total_cnt = 0;
+ dept_ut_results.evnt_stack_valid_cnt = 0;
+ dept_ut_results.evnt_stack_total_cnt = 0;
+
+ for (i = 0; i < ARRAY_SIZE(units); i++)
+ units[i].result = units[i].func();
+
+ pr_info("\n");
+ pr_info("******************************************\n");
+ pr_info("DEPT unit test results\n");
+ pr_info("******************************************\n");
+ for (i = 0; i < ARRAY_SIZE(units); i++) {
+ pr_info("(%s) %s\n", units[i].result ? "pass" : "fail",
+ units[i].name);
+ }
+ pr_info("ecxt stack valid count = %d/%d\n",
+ dept_ut_results.ecxt_stack_valid_cnt,
+ dept_ut_results.ecxt_stack_total_cnt);
+ pr_info("wait stack valid count = %d/%d\n",
+ dept_ut_results.wait_stack_valid_cnt,
+ dept_ut_results.wait_stack_total_cnt);
+ pr_info("event stack valid count = %d/%d\n",
+ dept_ut_results.evnt_stack_valid_cnt,
+ dept_ut_results.evnt_stack_total_cnt);
+ pr_info("******************************************\n");
+ pr_info("\n");
+
+ lockdep_on();
+
+ return 0;
+}
+
+static void dept_ut_cleanup(void)
+{
+ /*
+ * Do nothing for now.
+ */
+}
+
+module_init(dept_ut_init);
+module_exit(dept_ut_cleanup);
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 5c7f22ba253e..41c822f7b75a 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -1477,6 +1477,18 @@ config DEPT_AGGRESSIVE_TIMEOUT_WAIT
that timeout is used to avoid a deadlock. Say N if you'd like
to avoid verbose reports.
+config DEPT_UNIT_TEST
+ tristate "unit test for DEPT"
+ depends on DEBUG_KERNEL && DEPT
+ default n
+ help
+ This option provides a kernel module that runs unit test for
+ DEPT.
+
+ Say Y if you want DEPT unit test to be built into the kernel.
+ Say M if you want DEPT unit test to build as a module.
+ Say N if you are unsure.
+
config LOCK_DEBUGGING_SUPPORT
bool
depends on TRACE_IRQFLAGS_SUPPORT && STACKTRACE_SUPPORT && LOCKDEP_SUPPORT
--
2.17.1
^ permalink raw reply related
* [PATCH v19 39/40] rust: completion: Add __rust_helper to rust_helper_wait_for_completion()
From: Byungchul Park @ 2026-07-06 6:19 UTC (permalink / raw)
To: linux-kernel
Cc: max.byungchul.park, kernel_team, torvalds, damien.lemoal,
linux-ide, adilger.kernel, linux-ext4, mingo, peterz, will, tglx,
rostedt, joel, sashal, daniel.vetter, duyuyang, johannes.berg, tj,
tytso, willy, david, amir73il, gregkh, kernel-team, linux-mm,
akpm, mhocko, minchan, hannes, vdavydov.dev, sj, jglisse, dennis,
cl, penberg, rientjes, vbabka, ngupta, linux-block, josef,
linux-fsdevel, jack, jlayton, dan.j.williams, hch, djwong,
dri-devel, rodrigosiqueiramelo, melissa.srw, hamohammed.sa,
harry.yoo, chris.p.wilson, gwan-gyeong.mun, boqun.feng, longman,
yunseong.kim, ysk, yeoreum.yun, netdev, matthew.brost, her0gyugyu,
corbet, catalin.marinas, bp, x86, hpa, luto, sumit.semwal,
gustavo, christian.koenig, andi.shyti, arnd, lorenzo.stoakes,
Liam.Howlett, rppt, surenb, mcgrof, petr.pavlu, da.gomez,
samitolvanen, paulmck, frederic, neeraj.upadhyay, joelagnelf,
josh, urezki, mathieu.desnoyers, jiangshanlai, qiang.zhang,
juri.lelli, vincent.guittot, dietmar.eggemann, bsegall, mgorman,
vschneid, chuck.lever, neil, okorniev, Dai.Ngo, tom, trondmy,
anna, kees, bigeasy, clrkwllms, mark.rutland, ada.coupriediaz,
kristina.martsenko, wangkefeng.wang, broonie, kevin.brodsky, dwmw,
shakeel.butt, ast, ziy, yuzhao, baolin.wang, usamaarif642,
joel.granados, richard.weiyang, geert+renesas, tim.c.chen, linux,
alexander.shishkin, lillian, chenhuacai, francesco,
guoweikang.kernel, link, jpoimboe, masahiroy, brauner,
thomas.weissschuh, oleg, mjguzik, andrii, wangfushuai, linux-doc,
linux-arm-kernel, linux-media, linaro-mm-sig, linux-i2c,
linux-arch, linux-modules, rcu, linux-nfs, linux-rt-devel,
2407018371, dakr, miguel.ojeda.sandonis, neilb, bagasdotme,
wsa+renesas, dave.hansen, geert, ojeda, alex.gaynor, gary,
bjorn3_gh, lossin, a.hindborg, aliceryhl, tmgross, rust-for-linux
In-Reply-To: <20260706061928.66713-1-byungchul@sk.com>
This is needed to inline these helpers into Rust code, which is required
for DEPT to play with wait_for_completion().
Signed-off-by: Byungchul Park <byungchul@sk.com>
---
rust/helpers/completion.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/rust/helpers/completion.c b/rust/helpers/completion.c
index 5ea2eef74abc..7b55c960fe22 100644
--- a/rust/helpers/completion.c
+++ b/rust/helpers/completion.c
@@ -7,7 +7,7 @@ __rust_helper void rust_helper_init_completion(struct completion *x)
init_completion(x);
}
-void rust_helper_wait_for_completion(struct completion *x)
+__rust_helper void rust_helper_wait_for_completion(struct completion *x)
{
wait_for_completion(x);
}
--
2.17.1
^ permalink raw reply related
* [PATCH v19 38/40] mm: percpu: increase PERCPU_DYNAMIC_SIZE_SHIFT on DEPT and large PAGE_SIZE
From: Byungchul Park @ 2026-07-06 6:19 UTC (permalink / raw)
To: linux-kernel
Cc: max.byungchul.park, kernel_team, torvalds, damien.lemoal,
linux-ide, adilger.kernel, linux-ext4, mingo, peterz, will, tglx,
rostedt, joel, sashal, daniel.vetter, duyuyang, johannes.berg, tj,
tytso, willy, david, amir73il, gregkh, kernel-team, linux-mm,
akpm, mhocko, minchan, hannes, vdavydov.dev, sj, jglisse, dennis,
cl, penberg, rientjes, vbabka, ngupta, linux-block, josef,
linux-fsdevel, jack, jlayton, dan.j.williams, hch, djwong,
dri-devel, rodrigosiqueiramelo, melissa.srw, hamohammed.sa,
harry.yoo, chris.p.wilson, gwan-gyeong.mun, boqun.feng, longman,
yunseong.kim, ysk, yeoreum.yun, netdev, matthew.brost, her0gyugyu,
corbet, catalin.marinas, bp, x86, hpa, luto, sumit.semwal,
gustavo, christian.koenig, andi.shyti, arnd, lorenzo.stoakes,
Liam.Howlett, rppt, surenb, mcgrof, petr.pavlu, da.gomez,
samitolvanen, paulmck, frederic, neeraj.upadhyay, joelagnelf,
josh, urezki, mathieu.desnoyers, jiangshanlai, qiang.zhang,
juri.lelli, vincent.guittot, dietmar.eggemann, bsegall, mgorman,
vschneid, chuck.lever, neil, okorniev, Dai.Ngo, tom, trondmy,
anna, kees, bigeasy, clrkwllms, mark.rutland, ada.coupriediaz,
kristina.martsenko, wangkefeng.wang, broonie, kevin.brodsky, dwmw,
shakeel.butt, ast, ziy, yuzhao, baolin.wang, usamaarif642,
joel.granados, richard.weiyang, geert+renesas, tim.c.chen, linux,
alexander.shishkin, lillian, chenhuacai, francesco,
guoweikang.kernel, link, jpoimboe, masahiroy, brauner,
thomas.weissschuh, oleg, mjguzik, andrii, wangfushuai, linux-doc,
linux-arm-kernel, linux-media, linaro-mm-sig, linux-i2c,
linux-arch, linux-modules, rcu, linux-nfs, linux-rt-devel,
2407018371, dakr, miguel.ojeda.sandonis, neilb, bagasdotme,
wsa+renesas, dave.hansen, geert, ojeda, alex.gaynor, gary,
bjorn3_gh, lossin, a.hindborg, aliceryhl, tmgross, rust-for-linux
In-Reply-To: <20260706061928.66713-1-byungchul@sk.com>
Yunseong reported a build failure due to the BUILD_BUG_ON() statement in
alloc_kmem_cache_cpus(). In the following test:
PERCPU_DYNAMIC_EARLY_SIZE < NR_KMALLOC_TYPES * KMALLOC_SHIFT_HIGH * sizeof(struct kmem_cache_cpu)
The following factors increase the right side of the equation:
1. PAGE_SIZE > 4KiB increases KMALLOC_SHIFT_HIGH.
2. DEPT increases the size of the local_lock_t in kmem_cache_cpu.
Increase PERCPU_DYNAMIC_SIZE_SHIFT to 11 on configs with PAGE_SIZE
larger than 4KiB and DEPT enabled.
Reported-by: Yunseong Kim <ysk@kzalloc.com>
Signed-off-by: Byungchul Park <byungchul@sk.com>
---
include/linux/percpu.h | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/include/linux/percpu.h b/include/linux/percpu.h
index 85bf8dd9f087..dd74321d4bbd 100644
--- a/include/linux/percpu.h
+++ b/include/linux/percpu.h
@@ -43,7 +43,11 @@
# define PERCPU_DYNAMIC_SIZE_SHIFT 12
#endif /* LOCKDEP and PAGE_SIZE > 4KiB */
#else
+#if defined(CONFIG_DEPT) && !defined(CONFIG_PAGE_SIZE_4KB)
+#define PERCPU_DYNAMIC_SIZE_SHIFT 11
+#else
#define PERCPU_DYNAMIC_SIZE_SHIFT 10
+#endif /* DEPT and PAGE_SIZE > 4KiB */
#endif
/*
--
2.17.1
^ permalink raw reply related
* [PATCH v19 37/40] SUNRPC: relocate struct rcu_head to the first field of struct rpc_xprt
From: Byungchul Park @ 2026-07-06 6:19 UTC (permalink / raw)
To: linux-kernel
Cc: max.byungchul.park, kernel_team, torvalds, damien.lemoal,
linux-ide, adilger.kernel, linux-ext4, mingo, peterz, will, tglx,
rostedt, joel, sashal, daniel.vetter, duyuyang, johannes.berg, tj,
tytso, willy, david, amir73il, gregkh, kernel-team, linux-mm,
akpm, mhocko, minchan, hannes, vdavydov.dev, sj, jglisse, dennis,
cl, penberg, rientjes, vbabka, ngupta, linux-block, josef,
linux-fsdevel, jack, jlayton, dan.j.williams, hch, djwong,
dri-devel, rodrigosiqueiramelo, melissa.srw, hamohammed.sa,
harry.yoo, chris.p.wilson, gwan-gyeong.mun, boqun.feng, longman,
yunseong.kim, ysk, yeoreum.yun, netdev, matthew.brost, her0gyugyu,
corbet, catalin.marinas, bp, x86, hpa, luto, sumit.semwal,
gustavo, christian.koenig, andi.shyti, arnd, lorenzo.stoakes,
Liam.Howlett, rppt, surenb, mcgrof, petr.pavlu, da.gomez,
samitolvanen, paulmck, frederic, neeraj.upadhyay, joelagnelf,
josh, urezki, mathieu.desnoyers, jiangshanlai, qiang.zhang,
juri.lelli, vincent.guittot, dietmar.eggemann, bsegall, mgorman,
vschneid, chuck.lever, neil, okorniev, Dai.Ngo, tom, trondmy,
anna, kees, bigeasy, clrkwllms, mark.rutland, ada.coupriediaz,
kristina.martsenko, wangkefeng.wang, broonie, kevin.brodsky, dwmw,
shakeel.butt, ast, ziy, yuzhao, baolin.wang, usamaarif642,
joel.granados, richard.weiyang, geert+renesas, tim.c.chen, linux,
alexander.shishkin, lillian, chenhuacai, francesco,
guoweikang.kernel, link, jpoimboe, masahiroy, brauner,
thomas.weissschuh, oleg, mjguzik, andrii, wangfushuai, linux-doc,
linux-arm-kernel, linux-media, linaro-mm-sig, linux-i2c,
linux-arch, linux-modules, rcu, linux-nfs, linux-rt-devel,
2407018371, dakr, miguel.ojeda.sandonis, neilb, bagasdotme,
wsa+renesas, dave.hansen, geert, ojeda, alex.gaynor, gary,
bjorn3_gh, lossin, a.hindborg, aliceryhl, tmgross, rust-for-linux
In-Reply-To: <20260706061928.66713-1-byungchul@sk.com>
While compiling Linux kernel with DEPT on, the following error was
observed:
./include/linux/rcupdate.h:1084:17: note: in expansion of macro
‘BUILD_BUG_ON’
1084 | BUILD_BUG_ON(offsetof(typeof(*(ptr)), rhf) >= 4096); \
| ^~~~~~~~~~~~
./include/linux/rcupdate.h:1047:29: note: in expansion of macro
'kvfree_rcu_arg_2'
1047 | #define kfree_rcu(ptr, rhf) kvfree_rcu_arg_2(ptr, rhf)
| ^~~~~~~~~~~~~~~~
net/sunrpc/xprt.c:1856:9: note: in expansion of macro 'kfree_rcu'
1856 | kfree_rcu(xprt, rcu);
| ^~~~~~~~~
CC net/kcm/kcmproc.o
make[4]: *** [scripts/Makefile.build:203: net/sunrpc/xprt.o] Error 1
Since kfree_rcu() assumes 'offset of struct rcu_head in a rcu-managed
struct < 4096', the offest of struct rcu_head in struct rpc_xprt should
not exceed 4096 but does, due to the debug information added by DEPT.
Relocate struct rcu_head to the first field of struct rpc_xprt from an
arbitrary location to avoid the issue and meet the assumption.
Reported-by: Yunseong Kim <ysk@kzalloc.com>
Signed-off-by: Byungchul Park <byungchul@sk.com>
Reviewed-by: Jeff Layton <jlayton@kernel.org>
---
include/linux/sunrpc/xprt.h | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/include/linux/sunrpc/xprt.h b/include/linux/sunrpc/xprt.h
index f46d1fb8f71a..666e42a17a31 100644
--- a/include/linux/sunrpc/xprt.h
+++ b/include/linux/sunrpc/xprt.h
@@ -211,6 +211,14 @@ enum xprt_transports {
struct rpc_sysfs_xprt;
struct rpc_xprt {
+ /*
+ * Place struct rcu_head within the first 4096 bytes of struct
+ * rpc_xprt if sizeof(struct rpc_xprt) > 4096, so that
+ * kfree_rcu() can simply work assuming that. See the comment
+ * in kfree_rcu().
+ */
+ struct rcu_head rcu;
+
struct kref kref; /* Reference count */
const struct rpc_xprt_ops *ops; /* transport methods */
unsigned int id; /* transport id */
@@ -317,7 +325,6 @@ struct rpc_xprt {
#if IS_ENABLED(CONFIG_SUNRPC_DEBUG)
struct dentry *debugfs; /* debugfs directory */
#endif
- struct rcu_head rcu;
const struct xprt_class *xprt_class;
struct rpc_sysfs_xprt *xprt_sysfs;
bool main; /*mark if this is the 1st transport */
--
2.17.1
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox