[PATCH v17 05/17] iov_iter: Add an API document for the bvecq structure

[David Howells <dhowells@redhat.com>]

Add an API description document for the bvecq structure.

Signed-off-by: David Howells <dhowells@redhat.com>
cc: Paulo Alcantara (Red Hat) <pc@manguebit.org>
cc: Matthew Wilcox <willy@infradead.org>
cc: Jens Axboe <axboe@kernel.dk>
cc: Christoph Hellwig <hch@infradead.org>
cc: linux-block@vger.kernel.org
cc: netfs@lists.linux.dev
cc: linux-fsdevel@vger.kernel.org
---
 Documentation/core-api/bvecq.rst | 229 +++++++++++++++++++++++++++++++
 Documentation/core-api/index.rst |   1 +
 MAINTAINERS                      |   1 +
 fs/netfs/bvecq.c                 |   2 +
 include/linux/bvecq.h            |   2 +
 5 files changed, 235 insertions(+)
 create mode 100644 Documentation/core-api/bvecq.rst

diff --git a/Documentation/core-api/bvecq.rst b/Documentation/core-api/bvecq.rst
new file mode 100644
index 000000000000..4a2b3e288982
--- /dev/null
+++ b/Documentation/core-api/bvecq.rst
@@ -0,0 +1,229 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=========================================
+bvecq - Chainable bio_vec queue structure
+=========================================
+
+Author: David Howells <dhowells@redhat.com>
+
+.. Contents:
+
+ * Overview
+ * Allocating bvecqs
+ * Refcounting
+ * Expandable buffer
+ * Producer functions
+ * Consumer functions
+ * Utility functions
+ * I/O iterators
+ * API Function Reference
+
+
+Overview
+========
+
+The ``bvecq`` struct can be used to form chains of arrays of ``bio_vec``
+structures - kind of a single layer of indirection.  Each bvecq segment points
+to the previous and next bvecq segments in the chain, but is not intended to
+form a circular queue as there are applications where the producer is adding to
+one end whilst the consumer is taking off the other.
+
+To use this, the caller should do::
+
+	#include <linux/bvecq.h>
+
+The publicly accessible members of the structure are::
+
+	struct bvecq {
+		struct bvecq	*next;
+		struct bvecq	*prev;
+		u32		priv;
+		u16		nr_slots;
+		u16		max_slots;
+		enum bvecq_mem	mem_type:3;
+		struct bio_vec	*bv;
+		...
+	};
+
+A pair of pointers are provided, ``next`` and ``prev``, that point to the
+segments on either side of the segment being accessed.  Whilst this is a
+doubly-linked list, it is intentionally not a circular list; the outward
+sibling pointers in terminal segments should be NULL.
+
+The ``priv`` member is for the owner of the bvecq to use as they will.
+
+The ``bv`` pointer points at an array of bio_vecs that list a number of regions
+of memory.  ``max_slots`` indicates how large the array is and ``nr_slots``
+says how much of the array is being used.  Note that there are access functions
+for nr_slots that insert appropriate barriering in the case that there are
+separate producer and consumer threads.
+
+The ``mem_type`` member informs bvecq_put() of how to clean up the bio_vecs
+attached to this segment.  It can be one of:
+
+ * ``BVECQ_MEM_EXTERNAL`` - Externally managed; no cleanup.
+ * ``BVECQ_MEM_PAGECACHE`` - Part of pagecache; put_page() used.
+ * ``BVECQ_MEM_GUP`` - Pinned userspace memory; unpin_user_page() used.
+ * ``BVECQ_MEM_ALLOCED`` - Allocated page; __free_pages() used, may be pooled.
+
+
+Allocating bvecqs
+=================
+
+A number of functions are provided to allocate bvecqs::
+
+	struct bvecq *bvecq_alloc_one(size_t nr_slots,
+				      gfp_t gfp,
+				      bool for_writeback);
+	struct bvecq *bvecq_alloc_chain(size_t nr_slots,
+					gfp_t gfp,
+					bool for_writeback);
+	struct bvecq *bvecq_alloc_buffer(size_t size,
+					 gfp_t gfp,
+					 bool for_writeback);
+	struct bvecq *bvecq_alloc_buffer2(size_t size,
+					  unsigned int pre_slots,
+					  gfp_t gfp,
+					  bool for_writeback);
+
+The first of these, ``bvecq_alloc_one()`` allocates a single bvecq with at
+least the requested number of inline slots.  The size of the object is rounded
+up to the size of the slab granule that it will be allocated in and the number
+of slots increased appropriately to match.  If ``for_writeback`` is set, then
+the netfslib writeback bvecq pool will be used to backstop the allocation if
+the initial slab request fails; in such a case, the number of slots will get
+reduced to ``BVECQ_POOL_SLOTS``.
+
+Two convenience slot count constants are provided:
+
+ * ``BVECQ_POOL_SLOTS`` - The right number of slots to alloc from the mempool
+ * ``BVECQ_4KB_SLOTS`` - The right number of slots to get a 4K allocation
+
+``bvecq_alloc_chain()`` will allocate a sufficiently large chain of bvecqs to
+hold the requested number of slots, capping each allocation at 4K.
+
+``bvecq_alloc_buffer()`` will allocate a chain and populate it, allocating
+sufficient pages to provide a buffer of the specified size attached to the
+chain.  ``bvecq_alloc_buffer2()`` is similar, but it reserves some unused
+slots at the front of the chain.  Note that these functions may add multipage
+pages in the future.
+
+
+Refcounting
+===========
+
+Each bvecq has a refcount and two functions are provided to handle that::
+
+	struct bvecq *bvecq_get(struct bvecq *bq);
+	void bvecq_put(struct bvecq *bq);
+
+The first of these gets a new reference and the second drops a reference.  When
+the refcount hits zero, the second iteratively drops the refs along the chain
+until it hits a segment that has a refcount that doesn't reduce to zero.
+
+
+Expandable buffer
+=================
+
+This facility provides a function to manage an expandable buffer::
+
+	int bvecq_expand_buffer(struct bvecq **_buffer,
+				size_t *_cur_size,
+				size_t size,
+				gfp_t gfp);
+
+Pages are added to the bvecq chain and the chain expanded such that, on
+returning, it holds ``size`` amount of space (without losing any content).  The
+caller needs to retain ``*_cur_size`` between calls as that tracks the amount
+of space currently allocated.  Note that pages in the chain may be allocated as
+compound pages larger than PAGE_SIZE.
+
+
+Producer functions
+==================
+
+When building and filling a bvecq, there are a number of utility functions that
+can be used to provide the correct barriering and to query the fields in the
+bvecq currently being filled.
+
+Firstly, there's a function to check to see if a bvecq has any spare slots
+left::
+
+	bool bvecq_is_full(const struct bvecq *bvecq);
+
+If not, ``bvecq_alloc_one()`` can be called to allocate a new segment which can
+be initialised and then added to the chain with::
+
+	void bvecq_append(struct bvecq *to, struct bvecq *add);
+
+bio_vec slots in the segment can then be filled and the number of occupied
+slots updated by calling::
+
+	void bvecq_filled_to(struct bvecq *bvecq,
+			     unsigned int to);
+
+to insert the appropriate barrier.
+
+
+Consumer functions
+==================
+
+When consuming and destroying a bvecq, there are some utility functions for
+that too, providing the needed barriering.  Firstly, there's a function to find
+out how many slots have been used in the current bvecq::
+
+	unsigned int bvecq_nr_slots_acquire(const struct bvecq *bvecq);
+
+Or, more conveniently, if the current slot position is valid within the bvecq::
+
+	bool bvecq_acquire_slot(const struct bvecq *bvecq,
+				unsigned int slot);
+
+If this returns true, the slot is valid and can be looked at; if it returns
+false, the bvecq may have been consumed.  If it has, the next bvecq can be
+obtained by calling::
+
+	struct bvecq *bvecq_next(const struct bvecq *bq);
+
+which inserts the appropriate barrier - though bvecq_acquire_slot should be
+rechecked after calling this in case the producer added more slots and then
+extended the list.
+
+
+Utility functions
+=================
+
+A utility function is provided to list the bio_vecs pointed to by a bvecq
+chain::
+
+	void bvecq_dump(const struct bvecq *bq);
+
+
+I/O iterators
+=============
+
+bvecq chains can be used with the ``iov_iter`` facility too.  An iterator can
+be initialised to point at a chain by calling::
+
+	void iov_iter_bvec_queue(struct iov_iter *i,
+				 unsigned int direction,
+				 const struct bvecq *bvecq,
+				 unsigned int first_slot,
+				 unsigned int offset,
+				 size_t count);
+
+In addition to setting the iterator, direction and count, the first bvecq in
+the chain to be iterated over is set, along with the starting slot in that
+iterator and the offset into the slot.  Once the iterator is set up, it can be
+treated like any other I/O iterator.
+
+In a producer/consumer environment, it is expected that when an iterator is set
+up, all the bvecqs and all the slots within the count of the iterator must
+exist and must be populated before this function is called.
+
+
+API Function Reference
+======================
+
+.. kernel-doc:: include/linux/bvecq.h
+.. kernel-doc:: fs/netfs/bvecq.c
diff --git a/Documentation/core-api/index.rst b/Documentation/core-api/index.rst
index 92f91c6a0d79..e39edef58ab6 100644
--- a/Documentation/core-api/index.rst
+++ b/Documentation/core-api/index.rst
@@ -39,6 +39,7 @@ Library functionality that is used throughout the kernel.
    kref
    cleanup
    assoc_array
+   bvecq
    folio_queue
    xarray
    maple_tree
diff --git a/MAINTAINERS b/MAINTAINERS
index 6fa777744163..9cb3785a5bd2 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -10382,6 +10382,7 @@ M:	Paulo Alcantara <pc@manguebit.org>
 L:	netfs@lists.linux.dev
 L:	linux-fsdevel@vger.kernel.org
 S:	Supported
+F:	Documentation/core-api/bvecq.rst
 F:	Documentation/filesystems/caching/
 F:	Documentation/filesystems/netfs_library.rst
 F:	fs/netfs/
diff --git a/fs/netfs/bvecq.c b/fs/netfs/bvecq.c
index 1d4da3de9b9d..f1264ae52033 100644
--- a/fs/netfs/bvecq.c
+++ b/fs/netfs/bvecq.c
@@ -3,6 +3,8 @@
  *
  * Copyright (C) 2026 Red Hat, Inc. All Rights Reserved.
  * Written by David Howells (dhowells@redhat.com)
+ *
+ * See also Documentation/core-api/bvecq.rst
  */
 
 #include <linux/bvecq.h>
diff --git a/include/linux/bvecq.h b/include/linux/bvecq.h
index d9d3dd487a27..90e6eb9588bf 100644
--- a/include/linux/bvecq.h
+++ b/include/linux/bvecq.h
@@ -3,6 +3,8 @@
  *
  * Copyright (C) 2026 Red Hat, Inc. All Rights Reserved.
  * Written by David Howells (dhowells@redhat.com)
+ *
+ * See also Documentation/core-api/bvecq.rst
  */
 
 #ifndef _LINUX_BVECQ_H