Linux filesystem development
 help / color / mirror / Atom feed
From: Christoph Hellwig <hch@infradead.org>
To: David Howells <dhowells@redhat.com>
Cc: Christian Brauner <christian@brauner.io>,
	Matthew Wilcox <willy@infradead.org>,
	Christoph Hellwig <hch@infradead.org>,
	Paulo Alcantara <pc@manguebit.org>, Jens Axboe <axboe@kernel.dk>,
	Leon Romanovsky <leon@kernel.org>,
	Steve French <sfrench@samba.org>,
	ChenXiaoSong <chenxiaosong@chenxiaosong.com>,
	Marc Dionne <marc.dionne@auristor.com>,
	Stefan Metzmacher <metze@samba.org>,
	Eric Van Hensbergen <ericvh@kernel.org>,
	Dominique Martinet <asmadeus@codewreck.org>,
	Ilya Dryomov <idryomov@gmail.com>,
	netfs@lists.linux.dev, linux-afs@lists.infradead.org,
	linux-cifs@vger.kernel.org, linux-nfs@vger.kernel.org,
	ceph-devel@vger.kernel.org, v9fs@lists.linux.dev,
	linux-erofs@lists.ozlabs.org, linux-fsdevel@vger.kernel.org,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH v9 00/26] netfs: Keep track of folios in a segmented bio_vec[] chain
Date: Mon, 10 Aug 2026 07:51:07 -0700	[thread overview]
Message-ID: <annlWwsCPjNgcKa-@infradead.org> (raw)
In-Reply-To: <20260810144746.574036-1-dhowells@redhat.com>

On Mon, Aug 10, 2026 at 03:47:17PM +0100, David Howells wrote:
> Hi Christian,
> 
> Could you add these patches to the VFS tree for next?  Though, given it may
> be a little late for that, could you at least take the fix patch at the
> front?
> 
> The patches get rid of folio_queue, rolling_buffer and ITER_FOLIOQ,
> replacing the folio queue construct used to manage buffers in netfslib with
> one based around a segmented chain of bio_vec arrays instead.  There are
> three main aims here:

Christian had a mail that a previous version was applied, but I think
this is a bad idea.  This is a giant series that also touches core code
with no review whatsoever.  And I repeatedly asked for:

 1) the core changes to be split out
 2) come up with users for this very core primitive in the hot path
    (iov_iter) that are not just for your rather niche use case

There's been absolutely no follow on to this, and we should not treat
the absolute core I/O path like this.

> 
>  (1) The kernel file I/O subsystem seems to be moving towards consolidating
>      on the use of bio_vec arrays, so embrace this by moving netfslib to
>      keep track of its buffers for buffered I/O in bio_vec[] form.
> 
>  (2) Netfslib already uses a bio_vec[] to handle unbuffered/DIO, so the
>      number of different buffering schemes used can be reduced to just a
>      single one.
> 
>  (3) Always send an entire filesystem RPC request message to a TCP socket
>      with single kernel_sendmsg() call as this is faster, more efficient
>      and doesn't require the use of corking as it puts the entire
>      transmission loop inside of a single tcp_sendmsg().
> 
> For the replacement of folio_queue, a segmented chain of bio_vec arrays
> rather than a single monolithic array is provided:
> 
> 	struct bvecq {
> 		struct bvecq		*next;
> 		struct bvecq		*prev;
> 		unsigned long long	fpos;
> 		refcount_t		ref;
> 		u32			priv;
> 		u16			nr_slots;
> 		u16			max_slots;
> 		enum bvecq_mem		mem_type:2;
> 		bool			inline_bv:1;
> 		bool			discontig:1;
> 		bool			from_pool:1;
> 		struct bio_vec		*bv;
> 		struct bio_vec		__bv[];
> 	};
> 
> The fields are:
> 
>  (1) next, prev - Link segments together in a list.  I want this to be
>      NULL-terminated linear rather than circular to make it possible to
>      arbitrarily glue bits on the front.
> 
>  (2) fpos, discontig - Note the current file position of the first byte of
>      the segment and whether this bvecq is discontiguous with the previous.
>      When accessing the pagecache to clear flags/locks, the fpos can be
>      used to look up folios by file position rather than by finding those
>      folios from the info stored in the bio_vecs.
> 
>      When the file position is relevant, the model I'm working with is that
>      all the segments pointed to by a single bvecq must represent
>      contiguous data, but adjacent bvecqs within a chain need not be
>      contiguous.  This allows a bvecq chain to be used to provide bufferage
>      for a sparse read or write RPC such as can be done with Ceph.
> 
>      If a bvecq segment is not contiguous with the previous one,
>      ->discontig should be set (this is technically redundant if one keeps
>      track of the fpos as a bvecq chain is processed).
> 
>      Note that the beginning and end file positions in a segment need not
>      be aligned to any filesystem block size.
> 
>  (3) ref - Refcount.  Each bvecq keeps a ref on the next.  I'm not sure
>      this is entirely necessary, but it makes sharing slices easier.
> 
>  (4) priv - Private data for the owner.  Dispensible; currently only used
>      for storing a debug ID for tracing in a patch not included here.
> 
>  (5) max_slots, nr_slots.  The size of bv[] and the number of slots used.
>      I've assumed a maximum of 65535 bio_vecs in the array (which would
>      represent a ~1MiB allocation).
> 
>  (6) bv, __bv, inline_bv.  bv points to the bio_vec[] array handled by
>      this segment.  This may begin at __bv and if it does inline_bv should
>      be set (otherwise it's impossible to distinguish a separately
>      allocated bio_vec[] that follows immediately by coincidence).
> 
>  (7) mem_type.  Indicates how the memory attached to the bio_vecs should be
>      disposed of when the bvecq is destroyed.  It can be one of:
> 
> 	BVECQ_MEM_EXTERNAL	- Externally tracked ref; don't put
> 	BVECQ_MEM_PAGECACHE	- Pagecache; must be put
> 	BVECQ_MEM_GUP		- Pinned by from GUP; needs unpin
> 	BVECQ_MEM_ALLOCED	- Plain alloc'd pages; can be mempooled
> 
>      [!] I'm not sure that this is a good name for this member or for the
>      	 enum values.
> 
>  (8) from_pool.  Set if the bvecq is allocated from netfslib's mempool and
>      should be freed to it.
> 
> I've also defined an iov_iter iterator type ITER_BVECQ to walk this sort of
> construct so that it can be passed directly to sendmsg() or block-based DIO
> (as cachefiles does).
> 
> 
> This series makes the following changes to netfslib:
> 
>  (1) Remove the writethrough code as the locking is really tricky to get
>      right and it looks like it could deadlock with Ceph if snapshots are
>      used.
> 
>  (2) The folio_queue chain used to hold folios for buffered I/O is replaced
>      with a bvecq chain.  Each bio_vec then holds (a portion of) one folio.
>      Each bvecq holds a contiguous sequence of folios, but adjacent bvecqs
>      in a chain may be discontiguous.
> 
>  (3) For unbuffered/DIO, the source iov_iter is extracted into a bvecq
>      chain.
> 
>  (4) An abstract position representation ('bvecq_pos') is created that can
>      used to hold a position in a bvecq chain.  For the moment, this takes
>      a ref on the bvecq it points to, but that may be excessive.
> 
>  (5) Buffer tracking is managed with three cursors:  The load_cursor, at
>      which new folios are added as we go; the dispatch_cursor, at which new
>      subrequests' buffers start when they're created; and the
>      collect_cursor, the point at which folios are being unlocked.
> 
>      Not all cursors are necessarily needed in all situations and during
>      buffered writeback, we need a dispatch cursor per stream (one for the
>      network filesystem and one for the cache).
> 
>  (6) ->prepare_read(), buffer setting up and ->issue_read() are merged, as
>      are the write variants, with the filesystem calling back up to
>      netfslib to prepare its buffer.  This simplifies the process of
>      setting up a subrequest.  It may even make sense to have the
>      filesystem allocate the subrequest.
> 
>  (7) Retry dispatch tracking is added to netfs_io_request so that the
>      buffer preparation functions can find it.  Retry requires an
>      additional buffer cursor.
> 
>  (8) Netfslib dispatches I/O by accumulating enough bufferage to dispatch
>      at least one subrequest, then looping to generate as many as the
>      filesystem wants to (they may be limited by other constraints,
>      e.g. max RDMA segment count or negotiated max size).  This loop could
>      be moved down into the filesystem.  A new method is provided by which
>      netfslib can ask the filesystem to provide an estimate of the data
>      that should be accumulated before dispatch begins.
> 
>  (9) Reading from the cache is now managed by querying the cache to provide
>      a list of the next two data extents within the cache.
> 
> (10) AFS directories are switched to using a bvecq rather than a
>      folio_queue to hold their contents.
> 
> (11) CIFS is switch to using a bvecq rather than a folio_queue for holding
>      a temporary encryption buffer.
> 
> (12) CIFS RDMA is given the ability to extract ITER_BVECQ and support for
>      extracting ITER_FOLIOQ is removed.
> 
> (13) All the folio_queue and rolling_buffer code is removed.
> 
> Cachefiles is also modified:
> 
>  (1) On-demand mode for Erofs is removed.
> 
>  (2) The object type in the cachefiles file xattr is now correctly set to
>      CACHEFILES_CONTENT_{SINGLE,ALL,BACKFS_MAP} rather than just being 0,
>      to indicate whether we have a single monolithic blob, all the data up
>      to cache i_size with no holes or a sparse file with the data mapped by
>      the backing file system (as currently upstream).
> 
>  (3) For "ALL" type files, the cache's i_size is used to track how much
>      data is saved in the cache and no longer bears any relation to the
>      netfs i_size.  The actual object size is stored in the xattr.
> 
>  (4) For most typical files which are contiguous and written progressively,
>      the object type is now set to "ALL".  For anything else, cachefiles
>      uses SEEK_DATA/HOLE to find extent outlines at before (this is the
>      current behaviour and needs to be fixed, but in a separate set of
>      patches as it's not trivial).
> 
>  (5) Preset the xattr on a cachefile to try to avoid having to deal with
>      ENOSPC from setxattr when committing the object.
> 
> Two further things that I'm working on (but not in this branch) are:
> 
>  (1) Make it so that a filesystem can be given a copy of a subchain which
>      it can then tack header and trailer protocol elements upon to form a
>      single message (I have this working for cifs) and even join copies
>      together with intervening protocol elements to form compounds.
> 
>  (2) Make it so that a filesystem can 'splice' out the contents of the TCP
>      receive queue into a bvecq chain.  This allows the socket lock to be
>      dropped much more quickly and the copying of data read to the
>      destination buffers to happen without the lock.  I have this working
>      for cifs too.  Kernel recvmsg() doesn't then block kernel sendmsg()
>      for anywhere near as long.
> 
> There are also some things I want to consider for the future:
> 
>  (1) Create one or more batched iteration functions to 'unlock' all the
>      folios in a bio_vec[], where 'unlock' is the appropriate action for
>      ending a read or a write.  Batching should hopefully also improve the
>      efficiency of wrangling the marks on the xarray.  Very often these
>      marks are going to be represented by contiguous bits, so there may be
>      a way to change them in bulk.
> 
>  (2) Rather than walking the bvecq chain to get each individual folio out
>      via bv_page, use the file position stored on the bvecq and the sum of
>      bv_len to iterate over the appropriate range in i_pages.
> 
>  (3) Change iov_iter to store the initial starting point and for
>      iov_iter_revert() to reset to that and advance.  This would (a) help
>      prevent over-reversion and (b) dispense with the need for a prev
>      pointer.
> 
>  (4) Use bvecq to replace scatterlist.  One problem with replacing
>      scatterlist is that crypto drivers like to glue bits on the front of
>      the scatterlists they're given (something trivial with that API) - and
>      this is one way to achieve it.
> 
> The patches can also be found here:
> 
> 	https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/log/?h=netfs-next
> 
> Thanks,
> David
> 
> Changes
> =======
> ver #9)
> - Fixed a number of issues reported by Sashiko[7]:
>   - Fixed netfs_unlock_abandoned_read_pages() to make sure the refs are
>     dropped before unlocking.
>   - Fixed iov_iter_extract_bvecq_pages() to immediately return 0 if
>     there are no pages to be extracted from the iterator (e.g. if maxpages
>     is 0).
>   - Pass a flag into bvecq_alloc_one() to indicate if we can access
>     writeback memory pools rather than trying to divine this from gfp.
>   - Fixed bvecq_alloc_one() to use the same gfp adjustments as
>     mempool_alloc() when prospectively allocating what the caller asked for
>     during writeback.
>   - Simplified the bvecq-based rolling buffer in writeback by always adding
>     whole folios and then setting the offset appropriately.  The offset and
>     length of the part added can then be aligned at that point when
>     tracking the new folio in the cache stream rather than copying and
>     rounding out later.
>   - Changed the writeback loop to ignore ENOMEM.  There are now backup
>     memory pools for writeback.
>   - Simplified the writeback loop to update stream tracking as folios are
>     added and to flush at that point too if needed.
>   - Removed the warning given by netfs_extract_iter() if max_len or
>     max_pages is 0.
>   - Made netfs_extract_iter() set the fpos on the blank bvecq it allocates
>     in the alloc_empty path.
>   - Fixed netfs_writeback_single() to test NETFS_RREQ_UPLOAD_TO_SERVER
>     rather than test-and-setting it (nothing currently wants to
>     writeback-single to the server anyway, only to the cache).
>   - Removed cachefiles ondemand pincount.
>   - Fixed netfs_collect_write_results() to get the cancel/data flags to
>     netfs_collect_write_results() the right way round.
>   - Fixed netfs_writeback_unlock_folios() to just warn and oops in the
>     unlikely event that it sees a NULL (ie. already unlocked) folio
>     pointer.
>   - Moved some common initialisation of fields to rreq->start to
>     netfs_alloc_request().
>   - Fixed bvecq_load_from_ra() to set bq->fpos.
>   - Fixed read progress reporting to use an offset from rreq->start
>     (unchanging) rather than trying to use rreq->clean_to (tearable) as a
>     base.
> 
> ver #8)
> - Removed the fix patches to their own branches and posted separately.
> - Rebased on v7.2-rc6 as a number of fixes went in.
> - Fixed cachefiles_collect_write() to handle cres->cache_priv2 (file) being
>   NULL due to failure to create an object (e.g. due to ENOSPC).
> - Added a patch to cachefiles to preset the xattr on a cachefiles to avoid
>   dealing with ENOSPC from setxattr when committing an object.
> - Fixed a number of bugs reported by Sashiko[6]:
>   - Fixed bvecq_alloc_buffer2() to break out of the loop if pre_slots > 0
>     and size == 0.
>   - Made netfs_extract_iter() limit max_pages to protect iov_iter_npages().
>   - Remove include/uapi/linux/cachefiles.h when ondemand mode is removed.
>   - Remove more ondemand-related trace bits.
>   - Remove ondemand-related BPF bits.
>   - Fixed a missing ENOMEM check in netfs_pgpriv2_begin_copy_to_cache().
>   - Fixed netfs_retry_read_subrequests() to unset dispatch_cursor on the
>     "abandon_after" error path.
>   - Fixed netfs_issue_write() to clear stream->construct if a subreq is
>     terminated due to ENOMEM.
>   - Fixed netfs_collect_read_results() to unpause the app if it decides to
>     abandon the read.
>   - Fixed cachefiles_issue_read() to not wait for the object state to match
>     FSCACHE_WANT_READ as cachefiles_query_occupancy() should have done that
>     already.
>   - Fixed netfs_retry_write_stream() to handle superfluous subreqs
>     correctly (borrowing from read retry).
> 
> ver #7)
> - Rebased on v7.2-rc4.
> - Added three fix patches to the front from Yichong Chen that conflict with
>   other patches in the series.
> - Fixed a number of bugs reported by Sashiko[5].
>   - Fixed double page put in iov_kunit_destroy_bvecq().
>   - Fixed bvecq_alloc_buffer2() to handle pre_slots>0 and size==0
>     correctly.
>   - Fixes bvecq_expand_buffer() to use a barrer to set tail->nr_slots.
>   - Added functions to insert barriers when reading or setting bvecq::next
>     and fixed some places to use them.
>   - Fixed more meta block mapping in afs_edit_dir_remove().
>   - Added a patch to fix missing unmap in afs_dir_search_bucket().
>   - Removed the writethrough stat counter as the code gets removed.
>   - Cleaned up the cachefiles-ondemand removal patch:
>     - Fix removal of code that's conditional on !ONDEMAND.
>     - Removed msg_id_next field.
>     - Removed ondemand trace elements.
>     - Removed documentation.
>   - Fixed cachefiles_resize_cookie() to only consider a resize a shrink if
>     the new size is less than the object size.
>   - Fixed cachefiles_resize_cookie() to update object->object_size on
>     expansion.
>   - Add a comment on exclusion in cachefiles_collect_write().
>   - Fixed netfs_single_dispatch_read() to allow for the cache to have
>     rounded out the stored data to DIO size.
>   - Fixed netfs_prepare_read_iterator() to clamp subreq len to sreq_max_len
>     (obsoleted by last patch).
>   - Changed write subreq collection to never retry cache writes.  Shouldn't
>     see them in the retry path either then.
>   - Fixed netfs_pgpriv2_unlock_copied_folios() to return made_progress
>     instead of false if we hit the end of the bvecq chain.
>   - Modified buffered read collection to abandon the rest of a read request
>     if a subreq over-reads as the buffer may have been corrupted.  The read
>     termination function no longer winds subreq->transferred back.
>   - Fixed netfs_issue_read() to call the right termination func in the fail
>     path.
>   - Fixed netfs_dispatch_unbuffered_reads(), netfs_single_dispatch_read()
>     and netfs_retry_read_subrequests() to handle failed ->issue_read().
>   - Fixed afs_issue_read() to return after calling afs_end_read() as this
>     op is then released.
>   - Fixed cifs_issue_write() to always release credits on failure.
>   - Fixed cifs_issue_read() to not release credits twice.
>   - Fixed netfs_issue_writes() to advance stream->issued_to and
>     stream->dispatch_cursor on early failure of ->issue_write().
>   - Fixed netfs_issue_read() to only call netfs_all_subreqs_queued() if
>     stream->buffered is zero.
>   - Fixed ceph_netfs_issue_op_inline() to do a retry if file got uninlined
>     by someone else.
>   - Fixed ceph_netfs_issue_op_inline() to not return an error directly
>     once it's got a prepared buffer.
>   - Fixed nfs_netfs_issue_read() to return an int.
> 
> ver #6)
> - Rebased on v7.2-rc3.
> - Added a patch to remove erofs on-demand support from cachefiles.
> - Added a patch to remove the writethrough code as it's really tricky to
>   get the locking right and it looks like can deadlock itself with Ceph.
> - Fixed a number of bugs reported by Sashiko[4].
>   - Fixed iov_iter_single_seg_count(), iov_iter_alignment_bvecq() and
>     iov_npages_bvecq() to handle zero-length segments and segments with no
>     stuff remaining.
>   - Fixed bvecq_extract() to use GFP_NOFS rather than GFP_KERNEL.
>   - Fixed bvecq_alloc_buffer2()'s use of alloc_pages_bulk().
>   - Fixed netfs_extract_iter() to break out of the inner loop if max_pages
>     hits zero.
>   - Added a patch to fix meta block mapping in afs_edit_dir_remove().
>   - Fixed afs_init_new_symlink() to clear the tail of the symlink page.
>   - Fixed cachefiles_begin_operation() to deal with a NULL file.
>   - Fixed slot check in netfs_read_gaps() to be >=, not >.
>   - Fixed netfs_pgpriv2_copy_folio() to update creq->last_end.
>   - Fixed netfs_read_single() to return -EIO if the buffer isn't big
>     enough.
>   - Add a comment to netfs_write_folio() to indicate that truncate must
>     not run concurrently with writeback and must exclude/wait for it.
>   - In netfs_read_subreq_terminated(), move the netfs_sreq_trace_too_much
>     trace line earlier before ->transferred is clobbered.
>   - In netfs_read_subreq_terminated(), make the transfer-too-long case
>     avoid retrying and just end with EIO (in case data further along in the
>     buffer got corrupted).
>   - Fixed netfs_collect_read_results() if it sees front->transferred >
>     front->len, to just use front->len instead.
>   - Fixed ceph_netfs_issue_op_inline() to prep the buffer before setting
>     NETFS_SREQ_HIT_EOF so that subreq->content is set.
>   - Alter netfs_read_to_pagecache() to not round up reading from
>     zero-containing cache granules as no I/O is required.
>   - Made ->issue_write() return an error so that errors before any buffer
>     is sliced off don't result in an infinite loop in netfs_issue_writes().
>   - Made ->issue_read() return an error also for similiar reasons.
>   - In netfs_writepages(), handle -ENOMEM from bvecq prealloc before
>     caling netfs_queue_wb_folio() rather than doing it on the combined
>     error return of that and netfs_issue_streams() which can result in a
>     double unlock.
>   - Fix netfs_writeback_single() to write nothing if len == 0.
> 
> ver #5)
> - Rebased on v7.2-rc2 as that has a bunch of outstanding netfs and afs
>   bugfixes included.
> 
> ver #4)
> - Fixed a number of bugs reported by Sashiko[3].
>   - Added a patch to fix an underflow in iov_iter_extract_xarray_pages().
>   - Added a patch to fix alloc failure in iov_iter_extract_bvec_pages().
>   - Added a patch to remove an unused var in kunit code.
>   - Added a patch to fix the folio offset in extract_xarray_to_sg().
>   - Added a patch to fix the exclusion over writeback to make it cover
>     collection too.
>   - Fixed double fput() in cachefiles.
>   - Fixed the collection of cache writes to handle cancellation better.
>   - Fixed iterate_bvecq() to skip bvecq structs with nr_slots==0.
>   - Add a comment into iterate_bvecq() that a slot with bv_len>0 must have a
>     valid bv_page.
>   - Fixed iov_iter_bvecq_advance(), iov_iter_bvecq_revert(),
>     iter_count_bvecq_pages(), iov_iter_extract_bvecq_pages() and
>     extract_bvecq_to_sg() to correctly handle empty bvecqs.
>   - Fixed extract_bvecq_to_sg() to be limited by iter->count.
>   - Fixed bvecq_expand_buffer() to take an unsigned size param.
>   - Fixed bvecq_expand_buffer() to not mix memory types in alloc'd bvecqs.
>   - Fixed bvecq_shorten_buffer() occasional retention of zero-length slots.
>   - Fixed slot validity check polarity in bvecq_pos_advance(); also don't use
>     inner loop otherwise break then exits the wrong loop.
>   - Fixed bvecq_zero(), bvecq_slice() and bvecq_extract to use a barrier when
>     checking bq->nr_slots.
>   - Restructured bvecq_zero() to be similar to bvecq_pos_advance().
>   - Fixed an off-by-one error in bvecq_pos_step() and added a missing slot
>     reset.
>   - Fixed a break in netfs_extract_iter() that should have been a goto.
>   - Fixed netfs_extract_iter() to limit number of pages extracted to remnant
>     of max_pages.
>   - Fixed an uninit var in afs_do_read_symlink().
>   - Fixed netfs_read_gaps() to fill a multipart bvecq chain correctly.
>   - Fixed netfs_dispatch_unbuffered_reads() to initialise collect_cursor as
>     netfs_rreq_assess_dio() uses it to flush the data read.
>   - Fixed netfs_extract_iter() to init the slot counter outside the extract
>     loop to avoid overwriting already loaded slots.
>   - Fixed callers of bvecq_delete_spent() to update bvecq_pos::slot before
>     calling.
>   - Fixed netfs_reissue_write() to make sure subreq->content is unset before
>     setting.
>   - Altered netfs_extract_iter() to free any allocated bvecq chain if no pages
>     were extracted and an error occurred  (and to initialise the return
>     pointer to NULL).  Also, made it return an empty bvecq if nothing was
>     extracted, but no error occurred.
>   - Fixed ceph_netfs_issue_read() to just return if
>     ceph_netfs_issue_op_inline() returns anything other than 1 to avoid a
>     double termination.
>   - Fixed ceph_netfs_issue_read() to do the size calculation in the right
>     order to avoid the op expanding to larger than the buffer.
>   - Fixed netfs_issue_read(), in the NETFS_FILL_WITH_ZEROES case, to deduct
>     subreq->len from stream->buffered rather than just setting it to 0.
>   - Fixed netfs_perform_write() to put the folio if netfs_advance_writethrough()
>     fails.
>   - Restored the old ->prepare_write op specifically for
>     fscache_write_to_cache() which is still used by Ceph.
>   - Fixed undefined return in netfs_pgpriv2_issue_stream().
>   - Fixed netfs_collect_write_results() to try to make sure a request isn't
>     left paused if there are no further server-bound subreqs.
>   - Fixed netfs_queue_wb_folio() to redirty the folio before unlocking it if
>     it can't allocate a bvecq.
>   - Fixed netfs_writepages() to cancel the pagecache iteration after ENOMEM.
>   - Fixed netfs_advance/end_writethrough() to advance the dispatch cursor.
>   - Fixed netfs_retry_read_subrequests() to use barriers when walking
>     stream->subrequests as the app may add another subreq before pausing.
>   - Fixed netfs_prepare_write_retry_buffer() to use ->retry_start and
>     ->retry_buffered rather than ->issue_from and ->buffered.
>   - Fixed netfs_retry_write_stream() to use barriers when walking
>     stream->subrequests as the app may add another subreq before pausing.
>   - Fixed netfs_retry_write_stream() to check the correct length when adding
>     additional subreqs.
>   - Fixed nfs_netfs_issue_read() to set -ENOMEM, not 0, on alloc failure.
>   - Fixed nfs_netfs_issue_read() to only terminate the subreq once.
>   - Fixed cifs_issue_read() to release the credits if cifs_reopen_file()
>     fails.
> - Rebased on v7.1.
> 
> ver #3)
> - Rebased to -rc7 as the patches wouldn't apply for Christian.
> - Prepended a fix for a warning from generic/464 (the problem also exists
>   upstream, just not the warning).
> - Renamed kmap_local_bvec() to bvec_kmap_partial() as requested by
>   Christoph.
> - Adjusted smbdirect patch descriptions as requested by Stefan Metzmacher.
> 
> ver #2)
> - Fixed a number of bugs reported by Sashiko[1].
> - Split a bunch of fixes out and posted them separately[2].
> 
> [1] https://sashiko.dev/#/patchset/20260326104544.509518-1-dhowells%40redhat.com
> [2] https://lore.kernel.org/linux-fsdevel/20260512-infozentrum-becher-7f86c47c96c8@brauner/T/#t
> [3] https://sashiko.dev/#/patchset/20260608145432.681865-1-dhowells%40redhat.com
> [4] https://sashiko.dev/#/patchset/20260706153408.1231650-1-dhowells%40redhat.com
> [5] https://sashiko.dev/#/patchset/20260716103030.3065561-1-dhowells%40redhat.com
> [6] https://sashiko.dev/#/patchset/20260722130218.78958-1-dhowells%40redhat.com
> [7] https://sashiko.dev/#/patchset/20260804100224.2748935-1-dhowells%40redhat.com
> 
> David Howells (25):
>   netfs: Fix read progress reporting
>   mm: Make readahead store folio count in readahead_control
>   netfs: Bulk load the readahead-provided folios up front
>   Add a function to kmap one page of a multipage bio_vec
>   iov_iter: Make iov_iter_get_pages*() wrap iov_iter_extract_pages()
>   iov_iter: Add a segmented queue of bio_vec[]
>   netfs: Add some tools for managing bvecq chains
>   netfs: Make mempool available for bvecq
>   netfs: Add a function to extract from an iter into a bvecq
>   afs: Use a bvecq to hold dir content rather than folioq
>   cifs: Use a bvecq for buffering instead of a folioq
>   smbdirect: Support ITER_BVECQ in smbdirect_map_sges_from_iter()
>   netfs: Remove the writethrough code
>   cachefiles: Don't rely on backing fs storage map for most use cases
>   netfs: Add the cache object ID to netfs_read/write tracepoints
>   netfs: Switch to using bvecq rather than folio_queue and
>     rolling_buffer
>   smbdirect: Remove support for ITER_FOLIOQ from
>     smbdirect_map_sges_from_iter()
>   netfs: Remove netfs_alloc/free_folioq_buffer()
>   netfs: Remove netfs_extract_user_iter()
>   iov_iter: Remove ITER_FOLIOQ
>   netfs: Remove folio_queue and rolling_buffer
>   netfs: Simplify read abandonment
>   netfs: Check for too much data being read
>   netfs: Combine prepare and issue ops and grab the buffers on request
>   cachefiles: Preset the state xattr when creating a new file
> 
> Gao Xiang (1):
>   cachefiles,netfs: sunset ondemand mode
> 
>  Documentation/core-api/folio_queue.rst        | 209 ----
>  Documentation/core-api/index.rst              |   1 -
>  .../filesystems/caching/cachefiles.rst        | 179 ----
>  Documentation/filesystems/netfs_library.rst   |   2 +-
>  fs/9p/vfs_addr.c                              |  61 +-
>  fs/afs/dir.c                                  |  40 +-
>  fs/afs/dir_edit.c                             |  43 +-
>  fs/afs/dir_search.c                           |  33 +-
>  fs/afs/file.c                                 |  31 +-
>  fs/afs/fsclient.c                             |   8 +-
>  fs/afs/inode.c                                |   2 +-
>  fs/afs/internal.h                             |  14 +-
>  fs/afs/symlink.c                              |  37 +-
>  fs/afs/write.c                                |  36 +-
>  fs/afs/yfsclient.c                            |   6 +-
>  fs/cachefiles/Kconfig                         |  12 -
>  fs/cachefiles/Makefile                        |   1 -
>  fs/cachefiles/daemon.c                        |  96 +-
>  fs/cachefiles/interface.c                     |  98 +-
>  fs/cachefiles/internal.h                      | 147 +--
>  fs/cachefiles/io.c                            | 602 ++++++++---
>  fs/cachefiles/namei.c                         |  42 +-
>  fs/cachefiles/ondemand.c                      | 761 --------------
>  fs/cachefiles/xattr.c                         |  84 +-
>  fs/ceph/Kconfig                               |   1 +
>  fs/ceph/addr.c                                | 123 ++-
>  fs/netfs/Kconfig                              |   3 +
>  fs/netfs/Makefile                             |   4 +-
>  fs/netfs/buffered_read.c                      | 526 ++++++----
>  fs/netfs/buffered_write.c                     |  59 +-
>  fs/netfs/bvecq.c                              | 808 +++++++++++++++
>  fs/netfs/direct_read.c                        | 115 +--
>  fs/netfs/direct_write.c                       | 162 +--
>  fs/netfs/fscache_io.c                         |   4 +-
>  fs/netfs/internal.h                           | 109 +-
>  fs/netfs/iterator.c                           | 394 +++-----
>  fs/netfs/main.c                               |  13 +-
>  fs/netfs/misc.c                               | 168 +---
>  fs/netfs/objects.c                            |  57 +-
>  fs/netfs/read_collect.c                       | 377 ++++---
>  fs/netfs/read_pgpriv2.c                       | 204 ++--
>  fs/netfs/read_retry.c                         | 261 ++---
>  fs/netfs/read_single.c                        | 184 ++--
>  fs/netfs/rolling_buffer.c                     | 226 -----
>  fs/netfs/stats.c                              |  10 +-
>  fs/netfs/write_collect.c                      | 245 +++--
>  fs/netfs/write_issue.c                        | 935 ++++++++----------
>  fs/netfs/write_retry.c                        | 187 ++--
>  fs/nfs/Kconfig                                |   1 +
>  fs/nfs/fscache.c                              |  24 +-
>  fs/smb/client/cifsglob.h                      |   2 +-
>  fs/smb/client/cifssmb.c                       |  13 +-
>  fs/smb/client/file.c                          | 145 +--
>  fs/smb/client/smb2ops.c                       |  84 +-
>  fs/smb/client/smb2pdu.c                       |  28 +-
>  fs/smb/client/transport.c                     |  15 +-
>  fs/smb/smbdirect/connection.c                 | 135 ++-
>  include/linux/bvec.h                          |  18 +
>  include/linux/bvecq.h                         | 363 +++++++
>  include/linux/folio_queue.h                   | 282 ------
>  include/linux/fscache.h                       |  17 +
>  include/linux/iov_iter.h                      |  87 +-
>  include/linux/netfs.h                         | 188 ++--
>  include/linux/pagemap.h                       |  10 +
>  include/linux/rolling_buffer.h                |  61 --
>  include/linux/uio.h                           |  17 +-
>  include/trace/events/cachefiles.h             | 209 +---
>  include/trace/events/netfs.h                  | 162 ++-
>  include/uapi/linux/cachefiles.h               |  68 --
>  kernel/bpf/btf.c                              |   9 -
>  lib/iov_iter.c                                | 559 ++++++-----
>  lib/scatterlist.c                             |  82 +-
>  lib/tests/kunit_iov_iter.c                    | 131 ++-
>  mm/readahead.c                                |   5 +
>  net/9p/client.c                               |   8 +-
>  75 files changed, 5134 insertions(+), 5309 deletions(-)
>  delete mode 100644 Documentation/core-api/folio_queue.rst
>  delete mode 100644 fs/cachefiles/ondemand.c
>  create mode 100644 fs/netfs/bvecq.c
>  delete mode 100644 fs/netfs/rolling_buffer.c
>  create mode 100644 include/linux/bvecq.h
>  delete mode 100644 include/linux/folio_queue.h
>  delete mode 100644 include/linux/rolling_buffer.h
>  delete mode 100644 include/uapi/linux/cachefiles.h
> 
---end quoted text---

      parent reply	other threads:[~2026-08-10 14:51 UTC|newest]

Thread overview: 30+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-08-10 14:47 [PATCH v9 00/26] netfs: Keep track of folios in a segmented bio_vec[] chain David Howells
2026-08-10 14:47 ` [PATCH v9 01/26] netfs: Fix read progress reporting David Howells
2026-08-10 14:47 ` [PATCH v9 02/26] mm: Make readahead store folio count in readahead_control David Howells
2026-08-10 14:47 ` [PATCH v9 03/26] netfs: Bulk load the readahead-provided folios up front David Howells
2026-08-10 14:47 ` [PATCH v9 04/26] Add a function to kmap one page of a multipage bio_vec David Howells
2026-08-10 19:36   ` Matthew Wilcox
2026-08-10 20:31     ` David Howells
2026-08-10 14:47 ` [PATCH v9 05/26] iov_iter: Make iov_iter_get_pages*() wrap iov_iter_extract_pages() David Howells
2026-08-10 14:47 ` [PATCH v9 06/26] iov_iter: Add a segmented queue of bio_vec[] David Howells
2026-08-10 14:47 ` [PATCH v9 07/26] netfs: Add some tools for managing bvecq chains David Howells
2026-08-10 14:47 ` [PATCH v9 08/26] netfs: Make mempool available for bvecq David Howells
2026-08-10 14:47 ` [PATCH v9 09/26] netfs: Add a function to extract from an iter into a bvecq David Howells
2026-08-10 14:47 ` [PATCH v9 10/26] afs: Use a bvecq to hold dir content rather than folioq David Howells
2026-08-10 14:47 ` [PATCH v9 11/26] cifs: Use a bvecq for buffering instead of a folioq David Howells
2026-08-10 14:47 ` [PATCH v9 12/26] smbdirect: Support ITER_BVECQ in smbdirect_map_sges_from_iter() David Howells
2026-08-10 14:47 ` [PATCH v9 13/26] netfs: Remove the writethrough code David Howells
2026-08-10 14:47 ` [PATCH v9 14/26] cachefiles,netfs: sunset ondemand mode David Howells
2026-08-10 14:47 ` [PATCH v9 15/26] cachefiles: Don't rely on backing fs storage map for most use cases David Howells
2026-08-10 14:47 ` [PATCH v9 16/26] netfs: Add the cache object ID to netfs_read/write tracepoints David Howells
2026-08-10 14:47 ` [PATCH v9 17/26] netfs: Switch to using bvecq rather than folio_queue and rolling_buffer David Howells
2026-08-10 14:47 ` [PATCH v9 18/26] smbdirect: Remove support for ITER_FOLIOQ from smbdirect_map_sges_from_iter() David Howells
2026-08-10 14:47 ` [PATCH v9 19/26] netfs: Remove netfs_alloc/free_folioq_buffer() David Howells
2026-08-10 14:47 ` [PATCH v9 20/26] netfs: Remove netfs_extract_user_iter() David Howells
2026-08-10 14:47 ` [PATCH v9 21/26] iov_iter: Remove ITER_FOLIOQ David Howells
2026-08-10 14:47 ` [PATCH v9 22/26] netfs: Remove folio_queue and rolling_buffer David Howells
2026-08-10 14:47 ` [PATCH v9 23/26] netfs: Simplify read abandonment David Howells
2026-08-10 14:47 ` [PATCH v9 24/26] netfs: Check for too much data being read David Howells
2026-08-10 14:47 ` [PATCH v9 25/26] netfs: Combine prepare and issue ops and grab the buffers on request David Howells
2026-08-10 14:47 ` [PATCH v9 26/26] cachefiles: Preset the state xattr when creating a new file David Howells
2026-08-10 14:51 ` Christoph Hellwig [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=annlWwsCPjNgcKa-@infradead.org \
    --to=hch@infradead.org \
    --cc=asmadeus@codewreck.org \
    --cc=axboe@kernel.dk \
    --cc=ceph-devel@vger.kernel.org \
    --cc=chenxiaosong@chenxiaosong.com \
    --cc=christian@brauner.io \
    --cc=dhowells@redhat.com \
    --cc=ericvh@kernel.org \
    --cc=idryomov@gmail.com \
    --cc=leon@kernel.org \
    --cc=linux-afs@lists.infradead.org \
    --cc=linux-cifs@vger.kernel.org \
    --cc=linux-erofs@lists.ozlabs.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-nfs@vger.kernel.org \
    --cc=marc.dionne@auristor.com \
    --cc=metze@samba.org \
    --cc=netfs@lists.linux.dev \
    --cc=pc@manguebit.org \
    --cc=sfrench@samba.org \
    --cc=v9fs@lists.linux.dev \
    --cc=willy@infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox