From: Jeff Layton <jlayton@kernel.org>
To: David Howells <dhowells@redhat.com>, Steve French <smfrench@gmail.com>
Cc: Matthew Wilcox <willy@infradead.org>,
Marc Dionne <marc.dionne@auristor.com>,
Paulo Alcantara <pc@manguebit.com>,
Shyam Prasad N <sprasad@microsoft.com>,
Tom Talpey <tom@talpey.com>,
Dominique Martinet <asmadeus@codewreck.org>,
Eric Van Hensbergen <ericvh@kernel.org>,
Ilya Dryomov <idryomov@gmail.com>,
Christian Brauner <christian@brauner.io>,
linux-cachefs@redhat.com, linux-afs@lists.infradead.org,
linux-cifs@vger.kernel.org, linux-nfs@vger.kernel.org,
ceph-devel@vger.kernel.org, v9fs@lists.linux.dev,
linux-fsdevel@vger.kernel.org, linux-mm@kvack.org,
netdev@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH v4 12/39] netfs: Add iov_iters to (sub)requests to describe various buffers
Date: Wed, 13 Dec 2023 11:37:10 -0500 [thread overview]
Message-ID: <367107fa03540f7ddd2e8de51c751348bd7eb42c.camel@kernel.org> (raw)
In-Reply-To: <20231213152350.431591-13-dhowells@redhat.com>
On Wed, 2023-12-13 at 15:23 +0000, David Howells wrote:
> Add three iov_iter structs:
>
> (1) Add an iov_iter (->iter) to the I/O request to describe the
> unencrypted-side buffer.
>
> (2) Add an iov_iter (->io_iter) to the I/O request to describe the
> encrypted-side I/O buffer. This may be a different size to the buffer
> in (1).
>
> (3) Add an iov_iter (->io_iter) to the I/O subrequest to describe the part
> of the I/O buffer for that subrequest.
>
> This will allow future patches to point to a bounce buffer instead for
> purposes of handling oversize writes, decryption (where we want to save the
> encrypted data to the cache) and decompression.
>
> These iov_iters persist for the lifetime of the (sub)request, and so can be
> accessed multiple times without worrying about them being deallocated upon
> return to the caller.
>
> The network filesystem must appropriately advance the iterator before
> terminating the request.
>
> Signed-off-by: David Howells <dhowells@redhat.com>
> cc: Jeff Layton <jlayton@kernel.org>
> cc: linux-cachefs@redhat.com
> cc: linux-fsdevel@vger.kernel.org
> cc: linux-mm@kvack.org
> ---
> fs/afs/file.c | 6 +---
> fs/netfs/buffered_read.c | 13 ++++++++
> fs/netfs/io.c | 69 +++++++++++++++++++++++++++++-----------
> include/linux/netfs.h | 3 ++
> 4 files changed, 67 insertions(+), 24 deletions(-)
>
> diff --git a/fs/afs/file.c b/fs/afs/file.c
> index c5013ec3c1dc..aa95b4d6376c 100644
> --- a/fs/afs/file.c
> +++ b/fs/afs/file.c
> @@ -320,11 +320,7 @@ static void afs_issue_read(struct netfs_io_subrequest *subreq)
> fsreq->len = subreq->len - subreq->transferred;
> fsreq->key = key_get(subreq->rreq->netfs_priv);
> fsreq->vnode = vnode;
> - fsreq->iter = &fsreq->def_iter;
> -
> - iov_iter_xarray(&fsreq->def_iter, ITER_DEST,
> - &fsreq->vnode->netfs.inode.i_mapping->i_pages,
> - fsreq->pos, fsreq->len);
> + fsreq->iter = &subreq->io_iter;
>
> afs_fetch_data(fsreq->vnode, fsreq);
> afs_put_read(fsreq);
> diff --git a/fs/netfs/buffered_read.c b/fs/netfs/buffered_read.c
> index d39d0ffe75d2..751556faa70b 100644
> --- a/fs/netfs/buffered_read.c
> +++ b/fs/netfs/buffered_read.c
> @@ -199,6 +199,10 @@ void netfs_readahead(struct readahead_control *ractl)
>
> netfs_rreq_expand(rreq, ractl);
>
> + /* Set up the output buffer */
> + iov_iter_xarray(&rreq->iter, ITER_DEST, &ractl->mapping->i_pages,
> + rreq->start, rreq->len);
> +
> /* Drop the refs on the folios here rather than in the cache or
> * filesystem. The locks will be dropped in netfs_rreq_unlock().
> */
> @@ -251,6 +255,11 @@ int netfs_read_folio(struct file *file, struct folio *folio)
>
> netfs_stat(&netfs_n_rh_readpage);
> trace_netfs_read(rreq, rreq->start, rreq->len, netfs_read_trace_readpage);
> +
> + /* Set up the output buffer */
> + iov_iter_xarray(&rreq->iter, ITER_DEST, &mapping->i_pages,
> + rreq->start, rreq->len);
> +
> return netfs_begin_read(rreq, true);
>
> discard:
> @@ -408,6 +417,10 @@ int netfs_write_begin(struct netfs_inode *ctx,
> ractl._nr_pages = folio_nr_pages(folio);
> netfs_rreq_expand(rreq, &ractl);
>
> + /* Set up the output buffer */
> + iov_iter_xarray(&rreq->iter, ITER_DEST, &mapping->i_pages,
> + rreq->start, rreq->len);
Should the above be ITER_SOURCE ?
> +
> /* We hold the folio locks, so we can drop the references */
> folio_get(folio);
> while (readahead_folio(&ractl))
> diff --git a/fs/netfs/io.c b/fs/netfs/io.c
> index 7f753380e047..e9d408e211b8 100644
> --- a/fs/netfs/io.c
> +++ b/fs/netfs/io.c
> @@ -21,12 +21,7 @@
> */
> static void netfs_clear_unread(struct netfs_io_subrequest *subreq)
> {
> - struct iov_iter iter;
> -
> - iov_iter_xarray(&iter, ITER_DEST, &subreq->rreq->mapping->i_pages,
> - subreq->start + subreq->transferred,
> - subreq->len - subreq->transferred);
> - iov_iter_zero(iov_iter_count(&iter), &iter);
> + iov_iter_zero(iov_iter_count(&subreq->io_iter), &subreq->io_iter);
> }
>
> static void netfs_cache_read_terminated(void *priv, ssize_t transferred_or_error,
> @@ -46,14 +41,9 @@ static void netfs_read_from_cache(struct netfs_io_request *rreq,
> enum netfs_read_from_hole read_hole)
> {
> struct netfs_cache_resources *cres = &rreq->cache_resources;
> - struct iov_iter iter;
>
> netfs_stat(&netfs_n_rh_read);
> - iov_iter_xarray(&iter, ITER_DEST, &rreq->mapping->i_pages,
> - subreq->start + subreq->transferred,
> - subreq->len - subreq->transferred);
> -
> - cres->ops->read(cres, subreq->start, &iter, read_hole,
> + cres->ops->read(cres, subreq->start, &subreq->io_iter, read_hole,
> netfs_cache_read_terminated, subreq);
> }
>
> @@ -88,6 +78,11 @@ static void netfs_read_from_server(struct netfs_io_request *rreq,
> struct netfs_io_subrequest *subreq)
> {
> netfs_stat(&netfs_n_rh_download);
> + if (iov_iter_count(&subreq->io_iter) != subreq->len - subreq->transferred)
> + pr_warn("R=%08x[%u] ITER PRE-MISMATCH %zx != %zx-%zx %lx\n",
> + rreq->debug_id, subreq->debug_index,
> + iov_iter_count(&subreq->io_iter), subreq->len,
> + subreq->transferred, subreq->flags);
pr_warn is a bit alarmist, esp given the cryptic message. Maybe demote
this to INFO or DEBUG?
Does this indicate a bug in the client or that the server is sending us
malformed frames?
> rreq->netfs_ops->issue_read(subreq);
> }
>
> @@ -259,6 +254,30 @@ static void netfs_rreq_short_read(struct netfs_io_request *rreq,
> netfs_read_from_server(rreq, subreq);
> }
>
> +/*
> + * Reset the subrequest iterator prior to resubmission.
> + */
> +static void netfs_reset_subreq_iter(struct netfs_io_request *rreq,
> + struct netfs_io_subrequest *subreq)
> +{
> + size_t remaining = subreq->len - subreq->transferred;
> + size_t count = iov_iter_count(&subreq->io_iter);
> +
> + if (count == remaining)
> + return;
> +
> + _debug("R=%08x[%u] ITER RESUB-MISMATCH %zx != %zx-%zx-%llx %x\n",
> + rreq->debug_id, subreq->debug_index,
> + iov_iter_count(&subreq->io_iter), subreq->transferred,
> + subreq->len, rreq->i_size,
> + subreq->io_iter.iter_type);
> +
> + if (count < remaining)
> + iov_iter_revert(&subreq->io_iter, remaining - count);
> + else
> + iov_iter_advance(&subreq->io_iter, count - remaining);
> +}
> +
> /*
> * Resubmit any short or failed operations. Returns true if we got the rreq
> * ref back.
> @@ -287,6 +306,7 @@ static bool netfs_rreq_perform_resubmissions(struct netfs_io_request *rreq)
> trace_netfs_sreq(subreq, netfs_sreq_trace_download_instead);
> netfs_get_subrequest(subreq, netfs_sreq_trace_get_resubmit);
> atomic_inc(&rreq->nr_outstanding);
> + netfs_reset_subreq_iter(rreq, subreq);
> netfs_read_from_server(rreq, subreq);
> } else if (test_bit(NETFS_SREQ_SHORT_IO, &subreq->flags)) {
> netfs_rreq_short_read(rreq, subreq);
> @@ -399,9 +419,9 @@ void netfs_subreq_terminated(struct netfs_io_subrequest *subreq,
> struct netfs_io_request *rreq = subreq->rreq;
> int u;
>
> - _enter("[%u]{%llx,%lx},%zd",
> - subreq->debug_index, subreq->start, subreq->flags,
> - transferred_or_error);
> + _enter("R=%x[%x]{%llx,%lx},%zd",
> + rreq->debug_id, subreq->debug_index,
> + subreq->start, subreq->flags, transferred_or_error);
>
> switch (subreq->source) {
> case NETFS_READ_FROM_CACHE:
> @@ -501,7 +521,8 @@ static enum netfs_io_source netfs_cache_prepare_read(struct netfs_io_subrequest
> */
> static enum netfs_io_source
> netfs_rreq_prepare_read(struct netfs_io_request *rreq,
> - struct netfs_io_subrequest *subreq)
> + struct netfs_io_subrequest *subreq,
> + struct iov_iter *io_iter)
> {
> enum netfs_io_source source;
>
> @@ -528,9 +549,14 @@ netfs_rreq_prepare_read(struct netfs_io_request *rreq,
> }
> }
>
> - if (WARN_ON(subreq->len == 0))
> + if (WARN_ON(subreq->len == 0)) {
> source = NETFS_INVALID_READ;
> + goto out;
> + }
>
> + subreq->io_iter = *io_iter;
> + iov_iter_truncate(&subreq->io_iter, subreq->len);
> + iov_iter_advance(io_iter, subreq->len);
> out:
> subreq->source = source;
> trace_netfs_sreq(subreq, netfs_sreq_trace_prepare);
> @@ -541,6 +567,7 @@ netfs_rreq_prepare_read(struct netfs_io_request *rreq,
> * Slice off a piece of a read request and submit an I/O request for it.
> */
> static bool netfs_rreq_submit_slice(struct netfs_io_request *rreq,
> + struct iov_iter *io_iter,
> unsigned int *_debug_index)
> {
> struct netfs_io_subrequest *subreq;
> @@ -565,7 +592,7 @@ static bool netfs_rreq_submit_slice(struct netfs_io_request *rreq,
> * (the starts must coincide), in which case, we go around the loop
> * again and ask it to download the next piece.
> */
> - source = netfs_rreq_prepare_read(rreq, subreq);
> + source = netfs_rreq_prepare_read(rreq, subreq, io_iter);
> if (source == NETFS_INVALID_READ)
> goto subreq_failed;
>
> @@ -603,6 +630,7 @@ static bool netfs_rreq_submit_slice(struct netfs_io_request *rreq,
> */
> int netfs_begin_read(struct netfs_io_request *rreq, bool sync)
> {
> + struct iov_iter io_iter;
> unsigned int debug_index = 0;
> int ret;
>
> @@ -615,6 +643,8 @@ int netfs_begin_read(struct netfs_io_request *rreq, bool sync)
> return -EIO;
> }
>
> + rreq->io_iter = rreq->iter;
> +
> INIT_WORK(&rreq->work, netfs_rreq_work);
>
> if (sync)
> @@ -624,8 +654,9 @@ int netfs_begin_read(struct netfs_io_request *rreq, bool sync)
> * want and submit each one.
> */
> atomic_set(&rreq->nr_outstanding, 1);
> + io_iter = rreq->io_iter;
> do {
> - if (!netfs_rreq_submit_slice(rreq, &debug_index))
> + if (!netfs_rreq_submit_slice(rreq, &io_iter, &debug_index))
> break;
>
> } while (rreq->submitted < rreq->len);
> diff --git a/include/linux/netfs.h b/include/linux/netfs.h
> index fc6d9756a029..3da962e977f5 100644
> --- a/include/linux/netfs.h
> +++ b/include/linux/netfs.h
> @@ -150,6 +150,7 @@ struct netfs_cache_resources {
> struct netfs_io_subrequest {
> struct netfs_io_request *rreq; /* Supervising I/O request */
> struct list_head rreq_link; /* Link in rreq->subrequests */
> + struct iov_iter io_iter; /* Iterator for this subrequest */
> loff_t start; /* Where to start the I/O */
> size_t len; /* Size of the I/O */
> size_t transferred; /* Amount of data transferred */
> @@ -186,6 +187,8 @@ struct netfs_io_request {
> struct netfs_cache_resources cache_resources;
> struct list_head proc_link; /* Link in netfs_iorequests */
> struct list_head subrequests; /* Contributory I/O operations */
> + struct iov_iter iter; /* Unencrypted-side iterator */
> + struct iov_iter io_iter; /* I/O (Encrypted-side) iterator */
> void *netfs_priv; /* Private data for the netfs */
> unsigned int debug_id;
> atomic_t nr_outstanding; /* Number of ops in progress */
>
--
Jeff Layton <jlayton@kernel.org>
next prev parent reply other threads:[~2023-12-13 16:37 UTC|newest]
Thread overview: 64+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-12-13 15:23 [PATCH v4 00/39] netfs, afs, 9p: Delegate high-level I/O to netfslib David Howells
2023-12-13 15:23 ` [PATCH v4 01/39] netfs, fscache: Move fs/fscache/* into fs/netfs/ David Howells
2023-12-13 15:23 ` [PATCH v4 02/39] netfs, fscache: Combine fscache with netfs David Howells
2023-12-13 15:23 ` [PATCH v4 03/39] netfs, fscache: Remove ->begin_cache_operation David Howells
2023-12-13 15:23 ` [PATCH v4 04/39] netfs, fscache: Move /proc/fs/fscache to /proc/fs/netfs and put in a symlink David Howells
2023-12-13 15:23 ` [PATCH v4 05/39] netfs: Move pinning-for-writeback from fscache to netfs David Howells
2023-12-13 15:23 ` [PATCH v4 06/39] netfs: Add a procfile to list in-progress requests David Howells
2023-12-13 15:59 ` Jeff Layton
2023-12-13 15:23 ` [PATCH v4 07/39] netfs: Allow the netfs to make the io (sub)request alloc larger David Howells
2023-12-13 15:23 ` [PATCH v4 08/39] netfs: Add a ->free_subrequest() op David Howells
2023-12-13 15:23 ` [PATCH v4 09/39] afs: Don't use folio->private to record partial modification David Howells
2023-12-13 15:23 ` [PATCH v4 10/39] netfs: Provide invalidate_folio and release_folio calls David Howells
2023-12-13 16:05 ` Jeff Layton
2023-12-13 15:23 ` [PATCH v4 11/39] netfs: Implement unbuffered/DIO vs buffered I/O locking David Howells
2023-12-13 16:08 ` Jeff Layton
2023-12-13 16:30 ` Jeff Layton
2023-12-13 15:23 ` [PATCH v4 12/39] netfs: Add iov_iters to (sub)requests to describe various buffers David Howells
2023-12-13 16:37 ` Jeff Layton [this message]
2023-12-19 14:31 ` David Howells
2023-12-19 14:40 ` David Howells
2023-12-13 15:23 ` [PATCH v4 13/39] netfs: Add support for DIO buffering David Howells
2023-12-13 15:23 ` [PATCH v4 14/39] netfs: Provide tools to create a buffer in an xarray David Howells
2023-12-13 15:23 ` [PATCH v4 15/39] netfs: Add bounce buffering support David Howells
2023-12-13 15:23 ` [PATCH v4 16/39] netfs: Add func to calculate pagecount/size-limited span of an iterator David Howells
2023-12-13 15:23 ` [PATCH v4 17/39] netfs: Limit subrequest by size or number of segments David Howells
2023-12-13 15:23 ` [PATCH v4 18/39] netfs: Export netfs_put_subrequest() and some tracepoints David Howells
2023-12-13 18:01 ` Jeff Layton
2023-12-19 14:42 ` David Howells
2023-12-19 14:48 ` David Howells
2023-12-13 15:23 ` [PATCH v4 19/39] netfs: Extend the netfs_io_*request structs to handle writes David Howells
2023-12-13 15:23 ` [PATCH v4 20/39] netfs: Add a hook to allow tell the netfs to update its i_size David Howells
2023-12-13 15:23 ` [PATCH v4 21/39] netfs: Make netfs_put_request() handle a NULL pointer David Howells
2023-12-13 15:23 ` [PATCH v4 22/39] netfs: Make the refcounting of netfs_begin_read() easier to use David Howells
2023-12-13 15:23 ` [PATCH v4 23/39] netfs: Prep to use folio->private for write grouping and streaming write David Howells
2023-12-13 15:23 ` [PATCH v4 24/39] netfs: Dispatch write requests to process a writeback slice David Howells
2023-12-13 15:23 ` [PATCH v4 25/39] netfs: Provide func to copy data to pagecache for buffered write David Howells
2023-12-13 15:23 ` [PATCH v4 26/39] netfs: Make netfs_read_folio() handle streaming-write pages David Howells
2023-12-13 15:23 ` [PATCH v4 27/39] netfs: Allocate multipage folios in the writepath David Howells
2023-12-13 15:23 ` [PATCH v4 28/39] netfs: Implement support for unbuffered/DIO read David Howells
2023-12-14 12:43 ` Jeff Layton
2023-12-19 15:46 ` David Howells
2023-12-13 15:23 ` [PATCH v4 29/39] netfs: Implement unbuffered/DIO write support David Howells
2023-12-13 15:23 ` [PATCH v4 30/39] netfs: Implement buffered write API David Howells
2023-12-13 15:23 ` [PATCH v4 31/39] netfs: Allow buffered shared-writeable mmap through netfs_page_mkwrite() David Howells
2023-12-13 15:23 ` [PATCH v4 32/39] netfs: Provide netfs_file_read_iter() David Howells
2023-12-13 15:23 ` [PATCH v4 33/39] netfs, cachefiles: Pass upper bound length to allow expansion David Howells
2023-12-13 15:23 ` [PATCH v4 34/39] netfs: Provide a writepages implementation David Howells
2023-12-13 15:23 ` [PATCH v4 35/39] netfs: Provide a launder_folio implementation David Howells
2023-12-13 15:23 ` [PATCH v4 36/39] netfs: Implement a write-through caching option David Howells
2023-12-14 13:49 ` Jeff Layton
2023-12-19 16:51 ` David Howells
2023-12-19 17:19 ` Jeff Layton
2023-12-13 15:23 ` [PATCH v4 37/39] netfs: Optimise away reads above the point at which there can be no data David Howells
2023-12-14 14:07 ` Jeff Layton
2023-12-19 16:56 ` David Howells
2023-12-13 15:23 ` [PATCH v4 38/39] afs: Use the netfs write helpers David Howells
2023-12-13 15:23 ` [PATCH v4 39/39] 9p: Use netfslib read/write_iter David Howells
2023-12-13 15:39 ` Christian Schoenebeck
2023-12-14 14:11 ` [PATCH v4 00/39] netfs, afs, 9p: Delegate high-level I/O to netfslib Jeff Layton
2023-12-15 12:03 ` Christian Brauner
2023-12-15 13:29 ` Dominique Martinet
2023-12-18 11:05 ` Christian Brauner
2023-12-20 10:04 ` David Howells
2023-12-20 13:26 ` Christian Brauner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=367107fa03540f7ddd2e8de51c751348bd7eb42c.camel@kernel.org \
--to=jlayton@kernel.org \
--cc=asmadeus@codewreck.org \
--cc=ceph-devel@vger.kernel.org \
--cc=christian@brauner.io \
--cc=dhowells@redhat.com \
--cc=ericvh@kernel.org \
--cc=idryomov@gmail.com \
--cc=linux-afs@lists.infradead.org \
--cc=linux-cachefs@redhat.com \
--cc=linux-cifs@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-nfs@vger.kernel.org \
--cc=marc.dionne@auristor.com \
--cc=netdev@vger.kernel.org \
--cc=pc@manguebit.com \
--cc=smfrench@gmail.com \
--cc=sprasad@microsoft.com \
--cc=tom@talpey.com \
--cc=v9fs@lists.linux.dev \
--cc=willy@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).