netfs.lists.linux.dev archive mirror
 help / color / mirror / Atom feed
From: David Howells <dhowells@redhat.com>
To: Christian Brauner <christian@brauner.io>,
	Steve French <smfrench@gmail.com>,
	Matthew Wilcox <willy@infradead.org>
Cc: David Howells <dhowells@redhat.com>,
	Jeff Layton <jlayton@kernel.org>,
	Gao Xiang <hsiangkao@linux.alibaba.com>,
	Dominique Martinet <asmadeus@codewreck.org>,
	Marc Dionne <marc.dionne@auristor.com>,
	Paulo Alcantara <pc@manguebit.com>,
	Shyam Prasad N <sprasad@microsoft.com>,
	Tom Talpey <tom@talpey.com>,
	Eric Van Hensbergen <ericvh@kernel.org>,
	Ilya Dryomov <idryomov@gmail.com>,
	netfs@lists.linux.dev, linux-afs@lists.infradead.org,
	linux-cifs@vger.kernel.org, linux-nfs@vger.kernel.org,
	ceph-devel@vger.kernel.org, v9fs@lists.linux.dev,
	linux-erofs@lists.ozlabs.org, linux-fsdevel@vger.kernel.org,
	linux-mm@kvack.org, netdev@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	syzbot+af5c06208fa71bf31b16@syzkaller.appspotmail.com,
	Chang Yu <marcus.yu.56@gmail.com>
Subject: [PATCH v3 33/33] netfs: Report on NULL folioq in netfs_writeback_unlock_folios()
Date: Wed,  6 Nov 2024 12:35:57 +0000	[thread overview]
Message-ID: <20241106123559.724888-34-dhowells@redhat.com> (raw)
In-Reply-To: <20241106123559.724888-1-dhowells@redhat.com>

It seems that it's possible to get to netfs_writeback_unlock_folios() with
an empty rolling buffer during buffered writes.  This should not be
possible as the rolling buffer is initialised as the write request is set
up and thereafter maintains at least one folio_queue struct therein until
it gets destroyed.  This allows lockless addition and removal of
folio_queue structs in the buffer because, unlike with a ring buffer, the
producer and consumer each only need to look at and alter one pointer into
the buffer.

Now, the rolling buffer is only used for buffered I/O operations as
netfs_collect_write_results() should only call
netfs_writeback_unlock_folios() if the request is of origin type
NETFS_WRITEBACK, NETFS_WRITETHROUGH or NETFS_PGPRIV2_COPY_TO_CACHE.

So it would seem that one of the following occurred: (1) I/O started before
the request was fully initialised, (2) the origin got switched mid-flow or
(3) the request has already been freed and this is a UAF error.  I think the
last is the most likely.

Make netfs_writeback_unlock_folios() report information about the request
and subrequests if folioq is seen to be NULL to try and help debug this,
throw a warning and return.

Note that this does not try to fix the problem.

Reported-by: syzbot+af5c06208fa71bf31b16@syzkaller.appspotmail.com
Link: https://syzkaller.appspot.com/bug?extid=af5c06208fa71bf31b16
Signed-off-by: David Howells <dhowells@redhat.com>
cc: Chang Yu <marcus.yu.56@gmail.com>
Link: https://lore.kernel.org/r/ZxshMEW4U7MTgQYa@gmail.com/
cc: Jeff Layton <jlayton@kernel.org>
cc: netfs@lists.linux.dev
cc: linux-fsdevel@vger.kernel.org
---
 fs/netfs/write_collect.c | 34 ++++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)

diff --git a/fs/netfs/write_collect.c b/fs/netfs/write_collect.c
index 3d8b87c8e6a6..4a1499167770 100644
--- a/fs/netfs/write_collect.c
+++ b/fs/netfs/write_collect.c
@@ -21,6 +21,34 @@
 #define NEED_RETRY		0x10	/* A front op requests retrying */
 #define SAW_FAILURE		0x20	/* One stream or hit a permanent failure */
 
+static void netfs_dump_request(const struct netfs_io_request *rreq)
+{
+	pr_err("Request R=%08x r=%d fl=%lx or=%x e=%ld\n",
+	       rreq->debug_id, refcount_read(&rreq->ref), rreq->flags,
+	       rreq->origin, rreq->error);
+	pr_err("  st=%llx tsl=%zx/%llx/%llx\n",
+	       rreq->start, rreq->transferred, rreq->submitted, rreq->len);
+	pr_err("  cci=%llx/%llx/%llx\n",
+	       rreq->cleaned_to, rreq->collected_to, atomic64_read(&rreq->issued_to));
+	pr_err("  iw=%pSR\n", rreq->netfs_ops->issue_write);
+	for (int i = 0; i < NR_IO_STREAMS; i++) {
+		const struct netfs_io_subrequest *sreq;
+		const struct netfs_io_stream *s = &rreq->io_streams[i];
+
+		pr_err("  str[%x] s=%x e=%d acnf=%u,%u,%u,%u\n",
+		       s->stream_nr, s->source, s->error,
+		       s->avail, s->active, s->need_retry, s->failed);
+		pr_err("  str[%x] ct=%llx t=%zx\n",
+		       s->stream_nr, s->collected_to, s->transferred);
+		list_for_each_entry(sreq, &s->subrequests, rreq_link) {
+			pr_err("  sreq[%x:%x] sc=%u s=%llx t=%zx/%zx r=%d f=%lx\n",
+			       sreq->stream_nr, sreq->debug_index, sreq->source,
+			       sreq->start, sreq->transferred, sreq->len,
+			       refcount_read(&sreq->ref), sreq->flags);
+		}
+	}
+}
+
 /*
  * Successful completion of write of a folio to the server and/or cache.  Note
  * that we are not allowed to lock the folio here on pain of deadlocking with
@@ -87,6 +115,12 @@ static void netfs_writeback_unlock_folios(struct netfs_io_request *wreq,
 	unsigned long long collected_to = wreq->collected_to;
 	unsigned int slot = wreq->buffer.first_tail_slot;
 
+	if (WARN_ON_ONCE(!folioq)) {
+		pr_err("[!] Writeback unlock found empty rolling buffer!\n");
+		netfs_dump_request(wreq);
+		return;
+	}
+
 	if (wreq->origin == NETFS_PGPRIV2_COPY_TO_CACHE) {
 		if (netfs_pgpriv2_unlock_copied_folios(wreq))
 			*notes |= MADE_PROGRESS;


  parent reply	other threads:[~2024-11-06 12:40 UTC|newest]

Thread overview: 36+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-11-06 12:35 [PATCH v3 00/33] netfs: Read performance improvements and "single-blob" support David Howells
2024-11-06 12:35 ` [PATCH v3 01/33] kheaders: Ignore silly-rename files David Howells
2024-11-06 12:35 ` [PATCH v3 02/33] netfs: Remove call to folio_index() David Howells
2024-11-06 12:35 ` [PATCH v3 03/33] netfs: Fix a few minor bugs in netfs_page_mkwrite() David Howells
2024-11-06 12:35 ` [PATCH v3 04/33] netfs: Remove unnecessary references to pages David Howells
2024-11-06 12:35 ` [PATCH v3 05/33] netfs: Use a folio_queue allocation and free functions David Howells
2024-11-06 12:35 ` [PATCH v3 06/33] netfs: Add a tracepoint to log the lifespan of folio_queue structs David Howells
2024-11-06 12:35 ` [PATCH v3 07/33] netfs: Abstract out a rolling folio buffer implementation David Howells
2024-11-06 12:35 ` [PATCH v3 08/33] netfs: Make netfs_advance_write() return size_t David Howells
2024-11-06 12:35 ` [PATCH v3 09/33] netfs: Split retry code out of fs/netfs/write_collect.c David Howells
2024-11-06 12:35 ` [PATCH v3 10/33] netfs: Drop the error arg from netfs_read_subreq_terminated() David Howells
2024-11-06 12:35 ` [PATCH v3 11/33] netfs: Drop the was_async " David Howells
2024-11-06 12:35 ` [PATCH v3 12/33] netfs: Don't use bh spinlock David Howells
2024-11-06 12:35 ` [PATCH v3 13/33] afs: Don't use mutex for I/O operation lock David Howells
2024-11-06 12:35 ` [PATCH v3 14/33] afs: Fix EEXIST error returned from afs_rmdir() to be ENOTEMPTY David Howells
2024-11-06 12:35 ` [PATCH v3 15/33] afs: Fix directory format encoding struct David Howells
2024-11-06 12:35 ` [PATCH v3 16/33] netfs: Remove some extraneous directory invalidations David Howells
2024-11-06 12:35 ` [PATCH v3 17/33] cachefiles: Add some subrequest tracepoints David Howells
2024-11-06 12:35 ` [PATCH v3 18/33] cachefiles: Add auxiliary data trace David Howells
2024-11-06 12:35 ` [PATCH v3 19/33] afs: Add more tracepoints to do with tracking validity David Howells
2024-11-06 12:35 ` [PATCH v3 20/33] netfs: Add functions to build/clean a buffer in a folio_queue David Howells
2024-11-06 12:35 ` [PATCH v3 21/33] netfs: Add support for caching single monolithic objects such as AFS dirs David Howells
2024-11-06 12:35 ` [PATCH v3 22/33] afs: Make afs_init_request() get a key if not given a file David Howells
2024-11-06 12:35 ` [PATCH v3 23/33] afs: Use netfslib for directories David Howells
2024-11-06 12:35 ` [PATCH v3 24/33] afs: Use netfslib for symlinks, allowing them to be cached David Howells
2024-11-06 12:35 ` [PATCH v3 25/33] afs: Eliminate afs_read David Howells
2024-11-06 12:35 ` [PATCH v3 26/33] afs: Fix cleanup of immediately failed async calls David Howells
2024-11-06 12:35 ` [PATCH v3 27/33] afs: Make {Y,}FS.FetchData an asynchronous operation David Howells
2024-11-06 12:35 ` [PATCH v3 28/33] netfs: Change the read result collector to only use one work item David Howells
2024-11-06 12:35 ` [PATCH v3 29/33] afs: Make afs_mkdir() locally initialise a new directory's content David Howells
2024-11-06 12:35 ` [PATCH v3 30/33] afs: Use the contained hashtable to search a directory David Howells
2024-11-06 12:35 ` [PATCH v3 31/33] afs: Locally initialise the contents of a new symlink on creation David Howells
2024-11-06 12:35 ` [PATCH v3 32/33] afs: Add a tracepoint for afs_read_receive() David Howells
2024-11-06 12:35 ` David Howells [this message]
2024-11-08 17:03 ` [PATCH v3 28/33] netfs: Change the read result collector to only use one work item David Howells
2024-11-08 17:23 ` [PATCH v3 23/33] afs: Use netfslib for directories David Howells

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20241106123559.724888-34-dhowells@redhat.com \
    --to=dhowells@redhat.com \
    --cc=asmadeus@codewreck.org \
    --cc=ceph-devel@vger.kernel.org \
    --cc=christian@brauner.io \
    --cc=ericvh@kernel.org \
    --cc=hsiangkao@linux.alibaba.com \
    --cc=idryomov@gmail.com \
    --cc=jlayton@kernel.org \
    --cc=linux-afs@lists.infradead.org \
    --cc=linux-cifs@vger.kernel.org \
    --cc=linux-erofs@lists.ozlabs.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=linux-nfs@vger.kernel.org \
    --cc=marc.dionne@auristor.com \
    --cc=marcus.yu.56@gmail.com \
    --cc=netdev@vger.kernel.org \
    --cc=netfs@lists.linux.dev \
    --cc=pc@manguebit.com \
    --cc=smfrench@gmail.com \
    --cc=sprasad@microsoft.com \
    --cc=syzbot+af5c06208fa71bf31b16@syzkaller.appspotmail.com \
    --cc=tom@talpey.com \
    --cc=v9fs@lists.linux.dev \
    --cc=willy@infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).