linux-xfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Jeff Layton <jlayton@redhat.com>
To: Andrew Morton <akpm@linux-foundation.org>,
	Al Viro <viro@ZenIV.linux.org.uk>, Jan Kara <jack@suse.cz>,
	tytso@mit.edu, axboe@kernel.dk, mawilcox@microsoft.com,
	ross.zwisler@linux.intel.com, corbet@lwn.net,
	Chris Mason <clm@fb.com>, Josef Bacik <jbacik@fb.com>,
	David Sterba <dsterba@suse.com>,
	"Darrick J . Wong" <darrick.wong@oracle.com>
Cc: linux-fsdevel@vger.kernel.org, linux-mm@kvack.org,
	linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org,
	linux-btrfs@vger.kernel.org, linux-block@vger.kernel.org
Subject: [PATCH v6 11/13] Documentation: flesh out the section in vfs.txt on storing and reporting writeback errors
Date: Mon, 12 Jun 2017 08:23:04 -0400	[thread overview]
Message-ID: <20170612122316.13244-13-jlayton@redhat.com> (raw)
In-Reply-To: <20170612122316.13244-1-jlayton@redhat.com>

I waxed a little loquacious here, but I figured that more detail was
better, and writeback error handling is so hard to get right.

Although I think we'll eventually remove it once the transition is
complete, I've gone ahead and documented the FS_WB_ERRSEQ flag as well.

Cc: Jan Kara <jack@suse.cz>
Signed-off-by: Jeff Layton <jlayton@redhat.com>
---
 Documentation/filesystems/vfs.txt | 50 ++++++++++++++++++++++++++++++++++++---
 1 file changed, 47 insertions(+), 3 deletions(-)

diff --git a/Documentation/filesystems/vfs.txt b/Documentation/filesystems/vfs.txt
index f42b90687d40..c3efdd833a3d 100644
--- a/Documentation/filesystems/vfs.txt
+++ b/Documentation/filesystems/vfs.txt
@@ -576,7 +576,49 @@ should clear PG_Dirty and set PG_Writeback.  It can be actually
 written at any point after PG_Dirty is clear.  Once it is known to be
 safe, PG_Writeback is cleared.
 
-Writeback makes use of a writeback_control structure...
+Writeback makes use of a writeback_control structure to direct the
+operations.  This gives the the writepage and writepages operations some
+information about the nature of and reason for the writeback request,
+and the constraints under which it is being done.  It is also used to
+return information back to the caller about the result of a writepage or
+writepages request.
+
+Handling errors during writeback
+--------------------------------
+Most applications that utilize the pagecache will periodically call
+fsync to ensure that data written has made it to the backing store.
+When there is an error during writeback, expect that error to be
+reported when fsync is called.  After an error has been reported to
+fsync, subsequent fsync calls on the same file descriptor should return
+0, unless further writeback errors have occurred since the previous
+fsync.
+
+Ideally, the kernel would report an error only on file descriptions on
+which writes were done that subsequently failed to be written back.  The
+generic pagecache infrastructure does not track the file descriptions
+that have dirtied each individual page however, so determining which
+file descriptors should get back an error is not possible.
+
+Instead, the generic writeback error tracking infrastructure in the
+kernel settles for reporting errors to fsync on all file descriptions
+that were open at the time that the error occurred.  In a situation with
+multiple writers, all of them will get back an error on a subsequent fsync,
+even if all of the writes done through that particular file descriptor
+succeeded (or even if there were no writes on that file descriptor at all).
+
+Filesystems that wish to use this infrastructure should call
+filemap_set_wb_err to record the error in the address_space when it
+occurs.  Then, at the end of their fsync operation, they should call
+filemap_report_wb_err to ensure that the struct file's error cursor
+has advanced to the correct point in the stream of errors emitted by
+the backing device(s).
+
+Older kernels used a different method for tracking errors, based on flags
+in the address_space. We're currently switching everything over to use
+the infrastructure based on errseq_t values. During the transition,
+filesystem authors will want to also ensure their file_system_type has
+FS_WB_ERRSEQ set in fs_flags to ensure that shared infrastructure is
+aware of the model in use.
 
 struct address_space_operations
 -------------------------------
@@ -804,7 +846,8 @@ struct address_space_operations {
 The File Object
 ===============
 
-A file object represents a file opened by a process.
+A file object represents a file opened by a process. This is also known
+as an "open file description" in POSIX parlance.
 
 
 struct file_operations
@@ -887,7 +930,8 @@ otherwise noted.
 
   release: called when the last reference to an open file is closed
 
-  fsync: called by the fsync(2) system call
+  fsync: called by the fsync(2) system call. Also see the section above
+	 entitled "Handling errors during writeback".
 
   fasync: called by the fcntl(2) system call when asynchronous
 	(non-blocking) mode is enabled for a file
-- 
2.13.0


  parent reply	other threads:[~2017-06-12 12:23 UTC|newest]

Thread overview: 37+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-06-12 12:22 [PATCH v6 00/20] fs: enhanced writeback error reporting with errseq_t (pile #1) Jeff Layton
2017-06-12 12:22 ` [PATCH v6 01/20] mm: fix mapping_set_error call in me_pagecache_dirty Jeff Layton
2017-06-12 12:22 ` [PATCH v6 02/20] buffer: use mapping_set_error instead of setting the flag Jeff Layton
2017-06-12 12:22 ` [PATCH v6 03/20] fs: check for writeback errors after syncing out buffers in generic_file_fsync Jeff Layton
2017-06-12 12:22 ` [PATCH v6 04/20] buffer: set errors in mapping at the time that the error occurs Jeff Layton
2017-06-12 12:22 ` [PATCH v6 05/20] mm: don't TestClearPageError in __filemap_fdatawait_range Jeff Layton
2017-06-12 12:22 ` [PATCH v6 06/20] mm: drop "wait" parameter from write_one_page Jeff Layton
2017-06-12 12:22 ` [PATCH v6 07/20] mm: clean up error handling in write_one_page Jeff Layton
2017-06-12 12:23 ` [PATCH v6 08/20] lib: add errseq_t type and infrastructure for handling it Jeff Layton
2017-06-12 12:23 ` [PATCH v6 09/20] fs: new infrastructure for writeback error handling and reporting Jeff Layton
2017-06-12 12:23 ` [PATCH v6 10/13] ext4: add more robust reporting of metadata writeback errors Jeff Layton
2017-06-12 12:23 ` [PATCH v6 10/20] mm: tracepoints for writeback error events Jeff Layton
2017-06-12 12:23 ` Jeff Layton [this message]
2017-06-12 12:23 ` [PATCH v6 11/20] mm: set both AS_EIO/AS_ENOSPC and errseq_t in mapping_set_error Jeff Layton
2017-06-12 12:23 ` [PATCH v6 12/20] fs: add a new fstype flag to indicate how writeback errors are tracked Jeff Layton
2017-06-12 12:45   ` Christoph Hellwig
2017-06-13 10:24     ` Jeff Layton
2017-06-14  6:47       ` Christoph Hellwig
2017-06-14 17:24         ` Jeff Layton
2017-06-15  8:22           ` Christoph Hellwig
2017-06-15 10:42             ` Jeff Layton
2017-06-15 14:57               ` Christoph Hellwig
2017-06-15 15:03                 ` Jeff Layton
2017-06-12 12:23 ` [PATCH v6 12/13] xfs: minimal conversion to errseq_t writeback error reporting Jeff Layton
2017-06-12 12:23 ` [PATCH v6 13/13] btrfs: minimal conversion to errseq_t writeback error reporting on fsync Jeff Layton
2017-06-12 12:23 ` [PATCH v6 13/20] Documentation: flesh out the section in vfs.txt on storing and reporting writeback errors Jeff Layton
2017-06-12 12:23 ` [PATCH v6 14/20] dax: set errors in mapping when writeback fails Jeff Layton
2017-06-12 12:44   ` Christoph Hellwig
2017-06-12 12:23 ` [PATCH v6 15/20] fs: have call_fsync call filemap_report_wb_err if FS_WB_ERRSEQ is set Jeff Layton
2017-06-12 12:42   ` Christoph Hellwig
2017-06-12 12:23 ` [PATCH v6 16/20] block: convert to errseq_t based writeback error tracking Jeff Layton
2017-06-12 12:23 ` [PATCH v6 17/20] fs: add f_md_wb_err field to struct file for tracking metadata errors Jeff Layton
2017-06-12 12:23 ` [PATCH v6 18/20] ext4: use errseq_t based error handling for reporting data writeback errors Jeff Layton
2017-06-12 12:23 ` [PATCH v6 19/20] xfs: minimal conversion to errseq_t writeback error reporting Jeff Layton
2017-06-13  4:30   ` Darrick J. Wong
2017-06-13 10:27     ` Jeff Layton
2017-06-12 12:23 ` [PATCH v6 20/20] btrfs: minimal conversion to errseq_t writeback error reporting on fsync Jeff Layton

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170612122316.13244-13-jlayton@redhat.com \
    --to=jlayton@redhat.com \
    --cc=akpm@linux-foundation.org \
    --cc=axboe@kernel.dk \
    --cc=clm@fb.com \
    --cc=corbet@lwn.net \
    --cc=darrick.wong@oracle.com \
    --cc=dsterba@suse.com \
    --cc=jack@suse.cz \
    --cc=jbacik@fb.com \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-btrfs@vger.kernel.org \
    --cc=linux-ext4@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=linux-xfs@vger.kernel.org \
    --cc=mawilcox@microsoft.com \
    --cc=ross.zwisler@linux.intel.com \
    --cc=tytso@mit.edu \
    --cc=viro@ZenIV.linux.org.uk \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).