linux-fsdevel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Jeff Layton <jlayton@redhat.com>
To: Andrew Morton <akpm@linux-foundation.org>,
	Al Viro <viro@ZenIV.linux.org.uk>, Jan Kara <jack@suse.cz>,
	tytso@mit.edu, axboe@kernel.dk, mawilcox@microsoft.com,
	ross.zwisler@linux.intel.com, corbet@lwn.net
Cc: linux-ext4@vger.kernel.org, linux-fsdevel@vger.kernel.org,
	linux-kernel@vger.kernel.org, linux-block@vger.kernel.org,
	linux-doc@vger.kernel.org
Subject: [PATCH v5 05/17] Documentation: flesh out the section in vfs.txt on storing and reporting writeback errors
Date: Wed, 31 May 2017 08:45:28 -0400	[thread overview]
Message-ID: <20170531124540.8782-6-jlayton@redhat.com> (raw)
In-Reply-To: <20170531124540.8782-1-jlayton@redhat.com>

I waxed a little loquacious here, but I figured that more detail was
better, and writeback error handling is so hard to get right.

Although I think we'll eventually remove it once the transition is
complete, I've gone ahead and documented the FS_WB_ERRSEQ flag as well.

Cc: Jan Kara <jack@suse.cz>
Signed-off-by: Jeff Layton <jlayton@redhat.com>
---
 Documentation/filesystems/vfs.txt | 50 ++++++++++++++++++++++++++++++++++++---
 1 file changed, 47 insertions(+), 3 deletions(-)

diff --git a/Documentation/filesystems/vfs.txt b/Documentation/filesystems/vfs.txt
index f42b90687d40..c3efdd833a3d 100644
--- a/Documentation/filesystems/vfs.txt
+++ b/Documentation/filesystems/vfs.txt
@@ -576,7 +576,49 @@ should clear PG_Dirty and set PG_Writeback.  It can be actually
 written at any point after PG_Dirty is clear.  Once it is known to be
 safe, PG_Writeback is cleared.
 
-Writeback makes use of a writeback_control structure...
+Writeback makes use of a writeback_control structure to direct the
+operations.  This gives the the writepage and writepages operations some
+information about the nature of and reason for the writeback request,
+and the constraints under which it is being done.  It is also used to
+return information back to the caller about the result of a writepage or
+writepages request.
+
+Handling errors during writeback
+--------------------------------
+Most applications that utilize the pagecache will periodically call
+fsync to ensure that data written has made it to the backing store.
+When there is an error during writeback, expect that error to be
+reported when fsync is called.  After an error has been reported to
+fsync, subsequent fsync calls on the same file descriptor should return
+0, unless further writeback errors have occurred since the previous
+fsync.
+
+Ideally, the kernel would report an error only on file descriptions on
+which writes were done that subsequently failed to be written back.  The
+generic pagecache infrastructure does not track the file descriptions
+that have dirtied each individual page however, so determining which
+file descriptors should get back an error is not possible.
+
+Instead, the generic writeback error tracking infrastructure in the
+kernel settles for reporting errors to fsync on all file descriptions
+that were open at the time that the error occurred.  In a situation with
+multiple writers, all of them will get back an error on a subsequent fsync,
+even if all of the writes done through that particular file descriptor
+succeeded (or even if there were no writes on that file descriptor at all).
+
+Filesystems that wish to use this infrastructure should call
+filemap_set_wb_err to record the error in the address_space when it
+occurs.  Then, at the end of their fsync operation, they should call
+filemap_report_wb_err to ensure that the struct file's error cursor
+has advanced to the correct point in the stream of errors emitted by
+the backing device(s).
+
+Older kernels used a different method for tracking errors, based on flags
+in the address_space. We're currently switching everything over to use
+the infrastructure based on errseq_t values. During the transition,
+filesystem authors will want to also ensure their file_system_type has
+FS_WB_ERRSEQ set in fs_flags to ensure that shared infrastructure is
+aware of the model in use.
 
 struct address_space_operations
 -------------------------------
@@ -804,7 +846,8 @@ struct address_space_operations {
 The File Object
 ===============
 
-A file object represents a file opened by a process.
+A file object represents a file opened by a process. This is also known
+as an "open file description" in POSIX parlance.
 
 
 struct file_operations
@@ -887,7 +930,8 @@ otherwise noted.
 
   release: called when the last reference to an open file is closed
 
-  fsync: called by the fsync(2) system call
+  fsync: called by the fsync(2) system call. Also see the section above
+	 entitled "Handling errors during writeback".
 
   fasync: called by the fcntl(2) system call when asynchronous
 	(non-blocking) mode is enabled for a file
-- 
2.9.4

  parent reply	other threads:[~2017-05-31 12:45 UTC|newest]

Thread overview: 26+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-05-31 12:45 [PATCH v5 00/17] fs: introduce new writeback error reporting and convert ext2 and ext4 to use it Jeff Layton
2017-05-31 12:45 ` [PATCH v5 01/17] lib: add errseq_t type and infrastructure for handling it Jeff Layton
2017-05-31 12:45 ` [PATCH v5 02/17] fs: new infrastructure for writeback error handling and reporting Jeff Layton
2017-05-31 12:45 ` [PATCH v5 03/17] mm: tracepoints for writeback error events Jeff Layton
2017-05-31 12:45 ` [PATCH v5 04/17] fs: add a new fstype flag to indicate how writeback errors are tracked Jeff Layton
2017-05-31 12:45 ` Jeff Layton [this message]
2017-05-31 12:45 ` [PATCH v5 06/17] fs: adapt sync_file_range to new reporting infrastructure Jeff Layton
2017-05-31 12:45 ` [PATCH v5 07/17] mm: add filemap_fdatawait_range_since and filemap_write_and_wait_range_since Jeff Layton
2017-05-31 12:45 ` [PATCH v5 08/17] dax: set errors in mapping when writeback fails Jeff Layton
2017-06-06  1:01   ` Ross Zwisler
2017-06-06  1:08     ` Jeff Layton
2017-05-31 12:45 ` [PATCH v5 09/17] block: convert to errseq_t based writeback error tracking Jeff Layton
2017-05-31 12:45 ` [PATCH v5 10/17] block: add sync_blockdev_since and sync_filesystem_since Jeff Layton
2017-05-31 12:45 ` [PATCH v5 11/17] fs: add f_md_wb_err field to struct file for tracking metadata errors Jeff Layton
2017-05-31 12:45 ` [PATCH v5 12/17] fs: allow __generic_file_fsync to support both flavors of error reporting Jeff Layton
2017-05-31 12:45 ` [PATCH v5 13/17] jbd2: conditionally handle errors using errseq_t based on FS_WB_ERRSEQ flag Jeff Layton
2017-05-31 12:45 ` [PATCH v5 14/17] ext4: convert to errseq_t based error tracking Jeff Layton
2017-05-31 12:45 ` [PATCH v5 15/17] fs: add a write_one_page_since Jeff Layton
2017-05-31 12:45 ` [PATCH v5 16/17] ext2: convert to errseq_t based writeback error tracking Jeff Layton
2017-05-31 12:45 ` [PATCH v5 17/17] fs: convert ext2 to use write_one_page_since Jeff Layton
2017-05-31 20:27 ` [PATCH v5 00/17] fs: introduce new writeback error reporting and convert ext2 and ext4 to use it Andrew Morton
2017-05-31 21:31   ` Jeff Layton
2017-05-31 21:37     ` Andrew Morton
2017-05-31 22:01       ` Jeff Layton
2017-06-02  5:25 ` Ross Zwisler
2017-06-02 10:07   ` Jeff Layton

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170531124540.8782-6-jlayton@redhat.com \
    --to=jlayton@redhat.com \
    --cc=akpm@linux-foundation.org \
    --cc=axboe@kernel.dk \
    --cc=corbet@lwn.net \
    --cc=jack@suse.cz \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-ext4@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mawilcox@microsoft.com \
    --cc=ross.zwisler@linux.intel.com \
    --cc=tytso@mit.edu \
    --cc=viro@ZenIV.linux.org.uk \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).