public inbox for linux-btrfs@vger.kernel.org
 help / color / mirror / Atom feed
From: Josef Bacik <josef@toxicpanda.com>
To: linux-btrfs@vger.kernel.org, kernel-team@fb.com
Subject: [PATCH 0/8] Fix error handling on data bio submission
Date: Thu, 10 Feb 2022 17:44:18 -0500	[thread overview]
Message-ID: <cover.1644532798.git.josef@toxicpanda.com> (raw)

Internally we tried to enable a remediation to automatically re-provision
machines that had checksum errors.  This was based on dmesg scanning, however we
discovered that we were getting transienty csum error messages.  This came down
to getting a transient ENOMEM while trying to lookup checksums while doing a
data read (this was on memory constrained containers).

What we were doing was simply acting like there wasn't a checksum there, which
would print a scary message about missing checksums.  And then we'd do the read,
but because we didn't have a checksum we'd complain about a checksum mismatch.
Neither of these things were actually what was happening, we simply got an EIO
while looking up the checksums.

Fix this by properly returning an error and erroring out the BIO with the
correct error.  This is actually correct, it allows us to skip the IO and also
not erroneously tell the user that their checksums are invalid.

While testing this fix however I uncovered a variety of problems with our error
handling when we submit.  So the first two patches are to fix the main problem I
wanted to fix, and the next 6 are to fix problems that happen when injecting
errors into the checksum lookup path.

With these patches I'm no longer getting csum mismatch errors when I fail to
lookup csums, and I'm also able to survive a xfstests run while randomly
injecting errors into this path.  Thanks,

Josef

Josef Bacik (8):
  btrfs: make search_csum_tree return 0 if we get -EFBIG
  btrfs: handle csum lookup errors properly on reads
  btrfs: check correct bio in finish_compressed_bio_read
  btrfs: remove the bio argument from finish_compressed_bio_read
  btrfs: track compressed bio errors as blk_status_t
  btrfs: do not double complete bio on errors during compressed reads
  btrfs: do not try to repair bio that has no mirror set
  btrfs: do not clean up repair bio if submit fails

 fs/btrfs/compression.c | 50 +++++++++++++++++++++++-------------------
 fs/btrfs/compression.h |  2 +-
 fs/btrfs/extent_io.c   | 25 ++++++++++++++-------
 fs/btrfs/file-item.c   | 15 ++++++++++---
 fs/btrfs/inode.c       | 12 ++++++----
 5 files changed, 65 insertions(+), 39 deletions(-)

-- 
2.26.3


             reply	other threads:[~2022-02-10 22:44 UTC|newest]

Thread overview: 20+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-02-10 22:44 Josef Bacik [this message]
2022-02-10 22:44 ` [PATCH 1/8] btrfs: make search_csum_tree return 0 if we get -EFBIG Josef Bacik
2022-02-11 22:39   ` Boris Burkov
2022-02-15 16:10   ` Johannes Thumshirn
2022-02-10 22:44 ` [PATCH 2/8] btrfs: handle csum lookup errors properly on reads Josef Bacik
2022-02-11 22:28   ` Boris Burkov
2022-02-10 22:44 ` [PATCH 3/8] btrfs: check correct bio in finish_compressed_bio_read Josef Bacik
2022-02-11 22:43   ` Boris Burkov
2022-02-16  8:48   ` Johannes Thumshirn
2022-02-10 22:44 ` [PATCH 4/8] btrfs: remove the bio argument from finish_compressed_bio_read Josef Bacik
2022-02-16  8:50   ` Johannes Thumshirn
2022-02-10 22:44 ` [PATCH 5/8] btrfs: track compressed bio errors as blk_status_t Josef Bacik
2022-02-16  8:53   ` Johannes Thumshirn
2022-02-10 22:44 ` [PATCH 6/8] btrfs: do not double complete bio on errors during compressed reads Josef Bacik
2022-02-11 22:54   ` Boris Burkov
2022-02-14 17:06     ` David Sterba
2022-02-10 22:44 ` [PATCH 7/8] btrfs: do not try to repair bio that has no mirror set Josef Bacik
2022-02-11 22:56   ` Boris Burkov
2022-02-10 22:44 ` [PATCH 8/8] btrfs: do not clean up repair bio if submit fails Josef Bacik
2022-02-11 23:00   ` Boris Burkov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=cover.1644532798.git.josef@toxicpanda.com \
    --to=josef@toxicpanda.com \
    --cc=kernel-team@fb.com \
    --cc=linux-btrfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox