cluster-devel.redhat.com archive mirror
 help / color / mirror / Atom feed
From: Bob Peterson <rpeterso@redhat.com>
To: cluster-devel.redhat.com
Subject: [Cluster-devel] [PATCH 0/2] gfs2: improvements to recovery and withdraw process (v2)
Date: Wed, 21 Nov 2018 12:52:55 -0600	[thread overview]
Message-ID: <20181121185257.11206-1-rpeterso@redhat.com> (raw)

Hi,

This is a second draft of a two-patch set to fix some of the nasty
journal recovery problems I've found lately.

The original post from 08 November had horribly bad and inaccurate
comments, and Steve Whitehouse and Andreas Gruenbacher pointed out.
This version is hopefully better and more accurately describes what
the patches do and how they work. Also, I fixed a superblock flag
that was improperly declared as a glock flag.

Other than the renamed and re-valued superblock flag, the code
remains unchanged from the previous version. It probably needs a bit
more testing, but it seems to work well.
---
The problems have to do with file system corruption caused when recovery
replays a journal after the resource group blocks have been unlocked
by the recovery process. In other words, when no cluster node takes
responsibility to replay the journal of a withdrawing node, then it
gets replayed later on, after the blocks contents have been changed.

The first patch prevents gfs2 from attempting recovery if the file system
is withdrawn or has journal IO errors. Trying to recover your own journal
from either of these unstable conditions is dangerous and likely to corrupt
the file system.

The second patch is more extensive. When a node withdraws from a file system
it signals all other nodes with the file system mounted to perform recovery
on its journal, since it cannot safely recover its own journal. This is
accomplished by a new non-disk callback glop used exclusively by the
"live" glock, which sets up an lvb in the glock to indicate which
journal(s) need to be recovered.

Regards,

Bob Peterson
---
Bob Peterson (2):
  gfs2: Ignore recovery attempts if gfs2 has io error or is withdrawn
  gfs2: initiate journal recovery as soon as a node withdraws

 fs/gfs2/glock.c    |  5 ++-
 fs/gfs2/glops.c    | 47 +++++++++++++++++++++++
 fs/gfs2/incore.h   |  3 ++
 fs/gfs2/lock_dlm.c | 95 ++++++++++++++++++++++++++++++++++++++++++++++
 fs/gfs2/log.c      | 62 ++++++++++++++++--------------
 fs/gfs2/super.c    |  5 ++-
 fs/gfs2/super.h    |  1 +
 fs/gfs2/util.c     | 84 ++++++++++++++++++++++++++++++++++++++++
 fs/gfs2/util.h     | 13 +++++++
 9 files changed, 282 insertions(+), 33 deletions(-)

-- 
2.19.1



             reply	other threads:[~2018-11-21 18:52 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-11-21 18:52 Bob Peterson [this message]
2018-11-21 18:52 ` [Cluster-devel] [PATCH 1/2] gfs2: Ignore recovery attempts if gfs2 has io error or is withdrawn Bob Peterson
2018-11-29 20:56   ` Andreas Gruenbacher
2018-11-21 18:52 ` [Cluster-devel] [PATCH 2/2] gfs2: initiate journal recovery as soon as a node withdraws Bob Peterson
2018-11-22 10:16 ` [Cluster-devel] [PATCH 0/2] gfs2: improvements to recovery and withdraw process (v2) Steven Whitehouse

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20181121185257.11206-1-rpeterso@redhat.com \
    --to=rpeterso@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).