[Cluster-devel] GFS2 changes for the 5.7 merge window

[Cluster-devel] GFS2 changes for the 5.7 merge window

[Bob Peterson]

Hi Linus,

Could you please pull the following changes for gfs2?

Thanks,

Bob Peterson

The following changes since commit 6e5e41e2dc4e4413296d5a4af54ac92d7cd52317:

  gfs2: fix O_SYNC write handling (2020-02-06 18:49:41 +0100)

are available in the Git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/gfs2/linux-gfs2.git tags/gfs2-for-5.7

for you to fetch changes up to 75b46c437f6b0f8e37032a407c7373f85f5c26a8:

  gfs2: Fix oversight in gfs2_ail1_flush (2020-03-30 07:55:35 -0500)

----------------------------------------------------------------
We've got a lot of patches (39) for this merge window. Most of these patches
are related to corruption that occurs when journals are replayed.
For example:

   1. A node fails while writing to the file system.
   2. Other nodes use the metadata that was once used by the failed node.
   3. When the node returns to the cluster, its journal is replayed,
      but the older metadata blocks overwrite the changes from step 2.

- Fixed the recovery sequence to prevent corruption during journal replay.
- Many bug fixes found during recovery testing.
- New improved file system withdraw sequence.
- Fixed how resource group buffers are managed.
- Fixed how metadata revokes are tracked and written.
- Improve processing of IO errors hit by daemons like logd and quotad.
- Improved error checking in metadata writes.
- Fixed how qadata quota data structures are managed.

----------------------------------------------------------------
Andreas Gruenbacher (9):
      gfs2: Split gfs2_lm_withdraw into two functions
      gfs2: Report errors before withdraw
      gfs2: Remove usused cluster_wide arguments of gfs2_consist functions
      gfs2: Turn gfs2_consist into void functions
      gfs2: Return bool from gfs2_assert functions
      gfs2: Clean up inode initialization and teardown
      gfs2: Switch to list_{first,last}_entry
      gfs2: Split gfs2_rsqa_delete into gfs2_rs_delete and gfs2_qa_put
      gfs2: Remove unnecessary gfs2_qa_{get,put} pairs

Bob Peterson (30):
      gfs2: Introduce concept of a pending withdraw
      gfs2: clear ail1 list when gfs2 withdraws
      gfs2: Rework how rgrp buffer_heads are managed
      gfs2: log error reform
      gfs2: Only complain the first time an io error occurs in quota or log
      gfs2: Ignore dlm recovery requests if gfs2 is withdrawn
      gfs2: move check_journal_clean to util.c for future use
      gfs2: Allow some glocks to be used during withdraw
      gfs2: Force withdraw to replay journals and wait for it to finish
      gfs2: fix infinite loop when checking ail item count before go_inval
      gfs2: Add verbose option to check_journal_clean
      gfs2: Issue revokes more intelligently
      gfs2: Prepare to withdraw as soon as an IO error occurs in log write
      gfs2: Check for log write errors before telling dlm to unlock
      gfs2: Do log_flush in gfs2_ail_empty_gl even if ail list is empty
      gfs2: Withdraw in gfs2_ail1_flush if write_cache_pages fails
      gfs2: drain the ail2 list after io errors
      gfs2: Don't demote a glock until its revokes are written
      gfs2: Do proper error checking for go_sync family of glops functions
      gfs2: flesh out delayed withdraw for gfs2_log_flush
      gfs2: don't allow releasepage to free bd still used for revokes
      gfs2: allow journal replay to hold sd_log_flush_lock
      gfs2: leaf_dealloc needs to allocate one more revoke
      gfs2: Additional information when gfs2_ail1_flush withdraws
      gfs2: eliminate gfs2_rsqa_alloc in favor of gfs2_qa_alloc
      gfs2: Change inode qa_data to allow multiple users
      gfs2: don't lock sd_log_flush_lock in try_rgrp_unlink
      gfs2: instrumentation wrt ail1 stuck
      gfs2: change from write to read lock for sd_log_flush_lock in journal replay
      gfs2: Fix oversight in gfs2_ail1_flush

 fs/gfs2/acl.c        |   7 +-
 fs/gfs2/aops.c       |  11 +-
 fs/gfs2/bmap.c       |   9 +-
 fs/gfs2/dir.c        |   3 +-
 fs/gfs2/file.c       |  43 +++---
 fs/gfs2/glock.c      | 137 ++++++++++++++---
 fs/gfs2/glops.c      | 157 +++++++++++++++----
 fs/gfs2/incore.h     |  27 +++-
 fs/gfs2/inode.c      |  53 +++----
 fs/gfs2/lock_dlm.c   |  52 +++++++
 fs/gfs2/log.c        | 288 ++++++++++++++++++++++++-----------
 fs/gfs2/log.h        |   1 +
 fs/gfs2/lops.c       |  14 +-
 fs/gfs2/meta_io.c    |   3 +-
 fs/gfs2/ops_fstype.c |  59 ++------
 fs/gfs2/quota.c      |  76 ++++++----
 fs/gfs2/quota.h      |   4 +-
 fs/gfs2/recovery.c   |  12 +-
 fs/gfs2/rgrp.c       |  88 ++++-------
 fs/gfs2/rgrp.h       |   4 +-
 fs/gfs2/super.c      | 112 +++++++++-----
 fs/gfs2/super.h      |   1 -
 fs/gfs2/sys.c        |   5 +-
 fs/gfs2/trans.c      |   4 +
 fs/gfs2/util.c       | 419 ++++++++++++++++++++++++++++++++++++++++-----------
 fs/gfs2/util.h       |  76 +++++++---
 fs/gfs2/xattr.c      |  12 +-
 27 files changed, 1168 insertions(+), 509 deletions(-)

GFS2 changes for the 5.7 merge window

[Bob Peterson]

Hi Linus,

Could you please pull the following changes for gfs2?

Thanks,

Bob Peterson

The following changes since commit 6e5e41e2dc4e4413296d5a4af54ac92d7cd52317:

  gfs2: fix O_SYNC write handling (2020-02-06 18:49:41 +0100)

are available in the Git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/gfs2/linux-gfs2.git tags/gfs2-for-5.7

for you to fetch changes up to 75b46c437f6b0f8e37032a407c7373f85f5c26a8:

  gfs2: Fix oversight in gfs2_ail1_flush (2020-03-30 07:55:35 -0500)

----------------------------------------------------------------
We've got a lot of patches (39) for this merge window. Most of these patches
are related to corruption that occurs when journals are replayed.
For example:

   1. A node fails while writing to the file system.
   2. Other nodes use the metadata that was once used by the failed node.
   3. When the node returns to the cluster, its journal is replayed,
      but the older metadata blocks overwrite the changes from step 2.

- Fixed the recovery sequence to prevent corruption during journal replay.
- Many bug fixes found during recovery testing.
- New improved file system withdraw sequence.
- Fixed how resource group buffers are managed.
- Fixed how metadata revokes are tracked and written.
- Improve processing of IO errors hit by daemons like logd and quotad.
- Improved error checking in metadata writes.
- Fixed how qadata quota data structures are managed.

----------------------------------------------------------------
Andreas Gruenbacher (9):
      gfs2: Split gfs2_lm_withdraw into two functions
      gfs2: Report errors before withdraw
      gfs2: Remove usused cluster_wide arguments of gfs2_consist functions
      gfs2: Turn gfs2_consist into void functions
      gfs2: Return bool from gfs2_assert functions
      gfs2: Clean up inode initialization and teardown
      gfs2: Switch to list_{first,last}_entry
      gfs2: Split gfs2_rsqa_delete into gfs2_rs_delete and gfs2_qa_put
      gfs2: Remove unnecessary gfs2_qa_{get,put} pairs

Bob Peterson (30):
      gfs2: Introduce concept of a pending withdraw
      gfs2: clear ail1 list when gfs2 withdraws
      gfs2: Rework how rgrp buffer_heads are managed
      gfs2: log error reform
      gfs2: Only complain the first time an io error occurs in quota or log
      gfs2: Ignore dlm recovery requests if gfs2 is withdrawn
      gfs2: move check_journal_clean to util.c for future use
      gfs2: Allow some glocks to be used during withdraw
      gfs2: Force withdraw to replay journals and wait for it to finish
      gfs2: fix infinite loop when checking ail item count before go_inval
      gfs2: Add verbose option to check_journal_clean
      gfs2: Issue revokes more intelligently
      gfs2: Prepare to withdraw as soon as an IO error occurs in log write
      gfs2: Check for log write errors before telling dlm to unlock
      gfs2: Do log_flush in gfs2_ail_empty_gl even if ail list is empty
      gfs2: Withdraw in gfs2_ail1_flush if write_cache_pages fails
      gfs2: drain the ail2 list after io errors
      gfs2: Don't demote a glock until its revokes are written
      gfs2: Do proper error checking for go_sync family of glops functions
      gfs2: flesh out delayed withdraw for gfs2_log_flush
      gfs2: don't allow releasepage to free bd still used for revokes
      gfs2: allow journal replay to hold sd_log_flush_lock
      gfs2: leaf_dealloc needs to allocate one more revoke
      gfs2: Additional information when gfs2_ail1_flush withdraws
      gfs2: eliminate gfs2_rsqa_alloc in favor of gfs2_qa_alloc
      gfs2: Change inode qa_data to allow multiple users
      gfs2: don't lock sd_log_flush_lock in try_rgrp_unlink
      gfs2: instrumentation wrt ail1 stuck
      gfs2: change from write to read lock for sd_log_flush_lock in journal replay
      gfs2: Fix oversight in gfs2_ail1_flush

 fs/gfs2/acl.c        |   7 +-
 fs/gfs2/aops.c       |  11 +-
 fs/gfs2/bmap.c       |   9 +-
 fs/gfs2/dir.c        |   3 +-
 fs/gfs2/file.c       |  43 +++---
 fs/gfs2/glock.c      | 137 ++++++++++++++---
 fs/gfs2/glops.c      | 157 +++++++++++++++----
 fs/gfs2/incore.h     |  27 +++-
 fs/gfs2/inode.c      |  53 +++----
 fs/gfs2/lock_dlm.c   |  52 +++++++
 fs/gfs2/log.c        | 288 ++++++++++++++++++++++++-----------
 fs/gfs2/log.h        |   1 +
 fs/gfs2/lops.c       |  14 +-
 fs/gfs2/meta_io.c    |   3 +-
 fs/gfs2/ops_fstype.c |  59 ++------
 fs/gfs2/quota.c      |  76 ++++++----
 fs/gfs2/quota.h      |   4 +-
 fs/gfs2/recovery.c   |  12 +-
 fs/gfs2/rgrp.c       |  88 ++++-------
 fs/gfs2/rgrp.h       |   4 +-
 fs/gfs2/super.c      | 112 +++++++++-----
 fs/gfs2/super.h      |   1 -
 fs/gfs2/sys.c        |   5 +-
 fs/gfs2/trans.c      |   4 +
 fs/gfs2/util.c       | 419 ++++++++++++++++++++++++++++++++++++++++-----------
 fs/gfs2/util.h       |  76 +++++++---
 fs/gfs2/xattr.c      |  12 +-
 27 files changed, 1168 insertions(+), 509 deletions(-)

[Cluster-devel] GFS2 changes for the 5.7 merge window

[Linus Torvalds]

On Tue, Mar 31, 2020 at 9:41 AM Bob Peterson <rpeterso@redhat.com> wrote:
>
> Could you please pull the following changes for gfs2?

So I've pulled it, but I note that you didn't get the automated
notification of that like almost everybody else does.

The reason seems to be pretty simple: the pr-tracker-bot looks for
emails to lkml (and other mailing lists) with variations of the
subject line "[GIT PULL]" in it.

And while lkml was cc'd for your pull request, you don't use that
subject line prefix, and so pr-tracker-bot ignores your email.

I don't personally care - you do have the markers that _I_ look for,
with both "git" and "pull" in the message body, so your pull request
doesn't get lost in the chaos that is my inbox during the merge
window. So this is purely about that notification bot.

IOW, if you care, then you should change your scripting (or manual
habits, I don't know) to have that "[GIT PULL]" in the subject line,
and then you will get that nice timely notification when I've pulled
(and pushed out).

And if you don't want that notification, you can just continue doing
what you're doing.

                  Linus

Re: GFS2 changes for the 5.7 merge window

[Linus Torvalds]

On Tue, Mar 31, 2020 at 9:41 AM Bob Peterson <rpeterso@redhat.com> wrote:
>
> Could you please pull the following changes for gfs2?

So I've pulled it, but I note that you didn't get the automated
notification of that like almost everybody else does.

The reason seems to be pretty simple: the pr-tracker-bot looks for
emails to lkml (and other mailing lists) with variations of the
subject line "[GIT PULL]" in it.

And while lkml was cc'd for your pull request, you don't use that
subject line prefix, and so pr-tracker-bot ignores your email.

I don't personally care - you do have the markers that _I_ look for,
with both "git" and "pull" in the message body, so your pull request
doesn't get lost in the chaos that is my inbox during the merge
window. So this is purely about that notification bot.

IOW, if you care, then you should change your scripting (or manual
habits, I don't know) to have that "[GIT PULL]" in the subject line,
and then you will get that nice timely notification when I've pulled
(and pushed out).

And if you don't want that notification, you can just continue doing
what you're doing.

                  Linus