From: Bob Peterson <rpeterso@redhat.com>
To: cluster-devel.redhat.com
Subject: [Cluster-devel] [GFS2 PATCH 0/2] GFS2: Avoid inode shrinker-related deadlocks
Date: Thu, 19 Nov 2015 12:42:39 -0600 [thread overview]
Message-ID: <1447958561-2584-1-git-send-email-rpeterso@redhat.com> (raw)
Hi,
This set of two patches is very simple, even though it's somewhat big.
The problem is that GFS2 can livelock when the inode shrinker runs.
The failing scenario goes like this:
The inode shrinker tells GFS2 to evict inodes, but in order to evict them,
the evict code needs to unlock their glocks, and calls DLM to do so.
In this particular scenario, the DLM can't unlock the glocks because
it's blocked waiting on a pending fence operation to complete. The fence
operation can't complete because it's blocked waiting to allocate memory.
The allocation of memory can't complete because it's waiting for the
shrinker, which of course, is waiting for GFS2 to evict inodes.
The problem is that it's unsafe for GFS2's evict code to make a call into
DLM. The solution is to queue the final glock put to the glock state
machine. Since the glocks outlive the inode anyway, it's safe for the
evict code to continue and return successfully back to the shrinker,
which can then continue running and satisfy the memory needed for the
fence operation, and so forth.
The second patch reverts commit 35e478f, which made the evict code wait
for outstanding work on the glock state machine. That's also unsafe for
the same reason: It can block on the state machine, which can block on
DLM. That should be safe now, for reasons stated in that patch.
Bob Peterson (2):
GFS2: Make gfs2_clear_inode() queue the final put
GFS2: Revert 35e478f Flush pending glock work when evicting an inode
fs/gfs2/glock.c | 34 +++++++++++++++++-----------------
fs/gfs2/glock.h | 1 +
fs/gfs2/super.c | 6 ++++--
3 files changed, 22 insertions(+), 19 deletions(-)
--
2.5.0
next reply other threads:[~2015-11-19 18:42 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-11-19 18:42 Bob Peterson [this message]
2015-11-19 18:42 ` [Cluster-devel] [GFS2 PATCH 1/2] GFS2: Make gfs2_clear_inode() queue the final put Bob Peterson
2015-11-20 13:33 ` Steven Whitehouse
2015-11-25 14:22 ` Bob Peterson
2015-11-25 14:26 ` Steven Whitehouse
2015-12-01 15:42 ` Bob Peterson
2015-12-02 10:23 ` Steven Whitehouse
2015-12-02 16:42 ` Bob Peterson
2015-12-02 17:41 ` Bob Peterson
2015-12-03 11:18 ` Steven Whitehouse
2015-12-04 14:51 ` Bob Peterson
2015-12-04 15:51 ` David Teigland
2015-12-04 17:38 ` Bob Peterson
2015-12-08 7:57 ` Dave Chinner
2015-12-08 9:03 ` Steven Whitehouse
2015-11-19 18:42 ` [Cluster-devel] [GFS2 PATCH 2/2] GFS2: Revert 35e478f Flush pending glock work when evicting an inode Bob Peterson
2015-11-20 13:47 ` Steven Whitehouse
2015-11-25 14:36 ` Bob Peterson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1447958561-2584-1-git-send-email-rpeterso@redhat.com \
--to=rpeterso@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).