public inbox for linux-bcachefs@vger.kernel.org
 help / color / mirror / Atom feed
From: Brian Foster <bfoster@redhat.com>
To: linux-bcachefs@vger.kernel.org
Subject: [PATCH 5/5] RFC: bcachefs: use a timeout for the journal stuck condition
Date: Tue, 21 Mar 2023 09:20:14 -0400	[thread overview]
Message-ID: <20230321132014.1438249-6-bfoster@redhat.com> (raw)
In-Reply-To: <20230321132014.1438249-1-bfoster@redhat.com>

This is currently just a thought/experiment on how to make the
journal checking logic a bit more reliable. I've seen this actually
still result in blocked tasks in the journal reservation slow path,
which I think is actually due to the timeout allowing tasks to wind down
long enough after a legitimate stall such that there are no more journal
wake events to allow any one of the tasks to detect the stuck condition
and shut down the fs.

This would need to be addressed somehow or another for this sort of
thing to be useful. One approach could be a timer of some sort to
monitor things as the journal becomes full, but I think that is
approaching overkill given this is a rare enough problem. This is
primarily posted for thought and discussion.

Not-Signed-off-by: Brian Foster <bfoster@redhat.com>
---
 fs/bcachefs/journal.c | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/fs/bcachefs/journal.c b/fs/bcachefs/journal.c
index 3f0e6d71aa32..a83b753fbc3f 100644
--- a/fs/bcachefs/journal.c
+++ b/fs/bcachefs/journal.c
@@ -85,7 +85,8 @@ static void journal_pin_list_init(struct journal_entry_pin_list *p, int count)
  *
  * Consider the journal stuck when it appears full with no ability to commit
  * btree transactions, to discard journal buckets, nor acquire priority
- * (reserved watermark) reservation.
+ * (reserved watermark) reservation, and we have not been able to open a new
+ * journal entry for at least 30s.
  */
 static inline bool
 journal_error_check_stuck(struct journal *j, int error, unsigned flags)
@@ -93,6 +94,7 @@ journal_error_check_stuck(struct journal *j, int error, unsigned flags)
 	struct bch_fs *c = container_of(j, struct bch_fs, journal);
 	bool stuck = false;
 	struct printbuf buf = PRINTBUF;
+	u64 stuck_ts;
 
 	if (!(error == JOURNAL_ERR_journal_full ||
 	      error == JOURNAL_ERR_journal_pin_full) ||
@@ -102,7 +104,8 @@ journal_error_check_stuck(struct journal *j, int error, unsigned flags)
 
 	spin_lock(&j->lock);
 
-	if (j->can_discard) {
+	stuck_ts = j->res_get_blocked_start + (NSEC_PER_SEC * 30);
+	if (j->can_discard || time_before64(local_clock(), stuck_ts)) {
 		spin_unlock(&j->lock);
 		return stuck;
 	}
-- 
2.39.2


      parent reply	other threads:[~2023-03-21 13:19 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-03-21 13:20 [PATCH 0/5] bcachefs: journal stall fixes Brian Foster
2023-03-21 13:20 ` [PATCH 1/5] bcachefs: more aggressive fast path write buffer key flushing Brian Foster
2023-03-21 13:40   ` Brian Foster
2023-03-21 13:20 ` [PATCH 2/5] bcachefs: gracefully unwind journal res slowpath on shutdown Brian Foster
2023-03-21 13:20 ` [PATCH 3/5] bcachefs: refactor journal stuck checking into standalone helper Brian Foster
2023-03-21 13:20 ` [PATCH 4/5] bcachefs: drop unnecessary journal stuck check from space calculation Brian Foster
2023-03-21 13:20 ` Brian Foster [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230321132014.1438249-6-bfoster@redhat.com \
    --to=bfoster@redhat.com \
    --cc=linux-bcachefs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox