From: Ojaswin Mujoo <ojaswin@linux.ibm.com>
To: linux-ext4@vger.kernel.org, "Theodore Ts'o" <tytso@mit.edu>
Cc: Jan Kara <jack@suse.cz>, linux-kernel@vger.kernel.org
Subject: [PATCH 0/2] Fix a BUG_ON crashing the kernel in start_this_handle
Date: Sat, 22 Feb 2025 14:10:21 +0530 [thread overview]
Message-ID: <cover.1740212945.git.ojaswin@linux.ibm.com> (raw)
When running LTP stress tests on ext4, after a multiday run we seemed to
have hit the following BUG_ON:
[NIP : start_this_handle+268]
#3 [c000001067c27a40] start_this_handle at c008000004d40f74 [jbd2] (unreliable)
#4 [c000001067c27b60] jbd2__journal_start at c008000004d415cc [jbd2]
#5 [c000001067c27be0] update_super_work at c0080000053f9758 [ext4]
#6 [c000001067c27c70] process_one_work at c000000000188790
#7 [c000001067c27d20] worker_thread at c00000000018973c
#8 [c000001067c27dc0] kthread at c000000000196c84
#9 [c000001067c27e10] ret_from_kernel_thread at c00000000000cd64
Which comes out to
382 repeat:
383 read_lock(&journal->j_state_lock);
* 384 BUG_ON(journal->j_flags & JBD2_UNMOUNT);
385 if (is_journal_aborted(journal) ||
386 (journal->j_errno != 0 && !(journal->j_flags & JBD2_ACK_ERR))) {
387 read_unlock(&journal->j_state_lock);
Initially this seemed like it should never happen but upon crash
analysis it seems like it could indeed be hit as described in patch 1/2.
I would like to add that through the logs we only knew that:
- ext4_journal_bmap -> ext4_map_blocks is failing with EFSCORRUPTED.
- update_super_work had hit the BUG_ON
I was not able to hit this bug again (without modifying the kernel to
inject errors) but the above backtrace seems to be one possible paths
where this BUG_ON can be hit. Rest of the analysis and fix is in patch
1/2. Patch 2 is just a small tweak that i found helpful while debugging.
That being said, journalling is something I'm not very familiar with and
there might be gaps in my understanding so thoughts and suggestions are
welcome.
Ojaswin Mujoo (2):
ext4: only defer sb update on error if SB_ACTIVE
ext4: Make sb update interval tunable
fs/ext4/ext4.h | 9 +++++++++
fs/ext4/super.c | 17 +++++++++--------
fs/ext4/sysfs.c | 4 ++++
3 files changed, 22 insertions(+), 8 deletions(-)
--
2.48.1
next reply other threads:[~2025-02-22 8:40 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-02-22 8:40 Ojaswin Mujoo [this message]
2025-02-22 8:40 ` [PATCH 1/2] ext4: only defer sb update on error if SB_ACTIVE Ojaswin Mujoo
2025-02-24 14:52 ` Jan Kara
2025-02-27 11:20 ` Ojaswin Mujoo
2025-03-04 10:25 ` Jan Kara
2025-03-04 12:54 ` Ojaswin Mujoo
2025-03-05 4:14 ` Ritesh Harjani
2025-02-25 1:53 ` Baokun Li
2025-02-25 12:06 ` Ojaswin Mujoo
2025-02-26 1:55 ` Baokun Li
2025-02-27 11:43 ` Ojaswin Mujoo
2025-02-27 12:51 ` Baokun Li
2025-03-03 17:30 ` Ojaswin Mujoo
2025-02-22 8:40 ` [PATCH 2/2] ext4: Make sb update interval tunable Ojaswin Mujoo
2025-02-22 9:32 ` Ritesh Harjani (IBM)
2025-02-24 14:27 ` Jan Kara
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cover.1740212945.git.ojaswin@linux.ibm.com \
--to=ojaswin@linux.ibm.com \
--cc=jack@suse.cz \
--cc=linux-ext4@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=tytso@mit.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox