From: Theodore Ts'o <tytso@mit.edu>
To: Paul Gortmaker <paul.gortmaker@windriver.com>
Cc: linux-ext4@vger.kernel.org, linux-rt-users@vger.kernel.org
Subject: Re: [PATCH 2/6] jbd2/log_wait_for_space: drop checkpoint mutex when waiting
Date: Wed, 12 Jun 2013 22:55:26 -0400 [thread overview]
Message-ID: <20130613025526.GD16959@thunk.org> (raw)
In-Reply-To: <1370990670-49249-3-git-send-email-paul.gortmaker@windriver.com>
On Tue, Jun 11, 2013 at 06:44:26PM -0400, Paul Gortmaker wrote:
> While trying to debug an an issue under extreme I/O loading
> on preempt-rt kernels, the following backtrace was observed
> via SysRQ output:
>
> rm D ffff8802203afbc0 4600 4878 4748 0x00000000
> ffff8802217bfb78 0000000000000082 ffff88021fc2bb80 ffff88021fc2bb80
> ffff88021fc2bb80 ffff8802217bffd8 ffff8802217bffd8 ffff8802217bffd8
> ffff88021f1d4c80 ffff88021fc2bb80 ffff8802217bfb88 ffff88022437b000
> Call Trace:
> [<ffffffff8172dc34>] schedule+0x24/0x70
> [<ffffffff81225b5d>] jbd2_log_wait_commit+0xbd/0x140
> [<ffffffff81060390>] ? __init_waitqueue_head+0x50/0x50
> [<ffffffff81223635>] jbd2_log_do_checkpoint+0xf5/0x520
> [<ffffffff81223b09>] __jbd2_log_wait_for_space+0xa9/0x1f0
> [<ffffffff8121dc40>] start_this_handle.isra.10+0x2e0/0x530
> [<ffffffff81060390>] ? __init_waitqueue_head+0x50/0x50
> [<ffffffff8121e0a3>] jbd2__journal_start+0xc3/0x110
> [<ffffffff811de7ce>] ? ext4_rmdir+0x6e/0x230
> [<ffffffff8121e0fe>] jbd2_journal_start+0xe/0x10
> [<ffffffff811f308b>] ext4_journal_start_sb+0x5b/0x160
> [<ffffffff811de7ce>] ext4_rmdir+0x6e/0x230
> [<ffffffff811435c5>] vfs_rmdir+0xd5/0x140
> [<ffffffff8114370f>] do_rmdir+0xdf/0x120
> [<ffffffff8105c6b4>] ? task_work_run+0x44/0x80
> [<ffffffff81002889>] ? do_notify_resume+0x89/0x100
> [<ffffffff817361ae>] ? int_signal+0x12/0x17
> [<ffffffff81145d85>] sys_unlinkat+0x25/0x40
> [<ffffffff81735f22>] system_call_fastpath+0x16/0x1b
>
> What is interesting here, is that we call log_wait_commit, from
> within wait_for_space, but we are still holding the checkpoint_mutex
> as it surrounds mostly the whole of wait_for_space. And then, as we
> are waiting, journal_commit_transaction can run, and if the JBD2_FLUSHED
> bit is set, then we will also try to take the same checkpoint_mutex.
>
> It seems that we need to drop the checkpoint_mutex while sitting in
> jbd2_log_wait_commit, if we want to guarantee that progress can be made
> by jbd2_journal_commit_transaction(). There does not seem to be
> anything preempt-rt specific about this, other then perhaps increasing
> the odds of it happening.
>
> Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
Applied, thanks.
- Ted
next prev parent reply other threads:[~2013-06-13 2:55 UTC|newest]
Thread overview: 33+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-06-10 19:31 [RFC PATCH 0/4] ext4/jbd2: several possible mainline fixes Paul Gortmaker
2013-06-10 19:32 ` [PATCH 1/4] jbd2/journal_commit_transaction: relocate state lock to incorporate all users Paul Gortmaker
2013-06-11 2:12 ` Theodore Ts'o
2013-06-11 2:45 ` Paul Gortmaker
2013-06-11 2:52 ` Theodore Ts'o
2013-06-11 17:38 ` Paul Gortmaker
2013-06-11 17:53 ` Theodore Ts'o
2013-06-11 18:48 ` Paul Gortmaker
2013-06-11 21:54 ` Paul Gortmaker
2013-06-10 19:32 ` [PATCH 2/4] jbd2/log_wait_for_space: drop checkpoint mutex when waiting Paul Gortmaker
2013-06-11 2:33 ` Theodore Ts'o
2013-06-11 3:20 ` Paul Gortmaker
2013-06-11 13:03 ` Theodore Ts'o
2013-06-11 13:20 ` Paul Gortmaker
2013-06-10 19:32 ` [PATCH 3/4] jbd2: fix duplicate debug label for phase 2 Paul Gortmaker
2013-06-10 19:32 ` [PATCH 4/4] jbd/jbd2: relocate bit_spinlock header to jbd_common Paul Gortmaker
2013-06-10 23:38 ` [RFC PATCH 0/4] ext4/jbd2: several possible mainline fixes Theodore Ts'o
2013-06-11 3:09 ` Paul Gortmaker
2013-06-11 22:44 ` [PATCH v2 0/6] misc jbd2 fixes and cleanups Paul Gortmaker
2013-06-11 22:44 ` [PATCH 1/6] jbd2/journal_commit_transaction: relocate assert after state lock Paul Gortmaker
2013-06-13 2:42 ` Theodore Ts'o
2013-06-11 22:44 ` [PATCH 2/6] jbd2/log_wait_for_space: drop checkpoint mutex when waiting Paul Gortmaker
2013-06-13 2:55 ` Theodore Ts'o [this message]
2013-06-11 22:44 ` [PATCH 3/6] jbd2: fix duplicate debug label for phase 2 Paul Gortmaker
2013-06-13 2:57 ` Theodore Ts'o
2013-06-11 22:44 ` [PATCH 4/6] jbd/jbd2: relocate bit_spinlock header to jbd_common Paul Gortmaker
2013-06-13 3:02 ` Theodore Ts'o
2013-06-11 22:44 ` [PATCH 5/6] jbd2: make jbd_debug that won't split printk statements Paul Gortmaker
2013-06-11 22:44 ` [PATCH 6/6] jbd2: remove debug dependency on debug_fs; update help text Paul Gortmaker
2013-06-13 3:08 ` Theodore Ts'o
2013-06-13 13:51 ` Paul Gortmaker
2013-06-13 14:14 ` Theodore Ts'o
2013-06-13 14:47 ` Paul Gortmaker
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20130613025526.GD16959@thunk.org \
--to=tytso@mit.edu \
--cc=linux-ext4@vger.kernel.org \
--cc=linux-rt-users@vger.kernel.org \
--cc=paul.gortmaker@windriver.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).