From: Theodore Ts'o <tytso@mit.edu>
To: Paul Gortmaker <paul.gortmaker@windriver.com>
Cc: linux-ext4@vger.kernel.org, linux-rt-users@vger.kernel.org
Subject: Re: [PATCH 2/6] jbd2/log_wait_for_space: drop checkpoint mutex when waiting
Date: Wed, 12 Jun 2013 22:55:26 -0400 [thread overview]
Message-ID: <20130613025526.GD16959@thunk.org> (raw)
In-Reply-To: <1370990670-49249-3-git-send-email-paul.gortmaker@windriver.com>
On Tue, Jun 11, 2013 at 06:44:26PM -0400, Paul Gortmaker wrote:
> While trying to debug an an issue under extreme I/O loading
> on preempt-rt kernels, the following backtrace was observed
> via SysRQ output:
>
> rm D ffff8802203afbc0 4600 4878 4748 0x00000000
> ffff8802217bfb78 0000000000000082 ffff88021fc2bb80 ffff88021fc2bb80
> ffff88021fc2bb80 ffff8802217bffd8 ffff8802217bffd8 ffff8802217bffd8
> ffff88021f1d4c80 ffff88021fc2bb80 ffff8802217bfb88 ffff88022437b000
> Call Trace:
> [<ffffffff8172dc34>] schedule+0x24/0x70
> [<ffffffff81225b5d>] jbd2_log_wait_commit+0xbd/0x140
> [<ffffffff81060390>] ? __init_waitqueue_head+0x50/0x50
> [<ffffffff81223635>] jbd2_log_do_checkpoint+0xf5/0x520
> [<ffffffff81223b09>] __jbd2_log_wait_for_space+0xa9/0x1f0
> [<ffffffff8121dc40>] start_this_handle.isra.10+0x2e0/0x530
> [<ffffffff81060390>] ? __init_waitqueue_head+0x50/0x50
> [<ffffffff8121e0a3>] jbd2__journal_start+0xc3/0x110
> [<ffffffff811de7ce>] ? ext4_rmdir+0x6e/0x230
> [<ffffffff8121e0fe>] jbd2_journal_start+0xe/0x10
> [<ffffffff811f308b>] ext4_journal_start_sb+0x5b/0x160
> [<ffffffff811de7ce>] ext4_rmdir+0x6e/0x230
> [<ffffffff811435c5>] vfs_rmdir+0xd5/0x140
> [<ffffffff8114370f>] do_rmdir+0xdf/0x120
> [<ffffffff8105c6b4>] ? task_work_run+0x44/0x80
> [<ffffffff81002889>] ? do_notify_resume+0x89/0x100
> [<ffffffff817361ae>] ? int_signal+0x12/0x17
> [<ffffffff81145d85>] sys_unlinkat+0x25/0x40
> [<ffffffff81735f22>] system_call_fastpath+0x16/0x1b
>
> What is interesting here, is that we call log_wait_commit, from
> within wait_for_space, but we are still holding the checkpoint_mutex
> as it surrounds mostly the whole of wait_for_space. And then, as we
> are waiting, journal_commit_transaction can run, and if the JBD2_FLUSHED
> bit is set, then we will also try to take the same checkpoint_mutex.
>
> It seems that we need to drop the checkpoint_mutex while sitting in
> jbd2_log_wait_commit, if we want to guarantee that progress can be made
> by jbd2_journal_commit_transaction(). There does not seem to be
> anything preempt-rt specific about this, other then perhaps increasing
> the odds of it happening.
>
> Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
Applied, thanks.
- Ted
next prev parent reply other threads:[~2013-06-13 2:55 UTC|newest]
Thread overview: 33+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-06-10 19:31 [RFC PATCH 0/4] ext4/jbd2: several possible mainline fixes Paul Gortmaker
2013-06-10 19:32 ` [PATCH 1/4] jbd2/journal_commit_transaction: relocate state lock to incorporate all users Paul Gortmaker
2013-06-11 2:12 ` Theodore Ts'o
2013-06-11 2:45 ` Paul Gortmaker
2013-06-11 2:52 ` Theodore Ts'o
2013-06-11 17:38 ` Paul Gortmaker
2013-06-11 17:53 ` Theodore Ts'o
2013-06-11 18:48 ` Paul Gortmaker
2013-06-11 21:54 ` Paul Gortmaker
2013-06-10 19:32 ` [PATCH 2/4] jbd2/log_wait_for_space: drop checkpoint mutex when waiting Paul Gortmaker
2013-06-11 2:33 ` Theodore Ts'o
2013-06-11 3:20 ` Paul Gortmaker
2013-06-11 13:03 ` Theodore Ts'o
2013-06-11 13:20 ` Paul Gortmaker
2013-06-10 19:32 ` [PATCH 3/4] jbd2: fix duplicate debug label for phase 2 Paul Gortmaker
2013-06-10 19:32 ` [PATCH 4/4] jbd/jbd2: relocate bit_spinlock header to jbd_common Paul Gortmaker
2013-06-10 23:38 ` [RFC PATCH 0/4] ext4/jbd2: several possible mainline fixes Theodore Ts'o
2013-06-11 3:09 ` Paul Gortmaker
2013-06-11 22:44 ` [PATCH v2 0/6] misc jbd2 fixes and cleanups Paul Gortmaker
2013-06-11 22:44 ` [PATCH 1/6] jbd2/journal_commit_transaction: relocate assert after state lock Paul Gortmaker
2013-06-13 2:42 ` Theodore Ts'o
2013-06-11 22:44 ` [PATCH 2/6] jbd2/log_wait_for_space: drop checkpoint mutex when waiting Paul Gortmaker
2013-06-13 2:55 ` Theodore Ts'o [this message]
2013-06-11 22:44 ` [PATCH 3/6] jbd2: fix duplicate debug label for phase 2 Paul Gortmaker
2013-06-13 2:57 ` Theodore Ts'o
2013-06-11 22:44 ` [PATCH 4/6] jbd/jbd2: relocate bit_spinlock header to jbd_common Paul Gortmaker
2013-06-13 3:02 ` Theodore Ts'o
2013-06-11 22:44 ` [PATCH 5/6] jbd2: make jbd_debug that won't split printk statements Paul Gortmaker
2013-06-11 22:44 ` [PATCH 6/6] jbd2: remove debug dependency on debug_fs; update help text Paul Gortmaker
2013-06-13 3:08 ` Theodore Ts'o
2013-06-13 13:51 ` Paul Gortmaker
2013-06-13 14:14 ` Theodore Ts'o
2013-06-13 14:47 ` Paul Gortmaker
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20130613025526.GD16959@thunk.org \
--to=tytso@mit.edu \
--cc=linux-ext4@vger.kernel.org \
--cc=linux-rt-users@vger.kernel.org \
--cc=paul.gortmaker@windriver.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.