From: Jan Kara <jack@suse.cz>
To: Dmitry Monakhov <dmonakhov@openvz.org>
Cc: linux-ext4@vger.kernel.org, tytso@mit.edu, jack@suse.cz,
wenqing.lz@taobao.com
Subject: Re: [PATCH 1/3] jbd2: fix race between jbd2_journal_remove_checkpoint and ->j_commit_callback V2
Date: Wed, 27 Mar 2013 15:32:37 +0100 [thread overview]
Message-ID: <20130327143237.GA1771@quack.suse.cz> (raw)
In-Reply-To: <1364376164-31701-1-git-send-email-dmonakhov@openvz.org>
On Wed 27-03-13 13:22:42, Dmitry Monakhov wrote:
> Following race is possible
> [kjournald2] other_task
> jbd2_journal_commit_transaction()
> j_state = T_FINISHED;
> spin_unlock(&journal->j_list_lock);
> ->jbd2_journal_remove_checkpoint()
> ->jbd2_journal_free_transaction();
> ->kmem_cache_free(transaction)
> ->j_commit_callback(journal, transaction);
> -> USE_AFTER_FREE
>
> WARNING: at lib/list_debug.c:62 __list_del_entry+0x1c0/0x250()
> Hardware name:
> list_del corruption. prev->next should be ffff88019a4ec198, but was 6b6b6b6b6b6b6b6b
> Modules linked in: cpufreq_ondemand acpi_cpufreq freq_table mperf coretemp kvm_intel kvm crc32c_intel ghash_clmulni_intel microcode sg xhci_hcd button sd_mod crc_t10dif aesni_intel ablk_helper cryptd lrw aes_x86_64 xts gf128mul ahci libahci pata_acpi ata_generic dm_mirror dm_region_hash dm_log dm_mod
> Pid: 16400, comm: jbd2/dm-1-8 Tainted: G W 3.8.0-rc3+ #107
> Call Trace:
> [<ffffffff8106fb0d>] warn_slowpath_common+0xad/0xf0
> [<ffffffff8106fc06>] warn_slowpath_fmt+0x46/0x50
> [<ffffffff813637e9>] ? ext4_journal_commit_callback+0x99/0xc0
> [<ffffffff8148cae0>] __list_del_entry+0x1c0/0x250
> [<ffffffff813637bf>] ext4_journal_commit_callback+0x6f/0xc0
> [<ffffffff813ca336>] jbd2_journal_commit_transaction+0x23a6/0x2570
> [<ffffffff8108aa42>] ? try_to_del_timer_sync+0x82/0xa0
> [<ffffffff8108b491>] ? del_timer_sync+0x91/0x1e0
> [<ffffffff813d3ecf>] kjournald2+0x19f/0x6a0
> [<ffffffff810ad630>] ? wake_up_bit+0x40/0x40
> [<ffffffff813d3d30>] ? bit_spin_lock+0x80/0x80
> [<ffffffff810ac6be>] kthread+0x10e/0x120
> [<ffffffff810ac5b0>] ? __init_kthread_worker+0x70/0x70
> [<ffffffff818ff6ac>] ret_from_fork+0x7c/0xb0
> [<ffffffff810ac5b0>] ? __init_kthread_worker+0x70/0x70
>
> In order to demonstrace this issue one should mount ext4 with -odiscard option
> on SSD disk. This makes callback longer and race window becomes wider.
>
> In order to fix this we should mark transaction as finished only after
> callbacks have completed
>
> Changes since V1:
> - Simplify code-flow and add comments according to Jan's request
Looks good. Just one text correction below - Ted can you apply it please?
...
> -
> + /* Drop all spin_locks because commit_callback may be block.
> + * __journal_remove_checkpoint() can not destroy transaction
> + * under us because it is marked as T_FINISHED yet */
^^^ is *not*
> if (journal->j_commit_callback)
> journal->j_commit_callback(journal, commit_transaction);
>
> trace_jbd2_end_commit(journal, commit_transaction);
> jbd_debug(1, "JBD2: commit %d complete, head %d\n",
> journal->j_commit_sequence, journal->j_tail_sequence);
Honza
--
Jan Kara <jack@suse.cz>
SUSE Labs, CR
next prev parent reply other threads:[~2013-03-27 14:32 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-03-22 16:18 [PATCH 1/3] jbd2: fix race between jbd2_journal_remove_checkpoint and ->j_commit_callback Dmitry Monakhov
2013-03-22 16:18 ` [PATCH 2/3] ext4: fix journal callback list traversal Dmitry Monakhov
2013-03-26 21:48 ` Jan Kara
2013-03-22 16:18 ` [PATCH 3/3] ext4: undegister es_shrinker if mount failed Dmitry Monakhov
2013-03-23 2:31 ` Zheng Liu
2013-04-01 14:49 ` Theodore Ts'o
2013-03-26 21:34 ` [PATCH 1/3] jbd2: fix race between jbd2_journal_remove_checkpoint and ->j_commit_callback Jan Kara
2013-03-27 2:56 ` Theodore Ts'o
2013-03-27 9:22 ` [PATCH 1/3] jbd2: fix race between jbd2_journal_remove_checkpoint and ->j_commit_callback V2 Dmitry Monakhov
2013-03-27 9:22 ` [PATCH 2/3] ext4: fix journal callback list traversal V2 Dmitry Monakhov
2013-03-27 14:16 ` Theodore Ts'o
2013-03-27 9:22 ` [PATCH 3/3] ext4: undegister es_shrinker if mount failed Dmitry Monakhov
2013-03-27 14:17 ` Theodore Ts'o
2013-03-27 14:16 ` [PATCH 1/3] jbd2: fix race between jbd2_journal_remove_checkpoint and ->j_commit_callback V2 Theodore Ts'o
2013-03-27 14:32 ` Jan Kara [this message]
2013-03-27 14:49 ` Theodore Ts'o
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20130327143237.GA1771@quack.suse.cz \
--to=jack@suse.cz \
--cc=dmonakhov@openvz.org \
--cc=linux-ext4@vger.kernel.org \
--cc=tytso@mit.edu \
--cc=wenqing.lz@taobao.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).