From: Sasha Levin <sasha.levin@oracle.com>
To: Benjamin LaHaise <bcrl@kvack.org>,
Gu Zheng <guz.fnst@cn.fujitsu.com>,
Tang Chen <tangchen@cn.fujitsu.com>
Cc: Dave Jones <davej@redhat.com>, Al Viro <viro@zeniv.linux.org.uk>,
jmoyer@redhat.com, kosaki.motohiro@jp.fujitsu.com,
KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>,
Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>,
miaox@cn.fujitsu.com, linux-aio@kvack.org,
fsdevel <linux-fsdevel@vger.kernel.org>,
linux-kernel <linux-kernel@vger.kernel.org>,
Andrew Morton <akpm@linux-foundation.org>,
"linux-mm@kvack.org" <linux-mm@kvack.org>
Subject: Re: [PATCH] aio: ensure access to ctx->ring_pages is correctly serialised
Date: Mon, 24 Mar 2014 14:22:06 -0400 [thread overview]
Message-ID: <533077CE.6010204@oracle.com> (raw)
In-Reply-To: <20140321183509.GC23173@kvack.org>
On 03/21/2014 02:35 PM, Benjamin LaHaise wrote:
> Hi all,
>
> Based on the issues reported by Tang and Gu, I've come up with the an
> alternative fix that avoids adding additional locking in the event read
> code path. The fix is to take the ring_lock mutex during page migration,
> which is already used to syncronize event readers and thus does not add
> any new locking requirements in aio_read_events_ring(). I've dropped
> the patches from Tang and Gu as a result. This patch is now in my
> git://git.kvack.org/~bcrl/aio-next.git tree and will be sent to Linus
> once a few other people chime in with their reviews of this change.
> Please review Tang, Gu. Thanks!
Hi Benjamin,
This patch seems to trigger:
[ 433.476216] ======================================================
[ 433.478468] [ INFO: possible circular locking dependency detected ]
[ 433.480900] 3.14.0-rc7-next-20140324-sasha-00015-g1fb7de8 #267 Tainted: G W
[ 433.480900] -------------------------------------------------------
[ 433.480900] trinity-c57/13776 is trying to acquire lock:
[ 433.480900] (&ctx->ring_lock){+.+.+.}, at: aio_migratepage (include/linux/spinlock.h:303 fs/aio.c:306)
[ 433.480900]
[ 433.480900] but task is already holding lock:
[ 433.480900] (&mm->mmap_sem){++++++}, at: SYSC_move_pages (mm/migrate.c:1215 mm/migrate.c:1353 mm/migrate.c:1508)
[ 433.480900]
[ 433.480900] which lock already depends on the new lock.
[ 433.480900]
[ 433.480900]
[ 433.480900] the existing dependency chain (in reverse order) is:
[ 433.480900]
-> #1 (&mm->mmap_sem){++++++}:
[ 433.480900] lock_acquire (arch/x86/include/asm/current.h:14 kernel/locking/lockdep.c:3602)
[ 433.480900] down_write (arch/x86/include/asm/rwsem.h:130 kernel/locking/rwsem.c:50)
[ 433.480900] SyS_io_setup (fs/aio.c:442 fs/aio.c:689 fs/aio.c:1201 fs/aio.c:1184)
[ 433.480900] tracesys (arch/x86/kernel/entry_64.S:749)
[ 433.480900]
-> #0 (&ctx->ring_lock){+.+.+.}:
[ 433.480900] __lock_acquire (kernel/locking/lockdep.c:1840 kernel/locking/lockdep.c:1945 kernel/locking/lockdep.c:2131 kernel/locking/lockdep.c:3182)
[ 433.480900] lock_acquire (arch/x86/include/asm/current.h:14 kernel/locking/lockdep.c:3602)
[ 433.480900] mutex_lock_nested (kernel/locking/mutex.c:486 kernel/locking/mutex.c:587)
[ 433.480900] aio_migratepage (include/linux/spinlock.h:303 fs/aio.c:306)
[ 433.480900] move_to_new_page (mm/migrate.c:777)
[ 433.480900] migrate_pages (mm/migrate.c:921 mm/migrate.c:960 mm/migrate.c:1126)
[ 433.480900] SYSC_move_pages (mm/migrate.c:1278 mm/migrate.c:1353 mm/migrate.c:1508)
[ 433.480900] SyS_move_pages (mm/migrate.c:1456)
[ 433.480900] tracesys (arch/x86/kernel/entry_64.S:749)
[ 433.480900]
[ 433.480900] other info that might help us debug this:
[ 433.480900]
[ 433.480900] Possible unsafe locking scenario:
[ 433.480900]
[ 433.480900] CPU0 CPU1
[ 433.480900] ---- ----
[ 433.480900] lock(&mm->mmap_sem);
[ 433.480900] lock(&ctx->ring_lock);
[ 433.480900] lock(&mm->mmap_sem);
[ 433.480900] lock(&ctx->ring_lock);
[ 433.480900]
[ 433.480900] *** DEADLOCK ***
[ 433.480900]
[ 433.480900] 1 lock held by trinity-c57/13776:
[ 433.480900] #0: (&mm->mmap_sem){++++++}, at: SYSC_move_pages (mm/migrate.c:1215 mm/migrate.c:1353 mm/migrate.c:1508)
[ 433.480900]
[ 433.480900] stack backtrace:
[ 433.480900] CPU: 4 PID: 13776 Comm: trinity-c57 Tainted: G W 3.14.0-rc7-next-20140324-sasha-00015-g1fb7de8 #267
[ 433.480900] ffffffff87a80790 ffff8806abbbb9a8 ffffffff844bae02 0000000000000000
[ 433.480900] ffffffff87a80790 ffff8806abbbb9f8 ffffffff844ad86d 0000000000000001
[ 433.480900] ffff8806abbbba88 ffff8806abbbb9f8 ffff8806ab8fbcf0 ffff8806ab8fbd28
[ 433.480900] Call Trace:
[ 433.480900] dump_stack (lib/dump_stack.c:52)
[ 433.480900] print_circular_bug (kernel/locking/lockdep.c:1216)
[ 433.480900] __lock_acquire (kernel/locking/lockdep.c:1840 kernel/locking/lockdep.c:1945 kernel/locking/lockdep.c:2131 kernel/locking/lockdep.c:3182)
[ 433.480900] ? sched_clock (arch/x86/include/asm/paravirt.h:192 arch/x86/kernel/tsc.c:305)
[ 433.480900] ? sched_clock_local (kernel/sched/clock.c:214)
[ 433.480900] ? sched_clock_cpu (kernel/sched/clock.c:311)
[ 433.480900] ? __lock_acquire (kernel/locking/lockdep.c:3189)
[ 433.480900] lock_acquire (arch/x86/include/asm/current.h:14 kernel/locking/lockdep.c:3602)
[ 433.480900] ? aio_migratepage (include/linux/spinlock.h:303 fs/aio.c:306)
[ 433.480900] mutex_lock_nested (kernel/locking/mutex.c:486 kernel/locking/mutex.c:587)
[ 433.480900] ? aio_migratepage (include/linux/spinlock.h:303 fs/aio.c:306)
[ 433.480900] ? aio_migratepage (fs/aio.c:303)
[ 433.480900] ? aio_migratepage (include/linux/spinlock.h:303 fs/aio.c:306)
[ 433.480900] ? aio_migratepage (include/linux/rcupdate.h:324 include/linux/rcupdate.h:909 include/linux/percpu-refcount.h:117 fs/aio.c:297)
[ 433.480900] ? preempt_count_sub (kernel/sched/core.c:2527)
[ 433.480900] aio_migratepage (include/linux/spinlock.h:303 fs/aio.c:306)
[ 433.480900] ? aio_migratepage (include/linux/rcupdate.h:886 include/linux/percpu-refcount.h:108 fs/aio.c:297)
[ 433.480900] ? mutex_unlock (kernel/locking/mutex.c:220)
[ 433.480900] move_to_new_page (mm/migrate.c:777)
[ 433.480900] ? try_to_unmap (mm/rmap.c:1516)
[ 433.480900] ? try_to_unmap_nonlinear (mm/rmap.c:1113)
[ 433.480900] ? invalid_migration_vma (mm/rmap.c:1472)
[ 433.480900] ? page_remove_rmap (mm/rmap.c:1380)
[ 433.480900] ? anon_vma_fork (mm/rmap.c:446)
[ 433.480900] migrate_pages (mm/migrate.c:921 mm/migrate.c:960 mm/migrate.c:1126)
[ 433.480900] ? follow_page_mask (mm/memory.c:1544)
[ 433.480900] ? alloc_misplaced_dst_page (mm/migrate.c:1177)
[ 433.480900] SYSC_move_pages (mm/migrate.c:1278 mm/migrate.c:1353 mm/migrate.c:1508)
[ 433.480900] ? SYSC_move_pages (include/linux/rcupdate.h:800 mm/migrate.c:1472)
[ 433.480900] ? sched_clock (arch/x86/include/asm/paravirt.h:192 arch/x86/kernel/tsc.c:305)
[ 433.480900] SyS_move_pages (mm/migrate.c:1456)
[ 433.480900] tracesys (arch/x86/kernel/entry_64.S:749)
Thanks,
Sasha
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2014-03-24 18:22 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-03-20 5:46 [PATCH 2/2] aio: fix the confliction of read events and migrating ring page Gu Zheng
2014-03-20 14:32 ` Dave Jones
2014-03-20 16:30 ` Benjamin LaHaise
2014-03-21 1:56 ` Gu Zheng
2014-03-21 17:35 ` Benjamin LaHaise
2014-03-21 18:35 ` [PATCH] aio: ensure access to ctx->ring_pages is correctly serialised Benjamin LaHaise
2014-03-24 10:56 ` Gu Zheng
2014-03-24 10:59 ` [V2 PATCH 1/2] aio: clean up aio_migratepage() and related code much Gu Zheng
2014-03-24 13:20 ` Benjamin LaHaise
2014-03-25 10:11 ` Gu Zheng
2014-03-24 10:59 ` [V2 PATCH 2/2] aio: fix the confliction of aio read events and aio migrate page Gu Zheng
2014-03-24 18:22 ` Sasha Levin [this message]
2014-03-24 19:07 ` [PATCH] aio: ensure access to ctx->ring_pages is correctly serialised Benjamin LaHaise
2014-03-25 17:47 ` Sasha Levin
2014-03-25 18:57 ` Benjamin LaHaise
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=533077CE.6010204@oracle.com \
--to=sasha.levin@oracle.com \
--cc=akpm@linux-foundation.org \
--cc=bcrl@kvack.org \
--cc=davej@redhat.com \
--cc=guz.fnst@cn.fujitsu.com \
--cc=isimatu.yasuaki@jp.fujitsu.com \
--cc=jmoyer@redhat.com \
--cc=kamezawa.hiroyu@jp.fujitsu.com \
--cc=kosaki.motohiro@jp.fujitsu.com \
--cc=linux-aio@kvack.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=miaox@cn.fujitsu.com \
--cc=tangchen@cn.fujitsu.com \
--cc=viro@zeniv.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).