From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ross Zwisler Subject: Re: [4.3-rc1, regression] dax: hang on i_mmap_rwsem in generic/075 Date: Tue, 22 Sep 2015 22:14:12 -0600 Message-ID: <20150923041412.GA9909@linux.intel.com> References: <20150922030555.GA3902@dastard> <20150922100644.GA17788@node.dhcp.inet.fi> <20150922195631.GG3902@dastard> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: "Kirill A. Shutemov" , linux-fsdevel@vger.kernel.org, willy@linux.intel.com, kirill.shutemov@linux.intel.com, linux-kernel@vger.kernel.org To: Dave Chinner Return-path: Received: from mga09.intel.com ([134.134.136.24]:54383 "EHLO mga09.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750712AbbIWEOO (ORCPT ); Wed, 23 Sep 2015 00:14:14 -0400 Content-Disposition: inline In-Reply-To: <20150922195631.GG3902@dastard> Sender: linux-fsdevel-owner@vger.kernel.org List-ID: On Wed, Sep 23, 2015 at 05:56:31AM +1000, Dave Chinner wrote: > On Tue, Sep 22, 2015 at 01:06:45PM +0300, Kirill A. Shutemov wrote: > > On Tue, Sep 22, 2015 at 01:05:55PM +1000, Dave Chinner wrote: > > > Hi folks, > > > > > > I'm seeing hangs like this when using DAX on XFS on 4.3-rc1 running > > > xfstests generic/075 (fsx test): > > > > > > # echo w > /proc/sysrq-trigger > > > [71628.984872] sysrq: SysRq : Show Blocked State > > > [71628.985988] task PC stack pid father > > > [71628.987635] fsx D ffff88043fd756c0 12824 520 32636 0x00000000 > > > [71628.989251] ffff88007f557ba8 0000000000000086 ffff88042eb40580 ffff8803c8bcc180 > > > [71628.990645] ffff88007f558000 ffff88041d748e80 ffff88041d748e68 ffffffff00000000 > > > [71628.992068] 00000000fffffffe ffff88007f557bc0 ffffffff81d855ca ffff8803c8bcc180 > > > [71628.993639] Call Trace: > > > [71628.994097] [] schedule+0x3a/0x90 > > > [71628.994997] [] rwsem_down_write_failed+0x141/0x340 > > > [71628.996197] [] call_rwsem_down_write_failed+0x13/0x20 > > > [71628.997548] [] ? down_write+0x24/0x40 > > > [71628.998502] [] __dax_fault+0x546/0x6c0 > > > [71628.999469] [] ? xfs_get_blocks+0x20/0x20 > > > [71629.000515] [] xfs_filemap_fault+0xc8/0xf0 > > > [71629.001668] [] __do_fault+0x3d/0x80 > > > [71629.002589] [] handle_mm_fault+0xb8a/0xfd0 > > > [71629.003620] [] __do_page_fault+0x15f/0x420 > > > [71629.004680] [] trace_do_page_fault+0x43/0x110 > > > [71629.005877] [] do_async_page_fault+0x1a/0xa0 > > > [71629.006936] [] async_page_fault+0x28/0x30 > > > > > > __dax_fault() gets stuck on this lock: > > > > > > (gdb) l *(__dax_fault+0x546) > > > 0xffffffff812110b6 is in __dax_fault (include/linux/fs.h:499). > > > 494 > > > 495 int mapping_tagged(struct address_space *mapping, int tag); > > > 496 > > > 497 static inline void i_mmap_lock_write(struct address_space *mapping) > > > 498 { > > > 499 down_write(&mapping->i_mmap_rwsem); > > > 500 } > > > 501 > > > 502 static inline void i_mmap_unlock_write(struct address_space *mapping) > > > 503 { > > > > > > This didn't happen on 4.2 + the XFS for-next code that was merged > > > into 4.3-rc1, so it's come from changes somewhere else in the merge. > > > I suspect either of these two commits: > > > > > > 46c043e mm: take i_mmap_lock in unmap_mapping_range() for DAX > > > 8431729 dax: fix race between simultaneous faults > > > > > > as they both modified the i_mmap_lock usage for DAX page faults. > > > > It's likely we broke some locking ordering rules, but it's not obvious for > > me which one. > > > > No lockdep complain? Or it's disabled? > > Wasn't running lockdep, I don't always use it because of how slow it > can make things. I'll turn it on and see what happens... I just wanted to let you know that I was able to reproduce this on my test setup and am planning on tracking it down tomorrow, unless someone gets to it first.