From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757930AbcEEVrd (ORCPT ); Thu, 5 May 2016 17:47:33 -0400 Received: from mail-wm0-f43.google.com ([74.125.82.43]:36458 "EHLO mail-wm0-f43.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1757433AbcEEVr1 (ORCPT ); Thu, 5 May 2016 17:47:27 -0400 Date: Fri, 6 May 2016 00:47:19 +0300 From: Ebru Akagunduz To: kirill@shutemov.name, xiaolong.ye@intel.com Cc: sfr@canb.auug.org.au, riel@redhat.com, hughd@google.com, linux-kernel@vger.kernel.org, lkp@01.org Subject: Re: [lkp] [mm, thp] 409ca714ac: INFO: possible circular locking dependency detected Message-ID: <20160505214719.GA4185@debian> References: <20160505013245.GB10429@yexl-desktop> <20160505101843.GA13240@node.shutemov.name> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160505101843.GA13240@node.shutemov.name> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, May 05, 2016 at 01:18:43PM +0300, Kirill A. Shutemov wrote: > On Thu, May 05, 2016 at 09:32:45AM +0800, kernel test robot wrote: > > FYI, we noticed the following commit: > > > > https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git master > > commit 409ca714ac58768342cd39ca79c16f51e1825b3e ("mm, thp: avoid unnecessary swapin in khugepaged") > > > > on test machine: vm-kbuild-1G: 2 threads qemu-system-x86_64 -enable-kvm -cpu Haswell,+smep,+smap with 1G memory > > > > caused below changes: > > > > > > > > [ 21.116124] ====================================================== > > [ 21.116124] [ INFO: possible circular locking dependency detected ] > > [ 21.116127] 4.6.0-rc5-00302-g409ca71 #1 Not tainted > > [ 21.116127] ------------------------------------------------------- > > [ 21.116128] udevadm/221 is trying to acquire lock: > > [ 21.116138] (&mm->mmap_sem){++++++}, at: [] __might_fault+0x83/0x150 > > [ 21.116138] > > [ 21.116138] but task is already holding lock: > > [ 21.116144] (s_active#12){++++.+}, at: [] kernfs_fop_write+0x8e/0x250 > > [ 21.116144] > > [ 21.116144] which lock already depends on the new lock. > > [ 21.116144] > > [ 21.116145] the existing dependency chain (in reverse order) is: > > [ 21.116148] > > [ 21.116148] -> #2 (s_active#12){++++.+}: > > [ 21.116152] [] lock_acquire+0xac/0x180 > > [ 21.116155] [] __kernfs_remove+0x2da/0x410 > > [ 21.116158] [] kernfs_remove_by_name_ns+0x40/0x90 > > [ 21.116160] [] sysfs_remove_file_ns+0x2b/0x70 > > [ 21.116164] [] device_del+0x166/0x320 > > [ 21.116166] [] device_destroy+0x3c/0x50 > > [ 21.116170] [] cpuid_class_cpu_callback+0x51/0x70 > > [ 21.116173] [] notifier_call_chain+0x59/0x190 > > [ 21.116177] [] __raw_notifier_call_chain+0x9/0x10 > > [ 21.116180] [] __cpu_notify+0x40/0x90 > > [ 21.116182] [] cpu_notify_nofail+0x10/0x30 > > [ 21.116185] [] notify_dead+0x27/0x1e0 > > [ 21.116187] [] cpuhp_down_callbacks+0x93/0x190 > > [ 21.116192] [] _cpu_down+0xc2/0x1e0 > > [ 21.116194] [] do_cpu_down+0x37/0x50 > > [ 21.116197] [] cpu_down+0xb/0x10 > > [ 21.116201] [] _debug_hotplug_cpu+0x7d/0xd0 > > [ 21.116205] [] debug_hotplug_cpu+0xd/0x11 > > [ 21.116208] [] do_one_initcall+0x138/0x1cf > > [ 21.116211] [] kernel_init_freeable+0x24d/0x2de > > [ 21.116214] [] kernel_init+0xa/0x120 > > [ 21.116217] [] ret_from_fork+0x22/0x50 > > [ 21.116221] > > [ 21.116221] -> #1 (cpu_hotplug.lock#2){+.+.+.}: > > [ 21.116223] [] lock_acquire+0xac/0x180 > > [ 21.116226] [] mutex_lock_nested+0x71/0x4c0 > > [ 21.116228] [] get_online_cpus+0x66/0x80 > > [ 21.116232] [] sum_vm_event+0x23/0x1b0 > > [ 21.116236] [] collapse_huge_page+0x118/0x10b0 > > [ 21.116238] [] khugepaged+0x55d/0xe80 > > [ 21.116240] [] kthread+0x134/0x1a0 > > [ 21.116242] [] ret_from_fork+0x22/0x50 > > [ 21.116244] > > [ 21.116244] -> #0 (&mm->mmap_sem){++++++}: > > [ 21.116246] [] __lock_acquire+0x2861/0x31f0 > > [ 21.116248] [] lock_acquire+0xac/0x180 > > [ 21.116251] [] __might_fault+0xbe/0x150 > > [ 21.116253] [] kernfs_fop_write+0xaf/0x250 > > [ 21.116256] [] __vfs_write+0x43/0x1a0 > > [ 21.116258] [] vfs_write+0xda/0x240 > > [ 21.116260] [] SyS_write+0x44/0xa0 > > [ 21.116263] [] entry_SYSCALL_64_fastpath+0x1f/0xbd > > [ 21.116264] > > [ 21.116264] other info that might help us debug this: > > [ 21.116264] > > [ 21.116268] Chain exists of: > > [ 21.116268] &mm->mmap_sem --> cpu_hotplug.lock#2 --> s_active#12 > > [ 21.116268] > > [ 21.116268] Possible unsafe locking scenario: > > [ 21.116268] > > [ 21.116269] CPU0 CPU1 > > [ 21.116269] ---- ---- > > [ 21.116270] lock(s_active#12); > > [ 21.116271] lock(cpu_hotplug.lock#2); > > [ 21.116272] lock(s_active#12); > > [ 21.116273] lock(&mm->mmap_sem); > > [ 21.116274] > > [ 21.116274] *** DEADLOCK *** > > [ 21.116274] > > [ 21.116274] *** DEADLOCK *** > > [ 21.116274] > > [ 21.116274] 3 locks held by udevadm/221: > > [ 21.116278] #0: (sb_writers#3){.+.+.+}, at: [] __sb_start_write+0x6d/0x120 > > [ 21.116280] #1: (&of->mutex){+.+.+.}, at: [] kernfs_fop_write+0x86/0x250 > > [ 21.116282] #2: (s_active#12){++++.+}, at: [] kernfs_fop_write+0x8e/0x250 > > [ 21.116283] > > [ 21.116283] stack backtrace: > > [ 21.116283] > > [ 21.116283] stack backtrace: > > [ 21.116284] CPU: 1 PID: 221 Comm: udevadm Not tainted 4.6.0-rc5-00302-g409ca71 #1 > > [ 21.116287] ffff88003f698000 ffff88003f077bf0 ffffffff81444ef3 0000000000000011 > > [ 21.116288] ffffffff84bdd8f0 ffffffff84bf2630 ffff88003f077c40 ffffffff81173e91 > > [ 21.116290] 0000000000000000 ffffffff84fbdbc0 00ff88003f077c40 ffff88003f698bb8 > > [ 21.116290] Call Trace: > > [ 21.116293] [] dump_stack+0x86/0xd3 > > [ 21.116294] [] print_circular_bug+0x221/0x360 > > [ 21.116296] [] __lock_acquire+0x2861/0x31f0 > > [ 21.116297] [] lock_acquire+0xac/0x180 > > [ 21.116299] [] ? __might_fault+0x83/0x150 > > [ 21.116300] [] __might_fault+0xbe/0x150 > > [ 21.116302] [] ? __might_fault+0x83/0x150 > > [ 21.116303] [] kernfs_fop_write+0xaf/0x250 > > [ 21.116304] [] __vfs_write+0x43/0x1a0 > > [ 21.116306] [] ? update_fast_ctr+0x1d/0x80 > > [ 21.116308] [] ? percpu_down_read+0x57/0xa0 > > [ 21.116310] [] ? __sb_start_write+0x6d/0x120 > > [ 21.116311] [] ? __sb_start_write+0x6d/0x120 > > [ 21.116312] [] vfs_write+0xda/0x240 > > [ 21.116314] [] SyS_write+0x44/0xa0 > > [ 21.116315] [] entry_SYSCALL_64_fastpath+0x1f/0xbd > > If I read this correctly (I'm not sure about this), we shouldn't call > sum_vm_event() under mmap_sem. > > BTW, we do need mmap_sem for swapin, but there's no need for exclusive > one. It can be too expensive to do I/O with down_write(mmap_sem). > > Ebru, could look how to move sum_vm_event() outside mmap_sem and probably > have down_read(mmap_sem) during swapin? I don't have time for this right > now. I started to work on down_read(mmap_sem) a few days ago. But firstly, I'll take this issue before down_read. kind regards.