From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pf0-f169.google.com ([209.85.192.169]:34715 "EHLO mail-pf0-f169.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750801AbcCYAqw (ORCPT ); Thu, 24 Mar 2016 20:46:52 -0400 Received: by mail-pf0-f169.google.com with SMTP id x3so70584544pfb.1 for ; Thu, 24 Mar 2016 17:46:52 -0700 (PDT) Date: Thu, 24 Mar 2016 17:46:48 -0700 From: Yauhen Kharuzhy To: linux-btrfs@vger.kernel.org Subject: Lockdep warning at device replace finishing Message-ID: <20160325004648.GA1179@jek-Latitude-E7440> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: linux-btrfs-owner@vger.kernel.org List-ID: Hi. Can anybody point me to possible cause of this lockdep warning and say if it is dangerous in reality? It appeared when I started replacing from the missing drive ('btrfs replace start ). My locking-fu seems to be too weak to resolve this by myself. I use 4.4.5 kernel with Anand's global hotspare patches applied plus patch 'Btrfs: fix lockdep deadlock warning due to dev_replace' was applied also. This is reproduced rarely, so I cannot check this for sure with latest kernels. [ 1231.862717] BTRFS info (device sdf): dev_replace from /dev/sdf (devid 4) to /dev/sdh finished [ 1231.881938] [ 1231.888624] ====================================================== [ 1231.943857] [ INFO: possible circular locking dependency detected ] [ 1231.953686] 4.4.5-scst31x+ #16 Not tainted [ 1231.954755] ------------------------------------------------------- [ 1232.031044] btrfs/4453 is trying to acquire lock: [ 1232.031966] (sb_writers){.+.+.+}, at: [] __sb_start_write+0xb7/0xf0 [ 1232.033945] [ 1232.033945] but task is already holding lock: [ 1232.035238] (&fs_info->chunk_mutex){+.+.+.}, at: [] btrfs_dev_replace_finishing+0x121/0x9b0 [btrfs] [ 1232.037577] [ 1232.037577] which lock already depends on the new lock. [ 1232.037577] [ 1232.039352] [ 1232.039352] the existing dependency chain (in reverse order) is: [ 1232.040817] -> #4 (&fs_info->chunk_mutex){+.+.+.}: [ 1232.042429] [] lock_acquire+0xce/0x1e0 [ 1232.090783] [] mutex_lock_nested+0x69/0x3c0 [ 1232.093226] [] btrfs_dev_replace_finishing+0x121/0x9b0 [btrfs] [ 1232.094913] [] btrfs_dev_replace_start+0x390/0x560 [btrfs] [ 1232.096798] [] btrfs_ioctl+0x1e4c/0x2c00 [btrfs] [ 1232.098396] [] do_vfs_ioctl+0x2e2/0x530 [ 1232.099596] [] SyS_ioctl+0x79/0x90 [ 1232.100725] [] entry_SYSCALL_64_fastpath+0x16/0x7a [ 1232.102031] -> #3 (&fs_devs->device_list_mutex){+.+.+.}: [ 1232.103721] [] lock_acquire+0xce/0x1e0 [ 1232.104888] [] mutex_lock_nested+0x69/0x3c0 [ 1232.106106] [] btrfs_show_devname+0x36/0x210 [btrfs] [ 1232.107444] [] show_vfsmnt+0x49/0x150 [ 1232.108603] [] m_show+0x17/0x20 [ 1232.109696] [] seq_read+0x2d8/0x3b0 [ 1232.111160] [] __vfs_read+0x28/0xd0 [ 1232.113660] [] vfs_read+0x86/0x130 [ 1232.115847] [] SyS_read+0x49/0xa0 [ 1232.116963] [] entry_SYSCALL_64_fastpath+0x16/0x7a [ 1232.118268] -> #2 (namespace_sem){+++++.}: [ 1232.119795] [] lock_acquire+0xce/0x1e0 [ 1232.125023] [] down_write+0x49/0x80 [ 1232.150746] [] lock_mount+0x43/0x1c0 [ 1232.151914] [] do_add_mount+0x23/0xd0 [ 1232.157188] [] do_mount+0x27b/0xe30 [ 1232.158416] [] SyS_mount+0x8c/0xd0 [ 1232.159622] [] entry_SYSCALL_64_fastpath+0x16/0x7a [ 1232.189142] -> #1 (&sb->s_type->i_mutex_key#5){+.+.+.}: [ 1232.192923] [] lock_acquire+0xce/0x1e0 [ 1232.211173] [] mutex_lock_nested+0x69/0x3c0 [ 1232.223543] [] path_openat+0x468/0x1360 [ 1232.237187] [] do_filp_open+0x7e/0xe0 [ 1232.239007] [] do_sys_open+0x12b/0x210 [ 1232.241578] [] SyS_open+0x1e/0x20 [ 1232.242711] [] entry_SYSCALL_64_fastpath+0x16/0x7a [ 1232.244018] -> #0 (sb_writers){.+.+.+}: [ 1232.245504] [] __lock_acquire+0x17ba/0x1ae0 [ 1232.246742] [] lock_acquire+0xce/0x1e0 [ 1232.247934] [] percpu_down_read+0x4f/0xa0 [ 1232.260955] [] __sb_start_write+0xb7/0xf0 [ 1232.262264] [] mnt_want_write+0x24/0x50 [ 1232.263460] [] path_openat+0xd32/0x1360 [ 1232.264644] [] do_filp_open+0x7e/0xe0 [ 1232.267248] [] file_open_name+0xe4/0x130 [ 1232.268454] [] filp_open+0x33/0x60 [ 1232.269584] [] update_dev_time+0x16/0x40 [btrfs] [ 1232.314209] [] btrfs_scratch_superblocks+0x4e/0x90 [btrfs] [ 1232.315874] [] btrfs_rm_dev_replace_remove_srcdev+0xa2/0xc0 [btrfs] [ 1232.317610] [] btrfs_dev_replace_finishing+0x490/0x9b0 [btrfs] [ 1232.319291] [] btrfs_dev_replace_start+0x390/0x560 [btrfs] [ 1232.325306] [] btrfs_ioctl+0x1e4c/0x2c00 [btrfs] [ 1232.326612] [] do_vfs_ioctl+0x2e2/0x530 [ 1232.327808] [] SyS_ioctl+0x79/0x90 [ 1232.328935] [] entry_SYSCALL_64_fastpath+0x16/0x7a [ 1232.330243] [ 1232.330243] other info that might help us debug this: [ 1232.330243] [ 1232.414976] Chain exists of: sb_writers --> &fs_devs->device_list_mutex --> &fs_info->chunk_mutex [ 1232.417570] Possible unsafe locking scenario: [ 1232.417570] [ 1232.423489] CPU0 CPU1 [ 1232.424379] ---- ---- [ 1232.425273] lock(&fs_info->chunk_mutex); [ 1232.426341] lock(&fs_devs->device_list_mutex); [ 1232.427786] lock(&fs_info->chunk_mutex); [ 1232.429158] lock(sb_writers); [ 1232.430097] [ 1232.430097] *** DEADLOCK *** [ 1232.430097] [ 1232.466399] 4 locks held by btrfs/4453: [ 1232.467217] #0: (&fs_info->dev_replace.lock_finishing_cancel_unmount){+.+.+.}, at: [] btrfs_dev_replace_finishing+0x41/0x9b0 [btrfs] [ 1232.470045] #1: (uuid_mutex){+.+.+.}, at: [] btrfs_dev_replace_finishing+0xf3/0x9b0 [btrfs] [ 1232.521358] #2: (&fs_devs->device_list_mutex){+.+.+.}, at: [] btrfs_dev_replace_finishing+0x10c/0x9b0 [btrfs] [ 1232.523967] #3: (&fs_info->chunk_mutex){+.+.+.}, at: [] btrfs_dev_replace_finishing+0x121/0x9b0 [btrfs] [ 1232.526488] [ 1232.526488] stack backtrace: [ 1232.527578] CPU: 1 PID: 4453 Comm: btrfs Not tainted 4.4.5-scst31x+ #16 [ 1232.528722] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006 [ 1232.530296] 0000000000000000 ffff880062583800 ffffffff8134f233 ffffffff82614ca0 [ 1232.583206] ffffffff826403e0 ffff880062583840 ffffffff810d656c ffff8800625838b0 [ 1232.585123] ffff88006335acc0 ffff88006335ace8 0000000000000003 0000000000000004 [ 1232.672030] Call Trace: [ 1232.672679] [] dump_stack+0x85/0xc2 [ 1232.673658] [] print_circular_bug+0x1ec/0x260 [ 1232.674880] [] __lock_acquire+0x17ba/0x1ae0 [ 1232.676073] [] ? debug_lockdep_rcu_enabled+0x1d/0x20 [ 1232.677371] [] ? __raw_callee_save___pv_queued_spin_unlock+0x11/0x20 [ 1232.679149] [] lock_acquire+0xce/0x1e0 [ 1232.680301] [] ? __sb_start_write+0xb7/0xf0 [ 1232.681503] [] percpu_down_read+0x4f/0xa0 [ 1232.682680] [] ? __sb_start_write+0xb7/0xf0 [ 1232.683885] [] __sb_start_write+0xb7/0xf0 [ 1232.685056] [] mnt_want_write+0x24/0x50 [ 1232.686202] [] path_openat+0xd32/0x1360 [ 1232.687342] [] ? debug_check_no_obj_freed+0x214/0x230 [ 1232.688643] [] do_filp_open+0x7e/0xe0 [ 1232.689744] [] ? rcu_read_lock_sched_held+0x6d/0x80 [ 1232.691622] [] ? kmem_cache_alloc+0x18a/0x2c0 [ 1232.766936] [] file_open_name+0xe4/0x130 [ 1232.767963] [] filp_open+0x33/0x60 [ 1232.775605] [] update_dev_time+0x16/0x40 [btrfs] [ 1232.779776] [] btrfs_scratch_superblocks+0x4e/0x90 [btrfs] [ 1232.801170] [] btrfs_rm_dev_replace_remove_srcdev+0xa2/0xc0 [btrfs] [ 1232.830890] [] btrfs_dev_replace_finishing+0x490/0x9b0 [btrfs] [ 1232.832386] [] ? btrfs_dev_replace_finishing+0x1fc/0x9b0 [btrfs] [ 1232.833894] [] btrfs_dev_replace_start+0x390/0x560 [btrfs] [ 1232.835129] [] btrfs_ioctl+0x1e4c/0x2c00 [btrfs] [ 1232.836244] [] ? __raw_callee_save___pv_queued_spin_unlock+0x11/0x20 [ 1232.840742] [] ? handle_mm_fault+0xd76/0x1ac0 [ 1232.841817] [] ? do_raw_spin_unlock+0x50/0xc0 [ 1232.842917] [] ? _raw_spin_unlock+0x27/0x40 [ 1232.843968] [] ? handle_mm_fault+0xd76/0x1ac0 [ 1232.845040] [] ? debug_lockdep_rcu_enabled+0x1d/0x20 [ 1232.846196] [] do_vfs_ioctl+0x2e2/0x530 [ 1232.847207] [] ? up_read+0x1f/0x40 [ 1232.848156] [] ? debug_lockdep_rcu_enabled+0x1d/0x20 [ 1232.870945] [] ? __fget_light+0x66/0x90 [ 1232.877419] [] SyS_ioctl+0x79/0x90 [ 1232.878386] [] entry_SYSCALL_64_fastpath+0x16/0x7a