From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ob0-f174.google.com ([209.85.214.174]:35812 "EHLO mail-ob0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755021Ab2J2UGN (ORCPT ); Mon, 29 Oct 2012 16:06:13 -0400 Received: by mail-ob0-f174.google.com with SMTP id uo13so4996769obb.19 for ; Mon, 29 Oct 2012 13:06:13 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <1351333721-3220-1-git-send-email-bo.li.liu@oracle.com> References: <1351333721-3220-1-git-send-email-bo.li.liu@oracle.com> Date: Mon, 29 Oct 2012 15:06:12 -0500 Message-ID: Subject: Re: [PATCH 1/2 v4] Btrfs: snapshot-aware defrag From: Mitch Harder To: Liu Bo Cc: linux-btrfs@vger.kernel.org Content-Type: text/plain; charset=ISO-8859-1 Sender: linux-btrfs-owner@vger.kernel.org List-ID: On Sat, Oct 27, 2012 at 5:28 AM, Liu Bo wrote: > This comes from one of btrfs's project ideas, > As we defragment files, we break any sharing from other snapshots. > The balancing code will preserve the sharing, and defrag needs to grow this > as well. > > Now we're able to fill the blank with this patch, in which we make full use of > backref walking stuff. > > Here is the basic idea, > o set the writeback ranges started by defragment with flag EXTENT_DEFRAG > o at endio, after we finish updating fs tree, we use backref walking to find > all parents of the ranges and re-link them with the new COWed file layout by > adding corresponding backrefs. > > Originally patch by Li Zefan > Signed-off-by: Liu Bo > --- > v3->v4: > - fix duplicated refs bugs detected by mounting with autodefrag, thanks > for the bug report from Mitch and Chris. > I'm picking up many WARN_ON messages while testing this patch. I'm testing a snapshot script that uses kernel git sources along with some git manipulations. The kernel is a 3.6.4 kernel merged with the latest for-linus branch. I mounted with -o compress-force=lzo,autodefrag. I also have the second patch in this set (Btrfs: make snapshot-aware defrag as a mount option). However, I did not mount with 'snap_aware_defrag'. I did not find any corrupted data, and the partition passes a btrfsck without error after these warnings were observed. Here's a summary of the WARN_ON messages: $ cat local/dmesg-3.6.4-x+ | grep "WARNING:" [ 610.407561] WARNING: at fs/btrfs/inode.c:7779 btrfs_destroy_inode+0x2ac/0x2e6 [btrfs]() [ 610.407757] WARNING: at fs/btrfs/inode.c:7780 btrfs_destroy_inode+0x296/0x2e6 [btrfs]() [ 610.407929] WARNING: at fs/btrfs/inode.c:7782 btrfs_destroy_inode+0x26a/0x2e6 [btrfs]() [ 661.211849] WARNING: at fs/btrfs/inode.c:7779 btrfs_destroy_inode+0x2ac/0x2e6 [btrfs]() [ 661.212004] WARNING: at fs/btrfs/inode.c:7780 btrfs_destroy_inode+0x296/0x2e6 [btrfs]() [ 661.212236] WARNING: at fs/btrfs/inode.c:7782 btrfs_destroy_inode+0x26a/0x2e6 [btrfs]() [ 719.882942] WARNING: at fs/btrfs/inode.c:7779 btrfs_destroy_inode+0x2ac/0x2e6 [btrfs]() [ 719.883112] WARNING: at fs/btrfs/inode.c:7780 btrfs_destroy_inode+0x296/0x2e6 [btrfs]() [ 719.883232] WARNING: at fs/btrfs/inode.c:7782 btrfs_destroy_inode+0x26a/0x2e6 [btrfs]() [ 786.978869] WARNING: at fs/btrfs/inode.c:7779 btrfs_destroy_inode+0x2ac/0x2e6 [btrfs]() [ 786.979003] WARNING: at fs/btrfs/inode.c:7780 btrfs_destroy_inode+0x296/0x2e6 [btrfs]() [ 786.979140] WARNING: at fs/btrfs/inode.c:7782 btrfs_destroy_inode+0x26a/0x2e6 [btrfs]() [ 845.605176] WARNING: at fs/btrfs/inode.c:7779 btrfs_destroy_inode+0x2ac/0x2e6 [btrfs]() [ 845.605323] WARNING: at fs/btrfs/inode.c:7780 btrfs_destroy_inode+0x296/0x2e6 [btrfs]() [ 845.605445] WARNING: at fs/btrfs/inode.c:7782 btrfs_destroy_inode+0x26a/0x2e6 [btrfs]() [ 912.300307] WARNING: at fs/btrfs/inode.c:7779 btrfs_destroy_inode+0x2ac/0x2e6 [btrfs]() [ 912.300454] WARNING: at fs/btrfs/inode.c:7780 btrfs_destroy_inode+0x296/0x2e6 [btrfs]() [ 912.300577] WARNING: at fs/btrfs/inode.c:7782 btrfs_destroy_inode+0x26a/0x2e6 [btrfs]() [ 968.835873] WARNING: at fs/btrfs/inode.c:7779 btrfs_destroy_inode+0x2ac/0x2e6 [btrfs]() [ 968.836032] WARNING: at fs/btrfs/inode.c:7780 btrfs_destroy_inode+0x296/0x2e6 [btrfs]() [ 968.836156] WARNING: at fs/btrfs/inode.c:7782 btrfs_destroy_inode+0x26a/0x2e6 [btrfs]() [ 1023.778160] WARNING: at fs/btrfs/inode.c:7779 btrfs_destroy_inode+0x2ac/0x2e6 [btrfs]() [ 1023.778316] WARNING: at fs/btrfs/inode.c:7780 btrfs_destroy_inode+0x296/0x2e6 [btrfs]() [ 1023.778435] WARNING: at fs/btrfs/inode.c:7782 btrfs_destroy_inode+0x26a/0x2e6 [btrfs]() [ 1064.342768] WARNING: at fs/btrfs/inode.c:7779 btrfs_destroy_inode+0x2ac/0x2e6 [btrfs]() [ 1064.342914] WARNING: at fs/btrfs/inode.c:7780 btrfs_destroy_inode+0x296/0x2e6 [btrfs]() [ 1064.343112] WARNING: at fs/btrfs/inode.c:7782 btrfs_destroy_inode+0x26a/0x2e6 [btrfs]() [ 1177.892047] WARNING: at fs/btrfs/inode.c:7779 btrfs_destroy_inode+0x2ac/0x2e6 [btrfs]() [ 1177.892189] WARNING: at fs/btrfs/inode.c:7780 btrfs_destroy_inode+0x296/0x2e6 [btrfs]() [ 1177.892312] WARNING: at fs/btrfs/inode.c:7782 btrfs_destroy_inode+0x26a/0x2e6 [btrfs]() [ 1281.951715] WARNING: at fs/btrfs/inode.c:7779 btrfs_destroy_inode+0x2ac/0x2e6 [btrfs]() [ 1281.951857] WARNING: at fs/btrfs/inode.c:7780 btrfs_destroy_inode+0x296/0x2e6 [btrfs]() [ 1281.951978] WARNING: at fs/btrfs/inode.c:7782 btrfs_destroy_inode+0x26a/0x2e6 [btrfs]() [ 1282.804376] WARNING: at fs/btrfs/inode.c:7779 btrfs_destroy_inode+0x2ac/0x2e6 [btrfs]() [ 1282.804524] WARNING: at fs/btrfs/inode.c:7780 btrfs_destroy_inode+0x296/0x2e6 [btrfs]() [ 1282.804645] WARNING: at fs/btrfs/inode.c:7782 btrfs_destroy_inode+0x26a/0x2e6 [btrfs]() [ 1351.187114] WARNING: at fs/btrfs/inode.c:7779 btrfs_destroy_inode+0x2ac/0x2e6 [btrfs]() [ 1351.187263] WARNING: at fs/btrfs/inode.c:7780 btrfs_destroy_inode+0x296/0x2e6 [btrfs]() [ 1351.187391] WARNING: at fs/btrfs/inode.c:7782 btrfs_destroy_inode+0x26a/0x2e6 [btrfs]() [ 1399.022919] WARNING: at fs/btrfs/inode.c:7779 btrfs_destroy_inode+0x2ac/0x2e6 [btrfs]() [ 1399.023086] WARNING: at fs/btrfs/inode.c:7780 btrfs_destroy_inode+0x296/0x2e6 [btrfs]() [ 1399.023206] WARNING: at fs/btrfs/inode.c:7782 btrfs_destroy_inode+0x26a/0x2e6 [btrfs]() [ 1483.724756] WARNING: at fs/btrfs/inode.c:7779 btrfs_destroy_inode+0x2ac/0x2e6 [btrfs]() [ 1483.724895] WARNING: at fs/btrfs/inode.c:7780 btrfs_destroy_inode+0x296/0x2e6 [btrfs]() [ 1483.725031] WARNING: at fs/btrfs/inode.c:7782 btrfs_destroy_inode+0x26a/0x2e6 [btrfs]() [ 1523.081643] WARNING: at fs/btrfs/inode.c:7779 btrfs_destroy_inode+0x2ac/0x2e6 [btrfs]() [ 1523.081792] WARNING: at fs/btrfs/inode.c:7780 btrfs_destroy_inode+0x296/0x2e6 [btrfs]() [ 1523.081922] WARNING: at fs/btrfs/inode.c:7782 btrfs_destroy_inode+0x26a/0x2e6 [btrfs]() [ 1555.993160] WARNING: at fs/btrfs/inode.c:7779 btrfs_destroy_inode+0x2ac/0x2e6 [btrfs]() [ 1555.993299] WARNING: at fs/btrfs/inode.c:7780 btrfs_destroy_inode+0x296/0x2e6 [btrfs]() [ 1555.993420] WARNING: at fs/btrfs/inode.c:7782 btrfs_destroy_inode+0x26a/0x2e6 [btrfs]() [ 1556.252181] WARNING: at fs/btrfs/inode.c:7779 btrfs_destroy_inode+0x2ac/0x2e6 [btrfs]() [ 1556.252312] WARNING: at fs/btrfs/inode.c:7780 btrfs_destroy_inode+0x296/0x2e6 [btrfs]() [ 1556.252427] WARNING: at fs/btrfs/inode.c:7782 btrfs_destroy_inode+0x26a/0x2e6 [btrfs]() [ 1557.218561] WARNING: at fs/btrfs/inode.c:7779 btrfs_destroy_inode+0x2ac/0x2e6 [btrfs]() [ 1557.218691] WARNING: at fs/btrfs/inode.c:7780 btrfs_destroy_inode+0x296/0x2e6 [btrfs]() [ 1557.218805] WARNING: at fs/btrfs/inode.c:7782 btrfs_destroy_inode+0x26a/0x2e6 [btrfs]() [ 1557.895069] WARNING: at fs/btrfs/extent-tree.c:4417 btrfs_free_block_groups+0x2ad/0x34c [btrfs]() [ 1557.895261] WARNING: at fs/btrfs/extent-tree.c:4418 btrfs_free_block_groups+0x347/0x34c [btrfs]() [ 1557.895435] WARNING: at fs/btrfs/extent-tree.c:7687 btrfs_free_block_groups+0x23e/0x34c [btrfs]() [ 1557.895619] WARNING: at fs/btrfs/extent-tree.c:7687 btrfs_free_block_groups+0x23e/0x34c [btrfs]() Here's a snip of the code that has the WARN_ONs in inode.c: void btrfs_destroy_inode(struct inode *inode) { struct btrfs_ordered_extent *ordered; struct btrfs_root *root = BTRFS_I(inode)->root; WARN_ON(!hlist_empty(&inode->i_dentry)); WARN_ON(inode->i_data.nrpages); WARN_ON(BTRFS_I(inode)->outstanding_extents); // line 7779 WARN_ON(BTRFS_I(inode)->reserved_extents); // line 7780 WARN_ON(BTRFS_I(inode)->delalloc_bytes); // line 7781 WARN_ON(BTRFS_I(inode)->csum_bytes); // line 7782 Here's the dmesg output for just the last set of errors (the other WARN_ONs had similar backtraces): [ 1557.218529] ------------[ cut here ]------------ [ 1557.218561] WARNING: at fs/btrfs/inode.c:7779 btrfs_destroy_inode+0x2ac/0x2e6 [btrfs]() [ 1557.218564] Hardware name: OptiPlex 745 [ 1557.218566] Modules linked in: iTCO_wdt iTCO_vendor_support lpc_ich mfd_core xts gf128mul ablk_helper cryptd aes_x86_64 sha256_generic btrfs libcrc32c [ 1557.218582] Pid: 5728, comm: umount Tainted: G W 3.6.4-x+ #3 [ 1557.218584] Call Trace: [ 1557.218592] [] warn_slowpath_common+0x74/0xa2 [ 1557.218596] [] warn_slowpath_null+0x1a/0x1c [ 1557.218612] [] btrfs_destroy_inode+0x2ac/0x2e6 [btrfs] [ 1557.218618] [] destroy_inode+0x3c/0x5f [ 1557.218622] [] evict+0x122/0x1ac [ 1557.218626] [] dispose_list+0x3a/0x49 [ 1557.218629] [] evict_inodes+0x106/0x115 [ 1557.218635] [] generic_shutdown_super+0x4c/0xdc [ 1557.218639] [] kill_anon_super+0x16/0x24 [ 1557.218650] [] btrfs_kill_super+0x1a/0x8f [btrfs] [ 1557.218654] [] deactivate_locked_super+0x34/0x5a [ 1557.218658] [] deactivate_super+0x4e/0x66 [ 1557.218663] [] mntput_no_expire+0xf7/0x14d [ 1557.218666] [] sys_umount+0x63/0x37a [ 1557.218672] [] system_call_fastpath+0x16/0x1b [ 1557.218674] ---[ end trace 29a44c3c46b1a1b9 ]--- [ 1557.218676] ------------[ cut here ]------------ [ 1557.218691] WARNING: at fs/btrfs/inode.c:7780 btrfs_destroy_inode+0x296/0x2e6 [btrfs]() [ 1557.218693] Hardware name: OptiPlex 745 [ 1557.218694] Modules linked in: iTCO_wdt iTCO_vendor_support lpc_ich mfd_core xts gf128mul ablk_helper cryptd aes_x86_64 sha256_generic btrfs libcrc32c [ 1557.218708] Pid: 5728, comm: umount Tainted: G W 3.6.4-x+ #3 [ 1557.218709] Call Trace: [ 1557.218714] [] warn_slowpath_common+0x74/0xa2 [ 1557.218717] [] warn_slowpath_null+0x1a/0x1c [ 1557.218733] [] btrfs_destroy_inode+0x296/0x2e6 [btrfs] [ 1557.218737] [] destroy_inode+0x3c/0x5f [ 1557.218741] [] evict+0x122/0x1ac [ 1557.218744] [] dispose_list+0x3a/0x49 [ 1557.218748] [] evict_inodes+0x106/0x115 [ 1557.218752] [] generic_shutdown_super+0x4c/0xdc [ 1557.218757] [] kill_anon_super+0x16/0x24 [ 1557.218767] [] btrfs_kill_super+0x1a/0x8f [btrfs] [ 1557.218771] [] deactivate_locked_super+0x34/0x5a [ 1557.218775] [] deactivate_super+0x4e/0x66 [ 1557.218779] [] mntput_no_expire+0xf7/0x14d [ 1557.218783] [] sys_umount+0x63/0x37a [ 1557.218787] [] system_call_fastpath+0x16/0x1b [ 1557.218789] ---[ end trace 29a44c3c46b1a1ba ]--- [ 1557.218791] ------------[ cut here ]------------ [ 1557.218805] WARNING: at fs/btrfs/inode.c:7782 btrfs_destroy_inode+0x26a/0x2e6 [btrfs]() [ 1557.218807] Hardware name: OptiPlex 745 [ 1557.218809] Modules linked in: iTCO_wdt iTCO_vendor_support lpc_ich mfd_core xts gf128mul ablk_helper cryptd aes_x86_64 sha256_generic btrfs libcrc32c [ 1557.218822] Pid: 5728, comm: umount Tainted: G W 3.6.4-x+ #3 [ 1557.218823] Call Trace: [ 1557.218827] [] warn_slowpath_common+0x74/0xa2 [ 1557.218831] [] warn_slowpath_null+0x1a/0x1c [ 1557.218847] [] btrfs_destroy_inode+0x26a/0x2e6 [btrfs] [ 1557.218851] [] destroy_inode+0x3c/0x5f [ 1557.218854] [] evict+0x122/0x1ac [ 1557.218858] [] dispose_list+0x3a/0x49 [ 1557.218862] [] evict_inodes+0x106/0x115 [ 1557.218866] [] generic_shutdown_super+0x4c/0xdc [ 1557.218870] [] kill_anon_super+0x16/0x24 [ 1557.218881] [] btrfs_kill_super+0x1a/0x8f [btrfs] [ 1557.218885] [] deactivate_locked_super+0x34/0x5a [ 1557.218889] [] deactivate_super+0x4e/0x66 [ 1557.218893] [] mntput_no_expire+0xf7/0x14d [ 1557.218897] [] sys_umount+0x63/0x37a [ 1557.218901] [] system_call_fastpath+0x16/0x1b [ 1557.218903] ---[ end trace 29a44c3c46b1a1bb ]--- [ 1557.895037] ------------[ cut here ]------------ [ 1557.895069] WARNING: at fs/btrfs/extent-tree.c:4417 btrfs_free_block_groups+0x2ad/0x34c [btrfs]() [ 1557.895073] Hardware name: OptiPlex 745 [ 1557.895075] Modules linked in: iTCO_wdt iTCO_vendor_support lpc_ich mfd_core xts gf128mul ablk_helper cryptd aes_x86_64 sha256_generic btrfs libcrc32c [ 1557.895097] Pid: 5728, comm: umount Tainted: G W 3.6.4-x+ #3 [ 1557.895100] Call Trace: [ 1557.895112] [] warn_slowpath_common+0x74/0xa2 [ 1557.895118] [] warn_slowpath_null+0x1a/0x1c [ 1557.895136] [] btrfs_free_block_groups+0x2ad/0x34c [btrfs] [ 1557.895157] [] close_ctree+0x1bc/0x2f1 [btrfs] [ 1557.895165] [] ? evict_inodes+0x106/0x115 [ 1557.895179] [] btrfs_put_super+0x19/0x1b [btrfs] [ 1557.895187] [] generic_shutdown_super+0x5b/0xdc [ 1557.895193] [] kill_anon_super+0x16/0x24 [ 1557.895208] [] btrfs_kill_super+0x1a/0x8f [btrfs] [ 1557.895214] [] deactivate_locked_super+0x34/0x5a [ 1557.895220] [] deactivate_super+0x4e/0x66 [ 1557.895226] [] mntput_no_expire+0xf7/0x14d [ 1557.895231] [] sys_umount+0x63/0x37a [ 1557.895239] [] system_call_fastpath+0x16/0x1b [ 1557.895243] ---[ end trace 29a44c3c46b1a1bc ]--- [ 1557.895245] ------------[ cut here ]------------ [ 1557.895261] WARNING: at fs/btrfs/extent-tree.c:4418 btrfs_free_block_groups+0x347/0x34c [btrfs]() [ 1557.895264] Hardware name: OptiPlex 745 [ 1557.895266] Modules linked in: iTCO_wdt iTCO_vendor_support lpc_ich mfd_core xts gf128mul ablk_helper cryptd aes_x86_64 sha256_generic btrfs libcrc32c [ 1557.895285] Pid: 5728, comm: umount Tainted: G W 3.6.4-x+ #3 [ 1557.895287] Call Trace: [ 1557.895294] [] warn_slowpath_common+0x74/0xa2 [ 1557.895299] [] warn_slowpath_null+0x1a/0x1c [ 1557.895317] [] btrfs_free_block_groups+0x347/0x34c [btrfs] [ 1557.895337] [] close_ctree+0x1bc/0x2f1 [btrfs] [ 1557.895343] [] ? evict_inodes+0x106/0x115 [ 1557.895358] [] btrfs_put_super+0x19/0x1b [btrfs] [ 1557.895364] [] generic_shutdown_super+0x5b/0xdc [ 1557.895370] [] kill_anon_super+0x16/0x24 [ 1557.895385] [] btrfs_kill_super+0x1a/0x8f [btrfs] [ 1557.895391] [] deactivate_locked_super+0x34/0x5a [ 1557.895397] [] deactivate_super+0x4e/0x66 [ 1557.895402] [] mntput_no_expire+0xf7/0x14d [ 1557.895407] [] sys_umount+0x63/0x37a [ 1557.895413] [] system_call_fastpath+0x16/0x1b [ 1557.895417] ---[ end trace 29a44c3c46b1a1bd ]--- [ 1557.895419] ------------[ cut here ]------------ [ 1557.895435] WARNING: at fs/btrfs/extent-tree.c:7687 btrfs_free_block_groups+0x23e/0x34c [btrfs]() [ 1557.895438] Hardware name: OptiPlex 745 [ 1557.895440] Modules linked in: iTCO_wdt iTCO_vendor_support lpc_ich mfd_core xts gf128mul ablk_helper cryptd aes_x86_64 sha256_generic btrfs libcrc32c [ 1557.895458] Pid: 5728, comm: umount Tainted: G W 3.6.4-x+ #3 [ 1557.895461] Call Trace: [ 1557.895467] [] warn_slowpath_common+0x74/0xa2 [ 1557.895472] [] warn_slowpath_null+0x1a/0x1c [ 1557.895489] [] btrfs_free_block_groups+0x23e/0x34c [btrfs] [ 1557.895510] [] close_ctree+0x1bc/0x2f1 [btrfs] [ 1557.895516] [] ? evict_inodes+0x106/0x115 [ 1557.895533] [] btrfs_put_super+0x19/0x1b [btrfs] [ 1557.895540] [] generic_shutdown_super+0x5b/0xdc [ 1557.895547] [] kill_anon_super+0x16/0x24 [ 1557.895561] [] btrfs_kill_super+0x1a/0x8f [btrfs] [ 1557.895567] [] deactivate_locked_super+0x34/0x5a [ 1557.895573] [] deactivate_super+0x4e/0x66 [ 1557.895578] [] mntput_no_expire+0xf7/0x14d [ 1557.895583] [] sys_umount+0x63/0x37a [ 1557.895588] [] system_call_fastpath+0x16/0x1b [ 1557.895591] ---[ end trace 29a44c3c46b1a1be ]--- [ 1557.895595] space_info 1 has 806678528 free, is not full [ 1557.895600] space_info total=4303355904, used=3496611840, pinned=0, reserved=0, may_use=2998272, readonly=65536 [ 1557.895603] ------------[ cut here ]------------ [ 1557.895619] WARNING: at fs/btrfs/extent-tree.c:7687 btrfs_free_block_groups+0x23e/0x34c [btrfs]() [ 1557.895622] Hardware name: OptiPlex 745 [ 1557.895624] Modules linked in: iTCO_wdt iTCO_vendor_support lpc_ich mfd_core xts gf128mul ablk_helper cryptd aes_x86_64 sha256_generic btrfs libcrc32c [ 1557.895643] Pid: 5728, comm: umount Tainted: G W 3.6.4-x+ #3 [ 1557.895646] Call Trace: [ 1557.895652] [] warn_slowpath_common+0x74/0xa2 [ 1557.895656] [] warn_slowpath_null+0x1a/0x1c [ 1557.895673] [] btrfs_free_block_groups+0x23e/0x34c [btrfs] [ 1557.895693] [] close_ctree+0x1bc/0x2f1 [btrfs] [ 1557.895699] [] ? evict_inodes+0x106/0x115 [ 1557.895716] [] btrfs_put_super+0x19/0x1b [btrfs] [ 1557.895723] [] generic_shutdown_super+0x5b/0xdc [ 1557.895729] [] kill_anon_super+0x16/0x24 [ 1557.895743] [] btrfs_kill_super+0x1a/0x8f [btrfs] [ 1557.895749] [] deactivate_locked_super+0x34/0x5a [ 1557.895754] [] deactivate_super+0x4e/0x66 [ 1557.895760] [] mntput_no_expire+0xf7/0x14d [ 1557.895764] [] sys_umount+0x63/0x37a [ 1557.895770] [] system_call_fastpath+0x16/0x1b [ 1557.895773] ---[ end trace 29a44c3c46b1a1bf ]--- [ 1557.895776] space_info 4 has 92979200 free, is not full [ 1557.895780] space_info total=813694976, used=720715776, pinned=0, reserved=0, may_use=26738688, readonly=0