* [PATCH] btrfs: add missing discards when unpinning extents with, -o discard @ 2015-03-30 19:12 Jeff Mahoney 2015-03-30 19:29 ` Filipe David Manana 2015-08-21 19:25 ` Jun He 0 siblings, 2 replies; 4+ messages in thread From: Jeff Mahoney @ 2015-03-30 19:12 UTC (permalink / raw) To: linux-btrfs The block group removal patch adds an alternate path to forget extents other than btrfs_finish_extent_commit. As a result, any extents that would be freed when the block group is removed aren't discarded. In my test run, with a large copy of mixed sized files followed by removal, it left nearly 2/3 of extents undiscarded. As part of the extent removal review, I also added the discard call to btrfs_destroy_pinned_extent. Signed-off-by: Jeff Mahoney <jeffm@suse.com> --- fs/btrfs/disk-io.c | 7 +++++++ fs/btrfs/extent-tree.c | 9 +++++++++ 2 files changed, 16 insertions(+) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 639f266..f7dfcf8 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -4167,6 +4167,13 @@ again: if (ret) break; + if (btrfs_test_opt(root, DISCARD)) { + ret = btrfs_discard_extent(root, start, + end + 1 - start, NULL); + if (ret) + break; + } + clear_extent_dirty(unpin, start, end, GFP_NOFS); btrfs_error_unpin_extent_range(root, start, end); cond_resched(); diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 0bf45b8..6a7ff46 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -9630,6 +9630,15 @@ void btrfs_delete_unused_bgs(struct btrfs_fs_info *fs_info) * a BUG_ON() at btrfs_unpin_extent_range(). */ mutex_lock(&fs_info->unused_bg_unpin_mutex); + if (btrfs_test_opt(root, DISCARD)) { + ret = btrfs_discard_extent(root, start, end + 1 - start, + NULL); + if (ret) { + mutex_unlock(&fs_info->unused_bg_unpin_mutex); + btrfs_set_block_group_rw(root, block_group); + goto end_trans; + } + } ret = clear_extent_bits(&fs_info->freed_extents[0], start, end, EXTENT_DIRTY, GFP_NOFS); if (ret) { -- 1.8.5.6 -- Jeff Mahoney SUSE Labs ^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH] btrfs: add missing discards when unpinning extents with, -o discard 2015-03-30 19:12 [PATCH] btrfs: add missing discards when unpinning extents with, -o discard Jeff Mahoney @ 2015-03-30 19:29 ` Filipe David Manana 2015-03-30 21:36 ` Jeff Mahoney 2015-08-21 19:25 ` Jun He 1 sibling, 1 reply; 4+ messages in thread From: Filipe David Manana @ 2015-03-30 19:29 UTC (permalink / raw) To: Jeff Mahoney; +Cc: linux-btrfs On Mon, Mar 30, 2015 at 8:12 PM, Jeff Mahoney <jeffm@suse.com> wrote: > The block group removal patch adds an alternate path to forget extents > other than btrfs_finish_extent_commit. As a result, any extents that would > be freed when the block group is removed aren't discarded. In my > test run, with a large copy of mixed sized files followed by removal, it > left nearly 2/3 of extents undiscarded. > > As part of the extent removal review, I also added the discard call to > btrfs_destroy_pinned_extent. > > Signed-off-by: Jeff Mahoney <jeffm@suse.com> > --- > fs/btrfs/disk-io.c | 7 +++++++ > fs/btrfs/extent-tree.c | 9 +++++++++ > 2 files changed, 16 insertions(+) > > diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c > index 639f266..f7dfcf8 100644 > --- a/fs/btrfs/disk-io.c > +++ b/fs/btrfs/disk-io.c > @@ -4167,6 +4167,13 @@ again: > if (ret) > break; > > + if (btrfs_test_opt(root, DISCARD)) { > + ret = btrfs_discard_extent(root, start, > + end + 1 - start, NULL); > + if (ret) > + break; > + } > + Jeff we can't do a discard here, specially if unpin == fs_info->pinned_extents, as explained in: https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=678886bdc6378c1cbd5072da2c5a3035000214e3 > clear_extent_dirty(unpin, start, end, GFP_NOFS); > btrfs_error_unpin_extent_range(root, start, end); > cond_resched(); > diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c > index 0bf45b8..6a7ff46 100644 > --- a/fs/btrfs/extent-tree.c > +++ b/fs/btrfs/extent-tree.c > @@ -9630,6 +9630,15 @@ void btrfs_delete_unused_bgs(struct btrfs_fs_info *fs_info) > * a BUG_ON() at btrfs_unpin_extent_range(). > */ > mutex_lock(&fs_info->unused_bg_unpin_mutex); > + if (btrfs_test_opt(root, DISCARD)) { > + ret = btrfs_discard_extent(root, start, end + 1 - start, > + NULL); > + if (ret) { > + mutex_unlock(&fs_info->unused_bg_unpin_mutex); > + btrfs_set_block_group_rw(root, block_group); > + goto end_trans; > + } > + } This isn't safe either for the similar reason as above. For example, transaction N running, metadata bg X becomes empty and it's added to fs_info->unused_bgs, cleaner kthread running and gets bg X from fs_info->unused_bgs (running btrfs_delete_unused_bgs), does the discard, power failure, reboot, superblock points to metadata extents in the physical location of bg X that are now full of zeroes. thanks > ret = clear_extent_bits(&fs_info->freed_extents[0], start, end, > EXTENT_DIRTY, GFP_NOFS); > if (ret) { > -- > 1.8.5.6 > > > -- > Jeff Mahoney > SUSE Labs > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- Filipe David Manana, "Reasonable men adapt themselves to the world. Unreasonable men adapt the world to themselves. That's why all progress depends on unreasonable men." ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] btrfs: add missing discards when unpinning extents with, -o discard 2015-03-30 19:29 ` Filipe David Manana @ 2015-03-30 21:36 ` Jeff Mahoney 0 siblings, 0 replies; 4+ messages in thread From: Jeff Mahoney @ 2015-03-30 21:36 UTC (permalink / raw) To: fdmanana; +Cc: linux-btrfs -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 3/30/15 3:29 PM, Filipe David Manana wrote: > On Mon, Mar 30, 2015 at 8:12 PM, Jeff Mahoney <jeffm@suse.com> > wrote: >> The block group removal patch adds an alternate path to forget >> extents other than btrfs_finish_extent_commit. As a result, any >> extents that would be freed when the block group is removed >> aren't discarded. In my test run, with a large copy of mixed >> sized files followed by removal, it left nearly 2/3 of extents >> undiscarded. >> >> As part of the extent removal review, I also added the discard >> call to btrfs_destroy_pinned_extent. >> >> Signed-off-by: Jeff Mahoney <jeffm@suse.com> --- >> fs/btrfs/disk-io.c | 7 +++++++ fs/btrfs/extent-tree.c | 9 >> +++++++++ 2 files changed, 16 insertions(+) >> >> diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index >> 639f266..f7dfcf8 100644 --- a/fs/btrfs/disk-io.c +++ >> b/fs/btrfs/disk-io.c @@ -4167,6 +4167,13 @@ again: if (ret) >> break; >> >> + if (btrfs_test_opt(root, DISCARD)) { + ret = >> btrfs_discard_extent(root, start, + end + 1 - start, NULL); + >> if (ret) + break; + } + > > Jeff we can't do a discard here, specially if unpin == > fs_info->pinned_extents, as explained in: > > https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit /?id=678886bdc6378c1cbd5072da2c5a3035000214e3 > > > > > >> clear_extent_dirty(unpin, start, end, GFP_NOFS); >> btrfs_error_unpin_extent_range(root, start, end); cond_resched(); >> diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c >> index 0bf45b8..6a7ff46 100644 --- a/fs/btrfs/extent-tree.c +++ >> b/fs/btrfs/extent-tree.c @@ -9630,6 +9630,15 @@ void >> btrfs_delete_unused_bgs(struct btrfs_fs_info *fs_info) * a >> BUG_ON() at btrfs_unpin_extent_range(). */ >> mutex_lock(&fs_info->unused_bg_unpin_mutex); + if >> (btrfs_test_opt(root, DISCARD)) { + ret = >> btrfs_discard_extent(root, start, end + 1 - start, + NULL); + >> if (ret) { + mutex_unlock(&fs_info->unused_bg_unpin_mutex); + >> btrfs_set_block_group_rw(root, block_group); + goto end_trans; + >> } + } > > This isn't safe either for the similar reason as above. For > example, transaction N running, metadata bg X becomes empty and > it's added to fs_info->unused_bgs, cleaner kthread running and > gets bg X from fs_info->unused_bgs (running > btrfs_delete_unused_bgs), does the discard, power failure, reboot, > superblock points to metadata extents in the physical location of > bg X that are now full of zeroes. > > thanks > >> ret = clear_extent_bits(&fs_info->freed_extents[0], start, end, >> EXTENT_DIRTY, GFP_NOFS); if (ret) { Thanks for the review, Filipe. I think I understand what you mean. btrfs_finish_extent_commit is effectively the only safe place to perform the discards, since we know the transaction is finished. I'll need to rework block group deletions a bit since the clearing of the dirty bits for the whole block group mean that btrfs_finish_extent_commit never sees those blocks as eligible for discard. - -Jeff - -- Jeff Mahoney SUSE Labs -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.19 (Darwin) iQIcBAEBAgAGBQJVGcHYAAoJEB57S2MheeWy1AYP/ixGoegmr2xACfFRmtlOvI9q Wajf/9PiNbgr7kZ9pe5mWIi/Ab9X38OqPXPfnu7rx6qf/kBJ5ltHFsflCHEQazte /5n+dSClhn7dAjWvL36UbIhH+uqaD5kL2rkmU7D72j5l26TwlYxF7vTUIxE891mR 0DN/BjQ58dOY8vsQRU4ZY6cHoX5jpMU2b7tje/Uk+PrctNjFkiTJFR8NApetTV6d eebTomEoWGqSe8RCsZK9PvaRc1qdpuNl8XqdNppEapA9QOuWXaVr7hjYmVMwCAWi 01AVDd6NKwu6dQ2i7rt9+/o4GJrFORptBAtAZgHeTyd99WKFYbxPsWm3aBHrfGU2 4h/s5EEDbLcDV0p/5jojp3c/JdI9hOVL8LzCAI97LGZL9FNqx8CV1m2//TFYqRKk 2K1cmFVmzI+RwZ/Reo6/qW5CcwIeZJnmIZOAAHHnD97PZH69UHW+3RqRDUlcy7Mx srzTNLiA+R25MEEnv2qHNUcPduGmkjaENqq2TufQHsqpR8sIwpaavXNJKR2X128Y q+JwELlGi0XYqa9jgRjFwiX/Y+/nvjh1MQ/GZfArxZjooUpdfPMYrUUOCt4/djUG f0xkExvHr62t9IMdmaQkOhABb5HZ34lnJ6R3v2+p/RdAWIzhWtz6dd+M3x8s4ufZ a1eDIAuFL/wfl18QElBh =DQ57 -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] btrfs: add missing discards when unpinning extents with, -o discard 2015-03-30 19:12 [PATCH] btrfs: add missing discards when unpinning extents with, -o discard Jeff Mahoney 2015-03-30 19:29 ` Filipe David Manana @ 2015-08-21 19:25 ` Jun He 1 sibling, 0 replies; 4+ messages in thread From: Jun He @ 2015-08-21 19:25 UTC (permalink / raw) To: linux-btrfs I just want to let you guys know that there are people waiting for the final patches of the discard problem :) ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2015-08-21 19:40 UTC | newest] Thread overview: 4+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2015-03-30 19:12 [PATCH] btrfs: add missing discards when unpinning extents with, -o discard Jeff Mahoney 2015-03-30 19:29 ` Filipe David Manana 2015-03-30 21:36 ` Jeff Mahoney 2015-08-21 19:25 ` Jun He
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.