From: "Darrick J. Wong" <darrick.wong@oracle.com>
To: "Lukáš Czerner" <lczerner@redhat.com>
Cc: tytso@mit.edu, linux-ext4@vger.kernel.org
Subject: Re: [PATCH 22/31] libext2fs: During punch, only free a cluster if we're sure that all blocks in the cluster are being punched
Date: Thu, 10 Oct 2013 12:29:09 -0700 [thread overview]
Message-ID: <20131010192909.GP6860@birch.djwong.org> (raw)
In-Reply-To: <alpine.LFD.2.00.1310101737351.1925@localhost.localdomain>
On Thu, Oct 10, 2013 at 05:53:37PM +0200, Lukáš Czerner wrote:
> On Mon, 30 Sep 2013, Darrick J. Wong wrote:
>
> > Date: Mon, 30 Sep 2013 18:29:03 -0700
> > From: Darrick J. Wong <darrick.wong@oracle.com>
> > To: tytso@mit.edu, darrick.wong@oracle.com
> > Cc: linux-ext4@vger.kernel.org
> > Subject: [PATCH 22/31] libext2fs: During punch,
> > only free a cluster if we're sure that all blocks in the cluster are being
> > punched
> >
> > When bigalloc is enabled, using ext2fs_block_alloc_stats2() to free any block
> > in a cluster has the effect of freeing the entire cluster. This is problematic
> > if a caller instructs us to punch, say, blocks 12-15 of a 16-block cluster,
> > because blocks 0-11 now point to a "free" cluster.
> >
> > The naive way to solve this problem is to see if any of the other blocks in
> > this logical cluster map to a physical cluster. If so, then we know that the
> > cluster is still in use and it mustn't be freed. Otherwise, we are punching
> > the last mapped block in this cluster, so we can free the cluster.
> >
> > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> > ---
> > lib/ext2fs/bmap.c | 28 ++++++++++++++++++++++++++++
> > lib/ext2fs/ext2fs.h | 3 +++
> > lib/ext2fs/punch.c | 30 +++++++++++++++++++++++++-----
> > 3 files changed, 56 insertions(+), 5 deletions(-)
> >
> >
> > diff --git a/lib/ext2fs/bmap.c b/lib/ext2fs/bmap.c
> > index 5074587..a6e35a9 100644
> > --- a/lib/ext2fs/bmap.c
> > +++ b/lib/ext2fs/bmap.c
> > @@ -173,6 +173,34 @@ static errcode_t implied_cluster_alloc(ext2_filsys fs, ext2_ino_t ino,
> > return 0;
> > }
> >
> > +errcode_t ext2fs_map_cluster_block(ext2_filsys fs, ext2_ino_t ino,
> > + struct ext2_inode *inode, blk64_t lblk,
> > + blk64_t *pblk)
> > +{
> > + ext2_extent_handle_t handle;
> > + errcode_t retval;
> > +
> > + /* Need bigalloc and extents to be enabled */
> > + *pblk = 0;
> > + if (!EXT2_HAS_RO_COMPAT_FEATURE(fs->super,
> > + EXT4_FEATURE_RO_COMPAT_BIGALLOC) ||
> > + !(inode->i_flags & EXT4_EXTENTS_FL))
> > + return 0;
> > +
> > + retval = ext2fs_extent_open2(fs, ino, inode, &handle);
> > + if (retval)
> > + goto out;
> > +
> > + retval = implied_cluster_alloc(fs, ino, inode, handle, lblk, pblk);
> > + if (retval)
> > + goto out2;
> > +
> > +out2:
> > + ext2fs_extent_free(handle);
> > +out:
> > + return retval;
> > +}
> > +
> > static errcode_t extent_bmap(ext2_filsys fs, ext2_ino_t ino,
> > struct ext2_inode *inode,
> > ext2_extent_handle_t handle,
> > diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
> > index 9fef6d3..88da8db 100644
> > --- a/lib/ext2fs/ext2fs.h
> > +++ b/lib/ext2fs/ext2fs.h
> > @@ -925,6 +925,9 @@ extern errcode_t ext2fs_bmap2(ext2_filsys fs, ext2_ino_t ino,
> > struct ext2_inode *inode,
> > char *block_buf, int bmap_flags, blk64_t block,
> > int *ret_flags, blk64_t *phys_blk);
> > +errcode_t ext2fs_map_cluster_block(ext2_filsys fs, ext2_ino_t ino,
> > + struct ext2_inode *inode, blk64_t lblk,
> > + blk64_t *pblk);
> >
> > #if 0
> > /* bmove.c */
> > diff --git a/lib/ext2fs/punch.c b/lib/ext2fs/punch.c
> > index 4471f46..e0193b0 100644
> > --- a/lib/ext2fs/punch.c
> > +++ b/lib/ext2fs/punch.c
> > @@ -183,8 +183,8 @@ static errcode_t ext2fs_punch_extent(ext2_filsys fs, ext2_ino_t ino,
> > ext2_extent_handle_t handle = 0;
> > struct ext2fs_extent extent;
> > errcode_t retval;
> > - blk64_t free_start, next;
> > - __u32 free_count, newlen;
> > + blk64_t free_start, next, lfree_start, pblk;
> > + __u32 free_count, newlen, cluster_freed;
> > int freed = 0;
> > int op;
> >
> > @@ -210,6 +210,7 @@ static errcode_t ext2fs_punch_extent(ext2_filsys fs, ext2_ino_t ino,
> > /* Start of deleted region before extent;
> > adjust beginning of extent */
> > free_start = extent.e_pblk;
> > + lfree_start = extent.e_lblk;
> > if (next > end)
> > free_count = end - extent.e_lblk + 1;
> > else
> > @@ -225,6 +226,7 @@ static errcode_t ext2fs_punch_extent(ext2_filsys fs, ext2_ino_t ino,
> > dbg_printf("Case #%d\n", 2);
> > newlen = start - extent.e_lblk;
> > free_start = extent.e_pblk + newlen;
> > + lfree_start = extent.e_lblk + newlen;
> > free_count = extent.e_len - newlen;
> > extent.e_len = newlen;
> > } else {
> > @@ -240,6 +242,7 @@ static errcode_t ext2fs_punch_extent(ext2_filsys fs, ext2_ino_t ino,
> >
> > extent.e_len = start - extent.e_lblk;
> > free_start = extent.e_pblk + extent.e_len;
> > + lfree_start = extent.e_lblk + extent.e_len;
> > free_count = end - start + 1;
> >
> > dbg_print_extent("inserting", &newex);
> > @@ -280,9 +283,26 @@ static errcode_t ext2fs_punch_extent(ext2_filsys fs, ext2_ino_t ino,
> > goto errout;
> > dbg_printf("Free start %llu, free count = %u\n",
> > free_start, free_count);
> > - while (free_count-- > 0) {
> > - ext2fs_block_alloc_stats2(fs, free_start++, -1);
> > - freed++;
> > + while (free_count > 0) {
> > + retval = ext2fs_map_cluster_block(fs, ino, inode,
> > + lfree_start, &pblk);
> > + if (retval)
> > + goto errout;
> > + if (!pblk) {
> > + ext2fs_block_alloc_stats2(fs, free_start, -1);
> > + freed++;
> > + cluster_freed = EXT2FS_CLUSTER_RATIO(fs) -
> > + (free_start & EXT2FS_CLUSTER_MASK(fs));
> > + if (cluster_freed > free_count)
> > + cluster_freed = free_count;
> > + free_count -= cluster_freed;
> > + free_start += cluster_freed;
> > + lfree_start += cluster_freed;
> > + continue;
> > + }
> > + free_count--;
> > + free_start++;
> > + lfree_start++;
>
> I think that this is a little bit excessive. What I think we should
> do here is to identify first and last partial cluster and possibly
> call ext2fs_map_cluster_block() for those since we might or might
> not want to free then depending on whether there are other blocks in
> it in-use.
>
> Then just iterate over the whole clusters in between and free them
> all. Having to call ext2fs_map_cluster_block() for every single
> block we're freeing from the extent tree is not really necessary I think
> especially since we really need to get this information for those
> possibly partial clusters at the start and end of the extent.
Hmm. I think I could eliminate the middle ext2fs_map_cluster_block() calls by
only calling it if free_start is within cluster_ratio blocks of the pre-loop
value of free_start, or if free_count < cluster_ratio. I can also split the
whole thing into three loops (pre-cluster, clusters, and post-cluster), though
for the non-bigalloc case I'd skip to the middle loop.
--D
>
> Thanks!
> -Lukas
>
>
> > }
> > next_extent:
> > retval = ext2fs_extent_get(handle, op,
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
> >
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
next prev parent reply other threads:[~2013-10-10 19:29 UTC|newest]
Thread overview: 90+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-10-01 1:26 [PATCH v1 00/31] e2fsprogs September 2013 patchbomb Darrick J. Wong
2013-10-01 1:26 ` [PATCH 01/31] tune2fs: Don't convert block # to cluster # when clearing uninit_bg Darrick J. Wong
2013-10-03 16:53 ` Lukáš Czerner
2013-10-03 19:04 ` Darrick J. Wong
2013-10-07 12:49 ` Lukáš Czerner
2013-10-07 13:03 ` Theodore Ts'o
2013-10-09 22:10 ` Darrick J. Wong
2013-10-10 0:26 ` Theodore Ts'o
2013-10-10 22:04 ` Darrick J. Wong
2013-10-13 3:09 ` Theodore Ts'o
2013-10-01 1:26 ` [PATCH 02/31] libext2fs: Only link an inode into a directory once Darrick J. Wong
2013-10-01 15:37 ` jon ernst
2013-10-01 21:11 ` Darrick J. Wong
2013-10-07 13:17 ` Theodore Ts'o
2013-10-07 18:53 ` Darrick J. Wong
2013-10-01 1:27 ` [PATCH 03/31] Define an error code for block bitmap checksum failures Darrick J. Wong
2013-10-13 3:12 ` Theodore Ts'o
2013-10-01 1:27 ` [PATCH 04/31] libext2fs: Fix a minor grammatical error in the error catalog Darrick J. Wong
2013-10-07 13:20 ` Theodore Ts'o
2013-10-01 1:27 ` [PATCH 05/31] libext2fs: Add space for metadata checksum when unconverting a hashed directory block Darrick J. Wong
2013-10-13 3:16 ` Theodore Ts'o
2013-10-01 1:27 ` [PATCH 06/31] e2p: Fix f[gs]etflags argument size mismatch Darrick J. Wong
2013-10-07 13:33 ` Theodore Ts'o
2013-10-07 20:40 ` Darrick J. Wong
2013-10-07 23:23 ` Darrick J. Wong
2013-10-08 0:06 ` Theodore Ts'o
2013-10-08 0:28 ` Darrick J. Wong
2013-10-01 1:27 ` [PATCH 07/31] libext2fs: When writing a file that has a i_size > 2GB, set the large_file feature flag and update the superblock Darrick J. Wong
2013-10-07 13:14 ` Theodore Ts'o
2013-10-01 1:27 ` [PATCH 08/31] libext2fs: Fix off-by-one error in file truncation Darrick J. Wong
2013-10-07 14:02 ` Lukáš Czerner
2013-10-08 15:52 ` Theodore Ts'o
2013-10-01 1:27 ` [PATCH 09/31] libext2fs: Rewind extent pointer when totally deleting an extent Darrick J. Wong
2013-10-07 13:37 ` Theodore Ts'o
2013-10-07 18:24 ` Darrick J. Wong
2013-10-01 1:27 ` [PATCH 10/31] libext2fs: Allow callers to punch a single block Darrick J. Wong
2013-10-01 19:09 ` jon ernst
2013-10-01 21:25 ` Darrick J. Wong
2013-10-07 13:40 ` Theodore Ts'o
2013-10-08 15:54 ` Theodore Ts'o
2013-10-01 1:27 ` [PATCH 11/31] libext2fs: ind_punch() must not stop examining blocks prematurely Darrick J. Wong
2013-10-07 13:43 ` Theodore Ts'o
2013-10-01 1:27 ` [PATCH 12/31] e2fsprogs: Fix blk_t <- blk64_t assignment mismatches Darrick J. Wong
2013-10-07 13:52 ` Theodore Ts'o
2013-10-01 1:28 ` [PATCH 13/31] e2fsprogs: Less critical fixes to use the appropriate blk*t types Darrick J. Wong
2013-10-07 13:59 ` Theodore Ts'o
2013-10-01 1:28 ` [PATCH 14/31] libext2fs: Fix ext2fs_open2() truncation of the superblock parameter Darrick J. Wong
2013-10-07 14:30 ` Lukáš Czerner
2013-10-07 18:42 ` Darrick J. Wong
2013-10-08 15:58 ` Theodore Ts'o
2013-10-08 17:47 ` Darrick J. Wong
2013-10-01 1:28 ` [PATCH 15/31] e2fsck: Teach EA refcounting code to handle 48bit block addresses Darrick J. Wong
2013-10-07 15:30 ` Lukáš Czerner
2013-10-07 18:37 ` Darrick J. Wong
2013-10-08 16:01 ` Theodore Ts'o
2013-10-09 21:53 ` Darrick J. Wong
2013-10-01 1:28 ` [PATCH 16/31] debugfs: Handle 64bit block numbers Darrick J. Wong
2013-10-07 15:49 ` Lukáš Czerner
2013-10-07 18:49 ` Darrick J. Wong
2013-10-01 1:28 ` [PATCH 17/31] libext2fs: Refactor u32-list to handle 32 and 64-bit data types Darrick J. Wong
2013-10-10 14:46 ` Lukáš Czerner
2013-10-10 18:05 ` Darrick J. Wong
2013-10-01 1:28 ` [PATCH 18/31] libext2fs: Badblocks should handle 48-bit block numbers correctly Darrick J. Wong
2013-10-08 16:03 ` Theodore Ts'o
2013-10-09 21:57 ` Darrick J. Wong
2013-10-01 1:28 ` [PATCH 19/31] badblocks: Use the new badblocks APIs for 64-bit block numbers Darrick J. Wong
2013-10-10 15:01 ` Lukáš Czerner
2013-10-01 1:28 ` [PATCH 20/31] e2fsprogs: Add (optional) sparse checking to the build Darrick J. Wong
2013-10-12 3:13 ` Theodore Ts'o
2013-10-01 1:28 ` [PATCH 21/31] libext2fs: Be more thorough in searching a range of blocks for a cluster Darrick J. Wong
2013-10-08 16:09 ` Theodore Ts'o
2013-10-01 1:29 ` [PATCH 22/31] libext2fs: During punch, only free a cluster if we're sure that all blocks in the cluster are being punched Darrick J. Wong
2013-10-10 15:53 ` Lukáš Czerner
2013-10-10 19:29 ` Darrick J. Wong [this message]
2013-10-01 1:29 ` [PATCH 23/31] libext2fs: expanddir and mkjournal need not update the summary counts when performing an implied cluster allocation Darrick J. Wong
2013-10-10 16:02 ` Lukáš Czerner
2013-10-01 1:29 ` [PATCH 24/31] libext2fs: Use ext2fs_punch() to truncate quota file Darrick J. Wong
2013-10-10 16:06 ` Lukáš Czerner
2013-10-01 1:29 ` [PATCH 25/31] e2fsck: Only release clusters when shortening a directory during a rehash Darrick J. Wong
2013-10-10 16:13 ` Lukáš Czerner
2013-10-01 1:29 ` [PATCH 26/31] libext2fs: openfs() musn't allow bigalloc without EXT2_FLAGS_64BITS Darrick J. Wong
2013-10-07 12:50 ` Lukáš Czerner
2013-10-12 1:36 ` Theodore Ts'o
2013-10-01 1:29 ` [PATCH 27/31] resize2fs: Convert fs to and from 64bit mode Darrick J. Wong
2013-10-01 1:29 ` [PATCH 28/31] mke2fs: Complain about creating 64bit filesystems without extents Darrick J. Wong
2013-10-12 1:14 ` Theodore Ts'o
2013-10-01 1:29 ` [PATCH 29/31] e2fsck: Enable extents on all 64bit filesystems Darrick J. Wong
2013-10-12 1:19 ` Theodore Ts'o
2013-10-01 1:29 ` [PATCH 30/31] libext2fs: Support modifying arbitrary extended attributes Darrick J. Wong
2013-10-01 1:30 ` [PATCH 31/31] misc: Add fuse2fs, a FUSE server for e2fsprogs Darrick J. Wong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20131010192909.GP6860@birch.djwong.org \
--to=darrick.wong@oracle.com \
--cc=lczerner@redhat.com \
--cc=linux-ext4@vger.kernel.org \
--cc=tytso@mit.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).