From: "Lukáš Czerner" <lczerner@redhat.com>
To: "Darrick J. Wong" <darrick.wong@oracle.com>
Cc: tytso@mit.edu, linux-ext4@vger.kernel.org
Subject: Re: [PATCH 22/31] libext2fs: During punch, only free a cluster if we're sure that all blocks in the cluster are being punched
Date: Thu, 10 Oct 2013 17:53:37 +0200 (CEST) [thread overview]
Message-ID: <alpine.LFD.2.00.1310101737351.1925@localhost.localdomain> (raw)
In-Reply-To: <20131001012903.28415.39499.stgit@birch.djwong.org>
On Mon, 30 Sep 2013, Darrick J. Wong wrote:
> Date: Mon, 30 Sep 2013 18:29:03 -0700
> From: Darrick J. Wong <darrick.wong@oracle.com>
> To: tytso@mit.edu, darrick.wong@oracle.com
> Cc: linux-ext4@vger.kernel.org
> Subject: [PATCH 22/31] libext2fs: During punch,
> only free a cluster if we're sure that all blocks in the cluster are being
> punched
>
> When bigalloc is enabled, using ext2fs_block_alloc_stats2() to free any block
> in a cluster has the effect of freeing the entire cluster. This is problematic
> if a caller instructs us to punch, say, blocks 12-15 of a 16-block cluster,
> because blocks 0-11 now point to a "free" cluster.
>
> The naive way to solve this problem is to see if any of the other blocks in
> this logical cluster map to a physical cluster. If so, then we know that the
> cluster is still in use and it mustn't be freed. Otherwise, we are punching
> the last mapped block in this cluster, so we can free the cluster.
>
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> ---
> lib/ext2fs/bmap.c | 28 ++++++++++++++++++++++++++++
> lib/ext2fs/ext2fs.h | 3 +++
> lib/ext2fs/punch.c | 30 +++++++++++++++++++++++++-----
> 3 files changed, 56 insertions(+), 5 deletions(-)
>
>
> diff --git a/lib/ext2fs/bmap.c b/lib/ext2fs/bmap.c
> index 5074587..a6e35a9 100644
> --- a/lib/ext2fs/bmap.c
> +++ b/lib/ext2fs/bmap.c
> @@ -173,6 +173,34 @@ static errcode_t implied_cluster_alloc(ext2_filsys fs, ext2_ino_t ino,
> return 0;
> }
>
> +errcode_t ext2fs_map_cluster_block(ext2_filsys fs, ext2_ino_t ino,
> + struct ext2_inode *inode, blk64_t lblk,
> + blk64_t *pblk)
> +{
> + ext2_extent_handle_t handle;
> + errcode_t retval;
> +
> + /* Need bigalloc and extents to be enabled */
> + *pblk = 0;
> + if (!EXT2_HAS_RO_COMPAT_FEATURE(fs->super,
> + EXT4_FEATURE_RO_COMPAT_BIGALLOC) ||
> + !(inode->i_flags & EXT4_EXTENTS_FL))
> + return 0;
> +
> + retval = ext2fs_extent_open2(fs, ino, inode, &handle);
> + if (retval)
> + goto out;
> +
> + retval = implied_cluster_alloc(fs, ino, inode, handle, lblk, pblk);
> + if (retval)
> + goto out2;
> +
> +out2:
> + ext2fs_extent_free(handle);
> +out:
> + return retval;
> +}
> +
> static errcode_t extent_bmap(ext2_filsys fs, ext2_ino_t ino,
> struct ext2_inode *inode,
> ext2_extent_handle_t handle,
> diff --git a/lib/ext2fs/ext2fs.h b/lib/ext2fs/ext2fs.h
> index 9fef6d3..88da8db 100644
> --- a/lib/ext2fs/ext2fs.h
> +++ b/lib/ext2fs/ext2fs.h
> @@ -925,6 +925,9 @@ extern errcode_t ext2fs_bmap2(ext2_filsys fs, ext2_ino_t ino,
> struct ext2_inode *inode,
> char *block_buf, int bmap_flags, blk64_t block,
> int *ret_flags, blk64_t *phys_blk);
> +errcode_t ext2fs_map_cluster_block(ext2_filsys fs, ext2_ino_t ino,
> + struct ext2_inode *inode, blk64_t lblk,
> + blk64_t *pblk);
>
> #if 0
> /* bmove.c */
> diff --git a/lib/ext2fs/punch.c b/lib/ext2fs/punch.c
> index 4471f46..e0193b0 100644
> --- a/lib/ext2fs/punch.c
> +++ b/lib/ext2fs/punch.c
> @@ -183,8 +183,8 @@ static errcode_t ext2fs_punch_extent(ext2_filsys fs, ext2_ino_t ino,
> ext2_extent_handle_t handle = 0;
> struct ext2fs_extent extent;
> errcode_t retval;
> - blk64_t free_start, next;
> - __u32 free_count, newlen;
> + blk64_t free_start, next, lfree_start, pblk;
> + __u32 free_count, newlen, cluster_freed;
> int freed = 0;
> int op;
>
> @@ -210,6 +210,7 @@ static errcode_t ext2fs_punch_extent(ext2_filsys fs, ext2_ino_t ino,
> /* Start of deleted region before extent;
> adjust beginning of extent */
> free_start = extent.e_pblk;
> + lfree_start = extent.e_lblk;
> if (next > end)
> free_count = end - extent.e_lblk + 1;
> else
> @@ -225,6 +226,7 @@ static errcode_t ext2fs_punch_extent(ext2_filsys fs, ext2_ino_t ino,
> dbg_printf("Case #%d\n", 2);
> newlen = start - extent.e_lblk;
> free_start = extent.e_pblk + newlen;
> + lfree_start = extent.e_lblk + newlen;
> free_count = extent.e_len - newlen;
> extent.e_len = newlen;
> } else {
> @@ -240,6 +242,7 @@ static errcode_t ext2fs_punch_extent(ext2_filsys fs, ext2_ino_t ino,
>
> extent.e_len = start - extent.e_lblk;
> free_start = extent.e_pblk + extent.e_len;
> + lfree_start = extent.e_lblk + extent.e_len;
> free_count = end - start + 1;
>
> dbg_print_extent("inserting", &newex);
> @@ -280,9 +283,26 @@ static errcode_t ext2fs_punch_extent(ext2_filsys fs, ext2_ino_t ino,
> goto errout;
> dbg_printf("Free start %llu, free count = %u\n",
> free_start, free_count);
> - while (free_count-- > 0) {
> - ext2fs_block_alloc_stats2(fs, free_start++, -1);
> - freed++;
> + while (free_count > 0) {
> + retval = ext2fs_map_cluster_block(fs, ino, inode,
> + lfree_start, &pblk);
> + if (retval)
> + goto errout;
> + if (!pblk) {
> + ext2fs_block_alloc_stats2(fs, free_start, -1);
> + freed++;
> + cluster_freed = EXT2FS_CLUSTER_RATIO(fs) -
> + (free_start & EXT2FS_CLUSTER_MASK(fs));
> + if (cluster_freed > free_count)
> + cluster_freed = free_count;
> + free_count -= cluster_freed;
> + free_start += cluster_freed;
> + lfree_start += cluster_freed;
> + continue;
> + }
> + free_count--;
> + free_start++;
> + lfree_start++;
I think that this is a little bit excessive. What I think we should
do here is to identify first and last partial cluster and possibly
call ext2fs_map_cluster_block() for those since we might or might
not want to free then depending on whether there are other blocks in
it in-use.
Then just iterate over the whole clusters in between and free them
all. Having to call ext2fs_map_cluster_block() for every single
block we're freeing from the extent tree is not really necessary I think
especially since we really need to get this information for those
possibly partial clusters at the start and end of the extent.
Thanks!
-Lukas
> }
> next_extent:
> retval = ext2fs_extent_get(handle, op,
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
next prev parent reply other threads:[~2013-10-10 15:53 UTC|newest]
Thread overview: 90+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-10-01 1:26 [PATCH v1 00/31] e2fsprogs September 2013 patchbomb Darrick J. Wong
2013-10-01 1:26 ` [PATCH 01/31] tune2fs: Don't convert block # to cluster # when clearing uninit_bg Darrick J. Wong
2013-10-03 16:53 ` Lukáš Czerner
2013-10-03 19:04 ` Darrick J. Wong
2013-10-07 12:49 ` Lukáš Czerner
2013-10-07 13:03 ` Theodore Ts'o
2013-10-09 22:10 ` Darrick J. Wong
2013-10-10 0:26 ` Theodore Ts'o
2013-10-10 22:04 ` Darrick J. Wong
2013-10-13 3:09 ` Theodore Ts'o
2013-10-01 1:26 ` [PATCH 02/31] libext2fs: Only link an inode into a directory once Darrick J. Wong
2013-10-01 15:37 ` jon ernst
2013-10-01 21:11 ` Darrick J. Wong
2013-10-07 13:17 ` Theodore Ts'o
2013-10-07 18:53 ` Darrick J. Wong
2013-10-01 1:27 ` [PATCH 03/31] Define an error code for block bitmap checksum failures Darrick J. Wong
2013-10-13 3:12 ` Theodore Ts'o
2013-10-01 1:27 ` [PATCH 04/31] libext2fs: Fix a minor grammatical error in the error catalog Darrick J. Wong
2013-10-07 13:20 ` Theodore Ts'o
2013-10-01 1:27 ` [PATCH 05/31] libext2fs: Add space for metadata checksum when unconverting a hashed directory block Darrick J. Wong
2013-10-13 3:16 ` Theodore Ts'o
2013-10-01 1:27 ` [PATCH 06/31] e2p: Fix f[gs]etflags argument size mismatch Darrick J. Wong
2013-10-07 13:33 ` Theodore Ts'o
2013-10-07 20:40 ` Darrick J. Wong
2013-10-07 23:23 ` Darrick J. Wong
2013-10-08 0:06 ` Theodore Ts'o
2013-10-08 0:28 ` Darrick J. Wong
2013-10-01 1:27 ` [PATCH 07/31] libext2fs: When writing a file that has a i_size > 2GB, set the large_file feature flag and update the superblock Darrick J. Wong
2013-10-07 13:14 ` Theodore Ts'o
2013-10-01 1:27 ` [PATCH 08/31] libext2fs: Fix off-by-one error in file truncation Darrick J. Wong
2013-10-07 14:02 ` Lukáš Czerner
2013-10-08 15:52 ` Theodore Ts'o
2013-10-01 1:27 ` [PATCH 09/31] libext2fs: Rewind extent pointer when totally deleting an extent Darrick J. Wong
2013-10-07 13:37 ` Theodore Ts'o
2013-10-07 18:24 ` Darrick J. Wong
2013-10-01 1:27 ` [PATCH 10/31] libext2fs: Allow callers to punch a single block Darrick J. Wong
2013-10-01 19:09 ` jon ernst
2013-10-01 21:25 ` Darrick J. Wong
2013-10-07 13:40 ` Theodore Ts'o
2013-10-08 15:54 ` Theodore Ts'o
2013-10-01 1:27 ` [PATCH 11/31] libext2fs: ind_punch() must not stop examining blocks prematurely Darrick J. Wong
2013-10-07 13:43 ` Theodore Ts'o
2013-10-01 1:27 ` [PATCH 12/31] e2fsprogs: Fix blk_t <- blk64_t assignment mismatches Darrick J. Wong
2013-10-07 13:52 ` Theodore Ts'o
2013-10-01 1:28 ` [PATCH 13/31] e2fsprogs: Less critical fixes to use the appropriate blk*t types Darrick J. Wong
2013-10-07 13:59 ` Theodore Ts'o
2013-10-01 1:28 ` [PATCH 14/31] libext2fs: Fix ext2fs_open2() truncation of the superblock parameter Darrick J. Wong
2013-10-07 14:30 ` Lukáš Czerner
2013-10-07 18:42 ` Darrick J. Wong
2013-10-08 15:58 ` Theodore Ts'o
2013-10-08 17:47 ` Darrick J. Wong
2013-10-01 1:28 ` [PATCH 15/31] e2fsck: Teach EA refcounting code to handle 48bit block addresses Darrick J. Wong
2013-10-07 15:30 ` Lukáš Czerner
2013-10-07 18:37 ` Darrick J. Wong
2013-10-08 16:01 ` Theodore Ts'o
2013-10-09 21:53 ` Darrick J. Wong
2013-10-01 1:28 ` [PATCH 16/31] debugfs: Handle 64bit block numbers Darrick J. Wong
2013-10-07 15:49 ` Lukáš Czerner
2013-10-07 18:49 ` Darrick J. Wong
2013-10-01 1:28 ` [PATCH 17/31] libext2fs: Refactor u32-list to handle 32 and 64-bit data types Darrick J. Wong
2013-10-10 14:46 ` Lukáš Czerner
2013-10-10 18:05 ` Darrick J. Wong
2013-10-01 1:28 ` [PATCH 18/31] libext2fs: Badblocks should handle 48-bit block numbers correctly Darrick J. Wong
2013-10-08 16:03 ` Theodore Ts'o
2013-10-09 21:57 ` Darrick J. Wong
2013-10-01 1:28 ` [PATCH 19/31] badblocks: Use the new badblocks APIs for 64-bit block numbers Darrick J. Wong
2013-10-10 15:01 ` Lukáš Czerner
2013-10-01 1:28 ` [PATCH 20/31] e2fsprogs: Add (optional) sparse checking to the build Darrick J. Wong
2013-10-12 3:13 ` Theodore Ts'o
2013-10-01 1:28 ` [PATCH 21/31] libext2fs: Be more thorough in searching a range of blocks for a cluster Darrick J. Wong
2013-10-08 16:09 ` Theodore Ts'o
2013-10-01 1:29 ` [PATCH 22/31] libext2fs: During punch, only free a cluster if we're sure that all blocks in the cluster are being punched Darrick J. Wong
2013-10-10 15:53 ` Lukáš Czerner [this message]
2013-10-10 19:29 ` Darrick J. Wong
2013-10-01 1:29 ` [PATCH 23/31] libext2fs: expanddir and mkjournal need not update the summary counts when performing an implied cluster allocation Darrick J. Wong
2013-10-10 16:02 ` Lukáš Czerner
2013-10-01 1:29 ` [PATCH 24/31] libext2fs: Use ext2fs_punch() to truncate quota file Darrick J. Wong
2013-10-10 16:06 ` Lukáš Czerner
2013-10-01 1:29 ` [PATCH 25/31] e2fsck: Only release clusters when shortening a directory during a rehash Darrick J. Wong
2013-10-10 16:13 ` Lukáš Czerner
2013-10-01 1:29 ` [PATCH 26/31] libext2fs: openfs() musn't allow bigalloc without EXT2_FLAGS_64BITS Darrick J. Wong
2013-10-07 12:50 ` Lukáš Czerner
2013-10-12 1:36 ` Theodore Ts'o
2013-10-01 1:29 ` [PATCH 27/31] resize2fs: Convert fs to and from 64bit mode Darrick J. Wong
2013-10-01 1:29 ` [PATCH 28/31] mke2fs: Complain about creating 64bit filesystems without extents Darrick J. Wong
2013-10-12 1:14 ` Theodore Ts'o
2013-10-01 1:29 ` [PATCH 29/31] e2fsck: Enable extents on all 64bit filesystems Darrick J. Wong
2013-10-12 1:19 ` Theodore Ts'o
2013-10-01 1:29 ` [PATCH 30/31] libext2fs: Support modifying arbitrary extended attributes Darrick J. Wong
2013-10-01 1:30 ` [PATCH 31/31] misc: Add fuse2fs, a FUSE server for e2fsprogs Darrick J. Wong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=alpine.LFD.2.00.1310101737351.1925@localhost.localdomain \
--to=lczerner@redhat.com \
--cc=darrick.wong@oracle.com \
--cc=linux-ext4@vger.kernel.org \
--cc=tytso@mit.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox