[PATCH RFC v1 0/5]Factor common code from convert and split unwritten.

[PATCH RFC v1 0/5]Factor common code from convert and split unwritten.

[Yongqiang Yang]

v0->v1:
   fix a bug in ext4_ext_convert_to_initialized() reported by
   Allison <achender@linux.vnet.ibm.com>.

   optimize ext4_ext_convert_to_initialized().

-- factor common code
   These patches factor common code from ext4_ext_convert_to_initialized() and
   ext4_split_unwritten_extents() so that extent-move-on-write in snapshot and
   punch hole can be built on the common code.

-- optimization
   the 4th and the 5th patch optimize ext4_ext_convert_to_initialized() by
   zeroing out in memory.


[PATCH RFC v1 1/5] ext4:Add a function merging extent right and left.
[PATCH RFC v1 2/5] ext4:Add two functions splitting an extent.
[PATCH RFC v1 3/5] ext4:Reimplement convert and split_unwritten.
[PATCH RFC v1 4/5] ext4: Add a function ext4_ext_zeroout_mem().
[PATCH RFC v1 5/5] ext4: optimize ext4_ext_convert_to_initialized().

[PATCH RFC v1 1/5] ext4:Add a function merging extent right and left.

[Yongqiang Yang]

1] Rename ext4_ext_try_to_merge() to ext4_ext_try_to_merge_right().

2] Add a new function ext4_ext_try_to_merge() which tries to merge
   an extent both left and right.

3] Use the new function in ext4_ext_convert_unwritten_endio() and
   ext4_ext_insert_extent().

Signed-off-by: Yongqiang Yang <xiaoqiangnk@gmail.com>
---
 fs/ext4/extents.c |   65 ++++++++++++++++++++++++++++------------------------
 1 files changed, 35 insertions(+), 30 deletions(-)

diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c
index dd2cb50..11f30d2 100644
--- a/fs/ext4/extents.c
+++ b/fs/ext4/extents.c
@@ -1563,7 +1563,7 @@ ext4_can_extents_be_merged(struct inode *inode, struct ext4_extent *ex1,
  * Returns 0 if the extents (ex and ex+1) were _not_ merged and returns
  * 1 if they got merged.
  */
-static int ext4_ext_try_to_merge(struct inode *inode,
+static int ext4_ext_try_to_merge_right(struct inode *inode,
 				 struct ext4_ext_path *path,
 				 struct ext4_extent *ex)
 {
@@ -1603,6 +1603,31 @@ static int ext4_ext_try_to_merge(struct inode *inode,
 }
 
 /*
+ * This function tries to merge the @ex extent to neighbours in the tree.
+ * return 1 if merge left else 0.
+ */
+static int ext4_ext_try_to_merge(struct inode *inode,
+				  struct ext4_ext_path *path,
+				  struct ext4_extent *ex) {
+	struct ext4_extent_header *eh;
+	unsigned int depth;
+	int merge_done = 0;
+	int ret = 0;
+
+	depth = ext_depth(inode);
+	BUG_ON(path[depth].p_hdr == NULL);
+	eh = path[depth].p_hdr;
+
+	if (ex > EXT_FIRST_EXTENT(eh))
+		merge_done = ext4_ext_try_to_merge_right(inode, path, ex - 1);
+
+	if (!merge_done)
+		ret = ext4_ext_try_to_merge_right(inode, path, ex);
+
+	return ret;
+}
+
+/*
  * check if a portion of the "newext" extent overlaps with an
  * existing extent.
  *
@@ -3039,6 +3064,7 @@ fix_extent_len:
 	ext4_ext_dirty(handle, inode, path + depth);
 	return err;
 }
+
 static int ext4_convert_unwritten_extents_endio(handle_t *handle,
 					      struct inode *inode,
 					      struct ext4_ext_path *path)
@@ -3047,46 +3073,25 @@ static int ext4_convert_unwritten_extents_endio(handle_t *handle,
 	struct ext4_extent_header *eh;
 	int depth;
 	int err = 0;
-	int ret = 0;
 
 	depth = ext_depth(inode);
 	eh = path[depth].p_hdr;
 	ex = path[depth].p_ext;
 
+	ext_debug("ext4_convert_unwritten_extents_endio: inode %lu, logical"
+		"block %llu, max_blocks %u\n", inode->i_ino,
+		(unsigned long long)le32_to_cpu(ex->ee_block),
+		ext4_ext_get_actual_len(ex));
+
 	err = ext4_ext_get_access(handle, inode, path + depth);
 	if (err)
 		goto out;
 	/* first mark the extent as initialized */
 	ext4_ext_mark_initialized(ex);
 
-	/*
-	 * We have to see if it can be merged with the extent
-	 * on the left.
-	 */
-	if (ex > EXT_FIRST_EXTENT(eh)) {
-		/*
-		 * To merge left, pass "ex - 1" to try_to_merge(),
-		 * since it merges towards right _only_.
-		 */
-		ret = ext4_ext_try_to_merge(inode, path, ex - 1);
-		if (ret) {
-			err = ext4_ext_correct_indexes(handle, inode, path);
-			if (err)
-				goto out;
-			depth = ext_depth(inode);
-			ex--;
-		}
-	}
-	/*
-	 * Try to Merge towards right.
-	 */
-	ret = ext4_ext_try_to_merge(inode, path, ex);
-	if (ret) {
-		err = ext4_ext_correct_indexes(handle, inode, path);
-		if (err)
-			goto out;
-		depth = ext_depth(inode);
-	}
+	/* correct indexes is nt needed becasue borders are not changed */
+	ext4_ext_try_to_merge(inode, path, ex);
+
 	/* Mark modified extent as dirty */
 	err = ext4_ext_dirty(handle, inode, path + depth);
 out:
-- 
1.7.4.4

[PATCH RFC v1 2/5] ext4:Add two functions splitting an extent.

[Yongqiang Yang]

v0 -> v1:
   -- coding style
   -- try to merge extents in zeroout case too.

1] Add a function named ext4_split_extent_at() which splits an extent
   into two extents at given logical block.

2] Add a function called ext4_split_extent() which splits an extent
   into three extents.

Signed-off-by: Yongqiang Yang <xiaoqiangnk@gmail.com>
---
 fs/ext4/extents.c |  187 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 187 insertions(+), 0 deletions(-)

diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c
index 11f30d2..db1d67c 100644
--- a/fs/ext4/extents.c
+++ b/fs/ext4/extents.c
@@ -2554,6 +2554,193 @@ static int ext4_ext_zeroout(struct inode *inode, struct ext4_extent *ex)
 	return ret;
 }
 
+/*
+ * used by extent splitting.
+ */
+#define EXT4_EXT_MAY_ZEROOUT	0x1  /* safe to zeroout if split fails \
+					due to ENOSPC */
+#define EXT4_EXT_MARK_UNINIT1	0x2  /* mark first half uninitialized */
+#define EXT4_EXT_MARK_UNINIT2	0x4  /* mark second half uninitialized */
+
+/*
+ * ext4_split_extent_at() splits an extent at given block.
+ *
+ * @handle: the journal handle
+ * @inode: the file inode
+ * @path: the path to the extent
+ * @split: the logical block where the extent is splitted.
+ * @split_flags: indicates if the extent could be zeroout if split fails, and
+ *		 the states(init or uninit) of new extents.
+ * @flags: flags used to insert new extent to extent tree.
+ *
+ *
+ * Splits extent [a, b] into two extents [a, @split) and [@split, b], states
+ * of which are deterimined by split_flag.
+ *
+ * There are two cases:
+ *  a> the extent are splitted into two extent.
+ *  b> split is not needed, and just mark the extent.
+ *
+ * return 0 on success.
+ */
+static int ext4_split_extent_at(handle_t *handle,
+			     struct inode *inode,
+			     struct ext4_ext_path *path,
+			     ext4_lblk_t split,
+			     int split_flag,
+			     int flags)
+{
+	ext4_fsblk_t newblock;
+	ext4_lblk_t ee_block;
+	struct ext4_extent *ex, newex, orig_ex;
+	struct ext4_extent *ex2 = NULL;
+	unsigned int ee_len, depth;
+	int err = 0;
+
+	ext_debug("ext4_split_extents_at: inode %lu, logical"
+		"block %llu\n", inode->i_ino, (unsigned long long)split);
+
+	ext4_ext_show_leaf(inode, path);
+
+	depth = ext_depth(inode);
+	ex = path[depth].p_ext;
+	ee_block = le32_to_cpu(ex->ee_block);
+	ee_len = ext4_ext_get_actual_len(ex);
+	newblock = split - ee_block + ext4_ext_pblock(ex);
+
+	BUG_ON(split < ee_block || split >= (ee_block + ee_len));
+
+	err = ext4_ext_get_access(handle, inode, path + depth);
+	if (err)
+		goto out;
+
+	if (split == ee_block) {
+		/*
+		 * case b: block @split is the block that the extent begins with
+		 * then we just change the state of the extent, and splitting
+		 * is not needed.
+		 */
+		if (split_flag & EXT4_EXT_MARK_UNINIT2)
+			ext4_ext_mark_uninitialized(ex);
+		else
+			ext4_ext_mark_initialized(ex);
+
+		if (!(flags & EXT4_GET_BLOCKS_PRE_IO))
+			ext4_ext_try_to_merge(inode, path, ex);
+
+		err = ext4_ext_dirty(handle, inode, path + depth);
+		goto out;
+	}
+
+	/* case a */
+	memcpy(&orig_ex, ex, sizeof(orig_ex));
+	ex->ee_len = cpu_to_le16(split - ee_block);
+	if (split_flag & EXT4_EXT_MARK_UNINIT1)
+		ext4_ext_mark_uninitialized(ex);
+
+	/*
+	 * path may lead to new leaf, not to original leaf any more
+	 * after ext4_ext_insert_extent() returns,
+	 */
+	err = ext4_ext_dirty(handle, inode, path + depth);
+	if (err)
+		goto fix_extent_len;
+
+	ex2 = &newex;
+	ex2->ee_block = cpu_to_le32(split);
+	ex2->ee_len   = cpu_to_le16(ee_len - (split - ee_block));
+	ext4_ext_store_pblock(ex2, newblock);
+	if (split_flag & EXT4_EXT_MARK_UNINIT2)
+		ext4_ext_mark_uninitialized(ex2);
+
+	err = ext4_ext_insert_extent(handle, inode, path, &newex, flags);
+	if (err == -ENOSPC && (EXT4_EXT_MAY_ZEROOUT & split_flag)) {
+		err = ext4_ext_zeroout(inode, &orig_ex);
+		if (err)
+			goto fix_extent_len;
+		/* update the extent length and mark as initialized */
+		ex->ee_len = cpu_to_le32(ee_len);
+		ext4_ext_try_to_merge(inode, path, ex);
+		err = ext4_ext_dirty(handle, inode, path + depth);
+		goto out;
+	} else if (err)
+		goto fix_extent_len;
+
+out:
+	ext4_ext_show_leaf(inode, path);
+	return err;
+
+fix_extent_len:
+	ex->ee_len = orig_ex.ee_len;
+	ext4_ext_dirty(handle, inode, path + depth);
+	return err;
+}
+
+/*
+ * ext4_split_extents() splits an extent and mark extent which is covered
+ * by @map as split_flags indicates
+ *
+ * It may result in splitting the extent into multiple extents (upto three)
+ * There are three possibilities:
+ *   a> There is no split required
+ *   b> Splits in two extents: Split is happening at either end of the extent
+ *   c> Splits in three extents: Somone is splitting in middle of the extent
+ *
+ */
+static int ext4_split_extent(handle_t *handle,
+			      struct inode *inode,
+			      struct ext4_ext_path *path,
+			      struct ext4_map_blocks *map,
+			      int split_flag,
+			      int flags)
+{
+	ext4_lblk_t ee_block;
+	struct ext4_extent *ex;
+	unsigned int ee_len, depth;
+	int err = 0;
+	int uninitialized;
+	int split_flag1, flags1;
+
+	depth = ext_depth(inode);
+	ex = path[depth].p_ext;
+	ee_block = le32_to_cpu(ex->ee_block);
+	ee_len = ext4_ext_get_actual_len(ex);
+	uninitialized = ext4_ext_is_uninitialized(ex);
+
+	if (map->m_lblk + map->m_len < ee_block + ee_len) {
+		split_flag1 = split_flag & EXT4_EXT_MAY_ZEROOUT ?
+			      EXT4_EXT_MAY_ZEROOUT : 0;
+		flags1 = flags | EXT4_GET_BLOCKS_PRE_IO;
+		if (uninitialized)
+			split_flag1 |= EXT4_EXT_MARK_UNINIT1 |
+				       EXT4_EXT_MARK_UNINIT2;
+		err = ext4_split_extent_at(handle, inode, path,
+				map->m_lblk + map->m_len, split_flag1, flags1);
+	}
+
+	ext4_ext_drop_refs(path);
+	path = ext4_ext_find_extent(inode, map->m_lblk, path);
+	if (IS_ERR(path))
+		return PTR_ERR(path);
+
+	if (map->m_lblk >= ee_block) {
+		split_flag1 = split_flag & EXT4_EXT_MAY_ZEROOUT ?
+			      EXT4_EXT_MAY_ZEROOUT : 0;
+		if (uninitialized)
+			split_flag1 |= EXT4_EXT_MARK_UNINIT1;
+		if (split_flag & EXT4_EXT_MARK_UNINIT2)
+			split_flag1 |= EXT4_EXT_MARK_UNINIT2;
+		err = ext4_split_extent_at(handle, inode, path,
+				map->m_lblk, split_flag1, flags);
+		if (err)
+			goto out;
+	}
+
+	ext4_ext_show_leaf(inode, path);
+out:
+	return err ? err : map->m_len;
+}
+
 #define EXT4_EXT_ZERO_LEN 7
 /*
  * This function is called by ext4_ext_map_blocks() if someone tries to write
-- 
1.7.4.4

[PATCH RFC v1 3/5] ext4:Reimplement convert and split_unwritten.

[Yongqiang Yang]

v0->v1:
  -- ext4_ext_convert_initialized() zeroout whole extent when the extent's
     length is less than 14.

convert and split unwritten are reimplemented based on ext4_split_extent()
added in last patch.

Signed-off-by: Yongqiang Yang <xiaoqiangnk@gmail.com>
---
 fs/ext4/extents.c |  480 ++++++++---------------------------------------------
 1 files changed, 72 insertions(+), 408 deletions(-)

diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c
index db1d67c..9e7c7b3 100644
--- a/fs/ext4/extents.c
+++ b/fs/ext4/extents.c
@@ -2757,17 +2757,13 @@ static int ext4_ext_convert_to_initialized(handle_t *handle,
 					   struct ext4_map_blocks *map,
 					   struct ext4_ext_path *path)
 {
-	struct ext4_extent *ex, newex, orig_ex;
-	struct ext4_extent *ex1 = NULL;
-	struct ext4_extent *ex2 = NULL;
-	struct ext4_extent *ex3 = NULL;
-	struct ext4_extent_header *eh;
+	struct ext4_map_blocks split_map;
+	struct ext4_extent zero_ex;
+	struct ext4_extent *ex;
 	ext4_lblk_t ee_block, eof_block;
 	unsigned int allocated, ee_len, depth;
-	ext4_fsblk_t newblock;
 	int err = 0;
-	int ret = 0;
-	int may_zeroout;
+	int split_flag = 0;
 
 	ext_debug("ext4_ext_convert_to_initialized: inode %lu, logical"
 		"block %llu, max_blocks %u\n", inode->i_ino,
@@ -2779,280 +2775,87 @@ static int ext4_ext_convert_to_initialized(handle_t *handle,
 		eof_block = map->m_lblk + map->m_len;
 
 	depth = ext_depth(inode);
-	eh = path[depth].p_hdr;
 	ex = path[depth].p_ext;
 	ee_block = le32_to_cpu(ex->ee_block);
 	ee_len = ext4_ext_get_actual_len(ex);
 	allocated = ee_len - (map->m_lblk - ee_block);
-	newblock = map->m_lblk - ee_block + ext4_ext_pblock(ex);
-
-	ex2 = ex;
-	orig_ex.ee_block = ex->ee_block;
-	orig_ex.ee_len   = cpu_to_le16(ee_len);
-	ext4_ext_store_pblock(&orig_ex, ext4_ext_pblock(ex));
 
+	WARN_ON(map->m_lblk < ee_block);
 	/*
 	 * It is safe to convert extent to initialized via explicit
 	 * zeroout only if extent is fully insde i_size or new_size.
 	 */
-	may_zeroout = ee_block + ee_len <= eof_block;
+	split_flag |= ee_block + ee_len <= eof_block ? EXT4_EXT_MAY_ZEROOUT : 0;
 
-	err = ext4_ext_get_access(handle, inode, path + depth);
-	if (err)
-		goto out;
 	/* If extent has less than 2*EXT4_EXT_ZERO_LEN zerout directly */
-	if (ee_len <= 2*EXT4_EXT_ZERO_LEN && may_zeroout) {
-		err =  ext4_ext_zeroout(inode, &orig_ex);
+	if (ee_len <= 2*EXT4_EXT_ZERO_LEN &&
+	    (EXT4_EXT_MAY_ZEROOUT & split_flag)) {
+		err = ext4_ext_zeroout(inode, ex);
 		if (err)
-			goto fix_extent_len;
-		/* update the extent length and mark as initialized */
-		ex->ee_block = orig_ex.ee_block;
-		ex->ee_len   = orig_ex.ee_len;
-		ext4_ext_store_pblock(ex, ext4_ext_pblock(&orig_ex));
-		ext4_ext_dirty(handle, inode, path + depth);
-		/* zeroed the full extent */
-		return allocated;
-	}
-
-	/* ex1: ee_block to map->m_lblk - 1 : uninitialized */
-	if (map->m_lblk > ee_block) {
-		ex1 = ex;
-		ex1->ee_len = cpu_to_le16(map->m_lblk - ee_block);
-		ext4_ext_mark_uninitialized(ex1);
-		ex2 = &newex;
-	}
-	/*
-	 * for sanity, update the length of the ex2 extent before
-	 * we insert ex3, if ex1 is NULL. This is to avoid temporary
-	 * overlap of blocks.
-	 */
-	if (!ex1 && allocated > map->m_len)
-		ex2->ee_len = cpu_to_le16(map->m_len);
-	/* ex3: to ee_block + ee_len : uninitialised */
-	if (allocated > map->m_len) {
-		unsigned int newdepth;
-		/* If extent has less than EXT4_EXT_ZERO_LEN zerout directly */
-		if (allocated <= EXT4_EXT_ZERO_LEN && may_zeroout) {
-			/*
-			 * map->m_lblk == ee_block is handled by the zerouout
-			 * at the beginning.
-			 * Mark first half uninitialized.
-			 * Mark second half initialized and zero out the
-			 * initialized extent
-			 */
-			ex->ee_block = orig_ex.ee_block;
-			ex->ee_len   = cpu_to_le16(ee_len - allocated);
-			ext4_ext_mark_uninitialized(ex);
-			ext4_ext_store_pblock(ex, ext4_ext_pblock(&orig_ex));
-			ext4_ext_dirty(handle, inode, path + depth);
-
-			ex3 = &newex;
-			ex3->ee_block = cpu_to_le32(map->m_lblk);
-			ext4_ext_store_pblock(ex3, newblock);
-			ex3->ee_len = cpu_to_le16(allocated);
-			err = ext4_ext_insert_extent(handle, inode, path,
-							ex3, 0);
-			if (err == -ENOSPC) {
-				err =  ext4_ext_zeroout(inode, &orig_ex);
-				if (err)
-					goto fix_extent_len;
-				ex->ee_block = orig_ex.ee_block;
-				ex->ee_len   = orig_ex.ee_len;
-				ext4_ext_store_pblock(ex,
-					ext4_ext_pblock(&orig_ex));
-				ext4_ext_dirty(handle, inode, path + depth);
-				/* blocks available from map->m_lblk */
-				return allocated;
-
-			} else if (err)
-				goto fix_extent_len;
-
-			/*
-			 * We need to zero out the second half because
-			 * an fallocate request can update file size and
-			 * converting the second half to initialized extent
-			 * implies that we can leak some junk data to user
-			 * space.
-			 */
-			err =  ext4_ext_zeroout(inode, ex3);
-			if (err) {
-				/*
-				 * We should actually mark the
-				 * second half as uninit and return error
-				 * Insert would have changed the extent
-				 */
-				depth = ext_depth(inode);
-				ext4_ext_drop_refs(path);
-				path = ext4_ext_find_extent(inode, map->m_lblk,
-							    path);
-				if (IS_ERR(path)) {
-					err = PTR_ERR(path);
-					return err;
-				}
-				/* get the second half extent details */
-				ex = path[depth].p_ext;
-				err = ext4_ext_get_access(handle, inode,
-								path + depth);
-				if (err)
-					return err;
-				ext4_ext_mark_uninitialized(ex);
-				ext4_ext_dirty(handle, inode, path + depth);
-				return err;
-			}
-
-			/* zeroed the second half */
-			return allocated;
-		}
-		ex3 = &newex;
-		ex3->ee_block = cpu_to_le32(map->m_lblk + map->m_len);
-		ext4_ext_store_pblock(ex3, newblock + map->m_len);
-		ex3->ee_len = cpu_to_le16(allocated - map->m_len);
-		ext4_ext_mark_uninitialized(ex3);
-		err = ext4_ext_insert_extent(handle, inode, path, ex3, 0);
-		if (err == -ENOSPC && may_zeroout) {
-			err =  ext4_ext_zeroout(inode, &orig_ex);
-			if (err)
-				goto fix_extent_len;
-			/* update the extent length and mark as initialized */
-			ex->ee_block = orig_ex.ee_block;
-			ex->ee_len   = orig_ex.ee_len;
-			ext4_ext_store_pblock(ex, ext4_ext_pblock(&orig_ex));
-			ext4_ext_dirty(handle, inode, path + depth);
-			/* zeroed the full extent */
-			/* blocks available from map->m_lblk */
-			return allocated;
-
-		} else if (err)
-			goto fix_extent_len;
-		/*
-		 * The depth, and hence eh & ex might change
-		 * as part of the insert above.
-		 */
-		newdepth = ext_depth(inode);
-		/*
-		 * update the extent length after successful insert of the
-		 * split extent
-		 */
-		ee_len -= ext4_ext_get_actual_len(ex3);
-		orig_ex.ee_len = cpu_to_le16(ee_len);
-		may_zeroout = ee_block + ee_len <= eof_block;
-
-		depth = newdepth;
-		ext4_ext_drop_refs(path);
-		path = ext4_ext_find_extent(inode, map->m_lblk, path);
-		if (IS_ERR(path)) {
-			err = PTR_ERR(path);
 			goto out;
-		}
-		eh = path[depth].p_hdr;
-		ex = path[depth].p_ext;
-		if (ex2 != &newex)
-			ex2 = ex;
 
 		err = ext4_ext_get_access(handle, inode, path + depth);
 		if (err)
 			goto out;
-
-		allocated = map->m_len;
-
-		/* If extent has less than EXT4_EXT_ZERO_LEN and we are trying
-		 * to insert a extent in the middle zerout directly
-		 * otherwise give the extent a chance to merge to left
-		 */
-		if (le16_to_cpu(orig_ex.ee_len) <= EXT4_EXT_ZERO_LEN &&
-			map->m_lblk != ee_block && may_zeroout) {
-			err =  ext4_ext_zeroout(inode, &orig_ex);
-			if (err)
-				goto fix_extent_len;
-			/* update the extent length and mark as initialized */
-			ex->ee_block = orig_ex.ee_block;
-			ex->ee_len   = orig_ex.ee_len;
-			ext4_ext_store_pblock(ex, ext4_ext_pblock(&orig_ex));
-			ext4_ext_dirty(handle, inode, path + depth);
-			/* zero out the first half */
-			/* blocks available from map->m_lblk */
-			return allocated;
-		}
-	}
-	/*
-	 * If there was a change of depth as part of the
-	 * insertion of ex3 above, we need to update the length
-	 * of the ex1 extent again here
-	 */
-	if (ex1 && ex1 != ex) {
-		ex1 = ex;
-		ex1->ee_len = cpu_to_le16(map->m_lblk - ee_block);
-		ext4_ext_mark_uninitialized(ex1);
-		ex2 = &newex;
-	}
-	/* ex2: map->m_lblk to map->m_lblk + maxblocks-1 : initialised */
-	ex2->ee_block = cpu_to_le32(map->m_lblk);
-	ext4_ext_store_pblock(ex2, newblock);
-	ex2->ee_len = cpu_to_le16(allocated);
-	if (ex2 != ex)
-		goto insert;
-	/*
-	 * New (initialized) extent starts from the first block
-	 * in the current extent. i.e., ex2 == ex
-	 * We have to see if it can be merged with the extent
-	 * on the left.
-	 */
-	if (ex2 > EXT_FIRST_EXTENT(eh)) {
-		/*
-		 * To merge left, pass "ex2 - 1" to try_to_merge(),
-		 * since it merges towards right _only_.
-		 */
-		ret = ext4_ext_try_to_merge(inode, path, ex2 - 1);
-		if (ret) {
-			err = ext4_ext_correct_indexes(handle, inode, path);
-			if (err)
-				goto out;
-			depth = ext_depth(inode);
-			ex2--;
-		}
+		ext4_ext_mark_initialized(ex);
+		ext4_ext_try_to_merge(inode, path, ex);
+		err = ext4_ext_dirty(handle, inode, path + depth);
+		goto out;
 	}
+
 	/*
-	 * Try to Merge towards right. This might be required
-	 * only when the whole extent is being written to.
-	 * i.e. ex2 == ex and ex3 == NULL.
+	 * four cases:
+	 * 1. split the extent into three extents.
+	 * 2. split the extent into two extents, zeroout the first half.
+	 * 3. split the extent into two extents, zeroout the second half.
+	 * 4. split the extent into two extents with out zeroout.
 	 */
-	if (!ex3) {
-		ret = ext4_ext_try_to_merge(inode, path, ex2);
-		if (ret) {
-			err = ext4_ext_correct_indexes(handle, inode, path);
+	split_map.m_lblk = map->m_lblk;
+	split_map.m_len = map->m_len;
+
+	if (allocated > map->m_len) {
+		if (allocated <= EXT4_EXT_ZERO_LEN &&
+		    (EXT4_EXT_MAY_ZEROOUT & split_flag)) {
+			/* case 3 */
+			zero_ex.ee_block =
+					 cpu_to_le32(map->m_lblk + map->m_len);
+			zero_ex.ee_len = cpu_to_le16(allocated - map->m_len);
+			ext4_ext_store_pblock(&zero_ex,
+				ext4_ext_pblock(ex) + map->m_lblk - ee_block);
+			err = ext4_ext_zeroout(inode, &zero_ex);
 			if (err)
 				goto out;
+			split_map.m_lblk = map->m_lblk;
+			split_map.m_len = allocated;
+		} else if ((map->m_lblk - ee_block + map->m_len <
+			   EXT4_EXT_ZERO_LEN) &&
+			   (EXT4_EXT_MAY_ZEROOUT & split_flag)) {
+			/* case 2 */
+			if (map->m_lblk != ee_block) {
+				zero_ex.ee_block = ex->ee_block;
+				zero_ex.ee_len = cpu_to_le16(map->m_lblk -
+							ee_block);
+				ext4_ext_store_pblock(&zero_ex,
+						      ext4_ext_pblock(ex));
+				err = ext4_ext_zeroout(inode, &zero_ex);
+				if (err)
+					goto out;
+			}
+
+			allocated = map->m_lblk - ee_block + map->m_len;
+
+			split_map.m_lblk = ee_block;
+			split_map.m_len = allocated;
 		}
 	}
-	/* Mark modified extent as dirty */
-	err = ext4_ext_dirty(handle, inode, path + depth);
-	goto out;
-insert:
-	err = ext4_ext_insert_extent(handle, inode, path, &newex, 0);
-	if (err == -ENOSPC && may_zeroout) {
-		err =  ext4_ext_zeroout(inode, &orig_ex);
-		if (err)
-			goto fix_extent_len;
-		/* update the extent length and mark as initialized */
-		ex->ee_block = orig_ex.ee_block;
-		ex->ee_len   = orig_ex.ee_len;
-		ext4_ext_store_pblock(ex, ext4_ext_pblock(&orig_ex));
-		ext4_ext_dirty(handle, inode, path + depth);
-		/* zero out the first half */
-		return allocated;
-	} else if (err)
-		goto fix_extent_len;
+
+	allocated = ext4_split_extent(handle, inode, path,
+				       &split_map, split_flag, 0);
+	if (allocated < 0)
+		err = allocated;
+
 out:
-	ext4_ext_show_leaf(inode, path);
 	return err ? err : allocated;
-
-fix_extent_len:
-	ex->ee_block = orig_ex.ee_block;
-	ex->ee_len   = orig_ex.ee_len;
-	ext4_ext_store_pblock(ex, ext4_ext_pblock(&orig_ex));
-	ext4_ext_mark_uninitialized(ex);
-	ext4_ext_dirty(handle, inode, path + depth);
-	return err;
 }
 
 /*
@@ -3083,15 +2886,11 @@ static int ext4_split_unwritten_extents(handle_t *handle,
 					struct ext4_ext_path *path,
 					int flags)
 {
-	struct ext4_extent *ex, newex, orig_ex;
-	struct ext4_extent *ex1 = NULL;
-	struct ext4_extent *ex2 = NULL;
-	struct ext4_extent *ex3 = NULL;
-	ext4_lblk_t ee_block, eof_block;
-	unsigned int allocated, ee_len, depth;
-	ext4_fsblk_t newblock;
-	int err = 0;
-	int may_zeroout;
+	ext4_lblk_t eof_block;
+	ext4_lblk_t ee_block;
+	struct ext4_extent *ex;
+	unsigned int ee_len;
+	int split_flag = 0, depth;
 
 	ext_debug("ext4_split_unwritten_extents: inode %lu, logical"
 		"block %llu, max_blocks %u\n", inode->i_ino,
@@ -3101,155 +2900,20 @@ static int ext4_split_unwritten_extents(handle_t *handle,
 		inode->i_sb->s_blocksize_bits;
 	if (eof_block < map->m_lblk + map->m_len)
 		eof_block = map->m_lblk + map->m_len;
-
-	depth = ext_depth(inode);
-	ex = path[depth].p_ext;
-	ee_block = le32_to_cpu(ex->ee_block);
-	ee_len = ext4_ext_get_actual_len(ex);
-	allocated = ee_len - (map->m_lblk - ee_block);
-	newblock = map->m_lblk - ee_block + ext4_ext_pblock(ex);
-
-	ex2 = ex;
-	orig_ex.ee_block = ex->ee_block;
-	orig_ex.ee_len   = cpu_to_le16(ee_len);
-	ext4_ext_store_pblock(&orig_ex, ext4_ext_pblock(ex));
-
 	/*
 	 * It is safe to convert extent to initialized via explicit
 	 * zeroout only if extent is fully insde i_size or new_size.
 	 */
-	may_zeroout = ee_block + ee_len <= eof_block;
-
-	/*
- 	 * If the uninitialized extent begins at the same logical
- 	 * block where the write begins, and the write completely
- 	 * covers the extent, then we don't need to split it.
- 	 */
-	if ((map->m_lblk == ee_block) && (allocated <= map->m_len))
-		return allocated;
-
-	err = ext4_ext_get_access(handle, inode, path + depth);
-	if (err)
-		goto out;
-	/* ex1: ee_block to map->m_lblk - 1 : uninitialized */
-	if (map->m_lblk > ee_block) {
-		ex1 = ex;
-		ex1->ee_len = cpu_to_le16(map->m_lblk - ee_block);
-		ext4_ext_mark_uninitialized(ex1);
-		ex2 = &newex;
-	}
-	/*
-	 * for sanity, update the length of the ex2 extent before
-	 * we insert ex3, if ex1 is NULL. This is to avoid temporary
-	 * overlap of blocks.
-	 */
-	if (!ex1 && allocated > map->m_len)
-		ex2->ee_len = cpu_to_le16(map->m_len);
-	/* ex3: to ee_block + ee_len : uninitialised */
-	if (allocated > map->m_len) {
-		unsigned int newdepth;
-		ex3 = &newex;
-		ex3->ee_block = cpu_to_le32(map->m_lblk + map->m_len);
-		ext4_ext_store_pblock(ex3, newblock + map->m_len);
-		ex3->ee_len = cpu_to_le16(allocated - map->m_len);
-		ext4_ext_mark_uninitialized(ex3);
-		err = ext4_ext_insert_extent(handle, inode, path, ex3, flags);
-		if (err == -ENOSPC && may_zeroout) {
-			err =  ext4_ext_zeroout(inode, &orig_ex);
-			if (err)
-				goto fix_extent_len;
-			/* update the extent length and mark as initialized */
-			ex->ee_block = orig_ex.ee_block;
-			ex->ee_len   = orig_ex.ee_len;
-			ext4_ext_store_pblock(ex, ext4_ext_pblock(&orig_ex));
-			ext4_ext_dirty(handle, inode, path + depth);
-			/* zeroed the full extent */
-			/* blocks available from map->m_lblk */
-			return allocated;
-
-		} else if (err)
-			goto fix_extent_len;
-		/*
-		 * The depth, and hence eh & ex might change
-		 * as part of the insert above.
-		 */
-		newdepth = ext_depth(inode);
-		/*
-		 * update the extent length after successful insert of the
-		 * split extent
-		 */
-		ee_len -= ext4_ext_get_actual_len(ex3);
-		orig_ex.ee_len = cpu_to_le16(ee_len);
-		may_zeroout = ee_block + ee_len <= eof_block;
-
-		depth = newdepth;
-		ext4_ext_drop_refs(path);
-		path = ext4_ext_find_extent(inode, map->m_lblk, path);
-		if (IS_ERR(path)) {
-			err = PTR_ERR(path);
-			goto out;
-		}
-		ex = path[depth].p_ext;
-		if (ex2 != &newex)
-			ex2 = ex;
+	depth = ext_depth(inode);
+	ex = path[depth].p_ext;
+	ee_block = le32_to_cpu(ex->ee_block);
+	ee_len = ext4_ext_get_actual_len(ex);
 
-		err = ext4_ext_get_access(handle, inode, path + depth);
-		if (err)
-			goto out;
+	split_flag |= ee_block + ee_len <= eof_block ? EXT4_EXT_MAY_ZEROOUT : 0;
+	split_flag |= EXT4_EXT_MARK_UNINIT2;
 
-		allocated = map->m_len;
-	}
-	/*
-	 * If there was a change of depth as part of the
-	 * insertion of ex3 above, we need to update the length
-	 * of the ex1 extent again here
-	 */
-	if (ex1 && ex1 != ex) {
-		ex1 = ex;
-		ex1->ee_len = cpu_to_le16(map->m_lblk - ee_block);
-		ext4_ext_mark_uninitialized(ex1);
-		ex2 = &newex;
-	}
-	/*
-	 * ex2: map->m_lblk to map->m_lblk + map->m_len-1 : to be written
-	 * using direct I/O, uninitialised still.
-	 */
-	ex2->ee_block = cpu_to_le32(map->m_lblk);
-	ext4_ext_store_pblock(ex2, newblock);
-	ex2->ee_len = cpu_to_le16(allocated);
-	ext4_ext_mark_uninitialized(ex2);
-	if (ex2 != ex)
-		goto insert;
-	/* Mark modified extent as dirty */
-	err = ext4_ext_dirty(handle, inode, path + depth);
-	ext_debug("out here\n");
-	goto out;
-insert:
-	err = ext4_ext_insert_extent(handle, inode, path, &newex, flags);
-	if (err == -ENOSPC && may_zeroout) {
-		err =  ext4_ext_zeroout(inode, &orig_ex);
-		if (err)
-			goto fix_extent_len;
-		/* update the extent length and mark as initialized */
-		ex->ee_block = orig_ex.ee_block;
-		ex->ee_len   = orig_ex.ee_len;
-		ext4_ext_store_pblock(ex, ext4_ext_pblock(&orig_ex));
-		ext4_ext_dirty(handle, inode, path + depth);
-		/* zero out the first half */
-		return allocated;
-	} else if (err)
-		goto fix_extent_len;
-out:
-	ext4_ext_show_leaf(inode, path);
-	return err ? err : allocated;

[PATCH RFC v1 4/5] ext4: Add a function ext4_ext_zeroout_mem().

[Yongqiang Yang]

ext4_ext_zeroout_mem() zero out blocks in mem.

Signed-off-by: Yongqiang Yang <xiaoqiangnk@gmail.com>
---
 fs/ext4/extents.c |   70 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 70 insertions(+), 0 deletions(-)

diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c
index 9e7c7b3..30663b6 100644
--- a/fs/ext4/extents.c
+++ b/fs/ext4/extents.c
@@ -2555,6 +2555,76 @@ static int ext4_ext_zeroout(struct inode *inode, struct ext4_extent *ex)
 }
 
 /*
+ * ext4_ext_zeroout_mem() zero specified pages in page cache.
+ * this function is used by ext4_ext_convert_to_initialized() to
+ * zeroout small extents.
+ *
+ * @inode: the file inode
+ * @ex: the extent to be zeroout
+ *
+ * return 0 on success
+ */
+static int ext4_ext_zeroout_mem(struct inode *inode,
+				  struct ext4_extent *ex)
+{
+	ext4_fsblk_t pblk;
+	ext4_lblk_t lblk;
+	ext4_lblk_t end;
+	struct page *page;
+	struct buffer_head *bh;
+	struct block_device *bdev;
+	pgoff_t index;
+	unsigned blocksize, blks_per_page_bits;
+
+	bdev = inode->i_sb->s_bdev;
+
+	blocksize = 1 << inode->i_blkbits;
+	blks_per_page_bits = PAGE_CACHE_SHIFT - inode->i_blkbits;
+
+	pblk = ext4_ext_pblock(ex);
+	lblk = le32_to_cpu(ex->ee_block);
+	end = lblk + ext4_ext_get_actual_len(ex);
+
+	while (lblk < end) {
+		struct ext4_extent ext;
+		char *kaddr;
+
+		/* grab page */
+		index = lblk >> blks_per_page_bits;
+		page = grab_cache_page_write_begin(inode->i_mapping,
+						   index, AOP_FLAG_NOFS);
+		if (!page) {
+			ext.ee_block = cpu_to_le32(lblk);
+			ext.ee_len = cpu_to_le16(end - lblk);
+			ext4_ext_store_pblock(&ext, pblk);
+			return ext4_ext_zeroout(inode, &ext);
+		}
+
+		/* map buffers, mark dirty */
+		if (!page_has_buffers(page))
+			create_empty_buffers(page, blocksize, 0);
+
+		kaddr = kmap_atomic(page, KM_USER0);
+		bh = page_buffers(page);
+		do {
+			unmap_underlying_metadata(bdev, pblk);
+			memset(kaddr, 0, blocksize);
+			map_bh(bh, inode->i_sb, pblk++);
+			set_buffer_uptodate(bh);
+			mark_buffer_dirty(bh);
+			bh = bh->b_this_page;
+			lblk++;
+		} while (bh != page_buffers(page) && lblk < end);
+		kunmap_atomic(kaddr, KM_USER0);
+
+		unlock_page(page);
+		page_cache_release(page);
+	}
+
+	return 0;
+}
+
+/*
  * used by extent splitting.
  */
 #define EXT4_EXT_MAY_ZEROOUT	0x1  /* safe to zeroout if split fails \
-- 
1.7.4.4

[PATCH RFC v1 5/5] ext4: optimize ext4_ext_convert_to_initialized().

[Yongqiang Yang]

1] Rename ext4_ext_zroout() to ext4_ext_zeroout_dev().

2] Optimize ext4_ext_convert_to_initialized() by using
   ext4_ext_zeroout_mem().

Signed-off-by: Yongqiang Yang <xiaoqiangnk@gmail.com>
---
 fs/ext4/extents.c |   32 ++++++++++++++++++++++++--------
 1 files changed, 24 insertions(+), 8 deletions(-)

diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c
index 30663b6..0fc4ca1 100644
--- a/fs/ext4/extents.c
+++ b/fs/ext4/extents.c
@@ -2538,7 +2538,7 @@ void ext4_ext_release(struct super_block *sb)
 }
 
 /* FIXME!! we need to try to merge to left or right after zero-out  */
-static int ext4_ext_zeroout(struct inode *inode, struct ext4_extent *ex)
+static int ext4_ext_zeroout_dev(struct inode *inode, struct ext4_extent *ex)
 {
 	ext4_fsblk_t ee_pblock;
 	unsigned int ee_len;
@@ -2597,7 +2597,7 @@ static int ext4_ext_zeroout_mem(struct inode *inode,
 			ext.ee_block = cpu_to_le32(lblk);
 			ext.ee_len = cpu_to_le16(end - lblk);
 			ext4_ext_store_pblock(&ext, pblk);
-			return ext4_ext_zeroout(inode, &ext);
+			return ext4_ext_zeroout_dev(inode, &ext);
 		}
 
 		/* map buffers, mark dirty */
@@ -2725,7 +2725,7 @@ static int ext4_split_extent_at(handle_t *handle,
 
 	err = ext4_ext_insert_extent(handle, inode, path, &newex, flags);
 	if (err == -ENOSPC && (EXT4_EXT_MAY_ZEROOUT & split_flag)) {
-		err = ext4_ext_zeroout(inode, &orig_ex);
+		err = ext4_ext_zeroout_dev(inode, &orig_ex);
 		if (err)
 			goto fix_extent_len;
 		/* update the extent length and mark as initialized */
@@ -2860,9 +2860,25 @@ static int ext4_ext_convert_to_initialized(handle_t *handle,
 	/* If extent has less than 2*EXT4_EXT_ZERO_LEN zerout directly */
 	if (ee_len <= 2*EXT4_EXT_ZERO_LEN &&
 	    (EXT4_EXT_MAY_ZEROOUT & split_flag)) {
-		err = ext4_ext_zeroout(inode, ex);
-		if (err)
-			goto out;
+		if (map->m_lblk > ee_block) {
+			zero_ex.ee_block = ex->ee_block;
+			zero_ex.ee_len = cpu_to_le16(map->m_lblk - ee_block);
+			ext4_ext_store_pblock(&zero_ex, ext4_ext_pblock(ex));
+			err = ext4_ext_zeroout_mem(inode, &zero_ex);
+			if (err)
+				goto out;
+		}
+
+		if (map->m_lblk + map->m_len < ee_block + ee_len) {
+			zero_ex.ee_block = cpu_to_le32(map->m_lblk +
+						       map->m_len);
+			zero_ex.ee_len = cpu_to_le16(allocated - map->m_len);
+			ext4_ext_store_pblock(&zero_ex, ext4_ext_pblock(ex) +
+					(map->m_lblk - ee_block) + map->m_len);
+			err = ext4_ext_zeroout_mem(inode, &zero_ex);
+			if (err)
+				goto out;
+		}
 
 		err = ext4_ext_get_access(handle, inode, path + depth);
 		if (err)
@@ -2892,7 +2908,7 @@ static int ext4_ext_convert_to_initialized(handle_t *handle,
 			zero_ex.ee_len = cpu_to_le16(allocated - map->m_len);
 			ext4_ext_store_pblock(&zero_ex,
 				ext4_ext_pblock(ex) + map->m_lblk - ee_block);
-			err = ext4_ext_zeroout(inode, &zero_ex);
+			err = ext4_ext_zeroout_mem(inode, &zero_ex);
 			if (err)
 				goto out;
 			split_map.m_lblk = map->m_lblk;
@@ -2907,7 +2923,7 @@ static int ext4_ext_convert_to_initialized(handle_t *handle,
 							ee_block);
 				ext4_ext_store_pblock(&zero_ex,
 						      ext4_ext_pblock(ex));
-				err = ext4_ext_zeroout(inode, &zero_ex);
+				err = ext4_ext_zeroout_mem(inode, &zero_ex);
 				if (err)
 					goto out;
 			}
-- 
1.7.4.4

Re: [PATCH RFC v1 0/5]Factor common code from convert and split unwritten.

[Allison Henderson]

On 4/23/2011 1:44 AM, Yongqiang Yang wrote:
> v0->v1:
>     fix a bug in ext4_ext_convert_to_initialized() reported by
>     Allison<achender@linux.vnet.ibm.com>.
>
>     optimize ext4_ext_convert_to_initialized().
>
> -- factor common code
>     These patches factor common code from ext4_ext_convert_to_initialized() and
>     ext4_split_unwritten_extents() so that extent-move-on-write in snapshot and
>     punch hole can be built on the common code.
>
> -- optimization
>     the 4th and the 5th patch optimize ext4_ext_convert_to_initialized() by
>     zeroing out in memory.
>
>
> [PATCH RFC v1 1/5] ext4:Add a function merging extent right and left.
> [PATCH RFC v1 2/5] ext4:Add two functions splitting an extent.
> [PATCH RFC v1 3/5] ext4:Reimplement convert and split_unwritten.
> [PATCH RFC v1 4/5] ext4: Add a function ext4_ext_zeroout_mem().
> [PATCH RFC v1 5/5] ext4: optimize ext4_ext_convert_to_initialized().
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Hi there,

Just an update on your patch set.  Im am working on getting the punch 
hole patch to work with this new set, but I'm am having trouble getting 
it through the stress test.  It gets up to around 48265 file operations, 
and then hangs. So I am currently trying to narrow down the problem.  It 
looks like it does it with or with out the extra punch hole patches, but 
you may need to enable fallocate in the fsx Makefile to recreate the 
problem.  I will keep you posted if I find any more clues.

Allison Henderson

Re: [PATCH RFC v1 0/5]Factor common code from convert and split unwritten.

[Yongqiang Yang]

On Wed, Apr 27, 2011 at 3:08 AM, Allison Henderson
<achender@linux.vnet.ibm.com> wrote:
> On 4/23/2011 1:44 AM, Yongqiang Yang wrote:
>>
>> v0->v1:
>>    fix a bug in ext4_ext_convert_to_initialized() reported by
>>    Allison<achender@linux.vnet.ibm.com>.
>>
>>    optimize ext4_ext_convert_to_initialized().
>>
>> -- factor common code
>>    These patches factor common code from ext4_ext_convert_to_initialized()
>> and
>>    ext4_split_unwritten_extents() so that extent-move-on-write in snapshot
>> and
>>    punch hole can be built on the common code.
>>
>> -- optimization
>>    the 4th and the 5th patch optimize ext4_ext_convert_to_initialized() by
>>    zeroing out in memory.
>>
>>
>> [PATCH RFC v1 1/5] ext4:Add a function merging extent right and left.
>> [PATCH RFC v1 2/5] ext4:Add two functions splitting an extent.
>> [PATCH RFC v1 3/5] ext4:Reimplement convert and split_unwritten.
>> [PATCH RFC v1 4/5] ext4: Add a function ext4_ext_zeroout_mem().
>> [PATCH RFC v1 5/5] ext4: optimize ext4_ext_convert_to_initialized().
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
> Hi there,
>
> Just an update on your patch set.  Im am working on getting the punch hole
> patch to work with this new set, but I'm am having trouble getting it
> through the stress test.  It gets up to around 48265 file operations, and
> then hangs. So I am currently trying to narrow down the problem.  It looks
> like it does it with or with out the extra punch hole patches, but you may
> need to enable fallocate in the fsx Makefile to recreate the problem.  I
> will keep you posted if I find any more clues.
ok. Thank you.
>
> Allison Henderson
>



-- 
Best Wishes
Yongqiang Yang
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH RFC v1 0/5]Factor common code from convert and split unwritten.

[Yongqiang Yang]

On Wed, Apr 27, 2011 at 3:08 AM, Allison Henderson
<achender@linux.vnet.ibm.com> wrote:
> On 4/23/2011 1:44 AM, Yongqiang Yang wrote:
>>
>> v0->v1:
>>    fix a bug in ext4_ext_convert_to_initialized() reported by
>>    Allison<achender@linux.vnet.ibm.com>.
>>
>>    optimize ext4_ext_convert_to_initialized().
>>
>> -- factor common code
>>    These patches factor common code from ext4_ext_convert_to_initialized()
>> and
>>    ext4_split_unwritten_extents() so that extent-move-on-write in snapshot
>> and
>>    punch hole can be built on the common code.
>>
>> -- optimization
>>    the 4th and the 5th patch optimize ext4_ext_convert_to_initialized() by
>>    zeroing out in memory.
>>
>>
>> [PATCH RFC v1 1/5] ext4:Add a function merging extent right and left.
>> [PATCH RFC v1 2/5] ext4:Add two functions splitting an extent.
>> [PATCH RFC v1 3/5] ext4:Reimplement convert and split_unwritten.
>> [PATCH RFC v1 4/5] ext4: Add a function ext4_ext_zeroout_mem().
>> [PATCH RFC v1 5/5] ext4: optimize ext4_ext_convert_to_initialized().
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
> Hi there,
>
> Just an update on your patch set.  Im am working on getting the punch hole
> patch to work with this new set, but I'm am having trouble getting it
> through the stress test.  It gets up to around 48265 file operations, and
> then hangs. So I am currently trying to narrow down the problem.  It looks
> like it does it with or with out the extra punch hole patches, but you may
> need to enable fallocate in the fsx Makefile to recreate the problem.  I
> will keep you posted if I find any more clues.
Hi,

Could you tell me how to get fsx?  I can not find that.

Thank you,
Yongqiang.
>
> Allison Henderson
>



-- 
Best Wishes
Yongqiang Yang
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH RFC v1 0/5]Factor common code from convert and split unwritten.

[Allison Henderson]

On 4/26/2011 9:48 PM, Yongqiang Yang wrote:
> On Wed, Apr 27, 2011 at 3:08 AM, Allison Henderson
> <achender@linux.vnet.ibm.com>  wrote:
>> On 4/23/2011 1:44 AM, Yongqiang Yang wrote:
>>>
>>> v0->v1:
>>>     fix a bug in ext4_ext_convert_to_initialized() reported by
>>>     Allison<achender@linux.vnet.ibm.com>.
>>>
>>>     optimize ext4_ext_convert_to_initialized().
>>>
>>> -- factor common code
>>>     These patches factor common code from ext4_ext_convert_to_initialized()
>>> and
>>>     ext4_split_unwritten_extents() so that extent-move-on-write in snapshot
>>> and
>>>     punch hole can be built on the common code.
>>>
>>> -- optimization
>>>     the 4th and the 5th patch optimize ext4_ext_convert_to_initialized() by
>>>     zeroing out in memory.
>>>
>>>
>>> [PATCH RFC v1 1/5] ext4:Add a function merging extent right and left.
>>> [PATCH RFC v1 2/5] ext4:Add two functions splitting an extent.
>>> [PATCH RFC v1 3/5] ext4:Reimplement convert and split_unwritten.
>>> [PATCH RFC v1 4/5] ext4: Add a function ext4_ext_zeroout_mem().
>>> [PATCH RFC v1 5/5] ext4: optimize ext4_ext_convert_to_initialized().
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>> Hi there,
>>
>> Just an update on your patch set.  Im am working on getting the punch hole
>> patch to work with this new set, but I'm am having trouble getting it
>> through the stress test.  It gets up to around 48265 file operations, and
>> then hangs. So I am currently trying to narrow down the problem.  It looks
>> like it does it with or with out the extra punch hole patches, but you may
>> need to enable fallocate in the fsx Makefile to recreate the problem.  I
>> will keep you posted if I find any more clues.
> Hi,
>
> Could you tell me how to get fsx?  I can not find that.

Hi there,

I believe you can find it in both xfstests and ltp.  The one I am using 
I got from xfstests here:

http://xfs.org/index.php/Getting_the_latest_source_code

Once you have it configured and built, there is a sub folder called ltp. 
  Execute this command from that folder:

./fsx -d -b 1 -N 100000 -S 1 /mnt/ext4MntPt/holePunch/testFile

Where "/mnt/ext4MntPt/holePunch/testFile" is a file on an ext4 file 
system.  It should then try to run through 100000 random file 
operations, but get stuck on number 48256.

If you do not see any fallocate operations running, you may have to go 
enable it in the ltp/Makefile.  I had to change "ifeq 
($(HAVE_FALLOCATE), true)" to "ifeq ($(HAVE_FALLOCATE), yes)" to allow 
the extra fallocate code to compile in.  Let me know if you have any 
trouble recreating the bug.

>
> Thank you,
> Yongqiang.
>>
>> Allison Henderson
>>
>
>
>

Re: [PATCH RFC v1 0/5]Factor common code from convert and split unwritten.

[Yongqiang Yang]

On Wed, Apr 27, 2011 at 2:34 PM, Allison Henderson
<achender@linux.vnet.ibm.com> wrote:
> On 4/26/2011 9:48 PM, Yongqiang Yang wrote:
>>
>> On Wed, Apr 27, 2011 at 3:08 AM, Allison Henderson
>> <achender@linux.vnet.ibm.com>  wrote:
>>>
>>> On 4/23/2011 1:44 AM, Yongqiang Yang wrote:
>>>>
>>>> v0->v1:
>>>>    fix a bug in ext4_ext_convert_to_initialized() reported by
>>>>    Allison<achender@linux.vnet.ibm.com>.
>>>>
>>>>    optimize ext4_ext_convert_to_initialized().
>>>>
>>>> -- factor common code
>>>>    These patches factor common code from
>>>> ext4_ext_convert_to_initialized()
>>>> and
>>>>    ext4_split_unwritten_extents() so that extent-move-on-write in
>>>> snapshot
>>>> and
>>>>    punch hole can be built on the common code.
>>>>
>>>> -- optimization
>>>>    the 4th and the 5th patch optimize ext4_ext_convert_to_initialized()
>>>> by
>>>>    zeroing out in memory.
>>>>
>>>>
>>>> [PATCH RFC v1 1/5] ext4:Add a function merging extent right and left.
>>>> [PATCH RFC v1 2/5] ext4:Add two functions splitting an extent.
>>>> [PATCH RFC v1 3/5] ext4:Reimplement convert and split_unwritten.
>>>> [PATCH RFC v1 4/5] ext4: Add a function ext4_ext_zeroout_mem().
>>>> [PATCH RFC v1 5/5] ext4: optimize ext4_ext_convert_to_initialized().
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>> Hi there,
>>>
>>> Just an update on your patch set.  Im am working on getting the punch
>>> hole
>>> patch to work with this new set, but I'm am having trouble getting it
>>> through the stress test.  It gets up to around 48265 file operations, and
>>> then hangs. So I am currently trying to narrow down the problem.  It
>>> looks
>>> like it does it with or with out the extra punch hole patches, but you
>>> may
>>> need to enable fallocate in the fsx Makefile to recreate the problem.  I
>>> will keep you posted if I find any more clues.
>>
>> Hi,
>>
>> Could you tell me how to get fsx?  I can not find that.
>
> Hi there,
>
> I believe you can find it in both xfstests and ltp.  The one I am using I
> got from xfstests here:
>
> http://xfs.org/index.php/Getting_the_latest_source_code
>
> Once you have it configured and built, there is a sub folder called ltp.
>  Execute this command from that folder:
>
> ./fsx -d -b 1 -N 100000 -S 1 /mnt/ext4MntPt/holePunch/testFile
>
> Where "/mnt/ext4MntPt/holePunch/testFile" is a file on an ext4 file system.
>  It should then try to run through 100000 random file operations, but get
> stuck on number 48256.
>
> If you do not see any fallocate operations running, you may have to go
> enable it in the ltp/Makefile.  I had to change "ifeq ($(HAVE_FALLOCATE),
> true)" to "ifeq ($(HAVE_FALLOCATE), yes)" to allow the extra fallocate code
> to compile in.  Let me know if you have any trouble recreating the bug.
Thank you.
>
>>
>> Thank you,
>> Yongqiang.
>>>
>>> Allison Henderson
>>>
>>
>>
>>
>
>



-- 
Best Wishes
Yongqiang Yang
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH RFC v1 0/5]Factor common code from convert and split unwritten.

[Yongqiang Yang]

Hi Allison,

Could you send punch hole patch and fsx(if you modified it) to me?  I
can not reproduce the bug on my system without punch hole.

Thank you,
Yongqiang.

On Wed, Apr 27, 2011 at 2:34 PM, Allison Henderson
<achender@linux.vnet.ibm.com> wrote:
> On 4/26/2011 9:48 PM, Yongqiang Yang wrote:
>>
>> On Wed, Apr 27, 2011 at 3:08 AM, Allison Henderson
>> <achender@linux.vnet.ibm.com>  wrote:
>>>
>>> On 4/23/2011 1:44 AM, Yongqiang Yang wrote:
>>>>
>>>> v0->v1:
>>>>    fix a bug in ext4_ext_convert_to_initialized() reported by
>>>>    Allison<achender@linux.vnet.ibm.com>.
>>>>
>>>>    optimize ext4_ext_convert_to_initialized().
>>>>
>>>> -- factor common code
>>>>    These patches factor common code from
>>>> ext4_ext_convert_to_initialized()
>>>> and
>>>>    ext4_split_unwritten_extents() so that extent-move-on-write in
>>>> snapshot
>>>> and
>>>>    punch hole can be built on the common code.
>>>>
>>>> -- optimization
>>>>    the 4th and the 5th patch optimize ext4_ext_convert_to_initialized()
>>>> by
>>>>    zeroing out in memory.
>>>>
>>>>
>>>> [PATCH RFC v1 1/5] ext4:Add a function merging extent right and left.
>>>> [PATCH RFC v1 2/5] ext4:Add two functions splitting an extent.
>>>> [PATCH RFC v1 3/5] ext4:Reimplement convert and split_unwritten.
>>>> [PATCH RFC v1 4/5] ext4: Add a function ext4_ext_zeroout_mem().
>>>> [PATCH RFC v1 5/5] ext4: optimize ext4_ext_convert_to_initialized().
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>> Hi there,
>>>
>>> Just an update on your patch set.  Im am working on getting the punch
>>> hole
>>> patch to work with this new set, but I'm am having trouble getting it
>>> through the stress test.  It gets up to around 48265 file operations, and
>>> then hangs. So I am currently trying to narrow down the problem.  It
>>> looks
>>> like it does it with or with out the extra punch hole patches, but you
>>> may
>>> need to enable fallocate in the fsx Makefile to recreate the problem.  I
>>> will keep you posted if I find any more clues.
>>
>> Hi,
>>
>> Could you tell me how to get fsx?  I can not find that.
>
> Hi there,
>
> I believe you can find it in both xfstests and ltp.  The one I am using I
> got from xfstests here:
>
> http://xfs.org/index.php/Getting_the_latest_source_code
>
> Once you have it configured and built, there is a sub folder called ltp.
>  Execute this command from that folder:
>
> ./fsx -d -b 1 -N 100000 -S 1 /mnt/ext4MntPt/holePunch/testFile
>
> Where "/mnt/ext4MntPt/holePunch/testFile" is a file on an ext4 file system.
>  It should then try to run through 100000 random file operations, but get
> stuck on number 48256.
>
> If you do not see any fallocate operations running, you may have to go
> enable it in the ltp/Makefile.  I had to change "ifeq ($(HAVE_FALLOCATE),
> true)" to "ifeq ($(HAVE_FALLOCATE), yes)" to allow the extra fallocate code
> to compile in.  Let me know if you have any trouble recreating the bug.
>
>>
>> Thank you,
>> Yongqiang.
>>>
>>> Allison Henderson
>>>
>>
>>
>>
>
>



-- 
Best Wishes
Yongqiang Yang
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH RFC v1 0/5]Factor common code from convert and split unwritten.

[Allison Henderson]

On 4/27/2011 11:05 PM, Yongqiang Yang wrote:
> Hi Allison,
>
> Could you send punch hole patch and fsx(if you modified it) to me?  I
> can not reproduce the bug on my system without punch hole.
>
> Thank you,
> Yongqiang.
>

Sure, I have a patch for fsx to enable fallocate, and another patch that 
adds punch hole to the test suite.  Right now I'm just working on
  getting our patches through the fsx that only has fallocate enabled. 
I thought it might be best if we work out all the bugs with that first 
before we deal with the more complex tests.

Also, I have a debug patch that fits on top of the punch hole patch. 
Initially I hadn't planned to send it out, but maybe it might help you 
so I will include it too.

Allison Henderson

> On Wed, Apr 27, 2011 at 2:34 PM, Allison Henderson
> <achender@linux.vnet.ibm.com>  wrote:
>> On 4/26/2011 9:48 PM, Yongqiang Yang wrote:
>>>
>>> On Wed, Apr 27, 2011 at 3:08 AM, Allison Henderson
>>> <achender@linux.vnet.ibm.com>    wrote:
>>>>
>>>> On 4/23/2011 1:44 AM, Yongqiang Yang wrote:
>>>>>
>>>>> v0->v1:
>>>>>     fix a bug in ext4_ext_convert_to_initialized() reported by
>>>>>     Allison<achender@linux.vnet.ibm.com>.
>>>>>
>>>>>     optimize ext4_ext_convert_to_initialized().
>>>>>
>>>>> -- factor common code
>>>>>     These patches factor common code from
>>>>> ext4_ext_convert_to_initialized()
>>>>> and
>>>>>     ext4_split_unwritten_extents() so that extent-move-on-write in
>>>>> snapshot
>>>>> and
>>>>>     punch hole can be built on the common code.
>>>>>
>>>>> -- optimization
>>>>>     the 4th and the 5th patch optimize ext4_ext_convert_to_initialized()
>>>>> by
>>>>>     zeroing out in memory.
>>>>>
>>>>>
>>>>> [PATCH RFC v1 1/5] ext4:Add a function merging extent right and left.
>>>>> [PATCH RFC v1 2/5] ext4:Add two functions splitting an extent.
>>>>> [PATCH RFC v1 3/5] ext4:Reimplement convert and split_unwritten.
>>>>> [PATCH RFC v1 4/5] ext4: Add a function ext4_ext_zeroout_mem().
>>>>> [PATCH RFC v1 5/5] ext4: optimize ext4_ext_convert_to_initialized().
>>>>> --
>>>>> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
>>>>> the body of a message to majordomo@vger.kernel.org
>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>
>>>> Hi there,
>>>>
>>>> Just an update on your patch set.  Im am working on getting the punch
>>>> hole
>>>> patch to work with this new set, but I'm am having trouble getting it
>>>> through the stress test.  It gets up to around 48265 file operations, and
>>>> then hangs. So I am currently trying to narrow down the problem.  It
>>>> looks
>>>> like it does it with or with out the extra punch hole patches, but you
>>>> may
>>>> need to enable fallocate in the fsx Makefile to recreate the problem.  I
>>>> will keep you posted if I find any more clues.
>>>
>>> Hi,
>>>
>>> Could you tell me how to get fsx?  I can not find that.
>>
>> Hi there,
>>
>> I believe you can find it in both xfstests and ltp.  The one I am using I
>> got from xfstests here:
>>
>> http://xfs.org/index.php/Getting_the_latest_source_code
>>
>> Once you have it configured and built, there is a sub folder called ltp.
>>   Execute this command from that folder:
>>
>> ./fsx -d -b 1 -N 100000 -S 1 /mnt/ext4MntPt/holePunch/testFile
>>
>> Where "/mnt/ext4MntPt/holePunch/testFile" is a file on an ext4 file system.
>>   It should then try to run through 100000 random file operations, but get
>> stuck on number 48256.
>>
>> If you do not see any fallocate operations running, you may have to go
>> enable it in the ltp/Makefile.  I had to change "ifeq ($(HAVE_FALLOCATE),
>> true)" to "ifeq ($(HAVE_FALLOCATE), yes)" to allow the extra fallocate code
>> to compile in.  Let me know if you have any trouble recreating the bug.
>>
>>>
>>> Thank you,
>>> Yongqiang.
>>>>
>>>> Allison Henderson
>>>>
>>>
>>>
>>>
>>
>>
>
>
>

Re: [PATCH RFC v1 0/5]Factor common code from convert and split unwritten.

[Allison Henderson]

On 4/28/2011 12:51 PM, Allison Henderson wrote:
> On 4/27/2011 11:05 PM, Yongqiang Yang wrote:
>> Hi Allison,
>>
>> Could you send punch hole patch and fsx(if you modified it) to me? I
>> can not reproduce the bug on my system without punch hole.
>>
>> Thank you,
>> Yongqiang.
>>
>
> Sure, I have a patch for fsx to enable fallocate, and another patch that
> adds punch hole to the test suite. Right now I'm just working on
> getting our patches through the fsx that only has fallocate enabled. I
> thought it might be best if we work out all the bugs with that first
> before we deal with the more complex tests.
>
> Also, I have a debug patch that fits on top of the punch hole patch.
> Initially I hadn't planned to send it out, but maybe it might help you
> so I will include it too.
>
> Allison Henderson

Hi all,

Just some updates on this issue.  We are currently still trying to track 
down all the remaining bugs, and I hadn't planed on sending out another 
version of punch hole until we caught them all, but there is another 
suggestion to re-order the patch sets such that the RFC patch set 
applies on top of the punch hole set.

In the updated punch hole patch set that I have not yet sent out, the 
modifications to the split extents routine have been dropped (patch 2/6 
in the punch hole v5 patch set), because those changes were being done 
in the underlying RFC patch set.  If we re-order the patch sets, I would 
have to retain the split extents modifications, which means that the RFC 
set would have to be modified to deal with that.

I did think of one other option though.  There were two versions of the 
RFC patch set, and we only had to make a small modification to the first 
RFC set to make it work. After that, I didnt have any trouble with any 
of the test cases.  So a third option would be to push forward with an 
updated version of the first RFC set.  That would save Yongqaing from 
having to go back and deal with patch 2/6 of punch hole, and then only 
the remaining code that optimizes zeroing out extents would have to be 
made to apply on top of punch hole.  But I wasn't sure if that idea 
would be preferable to moving the entire set on top of punch hole, or if 
it is just better for us to continue pushing forward with debugging what 
we have now.  In the end, what ever combination of sets we choose to do 
should still be passing all the tests, but what would everyone prefer to 
do?  Thx!

Allison Henderson




>
>> On Wed, Apr 27, 2011 at 2:34 PM, Allison Henderson
>> <achender@linux.vnet.ibm.com> wrote:
>>> On 4/26/2011 9:48 PM, Yongqiang Yang wrote:
>>>>
>>>> On Wed, Apr 27, 2011 at 3:08 AM, Allison Henderson
>>>> <achender@linux.vnet.ibm.com> wrote:
>>>>>
>>>>> On 4/23/2011 1:44 AM, Yongqiang Yang wrote:
>>>>>>
>>>>>> v0->v1:
>>>>>> fix a bug in ext4_ext_convert_to_initialized() reported by
>>>>>> Allison<achender@linux.vnet.ibm.com>.
>>>>>>
>>>>>> optimize ext4_ext_convert_to_initialized().
>>>>>>
>>>>>> -- factor common code
>>>>>> These patches factor common code from
>>>>>> ext4_ext_convert_to_initialized()
>>>>>> and
>>>>>> ext4_split_unwritten_extents() so that extent-move-on-write in
>>>>>> snapshot
>>>>>> and
>>>>>> punch hole can be built on the common code.
>>>>>>
>>>>>> -- optimization
>>>>>> the 4th and the 5th patch optimize ext4_ext_convert_to_initialized()
>>>>>> by
>>>>>> zeroing out in memory.
>>>>>>
>>>>>>
>>>>>> [PATCH RFC v1 1/5] ext4:Add a function merging extent right and left.
>>>>>> [PATCH RFC v1 2/5] ext4:Add two functions splitting an extent.
>>>>>> [PATCH RFC v1 3/5] ext4:Reimplement convert and split_unwritten.
>>>>>> [PATCH RFC v1 4/5] ext4: Add a function ext4_ext_zeroout_mem().
>>>>>> [PATCH RFC v1 5/5] ext4: optimize ext4_ext_convert_to_initialized().
>>>>>> --
>>>>>> To unsubscribe from this list: send the line "unsubscribe
>>>>>> linux-ext4" in
>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>>>
>>>>> Hi there,
>>>>>
>>>>> Just an update on your patch set. Im am working on getting the punch
>>>>> hole
>>>>> patch to work with this new set, but I'm am having trouble getting it
>>>>> through the stress test. It gets up to around 48265 file
>>>>> operations, and
>>>>> then hangs. So I am currently trying to narrow down the problem. It
>>>>> looks
>>>>> like it does it with or with out the extra punch hole patches, but you
>>>>> may
>>>>> need to enable fallocate in the fsx Makefile to recreate the
>>>>> problem. I
>>>>> will keep you posted if I find any more clues.
>>>>
>>>> Hi,
>>>>
>>>> Could you tell me how to get fsx? I can not find that.
>>>
>>> Hi there,
>>>
>>> I believe you can find it in both xfstests and ltp. The one I am using I
>>> got from xfstests here:
>>>
>>> http://xfs.org/index.php/Getting_the_latest_source_code
>>>
>>> Once you have it configured and built, there is a sub folder called ltp.
>>> Execute this command from that folder:
>>>
>>> ./fsx -d -b 1 -N 100000 -S 1 /mnt/ext4MntPt/holePunch/testFile
>>>
>>> Where "/mnt/ext4MntPt/holePunch/testFile" is a file on an ext4 file
>>> system.
>>> It should then try to run through 100000 random file operations, but get
>>> stuck on number 48256.
>>>
>>> If you do not see any fallocate operations running, you may have to go
>>> enable it in the ltp/Makefile. I had to change "ifeq ($(HAVE_FALLOCATE),
>>> true)" to "ifeq ($(HAVE_FALLOCATE), yes)" to allow the extra
>>> fallocate code
>>> to compile in. Let me know if you have any trouble recreating the bug.
>>>
>>>>
>>>> Thank you,
>>>> Yongqiang.
>>>>>
>>>>> Allison Henderson
>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH RFC v1 0/5]Factor common code from convert and split unwritten.

[Yongqiang Yang]

On Sat, Apr 30, 2011 at 3:16 AM, Allison Henderson
<achender@linux.vnet.ibm.com> wrote:
> On 4/28/2011 12:51 PM, Allison Henderson wrote:
>>
>> On 4/27/2011 11:05 PM, Yongqiang Yang wrote:
>>>
>>> Hi Allison,
>>>
>>> Could you send punch hole patch and fsx(if you modified it) to me? I
>>> can not reproduce the bug on my system without punch hole.
>>>
>>> Thank you,
>>> Yongqiang.
>>>
>>
>> Sure, I have a patch for fsx to enable fallocate, and another patch that
>> adds punch hole to the test suite. Right now I'm just working on
>> getting our patches through the fsx that only has fallocate enabled. I
>> thought it might be best if we work out all the bugs with that first
>> before we deal with the more complex tests.
>>
>> Also, I have a debug patch that fits on top of the punch hole patch.
>> Initially I hadn't planned to send it out, but maybe it might help you
>> so I will include it too.
>>
>> Allison Henderson
>
> Hi all,
>
> Just some updates on this issue.  We are currently still trying to track
> down all the remaining bugs, and I hadn't planed on sending out another
> version of punch hole until we caught them all, but there is another
> suggestion to re-order the patch sets such that the RFC patch set applies on
> top of the punch hole set.
>
> In the updated punch hole patch set that I have not yet sent out, the
> modifications to the split extents routine have been dropped (patch 2/6 in
> the punch hole v5 patch set), because those changes were being done in the
> underlying RFC patch set.  If we re-order the patch sets, I would have to
> retain the split extents modifications, which means that the RFC set would
> have to be modified to deal with that.
>
> I did think of one other option though.  There were two versions of the RFC
> patch set, and we only had to make a small modification to the first RFC set
> to make it work. After that, I didnt have any trouble with any of the test
> cases.  So a third option would be to push forward with an updated version
> of the first RFC set.  That would save Yongqaing from having to go back and
> deal with patch 2/6 of punch hole, and then only the remaining code that
> optimizes zeroing out extents would have to be made to apply on top of punch
> hole.  But I wasn't sure if that idea would be preferable to moving the
> entire set on top of punch hole, or if it is just better for us to continue
> pushing forward with debugging what we have now.  In the end, what ever
> combination of sets we choose to do should still be passing all the tests,
> but what would everyone prefer to do?  Thx!
Hi,

I think we should push forward with the updated version without
optimization for zeroing out extents.  After split-extent and
punch-hole patches are merged into the tree, I will try to optimize
zeroing out extents.

I will send out the patches Allison has tested well.

Allison, thank you for testing the patches.

Yongqiang.

>
> Allison Henderson
>
>
>
>
>>
>>> On Wed, Apr 27, 2011 at 2:34 PM, Allison Henderson
>>> <achender@linux.vnet.ibm.com> wrote:
>>>>
>>>> On 4/26/2011 9:48 PM, Yongqiang Yang wrote:
>>>>>
>>>>> On Wed, Apr 27, 2011 at 3:08 AM, Allison Henderson
>>>>> <achender@linux.vnet.ibm.com> wrote:
>>>>>>
>>>>>> On 4/23/2011 1:44 AM, Yongqiang Yang wrote:
>>>>>>>
>>>>>>> v0->v1:
>>>>>>> fix a bug in ext4_ext_convert_to_initialized() reported by
>>>>>>> Allison<achender@linux.vnet.ibm.com>.
>>>>>>>
>>>>>>> optimize ext4_ext_convert_to_initialized().
>>>>>>>
>>>>>>> -- factor common code
>>>>>>> These patches factor common code from
>>>>>>> ext4_ext_convert_to_initialized()
>>>>>>> and
>>>>>>> ext4_split_unwritten_extents() so that extent-move-on-write in
>>>>>>> snapshot
>>>>>>> and
>>>>>>> punch hole can be built on the common code.
>>>>>>>
>>>>>>> -- optimization
>>>>>>> the 4th and the 5th patch optimize ext4_ext_convert_to_initialized()
>>>>>>> by
>>>>>>> zeroing out in memory.
>>>>>>>
>>>>>>>
>>>>>>> [PATCH RFC v1 1/5] ext4:Add a function merging extent right and left.
>>>>>>> [PATCH RFC v1 2/5] ext4:Add two functions splitting an extent.
>>>>>>> [PATCH RFC v1 3/5] ext4:Reimplement convert and split_unwritten.
>>>>>>> [PATCH RFC v1 4/5] ext4: Add a function ext4_ext_zeroout_mem().
>>>>>>> [PATCH RFC v1 5/5] ext4: optimize ext4_ext_convert_to_initialized().
>>>>>>> --
>>>>>>> To unsubscribe from this list: send the line "unsubscribe
>>>>>>> linux-ext4" in
>>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>>>>
>>>>>> Hi there,
>>>>>>
>>>>>> Just an update on your patch set. Im am working on getting the punch
>>>>>> hole
>>>>>> patch to work with this new set, but I'm am having trouble getting it
>>>>>> through the stress test. It gets up to around 48265 file
>>>>>> operations, and
>>>>>> then hangs. So I am currently trying to narrow down the problem. It
>>>>>> looks
>>>>>> like it does it with or with out the extra punch hole patches, but you
>>>>>> may
>>>>>> need to enable fallocate in the fsx Makefile to recreate the
>>>>>> problem. I
>>>>>> will keep you posted if I find any more clues.
>>>>>
>>>>> Hi,
>>>>>
>>>>> Could you tell me how to get fsx? I can not find that.
>>>>
>>>> Hi there,
>>>>
>>>> I believe you can find it in both xfstests and ltp. The one I am using I
>>>> got from xfstests here:
>>>>
>>>> http://xfs.org/index.php/Getting_the_latest_source_code
>>>>
>>>> Once you have it configured and built, there is a sub folder called ltp.
>>>> Execute this command from that folder:
>>>>
>>>> ./fsx -d -b 1 -N 100000 -S 1 /mnt/ext4MntPt/holePunch/testFile
>>>>
>>>> Where "/mnt/ext4MntPt/holePunch/testFile" is a file on an ext4 file
>>>> system.
>>>> It should then try to run through 100000 random file operations, but get
>>>> stuck on number 48256.
>>>>
>>>> If you do not see any fallocate operations running, you may have to go
>>>> enable it in the ltp/Makefile. I had to change "ifeq ($(HAVE_FALLOCATE),
>>>> true)" to "ifeq ($(HAVE_FALLOCATE), yes)" to allow the extra
>>>> fallocate code
>>>> to compile in. Let me know if you have any trouble recreating the bug.
>>>>
>>>>>
>>>>> Thank you,
>>>>> Yongqiang.
>>>>>>
>>>>>> Allison Henderson
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>



-- 
Best Wishes
Yongqiang Yang
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html