[PATCH 0/2] send: Avoid sending disknr==0 and PREALLOC extents when possible

[PATCH 0/2] send: Avoid sending disknr==0 and PREALLOC extents when possible

[Alex Lyakas]

These two patches address the issue of sending unneeded zero data for
disknr==0 and PREALLOC extents.
There is room for additional improvement for PREALLOC extents, but it
requires adding a new command, so for
now this is not addressed.

Please review and comment.

Thanks,
Alex.

Alex Lyakas (2):
  Avoid sending disknr==0 extents in the following cases:     1) full
    send 2) new inode in a diff-send 3) when disknr==0 extents are
    added to the end of an inode
  On a diff-send, avoid sending PREALLOC extents, if the parent root
    has only     PREALLOC extents on an appropriate file range.

 fs/btrfs/send.c |  178 +++++++++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 172 insertions(+), 6 deletions(-)

--
1.7.9.5

[PATCH 1/2] Avoid sending disknr==0 extents when possible

[Alex Lyakas]

Subject: [PATCH 1/2] Avoid sending disknr==0 extents in the following cases:
 1) full send 2) new inode in a diff-send 3) when
 disknr==0 extents are  added to the end of an inode

Original-version-by: Chen Yang <chenyang.fnst@cn.fujitsu.com>
Signed-off-by: Alex Lyakas <alex.btrfs@zadarastorage.com>
---
 fs/btrfs/send.c |   28 ++++++++++++++++++++++++++--
 1 file changed, 26 insertions(+), 2 deletions(-)

diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c
index 5445454..5ab584f 100644
--- a/fs/btrfs/send.c
+++ b/fs/btrfs/send.c
@@ -3844,7 +3844,8 @@ static int is_extent_unchanged(struct send_ctx *sctx,
 	btrfs_item_key_to_cpu(eb, &found_key, slot);
 	if (found_key.objectid != key.objectid ||
 	    found_key.type != key.type) {
-		ret = 0;
+		/* No need to send a no-data extent it in this case */
+		ret = (left_disknr == 0) ? 1 : 0;
 		goto out;
 	}

@@ -3870,7 +3871,8 @@ static int is_extent_unchanged(struct send_ctx *sctx,
 		 * This may only happen on the first iteration.
 		 */
 		if (found_key.offset + right_len <= ekey->offset) {
-			ret = 0;
+			/* No need to send a no-data extent it in this case */
+			ret = (left_disknr == 0) ? 1 : 0;
 			goto out;
 		}

@@ -3951,6 +3953,28 @@ static int process_extent(struct send_ctx *sctx,
 			ret = 0;
 			goto out;
 		}
+	} else {
+		struct extent_buffer *eb;
+		struct btrfs_file_extent_item *ei;
+		u8 extent_type;
+		u64 extent_disknr;
+
+		eb = path->nodes[0];
+		ei = btrfs_item_ptr(eb, path->slots[0],
+				struct btrfs_file_extent_item);
+
+		extent_type = btrfs_file_extent_type(eb, ei);
+		extent_disknr = btrfs_file_extent_disk_bytenr(eb, ei);
+		if (extent_type == BTRFS_FILE_EXTENT_REG && extent_disknr == 0) {
+			/*
+			 * This is disknr=0 extent in a full-send or a new inode
+			 * in a diff-send. Since we will send truncate command
+			 * in finish_inode_if_needed anyways, the inode size will be
+			 * correct, and we don't have to send all-zero data.
+			 */
+			ret = 0;
+			goto out;
+		}
 	}

 	ret = find_extent_clone(sctx, path, key->objectid, key->offset,
-- 
1.7.9.5

[PATCH 2/2] On a diff-send, avoid sending PREALLOC extent

[Alex Lyakas]

Subject: [PATCH 2/2] On a diff-send, avoid sending PREALLOC extents, if the
 parent root has only PREALLOC extents on an appropriate
 file range.

This does not fully avoids sending PREALLOC extents, because on full-send or
new inode we need a new send command to do that. But this patch improves
the situation by handling diff-sends.

Signed-off-by: Alex Lyakas <alex.btrfs@zadarastorage.com>
---
 fs/btrfs/send.c |  150 +++++++++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 146 insertions(+), 4 deletions(-)

diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c
index 5ab584f..456bc3e 100644
--- a/fs/btrfs/send.c
+++ b/fs/btrfs/send.c
@@ -3763,6 +3763,144 @@ out:
 	return ret;
 }

+static int is_prealloc_extent_unchanged(struct send_ctx *sctx,
+			       struct btrfs_path *left_path,
+			       struct btrfs_key *ekey)
+{
+	int ret = 0;
+	struct btrfs_key key;
+	struct btrfs_path *path = NULL;
+	struct extent_buffer *eb;
+	int slot;
+	struct btrfs_key found_key;
+	struct btrfs_file_extent_item *ei;
+	u64 left_len;
+	u64 right_len;
+	u8 right_type;
+
+	eb = left_path->nodes[0];
+	slot = left_path->slots[0];
+	ei = btrfs_item_ptr(eb, slot, struct btrfs_file_extent_item);
+	left_len = btrfs_file_extent_num_bytes(eb, ei);
+
+	/*
+	 * The logic is similar, but much simpler than in is_extent_unchanged().
+	 * We need to check extents on the parent root, and make sure that only
+	 * PREALLOC extents are on the same file range as our current extent.
+	 *
+	 * Following comments will refer to these graphics. L is the left
+	 * extents which we are checking at the moment. 1-8 are the right
+	 * extents that we iterate.
+	 *
+	 * 	  |-----L-----|
+	 * |-1-|-2a-|-3-|-4-|-5-|-6-|
+	 *
+	 * 	  |-----L-----|
+	 * |--1--|-2b-|...(same as above)
+	 *
+	 * Alternative situation. Happens on files where extents got split.
+	 * 	  |-----L-----|
+	 * |-----------7-----------|-6-|
+	 *
+	 * Alternative situation. Happens on files which got larger.
+	 * 	  |-----L-----|
+	 * |-8-|
+	 * Nothing follows after 8.
+	 */
+
+	path = alloc_path_for_send();
+	if (!path)
+		return -ENOMEM;
+
+	key.objectid = ekey->objectid;
+	key.type = BTRFS_EXTENT_DATA_KEY;
+	key.offset = ekey->offset;
+	ret = btrfs_search_slot_for_read(sctx->parent_root, &key, path, 0, 0);
+	if (ret < 0)
+		goto out;
+	if (ret) {
+		ret = 0;
+		goto out;
+	}
+
+	/*
+	 * Handle special case where the right side has no extents at all.
+	 */
+	eb = path->nodes[0];
+	slot = path->slots[0];
+	btrfs_item_key_to_cpu(eb, &found_key, slot);
+	if (found_key.objectid != key.objectid ||
+		found_key.type != key.type) {
+		/*
+		 * We need to send a "prealloc" command, which we don't have yet,
+		 * just send this extent fully.
+		 */
+		ret = 0;
+		goto out;
+	}
+
+	key = found_key;
+	while (key.offset < ekey->offset + left_len) {
+		ei = btrfs_item_ptr(eb, slot, struct btrfs_file_extent_item);
+		right_type = btrfs_file_extent_type(eb, ei);
+		right_len = btrfs_file_extent_num_bytes(eb, ei);
+
+		if (right_type != BTRFS_FILE_EXTENT_PREALLOC) {
+			ret = 0;
+			goto out;
+		}
+
+		/*
+		 * Are we at extent 8? If yes, we know the extent is changed.
+		 * This may only happen on the first iteration.
+		 */
+		if (found_key.offset + right_len <= ekey->offset) {
+			/*
+			 * We need to send a "prealloc" command, which we don't have yet,
+			 * just send this extent fully.
+			 */
+			ret = 0;
+			goto out;
+		}
+
+		/* The right extent is also PREALLOC, so up to here we are ok,
continue checking */
+
+		ret = btrfs_next_item(sctx->parent_root, path);
+		if (ret < 0)
+			goto out;
+		if (!ret) {
+			eb = path->nodes[0];
+			slot = path->slots[0];
+			btrfs_item_key_to_cpu(eb, &found_key, slot);
+		}
+		if (ret || found_key.objectid != key.objectid ||
+		    found_key.type != key.type) {
+			key.offset += right_len;
+			break;
+		} else {
+			if (found_key.offset != key.offset + right_len) {
+				/* Should really not happen */
+				ret = -EIO;
+				goto out;
+			}
+		}
+		key = found_key;
+	}
+
+	/*
+	 * We're now behind the left extent (treat as unchanged) or at the end
+	 * of the right side (treat as changed).
+	 */
+	if (key.offset >= ekey->offset + left_len)
+		ret = 1;
+	else
+		ret = 0;
+
+out:
+	btrfs_free_path(path);
+	return ret;
+}
+
 static int is_extent_unchanged(struct send_ctx *sctx,
 			       struct btrfs_path *left_path,
 			       struct btrfs_key *ekey)
@@ -3786,15 +3924,15 @@ static int is_extent_unchanged(struct send_ctx *sctx,
 	u8 left_type;
 	u8 right_type;

-	path = alloc_path_for_send();
-	if (!path)
-		return -ENOMEM;
-
 	eb = left_path->nodes[0];
 	slot = left_path->slots[0];
 	ei = btrfs_item_ptr(eb, slot, struct btrfs_file_extent_item);
 	left_type = btrfs_file_extent_type(eb, ei);

+	if (left_type == BTRFS_FILE_EXTENT_PREALLOC) {
+		ret = is_prealloc_extent_unchanged(sctx, left_path, ekey);
+		goto out;
+	}
 	if (left_type != BTRFS_FILE_EXTENT_REG) {
 		ret = 0;
 		goto out;
@@ -3825,6 +3963,10 @@ static int is_extent_unchanged(struct send_ctx *sctx,
 	 * Nothing follows after 8.
 	 */

+	path = alloc_path_for_send();
+	if (!path)
+		return -ENOMEM;
+
 	key.objectid = ekey->objectid;
 	key.type = BTRFS_EXTENT_DATA_KEY;
 	key.offset = ekey->offset;
-- 
1.7.9.5

Re: [PATCH 0/2] send: Avoid sending disknr==0 and PREALLOC extents when possible

[Chen Yang]

Hi, Alex
The patches can only solve part of questions, but some like "punch holes" problems 
as Alexander wrote are still unsolved. So I think these patches are unnecessary
if adding new command would help to solve all the questions.

Chen.

On 2013-1-9 4:11, Alex Lyakas wrote:
> These two patches address the issue of sending unneeded zero data for
> disknr==0 and PREALLOC extents.
> There is room for additional improvement for PREALLOC extents, but it
> requires adding a new command, so for
> now this is not addressed.
> 
> Please review and comment.
> 
> Thanks,
> Alex.
> 
> Alex Lyakas (2):
>   Avoid sending disknr==0 extents in the following cases:     1) full
>     send 2) new inode in a diff-send 3) when disknr==0 extents are
>     added to the end of an inode
>   On a diff-send, avoid sending PREALLOC extents, if the parent root
>     has only     PREALLOC extents on an appropriate file range.
> 
>  fs/btrfs/send.c |  178 +++++++++++++++++++++++++++++++++++++++++++++++++++++--
>  1 file changed, 172 insertions(+), 6 deletions(-)
> 
> --
> 1.7.9.5
> 

Re: [PATCH 0/2] send: Avoid sending disknr==0 and PREALLOC extents when possible

[Alex Lyakas]

Hi Chen,
thanks for reviewing.

I partially disagree with you. The new command(s) is required when the
code detects a change and wants to communicate this change to the
receive side. My patch addresses the cases when
- no new command is required to communicate the change (patch1)
- there is no change that is needed to be communicated (patch 2)

So even if/when new command is added, my patch still holds, I believe.
However, let's see what others will comment.

Thanks,
Alex.


On Thu, Jan 10, 2013 at 5:25 AM, Chen Yang <chenyang.fnst@cn.fujitsu.com> wrote:
> Hi, Alex
> The patches can only solve part of questions, but some like "punch holes" problems
> as Alexander wrote are still unsolved. So I think these patches are unnecessary
> if adding new command would help to solve all the questions.
>
> Chen.
>
> On 2013-1-9 4:11, Alex Lyakas wrote:
>> These two patches address the issue of sending unneeded zero data for
>> disknr==0 and PREALLOC extents.
>> There is room for additional improvement for PREALLOC extents, but it
>> requires adding a new command, so for
>> now this is not addressed.
>>
>> Please review and comment.
>>
>> Thanks,
>> Alex.
>>
>> Alex Lyakas (2):
>>   Avoid sending disknr==0 extents in the following cases:     1) full
>>     send 2) new inode in a diff-send 3) when disknr==0 extents are
>>     added to the end of an inode
>>   On a diff-send, avoid sending PREALLOC extents, if the parent root
>>     has only     PREALLOC extents on an appropriate file range.
>>
>>  fs/btrfs/send.c |  178 +++++++++++++++++++++++++++++++++++++++++++++++++++++--
>>  1 file changed, 172 insertions(+), 6 deletions(-)
>>
>> --
>> 1.7.9.5
>>
>

Re: [PATCH 2/2] On a diff-send, avoid sending PREALLOC extent

[Josef Bacik]

On Wed, Jan 09, 2013 at 10:41:10AM -0700, Alex Lyakas wrote:
> Subject: [PATCH 2/2] On a diff-send, avoid sending PREALLOC extents, if the
>  parent root has only PREALLOC extents on an appropriate
>  file range.
> 
> This does not fully avoids sending PREALLOC extents, because on full-send or
> new inode we need a new send command to do that. But this patch improves
> the situation by handling diff-sends.
> 
> Signed-off-by: Alex Lyakas <alex.btrfs@zadarastorage.com>
> ---

Malformed patch and it confused patchwork, please resend.  Thanks,

Josef

Re: [PATCH 2/2] On a diff-send, avoid sending PREALLOC extent

[Alex Lyakas]

Hi Josef,
I have resent the patches. There are still some "please, no space
before tabs" warnings, which I am not sure how to correct, because I
don't see space before tab in that place. If there is still a problem,
can you please advise what is wrong.

Thanks,
Alex.


On Mon, Jan 14, 2013 at 8:45 PM, Josef Bacik <jbacik@fusionio.com> wrote:
> On Wed, Jan 09, 2013 at 10:41:10AM -0700, Alex Lyakas wrote:
>> Subject: [PATCH 2/2] On a diff-send, avoid sending PREALLOC extents, if the
>>  parent root has only PREALLOC extents on an appropriate
>>  file range.
>>
>> This does not fully avoids sending PREALLOC extents, because on full-send or
>> new inode we need a new send command to do that. But this patch improves
>> the situation by handling diff-sends.
>>
>> Signed-off-by: Alex Lyakas <alex.btrfs@zadarastorage.com>
>> ---
>
> Malformed patch and it confused patchwork, please resend.  Thanks,
>
> Josef