* [RFC PATCH] Btrfs: btrfs_dedupe_file_range() ioctl, remove 16MiB restriction
@ 2017-08-27 20:06 Timofey Titovets
2017-08-27 20:28 ` Timofey Titovets
0 siblings, 1 reply; 3+ messages in thread
From: Timofey Titovets @ 2017-08-27 20:06 UTC (permalink / raw)
To: linux-btrfs; +Cc: Timofey Titovets
At now btrfs_dedupe_file_range() restricted to 16MiB range for
limit locking time and memory requirement for dedup ioctl()
For too big input rage code silently set range to 16MiB
Let's remove that restriction by do iterating over dedup range.
That's backward compatible and will not change anything for request
less then 16MiB.
Signed-off-by: Timofey Titovets <nefelim4ag@gmail.com>
---
fs/btrfs/ioctl.c | 22 ++++++++++++++++++----
1 file changed, 18 insertions(+), 4 deletions(-)
diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c
index ae8fbf9d3de2..7e77d192776f 100644
--- a/fs/btrfs/ioctl.c
+++ b/fs/btrfs/ioctl.c
@@ -3226,11 +3226,9 @@ ssize_t btrfs_dedupe_file_range(struct file *src_file, u64 loff, u64 olen,
struct inode *src = file_inode(src_file);
struct inode *dst = file_inode(dst_file);
u64 bs = BTRFS_I(src)->root->fs_info->sb->s_blocksize;
+ u64 i, tail_len, chunk_count;
ssize_t res;
- if (olen > BTRFS_MAX_DEDUPE_LEN)
- olen = BTRFS_MAX_DEDUPE_LEN;
-
if (WARN_ON_ONCE(bs < PAGE_SIZE)) {
/*
* Btrfs does not support blocksize < page_size. As a
@@ -3240,7 +3238,23 @@ ssize_t btrfs_dedupe_file_range(struct file *src_file, u64 loff, u64 olen,
return -EINVAL;
}
- res = btrfs_extent_same(src, loff, olen, dst, dst_loff);
+ tail_len = olen % BTRFS_MAX_DEDUPE_LEN;
+ chunk_count = div_u64(olen, BTRFS_MAX_DEDUPE_LEN);
+
+ for (i = 0; i < chunk_count; i++) {
+ res = btrfs_extent_same(src, loff, BTRFS_MAX_DEDUPE_LEN,
+ dst, dst_loff);
+ if (res)
+ return res;
+
+ loff += BTRFS_MAX_DEDUPE_LEN;
+ dst_loff += BTRFS_MAX_DEDUPE_LEN;
+ }
+
+ if (tail_len > 0)
+ res = btrfs_extent_same(src, loff, tail_len,
+ dst, dst_loff);
+
if (res)
return res;
return olen;
--
2.14.1
^ permalink raw reply related [flat|nested] 3+ messages in thread* Re: [RFC PATCH] Btrfs: btrfs_dedupe_file_range() ioctl, remove 16MiB restriction
2017-08-27 20:06 [RFC PATCH] Btrfs: btrfs_dedupe_file_range() ioctl, remove 16MiB restriction Timofey Titovets
@ 2017-08-27 20:28 ` Timofey Titovets
2017-08-27 23:46 ` Qu Wenruo
0 siblings, 1 reply; 3+ messages in thread
From: Timofey Titovets @ 2017-08-27 20:28 UTC (permalink / raw)
To: linux-btrfs; +Cc: Timofey Titovets
2017-08-27 23:06 GMT+03:00 Timofey Titovets <nefelim4ag@gmail.com>:
> At now btrfs_dedupe_file_range() restricted to 16MiB range for
> limit locking time and memory requirement for dedup ioctl()
>
> For too big input rage code silently set range to 16MiB
>
> Let's remove that restriction by do iterating over dedup range.
> That's backward compatible and will not change anything for request
> less then 16MiB.
>
> Signed-off-by: Timofey Titovets <nefelim4ag@gmail.com>
> ---
> fs/btrfs/ioctl.c | 22 ++++++++++++++++++----
> 1 file changed, 18 insertions(+), 4 deletions(-)
>
> diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c
> index ae8fbf9d3de2..7e77d192776f 100644
> --- a/fs/btrfs/ioctl.c
> +++ b/fs/btrfs/ioctl.c
> @@ -3226,11 +3226,9 @@ ssize_t btrfs_dedupe_file_range(struct file *src_file, u64 loff, u64 olen,
> struct inode *src = file_inode(src_file);
> struct inode *dst = file_inode(dst_file);
> u64 bs = BTRFS_I(src)->root->fs_info->sb->s_blocksize;
> + u64 i, tail_len, chunk_count;
> ssize_t res;
>
> - if (olen > BTRFS_MAX_DEDUPE_LEN)
> - olen = BTRFS_MAX_DEDUPE_LEN;
> -
> if (WARN_ON_ONCE(bs < PAGE_SIZE)) {
> /*
> * Btrfs does not support blocksize < page_size. As a
> @@ -3240,7 +3238,23 @@ ssize_t btrfs_dedupe_file_range(struct file *src_file, u64 loff, u64 olen,
> return -EINVAL;
> }
>
> - res = btrfs_extent_same(src, loff, olen, dst, dst_loff);
> + tail_len = olen % BTRFS_MAX_DEDUPE_LEN;
> + chunk_count = div_u64(olen, BTRFS_MAX_DEDUPE_LEN);
> +
> + for (i = 0; i < chunk_count; i++) {
> + res = btrfs_extent_same(src, loff, BTRFS_MAX_DEDUPE_LEN,
> + dst, dst_loff);
> + if (res)
> + return res;
> +
> + loff += BTRFS_MAX_DEDUPE_LEN;
> + dst_loff += BTRFS_MAX_DEDUPE_LEN;
> + }
> +
> + if (tail_len > 0)
> + res = btrfs_extent_same(src, loff, tail_len,
> + dst, dst_loff);
> +
> if (res)
> return res;
> return olen;
> --
> 2.14.1
P.S.
Also, if that acceptable, it's possible to decrease BTRFS_MAX_DEDUPE_LEN
to 1MiB or 128KiB, if inode use compression to make
btrfs_extent_same() more time/cpu/lock/io сheap
--
Have a nice day,
Timofey.
^ permalink raw reply [flat|nested] 3+ messages in thread* Re: [RFC PATCH] Btrfs: btrfs_dedupe_file_range() ioctl, remove 16MiB restriction
2017-08-27 20:28 ` Timofey Titovets
@ 2017-08-27 23:46 ` Qu Wenruo
0 siblings, 0 replies; 3+ messages in thread
From: Qu Wenruo @ 2017-08-27 23:46 UTC (permalink / raw)
To: Timofey Titovets, linux-btrfs
On 2017年08月28日 04:28, Timofey Titovets wrote:
> 2017-08-27 23:06 GMT+03:00 Timofey Titovets <nefelim4ag@gmail.com>:
>> At now btrfs_dedupe_file_range() restricted to 16MiB range for
>> limit locking time and memory requirement for dedup ioctl()
>>
>> For too big input rage code silently set range to 16MiB
>>
>> Let's remove that restriction by do iterating over dedup range.
>> That's backward compatible and will not change anything for request
>> less then 16MiB.
>>
>> Signed-off-by: Timofey Titovets <nefelim4ag@gmail.com>
>> ---
>> fs/btrfs/ioctl.c | 22 ++++++++++++++++++----
>> 1 file changed, 18 insertions(+), 4 deletions(-)
>>
>> diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c
>> index ae8fbf9d3de2..7e77d192776f 100644
>> --- a/fs/btrfs/ioctl.c
>> +++ b/fs/btrfs/ioctl.c
>> @@ -3226,11 +3226,9 @@ ssize_t btrfs_dedupe_file_range(struct file *src_file, u64 loff, u64 olen,
>> struct inode *src = file_inode(src_file);
>> struct inode *dst = file_inode(dst_file);
>> u64 bs = BTRFS_I(src)->root->fs_info->sb->s_blocksize;
>> + u64 i, tail_len, chunk_count;
>> ssize_t res;
>>
>> - if (olen > BTRFS_MAX_DEDUPE_LEN)
>> - olen = BTRFS_MAX_DEDUPE_LEN;
>> -
>> if (WARN_ON_ONCE(bs < PAGE_SIZE)) {
>> /*
>> * Btrfs does not support blocksize < page_size. As a
>> @@ -3240,7 +3238,23 @@ ssize_t btrfs_dedupe_file_range(struct file *src_file, u64 loff, u64 olen,
>> return -EINVAL;
>> }
>>
>> - res = btrfs_extent_same(src, loff, olen, dst, dst_loff);
>> + tail_len = olen % BTRFS_MAX_DEDUPE_LEN;
>> + chunk_count = div_u64(olen, BTRFS_MAX_DEDUPE_LEN);
>> +
>> + for (i = 0; i < chunk_count; i++) {
>> + res = btrfs_extent_same(src, loff, BTRFS_MAX_DEDUPE_LEN,
>> + dst, dst_loff);
>> + if (res)
>> + return res;
>> +
>> + loff += BTRFS_MAX_DEDUPE_LEN;
>> + dst_loff += BTRFS_MAX_DEDUPE_LEN;
>> + }
>> +
>> + if (tail_len > 0)
>> + res = btrfs_extent_same(src, loff, tail_len,
>> + dst, dst_loff);
>> +
>> if (res)
>> return res;
>> return olen;
>> --
>> 2.14.1
>
> P.S.
> Also, if that acceptable, it's possible to decrease BTRFS_MAX_DEDUPE_LEN
> to 1MiB or 128KiB, if inode use compression to make
> btrfs_extent_same() more time/cpu/lock/io сheap
The idea to decrease BTRFS_MAX_DEDUPE_LEN based on inode compression
flag seems good.
However the problem is, even an inode has compression flag, it doesn't
mean all its file extents are compressed.
Since compression flag only affects the behavior after setting, it's
quite possible a lot of plaintext extent remains.
Not to mention that compression may be aborted due to compression ratio.
BTW, the patch itself looks good.
Reviewed-by: Qu Wenruo <quwenruo.btrfs@gmx.com>
Thanks,
Qu
>
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2017-08-27 23:46 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2017-08-27 20:06 [RFC PATCH] Btrfs: btrfs_dedupe_file_range() ioctl, remove 16MiB restriction Timofey Titovets
2017-08-27 20:28 ` Timofey Titovets
2017-08-27 23:46 ` Qu Wenruo
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).