* [PATCH v2 0/2] btrfs: reduce div64 calls for __btrfs_map_block() and its variants
@ 2023-02-06 1:10 Qu Wenruo
2023-02-06 1:10 ` [PATCH v2 1/2] btrfs: remove map_lookup->stripe_len Qu Wenruo
2023-02-06 1:10 ` [PATCH v2 2/2] btrfs: reduce div64 calls by limiting the number of stripes of a chunk to u32 Qu Wenruo
0 siblings, 2 replies; 4+ messages in thread
From: Qu Wenruo @ 2023-02-06 1:10 UTC (permalink / raw)
To: linux-btrfs
Changelog:
v2:
- Fix a linkage error for 32bits platform in block-group.c
We have a line "stripe_nr = (stripe_nr * num_stripes + i) /
sub_stripes."
In that case, @stripe_nr itself is already u64, thus the division
is dividing u64 with u32.
And we can not easily do a forced u32 conversion, as we only have
ensured @stripe_nr can be contained in u32.
Even with 10G chunk size in mined, if the RAID10 array has over 26
devices, then we can overflow U32 and cause problems.
For now, keeps the old div_u64 call.
Div64 is much slower than 32 bit division, and only get improved in
the most recent CPUs, that's why we have dedicated div64* helpers.
One usage of div64 is in __btrfs_map_block() and its variants, where we
got @stripe_nr as u64, and all later division has to go the div64
helpers.
But the truth is, with our current chunk size limit (10G) and fixed
stripe length (64K), we can have at most 160K stripes in a chunk, which
is small enough for u32 already.
So this patchset would reduce div64 calls by:
- Remove map_lookup::stripe_len first
So now we don't need to call div64 to calculate @stripe_nr, just a
simple right shift, then truncate to u32.
This is a prerequisite for the 2nd patch, without the fixed stripe
length, we have to rely on div64.
- Reduce the width of various @stripe_nr to 32
- Use regular divitsion and module to do the calculation
Now we can get rid of the fear that we missed some div64 helpers.
Qu Wenruo (2):
btrfs: remove map_lookup->stripe_len
btrfs: reduce div64 calls by limiting the number of stripes of a chunk
to u32
fs/btrfs/block-group.c | 22 +++---
fs/btrfs/scrub.c | 43 ++++++------
fs/btrfs/tests/extent-map-tests.c | 1 -
fs/btrfs/tree-checker.c | 14 ++++
fs/btrfs/volumes.c | 110 +++++++++++++++---------------
fs/btrfs/volumes.h | 7 +-
6 files changed, 106 insertions(+), 91 deletions(-)
--
2.39.1
^ permalink raw reply [flat|nested] 4+ messages in thread
* [PATCH v2 1/2] btrfs: remove map_lookup->stripe_len
2023-02-06 1:10 [PATCH v2 0/2] btrfs: reduce div64 calls for __btrfs_map_block() and its variants Qu Wenruo
@ 2023-02-06 1:10 ` Qu Wenruo
2023-02-06 1:10 ` [PATCH v2 2/2] btrfs: reduce div64 calls by limiting the number of stripes of a chunk to u32 Qu Wenruo
1 sibling, 0 replies; 4+ messages in thread
From: Qu Wenruo @ 2023-02-06 1:10 UTC (permalink / raw)
To: linux-btrfs
Currently btrfs doesn't support stripe lengths other than 64KiB.
This is already in the tree-checker.
Thus there is really no meaning to record that fixed value in
map_lookup, and can all be replaced with BTRFS_STRIPE_LEN.
Furthermore we can introduce BTRFS_STRIPE_LEN_SHIFT to change a lot of
multiplication to shift.
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
fs/btrfs/block-group.c | 6 ++---
fs/btrfs/scrub.c | 43 +++++++++++++++----------------
fs/btrfs/tests/extent-map-tests.c | 1 -
fs/btrfs/volumes.c | 39 ++++++++++++----------------
fs/btrfs/volumes.h | 5 +++-
5 files changed, 44 insertions(+), 50 deletions(-)
diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
index 45ccb25c5b1f..1c74af23f54c 100644
--- a/fs/btrfs/block-group.c
+++ b/fs/btrfs/block-group.c
@@ -1994,12 +1994,12 @@ int btrfs_rmap_block(struct btrfs_fs_info *fs_info, u64 chunk_start,
map = em->map_lookup;
data_stripe_length = em->orig_block_len;
- io_stripe_size = map->stripe_len;
+ io_stripe_size = BTRFS_STRIPE_LEN;
chunk_start = em->start;
/* For RAID5/6 adjust to a full IO stripe length */
if (map->type & BTRFS_BLOCK_GROUP_RAID56_MASK)
- io_stripe_size = map->stripe_len * nr_data_stripes(map);
+ io_stripe_size = nr_data_stripes(map) << BTRFS_STRIPE_LEN_SHIFT;
buf = kcalloc(map->num_stripes, sizeof(u64), GFP_NOFS);
if (!buf) {
@@ -2021,7 +2021,7 @@ int btrfs_rmap_block(struct btrfs_fs_info *fs_info, u64 chunk_start,
continue;
stripe_nr = physical - map->stripes[i].physical;
- stripe_nr = div64_u64_rem(stripe_nr, map->stripe_len, &offset);
+ stripe_nr = div64_u64_rem(stripe_nr, BTRFS_STRIPE_LEN, &offset);
if (map->type & (BTRFS_BLOCK_GROUP_RAID0 |
BTRFS_BLOCK_GROUP_RAID10)) {
diff --git a/fs/btrfs/scrub.c b/fs/btrfs/scrub.c
index 69c93ae333f6..db0ee9354fda 100644
--- a/fs/btrfs/scrub.c
+++ b/fs/btrfs/scrub.c
@@ -2722,7 +2722,7 @@ static int scrub_extent(struct scrub_ctx *sctx, struct map_lookup *map,
if (flags & BTRFS_EXTENT_FLAG_DATA) {
if (map->type & BTRFS_BLOCK_GROUP_RAID56_MASK)
- blocksize = map->stripe_len;
+ blocksize = BTRFS_STRIPE_LEN;
else
blocksize = sctx->fs_info->sectorsize;
spin_lock(&sctx->stat_lock);
@@ -2731,7 +2731,7 @@ static int scrub_extent(struct scrub_ctx *sctx, struct map_lookup *map,
spin_unlock(&sctx->stat_lock);
} else if (flags & BTRFS_EXTENT_FLAG_TREE_BLOCK) {
if (map->type & BTRFS_BLOCK_GROUP_RAID56_MASK)
- blocksize = map->stripe_len;
+ blocksize = BTRFS_STRIPE_LEN;
else
blocksize = sctx->fs_info->nodesize;
spin_lock(&sctx->stat_lock);
@@ -2920,9 +2920,9 @@ static int get_raid56_logic_offset(u64 physical, int num,
*offset = last_offset;
for (i = 0; i < data_stripes; i++) {
- *offset = last_offset + i * map->stripe_len;
+ *offset = last_offset + (i << BTRFS_STRIPE_LEN_SHIFT);
- stripe_nr = div64_u64(*offset, map->stripe_len);
+ stripe_nr = div64_u64(*offset, BTRFS_STRIPE_LEN);
stripe_nr = div_u64(stripe_nr, data_stripes);
/* Work out the disk rotation on this stripe-set */
@@ -2935,7 +2935,7 @@ static int get_raid56_logic_offset(u64 physical, int num,
if (stripe_index < num)
j++;
}
- *offset = last_offset + j * map->stripe_len;
+ *offset = last_offset + (j << BTRFS_STRIPE_LEN_SHIFT);
return 1;
}
@@ -3205,7 +3205,7 @@ static int scrub_raid56_data_stripe_for_parity(struct scrub_ctx *sctx,
/* Path must not be populated */
ASSERT(!path->nodes[0]);
- while (cur_logical < logical + map->stripe_len) {
+ while (cur_logical < logical + BTRFS_STRIPE_LEN) {
struct btrfs_io_context *bioc = NULL;
struct btrfs_device *extent_dev;
u64 extent_start;
@@ -3217,7 +3217,7 @@ static int scrub_raid56_data_stripe_for_parity(struct scrub_ctx *sctx,
u64 extent_mirror_num;
ret = find_first_extent_item(extent_root, path, cur_logical,
- logical + map->stripe_len - cur_logical);
+ logical + BTRFS_STRIPE_LEN - cur_logical);
/* No more extent item in this data stripe */
if (ret > 0) {
ret = 0;
@@ -3231,7 +3231,7 @@ static int scrub_raid56_data_stripe_for_parity(struct scrub_ctx *sctx,
/* Metadata should not cross stripe boundaries */
if ((extent_flags & BTRFS_EXTENT_FLAG_TREE_BLOCK) &&
does_range_cross_boundary(extent_start, extent_size,
- logical, map->stripe_len)) {
+ logical, BTRFS_STRIPE_LEN)) {
btrfs_err(fs_info,
"scrub: tree block %llu spanning stripes, ignored. logical=%llu",
extent_start, logical);
@@ -3247,7 +3247,7 @@ static int scrub_raid56_data_stripe_for_parity(struct scrub_ctx *sctx,
/* Truncate the range inside this data stripe */
extent_size = min(extent_start + extent_size,
- logical + map->stripe_len) - cur_logical;
+ logical + BTRFS_STRIPE_LEN) - cur_logical;
extent_start = cur_logical;
ASSERT(extent_size <= U32_MAX);
@@ -3320,8 +3320,7 @@ static noinline_for_stack int scrub_raid56_parity(struct scrub_ctx *sctx,
path->search_commit_root = 1;
path->skip_locking = 1;
- ASSERT(map->stripe_len <= U32_MAX);
- nsectors = map->stripe_len >> fs_info->sectorsize_bits;
+ nsectors = BTRFS_STRIPE_LEN >> fs_info->sectorsize_bits;
ASSERT(nsectors <= BITS_PER_LONG);
sparity = kzalloc(sizeof(struct scrub_parity), GFP_NOFS);
if (!sparity) {
@@ -3332,8 +3331,7 @@ static noinline_for_stack int scrub_raid56_parity(struct scrub_ctx *sctx,
return -ENOMEM;
}
- ASSERT(map->stripe_len <= U32_MAX);
- sparity->stripe_len = map->stripe_len;
+ sparity->stripe_len = BTRFS_STRIPE_LEN;
sparity->nsectors = nsectors;
sparity->sctx = sctx;
sparity->scrub_dev = sdev;
@@ -3344,7 +3342,7 @@ static noinline_for_stack int scrub_raid56_parity(struct scrub_ctx *sctx,
ret = 0;
for (cur_logical = logic_start; cur_logical < logic_end;
- cur_logical += map->stripe_len) {
+ cur_logical += BTRFS_STRIPE_LEN) {
ret = scrub_raid56_data_stripe_for_parity(sctx, sparity, map,
sdev, path, cur_logical);
if (ret < 0)
@@ -3536,7 +3534,7 @@ static u64 simple_stripe_full_stripe_len(const struct map_lookup *map)
ASSERT(map->type & (BTRFS_BLOCK_GROUP_RAID0 |
BTRFS_BLOCK_GROUP_RAID10));
- return map->num_stripes / map->sub_stripes * map->stripe_len;
+ return (map->num_stripes / map->sub_stripes) << BTRFS_STRIPE_LEN_SHIFT;
}
/* Get the logical bytenr for the stripe */
@@ -3552,7 +3550,8 @@ static u64 simple_stripe_get_logical(struct map_lookup *map,
* (stripe_index / sub_stripes) gives how many data stripes we need to
* skip.
*/
- return (stripe_index / map->sub_stripes) * map->stripe_len + bg->start;
+ return ((stripe_index / map->sub_stripes) << BTRFS_STRIPE_LEN_SHIFT) +
+ bg->start;
}
/* Get the mirror number for the stripe */
@@ -3589,14 +3588,14 @@ static int scrub_simple_stripe(struct scrub_ctx *sctx,
* this stripe.
*/
ret = scrub_simple_mirror(sctx, extent_root, csum_root, bg, map,
- cur_logical, map->stripe_len, device,
+ cur_logical, BTRFS_STRIPE_LEN, device,
cur_physical, mirror_num);
if (ret)
return ret;
/* Skip to next stripe which belongs to the target device */
cur_logical += logical_increment;
/* For physical offset, we just go to next stripe */
- cur_physical += map->stripe_len;
+ cur_physical += BTRFS_STRIPE_LEN;
}
return ret;
}
@@ -3690,7 +3689,7 @@ static noinline_for_stack int scrub_stripe(struct scrub_ctx *sctx,
if (profile & (BTRFS_BLOCK_GROUP_RAID0 | BTRFS_BLOCK_GROUP_RAID10)) {
ret = scrub_simple_stripe(sctx, root, csum_root, bg, map,
scrub_dev, stripe_index);
- offset = map->stripe_len * (stripe_index / map->sub_stripes);
+ offset = (stripe_index / map->sub_stripes) << BTRFS_STRIPE_LEN_SHIFT;
goto out;
}
@@ -3705,7 +3704,7 @@ static noinline_for_stack int scrub_stripe(struct scrub_ctx *sctx,
/* Initialize @offset in case we need to go to out: label */
get_raid56_logic_offset(physical, stripe_index, map, &offset, NULL);
- increment = map->stripe_len * nr_data_stripes(map);
+ increment = nr_data_stripes(map) << BTRFS_STRIPE_LEN_SHIFT;
/*
* Due to the rotation, for RAID56 it's better to iterate each stripe
@@ -3736,13 +3735,13 @@ static noinline_for_stack int scrub_stripe(struct scrub_ctx *sctx,
* is still based on @mirror_num.
*/
ret = scrub_simple_mirror(sctx, root, csum_root, bg, map,
- logical, map->stripe_len,
+ logical, BTRFS_STRIPE_LEN,
scrub_dev, physical, 1);
if (ret < 0)
goto out;
next:
logical += increment;
- physical += map->stripe_len;
+ physical += BTRFS_STRIPE_LEN;
spin_lock(&sctx->stat_lock);
if (stop_loop)
sctx->stat.last_physical =
diff --git a/fs/btrfs/tests/extent-map-tests.c b/fs/btrfs/tests/extent-map-tests.c
index c5b3a631bf4f..5d09fc4e1c3c 100644
--- a/fs/btrfs/tests/extent-map-tests.c
+++ b/fs/btrfs/tests/extent-map-tests.c
@@ -486,7 +486,6 @@ static int test_rmap_block(struct btrfs_fs_info *fs_info,
em->map_lookup = map;
map->num_stripes = test->num_stripes;
- map->stripe_len = BTRFS_STRIPE_LEN;
map->type = test->raid_type;
for (i = 0; i < map->num_stripes; i++) {
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 883e5d2a23b6..4c91cc6471c3 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -5095,7 +5095,7 @@ static void init_alloc_chunk_ctl_policy_regular(
/* We don't want a chunk larger than 10% of writable space */
ctl->max_chunk_size = min(mult_perc(fs_devices->total_rw_bytes, 10),
ctl->max_chunk_size);
- ctl->dev_extent_min = BTRFS_STRIPE_LEN * ctl->dev_stripes;
+ ctl->dev_extent_min = ctl->dev_stripes << BTRFS_STRIPE_LEN_SHIFT;
}
static void init_alloc_chunk_ctl_policy_zoned(
@@ -5377,7 +5377,6 @@ static struct btrfs_block_group *create_chunk(struct btrfs_trans_handle *trans,
j * ctl->stripe_size;
}
}
- map->stripe_len = BTRFS_STRIPE_LEN;
map->io_align = BTRFS_STRIPE_LEN;
map->io_width = BTRFS_STRIPE_LEN;
map->type = type;
@@ -5585,11 +5584,11 @@ int btrfs_chunk_alloc_add_chunk_item(struct btrfs_trans_handle *trans,
btrfs_set_stack_chunk_length(chunk, bg->length);
btrfs_set_stack_chunk_owner(chunk, BTRFS_EXTENT_TREE_OBJECTID);
- btrfs_set_stack_chunk_stripe_len(chunk, map->stripe_len);
+ btrfs_set_stack_chunk_stripe_len(chunk, BTRFS_STRIPE_LEN);
btrfs_set_stack_chunk_type(chunk, map->type);
btrfs_set_stack_chunk_num_stripes(chunk, map->num_stripes);
- btrfs_set_stack_chunk_io_align(chunk, map->stripe_len);
- btrfs_set_stack_chunk_io_width(chunk, map->stripe_len);
+ btrfs_set_stack_chunk_io_align(chunk, BTRFS_STRIPE_LEN);
+ btrfs_set_stack_chunk_io_width(chunk, BTRFS_STRIPE_LEN);
btrfs_set_stack_chunk_sector_size(chunk, fs_info->sectorsize);
btrfs_set_stack_chunk_sub_stripes(chunk, map->sub_stripes);
@@ -5779,7 +5778,7 @@ unsigned long btrfs_full_stripe_len(struct btrfs_fs_info *fs_info,
if (!WARN_ON(IS_ERR(em))) {
map = em->map_lookup;
if (map->type & BTRFS_BLOCK_GROUP_RAID56_MASK)
- len = map->stripe_len * nr_data_stripes(map);
+ len = nr_data_stripes(map) << BTRFS_STRIPE_LEN_SHIFT;
free_extent_map(em);
}
return len;
@@ -5945,7 +5944,6 @@ struct btrfs_discard_stripe *btrfs_map_discard(struct btrfs_fs_info *fs_info,
u64 stripe_nr_end;
u64 stripe_end_offset;
u64 stripe_cnt;
- u64 stripe_len;
u64 stripe_offset;
u32 stripe_index;
u32 factor = 0;
@@ -5972,20 +5970,19 @@ struct btrfs_discard_stripe *btrfs_map_discard(struct btrfs_fs_info *fs_info,
length = min_t(u64, em->start + em->len - logical, length);
*length_ret = length;
- stripe_len = map->stripe_len;
/*
* stripe_nr counts the total number of stripes we have to stride
* to get to this block
*/
- stripe_nr = div64_u64(offset, stripe_len);
+ stripe_nr = div64_u64(offset, BTRFS_STRIPE_LEN);
/* stripe_offset is the offset of this block in its stripe */
- stripe_offset = offset - stripe_nr * stripe_len;
+ stripe_offset = offset - (stripe_nr << BTRFS_STRIPE_LEN_SHIFT);
- stripe_nr_end = round_up(offset + length, map->stripe_len);
- stripe_nr_end = div64_u64(stripe_nr_end, map->stripe_len);
+ stripe_nr_end = round_up(offset + length, BTRFS_STRIPE_LEN);
+ stripe_nr_end = div64_u64(stripe_nr_end, BTRFS_STRIPE_LEN);
stripe_cnt = stripe_nr_end - stripe_nr;
- stripe_end_offset = stripe_nr_end * map->stripe_len -
+ stripe_end_offset = (stripe_nr_end << BTRFS_STRIPE_LEN_SHIFT) -
(offset + length);
/*
* after this, stripe_nr is the number of stripes on this
@@ -6027,15 +6024,15 @@ struct btrfs_discard_stripe *btrfs_map_discard(struct btrfs_fs_info *fs_info,
for (i = 0; i < *num_stripes; i++) {
stripes[i].physical =
map->stripes[stripe_index].physical +
- stripe_offset + stripe_nr * map->stripe_len;
+ stripe_offset + (stripe_nr << BTRFS_STRIPE_LEN_SHIFT);
stripes[i].dev = map->stripes[stripe_index].dev;
if (map->type & (BTRFS_BLOCK_GROUP_RAID0 |
BTRFS_BLOCK_GROUP_RAID10)) {
- stripes[i].length = stripes_per_dev * map->stripe_len;
+ stripes[i].length = stripes_per_dev << BTRFS_STRIPE_LEN_SHIFT;
if (i / sub_stripes < remaining_stripes)
- stripes[i].length += map->stripe_len;
+ stripes[i].length += BTRFS_STRIPE_LEN;
/*
* Special for the first stripe and
@@ -6289,11 +6286,11 @@ int btrfs_get_io_geometry(struct btrfs_fs_info *fs_info, struct extent_map *em,
struct btrfs_io_geometry *io_geom)
{
struct map_lookup *map;
+ const u32 stripe_len = BTRFS_STRIPE_LEN;
u64 len;
u64 offset;
u64 stripe_offset;
u64 stripe_nr;
- u32 stripe_len;
u64 raid56_full_stripe_start = (u64)-1;
int data_stripes;
@@ -6302,8 +6299,6 @@ int btrfs_get_io_geometry(struct btrfs_fs_info *fs_info, struct extent_map *em,
map = em->map_lookup;
/* Offset of this logical address in the chunk */
offset = logical - em->start;
- /* Len of a stripe in a chunk */
- stripe_len = map->stripe_len;
/*
* Stripe_nr is where this block falls in
* stripe_offset is the offset of this block in its stripe.
@@ -6362,7 +6357,7 @@ static void set_io_stripe(struct btrfs_io_stripe *dst, const struct map_lookup *
{
dst->dev = map->stripes[stripe_index].dev;
dst->physical = map->stripes[stripe_index].physical +
- stripe_offset + stripe_nr * map->stripe_len;
+ stripe_offset + (stripe_nr << BTRFS_STRIPE_LEN_SHIFT);
}
int __btrfs_map_block(struct btrfs_fs_info *fs_info, enum btrfs_map_op op,
@@ -6481,7 +6476,6 @@ int __btrfs_map_block(struct btrfs_fs_info *fs_info, enum btrfs_map_op op,
}
} else if (map->type & BTRFS_BLOCK_GROUP_RAID56_MASK) {
- ASSERT(map->stripe_len == BTRFS_STRIPE_LEN);
if (need_raid_map && (need_full_stripe(op) || mirror_num > 1)) {
/* push stripe_nr back to the start of the full stripe */
stripe_nr = div64_u64(raid56_full_stripe_start,
@@ -6589,7 +6583,7 @@ int __btrfs_map_block(struct btrfs_fs_info *fs_info, enum btrfs_map_op op,
tmp = stripe_nr * data_stripes;
for (i = 0; i < data_stripes; i++)
bioc->raid_map[(i + rot) % num_stripes] =
- em->start + (tmp + i) * map->stripe_len;
+ em->start + ((tmp + i) << BTRFS_STRIPE_LEN_SHIFT);
bioc->raid_map[(i + rot) % map->num_stripes] = RAID5_P_STRIPE;
if (map->type & BTRFS_BLOCK_GROUP_RAID6)
@@ -6962,7 +6956,6 @@ static int read_one_chunk(struct btrfs_key *key, struct extent_buffer *leaf,
map->num_stripes = num_stripes;
map->io_width = btrfs_chunk_io_width(leaf, chunk);
map->io_align = btrfs_chunk_io_align(leaf, chunk);
- map->stripe_len = btrfs_chunk_stripe_len(leaf, chunk);
map->type = type;
/*
* We can't use the sub_stripes value, as for profiles other than
diff --git a/fs/btrfs/volumes.h b/fs/btrfs/volumes.h
index 6b7a05f6cf82..7d0d9f25864c 100644
--- a/fs/btrfs/volumes.h
+++ b/fs/btrfs/volumes.h
@@ -19,6 +19,10 @@ extern struct mutex uuid_mutex;
#define BTRFS_STRIPE_LEN SZ_64K
+#define BTRFS_STRIPE_LEN_SHIFT (16)
+
+static_assert((1 << BTRFS_STRIPE_LEN_SHIFT) == BTRFS_STRIPE_LEN);
+
/* Used by sanity check for btrfs_raid_types. */
#define const_ffs(n) (__builtin_ctzll(n) + 1)
@@ -461,7 +465,6 @@ struct map_lookup {
u64 type;
int io_align;
int io_width;
- u32 stripe_len;
int num_stripes;
int sub_stripes;
int verified_stripes; /* For mount time dev extent verification */
--
2.39.1
^ permalink raw reply related [flat|nested] 4+ messages in thread
* [PATCH v2 2/2] btrfs: reduce div64 calls by limiting the number of stripes of a chunk to u32
2023-02-06 1:10 [PATCH v2 0/2] btrfs: reduce div64 calls for __btrfs_map_block() and its variants Qu Wenruo
2023-02-06 1:10 ` [PATCH v2 1/2] btrfs: remove map_lookup->stripe_len Qu Wenruo
@ 2023-02-06 1:10 ` Qu Wenruo
2023-02-06 7:07 ` Anand Jain
1 sibling, 1 reply; 4+ messages in thread
From: Qu Wenruo @ 2023-02-06 1:10 UTC (permalink / raw)
To: linux-btrfs
There are quite some div64 calls inside btrfs_map_block() and its
variants.
One of such calls are for @stripe_nr, where @stripe_nr is the number of
stripes before our logical bytenr inside a chunk.
However we can eliminate such div64 calls by just reducing the width of
@stripe_nr from 64 to 32.
This can be done because our chunk size limit is already 10G, with fixed
stripe length 64K.
Thus a U32 is definitely enough to contain the number of stripes.
With such width reduction, we can get rid of slower div64, and extra
warning for certain 32bit arch.
This patch would do:
- Reduce the width of various stripe_nr variables
- Add a new tree-checker chunk validation on chunk length
Make sure no chunk can reach 256G, which can also act as a bitflip
checker.
- Replace unnecessary div64 calls with regular modulo and division
32bit division and modulo are much faster than 64bit operations, and
we are finally free of the div64 fear at least in those involved
functions.
Signed-off-by: Qu Wenruo <wqu@suse.com>
---
fs/btrfs/block-group.c | 18 +++++-----
fs/btrfs/tree-checker.c | 14 ++++++++
fs/btrfs/volumes.c | 79 ++++++++++++++++++++++-------------------
fs/btrfs/volumes.h | 2 +-
4 files changed, 67 insertions(+), 46 deletions(-)
diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
index 1c74af23f54c..93938e673f1e 100644
--- a/fs/btrfs/block-group.c
+++ b/fs/btrfs/block-group.c
@@ -2009,8 +2009,8 @@ int btrfs_rmap_block(struct btrfs_fs_info *fs_info, u64 chunk_start,
for (i = 0; i < map->num_stripes; i++) {
bool already_inserted = false;
- u64 stripe_nr;
- u64 offset;
+ u32 stripe_nr;
+ u32 offset;
int j;
if (!in_range(physical, map->stripes[i].physical,
@@ -2020,20 +2020,20 @@ int btrfs_rmap_block(struct btrfs_fs_info *fs_info, u64 chunk_start,
if (bdev && map->stripes[i].dev->bdev != bdev)
continue;
- stripe_nr = physical - map->stripes[i].physical;
- stripe_nr = div64_u64_rem(stripe_nr, BTRFS_STRIPE_LEN, &offset);
+ stripe_nr = (physical - map->stripes[i].physical) >>
+ BTRFS_STRIPE_LEN_SHIFT;
+ offset = (physical - map->stripes[i].physical) &
+ ((1 << BTRFS_STRIPE_LEN_SHIFT) - 1);
if (map->type & (BTRFS_BLOCK_GROUP_RAID0 |
- BTRFS_BLOCK_GROUP_RAID10)) {
- stripe_nr = stripe_nr * map->num_stripes + i;
- stripe_nr = div_u64(stripe_nr, map->sub_stripes);
- }
+ BTRFS_BLOCK_GROUP_RAID10))
+ stripe_nr = div_u64(stripe_nr * map->num_stripes + 1,
+ map->sub_stripes);
/*
* The remaining case would be for RAID56, multiply by
* nr_data_stripes(). Alternatively, just use rmap_len below
* instead of map->stripe_len
*/
-
bytenr = chunk_start + stripe_nr * io_stripe_size + offset;
/* Ensure we don't add duplicate addresses */
diff --git a/fs/btrfs/tree-checker.c b/fs/btrfs/tree-checker.c
index baad1ed7e111..7968fc89f278 100644
--- a/fs/btrfs/tree-checker.c
+++ b/fs/btrfs/tree-checker.c
@@ -849,6 +849,20 @@ int btrfs_check_chunk_valid(struct extent_buffer *leaf,
stripe_len);
return -EUCLEAN;
}
+ /*
+ * We artificially limit the chunk size, so that the number of stripes
+ * inside a chunk can be fit into a U32.
+ * The current limit (256G) would be way too large for real world usage
+ * anyway, and it's already way larger than our existing limit (10G).
+ *
+ * Thus it should be a good way to catch obvious bitflip.
+ */
+ if (unlikely(((u64)U32_MAX << BTRFS_STRIPE_LEN_SHIFT) <= length)) {
+ chunk_err(leaf, chunk, logical,
+ "chunk length too large: have %llu limit %llu",
+ length, (u64)U32_MAX << BTRFS_STRIPE_LEN_SHIFT);
+ return -EUCLEAN;
+ }
if (unlikely(type & ~(BTRFS_BLOCK_GROUP_TYPE_MASK |
BTRFS_BLOCK_GROUP_PROFILE_MASK))) {
chunk_err(leaf, chunk, logical,
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 4c91cc6471c3..ce073083b0dc 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -5940,15 +5940,15 @@ struct btrfs_discard_stripe *btrfs_map_discard(struct btrfs_fs_info *fs_info,
struct btrfs_discard_stripe *stripes;
u64 length = *length_ret;
u64 offset;
- u64 stripe_nr;
- u64 stripe_nr_end;
+ u32 stripe_nr;
+ u32 stripe_nr_end;
+ u32 stripe_cnt;
u64 stripe_end_offset;
- u64 stripe_cnt;
u64 stripe_offset;
u32 stripe_index;
u32 factor = 0;
u32 sub_stripes = 0;
- u64 stripes_per_dev = 0;
+ u32 stripes_per_dev = 0;
u32 remaining_stripes = 0;
u32 last_stripe = 0;
int ret;
@@ -5974,13 +5974,13 @@ struct btrfs_discard_stripe *btrfs_map_discard(struct btrfs_fs_info *fs_info,
* stripe_nr counts the total number of stripes we have to stride
* to get to this block
*/
- stripe_nr = div64_u64(offset, BTRFS_STRIPE_LEN);
+ stripe_nr = offset >> BTRFS_STRIPE_LEN_SHIFT;
/* stripe_offset is the offset of this block in its stripe */
stripe_offset = offset - (stripe_nr << BTRFS_STRIPE_LEN_SHIFT);
- stripe_nr_end = round_up(offset + length, BTRFS_STRIPE_LEN);
- stripe_nr_end = div64_u64(stripe_nr_end, BTRFS_STRIPE_LEN);
+ stripe_nr_end = round_up(offset + length, BTRFS_STRIPE_LEN) >>
+ BTRFS_STRIPE_LEN_SHIFT;
stripe_cnt = stripe_nr_end - stripe_nr;
stripe_end_offset = (stripe_nr_end << BTRFS_STRIPE_LEN_SHIFT) -
(offset + length);
@@ -6001,18 +6001,18 @@ struct btrfs_discard_stripe *btrfs_map_discard(struct btrfs_fs_info *fs_info,
factor = map->num_stripes / sub_stripes;
*num_stripes = min_t(u64, map->num_stripes,
sub_stripes * stripe_cnt);
- stripe_nr = div_u64_rem(stripe_nr, factor, &stripe_index);
+ stripe_index = stripe_nr % factor;
+ stripe_nr /= factor;
stripe_index *= sub_stripes;
stripes_per_dev = div_u64_rem(stripe_cnt, factor,
&remaining_stripes);
- div_u64_rem(stripe_nr_end - 1, factor, &last_stripe);
- last_stripe *= sub_stripes;
+ last_stripe = (stripe_nr_end - 1) % factor * sub_stripes;
} else if (map->type & (BTRFS_BLOCK_GROUP_RAID1_MASK |
BTRFS_BLOCK_GROUP_DUP)) {
*num_stripes = map->num_stripes;
} else {
- stripe_nr = div_u64_rem(stripe_nr, map->num_stripes,
- &stripe_index);
+ stripe_index = stripe_nr % map->num_stripes;
+ stripe_nr /= map->num_stripes;
}
stripes = kcalloc(*num_stripes, sizeof(*stripes), GFP_NOFS);
@@ -6286,11 +6286,10 @@ int btrfs_get_io_geometry(struct btrfs_fs_info *fs_info, struct extent_map *em,
struct btrfs_io_geometry *io_geom)
{
struct map_lookup *map;
- const u32 stripe_len = BTRFS_STRIPE_LEN;
u64 len;
u64 offset;
u64 stripe_offset;
- u64 stripe_nr;
+ u32 stripe_nr;
u64 raid56_full_stripe_start = (u64)-1;
int data_stripes;
@@ -6303,20 +6302,22 @@ int btrfs_get_io_geometry(struct btrfs_fs_info *fs_info, struct extent_map *em,
* Stripe_nr is where this block falls in
* stripe_offset is the offset of this block in its stripe.
*/
- stripe_nr = div64_u64_rem(offset, stripe_len, &stripe_offset);
+ stripe_offset = offset & ((1 << BTRFS_STRIPE_LEN_SHIFT) - 1);
+ stripe_nr = offset >> BTRFS_STRIPE_LEN_SHIFT;
ASSERT(stripe_offset < U32_MAX);
data_stripes = nr_data_stripes(map);
/* Only stripe based profiles needs to check against stripe length. */
if (map->type & BTRFS_BLOCK_GROUP_STRIPE_MASK) {
- u64 max_len = stripe_len - stripe_offset;
+ u64 max_len = BTRFS_STRIPE_LEN - stripe_offset;
/*
* In case of raid56, we need to know the stripe aligned start
*/
if (map->type & BTRFS_BLOCK_GROUP_RAID56_MASK) {
- unsigned long full_stripe_len = stripe_len * data_stripes;
+ unsigned long full_stripe_len = data_stripes <<
+ BTRFS_STRIPE_LEN_SHIFT;
raid56_full_stripe_start = offset;
/*
@@ -6333,7 +6334,7 @@ int btrfs_get_io_geometry(struct btrfs_fs_info *fs_info, struct extent_map *em,
* reads, just allow a single stripe (on a single disk).
*/
if (op == BTRFS_MAP_WRITE) {
- max_len = stripe_len * data_stripes -
+ max_len = (data_stripes << BTRFS_STRIPE_LEN_SHIFT) -
(offset - raid56_full_stripe_start);
}
}
@@ -6344,7 +6345,7 @@ int btrfs_get_io_geometry(struct btrfs_fs_info *fs_info, struct extent_map *em,
io_geom->len = len;
io_geom->offset = offset;
- io_geom->stripe_len = stripe_len;
+ io_geom->stripe_len = BTRFS_STRIPE_LEN;
io_geom->stripe_nr = stripe_nr;
io_geom->stripe_offset = stripe_offset;
io_geom->raid56_stripe_offset = raid56_full_stripe_start;
@@ -6353,7 +6354,7 @@ int btrfs_get_io_geometry(struct btrfs_fs_info *fs_info, struct extent_map *em,
}
static void set_io_stripe(struct btrfs_io_stripe *dst, const struct map_lookup *map,
- u32 stripe_index, u64 stripe_offset, u64 stripe_nr)
+ u32 stripe_index, u64 stripe_offset, u32 stripe_nr)
{
dst->dev = map->stripes[stripe_index].dev;
dst->physical = map->stripes[stripe_index].physical +
@@ -6369,8 +6370,8 @@ int __btrfs_map_block(struct btrfs_fs_info *fs_info, enum btrfs_map_op op,
struct extent_map *em;
struct map_lookup *map;
u64 stripe_offset;
- u64 stripe_nr;
u64 stripe_len;
+ u32 stripe_nr;
u32 stripe_index;
int data_stripes;
int i;
@@ -6433,8 +6434,8 @@ int __btrfs_map_block(struct btrfs_fs_info *fs_info, enum btrfs_map_op op,
num_stripes = 1;
stripe_index = 0;
if (map->type & BTRFS_BLOCK_GROUP_RAID0) {
- stripe_nr = div_u64_rem(stripe_nr, map->num_stripes,
- &stripe_index);
+ stripe_index = stripe_nr % map->num_stripes;
+ stripe_nr /= map->num_stripes;
if (!need_full_stripe(op))
mirror_num = 1;
} else if (map->type & BTRFS_BLOCK_GROUP_RAID1_MASK) {
@@ -6460,8 +6461,8 @@ int __btrfs_map_block(struct btrfs_fs_info *fs_info, enum btrfs_map_op op,
} else if (map->type & BTRFS_BLOCK_GROUP_RAID10) {
u32 factor = map->num_stripes / map->sub_stripes;
- stripe_nr = div_u64_rem(stripe_nr, factor, &stripe_index);
- stripe_index *= map->sub_stripes;
+ stripe_index = stripe_nr % factor * map->sub_stripes;
+ stripe_nr /= factor;
if (need_full_stripe(op))
num_stripes = map->sub_stripes;
@@ -6477,9 +6478,16 @@ int __btrfs_map_block(struct btrfs_fs_info *fs_info, enum btrfs_map_op op,
} else if (map->type & BTRFS_BLOCK_GROUP_RAID56_MASK) {
if (need_raid_map && (need_full_stripe(op) || mirror_num > 1)) {
- /* push stripe_nr back to the start of the full stripe */
- stripe_nr = div64_u64(raid56_full_stripe_start,
- stripe_len * data_stripes);
+ /*
+ * Push stripe_nr back to the start of the full stripe
+ * For those cases needing a full stripe, @stripe_nr
+ * is the full stripe number.
+ *
+ * Original we go raid56_full_stripe_start / full_stripe_len,
+ * but that can be expensive.
+ * Here we just divide @stripe_nr with @data_stripes.
+ */
+ stripe_nr /= data_stripes;
/* RAID[56] write or recovery. Return all stripes */
num_stripes = map->num_stripes;
@@ -6497,25 +6505,24 @@ int __btrfs_map_block(struct btrfs_fs_info *fs_info, enum btrfs_map_op op,
* Mirror #2 is RAID5 parity block.
* Mirror #3 is RAID6 Q block.
*/
- stripe_nr = div_u64_rem(stripe_nr,
- data_stripes, &stripe_index);
+ stripe_index = stripe_nr % data_stripes;
+ stripe_nr /= data_stripes;
if (mirror_num > 1)
stripe_index = data_stripes + mirror_num - 2;
/* We distribute the parity blocks across stripes */
- div_u64_rem(stripe_nr + stripe_index, map->num_stripes,
- &stripe_index);
+ stripe_index = (stripe_nr + stripe_index) % map->num_stripes;
if (!need_full_stripe(op) && mirror_num <= 1)
mirror_num = 1;
}
} else {
/*
- * after this, stripe_nr is the number of stripes on this
+ * After this, stripe_nr is the number of stripes on this
* device we have to walk to find the data, and stripe_index is
* the number of our device in the stripe array
*/
- stripe_nr = div_u64_rem(stripe_nr, map->num_stripes,
- &stripe_index);
+ stripe_index = stripe_nr % map->num_stripes;
+ stripe_nr /= map->num_stripes;
mirror_num = stripe_index + 1;
}
if (stripe_index >= map->num_stripes) {
@@ -6577,7 +6584,7 @@ int __btrfs_map_block(struct btrfs_fs_info *fs_info, enum btrfs_map_op op,
unsigned rot;
/* Work out the disk rotation on this stripe-set */
- div_u64_rem(stripe_nr, num_stripes, &rot);
+ rot = stripe_nr % num_stripes;
/* Fill in the logical address of each stripe */
tmp = stripe_nr * data_stripes;
diff --git a/fs/btrfs/volumes.h b/fs/btrfs/volumes.h
index 7d0d9f25864c..96dda0f4c564 100644
--- a/fs/btrfs/volumes.h
+++ b/fs/btrfs/volumes.h
@@ -67,7 +67,7 @@ struct btrfs_io_geometry {
/* offset of address in stripe */
u32 stripe_offset;
/* number of stripe where address falls */
- u64 stripe_nr;
+ u32 stripe_nr;
/* offset of raid56 stripe into the chunk */
u64 raid56_stripe_offset;
};
--
2.39.1
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH v2 2/2] btrfs: reduce div64 calls by limiting the number of stripes of a chunk to u32
2023-02-06 1:10 ` [PATCH v2 2/2] btrfs: reduce div64 calls by limiting the number of stripes of a chunk to u32 Qu Wenruo
@ 2023-02-06 7:07 ` Anand Jain
0 siblings, 0 replies; 4+ messages in thread
From: Anand Jain @ 2023-02-06 7:07 UTC (permalink / raw)
To: Qu Wenruo, linux-btrfs
On 2/6/23 09:10, Qu Wenruo wrote:
> There are quite some div64 calls inside btrfs_map_block() and its
> variants.
>
> One of such calls are for @stripe_nr, where @stripe_nr is the number of
> stripes before our logical bytenr inside a chunk.
>
> However we can eliminate such div64 calls by just reducing the width of
> @stripe_nr from 64 to 32.
>
> This can be done because our chunk size limit is already 10G, with fixed
> stripe length 64K.
> Thus a U32 is definitely enough to contain the number of stripes.
>
> With such width reduction, we can get rid of slower div64, and extra
> warning for certain 32bit arch.
>
> This patch would do:
>
> - Reduce the width of various stripe_nr variables
>
> - Add a new tree-checker chunk validation on chunk length
> Make sure no chunk can reach 256G, which can also act as a bitflip
> checker.
>
> - Replace unnecessary div64 calls with regular modulo and division
> 32bit division and modulo are much faster than 64bit operations, and
> we are finally free of the div64 fear at least in those involved
> functions.
>
Sound good. However, it appears that some of the changes should have
been part of the patch 1; for instance, the %full_stripe_len in
the function btrfs_get_io_geometry().
Thanks, Anand
> Signed-off-by: Qu Wenruo <wqu@suse.com>
> ---
> fs/btrfs/block-group.c | 18 +++++-----
> fs/btrfs/tree-checker.c | 14 ++++++++
> fs/btrfs/volumes.c | 79 ++++++++++++++++++++++-------------------
> fs/btrfs/volumes.h | 2 +-
> 4 files changed, 67 insertions(+), 46 deletions(-)
>
> diff --git a/fs/btrfs/block-group.c b/fs/btrfs/block-group.c
> index 1c74af23f54c..93938e673f1e 100644
> --- a/fs/btrfs/block-group.c
> +++ b/fs/btrfs/block-group.c
> @@ -2009,8 +2009,8 @@ int btrfs_rmap_block(struct btrfs_fs_info *fs_info, u64 chunk_start,
>
> for (i = 0; i < map->num_stripes; i++) {
> bool already_inserted = false;
> - u64 stripe_nr;
> - u64 offset;
> + u32 stripe_nr;
> + u32 offset;
> int j;
>
> if (!in_range(physical, map->stripes[i].physical,
> @@ -2020,20 +2020,20 @@ int btrfs_rmap_block(struct btrfs_fs_info *fs_info, u64 chunk_start,
> if (bdev && map->stripes[i].dev->bdev != bdev)
> continue;
>
> - stripe_nr = physical - map->stripes[i].physical;
> - stripe_nr = div64_u64_rem(stripe_nr, BTRFS_STRIPE_LEN, &offset);
> + stripe_nr = (physical - map->stripes[i].physical) >>
> + BTRFS_STRIPE_LEN_SHIFT;
> + offset = (physical - map->stripes[i].physical) &
> + ((1 << BTRFS_STRIPE_LEN_SHIFT) - 1);
>
> if (map->type & (BTRFS_BLOCK_GROUP_RAID0 |
> - BTRFS_BLOCK_GROUP_RAID10)) {
> - stripe_nr = stripe_nr * map->num_stripes + i;
> - stripe_nr = div_u64(stripe_nr, map->sub_stripes);
> - }
> + BTRFS_BLOCK_GROUP_RAID10))
> + stripe_nr = div_u64(stripe_nr * map->num_stripes + 1,
> + map->sub_stripes);
> /*
> * The remaining case would be for RAID56, multiply by
> * nr_data_stripes(). Alternatively, just use rmap_len below
> * instead of map->stripe_len
> */
> -
> bytenr = chunk_start + stripe_nr * io_stripe_size + offset;
>
> /* Ensure we don't add duplicate addresses */
> diff --git a/fs/btrfs/tree-checker.c b/fs/btrfs/tree-checker.c
> index baad1ed7e111..7968fc89f278 100644
> --- a/fs/btrfs/tree-checker.c
> +++ b/fs/btrfs/tree-checker.c
> @@ -849,6 +849,20 @@ int btrfs_check_chunk_valid(struct extent_buffer *leaf,
> stripe_len);
> return -EUCLEAN;
> }
> + /*
> + * We artificially limit the chunk size, so that the number of stripes
> + * inside a chunk can be fit into a U32.
> + * The current limit (256G) would be way too large for real world usage
> + * anyway, and it's already way larger than our existing limit (10G).
> + *
> + * Thus it should be a good way to catch obvious bitflip.
> + */
> + if (unlikely(((u64)U32_MAX << BTRFS_STRIPE_LEN_SHIFT) <= length)) {
> + chunk_err(leaf, chunk, logical,
> + "chunk length too large: have %llu limit %llu",
> + length, (u64)U32_MAX << BTRFS_STRIPE_LEN_SHIFT);
> + return -EUCLEAN;
> + }
> if (unlikely(type & ~(BTRFS_BLOCK_GROUP_TYPE_MASK |
> BTRFS_BLOCK_GROUP_PROFILE_MASK))) {
> chunk_err(leaf, chunk, logical,
> diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
> index 4c91cc6471c3..ce073083b0dc 100644
> --- a/fs/btrfs/volumes.c
> +++ b/fs/btrfs/volumes.c
> @@ -5940,15 +5940,15 @@ struct btrfs_discard_stripe *btrfs_map_discard(struct btrfs_fs_info *fs_info,
> struct btrfs_discard_stripe *stripes;
> u64 length = *length_ret;
> u64 offset;
> - u64 stripe_nr;
> - u64 stripe_nr_end;
> + u32 stripe_nr;
> + u32 stripe_nr_end;
> + u32 stripe_cnt;
> u64 stripe_end_offset;
> - u64 stripe_cnt;
> u64 stripe_offset;
> u32 stripe_index;
> u32 factor = 0;
> u32 sub_stripes = 0;
> - u64 stripes_per_dev = 0;
> + u32 stripes_per_dev = 0;
> u32 remaining_stripes = 0;
> u32 last_stripe = 0;
> int ret;
> @@ -5974,13 +5974,13 @@ struct btrfs_discard_stripe *btrfs_map_discard(struct btrfs_fs_info *fs_info,
> * stripe_nr counts the total number of stripes we have to stride
> * to get to this block
> */
> - stripe_nr = div64_u64(offset, BTRFS_STRIPE_LEN);
> + stripe_nr = offset >> BTRFS_STRIPE_LEN_SHIFT;
>
> /* stripe_offset is the offset of this block in its stripe */
> stripe_offset = offset - (stripe_nr << BTRFS_STRIPE_LEN_SHIFT);
>
> - stripe_nr_end = round_up(offset + length, BTRFS_STRIPE_LEN);
> - stripe_nr_end = div64_u64(stripe_nr_end, BTRFS_STRIPE_LEN);
> + stripe_nr_end = round_up(offset + length, BTRFS_STRIPE_LEN) >>
> + BTRFS_STRIPE_LEN_SHIFT;
> stripe_cnt = stripe_nr_end - stripe_nr;
> stripe_end_offset = (stripe_nr_end << BTRFS_STRIPE_LEN_SHIFT) -
> (offset + length);
> @@ -6001,18 +6001,18 @@ struct btrfs_discard_stripe *btrfs_map_discard(struct btrfs_fs_info *fs_info,
> factor = map->num_stripes / sub_stripes;
> *num_stripes = min_t(u64, map->num_stripes,
> sub_stripes * stripe_cnt);
> - stripe_nr = div_u64_rem(stripe_nr, factor, &stripe_index);
> + stripe_index = stripe_nr % factor;
> + stripe_nr /= factor;
> stripe_index *= sub_stripes;
> stripes_per_dev = div_u64_rem(stripe_cnt, factor,
> &remaining_stripes);
> - div_u64_rem(stripe_nr_end - 1, factor, &last_stripe);
> - last_stripe *= sub_stripes;
> + last_stripe = (stripe_nr_end - 1) % factor * sub_stripes;
> } else if (map->type & (BTRFS_BLOCK_GROUP_RAID1_MASK |
> BTRFS_BLOCK_GROUP_DUP)) {
> *num_stripes = map->num_stripes;
> } else {
> - stripe_nr = div_u64_rem(stripe_nr, map->num_stripes,
> - &stripe_index);
> + stripe_index = stripe_nr % map->num_stripes;
> + stripe_nr /= map->num_stripes;
> }
>
> stripes = kcalloc(*num_stripes, sizeof(*stripes), GFP_NOFS);
> @@ -6286,11 +6286,10 @@ int btrfs_get_io_geometry(struct btrfs_fs_info *fs_info, struct extent_map *em,
> struct btrfs_io_geometry *io_geom)
> {
> struct map_lookup *map;
> - const u32 stripe_len = BTRFS_STRIPE_LEN;
> u64 len;
> u64 offset;
> u64 stripe_offset;
> - u64 stripe_nr;
> + u32 stripe_nr;
> u64 raid56_full_stripe_start = (u64)-1;
> int data_stripes;
>
> @@ -6303,20 +6302,22 @@ int btrfs_get_io_geometry(struct btrfs_fs_info *fs_info, struct extent_map *em,
> * Stripe_nr is where this block falls in
> * stripe_offset is the offset of this block in its stripe.
> */
> - stripe_nr = div64_u64_rem(offset, stripe_len, &stripe_offset);
> + stripe_offset = offset & ((1 << BTRFS_STRIPE_LEN_SHIFT) - 1);
> + stripe_nr = offset >> BTRFS_STRIPE_LEN_SHIFT;
> ASSERT(stripe_offset < U32_MAX);
>
> data_stripes = nr_data_stripes(map);
>
> /* Only stripe based profiles needs to check against stripe length. */
> if (map->type & BTRFS_BLOCK_GROUP_STRIPE_MASK) {
> - u64 max_len = stripe_len - stripe_offset;
> + u64 max_len = BTRFS_STRIPE_LEN - stripe_offset;
>
> /*
> * In case of raid56, we need to know the stripe aligned start
> */
> if (map->type & BTRFS_BLOCK_GROUP_RAID56_MASK) {
> - unsigned long full_stripe_len = stripe_len * data_stripes;
> + unsigned long full_stripe_len = data_stripes <<
> + BTRFS_STRIPE_LEN_SHIFT;
> raid56_full_stripe_start = offset;
>
> /*
> @@ -6333,7 +6334,7 @@ int btrfs_get_io_geometry(struct btrfs_fs_info *fs_info, struct extent_map *em,
> * reads, just allow a single stripe (on a single disk).
> */
> if (op == BTRFS_MAP_WRITE) {
> - max_len = stripe_len * data_stripes -
> + max_len = (data_stripes << BTRFS_STRIPE_LEN_SHIFT) -
> (offset - raid56_full_stripe_start);
> }
> }
> @@ -6344,7 +6345,7 @@ int btrfs_get_io_geometry(struct btrfs_fs_info *fs_info, struct extent_map *em,
>
> io_geom->len = len;
> io_geom->offset = offset;
> - io_geom->stripe_len = stripe_len;
> + io_geom->stripe_len = BTRFS_STRIPE_LEN;
> io_geom->stripe_nr = stripe_nr;
> io_geom->stripe_offset = stripe_offset;
> io_geom->raid56_stripe_offset = raid56_full_stripe_start;
> @@ -6353,7 +6354,7 @@ int btrfs_get_io_geometry(struct btrfs_fs_info *fs_info, struct extent_map *em,
> }
>
> static void set_io_stripe(struct btrfs_io_stripe *dst, const struct map_lookup *map,
> - u32 stripe_index, u64 stripe_offset, u64 stripe_nr)
> + u32 stripe_index, u64 stripe_offset, u32 stripe_nr)
> {
> dst->dev = map->stripes[stripe_index].dev;
> dst->physical = map->stripes[stripe_index].physical +
> @@ -6369,8 +6370,8 @@ int __btrfs_map_block(struct btrfs_fs_info *fs_info, enum btrfs_map_op op,
> struct extent_map *em;
> struct map_lookup *map;
> u64 stripe_offset;
> - u64 stripe_nr;
> u64 stripe_len;
> + u32 stripe_nr;
> u32 stripe_index;
> int data_stripes;
> int i;
> @@ -6433,8 +6434,8 @@ int __btrfs_map_block(struct btrfs_fs_info *fs_info, enum btrfs_map_op op,
> num_stripes = 1;
> stripe_index = 0;
> if (map->type & BTRFS_BLOCK_GROUP_RAID0) {
> - stripe_nr = div_u64_rem(stripe_nr, map->num_stripes,
> - &stripe_index);
> + stripe_index = stripe_nr % map->num_stripes;
> + stripe_nr /= map->num_stripes;
> if (!need_full_stripe(op))
> mirror_num = 1;
> } else if (map->type & BTRFS_BLOCK_GROUP_RAID1_MASK) {
> @@ -6460,8 +6461,8 @@ int __btrfs_map_block(struct btrfs_fs_info *fs_info, enum btrfs_map_op op,
> } else if (map->type & BTRFS_BLOCK_GROUP_RAID10) {
> u32 factor = map->num_stripes / map->sub_stripes;
>
> - stripe_nr = div_u64_rem(stripe_nr, factor, &stripe_index);
> - stripe_index *= map->sub_stripes;
> + stripe_index = stripe_nr % factor * map->sub_stripes;
> + stripe_nr /= factor;
>
> if (need_full_stripe(op))
> num_stripes = map->sub_stripes;
> @@ -6477,9 +6478,16 @@ int __btrfs_map_block(struct btrfs_fs_info *fs_info, enum btrfs_map_op op,
>
> } else if (map->type & BTRFS_BLOCK_GROUP_RAID56_MASK) {
> if (need_raid_map && (need_full_stripe(op) || mirror_num > 1)) {
> - /* push stripe_nr back to the start of the full stripe */
> - stripe_nr = div64_u64(raid56_full_stripe_start,
> - stripe_len * data_stripes);
> + /*
> + * Push stripe_nr back to the start of the full stripe
> + * For those cases needing a full stripe, @stripe_nr
> + * is the full stripe number.
> + *
> + * Original we go raid56_full_stripe_start / full_stripe_len,
> + * but that can be expensive.
> + * Here we just divide @stripe_nr with @data_stripes.
> + */
> + stripe_nr /= data_stripes;
>
> /* RAID[56] write or recovery. Return all stripes */
> num_stripes = map->num_stripes;
> @@ -6497,25 +6505,24 @@ int __btrfs_map_block(struct btrfs_fs_info *fs_info, enum btrfs_map_op op,
> * Mirror #2 is RAID5 parity block.
> * Mirror #3 is RAID6 Q block.
> */
> - stripe_nr = div_u64_rem(stripe_nr,
> - data_stripes, &stripe_index);
> + stripe_index = stripe_nr % data_stripes;
> + stripe_nr /= data_stripes;
> if (mirror_num > 1)
> stripe_index = data_stripes + mirror_num - 2;
>
> /* We distribute the parity blocks across stripes */
> - div_u64_rem(stripe_nr + stripe_index, map->num_stripes,
> - &stripe_index);
> + stripe_index = (stripe_nr + stripe_index) % map->num_stripes;
> if (!need_full_stripe(op) && mirror_num <= 1)
> mirror_num = 1;
> }
> } else {
> /*
> - * after this, stripe_nr is the number of stripes on this
> + * After this, stripe_nr is the number of stripes on this
> * device we have to walk to find the data, and stripe_index is
> * the number of our device in the stripe array
> */
> - stripe_nr = div_u64_rem(stripe_nr, map->num_stripes,
> - &stripe_index);
> + stripe_index = stripe_nr % map->num_stripes;
> + stripe_nr /= map->num_stripes;
> mirror_num = stripe_index + 1;
> }
> if (stripe_index >= map->num_stripes) {
> @@ -6577,7 +6584,7 @@ int __btrfs_map_block(struct btrfs_fs_info *fs_info, enum btrfs_map_op op,
> unsigned rot;
>
> /* Work out the disk rotation on this stripe-set */
> - div_u64_rem(stripe_nr, num_stripes, &rot);
> + rot = stripe_nr % num_stripes;
>
> /* Fill in the logical address of each stripe */
> tmp = stripe_nr * data_stripes;
> diff --git a/fs/btrfs/volumes.h b/fs/btrfs/volumes.h
> index 7d0d9f25864c..96dda0f4c564 100644
> --- a/fs/btrfs/volumes.h
> +++ b/fs/btrfs/volumes.h
> @@ -67,7 +67,7 @@ struct btrfs_io_geometry {
> /* offset of address in stripe */
> u32 stripe_offset;
> /* number of stripe where address falls */
> - u64 stripe_nr;
> + u32 stripe_nr;
> /* offset of raid56 stripe into the chunk */
> u64 raid56_stripe_offset;
> };
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2023-02-06 7:07 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-02-06 1:10 [PATCH v2 0/2] btrfs: reduce div64 calls for __btrfs_map_block() and its variants Qu Wenruo
2023-02-06 1:10 ` [PATCH v2 1/2] btrfs: remove map_lookup->stripe_len Qu Wenruo
2023-02-06 1:10 ` [PATCH v2 2/2] btrfs: reduce div64 calls by limiting the number of stripes of a chunk to u32 Qu Wenruo
2023-02-06 7:07 ` Anand Jain
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox