* [RFC PATCH v2] mkfs.f2fs: recalculate sit_segments by max_sit_bitmap_size
@ 2016-02-23 3:46 Junling Zheng
2016-02-23 5:28 ` Chao Yu
0 siblings, 1 reply; 5+ messages in thread
From: Junling Zheng @ 2016-02-23 3:46 UTC (permalink / raw)
To: linux-f2fs-devel, jaegeuk
In most cases, sit_bitmap_size is smaller than MAX_SIT_BITMAP_SIZE.
However, in some extreme scenarios, such as 16TB, sit_bitmap_size
could be larger than MAX_SIT_BITMAP_SIZE.
In this case, we should recalculate the sit_segments through
max_sit_bitmap_size to prevent sit_ver_bitmap_bytesize got from
segment_count_sit in f2fs_write_check_point_pack() being over
MAX_SIT_BITMAP_SIZE.
Signed-off-by: Junling Zheng <zhengjunling@huawei.com>
---
mkfs/f2fs_format.c | 23 ++++++++++++++++-------
1 file changed, 16 insertions(+), 7 deletions(-)
diff --git a/mkfs/f2fs_format.c b/mkfs/f2fs_format.c
index 645c2aa..3a050e0 100644
--- a/mkfs/f2fs_format.c
+++ b/mkfs/f2fs_format.c
@@ -191,6 +191,22 @@ static int f2fs_prepare_super_block(void)
sit_segments = SEG_ALIGN(blocks_for_sit);
+ /*
+ * In most cases, sit_bitmap_size is smaller than MAX_SIT_BITMAP_SIZE.
+ * However, in an extreme scenario(16TB), sit_bitmap_size could be larger
+ * than MAX_SIT_BITMAP_SIZE. Thus, we should recalculate the sit_segments
+ * to prevent sit_ver_bitmap_bytesize got from segment_count_sit in
+ * f2fs_write_check_point_pack() being over MAX_SIT_BITMAP_SIZE.
+ */
+ sit_bitmap_size = (sit_segments << log_blks_per_seg) / 8;
+
+ if (sit_bitmap_size > MAX_SIT_BITMAP_SIZE) {
+ max_sit_bitmap_size = MAX_SIT_BITMAP_SIZE;
+ sit_segments = max_sit_bitmap_size * 8 >> log_blks_per_seg;
+ blocks_for_sit = sit_segments << log_blks_per_seg;
+ } else
+ max_sit_bitmap_size = sit_bitmap_size;
+
set_sb(segment_count_sit, sit_segments * 2);
set_sb(nat_blkaddr, get_sb(sit_blkaddr) + get_sb(segment_count_sit) *
@@ -208,13 +224,6 @@ static int f2fs_prepare_super_block(void)
* This number resizes NAT bitmap area in a CP page.
* So the threshold is determined not to overflow one CP page
*/
- sit_bitmap_size = ((get_sb(segment_count_sit) / 2) <<
- log_blks_per_seg) / 8;
-
- if (sit_bitmap_size > MAX_SIT_BITMAP_SIZE)
- max_sit_bitmap_size = MAX_SIT_BITMAP_SIZE;
- else
- max_sit_bitmap_size = sit_bitmap_size;
/*
* It should be reserved minimum 1 segment for nat.
--
1.9.1
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [RFC PATCH v2] mkfs.f2fs: recalculate sit_segments by max_sit_bitmap_size
2016-02-23 3:46 [RFC PATCH v2] mkfs.f2fs: recalculate sit_segments by max_sit_bitmap_size Junling Zheng
@ 2016-02-23 5:28 ` Chao Yu
2016-02-23 7:19 ` Junling Zheng
0 siblings, 1 reply; 5+ messages in thread
From: Chao Yu @ 2016-02-23 5:28 UTC (permalink / raw)
To: 'Junling Zheng', jaegeuk; +Cc: linux-f2fs-devel
Hi all,
> -----Original Message-----
> From: Junling Zheng [mailto:zhengjunling@huawei.com]
> Sent: Tuesday, February 23, 2016 11:47 AM
> To: linux-f2fs-devel@lists.sourceforge.net; jaegeuk@kernel.org
> Subject: [f2fs-dev] [RFC PATCH v2] mkfs.f2fs: recalculate sit_segments by max_sit_bitmap_size
>
> In most cases, sit_bitmap_size is smaller than MAX_SIT_BITMAP_SIZE.
>
> However, in some extreme scenarios, such as 16TB, sit_bitmap_size
> could be larger than MAX_SIT_BITMAP_SIZE.
>
> In this case, we should recalculate the sit_segments through
> max_sit_bitmap_size to prevent sit_ver_bitmap_bytesize got from
> segment_count_sit in f2fs_write_check_point_pack() being over
> MAX_SIT_BITMAP_SIZE.
>
> Signed-off-by: Junling Zheng <zhengjunling@huawei.com>
> ---
> mkfs/f2fs_format.c | 23 ++++++++++++++++-------
> 1 file changed, 16 insertions(+), 7 deletions(-)
>
> diff --git a/mkfs/f2fs_format.c b/mkfs/f2fs_format.c
> index 645c2aa..3a050e0 100644
> --- a/mkfs/f2fs_format.c
> +++ b/mkfs/f2fs_format.c
> @@ -191,6 +191,22 @@ static int f2fs_prepare_super_block(void)
>
> sit_segments = SEG_ALIGN(blocks_for_sit);
>
> + /*
> + * In most cases, sit_bitmap_size is smaller than MAX_SIT_BITMAP_SIZE.
> + * However, in an extreme scenario(16TB), sit_bitmap_size could be larger
> + * than MAX_SIT_BITMAP_SIZE. Thus, we should recalculate the sit_segments
> + * to prevent sit_ver_bitmap_bytesize got from segment_count_sit in
> + * f2fs_write_check_point_pack() being over MAX_SIT_BITMAP_SIZE.
> + */
> + sit_bitmap_size = (sit_segments << log_blks_per_seg) / 8;
> +
> + if (sit_bitmap_size > MAX_SIT_BITMAP_SIZE) {
> + max_sit_bitmap_size = MAX_SIT_BITMAP_SIZE;
> + sit_segments = max_sit_bitmap_size * 8 >> log_blks_per_seg;
> + blocks_for_sit = sit_segments << log_blks_per_seg;
> + } else
Still the minor coding style problem.
IMO, maybe it would be better to limit config.total_sectors with 16TB at
very beginning, so more fields in sb like block_count, segment_count could
be set correctly when we try to format a huge size image. Right?
Thanks,
> + max_sit_bitmap_size = sit_bitmap_size;
> +
> set_sb(segment_count_sit, sit_segments * 2);
>
> set_sb(nat_blkaddr, get_sb(sit_blkaddr) + get_sb(segment_count_sit) *
> @@ -208,13 +224,6 @@ static int f2fs_prepare_super_block(void)
> * This number resizes NAT bitmap area in a CP page.
> * So the threshold is determined not to overflow one CP page
> */
> - sit_bitmap_size = ((get_sb(segment_count_sit) / 2) <<
> - log_blks_per_seg) / 8;
> -
> - if (sit_bitmap_size > MAX_SIT_BITMAP_SIZE)
> - max_sit_bitmap_size = MAX_SIT_BITMAP_SIZE;
> - else
> - max_sit_bitmap_size = sit_bitmap_size;
>
> /*
> * It should be reserved minimum 1 segment for nat.
> --
> 1.9.1
>
>
> ------------------------------------------------------------------------------
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
> _______________________________________________
> Linux-f2fs-devel mailing list
> Linux-f2fs-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [RFC PATCH v2] mkfs.f2fs: recalculate sit_segments by max_sit_bitmap_size
2016-02-23 5:28 ` Chao Yu
@ 2016-02-23 7:19 ` Junling Zheng
2016-02-23 9:46 ` Chao Yu
0 siblings, 1 reply; 5+ messages in thread
From: Junling Zheng @ 2016-02-23 7:19 UTC (permalink / raw)
To: Chao Yu, jaegeuk; +Cc: linux-f2fs-devel
On 2016/2/23 13:28, Chao Yu wrote:
> Hi all,
>
>> -----Original Message-----
>> From: Junling Zheng [mailto:zhengjunling@huawei.com]
>> Sent: Tuesday, February 23, 2016 11:47 AM
>> To: linux-f2fs-devel@lists.sourceforge.net; jaegeuk@kernel.org
>> Subject: [f2fs-dev] [RFC PATCH v2] mkfs.f2fs: recalculate sit_segments by max_sit_bitmap_size
>>
>> In most cases, sit_bitmap_size is smaller than MAX_SIT_BITMAP_SIZE.
>>
>> However, in some extreme scenarios, such as 16TB, sit_bitmap_size
>> could be larger than MAX_SIT_BITMAP_SIZE.
>>
>> In this case, we should recalculate the sit_segments through
>> max_sit_bitmap_size to prevent sit_ver_bitmap_bytesize got from
>> segment_count_sit in f2fs_write_check_point_pack() being over
>> MAX_SIT_BITMAP_SIZE.
>>
>> Signed-off-by: Junling Zheng <zhengjunling@huawei.com>
>> ---
>> mkfs/f2fs_format.c | 23 ++++++++++++++++-------
>> 1 file changed, 16 insertions(+), 7 deletions(-)
>>
>> diff --git a/mkfs/f2fs_format.c b/mkfs/f2fs_format.c
>> index 645c2aa..3a050e0 100644
>> --- a/mkfs/f2fs_format.c
>> +++ b/mkfs/f2fs_format.c
>> @@ -191,6 +191,22 @@ static int f2fs_prepare_super_block(void)
>>
>> sit_segments = SEG_ALIGN(blocks_for_sit);
>>
>> + /*
>> + * In most cases, sit_bitmap_size is smaller than MAX_SIT_BITMAP_SIZE.
>> + * However, in an extreme scenario(16TB), sit_bitmap_size could be larger
>> + * than MAX_SIT_BITMAP_SIZE. Thus, we should recalculate the sit_segments
>> + * to prevent sit_ver_bitmap_bytesize got from segment_count_sit in
>> + * f2fs_write_check_point_pack() being over MAX_SIT_BITMAP_SIZE.
>> + */
>> + sit_bitmap_size = (sit_segments << log_blks_per_seg) / 8;
>> +
>> + if (sit_bitmap_size > MAX_SIT_BITMAP_SIZE) {
>> + max_sit_bitmap_size = MAX_SIT_BITMAP_SIZE;
>> + sit_segments = max_sit_bitmap_size * 8 >> log_blks_per_seg;
>> + blocks_for_sit = sit_segments << log_blks_per_seg;
>> + } else
>
> Still the minor coding style problem.
>
Do you mean the "else" also needs {} even though it has only one sentence?
> IMO, maybe it would be better to limit config.total_sectors with 16TB at
> very beginning, so more fields in sb like block_count, segment_count could
> be set correctly when we try to format a huge size image. Right?
>
Limiting the disk at very beginning means not starting from 0 sector?
Does it amount to reducing the size of disk to keep sit_bitmap_size smaller
than MAX_SIT_BITMAP_SIZE? It looks to be helpful:)
Indeed, the data in sb have no problem. Just the difference of calculating
methods between sit_bitmap_size and MAX_SIT_BITMAP_SIZE:
F2FS_MAX_SEGMENT:
16 * 1024 * 1024 / 2 = 8388608
max blocks_for_sit:
ALIGN(F2FS_MAX_SEGMENT, SIT_ENTRY_PER_BLOCK) = 152521 // Upward here, not all blocks are used, some are redundant.
max sit_segments:
ALIGN(blocks_for_sit, config.blks_per_seg) = 298 // Upward here, not all segments are used, some are redundant.
max segment_count_sit:
sit_segments * 2 = 596
max sit_ver_bitmap_bytesize:
((segment_count_sit / 2) << log_blocks_per_seg) / 8 = 19072 // Here, bitmap size is too large because of the redundant blocks and segments.
However, MAX_SIT_BITMAP_SIZE is defined as:
((F2FS_MAX_SEGMENT / SIT_ENTRY_PER_BLOCK) / 8) = 19065 // Downward here, abandon the redundant segments.
So, the best way is to unify the two calculating method:)
Thanks,
> Thanks,
>
>> + max_sit_bitmap_size = sit_bitmap_size;
>> +
>> set_sb(segment_count_sit, sit_segments * 2);
>>
>> set_sb(nat_blkaddr, get_sb(sit_blkaddr) + get_sb(segment_count_sit) *
>> @@ -208,13 +224,6 @@ static int f2fs_prepare_super_block(void)
>> * This number resizes NAT bitmap area in a CP page.
>> * So the threshold is determined not to overflow one CP page
>> */
>> - sit_bitmap_size = ((get_sb(segment_count_sit) / 2) <<
>> - log_blks_per_seg) / 8;
>> -
>> - if (sit_bitmap_size > MAX_SIT_BITMAP_SIZE)
>> - max_sit_bitmap_size = MAX_SIT_BITMAP_SIZE;
>> - else
>> - max_sit_bitmap_size = sit_bitmap_size;
>>
>> /*
>> * It should be reserved minimum 1 segment for nat.
>> --
>> 1.9.1
>>
>>
>> ------------------------------------------------------------------------------
>> Site24x7 APM Insight: Get Deep Visibility into Application Performance
>> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
>> Monitor end-to-end web transactions and take corrective actions now
>> Troubleshoot faster and improve end-user experience. Signup Now!
>> http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
>> _______________________________________________
>> Linux-f2fs-devel mailing list
>> Linux-f2fs-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
>
>
> .
>
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [RFC PATCH v2] mkfs.f2fs: recalculate sit_segments by max_sit_bitmap_size
2016-02-23 7:19 ` Junling Zheng
@ 2016-02-23 9:46 ` Chao Yu
2016-02-23 12:04 ` Junling Zheng
0 siblings, 1 reply; 5+ messages in thread
From: Chao Yu @ 2016-02-23 9:46 UTC (permalink / raw)
To: 'Junling Zheng', jaegeuk; +Cc: linux-f2fs-devel
Hi Junling,
> -----Original Message-----
> From: Junling Zheng [mailto:zhengjunling@huawei.com]
> Sent: Tuesday, February 23, 2016 3:20 PM
> To: Chao Yu; jaegeuk@kernel.org
> Cc: linux-f2fs-devel@lists.sourceforge.net
> Subject: Re: [f2fs-dev] [RFC PATCH v2] mkfs.f2fs: recalculate sit_segments by
> max_sit_bitmap_size
>
> On 2016/2/23 13:28, Chao Yu wrote:
> > Hi all,
> >
> >> -----Original Message-----
> >> From: Junling Zheng [mailto:zhengjunling@huawei.com]
> >> Sent: Tuesday, February 23, 2016 11:47 AM
> >> To: linux-f2fs-devel@lists.sourceforge.net; jaegeuk@kernel.org
> >> Subject: [f2fs-dev] [RFC PATCH v2] mkfs.f2fs: recalculate sit_segments by max_sit_bitmap_size
> >>
> >> In most cases, sit_bitmap_size is smaller than MAX_SIT_BITMAP_SIZE.
> >>
> >> However, in some extreme scenarios, such as 16TB, sit_bitmap_size
> >> could be larger than MAX_SIT_BITMAP_SIZE.
> >>
> >> In this case, we should recalculate the sit_segments through
> >> max_sit_bitmap_size to prevent sit_ver_bitmap_bytesize got from
> >> segment_count_sit in f2fs_write_check_point_pack() being over
> >> MAX_SIT_BITMAP_SIZE.
> >>
> >> Signed-off-by: Junling Zheng <zhengjunling@huawei.com>
> >> ---
> >> mkfs/f2fs_format.c | 23 ++++++++++++++++-------
> >> 1 file changed, 16 insertions(+), 7 deletions(-)
> >>
> >> diff --git a/mkfs/f2fs_format.c b/mkfs/f2fs_format.c
> >> index 645c2aa..3a050e0 100644
> >> --- a/mkfs/f2fs_format.c
> >> +++ b/mkfs/f2fs_format.c
> >> @@ -191,6 +191,22 @@ static int f2fs_prepare_super_block(void)
> >>
> >> sit_segments = SEG_ALIGN(blocks_for_sit);
> >>
> >> + /*
> >> + * In most cases, sit_bitmap_size is smaller than MAX_SIT_BITMAP_SIZE.
> >> + * However, in an extreme scenario(16TB), sit_bitmap_size could be larger
> >> + * than MAX_SIT_BITMAP_SIZE. Thus, we should recalculate the sit_segments
> >> + * to prevent sit_ver_bitmap_bytesize got from segment_count_sit in
> >> + * f2fs_write_check_point_pack() being over MAX_SIT_BITMAP_SIZE.
> >> + */
> >> + sit_bitmap_size = (sit_segments << log_blks_per_seg) / 8;
> >> +
> >> + if (sit_bitmap_size > MAX_SIT_BITMAP_SIZE) {
> >> + max_sit_bitmap_size = MAX_SIT_BITMAP_SIZE;
> >> + sit_segments = max_sit_bitmap_size * 8 >> log_blks_per_seg;
> >> + blocks_for_sit = sit_segments << log_blks_per_seg;
> >> + } else
> >
> > Still the minor coding style problem.
> >
>
> Do you mean the "else" also needs {} even though it has only one sentence?
That's right.
>
> > IMO, maybe it would be better to limit config.total_sectors with 16TB at
> > very beginning, so more fields in sb like block_count, segment_count could
> > be set correctly when we try to format a huge size image. Right?
> >
>
> Limiting the disk at very beginning means not starting from 0 sector?
Oh, what I mean here is 'at very beginning time', f2fs can support a
volume with size of 16 TB at most, so I'm thinking we'd better do the
limitation for config.total_sectors, then most parameters in cp/sb
calculated according to value of total_sectors could be limited or
corrected too.
Anyway, that is a separate issue. :)
> Does it amount to reducing the size of disk to keep sit_bitmap_size smaller
> than MAX_SIT_BITMAP_SIZE? It looks to be helpful:)
>
> Indeed, the data in sb have no problem. Just the difference of calculating
> methods between sit_bitmap_size and MAX_SIT_BITMAP_SIZE:
>
> F2FS_MAX_SEGMENT:
> 16 * 1024 * 1024 / 2 = 8388608
> max blocks_for_sit:
> ALIGN(F2FS_MAX_SEGMENT, SIT_ENTRY_PER_BLOCK) = 152521 // Upward here, not all blocks are
> used, some are redundant.
> max sit_segments:
> ALIGN(blocks_for_sit, config.blks_per_seg) = 298 // Upward here, not all segments are
> used, some are redundant.
> max segment_count_sit:
> sit_segments * 2 = 596
> max sit_ver_bitmap_bytesize:
> ((segment_count_sit / 2) << log_blocks_per_seg) / 8 = 19072 // Here, bitmap size is too
> large because of the redundant blocks and segments.
>
> However, MAX_SIT_BITMAP_SIZE is defined as:
> ((F2FS_MAX_SEGMENT / SIT_ENTRY_PER_BLOCK) / 8) = 19065 // Downward here, abandon the
> redundant segments.
>
> So, the best way is to unify the two calculating method:)
Agreed. :)
Thanks,
>
> Thanks,
>
> > Thanks,
> >
> >> + max_sit_bitmap_size = sit_bitmap_size;
> >> +
> >> set_sb(segment_count_sit, sit_segments * 2);
> >>
> >> set_sb(nat_blkaddr, get_sb(sit_blkaddr) + get_sb(segment_count_sit) *
> >> @@ -208,13 +224,6 @@ static int f2fs_prepare_super_block(void)
> >> * This number resizes NAT bitmap area in a CP page.
> >> * So the threshold is determined not to overflow one CP page
> >> */
> >> - sit_bitmap_size = ((get_sb(segment_count_sit) / 2) <<
> >> - log_blks_per_seg) / 8;
> >> -
> >> - if (sit_bitmap_size > MAX_SIT_BITMAP_SIZE)
> >> - max_sit_bitmap_size = MAX_SIT_BITMAP_SIZE;
> >> - else
> >> - max_sit_bitmap_size = sit_bitmap_size;
> >>
> >> /*
> >> * It should be reserved minimum 1 segment for nat.
> >> --
> >> 1.9.1
> >>
> >>
> >> ------------------------------------------------------------------------------
> >> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> >> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> >> Monitor end-to-end web transactions and take corrective actions now
> >> Troubleshoot faster and improve end-user experience. Signup Now!
> >> http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
> >> _______________________________________________
> >> Linux-f2fs-devel mailing list
> >> Linux-f2fs-devel@lists.sourceforge.net
> >> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
> >
> >
> > .
> >
>
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [RFC PATCH v2] mkfs.f2fs: recalculate sit_segments by max_sit_bitmap_size
2016-02-23 9:46 ` Chao Yu
@ 2016-02-23 12:04 ` Junling Zheng
0 siblings, 0 replies; 5+ messages in thread
From: Junling Zheng @ 2016-02-23 12:04 UTC (permalink / raw)
To: Chao Yu, jaegeuk; +Cc: linux-f2fs-devel
Hi Chao,
On 2016/2/23 17:46, Chao Yu wrote:
> Hi Junling,
>
>> -----Original Message-----
>> From: Junling Zheng [mailto:zhengjunling@huawei.com]
>> Sent: Tuesday, February 23, 2016 3:20 PM
>> To: Chao Yu; jaegeuk@kernel.org
>> Cc: linux-f2fs-devel@lists.sourceforge.net
>> Subject: Re: [f2fs-dev] [RFC PATCH v2] mkfs.f2fs: recalculate sit_segments by
>> max_sit_bitmap_size
>>
>> On 2016/2/23 13:28, Chao Yu wrote:
>>> Hi all,
>>>
>>>> -----Original Message-----
>>>> From: Junling Zheng [mailto:zhengjunling@huawei.com]
>>>> Sent: Tuesday, February 23, 2016 11:47 AM
>>>> To: linux-f2fs-devel@lists.sourceforge.net; jaegeuk@kernel.org
>>>> Subject: [f2fs-dev] [RFC PATCH v2] mkfs.f2fs: recalculate sit_segments by max_sit_bitmap_size
>>>>
>>>> In most cases, sit_bitmap_size is smaller than MAX_SIT_BITMAP_SIZE.
>>>>
>>>> However, in some extreme scenarios, such as 16TB, sit_bitmap_size
>>>> could be larger than MAX_SIT_BITMAP_SIZE.
>>>>
>>>> In this case, we should recalculate the sit_segments through
>>>> max_sit_bitmap_size to prevent sit_ver_bitmap_bytesize got from
>>>> segment_count_sit in f2fs_write_check_point_pack() being over
>>>> MAX_SIT_BITMAP_SIZE.
>>>>
>>>> Signed-off-by: Junling Zheng <zhengjunling@huawei.com>
>>>> ---
>>>> mkfs/f2fs_format.c | 23 ++++++++++++++++-------
>>>> 1 file changed, 16 insertions(+), 7 deletions(-)
>>>>
>>>> diff --git a/mkfs/f2fs_format.c b/mkfs/f2fs_format.c
>>>> index 645c2aa..3a050e0 100644
>>>> --- a/mkfs/f2fs_format.c
>>>> +++ b/mkfs/f2fs_format.c
>>>> @@ -191,6 +191,22 @@ static int f2fs_prepare_super_block(void)
>>>>
>>>> sit_segments = SEG_ALIGN(blocks_for_sit);
>>>>
>>>> + /*
>>>> + * In most cases, sit_bitmap_size is smaller than MAX_SIT_BITMAP_SIZE.
>>>> + * However, in an extreme scenario(16TB), sit_bitmap_size could be larger
>>>> + * than MAX_SIT_BITMAP_SIZE. Thus, we should recalculate the sit_segments
>>>> + * to prevent sit_ver_bitmap_bytesize got from segment_count_sit in
>>>> + * f2fs_write_check_point_pack() being over MAX_SIT_BITMAP_SIZE.
>>>> + */
>>>> + sit_bitmap_size = (sit_segments << log_blks_per_seg) / 8;
>>>> +
>>>> + if (sit_bitmap_size > MAX_SIT_BITMAP_SIZE) {
>>>> + max_sit_bitmap_size = MAX_SIT_BITMAP_SIZE;
>>>> + sit_segments = max_sit_bitmap_size * 8 >> log_blks_per_seg;
>>>> + blocks_for_sit = sit_segments << log_blks_per_seg;
>>>> + } else
>>>
>>> Still the minor coding style problem.
>>>
>>
>> Do you mean the "else" also needs {} even though it has only one sentence?
>
> That's right.
>
>>
>>> IMO, maybe it would be better to limit config.total_sectors with 16TB at
>>> very beginning, so more fields in sb like block_count, segment_count could
>>> be set correctly when we try to format a huge size image. Right?
>>>
>>
>> Limiting the disk at very beginning means not starting from 0 sector?
>
> Oh, what I mean here is 'at very beginning time', f2fs can support a
> volume with size of 16 TB at most, so I'm thinking we'd better do the
> limitation for config.total_sectors, then most parameters in cp/sb
> calculated according to value of total_sectors could be limited or
> corrected too.
>
> Anyway, that is a separate issue. :)
>
Good idea! I'll send another patch later:)
>> Does it amount to reducing the size of disk to keep sit_bitmap_size smaller
>> than MAX_SIT_BITMAP_SIZE? It looks to be helpful:)
>>
>> Indeed, the data in sb have no problem. Just the difference of calculating
>> methods between sit_bitmap_size and MAX_SIT_BITMAP_SIZE:
>>
>> F2FS_MAX_SEGMENT:
>> 16 * 1024 * 1024 / 2 = 8388608
>> max blocks_for_sit:
>> ALIGN(F2FS_MAX_SEGMENT, SIT_ENTRY_PER_BLOCK) = 152521 // Upward here, not all blocks are
>> used, some are redundant.
>> max sit_segments:
>> ALIGN(blocks_for_sit, config.blks_per_seg) = 298 // Upward here, not all segments are
>> used, some are redundant.
>> max segment_count_sit:
>> sit_segments * 2 = 596
>> max sit_ver_bitmap_bytesize:
>> ((segment_count_sit / 2) << log_blocks_per_seg) / 8 = 19072 // Here, bitmap size is too
>> large because of the redundant blocks and segments.
>>
>> However, MAX_SIT_BITMAP_SIZE is defined as:
>> ((F2FS_MAX_SEGMENT / SIT_ENTRY_PER_BLOCK) / 8) = 19065 // Downward here, abandon the
>> redundant segments.
>>
>> So, the best way is to unify the two calculating method:)
>
> Agreed. :)
>
So I'll resend a fix by redefining MAX_SIT_BITMAP_SIZE as the following instead of
recalculating segment_count_sit:)
SEG_ALIGN(ALIGN(F2FS_MAX_SEGMENT, SIT_ENTRY_PER_BLOCK)) * config.blks_per_seg / 8
Thanks,
> Thanks,
>
>>
>> Thanks,
>>
>>> Thanks,
>>>
>>>> + max_sit_bitmap_size = sit_bitmap_size;
>>>> +
>>>> set_sb(segment_count_sit, sit_segments * 2);
>>>>
>>>> set_sb(nat_blkaddr, get_sb(sit_blkaddr) + get_sb(segment_count_sit) *
>>>> @@ -208,13 +224,6 @@ static int f2fs_prepare_super_block(void)
>>>> * This number resizes NAT bitmap area in a CP page.
>>>> * So the threshold is determined not to overflow one CP page
>>>> */
>>>> - sit_bitmap_size = ((get_sb(segment_count_sit) / 2) <<
>>>> - log_blks_per_seg) / 8;
>>>> -
>>>> - if (sit_bitmap_size > MAX_SIT_BITMAP_SIZE)
>>>> - max_sit_bitmap_size = MAX_SIT_BITMAP_SIZE;
>>>> - else
>>>> - max_sit_bitmap_size = sit_bitmap_size;
>>>>
>>>> /*
>>>> * It should be reserved minimum 1 segment for nat.
>>>> --
>>>> 1.9.1
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Site24x7 APM Insight: Get Deep Visibility into Application Performance
>>>> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
>>>> Monitor end-to-end web transactions and take corrective actions now
>>>> Troubleshoot faster and improve end-user experience. Signup Now!
>>>> http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
>>>> _______________________________________________
>>>> Linux-f2fs-devel mailing list
>>>> Linux-f2fs-devel@lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
>>>
>>>
>>> .
>>>
>>
>
>
>
> .
>
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2016-02-23 12:04 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-02-23 3:46 [RFC PATCH v2] mkfs.f2fs: recalculate sit_segments by max_sit_bitmap_size Junling Zheng
2016-02-23 5:28 ` Chao Yu
2016-02-23 7:19 ` Junling Zheng
2016-02-23 9:46 ` Chao Yu
2016-02-23 12:04 ` Junling Zheng
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).