* [PATCH] mkfs.f2fs: recalculate sit_segments by max_sit_bitmap_size @ 2016-02-18 8:59 Junling Zheng 2016-02-22 17:31 ` Jaegeuk Kim 0 siblings, 1 reply; 3+ messages in thread From: Junling Zheng @ 2016-02-18 8:59 UTC (permalink / raw) To: linux-f2fs-devel, jaegeuk In most cases, sit_bitmap_size is smaller than MAX_SIT_BITMAP_SIZE. However, in some extreme scenarios, such as 16TB, sit_bitmap_size could be larger than MAX_SIT_BITMAP_SIZE. In this case, we should recalculate the sit_segments through max_sit_bitmap_size to prevent sit_ver_bitmap_bytesize got from segment_count_sit in f2fs_write_check_point_pack() being over MAX_SIT_BITMAP_SIZE. Signed-off-by: Junling Zheng <zhengjunling@huawei.com> --- mkfs/f2fs_format.c | 24 +++++++++++++++++------- 1 file changed, 17 insertions(+), 7 deletions(-) diff --git a/mkfs/f2fs_format.c b/mkfs/f2fs_format.c index 645c2aa..24b2cc1 100644 --- a/mkfs/f2fs_format.c +++ b/mkfs/f2fs_format.c @@ -191,6 +191,23 @@ static int f2fs_prepare_super_block(void) sit_segments = SEG_ALIGN(blocks_for_sit); + /* + * In most cases, sit_bitmap_size is smaller than MAX_SIT_BITMAP_SIZE. + * However, in an extreme scenario(16TB), sit_bitmap_size could be larger + * than MAX_SIT_BITMAP_SIZE. Thus, we should recalculate the sit_segments + * to prevent sit_ver_bitmap_bytesize got from segment_count_sit in + * f2fs_write_check_point_pack() being over MAX_SIT_BITMAP_SIZE. + */ + sit_bitmap_size = (sit_segments << log_blks_per_seg) / 8; + + if (sit_bitmap_size > MAX_SIT_BITMAP_SIZE) { + max_sit_bitmap_size = MAX_SIT_BITMAP_SIZE; + sit_segments = max_sit_bitmap_size * 8 >> log_blks_per_seg; + blocks_for_sit = sit_segments << log_blks_per_seg; + } + else + max_sit_bitmap_size = sit_bitmap_size; + set_sb(segment_count_sit, sit_segments * 2); set_sb(nat_blkaddr, get_sb(sit_blkaddr) + get_sb(segment_count_sit) * @@ -208,13 +225,6 @@ static int f2fs_prepare_super_block(void) * This number resizes NAT bitmap area in a CP page. * So the threshold is determined not to overflow one CP page */ - sit_bitmap_size = ((get_sb(segment_count_sit) / 2) << - log_blks_per_seg) / 8; - - if (sit_bitmap_size > MAX_SIT_BITMAP_SIZE) - max_sit_bitmap_size = MAX_SIT_BITMAP_SIZE; - else - max_sit_bitmap_size = sit_bitmap_size; /* * It should be reserved minimum 1 segment for nat. -- 1.9.1 ------------------------------------------------------------------------------ Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140 ^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH] mkfs.f2fs: recalculate sit_segments by max_sit_bitmap_size 2016-02-18 8:59 [PATCH] mkfs.f2fs: recalculate sit_segments by max_sit_bitmap_size Junling Zheng @ 2016-02-22 17:31 ` Jaegeuk Kim 2016-02-23 3:33 ` Junling Zheng 0 siblings, 1 reply; 3+ messages in thread From: Jaegeuk Kim @ 2016-02-22 17:31 UTC (permalink / raw) To: Junling Zheng; +Cc: linux-f2fs-devel Hi Junling, On Thu, Feb 18, 2016 at 04:59:04PM +0800, Junling Zheng wrote: > In most cases, sit_bitmap_size is smaller than MAX_SIT_BITMAP_SIZE. > > However, in some extreme scenarios, such as 16TB, sit_bitmap_size > could be larger than MAX_SIT_BITMAP_SIZE. > > In this case, we should recalculate the sit_segments through > max_sit_bitmap_size to prevent sit_ver_bitmap_bytesize got from > segment_count_sit in f2fs_write_check_point_pack() being over > MAX_SIT_BITMAP_SIZE. > > Signed-off-by: Junling Zheng <zhengjunling@huawei.com> > --- > mkfs/f2fs_format.c | 24 +++++++++++++++++------- > 1 file changed, 17 insertions(+), 7 deletions(-) > > diff --git a/mkfs/f2fs_format.c b/mkfs/f2fs_format.c > index 645c2aa..24b2cc1 100644 > --- a/mkfs/f2fs_format.c > +++ b/mkfs/f2fs_format.c > @@ -191,6 +191,23 @@ static int f2fs_prepare_super_block(void) > > sit_segments = SEG_ALIGN(blocks_for_sit); > > + /* > + * In most cases, sit_bitmap_size is smaller than MAX_SIT_BITMAP_SIZE. > + * However, in an extreme scenario(16TB), sit_bitmap_size could be larger > + * than MAX_SIT_BITMAP_SIZE. Thus, we should recalculate the sit_segments > + * to prevent sit_ver_bitmap_bytesize got from segment_count_sit in > + * f2fs_write_check_point_pack() being over MAX_SIT_BITMAP_SIZE. > + */ > + sit_bitmap_size = (sit_segments << log_blks_per_seg) / 8; > + > + if (sit_bitmap_size > MAX_SIT_BITMAP_SIZE) { > + max_sit_bitmap_size = MAX_SIT_BITMAP_SIZE; > + sit_segments = max_sit_bitmap_size * 8 >> log_blks_per_seg; > + blocks_for_sit = sit_segments << log_blks_per_seg; > + } > + else > + max_sit_bitmap_size = sit_bitmap_size; Codling style. if { } else { } Anyway, what about just redefining MAX_SIT_BITMAP_SIZE as (ALIGN(F2FS_MAX_SEGMENT, SIT_ENTRY_PER_BLOCK) / 8) ? Thanks, > + > set_sb(segment_count_sit, sit_segments * 2); > > set_sb(nat_blkaddr, get_sb(sit_blkaddr) + get_sb(segment_count_sit) * > @@ -208,13 +225,6 @@ static int f2fs_prepare_super_block(void) > * This number resizes NAT bitmap area in a CP page. > * So the threshold is determined not to overflow one CP page > */ > - sit_bitmap_size = ((get_sb(segment_count_sit) / 2) << > - log_blks_per_seg) / 8; > - > - if (sit_bitmap_size > MAX_SIT_BITMAP_SIZE) > - max_sit_bitmap_size = MAX_SIT_BITMAP_SIZE; > - else > - max_sit_bitmap_size = sit_bitmap_size; > > /* > * It should be reserved minimum 1 segment for nat. > -- > 1.9.1 > > > ------------------------------------------------------------------------------ > Site24x7 APM Insight: Get Deep Visibility into Application Performance > APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month > Monitor end-to-end web transactions and take corrective actions now > Troubleshoot faster and improve end-user experience. Signup Now! > http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140 > _______________________________________________ > Linux-f2fs-devel mailing list > Linux-f2fs-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel ------------------------------------------------------------------------------ Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140 ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH] mkfs.f2fs: recalculate sit_segments by max_sit_bitmap_size 2016-02-22 17:31 ` Jaegeuk Kim @ 2016-02-23 3:33 ` Junling Zheng 0 siblings, 0 replies; 3+ messages in thread From: Junling Zheng @ 2016-02-23 3:33 UTC (permalink / raw) To: Jaegeuk Kim; +Cc: linux-f2fs-devel On 2016/2/23 1:31, Jaegeuk Kim wrote: > Hi Junling, > > On Thu, Feb 18, 2016 at 04:59:04PM +0800, Junling Zheng wrote: >> In most cases, sit_bitmap_size is smaller than MAX_SIT_BITMAP_SIZE. >> >> However, in some extreme scenarios, such as 16TB, sit_bitmap_size >> could be larger than MAX_SIT_BITMAP_SIZE. >> >> In this case, we should recalculate the sit_segments through >> max_sit_bitmap_size to prevent sit_ver_bitmap_bytesize got from >> segment_count_sit in f2fs_write_check_point_pack() being over >> MAX_SIT_BITMAP_SIZE. >> >> Signed-off-by: Junling Zheng <zhengjunling@huawei.com> >> --- >> mkfs/f2fs_format.c | 24 +++++++++++++++++------- >> 1 file changed, 17 insertions(+), 7 deletions(-) >> >> diff --git a/mkfs/f2fs_format.c b/mkfs/f2fs_format.c >> index 645c2aa..24b2cc1 100644 >> --- a/mkfs/f2fs_format.c >> +++ b/mkfs/f2fs_format.c >> @@ -191,6 +191,23 @@ static int f2fs_prepare_super_block(void) >> >> sit_segments = SEG_ALIGN(blocks_for_sit); >> >> + /* >> + * In most cases, sit_bitmap_size is smaller than MAX_SIT_BITMAP_SIZE. >> + * However, in an extreme scenario(16TB), sit_bitmap_size could be larger >> + * than MAX_SIT_BITMAP_SIZE. Thus, we should recalculate the sit_segments >> + * to prevent sit_ver_bitmap_bytesize got from segment_count_sit in >> + * f2fs_write_check_point_pack() being over MAX_SIT_BITMAP_SIZE. >> + */ >> + sit_bitmap_size = (sit_segments << log_blks_per_seg) / 8; >> + >> + if (sit_bitmap_size > MAX_SIT_BITMAP_SIZE) { >> + max_sit_bitmap_size = MAX_SIT_BITMAP_SIZE; >> + sit_segments = max_sit_bitmap_size * 8 >> log_blks_per_seg; >> + blocks_for_sit = sit_segments << log_blks_per_seg; >> + } >> + else >> + max_sit_bitmap_size = sit_bitmap_size; > > Codling style. > if { > } else { > } > > Anyway, what about just redefining MAX_SIT_BITMAP_SIZE as > (ALIGN(F2FS_MAX_SEGMENT, SIT_ENTRY_PER_BLOCK) / 8) ? > (ALIGN(F2FS_MAX_SEGMENT, SIT_ENTRY_PER_BLOCK) / 8) is also 19065, is the same with the original. But the sit_ver_bitmap_bytesize in cp could be 19072. Please see the following calculations: F2FS_MAX_SEGMENT: 16 * 1024 * 1024 / 2 = 8388608 max blocks_for_sit: ALIGN(F2FS_MAX_SEGMENT, SIT_ENTRY_PER_BLOCK) = 152521 // Here, not all blocks are used, some are redundant. max sit_segments: ALIGN(blocks_for_sit, config.blks_per_seg) = 298 // Here, not all segments are used, some are redundant. max segment_count_sit: sit_segments * 2 = 596 max sit_ver_bitmap_bytesize: ((segment_count_sit / 2) << log_blocks_per_seg) / 8 = 19072 // Here, bitmap size is too large because of the redundant blocks and segments. At the beginning, I just doubted why segment_count_sit wasn't recalculated by max_sit_bitmap_size. And then, I found that the sit_ver_bitmap_bytesize got from segment_count_sit might exceed MAX_SIT_BITMAP_SIZE. So, if segment_count_sit is not recalculated by max_sit_bitmap_size, checking and limiting sit_ver_bitmap_bytesize in cp during f2fs_write_check_point_pack() is necessary. > Thanks, > >> + >> set_sb(segment_count_sit, sit_segments * 2); >> >> set_sb(nat_blkaddr, get_sb(sit_blkaddr) + get_sb(segment_count_sit) * >> @@ -208,13 +225,6 @@ static int f2fs_prepare_super_block(void) >> * This number resizes NAT bitmap area in a CP page. >> * So the threshold is determined not to overflow one CP page >> */ >> - sit_bitmap_size = ((get_sb(segment_count_sit) / 2) << >> - log_blks_per_seg) / 8; >> - >> - if (sit_bitmap_size > MAX_SIT_BITMAP_SIZE) >> - max_sit_bitmap_size = MAX_SIT_BITMAP_SIZE; >> - else >> - max_sit_bitmap_size = sit_bitmap_size; >> >> /* >> * It should be reserved minimum 1 segment for nat. >> -- >> 1.9.1 >> >> >> ------------------------------------------------------------------------------ >> Site24x7 APM Insight: Get Deep Visibility into Application Performance >> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month >> Monitor end-to-end web transactions and take corrective actions now >> Troubleshoot faster and improve end-user experience. Signup Now! >> http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140 >> _______________________________________________ >> Linux-f2fs-devel mailing list >> Linux-f2fs-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel > > . > ------------------------------------------------------------------------------ Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140 ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2016-02-23 3:33 UTC | newest] Thread overview: 3+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2016-02-18 8:59 [PATCH] mkfs.f2fs: recalculate sit_segments by max_sit_bitmap_size Junling Zheng 2016-02-22 17:31 ` Jaegeuk Kim 2016-02-23 3:33 ` Junling Zheng
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).