* [f2fs-dev] [PATCH] f2fs: modify f2fs_is_checkpoint_ready logic to allow more data to be written with the CP disable @ 2024-10-09 10:27 ` Qi Han 0 siblings, 0 replies; 16+ messages in thread From: Qi Han via Linux-f2fs-devel @ 2024-10-09 10:27 UTC (permalink / raw) To: jaegeuk, chao; +Cc: Qi Han, linux-kernel, linux-f2fs-devel When the free segment is used up during CP disable, many write or ioctl operations will get ENOSPC error codes, even if there are still many blocks available. We can reproduce it in the following steps: dd if=/dev/zero of=f2fs.img bs=1M count=55 mkfs.f2fs -f f2fs.img mount f2fs.img f2fs_dir -o checkpoint=disable:10% cd f2fs_dir dd if=/dev/zero of=bigfile bs=1M count=50 sync rm bigfile i=1; while [[ $i -lt 10000000 ]]; do (file_name=./file$i; dd \ if=/dev/random of=$file_name bs=1M count=0); i=$((i+1)); done stat -f ./ In f2fs_need_SSR() function, it is allowed to use SSR to allocate blocks when CP is disabled, so in f2fs_is_checkpoint_ready function, can we judge the number of invalid blocks when free segment is not enough, and return ENOSPC only if the number of invalid blocks is also not enough? Signed-off-by: Qi Han <hanqi@vivo.com> --- fs/f2fs/segment.h | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/fs/f2fs/segment.h b/fs/f2fs/segment.h index 71adb4a43bec..9bf0cf3a6a31 100644 --- a/fs/f2fs/segment.h +++ b/fs/f2fs/segment.h @@ -637,12 +637,33 @@ static inline bool has_enough_free_secs(struct f2fs_sb_info *sbi, return !has_not_enough_free_secs(sbi, freed, needed); } +static inline bool has_enough_available_blocks(struct f2fs_sb_info *sbi) +{ + unsigned int total_free_blocks = sbi->user_block_count - + valid_user_blocks(sbi) - + sbi->current_reserved_blocks; + + if (total_free_blocks <= sbi->unusable_block_count) + total_free_blocks = 0; + else + total_free_blocks -= sbi->unusable_block_count; + + if (total_free_blocks > F2FS_OPTION(sbi).root_reserved_blocks) + total_free_blocks -= F2FS_OPTION(sbi).root_reserved_blocks; + else + total_free_blocks = 0; + + return (total_free_blocks > 0) ? true : false; +} + static inline bool f2fs_is_checkpoint_ready(struct f2fs_sb_info *sbi) { if (likely(!is_sbi_flag_set(sbi, SBI_CP_DISABLED))) return true; if (likely(has_enough_free_secs(sbi, 0, 0))) return true; + if (likely(has_enough_available_blocks(sbi))) + return true; return false; } -- 2.39.0 _______________________________________________ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel ^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH] f2fs: modify f2fs_is_checkpoint_ready logic to allow more data to be written with the CP disable @ 2024-10-09 10:27 ` Qi Han 0 siblings, 0 replies; 16+ messages in thread From: Qi Han @ 2024-10-09 10:27 UTC (permalink / raw) To: jaegeuk, chao; +Cc: linux-f2fs-devel, linux-kernel, Qi Han When the free segment is used up during CP disable, many write or ioctl operations will get ENOSPC error codes, even if there are still many blocks available. We can reproduce it in the following steps: dd if=/dev/zero of=f2fs.img bs=1M count=55 mkfs.f2fs -f f2fs.img mount f2fs.img f2fs_dir -o checkpoint=disable:10% cd f2fs_dir dd if=/dev/zero of=bigfile bs=1M count=50 sync rm bigfile i=1; while [[ $i -lt 10000000 ]]; do (file_name=./file$i; dd \ if=/dev/random of=$file_name bs=1M count=0); i=$((i+1)); done stat -f ./ In f2fs_need_SSR() function, it is allowed to use SSR to allocate blocks when CP is disabled, so in f2fs_is_checkpoint_ready function, can we judge the number of invalid blocks when free segment is not enough, and return ENOSPC only if the number of invalid blocks is also not enough? Signed-off-by: Qi Han <hanqi@vivo.com> --- fs/f2fs/segment.h | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/fs/f2fs/segment.h b/fs/f2fs/segment.h index 71adb4a43bec..9bf0cf3a6a31 100644 --- a/fs/f2fs/segment.h +++ b/fs/f2fs/segment.h @@ -637,12 +637,33 @@ static inline bool has_enough_free_secs(struct f2fs_sb_info *sbi, return !has_not_enough_free_secs(sbi, freed, needed); } +static inline bool has_enough_available_blocks(struct f2fs_sb_info *sbi) +{ + unsigned int total_free_blocks = sbi->user_block_count - + valid_user_blocks(sbi) - + sbi->current_reserved_blocks; + + if (total_free_blocks <= sbi->unusable_block_count) + total_free_blocks = 0; + else + total_free_blocks -= sbi->unusable_block_count; + + if (total_free_blocks > F2FS_OPTION(sbi).root_reserved_blocks) + total_free_blocks -= F2FS_OPTION(sbi).root_reserved_blocks; + else + total_free_blocks = 0; + + return (total_free_blocks > 0) ? true : false; +} + static inline bool f2fs_is_checkpoint_ready(struct f2fs_sb_info *sbi) { if (likely(!is_sbi_flag_set(sbi, SBI_CP_DISABLED))) return true; if (likely(has_enough_free_secs(sbi, 0, 0))) return true; + if (likely(has_enough_available_blocks(sbi))) + return true; return false; } -- 2.39.0 ^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: [f2fs-dev] [PATCH] f2fs: modify f2fs_is_checkpoint_ready logic to allow more data to be written with the CP disable 2024-10-09 10:27 ` Qi Han @ 2024-10-17 9:54 ` Chao Yu -1 siblings, 0 replies; 16+ messages in thread From: Chao Yu via Linux-f2fs-devel @ 2024-10-17 9:54 UTC (permalink / raw) To: Qi Han, jaegeuk; +Cc: linux-kernel, linux-f2fs-devel On 2024/10/9 18:27, Qi Han wrote: > When the free segment is used up during CP disable, many write or > ioctl operations will get ENOSPC error codes, even if there are > still many blocks available. We can reproduce it in the following > steps: > > dd if=/dev/zero of=f2fs.img bs=1M count=55 > mkfs.f2fs -f f2fs.img > mount f2fs.img f2fs_dir -o checkpoint=disable:10% > cd f2fs_dir > dd if=/dev/zero of=bigfile bs=1M count=50 > sync > rm bigfile > i=1; while [[ $i -lt 10000000 ]]; do (file_name=./file$i; dd \ > if=/dev/random of=$file_name bs=1M count=0); i=$((i+1)); done > stat -f ./ > > In f2fs_need_SSR() function, it is allowed to use SSR to allocate > blocks when CP is disabled, so in f2fs_is_checkpoint_ready function, > can we judge the number of invalid blocks when free segment is not > enough, and return ENOSPC only if the number of invalid blocks is > also not enough? Can you please try below patch? From 38b7c97dcc55ba83de4220c3dc54c2eb66148dd5 Mon Sep 17 00:00:00 2001 From: Chao Yu <chao@kernel.org> Date: Thu, 17 Oct 2024 17:07:05 +0800 Subject: [PATCH] f2fs: revalidate empty segment when checkpoint is disabled If checkpoint is off, let's set segment as free once all newly written datas were removed. Signed-off-by: Chao Yu <chao@kernel.org> --- fs/f2fs/segment.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c index f8d6e601e084..9bad13d70afb 100644 --- a/fs/f2fs/segment.c +++ b/fs/f2fs/segment.c @@ -853,6 +853,17 @@ static void locate_dirty_segment(struct f2fs_sb_info *sbi, unsigned int segno) valid_blocks = get_valid_blocks(sbi, segno, false); ckpt_valid_blocks = get_ckpt_valid_blocks(sbi, segno, false); + /* + * If checkpoint is off, let's set segment as free once all newly + * written datas were removed. + */ + if (is_sbi_flag_set(sbi, SBI_CP_DISABLED) && + valid_blocks == 0 && ckpt_valid_blocks == 0) { + __remove_dirty_segment(sbi, segno, DIRTY); + __set_test_and_free(sbi, segno, false); + goto out_lock; + } + if (valid_blocks == 0 && (!is_sbi_flag_set(sbi, SBI_CP_DISABLED) || ckpt_valid_blocks == usable_blocks)) { __locate_dirty_segment(sbi, segno, PRE); @@ -863,7 +874,7 @@ static void locate_dirty_segment(struct f2fs_sb_info *sbi, unsigned int segno) /* Recovery routine with SSR needs this */ __remove_dirty_segment(sbi, segno, DIRTY); } - +out_lock: mutex_unlock(&dirty_i->seglist_lock); } -- 2.40.1 > > Signed-off-by: Qi Han <hanqi@vivo.com> > --- > fs/f2fs/segment.h | 21 +++++++++++++++++++++ > 1 file changed, 21 insertions(+) > > diff --git a/fs/f2fs/segment.h b/fs/f2fs/segment.h > index 71adb4a43bec..9bf0cf3a6a31 100644 > --- a/fs/f2fs/segment.h > +++ b/fs/f2fs/segment.h > @@ -637,12 +637,33 @@ static inline bool has_enough_free_secs(struct f2fs_sb_info *sbi, > return !has_not_enough_free_secs(sbi, freed, needed); > } > > +static inline bool has_enough_available_blocks(struct f2fs_sb_info *sbi) > +{ > + unsigned int total_free_blocks = sbi->user_block_count - > + valid_user_blocks(sbi) - > + sbi->current_reserved_blocks; > + > + if (total_free_blocks <= sbi->unusable_block_count) > + total_free_blocks = 0; > + else > + total_free_blocks -= sbi->unusable_block_count; > + > + if (total_free_blocks > F2FS_OPTION(sbi).root_reserved_blocks) > + total_free_blocks -= F2FS_OPTION(sbi).root_reserved_blocks; > + else > + total_free_blocks = 0; > + > + return (total_free_blocks > 0) ? true : false; > +} > + > static inline bool f2fs_is_checkpoint_ready(struct f2fs_sb_info *sbi) > { > if (likely(!is_sbi_flag_set(sbi, SBI_CP_DISABLED))) > return true; > if (likely(has_enough_free_secs(sbi, 0, 0))) > return true; > + if (likely(has_enough_available_blocks(sbi))) > + return true; > return false; > } > _______________________________________________ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel ^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: [PATCH] f2fs: modify f2fs_is_checkpoint_ready logic to allow more data to be written with the CP disable @ 2024-10-17 9:54 ` Chao Yu 0 siblings, 0 replies; 16+ messages in thread From: Chao Yu @ 2024-10-17 9:54 UTC (permalink / raw) To: Qi Han, jaegeuk; +Cc: Chao Yu, linux-f2fs-devel, linux-kernel On 2024/10/9 18:27, Qi Han wrote: > When the free segment is used up during CP disable, many write or > ioctl operations will get ENOSPC error codes, even if there are > still many blocks available. We can reproduce it in the following > steps: > > dd if=/dev/zero of=f2fs.img bs=1M count=55 > mkfs.f2fs -f f2fs.img > mount f2fs.img f2fs_dir -o checkpoint=disable:10% > cd f2fs_dir > dd if=/dev/zero of=bigfile bs=1M count=50 > sync > rm bigfile > i=1; while [[ $i -lt 10000000 ]]; do (file_name=./file$i; dd \ > if=/dev/random of=$file_name bs=1M count=0); i=$((i+1)); done > stat -f ./ > > In f2fs_need_SSR() function, it is allowed to use SSR to allocate > blocks when CP is disabled, so in f2fs_is_checkpoint_ready function, > can we judge the number of invalid blocks when free segment is not > enough, and return ENOSPC only if the number of invalid blocks is > also not enough? Can you please try below patch? From 38b7c97dcc55ba83de4220c3dc54c2eb66148dd5 Mon Sep 17 00:00:00 2001 From: Chao Yu <chao@kernel.org> Date: Thu, 17 Oct 2024 17:07:05 +0800 Subject: [PATCH] f2fs: revalidate empty segment when checkpoint is disabled If checkpoint is off, let's set segment as free once all newly written datas were removed. Signed-off-by: Chao Yu <chao@kernel.org> --- fs/f2fs/segment.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c index f8d6e601e084..9bad13d70afb 100644 --- a/fs/f2fs/segment.c +++ b/fs/f2fs/segment.c @@ -853,6 +853,17 @@ static void locate_dirty_segment(struct f2fs_sb_info *sbi, unsigned int segno) valid_blocks = get_valid_blocks(sbi, segno, false); ckpt_valid_blocks = get_ckpt_valid_blocks(sbi, segno, false); + /* + * If checkpoint is off, let's set segment as free once all newly + * written datas were removed. + */ + if (is_sbi_flag_set(sbi, SBI_CP_DISABLED) && + valid_blocks == 0 && ckpt_valid_blocks == 0) { + __remove_dirty_segment(sbi, segno, DIRTY); + __set_test_and_free(sbi, segno, false); + goto out_lock; + } + if (valid_blocks == 0 && (!is_sbi_flag_set(sbi, SBI_CP_DISABLED) || ckpt_valid_blocks == usable_blocks)) { __locate_dirty_segment(sbi, segno, PRE); @@ -863,7 +874,7 @@ static void locate_dirty_segment(struct f2fs_sb_info *sbi, unsigned int segno) /* Recovery routine with SSR needs this */ __remove_dirty_segment(sbi, segno, DIRTY); } - +out_lock: mutex_unlock(&dirty_i->seglist_lock); } -- 2.40.1 > > Signed-off-by: Qi Han <hanqi@vivo.com> > --- > fs/f2fs/segment.h | 21 +++++++++++++++++++++ > 1 file changed, 21 insertions(+) > > diff --git a/fs/f2fs/segment.h b/fs/f2fs/segment.h > index 71adb4a43bec..9bf0cf3a6a31 100644 > --- a/fs/f2fs/segment.h > +++ b/fs/f2fs/segment.h > @@ -637,12 +637,33 @@ static inline bool has_enough_free_secs(struct f2fs_sb_info *sbi, > return !has_not_enough_free_secs(sbi, freed, needed); > } > > +static inline bool has_enough_available_blocks(struct f2fs_sb_info *sbi) > +{ > + unsigned int total_free_blocks = sbi->user_block_count - > + valid_user_blocks(sbi) - > + sbi->current_reserved_blocks; > + > + if (total_free_blocks <= sbi->unusable_block_count) > + total_free_blocks = 0; > + else > + total_free_blocks -= sbi->unusable_block_count; > + > + if (total_free_blocks > F2FS_OPTION(sbi).root_reserved_blocks) > + total_free_blocks -= F2FS_OPTION(sbi).root_reserved_blocks; > + else > + total_free_blocks = 0; > + > + return (total_free_blocks > 0) ? true : false; > +} > + > static inline bool f2fs_is_checkpoint_ready(struct f2fs_sb_info *sbi) > { > if (likely(!is_sbi_flag_set(sbi, SBI_CP_DISABLED))) > return true; > if (likely(has_enough_free_secs(sbi, 0, 0))) > return true; > + if (likely(has_enough_available_blocks(sbi))) > + return true; > return false; > } > ^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: [f2fs-dev] [PATCH] f2fs: modify f2fs_is_checkpoint_ready logic to allow more data to be written with the CP disable 2024-10-17 9:54 ` Chao Yu @ 2024-10-18 1:52 ` Zhiguo Niu -1 siblings, 0 replies; 16+ messages in thread From: Zhiguo Niu @ 2024-10-18 1:52 UTC (permalink / raw) To: Chao Yu; +Cc: jaegeuk, Qi Han, linux-kernel, linux-f2fs-devel Chao Yu via Linux-f2fs-devel <linux-f2fs-devel@lists.sourceforge.net> 于2024年10月17日周四 17:57写道： > > On 2024/10/9 18:27, Qi Han wrote: > > When the free segment is used up during CP disable, many write or > > ioctl operations will get ENOSPC error codes, even if there are > > still many blocks available. We can reproduce it in the following > > steps: > > > > dd if=/dev/zero of=f2fs.img bs=1M count=55 > > mkfs.f2fs -f f2fs.img > > mount f2fs.img f2fs_dir -o checkpoint=disable:10% > > cd f2fs_dir > > dd if=/dev/zero of=bigfile bs=1M count=50 > > sync > > rm bigfile > > i=1; while [[ $i -lt 10000000 ]]; do (file_name=./file$i; dd \ > > if=/dev/random of=$file_name bs=1M count=0); i=$((i+1)); done > > stat -f ./ > > > > In f2fs_need_SSR() function, it is allowed to use SSR to allocate > > blocks when CP is disabled, so in f2fs_is_checkpoint_ready function, Hi Chao and Qi, based on this comment and codes, I have some doubts: unusable blocks are calculated from invalid blocks of Dirty segment in f2fs_get_unsuable_blocks then minus ovp_hole, but SSR is allowed when the checkpoint is disabled. So actually SSR can not use most invalid blocks of Dirty segments? Is this a contradiction? Thanks! > > can we judge the number of invalid blocks when free segment is not > > enough, and return ENOSPC only if the number of invalid blocks is > > also not enough? > > Can you please try below patch? > > From 38b7c97dcc55ba83de4220c3dc54c2eb66148dd5 Mon Sep 17 00:00:00 2001 > From: Chao Yu <chao@kernel.org> > Date: Thu, 17 Oct 2024 17:07:05 +0800 > Subject: [PATCH] f2fs: revalidate empty segment when checkpoint is disabled > > If checkpoint is off, let's set segment as free once all newly > written datas were removed. > > Signed-off-by: Chao Yu <chao@kernel.org> > --- > fs/f2fs/segment.c | 13 ++++++++++++- > 1 file changed, 12 insertions(+), 1 deletion(-) > > diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c > index f8d6e601e084..9bad13d70afb 100644 > --- a/fs/f2fs/segment.c > +++ b/fs/f2fs/segment.c > @@ -853,6 +853,17 @@ static void locate_dirty_segment(struct f2fs_sb_info *sbi, unsigned int segno) > valid_blocks = get_valid_blocks(sbi, segno, false); > ckpt_valid_blocks = get_ckpt_valid_blocks(sbi, segno, false); > > + /* > + * If checkpoint is off, let's set segment as free once all newly > + * written datas were removed. > + */ > + if (is_sbi_flag_set(sbi, SBI_CP_DISABLED) && > + valid_blocks == 0 && ckpt_valid_blocks == 0) { > + __remove_dirty_segment(sbi, segno, DIRTY); > + __set_test_and_free(sbi, segno, false); > + goto out_lock; > + } > + > if (valid_blocks == 0 && (!is_sbi_flag_set(sbi, SBI_CP_DISABLED) || > ckpt_valid_blocks == usable_blocks)) { > __locate_dirty_segment(sbi, segno, PRE); > @@ -863,7 +874,7 @@ static void locate_dirty_segment(struct f2fs_sb_info *sbi, unsigned int segno) > /* Recovery routine with SSR needs this */ > __remove_dirty_segment(sbi, segno, DIRTY); > } > - > +out_lock: > mutex_unlock(&dirty_i->seglist_lock); > } > > -- > 2.40.1 > > > > > Signed-off-by: Qi Han <hanqi@vivo.com> > > --- > > fs/f2fs/segment.h | 21 +++++++++++++++++++++ > > 1 file changed, 21 insertions(+) > > > > diff --git a/fs/f2fs/segment.h b/fs/f2fs/segment.h > > index 71adb4a43bec..9bf0cf3a6a31 100644 > > --- a/fs/f2fs/segment.h > > +++ b/fs/f2fs/segment.h > > @@ -637,12 +637,33 @@ static inline bool has_enough_free_secs(struct f2fs_sb_info *sbi, > > return !has_not_enough_free_secs(sbi, freed, needed); > > } > > > > +static inline bool has_enough_available_blocks(struct f2fs_sb_info *sbi) > > +{ > > + unsigned int total_free_blocks = sbi->user_block_count - > > + valid_user_blocks(sbi) - > > + sbi->current_reserved_blocks; > > + > > + if (total_free_blocks <= sbi->unusable_block_count) > > + total_free_blocks = 0; > > + else > > + total_free_blocks -= sbi->unusable_block_count; > > + > > + if (total_free_blocks > F2FS_OPTION(sbi).root_reserved_blocks) > > + total_free_blocks -= F2FS_OPTION(sbi).root_reserved_blocks; > > + else > > + total_free_blocks = 0; > > + > > + return (total_free_blocks > 0) ? true : false; > > +} > > + > > static inline bool f2fs_is_checkpoint_ready(struct f2fs_sb_info *sbi) > > { > > if (likely(!is_sbi_flag_set(sbi, SBI_CP_DISABLED))) > > return true; > > if (likely(has_enough_free_secs(sbi, 0, 0))) > > return true; > > + if (likely(has_enough_available_blocks(sbi))) > > + return true; > > return false; > > } > > > > > > _______________________________________________ > Linux-f2fs-devel mailing list > Linux-f2fs-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel _______________________________________________ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [f2fs-dev] [PATCH] f2fs: modify f2fs_is_checkpoint_ready logic to allow more data to be written with the CP disable @ 2024-10-18 1:52 ` Zhiguo Niu 0 siblings, 0 replies; 16+ messages in thread From: Zhiguo Niu @ 2024-10-18 1:52 UTC (permalink / raw) To: Chao Yu; +Cc: Qi Han, jaegeuk, linux-kernel, linux-f2fs-devel Chao Yu via Linux-f2fs-devel <linux-f2fs-devel@lists.sourceforge.net> 于2024年10月17日周四 17:57写道： > > On 2024/10/9 18:27, Qi Han wrote: > > When the free segment is used up during CP disable, many write or > > ioctl operations will get ENOSPC error codes, even if there are > > still many blocks available. We can reproduce it in the following > > steps: > > > > dd if=/dev/zero of=f2fs.img bs=1M count=55 > > mkfs.f2fs -f f2fs.img > > mount f2fs.img f2fs_dir -o checkpoint=disable:10% > > cd f2fs_dir > > dd if=/dev/zero of=bigfile bs=1M count=50 > > sync > > rm bigfile > > i=1; while [[ $i -lt 10000000 ]]; do (file_name=./file$i; dd \ > > if=/dev/random of=$file_name bs=1M count=0); i=$((i+1)); done > > stat -f ./ > > > > In f2fs_need_SSR() function, it is allowed to use SSR to allocate > > blocks when CP is disabled, so in f2fs_is_checkpoint_ready function, Hi Chao and Qi, based on this comment and codes, I have some doubts: unusable blocks are calculated from invalid blocks of Dirty segment in f2fs_get_unsuable_blocks then minus ovp_hole, but SSR is allowed when the checkpoint is disabled. So actually SSR can not use most invalid blocks of Dirty segments? Is this a contradiction? Thanks! > > can we judge the number of invalid blocks when free segment is not > > enough, and return ENOSPC only if the number of invalid blocks is > > also not enough? > > Can you please try below patch? > > From 38b7c97dcc55ba83de4220c3dc54c2eb66148dd5 Mon Sep 17 00:00:00 2001 > From: Chao Yu <chao@kernel.org> > Date: Thu, 17 Oct 2024 17:07:05 +0800 > Subject: [PATCH] f2fs: revalidate empty segment when checkpoint is disabled > > If checkpoint is off, let's set segment as free once all newly > written datas were removed. > > Signed-off-by: Chao Yu <chao@kernel.org> > --- > fs/f2fs/segment.c | 13 ++++++++++++- > 1 file changed, 12 insertions(+), 1 deletion(-) > > diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c > index f8d6e601e084..9bad13d70afb 100644 > --- a/fs/f2fs/segment.c > +++ b/fs/f2fs/segment.c > @@ -853,6 +853,17 @@ static void locate_dirty_segment(struct f2fs_sb_info *sbi, unsigned int segno) > valid_blocks = get_valid_blocks(sbi, segno, false); > ckpt_valid_blocks = get_ckpt_valid_blocks(sbi, segno, false); > > + /* > + * If checkpoint is off, let's set segment as free once all newly > + * written datas were removed. > + */ > + if (is_sbi_flag_set(sbi, SBI_CP_DISABLED) && > + valid_blocks == 0 && ckpt_valid_blocks == 0) { > + __remove_dirty_segment(sbi, segno, DIRTY); > + __set_test_and_free(sbi, segno, false); > + goto out_lock; > + } > + > if (valid_blocks == 0 && (!is_sbi_flag_set(sbi, SBI_CP_DISABLED) || > ckpt_valid_blocks == usable_blocks)) { > __locate_dirty_segment(sbi, segno, PRE); > @@ -863,7 +874,7 @@ static void locate_dirty_segment(struct f2fs_sb_info *sbi, unsigned int segno) > /* Recovery routine with SSR needs this */ > __remove_dirty_segment(sbi, segno, DIRTY); > } > - > +out_lock: > mutex_unlock(&dirty_i->seglist_lock); > } > > -- > 2.40.1 > > > > > Signed-off-by: Qi Han <hanqi@vivo.com> > > --- > > fs/f2fs/segment.h | 21 +++++++++++++++++++++ > > 1 file changed, 21 insertions(+) > > > > diff --git a/fs/f2fs/segment.h b/fs/f2fs/segment.h > > index 71adb4a43bec..9bf0cf3a6a31 100644 > > --- a/fs/f2fs/segment.h > > +++ b/fs/f2fs/segment.h > > @@ -637,12 +637,33 @@ static inline bool has_enough_free_secs(struct f2fs_sb_info *sbi, > > return !has_not_enough_free_secs(sbi, freed, needed); > > } > > > > +static inline bool has_enough_available_blocks(struct f2fs_sb_info *sbi) > > +{ > > + unsigned int total_free_blocks = sbi->user_block_count - > > + valid_user_blocks(sbi) - > > + sbi->current_reserved_blocks; > > + > > + if (total_free_blocks <= sbi->unusable_block_count) > > + total_free_blocks = 0; > > + else > > + total_free_blocks -= sbi->unusable_block_count; > > + > > + if (total_free_blocks > F2FS_OPTION(sbi).root_reserved_blocks) > > + total_free_blocks -= F2FS_OPTION(sbi).root_reserved_blocks; > > + else > > + total_free_blocks = 0; > > + > > + return (total_free_blocks > 0) ? true : false; > > +} > > + > > static inline bool f2fs_is_checkpoint_ready(struct f2fs_sb_info *sbi) > > { > > if (likely(!is_sbi_flag_set(sbi, SBI_CP_DISABLED))) > > return true; > > if (likely(has_enough_free_secs(sbi, 0, 0))) > > return true; > > + if (likely(has_enough_available_blocks(sbi))) > > + return true; > > return false; > > } > > > > > > _______________________________________________ > Linux-f2fs-devel mailing list > Linux-f2fs-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [f2fs-dev] [PATCH] f2fs: modify f2fs_is_checkpoint_ready logic to allow more data to be written with the CP disable 2024-10-18 1:52 ` Zhiguo Niu @ 2024-10-21 3:43 ` 韩棋 -1 siblings, 0 replies; 16+ messages in thread From: 韩棋 via Linux-f2fs-devel @ 2024-10-21 3:43 UTC (permalink / raw) To: Zhiguo Niu, Chao Yu Cc: jaegeuk@kernel.org, linux-kernel@vger.kernel.org, linux-f2fs-devel@lists.sourceforge.net 在 2024/10/18 9:52, Zhiguo Niu 写道: > Chao Yu via Linux-f2fs-devel <linux-f2fs-devel@lists.sourceforge.net> > 于2024年10月17日周四 17:57写道： >> On 2024/10/9 18:27, Qi Han wrote: >>> When the free segment is used up during CP disable, many write or >>> ioctl operations will get ENOSPC error codes, even if there are >>> still many blocks available. We can reproduce it in the following >>> steps: >>> >>> dd if=/dev/zero of=f2fs.img bs=1M count=55 >>> mkfs.f2fs -f f2fs.img >>> mount f2fs.img f2fs_dir -o checkpoint=disable:10% >>> cd f2fs_dir >>> dd if=/dev/zero of=bigfile bs=1M count=50 >>> sync >>> rm bigfile >>> i=1; while [[ $i -lt 10000000 ]]; do (file_name=./file$i; dd \ >>> if=/dev/random of=$file_name bs=1M count=0); i=$((i+1)); done >>> stat -f ./ >>> >>> In f2fs_need_SSR() function, it is allowed to use SSR to allocate >>> blocks when CP is disabled, so in f2fs_is_checkpoint_ready function, > Hi Chao and Qi, > based on this comment and codes, I have some doubts: > unusable blocks are calculated from invalid blocks of Dirty segment > in f2fs_get_unsuable_blocks then minus ovp_hole, > but SSR is allowed when the checkpoint is disabled. > So actually SSR can not use most invalid blocks of Dirty segments? > Is this a contradiction? > Thanks! I think that in the case of a disable CP, in order to restore the data to the state before the disable CP, SSR cannot reuse all the invalid blocks, for example, suppose there is a fileA which occupies 10 data blocks when the CP enable, and then we delete the fileA after the CP disable, the 10 blocks cannot be reused by SSR. >>> can we judge the number of invalid blocks when free segment is not >>> enough, and return ENOSPC only if the number of invalid blocks is >>> also not enough? >> Can you please try below patch? >> >> From 38b7c97dcc55ba83de4220c3dc54c2eb66148dd5 Mon Sep 17 00:00:00 2001 >> From: Chao Yu <chao@kernel.org> >> Date: Thu, 17 Oct 2024 17:07:05 +0800 >> Subject: [PATCH] f2fs: revalidate empty segment when checkpoint is disabled >> >> If checkpoint is off, let's set segment as free once all newly >> written datas were removed. >> >> Signed-off-by: Chao Yu <chao@kernel.org> >> --- >> fs/f2fs/segment.c | 13 ++++++++++++- >> 1 file changed, 12 insertions(+), 1 deletion(-) >> >> diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c >> index f8d6e601e084..9bad13d70afb 100644 >> --- a/fs/f2fs/segment.c >> +++ b/fs/f2fs/segment.c >> @@ -853,6 +853,17 @@ static void locate_dirty_segment(struct f2fs_sb_info *sbi, unsigned int segno) >> valid_blocks = get_valid_blocks(sbi, segno, false); >> ckpt_valid_blocks = get_ckpt_valid_blocks(sbi, segno, false); >> >> + /* >> + * If checkpoint is off, let's set segment as free once all newly >> + * written datas were removed. >> + */ >> + if (is_sbi_flag_set(sbi, SBI_CP_DISABLED) && >> + valid_blocks == 0 && ckpt_valid_blocks == 0) { >> + __remove_dirty_segment(sbi, segno, DIRTY); >> + __set_test_and_free(sbi, segno, false); >> + goto out_lock; >> + } >> + >> if (valid_blocks == 0 && (!is_sbi_flag_set(sbi, SBI_CP_DISABLED) || >> ckpt_valid_blocks == usable_blocks)) { >> __locate_dirty_segment(sbi, segno, PRE); >> @@ -863,7 +874,7 @@ static void locate_dirty_segment(struct f2fs_sb_info *sbi, unsigned int segno) >> /* Recovery routine with SSR needs this */ >> __remove_dirty_segment(sbi, segno, DIRTY); >> } >> - >> +out_lock: >> mutex_unlock(&dirty_i->seglist_lock); >> } >> >> -- >> 2.40.1 >> >>> Signed-off-by: Qi Han <hanqi@vivo.com> >>> --- >>> fs/f2fs/segment.h | 21 +++++++++++++++++++++ >>> 1 file changed, 21 insertions(+) >>> >>> diff --git a/fs/f2fs/segment.h b/fs/f2fs/segment.h >>> index 71adb4a43bec..9bf0cf3a6a31 100644 >>> --- a/fs/f2fs/segment.h >>> +++ b/fs/f2fs/segment.h >>> @@ -637,12 +637,33 @@ static inline bool has_enough_free_secs(struct f2fs_sb_info *sbi, >>> return !has_not_enough_free_secs(sbi, freed, needed); >>> } >>> >>> +static inline bool has_enough_available_blocks(struct f2fs_sb_info *sbi) >>> +{ >>> + unsigned int total_free_blocks = sbi->user_block_count - >>> + valid_user_blocks(sbi) - >>> + sbi->current_reserved_blocks; >>> + >>> + if (total_free_blocks <= sbi->unusable_block_count) >>> + total_free_blocks = 0; >>> + else >>> + total_free_blocks -= sbi->unusable_block_count; >>> + >>> + if (total_free_blocks > F2FS_OPTION(sbi).root_reserved_blocks) >>> + total_free_blocks -= F2FS_OPTION(sbi).root_reserved_blocks; >>> + else >>> + total_free_blocks = 0; >>> + >>> + return (total_free_blocks > 0) ? true : false; >>> +} >>> + >>> static inline bool f2fs_is_checkpoint_ready(struct f2fs_sb_info *sbi) >>> { >>> if (likely(!is_sbi_flag_set(sbi, SBI_CP_DISABLED))) >>> return true; >>> if (likely(has_enough_free_secs(sbi, 0, 0))) >>> return true; >>> + if (likely(has_enough_available_blocks(sbi))) >>> + return true; >>> return false; >>> } >>> >> >> >> _______________________________________________ >> Linux-f2fs-devel mailing list >> Linux-f2fs-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel _______________________________________________ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [f2fs-dev] [PATCH] f2fs: modify f2fs_is_checkpoint_ready logic to allow more data to be written with the CP disable @ 2024-10-21 3:43 ` 韩棋 0 siblings, 0 replies; 16+ messages in thread From: 韩棋 @ 2024-10-21 3:43 UTC (permalink / raw) To: Zhiguo Niu, Chao Yu Cc: jaegeuk@kernel.org, linux-kernel@vger.kernel.org, linux-f2fs-devel@lists.sourceforge.net 在 2024/10/18 9:52, Zhiguo Niu 写道: > Chao Yu via Linux-f2fs-devel <linux-f2fs-devel@lists.sourceforge.net> > 于2024年10月17日周四 17:57写道： >> On 2024/10/9 18:27, Qi Han wrote: >>> When the free segment is used up during CP disable, many write or >>> ioctl operations will get ENOSPC error codes, even if there are >>> still many blocks available. We can reproduce it in the following >>> steps: >>> >>> dd if=/dev/zero of=f2fs.img bs=1M count=55 >>> mkfs.f2fs -f f2fs.img >>> mount f2fs.img f2fs_dir -o checkpoint=disable:10% >>> cd f2fs_dir >>> dd if=/dev/zero of=bigfile bs=1M count=50 >>> sync >>> rm bigfile >>> i=1; while [[ $i -lt 10000000 ]]; do (file_name=./file$i; dd \ >>> if=/dev/random of=$file_name bs=1M count=0); i=$((i+1)); done >>> stat -f ./ >>> >>> In f2fs_need_SSR() function, it is allowed to use SSR to allocate >>> blocks when CP is disabled, so in f2fs_is_checkpoint_ready function, > Hi Chao and Qi, > based on this comment and codes, I have some doubts: > unusable blocks are calculated from invalid blocks of Dirty segment > in f2fs_get_unsuable_blocks then minus ovp_hole, > but SSR is allowed when the checkpoint is disabled. > So actually SSR can not use most invalid blocks of Dirty segments? > Is this a contradiction? > Thanks! I think that in the case of a disable CP, in order to restore the data to the state before the disable CP, SSR cannot reuse all the invalid blocks, for example, suppose there is a fileA which occupies 10 data blocks when the CP enable, and then we delete the fileA after the CP disable, the 10 blocks cannot be reused by SSR. >>> can we judge the number of invalid blocks when free segment is not >>> enough, and return ENOSPC only if the number of invalid blocks is >>> also not enough? >> Can you please try below patch? >> >> From 38b7c97dcc55ba83de4220c3dc54c2eb66148dd5 Mon Sep 17 00:00:00 2001 >> From: Chao Yu <chao@kernel.org> >> Date: Thu, 17 Oct 2024 17:07:05 +0800 >> Subject: [PATCH] f2fs: revalidate empty segment when checkpoint is disabled >> >> If checkpoint is off, let's set segment as free once all newly >> written datas were removed. >> >> Signed-off-by: Chao Yu <chao@kernel.org> >> --- >> fs/f2fs/segment.c | 13 ++++++++++++- >> 1 file changed, 12 insertions(+), 1 deletion(-) >> >> diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c >> index f8d6e601e084..9bad13d70afb 100644 >> --- a/fs/f2fs/segment.c >> +++ b/fs/f2fs/segment.c >> @@ -853,6 +853,17 @@ static void locate_dirty_segment(struct f2fs_sb_info *sbi, unsigned int segno) >> valid_blocks = get_valid_blocks(sbi, segno, false); >> ckpt_valid_blocks = get_ckpt_valid_blocks(sbi, segno, false); >> >> + /* >> + * If checkpoint is off, let's set segment as free once all newly >> + * written datas were removed. >> + */ >> + if (is_sbi_flag_set(sbi, SBI_CP_DISABLED) && >> + valid_blocks == 0 && ckpt_valid_blocks == 0) { >> + __remove_dirty_segment(sbi, segno, DIRTY); >> + __set_test_and_free(sbi, segno, false); >> + goto out_lock; >> + } >> + >> if (valid_blocks == 0 && (!is_sbi_flag_set(sbi, SBI_CP_DISABLED) || >> ckpt_valid_blocks == usable_blocks)) { >> __locate_dirty_segment(sbi, segno, PRE); >> @@ -863,7 +874,7 @@ static void locate_dirty_segment(struct f2fs_sb_info *sbi, unsigned int segno) >> /* Recovery routine with SSR needs this */ >> __remove_dirty_segment(sbi, segno, DIRTY); >> } >> - >> +out_lock: >> mutex_unlock(&dirty_i->seglist_lock); >> } >> >> -- >> 2.40.1 >> >>> Signed-off-by: Qi Han <hanqi@vivo.com> >>> --- >>> fs/f2fs/segment.h | 21 +++++++++++++++++++++ >>> 1 file changed, 21 insertions(+) >>> >>> diff --git a/fs/f2fs/segment.h b/fs/f2fs/segment.h >>> index 71adb4a43bec..9bf0cf3a6a31 100644 >>> --- a/fs/f2fs/segment.h >>> +++ b/fs/f2fs/segment.h >>> @@ -637,12 +637,33 @@ static inline bool has_enough_free_secs(struct f2fs_sb_info *sbi, >>> return !has_not_enough_free_secs(sbi, freed, needed); >>> } >>> >>> +static inline bool has_enough_available_blocks(struct f2fs_sb_info *sbi) >>> +{ >>> + unsigned int total_free_blocks = sbi->user_block_count - >>> + valid_user_blocks(sbi) - >>> + sbi->current_reserved_blocks; >>> + >>> + if (total_free_blocks <= sbi->unusable_block_count) >>> + total_free_blocks = 0; >>> + else >>> + total_free_blocks -= sbi->unusable_block_count; >>> + >>> + if (total_free_blocks > F2FS_OPTION(sbi).root_reserved_blocks) >>> + total_free_blocks -= F2FS_OPTION(sbi).root_reserved_blocks; >>> + else >>> + total_free_blocks = 0; >>> + >>> + return (total_free_blocks > 0) ? true : false; >>> +} >>> + >>> static inline bool f2fs_is_checkpoint_ready(struct f2fs_sb_info *sbi) >>> { >>> if (likely(!is_sbi_flag_set(sbi, SBI_CP_DISABLED))) >>> return true; >>> if (likely(has_enough_free_secs(sbi, 0, 0))) >>> return true; >>> + if (likely(has_enough_available_blocks(sbi))) >>> + return true; >>> return false; >>> } >>> >> >> >> _______________________________________________ >> Linux-f2fs-devel mailing list >> Linux-f2fs-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [f2fs-dev] [PATCH] f2fs: modify f2fs_is_checkpoint_ready logic to allow more data to be written with the CP disable 2024-10-09 10:27 ` Qi Han @ 2024-10-22 2:53 ` Chao Yu -1 siblings, 0 replies; 16+ messages in thread From: Chao Yu via Linux-f2fs-devel @ 2024-10-22 2:53 UTC (permalink / raw) To: Qi Han, jaegeuk; +Cc: linux-kernel, linux-f2fs-devel On 2024/10/9 18:27, Qi Han wrote: > When the free segment is used up during CP disable, many write or > ioctl operations will get ENOSPC error codes, even if there are > still many blocks available. We can reproduce it in the following > steps: > > dd if=/dev/zero of=f2fs.img bs=1M count=55 > mkfs.f2fs -f f2fs.img > mount f2fs.img f2fs_dir -o checkpoint=disable:10% > cd f2fs_dir > dd if=/dev/zero of=bigfile bs=1M count=50 > sync > rm bigfile > i=1; while [[ $i -lt 10000000 ]]; do (file_name=./file$i; dd \ > if=/dev/random of=$file_name bs=1M count=0); i=$((i+1)); done > stat -f ./ > > In f2fs_need_SSR() function, it is allowed to use SSR to allocate > blocks when CP is disabled, so in f2fs_is_checkpoint_ready function, > can we judge the number of invalid blocks when free segment is not > enough, and return ENOSPC only if the number of invalid blocks is > also not enough? > > Signed-off-by: Qi Han <hanqi@vivo.com> > --- > fs/f2fs/segment.h | 21 +++++++++++++++++++++ > 1 file changed, 21 insertions(+) > > diff --git a/fs/f2fs/segment.h b/fs/f2fs/segment.h > index 71adb4a43bec..9bf0cf3a6a31 100644 > --- a/fs/f2fs/segment.h > +++ b/fs/f2fs/segment.h > @@ -637,12 +637,33 @@ static inline bool has_enough_free_secs(struct f2fs_sb_info *sbi, > return !has_not_enough_free_secs(sbi, freed, needed); > } > > +static inline bool has_enough_available_blocks(struct f2fs_sb_info *sbi) > +{ > + unsigned int total_free_blocks = sbi->user_block_count - > + valid_user_blocks(sbi) - > + sbi->current_reserved_blocks; > + > + if (total_free_blocks <= sbi->unusable_block_count) > + total_free_blocks = 0; > + else > + total_free_blocks -= sbi->unusable_block_count; > + > + if (total_free_blocks > F2FS_OPTION(sbi).root_reserved_blocks) > + total_free_blocks -= F2FS_OPTION(sbi).root_reserved_blocks; > + else > + total_free_blocks = 0; > + > + return (total_free_blocks > 0) ? true : false; Can you please reuse get_available_block_count() as much as possible? and cover it w/ stat_lock? Thanks, > +} > + > static inline bool f2fs_is_checkpoint_ready(struct f2fs_sb_info *sbi) > { > if (likely(!is_sbi_flag_set(sbi, SBI_CP_DISABLED))) > return true; > if (likely(has_enough_free_secs(sbi, 0, 0))) > return true; > + if (likely(has_enough_available_blocks(sbi))) > + return true; > return false; > } > _______________________________________________ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH] f2fs: modify f2fs_is_checkpoint_ready logic to allow more data to be written with the CP disable @ 2024-10-22 2:53 ` Chao Yu 0 siblings, 0 replies; 16+ messages in thread From: Chao Yu @ 2024-10-22 2:53 UTC (permalink / raw) To: Qi Han, jaegeuk; +Cc: Chao Yu, linux-f2fs-devel, linux-kernel On 2024/10/9 18:27, Qi Han wrote: > When the free segment is used up during CP disable, many write or > ioctl operations will get ENOSPC error codes, even if there are > still many blocks available. We can reproduce it in the following > steps: > > dd if=/dev/zero of=f2fs.img bs=1M count=55 > mkfs.f2fs -f f2fs.img > mount f2fs.img f2fs_dir -o checkpoint=disable:10% > cd f2fs_dir > dd if=/dev/zero of=bigfile bs=1M count=50 > sync > rm bigfile > i=1; while [[ $i -lt 10000000 ]]; do (file_name=./file$i; dd \ > if=/dev/random of=$file_name bs=1M count=0); i=$((i+1)); done > stat -f ./ > > In f2fs_need_SSR() function, it is allowed to use SSR to allocate > blocks when CP is disabled, so in f2fs_is_checkpoint_ready function, > can we judge the number of invalid blocks when free segment is not > enough, and return ENOSPC only if the number of invalid blocks is > also not enough? > > Signed-off-by: Qi Han <hanqi@vivo.com> > --- > fs/f2fs/segment.h | 21 +++++++++++++++++++++ > 1 file changed, 21 insertions(+) > > diff --git a/fs/f2fs/segment.h b/fs/f2fs/segment.h > index 71adb4a43bec..9bf0cf3a6a31 100644 > --- a/fs/f2fs/segment.h > +++ b/fs/f2fs/segment.h > @@ -637,12 +637,33 @@ static inline bool has_enough_free_secs(struct f2fs_sb_info *sbi, > return !has_not_enough_free_secs(sbi, freed, needed); > } > > +static inline bool has_enough_available_blocks(struct f2fs_sb_info *sbi) > +{ > + unsigned int total_free_blocks = sbi->user_block_count - > + valid_user_blocks(sbi) - > + sbi->current_reserved_blocks; > + > + if (total_free_blocks <= sbi->unusable_block_count) > + total_free_blocks = 0; > + else > + total_free_blocks -= sbi->unusable_block_count; > + > + if (total_free_blocks > F2FS_OPTION(sbi).root_reserved_blocks) > + total_free_blocks -= F2FS_OPTION(sbi).root_reserved_blocks; > + else > + total_free_blocks = 0; > + > + return (total_free_blocks > 0) ? true : false; Can you please reuse get_available_block_count() as much as possible? and cover it w/ stat_lock? Thanks, > +} > + > static inline bool f2fs_is_checkpoint_ready(struct f2fs_sb_info *sbi) > { > if (likely(!is_sbi_flag_set(sbi, SBI_CP_DISABLED))) > return true; > if (likely(has_enough_free_secs(sbi, 0, 0))) > return true; > + if (likely(has_enough_available_blocks(sbi))) > + return true; > return false; > } > ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [f2fs-dev] [PATCH] f2fs: modify f2fs_is_checkpoint_ready logic to allow more data to be written with the CP disable 2024-10-22 2:53 ` Chao Yu @ 2024-10-22 3:14 ` 韩棋 -1 siblings, 0 replies; 16+ messages in thread From: 韩棋 via Linux-f2fs-devel @ 2024-10-22 3:14 UTC (permalink / raw) To: Chao Yu, jaegeuk@kernel.org Cc: linux-kernel@vger.kernel.org, linux-f2fs-devel@lists.sourceforge.net 在 2024/10/22 10:53, Chao Yu 写道: > On 2024/10/9 18:27, Qi Han wrote: >> When the free segment is used up during CP disable, many write or >> ioctl operations will get ENOSPC error codes, even if there are >> still many blocks available. We can reproduce it in the following >> steps: >> >> dd if=/dev/zero of=f2fs.img bs=1M count=55 >> mkfs.f2fs -f f2fs.img >> mount f2fs.img f2fs_dir -o checkpoint=disable:10% >> cd f2fs_dir >> dd if=/dev/zero of=bigfile bs=1M count=50 >> sync >> rm bigfile >> i=1; while [[ $i -lt 10000000 ]]; do (file_name=./file$i; dd \ >> if=/dev/random of=$file_name bs=1M count=0); i=$((i+1)); done >> stat -f ./ >> >> In f2fs_need_SSR() function, it is allowed to use SSR to allocate >> blocks when CP is disabled, so in f2fs_is_checkpoint_ready function, >> can we judge the number of invalid blocks when free segment is not >> enough, and return ENOSPC only if the number of invalid blocks is >> also not enough? >> >> Signed-off-by: Qi Han <hanqi@vivo.com> >> --- >> fs/f2fs/segment.h | 21 +++++++++++++++++++++ >> 1 file changed, 21 insertions(+) >> >> diff --git a/fs/f2fs/segment.h b/fs/f2fs/segment.h >> index 71adb4a43bec..9bf0cf3a6a31 100644 >> --- a/fs/f2fs/segment.h >> +++ b/fs/f2fs/segment.h >> @@ -637,12 +637,33 @@ static inline bool has_enough_free_secs(struct >> f2fs_sb_info *sbi, >> return !has_not_enough_free_secs(sbi, freed, needed); >> } >> +static inline bool has_enough_available_blocks(struct f2fs_sb_info >> *sbi) >> +{ >> + unsigned int total_free_blocks = sbi->user_block_count - >> + valid_user_blocks(sbi) - >> + sbi->current_reserved_blocks; >> + >> + if (total_free_blocks <= sbi->unusable_block_count) >> + total_free_blocks = 0; >> + else >> + total_free_blocks -= sbi->unusable_block_count; >> + >> + if (total_free_blocks > F2FS_OPTION(sbi).root_reserved_blocks) >> + total_free_blocks -= F2FS_OPTION(sbi).root_reserved_blocks; >> + else >> + total_free_blocks = 0; >> + >> + return (total_free_blocks > 0) ? true : false; > > Can you please reuse get_available_block_count() as much as possible? > and cover it w/ stat_lock? > > Thanks, Thank you for your suggestion. I will send the v2 version patch later. > >> +} >> + >> static inline bool f2fs_is_checkpoint_ready(struct f2fs_sb_info *sbi) >> { >> if (likely(!is_sbi_flag_set(sbi, SBI_CP_DISABLED))) >> return true; >> if (likely(has_enough_free_secs(sbi, 0, 0))) >> return true; >> + if (likely(has_enough_available_blocks(sbi))) >> + return true; >> return false; >> } > > _______________________________________________ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH] f2fs: modify f2fs_is_checkpoint_ready logic to allow more data to be written with the CP disable @ 2024-10-22 3:14 ` 韩棋 0 siblings, 0 replies; 16+ messages in thread From: 韩棋 @ 2024-10-22 3:14 UTC (permalink / raw) To: Chao Yu, jaegeuk@kernel.org Cc: linux-f2fs-devel@lists.sourceforge.net, linux-kernel@vger.kernel.org 在 2024/10/22 10:53, Chao Yu 写道: > On 2024/10/9 18:27, Qi Han wrote: >> When the free segment is used up during CP disable, many write or >> ioctl operations will get ENOSPC error codes, even if there are >> still many blocks available. We can reproduce it in the following >> steps: >> >> dd if=/dev/zero of=f2fs.img bs=1M count=55 >> mkfs.f2fs -f f2fs.img >> mount f2fs.img f2fs_dir -o checkpoint=disable:10% >> cd f2fs_dir >> dd if=/dev/zero of=bigfile bs=1M count=50 >> sync >> rm bigfile >> i=1; while [[ $i -lt 10000000 ]]; do (file_name=./file$i; dd \ >> if=/dev/random of=$file_name bs=1M count=0); i=$((i+1)); done >> stat -f ./ >> >> In f2fs_need_SSR() function, it is allowed to use SSR to allocate >> blocks when CP is disabled, so in f2fs_is_checkpoint_ready function, >> can we judge the number of invalid blocks when free segment is not >> enough, and return ENOSPC only if the number of invalid blocks is >> also not enough? >> >> Signed-off-by: Qi Han <hanqi@vivo.com> >> --- >> fs/f2fs/segment.h | 21 +++++++++++++++++++++ >> 1 file changed, 21 insertions(+) >> >> diff --git a/fs/f2fs/segment.h b/fs/f2fs/segment.h >> index 71adb4a43bec..9bf0cf3a6a31 100644 >> --- a/fs/f2fs/segment.h >> +++ b/fs/f2fs/segment.h >> @@ -637,12 +637,33 @@ static inline bool has_enough_free_secs(struct >> f2fs_sb_info *sbi, >> return !has_not_enough_free_secs(sbi, freed, needed); >> } >> +static inline bool has_enough_available_blocks(struct f2fs_sb_info >> *sbi) >> +{ >> + unsigned int total_free_blocks = sbi->user_block_count - >> + valid_user_blocks(sbi) - >> + sbi->current_reserved_blocks; >> + >> + if (total_free_blocks <= sbi->unusable_block_count) >> + total_free_blocks = 0; >> + else >> + total_free_blocks -= sbi->unusable_block_count; >> + >> + if (total_free_blocks > F2FS_OPTION(sbi).root_reserved_blocks) >> + total_free_blocks -= F2FS_OPTION(sbi).root_reserved_blocks; >> + else >> + total_free_blocks = 0; >> + >> + return (total_free_blocks > 0) ? true : false; > > Can you please reuse get_available_block_count() as much as possible? > and cover it w/ stat_lock? > > Thanks, Thank you for your suggestion. I will send the v2 version patch later. > >> +} >> + >> static inline bool f2fs_is_checkpoint_ready(struct f2fs_sb_info *sbi) >> { >> if (likely(!is_sbi_flag_set(sbi, SBI_CP_DISABLED))) >> return true; >> if (likely(has_enough_free_secs(sbi, 0, 0))) >> return true; >> + if (likely(has_enough_available_blocks(sbi))) >> + return true; >> return false; >> } > > ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [f2fs-dev] [PATCH] f2fs: modify f2fs_is_checkpoint_ready logic to allow more data to be written with the CP disable 2024-10-22 3:14 ` 韩棋 @ 2024-10-22 3:39 ` Chao Yu -1 siblings, 0 replies; 16+ messages in thread From: Chao Yu via Linux-f2fs-devel @ 2024-10-22 3:39 UTC (permalink / raw) To: 韩棋, jaegeuk@kernel.org Cc: linux-kernel@vger.kernel.org, linux-f2fs-devel@lists.sourceforge.net On 2024/10/22 11:14, 韩棋 wrote: > 在 2024/10/22 10:53, Chao Yu 写道: >> On 2024/10/9 18:27, Qi Han wrote: >>> When the free segment is used up during CP disable, many write or >>> ioctl operations will get ENOSPC error codes, even if there are >>> still many blocks available. We can reproduce it in the following >>> steps: >>> >>> dd if=/dev/zero of=f2fs.img bs=1M count=55 >>> mkfs.f2fs -f f2fs.img >>> mount f2fs.img f2fs_dir -o checkpoint=disable:10% >>> cd f2fs_dir >>> dd if=/dev/zero of=bigfile bs=1M count=50 >>> sync >>> rm bigfile >>> i=1; while [[ $i -lt 10000000 ]]; do (file_name=./file$i; dd \ >>> if=/dev/random of=$file_name bs=1M count=0); i=$((i+1)); done >>> stat -f ./ >>> >>> In f2fs_need_SSR() function, it is allowed to use SSR to allocate >>> blocks when CP is disabled, so in f2fs_is_checkpoint_ready function, >>> can we judge the number of invalid blocks when free segment is not >>> enough, and return ENOSPC only if the number of invalid blocks is >>> also not enough? >>> >>> Signed-off-by: Qi Han <hanqi@vivo.com> >>> --- >>> fs/f2fs/segment.h | 21 +++++++++++++++++++++ >>> 1 file changed, 21 insertions(+) >>> >>> diff --git a/fs/f2fs/segment.h b/fs/f2fs/segment.h >>> index 71adb4a43bec..9bf0cf3a6a31 100644 >>> --- a/fs/f2fs/segment.h >>> +++ b/fs/f2fs/segment.h >>> @@ -637,12 +637,33 @@ static inline bool has_enough_free_secs(struct >>> f2fs_sb_info *sbi, >>> return !has_not_enough_free_secs(sbi, freed, needed); >>> } >>> +static inline bool has_enough_available_blocks(struct f2fs_sb_info >>> *sbi) >>> +{ >>> + unsigned int total_free_blocks = sbi->user_block_count - >>> + valid_user_blocks(sbi) - >>> + sbi->current_reserved_blocks; >>> + >>> + if (total_free_blocks <= sbi->unusable_block_count) >>> + total_free_blocks = 0; >>> + else >>> + total_free_blocks -= sbi->unusable_block_count; >>> + >>> + if (total_free_blocks > F2FS_OPTION(sbi).root_reserved_blocks) >>> + total_free_blocks -= F2FS_OPTION(sbi).root_reserved_blocks; >>> + else >>> + total_free_blocks = 0; >>> + >>> + return (total_free_blocks > 0) ? true : false; >> >> Can you please reuse get_available_block_count() as much as possible? >> and cover it w/ stat_lock? >> >> Thanks, > > Thank you for your suggestion. I will send the v2 version patch later. BTW, it looks your testcase won't create sparse available space in segment for following reuse by SSR, can you please update your testcase? Thanks, > >> >>> +} >>> + >>> static inline bool f2fs_is_checkpoint_ready(struct f2fs_sb_info *sbi) >>> { >>> if (likely(!is_sbi_flag_set(sbi, SBI_CP_DISABLED))) >>> return true; >>> if (likely(has_enough_free_secs(sbi, 0, 0))) >>> return true; >>> + if (likely(has_enough_available_blocks(sbi))) >>> + return true; >>> return false; >>> } >> >> > _______________________________________________ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH] f2fs: modify f2fs_is_checkpoint_ready logic to allow more data to be written with the CP disable @ 2024-10-22 3:39 ` Chao Yu 0 siblings, 0 replies; 16+ messages in thread From: Chao Yu @ 2024-10-22 3:39 UTC (permalink / raw) To: 韩棋, jaegeuk@kernel.org Cc: Chao Yu, linux-f2fs-devel@lists.sourceforge.net, linux-kernel@vger.kernel.org On 2024/10/22 11:14, 韩棋 wrote: > 在 2024/10/22 10:53, Chao Yu 写道: >> On 2024/10/9 18:27, Qi Han wrote: >>> When the free segment is used up during CP disable, many write or >>> ioctl operations will get ENOSPC error codes, even if there are >>> still many blocks available. We can reproduce it in the following >>> steps: >>> >>> dd if=/dev/zero of=f2fs.img bs=1M count=55 >>> mkfs.f2fs -f f2fs.img >>> mount f2fs.img f2fs_dir -o checkpoint=disable:10% >>> cd f2fs_dir >>> dd if=/dev/zero of=bigfile bs=1M count=50 >>> sync >>> rm bigfile >>> i=1; while [[ $i -lt 10000000 ]]; do (file_name=./file$i; dd \ >>> if=/dev/random of=$file_name bs=1M count=0); i=$((i+1)); done >>> stat -f ./ >>> >>> In f2fs_need_SSR() function, it is allowed to use SSR to allocate >>> blocks when CP is disabled, so in f2fs_is_checkpoint_ready function, >>> can we judge the number of invalid blocks when free segment is not >>> enough, and return ENOSPC only if the number of invalid blocks is >>> also not enough? >>> >>> Signed-off-by: Qi Han <hanqi@vivo.com> >>> --- >>> fs/f2fs/segment.h | 21 +++++++++++++++++++++ >>> 1 file changed, 21 insertions(+) >>> >>> diff --git a/fs/f2fs/segment.h b/fs/f2fs/segment.h >>> index 71adb4a43bec..9bf0cf3a6a31 100644 >>> --- a/fs/f2fs/segment.h >>> +++ b/fs/f2fs/segment.h >>> @@ -637,12 +637,33 @@ static inline bool has_enough_free_secs(struct >>> f2fs_sb_info *sbi, >>> return !has_not_enough_free_secs(sbi, freed, needed); >>> } >>> +static inline bool has_enough_available_blocks(struct f2fs_sb_info >>> *sbi) >>> +{ >>> + unsigned int total_free_blocks = sbi->user_block_count - >>> + valid_user_blocks(sbi) - >>> + sbi->current_reserved_blocks; >>> + >>> + if (total_free_blocks <= sbi->unusable_block_count) >>> + total_free_blocks = 0; >>> + else >>> + total_free_blocks -= sbi->unusable_block_count; >>> + >>> + if (total_free_blocks > F2FS_OPTION(sbi).root_reserved_blocks) >>> + total_free_blocks -= F2FS_OPTION(sbi).root_reserved_blocks; >>> + else >>> + total_free_blocks = 0; >>> + >>> + return (total_free_blocks > 0) ? true : false; >> >> Can you please reuse get_available_block_count() as much as possible? >> and cover it w/ stat_lock? >> >> Thanks, > > Thank you for your suggestion. I will send the v2 version patch later. BTW, it looks your testcase won't create sparse available space in segment for following reuse by SSR, can you please update your testcase? Thanks, > >> >>> +} >>> + >>> static inline bool f2fs_is_checkpoint_ready(struct f2fs_sb_info *sbi) >>> { >>> if (likely(!is_sbi_flag_set(sbi, SBI_CP_DISABLED))) >>> return true; >>> if (likely(has_enough_free_secs(sbi, 0, 0))) >>> return true; >>> + if (likely(has_enough_available_blocks(sbi))) >>> + return true; >>> return false; >>> } >> >> > ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [f2fs-dev] [PATCH] f2fs: modify f2fs_is_checkpoint_ready logic to allow more data to be written with the CP disable 2024-10-22 3:39 ` Chao Yu @ 2024-10-22 3:45 ` 韩棋 -1 siblings, 0 replies; 16+ messages in thread From: 韩棋 via Linux-f2fs-devel @ 2024-10-22 3:45 UTC (permalink / raw) To: Chao Yu, jaegeuk@kernel.org Cc: linux-kernel@vger.kernel.org, linux-f2fs-devel@lists.sourceforge.net 在 2024/10/22 11:39, Chao Yu 写道: > On 2024/10/22 11:14, 韩棋 wrote: >> 在 2024/10/22 10:53, Chao Yu 写道: >>> On 2024/10/9 18:27, Qi Han wrote: >>>> When the free segment is used up during CP disable, many write or >>>> ioctl operations will get ENOSPC error codes, even if there are >>>> still many blocks available. We can reproduce it in the following >>>> steps: >>>> >>>> dd if=/dev/zero of=f2fs.img bs=1M count=55 >>>> mkfs.f2fs -f f2fs.img >>>> mount f2fs.img f2fs_dir -o checkpoint=disable:10% >>>> cd f2fs_dir >>>> dd if=/dev/zero of=bigfile bs=1M count=50 >>>> sync >>>> rm bigfile >>>> i=1; while [[ $i -lt 10000000 ]]; do (file_name=./file$i; dd \ >>>> if=/dev/random of=$file_name bs=1M count=0); i=$((i+1)); done >>>> stat -f ./ >>>> >>>> In f2fs_need_SSR() function, it is allowed to use SSR to allocate >>>> blocks when CP is disabled, so in f2fs_is_checkpoint_ready function, >>>> can we judge the number of invalid blocks when free segment is not >>>> enough, and return ENOSPC only if the number of invalid blocks is >>>> also not enough? >>>> >>>> Signed-off-by: Qi Han <hanqi@vivo.com> >>>> --- >>>> fs/f2fs/segment.h | 21 +++++++++++++++++++++ >>>> 1 file changed, 21 insertions(+) >>>> >>>> diff --git a/fs/f2fs/segment.h b/fs/f2fs/segment.h >>>> index 71adb4a43bec..9bf0cf3a6a31 100644 >>>> --- a/fs/f2fs/segment.h >>>> +++ b/fs/f2fs/segment.h >>>> @@ -637,12 +637,33 @@ static inline bool has_enough_free_secs(struct >>>> f2fs_sb_info *sbi, >>>> return !has_not_enough_free_secs(sbi, freed, needed); >>>> } >>>> +static inline bool has_enough_available_blocks(struct f2fs_sb_info >>>> *sbi) >>>> +{ >>>> + unsigned int total_free_blocks = sbi->user_block_count - >>>> + valid_user_blocks(sbi) - >>>> + sbi->current_reserved_blocks; >>>> + >>>> + if (total_free_blocks <= sbi->unusable_block_count) >>>> + total_free_blocks = 0; >>>> + else >>>> + total_free_blocks -= sbi->unusable_block_count; >>>> + >>>> + if (total_free_blocks > F2FS_OPTION(sbi).root_reserved_blocks) >>>> + total_free_blocks -= F2FS_OPTION(sbi).root_reserved_blocks; >>>> + else >>>> + total_free_blocks = 0; >>>> + >>>> + return (total_free_blocks > 0) ? true : false; >>> >>> Can you please reuse get_available_block_count() as much as possible? >>> and cover it w/ stat_lock? >>> >>> Thanks, >> >> Thank you for your suggestion. I will send the v2 version patch later. > > BTW, it looks your testcase won't create sparse available space in > segment for following reuse by SSR, can you please update your > testcase? > > Thanks, Okay, I got it. I'll update later. Thank you for your advice. > >> >>> >>>> +} >>>> + >>>> static inline bool f2fs_is_checkpoint_ready(struct f2fs_sb_info >>>> *sbi) >>>> { >>>> if (likely(!is_sbi_flag_set(sbi, SBI_CP_DISABLED))) >>>> return true; >>>> if (likely(has_enough_free_secs(sbi, 0, 0))) >>>> return true; >>>> + if (likely(has_enough_available_blocks(sbi))) >>>> + return true; >>>> return false; >>>> } >>> >>> >> > > _______________________________________________ Linux-f2fs-devel mailing list Linux-f2fs-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH] f2fs: modify f2fs_is_checkpoint_ready logic to allow more data to be written with the CP disable @ 2024-10-22 3:45 ` 韩棋 0 siblings, 0 replies; 16+ messages in thread From: 韩棋 @ 2024-10-22 3:45 UTC (permalink / raw) To: Chao Yu, jaegeuk@kernel.org Cc: linux-f2fs-devel@lists.sourceforge.net, linux-kernel@vger.kernel.org 在 2024/10/22 11:39, Chao Yu 写道: > On 2024/10/22 11:14, 韩棋 wrote: >> 在 2024/10/22 10:53, Chao Yu 写道: >>> On 2024/10/9 18:27, Qi Han wrote: >>>> When the free segment is used up during CP disable, many write or >>>> ioctl operations will get ENOSPC error codes, even if there are >>>> still many blocks available. We can reproduce it in the following >>>> steps: >>>> >>>> dd if=/dev/zero of=f2fs.img bs=1M count=55 >>>> mkfs.f2fs -f f2fs.img >>>> mount f2fs.img f2fs_dir -o checkpoint=disable:10% >>>> cd f2fs_dir >>>> dd if=/dev/zero of=bigfile bs=1M count=50 >>>> sync >>>> rm bigfile >>>> i=1; while [[ $i -lt 10000000 ]]; do (file_name=./file$i; dd \ >>>> if=/dev/random of=$file_name bs=1M count=0); i=$((i+1)); done >>>> stat -f ./ >>>> >>>> In f2fs_need_SSR() function, it is allowed to use SSR to allocate >>>> blocks when CP is disabled, so in f2fs_is_checkpoint_ready function, >>>> can we judge the number of invalid blocks when free segment is not >>>> enough, and return ENOSPC only if the number of invalid blocks is >>>> also not enough? >>>> >>>> Signed-off-by: Qi Han <hanqi@vivo.com> >>>> --- >>>> fs/f2fs/segment.h | 21 +++++++++++++++++++++ >>>> 1 file changed, 21 insertions(+) >>>> >>>> diff --git a/fs/f2fs/segment.h b/fs/f2fs/segment.h >>>> index 71adb4a43bec..9bf0cf3a6a31 100644 >>>> --- a/fs/f2fs/segment.h >>>> +++ b/fs/f2fs/segment.h >>>> @@ -637,12 +637,33 @@ static inline bool has_enough_free_secs(struct >>>> f2fs_sb_info *sbi, >>>> return !has_not_enough_free_secs(sbi, freed, needed); >>>> } >>>> +static inline bool has_enough_available_blocks(struct f2fs_sb_info >>>> *sbi) >>>> +{ >>>> + unsigned int total_free_blocks = sbi->user_block_count - >>>> + valid_user_blocks(sbi) - >>>> + sbi->current_reserved_blocks; >>>> + >>>> + if (total_free_blocks <= sbi->unusable_block_count) >>>> + total_free_blocks = 0; >>>> + else >>>> + total_free_blocks -= sbi->unusable_block_count; >>>> + >>>> + if (total_free_blocks > F2FS_OPTION(sbi).root_reserved_blocks) >>>> + total_free_blocks -= F2FS_OPTION(sbi).root_reserved_blocks; >>>> + else >>>> + total_free_blocks = 0; >>>> + >>>> + return (total_free_blocks > 0) ? true : false; >>> >>> Can you please reuse get_available_block_count() as much as possible? >>> and cover it w/ stat_lock? >>> >>> Thanks, >> >> Thank you for your suggestion. I will send the v2 version patch later. > > BTW, it looks your testcase won't create sparse available space in > segment for following reuse by SSR, can you please update your > testcase? > > Thanks, Okay, I got it. I'll update later. Thank you for your advice. > >> >>> >>>> +} >>>> + >>>> static inline bool f2fs_is_checkpoint_ready(struct f2fs_sb_info >>>> *sbi) >>>> { >>>> if (likely(!is_sbi_flag_set(sbi, SBI_CP_DISABLED))) >>>> return true; >>>> if (likely(has_enough_free_secs(sbi, 0, 0))) >>>> return true; >>>> + if (likely(has_enough_available_blocks(sbi))) >>>> + return true; >>>> return false; >>>> } >>> >>> >> > > ^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2024-10-22 3:45 UTC | newest] Thread overview: 16+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2024-10-09 10:27 [f2fs-dev] [PATCH] f2fs: modify f2fs_is_checkpoint_ready logic to allow more data to be written with the CP disable Qi Han via Linux-f2fs-devel 2024-10-09 10:27 ` Qi Han 2024-10-17 9:54 ` [f2fs-dev] " Chao Yu via Linux-f2fs-devel 2024-10-17 9:54 ` Chao Yu 2024-10-18 1:52 ` [f2fs-dev] " Zhiguo Niu 2024-10-18 1:52 ` Zhiguo Niu 2024-10-21 3:43 ` 韩棋 via Linux-f2fs-devel 2024-10-21 3:43 ` 韩棋 2024-10-22 2:53 ` Chao Yu via Linux-f2fs-devel 2024-10-22 2:53 ` Chao Yu 2024-10-22 3:14 ` [f2fs-dev] " 韩棋 via Linux-f2fs-devel 2024-10-22 3:14 ` 韩棋 2024-10-22 3:39 ` [f2fs-dev] " Chao Yu via Linux-f2fs-devel 2024-10-22 3:39 ` Chao Yu 2024-10-22 3:45 ` [f2fs-dev] " 韩棋 via Linux-f2fs-devel 2024-10-22 3:45 ` 韩棋
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.