[f2fs-dev] [question] Design rationale behind f2fs_map_blocks block reuse in LFS mode with DIO

[f2fs-dev] [question] Design rationale behind f2fs_map_blocks block reuse in LFS mode with DIO

[김규진]

Hi F2FS developers,

I'm testing multi-threaded direct I/O in LFS mode on Linux kernel
6.8.0-57.59, and noticed what seems to be an inefficiency in block
allocation behavior inside `fs/f2fs/data.c` (specifically
`f2fs_map_blocks()`):

1. In LFS mode with direct I/O, `f2fs_map_blocks()` always calls
`__allocate_data_block()` to reserve a new block and updates the
node/NAT entry, regardless of extent continuity.

2. If the new block is not physically contiguous with the current
extent, it submits the current bio and defers the write of the newly
reserved block (which is now recorded in the node) to the next
mapping.

3. On the next `f2fs_map_blocks()` call, it finds that the logical
block is already mapped in the node/NAT entry and skips over
it—despite the block never having been written—resulting in allocation
of yet another block. Over time, this leaves behind holes in the
current segment, especially under heavy multi-threaded DIO.


Since I'm still new to F2FS internals, I may be missing something —
I'd like to understand the design rationale behind this behavior in
LFS mode, if possible.

**My questions are:**

- Is there a specific reason F2FS does not distinguish between
reserved-but-unwritten and already-written blocks in this case?
- Would it be possible (or beneficial) to:

  1. Delay block allocation until the extent can actually be extended?

  2. Track "reserved-but-unwritten" blocks distinctly to avoid reallocation?


Thanks in advance for any clarification or insight.

Best regards,

Gyjin Kim


_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] [question] Design rationale behind f2fs_map_blocks block reuse in LFS mode with DIO

[Chao Yu via Linux-f2fs-devel]

Hi,

Sorry for the delay.

On 6/12/25 20:32, 김규진 wrote:
> Hi F2FS developers,
> 
> I'm testing multi-threaded direct I/O in LFS mode on Linux kernel
> 6.8.0-57.59, and noticed what seems to be an inefficiency in block
> allocation behavior inside `fs/f2fs/data.c` (specifically
> `f2fs_map_blocks()`):
> 
> 1. In LFS mode with direct I/O, `f2fs_map_blocks()` always calls
> `__allocate_data_block()` to reserve a new block and updates the
> node/NAT entry, regardless of extent continuity.
> 
> 2. If the new block is not physically contiguous with the current
> extent, it submits the current bio and defers the write of the newly
> reserved block (which is now recorded in the node) to the next
> mapping.
> 
> 3. On the next `f2fs_map_blocks()` call, it finds that the logical
> block is already mapped in the node/NAT entry and skips over
> it—despite the block never having been written—resulting in allocation
> of yet another block. Over time, this leaves behind holes in the
> current segment, especially under heavy multi-threaded DIO.

IIUC,

The problem is something like this, is my understanding right?

- user tries to write 768 blocks w/ direct IO.
- f2fs_iomap_begin(ofs:0, len:768)
 - f2fs_map_blocks allocates two extents [ofs:0, blk:512, len:512] and
   [ofs:512, blk:0, len:0], however f2fs_map_blocks() only return the first
   extent,
 - f2fs_iomap_begin(ofs:512, len:256)
   f2fs_map_blocks allocates another physical block for ofs:512 even there is
   a unwritten physical block allocated during previous f2fs_map_blocks.

Thanks,

> 
> 
> Since I'm still new to F2FS internals, I may be missing something —
> I'd like to understand the design rationale behind this behavior in
> LFS mode, if possible.
> 
> **My questions are:**
> 
> - Is there a specific reason F2FS does not distinguish between
> reserved-but-unwritten and already-written blocks in this case?
> - Would it be possible (or beneficial) to:
> 
>   1. Delay block allocation until the extent can actually be extended?
> 
>   2. Track "reserved-but-unwritten" blocks distinctly to avoid reallocation?
> 
> 
> Thanks in advance for any clarification or insight.
> 
> Best regards,
> 
> Gyjin Kim
> 
> 
> _______________________________________________
> Linux-f2fs-devel mailing list
> Linux-f2fs-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel



_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel

Re: [f2fs-dev] [question] Design rationale behind f2fs_map_blocks block reuse in LFS mode with DIO

[Chao Yu via Linux-f2fs-devel]

On 2025/7/15 16:56, Chao Yu wrote:
> Hi,
> 
> Sorry for the delay.
> 
> On 6/12/25 20:32, 김규진 wrote:
>> Hi F2FS developers,
>>
>> I'm testing multi-threaded direct I/O in LFS mode on Linux kernel
>> 6.8.0-57.59, and noticed what seems to be an inefficiency in block
>> allocation behavior inside `fs/f2fs/data.c` (specifically
>> `f2fs_map_blocks()`):
>>
>> 1. In LFS mode with direct I/O, `f2fs_map_blocks()` always calls
>> `__allocate_data_block()` to reserve a new block and updates the
>> node/NAT entry, regardless of extent continuity.
>>
>> 2. If the new block is not physically contiguous with the current
>> extent, it submits the current bio and defers the write of the newly
>> reserved block (which is now recorded in the node) to the next
>> mapping.
>>
>> 3. On the next `f2fs_map_blocks()` call, it finds that the logical
>> block is already mapped in the node/NAT entry and skips over
>> it—despite the block never having been written—resulting in allocation
>> of yet another block. Over time, this leaves behind holes in the
>> current segment, especially under heavy multi-threaded DIO.
> 
> IIUC,
> 
> The problem is something like this, is my understanding right?
> 
> - user tries to write 768 blocks w/ direct IO.
> - f2fs_iomap_begin(ofs:0, len:768)
>   - f2fs_map_blocks allocates two extents [ofs:0, blk:512, len:512] and
>     [ofs:512, blk:0, len:0], however f2fs_map_blocks() only return the first
>     extent,
>   - f2fs_iomap_begin(ofs:512, len:256)
>     f2fs_map_blocks allocates another physical block for ofs:512 even there is
>     a unwritten physical block allocated during previous f2fs_map_blocks.

If I'm not missing any thing, this issue has been fixed w/ below patch:

commit 0638a3197c194bed837c157c3574685e36febc7b
Author: Daejun Park <daejun7.park@samsung.com>
Date:   Thu Sep 5 14:24:33 2024 +0900

     f2fs: avoid unused block when dio write in LFS mode

     This patch addresses the problem that when using LFS mode, unused blocks
     may occur in f2fs_map_blocks() during block allocation for dio writes.

     If a new section is allocated during block allocation, it will not be
     included in the map struct by map_is_mergeable() if the LBA of the
     allocated block is not contiguous. However, the block already allocated
     in this process will remain unused due to the LFS mode.

     This patch avoids the possibility of unused blocks by escaping
     f2fs_map_blocks() when allocating the last block in a section.

     Signed-off-by: Daejun Park <daejun7.park@samsung.com>
     Reviewed-by: Chao Yu <chao@kernel.org>
     Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>

Thanks,

> 
> Thanks,
> 
>>
>>
>> Since I'm still new to F2FS internals, I may be missing something —
>> I'd like to understand the design rationale behind this behavior in
>> LFS mode, if possible.
>>
>> **My questions are:**
>>
>> - Is there a specific reason F2FS does not distinguish between
>> reserved-but-unwritten and already-written blocks in this case?
>> - Would it be possible (or beneficial) to:
>>
>>    1. Delay block allocation until the extent can actually be extended?
>>
>>    2. Track "reserved-but-unwritten" blocks distinctly to avoid reallocation?
>>
>>
>> Thanks in advance for any clarification or insight.
>>
>> Best regards,
>>
>> Gyjin Kim
>>
>>
>> _______________________________________________
>> Linux-f2fs-devel mailing list
>> Linux-f2fs-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
> 



_______________________________________________
Linux-f2fs-devel mailing list
Linux-f2fs-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel