Linux EXT4 FS development
 help / color / mirror / Atom feed
* [PATCH v3 0/2] ext4: avoid tail walks for cached large-folio writes
@ 2026-06-09  3:52 Jia Zhu
  2026-06-09  3:52 ` [PATCH v3 1/2] fs/buffer: avoid tail commit walk for uptodate folios Jia Zhu
                   ` (3 more replies)
  0 siblings, 4 replies; 5+ messages in thread
From: Jia Zhu @ 2026-06-09  3:52 UTC (permalink / raw)
  To: Theodore Ts'o, Andreas Dilger
  Cc: Matthew Wilcox, Alexander Viro, Christian Brauner, Jan Kara,
	Baokun Li, Ojaswin Mujoo, Ritesh Harjani, Zhang Yi, linux-ext4,
	linux-fsdevel, linux-kernel, Jia Zhu

Hi,

This series addresses a buffered-write regression we found during our
v6.12 -> v6.18 LTS upgrade testing on ext4.

The regression is in the remaining buffer_head path.  A small overwrite
of an already cached, uptodate large folio still walks every buffer_head
attached to the folio in both write_begin and write_end.  With order-0
folios this was bounded by the page size.  After ext4 enabled large
folios for regular files, the same loops became proportional to the
folio size.

I agree that converting ext4 buffered I/O to iomap is the right long-term
direction, and that would avoid this problem.  This series is meant as a
small fix for current and LTS kernels that still use the buffer_head path.

Patch 1 follows Willy's suggestion for block_commit_write(): if the folio
was already uptodate on entry, stop the commit walk once the copied range
has been processed.

Patch 2 applies the same conservative shape to ext4_block_write_begin().
It keeps walking from the first buffer, so prefix buffer state handling is
unchanged, and only skips the suffix for folios that were already
uptodate on entry.

The workload is from libMicro, which we use in kernel release testing:

  https://github.com/rzezeski/libMicro

The table below includes the v6.12 baseline from the same release
benchmark.  The v6.12 and v6.18 columns were run with THP=always.  The
last column is v6.18 with this series applied.  Results are usecs/call,
lower is better, and the improvement is relative to unpatched v6.18.

case           v6.12    v6.18   v6.18 + series   improvement
write_u1k      0.609    4.659       0.528           88.7%
write_u10k     1.408    4.869       0.809           83.4%
pwrite_u1k     0.609    4.659       0.538           88.5%
pwrite_u10k    1.399    4.889       0.819           83.2%
writev_u1k     2.238    5.277       1.179           77.7%
writev_u10k   11.057    8.029       4.219           47.5%

For the cases that regressed from v6.12 to v6.18 in this test, this
series brings the v6.18 numbers back below the v6.12 cost.

Previous versions:
v2:
  https://lore.kernel.org/r/20260608120131.45146-1-zhujia.zj@bytedance.com
v1:
  https://lore.kernel.org/r/20260603134800.25155-1-zhujia.zj@bytedance.com

Changes since v2:
- simplify the ext4 loop condition as suggested by Jan;
- add Reviewed-by tags from Jan;
- add stable Cc tags.

Changes since v1:
- replace the ext4 seek-to-@from optimization with a conservative tail
  break that preserves prefix buffer handling;
- add the block_commit_write() tail break suggested by Willy;
- add v6.12 and v6.18 benchmark results for the full series.

Jia Zhu (2):
  fs/buffer: avoid tail commit walk for uptodate folios
  ext4: avoid tail write_begin walk for uptodate folios

 fs/buffer.c     |  3 +++
 fs/ext4/inode.c | 11 ++++++-----
 2 files changed, 9 insertions(+), 5 deletions(-)

-- 
2.20.1

^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH v3 1/2] fs/buffer: avoid tail commit walk for uptodate folios
  2026-06-09  3:52 [PATCH v3 0/2] ext4: avoid tail walks for cached large-folio writes Jia Zhu
@ 2026-06-09  3:52 ` Jia Zhu
  2026-06-09  3:52 ` [PATCH v3 2/2] ext4: avoid tail write_begin " Jia Zhu
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 5+ messages in thread
From: Jia Zhu @ 2026-06-09  3:52 UTC (permalink / raw)
  To: Theodore Ts'o, Andreas Dilger
  Cc: Matthew Wilcox, Alexander Viro, Christian Brauner, Jan Kara,
	Baokun Li, Ojaswin Mujoo, Ritesh Harjani, Zhang Yi, linux-ext4,
	linux-fsdevel, linux-kernel, Jia Zhu, stable

block_commit_write() always walks every buffer_head attached to the
folio.  That was cheap for order-0 folios, but large folios can contain
hundreds of buffer_heads.  For a small buffered overwrite of an
already-uptodate large folio, the commit work is therefore proportional
to the folio size rather than the copied range.

This became visible with ext4 regular-file large folios, where cached
small overwrites reach block_commit_write() through block_write_end().
Before ext4 enabled large folios for regular files, this path was only
hit with order-0 folios for normal ext4 buffered writes, so the full walk
was bounded.  The ext4 large-folio commit is therefore the regression
point for this generic helper cost.

The full walk is still needed when the folio is not uptodate, because
block_commit_write() uses per-buffer uptodate state to decide whether
the whole folio can be marked uptodate.  Keep those folios on the old
full-buffer path.

For a folio that was already uptodate on entry, the commit no longer
needs tail buffers for folio-uptodate discovery.  The copied range has
already been processed once block_start reaches @to, so stop there and
avoid the suffix walk.

Fixes: 7ac67301e82f0 ("ext4: enable large folio for regular file")
Suggested-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: stable@vger.kernel.org # v6.16+
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Jia Zhu <zhujia.zj@bytedance.com>
---
 fs/buffer.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/fs/buffer.c b/fs/buffer.c
index b0b3792b1496e..c8c41c799030d 100644
--- a/fs/buffer.c
+++ b/fs/buffer.c
@@ -2096,6 +2096,7 @@ void block_commit_write(struct folio *folio, size_t from, size_t to)
 {
 	size_t block_start, block_end;
 	bool partial = false;
+	bool uptodate = folio_test_uptodate(folio);
 	unsigned blocksize;
 	struct buffer_head *bh, *head;
 
@@ -2118,6 +2119,8 @@ void block_commit_write(struct folio *folio, size_t from, size_t to)
 			clear_buffer_new(bh);
 
 		block_start = block_end;
+		if (uptodate && block_start >= to)
+			break;
 		bh = bh->b_this_page;
 	} while (bh != head);
 
-- 
2.20.1

^ permalink raw reply related	[flat|nested] 5+ messages in thread

* [PATCH v3 2/2] ext4: avoid tail write_begin walk for uptodate folios
  2026-06-09  3:52 [PATCH v3 0/2] ext4: avoid tail walks for cached large-folio writes Jia Zhu
  2026-06-09  3:52 ` [PATCH v3 1/2] fs/buffer: avoid tail commit walk for uptodate folios Jia Zhu
@ 2026-06-09  3:52 ` Jia Zhu
  2026-07-23 13:08 ` [PATCH v3 0/2] ext4: avoid tail walks for cached large-folio writes Theodore Ts'o
  2026-11-18  9:12 ` Chengfeng Lin
  3 siblings, 0 replies; 5+ messages in thread
From: Jia Zhu @ 2026-06-09  3:52 UTC (permalink / raw)
  To: Theodore Ts'o, Andreas Dilger
  Cc: Matthew Wilcox, Alexander Viro, Christian Brauner, Jan Kara,
	Baokun Li, Ojaswin Mujoo, Ritesh Harjani, Zhang Yi, linux-ext4,
	linux-fsdevel, linux-kernel, Jia Zhu, stable

Ext4 buffered writes into large folios also pay a full buffer_head
walk in ext4_block_write_begin().  For a small overwrite of an existing
cached folio, the folio is already uptodate and the write only needs to
prepare the buffers through the written range.  Walking the suffix still
makes the write_begin cost proportional to the folio size.

Before ext4 enabled large folios for regular files, the same loop was
bounded by a single page of buffers.  That commit made the existing
full-folio walk visible as a regression for cached small overwrites.

The suffix walk is needed for non-uptodate folios, where ext4 may have
to submit reads for partial blocks, preserve new-buffer cleanup, and run
error zeroing.  Keep those folios on the old full walk.

For already-uptodate folios, keep the walk starting at the first buffer
rather than seeking directly to from.  This preserves the existing prefix
buffer state handling.  Stop once block_start reaches the end of the
write range, because the skipped suffix would only repeat the
outside-range uptodate handling for buffers beyond @to.

On current master, the libMicro ext4 large-folio overwrite test shows
the following full-series result.  Results are median usecs/call over 10
runs, lower is better:

case        nofix     this series   improvement
write_u1k   1.418     0.3405        76.0%
write_u10k  1.887     0.4175        77.9%
pwrite_u1k  1.6775    0.3390        79.8%
pwrite_u10k 1.9035    0.4130        78.3%

Fixes: 7ac67301e82f0 ("ext4: enable large folio for regular file")
Cc: stable@vger.kernel.org # v6.16+
Reviewed-by: Jan Kara <jack@suse.cz>
Signed-off-by: Jia Zhu <zhujia.zj@bytedance.com>
---
 fs/ext4/inode.c | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index c2c2d6ac7f3d1..0fccb8f6a2116 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -1182,6 +1182,7 @@ int ext4_block_write_begin(handle_t *handle, struct folio *folio,
 	int nr_wait = 0;
 	int i;
 	bool should_journal_data = ext4_should_journal_data(inode);
+	bool folio_uptodate = folio_test_uptodate(folio);
 
 	BUG_ON(!folio_test_locked(folio));
 	BUG_ON(to > folio_size(folio));
@@ -1193,13 +1194,13 @@ int ext4_block_write_begin(handle_t *handle, struct folio *folio,
 		head = create_empty_buffers(folio, blocksize, 0);
 	block = EXT4_PG_TO_LBLK(inode, folio->index);
 
-	for (bh = head, block_start = 0; bh != head || !block_start;
+	for (bh = head, block_start = 0;
+	     block_start < to || (!folio_uptodate && bh != head);
 	    block++, block_start = block_end, bh = bh->b_this_page) {
 		block_end = block_start + blocksize;
 		if (block_end <= from || block_start >= to) {
-			if (folio_test_uptodate(folio)) {
+			if (folio_uptodate)
 				set_buffer_uptodate(bh);
-			}
 			continue;
 		}
 		if (WARN_ON_ONCE(buffer_new(bh)))
@@ -1220,7 +1221,7 @@ int ext4_block_write_begin(handle_t *handle, struct folio *folio,
 				if (should_journal_data)
 					do_journal_get_write_access(handle,
 								    inode, bh);
-				if (folio_test_uptodate(folio)) {
+				if (folio_uptodate) {
 					/*
 					 * Unlike __block_write_begin() we leave
 					 * dirtying of new uptodate buffers to
@@ -1237,7 +1238,7 @@ int ext4_block_write_begin(handle_t *handle, struct folio *folio,
 				continue;
 			}
 		}
-		if (folio_test_uptodate(folio)) {
+		if (folio_uptodate) {
 			set_buffer_uptodate(bh);
 			continue;
 		}
-- 
2.20.1

^ permalink raw reply related	[flat|nested] 5+ messages in thread

* Re: [PATCH v3 0/2] ext4: avoid tail walks for cached large-folio writes
  2026-06-09  3:52 [PATCH v3 0/2] ext4: avoid tail walks for cached large-folio writes Jia Zhu
  2026-06-09  3:52 ` [PATCH v3 1/2] fs/buffer: avoid tail commit walk for uptodate folios Jia Zhu
  2026-06-09  3:52 ` [PATCH v3 2/2] ext4: avoid tail write_begin " Jia Zhu
@ 2026-07-23 13:08 ` Theodore Ts'o
  2026-11-18  9:12 ` Chengfeng Lin
  3 siblings, 0 replies; 5+ messages in thread
From: Theodore Ts'o @ 2026-07-23 13:08 UTC (permalink / raw)
  To: Andreas Dilger, Jia Zhu
  Cc: Theodore Ts'o, Matthew Wilcox, Alexander Viro,
	Christian Brauner, Jan Kara, Baokun Li, Ojaswin Mujoo,
	Ritesh Harjani, Zhang Yi, linux-ext4, linux-fsdevel, linux-kernel


On Tue, 09 Jun 2026 11:52:00 +0800, Jia Zhu wrote:
> This series addresses a buffered-write regression we found during our
> v6.12 -> v6.18 LTS upgrade testing on ext4.
> 
> The regression is in the remaining buffer_head path.  A small overwrite
> of an already cached, uptodate large folio still walks every buffer_head
> attached to the folio in both write_begin and write_end.  With order-0
> folios this was bounded by the page size.  After ext4 enabled large
> folios for regular files, the same loops became proportional to the
> folio size.
> 
> [...]

Applied, thanks!

[1/2] fs/buffer: avoid tail commit walk for uptodate folios
      commit: f10b9cc1eb20637351f4e33372bfb464f89de59b
[2/2] ext4: avoid tail write_begin walk for uptodate folios
      commit: d09811183db2891776dbf0c0f1094540e29938f6

Best regards,
-- 
Theodore Ts'o <tytso@mit.edu>

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [PATCH v3 0/2] ext4: avoid tail walks for cached large-folio writes
  2026-06-09  3:52 [PATCH v3 0/2] ext4: avoid tail walks for cached large-folio writes Jia Zhu
                   ` (2 preceding siblings ...)
  2026-07-23 13:08 ` [PATCH v3 0/2] ext4: avoid tail walks for cached large-folio writes Theodore Ts'o
@ 2026-11-18  9:12 ` Chengfeng Lin
  3 siblings, 0 replies; 5+ messages in thread
From: Chengfeng Lin @ 2026-11-18  9:12 UTC (permalink / raw)
  To: Jia Zhu
  Cc: Theodore Ts'o, Andreas Dilger, Matthew Wilcox, Jan Kara,
	linux-ext4, linux-fsdevel

Hi Jia and all,

In libMicro write_t1k, one worker overwrites a prepared ext4 file in
1 KiB buffered writes. In our v6.15.11 / v7.3 / v6.15.11 comparison,
with nine runs per endpoint, libMicro's reported time per write rose
from about 0.233 to 0.502 us (about 115%). As discussed in this series,
large folios amplify the existing buffer_head walks. The measurements
below concern the remaining prefix cost after d09811183db2 and
f10b9cc1eb20, both included in the tested v7.3 kernel.

Before each run, I created a fresh 10 MiB file on ext4 with ten buffered
1 MiB writes of zeros. Each batch seeks back to offset zero and performs
10,000 sequential overwrites; each invocation runs for 30 seconds.
The machine is an i7-12700KF, with THP=madvise. The worker was allowed on
all 20 logical CPUs, without another benchmark running.

I compared v7.3, the same kernel with ext4's maximum folio order capped
at its minimum order, and v7.3 again. Each endpoint has nine clean runs:

                                v7.3 A   min-order   v7.3 B
  Buffered write, us/1 KiB call  0.500237  0.245877    0.496312
  Observed folio size            1 MiB     4 KiB       1 MiB
  Buffers visited per walk       128.5      1          128.5
  Of those, before write range   127.5      0          127.5

The timing row averages libMicro's nine per-run medians at each
endpoint. It covers the 1 KiB buffered overwrite loop, excluding file
preparation. Maximum run-to-run CV was 0.59%; baseline drift was -0.78%.

For the traversal counts, I instrumented one separate run of the same
workload at each endpoint. I counted buffer visits in the begin and
commit walks separately for the first 1,024 writes from offset zero.
At 1 KiB per write, that covers one full 1 MiB folio on the unmodified
kernel.

The min-order control reduced the reported time per 1 KiB buffered write
by about 50.5%, or 0.25 us/call. This supports a remaining prefix-related
cost, although changing folio size also changes allocation/cache
granularity; the control is diagnostic, not a proposal to disable large
folios generally.

These counts quantify the prefix traversal that remains after the tail
fixes and may be useful for future work on this path.

The libMicro source is gormanm/libMicro at
976d502afaa2d9b919564d7fa8a08fec286dfc89. The command was:

  ./write -E -C 0 -D 30000 -B 10000 -L -S -W \
      -N write_t1k -s 1k -f /path/on/ext4/data

Thanks.

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2026-11-18  9:36 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-06-09  3:52 [PATCH v3 0/2] ext4: avoid tail walks for cached large-folio writes Jia Zhu
2026-06-09  3:52 ` [PATCH v3 1/2] fs/buffer: avoid tail commit walk for uptodate folios Jia Zhu
2026-06-09  3:52 ` [PATCH v3 2/2] ext4: avoid tail write_begin " Jia Zhu
2026-07-23 13:08 ` [PATCH v3 0/2] ext4: avoid tail walks for cached large-folio writes Theodore Ts'o
2026-11-18  9:12 ` Chengfeng Lin

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox