Re: ext4 writeback performance issue in 6.12

[Matt Fleming <matt@readmodwrite.com>]

On Thu, Oct 09, 2025 at 06:06:27PM +0530, Ojaswin Mujoo wrote:
> 
> Hi Matt,
> 
> Thanks for the details, we have had issues in past where the allocator
> gets stuck in a loop trying too hard to find blocks that are aligned to
> the stripe size [1] but this particular issue was patched in an pre 6.12
> kernel.
 
Yeah, we (Cloudflare) hit this exact issue last year.

> Coming to the above details, ext4_mb_find_good_group_avg_frag_list()
> exits early if there are no groups of the needed so if we do have many
> order 9+ allocations we shouldn't have been spending more time there.
> The issue I think are the order 9 allocations, which allocator thinks it
> can satisfy but it ends up not being able to find the space easily.
> If ext4_mb_find_group_avg_frag_list() is indeed a bottleneck, there
> are 2 places where it could be getting called from:
> 
> - ext4_mb_choose_next_group_goal_fast (criteria =
> 	EXT4_MB_CR_GOAL_LEN_FAST)
> - ext4_mb_choose_next_group_best_avail (criteria =
> 	EXT4_MB_CR_BEST_AVAIL_LEN)
> 
> Will it be possible for you to use bpf to try to figure out which one of
> the callers is actually the one bottlenecking (mihgt be tricky since
> they will mostly get inlined) and a sample of values for ac_g_ex->fe_len
> and ac_b_ex->fe_len if possible.
 
Mostly we go through ext4_mb_choose_next_group_goal_fast() but we also
go through ext4_mb_choose_next_group_best_avail().

> Also, can you share the ext4 mb stats by enabling it via:
> 
>  echo 1 > /sys/fs/ext4/vda2/mb_stats
> 
> And then once you are able to replicate it for a few mins: 
> 
>   cat /proc/fs/ext4/vda2/mb_stats
> 
> This will also give some idea on where the allocator is spending more
> time.
> 
> Also, as Ted suggested, switching stripe off might also help here.

Preliminary results look very promising with stripe disabled.