Re: [PATCH v3] writeback, memcg: skip foreign dirty tracking for bdev inodes

[Julian Sun <sunjunchao@bytedance.com>]

On 9/11/26 7:20 PM, Jan Kara wrote:
> On Wed 09-09-26 21:18:11, Julian Sun wrote:
>> On Wed, Sep 9, 2026 at 7:13 PM Jan Kara <jack@suse.cz> wrote:
>>>
>>> On Wed 09-09-26 11:43:43, Julian Sun wrote:
>>>> The expectation in wbc_detach_inode() that "concurrent write sharing of an
>>>> inode is expected to be very rare" does not hold for bdev inodes. On ext4,
>>>> metadata from many memcgs shares the same bdev inode, so dirty throttling
>>>> in those memcgs can repeatedly trigger foreign flushes of the owner wb.
>>>> These flushes can also write ordinary file data, reducing overwrite
>>>> coalescing under continuous buffered overwrites. The resulting extra I/O
>>>> leaves less device bandwidth for other workloads.
>>>
>>> Hum. Are you aware of [1]? The guy is from the same company so I was
>>> assuming you two are working together on the problem :).
>>
>> Yes, Xin and I are on the same team, and we are looking at different
>> ways to mitigate this issue.
> 
> OK :).
> 
>> We will continue trying to gather more evidence from production to
>> better understand its actual impact. The number of calls to
>> `balance_dirty_pages()` and the time spent in it might be useful
>> metrics for this purpose.
>>
>> Jan, if we can show that this patch provides a measurable improvement
>> in production, would you consider accepting it? Or do you think we
>> should still look for a better approach? If you think there is a
>> better way to address this, we'd be very interested in your
>> suggestion.
> 
> I definitely acknowledge that there's a problem with foreign flushing due
> to bdev inodes that needs to be fixed because as Tejun wrote, bdev inodes
> break the fundamental assumptions of foreign flushes. But so far I haven't
> seen a solution which I'd definitely support.
> 
> Having data from production to understand what exactly is the practical
> problem will hopefully help us understand how the solution should look
> like. Is it just that we trigger too many flushes and that slows down
> things? Or is even one foreign flush of the whole wb due to bdev inode
> making performance bad? Is it that we have many foreign pages and foreign
> flushes fail to properly write them?  These are questions we need to answer
> to decide how the solution should look like. And the solution could be
> anything from just limiting number of foreign flushes for a wb to 1
> (because there really isn't big point in more of them being queued) to more
> involved foreign page tracking for bdev inodes so that flushes can be more
> targetted.

Hi Jan,

Thanks for your feedback.

All the foreign flushes we've observed were triggered by bdev inodes.
What we've observed is that the bdev inode has only around 10 MB of
dirty pages, but it triggers writeback of around 40 GB of dirty pages
across the entire wb. The actual foreign pages account for less than
10 MB, yet flushing them triggers writeback of the whole wb, and this
happens repeatedly.

This appears to cause significant write amplification.

Limiting foreign flushes to one may not be sufficient in this case,
since even a single flush can trigger a large amount of unnecessary
writeback. I'm currently working on tracking foreign inodes so that
foreign flushes can target those inodes specifically.

The impact on production workloads is not easy to measure directly,
which is another challenge we're facing. We've had reports of slow
dirty page writeback from production workloads. Although we haven't
established the full causal chain yet, we suspect foreign flushes may
be contributing to the problem.
> 
> 								Honza

Thanks,
-- 
Julian Sun <sunjunchao@bytedance.com>