Re: [PATCH v2] writeback, memcg: skip foreign dirty tracking for bdev inodes

[Julian Sun <sunjunchao@bytedance.com>]

On 9/4/26 5:00 PM, Jan Kara wrote:
> On Fri 04-09-26 11:39:23, Julian Sun wrote:
>> On 9/4/26 3:34 AM, Andrew Morton wrote:
>>> On Thu,  3 Sep 2026 16:33:03 +0800 Julian Sun <sunjunchao@bytedance.com> wrote:
>>>
>>>> Foreign dirty flushing handles cases where a dirtied folio's memcg and
>>>> the memcg owning its inode wb differ.
>>>>
>>>> wbc_detach_inode() notes that "concurrent write sharing of an inode is
>>>> expected to be very rare". This expectation does not hold for bdev inodes.
>>>> A bdev inode and mapping are shared by buffered users of the block device,
>>>> while the inode has a single wb owner. On ext4, buffer-cache folios used
>>>> by metadata and journal I/O can therefore be charged to different memcgs
>>>> while sharing the same mapping. Normal bdev activity can thus produce
>>>> frequent folio/wb ownership mismatches.
>>>>
>>>> This was reproduced on cgroup v2 with the memory and I/O controllers
>>>> enabled, using ext4 on a loop device. jbd2 repeatedly generated
>>>> track_foreign_dirty events for bdev folios charged to unrelated memcgs.
>>>> When one of those memcgs entered dirty throttling, the resulting record led
>>>> to a flush_foreign event and queued writeback with reason=foreign_flush for
>>>> the bdev inode's root wb.
>>>>
>>>> Skip foreign dirty tracking when the folio mapping's host inode is on the
>>>> blockdev pseudo superblock. This prevents bdev-originated records from
>>>> triggering later foreign flushes.
>>>>
>>>> The tradeoff is that dirty throttling in the folio's memcg no longer uses
>>>> bdev dirtiness to queue writeback on the bdev inode's owner wb. Those
>>>> folios remain subject to normal writeback. This is preferable because a
>>>> single bdev wb owner does not identify which cgroup is responsible for a
>>>> mapping shared by buffered bdev users. Flushing it can write back data
>>>> unrelated to the throttled memcg. Dirty accounting, normal writeback, and
>>>> foreign dirty tracking for non-bdev inodes remain unchanged.
>>>
>>> When fixing something, please always describe the userspace-visible
>>> runtime effects.
>>
>> Sorry, I overlooked this aspect.
>>>
>>> I'm thinking probably "unnecessary I/O"?  If so, has this been
>>> quantified/measured?  If not, can the effects be guessed about?
>>
>> We suspect that a large amount of prematurely triggered foreign writeback
>> consumes device writeback bandwidth, thereby degrading the performance of
>> the entire writeback path.
>>
>> On one production machine, the total nr_pages of the foreign-flush works was
>> 630,681,285, or about 2.4 TiB, while the machine had only 400 GiB of memory.
>> Although nr_pages does not represent the number of dirty pages that will
>> ultimately be written back, the actual number of pages written back may
>> still be very large because the corresponding wb was continuously
>> accumulating dirty pages.
> 
> As Tejun also wrote, it would be good to really quantify these effects -
> best by demonstrating how this change improved writeback behavior (and we
> are not so much interested in the number of submitted writeback works but
> more in the observed load on the storage or completion of some workload).
> The problem with your solution is that it has its downsided as well
> (possible stalls in dirty throttling) so it should be very clear what are
> we gaining to outweight this risk.
> 
> In principle I agree that a lot of submitted writeback works can degrade
> writeback behavior - a lot of workloads has some set of folios that is
> practically permanently dirty and if you are pushing writeback too much, we
> will be just writing these folios over and over again without useful work
> really done. Also if there's comparably small amount of folios dirty, the
> IO resulting from writeback is generally less efficient. But how severe
> these effects can get in case you describe is not clear...

Thanks for the suggestion. Following your point about repeatedly writing hot
pages, I tested buffered overwrites alongside a concurrent direct-I/O workload.
Skipping bdev tracking reduced actual writes to the overwritten file by 69.6%
and the concurrent workload's completion time by 14.4%.

I also reproduced the downside you described: a metadata-heavy workload
completed 31-33% more slowly with the patch, including final sync. So these
results do not establish that blanket skipping is a net improvement.

I've sent the detailed test setup and results in a separate reply.

https://lore.kernel.org/cgroups/apnYouiUINQR-98e@slm.duckdns.org/T/#mdc87094720abce0162c14cdf8f3d0ab218eea686
> 
> 								Honza

Thanks,
-- 
Julian Sun <sunjunchao@bytedance.com>