Re: [RFC] madvise: best-effort deferred writeback for shared file mappings

["David Hildenbrand (Arm)" <david@kernel.org>]

On 9/30/26 12:27, 天狼 wrote:
> Hi David, Lorenzo,
>

Hi,

>> You really need an actual file on disk. Shared memory / shmem / memfd is
>> not sufficient?
> 
> Yes, an ordinary disk-backed file is the intended output, and subsequent
> consumers access it through mmap.
> 
> Shared memory can preserve state across a process crash if the shared
> object remains alive. However, using shmem/memfd would introduce a separate
> staging and transfer step to put the data into the final file. Building
> directly in a file-backed MAP_SHARED mapping lets us preserve the latest
> userspace writes across process crashes and retain the same file-cache
> pages for subsequent consumers.

Why do you really need the data on a file in disk?

What you could do is, load it once from the file into shmem, then let everybody
work on shmem, and have some background thread that periodically writes the
shmem content out to the real file on disk.

Then, you could completely control when I/O would happen.

[...]

>> It'd be problematic to effectively corrupt the page cache by saying "hey
>> this is dirty but just clear dirty state and pretend this is what the
>> disk has".
> 
> To clarify, the proposal does not involve clearing dirty state or
> pretending that memory and disk contents match. The pages would remain
> dirty and continue to count toward the existing dirty-page limits.
> 

IIUC, vmscan would still write out the folio if dirty, so I'd assume that at
least memory reclaim would not be affected, only background writeback.

> The requested hint only expresses that the application expects to modify
> the data again soon and would prefer background writeback to happen later.
> 
>> Delaying writeback could interfere with the writeback balancing logic.
> 
> I understand that concern. The hint should not exempt the application from
> dirty-page limits or throttling. If writeback is needed for balancing,
> memory pressure, or explicit synchronization, the kernel should override
> the hint.
> 
> The intended benefit is to avoid writing intermediate versions when the
> kernel has room to defer that work. There is no requirement to prevent
> writeback until construction finishes.
> 
> madvise is not the only option, deferring writeback for the entire file
> would also be acceptable.
> 
> Could such an advisory preference be accommodated within the existing
> writeback balancing policy while retaining its limits and fairness?
When using MAP_SHARED with disk-based files for VM memory, people ran into
similar problems: the VM will constantly dirty many folios and background
writeback will just permanently write these out and wear storage. For VMs you
really only want the memory persisted in the file once you e.g., migrate the VM.
Essentially, once the VM is paused.

One of the reasons why we tell people to not use that combination and use
shmem/hugetlb instead. I do wonder whether there is some similarity ... we don't
want brackground writeback as long as there is heavy activity on these files.

-- 
Cheers,

David