[RFC PATCH 0/5] mm: sub-folio dirty tracking for PTE-mapped mmap writes

[RFC PATCH 0/5] mm: sub-folio dirty tracking for PTE-mapped mmap writes

[Kiryl Shutsemau]

From: "Kiryl Shutsemau (Meta)" <kas@kernel.org>

A store through a shared file mapping dirties the whole folio. With large
page cache folios that turns a 4K store into 2M of writeback: one dirty
bit per folio, and writeback has no way to know which part changed.

XFS already knows better. iomap tracks dirty state per block and
iomap_writeback_folio() submits only the dirty ranges, and the buffered
write path sets just the range it copied. Only the mmap path throws that
away, because iomap_dirty_folio() covers the whole folio.

Narrowing the dirtying at page_mkwrite() time does not work on its own:
set_pte_range() batch-maps a whole folio writable on the first shared
write fault, so the stores that follow never fault and never reach the
filesystem.

So harvest the hardware instead. folio_clear_dirty_for_io() already calls
folio_mkclean(), whose rmap walk reads pte_dirty() for every entry of the
folio and throws it away. Those bits are the only record of which parts
of a large folio were written through a mapping. Collect them there and
hand the filesystem the runs that were dirty, through a new
a_ops->dirty_folio_range().

All of this is about PTE-mapped folios. A PMD-mapped folio has a single
dirty bit for the 2M it maps, so there is nothing finer to harvest, and
it keeps writing back whole. Keeping shared write faults off PMDs is a
separate patch and not part of this posting.

On a 512M file in 2M folios on XFS, storing one byte per folio and
calling msync() wrote 512M before and writes 1M after, with identical
minor fault counts.

Not addressed here:

 - Dirty accounting stays folio-granular. A 4K store still counts as 2M
   against dirty_ratio and balance_dirty_pages().
 - iomap_page_mkwrite() still allocates blocks for the whole folio.
 - Filesystems without per-block dirty state see no change.

Kiryl Shutsemau (Meta) (5):
  mm: let folio_mkclean() report which pages had dirty PTEs
  mm: add a_ops->dirty_folio_range() and use the mkclean dirty harvest
  mm: keep the mmap dirty range down to the faulting page
  iomap: narrow page_mkwrite() dirtying to the faulting page
  xfs: track mmap dirty state per block

 fs/iomap/buffered-io.c | 37 +++++++++++++-----
 fs/xfs/xfs_aops.c      |  2 +-
 include/linux/fs.h     |  3 ++
 include/linux/iomap.h  |  2 +
 include/linux/mm.h     |  1 +
 include/linux/rmap.h   |  7 ++++
 mm/memory.c            | 58 ++++++++++++++++++++++++++--
 mm/page-writeback.c    | 87 +++++++++++++++++++++++++++++++++++++++---
 mm/rmap.c              | 56 +++++++++++++++++++++------
 9 files changed, 222 insertions(+), 31 deletions(-)


base-commit: 0a0d1d55dad570724bf8c7ea83409639cfb4be9b
-- 
2.54.0

[RFC PATCH 1/5] mm: let folio_mkclean() report which pages had dirty PTEs

[Kiryl Shutsemau]

From: "Kiryl Shutsemau (Meta)" <kas@kernel.org>

folio_mkclean() walks every mapping of a folio and clears the dirty and
write bits of each page table entry. It has the per-entry dirty bit in
hand while doing that, but only counts how many entries it cleaned.

For a large folio those bits are the only record of which parts of the
folio were written through a mapping. Everything downstream has to
assume the whole folio changed because that information is dropped here.

Add folio_mkclean_dirtymap(), which takes a bitmap and sets a bit for every
page of the folio whose entry was dirty. folio_mkclean() becomes a
wrapper that passes no bitmap, so there is no change in behaviour yet.

A PMD entry has one dirty bit for the whole folio, so a PMD-mapped folio
reports all of its pages as dirty. That is the best that can be done:
the hardware does not track anything finer for a PMD.

Assisted-by: Claude:claude-opus-5
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
---
 include/linux/rmap.h |  7 ++++++
 mm/rmap.c            | 56 +++++++++++++++++++++++++++++++++++---------
 2 files changed, 52 insertions(+), 11 deletions(-)

diff --git a/include/linux/rmap.h b/include/linux/rmap.h
index 8dc0871e5f00..6fc0a6020252 100644
--- a/include/linux/rmap.h
+++ b/include/linux/rmap.h
@@ -927,6 +927,7 @@ unsigned long page_address_in_vma(const struct folio *folio,
  * returns the number of cleaned PTEs.
  */
 int folio_mkclean(struct folio *);
+int folio_mkclean_dirtymap(struct folio *folio, unsigned long *dirty_map);
 
 int mapping_wrprotect_range(struct address_space *mapping, pgoff_t pgoff,
 		unsigned long pfn, unsigned long nr_pages);
@@ -990,6 +991,12 @@ static inline int folio_mkclean(struct folio *folio)
 {
 	return 0;
 }
+
+static inline int folio_mkclean_dirtymap(struct folio *folio,
+					 unsigned long *dirty_map)
+{
+	return 0;
+}
 #endif	/* CONFIG_MMU */
 
 #endif	/* _LINUX_RMAP_H */
diff --git a/mm/rmap.c b/mm/rmap.c
index 1f72d279ba68..aaf45ac79fa8 100644
--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -1100,12 +1100,18 @@ int folio_referenced(struct folio *folio, int is_locked,
 	return rwc.contended ? -1 : pra.referenced;
 }
 
-static int page_vma_mkclean_one(struct page_vma_mapped_walk *pvmw)
+struct mkclean_state {
+	unsigned long *dirty_map;
+	int cleaned;
+};
+
+static int page_vma_mkclean_one(struct page_vma_mapped_walk *pvmw,
+				unsigned long *dirty_map)
 {
-	int cleaned = 0;
 	struct vm_area_struct *vma = pvmw->vma;
-	struct mmu_notifier_range range;
 	unsigned long address = pvmw->address;
+	struct mmu_notifier_range range;
+	int cleaned = 0;
 
 	/*
 	 * We have to assume the worse case ie pmd for invalidation. Note that
@@ -1134,6 +1140,15 @@ static int page_vma_mkclean_one(struct page_vma_mapped_walk *pvmw)
 			if (!pte_dirty(entry) && !pte_write(entry))
 				continue;
 
+			if (dirty_map && pte_dirty(entry)) {
+				pgoff_t idx = linear_page_index(vma, address) -
+					      pvmw->pgoff;
+
+				/* The walk only visits pages of this folio */
+				VM_WARN_ON_ONCE(idx >= pvmw->nr_pages);
+				__set_bit(idx, dirty_map);
+			}
+
 			flush_cache_page(vma, address, pte_pfn(entry));
 			entry = ptep_clear_flush(vma, address, pte);
 			entry = pte_wrprotect(entry);
@@ -1155,6 +1170,9 @@ static int page_vma_mkclean_one(struct page_vma_mapped_walk *pvmw)
 			if (!pmd_dirty(entry) && !pmd_write(entry))
 				continue;
 
+			if (dirty_map && pmd_dirty(entry))
+				bitmap_set(dirty_map, 0, pvmw->nr_pages);
+
 			flush_cache_range(vma, address,
 					  address + HPAGE_PMD_SIZE);
 			entry = pmdp_invalidate(vma, address, pmd);
@@ -1181,9 +1199,9 @@ static bool page_mkclean_one(struct folio *folio, struct vm_area_struct *vma,
 			     unsigned long address, void *arg)
 {
 	DEFINE_FOLIO_VMA_WALK(pvmw, folio, vma, address, PVMW_SYNC);
-	int *cleaned = arg;
+	struct mkclean_state *state = arg;
 
-	*cleaned += page_vma_mkclean_one(&pvmw);
+	state->cleaned += page_vma_mkclean_one(&pvmw, state->dirty_map);
 
 	return true;
 }
@@ -1196,12 +1214,23 @@ static bool invalid_mkclean_vma(struct vm_area_struct *vma, void *arg)
 	return true;
 }
 
-int folio_mkclean(struct folio *folio)
+/**
+ * folio_mkclean_dirtymap - Write-protect a folio and report what was dirty.
+ * @folio: The folio to clean.
+ * @dirty_map: Bitmap of at least folio_nr_pages(@folio) bits, or NULL.
+ *
+ * Write-protects and cleans every mapping of @folio.  With @dirty_map, sets a
+ * bit for each page whose entry was dirty; a PMD-mapped folio has one dirty
+ * bit for all of it, so every page is reported.
+ *
+ * Return: the number of page table entries cleaned.
+ */
+int folio_mkclean_dirtymap(struct folio *folio, unsigned long *dirty_map)
 {
-	int cleaned = 0;
+	struct mkclean_state state = { .dirty_map = dirty_map };
 	struct address_space *mapping;
 	struct rmap_walk_control rwc = {
-		.arg = (void *)&cleaned,
+		.arg = (void *)&state,
 		.rmap_one = page_mkclean_one,
 		.invalid_vma = invalid_mkclean_vma,
 	};
@@ -1217,7 +1246,12 @@ int folio_mkclean(struct folio *folio)
 
 	rmap_walk(folio, &rwc);
 
-	return cleaned;
+	return state.cleaned;
+}
+
+int folio_mkclean(struct folio *folio)
+{
+	return folio_mkclean_dirtymap(folio, NULL);
 }
 EXPORT_SYMBOL_GPL(folio_mkclean);
 
@@ -1241,7 +1275,7 @@ static bool mapping_wrprotect_range_one(struct folio *folio,
 		.flags		= PVMW_SYNC,
 	};
 
-	state->cleaned += page_vma_mkclean_one(&pvmw);
+	state->cleaned += page_vma_mkclean_one(&pvmw, NULL);
 
 	return true;
 }
@@ -1324,7 +1358,7 @@ int pfn_mkclean_range(unsigned long pfn, unsigned long nr_pages, pgoff_t pgoff,
 	pvmw.address = vma_address(vma, pgoff, nr_pages);
 	VM_BUG_ON_VMA(pvmw.address == -EFAULT, vma);
 
-	return page_vma_mkclean_one(&pvmw);
+	return page_vma_mkclean_one(&pvmw, NULL);
 }
 
 static void __folio_mod_stat(struct folio *folio, int nr, int nr_pmdmapped)
-- 
2.54.0

[RFC PATCH 2/5] mm: add a_ops->dirty_folio_range() and use the mkclean dirty harvest

[Kiryl Shutsemau]

From: "Kiryl Shutsemau (Meta)" <kas@kernel.org>

Every way of dirtying part of a folio through a mapping ends up at
folio_mark_dirty(), which has no way to say which part changed, so
a_ops->dirty_folio() dirties all of it. A filesystem that tracks dirty
state per block then writes back the whole folio for a single stored
byte.

Add a_ops->dirty_folio_range() and folio_mark_dirty_range() to pass the
range on. The new operation can express everything a_ops->dirty_folio()
can, so folio_mark_dirty() goes through it with a range covering the
folio and a filesystem needs only one of the two. Filesystems without it
dirty the whole folio.

Use it in folio_clear_dirty_for_io(), where the page table dirty bits
were being turned into a whole-folio dirty. It now collects them with
folio_mkclean_dirtymap() and hands the filesystem the runs that were
dirty. A folio that is not already dirty still dirties whole, because
the clean to dirty transition needs the accounting in folio_mark_dirty().

Assisted-by: Claude:claude-opus-5
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
---
 include/linux/fs.h  |  3 ++
 include/linux/mm.h  |  1 +
 mm/page-writeback.c | 87 +++++++++++++++++++++++++++++++++++++++++----
 3 files changed, 85 insertions(+), 6 deletions(-)

diff --git a/include/linux/fs.h b/include/linux/fs.h
index 072d8cd09a0b..1d98b6c0b880 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -406,6 +406,9 @@ struct address_space_operations {
 
 	/* Mark a folio dirty.  Return true if this dirtied it */
 	bool (*dirty_folio)(struct address_space *, struct folio *);
+	/* Mark [off, off + len) of a folio dirty */
+	bool (*dirty_folio_range)(struct address_space *mapping,
+				  struct folio *folio, size_t off, size_t len);
 
 	void (*readahead)(struct readahead_control *);
 
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 87feaa5a2b78..7628262c17e1 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -3337,6 +3337,7 @@ struct kvec;
 struct page *get_dump_page(unsigned long addr, int *locked);
 
 bool folio_mark_dirty(struct folio *folio);
+bool folio_mark_dirty_range(struct folio *folio, size_t off, size_t len);
 bool folio_mark_dirty_lock(struct folio *folio);
 bool set_page_dirty(struct page *page);
 int set_page_dirty_lock(struct page *page);
diff --git a/mm/page-writeback.c b/mm/page-writeback.c
index 6c9c7ba89b8a..39b54c25a9fa 100644
--- a/mm/page-writeback.c
+++ b/mm/page-writeback.c
@@ -2751,6 +2751,21 @@ bool folio_redirty_for_writepage(struct writeback_control *wbc,
 }
 EXPORT_SYMBOL(folio_redirty_for_writepage);
 
+/*
+ * Hand a dirtied range of @folio to the filesystem.  ->dirty_folio_range() can
+ * express everything ->dirty_folio() can, so a filesystem that implements it
+ * does not need both, and a whole-folio dirty comes through here as a range
+ * covering the folio.
+ */
+static bool mapping_dirty_range(struct address_space *mapping,
+				struct folio *folio, size_t off, size_t len)
+{
+	if (!mapping->a_ops->dirty_folio_range)
+		return mapping->a_ops->dirty_folio(mapping, folio);
+
+	return mapping->a_ops->dirty_folio_range(mapping, folio, off, len);
+}
+
 /**
  * folio_mark_dirty - Mark a folio as being modified.
  * @folio: The folio.
@@ -2782,13 +2797,39 @@ bool folio_mark_dirty(struct folio *folio)
 		 */
 		if (folio_test_reclaim(folio))
 			folio_clear_reclaim(folio);
-		return mapping->a_ops->dirty_folio(mapping, folio);
+		return mapping_dirty_range(mapping, folio, 0,
+					   folio_size(folio));
 	}
 
 	return noop_dirty_folio(mapping, folio);
 }
 EXPORT_SYMBOL(folio_mark_dirty);
 
+/**
+ * folio_mark_dirty_range - Mark part of a folio as being modified.
+ * @folio: The folio.
+ * @off: Offset of the modified range within the folio.
+ * @len: Length of the modified range.
+ *
+ * Like folio_mark_dirty(), but tells a filesystem that tracks dirty state per
+ * block that only [@off, @off + @len) changed, so writeback can skip the rest
+ * of the folio.  Filesystems without that tracking dirty the whole folio.
+ *
+ * Return: True if the folio was newly dirtied, false if it was already dirty.
+ */
+bool folio_mark_dirty_range(struct folio *folio, size_t off, size_t len)
+{
+	struct address_space *mapping = folio_mapping(folio);
+
+	if (likely(mapping)) {
+		if (folio_test_reclaim(folio))
+			folio_clear_reclaim(folio);
+		return mapping_dirty_range(mapping, folio, off, len);
+	}
+
+	return noop_dirty_folio(mapping, folio);
+}
+
 /*
  * folio_mark_dirty() is racy if the caller has no reference against
  * folio->mapping->host, and if the folio is unlocked.  This is because another
@@ -2844,6 +2885,41 @@ void __folio_cancel_dirty(struct folio *folio)
 }
 EXPORT_SYMBOL(__folio_cancel_dirty);
 
+/*
+ * Write-protect every mapping of @folio and hand the filesystem the parts that
+ * were dirty in a page table.
+ *
+ * Without ->dirty_folio_range() there is nowhere to put per-block state, so
+ * any PTE dirty bit dirties the whole folio.  Same when the folio is not
+ * already dirty, because then the dirty transition needs the full accounting
+ * in folio_mark_dirty(), and for a folio too large for the bitmap, which the
+ * page cache does not make.
+ */
+static void folio_mkclean_for_io(struct folio *folio,
+				 struct address_space *mapping)
+{
+	DECLARE_BITMAP(map, 1UL << MAX_PAGECACHE_ORDER);
+	unsigned int nr = folio_nr_pages(folio);
+	unsigned int start, end;
+
+	if (!mapping->a_ops->dirty_folio_range || !folio_test_dirty(folio) ||
+	    WARN_ON_ONCE(nr > (1UL << MAX_PAGECACHE_ORDER))) {
+		if (folio_mkclean(folio))
+			folio_mark_dirty(folio);
+		return;
+	}
+
+	bitmap_zero(map, nr);
+	if (!folio_mkclean_dirtymap(folio, map))
+		return;
+
+	for_each_set_bitrange(start, end, map, nr) {
+		mapping_dirty_range(mapping, folio,
+				    (size_t)start << PAGE_SHIFT,
+				    (size_t)(end - start) << PAGE_SHIFT);
+	}
+}
+
 /*
  * Clear a folio's dirty flag, while caring for dirty memory accounting.
  * Returns true if the folio was previously dirty.
@@ -2875,9 +2951,9 @@ bool folio_clear_dirty_for_io(struct folio *folio)
 		 *
 		 * We use this sequence to make sure that
 		 *  (a) we account for dirty stats properly
-		 *  (b) we tell the low-level filesystem to
-		 *      mark the whole folio dirty if it was
-		 *      dirty in a pagetable. Only to then
+		 *  (b) we tell the low-level filesystem which
+		 *      parts of the folio were dirty in a
+		 *      pagetable. Only to then
 		 *  (c) clean the folio again and return 1 to
 		 *      cause the writeback.
 		 *
@@ -2895,8 +2971,7 @@ bool folio_clear_dirty_for_io(struct folio *folio)
 		 * as a serialization point for all the different
 		 * threads doing their things.
 		 */
-		if (folio_mkclean(folio))
-			folio_mark_dirty(folio);
+		folio_mkclean_for_io(folio, mapping);
 		/*
 		 * We carefully synchronise fault handlers against
 		 * installing a dirty pte and marking the folio dirty
-- 
2.54.0

[RFC PATCH 3/5] mm: keep the mmap dirty range down to the faulting page

[Kiryl Shutsemau]

From: "Kiryl Shutsemau (Meta)" <kas@kernel.org>

Two places in the fault path dirty a whole folio when a single page was
written.

fault_dirty_shared_page() marks the folio dirty after a write fault on a
shared mapping. Only the page the fault was for has been stored to, so
pass that range instead. A PMD-mapped folio is still dirtied whole,
since a PMD has one dirty bit and there is nothing finer to report.

set_pte_range() marks a whole batch of entries dirty on a write fault,
which destroys the per-page dirty information before the pages have been
written to. Leave the batch clean for a shared mapping of a filesystem
that implements a_ops->dirty_folio_range(), and let the hardware set the
bit on the first store to each page. Anywhere else nothing reads those
bits, and on architectures without a hardware dirty bit a clean batch
costs a fault per page for nothing. That covers shmem, which has no
dirty state below the folio.

The page the fault was for is dirtied right away. It is about to be
stored to, so leaving it clean would only move the work to a second page
table walk or fault.

Assisted-by: Claude:claude-opus-5
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
---
 mm/memory.c | 58 +++++++++++++++++++++++++++++++++++++++++++++++++----
 1 file changed, 54 insertions(+), 4 deletions(-)

diff --git a/mm/memory.c b/mm/memory.c
index 8da0f945141b..27a059e0c016 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -3778,10 +3778,19 @@ static vm_fault_t fault_dirty_shared_page(struct vm_fault *vmf)
 	struct vm_area_struct *vma = vmf->vma;
 	struct address_space *mapping;
 	struct folio *folio = page_folio(vmf->page);
+	size_t off = folio_page_idx(folio, vmf->page) << PAGE_SHIFT;
 	bool dirtied;
 	bool page_mkwrite = vma->vm_ops && vma->vm_ops->page_mkwrite;
 
-	dirtied = folio_mark_dirty(folio);
+	/*
+	 * A PMD entry has one dirty bit for the whole folio, so there is no
+	 * finer information to pass on. A PTE-mapped folio only has the page
+	 * the fault was for dirtied so far.
+	 */
+	if (pmd_trans_huge(pmdp_get_lockless(vmf->pmd)))
+		dirtied = folio_mark_dirty(folio);
+	else
+		dirtied = folio_mark_dirty_range(folio, off, PAGE_SIZE);
 	VM_BUG_ON_FOLIO(folio_test_anon(folio), folio);
 	/*
 	 * Take a local copy of the address_space - folio.mapping may be zeroed
@@ -5625,6 +5634,31 @@ vm_fault_t do_set_pmd(struct vm_fault *vmf, struct folio *folio, struct page *pa
 }
 #endif
 
+/*
+ * May the whole batch be marked dirty?
+ *
+ * Only a shared mapping of a filesystem that tracks dirty state per block says
+ * no. There the page table dirty bits are the only record of which parts of a
+ * large folio were written through the mapping, so pages nobody has stored to
+ * have to stay clean and let the hardware set the bit on the first store.
+ *
+ * Everywhere else nothing ever reads those bits, and on hardware without a
+ * dirty bit leaving them clean costs a fault per page for nothing. That covers
+ * shmem, which has no dirty state below the folio.
+ */
+static bool can_dirty_whole_batch(struct vm_fault *vmf, unsigned int nr,
+				  bool prefault)
+{
+	struct vm_area_struct *vma = vmf->vma;
+
+	if (!(vmf->flags & FAULT_FLAG_WRITE) || prefault || nr == 1)
+		return true;
+	if (!(vma->vm_flags & VM_SHARED))
+		return true;
+
+	return !vma->vm_file->f_mapping->a_ops->dirty_folio_range;
+}
+
 /**
  * set_pte_range - Set a range of PTEs to point to pages in a folio.
  * @vmf: Fault description.
@@ -5639,6 +5673,7 @@ void set_pte_range(struct vm_fault *vmf, struct folio *folio,
 	struct vm_area_struct *vma = vmf->vma;
 	bool write = vmf->flags & FAULT_FLAG_WRITE;
 	bool prefault = !in_range(vmf->address, addr, nr * PAGE_SIZE);
+	bool dirty_batch = can_dirty_whole_batch(vmf, nr, prefault);
 	pte_t entry;
 
 	flush_icache_pages(vma, page, nr);
@@ -5649,10 +5684,13 @@ void set_pte_range(struct vm_fault *vmf, struct folio *folio,
 	else
 		entry = pte_sw_mkyoung(entry);
 
-	if (write)
-		entry = maybe_mkwrite(pte_mkdirty(entry), vma);
-	else if (pte_write(entry) && folio_test_dirty(folio))
+	if (write) {
+		if (dirty_batch)
+			entry = pte_mkdirty(entry);
+		entry = maybe_mkwrite(entry, vma);
+	} else if (pte_write(entry) && folio_test_dirty(folio)) {
 		entry = pte_mkdirty(entry);
+	}
 	if (unlikely(vmf_orig_pte_uffd_wp(vmf)))
 		entry = pte_mkuffd(entry);
 	/* copy-on-write page */
@@ -5665,6 +5703,18 @@ void set_pte_range(struct vm_fault *vmf, struct folio *folio,
 	}
 	set_ptes(vma->vm_mm, addr, vmf->pte, entry, nr);
 
+	/*
+	 * The page the fault was for is about to be stored to, so dirty it
+	 * here rather than leave the store to a second page table walk, or to
+	 * a second fault where the dirty bit is maintained in software.
+	 */
+	if (!dirty_batch) {
+		pte_t *ptep = vmf->pte + ((vmf->address - addr) >> PAGE_SHIFT);
+
+		ptep_set_access_flags(vma, vmf->address, ptep,
+				      pte_mkdirty(ptep_get(ptep)), 1);
+	}
+
 	/* no need to invalidate: a not-present page won't be cached */
 	update_mmu_cache_range(vmf, vma, addr, vmf->pte, nr);
 }
-- 
2.54.0

[RFC PATCH 4/5] iomap: narrow page_mkwrite() dirtying to the faulting page

[Kiryl Shutsemau]

From: "Kiryl Shutsemau (Meta)" <kas@kernel.org>

iomap already tracks dirty state per block and iomap_writeback_folio()
already walks the bitmap and submits only the dirty ranges. The
buffered write path sets just the range it copied. Only the mmap path
sets everything, because iomap_dirty_folio() covers the whole folio.

Add iomap_dirty_folio_range() for the new a_ops->dirty_folio_range(),
and have iomap_page_mkwrite() dirty just the page the fault was for.
Blocks are still allocated for the whole folio; only the dirty range
narrows.

iomap_dirty_folio() becomes a call to it covering the whole folio.

Assisted-by: Claude:claude-opus-5
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
---
 fs/iomap/buffered-io.c | 37 ++++++++++++++++++++++++++++---------
 include/linux/iomap.h  |  2 ++
 2 files changed, 30 insertions(+), 9 deletions(-)

diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c
index 0a5ebfda90f1..8d8a8acab01a 100644
--- a/fs/iomap/buffered-io.c
+++ b/fs/iomap/buffered-io.c
@@ -847,14 +847,18 @@ void iomap_invalidate_folio(struct folio *folio, size_t offset, size_t len)
 }
 EXPORT_SYMBOL_GPL(iomap_invalidate_folio);
 
+bool iomap_dirty_folio_range(struct address_space *mapping, struct folio *folio,
+		size_t off, size_t len)
+{
+	ifs_alloc(mapping->host, folio, 0);
+	iomap_set_range_dirty(folio, off, len);
+	return filemap_dirty_folio(mapping, folio);
+}
+EXPORT_SYMBOL_GPL(iomap_dirty_folio_range);
+
 bool iomap_dirty_folio(struct address_space *mapping, struct folio *folio)
 {
-	struct inode *inode = mapping->host;
-	size_t len = folio_size(folio);
-
-	ifs_alloc(inode, folio, 0);
-	iomap_set_range_dirty(folio, 0, len);
-	return filemap_dirty_folio(mapping, folio);
+	return iomap_dirty_folio_range(mapping, folio, 0, folio_size(folio));
 }
 EXPORT_SYMBOL_GPL(iomap_dirty_folio);
 
@@ -1804,7 +1808,7 @@ iomap_truncate_page(struct inode *inode, loff_t pos, bool *did_zero,
 EXPORT_SYMBOL_GPL(iomap_truncate_page);
 
 static int iomap_folio_mkwrite_iter(struct iomap_iter *iter,
-		struct folio *folio)
+		struct folio *folio, size_t dirty_off, size_t dirty_len)
 {
 	loff_t length = iomap_length(iter);
 	int ret;
@@ -1817,7 +1821,7 @@ static int iomap_folio_mkwrite_iter(struct iomap_iter *iter,
 		block_commit_write(folio, 0, length);
 	} else {
 		WARN_ON_ONCE(!folio_test_uptodate(folio));
-		folio_mark_dirty(folio);
+		folio_mark_dirty_range(folio, dirty_off, dirty_len);
 	}
 
 	return iomap_iter_advance(iter, length);
@@ -1832,6 +1836,7 @@ vm_fault_t iomap_page_mkwrite(struct vm_fault *vmf, const struct iomap_ops *ops,
 		.private	= private,
 	};
 	struct folio *folio = page_folio(vmf->page);
+	size_t dirty_off, dirty_len;
 	ssize_t ret;
 
 	folio_lock(folio);
@@ -1840,8 +1845,22 @@ vm_fault_t iomap_page_mkwrite(struct vm_fault *vmf, const struct iomap_ops *ops,
 		goto out_unlock;
 	iter.pos = folio_pos(folio);
 	iter.len = ret;
+
+	/*
+	 * Blocks are still allocated for the whole folio, but only the page
+	 * the fault was for has been written, so that is all that has to be
+	 * written back.  A racing truncate can leave that page beyond i_size,
+	 * and then there is nothing to dirty.
+	 */
+	dirty_off = folio_page_idx(folio, vmf->page) << PAGE_SHIFT;
+	if (dirty_off < ret)
+		dirty_len = min_t(size_t, PAGE_SIZE, ret - dirty_off);
+	else
+		dirty_len = 0;
+
 	while ((ret = iomap_iter(&iter, ops)) > 0)
-		iter.status = iomap_folio_mkwrite_iter(&iter, folio);
+		iter.status = iomap_folio_mkwrite_iter(&iter, folio,
+						       dirty_off, dirty_len);
 
 	if (ret < 0)
 		goto out_unlock;
diff --git a/include/linux/iomap.h b/include/linux/iomap.h
index bc7ae6327dbf..3cf2ac751409 100644
--- a/include/linux/iomap.h
+++ b/include/linux/iomap.h
@@ -443,6 +443,8 @@ struct folio *iomap_get_folio(struct iomap_iter *iter, loff_t pos, size_t len);
 bool iomap_release_folio(struct folio *folio, gfp_t gfp_flags);
 void iomap_invalidate_folio(struct folio *folio, size_t offset, size_t len);
 bool iomap_dirty_folio(struct address_space *mapping, struct folio *folio);
+bool iomap_dirty_folio_range(struct address_space *mapping, struct folio *folio,
+		size_t off, size_t len);
 void iomap_folio_mark_uptodate(struct folio *folio);
 int iomap_file_unshare(struct inode *inode, loff_t pos, loff_t len,
 		const struct iomap_ops *ops,
-- 
2.54.0

[RFC PATCH 5/5] xfs: track mmap dirty state per block

[Kiryl Shutsemau]

From: "Kiryl Shutsemau (Meta)" <kas@kernel.org>

With a_ops->dirty_folio_range() wired up, a store through a shared
mapping writes back only the block it touched instead of the whole
folio.

a_ops->dirty_folio() goes away with it. folio_mark_dirty() reaches
a_ops->dirty_folio_range() with a range covering the folio, which is
what iomap_dirty_folio() does.

On a 512M file held in 2M page cache folios, storing one byte per folio
and then calling msync() wrote 512M of the file before this series and
writes 1M after it. Minor fault counts are the same either way, in both
the PTE-mapped and the batch-mapped case.

A PMD-mapped folio still writes back whole. A PMD carries one dirty bit
for the 2M it maps, so there is no per-block state to recover.

Assisted-by: Claude:claude-opus-5
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
---
 fs/xfs/xfs_aops.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/fs/xfs/xfs_aops.c b/fs/xfs/xfs_aops.c
index 74a6089abadf..600e7de2e599 100644
--- a/fs/xfs/xfs_aops.c
+++ b/fs/xfs/xfs_aops.c
@@ -863,7 +863,7 @@ const struct address_space_operations xfs_address_space_operations = {
 	.read_folio		= xfs_vm_read_folio,
 	.readahead		= xfs_vm_readahead,
 	.writepages		= xfs_vm_writepages,
-	.dirty_folio		= iomap_dirty_folio,
+	.dirty_folio_range	= iomap_dirty_folio_range,
 	.release_folio		= iomap_release_folio,
 	.invalidate_folio	= iomap_invalidate_folio,
 	.bmap			= xfs_vm_bmap,
-- 
2.54.0

Re: [RFC PATCH 0/5] mm: sub-folio dirty tracking for PTE-mapped mmap writes

[Pedro Falcato]

On Thu, Sep 03, 2026 at 07:29:38PM +0100, Kiryl Shutsemau wrote:
> From: "Kiryl Shutsemau (Meta)" <kas@kernel.org>
> 
> A store through a shared file mapping dirties the whole folio. With large
> page cache folios that turns a 4K store into 2M of writeback: one dirty
> bit per folio, and writeback has no way to know which part changed.
> 
> XFS already knows better. iomap tracks dirty state per block and
> iomap_writeback_folio() submits only the dirty ranges, and the buffered
> write path sets just the range it copied. Only the mmap path throws that
> away, because iomap_dirty_folio() covers the whole folio.
> 

Hmm, I believe Willy already had some patches; I don't know what came of
them (I think he's out for now).

> Narrowing the dirtying at page_mkwrite() time does not work on its own:
> set_pte_range() batch-maps a whole folio writable on the first shared
> write fault, so the stores that follow never fault and never reach the
> filesystem.

Help me out here: in which case does this happen? page fault handling is a
mess... I think page_mkwrite is always called, no? in do_shared_fault().

> 
> So harvest the hardware instead. folio_clear_dirty_for_io() already calls
> folio_mkclean(), whose rmap walk reads pte_dirty() for every entry of the
> folio and throws it away. Those bits are the only record of which parts
> of a large folio were written through a mapping. Collect them there and
> hand the filesystem the runs that were dirty, through a new
> a_ops->dirty_folio_range().

I think this goes against what we're trying to pull off. We want a single
method of dirtying folios, not 3.

Ideally(tm) we would have a single mkwrite interface.

> 
> All of this is about PTE-mapped folios. A PMD-mapped folio has a single
> dirty bit for the 2M it maps, so there is nothing finer to harvest, and
> it keeps writing back whole. Keeping shared write faults off PMDs is a
> separate patch and not part of this posting.
> 
> On a 512M file in 2M folios on XFS, storing one byte per folio and
> calling msync() wrote 512M before and writes 1M after, with identical
> minor fault counts.

But the effects are definitely nice though :)

> 
> Not addressed here:
> 
>  - Dirty accounting stays folio-granular. A 4K store still counts as 2M
>    against dirty_ratio and balance_dirty_pages().

That is correct; I don't think doing subpage accounting makes any sense,
the whole folio is still dirty, and for reclaim purposes can't be thrown
away (unless you added gnarly split logic for file folios as well).

>  - iomap_page_mkwrite() still allocates blocks for the whole folio.
>  - Filesystems without per-block dirty state see no change.
> 
> Kiryl Shutsemau (Meta) (5):
>   mm: let folio_mkclean() report which pages had dirty PTEs
>   mm: add a_ops->dirty_folio_range() and use the mkclean dirty harvest
>   mm: keep the mmap dirty range down to the faulting page
>   iomap: narrow page_mkwrite() dirtying to the faulting page
>   xfs: track mmap dirty state per block
> 
>  fs/iomap/buffered-io.c | 37 +++++++++++++-----
>  fs/xfs/xfs_aops.c      |  2 +-
>  include/linux/fs.h     |  3 ++
>  include/linux/iomap.h  |  2 +
>  include/linux/mm.h     |  1 +
>  include/linux/rmap.h   |  7 ++++
>  mm/memory.c            | 58 ++++++++++++++++++++++++++--
>  mm/page-writeback.c    | 87 +++++++++++++++++++++++++++++++++++++++---
>  mm/rmap.c              | 56 +++++++++++++++++++++------
>  9 files changed, 222 insertions(+), 31 deletions(-)
> 
> 
> base-commit: 0a0d1d55dad570724bf8c7ea83409639cfb4be9b
> -- 
> 2.54.0
> 

-- 
Pedro

Re: [RFC PATCH 0/5] mm: sub-folio dirty tracking for PTE-mapped mmap writes

[Kiryl Shutsemau]

On Thu, Sep 03, 2026 at 08:55:36PM +0100, Pedro Falcato wrote:
> On Thu, Sep 03, 2026 at 07:29:38PM +0100, Kiryl Shutsemau wrote:
> > Narrowing the dirtying at page_mkwrite() time does not work on its own:
> > set_pte_range() batch-maps a whole folio writable on the first shared
> > write fault, so the stores that follow never fault and never reach the
> > filesystem.
> 
> Help me out here: in which case does this happen? page fault handling is a
> mess... I think page_mkwrite is always called, no? in do_shared_fault().

It is always called, but once per folio, not once per page. It is the
filesystem's chance to preallocate whatever it needs to track dirty
state for the *folio*.

Later finish_fault() maps the folio. It tries to map it fully when it
can, so a write fault creates up to 512 writable PTEs on x86. And we
really do need to map the folio fully whenever we can. Otherwise we
significantly undercount mlocked memory. See commit 19773df031bc
("mm/fault: try to map the entire file folio in finish_fault()").

And once the whole folio is mapped with writable PTEs we cannot narrow
the dirtying to a subset of pages. Any of them can turn dirty at any
time, with nothing to record it.

-- 
  Kiryl Shutsemau / Kirill A. Shutemov