[PATCH v9 3/6] mm: add folio_mark_dropbehind() to mark a folio dropbehind and account it

[Mike Snitzer <snitzer@kernel.org>]

A folio allocated by an RWF_DONTCACHE write is marked dropbehind before
it is dirtied, so folio_account_dirtied() adds it to WB_DONTCACHE_DIRTY
and folio_clear_dirty_for_io() or folio_account_cleaned() subtracts it
when it is cleaned.  Nothing marks a folio that already exists.  A
caller that sets PG_dropbehind on a folio that is already dirty
unbalances the counter: the clean subtracts what was never added, and
once the counter goes negative, wb_stat_sum() reads it as zero and the
DONTCACHE writeback kick stops writing anything back for every writer
on that bdi_writeback.

Add folio_mark_dropbehind(), the inverse of the clear in
__filemap_get_folio_mpol(): with the folio locked, mark it and, if it
has been counted as dirty, add it to WB_DONTCACHE_DIRTY.  Neither
PG_dirty nor the dirty tag alone says that the folio is counted.
filemap_dirty_folio() sets PG_dirty before folio_account_dirtied()
counts the folio, under the i_pages lock and without the folio lock.  A
folio marked between the two would be counted twice.
__folio_cancel_dirty() uncounts a folio and clears PG_dirty, but leaves
the tag until writeback or truncation clears it.  A folio marked with
that stale tag would be counted and never uncounted.
folio_account_dirtied() sets the tag in the same i_pages section that
counts the folio, and every path that uncounts a dirty folio holds the
folio lock.  So set the flag and test both under the i_pages lock, and
count the folio only when both are set.  A dirtier not yet counting the
folio finds the flag set and counts it itself.  Under that lock the wb
is inode_to_wb(),
as in folio_account_dirtied().  A concurrent lookup without
FGP_DONTCACHE that clears the flag again subtracts what this added.

The name follows folio_set_dirty() and folio_mark_dirty(): the first
only sets PG_dirty, the second also accounts the dirty folio.
folio_set_dropbehind() likewise only sets PG_dropbehind, and
folio_mark_dropbehind() also accounts it in WB_DONTCACHE_DIRTY.

A later commit uses it in NFSD to drop a page shared by two misaligned
WRITEs once both have written it.

Signed-off-by: Mike Snitzer <snitzer@kernel.org>
---

Notes:
    Notes on the v8 review questions, with the code that answers each:
    
    - Counting rule.  The same rule as filemap_clear_dropbehind() in
      1/6:  folio_mark_dropbehind() counts the folio only when
      folio_test_dirty() and PAGECACHE_TAG_DIRTY are both true.  The
      test is under the folio lock (VM_WARN_ON_ONCE_FOLIO checks it) and
      the i_pages lock.  A stale tag on a clean folio does not add a
      count.
    - Double count.  A folio that has PG_dropbehind returns at the first
      test, before it changes the count.
    - No writeback.  A mapping without writeback gets only the flag.
      folio_account_dirtied() does not count such a folio either
      (mapping_can_writeback()).

 include/linux/pagemap.h |  1 +
 mm/filemap.c            | 42 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 43 insertions(+)

diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h
index 0adfa6605653d..3012bd95812a1 100644
--- a/include/linux/pagemap.h
+++ b/include/linux/pagemap.h
@@ -1310,6 +1310,7 @@ void end_page_writeback(struct page *page);
 void folio_end_writeback(struct folio *folio);
 void folio_end_writeback_no_dropbehind(struct folio *folio);
 void folio_end_dropbehind(struct folio *folio);
+void folio_mark_dropbehind(struct folio *folio);
 void folio_wait_stable(struct folio *folio);
 void __folio_mark_dirty(struct folio *folio, struct address_space *, int warn);
 void folio_account_cleaned(struct folio *folio, struct bdi_writeback *wb);
diff --git a/mm/filemap.c b/mm/filemap.c
index 5176c4125a4db..d0e4d1788fd40 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -1644,6 +1644,48 @@ void folio_end_dropbehind(struct folio *folio)
 }
 EXPORT_SYMBOL_GPL(folio_end_dropbehind);
 
+/**
+ * folio_mark_dropbehind - mark an existing folio dropbehind
+ * @folio: the folio, locked by the caller
+ *
+ * Mark @folio so that the writeback that cleans it drops it.  A folio
+ * already counted as dirty is added to WB_DONTCACHE_DIRTY, as
+ * folio_account_dirtied() would have added it had it been marked when
+ * dirtied, so that the clean that subtracts it leaves the counter balanced.
+ *
+ * folio_account_dirtied() counts a folio under the i_pages lock, in the
+ * section that sets its dirty tag, after PG_dirty has been set without the
+ * folio lock.  Every path that uncounts a dirty folio holds the folio lock.
+ * So with the folio locked and the i_pages lock held, the folio is counted
+ * when both PG_dirty and the dirty tag are set.  PG_dirty alone is not
+ * enough: a dirtier past folio_test_set_dirty() but not yet counting finds
+ * the flag set and adds the folio itself.  The tag alone is not enough
+ * either: __folio_cancel_dirty() uncounts a folio and clears PG_dirty, but
+ * leaves the tag until writeback or truncation clears it.
+ */
+void folio_mark_dropbehind(struct folio *folio)
+{
+	struct address_space *mapping = folio->mapping;
+	unsigned long flags;
+
+	VM_WARN_ON_ONCE_FOLIO(!folio_test_locked(folio), folio);
+
+	if (folio_test_dropbehind(folio))
+		return;
+	if (!mapping || !mapping_can_writeback(mapping)) {
+		folio_set_dropbehind(folio);
+		return;
+	}
+	xa_lock_irqsave(&mapping->i_pages, flags);
+	folio_set_dropbehind(folio);
+	if (folio_test_dirty(folio) &&
+	    xa_get_mark(&mapping->i_pages, folio->index, PAGECACHE_TAG_DIRTY))
+		wb_stat_mod(inode_to_wb(mapping->host), WB_DONTCACHE_DIRTY,
+			    folio_nr_pages(folio));
+	xa_unlock_irqrestore(&mapping->i_pages, flags);
+}
+EXPORT_SYMBOL_GPL(folio_mark_dropbehind);
+
 /**
  * folio_end_writeback_no_dropbehind - End writeback against a folio.
  * @folio: The folio.
-- 
2.52.0