[PATCH v3 00/14] Remove PG_private by using page/folio->private checks instead

[PATCH v3 00/14] Remove PG_private by using page/folio->private checks instead

[Zi Yan]

Hi all,

This patchset removes PG_private to make space for upcoming PG_folio
(reserved as __PG_folio) for identifying pages from a folio (more details
in Note below). Instead of checking PG_private, all code is changed to
check page/folio->private != NULL instead.

MM people are cc'd on all patches and subsystem people are cc'd on the
cover letter and corresponding patches.

Patch 6 is picked up separately in f2fs tree, but since mm-new does not
have it yet, it is sent for MM testing.

Overview
===
Most code uses folio_attach/detach/change_private() functions, so folio
refcount is increased and decreased when folio->private is set and reset,
respectively. There is no need to change them.

Changes are needed for exceptional users:
1. zsmalloc uses PG_private to indicate first component zpdesc page and
   page->private is used to store zspage in zpdesc. To remove PG_private,
   is_first_zpdesc() is replaced by pointer comparison.

2. kernel/events/ring_buffer.c stores page order in page->private.
   Replacing PG_private with page->private != NULL works.

3. drivers/xen/grant-table.c stores xen_page_foreign in page->private,
   where on 32-bit, a pointer to xen_page_foreign is stored; on 64-bit,
   page->private is used as xen_page_foreign. PG_private check is replaced
   by page->private != NULL on 32-bit for xen_page_foreign deallocation.
   On 64-bit, page->private is cleared unconditionally since {domid=0,
   gref=0} (xen_page_foreign can be 0) is valid.

4. fs/crypto/crypto.c stores a folio pointer in page->private, PG_private
   checks are replaced by page->private != NULL.

5. fs/erofs has two different uses:

    5a. folio->private is used to form a reversed list of
    the outputs of readahead_folio(). readahead_folio_last() is added to
    output folios in reversed order, so that ->private is no longer needed.

    5b. folio->private is used as an in-flight I/O counter. Convert the
    code to use folio_attach/detach/get_private() and add bias==1 to the
    counter to avoid folio->private being zero.

6. fs/nfs/write.c: folio refcount maintenance is in a bigger scope than
   folio->private. So folio_attach/detach/get_private() is not used.
   Nothing to change.

7. fs/f2fs uses attach_page_private() to first reset folio->private then
   immediately sets PAGE_PRIVATE_NOT_POINTER bit on it. Change it to use
   attach_page_private() to set PAGE_PRIVATE_NOT_POINTER bit directly to
   avoid folio->private == NULL gap inside set_page_private_##name().

8. hugetlb uses folio_change_private(folio, NULL) without folio refcount
   maintenance. Change it to folio->private = NULL.

After the above changes, PG_private ops are converted to
page/folio->private ops.

folio_test_fs_private() is added to check filesystem-only private data by
excluding swapcache and hugetlb folios, because swapcache folios overlap
swp_entry_t swap with ->private and hugetlb sets its own flags in
->private.

Note
===
1. KPF_PRIVATE is removed after PG_private is removed.

2. Documentation/mm/hugetlbfs_reserv.rst is outdated, so I did not remove
   PG_private related text. It should be rewritten.

3. PG_folio is planned to be set on every page from a folio in
   page_rmappable_folio(), so folios with any order (currently
   PG_large_rmappable is used to identify >0 order folios, but not order-0
   folios) can be identified. Then vm_insert_*() can correctly reject all
   folios and rmap code will only see folios. Eventually, page_folio()
   will return NULL for non-folio pages by checking PG_folio, but before
   that all existing users that treat compound pages as folios will need
   to be converted.

Tests
===
1. allmodconfig build passed.

2. zsmalloc is tested using ext4 on a 1GB lz4 zram:
    2a. zram load + zsmalloc compaction;
    2b. concurrent zspage migration via memory compaction;
    2c. confirmed that multi-page zspages actually formed.

    Details: https://github.com/x-y-z/linux-dev/blob/b4/remove-pg_private/test_zsmalloc.md

3. erofs is tested on images created with -C4096 and lz4hc, lzma,
   deflate, and zstd algorithms:
   3a. cold read of all files, verify checksums match source;
   3b. readahead + reclaim/migration race.

   Details: https://github.com/x-y-z/linux-dev/blob/b4/remove-pg_private/test_erofs.md

4. fscrypt is tested on software-encrypted ext4 with writes to exercise
   bounce pages.

   Details: https://github.com/x-y-z/linux-dev/blob/b4/remove-pg_private/test_fscrypt.md

5. f2fs is tested on an image with inline_data,compress_algorithm=lz4:
    5a. INLINE_INODE — lots of tiny files;
    5b. REF_RESOURCE + general writeback — buffered write churn with fsync;
    5c. ONGOING_MIGRATION — force GC / page migration;
    5d. ATOMIC_WRITE — atomic-write ioctl path.

    Details: https://github.com/x-y-z/linux-dev/blob/b4/remove-pg_private/test_f2fs.md
    (I did not run xfstests)

6. MM selftests passed.

LLM use
===
Claude was used to form a concrete plan on what code needs to be changed
and how to change them. The plan was reviewed by Codex until no issue was
spotted.

Plan is at: https://github.com/x-y-z/linux-dev/blob/b4/remove-pg_private/plan.md

I then followed the plan to make code changes. I did bounce ideas with
Claude how to change fs/erofs, since I did not like the original idea.
After each change, I asked Claude to review my code and git commit message.
I also asked Claude to give me test plans (see above).

At last, Codex was used to review all patches.

Comments and suggestions are welcome. Thanks.

Assisted-by: Claude:claude-opus-4-8
Assisted-by: Codex:gpt-5
Signed-off-by: Zi Yan <ziy@nvidia.com>
---
Changes in v3:
1. changed folio_test_fs_private() to check PG_swapbacked instead of
   PG_swapcache for excluding swapcache folios. Because folio->private and
   PG_swapcache are not set as a whole, making folio_test_fs_private() give
   false positive, whereas PG_swapbacked is always set for swapcache
   folios.
2. added __DEF_PAGEFLAG_NAME() to show __PG_folio instead of open code.
3. f2fs change is picked up at
   https://git.kernel.org/jaegeuk/f2fs/c/5ad9409a9533, mm-new currently
   does not have it, so the patch is sent for MM testing purpose.
- Link to v2: https://patch.msgid.link/20260831-remove-pg_private-v2-0-3668159cd9e8@nvidia.com

Changes in v2:
1. removed is_first_zpdesc() in patch 1 and open coded the checks.
2. fixed wording in patch 2's commit message and clarified page_private()
   also works when ring buffer's AUX page order is 0.
3. removed the empty loop in 64-bit gnttab_pages_set_private().
4. clarified folio->private will be reset to NULL by
   fscrypt_free_bounce_page() in the commit message.
5. clarified why hugetlb needs to restore hugetlb_vmemmap_optimized.
6. renamed readahead_folio_reverse() readahead_folio_last() and
   reimplemented readahead_folio_last() by adding a new readahead_control
   private member, _forward, and a new helper __readahead_advance().
7. added a bias, 1, to erofs I/O counter, so that folio->private stays non
   NULL between folio_attach_private() and folio_detach_private().
8. converted more call sites to use folio_test_fs_private().
- Link to v1: https://lore.kernel.org/r/20260731-remove-pg_private-v1-0-142c97ba3562@nvidia.com

---
Zi Yan (14):
      mm/zsmalloc: replace PG_private with pointer comparison
      perf/ring_buffer: stop using PG_private as AUX page high-order marker
      xen/grant-table: stop setting PG_private on pages for grant mapping
      fscrypt: stop setting PG_private on bounce page
      mm/hugetlb: use direct assignment instead of folio_change_private()
      f2fs: stop using PG_private
      erofs: mm/pagemap: add readahead_folio_last() to avoid folio->private
      erofs: use folio_attach/detach_private() instead of direct assignment
      mm/page-flags: check page/folio->private instead of PG_private
      mm/page-flags: introduce folio_test_fs_private()
      treewide: remove folio_set/clear_private()
      treewide: replace PagePrivate() with page_private()
      treewide: adjust comments on PagePrivate and PG_private
      mm/page-flags: remove PG_private

 Documentation/admin-guide/kdump/vmcoreinfo.rst |  2 +-
 Documentation/filesystems/vfs.rst              |  6 +--
 arch/x86/events/intel/bts.c                    |  3 --
 arch/x86/events/intel/pt.c                     |  6 +--
 drivers/md/md-bitmap.c                         |  6 +--
 drivers/xen/balloon.c                          |  5 +++
 drivers/xen/grant-table.c                      | 11 +++--
 fs/ceph/addr.c                                 |  8 ++--
 fs/crypto/crypto.c                             |  2 -
 fs/erofs/data.c                                | 16 ++++---
 fs/erofs/zdata.c                               | 13 ++----
 fs/f2fs/f2fs.h                                 |  8 ++--
 fs/nfs/file.c                                  |  4 +-
 fs/nfs/write.c                                 |  2 -
 fs/proc/page.c                                 |  1 -
 fs/ubifs/file.c                                |  8 ++--
 include/linux/buffer_head.h                    |  6 ---
 include/linux/kernel-page-flags.h              |  1 -
 include/linux/mm.h                             | 35 +++++++++------
 include/linux/mm_types.h                       |  4 +-
 include/linux/page-flags.h                     | 43 +++++++++++++-----
 include/linux/pagemap.h                        | 60 +++++++++++++++++++++-----
 include/trace/events/mmflags.h                 |  3 +-
 include/trace/events/pagemap.h                 |  3 +-
 kernel/events/ring_buffer.c                    |  7 ++-
 kernel/vmcore_info.c                           |  1 -
 mm/huge_memory.c                               |  3 +-
 mm/hugetlb.c                                   |  6 +--
 mm/migrate.c                                   |  3 +-
 mm/page-writeback.c                            |  3 +-
 mm/vmscan.c                                    |  2 +-
 mm/zpdesc.h                                    |  2 +-
 mm/zsmalloc.c                                  | 24 +++--------
 tools/mm/page-types.c                          |  2 -
 34 files changed, 172 insertions(+), 137 deletions(-)
---
base-commit: 5f4c0999b2bde1fbbba3c499208b415447ca4c7a
change-id: 20260728-remove-pg_private-cfe926c7f83c

Best regards,
--  
Yan, Zi

[PATCH v3 07/14] erofs: mm/pagemap: add readahead_folio_last() to avoid folio->private

[Zi Yan]

erofs needs to traverse readahead folios in reverse order to achieve
maximum performance by
1. reading all folios from readahead_folio();
2. storing the prior folio pointer in folio->private;
3. traverse from the last folio to the first one.

Add readahead_folio_last() to achieve the same function without using
folio->private. __readahead_advance() helper shares readahead_control
adjustment code among __readahead_folio(), readahead_folio_last(), and
__readahead_batch() by checking new private member, _forward, of
readahead_control.

It prepares for a future commit that replaces PG_private checks with
!folio->private checks. After switching the checks, erofs's use of
folio->private without bumping folio refcount can cause unexpected
outcomes, e.g., in filemap_release_folio(), try_to_free_buffers() becomes
reachable.

No functional change intended.

Assisted-by: Claude:claude-opus-4-8
Assisted-by: Codex:gpt-5
Signed-off-by: Zi Yan <ziy@nvidia.com>
To: Gao Xiang <xiang@kernel.org>
To: Chao Yu <chao@kernel.org>
To: "Matthew Wilcox (Oracle)" <willy@infradead.org>
To: Jan Kara <jack@suse.cz>
Cc: Yue Hu <zbestahu@gmail.com>
Cc: Jeffle Xu <jefflexu@linux.alibaba.com>
Cc: Sandeep Dhavale <dhavale@google.com>
Cc: Hongbo Li <hongbohbli@tencent.com>
Cc: Chunhai Guo <guochunhai@vivo.com>
Cc: linux-erofs@lists.ozlabs.org
Cc: linux-kernel@vger.kernel.org
Cc: linux-fsdevel@vger.kernel.org
Cc: linux-mm@kvack.org
---
 fs/erofs/zdata.c        | 13 +++---------
 include/linux/pagemap.h | 56 ++++++++++++++++++++++++++++++++++++++++++-------
 2 files changed, 51 insertions(+), 18 deletions(-)

diff --git a/fs/erofs/zdata.c b/fs/erofs/zdata.c
index e1e25ca0d1904..78fd7d980e957 100644
--- a/fs/erofs/zdata.c
+++ b/fs/erofs/zdata.c
@@ -1898,21 +1898,14 @@ static void z_erofs_readahead(struct readahead_control *rac)
 	struct inode *realinode = erofs_real_inode(sharedinode, &need_iput);
 	Z_EROFS_DEFINE_FRONTEND(f, realinode, sharedinode, readahead_pos(rac));
 	unsigned int nrpages = readahead_count(rac);
-	struct folio *head = NULL, *folio;
+	struct folio *folio;
 	int err;
 
 	trace_erofs_readahead(realinode, readahead_index(rac), nrpages, false);
 	z_erofs_pcluster_readmore(&f, rac, true);
-	while ((folio = readahead_folio(rac))) {
-		folio->private = head;
-		head = folio;
-	}
-
-	/* traverse in reverse order for best metadata I/O performance */
-	while (head) {
-		folio = head;
-		head = folio_get_private(folio);
 
+	/* traverse from last to first for best metadata I/O performance */
+	while ((folio = readahead_folio_last(rac))) {
 		err = z_erofs_scan_folio(&f, folio, true);
 		if (err && err != -EINTR)
 			erofs_err(realinode->i_sb, "readahead error at folio %lu @ nid %llu",
diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h
index 939f3a5e973f6..2257df004305e 100644
--- a/include/linux/pagemap.h
+++ b/include/linux/pagemap.h
@@ -1415,6 +1415,7 @@ struct readahead_control {
 	bool dropbehind;
 	bool _workingset;
 	unsigned long _pflags;
+	bool _forward;
 };
 
 #define DEFINE_READAHEAD(ractl, f, r, m, i)				\
@@ -1479,18 +1480,25 @@ void page_cache_async_readahead(struct address_space *mapping,
 	page_cache_async_ra(&ractl, folio, req_count);
 }
 
+static inline void __readahead_advance(struct readahead_control *rac)
+{
+	if (rac->_forward)
+		rac->_index += rac->_batch_count;
+
+	rac->_nr_pages -= rac->_batch_count;
+	rac->_batch_count = 0;
+}
+
 static inline struct folio *__readahead_folio(struct readahead_control *ractl)
 {
 	struct folio *folio;
 
 	BUG_ON(ractl->_batch_count > ractl->_nr_pages);
-	ractl->_nr_pages -= ractl->_batch_count;
-	ractl->_index += ractl->_batch_count;
+	__readahead_advance(ractl);
+	ractl->_forward = true;
 
-	if (!ractl->_nr_pages) {
-		ractl->_batch_count = 0;
+	if (!ractl->_nr_pages)
 		return NULL;
-	}
 
 	folio = xa_load(&ractl->mapping->i_pages, ractl->_index);
 	VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio);
@@ -1516,6 +1524,39 @@ static inline struct folio *readahead_folio(struct readahead_control *ractl)
 	return folio;
 }
 
+/**
+ * readahead_folio_last - Get the next folio to read, from the tail.
+ * @ractl: The current readahead request.
+ *
+ * Like readahead_folio(), but walks the range back-to-front. The folio is
+ * returned locked with its refcount dropped; the caller unlocks it once I/O
+ * completes. Compound folios are returned once, at their head index.
+ *
+ * Context: The folio is locked.
+ * Return: A pointer to the next folio, or %NULL when done.
+ */
+static inline struct folio *readahead_folio_last(struct readahead_control *ractl)
+{
+	struct folio *folio;
+
+	/* Drop the previously returned batch from the remaining range. */
+	__readahead_advance(ractl);
+	ractl->_forward = false;
+
+	if (!ractl->_nr_pages)
+		return NULL;
+
+	/* xa_load() follows sibling entries, so a tail index returns the head */
+	folio = xa_load(&ractl->mapping->i_pages,
+			ractl->_index + ractl->_nr_pages - 1);
+	VM_WARN_ON_ONCE_FOLIO(!folio_test_locked(folio), folio);
+
+	ractl->_batch_count = folio_nr_pages(folio);
+
+	folio_put(folio);
+	return folio;
+}
+
 static inline unsigned int __readahead_batch(struct readahead_control *rac,
 		struct page **array, unsigned int array_sz)
 {
@@ -1524,9 +1565,8 @@ static inline unsigned int __readahead_batch(struct readahead_control *rac,
 	struct folio *folio;
 
 	BUG_ON(rac->_batch_count > rac->_nr_pages);
-	rac->_nr_pages -= rac->_batch_count;
-	rac->_index += rac->_batch_count;
-	rac->_batch_count = 0;
+	__readahead_advance(rac);
+	rac->_forward = true;
 
 	xas_set(&xas, rac->_index);
 	rcu_read_lock();

-- 
2.53.0

[PATCH v3 08/14] erofs: use folio_attach/detach_private() instead of direct assignment

[Zi Yan]

erofs_onlinefolio_init/split/end() use folio->private without setting
PG_private or increasing folio refcount and it works. But after PG_private
is replaced by checking folio->private in a future commit, it can break
folio_expected_ref_count(), since the folio has private data without
elevated refcount. Change them to use folio_attach/detach_private().

Furthermore, because folio->private is used to store in-flight I/O counter
and the counter reaches 0 when all I/O completes successfully without error
or being dirty, ->private=0 causes folio_detach_private() to not drop the
elevated folio refcount. Solve this issue by using bias=1 for the counter,
so that ->private stays non NULL throughout every attach-to-detach process.
Add a macro EROFS_ONLINEFOLIO_BIAS=1. While at it, fix the comment about
->private bit layout and add EROFS_ONLINEFOLIO_COUNT_MASK.

It prepares for a future commit that removes PG_private.

Assisted-by: Claude:claude-opus-4-8
Assisted-by: Codex:gpt-5
To: Gao Xiang <xiang@kernel.org>
To: Chao Yu <chao@kernel.org>
Cc: Yue Hu <zbestahu@gmail.com>
Cc: Jeffle Xu <jefflexu@linux.alibaba.com>
Cc: Sandeep Dhavale <dhavale@google.com>
Cc: Hongbo Li <hongbohbli@tencent.com>
Cc: Chunhai Guo <guochunhai@vivo.com>
Cc: linux-erofs@lists.ozlabs.org
Cc: linux-kernel@vger.kernel.org
Reviewed-by: Gao Xiang <xiang@kernel.org>
Signed-off-by: Zi Yan <ziy@nvidia.com>
---
 fs/erofs/data.c | 16 ++++++++++------
 1 file changed, 10 insertions(+), 6 deletions(-)

diff --git a/fs/erofs/data.c b/fs/erofs/data.c
index 0885b1f2fc925..f71c7b51bb720 100644
--- a/fs/erofs/data.c
+++ b/fs/erofs/data.c
@@ -239,19 +239,23 @@ int erofs_map_dev(struct super_block *sb, struct erofs_map_dev *map)
 /*
  * bit 30: I/O error occurred on this folio
  * bit 29: CPU has dirty data in D-cache (needs aliasing handling);
- * bit 0 - 29: remaining parts to complete this folio
+ * bit 0 - 28: remaining parts to complete this folio, biased by 1 so that
+ *	       ->private stays non-NULL while the folio is attached
  */
 #define EROFS_ONLINEFOLIO_EIO		30
 #define EROFS_ONLINEFOLIO_DIRTY		29
+#define EROFS_ONLINEFOLIO_COUNT_MASK	(BIT(EROFS_ONLINEFOLIO_DIRTY) - 1)
+#define EROFS_ONLINEFOLIO_BIAS		1
 
 void erofs_onlinefolio_init(struct folio *folio)
 {
 	union {
 		atomic_t o;
 		void *v;
-	} u = { .o = ATOMIC_INIT(1) };
+	} u = { .o = ATOMIC_INIT(1 + EROFS_ONLINEFOLIO_BIAS) };
 
-	folio->private = u.v;	/* valid only if file-backed folio is locked */
+	/* valid only if file-backed folio is locked */
+	folio_attach_private(folio, u.v);
 }
 
 void erofs_onlinefolio_split(struct folio *folio)
@@ -265,14 +269,14 @@ void erofs_onlinefolio_end(struct folio *folio, int err, bool dirty)
 
 	do {
 		orig = atomic_read((atomic_t *)&folio->private);
-		DBG_BUGON(orig <= 0);
+		DBG_BUGON((orig & EROFS_ONLINEFOLIO_COUNT_MASK) <= EROFS_ONLINEFOLIO_BIAS);
 		v = dirty << EROFS_ONLINEFOLIO_DIRTY;
 		v |= (orig - 1) | (!!err << EROFS_ONLINEFOLIO_EIO);
 	} while (atomic_cmpxchg((atomic_t *)&folio->private, orig, v) != orig);
 
-	if (v & (BIT(EROFS_ONLINEFOLIO_DIRTY) - 1))
+	if ((v & EROFS_ONLINEFOLIO_COUNT_MASK) != EROFS_ONLINEFOLIO_BIAS)
 		return;
-	folio->private = 0;
+	folio_detach_private(folio);
 	if (v & BIT(EROFS_ONLINEFOLIO_DIRTY))
 		flush_dcache_folio(folio);
 	folio_end_read(folio, !(v & BIT(EROFS_ONLINEFOLIO_EIO)));

-- 
2.53.0

Re: [PATCH v3 07/14] erofs: mm/pagemap: add readahead_folio_last() to avoid folio->private

[David Hildenbrand (Arm)]

On 9/8/26 04:56, Zi Yan wrote:
> erofs needs to traverse readahead folios in reverse order to achieve
> maximum performance by
> 1. reading all folios from readahead_folio();
> 2. storing the prior folio pointer in folio->private;
> 3. traverse from the last folio to the first one.
> 
> Add readahead_folio_last() to achieve the same function without using
> folio->private. __readahead_advance() helper shares readahead_control
> adjustment code among __readahead_folio(), readahead_folio_last(), and
> __readahead_batch() by checking new private member, _forward, of
> readahead_control.
> 
> It prepares for a future commit that replaces PG_private checks with
> !folio->private checks. After switching the checks, erofs's use of
> folio->private without bumping folio refcount can cause unexpected
> outcomes, e.g., in filemap_release_folio(), try_to_free_buffers() becomes
> reachable.

Ah, I was just about to ask. So it's really about folios never using
folio->private manually (without the attach/detach).

> 
> No functional change intended.
> 
> Assisted-by: Claude:claude-opus-4-8
> Assisted-by: Codex:gpt-5
> Signed-off-by: Zi Yan <ziy@nvidia.com>
> To: Gao Xiang <xiang@kernel.org>
> To: Chao Yu <chao@kernel.org>
> To: "Matthew Wilcox (Oracle)" <willy@infradead.org>
> To: Jan Kara <jack@suse.cz>
> Cc: Yue Hu <zbestahu@gmail.com>
> Cc: Jeffle Xu <jefflexu@linux.alibaba.com>
> Cc: Sandeep Dhavale <dhavale@google.com>
> Cc: Hongbo Li <hongbohbli@tencent.com>
> Cc: Chunhai Guo <guochunhai@vivo.com>
> Cc: linux-erofs@lists.ozlabs.org
> Cc: linux-kernel@vger.kernel.org
> Cc: linux-fsdevel@vger.kernel.org
> Cc: linux-mm@kvack.org
> ---
>  fs/erofs/zdata.c        | 13 +++---------
>  include/linux/pagemap.h | 56 ++++++++++++++++++++++++++++++++++++++++++-------
>  2 files changed, 51 insertions(+), 18 deletions(-)
> 
> diff --git a/fs/erofs/zdata.c b/fs/erofs/zdata.c
> index e1e25ca0d1904..78fd7d980e957 100644
> --- a/fs/erofs/zdata.c
> +++ b/fs/erofs/zdata.c
> @@ -1898,21 +1898,14 @@ static void z_erofs_readahead(struct readahead_control *rac)
>  	struct inode *realinode = erofs_real_inode(sharedinode, &need_iput);
>  	Z_EROFS_DEFINE_FRONTEND(f, realinode, sharedinode, readahead_pos(rac));
>  	unsigned int nrpages = readahead_count(rac);
> -	struct folio *head = NULL, *folio;
> +	struct folio *folio;
>  	int err;
>  
>  	trace_erofs_readahead(realinode, readahead_index(rac), nrpages, false);
>  	z_erofs_pcluster_readmore(&f, rac, true);
> -	while ((folio = readahead_folio(rac))) {
> -		folio->private = head;
> -		head = folio;
> -	}
> -
> -	/* traverse in reverse order for best metadata I/O performance */
> -	while (head) {
> -		folio = head;
> -		head = folio_get_private(folio);
>  
> +	/* traverse from last to first for best metadata I/O performance */
> +	while ((folio = readahead_folio_last(rac))) {

Intuitively, this should be called readahead_folio_reverse /
readahead_folio_reversed, thinking of list_for_each_entry_reverse()?

	list_for_each_entry_reverse - iterate backwards over list of given type.

or maybe readahead_folio_backwards (which matches the forward below)

But I'm not a readahead expert :)

>  		err = z_erofs_scan_folio(&f, folio, true);
>  		if (err && err != -EINTR)
>  			erofs_err(realinode->i_sb, "readahead error at folio %lu @ nid %llu",
> diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h
> index 939f3a5e973f6..2257df004305e 100644
> --- a/include/linux/pagemap.h
> +++ b/include/linux/pagemap.h
> @@ -1415,6 +1415,7 @@ struct readahead_control {
>  	bool dropbehind;
>  	bool _workingset;
>  	unsigned long _pflags;
> +	bool _forward;
>  };
>  
>  #define DEFINE_READAHEAD(ractl, f, r, m, i)				\
> @@ -1479,18 +1480,25 @@ void page_cache_async_readahead(struct address_space *mapping,
>  	page_cache_async_ra(&ractl, folio, req_count);
>  }
>  
> +static inline void __readahead_advance(struct readahead_control *rac)
> +{
> +	if (rac->_forward)
> +		rac->_index += rac->_batch_count;

No expert, but shouldn't we decrement the _index somewhere in the other case? Or
where is that done? A comment might help :)


-- 
Cheers,

David

Re: [PATCH v3 08/14] erofs: use folio_attach/detach_private() instead of direct assignment

[David Hildenbrand (Arm)]

On 9/8/26 04:56, Zi Yan wrote:
> erofs_onlinefolio_init/split/end() use folio->private without setting
> PG_private or increasing folio refcount and it works. But after PG_private
> is replaced by checking folio->private in a future commit, it can break
> folio_expected_ref_count(), since the folio has private data without
> elevated refcount. Change them to use folio_attach/detach_private().

Freezing the folio will properly work I assume, as we'd always freeze with that
refcount in mind or remove the reference (detach private) early?

I'm thinking of folio_ref_freeze() in __folio_freeze_and_split_unmapped().

Or is the ref only very short-lived either way such that we don't care?



-- 
Cheers,

David

Re: [PATCH v3 07/14] erofs: mm/pagemap: add readahead_folio_last() to avoid folio->private

[Zi Yan]

On Tue Sep 8, 2026 at 12:04 PM EDT, David Hildenbrand (Arm) wrote:
> On 9/8/26 04:56, Zi Yan wrote:
>> erofs needs to traverse readahead folios in reverse order to achieve
>> maximum performance by
>> 1. reading all folios from readahead_folio();
>> 2. storing the prior folio pointer in folio->private;
>> 3. traverse from the last folio to the first one.
>> 
>> Add readahead_folio_last() to achieve the same function without using
>> folio->private. __readahead_advance() helper shares readahead_control
>> adjustment code among __readahead_folio(), readahead_folio_last(), and
>> __readahead_batch() by checking new private member, _forward, of
>> readahead_control.
>> 
>> It prepares for a future commit that replaces PG_private checks with
>> !folio->private checks. After switching the checks, erofs's use of
>> folio->private without bumping folio refcount can cause unexpected
>> outcomes, e.g., in filemap_release_folio(), try_to_free_buffers() becomes
>> reachable.
>
> Ah, I was just about to ask. So it's really about folios never using
> folio->private manually (without the attach/detach).
>
>> 
>> No functional change intended.
>> 
>> Assisted-by: Claude:claude-opus-4-8
>> Assisted-by: Codex:gpt-5
>> Signed-off-by: Zi Yan <ziy@nvidia.com>
>> To: Gao Xiang <xiang@kernel.org>
>> To: Chao Yu <chao@kernel.org>
>> To: "Matthew Wilcox (Oracle)" <willy@infradead.org>
>> To: Jan Kara <jack@suse.cz>
>> Cc: Yue Hu <zbestahu@gmail.com>
>> Cc: Jeffle Xu <jefflexu@linux.alibaba.com>
>> Cc: Sandeep Dhavale <dhavale@google.com>
>> Cc: Hongbo Li <hongbohbli@tencent.com>
>> Cc: Chunhai Guo <guochunhai@vivo.com>
>> Cc: linux-erofs@lists.ozlabs.org
>> Cc: linux-kernel@vger.kernel.org
>> Cc: linux-fsdevel@vger.kernel.org
>> Cc: linux-mm@kvack.org
>> ---
>>  fs/erofs/zdata.c        | 13 +++---------
>>  include/linux/pagemap.h | 56 ++++++++++++++++++++++++++++++++++++++++++-------
>>  2 files changed, 51 insertions(+), 18 deletions(-)
>> 
>> diff --git a/fs/erofs/zdata.c b/fs/erofs/zdata.c
>> index e1e25ca0d1904..78fd7d980e957 100644
>> --- a/fs/erofs/zdata.c
>> +++ b/fs/erofs/zdata.c
>> @@ -1898,21 +1898,14 @@ static void z_erofs_readahead(struct readahead_control *rac)
>>  	struct inode *realinode = erofs_real_inode(sharedinode, &need_iput);
>>  	Z_EROFS_DEFINE_FRONTEND(f, realinode, sharedinode, readahead_pos(rac));
>>  	unsigned int nrpages = readahead_count(rac);
>> -	struct folio *head = NULL, *folio;
>> +	struct folio *folio;
>>  	int err;
>>  
>>  	trace_erofs_readahead(realinode, readahead_index(rac), nrpages, false);
>>  	z_erofs_pcluster_readmore(&f, rac, true);
>> -	while ((folio = readahead_folio(rac))) {
>> -		folio->private = head;
>> -		head = folio;
>> -	}
>> -
>> -	/* traverse in reverse order for best metadata I/O performance */
>> -	while (head) {
>> -		folio = head;
>> -		head = folio_get_private(folio);
>>  
>> +	/* traverse from last to first for best metadata I/O performance */
>> +	while ((folio = readahead_folio_last(rac))) {
>
> Intuitively, this should be called readahead_folio_reverse /
> readahead_folio_reversed, thinking of list_for_each_entry_reverse()?
>
> 	list_for_each_entry_reverse - iterate backwards over list of given type.
>
> or maybe readahead_folio_backwards (which matches the forward below)
>
> But I'm not a readahead expert :)

Jan suggested the name[1]. It can be readahead_folio_reverse() if you
prefer it, like Jan said.

[1] https://lore.kernel.org/all/332rknj4vo3cfhvfhhlf6pvg37s3lbrnzbbnv4swa6gctsiu6a@ndotgokvnglc/

>
>>  		err = z_erofs_scan_folio(&f, folio, true);
>>  		if (err && err != -EINTR)
>>  			erofs_err(realinode->i_sb, "readahead error at folio %lu @ nid %llu",
>> diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h
>> index 939f3a5e973f6..2257df004305e 100644
>> --- a/include/linux/pagemap.h
>> +++ b/include/linux/pagemap.h
>> @@ -1415,6 +1415,7 @@ struct readahead_control {
>>  	bool dropbehind;
>>  	bool _workingset;
>>  	unsigned long _pflags;
>> +	bool _forward;
>>  };
>>  
>>  #define DEFINE_READAHEAD(ractl, f, r, m, i)				\
>> @@ -1479,18 +1480,25 @@ void page_cache_async_readahead(struct address_space *mapping,
>>  	page_cache_async_ra(&ractl, folio, req_count);
>>  }
>>  
>> +static inline void __readahead_advance(struct readahead_control *rac)
>> +{
>> +	if (rac->_forward)
>> +		rac->_index += rac->_batch_count;
>
> No expert, but shouldn't we decrement the _index somewhere in the other case? Or
> where is that done? A comment might help :)

A readahead folio comes from [_index, _index + _nr_pages), so for
last/reverse/backwards case, the code only needs to decrease _nr_pages.

Will add a comment about this.

-- 
Best Regards,
Yan, Zi

Re: [PATCH v3 08/14] erofs: use folio_attach/detach_private() instead of direct assignment

[Zi Yan]

On Tue Sep 8, 2026 at 12:13 PM EDT, David Hildenbrand (Arm) wrote:
> On 9/8/26 04:56, Zi Yan wrote:
>> erofs_onlinefolio_init/split/end() use folio->private without setting
>> PG_private or increasing folio refcount and it works. But after PG_private
>> is replaced by checking folio->private in a future commit, it can break
>> folio_expected_ref_count(), since the folio has private data without
>> elevated refcount. Change them to use folio_attach/detach_private().
>
> Freezing the folio will properly work I assume, as we'd always freeze with that
> refcount in mind or remove the reference (detach private) early?

Yes.

>
> I'm thinking of folio_ref_freeze() in __folio_freeze_and_split_unmapped().
>
> Or is the ref only very short-lived either way such that we don't care?

The folios here are locked since they are in the process of
readahead, which requires all folios locked. So they are not reachable
from folio split code. In additio, the private is deatched and ref is
decreased before the folio lock is dropped.


-- 
Best Regards,
Yan, Zi

Re: [PATCH v3 08/14] erofs: use folio_attach/detach_private() instead of direct assignment

[David Hildenbrand (Arm)]

On 9/8/26 19:19, Zi Yan wrote:
> On Tue Sep 8, 2026 at 12:13 PM EDT, David Hildenbrand (Arm) wrote:
>> On 9/8/26 04:56, Zi Yan wrote:
>>> erofs_onlinefolio_init/split/end() use folio->private without setting
>>> PG_private or increasing folio refcount and it works. But after PG_private
>>> is replaced by checking folio->private in a future commit, it can break
>>> folio_expected_ref_count(), since the folio has private data without
>>> elevated refcount. Change them to use folio_attach/detach_private().
>>
>> Freezing the folio will properly work I assume, as we'd always freeze with that
>> refcount in mind or remove the reference (detach private) early?
> 
> Yes.
> 
>>
>> I'm thinking of folio_ref_freeze() in __folio_freeze_and_split_unmapped().
>>
>> Or is the ref only very short-lived either way such that we don't care?
> 
> The folios here are locked since they are in the process of
> readahead, which requires all folios locked. So they are not reachable
> from folio split code. In additio, the private is deatched and ref is
> decreased before the folio lock is dropped.

It would be great to summarize this interaction with core-mm (migration/split)
in the patch description.

-- 
Cheers,

David

Re: [PATCH v3 07/14] erofs: mm/pagemap: add readahead_folio_last() to avoid folio->private

[David Hildenbrand (Arm)]

On 9/8/26 19:05, Zi Yan wrote:
> On Tue Sep 8, 2026 at 12:04 PM EDT, David Hildenbrand (Arm) wrote:
>> On 9/8/26 04:56, Zi Yan wrote:
>>> erofs needs to traverse readahead folios in reverse order to achieve
>>> maximum performance by
>>> 1. reading all folios from readahead_folio();
>>> 2. storing the prior folio pointer in folio->private;
>>> 3. traverse from the last folio to the first one.
>>>
>>> Add readahead_folio_last() to achieve the same function without using
>>> folio->private. __readahead_advance() helper shares readahead_control
>>> adjustment code among __readahead_folio(), readahead_folio_last(), and
>>> __readahead_batch() by checking new private member, _forward, of
>>> readahead_control.
>>>
>>> It prepares for a future commit that replaces PG_private checks with
>>> !folio->private checks. After switching the checks, erofs's use of
>>> folio->private without bumping folio refcount can cause unexpected
>>> outcomes, e.g., in filemap_release_folio(), try_to_free_buffers() becomes
>>> reachable.
>>
>> Ah, I was just about to ask. So it's really about folios never using
>> folio->private manually (without the attach/detach).
>>
>>>
>>> No functional change intended.
>>>
>>> Assisted-by: Claude:claude-opus-4-8
>>> Assisted-by: Codex:gpt-5
>>> Signed-off-by: Zi Yan <ziy@nvidia.com>
>>> To: Gao Xiang <xiang@kernel.org>
>>> To: Chao Yu <chao@kernel.org>
>>> To: "Matthew Wilcox (Oracle)" <willy@infradead.org>
>>> To: Jan Kara <jack@suse.cz>
>>> Cc: Yue Hu <zbestahu@gmail.com>
>>> Cc: Jeffle Xu <jefflexu@linux.alibaba.com>
>>> Cc: Sandeep Dhavale <dhavale@google.com>
>>> Cc: Hongbo Li <hongbohbli@tencent.com>
>>> Cc: Chunhai Guo <guochunhai@vivo.com>
>>> Cc: linux-erofs@lists.ozlabs.org
>>> Cc: linux-kernel@vger.kernel.org
>>> Cc: linux-fsdevel@vger.kernel.org
>>> Cc: linux-mm@kvack.org
>>> ---
>>>  fs/erofs/zdata.c        | 13 +++---------
>>>  include/linux/pagemap.h | 56 ++++++++++++++++++++++++++++++++++++++++++-------
>>>  2 files changed, 51 insertions(+), 18 deletions(-)
>>>
>>> diff --git a/fs/erofs/zdata.c b/fs/erofs/zdata.c
>>> index e1e25ca0d1904..78fd7d980e957 100644
>>> --- a/fs/erofs/zdata.c
>>> +++ b/fs/erofs/zdata.c
>>> @@ -1898,21 +1898,14 @@ static void z_erofs_readahead(struct readahead_control *rac)
>>>  	struct inode *realinode = erofs_real_inode(sharedinode, &need_iput);
>>>  	Z_EROFS_DEFINE_FRONTEND(f, realinode, sharedinode, readahead_pos(rac));
>>>  	unsigned int nrpages = readahead_count(rac);
>>> -	struct folio *head = NULL, *folio;
>>> +	struct folio *folio;
>>>  	int err;
>>>  
>>>  	trace_erofs_readahead(realinode, readahead_index(rac), nrpages, false);
>>>  	z_erofs_pcluster_readmore(&f, rac, true);
>>> -	while ((folio = readahead_folio(rac))) {
>>> -		folio->private = head;
>>> -		head = folio;
>>> -	}
>>> -
>>> -	/* traverse in reverse order for best metadata I/O performance */
>>> -	while (head) {
>>> -		folio = head;
>>> -		head = folio_get_private(folio);
>>>  
>>> +	/* traverse from last to first for best metadata I/O performance */
>>> +	while ((folio = readahead_folio_last(rac))) {
>>
>> Intuitively, this should be called readahead_folio_reverse /
>> readahead_folio_reversed, thinking of list_for_each_entry_reverse()?
>>
>> 	list_for_each_entry_reverse - iterate backwards over list of given type.
>>
>> or maybe readahead_folio_backwards (which matches the forward below)
>>
>> But I'm not a readahead expert :)
> 
> Jan suggested the name[1]. It can be readahead_folio_reverse() if you
> prefer it, like Jan said.
> 
> [1] https://lore.kernel.org/all/332rknj4vo3cfhvfhhlf6pvg37s3lbrnzbbnv4swa6gctsiu6a@ndotgokvnglc/

Heh, to me _reverse() is clearer; whatever people prefer.

Acked-by: David Hildenbrand (Arm) <david@kernel.org>

-- 
Cheers,

David

Re: [PATCH v3 08/14] erofs: use folio_attach/detach_private() instead of direct assignment

[Zi Yan]

On Wed Sep 9, 2026 at 9:27 AM EDT, David Hildenbrand (Arm) wrote:
> On 9/8/26 19:19, Zi Yan wrote:
>> On Tue Sep 8, 2026 at 12:13 PM EDT, David Hildenbrand (Arm) wrote:
>>> On 9/8/26 04:56, Zi Yan wrote:
>>>> erofs_onlinefolio_init/split/end() use folio->private without setting
>>>> PG_private or increasing folio refcount and it works. But after PG_private
>>>> is replaced by checking folio->private in a future commit, it can break
>>>> folio_expected_ref_count(), since the folio has private data without
>>>> elevated refcount. Change them to use folio_attach/detach_private().
>>>
>>> Freezing the folio will properly work I assume, as we'd always freeze with that
>>> refcount in mind or remove the reference (detach private) early?
>> 
>> Yes.
>> 
>>>
>>> I'm thinking of folio_ref_freeze() in __folio_freeze_and_split_unmapped().
>>>
>>> Or is the ref only very short-lived either way such that we don't care?
>> 
>> The folios here are locked since they are in the process of
>> readahead, which requires all folios locked. So they are not reachable
>> from folio split code. In additio, the private is deatched and ref is
>> decreased before the folio lock is dropped.
>
> It would be great to summarize this interaction with core-mm (migration/split)
> in the patch description.

Will update the commit message.



-- 
Best Regards,
Yan, Zi