[PATCH net-next v7 0/7] Split netmem from struct page

[PATCH net-next v7 0/7] Split netmem from struct page

[Byungchul Park]

Hi all,

In this version, I'm posting non-controversial patches first since there
are pending works that should be based on this series so that those can
be started shortly.  I will post the rest later.

The MM subsystem is trying to reduce struct page to a single pointer.
The first step towards that is splitting struct page by its individual
users, as has already been done with folio and slab.  This patchset does
that for netmem which is used for page pools.

Matthew Wilcox tried and stopped the same work, you can see in:

   https://lore.kernel.org/linux-mm/20230111042214.907030-1-willy@infradead.org/

Mina Almasry already has done a lot fo prerequisite works by luck.  I
stacked my patches on the top of his work e.i. netmem.

I focused on removing the page pool members in struct page this time,
not moving the allocation code of page pool from net to mm.  It can be
done later if needed.

The final patch removing the page pool fields will be submitted once
all the converting work of page to netmem are done:

   1. converting of libeth_fqe by Tony Nguyen.
   2. converting of mlx5 by Tariq Toukan.
   3. converting of prueth_swdata.
   4. converting of freescale driver.

For our discussion, I'm sharing what the final patch looks like the
following.

	Byungchul
--8<--
commit 1847d9890f798456b21ccb27aac7545303048492
Author: Byungchul Park <byungchul@sk.com>
Date:   Wed May 28 20:44:55 2025 +0900

    mm, netmem: remove the page pool members in struct page
    
    Now that all the users of the page pool members in struct page have been
    gone, the members can be removed from struct page.
    
    However, since struct netmem_desc still uses the space in struct page,
    the important offsets should be checked properly, until struct
    netmem_desc has its own instance from slab.
    
    Remove the page pool members in struct page and modify static checkers
    for the offsets.
    
    Signed-off-by: Byungchul Park <byungchul@sk.com>

diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 32ba5126e221..db2fe0d0ebbf 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -120,17 +120,6 @@ struct page {
 			 */
 			unsigned long private;
 		};
-		struct {	/* page_pool used by netstack */
-			/**
-			 * @pp_magic: magic value to avoid recycling non
-			 * page_pool allocated pages.
-			 */
-			unsigned long pp_magic;
-			struct page_pool *pp;
-			unsigned long _pp_mapping_pad;
-			unsigned long dma_addr;
-			atomic_long_t pp_ref_count;
-		};
 		struct {	/* Tail pages of compound page */
 			unsigned long compound_head;	/* Bit zero is set */
 		};
diff --git a/include/net/netmem.h b/include/net/netmem.h
index 8f354ae7d5c3..3414f184d018 100644
--- a/include/net/netmem.h
+++ b/include/net/netmem.h
@@ -42,11 +42,8 @@ struct netmem_desc {
 	static_assert(offsetof(struct page, pg) == \
 		      offsetof(struct netmem_desc, desc))
 NETMEM_DESC_ASSERT_OFFSET(flags, _flags);
-NETMEM_DESC_ASSERT_OFFSET(pp_magic, pp_magic);
-NETMEM_DESC_ASSERT_OFFSET(pp, pp);
-NETMEM_DESC_ASSERT_OFFSET(_pp_mapping_pad, _pp_mapping_pad);
-NETMEM_DESC_ASSERT_OFFSET(dma_addr, dma_addr);
-NETMEM_DESC_ASSERT_OFFSET(pp_ref_count, pp_ref_count);
+NETMEM_DESC_ASSERT_OFFSET(lru, pp_magic);
+NETMEM_DESC_ASSERT_OFFSET(mapping, _pp_mapping_pad);
 #undef NETMEM_DESC_ASSERT_OFFSET
 
 /*
---
Changes from v5 (no actual updates):
	1. Rebase on net-next/main as of Jun 25.
	2. Supplement a comment describing struct net_iov.
	3. Exclude a controversial patch, "page_pool: access ->pp_magic
	   through struct netmem_desc in page_pool_page_is_pp()".
	4. Exclude "netmem: remove __netmem_get_pp()" since the API
	   started to be used again by libeth.

Changes from v5 (no actual updates):
	1. Rebase on net-next/main as of Jun 20.
	2. Add given 'Reviewed-by's and 'Acked-by's, thanks to all.
	3. Add missing cc's.

Changes from v4:
	1. Add given 'Reviewed-by's, thanks to all.
	2. Exclude potentially controversial patches.

Changes from v3:
	1. Relocates ->owner and ->type of net_iov out of netmem_desc
	   and make them be net_iov specific.
	2. Remove __force when casting struct page to struct netmem_desc.

Changes from v2:
	1. Introduce a netmem API, virt_to_head_netmem(), and use it
	   when it's needed.
	2. Introduce struct netmem_desc as a new struct and union'ed
	   with the existing fields in struct net_iov.
	3. Make page_pool_page_is_pp() access ->pp_magic through struct
	   netmem_desc instead of struct page.
	4. Move netmem alloc APIs from include/net/netmem.h to
	   net/core/netmem_priv.h.
	5. Apply trivial feedbacks, thanks to Mina, Pavel, and Toke.
	6. Add given 'Reviewed-by's, thanks to Mina.

Changes from v1:
	1. Rebase on net-next's main as of May 26.
	2. Check checkpatch.pl, feedbacked by SJ Park.
	3. Add converting of page to netmem in mt76.
	4. Revert 'mlx5: use netmem descriptor and APIs for page pool'
	   since it's on-going by Tariq Toukan.  I will wait for his
	   work to be done.
	5. Revert 'page_pool: use netmem APIs to access page->pp_magic
	   in page_pool_page_is_pp()' since we need more discussion.
	6. Revert 'mm, netmem: remove the page pool members in struct
	   page' since there are some prerequisite works to remove the
	   page pool fields from struct page.  I can submit this patch
	   separatedly later.
	7. Cancel relocating a page pool member in struct page.
	8. Modify static assert for offests and size of struct
	   netmem_desc.

Changes from rfc:
	1. Rebase on net-next's main branch.
	   https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/
	2. Fix a build error reported by kernel test robot.
	   https://lore.kernel.org/all/202505100932.uzAMBW1y-lkp@intel.com/
	3. Add given 'Reviewed-by's, thanks to Mina and Ilias.
	4. Do static_assert() on the size of struct netmem_desc instead
	   of placing place-holder in struct page, feedbacked by
	   Matthew.
	5. Do struct_group_tagged(netmem_desc) on struct net_iov instead
	   of wholly renaming it to strcut netmem_desc, feedbacked by
	   Mina and Pavel.

Byungchul Park (7):
  netmem: introduce struct netmem_desc mirroring struct page
  page_pool: rename page_pool_return_page() to page_pool_return_netmem()
  page_pool: rename __page_pool_release_page_dma() to
    __page_pool_release_netmem_dma()
  page_pool: rename __page_pool_alloc_pages_slow() to
    __page_pool_alloc_netmems_slow()
  netmem: use _Generic to cover const casting for page_to_netmem()
  page_pool: make page_pool_get_dma_addr() just wrap
    page_pool_get_dma_addr_netmem()
  netmem: introduce a netmem API, virt_to_head_netmem()

 include/net/netmem.h            | 130 ++++++++++++++++++++++++++------
 include/net/page_pool/helpers.h |   7 +-
 net/core/page_pool.c            |  36 ++++-----
 3 files changed, 124 insertions(+), 49 deletions(-)


base-commit: 8dacfd92dbefee829ca555a860e86108fdd1d55b
-- 
2.17.1

[PATCH net-next v7 1/7] netmem: introduce struct netmem_desc mirroring struct page

[Byungchul Park]

To simplify struct page, the page pool members of struct page should be
moved to other, allowing these members to be removed from struct page.

Introduce a network memory descriptor to store the members, struct
netmem_desc, and make it union'ed with the existing fields in struct
net_iov, allowing to organize the fields of struct net_iov.

Signed-off-by: Byungchul Park <byungchul@sk.com>
Reviewed-by: Toke Høiland-Jørgensen <toke@redhat.com>
Reviewed-by: Pavel Begunkov <asml.silence@gmail.com>
Reviewed-by: Mina Almasry <almasrymina@google.com>
Reviewed-by: Vlastimil Babka <vbabka@suse.cz>
Acked-by: Harry Yoo <harry.yoo@oracle.com>
---
 include/net/netmem.h | 116 +++++++++++++++++++++++++++++++++++--------
 1 file changed, 95 insertions(+), 21 deletions(-)

diff --git a/include/net/netmem.h b/include/net/netmem.h
index 7a1dafa3f080..e9eee8f680d5 100644
--- a/include/net/netmem.h
+++ b/include/net/netmem.h
@@ -12,6 +12,50 @@
 #include <linux/mm.h>
 #include <net/net_debug.h>
 
+/* These fields in struct page are used by the page_pool and net stack:
+ *
+ *        struct {
+ *                unsigned long pp_magic;
+ *                struct page_pool *pp;
+ *                unsigned long _pp_mapping_pad;
+ *                unsigned long dma_addr;
+ *                atomic_long_t pp_ref_count;
+ *        };
+ *
+ * We mirror the page_pool fields here so the page_pool can access these
+ * fields without worrying whether the underlying fields belong to a
+ * page or netmem_desc.
+ *
+ * CAUTION: Do not update the fields in netmem_desc without also
+ * updating the anonymous aliasing union in struct net_iov.
+ */
+struct netmem_desc {
+	unsigned long _flags;
+	unsigned long pp_magic;
+	struct page_pool *pp;
+	unsigned long _pp_mapping_pad;
+	unsigned long dma_addr;
+	atomic_long_t pp_ref_count;
+};
+
+#define NETMEM_DESC_ASSERT_OFFSET(pg, desc)        \
+	static_assert(offsetof(struct page, pg) == \
+		      offsetof(struct netmem_desc, desc))
+NETMEM_DESC_ASSERT_OFFSET(flags, _flags);
+NETMEM_DESC_ASSERT_OFFSET(pp_magic, pp_magic);
+NETMEM_DESC_ASSERT_OFFSET(pp, pp);
+NETMEM_DESC_ASSERT_OFFSET(_pp_mapping_pad, _pp_mapping_pad);
+NETMEM_DESC_ASSERT_OFFSET(dma_addr, dma_addr);
+NETMEM_DESC_ASSERT_OFFSET(pp_ref_count, pp_ref_count);
+#undef NETMEM_DESC_ASSERT_OFFSET
+
+/*
+ * Since struct netmem_desc uses the space in struct page, the size
+ * should be checked, until struct netmem_desc has its own instance from
+ * slab, to avoid conflicting with other members within struct page.
+ */
+static_assert(sizeof(struct netmem_desc) <= offsetof(struct page, _refcount));
+
 /* net_iov */
 
 DECLARE_STATIC_KEY_FALSE(page_pool_mem_providers);
@@ -30,13 +74,48 @@ enum net_iov_type {
 	NET_IOV_MAX = ULONG_MAX
 };
 
+/* A memory descriptor representing abstract networking I/O vectors,
+ * generally for non-pages memory that doesn't have its corresponding
+ * struct page and needs to be explicitly allocated through slab.
+ *
+ * net_iovs are allocated and used by networking code, and the size of
+ * the chunk is PAGE_SIZE.
+ *
+ * This memory can be any form of non-struct paged memory.  Examples
+ * include imported dmabuf memory and imported io_uring memory.  See
+ * net_iov_type for all the supported types.
+ *
+ * @pp_magic:	pp field, similar to the one in struct page/struct
+ *		netmem_desc.
+ * @pp:		the pp this net_iov belongs to, if any.
+ * @dma_addr:	the dma addrs of the net_iov. Needed for the network
+ *		card to send/receive this net_iov.
+ * @pp_ref_count: the pp ref count of this net_iov, exactly the same
+ *		usage as struct page/struct netmem_desc.
+ * @owner:	the net_iov_area this net_iov belongs to, if any.
+ * @type:	the type of the memory.  Different types of net_iovs are
+ *		supported.
+ */
 struct net_iov {
-	enum net_iov_type type;
-	unsigned long pp_magic;
-	struct page_pool *pp;
+	union {
+		struct netmem_desc desc;
+
+		/* XXX: The following part should be removed once all
+		 * the references to them are converted so as to be
+		 * accessed via netmem_desc e.g. niov->desc.pp instead
+		 * of niov->pp.
+		 */
+		struct {
+			unsigned long _flags;
+			unsigned long pp_magic;
+			struct page_pool *pp;
+			unsigned long _pp_mapping_pad;
+			unsigned long dma_addr;
+			atomic_long_t pp_ref_count;
+		};
+	};
 	struct net_iov_area *owner;
-	unsigned long dma_addr;
-	atomic_long_t pp_ref_count;
+	enum net_iov_type type;
 };
 
 struct net_iov_area {
@@ -48,27 +127,22 @@ struct net_iov_area {
 	unsigned long base_virtual;
 };
 
-/* These fields in struct page are used by the page_pool and net stack:
+/* net_iov is union'ed with struct netmem_desc mirroring struct page, so
+ * the page_pool can access these fields without worrying whether the
+ * underlying fields are accessed via netmem_desc or directly via
+ * net_iov, until all the references to them are converted so as to be
+ * accessed via netmem_desc e.g. niov->desc.pp instead of niov->pp.
  *
- *        struct {
- *                unsigned long pp_magic;
- *                struct page_pool *pp;
- *                unsigned long _pp_mapping_pad;
- *                unsigned long dma_addr;
- *                atomic_long_t pp_ref_count;
- *        };
- *
- * We mirror the page_pool fields here so the page_pool can access these fields
- * without worrying whether the underlying fields belong to a page or net_iov.
- *
- * The non-net stack fields of struct page are private to the mm stack and must
- * never be mirrored to net_iov.
+ * The non-net stack fields of struct page are private to the mm stack
+ * and must never be mirrored to net_iov.
  */
-#define NET_IOV_ASSERT_OFFSET(pg, iov)             \
-	static_assert(offsetof(struct page, pg) == \
+#define NET_IOV_ASSERT_OFFSET(desc, iov)                    \
+	static_assert(offsetof(struct netmem_desc, desc) == \
 		      offsetof(struct net_iov, iov))
+NET_IOV_ASSERT_OFFSET(_flags, _flags);
 NET_IOV_ASSERT_OFFSET(pp_magic, pp_magic);
 NET_IOV_ASSERT_OFFSET(pp, pp);
+NET_IOV_ASSERT_OFFSET(_pp_mapping_pad, _pp_mapping_pad);
 NET_IOV_ASSERT_OFFSET(dma_addr, dma_addr);
 NET_IOV_ASSERT_OFFSET(pp_ref_count, pp_ref_count);
 #undef NET_IOV_ASSERT_OFFSET
-- 
2.17.1

[PATCH net-next v7 2/7] page_pool: rename page_pool_return_page() to page_pool_return_netmem()

[Byungchul Park]

Now that page_pool_return_page() is for returning netmem, not struct
page, rename it to page_pool_return_netmem() to reflect what it does.

Signed-off-by: Byungchul Park <byungchul@sk.com>
Reviewed-by: Mina Almasry <almasrymina@google.com>
Reviewed-by: Toke Høiland-Jørgensen <toke@redhat.com>
Reviewed-by: Pavel Begunkov <asml.silence@gmail.com>
Reviewed-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>
---
 net/core/page_pool.c | 22 +++++++++++-----------
 1 file changed, 11 insertions(+), 11 deletions(-)

diff --git a/net/core/page_pool.c b/net/core/page_pool.c
index ba7cf3e3c32f..3bf25e554f96 100644
--- a/net/core/page_pool.c
+++ b/net/core/page_pool.c
@@ -371,7 +371,7 @@ struct page_pool *page_pool_create(const struct page_pool_params *params)
 }
 EXPORT_SYMBOL(page_pool_create);
 
-static void page_pool_return_page(struct page_pool *pool, netmem_ref netmem);
+static void page_pool_return_netmem(struct page_pool *pool, netmem_ref netmem);
 
 static noinline netmem_ref page_pool_refill_alloc_cache(struct page_pool *pool)
 {
@@ -409,7 +409,7 @@ static noinline netmem_ref page_pool_refill_alloc_cache(struct page_pool *pool)
 			 * (2) break out to fallthrough to alloc_pages_node.
 			 * This limit stress on page buddy alloactor.
 			 */
-			page_pool_return_page(pool, netmem);
+			page_pool_return_netmem(pool, netmem);
 			alloc_stat_inc(pool, waive);
 			netmem = 0;
 			break;
@@ -712,7 +712,7 @@ static __always_inline void __page_pool_release_page_dma(struct page_pool *pool,
  * a regular page (that will eventually be returned to the normal
  * page-allocator via put_page).
  */
-void page_pool_return_page(struct page_pool *pool, netmem_ref netmem)
+static void page_pool_return_netmem(struct page_pool *pool, netmem_ref netmem)
 {
 	int count;
 	bool put;
@@ -826,7 +826,7 @@ __page_pool_put_page(struct page_pool *pool, netmem_ref netmem,
 	 * will be invoking put_page.
 	 */
 	recycle_stat_inc(pool, released_refcnt);
-	page_pool_return_page(pool, netmem);
+	page_pool_return_netmem(pool, netmem);
 
 	return 0;
 }
@@ -869,7 +869,7 @@ void page_pool_put_unrefed_netmem(struct page_pool *pool, netmem_ref netmem,
 	if (netmem && !page_pool_recycle_in_ring(pool, netmem)) {
 		/* Cache full, fallback to free pages */
 		recycle_stat_inc(pool, ring_full);
-		page_pool_return_page(pool, netmem);
+		page_pool_return_netmem(pool, netmem);
 	}
 }
 EXPORT_SYMBOL(page_pool_put_unrefed_netmem);
@@ -912,7 +912,7 @@ static void page_pool_recycle_ring_bulk(struct page_pool *pool,
 	 * since put_page() with refcnt == 1 can be an expensive operation.
 	 */
 	for (; i < bulk_len; i++)
-		page_pool_return_page(pool, bulk[i]);
+		page_pool_return_netmem(pool, bulk[i]);
 }
 
 /**
@@ -995,7 +995,7 @@ static netmem_ref page_pool_drain_frag(struct page_pool *pool,
 		return netmem;
 	}
 
-	page_pool_return_page(pool, netmem);
+	page_pool_return_netmem(pool, netmem);
 	return 0;
 }
 
@@ -1009,7 +1009,7 @@ static void page_pool_free_frag(struct page_pool *pool)
 	if (!netmem || page_pool_unref_netmem(netmem, drain_count))
 		return;
 
-	page_pool_return_page(pool, netmem);
+	page_pool_return_netmem(pool, netmem);
 }
 
 netmem_ref page_pool_alloc_frag_netmem(struct page_pool *pool,
@@ -1076,7 +1076,7 @@ static void page_pool_empty_ring(struct page_pool *pool)
 			pr_crit("%s() page_pool refcnt %d violation\n",
 				__func__, netmem_ref_count(netmem));
 
-		page_pool_return_page(pool, netmem);
+		page_pool_return_netmem(pool, netmem);
 	}
 }
 
@@ -1109,7 +1109,7 @@ static void page_pool_empty_alloc_cache_once(struct page_pool *pool)
 	 */
 	while (pool->alloc.count) {
 		netmem = pool->alloc.cache[--pool->alloc.count];
-		page_pool_return_page(pool, netmem);
+		page_pool_return_netmem(pool, netmem);
 	}
 }
 
@@ -1253,7 +1253,7 @@ void page_pool_update_nid(struct page_pool *pool, int new_nid)
 	/* Flush pool alloc cache, as refill will check NUMA node */
 	while (pool->alloc.count) {
 		netmem = pool->alloc.cache[--pool->alloc.count];
-		page_pool_return_page(pool, netmem);
+		page_pool_return_netmem(pool, netmem);
 	}
 }
 EXPORT_SYMBOL(page_pool_update_nid);
-- 
2.17.1

[PATCH net-next v7 3/7] page_pool: rename __page_pool_release_page_dma() to __page_pool_release_netmem_dma()

[Byungchul Park]

Now that __page_pool_release_page_dma() is for releasing netmem, not
struct page, rename it to __page_pool_release_netmem_dma() to reflect
what it does.

Signed-off-by: Byungchul Park <byungchul@sk.com>
Reviewed-by: Mina Almasry <almasrymina@google.com>
Reviewed-by: Toke Høiland-Jørgensen <toke@redhat.com>
Reviewed-by: Pavel Begunkov <asml.silence@gmail.com>
Reviewed-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>
---
 net/core/page_pool.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/net/core/page_pool.c b/net/core/page_pool.c
index 3bf25e554f96..95ffa48c7c67 100644
--- a/net/core/page_pool.c
+++ b/net/core/page_pool.c
@@ -673,8 +673,8 @@ void page_pool_clear_pp_info(netmem_ref netmem)
 	netmem_set_pp(netmem, NULL);
 }
 
-static __always_inline void __page_pool_release_page_dma(struct page_pool *pool,
-							 netmem_ref netmem)
+static __always_inline void __page_pool_release_netmem_dma(struct page_pool *pool,
+							   netmem_ref netmem)
 {
 	struct page *old, *page = netmem_to_page(netmem);
 	unsigned long id;
@@ -721,7 +721,7 @@ static void page_pool_return_netmem(struct page_pool *pool, netmem_ref netmem)
 	if (static_branch_unlikely(&page_pool_mem_providers) && pool->mp_ops)
 		put = pool->mp_ops->release_netmem(pool, netmem);
 	else
-		__page_pool_release_page_dma(pool, netmem);
+		__page_pool_release_netmem_dma(pool, netmem);
 
 	/* This may be the last page returned, releasing the pool, so
 	 * it is not safe to reference pool afterwards.
@@ -1136,7 +1136,7 @@ static void page_pool_scrub(struct page_pool *pool)
 		}
 
 		xa_for_each(&pool->dma_mapped, id, ptr)
-			__page_pool_release_page_dma(pool, page_to_netmem(ptr));
+			__page_pool_release_netmem_dma(pool, page_to_netmem((struct page *)ptr));
 	}
 
 	/* No more consumers should exist, but producers could still
-- 
2.17.1

[PATCH net-next v7 4/7] page_pool: rename __page_pool_alloc_pages_slow() to __page_pool_alloc_netmems_slow()

[Byungchul Park]

Now that __page_pool_alloc_pages_slow() is for allocating netmem, not
struct page, rename it to __page_pool_alloc_netmems_slow() to reflect
what it does.

Signed-off-by: Byungchul Park <byungchul@sk.com>
Reviewed-by: Mina Almasry <almasrymina@google.com>
Reviewed-by: Toke Høiland-Jørgensen <toke@redhat.com>
---
 net/core/page_pool.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/net/core/page_pool.c b/net/core/page_pool.c
index 95ffa48c7c67..05e2e22a8f7c 100644
--- a/net/core/page_pool.c
+++ b/net/core/page_pool.c
@@ -544,8 +544,8 @@ static struct page *__page_pool_alloc_page_order(struct page_pool *pool,
 }
 
 /* slow path */
-static noinline netmem_ref __page_pool_alloc_pages_slow(struct page_pool *pool,
-							gfp_t gfp)
+static noinline netmem_ref __page_pool_alloc_netmems_slow(struct page_pool *pool,
+							  gfp_t gfp)
 {
 	const int bulk = PP_ALLOC_CACHE_REFILL;
 	unsigned int pp_order = pool->p.order;
@@ -615,7 +615,7 @@ netmem_ref page_pool_alloc_netmems(struct page_pool *pool, gfp_t gfp)
 	if (static_branch_unlikely(&page_pool_mem_providers) && pool->mp_ops)
 		netmem = pool->mp_ops->alloc_netmems(pool, gfp);
 	else
-		netmem = __page_pool_alloc_pages_slow(pool, gfp);
+		netmem = __page_pool_alloc_netmems_slow(pool, gfp);
 	return netmem;
 }
 EXPORT_SYMBOL(page_pool_alloc_netmems);
-- 
2.17.1

[PATCH net-next v7 5/7] netmem: use _Generic to cover const casting for page_to_netmem()

[Byungchul Park]

The current page_to_netmem() doesn't cover const casting resulting in
trying to cast const struct page * to const netmem_ref fails.

To cover the case, change page_to_netmem() to use macro and _Generic.

Signed-off-by: Byungchul Park <byungchul@sk.com>
Reviewed-by: Mina Almasry <almasrymina@google.com>
Reviewed-by: Toke Høiland-Jørgensen <toke@redhat.com>
Reviewed-by: Pavel Begunkov <asml.silence@gmail.com>
---
 include/net/netmem.h | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/include/net/netmem.h b/include/net/netmem.h
index e9eee8f680d5..535cf17b9134 100644
--- a/include/net/netmem.h
+++ b/include/net/netmem.h
@@ -213,10 +213,9 @@ static inline netmem_ref net_iov_to_netmem(struct net_iov *niov)
 	return (__force netmem_ref)((unsigned long)niov | NET_IOV);
 }
 
-static inline netmem_ref page_to_netmem(const struct page *page)
-{
-	return (__force netmem_ref)page;
-}
+#define page_to_netmem(p)	(_Generic((p),			\
+	const struct page * :	(__force const netmem_ref)(p),	\
+	struct page * :		(__force netmem_ref)(p)))
 
 /**
  * virt_to_netmem - convert virtual memory pointer to a netmem reference
-- 
2.17.1

[PATCH net-next v7 6/7] page_pool: make page_pool_get_dma_addr() just wrap page_pool_get_dma_addr_netmem()

[Byungchul Park]

The page pool members in struct page cannot be removed unless it's not
allowed to access any of them via struct page.

Do not access 'page->dma_addr' directly in page_pool_get_dma_addr() but
just wrap page_pool_get_dma_addr_netmem() safely.

Signed-off-by: Byungchul Park <byungchul@sk.com>
Reviewed-by: Mina Almasry <almasrymina@google.com>
Reviewed-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>
Reviewed-by: Toke Høiland-Jørgensen <toke@redhat.com>
Reviewed-by: Pavel Begunkov <asml.silence@gmail.com>
---
 include/net/page_pool/helpers.h | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)

diff --git a/include/net/page_pool/helpers.h b/include/net/page_pool/helpers.h
index 773fc65780b5..db180626be06 100644
--- a/include/net/page_pool/helpers.h
+++ b/include/net/page_pool/helpers.h
@@ -444,12 +444,7 @@ static inline dma_addr_t page_pool_get_dma_addr_netmem(netmem_ref netmem)
  */
 static inline dma_addr_t page_pool_get_dma_addr(const struct page *page)
 {
-	dma_addr_t ret = page->dma_addr;
-
-	if (PAGE_POOL_32BIT_ARCH_WITH_64BIT_DMA)
-		ret <<= PAGE_SHIFT;
-
-	return ret;
+	return page_pool_get_dma_addr_netmem(page_to_netmem(page));
 }
 
 static inline void __page_pool_dma_sync_for_cpu(const struct page_pool *pool,
-- 
2.17.1

[PATCH net-next v7 7/7] netmem: introduce a netmem API, virt_to_head_netmem()

[Byungchul Park]

To eliminate the use of struct page in page pool, the page pool code
should use netmem descriptor and APIs instead.

As part of the work, introduce a netmem API to convert a virtual address
to a head netmem allowing the code to use it rather than the existing
API, virt_to_head_page() for struct page.

Signed-off-by: Byungchul Park <byungchul@sk.com>
Reviewed-by: Toke Høiland-Jørgensen <toke@redhat.com>
Reviewed-by: Pavel Begunkov <asml.silence@gmail.com>
Reviewed-by: Mina Almasry <almasrymina@google.com>
---
 include/net/netmem.h | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/include/net/netmem.h b/include/net/netmem.h
index 535cf17b9134..c7b1fc4b6c28 100644
--- a/include/net/netmem.h
+++ b/include/net/netmem.h
@@ -314,6 +314,13 @@ static inline netmem_ref netmem_compound_head(netmem_ref netmem)
 	return page_to_netmem(compound_head(netmem_to_page(netmem)));
 }
 
+static inline netmem_ref virt_to_head_netmem(const void *x)
+{
+	netmem_ref netmem = virt_to_netmem(x);
+
+	return netmem_compound_head(netmem);
+}
+
 /**
  * __netmem_address - unsafely get pointer to the memory backing @netmem
  * @netmem: netmem reference to get the pointer for
-- 
2.17.1

Re: [PATCH net-next v7 0/7] Split netmem from struct page

[Byungchul Park]

On Wed, Jun 25, 2025 at 01:33:43PM +0900, Byungchul Park wrote:
> Hi all,
> 
> In this version, I'm posting non-controversial patches first since there

To Jakub and net folks,

I believe v7 doesn't include any controversial patches.  It'd be
appreciated to lemme know if any.

	Byungchul

> are pending works that should be based on this series so that those can
> be started shortly.  I will post the rest later.
> 
> The MM subsystem is trying to reduce struct page to a single pointer.
> The first step towards that is splitting struct page by its individual
> users, as has already been done with folio and slab.  This patchset does
> that for netmem which is used for page pools.
> 
> Matthew Wilcox tried and stopped the same work, you can see in:
> 
>    https://lore.kernel.org/linux-mm/20230111042214.907030-1-willy@infradead.org/
> 
> Mina Almasry already has done a lot fo prerequisite works by luck.  I
> stacked my patches on the top of his work e.i. netmem.
> 
> I focused on removing the page pool members in struct page this time,
> not moving the allocation code of page pool from net to mm.  It can be
> done later if needed.
> 
> The final patch removing the page pool fields will be submitted once
> all the converting work of page to netmem are done:
> 
>    1. converting of libeth_fqe by Tony Nguyen.
>    2. converting of mlx5 by Tariq Toukan.
>    3. converting of prueth_swdata.
>    4. converting of freescale driver.
> 
> For our discussion, I'm sharing what the final patch looks like the
> following.
> 
> 	Byungchul
> --8<--
> commit 1847d9890f798456b21ccb27aac7545303048492
> Author: Byungchul Park <byungchul@sk.com>
> Date:   Wed May 28 20:44:55 2025 +0900
> 
>     mm, netmem: remove the page pool members in struct page
>     
>     Now that all the users of the page pool members in struct page have been
>     gone, the members can be removed from struct page.
>     
>     However, since struct netmem_desc still uses the space in struct page,
>     the important offsets should be checked properly, until struct
>     netmem_desc has its own instance from slab.
>     
>     Remove the page pool members in struct page and modify static checkers
>     for the offsets.
>     
>     Signed-off-by: Byungchul Park <byungchul@sk.com>
> 
> diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
> index 32ba5126e221..db2fe0d0ebbf 100644
> --- a/include/linux/mm_types.h
> +++ b/include/linux/mm_types.h
> @@ -120,17 +120,6 @@ struct page {
>  			 */
>  			unsigned long private;
>  		};
> -		struct {	/* page_pool used by netstack */
> -			/**
> -			 * @pp_magic: magic value to avoid recycling non
> -			 * page_pool allocated pages.
> -			 */
> -			unsigned long pp_magic;
> -			struct page_pool *pp;
> -			unsigned long _pp_mapping_pad;
> -			unsigned long dma_addr;
> -			atomic_long_t pp_ref_count;
> -		};
>  		struct {	/* Tail pages of compound page */
>  			unsigned long compound_head;	/* Bit zero is set */
>  		};
> diff --git a/include/net/netmem.h b/include/net/netmem.h
> index 8f354ae7d5c3..3414f184d018 100644
> --- a/include/net/netmem.h
> +++ b/include/net/netmem.h
> @@ -42,11 +42,8 @@ struct netmem_desc {
>  	static_assert(offsetof(struct page, pg) == \
>  		      offsetof(struct netmem_desc, desc))
>  NETMEM_DESC_ASSERT_OFFSET(flags, _flags);
> -NETMEM_DESC_ASSERT_OFFSET(pp_magic, pp_magic);
> -NETMEM_DESC_ASSERT_OFFSET(pp, pp);
> -NETMEM_DESC_ASSERT_OFFSET(_pp_mapping_pad, _pp_mapping_pad);
> -NETMEM_DESC_ASSERT_OFFSET(dma_addr, dma_addr);
> -NETMEM_DESC_ASSERT_OFFSET(pp_ref_count, pp_ref_count);
> +NETMEM_DESC_ASSERT_OFFSET(lru, pp_magic);
> +NETMEM_DESC_ASSERT_OFFSET(mapping, _pp_mapping_pad);
>  #undef NETMEM_DESC_ASSERT_OFFSET
>  
>  /*
> ---
> Changes from v5 (no actual updates):
> 	1. Rebase on net-next/main as of Jun 25.
> 	2. Supplement a comment describing struct net_iov.
> 	3. Exclude a controversial patch, "page_pool: access ->pp_magic
> 	   through struct netmem_desc in page_pool_page_is_pp()".
> 	4. Exclude "netmem: remove __netmem_get_pp()" since the API
> 	   started to be used again by libeth.
> 
> Changes from v5 (no actual updates):
> 	1. Rebase on net-next/main as of Jun 20.
> 	2. Add given 'Reviewed-by's and 'Acked-by's, thanks to all.
> 	3. Add missing cc's.
> 
> Changes from v4:
> 	1. Add given 'Reviewed-by's, thanks to all.
> 	2. Exclude potentially controversial patches.
> 
> Changes from v3:
> 	1. Relocates ->owner and ->type of net_iov out of netmem_desc
> 	   and make them be net_iov specific.
> 	2. Remove __force when casting struct page to struct netmem_desc.
> 
> Changes from v2:
> 	1. Introduce a netmem API, virt_to_head_netmem(), and use it
> 	   when it's needed.
> 	2. Introduce struct netmem_desc as a new struct and union'ed
> 	   with the existing fields in struct net_iov.
> 	3. Make page_pool_page_is_pp() access ->pp_magic through struct
> 	   netmem_desc instead of struct page.
> 	4. Move netmem alloc APIs from include/net/netmem.h to
> 	   net/core/netmem_priv.h.
> 	5. Apply trivial feedbacks, thanks to Mina, Pavel, and Toke.
> 	6. Add given 'Reviewed-by's, thanks to Mina.
> 
> Changes from v1:
> 	1. Rebase on net-next's main as of May 26.
> 	2. Check checkpatch.pl, feedbacked by SJ Park.
> 	3. Add converting of page to netmem in mt76.
> 	4. Revert 'mlx5: use netmem descriptor and APIs for page pool'
> 	   since it's on-going by Tariq Toukan.  I will wait for his
> 	   work to be done.
> 	5. Revert 'page_pool: use netmem APIs to access page->pp_magic
> 	   in page_pool_page_is_pp()' since we need more discussion.
> 	6. Revert 'mm, netmem: remove the page pool members in struct
> 	   page' since there are some prerequisite works to remove the
> 	   page pool fields from struct page.  I can submit this patch
> 	   separatedly later.
> 	7. Cancel relocating a page pool member in struct page.
> 	8. Modify static assert for offests and size of struct
> 	   netmem_desc.
> 
> Changes from rfc:
> 	1. Rebase on net-next's main branch.
> 	   https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/
> 	2. Fix a build error reported by kernel test robot.
> 	   https://lore.kernel.org/all/202505100932.uzAMBW1y-lkp@intel.com/
> 	3. Add given 'Reviewed-by's, thanks to Mina and Ilias.
> 	4. Do static_assert() on the size of struct netmem_desc instead
> 	   of placing place-holder in struct page, feedbacked by
> 	   Matthew.
> 	5. Do struct_group_tagged(netmem_desc) on struct net_iov instead
> 	   of wholly renaming it to strcut netmem_desc, feedbacked by
> 	   Mina and Pavel.
> 
> Byungchul Park (7):
>   netmem: introduce struct netmem_desc mirroring struct page
>   page_pool: rename page_pool_return_page() to page_pool_return_netmem()
>   page_pool: rename __page_pool_release_page_dma() to
>     __page_pool_release_netmem_dma()
>   page_pool: rename __page_pool_alloc_pages_slow() to
>     __page_pool_alloc_netmems_slow()
>   netmem: use _Generic to cover const casting for page_to_netmem()
>   page_pool: make page_pool_get_dma_addr() just wrap
>     page_pool_get_dma_addr_netmem()
>   netmem: introduce a netmem API, virt_to_head_netmem()
> 
>  include/net/netmem.h            | 130 ++++++++++++++++++++++++++------
>  include/net/page_pool/helpers.h |   7 +-
>  net/core/page_pool.c            |  36 ++++-----
>  3 files changed, 124 insertions(+), 49 deletions(-)
> 
> 
> base-commit: 8dacfd92dbefee829ca555a860e86108fdd1d55b
> -- 
> 2.17.1

Re: [PATCH net-next v7 1/7] netmem: introduce struct netmem_desc mirroring struct page

[Jakub Kicinski]

On Wed, 25 Jun 2025 13:33:44 +0900 Byungchul Park wrote:
> +/* A memory descriptor representing abstract networking I/O vectors,
> + * generally for non-pages memory that doesn't have its corresponding
> + * struct page and needs to be explicitly allocated through slab.

I still don't get what your final object set is going to be.

We have 
 - CPU-readable buffers (struct page)
 - un-readable buffers (struct net_iov)
 - abstract reference which can be a pointer to either of the
   above two (bitwise netmem_ref)

You say you want to evacuate page pool state from struct page
so I'd expect you to add a type which can always be fed into
some form of $type_to_virt(). A type which can always be cast
to net_iov, but not vice versa. So why are you putting things
inside net_iov, not outside.

> + * net_iovs are allocated and used by networking code, and the size of
> + * the chunk is PAGE_SIZE.

FWIW not for long. Patches to make the size of net_iov configurable 
are in progress.

Re: [PATCH net-next v7 7/7] netmem: introduce a netmem API, virt_to_head_netmem()

[Jakub Kicinski]

On Wed, 25 Jun 2025 13:33:50 +0900 Byungchul Park wrote:
> To eliminate the use of struct page in page pool, the page pool code
> should use netmem descriptor and APIs instead.
> 
> As part of the work, introduce a netmem API to convert a virtual address
> to a head netmem allowing the code to use it rather than the existing
> API, virt_to_head_page() for struct page.

nit: this one needs a caller for the new function to be merged

Re: [PATCH net-next v7 0/7] Split netmem from struct page

[Jakub Kicinski]

On Thu, 26 Jun 2025 15:41:19 +0900 Byungchul Park wrote:
> To Jakub and net folks,
> 
> I believe v7 doesn't include any controversial patches.  It'd be
> appreciated to lemme know if any.

I'm still a bit lost. But indeed patches 2-6 look fine.
If you were to repost those 5 they'll go right in.

Re: [PATCH net-next v7 1/7] netmem: introduce struct netmem_desc mirroring struct page

[Byungchul Park]

On Thu, Jun 26, 2025 at 05:49:04PM -0700, Jakub Kicinski wrote:
> On Wed, 25 Jun 2025 13:33:44 +0900 Byungchul Park wrote:
> > +/* A memory descriptor representing abstract networking I/O vectors,
> > + * generally for non-pages memory that doesn't have its corresponding
> > + * struct page and needs to be explicitly allocated through slab.
> 
> I still don't get what your final object set is going to be.

The ultimate goal is:

   Remove the pp fields from struct page

The second important goal is:

   Introduce a network pp descriptor, netmem_desc

While working on these two goals, I added some extra patches too, to
clean up related code if it's obvious e.g. patches for renaming and so
on.

> We have
>  - CPU-readable buffers (struct page)
>  - un-readable buffers (struct net_iov)
>  - abstract reference which can be a pointer to either of the
>    above two (bitwise netmem_ref)
> 
> You say you want to evacuate page pool state from struct page
> so I'd expect you to add a type which can always be fed into
> some form of $type_to_virt(). A type which can always be cast
> to net_iov, but not vice versa. So why are you putting things
> inside net_iov, not outside.

The type, struct netmem_desc, is declared outside.  Even though it's
used overlaying on struct page *for now*, it will be dynamically
allocated through slab shortly - it's also one of mm's plan.

As you know, net_iov is working with the assumption that it overlays on
struct page *for now* indeed, when it comes to netmem_ref.  See the
following APIs as example:

static inline struct net_iov *__netmem_clear_lsb(netmem_ref netmem)
{
	return (struct net_iov *)((__force unsigned long)netmem & ~NET_IOV);
}

static inline void netmem_set_pp(netmem_ref netmem, struct page_pool *pool)
{
	__netmem_clear_lsb(netmem)->pp = pool;
}

I'd say, I replaced the overlaying (on struct page) part with a
well-defined struct, netmem_desc that will play the role of struct page
for pp usage, instead of a set of the current overlaying fields of
net_iov.

This 'introduction of netmem_desc' patch can be the base for network
code to use netmem_desc as pp descriptor instead of struct page.  That's
what I meant.

Am I missing something or got you wrong?  If yes, please explain in more
detail then I will get back with the answer.

	Byungchul

> > + * net_iovs are allocated and used by networking code, and the size of
> > + * the chunk is PAGE_SIZE.
> 
> FWIW not for long. Patches to make the size of net_iov configurable
> are in progress.

Re: [PATCH net-next v7 1/7] netmem: introduce struct netmem_desc mirroring struct page

[Jakub Kicinski]

On Fri, 27 Jun 2025 12:54:05 +0900 Byungchul Park wrote:
> On Thu, Jun 26, 2025 at 05:49:04PM -0700, Jakub Kicinski wrote:
> > On Wed, 25 Jun 2025 13:33:44 +0900 Byungchul Park wrote:  
> > > +/* A memory descriptor representing abstract networking I/O vectors,
> > > + * generally for non-pages memory that doesn't have its corresponding
> > > + * struct page and needs to be explicitly allocated through slab.  
> > 
> > I still don't get what your final object set is going to be.  
> 
> The ultimate goal is:
> 
>    Remove the pp fields from struct page
> 
> The second important goal is:
> 
>    Introduce a network pp descriptor, netmem_desc
> 
> While working on these two goals, I added some extra patches too, to
> clean up related code if it's obvious e.g. patches for renaming and so
> on.

Object set. Not objective.

> > We have
> >  - CPU-readable buffers (struct page)
> >  - un-readable buffers (struct net_iov)
> >  - abstract reference which can be a pointer to either of the
> >    above two (bitwise netmem_ref)
> > 
> > You say you want to evacuate page pool state from struct page
> > so I'd expect you to add a type which can always be fed into
> > some form of $type_to_virt(). A type which can always be cast
> > to net_iov, but not vice versa. So why are you putting things
> > inside net_iov, not outside.  
> 
> The type, struct netmem_desc, is declared outside.  Even though it's
> used overlaying on struct page *for now*, it will be dynamically
> allocated through slab shortly - it's also one of mm's plan.
> 
> As you know, net_iov is working with the assumption that it overlays on
> struct page *for now* indeed, when it comes to netmem_ref.  See the
> following APIs as example:
> 
> static inline struct net_iov *__netmem_clear_lsb(netmem_ref netmem)
> {
> 	return (struct net_iov *)((__force unsigned long)netmem & ~NET_IOV);
> }
> 
> static inline void netmem_set_pp(netmem_ref netmem, struct page_pool *pool)
> {
> 	__netmem_clear_lsb(netmem)->pp = pool;
> }
> 
> I'd say, I replaced the overlaying (on struct page) part with a
> well-defined struct, netmem_desc that will play the role of struct page
> for pp usage, instead of a set of the current overlaying fields of
> net_iov.
> 
> This 'introduction of netmem_desc' patch can be the base for network
> code to use netmem_desc as pp descriptor instead of struct page.  That's
> what I meant.
> 
> Am I missing something or got you wrong?  If yes, please explain in more
> detail then I will get back with the answer.

Ugh, you keep explaining the mechanics to me. Our goal here is not
just to move fields around and make it still compile :/

Let me ask you this way: you said "netmem_desc" will be allocated
thru slab "shortly". How will calling the equivalent of page_address()
on netmem_desc work at that stage? Feel free to refer me to the existing
docs if its covered..

Re: [PATCH net-next v7 1/7] netmem: introduce struct netmem_desc mirroring struct page

[Harry Yoo]

On Fri, Jun 27, 2025 at 05:37:30PM -0700, Jakub Kicinski wrote:
> On Fri, 27 Jun 2025 12:54:05 +0900 Byungchul Park wrote:
> > On Thu, Jun 26, 2025 at 05:49:04PM -0700, Jakub Kicinski wrote:
> > > On Wed, 25 Jun 2025 13:33:44 +0900 Byungchul Park wrote:  
> > > > +/* A memory descriptor representing abstract networking I/O vectors,
> > > > + * generally for non-pages memory that doesn't have its corresponding
> > > > + * struct page and needs to be explicitly allocated through slab.  
> > > 
> > > I still don't get what your final object set is going to be.  
> > 
> > The ultimate goal is:
> > 
> >    Remove the pp fields from struct page
> > 
> > The second important goal is:
> > 
> >    Introduce a network pp descriptor, netmem_desc
> > 
> > While working on these two goals, I added some extra patches too, to
> > clean up related code if it's obvious e.g. patches for renaming and so
> > on.
> 
> Object set. Not objective.
> 
> > > We have
> > >  - CPU-readable buffers (struct page)
> > >  - un-readable buffers (struct net_iov)
> > >  - abstract reference which can be a pointer to either of the
> > >    above two (bitwise netmem_ref)
> > > 
> > > You say you want to evacuate page pool state from struct page
> > > so I'd expect you to add a type which can always be fed into
> > > some form of $type_to_virt(). A type which can always be cast
> > > to net_iov, but not vice versa. So why are you putting things
> > > inside net_iov, not outside.  
> > 
> > The type, struct netmem_desc, is declared outside.  Even though it's
> > used overlaying on struct page *for now*, it will be dynamically
> > allocated through slab shortly - it's also one of mm's plan.
> > 
> > As you know, net_iov is working with the assumption that it overlays on
> > struct page *for now* indeed, when it comes to netmem_ref.  See the
> > following APIs as example:
> > 
> > static inline struct net_iov *__netmem_clear_lsb(netmem_ref netmem)
> > {
> > 	return (struct net_iov *)((__force unsigned long)netmem & ~NET_IOV);
> > }
> > 
> > static inline void netmem_set_pp(netmem_ref netmem, struct page_pool *pool)
> > {
> > 	__netmem_clear_lsb(netmem)->pp = pool;
> > }
> > 
> > I'd say, I replaced the overlaying (on struct page) part with a
> > well-defined struct, netmem_desc that will play the role of struct page
> > for pp usage, instead of a set of the current overlaying fields of
> > net_iov.
> > 
> > This 'introduction of netmem_desc' patch can be the base for network
> > code to use netmem_desc as pp descriptor instead of struct page.  That's
> > what I meant.
> > 
> > Am I missing something or got you wrong?  If yes, please explain in more
> > detail then I will get back with the answer.
> 
> Ugh, you keep explaining the mechanics to me. Our goal here is not
> just to move fields around and make it still compile :/
> 
> Let me ask you this way: you said "netmem_desc" will be allocated
> thru slab "shortly". How will calling the equivalent of page_address()
> on netmem_desc work at that stage? Feel free to refer me to the existing
> docs if its covered..

https://kernelnewbies.org/MatthewWilcox/Memdescs/Path
https://kernelnewbies.org/MatthewWilcox/Memdescs

May not be the exact document you're looking for,
but with this article I can imagine:

- The ultimate goal is to shrink struct page to eventually from 64 bytes
  to 8 bytes, by allocating only the minimum required metadata per 4k page
  statically and moving the rest of metadata to dynamically-allocated
  descriptors (netmem_desc, anon, file, ptdesc, zpdesc, etc.) using slab
  at page allocation time.

- We can't achieve that goal just yet, because several subsystems
  still use struct page fields for their own purposes.

  To achieve that, each of these subsystems needs to define
  its own descriptor, which, for now, overlays struct page, and should be
  converted to use the new descriptor.

  Eventually, these descriptors will be allocated using slab.

- For CPU-readable buffers, page->memdesc will point to a netmem_desc,
  with a lower bit set indicating that it's a netmem_desc rather than
  other type. Networking code will need to cast it to (netmem_desc *)
  and dereference it to access networking specific fields.

- The struct page array (vmemmap) will still be statically allocated
  at boot time (or during memory hotplug time).
  So no change in how page_address() works.

net_iovs will continue to be not associated with struct pages,
as the buffers don't have corresponding struct pages.
net_iovs are already allocated using slab.

HTH

-- 
Cheers,
Harry / Hyeonggon

Re: [PATCH net-next v7 1/7] netmem: introduce struct netmem_desc mirroring struct page

[Jakub Kicinski]

On Mon, 30 Jun 2025 08:34:48 +0900 Harry Yoo wrote:
> > Ugh, you keep explaining the mechanics to me. Our goal here is not
> > just to move fields around and make it still compile :/
> > 
> > Let me ask you this way: you said "netmem_desc" will be allocated
> > thru slab "shortly". How will calling the equivalent of page_address()
> > on netmem_desc work at that stage? Feel free to refer me to the existing
> > docs if its covered..  
> 
> https://kernelnewbies.org/MatthewWilcox/Memdescs/Path
> https://kernelnewbies.org/MatthewWilcox/Memdescs
> 
> May not be the exact document you're looking for,
> but with this article I can imagine:
> 
> - The ultimate goal is to shrink struct page to eventually from 64 bytes
>   to 8 bytes, by allocating only the minimum required metadata per 4k page
>   statically and moving the rest of metadata to dynamically-allocated
>   descriptors (netmem_desc, anon, file, ptdesc, zpdesc, etc.) using slab
>   at page allocation time.
> 
> - We can't achieve that goal just yet, because several subsystems
>   still use struct page fields for their own purposes.
> 
>   To achieve that, each of these subsystems needs to define
>   its own descriptor, which, for now, overlays struct page, and should be
>   converted to use the new descriptor.
> 
>   Eventually, these descriptors will be allocated using slab.
> 
> - For CPU-readable buffers, page->memdesc will point to a netmem_desc,
>   with a lower bit set indicating that it's a netmem_desc rather than
>   other type. Networking code will need to cast it to (netmem_desc *)
>   and dereference it to access networking specific fields.
> 
> - The struct page array (vmemmap) will still be statically allocated
>   at boot time (or during memory hotplug time).
>   So no change in how page_address() works.
> 
> net_iovs will continue to be not associated with struct pages,
> as the buffers don't have corresponding struct pages.
> net_iovs are already allocated using slab.

Thanks a lot, this clarifies things for me.

Unfortunately, I still think that it's hard to judge patches 1 and 7 
in context limited to this series, so let's proceed to reposting just
the "middle 5" patches.

Re: [PATCH net-next v7 1/7] netmem: introduce struct netmem_desc mirroring struct page

[Byungchul Park]

On Tue, Jul 01, 2025 at 04:45:08PM -0700, Jakub Kicinski wrote:
> On Mon, 30 Jun 2025 08:34:48 +0900 Harry Yoo wrote:
> > > Ugh, you keep explaining the mechanics to me. Our goal here is not
> > > just to move fields around and make it still compile :/
> > >
> > > Let me ask you this way: you said "netmem_desc" will be allocated
> > > thru slab "shortly". How will calling the equivalent of page_address()
> > > on netmem_desc work at that stage? Feel free to refer me to the existing
> > > docs if its covered..
> >
> > https://kernelnewbies.org/MatthewWilcox/Memdescs/Path
> > https://kernelnewbies.org/MatthewWilcox/Memdescs
> >
> > May not be the exact document you're looking for,
> > but with this article I can imagine:
> >
> > - The ultimate goal is to shrink struct page to eventually from 64 bytes
> >   to 8 bytes, by allocating only the minimum required metadata per 4k page
> >   statically and moving the rest of metadata to dynamically-allocated
> >   descriptors (netmem_desc, anon, file, ptdesc, zpdesc, etc.) using slab
> >   at page allocation time.
> >
> > - We can't achieve that goal just yet, because several subsystems
> >   still use struct page fields for their own purposes.
> >
> >   To achieve that, each of these subsystems needs to define
> >   its own descriptor, which, for now, overlays struct page, and should be
> >   converted to use the new descriptor.
> >
> >   Eventually, these descriptors will be allocated using slab.
> >
> > - For CPU-readable buffers, page->memdesc will point to a netmem_desc,
> >   with a lower bit set indicating that it's a netmem_desc rather than
> >   other type. Networking code will need to cast it to (netmem_desc *)
> >   and dereference it to access networking specific fields.
> >
> > - The struct page array (vmemmap) will still be statically allocated
> >   at boot time (or during memory hotplug time).
> >   So no change in how page_address() works.
> >
> > net_iovs will continue to be not associated with struct pages,
> > as the buffers don't have corresponding struct pages.
> > net_iovs are already allocated using slab.
> 
> Thanks a lot, this clarifies things for me.
> 
> Unfortunately, I still think that it's hard to judge patches 1 and 7
> in context limited to this series, so let's proceed to reposting just
> the "middle 5" patches.

Just in case, I sent v8 with the "middle 5" last week as you requested.
I'm convinced they are non-controversial but lemme know if any.

	Byungchul

Re: [PATCH net-next v7 1/7] netmem: introduce struct netmem_desc mirroring struct page

[Jakub Kicinski]

On Mon, 7 Jul 2025 09:21:41 +0900 Byungchul Park wrote:
> > Thanks a lot, this clarifies things for me.
> > 
> > Unfortunately, I still think that it's hard to judge patches 1 and 7
> > in context limited to this series, so let's proceed to reposting just
> > the "middle 5" patches.  
> 
> Just in case, I sent v8 with the "middle 5" last week as you requested.
> I'm convinced they are non-controversial but lemme know if any.

Please don't ping maintainers like this.
It was 4th of July weekend in the US.
We'll get to your patches when we do.

Re: [PATCH net-next v7 1/7] netmem: introduce struct netmem_desc mirroring struct page

[Harry Yoo]

On Tue, Jul 01, 2025 at 04:45:08PM -0700, Jakub Kicinski wrote:
> On Mon, 30 Jun 2025 08:34:48 +0900 Harry Yoo wrote:
> > > Ugh, you keep explaining the mechanics to me. Our goal here is not
> > > just to move fields around and make it still compile :/
> > > 
> > > Let me ask you this way: you said "netmem_desc" will be allocated
> > > thru slab "shortly". How will calling the equivalent of page_address()
> > > on netmem_desc work at that stage? Feel free to refer me to the existing
> > > docs if its covered..  
> > 
> > https://kernelnewbies.org/MatthewWilcox/Memdescs/Path
> > https://kernelnewbies.org/MatthewWilcox/Memdescs
> > 
> > May not be the exact document you're looking for,
> > but with this article I can imagine:
> > 
> > - The ultimate goal is to shrink struct page to eventually from 64 bytes
> >   to 8 bytes, by allocating only the minimum required metadata per 4k page
> >   statically and moving the rest of metadata to dynamically-allocated
> >   descriptors (netmem_desc, anon, file, ptdesc, zpdesc, etc.) using slab
> >   at page allocation time.
> > 
> > - We can't achieve that goal just yet, because several subsystems
> >   still use struct page fields for their own purposes.
> > 
> >   To achieve that, each of these subsystems needs to define
> >   its own descriptor, which, for now, overlays struct page, and should be
> >   converted to use the new descriptor.
> > 
> >   Eventually, these descriptors will be allocated using slab.
> > 
> > - For CPU-readable buffers, page->memdesc will point to a netmem_desc,
> >   with a lower bit set indicating that it's a netmem_desc rather than
> >   other type. Networking code will need to cast it to (netmem_desc *)
> >   and dereference it to access networking specific fields.
> > 
> > - The struct page array (vmemmap) will still be statically allocated
> >   at boot time (or during memory hotplug time).
> >   So no change in how page_address() works.
> > 
> > net_iovs will continue to be not associated with struct pages,
> > as the buffers don't have corresponding struct pages.
> > net_iovs are already allocated using slab.
> 
> Thanks a lot, this clarifies things for me.

You're welcome :)

> Unfortunately, I still think that it's hard to judge patches 1 and 7 
> in context limited to this series, so let's proceed to reposting just
> the "middle 5" patches.

Could you please share your thoughts on why it's hard to judge them and
what's missing from the series, such as in the comments, changelog, or
the cover letter?

-- 
Cheers,
Harry / Hyeonggon

Re: [PATCH net-next v7 1/7] netmem: introduce struct netmem_desc mirroring struct page

[Vlastimil Babka]

On 7/14/25 11:42, Harry Yoo wrote:
> On Tue, Jul 01, 2025 at 04:45:08PM -0700, Jakub Kicinski wrote:
>> Thanks a lot, this clarifies things for me.
> 
> You're welcome :)
> 
>> Unfortunately, I still think that it's hard to judge patches 1 and 7 
>> in context limited to this series, so let's proceed to reposting just
>> the "middle 5" patches.
> 
> Could you please share your thoughts on why it's hard to judge them and
> what's missing from the series, such as in the comments, changelog, or
> the cover letter?

I think we moved on in the discussion since then? "middle 5" patches are now
merged, and 1+7 was, along with more patches (that make the context less
limited hopefully), posted as v9/v10 now?

Re: [PATCH net-next v7 1/7] netmem: introduce struct netmem_desc mirroring struct page

[Harry Yoo]

On Mon, Jul 14, 2025 at 03:24:02PM +0200, Vlastimil Babka wrote:
> On 7/14/25 11:42, Harry Yoo wrote:
> > On Tue, Jul 01, 2025 at 04:45:08PM -0700, Jakub Kicinski wrote:
> >> Thanks a lot, this clarifies things for me.
> > 
> > You're welcome :)
> > 
> >> Unfortunately, I still think that it's hard to judge patches 1 and 7 
> >> in context limited to this series, so let's proceed to reposting just
> >> the "middle 5" patches.
> > 
> > Could you please share your thoughts on why it's hard to judge them and
> > what's missing from the series, such as in the comments, changelog, or
> > the cover letter?
> 
> I think we moved on in the discussion since then? "middle 5" patches are now
> merged, and 1+7 was, along with more patches (that make the context less
> limited hopefully), posted as v9/v10 now?

On v7 Jakub asked [1] [2] few questions about the patch series
and I answered [3].

[1] https://lore.kernel.org/linux-mm/20250626174904.4a6125c9@kernel.org
[2] https://lore.kernel.org/linux-mm/20250627173730.15b25a8c@kernel.org 
[3] https://lore.kernel.org/linux-mm/aGHNmKRng9H6kTqz@hyeyoo

To me Jakub's comment "Thanks a lot, this clarifies things for me"
and still "hard to judge patches 1 and 7 in context limited to this series"
after reading my clarification (which wasn't part of the series)
sounded like he wants some of the clarification to be added to
later versions of the patchset.

But I don't speak for Jakub, I'll leave the answer to him.
Maybe I just misinterpreted his reply.

-- 
Cheers,
Harry / Hyeonggon

Re: [PATCH net-next v7 1/7] netmem: introduce struct netmem_desc mirroring struct page

[Jakub Kicinski]

On Mon, 14 Jul 2025 22:58:31 +0900 Harry Yoo wrote:
> > > Could you please share your thoughts on why it's hard to judge them and
> > > what's missing from the series, such as in the comments, changelog, or
> > > the cover letter?  

My main concern (as shared on earlier revisions) is the type hierarchy
exposed to the drivers. Converting things back and forth or blindly
downcasting to netmem and upcasting back to the CPU-readable type is
no good.

Re: [PATCH net-next v7 1/7] netmem: introduce struct netmem_desc mirroring struct page

[Byungchul Park]

On Mon, Jul 14, 2025 at 06:47:43PM -0700, Jakub Kicinski wrote:
> 
> On Mon, 14 Jul 2025 22:58:31 +0900 Harry Yoo wrote:
> > > > Could you please share your thoughts on why it's hard to judge them and
> > > > what's missing from the series, such as in the comments, changelog, or
> > > > the cover letter?
> 
> My main concern (as shared on earlier revisions) is the type hierarchy
> exposed to the drivers. Converting things back and forth or blindly
> downcasting to netmem and upcasting back to the CPU-readable type is
> no good.

I understand your concern.  I removed a lot of converting things but
left essencial things, that is inevitable to remove accessing the pp
fields through struct page.  Is it still not okay with v10 [1]?

There are some points under disscussion with Mina but I'm curious about
how you think about the direction changed.

[1] https://lore.kernel.org/all/20250714120047.35901-1-byungchul@sk.com/

	Byungchul

Re: [PATCH net-next v7 1/7] netmem: introduce struct netmem_desc mirroring struct page

[Pavel Begunkov]

On 7/15/25 02:47, Jakub Kicinski wrote:
> On Mon, 14 Jul 2025 22:58:31 +0900 Harry Yoo wrote:
>>>> Could you please share your thoughts on why it's hard to judge them and
>>>> what's missing from the series, such as in the comments, changelog, or
>>>> the cover letter?
> 
> My main concern (as shared on earlier revisions) is the type hierarchy

Quick overview. struct netmem_desc is a common denominator b/w pp pages
and net_iov, and contains fields used by the pp generic path, e.g.
refcount, dma_addr. Before, pp was using type casting hacks to keep
code generic with some overhead on bit masking, now it'll be able to
look up netmem_desc from a netmem and use it directly.

That's pretty much what I was suggesting niov / page aliasing to be
1+ years ago, but unfortunately that didn't happen. It definitely
removes some type casting hackiness.

> exposed to the drivers. 

v10 adds a bunch of "pp_page_to_nmdesc(page)->pp" in the drivers,
Not sure I have a strong opinion, but it can be turned into a helper.

Converting things back and forth or blindly
> downcasting to netmem and upcasting back to the CPU-readable type is
> no good.

-- 
Pavel Begunkov