Re: [PATCH 00/18] Split netmem from struct page

[Byungchul Park <byungchul@sk.com>]

On Fri, May 23, 2025 at 03:20:27PM +0900, Taehee Yoo wrote:
> On Fri, May 23, 2025 at 12:36 PM Byungchul Park <byungchul@sk.com> wrote:
> >
> 
> Hi Byungchul,
> Thanks a lot for this work!
> 
> > The MM subsystem is trying to reduce struct page to a single pointer.
> > The first step towards that is splitting struct page by its individual
> > users, as has already been done with folio and slab.  This patchset does
> > that for netmem which is used for page pools.
> >
> > Matthew Wilcox tried and stopped the same work, you can see in:
> >
> >    https://lore.kernel.org/linux-mm/20230111042214.907030-1-willy@infradead.org/
> >
> > Mina Almasry already has done a lot fo prerequisite works by luck, he
> > said :).  I stacked my patches on the top of his work e.i. netmem.
> >
> > I focused on removing the page pool members in struct page this time,
> > not moving the allocation code of page pool from net to mm.  It can be
> > done later if needed.
> >
> > My rfc version of this work is:
> >
> >    https://lore.kernel.org/all/20250509115126.63190-1-byungchul@sk.com/
> >
> > There are still a lot of works to do, to remove the dependency on struct
> > page in the network subsystem.  I will continue to work on this after
> > this base patchset is merged.
> 
> There is a compile failure.

Thanks a lot.  I will fix it.

	Byungchul
> 
> In file included from drivers/net/ethernet/intel/libeth/rx.c:4:
> ./include/net/libeth/rx.h: In function ‘libeth_rx_sync_for_cpu’:
> ./include/net/libeth/rx.h:140:40: error: ‘struct page’ has no member named ‘pp’
>   140 |         page_pool_dma_sync_for_cpu(page->pp, page, fqe->offset, len);
>       |                                        ^~
> drivers/net/ethernet/intel/libeth/rx.c: In function ‘libeth_rx_recycle_slow’:
> drivers/net/ethernet/intel/libeth/rx.c:210:38: error: ‘struct page’
> has no member named ‘pp’
>   210 |         page_pool_recycle_direct(page->pp, page);
>       |                                      ^~
> make[7]: *** [scripts/Makefile.build:203:
> drivers/net/ethernet/intel/libeth/rx.o] Error 1
> make[6]: *** [scripts/Makefile.build:461:
> drivers/net/ethernet/intel/libeth] Error 2
> make[5]: *** [scripts/Makefile.build:461: drivers/net/ethernet/intel] Error 2
> make[5]: *** Waiting for unfinished jobs....
> 
> There are page->pp usecases in drivers/net
> ./drivers/net/ethernet/marvell/octeontx2/nic/otx2_txrx.c:1574:
>  } else if (page->pp) {
> ./drivers/net/ethernet/freescale/fec_main.c:1046:
>                  page_pool_put_page(page->pp, page, 0, false);
> ./drivers/net/ethernet/freescale/fec_main.c:1584:
>  page_pool_put_page(page->pp, page, 0, true);
> ./drivers/net/ethernet/freescale/fec_main.c:3351:
>          page_pool_put_page(page->pp, page, 0, false);
> ./drivers/net/ethernet/ti/icssg/icssg_prueth_sr1.c:370:
> page_pool_recycle_direct(page->pp, page);
> ./drivers/net/ethernet/ti/icssg/icssg_prueth_sr1.c:395:
> page_pool_recycle_direct(page->pp, page);
> ./drivers/net/ethernet/ti/icssg/icssg_common.c:111:
> page_pool_recycle_direct(page->pp, swdata->data.page);
> ./drivers/net/ethernet/intel/idpf/idpf_txrx.c:389:
> page_pool_put_full_page(rx_buf->page->pp, rx_buf->page, false);
> ./drivers/net/ethernet/intel/idpf/idpf_txrx.c:3254:     u32 hr =
> rx_buf->page->pp->p.offset;
> ./drivers/net/ethernet/intel/idpf/idpf_txrx.c:3286:     dst =
> page_address(hdr->page) + hdr->offset + hdr->page->pp->p.offset;
> ./drivers/net/ethernet/intel/idpf/idpf_txrx.c:3287:     src =
> page_address(buf->page) + buf->offset + buf->page->pp->p.offset;
> ./drivers/net/ethernet/intel/idpf/idpf_txrx.c:3305:     u32 hr =
> buf->page->pp->p.offset;
> ./drivers/net/ethernet/intel/libeth/rx.c:210:
> page_pool_recycle_direct(page->pp, page);
> ./drivers/net/ethernet/intel/iavf/iavf_txrx.c:1200:     u32 hr =
> rx_buffer->page->pp->p.offset;
> ./drivers/net/ethernet/intel/iavf/iavf_txrx.c:1217:     u32 hr =
> rx_buffer->page->pp->p.offset;
> ./drivers/net/wireless/mediatek/mt76/mt76.h:1800:
> page_pool_put_full_page(page->pp, page, allow_direct);
> ./include/net/libeth/rx.h:140:  page_pool_dma_sync_for_cpu(page->pp,
> page, fqe->offset, len);
> 
> Thanks a lot!
> Taehee Yoo
> 
> >
> > ---
> >
> > Changes from rfc:
> >         1. Rebase on net-next's main branch
> >            https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/
> >         2. Fix a build error reported by kernel test robot
> >            https://lore.kernel.org/all/202505100932.uzAMBW1y-lkp@intel.com/
> >         3. Add given 'Reviewed-by's, thanks to Mina and Ilias
> >         4. Do static_assert() on the size of struct netmem_desc instead
> >            of placing place-holder in struct page, feedbacked by Matthew
> >         5. Do struct_group_tagged(netmem_desc) on struct net_iov instead
> >            of wholly renaming it to strcut netmem_desc, feedbacked by
> >            Mina and Pavel
> >
> > Byungchul Park (18):
> >   netmem: introduce struct netmem_desc struct_group_tagged()'ed on
> >     struct net_iov
> >   netmem: introduce netmem alloc APIs to wrap page alloc APIs
> >   page_pool: use netmem alloc/put APIs in __page_pool_alloc_page_order()
> >   page_pool: rename __page_pool_alloc_page_order() to
> >     __page_pool_alloc_large_netmem()
> >   page_pool: use netmem alloc/put APIs in __page_pool_alloc_pages_slow()
> >   page_pool: rename page_pool_return_page() to page_pool_return_netmem()
> >   page_pool: use netmem put API in page_pool_return_netmem()
> >   page_pool: rename __page_pool_release_page_dma() to
> >     __page_pool_release_netmem_dma()
> >   page_pool: rename __page_pool_put_page() to __page_pool_put_netmem()
> >   page_pool: rename __page_pool_alloc_pages_slow() to
> >     __page_pool_alloc_netmems_slow()
> >   mlx4: use netmem descriptor and APIs for page pool
> >   page_pool: use netmem APIs to access page->pp_magic in
> >     page_pool_page_is_pp()
> >   mlx5: use netmem descriptor and APIs for page pool
> >   netmem: use _Generic to cover const casting for page_to_netmem()
> >   netmem: remove __netmem_get_pp()
> >   page_pool: make page_pool_get_dma_addr() just wrap
> >     page_pool_get_dma_addr_netmem()
> >   netdevsim: use netmem descriptor and APIs for page pool
> >   mm, netmem: remove the page pool members in struct page
> >
> >  drivers/net/ethernet/mellanox/mlx4/en_rx.c    |  46 ++++----
> >  drivers/net/ethernet/mellanox/mlx4/en_tx.c    |   8 +-
> >  drivers/net/ethernet/mellanox/mlx4/mlx4_en.h  |   4 +-
> >  drivers/net/ethernet/mellanox/mlx5/core/en.h  |   4 +-
> >  .../net/ethernet/mellanox/mlx5/core/en/xdp.c  |  18 ++--
> >  .../net/ethernet/mellanox/mlx5/core/en/xdp.h  |   2 +-
> >  .../net/ethernet/mellanox/mlx5/core/en_main.c |  15 ++-
> >  .../net/ethernet/mellanox/mlx5/core/en_rx.c   |  66 ++++++------
> >  drivers/net/netdevsim/netdev.c                |  18 ++--
> >  drivers/net/netdevsim/netdevsim.h             |   2 +-
> >  include/linux/mm.h                            |   5 +-
> >  include/linux/mm_types.h                      |  11 --
> >  include/linux/skbuff.h                        |  14 +++
> >  include/net/netmem.h                          | 101 ++++++++++--------
> >  include/net/page_pool/helpers.h               |  11 +-
> >  net/core/page_pool.c                          |  97 +++++++++--------
> >  16 files changed, 221 insertions(+), 201 deletions(-)
> >
> >
> > base-commit: f44092606a3f153bb7e6b277006b1f4a5b914cfc
> > --
> > 2.17.1
> >
> >