linux-mm.kvack.org archive mirror
 help / color / mirror / Atom feed
From: "Lorenzo Stoakes (ARM)" <ljs@kernel.org>
To: Luigi Rizzo <lrizzo@google.com>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	 David Hildenbrand <david@kernel.org>,
	Johannes Weiner <hannes@cmpxchg.org>,
	 Vlastimil Babka <vbabka@kernel.org>,
	Joerg Roedel <joro@8bytes.org>, Will Deacon <will@kernel.org>,
	 Robin Murphy <robin.murphy@arm.com>,
	Marek Szyprowski <m.szyprowski@samsung.com>,
	 Jens Axboe <axboe@kernel.dk>,
	"David S . Miller" <davem@davemloft.net>,
	 Eric Dumazet <edumazet@kernel.org>,
	Jakub Kicinski <kuba@kernel.org>,
	 Paolo Abeni <pabeni@redhat.com>,
	Luigi Rizzo <rizzo.unipi@gmail.com>,
	 "Liam R . Howlett" <liam@infradead.org>,
	Mike Rapoport <rppt@kernel.org>,
	 Suren Baghdasaryan <surenb@google.com>,
	Michal Hocko <mhocko@suse.com>,
	 Brendan Jackman <brendan.jackman@linux.dev>,
	Zi Yan <ziy@nvidia.com>, Harry Yoo <harry@kernel.org>,
	 Christoph Lameter <cl@gentwo.org>,
	David Rientjes <rientjes@google.com>,
	 Roman Gushchin <roman.gushchin@linux.dev>,
	Hao Li <hao.li@linux.dev>, Matthew Wilcox <willy@infradead.org>,
	 Jan Kara <jack@suse.cz>, Jann Horn <jannh@google.com>,
	 Catalin Marinas <catalin.marinas@arm.com>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	 "Rafael J . Wysocki" <rafael@kernel.org>,
	Danilo Krummrich <dakr@kernel.org>,
	 Jonathan Corbet <corbet@lwn.net>,
	Simon Horman <horms@kernel.org>,
	 Jesper Dangaard Brouer <hawk@kernel.org>,
	Ilias Apalodimas <ilias.apalodimas@linaro.org>,
	 Willem de Bruijn <willemb@google.com>,
	Kuniyuki Iwashima <kuniyu@google.com>,
	 Joshua Washington <joshwash@google.com>,
	Harshitha Ramamurthy <hramamurthy@google.com>,
	 Alexander Lobakin <aleksander.lobakin@intel.com>,
	Christoph Hellwig <hch@lst.de>, Jason Gunthorpe <jgg@ziepe.ca>,
	 Dragos Tatulea <dtatulea@nvidia.com>,
	Naman Gulati <namangulati@google.com>,
	 Pranjal Shrivastava <praan@google.com>,
	Mostafa Saleh <smostafa@google.com>,
	 Randy Dunlap <rdunlap@infradead.org>,
	linux-mm@kvack.org, iommu@lists.linux.dev,
	 netdev@vger.kernel.org, linux-block@vger.kernel.org,
	linux-fsdevel@vger.kernel.org,  driver-core@lists.linux.dev,
	linux-arm-kernel@lists.infradead.org, linux-api@vger.kernel.org,
	 linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [RFC PATCH v3 11/25] mm: Migrate user pages into PISA
Date: Fri, 20 Nov 2026 14:10:13 +0000	[thread overview]
Message-ID: <awBUG15auEfX2Uc6@gremlin> (raw)
In-Reply-To: <20261120010655.467825-12-lrizzo@google.com>

On Fri, Nov 20, 2026 at 01:06:41AM +0000, Luigi Rizzo wrote:
> Zero-copy DMA paths (such as O_DIRECT disk I/O and MSG_ZEROCOPY sockets)
> map each user page with a per-buffer IOMMU mapping and IOTLB flush.
> Moving user buffers into PISA blocks once allows subsequent DMA
> operations to reuse pre-mapped 2MB IOMMU entries.
>
> Add mm/pisa-user.c to migrate private anonymous order-0 user pages into
> per-node DMA_TO_DEVICE PISA allocators, controlled by
> pisa.user_max_pages (0 = disabled), via madvise(MADV_PISA) /
> MADV_NOPISA (also supported via process_madvise()) and
> pisa_upgrade_user_pages().

No, this isn't how madvise works. It's not a place to stuff custom hooks for a
feature you want.

>
> Large anonymous folios are split to order 0 first, while budget remains,
> with split_folio_to_list_keep_zero(): a variant of split_folio_to_list()
> that leaves zero-filled subpages mapped instead of remapping them to the
> shared zeropage, so that pages the caller is about to write to (e.g. the
> target of an O_DIRECT read) are still there to migrate.
>
> Signed-off-by: Luigi Rizzo <lrizzo@google.com>

No thanks. This whole approach seems completely wrong.

> ---
>  include/linux/huge_mm.h                |   7 +
>  include/linux/pisa-mm.h                |   8 +
>  include/linux/pisa.h                   |  18 ++
>  include/linux/swap.h                   |   1 +
>  include/uapi/asm-generic/mman-common.h |   2 +
>  mm/Makefile                            |   2 +-
>  mm/huge_memory.c                       |  36 ++-
>  mm/madvise.c                           |  19 ++
>  mm/pisa-alloc.c                        |   2 +
>  mm/pisa-priv.h                         |   2 +
>  mm/pisa-user.c                         | 394 +++++++++++++++++++++++++
>  11 files changed, 483 insertions(+), 8 deletions(-)
>  create mode 100644 mm/pisa-user.c
>
> diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
> index fac05120ab774..d679de8cf261d 100644
> --- a/include/linux/huge_mm.h
> +++ b/include/linux/huge_mm.h
> @@ -404,6 +404,7 @@ int __split_huge_page_to_list_to_order(struct page *page, struct list_head *list
>  int folio_split_unmapped(struct folio *folio, unsigned int new_order);
>  unsigned int min_order_for_split(struct folio *folio);
>  int split_folio_to_list(struct folio *folio, struct list_head *list);
> +int split_folio_to_list_keep_zero(struct folio *folio, struct list_head *list);
>  int folio_check_splittable(struct folio *folio, unsigned int new_order,
>  			   enum split_type split_type);
>  int folio_split(struct folio *folio, unsigned int new_order, struct page *page,
> @@ -671,6 +672,12 @@ static inline int split_folio_to_list(struct folio *folio, struct list_head *lis
>  	return -EINVAL;
>  }
>
> +static inline int split_folio_to_list_keep_zero(struct folio *folio, struct list_head *list)
> +{
> +	VM_WARN_ON_ONCE_FOLIO(1, folio);
> +	return -EINVAL;
> +}
> +
>  static inline int folio_split(struct folio *folio, unsigned int new_order,
>  		struct page *page, struct list_head *list)
>  {
> diff --git a/include/linux/pisa-mm.h b/include/linux/pisa-mm.h
> index d55a42397c4b3..1e82e081b4004 100644
> --- a/include/linux/pisa-mm.h
> +++ b/include/linux/pisa-mm.h
> @@ -11,6 +11,8 @@ unsigned long __pisa_alloc_pages_bulk(gfp_t gfp, unsigned int order,
>  				      int preferred_nid, int nr_pages,
>  				      struct page **page_array, bool frozen);
>  bool __pisa_free_page(struct page *page, unsigned int order);
> +int pisa_madvise(struct vm_area_struct *vma, unsigned long start,
> +		 unsigned long end, bool to_pisa);
>
>  /*
>   * ============================================================================
> @@ -77,5 +79,11 @@ static inline bool pisa_free_page(struct page *page, unsigned int order)
>  	return false;
>  }
>
> +static inline int pisa_madvise(struct vm_area_struct *vma, unsigned long start,
> +			       unsigned long end, bool to_pisa)
> +{
> +	return -EINVAL;
> +}
> +
>  #endif /* CONFIG_PISA */
>  #endif /* _LINUX_PISA_MM_H */
> diff --git a/include/linux/pisa.h b/include/linux/pisa.h
> index edd34771ad1c4..9321e7ff00042 100644
> --- a/include/linux/pisa.h
> +++ b/include/linux/pisa.h
> @@ -14,6 +14,7 @@
>
>  struct iommu_domain;
>  struct iova_domain;
> +struct iov_iter;
>
>  /**
>   * struct pisa_iova - A domain's IOVA window reserved for PISA pages
> @@ -70,9 +71,11 @@ enum {
>   * Static keys guarding all kernel hooks:
>   * - @pisa_key: enabled on first device opt-in (pisa_meta_init()).
>   * - @pisa_recycle_check_key: enabled when pisa.compact or pisa.target_usage_percent > 0.
> + * - @pisa_user_key: enabled when pisa.user_max_pages > 0.
>   */
>  DECLARE_STATIC_KEY_FALSE(pisa_key);
>  DECLARE_STATIC_KEY_FALSE(pisa_recycle_check_key);
> +DECLARE_STATIC_KEY_FALSE(pisa_user_key);
>
>  extern u32 *pisa_pfn2slot;
>  extern unsigned int pisa_nr_slots;
> @@ -89,12 +92,19 @@ void *__pisa_alloc_map_coherent_dev(struct device *dev, size_t size,
>  				    unsigned long attrs);
>  bool __pisa_has_free_blocks_rx(unsigned int order, int nid);
>  bool __pisa_page_can_recycle(const struct page *page);
> +bool pisa_upgrade_user_pages(struct iov_iter *iter, struct page **pages,
> +			     size_t size, size_t offset);
>
>  static inline bool pisa_enabled(void)
>  {
>  	return static_branch_unlikely(&pisa_key);
>  }
>
> +static inline bool pisa_user_enabled(void)
> +{
> +	return static_branch_unlikely(&pisa_user_key);
> +}
> +
>  static inline u32 pisa_pfn_flags(unsigned long pfn)
>  {
>  	unsigned long frame = pfn >> PISA_ORDER;
> @@ -195,6 +205,7 @@ static inline bool pisa_page_can_recycle(const struct page *page)
>
>  static inline int pisa_meta_init(void) { return -EOPNOTSUPP; }
>  static inline bool pisa_enabled(void) { return false; }
> +static inline bool pisa_user_enabled(void) { return false; }
>  static inline bool is_pisa_pfn(unsigned long pfn) { return false; }
>  static inline bool is_pisa_page(const struct page *page) { return false; }
>  static inline bool is_pisa_phys(phys_addr_t phys) { return false; }
> @@ -228,6 +239,13 @@ static inline bool pisa_page_can_recycle(const struct page *page)
>  	return true;
>  }
>
> +static inline bool pisa_upgrade_user_pages(struct iov_iter *iter,
> +					   struct page **pages, size_t size,
> +					   size_t offset)
> +{
> +	return false;
> +}
> +
>  #endif /* CONFIG_PISA */
>
>  #if defined(CONFIG_PISA) && defined(CONFIG_IOMMU_DMA)
> diff --git a/include/linux/swap.h b/include/linux/swap.h
> index 4f686709d71f0..3736786ccc5a3 100644
> --- a/include/linux/swap.h
> +++ b/include/linux/swap.h
> @@ -314,6 +314,7 @@ extern unsigned long totalreserve_pages;
>  /* linux/mm/folio.c */
>  void folio_add_lru(struct folio *folio);
>  void folio_mark_accessed(struct folio *folio);
> +void lru_add_drain(void);
>  void lru_add_drain_all(void);
>
>  enum lru_cache_drained {
> diff --git a/include/uapi/asm-generic/mman-common.h b/include/uapi/asm-generic/mman-common.h
> index ef1c27fa3c570..c4332db8e56ff 100644
> --- a/include/uapi/asm-generic/mman-common.h
> +++ b/include/uapi/asm-generic/mman-common.h
> @@ -81,6 +81,8 @@
>
>  #define MADV_GUARD_INSTALL 102		/* fatal signal on access to range */
>  #define MADV_GUARD_REMOVE 103		/* unguard range */
> +#define MADV_PISA		210	/* move pages into PISA memory */
> +#define MADV_NOPISA		211	/* move pages out of PISA memory */
>
>  /* compatibility flags */
>  #define MAP_FILE	0
> diff --git a/mm/Makefile b/mm/Makefile
> index 7d624e4bad456..d6b54a6eb3dc0 100644
> --- a/mm/Makefile
> +++ b/mm/Makefile
> @@ -147,7 +147,7 @@ obj-$(CONFIG_GENERIC_IOREMAP) += ioremap.o
>  obj-$(CONFIG_SHRINKER_DEBUG) += shrinker_debug.o
>  obj-$(CONFIG_EXECMEM) += execmem.o
>  obj-$(CONFIG_PISA) += pisa.o
> -pisa-y := pisa-meta.o pisa-alloc.o pisa-node.o
> +pisa-y := pisa-meta.o pisa-alloc.o pisa-node.o pisa-user.o
>  obj-$(CONFIG_PISA_META_KUNIT_TEST) += pisa-kunit.o
>  obj-$(CONFIG_TMPFS_QUOTA) += shmem_quota.o
>  obj-$(CONFIG_LAZY_MMU_MODE_KUNIT_TEST) += tests/lazy_mmu_mode_kunit.o
> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> index 7791872f14af7..ffa3912f2418b 100644
> --- a/mm/huge_memory.c
> +++ b/mm/huge_memory.c
> @@ -4113,7 +4113,8 @@ static void folio_reset_partially_mapped(struct folio *folio)
>
>  static int __folio_freeze_split_anon(struct folio *folio,
>  		unsigned int new_order, struct page *split_at, bool do_lru,
> -		struct list_head *list, enum split_type split_type)
> +		struct list_head *list, enum split_type split_type,
> +		bool keep_zero)
>  {
>  	struct folio *end_folio = folio_next(folio);
>  	struct swap_cluster_info *ci = NULL;
> @@ -4207,7 +4208,13 @@ static int __folio_freeze_split_anon(struct folio *folio,
>  out_no_split:
>  	local_irq_enable();
>  	if (anon_vma) {
> -		if (!ret && !folio_is_device_private(folio))
> +		/*
> +		 * Zero-filled anon subpages are normally unmapped, freed and
> +		 * replaced by the shared zeropage. A @keep_zero caller needs
> +		 * the subpages themselves, e.g. to migrate them before a
> +		 * device writes to them.
> +		 */
> +		if (!ret && !folio_is_device_private(folio) && !keep_zero)
>  			ttu_flags = TTU_USE_SHARED_ZEROPAGE;
>  		remap_anon_folio(folio, 1 << old_order, ttu_flags);
>  	}
> @@ -4387,6 +4394,7 @@ static int __folio_freeze_split_file(struct folio *folio,
>   * @lock_at: a page within @folio to be left locked to caller
>   * @list: after-split folios will be put on it if non NULL
>   * @split_type: perform uniform split or not (non-uniform split)
> + * @keep_zero: do not remap zero-filled anon subpages to the shared zeropage
>   *
>   * It calls __split_frozen_folio() to perform uniform and non-uniform split.
>   * It is in charge of checking whether the split is supported or not and
> @@ -4402,7 +4410,8 @@ static int __folio_freeze_split_file(struct folio *folio,
>   */
>  static int __folio_split(struct folio *folio, unsigned int new_order,
>  		struct page *split_at, struct page *lock_at,
> -		struct list_head *list, enum split_type split_type)
> +		struct list_head *list, enum split_type split_type,
> +		bool keep_zero)
>  {
>  	struct folio *end_folio = folio_next(folio);
>  	const bool is_anon = folio_test_anon(folio);
> @@ -4432,7 +4441,8 @@ static int __folio_split(struct folio *folio, unsigned int new_order,
>
>  	if (is_anon)
>  		ret = __folio_freeze_split_anon(folio, new_order, split_at,
> -						true, list, split_type);
> +						true, list, split_type,
> +						keep_zero);
>  	else
>  		ret = __folio_freeze_split_file(folio, new_order, split_at,
>  						list, split_type);
> @@ -4495,7 +4505,7 @@ int folio_split_unmapped(struct folio *folio, unsigned int new_order)
>  	VM_WARN_ON_ONCE_FOLIO(!folio_test_anon(folio), folio);
>
>  	return __folio_freeze_split_anon(folio, new_order, &folio->page,
> -					 false, NULL, SPLIT_TYPE_UNIFORM);
> +					 false, NULL, SPLIT_TYPE_UNIFORM, false);
>  }
>
>  /*
> @@ -4551,7 +4561,7 @@ int __split_huge_page_to_list_to_order(struct page *page, struct list_head *list
>  	struct folio *folio = page_folio(page);
>
>  	return __folio_split(folio, new_order, &folio->page, page, list,
> -			     SPLIT_TYPE_UNIFORM);
> +			     SPLIT_TYPE_UNIFORM, false);
>  }
>
>  /**
> @@ -4582,7 +4592,7 @@ int folio_split(struct folio *folio, unsigned int new_order,
>  		struct page *split_at, struct list_head *list)
>  {
>  	return __folio_split(folio, new_order, split_at, &folio->page, list,
> -			     SPLIT_TYPE_NON_UNIFORM);
> +			     SPLIT_TYPE_NON_UNIFORM, false);
>  }
>
>  /**
> @@ -4617,6 +4627,18 @@ int split_folio_to_list(struct folio *folio, struct list_head *list)
>  	return split_huge_page_to_list_to_order(&folio->page, list, 0);
>  }
>
> +/*
> + * As split_folio_to_list(), but zero-filled subpages of an anon folio stay
> + * mapped instead of being remapped to the shared zeropage. For callers that
> + * are about to take write references on the subpages: PISA uses it to migrate
> + * the pages of an I/O buffer, e.g. the target of an O_DIRECT read.
> + */
> +int split_folio_to_list_keep_zero(struct folio *folio, struct list_head *list)
> +{
> +	return __folio_split(folio, 0, &folio->page, &folio->page, list,
> +			     SPLIT_TYPE_UNIFORM, true);
> +}
> +
>  /*
>   * __folio_unqueue_deferred_split() is not to be called directly:
>   * the folio_unqueue_deferred_split() inline wrapper in mm/internal.h
> diff --git a/mm/madvise.c b/mm/madvise.c
> index 0214efd2a014f..46981ad295aa5 100644
> --- a/mm/madvise.c
> +++ b/mm/madvise.c
> @@ -6,6 +6,7 @@
>   * Copyright (C) 2002  Christoph Hellwig
>   */
>
> +#include <linux/pisa-mm.h>
>  #include <linux/mman.h>
>  #include <linux/pagemap.h>
>  #include <linux/syscalls.h>
> @@ -1393,6 +1394,11 @@ static int madvise_vma_behavior(struct madvise_behavior *madv_behavior)
>  		return madvise_guard_install(madv_behavior);
>  	case MADV_GUARD_REMOVE:
>  		return madvise_guard_remove(madv_behavior);
> +#ifdef MADV_PISA
> +	case MADV_PISA:
> +	case MADV_NOPISA:
> +		return pisa_madvise(vma, range->start, range->end, behavior == MADV_PISA);
> +#endif
>
>  	/* The below behaviours update VMAs via madvise_update_vma(). */
>
> @@ -1582,6 +1588,11 @@ madvise_behavior_valid(int behavior)
>  	case MADV_HWPOISON:
>  #endif
>  		return true;
> +#ifdef MADV_PISA
> +	case MADV_PISA:
> +	case MADV_NOPISA:
> +		return IS_ENABLED(CONFIG_PISA);
> +#endif
>
>  	default:
>  		return false;
> @@ -1596,6 +1607,10 @@ static bool process_madvise_remote_valid(int behavior)
>  	case MADV_PAGEOUT:
>  	case MADV_WILLNEED:
>  	case MADV_COLLAPSE:
> +#ifdef MADV_PISA
> +	case MADV_PISA:
> +	case MADV_NOPISA:
> +#endif
>  		return true;
>  	default:
>  		return false;
> @@ -1782,6 +1797,10 @@ static enum madvise_lock_mode get_lock_mode(struct madvise_behavior *madv_behavi
>  	case MADV_POPULATE_READ:
>  	case MADV_POPULATE_WRITE:
>  	case MADV_COLLAPSE:
> +#ifdef MADV_PISA
> +	case MADV_PISA:
> +	case MADV_NOPISA:
> +#endif
>  		return MADVISE_MMAP_READ_LOCK;
>  	case MADV_GUARD_INSTALL:
>  	case MADV_GUARD_REMOVE:
> diff --git a/mm/pisa-alloc.c b/mm/pisa-alloc.c
> index ccba2bf01bd59..2213a56de1242 100644
> --- a/mm/pisa-alloc.c
> +++ b/mm/pisa-alloc.c
> @@ -1432,6 +1432,8 @@ static int __init pisa_init(void)
>  				  "pisa_ro_o%u_n%d", SKB_FRAG_PAGE_ORDER, nid);
>  	}
>
> +	pisa_user_boot_enable();
> +
>  	/* Dedicated WQ_MEM_RECLAIM workqueue so reclaim never blocks behind system_wq. */
>  	pisa_wq = alloc_workqueue("pisa", WQ_MEM_RECLAIM, 0);
>  	if (!pisa_wq)
> diff --git a/mm/pisa-priv.h b/mm/pisa-priv.h
> index 9f01173d30f5f..b18d461198ac5 100644
> --- a/mm/pisa-priv.h
> +++ b/mm/pisa-priv.h
> @@ -133,6 +133,7 @@ static inline long pisa_alloc_held_pages(const struct pisa_alloc *pa)
>  struct pisa_node {
>  	unsigned long		next_alloc;	/* atomic alloc backoff jiffies */
>  	struct pisa_alloc	*pa[2][PISA_MAX_ALLOC_ORDER + 1]; /* [0]=RW, [1]=RO */
> +	struct pisa_alloc	*user_pa;	/* user page migration pa */
>
>  	spinlock_t		lock;		/* protects @pages and @nr_pages */
>  	unsigned int		nr_pages;	/* entries in @pages */
> @@ -245,6 +246,7 @@ void pisa_schedule_scrub_all(void);
>  struct pisa_meta *pisa_reservoir_pop(int nid, gfp_t gfp, bool *clean);
>  unsigned long pisa_reservoir_count(void);
>  unsigned long pisa_reservoir_shrink(struct list_head *release_list, unsigned long nr_to_scan);
> +void pisa_user_boot_enable(void);
>
>  #endif /* CONFIG_PISA */
>  #endif /* _MM_PISA_PRIV_H */
> diff --git a/mm/pisa-user.c b/mm/pisa-user.c
> new file mode 100644
> index 0000000000000..6c3efaee4ecda
> --- /dev/null
> +++ b/mm/pisa-user.c
> @@ -0,0 +1,394 @@
> +// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause
> +/*
> + * PISA user and page-cache folios: allocate or migrate order-0 folios into
> + * per-node PISA allocators so zero-copy TX and block I/O use the PISA IOMMU mappings.
> + */
> +#include <linux/pisa.h>
> +#include <linux/gfp.h>
> +#include <linux/huge_mm.h>
> +#include <linux/migrate.h>
> +#include <linux/mm.h>
> +#include <linux/moduleparam.h>
> +#include <linux/nodemask.h>
> +#include <linux/pagewalk.h>
> +#include <linux/swap.h>
> +#include <linux/uio.h>
> +
> +#include "pisa-priv.h"
> +
> +/* Soft cap on user (PTE) pages held in PISA blocks, in PISA pages (0 = disabled). */
> +static unsigned long pisa_user_max_pages __read_mostly;
> +DEFINE_STATIC_KEY_FALSE(pisa_user_key);
> +
> +/* Create per-node user allocators and enable pisa_user_key (serialized by param_lock/init). */
> +static int pisa_user_init(void)
> +{
> +	int nid;
> +
> +	if (pisa_user_enabled())
> +		return 0;
> +	if (pisa_meta_init())
> +		return -ENOMEM;
> +	for_each_node_state(nid, N_MEMORY) {
> +		if (!pisa_alloc_ensure(&pisa_nodes[nid].user_pa, nid, 0,
> +				       DMA_TO_DEVICE, false, "user_tx_n%d", nid))
> +			return -ENOMEM;
> +	}
> +	static_branch_enable(&pisa_user_key);
> +	return 0;
> +}
> +
> +static int pisa_user_max_set(const char *val, const struct kernel_param *kp)
> +{
> +	unsigned long max_pages;
> +	int ret;
> +
> +	ret = kstrtoul(val, 0, &max_pages);
> +	if (ret)
> +		return ret;
> +	WRITE_ONCE(pisa_user_max_pages, max_pages);
> +	if (!max_pages) {
> +		static_branch_disable(&pisa_user_key);
> +	} else if (pisa_nodes[first_memory_node].pa[0][0] && pisa_user_init()) {
> +		WRITE_ONCE(pisa_user_max_pages, 0);
> +		return -ENOMEM;
> +	}
> +	return 0;
> +}
> +
> +static const struct kernel_param_ops pisa_user_max_ops = {
> +	.set = pisa_user_max_set,
> +	.get = param_get_ulong,
> +};
> +module_param_cb(user_max_pages, &pisa_user_max_ops, &pisa_user_max_pages, 0644);
> +
> +/*
> + * Initialize per-node user allocators if pisa.user_max_pages was set on
> + * the kernel command line before pisa_init() ran.
> + */
> +void __init pisa_user_boot_enable(void)
> +{
> +	if (READ_ONCE(pisa_user_max_pages))
> +		pisa_user_init();
> +}
> +
> +/* User pages currently held in PISA blocks. */
> +static unsigned long pisa_user_in_use(void)
> +{
> +	unsigned long used = 0;
> +	int nid;
> +
> +	for_each_node_state(nid, N_MEMORY) {
> +		/* Pairs with smp_store_release() in pisa_user_init(). */
> +		const struct pisa_alloc *pa = smp_load_acquire(&pisa_nodes[nid].user_pa);
> +
> +		used += pa ? READ_ONCE(pa->blocks_in_use) : 0;
> +	}
> +	return used;
> +}
> +
> +/* Pages that can still be moved in before reaching pisa.user_max_pages. */
> +static long pisa_user_budget(void)
> +{
> +	return (long)((READ_ONCE(pisa_user_max_pages) << PISA_ORDER) - pisa_user_in_use());
> +}
> +
> +/* Is @folio worth moving into (@to_pisa) or out of PISA memory? */
> +static bool pisa_user_want(struct folio *folio, bool to_pisa)
> +{
> +	if (!folio_test_anon(folio) || folio_is_zone_device(folio) ||
> +	    folio_test_ksm(folio) || is_huge_zero_folio(folio) ||
> +	    (folio_test_large(folio) && !to_pisa) ||
> +	    folio_maybe_mapped_shared(folio))
> +		return false;
> +	return is_pisa_pfn(folio_pfn(folio)) != to_pisa;
> +}
> +
> +/* Return true if @folio has references beyond its mappings, swap cache, and @extra. */
> +static bool pisa_user_busy(struct folio *folio, int extra)
> +{
> +	return folio_ref_count(folio) > folio_expected_ref_count(folio) + extra;
> +}
> +
> +struct pisa_user_mtc {
> +	gfp_t	gfp;
> +	long	budget;
> +};
> +
> +/* new_folio_t for migration into PISA memory. */
> +static struct folio *pisa_user_new_folio(struct folio *src, unsigned long private)
> +{
> +	struct pisa_user_mtc *mtc = (void *)private;
> +	int nid = folio_nid(src);
> +	struct page *page;
> +
> +	/* Callers split large folios first (see pisa_user_split()); none is expected here. */
> +	if (mtc->budget <= 0 || folio_test_large(src))
> +		return NULL;
> +	page = pisa_alloc(pisa_nodes[nid].user_pa, mtc->gfp);
> +	if (!page)
> +		return NULL;
> +	mtc->budget--;
> +	return page_folio(page);
> +}
> +
> +/* free_folio_t: give back an unused destination block and the budget it was charged. */
> +static void pisa_user_put_folio(struct folio *dst, unsigned long private)
> +{
> +	struct pisa_user_mtc *mtc = (void *)private;
> +
> +	mtc->budget++;
> +	folio_put(dst);
> +}
> +
> +/* new_folio_t for migration out of PISA memory: allocators never serve these. */
> +static struct folio *pisa_user_old_folio(struct folio *src, unsigned long private)
> +{
> +	return __folio_alloc_node(GFP_HIGHUSER_MOVABLE | __GFP_NOWARN, 0, folio_nid(src));
> +}
> +
> +/* Migrate the isolated folios on @list; failures are put back. */
> +static void pisa_user_migrate(struct list_head *list, bool to_pisa, gfp_t gfp)
> +{
> +	struct pisa_user_mtc mtc = { .gfp = gfp, .budget = LONG_MAX };
> +	enum migrate_mode mode = MIGRATE_SYNC;
> +
> +	if (list_empty(list))
> +		return;
> +	if (to_pisa) {
> +		mtc.budget = pisa_user_budget();
> +		if (!gfpflags_allow_blocking(gfp))
> +			mode = MIGRATE_ASYNC;
> +	}
> +	if (migrate_pages(list, to_pisa ? pisa_user_new_folio : pisa_user_old_folio,
> +			  to_pisa ? pisa_user_put_folio : NULL, (unsigned long)&mtc,
> +			  mode, MR_SYSCALL, NULL))
> +		putback_movable_pages(list);
> +}
> +
> +/*
> + * Split large anon @folio to order 0, with the caller holding the only extra
> + * reference. Zero-filled subpages stay mapped instead of being replaced by the
> + * shared zeropage, so that a GUP for write that follows (e.g. an O_DIRECT
> + * read) finds them. After-split folios go to @list, or to the LRU if NULL.
> + */
> +static bool pisa_user_split(struct folio *folio, struct list_head *list, bool can_wait)
> +{
> +	int ret;
> +
> +	if (can_wait)
> +		folio_lock(folio);
> +	else if (!folio_trylock(folio))
> +		return false;
> +	ret = split_folio_to_list_keep_zero(folio, list);
> +	folio_unlock(folio);
> +	return !ret;
> +}
> +
> +/*
> + * Split isolated large @folio and move to @out the subpages found in @pages.
> + * The others, or all of @folio if @split is false or the split fails, go back
> + * to the LRU. Returns the number of subpages moved to @out.
> + */
> +static unsigned int pisa_user_split_isolated(struct folio *folio, struct list_head *out,
> +					     struct page **pages, unsigned int nr, bool split)
> +{
> +	const struct page *lo = NULL, *hi = NULL;
> +	unsigned int kept = 0;
> +	struct folio *f, *next;
> +	LIST_HEAD(pieces);
> +
> +	/* @folio is held isolated, so page_folio() of its subpages is stable. */
> +	for (unsigned int i = 0; i < nr; i++) {
> +		if (page_folio(pages[i]) != folio)
> +			continue;
> +		if (!lo || pages[i] < lo)
> +			lo = pages[i];
> +		if (!hi || pages[i] > hi)
> +			hi = pages[i];
> +	}
> +	list_move(&folio->lru, &pieces);
> +	if (split && pisa_user_split(folio, &pieces, false)) {
> +		list_for_each_entry_safe(f, next, &pieces, lru) {
> +			if (!folio_test_large(f) && &f->page >= lo && &f->page <= hi) {
> +				list_move_tail(&f->lru, out);
> +				kept++;
> +			}
> +		}
> +	}
> +	putback_movable_pages(&pieces);
> +	return kept;
> +}
> +
> +/**
> + * pisa_upgrade_user_pages - Migrate extracted user @pages into PISA
> + * @iter: Iterator from which @pages were just extracted
> + * @pages: Extracted page array, one reference each
> + * @size: Number of bytes extracted from @iter
> + * @offset: Byte offset within @pages[0]
> + *
> + * Return: false if no pages were isolated (@pages and @iter are untouched);
> + * true otherwise, so the caller must re-extract @pages from the reverted @iter.
> + */
> +bool pisa_upgrade_user_pages(struct iov_iter *iter, struct page **pages,
> +			     size_t size, size_t offset)
> +{
> +	unsigned int nr = DIV_ROUND_UP(offset + size, PAGE_SIZE);
> +	unsigned int want = 0;
> +	struct folio *f, *next;
> +	bool drained = false;
> +	LIST_HEAD(small);
> +	LIST_HEAD(list);
> +	long budget;
> +
> +	if (!pisa_user_enabled())
> +		return false;
> +	/*
> +	 * First pass: check if any page needs migration and whether budget is
> +	 * available before draining per-CPU LRU caches or isolating folios.
> +	 */
> +	for (unsigned int i = 0; i < nr; i++)
> +		want += pisa_user_want(page_folio(pages[i]), true);
> +	if (!want || pisa_user_budget() <= 0)
> +		return false;
> +	for (unsigned int i = 0; i < nr; i++) {
> +		struct folio *folio = page_folio(pages[i]);
> +		unsigned int refs = 0;
> +
> +		if (!pisa_user_want(folio, true))
> +			continue;
> +		/* Drain per-CPU LRU batches once so freshly faulted folios can be isolated. */
> +		if (!drained && !folio_test_lru(folio)) {
> +			lru_add_drain();
> +			drained = true;
> +		}
> +		if (!folio_test_lru(folio))
> +			continue;
> +		for (unsigned int j = 0; j < nr; j++)
> +			refs += page_folio(pages[j]) == folio;
> +		/* The references from iov_iter_get_pages2() are expected. */
> +		if (folio_maybe_dma_pinned(folio) || pisa_user_busy(folio, refs))
> +			continue;
> +		if (isolate_folio_to_list(folio, &list))
> +			list_move_tail(&folio->lru, &list);
> +	}
> +	if (list_empty(&list))
> +		return false;
> +	/*
> +	 * Release the caller's references on @pages before migrating
> +	 * (migrate_pages() rejects folios with extra references).
> +	 */
> +	for (unsigned int i = 0; i < nr; i++)
> +		put_page(pages[i]);
> +	/*
> +	 * Split large folios while budget remains and keep only their subpages
> +	 * that are in @pages: migrate_pages() would split them regardless of
> +	 * the budget, migrate every subpage and drop the zero-filled ones.
> +	 */
> +	budget = pisa_user_budget();
> +	list_for_each_entry_safe(f, next, &list, lru) {
> +		if (folio_test_large(f))
> +			budget -= pisa_user_split_isolated(f, &small, pages, nr, budget > 0);
> +		else
> +			budget -= !is_pisa_pfn(folio_pfn(f));
> +	}
> +	list_splice_tail(&small, &list);
> +	pisa_user_migrate(&list, true,
> +			  GFP_NOWAIT | __GFP_HARDWALL | __GFP_NOWARN);
> +	/* Rewind @iter so the caller re-extracts the updated page pointers. */
> +	iov_iter_revert(iter, size);
> +	return true;
> +}
> +
> +#define PISA_USER_BATCH	512
> +
> +struct pisa_madvise_walk {
> +	struct list_head	list;	/* isolated folios waiting to be migrated */
> +	unsigned int		n;	/* pages on @list */
> +	bool			to_pisa;
> +};
> +
> +#define PISA_MADVISE_GFP	(GFP_USER | __GFP_NOWARN)
> +
> +/* Isolate the wanted folios mapped in one PMD's range, splitting the large ones first. */
> +static int pisa_madvise_pmd_entry(pmd_t *pmd, unsigned long addr, unsigned long end,
> +				  struct mm_walk *walk)
> +{
> +	struct pisa_madvise_walk *w = walk->private;
> +	bool to_pisa = w->to_pisa;
> +	struct folio_walk fw;
> +	struct folio *folio;
> +
> +	while (addr < end) {
> +		unsigned long nr = 1;
> +
> +		if (fatal_signal_pending(current))
> +			return -EINTR;
> +		/* Stop the walk, without an error, once the cap is reached. */
> +		if (to_pisa && pisa_user_budget() <= (long)w->n)
> +			return 1;
> +		folio = folio_walk_start(&fw, walk->vma, addr, 0);
> +		if (folio) {
> +			bool want = pisa_user_want(folio, to_pisa) && folio_test_lru(folio);
> +			bool busy = want && pisa_user_busy(folio, 0);
> +			bool split = want && !busy && folio_test_large(folio);
> +
> +			/* A PMD mapping covers the rest of the range, a PTE maps one page. */
> +			if (fw.level != FW_LEVEL_PTE)
> +				nr = (end - addr) >> PAGE_SHIFT;
> +			if (split) {
> +				folio_get(folio);
> +			} else if (want && !busy && isolate_folio_to_list(folio, &w->list)) {
> +				list_move_tail(&folio->lru, &w->list);
> +				w->n++;
> +			}
> +			folio_walk_end(&fw, walk->vma);
> +			if (split) {
> +				split = pisa_user_split(folio, NULL, true);
> +				folio_put(folio);
> +				/* Look at @addr again: it now maps an order-0 folio. */
> +				if (split)
> +					continue;
> +			}
> +		}
> +		if (w->n >= PISA_USER_BATCH) {
> +			pisa_user_migrate(&w->list, to_pisa, PISA_MADVISE_GFP);
> +			w->n = 0;
> +		}
> +		addr += nr << PAGE_SHIFT;
> +		cond_resched();
> +	}
> +	return 0;
> +}
> +
> +static const struct mm_walk_ops pisa_madvise_walk_ops = {
> +	.pmd_entry	= pisa_madvise_pmd_entry,
> +	.walk_lock	= PGWALK_RDLOCK,
> +};
> +
> +/**
> + * pisa_madvise - MADV_PISA / MADV_NOPISA on [@start, @end) of @vma
> + * @vma: VMA, with the mmap lock held for reading
> + * @start: Start address
> + * @end: End address
> + * @to_pisa: true to move present pages into PISA, false to move them out
> + *
> + * Return: 0, -EINVAL for unsupported mappings, or -EOPNOTSUPP if disabled.
> + */
> +int pisa_madvise(struct vm_area_struct *vma, unsigned long start, unsigned long end, bool to_pisa)
> +{
> +	struct pisa_madvise_walk w = { .list = LIST_HEAD_INIT(w.list), .to_pisa = to_pisa };
> +	int ret;
> +
> +	if (vma->vm_flags & (VM_SHARED | VM_IO | VM_PFNMAP | VM_HUGETLB))
> +		return -EINVAL;
> +	if (to_pisa && !pisa_user_enabled())
> +		return -EOPNOTSUPP;
> +	if (!pisa_enabled() && !to_pisa)
> +		return 0;
> +	lru_add_drain();
> +	/* The page walker skips the unpopulated page-table levels of sparse mappings. */
> +	ret = walk_page_range_vma(vma, start, end, &pisa_madvise_walk_ops, &w);
> +	pisa_user_migrate(&w.list, to_pisa, PISA_MADVISE_GFP);
> +	return ret < 0 ? ret : 0;
> +}
> --
> 2.56.1.809.ga0b61035b5-goog
>

--
Cheers, Lorenzo


  reply	other threads:[~2026-11-20 14:10 UTC|newest]

Thread overview: 137+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-10-03 21:22 [RFC: DMA_PMD 00/22] DMA_PMD: PMD_SIZE-backed IO buffer pools Luigi Rizzo
2026-10-03 21:22 ` [RFC: DMA_PMD 01/22] iommu/dma: introduce CONFIG_DMA_PMD and metadata table Luigi Rizzo
2026-10-03 21:44   ` Randy Dunlap
2026-10-04  9:14     ` Luigi Rizzo
2026-10-03 21:22 ` [RFC: DMA_PMD 02/22] iommu/dma: add DMA_PMD pool lifecycle and page recycle hook Luigi Rizzo
2026-10-10  2:59   ` kernel test robot
2026-10-03 21:22 ` [RFC: DMA_PMD 03/22] mm: Add split_page_compound() Luigi Rizzo
2026-10-03 21:22 ` [RFC: DMA_PMD 04/22] iommu/dma: add DMA_PMD pool block allocation Luigi Rizzo
2026-10-03 21:22 ` [RFC: DMA_PMD 05/22] iommu/dma: Global cap and shrinker for DMA_PMD pool memory Luigi Rizzo
2026-10-03 21:22 ` [RFC: DMA_PMD 06/22] iommu/dma: reserve a per-domain IOVA window for DMA_PMD pages Luigi Rizzo
2026-10-03 21:22 ` [RFC: DMA_PMD 07/22] iommu/dma: release DMA_PMD domain mappings on domain teardown Luigi Rizzo
2026-10-03 21:22 ` [RFC: DMA_PMD 08/22] iommu/dma: use per-domain IOVA window to map DMA_PMD memory Luigi Rizzo
2026-10-03 21:22 ` [RFC: DMA_PMD 09/22] iommu/dma: Add DMA_PMD arena allocator Luigi Rizzo
2026-10-03 21:22 ` [RFC: DMA_PMD 10/22] driver core: Add per-device dma_pmd_* sysfs attributes Luigi Rizzo
2026-10-03 21:22 ` [RFC: DMA_PMD 11/22] dma-mapping: Use DMA_PMD arena for dma_alloc_attrs() Luigi Rizzo
2026-10-03 21:22 ` [RFC: DMA_PMD 12/22] net/core: Use per-CPU DMA_PMD pools for skb_page_frag_refill() Luigi Rizzo
2026-10-03 21:22 ` [RFC: DMA_PMD 13/22] net/core: Use DMA_PMD for page_pool memory Luigi Rizzo
2026-10-03 21:22 ` [RFC: DMA_PMD 14/22] iommu/dma: Support decrypted and pinned DMA_PMD pages Luigi Rizzo
2026-10-03 21:22 ` [RFC: DMA_PMD 15/22] iommu/dma: Add background page scrubber for DMA_PMD pools Luigi Rizzo
2026-10-03 21:22 ` [RFC: DMA_PMD 16/22] iommu/dma: Add per-NUMA-node PMD page reservoir Luigi Rizzo
2026-10-03 21:22 ` [RFC: DMA_PMD 17/22] net/gve: Use DMA_PMD memory for RX buffers Luigi Rizzo
2026-10-03 21:22 ` [RFC: DMA_PMD 18/22] net/gve: Use DMA_PMD memory for tx header bounce buffers Luigi Rizzo
2026-10-03 21:22 ` [RFC: DMA_PMD 19/22] net/mlx5e: " Luigi Rizzo
2026-10-14 17:22   ` Dragos Tatulea
2026-10-14 17:50     ` Dragos Tatulea
2026-10-14 18:28       ` Luigi Rizzo
2026-10-03 21:22 ` [RFC: DMA_PMD 20/22] net/idpf: " Luigi Rizzo
2026-10-03 21:22 ` [RFC: DMA_PMD 21/22] net/bnxt: " Luigi Rizzo
2026-10-03 21:22 ` [RFC: DMA_PMD 22/22] iommu/dma: Add DMA_PMD statistics and debugfs Luigi Rizzo
2026-10-11  9:02 ` [RFC: DMA_PMD 00/22] DMA_PMD: PMD_SIZE-backed IO buffer pools Pranjal Shrivastava
2026-10-12 13:51 ` David Hildenbrand (Arm)
2026-10-12 14:45   ` Luigi Rizzo
2026-10-12 19:43     ` David Hildenbrand (Arm)
2026-10-13 10:42       ` Robin Murphy
2026-10-14 10:47         ` David Hildenbrand (Arm)
2026-10-14 11:05           ` Luigi Rizzo
2026-10-17 17:11       ` Luigi Rizzo
2026-10-22 20:12         ` Jason Gunthorpe
2026-10-22 21:51           ` Luigi Rizzo
2026-10-23 14:01             ` Jason Gunthorpe
2026-10-23 14:30               ` Luigi Rizzo
2026-10-23 14:50                 ` Jason Gunthorpe
2026-10-23 15:12                   ` Luigi Rizzo
2026-10-23 17:27                   ` Jakub Kicinski
2026-10-23 20:22                     ` Luigi Rizzo
2026-11-03  8:09                   ` Christoph Hellwig
2026-11-03  9:13                     ` Luigi Rizzo
2026-11-03 15:10                     ` Jason Gunthorpe
2026-11-06 17:21                       ` Naman Gulati
2026-11-06 17:30                         ` Jason Gunthorpe
2026-11-06 18:00                           ` Naman Gulati
2026-11-06 18:07                             ` Jason Gunthorpe
2026-11-06 18:17                               ` Naman Gulati
2026-11-06 18:43                                 ` Jason Gunthorpe
2026-11-06 19:01                                   ` Luigi Rizzo
2026-11-06 19:11                                   ` Naman Gulati
2026-10-12 14:49   ` Mostafa Saleh
2026-10-12 15:02     ` Luigi Rizzo
2026-10-14 17:28 ` Dragos Tatulea
2026-10-14 18:14   ` Luigi Rizzo
2026-11-09 18:25 ` [RFC PATCH v2 00/25] PISA: Pre-mapped IOVA Superpage Allocator for DMA and CoCo Luigi Rizzo
2026-11-20  1:06   ` [RFC PATCH v3 " Luigi Rizzo
2026-11-20  1:06     ` [RFC PATCH v3 01/25] mm: Introduce CONFIG_PISA and metadata table Luigi Rizzo
2026-11-20  1:06     ` [RFC PATCH v3 02/25] mm: Add PISA allocator lifecycle and page recycle hook Luigi Rizzo
2026-11-20  1:06     ` [RFC PATCH v3 03/25] mm: Add per-NUMA-node PISA page reservoir Luigi Rizzo
2026-11-20  1:06     ` [RFC PATCH v3 04/25] mm/page_alloc: Extend alloc_pages() to __GFP_PISA and PISA block allocation Luigi Rizzo
2026-11-20  1:06     ` [RFC PATCH v3 05/25] mm: Add global cap and shrinker for PISA pages Luigi Rizzo
2026-11-20  1:06     ` [RFC PATCH v3 06/25] iommu/dma: Map PISA memory in a per-domain IOVA window Luigi Rizzo
2026-11-20  1:06     ` [RFC PATCH v3 07/25] driver core: Add per-device pisa_* sysfs attributes for PISA optin Luigi Rizzo
2026-11-20  1:06     ` [RFC PATCH v3 08/25] iommu/dma: Use PISA for coherent DMA allocations Luigi Rizzo
2026-11-20  1:06     ` [RFC PATCH v3 09/25] mm: Add per-CPU block caches to PISA allocators Luigi Rizzo
2026-11-20  1:06     ` [RFC PATCH v3 10/25] mm: Add background page scrubber for " Luigi Rizzo
2026-11-20  1:06     ` [RFC PATCH v3 11/25] mm: Migrate user pages into PISA Luigi Rizzo
2026-11-20 14:10       ` Lorenzo Stoakes (ARM) [this message]
2026-11-20  1:06     ` [RFC PATCH v3 12/25] block, mm: Support PISA for disk I/O and scrub migration Luigi Rizzo
2026-11-20  1:06     ` [RFC PATCH v3 13/25] net: Cache leaf PISA TX flags on struct net_device Luigi Rizzo
2026-11-20  1:06     ` [RFC PATCH v3 14/25] net: Use per-node PISA allocators for skb_page_frag_refill() Luigi Rizzo
2026-11-20  1:06     ` [RFC PATCH v3 15/25] net: Use PISA for page_pool memory Luigi Rizzo
2026-11-20  1:06     ` [RFC PATCH v3 16/25] mm/slub: Add SLAB_CACHE_PISA_RO for PISA-backed slab caches Luigi Rizzo
2026-11-20  1:06     ` [RFC PATCH v3 17/25] net: Allocate small skb heads from PISA Luigi Rizzo
2026-11-20  1:06     ` [RFC PATCH v3 18/25] net: Migrate user pages into PISA for MSG_ZEROCOPY Luigi Rizzo
2026-11-20  1:06     ` [RFC PATCH v3 19/25] mm, iommu/dma: Support decrypted PISA pages Luigi Rizzo
2026-11-20  1:06     ` [RFC PATCH v3 20/25] arm64/mm, mm: Support decrypted PISA pages in Realms Luigi Rizzo
2026-11-20  1:06     ` [RFC PATCH v3 21/25] mm: Add PISA statistics and debugfs Luigi Rizzo
2026-11-20  1:06     ` [RFC PATCH v3 22/25] net/libeth: Use GFP_KERNEL when pre-filling RX rings Luigi Rizzo
2026-11-20  1:06     ` [RFC PATCH v3 23/25] net/gve: " Luigi Rizzo
2026-11-20  1:06     ` [RFC PATCH v3 24/25] net/gve: Use PISA memory for RX buffers Luigi Rizzo
2026-11-20  1:06     ` [RFC PATCH v3 25/25] Documentation: Add PISA architecture documentation Luigi Rizzo
2026-11-20 13:50     ` [RFC PATCH v3 00/25] PISA: Pre-mapped IOVA Superpage Allocator for DMA and CoCo Jason Gunthorpe
2026-11-20 14:18       ` Lorenzo Stoakes (ARM)
2026-11-20 14:19       ` Luigi Rizzo
2026-11-09 18:25 ` [RFC PATCH v2 01/25] iommu/dma: Introduce CONFIG_PISA and metadata table Luigi Rizzo
2026-11-10 23:12   ` sashiko-bot
2026-11-09 18:25 ` [RFC PATCH v2 02/25] iommu/dma: Add PISA allocator lifecycle and page recycle hook Luigi Rizzo
2026-11-09 18:25 ` [RFC PATCH v2 03/25] iommu/dma: Add per-NUMA-node PISA page reservoir Luigi Rizzo
2026-11-10 23:12   ` sashiko-bot
2026-11-09 18:25 ` [RFC PATCH v2 04/25] iommu/dma: Add PISA block allocation Luigi Rizzo
2026-11-10 23:12   ` sashiko-bot
2026-11-09 18:25 ` [RFC PATCH v2 05/25] iommu/dma: Add global cap and shrinker for PISA pages Luigi Rizzo
2026-11-09 18:25 ` [RFC PATCH v2 06/25] iommu/dma: Reserve a per-domain IOVA window " Luigi Rizzo
2026-11-09 18:25 ` [RFC PATCH v2 07/25] iommu/dma: Release PISA domain mappings on domain teardown Luigi Rizzo
2026-11-10 23:12   ` sashiko-bot
2026-11-09 18:25 ` [RFC PATCH v2 08/25] iommu/dma: Use per-domain IOVA window to map PISA memory Luigi Rizzo
2026-11-10 23:12   ` sashiko-bot
2026-11-09 18:25 ` [RFC PATCH v2 09/25] driver core: Add per-device pisa_* sysfs attributes for PISA optin Luigi Rizzo
2026-11-10 23:12   ` sashiko-bot
2026-11-09 18:25 ` [RFC PATCH v2 10/25] iommu/dma: Use PISA for coherent DMA allocations Luigi Rizzo
2026-11-10 23:12   ` sashiko-bot
2026-11-09 18:25 ` [RFC PATCH v2 11/25] iommu/dma: Add per-CPU block caches to PISA allocators Luigi Rizzo
2026-11-10 23:12   ` sashiko-bot
2026-11-09 18:25 ` [RFC PATCH v2 12/25] net/core: Use per-node PISA allocators for skb_page_frag_refill() Luigi Rizzo
2026-11-10 23:50   ` sashiko-bot
2026-11-09 18:25 ` [RFC PATCH v2 13/25] net/core: Use PISA for page_pool memory Luigi Rizzo
2026-11-09 18:25 ` [RFC PATCH v2 14/25] iommu/dma: Support decrypted PISA pages Luigi Rizzo
2026-11-11  0:19   ` sashiko-bot
2026-11-09 18:25 ` [RFC PATCH v2 15/25] iommu/dma: Add background page scrubber for PISA allocators Luigi Rizzo
2026-11-11  0:37   ` sashiko-bot
2026-11-09 18:25 ` [RFC PATCH v2 16/25] mm/slub: Add SLAB_CACHE_PISA_RO for PISA-backed slab caches Luigi Rizzo
2026-11-11  1:03   ` sashiko-bot
2026-11-09 18:25 ` [RFC PATCH v2 17/25] net: Allocate small skb heads from PISA Luigi Rizzo
2026-11-11  1:33   ` sashiko-bot
2026-11-09 18:25 ` [RFC PATCH v2 18/25] iommu/dma: Migrate user pages into PISA for TX zerocopy Luigi Rizzo
2026-11-11  2:41   ` sashiko-bot
2026-11-09 18:25 ` [RFC PATCH v2 19/25] iommu/dma: Stop recycling old PISA pages when allocator waste is high Luigi Rizzo
2026-11-11  2:57   ` sashiko-bot
2026-11-09 18:25 ` [RFC PATCH v2 20/25] net/libeth: Use GFP_KERNEL when pre-filling RX rings Luigi Rizzo
2026-11-11  3:13   ` sashiko-bot
2026-11-09 18:25 ` [RFC PATCH v2 21/25] net/gve: " Luigi Rizzo
2026-11-11  3:22   ` sashiko-bot
2026-11-09 18:25 ` [RFC PATCH v2 22/25] net/gve: Use PISA memory for RX buffers Luigi Rizzo
2026-11-11  3:43   ` sashiko-bot
2026-11-09 18:25 ` [RFC PATCH v2 23/25] block, mm, iommu/dma: Support PISA for disk I/O and scrub migration Luigi Rizzo
2026-11-11  4:16   ` sashiko-bot
2026-11-09 18:25 ` [RFC PATCH v2 24/25] Documentation: Add PISA architecture documentation Luigi Rizzo
2026-11-11  4:24   ` sashiko-bot
2026-11-09 18:25 ` [RFC PATCH v2 25/25] iommu/dma: Add PISA statistics and debugfs Luigi Rizzo
2026-11-11  4:37   ` sashiko-bot

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=awBUG15auEfX2Uc6@gremlin \
    --to=ljs@kernel.org \
    --cc=akpm@linux-foundation.org \
    --cc=aleksander.lobakin@intel.com \
    --cc=axboe@kernel.dk \
    --cc=brendan.jackman@linux.dev \
    --cc=catalin.marinas@arm.com \
    --cc=cl@gentwo.org \
    --cc=corbet@lwn.net \
    --cc=dakr@kernel.org \
    --cc=davem@davemloft.net \
    --cc=david@kernel.org \
    --cc=driver-core@lists.linux.dev \
    --cc=dtatulea@nvidia.com \
    --cc=edumazet@kernel.org \
    --cc=gregkh@linuxfoundation.org \
    --cc=hannes@cmpxchg.org \
    --cc=hao.li@linux.dev \
    --cc=harry@kernel.org \
    --cc=hawk@kernel.org \
    --cc=hch@lst.de \
    --cc=horms@kernel.org \
    --cc=hramamurthy@google.com \
    --cc=ilias.apalodimas@linaro.org \
    --cc=iommu@lists.linux.dev \
    --cc=jack@suse.cz \
    --cc=jannh@google.com \
    --cc=jgg@ziepe.ca \
    --cc=joro@8bytes.org \
    --cc=joshwash@google.com \
    --cc=kuba@kernel.org \
    --cc=kuniyu@google.com \
    --cc=liam@infradead.org \
    --cc=linux-api@vger.kernel.org \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=lrizzo@google.com \
    --cc=m.szyprowski@samsung.com \
    --cc=mhocko@suse.com \
    --cc=namangulati@google.com \
    --cc=netdev@vger.kernel.org \
    --cc=pabeni@redhat.com \
    --cc=praan@google.com \
    --cc=rafael@kernel.org \
    --cc=rdunlap@infradead.org \
    --cc=rientjes@google.com \
    --cc=rizzo.unipi@gmail.com \
    --cc=robin.murphy@arm.com \
    --cc=roman.gushchin@linux.dev \
    --cc=rppt@kernel.org \
    --cc=smostafa@google.com \
    --cc=surenb@google.com \
    --cc=vbabka@kernel.org \
    --cc=will@kernel.org \
    --cc=willemb@google.com \
    --cc=willy@infradead.org \
    --cc=ziy@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).