* [PATCH v6 01/14] mm/vmalloc: support vmalloc-backed VM_SPARSE areas
2026-11-22 12:39 [PATCH v6 00/14] mm, swap: extendable swap devices (xswap phase I) Baoquan He
@ 2026-11-22 12:39 ` Baoquan He
2026-11-22 12:39 ` [PATCH v6 02/14] mm: xswap support for zswap Baoquan He
` (8 subsequent siblings)
9 siblings, 0 replies; 15+ messages in thread
From: Baoquan He @ 2026-11-22 12:39 UTC (permalink / raw)
To: linux-mm
Cc: akpm, david, urezki, chrisl, kasong, nphamcs, baohua,
youngjun.park, shikemeng, hannes, yosry, chengming.zhou, corbet,
kunwu.chan, klarasmodin, linux-kernel, baoquan.he, Baoquan He
From: Baoquan He <hebaoquan@kylinos.cn>
VM_SPARSE areas are created with get_vm_area() and filled in by the
caller via vm_area_map_pages(), so the caller need to allocate the pages,
keep the struct page array and free everything by itself.
Add vm_area_grow_pages() and vm_area_shrink_pages() for VM_SPARSE |
VM_ALLOC areas. Pages come from the page allocator and are tracked in
area->pages, so the area is released with a single vfree(). The backed
pages always start at the beginning of the area: growing maps
[nr_pages, end) and shrinking unmaps and frees [end, nr_pages). Callers
serialize the two helpers.
nr_pages keeps its existing meaning of "number of backed pages", as
with vrealloc() and vread_iter(), and is updated under the vmap node's
busy.lock so /proc/vmallocinfo never sees a page that is already gone.
Only areas populated through vm_area_map_pages() stay a memory hole.
vm_area_map_pages() now rejects VM_ALLOC areas, because vfree() owns
their pages, to avoid double frees; free_vm_area() warns for the same
combination but not for plain VM_ALLOC areas.
Suggested-by: Chris Li <chrisl@kernel.org>
Signed-off-by: Baoquan He <hebaoquan@kylinos.cn>
---
include/linux/vmalloc.h | 2 +
mm/vmalloc.c | 149 ++++++++++++++++++++++++++++++++++++++--
2 files changed, 147 insertions(+), 4 deletions(-)
diff --git a/include/linux/vmalloc.h b/include/linux/vmalloc.h
index 034a693777ca..9f40edcddfa8 100644
--- a/include/linux/vmalloc.h
+++ b/include/linux/vmalloc.h
@@ -296,6 +296,8 @@ int vm_area_map_pages(struct vm_struct *area, unsigned long start,
unsigned long end, struct page **pages);
void vm_area_unmap_pages(struct vm_struct *area, unsigned long start,
unsigned long end);
+int vm_area_grow_pages(struct vm_struct *area, unsigned long end, gfp_t gfp);
+void vm_area_shrink_pages(struct vm_struct *area, unsigned long end);
void vunmap_range(unsigned long addr, unsigned long end);
static inline void set_vm_flush_reset_perms(void *addr)
diff --git a/mm/vmalloc.c b/mm/vmalloc.c
index 320e981188c9..0fe2f1c39b96 100644
--- a/mm/vmalloc.c
+++ b/mm/vmalloc.c
@@ -788,6 +788,8 @@ static int check_sparse_vm_area(struct vm_struct *area, unsigned long start,
return -EINVAL;
if (WARN_ON_ONCE(!(area->flags & VM_SPARSE)))
return -EINVAL;
+ if (WARN_ON_ONCE(area->flags & VM_ALLOC))
+ return -EINVAL;
if ((end - start) >> PAGE_SHIFT > totalram_pages())
return -E2BIG;
if (start < (unsigned long)area->addr ||
@@ -4174,6 +4176,135 @@ static void *__vmalloc_area_node(struct vm_struct *area, gfp_t gfp_mask,
return NULL;
}
+/**
+ * vm_area_grow_pages - back more of a sparse area with pages
+ * @area: area created with %VM_SPARSE | %VM_ALLOC
+ * @end: end of the range to back, %PAGE_SIZE aligned
+ * @gfp: gfp flags for the page allocation
+ *
+ * Allocate and map pages for [@area->nr_pages << PAGE_SHIFT, @end). The
+ * pages are zeroed when %__GFP_ZERO is set. The backed range always forms
+ * a prefix of the area, and growing to a range that is already backed is a
+ * no-op. On failure nothing is mapped and @area->nr_pages is unchanged.
+ *
+ * The area must be released with vfree(@area->addr). Callers must
+ * serialize calls to this function.
+ *
+ * Return: 0 on success, -errno on failure.
+ */
+int vm_area_grow_pages(struct vm_struct *area, unsigned long end, gfp_t gfp)
+{
+ unsigned long start = (unsigned long)area->nr_pages << PAGE_SHIFT;
+ unsigned long addr = (unsigned long)area->addr;
+ unsigned long nr_pages, nr_allocated;
+ gfp_t nested_gfp = (gfp & GFP_RECLAIM_MASK) | __GFP_ZERO;
+ struct vmap_node *vn;
+ unsigned int flags;
+ int ret;
+
+ might_sleep();
+
+ if (WARN_ON_ONCE(!(area->flags & VM_ALLOC) ||
+ !(area->flags & VM_SPARSE)))
+ return -EINVAL;
+ if (WARN_ON_ONCE(!IS_ALIGNED(end, PAGE_SIZE)))
+ return -EINVAL;
+ if (WARN_ON_ONCE(end > get_vm_area_size(area)))
+ return -EINVAL;
+ if (end <= start)
+ return 0;
+ if ((end - start) >> PAGE_SHIFT > totalram_pages())
+ return -E2BIG;
+
+ if (!area->pages) {
+ unsigned long max_pages = get_vm_area_size(area) >> PAGE_SHIFT;
+
+ area->pages = kvmalloc_array(max_pages, sizeof(*area->pages),
+ nested_gfp);
+ if (!area->pages)
+ return -ENOMEM;
+ }
+
+ if (!(gfp & (GFP_DMA | GFP_DMA32)))
+ gfp |= __GFP_HIGHMEM;
+
+ nr_pages = (end - start) >> PAGE_SHIFT;
+ nr_allocated = vm_area_alloc_pages(vmalloc_gfp_adjust(gfp, 0),
+ NUMA_NO_NODE, 0, nr_pages,
+ area->pages + (start >> PAGE_SHIFT));
+ if (nr_allocated != nr_pages) {
+ vm_area_free_pages(area, start >> PAGE_SHIFT,
+ (start >> PAGE_SHIFT) + nr_allocated);
+ return -ENOMEM;
+ }
+
+ flags = memalloc_apply_gfp_scope(gfp);
+ ret = __vmap_pages_range(addr + start, addr + end, PAGE_KERNEL,
+ area->pages + (start >> PAGE_SHIFT),
+ PAGE_SHIFT, nested_gfp);
+ memalloc_restore_scope(flags);
+ if (ret)
+ goto free_pages;
+
+ kasan_unpoison_vmalloc((void *)(addr + start), end - start,
+ KASAN_VMALLOC_PROT_NORMAL |
+ KASAN_VMALLOC_KEEP_TAG);
+
+ /*
+ * Publish under busy.lock, like clear_vm_uninitialized_flag():
+ * show_busy_info() reads nr_pages and pages[] under that lock.
+ */
+ vn = addr_to_node(addr);
+ spin_lock(&vn->busy.lock);
+ WRITE_ONCE(area->nr_pages, end >> PAGE_SHIFT);
+ spin_unlock(&vn->busy.lock);
+ return 0;
+
+free_pages:
+ vunmap_range(addr + start, addr + end);
+ vm_area_free_pages(area, start >> PAGE_SHIFT, end >> PAGE_SHIFT);
+ return -ENOMEM;
+}
+
+/**
+ * vm_area_shrink_pages - release the pages backing the end of a sparse area
+ * @area: area created with %VM_SPARSE | %VM_ALLOC
+ * @end: new end of the backed range, %PAGE_SIZE aligned
+ *
+ * Unmap and free pages for [@end, @area->nr_pages << PAGE_SHIFT). The
+ * released range can be backed again with vm_area_grow_pages(). Callers
+ * must serialize calls to this function.
+ */
+void vm_area_shrink_pages(struct vm_struct *area, unsigned long end)
+{
+ unsigned long start = (unsigned long)area->nr_pages << PAGE_SHIFT;
+ unsigned long addr = (unsigned long)area->addr;
+ struct vmap_node *vn;
+
+ might_sleep();
+
+ if (WARN_ON_ONCE(!(area->flags & VM_ALLOC) ||
+ !(area->flags & VM_SPARSE)))
+ return;
+ if (WARN_ON_ONCE(!IS_ALIGNED(end, PAGE_SIZE)))
+ return;
+ if (end >= start)
+ return;
+
+ /*
+ * Lower nr_pages under busy.lock before the pages go away: readers
+ * walk pages[] up to nr_pages under that lock.
+ */
+ vn = addr_to_node(addr);
+ spin_lock(&vn->busy.lock);
+ WRITE_ONCE(area->nr_pages, end >> PAGE_SHIFT);
+ spin_unlock(&vn->busy.lock);
+
+ kasan_poison_vmalloc((void *)(addr + end), start - end);
+ vunmap_range(addr + end, addr + start);
+ vm_area_free_pages(area, end >> PAGE_SHIFT, start >> PAGE_SHIFT);
+}
+
/*
* See __vmalloc_node_range() for a clear list of supported vmalloc flags.
* This gfp lists all flags currently passed through vmalloc. Currently,
@@ -4983,12 +5114,18 @@ long vread_iter(struct iov_iter *iter, const char *addr, size_t count)
if (n > remains)
n = remains;
- if (flags & VMAP_RAM)
+ if (flags & VMAP_RAM) {
copied = vmap_ram_vread_iter(iter, addr, n, flags);
- else if (!(vm && (vm->flags & (VM_IOREMAP | VM_SPARSE))))
- copied = aligned_vread_iter(iter, addr, n);
- else /* IOREMAP | SPARSE area is treated as memory hole */
+ } else if (vm && (vm->flags & VM_IOREMAP)) {
+ /* Not backed by struct pages. */
+ copied = zero_iter(iter, n);
+ } else if (vm && (vm->flags & VM_SPARSE) &&
+ !(vm->flags & VM_ALLOC)) {
+ /* Populated by the caller; treat as a memory hole. */
copied = zero_iter(iter, n);
+ } else {
+ copied = aligned_vread_iter(iter, addr, n);
+ }
addr += copied;
remains -= copied;
@@ -5104,6 +5241,10 @@ EXPORT_SYMBOL(remap_vmalloc_range);
void free_vm_area(struct vm_struct *area)
{
struct vm_struct *ret;
+
+ /* VM_SPARSE | VM_ALLOC areas own their pages and need vfree(). */
+ WARN_ON_ONCE((area->flags & (VM_ALLOC | VM_SPARSE)) ==
+ (VM_ALLOC | VM_SPARSE));
ret = remove_vm_area(area->addr);
BUG_ON(ret != area);
kfree(area);
--
2.54.0
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v6 02/14] mm: xswap support for zswap
2026-11-22 12:39 [PATCH v6 00/14] mm, swap: extendable swap devices (xswap phase I) Baoquan He
2026-11-22 12:39 ` [PATCH v6 01/14] mm/vmalloc: support vmalloc-backed VM_SPARSE areas Baoquan He
@ 2026-11-22 12:39 ` Baoquan He
2026-11-22 12:39 ` [PATCH v6 03/14] mm, swap: refactor free_swap_cluster_info to take swap_info_struct Baoquan He
` (7 subsequent siblings)
9 siblings, 0 replies; 15+ messages in thread
From: Baoquan He @ 2026-11-22 12:39 UTC (permalink / raw)
To: linux-mm
Cc: akpm, david, urezki, chrisl, kasong, nphamcs, baohua,
youngjun.park, shikemeng, hannes, yosry, chengming.zhou, corbet,
kunwu.chan, klarasmodin, linux-kernel, baoquan.he, Kunwu Chan,
Baoquan He
From: Chris Li <chrisl@kernel.org>
Introduce extendable swap device support - xswap.
An xswap device has no backing storage and no swap data section, so
it wastes no disk space. Creation is via swapon(2) with the name
"xswap", added in a later patch.
Zswap writeback is gated on whether a real (non-xswap) swap device is
active. nr_real_swapfiles counts such devices and is maintained at
swapon/swapoff only, so the gate reflects "a device exists to write
back to" rather than "a device currently has free slots". This keeps
writeback working even when the real swap device is full, and avoids
a double decrement when a full device is swapped off.
An xswap entry is not added to the zswap writeback LRU: with no backing
store there is nothing to write it back to. zswap_lru_del() tolerates
that, since an entry that was never added is on no list.
An incompressible page is stored uncompressed only because it can be
written back later, in LRU order. In this phase an xswap entry has no
writeback target and never reaches the LRU, so such a page would be
pinned in the pool for the lifetime of the entry. Reject it instead,
the same way a writeback-disabled memcg is handled. A later phase gives
xswap a physical backend, and the test becomes "this device has a
writeback target".
A VMA readahead walks the page table and can reach an entry on another
device. Skip an xswap device there: its entries live only in zswap, so
there is no IO to read ahead and a speculative read would take the entry
out of zswap.
An xswap entry is kept out of the zswap writeback LRU but still counts
towards the pool size. With memcg disabled the shrinker walks the root
LRU, finds no entry it can write back and is told -ENOENT without
counting a failure, so the accept-threshold loop would spin. Count that
as a failure; the worker then stops after its retry budget.
Reviewed-by: Kunwu Chan <chentao@kylinos.cn>
Co-developed-by: Baoquan He <hebaoquan@kylinos.cn>
Signed-off-by: Baoquan He <hebaoquan@kylinos.cn>
Signed-off-by: Chris Li <chrisl@kernel.org>
---
include/linux/swap.h | 2 ++
mm/page_io.c | 13 +++++++++++++
mm/swap_state.c | 24 +++++++++++++++++++-----
mm/swapfile.c | 34 ++++++++++++++++++++++++++++------
mm/zswap.c | 34 +++++++++++++++++++++++++++-------
5 files changed, 89 insertions(+), 18 deletions(-)
diff --git a/include/linux/swap.h b/include/linux/swap.h
index 4f686709d71f..9073d29377d8 100644
--- a/include/linux/swap.h
+++ b/include/linux/swap.h
@@ -201,6 +201,7 @@ enum {
SWP_STABLE_WRITES = (1 << 11), /* no overwrite PG_writeback pages */
SWP_SYNCHRONOUS_IO = (1 << 12), /* synchronous IO is efficient */
SWP_HIBERNATION = (1 << 13), /* pinned for hibernation */
+ SWP_XSWAP = (1 << 14), /* extendable swap device */
/* add others here before... */
};
@@ -378,6 +379,7 @@ void free_folio_and_swap_cache(struct folio *folio);
void free_pages_and_swap_cache(struct encoded_page **, int);
/* linux/mm/swapfile.c */
extern atomic_long_t nr_swap_pages;
+extern atomic_t nr_real_swapfiles;
extern long total_swap_pages;
extern atomic_t nr_rotate_swap;
diff --git a/mm/page_io.c b/mm/page_io.c
index a57cdcf2892d..4b541c50cd59 100644
--- a/mm/page_io.c
+++ b/mm/page_io.c
@@ -258,6 +258,12 @@ int swap_writeout(struct swap_io_ctx *ctx, struct folio *folio)
}
rcu_read_unlock();
+ /* No backing store: keep the folio; ctx->sis is not set yet. */
+ if (unlikely(__swap_entry_to_info(folio->swap)->flags & SWP_XSWAP)) {
+ folio_mark_dirty(folio);
+ return AOP_WRITEPAGE_ACTIVATE;
+ }
+
__swap_writeout(ctx, folio);
return 0;
out_unlock:
@@ -515,6 +521,13 @@ void swap_read_folio(struct swap_io_ctx *ctx, struct folio *folio)
if (zswap_load(folio) != -ENOENT)
return;
+ if (unlikely(sis->flags & SWP_XSWAP)) {
+ /* An xswap entry only lives in zswap: zswap_load() must hit. */
+ WARN_ON_ONCE(1);
+ folio_unlock(folio);
+ return;
+ }
+
/* We have to read from slower devices. Increase zswap protection. */
zswap_folio_swapin(folio);
swap_add_folio(ctx, folio, READ);
diff --git a/mm/swap_state.c b/mm/swap_state.c
index da65d75ceead..90cd99c25dfe 100644
--- a/mm/swap_state.c
+++ b/mm/swap_state.c
@@ -899,6 +899,10 @@ struct folio *swap_cluster_readahead(swp_entry_t entry, gfp_t gfp_mask,
struct blk_plug plug;
swp_entry_t ra_entry;
+ /* xswap entries live only in zswap; readahead does not help. */
+ if (si->flags & SWP_XSWAP)
+ goto skip;
+
mask = swapin_nr_pages(offset) - 1;
if (!mask)
goto skip;
@@ -984,6 +988,7 @@ static int swap_vma_ra_win(struct vm_fault *vmf, unsigned long *start,
static struct folio *swap_vma_readahead(swp_entry_t targ_entry, gfp_t gfp_mask,
struct mempolicy *mpol, pgoff_t targ_ilx, struct vm_fault *vmf)
{
+ struct swap_info_struct *si = __swap_entry_to_info(targ_entry);
struct swap_io_ctx ctx = {};
struct blk_plug plug;
struct folio *folio;
@@ -992,6 +997,10 @@ static struct folio *swap_vma_readahead(swp_entry_t targ_entry, gfp_t gfp_mask,
unsigned long start, end, addr;
pgoff_t ilx = targ_ilx;
+ /* xswap entries live only in zswap; readahead does not help. */
+ if (si->flags & SWP_XSWAP)
+ goto skip;
+
win = swap_vma_ra_win(vmf, &start, &end);
if (win == 1)
goto skip;
@@ -1000,7 +1009,7 @@ static struct folio *swap_vma_readahead(swp_entry_t targ_entry, gfp_t gfp_mask,
blk_start_plug(&plug);
for (addr = start; addr < end; ilx++, addr += PAGE_SIZE) {
- struct swap_info_struct *si = NULL;
+ struct swap_info_struct *entry_si = NULL;
softleaf_t entry;
if (!pte++) {
@@ -1020,14 +1029,19 @@ static struct folio *swap_vma_readahead(swp_entry_t targ_entry, gfp_t gfp_mask,
* holding a reference to, try to grab a reference, or skip.
*/
if (swp_type(entry) != swp_type(targ_entry)) {
- si = get_swap_device(entry);
- if (IS_ERR_OR_NULL(si))
+ entry_si = get_swap_device(entry);
+ if (IS_ERR_OR_NULL(entry_si))
+ continue;
+ /* An xswap entry lives only in zswap; there is no IO. */
+ if (entry_si->flags & SWP_XSWAP) {
+ put_swap_device(entry_si);
continue;
+ }
}
folio = swap_cache_read_folio(&ctx, entry, gfp_mask, mpol, ilx,
addr != vmf->address);
- if (si)
- put_swap_device(si);
+ if (entry_si)
+ put_swap_device(entry_si);
if (!folio)
continue;
folio_put(folio);
diff --git a/mm/swapfile.c b/mm/swapfile.c
index d59fa8220862..558c4b75bf92 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -66,6 +66,7 @@ static void move_cluster(struct swap_info_struct *si,
static DEFINE_SPINLOCK(swap_lock);
static unsigned int nr_swapfiles;
atomic_long_t nr_swap_pages;
+atomic_t nr_real_swapfiles;
/*
* Some modules use swappable objects and may try to swap them out under
* memory pressure (via the shrinker). Before doing so, they may wish to
@@ -775,7 +776,8 @@ static void free_cluster(struct swap_info_struct *si, struct swap_cluster_info *
/*
* If the swap is discardable, prepare discard the cluster
* instead of free it immediately. The cluster will be freed
- * after discard.
+ * after discard. xswap has no bdev and never sets
+ * SWP_PAGE_DISCARD, so it always takes the free path below.
*/
if ((si->flags & (SWP_WRITEOK | SWP_PAGE_DISCARD)) ==
(SWP_WRITEOK | SWP_PAGE_DISCARD)) {
@@ -1277,6 +1279,9 @@ static void del_from_avail_list(struct swap_info_struct *si, bool swapoff)
*/
lockdep_assert_held(&si->lock);
si->flags &= ~SWP_WRITEOK;
+ /* Count active devices, not merely those on the avail list. */
+ if (!(si->flags & SWP_XSWAP))
+ atomic_sub(1, &nr_real_swapfiles);
atomic_long_or(SWAP_USAGE_OFFLIST_BIT, &si->inuse_pages);
} else {
/*
@@ -1310,6 +1315,9 @@ static void add_to_avail_list(struct swap_info_struct *si, bool swapon)
if (swapon) {
lockdep_assert_held(&si->lock);
si->flags |= SWP_WRITEOK;
+ /* A full device is still active and skips the add below. */
+ if (!(si->flags & SWP_XSWAP))
+ atomic_add(1, &nr_real_swapfiles);
} else {
if (!(READ_ONCE(si->flags) & SWP_WRITEOK))
goto skip;
@@ -1323,10 +1331,11 @@ static void add_to_avail_list(struct swap_info_struct *si, bool swapon)
/*
* When device is full and device is on the plist, only one updater will
* see (inuse_pages == si->pages) and will call del_from_avail_list. If
- * that updater happen to be here, just skip adding.
+ * that updater happen to be here, just skip adding. @val still has
+ * the off-list bit, so compare its counter part.
*/
pages = si->pages;
- if (val == pages) {
+ if ((val & SWAP_USAGE_COUNTER_MASK) == pages) {
/* Just like the cmpxchg in del_from_avail_list */
if (atomic_long_try_cmpxchg(&si->inuse_pages, &pages,
pages | SWAP_USAGE_OFFLIST_BIT))
@@ -3330,7 +3339,8 @@ SYSCALL_DEFINE1(swapoff, const char __user *, specialfile)
destroy_swap_extents(p, p->swap_file);
- if (!(p->flags & SWP_SOLIDSTATE))
+ if (!(p->flags & SWP_XSWAP) &&
+ !(p->flags & SWP_SOLIDSTATE))
atomic_dec(&nr_rotate_swap);
mutex_lock(&swapon_mutex);
@@ -3440,6 +3450,19 @@ static void swap_stop(struct seq_file *swap, void *v)
mutex_unlock(&swapon_mutex);
}
+static const char *swap_type_str(struct swap_info_struct *si)
+{
+ struct file *file = si->swap_file;
+
+ if (si->flags & SWP_XSWAP)
+ return "xswap\t";
+
+ if (S_ISBLK(file_inode(file)->i_mode))
+ return "partition";
+
+ return "file\t";
+}
+
static int swap_show(struct seq_file *swap, void *v)
{
struct swap_info_struct *si = v;
@@ -3459,8 +3482,7 @@ static int swap_show(struct seq_file *swap, void *v)
len = seq_file_path(swap, file, " \t\n\\");
seq_printf(swap, "%*s%s\t%lu\t%s%lu\t%s%d\n",
len < 40 ? 40 - len : 1, " ",
- S_ISBLK(file_inode(file)->i_mode) ?
- "partition" : "file\t",
+ swap_type_str(si),
bytes, bytes < 10000000 ? "\t" : "",
inuse, inuse < 10000000 ? "\t" : "",
si->prio);
diff --git a/mm/zswap.c b/mm/zswap.c
index a2e0314db1cf..5603d283f9b5 100644
--- a/mm/zswap.c
+++ b/mm/zswap.c
@@ -885,7 +885,8 @@ static int zswap_cpu_comp_prepare(unsigned int cpu, struct hlist_node *node)
}
static bool zswap_compress(struct folio *folio, long index,
- struct zswap_entry *entry, struct zswap_pool *pool)
+ struct zswap_entry *entry, struct zswap_pool *pool,
+ bool can_writeback)
{
struct zswap_comp_ctx *comp_ctx;
struct scatterlist input, output;
@@ -934,9 +935,13 @@ static bool zswap_compress(struct folio *folio, long index,
* save the content as is without a compression, to keep the LRU order
* of writebacks. If writeback is disabled, reject the page since it
* only adds metadata overhead. swap_writeout() will put the page back
- * to the active LRU list in the case.
+ * to the active LRU list in the case. An xswap entry in this phase
+ * is never put on the LRU, so an incompressible page stored for it
+ * could never be reclaimed: reject it the same way.
*/
if (comp_ret) {
+ if (!can_writeback)
+ goto unlock;
rcu_read_lock();
if (!mem_cgroup_zswap_writeback_enabled(folio_memcg(folio))) {
rcu_read_unlock();
@@ -1093,6 +1098,11 @@ static int zswap_writeback_entry(struct swap_io_ctx *ctx,
if (IS_ERR_OR_NULL(si))
return -ENOENT;
+ if (si->flags & SWP_XSWAP) {
+ put_swap_device(si);
+ return -EINVAL;
+ }
+
mpol = get_task_policy(current);
folio = __swap_cache_alloc_folio(swpentry, GFP_KERNEL, BIT(0), NULL, mpol,
NO_INTERLEAVE_INDEX);
@@ -1508,10 +1518,15 @@ static void shrink_worker(struct work_struct *w)
* There are no writeback-candidate pages in the memcg.
* This is not an issue as long as we can find another memcg
* with pages in zswap. Skip this without incrementing attempts
- * and failures.
+ * and failures. With memcg disabled there is no other memcg,
+ * so an empty walk is the last one and has to count.
*/
- if (ret == -ENOENT)
+ if (ret == -ENOENT) {
+ if (mem_cgroup_disabled() &&
+ ++failures == MAX_RECLAIM_RETRIES)
+ break;
goto resched;
+ }
++attempts;
if (ret && ++failures == MAX_RECLAIM_RETRIES)
@@ -1531,6 +1546,7 @@ static bool zswap_store_page(struct folio *folio, long index,
{
swp_entry_t page_swpentry = folio_swap_entry(folio, index);
struct zswap_entry *entry, *old;
+ bool can_writeback;
/* allocate entry */
entry = zswap_entry_cache_alloc(GFP_KERNEL, folio_nid(folio));
@@ -1539,7 +1555,10 @@ static bool zswap_store_page(struct folio *folio, long index,
return false;
}
- if (!zswap_compress(folio, index, entry, pool))
+ /* In this phase xswap has no device to write the page back to. */
+ can_writeback = !(__swap_entry_to_info(page_swpentry)->flags & SWP_XSWAP);
+
+ if (!zswap_compress(folio, index, entry, pool, can_writeback))
goto compress_failed;
/*
@@ -1599,7 +1618,8 @@ static bool zswap_store_page(struct folio *folio, long index,
entry->referenced = true;
if (entry->length) {
INIT_LIST_HEAD(&entry->lru);
- zswap_lru_add(entry);
+ if (can_writeback)
+ zswap_lru_add(entry);
}
return true;
@@ -1669,7 +1689,7 @@ bool zswap_store(struct folio *folio)
zswap_pool_put(pool);
put_objcg:
obj_cgroup_put(objcg);
- if (!ret && zswap_pool_reached_full)
+ if (!ret && zswap_pool_reached_full && atomic_read(&nr_real_swapfiles))
queue_work(shrink_wq, &zswap_shrink_work);
check_old:
/*
--
2.54.0
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v6 03/14] mm, swap: refactor free_swap_cluster_info to take swap_info_struct
2026-11-22 12:39 [PATCH v6 00/14] mm, swap: extendable swap devices (xswap phase I) Baoquan He
2026-11-22 12:39 ` [PATCH v6 01/14] mm/vmalloc: support vmalloc-backed VM_SPARSE areas Baoquan He
2026-11-22 12:39 ` [PATCH v6 02/14] mm: xswap support for zswap Baoquan He
@ 2026-11-22 12:39 ` Baoquan He
2026-11-22 12:39 ` [PATCH v6 04/14] mm, swap: use a VM_SPARSE cluster array for xswap devices Baoquan He
` (6 subsequent siblings)
9 siblings, 0 replies; 15+ messages in thread
From: Baoquan He @ 2026-11-22 12:39 UTC (permalink / raw)
To: linux-mm
Cc: akpm, david, urezki, chrisl, kasong, nphamcs, baohua,
youngjun.park, shikemeng, hannes, yosry, chengming.zhou, corbet,
kunwu.chan, klarasmodin, linux-kernel, baoquan.he, Baoquan He,
Kunwu Chan
From: Baoquan He <hebaoquan@kylinos.cn>
Change free_swap_cluster_info() to take struct swap_info_struct* instead
of (cluster_info, maxpages). It now extracts the fields from si and
clears si->cluster_info after freeing to avoid a double free on the
swapon() error path.
The new parameter also lets the xswap path access si->flags in the
function.
Reviewed-by: Chris Li <chrisl@kernel.org>
Reviewed-by: Kunwu Chan <chentao@kylinos.cn>
Signed-off-by: Baoquan He <hebaoquan@kylinos.cn>
---
mm/swapfile.c | 26 ++++++++++++++------------
1 file changed, 14 insertions(+), 12 deletions(-)
diff --git a/mm/swapfile.c b/mm/swapfile.c
index 558c4b75bf92..35a079f2bb76 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -3208,14 +3208,17 @@ static void wait_for_allocation(struct swap_info_struct *si)
}
}
-static void free_swap_cluster_info(struct swap_cluster_info *cluster_info,
- unsigned long maxpages)
+static void free_swap_cluster_info(struct swap_info_struct *si)
{
+ struct swap_cluster_info *cluster_info = si->cluster_info;
+ unsigned long maxpages = si->max;
struct swap_cluster_info *ci;
- int i, nr_clusters = DIV_ROUND_UP(maxpages, SWAPFILE_CLUSTER);
+ int i, nr_clusters;
if (!cluster_info)
return;
+
+ nr_clusters = DIV_ROUND_UP(maxpages, SWAPFILE_CLUSTER);
for (i = 0; i < nr_clusters; i++) {
ci = cluster_info + i;
/* Cluster with bad marks count will have a remaining table */
@@ -3227,6 +3230,7 @@ static void free_swap_cluster_info(struct swap_cluster_info *cluster_info,
spin_unlock(&ci->lock);
}
kvfree(cluster_info);
+ si->cluster_info = NULL;
}
/*
@@ -3254,11 +3258,9 @@ static void flush_percpu_swap_cluster(struct swap_info_struct *si)
SYSCALL_DEFINE1(swapoff, const char __user *, specialfile)
{
struct swap_info_struct *p = NULL;
- struct swap_cluster_info *cluster_info;
struct file *swap_file, *victim;
struct address_space *mapping;
struct inode *inode;
- unsigned int maxpages;
int err, found = 0;
if (!capable(CAP_SYS_ADMIN))
@@ -3350,10 +3352,6 @@ SYSCALL_DEFINE1(swapoff, const char __user *, specialfile)
swap_file = p->swap_file;
p->swap_file = NULL;
- maxpages = p->max;
- cluster_info = p->cluster_info;
- p->max = 0;
- p->cluster_info = NULL;
spin_unlock(&p->lock);
spin_unlock(&swap_lock);
arch_swap_invalidate_area(p->type);
@@ -3361,7 +3359,9 @@ SYSCALL_DEFINE1(swapoff, const char __user *, specialfile)
mutex_unlock(&swapon_mutex);
kfree(p->global_cluster);
p->global_cluster = NULL;
- free_swap_cluster_info(cluster_info, maxpages);
+ free_swap_cluster_info(p);
+ p->max = 0;
+ p->cluster_info = NULL;
inode = mapping->host;
@@ -3728,6 +3728,8 @@ static int setup_swap_clusters_info(struct swap_info_struct *si,
if (!cluster_info)
goto err;
+ si->cluster_info = cluster_info;
+
for (i = 0; i < nr_clusters; i++)
spin_lock_init(&cluster_info[i].lock);
@@ -3791,7 +3793,7 @@ static int setup_swap_clusters_info(struct swap_info_struct *si,
si->cluster_info = cluster_info;
return 0;
err:
- free_swap_cluster_info(cluster_info, maxpages);
+ free_swap_cluster_info(si);
return err;
}
@@ -4013,7 +4015,7 @@ SYSCALL_DEFINE2(swapon, const char __user *, specialfile, int, swap_flags)
si->global_cluster = NULL;
inode = NULL;
destroy_swap_extents(si, swap_file);
- free_swap_cluster_info(si->cluster_info, si->max);
+ free_swap_cluster_info(si);
si->cluster_info = NULL;
/*
* Clear the SWP_USED flag after all resources are freed so
--
2.54.0
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v6 04/14] mm, swap: use a VM_SPARSE cluster array for xswap devices
2026-11-22 12:39 [PATCH v6 00/14] mm, swap: extendable swap devices (xswap phase I) Baoquan He
` (2 preceding siblings ...)
2026-11-22 12:39 ` [PATCH v6 03/14] mm, swap: refactor free_swap_cluster_info to take swap_info_struct Baoquan He
@ 2026-11-22 12:39 ` Baoquan He
2026-11-22 12:39 ` [PATCH v6 05/14] mm, swap: widen swap_info_struct max/pages to unsigned long Baoquan He
` (5 subsequent siblings)
9 siblings, 0 replies; 15+ messages in thread
From: Baoquan He @ 2026-11-22 12:39 UTC (permalink / raw)
To: linux-mm
Cc: akpm, david, urezki, chrisl, kasong, nphamcs, baohua,
youngjun.park, shikemeng, hannes, yosry, chengming.zhou, corbet,
kunwu.chan, klarasmodin, linux-kernel, baoquan.he, Baoquan He
From: Baoquan He <hebaoquan@kylinos.cn>
Add CONFIG_XSWAP and the per-device state for the sparse cluster array.
An xswap device does not have to allocate cluster_info for its whole
address space at once. Create the array as a VM_SPARSE | VM_ALLOC area,
which reserves the whole range but lets vmalloc allocate and track the
backing pages, and map only the first chunk at setup. The map helper
extends the mapped range as clusters become visible, and a failed grow
leaves the previous backing in place.
Setup, free and the two cluster_info scans must change together; free
uses vfree(), which drops the mapping and the pages together.
Signed-off-by: Baoquan He <hebaoquan@kylinos.cn>
---
include/linux/swap.h | 6 ++
mm/Kconfig | 9 ++
mm/swapfile.c | 203 +++++++++++++++++++++++++++++++++++++++++--
3 files changed, 209 insertions(+), 9 deletions(-)
diff --git a/include/linux/swap.h b/include/linux/swap.h
index 9073d29377d8..4a30cf7ee186 100644
--- a/include/linux/swap.h
+++ b/include/linux/swap.h
@@ -242,6 +242,12 @@ struct swap_info_struct {
signed char type; /* strange name for an index */
unsigned int max; /* size of this swap device */
struct swap_cluster_info *cluster_info; /* array, one entry per cluster */
+#ifdef CONFIG_XSWAP
+ struct vm_struct *cluster_vm; /* VM_SPARSE | VM_ALLOC area */
+ unsigned long nr_clusters_max;/* total clusters in the xswap address space */
+ unsigned long nr_clusters_mapped; /* currently mapped cluster count */
+ struct mutex xswap_lock; /* serialize map/unmap operations */
+#endif
struct list_head free_clusters; /* free clusters list */
struct list_head full_clusters; /* full clusters list */
struct list_head nonfull_clusters[SWAP_NR_ORDERS];
diff --git a/mm/Kconfig b/mm/Kconfig
index 18c0beafd4b8..a8707bbd8d62 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -122,6 +122,15 @@ config ZSWAP_COMPRESSOR_DEFAULT
default "zstd" if ZSWAP_COMPRESSOR_DEFAULT_ZSTD
default ""
+config XSWAP
+ bool "Extendable (virtual) swap device"
+ depends on SWAP && 64BIT && ZSWAP
+ help
+ Adds support for extendable swap devices (xswap) that decouple
+ PTE swap entries from physical backing storage. The cluster_info
+ array is backed by a sparse vmalloc area that is mapped lazily
+ in chunks, avoiding static pre-allocation overhead.
+
config ZSMALLOC
tristate
diff --git a/mm/swapfile.c b/mm/swapfile.c
index 35a079f2bb76..de2feaa2a628 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -49,6 +49,15 @@
#include "internal.h"
#include "swap.h"
+#ifdef CONFIG_XSWAP
+/*
+ * Clusters mapped per grow: one page worth of cluster_info, so the
+ * vmalloc page table overhead stays proportional to the mapped range.
+ */
+#define XSWAP_GROW_CLUSTERS \
+ max_t(unsigned long, PAGE_SIZE / sizeof(struct swap_cluster_info), 16)
+#endif
+
static void swap_range_alloc(struct swap_info_struct *si,
unsigned int nr_entries);
static bool folio_swapcache_freeable(struct folio *folio);
@@ -1983,6 +1992,20 @@ struct swap_info_struct *get_swap_device(swp_entry_t entry)
offset = swp_offset(entry);
if (offset >= si->max)
goto put_out;
+#ifdef CONFIG_XSWAP
+ if (si->flags & SWP_XSWAP) {
+ /* Pairs with the release store in xswap_map_clusters(). */
+ unsigned long mapped = smp_load_acquire(&si->nr_clusters_mapped);
+
+ /*
+ * xswap maps cluster_info lazily. A stale entry can point
+ * past nr_clusters_mapped; reject it before anything
+ * dereferences cluster_info.
+ */
+ if (offset / SWAPFILE_CLUSTER >= mapped)
+ goto put_out;
+ }
+#endif
return si;
bad_nofile:
@@ -2863,10 +2886,27 @@ static unsigned int find_next_to_unuse(struct swap_info_struct *si,
unsigned int prev)
{
struct swap_cluster_info *ci;
- unsigned long i, end;
+ unsigned long i, cluster_end, end;
unsigned int ci_off;
unsigned long swp_tb;
+ end = si->max;
+#ifdef CONFIG_XSWAP
+ /*
+ * An xswap device maps its cluster_info in chunks as it grows, so
+ * the tail past nr_clusters_mapped has nothing behind it.
+ */
+ if (si->flags & SWP_XSWAP) {
+ unsigned long mapped_end;
+
+ /* Pairs with the smp_store_release() in xswap_map_clusters(). */
+ mapped_end = smp_load_acquire(&si->nr_clusters_mapped) *
+ SWAPFILE_CLUSTER;
+ if (mapped_end < end)
+ end = mapped_end;
+ }
+#endif
+
/*
* No need for swap_lock here: we're just looking
* for whether an entry is in use, not modifying it; false
@@ -2874,11 +2914,11 @@ static unsigned int find_next_to_unuse(struct swap_info_struct *si,
* allocations from this area (while holding swap_lock).
*/
i = prev + 1;
- while (i < si->max) {
+ while (i < end) {
ci = __swap_offset_to_cluster(si, i);
- end = min_t(unsigned long,
- ALIGN_DOWN(i, SWAPFILE_CLUSTER) + SWAPFILE_CLUSTER,
- si->max);
+ cluster_end = min_t(unsigned long,
+ ALIGN_DOWN(i, SWAPFILE_CLUSTER) + SWAPFILE_CLUSTER,
+ end);
/*
* An empty cluster has no slot in use, so skip it whole.
@@ -2888,13 +2928,13 @@ static unsigned int find_next_to_unuse(struct swap_info_struct *si,
* enough, unlike in every other cluster_is_empty() caller.
*/
if (!READ_ONCE(ci->count)) {
- i = end;
+ i = cluster_end;
cond_resched();
continue;
}
ci_off = i % SWAPFILE_CLUSTER;
- for (; i < end; ci_off++, i++) {
+ for (; i < cluster_end; ci_off++, i++) {
swp_tb = swap_table_get(ci, ci_off);
if (!swp_tb_is_null(swp_tb) && !swp_tb_is_bad(swp_tb))
return i;
@@ -2904,7 +2944,6 @@ static unsigned int find_next_to_unuse(struct swap_info_struct *si,
return 0;
}
-
static int try_to_unuse(unsigned int type)
{
struct mm_struct *prev_mm;
@@ -3202,6 +3241,14 @@ static void wait_for_allocation(struct swap_info_struct *si)
BUG_ON(si->flags & SWP_WRITEOK);
+#ifdef CONFIG_XSWAP
+ if (si->flags & SWP_XSWAP) {
+ /* Pairs with the smp_store_release() in xswap_map_clusters(). */
+ end = min(end, smp_load_acquire(&si->nr_clusters_mapped) *
+ SWAPFILE_CLUSTER);
+ }
+#endif
+
for (offset = 0; offset < end; offset += SWAPFILE_CLUSTER) {
ci = swap_cluster_lock(si, offset);
swap_cluster_unlock(ci);
@@ -3213,11 +3260,38 @@ static void free_swap_cluster_info(struct swap_info_struct *si)
struct swap_cluster_info *cluster_info = si->cluster_info;
unsigned long maxpages = si->max;
struct swap_cluster_info *ci;
- int i, nr_clusters;
+ unsigned long i, nr_clusters;
if (!cluster_info)
return;
+#ifdef CONFIG_XSWAP
+ if (si->flags & SWP_XSWAP) {
+ unsigned long nr_mapped;
+
+ /*
+ * Cluster 0 keeps the bad header slot, so it never empties
+ * and __free_cluster() never frees its table.
+ */
+ /* Pairs with the smp_store_release() in xswap_map_clusters(). */
+ nr_mapped = smp_load_acquire(&si->nr_clusters_mapped);
+ for (i = 0; i < nr_mapped; i++) {
+ ci = &cluster_info[i];
+ spin_lock(&ci->lock);
+ if (cluster_table_is_alloced(ci)) {
+ swap_cluster_assert_empty(ci, 0, SWAPFILE_CLUSTER, true);
+ swap_cluster_free_table(ci);
+ }
+ spin_unlock(&ci->lock);
+ }
+ vfree(si->cluster_info);
+ si->cluster_vm = NULL;
+ si->cluster_info = NULL;
+ si->nr_clusters_mapped = 0;
+ return;
+ }
+#endif
+
nr_clusters = DIV_ROUND_UP(maxpages, SWAPFILE_CLUSTER);
for (i = 0; i < nr_clusters; i++) {
ci = cluster_info + i;
@@ -3715,6 +3789,54 @@ static unsigned long read_swap_header(struct swap_info_struct *si,
return maxpages;
}
+#ifdef CONFIG_XSWAP
+static int xswap_map_clusters(struct swap_info_struct *si,
+ unsigned long start_idx, unsigned long nr)
+{
+ /* vm_area_grow_pages() takes an offset into the area, not an address. */
+ unsigned long vm_end = PAGE_ALIGN((size_t)(start_idx + nr) *
+ sizeof(struct swap_cluster_info));
+ unsigned int noreclaim_flags;
+ unsigned long i;
+ int err;
+
+ mutex_lock(&si->xswap_lock);
+
+ /* Refuse a stale range: the boundary moved since the caller read it. */
+ if (start_idx != READ_ONCE(si->nr_clusters_mapped)) {
+ mutex_unlock(&si->xswap_lock);
+ return -EAGAIN;
+ }
+ if (start_idx + nr > si->nr_clusters_max) {
+ mutex_unlock(&si->xswap_lock);
+ return -EAGAIN;
+ }
+
+ /* Prevent recursive reclaim during page table allocation. */
+ noreclaim_flags = memalloc_noreclaim_save();
+ /* __GFP_ZERO: cluster_info pointer fields must start NULL. */
+ err = vm_area_grow_pages(si->cluster_vm, vm_end,
+ GFP_KERNEL | __GFP_ZERO |
+ __GFP_HIGH | __GFP_NOMEMALLOC);
+ memalloc_noreclaim_restore(noreclaim_flags);
+ if (err) {
+ mutex_unlock(&si->xswap_lock);
+ return err;
+ }
+
+ for (i = start_idx; i < start_idx + nr; i++)
+ spin_lock_init(&si->cluster_info[i].lock);
+
+ /*
+ * Publish the new mappings and cluster lock initialization before
+ * the count; walkers without xswap_lock use smp_load_acquire().
+ */
+ smp_store_release(&si->nr_clusters_mapped, start_idx + nr);
+ mutex_unlock(&si->xswap_lock);
+ return 0;
+}
+#endif /* CONFIG_XSWAP */
+
static int setup_swap_clusters_info(struct swap_info_struct *si,
union swap_header *swap_header,
unsigned long maxpages)
@@ -3724,6 +3846,69 @@ static int setup_swap_clusters_info(struct swap_info_struct *si,
int err = -ENOMEM;
unsigned long i;
+#ifdef CONFIG_XSWAP
+ if (si->flags & SWP_XSWAP) {
+ unsigned long size = PAGE_ALIGN(nr_clusters * sizeof(*cluster_info));
+ struct vm_struct *vm;
+
+ vm = get_vm_area(size, VM_SPARSE | VM_ALLOC);
+ if (!vm)
+ goto err;
+
+ cluster_info = vm->addr;
+ si->cluster_vm = vm;
+ si->nr_clusters_max = nr_clusters;
+ si->cluster_info = cluster_info;
+
+ /* Must be initialized before xswap_map_clusters() locks it. */
+ mutex_init(&si->xswap_lock);
+
+ err = xswap_map_clusters(si, 0, min_t(unsigned long,
+ XSWAP_GROW_CLUSTERS, nr_clusters));
+ if (err)
+ goto err_free_vm;
+
+ /* xswap: only cluster 0 slot 0 is bad */
+ err = swap_cluster_setup_bad_slot(si, cluster_info, 0, false);
+ if (err)
+ goto err_free_vm;
+
+ INIT_LIST_HEAD(&si->free_clusters);
+ INIT_LIST_HEAD(&si->full_clusters);
+ INIT_LIST_HEAD(&si->discard_clusters);
+ for (i = 0; i < SWAP_NR_ORDERS; i++) {
+ INIT_LIST_HEAD(&si->nonfull_clusters[i]);
+ INIT_LIST_HEAD(&si->frag_clusters[i]);
+ }
+
+ /*
+ * Cluster 0 holds the header slot, which is marked bad, so it
+ * is not entirely free. si->max is cluster aligned, so no
+ * cluster holds a slot past the end of the device.
+ */
+ for (i = 0; i < si->nr_clusters_mapped; i++) {
+ struct swap_cluster_info *ci = &cluster_info[i];
+
+ if (ci->count) {
+ ci->flags = CLUSTER_FLAG_NONFULL;
+ list_add_tail(&ci->list, &si->nonfull_clusters[0]);
+ } else {
+ ci->flags = CLUSTER_FLAG_FREE;
+ list_add_tail(&ci->list, &si->free_clusters);
+ }
+ }
+
+ return 0;
+
+err_free_vm:
+ vfree(si->cluster_info);
+ si->cluster_vm = NULL;
+ si->cluster_info = NULL;
+ si->nr_clusters_mapped = 0;
+ return err;
+ }
+#endif /* CONFIG_XSWAP */
+
cluster_info = kvzalloc_objs(*cluster_info, nr_clusters);
if (!cluster_info)
goto err;
--
2.54.0
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v6 05/14] mm, swap: widen swap_info_struct max/pages to unsigned long
2026-11-22 12:39 [PATCH v6 00/14] mm, swap: extendable swap devices (xswap phase I) Baoquan He
` (3 preceding siblings ...)
2026-11-22 12:39 ` [PATCH v6 04/14] mm, swap: use a VM_SPARSE cluster array for xswap devices Baoquan He
@ 2026-11-22 12:39 ` Baoquan He
2026-11-22 12:39 ` [PATCH v6 06/14] mm, swap: create the xswap device from swapon(2) Baoquan He
` (4 subsequent siblings)
9 siblings, 0 replies; 15+ messages in thread
From: Baoquan He @ 2026-11-22 12:39 UTC (permalink / raw)
To: linux-mm
Cc: akpm, david, urezki, chrisl, kasong, nphamcs, baohua,
youngjun.park, shikemeng, hannes, yosry, chengming.zhou, corbet,
kunwu.chan, klarasmodin, linux-kernel, baoquan.he, Baoquan He
From: Baoquan He <hebaoquan@kylinos.cn>
si->max and si->pages are unsigned int. With 4 KiB pages, this limits
an xswap device to 16 TB, and xswap may need a larger address space.
Change both fields to unsigned long, together with the offsets and
indexes that carry them.
A real swap device is still limited by swap_header.info.last_page,
which is __u32, so this only widens the xswap address space. The old
overflow clamp in read_swap_header() is no longer needed; the header
check still limits real devices.
count_swap_pages() returns unsigned int and is only used by
hibernation, which never selects an xswap device. If that ever
changes, WARN and return 0 instead of silently truncating the value.
Signed-off-by: Baoquan He <hebaoquan@kylinos.cn>
---
include/linux/swap.h | 4 ++--
mm/swapfile.c | 57 +++++++++++++++++++++++---------------------
2 files changed, 32 insertions(+), 29 deletions(-)
diff --git a/include/linux/swap.h b/include/linux/swap.h
index 4a30cf7ee186..6747b577c405 100644
--- a/include/linux/swap.h
+++ b/include/linux/swap.h
@@ -240,7 +240,7 @@ struct swap_info_struct {
signed short prio; /* swap priority of this type */
struct plist_node list; /* entry in swap_active_head */
signed char type; /* strange name for an index */
- unsigned int max; /* size of this swap device */
+ unsigned long max; /* size of this swap device */
struct swap_cluster_info *cluster_info; /* array, one entry per cluster */
#ifdef CONFIG_XSWAP
struct vm_struct *cluster_vm; /* VM_SPARSE | VM_ALLOC area */
@@ -254,7 +254,7 @@ struct swap_info_struct {
/* list of cluster that contains at least one free slot */
struct list_head frag_clusters[SWAP_NR_ORDERS];
/* list of cluster that are fragmented or contented */
- unsigned int pages; /* total of usable pages of swap */
+ unsigned long pages; /* total of usable pages of swap */
atomic_long_t inuse_pages; /* number of those currently in use */
struct swap_sequential_cluster *global_cluster; /* Use one global cluster for rotating device */
spinlock_t global_cluster_lock; /* Serialize usage of global cluster */
diff --git a/mm/swapfile.c b/mm/swapfile.c
index de2feaa2a628..912cfa68ba01 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -412,10 +412,10 @@ static inline unsigned int cluster_index(struct swap_info_struct *si,
return ci - si->cluster_info;
}
-static inline unsigned int cluster_offset(struct swap_info_struct *si,
- struct swap_cluster_info *ci)
+static inline unsigned long cluster_offset(struct swap_info_struct *si,
+ struct swap_cluster_info *ci)
{
- return cluster_index(si, ci) * SWAPFILE_CLUSTER;
+ return (unsigned long)cluster_index(si, ci) * SWAPFILE_CLUSTER;
}
static void swap_cluster_free_table_folio_rcu_cb(struct rcu_head *head)
@@ -863,7 +863,7 @@ static int swap_cluster_setup_bad_slot(struct swap_info_struct *si,
/* si->max may got shrunk by swap swap_activate() */
if (offset >= si->max && !mask) {
- pr_debug("Ignoring bad slot %u (max: %u)\n", offset, si->max);
+ pr_debug("Ignoring bad slot %u (max: %lu)\n", offset, si->max);
return 0;
}
/*
@@ -1045,11 +1045,12 @@ static bool swap_prefer_free_cluster(struct swap_info_struct *si)
}
/* Try use a new cluster for current CPU and allocate from it. */
-static unsigned int alloc_swap_scan_cluster(struct swap_info_struct *si,
- struct swap_cluster_info *ci,
- struct folio *folio, unsigned long offset)
+static unsigned long alloc_swap_scan_cluster(struct swap_info_struct *si,
+ struct swap_cluster_info *ci,
+ struct folio *folio,
+ unsigned long offset)
{
- unsigned int next = SWAP_ENTRY_INVALID, found = SWAP_ENTRY_INVALID;
+ unsigned long next = SWAP_ENTRY_INVALID, found = SWAP_ENTRY_INVALID;
unsigned long start = ALIGN_DOWN(offset, SWAPFILE_CLUSTER);
unsigned int order = likely(folio) ? folio_order(folio) : 0;
unsigned long end = start + SWAPFILE_CLUSTER;
@@ -1096,12 +1097,12 @@ static unsigned int alloc_swap_scan_cluster(struct swap_info_struct *si,
return found;
}
-static unsigned int alloc_swap_scan_list(struct swap_info_struct *si,
- struct list_head *list,
- struct folio *folio,
- bool scan_all)
+static unsigned long alloc_swap_scan_list(struct swap_info_struct *si,
+ struct list_head *list,
+ struct folio *folio,
+ bool scan_all)
{
- unsigned int found = SWAP_ENTRY_INVALID;
+ unsigned long found = SWAP_ENTRY_INVALID;
do {
struct swap_cluster_info *ci = isolate_lock_cluster(si, list);
@@ -1184,7 +1185,7 @@ static unsigned long cluster_alloc_swap_entry(struct swap_info_struct *si,
{
struct swap_cluster_info *ci;
unsigned int order = likely(folio) ? folio_order(folio) : 0;
- unsigned int offset = SWAP_ENTRY_INVALID, found = SWAP_ENTRY_INVALID;
+ unsigned long offset = SWAP_ENTRY_INVALID, found = SWAP_ENTRY_INVALID;
/*
* Swapfile is not block device so unable
@@ -1454,7 +1455,7 @@ static bool swap_alloc_fast(struct folio *folio)
unsigned int order = folio_order(folio);
struct swap_cluster_info *ci;
struct swap_info_struct *si;
- unsigned int offset;
+ unsigned long offset;
/*
* Once allocated, swap_info_struct will never be completely freed,
@@ -2559,9 +2560,15 @@ unsigned int count_swap_pages(int type, int free)
spin_lock(&sis->lock);
if (sis->flags & SWP_WRITEOK) {
- n = sis->pages;
- if (free)
- n -= swap_usage_in_pages(sis);
+ /* Hibernation only; it never selects an xswap device. */
+ if (unlikely(sis->flags & SWP_XSWAP)) {
+ WARN_ON_ONCE(1);
+ n = 0;
+ } else {
+ n = sis->pages;
+ if (free)
+ n -= swap_usage_in_pages(sis);
+ }
}
spin_unlock(&sis->lock);
}
@@ -2882,8 +2889,8 @@ static int unuse_mm(struct mm_struct *mm, unsigned int type)
* Return 0 if there are no inuse entries after prev till end of
* the map.
*/
-static unsigned int find_next_to_unuse(struct swap_info_struct *si,
- unsigned int prev)
+static unsigned long find_next_to_unuse(struct swap_info_struct *si,
+ unsigned long prev)
{
struct swap_cluster_info *ci;
unsigned long i, cluster_end, end;
@@ -2953,7 +2960,7 @@ static int try_to_unuse(unsigned int type)
struct swap_info_struct *si = swap_info[type];
struct folio *folio;
swp_entry_t entry;
- unsigned int i;
+ unsigned long i;
if (!swap_usage_in_pages(si))
goto success;
@@ -3767,12 +3774,8 @@ static unsigned long read_swap_header(struct swap_info_struct *si,
pr_warn("Truncating oversized swap area, only using %luk out of %luk\n",
K(maxpages), K(last_page));
}
- if (maxpages > last_page) {
+ if (maxpages > last_page)
maxpages = last_page + 1;
- /* p->max is an unsigned int: don't overflow it */
- if ((unsigned int)maxpages == 0)
- maxpages = UINT_MAX;
- }
if (!maxpages)
return 0;
@@ -4086,7 +4089,7 @@ SYSCALL_DEFINE2(swapon, const char __user *, specialfile, int, swap_flags)
goto bad_swap_unlock_inode;
}
if (si->pages != si->max - 1) {
- pr_err("swap:%u != (max:%u - 1)\n", si->pages, si->max);
+ pr_err("swap:%lu != (max:%lu - 1)\n", si->pages, si->max);
error = -EINVAL;
goto bad_swap_unlock_inode;
}
--
2.54.0
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v6 06/14] mm, swap: create the xswap device from swapon(2)
2026-11-22 12:39 [PATCH v6 00/14] mm, swap: extendable swap devices (xswap phase I) Baoquan He
` (4 preceding siblings ...)
2026-11-22 12:39 ` [PATCH v6 05/14] mm, swap: widen swap_info_struct max/pages to unsigned long Baoquan He
@ 2026-11-22 12:39 ` Baoquan He
2026-11-22 12:39 ` [PATCH v6 07/14] mm, swap: refactor swapoff and add xswap_destroy Baoquan He
` (3 subsequent siblings)
9 siblings, 0 replies; 15+ messages in thread
From: Baoquan He @ 2026-11-22 12:39 UTC (permalink / raw)
To: linux-mm
Cc: akpm, david, urezki, chrisl, kasong, nphamcs, baohua,
youngjun.park, shikemeng, hannes, yosry, chengming.zhou, corbet,
kunwu.chan, klarasmodin, linux-kernel, baoquan.he, Baoquan He
From: Baoquan He <hebaoquan@kylinos.cn>
An xswap device has no backing file, so it has no path for swapon(2) to
open, and it needs a way to be activated. Give swapon(2) a name that
says exactly that:
swapon("xswap", SWAP_FLAG_PREFER | prio);
The name is not a path: swapon(2) opens specialfile the usual way first
and falls back to the device only when that fails with -ENOENT. A file
or directory called "xswap" therefore keeps precedence, and only the
exact string matches, so "./xswap" and absolute paths are files as
usual. This is the same rule swapoff(2) uses. There is no separate
flag and no second way to create it.
The call is still an activation: it takes a priority, it fails with
-EBUSY when the device exists, and /proc/swaps shows it as the fixed
"xswap" next to the real devices. Discard flags are rejected: there is
no storage to discard.
The device is sized at twice RAM, and its cluster array is mapped
lazily, so only the part that is used costs memory.
Hibernation device discovery skips xswap devices: there is no bdev to
store a resume image.
Signed-off-by: Baoquan He <hebaoquan@kylinos.cn>
---
include/linux/swap.h | 1 -
mm/swapfile.c | 187 +++++++++++++++++++++++++++++++++++++++----
2 files changed, 172 insertions(+), 16 deletions(-)
diff --git a/include/linux/swap.h b/include/linux/swap.h
index 6747b577c405..02c3861a336d 100644
--- a/include/linux/swap.h
+++ b/include/linux/swap.h
@@ -21,7 +21,6 @@
#define SWAP_FLAG_DISCARD 0x10000 /* enable discard for swap */
#define SWAP_FLAG_DISCARD_ONCE 0x20000 /* discard swap area at swapon-time */
#define SWAP_FLAG_DISCARD_PAGES 0x40000 /* discard page-clusters after use */
-
#define SWAP_FLAGS_VALID (SWAP_FLAG_PRIO_MASK | SWAP_FLAG_PREFER | \
SWAP_FLAG_DISCARD | SWAP_FLAG_DISCARD_ONCE | \
SWAP_FLAG_DISCARD_PAGES)
diff --git a/mm/swapfile.c b/mm/swapfile.c
index 912cfa68ba01..4e598ab8da65 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -48,6 +48,16 @@
#include "swap_table.h"
#include "internal.h"
#include "swap.h"
+#define DEF_SWAP_PRIO -1
+
+/*
+ * Highest priority: xswap must take the page before a real device does.
+ * At equal priority the list is FIFO, so an earlier real device wins.
+ */
+#define DEF_XSWAP_PRIO SWAP_FLAG_PRIO_MASK
+
+/* File-less, so /proc/swaps shows this name; swapoff -a passes it back. */
+#define XSWAP_NAME "xswap"
#ifdef CONFIG_XSWAP
/*
@@ -56,7 +66,18 @@
*/
#define XSWAP_GROW_CLUSTERS \
max_t(unsigned long, PAGE_SIZE / sizeof(struct swap_cluster_info), 16)
-#endif
+
+static int xswap_create(int prio);
+static int xswap_create_by_name(const char *name, int swap_flags);
+
+/* Serializes create and destroy: at most one xswap device exists. */
+static DEFINE_MUTEX(xswap_ctl_lock);
+#else /* !CONFIG_XSWAP */
+static inline int xswap_create_by_name(const char *name, int swap_flags)
+{
+ return -ENOENT;
+}
+#endif /* CONFIG_XSWAP */
static void swap_range_alloc(struct swap_info_struct *si,
unsigned int nr_entries);
@@ -84,7 +105,6 @@ atomic_t nr_real_swapfiles;
EXPORT_SYMBOL_GPL(nr_swap_pages);
/* protected with swap_lock. reading in vm_swap_full() doesn't need lock */
long total_swap_pages;
-#define DEF_SWAP_PRIO -1
unsigned long swapfile_maximum_size;
#ifdef CONFIG_MIGRATION
bool swap_migration_ad_supported;
@@ -2334,6 +2354,9 @@ static int __find_hibernation_swap_type(dev_t device, sector_t offset)
if (!(sis->flags & SWP_WRITEOK))
continue;
+ /* xswap has no bdev to match a resume device */
+ if (sis->flags & SWP_XSWAP)
+ continue;
if (device == sis->bdev->bd_dev) {
struct swap_extent *se = first_se(sis);
@@ -2521,6 +2544,8 @@ int find_first_swap(dev_t *device)
if (!(sis->flags & SWP_WRITEOK))
continue;
+ if (sis->flags & SWP_XSWAP)
+ continue;
*device = sis->bdev->bd_dev;
spin_unlock(&swap_lock);
return type;
@@ -3497,7 +3522,8 @@ static void *swap_start(struct seq_file *swap, loff_t *pos)
return SEQ_START_TOKEN;
for (type = 0; (si = swap_type_to_info(type)); type++) {
- if (!(si->swap_file))
+ if (!si->swap_file &&
+ !((si->flags & SWP_XSWAP) && (si->flags & SWP_WRITEOK)))
continue;
if (!--l)
return si;
@@ -3518,7 +3544,8 @@ static void *swap_next(struct seq_file *swap, void *v, loff_t *pos)
++(*pos);
for (; (si = swap_type_to_info(type)); type++) {
- if (!(si->swap_file))
+ if (!si->swap_file &&
+ !((si->flags & SWP_XSWAP) && (si->flags & SWP_WRITEOK)))
continue;
return si;
}
@@ -3560,7 +3587,13 @@ static int swap_show(struct seq_file *swap, void *v)
inuse = K(swap_usage_in_pages(si));
file = si->swap_file;
- len = seq_file_path(swap, file, " \t\n\\");
+ if (file) {
+ len = seq_file_path(swap, file, " \t\n\\");
+ } else {
+ /* The file-less xswap device; one per system. */
+ len = strlen(XSWAP_NAME);
+ seq_puts(swap, XSWAP_NAME);
+ }
seq_printf(swap, "%*s%s\t%lu\t%s%lu\t%s%d\n",
len < 40 ? 40 - len : 1, " ",
swap_type_str(si),
@@ -3874,7 +3907,7 @@ static int setup_swap_clusters_info(struct swap_info_struct *si,
/* xswap: only cluster 0 slot 0 is bad */
err = swap_cluster_setup_bad_slot(si, cluster_info, 0, false);
if (err)
- goto err_free_vm;
+ goto err_unmap;
INIT_LIST_HEAD(&si->free_clusters);
INIT_LIST_HEAD(&si->full_clusters);
@@ -3903,6 +3936,9 @@ static int setup_swap_clusters_info(struct swap_info_struct *si,
return 0;
+err_unmap:
+ /* swap_cluster_setup_bad_slot() allocates cluster 0's table. */
+ swap_cluster_free_table(&si->cluster_info[0]);
err_free_vm:
vfree(si->cluster_info);
si->cluster_vm = NULL;
@@ -3985,6 +4021,117 @@ static int setup_swap_clusters_info(struct swap_info_struct *si,
return err;
}
+#ifdef CONFIG_XSWAP
+/*
+ * Create the file-less xswap device, sized at twice RAM. Called from
+ * swapon(2) with SWAP_FLAG_XSWAP; there is at most one device per system.
+ */
+static int xswap_create(int prio)
+{
+ struct swap_info_struct *si;
+ unsigned long ram, maxpages;
+ unsigned int type;
+ int error;
+
+ mutex_lock(&xswap_ctl_lock);
+
+ spin_lock(&swap_lock);
+ for (type = 0; type < nr_swapfiles; type++) {
+ struct swap_info_struct *q = swap_info[type];
+
+ if (q && (q->flags & SWP_XSWAP)) {
+ spin_unlock(&swap_lock);
+ error = -EBUSY;
+ goto out;
+ }
+ }
+ spin_unlock(&swap_lock);
+
+ si = alloc_swap_info();
+ if (IS_ERR(si)) {
+ error = PTR_ERR(si);
+ goto out;
+ }
+
+ INIT_WORK(&si->discard_work, swap_discard_work);
+ INIT_WORK(&si->reclaim_work, swap_reclaim_work);
+
+ ram = totalram_pages();
+ maxpages = min_t(unsigned long, ram * 2, swapfile_maximum_size);
+ /* A cluster index is an unsigned int, so cap the cluster count. */
+ if (maxpages / SWAPFILE_CLUSTER > UINT_MAX)
+ maxpages = (unsigned long)UINT_MAX * SWAPFILE_CLUSTER;
+ /* Cluster-aligned, so no cluster holds a slot past si->max. */
+ if (maxpages > SWAPFILE_CLUSTER)
+ maxpages = rounddown(maxpages, SWAPFILE_CLUSTER);
+ if (maxpages < 2)
+ maxpages = 2;
+
+ si->bdev = NULL;
+ si->swap_file = NULL;
+ si->flags |= SWP_XSWAP | SWP_SOLIDSTATE;
+ si->max = maxpages;
+ si->pages = maxpages - 1;
+ /*
+ * setup_swap_extents() is file-only, so set ops here; only
+ * ops->flags is used, and the IO methods are never called.
+ */
+ si->ops = &swap_bdev_ops;
+
+ error = setup_swap_clusters_info(si, NULL, maxpages);
+ if (error)
+ goto bad_swap;
+
+ error = zswap_swapon(si->type, si->max);
+ if (error)
+ goto bad_swap;
+
+ mutex_lock(&swapon_mutex);
+ si->prio = prio;
+ si->list.prio = -si->prio;
+ si->avail_list.prio = -si->prio;
+ enable_swap_info(si);
+ pr_info("xswap: adding extendable swap device %s (prio %d, %lu pages)\n",
+ XSWAP_NAME, prio, si->pages);
+ mutex_unlock(&swapon_mutex);
+ atomic_inc(&proc_poll_event);
+ wake_up_interruptible(&proc_poll_wait);
+
+ error = 0;
+out:
+ mutex_unlock(&xswap_ctl_lock);
+ return error;
+
+bad_swap:
+ kfree(si->global_cluster);
+ si->global_cluster = NULL;
+ destroy_swap_extents(si, NULL); /* safe: xswap never sets SWP_ACTIVATED */
+ free_swap_cluster_info(si);
+ si->cluster_info = NULL;
+ spin_lock(&swap_lock);
+ si->flags = 0;
+ spin_unlock(&swap_lock);
+ goto out;
+}
+
+/* A name, not a path: only -ENOENT from opening it gets here. */
+static int xswap_create_by_name(const char *name, int swap_flags)
+{
+ int prio = DEF_XSWAP_PRIO;
+
+ if (strcmp(name, XSWAP_NAME))
+ return -ENOENT;
+
+ if (swap_flags & (SWAP_FLAG_DISCARD | SWAP_FLAG_DISCARD_ONCE |
+ SWAP_FLAG_DISCARD_PAGES))
+ return -EINVAL;
+ if (swap_flags & SWAP_FLAG_PREFER)
+ prio = swap_flags & SWAP_FLAG_PRIO_MASK;
+
+ return xswap_create(prio);
+}
+#endif /* CONFIG_XSWAP */
+
SYSCALL_DEFINE2(swapon, const char __user *, specialfile, int, swap_flags)
{
struct swap_info_struct *si;
@@ -4007,26 +4154,36 @@ SYSCALL_DEFINE2(swapon, const char __user *, specialfile, int, swap_flags)
if (!capable(CAP_SYS_ADMIN))
return -EPERM;
+ CLASS(filename, name)(specialfile);
+ if (IS_ERR(name))
+ return PTR_ERR(name);
+
+ swap_file = file_open_name(name, O_RDWR | O_LARGEFILE | O_EXCL, 0);
+ if (IS_ERR(swap_file)) {
+ error = PTR_ERR(swap_file);
+ /*
+ * Only -ENOENT is the request name; a file of that name
+ * keeps precedence, as it does for swapoff.
+ */
+ if (error == -ENOENT)
+ return xswap_create_by_name(name->name, swap_flags);
+ return error;
+ }
+
/*
* Allocate or reuse existing !SWP_USED swap_info. The returned
* si will stay in a dying status, so nothing will access its content
* until enable_swap_info resurrects its percpu ref and expose it.
*/
si = alloc_swap_info();
- if (IS_ERR(si))
+ if (IS_ERR(si)) {
+ filp_close(swap_file, NULL);
return PTR_ERR(si);
+ }
INIT_WORK(&si->discard_work, swap_discard_work);
INIT_WORK(&si->reclaim_work, swap_reclaim_work);
- CLASS(filename, name)(specialfile);
- swap_file = file_open_name(name, O_RDWR | O_LARGEFILE | O_EXCL, 0);
- if (IS_ERR(swap_file)) {
- error = PTR_ERR(swap_file);
- swap_file = NULL;
- goto bad_swap;
- }
-
mapping = swap_file->f_mapping;
dentry = swap_file->f_path.dentry;
inode = mapping->host;
--
2.54.0
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v6 07/14] mm, swap: refactor swapoff and add xswap_destroy
2026-11-22 12:39 [PATCH v6 00/14] mm, swap: extendable swap devices (xswap phase I) Baoquan He
` (5 preceding siblings ...)
2026-11-22 12:39 ` [PATCH v6 06/14] mm, swap: create the xswap device from swapon(2) Baoquan He
@ 2026-11-22 12:39 ` Baoquan He
2026-11-22 12:39 ` [PATCH v6 08/14] mm, swap: require zswap for xswap devices Baoquan He
` (2 subsequent siblings)
9 siblings, 0 replies; 15+ messages in thread
From: Baoquan He @ 2026-11-22 12:39 UTC (permalink / raw)
To: linux-mm
Cc: akpm, david, urezki, chrisl, kasong, nphamcs, baohua,
youngjun.park, shikemeng, hannes, yosry, chengming.zhou, corbet,
kunwu.chan, klarasmodin, linux-kernel, baoquan.he, Baoquan He
From: Baoquan He <hebaoquan@kylinos.cn>
Extract __swapoff() from sys_swapoff() so the teardown logic can be
shared, and make it work for file-less devices.
Add xswap_destroy(), reached through the name /proc/swaps prints.
"xswap" is not a path, so the name cannot be opened, and "swapoff -a"
passes it back unchanged.
A real file or directory keeps precedence: only a name that cannot be
opened (-ENOENT) is taken for the device, so a plain file is never
mistaken for it. The name is compared with strcmp() against the one
name the kernel prints, so there is no parsing.
Destroy is serialized with create by xswap_ctl_lock, and it refuses a
device that is pinned for hibernation.
Signed-off-by: Baoquan He <hebaoquan@kylinos.cn>
---
mm/swapfile.c | 148 +++++++++++++++++++++++++++++++++++++++++++-------
1 file changed, 127 insertions(+), 21 deletions(-)
diff --git a/mm/swapfile.c b/mm/swapfile.c
index 4e598ab8da65..1f319e365b68 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -69,6 +69,8 @@
static int xswap_create(int prio);
static int xswap_create_by_name(const char *name, int swap_flags);
+static int xswap_destroy(void);
+static int xswap_destroy_by_name(const char *name);
/* Serializes create and destroy: at most one xswap device exists. */
static DEFINE_MUTEX(xswap_ctl_lock);
@@ -77,6 +79,11 @@ static inline int xswap_create_by_name(const char *name, int swap_flags)
{
return -ENOENT;
}
+
+static inline int xswap_destroy_by_name(const char *name)
+{
+ return -ENOENT;
+}
#endif /* CONFIG_XSWAP */
static void swap_range_alloc(struct swap_info_struct *si,
@@ -3360,13 +3367,14 @@ static void flush_percpu_swap_cluster(struct swap_info_struct *si)
}
}
+static int swap_info_remove(struct swap_info_struct *p);
+static int __swapoff(struct swap_info_struct *p);
SYSCALL_DEFINE1(swapoff, const char __user *, specialfile)
{
struct swap_info_struct *p = NULL;
- struct file *swap_file, *victim;
+ struct file *victim;
struct address_space *mapping;
- struct inode *inode;
int err, found = 0;
if (!capable(CAP_SYS_ADMIN))
@@ -3375,15 +3383,30 @@ SYSCALL_DEFINE1(swapoff, const char __user *, specialfile)
BUG_ON(!current->mm);
CLASS(filename, pathname)(specialfile);
+ if (IS_ERR(pathname))
+ return PTR_ERR(pathname);
+
victim = file_open_name(pathname, O_RDWR|O_LARGEFILE, 0);
- if (IS_ERR(victim))
- return PTR_ERR(victim);
+ if (IS_ERR(victim)) {
+ /*
+ * /proc/swaps shows a name that cannot be opened; accept it
+ * so swapoff -a works. Only -ENOENT means "no such path",
+ * so a file of that name keeps precedence.
+ */
+ if (PTR_ERR(victim) != -ENOENT)
+ return PTR_ERR(victim);
+
+ err = xswap_destroy_by_name(pathname->name);
+ if (err == -ENOENT)
+ return PTR_ERR(victim);
+ return err;
+ }
mapping = victim->f_mapping;
spin_lock(&swap_lock);
plist_for_each_entry(p, &swap_active_head, list) {
if (p->flags & SWP_WRITEOK) {
- if (p->swap_file->f_mapping == mapping) {
+ if (p->swap_file && p->swap_file->f_mapping == mapping) {
found = 1;
break;
}
@@ -3402,20 +3425,50 @@ SYSCALL_DEFINE1(swapoff, const char __user *, specialfile)
goto out_dput;
}
- if (!security_vm_enough_memory_mm(current->mm, p->pages))
- vm_unacct_memory(p->pages);
- else {
- err = -ENOMEM;
+ err = swap_info_remove(p);
+ if (err) {
spin_unlock(&swap_lock);
goto out_dput;
}
+ spin_unlock(&swap_lock);
+
+ err = __swapoff(p);
+
+out_dput:
+ filp_close(victim, NULL);
+ return err;
+}
+
+/*
+ * Drop @p from the avail and active lists and undo its accounting. The
+ * caller must hold swap_lock and have checked that @p is WRITEOK and not
+ * pinned for hibernation.
+ *
+ * Returns 0, or -ENOMEM with swap_lock still held.
+ */
+static int swap_info_remove(struct swap_info_struct *p)
+{
+ if (!security_vm_enough_memory_mm(current->mm, p->pages))
+ vm_unacct_memory(p->pages);
+ else
+ return -ENOMEM;
+
spin_lock(&p->lock);
del_from_avail_list(p, true);
plist_del(&p->list, &swap_active_head);
atomic_long_sub(p->pages, &nr_swap_pages);
total_swap_pages -= p->pages;
spin_unlock(&p->lock);
- spin_unlock(&swap_lock);
+ return 0;
+}
+
+/* Common swap teardown after list removal; shared by sys_swapoff() and
+ * xswap_destroy().
+ */
+static int __swapoff(struct swap_info_struct *p)
+{
+ struct file *swap_file = NULL;
+ int err;
wait_for_allocation(p);
@@ -3426,7 +3479,7 @@ SYSCALL_DEFINE1(swapoff, const char __user *, specialfile)
if (err) {
/* re-insert swap space back into swap_list */
reinsert_swap_info(p);
- goto out_dput;
+ return err;
}
/*
@@ -3466,15 +3519,21 @@ SYSCALL_DEFINE1(swapoff, const char __user *, specialfile)
kfree(p->global_cluster);
p->global_cluster = NULL;
free_swap_cluster_info(p);
+ /*
+ * The device is off swap_active_head and no longer WRITEOK, so no
+ * reader can observe these; clearing them here needs no lock.
+ */
p->max = 0;
p->cluster_info = NULL;
- inode = mapping->host;
+ if (swap_file) {
+ struct inode *inode = swap_file->f_mapping->host;
- inode_lock(inode);
- inode->i_flags &= ~S_SWAPFILE;
- inode_unlock(inode);
- filp_close(swap_file, NULL);
+ inode_lock(inode);
+ inode->i_flags &= ~S_SWAPFILE;
+ inode_unlock(inode);
+ filp_close(swap_file, NULL);
+ }
/*
* Clear the SWP_USED flag after all resources are freed so that swapon
@@ -3485,13 +3544,10 @@ SYSCALL_DEFINE1(swapoff, const char __user *, specialfile)
p->flags = 0;
spin_unlock(&swap_lock);
- err = 0;
atomic_inc(&proc_poll_event);
wake_up_interruptible(&proc_poll_wait);
-out_dput:
- filp_close(victim, NULL);
- return err;
+ return 0;
}
#ifdef CONFIG_PROC_FS
@@ -4114,7 +4170,57 @@ static int xswap_create(int prio)
goto out;
}
-/* A name, not a path: only -ENOENT from opening it gets here. */
+/* Tear down the file-less xswap device. */
+static int xswap_destroy(void)
+{
+ struct swap_info_struct *p = NULL;
+ unsigned int type;
+ int err;
+
+ mutex_lock(&xswap_ctl_lock);
+
+ spin_lock(&swap_lock);
+ /* Only a fully enabled device; a half-created one does not count. */
+ for (type = 0; type < nr_swapfiles; type++) {
+ struct swap_info_struct *q = swap_info[type];
+
+ if (q && (q->flags & (SWP_XSWAP | SWP_WRITEOK)) ==
+ (SWP_XSWAP | SWP_WRITEOK)) {
+ p = q;
+ break;
+ }
+ }
+ if (!p) {
+ spin_unlock(&swap_lock);
+ err = -ENOENT;
+ goto out;
+ }
+ /* Refuse swapoff while the device is pinned for hibernation */
+ if (p->flags & SWP_HIBERNATION) {
+ spin_unlock(&swap_lock);
+ err = -EBUSY;
+ goto out;
+ }
+
+ err = swap_info_remove(p);
+ spin_unlock(&swap_lock);
+ if (!err)
+ err = __swapoff(p);
+out:
+ mutex_unlock(&xswap_ctl_lock);
+ return err;
+}
+
+/* /proc/swaps shows XSWAP_NAME; accept it so swapoff -a works. */
+static int xswap_destroy_by_name(const char *name)
+{
+ if (strcmp(name, XSWAP_NAME))
+ return -ENOENT;
+
+ return xswap_destroy();
+}
+
+/* The swapon counterpart of xswap_destroy_by_name(). */
static int xswap_create_by_name(const char *name, int swap_flags)
{
int prio = DEF_XSWAP_PRIO;
--
2.54.0
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v6 08/14] mm, swap: require zswap for xswap devices
2026-11-22 12:39 [PATCH v6 00/14] mm, swap: extendable swap devices (xswap phase I) Baoquan He
` (6 preceding siblings ...)
2026-11-22 12:39 ` [PATCH v6 07/14] mm, swap: refactor swapoff and add xswap_destroy Baoquan He
@ 2026-11-22 12:39 ` Baoquan He
2026-11-22 12:39 ` [PATCH v6 09/14] mm, swap: let the command line set the xswap device size Baoquan He
2026-11-22 12:48 ` [PATCH v6 10/14] mm, swap: add CONFIG_XSWAP_DEFAULT_MAX Baoquan He
9 siblings, 0 replies; 15+ messages in thread
From: Baoquan He @ 2026-11-22 12:39 UTC (permalink / raw)
To: linux-mm
Cc: akpm, david, urezki, chrisl, kasong, nphamcs, baohua,
youngjun.park, shikemeng, hannes, yosry, chengming.zhou, corbet,
kunwu.chan, klarasmodin, linux-kernel, baoquan.he, Baoquan He
From: Baoquan He <hebaoquan@kylinos.cn>
xswap pages live only in zswap. Without zswap, swapout cannot
free the page, but still takes a swap entry. So the device consumes
swap entries without freeing memory. Fail to create a device when
zswap is unavailable.
Acked-by: Chris Li <chrisl@kernel.org>
Signed-off-by: Baoquan He <hebaoquan@kylinos.cn>
---
mm/swapfile.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/mm/swapfile.c b/mm/swapfile.c
index 1f319e365b68..ba899e349e99 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -4089,6 +4089,10 @@ static int xswap_create(int prio)
unsigned int type;
int error;
+ /* xswap has no backing store, it relies on zswap. */
+ if (!zswap_is_enabled())
+ return -EOPNOTSUPP;
+
mutex_lock(&xswap_ctl_lock);
spin_lock(&swap_lock);
--
2.54.0
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v6 09/14] mm, swap: let the command line set the xswap device size
2026-11-22 12:39 [PATCH v6 00/14] mm, swap: extendable swap devices (xswap phase I) Baoquan He
` (7 preceding siblings ...)
2026-11-22 12:39 ` [PATCH v6 08/14] mm, swap: require zswap for xswap devices Baoquan He
@ 2026-11-22 12:39 ` Baoquan He
2026-11-22 12:48 ` [PATCH v6 10/14] mm, swap: add CONFIG_XSWAP_DEFAULT_MAX Baoquan He
9 siblings, 0 replies; 15+ messages in thread
From: Baoquan He @ 2026-11-22 12:39 UTC (permalink / raw)
To: linux-mm
Cc: akpm, david, urezki, chrisl, kasong, nphamcs, baohua,
youngjun.park, shikemeng, hannes, yosry, chengming.zhou, corbet,
kunwu.chan, klarasmodin, linux-kernel, baoquan.he, Baoquan He
From: Baoquan He <hebaoquan@kylinos.cn>
By default, a new xswap device is 200% of RAM, which is twice RAM.
Add xswap.max= to set the size of devices created after it:
xswap.max=300% three times RAM
xswap.max=8G eight GiB
xswap.max=max the largest size the device can address
The default is fine for most users. But a machine with a lot of RAM
and a small working set can get a SwapTotal much larger than it needs.
The size is resolved at device creation, not at parameter parsing,
because a percentage depends on RAM, which is not known yet when the
parameters are parsed.
An explicit size that is too large or overflows is refused; the default
is clamped instead. A size below one cluster is also refused.
Signed-off-by: Baoquan He <hebaoquan@kylinos.cn>
---
.../admin-guide/kernel-parameters.txt | 7 +
mm/swapfile.c | 135 ++++++++++++++++--
2 files changed, 130 insertions(+), 12 deletions(-)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 176648941aec..9928f0937514 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -8731,3 +8731,10 @@ Kernel parameters
memory, and other data can't be written using
xmon commands.
off xmon is disabled.
+
+ xswap.max= [KNL,SWAP]
+ Maximum size of a new xswap device. The value is a
+ size in bytes with an optional KMGTPE suffix, a
+ percentage of RAM, or "max" for the largest size
+ the device can address.
+ Format: <size>[KMGTPE] | <percent>% | max
diff --git a/mm/swapfile.c b/mm/swapfile.c
index ba899e349e99..6ce5dbb3b103 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -14,6 +14,7 @@
#include <linux/mman.h>
#include <linux/slab.h>
#include <linux/kernel_stat.h>
+#include <linux/overflow.h>
#include <linux/swap.h>
#include <linux/vmalloc.h>
#include <linux/pagemap.h>
@@ -74,6 +75,102 @@ static int xswap_destroy_by_name(const char *name);
/* Serializes create and destroy: at most one xswap device exists. */
static DEFINE_MUTEX(xswap_ctl_lock);
+
+/*
+ * Size of a new device: xswap.max= if given, otherwise the built-in
+ * default. A percentage unless bytes is set; "max" asks for the largest
+ * size the device can address.
+ */
+static unsigned int xswap_size_pct = 200;
+static unsigned long xswap_size_bytes;
+static bool xswap_size_max;
+static bool xswap_size_set; /* xswap.max= was given */
+
+/*
+ * Parse "200%", "2T" or "max". @str is not modified.
+ */
+static int __init xswap_parse_size(const char *str, const char *what)
+{
+ const char *pct;
+ char buf[16];
+ unsigned int pct_val;
+ unsigned long bytes;
+ char *end;
+
+ pct = strchr(str, '%');
+ if (!strcmp(str, "max")) {
+ xswap_size_max = true;
+ xswap_size_pct = 0;
+ xswap_size_bytes = 0;
+ } else if (pct) {
+ size_t len = pct - str;
+
+ if (pct[1] || !len || len >= sizeof(buf))
+ goto bad;
+ memcpy(buf, str, len);
+ buf[len] = '\0';
+ if (kstrtouint(buf, 0, &pct_val) || !pct_val)
+ goto bad;
+ xswap_size_pct = pct_val;
+ xswap_size_bytes = 0;
+ xswap_size_max = false;
+ } else {
+ bytes = memparse(str, &end);
+ if (*end || bytes < PAGE_SIZE)
+ goto bad;
+ xswap_size_bytes = bytes;
+ xswap_size_pct = 0;
+ xswap_size_max = false;
+ }
+
+ xswap_size_set = true;
+ return 0;
+bad:
+ pr_err("xswap: bad %s=%s\n", what, str);
+ return -EINVAL;
+}
+
+static int __init xswap_max_setup(char *str)
+{
+ xswap_parse_size(str, "xswap.max");
+ return 1;
+}
+__setup("xswap.max=", xswap_max_setup);
+
+/*
+ * The most a device can address, whatever xswap.max= asked for. An xswap
+ * slot is still a swap entry, so the swap offset limit applies even though
+ * the data never goes through a physical address.
+ */
+static unsigned long xswap_size_limit(void)
+{
+ unsigned long limit = swapfile_maximum_size;
+
+ /* A cluster index is an unsigned int. */
+ if (limit / SWAPFILE_CLUSTER > UINT_MAX)
+ limit = (unsigned long)UINT_MAX * SWAPFILE_CLUSTER;
+
+ return limit;
+}
+
+static unsigned long xswap_size_pages(void)
+{
+ unsigned long long pages;
+
+ if (xswap_size_max)
+ return xswap_size_limit();
+
+ if (xswap_size_bytes)
+ return xswap_size_bytes >> PAGE_SHIFT;
+
+ /* Let xswap_create() refuse a size that does not fit in the type. */
+ if (check_mul_overflow((unsigned long long)totalram_pages(),
+ (unsigned long long)xswap_size_pct, &pages))
+ return ULONG_MAX;
+
+ return div_u64(pages, 100);
+}
+
#else /* !CONFIG_XSWAP */
static inline int xswap_create_by_name(const char *name, int swap_flags)
{
@@ -4079,13 +4176,15 @@ static int setup_swap_clusters_info(struct swap_info_struct *si,
#ifdef CONFIG_XSWAP
/*
- * Create the file-less xswap device, sized at twice RAM. Called from
- * swapon(2) with SWAP_FLAG_XSWAP; there is at most one device per system.
+ * Create the file-less xswap device, sized by xswap.max= or the built-in
+ * default. Called from swapon(2) with SWAP_FLAG_XSWAP; there is at most
+ * one device per system.
*/
static int xswap_create(int prio)
{
struct swap_info_struct *si;
- unsigned long ram, maxpages;
+ unsigned long maxpages;
+ unsigned long limit;
unsigned int type;
int error;
@@ -4093,6 +4192,26 @@ static int xswap_create(int prio)
if (!zswap_is_enabled())
return -EOPNOTSUPP;
+ /*
+ * Refuse an explicitly requested size the device cannot address;
+ * clamp the default size, which nobody asked for.
+ */
+ limit = xswap_size_limit();
+ maxpages = xswap_size_pages();
+ if (xswap_size_set && maxpages > limit) {
+ pr_err("xswap: xswap.max is larger than the device can address\n");
+ return -EINVAL;
+ }
+ maxpages = min(maxpages, limit);
+ /*
+ * Below one cluster, si->max is not cluster-aligned and a slot in
+ * the only cluster would be past the end of the device.
+ */
+ if (maxpages < SWAPFILE_CLUSTER) {
+ pr_err("xswap: the configured size is smaller than one cluster\n");
+ return -EINVAL;
+ }
+
mutex_lock(&xswap_ctl_lock);
spin_lock(&swap_lock);
@@ -4116,16 +4235,8 @@ static int xswap_create(int prio)
INIT_WORK(&si->discard_work, swap_discard_work);
INIT_WORK(&si->reclaim_work, swap_reclaim_work);
- ram = totalram_pages();
- maxpages = min_t(unsigned long, ram * 2, swapfile_maximum_size);
- /* A cluster index is an unsigned int, so cap the cluster count. */
- if (maxpages / SWAPFILE_CLUSTER > UINT_MAX)
- maxpages = (unsigned long)UINT_MAX * SWAPFILE_CLUSTER;
/* Cluster-aligned, so no cluster holds a slot past si->max. */
- if (maxpages > SWAPFILE_CLUSTER)
- maxpages = rounddown(maxpages, SWAPFILE_CLUSTER);
- if (maxpages < 2)
- maxpages = 2;
+ maxpages = rounddown(maxpages, SWAPFILE_CLUSTER);
si->bdev = NULL;
si->swap_file = NULL;
--
2.54.0
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v6 10/14] mm, swap: add CONFIG_XSWAP_DEFAULT_MAX
2026-11-22 12:39 [PATCH v6 00/14] mm, swap: extendable swap devices (xswap phase I) Baoquan He
` (8 preceding siblings ...)
2026-11-22 12:39 ` [PATCH v6 09/14] mm, swap: let the command line set the xswap device size Baoquan He
@ 2026-11-22 12:48 ` Baoquan He
2026-11-22 12:48 ` [PATCH v6 11/14] mm, swap: add xswap grow trigger on cluster allocation Baoquan He
` (3 more replies)
9 siblings, 4 replies; 15+ messages in thread
From: Baoquan He @ 2026-11-22 12:48 UTC (permalink / raw)
To: linux-mm
Cc: akpm, david, urezki, chrisl, kasong, nphamcs, baohua,
youngjun.park, shikemeng, hannes, yosry, chengming.zhou, corbet,
kunwu.chan, klarasmodin, linux-kernel, baoquan.he, Baoquan He
From: Baoquan He <hebaoquan@kylinos.cn>
Add a build-time default for the size of a new xswap device. The value
can be a percentage of RAM, a size in bytes, or "max". "max" means the
largest size the device can address. xswap.max= on the command line
overrides this default.
The build-time default is clamped to the device limit. An explicit
xswap.max= that is too large is refused.
If the build-time default cannot be parsed, or is a byte size below one
cluster, use 200% instead. A percentage default can still be below one
cluster after RAM is known. In that case, device creation fails.
Signed-off-by: Baoquan He <hebaoquan@kylinos.cn>
---
.../admin-guide/kernel-parameters.txt | 6 ++--
mm/Kconfig | 11 ++++++
mm/swapfile.c | 34 ++++++++++++-------
3 files changed, 37 insertions(+), 14 deletions(-)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 9928f0937514..06d37212399f 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -8733,8 +8733,10 @@ Kernel parameters
off xmon is disabled.
xswap.max= [KNL,SWAP]
- Maximum size of a new xswap device. The value is a
+ Maximum size of a new xswap device, which is created
+ with swapon(2) and the spec "xswap". The value is a
size in bytes with an optional KMGTPE suffix, a
percentage of RAM, or "max" for the largest size
- the device can address.
+ the device can address. The default is set by
+ CONFIG_XSWAP_DEFAULT_MAX.
Format: <size>[KMGTPE] | <percent>% | max
diff --git a/mm/Kconfig b/mm/Kconfig
index a8707bbd8d62..f13639a811e2 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -131,6 +131,17 @@ config XSWAP
array is backed by a sparse vmalloc area that is mapped lazily
in chunks, avoiding static pre-allocation overhead.
+config XSWAP_DEFAULT_MAX
+ string "Default maximum size of a new xswap device"
+ depends on XSWAP
+ default "200%"
+ help
+ The maximum size of a newly created xswap device. It accepts a
+ percentage of RAM (for example "200%"), a size (for example "2T"),
+ or "max" for the largest size the device can address.
+
+ xswap.max= on the kernel command line overrides this value.
+
config ZSMALLOC
tristate
diff --git a/mm/swapfile.c b/mm/swapfile.c
index 6ce5dbb3b103..074c5114d129 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -77,9 +77,9 @@ static int xswap_destroy_by_name(const char *name);
static DEFINE_MUTEX(xswap_ctl_lock);
/*
- * Size of a new device: xswap.max= if given, otherwise the built-in
- * default. A percentage unless bytes is set; "max" asks for the largest
- * size the device can address.
+ * Size of a new device: xswap.max= if given, otherwise
+ * CONFIG_XSWAP_DEFAULT_MAX. A percentage unless bytes is set; "max" asks
+ * for the largest size the device can address.
*/
static unsigned int xswap_size_pct = 200;
static unsigned long xswap_size_bytes;
@@ -87,9 +87,12 @@ static bool xswap_size_max;
static bool xswap_size_set; /* xswap.max= was given */
/*
- * Parse "200%", "2T" or "max". @str is not modified.
+ * Parse "200%", "2T" or "max". @str is not modified. @cmdline tells an
+ * explicit xswap.max= from the build-time default, which is clamped
+ * instead of refused when it is too large.
*/
-static int __init xswap_parse_size(const char *str, const char *what)
+static int __init xswap_parse_size(const char *str, const char *what,
+ bool cmdline)
{
const char *pct;
char buf[16];
@@ -118,12 +121,16 @@ static int __init xswap_parse_size(const char *str, const char *what)
bytes = memparse(str, &end);
if (*end || bytes < PAGE_SIZE)
goto bad;
+ /* A default below one cluster would fail every create. */
+ if (!cmdline && bytes < (unsigned long)SWAPFILE_CLUSTER * PAGE_SIZE)
+ goto bad;
xswap_size_bytes = bytes;
xswap_size_pct = 0;
xswap_size_max = false;
}
- xswap_size_set = true;
+ if (cmdline)
+ xswap_size_set = true;
return 0;
bad:
pr_err("xswap: bad %s=%s\n", what, str);
@@ -132,7 +139,7 @@ static int __init xswap_parse_size(const char *str, const char *what)
static int __init xswap_max_setup(char *str)
{
- xswap_parse_size(str, "xswap.max");
+ xswap_parse_size(str, "xswap.max", true);
return 1;
}
__setup("xswap.max=", xswap_max_setup);
@@ -4175,11 +4182,7 @@ static int setup_swap_clusters_info(struct swap_info_struct *si,
}
#ifdef CONFIG_XSWAP
-/*
- * Create the file-less xswap device, sized by xswap.max= or the built-in
- * default. Called from swapon(2) with SWAP_FLAG_XSWAP; there is at most
- * one device per system.
- */
+/* Create the file-less xswap device from swapon(2). */
static int xswap_create(int prio)
{
struct swap_info_struct *si;
@@ -4694,6 +4697,13 @@ static int __init swapfile_init(void)
{
swapfile_maximum_size = arch_max_swapfile_size();
+#ifdef CONFIG_XSWAP
+ /* xswap.max= on the command line overrides the build-time default. */
+ if (!xswap_size_set)
+ xswap_parse_size(CONFIG_XSWAP_DEFAULT_MAX,
+ "CONFIG_XSWAP_DEFAULT_MAX", false);
+#endif
+
/*
* Once a cluster is freed, it's swap table content is read
* only, and all swap cache readers (swap_cache_*) verifies
--
2.54.0
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v6 11/14] mm, swap: add xswap grow trigger on cluster allocation
2026-11-22 12:48 ` [PATCH v6 10/14] mm, swap: add CONFIG_XSWAP_DEFAULT_MAX Baoquan He
@ 2026-11-22 12:48 ` Baoquan He
2026-11-22 12:48 ` [PATCH v6 12/14] mm, swap: add debugfs counters for xswap Baoquan He
` (2 subsequent siblings)
3 siblings, 0 replies; 15+ messages in thread
From: Baoquan He @ 2026-11-22 12:48 UTC (permalink / raw)
To: linux-mm
Cc: akpm, david, urezki, chrisl, kasong, nphamcs, baohua,
youngjun.park, shikemeng, hannes, yosry, chengming.zhou, corbet,
kunwu.chan, klarasmodin, linux-kernel, baoquan.he, Baoquan He
From: Baoquan He <hebaoquan@kylinos.cn>
Grow the mapped xswap range before it runs out. When usage reaches 85%
of the mapped range, map more clusters from cluster_alloc_swap_entry().
Grow to about 65% usage. This leaves free space in the mapped range
before the next grow at 85%, so one grow is not immediately followed by
another.
Mapping pages can sleep. Grow before the allocation that needs them,
not after it fails.
Signed-off-by: Baoquan He <hebaoquan@kylinos.cn>
---
include/linux/swap.h | 1 +
mm/swapfile.c | 124 ++++++++++++++++++++++++++++++++++++++++---
2 files changed, 118 insertions(+), 7 deletions(-)
diff --git a/include/linux/swap.h b/include/linux/swap.h
index 02c3861a336d..5d9e5b6f2bd6 100644
--- a/include/linux/swap.h
+++ b/include/linux/swap.h
@@ -246,6 +246,7 @@ struct swap_info_struct {
unsigned long nr_clusters_max;/* total clusters in the xswap address space */
unsigned long nr_clusters_mapped; /* currently mapped cluster count */
struct mutex xswap_lock; /* serialize map/unmap operations */
+ atomic_long_t xswap_grows; /* grow operations so far */
#endif
struct list_head free_clusters; /* free clusters list */
struct list_head full_clusters; /* full clusters list */
diff --git a/mm/swapfile.c b/mm/swapfile.c
index 074c5114d129..213b66bb1d6d 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -68,6 +68,21 @@
#define XSWAP_GROW_CLUSTERS \
max_t(unsigned long, PAGE_SIZE / sizeof(struct swap_cluster_info), 16)
+/*
+ * Grow thresholds, as a percentage of the mapped range in use. The target
+ * sits below the next grow trigger, so a grow that follows a rise in usage
+ * does not immediately trigger another one.
+ */
+#define XSWAP_GROW_UNTIL 65
+#define XSWAP_GROW_WHEN 85
+#define XSWAP_GROW_CHUNKS 4
+
+static bool xswap_should_grow(struct swap_info_struct *si);
+static unsigned long xswap_grow(struct swap_info_struct *si);
+static int xswap_map_clusters(struct swap_info_struct *si,
+ unsigned long start_idx, unsigned long nr,
+ bool link_free);
+
static int xswap_create(int prio);
static int xswap_create_by_name(const char *name, int swap_flags);
static int xswap_destroy(void);
@@ -1325,6 +1340,13 @@ static unsigned long cluster_alloc_swap_entry(struct swap_info_struct *si,
if (order && !(si->flags & SWP_BLKDEV))
return 0;
+#ifdef CONFIG_XSWAP
+ /* Grow before allocating: mapping pages can sleep. */
+ if ((si->flags & SWP_XSWAP) && (si->flags & SWP_WRITEOK) &&
+ xswap_should_grow(si))
+ xswap_grow(si);
+#endif
+
if (!(si->flags & SWP_SOLIDSTATE)) {
/* Serialize HDD SWAP allocation for each device. */
spin_lock(&si->global_cluster_lock);
@@ -1397,6 +1419,12 @@ static unsigned long cluster_alloc_swap_entry(struct swap_info_struct *si,
if (found)
goto done;
}
+
+#ifdef CONFIG_XSWAP
+ /* A concurrent free or grow may have added clusters; retry once. */
+ if (!found && (si->flags & SWP_XSWAP))
+ found = alloc_swap_scan_list(si, &si->free_clusters, folio, false);
+#endif
done:
if (!(si->flags & SWP_SOLIDSTATE))
spin_unlock(&si->global_cluster_lock);
@@ -3986,8 +4014,39 @@ static unsigned long read_swap_header(struct swap_info_struct *si,
}
#ifdef CONFIG_XSWAP
+/*
+ * Link the new clusters and publish the larger range in one si->lock
+ * section: linking first makes a cluster visible to an allocator only
+ * after it is on the free list. @link_free is false for the first
+ * chunk, whose free list is not initialized yet.
+ */
+static void xswap_publish_clusters(struct swap_info_struct *si,
+ unsigned long start_idx, unsigned long nr,
+ bool link_free)
+{
+ unsigned long i;
+
+ spin_lock(&si->lock);
+ if (link_free) {
+ for (i = start_idx; i < start_idx + nr; i++) {
+ struct swap_cluster_info *ci = &si->cluster_info[i];
+
+ /* A concurrent grower cannot take these: xswap_lock. */
+ if (ci->flags != CLUSTER_FLAG_NONE)
+ continue;
+ list_add_tail(&ci->list, &si->free_clusters);
+ WRITE_ONCE(ci->flags, CLUSTER_FLAG_FREE);
+ ci->order = 0;
+ }
+ }
+ /* Pairs with the smp_load_acquire() in the readers of the range. */
+ smp_store_release(&si->nr_clusters_mapped, start_idx + nr);
+ spin_unlock(&si->lock);
+}
+
static int xswap_map_clusters(struct swap_info_struct *si,
- unsigned long start_idx, unsigned long nr)
+ unsigned long start_idx, unsigned long nr,
+ bool link_free)
{
/* vm_area_grow_pages() takes an offset into the area, not an address. */
unsigned long vm_end = PAGE_ALIGN((size_t)(start_idx + nr) *
@@ -4023,14 +4082,62 @@ static int xswap_map_clusters(struct swap_info_struct *si,
for (i = start_idx; i < start_idx + nr; i++)
spin_lock_init(&si->cluster_info[i].lock);
- /*
- * Publish the new mappings and cluster lock initialization before
- * the count; walkers without xswap_lock use smp_load_acquire().
- */
- smp_store_release(&si->nr_clusters_mapped, start_idx + nr);
+ /* The mappings and the locks are in place; make them visible. */
+ xswap_publish_clusters(si, start_idx, nr, link_free);
mutex_unlock(&si->xswap_lock);
return 0;
}
+static bool xswap_should_grow(struct swap_info_struct *si)
+{
+ unsigned long mapped = READ_ONCE(si->nr_clusters_mapped);
+
+ if (mapped >= READ_ONCE(si->nr_clusters_max))
+ return false;
+
+ return swap_usage_in_pages(si) * 100 >=
+ mapped * SWAPFILE_CLUSTER * XSWAP_GROW_WHEN;
+}
+
+/*
+ * Map more of the cluster_info array, up to GROW_UNTIL in use. The caller
+ * holds percpu_swap_cluster.lock, which this drops while it sleeps.
+ */
+static unsigned long xswap_grow(struct swap_info_struct *si)
+{
+ unsigned long mapped = READ_ONCE(si->nr_clusters_mapped);
+ unsigned long nr_new, want;
+ int ret;
+
+ lockdep_assert_held(&this_cpu_ptr(&percpu_swap_cluster)->lock);
+
+ want = DIV_ROUND_UP(swap_usage_in_pages(si) * 100,
+ XSWAP_GROW_UNTIL * SWAPFILE_CLUSTER);
+ if (want <= mapped)
+ return 0;
+
+ nr_new = rounddown(want - mapped, XSWAP_GROW_CLUSTERS);
+ if (!nr_new)
+ nr_new = XSWAP_GROW_CLUSTERS;
+ if (nr_new > XSWAP_GROW_CHUNKS * XSWAP_GROW_CLUSTERS)
+ nr_new = XSWAP_GROW_CHUNKS * XSWAP_GROW_CLUSTERS;
+ nr_new = min(nr_new, READ_ONCE(si->nr_clusters_max) - mapped);
+ if (!nr_new)
+ return 0;
+
+ /*
+ * Mapping pages can sleep. The lock only guards the per-cpu
+ * cluster cache, which this path does not touch.
+ */
+ local_unlock(&percpu_swap_cluster.lock);
+ ret = xswap_map_clusters(si, mapped, nr_new, true);
+ local_lock(&percpu_swap_cluster.lock);
+ if (ret)
+ return 0;
+
+ atomic_long_inc(&si->xswap_grows);
+ return nr_new;
+}
+
#endif /* CONFIG_XSWAP */
static int setup_swap_clusters_info(struct swap_info_struct *si,
@@ -4060,7 +4167,8 @@ static int setup_swap_clusters_info(struct swap_info_struct *si,
mutex_init(&si->xswap_lock);
err = xswap_map_clusters(si, 0, min_t(unsigned long,
- XSWAP_GROW_CLUSTERS, nr_clusters));
+ XSWAP_GROW_CLUSTERS, nr_clusters),
+ false);
if (err)
goto err_free_vm;
@@ -4252,6 +4360,8 @@ static int xswap_create(int prio)
*/
si->ops = &swap_bdev_ops;
+ atomic_long_set(&si->xswap_grows, 0);
+
error = setup_swap_clusters_info(si, NULL, maxpages);
if (error)
goto bad_swap;
--
2.54.0
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v6 12/14] mm, swap: add debugfs counters for xswap
2026-11-22 12:48 ` [PATCH v6 10/14] mm, swap: add CONFIG_XSWAP_DEFAULT_MAX Baoquan He
2026-11-22 12:48 ` [PATCH v6 11/14] mm, swap: add xswap grow trigger on cluster allocation Baoquan He
@ 2026-11-22 12:48 ` Baoquan He
2026-11-22 12:48 ` [PATCH v6 13/14] mm, swap: do not give an xswap slot to a cgroup that cannot zswap Baoquan He
2026-11-22 12:48 ` [PATCH v6 14/14] mm, swap: skip the overcommit probe when removing xswap Baoquan He
3 siblings, 0 replies; 15+ messages in thread
From: Baoquan He @ 2026-11-22 12:48 UTC (permalink / raw)
To: linux-mm
Cc: akpm, david, urezki, chrisl, kasong, nphamcs, baohua,
youngjun.park, shikemeng, hannes, yosry, chengming.zhou, corbet,
kunwu.chan, klarasmodin, linux-kernel, baoquan.he, Baoquan He
From: Baoquan He <hebaoquan@kylinos.cn>
The mapped range and the grow count are not visible from userspace.
This makes it hard to check if the grow thresholds work: we cannot see
how large the mapped range is, how much it grows, or how much free tail
it keeps.
Put this information in debugfs. It is a debugging aid, not a stable
interface, so it does not belong in sysfs or the ABI:
# cat /sys/kernel/debug/xswap/xswap.stats
type 0
clusters_max 2038783
clusters_mapped 255
usage_pages 91456
tail_free 16
grows 3
Signed-off-by: Baoquan He <hebaoquan@kylinos.cn>
---
include/linux/swap.h | 1 +
mm/swapfile.c | 69 ++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 70 insertions(+)
diff --git a/include/linux/swap.h b/include/linux/swap.h
index 5d9e5b6f2bd6..9daf0ac40c39 100644
--- a/include/linux/swap.h
+++ b/include/linux/swap.h
@@ -246,6 +246,7 @@ struct swap_info_struct {
unsigned long nr_clusters_max;/* total clusters in the xswap address space */
unsigned long nr_clusters_mapped; /* currently mapped cluster count */
struct mutex xswap_lock; /* serialize map/unmap operations */
+ struct dentry *xswap_debugfs; /* mapped range and counters */
atomic_long_t xswap_grows; /* grow operations so far */
#endif
struct list_head free_clusters; /* free clusters list */
diff --git a/mm/swapfile.c b/mm/swapfile.c
index 213b66bb1d6d..462efeef887a 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -43,12 +43,14 @@
#include <linux/suspend.h>
#include <linux/zswap.h>
#include <linux/plist.h>
+#include <linux/debugfs.h>
#include <asm/tlbflush.h>
#include <linux/leafops.h>
#include "swap_table.h"
#include "internal.h"
#include "swap.h"
+
#define DEF_SWAP_PRIO -1
/*
@@ -79,6 +81,7 @@
static bool xswap_should_grow(struct swap_info_struct *si);
static unsigned long xswap_grow(struct swap_info_struct *si);
+static void xswap_debugfs_del(struct swap_info_struct *si);
static int xswap_map_clusters(struct swap_info_struct *si,
unsigned long start_idx, unsigned long nr,
bool link_free);
@@ -3455,6 +3458,7 @@ static void free_swap_cluster_info(struct swap_info_struct *si)
}
spin_unlock(&ci->lock);
}
+ xswap_debugfs_del(si);
vfree(si->cluster_info);
si->cluster_vm = NULL;
si->cluster_info = NULL;
@@ -4087,6 +4091,7 @@ static int xswap_map_clusters(struct swap_info_struct *si,
mutex_unlock(&si->xswap_lock);
return 0;
}
+
static bool xswap_should_grow(struct swap_info_struct *si)
{
unsigned long mapped = READ_ONCE(si->nr_clusters_mapped);
@@ -4138,6 +4143,68 @@ static unsigned long xswap_grow(struct swap_info_struct *si)
return nr_new;
}
+static ssize_t xswap_debugfs_read(struct file *file, char __user *buf,
+ size_t count, loff_t *ppos)
+{
+ struct swap_info_struct *si = file->private_data;
+ struct swap_cluster_info *ci;
+ unsigned long mapped, tail = 0;
+ char tmp[192];
+ int len;
+
+ /*
+ * Count the free clusters at the end of the mapped range. The
+ * xswap_lock keeps the range from moving under the scan.
+ */
+ mutex_lock(&si->xswap_lock);
+ mapped = READ_ONCE(si->nr_clusters_mapped);
+ if (si->cluster_info) {
+ while (mapped - tail > 1) {
+ ci = &si->cluster_info[mapped - tail - 1];
+ if (READ_ONCE(ci->count) ||
+ READ_ONCE(ci->flags) != CLUSTER_FLAG_FREE)
+ break;
+ tail++;
+ }
+ }
+ len = scnprintf(tmp, sizeof(tmp),
+ "type %d\nclusters_max %lu\nclusters_mapped %lu\nusage_pages %lu\ntail_free %lu\ngrows %lu\n",
+ si->type, READ_ONCE(si->nr_clusters_max), mapped,
+ swap_usage_in_pages(si), tail,
+ atomic_long_read(&si->xswap_grows));
+ mutex_unlock(&si->xswap_lock);
+
+ return simple_read_from_buffer(buf, count, ppos, tmp, len);
+}
+
+static const struct file_operations xswap_debugfs_fops = {
+ .read = xswap_debugfs_read,
+ .open = simple_open,
+ .llseek = default_llseek,
+};
+
+static struct dentry *xswap_debugfs_root;
+
+static void xswap_debugfs_add(struct swap_info_struct *si)
+{
+ if (!xswap_debugfs_root)
+ xswap_debugfs_root = debugfs_create_dir("xswap", NULL);
+ if (IS_ERR_OR_NULL(xswap_debugfs_root))
+ return;
+
+ /* One device, so one file: /sys/kernel/debug/xswap/xswap.stats. */
+ si->xswap_debugfs = debugfs_create_file("xswap.stats", 0444,
+ xswap_debugfs_root, si,
+ &xswap_debugfs_fops);
+ if (IS_ERR(si->xswap_debugfs))
+ si->xswap_debugfs = NULL;
+}
+
+static void xswap_debugfs_del(struct swap_info_struct *si)
+{
+ debugfs_remove(si->xswap_debugfs);
+ si->xswap_debugfs = NULL;
+}
#endif /* CONFIG_XSWAP */
static int setup_swap_clusters_info(struct swap_info_struct *si,
@@ -4360,6 +4427,7 @@ static int xswap_create(int prio)
*/
si->ops = &swap_bdev_ops;
+ si->xswap_debugfs = NULL;
atomic_long_set(&si->xswap_grows, 0);
error = setup_swap_clusters_info(si, NULL, maxpages);
@@ -4371,6 +4439,7 @@ static int xswap_create(int prio)
goto bad_swap;
mutex_lock(&swapon_mutex);
+ xswap_debugfs_add(si);
si->prio = prio;
si->list.prio = -si->prio;
si->avail_list.prio = -si->prio;
--
2.54.0
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v6 13/14] mm, swap: do not give an xswap slot to a cgroup that cannot zswap
2026-11-22 12:48 ` [PATCH v6 10/14] mm, swap: add CONFIG_XSWAP_DEFAULT_MAX Baoquan He
2026-11-22 12:48 ` [PATCH v6 11/14] mm, swap: add xswap grow trigger on cluster allocation Baoquan He
2026-11-22 12:48 ` [PATCH v6 12/14] mm, swap: add debugfs counters for xswap Baoquan He
@ 2026-11-22 12:48 ` Baoquan He
2026-11-22 12:48 ` [PATCH v6 14/14] mm, swap: skip the overcommit probe when removing xswap Baoquan He
3 siblings, 0 replies; 15+ messages in thread
From: Baoquan He @ 2026-11-22 12:48 UTC (permalink / raw)
To: linux-mm
Cc: akpm, david, urezki, chrisl, kasong, nphamcs, baohua,
youngjun.park, shikemeng, hannes, yosry, chengming.zhou, corbet,
kunwu.chan, klarasmodin, linux-kernel, baoquan.he, Baoquan He
From: Baoquan He <hebaoquan@kylinos.cn>
At this point, an xswap slot holds only a zswap-backed page. It has no
storage of its own. If the folio's cgroup cannot take another zswap
page, giving it an xswap slot is wasted: zswap refuses the page and the
slot cannot hold it.
Skip xswap devices when the cgroup cannot zswap, and try a physical swap
device instead.
The check is obj_cgroup_may_zswap(): zswap_store() uses the same check
but first tries to shrink the memcg, while the allocation-time check does
not, so it is stricter.
Signed-off-by: Baoquan He <hebaoquan@kylinos.cn>
---
mm/swapfile.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++-----
1 file changed, 66 insertions(+), 6 deletions(-)
diff --git a/mm/swapfile.c b/mm/swapfile.c
index 462efeef887a..90f17edce1d0 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -226,6 +226,8 @@ static DEFINE_SPINLOCK(swap_lock);
static unsigned int nr_swapfiles;
atomic_long_t nr_swap_pages;
atomic_t nr_real_swapfiles;
+/* Active xswap devices, as in nr_real_swapfiles. */
+static atomic_t nr_xswap_files;
/*
* Some modules use swappable objects and may try to swap them out under
* memory pressure (via the shrinker). Before doing so, they may wish to
@@ -1452,7 +1454,9 @@ static void del_from_avail_list(struct swap_info_struct *si, bool swapoff)
lockdep_assert_held(&si->lock);
si->flags &= ~SWP_WRITEOK;
/* Count active devices, not merely those on the avail list. */
- if (!(si->flags & SWP_XSWAP))
+ if (si->flags & SWP_XSWAP)
+ atomic_sub(1, &nr_xswap_files);
+ else
atomic_sub(1, &nr_real_swapfiles);
atomic_long_or(SWAP_USAGE_OFFLIST_BIT, &si->inuse_pages);
} else {
@@ -1488,7 +1492,9 @@ static void add_to_avail_list(struct swap_info_struct *si, bool swapon)
lockdep_assert_held(&si->lock);
si->flags |= SWP_WRITEOK;
/* A full device is still active and skips the add below. */
- if (!(si->flags & SWP_XSWAP))
+ if (si->flags & SWP_XSWAP)
+ atomic_add(1, &nr_xswap_files);
+ else
atomic_add(1, &nr_real_swapfiles);
} else {
if (!(READ_ONCE(si->flags) & SWP_WRITEOK))
@@ -1612,7 +1618,7 @@ static bool get_swap_device_info(struct swap_info_struct *si)
* Fast path try to get swap entries with specified order from current
* CPU's swap entry pool (a cluster).
*/
-static bool swap_alloc_fast(struct folio *folio)
+static bool swap_alloc_fast(struct folio *folio, bool may_zswap)
{
unsigned int order = folio_order(folio);
struct swap_cluster_info *ci;
@@ -1628,6 +1634,11 @@ static bool swap_alloc_fast(struct folio *folio)
if (!si || !offset || !get_swap_device_info(si))
return false;
+ if (!may_zswap && (si->flags & SWP_XSWAP)) {
+ put_swap_device(si);
+ return false;
+ }
+
ci = swap_cluster_lock(si, offset);
if (cluster_is_usable(ci, order)) {
if (cluster_is_empty(ci))
@@ -1642,13 +1653,19 @@ static bool swap_alloc_fast(struct folio *folio)
}
/* Rotate the device and switch to a new cluster */
-static void swap_alloc_slow(struct folio *folio)
+static void swap_alloc_slow(struct folio *folio, bool may_zswap)
{
struct swap_info_struct *si, *next;
spin_lock(&swap_avail_lock);
start_over:
plist_for_each_entry_safe(si, next, &swap_avail_head, avail_list) {
+ /*
+ * Do not rotate a skipped device: the walk follows the
+ * rotation and would spin forever.
+ */
+ if (!may_zswap && (si->flags & SWP_XSWAP))
+ continue;
/* Rotate the device and switch to a new cluster */
plist_requeue(&si->avail_list, &swap_avail_head);
spin_unlock(&swap_avail_lock);
@@ -1971,6 +1988,17 @@ static int swap_dup_entries_cluster(struct swap_info_struct *si,
return err;
}
+/* Can the folio's cgroup take another zswap page? */
+static bool folio_may_zswap(struct folio *folio)
+{
+ struct obj_cgroup *objcg = get_obj_cgroup_from_folio(folio);
+ bool may_zswap = !objcg || obj_cgroup_may_zswap(objcg);
+
+ if (objcg)
+ obj_cgroup_put(objcg);
+ return may_zswap;
+}
+
/**
* folio_alloc_swap - allocate swap space for a folio
* @folio: folio we want to move to swap
@@ -1987,10 +2015,21 @@ int folio_alloc_swap(struct folio *folio)
{
unsigned int order = folio_order(folio);
unsigned int size = 1 << order;
+ bool may_zswap = true;
+ bool zswap_checked = false;
VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio);
VM_BUG_ON_FOLIO(!folio_test_uptodate(folio), folio);
+ /*
+ * A folio zswap will not take must not get an xswap slot; without
+ * a device to route away from, skip the lookup.
+ */
+ if (atomic_read(&nr_xswap_files)) {
+ may_zswap = folio_may_zswap(folio);
+ zswap_checked = true;
+ }
+
if (order) {
/*
* Reject large allocation when THP_SWAP is disabled. Check below
@@ -2011,8 +2050,8 @@ int folio_alloc_swap(struct folio *folio)
again:
local_lock(&percpu_swap_cluster.lock);
- if (!swap_alloc_fast(folio))
- swap_alloc_slow(folio);
+ if (!swap_alloc_fast(folio, may_zswap))
+ swap_alloc_slow(folio, may_zswap);
local_unlock(&percpu_swap_cluster.lock);
if (!order && unlikely(!folio_test_swapcache(folio))) {
@@ -2020,6 +2059,20 @@ int folio_alloc_swap(struct folio *folio)
goto again;
}
+ /*
+ * The gate was skipped when there was no device; one may have
+ * appeared since, so ask the slot itself.
+ */
+ if (unlikely(!zswap_checked && folio_test_swapcache(folio) &&
+ (__swap_entry_to_info(folio->swap)->flags & SWP_XSWAP))) {
+ zswap_checked = true;
+ may_zswap = folio_may_zswap(folio);
+ if (!may_zswap) {
+ swap_cache_del_folio(folio);
+ goto again;
+ }
+ }
+
/* Need to call this even if allocation failed, for MEMCG_SWAP_FAIL. */
if (unlikely(mem_cgroup_try_charge_swap(folio))) {
swap_cache_del_folio(folio);
@@ -2037,6 +2090,13 @@ int folio_alloc_swap(struct folio *folio)
if (mem_cgroup_get_folio_swap_margin(folio) <= 0)
return -ENOMEM;
+ /*
+ * Only xswap devices exist and the folio cannot use them; splitting
+ * cannot help.
+ */
+ if (!may_zswap && !atomic_read(&nr_real_swapfiles))
+ return -ENOMEM;
+
return order ? -E2BIG : -ENOMEM;
}
--
2.54.0
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH v6 14/14] mm, swap: skip the overcommit probe when removing xswap
2026-11-22 12:48 ` [PATCH v6 10/14] mm, swap: add CONFIG_XSWAP_DEFAULT_MAX Baoquan He
` (2 preceding siblings ...)
2026-11-22 12:48 ` [PATCH v6 13/14] mm, swap: do not give an xswap slot to a cgroup that cannot zswap Baoquan He
@ 2026-11-22 12:48 ` Baoquan He
3 siblings, 0 replies; 15+ messages in thread
From: Baoquan He @ 2026-11-22 12:48 UTC (permalink / raw)
To: linux-mm
Cc: akpm, david, urezki, chrisl, kasong, nphamcs, baohua,
youngjun.park, shikemeng, hannes, yosry, chengming.zhou, corbet,
kunwu.chan, klarasmodin, linux-kernel, baoquan.he, Baoquan He
From: Baoquan He <hebaoquan@kylinos.cn>
swap_info_remove() probes whether Committed_AS still fits under the
commit limit left after the device's pages are subtracted. An xswap
device's pages are its address space, not backing storage, but they
raised the commit limit at creation. A workload can commit into that
headroom, after which the probe refuses to destroy the device.
Skip the probe for xswap until the overcommit accounting for it is
settled. Real swap devices keep the existing protection.
Signed-off-by: Baoquan He <hebaoquan@kylinos.cn>
---
mm/swapfile.c | 15 +++++++++++----
1 file changed, 11 insertions(+), 4 deletions(-)
diff --git a/mm/swapfile.c b/mm/swapfile.c
index 90f17edce1d0..a65afd35c7d9 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -3644,10 +3644,17 @@ SYSCALL_DEFINE1(swapoff, const char __user *, specialfile)
*/
static int swap_info_remove(struct swap_info_struct *p)
{
- if (!security_vm_enough_memory_mm(current->mm, p->pages))
- vm_unacct_memory(p->pages);
- else
- return -ENOMEM;
+ /*
+ * The device's pages are its address space, not backing storage.
+ * The commit limit it added may have been committed against, so
+ * skip the probe until the limit stops counting that space.
+ */
+ if (!(p->flags & SWP_XSWAP)) {
+ if (!security_vm_enough_memory_mm(current->mm, p->pages))
+ vm_unacct_memory(p->pages);
+ else
+ return -ENOMEM;
+ }
spin_lock(&p->lock);
del_from_avail_list(p, true);
--
2.54.0
^ permalink raw reply related [flat|nested] 15+ messages in thread