* Re: [RFC PATCH v3 0/4] mm/zsmalloc: per-cpu deferred free to accelerate swap entry release
From: Nhat Pham @ 2026-05-09 0:08 UTC (permalink / raw)
To: Wenchao Hao
Cc: Andrew Morton, Barry Song, Chengming Zhou, Jens Axboe,
Johannes Weiner, linux-block, linux-kernel, linux-mm, Minchan Kim,
Sergey Senozhatsky, Yosry Ahmed, Wenchao Hao
In-Reply-To: <20260508060724.3810904-1-haowenchao@xiaomi.com>
On Thu, May 7, 2026 at 11:08 PM Wenchao Hao <haowenchao22@gmail.com> wrote:
>
> Swap freeing can be expensive when unmapping a VMA containing many swap
> entries. This has been reported to significantly delay memory reclamation
> during Android's low-memory killing, especially when multiple processes
> are terminated to free memory, with slot_free() accounting for more than
> 80% of the total cost of freeing swap entries.
>
> Two earlier attempts by Lei and Zhiguo added a new thread in the mm core
> to asynchronously collect and free swap entries [1][2], but the design
> itself is fairly complex.
>
> When anon folios and swap entries are mixed within a process, reclaiming
> anon folios from killed processes helps return memory to the system as
> quickly as possible, so that newly launched applications can satisfy
> their memory demands. It is not ideal for swap freeing to block anon
> folio freeing. On the other hand, swap freeing can still return memory
> to the system, although at a slower rate due to memory compression.
>
> This series introduces a callback-based deferred free framework in
> zsmalloc. Callers (zram, zswap) register push/drain callbacks to
> define what gets buffered and how it gets drained. The entire free
> path including caller-side bookkeeping (slot_free, zswap_entry_free)
> is deferred to a background worker.
>
> Implementation:
> - Each CPU owns a single-page buffer. The hot path writes a value
> via the push callback with preemption disabled (no locks).
> - When the buffer fills, it is swapped with a fresh page from a
> pre-allocated page pool. The full page is queued to a WQ_UNBOUND
> worker for drain.
> - The drain callback performs the actual expensive work (zs_free,
> slot_free, zswap_entry_free, etc.) in batch, off the hot path.
> - If no free page is available, the caller falls back to synchronous
> processing.
>
> The speedup comes from moving expensive swap slot freeing off the
> munmap hot path into a background worker, so that intact anonymous
> folios are released back to the system without blocking. The worker
> drains at a slower rate since compressed objects are small and freeing
> a single handle may not release an entire page until the zspage is
> fully empty.
>
> Performance results (Raspberry Pi 4B, ARM64, 8GB RAM):
>
> Test 1: munmap latency for 256MB swap-filled VMA (zram backend)
>
> mode Base Patched Speedup
> single 61.82ms 8.62ms 7.17x
> multi 2p 94.75ms 54.11ms 1.75x
> multi 3p 154.64ms 104.83ms 1.48x
>
> Test 2: munmap latency for different sizes (zram, single process)
>
> Size Base Patched Speedup
> 64MB 14.11ms 2.18ms 6.47x
> 128MB 29.45ms 4.48ms 6.57x
> 192MB 43.85ms 6.62ms 6.62x
> 256MB 57.01ms 9.08ms 6.28x
> 512MB 115.13ms 55.58ms 2.07x
> 1024MB 229.66ms 153.28ms 1.50x
>
> Test 3: munmap latency for 256MB swap-filled VMA (zswap backend)
>
> mode Base Patched Speedup
> single 152.14ms 51.26ms 2.97x
> multi 2p 186.56ms 105.42ms 1.77x
> multi 3p 205.83ms 153.32ms 1.34x
>
> Test 4: munmap latency for different sizes (zswap, single process)
>
> Size Base Patched Speedup
> 64MB 37.83ms 13.26ms 2.85x
> 128MB 75.11ms 26.73ms 2.81x
> 256MB 150.78ms 52.97ms 2.85x
> 512MB 303.04ms 130.38ms 2.32x
> 1024MB 599.95ms 287.10ms 2.09x
>
Hmmm, why are we batching at the zswap/zsmalloc level like this? I
agree with Yosry that this seems like somewhat of an unnecessary
layering violation. For example, do we observe a lot more performance
wins by doing this instead of just simply:
static void zswap_entry_free(swp_entry_t swp, bool deferred)
{
...
if (!deferred || !zs_deferred_free(entry->pool->zs_pool , entry->handle))
zs_free(entry->pool->zs_pool , entry->handle);
}
(basically what you had in the last version).
One weird effect of doing deferred zswap entry freeing like what you
are proposing here, is that the zswap LRU will be littered with stale
zswap entries. Seems like you removed them from the zswap xarray, but
they're still linked into the zswap LRU? At writeback time, that will
throw off the statistics used in the heuristics, and will make
writeback go through a bunch of stale entries, wasting more cycles :)
Seems a bit inelegant, no?
^ permalink raw reply
* Re: [PATCH v5 2/2] blk-mq: expose tag starvation counts via debugfs
From: kernel test robot @ 2026-05-09 0:12 UTC (permalink / raw)
To: Aaron Tomlin, axboe, rostedt, mhiramat, mathieu.desnoyers
Cc: llvm, oe-kbuild-all, bvanassche, johannes.thumshirn, kch, dlemoal,
ritesh.list, loberman, neelx, sean, mproche, chjohnst,
linux-block, linux-kernel, linux-trace-kernel
In-Reply-To: <20260427020142.358912-3-atomlin@atomlin.com>
Hi Aaron,
kernel test robot noticed the following build errors:
[auto build test ERROR on axboe/for-next]
[also build test ERROR on trace/for-next linus/master v7.1-rc2 next-20260508]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Aaron-Tomlin/blk-mq-add-tracepoint-block_rq_tag_wait/20260428-013259
base: https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux.git for-next
patch link: https://lore.kernel.org/r/20260427020142.358912-3-atomlin%40atomlin.com
patch subject: [PATCH v5 2/2] blk-mq: expose tag starvation counts via debugfs
config: x86_64-rhel-9.4-rust (https://download.01.org/0day-ci/archive/20260509/202605090233.BZVa3x9s-lkp@intel.com/config)
compiler: clang version 20.1.8 (https://github.com/llvm/llvm-project 87f0227cb60147a26a1eeb4fb06e3b505e9c7261)
rustc: rustc 1.88.0 (6b00bc388 2025-06-23)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20260509/202605090233.BZVa3x9s-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202605090233.BZVa3x9s-lkp@intel.com/
All errors (new ones prefixed by >>):
In file included from block/blk-core.c:45:
In file included from include/trace/events/block.h:726:
In file included from include/trace/define_trace.h:132:
In file included from include/trace/trace_events.h:468:
>> include/trace/events/block.h:255:47: error: expected ':'
255 | __entry->dev = q->disk ? disk_devt(q->disk);
| ^
| :
include/trace/stages/stage6_event_callback.h:133:33: note: expanded from macro 'TP_fast_assign'
133 | #define TP_fast_assign(args...) args
| ^
include/trace/trace_events.h:44:16: note: expanded from macro 'TRACE_EVENT'
44 | PARAMS(assign), \
| ^
include/linux/tracepoint.h:160:25: note: expanded from macro 'PARAMS'
160 | #define PARAMS(args...) args
| ^
include/trace/trace_events.h:435:16: note: expanded from macro 'DECLARE_EVENT_CLASS'
435 | PARAMS(assign), PARAMS(print)) \
| ^
include/linux/tracepoint.h:160:25: note: expanded from macro 'PARAMS'
160 | #define PARAMS(args...) args
| ^
include/trace/trace_events.h:427:4: note: expanded from macro '\
__DECLARE_EVENT_CLASS'
427 | { assign; } \
| ^
include/trace/events/block.h:255:27: note: to match this '?'
255 | __entry->dev = q->disk ? disk_devt(q->disk);
| ^
>> include/trace/events/block.h:255:47: error: expected expression
255 | __entry->dev = q->disk ? disk_devt(q->disk);
| ^
In file included from block/blk-core.c:45:
In file included from include/trace/events/block.h:726:
In file included from include/trace/define_trace.h:133:
In file included from include/trace/perf.h:110:
>> include/trace/events/block.h:255:47: error: expected ':'
255 | __entry->dev = q->disk ? disk_devt(q->disk);
| ^
| :
include/trace/stages/stage6_event_callback.h:133:33: note: expanded from macro 'TP_fast_assign'
133 | #define TP_fast_assign(args...) args
| ^
include/trace/trace_events.h:44:16: note: expanded from macro 'TRACE_EVENT'
44 | PARAMS(assign), \
| ^
include/linux/tracepoint.h:160:25: note: expanded from macro 'PARAMS'
160 | #define PARAMS(args...) args
| ^
include/trace/perf.h:67:16: note: expanded from macro 'DECLARE_EVENT_CLASS'
67 | PARAMS(assign), PARAMS(print)) \
| ^
include/linux/tracepoint.h:160:25: note: expanded from macro 'PARAMS'
160 | #define PARAMS(args...) args
| ^
include/trace/perf.h:51:4: note: expanded from macro '\
__DECLARE_EVENT_CLASS'
51 | { assign; } \
| ^
include/trace/events/block.h:255:27: note: to match this '?'
255 | __entry->dev = q->disk ? disk_devt(q->disk);
| ^
>> include/trace/events/block.h:255:47: error: expected expression
255 | __entry->dev = q->disk ? disk_devt(q->disk);
| ^
4 errors generated.
vim +255 include/trace/events/block.h
242
243 TP_PROTO(struct request_queue *q, struct blk_mq_hw_ctx *hctx, bool is_sched_tag),
244
245 TP_ARGS(q, hctx, is_sched_tag),
246
247 TP_STRUCT__entry(
248 __field( dev_t, dev )
249 __field( u32, hctx_id )
250 __field( u32, nr_tags )
251 __field( bool, is_sched_tag )
252 ),
253
254 TP_fast_assign(
> 255 __entry->dev = q->disk ? disk_devt(q->disk);
256 __entry->hctx_id = hctx->queue_num;
257 __entry->is_sched_tag = is_sched_tag;
258
259 if (is_sched_tag)
260 __entry->nr_tags = hctx->sched_tags->nr_tags;
261 else
262 __entry->nr_tags = hctx->tags->nr_tags;
263 ),
264
265 TP_printk("%d,%d hctx=%u starved on %s tags (depth=%u)",
266 MAJOR(__entry->dev), MINOR(__entry->dev),
267 __entry->hctx_id,
268 __entry->is_sched_tag ? "scheduler" : "hardware",
269 __entry->nr_tags)
270 );
271
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply
* Re: [PATCH v3 0/2] drbd: switch from genl_magic to YNL
From: Jakub Kicinski @ 2026-05-09 0:19 UTC (permalink / raw)
To: Christoph Böhmwalder
Cc: Jens Axboe, drbd-dev, linux-kernel, Lars Ellenberg,
Philipp Reisner, linux-block, Donald Hunter, Eric Dumazet, netdev
In-Reply-To: <20260506124541.1951772-1-christoph.boehmwalder@linbit.com>
On Wed, 6 May 2026 14:45:39 +0200 Christoph Böhmwalder wrote:
> DRBD's genetlink interface was defined using a custom multi-include
> macro system, genl_magic_{func,struct}.h. This system generated struct
> definitions, netlink policies, serialization functions and more
> from a single "magic" header.
> It never really caught on; DRBD is its only user, its internal macro
> jungle is next to impossible to understand, and even harder to debug.
>
> This series replaces it with the standard solution, YNL.
> The *_gen.[ch] files were created with a modified YNL generator, but
> these modifications are not shipped because the current DRBD family is
> effectively frozen.
>
> Note: this family primarily aims for compatibility with existing
> userspace. The next planned step is a new (also YNL-based) family,
> "drbd2", which will implement all the actual modern recommendations for
> new netlink families.
FWIW:
Acked-by: Jakub Kicinski <kuba@kernel.org>
^ permalink raw reply
* Re: [RFC PATCH v3 1/4] mm/zsmalloc: introduce deferred free framework with callback ops
From: Nhat Pham @ 2026-05-09 0:29 UTC (permalink / raw)
To: Wenchao Hao
Cc: Andrew Morton, Barry Song, Chengming Zhou, Jens Axboe,
Johannes Weiner, linux-block, linux-kernel, linux-mm, Minchan Kim,
Sergey Senozhatsky, Yosry Ahmed, Wenchao Hao
In-Reply-To: <20260508060724.3810904-2-haowenchao@xiaomi.com>
On Thu, May 7, 2026 at 11:08 PM Wenchao Hao <haowenchao22@gmail.com> wrote:
>
> Add a per-cpu deferred free mechanism to zsmalloc with a callback
> interface that lets callers (zram, zswap) customize push and drain
> behavior.
>
> Each CPU owns a single-page buffer. The hot path (zs_free_deferred)
> writes a value into the current CPU's buffer via the push callback
> with preemption disabled — no locks, no atomics. When the buffer
> fills, it is swapped with a fresh page from a pre-allocated page
> pool and the full page is queued to a WQ_UNBOUND worker for drain.
>
> The drain worker invokes the drain callback which performs the actual
> expensive work (zs_free, slot_free, etc.) in batch, away from the
> original hot path.
>
> Page pool management:
> - Pool is pre-allocated at enable time (ZS_DEFERRED_POOL_SIZE pages)
> - Full buffers are drained and returned to the pool
> - If no free page is available when buffer is full, the push falls
> back to synchronous processing by the caller
>
> Signed-off-by: Wenchao Hao <haowenchao@xiaomi.com>
> ---
> +#define ZS_DEFERRED_POOL_SIZE (256 * 1024 / PAGE_SIZE)
Seems oddly specific? :) And this doesn't quite scale with number of
CPUs, or memory size?
> +
> +struct zs_deferred_percpu {
> + unsigned int count;
> + void *buf;
> +};
> +
> struct zs_pool {
> const char *name;
>
> @@ -217,6 +224,18 @@ struct zs_pool {
> /* protect zspage migration/compaction */
> rwlock_t lock;
> atomic_t compaction_in_progress;
> +
> + /* per-cpu deferred free */
> + const struct zs_deferred_ops *deferred_ops;
> + void *deferred_private;
> + struct zs_deferred_percpu __percpu *deferred;
> + struct work_struct deferred_work;
> + struct workqueue_struct *deferred_wq;
> + struct list_head deferred_pool;
> + unsigned int deferred_pool_count;
> + spinlock_t deferred_pool_lock;
> + struct list_head deferred_drain_list;
> + spinlock_t deferred_drain_lock;
> };
>
> static inline void zpdesc_set_first(struct zpdesc *zpdesc)
> @@ -1416,6 +1435,171 @@ void zs_free(struct zs_pool *pool, unsigned long handle)
> }
> EXPORT_SYMBOL_GPL(zs_free);
>
> +static struct page *deferred_pool_get(struct zs_pool *pool)
> +{
> + struct page *page = NULL;
> +
> + spin_lock(&pool->deferred_pool_lock);
> + if (!list_empty(&pool->deferred_pool)) {
> + page = list_first_entry(&pool->deferred_pool, struct page, lru);
> + list_del(&page->lru);
> + pool->deferred_pool_count--;
> + }
> + spin_unlock(&pool->deferred_pool_lock);
> + return page;
> +}
> +
> +static void deferred_pool_put(struct zfs_pool *pool, struct page *page)
> +{
> + spin_lock(&pool->deferred_pool_lock);
> + list_add_tail(&page->lru, &pool->deferred_pool);
> + pool->deferred_pool_count++;
> + spin_unlock(&pool->deferred_pool_lock);
> +}
> +
> +static void zs_deferred_work_fn(struct work_struct *work)
> +{
> + struct zs_pool *pool = container_of(work, struct zs_pool, deferred_work);
> + struct page *page;
> +
> + while (true) {
> + unsigned int count;
> +
> + spin_lock(&pool->deferred_drain_lock);
> + if (list_empty(&pool->deferred_drain_list)) {
> + spin_unlock(&pool->deferred_drain_lock);
> + break;
> + }
> + page = list_first_entry(&pool->deferred_drain_list,
> + struct page, lru);
> + list_del(&page->lru);
> + count = page_private(page);
> + spin_unlock(&pool->deferred_drain_lock);
> +
> + pool->deferred_ops->drain(pool->deferred_private,
> + page_address(page), count);
> + deferred_pool_put(pool, page);
> + cond_resched();
> + }
> +}
> +
> +bool zs_free_deferred(struct zs_pool *pool, unsigned long value)
> +{
> + struct zs_deferred_percpu *def;
> + struct page *new_page, *full_page;
> + enum zs_push_ret ret;
> +
> + if (!pool->deferred)
> + return false;
> +
> + def = get_cpu_ptr(pool->deferred);
> +
> + ret = pool->deferred_ops->push(def->buf, def->count, value);
> + if (ret == ZS_PUSH_OK) {
> + def->count++;
> + put_cpu_ptr(pool->deferred);
> + return true;
> + }
> +
> + if (ret == ZS_PUSH_FULL_QUEUED)
> + def->count++;
> +
> + new_page = deferred_pool_get(pool);
> + if (new_page) {
> + full_page = virt_to_page(def->buf);
> + set_page_private(full_page, def->count);
> + def->buf = page_address(new_page);
> + def->count = 0;
> +
> + if (ret == ZS_PUSH_FULL) {
> + pool->deferred_ops->push(def->buf, 0, value);
> + def->count = 1;
> + }
> + put_cpu_ptr(pool->deferred);
> +
> + spin_lock(&pool->deferred_drain_lock);
> + list_add_tail(&full_page->lru, &pool->deferred_drain_list);
> + spin_unlock(&pool->deferred_drain_lock);
> + queue_work(pool->deferred_wq, &pool->deferred_work);
> + return true;
> + }
> + put_cpu_ptr(pool->deferred);
> +
> + /* ret==2: value already queued, will be drained eventually */
> + if (ret == 2)
== 2? :)
> + return true;
> +
> + /* ret==1: value not queued, caller must fallback */
> + return false;
> +}
> +EXPORT_SYMBOL_GPL(zs_free_deferred);
^ permalink raw reply
* Re: [PATCH v2] zram: fix use-after-free in zram_writeback_endio
From: Sergey Senozhatsky @ 2026-05-09 2:18 UTC (permalink / raw)
To: Richard Chang
Cc: Minchan Kim, Sergey Senozhatsky, Jens Axboe, Andrew Morton,
bgeffon, liumartin, linux-kernel, linux-block, linux-mm
In-Reply-To: <20260508084933.3730661-1-richardycc@google.com>
On (26/05/08 08:49), Richard Chang wrote:
> A crash was observed in zram_writeback_endio due to a NULL pointer
> dereference in wake_up. The root cause is a race condition between the
> bio completion handler (zram_writeback_endio) and the writeback task.
>
> In zram_writeback_endio, wake_up() is called on &wb_ctl->done_wait after
> releasing wb_ctl->done_lock. This creates a race window where the
> writeback task can see num_inflight become 0, return, and free wb_ctl
> before zram_writeback_endio calls wake_up().
>
> CPU 0 (zram_writeback_endio) CPU 1 (writeback_store)
> ============================ ============================
> zram_writeback_slots
> zram_submit_wb_request
> zram_submit_wb_request
> wait_event(wb_ctl->done_wait)
> spin_lock(&wb_ctl->done_lock);
> list_add(&req->entry, &wb_ctl->done_reqs);
> spin_unlock(&wb_ctl->done_lock);
> wake_up(&wb_ctl->done_wait);
> zram_complete_done_reqs
> spin_lock(&wb_ctl->done_lock);
> list_add(&req->entry, &wb_ctl->done_reqs);
> spin_unlock(&wb_ctl->done_lock);
> while (num_inflight) > 0)
> spin_lock(&wb_ctl->done_lock);
> list_del(&req->entry);
> spin_unlock(&wb_ctl->done_lock);
> // num_inflight becomes 0
> atomic_dec(num_inflight);
>
> // Leave zram_writeback_slots
> // Free wb_ctl
> release_wb_ctl(wb_ctl);
> // UAF crash!
> wake_up(&wb_ctl->done_wait);
>
> This patch fixes this race by using RCU. By protecting wb_ctl with
> rcu_read_lock() in zram_writeback_endio and using kfree_rcu() to free
> it, we ensure that wb_ctl remains valid during the execution of
> zram_writeback_endio.
>
> Fixes: f405066a1f0d ("zram: introduce writeback bio batching")
> Suggested-by: Sergey Senozhatsky <senozhatsky@chromium.org>
> Suggested-by: Minchan Kim <minchan@kernel.org>
> Signed-off-by: Richard Chang <richardycc@google.com>
Acked-by: Sergey Senozhatsky <senozhatsky@chromium.org>
^ permalink raw reply
* [PATCH] block: fix pages array leak in bio_map_user_iov error path
From: Xin Yin @ 2026-05-09 4:01 UTC (permalink / raw)
To: axboe, dhowells; +Cc: linux-block, linux-kernel, Xin Yin, stable
In bio_map_user_iov(), when iov_iter_extract_pages() is called with pages
set to NULL (because nr_vecs > UIO_FASTIOV), want_pages_array() internally
allocates a pages array via kvmalloc_array(). If iov_iter_extract_pages()
subsequently returns bytes <= 0 (e.g., due to pin_user_pages_fast()
failure), the code jumps to out_unmap without freeing the dynamically
allocated pages array, causing a memory leak detectable by kmemleak.
This can be triggered from userspace by issuing an SG_IO v4 ioctl on a
bsg device with a large din_xfer_len and an invalid din_xferp (mapped
PROT_NONE), which causes pin_user_pages_fast() to fail after the pages
array has already been allocated by want_pages_array().
The kmemleak backtrace looks like:
unreferenced object 0xffff... (size 2048):
backtrace (crc 0):
kvmalloc_node+0x...
want_pages_array+0x...
iov_iter_extract_pages+0x...
bio_map_user_iov+0x...
blk_rq_map_user_iov+0x...
blk_rq_map_user+0x...
bsg_transport_sg_io_fn+0x...
Fix this by freeing the dynamically allocated pages array (when it
differs from the on-stack stack_pages) before jumping to the error path.
Fixes: 403b6fb8dac1 ("block: convert bio_map_user_iov to use iov_iter_extract_pages")
Cc: stable@vger.kernel.org # 6.5+
Signed-off-by: Xin Yin <yinxin.x@bytedance.com>
---
block/blk-map.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/block/blk-map.c b/block/blk-map.c
index 0aadbaf7a9ddd..5b9f14caad4f9 100644
--- a/block/blk-map.c
+++ b/block/blk-map.c
@@ -304,6 +304,8 @@ static int bio_map_user_iov(struct request *rq, struct iov_iter *iter,
bytes = iov_iter_extract_pages(iter, &pages, LONG_MAX,
nr_vecs, extraction_flags, &offs);
if (unlikely(bytes <= 0)) {
+ if (pages != stack_pages)
+ kvfree(pages);
ret = bytes ? bytes : -EFAULT;
goto out_unmap;
}
--
2.20.1
^ permalink raw reply related
* Re: [RFC PATCH v3 0/4] mm/zsmalloc: per-cpu deferred free to accelerate swap entry release
From: Wenchao Hao @ 2026-05-09 8:32 UTC (permalink / raw)
To: Yosry Ahmed
Cc: Andrew Morton, Barry Song, Chengming Zhou, Jens Axboe,
Johannes Weiner, linux-block, linux-kernel, linux-mm, Minchan Kim,
Nhat Pham, Sergey Senozhatsky, Wenchao Hao
In-Reply-To: <CAO9r8zO3GrMLDV57aRaiwRvZhwcduihKhc3rBpFGkbZKoq_iSw@mail.gmail.com>
On Sat, May 9, 2026 at 4:13 AM Yosry Ahmed <yosry@kernel.org> wrote:
>
> On Thu, May 7, 2026 at 11:08 PM Wenchao Hao <haowenchao22@gmail.com> wrote:
> >
> > Swap freeing can be expensive when unmapping a VMA containing many swap
> > entries. This has been reported to significantly delay memory reclamation
> > during Android's low-memory killing, especially when multiple processes
> > are terminated to free memory, with slot_free() accounting for more than
> > 80% of the total cost of freeing swap entries.
> >
> > This series introduces a callback-based deferred free framework in
> > zsmalloc. Callers (zram, zswap) register push/drain callbacks to
> > define what gets buffered and how it gets drained. The entire free
> > path including caller-side bookkeeping (slot_free, zswap_entry_free)
> > is deferred to a background worker.
>
> How much of the speedup comes from avoiding the per-class lock,
> free_zspage(), other work in zswap, etc.
This series doesn't avoid the per-class lock. The pool->lock part
has been split out and posted as a separate series, so this series
focuses purely on the defer scheme:
https://lore.kernel.org/linux-mm/20260508061910.3882831-1-haowenchao@xiaomi.com/
>
> I ask because I think the design here is still fairly complex. I don't
> like how zswap and zram are registering callbacks into zsmalloc to do
> their own freeing work, and they fill the buffers on behalf of
> zsmalloc which seems like a layering violation.
The callback design was motivated by code reuse -- deferring only
zs_free() inside zsmalloc gave less speedup, and the machinery
needed to defer caller-side bookkeeping turns out to be the same
on both sides (per-cpu page buffer, drain worker, fallback). So I
folded the common parts into zsmalloc.
I agree it's not clean from a layering standpoint, and I'm happy to
revisit if the reuse isn't worth the cost.
>
> I wonder how much of the speedup we get by just deferring
> free_zspage()?
Below is the perf breakdown, sampled only during munmap() of a
256MB zram-filled VMA on a Raspberry Pi 4B.
Base kernel:
# Samples: 491 of event 'cycles'
# Event count (approx.): 214056923
#
# Children Self Symbol
# ........ ........ ..........................................
99.55% 0.41% [k] __zap_vma_range
97.27% 2.91% [k] swap_put_entries_cluster
94.37% 1.65% [k] __swap_cluster_free_entries
88.99% 8.91% [k] zram_slot_free_notify
79.87% 10.78% [k] slot_free
56.27% 5.99% [k] zs_free
47.61% 4.35% [k] free_zspage
36.85% 4.96% [k] __free_zspage
19.27% 0.21% [k] __folio_put
12.64% 2.91% [k] __free_frozen_pages
9.50% 6.40% [k] kmem_cache_free
8.28% 8.28% [k] _raw_spin_unlock_irqrestore
6.83% 1.85% [k] dec_zone_page_state
5.18% 5.18% [k] _raw_spin_unlock
5.18% 5.18% [k] folio_unlock
4.98% 4.98% [k] mod_zone_state
4.12% 4.12% [k] _raw_spin_lock
3.30% 3.30% [k] __swap_cgroup_id_xchg
Perf of the zsmalloc-only variant (same 256MB zram workload):
My first attempt for this RFC was exactly that -- defer only the
handle free inside zsmalloc, keep zram/zswap caller-side bookkeeping
synchronous. (I would post this version after this thread)
# Samples: 164 of event 'cycles'
# Event count (approx.): 68803872
#
# Children Self Symbol
# ........ ........ ..........................................
99.24% 1.28% [k] __zap_vma_range
94.17% 4.49% [k] swap_put_entries_cluster
87.77% 12.09% [k] __swap_cluster_free_entries
43.62% 24.33% [k] zram_slot_free_notify
21.80% 21.80% [k] slot_free_extract
19.29% 6.42% [k] zs_free_deferred
12.23% 0.64% [k] zs_free <- sync fallback only
8.96% 8.96% [k] __swap_cgroup_id_xchg
4.51% 1.93% [k] __free_frozen_pages
Zsmalloc-internal items drop out or shrink dramatically. zs_free at
0.64% is only the synchronous fallback when the per-cpu page pool is
temporarily empty. zram_slot_free_notify remains high (24.33%)
because slot_free_extract() still runs synchronously on the hot path
-- it's a new helper this variant introduces to do the zram-side
cleanup (slot flag clears, atomic stats updates, handle extraction)
before the handle is queued.
The perf numbers showed that zram also has non-trivial caller-side
bookkeeping cost -- the work in slot_free_extract() in particular.
I tried to reduce that without deferring it (per-cpu stats, swapping
the bit-lock for a different primitive), but the results were
basically a wash and sometimes slightly worse. That's what led to v3
extending the defer to cover the caller side as well, via the
push/drain callbacks.
v3 (this series, defer all zram slot notify):
# Samples: 82 of event 'cycles'
# Event count (approx.): 33089591
#
# Children Self Symbol
# ........ ........ ..........................................
91.46% 1.32% [k] __zap_vma_range
75.77% 8.35% [k] swap_put_entries_cluster
64.71% 10.72% [k] __swap_cluster_free_entries
33.36% 17.43% [k] zram_slot_free_notify
18.03% 18.03% [k] __swap_cgroup_id_xchg
13.31% 11.82% [k] zs_free_deferred
9.10% 9.10% [k] lookup_swap_cgroup_id
4.03% 4.03% [k] zswap_invalidate
3.94% 3.94% [k] swap_pte_batch
Absolute cycles in the unmap window drop from 214M to 33M (~6.5x),
matching the observed munmap latency (57ms -> 9ms). The defer path
moves the following items out of the hot path (base kernel Self%):
_raw_spin_unlock_irqrestore 8.28 class/pool locks
kmem_cache_free 6.40 zspage struct slab free
zs_free 5.99
_raw_spin_unlock 5.18
folio_unlock 5.18
__free_zspage 4.96 zspage teardown
mod_zone_state 4.98 zone stats
free_zspage 4.35
_raw_spin_lock 4.12
__free_frozen_pages 2.91 buddy page release
_raw_spin_trylock 2.48
dec_zone_page_state 1.85
free_frozen_page_commit 1.66
That accounts for ~55% of the base-kernel munmap hot path, all
moved to the drain worker.
Benchmark (zram, single process, avg of 3):
size base v3 zs-only v3/base zs-only/base
64MB 14.38 2.12 4.34 6.8x 3.3x
128MB 29.73 4.26 8.54 7.0x 3.5x
256MB 57.93 8.54 19.90 6.8x 2.9x
512MB 116.77 55.41 47.90 2.1x 2.4x
1024MB 234.43 150.11 105.06 1.6x 2.2x
> That part can be done much more simply by just putting
> the pages on a per-class list and having an async worker or a kthread
> consume them and batch-free them. If the rest of zs_free() is also
> expensive, we can do the deferred freeing on that level although it
> would be more complicated as we need to have a fixed size buffer to
> store them and handle running out of space.
>
I hesitated on per-class because there are ~255 classes, so a
worker walking them would often find single-entry or empty lists,
defeating the batching. Using a per-cpu buffer of handles and
sorting by class inside the drain gets "batched under one
class->lock" without the many-short-lists problem.
Thanks,
Wenchao
> A breakdown of where the slowdown is coming from would be helpful to
> understand what to focus on.
>
> >
> > Implementation:
> > - Each CPU owns a single-page buffer. The hot path writes a value
> > via the push callback with preemption disabled (no locks).
> > - When the buffer fills, it is swapped with a fresh page from a
> > pre-allocated page pool. The full page is queued to a WQ_UNBOUND
> > worker for drain.
> > - The drain callback performs the actual expensive work (zs_free,
> > slot_free, zswap_entry_free, etc.) in batch, off the hot path.
> > - If no free page is available, the caller falls back to synchronous
> > processing.
> >
> > The speedup comes from moving expensive swap slot freeing off the
> > munmap hot path into a background worker, so that intact anonymous
> > folios are released back to the system without blocking. The worker
> > drains at a slower rate since compressed objects are small and freeing
> > a single handle may not release an entire page until the zspage is
> > fully empty.
> >
> > Performance results (Raspberry Pi 4B, ARM64, 8GB RAM):
> >
> > Test 1: munmap latency for 256MB swap-filled VMA (zram backend)
> >
> > mode Base Patched Speedup
> > single 61.82ms 8.62ms 7.17x
> > multi 2p 94.75ms 54.11ms 1.75x
> > multi 3p 154.64ms 104.83ms 1.48x
> >
> > Test 2: munmap latency for different sizes (zram, single process)
> >
> > Size Base Patched Speedup
> > 64MB 14.11ms 2.18ms 6.47x
> > 128MB 29.45ms 4.48ms 6.57x
> > 192MB 43.85ms 6.62ms 6.62x
> > 256MB 57.01ms 9.08ms 6.28x
> > 512MB 115.13ms 55.58ms 2.07x
> > 1024MB 229.66ms 153.28ms 1.50x
> >
> > Test 3: munmap latency for 256MB swap-filled VMA (zswap backend)
> >
> > mode Base Patched Speedup
> > single 152.14ms 51.26ms 2.97x
> > multi 2p 186.56ms 105.42ms 1.77x
> > multi 3p 205.83ms 153.32ms 1.34x
> >
> > Test 4: munmap latency for different sizes (zswap, single process)
> >
> > Size Base Patched Speedup
> > 64MB 37.83ms 13.26ms 2.85x
> > 128MB 75.11ms 26.73ms 2.81x
> > 256MB 150.78ms 52.97ms 2.85x
> > 512MB 303.04ms 130.38ms 2.32x
> > 1024MB 599.95ms 287.10ms 2.09x
> >
> > [1] https://lore.kernel.org/all/20240805153639.1057-1-justinjiang@vivo.com/
> > [2] https://lore.kernel.org/all/20250909065349.574894-1-liulei.rjpt@vivo.com/
> > [3] https://lore.kernel.org/linux-mm/20260412060450.15813-1-baohua@kernel.org/
> >
> > Changes since v2:
> > - Use per-cpu single-page buffers instead of a global list; the hot
> > path only writes into the local CPU's buffer with preemption disabled
> > - Add a page pool for buffer rotation: when the current buffer is full,
> > swap it with a free page from the pool and queue the full page for
> > drain
> > - Introduce push/drain callback ops so that zram and zswap can each
> > define their own element size and drain logic (zram stores u32 slot
> > indices, zswap stores unsigned long handles)
> > - Drop the lock optimization patches it will be submitted separately
> > as part of a dedicated zsmalloc lock contention series
> > - Link to v2: https://lore.kernel.org/all/20260421121616.3298845-1-haowenchao@xiaomi.com/
> >
> > Barry Song (1):
> > zram: use zsmalloc deferred free callback for async slot free
> >
> > Wenchao Hao (3):
> > mm/zsmalloc: introduce deferred free framework with callback ops
> > mm/zswap: use zsmalloc deferred free callback for async invalidate
> > zram: batch clear flags in slot_free with single write
> >
> > drivers/block/zram/zram_drv.c | 44 ++++++-
> > drivers/block/zram/zram_drv.h | 6 +
> > include/linux/zsmalloc.h | 16 +++
> > mm/zsmalloc.c | 208 +++++++++++++++++++++++++++++++++-
> > mm/zswap.c | 38 ++++++-
> > 5 files changed, 306 insertions(+), 6 deletions(-)
> >
> > --
> > 2.34.1
> >
^ permalink raw reply
* Re: [RFC PATCH v3 0/4] mm/zsmalloc: per-cpu deferred free to accelerate swap entry release
From: Wenchao Hao @ 2026-05-09 8:38 UTC (permalink / raw)
To: haowenchao22
Cc: 21cnbao, akpm, axboe, chengming.zhou, hannes, haowenchao,
linux-block, linux-kernel, linux-mm, minchan, nphamcs,
senozhatsky, yosry
In-Reply-To: <CAOptpSPY3YL5VFJW9KKP99Yb17+_rdXKsKj93FdEn3_Zb350ow@mail.gmail.com>
The three patches below implement the zsmalloc-only variant --
deferring just zs_free(). They partially depend on the pool->lock
removal series, and also reduce the number of class->lock
acquire/release pairs on the drain path.
----- [1/3] mm/zsmalloc: introduce per-cpu deferred free with page pool -----
Introduce zs_free_deferred() that enqueues handles into per-cpu
buffers backed by single pages (PAGE_SIZE/8 entries each).
A pre-allocated page pool provides fresh pages for buffer swap on the
hot path without any allocation. When a per-cpu buffer fills up, the
producer swaps in a page from the pool, moves the full page to a drain
list, and resets count — all within preempt_disable, no waiting for the
worker.
The drain worker runs on a WQ_UNBOUND workqueue to avoid preempting
the producer on its CPU. It picks pages off the drain list one at a
time, drains them using consecutive-class batching (holding class->lock
across runs of same-class handles), and returns drained pages to the
pool. It processes at most pool_size/2 pages per invocation to avoid
monopolizing CPU, rescheduling itself if more pages remain.
Extract __zs_free_handle() from zs_free() as the locked free primitive
shared by both synchronous and deferred paths. Empty zspages are
collected on a list and released after dropping class->lock.
Also introduce zs_free_deferred_flush() for use before zs_compact()
and zs_deferred_free_all() for pool teardown.
Signed-off-by: Wenchao Hao <haowenchao@xiaomi.com>
---
include/linux/zsmalloc.h | 2 +
mm/zsmalloc.c | 342 +++++++++++++++++++++++++++++++++++----
2 files changed, 316 insertions(+), 28 deletions(-)
diff --git a/include/linux/zsmalloc.h b/include/linux/zsmalloc.h
index 478410c880b1..1e5ac1a39d41 100644
--- a/include/linux/zsmalloc.h
+++ b/include/linux/zsmalloc.h
@@ -30,6 +30,8 @@ void zs_destroy_pool(struct zs_pool *pool);
unsigned long zs_malloc(struct zs_pool *pool, size_t size, gfp_t flags,
const int nid);
void zs_free(struct zs_pool *pool, unsigned long obj);
+void zs_free_deferred(struct zs_pool *pool, unsigned long handle);
+void zs_free_deferred_flush(struct zs_pool *pool);
size_t zs_huge_class_size(struct zs_pool *pool);
diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 176d3ad4f6e9..f483937cf34f 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -42,6 +42,7 @@
#include <linux/zsmalloc.h>
#include <linux/fs.h>
#include <linux/workqueue.h>
+#include <linux/percpu.h>
#include "zpdesc.h"
#define ZSPAGE_MAGIC 0x58
@@ -56,6 +57,9 @@
#define ZS_HANDLE_SIZE (sizeof(unsigned long))
+#define ZS_DEFERRED_BUF_ENTRIES (PAGE_SIZE / sizeof(unsigned long))
+#define ZS_DEFERRED_POOL_SIZE (256 * 1024 / PAGE_SIZE)
+
/*
* Object location (<PFN>, <obj_idx>) is encoded as
* a single (unsigned long) handle value.
@@ -174,6 +178,7 @@ static_assert(_PFN_BITS + OBJ_CLASS_BITS_NEEDED + OBJ_IDX_BITS_NEEDED
#define ZS_SIZE_CLASSES (DIV_ROUND_UP(ZS_MAX_ALLOC_SIZE - ZS_MIN_ALLOC_SIZE, \
ZS_SIZE_CLASS_DELTA) + 1)
+
/*
* Pages are distinguished by the ratio of used memory (that is the ratio
* of ->inuse objects to all objects that page can store). For example,
@@ -246,6 +251,11 @@ struct link_free {
};
};
+struct zs_deferred_percpu {
+ unsigned int count;
+ unsigned long *handles;
+};
+
static struct kmem_cache *handle_cachep;
static struct kmem_cache *zspage_cachep;
@@ -270,6 +280,20 @@ struct zs_pool {
/* protect zspage migration/compaction */
rwlock_t lock;
atomic_t compaction_in_progress;
+
+ /* per-cpu deferred free */
+ struct zs_deferred_percpu __percpu *deferred;
+ struct work_struct deferred_drain_work;
+ struct workqueue_struct *drain_wq;
+
+ /* page pool: free pages available for buffer swap */
+ struct list_head page_pool;
+ unsigned int page_pool_count;
+ spinlock_t page_pool_lock;
+
+ /* drain list: full pages waiting to be drained */
+ struct list_head drain_list;
+ spinlock_t drain_list_lock;
};
static inline void zpdesc_set_first(struct zpdesc *zpdesc)
@@ -788,12 +812,6 @@ static unsigned int obj_to_class_idx(unsigned long obj)
return (obj >> OBJ_IDX_BITS) & OBJ_CLASS_MASK;
}
-/**
- * location_to_obj - encode (<zpdesc>, <obj_idx>, <class_idx>) into obj value
- * @zpdesc: zpdesc object resides in zspage
- * @obj_idx: object index
- * @class_idx: size class index
- */
static unsigned long location_to_obj(struct zpdesc *zpdesc, unsigned int obj_idx,
unsigned int class_idx)
{
@@ -1454,23 +1472,14 @@ static void obj_free(int class_size, unsigned long obj)
mod_zspage_inuse(zspage, -1);
}
-void zs_free(struct zs_pool *pool, unsigned long handle)
+static void __zs_free_handle(struct zs_pool *pool, struct size_class *class,
+ unsigned long handle, struct list_head *free_list)
{
- struct zspage *zspage;
- struct zspage *zspage_to_free = NULL;
struct zpdesc *f_zpdesc;
+ struct zspage *zspage;
unsigned long obj;
- struct size_class *class;
int fullness;
- if (IS_ERR_OR_NULL((void *)handle))
- return;
-
- obj = handle_to_obj(handle);
- class = pool->size_class[obj_to_class_idx(obj)];
-
- spin_lock(&class->lock);
-
obj = handle_to_obj(handle);
obj_to_zpdesc(obj, &f_zpdesc);
zspage = get_zspage(f_zpdesc);
@@ -1480,31 +1489,231 @@ void zs_free(struct zs_pool *pool, unsigned long handle)
fullness = fix_fullness_group(class, zspage);
if (fullness == ZS_INUSE_RATIO_0) {
- /*
- * Perform bookkeeping under class->lock, but defer the
- * actual page release (which may contend on zone->lock)
- * until after dropping class->lock.
- */
if (trylock_zspage(zspage)) {
remove_zspage(class, zspage);
class_stat_sub(class, ZS_OBJS_ALLOCATED,
class->objs_per_zspage);
atomic_long_sub(class->pages_per_zspage,
&pool->pages_allocated);
- zspage_to_free = zspage;
+ list_add(&zspage->list, free_list);
} else {
kick_deferred_free(pool);
}
}
+}
+static void free_zspage_list(struct zs_pool *pool, struct list_head *list)
+{
+ struct zspage *zspage, *tmp;
+
+ list_for_each_entry_safe(zspage, tmp, list, list) {
+ list_del(&zspage->list);
+ free_zspage_pages(pool, zspage);
+ }
+}
+
+void zs_free(struct zs_pool *pool, unsigned long handle)
+{
+ struct size_class *class;
+ unsigned long obj;
+ LIST_HEAD(free_list);
+
+ if (IS_ERR_OR_NULL((void *)handle))
+ return;
+
+ obj = handle_to_obj(handle);
+ class = pool->size_class[obj_to_class_idx(obj)];
+ spin_lock(&class->lock);
+
+ __zs_free_handle(pool, class, handle, &free_list);
spin_unlock(&class->lock);
- if (zspage_to_free)
- free_zspage_pages(pool, zspage_to_free);
+ free_zspage_list(pool, &free_list);
cache_free_handle(handle);
}
EXPORT_SYMBOL_GPL(zs_free);
+static void zs_deferred_drain_batch(struct zs_pool *pool,
+ unsigned long *handles, unsigned int count)
+{
+ struct size_class *class = NULL;
+ unsigned int cur_cls = UINT_MAX;
+ LIST_HEAD(free_list);
+ unsigned int i;
+
+ for (i = 0; i < count; i++) {
+ unsigned long obj = handle_to_obj(handles[i]);
+ unsigned int cls = obj_to_class_idx(obj);
+
+ if (cls != cur_cls) {
+ if (class) {
+ spin_unlock(&class->lock);
+ free_zspage_list(pool, &free_list);
+ cond_resched();
+ }
+ cur_cls = cls;
+ class = pool->size_class[cls];
+ spin_lock(&class->lock);
+ }
+ __zs_free_handle(pool, class, handles[i], &free_list);
+ }
+
+ if (class) {
+ spin_unlock(&class->lock);
+ free_zspage_list(pool, &free_list);
+ }
+
+ for (i = 0; i < count; i++)
+ cache_free_handle(handles[i]);
+}
+
+static struct page *deferred_pool_get(struct zs_pool *pool)
+{
+ struct page *page = NULL;
+
+ spin_lock(&pool->page_pool_lock);
+ if (!list_empty(&pool->page_pool)) {
+ page = list_first_entry(&pool->page_pool, struct page, lru);
+ list_del(&page->lru);
+ pool->page_pool_count--;
+ }
+ spin_unlock(&pool->page_pool_lock);
+ return page;
+}
+
+static void deferred_pool_put(struct zs_pool *pool, struct page *page)
+{
+ spin_lock(&pool->page_pool_lock);
+ list_add_tail(&page->lru, &pool->page_pool);
+ pool->page_pool_count++;
+ spin_unlock(&pool->page_pool_lock);
+}
+
+static void deferred_drain_enqueue(struct zs_pool *pool, struct page *page)
+{
+ spin_lock(&pool->drain_list_lock);
+ list_add_tail(&page->lru, &pool->drain_list);
+ spin_unlock(&pool->drain_list_lock);
+}
+
+static struct page *deferred_drain_dequeue(struct zs_pool *pool)
+{
+ struct page *page = NULL;
+
+ spin_lock(&pool->drain_list_lock);
+ if (!list_empty(&pool->drain_list)) {
+ page = list_first_entry(&pool->drain_list, struct page, lru);
+ list_del(&page->lru);
+ }
+ spin_unlock(&pool->drain_list_lock);
+ return page;
+}
+
+static void zs_deferred_drain_work(struct work_struct *work)
+{
+ struct zs_pool *pool = container_of(work, struct zs_pool,
+ deferred_drain_work);
+ struct page *page;
+ unsigned int drained = 0;
+ unsigned int max_drain = ZS_DEFERRED_POOL_SIZE / 2;
+
+ while (drained < max_drain) {
+ page = deferred_drain_dequeue(pool);
+ if (!page)
+ break;
+
+ zs_deferred_drain_batch(pool, page_address(page),
+ ZS_DEFERRED_BUF_ENTRIES);
+ deferred_pool_put(pool, page);
+ drained++;
+ cond_resched();
+ }
+
+ /* If drain list still has pages, reschedule */
+ spin_lock(&pool->drain_list_lock);
+ if (!list_empty(&pool->drain_list))
+ queue_work(pool->drain_wq, &pool->deferred_drain_work);
+ spin_unlock(&pool->drain_list_lock);
+}
+
+void zs_free_deferred(struct zs_pool *pool, unsigned long handle)
+{
+ struct zs_deferred_percpu *def;
+ struct page *new_page, *full_page;
+ bool queued = false;
+
+ if (IS_ERR_OR_NULL((void *)handle))
+ return;
+
+ def = get_cpu_ptr(pool->deferred);
+
+ if (likely(def->count < ZS_DEFERRED_BUF_ENTRIES)) {
+ def->handles[def->count++] = handle;
+ queued = true;
+ if (def->count < ZS_DEFERRED_BUF_ENTRIES) {
+ put_cpu_ptr(pool->deferred);
+ return;
+ }
+ }
+
+ /* Buffer is full, try to swap in a fresh page */
+ new_page = deferred_pool_get(pool);
+ if (new_page) {
+ full_page = virt_to_page(def->handles);
+ def->handles = page_address(new_page);
+ def->count = 0;
+ if (!queued)
+ def->handles[def->count++] = handle;
+ put_cpu_ptr(pool->deferred);
+ deferred_drain_enqueue(pool, full_page);
+ queue_work(pool->drain_wq, &pool->deferred_drain_work);
+ return;
+ }
+ put_cpu_ptr(pool->deferred);
+
+ if (!queued)
+ zs_free(pool, handle);
+}
+EXPORT_SYMBOL_GPL(zs_free_deferred);
+
+/*
+ * Called only from zs_destroy_pool() when no producers are running.
+ * Drains all per-cpu buffers regardless of whether they are full.
+ */
+static void zs_deferred_free_all(struct zs_pool *pool)
+{
+ struct page *page;
+ int cpu;
+
+ flush_work(&pool->deferred_drain_work);
+
+ /* Drain remaining pages on drain list */
+ while ((page = deferred_drain_dequeue(pool)) != NULL) {
+ zs_deferred_drain_batch(pool, page_address(page),
+ ZS_DEFERRED_BUF_ENTRIES);
+ deferred_pool_put(pool, page);
+ }
+
+ /* Drain partially-filled per-cpu buffers */
+ for_each_possible_cpu(cpu) {
+ struct zs_deferred_percpu *def;
+ unsigned int count;
+
+ def = per_cpu_ptr(pool->deferred, cpu);
+ count = def->count;
+ if (!count)
+ continue;
+ zs_deferred_drain_batch(pool, def->handles, count);
+ def->count = 0;
+ }
+}
+
+void zs_free_deferred_flush(struct zs_pool *pool)
+{
+ flush_work(&pool->deferred_drain_work);
+}
+EXPORT_SYMBOL_GPL(zs_free_deferred_flush);
+
static void zs_object_copy(struct size_class *class, unsigned long dst,
unsigned long src)
{
@@ -2053,6 +2262,8 @@ unsigned long zs_compact(struct zs_pool *pool)
if (atomic_xchg(&pool->compaction_in_progress, 1))
return 0;
+ zs_free_deferred_flush(pool);
+
for (i = ZS_SIZE_CLASSES - 1; i >= 0; i--) {
class = pool->size_class[i];
if (class->index != i)
@@ -2161,9 +2372,11 @@ static int calculate_zspage_chain_size(int class_size)
*/
struct zs_pool *zs_create_pool(const char *name)
{
- int i;
+ int i, cpu;
+ unsigned int pg_idx;
struct zs_pool *pool;
struct size_class *prev_class = NULL;
+ struct page *page, *tmp;
pool = kzalloc_obj(*pool);
if (!pool)
@@ -2172,11 +2385,67 @@ struct zs_pool *zs_create_pool(const char *name)
init_deferred_free(pool);
rwlock_init(&pool->lock);
atomic_set(&pool->compaction_in_progress, 0);
+ INIT_WORK(&pool->deferred_drain_work, zs_deferred_drain_work);
+
+ pool->drain_wq = alloc_workqueue("zs_drain", WQ_UNBOUND, 0);
+ if (!pool->drain_wq) {
+ kfree(pool);
+ return NULL;
+ }
+
+ /* Initialize page pool and drain list */
+ INIT_LIST_HEAD(&pool->page_pool);
+ spin_lock_init(&pool->page_pool_lock);
+ pool->page_pool_count = 0;
+ INIT_LIST_HEAD(&pool->drain_list);
+ spin_lock_init(&pool->drain_list_lock);
+
+ for (pg_idx = 0; pg_idx < ZS_DEFERRED_POOL_SIZE; pg_idx++) {
+ page = alloc_page(GFP_KERNEL);
+ if (!page)
+ goto err_pool_pages;
+ list_add_tail(&page->lru, &pool->page_pool);
+ pool->page_pool_count++;
+ }
+
+ pool->deferred = alloc_percpu(struct zs_deferred_percpu);
+ if (!pool->deferred)
+ goto err_pool_pages;
+ for_each_possible_cpu(cpu) {
+ struct zs_deferred_percpu *def = per_cpu_ptr(pool->deferred, cpu);
+
+ page = deferred_pool_get(pool);
+ if (!page) {
+ for_each_possible_cpu(cpu) {
+ def = per_cpu_ptr(pool->deferred, cpu);
+ if (def->handles)
+ deferred_pool_put(pool,
+ virt_to_page(def->handles));
+ }
+ free_percpu(pool->deferred);
+ goto err_pool_pages;
+ }
+ def->handles = page_address(page);
+ def->count = 0;
+ }
pool->name = kstrdup(name, GFP_KERNEL);
if (!pool->name)
goto err;
+ goto pool_init_done;
+
+err_pool_pages:
+ list_for_each_entry_safe(page, tmp, &pool->page_pool, lru) {
+ list_del(&page->lru);
+ __free_page(page);
+ }
+ destroy_workqueue(pool->drain_wq);
+ kfree(pool);
+ return NULL;
+
+pool_init_done:
+
/*
* Iterate reversely, because, size of size_class that we want to use
* for merging should be larger or equal to current size.
@@ -2272,9 +2541,11 @@ EXPORT_SYMBOL_GPL(zs_create_pool);
void zs_destroy_pool(struct zs_pool *pool)
{
- int i;
+ int i, cpu;
+ struct page *page, *tmp;
zs_unregister_shrinker(pool);
+ zs_deferred_free_all(pool);
zs_flush_migration(pool);
zs_pool_stat_destroy(pool);
@@ -2298,6 +2569,21 @@ void zs_destroy_pool(struct zs_pool *pool)
kfree(class);
}
+ /* Return per-cpu buffers to page pool */
+ for_each_possible_cpu(cpu) {
+ struct zs_deferred_percpu *def = per_cpu_ptr(pool->deferred, cpu);
+
+ if (def->handles)
+ deferred_pool_put(pool, virt_to_page(def->handles));
+ }
+
+ /* Free all pages in page pool */
+ list_for_each_entry_safe(page, tmp, &pool->page_pool, lru) {
+ list_del(&page->lru);
+ __free_page(page);
+ }
+ free_percpu(pool->deferred);
+ destroy_workqueue(pool->drain_wq);
kfree(pool->name);
kfree(pool);
}
----- [2/3] mm/zswap: use zs_free_deferred() in entry free path -----
Replace zs_free() with zs_free_deferred() in zswap_entry_free() to
avoid the overhead of zsmalloc class->lock and potential zone->lock
contention in the zswap invalidation/reclaim hot path.
The store failure path still uses zs_free() directly since it is not
performance critical.
Signed-off-by: Wenchao Hao <haowenchao@xiaomi.com>
---
mm/zswap.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/mm/zswap.c b/mm/zswap.c
index 4b5149173b0e..f2a38c07579f 100644
--- a/mm/zswap.c
+++ b/mm/zswap.c
@@ -765,7 +765,7 @@ static void zswap_entry_cache_free(struct zswap_entry *entry)
static void zswap_entry_free(struct zswap_entry *entry)
{
zswap_lru_del(&zswap_list_lru, entry);
- zs_free(entry->pool->zs_pool, entry->handle);
+ zs_free_deferred(entry->pool->zs_pool, entry->handle);
zswap_pool_put(entry->pool);
if (entry->objcg) {
obj_cgroup_uncharge_zswap(entry->objcg, entry->length);
----- [3/3] zram: defer zs_free() in swap slot free notification path -----
zram_slot_free_notify() is called on the process exit path when
unmapping swap entries. The zs_free() it invokes accounts for ~87%
of slot_free() cost due to zsmalloc locking, blocking memory release
during Android low-memory killing.
Split slot_free() into slot_free_extract() and the actual zs_free():
slot_free_extract() handles slot metadata cleanup (flags, stats,
handle/size zeroing) and returns the zsmalloc handle.
The returned handle is passed to zs_free_deferred() in the
notification path, deferring the expensive zs_free() to a
workqueue so the exit path can release anon folios faster.
All other slot_free() callers (write, discard, meta_free) continue
to use synchronous zs_free() through the unchanged slot_free().
Signed-off-by: Wenchao Hao <haowenchao@xiaomi.com>
---
drivers/block/zram/zram_drv.c | 41 ++++++++++++++++++++---------------
1 file changed, 23 insertions(+), 18 deletions(-)
diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c
index aebc710f0d6a..c67a7442d283 100644
--- a/drivers/block/zram/zram_drv.c
+++ b/drivers/block/zram/zram_drv.c
@@ -2000,24 +2000,26 @@ static bool zram_meta_alloc(struct zram *zram, u64 disksize)
return true;
}
-static void slot_free(struct zram *zram, u32 index)
+/*
+ * Clear slot metadata and extract the zsmalloc handle that needs freeing.
+ * Returns the handle, or 0 if no zsmalloc free is required (e.g. same-filled
+ * or writeback slots).
+ */
+#define ZRAM_SLOT_CLEAR_MASK \
+ (BIT(ZRAM_IDLE) | BIT(ZRAM_INCOMPRESSIBLE) | BIT(ZRAM_PP_SLOT) | \
+ (ZRAM_COMP_PRIORITY_MASK << ZRAM_COMP_PRIORITY_BIT1))
+
+static unsigned long slot_free_extract(struct zram *zram, u32 index)
{
- unsigned long handle;
+ unsigned long handle = 0;
#ifdef CONFIG_ZRAM_TRACK_ENTRY_ACTIME
zram->table[index].attr.ac_time = 0;
#endif
- clear_slot_flag(zram, index, ZRAM_IDLE);
- clear_slot_flag(zram, index, ZRAM_INCOMPRESSIBLE);
- clear_slot_flag(zram, index, ZRAM_PP_SLOT);
- set_slot_comp_priority(zram, index, 0);
+ zram->table[index].attr.flags &= ~ZRAM_SLOT_CLEAR_MASK;
if (test_slot_flag(zram, index, ZRAM_HUGE)) {
- /*
- * Writeback completion decrements ->huge_pages but keeps
- * ZRAM_HUGE flag for deferred decompression path.
- */
if (!test_slot_flag(zram, index, ZRAM_WB))
atomic64_dec(&zram->stats.huge_pages);
clear_slot_flag(zram, index, ZRAM_HUGE);
@@ -2029,10 +2031,6 @@ static void slot_free(struct zram *zram, u32 index)
goto out;
}
- /*
- * No memory is allocated for same element filled pages.
- * Simply clear same page flag.
- */
if (test_slot_flag(zram, index, ZRAM_SAME)) {
clear_slot_flag(zram, index, ZRAM_SAME);
atomic64_dec(&zram->stats.same_pages);
@@ -2041,9 +2039,7 @@ static void slot_free(struct zram *zram, u32 index)
handle = get_slot_handle(zram, index);
if (!handle)
- return;
-
- zs_free(zram->mem_pool, handle);
+ return 0;
atomic64_sub(get_slot_size(zram, index),
&zram->stats.compr_data_size);
@@ -2051,6 +2047,15 @@ static void slot_free(struct zram *zram, u32 index)
atomic64_dec(&zram->stats.pages_stored);
set_slot_handle(zram, index, 0);
set_slot_size(zram, index, 0);
+
+ return handle;
+}
+
+static void slot_free(struct zram *zram, u32 index)
+{
+ unsigned long handle = slot_free_extract(zram, index);
+
+ zs_free(zram->mem_pool, handle);
}
static int read_same_filled_page(struct zram *zram, struct page *page,
@@ -2797,7 +2802,7 @@ static void zram_slot_free_notify(struct block_device *bdev,
return;
}
- slot_free(zram, index);
+ zs_free_deferred(zram->mem_pool, slot_free_extract(zram, index));
slot_unlock(zram, index);
}
^ permalink raw reply related
* Re: [RFC PATCH v3 0/4] mm/zsmalloc: per-cpu deferred free to accelerate swap entry release
From: Wenchao Hao @ 2026-05-09 8:45 UTC (permalink / raw)
To: Nhat Pham
Cc: Andrew Morton, Barry Song, Chengming Zhou, Jens Axboe,
Johannes Weiner, linux-block, linux-kernel, linux-mm, Minchan Kim,
Sergey Senozhatsky, Yosry Ahmed, Wenchao Hao
In-Reply-To: <CAKEwX=Ood7Q4zv0zc+GCCGtWu2GLVdtod=4VC9151r87AZXpWA@mail.gmail.com>
On Sat, May 9, 2026 at 8:08 AM Nhat Pham <nphamcs@gmail.com> wrote:
>
> On Thu, May 7, 2026 at 11:08 PM Wenchao Hao <haowenchao22@gmail.com> wrote:
> >
> > Swap freeing can be expensive when unmapping a VMA containing many swap
> > entries. This has been reported to significantly delay memory reclamation
> > during Android's low-memory killing, especially when multiple processes
> > are terminated to free memory, with slot_free() accounting for more than
> > 80% of the total cost of freeing swap entries.
> >
> > Two earlier attempts by Lei and Zhiguo added a new thread in the mm core
> > to asynchronously collect and free swap entries [1][2], but the design
> > itself is fairly complex.
> >
> > When anon folios and swap entries are mixed within a process, reclaiming
> > anon folios from killed processes helps return memory to the system as
> > quickly as possible, so that newly launched applications can satisfy
> > their memory demands. It is not ideal for swap freeing to block anon
> > folio freeing. On the other hand, swap freeing can still return memory
> > to the system, although at a slower rate due to memory compression.
> >
> > This series introduces a callback-based deferred free framework in
> > zsmalloc. Callers (zram, zswap) register push/drain callbacks to
> > define what gets buffered and how it gets drained. The entire free
> > path including caller-side bookkeeping (slot_free, zswap_entry_free)
> > is deferred to a background worker.
> >
> > Implementation:
> > - Each CPU owns a single-page buffer. The hot path writes a value
> > via the push callback with preemption disabled (no locks).
> > - When the buffer fills, it is swapped with a fresh page from a
> > pre-allocated page pool. The full page is queued to a WQ_UNBOUND
> > worker for drain.
> > - The drain callback performs the actual expensive work (zs_free,
> > slot_free, zswap_entry_free, etc.) in batch, off the hot path.
> > - If no free page is available, the caller falls back to synchronous
> > processing.
> >
> > The speedup comes from moving expensive swap slot freeing off the
> > munmap hot path into a background worker, so that intact anonymous
> > folios are released back to the system without blocking. The worker
> > drains at a slower rate since compressed objects are small and freeing
> > a single handle may not release an entire page until the zspage is
> > fully empty.
> >
> > Performance results (Raspberry Pi 4B, ARM64, 8GB RAM):
> >
> > Test 1: munmap latency for 256MB swap-filled VMA (zram backend)
> >
> > mode Base Patched Speedup
> > single 61.82ms 8.62ms 7.17x
> > multi 2p 94.75ms 54.11ms 1.75x
> > multi 3p 154.64ms 104.83ms 1.48x
> >
> > Test 2: munmap latency for different sizes (zram, single process)
> >
> > Size Base Patched Speedup
> > 64MB 14.11ms 2.18ms 6.47x
> > 128MB 29.45ms 4.48ms 6.57x
> > 192MB 43.85ms 6.62ms 6.62x
> > 256MB 57.01ms 9.08ms 6.28x
> > 512MB 115.13ms 55.58ms 2.07x
> > 1024MB 229.66ms 153.28ms 1.50x
> >
> > Test 3: munmap latency for 256MB swap-filled VMA (zswap backend)
> >
> > mode Base Patched Speedup
> > single 152.14ms 51.26ms 2.97x
> > multi 2p 186.56ms 105.42ms 1.77x
> > multi 3p 205.83ms 153.32ms 1.34x
> >
> > Test 4: munmap latency for different sizes (zswap, single process)
> >
> > Size Base Patched Speedup
> > 64MB 37.83ms 13.26ms 2.85x
> > 128MB 75.11ms 26.73ms 2.81x
> > 256MB 150.78ms 52.97ms 2.85x
> > 512MB 303.04ms 130.38ms 2.32x
> > 1024MB 599.95ms 287.10ms 2.09x
> >
>
> Hmmm, why are we batching at the zswap/zsmalloc level like this? I
> agree with Yosry that this seems like somewhat of an unnecessary
> layering violation. For example, do we observe a lot more performance
> wins by doing this instead of just simply:
>
Thanks for the reply, refer following thread for the perf breakdown
and detail data:
https://lore.kernel.org/linux-mm/CAOptpSPY3YL5VFJW9KKP99Yb17+_rdXKsKj93FdEn3_Zb350ow@mail.gmail.com/
> static void zswap_entry_free(swp_entry_t swp, bool deferred)
> {
> ...
> if (!deferred || !zs_deferred_free(entry->pool->zs_pool , entry->handle))
> zs_free(entry->pool->zs_pool , entry->handle);
> }
>
> (basically what you had in the last version).
>
> One weird effect of doing deferred zswap entry freeing like what you
> are proposing here, is that the zswap LRU will be littered with stale
> zswap entries. Seems like you removed them from the zswap xarray, but
> they're still linked into the zswap LRU? At writeback time, that will
> throw off the statistics used in the heuristics, and will make
> writeback go through a bunch of stale entries, wasting more cycles :)
> Seems a bit inelegant, no?
You're right, that was an oversight -- thanks for pointing it. The
zsmalloc-only variant avoids this entirely: zswap_lru_del() stays
synchronous before the handle is queued, so the LRU never contains
torn-down entries. I'll make sure v4 doesn't have this issue
regardless of which direction we go.
^ permalink raw reply
* Re: [RFC PATCH v3 1/4] mm/zsmalloc: introduce deferred free framework with callback ops
From: Wenchao Hao @ 2026-05-09 8:47 UTC (permalink / raw)
To: Nhat Pham
Cc: Andrew Morton, Barry Song, Chengming Zhou, Jens Axboe,
Johannes Weiner, linux-block, linux-kernel, linux-mm, Minchan Kim,
Sergey Senozhatsky, Yosry Ahmed, Wenchao Hao
In-Reply-To: <CAKEwX=MzLKTF2uQRaJU8UK5B9UeW-Sh-LE_8fdmo7y-MPPbwyg@mail.gmail.com>
On Sat, May 9, 2026 at 8:29 AM Nhat Pham <nphamcs@gmail.com> wrote:
>
> On Thu, May 7, 2026 at 11:08 PM Wenchao Hao <haowenchao22@gmail.com> wrote:
> >
> > Add a per-cpu deferred free mechanism to zsmalloc with a callback
> > interface that lets callers (zram, zswap) customize push and drain
> > behavior.
> >
> > Each CPU owns a single-page buffer. The hot path (zs_free_deferred)
> > writes a value into the current CPU's buffer via the push callback
> > with preemption disabled — no locks, no atomics. When the buffer
> > fills, it is swapped with a fresh page from a pre-allocated page
> > pool and the full page is queued to a WQ_UNBOUND worker for drain.
> >
> > The drain worker invokes the drain callback which performs the actual
> > expensive work (zs_free, slot_free, etc.) in batch, away from the
> > original hot path.
> >
> > Page pool management:
> > - Pool is pre-allocated at enable time (ZS_DEFERRED_POOL_SIZE pages)
> > - Full buffers are drained and returned to the pool
> > - If no free page is available when buffer is full, the push falls
> > back to synchronous processing by the caller
> >
> > Signed-off-by: Wenchao Hao <haowenchao@xiaomi.com>
> > ---
> > +#define ZS_DEFERRED_POOL_SIZE (256 * 1024 / PAGE_SIZE)
>
> Seems oddly specific? :) And this doesn't quite scale with number of
> CPUs, or memory size?
>
256K holds the deferred metadata for ~128MB zswap or ~256MB zram
entries, which matches what a killed process typically has swapped
out. Pages sitting in the pool are memory that can't be used
elsewhere, so I didn't want it to grow with RAM/CPU. Happy to
parameterize it if you'd prefer.
> > +
> > +struct zs_deferred_percpu {
> > + unsigned int count;
> > + void *buf;
> > +};
> > +
> > struct zs_pool {
> > const char *name;
> >
> > @@ -217,6 +224,18 @@ struct zs_pool {
> > /* protect zspage migration/compaction */
> > rwlock_t lock;
> > atomic_t compaction_in_progress;
> > +
> > + /* per-cpu deferred free */
> > + const struct zs_deferred_ops *deferred_ops;
> > + void *deferred_private;
> > + struct zs_deferred_percpu __percpu *deferred;
> > + struct work_struct deferred_work;
> > + struct workqueue_struct *deferred_wq;
> > + struct list_head deferred_pool;
> > + unsigned int deferred_pool_count;
> > + spinlock_t deferred_pool_lock;
> > + struct list_head deferred_drain_list;
> > + spinlock_t deferred_drain_lock;
> > };
> >
> > static inline void zpdesc_set_first(struct zpdesc *zpdesc)
> > @@ -1416,6 +1435,171 @@ void zs_free(struct zs_pool *pool, unsigned long handle)
> > }
> > EXPORT_SYMBOL_GPL(zs_free);
> >
> > +static struct page *deferred_pool_get(struct zs_pool *pool)
> > +{
> > + struct page *page = NULL;
> > +
> > + spin_lock(&pool->deferred_pool_lock);
> > + if (!list_empty(&pool->deferred_pool)) {
> > + page = list_first_entry(&pool->deferred_pool, struct page, lru);
> > + list_del(&page->lru);
> > + pool->deferred_pool_count--;
> > + }
> > + spin_unlock(&pool->deferred_pool_lock);
> > + return page;
> > +}
> > +
> > +static void deferred_pool_put(struct zfs_pool *pool, struct page *page)
> > +{
> > + spin_lock(&pool->deferred_pool_lock);
> > + list_add_tail(&page->lru, &pool->deferred_pool);
> > + pool->deferred_pool_count++;
> > + spin_unlock(&pool->deferred_pool_lock);
> > +}
> > +
> > +static void zs_deferred_work_fn(struct work_struct *work)
> > +{
> > + struct zs_pool *pool = container_of(work, struct zs_pool, deferred_work);
> > + struct page *page;
> > +
> > + while (true) {
> > + unsigned int count;
> > +
> > + spin_lock(&pool->deferred_drain_lock);
> > + if (list_empty(&pool->deferred_drain_list)) {
> > + spin_unlock(&pool->deferred_drain_lock);
> > + break;
> > + }
> > + page = list_first_entry(&pool->deferred_drain_list,
> > + struct page, lru);
> > + list_del(&page->lru);
> > + count = page_private(page);
> > + spin_unlock(&pool->deferred_drain_lock);
> > +
> > + pool->deferred_ops->drain(pool->deferred_private,
> > + page_address(page), count);
> > + deferred_pool_put(pool, page);
> > + cond_resched();
> > + }
> > +}
> > +
> > +bool zs_free_deferred(struct zs_pool *pool, unsigned long value)
> > +{
> > + struct zs_deferred_percpu *def;
> > + struct page *new_page, *full_page;
> > + enum zs_push_ret ret;
> > +
> > + if (!pool->deferred)
> > + return false;
> > +
> > + def = get_cpu_ptr(pool->deferred);
> > +
> > + ret = pool->deferred_ops->push(def->buf, def->count, value);
> > + if (ret == ZS_PUSH_OK) {
> > + def->count++;
> > + put_cpu_ptr(pool->deferred);
> > + return true;
> > + }
> > +
> > + if (ret == ZS_PUSH_FULL_QUEUED)
> > + def->count++;
> > +
> > + new_page = deferred_pool_get(pool);
> > + if (new_page) {
> > + full_page = virt_to_page(def->buf);
> > + set_page_private(full_page, def->count);
> > + def->buf = page_address(new_page);
> > + def->count = 0;
> > +
> > + if (ret == ZS_PUSH_FULL) {
> > + pool->deferred_ops->push(def->buf, 0, value);
> > + def->count = 1;
> > + }
> > + put_cpu_ptr(pool->deferred);
> > +
> > + spin_lock(&pool->deferred_drain_lock);
> > + list_add_tail(&full_page->lru, &pool->deferred_drain_list);
> > + spin_unlock(&pool->deferred_drain_lock);
> > + queue_work(pool->deferred_wq, &pool->deferred_work);
> > + return true;
> > + }
> > + put_cpu_ptr(pool->deferred);
> > +
> > + /* ret==2: value already queued, will be drained eventually */
> > + if (ret == 2)
>
> == 2? :)
>
Will replace with ZS_PUSH_FULL_QUEUED, if v4 still has
this logic.
Thanks,
Wenchao
^ permalink raw reply
* Re: [RFC PATCH v2 0/4] mm/zsmalloc: reduce zs_free() latency on swap release path
From: Wenchao Hao @ 2026-05-09 9:08 UTC (permalink / raw)
To: Nhat Pham
Cc: Yosry Ahmed, Andrew Morton, Chengming Zhou, Jens Axboe,
Johannes Weiner, Minchan Kim, Sergey Senozhatsky, linux-block,
linux-kernel, linux-mm, Barry Song, Xueyuan Chen, Wenchao Hao
In-Reply-To: <CAKEwX=PgsM4Z0URJcF5VSXX_jdy-cAs2JxDCCR1a8CGwV4tCAg@mail.gmail.com>
On Sat, May 9, 2026 at 7:33 AM Nhat Pham <nphamcs@gmail.com> wrote:
>
> On Wed, May 6, 2026 at 6:55 AM Wenchao Hao <haowenchao22@gmail.com> wrote:
> >
> > On Sat, May 2, 2026 at 3:21 PM Nhat Pham <nphamcs@gmail.com> wrote:
> > >
> > >
> > > Oh man, you are eliminating pool lock here right? This would help my
> > > other patch series a lot too :)
> > >
> > > https://lore.kernel.org/all/CAKEwX=M5YpR0cQrryX_y4pm_BuxyUWZ_8MbhWodwbf1Fe=gzew@mail.gmail.com/
> > > https://lore.kernel.org/all/CAKEwX=PkFiP+u+ThrzjTKBi+usQf2uuhTZcfB2BNNA8RboOFDQ@mail.gmail.com/
> > >
> >
> > Yes, exactly. With class_idx encoded in the obj value,
> > zs_free() can determine the correct size_class without
> > any pool-level lock. The lockless read gives a valid
> > class_idx because it's invariant across migration (only
> > PFN changes), and we re-read obj under class->lock to
> > get a stable PFN.
> >
> > >
> > > /*
> > > * The pool->lock protects the race with zpage's migration
> > > * so it's safe to get the page from handle.
> > > */
> > > read_lock(&pool->lock);
> > > obj = handle_to_obj(handle);
> > > obj_to_zpdesc(obj, &f_zpdesc);
> > > zspage = get_zspage(f_zpdesc);
> > > class = zspage_class(pool, zspage);
> > > spin_lock(&class->lock);
> > > read_unlock(&pool->lock);
> > >
> > > It's basically just this blob right?
> > >
> >
> > Yes, that's the blob being replaced. On the
> > ZS_OBJ_CLASS_IDX path (64-bit systems), it becomes:
> >
> > obj = handle_to_obj(handle);
> > class = pool->size_class[obj_to_class_idx(obj)];
> > spin_lock(&class->lock);
> > obj = handle_to_obj(handle); /* re-read for stable PFN */
> >
> > No pool->lock at all. We've also added compile-time
> > gating (#if BITS_PER_LONG >= 64) since 32-bit systems
> > lack the spare bits in OBJ_INDEX to fit class_idx. On
> > 32-bit, it falls back to the original pool->lock path.
> >
>
> BTW, I've tested your idea with a hacky prototype, when I was playing
> with my vswap series. It absolutely improves free time in the usemem
> benchmark :) Idea is very promising - I won't scoop your work of
> course, just letting you know that at least in my use case, it works
> :) Look forward to seeing it submitted soon!!!
Thanks, Nhat, that's great to hear.
I've split this part out and posted it as its own series:
https://lore.kernel.org/linux-mm/20260508061910.3882831-1-haowenchao@xiaomi.com
Review there would be very welcome.
Also, could you share the details of your usemem setup? I'd like
to reproduce it locally on the same baseline.
Thanks,
Wenchao
^ permalink raw reply
* Re: [PATCH blktests] blktrace/001: Skip test when kernel lockdown is enabled
From: Shin'ichiro Kawasaki @ 2026-05-09 10:35 UTC (permalink / raw)
To: Disha Goel; +Cc: linux-block
In-Reply-To: <f3617b18-526e-4006-a692-b704925de24b@linux.ibm.com>
On May 08, 2026 / 16:51, Disha Goel wrote:
> On 29/04/26 7:22 pm, Shin'ichiro Kawasaki wrote:
[...]
> > I would like to understand why the blktrace error happens in your environment
> > and does not happen in my environment. It will affect how to judge the skip of
> > the test case blktrace/001.
> >
> > Could you share your system set up conditions? FYI, I used Fedora 43, QEMU VM,
> > Intel server and v7.1-rc1 kernel for the trial above. I'm guessing any
> > difference between the two environments causes the blktrace behavior difference.
> >
>
> Hi Shin'ichiro,
>
> Thank you for the detailed testing and feedback.
>
> After further investigation, I've identified the root cause. This
> failure is seen on SLES 16.x and RHEL 10.x with 6.12-based kernels when
> Secure Boot is enabled (kernel lockdown active).
>
> I tested on upstream kernel v7.1.0-rc1 with lockdown enabled, and the
> test ran fine. My apologies for not testing with upstream earlier.
>
> The issue is caused by missing debugfs fixes in the distro kernels.
> I'll work with SUSE and Red Hat to get these patches backported to their
> 6.12-based kernels.
>
> I'm withdrawing this patch as it's a kernel bug, not a test framework issue.
> Thank you for the thorough review.
No worries, and thanks to you. This patch post revealed and shared the problem,
which is great :)
^ permalink raw reply
* Re: [PATCH v3 0/2] drbd: switch from genl_magic to YNL
From: Jens Axboe @ 2026-05-09 13:58 UTC (permalink / raw)
To: Christoph Böhmwalder
Cc: drbd-dev, linux-kernel, Lars Ellenberg, Philipp Reisner,
linux-block, Donald Hunter, Eric Dumazet, Jakub Kicinski, netdev
In-Reply-To: <20260506124541.1951772-1-christoph.boehmwalder@linbit.com>
On Wed, 06 May 2026 14:45:39 +0200, Christoph Böhmwalder wrote:
> DRBD's genetlink interface was defined using a custom multi-include
> macro system, genl_magic_{func,struct}.h. This system generated struct
> definitions, netlink policies, serialization functions and more
> from a single "magic" header.
> It never really caught on; DRBD is its only user, its internal macro
> jungle is next to impossible to understand, and even harder to debug.
>
> [...]
Applied, thanks!
[1/2] drbd: move UAPI headers to include/uapi/linux/
(no commit info)
[2/2] drbd: replace genl_magic with explicit netlink serialization
(no commit info)
Best regards,
--
Jens Axboe
^ permalink raw reply
* [PATCH] ublk: reject max_sectors smaller than PAGE_SECTORS in parameter validation
From: Ming Lei @ 2026-05-10 14:48 UTC (permalink / raw)
To: Jens Axboe, linux-block; +Cc: Caleb Sander Mateos, Uday Shankar, Ming Lei
blk_validate_limits() requires max_hw_sectors >= PAGE_SECTORS and fires
a WARN_ON_ONCE if this invariant is violated. ublk_validate_params()
only checked the upper bound of max_sectors against max_io_buf_bytes,
allowing userspace to pass small values (including zero) that trigger
the warning when blk_mq_alloc_disk() is called from
ublk_ctrl_start_dev().
Before 494ea040bcb5, ublk used blk_queue_max_hw_sectors() which silently
clamped small values up to PAGE_SECTORS. The conversion to passing
queue_limits directly to blk_mq_alloc_disk() lost that clamping and now
hits blk_validate_limits()'s WARN_ON_ONCE instead.
Validate that max_sectors is at least PAGE_SECTORS in
ublk_validate_params() so invalid values are rejected early with
-EINVAL instead of reaching the block layer.
Fixes: 494ea040bcb5 ("ublk: pass queue_limits to blk_mq_alloc_disk")
Signed-off-by: Ming Lei <tom.leiming@gmail.com>
---
drivers/block/ublk_drv.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/block/ublk_drv.c b/drivers/block/ublk_drv.c
index 6d13f1481de0..6c041eaebdb9 100644
--- a/drivers/block/ublk_drv.c
+++ b/drivers/block/ublk_drv.c
@@ -920,6 +920,9 @@ static int ublk_validate_params(const struct ublk_device *ub)
if (p->max_sectors > (ub->dev_info.max_io_buf_bytes >> 9))
return -EINVAL;
+ if (p->max_sectors < PAGE_SECTORS)
+ return -EINVAL;
+
if (ublk_dev_is_zoned(ub) && !p->chunk_sectors)
return -EINVAL;
} else
--
2.53.0
^ permalink raw reply related
* Re: [PATCH v3 02/12] block/bdev: Annotate the blk_holder_ops callback invocations
From: Marco Elver @ 2026-05-11 6:00 UTC (permalink / raw)
To: Bart Van Assche
Cc: Christoph Hellwig, Jens Axboe, linux-block, Damien Le Moal,
Nathan Chancellor, Peter Zijlstra
In-Reply-To: <CANpmjNPzuCUKRPFXZ8ckiCE01kc85xzOiMB05Y9OC9J4Lwi61w@mail.gmail.com>
On Mon, 13 Apr 2026 at 22:03, Marco Elver <elver@google.com> wrote:
[...]
> I already sent the patch to bump min. Clang to 23 for Context
> Analysis: https://lore.kernel.org/all/acqELlVC6vEeEF-W@elver.google.com/
> [1]
>
> While for 1-2 new features we might be fine with more macros (as I
> initially tried for the multi-arg guarded_by), I think that doesn't
> scale if we need to support more versions and even more features. I
> think that's also what Peter suggested, and last I checked he's just
> waiting for the dev version of Clang 23 to be new enough to take the
> above patch [1].
>
> I hope the function pointer support will land in Clang 23 as well
> (when it will be released in ~August). So once the function pointer
> support lands, we should definitely bump min. Clang version.
Clang 23 will have function pointer support:
https://github.com/llvm/llvm-project/commit/d1e84bb9ad3b6a844cd902cf70e2764cb8126979
^ permalink raw reply
* Re: [PATCH] lib: free pagelist on error in iov_iter_extract_pages()
From: Christoph Hellwig @ 2026-05-11 6:35 UTC (permalink / raw)
To: Caleb Sander Mateos
Cc: Dmitry Antipov, Jens Axboe, Andrew Morton, Christoph Hellwig,
linux-block, linux-fsdevel, lvc-project, Fedor Pchelkin,
David Howells, ric Van Hensbergen, Latchesar Ionkov,
Dominique Martinet, v9fs, Ilya Dryomov, Alex Markuze,
Viacheslav Dubeyko, ceph-devel, Pranjal Shrivastava, linux-nfs,
Christian Schoenebeck
In-Reply-To: <CADUfDZpC5WOBY4_xvAy6ORtgQsxwFYj3Px81RdN7NKQZBFdJSQ@mail.gmail.com>
[adding authors and maintainers of relevant code]
On Fri, May 08, 2026 at 11:33:50AM -0700, Caleb Sander Mateos wrote:
> iov_iter_extract_pages() will only allocate a pages array if the
> initial struct page ** passed is NULL (see want_pages_array()). So the
> condition pages != stack_pages will never be true. Indeed, it looks
> like *all* callers of iov_iter_extract_pages() pass a non-NULL struct
> page **. Would it make sense for iov_iter_extract_pages() to require a
> pre-allocated pages array and remove support for allocating one?
I think the idea was to support this to replace existing users of
iov_iter_get_pages_alloc2. Which is urgently neede as those missing
the proper page pinning support. OTOH we should not keep dead code
around just in case.
For NFS, Pranjal has an initial series to convert away from
iov_iter_get_pages_alloc2, which makes use of the NULL pages argument
to iov_iter_extract_pages.
For 9p, Dominique had an untested patch in December that drops
iov_iter_get_pages_alloc2 in favor of a much better high level approach
that doesn't even involve iov_iter_extract_pages, which seems to not have
made it anywhere. It would be great to get this going again.
net/ceph/ needs to also do work. Not an expert on the code, but given
how it is based off the encryption flag it looks kinda fishy and
iov_iter_extract_pages might not be the best direct replacement.
^ permalink raw reply
* Re: [PATCH blktests] blktrace/001: Skip test when kernel lockdown is enabled
From: Disha Goel @ 2026-05-11 7:46 UTC (permalink / raw)
To: Daniel Wagner; +Cc: Shin'ichiro Kawasaki, linux-block
In-Reply-To: <440d1e82-e54f-4709-829b-24d8fb16246e@flourine.local>
On 08/05/26 11:09 pm, Daniel Wagner wrote:
> On Fri, May 08, 2026 at 04:51:41PM +0530, Disha Goel wrote:
>> After further investigation, I've identified the root cause. This
>> failure is seen on SLES 16.x and RHEL 10.x with 6.12-based kernels when
>> Secure Boot is enabled (kernel lockdown active).
>
> SLES ships an bunch of additional patches which hardens the lockdown
> feature a bit more. IIRC these are the same as RHEL uses. Not sure if
> this is the source of the problem though.
>
> https://raw.githubusercontent.com/openSUSE/kernel-source/refs/heads/SL-16.1/series.conf
>
> # Lock down functions for secure boot
> patches.suse/0001-security-lockdown-expose-a-hook-to-lock-the-kernel-down.patch
> patches.suse/lockdown-fix-kernel-lockdown-enforcement-issue-when-secure.patch
> patches.suse/0002-efi-Add-an-EFI_SECURE_BOOT-flag-to-indicate-secure-boot-mode.patch
> patches.suse/0003-efi-Lock-down-the-kernel-if-booted-in-secure-boot-mode.patch
> patches.suse/0004-efi-Lock-down-the-kernel-at-the-integrity-level-if-b.patch
>
> The patches are in https://github.com/openSUSE/kernel-source/tree/SL-16.1/patches.suse
Hi Daniel,
Thank you for pointing me to the SUSE lockdown patches. After analysing
them, I found that the second patch (lockdown-fix-kernel-lockdown-
enforcement-issue-when-secure.patch) is causing the issue.
This patch fixes bsc#1237521 by directly calling lockdown hooks, but it
appears to be too restrictive - it blocks blktrace from accessing
debugfs even though upstream kernel 7.1+ allows this with lockdown enabled.
The patch works around a timing issue but has the side effect of
blocking legitimate debugging tools. Is this intentional, or should
debugging tools like blktrace be allowed even with lockdown enabled?
--
Regards,
Disha
^ permalink raw reply
* Re: [PATCH blktests v3] nvme/068: add a test for multipath delayed removal
From: Daniel Wagner @ 2026-05-11 7:51 UTC (permalink / raw)
To: John Garry, g
Cc: shinichiro.kawasaki, linux-block, linux-nvme, nilay,
Chaitanya Kulkarni
In-Reply-To: <20260430084635.2438048-1-john.g.garry@oracle.com>
Hi John,
On Thu, Apr 30, 2026 at 08:46:35AM +0000, John Garry wrote:
> For NVMe multipath, the delayed removal feature allows the multipath
> gendisk to remain present when all available paths are gone. The purpose of
> this feature is to ensure that we keep the gendisk for intermittent path
> failures.
>
> The delayed removal works on a timer - when all paths are gone, a timer is
> kicked off; once the timer expires and no paths have returned, the gendisk
> is removed.
>
> When all paths are gone and the gendisk is still present, all reads and
> writes to the disk are queued. If a path returns before the timer
> expiration, the timer canceled and the queued IO is submitted;
> otherwise they fail when the timer expires.
>
> This testcase covers two scenarios in separate parts:
> a. test that IOs submitted after all paths are removed (and do not return)
> fail
> b. test that IOs submitted between all paths removed and a path
> returning succeed
>
> During the period of the timer being active, it must be ensured that the
> nvme-core module is not removed. Otherwise the driver may not be present
> to handle the timeout expiry. The kernel ensures this by taking a
> reference to the module. Ideally, we would try to remove the module during
> this test to prove that this is not possible (and the kernel behaves as
> expected), but that module will probably not be removable anyway due to
> many references. To test this feature, check that the refcount of the
> nvme-core module is incremented when the delayed timer is active.
>
> Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>
> Reviewed-by: Nilay Shroff <nilay@linux.ibm.com>
> Signed-off-by: John Garry <john.g.garry@oracle.com>
> ---
>
> @Chaitanya, @Nilay, Please check changes since RB tag granted
>
> Differences to v2:
> - Stop using sleeps so often (Daniel)
Thanks! Looks good to me.
Reviewed-by: Daniel Wagner <dwagner@suse.de>
^ permalink raw reply
* Re: [PATCH blktests] blktrace/001: Skip test when kernel lockdown is enabled
From: Daniel Wagner @ 2026-05-11 7:58 UTC (permalink / raw)
To: Disha Goel; +Cc: Shin'ichiro Kawasaki, linux-block
In-Reply-To: <1621bb15-b425-42d8-9ced-b3886dce1060@linux.ibm.com>
Hi Disha,
On Mon, May 11, 2026 at 01:16:03PM +0530, Disha Goel wrote:
> Thank you for pointing me to the SUSE lockdown patches. After analysing
> them, I found that the second patch (lockdown-fix-kernel-lockdown-
> enforcement-issue-when-secure.patch) is causing the issue.
>
> This patch fixes bsc#1237521 by directly calling lockdown hooks, but it
> appears to be too restrictive - it blocks blktrace from accessing
> debugfs even though upstream kernel 7.1+ allows this with lockdown
> enabled.
Right, so it's a downstream problem as expected.
> The patch works around a timing issue but has the side effect of
> blocking legitimate debugging tools. Is this intentional, or should
> debugging tools like blktrace be allowed even with lockdown enabled?
I am not really involved into this part of our downstream kernel and
don't know why these patches didn't make it upstream :)
Would you mind to you create a SUSE bugzilla issue and we can figure out
a solution there?
Thanks a lot!
Daniel
^ permalink raw reply
* Re: [PATCH v2 5/8] net: of_net: Add of_get_nvmem_eui48() helper for EUI-48 lookup
From: Bartosz Golaszewski @ 2026-05-11 10:36 UTC (permalink / raw)
To: Loic Poulain
Cc: linux-mmc, devicetree, linux-kernel, linux-arm-msm, linux-block,
linux-wireless, ath10k, linux-bluetooth, netdev, daniel,
Ulf Hansson, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Bjorn Andersson, Konrad Dybcio, Jens Axboe, Johannes Berg,
Jeff Johnson, Bartosz Golaszewski, Marcel Holtmann,
Luiz Augusto von Dentz, Balakrishna Godavarthi, Rocky Liao,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Simon Horman, Srinivas Kandagatla, Andrew Lunn, Heiner Kallweit,
Russell King, Saravana Kannan
In-Reply-To: <20260507-block-as-nvmem-v2-5-bf17edd5134e@oss.qualcomm.com>
On Thu, 7 May 2026 17:24:40 +0200, Loic Poulain
<loic.poulain@oss.qualcomm.com> said:
> Factor out the common NVMEM EUI-48 retrieval logic from
> of_get_mac_address_nvmem() into a new of_get_nvmem_eui48() helper that
> accepts the NVMEM cell name as a parameter. This allows other subsystems
> (e.g. Bluetooth) to reuse the same lookup-validate-copy pattern with a
> different cell name, without duplicating code.
>
> of_get_mac_address_nvmem() is updated to call of_get_nvmem_eui48() with
> "mac-address", preserving its existing behavior.
>
> Signed-off-by: Loic Poulain <loic.poulain@oss.qualcomm.com>
> ---
Reviewed-by: Bartosz Golaszewski <bartosz.golaszewski@oss.qualcomm.com>
^ permalink raw reply
* Re: [PATCH 1/2] block: pass a minsize argument to bio_iov_iter_bounce
From: Hannes Reinecke @ 2026-05-11 10:40 UTC (permalink / raw)
To: Christoph Hellwig, Jens Axboe
Cc: Christian Brauner, Darrick J. Wong, Pankaj Raghav, linux-block,
linux-xfs, linux-fsdevel
In-Reply-To: <20260507050153.1298375-2-hch@lst.de>
On 5/7/26 07:01, Christoph Hellwig wrote:
> When bouncing for block size > PAGE_SIZE file systems that require
> file system block size alignment (e.g. zoned XFS), the bio needs to
> be big enough to fit an entire block.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> block/bio.c | 23 +++++++++++++----------
> fs/iomap/direct-io.c | 2 +-
> include/linux/bio.h | 3 ++-
> 3 files changed, 16 insertions(+), 12 deletions(-)
>
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Cheers,
Hannes
--
Dr. Hannes Reinecke Kernel Storage Architect
hare@suse.de +49 911 74053 688
SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg
HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich
^ permalink raw reply
* Re: [PATCH 2/2] block: align down bounces bios
From: Hannes Reinecke @ 2026-05-11 10:40 UTC (permalink / raw)
To: Christoph Hellwig, Jens Axboe
Cc: Christian Brauner, Darrick J. Wong, Pankaj Raghav, linux-block,
linux-xfs, linux-fsdevel
In-Reply-To: <20260507050153.1298375-3-hch@lst.de>
On 5/7/26 07:01, Christoph Hellwig wrote:
> Just like for the extract user pages path, we need to align down the
> size to the supported boundary.
>
> Signed-off-by: Christoph Hellwig <hch@lst.de>
> ---
> block/bio.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/block/bio.c b/block/bio.c
> index f3e5d8bea08c..5f10900b3f42 100644
> --- a/block/bio.c
> +++ b/block/bio.c
> @@ -1345,7 +1345,7 @@ static int bio_iov_iter_bounce_write(struct bio *bio, struct iov_iter *iter,
>
> if (!bio->bi_iter.bi_size)
> return -ENOMEM;
> - return 0;
> + return bio_iov_iter_align_down(bio, iter, minsize - 1);
> }
>
> static int bio_iov_iter_bounce_read(struct bio *bio, struct iov_iter *iter,
> @@ -1383,7 +1383,7 @@ static int bio_iov_iter_bounce_read(struct bio *bio, struct iov_iter *iter,
> bvec_set_folio(&bio->bi_io_vec[0], folio, bio->bi_iter.bi_size, 0);
> if (iov_iter_extract_will_pin(iter))
> bio_set_flag(bio, BIO_PAGE_PINNED);
> - return 0;
> + return bio_iov_iter_align_down(bio, iter, minsize - 1);
> }
>
> /**
Reviewed-by: Hannes Reinecke <hare@kernel.org>
Cheers,
Hannes
--
Dr. Hannes Reinecke Kernel Storage Architect
hare@suse.de +49 911 74053 688
SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg
HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich
^ permalink raw reply
* Re: [PATCH v2 4/8] block: implement NVMEM provider
From: Bartosz Golaszewski @ 2026-05-11 11:27 UTC (permalink / raw)
To: Loic Poulain
Cc: linux-mmc, devicetree, linux-kernel, linux-arm-msm, linux-block,
linux-wireless, ath10k, linux-bluetooth, netdev, daniel,
Ulf Hansson, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Bjorn Andersson, Konrad Dybcio, Jens Axboe, Johannes Berg,
Jeff Johnson, Bartosz Golaszewski, Marcel Holtmann,
Luiz Augusto von Dentz, Balakrishna Godavarthi, Rocky Liao,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Simon Horman, Srinivas Kandagatla, Andrew Lunn, Heiner Kallweit,
Russell King, Saravana Kannan
In-Reply-To: <20260507-block-as-nvmem-v2-4-bf17edd5134e@oss.qualcomm.com>
On Thu, 7 May 2026 17:24:39 +0200, Loic Poulain
<loic.poulain@oss.qualcomm.com> said:
> From: Daniel Golle <daniel@makrotopia.org>
>
> On embedded devices using an eMMC it is common that one or more partitions
> on the eMMC are used to store MAC addresses and Wi-Fi calibration EEPROM
> data. Allow referencing the partition in device tree for the kernel and
> Wi-Fi drivers accessing it via the NVMEM layer.
>
> To safely defer the freeing of the provider private data until all
> consumers have released their cells, a nvmem_dev() accessor is added to
> the NVMEM core to expose the struct device embedded in struct nvmem_device.
> This allows registering a devm action on the nvmem device itself, ensuring
> the private data outlives any active cell references.
>
> Signed-off-by: Daniel Golle <daniel@makrotopia.org>
> Co-developed-by: Loic Poulain <loic.poulain@oss.qualcomm.com>
> Signed-off-by: Loic Poulain <loic.poulain@oss.qualcomm.com>
> ---
...
> diff --git a/block/blk-nvmem.c b/block/blk-nvmem.c
> new file mode 100644
> index 0000000000000000000000000000000000000000..96c0ffc51b1862a75644f3f94add35d59577d86b
> --- /dev/null
> +++ b/block/blk-nvmem.c
> @@ -0,0 +1,188 @@
> +// SPDX-License-Identifier: GPL-2.0-or-later
> +/*
> + * block device NVMEM provider
> + *
> + * Copyright (c) 2024 Daniel Golle <daniel@makrotopia.org>
> + *
> + * Useful on devices using a partition on an eMMC for MAC addresses or
> + * Wi-Fi calibration EEPROM data.
> + */
> +
> +#include "blk.h"
Local includes typically go after system-wide ones. I thought that maybe it's
a block subsystem thing but no, it too goes after here.
> +#include <linux/nvmem-provider.h>
> +#include <linux/nvmem-consumer.h>
> +#include <linux/of.h>
> +#include <linux/pagemap.h>
> +#include <linux/property.h>
> +
> +static void blk_nvmem_free(void *data)
> +{
> + kfree(data);
> +}
> +
> +/* List of all NVMEM devices */
> +static LIST_HEAD(nvmem_devices);
> +static DEFINE_MUTEX(devices_mutex);
> +
> +struct blk_nvmem {
> + struct nvmem_device *nvmem;
> + dev_t devt;
> + bool removed;
> + struct list_head list;
> +};
> +
> +static int blk_nvmem_reg_read(void *priv, unsigned int from,
> + void *val, size_t bytes)
> +{
> + blk_mode_t mode = BLK_OPEN_READ | BLK_OPEN_RESTRICT_WRITES;
> + unsigned long offs = from & ~PAGE_MASK, to_read;
> + pgoff_t f_index = from >> PAGE_SHIFT;
> + struct blk_nvmem *bnv = priv;
> + size_t bytes_left = bytes;
> + struct file *bdev_file;
> + struct folio *folio;
> + void *p;
> + int ret = 0;
> +
> + if (bnv->removed)
> + return -ENODEV;
> +
> + bdev_file = bdev_file_open_by_dev(bnv->devt, mode, priv, NULL);
> + if (!bdev_file)
> + return -ENODEV;
> +
> + if (IS_ERR(bdev_file))
> + return PTR_ERR(bdev_file);
> +
> + while (bytes_left) {
> + folio = read_mapping_folio(bdev_file->f_mapping, f_index++, NULL);
> + if (IS_ERR(folio)) {
> + ret = PTR_ERR(folio);
> + goto err_release_bdev;
> + }
> + to_read = min_t(unsigned long, bytes_left, PAGE_SIZE - offs);
> + p = folio_address(folio) + offset_in_folio(folio, offs);
> + memcpy(val, p, to_read);
> + offs = 0;
> + bytes_left -= to_read;
> + val += to_read;
> + folio_put(folio);
> + }
> +
> +err_release_bdev:
> + fput(bdev_file);
> +
> + return ret;
> +}
> +
> +static int blk_nvmem_register(struct device *dev)
> +{
> + struct device_node *np = dev_of_node(dev);
> + struct block_device *bdev = dev_to_bdev(dev);
> + struct nvmem_config config = {};
> + struct blk_nvmem *bnv;
> +
> + /* skip devices which do not have a device tree node */
> + if (!np)
> + return 0;
> +
> + /* skip devices without an nvmem layout defined */
> + if (!of_get_child_by_name(np, "nvmem-layout"))
> + return 0;
> +
> + /*
> + * skip block device too large to be represented as NVMEM devices
> + * which are using an 'int' as address
> + */
> + if (bdev_nr_bytes(bdev) > INT_MAX)
> + return -EFBIG;
> +
> + bnv = kzalloc_obj(*bnv);
> + if (!bnv)
> + return -ENOMEM;
> +
> + config.id = NVMEM_DEVID_NONE;
> + config.dev = &bdev->bd_device;
> + config.name = dev_name(&bdev->bd_device);
> + config.owner = THIS_MODULE;
> + config.priv = bnv;
> + config.reg_read = blk_nvmem_reg_read;
> + config.size = bdev_nr_bytes(bdev);
> + config.word_size = 1;
> + config.stride = 1;
> + config.read_only = true;
> + config.root_only = true;
> + config.ignore_wp = true;
> + config.of_node = to_of_node(dev->fwnode);
> +
> + bnv->devt = bdev->bd_device.devt;
> + bnv->nvmem = nvmem_register(&config);
> + if (IS_ERR(bnv->nvmem)) {
> + dev_err_probe(&bdev->bd_device, PTR_ERR(bnv->nvmem),
> + "Failed to register NVMEM device\n");
> +
> + kfree(bnv);
> + return PTR_ERR(bnv->nvmem);
> + }
> +
> + /*
> + * Free bnv only when the nvmem device is fully released (i.e. when
> + * its kref hits zero), not at unregister time. This prevents a
> + * use-after-free if a consumer still holds an nvmem_cell reference
> + * when the block device is removed: nvmem_unregister() only does a
> + * kref_put(), so reg_read could still be called with bnv as priv
> + * until the last consumer drops its cell.
> + */
> + if (devm_add_action(nvmem_dev(bnv->nvmem), blk_nvmem_free, bnv)) {
> + nvmem_unregister(bnv->nvmem);
> + kfree(bnv);
> + return -ENOMEM;
> + }
Please take a look at the series[1] I sent, it seems to address this issue at
the nvmem core level. Help reviewing it is appreciated. :)
In any case: I don't think it will work the way you intend it to: the devres
action will be executed when the nvmem device is "unbound" not when it's
"released". Not only that but the device you retrieve here is not the "parent"
device that's actually bound to the driver but the "logical" nvmem device, the
nvmem core creates to back the struct nvmem_device's functionality and reference
counting. In other words: I believe the devres action will never be called.
In general, it's a very bad idea to schedule devres actions on devices you
don't control and in context different than at probe() time.
> +
> + mutex_lock(&devices_mutex);
> + list_add_tail(&bnv->list, &nvmem_devices);
> + mutex_unlock(&devices_mutex);
> +
> + return 0;
> +}
> +
> +static void blk_nvmem_unregister(struct device *dev)
> +{
> + struct blk_nvmem *bnv_c, *bnv = NULL;
> +
> + mutex_lock(&devices_mutex);
> + list_for_each_entry(bnv_c, &nvmem_devices, list) {
> + if (bnv_c->devt == dev_to_bdev(dev)->bd_device.devt) {
> + bnv = bnv_c;
> + break;
> + }
> + }
> +
> + if (!bnv) {
> + mutex_unlock(&devices_mutex);
> + return;
> + }
> +
> + list_del(&bnv->list);
> + mutex_unlock(&devices_mutex);
> + bnv->removed = true;
> + nvmem_unregister(bnv->nvmem);
> +}
> +
> +static struct class_interface blk_nvmem_bus_interface __refdata = {
> + .class = &block_class,
> + .add_dev = &blk_nvmem_register,
> + .remove_dev = &blk_nvmem_unregister,
> +};
> +
> +static int __init blk_nvmem_init(void)
> +{
> + int ret;
> +
> + ret = class_interface_register(&blk_nvmem_bus_interface);
> + if (ret)
> + return ret;
> +
> + return 0;
> +}
> +device_initcall(blk_nvmem_init);
> diff --git a/drivers/nvmem/core.c b/drivers/nvmem/core.c
> index 311cb2e5a5c02d2c6979d7c9bbb7f94abdfbdad1..ee3481229c20b3063c86d0dd66aabbf6b5e29169 100644
> --- a/drivers/nvmem/core.c
> +++ b/drivers/nvmem/core.c
> @@ -2154,6 +2154,19 @@ const char *nvmem_dev_name(struct nvmem_device *nvmem)
> }
> EXPORT_SYMBOL_GPL(nvmem_dev_name);
>
> +/**
> + * nvmem_dev() - Get the struct device of a given nvmem device.
> + *
> + * @nvmem: nvmem device.
> + *
> + * Return: pointer to the struct device of the nvmem device.
> + */
> +struct device *nvmem_dev(struct nvmem_device *nvmem)
> +{
> + return &nvmem->dev;
> +}
> +EXPORT_SYMBOL_GPL(nvmem_dev);
> +
That should be done in a separate patch but see my response above.
> /**
> * nvmem_dev_size() - Get the size of a given nvmem device.
> *
> diff --git a/include/linux/nvmem-consumer.h b/include/linux/nvmem-consumer.h
> index 34c0e58dfa26636d2804fcc7e0bc4a875ee73dae..ce387c89dc8e4bc1241f3b6f36be8c6c95e282ed 100644
> --- a/include/linux/nvmem-consumer.h
> +++ b/include/linux/nvmem-consumer.h
> @@ -82,6 +82,7 @@ int nvmem_device_cell_write(struct nvmem_device *nvmem,
>
> const char *nvmem_dev_name(struct nvmem_device *nvmem);
> size_t nvmem_dev_size(struct nvmem_device *nvmem);
> +struct device *nvmem_dev(struct nvmem_device *nvmem);
>
> void nvmem_add_cell_lookups(struct nvmem_cell_lookup *entries,
> size_t nentries);
> @@ -220,6 +221,11 @@ static inline const char *nvmem_dev_name(struct nvmem_device *nvmem)
> return NULL;
> }
>
> +static inline struct device *nvmem_dev(struct nvmem_device *nvmem)
> +{
> + return NULL;
> +}
> +
> static inline void
> nvmem_add_cell_lookups(struct nvmem_cell_lookup *entries, size_t nentries) {}
> static inline void
>
> --
> 2.34.1
>
>
Thanks,
Bartosz
[1] https://lore.kernel.org/all/20260429-nvmem-unbind-v3-0-2a694f95395b@oss.qualcomm.com/
^ permalink raw reply
* Re: [PATCH v2 7/8] Bluetooth: qca: Set NVMEM BD address quirks when address is invalid
From: Bartosz Golaszewski @ 2026-05-11 11:29 UTC (permalink / raw)
To: Loic Poulain
Cc: linux-mmc, devicetree, linux-kernel, linux-arm-msm, linux-block,
linux-wireless, ath10k, linux-bluetooth, netdev, daniel,
Ulf Hansson, Rob Herring, Krzysztof Kozlowski, Conor Dooley,
Bjorn Andersson, Konrad Dybcio, Jens Axboe, Johannes Berg,
Jeff Johnson, Bartosz Golaszewski, Marcel Holtmann,
Luiz Augusto von Dentz, Balakrishna Godavarthi, Rocky Liao,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Simon Horman, Srinivas Kandagatla, Andrew Lunn, Heiner Kallweit,
Russell King, Saravana Kannan
In-Reply-To: <20260507-block-as-nvmem-v2-7-bf17edd5134e@oss.qualcomm.com>
On Thu, 7 May 2026 17:24:42 +0200, Loic Poulain
<loic.poulain@oss.qualcomm.com> said:
> When the controller BD address is invalid (zero or default),
> set the NVMEM quirks to allow retrieving the address from a
> 'local-bd-address' NVMEM cell. The BD address is often stored
> alongside the WiFi MAC address in big-endian format, so also
> set the big-endian quirk.
>
> Signed-off-by: Loic Poulain <loic.poulain@oss.qualcomm.com>
> ---
> drivers/bluetooth/btqca.c | 5 ++++-
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/bluetooth/btqca.c b/drivers/bluetooth/btqca.c
> index dda76365726f0bfe0e80e05fe04859fa4f0592e1..df33eacfd29fa680f393f90215150743e6001d5b 100644
> --- a/drivers/bluetooth/btqca.c
> +++ b/drivers/bluetooth/btqca.c
> @@ -721,8 +721,11 @@ static int qca_check_bdaddr(struct hci_dev *hdev, const struct qca_fw_config *co
> }
>
> bda = (struct hci_rp_read_bd_addr *)skb->data;
> - if (!bacmp(&bda->bdaddr, &config->bdaddr))
> + if (!bacmp(&bda->bdaddr, &config->bdaddr)) {
> hci_set_quirk(hdev, HCI_QUIRK_USE_BDADDR_PROPERTY);
> + hci_set_quirk(hdev, HCI_QUIRK_USE_BDADDR_NVMEM);
> + hci_set_quirk(hdev, HCI_QUIRK_BDADDR_NVMEM_BE);
> + }
>
> kfree_skb(skb);
>
>
> --
> 2.34.1
>
>
Reviewed-by: Bartosz Golaszewski <bartosz.golaszewski@oss.qualcomm.com>
^ permalink raw reply
* Re: [PATCH v2 8/8] arm64: dts: qcom: arduino-imola: Describe NVMEM layout for WiFi/BT addresses
From: Bartosz Golaszewski @ 2026-05-11 11:30 UTC (permalink / raw)
To: Loic Poulain
Cc: linux-mmc, devicetree, linux-kernel, linux-arm-msm, linux-block,
linux-wireless, ath10k, linux-bluetooth, netdev, daniel,
Konrad Dybcio, Ulf Hansson, Rob Herring, Krzysztof Kozlowski,
Conor Dooley, Bjorn Andersson, Konrad Dybcio, Jens Axboe,
Johannes Berg, Jeff Johnson, Bartosz Golaszewski, Marcel Holtmann,
Luiz Augusto von Dentz, Balakrishna Godavarthi, Rocky Liao,
David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
Simon Horman, Srinivas Kandagatla, Andrew Lunn, Heiner Kallweit,
Russell King, Saravana Kannan
In-Reply-To: <20260507-block-as-nvmem-v2-8-bf17edd5134e@oss.qualcomm.com>
On Thu, 7 May 2026 17:24:43 +0200, Loic Poulain
<loic.poulain@oss.qualcomm.com> said:
> On Arduino Uno-Q, the eMMC boot1 partition is factory provisioned
> with device-specific information such as the WiFi MAC address
> and the Bluetooth BD address. This partition can serve as an
> alternative to additional non-volatile memory, such as a
> dedicated EEPROM.
>
> The eMMC boot partitions are typically good candidates, as they
> are relatively small, read-only by default (and can be enforced
> as hardware read-only), and are not affected by board reflashing
> procedures, which generally target the eMMC user or GP partitions.
>
> Describe the corresponding nvmem-layout for the WiFi and Bluetooth
> addresses, and point the WiFi and Bluetooth nodes to the appropriate
> NVMEM cells to retrieve them.
>
> Signed-off-by: Loic Poulain <loic.poulain@oss.qualcomm.com>
> Reviewed-by: Konrad Dybcio <konrad.dybcio@oss.qualcomm.com>
> ---
Reviewed-by: Bartosz Golaszewski <bartosz.golaszewski@oss.qualcomm.com>
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox