[PATCH 1/2] mm/slab: Add kvfree_atomic() helper

[PATCH 1/2] mm/slab: Add kvfree_atomic() helper

[Uladzislau Rezki (Sony)]

kvmalloc() now supports non-sleeping GFP flags, including
the vmalloc fallback path. This means it may return vmalloc
memory even for GFP_ATOMIC and GFP_NOWAIT allocations.

Freeing such memory with kvfree() may then end up calling
vfree(), which is not safe for non-sleeping contexts.

Introduce kvfree_atomic() helper for such cases. It mirrors
kvfree(), but uses vfree_atomic() for vmalloced memory.

Signed-off-by: Uladzislau Rezki (Sony) <urezki@gmail.com>
---
 include/linux/slab.h |  3 +++
 mm/slub.c            | 16 ++++++++++++++++
 2 files changed, 19 insertions(+)

diff --git a/include/linux/slab.h b/include/linux/slab.h
index 15a60b501b95..2b5ab488e96b 100644
--- a/include/linux/slab.h
+++ b/include/linux/slab.h
@@ -1234,6 +1234,9 @@ void *kvrealloc_node_align_noprof(const void *p, size_t size, unsigned long alig
 extern void kvfree(const void *addr);
 DEFINE_FREE(kvfree, void *, if (!IS_ERR_OR_NULL(_T)) kvfree(_T))
 
+extern void kvfree_atomic(const void *addr);
+DEFINE_FREE(kvfree_atomic, void *, if (!IS_ERR_OR_NULL(_T)) kvfree_atomic(_T))
+
 extern void kvfree_sensitive(const void *addr, size_t len);
 
 unsigned int kmem_cache_size(struct kmem_cache *s);
diff --git a/mm/slub.c b/mm/slub.c
index 2b2d33cc735c..b096677c8152 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -6802,6 +6802,22 @@ void kvfree(const void *addr)
 }
 EXPORT_SYMBOL(kvfree);
 
+/**
+ * kvfree_atomic() - Free memory.
+ * @addr: Pointer to allocated memory.
+ *
+ * Same as kvfree(), but uses vfree_atomic() for vmalloc
+ * backed memory. Must not be called from NMI context.
+ */
+void kvfree_atomic(const void *addr)
+{
+	if (is_vmalloc_addr(addr))
+		vfree_atomic(addr);
+	else
+		kfree(addr);
+}
+EXPORT_SYMBOL(kvfree_atomic);
+
 /**
  * kvfree_sensitive - Free a data object containing sensitive information.
  * @addr: address of the data object to be freed.
-- 
2.47.3

[PATCH 2/2] rhashtable: Add bucket_table_free_atomic() helper

[Uladzislau Rezki (Sony)]

rhashtable_insert_rehash() allocates a new bucket table
with GFP_ATOMIC, as it is called from an RCU read-side
critical section.

If rhashtable_rehash_attach() then fails, the new table
is freed via kvfree(). This is unsafe, since kvfree() may
fall back to vfree() for vmalloc-backed allocations, which
can sleep and trigger:

  BUG: sleeping function called from invalid context

Add bucket_table_free_atomic(), which uses kvfree_atomic()
so the table can be freed safely from non-sleeping context.

Signed-off-by: Uladzislau Rezki (Sony) <urezki@gmail.com>
---
 lib/rhashtable.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/lib/rhashtable.c b/lib/rhashtable.c
index 6074ed5f66f3..4111aab8cee4 100644
--- a/lib/rhashtable.c
+++ b/lib/rhashtable.c
@@ -114,6 +114,14 @@ static void bucket_table_free(const struct bucket_table *tbl)
 	kvfree(tbl);
 }
 
+static void bucket_table_free_atomic(const struct bucket_table *tbl)
+{
+	if (tbl->nest)
+		nested_bucket_table_free(tbl);
+
+	kvfree_atomic(tbl);
+}
+
 static void bucket_table_free_rcu(struct rcu_head *head)
 {
 	bucket_table_free(container_of(head, struct bucket_table, rcu));
@@ -473,7 +481,7 @@ static int rhashtable_insert_rehash(struct rhashtable *ht,
 
 	err = rhashtable_rehash_attach(ht, tbl, new_tbl);
 	if (err) {
-		bucket_table_free(new_tbl);
+		bucket_table_free_atomic(new_tbl);
 		if (err == -EEXIST)
 			err = 0;
 	} else
-- 
2.47.3

Re: [PATCH 1/2] mm/slab: Add kvfree_atomic() helper

[Vlastimil Babka]

+Cc SLAB maintainers. Please use get_maintainers.pl next time.

On 4/28/26 18:14, Uladzislau Rezki (Sony) wrote:
> kvmalloc() now supports non-sleeping GFP flags, including
> the vmalloc fallback path. This means it may return vmalloc
> memory even for GFP_ATOMIC and GFP_NOWAIT allocations.
> 
> Freeing such memory with kvfree() may then end up calling
> vfree(), which is not safe for non-sleeping contexts.
> 
> Introduce kvfree_atomic() helper for such cases. It mirrors
> kvfree(), but uses vfree_atomic() for vmalloced memory.
> 
> Signed-off-by: Uladzislau Rezki (Sony) <urezki@gmail.com>

Acked-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>

> ---
>  include/linux/slab.h |  3 +++
>  mm/slub.c            | 16 ++++++++++++++++
>  2 files changed, 19 insertions(+)
> 
> diff --git a/include/linux/slab.h b/include/linux/slab.h
> index 15a60b501b95..2b5ab488e96b 100644
> --- a/include/linux/slab.h
> +++ b/include/linux/slab.h
> @@ -1234,6 +1234,9 @@ void *kvrealloc_node_align_noprof(const void *p, size_t size, unsigned long alig
>  extern void kvfree(const void *addr);
>  DEFINE_FREE(kvfree, void *, if (!IS_ERR_OR_NULL(_T)) kvfree(_T))
>  
> +extern void kvfree_atomic(const void *addr);
> +DEFINE_FREE(kvfree_atomic, void *, if (!IS_ERR_OR_NULL(_T)) kvfree_atomic(_T))
> +
>  extern void kvfree_sensitive(const void *addr, size_t len);
>  
>  unsigned int kmem_cache_size(struct kmem_cache *s);
> diff --git a/mm/slub.c b/mm/slub.c
> index 2b2d33cc735c..b096677c8152 100644
> --- a/mm/slub.c
> +++ b/mm/slub.c
> @@ -6802,6 +6802,22 @@ void kvfree(const void *addr)
>  }
>  EXPORT_SYMBOL(kvfree);
>  
> +/**
> + * kvfree_atomic() - Free memory.
> + * @addr: Pointer to allocated memory.
> + *
> + * Same as kvfree(), but uses vfree_atomic() for vmalloc
> + * backed memory. Must not be called from NMI context.
> + */
> +void kvfree_atomic(const void *addr)
> +{
> +	if (is_vmalloc_addr(addr))
> +		vfree_atomic(addr);
> +	else
> +		kfree(addr);
> +}
> +EXPORT_SYMBOL(kvfree_atomic);
> +
>  /**
>   * kvfree_sensitive - Free a data object containing sensitive information.
>   * @addr: address of the data object to be freed.

Re: [PATCH 1/2] mm/slab: Add kvfree_atomic() helper

[Uladzislau Rezki]

On Wed, Apr 29, 2026 at 10:28:02AM +0200, Vlastimil Babka wrote:
> +Cc SLAB maintainers. Please use get_maintainers.pl next time.
> 
> On 4/28/26 18:14, Uladzislau Rezki (Sony) wrote:
> > kvmalloc() now supports non-sleeping GFP flags, including
> > the vmalloc fallback path. This means it may return vmalloc
> > memory even for GFP_ATOMIC and GFP_NOWAIT allocations.
> > 
> > Freeing such memory with kvfree() may then end up calling
> > vfree(), which is not safe for non-sleeping contexts.
> > 
> > Introduce kvfree_atomic() helper for such cases. It mirrors
> > kvfree(), but uses vfree_atomic() for vmalloced memory.
> > 
> > Signed-off-by: Uladzislau Rezki (Sony) <urezki@gmail.com>
> 
> Acked-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
> 
Thank you and thank you for adding SLAB maintainers!

--
Uladzislau Rezki

Re: [PATCH 1/2] mm/slab: Add kvfree_atomic() helper

[Harry Yoo (Oracle)]

On Tue, Apr 28, 2026 at 06:14:18PM +0200, Uladzislau Rezki (Sony) wrote:
> kvmalloc() now supports non-sleeping GFP flags, including
> the vmalloc fallback path. This means it may return vmalloc
> memory even for GFP_ATOMIC and GFP_NOWAIT allocations.
> 
> Freeing such memory with kvfree() may then end up calling
> vfree(), which is not safe for non-sleeping contexts.
> 
> Introduce kvfree_atomic() helper for such cases. It mirrors
> kvfree(), but uses vfree_atomic() for vmalloced memory.
> 
> Signed-off-by: Uladzislau Rezki (Sony) <urezki@gmail.com>
> ---

Oh, allowing k[v]free() to be called in interrupt context but not in
non-sleepable context is confusing... but that's not new.

Looks good to me,
Acked-by: Harry Yoo (Oracle) <harry@kernel.org>

Thanks for fixing it, Ulad!

-- 
Cheers,
Harry / Hyeonggon

Re: [PATCH 1/2] mm/slab: Add kvfree_atomic() helper

[Herbert Xu]

On Tue, Apr 28, 2026 at 06:14:18PM +0200, Uladzislau Rezki (Sony) wrote:
> kvmalloc() now supports non-sleeping GFP flags, including
> the vmalloc fallback path. This means it may return vmalloc
> memory even for GFP_ATOMIC and GFP_NOWAIT allocations.
> 
> Freeing such memory with kvfree() may then end up calling
> vfree(), which is not safe for non-sleeping contexts.
> 
> Introduce kvfree_atomic() helper for such cases. It mirrors
> kvfree(), but uses vfree_atomic() for vmalloced memory.
> 
> Signed-off-by: Uladzislau Rezki (Sony) <urezki@gmail.com>
> ---
>  include/linux/slab.h |  3 +++
>  mm/slub.c            | 16 ++++++++++++++++
>  2 files changed, 19 insertions(+)

All applied.  Thanks.
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt