[PATCH bpf-next 0/2] bpf: Fix task work use-after-free on deletion

[PATCH bpf-next 0/2] bpf: Fix task work use-after-free on deletion

[Yun Lu]

From: Yun Lu <luyun@kylinos.cn>

This series fixes a task-work use-after-free when a BPF program schedules
through a map value pointer retained across deletion. Review of the
earlier task-work series [1] pointed out that cancellation can leave
the deleted value open to context reinitialization.

The original reader keeps the element alive while it queues new work,
but that does not make the new callback part of the grace period freeing
the element. If the callback enters RUNNING outside that waiting set, the
final destructor cannot stop it from accessing the freed key or value.
Patch 1 includes an allowed interleaving and a KASAN stack excerpt from a
controlled reproducer on the base unfixed kernel.

Patch 1 retires task-work slots before non-preallocated HASH/RHASH storage
is returned to the allocator, including abandoned RHASH candidates. It
also orders ordinary cancellation as FREED, conditional unpublish, then
map-owned reference disposal. Existing callback read-side protection and
element grace periods provide the lifetime guarantee.

Patch 2 adds 24 selftests for the retirement gate and compatibility: both
notification modes, live updates, and last-user-reference cleanup followed
by reopening the map by ID. Its callback only counts deliveries, allowing
an unfixed-kernel comparison without deliberately dereferencing freed
storage.

This is a separate series from the captured-task and irq_work BUSY fixes.
It is based on bpf-next commit 7c3188f84b54, which contains those fixes.

Testing on x86-64 QEMU/KVM:

- Same test_progs binary on both kernels, with KASAN, PROVE_LOCKING and
  PROVE_RCU: the unfixed kernel fails exactly the six retirement
  return-value assertions and passes the other 18 cases; the fixed
  kernel passes all 24, with no skips.
- An out-of-tree reproducer triggers the native KASAN report quoted in
  patch 1 on the unfixed kernel on every tested boot; on the fixed kernel
  the stale-pointer schedule is rejected with -ENOENT and no report
  occurs.

[1] https://lore.kernel.org/all/20261012061915.231254-1-luyun_611@163.com/

Yun Lu (2):
  bpf: Retire task work before reclaiming hash elements
  selftests/bpf: Check task work scheduling on retired values

 include/linux/bpf.h                           |   2 +
 kernel/bpf/hashtab.c                          |   7 +
 kernel/bpf/helpers.c                          |  80 +++++-
 .../bpf/prog_tests/task_work_retire.c         | 240 ++++++++++++++++++
 .../selftests/bpf/progs/task_work_retire.c    |  84 ++++++
 5 files changed, 403 insertions(+), 10 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/task_work_retire.c
 create mode 100644 tools/testing/selftests/bpf/progs/task_work_retire.c


base-commit: 7c3188f84b54d8e7643f6ec8454824df2823ba73
-- 
2.43.0

[PATCH bpf-next 1/2] bpf: Retire task work before reclaiming hash elements

[Yun Lu]

From: Yun Lu <luyun@kylinos.cn>

bpf_task_work_cancel_and_free() clears the context pointer in a map value
to NULL. A BPF program can still hold that value across deletion within
its trace read-side critical section. Scheduling through the old pointer
can then install a new context on the deleted element.

The context reference protects the context, not the element referenced by
ctx->map_val. If the new callback is outside the waiting set of the trace
grace period freeing the element, it can outlive the element. The final
destructor marks a RUNNING context FREED and drops the map-owned context
reference, but does not wait for the callback before freeing the element.

One execution allowed by these lifetime rules is:

  1. A BPF program looks up element E and retains its value pointer in a
     trace read-side critical section.

  2. Another task deletes E, clears its task-work slot to NULL during
     cancellation, and returns E to the allocator's deferred free path.

  3. The original reader installs a new context C through the old value
     pointer, queues task work, and leaves its trace read-side section.

  4. The target task enters the callback's trace read-side section and
     changes C to RUNNING before the final destructor writes FREED. Its
     read-side section is not covered by the grace period freeing E.
     With SRCU-fast tasks trace, this can happen after the index flip.

  5. The allocator runs the final destructor and frees E. C remains alive
     through the callback reference, but the callback still accesses E's
     key or value through ctx->map_val.

A controlled reproducer on an x86-64 KVM guest running this series'
base commit with KASAN reported the following fault. The reproducer
keeps a value pointer across deletion as in the interleaving above;
its task-work callback passes the freed value and key bytes to
bpf_map_update_elem() and bpf_map_delete_elem(), whose kernel-side
reads are KASAN-instrumented regardless of BPF JIT state. A production
callback dereferencing ctx->map_val reads the same freed bytes:

  BUG: KASAN: slab-use-after-free in array_map_update_elem+0x4b8/0x7f0
  Read of size 8 at addr ffff88800e61be98 by task uaf_repro/91
  Call Trace:
   __asan_memcpy+0x23/0x60
   array_map_update_elem+0x4b8/0x7f0
   bpf_prog_976f3fa6d36fbef6_cb_probe_step+0xaf/0xe5
   bpf_prog_ac7215eae4609044_process_work_cb+0x12c/0x234
   bpf_task_work_callback+0x351/0x970
   task_work_run+0x116/0x200

  Allocated by task 90:
   alloc_bulk+0x238/0x360
   bpf_mem_alloc_init+0x2fd/0x930
   rhtab_map_alloc+0x352/0x430

  Freed by task 9:
   kfree+0x1b8/0x550
   free_all.isra.0+0x5b/0x110
   __free_rcu+0x87/0x1b0
   srcu_invoke_callbacks+0x360/0x5c0

The fault reproduced on all three boots. With this patch applied, the
same reproducer observes scheduling through the stale pointer rejected
with -ENOENT, no callback entry on the deleted element, and no KASAN
reports across three boots.

Leave ERR_PTR(-ENOENT) in task-work slots before returning non-preallocated
HASH and RHASH elements to the allocator. The existing fetch/acquire error
path rejects the retired slot. Reopen it only when the allocator acquires
the storage again. Also retire a reinitialized RHASH candidate abandoned
after insertion finds an existing element or fails: a reader of a previous
incarnation can still hold its value pointer and install a context there.

Ordinary cancellation must mark the context FREED before detaching it, so
retirement cannot miss a detached context that can still enter RUNNING.
Use a conditional slot update to preserve DEAD or a context installed
after reuse. Protect competing cancellers with regular RCU, and let only
the first FREED transition dispose of the map-owned context reference.
Drop or transfer that reference only after the slot no longer publishes
the context; otherwise late readers could obtain it after reclamation was
already submitted.

A callback now either loses to FREED or establishes its trace read-side
section before retirement completes and the element is submitted for
reclamation. The existing trace grace period therefore covers callbacks
that can access the element. No element reference, additional grace
period or synchronous wait is needed. Preallocated HASH/LRU slots retain
their existing cancellation semantics, as do live updates and uref
cleanup. A retired slot returns -ENOENT until its storage is allocated
again; a scheduling attempt racing with retirement may still succeed and
then be cancelled.

Fixes: 38aa7003e369 ("bpf: task work scheduling kfuncs")
Link: https://sashiko.dev/#/patchset/20261012061915.231254-1-luyun_611@163.com?part=3
Signed-off-by: Yun Lu <luyun@kylinos.cn>
---
 include/linux/bpf.h  |  2 ++
 kernel/bpf/hashtab.c |  7 ++++
 kernel/bpf/helpers.c | 80 ++++++++++++++++++++++++++++++++++++++------
 3 files changed, 79 insertions(+), 10 deletions(-)

diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index 580f89c450b5..8e0b2652b56d 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -667,6 +667,8 @@ void copy_map_value_locked(struct bpf_map *map, void *dst, void *src,
 void bpf_timer_cancel_and_free(void *timer);
 void bpf_wq_cancel_and_free(void *timer);
 void bpf_task_work_cancel_and_free(void *timer);
+void bpf_task_work_free(struct bpf_map *map, void *value);
+void bpf_task_work_init(struct bpf_map *map, void *value);
 void bpf_list_head_free(const struct btf_field *field, void *list_head,
 			struct bpf_spin_lock *spin_lock);
 void bpf_rb_root_free(const struct btf_field *field, void *rb_root,
diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c
index bf78e09f4d51..1b90f0854bbe 100644
--- a/kernel/bpf/hashtab.c
+++ b/kernel/bpf/hashtab.c
@@ -1142,6 +1142,7 @@ static int htab_map_get_next_key(struct bpf_map *map, void *key, void *next_key)
 
 static void htab_elem_free(struct bpf_htab *htab, struct htab_elem *l)
 {
+	bpf_task_work_free(&htab->map, htab_elem_value(l, htab->map.key_size));
 	check_and_cancel_fields(htab, l);
 
 	if (htab->map.map_type == BPF_MAP_TYPE_PERCPU_HASH)
@@ -1317,6 +1318,7 @@ static struct htab_elem *alloc_htab_elem(struct bpf_htab *htab, void *key,
 			goto dec_count;
 		}
 		l_new = container + htab->elem_offset;
+		bpf_task_work_init(&htab->map, htab_elem_value(l_new, key_size));
 	}
 
 	if (htab_has_hash(htab))
@@ -3105,6 +3107,7 @@ static void rhtab_free_elem(void *ptr, void *arg)
 	struct bpf_rhtab *rhtab = arg;
 	struct rhtab_elem *elem = ptr;
 
+	bpf_task_work_free(&rhtab->map, rhtab_elem_value(elem, rhtab->map.key_size));
 	bpf_map_free_internal_structs(&rhtab->map, rhtab_elem_value(elem, rhtab->map.key_size));
 	bpf_mem_cache_free_rcu(&rhtab->ma, elem);
 }
@@ -3177,6 +3180,7 @@ static int rhtab_delete_elem(struct bpf_rhtab *rhtab, struct rhtab_elem *elem, v
 		rhtab_read_elem_value(&rhtab->map, copy, elem, flags);
 		check_and_init_map_value(&rhtab->map, copy);
 	}
+	bpf_task_work_free(&rhtab->map, rhtab_elem_value(elem, rhtab->map.key_size));
 	bpf_obj_cancel_fields(&rhtab->map,
 			      rhtab_elem_value(elem, rhtab->map.key_size));
 	bpf_mem_cache_free_rcu(&rhtab->ma, elem);
@@ -3277,6 +3281,7 @@ static long rhtab_map_update_elem(struct bpf_map *map, void *key, void *value, u
 	if (!elem)
 		return -ENOMEM;
 
+	bpf_task_work_init(map, rhtab_elem_value(elem, map->key_size));
 	memcpy(elem->data, key, map->key_size);
 	copy_map_value(map, rhtab_elem_value(elem, map->key_size), value);
 
@@ -3291,6 +3296,8 @@ static long rhtab_map_update_elem(struct bpf_map *map, void *key, void *value, u
 	bpf_enable_instrumentation();
 
 	if (tmp) {
+		/* A reader of a previous incarnation can still hold this value. */
+		bpf_task_work_free(map, rhtab_elem_value(elem, map->key_size));
 		bpf_mem_cache_free(&rhtab->ma, elem);
 		if (IS_ERR(tmp))
 			return PTR_ERR(tmp);
diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c
index 648abe1ba63a..741877b99b02 100644
--- a/kernel/bpf/helpers.c
+++ b/kernel/bpf/helpers.c
@@ -4531,6 +4531,8 @@ struct bpf_task_work_kern {
 	struct bpf_task_work_ctx *ctx;
 };
 
+#define BPF_TASK_WORK_DEAD ERR_PTR(-ENOENT)
+
 static void bpf_task_work_ctx_reset(struct bpf_task_work_ctx *ctx)
 {
 	if (ctx->prog) {
@@ -4809,7 +4811,13 @@ static int bpf_task_work_schedule(struct task_struct *task, struct bpf_task_work
  * @callback: pointer to BPF subprogram to call
  * @aux: pointer to bpf_prog_aux of the caller BPF program, implicitly set by the verifier
  *
- * Return: 0 if task work has been scheduled successfully, negative error code otherwise
+ * After deletion completes, a non-preallocated HASH/RHASH element rejects
+ * new context acquisition until its storage is allocated again. Replacing
+ * a non-preallocated HASH element has the same effect. Scheduling racing
+ * with retirement may succeed and then be cancelled.
+ *
+ * Return: 0 if scheduling irq_work was queued, -ENOENT for a retired slot,
+ * or another negative error code on failure.
  */
 __bpf_kfunc int bpf_task_work_schedule_signal(struct task_struct *task, struct bpf_task_work *tw,
 					      void *map__const_map, bpf_task_work_callback_t callback,
@@ -4827,7 +4835,10 @@ __bpf_kfunc int bpf_task_work_schedule_signal(struct task_struct *task, struct b
  * @callback: pointer to BPF subprogram to call
  * @aux: pointer to bpf_prog_aux of the caller BPF program, implicitly set by the verifier
  *
- * Return: 0 if task work has been scheduled successfully, negative error code otherwise
+ * See bpf_task_work_schedule_signal() for retired-element handling.
+ *
+ * Return: 0 if scheduling irq_work was queued, -ENOENT for a retired slot,
+ * or another negative error code on failure.
  */
 __bpf_kfunc int bpf_task_work_schedule_resume(struct task_struct *task, struct bpf_task_work *tw,
 					      void *map__const_map, bpf_task_work_callback_t callback,
@@ -5047,25 +5058,74 @@ static void bpf_task_work_cancel_scheduled(struct irq_work *irq_work)
 	bpf_task_work_ctx_put(ctx); /* and here we put map's own ref that was transferred to us */
 }
 
+/* ctx must no longer be published in its map value's slot. */
+static void bpf_task_work_finish_cancel(struct bpf_task_work_ctx *ctx,
+					enum bpf_task_work_state state)
+{
+	/* The first canceller owns the map's reference. */
+	if (state == BPF_TW_FREED)
+		return;
+	if (state == BPF_TW_SCHEDULED) {
+		/* run in irq_work to avoid locks in NMI */
+		init_irq_work(&ctx->irq_work, bpf_task_work_cancel_scheduled);
+		irq_work_queue(&ctx->irq_work);
+		return;
+	}
+
+	bpf_task_work_ctx_put(ctx); /* put bpf map's ref */
+}
+
 void bpf_task_work_cancel_and_free(void *val)
 {
 	struct bpf_task_work_kern *twk = val;
 	struct bpf_task_work_ctx *ctx;
 	enum bpf_task_work_state state;
 
-	ctx = xchg(&twk->ctx, NULL);
-	if (!ctx)
+	/* Another canceller can drop the map's reference. */
+	guard(rcu)();
+	ctx = READ_ONCE(twk->ctx);
+	if (IS_ERR_OR_NULL(ctx))
 		return;
 
+	/*
+	 * Stop new callbacks before detaching the context, so deletion cannot
+	 * miss a context that can still enter RUNNING. Preserve a concurrent
+	 * deletion's marker, or a context installed after element reuse.
+	 */
 	state = xchg(&ctx->state, BPF_TW_FREED);
-	if (state == BPF_TW_SCHEDULED) {
-		/* run in irq_work to avoid locks in NMI */
-		init_irq_work(&ctx->irq_work, bpf_task_work_cancel_scheduled);
-		irq_work_queue(&ctx->irq_work);
+	(void)cmpxchg(&twk->ctx, ctx, NULL);
+	/* A failed CAS also means ctx was detached. Only now may it be freed. */
+	bpf_task_work_finish_cancel(ctx, state);
+}
+
+/* Called before returning an element to the allocator. */
+void bpf_task_work_free(struct bpf_map *map, void *value)
+{
+	struct bpf_task_work_kern *twk;
+	struct bpf_task_work_ctx *ctx;
+	enum bpf_task_work_state state;
+
+	if (!btf_record_has_field(map->record, BPF_TASK_WORK))
 		return;
-	}
+	twk = value + map->record->task_work_off;
 
-	bpf_task_work_ctx_put(ctx); /* put bpf map's ref */
+	guard(rcu)();
+	ctx = xchg(&twk->ctx, BPF_TASK_WORK_DEAD);
+	if (IS_ERR_OR_NULL(ctx))
+		return;
+	state = xchg(&ctx->state, BPF_TW_FREED);
+	bpf_task_work_finish_cancel(ctx, state);
+}
+
+/* Only allocator acquisition may reopen a retired task-work slot. */
+void bpf_task_work_init(struct bpf_map *map, void *value)
+{
+	struct bpf_task_work_kern *twk;
+
+	if (!btf_record_has_field(map->record, BPF_TASK_WORK))
+		return;
+	twk = value + map->record->task_work_off;
+	(void)cmpxchg(&twk->ctx, BPF_TASK_WORK_DEAD, NULL);
 }
 
 BTF_KFUNCS_START(generic_btf_ids)
-- 
2.43.0

Re: [PATCH bpf-next 1/2] bpf: Retire task work before reclaiming hash elements

[Alexei Starovoitov]

On Wed, Oct 28, 2026 at 12:35 AM Yun Lu <luyun_611@163.com> wrote:
> Leave ERR_PTR(-ENOENT) in task-work slots before returning non-preallocated
> HASH and RHASH elements to the allocator. The existing fetch/acquire error
> path rejects the retired slot. Reopen it only when the allocator acquires
> the storage again. Also retire a reinitialized RHASH candidate abandoned
> after insertion finds an existing element or fails: a reader of a previous
> incarnation can still hold its value pointer and install a context there.

This is overkill.
It adds a 3rd value of twk->ctx and two new functions that hashtab.c
calls in 6 places.
The kfuncs start to work differently for prealloc and non-prealloc maps.

The map knows whether the element is deleted. Look it up.
In bpf_task_work_acquire_ctx() the prog owns the ctx in PENDING state.
The irq_work is not queued yet.

  prog                                   delete
  cmpxchg(&twk->ctx, NULL, ctx)          unlink the element
  cmpxchg(&ctx->state, .., PENDING)      xchg(&twk->ctx, NULL)
  lookup

Either the lookup doesn't find the element or the xchg() finds the ctx.
It's the same scheme as the usercnt check in __bpf_async_init().

hashtab.c doesn't change. A deleted element gets -ENOENT in all map types.
The cost is one lookup per bpf_task_work_schedule_*().

The diff doesn't have the reorder of FREED and detach in
bpf_task_work_cancel_and_free().
That's a fix for a different race. It doesn't need a new ctx in a deleted
element. Pls send it as a separate patch with its own steps.

Something like this (compile tested only):

diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c
index 648abe1..acd62ef 100644
--- a/kernel/bpf/helpers.c
+++ b/kernel/bpf/helpers.c
@@ -1148,6 +1148,14 @@ static void *map_key_from_value(struct bpf_map *map, void *value, u32 *arr_idx)
 	return (void *)value - round_up(map->key_size, 8);
 }

+/* Is the element that holds this value still in the map? */
+static bool bpf_map_value_is_linked(struct bpf_map *map, void *value)
+{
+	u32 idx;
+
+	return map->ops->map_lookup_elem(map, map_key_from_value(map, value, &idx)) == value;
+}
+
 enum bpf_async_type {
 	BPF_ASYNC_TYPE_TIMER = 0,
 	BPF_ASYNC_TYPE_WQ,
@@ -4741,6 +4749,19 @@ static struct bpf_task_work_ctx *bpf_task_work_acquire_ctx(struct bpf_task_work
 		return ERR_PTR(-EBUSY);
 	}

+	/*
+	 * The prog can still hold a value of an element that was deleted from the map.
+	 * bpf_task_work_cancel_and_free() is done with it and nothing stops a new callback
+	 * when the element is freed. The ctx is published and PENDING at this point: either
+	 * the lookup doesn't find the element or bpf_task_work_cancel_and_free() finds the ctx.
+	 * An idle ctx stays in the element until the element is reused or freed.
+	 */
+	if (!bpf_map_value_is_linked(map, (void *)tw - map->record->task_work_off)) {
+		(void)cmpxchg(&ctx->state, BPF_TW_PENDING, BPF_TW_STANDBY);
+		bpf_task_work_ctx_put(ctx);
+		return ERR_PTR(-ENOENT);
+	}
+
 	/*
 	 * If no process or bpffs is holding a reference to the map, no new callbacks should be
 	 * scheduled. This does not address any race or correctness issue, but rather is a policy

[...]

> +	bpf_task_work_free(&rhtab->map, rhtab_elem_value(elem, rhtab->map.key_size));
>  	bpf_obj_cancel_fields(&rhtab->map,
>  			      rhtab_elem_value(elem, rhtab->map.key_size));
>  	bpf_mem_cache_free_rcu(&rhtab->ma, elem);

bpf_wq and bpf_timer have the same problem.
bpf_wq_init() on a deleted element installs a new cb. cb->value points to
the element. htab_mem_dtor() ends in cancel_work(). It doesn't wait for
bpf_wq_work(). The callback still runs when the element is freed.
Pls fix all three.

Also trim the commit log. The 5 steps and the splat are enough.

pw-bot: cr

Re: [PATCH bpf-next 1/2] bpf: Retire task work before reclaiming hash elements

[luyun]

Hi Alexei and Mykyta,

Thanks for the review and suggestions.

I've reworked and validated the task_work fixes following Alexei's approach.
The revised fixes check that a map lookup returns the same value before
queueing work, and mark the context FREED before detaching it, without
adding special-field poisoning or allocator hooks.

Now working on the corresponding fixes and validation for bpf_wq and
bpf_timer. I'll send the complete patch series once validation is complete.

---

Thanks,

Yun Lu

在 2026/10/29 01:22, Alexei Starovoitov 写道:
> On Wed, Oct 28, 2026 at 12:35 AM Yun Lu <luyun_611@163.com> wrote:
>> Leave ERR_PTR(-ENOENT) in task-work slots before returning non-preallocated
>> HASH and RHASH elements to the allocator. The existing fetch/acquire error
>> path rejects the retired slot. Reopen it only when the allocator acquires
>> the storage again. Also retire a reinitialized RHASH candidate abandoned
>> after insertion finds an existing element or fails: a reader of a previous
>> incarnation can still hold its value pointer and install a context there.
> This is overkill.
> It adds a 3rd value of twk->ctx and two new functions that hashtab.c
> calls in 6 places.
> The kfuncs start to work differently for prealloc and non-prealloc maps.
>
> The map knows whether the element is deleted. Look it up.
> In bpf_task_work_acquire_ctx() the prog owns the ctx in PENDING state.
> The irq_work is not queued yet.
>
>    prog                                   delete
>    cmpxchg(&twk->ctx, NULL, ctx)          unlink the element
>    cmpxchg(&ctx->state, .., PENDING)      xchg(&twk->ctx, NULL)
>    lookup
>
> Either the lookup doesn't find the element or the xchg() finds the ctx.
> It's the same scheme as the usercnt check in __bpf_async_init().
>
> hashtab.c doesn't change. A deleted element gets -ENOENT in all map types.
> The cost is one lookup per bpf_task_work_schedule_*().
>
> The diff doesn't have the reorder of FREED and detach in
> bpf_task_work_cancel_and_free().
> That's a fix for a different race. It doesn't need a new ctx in a deleted
> element. Pls send it as a separate patch with its own steps.
>
> Something like this (compile tested only):
>
> diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c
> index 648abe1..acd62ef 100644
> --- a/kernel/bpf/helpers.c
> +++ b/kernel/bpf/helpers.c
> @@ -1148,6 +1148,14 @@ static void *map_key_from_value(struct bpf_map *map, void *value, u32 *arr_idx)
>   	return (void *)value - round_up(map->key_size, 8);
>   }
>
> +/* Is the element that holds this value still in the map? */
> +static bool bpf_map_value_is_linked(struct bpf_map *map, void *value)
> +{
> +	u32 idx;
> +
> +	return map->ops->map_lookup_elem(map, map_key_from_value(map, value, &idx)) == value;
> +}
> +
>   enum bpf_async_type {
>   	BPF_ASYNC_TYPE_TIMER = 0,
>   	BPF_ASYNC_TYPE_WQ,
> @@ -4741,6 +4749,19 @@ static struct bpf_task_work_ctx *bpf_task_work_acquire_ctx(struct bpf_task_work
>   		return ERR_PTR(-EBUSY);
>   	}
>
> +	/*
> +	 * The prog can still hold a value of an element that was deleted from the map.
> +	 * bpf_task_work_cancel_and_free() is done with it and nothing stops a new callback
> +	 * when the element is freed. The ctx is published and PENDING at this point: either
> +	 * the lookup doesn't find the element or bpf_task_work_cancel_and_free() finds the ctx.
> +	 * An idle ctx stays in the element until the element is reused or freed.
> +	 */
> +	if (!bpf_map_value_is_linked(map, (void *)tw - map->record->task_work_off)) {
> +		(void)cmpxchg(&ctx->state, BPF_TW_PENDING, BPF_TW_STANDBY);
> +		bpf_task_work_ctx_put(ctx);
> +		return ERR_PTR(-ENOENT);
> +	}
> +
>   	/*
>   	 * If no process or bpffs is holding a reference to the map, no new callbacks should be
>   	 * scheduled. This does not address any race or correctness issue, but rather is a policy
>
> [...]
>
>> +	bpf_task_work_free(&rhtab->map, rhtab_elem_value(elem, rhtab->map.key_size));
>>   	bpf_obj_cancel_fields(&rhtab->map,
>>   			      rhtab_elem_value(elem, rhtab->map.key_size));
>>   	bpf_mem_cache_free_rcu(&rhtab->ma, elem);
> bpf_wq and bpf_timer have the same problem.
> bpf_wq_init() on a deleted element installs a new cb. cb->value points to
> the element. htab_mem_dtor() ends in cancel_work(). It doesn't wait for
> bpf_wq_work(). The callback still runs when the element is freed.
> Pls fix all three.
>
> Also trim the commit log. The 5 steps and the splat are enough.
>
> pw-bot: cr

[PATCH bpf-next 2/2] selftests/bpf: Check task work scheduling on

[Yun Lu]

From: Yun Lu <luyun@kylinos.cn>

Keep a map value pointer across deletion or non-preallocated HASH
replacement in a syscall program, then require scheduling through the old
pointer to return -ENOENT. Run both TWA_RESUME and TWA_SIGNAL cases for
HASH deletion, RHASH deletion and HASH replacement.

Run 32 rounds per retirement case. The first exercises a slot with no
context; subsequent rounds exercise an idle context. After each mutation,
schedule on the live element and wait for its callback to check that
retirement does not prevent subsequent scheduling on live storage. This
does not assume that the allocator returns the same address.

Also check live updates on RHASH, ARRAY, preallocated HASH and LRU_HASH in
both notification modes. Keep a spare LRU slot so an update does not
evict the target while obtaining a candidate. Drop the last map user
reference and reopen each of the five map types by ID, covering both an
unused slot and one with a completed callback. Loaded programs keep the
map alive while its user reference count is zero. Reserve the old map FD
number with /dev/null until the reopened map FD is restored, so skeleton
cleanup cannot close an unrelated descriptor.

Target the calling task and wait with bounded polling and subsequent
syscalls. A successful scheduling call only queues irq_work, which may
publish the task work after test_run has returned. Returning from another
syscall gives TWA_RESUME a further opportunity to execute the callback.

The 24 subtests comprise six retirement cases, eight live-update cases
and ten uref-reopen cases. Without the fix, the six retirement cases fail
at the exact scheduling return-value assertion (0 instead of -ENOENT),
while the other 18 pass. With the fix all 24 pass. The callback only
increments a counter and does not dereference the old value, so the
unfixed-kernel comparison can safely test the retirement gate.

These are retirement and compatibility tests. They do not force physical
reclamation, allocator address reuse or concurrent cancellation.

Signed-off-by: Yun Lu <luyun@kylinos.cn>
---
 .../bpf/prog_tests/task_work_retire.c         | 240 ++++++++++++++++++
 .../selftests/bpf/progs/task_work_retire.c    |  84 ++++++
 2 files changed, 324 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/task_work_retire.c
 create mode 100644 tools/testing/selftests/bpf/progs/task_work_retire.c

diff --git a/tools/testing/selftests/bpf/prog_tests/task_work_retire.c b/tools/testing/selftests/bpf/prog_tests/task_work_retire.c
new file mode 100644
index 000000000000..af7637749a06
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/task_work_retire.c
@@ -0,0 +1,240 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 KylinSoft Corporation. */
+#include <test_progs.h>
+#include <fcntl.h>
+#include <sys/syscall.h>
+#include <unistd.h>
+#include "task_work_retire.skel.h"
+
+#define REUSE_ROUNDS 32
+#define COMPLETION_WAIT_NS 2000000000ULL
+#define COMPLETION_WAIT_ITERS 2000
+
+struct retire_value {
+	__u64 data;
+	struct bpf_task_work tw;
+};
+
+static bool run_prog(struct bpf_program *prog, int expected)
+{
+	LIBBPF_OPTS(bpf_test_run_opts, opts);
+	int err;
+
+	err = bpf_prog_test_run_opts(bpf_program__fd(prog), &opts);
+	return ASSERT_OK(err, "test_run") &&
+	       ASSERT_EQ((__s32)opts.retval, expected, "retval");
+}
+
+static __u64 completed(struct task_work_retire *skel)
+{
+	return __atomic_load_n(&skel->bss->completed, __ATOMIC_ACQUIRE);
+}
+
+static bool wait_completed(struct task_work_retire *skel, __u64 expected)
+{
+	__u64 deadline = get_time_ns() + COMPLETION_WAIT_NS;
+	__u64 count;
+	int i;
+
+	for (i = 0; i < COMPLETION_WAIT_ITERS; i++) {
+		/* irq_work may enqueue task work after test_run returned. */
+		syscall(__NR_gettid); /* Give TWA_RESUME another return to user mode. */
+		count = completed(skel);
+		if (count >= expected)
+			return ASSERT_EQ(count, expected, "completed");
+		if (get_time_ns() >= deadline)
+			break;
+		usleep(1000);
+	}
+	return ASSERT_EQ(completed(skel), expected, "callback_timeout");
+}
+
+static struct task_work_retire *open_test(enum bpf_map_type type, __u32 flags,
+					  bool use_signal)
+{
+	struct task_work_retire *skel;
+	int err;
+
+	skel = task_work_retire__open();
+	if (!ASSERT_OK_PTR(skel, "open"))
+		return NULL;
+	err = bpf_map__set_type(skel->maps.values, type);
+	if (!ASSERT_OK(err, "map_type"))
+		goto fail;
+	err = bpf_map__set_map_flags(skel->maps.values, flags);
+	if (!ASSERT_OK(err, "map_flags"))
+		goto fail;
+	/* A full LRU map evicts the target in prealloc_lru_pop() before the
+	 * BPF_EXIST lookup, so keep one spare slot for the update cases.
+	 */
+	if (type == BPF_MAP_TYPE_LRU_HASH) {
+		err = bpf_map__set_max_entries(skel->maps.values, 2);
+		if (!ASSERT_OK(err, "map_max_entries"))
+			goto fail;
+	}
+	if (!ASSERT_OK(task_work_retire__load(skel), "load"))
+		goto fail;
+	/* __open() already exposes the skeleton BSS mapping; writing after
+	 * load keeps the mode next to the other post-load setup.
+	 */
+	skel->bss->use_signal = use_signal;
+	return skel;
+fail:
+	task_work_retire__destroy(skel);
+	return NULL;
+}
+
+static bool insert_value(struct task_work_retire *skel)
+{
+	struct retire_value value = {};
+	int key = 0;
+
+	return ASSERT_OK(bpf_map_update_elem(bpf_map__fd(skel->maps.values),
+					   &key, &value, BPF_ANY), "insert");
+}
+
+static void retired_value(enum bpf_map_type type, bool use_signal, bool replace)
+{
+	struct task_work_retire *skel;
+	struct bpf_program *mutate;
+	int i;
+
+	skel = open_test(type, BPF_F_NO_PREALLOC, use_signal);
+	if (!skel)
+		return;
+	mutate = replace ? skel->progs.update_then_schedule :
+			   skel->progs.delete_then_schedule;
+	if (!insert_value(skel))
+		goto out;
+
+	for (i = 0; i < REUSE_ROUNDS; i++) {
+		/* First round has no ctx; later rounds have an idle ctx. */
+		if (!run_prog(mutate, -ENOENT) ||
+		    !ASSERT_OK(skel->bss->mutation_err, "mutation") ||
+		    !ASSERT_EQ(completed(skel), i, "retired_not_called"))
+			goto out;
+		if (!replace && !insert_value(skel))
+			goto out;
+		if (!run_prog(skel->progs.schedule_live, 0) ||
+		    !wait_completed(skel, i + 1))
+			goto out;
+	}
+out:
+	task_work_retire__destroy(skel);
+}
+
+static void live_update(enum bpf_map_type type, __u32 flags, bool use_signal)
+{
+	struct task_work_retire *skel;
+
+	skel = open_test(type, flags, use_signal);
+	if (!skel)
+		return;
+	if (!insert_value(skel) || !run_prog(skel->progs.schedule_live, 0) ||
+	    !wait_completed(skel, 1))
+		goto out;
+	if (!run_prog(skel->progs.update_then_schedule, 0) ||
+	    !ASSERT_OK(skel->bss->mutation_err, "update"))
+		goto out;
+	wait_completed(skel, 2);
+out:
+	task_work_retire__destroy(skel);
+}
+
+static void uref_reopen(enum bpf_map_type type, __u32 flags, bool used)
+{
+	struct bpf_map_info info = {};
+	__u32 info_len = sizeof(info);
+	struct task_work_retire *skel;
+	int map_fd, null_fd = -1, reopened_fd = -1;
+	int err;
+
+	skel = open_test(type, flags, false);
+	if (!skel)
+		return;
+	if (!insert_value(skel))
+		goto out;
+	if (used && (!run_prog(skel->progs.schedule_live, 0) ||
+		     !wait_completed(skel, 1)))
+		goto out;
+	map_fd = bpf_map__fd(skel->maps.values);
+	err = bpf_map_get_info_by_fd(map_fd, &info, &info_len);
+	if (!ASSERT_OK(err, "map_info"))
+		goto out;
+	null_fd = open("/dev/null", O_RDONLY | O_CLOEXEC);
+	if (!ASSERT_GE(null_fd, 0, "open_dev_null"))
+		goto out;
+	/*
+	 * Drop the sole map FD, retaining only program references. Reserve
+	 * its number so skeleton cleanup cannot close an unrelated new FD.
+	 */
+	if (!ASSERT_EQ(dup3(null_fd, map_fd, O_CLOEXEC), map_fd, "drop_uref"))
+		goto out;
+	/* Leave the unused case uninitialized until after reopening. */
+	if (used && !run_prog(skel->progs.schedule_live, -EBUSY))
+		goto out;
+	reopened_fd = bpf_map_get_fd_by_id(info.id);
+	if (!ASSERT_GE(reopened_fd, 0, "reopen_by_id"))
+		goto out;
+	if (!ASSERT_EQ(dup3(reopened_fd, map_fd, O_CLOEXEC), map_fd, "restore_map_fd"))
+		goto out;
+	if (!run_prog(skel->progs.schedule_live, 0))
+		goto out;
+	wait_completed(skel, used ? 2 : 1);
+out:
+	if (reopened_fd >= 0)
+		close(reopened_fd);
+	if (null_fd >= 0)
+		close(null_fd);
+	task_work_retire__destroy(skel);
+}
+
+void test_task_work_retire(void)
+{
+	static const struct {
+		const char *name;
+		enum bpf_map_type type;
+		__u32 flags;
+	} maps[] = {
+		{ "hash", BPF_MAP_TYPE_HASH, BPF_F_NO_PREALLOC },
+		{ "rhash", BPF_MAP_TYPE_RHASH, BPF_F_NO_PREALLOC },
+		{ "array", BPF_MAP_TYPE_ARRAY, 0 },
+		{ "prealloc_hash", BPF_MAP_TYPE_HASH, 0 },
+		{ "lru", BPF_MAP_TYPE_LRU_HASH, 0 },
+	};
+	char name[80];
+	int signal, i, used;
+
+	for (signal = 0; signal < 2; signal++) {
+		snprintf(name, sizeof(name), "hash_delete_%s", signal ? "signal" : "resume");
+		if (test__start_subtest(name))
+			retired_value(BPF_MAP_TYPE_HASH, signal, false);
+		snprintf(name, sizeof(name), "rhash_delete_%s", signal ? "signal" : "resume");
+		if (test__start_subtest(name))
+			retired_value(BPF_MAP_TYPE_RHASH, signal, false);
+		snprintf(name, sizeof(name), "hash_replace_%s", signal ? "signal" : "resume");
+		if (test__start_subtest(name))
+			retired_value(BPF_MAP_TYPE_HASH, signal, true);
+		snprintf(name, sizeof(name), "rhash_update_%s", signal ? "signal" : "resume");
+		if (test__start_subtest(name))
+			live_update(BPF_MAP_TYPE_RHASH, BPF_F_NO_PREALLOC, signal);
+		snprintf(name, sizeof(name), "array_update_%s", signal ? "signal" : "resume");
+		if (test__start_subtest(name))
+			live_update(BPF_MAP_TYPE_ARRAY, 0, signal);
+		snprintf(name, sizeof(name), "prealloc_hash_update_%s",
+			 signal ? "signal" : "resume");
+		if (test__start_subtest(name))
+			live_update(BPF_MAP_TYPE_HASH, 0, signal);
+		snprintf(name, sizeof(name), "lru_update_%s", signal ? "signal" : "resume");
+		if (test__start_subtest(name))
+			live_update(BPF_MAP_TYPE_LRU_HASH, 0, signal);
+	}
+	for (i = 0; i < ARRAY_SIZE(maps); i++) {
+		for (used = 0; used < 2; used++) {
+			snprintf(name, sizeof(name), "%s_uref_reopen_%s", maps[i].name,
+				 used ? "used" : "unused");
+			if (test__start_subtest(name))
+				uref_reopen(maps[i].type, maps[i].flags, used);
+		}
+	}
+}
diff --git a/tools/testing/selftests/bpf/progs/task_work_retire.c b/tools/testing/selftests/bpf/progs/task_work_retire.c
new file mode 100644
index 000000000000..69dc19261724
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/task_work_retire.c
@@ -0,0 +1,84 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2026 KylinSoft Corporation. */
+#include <vmlinux.h>
+#include <errno.h>
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+#include "bpf_misc.h"
+
+char _license[] SEC("license") = "GPL";
+
+struct elem {
+	__u64 data;
+	struct bpf_task_work tw;
+};
+
+struct {
+	__uint(type, BPF_MAP_TYPE_HASH);
+	__uint(map_flags, BPF_F_NO_PREALLOC);
+	__uint(max_entries, 1);
+	__type(key, int);
+	__type(value, struct elem);
+} values SEC(".maps");
+
+bool use_signal;
+int mutation_err;
+__u64 completed;
+
+static int callback(struct bpf_map *map, void *key, void *value)
+{
+	/* Also safe on the unfixed kernel: do not dereference key or value. */
+	__sync_fetch_and_add(&completed, 1);
+	return 0;
+}
+
+static __always_inline int schedule_value(struct elem *value)
+{
+	struct task_struct *task = bpf_get_current_task_btf();
+
+	if (use_signal)
+		return bpf_task_work_schedule_signal(task, &value->tw, &values, callback);
+	return bpf_task_work_schedule_resume(task, &value->tw, &values, callback);
+}
+
+SEC("syscall")
+int schedule_live(void *ctx)
+{
+	int key = 0;
+	struct elem *value = bpf_map_lookup_elem(&values, &key);
+
+	return value ? schedule_value(value) : -EINVAL;
+}
+
+SEC("syscall")
+int delete_then_schedule(void *ctx)
+{
+	int key = 0;
+	struct elem *value = bpf_map_lookup_elem(&values, &key);
+
+	if (!value)
+		return -EINVAL;
+	mutation_err = bpf_map_delete_elem(&values, &key);
+	if (mutation_err)
+		return mutation_err;
+
+	/* test_run holds the trace read-side section across both operations. */
+	return schedule_value(value);
+}
+
+SEC("syscall")
+int update_then_schedule(void *ctx)
+{
+	struct elem replacement = {};
+	int key = 0;
+	struct elem *value = bpf_map_lookup_elem(&values, &key);
+
+	if (!value)
+		return -EINVAL;
+	mutation_err = bpf_map_update_elem(&values, &key, &replacement, BPF_EXIST);
+	if (mutation_err)
+		return mutation_err;
+
+	/* HASH replaces the element; RHASH and ARRAY update it in place. */
+	return schedule_value(value);
+}
-- 
2.43.0

Re: [PATCH bpf-next 2/2] selftests/bpf: Check task work scheduling on

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 1222 bytes --]

> selftests/bpf: Check task work scheduling on

TL;DR: Commit message only: the subject line "selftests/bpf: Check task work
scheduling on" seems truncated, ending on a preposition; a word like "retired
map elements" may be missing.

>     selftests/bpf: Check task work scheduling on
>
>     Keep a map value pointer across deletion or non-preallocated HASH
>     replacement in a syscall program, then require scheduling through the old
>     pointer to return -ENOENT. Run both TWA_RESUME and TWA_SIGNAL cases for
>     HASH deletion, RHASH deletion and HASH replacement.
>
>     [ ... ]
>
>     Signed-off-by: Yun Lu <luyun@kylinos.cn>

This isn't a bug, but the subject line ends on a preposition: "Check task
work scheduling on".  Is a word missing at the end?

The body begins a new paragraph right after it, so the object of "on" seems
to have been dropped.  Could the subject be completed, for example:

  selftests/bpf: Check task work scheduling on retired map elements


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/40653588750

Re: [PATCH bpf-next 0/2] bpf: Fix task work use-after-free on deletion

[Mykyta Yatsenko]

On 10/28/26 7:35 AM, Yun Lu wrote:
> From: Yun Lu <luyun@kylinos.cn>
> 
> This series fixes a task-work use-after-free when a BPF program schedules
> through a map value pointer retained across deletion. Review of the
> earlier task-work series [1] pointed out that cancellation can leave
> the deleted value open to context reinitialization.
> 
> The original reader keeps the element alive while it queues new work,
> but that does not make the new callback part of the grace period freeing
> the element. If the callback enters RUNNING outside that waiting set, the
> final destructor cannot stop it from accessing the freed key or value.
> Patch 1 includes an allowed interleaving and a KASAN stack excerpt from a
> controlled reproducer on the base unfixed kernel.
> 
> Patch 1 retires task-work slots before non-preallocated HASH/RHASH storage
> is returned to the allocator, including abandoned RHASH candidates. It
> also orders ordinary cancellation as FREED, conditional unpublish, then
> map-owned reference disposal. Existing callback read-side protection and
> element grace periods provide the lifetime guarantee.
> 
> Patch 2 adds 24 selftests for the retirement gate and compatibility: both
> notification modes, live updates, and last-user-reference cleanup followed
> by reopening the map by ID. Its callback only counts deliveries, allowing
> an unfixed-kernel comparison without deliberately dereferencing freed
> storage.
> 
> This is a separate series from the captured-task and irq_work BUSY fixes.
> It is based on bpf-next commit 7c3188f84b54, which contains those fixes.
> 
> Testing on x86-64 QEMU/KVM:
> 
> - Same test_progs binary on both kernels, with KASAN, PROVE_LOCKING and
>   PROVE_RCU: the unfixed kernel fails exactly the six retirement
>   return-value assertions and passes the other 18 cases; the fixed
>   kernel passes all 24, with no skips.
> - An out-of-tree reproducer triggers the native KASAN report quoted in
>   patch 1 on the unfixed kernel on every tested boot; on the fixed kernel
>   the stale-pointer schedule is rejected with -ENOENT and no report
>   occurs.

It looks like this problem should affect wq and timers (I did not validate though).
The change looks very invasive, especially if we take wq and timers into 
consideration.
We rejected poisoning of the special fields before
https://lore.kernel.org/all/20260216131341.1285427-1-mykyta.yatsenko5@gmail.com/
Maybe this time we should fix it in the allocator as well?

> 
> [1] https://lore.kernel.org/all/20261012061915.231254-1-luyun_611@163.com/
> 
> Yun Lu (2):
>   bpf: Retire task work before reclaiming hash elements
>   selftests/bpf: Check task work scheduling on retired values
> 
>  include/linux/bpf.h                           |   2 +
>  kernel/bpf/hashtab.c                          |   7 +
>  kernel/bpf/helpers.c                          |  80 +++++-
>  .../bpf/prog_tests/task_work_retire.c         | 240 ++++++++++++++++++
>  .../selftests/bpf/progs/task_work_retire.c    |  84 ++++++
>  5 files changed, 403 insertions(+), 10 deletions(-)
>  create mode 100644 tools/testing/selftests/bpf/prog_tests/task_work_retire.c
>  create mode 100644 tools/testing/selftests/bpf/progs/task_work_retire.c
> 
> 
> base-commit: 7c3188f84b54d8e7643f6ec8454824df2823ba73