[PATCH bpf v2] bpf: keep trampoline progs alive until image release

[PATCH bpf v2] bpf: keep trampoline progs alive until image release

[Junseo Lim]

arch_prepare_bpf_trampoline() embeds program pointers in the generated
image and passes them to __bpf_prog_enter_recur(). After
bpf_trampoline_update() replaces the image, bpf_tramp_image_put() can keep
the old image executable past a normal RCU grace period.

A detached non-sleepable prog can therefore be freed before a preempted
task reaches rcu_read_lock_dont_migrate() in __bpf_prog_enter_recur(),
leading to a use-after-free.

Keep image-local prog refs and drop them from bpf_tramp_image_free().

Fixes: e21aa341785c ("bpf: Fix fexit trampoline.")
Reported-by: Sechang Lim <rhkrqnwk98@gmail.com>
Signed-off-by: Junseo Lim <zirajs7@gmail.com>
---
v1 -> v2:
- Factor prog ref get/put logic into helpers, per Leon Hwang's suggestion.

v1: https://lore.kernel.org/bpf/20260815071927.147049-1-zirajs7@gmail.com/T/

 include/linux/bpf.h     |  5 +++++
 kernel/bpf/trampoline.c | 30 ++++++++++++++++++++++++++++++
 2 files changed, 35 insertions(+)

diff --git a/include/linux/bpf.h b/include/linux/bpf.h
index 7719f6528445..bacb6bc2e27b 100644
--- a/include/linux/bpf.h
+++ b/include/linux/bpf.h
@@ -1376,6 +1376,11 @@ struct bpf_tramp_image {
 		struct rcu_head rcu;
 		struct work_struct work;
 	};
+#ifdef CONFIG_PREEMPTION
+	/* Programs called from this image must outlive deferred image freeing. */
+	struct bpf_prog *progs[BPF_MAX_TRAMP_LINKS];
+	int nr_progs;
+#endif
 };
 
 struct bpf_trampoline {
diff --git a/kernel/bpf/trampoline.c b/kernel/bpf/trampoline.c
index 1a721fc4bef5..33383d1dc1e0 100644
--- a/kernel/bpf/trampoline.c
+++ b/kernel/bpf/trampoline.c
@@ -529,8 +529,36 @@ bpf_trampoline_get_progs(const struct bpf_trampoline *tr, int *total, bool *ip_a
 	return tnodes;
 }
 
+static void bpf_tramp_image_get_progs(struct bpf_tramp_image *im,
+				      struct bpf_tramp_nodes *tnodes)
+{
+#ifdef CONFIG_PREEMPTION
+	int i, kind;
+
+	for (kind = 0; kind < BPF_TRAMP_MAX; kind++)
+		for (i = 0; i < tnodes[kind].nr_nodes; i++) {
+			struct bpf_prog *prog = tnodes[kind].nodes[i]->link->prog;
+
+			bpf_prog_inc(prog);
+			im->progs[im->nr_progs++] = prog;
+		}
+#endif
+}
+
+static void bpf_tramp_image_put_progs(struct bpf_tramp_image *im)
+{
+#ifdef CONFIG_PREEMPTION
+	int i;
+
+	for (i = 0; i < im->nr_progs; i++)
+		bpf_prog_put(im->progs[i]);
+#endif
+}
+
 static void bpf_tramp_image_free(struct bpf_tramp_image *im)
 {
+	bpf_tramp_image_put_progs(im);
+
 	bpf_image_ksym_del(&im->ksym);
 	arch_free_bpf_trampoline(im->image, im->size);
 	bpf_jit_uncharge_modmem(im->size);
@@ -740,6 +768,8 @@ static int bpf_trampoline_update(struct bpf_trampoline *tr, bool lock_direct_mut
 		goto out;
 	}
 
+	bpf_tramp_image_get_progs(im, tnodes);
+
 	err = arch_prepare_bpf_trampoline(im, im->image, im->image + size,
 					  &tr->func.model, tr->flags, tnodes,
 					  tr->func.addr);
-- 
2.55.0

Re: [PATCH bpf v2] bpf: keep trampoline progs alive until image release

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] The patch restricts the `bpf_tramp_image` program reference counting fix to `CONFIG_PREEMPTION=y`, leaving `CONFIG_PREEMPTION=n` kernels vulnerable to a Use-After-Free if `bpf_trampoline_update()` fails during program unlinking.
--

--- Patch [1]: [PATCH bpf v2] bpf: keep trampoline progs alive until image release ---
commit e97f38fbbba7062b73e2785973928c70f36f1661
Author: Junseo Lim <zirajs7@gmail.com>

bpf: keep trampoline progs alive until image release

arch_prepare_bpf_trampoline() embeds program pointers in the generated
image. After bpf_trampoline_update() replaces the image,
bpf_tramp_image_put() can keep the old image executable past a normal RCU
grace period, allowing a detached non-sleepable prog to be freed before a
preempted task finishes, leading to a use-after-free. This patch keeps
image-local prog refs and drops them from bpf_tramp_image_free().

> diff --git a/include/linux/bpf.h b/include/linux/bpf.h
> --- a/include/linux/bpf.h
> +++ b/include/linux/bpf.h
> @@ -1376,6 +1376,11 @@ struct bpf_tramp_image {
>  		struct rcu_head rcu;
>  		struct work_struct work;
>  	};
> +#ifdef CONFIG_PREEMPTION
> +	/* Programs called from this image must outlive deferred image freeing. */
> +	struct bpf_prog *progs[BPF_MAX_TRAMP_LINKS];
> +	int nr_progs;
> +#endif
>  };

[ ... ]

> diff --git a/kernel/bpf/trampoline.c b/kernel/bpf/trampoline.c
> --- a/kernel/bpf/trampoline.c
> +++ b/kernel/bpf/trampoline.c
> @@ -529,8 +529,36 @@ bpf_trampoline_get_progs(const struct bpf_trampoline *tr, int *total, bool *ip_a
>  	return tnodes;
>  }
>  
> +static void bpf_tramp_image_get_progs(struct bpf_tramp_image *im,
> +				      struct bpf_tramp_nodes *tnodes)
> +{
> +#ifdef CONFIG_PREEMPTION
        ^^^^

[Severity: High]
This is a pre-existing issue, but does restricting this reference counting to
CONFIG_PREEMPTION=y leave CONFIG_PREEMPTION=n kernels vulnerable to a
use-after-free if bpf_trampoline_update() fails during program unlinking?

If a user detaches a BPF tracing program, bpf_tracing_link_release() calls
__bpf_trampoline_unlink_prog(), which removes the program from the trampoline
list and calls bpf_trampoline_update().

If bpf_trampoline_update() fails (for example, due to -ENOMEM in
bpf_tramp_image_alloc()), the error is absorbed by a WARN_ON_ONCE. The link's
reference to the program is subsequently dropped, and the program is freed.

However, because the update failed, the old executable image remains active
and continues to execute the embedded pointers to the freed program memory.

Could the CONFIG_PREEMPTION ifdef guards be removed so that the image always
holds references to its programs, preventing the use-after-free for
CONFIG_PREEMPTION=n kernels as well?

> +	int i, kind;
> +
> +	for (kind = 0; kind < BPF_TRAMP_MAX; kind++)
> +		for (i = 0; i < tnodes[kind].nr_nodes; i++) {
> +			struct bpf_prog *prog = tnodes[kind].nodes[i]->link->prog;
> +
> +			bpf_prog_inc(prog);
> +			im->progs[im->nr_progs++] = prog;
> +		}
> +#endif
> +}

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260820022831.18601-1-zirajs7@gmail.com?part=1

Re: [PATCH bpf v2] bpf: keep trampoline progs alive until image release

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 1698 bytes --]

> diff --git a/include/linux/bpf.h b/include/linux/bpf.h
> index 7719f6528445..bacb6bc2e27b 100644
> --- a/include/linux/bpf.h
> +++ b/include/linux/bpf.h

[ ... ]

> diff --git a/kernel/bpf/trampoline.c b/kernel/bpf/trampoline.c
> index 1a721fc4bef5..33383d1dc1e0 100644
> --- a/kernel/bpf/trampoline.c
> +++ b/kernel/bpf/trampoline.c
> @@ -529,8 +529,36 @@ bpf_trampoline_get_progs(const struct bpf_trampoline *tr, int *total, bool *ip_a
>  	return tnodes;
>  }
>
> +static void bpf_tramp_image_get_progs(struct bpf_tramp_image *im,
> +				      struct bpf_tramp_nodes *tnodes)
> +{
> +#ifdef CONFIG_PREEMPTION
> +	int i, kind;
> +
> +	for (kind = 0; kind < BPF_TRAMP_MAX; kind++)
> +		for (i = 0; i < tnodes[kind].nr_nodes; i++) {
> +			struct bpf_prog *prog = tnodes[kind].nodes[i]->link->prog;
> +
> +			bpf_prog_inc(prog);
> +			im->progs[im->nr_progs++] = prog;
> +		}
> +#endif
> +}

This isn't a bug, but would renaming bpf_tramp_image_get_progs() to
something like bpf_tramp_image_hold_progs() help avoid confusion with
the existing bpf_trampoline_get_progs() right above it? The two helpers
do completely different things: bpf_trampoline_get_progs() collects nodes
from hlists while bpf_tramp_image_get_progs() takes refcounts.

> +
> +static void bpf_tramp_image_put_progs(struct bpf_tramp_image *im)
> +{
> +#ifdef CONFIG_PREEMPTION
> +	int i;
> +
> +	for (i = 0; i < im->nr_progs; i++)
> +		bpf_prog_put(im->progs[i]);
> +#endif
> +}

[ ... ]


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/32326053450

Re: [PATCH bpf v2] bpf: keep trampoline progs alive until image release

[Leon Hwang]

Hi Junseo,

On 20/8/26 10:28, Junseo Lim wrote:
> arch_prepare_bpf_trampoline() embeds program pointers in the generated
> image and passes them to __bpf_prog_enter_recur(). After
> bpf_trampoline_update() replaces the image, bpf_tramp_image_put() can keep
> the old image executable past a normal RCU grace period.
> 
> A detached non-sleepable prog can therefore be freed before a preempted
> task reaches rcu_read_lock_dont_migrate() in __bpf_prog_enter_recur(),
> leading to a use-after-free.
> 
> Keep image-local prog refs and drop them from bpf_tramp_image_free().
> 
> Fixes: e21aa341785c ("bpf: Fix fexit trampoline.")

There was an existing fix [1]. Could you take a look at that fix?

[1]
https://lore.kernel.org/bpf/20260819122252.1782790-1-florent.revest@linux.dev/

Thanks,
Leon

> Reported-by: Sechang Lim <rhkrqnwk98@gmail.com>
> Signed-off-by: Junseo Lim <zirajs7@gmail.com>
> --- [...]

Re: [PATCH bpf v2] bpf: keep trampoline progs alive until image release

[Junseo Lim]

On Thu, Aug 20, 2026 at 05:26:18PM +0800, Leon Hwang wrote:
> 
> There was an existing fix [1]. Could you take a look at that fix?
> 
> [1]
> https://lore.kernel.org/bpf/20260819122252.1782790-1-florent.revest@linux.dev/
> 

Thanks for the pointer. This appears to be the same issue as my v1. I took
a look at Florent's patch, and I think its flexible-array approach is cleaner,
so I'm fine with that version going forward.

Thanks,
Junseo