public inbox for linux-kernel-mentees@lists.linux-foundation.org
 help / color / mirror / Atom feed
* [PATCH] bpf: sync pending IRQ work before freeing ring buffer
@ 2025-10-18 18:11 neqbal
  2025-10-19  1:13 ` Alexei Starovoitov
  0 siblings, 1 reply; 6+ messages in thread
From: neqbal @ 2025-10-18 18:11 UTC (permalink / raw)
  To: ast, andrii
  Cc: daniel, martin.lau, eddyz87, song, yonghong.song, john.fastabend,
	kpsingh, sdf, haoluo, jolsa, bpf, linux-kernel, skhan,
	david.hunter, syzbot+2617fc732430968b45d2, linux-kernel-mentees,
	neqbal

Fix a possible vmalloc out-of-bounds access caused by pending IRQ work
by ensuring all pending IRQ work completes before freeing.

Fixes: 457f44363a88 ("bpf: Implement BPF ring buffer and verifier support for it")
Reported-by: syzbot+2617fc732430968b45d2@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=2617fc732430968b45d2
Tested-by: syzbot+2617fc732430968b45d2@syzkaller.appspotmail.com
Signed-off-by: neqbal <nooraineqbal@gmail.com>
---
 kernel/bpf/ringbuf.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/kernel/bpf/ringbuf.c b/kernel/bpf/ringbuf.c
index 719d73299397..d706c4b7f532 100644
--- a/kernel/bpf/ringbuf.c
+++ b/kernel/bpf/ringbuf.c
@@ -216,6 +216,8 @@ static struct bpf_map *ringbuf_map_alloc(union bpf_attr *attr)
 
 static void bpf_ringbuf_free(struct bpf_ringbuf *rb)
 {
+	irq_work_sync(&rb->work);
+
 	/* copy pages pointer and nr_pages to local variable, as we are going
 	 * to unmap rb itself with vunmap() below
 	 */
-- 
2.51.0


^ permalink raw reply related	[flat|nested] 6+ messages in thread

* Re: [PATCH] bpf: sync pending IRQ work before freeing ring buffer
  2025-10-18 18:11 [PATCH] bpf: sync pending IRQ work before freeing ring buffer neqbal
@ 2025-10-19  1:13 ` Alexei Starovoitov
  2025-10-19 22:30   ` Noorain Eqbal
  0 siblings, 1 reply; 6+ messages in thread
From: Alexei Starovoitov @ 2025-10-19  1:13 UTC (permalink / raw)
  To: neqbal
  Cc: Alexei Starovoitov, Andrii Nakryiko, Daniel Borkmann,
	Martin KaFai Lau, Eduard, Song Liu, Yonghong Song, John Fastabend,
	KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa, bpf, LKML,
	Shuah Khan, david.hunter, syzbot+2617fc732430968b45d2,
	linux-kernel-mentees

On Sat, Oct 18, 2025 at 11:12 AM neqbal <nooraineqbal@gmail.com> wrote:
>
> Fix a possible vmalloc out-of-bounds access caused by pending IRQ work
> by ensuring all pending IRQ work completes before freeing.
>
> Fixes: 457f44363a88 ("bpf: Implement BPF ring buffer and verifier support for it")
> Reported-by: syzbot+2617fc732430968b45d2@syzkaller.appspotmail.com
> Closes: https://syzkaller.appspot.com/bug?extid=2617fc732430968b45d2

Why do you think irq_work_run_list() processes bpf ringbuf in
the above splat?

> Tested-by: syzbot+2617fc732430968b45d2@syzkaller.appspotmail.com
> Signed-off-by: neqbal <nooraineqbal@gmail.com>

Use your real name.

> ---
>  kernel/bpf/ringbuf.c | 2 ++
>  1 file changed, 2 insertions(+)
>
> diff --git a/kernel/bpf/ringbuf.c b/kernel/bpf/ringbuf.c
> index 719d73299397..d706c4b7f532 100644
> --- a/kernel/bpf/ringbuf.c
> +++ b/kernel/bpf/ringbuf.c
> @@ -216,6 +216,8 @@ static struct bpf_map *ringbuf_map_alloc(union bpf_attr *attr)
>
>  static void bpf_ringbuf_free(struct bpf_ringbuf *rb)
>  {
> +       irq_work_sync(&rb->work);

Sort-of kind-of makes sense, but bpf_ringbuf_free() is called
when no references to bpf map are left. User space and bpf progs
are not using it anymore, so irq_work callbacks should have completed
long ago.

pw-bot: cr

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] bpf: sync pending IRQ work before freeing ring buffer
  2025-10-19  1:13 ` Alexei Starovoitov
@ 2025-10-19 22:30   ` Noorain Eqbal
  2025-10-20 16:17     ` Andrii Nakryiko
  0 siblings, 1 reply; 6+ messages in thread
From: Noorain Eqbal @ 2025-10-19 22:30 UTC (permalink / raw)
  To: alexei.starovoitov
  Cc: andrii, ast, bpf, daniel, david.hunter, eddyz87, haoluo,
	john.fastabend, jolsa, kpsingh, linux-kernel-mentees,
	linux-kernel, martin.lau, nooraineqbal, sdf, skhan, song,
	syzbot+2617fc732430968b45d2, yonghong.song

On Sat, Oct 19, 2025 at 1:13 UTC, Alexei Starovoitov wrote:
> Why do you think irq_work_run_list() processes bpf ringbuf in
> the above splat?

In the syzbot reproducer, GDB shows that when bpf_ringbuf_free() is entered 
the ring buffer's irq_work was still pending when the map was being freed.

    (gdb) p rb->work
    $5 = {
      node = {llist = {next = 0xffffffff8dc055c0 <wake_up_kfence_timer_work>},
              {u_flags = 35, a_flags = {counter = 35}}},
      func = 0xffffffff8223ac60 <bpf_ringbuf_notify>,
      irqwait = {task = 0x0}
    }

Here, `u_flags = 0x23` indicates IRQ_WORK_PENDING and IRQ_WORK_BUSY
are set, which shows that irq_work for the ring buffer was still queued
at the time of free. This confirms that `irq_work_run_list()` could
process the ring buffer after memory was freed.

On Sat, Oct 19, 2025 at 1:13 UTC, Alexei Starovoitov wrote:
> Sort-of kind-of makes sense, but bpf_ringbuf_free() is called
> when no references to bpf map are left. User space and bpf progs
> are not using it anymore, so irq_work callbacks should have completed
> long ago.

You're correct that normally all irq_work callbacks should have completed
by the time bpf_ringbuf_free() is called. However, there is a small
race window. In the syzbot reproducer (https://syzkaller.appspot.com/text?tag=ReproC&x=17a24b34580000),
the BPF program is attached to sched_switch and it also writes to the
ring buffer on every context switch. Each forked child creates the
BPF program and quickly drops the last reference after bpf_ringbuf_commit()
queues an irq_work. Because the irq_work runs asynchronously, it may still
be pending when bpf_ringbuf_free() executes, thus creating a small race
window that can lead to use-after-free.

Adding `irq_work_sync(&rb->work)` ensures that all pending notifications
complete before freeing the buffer.

Thanks,
Noorain Eqbal

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] bpf: sync pending IRQ work before freeing ring buffer
  2025-10-19 22:30   ` Noorain Eqbal
@ 2025-10-20 16:17     ` Andrii Nakryiko
  2025-10-20 18:03       ` [PATCH v2] " Noorain Eqbal
  0 siblings, 1 reply; 6+ messages in thread
From: Andrii Nakryiko @ 2025-10-20 16:17 UTC (permalink / raw)
  To: Noorain Eqbal
  Cc: alexei.starovoitov, andrii, ast, bpf, daniel, david.hunter,
	eddyz87, haoluo, john.fastabend, jolsa, kpsingh,
	linux-kernel-mentees, linux-kernel, martin.lau, sdf, skhan, song,
	syzbot+2617fc732430968b45d2, yonghong.song

On Sun, Oct 19, 2025 at 3:30 PM Noorain Eqbal <nooraineqbal@gmail.com> wrote:
>
> On Sat, Oct 19, 2025 at 1:13 UTC, Alexei Starovoitov wrote:
> > Why do you think irq_work_run_list() processes bpf ringbuf in
> > the above splat?
>
> In the syzbot reproducer, GDB shows that when bpf_ringbuf_free() is entered
> the ring buffer's irq_work was still pending when the map was being freed.
>
>     (gdb) p rb->work
>     $5 = {
>       node = {llist = {next = 0xffffffff8dc055c0 <wake_up_kfence_timer_work>},
>               {u_flags = 35, a_flags = {counter = 35}}},
>       func = 0xffffffff8223ac60 <bpf_ringbuf_notify>,
>       irqwait = {task = 0x0}
>     }
>
> Here, `u_flags = 0x23` indicates IRQ_WORK_PENDING and IRQ_WORK_BUSY
> are set, which shows that irq_work for the ring buffer was still queued
> at the time of free. This confirms that `irq_work_run_list()` could
> process the ring buffer after memory was freed.
>
> On Sat, Oct 19, 2025 at 1:13 UTC, Alexei Starovoitov wrote:
> > Sort-of kind-of makes sense, but bpf_ringbuf_free() is called
> > when no references to bpf map are left. User space and bpf progs
> > are not using it anymore, so irq_work callbacks should have completed
> > long ago.
>
> You're correct that normally all irq_work callbacks should have completed
> by the time bpf_ringbuf_free() is called. However, there is a small
> race window. In the syzbot reproducer (https://syzkaller.appspot.com/text?tag=ReproC&x=17a24b34580000),
> the BPF program is attached to sched_switch and it also writes to the
> ring buffer on every context switch. Each forked child creates the
> BPF program and quickly drops the last reference after bpf_ringbuf_commit()
> queues an irq_work. Because the irq_work runs asynchronously, it may still
> be pending when bpf_ringbuf_free() executes, thus creating a small race
> window that can lead to use-after-free.
>
> Adding `irq_work_sync(&rb->work)` ensures that all pending notifications
> complete before freeing the buffer.

I think this all makes sense and the fix should be good. Please add
the above details (perhaps in a bit more condensed form) to the commit
message.

>
> Thanks,
> Noorain Eqbal

^ permalink raw reply	[flat|nested] 6+ messages in thread

* [PATCH v2] bpf: sync pending IRQ work before freeing ring buffer
  2025-10-20 16:17     ` Andrii Nakryiko
@ 2025-10-20 18:03       ` Noorain Eqbal
  2025-10-21 17:10         ` patchwork-bot+netdevbpf
  0 siblings, 1 reply; 6+ messages in thread
From: Noorain Eqbal @ 2025-10-20 18:03 UTC (permalink / raw)
  To: andrii.nakryiko
  Cc: alexei.starovoitov, andrii, ast, bpf, daniel, david.hunter,
	eddyz87, haoluo, john.fastabend, jolsa, kpsingh,
	linux-kernel-mentees, linux-kernel, martin.lau, nooraineqbal, sdf,
	skhan, song, syzbot+2617fc732430968b45d2, yonghong.song

Fix a race where irq_work can be queued in bpf_ringbuf_commit()
but the ring buffer is freed before the work executes.
In the syzbot reproducer, a BPF program attached to sched_switch
triggers bpf_ringbuf_commit(), queuing an irq_work. If the ring buffer
is freed before this work executes, the irq_work thread may accesses
freed memory.
Calling `irq_work_sync(&rb->work)` ensures that all pending irq_work
complete before freeing the buffer

Fixes: 457f44363a88 ("bpf: Implement BPF ring buffer and verifier support for it")
Reported-by: syzbot+2617fc732430968b45d2@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=2617fc732430968b45d2
Tested-by: syzbot+2617fc732430968b45d2@syzkaller.appspotmail.com
Signed-off-by: Noorain Eqbal <nooraineqbal@gmail.com>
---
v2: Updated the commit message for clarity.
---
 kernel/bpf/ringbuf.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/kernel/bpf/ringbuf.c b/kernel/bpf/ringbuf.c
index 719d73299397..d706c4b7f532 100644
--- a/kernel/bpf/ringbuf.c
+++ b/kernel/bpf/ringbuf.c
@@ -216,6 +216,8 @@ static struct bpf_map *ringbuf_map_alloc(union bpf_attr *attr)
 
 static void bpf_ringbuf_free(struct bpf_ringbuf *rb)
 {
+	irq_work_sync(&rb->work);
+
 	/* copy pages pointer and nr_pages to local variable, as we are going
 	 * to unmap rb itself with vunmap() below
 	 */
-- 
2.51.0


^ permalink raw reply related	[flat|nested] 6+ messages in thread

* Re: [PATCH v2] bpf: sync pending IRQ work before freeing ring buffer
  2025-10-20 18:03       ` [PATCH v2] " Noorain Eqbal
@ 2025-10-21 17:10         ` patchwork-bot+netdevbpf
  0 siblings, 0 replies; 6+ messages in thread
From: patchwork-bot+netdevbpf @ 2025-10-21 17:10 UTC (permalink / raw)
  To: Noorain Eqbal
  Cc: andrii.nakryiko, alexei.starovoitov, andrii, ast, bpf, daniel,
	david.hunter, eddyz87, haoluo, john.fastabend, jolsa, kpsingh,
	linux-kernel-mentees, linux-kernel, martin.lau, sdf, skhan, song,
	syzbot+2617fc732430968b45d2, yonghong.song

Hello:

This patch was applied to bpf/bpf.git (master)
by Alexei Starovoitov <ast@kernel.org>:

On Mon, 20 Oct 2025 23:33:01 +0530 you wrote:
> Fix a race where irq_work can be queued in bpf_ringbuf_commit()
> but the ring buffer is freed before the work executes.
> In the syzbot reproducer, a BPF program attached to sched_switch
> triggers bpf_ringbuf_commit(), queuing an irq_work. If the ring buffer
> is freed before this work executes, the irq_work thread may accesses
> freed memory.
> Calling `irq_work_sync(&rb->work)` ensures that all pending irq_work
> complete before freeing the buffer
> 
> [...]

Here is the summary with links:
  - [v2] bpf: sync pending IRQ work before freeing ring buffer
    https://git.kernel.org/bpf/bpf/c/4e9077638301

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html



^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2025-10-21 17:10 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-10-18 18:11 [PATCH] bpf: sync pending IRQ work before freeing ring buffer neqbal
2025-10-19  1:13 ` Alexei Starovoitov
2025-10-19 22:30   ` Noorain Eqbal
2025-10-20 16:17     ` Andrii Nakryiko
2025-10-20 18:03       ` [PATCH v2] " Noorain Eqbal
2025-10-21 17:10         ` patchwork-bot+netdevbpf

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox