[syzbot] KASAN: use-after-free Read in put_pmu_ctx

[syzbot] KASAN: use-after-free Read in put_pmu_ctx

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit:    13e3c7793e2f Merge tag 'for-netdev' of https://git.kernel...
git tree:       bpf
console+strace: https://syzkaller.appspot.com/x/log.txt?x=177df7e0480000
kernel config:  https://syzkaller.appspot.com/x/.config?x=b0e91ad4b5f69c47
dashboard link: https://syzkaller.appspot.com/bug?extid=b8e8c01c8ade4fe6e48f
compiler:       gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2
syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=15e87100480000
C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=16ceeb13880000

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/373a99daa295/disk-13e3c779.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/7fa71ed0fe17/vmlinux-13e3c779.xz
kernel image: https://storage.googleapis.com/syzbot-assets/2842ad5c698b/bzImage-13e3c779.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+b8e8c01c8ade4fe6e48f@syzkaller.appspotmail.com

==================================================================
BUG: KASAN: use-after-free in __lock_acquire+0x3ee7/0x56d0 kernel/locking/lockdep.c:4925
Read of size 8 at addr ffff8880237d6018 by task syz-executor287/8300

CPU: 0 PID: 8300 Comm: syz-executor287 Not tainted 6.1.0-syzkaller-09661-g13e3c7793e2f #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/26/2022
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0xd1/0x138 lib/dump_stack.c:106
 print_address_description mm/kasan/report.c:284 [inline]
 print_report+0x15e/0x45d mm/kasan/report.c:395
 kasan_report+0xbf/0x1f0 mm/kasan/report.c:495
 __lock_acquire+0x3ee7/0x56d0 kernel/locking/lockdep.c:4925
 lock_acquire kernel/locking/lockdep.c:5668 [inline]
 lock_acquire+0x1e3/0x630 kernel/locking/lockdep.c:5633
 __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
 _raw_spin_lock_irqsave+0x3d/0x60 kernel/locking/spinlock.c:162
 put_pmu_ctx kernel/events/core.c:4913 [inline]
 put_pmu_ctx+0xad/0x390 kernel/events/core.c:4893
 _free_event+0x3c5/0x13d0 kernel/events/core.c:5196
 free_event+0x58/0xc0 kernel/events/core.c:5224
 __do_sys_perf_event_open+0x66d/0x2980 kernel/events/core.c:12701
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x63/0xcd
RIP: 0033:0x7f3a2b1b3f29
Code: 28 00 00 00 75 05 48 83 c4 28 c3 e8 b1 14 00 00 90 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 c0 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007fff4215df68 EFLAGS: 00000246 ORIG_RAX: 000000000000012a
RAX: ffffffffffffffda RBX: 00000000000f4240 RCX: 00007f3a2b1b3f29
RDX: 0000000000000000 RSI: 0000000000002070 RDI: 0000000020000480
RBP: 0000000000000000 R08: 0000000000000008 R09: 0000000000000001
R10: 00000000ffffffff R11: 0000000000000246 R12: 0000000000012a59
R13: 00007fff4215df7c R14: 00007fff4215df90 R15: 00007fff4215df80
 </TASK>

Allocated by task 8300:
 kasan_save_stack+0x22/0x40 mm/kasan/common.c:45
 kasan_set_track+0x25/0x30 mm/kasan/common.c:52
 ____kasan_kmalloc mm/kasan/common.c:371 [inline]
 ____kasan_kmalloc mm/kasan/common.c:330 [inline]
 __kasan_kmalloc+0xa5/0xb0 mm/kasan/common.c:380
 kmalloc include/linux/slab.h:580 [inline]
 kzalloc include/linux/slab.h:720 [inline]
 alloc_perf_context kernel/events/core.c:4693 [inline]
 find_get_context+0xcc/0x810 kernel/events/core.c:4763
 __do_sys_perf_event_open+0x963/0x2980 kernel/events/core.c:12476
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x63/0xcd

Freed by task 5310:
 kasan_save_stack+0x22/0x40 mm/kasan/common.c:45
 kasan_set_track+0x25/0x30 mm/kasan/common.c:52
 kasan_save_free_info+0x2e/0x40 mm/kasan/generic.c:518
 ____kasan_slab_free mm/kasan/common.c:236 [inline]
 ____kasan_slab_free+0x160/0x1c0 mm/kasan/common.c:200
 kasan_slab_free include/linux/kasan.h:177 [inline]
 slab_free_hook mm/slub.c:1781 [inline]
 slab_free_freelist_hook+0x8b/0x1c0 mm/slub.c:1807
 slab_free mm/slub.c:3787 [inline]
 __kmem_cache_free+0xaf/0x3b0 mm/slub.c:3800
 rcu_do_batch kernel/rcu/tree.c:2244 [inline]
 rcu_core+0x81f/0x1980 kernel/rcu/tree.c:2504
 __do_softirq+0x1fb/0xadc kernel/softirq.c:571

Last potentially related work creation:
 kasan_save_stack+0x22/0x40 mm/kasan/common.c:45
 __kasan_record_aux_stack+0xbc/0xd0 mm/kasan/generic.c:488
 __call_rcu_common.constprop.0+0x99/0x820 kernel/rcu/tree.c:2753
 put_ctx kernel/events/core.c:1180 [inline]
 put_ctx+0x116/0x1e0 kernel/events/core.c:1173
 perf_event_exit_task_context kernel/events/core.c:13046 [inline]
 perf_event_exit_task+0x556/0x760 kernel/events/core.c:13073
 do_exit+0xb4d/0x2a30 kernel/exit.c:829
 __do_sys_exit kernel/exit.c:917 [inline]
 __se_sys_exit kernel/exit.c:915 [inline]
 __x64_sys_exit+0x42/0x50 kernel/exit.c:915
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x63/0xcd

The buggy address belongs to the object at ffff8880237d6000
 which belongs to the cache kmalloc-512 of size 512
The buggy address is located 24 bytes inside of
 512-byte region [ffff8880237d6000, ffff8880237d6200)

The buggy address belongs to the physical page:
page:ffffea00008df500 refcount:1 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x237d4
head:ffffea00008df500 order:2 compound_mapcount:0 compound_pincount:0
anon flags: 0xfff00000010200(slab|head|node=0|zone=1|lastcpupid=0x7ff)
raw: 00fff00000010200 ffff888012441c80 0000000000000000 dead000000000001
raw: 0000000000000000 0000000080100010 00000001ffffffff 0000000000000000
page dumped because: kasan: bad access detected
page_owner tracks the page as allocated
page last allocated via order 2, migratetype Unmovable, gfp_mask 0xd20c0(__GFP_IO|__GFP_FS|__GFP_NOWARN|__GFP_NORETRY|__GFP_COMP|__GFP_NOMEMALLOC), pid 2173, tgid 2173 (kworker/u4:3), ts 10211275854, free_ts 0
 prep_new_page mm/page_alloc.c:2539 [inline]
 get_page_from_freelist+0x10b5/0x2d50 mm/page_alloc.c:4291
 __alloc_pages+0x1cb/0x5b0 mm/page_alloc.c:5558
 alloc_pages+0x1aa/0x270 mm/mempolicy.c:2285
 alloc_slab_page mm/slub.c:1851 [inline]
 allocate_slab+0x25f/0x350 mm/slub.c:1998
 new_slab mm/slub.c:2051 [inline]
 ___slab_alloc+0xa91/0x1400 mm/slub.c:3193
 __slab_alloc.constprop.0+0x56/0xa0 mm/slub.c:3292
 __slab_alloc_node mm/slub.c:3345 [inline]
 slab_alloc_node mm/slub.c:3442 [inline]
 __kmem_cache_alloc_node+0x1a4/0x430 mm/slub.c:3491
 kmalloc_trace+0x26/0x60 mm/slab_common.c:1062
 kmalloc include/linux/slab.h:580 [inline]
 kzalloc include/linux/slab.h:720 [inline]
 alloc_bprm+0x51/0x900 fs/exec.c:1510
 kernel_execve+0xaf/0x500 fs/exec.c:1981
 call_usermodehelper_exec_async+0x2e7/0x580 kernel/umh.c:113
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:306
page_owner free stack trace missing

Memory state around the buggy address:
 ffff8880237d5f00: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
 ffff8880237d5f80: fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc fc
>ffff8880237d6000: fa fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
                            ^
 ffff8880237d6080: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
 ffff8880237d6100: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
==================================================================


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzkaller@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
syzbot can test patches for this issue, for details see:
https://goo.gl/tpsmEJ#testing-patches

Re: [syzbot] KASAN: use-after-free Read in put_pmu_ctx

[Peter Zijlstra]

On Mon, Dec 19, 2022 at 12:04:43AM -0800, syzbot wrote:
> Hello,
> 
> syzbot found the following issue on:
> 
> HEAD commit:    13e3c7793e2f Merge tag 'for-netdev' of https://git.kernel...
> git tree:       bpf
> console+strace: https://syzkaller.appspot.com/x/log.txt?x=177df7e0480000
> kernel config:  https://syzkaller.appspot.com/x/.config?x=b0e91ad4b5f69c47
> dashboard link: https://syzkaller.appspot.com/bug?extid=b8e8c01c8ade4fe6e48f
> compiler:       gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2
> syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=15e87100480000
> C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=16ceeb13880000
> 
> Downloadable assets:
> disk image: https://storage.googleapis.com/syzbot-assets/373a99daa295/disk-13e3c779.raw.xz
> vmlinux: https://storage.googleapis.com/syzbot-assets/7fa71ed0fe17/vmlinux-13e3c779.xz
> kernel image: https://storage.googleapis.com/syzbot-assets/2842ad5c698b/bzImage-13e3c779.xz
> 
> IMPORTANT: if you fix the issue, please add the following tag to the commit:
> Reported-by: syzbot+b8e8c01c8ade4fe6e48f@syzkaller.appspotmail.com
> 
> ==================================================================
> BUG: KASAN: use-after-free in __lock_acquire+0x3ee7/0x56d0 kernel/locking/lockdep.c:4925
> Read of size 8 at addr ffff8880237d6018 by task syz-executor287/8300
> 
> CPU: 0 PID: 8300 Comm: syz-executor287 Not tainted 6.1.0-syzkaller-09661-g13e3c7793e2f #0
> Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/26/2022
> Call Trace:
>  <TASK>
>  __dump_stack lib/dump_stack.c:88 [inline]
>  dump_stack_lvl+0xd1/0x138 lib/dump_stack.c:106
>  print_address_description mm/kasan/report.c:284 [inline]
>  print_report+0x15e/0x45d mm/kasan/report.c:395
>  kasan_report+0xbf/0x1f0 mm/kasan/report.c:495
>  __lock_acquire+0x3ee7/0x56d0 kernel/locking/lockdep.c:4925
>  lock_acquire kernel/locking/lockdep.c:5668 [inline]
>  lock_acquire+0x1e3/0x630 kernel/locking/lockdep.c:5633
>  __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
>  _raw_spin_lock_irqsave+0x3d/0x60 kernel/locking/spinlock.c:162
>  put_pmu_ctx kernel/events/core.c:4913 [inline]
>  put_pmu_ctx+0xad/0x390 kernel/events/core.c:4893
>  _free_event+0x3c5/0x13d0 kernel/events/core.c:5196
>  free_event+0x58/0xc0 kernel/events/core.c:5224
>  __do_sys_perf_event_open+0x66d/0x2980 kernel/events/core.c:12701
>  do_syscall_x64 arch/x86/entry/common.c:50 [inline]
>  do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
>  entry_SYSCALL_64_after_hwframe+0x63/0xcd

Does this help?

diff --git a/kernel/events/core.c b/kernel/events/core.c
index e47914ac8732..bbff551783e1 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -12689,7 +12689,8 @@ SYSCALL_DEFINE5(perf_event_open,
 	return event_fd;
 
 err_context:
-	/* event->pmu_ctx freed by free_event() */
+	put_pmu_ctx(event->pmu_ctx);
+	event->pmu_ctx = NULL; /* _free_event() */
 err_locked:
 	mutex_unlock(&ctx->mutex);
 	perf_unpin_context(ctx);

Re: [syzbot] KASAN: use-after-free Read in put_pmu_ctx

[sdf]

On 12/19, Peter Zijlstra wrote:
> On Mon, Dec 19, 2022 at 12:04:43AM -0800, syzbot wrote:
> > Hello,
> >
> > syzbot found the following issue on:
> >
> > HEAD commit:    13e3c7793e2f Merge tag 'for-netdev' of  
> https://git.kernel...
> > git tree:       bpf
> > console+strace: https://syzkaller.appspot.com/x/log.txt?x=177df7e0480000
> > kernel config:   
> https://syzkaller.appspot.com/x/.config?x=b0e91ad4b5f69c47
> > dashboard link:  
> https://syzkaller.appspot.com/bug?extid=b8e8c01c8ade4fe6e48f
> > compiler:       gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU  
> Binutils for Debian) 2.35.2
> > syz repro:       
> https://syzkaller.appspot.com/x/repro.syz?x=15e87100480000
> > C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=16ceeb13880000
> >
> > Downloadable assets:
> > disk image:  
> https://storage.googleapis.com/syzbot-assets/373a99daa295/disk-13e3c779.raw.xz
> > vmlinux:  
> https://storage.googleapis.com/syzbot-assets/7fa71ed0fe17/vmlinux-13e3c779.xz
> > kernel image:  
> https://storage.googleapis.com/syzbot-assets/2842ad5c698b/bzImage-13e3c779.xz
> >
> > IMPORTANT: if you fix the issue, please add the following tag to the  
> commit:
> > Reported-by: syzbot+b8e8c01c8ade4fe6e48f@syzkaller.appspotmail.com
> >
> > ==================================================================
> > BUG: KASAN: use-after-free in __lock_acquire+0x3ee7/0x56d0  
> kernel/locking/lockdep.c:4925
> > Read of size 8 at addr ffff8880237d6018 by task syz-executor287/8300
> >
> > CPU: 0 PID: 8300 Comm: syz-executor287 Not tainted  
> 6.1.0-syzkaller-09661-g13e3c7793e2f #0
> > Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS  
> Google 10/26/2022
> > Call Trace:
> >  <TASK>
> >  __dump_stack lib/dump_stack.c:88 [inline]
> >  dump_stack_lvl+0xd1/0x138 lib/dump_stack.c:106
> >  print_address_description mm/kasan/report.c:284 [inline]
> >  print_report+0x15e/0x45d mm/kasan/report.c:395
> >  kasan_report+0xbf/0x1f0 mm/kasan/report.c:495
> >  __lock_acquire+0x3ee7/0x56d0 kernel/locking/lockdep.c:4925
> >  lock_acquire kernel/locking/lockdep.c:5668 [inline]
> >  lock_acquire+0x1e3/0x630 kernel/locking/lockdep.c:5633
> >  __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
> >  _raw_spin_lock_irqsave+0x3d/0x60 kernel/locking/spinlock.c:162
> >  put_pmu_ctx kernel/events/core.c:4913 [inline]
> >  put_pmu_ctx+0xad/0x390 kernel/events/core.c:4893
> >  _free_event+0x3c5/0x13d0 kernel/events/core.c:5196
> >  free_event+0x58/0xc0 kernel/events/core.c:5224
> >  __do_sys_perf_event_open+0x66d/0x2980 kernel/events/core.c:12701
> >  do_syscall_x64 arch/x86/entry/common.c:50 [inline]
> >  do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
> >  entry_SYSCALL_64_after_hwframe+0x63/0xcd

> Does this help?

Let's maybe try it this way:

#syz test: https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git  
13e3c7793e2f

diff --git a/kernel/events/core.c b/kernel/events/core.c
index e47914ac8732..bbff551783e1 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -12689,7 +12689,8 @@ SYSCALL_DEFINE5(perf_event_open,
  	return event_fd;

  err_context:
-	/* event->pmu_ctx freed by free_event() */
+	put_pmu_ctx(event->pmu_ctx);
+	event->pmu_ctx = NULL; /* _free_event() */
  err_locked:
  	mutex_unlock(&ctx->mutex);
  	perf_unpin_context(ctx);

Re: [syzbot] KASAN: use-after-free Read in put_pmu_ctx

[syzbot]

Hello,

syzbot tried to test the proposed patch but the build/boot failed:

failed to apply patch:
checking file kernel/events/core.c
patch: **** unexpected end of file in patch



Tested on:

commit:         13e3c779 Merge tag 'for-netdev' of https://git.kernel...
git tree:       https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git
dashboard link: https://syzkaller.appspot.com/bug?extid=b8e8c01c8ade4fe6e48f
compiler:       
patch:          https://syzkaller.appspot.com/x/patch.diff?x=15861a9f880000

Re: [syzbot] KASAN: use-after-free Read in put_pmu_ctx

[sdf]

On 12/19, syzbot wrote:
> Hello,

> syzbot tried to test the proposed patch but the build/boot failed:

> failed to apply patch:
> checking file kernel/events/core.c
> patch: **** unexpected end of file in patch



> Tested on:

> commit:         13e3c779 Merge tag 'for-netdev' of https://git.kernel...
> git tree:        
> https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git
> dashboard link:  
> https://syzkaller.appspot.com/bug?extid=b8e8c01c8ade4fe6e48f
> compiler:
> patch:           
> https://syzkaller.appspot.com/x/patch.diff?x=15861a9f880000


Let's try again with hopefully a better formatted patch..

#syz test: https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git  
13e3c7793e2f

diff --git a/kernel/events/core.c b/kernel/events/core.c
index e47914ac8732..bbff551783e1 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -12689,7 +12689,8 @@ SYSCALL_DEFINE5(perf_event_open,
  	return event_fd;

  err_context:
-	/* event->pmu_ctx freed by free_event() */
+	put_pmu_ctx(event->pmu_ctx);
+	event->pmu_ctx = NULL; /* _free_event() */
  err_locked:
  	mutex_unlock(&ctx->mutex);
  	perf_unpin_context(ctx);

Re: [syzbot] KASAN: use-after-free Read in put_pmu_ctx

[syzbot]

Hello,

syzbot tried to test the proposed patch but the build/boot failed:

failed to apply patch:
checking file kernel/events/core.c
patch: **** unexpected end of file in patch



Tested on:

commit:         13e3c779 Merge tag 'for-netdev' of https://git.kernel...
git tree:       https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git
dashboard link: https://syzkaller.appspot.com/bug?extid=b8e8c01c8ade4fe6e48f
compiler:       
patch:          https://syzkaller.appspot.com/x/patch.diff?x=1451cef0480000

Re: [syzbot] KASAN: use-after-free Read in put_pmu_ctx

[Peter Zijlstra]

On Mon, Dec 19, 2022 at 11:33:29AM -0800, sdf@google.com wrote:
> On 12/19, Peter Zijlstra wrote:
> > On Mon, Dec 19, 2022 at 12:04:43AM -0800, syzbot wrote:

> > > HEAD commit:    13e3c7793e2f Merge tag 'for-netdev' of
> > https://git.kernel...
> > > git tree:       bpf
> > > console+strace: https://syzkaller.appspot.com/x/log.txt?x=177df7e0480000
> > > kernel config:
> > https://syzkaller.appspot.com/x/.config?x=b0e91ad4b5f69c47
> > > dashboard link:
> > https://syzkaller.appspot.com/bug?extid=b8e8c01c8ade4fe6e48f

^ so syzbot knows what tree and config were used to trigger the report,
then why:

> Let's maybe try it this way:
> 
> #syz test: https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git
> 13e3c7793e2f

do you have to repeat that again in order for it to test something?

Re: [syzbot] KASAN: use-after-free Read in put_pmu_ctx

[Peter Zijlstra]

On Mon, Dec 19, 2022 at 12:04:43AM -0800, syzbot wrote:
> Hello,
> 
> syzbot found the following issue on:
> 
> HEAD commit:    13e3c7793e2f Merge tag 'for-netdev' of https://git.kernel...
> git tree:       bpf
> console+strace: https://syzkaller.appspot.com/x/log.txt?x=177df7e0480000
> kernel config:  https://syzkaller.appspot.com/x/.config?x=b0e91ad4b5f69c47
> dashboard link: https://syzkaller.appspot.com/bug?extid=b8e8c01c8ade4fe6e48f
> compiler:       gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2
> syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=15e87100480000
> C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=16ceeb13880000
> 
> Downloadable assets:
> disk image: https://storage.googleapis.com/syzbot-assets/373a99daa295/disk-13e3c779.raw.xz
> vmlinux: https://storage.googleapis.com/syzbot-assets/7fa71ed0fe17/vmlinux-13e3c779.xz
> kernel image: https://storage.googleapis.com/syzbot-assets/2842ad5c698b/bzImage-13e3c779.xz
> 
> IMPORTANT: if you fix the issue, please add the following tag to the commit:
> Reported-by: syzbot+b8e8c01c8ade4fe6e48f@syzkaller.appspotmail.com
> 
> ==================================================================
> BUG: KASAN: use-after-free in __lock_acquire+0x3ee7/0x56d0 kernel/locking/lockdep.c:4925
> Read of size 8 at addr ffff8880237d6018 by task syz-executor287/8300

OK, lemme try this.. still think having to repeat the tree it already
has is daft..

#syz test: https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git 13e3c7793e2f


diff --git a/kernel/events/core.c b/kernel/events/core.c
index eacc3702654d..7da593504c5b 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -12689,7 +12689,8 @@ SYSCALL_DEFINE5(perf_event_open,
 	return event_fd;
 
 err_context:
-	/* event->pmu_ctx freed by free_event() */
+	put_pmu_ctx(event->pmu_ctx);
+	event->pmu_ctx = NULL; /* _free_event() */
 err_locked:
 	mutex_unlock(&ctx->mutex);
 	perf_unpin_context(ctx);

Re: [syzbot] KASAN: use-after-free Read in put_pmu_ctx

[syzbot]

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:
INFO: rcu detected stall in corrupted

rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P5791 } 2673 jiffies s: 2805 root: 0x0/T
rcu: blocking rcu_node structures (internal RCU debug):


Tested on:

commit:         13e3c779 Merge tag 'for-netdev' of https://git.kernel...
git tree:       https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git
console output: https://syzkaller.appspot.com/x/log.txt?x=121ea2e8480000
kernel config:  https://syzkaller.appspot.com/x/.config?x=b0e91ad4b5f69c47
dashboard link: https://syzkaller.appspot.com/bug?extid=b8e8c01c8ade4fe6e48f
compiler:       gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2
patch:          https://syzkaller.appspot.com/x/patch.diff?x=12d4ad4f880000

Re: [syzbot] KASAN: use-after-free Read in put_pmu_ctx

[Peter Zijlstra]

On Tue, Dec 20, 2022 at 12:43:25AM -0800, syzbot wrote:
> Hello,
> 
> syzbot has tested the proposed patch but the reproducer is still triggering an issue:
> INFO: rcu detected stall in corrupted
> 
> rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P5791 } 2673 jiffies s: 2805 root: 0x0/T
> rcu: blocking rcu_node structures (internal RCU debug):

That is an entirely different issue methinks. Let me go write up a
Changelog for that thing and stuff it somewhere /urgent.

Re: [syzbot] KASAN: use-after-free Read in put_pmu_ctx

[Stanislav Fomichev]

[-- Attachment #1: Type: text/plain, Size: 1343 bytes --]

On Tue, Dec 20, 2022 at 12:22 AM Peter Zijlstra <peterz@infradead.org> wrote:
>
> On Mon, Dec 19, 2022 at 11:33:29AM -0800, sdf@google.com wrote:
> > On 12/19, Peter Zijlstra wrote:
> > > On Mon, Dec 19, 2022 at 12:04:43AM -0800, syzbot wrote:
>
> > > > HEAD commit:    13e3c7793e2f Merge tag 'for-netdev' of
> > > https://git.kernel...
> > > > git tree:       bpf
> > > > console+strace: https://syzkaller.appspot.com/x/log.txt?x=177df7e0480000
> > > > kernel config:
> > > https://syzkaller.appspot.com/x/.config?x=b0e91ad4b5f69c47
> > > > dashboard link:
> > > https://syzkaller.appspot.com/bug?extid=b8e8c01c8ade4fe6e48f
>
> ^ so syzbot knows what tree and config were used to trigger the report,
> then why:

I haven't used it before and wasn't sure whether it would take the
last commit from the branch of the one where it failed. Adding them
shouldn't hurt, right?

> > Let's maybe try it this way:
> >
> > #syz test: https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git
> > 13e3c7793e2f
>
> do you have to repeat that again in order for it to test something?

Yeah, I was trying to understand why it doesn't like my patch. It
seems I can retry with the patch attached which hopefully should fix
the possible formatting issue; let's see.

#syz test: https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git
13e3c7793e2f

[-- Attachment #2: pmu.patch --]
[-- Type: text/x-patch, Size: 433 bytes --]

diff --git a/kernel/events/core.c b/kernel/events/core.c
index e47914ac8732..bbff551783e1 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -12689,7 +12689,8 @@ SYSCALL_DEFINE5(perf_event_open,
 	return event_fd;
 
 err_context:
-	/* event->pmu_ctx freed by free_event() */
+	put_pmu_ctx(event->pmu_ctx);
+	event->pmu_ctx = NULL; /* _free_event() */
 err_locked:
 	mutex_unlock(&ctx->mutex);
 	perf_unpin_context(ctx);

Re: [syzbot] KASAN: use-after-free Read in put_pmu_ctx

[syzbot]

Hello,

syzbot has tested the proposed patch but the reproducer is still triggering an issue:
INFO: rcu detected stall in corrupted

rcu: INFO: rcu_preempt detected expedited stalls on CPUs/tasks: { P5778 } 2673 jiffies s: 2773 root: 0x0/T
rcu: blocking rcu_node structures (internal RCU debug):


Tested on:

commit:         13e3c779 Merge tag 'for-netdev' of https://git.kernel...
git tree:       https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git
console output: https://syzkaller.appspot.com/x/log.txt?x=12cb0e5d880000
kernel config:  https://syzkaller.appspot.com/x/.config?x=b0e91ad4b5f69c47
dashboard link: https://syzkaller.appspot.com/bug?extid=b8e8c01c8ade4fe6e48f
compiler:       gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2
patch:          https://syzkaller.appspot.com/x/patch.diff?x=12df6a9b880000

Re: [syzbot] KASAN: use-after-free Read in put_pmu_ctx

[Chengming Zhou]

On 2022/12/19 22:40, Peter Zijlstra wrote:
> On Mon, Dec 19, 2022 at 12:04:43AM -0800, syzbot wrote:
>> Hello,
>>
>> syzbot found the following issue on:
>>
>> HEAD commit:    13e3c7793e2f Merge tag 'for-netdev' of https://git.kernel...
>> git tree:       bpf
>> console+strace: https://syzkaller.appspot.com/x/log.txt?x=177df7e0480000
>> kernel config:  https://syzkaller.appspot.com/x/.config?x=b0e91ad4b5f69c47
>> dashboard link: https://syzkaller.appspot.com/bug?extid=b8e8c01c8ade4fe6e48f
>> compiler:       gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2
>> syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=15e87100480000
>> C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=16ceeb13880000
>>
>> Downloadable assets:
>> disk image: https://storage.googleapis.com/syzbot-assets/373a99daa295/disk-13e3c779.raw.xz
>> vmlinux: https://storage.googleapis.com/syzbot-assets/7fa71ed0fe17/vmlinux-13e3c779.xz
>> kernel image: https://storage.googleapis.com/syzbot-assets/2842ad5c698b/bzImage-13e3c779.xz
>>
>> IMPORTANT: if you fix the issue, please add the following tag to the commit:
>> Reported-by: syzbot+b8e8c01c8ade4fe6e48f@syzkaller.appspotmail.com
>>
>> ==================================================================
>> BUG: KASAN: use-after-free in __lock_acquire+0x3ee7/0x56d0 kernel/locking/lockdep.c:4925
>> Read of size 8 at addr ffff8880237d6018 by task syz-executor287/8300
>>
>> CPU: 0 PID: 8300 Comm: syz-executor287 Not tainted 6.1.0-syzkaller-09661-g13e3c7793e2f #0
>> Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/26/2022
>> Call Trace:
>>  <TASK>
>>  __dump_stack lib/dump_stack.c:88 [inline]
>>  dump_stack_lvl+0xd1/0x138 lib/dump_stack.c:106
>>  print_address_description mm/kasan/report.c:284 [inline]
>>  print_report+0x15e/0x45d mm/kasan/report.c:395
>>  kasan_report+0xbf/0x1f0 mm/kasan/report.c:495
>>  __lock_acquire+0x3ee7/0x56d0 kernel/locking/lockdep.c:4925
>>  lock_acquire kernel/locking/lockdep.c:5668 [inline]
>>  lock_acquire+0x1e3/0x630 kernel/locking/lockdep.c:5633
>>  __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
>>  _raw_spin_lock_irqsave+0x3d/0x60 kernel/locking/spinlock.c:162
>>  put_pmu_ctx kernel/events/core.c:4913 [inline]
>>  put_pmu_ctx+0xad/0x390 kernel/events/core.c:4893
>>  _free_event+0x3c5/0x13d0 kernel/events/core.c:5196
>>  free_event+0x58/0xc0 kernel/events/core.c:5224
>>  __do_sys_perf_event_open+0x66d/0x2980 kernel/events/core.c:12701
>>  do_syscall_x64 arch/x86/entry/common.c:50 [inline]
>>  do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80
>>  entry_SYSCALL_64_after_hwframe+0x63/0xcd
> 
> Does this help?
> 
> diff --git a/kernel/events/core.c b/kernel/events/core.c
> index e47914ac8732..bbff551783e1 100644
> --- a/kernel/events/core.c
> +++ b/kernel/events/core.c
> @@ -12689,7 +12689,8 @@ SYSCALL_DEFINE5(perf_event_open,
>  	return event_fd;
>  
>  err_context:
> -	/* event->pmu_ctx freed by free_event() */
> +	put_pmu_ctx(event->pmu_ctx);
> +	event->pmu_ctx = NULL; /* _free_event() */
>  err_locked:
>  	mutex_unlock(&ctx->mutex);
>  	perf_unpin_context(ctx);

Tested-by: Chengming Zhou <zhouchengming@bytedance.com>

While reviewing the code, I found perf_event_create_kernel_counter()
has the similar problem in the "err_pmu_ctx" error handling path:

CPU0					CPU1
perf_event_create_kernel_counter()
  // inc ctx refcnt
  find_get_context(task, event) (1)

  // inc pmu_ctx refcnt
  pmu_ctx = find_get_pmu_context()

  event->pmu_ctx = pmu_ctx
  ...
  goto err_pmu_ctx:
    // dec pmu_ctx refcnt
    put_pmu_ctx(pmu_ctx) (2)

    mutex_unlock(&ctx->mutex)
    // dec ctx refcnt
    put_ctx(ctx)
					perf_event_exit_task_context()
					  mutex_lock()
					  mutex_unlock()
					  // last refcnt put
					  put_ctx()
    free_event(event)
      if (event->pmu_ctx) // True
        put_pmu_ctx() (3)
          // will access freed pmu_ctx or ctx

      if (event->ctx) // False
        put_ctx()

(3) has UAF problem since the pmu_ctx maybe freed in (2), so also
should have "event->pmu_ctx = NULL;" in (2).

Re: [syzbot] KASAN: use-after-free Read in put_pmu_ctx

[Peter Zijlstra]

On Wed, Dec 21, 2022 at 10:42:39AM +0800, Chengming Zhou wrote:

> > Does this help?
> > 
> > diff --git a/kernel/events/core.c b/kernel/events/core.c
> > index e47914ac8732..bbff551783e1 100644
> > --- a/kernel/events/core.c
> > +++ b/kernel/events/core.c
> > @@ -12689,7 +12689,8 @@ SYSCALL_DEFINE5(perf_event_open,
> >  	return event_fd;
> >  
> >  err_context:
> > -	/* event->pmu_ctx freed by free_event() */
> > +	put_pmu_ctx(event->pmu_ctx);
> > +	event->pmu_ctx = NULL; /* _free_event() */
> >  err_locked:
> >  	mutex_unlock(&ctx->mutex);
> >  	perf_unpin_context(ctx);
> 
> Tested-by: Chengming Zhou <zhouchengming@bytedance.com>
> 
> While reviewing the code, I found perf_event_create_kernel_counter()
> has the similar problem in the "err_pmu_ctx" error handling path:

Right you are, updated the patch, thanks!

> CPU0					CPU1
> perf_event_create_kernel_counter()
>   // inc ctx refcnt
>   find_get_context(task, event) (1)
> 
>   // inc pmu_ctx refcnt
>   pmu_ctx = find_get_pmu_context()
> 
>   event->pmu_ctx = pmu_ctx
>   ...
>   goto err_pmu_ctx:
>     // dec pmu_ctx refcnt
>     put_pmu_ctx(pmu_ctx) (2)
> 
>     mutex_unlock(&ctx->mutex)
>     // dec ctx refcnt
>     put_ctx(ctx)
> 					perf_event_exit_task_context()
> 					  mutex_lock()
> 					  mutex_unlock()
> 					  // last refcnt put
> 					  put_ctx()
>     free_event(event)
>       if (event->pmu_ctx) // True
>         put_pmu_ctx() (3)
>           // will access freed pmu_ctx or ctx
> 
>       if (event->ctx) // False
>         put_ctx()

This doesn't look right; iirc you can hit this without concurrency,
something like so:


	// note that when getting here, we've not passed
	// perf_install_in_context() and event->ctx == NULL.
err_pmu_ctx:
	put_pmu_ctx();
	put_ctx(); // last, actually frees ctx
	..
err_alloc:
	free_event()
	  _free_event()
	    if (event->pmu_ctx) // true, because we forgot to clear
	      put_pmu_ctx() // hits 0 because double put
	        // goes and touch epc->ctx and UaF

Re: [syzbot] KASAN: use-after-free Read in put_pmu_ctx

[Chengming Zhou]

On 2022/12/23 04:59, Peter Zijlstra wrote:
> On Wed, Dec 21, 2022 at 10:42:39AM +0800, Chengming Zhou wrote:
> 
>>> Does this help?
>>>
>>> diff --git a/kernel/events/core.c b/kernel/events/core.c
>>> index e47914ac8732..bbff551783e1 100644
>>> --- a/kernel/events/core.c
>>> +++ b/kernel/events/core.c
>>> @@ -12689,7 +12689,8 @@ SYSCALL_DEFINE5(perf_event_open,
>>>  	return event_fd;
>>>  
>>>  err_context:
>>> -	/* event->pmu_ctx freed by free_event() */
>>> +	put_pmu_ctx(event->pmu_ctx);
>>> +	event->pmu_ctx = NULL; /* _free_event() */
>>>  err_locked:
>>>  	mutex_unlock(&ctx->mutex);
>>>  	perf_unpin_context(ctx);
>>
>> Tested-by: Chengming Zhou <zhouchengming@bytedance.com>
>>
>> While reviewing the code, I found perf_event_create_kernel_counter()
>> has the similar problem in the "err_pmu_ctx" error handling path:
> 
> Right you are, updated the patch, thanks!
> 
>> CPU0					CPU1
>> perf_event_create_kernel_counter()
>>   // inc ctx refcnt
>>   find_get_context(task, event) (1)
>>
>>   // inc pmu_ctx refcnt
>>   pmu_ctx = find_get_pmu_context()
>>
>>   event->pmu_ctx = pmu_ctx
>>   ...
>>   goto err_pmu_ctx:
>>     // dec pmu_ctx refcnt
>>     put_pmu_ctx(pmu_ctx) (2)
>>
>>     mutex_unlock(&ctx->mutex)
>>     // dec ctx refcnt
>>     put_ctx(ctx)
>> 					perf_event_exit_task_context()
>> 					  mutex_lock()
>> 					  mutex_unlock()
>> 					  // last refcnt put
>> 					  put_ctx()
>>     free_event(event)
>>       if (event->pmu_ctx) // True
>>         put_pmu_ctx() (3)
>>           // will access freed pmu_ctx or ctx
>>
>>       if (event->ctx) // False
>>         put_ctx()
> 
> This doesn't look right; iirc you can hit this without concurrency,
> something like so:

Right, pmu_ctx UaF can hit without concurrency.

But ctx has been created with refcnt == 1, which referenced by the task,
so the last refcnt put must be in perf_event_exit_task_context().

Maybe we can improve this, don't let ctx referenced by the task? Then ctx
can be freed when all perf_events are removed, instead of having to wait
for the task to exit. Maybe I missed something...

> 
> 
> 	// note that when getting here, we've not passed
> 	// perf_install_in_context() and event->ctx == NULL.
> err_pmu_ctx:
> 	put_pmu_ctx();
> 	put_ctx(); // last, actually frees ctx

This put_ctx() dec refcnt from 2 to 1, perf_event_exit_task_context()
will put the last refcnt and free it.

> 	..
> err_alloc:
> 	free_event()
> 	  _free_event()
> 	    if (event->pmu_ctx) // true, because we forgot to clear
> 	      put_pmu_ctx() // hits 0 because double put
> 	        // goes and touch epc->ctx and UaF
> 
>