[syzbot] [bpf?] UBSAN: array-index-out-of-bounds in check_stack_range_initialized

[syzbot] [bpf?] UBSAN: array-index-out-of-bounds in check_stack_range_initialized

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit:    0740b6427e90 Merge branch 'bpf-arena-followups'
git tree:       bpf
console+strace: https://syzkaller.appspot.com/x/log.txt?x=12fed769180000
kernel config:  https://syzkaller.appspot.com/x/.config?x=6fb1be60a193d440
dashboard link: https://syzkaller.appspot.com/bug?extid=33f4297b5f927648741a
compiler:       Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=1763a479180000
C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=15c38711180000

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/c9e6e9f97566/disk-0740b642.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/78476a588b62/vmlinux-0740b642.xz
kernel image: https://storage.googleapis.com/syzbot-assets/50cd6fab9ead/bzImage-0740b642.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+33f4297b5f927648741a@syzkaller.appspotmail.com

------------[ cut here ]------------
UBSAN: array-index-out-of-bounds in kernel/bpf/verifier.c:7190:12
index -1 is out of range for type 'u8[8]' (aka 'unsigned char[8]')
CPU: 0 PID: 5071 Comm: syz-executor474 Not tainted 6.8.0-syzkaller-05226-g0740b6427e90 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 02/29/2024
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x1e7/0x2e0 lib/dump_stack.c:106
 ubsan_epilogue lib/ubsan.c:217 [inline]
 __ubsan_handle_out_of_bounds+0x121/0x150 lib/ubsan.c:415
 check_stack_range_initialized+0x1668/0x19a0 kernel/bpf/verifier.c:7190
 check_helper_mem_access+0x2eb/0xfa0 kernel/bpf/verifier.c:7294
 check_helper_call+0x263c/0x7220 kernel/bpf/verifier.c:10252
 do_check+0x9e29/0x10530 kernel/bpf/verifier.c:17801
 do_check_common+0x14bd/0x1dd0 kernel/bpf/verifier.c:20500
 do_check_main kernel/bpf/verifier.c:20591 [inline]
 bpf_check+0x136ab/0x19010 kernel/bpf/verifier.c:21261
 bpf_prog_load+0x1667/0x20f0 kernel/bpf/syscall.c:2895
 __sys_bpf+0x4ee/0x810 kernel/bpf/syscall.c:5631
 __do_sys_bpf kernel/bpf/syscall.c:5738 [inline]
 __se_sys_bpf kernel/bpf/syscall.c:5736 [inline]
 __x64_sys_bpf+0x7c/0x90 kernel/bpf/syscall.c:5736
 do_syscall_64+0xfb/0x240
 entry_SYSCALL_64_after_hwframe+0x6d/0x75
RIP: 0033:0x7f8416194629
Code: 48 83 c4 28 c3 e8 37 17 00 00 0f 1f 80 00 00 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007ffdc6f0fdb8 EFLAGS: 00000246 ORIG_RAX: 0000000000000141
RAX: ffffffffffffffda RBX: 00007ffdc6f0ff88 RCX: 00007f8416194629
RDX: 0000000000000090 RSI: 00000000200000c0 RDI: 0000000000000005
RBP: 00007f8416207610 R08: 0000000000000000 R09: 00007ffdc6f0ff88
R10: 00000000fffffff8 R11: 0000000000000246 R12: 0000000000000001
R13: 00007ffdc6f0ff78 R14: 0000000000000001 R15: 0000000000000001
 </TASK>
---[ end trace ]---


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzkaller@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

If the report is already addressed, let syzbot know by replying with:
#syz fix: exact-commit-title

If you want syzbot to run the reproducer, reply with:
#syz test: git://repo/address.git branch-or-commit-hash
If you attach or paste a git patch, syzbot will apply it before testing.

If you want to overwrite report's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)

If the report is a duplicate of another one, reply with:
#syz dup: exact-subject-of-another-report

If you want to undo deduplication, reply with:
#syz undup

stack access issue. Re: [syzbot] [bpf?] UBSAN: array-index-out-of-bounds in check_stack_range_initialized

[Alexei Starovoitov]

Hi Andrei,

looks like the refactoring of stack access introduced a bug.
See the reproducer below.
positive offsets are not caught by check_stack_access_within_bounds().
So both slot and spi become negative and access
stack[spi].slot_type[slot % BPF_REG_SIZE]
returns garbage.

On Tue, Mar 19, 2024 at 11:12 AM syzbot
<syzbot+33f4297b5f927648741a@syzkaller.appspotmail.com> wrote:
>
> Hello,
>
> syzbot found the following issue on:
>
> HEAD commit:    0740b6427e90 Merge branch 'bpf-arena-followups'
> git tree:       bpf
> console+strace: https://syzkaller.appspot.com/x/log.txt?x=12fed769180000
> kernel config:  https://syzkaller.appspot.com/x/.config?x=6fb1be60a193d440
> dashboard link: https://syzkaller.appspot.com/bug?extid=33f4297b5f927648741a
> compiler:       Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
> syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=1763a479180000
> C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=15c38711180000
>
> Downloadable assets:
> disk image: https://storage.googleapis.com/syzbot-assets/c9e6e9f97566/disk-0740b642.raw.xz
> vmlinux: https://storage.googleapis.com/syzbot-assets/78476a588b62/vmlinux-0740b642.xz
> kernel image: https://storage.googleapis.com/syzbot-assets/50cd6fab9ead/bzImage-0740b642.xz
>
> IMPORTANT: if you fix the issue, please add the following tag to the commit:
> Reported-by: syzbot+33f4297b5f927648741a@syzkaller.appspotmail.com
>
> ------------[ cut here ]------------
> UBSAN: array-index-out-of-bounds in kernel/bpf/verifier.c:7190:12
> index -1 is out of range for type 'u8[8]' (aka 'unsigned char[8]')
> CPU: 0 PID: 5071 Comm: syz-executor474 Not tainted 6.8.0-syzkaller-05226-g0740b6427e90 #0
> Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 02/29/2024
> Call Trace:
>  <TASK>
>  __dump_stack lib/dump_stack.c:88 [inline]
>  dump_stack_lvl+0x1e7/0x2e0 lib/dump_stack.c:106
>  ubsan_epilogue lib/ubsan.c:217 [inline]
>  __ubsan_handle_out_of_bounds+0x121/0x150 lib/ubsan.c:415
>  check_stack_range_initialized+0x1668/0x19a0 kernel/bpf/verifier.c:7190
>  check_helper_mem_access+0x2eb/0xfa0 kernel/bpf/verifier.c:7294
>  check_helper_call+0x263c/0x7220 kernel/bpf/verifier.c:10252
>  do_check+0x9e29/0x10530 kernel/bpf/verifier.c:17801
>  do_check_common+0x14bd/0x1dd0 kernel/bpf/verifier.c:20500
>  do_check_main kernel/bpf/verifier.c:20591 [inline]
>  bpf_check+0x136ab/0x19010 kernel/bpf/verifier.c:21261
>  bpf_prog_load+0x1667/0x20f0 kernel/bpf/syscall.c:2895
>  __sys_bpf+0x4ee/0x810 kernel/bpf/syscall.c:5631
>  __do_sys_bpf kernel/bpf/syscall.c:5738 [inline]
>  __se_sys_bpf kernel/bpf/syscall.c:5736 [inline]
>  __x64_sys_bpf+0x7c/0x90 kernel/bpf/syscall.c:5736
>  do_syscall_64+0xfb/0x240
>  entry_SYSCALL_64_after_hwframe+0x6d/0x75
> RIP: 0033:0x7f8416194629
> Code: 48 83 c4 28 c3 e8 37 17 00 00 0f 1f 80 00 00 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48
> RSP: 002b:00007ffdc6f0fdb8 EFLAGS: 00000246 ORIG_RAX: 0000000000000141
> RAX: ffffffffffffffda RBX: 00007ffdc6f0ff88 RCX: 00007f8416194629
> RDX: 0000000000000090 RSI: 00000000200000c0 RDI: 0000000000000005
> RBP: 00007f8416207610 R08: 0000000000000000 R09: 00007ffdc6f0ff88
> R10: 00000000fffffff8 R11: 0000000000000246 R12: 0000000000000001
> R13: 00007ffdc6f0ff78 R14: 0000000000000001 R15: 0000000000000001
>  </TASK>
> ---[ end trace ]---
>
>
> ---
> This report is generated by a bot. It may contain errors.
> See https://goo.gl/tpsmEJ for more information about syzbot.
> syzbot engineers can be reached at syzkaller@googlegroups.com.
>
> syzbot will keep track of this issue. See:
> https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
>
> If the report is already addressed, let syzbot know by replying with:
> #syz fix: exact-commit-title
>
> If you want syzbot to run the reproducer, reply with:
> #syz test: git://repo/address.git branch-or-commit-hash
> If you attach or paste a git patch, syzbot will apply it before testing.
>
> If you want to overwrite report's subsystems, reply with:
> #syz set subsystems: new-subsystem
> (See the list of subsystem names on the web dashboard)
>
> If the report is a duplicate of another one, reply with:
> #syz dup: exact-subject-of-another-report
>
> If you want to undo deduplication, reply with:
> #syz undup

Re: stack access issue. Re: [syzbot] [bpf?] UBSAN: array-index-out-of-bounds in check_stack_range_initialized

[Andrei Matei]

Thanks for the report! Will look in a bit.

On Thu, Mar 21, 2024 at 3:33 AM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> Hi Andrei,
>
> looks like the refactoring of stack access introduced a bug.
> See the reproducer below.
> positive offsets are not caught by check_stack_access_within_bounds().
> So both slot and spi become negative and access
> stack[spi].slot_type[slot % BPF_REG_SIZE]
> returns garbage.
>
> On Tue, Mar 19, 2024 at 11:12 AM syzbot
> <syzbot+33f4297b5f927648741a@syzkaller.appspotmail.com> wrote:
> >
> > Hello,
> >
> > syzbot found the following issue on:
> >
> > HEAD commit:    0740b6427e90 Merge branch 'bpf-arena-followups'
> > git tree:       bpf
> > console+strace: https://syzkaller.appspot.com/x/log.txt?x=12fed769180000
> > kernel config:  https://syzkaller.appspot.com/x/.config?x=6fb1be60a193d440
> > dashboard link: https://syzkaller.appspot.com/bug?extid=33f4297b5f927648741a
> > compiler:       Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
> > syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=1763a479180000
> > C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=15c38711180000
> >
> > Downloadable assets:
> > disk image: https://storage.googleapis.com/syzbot-assets/c9e6e9f97566/disk-0740b642.raw.xz
> > vmlinux: https://storage.googleapis.com/syzbot-assets/78476a588b62/vmlinux-0740b642.xz
> > kernel image: https://storage.googleapis.com/syzbot-assets/50cd6fab9ead/bzImage-0740b642.xz
> >
> > IMPORTANT: if you fix the issue, please add the following tag to the commit:
> > Reported-by: syzbot+33f4297b5f927648741a@syzkaller.appspotmail.com
> >
> > ------------[ cut here ]------------
> > UBSAN: array-index-out-of-bounds in kernel/bpf/verifier.c:7190:12
> > index -1 is out of range for type 'u8[8]' (aka 'unsigned char[8]')
> > CPU: 0 PID: 5071 Comm: syz-executor474 Not tainted 6.8.0-syzkaller-05226-g0740b6427e90 #0
> > Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 02/29/2024
> > Call Trace:
> >  <TASK>
> >  __dump_stack lib/dump_stack.c:88 [inline]
> >  dump_stack_lvl+0x1e7/0x2e0 lib/dump_stack.c:106
> >  ubsan_epilogue lib/ubsan.c:217 [inline]
> >  __ubsan_handle_out_of_bounds+0x121/0x150 lib/ubsan.c:415
> >  check_stack_range_initialized+0x1668/0x19a0 kernel/bpf/verifier.c:7190
> >  check_helper_mem_access+0x2eb/0xfa0 kernel/bpf/verifier.c:7294
> >  check_helper_call+0x263c/0x7220 kernel/bpf/verifier.c:10252
> >  do_check+0x9e29/0x10530 kernel/bpf/verifier.c:17801
> >  do_check_common+0x14bd/0x1dd0 kernel/bpf/verifier.c:20500
> >  do_check_main kernel/bpf/verifier.c:20591 [inline]
> >  bpf_check+0x136ab/0x19010 kernel/bpf/verifier.c:21261
> >  bpf_prog_load+0x1667/0x20f0 kernel/bpf/syscall.c:2895
> >  __sys_bpf+0x4ee/0x810 kernel/bpf/syscall.c:5631
> >  __do_sys_bpf kernel/bpf/syscall.c:5738 [inline]
> >  __se_sys_bpf kernel/bpf/syscall.c:5736 [inline]
> >  __x64_sys_bpf+0x7c/0x90 kernel/bpf/syscall.c:5736
> >  do_syscall_64+0xfb/0x240
> >  entry_SYSCALL_64_after_hwframe+0x6d/0x75
> > RIP: 0033:0x7f8416194629
> > Code: 48 83 c4 28 c3 e8 37 17 00 00 0f 1f 80 00 00 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48
> > RSP: 002b:00007ffdc6f0fdb8 EFLAGS: 00000246 ORIG_RAX: 0000000000000141
> > RAX: ffffffffffffffda RBX: 00007ffdc6f0ff88 RCX: 00007f8416194629
> > RDX: 0000000000000090 RSI: 00000000200000c0 RDI: 0000000000000005
> > RBP: 00007f8416207610 R08: 0000000000000000 R09: 00007ffdc6f0ff88
> > R10: 00000000fffffff8 R11: 0000000000000246 R12: 0000000000000001
> > R13: 00007ffdc6f0ff78 R14: 0000000000000001 R15: 0000000000000001
> >  </TASK>
> > ---[ end trace ]---
> >
> >
> > ---
> > This report is generated by a bot. It may contain errors.
> > See https://goo.gl/tpsmEJ for more information about syzbot.
> > syzbot engineers can be reached at syzkaller@googlegroups.com.
> >
> > syzbot will keep track of this issue. See:
> > https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
> >
> > If the report is already addressed, let syzbot know by replying with:
> > #syz fix: exact-commit-title
> >
> > If you want syzbot to run the reproducer, reply with:
> > #syz test: git://repo/address.git branch-or-commit-hash
> > If you attach or paste a git patch, syzbot will apply it before testing.
> >
> > If you want to overwrite report's subsystems, reply with:
> > #syz set subsystems: new-subsystem
> > (See the list of subsystem names on the web dashboard)
> >
> > If the report is a duplicate of another one, reply with:
> > #syz dup: exact-subject-of-another-report
> >
> > If you want to undo deduplication, reply with:
> > #syz undup

Re: stack access issue. Re: [syzbot] [bpf?] UBSAN: array-index-out-of-bounds in check_stack_range_initialized

[Andrei Matei]

+ Edward

On Thu, Mar 21, 2024 at 3:33 AM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> Hi Andrei,
>
> looks like the refactoring of stack access introduced a bug.
> See the reproducer below.
> positive offsets are not caught by check_stack_access_within_bounds().

check_stack_access_within_bounds() tries to catch positive offsets;
It does: [1]

err = check_stack_slot_within_bounds(env, min_off, state, type);
if (!err && max_off > 0)
  err = -EINVAL; /* out of stack access into non-negative offsets */

Notice the max_off > 0 in there.
And we have various tests that seem to check that positive offsets are
rejected. Do you know what the bug is?
I'm thinking maybe there's some overflow going on, except that UBSAN
reported an index of -1 as being the problem.

Edward, I see that you've been tickling the robot trying to narrow the issue;
perhaps you've figured it out?

If the bug is not immediately apparent to anyone, I would really appreciate a
bit of tutoring around how to reproduce and get verifier logs. I have tried a
bunch of cases of constant- and variable-offset accesses, and couldn't repro. I
can run syzkaller's repro on its own vm image, and indeed it crashes. But I'm
not sure how to get verifier logs out of the C reproducer.  Alternatively, I'm
not sure how to figure out the actual BPF program corresponding to the "syz
repro" in [2] and turn it into a test_progs test. How do you guys do it?

Thanks a lot!

[1] https://github.com/torvalds/linux/blob/70293240c5ce675a67bfc48f419b093023b862b3/kernel/bpf/verifier.c#L6695
[2] https://syzkaller.appspot.com/x/repro.syz?x=1763a479180000


> So both slot and spi become negative and access
> stack[spi].slot_type[slot % BPF_REG_SIZE]
> returns garbage.
>
> On Tue, Mar 19, 2024 at 11:12 AM syzbot
> <syzbot+33f4297b5f927648741a@syzkaller.appspotmail.com> wrote:
> >
> > Hello,
> >
> > syzbot found the following issue on:
> >
> > HEAD commit:    0740b6427e90 Merge branch 'bpf-arena-followups'
> > git tree:       bpf
> > console+strace: https://syzkaller.appspot.com/x/log.txt?x=12fed769180000
> > kernel config:  https://syzkaller.appspot.com/x/.config?x=6fb1be60a193d440
> > dashboard link: https://syzkaller.appspot.com/bug?extid=33f4297b5f927648741a
> > compiler:       Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
> > syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=1763a479180000
> > C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=15c38711180000
> >
> > Downloadable assets:
> > disk image: https://storage.googleapis.com/syzbot-assets/c9e6e9f97566/disk-0740b642.raw.xz
> > vmlinux: https://storage.googleapis.com/syzbot-assets/78476a588b62/vmlinux-0740b642.xz
> > kernel image: https://storage.googleapis.com/syzbot-assets/50cd6fab9ead/bzImage-0740b642.xz
> >
> > IMPORTANT: if you fix the issue, please add the following tag to the commit:
> > Reported-by: syzbot+33f4297b5f927648741a@syzkaller.appspotmail.com
> >
> > ------------[ cut here ]------------
> > UBSAN: array-index-out-of-bounds in kernel/bpf/verifier.c:7190:12
> > index -1 is out of range for type 'u8[8]' (aka 'unsigned char[8]')
> > CPU: 0 PID: 5071 Comm: syz-executor474 Not tainted 6.8.0-syzkaller-05226-g0740b6427e90 #0
> > Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 02/29/2024
> > Call Trace:
> >  <TASK>
> >  __dump_stack lib/dump_stack.c:88 [inline]
> >  dump_stack_lvl+0x1e7/0x2e0 lib/dump_stack.c:106
> >  ubsan_epilogue lib/ubsan.c:217 [inline]
> >  __ubsan_handle_out_of_bounds+0x121/0x150 lib/ubsan.c:415
> >  check_stack_range_initialized+0x1668/0x19a0 kernel/bpf/verifier.c:7190
> >  check_helper_mem_access+0x2eb/0xfa0 kernel/bpf/verifier.c:7294
> >  check_helper_call+0x263c/0x7220 kernel/bpf/verifier.c:10252
> >  do_check+0x9e29/0x10530 kernel/bpf/verifier.c:17801
> >  do_check_common+0x14bd/0x1dd0 kernel/bpf/verifier.c:20500
> >  do_check_main kernel/bpf/verifier.c:20591 [inline]
> >  bpf_check+0x136ab/0x19010 kernel/bpf/verifier.c:21261
> >  bpf_prog_load+0x1667/0x20f0 kernel/bpf/syscall.c:2895
> >  __sys_bpf+0x4ee/0x810 kernel/bpf/syscall.c:5631
> >  __do_sys_bpf kernel/bpf/syscall.c:5738 [inline]
> >  __se_sys_bpf kernel/bpf/syscall.c:5736 [inline]
> >  __x64_sys_bpf+0x7c/0x90 kernel/bpf/syscall.c:5736
> >  do_syscall_64+0xfb/0x240
> >  entry_SYSCALL_64_after_hwframe+0x6d/0x75
> > RIP: 0033:0x7f8416194629
> > Code: 48 83 c4 28 c3 e8 37 17 00 00 0f 1f 80 00 00 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48
> > RSP: 002b:00007ffdc6f0fdb8 EFLAGS: 00000246 ORIG_RAX: 0000000000000141
> > RAX: ffffffffffffffda RBX: 00007ffdc6f0ff88 RCX: 00007f8416194629
> > RDX: 0000000000000090 RSI: 00000000200000c0 RDI: 0000000000000005
> > RBP: 00007f8416207610 R08: 0000000000000000 R09: 00007ffdc6f0ff88
> > R10: 00000000fffffff8 R11: 0000000000000246 R12: 0000000000000001
> > R13: 00007ffdc6f0ff78 R14: 0000000000000001 R15: 0000000000000001
> >  </TASK>
> > ---[ end trace ]---
> >
> >
> > ---
> > This report is generated by a bot. It may contain errors.
> > See https://goo.gl/tpsmEJ for more information about syzbot.
> > syzbot engineers can be reached at syzkaller@googlegroups.com.
> >
> > syzbot will keep track of this issue. See:
> > https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
> >
> > If the report is already addressed, let syzbot know by replying with:
> > #syz fix: exact-commit-title
> >
> > If you want syzbot to run the reproducer, reply with:
> > #syz test: git://repo/address.git branch-or-commit-hash
> > If you attach or paste a git patch, syzbot will apply it before testing.
> >
> > If you want to overwrite report's subsystems, reply with:
> > #syz set subsystems: new-subsystem
> > (See the list of subsystem names on the web dashboard)
> >
> > If the report is a duplicate of another one, reply with:
> > #syz dup: exact-subject-of-another-report
> >
> > If you want to undo deduplication, reply with:
> > #syz undup

Re: stack access issue. Re: [syzbot] [bpf?] UBSAN: array-index-out-of-bounds in check_stack_range_initialized

[Alexei Starovoitov]

On Sat, Mar 23, 2024 at 5:50 PM Andrei Matei <andreimatei1@gmail.com> wrote:
>
> + Edward
>
> On Thu, Mar 21, 2024 at 3:33 AM Alexei Starovoitov
> <alexei.starovoitov@gmail.com> wrote:
> >
> > Hi Andrei,
> >
> > looks like the refactoring of stack access introduced a bug.
> > See the reproducer below.
> > positive offsets are not caught by check_stack_access_within_bounds().
>
> check_stack_access_within_bounds() tries to catch positive offsets;
> It does: [1]
>
> err = check_stack_slot_within_bounds(env, min_off, state, type);
> if (!err && max_off > 0)
>   err = -EINVAL; /* out of stack access into non-negative offsets */
>
> Notice the max_off > 0 in there.
> And we have various tests that seem to check that positive offsets are
> rejected. Do you know what the bug is?
> I'm thinking maybe there's some overflow going on, except that UBSAN
> reported an index of -1 as being the problem.
>
> Edward, I see that you've been tickling the robot trying to narrow the issue;
> perhaps you've figured it out?
>
> If the bug is not immediately apparent to anyone, I would really appreciate a
> bit of tutoring around how to reproduce and get verifier logs.

The repro is right there in the email I forwarded:

> C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=15c38711180000

Re: stack access issue. Re: [syzbot] [bpf?] UBSAN: array-index-out-of-bounds in check_stack_range_initialized

[Andrei Matei]

On Sat, Mar 23, 2024 at 8:52 PM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Sat, Mar 23, 2024 at 5:50 PM Andrei Matei <andreimatei1@gmail.com> wrote:
> >
> > + Edward
> >
> > On Thu, Mar 21, 2024 at 3:33 AM Alexei Starovoitov
> > <alexei.starovoitov@gmail.com> wrote:
> > >
> > > Hi Andrei,
> > >
> > > looks like the refactoring of stack access introduced a bug.
> > > See the reproducer below.
> > > positive offsets are not caught by check_stack_access_within_bounds().
> >
> > check_stack_access_within_bounds() tries to catch positive offsets;
> > It does: [1]
> >
> > err = check_stack_slot_within_bounds(env, min_off, state, type);
> > if (!err && max_off > 0)
> >   err = -EINVAL; /* out of stack access into non-negative offsets */
> >
> > Notice the max_off > 0 in there.
> > And we have various tests that seem to check that positive offsets are
> > rejected. Do you know what the bug is?
> > I'm thinking maybe there's some overflow going on, except that UBSAN
> > reported an index of -1 as being the problem.
> >
> > Edward, I see that you've been tickling the robot trying to narrow the issue;
> > perhaps you've figured it out?
> >
> > If the bug is not immediately apparent to anyone, I would really appreciate a
> > bit of tutoring around how to reproduce and get verifier logs.
>
> The repro is right there in the email I forwarded:
>
> > C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=15c38711180000

I understand, but how does one go from this to either BPF assembly,
or to running it in such a way that you also get verifier logs?

Re: stack access issue. Re: [syzbot] [bpf?] UBSAN: array-index-out-of-bounds in check_stack_range_initialized

[Alexei Starovoitov]

On Sat, Mar 23, 2024 at 7:12 PM Andrei Matei <andreimatei1@gmail.com> wrote:
>
> On Sat, Mar 23, 2024 at 8:52 PM Alexei Starovoitov
> <alexei.starovoitov@gmail.com> wrote:
> >
> > On Sat, Mar 23, 2024 at 5:50 PM Andrei Matei <andreimatei1@gmail.com> wrote:
> > >
> > > + Edward
> > >
> > > On Thu, Mar 21, 2024 at 3:33 AM Alexei Starovoitov
> > > <alexei.starovoitov@gmail.com> wrote:
> > > >
> > > > Hi Andrei,
> > > >
> > > > looks like the refactoring of stack access introduced a bug.
> > > > See the reproducer below.
> > > > positive offsets are not caught by check_stack_access_within_bounds().
> > >
> > > check_stack_access_within_bounds() tries to catch positive offsets;
> > > It does: [1]
> > >
> > > err = check_stack_slot_within_bounds(env, min_off, state, type);
> > > if (!err && max_off > 0)
> > >   err = -EINVAL; /* out of stack access into non-negative offsets */
> > >
> > > Notice the max_off > 0 in there.
> > > And we have various tests that seem to check that positive offsets are
> > > rejected. Do you know what the bug is?
> > > I'm thinking maybe there's some overflow going on, except that UBSAN
> > > reported an index of -1 as being the problem.
> > >
> > > Edward, I see that you've been tickling the robot trying to narrow the issue;
> > > perhaps you've figured it out?
> > >
> > > If the bug is not immediately apparent to anyone, I would really appreciate a
> > > bit of tutoring around how to reproduce and get verifier logs.
> >
> > The repro is right there in the email I forwarded:
> >
> > > C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=15c38711180000
>
> I understand, but how does one go from this to either BPF assembly,
> or to running it in such a way that you also get verifier logs?

Adding logs to repro.c is too hard, but you can
hack the kernel with printk-s.

Like the following:

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index de7813947981..d158b83ed16c 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -7179,6 +7179,7 @@ static int check_stack_range_initialized(
                        return -EFAULT;
                }

+               printk("slot %d %d spi %d\n", slot, slot % BPF_REG_SIZE, spi);
                stype = &state->stack[spi].slot_type[slot % BPF_REG_SIZE];


shows that spi and slot get negative: -1, -2, ...

Re: stack access issue. Re: [syzbot] [bpf?] UBSAN: array-index-out-of-bounds in check_stack_range_initialized

[Andrei Matei]

Fixing in https://lore.kernel.org/bpf/20240324230323.1097685-1-andreimatei1@gmail.com/

FWIW, I managed to decode the BPF program that syzkaller used:

0: (18) r0 = 0x0
2: (18) r1 = map[id:4]
4: (b7) r8 = 0
5: (7b) *(u64 *)(r10 -8) = r8
6: (bf) r2 = r10
7: (07) r2 += -8
8: (b7) r3 = 8
9: (b7) r4 = 0
10: (85) call bloom_map_peek_elem#322320
11: (95) exit

Where the map is a bloom filter (as Alexei somehow already knew on the patch
thread) with a humongous value size.

4: type 30  flags 0x0
        key 0B  value 2147483649B  max_entries 255  memlock 720B

On Sat, Mar 23, 2024 at 10:55 PM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Sat, Mar 23, 2024 at 7:12 PM Andrei Matei <andreimatei1@gmail.com> wrote:
> >
> > On Sat, Mar 23, 2024 at 8:52 PM Alexei Starovoitov
> > <alexei.starovoitov@gmail.com> wrote:
> > >
> > > On Sat, Mar 23, 2024 at 5:50 PM Andrei Matei <andreimatei1@gmail.com> wrote:
> > > >
> > > > + Edward
> > > >
> > > > On Thu, Mar 21, 2024 at 3:33 AM Alexei Starovoitov
> > > > <alexei.starovoitov@gmail.com> wrote:
> > > > >
> > > > > Hi Andrei,
> > > > >
> > > > > looks like the refactoring of stack access introduced a bug.
> > > > > See the reproducer below.
> > > > > positive offsets are not caught by check_stack_access_within_bounds().
> > > >
> > > > check_stack_access_within_bounds() tries to catch positive offsets;
> > > > It does: [1]
> > > >
> > > > err = check_stack_slot_within_bounds(env, min_off, state, type);
> > > > if (!err && max_off > 0)
> > > >   err = -EINVAL; /* out of stack access into non-negative offsets */
> > > >
> > > > Notice the max_off > 0 in there.
> > > > And we have various tests that seem to check that positive offsets are
> > > > rejected. Do you know what the bug is?
> > > > I'm thinking maybe there's some overflow going on, except that UBSAN
> > > > reported an index of -1 as being the problem.
> > > >
> > > > Edward, I see that you've been tickling the robot trying to narrow the issue;
> > > > perhaps you've figured it out?
> > > >
> > > > If the bug is not immediately apparent to anyone, I would really appreciate a
> > > > bit of tutoring around how to reproduce and get verifier logs.
> > >
> > > The repro is right there in the email I forwarded:
> > >
> > > > C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=15c38711180000
> >
> > I understand, but how does one go from this to either BPF assembly,
> > or to running it in such a way that you also get verifier logs?
>
> Adding logs to repro.c is too hard, but you can
> hack the kernel with printk-s.
>
> Like the following:
>
> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index de7813947981..d158b83ed16c 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
> @@ -7179,6 +7179,7 @@ static int check_stack_range_initialized(
>                         return -EFAULT;
>                 }
>
> +               printk("slot %d %d spi %d\n", slot, slot % BPF_REG_SIZE, spi);
>                 stype = &state->stack[spi].slot_type[slot % BPF_REG_SIZE];
>
>
> shows that spi and slot get negative: -1, -2, ...