[syzbot] kernel BUG in dnotify_free_mark

[syzbot] kernel BUG in dnotify_free_mark

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit:    247f34f7b803 Linux 6.1-rc2
git tree:       upstream
console+strace: https://syzkaller.appspot.com/x/log.txt?x=157f594a880000
kernel config:  https://syzkaller.appspot.com/x/.config?x=1d3548a4365ba17d
dashboard link: https://syzkaller.appspot.com/bug?extid=06cc05ddc896f12b7ec5
compiler:       Debian clang version 13.0.1-++20220126092033+75e33f71c2da-1~exp1~20220126212112.63, GNU ld (GNU Binutils for Debian) 2.35.2
syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=15585936880000
C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=14ec85ba880000

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/a5f39164dea4/disk-247f34f7.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/8d1b92f5a01f/vmlinux-247f34f7.xz
mounted in repro: https://storage.googleapis.com/syzbot-assets/1a4d2943796c/mount_0.gz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+06cc05ddc896f12b7ec5@syzkaller.appspotmail.com

------------[ cut here ]------------
kernel BUG at fs/notify/dnotify/dnotify.c:136!
invalid opcode: 0000 [#1] PREEMPT SMP KASAN
CPU: 0 PID: 56 Comm: kworker/u4:4 Not tainted 6.1.0-rc2-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/11/2022
Workqueue: events_unbound fsnotify_mark_destroy_workfn
RIP: 0010:dnotify_free_mark+0x53/0x60 fs/notify/dnotify/dnotify.c:136
Code: 48 89 df e8 ff b3 dd ff 48 83 3b 00 75 17 e8 e4 bc 89 ff 48 8b 3d 4d ce 0f 0c 4c 89 f6 5b 41 5e e9 a2 de dc ff e8 cd bc 89 ff <0f> 0b cc cc cc cc cc cc cc cc cc cc cc 55 41 57 41 56 41 55 41 54
RSP: 0018:ffffc90001577b68 EFLAGS: 00010293
RAX: ffffffff81fe1253 RBX: ffff888075d2b080 RCX: ffff888018d40000
RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff888075d2b000
RBP: ffffc90001577c30 R08: dffffc0000000000 R09: fffffbfff2325fe4
R10: fffffbfff2325fe4 R11: 1ffffffff2325fe3 R12: ffff888145e77800
R13: ffffc90001577bc0 R14: ffff888075d2b000 R15: ffff888075d2b000
FS:  0000000000000000(0000) GS:ffff8880b9a00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f127cdcaa38 CR3: 000000001dd46000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
 <TASK>
 fsnotify_final_mark_destroy fs/notify/mark.c:278 [inline]
 fsnotify_mark_destroy_workfn+0x2cc/0x340 fs/notify/mark.c:902
 process_one_work+0x877/0xdb0 kernel/workqueue.c:2289
 worker_thread+0xb14/0x1330 kernel/workqueue.c:2436
 kthread+0x266/0x300 kernel/kthread.c:376
 ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:306
 </TASK>
Modules linked in:
---[ end trace 0000000000000000 ]---
RIP: 0010:dnotify_free_mark+0x53/0x60 fs/notify/dnotify/dnotify.c:136
Code: 48 89 df e8 ff b3 dd ff 48 83 3b 00 75 17 e8 e4 bc 89 ff 48 8b 3d 4d ce 0f 0c 4c 89 f6 5b 41 5e e9 a2 de dc ff e8 cd bc 89 ff <0f> 0b cc cc cc cc cc cc cc cc cc cc cc 55 41 57 41 56 41 55 41 54
RSP: 0018:ffffc90001577b68 EFLAGS: 00010293
RAX: ffffffff81fe1253 RBX: ffff888075d2b080 RCX: ffff888018d40000
RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff888075d2b000
RBP: ffffc90001577c30 R08: dffffc0000000000 R09


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzkaller@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
syzbot can test patches for this issue, for details see:
https://goo.gl/tpsmEJ#testing-patches

Re: [syzbot] kernel BUG in dnotify_free_mark

[Jan Kara]

Hello!

[added some CCs to gather more ideas]

On Fri 28-10-22 16:45:33, syzbot wrote:
> syzbot found the following issue on:
> 
> HEAD commit:    247f34f7b803 Linux 6.1-rc2
> git tree:       upstream
> console+strace: https://syzkaller.appspot.com/x/log.txt?x=157f594a880000
> kernel config:  https://syzkaller.appspot.com/x/.config?x=1d3548a4365ba17d
> dashboard link: https://syzkaller.appspot.com/bug?extid=06cc05ddc896f12b7ec5
> compiler:       Debian clang version 13.0.1-++20220126092033+75e33f71c2da-1~exp1~20220126212112.63, GNU ld (GNU Binutils for Debian) 2.35.2
> syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=15585936880000
> C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=14ec85ba880000
> 
> Downloadable assets:
> disk image: https://storage.googleapis.com/syzbot-assets/a5f39164dea4/disk-247f34f7.raw.xz
> vmlinux: https://storage.googleapis.com/syzbot-assets/8d1b92f5a01f/vmlinux-247f34f7.xz
> mounted in repro: https://storage.googleapis.com/syzbot-assets/1a4d2943796c/mount_0.gz
> 
> IMPORTANT: if you fix the issue, please add the following tag to the commit:
> Reported-by: syzbot+06cc05ddc896f12b7ec5@syzkaller.appspotmail.com
> 
> ------------[ cut here ]------------
> kernel BUG at fs/notify/dnotify/dnotify.c:136!

OK, I've tracked this down to the problem in ntfs3 driver or maybe more
exactly in bad inode handling. What the reproducer does is that it mounts
ntfs3 image, places dnotify mark on filesystem's /, then accesses something
which finds that / is corrupted.  This calls ntfs_bad_inode() which calls
make_bad_inode() which sets inode->i_mode to S_IFREG. So when the file
descriptor is closed, dnotify doesn't get properly shutdown because it
works only on directories. Now calling make_bad_inode() on live inode is
problematic because it can change inode type (e.g. from directory to
regular file) and that tends to confuse things - dnotify in this case.

Now it is easy to blame filesystem driver for calling make_bad_inode() on
live inode but given it seems to be relatively widespread maybe
make_bad_inode() should be more careful not to screw VFS? What do other
people think?

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

Re: [syzbot] kernel BUG in dnotify_free_mark

[Amir Goldstein]

On Mon, Oct 31, 2022 at 7:50 PM Jan Kara <jack@suse.cz> wrote:
>
> Hello!
>
> [added some CCs to gather more ideas]
>
> On Fri 28-10-22 16:45:33, syzbot wrote:
> > syzbot found the following issue on:
> >
> > HEAD commit:    247f34f7b803 Linux 6.1-rc2
> > git tree:       upstream
> > console+strace: https://syzkaller.appspot.com/x/log.txt?x=157f594a880000
> > kernel config:  https://syzkaller.appspot.com/x/.config?x=1d3548a4365ba17d
> > dashboard link: https://syzkaller.appspot.com/bug?extid=06cc05ddc896f12b7ec5
> > compiler:       Debian clang version 13.0.1-++20220126092033+75e33f71c2da-1~exp1~20220126212112.63, GNU ld (GNU Binutils for Debian) 2.35.2
> > syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=15585936880000
> > C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=14ec85ba880000
> >
> > Downloadable assets:
> > disk image: https://storage.googleapis.com/syzbot-assets/a5f39164dea4/disk-247f34f7.raw.xz
> > vmlinux: https://storage.googleapis.com/syzbot-assets/8d1b92f5a01f/vmlinux-247f34f7.xz
> > mounted in repro: https://storage.googleapis.com/syzbot-assets/1a4d2943796c/mount_0.gz
> >
> > IMPORTANT: if you fix the issue, please add the following tag to the commit:
> > Reported-by: syzbot+06cc05ddc896f12b7ec5@syzkaller.appspotmail.com
> >
> > ------------[ cut here ]------------
> > kernel BUG at fs/notify/dnotify/dnotify.c:136!
>
> OK, I've tracked this down to the problem in ntfs3 driver or maybe more
> exactly in bad inode handling. What the reproducer does is that it mounts
> ntfs3 image, places dnotify mark on filesystem's /, then accesses something
> which finds that / is corrupted.  This calls ntfs_bad_inode() which calls
> make_bad_inode() which sets inode->i_mode to S_IFREG. So when the file
> descriptor is closed, dnotify doesn't get properly shutdown because it
> works only on directories. Now calling make_bad_inode() on live inode is
> problematic because it can change inode type (e.g. from directory to
> regular file) and that tends to confuse things - dnotify in this case.
>
> Now it is easy to blame filesystem driver for calling make_bad_inode() on
> live inode but given it seems to be relatively widespread maybe
> make_bad_inode() should be more careful not to screw VFS? What do other
> people think?

Do you know why make_bad_inode() sets inode->i_mode to S_IFREG?
If it did not do that, would it solve the dnotify issue?

Thanks,
Amir.

Re: [syzbot] kernel BUG in dnotify_free_mark

[Jan Kara]

On Mon 31-10-22 20:18:25, Amir Goldstein wrote:
> On Mon, Oct 31, 2022 at 7:50 PM Jan Kara <jack@suse.cz> wrote:
> > [added some CCs to gather more ideas]
> >
> > On Fri 28-10-22 16:45:33, syzbot wrote:
> > > syzbot found the following issue on:
> > >
> > > HEAD commit:    247f34f7b803 Linux 6.1-rc2
> > > git tree:       upstream
> > > console+strace: https://syzkaller.appspot.com/x/log.txt?x=157f594a880000
> > > kernel config:  https://syzkaller.appspot.com/x/.config?x=1d3548a4365ba17d
> > > dashboard link: https://syzkaller.appspot.com/bug?extid=06cc05ddc896f12b7ec5
> > > compiler:       Debian clang version 13.0.1-++20220126092033+75e33f71c2da-1~exp1~20220126212112.63, GNU ld (GNU Binutils for Debian) 2.35.2
> > > syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=15585936880000
> > > C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=14ec85ba880000
> > >
> > > Downloadable assets:
> > > disk image: https://storage.googleapis.com/syzbot-assets/a5f39164dea4/disk-247f34f7.raw.xz
> > > vmlinux: https://storage.googleapis.com/syzbot-assets/8d1b92f5a01f/vmlinux-247f34f7.xz
> > > mounted in repro: https://storage.googleapis.com/syzbot-assets/1a4d2943796c/mount_0.gz
> > >
> > > IMPORTANT: if you fix the issue, please add the following tag to the commit:
> > > Reported-by: syzbot+06cc05ddc896f12b7ec5@syzkaller.appspotmail.com
> > >
> > > ------------[ cut here ]------------
> > > kernel BUG at fs/notify/dnotify/dnotify.c:136!
> >
> > OK, I've tracked this down to the problem in ntfs3 driver or maybe more
> > exactly in bad inode handling. What the reproducer does is that it mounts
> > ntfs3 image, places dnotify mark on filesystem's /, then accesses something
> > which finds that / is corrupted.  This calls ntfs_bad_inode() which calls
> > make_bad_inode() which sets inode->i_mode to S_IFREG. So when the file
> > descriptor is closed, dnotify doesn't get properly shutdown because it
> > works only on directories. Now calling make_bad_inode() on live inode is
> > problematic because it can change inode type (e.g. from directory to
> > regular file) and that tends to confuse things - dnotify in this case.
> >
> > Now it is easy to blame filesystem driver for calling make_bad_inode() on
> > live inode but given it seems to be relatively widespread maybe
> > make_bad_inode() should be more careful not to screw VFS? What do other
> > people think?
> 
> Do you know why make_bad_inode() sets inode->i_mode to S_IFREG?

I suppose because i_mode can be set to some bogus value (e.g. when
make_bad_inode() is called while reading the inode from the disk). One idea
I had was that we'd do this setting only if i_mode was indeed invalid. But
note that make_bad_inode() also sets inode->i_op and inode->i_fop and that
can also cause some surprises for a live inode (e.g. if some concurrent
process is in the middle of some operation on the inode).

> If it did not do that, would it solve the dnotify issue?

Yes, if i_mode was kept untouched, dnotify problem would be fixed.

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

Re: [syzbot] [ntfs3?] kernel BUG in dnotify_free_mark

[syzbot]

syzbot suspects this issue was fixed by commit:

commit 55ad333de0f80bc0caee10c6c27196cdcf8891bb
Author: Konstantin Komarov <almaz.alexandrovich@paragon-software.com>
Date:   Mon Dec 30 07:34:08 2024 +0000

    fs/ntfs3: Unify inode corruption marking with _ntfs_bad_inode()

bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=15420e42580000
start commit:   fe46a7dd189e Merge tag 'sound-6.9-rc1' of git://git.kernel..
git tree:       upstream
kernel config:  https://syzkaller.appspot.com/x/.config?x=1a07d5da4eb21586
dashboard link: https://syzkaller.appspot.com/bug?extid=06cc05ddc896f12b7ec5
syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=16e3dffd180000
C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=13f9e08d180000

If the result looks correct, please mark the issue as fixed by replying with:

#syz fix: fs/ntfs3: Unify inode corruption marking with _ntfs_bad_inode()

For information about bisection process see: https://goo.gl/tpsmEJ#bisection