* Re: INFO: task hung in vhost_net_stop_vq [not found] ` <0000000000008ee87005847fc985@google.com> @ 2019-03-25 14:02 ` Michael S. Tsirkin 2019-03-26 10:17 ` Jason Wang [not found] ` <df4f2cf6-8469-f894-8f45-7c48a6a1801f@redhat.com> 0 siblings, 2 replies; 4+ messages in thread From: Michael S. Tsirkin @ 2019-03-25 14:02 UTC (permalink / raw) To: syzbot; +Cc: weiyj.lk, kvm, netdev, syzkaller-bugs, linux-kernel, virtualization Looks like more iotlb locking mess? On Tue, Mar 19, 2019 at 10:21:00PM -0700, syzbot wrote: > syzbot has bisected this bug to: > > commit 6b1e6cc7855b09a0a9bfa1d9f30172ba366f161c > Author: Jason Wang <jasowang@redhat.com> > Date: Thu Jun 23 06:04:32 2016 +0000 > > vhost: new device IOTLB API > > bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=1486ad27200000 > start commit: 6b1e6cc7 vhost: new device IOTLB API > git tree: upstream > final crash: https://syzkaller.appspot.com/x/report.txt?x=1686ad27200000 > console output: https://syzkaller.appspot.com/x/log.txt?x=1286ad27200000 > kernel config: https://syzkaller.appspot.com/x/.config?x=c94f9f0c0363db4b > dashboard link: https://syzkaller.appspot.com/bug?extid=d21e6e297322a900c128 > syz repro: https://syzkaller.appspot.com/x/repro.syz?x=141db34d400000 > C reproducer: https://syzkaller.appspot.com/x/repro.c?x=108ef293400000 > > Reported-by: syzbot+d21e6e297322a900c128@syzkaller.appspotmail.com > Fixes: 6b1e6cc7 ("vhost: new device IOTLB API") ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: INFO: task hung in vhost_net_stop_vq 2019-03-25 14:02 ` INFO: task hung in vhost_net_stop_vq Michael S. Tsirkin @ 2019-03-26 10:17 ` Jason Wang [not found] ` <df4f2cf6-8469-f894-8f45-7c48a6a1801f@redhat.com> 1 sibling, 0 replies; 4+ messages in thread From: Jason Wang @ 2019-03-26 10:17 UTC (permalink / raw) To: Michael S. Tsirkin, syzbot Cc: weiyj.lk, kvm, netdev, syzkaller-bugs, linux-kernel, virtualization On 2019/3/25 下午10:02, Michael S. Tsirkin wrote: > Looks like more iotlb locking mess? Looking at the calltrace: [ 221.743675] ============================================= [ 221.744297] [ INFO: possible recursive locking detected ] [ 221.744944] 4.7.0+ #1 Not tainted [ 221.745326] --------------------------------------------- [ 221.746128] syz-executor1/6823 is trying to acquire lock: [ 221.746737] (&vq->mutex){+.+...}, at: [<ffffffff84484b70>] vhost_process_iotlb_msg+0xe0/0x9e0 [ 221.747789] [ 221.747789] but task is already holding lock: [ 221.748470] (&vq->mutex){+.+...}, at: [<ffffffff84484b70>] vhost_process_iotlb_msg+0xe0/0x9e0 [ 221.749535] [ 221.749535] other info that might help us debug this: [ 221.750280] Possible unsafe locking scenario: [ 221.750280] [ 221.750946] CPU0 [ 221.751232] ---- [ 221.751523] lock(&vq->mutex); [ 221.751922] lock(&vq->mutex); [ 221.752339] [ 221.752339] *** DEADLOCK *** [ 221.752339] I could not think of a path that can hit this. And I could not reproduce with the reproducer in the link in net-next. Thanks > > On Tue, Mar 19, 2019 at 10:21:00PM -0700, syzbot wrote: >> syzbot has bisected this bug to: >> >> commit 6b1e6cc7855b09a0a9bfa1d9f30172ba366f161c >> Author: Jason Wang <jasowang@redhat.com> >> Date: Thu Jun 23 06:04:32 2016 +0000 >> >> vhost: new device IOTLB API >> >> bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=1486ad27200000 >> start commit: 6b1e6cc7 vhost: new device IOTLB API >> git tree: upstream >> final crash: https://syzkaller.appspot.com/x/report.txt?x=1686ad27200000 >> console output: https://syzkaller.appspot.com/x/log.txt?x=1286ad27200000 >> kernel config: https://syzkaller.appspot.com/x/.config?x=c94f9f0c0363db4b >> dashboard link: https://syzkaller.appspot.com/bug?extid=d21e6e297322a900c128 >> syz repro: https://syzkaller.appspot.com/x/repro.syz?x=141db34d400000 >> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=108ef293400000 >> >> Reported-by: syzbot+d21e6e297322a900c128@syzkaller.appspotmail.com >> Fixes: 6b1e6cc7 ("vhost: new device IOTLB API") _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization ^ permalink raw reply [flat|nested] 4+ messages in thread
[parent not found: <df4f2cf6-8469-f894-8f45-7c48a6a1801f@redhat.com>]
* Re: INFO: task hung in vhost_net_stop_vq [not found] ` <df4f2cf6-8469-f894-8f45-7c48a6a1801f@redhat.com> @ 2019-03-26 10:28 ` Dmitry Vyukov via Virtualization [not found] ` <CACT4Y+Z1s-Lx1UXHKj88kQoOcbiD8gwyuRU3F_+cceP3pzbbrw@mail.gmail.com> 1 sibling, 0 replies; 4+ messages in thread From: Dmitry Vyukov via Virtualization @ 2019-03-26 10:28 UTC (permalink / raw) To: Jason Wang Cc: syzbot, weiyj.lk, KVM list, Michael S. Tsirkin, netdev, syzkaller-bugs, LKML, virtualization On Tue, Mar 26, 2019 at 11:17 AM Jason Wang <jasowang@redhat.com> wrote: > > > On 2019/3/25 下午10:02, Michael S. Tsirkin wrote: > > Looks like more iotlb locking mess? > > > Looking at the calltrace: > > [ 221.743675] ============================================= > [ 221.744297] [ INFO: possible recursive locking detected ] > [ 221.744944] 4.7.0+ #1 Not tainted > [ 221.745326] --------------------------------------------- > [ 221.746128] syz-executor1/6823 is trying to acquire lock: > [ 221.746737] (&vq->mutex){+.+...}, at: [<ffffffff84484b70>] vhost_process_iotlb_msg+0xe0/0x9e0 > [ 221.747789] > [ 221.747789] but task is already holding lock: > [ 221.748470] (&vq->mutex){+.+...}, at: [<ffffffff84484b70>] vhost_process_iotlb_msg+0xe0/0x9e0 > [ 221.749535] > [ 221.749535] other info that might help us debug this: > [ 221.750280] Possible unsafe locking scenario: > [ 221.750280] > [ 221.750946] CPU0 > [ 221.751232] ---- > [ 221.751523] lock(&vq->mutex); > [ 221.751922] lock(&vq->mutex); > [ 221.752339] > [ 221.752339] *** DEADLOCK *** > [ 221.752339] > > I could not think of a path that can hit this. And I could not reproduce with the reproducer in the link in net-next. Looking at the bisection log, syzbot is able to reproduce this super-reliably on multiple kernel revisions. Are you sure you are using the right config/revision? What else can be in play? syzbot uses VMs. The image is available. > Thanks > > > > > > On Tue, Mar 19, 2019 at 10:21:00PM -0700, syzbot wrote: > >> syzbot has bisected this bug to: > >> > >> commit 6b1e6cc7855b09a0a9bfa1d9f30172ba366f161c > >> Author: Jason Wang <jasowang@redhat.com> > >> Date: Thu Jun 23 06:04:32 2016 +0000 > >> > >> vhost: new device IOTLB API > >> > >> bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=1486ad27200000 > >> start commit: 6b1e6cc7 vhost: new device IOTLB API > >> git tree: upstream > >> final crash: https://syzkaller.appspot.com/x/report.txt?x=1686ad27200000 > >> console output: https://syzkaller.appspot.com/x/log.txt?x=1286ad27200000 > >> kernel config: https://syzkaller.appspot.com/x/.config?x=c94f9f0c0363db4b > >> dashboard link: https://syzkaller.appspot.com/bug?extid=d21e6e297322a900c128 > >> syz repro: https://syzkaller.appspot.com/x/repro.syz?x=141db34d400000 > >> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=108ef293400000 > >> > >> Reported-by: syzbot+d21e6e297322a900c128@syzkaller.appspotmail.com > >> Fixes: 6b1e6cc7 ("vhost: new device IOTLB API") > > -- > You received this message because you are subscribed to the Google Groups "syzkaller-bugs" group. > To unsubscribe from this group and stop receiving emails from it, send an email to syzkaller-bugs+unsubscribe@googlegroups.com. > To view this discussion on the web visit https://groups.google.com/d/msgid/syzkaller-bugs/df4f2cf6-8469-f894-8f45-7c48a6a1801f%40redhat.com. > For more options, visit https://groups.google.com/d/optout. _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization ^ permalink raw reply [flat|nested] 4+ messages in thread
[parent not found: <CACT4Y+Z1s-Lx1UXHKj88kQoOcbiD8gwyuRU3F_+cceP3pzbbrw@mail.gmail.com>]
* Re: INFO: task hung in vhost_net_stop_vq [not found] ` <CACT4Y+Z1s-Lx1UXHKj88kQoOcbiD8gwyuRU3F_+cceP3pzbbrw@mail.gmail.com> @ 2019-04-09 3:31 ` Jason Wang 0 siblings, 0 replies; 4+ messages in thread From: Jason Wang @ 2019-04-09 3:31 UTC (permalink / raw) To: Dmitry Vyukov Cc: syzbot, weiyj.lk, KVM list, Michael S. Tsirkin, netdev, syzkaller-bugs, LKML, virtualization On 2019/3/26 下午6:28, Dmitry Vyukov wrote: > On Tue, Mar 26, 2019 at 11:17 AM Jason Wang<jasowang@redhat.com> wrote: >> On 2019/3/25 下午10:02, Michael S. Tsirkin wrote: >>> Looks like more iotlb locking mess? >> Looking at the calltrace: >> >> [ 221.743675] ============================================= >> [ 221.744297] [ INFO: possible recursive locking detected ] >> [ 221.744944] 4.7.0+ #1 Not tainted >> [ 221.745326] --------------------------------------------- >> [ 221.746128] syz-executor1/6823 is trying to acquire lock: >> [ 221.746737] (&vq->mutex){+.+...}, at: [<ffffffff84484b70>] vhost_process_iotlb_msg+0xe0/0x9e0 >> [ 221.747789] >> [ 221.747789] but task is already holding lock: >> [ 221.748470] (&vq->mutex){+.+...}, at: [<ffffffff84484b70>] vhost_process_iotlb_msg+0xe0/0x9e0 >> [ 221.749535] >> [ 221.749535] other info that might help us debug this: >> [ 221.750280] Possible unsafe locking scenario: >> [ 221.750280] >> [ 221.750946] CPU0 >> [ 221.751232] ---- >> [ 221.751523] lock(&vq->mutex); >> [ 221.751922] lock(&vq->mutex); >> [ 221.752339] >> [ 221.752339] *** DEADLOCK *** >> [ 221.752339] >> >> I could not think of a path that can hit this. And I could not reproduce with the reproducer in the link in net-next. > Looking at the bisection log, syzbot is able to reproduce this > super-reliably on multiple kernel revisions. Are you sure you are > using the right config/revision? What else can be in play? syzbot uses > VMs. The image is available. > > Yes, looks like the reason is vhost accept zero size iova range which lead a infinite loop when trying to translate iova. Will post a patch to fix this. Thanks _______________________________________________ Virtualization mailing list Virtualization@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/virtualization ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2019-04-09 3:31 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <0000000000004f8499057be95e1d@google.com>
[not found] ` <0000000000008ee87005847fc985@google.com>
2019-03-25 14:02 ` INFO: task hung in vhost_net_stop_vq Michael S. Tsirkin
2019-03-26 10:17 ` Jason Wang
[not found] ` <df4f2cf6-8469-f894-8f45-7c48a6a1801f@redhat.com>
2019-03-26 10:28 ` Dmitry Vyukov via Virtualization
[not found] ` <CACT4Y+Z1s-Lx1UXHKj88kQoOcbiD8gwyuRU3F_+cceP3pzbbrw@mail.gmail.com>
2019-04-09 3:31 ` Jason Wang
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).