* irqfd issue @ 2013-05-07 1:37 Asias He 2013-05-07 7:55 ` Gleb Natapov 0 siblings, 1 reply; 13+ messages in thread From: Asias He @ 2013-05-07 1:37 UTC (permalink / raw) To: kvm; +Cc: Gleb Natapov, Michael S. Tsirkin Hi, I am seeing this with linus/master. Any ideas? [ 34.168356] IPv6: ADDRCONF(NETDEV_UP): virbr0: link is not ready [ 36.743758] BUG: unable to handle kernel paging request at 0000000300000029 [ 36.745177] IP: [<ffffffff81c08584>] __mutex_lock_slowpath+0x34/0x240 [ 36.746576] PGD 0 [ 36.747962] Oops: 0000 [#1] SMP [ 36.749343] Modules linked in: ipt_MASQUERADE iptable_nat nf_nat_ipv4 nf_nat vhost_net vhost [ 36.750753] CPU: 0 PID: 4260 Comm: qemu-kvm Not tainted 3.9.0+ #752 [ 36.752130] Hardware name: Dell Inc. OptiPlex 790/0V5HMK, BIOS A11 12/30/2011 [ 36.753495] task: ffff88021fba8000 ti: ffff88021f0c4000 task.ti: ffff88021f0c4000 [ 36.754847] RIP: 0010:[<ffffffff81c08584>] [<ffffffff81c08584>] __mutex_lock_slowpath+0x34/0x240 [ 36.756228] RSP: 0018:ffff88021f0c5c88 EFLAGS: 00010202 [ 36.757584] RAX: 0000000000000001 RBX: ffff880223ffb420 RCX: 0000000000000000 [ 36.758926] RDX: 0000000300000001 RSI: ffff88021f0c5d60 RDI: ffff880223ffb420 [ 36.760268] RBP: ffff88021f0c5cf8 R08: ffff88021f0c0000 R09: ffffffff00000000 [ 36.761602] R10: ffff8802209c3f10 R11: 0000000000000000 R12: ffff880223ffb420 [ 36.762924] R13: ffff88022236c000 R14: ffff8802236213b0 R15: ffff880223ffb420 [ 36.764232] FS: 0000000000000000(0000) GS:ffff88022dc00000(0000) knlGS:0000000000000000 [ 36.765543] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 36.766869] CR2: 0000000300000029 CR3: 000000000240b000 CR4: 00000000000427f0 [ 36.768220] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 36.769565] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 36.770889] Stack: [ 36.772200] 0000000000000092 ffff88021fba8000 ffff88021f0c5cb8 ffffffff81c0b2c7 [ 36.773551] ffff8802210a4fc0 ffff8802210a4fc0 ffff88021f0c5d08 ffffffff810c0f2c [ 36.774884] 000000000000000e ffff880223ffb420 ffff88021f0c5d38 ffff88022236c000 [ 36.776194] Call Trace: [ 36.777482] [<ffffffff81c0b2c7>] ? _raw_spin_unlock_irqrestore+0x37/0x40 [ 36.778789] [<ffffffff810c0f2c>] ? try_to_wake_up+0x1ec/0x290 [ 36.780107] [<ffffffff81c0852b>] mutex_lock+0x2b/0x50 [ 36.781420] [<ffffffff810a9b6d>] flush_workqueue+0x9d/0x560 [ 36.782729] [<ffffffff8100933f>] kvm_irqfd_release+0x8f/0xa0 [ 36.784046] [<ffffffff8100456d>] kvm_vm_release+0x1d/0x30 [ 36.785367] [<ffffffff811a732a>] __fput+0xba/0x240 [ 36.786693] [<ffffffff811a751e>] ____fput+0xe/0x10 [ 36.788007] [<ffffffff810af685>] task_work_run+0xa5/0xe0 [ 36.789317] [<ffffffff81092cd7>] do_exit+0x2d7/0xac0 [ 36.790622] [<ffffffff811a4a04>] ? fsnotify_modify+0x64/0x80 [ 36.791896] [<ffffffff8140327a>] ? trace_hardirqs_off_thunk+0x3a/0x6c [ 36.793141] [<ffffffff81093511>] do_group_exit+0x51/0xc0 [ 36.794358] [<ffffffff81093597>] SyS_exit_group+0x17/0x20 [ 36.795547] [<ffffffff81c13882>] system_call_fastpath+0x16/0x1b [ 36.796731] Code: 55 41 54 53 48 83 ec 48 66 66 66 66 90 65 48 8b 04 25 00 b8 00 00 49 89 fc 48 89 45 98 48 8b 57 18 b8 01 00 00 00 48 85 d2 74 03 <8b> 42 28 85 c0 0f 84 e6 00 00 00 65 48 8b 04 25 08 b8 00 00 48 [ 36.798194] RIP [<ffffffff81c08584>] __mutex_lock_slowpath+0x34/0x240 [ 36.799567] RSP <ffff88021f0c5c88> [ 36.800943] CR2: 0000000300000029 [ 36.813185] ---[ end trace 4877613defb9fc19 ]--- [ 36.813188] Fixing recursive fault but reboot is needed! [ 37.011566] usb 2-1.1: link qh8-0601/ffff880223a9c600 start 3 [1/2 us] [ 70.539341] usb usb1: usb port1's DeviceRemovable is changed to 1 according to platform information. [ 70.539546] usb usb2: usb port1's DeviceRemovable is changed to 1 according to platform information. [ 70.862490] nr_pdflush_threads exported in /proc is scheduled for removal [ 70.862558] sysctl: The scan_unevictable_pages sysctl/node-interface has been disabled for lack of a legitimate use case. If you have one, please send an email to linux-mm@kvack.org. -- Asias ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: irqfd issue 2013-05-07 1:37 irqfd issue Asias He @ 2013-05-07 7:55 ` Gleb Natapov 2013-05-07 8:14 ` Asias He 0 siblings, 1 reply; 13+ messages in thread From: Gleb Natapov @ 2013-05-07 7:55 UTC (permalink / raw) To: Asias He; +Cc: kvm, Michael S. Tsirkin On Tue, May 07, 2013 at 09:37:30AM +0800, Asias He wrote: > Hi, > > I am seeing this with linus/master. Any ideas? > How reproducible it this? What HEAD are you seeing this with? -- Gleb. ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: irqfd issue 2013-05-07 7:55 ` Gleb Natapov @ 2013-05-07 8:14 ` Asias He 2013-05-07 8:18 ` Gleb Natapov 0 siblings, 1 reply; 13+ messages in thread From: Asias He @ 2013-05-07 8:14 UTC (permalink / raw) To: Gleb Natapov; +Cc: kvm, Michael S. Tsirkin On Tue, May 07, 2013 at 10:55:36AM +0300, Gleb Natapov wrote: > On Tue, May 07, 2013 at 09:37:30AM +0800, Asias He wrote: > > Hi, > > > > I am seeing this with linus/master. Any ideas? > > > How reproducible it this? What HEAD are you seeing this with? Almost always. Start a guest with vhost-net on. QEMU : e3351000cd682200835763caca87adf708ed1c65 KERNEL: 51a26ae7a14b85c99c9be470c2d28eeeba0f26a3 > -- > Gleb. -- Asias ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: irqfd issue 2013-05-07 8:14 ` Asias He @ 2013-05-07 8:18 ` Gleb Natapov 2013-05-07 8:28 ` Asias He 2013-05-07 8:36 ` Asias He 0 siblings, 2 replies; 13+ messages in thread From: Gleb Natapov @ 2013-05-07 8:18 UTC (permalink / raw) To: Asias He; +Cc: kvm, Michael S. Tsirkin On Tue, May 07, 2013 at 04:14:50PM +0800, Asias He wrote: > On Tue, May 07, 2013 at 10:55:36AM +0300, Gleb Natapov wrote: > > On Tue, May 07, 2013 at 09:37:30AM +0800, Asias He wrote: > > > Hi, > > > > > > I am seeing this with linus/master. Any ideas? > > > > > How reproducible it this? What HEAD are you seeing this with? > > Almost always. Start a guest with vhost-net on. > It happens during start or on VM exit? The trace shows do_exit() > QEMU : e3351000cd682200835763caca87adf708ed1c65 > KERNEL: 51a26ae7a14b85c99c9be470c2d28eeeba0f26a3 > Can you try with kvm.git next branch? -- Gleb. ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: irqfd issue 2013-05-07 8:18 ` Gleb Natapov @ 2013-05-07 8:28 ` Asias He 2013-05-07 8:36 ` Asias He 1 sibling, 0 replies; 13+ messages in thread From: Asias He @ 2013-05-07 8:28 UTC (permalink / raw) To: Gleb Natapov; +Cc: kvm, Michael S. Tsirkin On Tue, May 07, 2013 at 11:18:38AM +0300, Gleb Natapov wrote: > On Tue, May 07, 2013 at 04:14:50PM +0800, Asias He wrote: > > On Tue, May 07, 2013 at 10:55:36AM +0300, Gleb Natapov wrote: > > > On Tue, May 07, 2013 at 09:37:30AM +0800, Asias He wrote: > > > > Hi, > > > > > > > > I am seeing this with linus/master. Any ideas? > > > > > > > How reproducible it this? What HEAD are you seeing this with? > > > > Almost always. Start a guest with vhost-net on. > > > It happens during start or on VM exit? The trace shows do_exit() Yes. It happens when you shutdown the VM. > > QEMU : e3351000cd682200835763caca87adf708ed1c65 > > KERNEL: 51a26ae7a14b85c99c9be470c2d28eeeba0f26a3 > > > Can you try with kvm.git next branch? /me compiling > -- > Gleb. -- Asias ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: irqfd issue 2013-05-07 8:18 ` Gleb Natapov 2013-05-07 8:28 ` Asias He @ 2013-05-07 8:36 ` Asias He 2013-05-07 8:42 ` Gleb Natapov 1 sibling, 1 reply; 13+ messages in thread From: Asias He @ 2013-05-07 8:36 UTC (permalink / raw) To: Gleb Natapov; +Cc: kvm, Michael S. Tsirkin On Tue, May 07, 2013 at 11:18:38AM +0300, Gleb Natapov wrote: > On Tue, May 07, 2013 at 04:14:50PM +0800, Asias He wrote: > > On Tue, May 07, 2013 at 10:55:36AM +0300, Gleb Natapov wrote: > > > On Tue, May 07, 2013 at 09:37:30AM +0800, Asias He wrote: > > > > Hi, > > > > > > > > I am seeing this with linus/master. Any ideas? > > > > > > > How reproducible it this? What HEAD are you seeing this with? > > > > Almost always. Start a guest with vhost-net on. > > > It happens during start or on VM exit? The trace shows do_exit() > > > QEMU : e3351000cd682200835763caca87adf708ed1c65 > > KERNEL: 51a26ae7a14b85c99c9be470c2d28eeeba0f26a3 > > > Can you try with kvm.git next branch? With next branch db6ae6158186a17165ef990bda2895ae7594b039, [ 53.386936] device tap0 entered promiscuous mode [ 53.386990] br0: port 2(tap0) entered forwarding state [ 53.386997] br0: port 2(tap0) entered forwarding state [ 80.096275] BUG: unable to handle kernel NULL pointer dereference at (null) [ 80.098592] IP: [<ffffffff81c0721e>] _raw_spin_lock+0xe/0x30 [ 80.100988] PGD 0 [ 80.103339] Oops: 0002 [#1] SMP [ 80.105668] Modules linked in: vhost_net [ 80.108005] CPU 6 [ 80.108026] Pid: 4257, comm: qemu-system-x86 Not tainted 3.9.0-rc3+ #757 Dell Inc. OptiPlex 790/0V5HMK [ 80.112662] RIP: 0010:[<ffffffff81c0721e>] [<ffffffff81c0721e>] _raw_spin_lock+0xe/0x30 [ 80.115030] RSP: 0018:ffff880221721cc8 EFLAGS: 00010046 [ 80.117365] RAX: 0000000000000100 RBX: ffff88022dcc003f RCX: ffff880221734950 [ 80.119688] RDX: ffff8802208f6ca8 RSI: 000000007fffffff RDI: 0000000000000000 [ 80.121982] RBP: ffff880221721cc8 R08: 0000000000000002 R09: 0000000000000002 [ 80.124230] R10: 00007f7fd01087e0 R11: 0000000000000246 R12: ffff8802208f6ca8 [ 80.126489] R13: 0000000000000080 R14: ffff880223e2a900 R15: 0000000000000000 [ 80.128718] FS: 00007f7fd38488e0(0000) GS:ffff88022dcc0000(0000) knlGS:0000000000000000 [ 80.130921] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 80.133130] CR2: 0000000000000000 CR3: 000000022309f000 CR4: 00000000000427e0 [ 80.135345] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 80.137543] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [ 80.139708] Process qemu-system-x86 (pid: 4257, threadinfo ffff880221720000, task ffff880222bd5640) [ 80.141887] Stack: [ 80.144064] ffff880221721d08 ffffffff810ac5c5 ffff88022431dc00 0000000000000086 [ 80.146316] 0000000000000080 ffff880223e2a900 ffff8802208f6ca8 0000000000000000 [ 80.148558] ffff880221721d48 ffffffff810ac8fe 0000000000000000 ffff880221734000 [ 80.150784] Call Trace: [ 80.152975] [<ffffffff810ac5c5>] __queue_work+0x45/0x2d0 [ 80.155193] [<ffffffff810ac8fe>] queue_work_on+0x8e/0xa0 [ 80.157374] [<ffffffff810ac949>] queue_work+0x19/0x20 [ 80.159505] [<ffffffff81009b6b>] irqfd_deactivate+0x4b/0x60 [ 80.161621] [<ffffffff8100a69d>] kvm_irqfd+0x39d/0x580 [ 80.163717] [<ffffffff81007a27>] kvm_vm_ioctl+0x207/0x5b0 [ 80.165832] [<ffffffff810c9545>] ? update_curr+0xf5/0x180 [ 80.167931] [<ffffffff811b66e8>] do_vfs_ioctl+0x98/0x550 [ 80.170013] [<ffffffff810c1f5e>] ? finish_task_switch+0x4e/0xe0 [ 80.172109] [<ffffffff81c054aa>] ? __schedule+0x2ea/0x710 [ 80.174218] [<ffffffff811b6bf7>] sys_ioctl+0x57/0x90 [ 80.176338] [<ffffffff8140ae9e>] ? trace_hardirqs_on_thunk+0x3a/0x3c [ 80.178467] [<ffffffff81c0f602>] system_call_fastpath+0x16/0x1b [ 80.180598] Code: c1 ea 08 38 c2 74 0f 66 0f 1f 44 00 00 f3 90 0f b6 03 38 c2 75 f7 48 83 c4 08 5b c9 c3 55 48 89 e5 66 66 66 66 90 b8 00 01 00 00 <f0> 66 0f c1 07 89 c2 66 c1 ea 08 38 c2 74 0c 0f 1f 00 f3 90 0f [ 80.183098] RIP [<ffffffff81c0721e>] _raw_spin_lock+0xe/0x30 [ 80.185372] RSP <ffff880221721cc8> [ 80.187584] CR2: 0000000000000000 [ 80.208627] ---[ end trace 13fb1e4b6e5ab21f ]--- > -- > Gleb. -- Asias ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: irqfd issue 2013-05-07 8:36 ` Asias He @ 2013-05-07 8:42 ` Gleb Natapov 2013-05-07 9:10 ` Asias He 0 siblings, 1 reply; 13+ messages in thread From: Gleb Natapov @ 2013-05-07 8:42 UTC (permalink / raw) To: Asias He; +Cc: kvm, Michael S. Tsirkin, Cornelia Huck On Tue, May 07, 2013 at 04:36:50PM +0800, Asias He wrote: > On Tue, May 07, 2013 at 11:18:38AM +0300, Gleb Natapov wrote: > > On Tue, May 07, 2013 at 04:14:50PM +0800, Asias He wrote: > > > On Tue, May 07, 2013 at 10:55:36AM +0300, Gleb Natapov wrote: > > > > On Tue, May 07, 2013 at 09:37:30AM +0800, Asias He wrote: > > > > > Hi, > > > > > > > > > > I am seeing this with linus/master. Any ideas? > > > > > > > > > How reproducible it this? What HEAD are you seeing this with? > > > > > > Almost always. Start a guest with vhost-net on. > > > > > It happens during start or on VM exit? The trace shows do_exit() > > > > > QEMU : e3351000cd682200835763caca87adf708ed1c65 > > > KERNEL: 51a26ae7a14b85c99c9be470c2d28eeeba0f26a3 > > > > > Can you try with kvm.git next branch? > > With next branch db6ae6158186a17165ef990bda2895ae7594b039, > Can you revert a0f155e9646d5f1c263f6f9aae880151100243bb and try again? > [ 53.386936] device tap0 entered promiscuous mode > [ 53.386990] br0: port 2(tap0) entered forwarding state > [ 53.386997] br0: port 2(tap0) entered forwarding state > [ 80.096275] BUG: unable to handle kernel NULL pointer dereference at (null) > [ 80.098592] IP: [<ffffffff81c0721e>] _raw_spin_lock+0xe/0x30 > [ 80.100988] PGD 0 > [ 80.103339] Oops: 0002 [#1] SMP > [ 80.105668] Modules linked in: vhost_net > [ 80.108005] CPU 6 > [ 80.108026] Pid: 4257, comm: qemu-system-x86 Not tainted 3.9.0-rc3+ #757 Dell Inc. OptiPlex 790/0V5HMK > [ 80.112662] RIP: 0010:[<ffffffff81c0721e>] [<ffffffff81c0721e>] _raw_spin_lock+0xe/0x30 > [ 80.115030] RSP: 0018:ffff880221721cc8 EFLAGS: 00010046 > [ 80.117365] RAX: 0000000000000100 RBX: ffff88022dcc003f RCX: ffff880221734950 > [ 80.119688] RDX: ffff8802208f6ca8 RSI: 000000007fffffff RDI: 0000000000000000 > [ 80.121982] RBP: ffff880221721cc8 R08: 0000000000000002 R09: 0000000000000002 > [ 80.124230] R10: 00007f7fd01087e0 R11: 0000000000000246 R12: ffff8802208f6ca8 > [ 80.126489] R13: 0000000000000080 R14: ffff880223e2a900 R15: 0000000000000000 > [ 80.128718] FS: 00007f7fd38488e0(0000) GS:ffff88022dcc0000(0000) knlGS:0000000000000000 > [ 80.130921] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [ 80.133130] CR2: 0000000000000000 CR3: 000000022309f000 CR4: 00000000000427e0 > [ 80.135345] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 > [ 80.137543] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 > [ 80.139708] Process qemu-system-x86 (pid: 4257, threadinfo ffff880221720000, task ffff880222bd5640) > [ 80.141887] Stack: > [ 80.144064] ffff880221721d08 ffffffff810ac5c5 ffff88022431dc00 0000000000000086 > [ 80.146316] 0000000000000080 ffff880223e2a900 ffff8802208f6ca8 0000000000000000 > [ 80.148558] ffff880221721d48 ffffffff810ac8fe 0000000000000000 ffff880221734000 > [ 80.150784] Call Trace: > [ 80.152975] [<ffffffff810ac5c5>] __queue_work+0x45/0x2d0 > [ 80.155193] [<ffffffff810ac8fe>] queue_work_on+0x8e/0xa0 > [ 80.157374] [<ffffffff810ac949>] queue_work+0x19/0x20 > [ 80.159505] [<ffffffff81009b6b>] irqfd_deactivate+0x4b/0x60 > [ 80.161621] [<ffffffff8100a69d>] kvm_irqfd+0x39d/0x580 > [ 80.163717] [<ffffffff81007a27>] kvm_vm_ioctl+0x207/0x5b0 > [ 80.165832] [<ffffffff810c9545>] ? update_curr+0xf5/0x180 > [ 80.167931] [<ffffffff811b66e8>] do_vfs_ioctl+0x98/0x550 > [ 80.170013] [<ffffffff810c1f5e>] ? finish_task_switch+0x4e/0xe0 > [ 80.172109] [<ffffffff81c054aa>] ? __schedule+0x2ea/0x710 > [ 80.174218] [<ffffffff811b6bf7>] sys_ioctl+0x57/0x90 > [ 80.176338] [<ffffffff8140ae9e>] ? trace_hardirqs_on_thunk+0x3a/0x3c > [ 80.178467] [<ffffffff81c0f602>] system_call_fastpath+0x16/0x1b > [ 80.180598] Code: c1 ea 08 38 c2 74 0f 66 0f 1f 44 00 00 f3 90 0f b6 03 38 c2 75 f7 48 83 c4 08 5b c9 c3 55 48 89 e5 66 66 66 66 90 b8 00 01 00 00 <f0> 66 0f c1 07 89 c2 66 c1 ea 08 38 c2 74 0c 0f 1f 00 f3 90 0f > [ 80.183098] RIP [<ffffffff81c0721e>] _raw_spin_lock+0xe/0x30 > [ 80.185372] RSP <ffff880221721cc8> > [ 80.187584] CR2: 0000000000000000 > [ 80.208627] ---[ end trace 13fb1e4b6e5ab21f ]--- > > > -- > > Gleb. > > -- > Asias -- Gleb. ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: irqfd issue 2013-05-07 8:42 ` Gleb Natapov @ 2013-05-07 9:10 ` Asias He 2013-05-07 9:12 ` Gleb Natapov 0 siblings, 1 reply; 13+ messages in thread From: Asias He @ 2013-05-07 9:10 UTC (permalink / raw) To: Gleb Natapov; +Cc: kvm, Michael S. Tsirkin, Cornelia Huck On Tue, May 07, 2013 at 11:42:07AM +0300, Gleb Natapov wrote: > On Tue, May 07, 2013 at 04:36:50PM +0800, Asias He wrote: > > On Tue, May 07, 2013 at 11:18:38AM +0300, Gleb Natapov wrote: > > > On Tue, May 07, 2013 at 04:14:50PM +0800, Asias He wrote: > > > > On Tue, May 07, 2013 at 10:55:36AM +0300, Gleb Natapov wrote: > > > > > On Tue, May 07, 2013 at 09:37:30AM +0800, Asias He wrote: > > > > > > Hi, > > > > > > > > > > > > I am seeing this with linus/master. Any ideas? > > > > > > > > > > > How reproducible it this? What HEAD are you seeing this with? > > > > > > > > Almost always. Start a guest with vhost-net on. > > > > > > > It happens during start or on VM exit? The trace shows do_exit() > > > > > > > QEMU : e3351000cd682200835763caca87adf708ed1c65 > > > > KERNEL: 51a26ae7a14b85c99c9be470c2d28eeeba0f26a3 > > > > > > > Can you try with kvm.git next branch? > > > > With next branch db6ae6158186a17165ef990bda2895ae7594b039, > > > Can you revert a0f155e9646d5f1c263f6f9aae880151100243bb and try again? The issue is gone with a0f155e9646d5f1c263f6f9aae880151100243bb reverted. > > [ 53.386936] device tap0 entered promiscuous mode > > [ 53.386990] br0: port 2(tap0) entered forwarding state > > [ 53.386997] br0: port 2(tap0) entered forwarding state > > [ 80.096275] BUG: unable to handle kernel NULL pointer dereference at (null) > > [ 80.098592] IP: [<ffffffff81c0721e>] _raw_spin_lock+0xe/0x30 > > [ 80.100988] PGD 0 > > [ 80.103339] Oops: 0002 [#1] SMP > > [ 80.105668] Modules linked in: vhost_net > > [ 80.108005] CPU 6 > > [ 80.108026] Pid: 4257, comm: qemu-system-x86 Not tainted 3.9.0-rc3+ #757 Dell Inc. OptiPlex 790/0V5HMK > > [ 80.112662] RIP: 0010:[<ffffffff81c0721e>] [<ffffffff81c0721e>] _raw_spin_lock+0xe/0x30 > > [ 80.115030] RSP: 0018:ffff880221721cc8 EFLAGS: 00010046 > > [ 80.117365] RAX: 0000000000000100 RBX: ffff88022dcc003f RCX: ffff880221734950 > > [ 80.119688] RDX: ffff8802208f6ca8 RSI: 000000007fffffff RDI: 0000000000000000 > > [ 80.121982] RBP: ffff880221721cc8 R08: 0000000000000002 R09: 0000000000000002 > > [ 80.124230] R10: 00007f7fd01087e0 R11: 0000000000000246 R12: ffff8802208f6ca8 > > [ 80.126489] R13: 0000000000000080 R14: ffff880223e2a900 R15: 0000000000000000 > > [ 80.128718] FS: 00007f7fd38488e0(0000) GS:ffff88022dcc0000(0000) knlGS:0000000000000000 > > [ 80.130921] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > > [ 80.133130] CR2: 0000000000000000 CR3: 000000022309f000 CR4: 00000000000427e0 > > [ 80.135345] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 > > [ 80.137543] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 > > [ 80.139708] Process qemu-system-x86 (pid: 4257, threadinfo ffff880221720000, task ffff880222bd5640) > > [ 80.141887] Stack: > > [ 80.144064] ffff880221721d08 ffffffff810ac5c5 ffff88022431dc00 0000000000000086 > > [ 80.146316] 0000000000000080 ffff880223e2a900 ffff8802208f6ca8 0000000000000000 > > [ 80.148558] ffff880221721d48 ffffffff810ac8fe 0000000000000000 ffff880221734000 > > [ 80.150784] Call Trace: > > [ 80.152975] [<ffffffff810ac5c5>] __queue_work+0x45/0x2d0 > > [ 80.155193] [<ffffffff810ac8fe>] queue_work_on+0x8e/0xa0 > > [ 80.157374] [<ffffffff810ac949>] queue_work+0x19/0x20 > > [ 80.159505] [<ffffffff81009b6b>] irqfd_deactivate+0x4b/0x60 > > [ 80.161621] [<ffffffff8100a69d>] kvm_irqfd+0x39d/0x580 > > [ 80.163717] [<ffffffff81007a27>] kvm_vm_ioctl+0x207/0x5b0 > > [ 80.165832] [<ffffffff810c9545>] ? update_curr+0xf5/0x180 > > [ 80.167931] [<ffffffff811b66e8>] do_vfs_ioctl+0x98/0x550 > > [ 80.170013] [<ffffffff810c1f5e>] ? finish_task_switch+0x4e/0xe0 > > [ 80.172109] [<ffffffff81c054aa>] ? __schedule+0x2ea/0x710 > > [ 80.174218] [<ffffffff811b6bf7>] sys_ioctl+0x57/0x90 > > [ 80.176338] [<ffffffff8140ae9e>] ? trace_hardirqs_on_thunk+0x3a/0x3c > > [ 80.178467] [<ffffffff81c0f602>] system_call_fastpath+0x16/0x1b > > [ 80.180598] Code: c1 ea 08 38 c2 74 0f 66 0f 1f 44 00 00 f3 90 0f b6 03 38 c2 75 f7 48 83 c4 08 5b c9 c3 55 48 89 e5 66 66 66 66 90 b8 00 01 00 00 <f0> 66 0f c1 07 89 c2 66 c1 ea 08 38 c2 74 0c 0f 1f 00 f3 90 0f > > [ 80.183098] RIP [<ffffffff81c0721e>] _raw_spin_lock+0xe/0x30 > > [ 80.185372] RSP <ffff880221721cc8> > > [ 80.187584] CR2: 0000000000000000 > > [ 80.208627] ---[ end trace 13fb1e4b6e5ab21f ]--- > > > > > -- > > > Gleb. > > > > -- > > Asias > > -- > Gleb. -- Asias ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: irqfd issue 2013-05-07 9:10 ` Asias He @ 2013-05-07 9:12 ` Gleb Natapov 2013-05-07 9:21 ` Cornelia Huck 0 siblings, 1 reply; 13+ messages in thread From: Gleb Natapov @ 2013-05-07 9:12 UTC (permalink / raw) To: Asias He; +Cc: kvm, Michael S. Tsirkin, Cornelia Huck On Tue, May 07, 2013 at 05:10:41PM +0800, Asias He wrote: > On Tue, May 07, 2013 at 11:42:07AM +0300, Gleb Natapov wrote: > > On Tue, May 07, 2013 at 04:36:50PM +0800, Asias He wrote: > > > On Tue, May 07, 2013 at 11:18:38AM +0300, Gleb Natapov wrote: > > > > On Tue, May 07, 2013 at 04:14:50PM +0800, Asias He wrote: > > > > > On Tue, May 07, 2013 at 10:55:36AM +0300, Gleb Natapov wrote: > > > > > > On Tue, May 07, 2013 at 09:37:30AM +0800, Asias He wrote: > > > > > > > Hi, > > > > > > > > > > > > > > I am seeing this with linus/master. Any ideas? > > > > > > > > > > > > > How reproducible it this? What HEAD are you seeing this with? > > > > > > > > > > Almost always. Start a guest with vhost-net on. > > > > > > > > > It happens during start or on VM exit? The trace shows do_exit() > > > > > > > > > QEMU : e3351000cd682200835763caca87adf708ed1c65 > > > > > KERNEL: 51a26ae7a14b85c99c9be470c2d28eeeba0f26a3 > > > > > > > > > Can you try with kvm.git next branch? > > > > > > With next branch db6ae6158186a17165ef990bda2895ae7594b039, > > > > > Can you revert a0f155e9646d5f1c263f6f9aae880151100243bb and try again? > > The issue is gone with a0f155e9646d5f1c263f6f9aae880151100243bb > reverted. > Cornelia, any ideas? > > > [ 53.386936] device tap0 entered promiscuous mode > > > [ 53.386990] br0: port 2(tap0) entered forwarding state > > > [ 53.386997] br0: port 2(tap0) entered forwarding state > > > [ 80.096275] BUG: unable to handle kernel NULL pointer dereference at (null) > > > [ 80.098592] IP: [<ffffffff81c0721e>] _raw_spin_lock+0xe/0x30 > > > [ 80.100988] PGD 0 > > > [ 80.103339] Oops: 0002 [#1] SMP > > > [ 80.105668] Modules linked in: vhost_net > > > [ 80.108005] CPU 6 > > > [ 80.108026] Pid: 4257, comm: qemu-system-x86 Not tainted 3.9.0-rc3+ #757 Dell Inc. OptiPlex 790/0V5HMK > > > [ 80.112662] RIP: 0010:[<ffffffff81c0721e>] [<ffffffff81c0721e>] _raw_spin_lock+0xe/0x30 > > > [ 80.115030] RSP: 0018:ffff880221721cc8 EFLAGS: 00010046 > > > [ 80.117365] RAX: 0000000000000100 RBX: ffff88022dcc003f RCX: ffff880221734950 > > > [ 80.119688] RDX: ffff8802208f6ca8 RSI: 000000007fffffff RDI: 0000000000000000 > > > [ 80.121982] RBP: ffff880221721cc8 R08: 0000000000000002 R09: 0000000000000002 > > > [ 80.124230] R10: 00007f7fd01087e0 R11: 0000000000000246 R12: ffff8802208f6ca8 > > > [ 80.126489] R13: 0000000000000080 R14: ffff880223e2a900 R15: 0000000000000000 > > > [ 80.128718] FS: 00007f7fd38488e0(0000) GS:ffff88022dcc0000(0000) knlGS:0000000000000000 > > > [ 80.130921] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > > > [ 80.133130] CR2: 0000000000000000 CR3: 000000022309f000 CR4: 00000000000427e0 > > > [ 80.135345] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 > > > [ 80.137543] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 > > > [ 80.139708] Process qemu-system-x86 (pid: 4257, threadinfo ffff880221720000, task ffff880222bd5640) > > > [ 80.141887] Stack: > > > [ 80.144064] ffff880221721d08 ffffffff810ac5c5 ffff88022431dc00 0000000000000086 > > > [ 80.146316] 0000000000000080 ffff880223e2a900 ffff8802208f6ca8 0000000000000000 > > > [ 80.148558] ffff880221721d48 ffffffff810ac8fe 0000000000000000 ffff880221734000 > > > [ 80.150784] Call Trace: > > > [ 80.152975] [<ffffffff810ac5c5>] __queue_work+0x45/0x2d0 > > > [ 80.155193] [<ffffffff810ac8fe>] queue_work_on+0x8e/0xa0 > > > [ 80.157374] [<ffffffff810ac949>] queue_work+0x19/0x20 > > > [ 80.159505] [<ffffffff81009b6b>] irqfd_deactivate+0x4b/0x60 > > > [ 80.161621] [<ffffffff8100a69d>] kvm_irqfd+0x39d/0x580 > > > [ 80.163717] [<ffffffff81007a27>] kvm_vm_ioctl+0x207/0x5b0 > > > [ 80.165832] [<ffffffff810c9545>] ? update_curr+0xf5/0x180 > > > [ 80.167931] [<ffffffff811b66e8>] do_vfs_ioctl+0x98/0x550 > > > [ 80.170013] [<ffffffff810c1f5e>] ? finish_task_switch+0x4e/0xe0 > > > [ 80.172109] [<ffffffff81c054aa>] ? __schedule+0x2ea/0x710 > > > [ 80.174218] [<ffffffff811b6bf7>] sys_ioctl+0x57/0x90 > > > [ 80.176338] [<ffffffff8140ae9e>] ? trace_hardirqs_on_thunk+0x3a/0x3c > > > [ 80.178467] [<ffffffff81c0f602>] system_call_fastpath+0x16/0x1b > > > [ 80.180598] Code: c1 ea 08 38 c2 74 0f 66 0f 1f 44 00 00 f3 90 0f b6 03 38 c2 75 f7 48 83 c4 08 5b c9 c3 55 48 89 e5 66 66 66 66 90 b8 00 01 00 00 <f0> 66 0f c1 07 89 c2 66 c1 ea 08 38 c2 74 0c 0f 1f 00 f3 90 0f > > > [ 80.183098] RIP [<ffffffff81c0721e>] _raw_spin_lock+0xe/0x30 > > > [ 80.185372] RSP <ffff880221721cc8> > > > [ 80.187584] CR2: 0000000000000000 > > > [ 80.208627] ---[ end trace 13fb1e4b6e5ab21f ]--- > > > > > > > -- > > > > Gleb. > > > > > > -- > > > Asias > > > > -- > > Gleb. > > -- > Asias -- Gleb. ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: irqfd issue 2013-05-07 9:12 ` Gleb Natapov @ 2013-05-07 9:21 ` Cornelia Huck 2013-05-07 11:07 ` Cornelia Huck 0 siblings, 1 reply; 13+ messages in thread From: Cornelia Huck @ 2013-05-07 9:21 UTC (permalink / raw) To: Gleb Natapov; +Cc: Asias He, kvm, Michael S. Tsirkin On Tue, 7 May 2013 12:12:09 +0300 Gleb Natapov <gleb@redhat.com> wrote: > On Tue, May 07, 2013 at 05:10:41PM +0800, Asias He wrote: > > On Tue, May 07, 2013 at 11:42:07AM +0300, Gleb Natapov wrote: > > > On Tue, May 07, 2013 at 04:36:50PM +0800, Asias He wrote: > > > > On Tue, May 07, 2013 at 11:18:38AM +0300, Gleb Natapov wrote: > > > > > On Tue, May 07, 2013 at 04:14:50PM +0800, Asias He wrote: > > > > > > On Tue, May 07, 2013 at 10:55:36AM +0300, Gleb Natapov wrote: > > > > > > > On Tue, May 07, 2013 at 09:37:30AM +0800, Asias He wrote: > > > > > > > > Hi, > > > > > > > > > > > > > > > > I am seeing this with linus/master. Any ideas? > > > > > > > > > > > > > > > How reproducible it this? What HEAD are you seeing this with? > > > > > > > > > > > > Almost always. Start a guest with vhost-net on. > > > > > > > > > > > It happens during start or on VM exit? The trace shows do_exit() > > > > > > > > > > > QEMU : e3351000cd682200835763caca87adf708ed1c65 > > > > > > KERNEL: 51a26ae7a14b85c99c9be470c2d28eeeba0f26a3 > > > > > > > > > > > Can you try with kvm.git next branch? > > > > > > > > With next branch db6ae6158186a17165ef990bda2895ae7594b039, > > > > > > > Can you revert a0f155e9646d5f1c263f6f9aae880151100243bb and try again? > > > > The issue is gone with a0f155e9646d5f1c263f6f9aae880151100243bb > > reverted. > > > Cornelia, any ideas? irqfd_deactivate before kvm_init or after kvm_exit? Let me look... > > > > > [ 53.386936] device tap0 entered promiscuous mode > > > > [ 53.386990] br0: port 2(tap0) entered forwarding state > > > > [ 53.386997] br0: port 2(tap0) entered forwarding state > > > > [ 80.096275] BUG: unable to handle kernel NULL pointer dereference at (null) > > > > [ 80.098592] IP: [<ffffffff81c0721e>] _raw_spin_lock+0xe/0x30 > > > > [ 80.100988] PGD 0 > > > > [ 80.103339] Oops: 0002 [#1] SMP > > > > [ 80.105668] Modules linked in: vhost_net > > > > [ 80.108005] CPU 6 > > > > [ 80.108026] Pid: 4257, comm: qemu-system-x86 Not tainted 3.9.0-rc3+ #757 Dell Inc. OptiPlex 790/0V5HMK > > > > [ 80.112662] RIP: 0010:[<ffffffff81c0721e>] [<ffffffff81c0721e>] _raw_spin_lock+0xe/0x30 > > > > [ 80.115030] RSP: 0018:ffff880221721cc8 EFLAGS: 00010046 > > > > [ 80.117365] RAX: 0000000000000100 RBX: ffff88022dcc003f RCX: ffff880221734950 > > > > [ 80.119688] RDX: ffff8802208f6ca8 RSI: 000000007fffffff RDI: 0000000000000000 > > > > [ 80.121982] RBP: ffff880221721cc8 R08: 0000000000000002 R09: 0000000000000002 > > > > [ 80.124230] R10: 00007f7fd01087e0 R11: 0000000000000246 R12: ffff8802208f6ca8 > > > > [ 80.126489] R13: 0000000000000080 R14: ffff880223e2a900 R15: 0000000000000000 > > > > [ 80.128718] FS: 00007f7fd38488e0(0000) GS:ffff88022dcc0000(0000) knlGS:0000000000000000 > > > > [ 80.130921] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > > > > [ 80.133130] CR2: 0000000000000000 CR3: 000000022309f000 CR4: 00000000000427e0 > > > > [ 80.135345] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 > > > > [ 80.137543] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 > > > > [ 80.139708] Process qemu-system-x86 (pid: 4257, threadinfo ffff880221720000, task ffff880222bd5640) > > > > [ 80.141887] Stack: > > > > [ 80.144064] ffff880221721d08 ffffffff810ac5c5 ffff88022431dc00 0000000000000086 > > > > [ 80.146316] 0000000000000080 ffff880223e2a900 ffff8802208f6ca8 0000000000000000 > > > > [ 80.148558] ffff880221721d48 ffffffff810ac8fe 0000000000000000 ffff880221734000 > > > > [ 80.150784] Call Trace: > > > > [ 80.152975] [<ffffffff810ac5c5>] __queue_work+0x45/0x2d0 > > > > [ 80.155193] [<ffffffff810ac8fe>] queue_work_on+0x8e/0xa0 > > > > [ 80.157374] [<ffffffff810ac949>] queue_work+0x19/0x20 > > > > [ 80.159505] [<ffffffff81009b6b>] irqfd_deactivate+0x4b/0x60 > > > > [ 80.161621] [<ffffffff8100a69d>] kvm_irqfd+0x39d/0x580 > > > > [ 80.163717] [<ffffffff81007a27>] kvm_vm_ioctl+0x207/0x5b0 > > > > [ 80.165832] [<ffffffff810c9545>] ? update_curr+0xf5/0x180 > > > > [ 80.167931] [<ffffffff811b66e8>] do_vfs_ioctl+0x98/0x550 > > > > [ 80.170013] [<ffffffff810c1f5e>] ? finish_task_switch+0x4e/0xe0 > > > > [ 80.172109] [<ffffffff81c054aa>] ? __schedule+0x2ea/0x710 > > > > [ 80.174218] [<ffffffff811b6bf7>] sys_ioctl+0x57/0x90 > > > > [ 80.176338] [<ffffffff8140ae9e>] ? trace_hardirqs_on_thunk+0x3a/0x3c > > > > [ 80.178467] [<ffffffff81c0f602>] system_call_fastpath+0x16/0x1b > > > > [ 80.180598] Code: c1 ea 08 38 c2 74 0f 66 0f 1f 44 00 00 f3 90 0f b6 03 38 c2 75 f7 48 83 c4 08 5b c9 c3 55 48 89 e5 66 66 66 66 90 b8 00 01 00 00 <f0> 66 0f c1 07 89 c2 66 c1 ea 08 38 c2 74 0c 0f 1f 00 f3 90 0f > > > > [ 80.183098] RIP [<ffffffff81c0721e>] _raw_spin_lock+0xe/0x30 > > > > [ 80.185372] RSP <ffff880221721cc8> > > > > [ 80.187584] CR2: 0000000000000000 > > > > [ 80.208627] ---[ end trace 13fb1e4b6e5ab21f ]--- > > > > ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: irqfd issue 2013-05-07 9:21 ` Cornelia Huck @ 2013-05-07 11:07 ` Cornelia Huck 2013-05-07 14:29 ` Gleb Natapov 0 siblings, 1 reply; 13+ messages in thread From: Cornelia Huck @ 2013-05-07 11:07 UTC (permalink / raw) To: Asias He; +Cc: Gleb Natapov, kvm, Michael S. Tsirkin On Tue, 7 May 2013 11:21:09 +0200 Cornelia Huck <cornelia.huck@de.ibm.com> wrote: > On Tue, 7 May 2013 12:12:09 +0300 > Gleb Natapov <gleb@redhat.com> wrote: > > > On Tue, May 07, 2013 at 05:10:41PM +0800, Asias He wrote: > > > On Tue, May 07, 2013 at 11:42:07AM +0300, Gleb Natapov wrote: > > > > On Tue, May 07, 2013 at 04:36:50PM +0800, Asias He wrote: > > > > > On Tue, May 07, 2013 at 11:18:38AM +0300, Gleb Natapov wrote: > > > > > > On Tue, May 07, 2013 at 04:14:50PM +0800, Asias He wrote: > > > > > > > On Tue, May 07, 2013 at 10:55:36AM +0300, Gleb Natapov wrote: > > > > > > > > On Tue, May 07, 2013 at 09:37:30AM +0800, Asias He wrote: > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > I am seeing this with linus/master. Any ideas? > > > > > > > > > > > > > > > > > How reproducible it this? What HEAD are you seeing this with? > > > > > > > > > > > > > > Almost always. Start a guest with vhost-net on. > > > > > > > > > > > > > It happens during start or on VM exit? The trace shows do_exit() > > > > > > > > > > > > > QEMU : e3351000cd682200835763caca87adf708ed1c65 > > > > > > > KERNEL: 51a26ae7a14b85c99c9be470c2d28eeeba0f26a3 > > > > > > > > > > > > > Can you try with kvm.git next branch? > > > > > > > > > > With next branch db6ae6158186a17165ef990bda2895ae7594b039, > > > > > > > > > Can you revert a0f155e9646d5f1c263f6f9aae880151100243bb and try again? > > > > > > The issue is gone with a0f155e9646d5f1c263f6f9aae880151100243bb > > > reverted. > > > > > Cornelia, any ideas? > > irqfd_deactivate before kvm_init or after kvm_exit? Let me look... > Hm, no idea. Asias, could you try whether the patch below yields any interesting backtraces? diff --git a/virt/kvm/eventfd.c b/virt/kvm/eventfd.c index 64ee720..59c3810 100644 --- a/virt/kvm/eventfd.c +++ b/virt/kvm/eventfd.c @@ -92,6 +92,8 @@ struct _irqfd { }; static struct workqueue_struct *irqfd_cleanup_wq; +static int irqfd_ready; +static unsigned long irqfd_cnt; static void irqfd_inject(struct work_struct *work) @@ -410,6 +412,8 @@ kvm_irqfd_assign(struct kvm *kvm, struct kvm_irqfd *args) */ fput(file); + irqfd_cnt++; + return 0; fail: @@ -483,12 +487,16 @@ kvm_irqfd_deassign(struct kvm *kvm, struct kvm_irqfd *args) */ flush_workqueue(irqfd_cleanup_wq); + irqfd_cnt--; + return 0; } int kvm_irqfd(struct kvm *kvm, struct kvm_irqfd *args) { + WARN_ON(!irqfd_ready); + if (args->flags & ~(KVM_IRQFD_FLAG_DEASSIGN | KVM_IRQFD_FLAG_RESAMPLE)) return -EINVAL; @@ -548,15 +556,22 @@ void kvm_irq_routing_update(struct kvm *kvm, */ int kvm_irqfd_init(void) { + WARN_ON(irqfd_cnt); + irqfd_cleanup_wq = create_singlethread_workqueue("kvm-irqfd-cleanup"); if (!irqfd_cleanup_wq) return -ENOMEM; + irqfd_ready = 1; + return 0; } void kvm_irqfd_exit(void) { + WARN_ON(irqfd_cnt); + irqfd_ready = 0; + destroy_workqueue(irqfd_cleanup_wq); } #endif ^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: irqfd issue 2013-05-07 11:07 ` Cornelia Huck @ 2013-05-07 14:29 ` Gleb Natapov 2013-05-07 14:38 ` Asias He 0 siblings, 1 reply; 13+ messages in thread From: Gleb Natapov @ 2013-05-07 14:29 UTC (permalink / raw) To: Cornelia Huck; +Cc: Asias He, kvm, Michael S. Tsirkin On Tue, May 07, 2013 at 01:07:06PM +0200, Cornelia Huck wrote: > On Tue, 7 May 2013 11:21:09 +0200 > Cornelia Huck <cornelia.huck@de.ibm.com> wrote: > > > On Tue, 7 May 2013 12:12:09 +0300 > > Gleb Natapov <gleb@redhat.com> wrote: > > > > > On Tue, May 07, 2013 at 05:10:41PM +0800, Asias He wrote: > > > > On Tue, May 07, 2013 at 11:42:07AM +0300, Gleb Natapov wrote: > > > > > On Tue, May 07, 2013 at 04:36:50PM +0800, Asias He wrote: > > > > > > On Tue, May 07, 2013 at 11:18:38AM +0300, Gleb Natapov wrote: > > > > > > > On Tue, May 07, 2013 at 04:14:50PM +0800, Asias He wrote: > > > > > > > > On Tue, May 07, 2013 at 10:55:36AM +0300, Gleb Natapov wrote: > > > > > > > > > On Tue, May 07, 2013 at 09:37:30AM +0800, Asias He wrote: > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > I am seeing this with linus/master. Any ideas? > > > > > > > > > > > > > > > > > > > How reproducible it this? What HEAD are you seeing this with? > > > > > > > > > > > > > > > > Almost always. Start a guest with vhost-net on. > > > > > > > > > > > > > > > It happens during start or on VM exit? The trace shows do_exit() > > > > > > > > > > > > > > > QEMU : e3351000cd682200835763caca87adf708ed1c65 > > > > > > > > KERNEL: 51a26ae7a14b85c99c9be470c2d28eeeba0f26a3 > > > > > > > > > > > > > > > Can you try with kvm.git next branch? > > > > > > > > > > > > With next branch db6ae6158186a17165ef990bda2895ae7594b039, > > > > > > > > > > > Can you revert a0f155e9646d5f1c263f6f9aae880151100243bb and try again? > > > > > > > > The issue is gone with a0f155e9646d5f1c263f6f9aae880151100243bb > > > > reverted. > > > > > > > Cornelia, any ideas? > > > > irqfd_deactivate before kvm_init or after kvm_exit? Let me look... > > > > Hm, no idea. > For am quick look is seems that after the patch irqfd_cleanup_wq is initialized as part of kvm-intel/kvm-amd module, but before the patch is was initialized as part of kvm module. The later is how it should be, but I do not see why would it cause the problem. > Asias, could you try whether the patch below yields any interesting > backtraces? > > diff --git a/virt/kvm/eventfd.c b/virt/kvm/eventfd.c > index 64ee720..59c3810 100644 > --- a/virt/kvm/eventfd.c > +++ b/virt/kvm/eventfd.c > @@ -92,6 +92,8 @@ struct _irqfd { > }; > > static struct workqueue_struct *irqfd_cleanup_wq; > +static int irqfd_ready; > +static unsigned long irqfd_cnt; > > static void > irqfd_inject(struct work_struct *work) > @@ -410,6 +412,8 @@ kvm_irqfd_assign(struct kvm *kvm, struct kvm_irqfd *args) > */ > fput(file); > > + irqfd_cnt++; > + > return 0; > > fail: > @@ -483,12 +487,16 @@ kvm_irqfd_deassign(struct kvm *kvm, struct kvm_irqfd *args) > */ > flush_workqueue(irqfd_cleanup_wq); > > + irqfd_cnt--; > + > return 0; > } > > int > kvm_irqfd(struct kvm *kvm, struct kvm_irqfd *args) > { > + WARN_ON(!irqfd_ready); > + > if (args->flags & ~(KVM_IRQFD_FLAG_DEASSIGN | KVM_IRQFD_FLAG_RESAMPLE)) > return -EINVAL; > > @@ -548,15 +556,22 @@ void kvm_irq_routing_update(struct kvm *kvm, > */ > int kvm_irqfd_init(void) > { > + WARN_ON(irqfd_cnt); > + > irqfd_cleanup_wq = create_singlethread_workqueue("kvm-irqfd-cleanup"); > if (!irqfd_cleanup_wq) > return -ENOMEM; > > + irqfd_ready = 1; > + > return 0; > } > > void kvm_irqfd_exit(void) > { > + WARN_ON(irqfd_cnt); > + irqfd_ready = 0; > + > destroy_workqueue(irqfd_cleanup_wq); > } > #endif -- Gleb. ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: irqfd issue 2013-05-07 14:29 ` Gleb Natapov @ 2013-05-07 14:38 ` Asias He 0 siblings, 0 replies; 13+ messages in thread From: Asias He @ 2013-05-07 14:38 UTC (permalink / raw) To: Gleb Natapov; +Cc: Cornelia Huck, kvm, Michael S. Tsirkin On Tue, May 07, 2013 at 05:29:09PM +0300, Gleb Natapov wrote: > On Tue, May 07, 2013 at 01:07:06PM +0200, Cornelia Huck wrote: > > On Tue, 7 May 2013 11:21:09 +0200 > > Cornelia Huck <cornelia.huck@de.ibm.com> wrote: > > > > > On Tue, 7 May 2013 12:12:09 +0300 > > > Gleb Natapov <gleb@redhat.com> wrote: > > > > > > > On Tue, May 07, 2013 at 05:10:41PM +0800, Asias He wrote: > > > > > On Tue, May 07, 2013 at 11:42:07AM +0300, Gleb Natapov wrote: > > > > > > On Tue, May 07, 2013 at 04:36:50PM +0800, Asias He wrote: > > > > > > > On Tue, May 07, 2013 at 11:18:38AM +0300, Gleb Natapov wrote: > > > > > > > > On Tue, May 07, 2013 at 04:14:50PM +0800, Asias He wrote: > > > > > > > > > On Tue, May 07, 2013 at 10:55:36AM +0300, Gleb Natapov wrote: > > > > > > > > > > On Tue, May 07, 2013 at 09:37:30AM +0800, Asias He wrote: > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > I am seeing this with linus/master. Any ideas? > > > > > > > > > > > > > > > > > > > > > How reproducible it this? What HEAD are you seeing this with? > > > > > > > > > > > > > > > > > > Almost always. Start a guest with vhost-net on. > > > > > > > > > > > > > > > > > It happens during start or on VM exit? The trace shows do_exit() > > > > > > > > > > > > > > > > > QEMU : e3351000cd682200835763caca87adf708ed1c65 > > > > > > > > > KERNEL: 51a26ae7a14b85c99c9be470c2d28eeeba0f26a3 > > > > > > > > > > > > > > > > > Can you try with kvm.git next branch? > > > > > > > > > > > > > > With next branch db6ae6158186a17165ef990bda2895ae7594b039, > > > > > > > > > > > > > Can you revert a0f155e9646d5f1c263f6f9aae880151100243bb and try again? > > > > > > > > > > The issue is gone with a0f155e9646d5f1c263f6f9aae880151100243bb > > > > > reverted. > > > > > > > > > Cornelia, any ideas? > > > > > > irqfd_deactivate before kvm_init or after kvm_exit? Let me look... > > > > > > > Hm, no idea. > > > For am quick look is seems that after the patch irqfd_cleanup_wq is > initialized as part of kvm-intel/kvm-amd module, but before the patch is > was initialized as part of kvm module. The later is how it should be, > but I do not see why would it cause the problem. I have a fix for this now. Will send out patch shortly. > > Asias, could you try whether the patch below yields any interesting > > backtraces? > > > > diff --git a/virt/kvm/eventfd.c b/virt/kvm/eventfd.c > > index 64ee720..59c3810 100644 > > --- a/virt/kvm/eventfd.c > > +++ b/virt/kvm/eventfd.c > > @@ -92,6 +92,8 @@ struct _irqfd { > > }; > > > > static struct workqueue_struct *irqfd_cleanup_wq; > > +static int irqfd_ready; > > +static unsigned long irqfd_cnt; > > > > static void > > irqfd_inject(struct work_struct *work) > > @@ -410,6 +412,8 @@ kvm_irqfd_assign(struct kvm *kvm, struct kvm_irqfd *args) > > */ > > fput(file); > > > > + irqfd_cnt++; > > + > > return 0; > > > > fail: > > @@ -483,12 +487,16 @@ kvm_irqfd_deassign(struct kvm *kvm, struct kvm_irqfd *args) > > */ > > flush_workqueue(irqfd_cleanup_wq); > > > > + irqfd_cnt--; > > + > > return 0; > > } > > > > int > > kvm_irqfd(struct kvm *kvm, struct kvm_irqfd *args) > > { > > + WARN_ON(!irqfd_ready); > > + > > if (args->flags & ~(KVM_IRQFD_FLAG_DEASSIGN | KVM_IRQFD_FLAG_RESAMPLE)) > > return -EINVAL; > > > > @@ -548,15 +556,22 @@ void kvm_irq_routing_update(struct kvm *kvm, > > */ > > int kvm_irqfd_init(void) > > { > > + WARN_ON(irqfd_cnt); > > + > > irqfd_cleanup_wq = create_singlethread_workqueue("kvm-irqfd-cleanup"); > > if (!irqfd_cleanup_wq) > > return -ENOMEM; > > > > + irqfd_ready = 1; > > + > > return 0; > > } > > > > void kvm_irqfd_exit(void) > > { > > + WARN_ON(irqfd_cnt); > > + irqfd_ready = 0; > > + > > destroy_workqueue(irqfd_cleanup_wq); > > } > > #endif > > -- > Gleb. -- Asias ^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2013-05-07 14:38 UTC | newest] Thread overview: 13+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2013-05-07 1:37 irqfd issue Asias He 2013-05-07 7:55 ` Gleb Natapov 2013-05-07 8:14 ` Asias He 2013-05-07 8:18 ` Gleb Natapov 2013-05-07 8:28 ` Asias He 2013-05-07 8:36 ` Asias He 2013-05-07 8:42 ` Gleb Natapov 2013-05-07 9:10 ` Asias He 2013-05-07 9:12 ` Gleb Natapov 2013-05-07 9:21 ` Cornelia Huck 2013-05-07 11:07 ` Cornelia Huck 2013-05-07 14:29 ` Gleb Natapov 2013-05-07 14:38 ` Asias He
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox