* qemu-kvm loops after kernel udpate @ 2012-09-11 18:11 Jiri Slaby 2012-09-11 19:03 ` Marcelo Tosatti 0 siblings, 1 reply; 16+ messages in thread From: Jiri Slaby @ 2012-09-11 18:11 UTC (permalink / raw) To: Avi Kivity, Marcelo Tosatti, kvm, LKML, Jiri Slaby Hi, it looks like an update from next-20120824 to next-20120910 makes kvm defunct. When I try to run qemu, it loops forever without printing anything on the monitor. -no-kvm makes it indeed work. Cmdline I use: qemu-kvm -k en-us -usbdevice tablet -balloon virtio -hda IMAGE -smp 2 -m 1000M -net user -net nic,model=e1000 -usb -serial pty Before I start investigating further, like biscection, is this a known issue already and fix available somewhere? thanks, -- js suse labs ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: qemu-kvm loops after kernel udpate 2012-09-11 18:11 qemu-kvm loops after kernel udpate Jiri Slaby @ 2012-09-11 19:03 ` Marcelo Tosatti 2012-09-11 19:41 ` Jiri Slaby 0 siblings, 1 reply; 16+ messages in thread From: Marcelo Tosatti @ 2012-09-11 19:03 UTC (permalink / raw) To: Jiri Slaby; +Cc: Avi Kivity, kvm, LKML, Jiri Slaby On Tue, Sep 11, 2012 at 08:11:36PM +0200, Jiri Slaby wrote: > Hi, > > it looks like an update from next-20120824 to next-20120910 makes kvm > defunct. When I try to run qemu, it loops forever without printing > anything on the monitor. > > -no-kvm makes it indeed work. > > Cmdline I use: > qemu-kvm -k en-us -usbdevice tablet -balloon virtio -hda IMAGE -smp 2 -m > 1000M -net user -net nic,model=e1000 -usb -serial pty > > Before I start investigating further, like biscection, is this a known > issue already and fix available somewhere? > > thanks, Its not a known issue. 'info registers' (for both vcpus) and 'x /20i $eip' might help track it down. ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: qemu-kvm loops after kernel udpate 2012-09-11 19:03 ` Marcelo Tosatti @ 2012-09-11 19:41 ` Jiri Slaby 2012-09-12 8:06 ` Avi Kivity 0 siblings, 1 reply; 16+ messages in thread From: Jiri Slaby @ 2012-09-11 19:41 UTC (permalink / raw) To: Marcelo Tosatti; +Cc: Avi Kivity, kvm, LKML, Jiri Slaby On 09/11/2012 09:03 PM, Marcelo Tosatti wrote: > On Tue, Sep 11, 2012 at 08:11:36PM +0200, Jiri Slaby wrote: >> Hi, >> >> it looks like an update from next-20120824 to next-20120910 makes kvm >> defunct. When I try to run qemu, it loops forever without printing >> anything on the monitor. >> >> -no-kvm makes it indeed work. >> >> Cmdline I use: >> qemu-kvm -k en-us -usbdevice tablet -balloon virtio -hda IMAGE -smp 2 -m >> 1000M -net user -net nic,model=e1000 -usb -serial pty >> >> Before I start investigating further, like biscection, is this a known >> issue already and fix available somewhere? > > Its not a known issue. 'info registers' (for both vcpus) and > 'x /20i $eip' might help track it down. I don't think it's in a state with something loaded. (qemu) cpu 0 (qemu) info registers EAX=00000000 EBX=00000000 ECX=00000000 EDX=00000623 ESI=00000000 EDI=00000000 EBP=00000000 ESP=00000000 EIP=0000fff0 EFL=00010002 [-------] CPL=3 II=0 A20=1 SMM=0 HLT=0 ES =0000 00000000 0000ffff 00009300 CS =f000 000f0000 0000ffff 0000f300 SS =0000 00000000 0000ffff 0000f300 DS =0000 00000000 0000ffff 00009300 FS =0000 00000000 0000ffff 00009300 GS =0000 00000000 0000ffff 00009300 LDT=0000 00000000 0000ffff 00008200 TR =0000 00000000 0000ffff 00008b00 GDT= 00000000 0000ffff IDT= 00000000 0000ffff CR0=60000010 CR2=00000000 CR3=00000000 CR4=00000000 DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 DR6=00000000ffff0ff0 DR7=0000000000000400 EFER=0000000000000000 FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00001f80 FPR0=0000000000000000 0000 FPR1=0000000000000000 0000 FPR2=0000000000000000 0000 FPR3=0000000000000000 0000 FPR4=0000000000000000 0000 FPR5=0000000000000000 0000 FPR6=0000000000000000 0000 FPR7=0000000000000000 0000 XMM00=00000000000000000000000000000000 XMM01=00000000000000000000000000000000 XMM02=00000000000000000000000000000000 XMM03=00000000000000000000000000000000 XMM04=00000000000000000000000000000000 XMM05=00000000000000000000000000000000 XMM06=00000000000000000000000000000000 XMM07=00000000000000000000000000000000 (qemu) cpu 1 (qemu) info registers EAX=00000000 EBX=00000000 ECX=00000000 EDX=00000623 ESI=00000000 EDI=00000000 EBP=00000000 ESP=00000000 EIP=0000fff0 EFL=00000002 [-------] CPL=3 II=0 A20=1 SMM=0 HLT=0 ES =0000 00000000 0000ffff 00009300 CS =f000 000f0000 0000ffff 0000f300 SS =0000 00000000 0000ffff 0000f300 DS =0000 00000000 0000ffff 00009300 FS =0000 00000000 0000ffff 00009300 GS =0000 00000000 0000ffff 00009300 LDT=0000 00000000 0000ffff 00008200 TR =0000 00000000 0000ffff 00008b00 GDT= 00000000 0000ffff IDT= 00000000 0000ffff CR0=60000010 CR2=00000000 CR3=00000000 CR4=00000000 DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 DR6=00000000ffff0ff0 DR7=0000000000000400 EFER=0000000000000000 FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00001f80 FPR0=0000000000000000 0000 FPR1=0000000000000000 0000 FPR2=0000000000000000 0000 FPR3=0000000000000000 0000 FPR4=0000000000000000 0000 FPR5=0000000000000000 0000 FPR6=0000000000000000 0000 FPR7=0000000000000000 0000 XMM00=00000000000000000000000000000000 XMM01=00000000000000000000000000000000 XMM02=00000000000000000000000000000000 XMM03=00000000000000000000000000000000 XMM04=00000000000000000000000000000000 XMM05=00000000000000000000000000000000 XMM06=00000000000000000000000000000000 XMM07=00000000000000000000000000000000 (qemu) x /20i $eip 0x000000000000fff0: add %al,(%bx,%si) 0x000000000000fff2: add %al,(%bx,%si) 0x000000000000fff4: add %al,(%bx,%si) 0x000000000000fff6: add %al,(%bx,%si) 0x000000000000fff8: add %al,(%bx,%si) 0x000000000000fffa: add %al,(%bx,%si) 0x000000000000fffc: add %al,(%bx,%si) 0x000000000000fffe: add %al,(%bx,%si) 0x0000000000010000: add %al,(%bx,%si) 0x0000000000010002: add %al,(%bx,%si) 0x0000000000010004: add %al,(%bx,%si) 0x0000000000010006: add %al,(%bx,%si) 0x0000000000010008: add %al,(%bx,%si) 0x000000000001000a: add %al,(%bx,%si) 0x000000000001000c: add %al,(%bx,%si) 0x000000000001000e: add %al,(%bx,%si) 0x0000000000010010: add %al,(%bx,%si) 0x0000000000010012: add %al,(%bx,%si) 0x0000000000010014: add %al,(%bx,%si) 0x0000000000010016: add %al,(%bx,%si) thanks, -- js suse labs ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: qemu-kvm loops after kernel udpate 2012-09-11 19:41 ` Jiri Slaby @ 2012-09-12 8:06 ` Avi Kivity 2012-09-12 8:13 ` Jiri Slaby 0 siblings, 1 reply; 16+ messages in thread From: Avi Kivity @ 2012-09-12 8:06 UTC (permalink / raw) To: Jiri Slaby; +Cc: Marcelo Tosatti, kvm, LKML, Jiri Slaby On 09/11/2012 10:41 PM, Jiri Slaby wrote: > On 09/11/2012 09:03 PM, Marcelo Tosatti wrote: >> On Tue, Sep 11, 2012 at 08:11:36PM +0200, Jiri Slaby wrote: >>> Hi, >>> >>> it looks like an update from next-20120824 to next-20120910 makes kvm >>> defunct. When I try to run qemu, it loops forever without printing >>> anything on the monitor. >>> >>> -no-kvm makes it indeed work. >>> >>> Cmdline I use: >>> qemu-kvm -k en-us -usbdevice tablet -balloon virtio -hda IMAGE -smp 2 -m >>> 1000M -net user -net nic,model=e1000 -usb -serial pty >>> >>> Before I start investigating further, like biscection, is this a known >>> issue already and fix available somewhere? >> >> Its not a known issue. 'info registers' (for both vcpus) and >> 'x /20i $eip' might help track it down. > > I don't think it's in a state with something loaded. > > (qemu) cpu 0 > (qemu) info registers > EAX=00000000 EBX=00000000 ECX=00000000 EDX=00000623 > ESI=00000000 EDI=00000000 EBP=00000000 ESP=00000000 > EIP=0000fff0 EFL=00010002 [-------] CPL=3 II=0 A20=1 SMM=0 HLT=0 > ES =0000 00000000 0000ffff 00009300 > CS =f000 000f0000 0000ffff 0000f300 This is before the first instruction is executed. You're on an Intel host, yes? Please provide the output of vmxcap (http://goo.gl/c5lUO), and a snapshot of kvm_stat while the guest is hung. -- error compiling committee.c: too many arguments to function ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: qemu-kvm loops after kernel udpate 2012-09-12 8:06 ` Avi Kivity @ 2012-09-12 8:13 ` Jiri Slaby 2012-09-12 8:18 ` Avi Kivity 0 siblings, 1 reply; 16+ messages in thread From: Jiri Slaby @ 2012-09-12 8:13 UTC (permalink / raw) To: Avi Kivity; +Cc: Jiri Slaby, Marcelo Tosatti, kvm, LKML On 09/12/2012 10:06 AM, Avi Kivity wrote: > On 09/11/2012 10:41 PM, Jiri Slaby wrote: >> On 09/11/2012 09:03 PM, Marcelo Tosatti wrote: >>> On Tue, Sep 11, 2012 at 08:11:36PM +0200, Jiri Slaby wrote: >>>> Hi, >>>> >>>> it looks like an update from next-20120824 to next-20120910 makes kvm >>>> defunct. When I try to run qemu, it loops forever without printing >>>> anything on the monitor. >>>> >>>> -no-kvm makes it indeed work. >>>> >>>> Cmdline I use: >>>> qemu-kvm -k en-us -usbdevice tablet -balloon virtio -hda IMAGE -smp 2 -m >>>> 1000M -net user -net nic,model=e1000 -usb -serial pty >>>> >>>> Before I start investigating further, like biscection, is this a known >>>> issue already and fix available somewhere? >>> >>> Its not a known issue. 'info registers' (for both vcpus) and >>> 'x /20i $eip' might help track it down. >> >> I don't think it's in a state with something loaded. >> >> (qemu) cpu 0 >> (qemu) info registers >> EAX=00000000 EBX=00000000 ECX=00000000 EDX=00000623 >> ESI=00000000 EDI=00000000 EBP=00000000 ESP=00000000 >> EIP=0000fff0 EFL=00010002 [-------] CPL=3 II=0 A20=1 SMM=0 HLT=0 >> ES =0000 00000000 0000ffff 00009300 >> CS =f000 000f0000 0000ffff 0000f300 > > This is before the first instruction is executed. > > You're on an Intel host, yes? Yes. processor : 1 vendor_id : GenuineIntel cpu family : 6 model : 15 model name : Intel(R) Core(TM)2 Duo CPU E6850 @ 3.00GHz stepping : 11 microcode : 0xba ... flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts nopl aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm lahf_lm dtherm tpr_shadow vnmi flexpriority > Please provide the output of vmxcap > (http://goo.gl/c5lUO), pin-based controls External interrupt exiting yes NMI exiting yes Virtual NMIs yes Activate VMX-preemption timer no primary processor-based controls Interrupt window exiting yes Use TSC offsetting yes HLT exiting yes INVLPG exiting yes MWAIT exiting yes RDPMC exiting yes RDTSC exiting yes CR3-load exiting forced CR3-store exiting forced CR8-load exiting yes CR8-store exiting yes Use TPR shadow yes NMI-window exiting yes MOV-DR exiting yes Unconditional I/O exiting yes Use I/O bitmaps yes Monitor trap flag no Use MSR bitmaps yes MONITOR exiting yes PAUSE exiting yes Activate secondary control yes secondary processor-based controls Virtualize APIC accesses yes Enable EPT no Descriptor-table exiting no Virtualize x2APIC mode no Enable VPID no WBINVD exiting no Unrestricted guest no PAUSE-loop exiting no RDRAND exiting no Enable INVPCID no Enable VM functions no VM-Exit controls Save debug controls forced Host address-space size yes Load IA32_PERF_GLOBAL_CTRL no Acknowledge interrupt on exit yes Save IA32_PAT no Load IA32_PAT no Save IA32_EFER no Load IA32_EFER no Save VMX-preemption timer value no VM-Entry controls Load debug controls forced IA-64 mode guest yes Entry to SMM yes Deactivate dual-monitor treatment yes Load IA32_PERF_GLOBAL_CTRL no Load IA32_PAT no Load IA32_EFER no Miscellaneous data VMX-preemption timer scale (log2) 0 Store EFER.LMA into IA-32e mode guest control no HLT activity state yes Shutdown activity state yes Wait-for-SIPI activity state yes Number of CR3-target values 4 MSR-load/store count recommenation 0 IA32_SMM_MONITOR_CTL[2] can be set to 1 no MSEG revision identifier 0 VPID and EPT capabilities Execute-only EPT translations no Page-walk length 4 no Paging-structure memory type UC no Paging-structure memory type WB no 2MB EPT pages no 1GB EPT pages no INVEPT supported no EPT accessed and dirty flags no Single-context INVEPT no All-context INVEPT no INVVPID supported no Individual-address INVVPID no Single-context INVVPID no All-context INVVPID no Single-context-retaining-globals INVVPID no VM Functions EPTP Switching no > and a snapshot of kvm_stat while the guest is hung. kvm statistics exits 6778198 615942 host_state_reload 1988 187 irq_exits 1523 138 mmu_cache_miss 4 0 fpu_reload 1 0 thanks, -- js suse labs ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: qemu-kvm loops after kernel udpate 2012-09-12 8:13 ` Jiri Slaby @ 2012-09-12 8:18 ` Avi Kivity 2012-09-12 8:20 ` Jiri Slaby 2012-09-12 18:11 ` Jiri Slaby 0 siblings, 2 replies; 16+ messages in thread From: Avi Kivity @ 2012-09-12 8:18 UTC (permalink / raw) To: Jiri Slaby; +Cc: Jiri Slaby, Marcelo Tosatti, kvm, LKML On 09/12/2012 11:13 AM, Jiri Slaby wrote: > >> Please provide the output of vmxcap >> (http://goo.gl/c5lUO), > > Unrestricted guest no The big real mode fixes. > >> and a snapshot of kvm_stat while the guest is hung. > > kvm statistics > > exits 6778198 615942 > host_state_reload 1988 187 > irq_exits 1523 138 > mmu_cache_miss 4 0 > fpu_reload 1 0 Please run this as root so we get the tracepoint based output; and press 'x' when it's running so we get more detailed output. -- error compiling committee.c: too many arguments to function ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: qemu-kvm loops after kernel udpate 2012-09-12 8:18 ` Avi Kivity @ 2012-09-12 8:20 ` Jiri Slaby 2012-09-12 18:11 ` Jiri Slaby 1 sibling, 0 replies; 16+ messages in thread From: Jiri Slaby @ 2012-09-12 8:20 UTC (permalink / raw) To: Avi Kivity; +Cc: Jiri Slaby, Marcelo Tosatti, kvm, LKML On 09/12/2012 10:18 AM, Avi Kivity wrote: > On 09/12/2012 11:13 AM, Jiri Slaby wrote: >> kvm statistics >> >> exits 6778198 615942 >> host_state_reload 1988 187 >> irq_exits 1523 138 >> mmu_cache_miss 4 0 >> fpu_reload 1 0 > > Please run this as root so we get the tracepoint based output; and press > 'x' when it's running so we get more detailed output. I need to build a kernel with trace points enabled first... I will return later today. thanks, -- js suse labs ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: qemu-kvm loops after kernel udpate 2012-09-12 8:18 ` Avi Kivity 2012-09-12 8:20 ` Jiri Slaby @ 2012-09-12 18:11 ` Jiri Slaby 2012-09-13 9:59 ` Avi Kivity 1 sibling, 1 reply; 16+ messages in thread From: Jiri Slaby @ 2012-09-12 18:11 UTC (permalink / raw) To: Avi Kivity; +Cc: Jiri Slaby, Marcelo Tosatti, kvm, LKML On 09/12/2012 10:18 AM, Avi Kivity wrote: > On 09/12/2012 11:13 AM, Jiri Slaby wrote: >> >>> Please provide the output of vmxcap >>> (http://goo.gl/c5lUO), >> >> Unrestricted guest no > > The big real mode fixes. > > >> >>> and a snapshot of kvm_stat while the guest is hung. >> >> kvm statistics >> >> exits 6778198 615942 >> host_state_reload 1988 187 >> irq_exits 1523 138 >> mmu_cache_miss 4 0 >> fpu_reload 1 0 > > Please run this as root so we get the tracepoint based output; and press > 'x' when it's running so we get more detailed output. kvm statistics kvm_exit 13798699 330708 kvm_entry 13799110 330708 kvm_page_fault 13793650 330604 kvm_exit(EXCEPTION_NMI) 6188458 330604 kvm_exit(EXTERNAL_INTERRUPT) 2169 105 kvm_exit(TPR_BELOW_THRESHOLD) 82 0 kvm_exit(IO_INSTRUCTION) 6 0 kvm_exit(PAUSE_INSTRUCTION) 5 0 kvm_exit(MCE_DURING_VMENTRY) 5 0 kvm_exit(DR_ACCESS) 5 0 kvm_exit(VMPTRLD) 5 0 kvm_exit(VMLAUNCH) 5 0 kvm_exit(INVLPG) 5 0 kvm_exit(VMCALL) 5 0 kvm_exit(WBINVD) 5 0 kvm_exit(MONITOR_INSTRUCTION) 5 0 kvm_exit(MWAIT_INSTRUCTION) 4 0 kvm_exit(NMI_WINDOW) 4 0 kvm_exit(VMREAD) 4 0 kvm_exit(XSETBV) 4 0 kvm_exit(RDPMC) 4 0 kvm_exit(CPUID) 4 0 kvm_exit(VMCLEAR) 4 0 kvm_exit(PENDING_INTERRUPT) 4 0 kvm_exit(VMRESUME) 4 0 kvm_exit(APIC_ACCESS) 4 0 kvm_exit(INVALID_STATE) 4 0 kvm_exit(MSR_READ) 4 0 kvm_exit(VMOFF) 4 0 kvm_exit(RDTSC) 4 0 kvm_exit(MSR_WRITE) 4 0 kvm_exit(VMPTRST) 4 0 kvm_exit(VMWRITE) 4 0 kvm_exit(HLT) 3 0 kvm_exit(EPT_VIOLATION) 3 0 kvm_exit(TASK_SWITCH) 3 0 kvm_exit(CR_ACCESS) 3 0 kvm_exit(TRIPLE_FAULT) 3 0 kvm_exit(VMON) 3 0 kvm_exit(EPT_MISCONFIG) 3 0 thanks, -- js suse labs ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: qemu-kvm loops after kernel udpate 2012-09-12 18:11 ` Jiri Slaby @ 2012-09-13 9:59 ` Avi Kivity 2012-09-13 18:32 ` Jiri Slaby 2012-09-18 19:40 ` Jiri Slaby 0 siblings, 2 replies; 16+ messages in thread From: Avi Kivity @ 2012-09-13 9:59 UTC (permalink / raw) To: Jiri Slaby; +Cc: Jiri Slaby, Marcelo Tosatti, kvm, LKML On 09/12/2012 09:11 PM, Jiri Slaby wrote: > On 09/12/2012 10:18 AM, Avi Kivity wrote: >> On 09/12/2012 11:13 AM, Jiri Slaby wrote: >>> >>>> Please provide the output of vmxcap >>>> (http://goo.gl/c5lUO), >>> >>> Unrestricted guest no >> >> The big real mode fixes. >> >> >>> >>>> and a snapshot of kvm_stat while the guest is hung. >>> >>> kvm statistics >>> >>> exits 6778198 615942 >>> host_state_reload 1988 187 >>> irq_exits 1523 138 >>> mmu_cache_miss 4 0 >>> fpu_reload 1 0 >> >> Please run this as root so we get the tracepoint based output; and press >> 'x' when it's running so we get more detailed output. > > kvm statistics > > kvm_exit 13798699 330708 > kvm_entry 13799110 330708 > kvm_page_fault 13793650 330604 > kvm_exit(EXCEPTION_NMI) 6188458 330604 > kvm_exit(EXTERNAL_INTERRUPT) 2169 105 > kvm_exit(TPR_BELOW_THRESHOLD) 82 0 > kvm_exit(IO_INSTRUCTION) 6 0 Strange, it's unable to fault in the very first page. Please provide a trace as per http://www.linux-kvm.org/page/Tracing (but append -e kvmmmu to the command line). -- error compiling committee.c: too many arguments to function ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: qemu-kvm loops after kernel udpate 2012-09-13 9:59 ` Avi Kivity @ 2012-09-13 18:32 ` Jiri Slaby 2012-09-18 19:40 ` Jiri Slaby 1 sibling, 0 replies; 16+ messages in thread From: Jiri Slaby @ 2012-09-13 18:32 UTC (permalink / raw) To: Avi Kivity; +Cc: Jiri Slaby, Marcelo Tosatti, kvm, LKML [-- Attachment #1: Type: text/plain, Size: 1702 bytes --] On 09/13/2012 11:59 AM, Avi Kivity wrote: > On 09/12/2012 09:11 PM, Jiri Slaby wrote: >> On 09/12/2012 10:18 AM, Avi Kivity wrote: >>> On 09/12/2012 11:13 AM, Jiri Slaby wrote: >>>> >>>>> Please provide the output of vmxcap >>>>> (http://goo.gl/c5lUO), >>>> >>>> Unrestricted guest no >>> >>> The big real mode fixes. >>> >>> >>>> >>>>> and a snapshot of kvm_stat while the guest is hung. >>>> >>>> kvm statistics >>>> >>>> exits 6778198 615942 >>>> host_state_reload 1988 187 >>>> irq_exits 1523 138 >>>> mmu_cache_miss 4 0 >>>> fpu_reload 1 0 >>> >>> Please run this as root so we get the tracepoint based output; and press >>> 'x' when it's running so we get more detailed output. >> >> kvm statistics >> >> kvm_exit 13798699 330708 >> kvm_entry 13799110 330708 >> kvm_page_fault 13793650 330604 >> kvm_exit(EXCEPTION_NMI) 6188458 330604 >> kvm_exit(EXTERNAL_INTERRUPT) 2169 105 >> kvm_exit(TPR_BELOW_THRESHOLD) 82 0 >> kvm_exit(IO_INSTRUCTION) 6 0 > > Strange, it's unable to fault in the very first page. > > Please provide a trace as per http://www.linux-kvm.org/page/Tracing (but > append -e kvmmmu to the command line). Attached. Does it make sense? It wrote things like: failed to read event print fmt for kvm_mmu_unsync_page to the stderr. thanks, -- js suse labs [-- Attachment #2: report --] [-- Type: text/plain, Size: 1627 bytes --] version = 6 CPU 0 is empty cpus=2 qemu-kvm-6170 [001] 457.811896: kvm_mmu_get_page: [FAILED TO PARSE] gfn=0 role=122882 root_count=0 unsync=0 created=1 qemu-kvm-6170 [001] 457.811899: kvm_mmu_get_page: [FAILED TO PARSE] gfn=262144 role=122882 root_count=0 unsync=0 created=1 qemu-kvm-6170 [001] 457.811900: kvm_mmu_get_page: [FAILED TO PARSE] gfn=524288 role=122882 root_count=0 unsync=0 created=1 qemu-kvm-6170 [001] 457.811902: kvm_mmu_get_page: [FAILED TO PARSE] gfn=786432 role=122882 root_count=0 unsync=0 created=1 qemu-kvm-6171 [001] 462.416705: kvm_mmu_prepare_zap_page: [FAILED TO PARSE] gfn=786432 role=122882 root_count=1 unsync=0 qemu-kvm-6171 [001] 462.416712: kvm_mmu_prepare_zap_page: [FAILED TO PARSE] gfn=524288 role=122882 root_count=1 unsync=0 qemu-kvm-6171 [001] 462.416715: kvm_mmu_prepare_zap_page: [FAILED TO PARSE] gfn=262144 role=122882 root_count=1 unsync=0 qemu-kvm-6171 [001] 462.416717: kvm_mmu_prepare_zap_page: [FAILED TO PARSE] gfn=0 role=122882 root_count=1 unsync=0 qemu-kvm-6171 [001] 462.485197: kvm_mmu_prepare_zap_page: [FAILED TO PARSE] gfn=0 role=253954 root_count=0 unsync=0 qemu-kvm-6171 [001] 462.485202: kvm_mmu_prepare_zap_page: [FAILED TO PARSE] gfn=262144 role=253954 root_count=0 unsync=0 qemu-kvm-6171 [001] 462.485205: kvm_mmu_prepare_zap_page: [FAILED TO PARSE] gfn=524288 role=253954 root_count=0 unsync=0 qemu-kvm-6171 [001] 462.485209: kvm_mmu_prepare_zap_page: [FAILED TO PARSE] gfn=786432 role=253954 root_count=0 unsync=0 ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: qemu-kvm loops after kernel udpate 2012-09-13 9:59 ` Avi Kivity 2012-09-13 18:32 ` Jiri Slaby @ 2012-09-18 19:40 ` Jiri Slaby 2012-09-18 19:46 ` Andrew Morton 2012-09-19 14:18 ` [PATCH] mm: call invalidate_range_end in do_wp_page even for zero pages Haggai Eran 1 sibling, 2 replies; 16+ messages in thread From: Jiri Slaby @ 2012-09-18 19:40 UTC (permalink / raw) To: Avi Kivity Cc: Jiri Slaby, Marcelo Tosatti, kvm, LKML, Haggai Eran, Andrew Morton, linux-mm On 09/13/2012 11:59 AM, Avi Kivity wrote: > On 09/12/2012 09:11 PM, Jiri Slaby wrote: >> On 09/12/2012 10:18 AM, Avi Kivity wrote: >>> On 09/12/2012 11:13 AM, Jiri Slaby wrote: >>>> >>>>> Please provide the output of vmxcap >>>>> (http://goo.gl/c5lUO), >>>> >>>> Unrestricted guest no >>> >>> The big real mode fixes. >>> >>> >>>> >>>>> and a snapshot of kvm_stat while the guest is hung. >>>> >>>> kvm statistics >>>> >>>> exits 6778198 615942 >>>> host_state_reload 1988 187 >>>> irq_exits 1523 138 >>>> mmu_cache_miss 4 0 >>>> fpu_reload 1 0 >>> >>> Please run this as root so we get the tracepoint based output; and press >>> 'x' when it's running so we get more detailed output. >> >> kvm statistics >> >> kvm_exit 13798699 330708 >> kvm_entry 13799110 330708 >> kvm_page_fault 13793650 330604 >> kvm_exit(EXCEPTION_NMI) 6188458 330604 >> kvm_exit(EXTERNAL_INTERRUPT) 2169 105 >> kvm_exit(TPR_BELOW_THRESHOLD) 82 0 >> kvm_exit(IO_INSTRUCTION) 6 0 > > Strange, it's unable to fault in the very first page. I bisected that. Note the bisection log. I have never seen something like that :D: git bisect start git bisect bad 3de9d1a1500472bc80478bd75e33fa9c1eba1422 git bisect good fea7a08acb13524b47711625eebea40a0ede69a0 git bisect good 95a2fe4baa1ad444df5f94bfc9416fc6b4b34cef git bisect good f42c0d57a5a60da03c705bdea9fbba381112dd60 git bisect good 31a2e241a9e37a133278959044960c229acc5714 git bisect good f15fb01c5593fa1b58cc7a8a9c59913e2625bf2e git bisect good 16d21ff46f5d50e311d07406c31f96916e5e8e1a git bisect good 0b84592f458b4e8567aa7d803aff382c1d3b64fd git bisect bad b955428e7f14cd29fe9d8059efa3ea4be679c83d git bisect bad 20c4da4f68fcade05eda9c9b7dbad0a78cc5efe8 git bisect bad 31b90ed2a90f80fb528ac55ee357a815e1dedc36 git bisect bad b273fe14ee5b38cecc7bce94f7777f35a0bf9ee4 git bisect bad de426dbe9a60706b91b40397f69f819a39a06b6b git bisect bad 6b998094ec50248e72b9f251d0607b58b18dba38 git bisect bad cf9b81d47a89f5d404a0cd8013b461617751e520 === 8< === Reverting cf9b81d47a89 (mm: wrap calls to set_pte_at_notify with invalidate_range_start and invalidate_range_end) on the top of today's -next fixes the issue. thanks, -- js suse labs ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: qemu-kvm loops after kernel udpate 2012-09-18 19:40 ` Jiri Slaby @ 2012-09-18 19:46 ` Andrew Morton 2012-09-19 0:00 ` Stephen Rothwell 2012-09-19 14:18 ` [PATCH] mm: call invalidate_range_end in do_wp_page even for zero pages Haggai Eran 1 sibling, 1 reply; 16+ messages in thread From: Andrew Morton @ 2012-09-18 19:46 UTC (permalink / raw) To: Jiri Slaby Cc: Avi Kivity, Jiri Slaby, Marcelo Tosatti, kvm, LKML, Haggai Eran, linux-mm, Sagi Grimberg, Shachar Raindel, Liran Liss On Tue, 18 Sep 2012 21:40:31 +0200 Jiri Slaby <jslaby@suse.cz> wrote: > On 09/13/2012 11:59 AM, Avi Kivity wrote: > > On 09/12/2012 09:11 PM, Jiri Slaby wrote: > >> On 09/12/2012 10:18 AM, Avi Kivity wrote: > >>> On 09/12/2012 11:13 AM, Jiri Slaby wrote: > >>>> > >>>>> Please provide the output of vmxcap > >>>>> (http://goo.gl/c5lUO), > >>>> > >>>> Unrestricted guest no > >>> > >>> The big real mode fixes. > >>> > >>> > >>>> > >>>>> and a snapshot of kvm_stat while the guest is hung. > >>>> > >>>> kvm statistics > >>>> > >>>> exits 6778198 615942 > >>>> host_state_reload 1988 187 > >>>> irq_exits 1523 138 > >>>> mmu_cache_miss 4 0 > >>>> fpu_reload 1 0 > >>> > >>> Please run this as root so we get the tracepoint based output; and press > >>> 'x' when it's running so we get more detailed output. > >> > >> kvm statistics > >> > >> kvm_exit 13798699 330708 > >> kvm_entry 13799110 330708 > >> kvm_page_fault 13793650 330604 > >> kvm_exit(EXCEPTION_NMI) 6188458 330604 > >> kvm_exit(EXTERNAL_INTERRUPT) 2169 105 > >> kvm_exit(TPR_BELOW_THRESHOLD) 82 0 > >> kvm_exit(IO_INSTRUCTION) 6 0 > > > > Strange, it's unable to fault in the very first page. > > I bisected that. Note the bisection log. I have never seen something > like that :D: > git bisect start > git bisect bad 3de9d1a1500472bc80478bd75e33fa9c1eba1422 > git bisect good fea7a08acb13524b47711625eebea40a0ede69a0 > git bisect good 95a2fe4baa1ad444df5f94bfc9416fc6b4b34cef > git bisect good f42c0d57a5a60da03c705bdea9fbba381112dd60 > git bisect good 31a2e241a9e37a133278959044960c229acc5714 > git bisect good f15fb01c5593fa1b58cc7a8a9c59913e2625bf2e > git bisect good 16d21ff46f5d50e311d07406c31f96916e5e8e1a > git bisect good 0b84592f458b4e8567aa7d803aff382c1d3b64fd > git bisect bad b955428e7f14cd29fe9d8059efa3ea4be679c83d > git bisect bad 20c4da4f68fcade05eda9c9b7dbad0a78cc5efe8 > git bisect bad 31b90ed2a90f80fb528ac55ee357a815e1dedc36 > git bisect bad b273fe14ee5b38cecc7bce94f7777f35a0bf9ee4 > git bisect bad de426dbe9a60706b91b40397f69f819a39a06b6b > git bisect bad 6b998094ec50248e72b9f251d0607b58b18dba38 > git bisect bad cf9b81d47a89f5d404a0cd8013b461617751e520 > > === 8< === > > Reverting cf9b81d47a89 (mm: wrap calls to set_pte_at_notify with > invalidate_range_start and invalidate_range_end) on the top of today's > -next fixes the issue. hm, thanks. This will probably take some time to resolve so I think I'll drop mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock.patch mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock-fix.patch mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock-fix-fix.patch mm-wrap-calls-to-set_pte_at_notify-with-invalidate_range_start-and-invalidate_range_end.patch ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: qemu-kvm loops after kernel udpate 2012-09-18 19:46 ` Andrew Morton @ 2012-09-19 0:00 ` Stephen Rothwell 2012-09-19 0:20 ` Andrew Morton 0 siblings, 1 reply; 16+ messages in thread From: Stephen Rothwell @ 2012-09-19 0:00 UTC (permalink / raw) To: Andrew Morton Cc: Jiri Slaby, Avi Kivity, Jiri Slaby, Marcelo Tosatti, kvm, LKML, Haggai Eran, linux-mm, Sagi Grimberg, Shachar Raindel, Liran Liss [-- Attachment #1: Type: text/plain, Size: 721 bytes --] Hi Andrew, On Tue, 18 Sep 2012 12:46:46 -0700 Andrew Morton <akpm@linux-foundation.org> wrote: > > hm, thanks. This will probably take some time to resolve so I think > I'll drop > > mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock.patch > mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock-fix.patch > mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock-fix-fix.patch > mm-wrap-calls-to-set_pte_at_notify-with-invalidate_range_start-and-invalidate_range_end.patch Should I attempt to remove these from the akpm tree in linux-next today? Or should I just wait for a new mmotm? -- Cheers, Stephen Rothwell sfr@canb.auug.org.au [-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --] ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: qemu-kvm loops after kernel udpate 2012-09-19 0:00 ` Stephen Rothwell @ 2012-09-19 0:20 ` Andrew Morton 2012-09-19 1:36 ` Stephen Rothwell 0 siblings, 1 reply; 16+ messages in thread From: Andrew Morton @ 2012-09-19 0:20 UTC (permalink / raw) To: Stephen Rothwell Cc: Jiri Slaby, Avi Kivity, Jiri Slaby, Marcelo Tosatti, kvm, LKML, Haggai Eran, linux-mm, Sagi Grimberg, Shachar Raindel, Liran Liss On Wed, 19 Sep 2012 10:00:34 +1000 Stephen Rothwell <sfr@canb.auug.org.au> wrote: > Hi Andrew, > > On Tue, 18 Sep 2012 12:46:46 -0700 Andrew Morton <akpm@linux-foundation.org> wrote: > > > > hm, thanks. This will probably take some time to resolve so I think > > I'll drop > > > > mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock.patch > > mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock-fix.patch > > mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock-fix-fix.patch > > mm-wrap-calls-to-set_pte_at_notify-with-invalidate_range_start-and-invalidate_range_end.patch > > Should I attempt to remove these from the akpm tree in linux-next today? That would be best - there's no point in having people test (and debug) dead stuff. > Or should I just wait for a new mmotm? You could be brave and test http://ozlabs.org/~akpm/mmots/ for me :) ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: qemu-kvm loops after kernel udpate 2012-09-19 0:20 ` Andrew Morton @ 2012-09-19 1:36 ` Stephen Rothwell 0 siblings, 0 replies; 16+ messages in thread From: Stephen Rothwell @ 2012-09-19 1:36 UTC (permalink / raw) To: Andrew Morton Cc: Jiri Slaby, Avi Kivity, Jiri Slaby, Marcelo Tosatti, kvm, LKML, Haggai Eran, linux-mm, Sagi Grimberg, Shachar Raindel, Liran Liss [-- Attachment #1: Type: text/plain, Size: 1174 bytes --] Hi Andrew, On Tue, 18 Sep 2012 17:20:29 -0700 Andrew Morton <akpm@linux-foundation.org> wrote: > > On Wed, 19 Sep 2012 10:00:34 +1000 Stephen Rothwell <sfr@canb.auug.org.au> wrote: > > > On Tue, 18 Sep 2012 12:46:46 -0700 Andrew Morton <akpm@linux-foundation.org> wrote: > > > > > > hm, thanks. This will probably take some time to resolve so I think > > > I'll drop > > > > > > mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock.patch > > > mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock-fix.patch > > > mm-move-all-mmu-notifier-invocations-to-be-done-outside-the-pt-lock-fix-fix.patch > > > mm-wrap-calls-to-set_pte_at_notify-with-invalidate_range_start-and-invalidate_range_end.patch > > > > Should I attempt to remove these from the akpm tree in linux-next today? > > That would be best - there's no point in having people test (and debug) > dead stuff. OK, I removed them. > > Or should I just wait for a new mmotm? > > You could be brave and test http://ozlabs.org/~akpm/mmots/ for me :) Brave? maybe. Stupid? no :-) -- Cheers, Stephen Rothwell sfr@canb.auug.org.au [-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --] ^ permalink raw reply [flat|nested] 16+ messages in thread
* [PATCH] mm: call invalidate_range_end in do_wp_page even for zero pages 2012-09-18 19:40 ` Jiri Slaby 2012-09-18 19:46 ` Andrew Morton @ 2012-09-19 14:18 ` Haggai Eran 1 sibling, 0 replies; 16+ messages in thread From: Haggai Eran @ 2012-09-19 14:18 UTC (permalink / raw) To: Jiri Slaby Cc: linux-kernel, kvm, linux-mm, Shachar Raindel, Haggai Eran, Avi Kivity, Andrew Morton The previous patch "mm: wrap calls to set_pte_at_notify with invalidate_range_start and invalidate_range_end" only called the invalidate_range_end mmu notifier function in do_wp_page when the new_page variable wasn't NULL. This was done in order to only call invalidate_range_end after invalidate_range_start was called. Unfortunately, there are situations where new_page is NULL and invalidate_range_start is called. This caused invalidate_range_start to be called without a matching invalidate_range_end, causing kvm to loop indefinitely on the first page fault. This patch adds a flag variable to do_wp_page that marks whether the invalidate_range_start notifier was called. invalidate_range_end is then called if the flag is true. Reported-by: Jiri Slaby <jslaby@suse.cz> Cc: Avi Kivity <avi@redhat.com> Cc: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Haggai Eran <haggaie@mellanox.com> --- I tested this patch against yesterday's linux-next (next-20120918), and it seems to solve the problem with kvm. I used the same command line you reported: qemu-kvm -k en-us -usbdevice tablet -balloon virtio -hda IMAGE -smp 2 \ -m 1000M -net user -net nic,model=e1000 -usb -serial pty I was hoping you could also test it yourself, and see that it also works for you, if you don't mind. mm/memory.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 1a92d87..76ec199 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -2529,6 +2529,7 @@ static int do_wp_page(struct mm_struct *mm, struct vm_area_struct *vma, struct page *dirty_page = NULL; unsigned long mmun_start; /* For mmu_notifiers */ unsigned long mmun_end; /* For mmu_notifiers */ + bool mmun_called = false; /* For mmu_notifiers */ old_page = vm_normal_page(vma, address, orig_pte); if (!old_page) { @@ -2706,8 +2707,9 @@ gotten: if (mem_cgroup_newpage_charge(new_page, mm, GFP_KERNEL)) goto oom_free_new; - mmun_start = address & PAGE_MASK; - mmun_end = (address & PAGE_MASK) + PAGE_SIZE; + mmun_start = address & PAGE_MASK; + mmun_end = (address & PAGE_MASK) + PAGE_SIZE; + mmun_called = true; mmu_notifier_invalidate_range_start(mm, mmun_start, mmun_end); /* @@ -2776,8 +2778,7 @@ gotten: page_cache_release(new_page); unlock: pte_unmap_unlock(page_table, ptl); - if (new_page) - /* Only call the end notifier if the begin was called. */ + if (mmun_called) mmu_notifier_invalidate_range_end(mm, mmun_start, mmun_end); if (old_page) { /* -- 1.7.11.2 ^ permalink raw reply related [flat|nested] 16+ messages in thread
end of thread, other threads:[~2012-09-19 14:19 UTC | newest] Thread overview: 16+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2012-09-11 18:11 qemu-kvm loops after kernel udpate Jiri Slaby 2012-09-11 19:03 ` Marcelo Tosatti 2012-09-11 19:41 ` Jiri Slaby 2012-09-12 8:06 ` Avi Kivity 2012-09-12 8:13 ` Jiri Slaby 2012-09-12 8:18 ` Avi Kivity 2012-09-12 8:20 ` Jiri Slaby 2012-09-12 18:11 ` Jiri Slaby 2012-09-13 9:59 ` Avi Kivity 2012-09-13 18:32 ` Jiri Slaby 2012-09-18 19:40 ` Jiri Slaby 2012-09-18 19:46 ` Andrew Morton 2012-09-19 0:00 ` Stephen Rothwell 2012-09-19 0:20 ` Andrew Morton 2012-09-19 1:36 ` Stephen Rothwell 2012-09-19 14:18 ` [PATCH] mm: call invalidate_range_end in do_wp_page even for zero pages Haggai Eran
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).