* [PATCH] Revert "qemu-kvm: Bring qemu_init_vcpu back home" @ 2011-02-23 14:38 Ryousei Takano 2011-02-23 14:47 ` Jan Kiszka 0 siblings, 1 reply; 6+ messages in thread From: Ryousei Takano @ 2011-02-23 14:38 UTC (permalink / raw) To: kvm; +Cc: Jan Kiszka, Marcelo Tosatti, takano-ryousei This reverts commit c65827770b5e7282b5fb9da29c1de85ed08e816e Signed-off-by: Ryousei Takano <takano-ryousei@aist.go.jp> --- I am trying to run OpenMP (i.e., multi-threaded) programs using Intel Math Kernel Library (MKL) on a guest OS. In the qemu-kvm-0.12.5, my program works fine. In the qemu-kvm-0.13.0, however, it does not. The Linux kernel is 2.6.32.28. The outputs of 'ps -L' are shown as follows: qemu-kvm-0.12.5: PID LWP TTY TIME CMD 2731 2731 pts/0 00:00:00 bash 2799 2799 pts/0 00:00:19 a.out 2799 2803 pts/0 00:00:00 a.out 2799 2804 pts/0 00:00:04 a.out <- 2799 2805 pts/0 00:00:03 a.out <- these threads are working properly. 2799 2806 pts/0 00:00:04 a.out <- 2807 2807 pts/0 00:00:00 ps qemu-0.13.0: PID LWP TTY TIME CMD 2725 2725 pts/0 00:00:00 bash 2748 2748 pts/0 00:00:19 a.out 2748 2752 pts/0 00:00:00 a.out 2748 2753 pts/0 00:00:00 a.out <- 2748 2754 pts/0 00:00:00 a.out <- these threads are blocked by pthread_cond_wait(3). 2748 2755 pts/0 00:00:00 a.out <- 2756 2756 pts/0 00:00:00 ps This problem seems to be introduced by the commit c65827770b5e7282b5fb9da29c1de85ed08e816e. The attached revert patch solves it. But I do not understand what is happened;-) Any comments and suggestions will be appreciated. Regards, Ryousei hw/pc.c | 6 ++++++ target-i386/helper.c | 2 -- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/hw/pc.c b/hw/pc.c index ec97181..577abe7 100644 --- a/hw/pc.c +++ b/hw/pc.c @@ -937,6 +937,12 @@ CPUState *pc_new_cpu(const char *cpu_model) } qemu_register_reset(pc_cpu_reset, env); pc_cpu_reset(env); + + /* kvm needs this to run after the apic is initialized. Otherwise, + * it can access invalid state and crash. + */ + qemu_init_vcpu(env); + return env; } diff --git a/target-i386/helper.c b/target-i386/helper.c index 86202e6..69cda5e 100644 --- a/target-i386/helper.c +++ b/target-i386/helper.c @@ -1224,8 +1224,6 @@ CPUX86State *cpu_x86_init(const char *cpu_model) } mce_init(env); - qemu_init_vcpu(env); - return env; } -- 1.7.4.1 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH] Revert "qemu-kvm: Bring qemu_init_vcpu back home" 2011-02-23 14:38 [PATCH] Revert "qemu-kvm: Bring qemu_init_vcpu back home" Ryousei Takano @ 2011-02-23 14:47 ` Jan Kiszka 2011-02-23 15:53 ` Ryousei Takano 0 siblings, 1 reply; 6+ messages in thread From: Jan Kiszka @ 2011-02-23 14:47 UTC (permalink / raw) To: Ryousei Takano; +Cc: kvm@vger.kernel.org, Marcelo Tosatti On 2011-02-23 15:38, Ryousei Takano wrote: > This reverts commit c65827770b5e7282b5fb9da29c1de85ed08e816e Reasoning would belong here, not below the separator. > > Signed-off-by: Ryousei Takano <takano-ryousei@aist.go.jp> > --- > I am trying to run OpenMP (i.e., multi-threaded) programs using Intel > Math Kernel Library (MKL) on a guest OS. In the qemu-kvm-0.12.5, > my program works fine. In the qemu-kvm-0.13.0, however, it does not. What about 0.14? > The Linux kernel is 2.6.32.28. The outputs of 'ps -L' are shown as follows: > > qemu-kvm-0.12.5: > PID LWP TTY TIME CMD > 2731 2731 pts/0 00:00:00 bash > 2799 2799 pts/0 00:00:19 a.out > 2799 2803 pts/0 00:00:00 a.out > 2799 2804 pts/0 00:00:04 a.out <- > 2799 2805 pts/0 00:00:03 a.out <- these threads are working properly. > 2799 2806 pts/0 00:00:04 a.out <- > 2807 2807 pts/0 00:00:00 ps > > qemu-0.13.0: > PID LWP TTY TIME CMD > 2725 2725 pts/0 00:00:00 bash > 2748 2748 pts/0 00:00:19 a.out > 2748 2752 pts/0 00:00:00 a.out > 2748 2753 pts/0 00:00:00 a.out <- > 2748 2754 pts/0 00:00:00 a.out <- these threads are blocked by > pthread_cond_wait(3). > 2748 2755 pts/0 00:00:00 a.out <- > 2756 2756 pts/0 00:00:00 ps > > This problem seems to be introduced by the commit > c65827770b5e7282b5fb9da29c1de85ed08e816e. > The attached revert patch solves it. But I do not understand what is > happened;-) > Any comments and suggestions will be appreciated. > > Regards, > Ryousei > > hw/pc.c | 6 ++++++ > target-i386/helper.c | 2 -- > 2 files changed, 6 insertions(+), 2 deletions(-) > > diff --git a/hw/pc.c b/hw/pc.c > index ec97181..577abe7 100644 > --- a/hw/pc.c > +++ b/hw/pc.c > @@ -937,6 +937,12 @@ CPUState *pc_new_cpu(const char *cpu_model) > } > qemu_register_reset(pc_cpu_reset, env); > pc_cpu_reset(env); > + > + /* kvm needs this to run after the apic is initialized. Otherwise, > + * it can access invalid state and crash. > + */ > + qemu_init_vcpu(env); > + > return env; > } > > diff --git a/target-i386/helper.c b/target-i386/helper.c > index 86202e6..69cda5e 100644 > --- a/target-i386/helper.c > +++ b/target-i386/helper.c > @@ -1224,8 +1224,6 @@ CPUX86State *cpu_x86_init(const char *cpu_model) > } > mce_init(env); > > - qemu_init_vcpu(env); > - > return env; > } > I don't believe yet that the patch is the reason, specifically as it targets the setup phase (except for CPU hotplug, but that's broken for ages), not the operational phase of the VM. What's your qemu-kvm command line? Also, please attach gdb to the stuck VM and print the backtraces of all threads ("thread apply all bt"). Thanks, Jan -- Siemens AG, Corporate Technology, CT T DE IT 1 Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] Revert "qemu-kvm: Bring qemu_init_vcpu back home" 2011-02-23 14:47 ` Jan Kiszka @ 2011-02-23 15:53 ` Ryousei Takano 2011-02-23 17:26 ` Jan Kiszka 0 siblings, 1 reply; 6+ messages in thread From: Ryousei Takano @ 2011-02-23 15:53 UTC (permalink / raw) To: Jan Kiszka; +Cc: kvm@vger.kernel.org, Marcelo Tosatti Hi Jan, On Wed, Feb 23, 2011 at 11:47 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote: > On 2011-02-23 15:38, Ryousei Takano wrote: >> This reverts commit c65827770b5e7282b5fb9da29c1de85ed08e816e > > Reasoning would belong here, not below the separator. > >> >> Signed-off-by: Ryousei Takano <takano-ryousei@aist.go.jp> >> --- >> I am trying to run OpenMP (i.e., multi-threaded) programs using Intel >> Math Kernel Library (MKL) on a guest OS. In the qemu-kvm-0.12.5, >> my program works fine. In the qemu-kvm-0.13.0, however, it does not. > > What about 0.14? > 0.14-rc1 also has problem. >> The Linux kernel is 2.6.32.28. The outputs of 'ps -L' are shown as follows: >> >> qemu-kvm-0.12.5: >> PID LWP TTY TIME CMD >> 2731 2731 pts/0 00:00:00 bash >> 2799 2799 pts/0 00:00:19 a.out >> 2799 2803 pts/0 00:00:00 a.out >> 2799 2804 pts/0 00:00:04 a.out <- >> 2799 2805 pts/0 00:00:03 a.out <- these threads are working properly. >> 2799 2806 pts/0 00:00:04 a.out <- >> 2807 2807 pts/0 00:00:00 ps >> >> qemu-0.13.0: >> PID LWP TTY TIME CMD >> 2725 2725 pts/0 00:00:00 bash >> 2748 2748 pts/0 00:00:19 a.out >> 2748 2752 pts/0 00:00:00 a.out >> 2748 2753 pts/0 00:00:00 a.out <- >> 2748 2754 pts/0 00:00:00 a.out <- these threads are blocked by >> pthread_cond_wait(3). >> 2748 2755 pts/0 00:00:00 a.out <- >> 2756 2756 pts/0 00:00:00 ps >> >> This problem seems to be introduced by the commit >> c65827770b5e7282b5fb9da29c1de85ed08e816e. >> The attached revert patch solves it. But I do not understand what is >> happened;-) >> Any comments and suggestions will be appreciated. >> >> Regards, >> Ryousei >> >> hw/pc.c | 6 ++++++ >> target-i386/helper.c | 2 -- >> 2 files changed, 6 insertions(+), 2 deletions(-) >> >> diff --git a/hw/pc.c b/hw/pc.c >> index ec97181..577abe7 100644 >> --- a/hw/pc.c >> +++ b/hw/pc.c >> @@ -937,6 +937,12 @@ CPUState *pc_new_cpu(const char *cpu_model) >> } >> qemu_register_reset(pc_cpu_reset, env); >> pc_cpu_reset(env); >> + >> + /* kvm needs this to run after the apic is initialized. Otherwise, >> + * it can access invalid state and crash. >> + */ >> + qemu_init_vcpu(env); >> + >> return env; >> } >> >> diff --git a/target-i386/helper.c b/target-i386/helper.c >> index 86202e6..69cda5e 100644 >> --- a/target-i386/helper.c >> +++ b/target-i386/helper.c >> @@ -1224,8 +1224,6 @@ CPUX86State *cpu_x86_init(const char *cpu_model) >> } >> mce_init(env); >> >> - qemu_init_vcpu(env); >> - >> return env; >> } >> > > I don't believe yet that the patch is the reason, specifically as it > targets the setup phase (except for CPU hotplug, but that's broken for > ages), not the operational phase of the VM. > > What's your qemu-kvm command line? > Here is my command line. sudo qemu-system-x86_64 \ -cpu host \ -smp 8 \ -m 46080 \ -net nic,model=virtio,macaddr=00:16:3e:1d:ff:00 \ -net tap,ifname=tap0,script=no,downscript=no \ -monitor telnet::5963,server,nowait \ -daemonize \ disk.img > Also, please attach gdb to the stuck VM and print the backtraces of all > threads ("thread apply all bt"). > (gdb) thread apply all bt Thread 9 (Thread 0x41129940 (LWP 8118)): #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 #1 0x000000000042e56d in pthread_attr_setdetachstate () #2 0x000000000042ea99 in pthread_attr_setdetachstate () #3 0x000000000042ecf0 in pthread_attr_setdetachstate () #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 Thread 8 (Thread 0x41ca8940 (LWP 8119)): #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 #1 0x000000000042e56d in pthread_attr_setdetachstate () #2 0x000000000042ea99 in pthread_attr_setdetachstate () #3 0x000000000042ecf0 in pthread_attr_setdetachstate () #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 Thread 7 (Thread 0x425f2940 (LWP 8120)): #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 #1 0x000000000042e56d in pthread_attr_setdetachstate () #2 0x000000000042ea99 in pthread_attr_setdetachstate () #3 0x000000000042ecf0 in pthread_attr_setdetachstate () #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 Thread 6 (Thread 0x42df3940 (LWP 8121)): #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 #1 0x000000000042e56d in pthread_attr_setdetachstate () #2 0x000000000042ea99 in pthread_attr_setdetachstate () #3 0x000000000042ecf0 in pthread_attr_setdetachstate () #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 Thread 5 (Thread 0x435f4940 (LWP 8122)): #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 #1 0x000000000042e56d in pthread_attr_setdetachstate () #2 0x000000000042ea99 in pthread_attr_setdetachstate () #3 0x000000000042ecf0 in pthread_attr_setdetachstate () #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 Thread 4 (Thread 0x43df5940 (LWP 8123)): #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 #1 0x000000000042e56d in pthread_attr_setdetachstate () #2 0x000000000042ea99 in pthread_attr_setdetachstate () #3 0x000000000042ecf0 in pthread_attr_setdetachstate () #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 Thread 3 (Thread 0x445f6940 (LWP 8124)): #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 #1 0x000000000042e56d in pthread_attr_setdetachstate () #2 0x000000000042ea99 in pthread_attr_setdetachstate () #3 0x000000000042ecf0 in pthread_attr_setdetachstate () #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 Thread 2 (Thread 0x44df7940 (LWP 8125)): #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 #1 0x000000000042e56d in pthread_attr_setdetachstate () #2 0x000000000042ea99 in pthread_attr_setdetachstate () #3 0x000000000042ecf0 in pthread_attr_setdetachstate () #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 Thread 1 (Thread 0x7fe796ec1740 (LWP 8117)): #0 0x0000003dd38ccfc2 in select () from /lib64/libc.so.6 #1 0x0000000000419620 in pthread_attr_setdetachstate () #2 0x000000000042be07 in pthread_attr_setdetachstate () #3 0x000000000041ae20 in pthread_attr_setdetachstate () #4 0x0000003dd381d994 in __libc_start_main () from /lib64/libc.so.6 #5 0x00000000004063d9 in pthread_attr_setdetachstate () #6 0x00007fff8050c988 in ?? () #7 0x0000000000000000 in ?? () Regards, Ryousei > Thanks, > Jan > > -- > Siemens AG, Corporate Technology, CT T DE IT 1 > Corporate Competence Center Embedded Linux > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] Revert "qemu-kvm: Bring qemu_init_vcpu back home" 2011-02-23 15:53 ` Ryousei Takano @ 2011-02-23 17:26 ` Jan Kiszka 2011-02-24 1:29 ` Ryousei Takano 0 siblings, 1 reply; 6+ messages in thread From: Jan Kiszka @ 2011-02-23 17:26 UTC (permalink / raw) To: Ryousei Takano; +Cc: kvm@vger.kernel.org, Marcelo Tosatti On 2011-02-23 16:53, Ryousei Takano wrote: > Hi Jan, > > On Wed, Feb 23, 2011 at 11:47 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote: >> On 2011-02-23 15:38, Ryousei Takano wrote: >>> This reverts commit c65827770b5e7282b5fb9da29c1de85ed08e816e >> >> Reasoning would belong here, not below the separator. >> >>> >>> Signed-off-by: Ryousei Takano <takano-ryousei@aist.go.jp> >>> --- >>> I am trying to run OpenMP (i.e., multi-threaded) programs using Intel >>> Math Kernel Library (MKL) on a guest OS. In the qemu-kvm-0.12.5, >>> my program works fine. In the qemu-kvm-0.13.0, however, it does not. >> >> What about 0.14? >> > 0.14-rc1 also has problem. Not automagically healed - what a pity. > >>> The Linux kernel is 2.6.32.28. The outputs of 'ps -L' are shown as follows: >>> >>> qemu-kvm-0.12.5: >>> PID LWP TTY TIME CMD >>> 2731 2731 pts/0 00:00:00 bash >>> 2799 2799 pts/0 00:00:19 a.out >>> 2799 2803 pts/0 00:00:00 a.out >>> 2799 2804 pts/0 00:00:04 a.out <- >>> 2799 2805 pts/0 00:00:03 a.out <- these threads are working properly. >>> 2799 2806 pts/0 00:00:04 a.out <- >>> 2807 2807 pts/0 00:00:00 ps >>> >>> qemu-0.13.0: >>> PID LWP TTY TIME CMD >>> 2725 2725 pts/0 00:00:00 bash >>> 2748 2748 pts/0 00:00:19 a.out >>> 2748 2752 pts/0 00:00:00 a.out >>> 2748 2753 pts/0 00:00:00 a.out <- >>> 2748 2754 pts/0 00:00:00 a.out <- these threads are blocked by >>> pthread_cond_wait(3). >>> 2748 2755 pts/0 00:00:00 a.out <- >>> 2756 2756 pts/0 00:00:00 ps >>> >>> This problem seems to be introduced by the commit >>> c65827770b5e7282b5fb9da29c1de85ed08e816e. >>> The attached revert patch solves it. But I do not understand what is >>> happened;-) >>> Any comments and suggestions will be appreciated. >>> >>> Regards, >>> Ryousei >>> >>> hw/pc.c | 6 ++++++ >>> target-i386/helper.c | 2 -- >>> 2 files changed, 6 insertions(+), 2 deletions(-) >>> >>> diff --git a/hw/pc.c b/hw/pc.c >>> index ec97181..577abe7 100644 >>> --- a/hw/pc.c >>> +++ b/hw/pc.c >>> @@ -937,6 +937,12 @@ CPUState *pc_new_cpu(const char *cpu_model) >>> } >>> qemu_register_reset(pc_cpu_reset, env); >>> pc_cpu_reset(env); >>> + >>> + /* kvm needs this to run after the apic is initialized. Otherwise, >>> + * it can access invalid state and crash. >>> + */ >>> + qemu_init_vcpu(env); >>> + >>> return env; >>> } >>> >>> diff --git a/target-i386/helper.c b/target-i386/helper.c >>> index 86202e6..69cda5e 100644 >>> --- a/target-i386/helper.c >>> +++ b/target-i386/helper.c >>> @@ -1224,8 +1224,6 @@ CPUX86State *cpu_x86_init(const char *cpu_model) >>> } >>> mce_init(env); >>> >>> - qemu_init_vcpu(env); >>> - >>> return env; >>> } >>> >> >> I don't believe yet that the patch is the reason, specifically as it >> targets the setup phase (except for CPU hotplug, but that's broken for >> ages), not the operational phase of the VM. >> >> What's your qemu-kvm command line? >> > Here is my command line. > > sudo qemu-system-x86_64 \ > -cpu host \ > -smp 8 \ > -m 46080 \ > -net nic,model=virtio,macaddr=00:16:3e:1d:ff:00 \ > -net tap,ifname=tap0,script=no,downscript=no \ > -monitor telnet::5963,server,nowait \ > -daemonize \ > disk.img How many cores are available on the host? >= 8 or less? > >> Also, please attach gdb to the stuck VM and print the backtraces of all >> threads ("thread apply all bt"). >> > (gdb) thread apply all bt > Thread 9 (Thread 0x41129940 (LWP 8118)): > #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 > #1 0x000000000042e56d in pthread_attr_setdetachstate () > #2 0x000000000042ea99 in pthread_attr_setdetachstate () > #3 0x000000000042ecf0 in pthread_attr_setdetachstate () > #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 > #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 > > Thread 8 (Thread 0x41ca8940 (LWP 8119)): > #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 > #1 0x000000000042e56d in pthread_attr_setdetachstate () > #2 0x000000000042ea99 in pthread_attr_setdetachstate () > #3 0x000000000042ecf0 in pthread_attr_setdetachstate () > #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 > #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 > > Thread 7 (Thread 0x425f2940 (LWP 8120)): > #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 > #1 0x000000000042e56d in pthread_attr_setdetachstate () > #2 0x000000000042ea99 in pthread_attr_setdetachstate () > #3 0x000000000042ecf0 in pthread_attr_setdetachstate () > #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 > #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 > > Thread 6 (Thread 0x42df3940 (LWP 8121)): > #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 > #1 0x000000000042e56d in pthread_attr_setdetachstate () > #2 0x000000000042ea99 in pthread_attr_setdetachstate () > #3 0x000000000042ecf0 in pthread_attr_setdetachstate () > #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 > #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 > > Thread 5 (Thread 0x435f4940 (LWP 8122)): > #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 > #1 0x000000000042e56d in pthread_attr_setdetachstate () > #2 0x000000000042ea99 in pthread_attr_setdetachstate () > #3 0x000000000042ecf0 in pthread_attr_setdetachstate () > #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 > #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 > > Thread 4 (Thread 0x43df5940 (LWP 8123)): > #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 > #1 0x000000000042e56d in pthread_attr_setdetachstate () > #2 0x000000000042ea99 in pthread_attr_setdetachstate () > #3 0x000000000042ecf0 in pthread_attr_setdetachstate () > #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 > #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 > > Thread 3 (Thread 0x445f6940 (LWP 8124)): > #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 > #1 0x000000000042e56d in pthread_attr_setdetachstate () > #2 0x000000000042ea99 in pthread_attr_setdetachstate () > #3 0x000000000042ecf0 in pthread_attr_setdetachstate () > #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 > #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 > > Thread 2 (Thread 0x44df7940 (LWP 8125)): > #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 > #1 0x000000000042e56d in pthread_attr_setdetachstate () > #2 0x000000000042ea99 in pthread_attr_setdetachstate () > #3 0x000000000042ecf0 in pthread_attr_setdetachstate () > #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 > #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 > > Thread 1 (Thread 0x7fe796ec1740 (LWP 8117)): > #0 0x0000003dd38ccfc2 in select () from /lib64/libc.so.6 > #1 0x0000000000419620 in pthread_attr_setdetachstate () > #2 0x000000000042be07 in pthread_attr_setdetachstate () > #3 0x000000000041ae20 in pthread_attr_setdetachstate () > #4 0x0000003dd381d994 in __libc_start_main () from /lib64/libc.so.6 > #5 0x00000000004063d9 in pthread_attr_setdetachstate () > #6 0x00007fff8050c988 in ?? () > #7 0x0000000000000000 in ?? () > Can you retry with --enable-debug set during qemu-kvm configure? We need the symbols of the qemu functions. Despite the lacking symbols, this rather looks like we are stuck in kernel space, not on some condition variable (ie. futex). Then probably also interesting would be a trace of the host kernel (trace-cmd record -e kvm, see e.g. [1]). Thanks, Jan [1] http://lwn.net/Articles/341902/ -- Siemens AG, Corporate Technology, CT T DE IT 1 Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] Revert "qemu-kvm: Bring qemu_init_vcpu back home" 2011-02-23 17:26 ` Jan Kiszka @ 2011-02-24 1:29 ` Ryousei Takano 2011-02-24 7:56 ` Jan Kiszka 0 siblings, 1 reply; 6+ messages in thread From: Ryousei Takano @ 2011-02-24 1:29 UTC (permalink / raw) To: Jan Kiszka; +Cc: kvm@vger.kernel.org, Marcelo Tosatti Hi Jan, Thanks for your help. On Thu, Feb 24, 2011 at 2:26 AM, Jan Kiszka <jan.kiszka@siemens.com> wrote: > On 2011-02-23 16:53, Ryousei Takano wrote: >> Hi Jan, >> >> On Wed, Feb 23, 2011 at 11:47 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote: >>> On 2011-02-23 15:38, Ryousei Takano wrote: >>>> This reverts commit c65827770b5e7282b5fb9da29c1de85ed08e816e >>> >>> Reasoning would belong here, not below the separator. >>> >>>> >>>> Signed-off-by: Ryousei Takano <takano-ryousei@aist.go.jp> >>>> --- >>>> I am trying to run OpenMP (i.e., multi-threaded) programs using Intel >>>> Math Kernel Library (MKL) on a guest OS. In the qemu-kvm-0.12.5, >>>> my program works fine. In the qemu-kvm-0.13.0, however, it does not. >>> >>> What about 0.14? >>> >> 0.14-rc1 also has problem. > > Not automagically healed - what a pity. > >> >>>> The Linux kernel is 2.6.32.28. The outputs of 'ps -L' are shown as follows: >>>> >>>> qemu-kvm-0.12.5: >>>> PID LWP TTY TIME CMD >>>> 2731 2731 pts/0 00:00:00 bash >>>> 2799 2799 pts/0 00:00:19 a.out >>>> 2799 2803 pts/0 00:00:00 a.out >>>> 2799 2804 pts/0 00:00:04 a.out <- >>>> 2799 2805 pts/0 00:00:03 a.out <- these threads are working properly. >>>> 2799 2806 pts/0 00:00:04 a.out <- >>>> 2807 2807 pts/0 00:00:00 ps >>>> >>>> qemu-0.13.0: >>>> PID LWP TTY TIME CMD >>>> 2725 2725 pts/0 00:00:00 bash >>>> 2748 2748 pts/0 00:00:19 a.out >>>> 2748 2752 pts/0 00:00:00 a.out >>>> 2748 2753 pts/0 00:00:00 a.out <- >>>> 2748 2754 pts/0 00:00:00 a.out <- these threads are blocked by >>>> pthread_cond_wait(3). >>>> 2748 2755 pts/0 00:00:00 a.out <- >>>> 2756 2756 pts/0 00:00:00 ps >>>> >>>> This problem seems to be introduced by the commit >>>> c65827770b5e7282b5fb9da29c1de85ed08e816e. >>>> The attached revert patch solves it. But I do not understand what is >>>> happened;-) >>>> Any comments and suggestions will be appreciated. >>>> >>>> Regards, >>>> Ryousei >>>> >>>> hw/pc.c | 6 ++++++ >>>> target-i386/helper.c | 2 -- >>>> 2 files changed, 6 insertions(+), 2 deletions(-) >>>> >>>> diff --git a/hw/pc.c b/hw/pc.c >>>> index ec97181..577abe7 100644 >>>> --- a/hw/pc.c >>>> +++ b/hw/pc.c >>>> @@ -937,6 +937,12 @@ CPUState *pc_new_cpu(const char *cpu_model) >>>> } >>>> qemu_register_reset(pc_cpu_reset, env); >>>> pc_cpu_reset(env); >>>> + >>>> + /* kvm needs this to run after the apic is initialized. Otherwise, >>>> + * it can access invalid state and crash. >>>> + */ >>>> + qemu_init_vcpu(env); >>>> + >>>> return env; >>>> } >>>> >>>> diff --git a/target-i386/helper.c b/target-i386/helper.c >>>> index 86202e6..69cda5e 100644 >>>> --- a/target-i386/helper.c >>>> +++ b/target-i386/helper.c >>>> @@ -1224,8 +1224,6 @@ CPUX86State *cpu_x86_init(const char *cpu_model) >>>> } >>>> mce_init(env); >>>> >>>> - qemu_init_vcpu(env); >>>> - >>>> return env; >>>> } >>>> >>> >>> I don't believe yet that the patch is the reason, specifically as it >>> targets the setup phase (except for CPU hotplug, but that's broken for >>> ages), not the operational phase of the VM. >>> >>> What's your qemu-kvm command line? >>> >> Here is my command line. >> >> sudo qemu-system-x86_64 \ >> -cpu host \ >> -smp 8 \ >> -m 46080 \ >> -net nic,model=virtio,macaddr=00:16:3e:1d:ff:00 \ >> -net tap,ifname=tap0,script=no,downscript=no \ >> -monitor telnet::5963,server,nowait \ >> -daemonize \ >> disk.img > > How many cores are available on the host? >= 8 or less? > 8 cores. All cores are allocated to the VM. If 7 cores are allocated, the problem is reproduced. >> >>> Also, please attach gdb to the stuck VM and print the backtraces of all >>> threads ("thread apply all bt"). >>> >> (gdb) thread apply all bt >> Thread 9 (Thread 0x41129940 (LWP 8118)): >> #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 >> #1 0x000000000042e56d in pthread_attr_setdetachstate () >> #2 0x000000000042ea99 in pthread_attr_setdetachstate () >> #3 0x000000000042ecf0 in pthread_attr_setdetachstate () >> #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 >> #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 >> >> Thread 8 (Thread 0x41ca8940 (LWP 8119)): >> #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 >> #1 0x000000000042e56d in pthread_attr_setdetachstate () >> #2 0x000000000042ea99 in pthread_attr_setdetachstate () >> #3 0x000000000042ecf0 in pthread_attr_setdetachstate () >> #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 >> #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 >> >> Thread 7 (Thread 0x425f2940 (LWP 8120)): >> #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 >> #1 0x000000000042e56d in pthread_attr_setdetachstate () >> #2 0x000000000042ea99 in pthread_attr_setdetachstate () >> #3 0x000000000042ecf0 in pthread_attr_setdetachstate () >> #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 >> #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 >> >> Thread 6 (Thread 0x42df3940 (LWP 8121)): >> #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 >> #1 0x000000000042e56d in pthread_attr_setdetachstate () >> #2 0x000000000042ea99 in pthread_attr_setdetachstate () >> #3 0x000000000042ecf0 in pthread_attr_setdetachstate () >> #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 >> #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 >> >> Thread 5 (Thread 0x435f4940 (LWP 8122)): >> #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 >> #1 0x000000000042e56d in pthread_attr_setdetachstate () >> #2 0x000000000042ea99 in pthread_attr_setdetachstate () >> #3 0x000000000042ecf0 in pthread_attr_setdetachstate () >> #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 >> #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 >> >> Thread 4 (Thread 0x43df5940 (LWP 8123)): >> #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 >> #1 0x000000000042e56d in pthread_attr_setdetachstate () >> #2 0x000000000042ea99 in pthread_attr_setdetachstate () >> #3 0x000000000042ecf0 in pthread_attr_setdetachstate () >> #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 >> #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 >> >> Thread 3 (Thread 0x445f6940 (LWP 8124)): >> #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 >> #1 0x000000000042e56d in pthread_attr_setdetachstate () >> #2 0x000000000042ea99 in pthread_attr_setdetachstate () >> #3 0x000000000042ecf0 in pthread_attr_setdetachstate () >> #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 >> #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 >> >> Thread 2 (Thread 0x44df7940 (LWP 8125)): >> #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 >> #1 0x000000000042e56d in pthread_attr_setdetachstate () >> #2 0x000000000042ea99 in pthread_attr_setdetachstate () >> #3 0x000000000042ecf0 in pthread_attr_setdetachstate () >> #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 >> #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 >> >> Thread 1 (Thread 0x7fe796ec1740 (LWP 8117)): >> #0 0x0000003dd38ccfc2 in select () from /lib64/libc.so.6 >> #1 0x0000000000419620 in pthread_attr_setdetachstate () >> #2 0x000000000042be07 in pthread_attr_setdetachstate () >> #3 0x000000000041ae20 in pthread_attr_setdetachstate () >> #4 0x0000003dd381d994 in __libc_start_main () from /lib64/libc.so.6 >> #5 0x00000000004063d9 in pthread_attr_setdetachstate () >> #6 0x00007fff8050c988 in ?? () >> #7 0x0000000000000000 in ?? () >> > > Can you retry with --enable-debug set during qemu-kvm configure? We need > the symbols of the qemu functions. Despite the lacking symbols, this > rather looks like we are stuck in kernel space, not on some condition > variable (ie. futex). > Here is the result with --enable-debug. Thread 9 (Thread 0x410c5940 (LWP 19303)): #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 #1 0x000000000043b85b in kvm_run (env=0x284c7c0) at /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:582 #2 0x000000000043cf7f in kvm_cpu_exec (env=0x284c7c0) at /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1233 #3 0x000000000043d73e in kvm_main_loop_cpu (env=0x284c7c0) at /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1419 #4 0x000000000043d8ac in ap_main_loop (_env=0x284c7c0) at /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1466 #5 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 #6 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 Thread 8 (Thread 0x418c6940 (LWP 19304)): #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 #1 0x000000000043b85b in kvm_run (env=0x2868180) at /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:582 #2 0x000000000043cf7f in kvm_cpu_exec (env=0x2868180) at /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1233 #3 0x000000000043d73e in kvm_main_loop_cpu (env=0x2868180) at /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1419 #4 0x000000000043d8ac in ap_main_loop (_env=0x2868180) at /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1466 #5 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 #6 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 Thread 7 (Thread 0x420c7940 (LWP 19305)): #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 #1 0x000000000043b85b in kvm_run (env=0x2877c70) at /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:582 #2 0x000000000043cf7f in kvm_cpu_exec (env=0x2877c70) at /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1233 #3 0x000000000043d73e in kvm_main_loop_cpu (env=0x2877c70) at /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1419 #4 0x000000000043d8ac in ap_main_loop (_env=0x2877c70) at /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1466 #5 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 #6 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 Thread 6 (Thread 0x428c8940 (LWP 19306)): #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 #1 0x000000000043b85b in kvm_run (env=0x2887740) at /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:582 #2 0x000000000043cf7f in kvm_cpu_exec (env=0x2887740) at /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1233 #3 0x000000000043d73e in kvm_main_loop_cpu (env=0x2887740) at /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1419 #4 0x000000000043d8ac in ap_main_loop (_env=0x2887740) at /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1466 #5 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 #6 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 Thread 5 (Thread 0x430c9940 (LWP 19307)): #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 #1 0x000000000043b85b in kvm_run (env=0x2897210) at /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:582 #2 0x000000000043cf7f in kvm_cpu_exec (env=0x2897210) at /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1233 #3 0x000000000043d73e in kvm_main_loop_cpu (env=0x2897210) at /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1419 #4 0x000000000043d8ac in ap_main_loop (_env=0x2897210) at /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1466 #5 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 #6 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 Thread 4 (Thread 0x438ca940 (LWP 19308)): #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 #1 0x000000000043b85b in kvm_run (env=0x28a6ce0) at /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:582 #2 0x000000000043cf7f in kvm_cpu_exec (env=0x28a6ce0) at /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1233 #3 0x000000000043d73e in kvm_main_loop_cpu (env=0x28a6ce0) at /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1419 #4 0x000000000043d8ac in ap_main_loop (_env=0x28a6ce0) at /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1466 #5 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 #6 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 Thread 3 (Thread 0x440cb940 (LWP 19309)): #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 #1 0x000000000043b85b in kvm_run (env=0x28b67b0) at /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:582 #2 0x000000000043cf7f in kvm_cpu_exec (env=0x28b67b0) at /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1233 #3 0x000000000043d73e in kvm_main_loop_cpu (env=0x28b67b0) at /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1419 #4 0x000000000043d8ac in ap_main_loop (_env=0x28b67b0) at /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1466 #5 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 #6 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 Thread 2 (Thread 0x448cc940 (LWP 19310)): #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 #1 0x000000000043b85b in kvm_run (env=0x28c6280) at /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:582 #2 0x000000000043cf7f in kvm_cpu_exec (env=0x28c6280) at /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1233 #3 0x000000000043d73e in kvm_main_loop_cpu (env=0x28c6280) at /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1419 #4 0x000000000043d8ac in ap_main_loop (_env=0x28c6280) at /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1466 #5 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 #6 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 Thread 1 (Thread 0x7fd303b9c740 (LWP 19302)): #0 0x0000003dd38ccfc2 in select () from /lib64/libc.so.6 #1 0x00000000004207a5 in main_loop_wait (nonblocking=0) at /home/takano/qemu-kvm-0.14.0-rc1/vl.c:1376 #2 0x000000000043dd48 in kvm_main_loop () at /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1589 #3 0x0000000000420ad2 in main_loop () at /home/takano/qemu-kvm-0.14.0-rc1/vl.c:1429 #4 0x0000000000424b39 in main (argc=15, argv=0x7fffeb76db58, envp=0x7fffeb76dbd8) at /home/takano/qemu-kvm-0.14.0-rc1/vl.c:3201 > Then probably also interesting would be a trace of the host kernel > (trace-cmd record -e kvm, see e.g. [1]). > OK. I will try trace as soon as possible. Regards, Ryousei > Thanks, > Jan > > [1] http://lwn.net/Articles/341902/ > > -- > Siemens AG, Corporate Technology, CT T DE IT 1 > Corporate Competence Center Embedded Linux > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] Revert "qemu-kvm: Bring qemu_init_vcpu back home" 2011-02-24 1:29 ` Ryousei Takano @ 2011-02-24 7:56 ` Jan Kiszka 0 siblings, 0 replies; 6+ messages in thread From: Jan Kiszka @ 2011-02-24 7:56 UTC (permalink / raw) To: Ryousei Takano; +Cc: kvm@vger.kernel.org, Marcelo Tosatti On 2011-02-24 02:29, Ryousei Takano wrote: > Hi Jan, > > Thanks for your help. > > On Thu, Feb 24, 2011 at 2:26 AM, Jan Kiszka <jan.kiszka@siemens.com> wrote: >> On 2011-02-23 16:53, Ryousei Takano wrote: >>> Hi Jan, >>> >>> On Wed, Feb 23, 2011 at 11:47 PM, Jan Kiszka <jan.kiszka@siemens.com> wrote: >>>> On 2011-02-23 15:38, Ryousei Takano wrote: >>>>> This reverts commit c65827770b5e7282b5fb9da29c1de85ed08e816e >>>> >>>> Reasoning would belong here, not below the separator. >>>> >>>>> >>>>> Signed-off-by: Ryousei Takano <takano-ryousei@aist.go.jp> >>>>> --- >>>>> I am trying to run OpenMP (i.e., multi-threaded) programs using Intel >>>>> Math Kernel Library (MKL) on a guest OS. In the qemu-kvm-0.12.5, >>>>> my program works fine. In the qemu-kvm-0.13.0, however, it does not. >>>> >>>> What about 0.14? >>>> >>> 0.14-rc1 also has problem. >> >> Not automagically healed - what a pity. >> >>> >>>>> The Linux kernel is 2.6.32.28. The outputs of 'ps -L' are shown as follows: >>>>> >>>>> qemu-kvm-0.12.5: >>>>> PID LWP TTY TIME CMD >>>>> 2731 2731 pts/0 00:00:00 bash >>>>> 2799 2799 pts/0 00:00:19 a.out >>>>> 2799 2803 pts/0 00:00:00 a.out >>>>> 2799 2804 pts/0 00:00:04 a.out <- >>>>> 2799 2805 pts/0 00:00:03 a.out <- these threads are working properly. >>>>> 2799 2806 pts/0 00:00:04 a.out <- >>>>> 2807 2807 pts/0 00:00:00 ps >>>>> >>>>> qemu-0.13.0: >>>>> PID LWP TTY TIME CMD >>>>> 2725 2725 pts/0 00:00:00 bash >>>>> 2748 2748 pts/0 00:00:19 a.out >>>>> 2748 2752 pts/0 00:00:00 a.out >>>>> 2748 2753 pts/0 00:00:00 a.out <- >>>>> 2748 2754 pts/0 00:00:00 a.out <- these threads are blocked by >>>>> pthread_cond_wait(3). >>>>> 2748 2755 pts/0 00:00:00 a.out <- >>>>> 2756 2756 pts/0 00:00:00 ps >>>>> >>>>> This problem seems to be introduced by the commit >>>>> c65827770b5e7282b5fb9da29c1de85ed08e816e. >>>>> The attached revert patch solves it. But I do not understand what is >>>>> happened;-) >>>>> Any comments and suggestions will be appreciated. >>>>> >>>>> Regards, >>>>> Ryousei >>>>> >>>>> hw/pc.c | 6 ++++++ >>>>> target-i386/helper.c | 2 -- >>>>> 2 files changed, 6 insertions(+), 2 deletions(-) >>>>> >>>>> diff --git a/hw/pc.c b/hw/pc.c >>>>> index ec97181..577abe7 100644 >>>>> --- a/hw/pc.c >>>>> +++ b/hw/pc.c >>>>> @@ -937,6 +937,12 @@ CPUState *pc_new_cpu(const char *cpu_model) >>>>> } >>>>> qemu_register_reset(pc_cpu_reset, env); >>>>> pc_cpu_reset(env); >>>>> + >>>>> + /* kvm needs this to run after the apic is initialized. Otherwise, >>>>> + * it can access invalid state and crash. >>>>> + */ >>>>> + qemu_init_vcpu(env); >>>>> + >>>>> return env; >>>>> } >>>>> >>>>> diff --git a/target-i386/helper.c b/target-i386/helper.c >>>>> index 86202e6..69cda5e 100644 >>>>> --- a/target-i386/helper.c >>>>> +++ b/target-i386/helper.c >>>>> @@ -1224,8 +1224,6 @@ CPUX86State *cpu_x86_init(const char *cpu_model) >>>>> } >>>>> mce_init(env); >>>>> >>>>> - qemu_init_vcpu(env); >>>>> - >>>>> return env; >>>>> } >>>>> >>>> >>>> I don't believe yet that the patch is the reason, specifically as it >>>> targets the setup phase (except for CPU hotplug, but that's broken for >>>> ages), not the operational phase of the VM. >>>> >>>> What's your qemu-kvm command line? >>>> >>> Here is my command line. >>> >>> sudo qemu-system-x86_64 \ >>> -cpu host \ >>> -smp 8 \ >>> -m 46080 \ >>> -net nic,model=virtio,macaddr=00:16:3e:1d:ff:00 \ >>> -net tap,ifname=tap0,script=no,downscript=no \ >>> -monitor telnet::5963,server,nowait \ >>> -daemonize \ >>> disk.img >> >> How many cores are available on the host? >= 8 or less? >> > 8 cores. All cores are allocated to the VM. > If 7 cores are allocated, the problem is reproduced. > >>> >>>> Also, please attach gdb to the stuck VM and print the backtraces of all >>>> threads ("thread apply all bt"). >>>> >>> (gdb) thread apply all bt >>> Thread 9 (Thread 0x41129940 (LWP 8118)): >>> #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 >>> #1 0x000000000042e56d in pthread_attr_setdetachstate () >>> #2 0x000000000042ea99 in pthread_attr_setdetachstate () >>> #3 0x000000000042ecf0 in pthread_attr_setdetachstate () >>> #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 >>> #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 >>> >>> Thread 8 (Thread 0x41ca8940 (LWP 8119)): >>> #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 >>> #1 0x000000000042e56d in pthread_attr_setdetachstate () >>> #2 0x000000000042ea99 in pthread_attr_setdetachstate () >>> #3 0x000000000042ecf0 in pthread_attr_setdetachstate () >>> #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 >>> #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 >>> >>> Thread 7 (Thread 0x425f2940 (LWP 8120)): >>> #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 >>> #1 0x000000000042e56d in pthread_attr_setdetachstate () >>> #2 0x000000000042ea99 in pthread_attr_setdetachstate () >>> #3 0x000000000042ecf0 in pthread_attr_setdetachstate () >>> #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 >>> #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 >>> >>> Thread 6 (Thread 0x42df3940 (LWP 8121)): >>> #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 >>> #1 0x000000000042e56d in pthread_attr_setdetachstate () >>> #2 0x000000000042ea99 in pthread_attr_setdetachstate () >>> #3 0x000000000042ecf0 in pthread_attr_setdetachstate () >>> #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 >>> #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 >>> >>> Thread 5 (Thread 0x435f4940 (LWP 8122)): >>> #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 >>> #1 0x000000000042e56d in pthread_attr_setdetachstate () >>> #2 0x000000000042ea99 in pthread_attr_setdetachstate () >>> #3 0x000000000042ecf0 in pthread_attr_setdetachstate () >>> #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 >>> #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 >>> >>> Thread 4 (Thread 0x43df5940 (LWP 8123)): >>> #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 >>> #1 0x000000000042e56d in pthread_attr_setdetachstate () >>> #2 0x000000000042ea99 in pthread_attr_setdetachstate () >>> #3 0x000000000042ecf0 in pthread_attr_setdetachstate () >>> #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 >>> #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 >>> >>> Thread 3 (Thread 0x445f6940 (LWP 8124)): >>> #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 >>> #1 0x000000000042e56d in pthread_attr_setdetachstate () >>> #2 0x000000000042ea99 in pthread_attr_setdetachstate () >>> #3 0x000000000042ecf0 in pthread_attr_setdetachstate () >>> #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 >>> #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 >>> >>> Thread 2 (Thread 0x44df7940 (LWP 8125)): >>> #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 >>> #1 0x000000000042e56d in pthread_attr_setdetachstate () >>> #2 0x000000000042ea99 in pthread_attr_setdetachstate () >>> #3 0x000000000042ecf0 in pthread_attr_setdetachstate () >>> #4 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 >>> #5 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 >>> >>> Thread 1 (Thread 0x7fe796ec1740 (LWP 8117)): >>> #0 0x0000003dd38ccfc2 in select () from /lib64/libc.so.6 >>> #1 0x0000000000419620 in pthread_attr_setdetachstate () >>> #2 0x000000000042be07 in pthread_attr_setdetachstate () >>> #3 0x000000000041ae20 in pthread_attr_setdetachstate () >>> #4 0x0000003dd381d994 in __libc_start_main () from /lib64/libc.so.6 >>> #5 0x00000000004063d9 in pthread_attr_setdetachstate () >>> #6 0x00007fff8050c988 in ?? () >>> #7 0x0000000000000000 in ?? () >>> >> >> Can you retry with --enable-debug set during qemu-kvm configure? We need >> the symbols of the qemu functions. Despite the lacking symbols, this >> rather looks like we are stuck in kernel space, not on some condition >> variable (ie. futex). >> > Here is the result with --enable-debug. > > Thread 9 (Thread 0x410c5940 (LWP 19303)): > #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 > #1 0x000000000043b85b in kvm_run (env=0x284c7c0) at > /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:582 > #2 0x000000000043cf7f in kvm_cpu_exec (env=0x284c7c0) at > /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1233 > #3 0x000000000043d73e in kvm_main_loop_cpu (env=0x284c7c0) at > /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1419 > #4 0x000000000043d8ac in ap_main_loop (_env=0x284c7c0) at > /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1466 > #5 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 > #6 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 > > Thread 8 (Thread 0x418c6940 (LWP 19304)): > #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 > #1 0x000000000043b85b in kvm_run (env=0x2868180) at > /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:582 > #2 0x000000000043cf7f in kvm_cpu_exec (env=0x2868180) at > /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1233 > #3 0x000000000043d73e in kvm_main_loop_cpu (env=0x2868180) at > /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1419 > #4 0x000000000043d8ac in ap_main_loop (_env=0x2868180) at > /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1466 > #5 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 > #6 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 > > Thread 7 (Thread 0x420c7940 (LWP 19305)): > #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 > #1 0x000000000043b85b in kvm_run (env=0x2877c70) at > /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:582 > #2 0x000000000043cf7f in kvm_cpu_exec (env=0x2877c70) at > /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1233 > #3 0x000000000043d73e in kvm_main_loop_cpu (env=0x2877c70) at > /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1419 > #4 0x000000000043d8ac in ap_main_loop (_env=0x2877c70) at > /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1466 > #5 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 > #6 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 > > Thread 6 (Thread 0x428c8940 (LWP 19306)): > #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 > #1 0x000000000043b85b in kvm_run (env=0x2887740) at > /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:582 > #2 0x000000000043cf7f in kvm_cpu_exec (env=0x2887740) at > /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1233 > #3 0x000000000043d73e in kvm_main_loop_cpu (env=0x2887740) at > /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1419 > #4 0x000000000043d8ac in ap_main_loop (_env=0x2887740) at > /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1466 > #5 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 > #6 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 > > Thread 5 (Thread 0x430c9940 (LWP 19307)): > #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 > #1 0x000000000043b85b in kvm_run (env=0x2897210) at > /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:582 > #2 0x000000000043cf7f in kvm_cpu_exec (env=0x2897210) at > /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1233 > #3 0x000000000043d73e in kvm_main_loop_cpu (env=0x2897210) at > /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1419 > #4 0x000000000043d8ac in ap_main_loop (_env=0x2897210) at > /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1466 > #5 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 > #6 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 > > Thread 4 (Thread 0x438ca940 (LWP 19308)): > #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 > #1 0x000000000043b85b in kvm_run (env=0x28a6ce0) at > /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:582 > #2 0x000000000043cf7f in kvm_cpu_exec (env=0x28a6ce0) at > /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1233 > #3 0x000000000043d73e in kvm_main_loop_cpu (env=0x28a6ce0) at > /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1419 > #4 0x000000000043d8ac in ap_main_loop (_env=0x28a6ce0) at > /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1466 > #5 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 > #6 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 > > Thread 3 (Thread 0x440cb940 (LWP 19309)): > #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 > #1 0x000000000043b85b in kvm_run (env=0x28b67b0) at > /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:582 > #2 0x000000000043cf7f in kvm_cpu_exec (env=0x28b67b0) at > /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1233 > #3 0x000000000043d73e in kvm_main_loop_cpu (env=0x28b67b0) at > /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1419 > #4 0x000000000043d8ac in ap_main_loop (_env=0x28b67b0) at > /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1466 > #5 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 > #6 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 > > Thread 2 (Thread 0x448cc940 (LWP 19310)): > #0 0x0000003dd38cc647 in ioctl () from /lib64/libc.so.6 > #1 0x000000000043b85b in kvm_run (env=0x28c6280) at > /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:582 > #2 0x000000000043cf7f in kvm_cpu_exec (env=0x28c6280) at > /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1233 > #3 0x000000000043d73e in kvm_main_loop_cpu (env=0x28c6280) at > /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1419 > #4 0x000000000043d8ac in ap_main_loop (_env=0x28c6280) at > /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1466 > #5 0x0000003dd400673d in start_thread () from /lib64/libpthread.so.0 > #6 0x0000003dd38d3d1d in clone () from /lib64/libc.so.6 > > Thread 1 (Thread 0x7fd303b9c740 (LWP 19302)): > #0 0x0000003dd38ccfc2 in select () from /lib64/libc.so.6 > #1 0x00000000004207a5 in main_loop_wait (nonblocking=0) at > /home/takano/qemu-kvm-0.14.0-rc1/vl.c:1376 > #2 0x000000000043dd48 in kvm_main_loop () at > /home/takano/qemu-kvm-0.14.0-rc1/qemu-kvm.c:1589 > #3 0x0000000000420ad2 in main_loop () at > /home/takano/qemu-kvm-0.14.0-rc1/vl.c:1429 > #4 0x0000000000424b39 in main (argc=15, argv=0x7fffeb76db58, > envp=0x7fffeb76dbd8) at /home/takano/qemu-kvm-0.14.0-rc1/vl.c:3201 OK, this finally makes me realize that we are neither talking about a locked up qemu-kvm nor guest kernel but rather some deadlock the guest user space application is seeing. Did you already debugged this to more details? If the guest app is stuck on some cond variable, is the associated condition true now? Did the related cond_signal/broadcast took place? > >> Then probably also interesting would be a trace of the host kernel >> (trace-cmd record -e kvm, see e.g. [1]). >> > OK. I will try trace as soon as possible. Focus tracing (ftrace or even just strace) first of all on the application from inside the guest. That provides more details. Also, compare the activities with working environments (older qemu or even native). Jan -- Siemens AG, Corporate Technology, CT T DE IT 1 Corporate Competence Center Embedded Linux ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2011-02-24 7:57 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2011-02-23 14:38 [PATCH] Revert "qemu-kvm: Bring qemu_init_vcpu back home" Ryousei Takano 2011-02-23 14:47 ` Jan Kiszka 2011-02-23 15:53 ` Ryousei Takano 2011-02-23 17:26 ` Jan Kiszka 2011-02-24 1:29 ` Ryousei Takano 2011-02-24 7:56 ` Jan Kiszka
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox