Re: Can we force a KVM VCPU in Guest Mode to Exit to User Mode From User Mode ?

[Avi Kivity <avi@redhat.com>]

On 07/26/2012 01:34 PM, Mian M. Hamayun wrote:
> 
> On 07/26/2012 12:06 PM, Avi Kivity wrote:
>> On 07/26/2012 12:38 PM, Mian M. Hamayun wrote:
>>
>> <snip>
>>
>>> This mechanism 'seems' to work fine when both vcpu threads are in User
>>> Mode. But when booting an SMP Guest, the boot processor (BSP) initially
>>> executes the bootstrap code while the non-boot processors (APs) are
>>> waiting for initial INIT-SIPI-SIPI messages.
>>>
>>> What I fail to understand is if an AP is currently waiting for an INIT
>>> signal, and we call the "run_on_cpu" function above for this cpu, it
>>> blocks the whole system, as the AP is in Guest mode and cannot call the
>>> "flush_queued_work" and the BSP is waiting for this to happen.
>>>
>>> How can we resolve this deadlock ? Is there a way to force the AP to
>>> quit the Guest Mode, by using some specific mechanism from the User
>>> mode ?
>> When a vcpu is waiting for an INIT, it still responds to signals and
>> will return to userspace if a signal is received.  Did you observe
>> something different?
>>
> 
> Hi Avi,
> 
> So it means that when we execute the following:
> 
> err = pthread_kill(env->thread, SIG_IPI);
> 
> then this VCPU thread should wake-up and force the VCPU to quit the
> guest mode ?

Yes.

> But I am not getting this behavior as the VCPU thread remains blocked in
> Guest Mode.
> What could be wrong here ?
> 

Perhaps signals are blocked?

Can you share your reproducer?

> Many Thanks,
> Hamayun
> 
> P.S. Please see the following trace; As it might help understanding the
> problem. (thread 2 is the BSP and thread 3 is for AP)
> 
> run_on_cpu: Kicked CPU#1 ... Waiting on qemu_work_cond
> 
> Program received signal SIGUSR1, User defined signal 1.
> ^C
> Program received signal SIGINT, Interrupt.
> 0xb7fdd424 in __kernel_vsyscall ()
> (gdb) info threads
>   Id   Target Id         Frame
>   3    Thread 0xaea17b40 (LWP 2843) "arch.x" 0xb7fdd424 in
> __kernel_vsyscall ()
>   2    Thread 0xaf218b40 (LWP 2842) "arch.x" 0xb7fdd424 in
> __kernel_vsyscall ()
> * 1    Thread 0xb7359940 (LWP 2822) "arch.x" 0xb7fdd424 in
> __kernel_vsyscall ()
> (gdb) thread 2
> [Switching to thread 2 (Thread 0xaf218b40 (LWP 2842))]
> #0  0xb7fdd424 in __kernel_vsyscall ()
> (gdb) bt
> #0  0xb7fdd424 in __kernel_vsyscall ()
> #1  0xb7f7096b in pthread_cond_wait@@GLIBC_2.3.2 () from
> /lib/i386-linux-gnu/libpthread.so.0
> #2  0xb7fc5715 in qemu_cond_wait (cond=0xb7fda640, lock=0xb7fda670) at
> gdb_srv_arch.c:578
> #3  0xb7fc5981 in run_on_cpu (env=0x8465818, func=0xb7fc5417
> <kvm_invoke_set_guest_debug>, data=0xaf20fe50) at gdb_srv_arch.c:782
> #4  0xb7fc5b90 in kvm_update_guest_debug (env=0x8465818,
> reinject_trap=0) at gdb_srv_arch.c:863
> #5  0xb7fc5ef1 in kvm_remove_all_breakpoints (current_env=0x8465418) at
> gdb_srv_arch.c:983
> #6  0xb7fc2e0a in gdb_breakpoint_remove_all (env=0x8465418) at
> gdb_srv.c:369
> #7  0xb7fc3139 in gdb_handle_packet (s=0x80ac3e0, line_buf=0x80ac3fc
> "?") at gdb_srv.c:497
> #8  0xb7fc439c in gdb_read_byte (s=0x80ac3e0, ch=102) at gdb_srv.c:830
> #9  0xb7fc4522 in gdb_loop (env=0x8465418) at gdb_srv.c:878
> #10 0xb7fc4621 in gdb_srv_handle_debug (env=0x8465418) at gdb_srv.c:924
> #11 0xb7fc5265 in kvm_arch_handle_debug (env=0x8465418) at
> gdb_srv_arch.c:424
> #12 0xb7fac3a8 in kvm_cpu__start (cpu=0x8465418) at kvm-cpu.c:588
> #13 0xb7fc6521 in kvm_cpu_thread (arg=0x8465418) at libkvm-main.c:276
> #14 0xb7f6cd4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
> #15 0xb7d7bace in clone () from /lib/i386-linux-gnu/libc.so.6
> (gdb) thread 3
> [Switching to thread 3 (Thread 0xaea17b40 (LWP 2843))]
> #0  0xb7fdd424 in __kernel_vsyscall ()
> (gdb) bt
> #0  0xb7fdd424 in __kernel_vsyscall ()
> #1  0xb7d73869 in ioctl () from /lib/i386-linux-gnu/libc.so.6
> #2  0xb7fabe9c in kvm_cpu__run (vcpu=0x8465818) at kvm-cpu.c:429
> #3  0xb7fac376 in kvm_cpu__start (cpu=0x8465818) at kvm-cpu.c:578
> #4  0xb7fc6521 in kvm_cpu_thread (arg=0x8465818) at libkvm-main.c:276
> #5  0xb7f6cd4c in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
> #6  0xb7d7bace in clone () from /lib/i386-linux-gnu/libc.so.6
> 

Are you sure thread 3 did not receive the signal?

Try stracing the run.


-- 
error compiling committee.c: too many arguments to function