VMEXIT and Threads

VMEXIT and Threads

[Oscar Garcia]

Hello,

I have a question about parallel vmexit calls, I would appreciate any 
answer or suggestion.

I have a host with debian 7 (intel i7 - RAM 8GB), the guest OS is also 
debian. I am running a program with some threads, every thread makes a 
vmexit call. Also every thread runs in a isolated vcpu. The problem is 
that the program does not run fluently, it looks like that every thread 
interfere with each other. This situation does not happen when 
separately processes call vmexit simultaneously. The question is: there 
is any restriction (any lock) that block the threads. I am not sure 
maybe libc, RCU, on even Qemu and KVM?

Thank you

Oscar Gracia

Re: VMEXIT and Threads

[Paolo Bonzini]

Il 06/10/2014 06:45, Oscar Garcia ha scritto:
> 
> I have a host with debian 7 (intel i7 - RAM 8GB), the guest OS is also
> debian. I am running a program with some threads, every thread makes a
> vmexit call. Also every thread runs in a isolated vcpu. The problem is
> that the program does not run fluently, it looks like that every thread
> interfere with each other. This situation does not happen when
> separately processes call vmexit simultaneously. The question is: there
> is any restriction (any lock) that block the threads. I am not sure
> maybe libc, RCU, on even Qemu and KVM?

At the KVM level, _most_ VCPU ioctls can run concurrently because they
only take a VCPU-level mutex.

QEMU however will take a global mutex on each exit to userspace.  What
vmexits are these?

Paolo

Re: VMEXIT and Threads

[Jan Kiszka]

On 2014-10-06 08:51, Paolo Bonzini wrote:
> Il 06/10/2014 06:45, Oscar Garcia ha scritto:
>>
>> I have a host with debian 7 (intel i7 - RAM 8GB), the guest OS is also
>> debian. I am running a program with some threads, every thread makes a
>> vmexit call. Also every thread runs in a isolated vcpu. The problem is
>> that the program does not run fluently, it looks like that every thread
>> interfere with each other. This situation does not happen when
>> separately processes call vmexit simultaneously. The question is: there
>> is any restriction (any lock) that block the threads. I am not sure
>> maybe libc, RCU, on even Qemu and KVM?
> 
> At the KVM level, _most_ VCPU ioctls can run concurrently because they
> only take a VCPU-level mutex.
> 
> QEMU however will take a global mutex on each exit to userspace.  What
> vmexits are these?

Expensive is usually anything related to VGA emulation - as it may draw
synchronously. But also writing to the emulated serial port when that
one dumps to the host's console under X. If only one VCPU is stuck in
this, also if that is unrelated to you primary guest load, even a
trivial "kick NIC in order to send my packet" may have to wait for
milliseconds.

Jan

-- 
Siemens AG, Corporate Technology, CT RTC ITP SES-DE
Corporate Competence Center Embedded Linux

Re: VMEXIT and Threads

[Marcelo Tosatti]

On Mon, Oct 06, 2014 at 01:45:24PM +0900, Oscar Garcia wrote:
> Hello,
>
> I have a question about parallel vmexit calls, I would appreciate
> any answer or suggestion.
>
> I have a host with debian 7 (intel i7 - RAM 8GB), the guest OS is
> also debian. I am running a program with some threads, every thread
> makes a vmexit call. 

How is that vmexit call made?

> Also every thread runs in a isolated vcpu. The
> problem is that the program does not run fluently, it looks like
> that every thread interfere with each other.

Can you be more precise about this? Did you perform a measurement?

>  This situation does not happen when separately processes call vmexit
> simultaneously. The question is: there is any restriction (any lock)
> that block the threads. I am not sure maybe libc, RCU, on even Qemu
> and KVM?

In QEMU there is a mutex which is shared by all vcpus, and is acquired 
right after entry in QEMU, named qemu_mutex.