Re: [patch 2/2] KVM: x86: add option to advance tscdeadline hrtimer expiration

[Paolo Bonzini <pbonzini@redhat.com>]

On 11/12/2014 21:48, Andy Lutomirski wrote:
> On 12/10/2014 07:07 PM, Marcelo Tosatti wrote:
>> On Thu, Dec 11, 2014 at 12:37:57AM +0100, Paolo Bonzini wrote:
>>>
>>>
>>> On 10/12/2014 21:57, Marcelo Tosatti wrote:
>>>> For the hrtimer which emulates the tscdeadline timer in the guest,
>>>> add an option to advance expiration, and busy spin on VM-entry waiting
>>>> for the actual expiration time to elapse.
>>>>
>>>> This allows achieving low latencies in cyclictest (or any scenario 
>>>> which requires strict timing regarding timer expiration).
>>>>
>>>> Reduces cyclictest avg latency by 50%.
>>>>
>>>> Note: this option requires tuning to find the appropriate value 
>>>> for a particular hardware/guest combination. One method is to measure the 
>>>> average delay between apic_timer_fn and VM-entry. 
>>>> Another method is to start with 1000ns, and increase the value
>>>> in say 500ns increments until avg cyclictest numbers stop decreasing.
>>>
>>> What values are you using in practice for the parameter?
>>
>> 7us.
> 
> It takes 7us to get from TSC deadline expiration to the *start* of
> vmresume?  That seems rather extreme.

No, to the end.  7us is 21000 clock cycles, and the vmexit+vmentry alone
costs about 1300.

> Is it possible that almost all of that latency is from deadline
> expiration to C-state exit?

No, I don't think so.  Marcelo confirmed that C-states are disabled, bt
anyway none of the C-state latency matches Marcelo's data: C1 is really
small (1 us), C1e is too large (~10 us).

To see the effect of C-state exit, go to the plots I made on a normal
laptop and see latency jumping up to 200000 or 400000 cycles
(respectively 70 and 140 us, corresponding to C3 and C6 latencies of 60
and 80 us).

> If so, can we teach the timer code to wake
> up early to account for that?

What, it doesn't already do that?

Paolo