From mboxrd@z Thu Jan  1 00:00:00 1970
From: Marcelo Tosatti <mtosatti@redhat.com>
Subject: Re: [patch 2/2] KVM: x86: add option to advance tscdeadline hrtimer
 expiration
Date: Thu, 11 Dec 2014 18:58:03 -0200
Message-ID: <20141211205803.GA22650@amt.cnet>
References: <20141210205749.035440781@redhat.com>
 <20141210205904.415174860@redhat.com>
 <5488D955.1050002@redhat.com>
 <20141211030753.GA6358@amt.cnet>
 <548A0324.7070504@amacapital.net>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Cc: Paolo Bonzini <pbonzini@redhat.com>, kvm@vger.kernel.org,
	Luiz Capitulino <lcapitulino@redhat.com>,
	Rik van Riel <riel@redhat.com>,
	Radim Krcmar <rkrcmar@redhat.com>
To: Andy Lutomirski <luto@amacapital.net>
Return-path: <kvm-owner@vger.kernel.org>
Received: from mx1.redhat.com ([209.132.183.28]:41787 "EHLO mx1.redhat.com"
	rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
	id S965245AbaLKU6Y (ORCPT <rfc822;kvm@vger.kernel.org>);
	Thu, 11 Dec 2014 15:58:24 -0500
Content-Disposition: inline
In-Reply-To: <548A0324.7070504@amacapital.net>
Sender: kvm-owner@vger.kernel.org
List-ID: <kvm.vger.kernel.org>

On Thu, Dec 11, 2014 at 12:48:36PM -0800, Andy Lutomirski wrote:
> On 12/10/2014 07:07 PM, Marcelo Tosatti wrote:
> > On Thu, Dec 11, 2014 at 12:37:57AM +0100, Paolo Bonzini wrote:
> >>
> >>
> >> On 10/12/2014 21:57, Marcelo Tosatti wrote:
> >>> For the hrtimer which emulates the tscdeadline timer in the guest,
> >>> add an option to advance expiration, and busy spin on VM-entry waiting
> >>> for the actual expiration time to elapse.
> >>>
> >>> This allows achieving low latencies in cyclictest (or any scenario 
> >>> which requires strict timing regarding timer expiration).
> >>>
> >>> Reduces cyclictest avg latency by 50%.
> >>>
> >>> Note: this option requires tuning to find the appropriate value 
> >>> for a particular hardware/guest combination. One method is to measure the 
> >>> average delay between apic_timer_fn and VM-entry. 
> >>> Another method is to start with 1000ns, and increase the value
> >>> in say 500ns increments until avg cyclictest numbers stop decreasing.
> >>
> >> What values are you using in practice for the parameter?
> > 
> > 7us.
> 
> 
> It takes 7us to get from TSC deadline expiration to the *start* of
> vmresume?  That seems rather extreme.
> 
> Is it possible that almost all of that latency is from deadline
> expiration to C-state exit?  If so, can we teach the timer code to wake
> up early to account for that?  We're supposed to know our idle exit
> latency these days.

7us includes: 

idle thread wakeup
idle schedout
ksoftirqd schedin
ksoftirqd schedout
qemu schedin
vm-entry

C-states are disabled of course.