From mboxrd@z Thu Jan 1 00:00:00 1970 From: Paolo Bonzini Subject: Re: [patch 2/2] KVM: x86: add option to advance tscdeadline hrtimer expiration Date: Thu, 11 Dec 2014 22:10:30 +0100 Message-ID: <548A0846.1020406@redhat.com> References: <20141210205749.035440781@redhat.com> <20141210205904.415174860@redhat.com> <5488D955.1050002@redhat.com> <20141211030753.GA6358@amt.cnet> <548A0324.7070504@amacapital.net> Mime-Version: 1.0 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit Cc: kvm@vger.kernel.org, Luiz Capitulino , Rik van Riel , Radim Krcmar To: Andy Lutomirski , Marcelo Tosatti Return-path: Received: from mx1.redhat.com ([209.132.183.28]:46445 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933337AbaLKVKi (ORCPT ); Thu, 11 Dec 2014 16:10:38 -0500 In-Reply-To: <548A0324.7070504@amacapital.net> Sender: kvm-owner@vger.kernel.org List-ID: On 11/12/2014 21:48, Andy Lutomirski wrote: > On 12/10/2014 07:07 PM, Marcelo Tosatti wrote: >> On Thu, Dec 11, 2014 at 12:37:57AM +0100, Paolo Bonzini wrote: >>> >>> >>> On 10/12/2014 21:57, Marcelo Tosatti wrote: >>>> For the hrtimer which emulates the tscdeadline timer in the guest, >>>> add an option to advance expiration, and busy spin on VM-entry waiting >>>> for the actual expiration time to elapse. >>>> >>>> This allows achieving low latencies in cyclictest (or any scenario >>>> which requires strict timing regarding timer expiration). >>>> >>>> Reduces cyclictest avg latency by 50%. >>>> >>>> Note: this option requires tuning to find the appropriate value >>>> for a particular hardware/guest combination. One method is to measure the >>>> average delay between apic_timer_fn and VM-entry. >>>> Another method is to start with 1000ns, and increase the value >>>> in say 500ns increments until avg cyclictest numbers stop decreasing. >>> >>> What values are you using in practice for the parameter? >> >> 7us. > > It takes 7us to get from TSC deadline expiration to the *start* of > vmresume? That seems rather extreme. No, to the end. 7us is 21000 clock cycles, and the vmexit+vmentry alone costs about 1300. > Is it possible that almost all of that latency is from deadline > expiration to C-state exit? No, I don't think so. Marcelo confirmed that C-states are disabled, bt anyway none of the C-state latency matches Marcelo's data: C1 is really small (1 us), C1e is too large (~10 us). To see the effect of C-state exit, go to the plots I made on a normal laptop and see latency jumping up to 200000 or 400000 cycles (respectively 70 and 140 us, corresponding to C3 and C6 latencies of 60 and 80 us). > If so, can we teach the timer code to wake > up early to account for that? What, it doesn't already do that? Paolo