From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.5 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_PASS,USER_AGENT_MUTT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E84F1C282DA for ; Tue, 16 Apr 2019 16:39:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id B71D420821 for ; Tue, 16 Apr 2019 16:39:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729970AbfDPQj0 (ORCPT ); Tue, 16 Apr 2019 12:39:26 -0400 Received: from mga07.intel.com ([134.134.136.100]:39532 "EHLO mga07.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729936AbfDPQj0 (ORCPT ); Tue, 16 Apr 2019 12:39:26 -0400 X-Amp-Result: UNSCANNABLE X-Amp-File-Uploaded: False Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga105.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 16 Apr 2019 09:39:25 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,358,1549958400"; d="scan'208";a="134851511" Received: from sjchrist-coffee.jf.intel.com (HELO linux.intel.com) ([10.54.74.181]) by orsmga008.jf.intel.com with ESMTP; 16 Apr 2019 09:39:24 -0700 Date: Tue, 16 Apr 2019 09:39:25 -0700 From: Sean Christopherson To: Liran Alon Cc: Paolo Bonzini , Radim =?utf-8?B?S3LEjW3DocWZ?= , kvm@vger.kernel.org, Wanpeng Li Subject: Re: [PATCH 6/7] KVM: lapic: Clean up the code for handling of a pre-expired hv_timer Message-ID: <20190416163925.GA21674@linux.intel.com> References: <20190412201834.10831-1-sean.j.christopherson@intel.com> <20190412201834.10831-7-sean.j.christopherson@intel.com> <9DCCF040-F617-4E10-9AA3-715F6EB51ADA@oracle.com> <20190415163251.GE24010@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org On Mon, Apr 15, 2019 at 08:25:48PM +0300, Liran Alon wrote: > > > > On 15 Apr 2019, at 19:32, Sean Christopherson wrote: > > > > On Sun, Apr 14, 2019 at 03:15:41PM +0300, Liran Alon wrote: > >> > >> > >>> On 12 Apr 2019, at 23:18, Sean Christopherson wrote: > >>> > >>> Calling apic_timer_expired() is a nop when a timer interrupt is already > >>> pending, i.e. there's no need to call apic_timer_expired() when there's > >>> a pending interrupt and the hv_timer wants to pend its own interrupt. > >>> Separate the two flows to make the code more readable and to avoid an > >>> unnecessary function call and read to ktimer->pending. > >> > >> In case timer is not periodic and r==1, atomic_read(&ktimer->pending) is not executed. > >> > >>> > >>> Cc: Wanpeng Li > >>> Signed-off-by: Sean Christopherson > >>> --- > >>> arch/x86/kvm/lapic.c | 9 ++++++--- > >>> 1 file changed, 6 insertions(+), 3 deletions(-) > >>> > >>> diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c > >>> index 1d649a2af04c..f0be6f148a47 100644 > >>> --- a/arch/x86/kvm/lapic.c > >>> +++ b/arch/x86/kvm/lapic.c > >>> @@ -1703,9 +1703,12 @@ static bool start_hv_timer(struct kvm_lapic *apic) > >>> * the window. For periodic timer, leave the hv timer running for > >>> * simplicity, and the deadline will be recomputed on the next vmexit. > >>> */ > >>> - if (!apic_lvtt_period(apic) && (r || atomic_read(&ktimer->pending))) { > >>> - if (r) > >>> - apic_timer_expired(apic); > >>> + if (!apic_lvtt_period(apic) && atomic_read(&ktimer->pending)) > >>> + return false; > >>> + > >>> + /* set_hv_timer() returns '1' when the timer has already expired. */ > >>> + if (r) { > >>> + apic_timer_expired(apic); > >>> return false; > >>> } > >>> > >>> -- > >>> 2.21.0 > >>> > >> > >> First, I think you should emphasise in commit message that you have actually > >> fixed a rare bug here. In case timer is periodic but given > >> ktimer->tscdeadline has already expired on host, we should call > >> apic_timer_expired(). > > > > Heh, I actually didn't even catch that bug, I was simply cleaning up the > > code because I had a hard time following the logic. > > LOL. So you can put me in the Reported-by tag :P Actually, thinking about this more, I believe the original behavior was correct, if poorly documented. More info below. > >> In addition, when start_hv_timer() returns false, restart_apic_timer() just > >> calls start_sw_timer() which use hrtimer instead of VMX preemption timer. > >> Therefore, it seems a bit ineffective to me for start_hv_timer() to return > >> false in case ktimer->pending or when ktimer->tscdeadline already expired. > >> Shouldn’t we return true in these cases? > > > > That also seemed weird to me. Again, I had a hell of a time following the > > intended logic and didn't want to break anything. AFAICT, the motivation > > for calling start_sw_timer() is to cancel the HV timer, and possibly to > > ensure start_sw_period() is called when necessary. > > I think the motivation is that if there is any reason why hardware > accelerated timer (i.e. VMX preemption timer), can't be used to emulate the > LAPIC timer, then utilise a software hrtimer based implementation instead. My comment was regarding why start_hv_timer() returns was when the hv_timer as already expired. > This does align with why we return false when (!kvm_x86_ops->set_hv_timer) or > (kvm_x86_ops->set_hv_timer() < 0). However, this doesn’t align in case we > have a (non-periodic timer and ktimer->pending) OR ktimer->tscdeadline > already expired OR (!ktimer->tscdeadline). > > In fact, note that start_sw_timer() early-exit when non-periodic timer and > ktimer->pending… Same is also true for start_sw_tscdeadline() early-exit when > (!ktimer->tscdeadline). > > > But the latter will be > > handled by virtue of checking "r" after apic_lvtt_period(), so this? > > > > if (r) { > > apic_timer_expired(apic); > > ktimer->hv_timer_in_use = false; > > return true; > > } > > I think I will just submit a patch to fix all the above examples I made as > this just seems wrong to me. Unless you find something I have missed. :P When the timer is periodic, we're relying on the timer handler to invoke advance_periodic_target_expiration() by way of kvm_lapic_expired_hv_timer(). That's why the original code only checks @r if apic_lvtt_period()==false, i.e. to actually trigger a VMX preemption timer VM-Exit. Note that the return from set_hv_timer() is essentially a hint, e.g. VMX is perfectly fine programming a preemption timer with a value of zero. I think Paolo's suggestion of moving the logic up into restart_apic_timer() is the way to go as it reduces the multiplexing down on start_hv_timer()'s return value.