From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 44A3EC433EF for ; Sun, 26 Sep 2021 09:02:37 +0000 (UTC) Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by mail.kernel.org (Postfix) with ESMTP id BB98861029 for ; Sun, 26 Sep 2021 09:02:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org BB98861029 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.cs.columbia.edu Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 4156B4086D; Sun, 26 Sep 2021 05:02:36 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pP1omsSCYS13; Sun, 26 Sep 2021 05:02:35 -0400 (EDT) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 2AA294B0B6; Sun, 26 Sep 2021 05:02:35 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id BC45F4B0B6 for ; Sun, 26 Sep 2021 05:02:33 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 08wRIFV03kW0 for ; Sun, 26 Sep 2021 05:02:32 -0400 (EDT) Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by mm01.cs.columbia.edu (Postfix) with ESMTPS id 830804086D for ; Sun, 26 Sep 2021 05:02:32 -0400 (EDT) Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 6F4DC60FC1; Sun, 26 Sep 2021 09:02:31 +0000 (UTC) Received: from sofa.misterjones.org ([185.219.108.64] helo=wait-a-minute.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1mUQ3J-00D1iV-6U; Sun, 26 Sep 2021 10:02:29 +0100 Date: Sun, 26 Sep 2021 10:02:28 +0100 Message-ID: <877df3btgb.wl-maz@kernel.org> From: Marc Zyngier To: Paolo Bonzini Subject: Re: [PATCH 07/14] KVM: Don't block+unblock when halt-polling is successful In-Reply-To: <80d90ee6-0d43-3735-5c26-be8c3d72d493@redhat.com> References: <20210925005528.1145584-1-seanjc@google.com> <20210925005528.1145584-8-seanjc@google.com> <878rzlass2.wl-maz@kernel.org> <80d90ee6-0d43-3735-5c26-be8c3d72d493@redhat.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: pbonzini@redhat.com, seanjc@google.com, chenhuacai@kernel.org, aleksandar.qemu.devel@gmail.com, paulus@ozlabs.org, borntraeger@de.ibm.com, frankja@linux.ibm.com, james.morse@arm.com, alexandru.elisei@arm.com, suzuki.poulose@arm.com, david@redhat.com, cohuck@redhat.com, imbrenda@linux.ibm.com, vkuznets@redhat.com, wanpengli@tencent.com, jmattson@google.com, joro@8bytes.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-mips@vger.kernel.org, kvm@vger.kernel.org, kvm-ppc@vger.kernel.org, linux-kernel@vger.kernel.org, dmatlack@google.com, jingzhangos@google.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Cc: Wanpeng Li , kvm@vger.kernel.org, David Hildenbrand , linux-kernel@vger.kernel.org, Paul Mackerras , Claudio Imbrenda , kvmarm@lists.cs.columbia.edu, Janosch Frank , Joerg Roedel , Huacai Chen , Christian Borntraeger , Aleksandar Markovic , kvm-ppc@vger.kernel.org, David Matlack , linux-arm-kernel@lists.infradead.org, Jim Mattson , Cornelia Huck , linux-mips@vger.kernel.org, Vitaly Kuznetsov X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu On Sun, 26 Sep 2021 07:27:28 +0100, Paolo Bonzini wrote: > > On 25/09/21 11:50, Marc Zyngier wrote: > >> there is no need for arm64 to put/load > >> the vGIC as KVM hasn't relinquished control of the vCPU in any way. > > > > This doesn't mean that there is no requirement for any state > > change. The put/load on GICv4 is crucial for performance, and the VMCR > > resync is a correctness requirement. > > I wouldn't even say it's crucial for performance: halt polling cannot > work and is a waste of time without (the current implementation of) > put/load. Not quite. A non-V{LPI,SGI} could still be used as the a wake-up from WFI (which is the only reason we end-up on this path). Only LPIs (and SGIs on GICv4.1) can be directly injected, meaning that SPIs and PPIs still follow the standard SW injection model. However, there is still the ICH_VMCR_EL2 requirement (to get the up-to-date priority mask and group enable bits) for SW-injected interrupt wake-up to work correctly, and I really don't want to save that one eagerly on each shallow exit. > However, is activating the doorbell necessary? If possible, polling > the VGIC directly for pending VLPIs without touching the ITS (for > example by emulating IAR reads) may make sense. IIUC that must be > done at EL2 though, so maybe it would even make sense to move all of > halt polling to EL2 for the nVHE case. It all depends on benchmark > results, of course. No, there is no architectural way to observe the VLPI state. EL2 cannot impersonate the guest an read ICV_IAR1_EL1 (because it conveniently has the same encoding as ICC_IAR1_EL1), and if it could, it would be *destructive* (not what you want). The equivalent of the LR that is used to hold the highest priority VLPI presented to the virtual CPU interface is not visible to SW at all. There are exactly two ways for the hypervisor to get a hint about the VLPI state (and that's only a hint, as everything can be spurious): - Make the vPE non resident and use GICR_VPENDBASER.PendingLast bit to find out whether there are pending VLPIs - Make the vPE non resident and get a doorbell interrupt See the common pattern? There is no polling mechanism, and the only way to flush the VLPI state to memory is to destroy the GIC view of the vPE, which is a bit counter-productive. It also only work on GICv4.1, and not GICv4 (which is why we don't support live migration on GICv4). > Sorry for the many stupid questions I'm asking lately, but I'm trying > to pay more attention to ARM and understand the VGIC and EL1/EL2 split > better. Feel free to ask any question. The more people understand how the architecture works, the better. M. -- Without deviation from the norm, progress is not possible. _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm