From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9A856C433F5 for ; Mon, 20 Dec 2021 14:33:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=TbM5G5XBVuMLv2G/71stnLJMnBYjnO+QZXvgSLoUVUU=; b=U0Een5U0jxe4/H KHmCZd3bQrzlLcD1IRVuSSlfVR+HCYQpTXsKAMifxXtfIBMKcLRcwBuURU2H9vJ56eeVY+GN+sB2S eh9R8asL8+24Z6UDCuE6weytd1MkcZeIOQOU0OajntOJycfkTlswIBPUafdRDB47z/8S6bdzPJwf8 jKSo0XrGnQhscgNnaspNAsG5Ao+NSUJu5zDBbjSc9X0j5gcQXus5Rgw8Khf71a40brNNHEL2k9+Ks rhEEzD4wWR7z5olf+UAyT8FAp9qLHUfm+cJ4RAZRr3A7okymP0blfFfG9En7Jx4+2tfwuAwOORlIy N2kbGve3+BETh9FMNdTg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mzJi1-002oIf-RL; Mon, 20 Dec 2021 14:32:15 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mzJeT-002n9e-Tn for linux-arm-kernel@lists.infradead.org; Mon, 20 Dec 2021 14:28:35 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 7529661185; Mon, 20 Dec 2021 14:28:33 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D9AF9C36AE8; Mon, 20 Dec 2021 14:28:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1640010512; bh=1eIe8Rm8nXRPWnknyt0IFILgScA81zHkg8rD11G9fF8=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=p9zS+D9+lM9BGBVlOOR7BEGaTWRoP5RvU6xcAPjrrRXKF51uBZ/B+/gpZKV0njXk8 pRVnc1fLji+Rz2I+yGK3muR3ShtgOg79+EUlLKOdT007Y9vuEpxyHkTuVcIkkn4J2d 8WpundwdMKXb5vMUNIfkkJMKfuB7sM01Zj86B/otdfMzfbB/bv+pGEib6hSU8zllDR ihqBT/zyAC4vVgjCyvZtIdgRBkOAAZg3/Kw6VhD+b0KGT5W2av0w3x1AztV5xxY5zE +GoMmoHcveprG/gKQAtTKJz4KaAnDx6gNIDMp19XQVLIlbVnW9dfPva5sPz7LGuOyz CdjfY5mGXGdFw== Received: from cfbb000407.r.cam.camfibre.uk ([185.219.108.64] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1mzJeQ-00DIsl-SU; Mon, 20 Dec 2021 14:28:30 +0000 Date: Mon, 20 Dec 2021 14:28:30 +0000 Message-ID: <875yrjwdtd.wl-maz@kernel.org> From: Marc Zyngier To: Mark Rutland Cc: Nicolas Saenz Julienne , Will Deacon , paulmck , linux-arm-kernel , rcu , Thomas Gleixner , frederic , kvmarm@lists.cs.columbia.edu, linux-kernel Subject: Re: Possible nohz-full/RCU issue in arm64 KVM In-Reply-To: References: User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: mark.rutland@arm.com, nsaenzju@redhat.com, will@kernel.org, paulmck@kernel.org, linux-arm-kernel@lists.infradead.org, rcu@vger.kernel.org, tglx@linutronix.de, frederic@kernel.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211220_062834_092263_283B3EE2 X-CRM114-Status: GOOD ( 32.57 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, 17 Dec 2021 13:21:39 +0000, Mark Rutland wrote: > > On Fri, Dec 17, 2021 at 12:51:57PM +0100, Nicolas Saenz Julienne wrote: > > Hi All, > > Hi, > > > arm64's guest entry code does the following: > > > > int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) > > { > > [...] > > > > guest_enter_irqoff(); > > > > ret = kvm_call_hyp_ret(__kvm_vcpu_run, vcpu); > > > > [...] > > > > local_irq_enable(); > > > > /* > > * We do local_irq_enable() before calling guest_exit() so > > * that if a timer interrupt hits while running the guest we > > * account that tick as being spent in the guest. We enable > > * preemption after calling guest_exit() so that if we get > > * preempted we make sure ticks after that is not counted as > > * guest time. > > */ > > guest_exit(); > > [...] > > } > > > > > > On a nohz-full CPU, guest_{enter,exit}() delimit an RCU extended quiescent > > state (EQS). Any interrupt happening between local_irq_enable() and > > guest_exit() should disable that EQS. Now, AFAICT all el0 interrupt handlers > > do the right thing if trggered in this context, but el1's won't. Is it > > possible to hit an el1 handler (for example __el1_irq()) there? > > I think you're right that the EL1 handlers can trigger here and > won't exit the EQS. > > I'm not immediately sure what we *should* do here. What does x86 do > for an IRQ taken from a guest mode? I couldn't spot any handling of > that case, but I'm not familiar enough with the x86 exception model > to know if I'm looking in the right place. > > Note that the EL0 handlers *cannot* trigger for an exception taken > from a guest. We use separate vectors while running a guest (for > both VHE and nVHE modes), and from the main kernel's PoV we return > from kvm_call_hyp_ret(). We can ony take IRQ from EL1 *after* that > returns. > > We *might* need to audit the KVM vector handlers to make sure they're not > dependent on RCU protection (I assume they're not, but it's possible something > has leaked into the VHE code). The *intent* certainly is that whatever is used in the VHE code to handle exceptions arising whilst running in guest context must be independent from RCU, if only because we share a bunch with the !VHE code, and RCU is, unfortunately, not a thing there. My most immediate concern is that the VHE/nVHE split now allows all sort of instrumentation in VHE, which may rely on RCU. At the very least, we should make most of the VHE switch code noinstr. Thanks, M. -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel