From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C61AEC4332F for ; Wed, 4 Jan 2023 18:56:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=3xXcC97VyiOboXh7JLTPqMgerqD45XqRUjRXxogVYm0=; b=rsmV532w0cjRSP 6xzxQ/J6eNNu+2MXZ6BZHuoBqDN7uMpG/2D4a0O4/oI1AFrOFquGWGQxRDEtrYKvOeAfaKyCclrTS yBi/7r4lMOQcVKpYxvOHR8PbXCuO97ecOhp9ItY/Aue1Ti2jGsU40v0zngbs37O6eNg5HZMn/XHwq rv1DpFdxamLBPoIIrOhRxIqUnzepyFLCFf0X2H7H4FuFlNiqII73KkThKeEl97R5u4kbNro+Pm8V+ J6UEHSoz6AR/F+xfDNSP6gXee2c5JCJqEEaHw28HeRUZNwH/pHtr7zApQzXUwqNp4NQAkNnh49pad WLnmgnQJpmuGGT02xQlg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1pD8vB-00BFCw-HO; Wed, 04 Jan 2023 18:55:29 +0000 Received: from ams.source.kernel.org ([2604:1380:4601:e00::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1pD8op-00B9e7-Jw for linux-arm-kernel@lists.infradead.org; Wed, 04 Jan 2023 18:48:57 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 07B5BB8189F; Wed, 4 Jan 2023 18:48:54 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id AA084C433EF; Wed, 4 Jan 2023 18:48:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1672858132; bh=+SF+kfQbx4VUQcJZoSCrmg774PV5XNx9OhIJQIGlILM=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=FFk/QTK9xuaSAVGDgY679bx3pLt6t1WbEw6FIB4xnD8bfeLzhO+jEBZt9JDPZwsA1 1ltLtkKfbiyCus/L75iSeLNddO9ug4m7WiDtz1m8IoXONAMfrvQkbuzVjQgW/MhQ8u uytqWxSuNiGSw5o0Gd31d+U52AgsGmhaZI+w+cuWnWgaBFUVA72jt9vjGi9Mk69Tfr WfzVRNnfe2rPdzja5VL1R13fo7p8dR0bwYZQKeimH1m8izvIJeiDttYtWCeb8C2grD aYWboTcmx46hOcIhtn9Gdi41XudWHKMqcydO+9mgyJhZZat96a6tZ+oB7DHFJRRfju cuT2vEYOyKV1g== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1pD8ok-00Gotr-Hu; Wed, 04 Jan 2023 18:48:50 +0000 Date: Wed, 04 Jan 2023 18:48:50 +0000 Message-ID: <86o7re6sq5.wl-maz@kernel.org> From: Marc Zyngier To: Joakim Tjernlund Cc: "mark.rutland@arm.com" , "linux-arm-kernel@lists.infradead.org" Subject: Re: [PATCH] GICv3: Add restart handler to detach CPU from GICv3 In-Reply-To: References: <20221216162128.10808-1-joakim.tjernlund@infinera.com> <86pmbu6y7k.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/28.2 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: Joakim.Tjernlund@infinera.com, mark.rutland@arm.com, linux-arm-kernel@lists.infradead.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230104_104855_997661_B343284E X-CRM114-Status: GOOD ( 47.03 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, 04 Jan 2023 17:23:42 +0000, Joakim Tjernlund wrote: > > On Wed, 2023-01-04 at 16:50 +0000, Marc Zyngier wrote: > > On Wed, 04 Jan 2023 16:04:14 +0000, > > Mark Rutland wrote: > > > > > > On Tue, Jan 03, 2023 at 04:27:07PM +0000, Joakim Tjernlund wrote: > > > > On Fri, 2022-12-16 at 17:21 +0100, Joakim Tjernlund wrote: > > > > > > > > Ping? > > > > > > To whom? > > > > > > You don't appeared to have Cc'd any relevant maintainer, and people are still > > > on holiday, so it's extremely likely this will be missed. > > > > That, plus nobody reads the list looking for this sort of things. > > > > > > > > For the maintainer, please use scripts/get_maintainer.pl, e.g. > > > > > > > [mark@lakrids:~/src/linux]% ./scripts/get_maintainer.pl -f drivers/irqchip/irq-gic-v3.c > > > > Thomas Gleixner (maintainer:IRQCHIP DRIVERS) > > > > Marc Zyngier (maintainer:IRQCHIP DRIVERS) > > > > linux-kernel@vger.kernel.org (open list:IRQCHIP DRIVERS) > > > > > > Note: I've Cc'd Marc, who wrote the GICv3 driver. > > > > Cheers Mark, much appreciated. > > Sorry for missing that extra maintainer CC: > > > > > > > > > > > Needed for reboot without resetting the whole GIC > > > > > > This doesn't really explain what you're trying to do nor why. > > > > > > Why do you need to "reboot without resetting the whole GIC" ? > > > > > > Do you encounter a problem if we try to reset the whole GIC? > > > > > > Is this for kexec? > > > > > > Is this for some use-case enabled by out-of-tree code? > > > > All valid questions. This smells of a terrible hack... > > Yes, all god Q's. And no answer? > > > > > The interesting aspect is that this is only done when DS=1, probably > > meaning that they are doing this in a VM. it also rely on some > > Nope, on our custom target. And you run with DS=1? On bare metal? Humpf... > > > (unbounded) UNPRED behaviour as ProcessorSleep is entered without > > any consideration for Group0... Good luck with that. > > hmm, I am doing the same as PM does which also needs DS=1 so I figured > this was uncontroversial. I seriously doubt anyone is actually using that code in anger. I always found it dodgy, and I'm pretty sure it is totally broken. > > > > > Anyway, I don't think we want any of this stuff. Certainly not without > > a cast-iron justification. > > We use several cores but only one run Linux so a Linux reboot will > only reset its own core. And what happens with the distributor? One of the key assumption of the GIC architecture is that there is only one operating system in control of it, the whole of it. The only way to share it is by virtualising it. Shades of Jailhouse... > > Without this patch I either need to reset the GIC as part of the > reboot or I get RWP timeout when linux starts again. Resetting the > GIC kills IRQ on the other cores for a long time which is unwanted. But that's what happens anyway when Linux boots (we reinitialise the whole distributor). So what is this about? > > The RWP timeout comes from lost HW handshake between core and GIC. > Is there another way to regain the HW handshake ? That's the firmware's job. But overall, getting these timeouts means your GIC has locked up. It would be more interesting to understand *why* you get in that situation, which I presume is due to the way the driver initialises itself. Assuming you use kexec to reboot your Linux instance, does the following help (totally untested)? M. diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c index 997104d4338e..0db35a07ffdb 100644 --- a/drivers/irqchip/irq-gic-v3.c +++ b/drivers/irqchip/irq-gic-v3.c @@ -1186,12 +1186,25 @@ static int gic_dist_supports_lpis(void) static void gic_cpu_init(void) { void __iomem *rbase; + bool g0; int i; /* Register ourselves with the rest of the world */ if (gic_populate_rdist()) return; + g0 = gic_has_group0(); + + if (read_gicreg(ICC_IGRPEN1_EL1) || + (g0 && read_gicreg(ICC_IGRPEN0_EL1))) { + if (g0) + write_gicreg(0, ICC_IGRPEN0_EL1); + + write_gicreg(0, ICC_IGRPEN1_EL1); + isb(); + gic_enable_redist(false); + } + gic_enable_redist(true); WARN((gic_data.ppi_nr > 16 || GIC_ESPI_NR != 0) && -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel