From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from fhigh-b5-smtp.messagingengine.com (fhigh-b5-smtp.messagingengine.com [202.12.124.156]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9806D449EA8 for ; Wed, 17 Jun 2026 15:07:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=202.12.124.156 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781708841; cv=none; b=lEHjeSkHI3aC10AQekQNVuOYeBHrkjI8lklPmQL5FhmiOz7GSOHxKxyW/jXYaQd3deIPFnID4qv62pT2XLltnIDTBM5DJ70M9ZkbFw+4HpJyaqmMveNMzn4o8R451Cn7Kf02rfKDmduuAz/t2lBR337lqQfoeL0tCGFFWtgOLSw= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1781708841; c=relaxed/simple; bh=jATLY5e3ANpoAJ5c0Q6/7D+MjyZo+jWvO3sQ5r5D75s=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=KDDJUNI6ifTDU5Ejf8JSU5/7ntm641KDbyOi7OlXF2b92EC7IsjO5JVVzMSbY/IiHJJP+gNra0XwYBJFWhuO9yzbm3TW/xthT1Aji3tF2qHd8UvQnEQXRLiLmyR8B7CN6PMHut6GpsPeo6zWavyi2a9vD43iBX1PWV63/0vHSMc= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=shutemov.name; spf=pass smtp.mailfrom=shutemov.name; dkim=pass (2048-bit key) header.d=shutemov.name header.i=@shutemov.name header.b=UQAJ2yL1; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=D3Zr5o+v; arc=none smtp.client-ip=202.12.124.156 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=shutemov.name Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=shutemov.name Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=shutemov.name header.i=@shutemov.name header.b="UQAJ2yL1"; dkim=pass (2048-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b="D3Zr5o+v" Received: from phl-compute-02.internal (phl-compute-02.internal [10.202.2.42]) by mailfhigh.stl.internal (Postfix) with ESMTP id 8ACBE7A0126; Wed, 17 Jun 2026 11:07:17 -0400 (EDT) Received: from phl-frontend-04 ([10.202.2.163]) by phl-compute-02.internal (MEProxy); Wed, 17 Jun 2026 11:07:18 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=shutemov.name; h=cc:cc:content-transfer-encoding:content-type:content-type :date:date:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:subject:subject:to:to; s=fm2; t=1781708837; x=1781795237; bh=329EFRDqzROnZ+um2EYZ/sgk7lK0yO4O f5gjiVQoqCY=; b=UQAJ2yL1P+ASFFm6ATRMzHKvl1utylIcRlpPxFAAqOjXU+RM 3wgu6C6hue+SwMQk7KPxzDR+X2GtjaZR5QR4xRM3r+quDOVaPUDIU31qcevOVHXi +3TSzvcKT/3grnaKsWfubkjhqkNt9HBVfY0VXqF8lVO8paBiS7VnoGJydAAqfUEU xXQY2lmIdzuMU7q/T8/BezBNN4CvCdA7WQNdP+obdo8b3iFlckXc6XLRf78PYN0M OdsQG8oPn0oserExiIcgphaFHAMebeFl7rdFDu00sFYveAee2ndJCIGsb3Ani+r/ SoN0OsISh/IztOnKwLJhy0SAOSKIBE8n+Uq05A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1781708837; x= 1781795237; bh=329EFRDqzROnZ+um2EYZ/sgk7lK0yO4Of5gjiVQoqCY=; b=D 3Zr5o+vci0dehxi0qn5lRP+rzb8Wzz7gFxXL61FWmHui6MTwo9ynBBLEGRK/k+5T TnKlVXyy3C9PVk6/6bJipFeUPn4f01FqYstgVkUKqqQ4gJFq7BsHu9SNlt1rgwZL wTELvt/CCbWcRZe+5R6fNyiQlLF3VjwPdyekxYNnT+eae+V4UYn8nIBWMfCoWdT2 yY3MF7/bQzDB4UoBQk3+i7kh5RqV9Ir0yEw98dzQU2yfntRY9rypqKrgGbTxIA3S kdmQERafK5DRglP6odFInRQRLxCvfrImhHCI1zdV34+x1exWsVkUgMMf9uxetjg+ ZSJU+fUVvEGIEwBQ+p2RQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: dmFkZTEEu0tieGgjfSoFYH7SZTLMospZf0X29ieHf2Cms5WHFP9sCH2U6lygDU8tZTUi1D y7jdlijROYk9RiNv+PRHa/mwW8lqsO03PtCMfo1gv5Tz6Xao5z9yyG1zr0GjMjRdLyAP4+ xZ8J2UOJpRihZmffAWvCVDECM3i6MxaxZ7jK8QNK94MQ0awHDRYP9V4vRDmirodEdFAeKK E/YjYui6fNSubEWL2LTlqZqDSCSXe772OC9pjFicyjp6CjoJ9Tfhl68TZ47J/41scSfO72 HOX937aiUJaNPMEaFB/R9ZFHKSYJUZNoquCFDlyBsMlV6fGTqdaCfuowNj1Cv1shrZ8qBp QUDWVwqeO9/XINwi2IwvT6/e8CTTqpesOZlCoOj4Smm4DOJuaHL9dwSUAM7orxYswdKJ6Y zHNEreSxrgiAsnhG/ZKkj8CgnrD3gw6DU11n4u8ykwUB850TlmucJkr4koBbgwLzqszwXj Cc96dknw2AyFCUq0Bg5j3x5pN3HQ743L2xqMauwjLAHc92WOHJPBBpQk8wCT7jMtvO91jb 20h7msK7uAvmBccexm/eBehdR0b3KSI8zVPTYkPHQfygryRjTqJfgpY23dvENTYabN/9bk wnS5kOD1jXpFg3RAqeKE8yeg8hsp07J8pZEovvZ5WjfQi2n/a1+TmR9CLNFA X-ME-Proxy: Feedback-ID: ie3994620:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Wed, 17 Jun 2026 11:07:14 -0400 (EDT) Date: Wed, 17 Jun 2026 16:07:08 +0100 From: Kiryl Shutsemau To: Doug Anderson Cc: Catalin Marinas , Will Deacon , James Morse , Mark Rutland , Marc Zyngier , Petr Mladek , Thomas Gleixner , Andrew Morton , Baoquan He , Puranjay Mohan , Usama Arif , Breno Leitao , Julien Thierry , Lecopzer Chen , Sumit Garg , kernel-team@meta.com, kexec@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH v3 3/3] arm64: escalate smp_send_stop() to an SDEI NMI as a last resort Message-ID: References: <167493d30ef6c99a44291de14cddd41ced8149c4.1781490440.git.kas@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: On Tue, Jun 16, 2026 at 02:38:16PM -0700, Doug Anderson wrote: > Hi, > > On Sun, Jun 14, 2026 at 7:36 PM Kiryl Shutsemau wrote: > > > > +/* > > + * Bring the local CPU to a stop, saving its register state into the vmcore > > + * on the kdump crash path first. The single point every arm64 stop path > > + * funnels through, so the bookkeeping (mask interrupts, mark offline, mask > > + * SDEI, optionally power off) lives in one place: > > + * > > + * - the regular IPI_CPU_STOP and pseudo-NMI IPI_CPU_STOP_NMI handlers; > > + * - panic_smp_self_stop(), a CPU parking itself on a parallel panic(); > > + * - the SDEI cross-CPU NMI handler (drivers/firmware/arm_sdei_nmi.c), > > + * which reaches CPUs the stop IPIs could not. > > + * > > + * @regs is the register state to record in the vmcore on a crash stop; NULL > > + * means "capture the current context". @die_on_crash decides the kdump crash > > + * path: the IPI stop handlers pass true and power the CPU off (PSCI CPU_OFF, > > + * via __cpu_try_die()) so a capture kernel can reclaim it. The SDEI handler > > + * and panic_smp_self_stop() pass false and only park. For SDEI that is > > + * required, not just conservative: it runs inside an SDEI event that is > > + * deliberately never completed (completing it has firmware resume the wedged > > + * context), and a CPU_OFF from that not-yet-completed context wedges EL3 on > > + * some firmware -- a documented follow-up. Parking also matches this path's > > + * own fallback when CPU_OFF is unavailable. > > Nice to have all the details in the function comment. Any reason why > you didn't use kernel-doc format? Nothing else in this file does, I > guess, but it doesn't seem like it would be a problem to start the > trend... ;-) No reason -- switched it to kernel-doc in v4. > > + if (READ_ONCE(sdei_nmi_stopping)) { > > Don't you need a smp_rmb() before that, to match with the smp_wmb()? No -- there's nothing for it to pair against. An smp_rmb() orders a flag-load before a later *data*-load (the message-passing pattern), but the flag is the only value shared here: the handler reads sdei_nmi_stopping and nothing else published through it. crash_stop, which the stop path reads afterwards, was written before the flag, so reordering the reads can't expose a stale value either. And the handler isn't spinning on the flag: it runs only because firmware delivered the SDEI event, which is a full round-trip (SMC -> EL3 -> GIC -> exception entry) strictly after sdei_nmi_stop_cpus()'s WRITE_ONCE() + smp_wmb() + signal. By the time the read executes the store is already globally visible -- there's no window for a stale read. The publish-before-signal barrier is the half that matters. The IPI core does exactly this. gic_ipi_send_mask() (drivers/irqchip/irq-gic-v3.c) issues dsb(ishst) -- "stores to Normal memory ... visible to the other CPUs before issuing the IPI" -- before sending the SGI, and no IPI handler takes a matching read-side barrier. I use an explicit smp_wmb() only because the SDEI signal goes through an SMC, which carries no such barrier, where gic_ipi_send_mask() bakes it into the SGI send. The backtrace framework this driver plugs into is the same shape: nmi_cpu_backtrace() reads backtrace_mask -- set by nmi_trigger_cpumask_backtrace() before raise() -- with no smp_rmb(). A bare smp_rmb() with no second load to order would just be confusing, so v4 adds a comment at the READ_ONCE() explaining why there isn't one. -- Kiryl Shutsemau / Kirill A. Shutemov