From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E64FCEB64DD for ; Mon, 7 Aug 2023 10:13:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=G/kuYR5XqFedyL9MeCC7Wuca1hW4ZzBb7ZsfELsGKoc=; b=AO3J6LDBXIxnaY hVM3OGZDuaPKG7hqUt0us1bbk/9+X7bp9ZtMVD4+Be6+oZJSLcT96toK/0NfL0W4jxjGiCSvx/FsP trYs4Eg2hcCRCuCTXy2RXswVY/4Rsr4xXRC3O4CVHOO+fNEKWvZlHZcKzWyeGFGy12wS8yRGyDZ9t +DOiDXb9G8onO+xPjsamRxFCu/goupXyyxgQNUtx/hWCYCtvd980CB06xGLJolxCjh/oYSYMgui/P IZkEEsBMrwCk8/V/Kfw7itskjbutm2PVKBDlV3E4xseGDyKgm5vewNPzMZf/3QU2iMWaDItAMc5sc prtURnkX9B43MUbvxNFQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qSxDz-00GfZt-2x; Mon, 07 Aug 2023 10:12:31 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qSxDt-00GfYQ-2M for linux-arm-kernel@lists.infradead.org; Mon, 07 Aug 2023 10:12:28 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 126241FB; Mon, 7 Aug 2023 03:13:06 -0700 (PDT) Received: from FVFF77S0Q05N.cambridge.arm.com (FVFF77S0Q05N.cambridge.arm.com [10.1.32.139]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3B1A13F59C; Mon, 7 Aug 2023 03:12:20 -0700 (PDT) Date: Mon, 7 Aug 2023 11:12:17 +0100 From: Mark Rutland To: Douglas Anderson Cc: Catalin Marinas , Will Deacon , Sumit Garg , Daniel Thompson , Marc Zyngier , linux-perf-users@vger.kernel.org, ito-yuichi@fujitsu.com, Chen-Yu Tsai , Ard Biesheuvel , Stephen Boyd , Peter Zijlstra , Thomas Gleixner , linux-arm-kernel@lists.infradead.org, kgdb-bugreport@lists.sourceforge.net, Masayoshi Mizuma , "Rafael J . Wysocki" , Lecopzer Chen , Andrey Konovalov , Masahiro Yamada , linux-kernel@vger.kernel.org Subject: Re: [PATCH v9 3/7] arm64: Add framework for a debug IPI Message-ID: References: <20230601213440.2488667-1-dianders@chromium.org> <20230601143109.v9.3.Ie6c132b96ebbbcddbf6954b9469ed40a6960343c@changeid> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20230601143109.v9.3.Ie6c132b96ebbbcddbf6954b9469ed40a6960343c@changeid> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230807_031225_869501_9777FA33 X-CRM114-Status: GOOD ( 44.24 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Thu, Jun 01, 2023 at 02:31:47PM -0700, Douglas Anderson wrote: > From: Sumit Garg > > Introduce a framework for an IPI that will be used for debug > purposes. The primary use case of this IPI will be to generate stack > crawls on other CPUs, but it will also be used to round up CPUs for > kgdb. > > When possible, we try to allocate this debug IPI as an NMI (or a > pseudo NMI). If that fails (due to CONFIG, an incompatible interrupt > controller, a quirk, missing the "irqchip.gicv3_pseudo_nmi=1" kernel > parameter, etc) we fall back to a normal IPI. > > NOTE: hooking this up for CPU backtrace / kgdb will happen in a future > patch, this just adds the framework. > > Signed-off-by: Sumit Garg > Signed-off-by: Douglas Anderson I think that we shouldn't add a framework in a separate file for this: * This is very similar to our existing IPI management in smp.c, so it feels like duplication, or at least another thing we'd like to keep in-sync. * We're going to want an NMI backtrace regardless of KGDB * We're going to want the IPI_CPU_STOP and IPI_CRASH_CPU_STOP IPIs to be NMIs too. I reckon it'd be better to extend the existing IPI logic in smp.c to allow IPIs to be requested as NMIs, e.g. ---- diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index edd63894d61e8..48e6aa62c473e 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -33,6 +33,7 @@ #include #include #include +#include #include #include @@ -926,6 +927,21 @@ static void smp_cross_call(const struct cpumask *target, unsigned int ipinr) __ipi_send_mask(ipi_desc[ipinr], target); } +static bool ipi_should_be_nmi(enum ipi_msg_type ipi) +{ + if (!system_uses_irq_prio_masking()) + return false; + + switch (ipi) { + /* + * TODO: select NMI IPIs here + */ + return true; + default: + return false; + } +} + static void ipi_setup(int cpu) { int i; @@ -933,8 +949,14 @@ static void ipi_setup(int cpu) if (WARN_ON_ONCE(!ipi_irq_base)) return; - for (i = 0; i < nr_ipi; i++) - enable_percpu_irq(ipi_irq_base + i, 0); + for (i = 0; i < nr_ipi; i++) { + if (ipi_should_be_nmi(i)) { + prepare_percpu_nmi(ipi_irq_base + i); + enable_percpu_nmi(ipi_irq_base + i, 0); + } else { + enable_percpu_irq(ipi_irq_base + i, 0); + } + } } #ifdef CONFIG_HOTPLUG_CPU @@ -945,8 +967,14 @@ static void ipi_teardown(int cpu) if (WARN_ON_ONCE(!ipi_irq_base)) return; - for (i = 0; i < nr_ipi; i++) - disable_percpu_irq(ipi_irq_base + i); + for (i = 0; i < nr_ipi; i++) { + if (ipi_should_be_nmi(i)) { + disable_percpu_nmi(ipi_irq_base + i); + teardown_percpu_nmi(ipi_irq_base + i); + } else { + disable_percpu_irq(ipi_irq_base + i); + } + } } #endif @@ -958,11 +986,19 @@ void __init set_smp_ipi_range(int ipi_base, int n) nr_ipi = min(n, NR_IPI); for (i = 0; i < nr_ipi; i++) { - int err; - - err = request_percpu_irq(ipi_base + i, ipi_handler, - "IPI", &cpu_number); - WARN_ON(err); + int err = -EINVAL; + + if (ipi_should_be_nmi(i)) { + err = request_percpu_nmi(ipi_base + i, ipi_handler, + "IPI", &cpu_number); + WARN(err, "Could not request IPI %d as NMI, err=%d\n", + i, err); + } else { + err = request_percpu_irq(ipi_base + i, ipi_handler, + "IPI", &cpu_number); + WARN(err, "Could not request IPI %d as IRQ, err=%d\n", + i, err); + } ipi_desc[i] = irq_to_desc(ipi_base + i); irq_set_status_flags(ipi_base + i, IRQ_HIDDEN); ---- ... and then if we need an IPI for KGDB, we can add that to the existing list of IPIs, and have it requested/enabled/disabled as usual. Thanks, Mark. > --- > I didn't get any feedback from v8 patch #10 [1], but I went ahead and > folded it in here anyway since it really simplfies things. If people > don't like the fallback to regular IPI, I can also undo it. > > [1] https://lore.kernel.org/r/20230419155341.v8.10.Ic3659997d6243139d0522fc3afcdfd88d7a5f030@changeid/ > > Changes in v9: > - Fold in v8 patch #10 ("Fallback to a regular IPI if NMI isn't enabled") > - Moved header file out of "include" since it didn't need to be there. > - Remove arm64_supports_nmi() > - Renamed "NMI IPI" to "debug IPI" since it might not be backed by NMI. > > Changes in v8: > - debug_ipi_setup() and debug_ipi_teardown() no longer take cpu param > > arch/arm64/kernel/Makefile | 2 +- > arch/arm64/kernel/ipi_debug.c | 76 +++++++++++++++++++++++++++++++++++ > arch/arm64/kernel/ipi_debug.h | 13 ++++++ > 3 files changed, 90 insertions(+), 1 deletion(-) > create mode 100644 arch/arm64/kernel/ipi_debug.c > create mode 100644 arch/arm64/kernel/ipi_debug.h > > diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile > index cc22011ab66a..737838f803b7 100644 > --- a/arch/arm64/kernel/Makefile > +++ b/arch/arm64/kernel/Makefile > @@ -34,7 +34,7 @@ obj-y := debug-monitors.o entry.o irq.o fpsimd.o \ > cpufeature.o alternative.o cacheinfo.o \ > smp.o smp_spin_table.o topology.o smccc-call.o \ > syscall.o proton-pack.o idreg-override.o idle.o \ > - patching.o > + patching.o ipi_debug.o > > obj-$(CONFIG_COMPAT) += sys32.o signal32.o \ > sys_compat.o > diff --git a/arch/arm64/kernel/ipi_debug.c b/arch/arm64/kernel/ipi_debug.c > new file mode 100644 > index 000000000000..b57833e31eaf > --- /dev/null > +++ b/arch/arm64/kernel/ipi_debug.c > @@ -0,0 +1,76 @@ > +// SPDX-License-Identifier: GPL-2.0-only > +/* > + * Debug IPI support > + * > + * Copyright (C) 2020 Linaro Limited > + * Author: Sumit Garg > + */ > + > +#include > +#include > +#include > + > +#include "ipi_debug.h" > + > +static struct irq_desc *ipi_debug_desc __read_mostly; > +static int ipi_debug_id __read_mostly; > +static bool is_nmi; > + > +void arm64_debug_ipi(cpumask_t *mask) > +{ > + if (WARN_ON_ONCE(!ipi_debug_desc)) > + return; > + > + __ipi_send_mask(ipi_debug_desc, mask); > +} > + > +static irqreturn_t ipi_debug_handler(int irq, void *data) > +{ > + /* nop, NMI handlers for special features can be added here. */ > + > + return IRQ_NONE; > +} > + > +void debug_ipi_setup(void) > +{ > + if (!ipi_debug_desc) > + return; > + > + if (is_nmi) { > + if (!prepare_percpu_nmi(ipi_debug_id)) > + enable_percpu_nmi(ipi_debug_id, IRQ_TYPE_NONE); > + } else { > + enable_percpu_irq(ipi_debug_id, IRQ_TYPE_NONE); > + } > +} > + > +void debug_ipi_teardown(void) > +{ > + if (!ipi_debug_desc) > + return; > + > + if (is_nmi) { > + disable_percpu_nmi(ipi_debug_id); > + teardown_percpu_nmi(ipi_debug_id); > + } else { > + disable_percpu_irq(ipi_debug_id); > + } > +} > + > +void __init set_smp_debug_ipi(int ipi) > +{ > + int err; > + > + if (!request_percpu_nmi(ipi, ipi_debug_handler, "IPI", &cpu_number)) { > + is_nmi = true; > + } else { > + err = request_percpu_irq(ipi, ipi_debug_handler, "IPI", &cpu_number); > + if (WARN_ON(err)) > + return; > + > + irq_set_status_flags(ipi, IRQ_HIDDEN); > + } > + > + ipi_debug_desc = irq_to_desc(ipi); > + ipi_debug_id = ipi; > +} > diff --git a/arch/arm64/kernel/ipi_debug.h b/arch/arm64/kernel/ipi_debug.h > new file mode 100644 > index 000000000000..f6011a09282f > --- /dev/null > +++ b/arch/arm64/kernel/ipi_debug.h > @@ -0,0 +1,13 @@ > +/* SPDX-License-Identifier: GPL-2.0 */ > +#ifndef __ASM_NMI_H > +#define __ASM_NMI_H > + > +#include > + > +void arm64_debug_ipi(cpumask_t *mask); > + > +void set_smp_debug_ipi(int ipi); > +void debug_ipi_setup(void); > +void debug_ipi_teardown(void); > + > +#endif > -- > 2.41.0.rc2.161.g9c6817b8e7-goog > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel