From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C0DB7C3ABC3 for ; Mon, 12 May 2025 18:08:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date :Subject:To:From:Reply-To:Content-Type:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=8Dj0By3OWLsdS2Je45l1/6dWMPHId5/YJCpBWSug84s=; b=UmIN6oA832XrL2 VYEXToqTh9fjNvkPs5Zdoia4RwBiD1tL0PcCa//XJUYt0HhO+f9gOCyfv14uKHmQep80lb+1kL61r 0fqhOCRs7SzcOiDNgk0dMZh5jsAmYrSHqF+eG9NTwWOkzqDfpXCXbawZOlHs18DfiqocAoGnrOR2O nlm8Ajj1QAVt2nXsJeYQ7IrKMUEXfxe6egH18+/0hc56nPDezkUAYwIor+v4Va7Ewss93amqQgKO8 k4XqRu0RykEc49t2NQuIMw+V8i0jvoKAH4PMkxfZRiOUAZANtZxlQe1i4ECm2DT+AlGdyM5YRPuBm HYX4T/SwS+61sERn8kCg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.2 #2 (Red Hat Linux)) id 1uEXZu-0000000AIBG-3ecI; Mon, 12 May 2025 18:08:38 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.98.2 #2 (Red Hat Linux)) id 1uEXC2-0000000AFBn-3qi1 for linux-arm-kernel@lists.infradead.org; Mon, 12 May 2025 17:44:00 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9820B202C; Mon, 12 May 2025 10:43:47 -0700 (PDT) Received: from e137867.cambridge.arm.com (e137867.arm.com [10.1.32.174]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 063493F673; Mon, 12 May 2025 10:43:56 -0700 (PDT) From: Ada Couprie Diaz To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v2 06/11] arm64: debug: split hardware breakpoint exeception entry Date: Mon, 12 May 2025 18:43:21 +0100 Message-ID: <20250512174326.133905-7-ada.coupriediaz@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250512174326.133905-1-ada.coupriediaz@arm.com> References: <20250512174326.133905-1-ada.coupriediaz@arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250512_104359_037923_6A8F66E5 X-CRM114-Status: GOOD ( 16.78 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , "Luis Claudio R. Goncalves" , Catalin Marinas , Sebastian Andrzej Siewior , Will Deacon Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently all debug exceptions share common entry code and are routed to `do_debug_exception()`, which calls dynamically-registered handlers for each specific debug exception. This is unfortunate as different debug exceptions have different entry handling requirements, and it would be better to handle these distinct requirements earlier. Hardware breakpoints exceptions are generated by the hardware after user configuration. As such, they can be exploited when training branch predictors outisde of the userspace VA range: they still need to call `arm64_apply_bp_hardening()` if needed to mitigate against this attack. Move the call to `arm64_apply_bp_hardening()` to `entry-common.c` as it is needed for exceptions coming from EL0 only. However, they do not need to handle the Cortex-A76 erratum #1463225 as it only applies to single stepping exceptions. It does not set an address in FAR_EL1 either, only the hardware watchpoint does. Split the hardware breakpoint exception entry, adjust the function signature, and handling of the Cortex-A76 erratum to fit the behaviour of the exception. Signed-off-by: Ada Couprie Diaz --- arch/arm64/include/asm/exception.h | 1 + arch/arm64/kernel/entry-common.c | 28 ++++++++++++++++++++++++++++ arch/arm64/kernel/hw_breakpoint.c | 15 +++++++++++---- 3 files changed, 40 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/exception.h b/arch/arm64/include/asm/exception.h index d48fc16584cd..c593fe639697 100644 --- a/arch/arm64/include/asm/exception.h +++ b/arch/arm64/include/asm/exception.h @@ -61,6 +61,7 @@ void do_el0_gcs(struct pt_regs *regs, unsigned long esr); void do_el1_gcs(struct pt_regs *regs, unsigned long esr); void do_debug_exception(unsigned long addr_if_watchpoint, unsigned long esr, struct pt_regs *regs); +void do_breakpoint(unsigned long esr, struct pt_regs *regs); void do_fpsimd_acc(unsigned long esr, struct pt_regs *regs); void do_sve_acc(unsigned long esr, struct pt_regs *regs); void do_sme_acc(unsigned long esr, struct pt_regs *regs); diff --git a/arch/arm64/kernel/entry-common.c b/arch/arm64/kernel/entry-common.c index 92d78b329e67..6ff52fc94da7 100644 --- a/arch/arm64/kernel/entry-common.c +++ b/arch/arm64/kernel/entry-common.c @@ -503,6 +503,15 @@ static void noinstr el1_mops(struct pt_regs *regs, unsigned long esr) exit_to_kernel_mode(regs); } +static void noinstr el1_breakpt(struct pt_regs *regs, unsigned long esr) +{ + arm64_enter_el1_dbg(regs); + debug_exception_enter(regs); + do_breakpoint(esr, regs); + debug_exception_exit(regs); + arm64_exit_el1_dbg(regs); +} + static void noinstr el1_dbg(struct pt_regs *regs, unsigned long esr) { unsigned long far = read_sysreg(far_el1); @@ -552,6 +561,8 @@ asmlinkage void noinstr el1h_64_sync_handler(struct pt_regs *regs) el1_mops(regs, esr); break; case ESR_ELx_EC_BREAKPT_CUR: + el1_breakpt(regs, esr); + break; case ESR_ELx_EC_SOFTSTP_CUR: case ESR_ELx_EC_WATCHPT_CUR: case ESR_ELx_EC_BRK64: @@ -746,6 +757,19 @@ static void noinstr el0_inv(struct pt_regs *regs, unsigned long esr) exit_to_user_mode(regs); } +static void noinstr el0_breakpt(struct pt_regs *regs, unsigned long esr) +{ + if (!is_ttbr0_addr(regs->pc)) + arm64_apply_bp_hardening(); + + enter_from_user_mode(regs); + debug_exception_enter(regs); + do_breakpoint(esr, regs); + debug_exception_exit(regs); + local_daif_restore(DAIF_PROCCTX); + exit_to_user_mode(regs); +} + static void noinstr el0_dbg(struct pt_regs *regs, unsigned long esr) { /* Only watchpoints write FAR_EL1, otherwise its UNKNOWN */ @@ -824,6 +848,8 @@ asmlinkage void noinstr el0t_64_sync_handler(struct pt_regs *regs) el0_gcs(regs, esr); break; case ESR_ELx_EC_BREAKPT_LOW: + el0_breakpt(regs, esr); + break; case ESR_ELx_EC_SOFTSTP_LOW: case ESR_ELx_EC_WATCHPT_LOW: case ESR_ELx_EC_BRK64: @@ -944,6 +970,8 @@ asmlinkage void noinstr el0t_32_sync_handler(struct pt_regs *regs) el0_cp15(regs, esr); break; case ESR_ELx_EC_BREAKPT_LOW: + el0_breakpt(regs, esr); + break; case ESR_ELx_EC_SOFTSTP_LOW: case ESR_ELx_EC_WATCHPT_LOW: case ESR_ELx_EC_BKPT32: diff --git a/arch/arm64/kernel/hw_breakpoint.c b/arch/arm64/kernel/hw_breakpoint.c index 722ac45f9f7b..b7253ddac230 100644 --- a/arch/arm64/kernel/hw_breakpoint.c +++ b/arch/arm64/kernel/hw_breakpoint.c @@ -22,6 +22,7 @@ #include #include #include +#include #include #include #include @@ -618,8 +619,7 @@ NOKPROBE_SYMBOL(toggle_bp_registers); /* * Debug exception handlers. */ -static int breakpoint_handler(unsigned long unused, unsigned long esr, - struct pt_regs *regs) +static int breakpoint_handler(unsigned long esr, struct pt_regs *regs) { int i, step = 0, *kernel_step; u32 ctrl_reg; @@ -695,6 +695,15 @@ static int breakpoint_handler(unsigned long unused, unsigned long esr, } NOKPROBE_SYMBOL(breakpoint_handler); +void do_breakpoint(unsigned long esr, struct pt_regs *regs) +{ + if (breakpoint_handler(esr, regs)) { + arm64_notify_die("hw-breakpoint handler", regs, SIGTRAP, TRAP_HWBKPT, + regs->pc, esr); + } +} +NOKPROBE_SYMBOL(do_breakpoint); + /* * Arm64 hardware does not always report a watchpoint hit address that matches * one of the watchpoints set. It can also report an address "near" the @@ -988,8 +997,6 @@ static int __init arch_hw_breakpoint_init(void) core_num_brps, core_num_wrps); /* Register debug fault handlers. */ - hook_debug_fault_code(DBG_ESR_EVT_HWBP, breakpoint_handler, SIGTRAP, - TRAP_HWBKPT, "hw-breakpoint handler"); hook_debug_fault_code(DBG_ESR_EVT_HWWP, watchpoint_handler, SIGTRAP, TRAP_HWBKPT, "hw-watchpoint handler"); -- 2.43.0