From mboxrd@z Thu Jan 1 00:00:00 1970 Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.remote-ip=209.85.215.200 ARC-Seal: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; cv=none; t=1794881230; b=JAL9cdp9x33Gc9m23qYutoJ4PkjFEXnZLz8PNKLN+LuTn9y40+SUGVf1QotO8QQRmfxP WJ53SXHqb58RxM7ZZpF5nyaJraxxQGt67sql1akCs86y5SoW0rMZ7ELoeO9qv632uaFfi yuAKyBVpTJodmJ5PbwHRblxngkeOPY+KuDnb5YQZFRNLeciqR5iagdc5ySikAzoonHtar 1Pu6FYe+jIN9v6M/onFhehP2+vyaJWGyhbonITmNzXPfaGGEPcgy/4Ify1U3bme8gsUu+ ni4qhwfuFmd+xUm/rITAWnFyXJTdsBn2Gu+vGs2J2ltnmXFA7d9R4q3E/pxboi5aF+A== ARC-Message-Signature: i=1; d=subspace.kernel.org; s=arc20261015; a=rsa-sha256; c=relaxed/relaxed; t=1794881230; h=DMARC-Filter:Received:DKIM-Signature:X-Google-DKIM-Signature: X-Forwarded-Encrypted:X-Gm-Message-State:X-Received:Date:In-Reply-To: Mime-Version:References:X-Mailer:Message-ID:Subject:From:To:Cc: Content-Type; bh=vA+hkqkTbHX9a8DOepJDZfPRSPWPvv4RpwYIaThVROM=; b=cbgXf/DTBWwSEr94JbsEaVCxo7MbkLHTxN/lGEor07DTIDsAjaGZwR1pn+FPdB+LAGqH TumcwGKmdyLCpbo82ERP+9bAUgBjibzm5H8XUp3wbvnvUCWv0fjN6wmyjgFB4C3OoOp0E GyghTn4yuEqvXmH3RdQYt7XGlX1ipul3u/0VsgefFRXLUcMqoBEn5/oW88VMCCZoXUq6K HRz3GTijOq4HZsa5CRBOTiampAaHRLMgwkDgr+/ZaDO2JtvyRJb7IAWQ2MPlJndpd5caN eaB1g0hGLOt2d2tlUTwJxy1lNJmjk2SyWWXK8mqPzLRLo7AQ3tOJ8pmt5/rbvbewEDQ== ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass header.from=google.com; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com; dkim=pass header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20260707 header.b=brA4y9Yi; arc=none smtp.remote-ip=209.85.215.200 Received: from mail-pg1-f200.google.com (mail-pg1-f200.google.com [209.85.215.200]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature ECDSA (prime256v1) server-digest SHA256) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 76C8E606A88F for ; Tue, 17 Nov 2026 02:07:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.2 smtp.subspace.kernel.org 76C8E606A88F Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--dylanbhatch.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.a=rsa-sha256 header.s=20260707 header.b=brA4y9Yi Received: by mail-pg1-f200.google.com with SMTP id 41be03b00d2f7-cddc4bde4adso1879348a12.2 for ; Mon, 16 Nov 2026 18:07:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20260707; t=1794881230; x=1795486030; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=vA+hkqkTbHX9a8DOepJDZfPRSPWPvv4RpwYIaThVROM=; b=brA4y9YiVZeZWAxSwQVEU8SVfn4A7zza8CU/05H6YVzEOzwC/V3jDv6XoG9UEy2mjh yHcXADr8oxBWxXE6nuHQ/hmeGgy3xofusAJOyhRiB/Z9xaeMLxUNoWLK8Y9+NXlZvsi8 5o9oQi5T1U90TwX0RsqWv2KnVvzXJeKgnMZT6cLjFyTBPwBcqVyquvaK/HzWv/ffjr7w 0jPW4ufAdBlmY1Z7XigIcbWgMIhGcy9b/FK+R5/EFqECeQ54d5BsYz4kulp0w+nahOzj HS6riKVUSg7hQDTYMg3hXK0rebyylq+ubrXm5rBQFVLqA2gHT1AAcJEla3Dzp818nmlr d2Mg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1794881230; x=1795486030; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=vA+hkqkTbHX9a8DOepJDZfPRSPWPvv4RpwYIaThVROM=; b=PlxyEa6D1CUCHPMwPkueNlichmVk+AcGT2SQ6E9UOsu64V8jF5ro9LBVz3LGwcvqCR RUyq3fa2cMFkskpzcJtPmkFt5doG1NRRaVts6+DG3Tdv7jG9K9gGj6OWe+GyIW9OVpck 5TDdorrA0/zXb/KOIpgcUdbumflwu4nm57CmbzQ4A9T/9ZmBBl9vQfnYFMmu7m+OgUE0 ORFn6sm+oVD0oOS6colz5+yd+IxrG1ia8nbZwnbFJ1/mJtvhVhrF/qMMR4Iy0zjMnpIx +BZMujMKSHD6CXl6JOQCFMOdvfx9R/EExO24AvkFiSguu/bgDB+bK2vnP9V/B81jZ6ju VEsQ== X-Forwarded-Encrypted: i=1; AL+hErHvkkcW+5J5sPPqc/qEECroxrMGDXeHdghaW3UFY8RFxLhVXdsqdFLjZKH9cMC09Uf+Ip7yZxep+bJnAB3Jc3Nt@vger.kernel.org X-Gm-Message-State: APuU4Z+fykENszTNNU+VNox0JtGOi+8mayo8bT3bKi2xKXaF2NLT2soX DFToSbxQv+ALP9d7VYMse5mHHgMaqoipdM/WIZpENPK52uRHOZblD9aoXC1k8kZOBmtGINq45su J88CtT6SN751kRiHA5oiJc046vQ== X-Received: from pgjk8-n1.prod.google.com ([2002:a05:6a02:81c8:10b0:cdb:932c:948b]) (user=dylanbhatch job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a21:7e05:b0:3e2:b85c:4b2a with SMTP id adf61e73a8af0-3f7a8fe1586mr9088034637.73.1794881229362; Mon, 16 Nov 2026 18:07:09 -0800 (PST) Date: Tue, 17 Nov 2026 02:06:48 +0000 In-Reply-To: <20261117020648.323464-1-dylanbhatch@google.com> X-Mailing-List: linux-toolchains@vger.kernel.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list Mime-Version: 1.0 References: <20261117020648.323464-1-dylanbhatch@google.com> X-Mailer: git-send-email 2.56.0.679.gc527c65306-goog Message-ID: <20261117020648.323464-12-dylanbhatch@google.com> Subject: [PATCH v9 11/11] unwind: arm64: Use sframe to unwind interrupt frames From: Dylan Hatch To: Roman Gushchin , Weinan Liu , Will Deacon , Josh Poimboeuf , Indu Bhagat , Peter Zijlstra , Steven Rostedt , Catalin Marinas , Jiri Kosina , Mark Rutland , Jens Remus Cc: Dylan Hatch , Prasanna Kumar T S M , Puranjay Mohan , Song Liu , joe.lawrence@redhat.com, linux-toolchains@vger.kernel.org, linux-kernel@vger.kernel.org, live-patching@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Randy Dunlap , Mostafa Saleh , Herbert Xu , "David S . Miller" Content-Type: text/plain; charset="UTF-8" Add kunwind_next_regs_sframe() function to unwind by sframe info if present. Use this method at exception boundaries, falling back to frame-pointer unwind only on failure. In such failure cases, the stacktrace is considered unreliable. During normal unwind, prefer frame pointer unwind (for better performance). This change restores the LR behavior originally introduced in commit c2c6b27b5aa14fa2 ("arm64: stacktrace: unwind exception boundaries"), But later removed in commit 32ed1205682e ("arm64: stacktrace: Skip reporting LR at exception boundaries") This can be done because the sframe data can be used to determine whether the LR is current for the PC value recovered from pt_regs at the exception boundary. Co-developed-by: Weinan Liu Signed-off-by: Weinan Liu Reviewed-by: Prasanna Kumar T S M Reviewed-by: Jens Remus Signed-off-by: Dylan Hatch --- Changes since v8: - (sashiko) Restrict SFrame unwinding at KUNWIND_SOURCE_REGS_PC to real on-stack FRAME_META_TYPE_PT_REGS exception frames so synthetic pt_regs fall back to frame-pointer unwinding. - (sashiko) Remove invalid cross-stack 'fp < sp' check in UNWIND_CFA_RULE_FP_OFFSET and enforce 8-byte alignment in get_word(). - (sashiko) Propagate state->unreliable when kunwind_next() terminates with -ENOENT in do_kunwind(). --- arch/arm64/kernel/stacktrace.c | 211 +++++++++++++++++++++++++++++++-- 1 file changed, 198 insertions(+), 13 deletions(-) diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c index 3ebcf8c53fb04..e876af54a4055 100644 --- a/arch/arm64/kernel/stacktrace.c +++ b/arch/arm64/kernel/stacktrace.c @@ -13,6 +13,7 @@ #include #include #include +#include #include #include @@ -26,6 +27,7 @@ enum kunwind_source { KUNWIND_SOURCE_CALLER, KUNWIND_SOURCE_TASK, KUNWIND_SOURCE_REGS_PC, + KUNWIND_SOURCE_REGS_LR, }; union unwind_flags { @@ -45,6 +47,7 @@ union unwind_flags { * @kr_cur: When KRETPROBES is selected, holds the kretprobe instance * associated with the most recently encountered replacement lr * value. + * @unreliable: Stacktrace is unreliable. */ struct kunwind_state { struct unwind_state common; @@ -56,6 +59,7 @@ struct kunwind_state { enum kunwind_source source; union unwind_flags flags; struct pt_regs *regs; + bool unreliable; }; static __always_inline void @@ -80,12 +84,31 @@ static __always_inline void kunwind_init_from_regs(struct kunwind_state *state, struct pt_regs *regs) { + struct stack_info *info; + kunwind_init(state, current); - state->regs = regs; state->common.fp = regs->regs[29]; state->common.pc = regs->pc; state->source = KUNWIND_SOURCE_REGS_PC; + + /* + * Synthetic pt_regs (e.g. from perf_fetch_caller_regs() or + * ftrace_fill_perf_regs()) do not have a full register state + * consistent with regs->pc and cannot be unwound with SFrame. + * Real exception pt_regs have FRAME_META_TYPE_PT_REGS. + */ + if (!regs->stackframe.record.fp && !regs->stackframe.record.lr && + regs->stackframe.type == FRAME_META_TYPE_PT_REGS) { + info = unwind_find_stack(&state->common, (unsigned long)regs, + sizeof(*regs)); + if (info) { + unwind_consume_stack(&state->common, info, + (unsigned long)regs, + sizeof(*regs)); + state->regs = regs; + } + } } /* @@ -181,7 +204,6 @@ int kunwind_next_regs_pc(struct kunwind_state *state) state->regs = regs; state->common.pc = regs->pc; state->common.fp = regs->regs[29]; - state->regs = NULL; state->source = KUNWIND_SOURCE_REGS_PC; return 0; } @@ -244,6 +266,163 @@ kunwind_next_frame_record(struct kunwind_state *state) return 0; } +#ifdef CONFIG_HAVE_UNWIND_KERNEL_SFRAME + +static __always_inline struct stack_info * +get_word(struct unwind_state *state, unsigned long *word) +{ + unsigned long addr = *word; + struct stack_info *info; + + if (addr & 0x7) + return NULL; + + info = unwind_find_stack(state, addr, sizeof(addr)); + if (!info) + return info; + + *word = READ_ONCE(*(unsigned long *)addr); + + return info; +} + +static __always_inline int +get_consume_word(struct unwind_state *state, unsigned long *word) +{ + struct stack_info *info; + unsigned long addr = *word; + + info = get_word(state, word); + if (!info) + return -EINVAL; + + unwind_consume_stack(state, info, addr, sizeof(addr)); + return 0; +} + +/* + * Unwind from a pt_regs according to sframe. + */ +static __always_inline int +kunwind_next_regs_sframe(struct kunwind_state *state) +{ + struct unwind_frame frame; + unsigned long cfa, fp, ra; + enum kunwind_source source = KUNWIND_SOURCE_FRAME; + struct pt_regs *regs = state->regs; + + int err; + + if (WARN_ON_ONCE(state->source != KUNWIND_SOURCE_REGS_PC)) + return -EINVAL; + if (WARN_ON_ONCE(!state->regs)) + return -EINVAL; + + /* FP/SP alignment 8 bytes */ + if (state->common.fp & 0x7) + return -EINVAL; + + err = sframe_find(state->common.pc, &frame); + if (err) + return -EINVAL; + + /* + * A kernel unwind should always end at a FRAME_META_TYPE_FINAL + * frame. There should be no outermost frames within the kernel. + */ + if (frame.outermost) + return -EINVAL; + + /* Get the Canonical Frame Address (CFA) */ + switch (frame.cfa.rule) { + case UNWIND_CFA_RULE_SP_OFFSET: + cfa = state->regs->sp; + break; + case UNWIND_CFA_RULE_FP_OFFSET: + cfa = state->common.fp; + break; + /* + * UNWIND_CFA_RULE_REG_OFFSET and UNWIND_CFA_RULE_REG_OFFSET_DEREF not + * implemented -- flexible FDEs are not currently generated by assembler + * for arm64. + */ + default: + WARN_ON_ONCE(1); + return -EINVAL; + } + cfa += frame.cfa.offset; + + /* CFA alignment 16 bytes */ + if (cfa & 0xf) + return -EINVAL; + + /* + * Consume RA and FP from the stack. The frame record puts FP at a lower + * address than RA, so we read FP first. + */ + switch (frame.fp.rule) { + case UNWIND_RULE_RETAIN: + fp = state->common.fp; + break; + /* + * UNWIND_RULE_CFA_OFFSET is currently not used for FP + * (e.g. SFrame cannot represent this rule). + */ + case UNWIND_RULE_CFA_OFFSET_DEREF: + fp = cfa + frame.fp.offset; + if (!get_word(&state->common, &fp)) + return -EINVAL; + break; + /* + * UNWIND_RULE_REG_OFFSET and UNWIND_RULE_REG_OFFSET_DEREF not + * implemented -- flexible FDEs are not currently generated by assembler + * for arm64. + */ + default: + WARN_ON_ONCE(1); + return -EINVAL; + } + + switch (frame.ra.rule) { + case UNWIND_RULE_RETAIN: + ra = regs->regs[30]; + source = KUNWIND_SOURCE_REGS_LR; + break; + + /* + * UNWIND_RULE_CFA_OFFSET doesn't make sense for RA. + * The return address cannot legitimately be a stack address. + */ + case UNWIND_RULE_CFA_OFFSET_DEREF: + ra = cfa + frame.ra.offset; + if (get_consume_word(&state->common, &ra)) + return -EINVAL; + break; + /* + * UNWIND_RULE_REG_OFFSET and UNWIND_RULE_REG_OFFSET_DEREF not + * implemented -- flexible FDEs are not currently generated by assembler + * for arm64. + */ + default: + WARN_ON_ONCE(1); + return -EINVAL; + } + + state->common.pc = ra; + state->common.fp = fp; + + state->source = source; + + return 0; +} + +#else /* !CONFIG_HAVE_UNWIND_KERNEL_SFRAME */ + +static __always_inline int +kunwind_next_regs_sframe(struct kunwind_state *state) { return -EINVAL; } + +#endif /* !CONFIG_HAVE_UNWIND_KERNEL_SFRAME*/ + /* * Unwind from one frame record (A) to the next frame record (B). * @@ -259,10 +438,23 @@ kunwind_next(struct kunwind_state *state) state->flags.all = 0; switch (state->source) { + case KUNWIND_SOURCE_REGS_PC: + if (state->regs) + err = kunwind_next_regs_sframe(state); + else + err = -EINVAL; + + if (err) { + /* Fallback to FP based unwinder */ + err = kunwind_next_frame_record(state); + state->unreliable = true; + } + state->regs = NULL; + break; case KUNWIND_SOURCE_FRAME: case KUNWIND_SOURCE_CALLER: case KUNWIND_SOURCE_TASK: - case KUNWIND_SOURCE_REGS_PC: + case KUNWIND_SOURCE_REGS_LR: err = kunwind_next_frame_record(state); break; default: @@ -294,7 +486,7 @@ do_kunwind(struct kunwind_state *state, kunwind_consume_fn consume_state, return -EINVAL; ret = kunwind_next(state); if (ret == -ENOENT) - return 0; + return state->unreliable ? -EINVAL : 0; if (ret < 0) return ret; } @@ -393,15 +585,7 @@ noinline noinstr void arch_stack_walk(stack_trace_consume_fn consume_entry, static __always_inline bool arch_reliable_kunwind_consume_entry(const struct kunwind_state *state, void *cookie) { - /* - * At an exception boundary we can reliably consume the saved PC. We do - * not know whether the LR was live when the exception was taken, and - * so we cannot perform the next unwind step reliably. - * - * All that matters is whether the *entire* unwind is reliable, so give - * up as soon as we hit an exception boundary. - */ - if (state->source == KUNWIND_SOURCE_REGS_PC) + if (state->unreliable) return false; return arch_kunwind_consume_entry(state, cookie); @@ -452,6 +636,7 @@ static const char *state_source_string(const struct kunwind_state *state) case KUNWIND_SOURCE_CALLER: return "C"; case KUNWIND_SOURCE_TASK: return "T"; case KUNWIND_SOURCE_REGS_PC: return "P"; + case KUNWIND_SOURCE_REGS_LR: return "L"; default: return "U"; } } -- 2.56.0.679.gc527c65306-goog