From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD1A5C433EF for ; Thu, 14 Oct 2021 09:59:29 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6EBA66109E for ; Thu, 14 Oct 2021 09:59:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 6EBA66109E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Mime-Version:References:In-Reply-To: Message-Id:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=vX+EmfcAUkGj6hFo17VtXhlrFZZP3hOkBlOb47SgzGQ=; b=wESCTesXqAJ18Y +EVvQ8vSr28/33mpRCPvk6jQvZbVp0Ms5Zd/qTz/52jq91CP3jSZVEer3do3E9qLwS/O4TqamUzgA xU06+yIcTCVEapVIRKZiwvboBSJatb/vzs1WVSdNCqnWNxC5mJkZTMUPu2W4SF2Hx4Jrf5NSVjtDO LOt85TfJpIU+0GenIGUktx/NYNKXs0f4SkYiiK6VFWAwur+31z0YnNm0PHkB6nGGdMCQSdgrwqvrT we/Hwaud8Iy/MdWQ6GzeKuvdE4Zo9LZhKzJuuIxpWGoJz9Uwmlg0Pnq0+qB1l3OxhQCZ71hZ/XR6K 3Kz+wNc4qIjLzt9buE2Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1maxUa-002TUp-RO; Thu, 14 Oct 2021 09:57:41 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1maxUK-002TPx-Gh for linux-arm-kernel@lists.infradead.org; Thu, 14 Oct 2021 09:57:26 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 4BFD66109E; Thu, 14 Oct 2021 09:57:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1634205444; bh=/IuaOK0r3stsDLvCgSeDG+yYRnpvsEN7DotBaEMiSwE=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=RAbNLk9PSRdvFfkJotl9eFkUHq+eysJdC4bPUFZk8kzi9Got4Jc4kbly266xzcEbL GMT3VwxJzGKSO9awduK8WpZjDMboF9hvRjxjjLFwzW89KDcL7MjPLPSDm/1ZoAHHQY /F+7vTUn59P99ssyn8JWar/H+1E38A3IMJ7V1t2g31v79Ru3DdF2fcrlsY7lUnYEuZ 7HKJKJ3ok+MbPe2Ce0Ya52YuM9N7UkJN0b1Mdf3WQhoJLY9frRVZrQ5P5zoyQjZip1 S3nG3K/pglPTo9b1eZUjMC4LC2jfXaaM9cI/7Q1e9MBx7M0DIryQ7oWgLlrAGZoyos KfOzxNWBVc+Zg== Date: Thu, 14 Oct 2021 18:57:20 +0900 From: Masami Hiramatsu To: Mark Rutland Cc: Steven Rostedt , "Naveen N . Rao" , Ananth N Mavinakayanahalli , Ingo Molnar , linux-kernel@vger.kernel.org, Sven Schnelle , Catalin Marinas , Will Deacon , Russell King , Nathan Chancellor , Nick Desaulniers , linux-arm-kernel@lists.infradead.org Subject: Re: [PATCH 5/8] arm64: Recover kretprobe modified return address in stacktrace Message-Id: <20211014185720.67262334328b383e082d9db4@kernel.org> In-Reply-To: <20211013101351.GB3187@C02TD0UTHF1T.local> References: <163369609308.636038.15295764725220907794.stgit@devnote2> <163369613866.636038.15240679956943005288.stgit@devnote2> <20211013101351.GB3187@C02TD0UTHF1T.local> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.32; x86_64-pc-linux-gnu) Mime-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211014_025724_630889_40118FC1 X-CRM114-Status: GOOD ( 37.32 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, 13 Oct 2021 11:13:51 +0100 Mark Rutland wrote: > On Fri, Oct 08, 2021 at 09:28:58PM +0900, Masami Hiramatsu wrote: > > Since the kretprobe replaces the function return address with > > the kretprobe_trampoline on the stack, stack unwinder shows it > > instead of the correct return address. > > > > This checks whether the next return address is the > > __kretprobe_trampoline(), and if so, try to find the correct > > return address from the kretprobe instance list. > > > > With this fix, now arm64 can enable > > CONFIG_ARCH_CORRECT_STACKTRACE_ON_KRETPROBE, and pass the > > kprobe self tests. > > > > Signed-off-by: Masami Hiramatsu > > --- > > arch/arm64/Kconfig | 1 + > > arch/arm64/include/asm/stacktrace.h | 2 ++ > > arch/arm64/kernel/stacktrace.c | 3 +++ > > 3 files changed, 6 insertions(+) > > > > diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig > > index 5c7ae4c3954b..edde5171ffb2 100644 > > --- a/arch/arm64/Kconfig > > +++ b/arch/arm64/Kconfig > > @@ -11,6 +11,7 @@ config ARM64 > > select ACPI_PPTT if ACPI > > select ARCH_HAS_DEBUG_WX > > select ARCH_BINFMT_ELF_STATE > > + select ARCH_CORRECT_STACKTRACE_ON_KRETPROBE > > select ARCH_ENABLE_HUGEPAGE_MIGRATION if HUGETLB_PAGE && MIGRATION > > select ARCH_ENABLE_MEMORY_HOTPLUG > > select ARCH_ENABLE_MEMORY_HOTREMOVE > > diff --git a/arch/arm64/include/asm/stacktrace.h b/arch/arm64/include/asm/stacktrace.h > > index 8aebc00c1718..8f997a602651 100644 > > --- a/arch/arm64/include/asm/stacktrace.h > > +++ b/arch/arm64/include/asm/stacktrace.h > > @@ -9,6 +9,7 @@ > > #include > > #include > > #include > > +#include > > > > #include > > #include > > @@ -59,6 +60,7 @@ struct stackframe { > > #ifdef CONFIG_FUNCTION_GRAPH_TRACER > > int graph; > > #endif > > + struct llist_node *kr_cur; > > As with the fgraph bits above, please make this depedn on the relevant > Kconfig, i.e. > > | #ifdef CONFIG_KRETPROBES > | struct llist_node *kr_cur; > | #endif > > > }; OK. > > > > extern int unwind_frame(struct task_struct *tsk, struct stackframe *frame); > > diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c > > index 8982a2b78acf..f1eef5745542 100644 > > --- a/arch/arm64/kernel/stacktrace.c > > +++ b/arch/arm64/kernel/stacktrace.c > > @@ -129,6 +129,8 @@ int notrace unwind_frame(struct task_struct *tsk, struct stackframe *frame) > > frame->pc = ret_stack->ret; > > } > > #endif /* CONFIG_FUNCTION_GRAPH_TRACER */ > > + if (is_kretprobe_trampoline(frame->pc)) > > + frame->pc = kretprobe_find_ret_addr(tsk, (void *)frame->fp, &frame->kr_cur); > > Please ifdef this, like the CONFIG_FUNCTION_GRAPH_TRACER bits above. > i.e. > > | #ifdef CONFIG_KRETPROBES > | if (is_kretprobe_trampoline(frame->pc)) > | frame->pc = kretprobe_find_ret_addr(tsk, (void *)frame->fp, &frame->kr_cur); > | #endif OK. > > What does kretprobe_find_ret_addr() do when it can't find the original > address? I couldn't spot it in this series or in mainline. If it couldn't find, it returns NULL. Hmm, should we check it and keep frame->pc if the return value is NULL? (anyway, it must not happen. If it happens, that task can not continue to run.) > As a future thing, I'd like to be able to have monotonicity and > completeness checks as part of the unwind, i.e. checking that we consume > the kretprobe address *in-order*, and can identify whether we've skipped > any, so that we can identify when unwinding has gone wrong. Does it do > that today? Good question. No today, but is easy to do since we have the loop cursor (frame->kr_cur). unsigned long kretprobe_find_ret_addr(struct task_struct *tsk, void *fp, struct llist_node **cur) { ... do { ret = __kretprobe_find_ret_addr(tsk, cur); if (!ret) break; ri = container_of(*cur, struct kretprobe_instance, llist); } while (ri->fp != fp); ... Change this loop to; ri = container_of(*cur, struct kretprobe_instance, llist); prev_fp = ri->fp; do { ret = __kretprobe_find_ret_addr(tsk, cur); if (WARN_ON_ONCE(!ret)) return ERR_PTR(-ENOENT); ri = container_of(*cur, struct kretprobe_instance, llist); } while (ri->fp == prev_fp); if (ri->fp != fp) return ERR_PTR(-EILSEQ); Then, we can detect that wrong sequence from stacktrace side. > > It'd be nice if it could signal failure reliably (without causing a > BUG() or similar), e.g. by returning an error code. As above, -EILSEQ is OK? > > > frame->pc = ptrauth_strip_insn_pac(frame->pc); > > > > @@ -224,6 +226,7 @@ noinline notrace void arch_stack_walk(stack_trace_consume_fn consume_entry, > > { > > struct stackframe frame; > > > > + memset(&frame, 0, sizeof(frame)); > > Please initialise stackframe::kr_cur in start_backtrace, where we > initialize all the other fields in struct stackframe, i.e. just after > the CONFIG_FUNCTION_GRAPH_TRACER bit, have: > > | #ifdef CONFIG_KRETPROBES > | frame->kr_cur = NULL; > | #endif OK, let me update it. Thank you! > > Thanks, > Mark. > > > if (regs) > > start_backtrace(&frame, regs->regs[29], regs->pc); > > else if (task == current) > > -- Masami Hiramatsu _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel