From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95CD6C433C1 for ; Tue, 23 Mar 2021 13:40:56 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1B1C461931 for ; Tue, 23 Mar 2021 13:40:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1B1C461931 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.microsoft.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From: References:Cc:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=bv/yponTIKtpswqaVxqYLAqzioWe48lh3tzKkwDjj40=; b=mM0EGGmep4MELHC96VzGbAp1X NMugqdm+gaVy/jW8NRj70jM0eoJw38O39Vu/PZmw9toC9r62st6zAY22wK9sxLwmwUDjIvuMtOfOR OOhooTms4M/nqdgH0MD+EqhCYEist7+ENxyjtc4T3Xj7Ke9KTvwTZcfsiUhonEJThFgzJEQ1cCjRE JM5OLxEK8Dw4y+N4te8FwU4xuNk6sDugPyad3IX+QoyekIesoK0lHJCgN30yCqZgqja2pdKxfnrGo IZLgwxk/yjHI8cTa0g1sAGUchxwKOWRFWu+QSI0YPu/NYuEsUx3xa8oNHhI/1evDzroGL4QM/WusJ Jop/YT1hw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lOhFT-00F3oO-2c; Tue, 23 Mar 2021 13:39:07 +0000 Received: from linux.microsoft.com ([13.77.154.182]) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lOhFN-00F3nm-AL for linux-arm-kernel@lists.infradead.org; Tue, 23 Mar 2021 13:39:03 +0000 Received: from [192.168.254.32] (unknown [47.187.194.202]) by linux.microsoft.com (Postfix) with ESMTPSA id A4DD020B5680; Tue, 23 Mar 2021 06:38:59 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com A4DD020B5680 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1616506740; bh=DwRsLoQPY0fFMEhsi2TeLN5GvE6GzKWHyVlv0powA4Y=; h=Subject:To:Cc:References:From:Date:In-Reply-To:From; b=rPua/ORFsVGJ5Wqu8WYJ/bMN52OnjWRzUx3NyvoZWpxTafNMBz5nQdnB+qWKzx9F9 XIMs4+6hvCF9Q+U/qs6sPqZbfKdnyt6CdwpEdfoib3PD98nlVoWgud+3SR6lmAMDj7 l3oKoMYWqJtabVYmoxI79DJge2dhumkOH7d/FSfI= Subject: Re: [RFC PATCH v2 5/8] arm64: Detect an FTRACE frame and mark a stack trace unreliable To: Mark Rutland Cc: broonie@kernel.org, jpoimboe@redhat.com, jthierry@redhat.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, live-patching@vger.kernel.org, linux-kernel@vger.kernel.org References: <5997dfe8d261a3a543667b83c902883c1e4bd270> <20210315165800.5948-1-madvenka@linux.microsoft.com> <20210315165800.5948-6-madvenka@linux.microsoft.com> <20210323105118.GE95840@C02TD0UTHF1T.local> <2167f3c5-e7d0-40c8-99e3-ae89ceb2d60e@linux.microsoft.com> <20210323133611.GB98545@C02TD0UTHF1T.local> From: "Madhavan T. Venkataraman" Message-ID: Date: Tue, 23 Mar 2021 08:38:58 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.7.1 MIME-Version: 1.0 In-Reply-To: <20210323133611.GB98545@C02TD0UTHF1T.local> Content-Language: en-US X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210323_133901_729500_81CEE1D3 X-CRM114-Status: GOOD ( 32.74 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 3/23/21 8:36 AM, Mark Rutland wrote: > On Tue, Mar 23, 2021 at 07:56:40AM -0500, Madhavan T. Venkataraman wrote: >> >> >> On 3/23/21 5:51 AM, Mark Rutland wrote: >>> On Mon, Mar 15, 2021 at 11:57:57AM -0500, madvenka@linux.microsoft.com wrote: >>>> From: "Madhavan T. Venkataraman" >>>> >>>> When CONFIG_DYNAMIC_FTRACE_WITH_REGS is enabled and tracing is activated >>>> for a function, the ftrace infrastructure is called for the function at >>>> the very beginning. Ftrace creates two frames: >>>> >>>> - One for the traced function >>>> >>>> - One for the caller of the traced function >>>> >>>> That gives a reliable stack trace while executing in the ftrace >>>> infrastructure code. When ftrace returns to the traced function, the frames >>>> are popped and everything is back to normal. >>>> >>>> However, in cases like live patch, execution is redirected to a different >>>> function when ftrace returns. A stack trace taken while still in the ftrace >>>> infrastructure code will not show the target function. The target function >>>> is the real function that we want to track. >>>> >>>> So, if an FTRACE frame is detected on the stack, just mark the stack trace >>>> as unreliable. >>> >>> To identify this case, please identify the ftrace trampolines instead, >>> e.g. ftrace_regs_caller, return_to_handler. >>> >> >> Yes. As part of the return address checking, I will check this. IIUC, I think that >> I need to check for the inner labels that are defined at the point where the >> instructions are patched for ftrace. E.g., ftrace_call and ftrace_graph_call. >> >> SYM_INNER_LABEL(ftrace_call, SYM_L_GLOBAL) >> bl ftrace_stub <==================================== >> >> #ifdef CONFIG_FUNCTION_GRAPH_TRACER >> SYM_INNER_LABEL(ftrace_graph_call, SYM_L_GLOBAL) // ftrace_graph_caller(); >> nop <======= // If enabled, this will be replaced >> // "b ftrace_graph_caller" >> #endif >> >> For instance, the stack trace I got while tracing do_mmap() with the stack trace >> tracer looks like this: >> >> ... >> [ 338.911793] trace_function+0xc4/0x160 >> [ 338.911801] function_stack_trace_call+0xac/0x130 >> [ 338.911807] ftrace_graph_call+0x0/0x4 >> [ 338.911813] do_mmap+0x8/0x598 >> [ 338.911820] vm_mmap_pgoff+0xf4/0x188 >> [ 338.911826] ksys_mmap_pgoff+0x1d8/0x220 >> [ 338.911832] __arm64_sys_mmap+0x38/0x50 >> [ 338.911839] el0_svc_common.constprop.0+0x70/0x1a8 >> [ 338.911846] do_el0_svc+0x2c/0x98 >> [ 338.911851] el0_svc+0x2c/0x70 >> [ 338.911859] el0_sync_handler+0xb0/0xb8 >> [ 338.911864] el0_sync+0x180/0x1c0 >> >>> It'd be good to check *exactly* when we need to reject, since IIUC when >>> we have a graph stack entry the unwind will be correct from livepatch's >>> PoV. >>> >> >> The current unwinder already handles this like this: >> >> #ifdef CONFIG_FUNCTION_GRAPH_TRACER >> if (tsk->ret_stack && >> (ptrauth_strip_insn_pac(frame->pc) == (unsigned long)return_to_handler)) { >> struct ftrace_ret_stack *ret_stack; >> /* >> * This is a case where function graph tracer has >> * modified a return address (LR) in a stack frame >> * to hook a function return. >> * So replace it to an original value. >> */ >> ret_stack = ftrace_graph_get_ret_stack(tsk, frame->graph++); >> if (WARN_ON_ONCE(!ret_stack)) >> return -EINVAL; >> frame->pc = ret_stack->ret; >> } >> #endif /* CONFIG_FUNCTION_GRAPH_TRACER */ > > Beware that this handles the case where a function will return to > return_to_handler, but doesn't handle unwinding from *within* > return_to_handler, which we can't do reliably today, so that might need > special handling. > OK. I will take a look at this. >> Is there anything else that needs handling here? > > I wrote up a few cases to consider in: > > https://www.kernel.org/doc/html/latest/livepatch/reliable-stacktrace.html > > ... e.g. the "Obscuring of return addresses" case. > > It might be that we're fine so long as we refuse to unwind across > exception boundaries, but it needs some thought. We probably need to go > over each of the trampolines instruction-by-instruction to consider > that. > > As mentioned above, within return_to_handler when we call > ftrace_return_to_handler, there's a period where the real return address > has been removed from the ftrace return stack, but hasn't yet been > placed in x30, and wouldn't show up in a trace (e.g. if we could somehow > hook the return from ftrace_return_to_handler). > > We might be saved by the fact we'll mark traces across exception > boundaries as unreliable, but I haven't thought very hard about it. We > might want to explciitly reject unwinds within return_to_handler in case > it's possible to interpose ftrace_return_to_handler somehow. > OK. I will study the above. Thanks. Madhavan _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel