From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8DA12EE0213 for ; Wed, 11 Sep 2024 15:34:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: Content-Type:Mime-Version:References:In-Reply-To:Message-Id:Subject:Cc:To: From:Date:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=LWPvx8OAD9WbM6BkilLqynuI3WVznZQOz+vgeV8MyeM=; b=BS3cDG8WwvjOIGhWNUwT9GrJVF XLFsn8wbwmXYo5RDM/EChI704c+QlTzera0wPh494O3AnQn0yco7q2BuvnqAXTkaa8pmLizqmu86u 1mycTfHPzKLVP38Rdo/AR/M3mWVssMWe+GVHOEErph0jRj700w8rC6UpNcHtZq5SGPsbocHpwP/kT F3sMWOUFv49cOcLWNOShTnFalEklSQnRLH6doB7Dzd6dwAjnT67ZgzFlNIoD8AKUXykRvxm/F3o/w SH8Wk4k/y1DZ3C+WETam+b0JVuPRZF/bSUyQrLUDFgs79iBGyWSU8UMvIIX7UwHV4Y3Mpr98pkQOd cCFFozTw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1soPM6-0000000AAqP-1PzY; Wed, 11 Sep 2024 15:34:06 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1soP7O-0000000A6jI-2ays for linux-arm-kernel@lists.infradead.org; Wed, 11 Sep 2024 15:18:56 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id CF3DA5C06DA; Wed, 11 Sep 2024 15:18:49 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 38B85C4CEC0; Wed, 11 Sep 2024 15:18:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1726067933; bh=dIMTjUUU6Z+9RVyokzndesj6W8X7N4m9fc53dLvlP88=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=M+2YFp+Jfs373JDhFk6aFPyisWNPDkvgBdhWdiHTa8+1cxhN6kvtNXDIC8JZzUEs9 jotL9FYDXo05Hg5u+Hcrt4x2ilKB5TDwzeOg2pNdfz67HlRM75frRCYmPa6Pbj2J5q amh44oTixkfp6AZplnP4U9FBNe9VCZKt70j+7aCsoW9+gkBjbRgBRmBaaXIDPSfl0S 1++1jhpqYsaDoDrPhA6Tem3zCgumFqpbsDimb77BXuH0VmxCnMmHWq64o6KyGbs56T X+0Ly9Uli8JxGCXAsBzqwTtxmXoA5emY/Tc/EqDzVARleqVW4H0rDjhtDygr9+IA1G DZKhrv7M3kQ5g== Date: Thu, 12 Sep 2024 00:18:48 +0900 From: Masami Hiramatsu (Google) To: Andrii Nakryiko Cc: Steven Rostedt , bpf , Linux trace kernel , adubey@linux.ibm.com, "Naveen N. Rao" , KP Singh , linux-arm-kernel , Mark Rutland , Will Deacon , Alexei Starovoitov , Catalin Marinas , Florent Revest , Puranjay Mohan Subject: Re: Unsupported CONFIG_FPROBE and CONFIG_RETHOOK on ARM64 Message-Id: <20240912001848.d9629a1579ea3ef6531a9a0b@kernel.org> In-Reply-To: References: <20240911091343.77c60bc2e5d96cbfd8787c19@kernel.org> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; x86_64-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240911_081854_815760_FCC927EB X-CRM114-Status: GOOD ( 47.75 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, 10 Sep 2024 17:37:48 -0700 Andrii Nakryiko wrote: > On Tue, Sep 10, 2024 at 5:13 PM Masami Hiramatsu wrote: > > > > On Tue, 10 Sep 2024 11:23:29 -0700 > > Andrii Nakryiko wrote: > > > > > + arm ML and maintainers > > > > > > On Wed, Sep 4, 2024 at 6:02 PM Andrii Nakryiko > > > wrote: > > > > > > > > Hey, > > > > > > > > I just recently realized that we are still missing multi-kprobe > > > > support for ARM64, which depends on CONFIG_FPROBE. And CONFIG_FPROBE > > > > seems to require CONFIG_HAVE_RETHOOK, which, it turns out, is not > > > > implemented for ARM64. > > > > > > > > It took me a while to realize what's going on, as I roughly remembered > > > > (and confirmed through lore search) that Masami's original rethook > > > > patches had arm64-specific bits. Long story short: > > > > > > > > 0f8f8030038a Revert "arm64: rethook: Add arm64 rethook implementation" > > > > 83acdce68949 arm64: rethook: Add arm64 rethook implementation > > > > > > > > The patch was landed and then reverted. I found some discussion online > > > > and it seems like the plan was to land arch-specific bits shortly > > > > after bpf-next PR. > > > > > > > > But it seems like that never happened. Why? > > > > > > > > I see s390x, RISC-V, loongarch (I'm not even mentioning x86-64) all > > > > have CONFIG_HAVE_RETHOOK, even powerpc is getting one (see [0]), it > > > > seems. How come ARM64 is the one left out? > > > > > > > > Can anyone please provide some context? And if that's just an > > > > oversight, can we prioritize landing this for ARM64 ASAP? > > > > > > > > [0] https://lore.kernel.org/bpf/20240830113131.7597-1-adubey@linux.ibm.com/ > > > > > > > > > > Masami, Steven, > > > > > > Does Linus have to be in CC to get any reply here? Come on, it's been > > > almost a full week. > > > > Sorry about bothering you, let me check that. But I think we eventually > > You don't bother me, but I'd appreciate a bit more timely replies in > the future, if that's OK. > > > need my fprobe-on-fgraph patch which allows all architecture uses ftrace_regs > > instead of pt_regs for ftrace/fgraph users. That allows arm64 to implement > > fprobe. > > Ok, thanks for a bit more context. I understand the end goal with > fprobe-on-fgraph, but see below. > > > > > > > > > Maybe ARM64 folks have some context?... And hopefully desire to see > > > this through so that ARM64 doesn't stick out as a lesser-supported > > > platform as far as tracing goes compared to loongarch, s390x, and > > > powerpc (which just landed rethook support, see [2]). > > > > I think lesser-supported or not is not a matter, but they need to keep > > their architecutre healthy. Mark said that the current rethook > > implementation is not acceptable because arm64 can not manually generate > > I don't see Mark's reply in the link you sent. But did he refer to the > code in kprobes_trampoline.S or is it something different? Sorry, here it is: https://lkml.org/lkml/2022/4/12/2233 > > By lesser-supported I mean that a very important functionality (BPF > multi-kprobe, which relies on CONFIG_FPROBE and thus > {HAVE|CONFIG}_RETHOOK) is currently still missing. And whether x86-64 > support landed more than 2 years ago (end of March 2022), the second > practically most popular (and thus important for tools and such) ARM64 > platform still doesn't have this functionality. > > And that's limiting, BPF multi-kprobes are a huge improvement in > tooling usability. Sorry for inconvenient. But I think this transformation is really important. > So while I get the desire to have a clean and nice > end goal, and that it might take a bit longer to get everything right. > But, maybe, landing a stop-gap solution meanwhile (especially as > isolated and thus easily backportable as the patch [0] you referenced) > is an OK path forward? I had not realized that the PSTATE register was not saved correctly at that point. This is one reason why I decided to move in the current fprobe-on-fgraph direction. > > I'm just lacking full understanding on what exactly the issue is/was, > and that's why I'm asking all these questions. I'm not sure if [0] is > just broken for some subtle reason, or it is just suboptimal in some > sense (performance, code duplication, whatnot)? If [0] was not broken, I pushed it and the current pt_regs to ftrace_regs series is separated series. But it was broken. So I tried to find the correct way to fix it, and finally introduced the current fprobe on fgraph series. performance improvement is just a side effect. Thank you, > > > [0] https://lore.kernel.org/bpf/164338038439.2429999.17564843625400931820.stgit@devnote2/ > > > pt_regs. So we need to use ftrace_regs for that. > > So eventually, we need my fprobe series. > > > > https://lore.kernel.org/bpf/164338038439.2429999.17564843625400931820.stgit@devnote2/ > > > > Thank you, > > > > > > > > Note that there was already an implementation (see [1]), but for some > > > reason it never made it. > > > > > > [1] https://lore.kernel.org/bpf/164338038439.2429999.17564843625400931820.stgit@devnote2/ > > > [2] https://lore.kernel.org/bpf/172562357215.467568.2172858907419105155.b4-ty@ellerman.id.au/ > > > > > > > > > > > -- Andrii > > > > > > -- > > Masami Hiramatsu (Google) -- Masami Hiramatsu (Google)