From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0A5C4C433F5 for ; Tue, 3 May 2022 14:39:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237202AbiECOmw (ORCPT ); Tue, 3 May 2022 10:42:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36782 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237201AbiECOmv (ORCPT ); Tue, 3 May 2022 10:42:51 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7720F35877 for ; Tue, 3 May 2022 07:39:19 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 1B49D617EC for ; Tue, 3 May 2022 14:39:19 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 527F3C385A4; Tue, 3 May 2022 14:39:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1651588758; bh=CS2zAsquoDYrnlAR3H/iaX6OZsbg9Vkx++mlmtpVBQA=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=qbidaDAM1pj18x+dwnGpeWpjhjpe3oZ5T3GSHYcn2vIQBw3cR75yaZlrfG07PyzU1 i8Wsaovhg/xjo07BU3W5sStS/xOPBlUjlGKu4x3v/eB7xmOIh5ETCEQOUCDjRiQLL0 58lzpLIzMt/R5aahc9+aS25yrvqWxz8pZ3mIH6n+sTI3YqqSPQoWKuyzPlO8b06y05 bZMcwoEiJMueZSCF+9Wh4ghIbni955ROhxbqjcvYIia8Ze+yiImAMZ0NDqIkJnVXl/ wMys1cXvFgYKrb8ZypXcJ36yRaWf6Mbnh5Nb31/urRzATNcRvMbsE+UnLQOtpxb8O6 M5poheek5E/hw== Received: by quaco.ghostprotocols.net (Postfix, from userid 1000) id 9D9CC400B1; Tue, 3 May 2022 11:39:15 -0300 (-03) Date: Tue, 3 May 2022 11:39:15 -0300 From: Arnaldo Carvalho de Melo To: Alexei Starovoitov Cc: Stephen Brennan , Andrii Nakryiko , Yonghong Song , Shung-Hsi Yu , bpf , Omar Sandoval , Arnaldo Carvalho de Melo Subject: Re: Question: missing vmlinux BTF variable declarations Message-ID: References: <586a6288-704a-f7a7-b256-e18a675927df@oracle.com> <8735jjw4rp.fsf@brennan.io> <878rt9hogh.fsf@brennan.io> <87r15iv0yd.fsf@stepbren-lnx.us.oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Url: http://acmel.wordpress.com Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org Em Fri, Apr 29, 2022 at 10:10:01AM -0700, Alexei Starovoitov escreveu: > On Wed, Apr 27, 2022 at 11:43 AM Stephen Brennan wrote: > > [2]: https://github.com/brenns10/drgn/tree/kallsyms_plus_btf > > Combining these three things, I've got a debugger which can open up a > > vmcore _without DWARF debuginfo_ and allow you to print out typed > > variable values. It just relies on BTF + kallsyms. > > So the proof of concept is proven, and I'm quite excited about it! > Exciting indeed. This is pretty cool. Indeed! > I'm afraid we cannot justify 2.5 Mb kernel memory increase for pure > debugging. The existing vmlinux BTF is used by the kernel itself to > validate bpf prog access. bpf progs cannot access normal global vars. > If/when they are we can reconsider. > As an alternative path I think we could introduce hierarchical > split BTF. Which we already have in the form of BTF for modules that use vmlinux as a base for common types. > Currently vmlinux BTF and BTF of kernel modules is a tree > of depth 2. > We can keep such representation of BTFs and introduce a fake kernel > module that contains kernel global vars. pahole would generate a naked BTF just with variables and types not present in the main vmlinux BTF and refer to it for all the other types. > drgn can parse vmlinux BTF plus BTFs of all ko-s including fake one > and obtain the same amount of debug info as if global vars > were part of vmlinux BTF. Right. > Consuming 2.5Mb on demand via ko would be acceptable in some scenarios > whereas unconditionally burning that much memory in vmlinux BTF (even > optional via kconfig) is probably not. And since it would be just an extra kernel module, the existing packaging processes (in distros, embedded systems, etc) that care about BTF would carry this without any modification to existing practices, i.e. selecting CONFIG_DEBUG_INFO_BTF=y would bt default enable CONFIG_DEBUG_INFO_GLOBAL_VARIABLES_BTF=y, which could be optionally disabled by people not wanting to carry this extra info. I.e. it would be always available but not always loaded. > Ideally we structure BTFs as a multi level tree. Where BTF with > global vars and other non essential BTF info can be added to vmlinux > BTF at run-time. BTF of kernel mods can add on top and mods can have > split BTF too. Yeah, reuses existing mechanizm, doesn't increase the kernel BTF footprint by default, allows for debuggers, profilers, tracers, etc to ask for extra info in the form of just loading btf_global_variables.ko. - Arnaldo