From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on archive.lwn.net X-Spam-Level: X-Spam-Status: No, score=-6.1 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham autolearn_force=no version=3.4.2 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by archive.lwn.net (Postfix) with ESMTP id 9BB4C7D08A for ; Thu, 14 Mar 2019 12:27:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726967AbfCNM1w (ORCPT ); Thu, 14 Mar 2019 08:27:52 -0400 Received: from mail-qt1-f193.google.com ([209.85.160.193]:33853 "EHLO mail-qt1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726776AbfCNM1v (ORCPT ); Thu, 14 Mar 2019 08:27:51 -0400 Received: by mail-qt1-f193.google.com with SMTP id k2so5777815qtm.1 for ; Thu, 14 Mar 2019 05:27:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=joelfernandes.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=6flR/SZ3bVyvGpeBMlCe2ZUtI2LHYS6gvOXI4e6BfpI=; b=kQh7eu4cXQZsGXIDCyZUWIYrnP4SyHjyEFvqeDX9qwPtnZcuVZwGG+PjPGwiNcoOW4 K6ThH9dTXFVHXQ8v4C8+VL2bakspMZATm2KkwyVhMgEQB5zCyrrNAkrL1M2cYBGHrKzN f+mj5cuB3C4v+kP1Bu40/h0NQwqAOdgzZb3J4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=6flR/SZ3bVyvGpeBMlCe2ZUtI2LHYS6gvOXI4e6BfpI=; b=A8cUMAurlOpBasDnPaITTa+KcmG9Kq8tJER+1FlZxVN0JrDD26GH/0eUPnj5wlUFz7 FSX85HdkdWtd+4ahyk8W80UWda3MMJ9Qvj5BuqVJQSZjpynBMfVF0wUecDmOEHPRh1Gg uJXT4JRcGfKabXaxJyqX0L6FVC5HZodkehUu6Ov6Pwg4pUllH8ntZw04WalCqcffzieB 2KqfUjkBD2vSZOhXJfKG+6aLg6p5ZJUbo7HxVLPnv91ZmWnJw77drY2Ye23XB9dLAiGj olzo9rQ0Yrg+90TL43EBOoxFZwycSxAdI0DawJSZ0f59tDJgu3BSk3+VtC+erLbGBAYs LeXQ== X-Gm-Message-State: APjAAAVJnFNr+4SXFr1GlifJH1yKFSCTiw6Cr2AHA/KFY/vUgRSu8iln P72pIj4J8QIcpqYwZhAIwu499H4dzyE= X-Google-Smtp-Source: APXvYqy2IZqfZEb8pYZbKHCl44Uhexadk5h0/fwVSteoQa1v1VP2dysxz0REKQPzCiHRqC15nDXZfg== X-Received: by 2002:ac8:2bc1:: with SMTP id n1mr37856804qtn.176.1552566470361; Thu, 14 Mar 2019 05:27:50 -0700 (PDT) Received: from localhost ([2620:0:1004:1100:cca9:fccc:8667:9bdc]) by smtp.gmail.com with ESMTPSA id k33sm8837364qte.8.2019.03.14.05.27.48 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 14 Mar 2019 05:27:48 -0700 (PDT) Date: Thu, 14 Mar 2019 08:27:48 -0400 From: Joel Fernandes To: Masami Hiramatsu Cc: Steven Rostedt , Daniel Colascione , Karim Yaghmour , Geert Uytterhoeven , Greg KH , LKML , Andrew Morton , Alexei Starovoitov , atish patra , Dan Williams , Dietmar Eggemann , Guenter Roeck , Jonathan Corbet , Kees Cook , Android Kernel Team , "open list:DOCUMENTATION" , "open list:KERNEL SELFTEST FRAMEWORK" , linux-trace-devel@vger.kernel.org, Manoj Rao , Masahiro Yamada , Qais Yousef , Randy Dunlap , Shuah Khan , Yonghong Song Subject: Re: [PATCH v4 1/2] Provide in-kernel headers for making it easy to extend the kernel Message-ID: <20190314122748.GA101545@google.com> References: <20190309121141.GA30173@kroah.com> <3e84e1ef-e266-e983-5874-6c26ac7f38b8@opersys.com> <20190311193612.4f09bf11@oasis.local.home> <20190312003912.GA170478@google.com> <20190311212823.60684182@oasis.local.home> <20190313101843.800dd8d6a9babb635580ebdc@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190313101843.800dd8d6a9babb635580ebdc@kernel.org> User-Agent: Mutt/1.10.1 (2018-07-13) Sender: linux-doc-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-doc@vger.kernel.org Hi Masami, Sorry for late reply as I was traveling. On Wed, Mar 13, 2019 at 10:18:43AM +0900, Masami Hiramatsu wrote: > On Mon, 11 Mar 2019 18:38:49 -0700 > Joel Fernandes wrote: > > > On Mon, Mar 11, 2019 at 6:28 PM Steven Rostedt wrote: > > > > > > On Mon, 11 Mar 2019 20:39:12 -0400 > > > Joel Fernandes wrote: > > > > > > > I think even though the kernel-headers can't have information about all data > > > > structures, they do already contain a lot of data structure definitions we > > > > need already. And anything needed can/should arguably be moved to include/ if > > > > they are really needed for kernel extension by something "external" to the > > > > kernel such as kernel modules or eBPF, right? > > > > > > That's not my worry. I would like to be able to easily walk data > > > structures from within the kernel, without having to do a lot of work > > > in userspace to get that information. The kprobe_events could then be > > > passed type casts or such to access data fields of arguments to > > > functions and such. > > > > Ok. > > > > > > In any case, such a solution such as what Steve suggested, still cannot do > > > > what we can with headers - such as build kernel modules on the fly using the > > > > C-compiler without any auto-generation of C code from any debug artifiacts. > > > > Think systemtap working with the module-backend without any need for > > > > linux-headers package on the file system. So such a solution would still be a > > > > bit orthogonal in scope to what this proposed solution can solve IMO. > > > > > > > > > > With the information I would like to have, it would be trivial to read > > > the data to create the header files needed for modules. > > > > But there are macros and other #define things too. We lose all of them > > and can't recreate them from just DWARF (AFAIK). Including > > include/generated/autoconf.h which #defines the CONFIG options. For > > that we either need headers, or full kernel's sources with build > > artifacts. > > What kind of macros would you concern? > > > I do see a use case for the debug info you are talking about as you > > mentioned for the kprobe_events argument list types, and I already > > thought about it. But it does not seem to work for all the use cases I > > am referring to here. > > But the eBPF is based on kprobe-events. What kind of usage would you > expected? (with macros??) eBPF C programs are compiled with kernel headers. They can execute inline functions or refer to macros in the kernel headers. They are similar to kernel modules where you build a C program that then later is executed in kernel context. It goes through the whole compiler pipeline. This is slightly different usage from pure kprobe-events. Also eBPF kprobe programs need LINUX_VERSION_CODE (or similarly named) macro which it provides to the bpf(2) syscall when loading kprobe programs. This is because eBPF implementation in the kernel checks if the eBPF programs that use kprobes are being loaded against the right kernel. thanks, - Joel