Re: Kernel ABI Monitoring and toolchain support

["Guillermo E. Martinez" <guillermo.e.martinez@oracle.com>]

Hello,

Any comment?

Thanks,
guillermo

> Hello guys, 
> 
> The paragraphs bellow summarize the proposal  for Kernel ABI analysis
> talk in LPC, these ideas comes from the thread mail bellow.
> 
> Please let me know your comments to submit it
> to https://lpc.events/event/16/abstracts/
> 
> guillermo
> 
> ---
> Kernel ABI Monitoring and toolchain support
> 
> The new CTF(Compact C Type Format) supported  in  libabigail  is  able
> to extract a  corpus  representation  for  the  debug  information  in
> Kernel binary and its modules, i.e, entire Kernel  release  (kernel  +
> modules).  Using CTF reader improvements the time to extract and build
> the corpus compared with DWARF reader,  for  example,  extracting  ABI
> information  from  the  Linux  kernel  takes  up  to  ~4.5times   less
> time, this was done using a Kernel  compiled  by  GCC,  nowadays  LLVM
> doesn't support binaries generation with CTF debug info, would be nice
> to have this.
> 
> But what about of the modules inserted  (loaded)  at  runtime  in  the
> Kernel image?.  To make the comparison it uses kABI  scripts  this  is
> useful among other things to load modules with compatible  kABI,  this
> mechanism allows modules to be used with a  different  kernel  version
> that of the kernel for which it was built.  So what of using a  single
> notion  of   ABI   (libabigail)   also   for   the   modules   loader?
> 
> Since we add support for  CTF  in  libabigail,  is  needed  the  patch
> for building the Kernel  with  CTF  enabled  in  the  Kernel  upstream
> configuration.  Also some GCC  attributes  that  affect  the  ABI  and
> are used by kernel hackers like noreturn,  interrupt,  etc.   are  not
> represented in DWARF/CTF debug  format  and  therefore  they  are  not
> present in the corpus.
> 
> A stricter conformance to DWARF standards would be nice, full DWARF  5
> support, getting things like ARM64 ABI extensions (e.g.,  for  HWASAN)
> into things like  elfutils  at  the  same  time  as  the  compile-link
> toolchain, more consistency between Clang and GCC debug info  for  the
> same sources, the same for Clang and Clang  with  full  LTO.   And  an
> extending ABI monitoring coverage beyond  just  architecture,  symbols
> and  types  /  dealing  with  header  constants,   macros   and   more
> 
> The  interest  in  discussing  ways  to  standardize  ABI   and   type
> information in a way that it can be embedded into binaries in  a  less
> ambiguous way.  In other words, what can we do to not rely entirely on
> intermediate formats like CTF or DWARF to make sense of an ABI?  Maybe
> CTF is already a good starting point, yet some  additions  are  needed
> (e.g. other language features like for C++)?
> 
> > On 6/21/22 15:19, Nick Desaulniers wrote:
> > > On Mon, Jun 20, 2022 at 6:42 AM Matthias Männich <maennich@google.com> wrote:
> > >> Hi!
> > >>
> > >> That is a very late response, sorry about that.
> > >>
> > >> In the Android MC we will have a talk about the latest news monitoring
> > >> Android Kernel ABIs already, but I definitely see value in a more
> > >> generic session in the Kernel+Toolchain MC. Is that schedule already
> > >> finalized or is there still a slot available and interest from your
> > >> side?
> > > We haven't finalized the schedule yet.  Matthias, Guillermo, Giuliano
> > > can one of you perhaps fill out a proposal then for the ideas
> > > discussed in this thread on https://lpc.events/event/16/abstracts/
> > > with the "Track" set to "Toolchains Track?"
> > >
> > 
> > Hi,
> > 
> > actually Guillermo was already working on that, will send something by the end of the day
> > 
> > elena
> > 
> > >> In particular, I would be interested in discussing ways to standardize
> > >> ABI and type information in a way that it can be embedded into
> > >> binaries in a less ambiguous way. In other words, what can we do to
> > >> not rely entirely on intermediate formats like CTF or DWARF to make
> > >> sense of an ABI? Maybe CTF is already a good starting point, yet some
> > >> additions are needed (e.g. other language features like for C++)?
> > >>
> > >> Cheers,
> > >> Matthias
> > >>
> > >>
> > >> On Sat, May 7, 2022 at 3:15 AM Guillermo E. Martinez
> > >> <guillermo.e.martinez@oracle.com> wrote:
> > >>> Hello guys,
> > >>>
> > >>> On our side we are working on:
> > >>>
> > >>>   * Gives the support  to extract the ABI for Linux kernel image
> > >>>   and Linux  kernel modules using the CFT backed in with
> > >>>   libabigail tools.
> > >>>   * Add support to compare packages with CTF debug format by using
> > >>>   abipkgdiff and other related libabigail tools.
> > >>>
> > >>>    * https://sourceware.org/pipermail/libabigail/2022q2/004340.html
> > >>>    * https://sourceware.org/pipermail/libabigail/2022q2/004345.html
> > >>>
> > >>> If you think that it could be a point to talk as part of your MC,  I can for sure
> > >>> provides more details about it.
> > >>>
> > >>> Thanks for your comments,
> > >>> Guillermo
> > >>>
> > >>> On Wednesday, April 20, 2022 12:34:59 PM CDT Giuliano Procida wrote:
> > >>>> Sorry for the late reply, I've been on leave and Mattias is still on leave.
> > >>>>
> > >>>> That sounds quite interesting.
> > >>>>
> > >>>> Matthias and I could certainly talk about what we've been busy with and
> > >>>> what are the main pain points for Android (not just the kernel).
> > >>>>
> > >>>> On what more would we like from the toolchain... it falls into at least two
> > >>>> buckets:
> > >>>>
> > >>>> 1. make the possible easier - stricter conformance to DWARF standards, full
> > >>>> DWARF 5 support, getting things like ARM64 ABI extensions (e.g., for
> > >>>> HWASAN) into things like elfutils at the same time as the compile-link
> > >>>> toolchain, more consistency between Clang and GCC debug info for the same
> > >>>> sources, the same for Clang and Clang with full LTO
> > >>>> 2. make the impossible possible - extending ABI monitoring coverage beyond
> > >>>> just architecture, symbols and types / dealing with header constants,
> > >>>> macros and more
> > >>>>
> > >>>> Giuliano.
> > >>>>
> > >>>>
> > >>>> On Wed, 20 Apr 2022 at 17:02, Jose E. Marchesi <jose.marchesi@oracle.com>
> > >>>> wrote:
> > >>>>
> > >>>>> [Adding Dodji Seketeli, libabigail maintainer,
> > >>>>>  and Guillermo Martinez, who works in the CTF support for libabigail
> > >>>>>  including kernel support.]
> > >>>>>
> > >>>>>> Hi Matthias and Giuliano,
> > >>>>>> Jose and I are currently in the planning process to put together a
> > >>>>>> Kernel+Toolchain microconference track at Linux Plumbers Conference
> > >>>>>> this year (Sept 12-14) in Dublin, Ireland.
> > >>>>>>
> > >>>>>> Would you all be interested in a leading session on Kernel ABI
> > >>>>>> Monitoring and perhaps what toolchain vendors can do to help, as part
> > >>>>>> of our MC (which itself is currently still in planning phase).
> > >>>
> > >>>
> > >>>
> > >
> > >
> > 
> > 
> 
> 
> 
> 
>