Re: Kernel ABI Monitoring and toolchain support

["Guillermo E. Martinez" <guillermo.e.martinez@oracle.com>]

On Tuesday, June 21, 2022 4:27:18 PM CDT Elena Zannoni wrote:

Hello guys, 

The paragraphs bellow summarize the proposal  for Kernel ABI analysis
talk in LPC, these ideas comes from the thread mail bellow.

Please let me know your comments to submit it
to https://lpc.events/event/16/abstracts/

guillermo

---
Kernel ABI Monitoring and toolchain support

The new CTF(Compact C Type Format) supported  in  libabigail  is  able
to extract a  corpus  representation  for  the  debug  information  in
Kernel binary and its modules, i.e, entire Kernel  release  (kernel  +
modules).  Using CTF reader improvements the time to extract and build
the corpus compared with DWARF reader,  for  example,  extracting  ABI
information  from  the  Linux  kernel  takes  up  to  ~4.5times   less
time, this was done using a Kernel  compiled  by  GCC,  nowadays  LLVM
doesn't support binaries generation with CTF debug info, would be nice
to have this.

But what about of the modules inserted  (loaded)  at  runtime  in  the
Kernel image?.  To make the comparison it uses kABI  scripts  this  is
useful among other things to load modules with compatible  kABI,  this
mechanism allows modules to be used with a  different  kernel  version
that of the kernel for which it was built.  So what of using a  single
notion  of   ABI   (libabigail)   also   for   the   modules   loader?

Since we add support for  CTF  in  libabigail,  is  needed  the  patch
for building the Kernel  with  CTF  enabled  in  the  Kernel  upstream
configuration.  Also some GCC  attributes  that  affect  the  ABI  and
are used by kernel hackers like noreturn,  interrupt,  etc.   are  not
represented in DWARF/CTF debug  format  and  therefore  they  are  not
present in the corpus.

A stricter conformance to DWARF standards would be nice, full DWARF  5
support, getting things like ARM64 ABI extensions (e.g.,  for  HWASAN)
into things like  elfutils  at  the  same  time  as  the  compile-link
toolchain, more consistency between Clang and GCC debug info  for  the
same sources, the same for Clang and Clang  with  full  LTO.   And  an
extending ABI monitoring coverage beyond  just  architecture,  symbols
and  types  /  dealing  with  header  constants,   macros   and   more

The  interest  in  discussing  ways  to  standardize  ABI   and   type
information in a way that it can be embedded into binaries in  a  less
ambiguous way.  In other words, what can we do to not rely entirely on
intermediate formats like CTF or DWARF to make sense of an ABI?  Maybe
CTF is already a good starting point, yet some  additions  are  needed
(e.g. other language features like for C++)?

> On 6/21/22 15:19, Nick Desaulniers wrote:
> > On Mon, Jun 20, 2022 at 6:42 AM Matthias Männich <maennich@google.com> wrote:
> >> Hi!
> >>
> >> That is a very late response, sorry about that.
> >>
> >> In the Android MC we will have a talk about the latest news monitoring
> >> Android Kernel ABIs already, but I definitely see value in a more
> >> generic session in the Kernel+Toolchain MC. Is that schedule already
> >> finalized or is there still a slot available and interest from your
> >> side?
> > We haven't finalized the schedule yet.  Matthias, Guillermo, Giuliano
> > can one of you perhaps fill out a proposal then for the ideas
> > discussed in this thread on https://lpc.events/event/16/abstracts/
> > with the "Track" set to "Toolchains Track?"
> >
> 
> Hi,
> 
> actually Guillermo was already working on that, will send something by the end of the day
> 
> elena
> 
> >> In particular, I would be interested in discussing ways to standardize
> >> ABI and type information in a way that it can be embedded into
> >> binaries in a less ambiguous way. In other words, what can we do to
> >> not rely entirely on intermediate formats like CTF or DWARF to make
> >> sense of an ABI? Maybe CTF is already a good starting point, yet some
> >> additions are needed (e.g. other language features like for C++)?
> >>
> >> Cheers,
> >> Matthias
> >>
> >>
> >> On Sat, May 7, 2022 at 3:15 AM Guillermo E. Martinez
> >> <guillermo.e.martinez@oracle.com> wrote:
> >>> Hello guys,
> >>>
> >>> On our side we are working on:
> >>>
> >>>   * Gives the support  to extract the ABI for Linux kernel image
> >>>   and Linux  kernel modules using the CFT backed in with
> >>>   libabigail tools.
> >>>   * Add support to compare packages with CTF debug format by using
> >>>   abipkgdiff and other related libabigail tools.
> >>>
> >>>    * https://sourceware.org/pipermail/libabigail/2022q2/004340.html
> >>>    * https://sourceware.org/pipermail/libabigail/2022q2/004345.html
> >>>
> >>> If you think that it could be a point to talk as part of your MC,  I can for sure
> >>> provides more details about it.
> >>>
> >>> Thanks for your comments,
> >>> Guillermo
> >>>
> >>> On Wednesday, April 20, 2022 12:34:59 PM CDT Giuliano Procida wrote:
> >>>> Sorry for the late reply, I've been on leave and Mattias is still on leave.
> >>>>
> >>>> That sounds quite interesting.
> >>>>
> >>>> Matthias and I could certainly talk about what we've been busy with and
> >>>> what are the main pain points for Android (not just the kernel).
> >>>>
> >>>> On what more would we like from the toolchain... it falls into at least two
> >>>> buckets:
> >>>>
> >>>> 1. make the possible easier - stricter conformance to DWARF standards, full
> >>>> DWARF 5 support, getting things like ARM64 ABI extensions (e.g., for
> >>>> HWASAN) into things like elfutils at the same time as the compile-link
> >>>> toolchain, more consistency between Clang and GCC debug info for the same
> >>>> sources, the same for Clang and Clang with full LTO
> >>>> 2. make the impossible possible - extending ABI monitoring coverage beyond
> >>>> just architecture, symbols and types / dealing with header constants,
> >>>> macros and more
> >>>>
> >>>> Giuliano.
> >>>>
> >>>>
> >>>> On Wed, 20 Apr 2022 at 17:02, Jose E. Marchesi <jose.marchesi@oracle.com>
> >>>> wrote:
> >>>>
> >>>>> [Adding Dodji Seketeli, libabigail maintainer,
> >>>>>  and Guillermo Martinez, who works in the CTF support for libabigail
> >>>>>  including kernel support.]
> >>>>>
> >>>>>> Hi Matthias and Giuliano,
> >>>>>> Jose and I are currently in the planning process to put together a
> >>>>>> Kernel+Toolchain microconference track at Linux Plumbers Conference
> >>>>>> this year (Sept 12-14) in Dublin, Ireland.
> >>>>>>
> >>>>>> Would you all be interested in a leading session on Kernel ABI
> >>>>>> Monitoring and perhaps what toolchain vendors can do to help, as part
> >>>>>> of our MC (which itself is currently still in planning phase).
> >>>
> >>>
> >>>
> >
> >
> 
>