[PATCHES 00/12] pahole: Support more rust tags and references to dwz alternate debug files

[Arnaldo Carvalho de Melo <acme@kernel.org>]

Hi,

Add support for Rust discriminated unions (enums with explicit
discriminants).

Rust enums like Option<u32> and Result<i32,u8> are represented in DWARF
as DW_TAG_variant_part containing DW_TAG_variant children with
discriminant values. These patches:

  - Load DW_TAG_variant_part containers with discriminant tracking
  - Populate DW_TAG_variant children with discriminant values
  - Handle DW_FORM_block encoding for discriminant values
  - Handle DW_TAG_subprogram inside DW_TAG_enumeration (Rust methods)
  - Encode variant parts as BTF_KIND_UNION members

Before: Rust Option<u32> showed as empty struct (0 members)

After:  Shows None and Some variants with correct layout

  === pahole --expand_types output ===
  struct Option<u32> {
      struct None {
      } __attribute__((__aligned__(4)));
      struct Some {
          /* XXX 4 bytes hole, try to pack */
          u32  __0 __attribute__((__aligned__(4)));  /*  4  4 */
      } __attribute__((__aligned__(4)));
  } __attribute__((__aligned__(4)));

BTF has no discriminated-union kind, so variant parts are tentatively
encoded as BTF_KIND_UNION with overlapping members at offset 0. Further
discussion is needed to see if we can keep that.

DW_FORM_block byte order fix: uses CU-recorded endianness instead of
assuming little-endian when decoding DW_AT_const_value and
DW_AT_default_value. Fixes cross-endian DWARF processing (e.g.  reading
big-endian s390x debug info on x86).

Also add support for cross-CU type references and dwz alternate debug
files.

This is needed to improve the support for, among other things, Rust, as
noticed in the pretty printing or a perf binary that has rust objects
linked, built by clang/llvm that use cross-CU references.

DW_TAG_imported_unit support:

  Handle same-file partial units where types are shared across CUs
  via DW_TAG_imported_unit. Force-merge CUs that contain inter-CU
  references (DW_FORM_ref_addr) so type lookups resolve correctly.

  Fix cus__merging_cu failing to detect DW_FORM_ref_addr when
  DW_FORM_implicit_const causes dwarf_getabbrevattr() to fail.

dwz alternate debug file support:

  Handle DW_FORM_GNU_ref_alt references to dwz-compressed alternate
  debug files (.dwz). Pre-processes all alternate partial units with
  separate hash tables and dedup tracking.

  Before (Firefox): 1774 "has no entry in cu" errors
  After:  processes cleanly with 0 errors

  Firefox uses dwz compression which moves common types into a
  separate .dwz file and replaces duplicates with DW_FORM_GNU_ref_alt
  references. pahole now pre-loads the alternate file's partial units,
  resolves all cross-file references, and processes Firefox DWARF
  cleanly.

Includes inter-CU type reference comparison test and
vmlinux_comparison.py for DWARF/BTF analysis across kernel configs.

Known limitations:
  - Pruning of unreferenced alt PUs is conservatively over-approximated
  - Merged-CU path is single-threaded by design

This makes pahole to be able to support more of the CONFIG_DEBUG_ DWARF
options, including DWARF5 and compression, see the latest patch in the
series for more details about how this table is produced:

  ┌───────────────────────────┬─────────┬─────┬───────────────────────────────────┬─────────┬───────┬──────────┬────────┬──────────┬─────────┐
  │ File                      │  Size   │ Ver │ Producer                          │  DWARF  │  BTF  │ DW-Hash  │ DW-Out │ BTF-Hash │ BTF-Out │
  ├───────────────────────────┼─────────┼─────┼───────────────────────────────────┼─────────┼───────┼──────────┼────────┼──────────┼─────────┤
  │ vmlinux.dwarf4            │ 835.7MB │   4 │ GNU C11 16.1.1 -gdwarf-4          │ 599.2MB │ 7.3MB │ 86e1b611 │  7.6MB │ 52123c24 │   7.4MB │
  │ vmlinux.dwarf5            │ 736.3MB │   5 │ GNU C11 16.1.1 -gdwarf-5          │ 499.7MB │ 7.3MB │ 86e1b611 │  7.6MB │ 52123c24 │   7.4MB │
  │ vmlinux.dwarf5.zlib       │ 497.8MB │   5 │ GNU C11 16.1.1 -gdwarf-5 -gz=zlib │ 261.2MB │ 7.3MB │ 86e1b611 │  7.6MB │ 52123c24 │   7.4MB │
  │ vmlinux.dwarf5.zstd       │ 454.7MB │   5 │ GNU C11 16.1.1 -gdwarf-5 -gz=zstd │ 218.1MB │ 7.3MB │ 86e1b611 │  7.6MB │ 52123c24 │   7.4MB │
  │ vmlinux.toolchain_default │ 736.3MB │   5 │ GNU C11 16.1.1                    │ 499.7MB │ 7.3MB │ 86e1b611 │  7.6MB │ 52123c24 │   7.4MB │
  └───────────────────────────┴─────────┴─────┴───────────────────────────────────┴─────────┴───────┴──────────┴────────┴──────────┴─────────┘

Split DWARF support (skeleton CUs) will be supported in upcoming work,
so as to cover all the possibilities the kernel build system offers
for generating DWARF.

- Arnaldo

Arnaldo Carvalho de Melo (12):
  dwarf_loader: Initial support for DW_TAG_variant_part
  dwarf_loader: Initial support for DW_TAG_subprogram in DW_TAG_enumeration
  dwarf_loader: Populate DW_TAG_variant children in DW_TAG_variant_part
  btf_encoder: Encode variant parts as union members in BTF
  dwarf_loader: Handle DW_FORM_block in attr_numeric for Rust discriminant values
  dwarf_loader: Allow forcing the merge of CUs for solving inter CU tag references
  dwarf_loader: Support DW_TAG_imported_unit for same-file partial units
  dwarf_loader: Fix cus__merging_cu failing to detect DW_FORM_ref_addr
  dwarf_loader: Add cu parameter to tag__set_spec() and dwarf_tag__set_attr_type()
  dwarf_loader: Support DW_FORM_GNU_ref_alt references to dwz alternate debug files
  tests: Add inter-CU type reference comparison test
  scripts: Add vmlinux_comparison.py for DWARF/BTF analysis

 btf_encoder.c                 |  66 ++-
 ctf_encoder.c                 |  34 +-
 dwarf_loader.c                | 873 ++++++++++++++++++++++++++++++----
 dwarves.c                     |  77 ++-
 dwarves.h                     |  63 ++-
 dwarves_emit.c                |  10 +-
 dwarves_fprintf.c             |  54 ++-
 man-pages/pahole.1            |  18 +-
 pahole.c                      |  20 +-
 scripts/vmlinux_comparison.py | 367 ++++++++++++++
 tests/block_endian.sh         | 146 ++++++
 tests/dwz_alt_file.sh         | 204 ++++++++
 tests/inter_cu_refs.sh        |  50 ++
 tests/prettify_perf.data.sh   |   4 +-
 14 files changed, 1854 insertions(+), 132 deletions(-)
 create mode 100755 scripts/vmlinux_comparison.py
 create mode 100755 tests/block_endian.sh
 create mode 100755 tests/dwz_alt_file.sh
 create mode 100755 tests/inter_cu_refs.sh

-- 
2.55.0