bpftool fails on 32-bit hosts when -Wa,--compress-debug-sections is used

[Sergei Trofimovich <slyich@gmail.com>]

Hi bpf@!

Tl;DR:

BPF programs built with `clang`'s `-g -Wa,--compress-debug-sections`
fail when processed by `bpftool` on section alignment checks as:

    $ cat src/core/bpf/socket_bind/bpf.c
    struct socket_bind_map_t { int value; };
    struct socket_bind_map_t sd_bind_allow __attribute__((section(".maps"), used));

    $ clang-16 -target bpf -fno-stack-protector -c src/core/bpf/socket_bind/bpf.c -o bpf.unstripped.o -g -Wa,--compress-debug-sections
    $ bpftool gen object bpf.o bpf.unstripped.o
    libbpf: ELF section #9 has inconsistent alignment addr=8 != d=4 in bpf.unstripped.o
    Error: failed to link 'bpf.unstripped.o': Invalid argument (22)

This happens only when `bpftool` is a 32-bit ELF binary. 64-bit
`bpftool` seems to work as expected.

More words:

Here is my understanding of the failure. Without the
`-Wa,--compress-debug-sections` option `clang` generates byte-aligned
uncompressed `.debug` sections:

    $ clang -target bpf -fno-stack-protector -c src/core/bpf/socket_bind/bpf.c -o bpf.unstripped.o -g
    $ readelf -SW bpf.unstripped.o
    There are 19 section headers, starting at offset 0x530:
    
    Section Headers:
      [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
      [ 0]                   NULL            0000000000000000 000000 000000 00      0   0  0
      [ 1] .strtab           STRTAB          0000000000000000 000471 0000b8 00      0   0  1
      [ 2] .text             PROGBITS        0000000000000000 000040 000000 00  AX  0   0  4
      [ 3] .maps             PROGBITS        0000000000000000 000040 000004 00  WA  0   0  4
      [ 4] .debug_abbrev     PROGBITS        0000000000000000 000044 00004c 00      0   0  1
      [ 5] .debug_info       PROGBITS        0000000000000000 000090 00003d 00      0   0  1
      [ 6] .rel.debug_info   REL             0000000000000000 000380 000040 10   I 18   5  8
      [ 7] .debug_str_offsets PROGBITS        0000000000000000 0000cd 000024 00      0   0  1
      [ 8] .rel.debug_str_offsets REL             0000000000000000 0003c0 000070 10   I 18   7  8
      [ 9] .debug_str        PROGBITS        0000000000000000 0000f1 000080 01  MS  0   0  1
      [10] .debug_addr       PROGBITS        0000000000000000 000171 000010 00      0   0  1
      [11] .rel.debug_addr   REL             0000000000000000 000430 000010 10   I 18  10  8
      [12] .BTF              PROGBITS        0000000000000000 000184 000099 00      0   0  4
      [13] .rel.BTF          REL             0000000000000000 000440 000010 10   I 18  12  8
      [14] .debug_line       PROGBITS        0000000000000000 00021d 000043 00      0   0  1
      [15] .rel.debug_line   REL             0000000000000000 000450 000020 10   I 18  14  8
      [16] .debug_line_str   PROGBITS        0000000000000000 000260 000041 01  MS  0   0  1
      [17] .llvm_addrsig     LOOS+0xfff4c03  0000000000000000 000470 000001 00   E 18   0  1
      [18] .symtab           SYMTAB          0000000000000000 0002a8 0000d8 18      1   8  8
    Key to Flags:
      W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
      L (link order), O (extra OS processing required), G (group), T (TLS),
      C (compressed), x (unknown), o (OS specific), E (exclude),
      D (mbind), p (processor specific)

Note how `.debug_str` has `alignment=1` here. Loading of such a section
has no problems on x86_64 or i686.

But when we add `-Wa,--compress-debug-sections` some debug sections
become aligned 8 bytes (usual `ELF64`) alignment:

    $ clang -target bpf -fno-stack-protector -c src/core/bpf/socket_bind/bpf.c -o bpf.unstripped.o -g -Wa,--compress-debug-sections
    $ LANG=C readelf -SW bpf.unstripped.o
    There are 19 section headers, starting at offset 0x530:

    Section Headers:
      [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
      [ 0]                   NULL            0000000000000000 000000 000000 00      0   0  0
      [ 1] .strtab           STRTAB          0000000000000000 000471 0000b8 00      0   0  1
      [ 2] .text             PROGBITS        0000000000000000 000040 000000 00  AX  0   0  4
      [ 3] .maps             PROGBITS        0000000000000000 000040 000004 00  WA  0   0  4
      [ 4] .debug_abbrev     PROGBITS        0000000000000000 000044 00004c 00      0   0  1
      [ 5] .debug_info       PROGBITS        0000000000000000 000090 00003d 00      0   0  1
      [ 6] .rel.debug_info   REL             0000000000000000 000380 000040 10   I 18   5  8
      [ 7] .debug_str_offsets PROGBITS        0000000000000000 0000cd 000024 00      0   0  1
      [ 8] .rel.debug_str_offsets REL             0000000000000000 0003c0 000070 10   I 18   7  8
      [ 9] .debug_str        PROGBITS        0000000000000000 0000f1 000086 01 MSC  0   0  8
      [10] .debug_addr       PROGBITS        0000000000000000 000177 000010 00      0   0  1
      [11] .rel.debug_addr   REL             0000000000000000 000430 000010 10   I 18  10  8
      [12] .BTF              PROGBITS        0000000000000000 000188 000099 00      0   0  4
      [13] .rel.BTF          REL             0000000000000000 000440 000010 10   I 18  12  8
      [14] .debug_line       PROGBITS        0000000000000000 000221 000043 00      0   0  1
      [15] .rel.debug_line   REL             0000000000000000 000450 000020 10   I 18  14  8
      [16] .debug_line_str   PROGBITS        0000000000000000 000264 000041 01  MS  0   0  1
      [17] .llvm_addrsig     LOOS+0xfff4c03  0000000000000000 000470 000001 00   E 18   0  1
      [18] .symtab           SYMTAB          0000000000000000 0002a8 0000d8 18      1   8  8
    Key to Flags:
      W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
      L (link order), O (extra OS processing required), G (group), T (TLS),
      C (compressed), x (unknown), o (OS specific), E (exclude),
      D (mbind), p (processor specific)

Note how `.debug_str` alignment changed from 1-byte to 8-byte alignment.

AFAIU 8-byte alignment of compressed section for `ELF64` files is expected:
compressed sections all have a small header that consistes of a few
64-bit values (compression type, uncompressed size, uncompressed data
alignment).

Thus the binary loads as expected by `x86_64` `bpftool` and fails when
loaded by `i686` `bpftool`.

Should `bpftool` work in this scenario? Or compressed sections are not
supported on 32-bit hosts?

It feels like debugging sections with strings should be easily
decompressable on any host type.

Thanks!

-- 

  Sergei