Re: Encountered error while encoding BTF due to Unsupported DW_TAG_unspecified_type(0x3b)

[Arnaldo Carvalho de Melo <acme@kernel.org>]

Em Fri, Oct 07, 2022 at 05:25:21PM -0700, Yonghong Song escreveu:
> 
> 
> On 10/7/22 1:21 PM, Arnaldo Carvalho de Melo wrote:
> > Em Thu, Oct 06, 2022 at 02:23:16PM -0300, Arnaldo Carvalho de Melo escreveu:
> > > Em Thu, Oct 06, 2022 at 09:04:58AM -0700, Andrii Nakryiko escreveu:
> > > > On Thu, Oct 6, 2022 at 7:00 AM Arnaldo Carvalho de Melo <arnaldo.melo@gmail.com> wrote:
> > > > > Em Thu, Oct 06, 2022 at 10:43:22AM -0300, Arnaldo Carvalho de Melo escreveu:
> > > > > > 15e20ce2324a:~/git/linux # readelf -wi ./arch/x86/entry/entry.o
> > > > > > Contents of the .debug_info section:
> > > 
> > > > > >    Compilation Unit @ offset 0:
> > > > > >     Length:        0x35 (32-bit)
> > > > > >     Version:       5
> > > > > >     Unit Type:     DW_UT_compile (1)
> > > > > >     Abbrev Offset: 0
> > > > > >     Pointer Size:  8
> > > > > >   <0><c>: Abbrev Number: 1 (DW_TAG_compile_unit)
> > > > > >      <d>   DW_AT_stmt_list   : 0
> > > > > >      <11>   DW_AT_low_pc      : 0
> > > > > >      <19>   DW_AT_high_pc     : 19
> > > > > >      <1a>   DW_AT_name        : (indirect string, offset: 0): arch/x86/entry/entry.S
> > > > > >      <1e>   DW_AT_comp_dir    : (indirect string, offset: 0x17): /root/git/linux
> > > > > >      <22>   DW_AT_producer    : (indirect string, offset: 0x27): GNU AS 2.39.50
> > > > > >      <26>   DW_AT_language    : 32769  (MIPS assembler)
> > > > > >   <1><28>: Abbrev Number: 2 (DW_TAG_subprogram)
> > > > > >      <29>   DW_AT_name        : (indirect string, offset: 0x36): entry_ibpb
> > > > > >      <2d>   DW_AT_external    : 1
> > > > > >      <2d>   DW_AT_type        : <0x37>
> > > > > >      <2e>   DW_AT_low_pc      : 0
> > > > > >      <36>   DW_AT_high_pc     : 19
> > > > > >   <1><37>: Abbrev Number: 3 (DW_TAG_unspecified_type)
> > > > > >   <1><38>: Abbrev Number: 0
> > > 
> > > > > > 15e20ce2324a:~/git/linux #
> > > 
> > > > > > Which pahole -J barfs on:
> > > 
> > > > > > 15e20ce2324a:~/git/linux # pahole -J ./arch/x86/entry/entry.o
> > > > > > [1] UNKNOWN (anon) Unexpected kind for reference
> > > > > > 15e20ce2324a:~/git/linux #
> > > 
> > > > > > But if we ask it to exclude asm CUs (<26>   DW_AT_language    : 32769
> > > > > > (MIPS assembler)) then it ignores it, so this is a workaround.
> > > 
> > > > > > 15e20ce2324a:~/git/linux # pahole --lang_exclude asm -V -J ./arch/x86/entry/entry.o
> > > > > > 15e20ce2324a:~/git/linux #
> > > 
> > > > > > Now I'm looking at how to get the BTF encoder grokking this.
> > > 
> > > > > This is what I came up with, Andrii, Yonghong, wdyt?
> > > > Does `const void` make sense? Why not just keeping it as "void"?
> > > > "const void" might confuse tooling and BTF verifier in kernel, but I
> > > > haven't checked. Just trying to understand why we need extra "const".
> > > 
> > > I was trying to have an entry for that DW_TAG_unspecified_type type
> > > instead of plain changing the type for entry_ibpb (a DW_TAG_subprogram)
> > > straight to 0 (void).
> > 
> > I think this is the best way forward at the moment, i.e. to just change
> > the type for such DW_TAG_subprogram to zero, aka void, losing this
> > information by default.
> > 
> > I'll have a --btf_encode_unspecified_type as this const void trick as an
> > opt-in, who knows if this ends up being useful :-)
> 
> For function entry_ibpb, actually the kernel has a prorotype,
> 
> include/asm/nospec-branch.h:extern void entry_ibpb(void);

> But unfortunately since the function is defined in asm code.
> The actual type information is not encoded in dwarf so pahole
> does not have enough info from vmlinux dwarf to get func
> types. The compiler does not generate func types based on
> declarations.

Right
 
> This may prevent people from tracing functions defined in
> asm code but this should be extremely rare.
> So I agree that BTF type id 0 is probably the best choice.

Thanks for your comments, that is the way I'll do it.
 
> > - Arnaldo
> > > I.e. we wouldn't lose that info that the return type for that "function"
> > > (it was encoded as a DW_TAG_subprogram) isn't defined. Using straight
> > > void would state that it doesn't return anything.
> > > 
> > > I couldn't find a way to use KCFLAGS to ask Kbuild to pass that -B flag
> > > to gcc so that I could rebuild the whole kernel with it to check if the
> > > build goes thru and then try booting with such BTF info.
> > > 
> > > - Arnaldo
> > > > > As an additional info clang 14 (haven't tested with other versions)
> > > > > encodes such ASM Labels as DW_TAG_label and this thus isn't an issue
> > > > > there.
> > > > > 
> > > > > - Arnaldo
> > > > > 
> > > > > commit 15ec614672da043008df31aa6ee85ebc5105d4fd
> > > > > Author: Arnaldo Carvalho de Melo <acme@redhat.com>
> > > > > Date:   Tue Oct 4 18:22:53 2022 -0300
> > > > > 
> > > > >      btf_encoder: Encode DW_TAG_unspecified_type as BTF_KIND_CONST
> > > > > 
> > > > >      This first appeared for assembler files in the Linux kernel with recent
> > > > >      GNU compilers, we don't have anything in BTF, AFAIK, to properly
> > > > >      represent that, so, for now, lets go with BTF_KIND_CONST.
> > > > > 
> > > > >      Testing it:
> > > > > 
> > > > >      Built binutils from git://sourceware.org/git/binutils-gdb.git, then used
> > > > >      gcc's -B option to point to the directory with the new as, that is built
> > > > >      as as-new, so make a symlink, ending up with:
> > > > > 
> > > > >        15e20ce2324a:~/git/linux # readelf -wi ./arch/x86/entry/entry.o
> > > > >        Contents of the .debug_info section:
> > > > > 
> > > > >          Compilation Unit @ offset 0:
> > > > >           Length:        0x35 (32-bit)
> > > > >           Version:       5
> > > > >           Unit Type:     DW_UT_compile (1)
> > > > >           Abbrev Offset: 0
> > > > >           Pointer Size:  8
> > > > >         <0><c>: Abbrev Number: 1 (DW_TAG_compile_unit)
> > > > >            <d>   DW_AT_stmt_list   : 0
> > > > >            <11>   DW_AT_low_pc      : 0
> > > > >            <19>   DW_AT_high_pc     : 19
> > > > >            <1a>   DW_AT_name        : (indirect string, offset: 0): arch/x86/entry/entry.S
> > > > >            <1e>   DW_AT_comp_dir    : (indirect string, offset: 0x17): /root/git/linux
> > > > >            <22>   DW_AT_producer    : (indirect string, offset: 0x27): GNU AS 2.39.50
> > > > >            <26>   DW_AT_language    : 32769  (MIPS assembler)
> > > > >         <1><28>: Abbrev Number: 2 (DW_TAG_subprogram)
> > > > >            <29>   DW_AT_name        : (indirect string, offset: 0x36): entry_ibpb
> > > > >            <2d>   DW_AT_external    : 1
> > > > >            <2d>   DW_AT_type        : <0x37>
> > > > >            <2e>   DW_AT_low_pc      : 0
> > > > >            <36>   DW_AT_high_pc     : 19
> > > > >         <1><37>: Abbrev Number: 3 (DW_TAG_unspecified_type)
> > > > >         <1><38>: Abbrev Number: 0
> > > > > 
> > > > >      So we have that asm label encoded by GNU AS 2.39.50 as a
> > > > >      DW_TAG_subprogram that has as its DW_AT_type the DW_TAG_unspecified_type
> > > > >      0x37 that we encode as a BTF_KIND_CONST pointing to 0 (void):
> > > > > 
> > > > >        15e20ce2324a:~/git/linux # pahole -J ./arch/x86/entry/entry.o
> > > > >        15e20ce2324a:~/git/linux # pahole -JV ./arch/x86/entry/entry.o
> > > > >        btf_encoder__new: './arch/x86/entry/entry.o' doesn't have '.data..percpu' section
> > > > >        Found 0 per-CPU variables!
> > > > >        Found 1 functions!
> > > > >        File ./arch/x86/entry/entry.o:
> > > > >        [1] CONST (anon) type_id=0
> > > > >        [2] FUNC_PROTO (anon) return=1 args=(void)
> > > > >        [3] FUNC entry_ibpb type_id=2
> > > > >        15e20ce2324a:~/git/linux # pfunct -F btf ./arch/x86/entry/entry.o
> > > > >        entry_ibpb
> > > > >        15e20ce2324a:~/git/linux # pfunct --proto -F btf ./arch/x86/entry/entry.o
> > > > >        const void  entry_ibpb(void);
> > > > >        15e20ce2324a:~/git/linux #
> > > > > 
> > > > >        15e20ce2324a:~/git/linux # tools/bpf/bpftool/bpftool btf dump file ./arch/x86/entry/entry.o format raw
> > > > >        [1] CONST '(anon)' type_id=0
> > > > >        [2] FUNC_PROTO '(anon)' ret_type_id=1 vlen=0
> > > > >        [3] FUNC 'entry_ibpb' type_id=2 linkage=static
> > > > >        15e20ce2324a:~/git/linux #
> > > > > 
> > > > >      I think this is what can be done to avoid having to skip ASM DWARF when
> > > > >      gets widely used, i.e. binutils gets updated.
> > > > > 
> > > > >      Reported-by: Martin Liška <mliska@suse.cz>
> > > > >      Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > > > >      Cc: Yonghong Song <yhs@fb.com>
> > > > >      Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
> > > > > 
> > > > > diff --git a/btf_encoder.c b/btf_encoder.c
> > > > > index 7ad3f29ef153d8d6..7e50ba711ccc14d6 100644
> > > > > --- a/btf_encoder.c
> > > > > +++ b/btf_encoder.c
> > > > > @@ -962,6 +962,8 @@ static int btf_encoder__encode_tag(struct btf_encoder *encoder, struct tag *tag,
> > > > >                  return btf_encoder__add_enum_type(encoder, tag, conf_load);
> > > > >          case DW_TAG_subroutine_type:
> > > > >                  return btf_encoder__add_func_proto(encoder, tag__ftype(tag), type_id_off);
> > > > > +       case DW_TAG_unspecified_type:
> > > > > +               return btf_encoder__add_ref_type(encoder, BTF_KIND_CONST, 0, NULL, false);
> > > > >          default:
> > > > >                  fprintf(stderr, "Unsupported DW_TAG_%s(0x%x): type: 0x%x\n",
> > > > >                          dwarf_tag_name(tag->tag), tag->tag, ref_type_id);
> > > 
> > > -- 
> > > 
> > > - Arnaldo
> > 

-- 

- Arnaldo