Re: ERROR: INT DW_ATE_unsigned_1 Error emitting BTF type

[Yonghong Song <yhs@fb.com>]

On 2/5/21 10:52 PM, Sedat Dilek wrote:
> On Sat, Feb 6, 2021 at 7:26 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>>
>> On Sat, Feb 6, 2021 at 6:53 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>>>
>>> On Sat, Feb 6, 2021 at 6:44 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>>>>
>>>> On Sat, Feb 6, 2021 at 4:34 AM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>>>>>
>>>>> On Fri, Feb 5, 2021 at 10:54 PM Yonghong Song <yhs@fb.com> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 2/5/21 12:31 PM, Sedat Dilek wrote:
>>>>>>> On Fri, Feb 5, 2021 at 9:03 PM Yonghong Song <yhs@fb.com> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 2/5/21 11:24 AM, Arnaldo Carvalho de Melo wrote:
>>>>>>>>> Em Fri, Feb 05, 2021 at 11:10:08AM -0800, Yonghong Song escreveu:
>>>>>>>>>> On 2/5/21 11:06 AM, Sedat Dilek wrote:
>>>>>>>>>>> On Fri, Feb 5, 2021 at 7:53 PM Sedat Dilek <sedat.dilek@gmail.com> wrote:
>>>>>>>>>>> Grepping through linux.git/tools I guess some BTF tools/libs need to
>>>>>>>>>>> know what BTF_INT_UNSIGNED is?
>>>>>>>>>
>>>>>>>>>> BTF_INT_UNSIGNED needs kernel support. Maybe to teach pahole to
>>>>>>>>>> ignore this for now until kernel infrastructure is ready.
>>>>>>>>>
>>>>>>>>> Yeah, I thought about doing that.
>>>>>>>>>
>>>>>>>>>> Not sure whether this information will be useful or not
>>>>>>>>>> for BTF. This needs to be discussed separately.
>>>>>>>>>
>>>>>>>>> Maybe search for the rationale for its introduction in DWARF.
>>>>>>>>
>>>>>>>> In LLVM, we have:
>>>>>>>>      uint8_t BTFEncoding;
>>>>>>>>      switch (Encoding) {
>>>>>>>>      case dwarf::DW_ATE_boolean:
>>>>>>>>        BTFEncoding = BTF::INT_BOOL;
>>>>>>>>        break;
>>>>>>>>      case dwarf::DW_ATE_signed:
>>>>>>>>      case dwarf::DW_ATE_signed_char:
>>>>>>>>        BTFEncoding = BTF::INT_SIGNED;
>>>>>>>>        break;
>>>>>>>>      case dwarf::DW_ATE_unsigned:
>>>>>>>>      case dwarf::DW_ATE_unsigned_char:
>>>>>>>>        BTFEncoding = 0;
>>>>>>>>        break;
>>>>>>>>
>>>>>>>> I think DW_ATE_unsigned can be ignored in pahole since
>>>>>>>> the default encoding = 0. A simple comment is enough.
>>>>>>>>
>>>>>>>
>>>>>>> Yonghong Son, do you have a patch/diff for me?
>>>>>>
>>>>>> Looking at error message from log:
>>>>>>
>>>>>>    LLVM_OBJCOPY=/opt/binutils/bin/objcopy /opt/pahole/bin/pahole -J
>>>>>> .tmp_vmlinux.btf
>>>>>> [115] INT DW_ATE_unsigned_1 Error emitting BTF type
>>>>>> Encountered error while encoding BTF.
>>>>>>
>>>>>> Not exactly what is the root cause. Maybe bt->bit_size is not
>>>>>> encoded correctly. Could you put vmlinux (in the above it is
>>>>>> .tmp_vmlinux.btf) somewhere, I or somebody else can investigate
>>>>>> and provide a proper fix.
>>>>>>
>>>>>
>>>>> [ TO: Masahiro ]
>>>>>
>>>>> Thanks for taking care Yonghong - hope this is your first name, if not
>>>>> I am sorry.
>>>>> In case of mixing my first and last name you will make me female -
>>>>> Dilek is a Turkish female first name :-).
>>>>> So, in some cultures you need to be careful.
>>>>>
>>>>> Anyway... back to business and facts.
>>>>>
>>>>> Out of frustration I killed my last build via `make distclean`.
>>>>> The whole day I tested diverse combination of GCC-10 and LLVM-12
>>>>> together with BTF Kconfigs, selfmade pahole, etc.
>>>>>
>>>>> I will do ne run with some little changes:
>>>>>
>>>>> #1: Pass LLVM_IAS=1 to make (means use Clang's Integrated ASsembler -
>>>>> as per Nick this leads to the same error - should be unrelated)
>>>>> #2: I did: DEBUG_INFO_COMPRESSED y -> n
>>>>>
>>>>> #2 I did in case you need vmlinux and I have to upload - I will
>>>>> compress the resulting vmlinux with ZSTD.
>>>>> You need vmlinux or .tmp_vmlinux.btf file?
>>>>> Nick was not allowed from his company to download from a Dropbox link.
>>>>> So, as an alternative I can offer GoogleDrive...
>>>>> ...or bomb into your INBOX :-).
>>>>>
>>>>> Now, why I CCed Masahiro:
>>>>>
>>>>> In case of ERRORs when running `scripts/link-vmlinux.sh` above files
>>>>> will be removed.
>>>>>
>>>>> Last, I found a hack to bypass this - means to keep these files (I
>>>>> need to check old emails).
>>>>>
>>>>> Masahiro, you see a possibility to have a way to keep these files in
>>>>> case of ERRORs without doing hackery?
>>>>>
>>>>>  From a previous post in this thread:
>>>>>
>>>>> + info BTF .btf.vmlinux.bin.o
>>>>> + [  != silent_ ]
>>>>> + printf   %-7s %s\n BTF .btf.vmlinux.bin.o
>>>>>   BTF     .btf.vmlinux.bin.o
>>>>> + LLVM_OBJCOPY=llvm-objcopy /opt/pahole/bin/pahole -J .tmp_vmlinux.btf
>>>>> [2] INT long unsigned int Error emitting BTF type
>>>>> Encountered error while encoding BTF.
>>>>> + llvm-objcopy --only-section=.BTF --set-section-flags
>>>>> .BTF=alloc,readonly --strip-all .tmp_vmlinux.btf .btf.vmlinux.bin.o
>>>>> ...
>>>>> + info BTFIDS vmlinux
>>>>> + [  != silent_ ]
>>>>> + printf   %-7s %s\n BTFIDS vmlinux
>>>>>   BTFIDS  vmlinux
>>>>> + ./tools/bpf/resolve_btfids/resolve_btfids vmlinux
>>>>> FAILED: load BTF from vmlinux: Invalid argument
>>>>> + on_exit
>>>>> + [ 255 -ne 0 ]
>>>>> + cleanup
>>>>> + rm -f .btf.vmlinux.bin.o
>>>>> + rm -f .tmp_System.map
>>>>> + rm -f .tmp_vmlinux.btf .tmp_vmlinux.kallsyms1
>>>>> .tmp_vmlinux.kallsyms1.S .tmp_vmlinux.kallsyms1.o
>>>>> .tmp_vmlinux.kallsyms2 .tmp_vmlinux.kallsyms2.S .tmp_vmlinux.kallsyms
>>>>> 2.o
>>>>> + rm -f System.map
>>>>> + rm -f vmlinux
>>>>> + rm -f vmlinux.o
>>>>> make[3]: *** [Makefile:1166: vmlinux] Error 255
>>>>>
>>>>> ^^^ Look here.
>>>>>
>>>>
>>>> With this diff:
>>>>
>>>> $ git diff scripts/link-vmlinux.sh
>>>> diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
>>>> index eef40fa9485d..40f1b6aae553 100755
>>>> --- a/scripts/link-vmlinux.sh
>>>> +++ b/scripts/link-vmlinux.sh
>>>> @@ -330,7 +330,7 @@ vmlinux_link vmlinux "${kallsymso}" ${btf_vmlinux_bin_o}
>>>> # fill in BTF IDs
>>>> if [ -n "${CONFIG_DEBUG_INFO_BTF}" -a -n "${CONFIG_BPF}" ]; then
>>>>         info BTFIDS vmlinux
>>>> -       ${RESOLVE_BTFIDS} vmlinux
>>>> +       ##${RESOLVE_BTFIDS} vmlinux
>>>> fi
>>>>
>>>> if [ -n "${CONFIG_BUILDTIME_TABLE_SORT}" ]; then
>>>>
>>>> This files are kept - not removed:
>>>>
>>>> $ LC_ALL=C ll .*btf* vmlinux vmlinux.o
>>>> -rwxr-xr-x 1 dileks dileks  31M Feb  6 06:37 .btf.vmlinux.bin.o
>>>> -rwxr-xr-x 1 dileks dileks 348M Feb  6 06:37 .tmp_vmlinux.btf
>>>> -rwxr-xr-x 1 dileks dileks 348M Feb  6 06:37 vmlinux
>>>> -rw-r--r-- 1 dileks dileks 344M Feb  6 06:37 vmlinux.o
>>>>
>>>> Pleas let me know where to upload - Dropbox or GoogleDrive or
>>>> elsewhere and give me a link.
>>>>
>>>
>>>
>>> WOW, ZSTD is great :-).
>>>
>>> $ zstd -19 -T0 -v vmlinux
>>> *** zstd command line interface 64-bits v1.4.8, by Yann Collet ***
>>> Note: 2 physical core(s) detected
>>> vmlinux              : 22.71%   (364466016 => 82784801 bytes, vmlinux.zst)
>>>
>>> $ du -m vmlinux*
>>> 348     vmlinux
>>> 79      vmlinux.zst
>>>
>>
>> Dropbox link:
>> https://www.dropbox.com/sh/kvyh8ps7na0r1h5/AABfyNfDZ2bESse_bo4h05fFa?dl=0
>>
>> I hope this is public available.
>>
> 
> Inspecting vmlinux with llvm-dwarf:
> 
> $ /opt/llvm-toolchain/bin/llvm-dwarfdump vmlinux | grep DW_AT_name |
> grep DW_ATE_ | sort | uniq
>                 DW_AT_name      ("DW_ATE_signed_1")
>                 DW_AT_name      ("DW_ATE_signed_16")
>                 DW_AT_name      ("DW_ATE_signed_32")
>                 DW_AT_name      ("DW_ATE_signed_64")
>                 DW_AT_name      ("DW_ATE_signed_8")
>                 DW_AT_name      ("DW_ATE_unsigned_1")
>                 DW_AT_name      ("DW_ATE_unsigned_128")
>                 DW_AT_name      ("DW_ATE_unsigned_16")
>                 DW_AT_name      ("DW_ATE_unsigned_24")
>                 DW_AT_name      ("DW_ATE_unsigned_32")
>                 DW_AT_name      ("DW_ATE_unsigned_40")
>                 DW_AT_name      ("DW_ATE_unsigned_64")
>                 DW_AT_name      ("DW_ATE_unsigned_8")
> 
> - Sedat -

Thanks for the above dropbot link, I am able to reproduce the issue.

I tried to use latest llvm + Nick's patch + latest pahole + dwarf5 
config option to compile kernel with LLVM=1 LLVM_IAS=1, somehow
I did not hit the issue. It complained like

   MODPOST vmlinux.symvers
WARNING: modpost: vmlinux.o(.text+0x6ce73): Section mismatch in 
reference from the function __nodes
_weight() to the variable .init.data:numa_nodes_parsed
The function __nodes_weight() references
the variable __initdata numa_nodes_parsed.
This is often because __nodes_weight lacks a __initdata
annotation or the annotation of numa_nodes_parsed is wrong.

but otherwise compilation is fine.

With the above vmlinux, the issue appears to be handling
DW_ATE_signed_1, DW_ATE_unsigned_{1,24,40}.

The following patch should fix the issue:

-bash-4.4$ git diff 

diff --git a/dwarf_loader.c b/dwarf_loader.c 

index b73d786..0341b5e 100644 

--- a/dwarf_loader.c 

+++ b/dwarf_loader.c 

@@ -467,8 +467,16 @@ static struct base_type *base_type__new(Dwarf_Die 
*die, struct cu *cu)
 

         if (bt != NULL) { 

                 tag__init(&bt->tag, cu, die); 

-               bt->name = strings__add(strings, attr_string(die, 
DW_AT_name));
-               bt->bit_size = attr_numeric(die, DW_AT_byte_size) * 8; 

+               const char *name = attr_string(die, DW_AT_name); 

+               bt->name = strings__add(strings, name); 

+               /* DW_ATE_unsigned_1 has DW_AT_byte_size == 0. 

+                * specially process it. 

+                */ 

+               if (strcmp(name, "DW_ATE_unsigned_1") == 0) 

+                       bt->bit_size = 1; 

+               else 

+                       bt->bit_size = attr_numeric(die, 
DW_AT_byte_size) * 8;
+ 

                 uint64_t encoding = attr_numeric(die, DW_AT_encoding);
                 bt->is_bool = encoding == DW_ATE_boolean;
                 bt->is_signed = encoding == DW_ATE_signed;
diff --git a/libbtf.c b/libbtf.c
index 9f76283..b5aa077 100644
--- a/libbtf.c
+++ b/libbtf.c
@@ -367,13 +367,32 @@ static void btf_log_func_param(const struct 
btf_elf *btfe,
         }
  }

+/* btf requires power-of-2 bytes, yet dwarf may have something like
+ * DW_ATE_unsigned_24 which encodes as 24 bits (3 bytes).
+ */
+static int bits_to_int_bytes(uint16_t bit_size)
+{
+       if (bit_size <= 8)
+               return 1;
+       if (bit_size <= 16)
+               return 2;
+       if (bit_size <= 32)
+               return 4;
+       if (bit_size <= 64) 

+               return 8; 

+       if (bit_size <= 128)
+               return 16;
+       /* BTF supports upto 16byte int (__int128). */
+       return -1;
+}
+
  int32_t btf_elf__add_base_type(struct btf_elf *btfe, const struct 
base_type *bt,
                                const char *name)
  {
         struct btf *btf = btfe->btf;
         const struct btf_type *t;
         uint8_t encoding = 0;
-       int32_t id;
+       int32_t id, nbytes;

         if (bt->is_signed) {
                 encoding = BTF_INT_SIGNED;
@@ -384,7 +403,13 @@ int32_t btf_elf__add_base_type(struct btf_elf 
*btfe, const struct base_type *b
t,
                 return -1;
         }
-       id = btf__add_int(btf, name, BITS_ROUNDUP_BYTES(bt->bit_size), 
encoding);
+       nbytes = bits_to_int_bytes(bt->bit_size);
+       if (nbytes < 0) {
+               fprintf(stderr, "not supported bit_size %hu\n", 
bt->bit_size);
+               return -1;
+       }
+
+       id = btf__add_int(btf, name, nbytes, encoding);
         if (id < 0) {
                 btf_elf__log_err(btfe, BTF_KIND_INT, name, true, "Error 
emitting BTF type");
         } else {
-bash-4.4$

Please help do a test. I can submit a formal patch tomorrow.