[PATCH dwarves] btf_encoder: Filter out __gendwarfksyms_ptr_

[PATCH dwarves] btf_encoder: Filter out __gendwarfksyms_ptr_

[Sami Tolvanen]

With CONFIG_GENDWARFKSYMS, __gendwarfksyms_ptr_<symbol>
variables are added to the kernel in EXPORT_SYMBOL() to ensure
DWARF type information is available for exported symbols in the
TUs where they're actually exported. These symbols are dropped
when linking vmlinux, but dangling references to them remain
in DWARF, which results in thousands of 0 address variables
that pahole needs to validate (since commit 9810758003ce
("btf_encoder: Verify 0 address DWARF variables are in ELF
section")).

Filter out symbols with the __gendwarfksyms_ptr_ name prefix in
filter_variable_name() instead of calling variable_in_sec()
for all of them. This reduces the time it takes to process
.tmp_vmlinux1 by ~77% on my test system:

Before: 35.775 +- 0.121  seconds time elapsed  ( +-  0.34% )
 After: 8.3516 +- 0.0407 seconds time elapsed  ( +-  0.49% )

Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
---
 btf_encoder.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/btf_encoder.c b/btf_encoder.c
index 1bde310..2bf7c59 100644
--- a/btf_encoder.c
+++ b/btf_encoder.c
@@ -2239,6 +2239,7 @@ static bool filter_variable_name(const char *name)
 		X("__UNIQUE_ID"),
 		X("__tpstrtab_"),
 		X("__exitcall_"),
+		X("__gendwarfksyms_ptr_"),
 		X("__func_stack_frame_non_standard_")
 		#undef X
 	};

base-commit: a0be596ae76c720d21eef257dec1cf2462130da1
-- 
2.49.0.rc1.451.g8f38331e32-goog

Re: [PATCH dwarves] btf_encoder: Filter out __gendwarfksyms_ptr_

[Jiri Olsa]

On Mon, Mar 17, 2025 at 10:24:24PM +0000, Sami Tolvanen wrote:
> With CONFIG_GENDWARFKSYMS, __gendwarfksyms_ptr_<symbol>
> variables are added to the kernel in EXPORT_SYMBOL() to ensure
> DWARF type information is available for exported symbols in the
> TUs where they're actually exported. These symbols are dropped
> when linking vmlinux, but dangling references to them remain
> in DWARF, which results in thousands of 0 address variables
> that pahole needs to validate (since commit 9810758003ce
> ("btf_encoder: Verify 0 address DWARF variables are in ELF
> section")).
> 
> Filter out symbols with the __gendwarfksyms_ptr_ name prefix in
> filter_variable_name() instead of calling variable_in_sec()
> for all of them. This reduces the time it takes to process
> .tmp_vmlinux1 by ~77% on my test system:
> 
> Before: 35.775 +- 0.121  seconds time elapsed  ( +-  0.34% )
>  After: 8.3516 +- 0.0407 seconds time elapsed  ( +-  0.49% )

makes sense to me, I just can't reproduce the speedup
could you please share your .config?

thanks,
jirka


> 
> Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
> ---
>  btf_encoder.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/btf_encoder.c b/btf_encoder.c
> index 1bde310..2bf7c59 100644
> --- a/btf_encoder.c
> +++ b/btf_encoder.c
> @@ -2239,6 +2239,7 @@ static bool filter_variable_name(const char *name)
>  		X("__UNIQUE_ID"),
>  		X("__tpstrtab_"),
>  		X("__exitcall_"),
> +		X("__gendwarfksyms_ptr_"),
>  		X("__func_stack_frame_non_standard_")
>  		#undef X
>  	};
> 
> base-commit: a0be596ae76c720d21eef257dec1cf2462130da1
> -- 
> 2.49.0.rc1.451.g8f38331e32-goog
> 

Re: [PATCH dwarves] btf_encoder: Filter out __gendwarfksyms_ptr_

[Sami Tolvanen]

Hi Jiri,

On Tue, Mar 18, 2025 at 9:52 AM Jiri Olsa <olsajiri@gmail.com> wrote:
>
> On Mon, Mar 17, 2025 at 10:24:24PM +0000, Sami Tolvanen wrote:
> > With CONFIG_GENDWARFKSYMS, __gendwarfksyms_ptr_<symbol>
> > variables are added to the kernel in EXPORT_SYMBOL() to ensure
> > DWARF type information is available for exported symbols in the
> > TUs where they're actually exported. These symbols are dropped
> > when linking vmlinux, but dangling references to them remain
> > in DWARF, which results in thousands of 0 address variables
> > that pahole needs to validate (since commit 9810758003ce
> > ("btf_encoder: Verify 0 address DWARF variables are in ELF
> > section")).
> >
> > Filter out symbols with the __gendwarfksyms_ptr_ name prefix in
> > filter_variable_name() instead of calling variable_in_sec()
> > for all of them. This reduces the time it takes to process
> > .tmp_vmlinux1 by ~77% on my test system:
> >
> > Before: 35.775 +- 0.121  seconds time elapsed  ( +-  0.34% )
> >  After: 8.3516 +- 0.0407 seconds time elapsed  ( +-  0.49% )
>
> makes sense to me, I just can't reproduce the speedup
> could you please share your .config?

Sure, here's the config I used to repro this:

https://gist.github.com/samitolvanen/dca66a1a779861be27579f88c9b6ba5d

This is essentially x86_64 defconfig with GENDWARFKSYMS and
DEBUG_INFO_BTF both enabled. When building this config with gcc, we
end up with 0 address __gendwarfksyms_ptr variables in DWARF:

...
0x0001b5c6:   DW_TAG_variable
                DW_AT_name      ("__gendwarfksyms_ptr_system_state")
                DW_AT_decl_file ("../init/main.c")
                DW_AT_decl_line (129)
                DW_AT_decl_column       (1)
                DW_AT_type      (0x0001b5dc "system_states *")
                DW_AT_location  (DW_OP_addr 0x0)
...

Note that this doesn't seem to happen when building with Clang.

Before commit 9810758003ce this resulted in pahole thinking all these
variables are in the .data..percpu section, which resulted in
btf_datasec_check_meta() failing with "Invalid offset" during boot.
pahole/next doesn't have this issue, but validating the 0 address
variables is unfortunately a bit slow when we have a lot of them.

Sami

Re: [PATCH dwarves] btf_encoder: Filter out __gendwarfksyms_ptr_

[Alan Maguire]

On 18/03/2025 16:14, Sami Tolvanen wrote:
> Hi Jiri,
> 
> On Tue, Mar 18, 2025 at 9:52 AM Jiri Olsa <olsajiri@gmail.com> wrote:
>>
>> On Mon, Mar 17, 2025 at 10:24:24PM +0000, Sami Tolvanen wrote:
>>> With CONFIG_GENDWARFKSYMS, __gendwarfksyms_ptr_<symbol>
>>> variables are added to the kernel in EXPORT_SYMBOL() to ensure
>>> DWARF type information is available for exported symbols in the
>>> TUs where they're actually exported. These symbols are dropped
>>> when linking vmlinux, but dangling references to them remain
>>> in DWARF, which results in thousands of 0 address variables
>>> that pahole needs to validate (since commit 9810758003ce
>>> ("btf_encoder: Verify 0 address DWARF variables are in ELF
>>> section")).
>>>
>>> Filter out symbols with the __gendwarfksyms_ptr_ name prefix in
>>> filter_variable_name() instead of calling variable_in_sec()
>>> for all of them. This reduces the time it takes to process
>>> .tmp_vmlinux1 by ~77% on my test system:
>>>
>>> Before: 35.775 +- 0.121  seconds time elapsed  ( +-  0.34% )
>>>  After: 8.3516 +- 0.0407 seconds time elapsed  ( +-  0.49% )
>>
>> makes sense to me, I just can't reproduce the speedup
>> could you please share your .config?
> 
> Sure, here's the config I used to repro this:
> 
> https://gist.github.com/samitolvanen/dca66a1a779861be27579f88c9b6ba5d
> 
> This is essentially x86_64 defconfig with GENDWARFKSYMS and
> DEBUG_INFO_BTF both enabled. When building this config with gcc, we
> end up with 0 address __gendwarfksyms_ptr variables in DWARF:
> 
> ...
> 0x0001b5c6:   DW_TAG_variable
>                 DW_AT_name      ("__gendwarfksyms_ptr_system_state")
>                 DW_AT_decl_file ("../init/main.c")
>                 DW_AT_decl_line (129)
>                 DW_AT_decl_column       (1)
>                 DW_AT_type      (0x0001b5dc "system_states *")
>                 DW_AT_location  (DW_OP_addr 0x0)
> ...
> 
> Note that this doesn't seem to happen when building with Clang.
> 
> Before commit 9810758003ce this resulted in pahole thinking all these
> variables are in the .data..percpu section, which resulted in
> btf_datasec_check_meta() failing with "Invalid offset" during boot.
> pahole/next doesn't have this issue, but validating the 0 address
> variables is unfortunately a bit slow when we have a lot of them.
>

Thanks for the fix Sami! I've tested it at my end and can reproduce the
longer time for BTF encoding on x86_64 prior to the fix and its
resolution. Let's wait a bit longer before landing it to see if anyone
else gets a chance to test/ack it, but I think we should probably also add a

Fixes: 9810758003ce9f ("btf_encoder: Verify 0 address DWARF variables
are in ELF section")

(no need to resend for this; I can add it when committing it)

I'm thinking we should also try and incorporate some performance tests
for vmlinux BTF encoding into the tests subdirectory to better catch
issues like this; perhaps the CI can baseline encoding performance on
the next branch versus the branch that has the changes..

Thanks again!

Alan
> Sami

Re: [PATCH dwarves] btf_encoder: Filter out __gendwarfksyms_ptr_

[Jiri Olsa]

On Thu, Mar 20, 2025 at 09:54:21AM +0000, Alan Maguire wrote:
> On 18/03/2025 16:14, Sami Tolvanen wrote:
> > Hi Jiri,
> > 
> > On Tue, Mar 18, 2025 at 9:52 AM Jiri Olsa <olsajiri@gmail.com> wrote:
> >>
> >> On Mon, Mar 17, 2025 at 10:24:24PM +0000, Sami Tolvanen wrote:
> >>> With CONFIG_GENDWARFKSYMS, __gendwarfksyms_ptr_<symbol>
> >>> variables are added to the kernel in EXPORT_SYMBOL() to ensure
> >>> DWARF type information is available for exported symbols in the
> >>> TUs where they're actually exported. These symbols are dropped
> >>> when linking vmlinux, but dangling references to them remain
> >>> in DWARF, which results in thousands of 0 address variables
> >>> that pahole needs to validate (since commit 9810758003ce
> >>> ("btf_encoder: Verify 0 address DWARF variables are in ELF
> >>> section")).
> >>>
> >>> Filter out symbols with the __gendwarfksyms_ptr_ name prefix in
> >>> filter_variable_name() instead of calling variable_in_sec()
> >>> for all of them. This reduces the time it takes to process
> >>> .tmp_vmlinux1 by ~77% on my test system:
> >>>
> >>> Before: 35.775 +- 0.121  seconds time elapsed  ( +-  0.34% )
> >>>  After: 8.3516 +- 0.0407 seconds time elapsed  ( +-  0.49% )
> >>
> >> makes sense to me, I just can't reproduce the speedup
> >> could you please share your .config?
> > 
> > Sure, here's the config I used to repro this:
> > 
> > https://gist.github.com/samitolvanen/dca66a1a779861be27579f88c9b6ba5d
> > 
> > This is essentially x86_64 defconfig with GENDWARFKSYMS and
> > DEBUG_INFO_BTF both enabled. When building this config with gcc, we
> > end up with 0 address __gendwarfksyms_ptr variables in DWARF:
> > 
> > ...
> > 0x0001b5c6:   DW_TAG_variable
> >                 DW_AT_name      ("__gendwarfksyms_ptr_system_state")
> >                 DW_AT_decl_file ("../init/main.c")
> >                 DW_AT_decl_line (129)
> >                 DW_AT_decl_column       (1)
> >                 DW_AT_type      (0x0001b5dc "system_states *")
> >                 DW_AT_location  (DW_OP_addr 0x0)
> > ...
> > 
> > Note that this doesn't seem to happen when building with Clang.
> > 
> > Before commit 9810758003ce this resulted in pahole thinking all these
> > variables are in the .data..percpu section, which resulted in
> > btf_datasec_check_meta() failing with "Invalid offset" during boot.
> > pahole/next doesn't have this issue, but validating the 0 address
> > variables is unfortunately a bit slow when we have a lot of them.
> >
> 
> Thanks for the fix Sami! I've tested it at my end and can reproduce the
> longer time for BTF encoding on x86_64 prior to the fix and its
> resolution. Let's wait a bit longer before landing it to see if anyone
> else gets a chance to test/ack it, but I think we should probably also add a
> 
> Fixes: 9810758003ce9f ("btf_encoder: Verify 0 address DWARF variables
> are in ELF section")

+1 

Acked-by: Jiri Olsa <jolsa@kernel.org>

thanks,
jirka


> 
> (no need to resend for this; I can add it when committing it)
> 
> I'm thinking we should also try and incorporate some performance tests
> for vmlinux BTF encoding into the tests subdirectory to better catch
> issues like this; perhaps the CI can baseline encoding performance on
> the next branch versus the branch that has the changes..
> 
> Thanks again!
> 
> Alan
> > Sami
> 

Re: [PATCH dwarves] btf_encoder: Filter out __gendwarfksyms_ptr_

[Alan Maguire]

On 17/03/2025 22:24, Sami Tolvanen wrote:
> With CONFIG_GENDWARFKSYMS, __gendwarfksyms_ptr_<symbol>
> variables are added to the kernel in EXPORT_SYMBOL() to ensure
> DWARF type information is available for exported symbols in the
> TUs where they're actually exported. These symbols are dropped
> when linking vmlinux, but dangling references to them remain
> in DWARF, which results in thousands of 0 address variables
> that pahole needs to validate (since commit 9810758003ce
> ("btf_encoder: Verify 0 address DWARF variables are in ELF
> section")).
> 
> Filter out symbols with the __gendwarfksyms_ptr_ name prefix in
> filter_variable_name() instead of calling variable_in_sec()
> for all of them. This reduces the time it takes to process
> .tmp_vmlinux1 by ~77% on my test system:
> 
> Before: 35.775 +- 0.121  seconds time elapsed  ( +-  0.34% )
>  After: 8.3516 +- 0.0407 seconds time elapsed  ( +-  0.49% )
> 
> Signed-off-by: Sami Tolvanen <samitolvanen@google.com>


applied to the next branch of

https://git.kernel.org/pub/scm/devel/pahole/pahole.git/

thanks!

> ---
>  btf_encoder.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/btf_encoder.c b/btf_encoder.c
> index 1bde310..2bf7c59 100644
> --- a/btf_encoder.c
> +++ b/btf_encoder.c
> @@ -2239,6 +2239,7 @@ static bool filter_variable_name(const char *name)
>  		X("__UNIQUE_ID"),
>  		X("__tpstrtab_"),
>  		X("__exitcall_"),
> +		X("__gendwarfksyms_ptr_"),
>  		X("__func_stack_frame_non_standard_")
>  		#undef X
>  	};
> 
> base-commit: a0be596ae76c720d21eef257dec1cf2462130da1