[PATCH v2] pahole: Apply CU-level filters early in loading

[PATCH v2] pahole: Apply CU-level filters early in loading

[Matthew Maurer]

Without this, even with `--lang_exclude=rust` set, running on `vmlinux`
with `CONFIG_RUST` enabled will lead to errors like:
die__process_function: tag not supported 0x2f (template_type_parameter)!
because the filtering doesn't happen until finalization, but unsupported
tags are reported during loading.

As an added bonus, this should speed up processing of large objects with
filtered CUs, as their details will no longer be walked.

Signed-off-by: Matthew Maurer <mmaurer@google.com>
---
 dwarf_loader.c | 10 ++++++++++
 dwarves.h      |  1 +
 pahole.c       |  4 ++++
 3 files changed, 15 insertions(+)

diff --git a/dwarf_loader.c b/dwarf_loader.c
index b832c93..02a17f4 100644
--- a/dwarf_loader.c
+++ b/dwarf_loader.c
@@ -2854,6 +2854,16 @@ static int die__process(Dwarf_Die *die, struct cu *cu, struct conf_load *conf)
 
 	cu->language = attr_numeric(die, DW_AT_language);
 
+	if (conf->early_cu_filter)
+		cu = conf->early_cu_filter(cu);
+
+	/*
+	 * If we filtered this CU out, we still want to keep iterating, but
+	 * there's no need to walk the rest of the CU info.
+	 */
+	if (cu == NULL)
+		return DWARF_CB_OK;
+
 	if (dwarf_child(die, &child) == 0) {
 		int err = die__process_unit(&child, cu, conf);
 		if (err)
diff --git a/dwarves.h b/dwarves.h
index f5ae79f..92d102b 100644
--- a/dwarves.h
+++ b/dwarves.h
@@ -72,6 +72,7 @@ struct conf_load {
 	enum load_steal_kind	(*steal)(struct cu *cu,
 					 struct conf_load *conf,
 					 void *thr_data);
+	struct cu *		(*early_cu_filter)(struct cu *cu);
 	int			(*thread_exit)(struct conf_load *conf, void *thr_data);
 	void			*cookie;
 	char			*format_path;
diff --git a/pahole.c b/pahole.c
index 954498d..42a9c16 100644
--- a/pahole.c
+++ b/pahole.c
@@ -3765,6 +3765,10 @@ int main(int argc, char *argv[])
 	memset(tab, ' ', sizeof(tab) - 1);
 
 	conf_load.steal = pahole_stealer;
+
+	if (languages.exclude)
+		conf_load.early_cu_filter = cu__filter;
+
 	conf_load.thread_exit = pahole_thread_exit;
 
 	if (conf_load.reproducible_build) {
-- 
2.46.0.rc2.264.g509ed76dc8-goog

Re: [PATCH v2] pahole: Apply CU-level filters early in loading

[Alan Maguire]

On 01/08/2024 19:50, Matthew Maurer wrote:
> Without this, even with `--lang_exclude=rust` set, running on `vmlinux`
> with `CONFIG_RUST` enabled will lead to errors like:
> die__process_function: tag not supported 0x2f (template_type_parameter)!
> because the filtering doesn't happen until finalization, but unsupported
> tags are reported during loading.
> 
> As an added bonus, this should speed up processing of large objects with
> filtered CUs, as their details will no longer be walked.
> 
> Signed-off-by: Matthew Maurer <mmaurer@google.com>

LGTM, thanks!

Reviewed-by: Alan Maguire <alan.maguire@oracle.com>
Tested-by: Alan Maguire <alan.maguire@oracle.com>

> ---
>  dwarf_loader.c | 10 ++++++++++
>  dwarves.h      |  1 +
>  pahole.c       |  4 ++++
>  3 files changed, 15 insertions(+)
> 
> diff --git a/dwarf_loader.c b/dwarf_loader.c
> index b832c93..02a17f4 100644
> --- a/dwarf_loader.c
> +++ b/dwarf_loader.c
> @@ -2854,6 +2854,16 @@ static int die__process(Dwarf_Die *die, struct cu *cu, struct conf_load *conf)
>  
>  	cu->language = attr_numeric(die, DW_AT_language);
>  
> +	if (conf->early_cu_filter)
> +		cu = conf->early_cu_filter(cu);
> +
> +	/*
> +	 * If we filtered this CU out, we still want to keep iterating, but
> +	 * there's no need to walk the rest of the CU info.
> +	 */
> +	if (cu == NULL)
> +		return DWARF_CB_OK;
> +
>  	if (dwarf_child(die, &child) == 0) {
>  		int err = die__process_unit(&child, cu, conf);
>  		if (err)
> diff --git a/dwarves.h b/dwarves.h
> index f5ae79f..92d102b 100644
> --- a/dwarves.h
> +++ b/dwarves.h
> @@ -72,6 +72,7 @@ struct conf_load {
>  	enum load_steal_kind	(*steal)(struct cu *cu,
>  					 struct conf_load *conf,
>  					 void *thr_data);
> +	struct cu *		(*early_cu_filter)(struct cu *cu);
>  	int			(*thread_exit)(struct conf_load *conf, void *thr_data);
>  	void			*cookie;
>  	char			*format_path;
> diff --git a/pahole.c b/pahole.c
> index 954498d..42a9c16 100644
> --- a/pahole.c
> +++ b/pahole.c
> @@ -3765,6 +3765,10 @@ int main(int argc, char *argv[])
>  	memset(tab, ' ', sizeof(tab) - 1);
>  
>  	conf_load.steal = pahole_stealer;
> +
> +	if (languages.exclude)
> +		conf_load.early_cu_filter = cu__filter;
> +
>  	conf_load.thread_exit = pahole_thread_exit;
>  
>  	if (conf_load.reproducible_build) {

Re: [PATCH v2] pahole: Apply CU-level filters early in loading

[Arnaldo Carvalho de Melo]

On Tue, Aug 06, 2024 at 01:10:03PM +0100, Alan Maguire wrote:
> On 01/08/2024 19:50, Matthew Maurer wrote:
> > Without this, even with `--lang_exclude=rust` set, running on `vmlinux`
> > with `CONFIG_RUST` enabled will lead to errors like:
> > die__process_function: tag not supported 0x2f (template_type_parameter)!
> > because the filtering doesn't happen until finalization, but unsupported
> > tags are reported during loading.
> > 
> > As an added bonus, this should speed up processing of large objects with
> > filtered CUs, as their details will no longer be walked.
> > 
> > Signed-off-by: Matthew Maurer <mmaurer@google.com>
> 
> LGTM, thanks!
> 
> Reviewed-by: Alan Maguire <alan.maguire@oracle.com>
> Tested-by: Alan Maguire <alan.maguire@oracle.com>

Thanks, I just split it into two patches, one adding the new early
filtering facility and with the second making then use of this new
facility in pahole:

⬢[acme@toolbox pahole]$ git log --oneline -5
fbcca0f6c2fbeb52 (HEAD -> master) pahole: Do --languages_exclude CU filtering earlier
1f8d83d16d6d6dfa dwarf_loader: Allow filtering CUs  early in loading
14be4e3d3fe56ce9 pahole: Only warn about multithreading not being available with older versions of elfutils in verbose mode
ee933f35484ac934 dwarf_loader: Simplify the tag__alloc() routine
cbecc3785266f0c5 (x1/master) dwarf_loader: Do just one alloc for 'struct dwarf_tag + struct tag'
⬢[acme@toolbox pahole]$ git show
commit fbcca0f6c2fbeb52349e28f03093a7efafb040ea (HEAD -> master)
Author: Matthew Maurer <mmaurer@google.com>
Date:   Thu Aug 1 18:50:54 2024 +0000

    pahole: Do --languages_exclude CU filtering earlier
    
    With this, we can avoid warnings for unsupported DWARF tags like:
    
      die__process_function: tag not supported 0x2f (template_type_parameter)!
    
    when processing object files generated from languages such as Rust, for
    instance when building the Linux kernel with `CONFIG_RUST`, after
    appltying the next patch in this series.
    
    As an added bonus, this should speed up processing of large objects with
    filtered CUs, as their details will no longer be walked.
    
    Reviewed-by: Alan Maguire <alan.maguire@oracle.com>
    Signed-off-by: Matthew Maurer <mmaurer@google.com>
    Tested-by: Alan Maguire <alan.maguire@oracle.com>
    Cc: Alice Ryhl <aliceryhl@google.com>
    Cc: rust-for-linux@vger.kernel.org
    Link: https://lore.kernel.org/r/20240801185054.2518383-1-mmaurer@google.com
    [ Split from a larger patch ]
    Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>

diff --git a/pahole.c b/pahole.c
index 4db51071b97d5f09..42ef6c262384a8e0 100644
--- a/pahole.c
+++ b/pahole.c
@@ -3766,6 +3766,10 @@ int main(int argc, char *argv[])
        memset(tab, ' ', sizeof(tab) - 1);
 
        conf_load.steal = pahole_stealer;
+
+       if (languages.exclude)
+               conf_load.early_cu_filter = cu__filter;
+
        conf_load.thread_exit = pahole_thread_exit;
 
        if (conf_load.reproducible_build) {
⬢[acme@toolbox pahole]$

Re: [PATCH v2] pahole: Apply CU-level filters early in loading

[Arnaldo Carvalho de Melo]

On Fri, Aug 09, 2024 at 09:45:13AM -0300, Arnaldo Carvalho de Melo wrote:
> On Tue, Aug 06, 2024 at 01:10:03PM +0100, Alan Maguire wrote:
> > On 01/08/2024 19:50, Matthew Maurer wrote:
> > > Without this, even with `--lang_exclude=rust` set, running on `vmlinux`
> > > with `CONFIG_RUST` enabled will lead to errors like:
> > > die__process_function: tag not supported 0x2f (template_type_parameter)!
> > > because the filtering doesn't happen until finalization, but unsupported
> > > tags are reported during loading.
> > > 
> > > As an added bonus, this should speed up processing of large objects with
> > > filtered CUs, as their details will no longer be walked.
> > > 
> > > Signed-off-by: Matthew Maurer <mmaurer@google.com>
> > 
> > LGTM, thanks!
> > 
> > Reviewed-by: Alan Maguire <alan.maguire@oracle.com>
> > Tested-by: Alan Maguire <alan.maguire@oracle.com>
> 
> Thanks, I just split it into two patches, one adding the new early
> filtering facility and with the second making then use of this new
> facility in pahole:
> 
> ⬢[acme@toolbox pahole]$ git log --oneline -5
> fbcca0f6c2fbeb52 (HEAD -> master) pahole: Do --languages_exclude CU filtering earlier
> 1f8d83d16d6d6dfa dwarf_loader: Allow filtering CUs  early in loading

Tested building a kernel with CONFIG_RUST, no warnings (that
template_type_param wouldn't appear anyway as I added initial support
for it, at least for creating some structs out of its DWARF info, so no
more warnings), but:

⬢[acme@toolbox linux]$ pahole -C '<kernel::str::RawFormatter as core::fmt::Write>::{vtable_type}' ../build/rust-kernel/rust/kernel.o 
die__process_class: tag not supported 0x33 (variant_part) at <14f43>!
struct <kernel::str::RawFormatter as core::fmt::Write>::{vtable_type} {
	() *                       drop_in_place __attribute__((__aligned__(8))); /*     0     8 */
	usize                      size __attribute__((__aligned__(8))); /*     8     8 */
	usize                      align __attribute__((__aligned__(8))); /*    16     8 */
	() *                       __method3 __attribute__((__aligned__(8))); /*    24     8 */
	() *                       __method4 __attribute__((__aligned__(8))); /*    32     8 */
	() *                       __method5 __attribute__((__aligned__(8))); /*    40     8 */

	/* size: 48, cachelines: 1, members: 6 */
	/* forced alignments: 6 */
	/* last cacheline: 48 bytes */
} __attribute__((__aligned__(8)));

⬢[acme@toolbox linux]$

That DW_TAG_variant_part warning would show up without this series, so
seems to be working as expected :-)

- Arnaldo