[RFC dwarves 0/6] prep for compiler-generated BTF

[RFC dwarves 0/6] prep for compiler-generated BTF

[Alan Maguire]

This series is intended to help pave the way to allow pahole to
handle compiler-generated BTF as input to its encoding process.

The BTF loader currently supports loading BTF for data structure
and function query but not for BTF generation.  [1] describes how
gcc can generate (and the linker deduplicate) BTF but even there
we will need pahole to post-process the results so it is valuable
to support reading BTF in pahole as input to start exploring how
to handle cases like [1].

The intention of this series is not yet to fully support
linker-deduplicated BTF, but to start down that road by supporting
reading BTF from (multiple) object files and support its deduplication
and BTF output.  To that end a few changes are needed in the
intermediate representation generated by the BTF loader; these are
done in patch 1.  Patch 2 then fixes up a few cases where we needed
to special-case the intermediate representation when it came from
BTF.  Patches 3/4 ensure that we retrieve ELF info when loading BTF
for encoding.  Patch 5 is intended to support cases where we read BTF from
multiple objects and - if the "encode_force" BTF feature is specified -
we continue as long as BTF is found in at least one of those sources.
Finally patch 6 tests pahole when BTF is used as input.

This allows us to experiment with "gcc -gbtf".  For example adding
-gbtf to the KCFLAGS environment variable for a kernel build, we can
then run pahole across the set of vmlinux objects via

pahole --format_path=btf -J --btf_features=encode_force \
        --btf_encode_detached=/tmp/vmlinux.btf $(ar t vmlinux.a)

This is clearly not the right way to do this for kernel builds -
we want to make use of linker deduplication as in [1] - but the fact
that even a hack like this works does demonstrate the viability of
handling BTF as input to vmlinux BTF encoding.  It also allows us
to provide a comparison between compiler-generated BTF with pahole
deduplication and compiler-generated BTF with linker deduplication
when the latter becomes available.

[1] https://lore.kernel.org/dwarves/87ldqf1i19.fsf@esperi.org.uk/

Alan Maguire (6):
  btf_loader: Make BTF representation match DWARF
  pfunct: Fix up function display with updated prototype representation
  pahole: Add btf_encode to conf_load
  btf_loader: read ELF for BTF encoding
  btf_encoder: Do not error out if BTF is not found in some input files
  tests: Add test of pahole using BTF as input to BTF generation

 btf_loader.c       |  36 +++++++++++--
 dwarves.c          |  14 +++++-
 dwarves.h          |   1 +
 dwarves_fprintf.c  |   2 +-
 pahole.c           |  15 +++---
 pfunct.c           |   2 +-
 tests/btf_2_btf.sh | 122 +++++++++++++++++++++++++++++++++++++++++++++
 7 files changed, 176 insertions(+), 16 deletions(-)
 create mode 100755 tests/btf_2_btf.sh

-- 
2.43.5

[RFC dwarves 1/6] btf_loader: Make BTF representation match DWARF

[Alan Maguire]

The function prototype representation for BTF needs to be modified to
fit with the DWARF-generated one.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 btf_loader.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/btf_loader.c b/btf_loader.c
index 64ea680..42bca92 100644
--- a/btf_loader.c
+++ b/btf_loader.c
@@ -84,6 +84,8 @@ out_free_parameters:
 static int create_new_function(struct cu *cu, const struct btf_type *tp, uint32_t id)
 {
 	struct function *func = tag__alloc(sizeof(*func));
+	struct btf *btf = cu->priv;
+	const struct btf_type *t;
 
 	if (func == NULL)
 		return -ENOMEM;
@@ -95,7 +97,9 @@ static int create_new_function(struct cu *cu, const struct btf_type *tp, uint32_
 	func->proto.tag.type = tp->type;
 	func->name = cu__btf_str(cu, tp->name_off);
 	INIT_LIST_HEAD(&func->lexblock.tags);
-	cu__add_tag_with_id(cu, &func->proto.tag, id);
+	INIT_LIST_HEAD(&func->annots);
+	t = btf__type_by_id(btf, tp->type);
+	cu__load_ftype(cu, &func->proto, DW_TAG_subprogram, t, id);
 
 	return 0;
 }
@@ -124,6 +128,7 @@ static void type__init(struct type *type, uint32_t tag, const char *name, size_t
 	type->size = size;
 	type->namespace.tag.tag = tag;
 	type->namespace.name = name;
+	INIT_LIST_HEAD(&type->namespace.annots);
 	type->template_parameter_pack = NULL;
 }
 
-- 
2.43.5

[RFC dwarves 2/6] pfunct: Fix up function display with updated prototype representation

[Alan Maguire]

Now that the function prototype representation has been updated
in the BTF-derived ftype, no special handling is needed for printing
BTF functions.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 dwarves_fprintf.c | 2 +-
 pfunct.c          | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/dwarves_fprintf.c b/dwarves_fprintf.c
index 1ec478c..557fe7a 100644
--- a/dwarves_fprintf.c
+++ b/dwarves_fprintf.c
@@ -1402,7 +1402,7 @@ static size_t function__fprintf(const struct tag *tag, const struct cu *cu,
 				const struct conf_fprintf *conf, FILE *fp)
 {
 	struct function *func = tag__function(tag);
-	struct ftype *ftype = func->btf ? tag__ftype(cu__type(cu, func->proto.tag.type)) : &func->proto;
+	struct ftype *ftype = &func->proto;
 	size_t printed = 0;
 	bool inlined = !conf->strip_inline && function__declared_inline(func);
 	int i;
diff --git a/pfunct.c b/pfunct.c
index 5a6dd59..61cefd5 100644
--- a/pfunct.c
+++ b/pfunct.c
@@ -326,7 +326,7 @@ static int function__emit_type_definitions(struct function *func,
 					   struct cu *cu, FILE *fp)
 {
 	struct parameter *pos;
-	struct ftype *proto = func->btf ? tag__ftype(cu__type(cu, func->proto.tag.type)) : &func->proto;
+	struct ftype *proto = &func->proto;
 	struct tag *type = cu__type(cu, proto->tag.type);
 
 retry_return_type:
-- 
2.43.5

[RFC dwarves 3/6] pahole: Add btf_encode to conf_load

[Alan Maguire]

This will allow us to take BTF encoding-specific actions in
the btf_loader.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 dwarves.h |  1 +
 pahole.c  | 15 +++++++--------
 2 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/dwarves.h b/dwarves.h
index 21d4166..feb7402 100644
--- a/dwarves.h
+++ b/dwarves.h
@@ -87,6 +87,7 @@ struct conf_load {
 	bool			ignore_inline_expansions;
 	bool			ignore_labels;
 	bool			ptr_table_stats;
+	bool			btf_encode;
 	bool			skip_encoding_btf_decl_tag;
 	bool			skip_missing;
 	bool			skip_encoding_btf_type_tag;
diff --git a/pahole.c b/pahole.c
index ef01e58..96c748b 100644
--- a/pahole.c
+++ b/pahole.c
@@ -32,7 +32,6 @@
 static struct btf_encoder *btf_encoder;
 static char *detached_btf_filename;
 struct cus *cus;
-static bool btf_encode;
 static bool ctf_encode;
 static bool sort_output;
 static bool need_resort;
@@ -1864,7 +1863,7 @@ static error_t pahole__options_parser(int key, char *arg,
 							break;
 	case ARGP_btf_encode_detached:
 		  detached_btf_filename = arg; // fallthru
-	case 'J': btf_encode = 1;
+	case 'J': conf_load.btf_encode = true;
 		  conf_load.get_addr_info = true;
 		  conf_load.ignore_alignment_attr = true;
 		  // XXX for now, test this more thoroughly
@@ -3217,7 +3216,7 @@ static enum load_steal_kind pahole_stealer(struct cu *cu, struct conf_load *conf
 	    print_enumeration_with_enumerator(cu, enumerator_name))
 		return LSK__DELETE; // Maybe we can find this in several CUs, so don't stop it
 
-	if (btf_encode) {
+	if (conf_load->btf_encode) {
 		return pahole_stealer__btf_encode(cu, conf_load);
 	}
 #if 0
@@ -3530,7 +3529,7 @@ int main(int argc, char *argv[])
 	// and as this is at this point unintended, avoid that.
 	// Next we need to just skip object files that don't have the format we
 	// expect as the source for BTF encoding, i.e. no DWARF, no BTF, no problema.
-	if (btf_encode && conf_load.format_path == NULL)
+	if (conf_load.btf_encode && conf_load.format_path == NULL)
 		conf_load.format_path = "dwarf";
 
 	if (show_running_kernel_vmlinux) {
@@ -3593,7 +3592,7 @@ int main(int argc, char *argv[])
 				base_btf_file, libbpf_get_error(conf_load.base_btf));
 			goto out;
 		}
-		if (!btf_encode && !ctf_encode) {
+		if (!conf_load.btf_encode && !ctf_encode) {
 			// Force "btf" since a btf_base is being informed
 			conf_load.format_path = "btf";
 		}
@@ -3641,7 +3640,7 @@ try_sole_arg_as_class_names:
 
 	err = cus__load_files(cus, &conf_load, argv + remaining);
 	if (err != 0) {
-		if (class_name == NULL && !btf_encode && !ctf_encode) {
+		if (class_name == NULL && !conf_load.btf_encode && !ctf_encode) {
 			class_name = argv[remaining];
 
 			if (class_name == NULL) {
@@ -3658,7 +3657,7 @@ try_sole_arg_as_class_names:
 			goto try_sole_arg_as_class_names;
 		}
 
-		if (btf_encode || ctf_encode) {
+		if (conf_load.btf_encode || ctf_encode) {
 			// If encoding is asked for and there is no DEBUG info to encode from,
 			// there are no errors, continue...
 			goto out_ok;
@@ -3717,7 +3716,7 @@ try_sole_arg_as_class_names:
 	type_instance__delete(header);
 	header = NULL;
 
-	if (btf_encode && btf_encoder) { // maybe all CUs were filtered out and thus we don't have an encoder?
+	if (conf_load.btf_encode && btf_encoder) { // maybe all CUs were filtered out and thus we don't have an encoder?
 		err = btf_encoder__encode(btf_encoder, &conf_load);
 		btf_encoder__delete(btf_encoder);
 		if (err) {
-- 
2.43.5

[RFC dwarves 4/6] btf_loader: read ELF for BTF encoding

[Alan Maguire]

When encoding BTF we need ELF information also; read it for BTF encoding case.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 btf_loader.c | 29 ++++++++++++++++++++++++++---
 1 file changed, 26 insertions(+), 3 deletions(-)

diff --git a/btf_loader.c b/btf_loader.c
index 42bca92..504a07d 100644
--- a/btf_loader.c
+++ b/btf_loader.c
@@ -735,7 +735,7 @@ struct debug_fmt_ops btf__ops;
 
 static int cus__load_btf(struct cus *cus, struct conf_load *conf, const char *filename)
 {
-	int err = -1;
+	int fd = -1, err = -1;
 
 	// Pass a zero for addr_size, we'll get it after we load via btf__pointer_size()
 	struct cu *cu = cu__new(filename, 0, NULL, 0, filename, false);
@@ -746,6 +746,23 @@ static int cus__load_btf(struct cus *cus, struct conf_load *conf, const char *fi
 	cu->uses_global_strings = false;
 	cu->dfops = &btf__ops;
 
+	/* Need ELF information for BTF encoding also */
+	if (conf->btf_encode) {
+		fd = open(filename, O_RDONLY), err = -1;
+		if (fd >= 0) {
+			if (elf_version(EV_CURRENT) == EV_NONE) {
+				fprintf(stderr, "%s: cannot set libelf version.\n", __func__);
+				goto out_free;
+			}
+			cu->elf = elf_begin(fd, ELF_C_READ_MMAP, NULL);
+			if (cu->elf == NULL) {
+				fprintf(stderr, "%s: cannot read %s ELF file.\n",
+					__func__, filename);
+				goto out_free;
+			}
+		}
+	}
+
 	libbpf_set_print(libbpf_log);
 
 	struct btf *btf = btf__parse_split(filename, conf->base_btf);
@@ -771,10 +788,16 @@ static int cus__load_btf(struct cus *cus, struct conf_load *conf, const char *fi
 		return 0;
 
 	cus__add(cus, cu);
-	return err;
 
 out_free:
-	cu__delete(cu); // will call btf__free(cu->priv);
+	if (cu->elf) {
+		elf_end(cu->elf);
+		cu->elf = NULL;
+	}
+	if (fd != -1)
+		close(fd);
+	if (err)
+		cu__delete(cu); // will call btf__free(cu->priv);
 	return err;
 }
 
-- 
2.43.5

[RFC dwarves 5/6] btf_encoder: Do not error out if BTF is not found in some input files

[Alan Maguire]

When encoding from a lot of object file sources, it is possible some
will not contain BTF, so if --btf_features=encode_force is set, drive
on as long as at least one _does_ contain a .BTF section.

With this in place, we can test how gcc -gbtf generation across vmlinux
works in a simple manner by doing the following on a kernel build which
specified -gbtf in the KCFLAGS environment variable (tested with gcc 14):

 pahole --format_path=btf -J --btf_features=encode_force \
        --btf_encode_detached=/tmp/vmlinux.btf $(ar t vmlinux.a)

("ar t vmlinux.a" gives a list of the object files comprising vmlinux)

This is no substitute for link-time BTF deduplication of course, but
it does provide a simple way to see the BTF that gcc generates for vmlinux.

The idea is that we can explore differences in BTF generation across
the various combinations

1. debug info source: DWARF; dedup done via pahole (traditional)
2. debug info source: compiler-generated BTF; dedup done via pahole (above)
3. debug info source: compiler-generated BTF; dedup done via linker (TBD)

Handling 3 - linker-based dedup - will require BTF archives so that is the
next step we need to explore.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 dwarves.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/dwarves.c b/dwarves.c
index ef93239..8b37b13 100644
--- a/dwarves.c
+++ b/dwarves.c
@@ -2948,17 +2948,27 @@ try_elf:
 int cus__load_files(struct cus *cus, struct conf_load *conf,
 		    char *filenames[])
 {
-	int i = 0;
+	int i = 0, loaded = 0, failed = 0;
 
 	while (filenames[i] != NULL) {
 		int err = cus__load_file(cus, conf, filenames[i]);
 		if (err) {
 			errno = -err;
-			return -++i;
+			if (conf->btf_encode_force)
+				failed = -(i + 1);
+			else
+				return -++i;
+		} else {
+			loaded++;
 		}
 		++i;
 	}
 
+	if (conf->btf_encode_force) {
+		/* If no files loaded, error out, otherwise return success */
+		if (loaded == 0)
+			return failed;
+	}
 	return i ? 0 : cus__load_running_kernel(cus, conf);
 }
 
-- 
2.43.5

[RFC dwarves 6/6] tests: Add test of pahole using BTF as input to BTF generation

[Alan Maguire]

Create two .o files containing BTF as a result of passing -gbtf
and ensure that we can run BTF generation (and dedup) on them
to get a valid detached BTF output.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 tests/btf_2_btf.sh | 122 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 122 insertions(+)
 create mode 100755 tests/btf_2_btf.sh

diff --git a/tests/btf_2_btf.sh b/tests/btf_2_btf.sh
new file mode 100755
index 0000000..dbc4f34
--- /dev/null
+++ b/tests/btf_2_btf.sh
@@ -0,0 +1,122 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# Copyright (c) 2025, Oracle and/or its affiliates.
+#
+# Use -gbtf to compile objects with BTF and combine/dedup the
+# BTF using pahole; this tests the ability to take BTF as
+# input for BTF encoding as well as DWARF.
+#
+
+outdir=
+
+fail()
+{
+	# Do not remove test dir; might be useful for analysis
+	trap - EXIT
+	if [[ -d "$outdir" ]]; then
+		echo "Test data is in $outdir"
+	fi
+	exit 1
+}
+
+cleanup()
+{
+	rm ${outdir}/*
+	rmdir $outdir
+}
+
+outdir=$(mktemp -d /tmp/btf_2_btf.sh.XXXXXX)
+
+trap cleanup EXIT
+
+echo -n "Validation of BTF encoding from cc-generated BTF: "
+
+cat <<EOF > ${outdir}/f1.c
+#include <stdlib.h>
+
+enum foo {
+	E1,
+	E2,
+	E3
+};
+
+struct bar {
+	enum foo f1;
+	int f2;
+	char *f3;
+	void *f4;
+};
+
+int baz(struct bar *b)
+{
+	return b->f2++;
+}
+EOF
+gcc -gbtf -c ${outdir}/f1.c -o ${outdir}/f1.o
+if [[ $? -ne 0 ]]; then
+	echo "skip: -gbtf not supported or gcc not present"
+	exit 1
+fi
+
+cat <<EOF > ${outdir}/f2.c
+#include <stdlib.h>
+
+enum foo {
+        E1,
+        E2,
+        E3
+};
+
+struct bar {
+        enum foo f1;
+        int f2;
+        char *f3;
+        void *f4;
+};
+
+struct bar2 {
+	void *f1;
+	int f2[3];
+};
+
+int othervar;
+
+void *baz2(struct bar2 *b2, char *o)
+{
+	if (b2->f2[0] > 2)
+		return NULL;
+	return o;
+}
+EOF
+gcc -gbtf -c ${outdir}/f2.c -o ${outdir}/f2.o
+
+pahole -J --format_path=btf --btf_features=default --lang_exclude=rust \
+	--btf_encode_detached=${outdir}/cc.btf ${outdir}/f1.o ${outdir}/f2.o
+
+for f in ${outdir}/f1.o ${outdir}/cc.btf ; do
+	struct=$(pahole $f |grep "struct bar" 2>/dev/null)
+	if [[ -z "$struct" ]]; then
+		echo "struct bar not found in '$f'"
+		fail
+	fi
+	fns=$(pfunct --all $f |grep baz 2>/dev/null)
+	if [[ -z "$fns" ]]; then
+		echo "function baz not found in '$f'"
+		fail
+	fi
+done
+struct=$(pahole $f |grep "struct bar2" 2>/dev/null)
+if [[ -z "$struct" ]]; then
+	echo "struct bar2 not found in '$f'"
+	fail
+fi
+fn=$(pfunct --all $f |grep "baz2" 2>/dev/null)
+if [[ -z "$fn" ]]; then
+	echo "function baz2 not found in '$f'"
+	fail
+fi
+
+echo "Ok"
+
+exit 0
-- 
2.43.5

Re: [RFC dwarves 5/6] btf_encoder: Do not error out if BTF is not found in some input files

[Alexei Starovoitov]

On Thu, Aug 7, 2025 at 7:42 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>
> This is no substitute for link-time BTF deduplication of course, but
> it does provide a simple way to see the BTF that gcc generates for vmlinux.
>
> The idea is that we can explore differences in BTF generation across
> the various combinations
>
> 1. debug info source: DWARF; dedup done via pahole (traditional)
> 2. debug info source: compiler-generated BTF; dedup done via pahole (above)
> 3. debug info source: compiler-generated BTF; dedup done via linker (TBD)
>
> Handling 3 - linker-based dedup - will require BTF archives so that is the
> next step we need to explore.

Overall, the patch set makes sense and we need to make this step in pahole,
but before we start any discussion about 3 and BTF archives
the 1 and 2 above need to reach parity.
Not just being close enough, but an exact equivalence.

But, frankly, gcc support for btf_decl_tags is much much higher priority
than any of this.
We're tired of adding hacks through the bpf subsystem, because
gcc cannot do decl_tags.
Here are the hacks that will be removed:
1. BTF_TYPE_SAFE*
2. raw_tp_null_args[]
3. KF_ARENA_ARG
and probably other cases.

Re: [RFC dwarves 5/6] btf_encoder: Do not error out if BTF is not found in some input files

[Jose E. Marchesi]

> On Thu, Aug 7, 2025 at 7:42 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>>
>> This is no substitute for link-time BTF deduplication of course, but
>> it does provide a simple way to see the BTF that gcc generates for vmlinux.
>>
>> The idea is that we can explore differences in BTF generation across
>> the various combinations
>>
>> 1. debug info source: DWARF; dedup done via pahole (traditional)
>> 2. debug info source: compiler-generated BTF; dedup done via pahole (above)
>> 3. debug info source: compiler-generated BTF; dedup done via linker (TBD)
>>
>> Handling 3 - linker-based dedup - will require BTF archives so that is the
>> next step we need to explore.
>
> Overall, the patch set makes sense and we need to make this step in pahole,
> but before we start any discussion about 3 and BTF archives
> the 1 and 2 above need to reach parity.
> Not just being close enough, but an exact equivalence.
>
> But, frankly, gcc support for btf_decl_tags is much much higher priority
> than any of this.
>
> We're tired of adding hacks through the bpf subsystem, because
> gcc cannot do decl_tags.
> Here are the hacks that will be removed:
> 1. BTF_TYPE_SAFE*
> 2. raw_tp_null_args[]
> 3. KF_ARENA_ARG
> and probably other cases.

We are getting there.  The C front-end maintainer just looked at the
latest version of the series [1] and, other than a small observation
concerning wide char strings, he seems to be ok with the attributes.

[1] https://gcc.gnu.org/pipermail/gcc-patches/2025-August/692057.html

Re: [RFC dwarves 5/6] btf_encoder: Do not error out if BTF is not found in some input files

[Alexei Starovoitov]

On Thu, Aug 7, 2025 at 9:37 AM Jose E. Marchesi
<jose.marchesi@oracle.com> wrote:
>
>
> > On Thu, Aug 7, 2025 at 7:42 AM Alan Maguire <alan.maguire@oracle.com> wrote:
> >>
> >> This is no substitute for link-time BTF deduplication of course, but
> >> it does provide a simple way to see the BTF that gcc generates for vmlinux.
> >>
> >> The idea is that we can explore differences in BTF generation across
> >> the various combinations
> >>
> >> 1. debug info source: DWARF; dedup done via pahole (traditional)
> >> 2. debug info source: compiler-generated BTF; dedup done via pahole (above)
> >> 3. debug info source: compiler-generated BTF; dedup done via linker (TBD)
> >>
> >> Handling 3 - linker-based dedup - will require BTF archives so that is the
> >> next step we need to explore.
> >
> > Overall, the patch set makes sense and we need to make this step in pahole,
> > but before we start any discussion about 3 and BTF archives
> > the 1 and 2 above need to reach parity.
> > Not just being close enough, but an exact equivalence.
> >
> > But, frankly, gcc support for btf_decl_tags is much much higher priority
> > than any of this.
> >
> > We're tired of adding hacks through the bpf subsystem, because
> > gcc cannot do decl_tags.
> > Here are the hacks that will be removed:
> > 1. BTF_TYPE_SAFE*
> > 2. raw_tp_null_args[]
> > 3. KF_ARENA_ARG
> > and probably other cases.
>
> We are getting there.  The C front-end maintainer just looked at the
> latest version of the series [1] and, other than a small observation
> concerning wide char strings, he seems to be ok with the attributes.
>
> [1] https://gcc.gnu.org/pipermail/gcc-patches/2025-August/692057.html

Good to know.

Yonghong, what does llvm do with wchar?

Re: [RFC dwarves 5/6] btf_encoder: Do not error out if BTF is not found in some input files

[Eduard Zingerman]

On Thu, 2025-08-07 at 10:12 -0700, Alexei Starovoitov wrote:
> On Thu, Aug 7, 2025 at 9:37 AM Jose E. Marchesi
> <jose.marchesi@oracle.com> wrote:
> >
> >
> > > On Thu, Aug 7, 2025 at 7:42 AM Alan Maguire <alan.maguire@oracle.com> wrote:
> > > >
> > > > This is no substitute for link-time BTF deduplication of course, but
> > > > it does provide a simple way to see the BTF that gcc generates for vmlinux.
> > > >
> > > > The idea is that we can explore differences in BTF generation across
> > > > the various combinations
> > > >
> > > > 1. debug info source: DWARF; dedup done via pahole (traditional)
> > > > 2. debug info source: compiler-generated BTF; dedup done via pahole (above)
> > > > 3. debug info source: compiler-generated BTF; dedup done via linker (TBD)
> > > >
> > > > Handling 3 - linker-based dedup - will require BTF archives so that is the
> > > > next step we need to explore.
> > >
> > > Overall, the patch set makes sense and we need to make this step in pahole,
> > > but before we start any discussion about 3 and BTF archives
> > > the 1 and 2 above need to reach parity.
> > > Not just being close enough, but an exact equivalence.
> > >
> > > But, frankly, gcc support for btf_decl_tags is much much higher priority
> > > than any of this.
> > >
> > > We're tired of adding hacks through the bpf subsystem, because
> > > gcc cannot do decl_tags.
> > > Here are the hacks that will be removed:
> > > 1. BTF_TYPE_SAFE*
> > > 2. raw_tp_null_args[]
> > > 3. KF_ARENA_ARG
> > > and probably other cases.
> >
> > We are getting there.  The C front-end maintainer just looked at the
> > latest version of the series [1] and, other than a small observation
> > concerning wide char strings, he seems to be ok with the attributes.
> >
> > [1] https://gcc.gnu.org/pipermail/gcc-patches/2025-August/692057.html
>
> Good to know.
>
> Yonghong, what does llvm do with wchar?

The literal is copied as-is with a warning.

  $ cat wide-string-test.c
  __attribute__((btf_decl_tag(u8"üüü")))
  int foo(void) { return 42; }

  $ clang --target=bpf -O2 -g wide-string-test.c -c -o wide-string-test.o
  wide-string-test.c:1:29: warning: encoding prefix 'u8' on an unevaluated string literal has no effect [-Winvalid-unevaluated-string]
      1 | __attribute__((btf_decl_tag(u8"üüü")))
        |                             ^~
  1 warning generated.

  $ bpftool btf dump file wide-string-test.o
  [1] FUNC_PROTO '(anon)' ret_type_id=2 vlen=0
  [2] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
  [3] FUNC 'foo' type_id=1 linkage=global
  [4] DECL_TAG 'üüü' type_id=3 component_idx=-1

"As-is" means using compiler internal encoding (UTF8),
e.g. above u8"üüü" is encoded as "c3 bc c3 bc c3 bc" in the final .BTF
section, same happens for UTF32 literal U"üüü".

Re: [RFC dwarves 5/6] btf_encoder: Do not error out if BTF is not found in some input files

[Yonghong Song]

On 8/7/25 12:18 PM, Eduard Zingerman wrote:
> On Thu, 2025-08-07 at 10:12 -0700, Alexei Starovoitov wrote:
>> On Thu, Aug 7, 2025 at 9:37 AM Jose E. Marchesi
>> <jose.marchesi@oracle.com> wrote:
>>>
>>>> On Thu, Aug 7, 2025 at 7:42 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>>>>> This is no substitute for link-time BTF deduplication of course, but
>>>>> it does provide a simple way to see the BTF that gcc generates for vmlinux.
>>>>>
>>>>> The idea is that we can explore differences in BTF generation across
>>>>> the various combinations
>>>>>
>>>>> 1. debug info source: DWARF; dedup done via pahole (traditional)
>>>>> 2. debug info source: compiler-generated BTF; dedup done via pahole (above)
>>>>> 3. debug info source: compiler-generated BTF; dedup done via linker (TBD)
>>>>>
>>>>> Handling 3 - linker-based dedup - will require BTF archives so that is the
>>>>> next step we need to explore.
>>>> Overall, the patch set makes sense and we need to make this step in pahole,
>>>> but before we start any discussion about 3 and BTF archives
>>>> the 1 and 2 above need to reach parity.
>>>> Not just being close enough, but an exact equivalence.
>>>>
>>>> But, frankly, gcc support for btf_decl_tags is much much higher priority
>>>> than any of this.
>>>>
>>>> We're tired of adding hacks through the bpf subsystem, because
>>>> gcc cannot do decl_tags.
>>>> Here are the hacks that will be removed:
>>>> 1. BTF_TYPE_SAFE*
>>>> 2. raw_tp_null_args[]
>>>> 3. KF_ARENA_ARG
>>>> and probably other cases.
>>> We are getting there.  The C front-end maintainer just looked at the
>>> latest version of the series [1] and, other than a small observation
>>> concerning wide char strings, he seems to be ok with the attributes.
>>>
>>> [1] https://gcc.gnu.org/pipermail/gcc-patches/2025-August/692057.html
>> Good to know.
>>
>> Yonghong, what does llvm do with wchar?
> The literal is copied as-is with a warning.
>
>    $ cat wide-string-test.c
>    __attribute__((btf_decl_tag(u8"üüü")))
>    int foo(void) { return 42; }
>
>    $ clang --target=bpf -O2 -g wide-string-test.c -c -o wide-string-test.o
>    wide-string-test.c:1:29: warning: encoding prefix 'u8' on an unevaluated string literal has no effect [-Winvalid-unevaluated-string]
>        1 | __attribute__((btf_decl_tag(u8"üüü")))
>          |                             ^~
>    1 warning generated.
>
>    $ bpftool btf dump file wide-string-test.o
>    [1] FUNC_PROTO '(anon)' ret_type_id=2 vlen=0
>    [2] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
>    [3] FUNC 'foo' type_id=1 linkage=global
>    [4] DECL_TAG 'üüü' type_id=3 component_idx=-1
>
> "As-is" means using compiler internal encoding (UTF8),
> e.g. above u8"üüü" is encoded as "c3 bc c3 bc c3 bc" in the final .BTF
> section, same happens for UTF32 literal U"üüü".

If we remove 'u8' prefix, there will be no warnings. For example,

$ cat wide-string-test.c
__attribute__((btf_decl_tag("üüü")))
int foo(void) { return 42; }
$ clang --target=bpf -O2 -g wide-string-test.c -c -o wide-string-test.o
$

"üüü" will be treated similar to other ascii strings w.r.t. decl/type tag.