* [PATCH bpf-next v1 1/4] resolve_btfids: rename object btf field to btf_path
2025-11-26 1:26 [PATCH bpf-next v1 0/4] resolve_btfids: Support for BTF modifications Ihor Solodrai
@ 2025-11-26 1:26 ` Ihor Solodrai
2025-11-26 1:26 ` [PATCH bpf-next v1 2/4] resolve_btfids: factor out load_btf() Ihor Solodrai
` (3 subsequent siblings)
4 siblings, 0 replies; 16+ messages in thread
From: Ihor Solodrai @ 2025-11-26 1:26 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Song Liu, Yonghong Song,
John Fastabend, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa,
Nathan Chancellor, Nicolas Schier, Nick Desaulniers,
Bill Wendling, Justin Stitt
Cc: bpf, dwarves, linux-kernel, linux-kbuild, Alan Maguire,
Donglin Peng
Rename the member of `struct object` holding the path to BTF data if
provided via --btf arg. `btf_path` is less ambiguous.
Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
---
tools/bpf/resolve_btfids/main.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/tools/bpf/resolve_btfids/main.c b/tools/bpf/resolve_btfids/main.c
index d47191c6e55e..d2762487ce5f 100644
--- a/tools/bpf/resolve_btfids/main.c
+++ b/tools/bpf/resolve_btfids/main.c
@@ -113,7 +113,7 @@ struct btf_id {
struct object {
const char *path;
- const char *btf;
+ const char *btf_path;
const char *base_btf_path;
struct {
@@ -550,11 +550,11 @@ static int symbols_resolve(struct object *obj)
}
}
- btf = btf__parse_split(obj->btf ?: obj->path, base_btf);
+ btf = btf__parse_split(obj->btf_path ?: obj->path, base_btf);
err = libbpf_get_error(btf);
if (err) {
pr_err("FAILED: load BTF from %s: %s\n",
- obj->btf ?: obj->path, strerror(-err));
+ obj->btf_path ?: obj->path, strerror(-err));
goto out;
}
@@ -790,7 +790,7 @@ int main(int argc, const char **argv)
struct option btfid_options[] = {
OPT_INCR('v', "verbose", &verbose,
"be more verbose (show errors, etc)"),
- OPT_STRING(0, "btf", &obj.btf, "BTF data",
+ OPT_STRING(0, "btf", &obj.btf_path, "BTF data",
"BTF data"),
OPT_STRING('b', "btf_base", &obj.base_btf_path, "file",
"path of file providing base BTF"),
--
2.52.0
^ permalink raw reply related [flat|nested] 16+ messages in thread* [PATCH bpf-next v1 2/4] resolve_btfids: factor out load_btf()
2025-11-26 1:26 [PATCH bpf-next v1 0/4] resolve_btfids: Support for BTF modifications Ihor Solodrai
2025-11-26 1:26 ` [PATCH bpf-next v1 1/4] resolve_btfids: rename object btf field to btf_path Ihor Solodrai
@ 2025-11-26 1:26 ` Ihor Solodrai
2025-11-26 1:26 ` [PATCH bpf-next v1 3/4] resolve_btfids: introduce enum btf_id_kind Ihor Solodrai
` (2 subsequent siblings)
4 siblings, 0 replies; 16+ messages in thread
From: Ihor Solodrai @ 2025-11-26 1:26 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Song Liu, Yonghong Song,
John Fastabend, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa,
Nathan Chancellor, Nicolas Schier, Nick Desaulniers,
Bill Wendling, Justin Stitt
Cc: bpf, dwarves, linux-kernel, linux-kbuild, Alan Maguire,
Donglin Peng
Increase the lifetime of parsed BTF in resolve_btfids by factoring
load_btf() routine out of symbols_resolve() and storing the base_btf
and btf pointers in the struct object.
Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
---
tools/bpf/resolve_btfids/main.c | 47 ++++++++++++++++++++++++---------
1 file changed, 34 insertions(+), 13 deletions(-)
diff --git a/tools/bpf/resolve_btfids/main.c b/tools/bpf/resolve_btfids/main.c
index d2762487ce5f..b7b44e72e765 100644
--- a/tools/bpf/resolve_btfids/main.c
+++ b/tools/bpf/resolve_btfids/main.c
@@ -116,6 +116,9 @@ struct object {
const char *btf_path;
const char *base_btf_path;
+ struct btf *btf;
+ struct btf *base_btf;
+
struct {
int fd;
Elf *elf;
@@ -529,16 +532,10 @@ static int symbols_collect(struct object *obj)
return 0;
}
-static int symbols_resolve(struct object *obj)
+static int load_btf(struct object *obj)
{
- int nr_typedefs = obj->nr_typedefs;
- int nr_structs = obj->nr_structs;
- int nr_unions = obj->nr_unions;
- int nr_funcs = obj->nr_funcs;
- struct btf *base_btf = NULL;
- int err, type_id;
- struct btf *btf;
- __u32 nr_types;
+ struct btf *base_btf = NULL, *btf = NULL;
+ int err;
if (obj->base_btf_path) {
base_btf = btf__parse(obj->base_btf_path, NULL);
@@ -546,7 +543,7 @@ static int symbols_resolve(struct object *obj)
if (err) {
pr_err("FAILED: load base BTF from %s: %s\n",
obj->base_btf_path, strerror(-err));
- return -1;
+ goto out_err;
}
}
@@ -555,9 +552,30 @@ static int symbols_resolve(struct object *obj)
if (err) {
pr_err("FAILED: load BTF from %s: %s\n",
obj->btf_path ?: obj->path, strerror(-err));
- goto out;
+ goto out_err;
}
+ obj->base_btf = base_btf;
+ obj->btf = btf;
+
+ return 0;
+
+out_err:
+ btf__free(base_btf);
+ btf__free(btf);
+ return err;
+}
+
+static int symbols_resolve(struct object *obj)
+{
+ int nr_typedefs = obj->nr_typedefs;
+ int nr_structs = obj->nr_structs;
+ int nr_unions = obj->nr_unions;
+ int nr_funcs = obj->nr_funcs;
+ struct btf *btf = obj->btf;
+ int err, type_id;
+ __u32 nr_types;
+
err = -1;
nr_types = btf__type_cnt(btf);
@@ -615,8 +633,6 @@ static int symbols_resolve(struct object *obj)
err = 0;
out:
- btf__free(base_btf);
- btf__free(btf);
return err;
}
@@ -824,6 +840,9 @@ int main(int argc, const char **argv)
if (symbols_collect(&obj))
goto out;
+ if (load_btf(&obj))
+ goto out;
+
if (symbols_resolve(&obj))
goto out;
@@ -833,6 +852,8 @@ int main(int argc, const char **argv)
if (!(fatal_warnings && warnings))
err = 0;
out:
+ btf__free(obj.base_btf);
+ btf__free(obj.btf);
if (obj.efile.elf) {
elf_end(obj.efile.elf);
close(obj.efile.fd);
--
2.52.0
^ permalink raw reply related [flat|nested] 16+ messages in thread* [PATCH bpf-next v1 3/4] resolve_btfids: introduce enum btf_id_kind
2025-11-26 1:26 [PATCH bpf-next v1 0/4] resolve_btfids: Support for BTF modifications Ihor Solodrai
2025-11-26 1:26 ` [PATCH bpf-next v1 1/4] resolve_btfids: rename object btf field to btf_path Ihor Solodrai
2025-11-26 1:26 ` [PATCH bpf-next v1 2/4] resolve_btfids: factor out load_btf() Ihor Solodrai
@ 2025-11-26 1:26 ` Ihor Solodrai
2025-11-26 2:09 ` bot+bpf-ci
2025-11-26 1:26 ` [PATCH bpf-next v1 4/4] resolve_btfids: change in-place update with raw binary output Ihor Solodrai
2025-11-26 12:36 ` [PATCH bpf-next v1 0/4] resolve_btfids: Support for BTF modifications Alan Maguire
4 siblings, 1 reply; 16+ messages in thread
From: Ihor Solodrai @ 2025-11-26 1:26 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Song Liu, Yonghong Song,
John Fastabend, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa,
Nathan Chancellor, Nicolas Schier, Nick Desaulniers,
Bill Wendling, Justin Stitt
Cc: bpf, dwarves, linux-kernel, linux-kbuild, Alan Maguire,
Donglin Peng
Instead of using multiple flags, make struct btf_id tagged with an
enum value indicating its kind in the context of resolve_btfids.
Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
---
tools/bpf/resolve_btfids/main.c | 61 ++++++++++++++++++++++-----------
1 file changed, 41 insertions(+), 20 deletions(-)
diff --git a/tools/bpf/resolve_btfids/main.c b/tools/bpf/resolve_btfids/main.c
index b7b44e72e765..7f5a9f7dde7f 100644
--- a/tools/bpf/resolve_btfids/main.c
+++ b/tools/bpf/resolve_btfids/main.c
@@ -98,6 +98,13 @@
# error "Unknown machine endianness!"
#endif
+enum btf_id_kind {
+ BTF_ID_KIND_NONE,
+ BTF_ID_KIND_SYM,
+ BTF_ID_KIND_SET,
+ BTF_ID_KIND_SET8
+};
+
struct btf_id {
struct rb_node rb_node;
char *name;
@@ -105,9 +112,8 @@ struct btf_id {
int id;
int cnt;
};
- int addr_cnt;
- bool is_set;
- bool is_set8;
+ enum btf_id_kind kind:8;
+ int addr_cnt:8;
Elf64_Addr addr[ADDR_CNT];
};
@@ -260,26 +266,33 @@ static char *get_id(const char *prefix_end)
return id;
}
-static struct btf_id *add_set(struct object *obj, char *name, bool is_set8)
+static struct btf_id *add_set(struct object *obj, char *name, enum btf_id_kind kind)
{
/*
* __BTF_ID__set__name
* name = ^
* id = ^
*/
- char *id = name + (is_set8 ? sizeof(BTF_SET8 "__") : sizeof(BTF_SET "__")) - 1;
+ int prefixlen = kind == BTF_ID_KIND_SET8 ? sizeof(BTF_SET8 "__") : sizeof(BTF_SET "__");
+ char *id = name + prefixlen - 1;
int len = strlen(name);
+ struct btf_id *btf_id;
if (id >= name + len) {
pr_err("FAILED to parse set name: %s\n", name);
return NULL;
}
- return btf_id__add(&obj->sets, id, true);
+ btf_id = btf_id__add(&obj->sets, id, true);
+ if (btf_id)
+ btf_id->kind = kind;
+
+ return btf_id;
}
static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)
{
+ struct btf_id *btf_id;
char *id;
id = get_id(name + size);
@@ -288,7 +301,10 @@ static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)
return NULL;
}
- return btf_id__add(root, id, false);
+ btf_id = btf_id__add(root, id, false);
+ btf_id->kind = BTF_ID_KIND_SYM;
+
+ return btf_id;
}
/* Older libelf.h and glibc elf.h might not yet define the ELF compression types. */
@@ -491,28 +507,24 @@ static int symbols_collect(struct object *obj)
id = add_symbol(&obj->funcs, prefix, sizeof(BTF_FUNC) - 1);
/* set8 */
} else if (!strncmp(prefix, BTF_SET8, sizeof(BTF_SET8) - 1)) {
- id = add_set(obj, prefix, true);
+ id = add_set(obj, prefix, BTF_ID_KIND_SET8);
/*
* SET8 objects store list's count, which is encoded
* in symbol's size, together with 'cnt' field hence
* that - 1.
*/
- if (id) {
+ if (id)
id->cnt = sym.st_size / sizeof(uint64_t) - 1;
- id->is_set8 = true;
- }
/* set */
} else if (!strncmp(prefix, BTF_SET, sizeof(BTF_SET) - 1)) {
- id = add_set(obj, prefix, false);
+ id = add_set(obj, prefix, BTF_ID_KIND_SET);
/*
* SET objects store list's count, which is encoded
* in symbol's size, together with 'cnt' field hence
* that - 1.
*/
- if (id) {
+ if (id)
id->cnt = sym.st_size / sizeof(int) - 1;
- id->is_set = true;
- }
} else {
pr_err("FAILED unsupported prefix %s\n", prefix);
return -1;
@@ -643,7 +655,7 @@ static int id_patch(struct object *obj, struct btf_id *id)
int i;
/* For set, set8, id->id may be 0 */
- if (!id->id && !id->is_set && !id->is_set8) {
+ if (!id->id && id->kind == BTF_ID_KIND_SYM) {
pr_err("WARN: resolve_btfids: unresolved symbol %s\n", id->name);
warnings++;
}
@@ -696,6 +708,7 @@ static int sets_patch(struct object *obj)
{
Elf_Data *data = obj->efile.idlist;
struct rb_node *next;
+ int cnt;
next = rb_first(&obj->sets);
while (next) {
@@ -715,11 +728,15 @@ static int sets_patch(struct object *obj)
return -1;
}
- if (id->is_set) {
+ switch (id->kind) {
+ case BTF_ID_KIND_SET:
set = data->d_buf + off;
+ cnt = set->cnt;
qsort(set->ids, set->cnt, sizeof(set->ids[0]), cmp_id);
- } else {
+ break;
+ case BTF_ID_KIND_SET8:
set8 = data->d_buf + off;
+ cnt = set8->cnt;
/*
* Make sure id is at the beginning of the pairs
* struct, otherwise the below qsort would not work.
@@ -744,10 +761,14 @@ static int sets_patch(struct object *obj)
bswap_32(set8->pairs[i].flags);
}
}
+ break;
+ case BTF_ID_KIND_SYM:
+ default:
+ pr_err("Unexpected btf_id_kind %d for set '%s'\n", id->kind, id->name);
+ return -1;
}
- pr_debug("sorting addr %5lu: cnt %6d [%s]\n",
- off, id->is_set ? set->cnt : set8->cnt, id->name);
+ pr_debug("sorting addr %5lu: cnt %6d [%s]\n", off, cnt, id->name);
next = rb_next(next);
}
--
2.52.0
^ permalink raw reply related [flat|nested] 16+ messages in thread* Re: [PATCH bpf-next v1 3/4] resolve_btfids: introduce enum btf_id_kind
2025-11-26 1:26 ` [PATCH bpf-next v1 3/4] resolve_btfids: introduce enum btf_id_kind Ihor Solodrai
@ 2025-11-26 2:09 ` bot+bpf-ci
2025-11-26 19:08 ` Ihor Solodrai
0 siblings, 1 reply; 16+ messages in thread
From: bot+bpf-ci @ 2025-11-26 2:09 UTC (permalink / raw)
To: ihor.solodrai, ast, daniel, andrii, martin.lau, eddyz87, song,
yonghong.song, john.fastabend, kpsingh, sdf, haoluo, jolsa,
nathan, nicolas.schier, nick.desaulniers+lkml, morbo, justinstitt
Cc: bpf, dwarves, linux-kernel, linux-kbuild, alan.maguire,
dolinux.peng, ast, andrii, daniel, martin.lau, eddyz87,
yonghong.song, clm, ihor.solodrai
[-- Attachment #1: Type: text/plain, Size: 2121 bytes --]
> diff --git a/tools/bpf/resolve_btfids/main.c b/tools/bpf/resolve_btfids/main.c
> index b7b44e72e..7f5a9f7dd 100644
> --- a/tools/bpf/resolve_btfids/main.c
> +++ b/tools/bpf/resolve_btfids/main.c
[ ... ]
> @@ -260,26 +266,33 @@ static char *get_id(const char *prefix_end)
> return id;
> }
>
> -static struct btf_id *add_set(struct object *obj, char *name, bool is_set8)
> +static struct btf_id *add_set(struct object *obj, char *name, enum btf_id_kind kind)
> {
> /*
> * __BTF_ID__set__name
> * name = ^
> * id = ^
> */
> - char *id = name + (is_set8 ? sizeof(BTF_SET8 "__") : sizeof(BTF_SET "__")) - 1;
> + int prefixlen = kind == BTF_ID_KIND_SET8 ? sizeof(BTF_SET8 "__") : sizeof(BTF_SET "__");
> + char *id = name + prefixlen - 1;
> int len = strlen(name);
> + struct btf_id *btf_id;
>
> if (id >= name + len) {
> pr_err("FAILED to parse set name: %s\n", name);
> return NULL;
> }
>
> - return btf_id__add(&obj->sets, id, true);
> + btf_id = btf_id__add(&obj->sets, id, true);
> + if (btf_id)
> + btf_id->kind = kind;
> +
> + return btf_id;
> }
>
> static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)
> {
> + struct btf_id *btf_id;
> char *id;
>
> id = get_id(name + size);
> @@ -288,7 +301,10 @@ static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)
> return NULL;
> }
>
> - return btf_id__add(root, id, false);
> + btf_id = btf_id__add(root, id, false);
> + btf_id->kind = BTF_ID_KIND_SYM;
^^^^
Can add_symbol() dereference a NULL pointer here? The function
btf_id__add() can return NULL on allocation failure (when zalloc fails),
but the new code sets btf_id->kind before checking if btf_id is NULL.
In contrast, add_set() checks for NULL before setting the kind field.
Should add_symbol() do the same?
> +
> + return btf_id;
> }
[ ... ]
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/19689674924
^ permalink raw reply [flat|nested] 16+ messages in thread* Re: [PATCH bpf-next v1 3/4] resolve_btfids: introduce enum btf_id_kind
2025-11-26 2:09 ` bot+bpf-ci
@ 2025-11-26 19:08 ` Ihor Solodrai
0 siblings, 0 replies; 16+ messages in thread
From: Ihor Solodrai @ 2025-11-26 19:08 UTC (permalink / raw)
To: bot+bpf-ci, ast, daniel, andrii, martin.lau, eddyz87, song,
yonghong.song, john.fastabend, kpsingh, sdf, haoluo, jolsa,
nathan, nicolas.schier, nick.desaulniers+lkml, morbo, justinstitt
Cc: bpf, dwarves, linux-kernel, linux-kbuild, alan.maguire,
dolinux.peng, martin.lau, clm
On 11/25/25 6:09 PM, bot+bpf-ci@kernel.org wrote:
>> diff --git a/tools/bpf/resolve_btfids/main.c b/tools/bpf/resolve_btfids/main.c
>> index b7b44e72e..7f5a9f7dd 100644
>> --- a/tools/bpf/resolve_btfids/main.c
>> +++ b/tools/bpf/resolve_btfids/main.c
>
> [ ... ]
>
>> @@ -260,26 +266,33 @@ static char *get_id(const char *prefix_end)
>> return id;
>> }
>>
>> -static struct btf_id *add_set(struct object *obj, char *name, bool is_set8)
>> +static struct btf_id *add_set(struct object *obj, char *name, enum btf_id_kind kind)
>> {
>> /*
>> * __BTF_ID__set__name
>> * name = ^
>> * id = ^
>> */
>> - char *id = name + (is_set8 ? sizeof(BTF_SET8 "__") : sizeof(BTF_SET "__")) - 1;
>> + int prefixlen = kind == BTF_ID_KIND_SET8 ? sizeof(BTF_SET8 "__") : sizeof(BTF_SET "__");
>> + char *id = name + prefixlen - 1;
>> int len = strlen(name);
>> + struct btf_id *btf_id;
>>
>> if (id >= name + len) {
>> pr_err("FAILED to parse set name: %s\n", name);
>> return NULL;
>> }
>>
>> - return btf_id__add(&obj->sets, id, true);
>> + btf_id = btf_id__add(&obj->sets, id, true);
>> + if (btf_id)
>> + btf_id->kind = kind;
>> +
>> + return btf_id;
>> }
>>
>> static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)
>> {
>> + struct btf_id *btf_id;
>> char *id;
>>
>> id = get_id(name + size);
>> @@ -288,7 +301,10 @@ static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)
>> return NULL;
>> }
>>
>> - return btf_id__add(root, id, false);
>> + btf_id = btf_id__add(root, id, false);
>> + btf_id->kind = BTF_ID_KIND_SYM;
> ^^^^
>
> Can add_symbol() dereference a NULL pointer here? The function
> btf_id__add() can return NULL on allocation failure (when zalloc fails),
> but the new code sets btf_id->kind before checking if btf_id is NULL.
>
> In contrast, add_set() checks for NULL before setting the kind field.
> Should add_symbol() do the same?
We pass unique=false to btf_id__add() and that means that it can only
return NULL if zalloc() fails, which is unlikely.
Checking for NULL here is little too defensive, but won't hurt.
>
>> +
>> + return btf_id;
>> }
>
> [ ... ]
>
>
> ---
> AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
> See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
>
> CI run summary: https://github.com/kernel-patches/bpf/actions/runs/19689674924
^ permalink raw reply [flat|nested] 16+ messages in thread
* [PATCH bpf-next v1 4/4] resolve_btfids: change in-place update with raw binary output
2025-11-26 1:26 [PATCH bpf-next v1 0/4] resolve_btfids: Support for BTF modifications Ihor Solodrai
` (2 preceding siblings ...)
2025-11-26 1:26 ` [PATCH bpf-next v1 3/4] resolve_btfids: introduce enum btf_id_kind Ihor Solodrai
@ 2025-11-26 1:26 ` Ihor Solodrai
2025-11-26 4:46 ` Donglin Peng
2025-11-26 13:03 ` Donglin Peng
2025-11-26 12:36 ` [PATCH bpf-next v1 0/4] resolve_btfids: Support for BTF modifications Alan Maguire
4 siblings, 2 replies; 16+ messages in thread
From: Ihor Solodrai @ 2025-11-26 1:26 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Song Liu, Yonghong Song,
John Fastabend, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa,
Nathan Chancellor, Nicolas Schier, Nick Desaulniers,
Bill Wendling, Justin Stitt
Cc: bpf, dwarves, linux-kernel, linux-kbuild, Alan Maguire,
Donglin Peng
Currently resolve_btfids updates .BTF_ids section of an ELF file
in-place, based on the contents of provided BTF, usually within the
same input file, and optionally a BTF base.
This patch changes resolve_btfids behavior to enable BTF
transformations as part of its main operation. To achieve this
in-place ELF write in resolve_btfids is replaced with generation of
the following binaries:
* ${1}.btf with .BTF section data
* ${1}.distilled_base.btf with .BTF.base section data (for modules)
* ${1}.btf_ids with .BTF_ids section data, if it exists in ${1}
The execution of resolve_btfids and consumption of its output is
orchestrated by scripts/gen-btf.sh introduced in this patch.
The rationale for this approach is that updating ELF in-place with
libelf API is complicated and bug-prone, especially in the context of
the kernel build. On the other hand applying objcopy to manipulate ELF
sections is simpler and more reliable.
There are two distinct paths for BTF generation and resolve_btfids
application in the kernel build: for vmlinux and for kernel modules.
For the vmlinux binary a .BTF section is added in a roundabout way to
ensure correct linking (details below). The patch doesn't change this
approach, only the implementation is a little different.
Before this patch it worked like follows:
* pahole consumed .tmp_vmlinux1 [1] and added .BTF section with
llvm-objcopy [2] to it
* then everything except the .BTF section was stripped from .tmp_vmlinux1
into a .tmp_vmlinux1.bpf.o object [1], later linked into vmlinux
* resolve_btfids was executed later on vmlinux.unstripped [3],
updating it in-place
After this patch gen-btf.sh implements the following:
* pahole consumes .tmp_vmlinux1 and produces a **detached** file
with raw BTF data
* resolve_btfids consumes .tmp_vmlinux1 and detached BTF to produce
(potentially modified) .BTF, and .BTF_ids sections data
* a .tmp_vmlinux1.bpf.o object is then produced with objcopy copying
BTF output of resolve_btfids
* .BTF_ids data gets embedded into vmlinux.unstripped in
link-vmlinux.sh by objcopy --update-section
For the kernel modules creating special .bpf.o file is not necessary,
and so embedding of sections data produced by resolve_btfids is
straightforward with the objcopy.
[1] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n115
[2] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tree/btf_encoder.c#n1835
[3] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n285
Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
---
MAINTAINERS | 1 +
scripts/Makefile.modfinal | 5 +-
scripts/gen-btf.sh | 166 ++++++++++++++++++++++++++++++++
scripts/link-vmlinux.sh | 42 ++------
tools/bpf/resolve_btfids/main.c | 124 ++++++++++++++++++------
5 files changed, 272 insertions(+), 66 deletions(-)
create mode 100755 scripts/gen-btf.sh
diff --git a/MAINTAINERS b/MAINTAINERS
index 48aabeeed029..5cd34419d952 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -4672,6 +4672,7 @@ F: net/sched/act_bpf.c
F: net/sched/cls_bpf.c
F: samples/bpf/
F: scripts/bpf_doc.py
+F: scripts/gen-btf.sh
F: scripts/Makefile.btf
F: scripts/pahole-version.sh
F: tools/bpf/
diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal
index 542ba462ed3e..86f843995556 100644
--- a/scripts/Makefile.modfinal
+++ b/scripts/Makefile.modfinal
@@ -38,9 +38,8 @@ quiet_cmd_btf_ko = BTF [M] $@
cmd_btf_ko = \
if [ ! -f $(objtree)/vmlinux ]; then \
printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \
- else \
- LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base $(objtree)/vmlinux $@; \
- $(RESOLVE_BTFIDS) -b $(objtree)/vmlinux $@; \
+ else \
+ $(objtree)/scripts/gen-btf.sh --btf_base $(objtree)/vmlinux $@; \
fi;
# Same as newer-prereqs, but allows to exclude specified extra dependencies
diff --git a/scripts/gen-btf.sh b/scripts/gen-btf.sh
new file mode 100755
index 000000000000..102f8296ae9e
--- /dev/null
+++ b/scripts/gen-btf.sh
@@ -0,0 +1,166 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2025 Meta Platforms, Inc. and affiliates.
+#
+# This script generates BTF data for the provided ELF file.
+#
+# Kernel BTF generation involves these conceptual steps:
+# 1. pahole generates BTF from DWARF data
+# 2. resolve_btfids applies kernel-specific btf2btf
+# transformations and computes data for .BTF_ids section
+# 3. the result gets linked/objcopied into the target binary
+#
+# How step (3) should be done differs between vmlinux, and
+# kernel modules, which is the primary reason for the existence
+# of this script.
+#
+# For modules the script expects vmlinux passed in as --btf_base.
+# Generated .BTF, .BTF.base and .BTF_ids sections become embedded
+# into the input ELF file with objcopy.
+#
+# For vmlinux the input file remains unchanged and two files are produced:
+# - ${1}.btf.o ready for linking into vmlinux
+# - ${1}.btf_ids with .BTF_ids data blob
+# This output is consumed by scripts/link-vmlinux.sh
+
+set -e
+
+usage()
+{
+ echo "Usage: $0 [--btf_base <file>] <target ELF file>"
+ exit 1
+}
+
+BTF_BASE=""
+
+while [ $# -gt 0 ]; do
+ case "$1" in
+ --btf_base)
+ BTF_BASE="$2"
+ shift 2
+ ;;
+ -*)
+ echo "Unknown option: $1" >&2
+ usage
+ ;;
+ *)
+ break
+ ;;
+ esac
+done
+
+if [ $# -ne 1 ]; then
+ usage
+fi
+
+ELF_FILE="$1"
+shift
+
+is_enabled() {
+ grep -q "^$1=y" ${objtree}/include/config/auto.conf
+}
+
+info()
+{
+ printf " %-7s %s\n" "${1}" "${2}"
+}
+
+case "${KBUILD_VERBOSE}" in
+*1*)
+ set -x
+ ;;
+esac
+
+if ! is_enabled CONFIG_DEBUG_INFO_BTF; then
+ exit 0
+fi
+
+gen_btf_data()
+{
+ info BTF "${ELF_FILE}"
+ btf1="${ELF_FILE}.btf.1"
+ ${PAHOLE} -J ${PAHOLE_FLAGS} \
+ ${BTF_BASE:+--btf_base ${BTF_BASE}} \
+ --btf_encode_detached=${btf1} \
+ "${ELF_FILE}"
+
+ info BTFIDS "${ELF_FILE}"
+ RESOLVE_BTFIDS_OPTS=""
+ if is_enabled CONFIG_WERROR; then
+ RESOLVE_BTFIDS_OPTS+=" --fatal_warnings "
+ fi
+ if [ -n "${KBUILD_VERBOSE}" ]; then
+ RESOLVE_BTFIDS_OPTS+=" -v "
+ fi
+ ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_OPTS} \
+ ${BTF_BASE:+--btf_base ${BTF_BASE}} \
+ --btf ${btf1} "${ELF_FILE}"
+}
+
+gen_btf_o()
+{
+ local btf_data=${ELF_FILE}.btf.o
+
+ # Create ${btf_data} which contains just .BTF section but no symbols. Add
+ # SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all
+ # deletes all symbols including __start_BTF and __stop_BTF, which will
+ # be redefined in the linker script.
+ info OBJCOPY "${btf_data}"
+ echo "" | ${CC} -c -x c -o ${btf_data} -
+ ${OBJCOPY} --add-section .BTF=${ELF_FILE}.btf \
+ --set-section-flags .BTF=alloc,readonly ${btf_data}
+ ${OBJCOPY} --only-section=.BTF --strip-all ${btf_data}
+
+ # Change e_type to ET_REL so that it can be used to link final vmlinux.
+ # GNU ld 2.35+ and lld do not allow an ET_EXEC input.
+ if is_enabled CONFIG_CPU_BIG_ENDIAN; then
+ et_rel='\0\1'
+ else
+ et_rel='\1\0'
+ fi
+ printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none
+}
+
+embed_btf_data()
+{
+ info OBJCOPY "${ELF_FILE}"
+
+ ${OBJCOPY} \
+ --add-section .BTF=${ELF_FILE}.btf \
+ --add-section .BTF.base=${ELF_FILE}.distilled_base.btf \
+ ${ELF_FILE}
+
+ # a module might not have a .BTF_ids section
+ if [ -f "${ELF_FILE}.btf_ids" ]; then
+ ${OBJCOPY} --update-section .BTF_ids=${ELF_FILE}.btf_ids ${ELF_FILE}
+ fi
+}
+
+cleanup()
+{
+ rm -f "${ELF_FILE}.btf.1"
+ rm -f "${ELF_FILE}.btf"
+ if [ "${BTFGEN_MODE}" == "module" ]; then
+ rm -f "${ELF_FILE}.distilled_base.btf"
+ rm -f "${ELF_FILE}.btf_ids"
+ fi
+}
+trap cleanup EXIT
+
+BTFGEN_MODE="vmlinux"
+if [ -n "${BTF_BASE}" ]; then
+ BTFGEN_MODE="module"
+fi
+
+gen_btf_data
+
+case "${BTFGEN_MODE}" in
+vmlinux)
+ gen_btf_o
+ ;;
+module)
+ embed_btf_data
+ ;;
+esac
+
+exit 0
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index 433849ff7529..728f82af24f6 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -105,34 +105,6 @@ vmlinux_link()
${kallsymso} ${btf_vmlinux_bin_o} ${arch_vmlinux_o} ${ldlibs}
}
-# generate .BTF typeinfo from DWARF debuginfo
-# ${1} - vmlinux image
-gen_btf()
-{
- local btf_data=${1}.btf.o
-
- info BTF "${btf_data}"
- LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS} ${1}
-
- # Create ${btf_data} which contains just .BTF section but no symbols. Add
- # SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all
- # deletes all symbols including __start_BTF and __stop_BTF, which will
- # be redefined in the linker script. Add 2>/dev/null to suppress GNU
- # objcopy warnings: "empty loadable segment detected at ..."
- ${OBJCOPY} --only-section=.BTF --set-section-flags .BTF=alloc,readonly \
- --strip-all ${1} "${btf_data}" 2>/dev/null
- # Change e_type to ET_REL so that it can be used to link final vmlinux.
- # GNU ld 2.35+ and lld do not allow an ET_EXEC input.
- if is_enabled CONFIG_CPU_BIG_ENDIAN; then
- et_rel='\0\1'
- else
- et_rel='\1\0'
- fi
- printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none
-
- btf_vmlinux_bin_o=${btf_data}
-}
-
# Create ${2}.o file with all symbols from the ${1} object file
kallsyms()
{
@@ -204,6 +176,7 @@ if is_enabled CONFIG_ARCH_WANTS_PRE_LINK_VMLINUX; then
fi
btf_vmlinux_bin_o=
+btfids_vmlinux=
kallsymso=
strip_debug=
generate_map=
@@ -224,11 +197,13 @@ if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then
fi
if is_enabled CONFIG_DEBUG_INFO_BTF; then
- if ! gen_btf .tmp_vmlinux1; then
+ if ! scripts/gen-btf.sh .tmp_vmlinux1; then
echo >&2 "Failed to generate BTF for vmlinux"
echo >&2 "Try to disable CONFIG_DEBUG_INFO_BTF"
exit 1
fi
+ btf_vmlinux_bin_o=.tmp_vmlinux1.btf.o
+ btfids_vmlinux=.tmp_vmlinux1.btf_ids
fi
if is_enabled CONFIG_KALLSYMS; then
@@ -281,14 +256,9 @@ fi
vmlinux_link "${VMLINUX}"
-# fill in BTF IDs
if is_enabled CONFIG_DEBUG_INFO_BTF; then
- info BTFIDS "${VMLINUX}"
- RESOLVE_BTFIDS_ARGS=""
- if is_enabled CONFIG_WERROR; then
- RESOLVE_BTFIDS_ARGS=" --fatal_warnings "
- fi
- ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_ARGS} "${VMLINUX}"
+ info OBJCOPY ${btfids_vmlinux}
+ ${OBJCOPY} --update-section .BTF_ids=${btfids_vmlinux} ${VMLINUX}
fi
mksysmap "${VMLINUX}" System.map
diff --git a/tools/bpf/resolve_btfids/main.c b/tools/bpf/resolve_btfids/main.c
index 7f5a9f7dde7f..4faf16b1ba6b 100644
--- a/tools/bpf/resolve_btfids/main.c
+++ b/tools/bpf/resolve_btfids/main.c
@@ -71,9 +71,11 @@
#include <fcntl.h>
#include <errno.h>
#include <linux/btf_ids.h>
+#include <linux/kallsyms.h>
#include <linux/rbtree.h>
#include <linux/zalloc.h>
#include <linux/err.h>
+#include <linux/limits.h>
#include <bpf/btf.h>
#include <bpf/libbpf.h>
#include <subcmd/parse-options.h>
@@ -429,14 +431,6 @@ static int elf_collect(struct object *obj)
obj->efile.idlist = data;
obj->efile.idlist_shndx = idx;
obj->efile.idlist_addr = sh.sh_addr;
- } else if (!strcmp(name, BTF_BASE_ELF_SEC)) {
- /* If a .BTF.base section is found, do not resolve
- * BTF ids relative to vmlinux; resolve relative
- * to the .BTF.base section instead. btf__parse_split()
- * will take care of this once the base BTF it is
- * passed is NULL.
- */
- obj->base_btf_path = NULL;
}
if (compressed_section_fix(elf, scn, &sh))
@@ -570,6 +564,19 @@ static int load_btf(struct object *obj)
obj->base_btf = base_btf;
obj->btf = btf;
+ if (obj->base_btf) {
+ err = btf__distill_base(obj->btf, &base_btf, &btf);
+ if (err) {
+ pr_err("FAILED to distill base BTF: %s\n", strerror(errno));
+ goto out_err;
+ }
+
+ btf__free(obj->btf);
+ btf__free(obj->base_btf);
+ obj->btf = btf;
+ obj->base_btf = base_btf;
+ }
+
return 0;
out_err:
@@ -777,8 +784,6 @@ static int sets_patch(struct object *obj)
static int symbols_patch(struct object *obj)
{
- off_t err;
-
if (__symbols_patch(obj, &obj->structs) ||
__symbols_patch(obj, &obj->unions) ||
__symbols_patch(obj, &obj->typedefs) ||
@@ -789,20 +794,67 @@ static int symbols_patch(struct object *obj)
if (sets_patch(obj))
return -1;
- /* Set type to ensure endian translation occurs. */
- obj->efile.idlist->d_type = ELF_T_WORD;
+ return 0;
+}
- elf_flagdata(obj->efile.idlist, ELF_C_SET, ELF_F_DIRTY);
+static int dump_raw_data(const char *out_path, const void *data, u32 size)
+{
+ int fd, ret;
- err = elf_update(obj->efile.elf, ELF_C_WRITE);
- if (err < 0) {
- pr_err("FAILED elf_update(WRITE): %s\n",
- elf_errmsg(-1));
+ fd = open(out_path, O_WRONLY | O_CREAT | O_TRUNC, 0640);
+ if (fd < 0) {
+ pr_err("Couldn't open %s for writing\n", out_path);
+ return fd;
+ }
+
+ ret = write(fd, data, size);
+ if (ret < 0 || ret != size) {
+ pr_err("Failed to write data to %s\n", out_path);
+ close(fd);
+ unlink(out_path);
+ return -1;
+ }
+
+ close(fd);
+ pr_debug("Dumped %lu bytes of data to %s\n", size, out_path);
+
+ return 0;
+}
+
+static int dump_raw_btf_ids(struct object *obj, const char *out_path)
+{
+ Elf_Data *data = obj->efile.idlist;
+ int fd, err;
+
+ if (!data || !data->d_buf) {
+ pr_debug("%s has no BTF_ids data to dump\n", obj->path);
+ return 0;
+ }
+
+ err = dump_raw_data(out_path, data->d_buf, data->d_size);
+ if (err)
+ return -1;
+
+ return 0;
+}
+
+static int dump_raw_btf(struct btf *btf, const char *out_path)
+{
+ const void *raw_btf_data;
+ u32 raw_btf_size;
+ int fd, err;
+
+ raw_btf_data = btf__raw_data(btf, &raw_btf_size);
+ if (raw_btf_data == NULL) {
+ pr_err("btf__raw_data() failed\n");
+ return -1;
}
- pr_debug("update %s for %s\n",
- err >= 0 ? "ok" : "failed", obj->path);
- return err < 0 ? -1 : 0;
+ err = dump_raw_data(out_path, raw_btf_data, raw_btf_size);
+ if (err)
+ return -1;
+
+ return 0;
}
static const char * const resolve_btfids_usage[] = {
@@ -823,12 +875,13 @@ int main(int argc, const char **argv)
.funcs = RB_ROOT,
.sets = RB_ROOT,
};
+ char out_path[PATH_MAX];
bool fatal_warnings = false;
struct option btfid_options[] = {
OPT_INCR('v', "verbose", &verbose,
"be more verbose (show errors, etc)"),
OPT_STRING(0, "btf", &obj.btf_path, "BTF data",
- "BTF data"),
+ "input BTF data"),
OPT_STRING('b', "btf_base", &obj.base_btf_path, "file",
"path of file providing base BTF"),
OPT_BOOLEAN(0, "fatal_warnings", &fatal_warnings,
@@ -844,6 +897,9 @@ int main(int argc, const char **argv)
obj.path = argv[0];
+ if (load_btf(&obj))
+ goto out;
+
if (elf_collect(&obj))
goto out;
@@ -853,23 +909,37 @@ int main(int argc, const char **argv)
*/
if (obj.efile.idlist_shndx == -1 ||
obj.efile.symbols_shndx == -1) {
- pr_debug("Cannot find .BTF_ids or symbols sections, nothing to do\n");
- err = 0;
- goto out;
+ pr_debug("Cannot find .BTF_ids or symbols sections, skip symbols resolution\n");
+ goto dump_btf;
}
if (symbols_collect(&obj))
goto out;
- if (load_btf(&obj))
- goto out;
-
if (symbols_resolve(&obj))
goto out;
if (symbols_patch(&obj))
goto out;
+ strcpy(out_path, obj.path);
+ strcat(out_path, ".btf_ids");
+ if (dump_raw_btf_ids(&obj, out_path))
+ goto out;
+
+dump_btf:
+ strcpy(out_path, obj.path);
+ strcat(out_path, ".btf");
+ if (dump_raw_btf(obj.btf, out_path))
+ goto out;
+
+ if (obj.base_btf) {
+ strcpy(out_path, obj.path);
+ strcat(out_path, ".distilled_base.btf");
+ if (dump_raw_btf(obj.base_btf, out_path))
+ goto out;
+ }
+
if (!(fatal_warnings && warnings))
err = 0;
out:
--
2.52.0
^ permalink raw reply related [flat|nested] 16+ messages in thread* Re: [PATCH bpf-next v1 4/4] resolve_btfids: change in-place update with raw binary output
2025-11-26 1:26 ` [PATCH bpf-next v1 4/4] resolve_btfids: change in-place update with raw binary output Ihor Solodrai
@ 2025-11-26 4:46 ` Donglin Peng
2025-11-26 18:22 ` Ihor Solodrai
2025-11-26 18:32 ` Ihor Solodrai
2025-11-26 13:03 ` Donglin Peng
1 sibling, 2 replies; 16+ messages in thread
From: Donglin Peng @ 2025-11-26 4:46 UTC (permalink / raw)
To: Ihor Solodrai
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Song Liu, Yonghong Song,
John Fastabend, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa,
Nathan Chancellor, Nicolas Schier, Nick Desaulniers,
Bill Wendling, Justin Stitt, bpf, dwarves, linux-kernel,
linux-kbuild, Alan Maguire
On Wed, Nov 26, 2025 at 9:29 AM Ihor Solodrai <ihor.solodrai@linux.dev> wrote:
>
> Currently resolve_btfids updates .BTF_ids section of an ELF file
> in-place, based on the contents of provided BTF, usually within the
> same input file, and optionally a BTF base.
>
> This patch changes resolve_btfids behavior to enable BTF
> transformations as part of its main operation. To achieve this
> in-place ELF write in resolve_btfids is replaced with generation of
> the following binaries:
> * ${1}.btf with .BTF section data
> * ${1}.distilled_base.btf with .BTF.base section data (for modules)
> * ${1}.btf_ids with .BTF_ids section data, if it exists in ${1}
>
> The execution of resolve_btfids and consumption of its output is
> orchestrated by scripts/gen-btf.sh introduced in this patch.
>
> The rationale for this approach is that updating ELF in-place with
> libelf API is complicated and bug-prone, especially in the context of
> the kernel build. On the other hand applying objcopy to manipulate ELF
> sections is simpler and more reliable.
>
> There are two distinct paths for BTF generation and resolve_btfids
> application in the kernel build: for vmlinux and for kernel modules.
>
> For the vmlinux binary a .BTF section is added in a roundabout way to
> ensure correct linking (details below). The patch doesn't change this
> approach, only the implementation is a little different.
>
> Before this patch it worked like follows:
>
> * pahole consumed .tmp_vmlinux1 [1] and added .BTF section with
> llvm-objcopy [2] to it
> * then everything except the .BTF section was stripped from .tmp_vmlinux1
> into a .tmp_vmlinux1.bpf.o object [1], later linked into vmlinux
> * resolve_btfids was executed later on vmlinux.unstripped [3],
> updating it in-place
>
> After this patch gen-btf.sh implements the following:
>
> * pahole consumes .tmp_vmlinux1 and produces a **detached** file
> with raw BTF data
> * resolve_btfids consumes .tmp_vmlinux1 and detached BTF to produce
> (potentially modified) .BTF, and .BTF_ids sections data
> * a .tmp_vmlinux1.bpf.o object is then produced with objcopy copying
> BTF output of resolve_btfids
> * .BTF_ids data gets embedded into vmlinux.unstripped in
> link-vmlinux.sh by objcopy --update-section
>
> For the kernel modules creating special .bpf.o file is not necessary,
> and so embedding of sections data produced by resolve_btfids is
> straightforward with the objcopy.
>
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n115
> [2] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tree/btf_encoder.c#n1835
> [3] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n285
>
> Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
> ---
> MAINTAINERS | 1 +
> scripts/Makefile.modfinal | 5 +-
> scripts/gen-btf.sh | 166 ++++++++++++++++++++++++++++++++
> scripts/link-vmlinux.sh | 42 ++------
> tools/bpf/resolve_btfids/main.c | 124 ++++++++++++++++++------
> 5 files changed, 272 insertions(+), 66 deletions(-)
> create mode 100755 scripts/gen-btf.sh
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 48aabeeed029..5cd34419d952 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -4672,6 +4672,7 @@ F: net/sched/act_bpf.c
> F: net/sched/cls_bpf.c
> F: samples/bpf/
> F: scripts/bpf_doc.py
> +F: scripts/gen-btf.sh
> F: scripts/Makefile.btf
> F: scripts/pahole-version.sh
> F: tools/bpf/
> diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal
> index 542ba462ed3e..86f843995556 100644
> --- a/scripts/Makefile.modfinal
> +++ b/scripts/Makefile.modfinal
> @@ -38,9 +38,8 @@ quiet_cmd_btf_ko = BTF [M] $@
> cmd_btf_ko = \
> if [ ! -f $(objtree)/vmlinux ]; then \
> printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \
> - else \
> - LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base $(objtree)/vmlinux $@; \
> - $(RESOLVE_BTFIDS) -b $(objtree)/vmlinux $@; \
> + else \
> + $(objtree)/scripts/gen-btf.sh --btf_base $(objtree)/vmlinux $@; \
> fi;
>
> # Same as newer-prereqs, but allows to exclude specified extra dependencies
> diff --git a/scripts/gen-btf.sh b/scripts/gen-btf.sh
> new file mode 100755
> index 000000000000..102f8296ae9e
> --- /dev/null
> +++ b/scripts/gen-btf.sh
> @@ -0,0 +1,166 @@
> +#!/bin/sh
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2025 Meta Platforms, Inc. and affiliates.
> +#
> +# This script generates BTF data for the provided ELF file.
> +#
> +# Kernel BTF generation involves these conceptual steps:
> +# 1. pahole generates BTF from DWARF data
> +# 2. resolve_btfids applies kernel-specific btf2btf
> +# transformations and computes data for .BTF_ids section
> +# 3. the result gets linked/objcopied into the target binary
> +#
> +# How step (3) should be done differs between vmlinux, and
> +# kernel modules, which is the primary reason for the existence
> +# of this script.
> +#
> +# For modules the script expects vmlinux passed in as --btf_base.
> +# Generated .BTF, .BTF.base and .BTF_ids sections become embedded
> +# into the input ELF file with objcopy.
> +#
> +# For vmlinux the input file remains unchanged and two files are produced:
> +# - ${1}.btf.o ready for linking into vmlinux
> +# - ${1}.btf_ids with .BTF_ids data blob
> +# This output is consumed by scripts/link-vmlinux.sh
> +
> +set -e
> +
> +usage()
> +{
> + echo "Usage: $0 [--btf_base <file>] <target ELF file>"
> + exit 1
> +}
> +
> +BTF_BASE=""
> +
> +while [ $# -gt 0 ]; do
> + case "$1" in
> + --btf_base)
> + BTF_BASE="$2"
> + shift 2
> + ;;
> + -*)
> + echo "Unknown option: $1" >&2
> + usage
> + ;;
> + *)
> + break
> + ;;
> + esac
> +done
> +
> +if [ $# -ne 1 ]; then
> + usage
> +fi
> +
> +ELF_FILE="$1"
> +shift
> +
> +is_enabled() {
> + grep -q "^$1=y" ${objtree}/include/config/auto.conf
> +}
> +
> +info()
> +{
> + printf " %-7s %s\n" "${1}" "${2}"
> +}
> +
> +case "${KBUILD_VERBOSE}" in
> +*1*)
> + set -x
> + ;;
> +esac
> +
> +if ! is_enabled CONFIG_DEBUG_INFO_BTF; then
> + exit 0
> +fi
> +
> +gen_btf_data()
> +{
> + info BTF "${ELF_FILE}"
> + btf1="${ELF_FILE}.btf.1"
> + ${PAHOLE} -J ${PAHOLE_FLAGS} \
> + ${BTF_BASE:+--btf_base ${BTF_BASE}} \
> + --btf_encode_detached=${btf1} \
> + "${ELF_FILE}"
> +
> + info BTFIDS "${ELF_FILE}"
> + RESOLVE_BTFIDS_OPTS=""
> + if is_enabled CONFIG_WERROR; then
> + RESOLVE_BTFIDS_OPTS+=" --fatal_warnings "
In POSIX sh, +=is undefined[1], and I encountered the following error:
./scripts/gen-btf.sh: 90: RESOLVE_BTFIDS_OPTS+= --fatal_warnings : not found
We should use the following syntax instead:
RESOLVE_BTFIDS_OPTS="${RESOLVE_BTFIDS_OPTS} --fatal_warnings "
[1] https://www.shellcheck.net/wiki/SC3024
Thanks,
Donglin
> + fi
> + if [ -n "${KBUILD_VERBOSE}" ]; then
> + RESOLVE_BTFIDS_OPTS+=" -v "
Ditto
Thanks,
Donglin
> + fi
> + ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_OPTS} \
> + ${BTF_BASE:+--btf_base ${BTF_BASE}} \
> + --btf ${btf1} "${ELF_FILE}"
> +}
> +
> +gen_btf_o()
> +{
> + local btf_data=${ELF_FILE}.btf.o
> +
> + # Create ${btf_data} which contains just .BTF section but no symbols. Add
> + # SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all
> + # deletes all symbols including __start_BTF and __stop_BTF, which will
> + # be redefined in the linker script.
> + info OBJCOPY "${btf_data}"
> + echo "" | ${CC} -c -x c -o ${btf_data} -
> + ${OBJCOPY} --add-section .BTF=${ELF_FILE}.btf \
> + --set-section-flags .BTF=alloc,readonly ${btf_data}
> + ${OBJCOPY} --only-section=.BTF --strip-all ${btf_data}
> +
> + # Change e_type to ET_REL so that it can be used to link final vmlinux.
> + # GNU ld 2.35+ and lld do not allow an ET_EXEC input.
> + if is_enabled CONFIG_CPU_BIG_ENDIAN; then
> + et_rel='\0\1'
> + else
> + et_rel='\1\0'
> + fi
> + printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none
> +}
> +
> +embed_btf_data()
> +{
> + info OBJCOPY "${ELF_FILE}"
> +
> + ${OBJCOPY} \
> + --add-section .BTF=${ELF_FILE}.btf \
> + --add-section .BTF.base=${ELF_FILE}.distilled_base.btf \
> + ${ELF_FILE}
> +
> + # a module might not have a .BTF_ids section
> + if [ -f "${ELF_FILE}.btf_ids" ]; then
> + ${OBJCOPY} --update-section .BTF_ids=${ELF_FILE}.btf_ids ${ELF_FILE}
> + fi
> +}
> +
> +cleanup()
> +{
> + rm -f "${ELF_FILE}.btf.1"
> + rm -f "${ELF_FILE}.btf"
> + if [ "${BTFGEN_MODE}" == "module" ]; then
In POSIX sh, == in place of = is undefined[2], and I encountered the following
error:
./scripts/gen-btf.sh: 143: [: module: unexpected operator
we should use = instead.
[2] https://www.shellcheck.net/wiki/SC3014
Thanks,
Donglin
> + rm -f "${ELF_FILE}.distilled_base.btf"
> + rm -f "${ELF_FILE}.btf_ids"
> + fi
> +}
> +trap cleanup EXIT
> +
> +BTFGEN_MODE="vmlinux"
> +if [ -n "${BTF_BASE}" ]; then
> + BTFGEN_MODE="module"
> +fi
> +
> +gen_btf_data
> +
> +case "${BTFGEN_MODE}" in
> +vmlinux)
> + gen_btf_o
> + ;;
> +module)
> + embed_btf_data
> + ;;
> +esac
> +
> +exit 0
> diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
> index 433849ff7529..728f82af24f6 100755
> --- a/scripts/link-vmlinux.sh
> +++ b/scripts/link-vmlinux.sh
> @@ -105,34 +105,6 @@ vmlinux_link()
> ${kallsymso} ${btf_vmlinux_bin_o} ${arch_vmlinux_o} ${ldlibs}
> }
>
> -# generate .BTF typeinfo from DWARF debuginfo
> -# ${1} - vmlinux image
> -gen_btf()
> -{
> - local btf_data=${1}.btf.o
> -
> - info BTF "${btf_data}"
> - LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS} ${1}
> -
> - # Create ${btf_data} which contains just .BTF section but no symbols. Add
> - # SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all
> - # deletes all symbols including __start_BTF and __stop_BTF, which will
> - # be redefined in the linker script. Add 2>/dev/null to suppress GNU
> - # objcopy warnings: "empty loadable segment detected at ..."
> - ${OBJCOPY} --only-section=.BTF --set-section-flags .BTF=alloc,readonly \
> - --strip-all ${1} "${btf_data}" 2>/dev/null
> - # Change e_type to ET_REL so that it can be used to link final vmlinux.
> - # GNU ld 2.35+ and lld do not allow an ET_EXEC input.
> - if is_enabled CONFIG_CPU_BIG_ENDIAN; then
> - et_rel='\0\1'
> - else
> - et_rel='\1\0'
> - fi
> - printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none
> -
> - btf_vmlinux_bin_o=${btf_data}
> -}
> -
> # Create ${2}.o file with all symbols from the ${1} object file
> kallsyms()
> {
> @@ -204,6 +176,7 @@ if is_enabled CONFIG_ARCH_WANTS_PRE_LINK_VMLINUX; then
> fi
>
> btf_vmlinux_bin_o=
> +btfids_vmlinux=
> kallsymso=
> strip_debug=
> generate_map=
> @@ -224,11 +197,13 @@ if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then
> fi
>
> if is_enabled CONFIG_DEBUG_INFO_BTF; then
> - if ! gen_btf .tmp_vmlinux1; then
> + if ! scripts/gen-btf.sh .tmp_vmlinux1; then
> echo >&2 "Failed to generate BTF for vmlinux"
> echo >&2 "Try to disable CONFIG_DEBUG_INFO_BTF"
> exit 1
> fi
> + btf_vmlinux_bin_o=.tmp_vmlinux1.btf.o
> + btfids_vmlinux=.tmp_vmlinux1.btf_ids
> fi
>
> if is_enabled CONFIG_KALLSYMS; then
> @@ -281,14 +256,9 @@ fi
>
> vmlinux_link "${VMLINUX}"
>
> -# fill in BTF IDs
> if is_enabled CONFIG_DEBUG_INFO_BTF; then
> - info BTFIDS "${VMLINUX}"
> - RESOLVE_BTFIDS_ARGS=""
> - if is_enabled CONFIG_WERROR; then
> - RESOLVE_BTFIDS_ARGS=" --fatal_warnings "
> - fi
> - ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_ARGS} "${VMLINUX}"
> + info OBJCOPY ${btfids_vmlinux}
> + ${OBJCOPY} --update-section .BTF_ids=${btfids_vmlinux} ${VMLINUX}
> fi
>
> mksysmap "${VMLINUX}" System.map
> diff --git a/tools/bpf/resolve_btfids/main.c b/tools/bpf/resolve_btfids/main.c
> index 7f5a9f7dde7f..4faf16b1ba6b 100644
> --- a/tools/bpf/resolve_btfids/main.c
> +++ b/tools/bpf/resolve_btfids/main.c
> @@ -71,9 +71,11 @@
> #include <fcntl.h>
> #include <errno.h>
> #include <linux/btf_ids.h>
> +#include <linux/kallsyms.h>
> #include <linux/rbtree.h>
> #include <linux/zalloc.h>
> #include <linux/err.h>
> +#include <linux/limits.h>
> #include <bpf/btf.h>
> #include <bpf/libbpf.h>
> #include <subcmd/parse-options.h>
> @@ -429,14 +431,6 @@ static int elf_collect(struct object *obj)
> obj->efile.idlist = data;
> obj->efile.idlist_shndx = idx;
> obj->efile.idlist_addr = sh.sh_addr;
> - } else if (!strcmp(name, BTF_BASE_ELF_SEC)) {
> - /* If a .BTF.base section is found, do not resolve
> - * BTF ids relative to vmlinux; resolve relative
> - * to the .BTF.base section instead. btf__parse_split()
> - * will take care of this once the base BTF it is
> - * passed is NULL.
> - */
> - obj->base_btf_path = NULL;
> }
>
> if (compressed_section_fix(elf, scn, &sh))
> @@ -570,6 +564,19 @@ static int load_btf(struct object *obj)
> obj->base_btf = base_btf;
> obj->btf = btf;
>
> + if (obj->base_btf) {
> + err = btf__distill_base(obj->btf, &base_btf, &btf);
> + if (err) {
> + pr_err("FAILED to distill base BTF: %s\n", strerror(errno));
> + goto out_err;
> + }
> +
> + btf__free(obj->btf);
> + btf__free(obj->base_btf);
> + obj->btf = btf;
> + obj->base_btf = base_btf;
> + }
> +
> return 0;
>
> out_err:
> @@ -777,8 +784,6 @@ static int sets_patch(struct object *obj)
>
> static int symbols_patch(struct object *obj)
> {
> - off_t err;
> -
> if (__symbols_patch(obj, &obj->structs) ||
> __symbols_patch(obj, &obj->unions) ||
> __symbols_patch(obj, &obj->typedefs) ||
> @@ -789,20 +794,67 @@ static int symbols_patch(struct object *obj)
> if (sets_patch(obj))
> return -1;
>
> - /* Set type to ensure endian translation occurs. */
> - obj->efile.idlist->d_type = ELF_T_WORD;
> + return 0;
> +}
>
> - elf_flagdata(obj->efile.idlist, ELF_C_SET, ELF_F_DIRTY);
> +static int dump_raw_data(const char *out_path, const void *data, u32 size)
> +{
> + int fd, ret;
>
> - err = elf_update(obj->efile.elf, ELF_C_WRITE);
> - if (err < 0) {
> - pr_err("FAILED elf_update(WRITE): %s\n",
> - elf_errmsg(-1));
> + fd = open(out_path, O_WRONLY | O_CREAT | O_TRUNC, 0640);
> + if (fd < 0) {
> + pr_err("Couldn't open %s for writing\n", out_path);
> + return fd;
> + }
> +
> + ret = write(fd, data, size);
> + if (ret < 0 || ret != size) {
> + pr_err("Failed to write data to %s\n", out_path);
> + close(fd);
> + unlink(out_path);
> + return -1;
> + }
> +
> + close(fd);
> + pr_debug("Dumped %lu bytes of data to %s\n", size, out_path);
> +
> + return 0;
> +}
> +
> +static int dump_raw_btf_ids(struct object *obj, const char *out_path)
> +{
> + Elf_Data *data = obj->efile.idlist;
> + int fd, err;
> +
> + if (!data || !data->d_buf) {
> + pr_debug("%s has no BTF_ids data to dump\n", obj->path);
> + return 0;
> + }
> +
> + err = dump_raw_data(out_path, data->d_buf, data->d_size);
> + if (err)
> + return -1;
> +
> + return 0;
> +}
> +
> +static int dump_raw_btf(struct btf *btf, const char *out_path)
> +{
> + const void *raw_btf_data;
> + u32 raw_btf_size;
> + int fd, err;
> +
> + raw_btf_data = btf__raw_data(btf, &raw_btf_size);
> + if (raw_btf_data == NULL) {
> + pr_err("btf__raw_data() failed\n");
> + return -1;
> }
>
> - pr_debug("update %s for %s\n",
> - err >= 0 ? "ok" : "failed", obj->path);
> - return err < 0 ? -1 : 0;
> + err = dump_raw_data(out_path, raw_btf_data, raw_btf_size);
> + if (err)
> + return -1;
> +
> + return 0;
> }
>
> static const char * const resolve_btfids_usage[] = {
> @@ -823,12 +875,13 @@ int main(int argc, const char **argv)
> .funcs = RB_ROOT,
> .sets = RB_ROOT,
> };
> + char out_path[PATH_MAX];
> bool fatal_warnings = false;
> struct option btfid_options[] = {
> OPT_INCR('v', "verbose", &verbose,
> "be more verbose (show errors, etc)"),
> OPT_STRING(0, "btf", &obj.btf_path, "BTF data",
> - "BTF data"),
> + "input BTF data"),
> OPT_STRING('b', "btf_base", &obj.base_btf_path, "file",
> "path of file providing base BTF"),
> OPT_BOOLEAN(0, "fatal_warnings", &fatal_warnings,
> @@ -844,6 +897,9 @@ int main(int argc, const char **argv)
>
> obj.path = argv[0];
>
> + if (load_btf(&obj))
> + goto out;
I think I can add the BTF sorting function here based on your patch.
Thanks,
Donglin
> +
> if (elf_collect(&obj))
> goto out;
>
> @@ -853,23 +909,37 @@ int main(int argc, const char **argv)
> */
> if (obj.efile.idlist_shndx == -1 ||
> obj.efile.symbols_shndx == -1) {
> - pr_debug("Cannot find .BTF_ids or symbols sections, nothing to do\n");
> - err = 0;
> - goto out;
> + pr_debug("Cannot find .BTF_ids or symbols sections, skip symbols resolution\n");
> + goto dump_btf;
> }
>
> if (symbols_collect(&obj))
> goto out;
>
> - if (load_btf(&obj))
> - goto out;
> -
> if (symbols_resolve(&obj))
> goto out;
>
> if (symbols_patch(&obj))
> goto out;
>
> + strcpy(out_path, obj.path);
> + strcat(out_path, ".btf_ids");
> + if (dump_raw_btf_ids(&obj, out_path))
> + goto out;
> +
> +dump_btf:
> + strcpy(out_path, obj.path);
> + strcat(out_path, ".btf");
Do we need to add .btf files to the .gitignore file?
Thanks,
Donglin
> + if (dump_raw_btf(obj.btf, out_path))
> + goto out;
> +
> + if (obj.base_btf) {
> + strcpy(out_path, obj.path);
> + strcat(out_path, ".distilled_base.btf");
> + if (dump_raw_btf(obj.base_btf, out_path))
> + goto out;
> + }
> +
> if (!(fatal_warnings && warnings))
> err = 0;
> out:
> --
> 2.52.0
>
^ permalink raw reply [flat|nested] 16+ messages in thread* Re: [PATCH bpf-next v1 4/4] resolve_btfids: change in-place update with raw binary output
2025-11-26 4:46 ` Donglin Peng
@ 2025-11-26 18:22 ` Ihor Solodrai
2025-11-26 18:32 ` Ihor Solodrai
1 sibling, 0 replies; 16+ messages in thread
From: Ihor Solodrai @ 2025-11-26 18:22 UTC (permalink / raw)
To: Donglin Peng
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Song Liu, Yonghong Song,
John Fastabend, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa,
Nathan Chancellor, Nicolas Schier, Nick Desaulniers,
Bill Wendling, Justin Stitt, bpf, dwarves, linux-kernel,
linux-kbuild, Alan Maguire
On 11/25/25 8:46 PM, Donglin Peng wrote:
> On Wed, Nov 26, 2025 at 9:29 AM Ihor Solodrai <ihor.solodrai@linux.dev> wrote:
>>
>> [...]
>> +
>> +gen_btf_data()
>> +{
>> + info BTF "${ELF_FILE}"
>> + btf1="${ELF_FILE}.btf.1"
>> + ${PAHOLE} -J ${PAHOLE_FLAGS} \
>> + ${BTF_BASE:+--btf_base ${BTF_BASE}} \
>> + --btf_encode_detached=${btf1} \
>> + "${ELF_FILE}"
>> +
>> + info BTFIDS "${ELF_FILE}"
>> + RESOLVE_BTFIDS_OPTS=""
>> + if is_enabled CONFIG_WERROR; then
>> + RESOLVE_BTFIDS_OPTS+=" --fatal_warnings "
>
> In POSIX sh, +=is undefined[1], and I encountered the following error:
>
> ./scripts/gen-btf.sh: 90: RESOLVE_BTFIDS_OPTS+= --fatal_warnings : not found
>
> We should use the following syntax instead:
>
> RESOLVE_BTFIDS_OPTS="${RESOLVE_BTFIDS_OPTS} --fatal_warnings "
Hi Donglin, thanks for taking a look.
These and a couple of other bugs have been caught by CI [1].
I am working on v2.
I changed the script to #!/bin/bash and will run the shellcheck
before submitting the next revision [2], when it's ready.
[1] https://github.com/kernel-patches/bpf/actions/runs/19689674975
[2] https://github.com/kernel-patches/bpf/pull/10370
>
> [1] https://www.shellcheck.net/wiki/SC3024
>
> Thanks,
> Donglin
>> [...]
^ permalink raw reply [flat|nested] 16+ messages in thread* Re: [PATCH bpf-next v1 4/4] resolve_btfids: change in-place update with raw binary output
2025-11-26 4:46 ` Donglin Peng
2025-11-26 18:22 ` Ihor Solodrai
@ 2025-11-26 18:32 ` Ihor Solodrai
1 sibling, 0 replies; 16+ messages in thread
From: Ihor Solodrai @ 2025-11-26 18:32 UTC (permalink / raw)
To: Donglin Peng
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Song Liu, Yonghong Song,
John Fastabend, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa,
Nathan Chancellor, Nicolas Schier, Nick Desaulniers,
Bill Wendling, Justin Stitt, bpf, dwarves, linux-kernel,
linux-kbuild, Alan Maguire
On 11/25/25 8:46 PM, Donglin Peng wrote:
> On Wed, Nov 26, 2025 at 9:29 AM Ihor Solodrai <ihor.solodrai@linux.dev> wrote:
>>
>> [...]
>> static const char * const resolve_btfids_usage[] = {
>> @@ -823,12 +875,13 @@ int main(int argc, const char **argv)
>> .funcs = RB_ROOT,
>> .sets = RB_ROOT,
>> };
>> + char out_path[PATH_MAX];
>> bool fatal_warnings = false;
>> struct option btfid_options[] = {
>> OPT_INCR('v', "verbose", &verbose,
>> "be more verbose (show errors, etc)"),
>> OPT_STRING(0, "btf", &obj.btf_path, "BTF data",
>> - "BTF data"),
>> + "input BTF data"),
>> OPT_STRING('b', "btf_base", &obj.base_btf_path, "file",
>> "path of file providing base BTF"),
>> OPT_BOOLEAN(0, "fatal_warnings", &fatal_warnings,
>> @@ -844,6 +897,9 @@ int main(int argc, const char **argv)
>>
>> obj.path = argv[0];
>>
>> + if (load_btf(&obj))
>> + goto out;
>
> I think I can add the BTF sorting function here based on your patch.
Correct. Any btf2btf transformations will have to happen before the
symbols resolution.
>
> Thanks,
> Donglin
>> +
>> if (elf_collect(&obj))
>> goto out;
>>
>> @@ -853,23 +909,37 @@ int main(int argc, const char **argv)
>> */
>> if (obj.efile.idlist_shndx == -1 ||
>> obj.efile.symbols_shndx == -1) {
>> - pr_debug("Cannot find .BTF_ids or symbols sections, nothing to do\n");
>> - err = 0;
>> - goto out;
>> + pr_debug("Cannot find .BTF_ids or symbols sections, skip symbols resolution\n");
>> + goto dump_btf;
>> }
>>
>> if (symbols_collect(&obj))
>> goto out;
>>
>> - if (load_btf(&obj))
>> - goto out;
>> -
>> if (symbols_resolve(&obj))
>> goto out;
>>
>> if (symbols_patch(&obj))
>> goto out;
>>
>> + strcpy(out_path, obj.path);
>> + strcat(out_path, ".btf_ids");
>> + if (dump_raw_btf_ids(&obj, out_path))
>> + goto out;
>> +
>> +dump_btf:
>> + strcpy(out_path, obj.path);
>> + strcat(out_path, ".btf");
>
> Do we need to add .btf files to the .gitignore file?
I don't know. Probably?
I take care to cleanup temporary files in the gen-btf.sh, but it makes
sense to .gitignore them anyways, since those are temporary build
artifacts.
>
> Thanks,
> Donglin
>
>> + if (dump_raw_btf(obj.btf, out_path))
>> + goto out;
>> +
>> + if (obj.base_btf) {
>> + strcpy(out_path, obj.path);
>> + strcat(out_path, ".distilled_base.btf");
>> + if (dump_raw_btf(obj.base_btf, out_path))
>> + goto out;
>> + }
>> +
>> if (!(fatal_warnings && warnings))
>> err = 0;
>> out:
>> --
>> 2.52.0
>>
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH bpf-next v1 4/4] resolve_btfids: change in-place update with raw binary output
2025-11-26 1:26 ` [PATCH bpf-next v1 4/4] resolve_btfids: change in-place update with raw binary output Ihor Solodrai
2025-11-26 4:46 ` Donglin Peng
@ 2025-11-26 13:03 ` Donglin Peng
2025-11-26 19:13 ` Ihor Solodrai
1 sibling, 1 reply; 16+ messages in thread
From: Donglin Peng @ 2025-11-26 13:03 UTC (permalink / raw)
To: Ihor Solodrai
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Song Liu, Yonghong Song,
John Fastabend, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa,
Nathan Chancellor, Nicolas Schier, Nick Desaulniers,
Bill Wendling, Justin Stitt, bpf, dwarves, linux-kernel,
linux-kbuild, Alan Maguire
On Wed, Nov 26, 2025 at 9:29 AM Ihor Solodrai <ihor.solodrai@linux.dev> wrote:
>
> Currently resolve_btfids updates .BTF_ids section of an ELF file
> in-place, based on the contents of provided BTF, usually within the
> same input file, and optionally a BTF base.
>
> This patch changes resolve_btfids behavior to enable BTF
> transformations as part of its main operation. To achieve this
> in-place ELF write in resolve_btfids is replaced with generation of
> the following binaries:
> * ${1}.btf with .BTF section data
> * ${1}.distilled_base.btf with .BTF.base section data (for modules)
> * ${1}.btf_ids with .BTF_ids section data, if it exists in ${1}
>
> The execution of resolve_btfids and consumption of its output is
> orchestrated by scripts/gen-btf.sh introduced in this patch.
>
> The rationale for this approach is that updating ELF in-place with
> libelf API is complicated and bug-prone, especially in the context of
> the kernel build. On the other hand applying objcopy to manipulate ELF
> sections is simpler and more reliable.
>
> There are two distinct paths for BTF generation and resolve_btfids
> application in the kernel build: for vmlinux and for kernel modules.
>
> For the vmlinux binary a .BTF section is added in a roundabout way to
> ensure correct linking (details below). The patch doesn't change this
> approach, only the implementation is a little different.
>
> Before this patch it worked like follows:
>
> * pahole consumed .tmp_vmlinux1 [1] and added .BTF section with
> llvm-objcopy [2] to it
> * then everything except the .BTF section was stripped from .tmp_vmlinux1
> into a .tmp_vmlinux1.bpf.o object [1], later linked into vmlinux
> * resolve_btfids was executed later on vmlinux.unstripped [3],
> updating it in-place
>
> After this patch gen-btf.sh implements the following:
>
> * pahole consumes .tmp_vmlinux1 and produces a **detached** file
> with raw BTF data
> * resolve_btfids consumes .tmp_vmlinux1 and detached BTF to produce
> (potentially modified) .BTF, and .BTF_ids sections data
> * a .tmp_vmlinux1.bpf.o object is then produced with objcopy copying
> BTF output of resolve_btfids
> * .BTF_ids data gets embedded into vmlinux.unstripped in
> link-vmlinux.sh by objcopy --update-section
>
> For the kernel modules creating special .bpf.o file is not necessary,
> and so embedding of sections data produced by resolve_btfids is
> straightforward with the objcopy.
The Makefile for the bpf selftests also needs be updated too:
https://elixir.bootlin.com/linux/v6.18-rc7/source/tools/testing/selftests/bpf/Makefile#L708
This results in the self-test for resolve_btfids failing:
$./vmtest.sh -- ./test_progs -t resolve_btfids -v
...
test_resolve_btfids:PASS:id_check 0 nsec
test_resolve_btfids:FAIL:id_check wrong ID for S (0 != 3)
>
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n115
> [2] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tree/btf_encoder.c#n1835
> [3] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n285
>
> Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
> ---
> MAINTAINERS | 1 +
> scripts/Makefile.modfinal | 5 +-
> scripts/gen-btf.sh | 166 ++++++++++++++++++++++++++++++++
> scripts/link-vmlinux.sh | 42 ++------
> tools/bpf/resolve_btfids/main.c | 124 ++++++++++++++++++------
> 5 files changed, 272 insertions(+), 66 deletions(-)
> create mode 100755 scripts/gen-btf.sh
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 48aabeeed029..5cd34419d952 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -4672,6 +4672,7 @@ F: net/sched/act_bpf.c
> F: net/sched/cls_bpf.c
> F: samples/bpf/
> F: scripts/bpf_doc.py
> +F: scripts/gen-btf.sh
> F: scripts/Makefile.btf
> F: scripts/pahole-version.sh
> F: tools/bpf/
> diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal
> index 542ba462ed3e..86f843995556 100644
> --- a/scripts/Makefile.modfinal
> +++ b/scripts/Makefile.modfinal
> @@ -38,9 +38,8 @@ quiet_cmd_btf_ko = BTF [M] $@
> cmd_btf_ko = \
> if [ ! -f $(objtree)/vmlinux ]; then \
> printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \
> - else \
> - LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base $(objtree)/vmlinux $@; \
> - $(RESOLVE_BTFIDS) -b $(objtree)/vmlinux $@; \
> + else \
> + $(objtree)/scripts/gen-btf.sh --btf_base $(objtree)/vmlinux $@; \
> fi;
>
> # Same as newer-prereqs, but allows to exclude specified extra dependencies
> diff --git a/scripts/gen-btf.sh b/scripts/gen-btf.sh
> new file mode 100755
> index 000000000000..102f8296ae9e
> --- /dev/null
> +++ b/scripts/gen-btf.sh
> @@ -0,0 +1,166 @@
> +#!/bin/sh
> +# SPDX-License-Identifier: GPL-2.0
> +# Copyright (c) 2025 Meta Platforms, Inc. and affiliates.
> +#
> +# This script generates BTF data for the provided ELF file.
> +#
> +# Kernel BTF generation involves these conceptual steps:
> +# 1. pahole generates BTF from DWARF data
> +# 2. resolve_btfids applies kernel-specific btf2btf
> +# transformations and computes data for .BTF_ids section
> +# 3. the result gets linked/objcopied into the target binary
> +#
> +# How step (3) should be done differs between vmlinux, and
> +# kernel modules, which is the primary reason for the existence
> +# of this script.
> +#
> +# For modules the script expects vmlinux passed in as --btf_base.
> +# Generated .BTF, .BTF.base and .BTF_ids sections become embedded
> +# into the input ELF file with objcopy.
> +#
> +# For vmlinux the input file remains unchanged and two files are produced:
> +# - ${1}.btf.o ready for linking into vmlinux
> +# - ${1}.btf_ids with .BTF_ids data blob
> +# This output is consumed by scripts/link-vmlinux.sh
> +
> +set -e
> +
> +usage()
> +{
> + echo "Usage: $0 [--btf_base <file>] <target ELF file>"
> + exit 1
> +}
> +
> +BTF_BASE=""
> +
> +while [ $# -gt 0 ]; do
> + case "$1" in
> + --btf_base)
> + BTF_BASE="$2"
> + shift 2
> + ;;
> + -*)
> + echo "Unknown option: $1" >&2
> + usage
> + ;;
> + *)
> + break
> + ;;
> + esac
> +done
> +
> +if [ $# -ne 1 ]; then
> + usage
> +fi
> +
> +ELF_FILE="$1"
> +shift
> +
> +is_enabled() {
> + grep -q "^$1=y" ${objtree}/include/config/auto.conf
> +}
> +
> +info()
> +{
> + printf " %-7s %s\n" "${1}" "${2}"
> +}
> +
> +case "${KBUILD_VERBOSE}" in
> +*1*)
> + set -x
> + ;;
> +esac
> +
> +if ! is_enabled CONFIG_DEBUG_INFO_BTF; then
> + exit 0
> +fi
> +
> +gen_btf_data()
> +{
> + info BTF "${ELF_FILE}"
> + btf1="${ELF_FILE}.btf.1"
> + ${PAHOLE} -J ${PAHOLE_FLAGS} \
> + ${BTF_BASE:+--btf_base ${BTF_BASE}} \
> + --btf_encode_detached=${btf1} \
> + "${ELF_FILE}"
> +
> + info BTFIDS "${ELF_FILE}"
> + RESOLVE_BTFIDS_OPTS=""
> + if is_enabled CONFIG_WERROR; then
> + RESOLVE_BTFIDS_OPTS+=" --fatal_warnings "
> + fi
> + if [ -n "${KBUILD_VERBOSE}" ]; then
> + RESOLVE_BTFIDS_OPTS+=" -v "
> + fi
> + ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_OPTS} \
> + ${BTF_BASE:+--btf_base ${BTF_BASE}} \
> + --btf ${btf1} "${ELF_FILE}"
> +}
> +
> +gen_btf_o()
> +{
> + local btf_data=${ELF_FILE}.btf.o
> +
> + # Create ${btf_data} which contains just .BTF section but no symbols. Add
> + # SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all
> + # deletes all symbols including __start_BTF and __stop_BTF, which will
> + # be redefined in the linker script.
> + info OBJCOPY "${btf_data}"
> + echo "" | ${CC} -c -x c -o ${btf_data} -
> + ${OBJCOPY} --add-section .BTF=${ELF_FILE}.btf \
> + --set-section-flags .BTF=alloc,readonly ${btf_data}
> + ${OBJCOPY} --only-section=.BTF --strip-all ${btf_data}
> +
> + # Change e_type to ET_REL so that it can be used to link final vmlinux.
> + # GNU ld 2.35+ and lld do not allow an ET_EXEC input.
> + if is_enabled CONFIG_CPU_BIG_ENDIAN; then
> + et_rel='\0\1'
> + else
> + et_rel='\1\0'
> + fi
> + printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none
> +}
> +
> +embed_btf_data()
> +{
> + info OBJCOPY "${ELF_FILE}"
> +
> + ${OBJCOPY} \
> + --add-section .BTF=${ELF_FILE}.btf \
> + --add-section .BTF.base=${ELF_FILE}.distilled_base.btf \
> + ${ELF_FILE}
> +
> + # a module might not have a .BTF_ids section
> + if [ -f "${ELF_FILE}.btf_ids" ]; then
> + ${OBJCOPY} --update-section .BTF_ids=${ELF_FILE}.btf_ids ${ELF_FILE}
> + fi
> +}
> +
> +cleanup()
> +{
> + rm -f "${ELF_FILE}.btf.1"
> + rm -f "${ELF_FILE}.btf"
> + if [ "${BTFGEN_MODE}" == "module" ]; then
> + rm -f "${ELF_FILE}.distilled_base.btf"
> + rm -f "${ELF_FILE}.btf_ids"
> + fi
> +}
> +trap cleanup EXIT
> +
> +BTFGEN_MODE="vmlinux"
> +if [ -n "${BTF_BASE}" ]; then
> + BTFGEN_MODE="module"
> +fi
> +
> +gen_btf_data
> +
> +case "${BTFGEN_MODE}" in
> +vmlinux)
> + gen_btf_o
> + ;;
> +module)
> + embed_btf_data
> + ;;
> +esac
> +
> +exit 0
> diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
> index 433849ff7529..728f82af24f6 100755
> --- a/scripts/link-vmlinux.sh
> +++ b/scripts/link-vmlinux.sh
> @@ -105,34 +105,6 @@ vmlinux_link()
> ${kallsymso} ${btf_vmlinux_bin_o} ${arch_vmlinux_o} ${ldlibs}
> }
>
> -# generate .BTF typeinfo from DWARF debuginfo
> -# ${1} - vmlinux image
> -gen_btf()
> -{
> - local btf_data=${1}.btf.o
> -
> - info BTF "${btf_data}"
> - LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS} ${1}
> -
> - # Create ${btf_data} which contains just .BTF section but no symbols. Add
> - # SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all
> - # deletes all symbols including __start_BTF and __stop_BTF, which will
> - # be redefined in the linker script. Add 2>/dev/null to suppress GNU
> - # objcopy warnings: "empty loadable segment detected at ..."
> - ${OBJCOPY} --only-section=.BTF --set-section-flags .BTF=alloc,readonly \
> - --strip-all ${1} "${btf_data}" 2>/dev/null
> - # Change e_type to ET_REL so that it can be used to link final vmlinux.
> - # GNU ld 2.35+ and lld do not allow an ET_EXEC input.
> - if is_enabled CONFIG_CPU_BIG_ENDIAN; then
> - et_rel='\0\1'
> - else
> - et_rel='\1\0'
> - fi
> - printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none
> -
> - btf_vmlinux_bin_o=${btf_data}
> -}
> -
> # Create ${2}.o file with all symbols from the ${1} object file
> kallsyms()
> {
> @@ -204,6 +176,7 @@ if is_enabled CONFIG_ARCH_WANTS_PRE_LINK_VMLINUX; then
> fi
>
> btf_vmlinux_bin_o=
> +btfids_vmlinux=
> kallsymso=
> strip_debug=
> generate_map=
> @@ -224,11 +197,13 @@ if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then
> fi
>
> if is_enabled CONFIG_DEBUG_INFO_BTF; then
> - if ! gen_btf .tmp_vmlinux1; then
> + if ! scripts/gen-btf.sh .tmp_vmlinux1; then
> echo >&2 "Failed to generate BTF for vmlinux"
> echo >&2 "Try to disable CONFIG_DEBUG_INFO_BTF"
> exit 1
> fi
> + btf_vmlinux_bin_o=.tmp_vmlinux1.btf.o
> + btfids_vmlinux=.tmp_vmlinux1.btf_ids
> fi
>
> if is_enabled CONFIG_KALLSYMS; then
> @@ -281,14 +256,9 @@ fi
>
> vmlinux_link "${VMLINUX}"
>
> -# fill in BTF IDs
> if is_enabled CONFIG_DEBUG_INFO_BTF; then
> - info BTFIDS "${VMLINUX}"
> - RESOLVE_BTFIDS_ARGS=""
> - if is_enabled CONFIG_WERROR; then
> - RESOLVE_BTFIDS_ARGS=" --fatal_warnings "
> - fi
> - ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_ARGS} "${VMLINUX}"
> + info OBJCOPY ${btfids_vmlinux}
> + ${OBJCOPY} --update-section .BTF_ids=${btfids_vmlinux} ${VMLINUX}
> fi
>
> mksysmap "${VMLINUX}" System.map
> diff --git a/tools/bpf/resolve_btfids/main.c b/tools/bpf/resolve_btfids/main.c
> index 7f5a9f7dde7f..4faf16b1ba6b 100644
> --- a/tools/bpf/resolve_btfids/main.c
> +++ b/tools/bpf/resolve_btfids/main.c
> @@ -71,9 +71,11 @@
> #include <fcntl.h>
> #include <errno.h>
> #include <linux/btf_ids.h>
> +#include <linux/kallsyms.h>
> #include <linux/rbtree.h>
> #include <linux/zalloc.h>
> #include <linux/err.h>
> +#include <linux/limits.h>
> #include <bpf/btf.h>
> #include <bpf/libbpf.h>
> #include <subcmd/parse-options.h>
> @@ -429,14 +431,6 @@ static int elf_collect(struct object *obj)
> obj->efile.idlist = data;
> obj->efile.idlist_shndx = idx;
> obj->efile.idlist_addr = sh.sh_addr;
> - } else if (!strcmp(name, BTF_BASE_ELF_SEC)) {
> - /* If a .BTF.base section is found, do not resolve
> - * BTF ids relative to vmlinux; resolve relative
> - * to the .BTF.base section instead. btf__parse_split()
> - * will take care of this once the base BTF it is
> - * passed is NULL.
> - */
> - obj->base_btf_path = NULL;
> }
>
> if (compressed_section_fix(elf, scn, &sh))
> @@ -570,6 +564,19 @@ static int load_btf(struct object *obj)
> obj->base_btf = base_btf;
> obj->btf = btf;
>
> + if (obj->base_btf) {
> + err = btf__distill_base(obj->btf, &base_btf, &btf);
> + if (err) {
> + pr_err("FAILED to distill base BTF: %s\n", strerror(errno));
> + goto out_err;
> + }
> +
> + btf__free(obj->btf);
> + btf__free(obj->base_btf);
> + obj->btf = btf;
> + obj->base_btf = base_btf;
> + }
> +
> return 0;
>
> out_err:
> @@ -777,8 +784,6 @@ static int sets_patch(struct object *obj)
>
> static int symbols_patch(struct object *obj)
> {
> - off_t err;
> -
> if (__symbols_patch(obj, &obj->structs) ||
> __symbols_patch(obj, &obj->unions) ||
> __symbols_patch(obj, &obj->typedefs) ||
> @@ -789,20 +794,67 @@ static int symbols_patch(struct object *obj)
> if (sets_patch(obj))
> return -1;
>
> - /* Set type to ensure endian translation occurs. */
> - obj->efile.idlist->d_type = ELF_T_WORD;
> + return 0;
> +}
>
> - elf_flagdata(obj->efile.idlist, ELF_C_SET, ELF_F_DIRTY);
> +static int dump_raw_data(const char *out_path, const void *data, u32 size)
> +{
> + int fd, ret;
>
> - err = elf_update(obj->efile.elf, ELF_C_WRITE);
> - if (err < 0) {
> - pr_err("FAILED elf_update(WRITE): %s\n",
> - elf_errmsg(-1));
> + fd = open(out_path, O_WRONLY | O_CREAT | O_TRUNC, 0640);
> + if (fd < 0) {
> + pr_err("Couldn't open %s for writing\n", out_path);
> + return fd;
> + }
> +
> + ret = write(fd, data, size);
> + if (ret < 0 || ret != size) {
> + pr_err("Failed to write data to %s\n", out_path);
> + close(fd);
> + unlink(out_path);
> + return -1;
> + }
> +
> + close(fd);
> + pr_debug("Dumped %lu bytes of data to %s\n", size, out_path);
> +
> + return 0;
> +}
> +
> +static int dump_raw_btf_ids(struct object *obj, const char *out_path)
> +{
> + Elf_Data *data = obj->efile.idlist;
> + int fd, err;
> +
> + if (!data || !data->d_buf) {
> + pr_debug("%s has no BTF_ids data to dump\n", obj->path);
> + return 0;
> + }
> +
> + err = dump_raw_data(out_path, data->d_buf, data->d_size);
> + if (err)
> + return -1;
> +
> + return 0;
> +}
> +
> +static int dump_raw_btf(struct btf *btf, const char *out_path)
> +{
> + const void *raw_btf_data;
> + u32 raw_btf_size;
> + int fd, err;
> +
> + raw_btf_data = btf__raw_data(btf, &raw_btf_size);
> + if (raw_btf_data == NULL) {
> + pr_err("btf__raw_data() failed\n");
> + return -1;
> }
>
> - pr_debug("update %s for %s\n",
> - err >= 0 ? "ok" : "failed", obj->path);
> - return err < 0 ? -1 : 0;
> + err = dump_raw_data(out_path, raw_btf_data, raw_btf_size);
> + if (err)
> + return -1;
> +
> + return 0;
> }
>
> static const char * const resolve_btfids_usage[] = {
> @@ -823,12 +875,13 @@ int main(int argc, const char **argv)
> .funcs = RB_ROOT,
> .sets = RB_ROOT,
> };
> + char out_path[PATH_MAX];
> bool fatal_warnings = false;
> struct option btfid_options[] = {
> OPT_INCR('v', "verbose", &verbose,
> "be more verbose (show errors, etc)"),
> OPT_STRING(0, "btf", &obj.btf_path, "BTF data",
> - "BTF data"),
> + "input BTF data"),
> OPT_STRING('b', "btf_base", &obj.base_btf_path, "file",
> "path of file providing base BTF"),
> OPT_BOOLEAN(0, "fatal_warnings", &fatal_warnings,
> @@ -844,6 +897,9 @@ int main(int argc, const char **argv)
>
> obj.path = argv[0];
>
> + if (load_btf(&obj))
> + goto out;
> +
> if (elf_collect(&obj))
> goto out;
>
> @@ -853,23 +909,37 @@ int main(int argc, const char **argv)
> */
> if (obj.efile.idlist_shndx == -1 ||
> obj.efile.symbols_shndx == -1) {
> - pr_debug("Cannot find .BTF_ids or symbols sections, nothing to do\n");
> - err = 0;
> - goto out;
> + pr_debug("Cannot find .BTF_ids or symbols sections, skip symbols resolution\n");
> + goto dump_btf;
> }
>
> if (symbols_collect(&obj))
> goto out;
>
> - if (load_btf(&obj))
> - goto out;
> -
> if (symbols_resolve(&obj))
> goto out;
>
> if (symbols_patch(&obj))
> goto out;
>
> + strcpy(out_path, obj.path);
> + strcat(out_path, ".btf_ids");
> + if (dump_raw_btf_ids(&obj, out_path))
> + goto out;
> +
> +dump_btf:
> + strcpy(out_path, obj.path);
> + strcat(out_path, ".btf");
> + if (dump_raw_btf(obj.btf, out_path))
> + goto out;
> +
> + if (obj.base_btf) {
> + strcpy(out_path, obj.path);
> + strcat(out_path, ".distilled_base.btf");
> + if (dump_raw_btf(obj.base_btf, out_path))
> + goto out;
> + }
> +
> if (!(fatal_warnings && warnings))
> err = 0;
> out:
> --
> 2.52.0
>
^ permalink raw reply [flat|nested] 16+ messages in thread* Re: [PATCH bpf-next v1 4/4] resolve_btfids: change in-place update with raw binary output
2025-11-26 13:03 ` Donglin Peng
@ 2025-11-26 19:13 ` Ihor Solodrai
2025-11-26 23:58 ` Ihor Solodrai
0 siblings, 1 reply; 16+ messages in thread
From: Ihor Solodrai @ 2025-11-26 19:13 UTC (permalink / raw)
To: Donglin Peng
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Song Liu, Yonghong Song,
John Fastabend, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa,
Nathan Chancellor, Nicolas Schier, Nick Desaulniers,
Bill Wendling, Justin Stitt, bpf, dwarves, linux-kernel,
linux-kbuild, Alan Maguire
On 11/26/25 5:03 AM, Donglin Peng wrote:
> On Wed, Nov 26, 2025 at 9:29 AM Ihor Solodrai <ihor.solodrai@linux.dev> wrote:
>>
>> [...]
>>
>> For the kernel modules creating special .bpf.o file is not necessary,
>> and so embedding of sections data produced by resolve_btfids is
>> straightforward with the objcopy.
>
> The Makefile for the bpf selftests also needs be updated too:
> https://elixir.bootlin.com/linux/v6.18-rc7/source/tools/testing/selftests/bpf/Makefile#L708
>
> This results in the self-test for resolve_btfids failing:
> $./vmtest.sh -- ./test_progs -t resolve_btfids -v
> ...
> test_resolve_btfids:PASS:id_check 0 nsec
> test_resolve_btfids:FAIL:id_check wrong ID for S (0 != 3)
Good catch, thanks.
I remember I noticed this at some point, and then forgot...
Interestingly this test passes on CI [1]:
2025-11-26T03:09:52.0908317Z #366 reg_bounds_rand_ranges_u64_u64:OK
2025-11-26T03:09:52.0925114Z #367 resolve_btfids:OK
2025-11-26T03:09:52.3904190Z #368/1 res_spin_lock_failure/res_spin_lock_arg:OK
I'll take a closer look.
[1] https://github.com/kernel-patches/bpf/actions/runs/19690981192/job/56406840021
>
>
>> [...]
>>
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH bpf-next v1 4/4] resolve_btfids: change in-place update with raw binary output
2025-11-26 19:13 ` Ihor Solodrai
@ 2025-11-26 23:58 ` Ihor Solodrai
0 siblings, 0 replies; 16+ messages in thread
From: Ihor Solodrai @ 2025-11-26 23:58 UTC (permalink / raw)
To: Donglin Peng
Cc: Alexei Starovoitov, Daniel Borkmann, Andrii Nakryiko,
Martin KaFai Lau, Eduard Zingerman, Song Liu, Yonghong Song,
John Fastabend, KP Singh, Stanislav Fomichev, Hao Luo, Jiri Olsa,
Nathan Chancellor, Nicolas Schier, Nick Desaulniers,
Bill Wendling, Justin Stitt, bpf, dwarves, linux-kernel,
linux-kbuild, Alan Maguire
On 11/26/25 11:13 AM, Ihor Solodrai wrote:
> On 11/26/25 5:03 AM, Donglin Peng wrote:
>> On Wed, Nov 26, 2025 at 9:29 AM Ihor Solodrai <ihor.solodrai@linux.dev> wrote:
>>>
>>> [...]
>>>
>>> For the kernel modules creating special .bpf.o file is not necessary,
>>> and so embedding of sections data produced by resolve_btfids is
>>> straightforward with the objcopy.
>>
>> The Makefile for the bpf selftests also needs be updated too:
>> https://elixir.bootlin.com/linux/v6.18-rc7/source/tools/testing/selftests/bpf/Makefile#L708
>>
>> This results in the self-test for resolve_btfids failing:
>> $./vmtest.sh -- ./test_progs -t resolve_btfids -v
>> ...
>> test_resolve_btfids:PASS:id_check 0 nsec
>> test_resolve_btfids:FAIL:id_check wrong ID for S (0 != 3)
>
> Good catch, thanks.
>
> I remember I noticed this at some point, and then forgot...
>
> Interestingly this test passes on CI [1]:
>
> 2025-11-26T03:09:52.0908317Z #366 reg_bounds_rand_ranges_u64_u64:OK
> 2025-11-26T03:09:52.0925114Z #367 resolve_btfids:OK
> 2025-11-26T03:09:52.3904190Z #368/1 res_spin_lock_failure/res_spin_lock_arg:OK
>
> I'll take a closer look.
I figured out why this test was flaky.
Even though I removed elf_update() call from resolve_btfids, the ELF
was opened with:
elf = elf_begin(fd, ELF_C_RDWR_MMAP, NULL);
And the buffers which resolve_btfids writes to are from Elf_Data
returned by elf_getdata(). And so the file might actually get written
to in-place, which is why the resolve_btfids test passed for me with
no changes to the selftests.
I switched ELF_C_RDWR_MMAP to ELF_C_READ_MMAP_PRIVATE, and then the
ELF reliably remains intact (and the test fails). From libelf.h:
ELF_C_READ_MMAP_PRIVATE, /* Read, but memory is writable, results are
not written to the file. */
It makes sense to use this for what resolve_btfids is doing.
I'll fix selftests/bpf/Makefile in the next revision.
>
> [1] https://github.com/kernel-patches/bpf/actions/runs/19690981192/job/56406840021
>
>>
>>
>>> [...]
>>>
>
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH bpf-next v1 0/4] resolve_btfids: Support for BTF modifications
2025-11-26 1:26 [PATCH bpf-next v1 0/4] resolve_btfids: Support for BTF modifications Ihor Solodrai
` (3 preceding siblings ...)
2025-11-26 1:26 ` [PATCH bpf-next v1 4/4] resolve_btfids: change in-place update with raw binary output Ihor Solodrai
@ 2025-11-26 12:36 ` Alan Maguire
2025-11-26 19:01 ` Ihor Solodrai
4 siblings, 1 reply; 16+ messages in thread
From: Alan Maguire @ 2025-11-26 12:36 UTC (permalink / raw)
To: Ihor Solodrai, Alexei Starovoitov, Daniel Borkmann,
Andrii Nakryiko, Martin KaFai Lau, Eduard Zingerman, Song Liu,
Yonghong Song, John Fastabend, KP Singh, Stanislav Fomichev,
Hao Luo, Jiri Olsa, Nathan Chancellor, Nicolas Schier,
Nick Desaulniers, Bill Wendling, Justin Stitt
Cc: bpf, dwarves, linux-kernel, linux-kbuild, Donglin Peng
On 26/11/2025 01:26, Ihor Solodrai wrote:
> This series changes resolve_btfids and kernel build scripts to enable
> BTF transformations in resolve_btfids. Main motivation for enhancing
> resolve_btfids is to reduce dependency of the kernel build on pahole
> capabilities [1] and enable BTF features and optimizations [2][3]
> particular to the kernel.
>
> Patches #1-#3 in the series are non-functional refactoring in
> resolve_btfids. The last patch (#4) makes significant changes in
> resolve_btfids and introduces scripts/gen-btf.sh. Implementation
> changes are described in detail in the patch description.
>
> One RFC item in this patchset is the --distilled_base [4] handling.
> Before this patchset .BTF.base was generated and added to target
> binary by pahole, based on these conditions [5]:
> * pahole version >=1.28
> * the kernel module is out-of-tree (KBUILD_EXTMOD)
>
> Since BTF finalization is now done by resolve_btfids, it requires
> btf__distill_base() to happen there. However, in my opinion, it is
> unnecessary to add and pass through a --distilled_base flag for
> resolve_btfids.
>
hi Ihor,
Can you say more about what constitutes BTF finalization and why BTF
distillation prior to finalization (i.e. in pahole) isn't workable? Is
it the concern that we eliminate types due to filtering, or is it a
problem with sorting/tracking type ids? Are there operations we
do/anticipate that make prior distillation infeasbile? Thanks!
> Logically, any split BTF referring to kernel BTF is not very useful
> without the .BTF.base, which is why the feature was developed in the
> first place. Therefore it makes sense to always emit .BTF.base for all
> modules, unconditionally. This is implemented in the series.
>
> However it might be argued that .BTF.base is redundant for in-tree
> modules: it takes space the module ELF and triggers unnecessary
> btf__relocate() call on load [6]. It can be avoided by special-casing
> in-tree module handling in resolve_btfids either with a flag or by
> checking env variables. The trade-off is slight performance impact vs
> code complexity.
>
I would say avoid distillation for in-tree modules if possible, as it
imposes runtime costs in relocation/type renumbering on module load. For
large modules (amdgpu take a bow) that could be non-trivial time-wise.
IMO the build-time costs/complexities are worth paying to avoid a
runtime tax on module load.
> [1] https://lore.kernel.org/dwarves/ba1650aa-fafd-49a8-bea4-bdddee7c38c9@linux.dev/
> [2] https://lore.kernel.org/bpf/20251029190113.3323406-1-ihor.solodrai@linux.dev/
> [3] https://lore.kernel.org/bpf/20251119031531.1817099-1-dolinux.peng@gmail.com/
> [4] https://docs.kernel.org/bpf/btf.html#btf-base-section
> [5] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/Makefile.btf#n29
> [6] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/kernel/bpf/btf.c#n6358
>
> Ihor Solodrai (4):
> resolve_btfids: rename object btf field to btf_path
> resolve_btfids: factor out load_btf()
> resolve_btfids: introduce enum btf_id_kind
> resolve_btfids: change in-place update with raw binary output
>
> MAINTAINERS | 1 +
> scripts/Makefile.modfinal | 5 +-
> scripts/gen-btf.sh | 166 ++++++++++++++++++++++
> scripts/link-vmlinux.sh | 42 +-----
> tools/bpf/resolve_btfids/main.c | 234 +++++++++++++++++++++++---------
> 5 files changed, 348 insertions(+), 100 deletions(-)
> create mode 100755 scripts/gen-btf.sh
>
^ permalink raw reply [flat|nested] 16+ messages in thread* Re: [PATCH bpf-next v1 0/4] resolve_btfids: Support for BTF modifications
2025-11-26 12:36 ` [PATCH bpf-next v1 0/4] resolve_btfids: Support for BTF modifications Alan Maguire
@ 2025-11-26 19:01 ` Ihor Solodrai
2025-12-02 12:56 ` Alan Maguire
0 siblings, 1 reply; 16+ messages in thread
From: Ihor Solodrai @ 2025-11-26 19:01 UTC (permalink / raw)
To: Alan Maguire, Alexei Starovoitov, Daniel Borkmann,
Andrii Nakryiko, Martin KaFai Lau, Eduard Zingerman, Song Liu,
Yonghong Song, John Fastabend, KP Singh, Stanislav Fomichev,
Hao Luo, Jiri Olsa, Nathan Chancellor, Nicolas Schier,
Nick Desaulniers, Bill Wendling, Justin Stitt
Cc: bpf, dwarves, linux-kernel, linux-kbuild, Donglin Peng
On 11/26/25 4:36 AM, Alan Maguire wrote:
> On 26/11/2025 01:26, Ihor Solodrai wrote:
>> This series changes resolve_btfids and kernel build scripts to enable
>> BTF transformations in resolve_btfids. Main motivation for enhancing
>> resolve_btfids is to reduce dependency of the kernel build on pahole
>> capabilities [1] and enable BTF features and optimizations [2][3]
>> particular to the kernel.
>>
>> Patches #1-#3 in the series are non-functional refactoring in
>> resolve_btfids. The last patch (#4) makes significant changes in
>> resolve_btfids and introduces scripts/gen-btf.sh. Implementation
>> changes are described in detail in the patch description.
>>
>> One RFC item in this patchset is the --distilled_base [4] handling.
>> Before this patchset .BTF.base was generated and added to target
>> binary by pahole, based on these conditions [5]:
>> * pahole version >=1.28
>> * the kernel module is out-of-tree (KBUILD_EXTMOD)
>>
>> Since BTF finalization is now done by resolve_btfids, it requires
>> btf__distill_base() to happen there. However, in my opinion, it is
>> unnecessary to add and pass through a --distilled_base flag for
>> resolve_btfids.
>>
> hi Ihor,
>
> Can you say more about what constitutes BTF finalization and why BTF
> distillation prior to finalization (i.e. in pahole) isn't workable? Is
> it the concern that we eliminate types due to filtering, or is it a
> problem with sorting/tracking type ids? Are there operations we
> do/anticipate that make prior distillation infeasbile? Thanks!
Hi Alan,
That's a good question. AFAIU the distillation should be done on the
final BTF, after all the transformations (sorting, adding/removing BTF
types) have been applied. At least this way we can be sure that the
distilled base is valid.
We certainly want BTF generation process to be the same for modules
and the kernel, which means that BTF modifications in resolve_btfids
have to be applied to module BTF also.
So the question is whether btf2btf will be safe to do *after*
distillation, and that of course depends on the specifics.
Let's say pahole generated BTF for a module and a distilled base. If
later some types are removed from module BTF, or a new type is added
(that might refer to a type absent in distilled base), is the btf/base
pair still valid?
My intuition is that it is more reliable to distill the final-final
BTF, and so with resolve_btfids taking over kernel BTF finalization it
makes sense to do it there. Otherwise we may be upfront limiting
ourselves in how module BTF can be changed in resolve_btfids.
What are the reasons to keep distillation in pahole? It's a simple
libbpf API call after all. Anything I might be missing?
>
>> Logically, any split BTF referring to kernel BTF is not very useful
>> without the .BTF.base, which is why the feature was developed in the
>> first place. Therefore it makes sense to always emit .BTF.base for all
>> modules, unconditionally. This is implemented in the series.
>>
>> However it might be argued that .BTF.base is redundant for in-tree
>> modules: it takes space the module ELF and triggers unnecessary
>> btf__relocate() call on load [6]. It can be avoided by special-casing
>> in-tree module handling in resolve_btfids either with a flag or by
>> checking env variables. The trade-off is slight performance impact vs
>> code complexity.
>>
>
> I would say avoid distillation for in-tree modules if possible, as it
> imposes runtime costs in relocation/type renumbering on module load. For
> large modules (amdgpu take a bow) that could be non-trivial time-wise.
> IMO the build-time costs/complexities are worth paying to avoid a
> runtime tax on module load.
Acked. I still would like to avoid passing flags around if possible.
Is it reasonable to simply check for KBUILD_EXTMOD env var from
withing resolve_btfids? Any drawbacks to that?
Thanks.
>
>> [1] https://lore.kernel.org/dwarves/ba1650aa-fafd-49a8-bea4-bdddee7c38c9@linux.dev/
>> [2] https://lore.kernel.org/bpf/20251029190113.3323406-1-ihor.solodrai@linux.dev/
>> [3] https://lore.kernel.org/bpf/20251119031531.1817099-1-dolinux.peng@gmail.com/
>> [4] https://docs.kernel.org/bpf/btf.html#btf-base-section
>> [5] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/Makefile.btf#n29
>> [6] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/kernel/bpf/btf.c#n6358
>>
>> Ihor Solodrai (4):
>> resolve_btfids: rename object btf field to btf_path
>> resolve_btfids: factor out load_btf()
>> resolve_btfids: introduce enum btf_id_kind
>> resolve_btfids: change in-place update with raw binary output
>>
>> MAINTAINERS | 1 +
>> scripts/Makefile.modfinal | 5 +-
>> scripts/gen-btf.sh | 166 ++++++++++++++++++++++
>> scripts/link-vmlinux.sh | 42 +-----
>> tools/bpf/resolve_btfids/main.c | 234 +++++++++++++++++++++++---------
>> 5 files changed, 348 insertions(+), 100 deletions(-)
>> create mode 100755 scripts/gen-btf.sh
>>
>
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH bpf-next v1 0/4] resolve_btfids: Support for BTF modifications
2025-11-26 19:01 ` Ihor Solodrai
@ 2025-12-02 12:56 ` Alan Maguire
0 siblings, 0 replies; 16+ messages in thread
From: Alan Maguire @ 2025-12-02 12:56 UTC (permalink / raw)
To: Ihor Solodrai, Alexei Starovoitov, Daniel Borkmann,
Andrii Nakryiko, Martin KaFai Lau, Eduard Zingerman, Song Liu,
Yonghong Song, John Fastabend, KP Singh, Stanislav Fomichev,
Hao Luo, Jiri Olsa, Nathan Chancellor, Nicolas Schier,
Nick Desaulniers, Bill Wendling, Justin Stitt
Cc: bpf, dwarves, linux-kernel, linux-kbuild, Donglin Peng
On 26/11/2025 19:01, Ihor Solodrai wrote:
> On 11/26/25 4:36 AM, Alan Maguire wrote:
>> On 26/11/2025 01:26, Ihor Solodrai wrote:
>>> This series changes resolve_btfids and kernel build scripts to enable
>>> BTF transformations in resolve_btfids. Main motivation for enhancing
>>> resolve_btfids is to reduce dependency of the kernel build on pahole
>>> capabilities [1] and enable BTF features and optimizations [2][3]
>>> particular to the kernel.
>>>
>>> Patches #1-#3 in the series are non-functional refactoring in
>>> resolve_btfids. The last patch (#4) makes significant changes in
>>> resolve_btfids and introduces scripts/gen-btf.sh. Implementation
>>> changes are described in detail in the patch description.
>>>
>>> One RFC item in this patchset is the --distilled_base [4] handling.
>>> Before this patchset .BTF.base was generated and added to target
>>> binary by pahole, based on these conditions [5]:
>>> * pahole version >=1.28
>>> * the kernel module is out-of-tree (KBUILD_EXTMOD)
>>>
>>> Since BTF finalization is now done by resolve_btfids, it requires
>>> btf__distill_base() to happen there. However, in my opinion, it is
>>> unnecessary to add and pass through a --distilled_base flag for
>>> resolve_btfids.
>>>
>> hi Ihor,
>>
>> Can you say more about what constitutes BTF finalization and why BTF
>> distillation prior to finalization (i.e. in pahole) isn't workable? Is
>> it the concern that we eliminate types due to filtering, or is it a
>> problem with sorting/tracking type ids? Are there operations we
>> do/anticipate that make prior distillation infeasbile? Thanks!
>
> Hi Alan,
>
> That's a good question. AFAIU the distillation should be done on the
> final BTF, after all the transformations (sorting, adding/removing BTF
> types) have been applied. At least this way we can be sure that the
> distilled base is valid.
>
> We certainly want BTF generation process to be the same for modules
> and the kernel, which means that BTF modifications in resolve_btfids
> have to be applied to module BTF also.
>
> So the question is whether btf2btf will be safe to do *after*
> distillation, and that of course depends on the specifics.
>
> Let's say pahole generated BTF for a module and a distilled base. If
> later some types are removed from module BTF, or a new type is added
> (that might refer to a type absent in distilled base), is the btf/base
> pair still valid?
>
> My intuition is that it is more reliable to distill the final-final
> BTF, and so with resolve_btfids taking over kernel BTF finalization it
> makes sense to do it there. Otherwise we may be upfront limiting
> ourselves in how module BTF can be changed in resolve_btfids.
>
> What are the reasons to keep distillation in pahole? It's a simple
> libbpf API call after all. Anything I might be missing?
>
>
Nope, I think doing distillation as the final step makes sense to me
too since we get a clearer picture of the types that the module references.
My only reservation was applying distillation to every module (rather
than just out-of-tree builds) but it sounds like that's not needed from below.
>>
>>> Logically, any split BTF referring to kernel BTF is not very useful
>>> without the .BTF.base, which is why the feature was developed in the
>>> first place. Therefore it makes sense to always emit .BTF.base for all
>>> modules, unconditionally. This is implemented in the series.
>>>
>>> However it might be argued that .BTF.base is redundant for in-tree
>>> modules: it takes space the module ELF and triggers unnecessary
>>> btf__relocate() call on load [6]. It can be avoided by special-casing
>>> in-tree module handling in resolve_btfids either with a flag or by
>>> checking env variables. The trade-off is slight performance impact vs
>>> code complexity.
>>>
>>
>> I would say avoid distillation for in-tree modules if possible, as it
>> imposes runtime costs in relocation/type renumbering on module load. For
>> large modules (amdgpu take a bow) that could be non-trivial time-wise.
>> IMO the build-time costs/complexities are worth paying to avoid a
>> runtime tax on module load.
>
> Acked. I still would like to avoid passing flags around if possible.
>
> Is it reasonable to simply check for KBUILD_EXTMOD env var from
> withing resolve_btfids? Any drawbacks to that?
>
None that I can think of; sounds like a good approach to me and is basically
equivalent to what we do now. I'll take a look at v2. Thanks!
Alan
> Thanks.
>
>
>>
>>> [1] https://lore.kernel.org/dwarves/ba1650aa-fafd-49a8-bea4-bdddee7c38c9@linux.dev/
>>> [2] https://lore.kernel.org/bpf/20251029190113.3323406-1-ihor.solodrai@linux.dev/
>>> [3] https://lore.kernel.org/bpf/20251119031531.1817099-1-dolinux.peng@gmail.com/
>>> [4] https://docs.kernel.org/bpf/btf.html#btf-base-section
>>> [5] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/Makefile.btf#n29
>>> [6] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/kernel/bpf/btf.c#n6358
>>>
>>> Ihor Solodrai (4):
>>> resolve_btfids: rename object btf field to btf_path
>>> resolve_btfids: factor out load_btf()
>>> resolve_btfids: introduce enum btf_id_kind
>>> resolve_btfids: change in-place update with raw binary output
>>>
>>> MAINTAINERS | 1 +
>>> scripts/Makefile.modfinal | 5 +-
>>> scripts/gen-btf.sh | 166 ++++++++++++++++++++++
>>> scripts/link-vmlinux.sh | 42 +-----
>>> tools/bpf/resolve_btfids/main.c | 234 +++++++++++++++++++++++---------
>>> 5 files changed, 348 insertions(+), 100 deletions(-)
>>> create mode 100755 scripts/gen-btf.sh
>>>
>>
>
^ permalink raw reply [flat|nested] 16+ messages in thread