* [PATCH bpf-next v10 00/13] Improve the performance of BTF type lookups with binary search
@ 2025-12-18 11:30 Donglin Peng
2025-12-18 11:30 ` [PATCH bpf-next v10 01/13] libbpf: Add BTF permutation support for type reordering Donglin Peng
` (12 more replies)
0 siblings, 13 replies; 72+ messages in thread
From: Donglin Peng @ 2025-12-18 11:30 UTC (permalink / raw)
To: ast, andrii.nakryiko, eddyz87
Cc: zhangxiaoqin, ihor.solodrai, linux-kernel, bpf, pengdonglin
From: pengdonglin <pengdonglin@xiaomi.com>
The series addresses the performance limitations of linear search in large
BTFs by:
1. Adding BTF permutation support
2. Using resolve_btfids to sort BTF during the build phase
3. Checking BTF sorting
4. Using binary search when looking up types
Patch #1 introduces an interface for btf__permute in libbpf to relay out BTF.
Patch #2 adds test cases to validate the functionality of btf__permute in base
and split BTF scenarios.
Patch #3 introduces a new phase in the resolve_btfids tool to sort BTF by name
in ascending order.
Patches #4-#7 implement the sorting check and binary search.
Patches #8-#10 optimize type lookup performance of some functions by skipping
anonymous types or invoking btf_find_by_name_kind.
Patches #11-#12 introduce btf_is_sorted and btf_sorted_start_id for clarity.
Patch #13 refactors the code by calling str_is_empty.
Here is a simple performance test result [1] for lookups to find 87,584 named
types in vmlinux BTF:
./vmtest.sh -- ./test_progs -t btf_permute/perf -v
Results:
| Condition | Lookup Time | Improvement |
|--------------------|-------------|--------------|
| Unsorted (Linear) | 36,534 ms | Baseline |
| Sorted (Binary) | 15 ms | 2437x faster |
The binary search implementation reduces lookup time from 36.5 seconds to 15
milliseconds, achieving a **2437x** speedup for large-scale type queries.
Changelog:
v10:
- Improve btf__permute() documentation (Eduard)
- Fall back to linear search when locating anonymous types (Eduard)
- Remove redundant NULL name check in libbpf's linear search path (Eduard)
- Simplify btf_check_sorted() implementation (Eduard)
- Treat kernel modules as unsorted by default
- Introduce btf_is_sorted and btf_sorted_start_id for clarity (Eduard)
- Fix optimizations in btf_find_decl_tag_value() and btf_prepare_func_args()
to support split BTF
- Remove linear search branch in determine_ptr_size()
- Rebase onto Ihor's v4 patch series [4]
v9:
- Link: https://lore.kernel.org/bpf/20251208062353.1702672-1-dolinux.peng@gmail.com/
- Optimize the performance of the function determine_ptr_size by invoking
btf__find_by_name_kind
- Optimize the performance of btf_find_decl_tag_value/btf_prepare_func_args/
bpf_core_add_cands by skipping anonymous types
- Rebase the patch series onto Ihor's v3 patch series [3]
v8
- Link: https://lore.kernel.org/bpf/20251126085025.784288-1-dolinux.peng@gmail.com/
- Remove the type dropping feature of btf__permute (Andrii)
- Refactor the code of btf__permute (Andrii, Eduard)
- Make the self-test code cleaner (Eduard)
- Reconstruct the BTF sorting patch based on Ihor's patch series [2]
- Simplify the sorting logic and place anonymous types before named types
(Andrii, Eduard)
- Optimize type lookup performance of two kernel functions
- Refactoring the binary search and type lookup logic achieves a 4.2%
performance gain, reducing the average lookup time (via the perf test
code in [1] for 60,995 named types in vmlinux BTF) from 10,217 us (v7) to
9,783 us (v8).
v7:
- Link: https://lore.kernel.org/all/20251119031531.1817099-1-dolinux.peng@gmail.com/
- btf__permute API refinement: Adjusted id_map and id_map_cnt parameter
usage so that for base BTF, id_map[0] now contains the new id of original
type id 1 (instead of VOID type id 0), improving logical consistency
- Selftest updates: Modified test cases to align with the API usage changes
- Refactor the code of resolve_btfids
v6:
- Link: https://lore.kernel.org/all/20251117132623.3807094-1-dolinux.peng@gmail.com/
- ID Map-based reimplementation of btf__permute (Andrii)
- Build-time BTF sorting using resolve_btfids (Alexei, Eduard)
- Binary search method refactoring (Andrii)
- Enhanced selftest coverage
v5:
- Link: https://lore.kernel.org/all/20251106131956.1222864-1-dolinux.peng@gmail.com/
- Refactor binary search implementation for improved efficiency
(Thanks to Andrii and Eduard)
- Extend btf__permute interface with 'ids_sz' parameter to support
type dropping feature (suggested by Andrii). Plan subsequent reimplementation of
id_map version for comparative analysis with current sequence interface
- Add comprehensive test coverage for type dropping functionality
- Enhance function comment clarity and accuracy
v4:
- Link: https://lore.kernel.org/all/20251104134033.344807-1-dolinux.peng@gmail.com/
- Abstracted btf_dedup_remap_types logic into a helper function (suggested by Eduard).
- Removed btf_sort.c and implemented sorting separately for libbpf and kernel (suggested by Andrii).
- Added test cases for both base BTF and split BTF scenarios (suggested by Eduard).
- Added validation for name-only sorting of types (suggested by Andrii)
- Refactored btf__permute implementation to reduce complexity (suggested by Andrii)
- Add doc comments for btf__permute (suggested by Andrii)
v3:
- Link: https://lore.kernel.org/all/20251027135423.3098490-1-dolinux.peng@gmail.com/
- Remove sorting logic from libbpf and provide a generic btf__permute() interface (suggested
by Andrii)
- Omitted the search direction patch to avoid conflicts with base BTF (suggested by Eduard).
- Include btf_sort.c directly in btf.c to reduce function call overhead
v2:
- Link: https://lore.kernel.org/all/20251020093941.548058-1-dolinux.peng@gmail.com/
- Moved sorting to the build phase to reduce overhead (suggested by Alexei).
- Integrated sorting into btf_dedup_compact_and_sort_types (suggested by Eduard).
- Added sorting checks during BTF parsing.
- Consolidated common logic into btf_sort.c for sharing (suggested by Alan).
v1:
- Link: https://lore.kernel.org/all/20251013131537.1927035-1-dolinux.peng@gmail.com/
[1] https://github.com/pengdonglin137/btf_sort_test
[2] https://lore.kernel.org/bpf/20251126012656.3546071-1-ihor.solodrai@linux.dev/
[3] https://lore.kernel.org/bpf/20251205223046.4155870-1-ihor.solodrai@linux.dev/
[4] https://lore.kernel.org/bpf/20251218003314.260269-1-ihor.solodrai@linux.dev/
pengdonglin (13):
libbpf: Add BTF permutation support for type reordering
selftests/bpf: Add test cases for btf__permute functionality
tools/resolve_btfids: Support BTF sorting feature
libbpf: Optimize type lookup with binary search for sorted BTF
libbpf: Verify BTF Sorting
btf: Optimize type lookup with binary search
btf: Verify BTF Sorting
bpf: Skip anonymous types in type lookup for performance
bpf: Optimize the performance of find_bpffs_btf_enums
libbpf: Optimize the performance of determine_ptr_size
libbpf: Add btf_is_sorted and btf_sorted_start_id helpers to refactor
the code
btf: Add btf_is_sorted to refactor the code
btf: Refactor the code by calling str_is_empty
include/linux/btf.h | 2 +
kernel/bpf/btf.c | 170 +++++++++-
kernel/bpf/inode.c | 42 ++-
kernel/bpf/verifier.c | 7 +-
tools/bpf/resolve_btfids/main.c | 68 ++++
tools/lib/bpf/btf.c | 321 +++++++++++++++---
tools/lib/bpf/btf.h | 36 ++
tools/lib/bpf/libbpf.c | 4 +-
tools/lib/bpf/libbpf.map | 1 +
tools/lib/bpf/libbpf_internal.h | 2 +
.../selftests/bpf/prog_tests/btf_permute.c | 228 +++++++++++++
11 files changed, 786 insertions(+), 95 deletions(-)
create mode 100644 tools/testing/selftests/bpf/prog_tests/btf_permute.c
--
2.34.1
^ permalink raw reply [flat|nested] 72+ messages in thread
* [PATCH bpf-next v10 01/13] libbpf: Add BTF permutation support for type reordering
2025-12-18 11:30 [PATCH bpf-next v10 00/13] Improve the performance of BTF type lookups with binary search Donglin Peng
@ 2025-12-18 11:30 ` Donglin Peng
2025-12-18 23:02 ` Andrii Nakryiko
2025-12-18 11:30 ` [PATCH bpf-next v10 02/13] selftests/bpf: Add test cases for btf__permute functionality Donglin Peng
` (11 subsequent siblings)
12 siblings, 1 reply; 72+ messages in thread
From: Donglin Peng @ 2025-12-18 11:30 UTC (permalink / raw)
To: ast, andrii.nakryiko, eddyz87
Cc: zhangxiaoqin, ihor.solodrai, linux-kernel, bpf, pengdonglin,
Alan Maguire
From: pengdonglin <pengdonglin@xiaomi.com>
Introduce btf__permute() API to allow in-place rearrangement of BTF types.
This function reorganizes BTF type order according to a provided array of
type IDs, updating all type references to maintain consistency.
Cc: Eduard Zingerman <eddyz87@gmail.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Cc: Alan Maguire <alan.maguire@oracle.com>
Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
---
tools/lib/bpf/btf.c | 119 +++++++++++++++++++++++++++++++++++++++
tools/lib/bpf/btf.h | 36 ++++++++++++
tools/lib/bpf/libbpf.map | 1 +
3 files changed, 156 insertions(+)
diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
index b136572e889a..ab204ca403dc 100644
--- a/tools/lib/bpf/btf.c
+++ b/tools/lib/bpf/btf.c
@@ -5887,3 +5887,122 @@ int btf__relocate(struct btf *btf, const struct btf *base_btf)
btf->owns_base = false;
return libbpf_err(err);
}
+
+struct btf_permute {
+ struct btf *btf;
+ __u32 *id_map;
+};
+
+/* Callback function to remap individual type ID references */
+static int btf_permute_remap_type_id(__u32 *type_id, void *ctx)
+{
+ struct btf_permute *p = ctx;
+ __u32 new_type_id = *type_id;
+
+ /* refer to the base BTF or VOID type */
+ if (new_type_id < p->btf->start_id)
+ return 0;
+
+ if (new_type_id >= btf__type_cnt(p->btf))
+ return -EINVAL;
+
+ *type_id = p->id_map[new_type_id - p->btf->start_id];
+ return 0;
+}
+
+int btf__permute(struct btf *btf, __u32 *id_map, __u32 id_map_cnt,
+ const struct btf_permute_opts *opts)
+{
+ struct btf_permute p;
+ struct btf_ext *btf_ext;
+ void *nt, *new_types = NULL;
+ __u32 *order_map = NULL;
+ int err = 0, i;
+ __u32 id;
+
+ if (!OPTS_VALID(opts, btf_permute_opts) || id_map_cnt != btf->nr_types)
+ return libbpf_err(-EINVAL);
+
+ /* record the sequence of types */
+ order_map = calloc(id_map_cnt, sizeof(*id_map));
+ if (!order_map) {
+ err = -ENOMEM;
+ goto done;
+ }
+
+ new_types = calloc(btf->hdr->type_len, 1);
+ if (!new_types) {
+ err = -ENOMEM;
+ goto done;
+ }
+
+ if (btf_ensure_modifiable(btf)) {
+ err = -ENOMEM;
+ goto done;
+ }
+
+ for (i = 0; i < id_map_cnt; i++) {
+ id = id_map[i];
+ if (id < btf->start_id || id >= btf__type_cnt(btf)) {
+ err = -EINVAL;
+ goto done;
+ }
+ id -= btf->start_id;
+ /* cannot be mapped to the same ID */
+ if (order_map[id]) {
+ err = -EINVAL;
+ goto done;
+ }
+ order_map[id] = i + btf->start_id;
+ }
+
+ p.btf = btf;
+ p.id_map = id_map;
+ nt = new_types;
+ for (i = 0; i < id_map_cnt; i++) {
+ struct btf_field_iter it;
+ const struct btf_type *t;
+ __u32 *type_id;
+ int type_size;
+
+ id = order_map[i];
+ t = btf__type_by_id(btf, id);
+ type_size = btf_type_size(t);
+ memcpy(nt, t, type_size);
+
+ /* fix up referenced IDs for BTF */
+ err = btf_field_iter_init(&it, nt, BTF_FIELD_ITER_IDS);
+ if (err)
+ goto done;
+ while ((type_id = btf_field_iter_next(&it))) {
+ err = btf_permute_remap_type_id(type_id, &p);
+ if (err)
+ goto done;
+ }
+
+ nt += type_size;
+ }
+
+ /* fix up referenced IDs for btf_ext */
+ btf_ext = OPTS_GET(opts, btf_ext, NULL);
+ if (btf_ext) {
+ err = btf_ext_visit_type_ids(btf_ext, btf_permute_remap_type_id, &p);
+ if (err)
+ goto done;
+ }
+
+ for (nt = new_types, i = 0; i < id_map_cnt; i++) {
+ btf->type_offs[i] = nt - new_types;
+ nt += btf_type_size(nt);
+ }
+
+ free(order_map);
+ free(btf->types_data);
+ btf->types_data = new_types;
+ return 0;
+
+done:
+ free(order_map);
+ free(new_types);
+ return libbpf_err(err);
+}
diff --git a/tools/lib/bpf/btf.h b/tools/lib/bpf/btf.h
index cc01494d6210..5d560571b1b5 100644
--- a/tools/lib/bpf/btf.h
+++ b/tools/lib/bpf/btf.h
@@ -281,6 +281,42 @@ LIBBPF_API int btf__dedup(struct btf *btf, const struct btf_dedup_opts *opts);
*/
LIBBPF_API int btf__relocate(struct btf *btf, const struct btf *base_btf);
+struct btf_permute_opts {
+ size_t sz;
+ /* optional .BTF.ext info along the main BTF info */
+ struct btf_ext *btf_ext;
+ size_t :0;
+};
+#define btf_permute_opts__last_field btf_ext
+
+/**
+ * @brief **btf__permute()** performs in-place BTF type rearrangement
+ * @param btf BTF object to permute
+ * @param id_map Array mapping original type IDs to new IDs
+ * @param id_map_cnt Number of elements in @id_map
+ * @param opts Optional parameters for BTF extension updates
+ * @return 0 on success, negative error code on failure
+ *
+ * **btf__permute()** rearranges BTF types according to the specified ID mapping.
+ * The @id_map array defines the new type ID for each original type ID.
+ *
+ * @id_map must include all types from ID `start_id` to `btf__type_cnt(btf) - 1`.
+ * @id_map_cnt should be `btf__type_cnt(btf) - start_id`
+ * The mapping is defined as: `id_map[original_id - start_id] = new_id`
+ *
+ * For base BTF, its `start_id` is fixed to 1, i.e. the VOID type can
+ * not be redefined or remapped and its ID is fixed to 0.
+ *
+ * For split BTF, its `start_id` can be retrieved by calling
+ * `btf__type_cnt(btf__base_btf(btf))`.
+ *
+ * On error, returns negative error code and sets errno:
+ * - `-EINVAL`: Invalid parameters or ID mapping (duplicates, out-of-range)
+ * - `-ENOMEM`: Memory allocation failure
+ */
+LIBBPF_API int btf__permute(struct btf *btf, __u32 *id_map, __u32 id_map_cnt,
+ const struct btf_permute_opts *opts);
+
struct btf_dump;
struct btf_dump_opts {
diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map
index 84fb90a016c9..d18fbcea7578 100644
--- a/tools/lib/bpf/libbpf.map
+++ b/tools/lib/bpf/libbpf.map
@@ -453,4 +453,5 @@ LIBBPF_1.7.0 {
bpf_map__exclusive_program;
bpf_prog_assoc_struct_ops;
bpf_program__assoc_struct_ops;
+ btf__permute;
} LIBBPF_1.6.0;
--
2.34.1
^ permalink raw reply related [flat|nested] 72+ messages in thread
* [PATCH bpf-next v10 02/13] selftests/bpf: Add test cases for btf__permute functionality
2025-12-18 11:30 [PATCH bpf-next v10 00/13] Improve the performance of BTF type lookups with binary search Donglin Peng
2025-12-18 11:30 ` [PATCH bpf-next v10 01/13] libbpf: Add BTF permutation support for type reordering Donglin Peng
@ 2025-12-18 11:30 ` Donglin Peng
2025-12-18 23:03 ` Andrii Nakryiko
2025-12-18 11:30 ` [PATCH bpf-next v10 03/13] tools/resolve_btfids: Support BTF sorting feature Donglin Peng
` (10 subsequent siblings)
12 siblings, 1 reply; 72+ messages in thread
From: Donglin Peng @ 2025-12-18 11:30 UTC (permalink / raw)
To: ast, andrii.nakryiko, eddyz87
Cc: zhangxiaoqin, ihor.solodrai, linux-kernel, bpf, pengdonglin,
Alan Maguire
From: pengdonglin <pengdonglin@xiaomi.com>
This patch introduces test cases for the btf__permute function to ensure
it works correctly with both base BTF and split BTF scenarios.
The test suite includes:
- test_permute_base: Validates permutation on base BTF
- test_permute_split: Tests permutation on split BTF
Cc: Eduard Zingerman <eddyz87@gmail.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Cc: Alan Maguire <alan.maguire@oracle.com>
Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
---
.../selftests/bpf/prog_tests/btf_permute.c | 228 ++++++++++++++++++
1 file changed, 228 insertions(+)
create mode 100644 tools/testing/selftests/bpf/prog_tests/btf_permute.c
diff --git a/tools/testing/selftests/bpf/prog_tests/btf_permute.c b/tools/testing/selftests/bpf/prog_tests/btf_permute.c
new file mode 100644
index 000000000000..9aa71cdf984a
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/btf_permute.c
@@ -0,0 +1,228 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2025 Xiaomi */
+
+#include <test_progs.h>
+#include <bpf/btf.h>
+#include "btf_helpers.h"
+
+static void permute_base_check(struct btf *btf)
+{
+ VALIDATE_RAW_BTF(
+ btf,
+ "[1] STRUCT 's2' size=4 vlen=1\n"
+ "\t'm' type_id=4 bits_offset=0",
+ "[2] FUNC 'f' type_id=6 linkage=static",
+ "[3] PTR '(anon)' type_id=4",
+ "[4] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED",
+ "[5] STRUCT 's1' size=4 vlen=1\n"
+ "\t'm' type_id=4 bits_offset=0",
+ "[6] FUNC_PROTO '(anon)' ret_type_id=4 vlen=1\n"
+ "\t'p' type_id=3");
+}
+
+/* Ensure btf__permute work as expected with base BTF */
+static void test_permute_base(void)
+{
+ struct btf *btf;
+ __u32 permute_ids[6];
+ int start_id = 1;
+ int err;
+
+ btf = btf__new_empty();
+ if (!ASSERT_OK_PTR(btf, "empty_main_btf"))
+ return;
+
+ btf__add_int(btf, "int", 4, BTF_INT_SIGNED); /* [1] int */
+ btf__add_ptr(btf, 1); /* [2] ptr to int */
+ btf__add_struct(btf, "s1", 4); /* [3] struct s1 { */
+ btf__add_field(btf, "m", 1, 0, 0); /* int m; */
+ /* } */
+ btf__add_struct(btf, "s2", 4); /* [4] struct s2 { */
+ btf__add_field(btf, "m", 1, 0, 0); /* int m; */
+ /* } */
+ btf__add_func_proto(btf, 1); /* [5] int (*)(int *p); */
+ btf__add_func_param(btf, "p", 2);
+ btf__add_func(btf, "f", BTF_FUNC_STATIC, 5); /* [6] int f(int *p); */
+
+ VALIDATE_RAW_BTF(
+ btf,
+ "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED",
+ "[2] PTR '(anon)' type_id=1",
+ "[3] STRUCT 's1' size=4 vlen=1\n"
+ "\t'm' type_id=1 bits_offset=0",
+ "[4] STRUCT 's2' size=4 vlen=1\n"
+ "\t'm' type_id=1 bits_offset=0",
+ "[5] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n"
+ "\t'p' type_id=2",
+ "[6] FUNC 'f' type_id=5 linkage=static");
+
+ permute_ids[1 - start_id] = 4; /* [1] -> [4] */
+ permute_ids[2 - start_id] = 3; /* [2] -> [3] */
+ permute_ids[3 - start_id] = 5; /* [3] -> [5] */
+ permute_ids[4 - start_id] = 1; /* [4] -> [1] */
+ permute_ids[5 - start_id] = 6; /* [5] -> [6] */
+ permute_ids[6 - start_id] = 2; /* [6] -> [2] */
+ err = btf__permute(btf, permute_ids, ARRAY_SIZE(permute_ids), NULL);
+ if (!ASSERT_OK(err, "btf__permute_base"))
+ goto done;
+ permute_base_check(btf);
+
+ /* id_map_cnt is invalid */
+ permute_ids[1 - start_id] = 4; /* [1] -> [4] */
+ permute_ids[2 - start_id] = 3; /* [2] -> [3] */
+ permute_ids[3 - start_id] = 5; /* [3] -> [5] */
+ permute_ids[4 - start_id] = 1; /* [4] -> [1] */
+ permute_ids[5 - start_id] = 6; /* [5] -> [6] */
+ permute_ids[6 - start_id] = 2; /* [6] -> [2] */
+ err = btf__permute(btf, permute_ids, ARRAY_SIZE(permute_ids) - 1, NULL);
+ if (!ASSERT_ERR(err, "btf__permute_base"))
+ goto done;
+ /* BTF is not modified */
+ permute_base_check(btf);
+
+ /* Multiple types can not be mapped to the same ID */
+ permute_ids[1 - start_id] = 4;
+ permute_ids[2 - start_id] = 4;
+ permute_ids[3 - start_id] = 5;
+ permute_ids[4 - start_id] = 1;
+ permute_ids[5 - start_id] = 6;
+ permute_ids[6 - start_id] = 2;
+ err = btf__permute(btf, permute_ids, ARRAY_SIZE(permute_ids), NULL);
+ if (!ASSERT_ERR(err, "btf__permute_base"))
+ goto done;
+ /* BTF is not modified */
+ permute_base_check(btf);
+
+ /* Type ID must be valid */
+ permute_ids[1 - start_id] = 4;
+ permute_ids[2 - start_id] = 3;
+ permute_ids[3 - start_id] = 5;
+ permute_ids[4 - start_id] = 1;
+ permute_ids[5 - start_id] = 7;
+ permute_ids[6 - start_id] = 2;
+ err = btf__permute(btf, permute_ids, ARRAY_SIZE(permute_ids), NULL);
+ if (!ASSERT_ERR(err, "btf__permute_base"))
+ goto done;
+ /* BTF is not modified */
+ permute_base_check(btf);
+
+done:
+ btf__free(btf);
+}
+
+static void permute_split_check(struct btf *btf)
+{
+ VALIDATE_RAW_BTF(
+ btf,
+ "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED",
+ "[2] PTR '(anon)' type_id=1",
+ "[3] STRUCT 's2' size=4 vlen=1\n"
+ "\t'm' type_id=1 bits_offset=0",
+ "[4] FUNC 'f' type_id=5 linkage=static",
+ "[5] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n"
+ "\t'p' type_id=2",
+ "[6] STRUCT 's1' size=4 vlen=1\n"
+ "\t'm' type_id=1 bits_offset=0");
+}
+
+/* Ensure btf__permute work as expected with split BTF */
+static void test_permute_split(void)
+{
+ struct btf *split_btf = NULL, *base_btf = NULL;
+ __u32 permute_ids[4];
+ int err;
+ int start_id;
+
+ base_btf = btf__new_empty();
+ if (!ASSERT_OK_PTR(base_btf, "empty_main_btf"))
+ return;
+
+ btf__add_int(base_btf, "int", 4, BTF_INT_SIGNED); /* [1] int */
+ btf__add_ptr(base_btf, 1); /* [2] ptr to int */
+ VALIDATE_RAW_BTF(
+ base_btf,
+ "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED",
+ "[2] PTR '(anon)' type_id=1");
+ split_btf = btf__new_empty_split(base_btf);
+ if (!ASSERT_OK_PTR(split_btf, "empty_split_btf"))
+ goto cleanup;
+ btf__add_struct(split_btf, "s1", 4); /* [3] struct s1 { */
+ btf__add_field(split_btf, "m", 1, 0, 0); /* int m; */
+ /* } */
+ btf__add_struct(split_btf, "s2", 4); /* [4] struct s2 { */
+ btf__add_field(split_btf, "m", 1, 0, 0); /* int m; */
+ /* } */
+ btf__add_func_proto(split_btf, 1); /* [5] int (*)(int p); */
+ btf__add_func_param(split_btf, "p", 2);
+ btf__add_func(split_btf, "f", BTF_FUNC_STATIC, 5); /* [6] int f(int *p); */
+
+ VALIDATE_RAW_BTF(
+ split_btf,
+ "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED",
+ "[2] PTR '(anon)' type_id=1",
+ "[3] STRUCT 's1' size=4 vlen=1\n"
+ "\t'm' type_id=1 bits_offset=0",
+ "[4] STRUCT 's2' size=4 vlen=1\n"
+ "\t'm' type_id=1 bits_offset=0",
+ "[5] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n"
+ "\t'p' type_id=2",
+ "[6] FUNC 'f' type_id=5 linkage=static");
+
+ start_id = btf__type_cnt(base_btf);
+ permute_ids[3 - start_id] = 6; /* [3] -> [6] */
+ permute_ids[4 - start_id] = 3; /* [4] -> [3] */
+ permute_ids[5 - start_id] = 5; /* [5] -> [5] */
+ permute_ids[6 - start_id] = 4; /* [6] -> [4] */
+ err = btf__permute(split_btf, permute_ids, ARRAY_SIZE(permute_ids), NULL);
+ if (!ASSERT_OK(err, "btf__permute_split"))
+ goto cleanup;
+ permute_split_check(split_btf);
+
+ /*
+ * For split BTF, id_map_cnt must equal to the number of types
+ * added on top of base BTF
+ */
+ permute_ids[3 - start_id] = 4;
+ permute_ids[4 - start_id] = 3;
+ permute_ids[5 - start_id] = 5;
+ permute_ids[6 - start_id] = 6;
+ err = btf__permute(split_btf, permute_ids, 3, NULL);
+ if (!ASSERT_ERR(err, "btf__permute_split"))
+ goto cleanup;
+ /* BTF is not modified */
+ permute_split_check(split_btf);
+
+ /* Multiple types can not be mapped to the same ID */
+ permute_ids[3 - start_id] = 4;
+ permute_ids[4 - start_id] = 3;
+ permute_ids[5 - start_id] = 3;
+ permute_ids[6 - start_id] = 6;
+ err = btf__permute(split_btf, permute_ids, 4, NULL);
+ if (!ASSERT_ERR(err, "btf__permute_split"))
+ goto cleanup;
+ /* BTF is not modified */
+ permute_split_check(split_btf);
+
+ /* Can not map to base ID */
+ permute_ids[3 - start_id] = 4;
+ permute_ids[4 - start_id] = 2;
+ permute_ids[5 - start_id] = 5;
+ permute_ids[6 - start_id] = 6;
+ err = btf__permute(split_btf, permute_ids, 4, NULL);
+ if (!ASSERT_ERR(err, "btf__permute_split"))
+ goto cleanup;
+ /* BTF is not modified */
+ permute_split_check(split_btf);
+
+cleanup:
+ btf__free(split_btf);
+ btf__free(base_btf);
+}
+
+void test_btf_permute(void)
+{
+ if (test__start_subtest("permute_base"))
+ test_permute_base();
+ if (test__start_subtest("permute_split"))
+ test_permute_split();
+}
--
2.34.1
^ permalink raw reply related [flat|nested] 72+ messages in thread
* [PATCH bpf-next v10 03/13] tools/resolve_btfids: Support BTF sorting feature
2025-12-18 11:30 [PATCH bpf-next v10 00/13] Improve the performance of BTF type lookups with binary search Donglin Peng
2025-12-18 11:30 ` [PATCH bpf-next v10 01/13] libbpf: Add BTF permutation support for type reordering Donglin Peng
2025-12-18 11:30 ` [PATCH bpf-next v10 02/13] selftests/bpf: Add test cases for btf__permute functionality Donglin Peng
@ 2025-12-18 11:30 ` Donglin Peng
2025-12-18 23:09 ` Andrii Nakryiko
2025-12-18 11:30 ` [PATCH bpf-next v10 04/13] libbpf: Optimize type lookup with binary search for sorted BTF Donglin Peng
` (9 subsequent siblings)
12 siblings, 1 reply; 72+ messages in thread
From: Donglin Peng @ 2025-12-18 11:30 UTC (permalink / raw)
To: ast, andrii.nakryiko, eddyz87
Cc: zhangxiaoqin, ihor.solodrai, linux-kernel, bpf, pengdonglin,
Alan Maguire
From: pengdonglin <pengdonglin@xiaomi.com>
This introduces a new BTF sorting phase that specifically sorts
BTF types by name in ascending order, so that the binary search
can be used to look up types.
Cc: Eduard Zingerman <eddyz87@gmail.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Cc: Alan Maguire <alan.maguire@oracle.com>
Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
---
tools/bpf/resolve_btfids/main.c | 68 +++++++++++++++++++++++++++++++++
1 file changed, 68 insertions(+)
diff --git a/tools/bpf/resolve_btfids/main.c b/tools/bpf/resolve_btfids/main.c
index 3e88dc862d87..659de35748ec 100644
--- a/tools/bpf/resolve_btfids/main.c
+++ b/tools/bpf/resolve_btfids/main.c
@@ -848,6 +848,71 @@ static int dump_raw_btf(struct btf *btf, const char *out_path)
return 0;
}
+/*
+ * Sort types by name in ascending order resulting in all
+ * anonymous types being placed before named types.
+ */
+static int cmp_type_names(const void *a, const void *b, void *priv)
+{
+ struct btf *btf = (struct btf *)priv;
+ const struct btf_type *ta = btf__type_by_id(btf, *(__u32 *)a);
+ const struct btf_type *tb = btf__type_by_id(btf, *(__u32 *)b);
+ const char *na, *nb;
+
+ na = btf__str_by_offset(btf, ta->name_off);
+ nb = btf__str_by_offset(btf, tb->name_off);
+ return strcmp(na, nb);
+}
+
+static int sort_btf_by_name(struct btf *btf)
+{
+ __u32 *permute_ids = NULL, *id_map = NULL;
+ int nr_types, i, err = 0;
+ __u32 start_id = 1, id;
+
+ if (btf__base_btf(btf))
+ start_id = btf__type_cnt(btf__base_btf(btf));
+ nr_types = btf__type_cnt(btf) - start_id;
+ if (nr_types < 2)
+ goto out;
+
+ permute_ids = calloc(nr_types, sizeof(*permute_ids));
+ if (!permute_ids) {
+ err = -ENOMEM;
+ goto out;
+ }
+
+ id_map = calloc(nr_types, sizeof(*id_map));
+ if (!id_map) {
+ err = -ENOMEM;
+ goto out;
+ }
+
+ for (i = 0, id = start_id; i < nr_types; i++, id++)
+ permute_ids[i] = id;
+
+ qsort_r(permute_ids, nr_types, sizeof(*permute_ids), cmp_type_names, btf);
+
+ for (i = 0; i < nr_types; i++) {
+ id = permute_ids[i] - start_id;
+ id_map[id] = i + start_id;
+ }
+
+ err = btf__permute(btf, id_map, nr_types, NULL);
+ if (err)
+ pr_err("FAILED: btf permute: %s\n", strerror(-err));
+
+out:
+ free(permute_ids);
+ free(id_map);
+ return err;
+}
+
+static int btf2btf(struct object *obj)
+{
+ return sort_btf_by_name(obj->btf);
+}
+
static inline int make_out_path(char *buf, u32 buf_sz, const char *in_path, const char *suffix)
{
int len = snprintf(buf, buf_sz, "%s%s", in_path, suffix);
@@ -906,6 +971,9 @@ int main(int argc, const char **argv)
if (load_btf(&obj))
goto out;
+ if (btf2btf(&obj))
+ goto out;
+
if (elf_collect(&obj))
goto out;
--
2.34.1
^ permalink raw reply related [flat|nested] 72+ messages in thread
* [PATCH bpf-next v10 04/13] libbpf: Optimize type lookup with binary search for sorted BTF
2025-12-18 11:30 [PATCH bpf-next v10 00/13] Improve the performance of BTF type lookups with binary search Donglin Peng
` (2 preceding siblings ...)
2025-12-18 11:30 ` [PATCH bpf-next v10 03/13] tools/resolve_btfids: Support BTF sorting feature Donglin Peng
@ 2025-12-18 11:30 ` Donglin Peng
2025-12-18 19:32 ` Eduard Zingerman
2025-12-18 23:29 ` Andrii Nakryiko
2025-12-18 11:30 ` [PATCH bpf-next v10 05/13] libbpf: Verify BTF Sorting Donglin Peng
` (8 subsequent siblings)
12 siblings, 2 replies; 72+ messages in thread
From: Donglin Peng @ 2025-12-18 11:30 UTC (permalink / raw)
To: ast, andrii.nakryiko, eddyz87
Cc: zhangxiaoqin, ihor.solodrai, linux-kernel, bpf, pengdonglin,
Alan Maguire
From: pengdonglin <pengdonglin@xiaomi.com>
This patch introduces binary search optimization for BTF type lookups
when the BTF instance contains sorted types.
The optimization significantly improves performance when searching for
types in large BTF instances with sorted types. For unsorted BTF, the
implementation falls back to the original linear search.
Cc: Eduard Zingerman <eddyz87@gmail.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Cc: Alan Maguire <alan.maguire@oracle.com>
Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
---
tools/lib/bpf/btf.c | 103 ++++++++++++++++++++++++++++++++++----------
1 file changed, 80 insertions(+), 23 deletions(-)
diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
index ab204ca403dc..2facb57d7e5f 100644
--- a/tools/lib/bpf/btf.c
+++ b/tools/lib/bpf/btf.c
@@ -92,6 +92,8 @@ struct btf {
* - for split BTF counts number of types added on top of base BTF.
*/
__u32 nr_types;
+ /* the start IDs of named types in sorted BTF */
+ int sorted_start_id;
/* if not NULL, points to the base BTF on top of which the current
* split BTF is based
*/
@@ -897,46 +899,98 @@ int btf__resolve_type(const struct btf *btf, __u32 type_id)
return type_id;
}
-__s32 btf__find_by_name(const struct btf *btf, const char *type_name)
+static __s32 btf_find_by_name_bsearch(const struct btf *btf, const char *name,
+ __s32 start_id, __s32 end_id)
{
- __u32 i, nr_types = btf__type_cnt(btf);
-
- if (!strcmp(type_name, "void"))
- return 0;
-
- for (i = 1; i < nr_types; i++) {
- const struct btf_type *t = btf__type_by_id(btf, i);
- const char *name = btf__name_by_offset(btf, t->name_off);
-
- if (name && !strcmp(type_name, name))
- return i;
+ const struct btf_type *t;
+ const char *tname;
+ __s32 l, r, m, lmost = -ENOENT;
+ int ret;
+
+ l = start_id;
+ r = end_id;
+ while (l <= r) {
+ m = l + (r - l) / 2;
+ t = btf_type_by_id(btf, m);
+ tname = btf__str_by_offset(btf, t->name_off);
+ ret = strcmp(tname, name);
+ if (ret < 0) {
+ l = m + 1;
+ } else {
+ if (ret == 0)
+ lmost = m;
+ r = m - 1;
+ }
}
- return libbpf_err(-ENOENT);
+ return lmost;
}
static __s32 btf_find_by_name_kind(const struct btf *btf, int start_id,
const char *type_name, __u32 kind)
{
- __u32 i, nr_types = btf__type_cnt(btf);
+ const struct btf_type *t;
+ const char *tname;
+ __s32 idx;
+
+ if (start_id < btf->start_id) {
+ idx = btf_find_by_name_kind(btf->base_btf, start_id,
+ type_name, kind);
+ if (idx >= 0)
+ return idx;
+ start_id = btf->start_id;
+ }
- if (kind == BTF_KIND_UNKN || !strcmp(type_name, "void"))
+ if (kind == BTF_KIND_UNKN || strcmp(type_name, "void") == 0)
return 0;
- for (i = start_id; i < nr_types; i++) {
- const struct btf_type *t = btf__type_by_id(btf, i);
- const char *name;
+ if (btf->sorted_start_id > 0 && type_name[0]) {
+ __s32 end_id = btf__type_cnt(btf) - 1;
+
+ /* skip anonymous types */
+ start_id = max(start_id, btf->sorted_start_id);
+ idx = btf_find_by_name_bsearch(btf, type_name, start_id, end_id);
+ if (unlikely(idx < 0))
+ return libbpf_err(-ENOENT);
+
+ if (unlikely(kind == -1))
+ return idx;
+
+ t = btf_type_by_id(btf, idx);
+ if (likely(BTF_INFO_KIND(t->info) == kind))
+ return idx;
+
+ for (idx++; idx <= end_id; idx++) {
+ t = btf__type_by_id(btf, idx);
+ tname = btf__str_by_offset(btf, t->name_off);
+ if (strcmp(tname, type_name) != 0)
+ return libbpf_err(-ENOENT);
+ if (btf_kind(t) == kind)
+ return idx;
+ }
+ } else {
+ __u32 i, total;
- if (btf_kind(t) != kind)
- continue;
- name = btf__name_by_offset(btf, t->name_off);
- if (name && !strcmp(type_name, name))
- return i;
+ total = btf__type_cnt(btf);
+ for (i = start_id; i < total; i++) {
+ t = btf_type_by_id(btf, i);
+ if (kind != -1 && btf_kind(t) != kind)
+ continue;
+ tname = btf__str_by_offset(btf, t->name_off);
+ if (strcmp(tname, type_name) == 0)
+ return i;
+ }
}
return libbpf_err(-ENOENT);
}
+/* the kind value of -1 indicates that kind matching should be skipped */
+__s32 btf__find_by_name(const struct btf *btf, const char *type_name)
+{
+ return btf_find_by_name_kind(btf, btf->start_id, type_name, -1);
+}
+
__s32 btf__find_by_name_kind_own(const struct btf *btf, const char *type_name,
__u32 kind)
{
@@ -1006,6 +1060,7 @@ static struct btf *btf_new_empty(struct btf *base_btf)
btf->fd = -1;
btf->ptr_sz = sizeof(void *);
btf->swapped_endian = false;
+ btf->sorted_start_id = 0;
if (base_btf) {
btf->base_btf = base_btf;
@@ -1057,6 +1112,7 @@ static struct btf *btf_new(const void *data, __u32 size, struct btf *base_btf, b
btf->start_id = 1;
btf->start_str_off = 0;
btf->fd = -1;
+ btf->sorted_start_id = 0;
if (base_btf) {
btf->base_btf = base_btf;
@@ -1715,6 +1771,7 @@ static void btf_invalidate_raw_data(struct btf *btf)
free(btf->raw_data_swapped);
btf->raw_data_swapped = NULL;
}
+ btf->sorted_start_id = 0;
}
/* Ensure BTF is ready to be modified (by splitting into a three memory
--
2.34.1
^ permalink raw reply related [flat|nested] 72+ messages in thread
* [PATCH bpf-next v10 05/13] libbpf: Verify BTF Sorting
2025-12-18 11:30 [PATCH bpf-next v10 00/13] Improve the performance of BTF type lookups with binary search Donglin Peng
` (3 preceding siblings ...)
2025-12-18 11:30 ` [PATCH bpf-next v10 04/13] libbpf: Optimize type lookup with binary search for sorted BTF Donglin Peng
@ 2025-12-18 11:30 ` Donglin Peng
2025-12-18 19:38 ` Eduard Zingerman
2025-12-18 23:44 ` Andrii Nakryiko
2025-12-18 11:30 ` [PATCH bpf-next v10 06/13] btf: Optimize type lookup with binary search Donglin Peng
` (7 subsequent siblings)
12 siblings, 2 replies; 72+ messages in thread
From: Donglin Peng @ 2025-12-18 11:30 UTC (permalink / raw)
To: ast, andrii.nakryiko, eddyz87
Cc: zhangxiaoqin, ihor.solodrai, linux-kernel, bpf, pengdonglin,
Alan Maguire
From: pengdonglin <pengdonglin@xiaomi.com>
This patch checks whether the BTF is sorted by name in ascending
order. If sorted, binary search will be used when looking up types.
Cc: Eduard Zingerman <eddyz87@gmail.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Cc: Alan Maguire <alan.maguire@oracle.com>
Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
---
tools/lib/bpf/btf.c | 41 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 41 insertions(+)
diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
index 2facb57d7e5f..c63d46b7d74b 100644
--- a/tools/lib/bpf/btf.c
+++ b/tools/lib/bpf/btf.c
@@ -899,6 +899,46 @@ int btf__resolve_type(const struct btf *btf, __u32 type_id)
return type_id;
}
+/*
+ * Assuming that types are sorted by name in ascending order.
+ */
+static int btf_compare_type_names(__u32 *a, __u32 *b, const struct btf *btf)
+{
+ struct btf_type *ta = btf_type_by_id(btf, *a);
+ struct btf_type *tb = btf_type_by_id(btf, *b);
+ const char *na, *nb;
+
+ na = btf__str_by_offset(btf, ta->name_off);
+ nb = btf__str_by_offset(btf, tb->name_off);
+ return strcmp(na, nb);
+}
+
+static void btf_check_sorted(struct btf *btf)
+{
+ const struct btf_type *t;
+ __u32 i, k, n;
+ __u32 sorted_start_id;
+
+ if (btf->nr_types < 2)
+ return;
+
+ sorted_start_id = 0;
+ n = btf__type_cnt(btf);
+ for (i = btf->start_id; i < n; i++) {
+ k = i + 1;
+ if (k < n && btf_compare_type_names(&i, &k, btf) > 0)
+ return;
+ if (sorted_start_id == 0) {
+ t = btf_type_by_id(btf, i);
+ if (t->name_off)
+ sorted_start_id = i;
+ }
+ }
+
+ if (sorted_start_id)
+ btf->sorted_start_id = sorted_start_id;
+}
+
static __s32 btf_find_by_name_bsearch(const struct btf *btf, const char *name,
__s32 start_id, __s32 end_id)
{
@@ -1147,6 +1187,7 @@ static struct btf *btf_new(const void *data, __u32 size, struct btf *base_btf, b
err = err ?: btf_sanity_check(btf);
if (err)
goto done;
+ btf_check_sorted(btf);
done:
if (err) {
--
2.34.1
^ permalink raw reply related [flat|nested] 72+ messages in thread
* [PATCH bpf-next v10 06/13] btf: Optimize type lookup with binary search
2025-12-18 11:30 [PATCH bpf-next v10 00/13] Improve the performance of BTF type lookups with binary search Donglin Peng
` (4 preceding siblings ...)
2025-12-18 11:30 ` [PATCH bpf-next v10 05/13] libbpf: Verify BTF Sorting Donglin Peng
@ 2025-12-18 11:30 ` Donglin Peng
2025-12-18 21:38 ` Eduard Zingerman
2025-12-18 11:30 ` [PATCH bpf-next v10 07/13] btf: Verify BTF Sorting Donglin Peng
` (6 subsequent siblings)
12 siblings, 1 reply; 72+ messages in thread
From: Donglin Peng @ 2025-12-18 11:30 UTC (permalink / raw)
To: ast, andrii.nakryiko, eddyz87
Cc: zhangxiaoqin, ihor.solodrai, linux-kernel, bpf, pengdonglin,
Alan Maguire
From: pengdonglin <pengdonglin@xiaomi.com>
Improve btf_find_by_name_kind() performance by adding binary search
support for sorted types. Falls back to linear search for compatibility.
Cc: Eduard Zingerman <eddyz87@gmail.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Cc: Alan Maguire <alan.maguire@oracle.com>
Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
---
kernel/bpf/btf.c | 85 +++++++++++++++++++++++++++++++++++++++++++-----
1 file changed, 76 insertions(+), 9 deletions(-)
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index 0de8fc8a0e0b..0394f0c8ef74 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -259,6 +259,7 @@ struct btf {
void *nohdr_data;
struct btf_header hdr;
u32 nr_types; /* includes VOID for base BTF */
+ u32 sorted_start_id;
u32 types_size;
u32 data_size;
refcount_t refcnt;
@@ -494,6 +495,11 @@ static bool btf_type_is_modifier(const struct btf_type *t)
return false;
}
+static int btf_start_id(const struct btf *btf)
+{
+ return btf->start_id + (btf->base_btf ? 0 : 1);
+}
+
bool btf_type_is_void(const struct btf_type *t)
{
return t == &btf_void;
@@ -544,21 +550,79 @@ u32 btf_nr_types(const struct btf *btf)
return total;
}
+static s32 btf_find_by_name_bsearch(const struct btf *btf, const char *name,
+ s32 start_id, s32 end_id)
+{
+ const struct btf_type *t;
+ const char *tname;
+ s32 l, r, m, lmost = -ENOENT;
+ int ret;
+
+ l = start_id;
+ r = end_id;
+ while (l <= r) {
+ m = l + (r - l) / 2;
+ t = btf_type_by_id(btf, m);
+ tname = btf_name_by_offset(btf, t->name_off);
+ ret = strcmp(tname, name);
+ if (ret < 0) {
+ l = m + 1;
+ } else {
+ if (ret == 0)
+ lmost = m;
+ r = m - 1;
+ }
+ }
+
+ return lmost;
+}
+
s32 btf_find_by_name_kind(const struct btf *btf, const char *name, u8 kind)
{
+ const struct btf *base_btf = btf_base_btf(btf);
const struct btf_type *t;
const char *tname;
- u32 i, total;
+ s32 idx;
- total = btf_nr_types(btf);
- for (i = 1; i < total; i++) {
- t = btf_type_by_id(btf, i);
- if (BTF_INFO_KIND(t->info) != kind)
- continue;
+ if (base_btf) {
+ idx = btf_find_by_name_kind(base_btf, name, kind);
+ if (idx > 0)
+ return idx;
+ }
- tname = btf_name_by_offset(btf, t->name_off);
- if (!strcmp(tname, name))
- return i;
+ if (btf->sorted_start_id > 0 && name[0]) {
+ /* skip anonymous types */
+ s32 start_id = btf->sorted_start_id;
+ s32 end_id = btf_nr_types(btf) - 1;
+
+ idx = btf_find_by_name_bsearch(btf, name, start_id, end_id);
+ if (idx < 0)
+ return -ENOENT;
+
+ t = btf_type_by_id(btf, idx);
+ if (BTF_INFO_KIND(t->info) == kind)
+ return idx;
+
+ for (idx++; idx <= end_id; idx++) {
+ t = btf_type_by_id(btf, idx);
+ tname = btf_name_by_offset(btf, t->name_off);
+ if (strcmp(tname, name) != 0)
+ return -ENOENT;
+ if (BTF_INFO_KIND(t->info) == kind)
+ return idx;
+ }
+ } else {
+ u32 i, total;
+
+ total = btf_nr_types(btf);
+ for (i = btf_start_id(btf); i < total; i++) {
+ t = btf_type_by_id(btf, i);
+ if (BTF_INFO_KIND(t->info) != kind)
+ continue;
+ tname = btf_name_by_offset(btf, t->name_off);
+ if (strcmp(tname, name) == 0)
+ return i;
+ }
}
return -ENOENT;
@@ -5791,6 +5855,7 @@ static struct btf *btf_parse(const union bpf_attr *attr, bpfptr_t uattr, u32 uat
goto errout;
}
env->btf = btf;
+ btf->sorted_start_id = 0;
data = kvmalloc(attr->btf_size, GFP_KERNEL | __GFP_NOWARN);
if (!data) {
@@ -6210,6 +6275,7 @@ static struct btf *btf_parse_base(struct btf_verifier_env *env, const char *name
btf->data = data;
btf->data_size = data_size;
btf->kernel_btf = true;
+ btf->sorted_start_id = 0;
snprintf(btf->name, sizeof(btf->name), "%s", name);
err = btf_parse_hdr(env);
@@ -6327,6 +6393,7 @@ static struct btf *btf_parse_module(const char *module_name, const void *data,
btf->start_id = base_btf->nr_types;
btf->start_str_off = base_btf->hdr.str_len;
btf->kernel_btf = true;
+ btf->sorted_start_id = 0;
snprintf(btf->name, sizeof(btf->name), "%s", module_name);
btf->data = kvmemdup(data, data_size, GFP_KERNEL | __GFP_NOWARN);
--
2.34.1
^ permalink raw reply related [flat|nested] 72+ messages in thread
* [PATCH bpf-next v10 07/13] btf: Verify BTF Sorting
2025-12-18 11:30 [PATCH bpf-next v10 00/13] Improve the performance of BTF type lookups with binary search Donglin Peng
` (5 preceding siblings ...)
2025-12-18 11:30 ` [PATCH bpf-next v10 06/13] btf: Optimize type lookup with binary search Donglin Peng
@ 2025-12-18 11:30 ` Donglin Peng
2025-12-18 21:43 ` Eduard Zingerman
2025-12-18 23:46 ` Andrii Nakryiko
2025-12-18 11:30 ` [PATCH bpf-next v10 08/13] bpf: Skip anonymous types in type lookup for performance Donglin Peng
` (5 subsequent siblings)
12 siblings, 2 replies; 72+ messages in thread
From: Donglin Peng @ 2025-12-18 11:30 UTC (permalink / raw)
To: ast, andrii.nakryiko, eddyz87
Cc: zhangxiaoqin, ihor.solodrai, linux-kernel, bpf, pengdonglin,
Alan Maguire
From: pengdonglin <pengdonglin@xiaomi.com>
This patch checks whether the BTF is sorted by name in ascending order.
If sorted, binary search will be used when looking up types.
Specifically, vmlinux and kernel module BTFs are always sorted during
the build phase with anonymous types placed before named types, so we
only need to identify the starting ID of named types.
Cc: Eduard Zingerman <eddyz87@gmail.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Cc: Alan Maguire <alan.maguire@oracle.com>
Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
---
kernel/bpf/btf.c | 56 ++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 56 insertions(+)
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index 0394f0c8ef74..a9e2345558c0 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -550,6 +550,60 @@ u32 btf_nr_types(const struct btf *btf)
return total;
}
+/*
+ * Assuming that types are sorted by name in ascending order.
+ */
+static int btf_compare_type_names(u32 *a, u32 *b, const struct btf *btf)
+{
+ const struct btf_type *ta = btf_type_by_id(btf, *a);
+ const struct btf_type *tb = btf_type_by_id(btf, *b);
+ const char *na, *nb;
+
+ na = btf_name_by_offset(btf, ta->name_off);
+ nb = btf_name_by_offset(btf, tb->name_off);
+ return strcmp(na, nb);
+}
+
+/* Note that vmlinux and kernel module BTFs are always sorted
+ * during the building phase.
+ */
+static void btf_check_sorted(struct btf *btf)
+{
+ const struct btf_type *t;
+ u32 sorted_start_id;
+ u32 i, n, k;
+
+ if (btf_is_kernel(btf) && !btf_is_module(btf)) {
+ for (i = btf_start_id(btf); i < n; i++) {
+ t = btf_type_by_id(btf, i);
+ if (t->name_off) {
+ btf->sorted_start_id = i;
+ return;
+ }
+ }
+ }
+
+ if (btf->nr_types < 2)
+ return;
+
+ sorted_start_id = 0;
+ n = btf_nr_types(btf);
+ for (i = btf_start_id(btf); i < n; i++) {
+ k = i + 1;
+ if (k < n && btf_compare_type_names(&i, &k, btf) > 0)
+ return;
+
+ if (sorted_start_id == 0) {
+ t = btf_type_by_id(btf, i);
+ if (t->name_off)
+ sorted_start_id = i;
+ }
+ }
+
+ if (sorted_start_id)
+ btf->sorted_start_id = sorted_start_id;
+}
+
static s32 btf_find_by_name_bsearch(const struct btf *btf, const char *name,
s32 start_id, s32 end_id)
{
@@ -6296,6 +6350,7 @@ static struct btf *btf_parse_base(struct btf_verifier_env *env, const char *name
if (err)
goto errout;
+ btf_check_sorted(btf);
refcount_set(&btf->refcnt, 1);
return btf;
@@ -6430,6 +6485,7 @@ static struct btf *btf_parse_module(const char *module_name, const void *data,
}
btf_verifier_env_free(env);
+ btf_check_sorted(btf);
refcount_set(&btf->refcnt, 1);
return btf;
--
2.34.1
^ permalink raw reply related [flat|nested] 72+ messages in thread
* [PATCH bpf-next v10 08/13] bpf: Skip anonymous types in type lookup for performance
2025-12-18 11:30 [PATCH bpf-next v10 00/13] Improve the performance of BTF type lookups with binary search Donglin Peng
` (6 preceding siblings ...)
2025-12-18 11:30 ` [PATCH bpf-next v10 07/13] btf: Verify BTF Sorting Donglin Peng
@ 2025-12-18 11:30 ` Donglin Peng
2025-12-18 22:21 ` Eduard Zingerman
2025-12-18 22:24 ` Eduard Zingerman
2025-12-18 11:30 ` [PATCH bpf-next v10 09/13] bpf: Optimize the performance of find_bpffs_btf_enums Donglin Peng
` (4 subsequent siblings)
12 siblings, 2 replies; 72+ messages in thread
From: Donglin Peng @ 2025-12-18 11:30 UTC (permalink / raw)
To: ast, andrii.nakryiko, eddyz87
Cc: zhangxiaoqin, ihor.solodrai, linux-kernel, bpf, pengdonglin,
Alan Maguire
From: pengdonglin <pengdonglin@xiaomi.com>
Currently, vmlinux and kernel module BTFs are unconditionally
sorted during the build phase, with named types placed at the
end. Thus, anonymous types should be skipped when starting the
search. In my vmlinux BTF, the number of anonymous types is
61,747, which means the loop count can be reduced by 61,747.
Cc: Eduard Zingerman <eddyz87@gmail.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Cc: Alan Maguire <alan.maguire@oracle.com>
Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
---
include/linux/btf.h | 1 +
kernel/bpf/btf.c | 24 ++++++++++++++++++++----
kernel/bpf/verifier.c | 7 +------
3 files changed, 22 insertions(+), 10 deletions(-)
diff --git a/include/linux/btf.h b/include/linux/btf.h
index f06976ffb63f..2d28f2b22ae5 100644
--- a/include/linux/btf.h
+++ b/include/linux/btf.h
@@ -220,6 +220,7 @@ bool btf_is_module(const struct btf *btf);
bool btf_is_vmlinux(const struct btf *btf);
struct module *btf_try_get_module(const struct btf *btf);
u32 btf_nr_types(const struct btf *btf);
+u32 btf_sorted_start_id(const struct btf *btf);
struct btf *btf_base_btf(const struct btf *btf);
bool btf_type_is_i32(const struct btf_type *t);
bool btf_type_is_i64(const struct btf_type *t);
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index a9e2345558c0..3aeb4f00cbfe 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -550,6 +550,11 @@ u32 btf_nr_types(const struct btf *btf)
return total;
}
+u32 btf_sorted_start_id(const struct btf *btf)
+{
+ return btf->sorted_start_id ?: (btf->start_id ?: 1);
+}
+
/*
* Assuming that types are sorted by name in ascending order.
*/
@@ -3540,9 +3545,14 @@ const char *btf_find_decl_tag_value(const struct btf *btf, const struct btf_type
{
const char *value = NULL;
const struct btf_type *t;
+ const struct btf *base_btf = btf;
int len, id;
- id = btf_find_next_decl_tag(btf, pt, comp_idx, tag_key, 0);
+ while (base_btf->base_btf)
+ base_btf = base_btf->base_btf;
+
+ id = btf_find_next_decl_tag(btf, pt, comp_idx, tag_key,
+ btf_sorted_start_id(base_btf) - 1);
if (id < 0)
return ERR_PTR(id);
@@ -7787,6 +7797,7 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
struct bpf_prog *prog = env->prog;
enum bpf_prog_type prog_type = prog->type;
struct btf *btf = prog->aux->btf;
+ struct btf *base_btf;
const struct btf_param *args;
const struct btf_type *t, *ref_t, *fn_t;
u32 i, nargs, btf_id;
@@ -7852,12 +7863,17 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
tname);
return -EINVAL;
}
+
+ base_btf = btf;
+ while (base_btf->base_btf)
+ base_btf = base_btf->base_btf;
+
/* Convert BTF function arguments into verifier types.
* Only PTR_TO_CTX and SCALAR are supported atm.
*/
for (i = 0; i < nargs; i++) {
u32 tags = 0;
- int id = 0;
+ int id = btf_sorted_start_id(base_btf) - 1;
/* 'arg:<tag>' decl_tag takes precedence over derivation of
* register type from BTF type itself
@@ -9338,7 +9354,7 @@ bpf_core_find_cands(struct bpf_core_ctx *ctx, u32 local_type_id)
}
/* Attempt to find target candidates in vmlinux BTF first */
- cands = bpf_core_add_cands(cands, main_btf, 1);
+ cands = bpf_core_add_cands(cands, main_btf, btf_sorted_start_id(main_btf));
if (IS_ERR(cands))
return ERR_CAST(cands);
@@ -9370,7 +9386,7 @@ bpf_core_find_cands(struct bpf_core_ctx *ctx, u32 local_type_id)
*/
btf_get(mod_btf);
spin_unlock_bh(&btf_idr_lock);
- cands = bpf_core_add_cands(cands, mod_btf, btf_nr_types(main_btf));
+ cands = bpf_core_add_cands(cands, mod_btf, btf_sorted_start_id(mod_btf));
btf_put(mod_btf);
if (IS_ERR(cands))
return ERR_CAST(cands);
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index d6b8a77fbe3b..1a9da59d8589 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -20651,12 +20651,7 @@ static int find_btf_percpu_datasec(struct btf *btf)
* types to look at only module's own BTF types.
*/
n = btf_nr_types(btf);
- if (btf_is_module(btf))
- i = btf_nr_types(btf_vmlinux);
- else
- i = 1;
-
- for(; i < n; i++) {
+ for (i = btf_sorted_start_id(btf); i < n; i++) {
t = btf_type_by_id(btf, i);
if (BTF_INFO_KIND(t->info) != BTF_KIND_DATASEC)
continue;
--
2.34.1
^ permalink raw reply related [flat|nested] 72+ messages in thread
* [PATCH bpf-next v10 09/13] bpf: Optimize the performance of find_bpffs_btf_enums
2025-12-18 11:30 [PATCH bpf-next v10 00/13] Improve the performance of BTF type lookups with binary search Donglin Peng
` (7 preceding siblings ...)
2025-12-18 11:30 ` [PATCH bpf-next v10 08/13] bpf: Skip anonymous types in type lookup for performance Donglin Peng
@ 2025-12-18 11:30 ` Donglin Peng
2025-12-19 0:01 ` Andrii Nakryiko
2025-12-18 11:30 ` [PATCH bpf-next v10 10/13] libbpf: Optimize the performance of determine_ptr_size Donglin Peng
` (3 subsequent siblings)
12 siblings, 1 reply; 72+ messages in thread
From: Donglin Peng @ 2025-12-18 11:30 UTC (permalink / raw)
To: ast, andrii.nakryiko, eddyz87
Cc: zhangxiaoqin, ihor.solodrai, linux-kernel, bpf, pengdonglin,
Alan Maguire
From: pengdonglin <pengdonglin@xiaomi.com>
Currently, vmlinux BTF is unconditionally sorted during
the build phase. The function btf_find_by_name_kind
executes the binary search branch, so find_bpffs_btf_enums
can be optimized by using btf_find_by_name_kind.
Cc: Eduard Zingerman <eddyz87@gmail.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Cc: Alan Maguire <alan.maguire@oracle.com>
Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
---
kernel/bpf/inode.c | 42 +++++++++++++++++++-----------------------
1 file changed, 19 insertions(+), 23 deletions(-)
diff --git a/kernel/bpf/inode.c b/kernel/bpf/inode.c
index 9f866a010dad..050fde1cf211 100644
--- a/kernel/bpf/inode.c
+++ b/kernel/bpf/inode.c
@@ -600,10 +600,18 @@ struct bpffs_btf_enums {
static int find_bpffs_btf_enums(struct bpffs_btf_enums *info)
{
+ struct {
+ const struct btf_type **type;
+ const char *name;
+ } btf_enums[] = {
+ {&info->cmd_t, "bpf_cmd"},
+ {&info->map_t, "bpf_map_type"},
+ {&info->prog_t, "bpf_prog_type"},
+ {&info->attach_t, "bpf_attach_type"},
+ };
const struct btf *btf;
const struct btf_type *t;
- const char *name;
- int i, n;
+ int i, id;
memset(info, 0, sizeof(*info));
@@ -615,30 +623,18 @@ static int find_bpffs_btf_enums(struct bpffs_btf_enums *info)
info->btf = btf;
- for (i = 1, n = btf_nr_types(btf); i < n; i++) {
- t = btf_type_by_id(btf, i);
- if (!btf_type_is_enum(t))
- continue;
+ for (i = 0; i < ARRAY_SIZE(btf_enums); i++) {
+ id = btf_find_by_name_kind(btf, btf_enums[i].name,
+ BTF_KIND_ENUM);
+ if (id < 0)
+ goto out;
- name = btf_name_by_offset(btf, t->name_off);
- if (!name)
- continue;
-
- if (strcmp(name, "bpf_cmd") == 0)
- info->cmd_t = t;
- else if (strcmp(name, "bpf_map_type") == 0)
- info->map_t = t;
- else if (strcmp(name, "bpf_prog_type") == 0)
- info->prog_t = t;
- else if (strcmp(name, "bpf_attach_type") == 0)
- info->attach_t = t;
- else
- continue;
-
- if (info->cmd_t && info->map_t && info->prog_t && info->attach_t)
- return 0;
+ t = btf_type_by_id(btf, id);
+ *btf_enums[i].type = t;
}
+ return 0;
+out:
return -ESRCH;
}
--
2.34.1
^ permalink raw reply related [flat|nested] 72+ messages in thread
* [PATCH bpf-next v10 10/13] libbpf: Optimize the performance of determine_ptr_size
2025-12-18 11:30 [PATCH bpf-next v10 00/13] Improve the performance of BTF type lookups with binary search Donglin Peng
` (8 preceding siblings ...)
2025-12-18 11:30 ` [PATCH bpf-next v10 09/13] bpf: Optimize the performance of find_bpffs_btf_enums Donglin Peng
@ 2025-12-18 11:30 ` Donglin Peng
2025-12-18 22:27 ` Eduard Zingerman
2025-12-19 0:03 ` Andrii Nakryiko
2025-12-18 11:30 ` [PATCH bpf-next v10 11/13] libbpf: Add btf_is_sorted and btf_sorted_start_id helpers to refactor the code Donglin Peng
` (2 subsequent siblings)
12 siblings, 2 replies; 72+ messages in thread
From: Donglin Peng @ 2025-12-18 11:30 UTC (permalink / raw)
To: ast, andrii.nakryiko, eddyz87
Cc: zhangxiaoqin, ihor.solodrai, linux-kernel, bpf, pengdonglin,
Alan Maguire
From: pengdonglin <pengdonglin@xiaomi.com>
Leverage the performance improvement of btf__find_by_name_kind() when
BTF is sorted. For sorted BTF, the function uses binary search with
O(log n) complexity instead of linear search, providing significant
performance benefits, especially for large BTF like vmlinux.
Cc: Eduard Zingerman <eddyz87@gmail.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Cc: Alan Maguire <alan.maguire@oracle.com>
Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
---
tools/lib/bpf/btf.c | 20 ++++++--------------
1 file changed, 6 insertions(+), 14 deletions(-)
diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
index c63d46b7d74b..b5b0898d033d 100644
--- a/tools/lib/bpf/btf.c
+++ b/tools/lib/bpf/btf.c
@@ -659,29 +659,21 @@ static int determine_ptr_size(const struct btf *btf)
"int long unsigned",
};
const struct btf_type *t;
- const char *name;
- int i, j, n;
+ int i, id;
if (btf->base_btf && btf->base_btf->ptr_sz > 0)
return btf->base_btf->ptr_sz;
- n = btf__type_cnt(btf);
- for (i = 1; i < n; i++) {
- t = btf__type_by_id(btf, i);
- if (!btf_is_int(t))
+ for (i = 0; i < ARRAY_SIZE(long_aliases); i++) {
+ id = btf__find_by_name_kind(btf, long_aliases[i], BTF_KIND_INT);
+ if (id < 0)
continue;
+ t = btf__type_by_id(btf, id);
if (t->size != 4 && t->size != 8)
continue;
- name = btf__name_by_offset(btf, t->name_off);
- if (!name)
- continue;
-
- for (j = 0; j < ARRAY_SIZE(long_aliases); j++) {
- if (strcmp(name, long_aliases[j]) == 0)
- return t->size;
- }
+ return t->size;
}
return -1;
--
2.34.1
^ permalink raw reply related [flat|nested] 72+ messages in thread
* [PATCH bpf-next v10 11/13] libbpf: Add btf_is_sorted and btf_sorted_start_id helpers to refactor the code
2025-12-18 11:30 [PATCH bpf-next v10 00/13] Improve the performance of BTF type lookups with binary search Donglin Peng
` (9 preceding siblings ...)
2025-12-18 11:30 ` [PATCH bpf-next v10 10/13] libbpf: Optimize the performance of determine_ptr_size Donglin Peng
@ 2025-12-18 11:30 ` Donglin Peng
2025-12-18 22:33 ` Eduard Zingerman
2025-12-19 0:05 ` Andrii Nakryiko
2025-12-18 11:30 ` [PATCH bpf-next v10 12/13] btf: Add btf_is_sorted " Donglin Peng
2025-12-18 11:30 ` [PATCH bpf-next v10 13/13] btf: Refactor the code by calling str_is_empty Donglin Peng
12 siblings, 2 replies; 72+ messages in thread
From: Donglin Peng @ 2025-12-18 11:30 UTC (permalink / raw)
To: ast, andrii.nakryiko, eddyz87
Cc: zhangxiaoqin, ihor.solodrai, linux-kernel, bpf, pengdonglin,
Alan Maguire
From: pengdonglin <pengdonglin@xiaomi.com>
Introduce two new helper functions to clarify the code and no
functional changes are introduced.
Cc: Eduard Zingerman <eddyz87@gmail.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Cc: Alan Maguire <alan.maguire@oracle.com>
Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
---
tools/lib/bpf/btf.c | 14 ++++++++++++--
tools/lib/bpf/libbpf_internal.h | 2 ++
2 files changed, 14 insertions(+), 2 deletions(-)
diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
index b5b0898d033d..571b72bd90b5 100644
--- a/tools/lib/bpf/btf.c
+++ b/tools/lib/bpf/btf.c
@@ -626,6 +626,16 @@ const struct btf *btf__base_btf(const struct btf *btf)
return btf->base_btf;
}
+int btf_sorted_start_id(const struct btf *btf)
+{
+ return btf->sorted_start_id;
+}
+
+bool btf_is_sorted(const struct btf *btf)
+{
+ return btf->sorted_start_id > 0;
+}
+
/* internal helper returning non-const pointer to a type */
struct btf_type *btf_type_by_id(const struct btf *btf, __u32 type_id)
{
@@ -976,11 +986,11 @@ static __s32 btf_find_by_name_kind(const struct btf *btf, int start_id,
if (kind == BTF_KIND_UNKN || strcmp(type_name, "void") == 0)
return 0;
- if (btf->sorted_start_id > 0 && type_name[0]) {
+ if (btf_is_sorted(btf) && type_name[0]) {
__s32 end_id = btf__type_cnt(btf) - 1;
/* skip anonymous types */
- start_id = max(start_id, btf->sorted_start_id);
+ start_id = max(start_id, btf_sorted_start_id(btf));
idx = btf_find_by_name_bsearch(btf, type_name, start_id, end_id);
if (unlikely(idx < 0))
return libbpf_err(-ENOENT);
diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_internal.h
index fc59b21b51b5..95e6848396b4 100644
--- a/tools/lib/bpf/libbpf_internal.h
+++ b/tools/lib/bpf/libbpf_internal.h
@@ -250,6 +250,8 @@ const struct btf_type *skip_mods_and_typedefs(const struct btf *btf, __u32 id, _
const struct btf_header *btf_header(const struct btf *btf);
void btf_set_base_btf(struct btf *btf, const struct btf *base_btf);
int btf_relocate(struct btf *btf, const struct btf *base_btf, __u32 **id_map);
+int btf_sorted_start_id(const struct btf *btf);
+bool btf_is_sorted(const struct btf *btf);
static inline enum btf_func_linkage btf_func_linkage(const struct btf_type *t)
{
--
2.34.1
^ permalink raw reply related [flat|nested] 72+ messages in thread
* [PATCH bpf-next v10 12/13] btf: Add btf_is_sorted to refactor the code
2025-12-18 11:30 [PATCH bpf-next v10 00/13] Improve the performance of BTF type lookups with binary search Donglin Peng
` (10 preceding siblings ...)
2025-12-18 11:30 ` [PATCH bpf-next v10 11/13] libbpf: Add btf_is_sorted and btf_sorted_start_id helpers to refactor the code Donglin Peng
@ 2025-12-18 11:30 ` Donglin Peng
2025-12-18 22:30 ` Eduard Zingerman
2025-12-19 0:05 ` Andrii Nakryiko
2025-12-18 11:30 ` [PATCH bpf-next v10 13/13] btf: Refactor the code by calling str_is_empty Donglin Peng
12 siblings, 2 replies; 72+ messages in thread
From: Donglin Peng @ 2025-12-18 11:30 UTC (permalink / raw)
To: ast, andrii.nakryiko, eddyz87
Cc: zhangxiaoqin, ihor.solodrai, linux-kernel, bpf, pengdonglin,
Alan Maguire
From: pengdonglin <pengdonglin@xiaomi.com>
Introduce a new helper function to clarify the code and no
functional changes are introduced.
Cc: Eduard Zingerman <eddyz87@gmail.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Cc: Alan Maguire <alan.maguire@oracle.com>
Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
---
include/linux/btf.h | 1 +
kernel/bpf/btf.c | 9 +++++++--
2 files changed, 8 insertions(+), 2 deletions(-)
diff --git a/include/linux/btf.h b/include/linux/btf.h
index 2d28f2b22ae5..947ed2abf632 100644
--- a/include/linux/btf.h
+++ b/include/linux/btf.h
@@ -221,6 +221,7 @@ bool btf_is_vmlinux(const struct btf *btf);
struct module *btf_try_get_module(const struct btf *btf);
u32 btf_nr_types(const struct btf *btf);
u32 btf_sorted_start_id(const struct btf *btf);
+bool btf_is_sorted(const struct btf *btf);
struct btf *btf_base_btf(const struct btf *btf);
bool btf_type_is_i32(const struct btf_type *t);
bool btf_type_is_i64(const struct btf_type *t);
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index 3aeb4f00cbfe..0f20887a6f02 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -555,6 +555,11 @@ u32 btf_sorted_start_id(const struct btf *btf)
return btf->sorted_start_id ?: (btf->start_id ?: 1);
}
+bool btf_is_sorted(const struct btf *btf)
+{
+ return btf->sorted_start_id > 0;
+}
+
/*
* Assuming that types are sorted by name in ascending order.
*/
@@ -649,9 +654,9 @@ s32 btf_find_by_name_kind(const struct btf *btf, const char *name, u8 kind)
return idx;
}
- if (btf->sorted_start_id > 0 && name[0]) {
+ if (btf_is_sorted(btf) && name[0]) {
/* skip anonymous types */
- s32 start_id = btf->sorted_start_id;
+ s32 start_id = btf_sorted_start_id(btf);
s32 end_id = btf_nr_types(btf) - 1;
idx = btf_find_by_name_bsearch(btf, name, start_id, end_id);
--
2.34.1
^ permalink raw reply related [flat|nested] 72+ messages in thread
* [PATCH bpf-next v10 13/13] btf: Refactor the code by calling str_is_empty
2025-12-18 11:30 [PATCH bpf-next v10 00/13] Improve the performance of BTF type lookups with binary search Donglin Peng
` (11 preceding siblings ...)
2025-12-18 11:30 ` [PATCH bpf-next v10 12/13] btf: Add btf_is_sorted " Donglin Peng
@ 2025-12-18 11:30 ` Donglin Peng
2025-12-18 22:34 ` Eduard Zingerman
12 siblings, 1 reply; 72+ messages in thread
From: Donglin Peng @ 2025-12-18 11:30 UTC (permalink / raw)
To: ast, andrii.nakryiko, eddyz87
Cc: zhangxiaoqin, ihor.solodrai, linux-kernel, bpf, pengdonglin,
Alan Maguire
From: pengdonglin <pengdonglin@xiaomi.com>
Calling the str_is_empty function to clarify the code and
no functional changes are introduced.
Cc: Eduard Zingerman <eddyz87@gmail.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Cc: Alan Maguire <alan.maguire@oracle.com>
Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
---
tools/lib/bpf/btf.c | 34 +++++++++++++++++-----------------
tools/lib/bpf/libbpf.c | 4 ++--
2 files changed, 19 insertions(+), 19 deletions(-)
diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
index 571b72bd90b5..013b1e5d396d 100644
--- a/tools/lib/bpf/btf.c
+++ b/tools/lib/bpf/btf.c
@@ -2169,7 +2169,7 @@ int btf__add_int(struct btf *btf, const char *name, size_t byte_sz, int encoding
int sz, name_off;
/* non-empty name */
- if (!name || !name[0])
+ if (str_is_empty(name))
return libbpf_err(-EINVAL);
/* byte_sz must be power of 2 */
if (!byte_sz || (byte_sz & (byte_sz - 1)) || byte_sz > 16)
@@ -2217,7 +2217,7 @@ int btf__add_float(struct btf *btf, const char *name, size_t byte_sz)
int sz, name_off;
/* non-empty name */
- if (!name || !name[0])
+ if (str_is_empty(name))
return libbpf_err(-EINVAL);
/* byte_sz must be one of the explicitly allowed values */
@@ -2272,7 +2272,7 @@ static int btf_add_ref_kind(struct btf *btf, int kind, const char *name, int ref
if (!t)
return libbpf_err(-ENOMEM);
- if (name && name[0]) {
+ if (!str_is_empty(name)) {
name_off = btf__add_str(btf, name);
if (name_off < 0)
return name_off;
@@ -2349,7 +2349,7 @@ static int btf_add_composite(struct btf *btf, int kind, const char *name, __u32
if (!t)
return libbpf_err(-ENOMEM);
- if (name && name[0]) {
+ if (!str_is_empty(name)) {
name_off = btf__add_str(btf, name);
if (name_off < 0)
return name_off;
@@ -2450,7 +2450,7 @@ int btf__add_field(struct btf *btf, const char *name, int type_id,
if (!m)
return libbpf_err(-ENOMEM);
- if (name && name[0]) {
+ if (!str_is_empty(name)) {
name_off = btf__add_str(btf, name);
if (name_off < 0)
return name_off;
@@ -2488,7 +2488,7 @@ static int btf_add_enum_common(struct btf *btf, const char *name, __u32 byte_sz,
if (!t)
return libbpf_err(-ENOMEM);
- if (name && name[0]) {
+ if (!str_is_empty(name)) {
name_off = btf__add_str(btf, name);
if (name_off < 0)
return name_off;
@@ -2546,7 +2546,7 @@ int btf__add_enum_value(struct btf *btf, const char *name, __s64 value)
return libbpf_err(-EINVAL);
/* non-empty name */
- if (!name || !name[0])
+ if (str_is_empty(name))
return libbpf_err(-EINVAL);
if (value < INT_MIN || value > UINT_MAX)
return libbpf_err(-E2BIG);
@@ -2623,7 +2623,7 @@ int btf__add_enum64_value(struct btf *btf, const char *name, __u64 value)
return libbpf_err(-EINVAL);
/* non-empty name */
- if (!name || !name[0])
+ if (str_is_empty(name))
return libbpf_err(-EINVAL);
/* decompose and invalidate raw data */
@@ -2663,7 +2663,7 @@ int btf__add_enum64_value(struct btf *btf, const char *name, __u64 value)
*/
int btf__add_fwd(struct btf *btf, const char *name, enum btf_fwd_kind fwd_kind)
{
- if (!name || !name[0])
+ if (str_is_empty(name))
return libbpf_err(-EINVAL);
switch (fwd_kind) {
@@ -2699,7 +2699,7 @@ int btf__add_fwd(struct btf *btf, const char *name, enum btf_fwd_kind fwd_kind)
*/
int btf__add_typedef(struct btf *btf, const char *name, int ref_type_id)
{
- if (!name || !name[0])
+ if (str_is_empty(name))
return libbpf_err(-EINVAL);
return btf_add_ref_kind(btf, BTF_KIND_TYPEDEF, name, ref_type_id, 0);
@@ -2751,7 +2751,7 @@ int btf__add_restrict(struct btf *btf, int ref_type_id)
*/
int btf__add_type_tag(struct btf *btf, const char *value, int ref_type_id)
{
- if (!value || !value[0])
+ if (str_is_empty(value))
return libbpf_err(-EINVAL);
return btf_add_ref_kind(btf, BTF_KIND_TYPE_TAG, value, ref_type_id, 0);
@@ -2768,7 +2768,7 @@ int btf__add_type_tag(struct btf *btf, const char *value, int ref_type_id)
*/
int btf__add_type_attr(struct btf *btf, const char *value, int ref_type_id)
{
- if (!value || !value[0])
+ if (str_is_empty(value))
return libbpf_err(-EINVAL);
return btf_add_ref_kind(btf, BTF_KIND_TYPE_TAG, value, ref_type_id, 1);
@@ -2787,7 +2787,7 @@ int btf__add_func(struct btf *btf, const char *name,
{
int id;
- if (!name || !name[0])
+ if (str_is_empty(name))
return libbpf_err(-EINVAL);
if (linkage != BTF_FUNC_STATIC && linkage != BTF_FUNC_GLOBAL &&
linkage != BTF_FUNC_EXTERN)
@@ -2873,7 +2873,7 @@ int btf__add_func_param(struct btf *btf, const char *name, int type_id)
if (!p)
return libbpf_err(-ENOMEM);
- if (name && name[0]) {
+ if (!str_is_empty(name)) {
name_off = btf__add_str(btf, name);
if (name_off < 0)
return name_off;
@@ -2908,7 +2908,7 @@ int btf__add_var(struct btf *btf, const char *name, int linkage, int type_id)
int sz, name_off;
/* non-empty name */
- if (!name || !name[0])
+ if (str_is_empty(name))
return libbpf_err(-EINVAL);
if (linkage != BTF_VAR_STATIC && linkage != BTF_VAR_GLOBAL_ALLOCATED &&
linkage != BTF_VAR_GLOBAL_EXTERN)
@@ -2957,7 +2957,7 @@ int btf__add_datasec(struct btf *btf, const char *name, __u32 byte_sz)
int sz, name_off;
/* non-empty name */
- if (!name || !name[0])
+ if (str_is_empty(name))
return libbpf_err(-EINVAL);
if (btf_ensure_modifiable(btf))
@@ -3034,7 +3034,7 @@ static int btf_add_decl_tag(struct btf *btf, const char *value, int ref_type_id,
struct btf_type *t;
int sz, value_off;
- if (!value || !value[0] || component_idx < -1)
+ if (str_is_empty(value) || component_idx < -1)
return libbpf_err(-EINVAL);
if (validate_type_id(ref_type_id))
diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index 1a52d818a76c..96c6db972855 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -2904,7 +2904,7 @@ static int bpf_object__init_user_btf_map(struct bpf_object *obj,
var_extra = btf_var(var);
map_name = btf__name_by_offset(obj->btf, var->name_off);
- if (map_name == NULL || map_name[0] == '\0') {
+ if (str_is_empty(map_name)) {
pr_warn("map #%d: empty name.\n", var_idx);
return -EINVAL;
}
@@ -4281,7 +4281,7 @@ static int bpf_object__collect_externs(struct bpf_object *obj)
if (!sym_is_extern(sym))
continue;
ext_name = elf_sym_str(obj, sym->st_name);
- if (!ext_name || !ext_name[0])
+ if (str_is_empty(ext_name))
continue;
ext = obj->externs;
--
2.34.1
^ permalink raw reply related [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 04/13] libbpf: Optimize type lookup with binary search for sorted BTF
2025-12-18 11:30 ` [PATCH bpf-next v10 04/13] libbpf: Optimize type lookup with binary search for sorted BTF Donglin Peng
@ 2025-12-18 19:32 ` Eduard Zingerman
2025-12-18 23:29 ` Andrii Nakryiko
1 sibling, 0 replies; 72+ messages in thread
From: Eduard Zingerman @ 2025-12-18 19:32 UTC (permalink / raw)
To: Donglin Peng, ast, andrii.nakryiko
Cc: zhangxiaoqin, ihor.solodrai, linux-kernel, bpf, pengdonglin,
Alan Maguire
On Thu, 2025-12-18 at 19:30 +0800, Donglin Peng wrote:
> From: pengdonglin <pengdonglin@xiaomi.com>
>
> This patch introduces binary search optimization for BTF type lookups
> when the BTF instance contains sorted types.
>
> The optimization significantly improves performance when searching for
> types in large BTF instances with sorted types. For unsorted BTF, the
> implementation falls back to the original linear search.
>
> Cc: Eduard Zingerman <eddyz87@gmail.com>
> Cc: Alexei Starovoitov <ast@kernel.org>
> Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> Cc: Alan Maguire <alan.maguire@oracle.com>
> Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> ---
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
[...]
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 05/13] libbpf: Verify BTF Sorting
2025-12-18 11:30 ` [PATCH bpf-next v10 05/13] libbpf: Verify BTF Sorting Donglin Peng
@ 2025-12-18 19:38 ` Eduard Zingerman
2025-12-18 23:44 ` Andrii Nakryiko
1 sibling, 0 replies; 72+ messages in thread
From: Eduard Zingerman @ 2025-12-18 19:38 UTC (permalink / raw)
To: Donglin Peng, ast, andrii.nakryiko
Cc: zhangxiaoqin, ihor.solodrai, linux-kernel, bpf, pengdonglin,
Alan Maguire
On Thu, 2025-12-18 at 19:30 +0800, Donglin Peng wrote:
> From: pengdonglin <pengdonglin@xiaomi.com>
>
> This patch checks whether the BTF is sorted by name in ascending
> order. If sorted, binary search will be used when looking up types.
>
> Cc: Eduard Zingerman <eddyz87@gmail.com>
> Cc: Alexei Starovoitov <ast@kernel.org>
> Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> Cc: Alan Maguire <alan.maguire@oracle.com>
> Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> ---
(But could you please fix the btf_compare_type_names() prototype?)
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
> tools/lib/bpf/btf.c | 41 +++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 41 insertions(+)
>
> diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> index 2facb57d7e5f..c63d46b7d74b 100644
> --- a/tools/lib/bpf/btf.c
> +++ b/tools/lib/bpf/btf.c
> @@ -899,6 +899,46 @@ int btf__resolve_type(const struct btf *btf, __u32 type_id)
> return type_id;
> }
>
> +/*
> + * Assuming that types are sorted by name in ascending order.
> + */
> +static int btf_compare_type_names(__u32 *a, __u32 *b, const struct btf *btf)
Nit: still no need for 'a' and 'b' to be pointers.
> +{
> + struct btf_type *ta = btf_type_by_id(btf, *a);
> + struct btf_type *tb = btf_type_by_id(btf, *b);
> + const char *na, *nb;
> +
> + na = btf__str_by_offset(btf, ta->name_off);
> + nb = btf__str_by_offset(btf, tb->name_off);
> + return strcmp(na, nb);
> +}
[...]
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 06/13] btf: Optimize type lookup with binary search
2025-12-18 11:30 ` [PATCH bpf-next v10 06/13] btf: Optimize type lookup with binary search Donglin Peng
@ 2025-12-18 21:38 ` Eduard Zingerman
2025-12-19 5:07 ` Donglin Peng
0 siblings, 1 reply; 72+ messages in thread
From: Eduard Zingerman @ 2025-12-18 21:38 UTC (permalink / raw)
To: Donglin Peng, ast, andrii.nakryiko
Cc: zhangxiaoqin, ihor.solodrai, linux-kernel, bpf, pengdonglin,
Alan Maguire
On Thu, 2025-12-18 at 19:30 +0800, Donglin Peng wrote:
> From: pengdonglin <pengdonglin@xiaomi.com>
>
> Improve btf_find_by_name_kind() performance by adding binary search
> support for sorted types. Falls back to linear search for compatibility.
>
> Cc: Eduard Zingerman <eddyz87@gmail.com>
> Cc: Alexei Starovoitov <ast@kernel.org>
> Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> Cc: Alan Maguire <alan.maguire@oracle.com>
> Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> ---
(One nit below).
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
> kernel/bpf/btf.c | 85 +++++++++++++++++++++++++++++++++++++++++++-----
> 1 file changed, 76 insertions(+), 9 deletions(-)
>
> diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
> index 0de8fc8a0e0b..0394f0c8ef74 100644
> --- a/kernel/bpf/btf.c
> +++ b/kernel/bpf/btf.c
> @@ -259,6 +259,7 @@ struct btf {
> void *nohdr_data;
> struct btf_header hdr;
> u32 nr_types; /* includes VOID for base BTF */
> + u32 sorted_start_id;
> u32 types_size;
> u32 data_size;
> refcount_t refcnt;
> @@ -494,6 +495,11 @@ static bool btf_type_is_modifier(const struct btf_type *t)
> return false;
> }
>
> +static int btf_start_id(const struct btf *btf)
> +{
> + return btf->start_id + (btf->base_btf ? 0 : 1);
> +}
> +
> bool btf_type_is_void(const struct btf_type *t)
> {
> return t == &btf_void;
> @@ -544,21 +550,79 @@ u32 btf_nr_types(const struct btf *btf)
> return total;
> }
>
> +static s32 btf_find_by_name_bsearch(const struct btf *btf, const char *name,
> + s32 start_id, s32 end_id)
> +{
> + const struct btf_type *t;
> + const char *tname;
> + s32 l, r, m, lmost = -ENOENT;
> + int ret;
> +
> + l = start_id;
> + r = end_id;
> + while (l <= r) {
> + m = l + (r - l) / 2;
> + t = btf_type_by_id(btf, m);
> + tname = btf_name_by_offset(btf, t->name_off);
> + ret = strcmp(tname, name);
> + if (ret < 0) {
> + l = m + 1;
> + } else {
> + if (ret == 0)
> + lmost = m;
> + r = m - 1;
> + }
> + }
> +
> + return lmost;
> +}
> +
> s32 btf_find_by_name_kind(const struct btf *btf, const char *name, u8 kind)
> {
> + const struct btf *base_btf = btf_base_btf(btf);
> const struct btf_type *t;
> const char *tname;
> - u32 i, total;
> + s32 idx;
>
> - total = btf_nr_types(btf);
> - for (i = 1; i < total; i++) {
> - t = btf_type_by_id(btf, i);
> - if (BTF_INFO_KIND(t->info) != kind)
> - continue;
> + if (base_btf) {
> + idx = btf_find_by_name_kind(base_btf, name, kind);
> + if (idx > 0)
> + return idx;
> + }
>
> - tname = btf_name_by_offset(btf, t->name_off);
> - if (!strcmp(tname, name))
> - return i;
> + if (btf->sorted_start_id > 0 && name[0]) {
^^^^^^^^^^^^^^^^^^^^^^^^
Nit: Maybe pull the is_sorted helper into this patch-set?
> + /* skip anonymous types */
> + s32 start_id = btf->sorted_start_id;
> + s32 end_id = btf_nr_types(btf) - 1;
> +
> + idx = btf_find_by_name_bsearch(btf, name, start_id, end_id);
> + if (idx < 0)
> + return -ENOENT;
> +
> + t = btf_type_by_id(btf, idx);
> + if (BTF_INFO_KIND(t->info) == kind)
> + return idx;
> +
> + for (idx++; idx <= end_id; idx++) {
> + t = btf_type_by_id(btf, idx);
> + tname = btf_name_by_offset(btf, t->name_off);
> + if (strcmp(tname, name) != 0)
> + return -ENOENT;
> + if (BTF_INFO_KIND(t->info) == kind)
> + return idx;
> + }
> + } else {
> + u32 i, total;
> +
> + total = btf_nr_types(btf);
> + for (i = btf_start_id(btf); i < total; i++) {
> + t = btf_type_by_id(btf, i);
> + if (BTF_INFO_KIND(t->info) != kind)
> + continue;
> + tname = btf_name_by_offset(btf, t->name_off);
> + if (strcmp(tname, name) == 0)
> + return i;
> + }
> }
>
> return -ENOENT;
> @@ -5791,6 +5855,7 @@ static struct btf *btf_parse(const union bpf_attr *attr, bpfptr_t uattr, u32 uat
> goto errout;
> }
> env->btf = btf;
> + btf->sorted_start_id = 0;
>
> data = kvmalloc(attr->btf_size, GFP_KERNEL | __GFP_NOWARN);
> if (!data) {
> @@ -6210,6 +6275,7 @@ static struct btf *btf_parse_base(struct btf_verifier_env *env, const char *name
> btf->data = data;
> btf->data_size = data_size;
> btf->kernel_btf = true;
> + btf->sorted_start_id = 0;
> snprintf(btf->name, sizeof(btf->name), "%s", name);
>
> err = btf_parse_hdr(env);
> @@ -6327,6 +6393,7 @@ static struct btf *btf_parse_module(const char *module_name, const void *data,
> btf->start_id = base_btf->nr_types;
> btf->start_str_off = base_btf->hdr.str_len;
> btf->kernel_btf = true;
> + btf->sorted_start_id = 0;
> snprintf(btf->name, sizeof(btf->name), "%s", module_name);
>
> btf->data = kvmemdup(data, data_size, GFP_KERNEL | __GFP_NOWARN);
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 07/13] btf: Verify BTF Sorting
2025-12-18 11:30 ` [PATCH bpf-next v10 07/13] btf: Verify BTF Sorting Donglin Peng
@ 2025-12-18 21:43 ` Eduard Zingerman
2025-12-19 5:10 ` Donglin Peng
2025-12-18 23:46 ` Andrii Nakryiko
1 sibling, 1 reply; 72+ messages in thread
From: Eduard Zingerman @ 2025-12-18 21:43 UTC (permalink / raw)
To: Donglin Peng, ast, andrii.nakryiko
Cc: zhangxiaoqin, ihor.solodrai, linux-kernel, bpf, pengdonglin,
Alan Maguire
On Thu, 2025-12-18 at 19:30 +0800, Donglin Peng wrote:
> From: pengdonglin <pengdonglin@xiaomi.com>
>
> This patch checks whether the BTF is sorted by name in ascending order.
> If sorted, binary search will be used when looking up types.
>
> Specifically, vmlinux and kernel module BTFs are always sorted during
> the build phase with anonymous types placed before named types, so we
> only need to identify the starting ID of named types.
>
> Cc: Eduard Zingerman <eddyz87@gmail.com>
> Cc: Alexei Starovoitov <ast@kernel.org>
> Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> Cc: Alan Maguire <alan.maguire@oracle.com>
> Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> ---
Lgtm, but please take a look at a few nits.
> kernel/bpf/btf.c | 56 ++++++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 56 insertions(+)
>
> diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
> index 0394f0c8ef74..a9e2345558c0 100644
> --- a/kernel/bpf/btf.c
> +++ b/kernel/bpf/btf.c
> @@ -550,6 +550,60 @@ u32 btf_nr_types(const struct btf *btf)
> return total;
> }
>
> +/*
> + * Assuming that types are sorted by name in ascending order.
> + */
> +static int btf_compare_type_names(u32 *a, u32 *b, const struct btf *btf)
Nit: no need for 'a' and 'b' to be pointers.
> +{
> + const struct btf_type *ta = btf_type_by_id(btf, *a);
> + const struct btf_type *tb = btf_type_by_id(btf, *b);
> + const char *na, *nb;
> +
> + na = btf_name_by_offset(btf, ta->name_off);
> + nb = btf_name_by_offset(btf, tb->name_off);
> + return strcmp(na, nb);
> +}
> +
> +/* Note that vmlinux and kernel module BTFs are always sorted
> + * during the building phase.
> + */
> +static void btf_check_sorted(struct btf *btf)
> +{
> + const struct btf_type *t;
> + u32 sorted_start_id;
> + u32 i, n, k;
> +
> + if (btf_is_kernel(btf) && !btf_is_module(btf)) {
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Nit: there is a btf_is_vmlinux() helper, which does the same thing.
> + for (i = btf_start_id(btf); i < n; i++) {
> + t = btf_type_by_id(btf, i);
> + if (t->name_off) {
> + btf->sorted_start_id = i;
> + return;
> + }
> + }
Nit: return here?
> + }
> +
> + if (btf->nr_types < 2)
> + return;
> +
> + sorted_start_id = 0;
> + n = btf_nr_types(btf);
> + for (i = btf_start_id(btf); i < n; i++) {
> + k = i + 1;
> + if (k < n && btf_compare_type_names(&i, &k, btf) > 0)
> + return;
> +
> + if (sorted_start_id == 0) {
> + t = btf_type_by_id(btf, i);
> + if (t->name_off)
> + sorted_start_id = i;
> + }
> + }
> +
> + if (sorted_start_id)
> + btf->sorted_start_id = sorted_start_id;
> +}
> +
> static s32 btf_find_by_name_bsearch(const struct btf *btf, const char *name,
> s32 start_id, s32 end_id)
> {
> @@ -6296,6 +6350,7 @@ static struct btf *btf_parse_base(struct btf_verifier_env *env, const char *name
> if (err)
> goto errout;
>
> + btf_check_sorted(btf);
> refcount_set(&btf->refcnt, 1);
>
> return btf;
> @@ -6430,6 +6485,7 @@ static struct btf *btf_parse_module(const char *module_name, const void *data,
> }
>
> btf_verifier_env_free(env);
> + btf_check_sorted(btf);
> refcount_set(&btf->refcnt, 1);
> return btf;
>
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 08/13] bpf: Skip anonymous types in type lookup for performance
2025-12-18 11:30 ` [PATCH bpf-next v10 08/13] bpf: Skip anonymous types in type lookup for performance Donglin Peng
@ 2025-12-18 22:21 ` Eduard Zingerman
2025-12-18 23:59 ` Andrii Nakryiko
2025-12-18 22:24 ` Eduard Zingerman
1 sibling, 1 reply; 72+ messages in thread
From: Eduard Zingerman @ 2025-12-18 22:21 UTC (permalink / raw)
To: Donglin Peng, ast, andrii.nakryiko
Cc: zhangxiaoqin, ihor.solodrai, linux-kernel, bpf, pengdonglin,
Alan Maguire
On Thu, 2025-12-18 at 19:30 +0800, Donglin Peng wrote:
> From: pengdonglin <pengdonglin@xiaomi.com>
>
> Currently, vmlinux and kernel module BTFs are unconditionally
> sorted during the build phase, with named types placed at the
> end. Thus, anonymous types should be skipped when starting the
> search. In my vmlinux BTF, the number of anonymous types is
> 61,747, which means the loop count can be reduced by 61,747.
>
> Cc: Eduard Zingerman <eddyz87@gmail.com>
> Cc: Alexei Starovoitov <ast@kernel.org>
> Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> Cc: Alan Maguire <alan.maguire@oracle.com>
> Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> ---
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
> include/linux/btf.h | 1 +
> kernel/bpf/btf.c | 24 ++++++++++++++++++++----
> kernel/bpf/verifier.c | 7 +------
> 3 files changed, 22 insertions(+), 10 deletions(-)
>
> diff --git a/include/linux/btf.h b/include/linux/btf.h
> index f06976ffb63f..2d28f2b22ae5 100644
> --- a/include/linux/btf.h
> +++ b/include/linux/btf.h
> @@ -220,6 +220,7 @@ bool btf_is_module(const struct btf *btf);
> bool btf_is_vmlinux(const struct btf *btf);
> struct module *btf_try_get_module(const struct btf *btf);
> u32 btf_nr_types(const struct btf *btf);
> +u32 btf_sorted_start_id(const struct btf *btf);
> struct btf *btf_base_btf(const struct btf *btf);
> bool btf_type_is_i32(const struct btf_type *t);
> bool btf_type_is_i64(const struct btf_type *t);
> diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
> index a9e2345558c0..3aeb4f00cbfe 100644
> --- a/kernel/bpf/btf.c
> +++ b/kernel/bpf/btf.c
> @@ -550,6 +550,11 @@ u32 btf_nr_types(const struct btf *btf)
> return total;
> }
>
> +u32 btf_sorted_start_id(const struct btf *btf)
Nit: the name is a bit confusing, given that it not always returns the
start id for sorted part. btf_maybe_first_named_id?
Can't figure out a good name :(
> +{
> + return btf->sorted_start_id ?: (btf->start_id ?: 1);
> +}
> +
> /*
> * Assuming that types are sorted by name in ascending order.
> */
[...]
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 08/13] bpf: Skip anonymous types in type lookup for performance
2025-12-18 11:30 ` [PATCH bpf-next v10 08/13] bpf: Skip anonymous types in type lookup for performance Donglin Peng
2025-12-18 22:21 ` Eduard Zingerman
@ 2025-12-18 22:24 ` Eduard Zingerman
1 sibling, 0 replies; 72+ messages in thread
From: Eduard Zingerman @ 2025-12-18 22:24 UTC (permalink / raw)
To: Donglin Peng, ast, andrii.nakryiko
Cc: zhangxiaoqin, ihor.solodrai, linux-kernel, bpf, pengdonglin,
Alan Maguire
On Thu, 2025-12-18 at 19:30 +0800, Donglin Peng wrote:
> From: pengdonglin <pengdonglin@xiaomi.com>
>
> Currently, vmlinux and kernel module BTFs are unconditionally
> sorted during the build phase, with named types placed at the
> end. Thus, anonymous types should be skipped when starting the
> search. In my vmlinux BTF, the number of anonymous types is
> 61,747, which means the loop count can be reduced by 61,747.
>
> Cc: Eduard Zingerman <eddyz87@gmail.com>
> Cc: Alexei Starovoitov <ast@kernel.org>
> Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> Cc: Alan Maguire <alan.maguire@oracle.com>
> Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> ---
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
[...]
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 10/13] libbpf: Optimize the performance of determine_ptr_size
2025-12-18 11:30 ` [PATCH bpf-next v10 10/13] libbpf: Optimize the performance of determine_ptr_size Donglin Peng
@ 2025-12-18 22:27 ` Eduard Zingerman
2025-12-19 0:03 ` Andrii Nakryiko
1 sibling, 0 replies; 72+ messages in thread
From: Eduard Zingerman @ 2025-12-18 22:27 UTC (permalink / raw)
To: Donglin Peng, ast, andrii.nakryiko
Cc: zhangxiaoqin, ihor.solodrai, linux-kernel, bpf, pengdonglin,
Alan Maguire
On Thu, 2025-12-18 at 19:30 +0800, Donglin Peng wrote:
> From: pengdonglin <pengdonglin@xiaomi.com>
>
> Leverage the performance improvement of btf__find_by_name_kind() when
> BTF is sorted. For sorted BTF, the function uses binary search with
> O(log n) complexity instead of linear search, providing significant
> performance benefits, especially for large BTF like vmlinux.
>
> Cc: Eduard Zingerman <eddyz87@gmail.com>
> Cc: Alexei Starovoitov <ast@kernel.org>
> Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> Cc: Alan Maguire <alan.maguire@oracle.com>
> Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> ---
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
[...]
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 12/13] btf: Add btf_is_sorted to refactor the code
2025-12-18 11:30 ` [PATCH bpf-next v10 12/13] btf: Add btf_is_sorted " Donglin Peng
@ 2025-12-18 22:30 ` Eduard Zingerman
2025-12-19 0:05 ` Andrii Nakryiko
1 sibling, 0 replies; 72+ messages in thread
From: Eduard Zingerman @ 2025-12-18 22:30 UTC (permalink / raw)
To: Donglin Peng, ast, andrii.nakryiko
Cc: zhangxiaoqin, ihor.solodrai, linux-kernel, bpf, pengdonglin,
Alan Maguire
On Thu, 2025-12-18 at 19:30 +0800, Donglin Peng wrote:
> From: pengdonglin <pengdonglin@xiaomi.com>
>
> Introduce a new helper function to clarify the code and no
> functional changes are introduced.
>
> Cc: Eduard Zingerman <eddyz87@gmail.com>
> Cc: Alexei Starovoitov <ast@kernel.org>
> Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> Cc: Alan Maguire <alan.maguire@oracle.com>
> Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> ---
Please squash this patch, there is no need to keep this change separate.
[...]
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 11/13] libbpf: Add btf_is_sorted and btf_sorted_start_id helpers to refactor the code
2025-12-18 11:30 ` [PATCH bpf-next v10 11/13] libbpf: Add btf_is_sorted and btf_sorted_start_id helpers to refactor the code Donglin Peng
@ 2025-12-18 22:33 ` Eduard Zingerman
2025-12-19 5:49 ` Donglin Peng
2025-12-19 0:05 ` Andrii Nakryiko
1 sibling, 1 reply; 72+ messages in thread
From: Eduard Zingerman @ 2025-12-18 22:33 UTC (permalink / raw)
To: Donglin Peng, ast, andrii.nakryiko
Cc: zhangxiaoqin, ihor.solodrai, linux-kernel, bpf, pengdonglin,
Alan Maguire
On Thu, 2025-12-18 at 19:30 +0800, Donglin Peng wrote:
> From: pengdonglin <pengdonglin@xiaomi.com>
>
> Introduce two new helper functions to clarify the code and no
> functional changes are introduced.
>
> Cc: Eduard Zingerman <eddyz87@gmail.com>
> Cc: Alexei Starovoitov <ast@kernel.org>
> Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> Cc: Alan Maguire <alan.maguire@oracle.com>
> Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> ---
> tools/lib/bpf/btf.c | 14 ++++++++++++--
> tools/lib/bpf/libbpf_internal.h | 2 ++
> 2 files changed, 14 insertions(+), 2 deletions(-)
>
> diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> index b5b0898d033d..571b72bd90b5 100644
> --- a/tools/lib/bpf/btf.c
> +++ b/tools/lib/bpf/btf.c
> @@ -626,6 +626,16 @@ const struct btf *btf__base_btf(const struct btf *btf)
> return btf->base_btf;
> }
>
> +int btf_sorted_start_id(const struct btf *btf)
> +{
> + return btf->sorted_start_id;
> +}
Having this function declared differently in kernel and in libbpf is a
bit confusing. Is it needed in libbpf at all?
> +bool btf_is_sorted(const struct btf *btf)
> +{
> + return btf->sorted_start_id > 0;
> +}
> +
Please squash this with the first btf_find_by_name_kind() change.
> /* internal helper returning non-const pointer to a type */
> struct btf_type *btf_type_by_id(const struct btf *btf, __u32 type_id)
> {
> @@ -976,11 +986,11 @@ static __s32 btf_find_by_name_kind(const struct btf *btf, int start_id,
> if (kind == BTF_KIND_UNKN || strcmp(type_name, "void") == 0)
> return 0;
>
> - if (btf->sorted_start_id > 0 && type_name[0]) {
> + if (btf_is_sorted(btf) && type_name[0]) {
> __s32 end_id = btf__type_cnt(btf) - 1;
>
> /* skip anonymous types */
> - start_id = max(start_id, btf->sorted_start_id);
> + start_id = max(start_id, btf_sorted_start_id(btf));
> idx = btf_find_by_name_bsearch(btf, type_name, start_id, end_id);
> if (unlikely(idx < 0))
> return libbpf_err(-ENOENT);
> diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_internal.h
> index fc59b21b51b5..95e6848396b4 100644
> --- a/tools/lib/bpf/libbpf_internal.h
> +++ b/tools/lib/bpf/libbpf_internal.h
> @@ -250,6 +250,8 @@ const struct btf_type *skip_mods_and_typedefs(const struct btf *btf, __u32 id, _
> const struct btf_header *btf_header(const struct btf *btf);
> void btf_set_base_btf(struct btf *btf, const struct btf *base_btf);
> int btf_relocate(struct btf *btf, const struct btf *base_btf, __u32 **id_map);
> +int btf_sorted_start_id(const struct btf *btf);
> +bool btf_is_sorted(const struct btf *btf);
>
> static inline enum btf_func_linkage btf_func_linkage(const struct btf_type *t)
> {
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 13/13] btf: Refactor the code by calling str_is_empty
2025-12-18 11:30 ` [PATCH bpf-next v10 13/13] btf: Refactor the code by calling str_is_empty Donglin Peng
@ 2025-12-18 22:34 ` Eduard Zingerman
0 siblings, 0 replies; 72+ messages in thread
From: Eduard Zingerman @ 2025-12-18 22:34 UTC (permalink / raw)
To: Donglin Peng, ast, andrii.nakryiko
Cc: zhangxiaoqin, ihor.solodrai, linux-kernel, bpf, pengdonglin,
Alan Maguire
On Thu, 2025-12-18 at 19:30 +0800, Donglin Peng wrote:
> From: pengdonglin <pengdonglin@xiaomi.com>
>
> Calling the str_is_empty function to clarify the code and
> no functional changes are introduced.
>
> Cc: Eduard Zingerman <eddyz87@gmail.com>
> Cc: Alexei Starovoitov <ast@kernel.org>
> Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> Cc: Alan Maguire <alan.maguire@oracle.com>
> Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> ---
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
[...]
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 01/13] libbpf: Add BTF permutation support for type reordering
2025-12-18 11:30 ` [PATCH bpf-next v10 01/13] libbpf: Add BTF permutation support for type reordering Donglin Peng
@ 2025-12-18 23:02 ` Andrii Nakryiko
2025-12-19 3:14 ` Donglin Peng
0 siblings, 1 reply; 72+ messages in thread
From: Andrii Nakryiko @ 2025-12-18 23:02 UTC (permalink / raw)
To: Donglin Peng
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
>
> From: pengdonglin <pengdonglin@xiaomi.com>
>
> Introduce btf__permute() API to allow in-place rearrangement of BTF types.
> This function reorganizes BTF type order according to a provided array of
> type IDs, updating all type references to maintain consistency.
>
> Cc: Eduard Zingerman <eddyz87@gmail.com>
> Cc: Alexei Starovoitov <ast@kernel.org>
> Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> Cc: Alan Maguire <alan.maguire@oracle.com>
> Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> Acked-by: Eduard Zingerman <eddyz87@gmail.com>
> ---
> tools/lib/bpf/btf.c | 119 +++++++++++++++++++++++++++++++++++++++
> tools/lib/bpf/btf.h | 36 ++++++++++++
> tools/lib/bpf/libbpf.map | 1 +
> 3 files changed, 156 insertions(+)
>
> diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> index b136572e889a..ab204ca403dc 100644
> --- a/tools/lib/bpf/btf.c
> +++ b/tools/lib/bpf/btf.c
> @@ -5887,3 +5887,122 @@ int btf__relocate(struct btf *btf, const struct btf *base_btf)
> btf->owns_base = false;
> return libbpf_err(err);
> }
> +
> +struct btf_permute {
> + struct btf *btf;
> + __u32 *id_map;
> +};
> +
> +/* Callback function to remap individual type ID references */
> +static int btf_permute_remap_type_id(__u32 *type_id, void *ctx)
> +{
> + struct btf_permute *p = ctx;
> + __u32 new_type_id = *type_id;
> +
> + /* refer to the base BTF or VOID type */
> + if (new_type_id < p->btf->start_id)
> + return 0;
> +
> + if (new_type_id >= btf__type_cnt(p->btf))
> + return -EINVAL;
> +
> + *type_id = p->id_map[new_type_id - p->btf->start_id];
> + return 0;
> +}
> +
> +int btf__permute(struct btf *btf, __u32 *id_map, __u32 id_map_cnt,
> + const struct btf_permute_opts *opts)
> +{
> + struct btf_permute p;
> + struct btf_ext *btf_ext;
> + void *nt, *new_types = NULL;
> + __u32 *order_map = NULL;
> + int err = 0, i;
> + __u32 id;
> +
> + if (!OPTS_VALID(opts, btf_permute_opts) || id_map_cnt != btf->nr_types)
> + return libbpf_err(-EINVAL);
> +
> + /* record the sequence of types */
> + order_map = calloc(id_map_cnt, sizeof(*id_map));
> + if (!order_map) {
> + err = -ENOMEM;
> + goto done;
> + }
> +
> + new_types = calloc(btf->hdr->type_len, 1);
> + if (!new_types) {
> + err = -ENOMEM;
> + goto done;
> + }
> +
> + if (btf_ensure_modifiable(btf)) {
> + err = -ENOMEM;
> + goto done;
> + }
> +
> + for (i = 0; i < id_map_cnt; i++) {
> + id = id_map[i];
> + if (id < btf->start_id || id >= btf__type_cnt(btf)) {
> + err = -EINVAL;
> + goto done;
> + }
> + id -= btf->start_id;
> + /* cannot be mapped to the same ID */
> + if (order_map[id]) {
> + err = -EINVAL;
> + goto done;
> + }
> + order_map[id] = i + btf->start_id;
> + }
> +
> + p.btf = btf;
> + p.id_map = id_map;
> + nt = new_types;
> + for (i = 0; i < id_map_cnt; i++) {
> + struct btf_field_iter it;
> + const struct btf_type *t;
> + __u32 *type_id;
> + int type_size;
> +
> + id = order_map[i];
> + t = btf__type_by_id(btf, id);
> + type_size = btf_type_size(t);
> + memcpy(nt, t, type_size);
> +
> + /* fix up referenced IDs for BTF */
> + err = btf_field_iter_init(&it, nt, BTF_FIELD_ITER_IDS);
> + if (err)
> + goto done;
> + while ((type_id = btf_field_iter_next(&it))) {
> + err = btf_permute_remap_type_id(type_id, &p);
> + if (err)
> + goto done;
> + }
> +
> + nt += type_size;
> + }
> +
> + /* fix up referenced IDs for btf_ext */
> + btf_ext = OPTS_GET(opts, btf_ext, NULL);
> + if (btf_ext) {
> + err = btf_ext_visit_type_ids(btf_ext, btf_permute_remap_type_id, &p);
> + if (err)
> + goto done;
> + }
> +
> + for (nt = new_types, i = 0; i < id_map_cnt; i++) {
> + btf->type_offs[i] = nt - new_types;
> + nt += btf_type_size(nt);
> + }
> +
> + free(order_map);
> + free(btf->types_data);
> + btf->types_data = new_types;
> + return 0;
> +
> +done:
> + free(order_map);
> + free(new_types);
> + return libbpf_err(err);
> +}
> diff --git a/tools/lib/bpf/btf.h b/tools/lib/bpf/btf.h
> index cc01494d6210..5d560571b1b5 100644
> --- a/tools/lib/bpf/btf.h
> +++ b/tools/lib/bpf/btf.h
> @@ -281,6 +281,42 @@ LIBBPF_API int btf__dedup(struct btf *btf, const struct btf_dedup_opts *opts);
> */
> LIBBPF_API int btf__relocate(struct btf *btf, const struct btf *base_btf);
>
> +struct btf_permute_opts {
> + size_t sz;
> + /* optional .BTF.ext info along the main BTF info */
> + struct btf_ext *btf_ext;
> + size_t :0;
> +};
> +#define btf_permute_opts__last_field btf_ext
> +
> +/**
> + * @brief **btf__permute()** performs in-place BTF type rearrangement
> + * @param btf BTF object to permute
> + * @param id_map Array mapping original type IDs to new IDs
> + * @param id_map_cnt Number of elements in @id_map
> + * @param opts Optional parameters for BTF extension updates
> + * @return 0 on success, negative error code on failure
> + *
> + * **btf__permute()** rearranges BTF types according to the specified ID mapping.
> + * The @id_map array defines the new type ID for each original type ID.
> + *
> + * @id_map must include all types from ID `start_id` to `btf__type_cnt(btf) - 1`.
> + * @id_map_cnt should be `btf__type_cnt(btf) - start_id`
> + * The mapping is defined as: `id_map[original_id - start_id] = new_id`
Would you mind paying attention to the feedback I left in [0]? Thank you.
The contract should be id_map[original_id] = new_id for base BTF and
id_map[original_id - btf__type_cnt(base_btf)] = new_id for split BTF.
Special BTF type #0 (VOID) is considered to be part of base BTF,
having id_map[0] = 0 is easy to check and enforce. And then it leaves
us with a simple and logical rule for id_map. For split BTF we make
necessary type ID shifts to avoid tons of wasted memory. But for base
BTF there is no need to shift anything. So mapping the original type
#X to #Y is id_map[X] = Y. Literally, "map X to Y", as simple as that.
[0] https://lore.kernel.org/bpf/CAEf4BzY_k721TBfRSUeq5mB-7fgJhVKCeXVKO-W2EjQ0aS9AgA@mail.gmail.com/
> + *
> + * For base BTF, its `start_id` is fixed to 1, i.e. the VOID type can
> + * not be redefined or remapped and its ID is fixed to 0.
> + *
> + * For split BTF, its `start_id` can be retrieved by calling
> + * `btf__type_cnt(btf__base_btf(btf))`.
> + *
> + * On error, returns negative error code and sets errno:
> + * - `-EINVAL`: Invalid parameters or ID mapping (duplicates, out-of-range)
> + * - `-ENOMEM`: Memory allocation failure
> + */
> +LIBBPF_API int btf__permute(struct btf *btf, __u32 *id_map, __u32 id_map_cnt,
> + const struct btf_permute_opts *opts);
> +
> struct btf_dump;
>
> struct btf_dump_opts {
> diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map
> index 84fb90a016c9..d18fbcea7578 100644
> --- a/tools/lib/bpf/libbpf.map
> +++ b/tools/lib/bpf/libbpf.map
> @@ -453,4 +453,5 @@ LIBBPF_1.7.0 {
> bpf_map__exclusive_program;
> bpf_prog_assoc_struct_ops;
> bpf_program__assoc_struct_ops;
> + btf__permute;
> } LIBBPF_1.6.0;
> --
> 2.34.1
>
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 02/13] selftests/bpf: Add test cases for btf__permute functionality
2025-12-18 11:30 ` [PATCH bpf-next v10 02/13] selftests/bpf: Add test cases for btf__permute functionality Donglin Peng
@ 2025-12-18 23:03 ` Andrii Nakryiko
2025-12-19 3:30 ` Donglin Peng
0 siblings, 1 reply; 72+ messages in thread
From: Andrii Nakryiko @ 2025-12-18 23:03 UTC (permalink / raw)
To: Donglin Peng
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
>
> From: pengdonglin <pengdonglin@xiaomi.com>
>
> This patch introduces test cases for the btf__permute function to ensure
> it works correctly with both base BTF and split BTF scenarios.
>
> The test suite includes:
> - test_permute_base: Validates permutation on base BTF
> - test_permute_split: Tests permutation on split BTF
>
> Cc: Eduard Zingerman <eddyz87@gmail.com>
> Cc: Alexei Starovoitov <ast@kernel.org>
> Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> Cc: Alan Maguire <alan.maguire@oracle.com>
> Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> Acked-by: Eduard Zingerman <eddyz87@gmail.com>
> ---
> .../selftests/bpf/prog_tests/btf_permute.c | 228 ++++++++++++++++++
> 1 file changed, 228 insertions(+)
> create mode 100644 tools/testing/selftests/bpf/prog_tests/btf_permute.c
>
> diff --git a/tools/testing/selftests/bpf/prog_tests/btf_permute.c b/tools/testing/selftests/bpf/prog_tests/btf_permute.c
> new file mode 100644
> index 000000000000..9aa71cdf984a
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/prog_tests/btf_permute.c
> @@ -0,0 +1,228 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/* Copyright (c) 2025 Xiaomi */
> +
> +#include <test_progs.h>
> +#include <bpf/btf.h>
> +#include "btf_helpers.h"
> +
> +static void permute_base_check(struct btf *btf)
> +{
> + VALIDATE_RAW_BTF(
> + btf,
> + "[1] STRUCT 's2' size=4 vlen=1\n"
> + "\t'm' type_id=4 bits_offset=0",
> + "[2] FUNC 'f' type_id=6 linkage=static",
> + "[3] PTR '(anon)' type_id=4",
> + "[4] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED",
> + "[5] STRUCT 's1' size=4 vlen=1\n"
> + "\t'm' type_id=4 bits_offset=0",
> + "[6] FUNC_PROTO '(anon)' ret_type_id=4 vlen=1\n"
> + "\t'p' type_id=3");
> +}
> +
> +/* Ensure btf__permute work as expected with base BTF */
> +static void test_permute_base(void)
> +{
> + struct btf *btf;
> + __u32 permute_ids[6];
> + int start_id = 1;
> + int err;
> +
> + btf = btf__new_empty();
> + if (!ASSERT_OK_PTR(btf, "empty_main_btf"))
> + return;
> +
> + btf__add_int(btf, "int", 4, BTF_INT_SIGNED); /* [1] int */
> + btf__add_ptr(btf, 1); /* [2] ptr to int */
> + btf__add_struct(btf, "s1", 4); /* [3] struct s1 { */
> + btf__add_field(btf, "m", 1, 0, 0); /* int m; */
> + /* } */
> + btf__add_struct(btf, "s2", 4); /* [4] struct s2 { */
> + btf__add_field(btf, "m", 1, 0, 0); /* int m; */
> + /* } */
> + btf__add_func_proto(btf, 1); /* [5] int (*)(int *p); */
> + btf__add_func_param(btf, "p", 2);
> + btf__add_func(btf, "f", BTF_FUNC_STATIC, 5); /* [6] int f(int *p); */
> +
> + VALIDATE_RAW_BTF(
> + btf,
> + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED",
> + "[2] PTR '(anon)' type_id=1",
> + "[3] STRUCT 's1' size=4 vlen=1\n"
> + "\t'm' type_id=1 bits_offset=0",
> + "[4] STRUCT 's2' size=4 vlen=1\n"
> + "\t'm' type_id=1 bits_offset=0",
> + "[5] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n"
> + "\t'p' type_id=2",
> + "[6] FUNC 'f' type_id=5 linkage=static");
> +
> + permute_ids[1 - start_id] = 4; /* [1] -> [4] */
> + permute_ids[2 - start_id] = 3; /* [2] -> [3] */
> + permute_ids[3 - start_id] = 5; /* [3] -> [5] */
> + permute_ids[4 - start_id] = 1; /* [4] -> [1] */
> + permute_ids[5 - start_id] = 6; /* [5] -> [6] */
> + permute_ids[6 - start_id] = 2; /* [6] -> [2] */
> + err = btf__permute(btf, permute_ids, ARRAY_SIZE(permute_ids), NULL);
> + if (!ASSERT_OK(err, "btf__permute_base"))
> + goto done;
> + permute_base_check(btf);
> +
> + /* id_map_cnt is invalid */
> + permute_ids[1 - start_id] = 4; /* [1] -> [4] */
> + permute_ids[2 - start_id] = 3; /* [2] -> [3] */
> + permute_ids[3 - start_id] = 5; /* [3] -> [5] */
> + permute_ids[4 - start_id] = 1; /* [4] -> [1] */
> + permute_ids[5 - start_id] = 6; /* [5] -> [6] */
> + permute_ids[6 - start_id] = 2; /* [6] -> [2] */
> + err = btf__permute(btf, permute_ids, ARRAY_SIZE(permute_ids) - 1, NULL);
> + if (!ASSERT_ERR(err, "btf__permute_base"))
> + goto done;
> + /* BTF is not modified */
> + permute_base_check(btf);
> +
> + /* Multiple types can not be mapped to the same ID */
> + permute_ids[1 - start_id] = 4;
> + permute_ids[2 - start_id] = 4;
> + permute_ids[3 - start_id] = 5;
> + permute_ids[4 - start_id] = 1;
> + permute_ids[5 - start_id] = 6;
> + permute_ids[6 - start_id] = 2;
> + err = btf__permute(btf, permute_ids, ARRAY_SIZE(permute_ids), NULL);
> + if (!ASSERT_ERR(err, "btf__permute_base"))
> + goto done;
> + /* BTF is not modified */
> + permute_base_check(btf);
> +
> + /* Type ID must be valid */
> + permute_ids[1 - start_id] = 4;
> + permute_ids[2 - start_id] = 3;
> + permute_ids[3 - start_id] = 5;
> + permute_ids[4 - start_id] = 1;
> + permute_ids[5 - start_id] = 7;
> + permute_ids[6 - start_id] = 2;
please make sure this base BTF test doesn't use start_id, see comment
on previous patch
> + err = btf__permute(btf, permute_ids, ARRAY_SIZE(permute_ids), NULL);
> + if (!ASSERT_ERR(err, "btf__permute_base"))
> + goto done;
> + /* BTF is not modified */
> + permute_base_check(btf);
> +
> +done:
> + btf__free(btf);
> +}
> +
> +static void permute_split_check(struct btf *btf)
> +{
> + VALIDATE_RAW_BTF(
> + btf,
> + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED",
> + "[2] PTR '(anon)' type_id=1",
> + "[3] STRUCT 's2' size=4 vlen=1\n"
> + "\t'm' type_id=1 bits_offset=0",
> + "[4] FUNC 'f' type_id=5 linkage=static",
> + "[5] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n"
> + "\t'p' type_id=2",
> + "[6] STRUCT 's1' size=4 vlen=1\n"
> + "\t'm' type_id=1 bits_offset=0");
> +}
> +
> +/* Ensure btf__permute work as expected with split BTF */
> +static void test_permute_split(void)
> +{
> + struct btf *split_btf = NULL, *base_btf = NULL;
> + __u32 permute_ids[4];
> + int err;
> + int start_id;
> +
> + base_btf = btf__new_empty();
> + if (!ASSERT_OK_PTR(base_btf, "empty_main_btf"))
> + return;
> +
> + btf__add_int(base_btf, "int", 4, BTF_INT_SIGNED); /* [1] int */
> + btf__add_ptr(base_btf, 1); /* [2] ptr to int */
> + VALIDATE_RAW_BTF(
> + base_btf,
> + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED",
> + "[2] PTR '(anon)' type_id=1");
> + split_btf = btf__new_empty_split(base_btf);
> + if (!ASSERT_OK_PTR(split_btf, "empty_split_btf"))
> + goto cleanup;
> + btf__add_struct(split_btf, "s1", 4); /* [3] struct s1 { */
> + btf__add_field(split_btf, "m", 1, 0, 0); /* int m; */
> + /* } */
> + btf__add_struct(split_btf, "s2", 4); /* [4] struct s2 { */
> + btf__add_field(split_btf, "m", 1, 0, 0); /* int m; */
> + /* } */
> + btf__add_func_proto(split_btf, 1); /* [5] int (*)(int p); */
> + btf__add_func_param(split_btf, "p", 2);
> + btf__add_func(split_btf, "f", BTF_FUNC_STATIC, 5); /* [6] int f(int *p); */
> +
> + VALIDATE_RAW_BTF(
> + split_btf,
> + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED",
> + "[2] PTR '(anon)' type_id=1",
> + "[3] STRUCT 's1' size=4 vlen=1\n"
> + "\t'm' type_id=1 bits_offset=0",
> + "[4] STRUCT 's2' size=4 vlen=1\n"
> + "\t'm' type_id=1 bits_offset=0",
> + "[5] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n"
> + "\t'p' type_id=2",
> + "[6] FUNC 'f' type_id=5 linkage=static");
> +
> + start_id = btf__type_cnt(base_btf);
> + permute_ids[3 - start_id] = 6; /* [3] -> [6] */
> + permute_ids[4 - start_id] = 3; /* [4] -> [3] */
> + permute_ids[5 - start_id] = 5; /* [5] -> [5] */
> + permute_ids[6 - start_id] = 4; /* [6] -> [4] */
> + err = btf__permute(split_btf, permute_ids, ARRAY_SIZE(permute_ids), NULL);
> + if (!ASSERT_OK(err, "btf__permute_split"))
> + goto cleanup;
> + permute_split_check(split_btf);
> +
> + /*
> + * For split BTF, id_map_cnt must equal to the number of types
> + * added on top of base BTF
> + */
> + permute_ids[3 - start_id] = 4;
> + permute_ids[4 - start_id] = 3;
> + permute_ids[5 - start_id] = 5;
> + permute_ids[6 - start_id] = 6;
> + err = btf__permute(split_btf, permute_ids, 3, NULL);
> + if (!ASSERT_ERR(err, "btf__permute_split"))
> + goto cleanup;
> + /* BTF is not modified */
> + permute_split_check(split_btf);
> +
> + /* Multiple types can not be mapped to the same ID */
> + permute_ids[3 - start_id] = 4;
> + permute_ids[4 - start_id] = 3;
> + permute_ids[5 - start_id] = 3;
> + permute_ids[6 - start_id] = 6;
> + err = btf__permute(split_btf, permute_ids, 4, NULL);
> + if (!ASSERT_ERR(err, "btf__permute_split"))
> + goto cleanup;
> + /* BTF is not modified */
> + permute_split_check(split_btf);
> +
> + /* Can not map to base ID */
> + permute_ids[3 - start_id] = 4;
> + permute_ids[4 - start_id] = 2;
> + permute_ids[5 - start_id] = 5;
> + permute_ids[6 - start_id] = 6;
> + err = btf__permute(split_btf, permute_ids, 4, NULL);
> + if (!ASSERT_ERR(err, "btf__permute_split"))
> + goto cleanup;
> + /* BTF is not modified */
> + permute_split_check(split_btf);
> +
> +cleanup:
> + btf__free(split_btf);
> + btf__free(base_btf);
> +}
> +
> +void test_btf_permute(void)
> +{
> + if (test__start_subtest("permute_base"))
> + test_permute_base();
> + if (test__start_subtest("permute_split"))
> + test_permute_split();
> +}
> --
> 2.34.1
>
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 03/13] tools/resolve_btfids: Support BTF sorting feature
2025-12-18 11:30 ` [PATCH bpf-next v10 03/13] tools/resolve_btfids: Support BTF sorting feature Donglin Peng
@ 2025-12-18 23:09 ` Andrii Nakryiko
2025-12-19 3:42 ` Donglin Peng
0 siblings, 1 reply; 72+ messages in thread
From: Andrii Nakryiko @ 2025-12-18 23:09 UTC (permalink / raw)
To: Donglin Peng
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
>
> From: pengdonglin <pengdonglin@xiaomi.com>
>
> This introduces a new BTF sorting phase that specifically sorts
> BTF types by name in ascending order, so that the binary search
> can be used to look up types.
>
> Cc: Eduard Zingerman <eddyz87@gmail.com>
> Cc: Alexei Starovoitov <ast@kernel.org>
> Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> Cc: Alan Maguire <alan.maguire@oracle.com>
> Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
Signed-off-by is supposed to use properly spelled full name, this
should be "Donglin Peng", right?
> Acked-by: Eduard Zingerman <eddyz87@gmail.com>
> ---
> tools/bpf/resolve_btfids/main.c | 68 +++++++++++++++++++++++++++++++++
> 1 file changed, 68 insertions(+)
>
> diff --git a/tools/bpf/resolve_btfids/main.c b/tools/bpf/resolve_btfids/main.c
> index 3e88dc862d87..659de35748ec 100644
> --- a/tools/bpf/resolve_btfids/main.c
> +++ b/tools/bpf/resolve_btfids/main.c
> @@ -848,6 +848,71 @@ static int dump_raw_btf(struct btf *btf, const char *out_path)
> return 0;
> }
>
> +/*
> + * Sort types by name in ascending order resulting in all
> + * anonymous types being placed before named types.
> + */
> +static int cmp_type_names(const void *a, const void *b, void *priv)
> +{
> + struct btf *btf = (struct btf *)priv;
> + const struct btf_type *ta = btf__type_by_id(btf, *(__u32 *)a);
> + const struct btf_type *tb = btf__type_by_id(btf, *(__u32 *)b);
> + const char *na, *nb;
> +
> + na = btf__str_by_offset(btf, ta->name_off);
> + nb = btf__str_by_offset(btf, tb->name_off);
> + return strcmp(na, nb);
> +}
> +
> +static int sort_btf_by_name(struct btf *btf)
> +{
> + __u32 *permute_ids = NULL, *id_map = NULL;
> + int nr_types, i, err = 0;
> + __u32 start_id = 1, id;
> +
> + if (btf__base_btf(btf))
> + start_id = btf__type_cnt(btf__base_btf(btf));
> + nr_types = btf__type_cnt(btf) - start_id;
> + if (nr_types < 2)
> + goto out;
why this check, will anything break if you don't do it?
> +
> + permute_ids = calloc(nr_types, sizeof(*permute_ids));
> + if (!permute_ids) {
> + err = -ENOMEM;
> + goto out;
> + }
> +
> + id_map = calloc(nr_types, sizeof(*id_map));
> + if (!id_map) {
> + err = -ENOMEM;
> + goto out;
> + }
> +
> + for (i = 0, id = start_id; i < nr_types; i++, id++)
> + permute_ids[i] = id;
> +
> + qsort_r(permute_ids, nr_types, sizeof(*permute_ids), cmp_type_names, btf);
> +
> + for (i = 0; i < nr_types; i++) {
> + id = permute_ids[i] - start_id;
> + id_map[id] = i + start_id;
> + }
> +
> + err = btf__permute(btf, id_map, nr_types, NULL);
> + if (err)
> + pr_err("FAILED: btf permute: %s\n", strerror(-err));
> +
> +out:
> + free(permute_ids);
> + free(id_map);
> + return err;
> +}
> +
> +static int btf2btf(struct object *obj)
what's the point of having this function?
> +{
> + return sort_btf_by_name(obj->btf);
> +}
> +
> static inline int make_out_path(char *buf, u32 buf_sz, const char *in_path, const char *suffix)
> {
> int len = snprintf(buf, buf_sz, "%s%s", in_path, suffix);
> @@ -906,6 +971,9 @@ int main(int argc, const char **argv)
> if (load_btf(&obj))
> goto out;
>
> + if (btf2btf(&obj))
> + goto out;
> +
> if (elf_collect(&obj))
> goto out;
>
> --
> 2.34.1
>
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 04/13] libbpf: Optimize type lookup with binary search for sorted BTF
2025-12-18 11:30 ` [PATCH bpf-next v10 04/13] libbpf: Optimize type lookup with binary search for sorted BTF Donglin Peng
2025-12-18 19:32 ` Eduard Zingerman
@ 2025-12-18 23:29 ` Andrii Nakryiko
2025-12-19 0:13 ` Eduard Zingerman
2025-12-19 2:53 ` Donglin Peng
1 sibling, 2 replies; 72+ messages in thread
From: Andrii Nakryiko @ 2025-12-18 23:29 UTC (permalink / raw)
To: Donglin Peng
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
>
> From: pengdonglin <pengdonglin@xiaomi.com>
>
> This patch introduces binary search optimization for BTF type lookups
> when the BTF instance contains sorted types.
>
> The optimization significantly improves performance when searching for
> types in large BTF instances with sorted types. For unsorted BTF, the
> implementation falls back to the original linear search.
>
> Cc: Eduard Zingerman <eddyz87@gmail.com>
> Cc: Alexei Starovoitov <ast@kernel.org>
> Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> Cc: Alan Maguire <alan.maguire@oracle.com>
> Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> ---
> tools/lib/bpf/btf.c | 103 ++++++++++++++++++++++++++++++++++----------
> 1 file changed, 80 insertions(+), 23 deletions(-)
>
[...]
> + l = start_id;
> + r = end_id;
> + while (l <= r) {
> + m = l + (r - l) / 2;
> + t = btf_type_by_id(btf, m);
> + tname = btf__str_by_offset(btf, t->name_off);
> + ret = strcmp(tname, name);
> + if (ret < 0) {
> + l = m + 1;
> + } else {
> + if (ret == 0)
> + lmost = m;
> + r = m - 1;
> + }
> }
this differs from what we discussed in [0], you said you'll use that
approach. Can you please elaborate on why you didn't?
[0] https://lore.kernel.org/bpf/CAEf4Bzb3Eu0J83O=Y4KA-LkzBMjtx7cbonxPzkiduzZ1Pedajg@mail.gmail.com/
>
> - return libbpf_err(-ENOENT);
> + return lmost;
> }
>
> static __s32 btf_find_by_name_kind(const struct btf *btf, int start_id,
> const char *type_name, __u32 kind)
kind is defined as u32 but you expect caller to pass -1 to ignore the
kind. Use int here.
> {
> - __u32 i, nr_types = btf__type_cnt(btf);
> + const struct btf_type *t;
> + const char *tname;
> + __s32 idx;
> +
> + if (start_id < btf->start_id) {
> + idx = btf_find_by_name_kind(btf->base_btf, start_id,
> + type_name, kind);
> + if (idx >= 0)
> + return idx;
> + start_id = btf->start_id;
> + }
>
> - if (kind == BTF_KIND_UNKN || !strcmp(type_name, "void"))
> + if (kind == BTF_KIND_UNKN || strcmp(type_name, "void") == 0)
> return 0;
>
> - for (i = start_id; i < nr_types; i++) {
> - const struct btf_type *t = btf__type_by_id(btf, i);
> - const char *name;
> + if (btf->sorted_start_id > 0 && type_name[0]) {
> + __s32 end_id = btf__type_cnt(btf) - 1;
> +
> + /* skip anonymous types */
> + start_id = max(start_id, btf->sorted_start_id);
can sorted_start_id ever be smaller than start_id?
> + idx = btf_find_by_name_bsearch(btf, type_name, start_id, end_id);
is there ever a time when btf_find_by_name_bsearch() will work with
different start_id and end_id? why is this not done inside the
btf_find_by_name_bsearch()?
> + if (unlikely(idx < 0))
> + return libbpf_err(-ENOENT);
pass through error returned from btf_find_by_name_bsearch(), why redefining it?
> +
> + if (unlikely(kind == -1))
> + return idx;
> +
> + t = btf_type_by_id(btf, idx);
> + if (likely(BTF_INFO_KIND(t->info) == kind))
use btf_kind(), but this whole extra check is just unnecessary, this
should be done in the loop below. We talked about all this already,
why do I feel like I'm being ignored?..
> + return idx;
drop all these likely and unlikely micro optimizations, please
> +
> + for (idx++; idx <= end_id; idx++) {
> + t = btf__type_by_id(btf, idx);
> + tname = btf__str_by_offset(btf, t->name_off);
> + if (strcmp(tname, type_name) != 0)
> + return libbpf_err(-ENOENT);
> + if (btf_kind(t) == kind)
> + return idx;
> + }
> + } else {
> + __u32 i, total;
>
> - if (btf_kind(t) != kind)
> - continue;
> - name = btf__name_by_offset(btf, t->name_off);
> - if (name && !strcmp(type_name, name))
> - return i;
> + total = btf__type_cnt(btf);
> + for (i = start_id; i < total; i++) {
> + t = btf_type_by_id(btf, i);
> + if (kind != -1 && btf_kind(t) != kind)
nit: kind < 0, no need to hard-code -1
> + continue;
> + tname = btf__str_by_offset(btf, t->name_off);
> + if (strcmp(tname, type_name) == 0)
> + return i;
> + }
> }
>
> return libbpf_err(-ENOENT);
> }
>
[...]
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 05/13] libbpf: Verify BTF Sorting
2025-12-18 11:30 ` [PATCH bpf-next v10 05/13] libbpf: Verify BTF Sorting Donglin Peng
2025-12-18 19:38 ` Eduard Zingerman
@ 2025-12-18 23:44 ` Andrii Nakryiko
2025-12-19 5:06 ` Donglin Peng
1 sibling, 1 reply; 72+ messages in thread
From: Andrii Nakryiko @ 2025-12-18 23:44 UTC (permalink / raw)
To: Donglin Peng
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
>
> From: pengdonglin <pengdonglin@xiaomi.com>
>
typo in subject: "Sorting" -> "sorting", it looks weird capitalized like that
> This patch checks whether the BTF is sorted by name in ascending
> order. If sorted, binary search will be used when looking up types.
>
> Cc: Eduard Zingerman <eddyz87@gmail.com>
> Cc: Alexei Starovoitov <ast@kernel.org>
> Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> Cc: Alan Maguire <alan.maguire@oracle.com>
> Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> ---
> tools/lib/bpf/btf.c | 41 +++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 41 insertions(+)
>
> diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> index 2facb57d7e5f..c63d46b7d74b 100644
> --- a/tools/lib/bpf/btf.c
> +++ b/tools/lib/bpf/btf.c
> @@ -899,6 +899,46 @@ int btf__resolve_type(const struct btf *btf, __u32 type_id)
> return type_id;
> }
>
> +/*
> + * Assuming that types are sorted by name in ascending order.
> + */
Unnecessary comment, and no, btf_compare_type_names() itself makes no
such assumption, it just compares two provided types by name. Drop the
comment, please.
> +static int btf_compare_type_names(__u32 *a, __u32 *b, const struct btf *btf)
> +{
> + struct btf_type *ta = btf_type_by_id(btf, *a);
> + struct btf_type *tb = btf_type_by_id(btf, *b);
> + const char *na, *nb;
> +
> + na = btf__str_by_offset(btf, ta->name_off);
> + nb = btf__str_by_offset(btf, tb->name_off);
> + return strcmp(na, nb);
> +}
you use this function only in one place, there is no real point having
it, especially that it uses **a pointer to type ID** as an
interface... just inline its logic in that one loop below
> +
> +static void btf_check_sorted(struct btf *btf)
> +{
> + const struct btf_type *t;
> + __u32 i, k, n;
> + __u32 sorted_start_id;
> +
> + if (btf->nr_types < 2)
> + return;
why special casing? does it not work with nr_types = 0 or nr_types = 1?
> +
> + sorted_start_id = 0;
nit: initialize in declaration
> + n = btf__type_cnt(btf);
> + for (i = btf->start_id; i < n; i++) {
> + k = i + 1;
> + if (k < n && btf_compare_type_names(&i, &k, btf) > 0)
> + return;
> + if (sorted_start_id == 0) {
> + t = btf_type_by_id(btf, i);
> + if (t->name_off)
I'd check actual string, not name_off. Technically, you can have empty
string with non-zero name_off, so why assume anything here?
> + sorted_start_id = i;
> + }
> + }
> +
> + if (sorted_start_id)
> + btf->sorted_start_id = sorted_start_id;
You actually made code more complicated by extracting that
btf_compare_type_names(). Compare to:
n = btf__type_cnt(btf);
btf->sorted_start_id = 0;
for (i = btf->start_id + 1; i < n; i++) {
struct btf_type *t1 = btf_type_by_id(btf, i - 1);
struct btf_type *t2 = btf_type_by_id(btf, i);
const char *n1 = btf__str_by_offset(btf, t1->name_off);
const char *n2 = btf__str_by_offset(btf, t2->name_off);
if (strcmp(n1, n2) > 0)
return;
if (btf->sorted_start_id == 0 && n1[0] != '\0')
btf->sorted_start_id = i - 1;
}
No extra k<n checks, no extra type_by_id lookups. It's minimalistic
and cleaner. And if it so happens that we get single type BTF that is
technically sorted, it doesn't matter, we always fallback to faster
linear search anyways.
Keep it simple.
> +}
> +
> static __s32 btf_find_by_name_bsearch(const struct btf *btf, const char *name,
> __s32 start_id, __s32 end_id)
> {
> @@ -1147,6 +1187,7 @@ static struct btf *btf_new(const void *data, __u32 size, struct btf *base_btf, b
> err = err ?: btf_sanity_check(btf);
> if (err)
> goto done;
> + btf_check_sorted(btf);
>
> done:
> if (err) {
> --
> 2.34.1
>
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 07/13] btf: Verify BTF Sorting
2025-12-18 11:30 ` [PATCH bpf-next v10 07/13] btf: Verify BTF Sorting Donglin Peng
2025-12-18 21:43 ` Eduard Zingerman
@ 2025-12-18 23:46 ` Andrii Nakryiko
2025-12-19 5:10 ` Donglin Peng
1 sibling, 1 reply; 72+ messages in thread
From: Andrii Nakryiko @ 2025-12-18 23:46 UTC (permalink / raw)
To: Donglin Peng
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
>
> From: pengdonglin <pengdonglin@xiaomi.com>
>
> This patch checks whether the BTF is sorted by name in ascending order.
> If sorted, binary search will be used when looking up types.
>
> Specifically, vmlinux and kernel module BTFs are always sorted during
> the build phase with anonymous types placed before named types, so we
> only need to identify the starting ID of named types.
>
> Cc: Eduard Zingerman <eddyz87@gmail.com>
> Cc: Alexei Starovoitov <ast@kernel.org>
> Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> Cc: Alan Maguire <alan.maguire@oracle.com>
> Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> ---
> kernel/bpf/btf.c | 56 ++++++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 56 insertions(+)
>
please make sure to apply feedback received for libbpf-side
implementation for kernel-side implementations as well
> diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
> index 0394f0c8ef74..a9e2345558c0 100644
> --- a/kernel/bpf/btf.c
> +++ b/kernel/bpf/btf.c
> @@ -550,6 +550,60 @@ u32 btf_nr_types(const struct btf *btf)
> return total;
> }
>
[...]
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 08/13] bpf: Skip anonymous types in type lookup for performance
2025-12-18 22:21 ` Eduard Zingerman
@ 2025-12-18 23:59 ` Andrii Nakryiko
2025-12-19 5:40 ` Donglin Peng
0 siblings, 1 reply; 72+ messages in thread
From: Andrii Nakryiko @ 2025-12-18 23:59 UTC (permalink / raw)
To: Eduard Zingerman
Cc: Donglin Peng, ast, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Thu, Dec 18, 2025 at 2:21 PM Eduard Zingerman <eddyz87@gmail.com> wrote:
>
> On Thu, 2025-12-18 at 19:30 +0800, Donglin Peng wrote:
> > From: pengdonglin <pengdonglin@xiaomi.com>
> >
> > Currently, vmlinux and kernel module BTFs are unconditionally
> > sorted during the build phase, with named types placed at the
> > end. Thus, anonymous types should be skipped when starting the
> > search. In my vmlinux BTF, the number of anonymous types is
> > 61,747, which means the loop count can be reduced by 61,747.
> >
> > Cc: Eduard Zingerman <eddyz87@gmail.com>
> > Cc: Alexei Starovoitov <ast@kernel.org>
> > Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > Cc: Alan Maguire <alan.maguire@oracle.com>
> > Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> > Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> > Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> > ---
>
> Acked-by: Eduard Zingerman <eddyz87@gmail.com>
>
> > include/linux/btf.h | 1 +
> > kernel/bpf/btf.c | 24 ++++++++++++++++++++----
> > kernel/bpf/verifier.c | 7 +------
> > 3 files changed, 22 insertions(+), 10 deletions(-)
> >
> > diff --git a/include/linux/btf.h b/include/linux/btf.h
> > index f06976ffb63f..2d28f2b22ae5 100644
> > --- a/include/linux/btf.h
> > +++ b/include/linux/btf.h
> > @@ -220,6 +220,7 @@ bool btf_is_module(const struct btf *btf);
> > bool btf_is_vmlinux(const struct btf *btf);
> > struct module *btf_try_get_module(const struct btf *btf);
> > u32 btf_nr_types(const struct btf *btf);
> > +u32 btf_sorted_start_id(const struct btf *btf);
> > struct btf *btf_base_btf(const struct btf *btf);
> > bool btf_type_is_i32(const struct btf_type *t);
> > bool btf_type_is_i64(const struct btf_type *t);
> > diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
> > index a9e2345558c0..3aeb4f00cbfe 100644
> > --- a/kernel/bpf/btf.c
> > +++ b/kernel/bpf/btf.c
> > @@ -550,6 +550,11 @@ u32 btf_nr_types(const struct btf *btf)
> > return total;
> > }
> >
> > +u32 btf_sorted_start_id(const struct btf *btf)
>
> Nit: the name is a bit confusing, given that it not always returns the
> start id for sorted part. btf_maybe_first_named_id?
> Can't figure out a good name :(
yeah, I agree, it is quite confusing overall. I think we should at
least add comments why we start with something different than 1 in
those few places where we use this optimization...
let's name it btf_named_start_id() and specify in the comment that for
non-sorted BTFs we conservatively fallback to the first type.
btw, maybe it would be good to have two versions of this (or bool
flag,but we all hate bool flags) to either return own start id (i.e.,
ignoring base BTF) or recursively go down to the base BTF.
Having that
while (base_btf->base_btf)
base_btf = base_btf->base_btf;
logic in a few places looks a bit too low-level and distracting, IMO.
>
> > +{
> > + return btf->sorted_start_id ?: (btf->start_id ?: 1);
> > +}
> > +
> > /*
> > * Assuming that types are sorted by name in ascending order.
> > */
>
> [...]
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 09/13] bpf: Optimize the performance of find_bpffs_btf_enums
2025-12-18 11:30 ` [PATCH bpf-next v10 09/13] bpf: Optimize the performance of find_bpffs_btf_enums Donglin Peng
@ 2025-12-19 0:01 ` Andrii Nakryiko
2025-12-19 5:41 ` Donglin Peng
0 siblings, 1 reply; 72+ messages in thread
From: Andrii Nakryiko @ 2025-12-19 0:01 UTC (permalink / raw)
To: Donglin Peng
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
>
> From: pengdonglin <pengdonglin@xiaomi.com>
>
> Currently, vmlinux BTF is unconditionally sorted during
> the build phase. The function btf_find_by_name_kind
> executes the binary search branch, so find_bpffs_btf_enums
> can be optimized by using btf_find_by_name_kind.
>
> Cc: Eduard Zingerman <eddyz87@gmail.com>
> Cc: Alexei Starovoitov <ast@kernel.org>
> Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> Cc: Alan Maguire <alan.maguire@oracle.com>
> Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> Acked-by: Eduard Zingerman <eddyz87@gmail.com>
> ---
> kernel/bpf/inode.c | 42 +++++++++++++++++++-----------------------
> 1 file changed, 19 insertions(+), 23 deletions(-)
>
> diff --git a/kernel/bpf/inode.c b/kernel/bpf/inode.c
> index 9f866a010dad..050fde1cf211 100644
> --- a/kernel/bpf/inode.c
> +++ b/kernel/bpf/inode.c
> @@ -600,10 +600,18 @@ struct bpffs_btf_enums {
>
> static int find_bpffs_btf_enums(struct bpffs_btf_enums *info)
> {
> + struct {
> + const struct btf_type **type;
> + const char *name;
> + } btf_enums[] = {
> + {&info->cmd_t, "bpf_cmd"},
> + {&info->map_t, "bpf_map_type"},
> + {&info->prog_t, "bpf_prog_type"},
> + {&info->attach_t, "bpf_attach_type"},
> + };
> const struct btf *btf;
> const struct btf_type *t;
> - const char *name;
> - int i, n;
> + int i, id;
>
> memset(info, 0, sizeof(*info));
>
> @@ -615,30 +623,18 @@ static int find_bpffs_btf_enums(struct bpffs_btf_enums *info)
>
> info->btf = btf;
>
> - for (i = 1, n = btf_nr_types(btf); i < n; i++) {
> - t = btf_type_by_id(btf, i);
> - if (!btf_type_is_enum(t))
> - continue;
> + for (i = 0; i < ARRAY_SIZE(btf_enums); i++) {
> + id = btf_find_by_name_kind(btf, btf_enums[i].name,
> + BTF_KIND_ENUM);
> + if (id < 0)
> + goto out;
>
> - name = btf_name_by_offset(btf, t->name_off);
> - if (!name)
> - continue;
return -ESRCH, why goto at all?
> -
> - if (strcmp(name, "bpf_cmd") == 0)
> - info->cmd_t = t;
> - else if (strcmp(name, "bpf_map_type") == 0)
> - info->map_t = t;
> - else if (strcmp(name, "bpf_prog_type") == 0)
> - info->prog_t = t;
> - else if (strcmp(name, "bpf_attach_type") == 0)
> - info->attach_t = t;
> - else
> - continue;
> -
> - if (info->cmd_t && info->map_t && info->prog_t && info->attach_t)
> - return 0;
> + t = btf_type_by_id(btf, id);
> + *btf_enums[i].type = t;
nit: drop t local variable, just assign directly:
*btf_enums[i].type = btf_type_by_id(btf, id);
> }
>
> + return 0;
> +out:
> return -ESRCH;
> }
>
> --
> 2.34.1
>
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 10/13] libbpf: Optimize the performance of determine_ptr_size
2025-12-18 11:30 ` [PATCH bpf-next v10 10/13] libbpf: Optimize the performance of determine_ptr_size Donglin Peng
2025-12-18 22:27 ` Eduard Zingerman
@ 2025-12-19 0:03 ` Andrii Nakryiko
2025-12-19 5:42 ` Donglin Peng
1 sibling, 1 reply; 72+ messages in thread
From: Andrii Nakryiko @ 2025-12-19 0:03 UTC (permalink / raw)
To: Donglin Peng
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
>
> From: pengdonglin <pengdonglin@xiaomi.com>
>
> Leverage the performance improvement of btf__find_by_name_kind() when
> BTF is sorted. For sorted BTF, the function uses binary search with
> O(log n) complexity instead of linear search, providing significant
> performance benefits, especially for large BTF like vmlinux.
>
> Cc: Eduard Zingerman <eddyz87@gmail.com>
> Cc: Alexei Starovoitov <ast@kernel.org>
> Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> Cc: Alan Maguire <alan.maguire@oracle.com>
> Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> ---
> tools/lib/bpf/btf.c | 20 ++++++--------------
> 1 file changed, 6 insertions(+), 14 deletions(-)
>
nice and clean
Acked-by: Andrii Nakryiko <andrii@kernel.org>
> diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> index c63d46b7d74b..b5b0898d033d 100644
> --- a/tools/lib/bpf/btf.c
> +++ b/tools/lib/bpf/btf.c
> @@ -659,29 +659,21 @@ static int determine_ptr_size(const struct btf *btf)
> "int long unsigned",
> };
> const struct btf_type *t;
> - const char *name;
> - int i, j, n;
> + int i, id;
>
> if (btf->base_btf && btf->base_btf->ptr_sz > 0)
> return btf->base_btf->ptr_sz;
>
> - n = btf__type_cnt(btf);
> - for (i = 1; i < n; i++) {
> - t = btf__type_by_id(btf, i);
> - if (!btf_is_int(t))
> + for (i = 0; i < ARRAY_SIZE(long_aliases); i++) {
> + id = btf__find_by_name_kind(btf, long_aliases[i], BTF_KIND_INT);
> + if (id < 0)
> continue;
>
> + t = btf__type_by_id(btf, id);
> if (t->size != 4 && t->size != 8)
> continue;
>
> - name = btf__name_by_offset(btf, t->name_off);
> - if (!name)
> - continue;
> -
> - for (j = 0; j < ARRAY_SIZE(long_aliases); j++) {
> - if (strcmp(name, long_aliases[j]) == 0)
> - return t->size;
> - }
> + return t->size;
> }
>
> return -1;
> --
> 2.34.1
>
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 11/13] libbpf: Add btf_is_sorted and btf_sorted_start_id helpers to refactor the code
2025-12-18 11:30 ` [PATCH bpf-next v10 11/13] libbpf: Add btf_is_sorted and btf_sorted_start_id helpers to refactor the code Donglin Peng
2025-12-18 22:33 ` Eduard Zingerman
@ 2025-12-19 0:05 ` Andrii Nakryiko
2025-12-19 5:51 ` Donglin Peng
1 sibling, 1 reply; 72+ messages in thread
From: Andrii Nakryiko @ 2025-12-19 0:05 UTC (permalink / raw)
To: Donglin Peng
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
>
> From: pengdonglin <pengdonglin@xiaomi.com>
>
> Introduce two new helper functions to clarify the code and no
> functional changes are introduced.
>
> Cc: Eduard Zingerman <eddyz87@gmail.com>
> Cc: Alexei Starovoitov <ast@kernel.org>
> Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> Cc: Alan Maguire <alan.maguire@oracle.com>
> Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> ---
> tools/lib/bpf/btf.c | 14 ++++++++++++--
> tools/lib/bpf/libbpf_internal.h | 2 ++
> 2 files changed, 14 insertions(+), 2 deletions(-)
>
It just adds more functions to jump to and check what it's doing. I
don't think this adds much value, just drop this patch
> diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> index b5b0898d033d..571b72bd90b5 100644
> --- a/tools/lib/bpf/btf.c
> +++ b/tools/lib/bpf/btf.c
> @@ -626,6 +626,16 @@ const struct btf *btf__base_btf(const struct btf *btf)
> return btf->base_btf;
> }
>
> +int btf_sorted_start_id(const struct btf *btf)
> +{
> + return btf->sorted_start_id;
> +}
> +
> +bool btf_is_sorted(const struct btf *btf)
> +{
> + return btf->sorted_start_id > 0;
> +}
> +
> /* internal helper returning non-const pointer to a type */
> struct btf_type *btf_type_by_id(const struct btf *btf, __u32 type_id)
> {
> @@ -976,11 +986,11 @@ static __s32 btf_find_by_name_kind(const struct btf *btf, int start_id,
> if (kind == BTF_KIND_UNKN || strcmp(type_name, "void") == 0)
> return 0;
>
> - if (btf->sorted_start_id > 0 && type_name[0]) {
> + if (btf_is_sorted(btf) && type_name[0]) {
> __s32 end_id = btf__type_cnt(btf) - 1;
>
> /* skip anonymous types */
> - start_id = max(start_id, btf->sorted_start_id);
> + start_id = max(start_id, btf_sorted_start_id(btf));
> idx = btf_find_by_name_bsearch(btf, type_name, start_id, end_id);
> if (unlikely(idx < 0))
> return libbpf_err(-ENOENT);
> diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_internal.h
> index fc59b21b51b5..95e6848396b4 100644
> --- a/tools/lib/bpf/libbpf_internal.h
> +++ b/tools/lib/bpf/libbpf_internal.h
> @@ -250,6 +250,8 @@ const struct btf_type *skip_mods_and_typedefs(const struct btf *btf, __u32 id, _
> const struct btf_header *btf_header(const struct btf *btf);
> void btf_set_base_btf(struct btf *btf, const struct btf *base_btf);
> int btf_relocate(struct btf *btf, const struct btf *base_btf, __u32 **id_map);
> +int btf_sorted_start_id(const struct btf *btf);
> +bool btf_is_sorted(const struct btf *btf);
>
> static inline enum btf_func_linkage btf_func_linkage(const struct btf_type *t)
> {
> --
> 2.34.1
>
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 12/13] btf: Add btf_is_sorted to refactor the code
2025-12-18 11:30 ` [PATCH bpf-next v10 12/13] btf: Add btf_is_sorted " Donglin Peng
2025-12-18 22:30 ` Eduard Zingerman
@ 2025-12-19 0:05 ` Andrii Nakryiko
2025-12-20 14:25 ` Donglin Peng
1 sibling, 1 reply; 72+ messages in thread
From: Andrii Nakryiko @ 2025-12-19 0:05 UTC (permalink / raw)
To: Donglin Peng
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
>
> From: pengdonglin <pengdonglin@xiaomi.com>
>
> Introduce a new helper function to clarify the code and no
> functional changes are introduced.
>
> Cc: Eduard Zingerman <eddyz87@gmail.com>
> Cc: Alexei Starovoitov <ast@kernel.org>
> Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> Cc: Alan Maguire <alan.maguire@oracle.com>
> Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> ---
> include/linux/btf.h | 1 +
> kernel/bpf/btf.c | 9 +++++++--
> 2 files changed, 8 insertions(+), 2 deletions(-)
>
let's drop, this is not necessary
> diff --git a/include/linux/btf.h b/include/linux/btf.h
> index 2d28f2b22ae5..947ed2abf632 100644
> --- a/include/linux/btf.h
> +++ b/include/linux/btf.h
> @@ -221,6 +221,7 @@ bool btf_is_vmlinux(const struct btf *btf);
> struct module *btf_try_get_module(const struct btf *btf);
> u32 btf_nr_types(const struct btf *btf);
> u32 btf_sorted_start_id(const struct btf *btf);
> +bool btf_is_sorted(const struct btf *btf);
> struct btf *btf_base_btf(const struct btf *btf);
> bool btf_type_is_i32(const struct btf_type *t);
> bool btf_type_is_i64(const struct btf_type *t);
> diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
> index 3aeb4f00cbfe..0f20887a6f02 100644
> --- a/kernel/bpf/btf.c
> +++ b/kernel/bpf/btf.c
> @@ -555,6 +555,11 @@ u32 btf_sorted_start_id(const struct btf *btf)
> return btf->sorted_start_id ?: (btf->start_id ?: 1);
> }
>
> +bool btf_is_sorted(const struct btf *btf)
> +{
> + return btf->sorted_start_id > 0;
> +}
> +
> /*
> * Assuming that types are sorted by name in ascending order.
> */
> @@ -649,9 +654,9 @@ s32 btf_find_by_name_kind(const struct btf *btf, const char *name, u8 kind)
> return idx;
> }
>
> - if (btf->sorted_start_id > 0 && name[0]) {
> + if (btf_is_sorted(btf) && name[0]) {
> /* skip anonymous types */
> - s32 start_id = btf->sorted_start_id;
> + s32 start_id = btf_sorted_start_id(btf);
> s32 end_id = btf_nr_types(btf) - 1;
>
> idx = btf_find_by_name_bsearch(btf, name, start_id, end_id);
> --
> 2.34.1
>
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 04/13] libbpf: Optimize type lookup with binary search for sorted BTF
2025-12-18 23:29 ` Andrii Nakryiko
@ 2025-12-19 0:13 ` Eduard Zingerman
2025-12-19 0:19 ` Andrii Nakryiko
2025-12-19 2:53 ` Donglin Peng
1 sibling, 1 reply; 72+ messages in thread
From: Eduard Zingerman @ 2025-12-19 0:13 UTC (permalink / raw)
To: Andrii Nakryiko, Donglin Peng
Cc: ast, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf, pengdonglin,
Alan Maguire
On Thu, 2025-12-18 at 15:29 -0800, Andrii Nakryiko wrote:
[...]
> > static __s32 btf_find_by_name_kind(const struct btf *btf, int start_id,
> > const char *type_name, __u32 kind)
>
> kind is defined as u32 but you expect caller to pass -1 to ignore the
> kind. Use int here.
>
> > {
> > - __u32 i, nr_types = btf__type_cnt(btf);
> > + const struct btf_type *t;
> > + const char *tname;
> > + __s32 idx;
> > +
> > + if (start_id < btf->start_id) {
> > + idx = btf_find_by_name_kind(btf->base_btf, start_id,
> > + type_name, kind);
> > + if (idx >= 0)
> > + return idx;
> > + start_id = btf->start_id;
> > + }
> >
> > - if (kind == BTF_KIND_UNKN || !strcmp(type_name, "void"))
> > + if (kind == BTF_KIND_UNKN || strcmp(type_name, "void") == 0)
> > return 0;
> >
> > - for (i = start_id; i < nr_types; i++) {
> > - const struct btf_type *t = btf__type_by_id(btf, i);
> > - const char *name;
> > + if (btf->sorted_start_id > 0 && type_name[0]) {
> > + __s32 end_id = btf__type_cnt(btf) - 1;
> > +
> > + /* skip anonymous types */
> > + start_id = max(start_id, btf->sorted_start_id);
>
> can sorted_start_id ever be smaller than start_id?
sorted_start_id can be zero, at two callsites for this function
start_id is passed as btf->start_id and 1.
>
> > + idx = btf_find_by_name_bsearch(btf, type_name, start_id, end_id);
>
> is there ever a time when btf_find_by_name_bsearch() will work with
> different start_id and end_id? why is this not done inside the
> btf_find_by_name_bsearch()?
>
[...]
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 04/13] libbpf: Optimize type lookup with binary search for sorted BTF
2025-12-19 0:13 ` Eduard Zingerman
@ 2025-12-19 0:19 ` Andrii Nakryiko
2025-12-19 0:24 ` Eduard Zingerman
0 siblings, 1 reply; 72+ messages in thread
From: Andrii Nakryiko @ 2025-12-19 0:19 UTC (permalink / raw)
To: Eduard Zingerman
Cc: Donglin Peng, ast, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Thu, Dec 18, 2025 at 4:13 PM Eduard Zingerman <eddyz87@gmail.com> wrote:
>
> On Thu, 2025-12-18 at 15:29 -0800, Andrii Nakryiko wrote:
>
> [...]
>
> > > static __s32 btf_find_by_name_kind(const struct btf *btf, int start_id,
> > > const char *type_name, __u32 kind)
> >
> > kind is defined as u32 but you expect caller to pass -1 to ignore the
> > kind. Use int here.
> >
> > > {
> > > - __u32 i, nr_types = btf__type_cnt(btf);
> > > + const struct btf_type *t;
> > > + const char *tname;
> > > + __s32 idx;
> > > +
> > > + if (start_id < btf->start_id) {
> > > + idx = btf_find_by_name_kind(btf->base_btf, start_id,
> > > + type_name, kind);
> > > + if (idx >= 0)
> > > + return idx;
> > > + start_id = btf->start_id;
> > > + }
> > >
> > > - if (kind == BTF_KIND_UNKN || !strcmp(type_name, "void"))
> > > + if (kind == BTF_KIND_UNKN || strcmp(type_name, "void") == 0)
> > > return 0;
> > >
> > > - for (i = start_id; i < nr_types; i++) {
> > > - const struct btf_type *t = btf__type_by_id(btf, i);
> > > - const char *name;
> > > + if (btf->sorted_start_id > 0 && type_name[0]) {
> > > + __s32 end_id = btf__type_cnt(btf) - 1;
> > > +
> > > + /* skip anonymous types */
> > > + start_id = max(start_id, btf->sorted_start_id);
> >
> > can sorted_start_id ever be smaller than start_id?
>
> sorted_start_id can be zero, at two callsites for this function
> start_id is passed as btf->start_id and 1.
Can it with the check above?
if (btf->sorted_start_id > 0 && type_name[0]) {
This branch is a known sorted case. That's why all these start_id
manipulations look weird and sloppy.
>
> >
> > > + idx = btf_find_by_name_bsearch(btf, type_name, start_id, end_id);
> >
> > is there ever a time when btf_find_by_name_bsearch() will work with
> > different start_id and end_id? why is this not done inside the
> > btf_find_by_name_bsearch()?
> >
>
> [...]
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 04/13] libbpf: Optimize type lookup with binary search for sorted BTF
2025-12-19 0:19 ` Andrii Nakryiko
@ 2025-12-19 0:24 ` Eduard Zingerman
2025-12-19 1:01 ` Andrii Nakryiko
0 siblings, 1 reply; 72+ messages in thread
From: Eduard Zingerman @ 2025-12-19 0:24 UTC (permalink / raw)
To: Andrii Nakryiko
Cc: Donglin Peng, ast, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Thu, 2025-12-18 at 16:19 -0800, Andrii Nakryiko wrote:
> On Thu, Dec 18, 2025 at 4:13 PM Eduard Zingerman <eddyz87@gmail.com> wrote:
> >
> > On Thu, 2025-12-18 at 15:29 -0800, Andrii Nakryiko wrote:
> >
> > [...]
> >
> > > > static __s32 btf_find_by_name_kind(const struct btf *btf, int start_id,
> > > > const char *type_name, __u32 kind)
> > >
> > > kind is defined as u32 but you expect caller to pass -1 to ignore the
> > > kind. Use int here.
> > >
> > > > {
> > > > - __u32 i, nr_types = btf__type_cnt(btf);
> > > > + const struct btf_type *t;
> > > > + const char *tname;
> > > > + __s32 idx;
> > > > +
> > > > + if (start_id < btf->start_id) {
> > > > + idx = btf_find_by_name_kind(btf->base_btf, start_id,
> > > > + type_name, kind);
> > > > + if (idx >= 0)
> > > > + return idx;
> > > > + start_id = btf->start_id;
> > > > + }
> > > >
> > > > - if (kind == BTF_KIND_UNKN || !strcmp(type_name, "void"))
> > > > + if (kind == BTF_KIND_UNKN || strcmp(type_name, "void") == 0)
> > > > return 0;
> > > >
> > > > - for (i = start_id; i < nr_types; i++) {
> > > > - const struct btf_type *t = btf__type_by_id(btf, i);
> > > > - const char *name;
> > > > + if (btf->sorted_start_id > 0 && type_name[0]) {
> > > > + __s32 end_id = btf__type_cnt(btf) - 1;
> > > > +
> > > > + /* skip anonymous types */
> > > > + start_id = max(start_id, btf->sorted_start_id);
> > >
> > > can sorted_start_id ever be smaller than start_id?
> >
> > sorted_start_id can be zero, at two callsites for this function
> > start_id is passed as btf->start_id and 1.
>
> Can it with the check above?
>
> if (btf->sorted_start_id > 0 && type_name[0]) {
>
>
> This branch is a known sorted case. That's why all these start_id
> manipulations look weird and sloppy.
Oops, it cannot.
But still it feels strange to pass a 'start_id' parameter to a
function and rely at exact values passed at callsites. Replace the
parameter with boolean 'own'?
> >
> > >
> > > > + idx = btf_find_by_name_bsearch(btf, type_name, start_id, end_id);
> > >
> > > is there ever a time when btf_find_by_name_bsearch() will work with
> > > different start_id and end_id? why is this not done inside the
> > > btf_find_by_name_bsearch()?
> > >
> >
> > [...]
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 04/13] libbpf: Optimize type lookup with binary search for sorted BTF
2025-12-19 0:24 ` Eduard Zingerman
@ 2025-12-19 1:01 ` Andrii Nakryiko
0 siblings, 0 replies; 72+ messages in thread
From: Andrii Nakryiko @ 2025-12-19 1:01 UTC (permalink / raw)
To: Eduard Zingerman
Cc: Donglin Peng, ast, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Thu, Dec 18, 2025 at 4:25 PM Eduard Zingerman <eddyz87@gmail.com> wrote:
>
> On Thu, 2025-12-18 at 16:19 -0800, Andrii Nakryiko wrote:
> > On Thu, Dec 18, 2025 at 4:13 PM Eduard Zingerman <eddyz87@gmail.com> wrote:
> > >
> > > On Thu, 2025-12-18 at 15:29 -0800, Andrii Nakryiko wrote:
> > >
> > > [...]
> > >
> > > > > static __s32 btf_find_by_name_kind(const struct btf *btf, int start_id,
> > > > > const char *type_name, __u32 kind)
> > > >
> > > > kind is defined as u32 but you expect caller to pass -1 to ignore the
> > > > kind. Use int here.
> > > >
> > > > > {
> > > > > - __u32 i, nr_types = btf__type_cnt(btf);
> > > > > + const struct btf_type *t;
> > > > > + const char *tname;
> > > > > + __s32 idx;
> > > > > +
> > > > > + if (start_id < btf->start_id) {
> > > > > + idx = btf_find_by_name_kind(btf->base_btf, start_id,
> > > > > + type_name, kind);
> > > > > + if (idx >= 0)
> > > > > + return idx;
> > > > > + start_id = btf->start_id;
> > > > > + }
> > > > >
> > > > > - if (kind == BTF_KIND_UNKN || !strcmp(type_name, "void"))
> > > > > + if (kind == BTF_KIND_UNKN || strcmp(type_name, "void") == 0)
> > > > > return 0;
> > > > >
> > > > > - for (i = start_id; i < nr_types; i++) {
> > > > > - const struct btf_type *t = btf__type_by_id(btf, i);
> > > > > - const char *name;
> > > > > + if (btf->sorted_start_id > 0 && type_name[0]) {
> > > > > + __s32 end_id = btf__type_cnt(btf) - 1;
> > > > > +
> > > > > + /* skip anonymous types */
> > > > > + start_id = max(start_id, btf->sorted_start_id);
> > > >
> > > > can sorted_start_id ever be smaller than start_id?
> > >
> > > sorted_start_id can be zero, at two callsites for this function
> > > start_id is passed as btf->start_id and 1.
> >
> > Can it with the check above?
> >
> > if (btf->sorted_start_id > 0 && type_name[0]) {
> >
> >
> > This branch is a known sorted case. That's why all these start_id
> > manipulations look weird and sloppy.
>
> Oops, it cannot.
> But still it feels strange to pass a 'start_id' parameter to a
> function and rely at exact values passed at callsites. Replace the
> parameter with boolean 'own'?
hm.. looking around a bit more, it seems like passed-in start_id might
be useful for iterator-like searches. We don't use that right now, but
maybe we should preserve this behavior? And then max() does make sense
(though "skip anonymous types" is a bit too brief, I'd mention that
start_id might be within the named types intentionally and we need to
jump forward)
>
> > >
> > > >
> > > > > + idx = btf_find_by_name_bsearch(btf, type_name, start_id, end_id);
> > > >
> > > > is there ever a time when btf_find_by_name_bsearch() will work with
> > > > different start_id and end_id? why is this not done inside the
> > > > btf_find_by_name_bsearch()?
> > > >
> > >
> > > [...]
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 04/13] libbpf: Optimize type lookup with binary search for sorted BTF
2025-12-18 23:29 ` Andrii Nakryiko
2025-12-19 0:13 ` Eduard Zingerman
@ 2025-12-19 2:53 ` Donglin Peng
2025-12-19 17:28 ` Andrii Nakryiko
1 sibling, 1 reply; 72+ messages in thread
From: Donglin Peng @ 2025-12-19 2:53 UTC (permalink / raw)
To: Andrii Nakryiko
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Fri, Dec 19, 2025 at 7:29 AM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
> >
> > From: pengdonglin <pengdonglin@xiaomi.com>
> >
> > This patch introduces binary search optimization for BTF type lookups
> > when the BTF instance contains sorted types.
> >
> > The optimization significantly improves performance when searching for
> > types in large BTF instances with sorted types. For unsorted BTF, the
> > implementation falls back to the original linear search.
> >
> > Cc: Eduard Zingerman <eddyz87@gmail.com>
> > Cc: Alexei Starovoitov <ast@kernel.org>
> > Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > Cc: Alan Maguire <alan.maguire@oracle.com>
> > Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> > Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> > Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> > ---
> > tools/lib/bpf/btf.c | 103 ++++++++++++++++++++++++++++++++++----------
> > 1 file changed, 80 insertions(+), 23 deletions(-)
> >
>
> [...]
>
> > + l = start_id;
> > + r = end_id;
> > + while (l <= r) {
> > + m = l + (r - l) / 2;
> > + t = btf_type_by_id(btf, m);
> > + tname = btf__str_by_offset(btf, t->name_off);
> > + ret = strcmp(tname, name);
> > + if (ret < 0) {
> > + l = m + 1;
> > + } else {
> > + if (ret == 0)
> > + lmost = m;
> > + r = m - 1;
> > + }
> > }
>
> this differs from what we discussed in [0], you said you'll use that
> approach. Can you please elaborate on why you didn't?
>
> [0] https://lore.kernel.org/bpf/CAEf4Bzb3Eu0J83O=Y4KA-LkzBMjtx7cbonxPzkiduzZ1Pedajg@mail.gmail.com/
Yes. As mentioned in the v8 changelog [1], the binary search approach
you referenced was implemented in versions v6 and v7 [2]. However,
testing revealed a slight performance regression. The root cause was
an extra strcmp operation introduced in v7, as discussed in [3]. Therefore,
in v8, I reverted to the approach from v5 [4] and refactored it for clarity.
Benchmark results show that v8 achieves a 4.2% performance improvement
over v7. If we don't care the performance gain, I will revert to the approach
in v7 in the next version.
[1] https://lore.kernel.org/bpf/20251126085025.784288-1-dolinux.peng@gmail.com/
[2] https://lore.kernel.org/all/20251119031531.1817099-1-dolinux.peng@gmail.com/
[3] https://lore.kernel.org/all/CAEf4BzaqEPD46LddJHO1-k5KPGyVWf6d=duDAxG1q=jykJkMBg@mail.gmail.com/
[4] https://lore.kernel.org/all/20251106131956.1222864-4-dolinux.peng@gmail.com/
>
> >
> > - return libbpf_err(-ENOENT);
> > + return lmost;
> > }
> >
> > static __s32 btf_find_by_name_kind(const struct btf *btf, int start_id,
> > const char *type_name, __u32 kind)
>
> kind is defined as u32 but you expect caller to pass -1 to ignore the
> kind. Use int here.
Thanks, I will fix it.
>
> > {
> > - __u32 i, nr_types = btf__type_cnt(btf);
> > + const struct btf_type *t;
> > + const char *tname;
> > + __s32 idx;
> > +
> > + if (start_id < btf->start_id) {
> > + idx = btf_find_by_name_kind(btf->base_btf, start_id,
> > + type_name, kind);
> > + if (idx >= 0)
> > + return idx;
> > + start_id = btf->start_id;
> > + }
> >
> > - if (kind == BTF_KIND_UNKN || !strcmp(type_name, "void"))
> > + if (kind == BTF_KIND_UNKN || strcmp(type_name, "void") == 0)
> > return 0;
> >
> > - for (i = start_id; i < nr_types; i++) {
> > - const struct btf_type *t = btf__type_by_id(btf, i);
> > - const char *name;
> > + if (btf->sorted_start_id > 0 && type_name[0]) {
> > + __s32 end_id = btf__type_cnt(btf) - 1;
> > +
> > + /* skip anonymous types */
> > + start_id = max(start_id, btf->sorted_start_id);
>
> can sorted_start_id ever be smaller than start_id?
>
> > + idx = btf_find_by_name_bsearch(btf, type_name, start_id, end_id);
>
> is there ever a time when btf_find_by_name_bsearch() will work with
> different start_id and end_id? why is this not done inside the
> btf_find_by_name_bsearch()?
Because the start_id could be specified by the caller.
>
> > + if (unlikely(idx < 0))
> > + return libbpf_err(-ENOENT);
>
> pass through error returned from btf_find_by_name_bsearch(), why redefining it?
Thanks, I will fix it.
>
> > +
> > + if (unlikely(kind == -1))
> > + return idx;
> > +
> > + t = btf_type_by_id(btf, idx);
> > + if (likely(BTF_INFO_KIND(t->info) == kind))
>
> use btf_kind(), but this whole extra check is just unnecessary, this
Thanks, I will do it.
> should be done in the loop below. We talked about all this already,
> why do I feel like I'm being ignored?..
Sorry for the confusion, and absolutely not ignoring you.
>
> > + return idx;
>
> drop all these likely and unlikely micro optimizations, please
Thanks, I will do it.
>
>
> > +
> > + for (idx++; idx <= end_id; idx++) {
> > + t = btf__type_by_id(btf, idx);
> > + tname = btf__str_by_offset(btf, t->name_off);
> > + if (strcmp(tname, type_name) != 0)
> > + return libbpf_err(-ENOENT);
> > + if (btf_kind(t) == kind)
> > + return idx;
> > + }
> > + } else {
> > + __u32 i, total;
> >
> > - if (btf_kind(t) != kind)
> > - continue;
> > - name = btf__name_by_offset(btf, t->name_off);
> > - if (name && !strcmp(type_name, name))
> > - return i;
> > + total = btf__type_cnt(btf);
> > + for (i = start_id; i < total; i++) {
> > + t = btf_type_by_id(btf, i);
> > + if (kind != -1 && btf_kind(t) != kind)
>
> nit: kind < 0, no need to hard-code -1
Good, I will fix it.
>
> > + continue;
> > + tname = btf__str_by_offset(btf, t->name_off);
> > + if (strcmp(tname, type_name) == 0)
> > + return i;
> > + }
> > }
> >
> > return libbpf_err(-ENOENT);
> > }
> >
>
> [...]
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 01/13] libbpf: Add BTF permutation support for type reordering
2025-12-18 23:02 ` Andrii Nakryiko
@ 2025-12-19 3:14 ` Donglin Peng
2025-12-19 3:16 ` Donglin Peng
2025-12-19 17:07 ` Andrii Nakryiko
0 siblings, 2 replies; 72+ messages in thread
From: Donglin Peng @ 2025-12-19 3:14 UTC (permalink / raw)
To: Andrii Nakryiko
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Fri, Dec 19, 2025 at 7:02 AM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
> >
> > From: pengdonglin <pengdonglin@xiaomi.com>
> >
> > Introduce btf__permute() API to allow in-place rearrangement of BTF types.
> > This function reorganizes BTF type order according to a provided array of
> > type IDs, updating all type references to maintain consistency.
> >
> > Cc: Eduard Zingerman <eddyz87@gmail.com>
> > Cc: Alexei Starovoitov <ast@kernel.org>
> > Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > Cc: Alan Maguire <alan.maguire@oracle.com>
> > Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> > Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> > Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> > Acked-by: Eduard Zingerman <eddyz87@gmail.com>
> > ---
> > tools/lib/bpf/btf.c | 119 +++++++++++++++++++++++++++++++++++++++
> > tools/lib/bpf/btf.h | 36 ++++++++++++
> > tools/lib/bpf/libbpf.map | 1 +
> > 3 files changed, 156 insertions(+)
> >
> > diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> > index b136572e889a..ab204ca403dc 100644
> > --- a/tools/lib/bpf/btf.c
> > +++ b/tools/lib/bpf/btf.c
> > @@ -5887,3 +5887,122 @@ int btf__relocate(struct btf *btf, const struct btf *base_btf)
> > btf->owns_base = false;
> > return libbpf_err(err);
> > }
> > +
> > +struct btf_permute {
> > + struct btf *btf;
> > + __u32 *id_map;
> > +};
> > +
> > +/* Callback function to remap individual type ID references */
> > +static int btf_permute_remap_type_id(__u32 *type_id, void *ctx)
> > +{
> > + struct btf_permute *p = ctx;
> > + __u32 new_type_id = *type_id;
> > +
> > + /* refer to the base BTF or VOID type */
> > + if (new_type_id < p->btf->start_id)
> > + return 0;
> > +
> > + if (new_type_id >= btf__type_cnt(p->btf))
> > + return -EINVAL;
> > +
> > + *type_id = p->id_map[new_type_id - p->btf->start_id];
> > + return 0;
> > +}
> > +
> > +int btf__permute(struct btf *btf, __u32 *id_map, __u32 id_map_cnt,
> > + const struct btf_permute_opts *opts)
> > +{
> > + struct btf_permute p;
> > + struct btf_ext *btf_ext;
> > + void *nt, *new_types = NULL;
> > + __u32 *order_map = NULL;
> > + int err = 0, i;
> > + __u32 id;
> > +
> > + if (!OPTS_VALID(opts, btf_permute_opts) || id_map_cnt != btf->nr_types)
> > + return libbpf_err(-EINVAL);
> > +
> > + /* record the sequence of types */
> > + order_map = calloc(id_map_cnt, sizeof(*id_map));
> > + if (!order_map) {
> > + err = -ENOMEM;
> > + goto done;
> > + }
> > +
> > + new_types = calloc(btf->hdr->type_len, 1);
> > + if (!new_types) {
> > + err = -ENOMEM;
> > + goto done;
> > + }
> > +
> > + if (btf_ensure_modifiable(btf)) {
> > + err = -ENOMEM;
> > + goto done;
> > + }
> > +
> > + for (i = 0; i < id_map_cnt; i++) {
> > + id = id_map[i];
> > + if (id < btf->start_id || id >= btf__type_cnt(btf)) {
> > + err = -EINVAL;
> > + goto done;
> > + }
> > + id -= btf->start_id;
> > + /* cannot be mapped to the same ID */
> > + if (order_map[id]) {
> > + err = -EINVAL;
> > + goto done;
> > + }
> > + order_map[id] = i + btf->start_id;
> > + }
> > +
> > + p.btf = btf;
> > + p.id_map = id_map;
> > + nt = new_types;
> > + for (i = 0; i < id_map_cnt; i++) {
> > + struct btf_field_iter it;
> > + const struct btf_type *t;
> > + __u32 *type_id;
> > + int type_size;
> > +
> > + id = order_map[i];
> > + t = btf__type_by_id(btf, id);
> > + type_size = btf_type_size(t);
> > + memcpy(nt, t, type_size);
> > +
> > + /* fix up referenced IDs for BTF */
> > + err = btf_field_iter_init(&it, nt, BTF_FIELD_ITER_IDS);
> > + if (err)
> > + goto done;
> > + while ((type_id = btf_field_iter_next(&it))) {
> > + err = btf_permute_remap_type_id(type_id, &p);
> > + if (err)
> > + goto done;
> > + }
> > +
> > + nt += type_size;
> > + }
> > +
> > + /* fix up referenced IDs for btf_ext */
> > + btf_ext = OPTS_GET(opts, btf_ext, NULL);
> > + if (btf_ext) {
> > + err = btf_ext_visit_type_ids(btf_ext, btf_permute_remap_type_id, &p);
> > + if (err)
> > + goto done;
> > + }
> > +
> > + for (nt = new_types, i = 0; i < id_map_cnt; i++) {
> > + btf->type_offs[i] = nt - new_types;
> > + nt += btf_type_size(nt);
> > + }
> > +
> > + free(order_map);
> > + free(btf->types_data);
> > + btf->types_data = new_types;
> > + return 0;
> > +
> > +done:
> > + free(order_map);
> > + free(new_types);
> > + return libbpf_err(err);
> > +}
> > diff --git a/tools/lib/bpf/btf.h b/tools/lib/bpf/btf.h
> > index cc01494d6210..5d560571b1b5 100644
> > --- a/tools/lib/bpf/btf.h
> > +++ b/tools/lib/bpf/btf.h
> > @@ -281,6 +281,42 @@ LIBBPF_API int btf__dedup(struct btf *btf, const struct btf_dedup_opts *opts);
> > */
> > LIBBPF_API int btf__relocate(struct btf *btf, const struct btf *base_btf);
> >
> > +struct btf_permute_opts {
> > + size_t sz;
> > + /* optional .BTF.ext info along the main BTF info */
> > + struct btf_ext *btf_ext;
> > + size_t :0;
> > +};
> > +#define btf_permute_opts__last_field btf_ext
> > +
> > +/**
> > + * @brief **btf__permute()** performs in-place BTF type rearrangement
> > + * @param btf BTF object to permute
> > + * @param id_map Array mapping original type IDs to new IDs
> > + * @param id_map_cnt Number of elements in @id_map
> > + * @param opts Optional parameters for BTF extension updates
> > + * @return 0 on success, negative error code on failure
> > + *
> > + * **btf__permute()** rearranges BTF types according to the specified ID mapping.
> > + * The @id_map array defines the new type ID for each original type ID.
> > + *
> > + * @id_map must include all types from ID `start_id` to `btf__type_cnt(btf) - 1`.
> > + * @id_map_cnt should be `btf__type_cnt(btf) - start_id`
> > + * The mapping is defined as: `id_map[original_id - start_id] = new_id`
>
> Would you mind paying attention to the feedback I left in [0]? Thank you.
Apologies for the delayed response, I would like to hear if someone has a
different idea.
>
> The contract should be id_map[original_id] = new_id for base BTF and
> id_map[original_id - btf__type_cnt(base_btf)] = new_id for split BTF.
> Special BTF type #0 (VOID) is considered to be part of base BTF,
> having id_map[0] = 0 is easy to check and enforce. And then it leaves
> us with a simple and logical rule for id_map. For split BTF we make
> necessary type ID shifts to avoid tons of wasted memory. But for base
> BTF there is no need to shift anything. So mapping the original type
> #X to #Y is id_map[X] = Y. Literally, "map X to Y", as simple as that.
>
> [0] https://lore.kernel.org/bpf/CAEf4BzY_k721TBfRSUeq5mB-7fgJhVKCeXVKO-W2EjQ0aS9AgA@mail.gmail.com/
Thanks. I implemented the approach in v6, but it had inconsistent internal
details for base and split BTF. It seems we prioritize external contract
consistency over internal inconsistencies, so I’ll revert to the v6 approach
and refine it for clarity.
>
> > + *
> > + * For base BTF, its `start_id` is fixed to 1, i.e. the VOID type can
> > + * not be redefined or remapped and its ID is fixed to 0.
> > + *
> > + * For split BTF, its `start_id` can be retrieved by calling
> > + * `btf__type_cnt(btf__base_btf(btf))`.
> > + *
> > + * On error, returns negative error code and sets errno:
> > + * - `-EINVAL`: Invalid parameters or ID mapping (duplicates, out-of-range)
> > + * - `-ENOMEM`: Memory allocation failure
> > + */
> > +LIBBPF_API int btf__permute(struct btf *btf, __u32 *id_map, __u32 id_map_cnt,
> > + const struct btf_permute_opts *opts);
> > +
> > struct btf_dump;
> >
> > struct btf_dump_opts {
> > diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map
> > index 84fb90a016c9..d18fbcea7578 100644
> > --- a/tools/lib/bpf/libbpf.map
> > +++ b/tools/lib/bpf/libbpf.map
> > @@ -453,4 +453,5 @@ LIBBPF_1.7.0 {
> > bpf_map__exclusive_program;
> > bpf_prog_assoc_struct_ops;
> > bpf_program__assoc_struct_ops;
> > + btf__permute;
> > } LIBBPF_1.6.0;
> > --
> > 2.34.1
> >
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 01/13] libbpf: Add BTF permutation support for type reordering
2025-12-19 3:14 ` Donglin Peng
@ 2025-12-19 3:16 ` Donglin Peng
2025-12-19 17:07 ` Andrii Nakryiko
1 sibling, 0 replies; 72+ messages in thread
From: Donglin Peng @ 2025-12-19 3:16 UTC (permalink / raw)
To: Andrii Nakryiko
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Fri, Dec 19, 2025 at 11:14 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
>
> On Fri, Dec 19, 2025 at 7:02 AM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
> >
> > On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
> > >
> > > From: pengdonglin <pengdonglin@xiaomi.com>
> > >
> > > Introduce btf__permute() API to allow in-place rearrangement of BTF types.
> > > This function reorganizes BTF type order according to a provided array of
> > > type IDs, updating all type references to maintain consistency.
> > >
> > > Cc: Eduard Zingerman <eddyz87@gmail.com>
> > > Cc: Alexei Starovoitov <ast@kernel.org>
> > > Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > > Cc: Alan Maguire <alan.maguire@oracle.com>
> > > Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> > > Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> > > Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> > > Acked-by: Eduard Zingerman <eddyz87@gmail.com>
> > > ---
> > > tools/lib/bpf/btf.c | 119 +++++++++++++++++++++++++++++++++++++++
> > > tools/lib/bpf/btf.h | 36 ++++++++++++
> > > tools/lib/bpf/libbpf.map | 1 +
> > > 3 files changed, 156 insertions(+)
> > >
> > > diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> > > index b136572e889a..ab204ca403dc 100644
> > > --- a/tools/lib/bpf/btf.c
> > > +++ b/tools/lib/bpf/btf.c
> > > @@ -5887,3 +5887,122 @@ int btf__relocate(struct btf *btf, const struct btf *base_btf)
> > > btf->owns_base = false;
> > > return libbpf_err(err);
> > > }
> > > +
> > > +struct btf_permute {
> > > + struct btf *btf;
> > > + __u32 *id_map;
> > > +};
> > > +
> > > +/* Callback function to remap individual type ID references */
> > > +static int btf_permute_remap_type_id(__u32 *type_id, void *ctx)
> > > +{
> > > + struct btf_permute *p = ctx;
> > > + __u32 new_type_id = *type_id;
> > > +
> > > + /* refer to the base BTF or VOID type */
> > > + if (new_type_id < p->btf->start_id)
> > > + return 0;
> > > +
> > > + if (new_type_id >= btf__type_cnt(p->btf))
> > > + return -EINVAL;
> > > +
> > > + *type_id = p->id_map[new_type_id - p->btf->start_id];
> > > + return 0;
> > > +}
> > > +
> > > +int btf__permute(struct btf *btf, __u32 *id_map, __u32 id_map_cnt,
> > > + const struct btf_permute_opts *opts)
> > > +{
> > > + struct btf_permute p;
> > > + struct btf_ext *btf_ext;
> > > + void *nt, *new_types = NULL;
> > > + __u32 *order_map = NULL;
> > > + int err = 0, i;
> > > + __u32 id;
> > > +
> > > + if (!OPTS_VALID(opts, btf_permute_opts) || id_map_cnt != btf->nr_types)
> > > + return libbpf_err(-EINVAL);
> > > +
> > > + /* record the sequence of types */
> > > + order_map = calloc(id_map_cnt, sizeof(*id_map));
> > > + if (!order_map) {
> > > + err = -ENOMEM;
> > > + goto done;
> > > + }
> > > +
> > > + new_types = calloc(btf->hdr->type_len, 1);
> > > + if (!new_types) {
> > > + err = -ENOMEM;
> > > + goto done;
> > > + }
> > > +
> > > + if (btf_ensure_modifiable(btf)) {
> > > + err = -ENOMEM;
> > > + goto done;
> > > + }
> > > +
> > > + for (i = 0; i < id_map_cnt; i++) {
> > > + id = id_map[i];
> > > + if (id < btf->start_id || id >= btf__type_cnt(btf)) {
> > > + err = -EINVAL;
> > > + goto done;
> > > + }
> > > + id -= btf->start_id;
> > > + /* cannot be mapped to the same ID */
> > > + if (order_map[id]) {
> > > + err = -EINVAL;
> > > + goto done;
> > > + }
> > > + order_map[id] = i + btf->start_id;
> > > + }
> > > +
> > > + p.btf = btf;
> > > + p.id_map = id_map;
> > > + nt = new_types;
> > > + for (i = 0; i < id_map_cnt; i++) {
> > > + struct btf_field_iter it;
> > > + const struct btf_type *t;
> > > + __u32 *type_id;
> > > + int type_size;
> > > +
> > > + id = order_map[i];
> > > + t = btf__type_by_id(btf, id);
> > > + type_size = btf_type_size(t);
> > > + memcpy(nt, t, type_size);
> > > +
> > > + /* fix up referenced IDs for BTF */
> > > + err = btf_field_iter_init(&it, nt, BTF_FIELD_ITER_IDS);
> > > + if (err)
> > > + goto done;
> > > + while ((type_id = btf_field_iter_next(&it))) {
> > > + err = btf_permute_remap_type_id(type_id, &p);
> > > + if (err)
> > > + goto done;
> > > + }
> > > +
> > > + nt += type_size;
> > > + }
> > > +
> > > + /* fix up referenced IDs for btf_ext */
> > > + btf_ext = OPTS_GET(opts, btf_ext, NULL);
> > > + if (btf_ext) {
> > > + err = btf_ext_visit_type_ids(btf_ext, btf_permute_remap_type_id, &p);
> > > + if (err)
> > > + goto done;
> > > + }
> > > +
> > > + for (nt = new_types, i = 0; i < id_map_cnt; i++) {
> > > + btf->type_offs[i] = nt - new_types;
> > > + nt += btf_type_size(nt);
> > > + }
> > > +
> > > + free(order_map);
> > > + free(btf->types_data);
> > > + btf->types_data = new_types;
> > > + return 0;
> > > +
> > > +done:
> > > + free(order_map);
> > > + free(new_types);
> > > + return libbpf_err(err);
> > > +}
> > > diff --git a/tools/lib/bpf/btf.h b/tools/lib/bpf/btf.h
> > > index cc01494d6210..5d560571b1b5 100644
> > > --- a/tools/lib/bpf/btf.h
> > > +++ b/tools/lib/bpf/btf.h
> > > @@ -281,6 +281,42 @@ LIBBPF_API int btf__dedup(struct btf *btf, const struct btf_dedup_opts *opts);
> > > */
> > > LIBBPF_API int btf__relocate(struct btf *btf, const struct btf *base_btf);
> > >
> > > +struct btf_permute_opts {
> > > + size_t sz;
> > > + /* optional .BTF.ext info along the main BTF info */
> > > + struct btf_ext *btf_ext;
> > > + size_t :0;
> > > +};
> > > +#define btf_permute_opts__last_field btf_ext
> > > +
> > > +/**
> > > + * @brief **btf__permute()** performs in-place BTF type rearrangement
> > > + * @param btf BTF object to permute
> > > + * @param id_map Array mapping original type IDs to new IDs
> > > + * @param id_map_cnt Number of elements in @id_map
> > > + * @param opts Optional parameters for BTF extension updates
> > > + * @return 0 on success, negative error code on failure
> > > + *
> > > + * **btf__permute()** rearranges BTF types according to the specified ID mapping.
> > > + * The @id_map array defines the new type ID for each original type ID.
> > > + *
> > > + * @id_map must include all types from ID `start_id` to `btf__type_cnt(btf) - 1`.
> > > + * @id_map_cnt should be `btf__type_cnt(btf) - start_id`
> > > + * The mapping is defined as: `id_map[original_id - start_id] = new_id`
> >
> > Would you mind paying attention to the feedback I left in [0]? Thank you.
>
> Apologies for the delayed response, I would like to hear if someone has a
> different idea.
>
> >
> > The contract should be id_map[original_id] = new_id for base BTF and
> > id_map[original_id - btf__type_cnt(base_btf)] = new_id for split BTF.
> > Special BTF type #0 (VOID) is considered to be part of base BTF,
> > having id_map[0] = 0 is easy to check and enforce. And then it leaves
> > us with a simple and logical rule for id_map. For split BTF we make
> > necessary type ID shifts to avoid tons of wasted memory. But for base
> > BTF there is no need to shift anything. So mapping the original type
> > #X to #Y is id_map[X] = Y. Literally, "map X to Y", as simple as that.
> >
> > [0] https://lore.kernel.org/bpf/CAEf4BzY_k721TBfRSUeq5mB-7fgJhVKCeXVKO-W2EjQ0aS9AgA@mail.gmail.com/
>
> Thanks. I implemented the approach in v6, but it had inconsistent internal
> details for base and split BTF. It seems we prioritize external contract
> consistency over internal inconsistencies, so I’ll revert to the v6 approach
> and refine it for clarity.
Link to v6: https://lore.kernel.org/all/20251117132623.3807094-2-dolinux.peng@gmail.com/
>
> >
> > > + *
> > > + * For base BTF, its `start_id` is fixed to 1, i.e. the VOID type can
> > > + * not be redefined or remapped and its ID is fixed to 0.
> > > + *
> > > + * For split BTF, its `start_id` can be retrieved by calling
> > > + * `btf__type_cnt(btf__base_btf(btf))`.
> > > + *
> > > + * On error, returns negative error code and sets errno:
> > > + * - `-EINVAL`: Invalid parameters or ID mapping (duplicates, out-of-range)
> > > + * - `-ENOMEM`: Memory allocation failure
> > > + */
> > > +LIBBPF_API int btf__permute(struct btf *btf, __u32 *id_map, __u32 id_map_cnt,
> > > + const struct btf_permute_opts *opts);
> > > +
> > > struct btf_dump;
> > >
> > > struct btf_dump_opts {
> > > diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map
> > > index 84fb90a016c9..d18fbcea7578 100644
> > > --- a/tools/lib/bpf/libbpf.map
> > > +++ b/tools/lib/bpf/libbpf.map
> > > @@ -453,4 +453,5 @@ LIBBPF_1.7.0 {
> > > bpf_map__exclusive_program;
> > > bpf_prog_assoc_struct_ops;
> > > bpf_program__assoc_struct_ops;
> > > + btf__permute;
> > > } LIBBPF_1.6.0;
> > > --
> > > 2.34.1
> > >
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 02/13] selftests/bpf: Add test cases for btf__permute functionality
2025-12-18 23:03 ` Andrii Nakryiko
@ 2025-12-19 3:30 ` Donglin Peng
0 siblings, 0 replies; 72+ messages in thread
From: Donglin Peng @ 2025-12-19 3:30 UTC (permalink / raw)
To: Andrii Nakryiko
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Fri, Dec 19, 2025 at 7:03 AM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
> >
> > From: pengdonglin <pengdonglin@xiaomi.com>
> >
> > This patch introduces test cases for the btf__permute function to ensure
> > it works correctly with both base BTF and split BTF scenarios.
> >
> > The test suite includes:
> > - test_permute_base: Validates permutation on base BTF
> > - test_permute_split: Tests permutation on split BTF
> >
> > Cc: Eduard Zingerman <eddyz87@gmail.com>
> > Cc: Alexei Starovoitov <ast@kernel.org>
> > Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > Cc: Alan Maguire <alan.maguire@oracle.com>
> > Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> > Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> > Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> > Acked-by: Eduard Zingerman <eddyz87@gmail.com>
> > ---
> > .../selftests/bpf/prog_tests/btf_permute.c | 228 ++++++++++++++++++
> > 1 file changed, 228 insertions(+)
> > create mode 100644 tools/testing/selftests/bpf/prog_tests/btf_permute.c
> >
> > diff --git a/tools/testing/selftests/bpf/prog_tests/btf_permute.c b/tools/testing/selftests/bpf/prog_tests/btf_permute.c
> > new file mode 100644
> > index 000000000000..9aa71cdf984a
> > --- /dev/null
> > +++ b/tools/testing/selftests/bpf/prog_tests/btf_permute.c
> > @@ -0,0 +1,228 @@
> > +// SPDX-License-Identifier: GPL-2.0
> > +/* Copyright (c) 2025 Xiaomi */
> > +
> > +#include <test_progs.h>
> > +#include <bpf/btf.h>
> > +#include "btf_helpers.h"
> > +
> > +static void permute_base_check(struct btf *btf)
> > +{
> > + VALIDATE_RAW_BTF(
> > + btf,
> > + "[1] STRUCT 's2' size=4 vlen=1\n"
> > + "\t'm' type_id=4 bits_offset=0",
> > + "[2] FUNC 'f' type_id=6 linkage=static",
> > + "[3] PTR '(anon)' type_id=4",
> > + "[4] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED",
> > + "[5] STRUCT 's1' size=4 vlen=1\n"
> > + "\t'm' type_id=4 bits_offset=0",
> > + "[6] FUNC_PROTO '(anon)' ret_type_id=4 vlen=1\n"
> > + "\t'p' type_id=3");
> > +}
> > +
> > +/* Ensure btf__permute work as expected with base BTF */
> > +static void test_permute_base(void)
> > +{
> > + struct btf *btf;
> > + __u32 permute_ids[6];
> > + int start_id = 1;
> > + int err;
> > +
> > + btf = btf__new_empty();
> > + if (!ASSERT_OK_PTR(btf, "empty_main_btf"))
> > + return;
> > +
> > + btf__add_int(btf, "int", 4, BTF_INT_SIGNED); /* [1] int */
> > + btf__add_ptr(btf, 1); /* [2] ptr to int */
> > + btf__add_struct(btf, "s1", 4); /* [3] struct s1 { */
> > + btf__add_field(btf, "m", 1, 0, 0); /* int m; */
> > + /* } */
> > + btf__add_struct(btf, "s2", 4); /* [4] struct s2 { */
> > + btf__add_field(btf, "m", 1, 0, 0); /* int m; */
> > + /* } */
> > + btf__add_func_proto(btf, 1); /* [5] int (*)(int *p); */
> > + btf__add_func_param(btf, "p", 2);
> > + btf__add_func(btf, "f", BTF_FUNC_STATIC, 5); /* [6] int f(int *p); */
> > +
> > + VALIDATE_RAW_BTF(
> > + btf,
> > + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED",
> > + "[2] PTR '(anon)' type_id=1",
> > + "[3] STRUCT 's1' size=4 vlen=1\n"
> > + "\t'm' type_id=1 bits_offset=0",
> > + "[4] STRUCT 's2' size=4 vlen=1\n"
> > + "\t'm' type_id=1 bits_offset=0",
> > + "[5] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n"
> > + "\t'p' type_id=2",
> > + "[6] FUNC 'f' type_id=5 linkage=static");
> > +
> > + permute_ids[1 - start_id] = 4; /* [1] -> [4] */
> > + permute_ids[2 - start_id] = 3; /* [2] -> [3] */
> > + permute_ids[3 - start_id] = 5; /* [3] -> [5] */
> > + permute_ids[4 - start_id] = 1; /* [4] -> [1] */
> > + permute_ids[5 - start_id] = 6; /* [5] -> [6] */
> > + permute_ids[6 - start_id] = 2; /* [6] -> [2] */
> > + err = btf__permute(btf, permute_ids, ARRAY_SIZE(permute_ids), NULL);
> > + if (!ASSERT_OK(err, "btf__permute_base"))
> > + goto done;
> > + permute_base_check(btf);
> > +
> > + /* id_map_cnt is invalid */
> > + permute_ids[1 - start_id] = 4; /* [1] -> [4] */
> > + permute_ids[2 - start_id] = 3; /* [2] -> [3] */
> > + permute_ids[3 - start_id] = 5; /* [3] -> [5] */
> > + permute_ids[4 - start_id] = 1; /* [4] -> [1] */
> > + permute_ids[5 - start_id] = 6; /* [5] -> [6] */
> > + permute_ids[6 - start_id] = 2; /* [6] -> [2] */
> > + err = btf__permute(btf, permute_ids, ARRAY_SIZE(permute_ids) - 1, NULL);
> > + if (!ASSERT_ERR(err, "btf__permute_base"))
> > + goto done;
> > + /* BTF is not modified */
> > + permute_base_check(btf);
> > +
> > + /* Multiple types can not be mapped to the same ID */
> > + permute_ids[1 - start_id] = 4;
> > + permute_ids[2 - start_id] = 4;
> > + permute_ids[3 - start_id] = 5;
> > + permute_ids[4 - start_id] = 1;
> > + permute_ids[5 - start_id] = 6;
> > + permute_ids[6 - start_id] = 2;
> > + err = btf__permute(btf, permute_ids, ARRAY_SIZE(permute_ids), NULL);
> > + if (!ASSERT_ERR(err, "btf__permute_base"))
> > + goto done;
> > + /* BTF is not modified */
> > + permute_base_check(btf);
> > +
> > + /* Type ID must be valid */
> > + permute_ids[1 - start_id] = 4;
> > + permute_ids[2 - start_id] = 3;
> > + permute_ids[3 - start_id] = 5;
> > + permute_ids[4 - start_id] = 1;
> > + permute_ids[5 - start_id] = 7;
> > + permute_ids[6 - start_id] = 2;
>
> please make sure this base BTF test doesn't use start_id, see comment
> on previous patch
Thanks. I will remove it in the next version.
>
> > + err = btf__permute(btf, permute_ids, ARRAY_SIZE(permute_ids), NULL);
> > + if (!ASSERT_ERR(err, "btf__permute_base"))
> > + goto done;
> > + /* BTF is not modified */
> > + permute_base_check(btf);
> > +
> > +done:
> > + btf__free(btf);
> > +}
> > +
> > +static void permute_split_check(struct btf *btf)
> > +{
> > + VALIDATE_RAW_BTF(
> > + btf,
> > + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED",
> > + "[2] PTR '(anon)' type_id=1",
> > + "[3] STRUCT 's2' size=4 vlen=1\n"
> > + "\t'm' type_id=1 bits_offset=0",
> > + "[4] FUNC 'f' type_id=5 linkage=static",
> > + "[5] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n"
> > + "\t'p' type_id=2",
> > + "[6] STRUCT 's1' size=4 vlen=1\n"
> > + "\t'm' type_id=1 bits_offset=0");
> > +}
> > +
> > +/* Ensure btf__permute work as expected with split BTF */
> > +static void test_permute_split(void)
> > +{
> > + struct btf *split_btf = NULL, *base_btf = NULL;
> > + __u32 permute_ids[4];
> > + int err;
> > + int start_id;
> > +
> > + base_btf = btf__new_empty();
> > + if (!ASSERT_OK_PTR(base_btf, "empty_main_btf"))
> > + return;
> > +
> > + btf__add_int(base_btf, "int", 4, BTF_INT_SIGNED); /* [1] int */
> > + btf__add_ptr(base_btf, 1); /* [2] ptr to int */
> > + VALIDATE_RAW_BTF(
> > + base_btf,
> > + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED",
> > + "[2] PTR '(anon)' type_id=1");
> > + split_btf = btf__new_empty_split(base_btf);
> > + if (!ASSERT_OK_PTR(split_btf, "empty_split_btf"))
> > + goto cleanup;
> > + btf__add_struct(split_btf, "s1", 4); /* [3] struct s1 { */
> > + btf__add_field(split_btf, "m", 1, 0, 0); /* int m; */
> > + /* } */
> > + btf__add_struct(split_btf, "s2", 4); /* [4] struct s2 { */
> > + btf__add_field(split_btf, "m", 1, 0, 0); /* int m; */
> > + /* } */
> > + btf__add_func_proto(split_btf, 1); /* [5] int (*)(int p); */
> > + btf__add_func_param(split_btf, "p", 2);
> > + btf__add_func(split_btf, "f", BTF_FUNC_STATIC, 5); /* [6] int f(int *p); */
> > +
> > + VALIDATE_RAW_BTF(
> > + split_btf,
> > + "[1] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED",
> > + "[2] PTR '(anon)' type_id=1",
> > + "[3] STRUCT 's1' size=4 vlen=1\n"
> > + "\t'm' type_id=1 bits_offset=0",
> > + "[4] STRUCT 's2' size=4 vlen=1\n"
> > + "\t'm' type_id=1 bits_offset=0",
> > + "[5] FUNC_PROTO '(anon)' ret_type_id=1 vlen=1\n"
> > + "\t'p' type_id=2",
> > + "[6] FUNC 'f' type_id=5 linkage=static");
> > +
> > + start_id = btf__type_cnt(base_btf);
> > + permute_ids[3 - start_id] = 6; /* [3] -> [6] */
> > + permute_ids[4 - start_id] = 3; /* [4] -> [3] */
> > + permute_ids[5 - start_id] = 5; /* [5] -> [5] */
> > + permute_ids[6 - start_id] = 4; /* [6] -> [4] */
> > + err = btf__permute(split_btf, permute_ids, ARRAY_SIZE(permute_ids), NULL);
> > + if (!ASSERT_OK(err, "btf__permute_split"))
> > + goto cleanup;
> > + permute_split_check(split_btf);
> > +
> > + /*
> > + * For split BTF, id_map_cnt must equal to the number of types
> > + * added on top of base BTF
> > + */
> > + permute_ids[3 - start_id] = 4;
> > + permute_ids[4 - start_id] = 3;
> > + permute_ids[5 - start_id] = 5;
> > + permute_ids[6 - start_id] = 6;
> > + err = btf__permute(split_btf, permute_ids, 3, NULL);
> > + if (!ASSERT_ERR(err, "btf__permute_split"))
> > + goto cleanup;
> > + /* BTF is not modified */
> > + permute_split_check(split_btf);
> > +
> > + /* Multiple types can not be mapped to the same ID */
> > + permute_ids[3 - start_id] = 4;
> > + permute_ids[4 - start_id] = 3;
> > + permute_ids[5 - start_id] = 3;
> > + permute_ids[6 - start_id] = 6;
> > + err = btf__permute(split_btf, permute_ids, 4, NULL);
> > + if (!ASSERT_ERR(err, "btf__permute_split"))
> > + goto cleanup;
> > + /* BTF is not modified */
> > + permute_split_check(split_btf);
> > +
> > + /* Can not map to base ID */
> > + permute_ids[3 - start_id] = 4;
> > + permute_ids[4 - start_id] = 2;
> > + permute_ids[5 - start_id] = 5;
> > + permute_ids[6 - start_id] = 6;
> > + err = btf__permute(split_btf, permute_ids, 4, NULL);
> > + if (!ASSERT_ERR(err, "btf__permute_split"))
> > + goto cleanup;
> > + /* BTF is not modified */
> > + permute_split_check(split_btf);
> > +
> > +cleanup:
> > + btf__free(split_btf);
> > + btf__free(base_btf);
> > +}
> > +
> > +void test_btf_permute(void)
> > +{
> > + if (test__start_subtest("permute_base"))
> > + test_permute_base();
> > + if (test__start_subtest("permute_split"))
> > + test_permute_split();
> > +}
> > --
> > 2.34.1
> >
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 03/13] tools/resolve_btfids: Support BTF sorting feature
2025-12-18 23:09 ` Andrii Nakryiko
@ 2025-12-19 3:42 ` Donglin Peng
2025-12-19 17:12 ` Andrii Nakryiko
0 siblings, 1 reply; 72+ messages in thread
From: Donglin Peng @ 2025-12-19 3:42 UTC (permalink / raw)
To: Andrii Nakryiko
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Fri, Dec 19, 2025 at 7:09 AM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
> >
> > From: pengdonglin <pengdonglin@xiaomi.com>
> >
> > This introduces a new BTF sorting phase that specifically sorts
> > BTF types by name in ascending order, so that the binary search
> > can be used to look up types.
> >
> > Cc: Eduard Zingerman <eddyz87@gmail.com>
> > Cc: Alexei Starovoitov <ast@kernel.org>
> > Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > Cc: Alan Maguire <alan.maguire@oracle.com>
> > Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> > Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> > Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
>
> Signed-off-by is supposed to use properly spelled full name, this
> should be "Donglin Peng", right?
Sorry, I will fix this in the next version. The reason is that our
company's Gerrit only accepts "pengdonglin" and does not
accept "Donglin Peng".
>
> > Acked-by: Eduard Zingerman <eddyz87@gmail.com>
> > ---
> > tools/bpf/resolve_btfids/main.c | 68 +++++++++++++++++++++++++++++++++
> > 1 file changed, 68 insertions(+)
> >
> > diff --git a/tools/bpf/resolve_btfids/main.c b/tools/bpf/resolve_btfids/main.c
> > index 3e88dc862d87..659de35748ec 100644
> > --- a/tools/bpf/resolve_btfids/main.c
> > +++ b/tools/bpf/resolve_btfids/main.c
> > @@ -848,6 +848,71 @@ static int dump_raw_btf(struct btf *btf, const char *out_path)
> > return 0;
> > }
> >
> > +/*
> > + * Sort types by name in ascending order resulting in all
> > + * anonymous types being placed before named types.
> > + */
> > +static int cmp_type_names(const void *a, const void *b, void *priv)
> > +{
> > + struct btf *btf = (struct btf *)priv;
> > + const struct btf_type *ta = btf__type_by_id(btf, *(__u32 *)a);
> > + const struct btf_type *tb = btf__type_by_id(btf, *(__u32 *)b);
> > + const char *na, *nb;
> > +
> > + na = btf__str_by_offset(btf, ta->name_off);
> > + nb = btf__str_by_offset(btf, tb->name_off);
> > + return strcmp(na, nb);
> > +}
> > +
> > +static int sort_btf_by_name(struct btf *btf)
> > +{
> > + __u32 *permute_ids = NULL, *id_map = NULL;
> > + int nr_types, i, err = 0;
> > + __u32 start_id = 1, id;
> > +
> > + if (btf__base_btf(btf))
> > + start_id = btf__type_cnt(btf__base_btf(btf));
> > + nr_types = btf__type_cnt(btf) - start_id;
> > + if (nr_types < 2)
> > + goto out;
>
> why this check, will anything break if you don't do it?
Because I think that if there are zero or only one type,
there is no need to sort.
>
> > +
> > + permute_ids = calloc(nr_types, sizeof(*permute_ids));
> > + if (!permute_ids) {
> > + err = -ENOMEM;
> > + goto out;
> > + }
> > +
> > + id_map = calloc(nr_types, sizeof(*id_map));
> > + if (!id_map) {
> > + err = -ENOMEM;
> > + goto out;
> > + }
> > +
> > + for (i = 0, id = start_id; i < nr_types; i++, id++)
> > + permute_ids[i] = id;
> > +
> > + qsort_r(permute_ids, nr_types, sizeof(*permute_ids), cmp_type_names, btf);
> > +
> > + for (i = 0; i < nr_types; i++) {
> > + id = permute_ids[i] - start_id;
> > + id_map[id] = i + start_id;
> > + }
> > +
> > + err = btf__permute(btf, id_map, nr_types, NULL);
> > + if (err)
> > + pr_err("FAILED: btf permute: %s\n", strerror(-err));
> > +
> > +out:
> > + free(permute_ids);
> > + free(id_map);
> > + return err;
> > +}
> > +
> > +static int btf2btf(struct object *obj)
>
> what's the point of having this function?
Sorting BTF is a type of `btf2btf` process. There may be other
types of `btf2btf` processes, which could be grouped together
here. If we currently don't care about these other processes,
I will retain only `sort_btf_by_name` in the next version.
>
> > +{
> > + return sort_btf_by_name(obj->btf);
> > +}
> > +
> > static inline int make_out_path(char *buf, u32 buf_sz, const char *in_path, const char *suffix)
> > {
> > int len = snprintf(buf, buf_sz, "%s%s", in_path, suffix);
> > @@ -906,6 +971,9 @@ int main(int argc, const char **argv)
> > if (load_btf(&obj))
> > goto out;
> >
> > + if (btf2btf(&obj))
> > + goto out;
> > +
> > if (elf_collect(&obj))
> > goto out;
> >
> > --
> > 2.34.1
> >
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 05/13] libbpf: Verify BTF Sorting
2025-12-18 23:44 ` Andrii Nakryiko
@ 2025-12-19 5:06 ` Donglin Peng
2025-12-19 17:33 ` Andrii Nakryiko
0 siblings, 1 reply; 72+ messages in thread
From: Donglin Peng @ 2025-12-19 5:06 UTC (permalink / raw)
To: Andrii Nakryiko
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Fri, Dec 19, 2025 at 7:44 AM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
> >
> > From: pengdonglin <pengdonglin@xiaomi.com>
> >
>
> typo in subject: "Sorting" -> "sorting", it looks weird capitalized like that
Thanks, I will do it.
>
> > This patch checks whether the BTF is sorted by name in ascending
> > order. If sorted, binary search will be used when looking up types.
> >
> > Cc: Eduard Zingerman <eddyz87@gmail.com>
> > Cc: Alexei Starovoitov <ast@kernel.org>
> > Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > Cc: Alan Maguire <alan.maguire@oracle.com>
> > Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> > Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> > Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> > ---
> > tools/lib/bpf/btf.c | 41 +++++++++++++++++++++++++++++++++++++++++
> > 1 file changed, 41 insertions(+)
> >
> > diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> > index 2facb57d7e5f..c63d46b7d74b 100644
> > --- a/tools/lib/bpf/btf.c
> > +++ b/tools/lib/bpf/btf.c
> > @@ -899,6 +899,46 @@ int btf__resolve_type(const struct btf *btf, __u32 type_id)
> > return type_id;
> > }
> >
> > +/*
> > + * Assuming that types are sorted by name in ascending order.
> > + */
>
> Unnecessary comment, and no, btf_compare_type_names() itself makes no
> such assumption, it just compares two provided types by name. Drop the
> comment, please.
>
> > +static int btf_compare_type_names(__u32 *a, __u32 *b, const struct btf *btf)
> > +{
> > + struct btf_type *ta = btf_type_by_id(btf, *a);
> > + struct btf_type *tb = btf_type_by_id(btf, *b);
> > + const char *na, *nb;
> > +
> > + na = btf__str_by_offset(btf, ta->name_off);
> > + nb = btf__str_by_offset(btf, tb->name_off);
> > + return strcmp(na, nb);
> > +}
>
> you use this function only in one place, there is no real point having
> it, especially that it uses **a pointer to type ID** as an
> interface... just inline its logic in that one loop below
>
> > +
> > +static void btf_check_sorted(struct btf *btf)
> > +{
> > + const struct btf_type *t;
> > + __u32 i, k, n;
> > + __u32 sorted_start_id;
> > +
> > + if (btf->nr_types < 2)
> > + return;
>
> why special casing? does it not work with nr_types = 0 or nr_types = 1?
No. I just think it doesn't make any sense to check the sorting
of BTF with zero or only one type.
>
> > +
> > + sorted_start_id = 0;
>
> nit: initialize in declaration
Thanks, I will do it.
>
>
> > + n = btf__type_cnt(btf);
> > + for (i = btf->start_id; i < n; i++) {
> > + k = i + 1;
> > + if (k < n && btf_compare_type_names(&i, &k, btf) > 0)
> > + return;
> > + if (sorted_start_id == 0) {
> > + t = btf_type_by_id(btf, i);
> > + if (t->name_off)
>
> I'd check actual string, not name_off. Technically, you can have empty
> string with non-zero name_off, so why assume anything here?
Thanks, I will do it.
>
> > + sorted_start_id = i;
> > + }
> > + }
> > +
> > + if (sorted_start_id)
> > + btf->sorted_start_id = sorted_start_id;
>
> You actually made code more complicated by extracting that
> btf_compare_type_names(). Compare to:
>
> n = btf__type_cnt(btf);
> btf->sorted_start_id = 0;
> for (i = btf->start_id + 1; i < n; i++) {
> struct btf_type *t1 = btf_type_by_id(btf, i - 1);
> struct btf_type *t2 = btf_type_by_id(btf, i);
> const char *n1 = btf__str_by_offset(btf, t1->name_off);
> const char *n2 = btf__str_by_offset(btf, t2->name_off);
>
> if (strcmp(n1, n2) > 0)
> return;
> if (btf->sorted_start_id == 0 && n1[0] != '\0')
> btf->sorted_start_id = i - 1;
> }
Thanks. I believe we shouldn't directly assign a value to
`btf->sorted_start_id` within the for loop, because
`btf->sorted_start_id` might be non-zero even when the
BTF isn't sorted.
>
>
> No extra k<n checks, no extra type_by_id lookups. It's minimalistic
> and cleaner. And if it so happens that we get single type BTF that is
> technically sorted, it doesn't matter, we always fallback to faster
> linear search anyways.
>
> Keep it simple.
Thank you. I will adopt this method in the next version.
>
> > +}
> > +
> > static __s32 btf_find_by_name_bsearch(const struct btf *btf, const char *name,
> > __s32 start_id, __s32 end_id)
> > {
> > @@ -1147,6 +1187,7 @@ static struct btf *btf_new(const void *data, __u32 size, struct btf *base_btf, b
> > err = err ?: btf_sanity_check(btf);
> > if (err)
> > goto done;
> > + btf_check_sorted(btf);
> >
> > done:
> > if (err) {
> > --
> > 2.34.1
> >
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 06/13] btf: Optimize type lookup with binary search
2025-12-18 21:38 ` Eduard Zingerman
@ 2025-12-19 5:07 ` Donglin Peng
0 siblings, 0 replies; 72+ messages in thread
From: Donglin Peng @ 2025-12-19 5:07 UTC (permalink / raw)
To: Eduard Zingerman
Cc: ast, andrii.nakryiko, zhangxiaoqin, ihor.solodrai, linux-kernel,
bpf, pengdonglin, Alan Maguire
On Fri, Dec 19, 2025 at 5:38 AM Eduard Zingerman <eddyz87@gmail.com> wrote:
>
> On Thu, 2025-12-18 at 19:30 +0800, Donglin Peng wrote:
> > From: pengdonglin <pengdonglin@xiaomi.com>
> >
> > Improve btf_find_by_name_kind() performance by adding binary search
> > support for sorted types. Falls back to linear search for compatibility.
> >
> > Cc: Eduard Zingerman <eddyz87@gmail.com>
> > Cc: Alexei Starovoitov <ast@kernel.org>
> > Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > Cc: Alan Maguire <alan.maguire@oracle.com>
> > Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> > Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> > Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> > ---
>
> (One nit below).
>
> Acked-by: Eduard Zingerman <eddyz87@gmail.com>
>
> > kernel/bpf/btf.c | 85 +++++++++++++++++++++++++++++++++++++++++++-----
> > 1 file changed, 76 insertions(+), 9 deletions(-)
> >
> > diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
> > index 0de8fc8a0e0b..0394f0c8ef74 100644
> > --- a/kernel/bpf/btf.c
> > +++ b/kernel/bpf/btf.c
> > @@ -259,6 +259,7 @@ struct btf {
> > void *nohdr_data;
> > struct btf_header hdr;
> > u32 nr_types; /* includes VOID for base BTF */
> > + u32 sorted_start_id;
> > u32 types_size;
> > u32 data_size;
> > refcount_t refcnt;
> > @@ -494,6 +495,11 @@ static bool btf_type_is_modifier(const struct btf_type *t)
> > return false;
> > }
> >
> > +static int btf_start_id(const struct btf *btf)
> > +{
> > + return btf->start_id + (btf->base_btf ? 0 : 1);
> > +}
> > +
> > bool btf_type_is_void(const struct btf_type *t)
> > {
> > return t == &btf_void;
> > @@ -544,21 +550,79 @@ u32 btf_nr_types(const struct btf *btf)
> > return total;
> > }
> >
> > +static s32 btf_find_by_name_bsearch(const struct btf *btf, const char *name,
> > + s32 start_id, s32 end_id)
> > +{
> > + const struct btf_type *t;
> > + const char *tname;
> > + s32 l, r, m, lmost = -ENOENT;
> > + int ret;
> > +
> > + l = start_id;
> > + r = end_id;
> > + while (l <= r) {
> > + m = l + (r - l) / 2;
> > + t = btf_type_by_id(btf, m);
> > + tname = btf_name_by_offset(btf, t->name_off);
> > + ret = strcmp(tname, name);
> > + if (ret < 0) {
> > + l = m + 1;
> > + } else {
> > + if (ret == 0)
> > + lmost = m;
> > + r = m - 1;
> > + }
> > + }
> > +
> > + return lmost;
> > +}
> > +
> > s32 btf_find_by_name_kind(const struct btf *btf, const char *name, u8 kind)
> > {
> > + const struct btf *base_btf = btf_base_btf(btf);
> > const struct btf_type *t;
> > const char *tname;
> > - u32 i, total;
> > + s32 idx;
> >
> > - total = btf_nr_types(btf);
> > - for (i = 1; i < total; i++) {
> > - t = btf_type_by_id(btf, i);
> > - if (BTF_INFO_KIND(t->info) != kind)
> > - continue;
> > + if (base_btf) {
> > + idx = btf_find_by_name_kind(base_btf, name, kind);
> > + if (idx > 0)
> > + return idx;
> > + }
> >
> > - tname = btf_name_by_offset(btf, t->name_off);
> > - if (!strcmp(tname, name))
> > - return i;
> > + if (btf->sorted_start_id > 0 && name[0]) {
> ^^^^^^^^^^^^^^^^^^^^^^^^
> Nit: Maybe pull the is_sorted helper into this patch-set?
Thanks, will do it.
>
> > + /* skip anonymous types */
> > + s32 start_id = btf->sorted_start_id;
> > + s32 end_id = btf_nr_types(btf) - 1;
> > +
> > + idx = btf_find_by_name_bsearch(btf, name, start_id, end_id);
> > + if (idx < 0)
> > + return -ENOENT;
> > +
> > + t = btf_type_by_id(btf, idx);
> > + if (BTF_INFO_KIND(t->info) == kind)
> > + return idx;
> > +
> > + for (idx++; idx <= end_id; idx++) {
> > + t = btf_type_by_id(btf, idx);
> > + tname = btf_name_by_offset(btf, t->name_off);
> > + if (strcmp(tname, name) != 0)
> > + return -ENOENT;
> > + if (BTF_INFO_KIND(t->info) == kind)
> > + return idx;
> > + }
> > + } else {
> > + u32 i, total;
> > +
> > + total = btf_nr_types(btf);
> > + for (i = btf_start_id(btf); i < total; i++) {
> > + t = btf_type_by_id(btf, i);
> > + if (BTF_INFO_KIND(t->info) != kind)
> > + continue;
> > + tname = btf_name_by_offset(btf, t->name_off);
> > + if (strcmp(tname, name) == 0)
> > + return i;
> > + }
> > }
> >
> > return -ENOENT;
> > @@ -5791,6 +5855,7 @@ static struct btf *btf_parse(const union bpf_attr *attr, bpfptr_t uattr, u32 uat
> > goto errout;
> > }
> > env->btf = btf;
> > + btf->sorted_start_id = 0;
> >
> > data = kvmalloc(attr->btf_size, GFP_KERNEL | __GFP_NOWARN);
> > if (!data) {
> > @@ -6210,6 +6275,7 @@ static struct btf *btf_parse_base(struct btf_verifier_env *env, const char *name
> > btf->data = data;
> > btf->data_size = data_size;
> > btf->kernel_btf = true;
> > + btf->sorted_start_id = 0;
> > snprintf(btf->name, sizeof(btf->name), "%s", name);
> >
> > err = btf_parse_hdr(env);
> > @@ -6327,6 +6393,7 @@ static struct btf *btf_parse_module(const char *module_name, const void *data,
> > btf->start_id = base_btf->nr_types;
> > btf->start_str_off = base_btf->hdr.str_len;
> > btf->kernel_btf = true;
> > + btf->sorted_start_id = 0;
> > snprintf(btf->name, sizeof(btf->name), "%s", module_name);
> >
> > btf->data = kvmemdup(data, data_size, GFP_KERNEL | __GFP_NOWARN);
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 07/13] btf: Verify BTF Sorting
2025-12-18 21:43 ` Eduard Zingerman
@ 2025-12-19 5:10 ` Donglin Peng
0 siblings, 0 replies; 72+ messages in thread
From: Donglin Peng @ 2025-12-19 5:10 UTC (permalink / raw)
To: Eduard Zingerman
Cc: ast, andrii.nakryiko, zhangxiaoqin, ihor.solodrai, linux-kernel,
bpf, pengdonglin, Alan Maguire
On Fri, Dec 19, 2025 at 5:43 AM Eduard Zingerman <eddyz87@gmail.com> wrote:
>
> On Thu, 2025-12-18 at 19:30 +0800, Donglin Peng wrote:
> > From: pengdonglin <pengdonglin@xiaomi.com>
> >
> > This patch checks whether the BTF is sorted by name in ascending order.
> > If sorted, binary search will be used when looking up types.
> >
> > Specifically, vmlinux and kernel module BTFs are always sorted during
> > the build phase with anonymous types placed before named types, so we
> > only need to identify the starting ID of named types.
> >
> > Cc: Eduard Zingerman <eddyz87@gmail.com>
> > Cc: Alexei Starovoitov <ast@kernel.org>
> > Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > Cc: Alan Maguire <alan.maguire@oracle.com>
> > Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> > Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> > Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> > ---
>
> Lgtm, but please take a look at a few nits.
>
> > kernel/bpf/btf.c | 56 ++++++++++++++++++++++++++++++++++++++++++++++++
> > 1 file changed, 56 insertions(+)
> >
> > diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
> > index 0394f0c8ef74..a9e2345558c0 100644
> > --- a/kernel/bpf/btf.c
> > +++ b/kernel/bpf/btf.c
> > @@ -550,6 +550,60 @@ u32 btf_nr_types(const struct btf *btf)
> > return total;
> > }
> >
> > +/*
> > + * Assuming that types are sorted by name in ascending order.
> > + */
> > +static int btf_compare_type_names(u32 *a, u32 *b, const struct btf *btf)
>
> Nit: no need for 'a' and 'b' to be pointers.
Thank you. I will remove this function in the next version, as
suggested by Adrill.
>
> > +{
> > + const struct btf_type *ta = btf_type_by_id(btf, *a);
> > + const struct btf_type *tb = btf_type_by_id(btf, *b);
> > + const char *na, *nb;
> > +
> > + na = btf_name_by_offset(btf, ta->name_off);
> > + nb = btf_name_by_offset(btf, tb->name_off);
> > + return strcmp(na, nb);
> > +}
> > +
> > +/* Note that vmlinux and kernel module BTFs are always sorted
> > + * during the building phase.
> > + */
> > +static void btf_check_sorted(struct btf *btf)
> > +{
> > + const struct btf_type *t;
> > + u32 sorted_start_id;
> > + u32 i, n, k;
> > +
> > + if (btf_is_kernel(btf) && !btf_is_module(btf)) {
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> Nit: there is a btf_is_vmlinux() helper, which does the same thing.
Good catch.
>
> > + for (i = btf_start_id(btf); i < n; i++) {
> > + t = btf_type_by_id(btf, i);
> > + if (t->name_off) {
> > + btf->sorted_start_id = i;
> > + return;
> > + }
> > + }
>
> Nit: return here?
Agreed.
>
> > + }
> > +
> > + if (btf->nr_types < 2)
> > + return;
> > +
> > + sorted_start_id = 0;
> > + n = btf_nr_types(btf);
> > + for (i = btf_start_id(btf); i < n; i++) {
> > + k = i + 1;
> > + if (k < n && btf_compare_type_names(&i, &k, btf) > 0)
> > + return;
> > +
> > + if (sorted_start_id == 0) {
> > + t = btf_type_by_id(btf, i);
> > + if (t->name_off)
> > + sorted_start_id = i;
> > + }
> > + }
> > +
> > + if (sorted_start_id)
> > + btf->sorted_start_id = sorted_start_id;
> > +}
> > +
> > static s32 btf_find_by_name_bsearch(const struct btf *btf, const char *name,
> > s32 start_id, s32 end_id)
> > {
> > @@ -6296,6 +6350,7 @@ static struct btf *btf_parse_base(struct btf_verifier_env *env, const char *name
> > if (err)
> > goto errout;
> >
> > + btf_check_sorted(btf);
> > refcount_set(&btf->refcnt, 1);
> >
> > return btf;
> > @@ -6430,6 +6485,7 @@ static struct btf *btf_parse_module(const char *module_name, const void *data,
> > }
> >
> > btf_verifier_env_free(env);
> > + btf_check_sorted(btf);
> > refcount_set(&btf->refcnt, 1);
> > return btf;
> >
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 07/13] btf: Verify BTF Sorting
2025-12-18 23:46 ` Andrii Nakryiko
@ 2025-12-19 5:10 ` Donglin Peng
0 siblings, 0 replies; 72+ messages in thread
From: Donglin Peng @ 2025-12-19 5:10 UTC (permalink / raw)
To: Andrii Nakryiko
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Fri, Dec 19, 2025 at 7:46 AM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
> >
> > From: pengdonglin <pengdonglin@xiaomi.com>
> >
> > This patch checks whether the BTF is sorted by name in ascending order.
> > If sorted, binary search will be used when looking up types.
> >
> > Specifically, vmlinux and kernel module BTFs are always sorted during
> > the build phase with anonymous types placed before named types, so we
> > only need to identify the starting ID of named types.
> >
> > Cc: Eduard Zingerman <eddyz87@gmail.com>
> > Cc: Alexei Starovoitov <ast@kernel.org>
> > Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > Cc: Alan Maguire <alan.maguire@oracle.com>
> > Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> > Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> > Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> > ---
> > kernel/bpf/btf.c | 56 ++++++++++++++++++++++++++++++++++++++++++++++++
> > 1 file changed, 56 insertions(+)
> >
>
> please make sure to apply feedback received for libbpf-side
> implementation for kernel-side implementations as well
Thanks, I will do it.
>
> > diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
> > index 0394f0c8ef74..a9e2345558c0 100644
> > --- a/kernel/bpf/btf.c
> > +++ b/kernel/bpf/btf.c
> > @@ -550,6 +550,60 @@ u32 btf_nr_types(const struct btf *btf)
> > return total;
> > }
> >
>
> [...]
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 08/13] bpf: Skip anonymous types in type lookup for performance
2025-12-18 23:59 ` Andrii Nakryiko
@ 2025-12-19 5:40 ` Donglin Peng
0 siblings, 0 replies; 72+ messages in thread
From: Donglin Peng @ 2025-12-19 5:40 UTC (permalink / raw)
To: Andrii Nakryiko
Cc: Eduard Zingerman, ast, zhangxiaoqin, ihor.solodrai, linux-kernel,
bpf, pengdonglin, Alan Maguire
On Fri, Dec 19, 2025 at 7:59 AM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Thu, Dec 18, 2025 at 2:21 PM Eduard Zingerman <eddyz87@gmail.com> wrote:
> >
> > On Thu, 2025-12-18 at 19:30 +0800, Donglin Peng wrote:
> > > From: pengdonglin <pengdonglin@xiaomi.com>
> > >
> > > Currently, vmlinux and kernel module BTFs are unconditionally
> > > sorted during the build phase, with named types placed at the
> > > end. Thus, anonymous types should be skipped when starting the
> > > search. In my vmlinux BTF, the number of anonymous types is
> > > 61,747, which means the loop count can be reduced by 61,747.
> > >
> > > Cc: Eduard Zingerman <eddyz87@gmail.com>
> > > Cc: Alexei Starovoitov <ast@kernel.org>
> > > Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > > Cc: Alan Maguire <alan.maguire@oracle.com>
> > > Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> > > Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> > > Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> > > ---
> >
> > Acked-by: Eduard Zingerman <eddyz87@gmail.com>
> >
> > > include/linux/btf.h | 1 +
> > > kernel/bpf/btf.c | 24 ++++++++++++++++++++----
> > > kernel/bpf/verifier.c | 7 +------
> > > 3 files changed, 22 insertions(+), 10 deletions(-)
> > >
> > > diff --git a/include/linux/btf.h b/include/linux/btf.h
> > > index f06976ffb63f..2d28f2b22ae5 100644
> > > --- a/include/linux/btf.h
> > > +++ b/include/linux/btf.h
> > > @@ -220,6 +220,7 @@ bool btf_is_module(const struct btf *btf);
> > > bool btf_is_vmlinux(const struct btf *btf);
> > > struct module *btf_try_get_module(const struct btf *btf);
> > > u32 btf_nr_types(const struct btf *btf);
> > > +u32 btf_sorted_start_id(const struct btf *btf);
> > > struct btf *btf_base_btf(const struct btf *btf);
> > > bool btf_type_is_i32(const struct btf_type *t);
> > > bool btf_type_is_i64(const struct btf_type *t);
> > > diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
> > > index a9e2345558c0..3aeb4f00cbfe 100644
> > > --- a/kernel/bpf/btf.c
> > > +++ b/kernel/bpf/btf.c
> > > @@ -550,6 +550,11 @@ u32 btf_nr_types(const struct btf *btf)
> > > return total;
> > > }
> > >
> > > +u32 btf_sorted_start_id(const struct btf *btf)
> >
> > Nit: the name is a bit confusing, given that it not always returns the
> > start id for sorted part. btf_maybe_first_named_id?
> > Can't figure out a good name :(
>
> yeah, I agree, it is quite confusing overall. I think we should at
> least add comments why we start with something different than 1 in
> those few places where we use this optimization...
Thanks, I will add comments to make it more clear.
>
> let's name it btf_named_start_id() and specify in the comment that for
> non-sorted BTFs we conservatively fallback to the first type.
Thanks, I will do it.
>
> btw, maybe it would be good to have two versions of this (or bool
> flag,but we all hate bool flags) to either return own start id (i.e.,
> ignoring base BTF) or recursively go down to the base BTF.
Thanks, I will implement it.
>
> Having that
>
> while (base_btf->base_btf)
> base_btf = base_btf->base_btf;
>
> logic in a few places looks a bit too low-level and distracting, IMO.
Agreed.
>
> >
> > > +{
> > > + return btf->sorted_start_id ?: (btf->start_id ?: 1);
> > > +}
> > > +
> > > /*
> > > * Assuming that types are sorted by name in ascending order.
> > > */
> >
> > [...]
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 09/13] bpf: Optimize the performance of find_bpffs_btf_enums
2025-12-19 0:01 ` Andrii Nakryiko
@ 2025-12-19 5:41 ` Donglin Peng
2025-12-20 14:27 ` Donglin Peng
0 siblings, 1 reply; 72+ messages in thread
From: Donglin Peng @ 2025-12-19 5:41 UTC (permalink / raw)
To: Andrii Nakryiko
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Fri, Dec 19, 2025 at 8:02 AM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
> >
> > From: pengdonglin <pengdonglin@xiaomi.com>
> >
> > Currently, vmlinux BTF is unconditionally sorted during
> > the build phase. The function btf_find_by_name_kind
> > executes the binary search branch, so find_bpffs_btf_enums
> > can be optimized by using btf_find_by_name_kind.
> >
> > Cc: Eduard Zingerman <eddyz87@gmail.com>
> > Cc: Alexei Starovoitov <ast@kernel.org>
> > Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > Cc: Alan Maguire <alan.maguire@oracle.com>
> > Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> > Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> > Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> > Acked-by: Eduard Zingerman <eddyz87@gmail.com>
> > ---
> > kernel/bpf/inode.c | 42 +++++++++++++++++++-----------------------
> > 1 file changed, 19 insertions(+), 23 deletions(-)
> >
> > diff --git a/kernel/bpf/inode.c b/kernel/bpf/inode.c
> > index 9f866a010dad..050fde1cf211 100644
> > --- a/kernel/bpf/inode.c
> > +++ b/kernel/bpf/inode.c
> > @@ -600,10 +600,18 @@ struct bpffs_btf_enums {
> >
> > static int find_bpffs_btf_enums(struct bpffs_btf_enums *info)
> > {
> > + struct {
> > + const struct btf_type **type;
> > + const char *name;
> > + } btf_enums[] = {
> > + {&info->cmd_t, "bpf_cmd"},
> > + {&info->map_t, "bpf_map_type"},
> > + {&info->prog_t, "bpf_prog_type"},
> > + {&info->attach_t, "bpf_attach_type"},
> > + };
> > const struct btf *btf;
> > const struct btf_type *t;
> > - const char *name;
> > - int i, n;
> > + int i, id;
> >
> > memset(info, 0, sizeof(*info));
> >
> > @@ -615,30 +623,18 @@ static int find_bpffs_btf_enums(struct bpffs_btf_enums *info)
> >
> > info->btf = btf;
> >
> > - for (i = 1, n = btf_nr_types(btf); i < n; i++) {
> > - t = btf_type_by_id(btf, i);
> > - if (!btf_type_is_enum(t))
> > - continue;
> > + for (i = 0; i < ARRAY_SIZE(btf_enums); i++) {
> > + id = btf_find_by_name_kind(btf, btf_enums[i].name,
> > + BTF_KIND_ENUM);
> > + if (id < 0)
> > + goto out;
> >
> > - name = btf_name_by_offset(btf, t->name_off);
> > - if (!name)
> > - continue;
>
> return -ESRCH, why goto at all?
>
> > -
> > - if (strcmp(name, "bpf_cmd") == 0)
> > - info->cmd_t = t;
> > - else if (strcmp(name, "bpf_map_type") == 0)
> > - info->map_t = t;
> > - else if (strcmp(name, "bpf_prog_type") == 0)
> > - info->prog_t = t;
> > - else if (strcmp(name, "bpf_attach_type") == 0)
> > - info->attach_t = t;
> > - else
> > - continue;
> > -
> > - if (info->cmd_t && info->map_t && info->prog_t && info->attach_t)
> > - return 0;
> > + t = btf_type_by_id(btf, id);
> > + *btf_enums[i].type = t;
>
> nit: drop t local variable, just assign directly:
>
> *btf_enums[i].type = btf_type_by_id(btf, id);
Thanks, I will do it.
>
> > }
> >
> > + return 0;
> > +out:
> > return -ESRCH;
> > }
> >
> > --
> > 2.34.1
> >
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 10/13] libbpf: Optimize the performance of determine_ptr_size
2025-12-19 0:03 ` Andrii Nakryiko
@ 2025-12-19 5:42 ` Donglin Peng
0 siblings, 0 replies; 72+ messages in thread
From: Donglin Peng @ 2025-12-19 5:42 UTC (permalink / raw)
To: Andrii Nakryiko
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Fri, Dec 19, 2025 at 8:03 AM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
> >
> > From: pengdonglin <pengdonglin@xiaomi.com>
> >
> > Leverage the performance improvement of btf__find_by_name_kind() when
> > BTF is sorted. For sorted BTF, the function uses binary search with
> > O(log n) complexity instead of linear search, providing significant
> > performance benefits, especially for large BTF like vmlinux.
> >
> > Cc: Eduard Zingerman <eddyz87@gmail.com>
> > Cc: Alexei Starovoitov <ast@kernel.org>
> > Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > Cc: Alan Maguire <alan.maguire@oracle.com>
> > Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> > Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> > Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> > ---
> > tools/lib/bpf/btf.c | 20 ++++++--------------
> > 1 file changed, 6 insertions(+), 14 deletions(-)
> >
>
> nice and clean
Thank you.
>
> Acked-by: Andrii Nakryiko <andrii@kernel.org>
>
> > diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> > index c63d46b7d74b..b5b0898d033d 100644
> > --- a/tools/lib/bpf/btf.c
> > +++ b/tools/lib/bpf/btf.c
> > @@ -659,29 +659,21 @@ static int determine_ptr_size(const struct btf *btf)
> > "int long unsigned",
> > };
> > const struct btf_type *t;
> > - const char *name;
> > - int i, j, n;
> > + int i, id;
> >
> > if (btf->base_btf && btf->base_btf->ptr_sz > 0)
> > return btf->base_btf->ptr_sz;
> >
> > - n = btf__type_cnt(btf);
> > - for (i = 1; i < n; i++) {
> > - t = btf__type_by_id(btf, i);
> > - if (!btf_is_int(t))
> > + for (i = 0; i < ARRAY_SIZE(long_aliases); i++) {
> > + id = btf__find_by_name_kind(btf, long_aliases[i], BTF_KIND_INT);
> > + if (id < 0)
> > continue;
> >
> > + t = btf__type_by_id(btf, id);
> > if (t->size != 4 && t->size != 8)
> > continue;
> >
> > - name = btf__name_by_offset(btf, t->name_off);
> > - if (!name)
> > - continue;
> > -
> > - for (j = 0; j < ARRAY_SIZE(long_aliases); j++) {
> > - if (strcmp(name, long_aliases[j]) == 0)
> > - return t->size;
> > - }
> > + return t->size;
> > }
> >
> > return -1;
> > --
> > 2.34.1
> >
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 11/13] libbpf: Add btf_is_sorted and btf_sorted_start_id helpers to refactor the code
2025-12-18 22:33 ` Eduard Zingerman
@ 2025-12-19 5:49 ` Donglin Peng
0 siblings, 0 replies; 72+ messages in thread
From: Donglin Peng @ 2025-12-19 5:49 UTC (permalink / raw)
To: Eduard Zingerman
Cc: ast, andrii.nakryiko, zhangxiaoqin, ihor.solodrai, linux-kernel,
bpf, pengdonglin, Alan Maguire
On Fri, Dec 19, 2025 at 6:33 AM Eduard Zingerman <eddyz87@gmail.com> wrote:
>
> On Thu, 2025-12-18 at 19:30 +0800, Donglin Peng wrote:
> > From: pengdonglin <pengdonglin@xiaomi.com>
> >
> > Introduce two new helper functions to clarify the code and no
> > functional changes are introduced.
> >
> > Cc: Eduard Zingerman <eddyz87@gmail.com>
> > Cc: Alexei Starovoitov <ast@kernel.org>
> > Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > Cc: Alan Maguire <alan.maguire@oracle.com>
> > Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> > Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> > Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> > ---
> > tools/lib/bpf/btf.c | 14 ++++++++++++--
> > tools/lib/bpf/libbpf_internal.h | 2 ++
> > 2 files changed, 14 insertions(+), 2 deletions(-)
> >
> > diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> > index b5b0898d033d..571b72bd90b5 100644
> > --- a/tools/lib/bpf/btf.c
> > +++ b/tools/lib/bpf/btf.c
> > @@ -626,6 +626,16 @@ const struct btf *btf__base_btf(const struct btf *btf)
> > return btf->base_btf;
> > }
> >
> > +int btf_sorted_start_id(const struct btf *btf)
> > +{
> > + return btf->sorted_start_id;
> > +}
>
> Having this function declared differently in kernel and in libbpf is a
> bit confusing. Is it needed in libbpf at all?
Thanks. Keeping it consistent between libbpf and kernel side.
Will remove in the next version.
>
> > +bool btf_is_sorted(const struct btf *btf)
> > +{
> > + return btf->sorted_start_id > 0;
> > +}
> > +
>
> Please squash this with the first btf_find_by_name_kind() change.
Yes.
>
> > /* internal helper returning non-const pointer to a type */
> > struct btf_type *btf_type_by_id(const struct btf *btf, __u32 type_id)
> > {
> > @@ -976,11 +986,11 @@ static __s32 btf_find_by_name_kind(const struct btf *btf, int start_id,
> > if (kind == BTF_KIND_UNKN || strcmp(type_name, "void") == 0)
> > return 0;
> >
> > - if (btf->sorted_start_id > 0 && type_name[0]) {
> > + if (btf_is_sorted(btf) && type_name[0]) {
> > __s32 end_id = btf__type_cnt(btf) - 1;
> >
> > /* skip anonymous types */
> > - start_id = max(start_id, btf->sorted_start_id);
> > + start_id = max(start_id, btf_sorted_start_id(btf));
> > idx = btf_find_by_name_bsearch(btf, type_name, start_id, end_id);
> > if (unlikely(idx < 0))
> > return libbpf_err(-ENOENT);
> > diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_internal.h
> > index fc59b21b51b5..95e6848396b4 100644
> > --- a/tools/lib/bpf/libbpf_internal.h
> > +++ b/tools/lib/bpf/libbpf_internal.h
> > @@ -250,6 +250,8 @@ const struct btf_type *skip_mods_and_typedefs(const struct btf *btf, __u32 id, _
> > const struct btf_header *btf_header(const struct btf *btf);
> > void btf_set_base_btf(struct btf *btf, const struct btf *base_btf);
> > int btf_relocate(struct btf *btf, const struct btf *base_btf, __u32 **id_map);
> > +int btf_sorted_start_id(const struct btf *btf);
> > +bool btf_is_sorted(const struct btf *btf);
> >
> > static inline enum btf_func_linkage btf_func_linkage(const struct btf_type *t)
> > {
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 11/13] libbpf: Add btf_is_sorted and btf_sorted_start_id helpers to refactor the code
2025-12-19 0:05 ` Andrii Nakryiko
@ 2025-12-19 5:51 ` Donglin Peng
2025-12-19 17:35 ` Andrii Nakryiko
0 siblings, 1 reply; 72+ messages in thread
From: Donglin Peng @ 2025-12-19 5:51 UTC (permalink / raw)
To: Andrii Nakryiko
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Fri, Dec 19, 2025 at 8:05 AM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
> >
> > From: pengdonglin <pengdonglin@xiaomi.com>
> >
> > Introduce two new helper functions to clarify the code and no
> > functional changes are introduced.
> >
> > Cc: Eduard Zingerman <eddyz87@gmail.com>
> > Cc: Alexei Starovoitov <ast@kernel.org>
> > Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > Cc: Alan Maguire <alan.maguire@oracle.com>
> > Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> > Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> > Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> > ---
> > tools/lib/bpf/btf.c | 14 ++++++++++++--
> > tools/lib/bpf/libbpf_internal.h | 2 ++
> > 2 files changed, 14 insertions(+), 2 deletions(-)
> >
>
> It just adds more functions to jump to and check what it's doing. I
> don't think this adds much value, just drop this patch
Could adding the __always_inline be acceptable?
>
>
> > diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> > index b5b0898d033d..571b72bd90b5 100644
> > --- a/tools/lib/bpf/btf.c
> > +++ b/tools/lib/bpf/btf.c
> > @@ -626,6 +626,16 @@ const struct btf *btf__base_btf(const struct btf *btf)
> > return btf->base_btf;
> > }
> >
> > +int btf_sorted_start_id(const struct btf *btf)
> > +{
> > + return btf->sorted_start_id;
> > +}
> > +
> > +bool btf_is_sorted(const struct btf *btf)
> > +{
> > + return btf->sorted_start_id > 0;
> > +}
> > +
> > /* internal helper returning non-const pointer to a type */
> > struct btf_type *btf_type_by_id(const struct btf *btf, __u32 type_id)
> > {
> > @@ -976,11 +986,11 @@ static __s32 btf_find_by_name_kind(const struct btf *btf, int start_id,
> > if (kind == BTF_KIND_UNKN || strcmp(type_name, "void") == 0)
> > return 0;
> >
> > - if (btf->sorted_start_id > 0 && type_name[0]) {
> > + if (btf_is_sorted(btf) && type_name[0]) {
> > __s32 end_id = btf__type_cnt(btf) - 1;
> >
> > /* skip anonymous types */
> > - start_id = max(start_id, btf->sorted_start_id);
> > + start_id = max(start_id, btf_sorted_start_id(btf));
> > idx = btf_find_by_name_bsearch(btf, type_name, start_id, end_id);
> > if (unlikely(idx < 0))
> > return libbpf_err(-ENOENT);
> > diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_internal.h
> > index fc59b21b51b5..95e6848396b4 100644
> > --- a/tools/lib/bpf/libbpf_internal.h
> > +++ b/tools/lib/bpf/libbpf_internal.h
> > @@ -250,6 +250,8 @@ const struct btf_type *skip_mods_and_typedefs(const struct btf *btf, __u32 id, _
> > const struct btf_header *btf_header(const struct btf *btf);
> > void btf_set_base_btf(struct btf *btf, const struct btf *base_btf);
> > int btf_relocate(struct btf *btf, const struct btf *base_btf, __u32 **id_map);
> > +int btf_sorted_start_id(const struct btf *btf);
> > +bool btf_is_sorted(const struct btf *btf);
> >
> > static inline enum btf_func_linkage btf_func_linkage(const struct btf_type *t)
> > {
> > --
> > 2.34.1
> >
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 01/13] libbpf: Add BTF permutation support for type reordering
2025-12-19 3:14 ` Donglin Peng
2025-12-19 3:16 ` Donglin Peng
@ 2025-12-19 17:07 ` Andrii Nakryiko
2025-12-20 8:39 ` Donglin Peng
1 sibling, 1 reply; 72+ messages in thread
From: Andrii Nakryiko @ 2025-12-19 17:07 UTC (permalink / raw)
To: Donglin Peng
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Thu, Dec 18, 2025 at 7:15 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
>
> On Fri, Dec 19, 2025 at 7:02 AM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
> >
> > On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
> > >
> > > From: pengdonglin <pengdonglin@xiaomi.com>
> > >
> > > Introduce btf__permute() API to allow in-place rearrangement of BTF types.
> > > This function reorganizes BTF type order according to a provided array of
> > > type IDs, updating all type references to maintain consistency.
> > >
> > > Cc: Eduard Zingerman <eddyz87@gmail.com>
> > > Cc: Alexei Starovoitov <ast@kernel.org>
> > > Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > > Cc: Alan Maguire <alan.maguire@oracle.com>
> > > Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> > > Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> > > Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> > > Acked-by: Eduard Zingerman <eddyz87@gmail.com>
> > > ---
> > > tools/lib/bpf/btf.c | 119 +++++++++++++++++++++++++++++++++++++++
> > > tools/lib/bpf/btf.h | 36 ++++++++++++
> > > tools/lib/bpf/libbpf.map | 1 +
> > > 3 files changed, 156 insertions(+)
> > >
[...]
> > > +/**
> > > + * @brief **btf__permute()** performs in-place BTF type rearrangement
> > > + * @param btf BTF object to permute
> > > + * @param id_map Array mapping original type IDs to new IDs
> > > + * @param id_map_cnt Number of elements in @id_map
> > > + * @param opts Optional parameters for BTF extension updates
> > > + * @return 0 on success, negative error code on failure
> > > + *
> > > + * **btf__permute()** rearranges BTF types according to the specified ID mapping.
> > > + * The @id_map array defines the new type ID for each original type ID.
> > > + *
> > > + * @id_map must include all types from ID `start_id` to `btf__type_cnt(btf) - 1`.
> > > + * @id_map_cnt should be `btf__type_cnt(btf) - start_id`
> > > + * The mapping is defined as: `id_map[original_id - start_id] = new_id`
> >
> > Would you mind paying attention to the feedback I left in [0]? Thank you.
>
> Apologies for the delayed response, I would like to hear if someone has a
> different idea.
Delayed response?.. You ignored my feedback and never even replied to
it. And then posted a new revision two days later, while still not
taking the feedback into account. This is not a delayed response, it's
ignoring the feedback. You don't have to agree with all the feedback,
but you have to respond to the feedback you disagree with and provide
your arguments, not just silently disregard it.
>
> >
> > The contract should be id_map[original_id] = new_id for base BTF and
> > id_map[original_id - btf__type_cnt(base_btf)] = new_id for split BTF.
> > Special BTF type #0 (VOID) is considered to be part of base BTF,
> > having id_map[0] = 0 is easy to check and enforce. And then it leaves
> > us with a simple and logical rule for id_map. For split BTF we make
> > necessary type ID shifts to avoid tons of wasted memory. But for base
> > BTF there is no need to shift anything. So mapping the original type
> > #X to #Y is id_map[X] = Y. Literally, "map X to Y", as simple as that.
> >
> > [0] https://lore.kernel.org/bpf/CAEf4BzY_k721TBfRSUeq5mB-7fgJhVKCeXVKO-W2EjQ0aS9AgA@mail.gmail.com/
>
> Thanks. I implemented the approach in v6, but it had inconsistent internal
> details for base and split BTF. It seems we prioritize external contract
> consistency over internal inconsistencies, so I’ll revert to the v6 approach
> and refine it for clarity.
Yes, we always prioritize external contract consistency, of course!
You are overpivoting on *internal implementation detail* of base BTF's
start_id being set to 1, which is convenient in some other places due
to type_offs shifted by one mapping due to &btf_void special handling.
We can always change that, if we wanted, but this shouldn't spill into
public API though. But conceptually BTF types start at type #0, which
is defined to be VOID and is not user controlled.
This is not much of a complication or inconsistency:
type_shift = base_btf ? btf__type_cnt(base_btf) : 0;
id_map[type_id - type_shift] = ...
>
> >
> > > + *
> > > + * For base BTF, its `start_id` is fixed to 1, i.e. the VOID type can
> > > + * not be redefined or remapped and its ID is fixed to 0.
> > > + *
> > > + * For split BTF, its `start_id` can be retrieved by calling
> > > + * `btf__type_cnt(btf__base_btf(btf))`.
> > > + *
> > > + * On error, returns negative error code and sets errno:
> > > + * - `-EINVAL`: Invalid parameters or ID mapping (duplicates, out-of-range)
> > > + * - `-ENOMEM`: Memory allocation failure
> > > + */
> > > +LIBBPF_API int btf__permute(struct btf *btf, __u32 *id_map, __u32 id_map_cnt,
> > > + const struct btf_permute_opts *opts);
> > > +
> > > struct btf_dump;
> > >
> > > struct btf_dump_opts {
> > > diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map
> > > index 84fb90a016c9..d18fbcea7578 100644
> > > --- a/tools/lib/bpf/libbpf.map
> > > +++ b/tools/lib/bpf/libbpf.map
> > > @@ -453,4 +453,5 @@ LIBBPF_1.7.0 {
> > > bpf_map__exclusive_program;
> > > bpf_prog_assoc_struct_ops;
> > > bpf_program__assoc_struct_ops;
> > > + btf__permute;
> > > } LIBBPF_1.6.0;
> > > --
> > > 2.34.1
> > >
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 03/13] tools/resolve_btfids: Support BTF sorting feature
2025-12-19 3:42 ` Donglin Peng
@ 2025-12-19 17:12 ` Andrii Nakryiko
2025-12-20 8:44 ` Donglin Peng
0 siblings, 1 reply; 72+ messages in thread
From: Andrii Nakryiko @ 2025-12-19 17:12 UTC (permalink / raw)
To: Donglin Peng
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Thu, Dec 18, 2025 at 7:42 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
>
> On Fri, Dec 19, 2025 at 7:09 AM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
> >
> > On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
> > >
> > > From: pengdonglin <pengdonglin@xiaomi.com>
> > >
> > > This introduces a new BTF sorting phase that specifically sorts
> > > BTF types by name in ascending order, so that the binary search
> > > can be used to look up types.
> > >
> > > Cc: Eduard Zingerman <eddyz87@gmail.com>
> > > Cc: Alexei Starovoitov <ast@kernel.org>
> > > Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > > Cc: Alan Maguire <alan.maguire@oracle.com>
> > > Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> > > Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> > > Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> >
> > Signed-off-by is supposed to use properly spelled full name, this
> > should be "Donglin Peng", right?
>
> Sorry, I will fix this in the next version. The reason is that our
> company's Gerrit only accepts "pengdonglin" and does not
> accept "Donglin Peng".
>
> >
> > > Acked-by: Eduard Zingerman <eddyz87@gmail.com>
> > > ---
> > > tools/bpf/resolve_btfids/main.c | 68 +++++++++++++++++++++++++++++++++
> > > 1 file changed, 68 insertions(+)
> > >
> > > diff --git a/tools/bpf/resolve_btfids/main.c b/tools/bpf/resolve_btfids/main.c
> > > index 3e88dc862d87..659de35748ec 100644
> > > --- a/tools/bpf/resolve_btfids/main.c
> > > +++ b/tools/bpf/resolve_btfids/main.c
> > > @@ -848,6 +848,71 @@ static int dump_raw_btf(struct btf *btf, const char *out_path)
> > > return 0;
> > > }
> > >
> > > +/*
> > > + * Sort types by name in ascending order resulting in all
> > > + * anonymous types being placed before named types.
> > > + */
> > > +static int cmp_type_names(const void *a, const void *b, void *priv)
> > > +{
> > > + struct btf *btf = (struct btf *)priv;
> > > + const struct btf_type *ta = btf__type_by_id(btf, *(__u32 *)a);
> > > + const struct btf_type *tb = btf__type_by_id(btf, *(__u32 *)b);
> > > + const char *na, *nb;
> > > +
> > > + na = btf__str_by_offset(btf, ta->name_off);
> > > + nb = btf__str_by_offset(btf, tb->name_off);
> > > + return strcmp(na, nb);
> > > +}
> > > +
> > > +static int sort_btf_by_name(struct btf *btf)
> > > +{
> > > + __u32 *permute_ids = NULL, *id_map = NULL;
> > > + int nr_types, i, err = 0;
> > > + __u32 start_id = 1, id;
> > > +
> > > + if (btf__base_btf(btf))
> > > + start_id = btf__type_cnt(btf__base_btf(btf));
> > > + nr_types = btf__type_cnt(btf) - start_id;
> > > + if (nr_types < 2)
> > > + goto out;
> >
> > why this check, will anything break if you don't do it?
>
> Because I think that if there are zero or only one type,
> there is no need to sort.
There is also no need to special-case and add more checks just for
these corner cases. Keep it simple.
>
> >
> > > +
> > > + permute_ids = calloc(nr_types, sizeof(*permute_ids));
> > > + if (!permute_ids) {
> > > + err = -ENOMEM;
> > > + goto out;
> > > + }
> > > +
> > > + id_map = calloc(nr_types, sizeof(*id_map));
> > > + if (!id_map) {
> > > + err = -ENOMEM;
> > > + goto out;
> > > + }
> > > +
> > > + for (i = 0, id = start_id; i < nr_types; i++, id++)
> > > + permute_ids[i] = id;
> > > +
> > > + qsort_r(permute_ids, nr_types, sizeof(*permute_ids), cmp_type_names, btf);
> > > +
> > > + for (i = 0; i < nr_types; i++) {
> > > + id = permute_ids[i] - start_id;
> > > + id_map[id] = i + start_id;
> > > + }
> > > +
> > > + err = btf__permute(btf, id_map, nr_types, NULL);
> > > + if (err)
> > > + pr_err("FAILED: btf permute: %s\n", strerror(-err));
> > > +
> > > +out:
> > > + free(permute_ids);
> > > + free(id_map);
> > > + return err;
> > > +}
> > > +
> > > +static int btf2btf(struct object *obj)
> >
> > what's the point of having this function?
>
> Sorting BTF is a type of `btf2btf` process. There may be other
> types of `btf2btf` processes, which could be grouped together
> here. If we currently don't care about these other processes,
> I will retain only `sort_btf_by_name` in the next version.
Let's have sort as is, when necessary we can refactor this into
logical steps further (or perhaps sorting will be its own logical
step, I don't know, but let's not over design it just yet)
>
> >
> > > +{
> > > + return sort_btf_by_name(obj->btf);
> > > +}
> > > +
> > > static inline int make_out_path(char *buf, u32 buf_sz, const char *in_path, const char *suffix)
> > > {
> > > int len = snprintf(buf, buf_sz, "%s%s", in_path, suffix);
> > > @@ -906,6 +971,9 @@ int main(int argc, const char **argv)
> > > if (load_btf(&obj))
> > > goto out;
> > >
> > > + if (btf2btf(&obj))
> > > + goto out;
> > > +
> > > if (elf_collect(&obj))
> > > goto out;
> > >
> > > --
> > > 2.34.1
> > >
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 04/13] libbpf: Optimize type lookup with binary search for sorted BTF
2025-12-19 2:53 ` Donglin Peng
@ 2025-12-19 17:28 ` Andrii Nakryiko
2025-12-20 9:38 ` Donglin Peng
0 siblings, 1 reply; 72+ messages in thread
From: Andrii Nakryiko @ 2025-12-19 17:28 UTC (permalink / raw)
To: Donglin Peng
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Thu, Dec 18, 2025 at 6:53 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
>
> On Fri, Dec 19, 2025 at 7:29 AM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
> >
> > On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
> > >
> > > From: pengdonglin <pengdonglin@xiaomi.com>
> > >
> > > This patch introduces binary search optimization for BTF type lookups
> > > when the BTF instance contains sorted types.
> > >
> > > The optimization significantly improves performance when searching for
> > > types in large BTF instances with sorted types. For unsorted BTF, the
> > > implementation falls back to the original linear search.
> > >
> > > Cc: Eduard Zingerman <eddyz87@gmail.com>
> > > Cc: Alexei Starovoitov <ast@kernel.org>
> > > Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > > Cc: Alan Maguire <alan.maguire@oracle.com>
> > > Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> > > Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> > > Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> > > ---
> > > tools/lib/bpf/btf.c | 103 ++++++++++++++++++++++++++++++++++----------
> > > 1 file changed, 80 insertions(+), 23 deletions(-)
> > >
> >
> > [...]
> >
> > > + l = start_id;
> > > + r = end_id;
> > > + while (l <= r) {
> > > + m = l + (r - l) / 2;
> > > + t = btf_type_by_id(btf, m);
> > > + tname = btf__str_by_offset(btf, t->name_off);
> > > + ret = strcmp(tname, name);
> > > + if (ret < 0) {
> > > + l = m + 1;
> > > + } else {
> > > + if (ret == 0)
> > > + lmost = m;
> > > + r = m - 1;
> > > + }
> > > }
> >
> > this differs from what we discussed in [0], you said you'll use that
> > approach. Can you please elaborate on why you didn't?
> >
> > [0] https://lore.kernel.org/bpf/CAEf4Bzb3Eu0J83O=Y4KA-LkzBMjtx7cbonxPzkiduzZ1Pedajg@mail.gmail.com/
>
> Yes. As mentioned in the v8 changelog [1], the binary search approach
> you referenced was implemented in versions v6 and v7 [2]. However,
> testing revealed a slight performance regression. The root cause was
> an extra strcmp operation introduced in v7, as discussed in [3]. Therefore,
> in v8, I reverted to the approach from v5 [4] and refactored it for clarity.
If you keep oscillating like that this patch set will never land. 4%
(500us) gain on artificial and unrealistic micro-benchmark is
meaningless and irrelevant, you are just adding more work for yourself
and for reviewers by constantly changing your implementation between
revisions for no good reason.
>
> Benchmark results show that v8 achieves a 4.2% performance improvement
> over v7. If we don't care the performance gain, I will revert to the approach
> in v7 in the next version.
>
> [1] https://lore.kernel.org/bpf/20251126085025.784288-1-dolinux.peng@gmail.com/
> [2] https://lore.kernel.org/all/20251119031531.1817099-1-dolinux.peng@gmail.com/
> [3] https://lore.kernel.org/all/CAEf4BzaqEPD46LddJHO1-k5KPGyVWf6d=duDAxG1q=jykJkMBg@mail.gmail.com/
> [4] https://lore.kernel.org/all/20251106131956.1222864-4-dolinux.peng@gmail.com/
>
> >
> > >
> > > - return libbpf_err(-ENOENT);
> > > + return lmost;
> > > }
> > >
> > > static __s32 btf_find_by_name_kind(const struct btf *btf, int start_id,
> > > const char *type_name, __u32 kind)
> >
> > kind is defined as u32 but you expect caller to pass -1 to ignore the
> > kind. Use int here.
>
> Thanks, I will fix it.
>
> >
> > > {
> > > - __u32 i, nr_types = btf__type_cnt(btf);
> > > + const struct btf_type *t;
> > > + const char *tname;
> > > + __s32 idx;
> > > +
> > > + if (start_id < btf->start_id) {
> > > + idx = btf_find_by_name_kind(btf->base_btf, start_id,
> > > + type_name, kind);
> > > + if (idx >= 0)
> > > + return idx;
> > > + start_id = btf->start_id;
> > > + }
> > >
> > > - if (kind == BTF_KIND_UNKN || !strcmp(type_name, "void"))
> > > + if (kind == BTF_KIND_UNKN || strcmp(type_name, "void") == 0)
> > > return 0;
> > >
> > > - for (i = start_id; i < nr_types; i++) {
> > > - const struct btf_type *t = btf__type_by_id(btf, i);
> > > - const char *name;
> > > + if (btf->sorted_start_id > 0 && type_name[0]) {
> > > + __s32 end_id = btf__type_cnt(btf) - 1;
> > > +
> > > + /* skip anonymous types */
> > > + start_id = max(start_id, btf->sorted_start_id);
> >
> > can sorted_start_id ever be smaller than start_id?
> >
> > > + idx = btf_find_by_name_bsearch(btf, type_name, start_id, end_id);
> >
> > is there ever a time when btf_find_by_name_bsearch() will work with
> > different start_id and end_id? why is this not done inside the
> > btf_find_by_name_bsearch()?
>
> Because the start_id could be specified by the caller.
Right, start_id has to be passed in. But end_id is always the same, so
maybe determine it internally instead? And let's not return -ENOENT
from btf_find_by_name_bsearch(), as I mentioned before, it would be
more streamlined if you return btf__type_cnt(btf) if search failed.
>
> >
> > > + if (unlikely(idx < 0))
> > > + return libbpf_err(-ENOENT);
> >
> > pass through error returned from btf_find_by_name_bsearch(), why redefining it?
>
> Thanks, I will fix it.
>
see above, by returning btf__type_cnt() you won't even have this error
handling, you'll just go through normal loop checking for a match and
won't find anything, returning -ENOENT then.
> >
> > > +
> > > + if (unlikely(kind == -1))
> > > + return idx;
> > > +
> > > + t = btf_type_by_id(btf, idx);
> > > + if (likely(BTF_INFO_KIND(t->info) == kind))
> >
> > use btf_kind(), but this whole extra check is just unnecessary, this
>
> Thanks, I will do it.
>
> > should be done in the loop below. We talked about all this already,
> > why do I feel like I'm being ignored?..
>
> Sorry for the confusion, and absolutely not ignoring you.
>
If you decide to change implementation due to some unforeseen factors
(like concern about 4% microbenchmark improvement), it would be
helpful for you to call this out in a reply to the original
discussion. A line somewhere in the cover letter changelog is way too
easy to miss and that doesn't give me an opportunity to stop you
before you go and produce another revision that I'll then be
rejecting.
> >
> > > + return idx;
> >
> > drop all these likely and unlikely micro optimizations, please
>
> Thanks, I will do it.
>
> >
> >
> > > +
> > > + for (idx++; idx <= end_id; idx++) {
> > > + t = btf__type_by_id(btf, idx);
> > > + tname = btf__str_by_offset(btf, t->name_off);
> > > + if (strcmp(tname, type_name) != 0)
> > > + return libbpf_err(-ENOENT);
> > > + if (btf_kind(t) == kind)
> > > + return idx;
> > > + }
> > > + } else {
> > > + __u32 i, total;
> > >
> > > - if (btf_kind(t) != kind)
> > > - continue;
> > > - name = btf__name_by_offset(btf, t->name_off);
> > > - if (name && !strcmp(type_name, name))
> > > - return i;
> > > + total = btf__type_cnt(btf);
> > > + for (i = start_id; i < total; i++) {
> > > + t = btf_type_by_id(btf, i);
> > > + if (kind != -1 && btf_kind(t) != kind)
> >
> > nit: kind < 0, no need to hard-code -1
>
> Good, I will fix it.
>
> >
> > > + continue;
> > > + tname = btf__str_by_offset(btf, t->name_off);
> > > + if (strcmp(tname, type_name) == 0)
> > > + return i;
> > > + }
> > > }
> > >
> > > return libbpf_err(-ENOENT);
> > > }
> > >
> >
> > [...]
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 05/13] libbpf: Verify BTF Sorting
2025-12-19 5:06 ` Donglin Peng
@ 2025-12-19 17:33 ` Andrii Nakryiko
[not found] ` <CAErzpmsvirekLBRrJYVgmRC0YKWCbo7OyRQXgNYrk83aF-Wz2Q@mail.gmail.com>
0 siblings, 1 reply; 72+ messages in thread
From: Andrii Nakryiko @ 2025-12-19 17:33 UTC (permalink / raw)
To: Donglin Peng
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Thu, Dec 18, 2025 at 9:06 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
>
> On Fri, Dec 19, 2025 at 7:44 AM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
> >
> > On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
> > >
> > > From: pengdonglin <pengdonglin@xiaomi.com>
> > >
> >
> > typo in subject: "Sorting" -> "sorting", it looks weird capitalized like that
>
> Thanks, I will do it.
>
> >
> > > This patch checks whether the BTF is sorted by name in ascending
> > > order. If sorted, binary search will be used when looking up types.
> > >
> > > Cc: Eduard Zingerman <eddyz87@gmail.com>
> > > Cc: Alexei Starovoitov <ast@kernel.org>
> > > Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > > Cc: Alan Maguire <alan.maguire@oracle.com>
> > > Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> > > Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> > > Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> > > ---
> > > tools/lib/bpf/btf.c | 41 +++++++++++++++++++++++++++++++++++++++++
> > > 1 file changed, 41 insertions(+)
> > >
> > > diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> > > index 2facb57d7e5f..c63d46b7d74b 100644
> > > --- a/tools/lib/bpf/btf.c
> > > +++ b/tools/lib/bpf/btf.c
> > > @@ -899,6 +899,46 @@ int btf__resolve_type(const struct btf *btf, __u32 type_id)
> > > return type_id;
> > > }
> > >
> > > +/*
> > > + * Assuming that types are sorted by name in ascending order.
> > > + */
> >
> > Unnecessary comment, and no, btf_compare_type_names() itself makes no
> > such assumption, it just compares two provided types by name. Drop the
> > comment, please.
> >
> > > +static int btf_compare_type_names(__u32 *a, __u32 *b, const struct btf *btf)
> > > +{
> > > + struct btf_type *ta = btf_type_by_id(btf, *a);
> > > + struct btf_type *tb = btf_type_by_id(btf, *b);
> > > + const char *na, *nb;
> > > +
> > > + na = btf__str_by_offset(btf, ta->name_off);
> > > + nb = btf__str_by_offset(btf, tb->name_off);
> > > + return strcmp(na, nb);
> > > +}
> >
> > you use this function only in one place, there is no real point having
> > it, especially that it uses **a pointer to type ID** as an
> > interface... just inline its logic in that one loop below
> >
> > > +
> > > +static void btf_check_sorted(struct btf *btf)
> > > +{
> > > + const struct btf_type *t;
> > > + __u32 i, k, n;
> > > + __u32 sorted_start_id;
> > > +
> > > + if (btf->nr_types < 2)
> > > + return;
> >
> > why special casing? does it not work with nr_types = 0 or nr_types = 1?
>
> No. I just think it doesn't make any sense to check the sorting
> of BTF with zero or only one type.
>
Look, I don't know how to emphasize this enough. Any special case like
this is nothing good, it adds more cases to consider and raises
questions why generic case code doesn't handle such special cases. It
implies that generic case handling might have some unhandled corner
case that we are trying to short-circuit with early exits like this.
It just basically means we don't trust our code, in a sense. It's just
unnecessary and thus sloppy. Don't do this unnecessarily.
> >
> > > +
> > > + sorted_start_id = 0;
> >
> > nit: initialize in declaration
>
> Thanks, I will do it.
>
> >
> >
> > > + n = btf__type_cnt(btf);
> > > + for (i = btf->start_id; i < n; i++) {
> > > + k = i + 1;
> > > + if (k < n && btf_compare_type_names(&i, &k, btf) > 0)
> > > + return;
> > > + if (sorted_start_id == 0) {
> > > + t = btf_type_by_id(btf, i);
> > > + if (t->name_off)
> >
> > I'd check actual string, not name_off. Technically, you can have empty
> > string with non-zero name_off, so why assume anything here?
>
> Thanks, I will do it.
>
> >
> > > + sorted_start_id = i;
> > > + }
> > > + }
> > > +
> > > + if (sorted_start_id)
> > > + btf->sorted_start_id = sorted_start_id;
> >
> > You actually made code more complicated by extracting that
> > btf_compare_type_names(). Compare to:
> >
> > n = btf__type_cnt(btf);
> > btf->sorted_start_id = 0;
> > for (i = btf->start_id + 1; i < n; i++) {
> > struct btf_type *t1 = btf_type_by_id(btf, i - 1);
> > struct btf_type *t2 = btf_type_by_id(btf, i);
> > const char *n1 = btf__str_by_offset(btf, t1->name_off);
> > const char *n2 = btf__str_by_offset(btf, t2->name_off);
> >
> > if (strcmp(n1, n2) > 0)
> > return;
> > if (btf->sorted_start_id == 0 && n1[0] != '\0')
> > btf->sorted_start_id = i - 1;
> > }
>
> Thanks. I believe we shouldn't directly assign a value to
> `btf->sorted_start_id` within the for loop, because
> `btf->sorted_start_id` might be non-zero even when the
> BTF isn't sorted.
Ah, right, we'd need to reset btf->sorted_start_id to zero in that
strcmp(n1, n2) > 0 branch. Using btf->sorted_start_id directly is not
the main point here, though, feel free to use a local variable, but
don't add unnecessary helper functions which don't do much, but
obscure the logic unnecessarily.
>
> >
> >
> > No extra k<n checks, no extra type_by_id lookups. It's minimalistic
> > and cleaner. And if it so happens that we get single type BTF that is
> > technically sorted, it doesn't matter, we always fallback to faster
> > linear search anyways.
> >
> > Keep it simple.
>
> Thank you. I will adopt this method in the next version.
>
> >
> > > +}
> > > +
> > > static __s32 btf_find_by_name_bsearch(const struct btf *btf, const char *name,
> > > __s32 start_id, __s32 end_id)
> > > {
> > > @@ -1147,6 +1187,7 @@ static struct btf *btf_new(const void *data, __u32 size, struct btf *base_btf, b
> > > err = err ?: btf_sanity_check(btf);
> > > if (err)
> > > goto done;
> > > + btf_check_sorted(btf);
> > >
> > > done:
> > > if (err) {
> > > --
> > > 2.34.1
> > >
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 11/13] libbpf: Add btf_is_sorted and btf_sorted_start_id helpers to refactor the code
2025-12-19 5:51 ` Donglin Peng
@ 2025-12-19 17:35 ` Andrii Nakryiko
2025-12-20 14:27 ` Donglin Peng
0 siblings, 1 reply; 72+ messages in thread
From: Andrii Nakryiko @ 2025-12-19 17:35 UTC (permalink / raw)
To: Donglin Peng
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Thu, Dec 18, 2025 at 9:51 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
>
> On Fri, Dec 19, 2025 at 8:05 AM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
> >
> > On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
> > >
> > > From: pengdonglin <pengdonglin@xiaomi.com>
> > >
> > > Introduce two new helper functions to clarify the code and no
> > > functional changes are introduced.
> > >
> > > Cc: Eduard Zingerman <eddyz87@gmail.com>
> > > Cc: Alexei Starovoitov <ast@kernel.org>
> > > Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > > Cc: Alan Maguire <alan.maguire@oracle.com>
> > > Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> > > Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> > > Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> > > ---
> > > tools/lib/bpf/btf.c | 14 ++++++++++++--
> > > tools/lib/bpf/libbpf_internal.h | 2 ++
> > > 2 files changed, 14 insertions(+), 2 deletions(-)
> > >
> >
> > It just adds more functions to jump to and check what it's doing. I
> > don't think this adds much value, just drop this patch
>
> Could adding the __always_inline be acceptable?
No! This is not a performance concern, just mental overhead when
reading the code.
>
> >
> >
> > > diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> > > index b5b0898d033d..571b72bd90b5 100644
> > > --- a/tools/lib/bpf/btf.c
> > > +++ b/tools/lib/bpf/btf.c
> > > @@ -626,6 +626,16 @@ const struct btf *btf__base_btf(const struct btf *btf)
> > > return btf->base_btf;
> > > }
> > >
> > > +int btf_sorted_start_id(const struct btf *btf)
> > > +{
> > > + return btf->sorted_start_id;
> > > +}
> > > +
> > > +bool btf_is_sorted(const struct btf *btf)
> > > +{
> > > + return btf->sorted_start_id > 0;
> > > +}
> > > +
> > > /* internal helper returning non-const pointer to a type */
> > > struct btf_type *btf_type_by_id(const struct btf *btf, __u32 type_id)
> > > {
> > > @@ -976,11 +986,11 @@ static __s32 btf_find_by_name_kind(const struct btf *btf, int start_id,
> > > if (kind == BTF_KIND_UNKN || strcmp(type_name, "void") == 0)
> > > return 0;
> > >
> > > - if (btf->sorted_start_id > 0 && type_name[0]) {
> > > + if (btf_is_sorted(btf) && type_name[0]) {
> > > __s32 end_id = btf__type_cnt(btf) - 1;
> > >
> > > /* skip anonymous types */
> > > - start_id = max(start_id, btf->sorted_start_id);
> > > + start_id = max(start_id, btf_sorted_start_id(btf));
> > > idx = btf_find_by_name_bsearch(btf, type_name, start_id, end_id);
> > > if (unlikely(idx < 0))
> > > return libbpf_err(-ENOENT);
> > > diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_internal.h
> > > index fc59b21b51b5..95e6848396b4 100644
> > > --- a/tools/lib/bpf/libbpf_internal.h
> > > +++ b/tools/lib/bpf/libbpf_internal.h
> > > @@ -250,6 +250,8 @@ const struct btf_type *skip_mods_and_typedefs(const struct btf *btf, __u32 id, _
> > > const struct btf_header *btf_header(const struct btf *btf);
> > > void btf_set_base_btf(struct btf *btf, const struct btf *base_btf);
> > > int btf_relocate(struct btf *btf, const struct btf *base_btf, __u32 **id_map);
> > > +int btf_sorted_start_id(const struct btf *btf);
> > > +bool btf_is_sorted(const struct btf *btf);
> > >
> > > static inline enum btf_func_linkage btf_func_linkage(const struct btf_type *t)
> > > {
> > > --
> > > 2.34.1
> > >
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 01/13] libbpf: Add BTF permutation support for type reordering
2025-12-19 17:07 ` Andrii Nakryiko
@ 2025-12-20 8:39 ` Donglin Peng
0 siblings, 0 replies; 72+ messages in thread
From: Donglin Peng @ 2025-12-20 8:39 UTC (permalink / raw)
To: Andrii Nakryiko
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Sat, Dec 20, 2025 at 1:07 AM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Thu, Dec 18, 2025 at 7:15 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
> >
> > On Fri, Dec 19, 2025 at 7:02 AM Andrii Nakryiko
> > <andrii.nakryiko@gmail.com> wrote:
> > >
> > > On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
> > > >
> > > > From: pengdonglin <pengdonglin@xiaomi.com>
> > > >
> > > > Introduce btf__permute() API to allow in-place rearrangement of BTF types.
> > > > This function reorganizes BTF type order according to a provided array of
> > > > type IDs, updating all type references to maintain consistency.
> > > >
> > > > Cc: Eduard Zingerman <eddyz87@gmail.com>
> > > > Cc: Alexei Starovoitov <ast@kernel.org>
> > > > Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > > > Cc: Alan Maguire <alan.maguire@oracle.com>
> > > > Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> > > > Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> > > > Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> > > > Acked-by: Eduard Zingerman <eddyz87@gmail.com>
> > > > ---
> > > > tools/lib/bpf/btf.c | 119 +++++++++++++++++++++++++++++++++++++++
> > > > tools/lib/bpf/btf.h | 36 ++++++++++++
> > > > tools/lib/bpf/libbpf.map | 1 +
> > > > 3 files changed, 156 insertions(+)
> > > >
>
> [...]
>
> > > > +/**
> > > > + * @brief **btf__permute()** performs in-place BTF type rearrangement
> > > > + * @param btf BTF object to permute
> > > > + * @param id_map Array mapping original type IDs to new IDs
> > > > + * @param id_map_cnt Number of elements in @id_map
> > > > + * @param opts Optional parameters for BTF extension updates
> > > > + * @return 0 on success, negative error code on failure
> > > > + *
> > > > + * **btf__permute()** rearranges BTF types according to the specified ID mapping.
> > > > + * The @id_map array defines the new type ID for each original type ID.
> > > > + *
> > > > + * @id_map must include all types from ID `start_id` to `btf__type_cnt(btf) - 1`.
> > > > + * @id_map_cnt should be `btf__type_cnt(btf) - start_id`
> > > > + * The mapping is defined as: `id_map[original_id - start_id] = new_id`
> > >
> > > Would you mind paying attention to the feedback I left in [0]? Thank you.
> >
> > Apologies for the delayed response, I would like to hear if someone has a
> > different idea.
>
> Delayed response?.. You ignored my feedback and never even replied to
> it. And then posted a new revision two days later, while still not
> taking the feedback into account. This is not a delayed response, it's
> ignoring the feedback. You don't have to agree with all the feedback,
> but you have to respond to the feedback you disagree with and provide
> your arguments, not just silently disregard it.
Thank you for the reminder, and I sincerely apologize for
my mistake in handling the feedback. You are absolutely right.
I should not have posted a new revision without first replying
to your comments on the previous version. The correct process,
as outlined in the kernel development documentation and community
norms, is to address all feedback—either by implementing the
suggested changes or by providing a clear explanation if I
disagree.
I appreciate you taking the time to review my work and for
holding me to the community standard. I will ensure this
does not happen again.
>
> >
> > >
> > > The contract should be id_map[original_id] = new_id for base BTF and
> > > id_map[original_id - btf__type_cnt(base_btf)] = new_id for split BTF.
> > > Special BTF type #0 (VOID) is considered to be part of base BTF,
> > > having id_map[0] = 0 is easy to check and enforce. And then it leaves
> > > us with a simple and logical rule for id_map. For split BTF we make
> > > necessary type ID shifts to avoid tons of wasted memory. But for base
> > > BTF there is no need to shift anything. So mapping the original type
> > > #X to #Y is id_map[X] = Y. Literally, "map X to Y", as simple as that.
> > >
> > > [0] https://lore.kernel.org/bpf/CAEf4BzY_k721TBfRSUeq5mB-7fgJhVKCeXVKO-W2EjQ0aS9AgA@mail.gmail.com/
> >
> > Thanks. I implemented the approach in v6, but it had inconsistent internal
> > details for base and split BTF. It seems we prioritize external contract
> > consistency over internal inconsistencies, so I’ll revert to the v6 approach
> > and refine it for clarity.
>
> Yes, we always prioritize external contract consistency, of course!
> You are overpivoting on *internal implementation detail* of base BTF's
> start_id being set to 1, which is convenient in some other places due
> to type_offs shifted by one mapping due to &btf_void special handling.
> We can always change that, if we wanted, but this shouldn't spill into
> public API though. But conceptually BTF types start at type #0, which
> is defined to be VOID and is not user controlled.
Thanks, I understood.
>
>
> This is not much of a complication or inconsistency:
>
> type_shift = base_btf ? btf__type_cnt(base_btf) : 0;
> id_map[type_id - type_shift] = ...
Thank you, I agree and will do it in the next version.
>
>
> >
> > >
> > > > + *
> > > > + * For base BTF, its `start_id` is fixed to 1, i.e. the VOID type can
> > > > + * not be redefined or remapped and its ID is fixed to 0.
> > > > + *
> > > > + * For split BTF, its `start_id` can be retrieved by calling
> > > > + * `btf__type_cnt(btf__base_btf(btf))`.
> > > > + *
> > > > + * On error, returns negative error code and sets errno:
> > > > + * - `-EINVAL`: Invalid parameters or ID mapping (duplicates, out-of-range)
> > > > + * - `-ENOMEM`: Memory allocation failure
> > > > + */
> > > > +LIBBPF_API int btf__permute(struct btf *btf, __u32 *id_map, __u32 id_map_cnt,
> > > > + const struct btf_permute_opts *opts);
> > > > +
> > > > struct btf_dump;
> > > >
> > > > struct btf_dump_opts {
> > > > diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map
> > > > index 84fb90a016c9..d18fbcea7578 100644
> > > > --- a/tools/lib/bpf/libbpf.map
> > > > +++ b/tools/lib/bpf/libbpf.map
> > > > @@ -453,4 +453,5 @@ LIBBPF_1.7.0 {
> > > > bpf_map__exclusive_program;
> > > > bpf_prog_assoc_struct_ops;
> > > > bpf_program__assoc_struct_ops;
> > > > + btf__permute;
> > > > } LIBBPF_1.6.0;
> > > > --
> > > > 2.34.1
> > > >
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 03/13] tools/resolve_btfids: Support BTF sorting feature
2025-12-19 17:12 ` Andrii Nakryiko
@ 2025-12-20 8:44 ` Donglin Peng
0 siblings, 0 replies; 72+ messages in thread
From: Donglin Peng @ 2025-12-20 8:44 UTC (permalink / raw)
To: Andrii Nakryiko
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Sat, Dec 20, 2025 at 1:12 AM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Thu, Dec 18, 2025 at 7:42 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
> >
> > On Fri, Dec 19, 2025 at 7:09 AM Andrii Nakryiko
> > <andrii.nakryiko@gmail.com> wrote:
> > >
> > > On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
> > > >
> > > > From: pengdonglin <pengdonglin@xiaomi.com>
> > > >
> > > > This introduces a new BTF sorting phase that specifically sorts
> > > > BTF types by name in ascending order, so that the binary search
> > > > can be used to look up types.
> > > >
> > > > Cc: Eduard Zingerman <eddyz87@gmail.com>
> > > > Cc: Alexei Starovoitov <ast@kernel.org>
> > > > Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > > > Cc: Alan Maguire <alan.maguire@oracle.com>
> > > > Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> > > > Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> > > > Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> > >
> > > Signed-off-by is supposed to use properly spelled full name, this
> > > should be "Donglin Peng", right?
> >
> > Sorry, I will fix this in the next version. The reason is that our
> > company's Gerrit only accepts "pengdonglin" and does not
> > accept "Donglin Peng".
> >
> > >
> > > > Acked-by: Eduard Zingerman <eddyz87@gmail.com>
> > > > ---
> > > > tools/bpf/resolve_btfids/main.c | 68 +++++++++++++++++++++++++++++++++
> > > > 1 file changed, 68 insertions(+)
> > > >
> > > > diff --git a/tools/bpf/resolve_btfids/main.c b/tools/bpf/resolve_btfids/main.c
> > > > index 3e88dc862d87..659de35748ec 100644
> > > > --- a/tools/bpf/resolve_btfids/main.c
> > > > +++ b/tools/bpf/resolve_btfids/main.c
> > > > @@ -848,6 +848,71 @@ static int dump_raw_btf(struct btf *btf, const char *out_path)
> > > > return 0;
> > > > }
> > > >
> > > > +/*
> > > > + * Sort types by name in ascending order resulting in all
> > > > + * anonymous types being placed before named types.
> > > > + */
> > > > +static int cmp_type_names(const void *a, const void *b, void *priv)
> > > > +{
> > > > + struct btf *btf = (struct btf *)priv;
> > > > + const struct btf_type *ta = btf__type_by_id(btf, *(__u32 *)a);
> > > > + const struct btf_type *tb = btf__type_by_id(btf, *(__u32 *)b);
> > > > + const char *na, *nb;
> > > > +
> > > > + na = btf__str_by_offset(btf, ta->name_off);
> > > > + nb = btf__str_by_offset(btf, tb->name_off);
> > > > + return strcmp(na, nb);
> > > > +}
> > > > +
> > > > +static int sort_btf_by_name(struct btf *btf)
> > > > +{
> > > > + __u32 *permute_ids = NULL, *id_map = NULL;
> > > > + int nr_types, i, err = 0;
> > > > + __u32 start_id = 1, id;
> > > > +
> > > > + if (btf__base_btf(btf))
> > > > + start_id = btf__type_cnt(btf__base_btf(btf));
> > > > + nr_types = btf__type_cnt(btf) - start_id;
> > > > + if (nr_types < 2)
> > > > + goto out;
> > >
> > > why this check, will anything break if you don't do it?
> >
> > Because I think that if there are zero or only one type,
> > there is no need to sort.
>
> There is also no need to special-case and add more checks just for
> these corner cases. Keep it simple.
Thanks, I agree and and will fix it.
>
>
> >
> > >
> > > > +
> > > > + permute_ids = calloc(nr_types, sizeof(*permute_ids));
> > > > + if (!permute_ids) {
> > > > + err = -ENOMEM;
> > > > + goto out;
> > > > + }
> > > > +
> > > > + id_map = calloc(nr_types, sizeof(*id_map));
> > > > + if (!id_map) {
> > > > + err = -ENOMEM;
> > > > + goto out;
> > > > + }
> > > > +
> > > > + for (i = 0, id = start_id; i < nr_types; i++, id++)
> > > > + permute_ids[i] = id;
> > > > +
> > > > + qsort_r(permute_ids, nr_types, sizeof(*permute_ids), cmp_type_names, btf);
> > > > +
> > > > + for (i = 0; i < nr_types; i++) {
> > > > + id = permute_ids[i] - start_id;
> > > > + id_map[id] = i + start_id;
> > > > + }
> > > > +
> > > > + err = btf__permute(btf, id_map, nr_types, NULL);
> > > > + if (err)
> > > > + pr_err("FAILED: btf permute: %s\n", strerror(-err));
> > > > +
> > > > +out:
> > > > + free(permute_ids);
> > > > + free(id_map);
> > > > + return err;
> > > > +}
> > > > +
> > > > +static int btf2btf(struct object *obj)
> > >
> > > what's the point of having this function?
> >
> > Sorting BTF is a type of `btf2btf` process. There may be other
> > types of `btf2btf` processes, which could be grouped together
> > here. If we currently don't care about these other processes,
> > I will retain only `sort_btf_by_name` in the next version.
>
> Let's have sort as is, when necessary we can refactor this into
> logical steps further (or perhaps sorting will be its own logical
> step, I don't know, but let's not over design it just yet)
Okay, I will invoke sort_btf_by_name directly in the next version.
>
> >
> > >
> > > > +{
> > > > + return sort_btf_by_name(obj->btf);
> > > > +}
> > > > +
> > > > static inline int make_out_path(char *buf, u32 buf_sz, const char *in_path, const char *suffix)
> > > > {
> > > > int len = snprintf(buf, buf_sz, "%s%s", in_path, suffix);
> > > > @@ -906,6 +971,9 @@ int main(int argc, const char **argv)
> > > > if (load_btf(&obj))
> > > > goto out;
> > > >
> > > > + if (btf2btf(&obj))
> > > > + goto out;
> > > > +
> > > > if (elf_collect(&obj))
> > > > goto out;
> > > >
> > > > --
> > > > 2.34.1
> > > >
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 04/13] libbpf: Optimize type lookup with binary search for sorted BTF
2025-12-19 17:28 ` Andrii Nakryiko
@ 2025-12-20 9:38 ` Donglin Peng
2025-12-22 1:58 ` Donglin Peng
2026-01-06 0:36 ` Andrii Nakryiko
0 siblings, 2 replies; 72+ messages in thread
From: Donglin Peng @ 2025-12-20 9:38 UTC (permalink / raw)
To: Andrii Nakryiko
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Sat, Dec 20, 2025 at 1:28 AM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Thu, Dec 18, 2025 at 6:53 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
> >
> > On Fri, Dec 19, 2025 at 7:29 AM Andrii Nakryiko
> > <andrii.nakryiko@gmail.com> wrote:
> > >
> > > On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
> > > >
> > > > From: pengdonglin <pengdonglin@xiaomi.com>
> > > >
> > > > This patch introduces binary search optimization for BTF type lookups
> > > > when the BTF instance contains sorted types.
> > > >
> > > > The optimization significantly improves performance when searching for
> > > > types in large BTF instances with sorted types. For unsorted BTF, the
> > > > implementation falls back to the original linear search.
> > > >
> > > > Cc: Eduard Zingerman <eddyz87@gmail.com>
> > > > Cc: Alexei Starovoitov <ast@kernel.org>
> > > > Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > > > Cc: Alan Maguire <alan.maguire@oracle.com>
> > > > Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> > > > Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> > > > Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> > > > ---
> > > > tools/lib/bpf/btf.c | 103 ++++++++++++++++++++++++++++++++++----------
> > > > 1 file changed, 80 insertions(+), 23 deletions(-)
> > > >
> > >
> > > [...]
> > >
> > > > + l = start_id;
> > > > + r = end_id;
> > > > + while (l <= r) {
> > > > + m = l + (r - l) / 2;
> > > > + t = btf_type_by_id(btf, m);
> > > > + tname = btf__str_by_offset(btf, t->name_off);
> > > > + ret = strcmp(tname, name);
> > > > + if (ret < 0) {
> > > > + l = m + 1;
> > > > + } else {
> > > > + if (ret == 0)
> > > > + lmost = m;
> > > > + r = m - 1;
> > > > + }
> > > > }
> > >
> > > this differs from what we discussed in [0], you said you'll use that
> > > approach. Can you please elaborate on why you didn't?
> > >
> > > [0] https://lore.kernel.org/bpf/CAEf4Bzb3Eu0J83O=Y4KA-LkzBMjtx7cbonxPzkiduzZ1Pedajg@mail.gmail.com/
> >
> > Yes. As mentioned in the v8 changelog [1], the binary search approach
> > you referenced was implemented in versions v6 and v7 [2]. However,
> > testing revealed a slight performance regression. The root cause was
> > an extra strcmp operation introduced in v7, as discussed in [3]. Therefore,
> > in v8, I reverted to the approach from v5 [4] and refactored it for clarity.
>
> If you keep oscillating like that this patch set will never land. 4%
> (500us) gain on artificial and unrealistic micro-benchmark is
> meaningless and irrelevant, you are just adding more work for yourself
> and for reviewers by constantly changing your implementation between
> revisions for no good reason.
Thank you, I understand and will learn from it. I think the performance gain
makes sense. I’d like to share a specific real-world case where this
optimization
could matter: the `btf_find_by_name_kind()` function is indeed infrequently
used by the BPF subsystem, but it’s heavily relied upon by the ftrace
subsystem’s features like `func-args`, `funcgraph-args` [1], and the upcoming
`funcgraph-retval` [2]. These features invoke the function nearly once per
trace line when outputting, with a call frequency that can reach **100 kHz**
in intensive tracing workloads.
In such scenarios, the extra `strcmp` operations translate to ~100,000
additional
string comparisons per second. While this might seem negligible in isolation,
the overhead accumulates under high-frequency tracing—potentially impacting
latency for users relying on detailed function argument/return value tracing.
Thanks again for pushing for rigor—it helps make the code more cleaner
and robust.
[1] https://lore.kernel.org/all/20250227185822.639418500@goodmis.org/
[2] https://lore.kernel.org/all/20251215034153.2367756-1-dolinux.peng@gmail.com/
>
>
> >
> > Benchmark results show that v8 achieves a 4.2% performance improvement
> > over v7. If we don't care the performance gain, I will revert to the approach
> > in v7 in the next version.
> >
> > [1] https://lore.kernel.org/bpf/20251126085025.784288-1-dolinux.peng@gmail.com/
> > [2] https://lore.kernel.org/all/20251119031531.1817099-1-dolinux.peng@gmail.com/
> > [3] https://lore.kernel.org/all/CAEf4BzaqEPD46LddJHO1-k5KPGyVWf6d=duDAxG1q=jykJkMBg@mail.gmail.com/
> > [4] https://lore.kernel.org/all/20251106131956.1222864-4-dolinux.peng@gmail.com/
> >
> > >
> > > >
> > > > - return libbpf_err(-ENOENT);
> > > > + return lmost;
> > > > }
> > > >
> > > > static __s32 btf_find_by_name_kind(const struct btf *btf, int start_id,
> > > > const char *type_name, __u32 kind)
> > >
> > > kind is defined as u32 but you expect caller to pass -1 to ignore the
> > > kind. Use int here.
> >
> > Thanks, I will fix it.
> >
> > >
> > > > {
> > > > - __u32 i, nr_types = btf__type_cnt(btf);
> > > > + const struct btf_type *t;
> > > > + const char *tname;
> > > > + __s32 idx;
> > > > +
> > > > + if (start_id < btf->start_id) {
> > > > + idx = btf_find_by_name_kind(btf->base_btf, start_id,
> > > > + type_name, kind);
> > > > + if (idx >= 0)
> > > > + return idx;
> > > > + start_id = btf->start_id;
> > > > + }
> > > >
> > > > - if (kind == BTF_KIND_UNKN || !strcmp(type_name, "void"))
> > > > + if (kind == BTF_KIND_UNKN || strcmp(type_name, "void") == 0)
> > > > return 0;
> > > >
> > > > - for (i = start_id; i < nr_types; i++) {
> > > > - const struct btf_type *t = btf__type_by_id(btf, i);
> > > > - const char *name;
> > > > + if (btf->sorted_start_id > 0 && type_name[0]) {
> > > > + __s32 end_id = btf__type_cnt(btf) - 1;
> > > > +
> > > > + /* skip anonymous types */
> > > > + start_id = max(start_id, btf->sorted_start_id);
> > >
> > > can sorted_start_id ever be smaller than start_id?
> > >
> > > > + idx = btf_find_by_name_bsearch(btf, type_name, start_id, end_id);
> > >
> > > is there ever a time when btf_find_by_name_bsearch() will work with
> > > different start_id and end_id? why is this not done inside the
> > > btf_find_by_name_bsearch()?
> >
> > Because the start_id could be specified by the caller.
>
> Right, start_id has to be passed in. But end_id is always the same, so
> maybe determine it internally instead? And let's not return -ENOENT
Thanks, I agree and will put the end_id into btf_find_by_name_bsearch.
> from btf_find_by_name_bsearch(), as I mentioned before, it would be
> more streamlined if you return btf__type_cnt(btf) if search failed.
Thanks, I agree.
>
> >
> > >
> > > > + if (unlikely(idx < 0))
> > > > + return libbpf_err(-ENOENT);
> > >
> > > pass through error returned from btf_find_by_name_bsearch(), why redefining it?
> >
> > Thanks, I will fix it.
> >
>
> see above, by returning btf__type_cnt() you won't even have this error
> handling, you'll just go through normal loop checking for a match and
> won't find anything, returning -ENOENT then.
Thanks, I agree.
>
> > >
> > > > +
> > > > + if (unlikely(kind == -1))
> > > > + return idx;
> > > > +
> > > > + t = btf_type_by_id(btf, idx);
> > > > + if (likely(BTF_INFO_KIND(t->info) == kind))
> > >
> > > use btf_kind(), but this whole extra check is just unnecessary, this
> >
> > Thanks, I will do it.
> >
> > > should be done in the loop below. We talked about all this already,
> > > why do I feel like I'm being ignored?..
> >
> > Sorry for the confusion, and absolutely not ignoring you.
> >
>
> If you decide to change implementation due to some unforeseen factors
> (like concern about 4% microbenchmark improvement), it would be
> helpful for you to call this out in a reply to the original
> discussion. A line somewhere in the cover letter changelog is way too
> easy to miss and that doesn't give me an opportunity to stop you
> before you go and produce another revision that I'll then be
> rejecting.
I will learn from it and thank you for the suggestion.
>
> > >
> > > > + return idx;
> > >
> > > drop all these likely and unlikely micro optimizations, please
> >
> > Thanks, I will do it.
> >
> > >
> > >
> > > > +
> > > > + for (idx++; idx <= end_id; idx++) {
> > > > + t = btf__type_by_id(btf, idx);
> > > > + tname = btf__str_by_offset(btf, t->name_off);
> > > > + if (strcmp(tname, type_name) != 0)
> > > > + return libbpf_err(-ENOENT);
> > > > + if (btf_kind(t) == kind)
> > > > + return idx;
> > > > + }
> > > > + } else {
> > > > + __u32 i, total;
> > > >
> > > > - if (btf_kind(t) != kind)
> > > > - continue;
> > > > - name = btf__name_by_offset(btf, t->name_off);
> > > > - if (name && !strcmp(type_name, name))
> > > > - return i;
> > > > + total = btf__type_cnt(btf);
> > > > + for (i = start_id; i < total; i++) {
> > > > + t = btf_type_by_id(btf, i);
> > > > + if (kind != -1 && btf_kind(t) != kind)
> > >
> > > nit: kind < 0, no need to hard-code -1
> >
> > Good, I will fix it.
> >
> > >
> > > > + continue;
> > > > + tname = btf__str_by_offset(btf, t->name_off);
> > > > + if (strcmp(tname, type_name) == 0)
> > > > + return i;
> > > > + }
> > > > }
> > > >
> > > > return libbpf_err(-ENOENT);
> > > > }
> > > >
> > >
> > > [...]
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 12/13] btf: Add btf_is_sorted to refactor the code
2025-12-19 0:05 ` Andrii Nakryiko
@ 2025-12-20 14:25 ` Donglin Peng
0 siblings, 0 replies; 72+ messages in thread
From: Donglin Peng @ 2025-12-20 14:25 UTC (permalink / raw)
To: Andrii Nakryiko
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Fri, Dec 19, 2025 at 8:05 AM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
> >
> > From: pengdonglin <pengdonglin@xiaomi.com>
> >
> > Introduce a new helper function to clarify the code and no
> > functional changes are introduced.
> >
> > Cc: Eduard Zingerman <eddyz87@gmail.com>
> > Cc: Alexei Starovoitov <ast@kernel.org>
> > Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > Cc: Alan Maguire <alan.maguire@oracle.com>
> > Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> > Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> > Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> > ---
> > include/linux/btf.h | 1 +
> > kernel/bpf/btf.c | 9 +++++++--
> > 2 files changed, 8 insertions(+), 2 deletions(-)
> >
>
> let's drop, this is not necessary
Okay, I will drop it.
>
>
> > diff --git a/include/linux/btf.h b/include/linux/btf.h
> > index 2d28f2b22ae5..947ed2abf632 100644
> > --- a/include/linux/btf.h
> > +++ b/include/linux/btf.h
> > @@ -221,6 +221,7 @@ bool btf_is_vmlinux(const struct btf *btf);
> > struct module *btf_try_get_module(const struct btf *btf);
> > u32 btf_nr_types(const struct btf *btf);
> > u32 btf_sorted_start_id(const struct btf *btf);
> > +bool btf_is_sorted(const struct btf *btf);
> > struct btf *btf_base_btf(const struct btf *btf);
> > bool btf_type_is_i32(const struct btf_type *t);
> > bool btf_type_is_i64(const struct btf_type *t);
> > diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
> > index 3aeb4f00cbfe..0f20887a6f02 100644
> > --- a/kernel/bpf/btf.c
> > +++ b/kernel/bpf/btf.c
> > @@ -555,6 +555,11 @@ u32 btf_sorted_start_id(const struct btf *btf)
> > return btf->sorted_start_id ?: (btf->start_id ?: 1);
> > }
> >
> > +bool btf_is_sorted(const struct btf *btf)
> > +{
> > + return btf->sorted_start_id > 0;
> > +}
> > +
> > /*
> > * Assuming that types are sorted by name in ascending order.
> > */
> > @@ -649,9 +654,9 @@ s32 btf_find_by_name_kind(const struct btf *btf, const char *name, u8 kind)
> > return idx;
> > }
> >
> > - if (btf->sorted_start_id > 0 && name[0]) {
> > + if (btf_is_sorted(btf) && name[0]) {
> > /* skip anonymous types */
> > - s32 start_id = btf->sorted_start_id;
> > + s32 start_id = btf_sorted_start_id(btf);
> > s32 end_id = btf_nr_types(btf) - 1;
> >
> > idx = btf_find_by_name_bsearch(btf, name, start_id, end_id);
> > --
> > 2.34.1
> >
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 09/13] bpf: Optimize the performance of find_bpffs_btf_enums
2025-12-19 5:41 ` Donglin Peng
@ 2025-12-20 14:27 ` Donglin Peng
0 siblings, 0 replies; 72+ messages in thread
From: Donglin Peng @ 2025-12-20 14:27 UTC (permalink / raw)
To: Andrii Nakryiko
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Fri, Dec 19, 2025 at 1:41 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
>
> On Fri, Dec 19, 2025 at 8:02 AM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
> >
> > On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
> > >
> > > From: pengdonglin <pengdonglin@xiaomi.com>
> > >
> > > Currently, vmlinux BTF is unconditionally sorted during
> > > the build phase. The function btf_find_by_name_kind
> > > executes the binary search branch, so find_bpffs_btf_enums
> > > can be optimized by using btf_find_by_name_kind.
> > >
> > > Cc: Eduard Zingerman <eddyz87@gmail.com>
> > > Cc: Alexei Starovoitov <ast@kernel.org>
> > > Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > > Cc: Alan Maguire <alan.maguire@oracle.com>
> > > Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> > > Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> > > Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> > > Acked-by: Eduard Zingerman <eddyz87@gmail.com>
> > > ---
> > > kernel/bpf/inode.c | 42 +++++++++++++++++++-----------------------
> > > 1 file changed, 19 insertions(+), 23 deletions(-)
> > >
> > > diff --git a/kernel/bpf/inode.c b/kernel/bpf/inode.c
> > > index 9f866a010dad..050fde1cf211 100644
> > > --- a/kernel/bpf/inode.c
> > > +++ b/kernel/bpf/inode.c
> > > @@ -600,10 +600,18 @@ struct bpffs_btf_enums {
> > >
> > > static int find_bpffs_btf_enums(struct bpffs_btf_enums *info)
> > > {
> > > + struct {
> > > + const struct btf_type **type;
> > > + const char *name;
> > > + } btf_enums[] = {
> > > + {&info->cmd_t, "bpf_cmd"},
> > > + {&info->map_t, "bpf_map_type"},
> > > + {&info->prog_t, "bpf_prog_type"},
> > > + {&info->attach_t, "bpf_attach_type"},
> > > + };
> > > const struct btf *btf;
> > > const struct btf_type *t;
> > > - const char *name;
> > > - int i, n;
> > > + int i, id;
> > >
> > > memset(info, 0, sizeof(*info));
> > >
> > > @@ -615,30 +623,18 @@ static int find_bpffs_btf_enums(struct bpffs_btf_enums *info)
> > >
> > > info->btf = btf;
> > >
> > > - for (i = 1, n = btf_nr_types(btf); i < n; i++) {
> > > - t = btf_type_by_id(btf, i);
> > > - if (!btf_type_is_enum(t))
> > > - continue;
> > > + for (i = 0; i < ARRAY_SIZE(btf_enums); i++) {
> > > + id = btf_find_by_name_kind(btf, btf_enums[i].name,
> > > + BTF_KIND_ENUM);
> > > + if (id < 0)
> > > + goto out;
> > >
> > > - name = btf_name_by_offset(btf, t->name_off);
> > > - if (!name)
> > > - continue;
> >
> > return -ESRCH, why goto at all?
Thanks, will do.
> >
> > > -
> > > - if (strcmp(name, "bpf_cmd") == 0)
> > > - info->cmd_t = t;
> > > - else if (strcmp(name, "bpf_map_type") == 0)
> > > - info->map_t = t;
> > > - else if (strcmp(name, "bpf_prog_type") == 0)
> > > - info->prog_t = t;
> > > - else if (strcmp(name, "bpf_attach_type") == 0)
> > > - info->attach_t = t;
> > > - else
> > > - continue;
> > > -
> > > - if (info->cmd_t && info->map_t && info->prog_t && info->attach_t)
> > > - return 0;
> > > + t = btf_type_by_id(btf, id);
> > > + *btf_enums[i].type = t;
> >
> > nit: drop t local variable, just assign directly:
> >
> > *btf_enums[i].type = btf_type_by_id(btf, id);
>
> Thanks, I will do it.
>
> >
> > > }
> > >
> > > + return 0;
> > > +out:
> > > return -ESRCH;
> > > }
> > >
> > > --
> > > 2.34.1
> > >
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 11/13] libbpf: Add btf_is_sorted and btf_sorted_start_id helpers to refactor the code
2025-12-19 17:35 ` Andrii Nakryiko
@ 2025-12-20 14:27 ` Donglin Peng
0 siblings, 0 replies; 72+ messages in thread
From: Donglin Peng @ 2025-12-20 14:27 UTC (permalink / raw)
To: Andrii Nakryiko
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
Andrii Nakryiko <andrii.nakryiko@gmail.com>于2025年12月20日 周六01:35写道：
>
> On Thu, Dec 18, 2025 at 9:51 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
> >
> > On Fri, Dec 19, 2025 at 8:05 AM Andrii Nakryiko
> > <andrii.nakryiko@gmail.com> wrote:
> > >
> > > On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
> > > >
> > > > From: pengdonglin <pengdonglin@xiaomi.com>
> > > >
> > > > Introduce two new helper functions to clarify the code and no
> > > > functional changes are introduced.
> > > >
> > > > Cc: Eduard Zingerman <eddyz87@gmail.com>
> > > > Cc: Alexei Starovoitov <ast@kernel.org>
> > > > Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > > > Cc: Alan Maguire <alan.maguire@oracle.com>
> > > > Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> > > > Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> > > > Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> > > > ---
> > > > tools/lib/bpf/btf.c | 14 ++++++++++++--
> > > > tools/lib/bpf/libbpf_internal.h | 2 ++
> > > > 2 files changed, 14 insertions(+), 2 deletions(-)
> > > >
> > >
> > > It just adds more functions to jump to and check what it's doing. I
> > > don't think this adds much value, just drop this patch
> >
> > Could adding the __always_inline be acceptable?
>
> No! This is not a performance concern, just mental overhead when
> reading the code.
Thanks, will remove it.
>
>
>
> >
> > >
> > >
> > > > diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> > > > index b5b0898d033d..571b72bd90b5 100644
> > > > --- a/tools/lib/bpf/btf.c
> > > > +++ b/tools/lib/bpf/btf.c
> > > > @@ -626,6 +626,16 @@ const struct btf *btf__base_btf(const struct btf *btf)
> > > > return btf->base_btf;
> > > > }
> > > >
> > > > +int btf_sorted_start_id(const struct btf *btf)
> > > > +{
> > > > + return btf->sorted_start_id;
> > > > +}
> > > > +
> > > > +bool btf_is_sorted(const struct btf *btf)
> > > > +{
> > > > + return btf->sorted_start_id > 0;
> > > > +}
> > > > +
> > > > /* internal helper returning non-const pointer to a type */
> > > > struct btf_type *btf_type_by_id(const struct btf *btf, __u32 type_id)
> > > > {
> > > > @@ -976,11 +986,11 @@ static __s32 btf_find_by_name_kind(const struct btf *btf, int start_id,
> > > > if (kind == BTF_KIND_UNKN || strcmp(type_name, "void") == 0)
> > > > return 0;
> > > >
> > > > - if (btf->sorted_start_id > 0 && type_name[0]) {
> > > > + if (btf_is_sorted(btf) && type_name[0]) {
> > > > __s32 end_id = btf__type_cnt(btf) - 1;
> > > >
> > > > /* skip anonymous types */
> > > > - start_id = max(start_id, btf->sorted_start_id);
> > > > + start_id = max(start_id, btf_sorted_start_id(btf));
> > > > idx = btf_find_by_name_bsearch(btf, type_name, start_id, end_id);
> > > > if (unlikely(idx < 0))
> > > > return libbpf_err(-ENOENT);
> > > > diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_internal.h
> > > > index fc59b21b51b5..95e6848396b4 100644
> > > > --- a/tools/lib/bpf/libbpf_internal.h
> > > > +++ b/tools/lib/bpf/libbpf_internal.h
> > > > @@ -250,6 +250,8 @@ const struct btf_type *skip_mods_and_typedefs(const struct btf *btf, __u32 id, _
> > > > const struct btf_header *btf_header(const struct btf *btf);
> > > > void btf_set_base_btf(struct btf *btf, const struct btf *base_btf);
> > > > int btf_relocate(struct btf *btf, const struct btf *base_btf, __u32 **id_map);
> > > > +int btf_sorted_start_id(const struct btf *btf);
> > > > +bool btf_is_sorted(const struct btf *btf);
> > > >
> > > > static inline enum btf_func_linkage btf_func_linkage(const struct btf_type *t)
> > > > {
> > > > --
> > > > 2.34.1
> > > >
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 04/13] libbpf: Optimize type lookup with binary search for sorted BTF
2025-12-20 9:38 ` Donglin Peng
@ 2025-12-22 1:58 ` Donglin Peng
2026-01-06 0:38 ` Andrii Nakryiko
2026-01-06 0:36 ` Andrii Nakryiko
1 sibling, 1 reply; 72+ messages in thread
From: Donglin Peng @ 2025-12-22 1:58 UTC (permalink / raw)
To: Andrii Nakryiko
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Sat, Dec 20, 2025 at 5:38 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
>
> On Sat, Dec 20, 2025 at 1:28 AM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
> >
> > On Thu, Dec 18, 2025 at 6:53 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
> > >
> > > On Fri, Dec 19, 2025 at 7:29 AM Andrii Nakryiko
> > > <andrii.nakryiko@gmail.com> wrote:
> > > >
> > > > On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
> > > > >
> > > > > From: pengdonglin <pengdonglin@xiaomi.com>
> > > > >
> > > > > This patch introduces binary search optimization for BTF type lookups
> > > > > when the BTF instance contains sorted types.
> > > > >
> > > > > The optimization significantly improves performance when searching for
> > > > > types in large BTF instances with sorted types. For unsorted BTF, the
> > > > > implementation falls back to the original linear search.
> > > > >
> > > > > Cc: Eduard Zingerman <eddyz87@gmail.com>
> > > > > Cc: Alexei Starovoitov <ast@kernel.org>
> > > > > Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > > > > Cc: Alan Maguire <alan.maguire@oracle.com>
> > > > > Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> > > > > Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> > > > > Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> > > > > ---
> > > > > tools/lib/bpf/btf.c | 103 ++++++++++++++++++++++++++++++++++----------
> > > > > 1 file changed, 80 insertions(+), 23 deletions(-)
> > > > >
> > > >
> > > > [...]
> > > >
> > > > > + l = start_id;
> > > > > + r = end_id;
> > > > > + while (l <= r) {
> > > > > + m = l + (r - l) / 2;
> > > > > + t = btf_type_by_id(btf, m);
> > > > > + tname = btf__str_by_offset(btf, t->name_off);
> > > > > + ret = strcmp(tname, name);
> > > > > + if (ret < 0) {
> > > > > + l = m + 1;
> > > > > + } else {
> > > > > + if (ret == 0)
> > > > > + lmost = m;
> > > > > + r = m - 1;
> > > > > + }
> > > > > }
> > > >
> > > > this differs from what we discussed in [0], you said you'll use that
> > > > approach. Can you please elaborate on why you didn't?
> > > >
> > > > [0] https://lore.kernel.org/bpf/CAEf4Bzb3Eu0J83O=Y4KA-LkzBMjtx7cbonxPzkiduzZ1Pedajg@mail.gmail.com/
> > >
> > > Yes. As mentioned in the v8 changelog [1], the binary search approach
> > > you referenced was implemented in versions v6 and v7 [2]. However,
> > > testing revealed a slight performance regression. The root cause was
> > > an extra strcmp operation introduced in v7, as discussed in [3]. Therefore,
> > > in v8, I reverted to the approach from v5 [4] and refactored it for clarity.
> >
> > If you keep oscillating like that this patch set will never land. 4%
> > (500us) gain on artificial and unrealistic micro-benchmark is
> > meaningless and irrelevant, you are just adding more work for yourself
> > and for reviewers by constantly changing your implementation between
> > revisions for no good reason.
>
> Thank you, I understand and will learn from it. I think the performance gain
> makes sense. I’d like to share a specific real-world case where this
> optimization
> could matter: the `btf_find_by_name_kind()` function is indeed infrequently
> used by the BPF subsystem, but it’s heavily relied upon by the ftrace
> subsystem’s features like `func-args`, `funcgraph-args` [1], and the upcoming
> `funcgraph-retval` [2]. These features invoke the function nearly once per
> trace line when outputting, with a call frequency that can reach **100 kHz**
> in intensive tracing workloads.
Hi Andrii,
I think we can refactor the code based on your suggestion like this:
1. If the binary search finds the matching name type, return its index.
Else, return btf__type_cnt(btf). It will make the code streamlined.
2. Skip the name checking in the first loop to eliminate the extra strcmp.
What do you think?
tatic __s32 btf_find_by_name_bsearch(const struct btf *btf, const char *name,
__s32 start_id)
{
const struct btf_type *t;
const char *tname;
__s32 end_id = btf__type_cnt(btf) - 1;
__s32 l, r, m, lmost = end_id + 1;
int ret;
l = start_id;
r = end_id;
while (l <= r) {
m = l + (r - l) / 2;
t = btf_type_by_id(btf, m);
tname = btf__str_by_offset(btf, t->name_off);
ret = strcmp(tname, name);
if (ret < 0) {
l = m + 1;
} else {
if (ret == 0)
lmost = m;
r = m - 1;
}
}
return lmost;
}
static __s32 btf_find_by_name_kind(const struct btf *btf, int start_id,
const char *type_name, __u32 kind)
{
......
if (btf_is_sorted(btf) && type_name[0]) {
bool first_loop = true;
start_id = max(start_id, btf_sorted_start_id(btf));
idx = btf_find_by_name_bsearch(btf, type_name, start_id);
for (; idx < btf__type_cnt(btf); idx++) {
t = btf__type_by_id(btf, idx);
tname = btf__str_by_offset(btf, t->name_off);
if (!first_loop && strcmp(tname, type_name) != 0)
return libbpf_err(-ENOENT);
if (kind == -1 || btf_kind(t) == kind)
return idx;
if (first_loop)
first_loop = false;
}
} else {
......
}
return libbpf_err(-ENOENT);
}
>
> In such scenarios, the extra `strcmp` operations translate to ~100,000
> additional
> string comparisons per second. While this might seem negligible in isolation,
> the overhead accumulates under high-frequency tracing—potentially impacting
> latency for users relying on detailed function argument/return value tracing.
>
> Thanks again for pushing for rigor—it helps make the code more cleaner
> and robust.
>
> [1] https://lore.kernel.org/all/20250227185822.639418500@goodmis.org/
> [2] https://lore.kernel.org/all/20251215034153.2367756-1-dolinux.peng@gmail.com/
>
> >
> >
> > >
> > > Benchmark results show that v8 achieves a 4.2% performance improvement
> > > over v7. If we don't care the performance gain, I will revert to the approach
> > > in v7 in the next version.
> > >
> > > [1] https://lore.kernel.org/bpf/20251126085025.784288-1-dolinux.peng@gmail.com/
> > > [2] https://lore.kernel.org/all/20251119031531.1817099-1-dolinux.peng@gmail.com/
> > > [3] https://lore.kernel.org/all/CAEf4BzaqEPD46LddJHO1-k5KPGyVWf6d=duDAxG1q=jykJkMBg@mail.gmail.com/
> > > [4] https://lore.kernel.org/all/20251106131956.1222864-4-dolinux.peng@gmail.com/
> > >
> > > >
> > > > >
> > > > > - return libbpf_err(-ENOENT);
> > > > > + return lmost;
> > > > > }
> > > > >
> > > > > static __s32 btf_find_by_name_kind(const struct btf *btf, int start_id,
> > > > > const char *type_name, __u32 kind)
> > > >
> > > > kind is defined as u32 but you expect caller to pass -1 to ignore the
> > > > kind. Use int here.
> > >
> > > Thanks, I will fix it.
> > >
> > > >
> > > > > {
> > > > > - __u32 i, nr_types = btf__type_cnt(btf);
> > > > > + const struct btf_type *t;
> > > > > + const char *tname;
> > > > > + __s32 idx;
> > > > > +
> > > > > + if (start_id < btf->start_id) {
> > > > > + idx = btf_find_by_name_kind(btf->base_btf, start_id,
> > > > > + type_name, kind);
> > > > > + if (idx >= 0)
> > > > > + return idx;
> > > > > + start_id = btf->start_id;
> > > > > + }
> > > > >
> > > > > - if (kind == BTF_KIND_UNKN || !strcmp(type_name, "void"))
> > > > > + if (kind == BTF_KIND_UNKN || strcmp(type_name, "void") == 0)
> > > > > return 0;
> > > > >
> > > > > - for (i = start_id; i < nr_types; i++) {
> > > > > - const struct btf_type *t = btf__type_by_id(btf, i);
> > > > > - const char *name;
> > > > > + if (btf->sorted_start_id > 0 && type_name[0]) {
> > > > > + __s32 end_id = btf__type_cnt(btf) - 1;
> > > > > +
> > > > > + /* skip anonymous types */
> > > > > + start_id = max(start_id, btf->sorted_start_id);
> > > >
> > > > can sorted_start_id ever be smaller than start_id?
> > > >
> > > > > + idx = btf_find_by_name_bsearch(btf, type_name, start_id, end_id);
> > > >
> > > > is there ever a time when btf_find_by_name_bsearch() will work with
> > > > different start_id and end_id? why is this not done inside the
> > > > btf_find_by_name_bsearch()?
> > >
> > > Because the start_id could be specified by the caller.
> >
> > Right, start_id has to be passed in. But end_id is always the same, so
> > maybe determine it internally instead? And let's not return -ENOENT
>
> Thanks, I agree and will put the end_id into btf_find_by_name_bsearch.
>
> > from btf_find_by_name_bsearch(), as I mentioned before, it would be
> > more streamlined if you return btf__type_cnt(btf) if search failed.
>
> Thanks, I agree.
>
> >
> > >
> > > >
> > > > > + if (unlikely(idx < 0))
> > > > > + return libbpf_err(-ENOENT);
> > > >
> > > > pass through error returned from btf_find_by_name_bsearch(), why redefining it?
> > >
> > > Thanks, I will fix it.
> > >
> >
> > see above, by returning btf__type_cnt() you won't even have this error
> > handling, you'll just go through normal loop checking for a match and
> > won't find anything, returning -ENOENT then.
>
> Thanks, I agree.
>
> >
> > > >
> > > > > +
> > > > > + if (unlikely(kind == -1))
> > > > > + return idx;
> > > > > +
> > > > > + t = btf_type_by_id(btf, idx);
> > > > > + if (likely(BTF_INFO_KIND(t->info) == kind))
> > > >
> > > > use btf_kind(), but this whole extra check is just unnecessary, this
> > >
> > > Thanks, I will do it.
> > >
> > > > should be done in the loop below. We talked about all this already,
> > > > why do I feel like I'm being ignored?..
> > >
> > > Sorry for the confusion, and absolutely not ignoring you.
> > >
> >
> > If you decide to change implementation due to some unforeseen factors
> > (like concern about 4% microbenchmark improvement), it would be
> > helpful for you to call this out in a reply to the original
> > discussion. A line somewhere in the cover letter changelog is way too
> > easy to miss and that doesn't give me an opportunity to stop you
> > before you go and produce another revision that I'll then be
> > rejecting.
>
> I will learn from it and thank you for the suggestion.
>
> >
> > > >
> > > > > + return idx;
> > > >
> > > > drop all these likely and unlikely micro optimizations, please
> > >
> > > Thanks, I will do it.
> > >
> > > >
> > > >
> > > > > +
> > > > > + for (idx++; idx <= end_id; idx++) {
> > > > > + t = btf__type_by_id(btf, idx);
> > > > > + tname = btf__str_by_offset(btf, t->name_off);
> > > > > + if (strcmp(tname, type_name) != 0)
> > > > > + return libbpf_err(-ENOENT);
> > > > > + if (btf_kind(t) == kind)
> > > > > + return idx;
> > > > > + }
> > > > > + } else {
> > > > > + __u32 i, total;
> > > > >
> > > > > - if (btf_kind(t) != kind)
> > > > > - continue;
> > > > > - name = btf__name_by_offset(btf, t->name_off);
> > > > > - if (name && !strcmp(type_name, name))
> > > > > - return i;
> > > > > + total = btf__type_cnt(btf);
> > > > > + for (i = start_id; i < total; i++) {
> > > > > + t = btf_type_by_id(btf, i);
> > > > > + if (kind != -1 && btf_kind(t) != kind)
> > > >
> > > > nit: kind < 0, no need to hard-code -1
> > >
> > > Good, I will fix it.
> > >
> > > >
> > > > > + continue;
> > > > > + tname = btf__str_by_offset(btf, t->name_off);
> > > > > + if (strcmp(tname, type_name) == 0)
> > > > > + return i;
> > > > > + }
> > > > > }
> > > > >
> > > > > return libbpf_err(-ENOENT);
> > > > > }
> > > > >
> > > >
> > > > [...]
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 04/13] libbpf: Optimize type lookup with binary search for sorted BTF
2025-12-20 9:38 ` Donglin Peng
2025-12-22 1:58 ` Donglin Peng
@ 2026-01-06 0:36 ` Andrii Nakryiko
2026-01-06 2:50 ` Donglin Peng
1 sibling, 1 reply; 72+ messages in thread
From: Andrii Nakryiko @ 2026-01-06 0:36 UTC (permalink / raw)
To: Donglin Peng
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Sat, Dec 20, 2025 at 1:38 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
>
> On Sat, Dec 20, 2025 at 1:28 AM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
> >
> > On Thu, Dec 18, 2025 at 6:53 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
> > >
> > > On Fri, Dec 19, 2025 at 7:29 AM Andrii Nakryiko
> > > <andrii.nakryiko@gmail.com> wrote:
> > > >
> > > > On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
> > > > >
> > > > > From: pengdonglin <pengdonglin@xiaomi.com>
> > > > >
> > > > > This patch introduces binary search optimization for BTF type lookups
> > > > > when the BTF instance contains sorted types.
> > > > >
> > > > > The optimization significantly improves performance when searching for
> > > > > types in large BTF instances with sorted types. For unsorted BTF, the
> > > > > implementation falls back to the original linear search.
> > > > >
> > > > > Cc: Eduard Zingerman <eddyz87@gmail.com>
> > > > > Cc: Alexei Starovoitov <ast@kernel.org>
> > > > > Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > > > > Cc: Alan Maguire <alan.maguire@oracle.com>
> > > > > Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> > > > > Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> > > > > Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> > > > > ---
> > > > > tools/lib/bpf/btf.c | 103 ++++++++++++++++++++++++++++++++++----------
> > > > > 1 file changed, 80 insertions(+), 23 deletions(-)
> > > > >
> > > >
> > > > [...]
> > > >
> > > > > + l = start_id;
> > > > > + r = end_id;
> > > > > + while (l <= r) {
> > > > > + m = l + (r - l) / 2;
> > > > > + t = btf_type_by_id(btf, m);
> > > > > + tname = btf__str_by_offset(btf, t->name_off);
> > > > > + ret = strcmp(tname, name);
> > > > > + if (ret < 0) {
> > > > > + l = m + 1;
> > > > > + } else {
> > > > > + if (ret == 0)
> > > > > + lmost = m;
> > > > > + r = m - 1;
> > > > > + }
> > > > > }
> > > >
> > > > this differs from what we discussed in [0], you said you'll use that
> > > > approach. Can you please elaborate on why you didn't?
> > > >
> > > > [0] https://lore.kernel.org/bpf/CAEf4Bzb3Eu0J83O=Y4KA-LkzBMjtx7cbonxPzkiduzZ1Pedajg@mail.gmail.com/
> > >
> > > Yes. As mentioned in the v8 changelog [1], the binary search approach
> > > you referenced was implemented in versions v6 and v7 [2]. However,
> > > testing revealed a slight performance regression. The root cause was
> > > an extra strcmp operation introduced in v7, as discussed in [3]. Therefore,
> > > in v8, I reverted to the approach from v5 [4] and refactored it for clarity.
> >
> > If you keep oscillating like that this patch set will never land. 4%
> > (500us) gain on artificial and unrealistic micro-benchmark is
> > meaningless and irrelevant, you are just adding more work for yourself
> > and for reviewers by constantly changing your implementation between
> > revisions for no good reason.
>
> Thank you, I understand and will learn from it. I think the performance gain
> makes sense. I’d like to share a specific real-world case where this
> optimization
> could matter: the `btf_find_by_name_kind()` function is indeed infrequently
> used by the BPF subsystem, but it’s heavily relied upon by the ftrace
> subsystem’s features like `func-args`, `funcgraph-args` [1], and the upcoming
> `funcgraph-retval` [2]. These features invoke the function nearly once per
> trace line when outputting, with a call frequency that can reach **100 kHz**
> in intensive tracing workloads.
>
> In such scenarios, the extra `strcmp` operations translate to ~100,000
> additional
> string comparisons per second. While this might seem negligible in isolation,
I'm pretty confident it is quite negligible compared to all other
*useful* work you'll have to do to use that BTF type to format those
arguments. Yes, 100KHz is pretty frequent, but it won't be anywhere
close to 4% if you profile the entire end-to-end overhead of emitting
arguments using func-args.
> the overhead accumulates under high-frequency tracing—potentially impacting
> latency for users relying on detailed function argument/return value tracing.
>
> Thanks again for pushing for rigor—it helps make the code more cleaner
> and robust.
>
> [1] https://lore.kernel.org/all/20250227185822.639418500@goodmis.org/
> [2] https://lore.kernel.org/all/20251215034153.2367756-1-dolinux.peng@gmail.com/
>
> >
> >
> > >
> > > Benchmark results show that v8 achieves a 4.2% performance improvement
> > > over v7. If we don't care the performance gain, I will revert to the approach
> > > in v7 in the next version.
> > >
> > > [1] https://lore.kernel.org/bpf/20251126085025.784288-1-dolinux.peng@gmail.com/
> > > [2] https://lore.kernel.org/all/20251119031531.1817099-1-dolinux.peng@gmail.com/
> > > [3] https://lore.kernel.org/all/CAEf4BzaqEPD46LddJHO1-k5KPGyVWf6d=duDAxG1q=jykJkMBg@mail.gmail.com/
> > > [4] https://lore.kernel.org/all/20251106131956.1222864-4-dolinux.peng@gmail.com/
> > >
[...]
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 04/13] libbpf: Optimize type lookup with binary search for sorted BTF
2025-12-22 1:58 ` Donglin Peng
@ 2026-01-06 0:38 ` Andrii Nakryiko
2026-01-06 2:42 ` Donglin Peng
0 siblings, 1 reply; 72+ messages in thread
From: Andrii Nakryiko @ 2026-01-06 0:38 UTC (permalink / raw)
To: Donglin Peng
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Sun, Dec 21, 2025 at 5:58 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
>
> On Sat, Dec 20, 2025 at 5:38 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
> >
> > On Sat, Dec 20, 2025 at 1:28 AM Andrii Nakryiko
> > <andrii.nakryiko@gmail.com> wrote:
> > >
> > > On Thu, Dec 18, 2025 at 6:53 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
> > > >
> > > > On Fri, Dec 19, 2025 at 7:29 AM Andrii Nakryiko
> > > > <andrii.nakryiko@gmail.com> wrote:
> > > > >
> > > > > On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
> > > > > >
> > > > > > From: pengdonglin <pengdonglin@xiaomi.com>
> > > > > >
> > > > > > This patch introduces binary search optimization for BTF type lookups
> > > > > > when the BTF instance contains sorted types.
> > > > > >
> > > > > > The optimization significantly improves performance when searching for
> > > > > > types in large BTF instances with sorted types. For unsorted BTF, the
> > > > > > implementation falls back to the original linear search.
> > > > > >
> > > > > > Cc: Eduard Zingerman <eddyz87@gmail.com>
> > > > > > Cc: Alexei Starovoitov <ast@kernel.org>
> > > > > > Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > > > > > Cc: Alan Maguire <alan.maguire@oracle.com>
> > > > > > Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> > > > > > Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> > > > > > Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> > > > > > ---
> > > > > > tools/lib/bpf/btf.c | 103 ++++++++++++++++++++++++++++++++++----------
> > > > > > 1 file changed, 80 insertions(+), 23 deletions(-)
> > > > > >
> > > > >
> > > > > [...]
> > > > >
> > > > > > + l = start_id;
> > > > > > + r = end_id;
> > > > > > + while (l <= r) {
> > > > > > + m = l + (r - l) / 2;
> > > > > > + t = btf_type_by_id(btf, m);
> > > > > > + tname = btf__str_by_offset(btf, t->name_off);
> > > > > > + ret = strcmp(tname, name);
> > > > > > + if (ret < 0) {
> > > > > > + l = m + 1;
> > > > > > + } else {
> > > > > > + if (ret == 0)
> > > > > > + lmost = m;
> > > > > > + r = m - 1;
> > > > > > + }
> > > > > > }
> > > > >
> > > > > this differs from what we discussed in [0], you said you'll use that
> > > > > approach. Can you please elaborate on why you didn't?
> > > > >
> > > > > [0] https://lore.kernel.org/bpf/CAEf4Bzb3Eu0J83O=Y4KA-LkzBMjtx7cbonxPzkiduzZ1Pedajg@mail.gmail.com/
> > > >
> > > > Yes. As mentioned in the v8 changelog [1], the binary search approach
> > > > you referenced was implemented in versions v6 and v7 [2]. However,
> > > > testing revealed a slight performance regression. The root cause was
> > > > an extra strcmp operation introduced in v7, as discussed in [3]. Therefore,
> > > > in v8, I reverted to the approach from v5 [4] and refactored it for clarity.
> > >
> > > If you keep oscillating like that this patch set will never land. 4%
> > > (500us) gain on artificial and unrealistic micro-benchmark is
> > > meaningless and irrelevant, you are just adding more work for yourself
> > > and for reviewers by constantly changing your implementation between
> > > revisions for no good reason.
> >
> > Thank you, I understand and will learn from it. I think the performance gain
> > makes sense. I’d like to share a specific real-world case where this
> > optimization
> > could matter: the `btf_find_by_name_kind()` function is indeed infrequently
> > used by the BPF subsystem, but it’s heavily relied upon by the ftrace
> > subsystem’s features like `func-args`, `funcgraph-args` [1], and the upcoming
> > `funcgraph-retval` [2]. These features invoke the function nearly once per
> > trace line when outputting, with a call frequency that can reach **100 kHz**
> > in intensive tracing workloads.
>
> Hi Andrii,
> I think we can refactor the code based on your suggestion like this:
>
> 1. If the binary search finds the matching name type, return its index.
> Else, return btf__type_cnt(btf). It will make the code streamlined.
> 2. Skip the name checking in the first loop to eliminate the extra strcmp.
>
> What do you think?
>
> tatic __s32 btf_find_by_name_bsearch(const struct btf *btf, const char *name,
> __s32 start_id)
> {
> const struct btf_type *t;
> const char *tname;
> __s32 end_id = btf__type_cnt(btf) - 1;
> __s32 l, r, m, lmost = end_id + 1;
> int ret;
>
> l = start_id;
> r = end_id;
> while (l <= r) {
> m = l + (r - l) / 2;
> t = btf_type_by_id(btf, m);
> tname = btf__str_by_offset(btf, t->name_off);
> ret = strcmp(tname, name);
> if (ret < 0) {
> l = m + 1;
> } else {
> if (ret == 0)
> lmost = m;
> r = m - 1;
> }
> }
>
> return lmost;
> }
>
> static __s32 btf_find_by_name_kind(const struct btf *btf, int start_id,
> const char *type_name, __u32 kind)
> {
> ......
> if (btf_is_sorted(btf) && type_name[0]) {
> bool first_loop = true;
>
> start_id = max(start_id, btf_sorted_start_id(btf));
> idx = btf_find_by_name_bsearch(btf, type_name, start_id);
> for (; idx < btf__type_cnt(btf); idx++) {
> t = btf__type_by_id(btf, idx);
> tname = btf__str_by_offset(btf, t->name_off);
> if (!first_loop && strcmp(tname, type_name) != 0)
> return libbpf_err(-ENOENT);
no, let's keep it simple, please revert to previous implementation we
agreed upon
> if (kind == -1 || btf_kind(t) == kind)
> return idx;
> if (first_loop)
> first_loop = false;
> }
> } else {
> ......
> }
>
> return libbpf_err(-ENOENT);
> }
>
> >
> > In such scenarios, the extra `strcmp` operations translate to ~100,000
> > additional
> > string comparisons per second. While this might seem negligible in isolation,
> > the overhead accumulates under high-frequency tracing—potentially impacting
> > latency for users relying on detailed function argument/return value tracing.
> >
> > Thanks again for pushing for rigor—it helps make the code more cleaner
> > and robust.
> >
> > [1] https://lore.kernel.org/all/20250227185822.639418500@goodmis.org/
> > [2] https://lore.kernel.org/all/20251215034153.2367756-1-dolinux.peng@gmail.com/
> >
[...]
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 04/13] libbpf: Optimize type lookup with binary search for sorted BTF
2026-01-06 0:38 ` Andrii Nakryiko
@ 2026-01-06 2:42 ` Donglin Peng
0 siblings, 0 replies; 72+ messages in thread
From: Donglin Peng @ 2026-01-06 2:42 UTC (permalink / raw)
To: Andrii Nakryiko
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Tue, Jan 6, 2026 at 8:38 AM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Sun, Dec 21, 2025 at 5:58 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
> >
> > On Sat, Dec 20, 2025 at 5:38 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
> > >
> > > On Sat, Dec 20, 2025 at 1:28 AM Andrii Nakryiko
> > > <andrii.nakryiko@gmail.com> wrote:
> > > >
> > > > On Thu, Dec 18, 2025 at 6:53 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
> > > > >
> > > > > On Fri, Dec 19, 2025 at 7:29 AM Andrii Nakryiko
> > > > > <andrii.nakryiko@gmail.com> wrote:
> > > > > >
> > > > > > On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
> > > > > > >
> > > > > > > From: pengdonglin <pengdonglin@xiaomi.com>
> > > > > > >
> > > > > > > This patch introduces binary search optimization for BTF type lookups
> > > > > > > when the BTF instance contains sorted types.
> > > > > > >
> > > > > > > The optimization significantly improves performance when searching for
> > > > > > > types in large BTF instances with sorted types. For unsorted BTF, the
> > > > > > > implementation falls back to the original linear search.
> > > > > > >
> > > > > > > Cc: Eduard Zingerman <eddyz87@gmail.com>
> > > > > > > Cc: Alexei Starovoitov <ast@kernel.org>
> > > > > > > Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > > > > > > Cc: Alan Maguire <alan.maguire@oracle.com>
> > > > > > > Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> > > > > > > Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> > > > > > > Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> > > > > > > ---
> > > > > > > tools/lib/bpf/btf.c | 103 ++++++++++++++++++++++++++++++++++----------
> > > > > > > 1 file changed, 80 insertions(+), 23 deletions(-)
> > > > > > >
> > > > > >
> > > > > > [...]
> > > > > >
> > > > > > > + l = start_id;
> > > > > > > + r = end_id;
> > > > > > > + while (l <= r) {
> > > > > > > + m = l + (r - l) / 2;
> > > > > > > + t = btf_type_by_id(btf, m);
> > > > > > > + tname = btf__str_by_offset(btf, t->name_off);
> > > > > > > + ret = strcmp(tname, name);
> > > > > > > + if (ret < 0) {
> > > > > > > + l = m + 1;
> > > > > > > + } else {
> > > > > > > + if (ret == 0)
> > > > > > > + lmost = m;
> > > > > > > + r = m - 1;
> > > > > > > + }
> > > > > > > }
> > > > > >
> > > > > > this differs from what we discussed in [0], you said you'll use that
> > > > > > approach. Can you please elaborate on why you didn't?
> > > > > >
> > > > > > [0] https://lore.kernel.org/bpf/CAEf4Bzb3Eu0J83O=Y4KA-LkzBMjtx7cbonxPzkiduzZ1Pedajg@mail.gmail.com/
> > > > >
> > > > > Yes. As mentioned in the v8 changelog [1], the binary search approach
> > > > > you referenced was implemented in versions v6 and v7 [2]. However,
> > > > > testing revealed a slight performance regression. The root cause was
> > > > > an extra strcmp operation introduced in v7, as discussed in [3]. Therefore,
> > > > > in v8, I reverted to the approach from v5 [4] and refactored it for clarity.
> > > >
> > > > If you keep oscillating like that this patch set will never land. 4%
> > > > (500us) gain on artificial and unrealistic micro-benchmark is
> > > > meaningless and irrelevant, you are just adding more work for yourself
> > > > and for reviewers by constantly changing your implementation between
> > > > revisions for no good reason.
> > >
> > > Thank you, I understand and will learn from it. I think the performance gain
> > > makes sense. I’d like to share a specific real-world case where this
> > > optimization
> > > could matter: the `btf_find_by_name_kind()` function is indeed infrequently
> > > used by the BPF subsystem, but it’s heavily relied upon by the ftrace
> > > subsystem’s features like `func-args`, `funcgraph-args` [1], and the upcoming
> > > `funcgraph-retval` [2]. These features invoke the function nearly once per
> > > trace line when outputting, with a call frequency that can reach **100 kHz**
> > > in intensive tracing workloads.
> >
> > Hi Andrii,
> > I think we can refactor the code based on your suggestion like this:
> >
> > 1. If the binary search finds the matching name type, return its index.
> > Else, return btf__type_cnt(btf). It will make the code streamlined.
> > 2. Skip the name checking in the first loop to eliminate the extra strcmp.
> >
> > What do you think?
> >
> > tatic __s32 btf_find_by_name_bsearch(const struct btf *btf, const char *name,
> > __s32 start_id)
> > {
> > const struct btf_type *t;
> > const char *tname;
> > __s32 end_id = btf__type_cnt(btf) - 1;
> > __s32 l, r, m, lmost = end_id + 1;
> > int ret;
> >
> > l = start_id;
> > r = end_id;
> > while (l <= r) {
> > m = l + (r - l) / 2;
> > t = btf_type_by_id(btf, m);
> > tname = btf__str_by_offset(btf, t->name_off);
> > ret = strcmp(tname, name);
> > if (ret < 0) {
> > l = m + 1;
> > } else {
> > if (ret == 0)
> > lmost = m;
> > r = m - 1;
> > }
> > }
> >
> > return lmost;
> > }
> >
> > static __s32 btf_find_by_name_kind(const struct btf *btf, int start_id,
> > const char *type_name, __u32 kind)
> > {
> > ......
> > if (btf_is_sorted(btf) && type_name[0]) {
> > bool first_loop = true;
> >
> > start_id = max(start_id, btf_sorted_start_id(btf));
> > idx = btf_find_by_name_bsearch(btf, type_name, start_id);
> > for (; idx < btf__type_cnt(btf); idx++) {
> > t = btf__type_by_id(btf, idx);
> > tname = btf__str_by_offset(btf, t->name_off);
> > if (!first_loop && strcmp(tname, type_name) != 0)
> > return libbpf_err(-ENOENT);
>
> no, let's keep it simple, please revert to previous implementation we
> agreed upon
Okay. I will do it.
>
> > if (kind == -1 || btf_kind(t) == kind)
> > return idx;
> > if (first_loop)
> > first_loop = false;
> > }
> > } else {
> > ......
> > }
> >
> > return libbpf_err(-ENOENT);
> > }
> >
> > >
> > > In such scenarios, the extra `strcmp` operations translate to ~100,000
> > > additional
> > > string comparisons per second. While this might seem negligible in isolation,
> > > the overhead accumulates under high-frequency tracing—potentially impacting
> > > latency for users relying on detailed function argument/return value tracing.
> > >
> > > Thanks again for pushing for rigor—it helps make the code more cleaner
> > > and robust.
> > >
> > > [1] https://lore.kernel.org/all/20250227185822.639418500@goodmis.org/
> > > [2] https://lore.kernel.org/all/20251215034153.2367756-1-dolinux.peng@gmail.com/
> > >
>
> [...]
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 04/13] libbpf: Optimize type lookup with binary search for sorted BTF
2026-01-06 0:36 ` Andrii Nakryiko
@ 2026-01-06 2:50 ` Donglin Peng
0 siblings, 0 replies; 72+ messages in thread
From: Donglin Peng @ 2026-01-06 2:50 UTC (permalink / raw)
To: Andrii Nakryiko
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Tue, Jan 6, 2026 at 8:36 AM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Sat, Dec 20, 2025 at 1:38 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
> >
> > On Sat, Dec 20, 2025 at 1:28 AM Andrii Nakryiko
> > <andrii.nakryiko@gmail.com> wrote:
> > >
> > > On Thu, Dec 18, 2025 at 6:53 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
> > > >
> > > > On Fri, Dec 19, 2025 at 7:29 AM Andrii Nakryiko
> > > > <andrii.nakryiko@gmail.com> wrote:
> > > > >
> > > > > On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
> > > > > >
> > > > > > From: pengdonglin <pengdonglin@xiaomi.com>
> > > > > >
> > > > > > This patch introduces binary search optimization for BTF type lookups
> > > > > > when the BTF instance contains sorted types.
> > > > > >
> > > > > > The optimization significantly improves performance when searching for
> > > > > > types in large BTF instances with sorted types. For unsorted BTF, the
> > > > > > implementation falls back to the original linear search.
> > > > > >
> > > > > > Cc: Eduard Zingerman <eddyz87@gmail.com>
> > > > > > Cc: Alexei Starovoitov <ast@kernel.org>
> > > > > > Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > > > > > Cc: Alan Maguire <alan.maguire@oracle.com>
> > > > > > Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> > > > > > Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> > > > > > Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> > > > > > ---
> > > > > > tools/lib/bpf/btf.c | 103 ++++++++++++++++++++++++++++++++++----------
> > > > > > 1 file changed, 80 insertions(+), 23 deletions(-)
> > > > > >
> > > > >
> > > > > [...]
> > > > >
> > > > > > + l = start_id;
> > > > > > + r = end_id;
> > > > > > + while (l <= r) {
> > > > > > + m = l + (r - l) / 2;
> > > > > > + t = btf_type_by_id(btf, m);
> > > > > > + tname = btf__str_by_offset(btf, t->name_off);
> > > > > > + ret = strcmp(tname, name);
> > > > > > + if (ret < 0) {
> > > > > > + l = m + 1;
> > > > > > + } else {
> > > > > > + if (ret == 0)
> > > > > > + lmost = m;
> > > > > > + r = m - 1;
> > > > > > + }
> > > > > > }
> > > > >
> > > > > this differs from what we discussed in [0], you said you'll use that
> > > > > approach. Can you please elaborate on why you didn't?
> > > > >
> > > > > [0] https://lore.kernel.org/bpf/CAEf4Bzb3Eu0J83O=Y4KA-LkzBMjtx7cbonxPzkiduzZ1Pedajg@mail.gmail.com/
> > > >
> > > > Yes. As mentioned in the v8 changelog [1], the binary search approach
> > > > you referenced was implemented in versions v6 and v7 [2]. However,
> > > > testing revealed a slight performance regression. The root cause was
> > > > an extra strcmp operation introduced in v7, as discussed in [3]. Therefore,
> > > > in v8, I reverted to the approach from v5 [4] and refactored it for clarity.
> > >
> > > If you keep oscillating like that this patch set will never land. 4%
> > > (500us) gain on artificial and unrealistic micro-benchmark is
> > > meaningless and irrelevant, you are just adding more work for yourself
> > > and for reviewers by constantly changing your implementation between
> > > revisions for no good reason.
> >
> > Thank you, I understand and will learn from it. I think the performance gain
> > makes sense. I’d like to share a specific real-world case where this
> > optimization
> > could matter: the `btf_find_by_name_kind()` function is indeed infrequently
> > used by the BPF subsystem, but it’s heavily relied upon by the ftrace
> > subsystem’s features like `func-args`, `funcgraph-args` [1], and the upcoming
> > `funcgraph-retval` [2]. These features invoke the function nearly once per
> > trace line when outputting, with a call frequency that can reach **100 kHz**
> > in intensive tracing workloads.
> >
> > In such scenarios, the extra `strcmp` operations translate to ~100,000
> > additional
> > string comparisons per second. While this might seem negligible in isolation,
>
> I'm pretty confident it is quite negligible compared to all other
> *useful* work you'll have to do to use that BTF type to format those
> arguments. Yes, 100KHz is pretty frequent, but it won't be anywhere
> close to 4% if you profile the entire end-to-end overhead of emitting
> arguments using func-args.
Thanks, I agree.
>
> > the overhead accumulates under high-frequency tracing—potentially impacting
> > latency for users relying on detailed function argument/return value tracing.
> >
> > Thanks again for pushing for rigor—it helps make the code more cleaner
> > and robust.
> >
> > [1] https://lore.kernel.org/all/20250227185822.639418500@goodmis.org/
> > [2] https://lore.kernel.org/all/20251215034153.2367756-1-dolinux.peng@gmail.com/
> >
> > >
> > >
> > > >
> > > > Benchmark results show that v8 achieves a 4.2% performance improvement
> > > > over v7. If we don't care the performance gain, I will revert to the approach
> > > > in v7 in the next version.
> > > >
> > > > [1] https://lore.kernel.org/bpf/20251126085025.784288-1-dolinux.peng@gmail.com/
> > > > [2] https://lore.kernel.org/all/20251119031531.1817099-1-dolinux.peng@gmail.com/
> > > > [3] https://lore.kernel.org/all/CAEf4BzaqEPD46LddJHO1-k5KPGyVWf6d=duDAxG1q=jykJkMBg@mail.gmail.com/
> > > > [4] https://lore.kernel.org/all/20251106131956.1222864-4-dolinux.peng@gmail.com/
> > > >
>
> [...]
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 05/13] libbpf: Verify BTF Sorting
[not found] ` <CAErzpmsvirekLBRrJYVgmRC0YKWCbo7OyRQXgNYrk83aF-Wz2Q@mail.gmail.com>
@ 2026-01-07 3:45 ` Donglin Peng
2026-01-07 21:50 ` Andrii Nakryiko
0 siblings, 1 reply; 72+ messages in thread
From: Donglin Peng @ 2026-01-07 3:45 UTC (permalink / raw)
To: Andrii Nakryiko
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Sat, Dec 20, 2025 at 10:36 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
>
>
>
> On Sat, Dec 20, 2025 at 1:33 AM Andrii Nakryiko <andrii.nakryiko@gmail.com> wrote:
> >
> > On Thu, Dec 18, 2025 at 9:06 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
> > >
> > > On Fri, Dec 19, 2025 at 7:44 AM Andrii Nakryiko
> > > <andrii.nakryiko@gmail.com> wrote:
> > > >
> > > > On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
> > > > >
> > > > > From: pengdonglin <pengdonglin@xiaomi.com>
> > > > >
> > > >
> > > > typo in subject: "Sorting" -> "sorting", it looks weird capitalized like that
> > >
> > > Thanks, I will do it.
> > >
> > > >
> > > > > This patch checks whether the BTF is sorted by name in ascending
> > > > > order. If sorted, binary search will be used when looking up types.
> > > > >
> > > > > Cc: Eduard Zingerman <eddyz87@gmail.com>
> > > > > Cc: Alexei Starovoitov <ast@kernel.org>
> > > > > Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > > > > Cc: Alan Maguire <alan.maguire@oracle.com>
> > > > > Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> > > > > Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> > > > > Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> > > > > ---
> > > > > tools/lib/bpf/btf.c | 41 +++++++++++++++++++++++++++++++++++++++++
> > > > > 1 file changed, 41 insertions(+)
> > > > >
> > > > > diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> > > > > index 2facb57d7e5f..c63d46b7d74b 100644
> > > > > --- a/tools/lib/bpf/btf.c
> > > > > +++ b/tools/lib/bpf/btf.c
> > > > > @@ -899,6 +899,46 @@ int btf__resolve_type(const struct btf *btf, __u32 type_id)
> > > > > return type_id;
> > > > > }
> > > > >
> > > > > +/*
> > > > > + * Assuming that types are sorted by name in ascending order.
> > > > > + */
> > > >
> > > > Unnecessary comment, and no, btf_compare_type_names() itself makes no
> > > > such assumption, it just compares two provided types by name. Drop the
> > > > comment, please.
> > > >
> > > > > +static int btf_compare_type_names(__u32 *a, __u32 *b, const struct btf *btf)
> > > > > +{
> > > > > + struct btf_type *ta = btf_type_by_id(btf, *a);
> > > > > + struct btf_type *tb = btf_type_by_id(btf, *b);
> > > > > + const char *na, *nb;
> > > > > +
> > > > > + na = btf__str_by_offset(btf, ta->name_off);
> > > > > + nb = btf__str_by_offset(btf, tb->name_off);
> > > > > + return strcmp(na, nb);
> > > > > +}
> > > >
> > > > you use this function only in one place, there is no real point having
> > > > it, especially that it uses **a pointer to type ID** as an
> > > > interface... just inline its logic in that one loop below
> > > >
> > > > > +
> > > > > +static void btf_check_sorted(struct btf *btf)
> > > > > +{
> > > > > + const struct btf_type *t;
> > > > > + __u32 i, k, n;
> > > > > + __u32 sorted_start_id;
> > > > > +
> > > > > + if (btf->nr_types < 2)
> > > > > + return;
> > > >
> > > > why special casing? does it not work with nr_types = 0 or nr_types = 1?
> > >
> > > No. I just think it doesn't make any sense to check the sorting
> > > of BTF with zero or only one type.
> > >
> >
> > Look, I don't know how to emphasize this enough. Any special case like
> > this is nothing good, it adds more cases to consider and raises
> > questions why generic case code doesn't handle such special cases. It
> > implies that generic case handling might have some unhandled corner
> > case that we are trying to short-circuit with early exits like this.
> > It just basically means we don't trust our code, in a sense. It's just
> > unnecessary and thus sloppy. Don't do this unnecessarily.
>
> Thank you for taking the time to explain this so clearly. I’m grateful for
> your patience in pushing for rigor — it’s exactly the kind of scrutiny that
> makes the codebase stronger. I’ll make sure to apply this lesson in future
> revisions.
>
> >
> > > >
> > > > > +
> > > > > + sorted_start_id = 0;
> > > >
> > > > nit: initialize in declaration
> > >
> > > Thanks, I will do it.
> > >
> > > >
> > > >
> > > > > + n = btf__type_cnt(btf);
> > > > > + for (i = btf->start_id; i < n; i++) {
> > > > > + k = i + 1;
> > > > > + if (k < n && btf_compare_type_names(&i, &k, btf) > 0)
> > > > > + return;
> > > > > + if (sorted_start_id == 0) {
> > > > > + t = btf_type_by_id(btf, i);
> > > > > + if (t->name_off)
> > > >
> > > > I'd check actual string, not name_off. Technically, you can have empty
> > > > string with non-zero name_off, so why assume anything here?
> > >
> > > Thanks, I will do it.
> > >
> > > >
> > > > > + sorted_start_id = i;
> > > > > + }
> > > > > + }
> > > > > +
> > > > > + if (sorted_start_id)
> > > > > + btf->sorted_start_id = sorted_start_id;
> > > >
> > > > You actually made code more complicated by extracting that
> > > > btf_compare_type_names(). Compare to:
> > > >
> > > > n = btf__type_cnt(btf);
> > > > btf->sorted_start_id = 0;
> > > > for (i = btf->start_id + 1; i < n; i++) {
> > > > struct btf_type *t1 = btf_type_by_id(btf, i - 1);
> > > > struct btf_type *t2 = btf_type_by_id(btf, i);
> > > > const char *n1 = btf__str_by_offset(btf, t1->name_off);
> > > > const char *n2 = btf__str_by_offset(btf, t2->name_off);
> > > >
> > > > if (strcmp(n1, n2) > 0)
> > > > return;
> > > > if (btf->sorted_start_id == 0 && n1[0] != '\0')
> > > > btf->sorted_start_id = i - 1;
> > > > }
> > >
> > > Thanks. I believe we shouldn't directly assign a value to
> > > `btf->sorted_start_id` within the for loop, because
> > > `btf->sorted_start_id` might be non-zero even when the
> > > BTF isn't sorted.
> >
> > Ah, right, we'd need to reset btf->sorted_start_id to zero in that
> > strcmp(n1, n2) > 0 branch. Using btf->sorted_start_id directly is not
> > the main point here, though, feel free to use a local variable, but
> > don't add unnecessary helper functions which don't do much, but
> > obscure the logic unnecessarily.
>
> Thanks, I will use a local variable and remove the helper functions.
Hi Andrii,
I noticed that the code above does not handle the scenario where
only the last btf_typein a sorted BTF has a name. So I've added an
additional check in the loop as follows:
static void btf_check_sorted(struct btf *btf)
{
__u32 i, n, named_start_id = 0;
n = btf__type_cnt(btf);
for (i = btf->start_id + 1; i < n; i++) {
struct btf_type *ta = btf_type_by_id(btf, i - 1);
struct btf_type *tb = btf_type_by_id(btf, i);
const char *na = btf__str_by_offset(btf, ta->name_off);
const char *nb = btf__str_by_offset(btf, tb->name_off);
if (strcmp(na, nb) > 0)
return;
if (named_start_id == 0) {
if (na[0] != '\0')
named_start_id = i - 1;
else if (i == (n - 1) && nb[0] != '\0') /*
only the last one has a name */
named_start_id = i;
}
}
if (named_start_id)
btf->named_start_id = named_start_id;
}
WDYT?
>
> >
> > >
> > > >
> > > >
> > > > No extra k<n checks, no extra type_by_id lookups. It's minimalistic
> > > > and cleaner. And if it so happens that we get single type BTF that is
> > > > technically sorted, it doesn't matter, we always fallback to faster
> > > > linear search anyways.
> > > >
> > > > Keep it simple.
> > >
> > > Thank you. I will adopt this method in the next version.
> > >
> > > >
> > > > > +}
> > > > > +
> > > > > static __s32 btf_find_by_name_bsearch(const struct btf *btf, const char *name,
> > > > > __s32 start_id, __s32 end_id)
> > > > > {
> > > > > @@ -1147,6 +1187,7 @@ static struct btf *btf_new(const void *data, __u32 size, struct btf *base_btf, b
> > > > > err = err ?: btf_sanity_check(btf);
> > > > > if (err)
> > > > > goto done;
> > > > > + btf_check_sorted(btf);
> > > > >
> > > > > done:
> > > > > if (err) {
> > > > > --
> > > > > 2.34.1
> > > > >
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 05/13] libbpf: Verify BTF Sorting
2026-01-07 3:45 ` Donglin Peng
@ 2026-01-07 21:50 ` Andrii Nakryiko
2026-01-08 1:50 ` Donglin Peng
0 siblings, 1 reply; 72+ messages in thread
From: Andrii Nakryiko @ 2026-01-07 21:50 UTC (permalink / raw)
To: Donglin Peng
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Tue, Jan 6, 2026 at 7:45 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
>
> On Sat, Dec 20, 2025 at 10:36 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
> >
> >
> >
> > On Sat, Dec 20, 2025 at 1:33 AM Andrii Nakryiko <andrii.nakryiko@gmail.com> wrote:
> > >
> > > On Thu, Dec 18, 2025 at 9:06 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
> > > >
> > > > On Fri, Dec 19, 2025 at 7:44 AM Andrii Nakryiko
> > > > <andrii.nakryiko@gmail.com> wrote:
> > > > >
> > > > > On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
> > > > > >
> > > > > > From: pengdonglin <pengdonglin@xiaomi.com>
> > > > > >
> > > > >
> > > > > typo in subject: "Sorting" -> "sorting", it looks weird capitalized like that
> > > >
> > > > Thanks, I will do it.
> > > >
> > > > >
> > > > > > This patch checks whether the BTF is sorted by name in ascending
> > > > > > order. If sorted, binary search will be used when looking up types.
> > > > > >
> > > > > > Cc: Eduard Zingerman <eddyz87@gmail.com>
> > > > > > Cc: Alexei Starovoitov <ast@kernel.org>
> > > > > > Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > > > > > Cc: Alan Maguire <alan.maguire@oracle.com>
> > > > > > Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> > > > > > Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> > > > > > Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> > > > > > ---
> > > > > > tools/lib/bpf/btf.c | 41 +++++++++++++++++++++++++++++++++++++++++
> > > > > > 1 file changed, 41 insertions(+)
> > > > > >
> > > > > > diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> > > > > > index 2facb57d7e5f..c63d46b7d74b 100644
> > > > > > --- a/tools/lib/bpf/btf.c
> > > > > > +++ b/tools/lib/bpf/btf.c
> > > > > > @@ -899,6 +899,46 @@ int btf__resolve_type(const struct btf *btf, __u32 type_id)
> > > > > > return type_id;
> > > > > > }
> > > > > >
> > > > > > +/*
> > > > > > + * Assuming that types are sorted by name in ascending order.
> > > > > > + */
> > > > >
> > > > > Unnecessary comment, and no, btf_compare_type_names() itself makes no
> > > > > such assumption, it just compares two provided types by name. Drop the
> > > > > comment, please.
> > > > >
> > > > > > +static int btf_compare_type_names(__u32 *a, __u32 *b, const struct btf *btf)
> > > > > > +{
> > > > > > + struct btf_type *ta = btf_type_by_id(btf, *a);
> > > > > > + struct btf_type *tb = btf_type_by_id(btf, *b);
> > > > > > + const char *na, *nb;
> > > > > > +
> > > > > > + na = btf__str_by_offset(btf, ta->name_off);
> > > > > > + nb = btf__str_by_offset(btf, tb->name_off);
> > > > > > + return strcmp(na, nb);
> > > > > > +}
> > > > >
> > > > > you use this function only in one place, there is no real point having
> > > > > it, especially that it uses **a pointer to type ID** as an
> > > > > interface... just inline its logic in that one loop below
> > > > >
> > > > > > +
> > > > > > +static void btf_check_sorted(struct btf *btf)
> > > > > > +{
> > > > > > + const struct btf_type *t;
> > > > > > + __u32 i, k, n;
> > > > > > + __u32 sorted_start_id;
> > > > > > +
> > > > > > + if (btf->nr_types < 2)
> > > > > > + return;
> > > > >
> > > > > why special casing? does it not work with nr_types = 0 or nr_types = 1?
> > > >
> > > > No. I just think it doesn't make any sense to check the sorting
> > > > of BTF with zero or only one type.
> > > >
> > >
> > > Look, I don't know how to emphasize this enough. Any special case like
> > > this is nothing good, it adds more cases to consider and raises
> > > questions why generic case code doesn't handle such special cases. It
> > > implies that generic case handling might have some unhandled corner
> > > case that we are trying to short-circuit with early exits like this.
> > > It just basically means we don't trust our code, in a sense. It's just
> > > unnecessary and thus sloppy. Don't do this unnecessarily.
> >
> > Thank you for taking the time to explain this so clearly. I’m grateful for
> > your patience in pushing for rigor — it’s exactly the kind of scrutiny that
> > makes the codebase stronger. I’ll make sure to apply this lesson in future
> > revisions.
> >
> > >
> > > > >
> > > > > > +
> > > > > > + sorted_start_id = 0;
> > > > >
> > > > > nit: initialize in declaration
> > > >
> > > > Thanks, I will do it.
> > > >
> > > > >
> > > > >
> > > > > > + n = btf__type_cnt(btf);
> > > > > > + for (i = btf->start_id; i < n; i++) {
> > > > > > + k = i + 1;
> > > > > > + if (k < n && btf_compare_type_names(&i, &k, btf) > 0)
> > > > > > + return;
> > > > > > + if (sorted_start_id == 0) {
> > > > > > + t = btf_type_by_id(btf, i);
> > > > > > + if (t->name_off)
> > > > >
> > > > > I'd check actual string, not name_off. Technically, you can have empty
> > > > > string with non-zero name_off, so why assume anything here?
> > > >
> > > > Thanks, I will do it.
> > > >
> > > > >
> > > > > > + sorted_start_id = i;
> > > > > > + }
> > > > > > + }
> > > > > > +
> > > > > > + if (sorted_start_id)
> > > > > > + btf->sorted_start_id = sorted_start_id;
> > > > >
> > > > > You actually made code more complicated by extracting that
> > > > > btf_compare_type_names(). Compare to:
> > > > >
> > > > > n = btf__type_cnt(btf);
> > > > > btf->sorted_start_id = 0;
> > > > > for (i = btf->start_id + 1; i < n; i++) {
> > > > > struct btf_type *t1 = btf_type_by_id(btf, i - 1);
> > > > > struct btf_type *t2 = btf_type_by_id(btf, i);
> > > > > const char *n1 = btf__str_by_offset(btf, t1->name_off);
> > > > > const char *n2 = btf__str_by_offset(btf, t2->name_off);
> > > > >
> > > > > if (strcmp(n1, n2) > 0)
> > > > > return;
> > > > > if (btf->sorted_start_id == 0 && n1[0] != '\0')
> > > > > btf->sorted_start_id = i - 1;
> > > > > }
> > > >
> > > > Thanks. I believe we shouldn't directly assign a value to
> > > > `btf->sorted_start_id` within the for loop, because
> > > > `btf->sorted_start_id` might be non-zero even when the
> > > > BTF isn't sorted.
> > >
> > > Ah, right, we'd need to reset btf->sorted_start_id to zero in that
> > > strcmp(n1, n2) > 0 branch. Using btf->sorted_start_id directly is not
> > > the main point here, though, feel free to use a local variable, but
> > > don't add unnecessary helper functions which don't do much, but
> > > obscure the logic unnecessarily.
> >
> > Thanks, I will use a local variable and remove the helper functions.
>
> Hi Andrii,
>
> I noticed that the code above does not handle the scenario where
> only the last btf_typein a sorted BTF has a name. So I've added an
> additional check in the loop as follows:
>
> static void btf_check_sorted(struct btf *btf)
> {
> __u32 i, n, named_start_id = 0;
>
> n = btf__type_cnt(btf);
> for (i = btf->start_id + 1; i < n; i++) {
> struct btf_type *ta = btf_type_by_id(btf, i - 1);
> struct btf_type *tb = btf_type_by_id(btf, i);
> const char *na = btf__str_by_offset(btf, ta->name_off);
> const char *nb = btf__str_by_offset(btf, tb->name_off);
>
> if (strcmp(na, nb) > 0)
> return;
>
> if (named_start_id == 0) {
> if (na[0] != '\0')
> named_start_id = i - 1;
> else if (i == (n - 1) && nb[0] != '\0') /*
> only the last one has a name */
> named_start_id = i;
> }
good observation!
How about a bit more uniform way?
if (named_start_id == 0 && na[0] != '\0')
named_start_id = i - 1;
if (named_start_id == 0 && nb[0] != '\0')
named_start_id = i;
less nested conditions, and code clearly says that we pick earliest
non-anon type in a comparison pair as named_start_id
> }
>
> if (named_start_id)
> btf->named_start_id = named_start_id;
> }
>
> WDYT?
>
> >
> > >
> > > >
> > > > >
> > > > >
> > > > > No extra k<n checks, no extra type_by_id lookups. It's minimalistic
> > > > > and cleaner. And if it so happens that we get single type BTF that is
> > > > > technically sorted, it doesn't matter, we always fallback to faster
> > > > > linear search anyways.
> > > > >
> > > > > Keep it simple.
> > > >
> > > > Thank you. I will adopt this method in the next version.
> > > >
> > > > >
> > > > > > +}
> > > > > > +
> > > > > > static __s32 btf_find_by_name_bsearch(const struct btf *btf, const char *name,
> > > > > > __s32 start_id, __s32 end_id)
> > > > > > {
> > > > > > @@ -1147,6 +1187,7 @@ static struct btf *btf_new(const void *data, __u32 size, struct btf *base_btf, b
> > > > > > err = err ?: btf_sanity_check(btf);
> > > > > > if (err)
> > > > > > goto done;
> > > > > > + btf_check_sorted(btf);
> > > > > >
> > > > > > done:
> > > > > > if (err) {
> > > > > > --
> > > > > > 2.34.1
> > > > > >
^ permalink raw reply [flat|nested] 72+ messages in thread
* Re: [PATCH bpf-next v10 05/13] libbpf: Verify BTF Sorting
2026-01-07 21:50 ` Andrii Nakryiko
@ 2026-01-08 1:50 ` Donglin Peng
0 siblings, 0 replies; 72+ messages in thread
From: Donglin Peng @ 2026-01-08 1:50 UTC (permalink / raw)
To: Andrii Nakryiko
Cc: ast, eddyz87, zhangxiaoqin, ihor.solodrai, linux-kernel, bpf,
pengdonglin, Alan Maguire
On Thu, Jan 8, 2026 at 5:51 AM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Tue, Jan 6, 2026 at 7:45 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
> >
> > On Sat, Dec 20, 2025 at 10:36 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
> > >
> > >
> > >
> > > On Sat, Dec 20, 2025 at 1:33 AM Andrii Nakryiko <andrii.nakryiko@gmail.com> wrote:
> > > >
> > > > On Thu, Dec 18, 2025 at 9:06 PM Donglin Peng <dolinux.peng@gmail.com> wrote:
> > > > >
> > > > > On Fri, Dec 19, 2025 at 7:44 AM Andrii Nakryiko
> > > > > <andrii.nakryiko@gmail.com> wrote:
> > > > > >
> > > > > > On Thu, Dec 18, 2025 at 3:31 AM Donglin Peng <dolinux.peng@gmail.com> wrote:
> > > > > > >
> > > > > > > From: pengdonglin <pengdonglin@xiaomi.com>
> > > > > > >
> > > > > >
> > > > > > typo in subject: "Sorting" -> "sorting", it looks weird capitalized like that
> > > > >
> > > > > Thanks, I will do it.
> > > > >
> > > > > >
> > > > > > > This patch checks whether the BTF is sorted by name in ascending
> > > > > > > order. If sorted, binary search will be used when looking up types.
> > > > > > >
> > > > > > > Cc: Eduard Zingerman <eddyz87@gmail.com>
> > > > > > > Cc: Alexei Starovoitov <ast@kernel.org>
> > > > > > > Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
> > > > > > > Cc: Alan Maguire <alan.maguire@oracle.com>
> > > > > > > Cc: Ihor Solodrai <ihor.solodrai@linux.dev>
> > > > > > > Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
> > > > > > > Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
> > > > > > > ---
> > > > > > > tools/lib/bpf/btf.c | 41 +++++++++++++++++++++++++++++++++++++++++
> > > > > > > 1 file changed, 41 insertions(+)
> > > > > > >
> > > > > > > diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> > > > > > > index 2facb57d7e5f..c63d46b7d74b 100644
> > > > > > > --- a/tools/lib/bpf/btf.c
> > > > > > > +++ b/tools/lib/bpf/btf.c
> > > > > > > @@ -899,6 +899,46 @@ int btf__resolve_type(const struct btf *btf, __u32 type_id)
> > > > > > > return type_id;
> > > > > > > }
> > > > > > >
> > > > > > > +/*
> > > > > > > + * Assuming that types are sorted by name in ascending order.
> > > > > > > + */
> > > > > >
> > > > > > Unnecessary comment, and no, btf_compare_type_names() itself makes no
> > > > > > such assumption, it just compares two provided types by name. Drop the
> > > > > > comment, please.
> > > > > >
> > > > > > > +static int btf_compare_type_names(__u32 *a, __u32 *b, const struct btf *btf)
> > > > > > > +{
> > > > > > > + struct btf_type *ta = btf_type_by_id(btf, *a);
> > > > > > > + struct btf_type *tb = btf_type_by_id(btf, *b);
> > > > > > > + const char *na, *nb;
> > > > > > > +
> > > > > > > + na = btf__str_by_offset(btf, ta->name_off);
> > > > > > > + nb = btf__str_by_offset(btf, tb->name_off);
> > > > > > > + return strcmp(na, nb);
> > > > > > > +}
> > > > > >
> > > > > > you use this function only in one place, there is no real point having
> > > > > > it, especially that it uses **a pointer to type ID** as an
> > > > > > interface... just inline its logic in that one loop below
> > > > > >
> > > > > > > +
> > > > > > > +static void btf_check_sorted(struct btf *btf)
> > > > > > > +{
> > > > > > > + const struct btf_type *t;
> > > > > > > + __u32 i, k, n;
> > > > > > > + __u32 sorted_start_id;
> > > > > > > +
> > > > > > > + if (btf->nr_types < 2)
> > > > > > > + return;
> > > > > >
> > > > > > why special casing? does it not work with nr_types = 0 or nr_types = 1?
> > > > >
> > > > > No. I just think it doesn't make any sense to check the sorting
> > > > > of BTF with zero or only one type.
> > > > >
> > > >
> > > > Look, I don't know how to emphasize this enough. Any special case like
> > > > this is nothing good, it adds more cases to consider and raises
> > > > questions why generic case code doesn't handle such special cases. It
> > > > implies that generic case handling might have some unhandled corner
> > > > case that we are trying to short-circuit with early exits like this.
> > > > It just basically means we don't trust our code, in a sense. It's just
> > > > unnecessary and thus sloppy. Don't do this unnecessarily.
> > >
> > > Thank you for taking the time to explain this so clearly. I’m grateful for
> > > your patience in pushing for rigor — it’s exactly the kind of scrutiny that
> > > makes the codebase stronger. I’ll make sure to apply this lesson in future
> > > revisions.
> > >
> > > >
> > > > > >
> > > > > > > +
> > > > > > > + sorted_start_id = 0;
> > > > > >
> > > > > > nit: initialize in declaration
> > > > >
> > > > > Thanks, I will do it.
> > > > >
> > > > > >
> > > > > >
> > > > > > > + n = btf__type_cnt(btf);
> > > > > > > + for (i = btf->start_id; i < n; i++) {
> > > > > > > + k = i + 1;
> > > > > > > + if (k < n && btf_compare_type_names(&i, &k, btf) > 0)
> > > > > > > + return;
> > > > > > > + if (sorted_start_id == 0) {
> > > > > > > + t = btf_type_by_id(btf, i);
> > > > > > > + if (t->name_off)
> > > > > >
> > > > > > I'd check actual string, not name_off. Technically, you can have empty
> > > > > > string with non-zero name_off, so why assume anything here?
> > > > >
> > > > > Thanks, I will do it.
> > > > >
> > > > > >
> > > > > > > + sorted_start_id = i;
> > > > > > > + }
> > > > > > > + }
> > > > > > > +
> > > > > > > + if (sorted_start_id)
> > > > > > > + btf->sorted_start_id = sorted_start_id;
> > > > > >
> > > > > > You actually made code more complicated by extracting that
> > > > > > btf_compare_type_names(). Compare to:
> > > > > >
> > > > > > n = btf__type_cnt(btf);
> > > > > > btf->sorted_start_id = 0;
> > > > > > for (i = btf->start_id + 1; i < n; i++) {
> > > > > > struct btf_type *t1 = btf_type_by_id(btf, i - 1);
> > > > > > struct btf_type *t2 = btf_type_by_id(btf, i);
> > > > > > const char *n1 = btf__str_by_offset(btf, t1->name_off);
> > > > > > const char *n2 = btf__str_by_offset(btf, t2->name_off);
> > > > > >
> > > > > > if (strcmp(n1, n2) > 0)
> > > > > > return;
> > > > > > if (btf->sorted_start_id == 0 && n1[0] != '\0')
> > > > > > btf->sorted_start_id = i - 1;
> > > > > > }
> > > > >
> > > > > Thanks. I believe we shouldn't directly assign a value to
> > > > > `btf->sorted_start_id` within the for loop, because
> > > > > `btf->sorted_start_id` might be non-zero even when the
> > > > > BTF isn't sorted.
> > > >
> > > > Ah, right, we'd need to reset btf->sorted_start_id to zero in that
> > > > strcmp(n1, n2) > 0 branch. Using btf->sorted_start_id directly is not
> > > > the main point here, though, feel free to use a local variable, but
> > > > don't add unnecessary helper functions which don't do much, but
> > > > obscure the logic unnecessarily.
> > >
> > > Thanks, I will use a local variable and remove the helper functions.
> >
> > Hi Andrii,
> >
> > I noticed that the code above does not handle the scenario where
> > only the last btf_typein a sorted BTF has a name. So I've added an
> > additional check in the loop as follows:
> >
> > static void btf_check_sorted(struct btf *btf)
> > {
> > __u32 i, n, named_start_id = 0;
> >
> > n = btf__type_cnt(btf);
> > for (i = btf->start_id + 1; i < n; i++) {
> > struct btf_type *ta = btf_type_by_id(btf, i - 1);
> > struct btf_type *tb = btf_type_by_id(btf, i);
> > const char *na = btf__str_by_offset(btf, ta->name_off);
> > const char *nb = btf__str_by_offset(btf, tb->name_off);
> >
> > if (strcmp(na, nb) > 0)
> > return;
> >
> > if (named_start_id == 0) {
> > if (na[0] != '\0')
> > named_start_id = i - 1;
> > else if (i == (n - 1) && nb[0] != '\0') /*
> > only the last one has a name */
> > named_start_id = i;
> > }
>
> good observation!
>
> How about a bit more uniform way?
>
> if (named_start_id == 0 && na[0] != '\0')
> named_start_id = i - 1;
> if (named_start_id == 0 && nb[0] != '\0')
> named_start_id = i;
>
>
> less nested conditions, and code clearly says that we pick earliest
> non-anon type in a comparison pair as named_start_id
Thanks, I will apply it in the next version.
>
> > }
> >
> > if (named_start_id)
> > btf->named_start_id = named_start_id;
> > }
> >
> > WDYT?
> >
> > >
> > > >
> > > > >
> > > > > >
> > > > > >
> > > > > > No extra k<n checks, no extra type_by_id lookups. It's minimalistic
> > > > > > and cleaner. And if it so happens that we get single type BTF that is
> > > > > > technically sorted, it doesn't matter, we always fallback to faster
> > > > > > linear search anyways.
> > > > > >
> > > > > > Keep it simple.
> > > > >
> > > > > Thank you. I will adopt this method in the next version.
> > > > >
> > > > > >
> > > > > > > +}
> > > > > > > +
> > > > > > > static __s32 btf_find_by_name_bsearch(const struct btf *btf, const char *name,
> > > > > > > __s32 start_id, __s32 end_id)
> > > > > > > {
> > > > > > > @@ -1147,6 +1187,7 @@ static struct btf *btf_new(const void *data, __u32 size, struct btf *base_btf, b
> > > > > > > err = err ?: btf_sanity_check(btf);
> > > > > > > if (err)
> > > > > > > goto done;
> > > > > > > + btf_check_sorted(btf);
> > > > > > >
> > > > > > > done:
> > > > > > > if (err) {
> > > > > > > --
> > > > > > > 2.34.1
> > > > > > >
^ permalink raw reply [flat|nested] 72+ messages in thread
end of thread, other threads:[~2026-01-08 1:50 UTC | newest]
Thread overview: 72+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-12-18 11:30 [PATCH bpf-next v10 00/13] Improve the performance of BTF type lookups with binary search Donglin Peng
2025-12-18 11:30 ` [PATCH bpf-next v10 01/13] libbpf: Add BTF permutation support for type reordering Donglin Peng
2025-12-18 23:02 ` Andrii Nakryiko
2025-12-19 3:14 ` Donglin Peng
2025-12-19 3:16 ` Donglin Peng
2025-12-19 17:07 ` Andrii Nakryiko
2025-12-20 8:39 ` Donglin Peng
2025-12-18 11:30 ` [PATCH bpf-next v10 02/13] selftests/bpf: Add test cases for btf__permute functionality Donglin Peng
2025-12-18 23:03 ` Andrii Nakryiko
2025-12-19 3:30 ` Donglin Peng
2025-12-18 11:30 ` [PATCH bpf-next v10 03/13] tools/resolve_btfids: Support BTF sorting feature Donglin Peng
2025-12-18 23:09 ` Andrii Nakryiko
2025-12-19 3:42 ` Donglin Peng
2025-12-19 17:12 ` Andrii Nakryiko
2025-12-20 8:44 ` Donglin Peng
2025-12-18 11:30 ` [PATCH bpf-next v10 04/13] libbpf: Optimize type lookup with binary search for sorted BTF Donglin Peng
2025-12-18 19:32 ` Eduard Zingerman
2025-12-18 23:29 ` Andrii Nakryiko
2025-12-19 0:13 ` Eduard Zingerman
2025-12-19 0:19 ` Andrii Nakryiko
2025-12-19 0:24 ` Eduard Zingerman
2025-12-19 1:01 ` Andrii Nakryiko
2025-12-19 2:53 ` Donglin Peng
2025-12-19 17:28 ` Andrii Nakryiko
2025-12-20 9:38 ` Donglin Peng
2025-12-22 1:58 ` Donglin Peng
2026-01-06 0:38 ` Andrii Nakryiko
2026-01-06 2:42 ` Donglin Peng
2026-01-06 0:36 ` Andrii Nakryiko
2026-01-06 2:50 ` Donglin Peng
2025-12-18 11:30 ` [PATCH bpf-next v10 05/13] libbpf: Verify BTF Sorting Donglin Peng
2025-12-18 19:38 ` Eduard Zingerman
2025-12-18 23:44 ` Andrii Nakryiko
2025-12-19 5:06 ` Donglin Peng
2025-12-19 17:33 ` Andrii Nakryiko
[not found] ` <CAErzpmsvirekLBRrJYVgmRC0YKWCbo7OyRQXgNYrk83aF-Wz2Q@mail.gmail.com>
2026-01-07 3:45 ` Donglin Peng
2026-01-07 21:50 ` Andrii Nakryiko
2026-01-08 1:50 ` Donglin Peng
2025-12-18 11:30 ` [PATCH bpf-next v10 06/13] btf: Optimize type lookup with binary search Donglin Peng
2025-12-18 21:38 ` Eduard Zingerman
2025-12-19 5:07 ` Donglin Peng
2025-12-18 11:30 ` [PATCH bpf-next v10 07/13] btf: Verify BTF Sorting Donglin Peng
2025-12-18 21:43 ` Eduard Zingerman
2025-12-19 5:10 ` Donglin Peng
2025-12-18 23:46 ` Andrii Nakryiko
2025-12-19 5:10 ` Donglin Peng
2025-12-18 11:30 ` [PATCH bpf-next v10 08/13] bpf: Skip anonymous types in type lookup for performance Donglin Peng
2025-12-18 22:21 ` Eduard Zingerman
2025-12-18 23:59 ` Andrii Nakryiko
2025-12-19 5:40 ` Donglin Peng
2025-12-18 22:24 ` Eduard Zingerman
2025-12-18 11:30 ` [PATCH bpf-next v10 09/13] bpf: Optimize the performance of find_bpffs_btf_enums Donglin Peng
2025-12-19 0:01 ` Andrii Nakryiko
2025-12-19 5:41 ` Donglin Peng
2025-12-20 14:27 ` Donglin Peng
2025-12-18 11:30 ` [PATCH bpf-next v10 10/13] libbpf: Optimize the performance of determine_ptr_size Donglin Peng
2025-12-18 22:27 ` Eduard Zingerman
2025-12-19 0:03 ` Andrii Nakryiko
2025-12-19 5:42 ` Donglin Peng
2025-12-18 11:30 ` [PATCH bpf-next v10 11/13] libbpf: Add btf_is_sorted and btf_sorted_start_id helpers to refactor the code Donglin Peng
2025-12-18 22:33 ` Eduard Zingerman
2025-12-19 5:49 ` Donglin Peng
2025-12-19 0:05 ` Andrii Nakryiko
2025-12-19 5:51 ` Donglin Peng
2025-12-19 17:35 ` Andrii Nakryiko
2025-12-20 14:27 ` Donglin Peng
2025-12-18 11:30 ` [PATCH bpf-next v10 12/13] btf: Add btf_is_sorted " Donglin Peng
2025-12-18 22:30 ` Eduard Zingerman
2025-12-19 0:05 ` Andrii Nakryiko
2025-12-20 14:25 ` Donglin Peng
2025-12-18 11:30 ` [PATCH bpf-next v10 13/13] btf: Refactor the code by calling str_is_empty Donglin Peng
2025-12-18 22:34 ` Eduard Zingerman
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox