* [PATCH bpf-next 0/4] libbpf: Resolve unambigous forward declarations
@ 2022-11-02 11:09 Eduard Zingerman
2022-11-02 11:09 ` [PATCH bpf-next 1/4] libbpf: hashmap interface update to uintptr_t -> uintptr_t Eduard Zingerman
` (3 more replies)
0 siblings, 4 replies; 8+ messages in thread
From: Eduard Zingerman @ 2022-11-02 11:09 UTC (permalink / raw)
To: bpf, ast; +Cc: andrii, daniel, kernel-team, yhs, alan.maguire, Eduard Zingerman
Resolve forward declarations that don't take part in type graphs
comparisons if declaration name is unambiguous.
Example:
CU #1:
struct foo; // standalone forward declaration
struct foo *some_global;
CU #2:
struct foo { int x; };
struct foo *another_global;
Currently the de-duplicated BTF for this example looks as follows:
[1] STRUCT 'foo' size=4 vlen=1 ...
[2] INT 'int' size=4 ...
[3] PTR '(anon)' type_id=1
[4] FWD 'foo' fwd_kind=struct
[5] PTR '(anon)' type_id=4
The goal of this patch-set is to simplify it as follows:
[1] STRUCT 'foo' size=4 vlen=1
'x' type_id=2 bits_offset=0
[2] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
[3] PTR '(anon)' type_id=1
The patch-set is consists of the following parts:
- A refactoring of the libbpf's hashmap interface to use `uintptr_t`
instead of `void*` for keys and values. The reasoning behind the
refactoring is that integer keys / values are used in libbpf more
often then pointer keys / values. Thus the refactoring reduces the
number of awkward looking casts like "(void *)(long)off".
`uintptr_t` is used to avoid necessity for temporary variables when
pointer keys / values are used on platforms with 32-bit pointers.
- A change to `lib/bpf/btf.c:btf__dedup` that adds a new pass named
"Resolve unambiguous forward declaration". This pass builds a
hashmap `name_off -> uniquely named struct or union` and uses it to
replace FWD types by structs or unions. This is necessary for corner
cases when FWD is not used as a part of some struct or union
definition de-duplicated by `btf_dedup_struct_types`.
For defconfig kernel with BTF enabled this removes 63 forward
declarations.
For allmodconfig kernel with BTF enabled this removes ~5K out of ~21K
forward declarations in ko objects. This unlocks some additional
de-duplication in ko objects, but impact is tiny: ~13K less BTF ids
out of ~2M.
Eduard Zingerman (4):
libbpf: hashmap interface update to uintptr_t -> uintptr_t
selftests/bpf: hashmap test cases updated for uintptr_t -> uintptr_t
interface
libbpf: Resolve unambigous forward declarations
selftests/bpf: Tests for btf_dedup_resolve_fwds
tools/bpf/bpftool/btf.c | 23 +--
tools/bpf/bpftool/common.c | 10 +-
tools/bpf/bpftool/gen.c | 19 +-
tools/bpf/bpftool/link.c | 8 +-
tools/bpf/bpftool/main.h | 4 +-
tools/bpf/bpftool/map.c | 8 +-
tools/bpf/bpftool/pids.c | 16 +-
tools/bpf/bpftool/prog.c | 8 +-
tools/lib/bpf/btf.c | 183 +++++++++++++++---
tools/lib/bpf/btf_dump.c | 16 +-
tools/lib/bpf/hashmap.c | 16 +-
tools/lib/bpf/hashmap.h | 35 ++--
tools/lib/bpf/libbpf.c | 18 +-
tools/lib/bpf/strset.c | 24 +--
tools/lib/bpf/usdt.c | 31 ++-
tools/testing/selftests/bpf/prog_tests/btf.c | 152 +++++++++++++++
.../bpf/prog_tests/btf_dedup_split.c | 45 +++--
.../selftests/bpf/prog_tests/hashmap.c | 68 +++----
.../bpf/prog_tests/kprobe_multi_test.c | 6 +-
19 files changed, 482 insertions(+), 208 deletions(-)
--
2.34.1
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH bpf-next 1/4] libbpf: hashmap interface update to uintptr_t -> uintptr_t
2022-11-02 11:09 [PATCH bpf-next 0/4] libbpf: Resolve unambigous forward declarations Eduard Zingerman
@ 2022-11-02 11:09 ` Eduard Zingerman
2022-11-02 11:09 ` [PATCH bpf-next 2/4] selftests/bpf: hashmap test cases updated for uintptr_t -> uintptr_t interface Eduard Zingerman
` (2 subsequent siblings)
3 siblings, 0 replies; 8+ messages in thread
From: Eduard Zingerman @ 2022-11-02 11:09 UTC (permalink / raw)
To: bpf, ast; +Cc: andrii, daniel, kernel-team, yhs, alan.maguire, Eduard Zingerman
An update for libbpf's hashmap interface from void* -> void* to
uintptr_t -> uintptr_t. Removes / simplifies some type casts when
hashmap keys or values are 32-bit integers. In libbpf hashmap is more
often used with integral keys / values rather than with pointer keys /
values.
This is a follow up for [1].
[1] https://lore.kernel.org/bpf/af1facf9-7bc8-8a3d-0db4-7b3f333589a2@meta.com/T/#m65b28f1d6d969fcd318b556db6a3ad499a42607d
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
---
tools/bpf/bpftool/btf.c | 23 ++++++++------------
tools/bpf/bpftool/common.c | 10 ++++-----
tools/bpf/bpftool/gen.c | 19 +++++++----------
tools/bpf/bpftool/link.c | 8 +++----
tools/bpf/bpftool/main.h | 4 ++--
tools/bpf/bpftool/map.c | 8 +++----
tools/bpf/bpftool/pids.c | 16 +++++++-------
tools/bpf/bpftool/prog.c | 8 +++----
tools/lib/bpf/btf.c | 43 +++++++++++++++++++-------------------
tools/lib/bpf/btf_dump.c | 16 +++++++-------
tools/lib/bpf/hashmap.c | 16 +++++++-------
tools/lib/bpf/hashmap.h | 35 ++++++++++++++++---------------
tools/lib/bpf/libbpf.c | 18 ++++++----------
tools/lib/bpf/strset.c | 24 ++++++++++-----------
tools/lib/bpf/usdt.c | 31 +++++++++++++--------------
15 files changed, 127 insertions(+), 152 deletions(-)
diff --git a/tools/bpf/bpftool/btf.c b/tools/bpf/bpftool/btf.c
index 68a70ac03c80..ccb3b8b0378b 100644
--- a/tools/bpf/bpftool/btf.c
+++ b/tools/bpf/bpftool/btf.c
@@ -815,8 +815,7 @@ build_btf_type_table(struct hashmap *tab, enum bpf_obj_type type,
if (!btf_id)
continue;
- err = hashmap__append(tab, u32_as_hash_field(btf_id),
- u32_as_hash_field(id));
+ err = hashmap__append(tab, btf_id, id);
if (err) {
p_err("failed to append entry to hashmap for BTF ID %u, object ID %u: %s",
btf_id, id, strerror(-err));
@@ -875,17 +874,15 @@ show_btf_plain(struct bpf_btf_info *info, int fd,
printf("size %uB", info->btf_size);
n = 0;
- hashmap__for_each_key_entry(btf_prog_table, entry,
- u32_as_hash_field(info->id)) {
+ hashmap__for_each_key_entry(btf_prog_table, entry, info->id) {
printf("%s%u", n++ == 0 ? " prog_ids " : ",",
- hash_field_as_u32(entry->value));
+ (__u32)entry->value);
}
n = 0;
- hashmap__for_each_key_entry(btf_map_table, entry,
- u32_as_hash_field(info->id)) {
+ hashmap__for_each_key_entry(btf_map_table, entry, info->id) {
printf("%s%u", n++ == 0 ? " map_ids " : ",",
- hash_field_as_u32(entry->value));
+ (__u32)entry->value);
}
emit_obj_refs_plain(refs_table, info->id, "\n\tpids ");
@@ -907,17 +904,15 @@ show_btf_json(struct bpf_btf_info *info, int fd,
jsonw_name(json_wtr, "prog_ids");
jsonw_start_array(json_wtr); /* prog_ids */
- hashmap__for_each_key_entry(btf_prog_table, entry,
- u32_as_hash_field(info->id)) {
- jsonw_uint(json_wtr, hash_field_as_u32(entry->value));
+ hashmap__for_each_key_entry(btf_prog_table, entry, info->id) {
+ jsonw_uint(json_wtr, entry->value);
}
jsonw_end_array(json_wtr); /* prog_ids */
jsonw_name(json_wtr, "map_ids");
jsonw_start_array(json_wtr); /* map_ids */
- hashmap__for_each_key_entry(btf_map_table, entry,
- u32_as_hash_field(info->id)) {
- jsonw_uint(json_wtr, hash_field_as_u32(entry->value));
+ hashmap__for_each_key_entry(btf_map_table, entry, info->id) {
+ jsonw_uint(json_wtr, entry->value);
}
jsonw_end_array(json_wtr); /* map_ids */
diff --git a/tools/bpf/bpftool/common.c b/tools/bpf/bpftool/common.c
index e4d33bc8bbbf..7bfb9ea1dc66 100644
--- a/tools/bpf/bpftool/common.c
+++ b/tools/bpf/bpftool/common.c
@@ -494,7 +494,7 @@ static int do_build_table_cb(const char *fpath, const struct stat *sb,
goto out_close;
}
- err = hashmap__append(build_fn_table, u32_as_hash_field(pinned_info.id), path);
+ err = hashmap__append(build_fn_table, pinned_info.id, (uintptr_t)path);
if (err) {
p_err("failed to append entry to hashmap for ID %u, path '%s': %s",
pinned_info.id, path, strerror(errno));
@@ -545,7 +545,7 @@ void delete_pinned_obj_table(struct hashmap *map)
return;
hashmap__for_each_entry(map, entry, bkt)
- free(entry->value);
+ free((char *)entry->value);
hashmap__free(map);
}
@@ -1041,12 +1041,12 @@ int map_parse_fd_and_info(int *argc, char ***argv, void *info, __u32 *info_len)
return fd;
}
-size_t hash_fn_for_key_as_id(const void *key, void *ctx)
+size_t hash_fn_for_key_as_id(uintptr_t key, void *ctx)
{
- return (size_t)key;
+ return key;
}
-bool equal_fn_for_key_as_id(const void *k1, const void *k2, void *ctx)
+bool equal_fn_for_key_as_id(uintptr_t k1, uintptr_t k2, void *ctx)
{
return k1 == k2;
}
diff --git a/tools/bpf/bpftool/gen.c b/tools/bpf/bpftool/gen.c
index cf8b4e525c88..756327fcab98 100644
--- a/tools/bpf/bpftool/gen.c
+++ b/tools/bpf/bpftool/gen.c
@@ -1660,21 +1660,16 @@ struct btfgen_info {
struct btf *marked_btf; /* btf structure used to mark used types */
};
-static size_t btfgen_hash_fn(const void *key, void *ctx)
+static size_t btfgen_hash_fn(uintptr_t key, void *ctx)
{
- return (size_t)key;
+ return key;
}
-static bool btfgen_equal_fn(const void *k1, const void *k2, void *ctx)
+static bool btfgen_equal_fn(uintptr_t k1, uintptr_t k2, void *ctx)
{
return k1 == k2;
}
-static void *u32_as_hash_key(__u32 x)
-{
- return (void *)(uintptr_t)x;
-}
-
static void btfgen_free_info(struct btfgen_info *info)
{
if (!info)
@@ -2086,18 +2081,18 @@ static int btfgen_record_obj(struct btfgen_info *info, const char *obj_path)
struct bpf_core_spec specs_scratch[3] = {};
struct bpf_core_relo_res targ_res = {};
struct bpf_core_cand_list *cands = NULL;
- const void *type_key = u32_as_hash_key(relo->type_id);
const char *sec_name = btf__name_by_offset(btf, sec->sec_name_off);
if (relo->kind != BPF_CORE_TYPE_ID_LOCAL &&
- !hashmap__find(cand_cache, type_key, (void **)&cands)) {
+ !hashmap__find(cand_cache, relo->type_id, (uintptr_t *)&cands)) {
cands = btfgen_find_cands(btf, info->src_btf, relo->type_id);
if (!cands) {
err = -errno;
goto out;
}
- err = hashmap__set(cand_cache, type_key, cands, NULL, NULL);
+ err = hashmap__set(cand_cache, relo->type_id, (uintptr_t)cands,
+ NULL, NULL);
if (err)
goto out;
}
@@ -2120,7 +2115,7 @@ static int btfgen_record_obj(struct btfgen_info *info, const char *obj_path)
if (!IS_ERR_OR_NULL(cand_cache)) {
hashmap__for_each_entry(cand_cache, entry, i) {
- bpf_core_free_cands(entry->value);
+ bpf_core_free_cands((struct bpf_core_cand_list *)entry->value);
}
hashmap__free(cand_cache);
}
diff --git a/tools/bpf/bpftool/link.c b/tools/bpf/bpftool/link.c
index 2863639706dd..54c7eccff8e1 100644
--- a/tools/bpf/bpftool/link.c
+++ b/tools/bpf/bpftool/link.c
@@ -204,9 +204,8 @@ static int show_link_close_json(int fd, struct bpf_link_info *info)
jsonw_name(json_wtr, "pinned");
jsonw_start_array(json_wtr);
- hashmap__for_each_key_entry(link_table, entry,
- u32_as_hash_field(info->id))
- jsonw_string(json_wtr, entry->value);
+ hashmap__for_each_key_entry(link_table, entry, info->id)
+ jsonw_string(json_wtr, (char *)entry->value);
jsonw_end_array(json_wtr);
}
@@ -309,8 +308,7 @@ static int show_link_close_plain(int fd, struct bpf_link_info *info)
if (!hashmap__empty(link_table)) {
struct hashmap_entry *entry;
- hashmap__for_each_key_entry(link_table, entry,
- u32_as_hash_field(info->id))
+ hashmap__for_each_key_entry(link_table, entry, info->id)
printf("\n\tpinned %s", (char *)entry->value);
}
emit_obj_refs_plain(refs_table, info->id, "\n\tpids ");
diff --git a/tools/bpf/bpftool/main.h b/tools/bpf/bpftool/main.h
index 467d8472df0c..e0be216da944 100644
--- a/tools/bpf/bpftool/main.h
+++ b/tools/bpf/bpftool/main.h
@@ -240,8 +240,8 @@ int do_filter_dump(struct tcmsg *ifinfo, struct nlattr **tb, const char *kind,
int print_all_levels(__maybe_unused enum libbpf_print_level level,
const char *format, va_list args);
-size_t hash_fn_for_key_as_id(const void *key, void *ctx);
-bool equal_fn_for_key_as_id(const void *k1, const void *k2, void *ctx);
+size_t hash_fn_for_key_as_id(uintptr_t key, void *ctx);
+bool equal_fn_for_key_as_id(uintptr_t k1, uintptr_t k2, void *ctx);
/* bpf_attach_type_input_str - convert the provided attach type value into a
* textual representation that we accept for input purposes.
diff --git a/tools/bpf/bpftool/map.c b/tools/bpf/bpftool/map.c
index f941ac5c7b73..dbee1bbd6a0c 100644
--- a/tools/bpf/bpftool/map.c
+++ b/tools/bpf/bpftool/map.c
@@ -518,9 +518,8 @@ static int show_map_close_json(int fd, struct bpf_map_info *info)
jsonw_name(json_wtr, "pinned");
jsonw_start_array(json_wtr);
- hashmap__for_each_key_entry(map_table, entry,
- u32_as_hash_field(info->id))
- jsonw_string(json_wtr, entry->value);
+ hashmap__for_each_key_entry(map_table, entry, info->id)
+ jsonw_string(json_wtr, (char *)entry->value);
jsonw_end_array(json_wtr);
}
@@ -595,8 +594,7 @@ static int show_map_close_plain(int fd, struct bpf_map_info *info)
if (!hashmap__empty(map_table)) {
struct hashmap_entry *entry;
- hashmap__for_each_key_entry(map_table, entry,
- u32_as_hash_field(info->id))
+ hashmap__for_each_key_entry(map_table, entry, info->id)
printf("\n\tpinned %s", (char *)entry->value);
}
diff --git a/tools/bpf/bpftool/pids.c b/tools/bpf/bpftool/pids.c
index bb6c969a114a..e01ff78610cf 100644
--- a/tools/bpf/bpftool/pids.c
+++ b/tools/bpf/bpftool/pids.c
@@ -36,8 +36,8 @@ static void add_ref(struct hashmap *map, struct pid_iter_entry *e)
int err, i;
void *tmp;
- hashmap__for_each_key_entry(map, entry, u32_as_hash_field(e->id)) {
- refs = entry->value;
+ hashmap__for_each_key_entry(map, entry, e->id) {
+ refs = (struct obj_refs *)entry->value;
for (i = 0; i < refs->ref_cnt; i++) {
if (refs->refs[i].pid == e->pid)
@@ -81,7 +81,7 @@ static void add_ref(struct hashmap *map, struct pid_iter_entry *e)
refs->has_bpf_cookie = e->has_bpf_cookie;
refs->bpf_cookie = e->bpf_cookie;
- err = hashmap__append(map, u32_as_hash_field(e->id), refs);
+ err = hashmap__append(map, e->id, (uintptr_t)refs);
if (err)
p_err("failed to append entry to hashmap for ID %u: %s",
e->id, strerror(errno));
@@ -183,7 +183,7 @@ void delete_obj_refs_table(struct hashmap *map)
return;
hashmap__for_each_entry(map, entry, bkt) {
- struct obj_refs *refs = entry->value;
+ struct obj_refs *refs = (struct obj_refs *)entry->value;
free(refs->refs);
free(refs);
@@ -200,8 +200,8 @@ void emit_obj_refs_json(struct hashmap *map, __u32 id,
if (hashmap__empty(map))
return;
- hashmap__for_each_key_entry(map, entry, u32_as_hash_field(id)) {
- struct obj_refs *refs = entry->value;
+ hashmap__for_each_key_entry(map, entry, id) {
+ struct obj_refs *refs = (struct obj_refs *)entry->value;
int i;
if (refs->ref_cnt == 0)
@@ -232,8 +232,8 @@ void emit_obj_refs_plain(struct hashmap *map, __u32 id, const char *prefix)
if (hashmap__empty(map))
return;
- hashmap__for_each_key_entry(map, entry, u32_as_hash_field(id)) {
- struct obj_refs *refs = entry->value;
+ hashmap__for_each_key_entry(map, entry, id) {
+ struct obj_refs *refs = (struct obj_refs *)entry->value;
int i;
if (refs->ref_cnt == 0)
diff --git a/tools/bpf/bpftool/prog.c b/tools/bpf/bpftool/prog.c
index a858b907da16..18d8da67c7e8 100644
--- a/tools/bpf/bpftool/prog.c
+++ b/tools/bpf/bpftool/prog.c
@@ -486,9 +486,8 @@ static void print_prog_json(struct bpf_prog_info *info, int fd)
jsonw_name(json_wtr, "pinned");
jsonw_start_array(json_wtr);
- hashmap__for_each_key_entry(prog_table, entry,
- u32_as_hash_field(info->id))
- jsonw_string(json_wtr, entry->value);
+ hashmap__for_each_key_entry(prog_table, entry, info->id)
+ jsonw_string(json_wtr, (char *)entry->value);
jsonw_end_array(json_wtr);
}
@@ -561,8 +560,7 @@ static void print_prog_plain(struct bpf_prog_info *info, int fd)
if (!hashmap__empty(prog_table)) {
struct hashmap_entry *entry;
- hashmap__for_each_key_entry(prog_table, entry,
- u32_as_hash_field(info->id))
+ hashmap__for_each_key_entry(prog_table, entry, info->id)
printf("\n\tpinned %s", (char *)entry->value);
}
diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
index 675a0df5c840..04db202aac3d 100644
--- a/tools/lib/bpf/btf.c
+++ b/tools/lib/bpf/btf.c
@@ -1559,15 +1559,15 @@ struct btf_pipe {
static int btf_rewrite_str(__u32 *str_off, void *ctx)
{
struct btf_pipe *p = ctx;
- void *mapped_off;
+ uintptr_t mapped_off;
int off, err;
if (!*str_off) /* nothing to do for empty strings */
return 0;
if (p->str_off_map &&
- hashmap__find(p->str_off_map, (void *)(long)*str_off, &mapped_off)) {
- *str_off = (__u32)(long)mapped_off;
+ hashmap__find(p->str_off_map, *str_off, &mapped_off)) {
+ *str_off = mapped_off;
return 0;
}
@@ -1579,7 +1579,7 @@ static int btf_rewrite_str(__u32 *str_off, void *ctx)
* performing expensive string comparisons.
*/
if (p->str_off_map) {
- err = hashmap__append(p->str_off_map, (void *)(long)*str_off, (void *)(long)off);
+ err = hashmap__append(p->str_off_map, *str_off, off);
if (err)
return err;
}
@@ -1630,8 +1630,8 @@ static int btf_rewrite_type_ids(__u32 *type_id, void *ctx)
return 0;
}
-static size_t btf_dedup_identity_hash_fn(const void *key, void *ctx);
-static bool btf_dedup_equal_fn(const void *k1, const void *k2, void *ctx);
+static size_t btf_dedup_identity_hash_fn(uintptr_t key, void *ctx);
+static bool btf_dedup_equal_fn(uintptr_t k1, uintptr_t k2, void *ctx);
int btf__add_btf(struct btf *btf, const struct btf *src_btf)
{
@@ -3126,12 +3126,11 @@ static long hash_combine(long h, long value)
}
#define for_each_dedup_cand(d, node, hash) \
- hashmap__for_each_key_entry(d->dedup_table, node, (void *)hash)
+ hashmap__for_each_key_entry(d->dedup_table, node, hash)
static int btf_dedup_table_add(struct btf_dedup *d, long hash, __u32 type_id)
{
- return hashmap__append(d->dedup_table,
- (void *)hash, (void *)(long)type_id);
+ return hashmap__append(d->dedup_table, hash, type_id);
}
static int btf_dedup_hypot_map_add(struct btf_dedup *d,
@@ -3178,17 +3177,17 @@ static void btf_dedup_free(struct btf_dedup *d)
free(d);
}
-static size_t btf_dedup_identity_hash_fn(const void *key, void *ctx)
+static size_t btf_dedup_identity_hash_fn(uintptr_t key, void *ctx)
{
- return (size_t)key;
+ return key;
}
-static size_t btf_dedup_collision_hash_fn(const void *key, void *ctx)
+static size_t btf_dedup_collision_hash_fn(uintptr_t key, void *ctx)
{
return 0;
}
-static bool btf_dedup_equal_fn(const void *k1, const void *k2, void *ctx)
+static bool btf_dedup_equal_fn(uintptr_t k1, uintptr_t k2, void *ctx)
{
return k1 == k2;
}
@@ -3753,7 +3752,7 @@ static int btf_dedup_prim_type(struct btf_dedup *d, __u32 type_id)
case BTF_KIND_INT:
h = btf_hash_int_decl_tag(t);
for_each_dedup_cand(d, hash_entry, h) {
- cand_id = (__u32)(long)hash_entry->value;
+ cand_id = hash_entry->value;
cand = btf_type_by_id(d->btf, cand_id);
if (btf_equal_int_tag(t, cand)) {
new_id = cand_id;
@@ -3765,7 +3764,7 @@ static int btf_dedup_prim_type(struct btf_dedup *d, __u32 type_id)
case BTF_KIND_ENUM:
h = btf_hash_enum(t);
for_each_dedup_cand(d, hash_entry, h) {
- cand_id = (__u32)(long)hash_entry->value;
+ cand_id = hash_entry->value;
cand = btf_type_by_id(d->btf, cand_id);
if (btf_equal_enum(t, cand)) {
new_id = cand_id;
@@ -3786,7 +3785,7 @@ static int btf_dedup_prim_type(struct btf_dedup *d, __u32 type_id)
case BTF_KIND_ENUM64:
h = btf_hash_enum(t);
for_each_dedup_cand(d, hash_entry, h) {
- cand_id = (__u32)(long)hash_entry->value;
+ cand_id = hash_entry->value;
cand = btf_type_by_id(d->btf, cand_id);
if (btf_equal_enum64(t, cand)) {
new_id = cand_id;
@@ -3808,7 +3807,7 @@ static int btf_dedup_prim_type(struct btf_dedup *d, __u32 type_id)
case BTF_KIND_FLOAT:
h = btf_hash_common(t);
for_each_dedup_cand(d, hash_entry, h) {
- cand_id = (__u32)(long)hash_entry->value;
+ cand_id = hash_entry->value;
cand = btf_type_by_id(d->btf, cand_id);
if (btf_equal_common(t, cand)) {
new_id = cand_id;
@@ -4313,7 +4312,7 @@ static int btf_dedup_struct_type(struct btf_dedup *d, __u32 type_id)
h = btf_hash_struct(t);
for_each_dedup_cand(d, hash_entry, h) {
- __u32 cand_id = (__u32)(long)hash_entry->value;
+ __u32 cand_id = hash_entry->value;
int eq;
/*
@@ -4418,7 +4417,7 @@ static int btf_dedup_ref_type(struct btf_dedup *d, __u32 type_id)
h = btf_hash_common(t);
for_each_dedup_cand(d, hash_entry, h) {
- cand_id = (__u32)(long)hash_entry->value;
+ cand_id = hash_entry->value;
cand = btf_type_by_id(d->btf, cand_id);
if (btf_equal_common(t, cand)) {
new_id = cand_id;
@@ -4435,7 +4434,7 @@ static int btf_dedup_ref_type(struct btf_dedup *d, __u32 type_id)
h = btf_hash_int_decl_tag(t);
for_each_dedup_cand(d, hash_entry, h) {
- cand_id = (__u32)(long)hash_entry->value;
+ cand_id = hash_entry->value;
cand = btf_type_by_id(d->btf, cand_id);
if (btf_equal_int_tag(t, cand)) {
new_id = cand_id;
@@ -4459,7 +4458,7 @@ static int btf_dedup_ref_type(struct btf_dedup *d, __u32 type_id)
h = btf_hash_array(t);
for_each_dedup_cand(d, hash_entry, h) {
- cand_id = (__u32)(long)hash_entry->value;
+ cand_id = hash_entry->value;
cand = btf_type_by_id(d->btf, cand_id);
if (btf_equal_array(t, cand)) {
new_id = cand_id;
@@ -4491,7 +4490,7 @@ static int btf_dedup_ref_type(struct btf_dedup *d, __u32 type_id)
h = btf_hash_fnproto(t);
for_each_dedup_cand(d, hash_entry, h) {
- cand_id = (__u32)(long)hash_entry->value;
+ cand_id = hash_entry->value;
cand = btf_type_by_id(d->btf, cand_id);
if (btf_equal_fnproto(t, cand)) {
new_id = cand_id;
diff --git a/tools/lib/bpf/btf_dump.c b/tools/lib/bpf/btf_dump.c
index bf0cc0e986dd..ad0585410e51 100644
--- a/tools/lib/bpf/btf_dump.c
+++ b/tools/lib/bpf/btf_dump.c
@@ -117,14 +117,14 @@ struct btf_dump {
struct btf_dump_data *typed_dump;
};
-static size_t str_hash_fn(const void *key, void *ctx)
+static size_t str_hash_fn(uintptr_t key, void *ctx)
{
- return str_hash(key);
+ return str_hash((void *)key);
}
-static bool str_equal_fn(const void *a, const void *b, void *ctx)
+static bool str_equal_fn(uintptr_t a, uintptr_t b, void *ctx)
{
- return strcmp(a, b) == 0;
+ return strcmp((void *)a, (void *)b) == 0;
}
static const char *btf_name_of(const struct btf_dump *d, __u32 name_off)
@@ -1536,18 +1536,18 @@ static size_t btf_dump_name_dups(struct btf_dump *d, struct hashmap *name_map,
const char *orig_name)
{
char *old_name, *new_name;
- size_t dup_cnt = 0;
+ uintptr_t dup_cnt = 0;
int err;
new_name = strdup(orig_name);
if (!new_name)
return 1;
- hashmap__find(name_map, orig_name, (void **)&dup_cnt);
+ hashmap__find(name_map, (uintptr_t)orig_name, &dup_cnt);
dup_cnt++;
- err = hashmap__set(name_map, new_name, (void *)dup_cnt,
- (const void **)&old_name, NULL);
+ err = hashmap__set(name_map, (uintptr_t)new_name, dup_cnt,
+ (uintptr_t *)&old_name, NULL);
if (err)
free(new_name);
diff --git a/tools/lib/bpf/hashmap.c b/tools/lib/bpf/hashmap.c
index aeb09c288716..0d880e6367d5 100644
--- a/tools/lib/bpf/hashmap.c
+++ b/tools/lib/bpf/hashmap.c
@@ -128,7 +128,7 @@ static int hashmap_grow(struct hashmap *map)
}
static bool hashmap_find_entry(const struct hashmap *map,
- const void *key, size_t hash,
+ const uintptr_t key, size_t hash,
struct hashmap_entry ***pprev,
struct hashmap_entry **entry)
{
@@ -151,18 +151,18 @@ static bool hashmap_find_entry(const struct hashmap *map,
return false;
}
-int hashmap__insert(struct hashmap *map, const void *key, void *value,
+int hashmap__insert(struct hashmap *map, uintptr_t key, uintptr_t value,
enum hashmap_insert_strategy strategy,
- const void **old_key, void **old_value)
+ uintptr_t *old_key, uintptr_t *old_value)
{
struct hashmap_entry *entry;
size_t h;
int err;
if (old_key)
- *old_key = NULL;
+ *old_key = 0;
if (old_value)
- *old_value = NULL;
+ *old_value = 0;
h = hash_bits(map->hash_fn(key, map->ctx), map->cap_bits);
if (strategy != HASHMAP_APPEND &&
@@ -203,7 +203,7 @@ int hashmap__insert(struct hashmap *map, const void *key, void *value,
return 0;
}
-bool hashmap__find(const struct hashmap *map, const void *key, void **value)
+bool hashmap__find(const struct hashmap *map, uintptr_t key, uintptr_t *value)
{
struct hashmap_entry *entry;
size_t h;
@@ -217,8 +217,8 @@ bool hashmap__find(const struct hashmap *map, const void *key, void **value)
return true;
}
-bool hashmap__delete(struct hashmap *map, const void *key,
- const void **old_key, void **old_value)
+bool hashmap__delete(struct hashmap *map, uintptr_t key,
+ uintptr_t *old_key, uintptr_t *old_value)
{
struct hashmap_entry **pprev, *entry;
size_t h;
diff --git a/tools/lib/bpf/hashmap.h b/tools/lib/bpf/hashmap.h
index 10a4c4cd13cf..8e72a238caa2 100644
--- a/tools/lib/bpf/hashmap.h
+++ b/tools/lib/bpf/hashmap.h
@@ -40,12 +40,15 @@ static inline size_t str_hash(const char *s)
return h;
}
-typedef size_t (*hashmap_hash_fn)(const void *key, void *ctx);
-typedef bool (*hashmap_equal_fn)(const void *key1, const void *key2, void *ctx);
+/* keys and values are represented by uintptr_t to allow usage of both
+ * pointers and 32-bit unsigned integers as keys or values.
+ */
+typedef size_t (*hashmap_hash_fn)(uintptr_t key, void *ctx);
+typedef bool (*hashmap_equal_fn)(uintptr_t key1, uintptr_t key2, void *ctx);
struct hashmap_entry {
- const void *key;
- void *value;
+ uintptr_t key;
+ uintptr_t value;
struct hashmap_entry *next;
};
@@ -109,42 +112,40 @@ enum hashmap_insert_strategy {
* through old_key and old_value to allow calling code do proper memory
* management.
*/
-int hashmap__insert(struct hashmap *map, const void *key, void *value,
+int hashmap__insert(struct hashmap *map, uintptr_t key, uintptr_t value,
enum hashmap_insert_strategy strategy,
- const void **old_key, void **old_value);
+ uintptr_t *old_key, uintptr_t *old_value);
-static inline int hashmap__add(struct hashmap *map,
- const void *key, void *value)
+static inline int hashmap__add(struct hashmap *map, uintptr_t key, uintptr_t value)
{
return hashmap__insert(map, key, value, HASHMAP_ADD, NULL, NULL);
}
static inline int hashmap__set(struct hashmap *map,
- const void *key, void *value,
- const void **old_key, void **old_value)
+ uintptr_t key, uintptr_t value,
+ uintptr_t *old_key, uintptr_t *old_value)
{
return hashmap__insert(map, key, value, HASHMAP_SET,
old_key, old_value);
}
static inline int hashmap__update(struct hashmap *map,
- const void *key, void *value,
- const void **old_key, void **old_value)
+ uintptr_t key, uintptr_t value,
+ uintptr_t *old_key, uintptr_t *old_value)
{
return hashmap__insert(map, key, value, HASHMAP_UPDATE,
old_key, old_value);
}
-static inline int hashmap__append(struct hashmap *map,
- const void *key, void *value)
+static inline int hashmap__append(struct hashmap *map, uintptr_t key, uintptr_t value)
{
return hashmap__insert(map, key, value, HASHMAP_APPEND, NULL, NULL);
}
-bool hashmap__delete(struct hashmap *map, const void *key,
- const void **old_key, void **old_value);
+bool hashmap__delete(struct hashmap *map, uintptr_t key,
+ uintptr_t *old_key, uintptr_t *old_value);
-bool hashmap__find(const struct hashmap *map, const void *key, void **value);
+bool hashmap__find(const struct hashmap *map, uintptr_t key, uintptr_t *value);
/*
* hashmap__for_each_entry - iterate over all entries in hashmap
diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index 5d7819edf074..7c9c1770db13 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -5601,21 +5601,16 @@ int bpf_core_types_match(const struct btf *local_btf, __u32 local_id,
return __bpf_core_types_match(local_btf, local_id, targ_btf, targ_id, false, 32);
}
-static size_t bpf_core_hash_fn(const void *key, void *ctx)
+static size_t bpf_core_hash_fn(const uintptr_t key, void *ctx)
{
- return (size_t)key;
+ return key;
}
-static bool bpf_core_equal_fn(const void *k1, const void *k2, void *ctx)
+static bool bpf_core_equal_fn(const uintptr_t k1, const uintptr_t k2, void *ctx)
{
return k1 == k2;
}
-static void *u32_as_hash_key(__u32 x)
-{
- return (void *)(uintptr_t)x;
-}
-
static int record_relo_core(struct bpf_program *prog,
const struct bpf_core_relo *core_relo, int insn_idx)
{
@@ -5658,7 +5653,6 @@ static int bpf_core_resolve_relo(struct bpf_program *prog,
struct bpf_core_relo_res *targ_res)
{
struct bpf_core_spec specs_scratch[3] = {};
- const void *type_key = u32_as_hash_key(relo->type_id);
struct bpf_core_cand_list *cands = NULL;
const char *prog_name = prog->name;
const struct btf_type *local_type;
@@ -5675,7 +5669,7 @@ static int bpf_core_resolve_relo(struct bpf_program *prog,
return -EINVAL;
if (relo->kind != BPF_CORE_TYPE_ID_LOCAL &&
- !hashmap__find(cand_cache, type_key, (void **)&cands)) {
+ !hashmap__find(cand_cache, local_id, (uintptr_t *)&cands)) {
cands = bpf_core_find_cands(prog->obj, local_btf, local_id);
if (IS_ERR(cands)) {
pr_warn("prog '%s': relo #%d: target candidate search failed for [%d] %s %s: %ld\n",
@@ -5683,7 +5677,7 @@ static int bpf_core_resolve_relo(struct bpf_program *prog,
local_name, PTR_ERR(cands));
return PTR_ERR(cands);
}
- err = hashmap__set(cand_cache, type_key, cands, NULL, NULL);
+ err = hashmap__set(cand_cache, local_id, (uintptr_t)cands, NULL, NULL);
if (err) {
bpf_core_free_cands(cands);
return err;
@@ -5806,7 +5800,7 @@ bpf_object__relocate_core(struct bpf_object *obj, const char *targ_btf_path)
if (!IS_ERR_OR_NULL(cand_cache)) {
hashmap__for_each_entry(cand_cache, entry, i) {
- bpf_core_free_cands(entry->value);
+ bpf_core_free_cands((struct bpf_core_cand_list *)entry->value);
}
hashmap__free(cand_cache);
}
diff --git a/tools/lib/bpf/strset.c b/tools/lib/bpf/strset.c
index ea655318153f..e467d0c0ab1a 100644
--- a/tools/lib/bpf/strset.c
+++ b/tools/lib/bpf/strset.c
@@ -19,19 +19,19 @@ struct strset {
struct hashmap *strs_hash;
};
-static size_t strset_hash_fn(const void *key, void *ctx)
+static size_t strset_hash_fn(uintptr_t key, void *ctx)
{
const struct strset *s = ctx;
- const char *str = s->strs_data + (long)key;
+ const char *str = s->strs_data + key;
return str_hash(str);
}
-static bool strset_equal_fn(const void *key1, const void *key2, void *ctx)
+static bool strset_equal_fn(uintptr_t key1, uintptr_t key2, void *ctx)
{
const struct strset *s = ctx;
- const char *str1 = s->strs_data + (long)key1;
- const char *str2 = s->strs_data + (long)key2;
+ const char *str1 = s->strs_data + key1;
+ const char *str2 = s->strs_data + key2;
return strcmp(str1, str2) == 0;
}
@@ -53,7 +53,7 @@ struct strset *strset__new(size_t max_data_sz, const char *init_data, size_t ini
set->strs_hash = hash;
if (init_data) {
- long off;
+ uintptr_t off;
set->strs_data = malloc(init_data_sz);
if (!set->strs_data)
@@ -67,7 +67,7 @@ struct strset *strset__new(size_t max_data_sz, const char *init_data, size_t ini
/* hashmap__add() returns EEXIST if string with the same
* content already is in the hash map
*/
- err = hashmap__add(hash, (void *)off, (void *)off);
+ err = hashmap__add(hash, off, off);
if (err == -EEXIST)
continue; /* duplicate */
if (err)
@@ -115,7 +115,7 @@ static void *strset_add_str_mem(struct strset *set, size_t add_sz)
*/
int strset__find_str(struct strset *set, const char *s)
{
- long old_off, new_off, len;
+ uintptr_t old_off, new_off, len;
void *p;
/* see strset__add_str() for why we do this */
@@ -127,7 +127,7 @@ int strset__find_str(struct strset *set, const char *s)
new_off = set->strs_data_len;
memcpy(p, s, len);
- if (hashmap__find(set->strs_hash, (void *)new_off, (void **)&old_off))
+ if (hashmap__find(set->strs_hash, new_off, &old_off))
return old_off;
return -ENOENT;
@@ -141,7 +141,7 @@ int strset__find_str(struct strset *set, const char *s)
*/
int strset__add_str(struct strset *set, const char *s)
{
- long old_off, new_off, len;
+ uintptr_t old_off, new_off, len;
void *p;
int err;
@@ -165,8 +165,8 @@ int strset__add_str(struct strset *set, const char *s)
* contents doesn't exist already (HASHMAP_ADD strategy). If such
* string exists, we'll get its offset in old_off (that's old_key).
*/
- err = hashmap__insert(set->strs_hash, (void *)new_off, (void *)new_off,
- HASHMAP_ADD, (const void **)&old_off, NULL);
+ err = hashmap__insert(set->strs_hash, new_off, new_off,
+ HASHMAP_ADD, &old_off, NULL);
if (err == -EEXIST)
return old_off; /* duplicated string, return existing offset */
if (err)
diff --git a/tools/lib/bpf/usdt.c b/tools/lib/bpf/usdt.c
index 28fa1b2283de..aa7ca3652db8 100644
--- a/tools/lib/bpf/usdt.c
+++ b/tools/lib/bpf/usdt.c
@@ -873,31 +873,27 @@ static void bpf_link_usdt_dealloc(struct bpf_link *link)
free(usdt_link);
}
-static size_t specs_hash_fn(const void *key, void *ctx)
+static size_t specs_hash_fn(uintptr_t key, void *ctx)
{
- const char *s = key;
-
- return str_hash(s);
+ return str_hash((char *)key);
}
-static bool specs_equal_fn(const void *key1, const void *key2, void *ctx)
+static bool specs_equal_fn(uintptr_t key1, uintptr_t key2, void *ctx)
{
- const char *s1 = key1;
- const char *s2 = key2;
-
- return strcmp(s1, s2) == 0;
+ return strcmp((char *)key1, (char *)key2) == 0;
}
static int allocate_spec_id(struct usdt_manager *man, struct hashmap *specs_hash,
struct bpf_link_usdt *link, struct usdt_target *target,
int *spec_id, bool *is_new)
{
- void *tmp;
+ uintptr_t tmp;
+ void *new_ids;
int err;
/* check if we already allocated spec ID for this spec string */
- if (hashmap__find(specs_hash, target->spec_str, &tmp)) {
- *spec_id = (long)tmp;
+ if (hashmap__find(specs_hash, (uintptr_t)target->spec_str, &tmp)) {
+ *spec_id = tmp;
*is_new = false;
return 0;
}
@@ -905,17 +901,18 @@ static int allocate_spec_id(struct usdt_manager *man, struct hashmap *specs_hash
/* otherwise it's a new ID that needs to be set up in specs map and
* returned back to usdt_manager when USDT link is detached
*/
- tmp = libbpf_reallocarray(link->spec_ids, link->spec_cnt + 1, sizeof(*link->spec_ids));
- if (!tmp)
+ new_ids = libbpf_reallocarray(link->spec_ids, link->spec_cnt + 1,
+ sizeof(*link->spec_ids));
+ if (!new_ids)
return -ENOMEM;
- link->spec_ids = tmp;
+ link->spec_ids = new_ids;
/* get next free spec ID, giving preference to free list, if not empty */
if (man->free_spec_cnt) {
*spec_id = man->free_spec_ids[man->free_spec_cnt - 1];
/* cache spec ID for current spec string for future lookups */
- err = hashmap__add(specs_hash, target->spec_str, (void *)(long)*spec_id);
+ err = hashmap__add(specs_hash, (uintptr_t)target->spec_str, *spec_id);
if (err)
return err;
@@ -928,7 +925,7 @@ static int allocate_spec_id(struct usdt_manager *man, struct hashmap *specs_hash
*spec_id = man->next_free_spec_id;
/* cache spec ID for current spec string for future lookups */
- err = hashmap__add(specs_hash, target->spec_str, (void *)(long)*spec_id);
+ err = hashmap__add(specs_hash, (uintptr_t)target->spec_str, *spec_id);
if (err)
return err;
--
2.34.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH bpf-next 2/4] selftests/bpf: hashmap test cases updated for uintptr_t -> uintptr_t interface
2022-11-02 11:09 [PATCH bpf-next 0/4] libbpf: Resolve unambigous forward declarations Eduard Zingerman
2022-11-02 11:09 ` [PATCH bpf-next 1/4] libbpf: hashmap interface update to uintptr_t -> uintptr_t Eduard Zingerman
@ 2022-11-02 11:09 ` Eduard Zingerman
2022-11-02 11:09 ` [PATCH bpf-next 3/4] libbpf: Resolve unambigous forward declarations Eduard Zingerman
2022-11-02 11:09 ` [PATCH bpf-next 4/4] selftests/bpf: Tests for btf_dedup_resolve_fwds Eduard Zingerman
3 siblings, 0 replies; 8+ messages in thread
From: Eduard Zingerman @ 2022-11-02 11:09 UTC (permalink / raw)
To: bpf, ast; +Cc: andrii, daniel, kernel-team, yhs, alan.maguire, Eduard Zingerman
Hashmap test cases require update after libbpf's hashmap interface
update from void* -> void* to uintptr_t -> uintptr_t. No logical
changes, types / casts updated to satisfy the type checker.
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
---
.../selftests/bpf/prog_tests/hashmap.c | 68 +++++++++----------
.../bpf/prog_tests/kprobe_multi_test.c | 6 +-
2 files changed, 37 insertions(+), 37 deletions(-)
diff --git a/tools/testing/selftests/bpf/prog_tests/hashmap.c b/tools/testing/selftests/bpf/prog_tests/hashmap.c
index 4747ab18f97f..dd705959f91d 100644
--- a/tools/testing/selftests/bpf/prog_tests/hashmap.c
+++ b/tools/testing/selftests/bpf/prog_tests/hashmap.c
@@ -10,14 +10,14 @@
static int duration = 0;
-static size_t hash_fn(const void *k, void *ctx)
+static size_t hash_fn(uintptr_t k, void *ctx)
{
- return (long)k;
+ return k;
}
-static bool equal_fn(const void *a, const void *b, void *ctx)
+static bool equal_fn(uintptr_t a, uintptr_t b, void *ctx)
{
- return (long)a == (long)b;
+ return a == b;
}
static inline size_t next_pow_2(size_t n)
@@ -52,8 +52,8 @@ static void test_hashmap_generic(void)
return;
for (i = 0; i < ELEM_CNT; i++) {
- const void *oldk, *k = (const void *)(long)i;
- void *oldv, *v = (void *)(long)(1024 + i);
+ uintptr_t oldk, k = i;
+ uintptr_t oldv, v = 1024 + i;
err = hashmap__update(map, k, v, &oldk, &oldv);
if (CHECK(err != -ENOENT, "hashmap__update",
@@ -64,8 +64,8 @@ static void test_hashmap_generic(void)
err = hashmap__add(map, k, v);
} else {
err = hashmap__set(map, k, v, &oldk, &oldv);
- if (CHECK(oldk != NULL || oldv != NULL, "check_kv",
- "unexpected k/v: %p=%p\n", oldk, oldv))
+ if (CHECK(oldk != 0 || oldv != 0, "check_kv",
+ "unexpected k/v: %ld=%ld\n", (long)oldk, (long)oldv))
goto cleanup;
}
@@ -91,8 +91,8 @@ static void test_hashmap_generic(void)
found_msk = 0;
hashmap__for_each_entry(map, entry, bkt) {
- long k = (long)entry->key;
- long v = (long)entry->value;
+ long k = entry->key;
+ long v = entry->value;
found_msk |= 1ULL << k;
if (CHECK(v - k != 1024, "check_kv",
@@ -104,8 +104,8 @@ static void test_hashmap_generic(void)
goto cleanup;
for (i = 0; i < ELEM_CNT; i++) {
- const void *oldk, *k = (const void *)(long)i;
- void *oldv, *v = (void *)(long)(256 + i);
+ uintptr_t oldk, k = i;
+ uintptr_t oldv, v = 256 + i;
err = hashmap__add(map, k, v);
if (CHECK(err != -EEXIST, "hashmap__add",
@@ -139,8 +139,8 @@ static void test_hashmap_generic(void)
found_msk = 0;
hashmap__for_each_entry_safe(map, entry, tmp, bkt) {
- long k = (long)entry->key;
- long v = (long)entry->value;
+ long k = entry->key;
+ long v = entry->value;
found_msk |= 1ULL << k;
if (CHECK(v - k != 256, "elem_check",
@@ -152,7 +152,7 @@ static void test_hashmap_generic(void)
goto cleanup;
found_cnt = 0;
- hashmap__for_each_key_entry(map, entry, (void *)0) {
+ hashmap__for_each_key_entry(map, entry, 0) {
found_cnt++;
}
if (CHECK(!found_cnt, "found_cnt",
@@ -161,9 +161,9 @@ static void test_hashmap_generic(void)
found_msk = 0;
found_cnt = 0;
- hashmap__for_each_key_entry_safe(map, entry, tmp, (void *)0) {
- const void *oldk, *k;
- void *oldv, *v;
+ hashmap__for_each_key_entry_safe(map, entry, tmp, 0) {
+ uintptr_t oldk, k;
+ uintptr_t oldv, v;
k = entry->key;
v = entry->value;
@@ -198,8 +198,8 @@ static void test_hashmap_generic(void)
goto cleanup;
hashmap__for_each_entry_safe(map, entry, tmp, bkt) {
- const void *oldk, *k;
- void *oldv, *v;
+ uintptr_t oldk, k;
+ uintptr_t oldv, v;
k = entry->key;
v = entry->value;
@@ -235,7 +235,7 @@ static void test_hashmap_generic(void)
hashmap__for_each_entry(map, entry, bkt) {
CHECK(false, "elem_exists",
"unexpected map entries left: %ld = %ld\n",
- (long)entry->key, (long)entry->value);
+ entry->key, entry->value);
goto cleanup;
}
@@ -243,7 +243,7 @@ static void test_hashmap_generic(void)
hashmap__for_each_entry(map, entry, bkt) {
CHECK(false, "elem_exists",
"unexpected map entries left: %ld = %ld\n",
- (long)entry->key, (long)entry->value);
+ entry->key, entry->value);
goto cleanup;
}
@@ -251,14 +251,14 @@ static void test_hashmap_generic(void)
hashmap__free(map);
}
-static size_t collision_hash_fn(const void *k, void *ctx)
+static size_t collision_hash_fn(uintptr_t k, void *ctx)
{
return 0;
}
static void test_hashmap_multimap(void)
{
- void *k1 = (void *)0, *k2 = (void *)1;
+ uintptr_t k1 = 0, k2 = 1;
struct hashmap_entry *entry;
struct hashmap *map;
long found_msk;
@@ -273,23 +273,23 @@ static void test_hashmap_multimap(void)
* [0] -> 1, 2, 4;
* [1] -> 8, 16, 32;
*/
- err = hashmap__append(map, k1, (void *)1);
+ err = hashmap__append(map, k1, 1);
if (CHECK(err, "elem_add", "failed to add k/v: %d\n", err))
goto cleanup;
- err = hashmap__append(map, k1, (void *)2);
+ err = hashmap__append(map, k1, 2);
if (CHECK(err, "elem_add", "failed to add k/v: %d\n", err))
goto cleanup;
- err = hashmap__append(map, k1, (void *)4);
+ err = hashmap__append(map, k1, 4);
if (CHECK(err, "elem_add", "failed to add k/v: %d\n", err))
goto cleanup;
- err = hashmap__append(map, k2, (void *)8);
+ err = hashmap__append(map, k2, 8);
if (CHECK(err, "elem_add", "failed to add k/v: %d\n", err))
goto cleanup;
- err = hashmap__append(map, k2, (void *)16);
+ err = hashmap__append(map, k2, 16);
if (CHECK(err, "elem_add", "failed to add k/v: %d\n", err))
goto cleanup;
- err = hashmap__append(map, k2, (void *)32);
+ err = hashmap__append(map, k2, 32);
if (CHECK(err, "elem_add", "failed to add k/v: %d\n", err))
goto cleanup;
@@ -300,7 +300,7 @@ static void test_hashmap_multimap(void)
/* verify global iteration still works and sees all values */
found_msk = 0;
hashmap__for_each_entry(map, entry, bkt) {
- found_msk |= (long)entry->value;
+ found_msk |= entry->value;
}
if (CHECK(found_msk != (1 << 6) - 1, "found_msk",
"not all keys iterated: %lx\n", found_msk))
@@ -309,7 +309,7 @@ static void test_hashmap_multimap(void)
/* iterate values for key 1 */
found_msk = 0;
hashmap__for_each_key_entry(map, entry, k1) {
- found_msk |= (long)entry->value;
+ found_msk |= entry->value;
}
if (CHECK(found_msk != (1 | 2 | 4), "found_msk",
"invalid k1 values: %lx\n", found_msk))
@@ -318,7 +318,7 @@ static void test_hashmap_multimap(void)
/* iterate values for key 2 */
found_msk = 0;
hashmap__for_each_key_entry(map, entry, k2) {
- found_msk |= (long)entry->value;
+ found_msk |= entry->value;
}
if (CHECK(found_msk != (8 | 16 | 32), "found_msk",
"invalid k2 values: %lx\n", found_msk))
@@ -333,7 +333,7 @@ static void test_hashmap_empty()
struct hashmap_entry *entry;
int bkt;
struct hashmap *map;
- void *k = (void *)0;
+ uintptr_t k = 0;
/* force collisions */
map = hashmap__new(hash_fn, equal_fn, NULL);
diff --git a/tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c b/tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c
index 287b3ac40227..df26b4d714d5 100644
--- a/tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c
+++ b/tools/testing/selftests/bpf/prog_tests/kprobe_multi_test.c
@@ -312,12 +312,12 @@ static inline __u64 get_time_ns(void)
return (__u64) t.tv_sec * 1000000000 + t.tv_nsec;
}
-static size_t symbol_hash(const void *key, void *ctx __maybe_unused)
+static size_t symbol_hash(uintptr_t key, void *ctx __maybe_unused)
{
return str_hash((const char *) key);
}
-static bool symbol_equal(const void *key1, const void *key2, void *ctx __maybe_unused)
+static bool symbol_equal(uintptr_t key1, uintptr_t key2, void *ctx __maybe_unused)
{
return strcmp((const char *) key1, (const char *) key2) == 0;
}
@@ -372,7 +372,7 @@ static int get_syms(char ***symsp, size_t *cntp)
sizeof("__ftrace_invalid_address__") - 1))
continue;
- err = hashmap__add(map, name, NULL);
+ err = hashmap__add(map, (uintptr_t)name, 0);
if (err == -EEXIST)
continue;
if (err)
--
2.34.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH bpf-next 3/4] libbpf: Resolve unambigous forward declarations
2022-11-02 11:09 [PATCH bpf-next 0/4] libbpf: Resolve unambigous forward declarations Eduard Zingerman
2022-11-02 11:09 ` [PATCH bpf-next 1/4] libbpf: hashmap interface update to uintptr_t -> uintptr_t Eduard Zingerman
2022-11-02 11:09 ` [PATCH bpf-next 2/4] selftests/bpf: hashmap test cases updated for uintptr_t -> uintptr_t interface Eduard Zingerman
@ 2022-11-02 11:09 ` Eduard Zingerman
2022-11-02 16:36 ` Alan Maguire
2022-11-02 11:09 ` [PATCH bpf-next 4/4] selftests/bpf: Tests for btf_dedup_resolve_fwds Eduard Zingerman
3 siblings, 1 reply; 8+ messages in thread
From: Eduard Zingerman @ 2022-11-02 11:09 UTC (permalink / raw)
To: bpf, ast; +Cc: andrii, daniel, kernel-team, yhs, alan.maguire, Eduard Zingerman
Resolve forward declarations that don't take part in type graphs
comparisons if declaration name is unambiguous. Example:
CU #1:
struct foo; // standalone forward declaration
struct foo *some_global;
CU #2:
struct foo { int x; };
struct foo *another_global;
The `struct foo` from CU #1 is not a part of any definition that is
compared against another definition while `btf_dedup_struct_types`
processes structural types. The the BTF after `btf_dedup_struct_types`
the BTF looks as follows:
[1] STRUCT 'foo' size=4 vlen=1 ...
[2] INT 'int' size=4 ...
[3] PTR '(anon)' type_id=1
[4] FWD 'foo' fwd_kind=struct
[5] PTR '(anon)' type_id=4
This commit adds a new pass `btf_dedup_resolve_fwds`, that maps such
forward declarations to structs or unions with identical name in case
if the name is not ambiguous.
The pass is positioned before `btf_dedup_ref_types` so that types
[3] and [5] could be merged as a same type after [1] and [4] are merged.
The final result for the example above looks as follows:
[1] STRUCT 'foo' size=4 vlen=1
'x' type_id=2 bits_offset=0
[2] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
[3] PTR '(anon)' type_id=1
For defconfig kernel with BTF enabled this removes 63 forward
declarations. Examples of removed declarations: `pt_regs`, `in6_addr`.
The running time of `btf__dedup` function is increased by about 3%.
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
---
tools/lib/bpf/btf.c | 140 ++++++++++++++++++++++++++++++++++++++++++--
1 file changed, 136 insertions(+), 4 deletions(-)
diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
index 04db202aac3d..d2f994d30af7 100644
--- a/tools/lib/bpf/btf.c
+++ b/tools/lib/bpf/btf.c
@@ -2881,6 +2881,7 @@ static int btf_dedup_strings(struct btf_dedup *d);
static int btf_dedup_prim_types(struct btf_dedup *d);
static int btf_dedup_struct_types(struct btf_dedup *d);
static int btf_dedup_ref_types(struct btf_dedup *d);
+static int btf_dedup_resolve_fwds(struct btf_dedup *d);
static int btf_dedup_compact_types(struct btf_dedup *d);
static int btf_dedup_remap_types(struct btf_dedup *d);
@@ -2988,15 +2989,16 @@ static int btf_dedup_remap_types(struct btf_dedup *d);
* Algorithm summary
* =================
*
- * Algorithm completes its work in 6 separate passes:
+ * Algorithm completes its work in 7 separate passes:
*
* 1. Strings deduplication.
* 2. Primitive types deduplication (int, enum, fwd).
* 3. Struct/union types deduplication.
- * 4. Reference types deduplication (pointers, typedefs, arrays, funcs, func
+ * 4. Resolve unambiguous forward declarations.
+ * 5. Reference types deduplication (pointers, typedefs, arrays, funcs, func
* protos, and const/volatile/restrict modifiers).
- * 5. Types compaction.
- * 6. Types remapping.
+ * 6. Types compaction.
+ * 7. Types remapping.
*
* Algorithm determines canonical type descriptor, which is a single
* representative type for each truly unique type. This canonical type is the
@@ -3060,6 +3062,11 @@ int btf__dedup(struct btf *btf, const struct btf_dedup_opts *opts)
pr_debug("btf_dedup_struct_types failed:%d\n", err);
goto done;
}
+ err = btf_dedup_resolve_fwds(d);
+ if (err < 0) {
+ pr_debug("btf_dedup_resolve_fwds failed:%d\n", err);
+ goto done;
+ }
err = btf_dedup_ref_types(d);
if (err < 0) {
pr_debug("btf_dedup_ref_types failed:%d\n", err);
@@ -4526,6 +4533,131 @@ static int btf_dedup_ref_types(struct btf_dedup *d)
return 0;
}
+/*
+ * Collect a map from type names to type ids for all canonical structs
+ * and unions. If the same name is shared by several canonical types
+ * use a special value 0 to indicate this fact.
+ */
+static int btf_dedup_fill_unique_names_map(struct btf_dedup *d, struct hashmap *names_map)
+{
+ __u32 nr_types = btf__type_cnt(d->btf);
+ struct btf_type *t;
+ __u32 type_id;
+ __u16 kind;
+ int err;
+
+ /*
+ * Iterate over base and split module ids in order to get all
+ * available structs in the map.
+ */
+ for (type_id = 1; type_id < nr_types; ++type_id) {
+ t = btf_type_by_id(d->btf, type_id);
+ kind = btf_kind(t);
+
+ if (kind != BTF_KIND_STRUCT && kind != BTF_KIND_UNION)
+ continue;
+
+ /* Skip non-canonical types */
+ if (type_id != d->map[type_id])
+ continue;
+
+ err = hashmap__add(names_map, t->name_off, type_id);
+ if (err == -EEXIST)
+ err = hashmap__set(names_map, t->name_off, 0, NULL, NULL);
+
+ if (err)
+ return err;
+ }
+
+ return 0;
+}
+
+static int btf_dedup_resolve_fwd(struct btf_dedup *d, struct hashmap *names_map, __u32 type_id)
+{
+ struct btf_type *t = btf_type_by_id(d->btf, type_id);
+ enum btf_fwd_kind fwd_kind = btf_kflag(t);
+ __u16 cand_kind, kind = btf_kind(t);
+ struct btf_type *cand_t;
+ uintptr_t cand_id = 0;
+
+ if (kind != BTF_KIND_FWD)
+ return 0;
+
+ /* Skip if this FWD already has a mapping */
+ if (type_id != d->map[type_id])
+ return 0;
+
+ hashmap__find(names_map, t->name_off, &cand_id);
+ if (!cand_id)
+ return 0;
+
+ cand_t = btf_type_by_id(d->btf, cand_id);
+ cand_kind = btf_kind(cand_t);
+ if (!(cand_kind == BTF_KIND_STRUCT && fwd_kind == BTF_FWD_STRUCT) &&
+ !(cand_kind == BTF_KIND_UNION && fwd_kind == BTF_FWD_UNION))
+ return 0;
+
+ d->map[type_id] = cand_id;
+
+ return 0;
+}
+
+/*
+ * Resolve unambiguous forward declarations.
+ *
+ * The lion's share of all FWD declarations is resolved during
+ * `btf_dedup_struct_types` phase when different type graphs are
+ * compared against each other. However, if in some compilation unit a
+ * FWD declaration is not a part of a type graph compared against
+ * another type graph that declaration's canonical type would not be
+ * changed. Example:
+ *
+ * CU #1:
+ *
+ * struct foo;
+ * struct foo *some_global;
+ *
+ * CU #2:
+ *
+ * struct foo { int u; };
+ * struct foo *another_global;
+ *
+ * After `btf_dedup_struct_types` the BTF looks as follows:
+ *
+ * [1] STRUCT 'foo' size=4 vlen=1 ...
+ * [2] INT 'int' size=4 ...
+ * [3] PTR '(anon)' type_id=1
+ * [4] FWD 'foo' fwd_kind=struct
+ * [5] PTR '(anon)' type_id=4
+ *
+ * This pass assumes that such FWD declarations should be mapped to
+ * structs or unions with identical name in case if the name is not
+ * ambiguous.
+ */
+static int btf_dedup_resolve_fwds(struct btf_dedup *d)
+{
+ int i, err;
+ struct hashmap *names_map =
+ hashmap__new(btf_dedup_identity_hash_fn, btf_dedup_equal_fn, NULL);
+
+ if (!names_map)
+ return -ENOMEM;
+
+ err = btf_dedup_fill_unique_names_map(d, names_map);
+ if (err < 0)
+ goto exit;
+
+ for (i = 0; i < d->btf->nr_types; i++) {
+ err = btf_dedup_resolve_fwd(d, names_map, d->btf->start_id + i);
+ if (err < 0)
+ goto exit;
+ }
+
+exit:
+ hashmap__free(names_map);
+ return err;
+}
+
/*
* Compact types.
*
--
2.34.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH bpf-next 4/4] selftests/bpf: Tests for btf_dedup_resolve_fwds
2022-11-02 11:09 [PATCH bpf-next 0/4] libbpf: Resolve unambigous forward declarations Eduard Zingerman
` (2 preceding siblings ...)
2022-11-02 11:09 ` [PATCH bpf-next 3/4] libbpf: Resolve unambigous forward declarations Eduard Zingerman
@ 2022-11-02 11:09 ` Eduard Zingerman
2022-11-02 16:47 ` Alan Maguire
3 siblings, 1 reply; 8+ messages in thread
From: Eduard Zingerman @ 2022-11-02 11:09 UTC (permalink / raw)
To: bpf, ast; +Cc: andrii, daniel, kernel-team, yhs, alan.maguire, Eduard Zingerman
Tests to verify the following behavior of `btf_dedup_resolve_fwds`:
- remapping for struct forward declarations;
- remapping for union forward declarations;
- no remapping if forward declaration kind does not match similarly
named struct or union declaration;
- no remapping if forward declaration name is ambiguous;
- base ids are considered for fwd resolution in split btf scenario.
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
---
tools/testing/selftests/bpf/prog_tests/btf.c | 152 ++++++++++++++++++
.../bpf/prog_tests/btf_dedup_split.c | 45 ++++--
2 files changed, 182 insertions(+), 15 deletions(-)
diff --git a/tools/testing/selftests/bpf/prog_tests/btf.c b/tools/testing/selftests/bpf/prog_tests/btf.c
index 127b8caa3dc1..f14020d51ab9 100644
--- a/tools/testing/selftests/bpf/prog_tests/btf.c
+++ b/tools/testing/selftests/bpf/prog_tests/btf.c
@@ -7598,6 +7598,158 @@ static struct btf_dedup_test dedup_tests[] = {
BTF_STR_SEC("\0e1\0e1_val"),
},
},
+{
+ .descr = "dedup: standalone fwd declaration struct",
+ /*
+ * // CU 1:
+ * struct foo { int x; };
+ * struct foo *a;
+ *
+ * // CU 2:
+ * struct foo;
+ * struct foo *b;
+ */
+ .input = {
+ .raw_types = {
+ BTF_STRUCT_ENC(NAME_NTH(1), 1, 4), /* [1] */
+ BTF_MEMBER_ENC(NAME_NTH(2), 2, 0),
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [2] */
+ BTF_PTR_ENC(1), /* [3] */
+ BTF_FWD_ENC(NAME_TBD, 0), /* [4] */
+ BTF_PTR_ENC(4), /* [5] */
+ BTF_END_RAW,
+ },
+ BTF_STR_SEC("\0foo\0x"),
+ },
+ .expect = {
+ .raw_types = {
+ BTF_STRUCT_ENC(NAME_NTH(1), 1, 4), /* [1] */
+ BTF_MEMBER_ENC(NAME_NTH(2), 2, 0),
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [2] */
+ BTF_PTR_ENC(1), /* [3] */
+ BTF_END_RAW,
+ },
+ BTF_STR_SEC("\0foo\0x"),
+ },
+},
+{
+ .descr = "dedup: standalone fwd declaration union",
+ /*
+ * // CU 1:
+ * union foo { int x; };
+ * union foo *another_global;
+ *
+ * // CU 2:
+ * union foo;
+ * union foo *some_global;
+ */
+ .input = {
+ .raw_types = {
+ BTF_UNION_ENC(NAME_NTH(1), 1, 4), /* [1] */
+ BTF_MEMBER_ENC(NAME_NTH(2), 2, 0),
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [2] */
+ BTF_PTR_ENC(1), /* [3] */
+ BTF_FWD_ENC(NAME_TBD, 1), /* [4] */
+ BTF_PTR_ENC(4), /* [5] */
+ BTF_END_RAW,
+ },
+ BTF_STR_SEC("\0foo\0x"),
+ },
+ .expect = {
+ .raw_types = {
+ BTF_UNION_ENC(NAME_NTH(1), 1, 4), /* [1] */
+ BTF_MEMBER_ENC(NAME_NTH(2), 2, 0),
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [2] */
+ BTF_PTR_ENC(1), /* [3] */
+ BTF_END_RAW,
+ },
+ BTF_STR_SEC("\0foo\0x"),
+ },
+},
+{
+ .descr = "dedup: standalone fwd declaration wrong kind",
+ /*
+ * // CU 1:
+ * struct foo { int x; };
+ * struct foo *b;
+ *
+ * // CU 2:
+ * union foo;
+ * union foo *a;
+ */
+ .input = {
+ .raw_types = {
+ BTF_STRUCT_ENC(NAME_NTH(1), 1, 4), /* [1] */
+ BTF_MEMBER_ENC(NAME_NTH(2), 2, 0),
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [2] */
+ BTF_PTR_ENC(1), /* [3] */
+ BTF_FWD_ENC(NAME_TBD, 1), /* [4] */
+ BTF_PTR_ENC(4), /* [5] */
+ BTF_END_RAW,
+ },
+ BTF_STR_SEC("\0foo\0x"),
+ },
+ .expect = {
+ .raw_types = {
+ BTF_STRUCT_ENC(NAME_NTH(1), 1, 4), /* [1] */
+ BTF_MEMBER_ENC(NAME_NTH(2), 2, 0),
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [2] */
+ BTF_PTR_ENC(1), /* [3] */
+ BTF_FWD_ENC(NAME_TBD, 1), /* [4] */
+ BTF_PTR_ENC(4), /* [5] */
+ BTF_END_RAW,
+ },
+ BTF_STR_SEC("\0foo\0x"),
+ },
+},
+{
+ .descr = "dedup: standalone fwd declaration name conflict",
+ /*
+ * // CU 1:
+ * struct foo { int x; };
+ * struct foo *a;
+ *
+ * // CU 2:
+ * struct foo;
+ * struct foo *b;
+ *
+ * // CU 3:
+ * struct foo { int x; int y; };
+ * struct foo *c;
+ */
+ .input = {
+ .raw_types = {
+ BTF_STRUCT_ENC(NAME_NTH(1), 1, 4), /* [1] */
+ BTF_MEMBER_ENC(NAME_NTH(2), 2, 0),
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [2] */
+ BTF_PTR_ENC(1), /* [3] */
+ BTF_FWD_ENC(NAME_TBD, 0), /* [4] */
+ BTF_PTR_ENC(4), /* [5] */
+ BTF_STRUCT_ENC(NAME_NTH(1), 2, 8), /* [6] */
+ BTF_MEMBER_ENC(NAME_NTH(2), 2, 0),
+ BTF_MEMBER_ENC(NAME_NTH(3), 2, 0),
+ BTF_PTR_ENC(6), /* [7] */
+ BTF_END_RAW,
+ },
+ BTF_STR_SEC("\0foo\0x\0y"),
+ },
+ .expect = {
+ .raw_types = {
+ BTF_STRUCT_ENC(NAME_NTH(1), 1, 4), /* [1] */
+ BTF_MEMBER_ENC(NAME_NTH(2), 2, 0),
+ BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [2] */
+ BTF_PTR_ENC(1), /* [3] */
+ BTF_FWD_ENC(NAME_TBD, 0), /* [4] */
+ BTF_PTR_ENC(4), /* [5] */
+ BTF_STRUCT_ENC(NAME_NTH(1), 2, 8), /* [6] */
+ BTF_MEMBER_ENC(NAME_NTH(2), 2, 0),
+ BTF_MEMBER_ENC(NAME_NTH(3), 2, 0),
+ BTF_PTR_ENC(6), /* [7] */
+ BTF_END_RAW,
+ },
+ BTF_STR_SEC("\0foo\0x\0y"),
+ },
+},
};
diff --git a/tools/testing/selftests/bpf/prog_tests/btf_dedup_split.c b/tools/testing/selftests/bpf/prog_tests/btf_dedup_split.c
index 90aac437576d..d9024c7a892a 100644
--- a/tools/testing/selftests/bpf/prog_tests/btf_dedup_split.c
+++ b/tools/testing/selftests/bpf/prog_tests/btf_dedup_split.c
@@ -141,6 +141,10 @@ static void test_split_fwd_resolve() {
btf__add_field(btf1, "f2", 3, 64, 0); /* struct s2 *f2; */
/* } */
btf__add_struct(btf1, "s2", 4); /* [5] struct s2 { */
+ btf__add_field(btf1, "f1", 1, 0, 0); /* int f1; */
+ /* } */
+ /* keep this not a part of type the graph to test btf_dedup_resolve_fwds */
+ btf__add_struct(btf1, "s3", 4); /* [6] struct s3 { */
btf__add_field(btf1, "f1", 1, 0, 0); /* int f1; */
/* } */
@@ -153,20 +157,24 @@ static void test_split_fwd_resolve() {
"\t'f1' type_id=2 bits_offset=0\n"
"\t'f2' type_id=3 bits_offset=64",
"[5] STRUCT 's2' size=4 vlen=1\n"
+ "\t'f1' type_id=1 bits_offset=0",
+ "[6] STRUCT 's3' size=4 vlen=1\n"
"\t'f1' type_id=1 bits_offset=0");
btf2 = btf__new_empty_split(btf1);
if (!ASSERT_OK_PTR(btf2, "empty_split_btf"))
goto cleanup;
- btf__add_int(btf2, "int", 4, BTF_INT_SIGNED); /* [6] int */
- btf__add_ptr(btf2, 10); /* [7] ptr to struct s1 */
- btf__add_fwd(btf2, "s2", BTF_FWD_STRUCT); /* [8] fwd for struct s2 */
- btf__add_ptr(btf2, 8); /* [9] ptr to fwd struct s2 */
- btf__add_struct(btf2, "s1", 16); /* [10] struct s1 { */
- btf__add_field(btf2, "f1", 7, 0, 0); /* struct s1 *f1; */
- btf__add_field(btf2, "f2", 9, 64, 0); /* struct s2 *f2; */
+ btf__add_int(btf2, "int", 4, BTF_INT_SIGNED); /* [7] int */
+ btf__add_ptr(btf2, 11); /* [8] ptr to struct s1 */
+ btf__add_fwd(btf2, "s2", BTF_FWD_STRUCT); /* [9] fwd for struct s2 */
+ btf__add_ptr(btf2, 9); /* [10] ptr to fwd struct s2 */
+ btf__add_struct(btf2, "s1", 16); /* [11] struct s1 { */
+ btf__add_field(btf2, "f1", 8, 0, 0); /* struct s1 *f1; */
+ btf__add_field(btf2, "f2", 10, 64, 0); /* struct s2 *f2; */
/* } */
+ btf__add_fwd(btf2, "s3", BTF_FWD_STRUCT); /* [12] fwd for struct s3 */
+ btf__add_ptr(btf2, 12); /* [13] ptr to struct s1 */
VALIDATE_RAW_BTF(
btf2,
@@ -178,13 +186,17 @@ static void test_split_fwd_resolve() {
"\t'f2' type_id=3 bits_offset=64",
"[5] STRUCT 's2' size=4 vlen=1\n"
"\t'f1' type_id=1 bits_offset=0",
- "[6] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED",
- "[7] PTR '(anon)' type_id=10",
- "[8] FWD 's2' fwd_kind=struct",
- "[9] PTR '(anon)' type_id=8",
- "[10] STRUCT 's1' size=16 vlen=2\n"
- "\t'f1' type_id=7 bits_offset=0\n"
- "\t'f2' type_id=9 bits_offset=64");
+ "[6] STRUCT 's3' size=4 vlen=1\n"
+ "\t'f1' type_id=1 bits_offset=0",
+ "[7] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED",
+ "[8] PTR '(anon)' type_id=11",
+ "[9] FWD 's2' fwd_kind=struct",
+ "[10] PTR '(anon)' type_id=9",
+ "[11] STRUCT 's1' size=16 vlen=2\n"
+ "\t'f1' type_id=8 bits_offset=0\n"
+ "\t'f2' type_id=10 bits_offset=64",
+ "[12] FWD 's3' fwd_kind=struct",
+ "[13] PTR '(anon)' type_id=12");
err = btf__dedup(btf2, NULL);
if (!ASSERT_OK(err, "btf_dedup"))
@@ -199,7 +211,10 @@ static void test_split_fwd_resolve() {
"\t'f1' type_id=2 bits_offset=0\n"
"\t'f2' type_id=3 bits_offset=64",
"[5] STRUCT 's2' size=4 vlen=1\n"
- "\t'f1' type_id=1 bits_offset=0");
+ "\t'f1' type_id=1 bits_offset=0",
+ "[6] STRUCT 's3' size=4 vlen=1\n"
+ "\t'f1' type_id=1 bits_offset=0",
+ "[7] PTR '(anon)' type_id=6");
cleanup:
btf__free(btf2);
--
2.34.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH bpf-next 3/4] libbpf: Resolve unambigous forward declarations
2022-11-02 11:09 ` [PATCH bpf-next 3/4] libbpf: Resolve unambigous forward declarations Eduard Zingerman
@ 2022-11-02 16:36 ` Alan Maguire
2022-11-02 16:46 ` Eduard Zingerman
0 siblings, 1 reply; 8+ messages in thread
From: Alan Maguire @ 2022-11-02 16:36 UTC (permalink / raw)
To: Eduard Zingerman, bpf, ast; +Cc: andrii, daniel, kernel-team, yhs
On 02/11/2022 11:09, Eduard Zingerman wrote:
> Resolve forward declarations that don't take part in type graphs
> comparisons if declaration name is unambiguous. Example:
>
> CU #1:
>
> struct foo; // standalone forward declaration
> struct foo *some_global;
>
> CU #2:
>
> struct foo { int x; };
> struct foo *another_global;
>
> The `struct foo` from CU #1 is not a part of any definition that is
> compared against another definition while `btf_dedup_struct_types`
> processes structural types. The the BTF after `btf_dedup_struct_types`
> the BTF looks as follows:
>
> [1] STRUCT 'foo' size=4 vlen=1 ...
> [2] INT 'int' size=4 ...
> [3] PTR '(anon)' type_id=1
> [4] FWD 'foo' fwd_kind=struct
> [5] PTR '(anon)' type_id=4
>
> This commit adds a new pass `btf_dedup_resolve_fwds`, that maps such
> forward declarations to structs or unions with identical name in case
> if the name is not ambiguous.
>
> The pass is positioned before `btf_dedup_ref_types` so that types
> [3] and [5] could be merged as a same type after [1] and [4] are merged.
> The final result for the example above looks as follows:
>
> [1] STRUCT 'foo' size=4 vlen=1
> 'x' type_id=2 bits_offset=0
> [2] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
> [3] PTR '(anon)' type_id=1
>
> For defconfig kernel with BTF enabled this removes 63 forward
> declarations. Examples of removed declarations: `pt_regs`, `in6_addr`.
> The running time of `btf__dedup` function is increased by about 3%.
>
> Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
A few small things below, but looks great!
Reviewed-by: Alan Maguire <alan.maguire@oracle.com>
> ---
> tools/lib/bpf/btf.c | 140 ++++++++++++++++++++++++++++++++++++++++++--
> 1 file changed, 136 insertions(+), 4 deletions(-)
>
> diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> index 04db202aac3d..d2f994d30af7 100644
> --- a/tools/lib/bpf/btf.c
> +++ b/tools/lib/bpf/btf.c
> @@ -2881,6 +2881,7 @@ static int btf_dedup_strings(struct btf_dedup *d);
> static int btf_dedup_prim_types(struct btf_dedup *d);
> static int btf_dedup_struct_types(struct btf_dedup *d);
> static int btf_dedup_ref_types(struct btf_dedup *d);
> +static int btf_dedup_resolve_fwds(struct btf_dedup *d);
> static int btf_dedup_compact_types(struct btf_dedup *d);
> static int btf_dedup_remap_types(struct btf_dedup *d);
>
> @@ -2988,15 +2989,16 @@ static int btf_dedup_remap_types(struct btf_dedup *d);
> * Algorithm summary
> * =================
> *
> - * Algorithm completes its work in 6 separate passes:
> + * Algorithm completes its work in 7 separate passes:
> *
> * 1. Strings deduplication.
> * 2. Primitive types deduplication (int, enum, fwd).
> * 3. Struct/union types deduplication.
> - * 4. Reference types deduplication (pointers, typedefs, arrays, funcs, func
> + * 4. Resolve unambiguous forward declarations.
> + * 5. Reference types deduplication (pointers, typedefs, arrays, funcs, func
> * protos, and const/volatile/restrict modifiers).
> - * 5. Types compaction.
> - * 6. Types remapping.
> + * 6. Types compaction.
> + * 7. Types remapping.
> *
> * Algorithm determines canonical type descriptor, which is a single
> * representative type for each truly unique type. This canonical type is the
> @@ -3060,6 +3062,11 @@ int btf__dedup(struct btf *btf, const struct btf_dedup_opts *opts)
> pr_debug("btf_dedup_struct_types failed:%d\n", err);
> goto done;
> }
> + err = btf_dedup_resolve_fwds(d);
> + if (err < 0) {
> + pr_debug("btf_dedup_resolve_fwds failed:%d\n", err);
> + goto done;
> + }
> err = btf_dedup_ref_types(d);
> if (err < 0) {
> pr_debug("btf_dedup_ref_types failed:%d\n", err);
> @@ -4526,6 +4533,131 @@ static int btf_dedup_ref_types(struct btf_dedup *d)
> return 0;
> }
>
> +/*
> + * Collect a map from type names to type ids for all canonical structs
> + * and unions. If the same name is shared by several canonical types
> + * use a special value 0 to indicate this fact.
> + */
> +static int btf_dedup_fill_unique_names_map(struct btf_dedup *d, struct hashmap *names_map)
> +{
> + __u32 nr_types = btf__type_cnt(d->btf);
> + struct btf_type *t;
> + __u32 type_id;
> + __u16 kind;
> + int err;
> +
> + /*
> + * Iterate over base and split module ids in order to get all
> + * available structs in the map.
> + */
> + for (type_id = 1; type_id < nr_types; ++type_id) {
> + t = btf_type_by_id(d->btf, type_id);
> + kind = btf_kind(t);
> +
> + if (kind != BTF_KIND_STRUCT && kind != BTF_KIND_UNION)
> + continue;
> +
> + /* Skip non-canonical types */
> + if (type_id != d->map[type_id])
> + continue;
> +
> + err = hashmap__add(names_map, t->name_off, type_id);
> + if (err == -EEXIST)
> + err = hashmap__set(names_map, t->name_off, 0, NULL, NULL);
> +> + if (err)
> + return err;
> + }
> +
> + return 0;
> +}
> +
> +static int btf_dedup_resolve_fwd(struct btf_dedup *d, struct hashmap *names_map, __u32 type_id)
> +{
> + struct btf_type *t = btf_type_by_id(d->btf, type_id);
> + enum btf_fwd_kind fwd_kind = btf_kflag(t);
> + __u16 cand_kind, kind = btf_kind(t);
> + struct btf_type *cand_t;
> + uintptr_t cand_id = 0;
> +
> + if (kind != BTF_KIND_FWD)
> + return 0;
> +
> + /* Skip if this FWD already has a mapping */
> + if (type_id != d->map[type_id])
> + return 0;
> +
> + hashmap__find(names_map, t->name_off, &cand_id);
would it be safer to do
if (!hashmap__find(names_map, t->name_off, &cand_id))
return 0;
> + if (!cand_id)
> + return 0;
> +
...and might be no harm to reiterate the special meaning of 0 here (multiple
name matches -> ambiguous) since it's a valid type id (void) in other cases.
While strictly you probably don't need separate conditions for not found
and found ambiguous name, it might read a bit easier and more consistently
with other users of hashmap__find().
> + cand_t = btf_type_by_id(d->btf, cand_id);
> + cand_kind = btf_kind(cand_t);
> + if (!(cand_kind == BTF_KIND_STRUCT && fwd_kind == BTF_FWD_STRUCT) &&
> + !(cand_kind == BTF_KIND_UNION && fwd_kind == BTF_FWD_UNION))
> + return 0;
> +
I'd find
if ((cand_id == BTF_KIND_STRUCT && fwd_kind != BTF_FWD_STRUCT) ||
(cand_id == BTF_KIND_UNION && fwd_kind != BTF_FWD_UNION))
...a bit easier to parse, but again not a big deal.
> + d->map[type_id] = cand_id;
> +
> + return 0;
> +}
> +
> +/*
> + * Resolve unambiguous forward declarations.
> + *
> + * The lion's share of all FWD declarations is resolved during
> + * `btf_dedup_struct_types` phase when different type graphs are
> + * compared against each other. However, if in some compilation unit a
> + * FWD declaration is not a part of a type graph compared against
> + * another type graph that declaration's canonical type would not be
> + * changed. Example:
> + *
> + * CU #1:
> + *
> + * struct foo;
> + * struct foo *some_global;
> + *
> + * CU #2:
> + *
> + * struct foo { int u; };
> + * struct foo *another_global;
> + *
> + * After `btf_dedup_struct_types` the BTF looks as follows:
> + *
> + * [1] STRUCT 'foo' size=4 vlen=1 ...
> + * [2] INT 'int' size=4 ...
> + * [3] PTR '(anon)' type_id=1
> + * [4] FWD 'foo' fwd_kind=struct
> + * [5] PTR '(anon)' type_id=4
> + *
> + * This pass assumes that such FWD declarations should be mapped to
> + * structs or unions with identical name in case if the name is not
> + * ambiguous.
> + */
> +static int btf_dedup_resolve_fwds(struct btf_dedup *d)
> +{
> + int i, err;
> + struct hashmap *names_map =
> + hashmap__new(btf_dedup_identity_hash_fn, btf_dedup_equal_fn, NULL);
> +
> + if (!names_map)
> + return -ENOMEM;
> +
> + err = btf_dedup_fill_unique_names_map(d, names_map);
> + if (err < 0)
> + goto exit;
> +
> + for (i = 0; i < d->btf->nr_types; i++) {
> + err = btf_dedup_resolve_fwd(d, names_map, d->btf->start_id + i);
> + if (err < 0)
> + goto exit;
could just break; here I suppose
> + }
> +
> +exit:
> + hashmap__free(names_map);
> + return err;
> +}
> +
> /*
> * Compact types.
> *
>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH bpf-next 3/4] libbpf: Resolve unambigous forward declarations
2022-11-02 16:36 ` Alan Maguire
@ 2022-11-02 16:46 ` Eduard Zingerman
0 siblings, 0 replies; 8+ messages in thread
From: Eduard Zingerman @ 2022-11-02 16:46 UTC (permalink / raw)
To: Alan Maguire, bpf, ast; +Cc: andrii, daniel, kernel-team, yhs
On Wed, 2022-11-02 at 16:36 +0000, Alan Maguire wrote:
> On 02/11/2022 11:09, Eduard Zingerman wrote:
> > Resolve forward declarations that don't take part in type graphs
> > comparisons if declaration name is unambiguous. Example:
> >
> > CU #1:
> >
> > struct foo; // standalone forward declaration
> > struct foo *some_global;
> >
> > CU #2:
> >
> > struct foo { int x; };
> > struct foo *another_global;
> >
> > The `struct foo` from CU #1 is not a part of any definition that is
> > compared against another definition while `btf_dedup_struct_types`
> > processes structural types. The the BTF after `btf_dedup_struct_types`
> > the BTF looks as follows:
> >
> > [1] STRUCT 'foo' size=4 vlen=1 ...
> > [2] INT 'int' size=4 ...
> > [3] PTR '(anon)' type_id=1
> > [4] FWD 'foo' fwd_kind=struct
> > [5] PTR '(anon)' type_id=4
> >
> > This commit adds a new pass `btf_dedup_resolve_fwds`, that maps such
> > forward declarations to structs or unions with identical name in case
> > if the name is not ambiguous.
> >
> > The pass is positioned before `btf_dedup_ref_types` so that types
> > [3] and [5] could be merged as a same type after [1] and [4] are merged.
> > The final result for the example above looks as follows:
> >
> > [1] STRUCT 'foo' size=4 vlen=1
> > 'x' type_id=2 bits_offset=0
> > [2] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
> > [3] PTR '(anon)' type_id=1
> >
> > For defconfig kernel with BTF enabled this removes 63 forward
> > declarations. Examples of removed declarations: `pt_regs`, `in6_addr`.
> > The running time of `btf__dedup` function is increased by about 3%.
> >
> > Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
>
> A few small things below, but looks great!
Hi Alan,
thanks for the review, I'll apply your suggestions in v2.
Will wait a bit in case anyone else would want to comment.
Thanks,
Eduard
>
> Reviewed-by: Alan Maguire <alan.maguire@oracle.com>
>
> > ---
> > tools/lib/bpf/btf.c | 140 ++++++++++++++++++++++++++++++++++++++++++--
> > 1 file changed, 136 insertions(+), 4 deletions(-)
> >
> > diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> > index 04db202aac3d..d2f994d30af7 100644
> > --- a/tools/lib/bpf/btf.c
> > +++ b/tools/lib/bpf/btf.c
> > @@ -2881,6 +2881,7 @@ static int btf_dedup_strings(struct btf_dedup *d);
> > static int btf_dedup_prim_types(struct btf_dedup *d);
> > static int btf_dedup_struct_types(struct btf_dedup *d);
> > static int btf_dedup_ref_types(struct btf_dedup *d);
> > +static int btf_dedup_resolve_fwds(struct btf_dedup *d);
> > static int btf_dedup_compact_types(struct btf_dedup *d);
> > static int btf_dedup_remap_types(struct btf_dedup *d);
> >
> > @@ -2988,15 +2989,16 @@ static int btf_dedup_remap_types(struct btf_dedup *d);
> > * Algorithm summary
> > * =================
> > *
> > - * Algorithm completes its work in 6 separate passes:
> > + * Algorithm completes its work in 7 separate passes:
> > *
> > * 1. Strings deduplication.
> > * 2. Primitive types deduplication (int, enum, fwd).
> > * 3. Struct/union types deduplication.
> > - * 4. Reference types deduplication (pointers, typedefs, arrays, funcs, func
> > + * 4. Resolve unambiguous forward declarations.
> > + * 5. Reference types deduplication (pointers, typedefs, arrays, funcs, func
> > * protos, and const/volatile/restrict modifiers).
> > - * 5. Types compaction.
> > - * 6. Types remapping.
> > + * 6. Types compaction.
> > + * 7. Types remapping.
> > *
> > * Algorithm determines canonical type descriptor, which is a single
> > * representative type for each truly unique type. This canonical type is the
> > @@ -3060,6 +3062,11 @@ int btf__dedup(struct btf *btf, const struct btf_dedup_opts *opts)
> > pr_debug("btf_dedup_struct_types failed:%d\n", err);
> > goto done;
> > }
> > + err = btf_dedup_resolve_fwds(d);
> > + if (err < 0) {
> > + pr_debug("btf_dedup_resolve_fwds failed:%d\n", err);
> > + goto done;
> > + }
> > err = btf_dedup_ref_types(d);
> > if (err < 0) {
> > pr_debug("btf_dedup_ref_types failed:%d\n", err);
> > @@ -4526,6 +4533,131 @@ static int btf_dedup_ref_types(struct btf_dedup *d)
> > return 0;
> > }
> >
> > +/*
> > + * Collect a map from type names to type ids for all canonical structs
> > + * and unions. If the same name is shared by several canonical types
> > + * use a special value 0 to indicate this fact.
> > + */
> > +static int btf_dedup_fill_unique_names_map(struct btf_dedup *d, struct hashmap *names_map)
> > +{
> > + __u32 nr_types = btf__type_cnt(d->btf);
> > + struct btf_type *t;
> > + __u32 type_id;
> > + __u16 kind;
> > + int err;
> > +
> > + /*
> > + * Iterate over base and split module ids in order to get all
> > + * available structs in the map.
> > + */
> > + for (type_id = 1; type_id < nr_types; ++type_id) {
> > + t = btf_type_by_id(d->btf, type_id);
> > + kind = btf_kind(t);
> > +
> > + if (kind != BTF_KIND_STRUCT && kind != BTF_KIND_UNION)
> > + continue;
> > +
> > + /* Skip non-canonical types */
> > + if (type_id != d->map[type_id])
> > + continue;
> > +
> > + err = hashmap__add(names_map, t->name_off, type_id);
> > + if (err == -EEXIST)
> > + err = hashmap__set(names_map, t->name_off, 0, NULL, NULL);
> > +> + if (err)
> > + return err;
> > + }
> > +
> > + return 0;
> > +}
> > +
> > +static int btf_dedup_resolve_fwd(struct btf_dedup *d, struct hashmap *names_map, __u32 type_id)
> > +{
> > + struct btf_type *t = btf_type_by_id(d->btf, type_id);
> > + enum btf_fwd_kind fwd_kind = btf_kflag(t);
> > + __u16 cand_kind, kind = btf_kind(t);
> > + struct btf_type *cand_t;
> > + uintptr_t cand_id = 0;
> > +
> > + if (kind != BTF_KIND_FWD)
> > + return 0;
> > +
> > + /* Skip if this FWD already has a mapping */
> > + if (type_id != d->map[type_id])
> > + return 0;
> > +
> > + hashmap__find(names_map, t->name_off, &cand_id);
>
> would it be safer to do
>
> if (!hashmap__find(names_map, t->name_off, &cand_id))
> return 0;
>
> > + if (!cand_id)
> > + return 0;
> > +
>
> ...and might be no harm to reiterate the special meaning of 0 here (multiple
> name matches -> ambiguous) since it's a valid type id (void) in other cases.
> While strictly you probably don't need separate conditions for not found
> and found ambiguous name, it might read a bit easier and more consistently
> with other users of hashmap__find().
>
> > + cand_t = btf_type_by_id(d->btf, cand_id);
> > + cand_kind = btf_kind(cand_t);
> > + if (!(cand_kind == BTF_KIND_STRUCT && fwd_kind == BTF_FWD_STRUCT) &&
> > + !(cand_kind == BTF_KIND_UNION && fwd_kind == BTF_FWD_UNION))
> > + return 0;
> > +
>
> I'd find
>
> if ((cand_id == BTF_KIND_STRUCT && fwd_kind != BTF_FWD_STRUCT) ||
> (cand_id == BTF_KIND_UNION && fwd_kind != BTF_FWD_UNION))
>
> ...a bit easier to parse, but again not a big deal.
>
> > + d->map[type_id] = cand_id;
> > +
> > + return 0;
> > +}
> > +
> > +/*
> > + * Resolve unambiguous forward declarations.
> > + *
> > + * The lion's share of all FWD declarations is resolved during
> > + * `btf_dedup_struct_types` phase when different type graphs are
> > + * compared against each other. However, if in some compilation unit a
> > + * FWD declaration is not a part of a type graph compared against
> > + * another type graph that declaration's canonical type would not be
> > + * changed. Example:
> > + *
> > + * CU #1:
> > + *
> > + * struct foo;
> > + * struct foo *some_global;
> > + *
> > + * CU #2:
> > + *
> > + * struct foo { int u; };
> > + * struct foo *another_global;
> > + *
> > + * After `btf_dedup_struct_types` the BTF looks as follows:
> > + *
> > + * [1] STRUCT 'foo' size=4 vlen=1 ...
> > + * [2] INT 'int' size=4 ...
> > + * [3] PTR '(anon)' type_id=1
> > + * [4] FWD 'foo' fwd_kind=struct
> > + * [5] PTR '(anon)' type_id=4
> > + *
> > + * This pass assumes that such FWD declarations should be mapped to
> > + * structs or unions with identical name in case if the name is not
> > + * ambiguous.
> > + */
> > +static int btf_dedup_resolve_fwds(struct btf_dedup *d)
> > +{
> > + int i, err;
> > + struct hashmap *names_map =
> > + hashmap__new(btf_dedup_identity_hash_fn, btf_dedup_equal_fn, NULL);
> > +
> > + if (!names_map)
> > + return -ENOMEM;
> > +
> > + err = btf_dedup_fill_unique_names_map(d, names_map);
> > + if (err < 0)
> > + goto exit;
> > +
> > + for (i = 0; i < d->btf->nr_types; i++) {
> > + err = btf_dedup_resolve_fwd(d, names_map, d->btf->start_id + i);
> > + if (err < 0)
> > + goto exit;
>
> could just break; here I suppose
>
> > + }
> > +
> > +exit:
> > + hashmap__free(names_map);
> > + return err;
> > +}
> > +
> > /*
> > * Compact types.
> > *
> >
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH bpf-next 4/4] selftests/bpf: Tests for btf_dedup_resolve_fwds
2022-11-02 11:09 ` [PATCH bpf-next 4/4] selftests/bpf: Tests for btf_dedup_resolve_fwds Eduard Zingerman
@ 2022-11-02 16:47 ` Alan Maguire
0 siblings, 0 replies; 8+ messages in thread
From: Alan Maguire @ 2022-11-02 16:47 UTC (permalink / raw)
To: Eduard Zingerman, bpf, ast; +Cc: andrii, daniel, kernel-team, yhs
On 02/11/2022 11:09, Eduard Zingerman wrote:
> Tests to verify the following behavior of `btf_dedup_resolve_fwds`:
> - remapping for struct forward declarations;
> - remapping for union forward declarations;
> - no remapping if forward declaration kind does not match similarly
> named struct or union declaration;
> - no remapping if forward declaration name is ambiguous;
> - base ids are considered for fwd resolution in split btf scenario.
>
> Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Really nice having positive and negative tests here!
Reviewed-by: Alan Maguire <alan.maguire@oracle.com>
> ---
> tools/testing/selftests/bpf/prog_tests/btf.c | 152 ++++++++++++++++++
> .../bpf/prog_tests/btf_dedup_split.c | 45 ++++--
> 2 files changed, 182 insertions(+), 15 deletions(-)
>
> diff --git a/tools/testing/selftests/bpf/prog_tests/btf.c b/tools/testing/selftests/bpf/prog_tests/btf.c
> index 127b8caa3dc1..f14020d51ab9 100644
> --- a/tools/testing/selftests/bpf/prog_tests/btf.c
> +++ b/tools/testing/selftests/bpf/prog_tests/btf.c
> @@ -7598,6 +7598,158 @@ static struct btf_dedup_test dedup_tests[] = {
> BTF_STR_SEC("\0e1\0e1_val"),
> },
> },
> +{
> + .descr = "dedup: standalone fwd declaration struct",
> + /*
> + * // CU 1:
> + * struct foo { int x; };
> + * struct foo *a;
> + *
> + * // CU 2:
> + * struct foo;
> + * struct foo *b;
> + */
> + .input = {
> + .raw_types = {
> + BTF_STRUCT_ENC(NAME_NTH(1), 1, 4), /* [1] */
> + BTF_MEMBER_ENC(NAME_NTH(2), 2, 0),
> + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [2] */
> + BTF_PTR_ENC(1), /* [3] */
> + BTF_FWD_ENC(NAME_TBD, 0), /* [4] */
> + BTF_PTR_ENC(4), /* [5] */
> + BTF_END_RAW,
> + },
> + BTF_STR_SEC("\0foo\0x"),
> + },
> + .expect = {
> + .raw_types = {
> + BTF_STRUCT_ENC(NAME_NTH(1), 1, 4), /* [1] */
> + BTF_MEMBER_ENC(NAME_NTH(2), 2, 0),
> + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [2] */
> + BTF_PTR_ENC(1), /* [3] */
> + BTF_END_RAW,
> + },
> + BTF_STR_SEC("\0foo\0x"),
> + },
> +},
> +{
> + .descr = "dedup: standalone fwd declaration union",
> + /*
> + * // CU 1:
> + * union foo { int x; };
> + * union foo *another_global;
> + *
> + * // CU 2:
> + * union foo;
> + * union foo *some_global;
> + */
> + .input = {
> + .raw_types = {
> + BTF_UNION_ENC(NAME_NTH(1), 1, 4), /* [1] */
> + BTF_MEMBER_ENC(NAME_NTH(2), 2, 0),
> + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [2] */
> + BTF_PTR_ENC(1), /* [3] */
> + BTF_FWD_ENC(NAME_TBD, 1), /* [4] */
> + BTF_PTR_ENC(4), /* [5] */
> + BTF_END_RAW,
> + },
> + BTF_STR_SEC("\0foo\0x"),
> + },
> + .expect = {
> + .raw_types = {
> + BTF_UNION_ENC(NAME_NTH(1), 1, 4), /* [1] */
> + BTF_MEMBER_ENC(NAME_NTH(2), 2, 0),
> + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [2] */
> + BTF_PTR_ENC(1), /* [3] */
> + BTF_END_RAW,
> + },
> + BTF_STR_SEC("\0foo\0x"),
> + },
> +},
> +{
> + .descr = "dedup: standalone fwd declaration wrong kind",
> + /*
> + * // CU 1:
> + * struct foo { int x; };
> + * struct foo *b;
> + *
> + * // CU 2:
> + * union foo;
> + * union foo *a;
> + */
> + .input = {
> + .raw_types = {
> + BTF_STRUCT_ENC(NAME_NTH(1), 1, 4), /* [1] */
> + BTF_MEMBER_ENC(NAME_NTH(2), 2, 0),
> + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [2] */
> + BTF_PTR_ENC(1), /* [3] */
> + BTF_FWD_ENC(NAME_TBD, 1), /* [4] */
> + BTF_PTR_ENC(4), /* [5] */
> + BTF_END_RAW,
> + },
> + BTF_STR_SEC("\0foo\0x"),
> + },
> + .expect = {
> + .raw_types = {
> + BTF_STRUCT_ENC(NAME_NTH(1), 1, 4), /* [1] */
> + BTF_MEMBER_ENC(NAME_NTH(2), 2, 0),
> + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [2] */
> + BTF_PTR_ENC(1), /* [3] */
> + BTF_FWD_ENC(NAME_TBD, 1), /* [4] */
> + BTF_PTR_ENC(4), /* [5] */
> + BTF_END_RAW,
> + },
> + BTF_STR_SEC("\0foo\0x"),
> + },
> +},
> +{
> + .descr = "dedup: standalone fwd declaration name conflict",
> + /*
> + * // CU 1:
> + * struct foo { int x; };
> + * struct foo *a;
> + *
> + * // CU 2:
> + * struct foo;
> + * struct foo *b;
> + *
> + * // CU 3:
> + * struct foo { int x; int y; };
> + * struct foo *c;
> + */
> + .input = {
> + .raw_types = {
> + BTF_STRUCT_ENC(NAME_NTH(1), 1, 4), /* [1] */
> + BTF_MEMBER_ENC(NAME_NTH(2), 2, 0),
> + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [2] */
> + BTF_PTR_ENC(1), /* [3] */
> + BTF_FWD_ENC(NAME_TBD, 0), /* [4] */
> + BTF_PTR_ENC(4), /* [5] */
> + BTF_STRUCT_ENC(NAME_NTH(1), 2, 8), /* [6] */
> + BTF_MEMBER_ENC(NAME_NTH(2), 2, 0),
> + BTF_MEMBER_ENC(NAME_NTH(3), 2, 0),
> + BTF_PTR_ENC(6), /* [7] */
> + BTF_END_RAW,
> + },
> + BTF_STR_SEC("\0foo\0x\0y"),
> + },
> + .expect = {
> + .raw_types = {
> + BTF_STRUCT_ENC(NAME_NTH(1), 1, 4), /* [1] */
> + BTF_MEMBER_ENC(NAME_NTH(2), 2, 0),
> + BTF_TYPE_INT_ENC(0, BTF_INT_SIGNED, 0, 32, 4), /* [2] */
> + BTF_PTR_ENC(1), /* [3] */
> + BTF_FWD_ENC(NAME_TBD, 0), /* [4] */
> + BTF_PTR_ENC(4), /* [5] */
> + BTF_STRUCT_ENC(NAME_NTH(1), 2, 8), /* [6] */
> + BTF_MEMBER_ENC(NAME_NTH(2), 2, 0),
> + BTF_MEMBER_ENC(NAME_NTH(3), 2, 0),
> + BTF_PTR_ENC(6), /* [7] */
> + BTF_END_RAW,
> + },
> + BTF_STR_SEC("\0foo\0x\0y"),
> + },
> +},
>
> };
>
> diff --git a/tools/testing/selftests/bpf/prog_tests/btf_dedup_split.c b/tools/testing/selftests/bpf/prog_tests/btf_dedup_split.c
> index 90aac437576d..d9024c7a892a 100644
> --- a/tools/testing/selftests/bpf/prog_tests/btf_dedup_split.c
> +++ b/tools/testing/selftests/bpf/prog_tests/btf_dedup_split.c
> @@ -141,6 +141,10 @@ static void test_split_fwd_resolve() {
> btf__add_field(btf1, "f2", 3, 64, 0); /* struct s2 *f2; */
> /* } */
> btf__add_struct(btf1, "s2", 4); /* [5] struct s2 { */
> + btf__add_field(btf1, "f1", 1, 0, 0); /* int f1; */
> + /* } */
> + /* keep this not a part of type the graph to test btf_dedup_resolve_fwds */
> + btf__add_struct(btf1, "s3", 4); /* [6] struct s3 { */
> btf__add_field(btf1, "f1", 1, 0, 0); /* int f1; */
> /* } */
>
> @@ -153,20 +157,24 @@ static void test_split_fwd_resolve() {
> "\t'f1' type_id=2 bits_offset=0\n"
> "\t'f2' type_id=3 bits_offset=64",
> "[5] STRUCT 's2' size=4 vlen=1\n"
> + "\t'f1' type_id=1 bits_offset=0",
> + "[6] STRUCT 's3' size=4 vlen=1\n"
> "\t'f1' type_id=1 bits_offset=0");
>
> btf2 = btf__new_empty_split(btf1);
> if (!ASSERT_OK_PTR(btf2, "empty_split_btf"))
> goto cleanup;
>
> - btf__add_int(btf2, "int", 4, BTF_INT_SIGNED); /* [6] int */
> - btf__add_ptr(btf2, 10); /* [7] ptr to struct s1 */
> - btf__add_fwd(btf2, "s2", BTF_FWD_STRUCT); /* [8] fwd for struct s2 */
> - btf__add_ptr(btf2, 8); /* [9] ptr to fwd struct s2 */
> - btf__add_struct(btf2, "s1", 16); /* [10] struct s1 { */
> - btf__add_field(btf2, "f1", 7, 0, 0); /* struct s1 *f1; */
> - btf__add_field(btf2, "f2", 9, 64, 0); /* struct s2 *f2; */
> + btf__add_int(btf2, "int", 4, BTF_INT_SIGNED); /* [7] int */
> + btf__add_ptr(btf2, 11); /* [8] ptr to struct s1 */
> + btf__add_fwd(btf2, "s2", BTF_FWD_STRUCT); /* [9] fwd for struct s2 */
> + btf__add_ptr(btf2, 9); /* [10] ptr to fwd struct s2 */
> + btf__add_struct(btf2, "s1", 16); /* [11] struct s1 { */
> + btf__add_field(btf2, "f1", 8, 0, 0); /* struct s1 *f1; */
> + btf__add_field(btf2, "f2", 10, 64, 0); /* struct s2 *f2; */
> /* } */
> + btf__add_fwd(btf2, "s3", BTF_FWD_STRUCT); /* [12] fwd for struct s3 */
> + btf__add_ptr(btf2, 12); /* [13] ptr to struct s1 */
>
> VALIDATE_RAW_BTF(
> btf2,
> @@ -178,13 +186,17 @@ static void test_split_fwd_resolve() {
> "\t'f2' type_id=3 bits_offset=64",
> "[5] STRUCT 's2' size=4 vlen=1\n"
> "\t'f1' type_id=1 bits_offset=0",
> - "[6] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED",
> - "[7] PTR '(anon)' type_id=10",
> - "[8] FWD 's2' fwd_kind=struct",
> - "[9] PTR '(anon)' type_id=8",
> - "[10] STRUCT 's1' size=16 vlen=2\n"
> - "\t'f1' type_id=7 bits_offset=0\n"
> - "\t'f2' type_id=9 bits_offset=64");
> + "[6] STRUCT 's3' size=4 vlen=1\n"
> + "\t'f1' type_id=1 bits_offset=0",
> + "[7] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED",
> + "[8] PTR '(anon)' type_id=11",
> + "[9] FWD 's2' fwd_kind=struct",
> + "[10] PTR '(anon)' type_id=9",
> + "[11] STRUCT 's1' size=16 vlen=2\n"
> + "\t'f1' type_id=8 bits_offset=0\n"
> + "\t'f2' type_id=10 bits_offset=64",
> + "[12] FWD 's3' fwd_kind=struct",
> + "[13] PTR '(anon)' type_id=12");
>
> err = btf__dedup(btf2, NULL);
> if (!ASSERT_OK(err, "btf_dedup"))
> @@ -199,7 +211,10 @@ static void test_split_fwd_resolve() {
> "\t'f1' type_id=2 bits_offset=0\n"
> "\t'f2' type_id=3 bits_offset=64",
> "[5] STRUCT 's2' size=4 vlen=1\n"
> - "\t'f1' type_id=1 bits_offset=0");
> + "\t'f1' type_id=1 bits_offset=0",
> + "[6] STRUCT 's3' size=4 vlen=1\n"
> + "\t'f1' type_id=1 bits_offset=0",
> + "[7] PTR '(anon)' type_id=6");
>
> cleanup:
> btf__free(btf2);
>
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2022-11-02 16:49 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-11-02 11:09 [PATCH bpf-next 0/4] libbpf: Resolve unambigous forward declarations Eduard Zingerman
2022-11-02 11:09 ` [PATCH bpf-next 1/4] libbpf: hashmap interface update to uintptr_t -> uintptr_t Eduard Zingerman
2022-11-02 11:09 ` [PATCH bpf-next 2/4] selftests/bpf: hashmap test cases updated for uintptr_t -> uintptr_t interface Eduard Zingerman
2022-11-02 11:09 ` [PATCH bpf-next 3/4] libbpf: Resolve unambigous forward declarations Eduard Zingerman
2022-11-02 16:36 ` Alan Maguire
2022-11-02 16:46 ` Eduard Zingerman
2022-11-02 11:09 ` [PATCH bpf-next 4/4] selftests/bpf: Tests for btf_dedup_resolve_fwds Eduard Zingerman
2022-11-02 16:47 ` Alan Maguire
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox