[PATCH bpf-next] libbpf: add identical pointer detection to btf_dedup_is_equiv()

[PATCH bpf-next] libbpf: add identical pointer detection to btf_dedup_is_equiv()

[Alan Maguire]

Recently as a side-effect of

commit ac053946f5c4 ("compiler.h: introduce TYPEOF_UNQUAL() macro")

issues were observed in deduplication between modules and kernel BTF
such that a large number of kernel types were not deduplicated so
were found in module BTF (task_struct, bpf_prog etc).  The root cause
appeared to be a failure to dedup struct types, specifically those
with members that were pointers with __percpu annotations.

The issue in dedup is at the point that we are deduplicating structures,
we have not yet deduplicated reference types like pointers.  If multiple
copies of a pointer point at the same (deduplicated) integer as in this
case, we do not see them as identical.  Special handling already exists
to deal with structures and arrays, so add pointer handling here too.

Reported-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 tools/lib/bpf/btf.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
index 24fc71ce5631..eea7fc10d19c 100644
--- a/tools/lib/bpf/btf.c
+++ b/tools/lib/bpf/btf.c
@@ -4396,6 +4396,19 @@ static bool btf_dedup_identical_structs(struct btf_dedup *d, __u32 id1, __u32 id
 	return true;
 }
 
+static bool btf_dedup_identical_ptrs(struct btf_dedup *d, __u32 id1,
+__u32 id2)
+{
+	struct btf_type *t1, *t2;
+
+	t1 = btf_type_by_id(d->btf, id1);
+	t2 = btf_type_by_id(d->btf, id2);
+
+	if (!btf_is_ptr(t1) || !btf_is_ptr(t2))
+		return false;
+	return t1->type == t2->type;
+}
+
 /*
  * Check equivalence of BTF type graph formed by candidate struct/union (we'll
  * call it "candidate graph" in this description for brevity) to a type graph
@@ -4528,6 +4541,9 @@ static int btf_dedup_is_equiv(struct btf_dedup *d, __u32 cand_id,
 		 */
 		if (btf_dedup_identical_structs(d, hypot_type_id, cand_id))
 			return 1;
+		/* A similar case is again observed for PTRs. */
+		if (btf_dedup_identical_ptrs(d, hypot_type_id, cand_id))
+			return 1;
 		return 0;
 	}
 
-- 
2.43.5

Re: [PATCH bpf-next] libbpf: add identical pointer detection to btf_dedup_is_equiv()

[patchwork-bot+netdevbpf]

Hello:

This patch was applied to bpf/bpf-next.git (master)
by Andrii Nakryiko <andrii@kernel.org>:

On Tue, 29 Apr 2025 17:10:42 +0100 you wrote:
> Recently as a side-effect of
> 
> commit ac053946f5c4 ("compiler.h: introduce TYPEOF_UNQUAL() macro")
> 
> issues were observed in deduplication between modules and kernel BTF
> such that a large number of kernel types were not deduplicated so
> were found in module BTF (task_struct, bpf_prog etc).  The root cause
> appeared to be a failure to dedup struct types, specifically those
> with members that were pointers with __percpu annotations.
> 
> [...]

Here is the summary with links:
  - [bpf-next] libbpf: add identical pointer detection to btf_dedup_is_equiv()
    https://git.kernel.org/bpf/bpf-next/c/8e64c387c942

You are awesome, thank you!
-- 
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html

Re: [PATCH bpf-next] libbpf: add identical pointer detection to btf_dedup_is_equiv()

[Andrii Nakryiko]

On Tue, Apr 29, 2025 at 9:10 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>
> Recently as a side-effect of
>
> commit ac053946f5c4 ("compiler.h: introduce TYPEOF_UNQUAL() macro")
>
> issues were observed in deduplication between modules and kernel BTF
> such that a large number of kernel types were not deduplicated so
> were found in module BTF (task_struct, bpf_prog etc).  The root cause
> appeared to be a failure to dedup struct types, specifically those
> with members that were pointers with __percpu annotations.
>
> The issue in dedup is at the point that we are deduplicating structures,
> we have not yet deduplicated reference types like pointers.  If multiple
> copies of a pointer point at the same (deduplicated) integer as in this
> case, we do not see them as identical.  Special handling already exists
> to deal with structures and arrays, so add pointer handling here too.
>
> Reported-by: Alexei Starovoitov <ast@kernel.org>
> Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
> ---
>  tools/lib/bpf/btf.c | 16 ++++++++++++++++
>  1 file changed, 16 insertions(+)
>
> diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> index 24fc71ce5631..eea7fc10d19c 100644
> --- a/tools/lib/bpf/btf.c
> +++ b/tools/lib/bpf/btf.c
> @@ -4396,6 +4396,19 @@ static bool btf_dedup_identical_structs(struct btf_dedup *d, __u32 id1, __u32 id
>         return true;
>  }
>
> +static bool btf_dedup_identical_ptrs(struct btf_dedup *d, __u32 id1,
> +__u32 id2)

fixed up this unintended line wrap, applied to bpf-next, and I've
already synced all the pending libbpf changes to Github ([0]). Alan,
feel free to pull all that into pahole master, so it propagates to BPF
CI (and we can start testing GCC 14)

  [0] https://github.com/libbpf/libbpf/pull/899

> +{
> +       struct btf_type *t1, *t2;
> +
> +       t1 = btf_type_by_id(d->btf, id1);
> +       t2 = btf_type_by_id(d->btf, id2);
> +
> +       if (!btf_is_ptr(t1) || !btf_is_ptr(t2))
> +               return false;
> +       return t1->type == t2->type;
> +}
> +
>  /*
>   * Check equivalence of BTF type graph formed by candidate struct/union (we'll
>   * call it "candidate graph" in this description for brevity) to a type graph
> @@ -4528,6 +4541,9 @@ static int btf_dedup_is_equiv(struct btf_dedup *d, __u32 cand_id,
>                  */
>                 if (btf_dedup_identical_structs(d, hypot_type_id, cand_id))
>                         return 1;
> +               /* A similar case is again observed for PTRs. */
> +               if (btf_dedup_identical_ptrs(d, hypot_type_id, cand_id))
> +                       return 1;
>                 return 0;
>         }
>
> --
> 2.43.5
>