[PATCH bpf v2 1/2] libbpf: fix deduplication of typedef with base definitions

[Antoine Tenart <atenart@kernel.org>]

When deduplicating definitions for a module, typedef defined in the base
are not removed. This is because the hash used for base types differs
from the one used in the deduplication logic in btf_dedup_struct_type.

This was introduced by the referenced commit when moving the typedef
deduplication logic handling from btf_dedup_ref_type to
btf_dedup_struct_type, as this also changed the hash logic
(btf_hash_common to btf_hash_typedef).

This also impacts other types referencing those typedef (e.g. const). In
my test, the BTF section size of the openvswitch module went from 31KB
to 45KB.

Cc: Paul Houssel <paulhoussel2@gmail.com>
Fixes: 3781413465df ("libbpf: Fix BTF dedup to support recursive typedef definitions").
Reviewed-by: Alan Maguire <alan.maguire@oracle.com>
Signed-off-by: Antoine Tenart <atenart@kernel.org>
---
 tools/lib/bpf/btf.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
index ceb57b46a878..771aeaa0262b 100644
--- a/tools/lib/bpf/btf.c
+++ b/tools/lib/bpf/btf.c
@@ -4578,12 +4578,14 @@ static int btf_dedup_prep(struct btf_dedup *d)
 		case BTF_KIND_RESTRICT:
 		case BTF_KIND_PTR:
 		case BTF_KIND_FWD:
-		case BTF_KIND_TYPEDEF:
 		case BTF_KIND_FUNC:
 		case BTF_KIND_FLOAT:
 		case BTF_KIND_TYPE_TAG:
 			h = btf_hash_common(t);
 			break;
+		case BTF_KIND_TYPEDEF:
+			h = btf_hash_typedef(t);
+			break;
 		case BTF_KIND_INT:
 		case BTF_KIND_DECL_TAG:
 			h = btf_hash_int_decl_tag(t);
-- 
2.53.0