Re: [RFC bpf-next 03/15] libbpf: Add option to retrieve map from old->new ids from btf__dedup()

[Andrii Nakryiko <andrii.nakryiko@gmail.com>]

On Fri, Oct 17, 2025 at 1:57 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>
> On 16/10/2025 19:39, Andrii Nakryiko wrote:
> > On Wed, Oct 8, 2025 at 10:35 AM Alan Maguire <alan.maguire@oracle.com> wrote:
> >>
> >> When creating split BTF for the .BTF.extra section to record location
> >> information, we need to add function prototypes that refer to base BTF
> >> (vmlinux) types.  However since .BTF.extra is split BTF we have a
> >> problem; since collecting those type ids for the parameters, the base
> >> vmlinux BTF has been deduplicated so the type ids are stale.  As a
> >> result it is valuable to be able to access the map from old->new type
> >> ids that is constructed as part of deduplication.  This allows us to
> >> update the out-of-date type ids in the FUNC_PROTOs.
> >>
> >> In order to pass the map back, we need to fill out all of the hypot
> >> map mappings; as an optimization normal dedup only computes type id
> >> mappings needed in existing BTF type id references.
> >
> > I probably should look at pahole patches to find out myself, but I'm
> > going to be lazy here. ;) Wouldn't you want to generate .BTF.extra
> > after base BTF was generated and deduped? Or is it too inconvenient?
> > Can you please elaborate a bit with more info?
> >
>
> Yep, the BTF.extra is indeed generated after base BTF+dedup, but the
> problem is we need to cache info about inline sites as we process DWARF
> CUs and collect inline info. Specifically at that time we need to cache
> info about function prototypes associated with inlines, and this is done
> - like it is done for real functions - via btf_encoder__save_func(). It
> saves a representation of the function prototype using BTF ids of
> function parameters, and these are pre-dedup BTF ids.
>
> And it's those BTF ids that are the problem. When we dedup with
> FUNC_PROTOs in the same BTF, all the id references get fixed up, but
> because we now have stale type id references in FUNC_PROTOs in the split
> BTF.extra (that were not fixed up by dedup) since we didn't dedup this
> split BTF yet, we are stuck.
>
> There are other alternatives here I suppose, but they seemed equally
> bad/worse.
>
> One is to rescan all the CUs for later inline site representation once
> vmlinux/module dedup is done. That would make pahole much slower as CU
> processing is the most time-consuming aspect of its operation. It seemed
> better to collect inline info at the same time we collect everything else.
>
> Another is to put the FUNC_PROTOs (that are only needed for inline
> sites) into the vmlinux/module BTF. That would work, but even that would
> exhibit the same problem as even those FUNC_PROTO type id references
> would also get remapped by vmlinux/module dedup.
>
> So it's not an ideal solution, but I couldn't figure out an easier one
> I'm afraid.

Ok, this makes sense at the conceptual level. This might be useful
overall. But I don't like the implementation, sorry.

The size of mapping "table" is fixed, it's btf__type_cnt(). So just
make caller allocate u32 array of that size, and pass it in. Libbpf
will then maintain/populate provided array with original type ID ->
deduped type ID with an absolutely minimal amount of overhead and
extra code.

so just

__u32 dedup_map;
size_t dedup_map_cnt;

inside btf_dedup_opts ? (and we request user to specify count just to
avoid surprises, we do know the size, but user should know it as well)

>
> Alan