On Mon, 2025-12-15 at 09:17 +0000, Alan Maguire wrote:

Hi Alan,

Overall makes sense, please fine one suggestion below.

> Support reading in kind layout fixing endian issues on reading;
> also support writing kind layout section to raw BTF object.
> There is not yet an API to populate the kind layout with meaningful
> information.
> 
> As part of this, we need to consider multiple valid BTF header
> sizes; the original or the kind layout-extended headers.
> So to support this, the "struct btf" representation is modified
> to always allocate a "struct btf_header" and copy the valid
> portion from the raw data to it; this means we can always safely
> check fields like btf->hdr->kind_layout_len.
> 
> Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
> ---
>  tools/lib/bpf/btf.c | 260 +++++++++++++++++++++++++++++++-------------
>  1 file changed, 183 insertions(+), 77 deletions(-)
> 
> diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> index b136572e889a..8835aee6ee84 100644
> --- a/tools/lib/bpf/btf.c
> +++ b/tools/lib/bpf/btf.c
> @@ -40,42 +40,53 @@ struct btf {
>  
>  	/*
>  	 * When BTF is loaded from an ELF or raw memory it is stored
> -	 * in a contiguous memory block. The hdr, type_data, and, strs_data
> +	 * in a contiguous memory block. The  type_data, and, strs_data
>  	 * point inside that memory region to their respective parts of BTF
>  	 * representation:
>  	 *
> -	 * +--------------------------------+
> -	 * |  Header  |  Types  |  Strings  |
> -	 * +--------------------------------+
> -	 * ^          ^         ^
> -	 * |          |         |
> -	 * hdr        |         |
> -	 * types_data-+         |
> -	 * strs_data------------+
> +	 * +--------------------------------+---------------------+
> +	 * |  Header  |  Types  |  Strings  |Optional kind layout |
> +	 * +--------------------------------+---------------------+
> +	 * ^          ^         ^           ^
> +	 * |          |         |           |
> +	 * raw_data   |         |           |
> +	 * types_data-+         |           |
> +	 * strs_data------------+           |
> +	 * kind_layout----------------------+
> +	 *
> +	 * A separate struct btf_header is allocated for btf->hdr,
> +	 * and header information is copied into it.  This allows us
> +	 * to handle header data for various header formats; the original,
> +	 * the extended header with kind layout, etc.
>  	 *
>  	 * If BTF data is later modified, e.g., due to types added or
>  	 * removed, BTF deduplication performed, etc, this contiguous
> -	 * representation is broken up into three independently allocated
> -	 * memory regions to be able to modify them independently.
> +	 * representation is broken up into four independent memory
> +	 * regions.
> +	 *
>  	 * raw_data is nulled out at that point, but can be later allocated
>  	 * and cached again if user calls btf__raw_data(), at which point
> -	 * raw_data will contain a contiguous copy of header, types, and
> -	 * strings:
> +	 * raw_data will contain a contiguous copy of header, types, strings
> +	 * and optionally kind_layout.  kind_layout optionally points to a
> +	 * kind_layout array - this allows us to encode information about
> +	 * the kinds known at encoding time.  If kind_layout is NULL no
> +	 * kind information is encoded.
>  	 *
> -	 * +----------+  +---------+  +-----------+
> -	 * |  Header  |  |  Types  |  |  Strings  |
> -	 * +----------+  +---------+  +-----------+
> -	 * ^             ^            ^
> -	 * |             |            |
> -	 * hdr           |            |
> -	 * types_data----+            |
> -	 * strset__data(strs_set)-----+
> +	 * +----------+  +---------+  +-----------+   +-----------+
> +	 * |  Header  |  |  Types  |  |  Strings  |   |kind_layout|
> +	 * +----------+  +---------+  +-----------+   +-----------+
> +	 * ^             ^            ^               ^
> +	 * |             |            |               |
> +	 * hdr           |            |               |
> +	 * types_data----+            |               |
> +	 * strset__data(strs_set)-----+               |
> +	 * kind_layout--------------------------------+
>  	 *
> -	 *               +----------+---------+-----------+
> -	 *               |  Header  |  Types  |  Strings  |
> -	 * raw_data----->+----------+---------+-----------+
> +	 *               +----------+---------+-----------+---------------------+
> +	 *               |  Header  |  Types  |  Strings  | Optional kind layout|
> +	 * raw_data----->+----------+---------+-----------+---------------------+
>  	 */
> -	struct btf_header *hdr;
> +	struct btf_header *hdr; /* separately-allocated header data */

Do we want to directly embed this structure?
E.g. as in the diff attached.
No need to bother with calloc/free in such case.

>  
>  	void *types_data;
>  	size_t types_data_cap; /* used size stored in hdr->type_len */
> @@ -123,6 +134,13 @@ struct btf {
>  	/* whether raw_data is a (read-only) mmap */
>  	bool raw_data_is_mmap;
>  
> +	/* is BTF modifiable? i.e. is it split into separate sections as described above? */
> +	bool modifiable;
> +	/* Points either at raw kind layout data in parsed BTF (if present), or
> +	 * at an allocated kind layout array when BTF is modifiable.
> +	 */
> +	void *kind_layout;
> +
>  	/* BTF object FD, if loaded into kernel */
>  	int fd;
>  
> @@ -214,7 +232,7 @@ static int btf_add_type_idx_entry(struct btf *btf, __u32 type_off)
>  	return 0;
>  }
>  
> -static void btf_bswap_hdr(struct btf_header *h)
> +static void btf_bswap_hdr(struct btf_header *h, __u32 hdr_len)
>  {
>  	h->magic = bswap_16(h->magic);
>  	h->hdr_len = bswap_32(h->hdr_len);

[...]

Thanks,
Eduard