[PATCH v6 bpf-next 00/10] Add kind layout to BTF

[PATCH v6 bpf-next 00/10] Add kind layout to BTF

[Alan Maguire]

Update struct btf_header to add a new "kind_layout" section containing
a description of how to parse the BTF kinds known about at BTF
encoding time.  This provides the opportunity for tools that might
not know all of these kinds - as is the case when older tools run
on more newly-generated BTF - to still parse the BTF provided,
even if it cannot all be used.

Also add support to bpftool to dump metadata about BTF; its size,
header information and kind layout section.

The ideas here were discussed at [1], with further discussion
at [2].

Patches for pahole will enable the kind layout addition during
BTF generation are at [3], but even absent these the addition of the
kind_layout feature in the final patch in this series should not break
anything since such unknown features are simply ignored during pahole
BTF generation.

Changes since v5: [4]:

- removed flags field from kind layout; it is not really workable
  since we would have to define semantics of all possible future
  flags today to be usable. Instead stick to parsing only, which
  means each kind just needs the length of the singular and
  vlen-specified objects (Alexei)
- added documentation for bpftool BTF metadata dump (Quentin, patch 9)

Changes since v4: [5]:

- removed CRC generation since it is not needed to handle modules
  built at different time than kernel; distilled base BTF supports
  this now
- fixed up bpftool display of empty kind names, comment/documentation
  indentation (Quentin, patches 8, 9)

Changes since v3 [6]:

- fixed mismerge issues with kbuild changes for BTF generation
  (patches 9, 14)
- fixed a few small issues in libbpf with kind layout representation
  (patches 2, 4)

Changes since v2 [7]:

- drop "optional" kind flag (Andrii, patch 1)
- allocate "struct btf_header" for struct btf to ensure
  we can always access new fields (Andrii, patch 2)
- use an internal BTF kind array in btf.c to simplify
  kind encoding (Andrii, patch 2)
- drop use of kind layout information for in-kernel parsing,
  since the kernel needs to be strict in what it accepts
  (Andrii, patch 6)
 added CRC verification for BTF objects and for matching
  with base object (Alexei, patches 7,8)
- fixed bpftool json output (Quentin, patch 10)
- added standalone module BTF support, tests (patches 13-17)

Changes since RFC

- Terminology change from meta -> kind_layout
  (Alexei and Andrii)
- Simplify representation, removing meta header
  and just having kind layout section (Alexei)
- Fixed bpftool to have JSON support, support
  prefix match, documented changes (Quentin)
- Separated metadata opts into add_kind_layout
  and add_crc
- Added additional positive/negative tests
  to cover basic unknown kind, one with an
  info_sz object following it and one with
  N elem_sz elements following it.
- Updated pahole-flags to use help output
  rather than version to see if features
  are present


[1] https://lore.kernel.org/bpf/CAEf4BzYjWHRdNNw4B=eOXOs_ONrDwrgX4bn=Nuc1g8JPFC34MA@mail.gmail.com/
[2] https://lore.kernel.org/bpf/20230531201936.1992188-1-alan.maguire@oracle.com/
[3] https://lore.kernel.org/dwarves/20251210202752.813919-1-alan.maguire@oracle.com/
[4] https://lore.kernel.org/bpf/20250528095743.791722-1-alan.maguire@oracle.com/
[5] https://lore.kernel.org/bpf/20231112124834.388735-1-alan.maguire@oracle.com/
[6] https://lore.kernel.org/bpf/20231110110304.63910-1-alan.maguire@oracle.com/
[7] https://lore.kernel.org/bpf/20230616171728.530116-1-alan.maguire@oracle.com/

Alan Maguire (10):
  btf: add kind layout encoding to UAPI
  libbpf: Support kind layout section handling in BTF
  libbpf: use kind layout to compute an unknown kind size
  libbpf: Add kind layout encoding support
  libbpf: BTF validation can use kind layout for unknown kinds
  btf: support kernel parsing of BTF with kind layout
  selftests/bpf: test kind encoding/decoding
  bpftool: add BTF dump "format meta" to dump header/metadata
  bpftool: Update doc to describe bpftool btf dump .. format metadata
  kbuild, bpf: Specify "kind_layout" optional feature

 include/uapi/linux/btf.h                      |  10 +
 kernel/bpf/btf.c                              |  96 ++++--
 scripts/Makefile.btf                          |   2 +
 .../bpf/bpftool/Documentation/bpftool-btf.rst |  28 +-
 tools/bpf/bpftool/bash-completion/bpftool     |   2 +-
 tools/bpf/bpftool/btf.c                       |  89 ++++-
 tools/include/uapi/linux/btf.h                |  10 +
 tools/lib/bpf/btf.c                           | 304 +++++++++++++-----
 tools/lib/bpf/btf.h                           |  20 ++
 tools/lib/bpf/libbpf.map                      |   1 +
 .../selftests/bpf/prog_tests/btf_kind.c       | 176 ++++++++++
 11 files changed, 639 insertions(+), 99 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/btf_kind.c

-- 
2.43.5

[PATCH v6 bpf-next 01/10] btf: add kind layout encoding to UAPI

[Alan Maguire]

BTF kind layouts provide information to parse BTF kinds. By separating
parsing BTF from using all the information it provides, we allow BTF
to encode new features even if they cannot be used by readers. This
will be helpful in particular for cases where older tools are used
to parse newer BTF with kinds the older tools do not recognize;
the BTF can still be parsed in such cases using kind layout.

The intent is to support encoding of kind layouts optionally so that
tools like pahole can add this information. For each kind, we record

- length of singular element following struct btf_type
- length of each of the btf_vlen() elements following

The ideas here were discussed at [1], [2]; hence

Suggested-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Alan Maguire <alan.maguire@oracle.com>

[1] https://lore.kernel.org/bpf/CAEf4BzYjWHRdNNw4B=eOXOs_ONrDwrgX4bn=Nuc1g8JPFC34MA@mail.gmail.com/
[2] https://lore.kernel.org/bpf/20230531201936.1992188-1-alan.maguire@oracle.com/
---
 include/uapi/linux/btf.h       | 10 ++++++++++
 tools/include/uapi/linux/btf.h | 10 ++++++++++
 2 files changed, 20 insertions(+)

diff --git a/include/uapi/linux/btf.h b/include/uapi/linux/btf.h
index 266d4ffa6c07..64dd681274f4 100644
--- a/include/uapi/linux/btf.h
+++ b/include/uapi/linux/btf.h
@@ -8,6 +8,14 @@
 #define BTF_MAGIC	0xeB9F
 #define BTF_VERSION	1
 
+/* kind layout section consists of a struct btf_kind_layout for each known
+ * kind at BTF encoding time.
+ */
+struct btf_kind_layout {
+	__u8 info_sz;		/* size of singular element after btf_type */
+	__u8 elem_sz;		/* size of each of btf_vlen(t) elements */
+};
+
 struct btf_header {
 	__u16	magic;
 	__u8	version;
@@ -19,6 +27,8 @@ struct btf_header {
 	__u32	type_len;	/* length of type section	*/
 	__u32	str_off;	/* offset of string section	*/
 	__u32	str_len;	/* length of string section	*/
+	__u32	kind_layout_off;/* offset of kind layout section */
+	__u32	kind_layout_len;/* length of kind layout section */
 };
 
 /* Max # of type identifier */
diff --git a/tools/include/uapi/linux/btf.h b/tools/include/uapi/linux/btf.h
index 266d4ffa6c07..64dd681274f4 100644
--- a/tools/include/uapi/linux/btf.h
+++ b/tools/include/uapi/linux/btf.h
@@ -8,6 +8,14 @@
 #define BTF_MAGIC	0xeB9F
 #define BTF_VERSION	1
 
+/* kind layout section consists of a struct btf_kind_layout for each known
+ * kind at BTF encoding time.
+ */
+struct btf_kind_layout {
+	__u8 info_sz;		/* size of singular element after btf_type */
+	__u8 elem_sz;		/* size of each of btf_vlen(t) elements */
+};
+
 struct btf_header {
 	__u16	magic;
 	__u8	version;
@@ -19,6 +27,8 @@ struct btf_header {
 	__u32	type_len;	/* length of type section	*/
 	__u32	str_off;	/* offset of string section	*/
 	__u32	str_len;	/* length of string section	*/
+	__u32	kind_layout_off;/* offset of kind layout section */
+	__u32	kind_layout_len;/* length of kind layout section */
 };
 
 /* Max # of type identifier */
-- 
2.43.5

[PATCH v6 bpf-next 02/10] libbpf: Support kind layout section handling in BTF

[Alan Maguire]

Support reading in kind layout fixing endian issues on reading;
also support writing kind layout section to raw BTF object.
There is not yet an API to populate the kind layout with meaningful
information.

As part of this, we need to consider multiple valid BTF header
sizes; the original or the kind layout-extended headers.
So to support this, the "struct btf" representation is modified
to always allocate a "struct btf_header" and copy the valid
portion from the raw data to it; this means we can always safely
check fields like btf->hdr->kind_layout_len.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 tools/lib/bpf/btf.c | 200 ++++++++++++++++++++++++++++++--------------
 1 file changed, 139 insertions(+), 61 deletions(-)

diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
index b136572e889a..737adc560818 100644
--- a/tools/lib/bpf/btf.c
+++ b/tools/lib/bpf/btf.c
@@ -40,42 +40,53 @@ struct btf {
 
 	/*
 	 * When BTF is loaded from an ELF or raw memory it is stored
-	 * in a contiguous memory block. The hdr, type_data, and, strs_data
+	 * in a contiguous memory block. The  type_data, and, strs_data
 	 * point inside that memory region to their respective parts of BTF
 	 * representation:
 	 *
-	 * +--------------------------------+
-	 * |  Header  |  Types  |  Strings  |
-	 * +--------------------------------+
-	 * ^          ^         ^
-	 * |          |         |
-	 * hdr        |         |
-	 * types_data-+         |
-	 * strs_data------------+
+	 * +--------------------------------+---------------------+
+	 * |  Header  |  Types  |  Strings  |Optional kind layout |
+	 * +--------------------------------+---------------------+
+	 * ^          ^         ^           ^
+	 * |          |         |           |
+	 * raw_data   |         |           |
+	 * types_data-+         |           |
+	 * strs_data------------+           |
+	 * kind_layout----------------------+
+	 *
+	 * A separate struct btf_header is allocated for btf->hdr,
+	 * and header information is copied into it.  This allows us
+	 * to handle header data for various header formats; the original,
+	 * the extended header with kind layout, etc.
 	 *
 	 * If BTF data is later modified, e.g., due to types added or
 	 * removed, BTF deduplication performed, etc, this contiguous
-	 * representation is broken up into three independently allocated
-	 * memory regions to be able to modify them independently.
+	 * representation is broken up into four independent memory
+	 * regions.
+	 *
 	 * raw_data is nulled out at that point, but can be later allocated
 	 * and cached again if user calls btf__raw_data(), at which point
-	 * raw_data will contain a contiguous copy of header, types, and
-	 * strings:
+	 * raw_data will contain a contiguous copy of header, types, strings
+	 * and optionally kind_layout.  kind_layout optionally points to a
+	 * kind_layout array - this allows us to encode information about
+	 * the kinds known at encoding time.  If kind_layout is NULL no
+	 * kind information is encoded.
 	 *
-	 * +----------+  +---------+  +-----------+
-	 * |  Header  |  |  Types  |  |  Strings  |
-	 * +----------+  +---------+  +-----------+
-	 * ^             ^            ^
-	 * |             |            |
-	 * hdr           |            |
-	 * types_data----+            |
-	 * strset__data(strs_set)-----+
+	 * +----------+  +---------+  +-----------+   +-----------+
+	 * |  Header  |  |  Types  |  |  Strings  |   |kind_layout|
+	 * +----------+  +---------+  +-----------+   +-----------+
+	 * ^             ^            ^               ^
+	 * |             |            |               |
+	 * hdr           |            |               |
+	 * types_data----+            |               |
+	 * strset__data(strs_set)-----+               |
+	 * kind_layout--------------------------------+
 	 *
-	 *               +----------+---------+-----------+
-	 *               |  Header  |  Types  |  Strings  |
-	 * raw_data----->+----------+---------+-----------+
+	 *               +----------+---------+-----------+---------------------+
+	 *               |  Header  |  Types  |  Strings  | Optional kind layout|
+	 * raw_data----->+----------+---------+-----------+---------------------+
 	 */
-	struct btf_header *hdr;
+	struct btf_header *hdr; /* separately-allocated header data */
 
 	void *types_data;
 	size_t types_data_cap; /* used size stored in hdr->type_len */
@@ -123,6 +134,13 @@ struct btf {
 	/* whether raw_data is a (read-only) mmap */
 	bool raw_data_is_mmap;
 
+	/* is BTF modifiable? i.e. is it split into separate sections as described above? */
+	bool modifiable;
+	/* Points either at raw kind layout data in parsed BTF (if present), or
+	 * at an allocated kind layout array when BTF is modifiable.
+	 */
+	void *kind_layout;
+
 	/* BTF object FD, if loaded into kernel */
 	int fd;
 
@@ -214,7 +232,7 @@ static int btf_add_type_idx_entry(struct btf *btf, __u32 type_off)
 	return 0;
 }
 
-static void btf_bswap_hdr(struct btf_header *h)
+static void btf_bswap_hdr(struct btf_header *h, __u32 hdr_len)
 {
 	h->magic = bswap_16(h->magic);
 	h->hdr_len = bswap_32(h->hdr_len);
@@ -222,50 +240,68 @@ static void btf_bswap_hdr(struct btf_header *h)
 	h->type_len = bswap_32(h->type_len);
 	h->str_off = bswap_32(h->str_off);
 	h->str_len = bswap_32(h->str_len);
+	/* May be operating on raw data with hdr_len that does not include below fields */
+	if (hdr_len >= sizeof(struct btf_header)) {
+		h->kind_layout_off = bswap_32(h->kind_layout_off);
+		h->kind_layout_len = bswap_32(h->kind_layout_len);
+	}
 }
 
 static int btf_parse_hdr(struct btf *btf)
 {
-	struct btf_header *hdr = btf->hdr;
+	struct btf_header *hdr = btf->raw_data;
+	__u32 hdr_len = hdr->hdr_len;
 	__u32 meta_left;
 
-	if (btf->raw_size < sizeof(struct btf_header)) {
+	if (btf->raw_size < offsetofend(struct btf_header, str_len)) {
 		pr_debug("BTF header not found\n");
 		return -EINVAL;
 	}
 
 	if (hdr->magic == bswap_16(BTF_MAGIC)) {
 		btf->swapped_endian = true;
-		if (bswap_32(hdr->hdr_len) != sizeof(struct btf_header)) {
+		hdr_len = bswap_32(hdr->hdr_len);
+		if (hdr_len < offsetofend(struct btf_header, str_len)) {
 			pr_warn("Can't load BTF with non-native endianness due to unsupported header length %u\n",
-				bswap_32(hdr->hdr_len));
+				hdr_len);
 			return -ENOTSUP;
 		}
-		btf_bswap_hdr(hdr);
 	} else if (hdr->magic != BTF_MAGIC) {
 		pr_debug("Invalid BTF magic: %x\n", hdr->magic);
 		return -EINVAL;
 	}
 
-	if (btf->raw_size < hdr->hdr_len) {
+	if (btf->raw_size < hdr_len) {
 		pr_debug("BTF header len %u larger than data size %u\n",
-			 hdr->hdr_len, btf->raw_size);
+			 hdr_len, btf->raw_size);
 		return -EINVAL;
 	}
 
-	meta_left = btf->raw_size - hdr->hdr_len;
-	if (meta_left < (long long)hdr->str_off + hdr->str_len) {
+	/* At this point, we have basic header information, so allocate btf->hdr */
+	btf->hdr = calloc(1, sizeof(struct btf_header));
+	if (!btf->hdr) {
+		pr_debug("BTF header allocation failed\n");
+		return -ENOMEM;
+	}
+	if (btf->swapped_endian)
+		btf_bswap_hdr(hdr, hdr_len);
+	memcpy(btf->hdr, hdr, hdr_len < sizeof(struct btf_header) ? hdr_len :
+								    sizeof(struct btf_header));
+
+	meta_left = btf->raw_size - hdr_len;
+	if (meta_left < (long long)btf->hdr->str_off + btf->hdr->str_len) {
 		pr_debug("Invalid BTF total size: %u\n", btf->raw_size);
 		return -EINVAL;
 	}
 
-	if ((long long)hdr->type_off + hdr->type_len > hdr->str_off) {
+	if ((long long)btf->hdr->type_off + btf->hdr->type_len > btf->hdr->str_off) {
 		pr_debug("Invalid BTF data sections layout: type data at %u + %u, strings data at %u + %u\n",
-			 hdr->type_off, hdr->type_len, hdr->str_off, hdr->str_len);
+			 btf->hdr->type_off, btf->hdr->type_len, btf->hdr->str_off,
+			 btf->hdr->str_len);
 		return -EINVAL;
 	}
 
-	if (hdr->type_off % 4) {
+	if (btf->hdr->type_off % 4) {
 		pr_debug("BTF type section is not aligned to 4 bytes\n");
 		return -EINVAL;
 	}
@@ -292,6 +328,22 @@ static int btf_parse_str_sec(struct btf *btf)
 	return 0;
 }
 
+static int btf_parse_kind_layout_sec(struct btf *btf)
+{
+	const struct btf_header *hdr = btf->hdr;
+
+	if (!hdr->kind_layout_off || !hdr->kind_layout_len)
+		return 0;
+
+	if (hdr->kind_layout_len % sizeof(struct btf_kind_layout) != 0) {
+		pr_debug("Invalid BTF kind layout section\n");
+		return -EINVAL;
+	}
+	btf->kind_layout = btf->raw_data + btf->hdr->hdr_len + btf->hdr->kind_layout_off;
+
+	return 0;
+}
+
 static int btf_type_size(const struct btf_type *t)
 {
 	const int base_size = sizeof(struct btf_type);
@@ -951,7 +1003,8 @@ __s32 btf__find_by_name_kind(const struct btf *btf, const char *type_name,
 
 static bool btf_is_modifiable(const struct btf *btf)
 {
-	return (void *)btf->hdr != btf->raw_data;
+	/* BTF is modifiable if split into multiple sections */
+	return btf->modifiable;
 }
 
 static void btf_free_raw_data(struct btf *btf)
@@ -980,10 +1033,11 @@ void btf__free(struct btf *btf)
 		 * might still have a cached contiguous raw data present,
 		 * which will be unconditionally freed below.
 		 */
-		free(btf->hdr);
 		free(btf->types_data);
 		strset__free(btf->strs_set);
+		free(btf->kind_layout);
 	}
+	free(btf->hdr);
 	btf_free_raw_data(btf);
 	free(btf->raw_data_swapped);
 	free(btf->type_offs);
@@ -994,6 +1048,7 @@ void btf__free(struct btf *btf)
 
 static struct btf *btf_new_empty(struct btf *base_btf)
 {
+	struct btf_header *hdr;
 	struct btf *btf;
 
 	btf = calloc(1, sizeof(*btf));
@@ -1022,14 +1077,20 @@ static struct btf *btf_new_empty(struct btf *base_btf)
 		return ERR_PTR(-ENOMEM);
 	}
 
-	btf->hdr = btf->raw_data;
-	btf->hdr->hdr_len = sizeof(struct btf_header);
-	btf->hdr->magic = BTF_MAGIC;
-	btf->hdr->version = BTF_VERSION;
+	hdr = btf->raw_data;
+	hdr->hdr_len = sizeof(struct btf_header);
+	hdr->magic = BTF_MAGIC;
+	hdr->version = BTF_VERSION;
 
-	btf->types_data = btf->raw_data + btf->hdr->hdr_len;
-	btf->strs_data = btf->raw_data + btf->hdr->hdr_len;
-	btf->hdr->str_len = base_btf ? 0 : 1; /* empty string at offset 0 */
+	btf->types_data = btf->raw_data + hdr->hdr_len;
+	btf->strs_data = btf->raw_data + hdr->hdr_len;
+	hdr->str_len = base_btf ? 0 : 1; /* empty string at offset 0 */
+	btf->hdr = calloc(1, sizeof(struct btf_header));
+	if (!btf->hdr) {
+		free(btf);
+		return ERR_PTR(-ENOMEM);
+	}
+	memcpy(btf->hdr, hdr, sizeof(*hdr));
 
 	return btf;
 }
@@ -1078,7 +1139,6 @@ static struct btf *btf_new(const void *data, __u32 size, struct btf *base_btf, b
 
 	btf->raw_size = size;
 
-	btf->hdr = btf->raw_data;
 	err = btf_parse_hdr(btf);
 	if (err)
 		goto done;
@@ -1087,6 +1147,7 @@ static struct btf *btf_new(const void *data, __u32 size, struct btf *base_btf, b
 	btf->types_data = btf->raw_data + btf->hdr->hdr_len + btf->hdr->type_off;
 
 	err = btf_parse_str_sec(btf);
+	err = err ?: btf_parse_kind_layout_sec(btf);
 	err = err ?: btf_parse_type_sec(btf);
 	err = err ?: btf_sanity_check(btf);
 	if (err)
@@ -1550,6 +1611,11 @@ static void *btf_get_raw_data(const struct btf *btf, __u32 *size, bool swap_endi
 	}
 
 	data_sz = hdr->hdr_len + hdr->type_len + hdr->str_len;
+	if (btf->kind_layout) {
+		data_sz = roundup(data_sz, 4);
+		data_sz += hdr->kind_layout_len;
+		hdr->kind_layout_off = roundup(hdr->type_len + hdr->str_len, 4);
+	}
 	data = calloc(1, data_sz);
 	if (!data)
 		return NULL;
@@ -1557,7 +1623,7 @@ static void *btf_get_raw_data(const struct btf *btf, __u32 *size, bool swap_endi
 
 	memcpy(p, hdr, hdr->hdr_len);
 	if (swap_endian)
-		btf_bswap_hdr(p);
+		btf_bswap_hdr(p, hdr->hdr_len);
 	p += hdr->hdr_len;
 
 	memcpy(p, btf->types_data, hdr->type_len);
@@ -1576,7 +1642,11 @@ static void *btf_get_raw_data(const struct btf *btf, __u32 *size, bool swap_endi
 	p += hdr->type_len;
 
 	memcpy(p, btf_strs_data(btf), hdr->str_len);
-	p += hdr->str_len;
+
+	if (btf->kind_layout) {
+		p = data + hdr->hdr_len + hdr->kind_layout_off;
+		memcpy(p, btf->kind_layout, hdr->kind_layout_len);
+	}
 
 	*size = data_sz;
 	return data;
@@ -1717,13 +1787,13 @@ static void btf_invalidate_raw_data(struct btf *btf)
 	}
 }
 
-/* Ensure BTF is ready to be modified (by splitting into a three memory
- * regions for header, types, and strings). Also invalidate cached
- * raw_data, if any.
+/* Ensure BTF is ready to be modified (by splitting into memory regions
+ * for types and strings, with kind layout section if needed (btf->hdr
+ * is already a separate region).  Also invalidate cached raw_data, if any.
  */
 static int btf_ensure_modifiable(struct btf *btf)
 {
-	void *hdr, *types;
+	void *types, *kind_layout = NULL;
 	struct strset *set = NULL;
 	int err = -ENOMEM;
 
@@ -1733,15 +1803,20 @@ static int btf_ensure_modifiable(struct btf *btf)
 		return 0;
 	}
 
-	/* split raw data into three memory regions */
-	hdr = malloc(btf->hdr->hdr_len);
+	/* split raw data into memory regions; btf->hdr is done already. */
 	types = malloc(btf->hdr->type_len);
-	if (!hdr || !types)
+	if (!types)
 		goto err_out;
-
-	memcpy(hdr, btf->hdr, btf->hdr->hdr_len);
 	memcpy(types, btf->types_data, btf->hdr->type_len);
 
+	if (btf->hdr->kind_layout_len && btf->hdr->kind_layout_off) {
+		kind_layout = malloc(btf->hdr->kind_layout_len);
+		if (!kind_layout)
+			goto err_out;
+		memcpy(kind_layout, btf->raw_data + btf->hdr->hdr_len + btf->hdr->kind_layout_off,
+		       btf->hdr->kind_layout_len);
+	}
+
 	/* build lookup index for all strings */
 	set = strset__new(BTF_MAX_STR_OFFSET, btf->strs_data, btf->hdr->str_len);
 	if (IS_ERR(set)) {
@@ -1750,11 +1825,12 @@ static int btf_ensure_modifiable(struct btf *btf)
 	}
 
 	/* only when everything was successful, update internal state */
-	btf->hdr = hdr;
 	btf->types_data = types;
 	btf->types_data_cap = btf->hdr->type_len;
 	btf->strs_data = NULL;
 	btf->strs_set = set;
+	if (kind_layout)
+		btf->kind_layout = kind_layout;
 	/* if BTF was created from scratch, all strings are guaranteed to be
 	 * unique and deduplicated
 	 */
@@ -1766,12 +1842,14 @@ static int btf_ensure_modifiable(struct btf *btf)
 	/* invalidate raw_data representation */
 	btf_invalidate_raw_data(btf);
 
+	btf->modifiable = true;
+
 	return 0;
 
 err_out:
 	strset__free(set);
-	free(hdr);
 	free(types);
+	free(kind_layout);
 	return err;
 }
 
-- 
2.43.5

[PATCH v6 bpf-next 03/10] libbpf: use kind layout to compute an unknown kind size

[Alan Maguire]

This allows BTF parsing to proceed even if we do not know the
kind.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 tools/lib/bpf/btf.c | 35 ++++++++++++++++++++++++++++-------
 1 file changed, 28 insertions(+), 7 deletions(-)

diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
index 737adc560818..4eb0704a0309 100644
--- a/tools/lib/bpf/btf.c
+++ b/tools/lib/bpf/btf.c
@@ -344,7 +344,29 @@ static int btf_parse_kind_layout_sec(struct btf *btf)
 	return 0;
 }
 
-static int btf_type_size(const struct btf_type *t)
+/* for unknown kinds, consult kind layout. */
+static int btf_type_size_unknown(const struct btf *btf, const struct btf_type *t)
+{
+	int size = sizeof(struct btf_type);
+	struct btf_kind_layout *k = NULL;
+	__u16 vlen = btf_vlen(t);
+	__u8 kind = btf_kind(t);
+
+	if (btf->kind_layout)
+		k = &((struct btf_kind_layout *)btf->kind_layout)[kind];
+
+	if (!k || (void *)k > ((void *)btf->kind_layout + btf->hdr->kind_layout_len)) {
+		pr_debug("Unsupported BTF_KIND: %u\n", btf_kind(t));
+		return -EINVAL;
+	}
+
+	size += k->info_sz;
+	size += vlen * k->elem_sz;
+
+	return size;
+}
+
+static int btf_type_size(const struct btf *btf, const struct btf_type *t)
 {
 	const int base_size = sizeof(struct btf_type);
 	__u16 vlen = btf_vlen(t);
@@ -380,8 +402,7 @@ static int btf_type_size(const struct btf_type *t)
 	case BTF_KIND_DECL_TAG:
 		return base_size + sizeof(struct btf_decl_tag);
 	default:
-		pr_debug("Unsupported BTF_KIND:%u\n", btf_kind(t));
-		return -EINVAL;
+		return btf_type_size_unknown(btf, t);
 	}
 }
 
@@ -480,7 +501,7 @@ static int btf_parse_type_sec(struct btf *btf)
 		if (btf->swapped_endian)
 			btf_bswap_type_base(next_type);
 
-		type_size = btf_type_size(next_type);
+		type_size = btf_type_size(btf, next_type);
 		if (type_size < 0)
 			return type_size;
 		if (next_type + type_size > end_type) {
@@ -1976,7 +1997,7 @@ static int btf_add_type(struct btf_pipe *p, const struct btf_type *src_type)
 	__u32 *str_off;
 	int sz, err;
 
-	sz = btf_type_size(src_type);
+	sz = btf_type_size(p->src, src_type);
 	if (sz < 0)
 		return libbpf_err(sz);
 
@@ -2058,7 +2079,7 @@ int btf__add_btf(struct btf *btf, const struct btf *src_btf)
 		struct btf_field_iter it;
 		__u32 *type_id, *str_off;
 
-		sz = btf_type_size(t);
+		sz = btf_type_size(src_btf, t);
 		if (sz < 0) {
 			/* unlikely, has to be corrupted src_btf */
 			err = sz;
@@ -5399,7 +5420,7 @@ static int btf_dedup_compact_types(struct btf_dedup *d)
 			continue;
 
 		t = btf__type_by_id(d->btf, id);
-		len = btf_type_size(t);
+		len = btf_type_size(d->btf, t);
 		if (len < 0)
 			return len;
 
-- 
2.43.5

[PATCH v6 bpf-next 04/10] libbpf: Add kind layout encoding support

[Alan Maguire]

Support encoding of BTF kind layout data via btf__new_empty_opts().

Current supported opts are base_btf and add_kind_layout.

Kind layout information is maintained in btf.c in the
kind_layouts[] array; when BTF is created with the
add_kind_layout option it represents the current view
of supported BTF kinds.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 tools/lib/bpf/btf.c      | 61 ++++++++++++++++++++++++++++++++++++++--
 tools/lib/bpf/btf.h      | 20 +++++++++++++
 tools/lib/bpf/libbpf.map |  1 +
 3 files changed, 79 insertions(+), 3 deletions(-)

diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
index 4eb0704a0309..2133e976cb9c 100644
--- a/tools/lib/bpf/btf.c
+++ b/tools/lib/bpf/btf.c
@@ -29,6 +29,35 @@
 
 static struct btf_type btf_void;
 
+/* Describe how kinds are laid out; some have a singular element following the "struct btf_type",
+ * some have BTF_INFO_VLEN(t->info) elements.  Specify sizes for both.  Flags are currently unused.
+ * Kind layout can be optionally added to the BTF representation in a dedicated section to
+ * facilitate parsing.  New kinds must be added here.
+ */
+struct btf_kind_layout kind_layouts[NR_BTF_KINDS] = {
+/*	singular element size		vlen element(s) size */
+{	0,				0				}, /* _UNKN */
+{	sizeof(__u32),			0				}, /* _INT */
+{	0,				0				}, /* _PTR */
+{	sizeof(struct btf_array),	0				}, /* _ARRAY */
+{	0,				sizeof(struct btf_member)	}, /* _STRUCT */
+{	0,				sizeof(struct btf_member)	}, /* _UNION */
+{	0,				sizeof(struct btf_enum)		}, /* _ENUM */
+{	0,				0				}, /* _FWD */
+{	0,				0				}, /* _TYPEDEF */
+{	0,				0				}, /* _VOLATILE */
+{	0,				0				}, /* _CONST */
+{	0,				0				}, /* _RESTRICT */
+{	0,				0				}, /* _FUNC */
+{	0,				sizeof(struct btf_param)	}, /* _FUNC_PROTO */
+{	sizeof(struct btf_var),		0				}, /* _VAR */
+{	0,				sizeof(struct btf_var_secinfo)	}, /* _DATASEC */
+{	0,				0				}, /* _FLOAT */
+{	sizeof(struct btf_decl_tag),	0				}, /* _DECL_TAG */
+{	0,				0				}, /* _TYPE_TAG */
+{	0,				sizeof(struct btf_enum64)	}, /* _ENUM64 */
+};
+
 struct btf {
 	/* raw BTF data in native endianness */
 	void *raw_data;
@@ -1067,8 +1096,9 @@ void btf__free(struct btf *btf)
 	free(btf);
 }
 
-static struct btf *btf_new_empty(struct btf *base_btf)
+static struct btf *btf_new_empty(struct btf_new_opts *opts)
 {
+	struct btf *base_btf = OPTS_GET(opts, base_btf, NULL);
 	struct btf_header *hdr;
 	struct btf *btf;
 
@@ -1111,6 +1141,17 @@ static struct btf *btf_new_empty(struct btf *base_btf)
 		free(btf);
 		return ERR_PTR(-ENOMEM);
 	}
+
+	if (opts->add_kind_layout) {
+		hdr->kind_layout_len = sizeof(kind_layouts);
+		btf->kind_layout = malloc(hdr->kind_layout_len);
+		if (!btf->kind_layout) {
+			free(btf->hdr);
+			free(btf);
+			return ERR_PTR(-ENOMEM);
+		}
+		memcpy(btf->kind_layout, kind_layouts, sizeof(kind_layouts));
+	}
 	memcpy(btf->hdr, hdr, sizeof(*hdr));
 
 	return btf;
@@ -1118,12 +1159,26 @@ static struct btf *btf_new_empty(struct btf *base_btf)
 
 struct btf *btf__new_empty(void)
 {
-	return libbpf_ptr(btf_new_empty(NULL));
+	LIBBPF_OPTS(btf_new_opts, opts);
+
+	return libbpf_ptr(btf_new_empty(&opts));
 }
 
 struct btf *btf__new_empty_split(struct btf *base_btf)
 {
-	return libbpf_ptr(btf_new_empty(base_btf));
+	LIBBPF_OPTS(btf_new_opts, opts);
+
+	opts.base_btf = base_btf;
+
+	return libbpf_ptr(btf_new_empty(&opts));
+}
+
+struct btf *btf__new_empty_opts(struct btf_new_opts *opts)
+{
+	if (!OPTS_VALID(opts, btf_new_opts))
+		return libbpf_err_ptr(-EINVAL);
+
+	return libbpf_ptr(btf_new_empty(opts));
 }
 
 static struct btf *btf_new(const void *data, __u32 size, struct btf *base_btf, bool is_mmap)
diff --git a/tools/lib/bpf/btf.h b/tools/lib/bpf/btf.h
index cc01494d6210..dcc166834937 100644
--- a/tools/lib/bpf/btf.h
+++ b/tools/lib/bpf/btf.h
@@ -109,6 +109,26 @@ LIBBPF_API struct btf *btf__new_empty(void);
  */
 LIBBPF_API struct btf *btf__new_empty_split(struct btf *base_btf);
 
+struct btf_new_opts {
+	size_t sz;
+	struct btf *base_btf;	/* optional base BTF */
+	bool add_kind_layout;	/* add BTF kind layout information */
+	size_t:0;
+};
+#define btf_new_opts__last_field add_kind_layout
+
+/**
+ * @brief **btf__new_empty_opts()** creates an unpopulated BTF object with
+ * optional *base_btf* and BTF kind layout description if *add_kind_layout*
+ * is set
+ * @return new BTF object instance which has to be eventually freed with
+ * **btf__free()**
+ *
+ * On error, NULL is returned and the thread-local `errno` variable is
+ * set to the error code.
+ */
+LIBBPF_API struct btf *btf__new_empty_opts(struct btf_new_opts *opts);
+
 /**
  * @brief **btf__distill_base()** creates new versions of the split BTF
  * *src_btf* and its base BTF. The new base BTF will only contain the types
diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map
index 84fb90a016c9..0fb9a1f70e72 100644
--- a/tools/lib/bpf/libbpf.map
+++ b/tools/lib/bpf/libbpf.map
@@ -453,4 +453,5 @@ LIBBPF_1.7.0 {
 		bpf_map__exclusive_program;
 		bpf_prog_assoc_struct_ops;
 		bpf_program__assoc_struct_ops;
+		btf__new_empty_opts;
 } LIBBPF_1.6.0;
-- 
2.43.5

[PATCH v6 bpf-next 05/10] libbpf: BTF validation can use kind layout for unknown kinds

[Alan Maguire]

BTF parsing can use kind layout to navigate unknown kinds, so
btf_validate_type() should take kind layout information into
account to avoid failure when an unrecognized kind is met.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 tools/lib/bpf/btf.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
index 2133e976cb9c..19e37a543e9a 100644
--- a/tools/lib/bpf/btf.c
+++ b/tools/lib/bpf/btf.c
@@ -691,8 +691,12 @@ static int btf_validate_type(const struct btf *btf, const struct btf_type *t, __
 		break;
 	}
 	default:
-		pr_warn("btf: type [%u]: unrecognized kind %u\n", id, kind);
-		return -EINVAL;
+		/* Kind may be represented in kind layout information. */
+		if (btf_type_size_unknown(btf, t) < 0) {
+			pr_warn("btf: type [%u]: unrecognized kind %u\n", id, kind);
+			return -EINVAL;
+		}
+		break;
 	}
 	return 0;
 }
-- 
2.43.5

[PATCH v6 bpf-next 06/10] btf: support kernel parsing of BTF with kind layout

[Alan Maguire]

Validate kind layout if present, but because the kernel must be
strict in what it accepts, reject BTF with unsupported kinds,
even if they are in the kind layout information.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 kernel/bpf/btf.c | 96 ++++++++++++++++++++++++++++++++++++++----------
 1 file changed, 76 insertions(+), 20 deletions(-)

diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index 0de8fc8a0e0b..eb4ac78b453a 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -268,6 +268,7 @@ struct btf {
 	struct btf_id_dtor_kfunc_tab *dtor_kfunc_tab;
 	struct btf_struct_metas *struct_meta_tab;
 	struct btf_struct_ops_tab *struct_ops_tab;
+	struct btf_kind_layout *kind_layout;
 
 	/* split BTF support */
 	struct btf *base_btf;
@@ -5215,23 +5216,36 @@ static s32 btf_check_meta(struct btf_verifier_env *env,
 		return -EINVAL;
 	}
 
-	if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX ||
-	    BTF_INFO_KIND(t->info) == BTF_KIND_UNKN) {
+	if (!btf_name_offset_valid(env->btf, t->name_off)) {
+		btf_verifier_log(env, "[%u] Invalid name_offset:%u",
+				 env->log_type_id, t->name_off);
+		return -EINVAL;
+	}
+
+	if (BTF_INFO_KIND(t->info) == BTF_KIND_UNKN) {
 		btf_verifier_log(env, "[%u] Invalid kind:%u",
 				 env->log_type_id, BTF_INFO_KIND(t->info));
 		return -EINVAL;
 	}
 
-	if (!btf_name_offset_valid(env->btf, t->name_off)) {
-		btf_verifier_log(env, "[%u] Invalid name_offset:%u",
-				 env->log_type_id, t->name_off);
+	if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX && env->btf->kind_layout &&
+	    (BTF_INFO_KIND(t->info) * sizeof(struct btf_kind_layout)) <
+	     env->btf->hdr.kind_layout_len) {
+		btf_verifier_log(env, "[%u] unknown but required kind %u",
+				 env->log_type_id,
+				 BTF_INFO_KIND(t->info));
 		return -EINVAL;
+	} else {
+		if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX) {
+			btf_verifier_log(env, "[%u] Invalid kind:%u",
+					 env->log_type_id, BTF_INFO_KIND(t->info));
+			return -EINVAL;
+		}
+		var_meta_size = btf_type_ops(t)->check_meta(env, t, meta_left);
+		if (var_meta_size < 0)
+			return var_meta_size;
 	}
 
-	var_meta_size = btf_type_ops(t)->check_meta(env, t, meta_left);
-	if (var_meta_size < 0)
-		return var_meta_size;
-
 	meta_left -= var_meta_size;
 
 	return saved_meta_left - meta_left;
@@ -5405,7 +5419,8 @@ static int btf_parse_str_sec(struct btf_verifier_env *env)
 	start = btf->nohdr_data + hdr->str_off;
 	end = start + hdr->str_len;
 
-	if (end != btf->data + btf->data_size) {
+	if (hdr->hdr_len < sizeof(struct btf_header) &&
+	    end != btf->data + btf->data_size) {
 		btf_verifier_log(env, "String section is not at the end");
 		return -EINVAL;
 	}
@@ -5426,9 +5441,41 @@ static int btf_parse_str_sec(struct btf_verifier_env *env)
 	return 0;
 }
 
+static int btf_parse_kind_layout_sec(struct btf_verifier_env *env)
+{
+	const struct btf_header *hdr = &env->btf->hdr;
+	struct btf *btf = env->btf;
+	void *start, *end;
+
+	if (hdr->hdr_len < sizeof(struct btf_header) ||
+	    hdr->kind_layout_len == 0)
+		return 0;
+
+	/* Kind layout section must align to 4 bytes */
+	if (hdr->kind_layout_off & (sizeof(u32) - 1)) {
+		btf_verifier_log(env, "Unaligned kind_layout_off");
+		return -EINVAL;
+	}
+	start = btf->nohdr_data + hdr->kind_layout_off;
+	end = start + hdr->kind_layout_len;
+
+	if (hdr->kind_layout_len < sizeof(struct btf_kind_layout)) {
+		btf_verifier_log(env, "Kind layout section is too small");
+		return -EINVAL;
+	}
+	if (end != btf->data + btf->data_size) {
+		btf_verifier_log(env, "Kind layout section is not at the end");
+		return -EINVAL;
+	}
+	btf->kind_layout = start;
+
+	return 0;
+}
+
 static const size_t btf_sec_info_offset[] = {
 	offsetof(struct btf_header, type_off),
 	offsetof(struct btf_header, str_off),
+	offsetof(struct btf_header, kind_layout_off),
 };
 
 static int btf_sec_info_cmp(const void *a, const void *b)
@@ -5443,44 +5490,49 @@ static int btf_check_sec_info(struct btf_verifier_env *env,
 			      u32 btf_data_size)
 {
 	struct btf_sec_info secs[ARRAY_SIZE(btf_sec_info_offset)];
-	u32 total, expected_total, i;
+	u32 nr_secs = ARRAY_SIZE(btf_sec_info_offset);
+	u32 total, expected_total, gap, i;
 	const struct btf_header *hdr;
 	const struct btf *btf;
 
 	btf = env->btf;
 	hdr = &btf->hdr;
 
+	if (hdr->hdr_len < sizeof(struct btf_header))
+		nr_secs--;
+
 	/* Populate the secs from hdr */
-	for (i = 0; i < ARRAY_SIZE(btf_sec_info_offset); i++)
+	for (i = 0; i < nr_secs; i++)
 		secs[i] = *(struct btf_sec_info *)((void *)hdr +
 						   btf_sec_info_offset[i]);
 
-	sort(secs, ARRAY_SIZE(btf_sec_info_offset),
+	sort(secs, nr_secs,
 	     sizeof(struct btf_sec_info), btf_sec_info_cmp, NULL);
 
 	/* Check for gaps and overlap among sections */
 	total = 0;
 	expected_total = btf_data_size - hdr->hdr_len;
-	for (i = 0; i < ARRAY_SIZE(btf_sec_info_offset); i++) {
+	for (i = 0; i < nr_secs; i++) {
 		if (expected_total < secs[i].off) {
 			btf_verifier_log(env, "Invalid section offset");
 			return -EINVAL;
 		}
-		if (total < secs[i].off) {
-			/* gap */
-			btf_verifier_log(env, "Unsupported section found");
-			return -EINVAL;
-		}
 		if (total > secs[i].off) {
 			btf_verifier_log(env, "Section overlap found");
 			return -EINVAL;
 		}
+		gap = secs[i].off - total;
+		if (gap >= 4) {
+			/* gap larger than alignment gap */
+			btf_verifier_log(env, "Unsupported section found");
+			return -EINVAL;
+		}
 		if (expected_total - total < secs[i].len) {
 			btf_verifier_log(env,
 					 "Total section length too long");
 			return -EINVAL;
 		}
-		total += secs[i].len;
+		total += secs[i].len + gap;
 	}
 
 	/* There is data other than hdr and known sections */
@@ -5816,6 +5868,10 @@ static struct btf *btf_parse(const union bpf_attr *attr, bpfptr_t uattr, u32 uat
 	if (err)
 		goto errout;
 
+	err = btf_parse_kind_layout_sec(env);
+	if (err)
+		goto errout;
+
 	err = btf_parse_type_sec(env);
 	if (err)
 		goto errout;
-- 
2.43.5

[PATCH v6 bpf-next 07/10] selftests/bpf: test kind encoding/decoding

[Alan Maguire]

verify btf__new_empty_opts() adds kind layouts for all kinds supported,
and after adding kind-related types for an unknown kind, ensure that
parsing uses this info when that kind is encountered rather than
giving up.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 .../selftests/bpf/prog_tests/btf_kind.c       | 176 ++++++++++++++++++
 1 file changed, 176 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/btf_kind.c

diff --git a/tools/testing/selftests/bpf/prog_tests/btf_kind.c b/tools/testing/selftests/bpf/prog_tests/btf_kind.c
new file mode 100644
index 000000000000..735ea0b588a5
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/btf_kind.c
@@ -0,0 +1,176 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2025, Oracle and/or its affiliates. */
+
+#include <test_progs.h>
+#include <bpf/btf.h>
+#include <bpf/libbpf.h>
+
+/* verify kind encoding exists for each kind */
+static void test_btf_kind_encoding(void)
+{
+	LIBBPF_OPTS(btf_new_opts, opts);
+	const struct btf_header *hdr;
+	const void *raw_btf;
+	struct btf *btf;
+	__u32 raw_size;
+
+	opts.add_kind_layout = true;
+	btf = btf__new_empty_opts(&opts);
+	if (!ASSERT_OK_PTR(btf, "btf_new"))
+		return;
+
+	raw_btf = btf__raw_data(btf, &raw_size);
+	if (!ASSERT_OK_PTR(raw_btf, "btf__raw_data"))
+		return;
+
+	hdr = raw_btf;
+
+	ASSERT_EQ(hdr->kind_layout_off % 4, 0, "kind_layout aligned");
+	ASSERT_EQ(hdr->kind_layout_len, sizeof(struct btf_kind_layout) * NR_BTF_KINDS,
+		  "kind_layout_len");
+	btf__free(btf);
+}
+
+static void write_raw_btf(const char *btf_path, void *raw_btf, size_t raw_size)
+{
+	int fd = open(btf_path, O_WRONLY | O_CREAT);
+
+	write(fd, raw_btf, raw_size);
+	close(fd);
+}
+
+/* fabricate an unrecognized kind at BTF_KIND_MAX + 1, and after adding
+ * the appropriate struct/typedefs to the BTF such that it recognizes
+ * this kind, ensure that parsing of BTF containing the unrecognized kind
+ * can succeed.
+ */
+void test_btf_kind_decoding(void)
+{
+	__s32 int_id, unrec_id, id, id2;
+	LIBBPF_OPTS(btf_new_opts, opts);
+	struct btf *btf, *new_btf;
+	struct btf_kind_layout *k;
+	struct btf_header *hdr;
+	const void *raw_btf;
+	struct btf_type *t;
+	char btf_path[64];
+	void *new_raw_btf;
+	__u32 raw_size;
+
+	opts.add_kind_layout = true;
+	btf = btf__new_empty_opts(&opts);
+	if (!ASSERT_OK_PTR(btf, "btf_new"))
+		return;
+
+	int_id = btf__add_int(btf, "test_char", 1, BTF_INT_CHAR);
+	if (!ASSERT_GT(int_id, 0, "add_int_id"))
+		return;
+
+	/* now create our type with unrecognized kind by adding a typedef kind
+	 * we will overwrite it with our unrecognized kind value.
+	 */
+	unrec_id = btf__add_typedef(btf, "unrec_kind", int_id);
+	if (!ASSERT_GT(unrec_id, 0, "add_unrec_id"))
+		return;
+
+	/* add an id after it that we will look up to verify we can parse
+	 * beyond unrecognized kinds.
+	 */
+	id = btf__add_typedef(btf, "test_lookup", int_id);
+	if (!ASSERT_GT(id, 0, "add_test_lookup_id"))
+		return;
+	id2 = btf__add_typedef(btf, "test_lookup2", int_id);
+	if (!ASSERT_GT(id2, 0, "add_test_lookup_id2"))
+		return;
+
+	raw_btf = (void *)btf__raw_data(btf, &raw_size);
+	if (!ASSERT_OK_PTR(raw_btf, "btf__raw_data"))
+		return;
+
+	new_raw_btf = calloc(1, raw_size + sizeof(*k));
+	memcpy(new_raw_btf, raw_btf, raw_size);
+
+	/* add new layout description */
+	hdr = new_raw_btf;
+	hdr->kind_layout_len += sizeof(*k);
+	k = new_raw_btf + hdr->hdr_len + hdr->kind_layout_off;
+	k[NR_BTF_KINDS].info_sz = 0;
+	k[NR_BTF_KINDS].elem_sz = 0;
+
+	/* now modify our typedef added above to be an unrecognized kind. */
+	t = (void *)hdr + hdr->hdr_len + hdr->type_off + sizeof(struct btf_type) +
+		sizeof(__u32);
+	t->info = (NR_BTF_KINDS << 24);
+
+	/* now write our BTF to a raw file, ready for parsing. */
+	snprintf(btf_path, sizeof(btf_path), "/tmp/btf_kind.%d", getpid());
+
+	write_raw_btf(btf_path, new_raw_btf, raw_size + sizeof(*k));
+
+	/* verify parsing succeeds, and that we can read type info past
+	 * the unrecognized kind.
+	 */
+	new_btf = btf__parse_raw(btf_path);
+	if (ASSERT_OK_PTR(new_btf, "btf__parse_raw")) {
+		ASSERT_EQ(btf__find_by_name_kind(new_btf, "test_lookup",
+						 BTF_KIND_TYPEDEF), id,
+			  "verify_id_lookup");
+		ASSERT_EQ(btf__find_by_name_kind(new_btf, "test_lookup2",
+						 BTF_KIND_TYPEDEF), id2,
+			  "verify_id2_lookup");
+	}
+	btf__free(new_btf);
+
+	/* next, change info_sz to equal sizeof(struct btf_type); this means the
+	 * "test_lookup" kind will be reinterpreted as a singular info element
+	 * following the unrecognized kind.
+	 */
+	k[NR_BTF_KINDS].info_sz = sizeof(struct btf_type);
+	write_raw_btf(btf_path, new_raw_btf, raw_size + sizeof(*k));
+
+	new_btf = btf__parse_raw(btf_path);
+	if (ASSERT_OK_PTR(new_btf, "btf__parse_raw")) {
+		ASSERT_EQ(btf__find_by_name_kind(new_btf, "test_lookup",
+						 BTF_KIND_TYPEDEF), -ENOENT,
+			  "verify_id_not_found");
+		/* id of "test_lookup2" will be id2 -1 as we have removed one type */
+		ASSERT_EQ(btf__find_by_name_kind(new_btf, "test_lookup2",
+						 BTF_KIND_TYPEDEF), id2 - 1,
+			  "verify_id_lookup2");
+
+	}
+	btf__free(new_btf);
+
+	/* next, change elem_sz to equal sizeof(struct btf_type)/2 and set
+	 * vlen associated with unrecognized type to 2; this allows us to verify
+	 * vlen-specified BTF can still be parsed.
+	 */
+	k[NR_BTF_KINDS].info_sz = 0;
+	k[NR_BTF_KINDS].elem_sz = sizeof(struct btf_type)/2;
+	t->info |= 2;
+	write_raw_btf(btf_path, new_raw_btf, raw_size + sizeof(*k));
+
+	new_btf = btf__parse_raw(btf_path);
+	if (ASSERT_OK_PTR(new_btf, "btf__parse_raw")) {
+		ASSERT_EQ(btf__find_by_name_kind(new_btf, "test_lookup",
+						 BTF_KIND_TYPEDEF), -ENOENT,
+			  "verify_id_not_found");
+		/* id of "test_lookup2" will be id2 -1 as we have removed one type */
+		ASSERT_EQ(btf__find_by_name_kind(new_btf, "test_lookup2",
+						 BTF_KIND_TYPEDEF), id2 - 1,
+			  "verify_id_lookup2");
+
+	}
+	btf__free(new_btf);
+	free(new_raw_btf);
+	unlink(btf_path);
+	btf__free(btf);
+}
+
+void test_btf_kind(void)
+{
+	if (test__start_subtest("btf_kind_encoding"))
+		test_btf_kind_encoding();
+	if (test__start_subtest("btf_kind_decoding"))
+		test_btf_kind_decoding();
+}
-- 
2.43.5

[PATCH v6 bpf-next 08/10] bpftool: add BTF dump "format meta" to dump header/metadata

[Alan Maguire]

Provide a way to dump BTF metadata info via bpftool; this
consists of BTF size, header fields and kind layout info
(if available); for example

$ bpftool btf dump file vmlinux format meta
size 5460436
magic 0xeb9f
version 1
flags 0x0
hdr_len 32
type_len 3194640
type_off 0
str_len 2265721
str_off 3194640
kind_layout_len 40
kind_layout_off 5460364
kind 0    UNKNOWN    info_sz 0    elem_sz 0
kind 1    INT        info_sz 4    elem_sz 0
kind 2    PTR        info_sz 0    elem_sz 0
kind 3    ARRAY      info_sz 12   elem_sz 0
kind 4    STRUCT     info_sz 0    elem_sz 12
kind 5    UNION      info_sz 0    elem_sz 12
kind 6    ENUM       info_sz 0    elem_sz 8
kind 7    FWD        info_sz 0    elem_sz 0
kind 8    TYPEDEF    info_sz 0    elem_sz 0
kind 9    VOLATILE   info_sz 0    elem_sz 0
kind 10   CONST      info_sz 0    elem_sz 0
kind 11   RESTRICT   info_sz 0    elem_sz 0
kind 12   FUNC       info_sz 0    elem_sz 0
kind 13   FUNC_PROTO info_sz 0    elem_sz 8
kind 14   VAR        info_sz 4    elem_sz 0
kind 15   DATASEC    info_sz 0    elem_sz 12
kind 16   FLOAT      info_sz 0    elem_sz 0
kind 17   DECL_TAG   info_sz 4    elem_sz 0
kind 18   TYPE_TAG   info_sz 0    elem_sz 0
kind 19   ENUM64     info_sz 0    elem_sz 12

JSON output is also supported:

$ $ bpftool -p btf dump file /var/tmp/vmlinux.kind_layout format meta
{
    "size": 5460436,
    "header": {
        "magic": 60319,
        "version": 1,
        "flags": 0,
        "hdr_len": 32,
        "type_len": 3194640,
        "type_off": 0,
        "str_len": 2265721,
        "str_off": 3194640,
        "kind_layout_len": 40,
        "kind_layout_offset": 5460364
    },
    "kind_layouts": [{
            "kind": 0,
            "name": "UNKNOWN",
            "info_sz": 0,
            "elem_sz": 0
        },{
            "kind": 1,
            "name": "INT",
            "info_sz": 4,
            "elem_sz": 0
        },{
...

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 tools/bpf/bpftool/bash-completion/bpftool |  2 +-
 tools/bpf/bpftool/btf.c                   | 89 ++++++++++++++++++++++-
 2 files changed, 87 insertions(+), 4 deletions(-)

diff --git a/tools/bpf/bpftool/bash-completion/bpftool b/tools/bpf/bpftool/bash-completion/bpftool
index 53bcfeb1a76e..a331172cf8de 100644
--- a/tools/bpf/bpftool/bash-completion/bpftool
+++ b/tools/bpf/bpftool/bash-completion/bpftool
@@ -950,7 +950,7 @@ _bpftool()
                             return 0
                             ;;
                         format)
-                            COMPREPLY=( $( compgen -W "c raw" -- "$cur" ) )
+                            COMPREPLY=( $( compgen -W "c raw meta" -- "$cur" ) )
                             ;;
                         root_id)
                             return 0;
diff --git a/tools/bpf/bpftool/btf.c b/tools/bpf/bpftool/btf.c
index 946612029dee..028448c0c7a0 100644
--- a/tools/bpf/bpftool/btf.c
+++ b/tools/bpf/bpftool/btf.c
@@ -835,6 +835,85 @@ static int dump_btf_c(const struct btf *btf,
 	return err;
 }
 
+static int dump_btf_meta(const struct btf *btf)
+{
+	const struct btf_header *hdr;
+	const struct btf_kind_layout *k;
+	__u8 i, nr_kinds = 0;
+	const void *data;
+	__u32 data_sz;
+
+	data = btf__raw_data(btf, &data_sz);
+	if (!data)
+		return -ENOMEM;
+	hdr = data;
+	if (json_output) {
+		jsonw_start_object(json_wtr);		/* metadata object */
+		jsonw_uint_field(json_wtr, "size", data_sz);
+		jsonw_name(json_wtr, "header");
+		jsonw_start_object(json_wtr);		/* header object */
+		jsonw_uint_field(json_wtr, "magic", hdr->magic);
+		jsonw_uint_field(json_wtr, "version", hdr->version);
+		jsonw_uint_field(json_wtr, "flags", hdr->flags);
+		jsonw_uint_field(json_wtr, "hdr_len", hdr->hdr_len);
+		jsonw_uint_field(json_wtr, "type_len", hdr->type_len);
+		jsonw_uint_field(json_wtr, "type_off", hdr->type_off);
+		jsonw_uint_field(json_wtr, "str_len", hdr->str_len);
+		jsonw_uint_field(json_wtr, "str_off", hdr->str_off);
+	} else {
+		printf("size %-10u\n", data_sz);
+		printf("magic 0x%-10x\nversion %-10d\nflags 0x%-10x\nhdr_len %-10u\n",
+		       hdr->magic, hdr->version, hdr->flags, hdr->hdr_len);
+		printf("type_len %-10u\ntype_off %-10u\n", hdr->type_len, hdr->type_off);
+		printf("str_len %-10u\nstr_off %-10u\n", hdr->str_len, hdr->str_off);
+	}
+
+	if (hdr->hdr_len < sizeof(struct btf_header)) {
+		if (json_output) {
+			jsonw_end_object(json_wtr);	/* end header object */
+			jsonw_end_object(json_wtr);	/* end metadata object */
+		}
+		return 0;
+	}
+	if (hdr->kind_layout_len > 0 && hdr->kind_layout_off > 0) {
+		k = (void *)hdr + hdr->hdr_len + hdr->kind_layout_off;
+		nr_kinds = hdr->kind_layout_len / sizeof(*k);
+	}
+	if (json_output) {
+		jsonw_uint_field(json_wtr, "kind_layout_len", hdr->kind_layout_len);
+		jsonw_uint_field(json_wtr, "kind_layout_offset", hdr->kind_layout_off);
+		jsonw_end_object(json_wtr);		/* end header object */
+
+		if (nr_kinds > 0) {
+			jsonw_name(json_wtr, "kind_layouts");
+			jsonw_start_array(json_wtr);
+			for (i = 0; i < nr_kinds; i++) {
+				jsonw_start_object(json_wtr);
+				jsonw_uint_field(json_wtr, "kind", i);
+				if (i < NR_BTF_KINDS)
+					jsonw_string_field(json_wtr, "name", btf_kind_str[i]);
+				else
+					jsonw_null_field(json_wtr, "name");
+				jsonw_uint_field(json_wtr, "info_sz", k[i].info_sz);
+				jsonw_uint_field(json_wtr, "elem_sz", k[i].elem_sz);
+				jsonw_end_object(json_wtr);
+			}
+			jsonw_end_array(json_wtr);
+		}
+		jsonw_end_object(json_wtr);		/* end metadata object */
+	} else {
+		printf("kind_layout_len %-10u\nkind_layout_off %-10u\n",
+		       hdr->kind_layout_len, hdr->kind_layout_off);
+		for (i = 0; i < nr_kinds; i++) {
+			printf("kind %-4d %-10s info_sz %-4d elem_sz %-4d\n",
+			       i, i < NR_BTF_KINDS ? btf_kind_str[i] : "?",
+			       k[i].info_sz, k[i].elem_sz);
+		}
+	}
+
+	return 0;
+}
+
 static const char sysfs_vmlinux[] = "/sys/kernel/btf/vmlinux";
 
 static struct btf *get_vmlinux_btf_from_sysfs(void)
@@ -880,7 +959,7 @@ static bool btf_is_kernel_module(__u32 btf_id)
 
 static int do_dump(int argc, char **argv)
 {
-	bool dump_c = false, sort_dump_c = true;
+	bool dump_c = false, sort_dump_c = true, dump_meta = false;
 	struct btf *btf = NULL, *base = NULL;
 	__u32 root_type_ids[MAX_ROOT_IDS];
 	bool have_id_filtering;
@@ -990,10 +1069,12 @@ static int do_dump(int argc, char **argv)
 			}
 			if (strcmp(*argv, "c") == 0) {
 				dump_c = true;
+			} else if (is_prefix(*argv, "meta")) {
+				dump_meta = true;
 			} else if (strcmp(*argv, "raw") == 0) {
 				dump_c = false;
 			} else {
-				p_err("unrecognized format specifier: '%s', possible values: raw, c",
+				p_err("unrecognized format specifier: '%s', possible values: raw, c, meta",
 				      *argv);
 				err = -EINVAL;
 				goto done;
@@ -1072,6 +1153,8 @@ static int do_dump(int argc, char **argv)
 			goto done;
 		}
 		err = dump_btf_c(btf, root_type_ids, root_type_cnt, sort_dump_c);
+	} else if (dump_meta) {
+		err = dump_btf_meta(btf);
 	} else {
 		err = dump_btf_raw(btf, root_type_ids, root_type_cnt);
 	}
@@ -1446,7 +1529,7 @@ static int do_help(int argc, char **argv)
 		"       %1$s %2$s help\n"
 		"\n"
 		"       BTF_SRC := { id BTF_ID | prog PROG | map MAP [{key | value | kv | all}] | file FILE }\n"
-		"       FORMAT  := { raw | c [unsorted] }\n"
+		"       FORMAT  := { raw | c [unsorted] | meta }\n"
 		"       " HELP_SPEC_MAP "\n"
 		"       " HELP_SPEC_PROGRAM "\n"
 		"       " HELP_SPEC_OPTIONS " |\n"
-- 
2.43.5

[PATCH v6 bpf-next 09/10] bpftool: Update doc to describe bpftool btf dump .. format metadata

[Alan Maguire]

...and provide an example of output.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 .../bpf/bpftool/Documentation/bpftool-btf.rst | 28 +++++++++++++++++--
 1 file changed, 26 insertions(+), 2 deletions(-)

diff --git a/tools/bpf/bpftool/Documentation/bpftool-btf.rst b/tools/bpf/bpftool/Documentation/bpftool-btf.rst
index d47dddc2b4ee..1c11b5647ab7 100644
--- a/tools/bpf/bpftool/Documentation/bpftool-btf.rst
+++ b/tools/bpf/bpftool/Documentation/bpftool-btf.rst
@@ -28,7 +28,7 @@ BTF COMMANDS
 | **bpftool** **btf help**
 |
 | *BTF_SRC* := { **id** *BTF_ID* | **prog** *PROG* | **map** *MAP* [{**key** | **value** | **kv** | **all**}] | **file** *FILE* }
-| *FORMAT* := { **raw** | **c** [**unsorted**] }
+| *FORMAT* := { **raw** | **c** [**unsorted**] | **meta** }
 | *MAP* := { **id** *MAP_ID* | **pinned** *FILE* }
 | *PROG* := { **id** *PROG_ID* | **pinned** *FILE* | **tag** *PROG_TAG* | **name** *PROG_NAME* }
 
@@ -65,7 +65,8 @@ bpftool btf dump *BTF_SRC* [format *FORMAT*] [root_id *ROOT_ID*]
     **format** option can be used to override default (raw) output format. Raw
     (**raw**) or C-syntax (**c**) output formats are supported. With C-style
     formatting, the output is sorted by default. Use the **unsorted** option
-    to avoid sorting the output.
+    to avoid sorting the output.  BTF metadata can be displayed with the
+    **meta** option.
 
     **root_id** option can be used to filter a dump to a single type and all
     its dependent types. It cannot be used with any other types of filtering
@@ -267,3 +268,26 @@ All the standard ways to specify map or program are supported:
   [104859] FUNC 'smbalert_work' type_id=9695 linkage=static
   [104860] FUNC 'smbus_alert' type_id=71367 linkage=static
   [104861] FUNC 'smbus_do_alert' type_id=84827 linkage=static
+
+Display BTF metadata from file vmlinux
+
+**# bpftool btf dump file vmlinux format meta**
+
+::
+
+  size 5161076
+  magic 0xeb9f
+  version 1
+  flags 0x1
+  hdr_len 40
+  type_len 3036368
+  type_off 0
+  str_len 2124588
+  str_off 3036368
+  kind_layout_len 80
+  kind_layout_off 5160956
+  kind 0    UNKNOWN    info_sz 0    elem_sz 0
+  kind 1    INT        info_sz 0    elem_sz 0
+  kind 2    PTR        info_sz 0    elem_sz 0
+  kind 3    ARRAY      info_sz 0    elem_sz 0
+  kind 4    STRUCT     info_sz 0    elem_sz 0
-- 
2.43.5

[PATCH v6 bpf-next 10/10] kbuild, bpf: Specify "kind_layout" optional feature

[Alan Maguire]

The "kind_layout" feature will add metadata about BTF kinds to the
generated BTF; its absence in pahole will not trigger an error so it
is safe to add unconditionally as it will simply be ignored if pahole
does not support it.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 scripts/Makefile.btf | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/scripts/Makefile.btf b/scripts/Makefile.btf
index db76335dd917..c20f9bbcabeb 100644
--- a/scripts/Makefile.btf
+++ b/scripts/Makefile.btf
@@ -25,6 +25,8 @@ pahole-flags-$(call test-ge, $(pahole-ver), 126)  = -j$(JOBS) --btf_features=enc
 
 pahole-flags-$(call test-ge, $(pahole-ver), 130) += --btf_features=attributes
 
+pahole-flags-$(call test-ge, $(pahole-ver), 131) += --btf_features=kind_layout
+
 ifneq ($(KBUILD_EXTMOD),)
 module-pahole-flags-$(call test-ge, $(pahole-ver), 128) += --btf_features=distilled_base
 endif
-- 
2.43.5

Re: [PATCH v6 bpf-next 01/10] btf: add kind layout encoding to UAPI

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 2541 bytes --]

> diff --git a/include/uapi/linux/btf.h b/include/uapi/linux/btf.h
> index 266d4ffa6c07..64dd681274f4 100644
> --- a/include/uapi/linux/btf.h
> +++ b/include/uapi/linux/btf.h
> @@ -8,6 +8,14 @@
>  #define BTF_MAGIC	0xeB9F
>  #define BTF_VERSION	1
>
> +/* kind layout section consists of a struct btf_kind_layout for each known
> + * kind at BTF encoding time.
> + */
> +struct btf_kind_layout {
> +	__u8 info_sz;		/* size of singular element after btf_type */
> +	__u8 elem_sz;		/* size of each of btf_vlen(t) elements */
> +};
> +
>  struct btf_header {
>  	__u16	magic;
>  	__u8	version;
> @@ -19,6 +27,8 @@ struct btf_header {
>  	__u32	type_len;	/* length of type section	*/
>  	__u32	str_off;	/* offset of string section	*/
>  	__u32	str_len;	/* length of string section	*/
> +	__u32	kind_layout_off;/* offset of kind layout section */
> +	__u32	kind_layout_len;/* length of kind layout section */
                             ^^^^

Adding these fields to struct btf_header expands it from 24 to 32 bytes.
This breaks compatibility in two ways without implementation support:

1. In kernel/bpf/btf.c, the btf_sec_info_offset[] array at
   btf_check_sec_info() only includes type_off and str_off, missing the
   new kind_layout_off. When btf_check_sec_info() validates sections, it
   checks that expected_total == total. If a BTF file contains a
   kind_layout section, the validation will fail with "Unsupported
   section found" and reject the BTF with -EINVAL.

   Can the kernel accept BTF files with kind_layout sections without
   updating btf_sec_info_offset[] to include the new field?

   Note: This is fixed in commit e82981d01e84 which adds kind_layout_off
   to the validation array.

2. In tools/lib/bpf/btf.c at btf_sanity_check(), when handling
   non-native endianness, there's an exact equality check:

     if (bswap_32(hdr->hdr_len) != sizeof(struct btf_header))

   With sizeof(struct btf_header) now 32 bytes, this check will reject
   old BTF files with 24-byte headers. Does this break loading of
   existing BTF files that have non-native endianness?

   Note: This is fixed in commit a018196d8c9e which changes the check to
   a minimum size comparison instead of exact equality.

> diff --git a/tools/include/uapi/linux/btf.h b/tools/include/uapi/linux/btf.h

[ ... ]


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/20112692486

Re: [PATCH v6 bpf-next 04/10] libbpf: Add kind layout encoding support

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 2328 bytes --]

> diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> index 4eb0704a0309..2133e976cb9c 100644
> --- a/tools/lib/bpf/btf.c
> +++ b/tools/lib/bpf/btf.c

[ ... ]

> -static struct btf *btf_new_empty(struct btf *base_btf)
> +static struct btf *btf_new_empty(struct btf_new_opts *opts)
>  {
> +	struct btf *base_btf = OPTS_GET(opts, base_btf, NULL);
>  	struct btf_header *hdr;
>  	struct btf *btf;
>
> @@ -1111,6 +1141,17 @@ static struct btf *btf_new_empty(struct btf *base_btf)
>  		free(btf);
>  		return ERR_PTR(-ENOMEM);
>  	}
> +
> +	if (opts->add_kind_layout) {

Can opts be NULL here? The OPTS_VALID() check in btf__new_empty_opts()
returns true when opts is NULL (!(opts) evaluates to true), so this code
could dereference NULL. Should this use OPTS_GET(opts, add_kind_layout,
false) instead?

> +		hdr->kind_layout_len = sizeof(kind_layouts);
> +		btf->kind_layout = malloc(hdr->kind_layout_len);
> +		if (!btf->kind_layout) {
> +			free(btf->hdr);
> +			free(btf);
> +			return ERR_PTR(-ENOMEM);
> +		}

Does this error path leak btf->raw_data? It was allocated earlier via
calloc() but isn't freed here, while btf->hdr is freed. The same issue
appears in the error path above at the check "if (!btf->hdr)" where
btf->raw_data is also not freed.

> +		memcpy(btf->kind_layout, kind_layouts, sizeof(kind_layouts));
> +	}
>  	memcpy(btf->hdr, hdr, sizeof(*hdr));
>
>  	return btf;

Also, when a BTF object is created via btf_new_empty() with
add_kind_layout=true, btf->kind_layout is allocated but btf->modifiable
remains false (never set to true). Later, when btf__free() is called, it
only frees btf->kind_layout if btf_is_modifiable() returns true (checking
the modifiable flag). Does this leak the kind_layout allocation for BTF
objects created with btf__new_empty_opts()?

> @@ -1118,12 +1159,26 @@ static struct btf *btf_new_empty(struct btf *base_btf)

[ ... ]

> +struct btf *btf__new_empty_opts(struct btf_new_opts *opts)
> +{
> +	if (!OPTS_VALID(opts, btf_new_opts))
> +		return libbpf_err_ptr(-EINVAL);
> +
> +	return libbpf_ptr(btf_new_empty(opts));
> +}

[ ... ]


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/20112692486

Re: [PATCH v6 bpf-next 03/10] libbpf: use kind layout to compute an unknown kind size

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 1943 bytes --]

> diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> index 737adc560818..4eb0704a0309 100644
> --- a/tools/lib/bpf/btf.c
> +++ b/tools/lib/bpf/btf.c

[ ... ]

> +/* for unknown kinds, consult kind layout. */
> +static int btf_type_size_unknown(const struct btf *btf, const struct btf_type *t)
> +{
> +	int size = sizeof(struct btf_type);
> +	struct btf_kind_layout *k = NULL;
> +	__u16 vlen = btf_vlen(t);
> +	__u8 kind = btf_kind(t);
> +
> +	if (btf->kind_layout)
> +		k = &((struct btf_kind_layout *)btf->kind_layout)[kind];
> +
> +	if (!k || (void *)k > ((void *)btf->kind_layout + btf->hdr->kind_layout_len)) {
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Can the bounds check be moved before the array access?  Currently the code
indexes into kind_layout[kind] before validating whether 'kind' is within
bounds.  While computing an out-of-bounds pointer may not cause immediate
issues, the bounds check itself appears incorrect.

The check uses '>' which allows access when k points exactly to the end of
the array. For example, if kind_layout_len is 40 bytes (20 kinds * 2 bytes
each) and kind is 20, then k would point to offset 40, and the check
'(ptr at 40) > (ptr at 40)' evaluates to false, allowing the subsequent
read of k->info_sz at an out-of-bounds location.

Should this be: 'if (!btf->kind_layout || kind >= (btf->hdr->kind_layout_len
/ sizeof(*k)))'?  This would validate the index before the array access and
ensure the full structure can be read safely.

> +		pr_debug("Unsupported BTF_KIND: %u\n", btf_kind(t));
> +		return -EINVAL;
> +	}
> +
> +	size += k->info_sz;
> +	size += vlen * k->elem_sz;
> +
> +	return size;
> +}

[ ... ]


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/20112692486

Re: [PATCH v6 bpf-next 08/10] bpftool: add BTF dump "format meta" to dump header/metadata

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 4541 bytes --]

> diff --git a/tools/bpf/bpftool/bash-completion/bpftool b/tools/bpf/bpftool/bash-completion/bpftool
> index 53bcfeb1a76e..a331172cf8de 100644
> --- a/tools/bpf/bpftool/bash-completion/bpftool
> +++ b/tools/bpf/bpftool/bash-completion/bpftool

[ ... ]

> diff --git a/tools/bpf/bpftool/btf.c b/tools/bpf/bpftool/btf.c
> index 946612029dee..028448c0c7a0 100644
> --- a/tools/bpf/bpftool/btf.c
> +++ b/tools/bpf/bpftool/btf.c
> @@ -835,6 +835,85 @@ static int dump_btf_c(const struct btf *btf,
>  	return err;
>  }
>
> +static int dump_btf_meta(const struct btf *btf)
> +{
> +	const struct btf_header *hdr;
> +	const struct btf_kind_layout *k;
> +	__u8 i, nr_kinds = 0;
> +	const void *data;
> +	__u32 data_sz;
> +
> +	data = btf__raw_data(btf, &data_sz);
> +	if (!data)
> +		return -ENOMEM;
> +	hdr = data;
> +	if (json_output) {
> +		jsonw_start_object(json_wtr);		/* metadata object */
> +		jsonw_uint_field(json_wtr, "size", data_sz);
> +		jsonw_name(json_wtr, "header");
> +		jsonw_start_object(json_wtr);		/* header object */
> +		jsonw_uint_field(json_wtr, "magic", hdr->magic);
> +		jsonw_uint_field(json_wtr, "version", hdr->version);
> +		jsonw_uint_field(json_wtr, "flags", hdr->flags);
> +		jsonw_uint_field(json_wtr, "hdr_len", hdr->hdr_len);
> +		jsonw_uint_field(json_wtr, "type_len", hdr->type_len);
> +		jsonw_uint_field(json_wtr, "type_off", hdr->type_off);
> +		jsonw_uint_field(json_wtr, "str_len", hdr->str_len);
> +		jsonw_uint_field(json_wtr, "str_off", hdr->str_off);
> +	} else {
> +		printf("size %-10u\n", data_sz);
> +		printf("magic 0x%-10x\nversion %-10d\nflags 0x%-10x\nhdr_len %-10u\n",
> +		       hdr->magic, hdr->version, hdr->flags, hdr->hdr_len);
> +		printf("type_len %-10u\ntype_off %-10u\n", hdr->type_len, hdr->type_off);
> +		printf("str_len %-10u\nstr_off %-10u\n", hdr->str_len, hdr->str_off);
> +	}
> +
> +	if (hdr->hdr_len < sizeof(struct btf_header)) {
> +		if (json_output) {
> +			jsonw_end_object(json_wtr);	/* end header object */
> +			jsonw_end_object(json_wtr);	/* end metadata object */
> +		}
> +		return 0;
> +	}
> +	if (hdr->kind_layout_len > 0 && hdr->kind_layout_off > 0) {
> +		k = (void *)hdr + hdr->hdr_len + hdr->kind_layout_off;
                    ^^^^

Can this read out of bounds? The code checks that kind_layout_len and
kind_layout_off are positive, but doesn't validate that hdr->hdr_len +
hdr->kind_layout_off + hdr->kind_layout_len stays within data_sz. If a
malformed BTF file has kind_layout_off pointing beyond the buffer, the
subsequent k[i] accesses could read past the end of the buffer.

This same calculation appears in libbpf's btf_parse_kind_layout_sec()
without bounds checking, suggesting the validation gap exists at the
parsing layer.

> +		nr_kinds = hdr->kind_layout_len / sizeof(*k);
> +	}
> +	if (json_output) {
> +		jsonw_uint_field(json_wtr, "kind_layout_len", hdr->kind_layout_len);
> +		jsonw_uint_field(json_wtr, "kind_layout_offset", hdr->kind_layout_off);
> +		jsonw_end_object(json_wtr);		/* end header object */
> +
> +		if (nr_kinds > 0) {
> +			jsonw_name(json_wtr, "kind_layouts");
> +			jsonw_start_array(json_wtr);
> +			for (i = 0; i < nr_kinds; i++) {
> +				jsonw_start_object(json_wtr);
> +				jsonw_uint_field(json_wtr, "kind", i);
> +				if (i < NR_BTF_KINDS)
> +					jsonw_string_field(json_wtr, "name", btf_kind_str[i]);
> +				else
> +					jsonw_null_field(json_wtr, "name");
> +				jsonw_uint_field(json_wtr, "info_sz", k[i].info_sz);
> +				jsonw_uint_field(json_wtr, "elem_sz", k[i].elem_sz);
                                                                         ^^^^

The k[i] accesses here will read from the out-of-bounds pointer if
kind_layout_off was invalid.

> +				jsonw_end_object(json_wtr);
> +			}
> +			jsonw_end_array(json_wtr);
> +		}
> +		jsonw_end_object(json_wtr);		/* end metadata object */
> +	} else {
> +		printf("kind_layout_len %-10u\nkind_layout_off %-10u\n",
> +		       hdr->kind_layout_len, hdr->kind_layout_off);
> +		for (i = 0; i < nr_kinds; i++) {
> +			printf("kind %-4d %-10s info_sz %-4d elem_sz %-4d\n",
> +			       i, i < NR_BTF_KINDS ? btf_kind_str[i] : "?",
> +			       k[i].info_sz, k[i].elem_sz);
                               ^^^^

Same out-of-bounds read in the plain text output path.

> +		}
> +	}
> +
> +	return 0;
> +}
> +

[ ... ]


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/20112692486

Re: [PATCH v6 bpf-next 05/10] libbpf: BTF validation can use kind layout for unknown kinds

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 2081 bytes --]

> diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> index 2133e976cb9c..19e37a543e9a 100644
> --- a/tools/lib/bpf/btf.c
> +++ b/tools/lib/bpf/btf.c
> @@ -691,8 +691,12 @@ static int btf_validate_type(const struct btf *btf, const struct btf_type *t, __
>  		break;
>  	}
>  	default:
> -		pr_warn("btf: type [%u]: unrecognized kind %u\n", id, kind);
> -		return -EINVAL;
> +		/* Kind may be represented in kind layout information. */
> +		if (btf_type_size_unknown(btf, t) < 0) {
> +			pr_warn("btf: type [%u]: unrecognized kind %u\n", id, kind);
> +			return -EINVAL;
> +		}
> +		break;
>  	}
>  	return 0;
>  }

The btf_type_size_unknown() function is called to validate unknown kinds
by checking the kind_layout array. Can the bounds check in
btf_type_size_unknown() read past the end of the kind_layout array?

In btf_type_size_unknown():

    if (btf->kind_layout)
        k = &((struct btf_kind_layout *)btf->kind_layout)[kind];

    if (!k || (void *)k > ((void *)btf->kind_layout + btf->hdr->kind_layout_len)) {
        pr_debug("Unsupported BTF_KIND: %u\n", btf_kind(t));
        return -EINVAL;
    }

    size += k->info_sz;
    size += vlen * k->elem_sz;

The check uses '>' to test if k exceeds the array bounds. If kind equals
the number of elements in the array (kind == kind_layout_len /
sizeof(struct btf_kind_layout)), then k points exactly at the array
boundary. The check evaluates k > end as false (since k == end), so the
validation passes and the code proceeds to dereference k, reading
out-of-bounds memory.

For comparison, btf__type_by_id() uses '>=' for its bounds check:

    if (type_id >= btf->start_id + btf->nr_types)
        return errno = EINVAL, NULL;

Should the bounds check in btf_type_size_unknown() use '>=' instead of
'>' to prevent reading one element past the end of the kind_layout array?


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/20112692486

Re: [PATCH v6 bpf-next 07/10] selftests/bpf: test kind encoding/decoding

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 2364 bytes --]

> diff --git a/tools/testing/selftests/bpf/prog_tests/btf_kind.c b/tools/testing/selftests/bpf/prog_tests/btf_kind.c
> new file mode 100644
> index 000000000000..735ea0b588a5
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/prog_tests/btf_kind.c
> @@ -0,0 +1,176 @@

[ ... ]

> +static void write_raw_btf(const char *btf_path, void *raw_btf, size_t raw_size)
> +{
> +	int fd = open(btf_path, O_WRONLY | O_CREAT);
                                                  ^^

Should open() include a mode parameter here? POSIX requires a mode
argument when O_CREAT is specified. Other tests in this directory use
0644, for example d_path.c and fs_kfuncs.c both use:

    open(path, O_CREAT | O_RDONLY, 0644)

> +
> +	write(fd, raw_btf, raw_size);
> +	close(fd);
> +}
> +

[ ... ]

> +void test_btf_kind_decoding(void)
> +{
> +	__s32 int_id, unrec_id, id, id2;
> +	LIBBPF_OPTS(btf_new_opts, opts);
> +	struct btf *btf, *new_btf;
> +	struct btf_kind_layout *k;
> +	struct btf_header *hdr;
> +	const void *raw_btf;
> +	struct btf_type *t;
> +	char btf_path[64];
> +	void *new_raw_btf;
> +	__u32 raw_size;

[ ... ]

> +	raw_btf = (void *)btf__raw_data(btf, &raw_size);
> +	if (!ASSERT_OK_PTR(raw_btf, "btf__raw_data"))
> +		return;
> +
> +	new_raw_btf = calloc(1, raw_size + sizeof(*k));
> +	memcpy(new_raw_btf, raw_btf, raw_size);
         ^^^^^^^

Can memcpy() dereference NULL here if calloc() fails? Other BPF tests
check calloc() before use. For example, btf.c uses:

    mapv = calloc(num_cpus, rounded_value_size);
    if (CHECK(!mapv, "mapv allocation failure")) {

And htab_update.c uses:

    value = calloc(1, value_size);
    if (!ASSERT_OK_PTR(value, "calloc value"))

> +
> +	/* add new layout description */
> +	hdr = new_raw_btf;
> +	hdr->kind_layout_len += sizeof(*k);
> +	k = new_raw_btf + hdr->hdr_len + hdr->kind_layout_off;
> +	k[NR_BTF_KINDS].info_sz = 0;
> +	k[NR_BTF_KINDS].elem_sz = 0;
> +
> +	/* now modify our typedef added above to be an unrecognized kind. */
> +	t = (void *)hdr + hdr->hdr_len + hdr->type_off + sizeof(struct btf_type) +
> +		sizeof(__u32);
> +	t->info = (NR_BTF_KINDS << 24);

[ ... ]


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/20112692486

Re: [PATCH v6 bpf-next 02/10] libbpf: Support kind layout section handling in BTF

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 1793 bytes --]

> diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> index b136572e889a..737adc560818 100644
> --- a/tools/lib/bpf/btf.c
> +++ b/tools/lib/bpf/btf.c

[ ... ]

> @@ -994,6 +1048,7 @@ void btf__free(struct btf *btf)
>
>  static struct btf *btf_new_empty(struct btf *base_btf)
>  {
> +	struct btf_header *hdr;
>  	struct btf *btf;
>
>  	btf = calloc(1, sizeof(*btf));
> @@ -1022,14 +1077,20 @@ static struct btf *btf_new_empty(struct btf *base_btf)
>  		return ERR_PTR(-ENOMEM);
>  	}
>
> -	btf->hdr = btf->raw_data;
> -	btf->hdr->hdr_len = sizeof(struct btf_header);
> -	btf->hdr->magic = BTF_MAGIC;
> -	btf->hdr->version = BTF_VERSION;
> +	hdr = btf->raw_data;
> +	hdr->hdr_len = sizeof(struct btf_header);
> +	hdr->magic = BTF_MAGIC;
> +	hdr->version = BTF_VERSION;
>
> -	btf->types_data = btf->raw_data + btf->hdr->hdr_len;
> -	btf->strs_data = btf->raw_data + btf->hdr->hdr_len;
> -	btf->hdr->str_len = base_btf ? 0 : 1; /* empty string at offset 0 */
> +	btf->types_data = btf->raw_data + hdr->hdr_len;
> +	btf->strs_data = btf->raw_data + hdr->hdr_len;
> +	hdr->str_len = base_btf ? 0 : 1; /* empty string at offset 0 */
> +	btf->hdr = calloc(1, sizeof(struct btf_header));
> +	if (!btf->hdr) {
> +		free(btf);
> +		return ERR_PTR(-ENOMEM);
> +	}
> +	memcpy(btf->hdr, hdr, sizeof(*hdr));
>
>  	return btf;
>  }

Does this leak btf->raw_data when the btf->hdr allocation fails? The
btf->raw_data was allocated earlier (at line 1074 in the new code), but
if we reach the error path here, we only free(btf) without freeing
btf->raw_data first.


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/20112692486

Re: [PATCH v6 bpf-next 02/10] libbpf: Support kind layout section handling in BTF

[Alan Maguire]

On 10/12/2025 20:55, bot+bpf-ci@kernel.org wrote:
>> diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
>> index b136572e889a..737adc560818 100644
>> --- a/tools/lib/bpf/btf.c
>> +++ b/tools/lib/bpf/btf.c
> 
> [ ... ]
> 
>> @@ -994,6 +1048,7 @@ void btf__free(struct btf *btf)
>>
>>  static struct btf *btf_new_empty(struct btf *base_btf)
>>  {
>> +	struct btf_header *hdr;
>>  	struct btf *btf;
>>
>>  	btf = calloc(1, sizeof(*btf));
>> @@ -1022,14 +1077,20 @@ static struct btf *btf_new_empty(struct btf *base_btf)
>>  		return ERR_PTR(-ENOMEM);
>>  	}
>>
>> -	btf->hdr = btf->raw_data;
>> -	btf->hdr->hdr_len = sizeof(struct btf_header);
>> -	btf->hdr->magic = BTF_MAGIC;
>> -	btf->hdr->version = BTF_VERSION;
>> +	hdr = btf->raw_data;
>> +	hdr->hdr_len = sizeof(struct btf_header);
>> +	hdr->magic = BTF_MAGIC;
>> +	hdr->version = BTF_VERSION;
>>
>> -	btf->types_data = btf->raw_data + btf->hdr->hdr_len;
>> -	btf->strs_data = btf->raw_data + btf->hdr->hdr_len;
>> -	btf->hdr->str_len = base_btf ? 0 : 1; /* empty string at offset 0 */
>> +	btf->types_data = btf->raw_data + hdr->hdr_len;
>> +	btf->strs_data = btf->raw_data + hdr->hdr_len;
>> +	hdr->str_len = base_btf ? 0 : 1; /* empty string at offset 0 */
>> +	btf->hdr = calloc(1, sizeof(struct btf_header));
>> +	if (!btf->hdr) {
>> +		free(btf);
>> +		return ERR_PTR(-ENOMEM);
>> +	}
>> +	memcpy(btf->hdr, hdr, sizeof(*hdr));
>>
>>  	return btf;
>>  }
> 
> Does this leak btf->raw_data when the btf->hdr allocation fails? The
> btf->raw_data was allocated earlier (at line 1074 in the new code), but
> if we reach the error path here, we only free(btf) without freeing
> btf->raw_data first.
>

Good catch; will fix. Additionally, elsewhere bot raises the issue of validation of
kind_layout_[len|off] in btf_parse_hdr(); we should add additional validation there
to ensure kind_layout_off is 4-byte aligned, points after the str section and does not
overrun the overall size. Will fix these in next round.

> 
> ---
> AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
> See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
> 
> CI run summary: https://github.com/kernel-patches/bpf/actions/runs/20112692486

Re: [PATCH v6 bpf-next 03/10] libbpf: use kind layout to compute an unknown kind size

[Alan Maguire]

On 10/12/2025 20:55, bot+bpf-ci@kernel.org wrote:
>> diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
>> index 737adc560818..4eb0704a0309 100644
>> --- a/tools/lib/bpf/btf.c
>> +++ b/tools/lib/bpf/btf.c
> 
> [ ... ]
> 
>> +/* for unknown kinds, consult kind layout. */
>> +static int btf_type_size_unknown(const struct btf *btf, const struct btf_type *t)
>> +{
>> +	int size = sizeof(struct btf_type);
>> +	struct btf_kind_layout *k = NULL;
>> +	__u16 vlen = btf_vlen(t);
>> +	__u8 kind = btf_kind(t);
>> +
>> +	if (btf->kind_layout)
>> +		k = &((struct btf_kind_layout *)btf->kind_layout)[kind];
>> +
>> +	if (!k || (void *)k > ((void *)btf->kind_layout + btf->hdr->kind_layout_len)) {
>                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> 
> Can the bounds check be moved before the array access?  Currently the code
> indexes into kind_layout[kind] before validating whether 'kind' is within
> bounds.  While computing an out-of-bounds pointer may not cause immediate
> issues, the bounds check itself appears incorrect.
> 
> The check uses '>' which allows access when k points exactly to the end of
> the array. For example, if kind_layout_len is 40 bytes (20 kinds * 2 bytes
> each) and kind is 20, then k would point to offset 40, and the check
> '(ptr at 40) > (ptr at 40)' evaluates to false, allowing the subsequent
> read of k->info_sz at an out-of-bounds location.
> 
> Should this be: 'if (!btf->kind_layout || kind >= (btf->hdr->kind_layout_len
> / sizeof(*k)))'?  This would validate the index before the array access and
> ensure the full structure can be read safely.
> 

Yep, good suggestion. Will fix.

>> +		pr_debug("Unsupported BTF_KIND: %u\n", btf_kind(t));
>> +		return -EINVAL;
>> +	}
>> +
>> +	size += k->info_sz;
>> +	size += vlen * k->elem_sz;
>> +
>> +	return size;
>> +}
> 
> [ ... ]
> 
> 
> ---
> AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
> See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
> 
> CI run summary: https://github.com/kernel-patches/bpf/actions/runs/20112692486

Re: [PATCH v6 bpf-next 04/10] libbpf: Add kind layout encoding support

[Alan Maguire]

On 10/12/2025 20:55, bot+bpf-ci@kernel.org wrote:
>> diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
>> index 4eb0704a0309..2133e976cb9c 100644
>> --- a/tools/lib/bpf/btf.c
>> +++ b/tools/lib/bpf/btf.c
> 
> [ ... ]
> 
>> -static struct btf *btf_new_empty(struct btf *base_btf)
>> +static struct btf *btf_new_empty(struct btf_new_opts *opts)
>>  {
>> +	struct btf *base_btf = OPTS_GET(opts, base_btf, NULL);
>>  	struct btf_header *hdr;
>>  	struct btf *btf;
>>
>> @@ -1111,6 +1141,17 @@ static struct btf *btf_new_empty(struct btf *base_btf)
>>  		free(btf);
>>  		return ERR_PTR(-ENOMEM);
>>  	}
>> +
>> +	if (opts->add_kind_layout) {
> 
> Can opts be NULL here? The OPTS_VALID() check in btf__new_empty_opts()
> returns true when opts is NULL (!(opts) evaluates to true), so this code
> could dereference NULL. Should this use OPTS_GET(opts, add_kind_layout,
> false) instead?

Will do.

> 
>> +		hdr->kind_layout_len = sizeof(kind_layouts);
>> +		btf->kind_layout = malloc(hdr->kind_layout_len);
>> +		if (!btf->kind_layout) {
>> +			free(btf->hdr);
>> +			free(btf);
>> +			return ERR_PTR(-ENOMEM);
>> +		}
> 
> Does this error path leak btf->raw_data? It was allocated earlier via
> calloc() but isn't freed here, while btf->hdr is freed. The same issue
> appears in the error path above at the check "if (!btf->hdr)" where
> btf->raw_data is also not freed.
>

Will fix.
 
>> +		memcpy(btf->kind_layout, kind_layouts, sizeof(kind_layouts));
>> +	}
>>  	memcpy(btf->hdr, hdr, sizeof(*hdr));
>>
>>  	return btf;
> 
> Also, when a BTF object is created via btf_new_empty() with
> add_kind_layout=true, btf->kind_layout is allocated but btf->modifiable
> remains false (never set to true). Later, when btf__free() is called, it
> only frees btf->kind_layout if btf_is_modifiable() returns true (checking
> the modifiable flag). Does this leak the kind_layout allocation for BTF
> objects created with btf__new_empty_opts()?
> 

We can free btf->kind_layout conditionally as long as it is not pointing into
the btf data section (as would be the case for parsed BTF). Will add the check
in btf__free().

>> @@ -1118,12 +1159,26 @@ static struct btf *btf_new_empty(struct btf *base_btf)
> 
> [ ... ]
> 
>> +struct btf *btf__new_empty_opts(struct btf_new_opts *opts)
>> +{
>> +	if (!OPTS_VALID(opts, btf_new_opts))
>> +		return libbpf_err_ptr(-EINVAL);
>> +
>> +	return libbpf_ptr(btf_new_empty(opts));
>> +}
> 
> [ ... ]
> 
> 
> ---
> AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
> See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
> 
> CI run summary: https://github.com/kernel-patches/bpf/actions/runs/20112692486

Re: [PATCH v6 bpf-next 04/10] libbpf: Add kind layout encoding support

[Alan Maguire]

On 11/12/2025 08:36, Alan Maguire wrote:
> On 10/12/2025 20:55, bot+bpf-ci@kernel.org wrote:
>>> diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
>>> index 4eb0704a0309..2133e976cb9c 100644
>>> --- a/tools/lib/bpf/btf.c
>>> +++ b/tools/lib/bpf/btf.c
>>
>> [ ... ]
>>
>>> -static struct btf *btf_new_empty(struct btf *base_btf)
>>> +static struct btf *btf_new_empty(struct btf_new_opts *opts)
>>>  {
>>> +	struct btf *base_btf = OPTS_GET(opts, base_btf, NULL);
>>>  	struct btf_header *hdr;
>>>  	struct btf *btf;
>>>
>>> @@ -1111,6 +1141,17 @@ static struct btf *btf_new_empty(struct btf *base_btf)
>>>  		free(btf);
>>>  		return ERR_PTR(-ENOMEM);
>>>  	}
>>> +
>>> +	if (opts->add_kind_layout) {
>>
>> Can opts be NULL here? The OPTS_VALID() check in btf__new_empty_opts()
>> returns true when opts is NULL (!(opts) evaluates to true), so this code
>> could dereference NULL. Should this use OPTS_GET(opts, add_kind_layout,
>> false) instead?
> 
> Will do.
> 
>>
>>> +		hdr->kind_layout_len = sizeof(kind_layouts);
>>> +		btf->kind_layout = malloc(hdr->kind_layout_len);
>>> +		if (!btf->kind_layout) {
>>> +			free(btf->hdr);
>>> +			free(btf);
>>> +			return ERR_PTR(-ENOMEM);
>>> +		}
>>
>> Does this error path leak btf->raw_data? It was allocated earlier via
>> calloc() but isn't freed here, while btf->hdr is freed. The same issue
>> appears in the error path above at the check "if (!btf->hdr)" where
>> btf->raw_data is also not freed.
>>
> 
> Will fix.
>  
>>> +		memcpy(btf->kind_layout, kind_layouts, sizeof(kind_layouts));
>>> +	}
>>>  	memcpy(btf->hdr, hdr, sizeof(*hdr));
>>>
>>>  	return btf;
>>
>> Also, when a BTF object is created via btf_new_empty() with
>> add_kind_layout=true, btf->kind_layout is allocated but btf->modifiable
>> remains false (never set to true). Later, when btf__free() is called, it
>> only frees btf->kind_layout if btf_is_modifiable() returns true (checking
>> the modifiable flag). Does this leak the kind_layout allocation for BTF
>> objects created with btf__new_empty_opts()?
>>
> 
> We can free btf->kind_layout conditionally as long as it is not pointing into
> the btf data section (as would be the case for parsed BTF). Will add the check
> in btf__free().
>

Actually the right answer here is for btf_new_empty() to place the kind layout
data in the raw contiguous representation when add_kind_layout is true; it then
becomes btf_ensure_modifiable()s job to break out and allocate sections as is true today
for type/string sections. With this approach, we do not need to allocate btf->kind_layout
in btf_new_empty() and the leak will not happen as far as I can see since allocation
will only happen when btf->modifiable is true.

>>> @@ -1118,12 +1159,26 @@ static struct btf *btf_new_empty(struct btf *base_btf)
>>
>> [ ... ]
>>
>>> +struct btf *btf__new_empty_opts(struct btf_new_opts *opts)
>>> +{
>>> +	if (!OPTS_VALID(opts, btf_new_opts))
>>> +		return libbpf_err_ptr(-EINVAL);
>>> +
>>> +	return libbpf_ptr(btf_new_empty(opts));
>>> +}
>>
>> [ ... ]
>>
>>
>> ---
>> AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
>> See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
>>
>> CI run summary: https://github.com/kernel-patches/bpf/actions/runs/20112692486
> 
> 

Re: [PATCH v6 bpf-next 01/10] btf: add kind layout encoding to UAPI

[Mykyta Yatsenko]

On 12/10/25 20:32, Alan Maguire wrote:
> BTF kind layouts provide information to parse BTF kinds. By separating
> parsing BTF from using all the information it provides, we allow BTF
> to encode new features even if they cannot be used by readers. This
> will be helpful in particular for cases where older tools are used
> to parse newer BTF with kinds the older tools do not recognize;
> the BTF can still be parsed in such cases using kind layout.
>
> The intent is to support encoding of kind layouts optionally so that
> tools like pahole can add this information. For each kind, we record
>
> - length of singular element following struct btf_type
> - length of each of the btf_vlen() elements following
>
> The ideas here were discussed at [1], [2]; hence
>
> Suggested-by: Andrii Nakryiko <andrii@kernel.org>
> Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
>
> [1] https://lore.kernel.org/bpf/CAEf4BzYjWHRdNNw4B=eOXOs_ONrDwrgX4bn=Nuc1g8JPFC34MA@mail.gmail.com/
> [2] https://lore.kernel.org/bpf/20230531201936.1992188-1-alan.maguire@oracle.com/
> ---
>   include/uapi/linux/btf.h       | 10 ++++++++++
>   tools/include/uapi/linux/btf.h | 10 ++++++++++
>   2 files changed, 20 insertions(+)
>
> diff --git a/include/uapi/linux/btf.h b/include/uapi/linux/btf.h
> index 266d4ffa6c07..64dd681274f4 100644
> --- a/include/uapi/linux/btf.h
> +++ b/include/uapi/linux/btf.h
> @@ -8,6 +8,14 @@
>   #define BTF_MAGIC	0xeB9F
>   #define BTF_VERSION	1
>   
> +/* kind layout section consists of a struct btf_kind_layout for each known
> + * kind at BTF encoding time.
> + */
We switched to kernel style comments in the new code:
/*
  * comment
  */

instead of
/* comment
  */
> +struct btf_kind_layout {
> +	__u8 info_sz;		/* size of singular element after btf_type */
> +	__u8 elem_sz;		/* size of each of btf_vlen(t) elements */
> +};
> +
>   struct btf_header {
>   	__u16	magic;
>   	__u8	version;
> @@ -19,6 +27,8 @@ struct btf_header {
>   	__u32	type_len;	/* length of type section	*/
>   	__u32	str_off;	/* offset of string section	*/
>   	__u32	str_len;	/* length of string section	*/
> +	__u32	kind_layout_off;/* offset of kind layout section */
> +	__u32	kind_layout_len;/* length of kind layout section */
>   };
>   
>   /* Max # of type identifier */
> diff --git a/tools/include/uapi/linux/btf.h b/tools/include/uapi/linux/btf.h
> index 266d4ffa6c07..64dd681274f4 100644
> --- a/tools/include/uapi/linux/btf.h
> +++ b/tools/include/uapi/linux/btf.h
> @@ -8,6 +8,14 @@
>   #define BTF_MAGIC	0xeB9F
>   #define BTF_VERSION	1
>   
> +/* kind layout section consists of a struct btf_kind_layout for each known
> + * kind at BTF encoding time.
> + */
> +struct btf_kind_layout {
> +	__u8 info_sz;		/* size of singular element after btf_type */
> +	__u8 elem_sz;		/* size of each of btf_vlen(t) elements */
> +};
> +
>   struct btf_header {
>   	__u16	magic;
>   	__u8	version;
> @@ -19,6 +27,8 @@ struct btf_header {
>   	__u32	type_len;	/* length of type section	*/
>   	__u32	str_off;	/* offset of string section	*/
>   	__u32	str_len;	/* length of string section	*/
> +	__u32	kind_layout_off;/* offset of kind layout section */
> +	__u32	kind_layout_len;/* length of kind layout section */
>   };
>   
>   /* Max # of type identifier */

Re: [PATCH v6 bpf-next 02/10] libbpf: Support kind layout section handling in BTF

[Mykyta Yatsenko]

On 12/10/25 20:32, Alan Maguire wrote:
> Support reading in kind layout fixing endian issues on reading;
> also support writing kind layout section to raw BTF object.
> There is not yet an API to populate the kind layout with meaningful
> information.
>
> As part of this, we need to consider multiple valid BTF header
> sizes; the original or the kind layout-extended headers.
> So to support this, the "struct btf" representation is modified
> to always allocate a "struct btf_header" and copy the valid
> portion from the raw data to it; this means we can always safely
> check fields like btf->hdr->kind_layout_len.
>
> Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
> ---
>   tools/lib/bpf/btf.c | 200 ++++++++++++++++++++++++++++++--------------
>   1 file changed, 139 insertions(+), 61 deletions(-)
>
> diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> index b136572e889a..737adc560818 100644
> --- a/tools/lib/bpf/btf.c
> +++ b/tools/lib/bpf/btf.c
> @@ -40,42 +40,53 @@ struct btf {
>   
>   	/*
>   	 * When BTF is loaded from an ELF or raw memory it is stored
> -	 * in a contiguous memory block. The hdr, type_data, and, strs_data
> +	 * in a contiguous memory block. The  type_data, and, strs_data
>   	 * point inside that memory region to their respective parts of BTF
>   	 * representation:
>   	 *
> -	 * +--------------------------------+
> -	 * |  Header  |  Types  |  Strings  |
> -	 * +--------------------------------+
> -	 * ^          ^         ^
> -	 * |          |         |
> -	 * hdr        |         |
> -	 * types_data-+         |
> -	 * strs_data------------+
> +	 * +--------------------------------+---------------------+
> +	 * |  Header  |  Types  |  Strings  |Optional kind layout |
> +	 * +--------------------------------+---------------------+
> +	 * ^          ^         ^           ^
> +	 * |          |         |           |
> +	 * raw_data   |         |           |
> +	 * types_data-+         |           |
> +	 * strs_data------------+           |
> +	 * kind_layout----------------------+
> +	 *
> +	 * A separate struct btf_header is allocated for btf->hdr,
> +	 * and header information is copied into it.  This allows us
> +	 * to handle header data for various header formats; the original,
> +	 * the extended header with kind layout, etc.
>   	 *
>   	 * If BTF data is later modified, e.g., due to types added or
>   	 * removed, BTF deduplication performed, etc, this contiguous
> -	 * representation is broken up into three independently allocated
> -	 * memory regions to be able to modify them independently.
> +	 * representation is broken up into four independent memory
> +	 * regions.
> +	 *
>   	 * raw_data is nulled out at that point, but can be later allocated
>   	 * and cached again if user calls btf__raw_data(), at which point
> -	 * raw_data will contain a contiguous copy of header, types, and
> -	 * strings:
> +	 * raw_data will contain a contiguous copy of header, types, strings
> +	 * and optionally kind_layout.  kind_layout optionally points to a
> +	 * kind_layout array - this allows us to encode information about
> +	 * the kinds known at encoding time.  If kind_layout is NULL no
> +	 * kind information is encoded.
>   	 *
> -	 * +----------+  +---------+  +-----------+
> -	 * |  Header  |  |  Types  |  |  Strings  |
> -	 * +----------+  +---------+  +-----------+
> -	 * ^             ^            ^
> -	 * |             |            |
> -	 * hdr           |            |
> -	 * types_data----+            |
> -	 * strset__data(strs_set)-----+
> +	 * +----------+  +---------+  +-----------+   +-----------+
> +	 * |  Header  |  |  Types  |  |  Strings  |   |kind_layout|
> +	 * +----------+  +---------+  +-----------+   +-----------+
> +	 * ^             ^            ^               ^
> +	 * |             |            |               |
> +	 * hdr           |            |               |
> +	 * types_data----+            |               |
> +	 * strset__data(strs_set)-----+               |
> +	 * kind_layout--------------------------------+
>   	 *
> -	 *               +----------+---------+-----------+
> -	 *               |  Header  |  Types  |  Strings  |
> -	 * raw_data----->+----------+---------+-----------+
> +	 *               +----------+---------+-----------+---------------------+
> +	 *               |  Header  |  Types  |  Strings  | Optional kind layout|
> +	 * raw_data----->+----------+---------+-----------+---------------------+
>   	 */
> -	struct btf_header *hdr;
> +	struct btf_header *hdr; /* separately-allocated header data */
>   
>   	void *types_data;
>   	size_t types_data_cap; /* used size stored in hdr->type_len */
> @@ -123,6 +134,13 @@ struct btf {
>   	/* whether raw_data is a (read-only) mmap */
>   	bool raw_data_is_mmap;
>   
> +	/* is BTF modifiable? i.e. is it split into separate sections as described above? */
> +	bool modifiable;
> +	/* Points either at raw kind layout data in parsed BTF (if present), or
> +	 * at an allocated kind layout array when BTF is modifiable.
> +	 */
> +	void *kind_layout;
> +
>   	/* BTF object FD, if loaded into kernel */
>   	int fd;
>   
> @@ -214,7 +232,7 @@ static int btf_add_type_idx_entry(struct btf *btf, __u32 type_off)
>   	return 0;
>   }
>   
> -static void btf_bswap_hdr(struct btf_header *h)
> +static void btf_bswap_hdr(struct btf_header *h, __u32 hdr_len)
>   {
>   	h->magic = bswap_16(h->magic);
>   	h->hdr_len = bswap_32(h->hdr_len);
> @@ -222,50 +240,68 @@ static void btf_bswap_hdr(struct btf_header *h)
>   	h->type_len = bswap_32(h->type_len);
>   	h->str_off = bswap_32(h->str_off);
>   	h->str_len = bswap_32(h->str_len);
> +	/* May be operating on raw data with hdr_len that does not include below fields */
> +	if (hdr_len >= sizeof(struct btf_header)) {
> +		h->kind_layout_off = bswap_32(h->kind_layout_off);
> +		h->kind_layout_len = bswap_32(h->kind_layout_len);
> +	}
>   }
>   
>   static int btf_parse_hdr(struct btf *btf)
>   {
> -	struct btf_header *hdr = btf->hdr;
> +	struct btf_header *hdr = btf->raw_data;
> +	__u32 hdr_len = hdr->hdr_len;
>   	__u32 meta_left;
>   
> -	if (btf->raw_size < sizeof(struct btf_header)) {
> +	if (btf->raw_size < offsetofend(struct btf_header, str_len)) {
>   		pr_debug("BTF header not found\n");
>   		return -EINVAL;
>   	}
>   
>   	if (hdr->magic == bswap_16(BTF_MAGIC)) {
>   		btf->swapped_endian = true;
> -		if (bswap_32(hdr->hdr_len) != sizeof(struct btf_header)) {
> +		hdr_len = bswap_32(hdr->hdr_len);
> +		if (hdr_len < offsetofend(struct btf_header, str_len)) {
>   			pr_warn("Can't load BTF with non-native endianness due to unsupported header length %u\n",
> -				bswap_32(hdr->hdr_len));
> +				hdr_len);
>   			return -ENOTSUP;
>   		}
> -		btf_bswap_hdr(hdr);
>   	} else if (hdr->magic != BTF_MAGIC) {
>   		pr_debug("Invalid BTF magic: %x\n", hdr->magic);
>   		return -EINVAL;
>   	}
>   
> -	if (btf->raw_size < hdr->hdr_len) {
> +	if (btf->raw_size < hdr_len) {
>   		pr_debug("BTF header len %u larger than data size %u\n",
> -			 hdr->hdr_len, btf->raw_size);
> +			 hdr_len, btf->raw_size);
>   		return -EINVAL;
>   	}
>   
> -	meta_left = btf->raw_size - hdr->hdr_len;
> -	if (meta_left < (long long)hdr->str_off + hdr->str_len) {
> +	/* At this point, we have basic header information, so allocate btf->hdr */
> +	btf->hdr = calloc(1, sizeof(struct btf_header));
> +	if (!btf->hdr) {
> +		pr_debug("BTF header allocation failed\n");
> +		return -ENOMEM;
> +	}
> +	if (btf->swapped_endian)
> +		btf_bswap_hdr(hdr, hdr_len);
> +	memcpy(btf->hdr, hdr, hdr_len < sizeof(struct btf_header) ? hdr_len :
> +								    sizeof(struct btf_header));
can we simplify this a little bit: memcpy(btf->hdr, hdr, min(hdr_len, 
sizeof(struct btf_header));?
min macro is available in libbpf_internal.h
> +
> +	meta_left = btf->raw_size - hdr_len;
> +	if (meta_left < (long long)btf->hdr->str_off + btf->hdr->str_len) {
>   		pr_debug("Invalid BTF total size: %u\n", btf->raw_size);
>   		return -EINVAL;
>   	}
>   
> -	if ((long long)hdr->type_off + hdr->type_len > hdr->str_off) {
> +	if ((long long)btf->hdr->type_off + btf->hdr->type_len > btf->hdr->str_off) {
>   		pr_debug("Invalid BTF data sections layout: type data at %u + %u, strings data at %u + %u\n",
> -			 hdr->type_off, hdr->type_len, hdr->str_off, hdr->str_len);
> +			 btf->hdr->type_off, btf->hdr->type_len, btf->hdr->str_off,
> +			 btf->hdr->str_len);
>   		return -EINVAL;
>   	}
>   
> -	if (hdr->type_off % 4) {
> +	if (btf->hdr->type_off % 4) {
>   		pr_debug("BTF type section is not aligned to 4 bytes\n");
>   		return -EINVAL;
>   	}
> @@ -292,6 +328,22 @@ static int btf_parse_str_sec(struct btf *btf)
>   	return 0;
>   }
>   
> +static int btf_parse_kind_layout_sec(struct btf *btf)
> +{
> +	const struct btf_header *hdr = btf->hdr;
> +
> +	if (!hdr->kind_layout_off || !hdr->kind_layout_len)
> +		return 0;
> +
> +	if (hdr->kind_layout_len % sizeof(struct btf_kind_layout) != 0) {
> +		pr_debug("Invalid BTF kind layout section\n");
> +		return -EINVAL;
> +	}
> +	btf->kind_layout = btf->raw_data + btf->hdr->hdr_len + btf->hdr->kind_layout_off;
> +
> +	return 0;
> +}
> +
>   static int btf_type_size(const struct btf_type *t)
>   {
>   	const int base_size = sizeof(struct btf_type);
> @@ -951,7 +1003,8 @@ __s32 btf__find_by_name_kind(const struct btf *btf, const char *type_name,
>   
>   static bool btf_is_modifiable(const struct btf *btf)
>   {
> -	return (void *)btf->hdr != btf->raw_data;
> +	/* BTF is modifiable if split into multiple sections */
> +	return btf->modifiable;
>   }
>   
>   static void btf_free_raw_data(struct btf *btf)
> @@ -980,10 +1033,11 @@ void btf__free(struct btf *btf)
>   		 * might still have a cached contiguous raw data present,
>   		 * which will be unconditionally freed below.
>   		 */
> -		free(btf->hdr);
>   		free(btf->types_data);
>   		strset__free(btf->strs_set);
> +		free(btf->kind_layout);
>   	}
> +	free(btf->hdr);
>   	btf_free_raw_data(btf);
>   	free(btf->raw_data_swapped);
>   	free(btf->type_offs);
> @@ -994,6 +1048,7 @@ void btf__free(struct btf *btf)
>   
>   static struct btf *btf_new_empty(struct btf *base_btf)
>   {
> +	struct btf_header *hdr;
>   	struct btf *btf;
>   
>   	btf = calloc(1, sizeof(*btf));
> @@ -1022,14 +1077,20 @@ static struct btf *btf_new_empty(struct btf *base_btf)
>   		return ERR_PTR(-ENOMEM);
>   	}
>   
> -	btf->hdr = btf->raw_data;
> -	btf->hdr->hdr_len = sizeof(struct btf_header);
> -	btf->hdr->magic = BTF_MAGIC;
> -	btf->hdr->version = BTF_VERSION;
> +	hdr = btf->raw_data;
> +	hdr->hdr_len = sizeof(struct btf_header);
> +	hdr->magic = BTF_MAGIC;
> +	hdr->version = BTF_VERSION;
>   
> -	btf->types_data = btf->raw_data + btf->hdr->hdr_len;
> -	btf->strs_data = btf->raw_data + btf->hdr->hdr_len;
> -	btf->hdr->str_len = base_btf ? 0 : 1; /* empty string at offset 0 */
> +	btf->types_data = btf->raw_data + hdr->hdr_len;
> +	btf->strs_data = btf->raw_data + hdr->hdr_len;
> +	hdr->str_len = base_btf ? 0 : 1; /* empty string at offset 0 */
> +	btf->hdr = calloc(1, sizeof(struct btf_header));
> +	if (!btf->hdr) {
> +		free(btf);
> +		return ERR_PTR(-ENOMEM);
> +	}
> +	memcpy(btf->hdr, hdr, sizeof(*hdr));
>   
>   	return btf;
>   }
> @@ -1078,7 +1139,6 @@ static struct btf *btf_new(const void *data, __u32 size, struct btf *base_btf, b
>   
>   	btf->raw_size = size;
>   
> -	btf->hdr = btf->raw_data;
>   	err = btf_parse_hdr(btf);
>   	if (err)
>   		goto done;
> @@ -1087,6 +1147,7 @@ static struct btf *btf_new(const void *data, __u32 size, struct btf *base_btf, b
>   	btf->types_data = btf->raw_data + btf->hdr->hdr_len + btf->hdr->type_off;
>   
>   	err = btf_parse_str_sec(btf);
> +	err = err ?: btf_parse_kind_layout_sec(btf);
>   	err = err ?: btf_parse_type_sec(btf);
>   	err = err ?: btf_sanity_check(btf);
>   	if (err)
> @@ -1550,6 +1611,11 @@ static void *btf_get_raw_data(const struct btf *btf, __u32 *size, bool swap_endi
>   	}
>   
>   	data_sz = hdr->hdr_len + hdr->type_len + hdr->str_len;
> +	if (btf->kind_layout) {
> +		data_sz = roundup(data_sz, 4);
> +		data_sz += hdr->kind_layout_len;
> +		hdr->kind_layout_off = roundup(hdr->type_len + hdr->str_len, 4);
> +	}
>   	data = calloc(1, data_sz);
>   	if (!data)
>   		return NULL;
> @@ -1557,7 +1623,7 @@ static void *btf_get_raw_data(const struct btf *btf, __u32 *size, bool swap_endi
>   
>   	memcpy(p, hdr, hdr->hdr_len);
>   	if (swap_endian)
> -		btf_bswap_hdr(p);
> +		btf_bswap_hdr(p, hdr->hdr_len);
>   	p += hdr->hdr_len;
>   
>   	memcpy(p, btf->types_data, hdr->type_len);
> @@ -1576,7 +1642,11 @@ static void *btf_get_raw_data(const struct btf *btf, __u32 *size, bool swap_endi
>   	p += hdr->type_len;
>   
>   	memcpy(p, btf_strs_data(btf), hdr->str_len);
> -	p += hdr->str_len;
> +
> +	if (btf->kind_layout) {
> +		p = data + hdr->hdr_len + hdr->kind_layout_off;
> +		memcpy(p, btf->kind_layout, hdr->kind_layout_len);
> +	}
>   
>   	*size = data_sz;
>   	return data;
> @@ -1717,13 +1787,13 @@ static void btf_invalidate_raw_data(struct btf *btf)
>   	}
>   }
>   
> -/* Ensure BTF is ready to be modified (by splitting into a three memory
> - * regions for header, types, and strings). Also invalidate cached
> - * raw_data, if any.
> +/* Ensure BTF is ready to be modified (by splitting into memory regions
> + * for types and strings, with kind layout section if needed (btf->hdr
> + * is already a separate region).  Also invalidate cached raw_data, if any.
>    */
>   static int btf_ensure_modifiable(struct btf *btf)
>   {
> -	void *hdr, *types;
> +	void *types, *kind_layout = NULL;
>   	struct strset *set = NULL;
>   	int err = -ENOMEM;
>   
> @@ -1733,15 +1803,20 @@ static int btf_ensure_modifiable(struct btf *btf)
>   		return 0;
>   	}
>   
> -	/* split raw data into three memory regions */
> -	hdr = malloc(btf->hdr->hdr_len);
> +	/* split raw data into memory regions; btf->hdr is done already. */
>   	types = malloc(btf->hdr->type_len);
> -	if (!hdr || !types)
> +	if (!types)
>   		goto err_out;
> -
> -	memcpy(hdr, btf->hdr, btf->hdr->hdr_len);
>   	memcpy(types, btf->types_data, btf->hdr->type_len);
>   
> +	if (btf->hdr->kind_layout_len && btf->hdr->kind_layout_off) {
> +		kind_layout = malloc(btf->hdr->kind_layout_len);
> +		if (!kind_layout)
> +			goto err_out;
> +		memcpy(kind_layout, btf->raw_data + btf->hdr->hdr_len + btf->hdr->kind_layout_off,
> +		       btf->hdr->kind_layout_len);
> +	}
> +
>   	/* build lookup index for all strings */
>   	set = strset__new(BTF_MAX_STR_OFFSET, btf->strs_data, btf->hdr->str_len);
>   	if (IS_ERR(set)) {
> @@ -1750,11 +1825,12 @@ static int btf_ensure_modifiable(struct btf *btf)
>   	}
>   
>   	/* only when everything was successful, update internal state */
> -	btf->hdr = hdr;
>   	btf->types_data = types;
>   	btf->types_data_cap = btf->hdr->type_len;
>   	btf->strs_data = NULL;
>   	btf->strs_set = set;
> +	if (kind_layout)
> +		btf->kind_layout = kind_layout;
>   	/* if BTF was created from scratch, all strings are guaranteed to be
>   	 * unique and deduplicated
>   	 */
> @@ -1766,12 +1842,14 @@ static int btf_ensure_modifiable(struct btf *btf)
>   	/* invalidate raw_data representation */
>   	btf_invalidate_raw_data(btf);
>   
> +	btf->modifiable = true;
> +
>   	return 0;
>   
>   err_out:
>   	strset__free(set);
> -	free(hdr);
>   	free(types);
> +	free(kind_layout);
>   	return err;
>   }
>   

Re: [PATCH v6 bpf-next 03/10] libbpf: use kind layout to compute an unknown kind size

[Mykyta Yatsenko]

On 12/11/25 08:33, Alan Maguire wrote:
> On 10/12/2025 20:55, bot+bpf-ci@kernel.org wrote:
>>> diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
>>> index 737adc560818..4eb0704a0309 100644
>>> --- a/tools/lib/bpf/btf.c
>>> +++ b/tools/lib/bpf/btf.c
>> [ ... ]
>>
>>> +/* for unknown kinds, consult kind layout. */
>>> +static int btf_type_size_unknown(const struct btf *btf, const struct btf_type *t)
>>> +{
>>> +	int size = sizeof(struct btf_type);
>>> +	struct btf_kind_layout *k = NULL;
>>> +	__u16 vlen = btf_vlen(t);
>>> +	__u8 kind = btf_kind(t);
>>> +
>>> +	if (btf->kind_layout)
>>> +		k = &((struct btf_kind_layout *)btf->kind_layout)[kind];
>>> +
>>> +	if (!k || (void *)k > ((void *)btf->kind_layout + btf->hdr->kind_layout_len)) {
>>                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>>
>> Can the bounds check be moved before the array access?  Currently the code
>> indexes into kind_layout[kind] before validating whether 'kind' is within
>> bounds.  While computing an out-of-bounds pointer may not cause immediate
>> issues, the bounds check itself appears incorrect.
>>
>> The check uses '>' which allows access when k points exactly to the end of
>> the array. For example, if kind_layout_len is 40 bytes (20 kinds * 2 bytes
>> each) and kind is 20, then k would point to offset 40, and the check
>> '(ptr at 40) > (ptr at 40)' evaluates to false, allowing the subsequent
>> read of k->info_sz at an out-of-bounds location.
>>
>> Should this be: 'if (!btf->kind_layout || kind >= (btf->hdr->kind_layout_len
>> / sizeof(*k)))'?  This would validate the index before the array access and
>> ensure the full structure can be read safely.
>>
> Yep, good suggestion. Will fix.
maybe something like this:

__u32 off = kind * sizeof(struct btf_kind_layout);

if (!btf->kind_layout || off >= btf->hdr->kind_layout_len) {
     pr_debug("Unsupported BTF_KIND: %u\n", kind);
     return -EINVAL;
}

k = btf->kind_layout + off;
>>> +		pr_debug("Unsupported BTF_KIND: %u\n", btf_kind(t));
>>> +		return -EINVAL;
>>> +	}
>>> +
>>> +	size += k->info_sz;
>>> +	size += vlen * k->elem_sz;
>>> +
>>> +	return size;
>>> +}
>> [ ... ]
>>
>>
>> ---
>> AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
>> See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
>>
>> CI run summary: https://github.com/kernel-patches/bpf/actions/runs/20112692486
>