[PATCH v8 bpf-next 00/10] Add kind layout to BTF

[PATCH v8 bpf-next 00/10] Add kind layout to BTF

[Alan Maguire]

Update struct btf_header to add a new "kind_layout" section containing
a description of how to parse the BTF kinds known about at BTF
encoding time.  This provides the opportunity for tools that might
not know all of these kinds - as is the case when older tools run
on more newly-generated BTF - to still parse the BTF provided,
even if it cannot all be used.

Also add support to bpftool to dump metadata about BTF; its size,
header information and kind layout section.

The ideas here were discussed at [1], with further discussion
at [2].

Patches for pahole will enable the kind layout addition during
BTF generation are at [3], but even absent these the addition of the
kind_layout feature in the final patch in this series should not break
anything since such unknown features are simply ignored during pahole
BTF generation.

Changes since v7: [4]:

- Fixed comment style in UAPI headers (Mykyta, patch 1)
- Simplify calcuation of header size using min() (Mykyta, patch 2)
- simplify computation of bounds for kind (Mykyta, patch 3)
- Added utility functions for updating type, string offsets when
  data is added; this simplifies the code and encapsulates such
  updates more clearly (patch 2)

Changes since v6: [5]:

- BPF review bot caught some memory leaks around freeing
  of kind layout; more importantly, it noted that we were
  breaking with the contiguous BTF representation for
  btf_new_empty_opts(). Doing so meant that freeing kind_layout
  could not be predicated on having btf->modifiable set, so
  adpoted the contiguous raw data layout for BTF to be
  consistent with type/string storage (patches 2,4)
- Moved checks for kind overflow prior to referencing kinds
  to avoid any risk of overrun (patches 3, 8)
- Tightened up kind layout header offset/len header validation
  to catch invalid combinations early in btf_parse_hdr()
  (patch 2)
- Fixed selftest to verify calloc success (patch 7)

Changes since v5: [6]:

- removed flags field from kind layout; it is not really workable
  since we would have to define semantics of all possible future
  flags today to be usable. Instead stick to parsing only, which
  means each kind just needs the length of the singular and
  vlen-specified objects (Alexei)
- added documentation for bpftool BTF metadata dump (Quentin, patch 9)

Changes since v4: [7]:

- removed CRC generation since it is not needed to handle modules
  built at different time than kernel; distilled base BTF supports
  this now
- fixed up bpftool display of empty kind names, comment/documentation
  indentation (Quentin, patches 8, 9)

Changes since v3 [8]:

- fixed mismerge issues with kbuild changes for BTF generation
  (patches 9, 14)
- fixed a few small issues in libbpf with kind layout representation
  (patches 2, 4)

Changes since v2 [9]:

- drop "optional" kind flag (Andrii, patch 1)
- allocate "struct btf_header" for struct btf to ensure
  we can always access new fields (Andrii, patch 2)
- use an internal BTF kind array in btf.c to simplify
  kind encoding (Andrii, patch 2)
- drop use of kind layout information for in-kernel parsing,
  since the kernel needs to be strict in what it accepts
  (Andrii, patch 6)
 added CRC verification for BTF objects and for matching
  with base object (Alexei, patches 7,8)
- fixed bpftool json output (Quentin, patch 10)
- added standalone module BTF support, tests (patches 13-17)

Changes since RFC
- Terminology change from meta -> kind_layout
  (Alexei and Andrii)
- Simplify representation, removing meta header
  and just having kind layout section (Alexei)
- Fixed bpftool to have JSON support, support
  prefix match, documented changes (Quentin)
- Separated metadata opts into add_kind_layout
  and add_crc
- Added additional positive/negative tests
  to cover basic unknown kind, one with an
  info_sz object following it and one with
  N elem_sz elements following it.
- Updated pahole-flags to use help output
  rather than version to see if features
  are present


[1] https://lore.kernel.org/bpf/CAEf4BzYjWHRdNNw4B=eOXOs_ONrDwrgX4bn=Nuc1g8JPFC34MA@mail.gmail.com/
[2] https://lore.kernel.org/bpf/20230531201936.1992188-1-alan.maguire@oracle.com/
[3] https://lore.kernel.org/dwarves/20251210202752.813919-1-alan.maguire@oracle.com/
[4] https://lore.kernel.org/dwarves/20251211164646.1219122-1-alan.maguire@oracle.com/
[5] https://lore.kernel.org/bpf/20251210203243.814529-1-alan.maguire@oracle.com/
[6] https://lore.kernel.org/bpf/20250528095743.791722-1-alan.maguire@oracle.com/
[7] https://lore.kernel.org/bpf/20231112124834.388735-1-alan.maguire@oracle.com/
[8] https://lore.kernel.org/bpf/20231110110304.63910-1-alan.maguire@oracle.com/
[9] https://lore.kernel.org/bpf/20230616171728.530116-1-alan.maguire@oracle.com/

Alan Maguire (10):
  btf: add kind layout encoding to UAPI
  libbpf: Support kind layout section handling in BTF
  libbpf: use kind layout to compute an unknown kind size
  libbpf: Add kind layout encoding support
  libbpf: BTF validation can use kind layout for unknown kinds
  btf: support kernel parsing of BTF with kind layout
  selftests/bpf: test kind encoding/decoding
  bpftool: add BTF dump "format meta" to dump header/metadata
  bpftool: Update doc to describe bpftool btf dump .. format metadata
  kbuild, bpf: Specify "kind_layout" optional feature

 include/uapi/linux/btf.h                      |  11 +
 kernel/bpf/btf.c                              |  96 ++++-
 scripts/Makefile.btf                          |   2 +
 .../bpf/bpftool/Documentation/bpftool-btf.rst |  28 +-
 tools/bpf/bpftool/bash-completion/bpftool     |   2 +-
 tools/bpf/bpftool/btf.c                       |  94 ++++-
 tools/include/uapi/linux/btf.h                |  11 +
 tools/lib/bpf/btf.c                           | 362 +++++++++++++-----
 tools/lib/bpf/btf.h                           |  20 +
 tools/lib/bpf/libbpf.map                      |   1 +
 .../selftests/bpf/prog_tests/btf_kind.c       | 178 +++++++++
 11 files changed, 690 insertions(+), 115 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/btf_kind.c

-- 
2.39.3

[PATCH v8 bpf-next 01/10] btf: add kind layout encoding to UAPI

[Alan Maguire]

BTF kind layouts provide information to parse BTF kinds. By separating
parsing BTF from using all the information it provides, we allow BTF
to encode new features even if they cannot be used by readers. This
will be helpful in particular for cases where older tools are used
to parse newer BTF with kinds the older tools do not recognize;
the BTF can still be parsed in such cases using kind layout.

The intent is to support encoding of kind layouts optionally so that
tools like pahole can add this information. For each kind, we record

- length of singular element following struct btf_type
- length of each of the btf_vlen() elements following

The ideas here were discussed at [1], [2]; hence

Suggested-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Alan Maguire <alan.maguire@oracle.com>

[1] https://lore.kernel.org/bpf/CAEf4BzYjWHRdNNw4B=eOXOs_ONrDwrgX4bn=Nuc1g8JPFC34MA@mail.gmail.com/
[2] https://lore.kernel.org/bpf/20230531201936.1992188-1-alan.maguire@oracle.com/
---
 include/uapi/linux/btf.h       | 11 +++++++++++
 tools/include/uapi/linux/btf.h | 11 +++++++++++
 2 files changed, 22 insertions(+)

diff --git a/include/uapi/linux/btf.h b/include/uapi/linux/btf.h
index 266d4ffa6c07..c1854a1c7b38 100644
--- a/include/uapi/linux/btf.h
+++ b/include/uapi/linux/btf.h
@@ -8,6 +8,15 @@
 #define BTF_MAGIC	0xeB9F
 #define BTF_VERSION	1
 
+/*
+ * kind layout section consists of a struct btf_kind_layout for each known
+ * kind at BTF encoding time.
+ */
+struct btf_kind_layout {
+	__u8 info_sz;		/* size of singular element after btf_type */
+	__u8 elem_sz;		/* size of each of btf_vlen(t) elements */
+};
+
 struct btf_header {
 	__u16	magic;
 	__u8	version;
@@ -19,6 +28,8 @@ struct btf_header {
 	__u32	type_len;	/* length of type section	*/
 	__u32	str_off;	/* offset of string section	*/
 	__u32	str_len;	/* length of string section	*/
+	__u32	kind_layout_off;/* offset of kind layout section */
+	__u32	kind_layout_len;/* length of kind layout section */
 };
 
 /* Max # of type identifier */
diff --git a/tools/include/uapi/linux/btf.h b/tools/include/uapi/linux/btf.h
index 266d4ffa6c07..c1854a1c7b38 100644
--- a/tools/include/uapi/linux/btf.h
+++ b/tools/include/uapi/linux/btf.h
@@ -8,6 +8,15 @@
 #define BTF_MAGIC	0xeB9F
 #define BTF_VERSION	1
 
+/*
+ * kind layout section consists of a struct btf_kind_layout for each known
+ * kind at BTF encoding time.
+ */
+struct btf_kind_layout {
+	__u8 info_sz;		/* size of singular element after btf_type */
+	__u8 elem_sz;		/* size of each of btf_vlen(t) elements */
+};
+
 struct btf_header {
 	__u16	magic;
 	__u8	version;
@@ -19,6 +28,8 @@ struct btf_header {
 	__u32	type_len;	/* length of type section	*/
 	__u32	str_off;	/* offset of string section	*/
 	__u32	str_len;	/* length of string section	*/
+	__u32	kind_layout_off;/* offset of kind layout section */
+	__u32	kind_layout_len;/* length of kind layout section */
 };
 
 /* Max # of type identifier */
-- 
2.39.3

[PATCH v8 bpf-next 02/10] libbpf: Support kind layout section handling in BTF

[Alan Maguire]

Support reading in kind layout fixing endian issues on reading;
also support writing kind layout section to raw BTF object.
There is not yet an API to populate the kind layout with meaningful
information.

As part of this, we need to consider multiple valid BTF header
sizes; the original or the kind layout-extended headers.
So to support this, the "struct btf" representation is modified
to always allocate a "struct btf_header" and copy the valid
portion from the raw data to it; this means we can always safely
check fields like btf->hdr->kind_layout_len.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 tools/lib/bpf/btf.c | 260 +++++++++++++++++++++++++++++++-------------
 1 file changed, 183 insertions(+), 77 deletions(-)

diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
index b136572e889a..8835aee6ee84 100644
--- a/tools/lib/bpf/btf.c
+++ b/tools/lib/bpf/btf.c
@@ -40,42 +40,53 @@ struct btf {
 
 	/*
 	 * When BTF is loaded from an ELF or raw memory it is stored
-	 * in a contiguous memory block. The hdr, type_data, and, strs_data
+	 * in a contiguous memory block. The  type_data, and, strs_data
 	 * point inside that memory region to their respective parts of BTF
 	 * representation:
 	 *
-	 * +--------------------------------+
-	 * |  Header  |  Types  |  Strings  |
-	 * +--------------------------------+
-	 * ^          ^         ^
-	 * |          |         |
-	 * hdr        |         |
-	 * types_data-+         |
-	 * strs_data------------+
+	 * +--------------------------------+---------------------+
+	 * |  Header  |  Types  |  Strings  |Optional kind layout |
+	 * +--------------------------------+---------------------+
+	 * ^          ^         ^           ^
+	 * |          |         |           |
+	 * raw_data   |         |           |
+	 * types_data-+         |           |
+	 * strs_data------------+           |
+	 * kind_layout----------------------+
+	 *
+	 * A separate struct btf_header is allocated for btf->hdr,
+	 * and header information is copied into it.  This allows us
+	 * to handle header data for various header formats; the original,
+	 * the extended header with kind layout, etc.
 	 *
 	 * If BTF data is later modified, e.g., due to types added or
 	 * removed, BTF deduplication performed, etc, this contiguous
-	 * representation is broken up into three independently allocated
-	 * memory regions to be able to modify them independently.
+	 * representation is broken up into four independent memory
+	 * regions.
+	 *
 	 * raw_data is nulled out at that point, but can be later allocated
 	 * and cached again if user calls btf__raw_data(), at which point
-	 * raw_data will contain a contiguous copy of header, types, and
-	 * strings:
+	 * raw_data will contain a contiguous copy of header, types, strings
+	 * and optionally kind_layout.  kind_layout optionally points to a
+	 * kind_layout array - this allows us to encode information about
+	 * the kinds known at encoding time.  If kind_layout is NULL no
+	 * kind information is encoded.
 	 *
-	 * +----------+  +---------+  +-----------+
-	 * |  Header  |  |  Types  |  |  Strings  |
-	 * +----------+  +---------+  +-----------+
-	 * ^             ^            ^
-	 * |             |            |
-	 * hdr           |            |
-	 * types_data----+            |
-	 * strset__data(strs_set)-----+
+	 * +----------+  +---------+  +-----------+   +-----------+
+	 * |  Header  |  |  Types  |  |  Strings  |   |kind_layout|
+	 * +----------+  +---------+  +-----------+   +-----------+
+	 * ^             ^            ^               ^
+	 * |             |            |               |
+	 * hdr           |            |               |
+	 * types_data----+            |               |
+	 * strset__data(strs_set)-----+               |
+	 * kind_layout--------------------------------+
 	 *
-	 *               +----------+---------+-----------+
-	 *               |  Header  |  Types  |  Strings  |
-	 * raw_data----->+----------+---------+-----------+
+	 *               +----------+---------+-----------+---------------------+
+	 *               |  Header  |  Types  |  Strings  | Optional kind layout|
+	 * raw_data----->+----------+---------+-----------+---------------------+
 	 */
-	struct btf_header *hdr;
+	struct btf_header *hdr; /* separately-allocated header data */
 
 	void *types_data;
 	size_t types_data_cap; /* used size stored in hdr->type_len */
@@ -123,6 +134,13 @@ struct btf {
 	/* whether raw_data is a (read-only) mmap */
 	bool raw_data_is_mmap;
 
+	/* is BTF modifiable? i.e. is it split into separate sections as described above? */
+	bool modifiable;
+	/* Points either at raw kind layout data in parsed BTF (if present), or
+	 * at an allocated kind layout array when BTF is modifiable.
+	 */
+	void *kind_layout;
+
 	/* BTF object FD, if loaded into kernel */
 	int fd;
 
@@ -214,7 +232,7 @@ static int btf_add_type_idx_entry(struct btf *btf, __u32 type_off)
 	return 0;
 }
 
-static void btf_bswap_hdr(struct btf_header *h)
+static void btf_bswap_hdr(struct btf_header *h, __u32 hdr_len)
 {
 	h->magic = bswap_16(h->magic);
 	h->hdr_len = bswap_32(h->hdr_len);
@@ -222,54 +240,87 @@ static void btf_bswap_hdr(struct btf_header *h)
 	h->type_len = bswap_32(h->type_len);
 	h->str_off = bswap_32(h->str_off);
 	h->str_len = bswap_32(h->str_len);
+	/* May be operating on raw data with hdr_len that does not include below fields */
+	if (hdr_len >= sizeof(struct btf_header)) {
+		h->kind_layout_off = bswap_32(h->kind_layout_off);
+		h->kind_layout_len = bswap_32(h->kind_layout_len);
+	}
 }
 
 static int btf_parse_hdr(struct btf *btf)
 {
-	struct btf_header *hdr = btf->hdr;
+	struct btf_header *hdr = btf->raw_data;
+	__u32 hdr_len = hdr->hdr_len;
 	__u32 meta_left;
 
-	if (btf->raw_size < sizeof(struct btf_header)) {
+	if (btf->raw_size < offsetofend(struct btf_header, str_len)) {
 		pr_debug("BTF header not found\n");
 		return -EINVAL;
 	}
 
 	if (hdr->magic == bswap_16(BTF_MAGIC)) {
 		btf->swapped_endian = true;
-		if (bswap_32(hdr->hdr_len) != sizeof(struct btf_header)) {
+		hdr_len = bswap_32(hdr->hdr_len);
+		if (hdr_len < offsetofend(struct btf_header, str_len)) {
 			pr_warn("Can't load BTF with non-native endianness due to unsupported header length %u\n",
-				bswap_32(hdr->hdr_len));
+				hdr_len);
 			return -ENOTSUP;
 		}
-		btf_bswap_hdr(hdr);
+		btf_bswap_hdr(hdr, hdr_len);
 	} else if (hdr->magic != BTF_MAGIC) {
 		pr_debug("Invalid BTF magic: %x\n", hdr->magic);
 		return -EINVAL;
 	}
 
-	if (btf->raw_size < hdr->hdr_len) {
+	if (btf->raw_size < hdr_len) {
 		pr_debug("BTF header len %u larger than data size %u\n",
-			 hdr->hdr_len, btf->raw_size);
+			 hdr_len, btf->raw_size);
 		return -EINVAL;
 	}
 
-	meta_left = btf->raw_size - hdr->hdr_len;
-	if (meta_left < (long long)hdr->str_off + hdr->str_len) {
+	/* At this point, we have basic header information, so allocate btf->hdr */
+	btf->hdr = calloc(1, sizeof(struct btf_header));
+	if (!btf->hdr) {
+		pr_debug("BTF header allocation failed\n");
+		return -ENOMEM;
+	}
+	memcpy(btf->hdr, hdr, min((size_t)hdr_len, sizeof(struct btf_header)));
+
+	meta_left = btf->raw_size - hdr_len;
+	if (meta_left < (long long)btf->hdr->str_off + btf->hdr->str_len) {
 		pr_debug("Invalid BTF total size: %u\n", btf->raw_size);
 		return -EINVAL;
 	}
 
-	if ((long long)hdr->type_off + hdr->type_len > hdr->str_off) {
+	if ((long long)btf->hdr->type_off + btf->hdr->type_len > btf->hdr->str_off) {
 		pr_debug("Invalid BTF data sections layout: type data at %u + %u, strings data at %u + %u\n",
-			 hdr->type_off, hdr->type_len, hdr->str_off, hdr->str_len);
+			 btf->hdr->type_off, btf->hdr->type_len, btf->hdr->str_off,
+			 btf->hdr->str_len);
 		return -EINVAL;
 	}
 
-	if (hdr->type_off % 4) {
+	if (btf->hdr->type_off % 4) {
 		pr_debug("BTF type section is not aligned to 4 bytes\n");
 		return -EINVAL;
 	}
 
+	if (btf->hdr->kind_layout_len == 0)
+		return 0;
+
+	if (btf->hdr->kind_layout_off % 4) {
+		pr_debug("BTF kind_layout section is not aligned to 4 bytes\n");
+		return -EINVAL;
+	}
+	if (btf->hdr->kind_layout_off < btf->hdr->str_off + btf->hdr->str_len) {
+		pr_debug("Invalid BTF data sections layout: strings data at %u + %u, kind layout data at %u + %u\n",
+			 btf->hdr->str_off, btf->hdr->str_len,
+			 btf->hdr->kind_layout_off, btf->hdr->kind_layout_len);
+		return -EINVAL;
+	}
+	if (btf->hdr->kind_layout_off + btf->hdr->kind_layout_len > meta_left) {
+		pr_debug("Invalid BTF total size: %u\n", btf->raw_size);
+		return -EINVAL;
+	}
 	return 0;
 }
 
@@ -292,6 +343,22 @@ static int btf_parse_str_sec(struct btf *btf)
 	return 0;
 }
 
+static int btf_parse_kind_layout_sec(struct btf *btf)
+{
+	const struct btf_header *hdr = btf->hdr;
+
+	if (!hdr->kind_layout_len)
+		return 0;
+
+	if (hdr->kind_layout_len % sizeof(struct btf_kind_layout) != 0) {
+		pr_debug("Invalid BTF kind layout section\n");
+		return -EINVAL;
+	}
+	btf->kind_layout = btf->raw_data + btf->hdr->hdr_len + btf->hdr->kind_layout_off;
+
+	return 0;
+}
+
 static int btf_type_size(const struct btf_type *t)
 {
 	const int base_size = sizeof(struct btf_type);
@@ -951,7 +1018,8 @@ __s32 btf__find_by_name_kind(const struct btf *btf, const char *type_name,
 
 static bool btf_is_modifiable(const struct btf *btf)
 {
-	return (void *)btf->hdr != btf->raw_data;
+	/* BTF is modifiable if split into multiple sections */
+	return btf->modifiable;
 }
 
 static void btf_free_raw_data(struct btf *btf)
@@ -980,10 +1048,11 @@ void btf__free(struct btf *btf)
 		 * might still have a cached contiguous raw data present,
 		 * which will be unconditionally freed below.
 		 */
-		free(btf->hdr);
 		free(btf->types_data);
 		strset__free(btf->strs_set);
+		free(btf->kind_layout);
 	}
+	free(btf->hdr);
 	btf_free_raw_data(btf);
 	free(btf->raw_data_swapped);
 	free(btf->type_offs);
@@ -994,6 +1063,7 @@ void btf__free(struct btf *btf)
 
 static struct btf *btf_new_empty(struct btf *base_btf)
 {
+	struct btf_header *hdr;
 	struct btf *btf;
 
 	btf = calloc(1, sizeof(*btf));
@@ -1022,14 +1092,21 @@ static struct btf *btf_new_empty(struct btf *base_btf)
 		return ERR_PTR(-ENOMEM);
 	}
 
-	btf->hdr = btf->raw_data;
-	btf->hdr->hdr_len = sizeof(struct btf_header);
-	btf->hdr->magic = BTF_MAGIC;
-	btf->hdr->version = BTF_VERSION;
+	hdr = btf->raw_data;
+	hdr->hdr_len = sizeof(struct btf_header);
+	hdr->magic = BTF_MAGIC;
+	hdr->version = BTF_VERSION;
 
-	btf->types_data = btf->raw_data + btf->hdr->hdr_len;
-	btf->strs_data = btf->raw_data + btf->hdr->hdr_len;
-	btf->hdr->str_len = base_btf ? 0 : 1; /* empty string at offset 0 */
+	btf->types_data = btf->raw_data + hdr->hdr_len;
+	btf->strs_data = btf->raw_data + hdr->hdr_len;
+	hdr->str_len = base_btf ? 0 : 1; /* empty string at offset 0 */
+	btf->hdr = calloc(1, sizeof(struct btf_header));
+	if (!btf->hdr) {
+		free(btf->raw_data);
+		free(btf);
+		return ERR_PTR(-ENOMEM);
+	}
+	memcpy(btf->hdr, hdr, sizeof(*hdr));
 
 	return btf;
 }
@@ -1078,7 +1155,6 @@ static struct btf *btf_new(const void *data, __u32 size, struct btf *base_btf, b
 
 	btf->raw_size = size;
 
-	btf->hdr = btf->raw_data;
 	err = btf_parse_hdr(btf);
 	if (err)
 		goto done;
@@ -1087,6 +1163,7 @@ static struct btf *btf_new(const void *data, __u32 size, struct btf *base_btf, b
 	btf->types_data = btf->raw_data + btf->hdr->hdr_len + btf->hdr->type_off;
 
 	err = btf_parse_str_sec(btf);
+	err = err ?: btf_parse_kind_layout_sec(btf);
 	err = err ?: btf_parse_type_sec(btf);
 	err = err ?: btf_sanity_check(btf);
 	if (err)
@@ -1550,6 +1627,10 @@ static void *btf_get_raw_data(const struct btf *btf, __u32 *size, bool swap_endi
 	}
 
 	data_sz = hdr->hdr_len + hdr->type_len + hdr->str_len;
+	if (btf->kind_layout) {
+		data_sz = roundup(data_sz, 4);
+		data_sz += hdr->kind_layout_len;
+	}
 	data = calloc(1, data_sz);
 	if (!data)
 		return NULL;
@@ -1557,7 +1638,7 @@ static void *btf_get_raw_data(const struct btf *btf, __u32 *size, bool swap_endi
 
 	memcpy(p, hdr, hdr->hdr_len);
 	if (swap_endian)
-		btf_bswap_hdr(p);
+		btf_bswap_hdr(p, hdr->hdr_len);
 	p += hdr->hdr_len;
 
 	memcpy(p, btf->types_data, hdr->type_len);
@@ -1576,7 +1657,11 @@ static void *btf_get_raw_data(const struct btf *btf, __u32 *size, bool swap_endi
 	p += hdr->type_len;
 
 	memcpy(p, btf_strs_data(btf), hdr->str_len);
-	p += hdr->str_len;
+
+	if (btf->kind_layout) {
+		p = data + hdr->hdr_len + hdr->kind_layout_off;
+		memcpy(p, btf->kind_layout, hdr->kind_layout_len);
+	}
 
 	*size = data_sz;
 	return data;
@@ -1717,13 +1802,13 @@ static void btf_invalidate_raw_data(struct btf *btf)
 	}
 }
 
-/* Ensure BTF is ready to be modified (by splitting into a three memory
- * regions for header, types, and strings). Also invalidate cached
- * raw_data, if any.
+/* Ensure BTF is ready to be modified (by splitting into memory regions
+ * for types and strings, with kind layout section if needed (btf->hdr
+ * is already a separate region).  Also invalidate cached raw_data, if any.
  */
 static int btf_ensure_modifiable(struct btf *btf)
 {
-	void *hdr, *types;
+	void *types, *kind_layout = NULL;
 	struct strset *set = NULL;
 	int err = -ENOMEM;
 
@@ -1733,15 +1818,20 @@ static int btf_ensure_modifiable(struct btf *btf)
 		return 0;
 	}
 
-	/* split raw data into three memory regions */
-	hdr = malloc(btf->hdr->hdr_len);
+	/* split raw data into memory regions; btf->hdr is done already. */
 	types = malloc(btf->hdr->type_len);
-	if (!hdr || !types)
+	if (!types)
 		goto err_out;
-
-	memcpy(hdr, btf->hdr, btf->hdr->hdr_len);
 	memcpy(types, btf->types_data, btf->hdr->type_len);
 
+	if (btf->hdr->kind_layout_len) {
+		kind_layout = malloc(btf->hdr->kind_layout_len);
+		if (!kind_layout)
+			goto err_out;
+		memcpy(kind_layout, btf->raw_data + btf->hdr->hdr_len + btf->hdr->kind_layout_off,
+		       btf->hdr->kind_layout_len);
+	}
+
 	/* build lookup index for all strings */
 	set = strset__new(BTF_MAX_STR_OFFSET, btf->strs_data, btf->hdr->str_len);
 	if (IS_ERR(set)) {
@@ -1750,11 +1840,12 @@ static int btf_ensure_modifiable(struct btf *btf)
 	}
 
 	/* only when everything was successful, update internal state */
-	btf->hdr = hdr;
 	btf->types_data = types;
 	btf->types_data_cap = btf->hdr->type_len;
 	btf->strs_data = NULL;
 	btf->strs_set = set;
+	if (kind_layout)
+		btf->kind_layout = kind_layout;
 	/* if BTF was created from scratch, all strings are guaranteed to be
 	 * unique and deduplicated
 	 */
@@ -1766,12 +1857,14 @@ static int btf_ensure_modifiable(struct btf *btf)
 	/* invalidate raw_data representation */
 	btf_invalidate_raw_data(btf);
 
+	btf->modifiable = true;
+
 	return 0;
 
 err_out:
 	strset__free(set);
-	free(hdr);
 	free(types);
+	free(kind_layout);
 	return err;
 }
 
@@ -1840,6 +1933,21 @@ static void btf_type_inc_vlen(struct btf_type *t)
 	t->info = btf_type_info(btf_kind(t), btf_vlen(t) + 1, btf_kflag(t));
 }
 
+static void btf_hdr_update_type_len(struct btf *btf, int new_len)
+{
+	btf->hdr->type_len = new_len;
+	btf->hdr->str_off = new_len;
+	if (btf->kind_layout)
+		btf->hdr->kind_layout_off = btf->hdr->type_len + roundup(btf->hdr->str_len, 4);
+}
+
+static void btf_hdr_update_str_len(struct btf *btf, int new_len)
+{
+	btf->hdr->str_len = new_len;
+	if (btf->kind_layout)
+		btf->hdr->kind_layout_off = btf->hdr->type_len + roundup(new_len, 4);
+}
+
 static int btf_commit_type(struct btf *btf, int data_sz)
 {
 	int err;
@@ -1848,8 +1956,7 @@ static int btf_commit_type(struct btf *btf, int data_sz)
 	if (err)
 		return libbpf_err(err);
 
-	btf->hdr->type_len += data_sz;
-	btf->hdr->str_off += data_sz;
+	btf_hdr_update_type_len(btf, btf->hdr->type_len + data_sz);
 	btf->nr_types++;
 	return btf->start_id + btf->nr_types - 1;
 }
@@ -2029,8 +2136,7 @@ int btf__add_btf(struct btf *btf, const struct btf *src_btf)
 	 * update type count and various internal offsets and sizes to
 	 * "commit" the changes and made them visible to the outside world.
 	 */
-	btf->hdr->type_len += data_sz;
-	btf->hdr->str_off += data_sz;
+	btf_hdr_update_type_len(btf, btf->hdr->type_len + data_sz);
 	btf->nr_types += cnt;
 
 	hashmap__free(p.str_off_map);
@@ -2047,7 +2153,7 @@ int btf__add_btf(struct btf *btf, const struct btf *src_btf)
 	/* and now restore original strings section size; types data size
 	 * wasn't modified, so doesn't need restoring, see big comment above
 	 */
-	btf->hdr->str_len = old_strs_len;
+	btf_hdr_update_str_len(btf, old_strs_len);
 
 	hashmap__free(p.str_off_map);
 
@@ -2365,8 +2471,7 @@ int btf__add_field(struct btf *btf, const char *name, int type_id,
 	/* update parent type's vlen and kflag */
 	t->info = btf_type_info(btf_kind(t), btf_vlen(t) + 1, is_bitfield || btf_kflag(t));
 
-	btf->hdr->type_len += sz;
-	btf->hdr->str_off += sz;
+	btf_hdr_update_type_len(btf, btf->hdr->type_len + sz);
 	return 0;
 }
 
@@ -2475,8 +2580,7 @@ int btf__add_enum_value(struct btf *btf, const char *name, __s64 value)
 	if (value < 0)
 		t->info = btf_type_info(btf_kind(t), btf_vlen(t), true);
 
-	btf->hdr->type_len += sz;
-	btf->hdr->str_off += sz;
+	btf_hdr_update_type_len(btf, btf->hdr->type_len + sz);
 	return 0;
 }
 
@@ -2547,8 +2651,7 @@ int btf__add_enum64_value(struct btf *btf, const char *name, __u64 value)
 	t = btf_last_type(btf);
 	btf_type_inc_vlen(t);
 
-	btf->hdr->type_len += sz;
-	btf->hdr->str_off += sz;
+	btf_hdr_update_type_len(btf, btf->hdr->type_len + sz);
 	return 0;
 }
 
@@ -2786,8 +2889,7 @@ int btf__add_func_param(struct btf *btf, const char *name, int type_id)
 	t = btf_last_type(btf);
 	btf_type_inc_vlen(t);
 
-	btf->hdr->type_len += sz;
-	btf->hdr->str_off += sz;
+	btf_hdr_update_type_len(btf, btf->hdr->type_len + sz);
 	return 0;
 }
 
@@ -2923,8 +3025,7 @@ int btf__add_datasec_var_info(struct btf *btf, int var_type_id, __u32 offset, __
 	t = btf_last_type(btf);
 	btf_type_inc_vlen(t);
 
-	btf->hdr->type_len += sz;
-	btf->hdr->str_off += sz;
+	btf_hdr_update_type_len(btf, btf->hdr->type_len + sz);
 	return 0;
 }
 
@@ -3888,7 +3989,7 @@ static int btf_dedup_strings(struct btf_dedup *d)
 
 	/* replace BTF string data and hash with deduped ones */
 	strset__free(d->btf->strs_set);
-	d->btf->hdr->str_len = strset__data_size(d->strs_set);
+	btf_hdr_update_str_len(d->btf, strset__data_size(d->strs_set));
 	d->btf->strs_set = d->strs_set;
 	d->strs_set = NULL;
 	d->btf->strs_deduped = true;
@@ -5343,6 +5444,11 @@ static int btf_dedup_compact_types(struct btf_dedup *d)
 	d->btf->type_offs = new_offs;
 	d->btf->hdr->str_off = d->btf->hdr->type_len;
 	d->btf->raw_size = d->btf->hdr->hdr_len + d->btf->hdr->type_len + d->btf->hdr->str_len;
+	if (d->btf->kind_layout) {
+		d->btf->hdr->kind_layout_off = d->btf->hdr->str_off + roundup(d->btf->hdr->str_len,
+									      4);
+		d->btf->raw_size = roundup(d->btf->raw_size, 4) + d->btf->hdr->kind_layout_len;
+	}
 	return 0;
 }
 
-- 
2.39.3

[PATCH v8 bpf-next 03/10] libbpf: use kind layout to compute an unknown kind size

[Alan Maguire]

This allows BTF parsing to proceed even if we do not know the
kind.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 tools/lib/bpf/btf.c | 34 +++++++++++++++++++++++++++-------
 1 file changed, 27 insertions(+), 7 deletions(-)

diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
index 8835aee6ee84..3936ee04a46a 100644
--- a/tools/lib/bpf/btf.c
+++ b/tools/lib/bpf/btf.c
@@ -359,7 +359,28 @@ static int btf_parse_kind_layout_sec(struct btf *btf)
 	return 0;
 }
 
-static int btf_type_size(const struct btf_type *t)
+/* for unknown kinds, consult kind layout. */
+static int btf_type_size_unknown(const struct btf *btf, const struct btf_type *t)
+{
+	int size = sizeof(struct btf_type);
+	struct btf_kind_layout *k = NULL;
+	__u16 vlen = btf_vlen(t);
+	__u8 kind = btf_kind(t);
+	__u32 off = kind * sizeof(struct btf_kind_layout);
+
+	if (!btf->kind_layout || off >= btf->hdr->kind_layout_len) {
+		pr_debug("Unsupported BTF_KIND: %u\n", btf_kind(t));
+		return -EINVAL;
+	}
+	k = btf->kind_layout + off;
+
+	size += k->info_sz;
+	size += vlen * k->elem_sz;
+
+	return size;
+}
+
+static int btf_type_size(const struct btf *btf, const struct btf_type *t)
 {
 	const int base_size = sizeof(struct btf_type);
 	__u16 vlen = btf_vlen(t);
@@ -395,8 +416,7 @@ static int btf_type_size(const struct btf_type *t)
 	case BTF_KIND_DECL_TAG:
 		return base_size + sizeof(struct btf_decl_tag);
 	default:
-		pr_debug("Unsupported BTF_KIND:%u\n", btf_kind(t));
-		return -EINVAL;
+		return btf_type_size_unknown(btf, t);
 	}
 }
 
@@ -495,7 +515,7 @@ static int btf_parse_type_sec(struct btf *btf)
 		if (btf->swapped_endian)
 			btf_bswap_type_base(next_type);
 
-		type_size = btf_type_size(next_type);
+		type_size = btf_type_size(btf, next_type);
 		if (type_size < 0)
 			return type_size;
 		if (next_type + type_size > end_type) {
@@ -2005,7 +2025,7 @@ static int btf_add_type(struct btf_pipe *p, const struct btf_type *src_type)
 	__u32 *str_off;
 	int sz, err;
 
-	sz = btf_type_size(src_type);
+	sz = btf_type_size(p->src, src_type);
 	if (sz < 0)
 		return libbpf_err(sz);
 
@@ -2087,7 +2107,7 @@ int btf__add_btf(struct btf *btf, const struct btf *src_btf)
 		struct btf_field_iter it;
 		__u32 *type_id, *str_off;
 
-		sz = btf_type_size(t);
+		sz = btf_type_size(src_btf, t);
 		if (sz < 0) {
 			/* unlikely, has to be corrupted src_btf */
 			err = sz;
@@ -5422,7 +5442,7 @@ static int btf_dedup_compact_types(struct btf_dedup *d)
 			continue;
 
 		t = btf__type_by_id(d->btf, id);
-		len = btf_type_size(t);
+		len = btf_type_size(d->btf, t);
 		if (len < 0)
 			return len;
 
-- 
2.39.3

[PATCH v8 bpf-next 04/10] libbpf: Add kind layout encoding support

[Alan Maguire]

Support encoding of BTF kind layout data via btf__new_empty_opts().

Current supported opts are base_btf and add_kind_layout.

Kind layout information is maintained in btf.c in the
kind_layouts[] array; when BTF is created with the
add_kind_layout option it represents the current view
of supported BTF kinds.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 tools/lib/bpf/btf.c      | 60 ++++++++++++++++++++++++++++++++++++++--
 tools/lib/bpf/btf.h      | 20 ++++++++++++++
 tools/lib/bpf/libbpf.map |  1 +
 3 files changed, 78 insertions(+), 3 deletions(-)

diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
index 3936ee04a46a..589a9632a630 100644
--- a/tools/lib/bpf/btf.c
+++ b/tools/lib/bpf/btf.c
@@ -29,6 +29,35 @@
 
 static struct btf_type btf_void;
 
+/* Describe how kinds are laid out; some have a singular element following the "struct btf_type",
+ * some have BTF_INFO_VLEN(t->info) elements.  Specify sizes for both.  Flags are currently unused.
+ * Kind layout can be optionally added to the BTF representation in a dedicated section to
+ * facilitate parsing.  New kinds must be added here.
+ */
+struct btf_kind_layout kind_layouts[NR_BTF_KINDS] = {
+/*	singular element size		vlen element(s) size */
+{	0,				0				}, /* _UNKN */
+{	sizeof(__u32),			0				}, /* _INT */
+{	0,				0				}, /* _PTR */
+{	sizeof(struct btf_array),	0				}, /* _ARRAY */
+{	0,				sizeof(struct btf_member)	}, /* _STRUCT */
+{	0,				sizeof(struct btf_member)	}, /* _UNION */
+{	0,				sizeof(struct btf_enum)		}, /* _ENUM */
+{	0,				0				}, /* _FWD */
+{	0,				0				}, /* _TYPEDEF */
+{	0,				0				}, /* _VOLATILE */
+{	0,				0				}, /* _CONST */
+{	0,				0				}, /* _RESTRICT */
+{	0,				0				}, /* _FUNC */
+{	0,				sizeof(struct btf_param)	}, /* _FUNC_PROTO */
+{	sizeof(struct btf_var),		0				}, /* _VAR */
+{	0,				sizeof(struct btf_var_secinfo)	}, /* _DATASEC */
+{	0,				0				}, /* _FLOAT */
+{	sizeof(struct btf_decl_tag),	0				}, /* _DECL_TAG */
+{	0,				0				}, /* _TYPE_TAG */
+{	0,				sizeof(struct btf_enum64)	}, /* _ENUM64 */
+};
+
 struct btf {
 	/* raw BTF data in native endianness */
 	void *raw_data;
@@ -1081,8 +1110,10 @@ void btf__free(struct btf *btf)
 	free(btf);
 }
 
-static struct btf *btf_new_empty(struct btf *base_btf)
+static struct btf *btf_new_empty(struct btf_new_opts *opts)
 {
+	bool add_kind_layout = OPTS_GET(opts, add_kind_layout, false);
+	struct btf *base_btf = OPTS_GET(opts, base_btf, NULL);
 	struct btf_header *hdr;
 	struct btf *btf;
 
@@ -1106,6 +1137,8 @@ static struct btf *btf_new_empty(struct btf *base_btf)
 
 	/* +1 for empty string at offset 0 */
 	btf->raw_size = sizeof(struct btf_header) + (base_btf ? 0 : 1);
+	if (add_kind_layout)
+		btf->raw_size = roundup(btf->raw_size, 4) + sizeof(kind_layouts);
 	btf->raw_data = calloc(1, btf->raw_size);
 	if (!btf->raw_data) {
 		free(btf);
@@ -1126,6 +1159,13 @@ static struct btf *btf_new_empty(struct btf *base_btf)
 		free(btf);
 		return ERR_PTR(-ENOMEM);
 	}
+
+	if (add_kind_layout) {
+		hdr->kind_layout_len = sizeof(kind_layouts);
+		hdr->kind_layout_off = roundup(hdr->str_len, 4);
+		btf->kind_layout = btf->raw_data + hdr->hdr_len + hdr->kind_layout_off;
+		memcpy(btf->kind_layout, kind_layouts, sizeof(kind_layouts));
+	}
 	memcpy(btf->hdr, hdr, sizeof(*hdr));
 
 	return btf;
@@ -1133,12 +1173,26 @@ static struct btf *btf_new_empty(struct btf *base_btf)
 
 struct btf *btf__new_empty(void)
 {
-	return libbpf_ptr(btf_new_empty(NULL));
+	LIBBPF_OPTS(btf_new_opts, opts);
+
+	return libbpf_ptr(btf_new_empty(&opts));
 }
 
 struct btf *btf__new_empty_split(struct btf *base_btf)
 {
-	return libbpf_ptr(btf_new_empty(base_btf));
+	LIBBPF_OPTS(btf_new_opts, opts);
+
+	opts.base_btf = base_btf;
+
+	return libbpf_ptr(btf_new_empty(&opts));
+}
+
+struct btf *btf__new_empty_opts(struct btf_new_opts *opts)
+{
+	if (!OPTS_VALID(opts, btf_new_opts))
+		return libbpf_err_ptr(-EINVAL);
+
+	return libbpf_ptr(btf_new_empty(opts));
 }
 
 static struct btf *btf_new(const void *data, __u32 size, struct btf *base_btf, bool is_mmap)
diff --git a/tools/lib/bpf/btf.h b/tools/lib/bpf/btf.h
index cc01494d6210..dcc166834937 100644
--- a/tools/lib/bpf/btf.h
+++ b/tools/lib/bpf/btf.h
@@ -109,6 +109,26 @@ LIBBPF_API struct btf *btf__new_empty(void);
  */
 LIBBPF_API struct btf *btf__new_empty_split(struct btf *base_btf);
 
+struct btf_new_opts {
+	size_t sz;
+	struct btf *base_btf;	/* optional base BTF */
+	bool add_kind_layout;	/* add BTF kind layout information */
+	size_t:0;
+};
+#define btf_new_opts__last_field add_kind_layout
+
+/**
+ * @brief **btf__new_empty_opts()** creates an unpopulated BTF object with
+ * optional *base_btf* and BTF kind layout description if *add_kind_layout*
+ * is set
+ * @return new BTF object instance which has to be eventually freed with
+ * **btf__free()**
+ *
+ * On error, NULL is returned and the thread-local `errno` variable is
+ * set to the error code.
+ */
+LIBBPF_API struct btf *btf__new_empty_opts(struct btf_new_opts *opts);
+
 /**
  * @brief **btf__distill_base()** creates new versions of the split BTF
  * *src_btf* and its base BTF. The new base BTF will only contain the types
diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map
index 84fb90a016c9..0fb9a1f70e72 100644
--- a/tools/lib/bpf/libbpf.map
+++ b/tools/lib/bpf/libbpf.map
@@ -453,4 +453,5 @@ LIBBPF_1.7.0 {
 		bpf_map__exclusive_program;
 		bpf_prog_assoc_struct_ops;
 		bpf_program__assoc_struct_ops;
+		btf__new_empty_opts;
 } LIBBPF_1.6.0;
-- 
2.39.3

[PATCH v8 bpf-next 05/10] libbpf: BTF validation can use kind layout for unknown kinds

[Alan Maguire]

BTF parsing can use kind layout to navigate unknown kinds, so
btf_validate_type() should take kind layout information into
account to avoid failure when an unrecognized kind is met.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 tools/lib/bpf/btf.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
index 589a9632a630..6098c8d1e26a 100644
--- a/tools/lib/bpf/btf.c
+++ b/tools/lib/bpf/btf.c
@@ -705,8 +705,12 @@ static int btf_validate_type(const struct btf *btf, const struct btf_type *t, __
 		break;
 	}
 	default:
-		pr_warn("btf: type [%u]: unrecognized kind %u\n", id, kind);
-		return -EINVAL;
+		/* Kind may be represented in kind layout information. */
+		if (btf_type_size_unknown(btf, t) < 0) {
+			pr_warn("btf: type [%u]: unrecognized kind %u\n", id, kind);
+			return -EINVAL;
+		}
+		break;
 	}
 	return 0;
 }
-- 
2.39.3

[PATCH v8 bpf-next 06/10] btf: support kernel parsing of BTF with kind layout

[Alan Maguire]

Validate kind layout if present, but because the kernel must be
strict in what it accepts, reject BTF with unsupported kinds,
even if they are in the kind layout information.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 kernel/bpf/btf.c | 96 ++++++++++++++++++++++++++++++++++++++----------
 1 file changed, 76 insertions(+), 20 deletions(-)

diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index 0de8fc8a0e0b..79ecc13decb3 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -268,6 +268,7 @@ struct btf {
 	struct btf_id_dtor_kfunc_tab *dtor_kfunc_tab;
 	struct btf_struct_metas *struct_meta_tab;
 	struct btf_struct_ops_tab *struct_ops_tab;
+	struct btf_kind_layout *kind_layout;
 
 	/* split BTF support */
 	struct btf *base_btf;
@@ -5215,23 +5216,36 @@ static s32 btf_check_meta(struct btf_verifier_env *env,
 		return -EINVAL;
 	}
 
-	if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX ||
-	    BTF_INFO_KIND(t->info) == BTF_KIND_UNKN) {
+	if (!btf_name_offset_valid(env->btf, t->name_off)) {
+		btf_verifier_log(env, "[%u] Invalid name_offset:%u",
+				 env->log_type_id, t->name_off);
+		return -EINVAL;
+	}
+
+	if (BTF_INFO_KIND(t->info) == BTF_KIND_UNKN) {
 		btf_verifier_log(env, "[%u] Invalid kind:%u",
 				 env->log_type_id, BTF_INFO_KIND(t->info));
 		return -EINVAL;
 	}
 
-	if (!btf_name_offset_valid(env->btf, t->name_off)) {
-		btf_verifier_log(env, "[%u] Invalid name_offset:%u",
-				 env->log_type_id, t->name_off);
+	if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX && env->btf->kind_layout &&
+	    ((BTF_INFO_KIND(t->info) + 1) * sizeof(struct btf_kind_layout)) <
+	     env->btf->hdr.kind_layout_len) {
+		btf_verifier_log(env, "[%u] unknown but required kind %u",
+				 env->log_type_id,
+				 BTF_INFO_KIND(t->info));
 		return -EINVAL;
+	} else {
+		if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX) {
+			btf_verifier_log(env, "[%u] Invalid kind:%u",
+					 env->log_type_id, BTF_INFO_KIND(t->info));
+			return -EINVAL;
+		}
+		var_meta_size = btf_type_ops(t)->check_meta(env, t, meta_left);
+		if (var_meta_size < 0)
+			return var_meta_size;
 	}
 
-	var_meta_size = btf_type_ops(t)->check_meta(env, t, meta_left);
-	if (var_meta_size < 0)
-		return var_meta_size;
-
 	meta_left -= var_meta_size;
 
 	return saved_meta_left - meta_left;
@@ -5405,7 +5419,8 @@ static int btf_parse_str_sec(struct btf_verifier_env *env)
 	start = btf->nohdr_data + hdr->str_off;
 	end = start + hdr->str_len;
 
-	if (end != btf->data + btf->data_size) {
+	if (hdr->hdr_len < sizeof(struct btf_header) &&
+	    end != btf->data + btf->data_size) {
 		btf_verifier_log(env, "String section is not at the end");
 		return -EINVAL;
 	}
@@ -5426,9 +5441,41 @@ static int btf_parse_str_sec(struct btf_verifier_env *env)
 	return 0;
 }
 
+static int btf_parse_kind_layout_sec(struct btf_verifier_env *env)
+{
+	const struct btf_header *hdr = &env->btf->hdr;
+	struct btf *btf = env->btf;
+	void *start, *end;
+
+	if (hdr->hdr_len < sizeof(struct btf_header) ||
+	    hdr->kind_layout_len == 0)
+		return 0;
+
+	/* Kind layout section must align to 4 bytes */
+	if (hdr->kind_layout_off & (sizeof(u32) - 1)) {
+		btf_verifier_log(env, "Unaligned kind_layout_off");
+		return -EINVAL;
+	}
+	start = btf->nohdr_data + hdr->kind_layout_off;
+	end = start + hdr->kind_layout_len;
+
+	if (hdr->kind_layout_len < sizeof(struct btf_kind_layout)) {
+		btf_verifier_log(env, "Kind layout section is too small");
+		return -EINVAL;
+	}
+	if (end > btf->data + btf->data_size) {
+		btf_verifier_log(env, "Kind layout section is too big");
+		return -EINVAL;
+	}
+	btf->kind_layout = start;
+
+	return 0;
+}
+
 static const size_t btf_sec_info_offset[] = {
 	offsetof(struct btf_header, type_off),
 	offsetof(struct btf_header, str_off),
+	offsetof(struct btf_header, kind_layout_off),
 };
 
 static int btf_sec_info_cmp(const void *a, const void *b)
@@ -5443,44 +5490,49 @@ static int btf_check_sec_info(struct btf_verifier_env *env,
 			      u32 btf_data_size)
 {
 	struct btf_sec_info secs[ARRAY_SIZE(btf_sec_info_offset)];
-	u32 total, expected_total, i;
+	u32 nr_secs = ARRAY_SIZE(btf_sec_info_offset);
+	u32 total, expected_total, gap, i;
 	const struct btf_header *hdr;
 	const struct btf *btf;
 
 	btf = env->btf;
 	hdr = &btf->hdr;
 
+	if (hdr->hdr_len < sizeof(struct btf_header))
+		nr_secs--;
+
 	/* Populate the secs from hdr */
-	for (i = 0; i < ARRAY_SIZE(btf_sec_info_offset); i++)
+	for (i = 0; i < nr_secs; i++)
 		secs[i] = *(struct btf_sec_info *)((void *)hdr +
 						   btf_sec_info_offset[i]);
 
-	sort(secs, ARRAY_SIZE(btf_sec_info_offset),
+	sort(secs, nr_secs,
 	     sizeof(struct btf_sec_info), btf_sec_info_cmp, NULL);
 
 	/* Check for gaps and overlap among sections */
 	total = 0;
 	expected_total = btf_data_size - hdr->hdr_len;
-	for (i = 0; i < ARRAY_SIZE(btf_sec_info_offset); i++) {
+	for (i = 0; i < nr_secs; i++) {
 		if (expected_total < secs[i].off) {
 			btf_verifier_log(env, "Invalid section offset");
 			return -EINVAL;
 		}
-		if (total < secs[i].off) {
-			/* gap */
-			btf_verifier_log(env, "Unsupported section found");
-			return -EINVAL;
-		}
 		if (total > secs[i].off) {
 			btf_verifier_log(env, "Section overlap found");
 			return -EINVAL;
 		}
+		gap = secs[i].off - total;
+		if (gap >= 4) {
+			/* gap larger than alignment gap */
+			btf_verifier_log(env, "Unsupported section found");
+			return -EINVAL;
+		}
 		if (expected_total - total < secs[i].len) {
 			btf_verifier_log(env,
 					 "Total section length too long");
 			return -EINVAL;
 		}
-		total += secs[i].len;
+		total += secs[i].len + gap;
 	}
 
 	/* There is data other than hdr and known sections */
@@ -5816,6 +5868,10 @@ static struct btf *btf_parse(const union bpf_attr *attr, bpfptr_t uattr, u32 uat
 	if (err)
 		goto errout;
 
+	err = btf_parse_kind_layout_sec(env);
+	if (err)
+		goto errout;
+
 	err = btf_parse_type_sec(env);
 	if (err)
 		goto errout;
-- 
2.39.3

[PATCH v8 bpf-next 07/10] selftests/bpf: test kind encoding/decoding

[Alan Maguire]

verify btf__new_empty_opts() adds kind layouts for all kinds supported,
and after adding kind-related types for an unknown kind, ensure that
parsing uses this info when that kind is encountered rather than
giving up.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 .../selftests/bpf/prog_tests/btf_kind.c       | 178 ++++++++++++++++++
 1 file changed, 178 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/btf_kind.c

diff --git a/tools/testing/selftests/bpf/prog_tests/btf_kind.c b/tools/testing/selftests/bpf/prog_tests/btf_kind.c
new file mode 100644
index 000000000000..322f207873c2
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/btf_kind.c
@@ -0,0 +1,178 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2025, Oracle and/or its affiliates. */
+
+#include <test_progs.h>
+#include <bpf/btf.h>
+#include <bpf/libbpf.h>
+
+/* verify kind encoding exists for each kind */
+static void test_btf_kind_encoding(void)
+{
+	LIBBPF_OPTS(btf_new_opts, opts);
+	const struct btf_header *hdr;
+	const void *raw_btf;
+	struct btf *btf;
+	__u32 raw_size;
+
+	opts.add_kind_layout = true;
+	btf = btf__new_empty_opts(&opts);
+	if (!ASSERT_OK_PTR(btf, "btf_new"))
+		return;
+
+	raw_btf = btf__raw_data(btf, &raw_size);
+	if (!ASSERT_OK_PTR(raw_btf, "btf__raw_data"))
+		return;
+
+	hdr = raw_btf;
+
+	ASSERT_EQ(hdr->kind_layout_off % 4, 0, "kind_layout aligned");
+	ASSERT_EQ(hdr->kind_layout_len, sizeof(struct btf_kind_layout) * NR_BTF_KINDS,
+		  "kind_layout_len");
+	btf__free(btf);
+}
+
+static void write_raw_btf(const char *btf_path, void *raw_btf, size_t raw_size)
+{
+	int fd = open(btf_path, O_WRONLY | O_CREAT);
+
+	write(fd, raw_btf, raw_size);
+	close(fd);
+}
+
+/* fabricate an unrecognized kind at BTF_KIND_MAX + 1, and after adding
+ * the appropriate struct/typedefs to the BTF such that it recognizes
+ * this kind, ensure that parsing of BTF containing the unrecognized kind
+ * can succeed.
+ */
+void test_btf_kind_decoding(void)
+{
+	__s32 int_id, unrec_id, id, id2;
+	LIBBPF_OPTS(btf_new_opts, opts);
+	struct btf *btf, *new_btf;
+	struct btf_kind_layout *k;
+	struct btf_header *hdr;
+	const void *raw_btf;
+	struct btf_type *t;
+	char btf_path[64];
+	void *new_raw_btf;
+	__u32 raw_size;
+
+	opts.add_kind_layout = true;
+	btf = btf__new_empty_opts(&opts);
+	if (!ASSERT_OK_PTR(btf, "btf_new"))
+		return;
+
+	int_id = btf__add_int(btf, "test_char", 1, BTF_INT_CHAR);
+	if (!ASSERT_GT(int_id, 0, "add_int_id"))
+		return;
+
+	/* now create our type with unrecognized kind by adding a typedef kind
+	 * we will overwrite it with our unrecognized kind value.
+	 */
+	unrec_id = btf__add_typedef(btf, "unrec_kind", int_id);
+	if (!ASSERT_GT(unrec_id, 0, "add_unrec_id"))
+		return;
+
+	/* add an id after it that we will look up to verify we can parse
+	 * beyond unrecognized kinds.
+	 */
+	id = btf__add_typedef(btf, "test_lookup", int_id);
+	if (!ASSERT_GT(id, 0, "add_test_lookup_id"))
+		return;
+	id2 = btf__add_typedef(btf, "test_lookup2", int_id);
+	if (!ASSERT_GT(id2, 0, "add_test_lookup_id2"))
+		return;
+
+	raw_btf = (void *)btf__raw_data(btf, &raw_size);
+	if (!ASSERT_OK_PTR(raw_btf, "btf__raw_data"))
+		return;
+
+	new_raw_btf = calloc(1, raw_size + sizeof(*k));
+	if (!ASSERT_OK_PTR(new_raw_btf, "calloc_raw_btf"))
+		return;
+	memcpy(new_raw_btf, raw_btf, raw_size);
+
+	/* add new layout description */
+	hdr = new_raw_btf;
+	hdr->kind_layout_len += sizeof(*k);
+	k = new_raw_btf + hdr->hdr_len + hdr->kind_layout_off;
+	k[NR_BTF_KINDS].info_sz = 0;
+	k[NR_BTF_KINDS].elem_sz = 0;
+
+	/* now modify our typedef added above to be an unrecognized kind. */
+	t = (void *)hdr + hdr->hdr_len + hdr->type_off + sizeof(struct btf_type) +
+		sizeof(__u32);
+	t->info = (NR_BTF_KINDS << 24);
+
+	/* now write our BTF to a raw file, ready for parsing. */
+	snprintf(btf_path, sizeof(btf_path), "/tmp/btf_kind.%d", getpid());
+
+	write_raw_btf(btf_path, new_raw_btf, raw_size + sizeof(*k));
+
+	/* verify parsing succeeds, and that we can read type info past
+	 * the unrecognized kind.
+	 */
+	new_btf = btf__parse_raw(btf_path);
+	if (ASSERT_OK_PTR(new_btf, "btf__parse_raw")) {
+		ASSERT_EQ(btf__find_by_name_kind(new_btf, "test_lookup",
+						 BTF_KIND_TYPEDEF), id,
+			  "verify_id_lookup");
+		ASSERT_EQ(btf__find_by_name_kind(new_btf, "test_lookup2",
+						 BTF_KIND_TYPEDEF), id2,
+			  "verify_id2_lookup");
+	}
+	btf__free(new_btf);
+
+	/* next, change info_sz to equal sizeof(struct btf_type); this means the
+	 * "test_lookup" kind will be reinterpreted as a singular info element
+	 * following the unrecognized kind.
+	 */
+	k[NR_BTF_KINDS].info_sz = sizeof(struct btf_type);
+	write_raw_btf(btf_path, new_raw_btf, raw_size + sizeof(*k));
+
+	new_btf = btf__parse_raw(btf_path);
+	if (ASSERT_OK_PTR(new_btf, "btf__parse_raw")) {
+		ASSERT_EQ(btf__find_by_name_kind(new_btf, "test_lookup",
+						 BTF_KIND_TYPEDEF), -ENOENT,
+			  "verify_id_not_found");
+		/* id of "test_lookup2" will be id2 -1 as we have removed one type */
+		ASSERT_EQ(btf__find_by_name_kind(new_btf, "test_lookup2",
+						 BTF_KIND_TYPEDEF), id2 - 1,
+			  "verify_id_lookup2");
+
+	}
+	btf__free(new_btf);
+
+	/* next, change elem_sz to equal sizeof(struct btf_type)/2 and set
+	 * vlen associated with unrecognized type to 2; this allows us to verify
+	 * vlen-specified BTF can still be parsed.
+	 */
+	k[NR_BTF_KINDS].info_sz = 0;
+	k[NR_BTF_KINDS].elem_sz = sizeof(struct btf_type)/2;
+	t->info |= 2;
+	write_raw_btf(btf_path, new_raw_btf, raw_size + sizeof(*k));
+
+	new_btf = btf__parse_raw(btf_path);
+	if (ASSERT_OK_PTR(new_btf, "btf__parse_raw")) {
+		ASSERT_EQ(btf__find_by_name_kind(new_btf, "test_lookup",
+						 BTF_KIND_TYPEDEF), -ENOENT,
+			  "verify_id_not_found");
+		/* id of "test_lookup2" will be id2 -1 as we have removed one type */
+		ASSERT_EQ(btf__find_by_name_kind(new_btf, "test_lookup2",
+						 BTF_KIND_TYPEDEF), id2 - 1,
+			  "verify_id_lookup2");
+
+	}
+	btf__free(new_btf);
+	free(new_raw_btf);
+	unlink(btf_path);
+	btf__free(btf);
+}
+
+void test_btf_kind(void)
+{
+	if (test__start_subtest("btf_kind_encoding"))
+		test_btf_kind_encoding();
+	if (test__start_subtest("btf_kind_decoding"))
+		test_btf_kind_decoding();
+}
-- 
2.39.3

[PATCH v8 bpf-next 08/10] bpftool: add BTF dump "format meta" to dump header/metadata

[Alan Maguire]

Provide a way to dump BTF metadata info via bpftool; this
consists of BTF size, header fields and kind layout info
(if available); for example

$ bpftool btf dump file vmlinux format meta
size 5460436
magic 0xeb9f
version 1
flags 0x0
hdr_len 32
type_len 3194640
type_off 0
str_len 2265721
str_off 3194640
kind_layout_len 40
kind_layout_off 5460364
kind 0    UNKNOWN    info_sz 0    elem_sz 0
kind 1    INT        info_sz 4    elem_sz 0
kind 2    PTR        info_sz 0    elem_sz 0
kind 3    ARRAY      info_sz 12   elem_sz 0
kind 4    STRUCT     info_sz 0    elem_sz 12
kind 5    UNION      info_sz 0    elem_sz 12
kind 6    ENUM       info_sz 0    elem_sz 8
kind 7    FWD        info_sz 0    elem_sz 0
kind 8    TYPEDEF    info_sz 0    elem_sz 0
kind 9    VOLATILE   info_sz 0    elem_sz 0
kind 10   CONST      info_sz 0    elem_sz 0
kind 11   RESTRICT   info_sz 0    elem_sz 0
kind 12   FUNC       info_sz 0    elem_sz 0
kind 13   FUNC_PROTO info_sz 0    elem_sz 8
kind 14   VAR        info_sz 4    elem_sz 0
kind 15   DATASEC    info_sz 0    elem_sz 12
kind 16   FLOAT      info_sz 0    elem_sz 0
kind 17   DECL_TAG   info_sz 4    elem_sz 0
kind 18   TYPE_TAG   info_sz 0    elem_sz 0
kind 19   ENUM64     info_sz 0    elem_sz 12

JSON output is also supported:

$ $ bpftool -p btf dump file /var/tmp/vmlinux.kind_layout format meta
{
    "size": 5460436,
    "header": {
        "magic": 60319,
        "version": 1,
        "flags": 0,
        "hdr_len": 32,
        "type_len": 3194640,
        "type_off": 0,
        "str_len": 2265721,
        "str_off": 3194640,
        "kind_layout_len": 40,
        "kind_layout_offset": 5460364
    },
    "kind_layouts": [{
            "kind": 0,
            "name": "UNKNOWN",
            "info_sz": 0,
            "elem_sz": 0
        },{
            "kind": 1,
            "name": "INT",
            "info_sz": 4,
            "elem_sz": 0
        },{
...

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 tools/bpf/bpftool/bash-completion/bpftool |  2 +-
 tools/bpf/bpftool/btf.c                   | 94 ++++++++++++++++++++++-
 2 files changed, 92 insertions(+), 4 deletions(-)

diff --git a/tools/bpf/bpftool/bash-completion/bpftool b/tools/bpf/bpftool/bash-completion/bpftool
index 53bcfeb1a76e..a331172cf8de 100644
--- a/tools/bpf/bpftool/bash-completion/bpftool
+++ b/tools/bpf/bpftool/bash-completion/bpftool
@@ -950,7 +950,7 @@ _bpftool()
                             return 0
                             ;;
                         format)
-                            COMPREPLY=( $( compgen -W "c raw" -- "$cur" ) )
+                            COMPREPLY=( $( compgen -W "c raw meta" -- "$cur" ) )
                             ;;
                         root_id)
                             return 0;
diff --git a/tools/bpf/bpftool/btf.c b/tools/bpf/bpftool/btf.c
index 946612029dee..fdb9b36fe106 100644
--- a/tools/bpf/bpftool/btf.c
+++ b/tools/bpf/bpftool/btf.c
@@ -835,6 +835,90 @@ static int dump_btf_c(const struct btf *btf,
 	return err;
 }
 
+static int dump_btf_meta(const struct btf *btf)
+{
+	const struct btf_header *hdr;
+	const struct btf_kind_layout *k;
+	__u8 i, nr_kinds = 0;
+	const void *data;
+	__u32 data_sz;
+
+	data = btf__raw_data(btf, &data_sz);
+	if (!data)
+		return -ENOMEM;
+	hdr = data;
+	if (json_output) {
+		jsonw_start_object(json_wtr);		/* metadata object */
+		jsonw_uint_field(json_wtr, "size", data_sz);
+		jsonw_name(json_wtr, "header");
+		jsonw_start_object(json_wtr);		/* header object */
+		jsonw_uint_field(json_wtr, "magic", hdr->magic);
+		jsonw_uint_field(json_wtr, "version", hdr->version);
+		jsonw_uint_field(json_wtr, "flags", hdr->flags);
+		jsonw_uint_field(json_wtr, "hdr_len", hdr->hdr_len);
+		jsonw_uint_field(json_wtr, "type_len", hdr->type_len);
+		jsonw_uint_field(json_wtr, "type_off", hdr->type_off);
+		jsonw_uint_field(json_wtr, "str_len", hdr->str_len);
+		jsonw_uint_field(json_wtr, "str_off", hdr->str_off);
+	} else {
+		printf("size %-10u\n", data_sz);
+		printf("magic 0x%-10x\nversion %-10d\nflags 0x%-10x\nhdr_len %-10u\n",
+		       hdr->magic, hdr->version, hdr->flags, hdr->hdr_len);
+		printf("type_len %-10u\ntype_off %-10u\n", hdr->type_len, hdr->type_off);
+		printf("str_len %-10u\nstr_off %-10u\n", hdr->str_len, hdr->str_off);
+	}
+
+	if (hdr->hdr_len < sizeof(struct btf_header)) {
+		if (json_output) {
+			jsonw_end_object(json_wtr);	/* end header object */
+			jsonw_end_object(json_wtr);	/* end metadata object */
+		}
+		return 0;
+	}
+
+	data_sz -= hdr->hdr_len;
+
+	if (hdr->kind_layout_len > 0 && hdr->kind_layout_off > 0) {
+		if (hdr->kind_layout_off + hdr->kind_layout_len <= data_sz) {
+			k = (void *)hdr + hdr->hdr_len + hdr->kind_layout_off;
+			nr_kinds = hdr->kind_layout_len / sizeof(*k);
+		}
+	}
+	if (json_output) {
+		jsonw_uint_field(json_wtr, "kind_layout_len", hdr->kind_layout_len);
+		jsonw_uint_field(json_wtr, "kind_layout_offset", hdr->kind_layout_off);
+		jsonw_end_object(json_wtr);		/* end header object */
+
+		if (nr_kinds > 0) {
+			jsonw_name(json_wtr, "kind_layouts");
+			jsonw_start_array(json_wtr);
+			for (i = 0; i < nr_kinds; i++) {
+				jsonw_start_object(json_wtr);
+				jsonw_uint_field(json_wtr, "kind", i);
+				if (i < NR_BTF_KINDS)
+					jsonw_string_field(json_wtr, "name", btf_kind_str[i]);
+				else
+					jsonw_null_field(json_wtr, "name");
+				jsonw_uint_field(json_wtr, "info_sz", k[i].info_sz);
+				jsonw_uint_field(json_wtr, "elem_sz", k[i].elem_sz);
+				jsonw_end_object(json_wtr);
+			}
+			jsonw_end_array(json_wtr);
+		}
+		jsonw_end_object(json_wtr);		/* end metadata object */
+	} else {
+		printf("kind_layout_len %-10u\nkind_layout_off %-10u\n",
+		       hdr->kind_layout_len, hdr->kind_layout_off);
+		for (i = 0; i < nr_kinds; i++) {
+			printf("kind %-4d %-10s info_sz %-4d elem_sz %-4d\n",
+			       i, i < NR_BTF_KINDS ? btf_kind_str[i] : "?",
+			       k[i].info_sz, k[i].elem_sz);
+		}
+	}
+
+	return 0;
+}
+
 static const char sysfs_vmlinux[] = "/sys/kernel/btf/vmlinux";
 
 static struct btf *get_vmlinux_btf_from_sysfs(void)
@@ -880,7 +964,7 @@ static bool btf_is_kernel_module(__u32 btf_id)
 
 static int do_dump(int argc, char **argv)
 {
-	bool dump_c = false, sort_dump_c = true;
+	bool dump_c = false, sort_dump_c = true, dump_meta = false;
 	struct btf *btf = NULL, *base = NULL;
 	__u32 root_type_ids[MAX_ROOT_IDS];
 	bool have_id_filtering;
@@ -990,10 +1074,12 @@ static int do_dump(int argc, char **argv)
 			}
 			if (strcmp(*argv, "c") == 0) {
 				dump_c = true;
+			} else if (is_prefix(*argv, "meta")) {
+				dump_meta = true;
 			} else if (strcmp(*argv, "raw") == 0) {
 				dump_c = false;
 			} else {
-				p_err("unrecognized format specifier: '%s', possible values: raw, c",
+				p_err("unrecognized format specifier: '%s', possible values: raw, c, meta",
 				      *argv);
 				err = -EINVAL;
 				goto done;
@@ -1072,6 +1158,8 @@ static int do_dump(int argc, char **argv)
 			goto done;
 		}
 		err = dump_btf_c(btf, root_type_ids, root_type_cnt, sort_dump_c);
+	} else if (dump_meta) {
+		err = dump_btf_meta(btf);
 	} else {
 		err = dump_btf_raw(btf, root_type_ids, root_type_cnt);
 	}
@@ -1446,7 +1534,7 @@ static int do_help(int argc, char **argv)
 		"       %1$s %2$s help\n"
 		"\n"
 		"       BTF_SRC := { id BTF_ID | prog PROG | map MAP [{key | value | kv | all}] | file FILE }\n"
-		"       FORMAT  := { raw | c [unsorted] }\n"
+		"       FORMAT  := { raw | c [unsorted] | meta }\n"
 		"       " HELP_SPEC_MAP "\n"
 		"       " HELP_SPEC_PROGRAM "\n"
 		"       " HELP_SPEC_OPTIONS " |\n"
-- 
2.39.3

[PATCH v8 bpf-next 09/10] bpftool: Update doc to describe bpftool btf dump .. format metadata

[Alan Maguire]

...and provide an example of output.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 .../bpf/bpftool/Documentation/bpftool-btf.rst | 28 +++++++++++++++++--
 1 file changed, 26 insertions(+), 2 deletions(-)

diff --git a/tools/bpf/bpftool/Documentation/bpftool-btf.rst b/tools/bpf/bpftool/Documentation/bpftool-btf.rst
index d47dddc2b4ee..1c11b5647ab7 100644
--- a/tools/bpf/bpftool/Documentation/bpftool-btf.rst
+++ b/tools/bpf/bpftool/Documentation/bpftool-btf.rst
@@ -28,7 +28,7 @@ BTF COMMANDS
 | **bpftool** **btf help**
 |
 | *BTF_SRC* := { **id** *BTF_ID* | **prog** *PROG* | **map** *MAP* [{**key** | **value** | **kv** | **all**}] | **file** *FILE* }
-| *FORMAT* := { **raw** | **c** [**unsorted**] }
+| *FORMAT* := { **raw** | **c** [**unsorted**] | **meta** }
 | *MAP* := { **id** *MAP_ID* | **pinned** *FILE* }
 | *PROG* := { **id** *PROG_ID* | **pinned** *FILE* | **tag** *PROG_TAG* | **name** *PROG_NAME* }
 
@@ -65,7 +65,8 @@ bpftool btf dump *BTF_SRC* [format *FORMAT*] [root_id *ROOT_ID*]
     **format** option can be used to override default (raw) output format. Raw
     (**raw**) or C-syntax (**c**) output formats are supported. With C-style
     formatting, the output is sorted by default. Use the **unsorted** option
-    to avoid sorting the output.
+    to avoid sorting the output.  BTF metadata can be displayed with the
+    **meta** option.
 
     **root_id** option can be used to filter a dump to a single type and all
     its dependent types. It cannot be used with any other types of filtering
@@ -267,3 +268,26 @@ All the standard ways to specify map or program are supported:
   [104859] FUNC 'smbalert_work' type_id=9695 linkage=static
   [104860] FUNC 'smbus_alert' type_id=71367 linkage=static
   [104861] FUNC 'smbus_do_alert' type_id=84827 linkage=static
+
+Display BTF metadata from file vmlinux
+
+**# bpftool btf dump file vmlinux format meta**
+
+::
+
+  size 5161076
+  magic 0xeb9f
+  version 1
+  flags 0x1
+  hdr_len 40
+  type_len 3036368
+  type_off 0
+  str_len 2124588
+  str_off 3036368
+  kind_layout_len 80
+  kind_layout_off 5160956
+  kind 0    UNKNOWN    info_sz 0    elem_sz 0
+  kind 1    INT        info_sz 0    elem_sz 0
+  kind 2    PTR        info_sz 0    elem_sz 0
+  kind 3    ARRAY      info_sz 0    elem_sz 0
+  kind 4    STRUCT     info_sz 0    elem_sz 0
-- 
2.39.3

[PATCH v8 bpf-next 10/10] kbuild, bpf: Specify "kind_layout" optional feature

[Alan Maguire]

The "kind_layout" feature will add metadata about BTF kinds to the
generated BTF; its absence in pahole will not trigger an error so it
is safe to add unconditionally as it will simply be ignored if pahole
does not support it.

Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
---
 scripts/Makefile.btf | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/scripts/Makefile.btf b/scripts/Makefile.btf
index db76335dd917..c20f9bbcabeb 100644
--- a/scripts/Makefile.btf
+++ b/scripts/Makefile.btf
@@ -25,6 +25,8 @@ pahole-flags-$(call test-ge, $(pahole-ver), 126)  = -j$(JOBS) --btf_features=enc
 
 pahole-flags-$(call test-ge, $(pahole-ver), 130) += --btf_features=attributes
 
+pahole-flags-$(call test-ge, $(pahole-ver), 131) += --btf_features=kind_layout
+
 ifneq ($(KBUILD_EXTMOD),)
 module-pahole-flags-$(call test-ge, $(pahole-ver), 128) += --btf_features=distilled_base
 endif
-- 
2.39.3

Re: [PATCH v8 bpf-next 01/10] btf: add kind layout encoding to UAPI

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 2119 bytes --]

> diff --git a/include/uapi/linux/btf.h b/include/uapi/linux/btf.h
> index 266d4ffa6c07..c1854a1c7b38 100644
> --- a/include/uapi/linux/btf.h
> +++ b/include/uapi/linux/btf.h
> @@ -8,6 +8,15 @@
>  #define BTF_MAGIC	0xeB9F
>  #define BTF_VERSION	1
>
> +/*
> + * kind layout section consists of a struct btf_kind_layout for each known
> + * kind at BTF encoding time.
> + */
> +struct btf_kind_layout {
> +	__u8 info_sz;		/* size of singular element after btf_type */
> +	__u8 elem_sz;		/* size of each of btf_vlen(t) elements */
> +};
> +
>  struct btf_header {
>  	__u16	magic;
>  	__u8	version;
> @@ -19,6 +28,8 @@ struct btf_header {
>  	__u32	type_len;	/* length of type section	*/
>  	__u32	str_off;	/* offset of string section	*/
>  	__u32	str_len;	/* length of string section	*/
> +	__u32	kind_layout_off;/* offset of kind layout section */
> +	__u32	kind_layout_len;/* length of kind layout section */
>  };

This adds two new __u32 fields to struct btf_header, growing it from 24
to 32 bytes. When BTF data needs to be byte-swapped for cross-endian
parsing, these new fields also need byte-swapping.

Looking at tools/lib/bpf/btf.c::btf_bswap_hdr() (not shown in this
UAPI-only patch), the function swaps magic, hdr_len, type_off, type_len,
str_off, and str_len, but the new kind_layout_off and kind_layout_len
fields are not present in the swapping logic. Will cross-endian BTF files
with kind layout sections fail to parse correctly in libbpf?

Note: This issue was fixed in the follow-up commit 16346d216f9b
("libbpf: Support kind layout section handling in BTF") which added
conditional byte-swapping for the new fields.

> diff --git a/tools/include/uapi/linux/btf.h b/tools/include/uapi/linux/btf.h
> index 266d4ffa6c07..c1854a1c7b38 100644
> --- a/tools/include/uapi/linux/btf.h
> +++ b/tools/include/uapi/linux/btf.h

[ ... identical change to tools copy ... ]


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/20226997366

Re: [PATCH v8 bpf-next 02/10] libbpf: Support kind layout section handling in BTF

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 3484 bytes --]

> diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> index b136572e889a..8835aee6ee84 100644
> --- a/tools/lib/bpf/btf.c
> +++ b/tools/lib/bpf/btf.c

[ ... ]

> @@ -214,7 +232,7 @@ static int btf_add_type_idx_entry(struct btf *btf, __u32 type_off)
>  	return 0;
>  }
>
> -static void btf_bswap_hdr(struct btf_header *h)
> +static void btf_bswap_hdr(struct btf_header *h, __u32 hdr_len)
>  {
>  	h->magic = bswap_16(h->magic);
>  	h->hdr_len = bswap_32(h->hdr_len);
> @@ -222,54 +240,87 @@ static void btf_bswap_hdr(struct btf_header *h)
>  	h->type_len = bswap_32(h->type_len);
>  	h->str_off = bswap_32(h->str_off);
>  	h->str_len = bswap_32(h->str_len);
> +	/* May be operating on raw data with hdr_len that does not include below fields */
> +	if (hdr_len >= sizeof(struct btf_header)) {
> +		h->kind_layout_off = bswap_32(h->kind_layout_off);
> +		h->kind_layout_len = bswap_32(h->kind_layout_len);
> +	}
>  }
>
>  static int btf_parse_hdr(struct btf *btf)
>  {
> -	struct btf_header *hdr = btf->hdr;
> +	struct btf_header *hdr = btf->raw_data;
> +	__u32 hdr_len = hdr->hdr_len;
>  	__u32 meta_left;
>
> -	if (btf->raw_size < sizeof(struct btf_header)) {
> +	if (btf->raw_size < offsetofend(struct btf_header, str_len)) {
>  		pr_debug("BTF header not found\n");
>  		return -EINVAL;
>  	}
>
>  	if (hdr->magic == bswap_16(BTF_MAGIC)) {
>  		btf->swapped_endian = true;
> -		if (bswap_32(hdr->hdr_len) != sizeof(struct btf_header)) {
> +		hdr_len = bswap_32(hdr->hdr_len);
> +		if (hdr_len < offsetofend(struct btf_header, str_len)) {
>  			pr_warn("Can't load BTF with non-native endianness due to unsupported header length %u\n",
> -				bswap_32(hdr->hdr_len));
> +				hdr_len);
>  			return -ENOTSUP;
>  		}
> -		btf_bswap_hdr(hdr);
> +		btf_bswap_hdr(hdr, hdr_len);
                       ^^^

Does this result in hdr->hdr_len being double-swapped? In the swapped
endianness path, hdr_len is set to bswap_32(hdr->hdr_len) and then
btf_bswap_hdr() is called, which does h->hdr_len = bswap_32(h->hdr_len)
again. This means hdr->hdr_len in the structure ends up with its original
(incorrect) endianness while the local hdr_len variable has the correct
swapped value.

Later code then copies the incorrectly-swapped header into btf->hdr:

>  	}
>
> -	if (btf->raw_size < hdr->hdr_len) {
> +	if (btf->raw_size < hdr_len) {
>  		pr_debug("BTF header len %u larger than data size %u\n",
> -			 hdr->hdr_len, btf->raw_size);
> +			 hdr_len, btf->raw_size);
>  		return -EINVAL;
>  	}

[ ... ]

> +	/* At this point, we have basic header information, so allocate btf->hdr */
> +	btf->hdr = calloc(1, sizeof(struct btf_header));
> +	if (!btf->hdr) {
> +		pr_debug("BTF header allocation failed\n");
> +		return -ENOMEM;
> +	}
> +	memcpy(btf->hdr, hdr, min((size_t)hdr_len, sizeof(struct btf_header)));
                           ^^^

The memcpy copies from hdr, which contains the double-swapped (therefore
incorrect) hdr_len field. So btf->hdr->hdr_len will have the wrong
endianness when btf_parse_hdr() returns. This affects subsequent pointer
arithmetic like:

    btf->types_data = btf->raw_data + btf->hdr->hdr_len + btf->hdr->type_off

causing types_data to point to the wrong location when loading BTF with
non-native endianness.


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/20226997366

Re: [PATCH v8 bpf-next 08/10] bpftool: add BTF dump "format meta" to dump header/metadata

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 2674 bytes --]

> diff --git a/tools/bpf/bpftool/btf.c b/tools/bpf/bpftool/btf.c
> index 946612029dee..fdb9b36fe106 100644
> --- a/tools/bpf/bpftool/btf.c
> +++ b/tools/bpf/bpftool/btf.c
> @@ -835,6 +835,90 @@ static int dump_btf_c(const struct btf *btf,
>  	return err;
>  }
>
> +static int dump_btf_meta(const struct btf *btf)
> +{
> +	const struct btf_header *hdr;
> +	const struct btf_kind_layout *k;
> +	__u8 i, nr_kinds = 0;
> +	const void *data;
> +	__u32 data_sz;
> +
> +	data = btf__raw_data(btf, &data_sz);
> +	if (!data)
> +		return -ENOMEM;
> +	hdr = data;

[ ... ]

> +	if (hdr->hdr_len < sizeof(struct btf_header)) {
> +		if (json_output) {
> +			jsonw_end_object(json_wtr);	/* end header object */
> +			jsonw_end_object(json_wtr);	/* end metadata object */
> +		}
> +		return 0;
> +	}
> +
> +	data_sz -= hdr->hdr_len;

Can data_sz underflow here? If hdr->hdr_len is larger than the
original data_sz, this subtraction wraps to a large value. The
check above only ensures hdr_len >= sizeof(struct btf_header),
not that hdr_len <= data_sz.

> +
> +	if (hdr->kind_layout_len > 0 && hdr->kind_layout_off > 0) {
> +		if (hdr->kind_layout_off + hdr->kind_layout_len <= data_sz) {

Can the addition here overflow? Both kind_layout_off and
kind_layout_len are __u32 values from the BTF header. If an attacker
crafts a BTF file with kind_layout_off = 0xFFFFFFF0 and
kind_layout_len = 0x20, the sum wraps to 0x10 and passes the bounds
check, but the actual range extends far beyond data_sz. The subsequent
pointer calculation and k[i] accesses would then read out-of-bounds.

> +			k = (void *)hdr + hdr->hdr_len + hdr->kind_layout_off;
> +			nr_kinds = hdr->kind_layout_len / sizeof(*k);

Does nr_kinds truncate correctly? Since nr_kinds is __u8 but
kind_layout_len is __u32, the division result gets truncated. If
kind_layout_len is 512 or larger, nr_kinds wraps (512/2 = 256 -> 0).
While this causes incorrect output rather than a crash, using __u32
for nr_kinds would be more appropriate given that BTF kinds are
designed to be extensible.

> +		}
> +	}

[ ... ]

> +	} else {
> +		printf("kind_layout_len %-10u\nkind_layout_off %-10u\n",
> +		       hdr->kind_layout_len, hdr->kind_layout_off);
> +		for (i = 0; i < nr_kinds; i++) {
> +			printf("kind %-4d %-10s info_sz %-4d elem_sz %-4d\n",
> +			       i, i < NR_BTF_KINDS ? btf_kind_str[i] : "?",
> +			       k[i].info_sz, k[i].elem_sz);
> +		}
> +	}
> +
> +	return 0;
> +}

[ ... ]


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/20226997366

Re: [PATCH v8 bpf-next 02/10] libbpf: Support kind layout section handling in BTF

[Alan Maguire]

On 15/12/2025 09:38, bot+bpf-ci@kernel.org wrote:
>> diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
>> index b136572e889a..8835aee6ee84 100644
>> --- a/tools/lib/bpf/btf.c
>> +++ b/tools/lib/bpf/btf.c
> 
> [ ... ]
> 
>> @@ -214,7 +232,7 @@ static int btf_add_type_idx_entry(struct btf *btf, __u32 type_off)
>>  	return 0;
>>  }
>>
>> -static void btf_bswap_hdr(struct btf_header *h)
>> +static void btf_bswap_hdr(struct btf_header *h, __u32 hdr_len)
>>  {
>>  	h->magic = bswap_16(h->magic);
>>  	h->hdr_len = bswap_32(h->hdr_len);
>> @@ -222,54 +240,87 @@ static void btf_bswap_hdr(struct btf_header *h)
>>  	h->type_len = bswap_32(h->type_len);
>>  	h->str_off = bswap_32(h->str_off);
>>  	h->str_len = bswap_32(h->str_len);
>> +	/* May be operating on raw data with hdr_len that does not include below fields */
>> +	if (hdr_len >= sizeof(struct btf_header)) {
>> +		h->kind_layout_off = bswap_32(h->kind_layout_off);
>> +		h->kind_layout_len = bswap_32(h->kind_layout_len);
>> +	}
>>  }
>>
>>  static int btf_parse_hdr(struct btf *btf)
>>  {
>> -	struct btf_header *hdr = btf->hdr;
>> +	struct btf_header *hdr = btf->raw_data;
>> +	__u32 hdr_len = hdr->hdr_len;
>>  	__u32 meta_left;
>>
>> -	if (btf->raw_size < sizeof(struct btf_header)) {
>> +	if (btf->raw_size < offsetofend(struct btf_header, str_len)) {
>>  		pr_debug("BTF header not found\n");
>>  		return -EINVAL;
>>  	}
>>
>>  	if (hdr->magic == bswap_16(BTF_MAGIC)) {
>>  		btf->swapped_endian = true;
>> -		if (bswap_32(hdr->hdr_len) != sizeof(struct btf_header)) {
>> +		hdr_len = bswap_32(hdr->hdr_len);
>> +		if (hdr_len < offsetofend(struct btf_header, str_len)) {
>>  			pr_warn("Can't load BTF with non-native endianness due to unsupported header length %u\n",
>> -				bswap_32(hdr->hdr_len));
>> +				hdr_len);
>>  			return -ENOTSUP;
>>  		}
>> -		btf_bswap_hdr(hdr);
>> +		btf_bswap_hdr(hdr, hdr_len);
>                        ^^^
> 
> Does this result in hdr->hdr_len being double-swapped? In the swapped
> endianness path, hdr_len is set to bswap_32(hdr->hdr_len) and then
> btf_bswap_hdr() is called, which does h->hdr_len = bswap_32(h->hdr_len)
> again. This means hdr->hdr_len in the structure ends up with its original
> (incorrect) endianness while the local hdr_len variable has the correct
> swapped value.
> 
> Later code then copies the incorrectly-swapped header into btf->hdr:
>

I don't think this is right. We do

	hdr_len = bswap_32(hdr->hdr_len);

and this will _return_ a bswapped value, but does not bswap hdr->hdr_len itself.
When we later bswap the header via btf_bswap_hdr(), we end up in a state where
both hdr_len and hdr->hdr_len will have the same (swapped once) value. If we
had 

	hdr->hdr_len = bswap_32(hdr->hdr_len);

...the concern would be valid. So AI wasn't far off here.
 
>>  	}
>>
>> -	if (btf->raw_size < hdr->hdr_len) {
>> +	if (btf->raw_size < hdr_len) {
>>  		pr_debug("BTF header len %u larger than data size %u\n",
>> -			 hdr->hdr_len, btf->raw_size);
>> +			 hdr_len, btf->raw_size);
>>  		return -EINVAL;
>>  	}
> 
> [ ... ]
> 
>> +	/* At this point, we have basic header information, so allocate btf->hdr */
>> +	btf->hdr = calloc(1, sizeof(struct btf_header));
>> +	if (!btf->hdr) {
>> +		pr_debug("BTF header allocation failed\n");
>> +		return -ENOMEM;
>> +	}
>> +	memcpy(btf->hdr, hdr, min((size_t)hdr_len, sizeof(struct btf_header)));
>                            ^^^
> 
> The memcpy copies from hdr, which contains the double-swapped (therefore
> incorrect) hdr_len field. So btf->hdr->hdr_len will have the wrong
> endianness when btf_parse_hdr() returns. This affects subsequent pointer
> arithmetic like:
> 
>     btf->types_data = btf->raw_data + btf->hdr->hdr_len + btf->hdr->type_off
> 
> causing types_data to point to the wrong location when loading BTF with
> non-native endianness.
>

AI has followed the chain of reasoning based on the non-existent byte-swap issue
above, so this isn't a concern either I believe.

Re: [PATCH v8 bpf-next 02/10] libbpf: Support kind layout section handling in BTF

[Eduard Zingerman]

[-- Attachment #1: Type: text/plain, Size: 5789 bytes --]

On Mon, 2025-12-15 at 09:17 +0000, Alan Maguire wrote:

Hi Alan,

Overall makes sense, please fine one suggestion below.

> Support reading in kind layout fixing endian issues on reading;
> also support writing kind layout section to raw BTF object.
> There is not yet an API to populate the kind layout with meaningful
> information.
> 
> As part of this, we need to consider multiple valid BTF header
> sizes; the original or the kind layout-extended headers.
> So to support this, the "struct btf" representation is modified
> to always allocate a "struct btf_header" and copy the valid
> portion from the raw data to it; this means we can always safely
> check fields like btf->hdr->kind_layout_len.
> 
> Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
> ---
>  tools/lib/bpf/btf.c | 260 +++++++++++++++++++++++++++++++-------------
>  1 file changed, 183 insertions(+), 77 deletions(-)
> 
> diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> index b136572e889a..8835aee6ee84 100644
> --- a/tools/lib/bpf/btf.c
> +++ b/tools/lib/bpf/btf.c
> @@ -40,42 +40,53 @@ struct btf {
>  
>  	/*
>  	 * When BTF is loaded from an ELF or raw memory it is stored
> -	 * in a contiguous memory block. The hdr, type_data, and, strs_data
> +	 * in a contiguous memory block. The  type_data, and, strs_data
>  	 * point inside that memory region to their respective parts of BTF
>  	 * representation:
>  	 *
> -	 * +--------------------------------+
> -	 * |  Header  |  Types  |  Strings  |
> -	 * +--------------------------------+
> -	 * ^          ^         ^
> -	 * |          |         |
> -	 * hdr        |         |
> -	 * types_data-+         |
> -	 * strs_data------------+
> +	 * +--------------------------------+---------------------+
> +	 * |  Header  |  Types  |  Strings  |Optional kind layout |
> +	 * +--------------------------------+---------------------+
> +	 * ^          ^         ^           ^
> +	 * |          |         |           |
> +	 * raw_data   |         |           |
> +	 * types_data-+         |           |
> +	 * strs_data------------+           |
> +	 * kind_layout----------------------+
> +	 *
> +	 * A separate struct btf_header is allocated for btf->hdr,
> +	 * and header information is copied into it.  This allows us
> +	 * to handle header data for various header formats; the original,
> +	 * the extended header with kind layout, etc.
>  	 *
>  	 * If BTF data is later modified, e.g., due to types added or
>  	 * removed, BTF deduplication performed, etc, this contiguous
> -	 * representation is broken up into three independently allocated
> -	 * memory regions to be able to modify them independently.
> +	 * representation is broken up into four independent memory
> +	 * regions.
> +	 *
>  	 * raw_data is nulled out at that point, but can be later allocated
>  	 * and cached again if user calls btf__raw_data(), at which point
> -	 * raw_data will contain a contiguous copy of header, types, and
> -	 * strings:
> +	 * raw_data will contain a contiguous copy of header, types, strings
> +	 * and optionally kind_layout.  kind_layout optionally points to a
> +	 * kind_layout array - this allows us to encode information about
> +	 * the kinds known at encoding time.  If kind_layout is NULL no
> +	 * kind information is encoded.
>  	 *
> -	 * +----------+  +---------+  +-----------+
> -	 * |  Header  |  |  Types  |  |  Strings  |
> -	 * +----------+  +---------+  +-----------+
> -	 * ^             ^            ^
> -	 * |             |            |
> -	 * hdr           |            |
> -	 * types_data----+            |
> -	 * strset__data(strs_set)-----+
> +	 * +----------+  +---------+  +-----------+   +-----------+
> +	 * |  Header  |  |  Types  |  |  Strings  |   |kind_layout|
> +	 * +----------+  +---------+  +-----------+   +-----------+
> +	 * ^             ^            ^               ^
> +	 * |             |            |               |
> +	 * hdr           |            |               |
> +	 * types_data----+            |               |
> +	 * strset__data(strs_set)-----+               |
> +	 * kind_layout--------------------------------+
>  	 *
> -	 *               +----------+---------+-----------+
> -	 *               |  Header  |  Types  |  Strings  |
> -	 * raw_data----->+----------+---------+-----------+
> +	 *               +----------+---------+-----------+---------------------+
> +	 *               |  Header  |  Types  |  Strings  | Optional kind layout|
> +	 * raw_data----->+----------+---------+-----------+---------------------+
>  	 */
> -	struct btf_header *hdr;
> +	struct btf_header *hdr; /* separately-allocated header data */

Do we want to directly embed this structure?
E.g. as in the diff attached.
No need to bother with calloc/free in such case.

>  
>  	void *types_data;
>  	size_t types_data_cap; /* used size stored in hdr->type_len */
> @@ -123,6 +134,13 @@ struct btf {
>  	/* whether raw_data is a (read-only) mmap */
>  	bool raw_data_is_mmap;
>  
> +	/* is BTF modifiable? i.e. is it split into separate sections as described above? */
> +	bool modifiable;
> +	/* Points either at raw kind layout data in parsed BTF (if present), or
> +	 * at an allocated kind layout array when BTF is modifiable.
> +	 */
> +	void *kind_layout;
> +
>  	/* BTF object FD, if loaded into kernel */
>  	int fd;
>  
> @@ -214,7 +232,7 @@ static int btf_add_type_idx_entry(struct btf *btf, __u32 type_off)
>  	return 0;
>  }
>  
> -static void btf_bswap_hdr(struct btf_header *h)
> +static void btf_bswap_hdr(struct btf_header *h, __u32 hdr_len)
>  {
>  	h->magic = bswap_16(h->magic);
>  	h->hdr_len = bswap_32(h->hdr_len);

[...]

Thanks,
Eduard

[-- Attachment #2: embed-hdr.diff --]
[-- Type: text/x-patch, Size: 16269 bytes --]

diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
index 6098c8d1e26a..af4eb27d2f54 100644
--- a/tools/lib/bpf/btf.c
+++ b/tools/lib/bpf/btf.c
@@ -115,7 +115,7 @@ struct btf {
 	 *               |  Header  |  Types  |  Strings  | Optional kind layout|
 	 * raw_data----->+----------+---------+-----------+---------------------+
 	 */
-	struct btf_header *hdr; /* separately-allocated header data */
+	struct btf_header hdr; /* copy of the header data */
 
 	void *types_data;
 	size_t types_data_cap; /* used size stored in hdr->type_len */
@@ -307,46 +307,41 @@ static int btf_parse_hdr(struct btf *btf)
 		return -EINVAL;
 	}
 
-	/* At this point, we have basic header information, so allocate btf->hdr */
-	btf->hdr = calloc(1, sizeof(struct btf_header));
-	if (!btf->hdr) {
-		pr_debug("BTF header allocation failed\n");
-		return -ENOMEM;
-	}
-	memcpy(btf->hdr, hdr, min((size_t)hdr_len, sizeof(struct btf_header)));
+	/* At this point, we have basic header information, so copy to btf->hdr */
+	memcpy(&btf->hdr, hdr, min((size_t)hdr_len, sizeof(struct btf_header)));
 
 	meta_left = btf->raw_size - hdr_len;
-	if (meta_left < (long long)btf->hdr->str_off + btf->hdr->str_len) {
+	if (meta_left < (long long)btf->hdr.str_off + btf->hdr.str_len) {
 		pr_debug("Invalid BTF total size: %u\n", btf->raw_size);
 		return -EINVAL;
 	}
 
-	if ((long long)btf->hdr->type_off + btf->hdr->type_len > btf->hdr->str_off) {
+	if ((long long)btf->hdr.type_off + btf->hdr.type_len > btf->hdr.str_off) {
 		pr_debug("Invalid BTF data sections layout: type data at %u + %u, strings data at %u + %u\n",
-			 btf->hdr->type_off, btf->hdr->type_len, btf->hdr->str_off,
-			 btf->hdr->str_len);
+			 btf->hdr.type_off, btf->hdr.type_len, btf->hdr.str_off,
+			 btf->hdr.str_len);
 		return -EINVAL;
 	}
 
-	if (btf->hdr->type_off % 4) {
+	if (btf->hdr.type_off % 4) {
 		pr_debug("BTF type section is not aligned to 4 bytes\n");
 		return -EINVAL;
 	}
 
-	if (btf->hdr->kind_layout_len == 0)
+	if (btf->hdr.kind_layout_len == 0)
 		return 0;
 
-	if (btf->hdr->kind_layout_off % 4) {
+	if (btf->hdr.kind_layout_off % 4) {
 		pr_debug("BTF kind_layout section is not aligned to 4 bytes\n");
 		return -EINVAL;
 	}
-	if (btf->hdr->kind_layout_off < btf->hdr->str_off + btf->hdr->str_len) {
+	if (btf->hdr.kind_layout_off < btf->hdr.str_off + btf->hdr.str_len) {
 		pr_debug("Invalid BTF data sections layout: strings data at %u + %u, kind layout data at %u + %u\n",
-			 btf->hdr->str_off, btf->hdr->str_len,
-			 btf->hdr->kind_layout_off, btf->hdr->kind_layout_len);
+			 btf->hdr.str_off, btf->hdr.str_len,
+			 btf->hdr.kind_layout_off, btf->hdr.kind_layout_len);
 		return -EINVAL;
 	}
-	if (btf->hdr->kind_layout_off + btf->hdr->kind_layout_len > meta_left) {
+	if (btf->hdr.kind_layout_off + btf->hdr.kind_layout_len > meta_left) {
 		pr_debug("Invalid BTF total size: %u\n", btf->raw_size);
 		return -EINVAL;
 	}
@@ -355,9 +350,9 @@ static int btf_parse_hdr(struct btf *btf)
 
 static int btf_parse_str_sec(struct btf *btf)
 {
-	const struct btf_header *hdr = btf->hdr;
+	const struct btf_header *hdr = &btf->hdr;
 	const char *start = btf->strs_data;
-	const char *end = start + btf->hdr->str_len;
+	const char *end = start + btf->hdr.str_len;
 
 	if (btf->base_btf && hdr->str_len == 0)
 		return 0;
@@ -374,7 +369,7 @@ static int btf_parse_str_sec(struct btf *btf)
 
 static int btf_parse_kind_layout_sec(struct btf *btf)
 {
-	const struct btf_header *hdr = btf->hdr;
+	const struct btf_header *hdr = &btf->hdr;
 
 	if (!hdr->kind_layout_len)
 		return 0;
@@ -383,7 +378,7 @@ static int btf_parse_kind_layout_sec(struct btf *btf)
 		pr_debug("Invalid BTF kind layout section\n");
 		return -EINVAL;
 	}
-	btf->kind_layout = btf->raw_data + btf->hdr->hdr_len + btf->hdr->kind_layout_off;
+	btf->kind_layout = btf->raw_data + btf->hdr.hdr_len + btf->hdr.kind_layout_off;
 
 	return 0;
 }
@@ -397,7 +392,7 @@ static int btf_type_size_unknown(const struct btf *btf, const struct btf_type *t
 	__u8 kind = btf_kind(t);
 	__u32 off = kind * sizeof(struct btf_kind_layout);
 
-	if (!btf->kind_layout || off >= btf->hdr->kind_layout_len) {
+	if (!btf->kind_layout || off >= btf->hdr.kind_layout_len) {
 		pr_debug("Unsupported BTF_KIND: %u\n", btf_kind(t));
 		return -EINVAL;
 	}
@@ -535,7 +530,7 @@ static int btf_bswap_type_rest(struct btf_type *t)
 
 static int btf_parse_type_sec(struct btf *btf)
 {
-	struct btf_header *hdr = btf->hdr;
+	struct btf_header *hdr = &btf->hdr;
 	void *next_type = btf->types_data;
 	void *end_type = next_type + hdr->type_len;
 	int err, type_size;
@@ -1105,7 +1100,6 @@ void btf__free(struct btf *btf)
 		strset__free(btf->strs_set);
 		free(btf->kind_layout);
 	}
-	free(btf->hdr);
 	btf_free_raw_data(btf);
 	free(btf->raw_data_swapped);
 	free(btf->type_offs);
@@ -1135,7 +1129,7 @@ static struct btf *btf_new_empty(struct btf_new_opts *opts)
 	if (base_btf) {
 		btf->base_btf = base_btf;
 		btf->start_id = btf__type_cnt(base_btf);
-		btf->start_str_off = base_btf->hdr->str_len + base_btf->start_str_off;
+		btf->start_str_off = base_btf->hdr.str_len + base_btf->start_str_off;
 		btf->swapped_endian = base_btf->swapped_endian;
 	}
 
@@ -1157,20 +1151,13 @@ static struct btf *btf_new_empty(struct btf_new_opts *opts)
 	btf->types_data = btf->raw_data + hdr->hdr_len;
 	btf->strs_data = btf->raw_data + hdr->hdr_len;
 	hdr->str_len = base_btf ? 0 : 1; /* empty string at offset 0 */
-	btf->hdr = calloc(1, sizeof(struct btf_header));
-	if (!btf->hdr) {
-		free(btf->raw_data);
-		free(btf);
-		return ERR_PTR(-ENOMEM);
-	}
-
 	if (add_kind_layout) {
 		hdr->kind_layout_len = sizeof(kind_layouts);
 		hdr->kind_layout_off = roundup(hdr->str_len, 4);
 		btf->kind_layout = btf->raw_data + hdr->hdr_len + hdr->kind_layout_off;
 		memcpy(btf->kind_layout, kind_layouts, sizeof(kind_layouts));
 	}
-	memcpy(btf->hdr, hdr, sizeof(*hdr));
+	memcpy(&btf->hdr, hdr, sizeof(*hdr));
 
 	return btf;
 }
@@ -1216,7 +1203,7 @@ static struct btf *btf_new(const void *data, __u32 size, struct btf *base_btf, b
 	if (base_btf) {
 		btf->base_btf = base_btf;
 		btf->start_id = btf__type_cnt(base_btf);
-		btf->start_str_off = base_btf->hdr->str_len + base_btf->start_str_off;
+		btf->start_str_off = base_btf->hdr.str_len + base_btf->start_str_off;
 	}
 
 	if (is_mmap) {
@@ -1237,8 +1224,8 @@ static struct btf *btf_new(const void *data, __u32 size, struct btf *base_btf, b
 	if (err)
 		goto done;
 
-	btf->strs_data = btf->raw_data + btf->hdr->hdr_len + btf->hdr->str_off;
-	btf->types_data = btf->raw_data + btf->hdr->hdr_len + btf->hdr->type_off;
+	btf->strs_data = btf->raw_data + btf->hdr.hdr_len + btf->hdr.str_off;
+	btf->types_data = btf->raw_data + btf->hdr.hdr_len + btf->hdr.type_off;
 
 	err = btf_parse_str_sec(btf);
 	err = err ?: btf_parse_kind_layout_sec(btf);
@@ -1692,7 +1679,7 @@ static const void *btf_strs_data(const struct btf *btf)
 
 static void *btf_get_raw_data(const struct btf *btf, __u32 *size, bool swap_endian)
 {
-	struct btf_header *hdr = btf->hdr;
+	const struct btf_header *hdr = &btf->hdr;
 	struct btf_type *t;
 	void *data, *p;
 	__u32 data_sz;
@@ -1774,7 +1761,7 @@ const char *btf__str_by_offset(const struct btf *btf, __u32 offset)
 {
 	if (offset < btf->start_str_off)
 		return btf__str_by_offset(btf->base_btf, offset);
-	else if (offset - btf->start_str_off < btf->hdr->str_len)
+	else if (offset - btf->start_str_off < btf->hdr.str_len)
 		return btf_strs_data(btf) + (offset - btf->start_str_off);
 	else
 		return errno = EINVAL, NULL;
@@ -1897,21 +1884,21 @@ static int btf_ensure_modifiable(struct btf *btf)
 	}
 
 	/* split raw data into memory regions; btf->hdr is done already. */
-	types = malloc(btf->hdr->type_len);
+	types = malloc(btf->hdr.type_len);
 	if (!types)
 		goto err_out;
-	memcpy(types, btf->types_data, btf->hdr->type_len);
+	memcpy(types, btf->types_data, btf->hdr.type_len);
 
-	if (btf->hdr->kind_layout_len) {
-		kind_layout = malloc(btf->hdr->kind_layout_len);
+	if (btf->hdr.kind_layout_len) {
+		kind_layout = malloc(btf->hdr.kind_layout_len);
 		if (!kind_layout)
 			goto err_out;
-		memcpy(kind_layout, btf->raw_data + btf->hdr->hdr_len + btf->hdr->kind_layout_off,
-		       btf->hdr->kind_layout_len);
+		memcpy(kind_layout, btf->raw_data + btf->hdr.hdr_len + btf->hdr.kind_layout_off,
+		       btf->hdr.kind_layout_len);
 	}
 
 	/* build lookup index for all strings */
-	set = strset__new(BTF_MAX_STR_OFFSET, btf->strs_data, btf->hdr->str_len);
+	set = strset__new(BTF_MAX_STR_OFFSET, btf->strs_data, btf->hdr.str_len);
 	if (IS_ERR(set)) {
 		err = PTR_ERR(set);
 		goto err_out;
@@ -1919,7 +1906,7 @@ static int btf_ensure_modifiable(struct btf *btf)
 
 	/* only when everything was successful, update internal state */
 	btf->types_data = types;
-	btf->types_data_cap = btf->hdr->type_len;
+	btf->types_data_cap = btf->hdr.type_len;
 	btf->strs_data = NULL;
 	btf->strs_set = set;
 	if (kind_layout)
@@ -1927,9 +1914,9 @@ static int btf_ensure_modifiable(struct btf *btf)
 	/* if BTF was created from scratch, all strings are guaranteed to be
 	 * unique and deduplicated
 	 */
-	if (btf->hdr->str_len == 0)
+	if (btf->hdr.str_len == 0)
 		btf->strs_deduped = true;
-	if (!btf->base_btf && btf->hdr->str_len == 1)
+	if (!btf->base_btf && btf->hdr.str_len == 1)
 		btf->strs_deduped = true;
 
 	/* invalidate raw_data representation */
@@ -1995,7 +1982,7 @@ int btf__add_str(struct btf *btf, const char *s)
 	if (off < 0)
 		return libbpf_err(off);
 
-	btf->hdr->str_len = strset__data_size(btf->strs_set);
+	btf->hdr.str_len = strset__data_size(btf->strs_set);
 
 	return btf->start_str_off + off;
 }
@@ -2003,7 +1990,7 @@ int btf__add_str(struct btf *btf, const char *s)
 static void *btf_add_type_mem(struct btf *btf, size_t add_sz)
 {
 	return libbpf_add_mem(&btf->types_data, &btf->types_data_cap, 1,
-			      btf->hdr->type_len, UINT_MAX, add_sz);
+			      btf->hdr.type_len, UINT_MAX, add_sz);
 }
 
 static void btf_type_inc_vlen(struct btf_type *t)
@@ -2013,28 +2000,28 @@ static void btf_type_inc_vlen(struct btf_type *t)
 
 static void btf_hdr_update_type_len(struct btf *btf, int new_len)
 {
-	btf->hdr->type_len = new_len;
-	btf->hdr->str_off = new_len;
+	btf->hdr.type_len = new_len;
+	btf->hdr.str_off = new_len;
 	if (btf->kind_layout)
-		btf->hdr->kind_layout_off = btf->hdr->type_len + roundup(btf->hdr->str_len, 4);
+		btf->hdr.kind_layout_off = btf->hdr.type_len + roundup(btf->hdr.str_len, 4);
 }
 
 static void btf_hdr_update_str_len(struct btf *btf, int new_len)
 {
-	btf->hdr->str_len = new_len;
+	btf->hdr.str_len = new_len;
 	if (btf->kind_layout)
-		btf->hdr->kind_layout_off = btf->hdr->type_len + roundup(new_len, 4);
+		btf->hdr.kind_layout_off = btf->hdr.type_len + roundup(new_len, 4);
 }
 
 static int btf_commit_type(struct btf *btf, int data_sz)
 {
 	int err;
 
-	err = btf_add_type_idx_entry(btf, btf->hdr->type_len);
+	err = btf_add_type_idx_entry(btf, btf->hdr.type_len);
 	if (err)
 		return libbpf_err(err);
 
-	btf_hdr_update_type_len(btf, btf->hdr->type_len + data_sz);
+	btf_hdr_update_type_len(btf, btf->hdr.type_len + data_sz);
 	btf->nr_types++;
 	return btf->start_id + btf->nr_types - 1;
 }
@@ -2138,9 +2125,9 @@ int btf__add_btf(struct btf *btf, const struct btf *src_btf)
 	/* remember original strings section size if we have to roll back
 	 * partial strings section changes
 	 */
-	old_strs_len = btf->hdr->str_len;
+	old_strs_len = btf->hdr.str_len;
 
-	data_sz = src_btf->hdr->type_len;
+	data_sz = src_btf->hdr.type_len;
 	cnt = btf__type_cnt(src_btf) - 1;
 
 	/* pre-allocate enough memory for new types */
@@ -2214,7 +2201,7 @@ int btf__add_btf(struct btf *btf, const struct btf *src_btf)
 	 * update type count and various internal offsets and sizes to
 	 * "commit" the changes and made them visible to the outside world.
 	 */
-	btf_hdr_update_type_len(btf, btf->hdr->type_len + data_sz);
+	btf_hdr_update_type_len(btf, btf->hdr.type_len + data_sz);
 	btf->nr_types += cnt;
 
 	hashmap__free(p.str_off_map);
@@ -2225,8 +2212,8 @@ int btf__add_btf(struct btf *btf, const struct btf *src_btf)
 	/* zero out preallocated memory as if it was just allocated with
 	 * libbpf_add_mem()
 	 */
-	memset(btf->types_data + btf->hdr->type_len, 0, data_sz);
-	memset(btf->strs_data + old_strs_len, 0, btf->hdr->str_len - old_strs_len);
+	memset(btf->types_data + btf->hdr.type_len, 0, data_sz);
+	memset(btf->strs_data + old_strs_len, 0, btf->hdr.str_len - old_strs_len);
 
 	/* and now restore original strings section size; types data size
 	 * wasn't modified, so doesn't need restoring, see big comment above
@@ -2549,7 +2536,7 @@ int btf__add_field(struct btf *btf, const char *name, int type_id,
 	/* update parent type's vlen and kflag */
 	t->info = btf_type_info(btf_kind(t), btf_vlen(t) + 1, is_bitfield || btf_kflag(t));
 
-	btf_hdr_update_type_len(btf, btf->hdr->type_len + sz);
+	btf_hdr_update_type_len(btf, btf->hdr.type_len + sz);
 	return 0;
 }
 
@@ -2658,7 +2645,7 @@ int btf__add_enum_value(struct btf *btf, const char *name, __s64 value)
 	if (value < 0)
 		t->info = btf_type_info(btf_kind(t), btf_vlen(t), true);
 
-	btf_hdr_update_type_len(btf, btf->hdr->type_len + sz);
+	btf_hdr_update_type_len(btf, btf->hdr.type_len + sz);
 	return 0;
 }
 
@@ -2729,7 +2716,7 @@ int btf__add_enum64_value(struct btf *btf, const char *name, __u64 value)
 	t = btf_last_type(btf);
 	btf_type_inc_vlen(t);
 
-	btf_hdr_update_type_len(btf, btf->hdr->type_len + sz);
+	btf_hdr_update_type_len(btf, btf->hdr.type_len + sz);
 	return 0;
 }
 
@@ -2967,7 +2954,7 @@ int btf__add_func_param(struct btf *btf, const char *name, int type_id)
 	t = btf_last_type(btf);
 	btf_type_inc_vlen(t);
 
-	btf_hdr_update_type_len(btf, btf->hdr->type_len + sz);
+	btf_hdr_update_type_len(btf, btf->hdr.type_len + sz);
 	return 0;
 }
 
@@ -3103,7 +3090,7 @@ int btf__add_datasec_var_info(struct btf *btf, int var_type_id, __u32 offset, __
 	t = btf_last_type(btf);
 	btf_type_inc_vlen(t);
 
-	btf_hdr_update_type_len(btf, btf->hdr->type_len + sz);
+	btf_hdr_update_type_len(btf, btf->hdr.type_len + sz);
 	return 0;
 }
 
@@ -5514,18 +5501,18 @@ static int btf_dedup_compact_types(struct btf_dedup *d)
 	/* shrink struct btf's internal types index and update btf_header */
 	d->btf->nr_types = next_type_id - d->btf->start_id;
 	d->btf->type_offs_cap = d->btf->nr_types;
-	d->btf->hdr->type_len = p - d->btf->types_data;
+	d->btf->hdr.type_len = p - d->btf->types_data;
 	new_offs = libbpf_reallocarray(d->btf->type_offs, d->btf->type_offs_cap,
 				       sizeof(*new_offs));
 	if (d->btf->type_offs_cap && !new_offs)
 		return -ENOMEM;
 	d->btf->type_offs = new_offs;
-	d->btf->hdr->str_off = d->btf->hdr->type_len;
-	d->btf->raw_size = d->btf->hdr->hdr_len + d->btf->hdr->type_len + d->btf->hdr->str_len;
+	d->btf->hdr.str_off = d->btf->hdr.type_len;
+	d->btf->raw_size = d->btf->hdr.hdr_len + d->btf->hdr.type_len + d->btf->hdr.str_len;
 	if (d->btf->kind_layout) {
-		d->btf->hdr->kind_layout_off = d->btf->hdr->str_off + roundup(d->btf->hdr->str_len,
+		d->btf->hdr.kind_layout_off = d->btf->hdr.str_off + roundup(d->btf->hdr.str_len,
 									      4);
-		d->btf->raw_size = roundup(d->btf->raw_size, 4) + d->btf->hdr->kind_layout_len;
+		d->btf->raw_size = roundup(d->btf->raw_size, 4) + d->btf->hdr.kind_layout_len;
 	}
 	return 0;
 }
@@ -5984,7 +5971,7 @@ int btf__distill_base(const struct btf *src_btf, struct btf **new_base_btf,
 		goto done;
 	}
 	dist.split_start_id = btf__type_cnt(old_base);
-	dist.split_start_str = old_base->hdr->str_len;
+	dist.split_start_str = old_base->hdr.str_len;
 
 	/* Pass over src split BTF; generate the list of base BTF type ids it
 	 * references; these will constitute our distilled BTF set to be
@@ -6053,14 +6040,14 @@ int btf__distill_base(const struct btf *src_btf, struct btf **new_base_btf,
 
 const struct btf_header *btf_header(const struct btf *btf)
 {
-	return btf->hdr;
+	return &btf->hdr;
 }
 
 void btf_set_base_btf(struct btf *btf, const struct btf *base_btf)
 {
 	btf->base_btf = (struct btf *)base_btf;
 	btf->start_id = btf__type_cnt(base_btf);
-	btf->start_str_off = base_btf->hdr->str_len + base_btf->start_str_off;
+	btf->start_str_off = base_btf->hdr.str_len + base_btf->start_str_off;
 }
 
 int btf__relocate(struct btf *btf, const struct btf *base_btf)

Re: [PATCH v8 bpf-next 04/10] libbpf: Add kind layout encoding support

[Eduard Zingerman]

On Mon, 2025-12-15 at 09:17 +0000, Alan Maguire wrote:

[...]

> @@ -1106,6 +1137,8 @@ static struct btf *btf_new_empty(struct btf *base_btf)
>  
>  	/* +1 for empty string at offset 0 */
>  	btf->raw_size = sizeof(struct btf_header) + (base_btf ? 0 : 1);
> +	if (add_kind_layout)
> +		btf->raw_size = roundup(btf->raw_size, 4) + sizeof(kind_layouts);
>  	btf->raw_data = calloc(1, btf->raw_size);
>  	if (!btf->raw_data) {
>  		free(btf);
> @@ -1126,6 +1159,13 @@ static struct btf *btf_new_empty(struct btf *base_btf)
>  		free(btf);
>  		return ERR_PTR(-ENOMEM);
>  	}
> +
> +	if (add_kind_layout) {
> +		hdr->kind_layout_len = sizeof(kind_layouts);
> +		hdr->kind_layout_off = roundup(hdr->str_len, 4);

btf_add_str() adjusts btf->hdr->str_len w/o moving hdr->kind_layout_off.
This should make hdr->kind_layout_off obsolete after each string addition.
btf_get_raw_data() does not seem to have any logic to adjust kind_layout_off.
Is this a problem, or do I miss something?

> +		btf->kind_layout = btf->raw_data + hdr->hdr_len + hdr->kind_layout_off;
> +		memcpy(btf->kind_layout, kind_layouts, sizeof(kind_layouts));
> +	}
>  	memcpy(btf->hdr, hdr, sizeof(*hdr));
>  
>  	return btf;
> @@ -1133,12 +1173,26 @@ static struct btf *btf_new_empty(struct btf *base_btf)
>  
>  struct btf *btf__new_empty(void)
>  {
> -	return libbpf_ptr(btf_new_empty(NULL));
> +	LIBBPF_OPTS(btf_new_opts, opts);
> +
> +	return libbpf_ptr(btf_new_empty(&opts));

Nit: btf_new_empty(NULL) should work fine because of OPTS_GET.

>  }
>  

[...]

Re: [PATCH v8 bpf-next 02/10] libbpf: Support kind layout section handling in BTF

[Eduard Zingerman]

On Mon, 2025-12-15 at 09:17 +0000, Alan Maguire wrote:

[...]

> @@ -951,7 +1018,8 @@ __s32 btf__find_by_name_kind(const struct btf *btf, const char *type_name,
>  
>  static bool btf_is_modifiable(const struct btf *btf)
>  {
> -	return (void *)btf->hdr != btf->raw_data;
> +	/* BTF is modifiable if split into multiple sections */
> +	return btf->modifiable;
>  }

One more thought.
If some kinds are not known to libbpf, BTF modifications are not safe,
e.g. endianness swap will fail for non-native endianness and anything
that depends on btf_field_iter will produce partial results.
So, maybe forbid conversion to modifiable in such scenarios?

[...]

Re: [PATCH v8 bpf-next 03/10] libbpf: use kind layout to compute an unknown kind size

[Eduard Zingerman]

On Mon, 2025-12-15 at 09:17 +0000, Alan Maguire wrote:

[...]

> @@ -395,8 +416,7 @@ static int btf_type_size(const struct btf_type *t)
>  	case BTF_KIND_DECL_TAG:
>  		return base_size + sizeof(struct btf_decl_tag);
>  	default:
> -		pr_debug("Unsupported BTF_KIND:%u\n", btf_kind(t));
> -		return -EINVAL;
> +		return btf_type_size_unknown(btf, t);
>  	}
>  }
>  

That's a matter of personal preference, of-course, but it seems to me
that using `kind_layouts` table from your next patch for size
computation for all kinds would be a bit more elegant.

Also, a question, should BTF validation check sizes for known kinds
and reject kind layout sections if those sizes differ from expected?

[...]

Re: [PATCH v8 bpf-next 06/10] btf: support kernel parsing of BTF with kind layout

[Eduard Zingerman]

On Mon, 2025-12-15 at 09:17 +0000, Alan Maguire wrote:

If strict kind layout checks are the goal, would it make sense to
check sizes declared in kind_layout for known types?

[...]

> @@ -5215,23 +5216,36 @@ static s32 btf_check_meta(struct btf_verifier_env *env,
>  		return -EINVAL;
>  	}
>  
> -	if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX ||
> -	    BTF_INFO_KIND(t->info) == BTF_KIND_UNKN) {
> +	if (!btf_name_offset_valid(env->btf, t->name_off)) {
> +		btf_verifier_log(env, "[%u] Invalid name_offset:%u",
> +				 env->log_type_id, t->name_off);
> +		return -EINVAL;
> +	}
> +
> +	if (BTF_INFO_KIND(t->info) == BTF_KIND_UNKN) {
>  		btf_verifier_log(env, "[%u] Invalid kind:%u",
>  				 env->log_type_id, BTF_INFO_KIND(t->info));
>  		return -EINVAL;
>  	}
>  
> -	if (!btf_name_offset_valid(env->btf, t->name_off)) {
> -		btf_verifier_log(env, "[%u] Invalid name_offset:%u",
> -				 env->log_type_id, t->name_off);
> +	if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX && env->btf->kind_layout &&
> +	    ((BTF_INFO_KIND(t->info) + 1) * sizeof(struct btf_kind_layout)) <
> +	     env->btf->hdr.kind_layout_len) {
> +		btf_verifier_log(env, "[%u] unknown but required kind %u",
> +				 env->log_type_id,
> +				 BTF_INFO_KIND(t->info));
>  		return -EINVAL;
> +	} else {
> +		if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX) {
> +			btf_verifier_log(env, "[%u] Invalid kind:%u",
> +					 env->log_type_id, BTF_INFO_KIND(t->info));
> +			return -EINVAL;
> +		}
> +		var_meta_size = btf_type_ops(t)->check_meta(env, t, meta_left);
> +		if (var_meta_size < 0)
> +			return var_meta_size;
>  	}
>  
> -	var_meta_size = btf_type_ops(t)->check_meta(env, t, meta_left);
> -	if (var_meta_size < 0)
> -		return var_meta_size;
> -
>  	meta_left -= var_meta_size;

It appears that a smaller change here would achieve same results:

    -        if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX ||
    +        u32 layout_kind_max = env->btf->hdr.kind_layout_len / sizeof(struct btf_kind_layout);
    +        if (BTF_INFO_KIND(t->info) > layout_kind_max ||
                 BTF_INFO_KIND(t->info) == BTF_KIND_UNKN) {
                     btf_verifier_log(env, "[%u] Invalid kind:%u",
                                      env->log_type_id, BTF_INFO_KIND(t->info));
                     return -EINVAL;
             }

    +        if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX) {
    +                btf_verifier_log(env, "[%u] unknown but required kind %u",
    +                                 env->log_type_id,
    +                                 BTF_INFO_KIND(t->info));
    +        }
    +
             if (!btf_name_offset_valid(env->btf, t->name_off)) {
                     btf_verifier_log(env, "[%u] Invalid name_offset:%u",
                                      env->log_type_id, t->name_off);

wdyt?

But tbh, the "unknown but required kind" message seems unnecessary,

>  
>  	return saved_meta_left - meta_left;
> @@ -5405,7 +5419,8 @@ static int btf_parse_str_sec(struct btf_verifier_env *env)
>  	start = btf->nohdr_data + hdr->str_off;
>  	end = start + hdr->str_len;
>  
> -	if (end != btf->data + btf->data_size) {
> +	if (hdr->hdr_len < sizeof(struct btf_header) &&
> +	    end != btf->data + btf->data_size) {

Given that btf_check_sec_info() checks for overlap and total size,
is this check needed at all?

>  		btf_verifier_log(env, "String section is not at the end");
>  		return -EINVAL;
>  	}

[...]

Re: [PATCH v8 bpf-next 02/10] libbpf: Support kind layout section handling in BTF

[Alan Maguire]

On 16/12/2025 06:01, Eduard Zingerman wrote:
> On Mon, 2025-12-15 at 09:17 +0000, Alan Maguire wrote:
> 
> [...]
> 
>> @@ -951,7 +1018,8 @@ __s32 btf__find_by_name_kind(const struct btf *btf, const char *type_name,
>>  
>>  static bool btf_is_modifiable(const struct btf *btf)
>>  {
>> -	return (void *)btf->hdr != btf->raw_data;
>> +	/* BTF is modifiable if split into multiple sections */
>> +	return btf->modifiable;
>>  }
> 
> One more thought.
> If some kinds are not known to libbpf, BTF modifications are not safe,
> e.g. endianness swap will fail for non-native endianness and anything
> that depends on btf_field_iter will produce partial results.
> So, maybe forbid conversion to modifiable in such scenarios?
>

Good idea; and ack on the embedded header in struct btf. Thanks!

Alan
 
> [...]

Re: [PATCH v8 bpf-next 03/10] libbpf: use kind layout to compute an unknown kind size

[Alan Maguire]

On 16/12/2025 06:07, Eduard Zingerman wrote:
> On Mon, 2025-12-15 at 09:17 +0000, Alan Maguire wrote:
> 
> [...]
> 
>> @@ -395,8 +416,7 @@ static int btf_type_size(const struct btf_type *t)
>>  	case BTF_KIND_DECL_TAG:
>>  		return base_size + sizeof(struct btf_decl_tag);
>>  	default:
>> -		pr_debug("Unsupported BTF_KIND:%u\n", btf_kind(t));
>> -		return -EINVAL;
>> +		return btf_type_size_unknown(btf, t);
>>  	}
>>  }
>>  
> 
> That's a matter of personal preference, of-course, but it seems to me
> that using `kind_layouts` table from your next patch for size
> computation for all kinds would be a bit more elegant.
> 
> Also, a question, should BTF validation check sizes for known kinds
> and reject kind layout sections if those sizes differ from expected?
>

yeah, I'd say we'd need your second suggestion for the first to be safe,
and it seems worthwhile doing both I think. Thanks!
 
> [...]

Re: [PATCH v8 bpf-next 01/10] btf: add kind layout encoding to UAPI

[Andrii Nakryiko]

On Mon, Dec 15, 2025 at 1:18 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>
> BTF kind layouts provide information to parse BTF kinds. By separating
> parsing BTF from using all the information it provides, we allow BTF
> to encode new features even if they cannot be used by readers. This
> will be helpful in particular for cases where older tools are used
> to parse newer BTF with kinds the older tools do not recognize;
> the BTF can still be parsed in such cases using kind layout.
>
> The intent is to support encoding of kind layouts optionally so that
> tools like pahole can add this information. For each kind, we record
>
> - length of singular element following struct btf_type
> - length of each of the btf_vlen() elements following
>
> The ideas here were discussed at [1], [2]; hence
>
> Suggested-by: Andrii Nakryiko <andrii@kernel.org>
> Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
>
> [1] https://lore.kernel.org/bpf/CAEf4BzYjWHRdNNw4B=eOXOs_ONrDwrgX4bn=Nuc1g8JPFC34MA@mail.gmail.com/
> [2] https://lore.kernel.org/bpf/20230531201936.1992188-1-alan.maguire@oracle.com/
> ---
>  include/uapi/linux/btf.h       | 11 +++++++++++
>  tools/include/uapi/linux/btf.h | 11 +++++++++++
>  2 files changed, 22 insertions(+)
>
> diff --git a/include/uapi/linux/btf.h b/include/uapi/linux/btf.h
> index 266d4ffa6c07..c1854a1c7b38 100644
> --- a/include/uapi/linux/btf.h
> +++ b/include/uapi/linux/btf.h
> @@ -8,6 +8,15 @@
>  #define BTF_MAGIC      0xeB9F
>  #define BTF_VERSION    1
>
> +/*
> + * kind layout section consists of a struct btf_kind_layout for each known
> + * kind at BTF encoding time.
> + */
> +struct btf_kind_layout {
> +       __u8 info_sz;           /* size of singular element after btf_type */
> +       __u8 elem_sz;           /* size of each of btf_vlen(t) elements */

So Eduard pointed out that at some point we discussed having a name of
a kind (i.e., "struct", "typedef", etc). By now I have no recollection
what were the arguments, do you remember? I'm not sure how I feel now
about having extra 4 bytes per kind, but that's not really a lot of
data (20*4 = 80 bytes added), so might as well add it, I suppose?

I think we were also discussing having flags per kind to designate
some extra semantics, where applicable. Again, don't remember
arguments for or against, but one case where I think this would be
very beneficial is when we add something like type_tag, which is
inevitably used from "normal" struct and will be almost inevitable in
normal vmlinux BTF. Think about it, we have some field which will be
CONST -> PTR -> TYPE_TAG -> STRUCT. That TYPE_TAG shouldn't just
totally break (old) bpftool's dump, as it really can be easily ignored
**if we know TYPE_TAG can be ignored and it is just a reference
type**. That reference type means that there is another type pointed
to using struct btf_type::type field (instead of that field being a
size).

So I think it would be nice to encode this as a flag that says a) kind
can be ignored without compromising type integrity (i.e., memory
layout is preserved) which will be true for all kinds of modifier
kinds (const/volatile/restrict/type_tag, even for typedef that should
be true) and b) kind is reference type, so struct btf_type::type is a
"pointer" to a valid other underlying type.

Thoughts?

> +};
> +
>  struct btf_header {
>         __u16   magic;
>         __u8    version;
> @@ -19,6 +28,8 @@ struct btf_header {
>         __u32   type_len;       /* length of type section       */
>         __u32   str_off;        /* offset of string section     */
>         __u32   str_len;        /* length of string section     */
> +       __u32   kind_layout_off;/* offset of kind layout section */
> +       __u32   kind_layout_len;/* length of kind layout section */

nit: kind_layout is a bit mouthful, have you considered "descr" (for
description/descriptor) or just "layout" as a name designator?



>  };
>
>  /* Max # of type identifier */
> diff --git a/tools/include/uapi/linux/btf.h b/tools/include/uapi/linux/btf.h
> index 266d4ffa6c07..c1854a1c7b38 100644
> --- a/tools/include/uapi/linux/btf.h
> +++ b/tools/include/uapi/linux/btf.h
> @@ -8,6 +8,15 @@
>  #define BTF_MAGIC      0xeB9F
>  #define BTF_VERSION    1
>
> +/*
> + * kind layout section consists of a struct btf_kind_layout for each known
> + * kind at BTF encoding time.
> + */
> +struct btf_kind_layout {
> +       __u8 info_sz;           /* size of singular element after btf_type */
> +       __u8 elem_sz;           /* size of each of btf_vlen(t) elements */
> +};
> +
>  struct btf_header {
>         __u16   magic;
>         __u8    version;
> @@ -19,6 +28,8 @@ struct btf_header {
>         __u32   type_len;       /* length of type section       */
>         __u32   str_off;        /* offset of string section     */
>         __u32   str_len;        /* length of string section     */
> +       __u32   kind_layout_off;/* offset of kind layout section */
> +       __u32   kind_layout_len;/* length of kind layout section */
>  };
>
>  /* Max # of type identifier */
> --
> 2.39.3
>

Re: [PATCH v8 bpf-next 02/10] libbpf: Support kind layout section handling in BTF

[Andrii Nakryiko]

On Mon, Dec 15, 2025 at 1:18 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>
> Support reading in kind layout fixing endian issues on reading;
> also support writing kind layout section to raw BTF object.
> There is not yet an API to populate the kind layout with meaningful
> information.
>
> As part of this, we need to consider multiple valid BTF header
> sizes; the original or the kind layout-extended headers.
> So to support this, the "struct btf" representation is modified
> to always allocate a "struct btf_header" and copy the valid
> portion from the raw data to it; this means we can always safely
> check fields like btf->hdr->kind_layout_len.
>
> Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
> ---
>  tools/lib/bpf/btf.c | 260 +++++++++++++++++++++++++++++++-------------
>  1 file changed, 183 insertions(+), 77 deletions(-)
>
> diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> index b136572e889a..8835aee6ee84 100644
> --- a/tools/lib/bpf/btf.c
> +++ b/tools/lib/bpf/btf.c
> @@ -40,42 +40,53 @@ struct btf {
>
>         /*
>          * When BTF is loaded from an ELF or raw memory it is stored
> -        * in a contiguous memory block. The hdr, type_data, and, strs_data
> +        * in a contiguous memory block. The  type_data, and, strs_data

nit: two spaces, and so many commas around and ;) let's leave Oxford
comma, but comma after and is weird

>          * point inside that memory region to their respective parts of BTF
>          * representation:
>          *
> -        * +--------------------------------+
> -        * |  Header  |  Types  |  Strings  |
> -        * +--------------------------------+
> -        * ^          ^         ^
> -        * |          |         |
> -        * hdr        |         |
> -        * types_data-+         |
> -        * strs_data------------+
> +        * +--------------------------------+---------------------+
> +        * |  Header  |  Types  |  Strings  |Optional kind layout |

Space missing, boo. Keep diagrams beautiful!..

> +        * +--------------------------------+---------------------+
> +        * ^          ^         ^           ^
> +        * |          |         |           |
> +        * raw_data   |         |           |
> +        * types_data-+         |           |
> +        * strs_data------------+           |
> +        * kind_layout----------------------+
> +        *
> +        * A separate struct btf_header is allocated for btf->hdr,
> +        * and header information is copied into it.  This allows us
> +        * to handle header data for various header formats; the original,
> +        * the extended header with kind layout, etc.
>          *
>          * If BTF data is later modified, e.g., due to types added or
>          * removed, BTF deduplication performed, etc, this contiguous
> -        * representation is broken up into three independently allocated
> -        * memory regions to be able to modify them independently.
> +        * representation is broken up into four independent memory
> +        * regions.
> +        *
>          * raw_data is nulled out at that point, but can be later allocated
>          * and cached again if user calls btf__raw_data(), at which point
> -        * raw_data will contain a contiguous copy of header, types, and
> -        * strings:
> +        * raw_data will contain a contiguous copy of header, types, strings
> +        * and optionally kind_layout.  kind_layout optionally points to a
> +        * kind_layout array - this allows us to encode information about
> +        * the kinds known at encoding time.  If kind_layout is NULL no
> +        * kind information is encoded.
>          *
> -        * +----------+  +---------+  +-----------+
> -        * |  Header  |  |  Types  |  |  Strings  |
> -        * +----------+  +---------+  +-----------+
> -        * ^             ^            ^
> -        * |             |            |
> -        * hdr           |            |
> -        * types_data----+            |
> -        * strset__data(strs_set)-----+
> +        * +----------+  +---------+  +-----------+   +-----------+
> +        * |  Header  |  |  Types  |  |  Strings  |   |kind_layout|
> +        * +----------+  +---------+  +-----------+   +-----------+

nit: spaces (and if we go with "layout" naming, this will be short and
beautiful " Layout " ;)

> +        * ^             ^            ^               ^
> +        * |             |            |               |
> +        * hdr           |            |               |
> +        * types_data----+            |               |
> +        * strset__data(strs_set)-----+               |
> +        * kind_layout--------------------------------+

[...]

> @@ -3888,7 +3989,7 @@ static int btf_dedup_strings(struct btf_dedup *d)
>
>         /* replace BTF string data and hash with deduped ones */
>         strset__free(d->btf->strs_set);
> -       d->btf->hdr->str_len = strset__data_size(d->strs_set);
> +       btf_hdr_update_str_len(d->btf, strset__data_size(d->strs_set));
>         d->btf->strs_set = d->strs_set;
>         d->strs_set = NULL;
>         d->btf->strs_deduped = true;
> @@ -5343,6 +5444,11 @@ static int btf_dedup_compact_types(struct btf_dedup *d)
>         d->btf->type_offs = new_offs;
>         d->btf->hdr->str_off = d->btf->hdr->type_len;
>         d->btf->raw_size = d->btf->hdr->hdr_len + d->btf->hdr->type_len + d->btf->hdr->str_len;
> +       if (d->btf->kind_layout) {
> +               d->btf->hdr->kind_layout_off = d->btf->hdr->str_off + roundup(d->btf->hdr->str_len,
> +                                                                             4);
> +               d->btf->raw_size = roundup(d->btf->raw_size, 4) + d->btf->hdr->kind_layout_len;

maybe put layout data after type data, but before strings? rounding up
string section which is byte-based feels weird. I think old libbpf
implementations should handle all this well, because btf_header
explicitly specifies string section offset, no?

> +       }
>         return 0;
>  }
>
> --
> 2.39.3
>

Re: [PATCH v8 bpf-next 03/10] libbpf: use kind layout to compute an unknown kind size

[Andrii Nakryiko]

On Mon, Dec 15, 2025 at 1:18 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>
> This allows BTF parsing to proceed even if we do not know the
> kind.
>
> Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
> ---
>  tools/lib/bpf/btf.c | 34 +++++++++++++++++++++++++++-------
>  1 file changed, 27 insertions(+), 7 deletions(-)
>
> diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> index 8835aee6ee84..3936ee04a46a 100644
> --- a/tools/lib/bpf/btf.c
> +++ b/tools/lib/bpf/btf.c
> @@ -359,7 +359,28 @@ static int btf_parse_kind_layout_sec(struct btf *btf)
>         return 0;
>  }
>
> -static int btf_type_size(const struct btf_type *t)
> +/* for unknown kinds, consult kind layout. */
> +static int btf_type_size_unknown(const struct btf *btf, const struct btf_type *t)
> +{
> +       int size = sizeof(struct btf_type);
> +       struct btf_kind_layout *k = NULL;
> +       __u16 vlen = btf_vlen(t);
> +       __u8 kind = btf_kind(t);
> +       __u32 off = kind * sizeof(struct btf_kind_layout);
> +
> +       if (!btf->kind_layout || off >= btf->hdr->kind_layout_len) {
> +               pr_debug("Unsupported BTF_KIND: %u\n", btf_kind(t));
> +               return -EINVAL;
> +       }
> +       k = btf->kind_layout + off;
> +
> +       size += k->info_sz;
> +       size += vlen * k->elem_sz;
> +
> +       return size;

return sizeof(struct btf_type) + k->info_sz + vlen * k->elem_sz;

Stateless, explicit, single line. All good properties.

> +}
> +
> +static int btf_type_size(const struct btf *btf, const struct btf_type *t)
>  {
>         const int base_size = sizeof(struct btf_type);
>         __u16 vlen = btf_vlen(t);
> @@ -395,8 +416,7 @@ static int btf_type_size(const struct btf_type *t)
>         case BTF_KIND_DECL_TAG:
>                 return base_size + sizeof(struct btf_decl_tag);
>         default:
> -               pr_debug("Unsupported BTF_KIND:%u\n", btf_kind(t));
> -               return -EINVAL;
> +               return btf_type_size_unknown(btf, t);
>         }
>  }
>

[...]

Re: [PATCH v8 bpf-next 03/10] libbpf: use kind layout to compute an unknown kind size

[Andrii Nakryiko]

On Tue, Dec 16, 2025 at 7:00 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>
> On 16/12/2025 06:07, Eduard Zingerman wrote:
> > On Mon, 2025-12-15 at 09:17 +0000, Alan Maguire wrote:
> >
> > [...]
> >
> >> @@ -395,8 +416,7 @@ static int btf_type_size(const struct btf_type *t)
> >>      case BTF_KIND_DECL_TAG:
> >>              return base_size + sizeof(struct btf_decl_tag);
> >>      default:
> >> -            pr_debug("Unsupported BTF_KIND:%u\n", btf_kind(t));
> >> -            return -EINVAL;
> >> +            return btf_type_size_unknown(btf, t);
> >>      }
> >>  }
> >>
> >
> > That's a matter of personal preference, of-course, but it seems to me
> > that using `kind_layouts` table from your next patch for size
> > computation for all kinds would be a bit more elegant.
> >
> > Also, a question, should BTF validation check sizes for known kinds
> > and reject kind layout sections if those sizes differ from expected?
> >
>
> yeah, I'd say we'd need your second suggestion for the first to be safe,
> and it seems worthwhile doing both I think. Thanks!

... but we will just blindly trust layout for unknown kinds, though?
So it's a bit inconsistent. I'd say let's keep it simple and don't
overdo the checking? btf_sanity_check() will validate that all known
kinds are well-formed, isn't that sufficient to ensure that subsequent
use of BTF data in libbpf won't crash? If some tool generated a subtly
invalid layout section which otherwise preserves BTF data
correctness... I don't know, this seems fine. The goal of sanity
checking is just to prevent more checks in all different places that
will subsequently rely on IDs being valid, and stuff like that. If
layout info is wrong for known kinds, so be it, we are not using that
information anyways.

>
> > [...]
>

Re: [PATCH v8 bpf-next 03/10] libbpf: use kind layout to compute an unknown kind size

[Eduard Zingerman]

On Tue, 2025-12-16 at 11:42 -0800, Andrii Nakryiko wrote:
> On Tue, Dec 16, 2025 at 7:00 AM Alan Maguire <alan.maguire@oracle.com> wrote:
> > 
> > On 16/12/2025 06:07, Eduard Zingerman wrote:
> > > On Mon, 2025-12-15 at 09:17 +0000, Alan Maguire wrote:
> > > 
> > > [...]
> > > 
> > > > @@ -395,8 +416,7 @@ static int btf_type_size(const struct btf_type *t)
> > > >      case BTF_KIND_DECL_TAG:
> > > >              return base_size + sizeof(struct btf_decl_tag);
> > > >      default:
> > > > -            pr_debug("Unsupported BTF_KIND:%u\n", btf_kind(t));
> > > > -            return -EINVAL;
> > > > +            return btf_type_size_unknown(btf, t);
> > > >      }
> > > >  }
> > > > 
> > > 
> > > That's a matter of personal preference, of-course, but it seems to me
> > > that using `kind_layouts` table from your next patch for size
> > > computation for all kinds would be a bit more elegant.
> > > 
> > > Also, a question, should BTF validation check sizes for known kinds
> > > and reject kind layout sections if those sizes differ from expected?
> > > 
> > 
> > yeah, I'd say we'd need your second suggestion for the first to be safe,
> > and it seems worthwhile doing both I think. Thanks!
> 
> ... but we will just blindly trust layout for unknown kinds, though?
> So it's a bit inconsistent. I'd say let's keep it simple and don't
> overdo the checking? btf_sanity_check() will validate that all known
> kinds are well-formed, isn't that sufficient to ensure that subsequent
> use of BTF data in libbpf won't crash? If some tool generated a subtly
> invalid layout section which otherwise preserves BTF data
> correctness... I don't know, this seems fine. The goal of sanity
> checking is just to prevent more checks in all different places that
> will subsequently rely on IDs being valid, and stuff like that. If
> layout info is wrong for known kinds, so be it, we are not using that
> information anyways.

Ignoring layout information for known kinds can lead to weird
scenarios: e.g. suppose type size is N, but kind layout specifies that
it is M > N, and the tool generating BTF uses M to actually layout the
binary data. We are being a bit inconsistent with such encoding.

Re: [PATCH v8 bpf-next 04/10] libbpf: Add kind layout encoding support

[Andrii Nakryiko]

On Mon, Dec 15, 2025 at 1:18 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>
> Support encoding of BTF kind layout data via btf__new_empty_opts().
>
> Current supported opts are base_btf and add_kind_layout.
>
> Kind layout information is maintained in btf.c in the
> kind_layouts[] array; when BTF is created with the
> add_kind_layout option it represents the current view
> of supported BTF kinds.
>
> Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
> ---
>  tools/lib/bpf/btf.c      | 60 ++++++++++++++++++++++++++++++++++++++--
>  tools/lib/bpf/btf.h      | 20 ++++++++++++++
>  tools/lib/bpf/libbpf.map |  1 +
>  3 files changed, 78 insertions(+), 3 deletions(-)
>
> diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> index 3936ee04a46a..589a9632a630 100644
> --- a/tools/lib/bpf/btf.c
> +++ b/tools/lib/bpf/btf.c
> @@ -29,6 +29,35 @@
>
>  static struct btf_type btf_void;
>
> +/* Describe how kinds are laid out; some have a singular element following the "struct btf_type",
> + * some have BTF_INFO_VLEN(t->info) elements.  Specify sizes for both.  Flags are currently unused.
> + * Kind layout can be optionally added to the BTF representation in a dedicated section to
> + * facilitate parsing.  New kinds must be added here.
> + */
> +struct btf_kind_layout kind_layouts[NR_BTF_KINDS] = {

static?

> +/*     singular element size           vlen element(s) size */
> +{      0,                              0                               }, /* _UNKN */

use

[BTF_KIND_UNKN] = { 0, 0 },
[BTF_KIND_INT] = { sizeof(__u32), 0 },
and so on

it's succinct and self-descriptive

[...]

>  static struct btf *btf_new(const void *data, __u32 size, struct btf *base_btf, bool is_mmap)
> diff --git a/tools/lib/bpf/btf.h b/tools/lib/bpf/btf.h
> index cc01494d6210..dcc166834937 100644
> --- a/tools/lib/bpf/btf.h
> +++ b/tools/lib/bpf/btf.h
> @@ -109,6 +109,26 @@ LIBBPF_API struct btf *btf__new_empty(void);
>   */
>  LIBBPF_API struct btf *btf__new_empty_split(struct btf *base_btf);
>
> +struct btf_new_opts {
> +       size_t sz;
> +       struct btf *base_btf;   /* optional base BTF */
> +       bool add_kind_layout;   /* add BTF kind layout information */

Why an option? Let's just always emit this. And teach libbpf to
"sanitize" layout if the host kernel doesn't support it. Looking at
kernel validation code, it should be enough to just zero-out layout
len/off fields in the header. (Maybe we'll need to zero out layout
data itself, but I don't see any zero checks for that, so probably is
fine without that).

As a general rule, let's avoid adding options if we can just make it
work unconditionally. Options are never a great solution, it's just
sometimes a necessary evil.

> +       size_t:0;
> +};
> +#define btf_new_opts__last_field add_kind_layout
> +
> +/**
> + * @brief **btf__new_empty_opts()** creates an unpopulated BTF object with
> + * optional *base_btf* and BTF kind layout description if *add_kind_layout*
> + * is set
> + * @return new BTF object instance which has to be eventually freed with
> + * **btf__free()**
> + *
> + * On error, NULL is returned and the thread-local `errno` variable is
> + * set to the error code.
> + */
> +LIBBPF_API struct btf *btf__new_empty_opts(struct btf_new_opts *opts);
> +
>  /**
>   * @brief **btf__distill_base()** creates new versions of the split BTF
>   * *src_btf* and its base BTF. The new base BTF will only contain the types
> diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map
> index 84fb90a016c9..0fb9a1f70e72 100644
> --- a/tools/lib/bpf/libbpf.map
> +++ b/tools/lib/bpf/libbpf.map
> @@ -453,4 +453,5 @@ LIBBPF_1.7.0 {
>                 bpf_map__exclusive_program;
>                 bpf_prog_assoc_struct_ops;
>                 bpf_program__assoc_struct_ops;
> +               btf__new_empty_opts;
>  } LIBBPF_1.6.0;
> --
> 2.39.3
>

Re: [PATCH v8 bpf-next 03/10] libbpf: use kind layout to compute an unknown kind size

[Andrii Nakryiko]

On Tue, Dec 16, 2025 at 11:58 AM Eduard Zingerman <eddyz87@gmail.com> wrote:
>
> On Tue, 2025-12-16 at 11:42 -0800, Andrii Nakryiko wrote:
> > On Tue, Dec 16, 2025 at 7:00 AM Alan Maguire <alan.maguire@oracle.com> wrote:
> > >
> > > On 16/12/2025 06:07, Eduard Zingerman wrote:
> > > > On Mon, 2025-12-15 at 09:17 +0000, Alan Maguire wrote:
> > > >
> > > > [...]
> > > >
> > > > > @@ -395,8 +416,7 @@ static int btf_type_size(const struct btf_type *t)
> > > > >      case BTF_KIND_DECL_TAG:
> > > > >              return base_size + sizeof(struct btf_decl_tag);
> > > > >      default:
> > > > > -            pr_debug("Unsupported BTF_KIND:%u\n", btf_kind(t));
> > > > > -            return -EINVAL;
> > > > > +            return btf_type_size_unknown(btf, t);
> > > > >      }
> > > > >  }
> > > > >
> > > >
> > > > That's a matter of personal preference, of-course, but it seems to me
> > > > that using `kind_layouts` table from your next patch for size
> > > > computation for all kinds would be a bit more elegant.
> > > >
> > > > Also, a question, should BTF validation check sizes for known kinds
> > > > and reject kind layout sections if those sizes differ from expected?
> > > >
> > >
> > > yeah, I'd say we'd need your second suggestion for the first to be safe,
> > > and it seems worthwhile doing both I think. Thanks!
> >
> > ... but we will just blindly trust layout for unknown kinds, though?
> > So it's a bit inconsistent. I'd say let's keep it simple and don't
> > overdo the checking? btf_sanity_check() will validate that all known
> > kinds are well-formed, isn't that sufficient to ensure that subsequent
> > use of BTF data in libbpf won't crash? If some tool generated a subtly
> > invalid layout section which otherwise preserves BTF data
> > correctness... I don't know, this seems fine. The goal of sanity
> > checking is just to prevent more checks in all different places that
> > will subsequently rely on IDs being valid, and stuff like that. If
> > layout info is wrong for known kinds, so be it, we are not using that
> > information anyways.
>
> Ignoring layout information for known kinds can lead to weird
> scenarios: e.g. suppose type size is N, but kind layout specifies that
> it is M > N, and the tool generating BTF uses M to actually layout the
> binary data. We are being a bit inconsistent with such encoding.

Who are "we" here? The tool that emitted incorrect layout information
-- yep, for sure. (But that shouldn't happen for correct BTF.)

We do btf_sanity_check() upfront to minimize various sanity checks
spread out in libbpf code when using BTF data later on. But the goal
there is not really to check "100% standards conformance" of whatever
BTF we are working with. Kernel is way more concerned about validity
and not letting anything unexpected get through, but libbpf is in user
space and it's a bit different approach there.

As long as BTF looks structurally sound (btf_sanity_check), it should
be fine for our needs. If BTF is corrupted or just uses invalid ID, or
whatnot, it will eventually fail somewhere, most probably. But the
goal is not to have NULL derefs and stuff like that.

Basically what I'm trying to say is that if some tool intentionally or
unintentionally generates invalid layout information, but the rest of
data is valid, and libbpf doesn't look at layout and otherwise makes
fine use of BTF. Then that's fine with me, that BTF fulfills its duty
as far as libbpf is concerned. Libbpf here is just a consumer that
tries to be as permissive (unlike kernel) as possible, while not
leading to a process crash.

With double checking that layout info matches our implicit knowledge
of type layout we are starting to move into BTF verification a bit,
IMO. And I think that's misleading and unnecessary.

Re: [PATCH v8 bpf-next 06/10] btf: support kernel parsing of BTF with kind layout

[Andrii Nakryiko]

On Mon, Dec 15, 2025 at 10:51 PM Eduard Zingerman <eddyz87@gmail.com> wrote:
>
> On Mon, 2025-12-15 at 09:17 +0000, Alan Maguire wrote:
>
> If strict kind layout checks are the goal, would it make sense to
> check sizes declared in kind_layout for known types?
>
> [...]
>
> > @@ -5215,23 +5216,36 @@ static s32 btf_check_meta(struct btf_verifier_env *env,
> >               return -EINVAL;
> >       }
> >
> > -     if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX ||
> > -         BTF_INFO_KIND(t->info) == BTF_KIND_UNKN) {
> > +     if (!btf_name_offset_valid(env->btf, t->name_off)) {
> > +             btf_verifier_log(env, "[%u] Invalid name_offset:%u",
> > +                              env->log_type_id, t->name_off);
> > +             return -EINVAL;
> > +     }
> > +
> > +     if (BTF_INFO_KIND(t->info) == BTF_KIND_UNKN) {
> >               btf_verifier_log(env, "[%u] Invalid kind:%u",
> >                                env->log_type_id, BTF_INFO_KIND(t->info));
> >               return -EINVAL;
> >       }
> >
> > -     if (!btf_name_offset_valid(env->btf, t->name_off)) {
> > -             btf_verifier_log(env, "[%u] Invalid name_offset:%u",
> > -                              env->log_type_id, t->name_off);
> > +     if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX && env->btf->kind_layout &&
> > +         ((BTF_INFO_KIND(t->info) + 1) * sizeof(struct btf_kind_layout)) <
> > +          env->btf->hdr.kind_layout_len) {
> > +             btf_verifier_log(env, "[%u] unknown but required kind %u",
> > +                              env->log_type_id,
> > +                              BTF_INFO_KIND(t->info));
> >               return -EINVAL;
> > +     } else {
> > +             if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX) {
> > +                     btf_verifier_log(env, "[%u] Invalid kind:%u",
> > +                                      env->log_type_id, BTF_INFO_KIND(t->info));
> > +                     return -EINVAL;
> > +             }
> > +             var_meta_size = btf_type_ops(t)->check_meta(env, t, meta_left);
> > +             if (var_meta_size < 0)
> > +                     return var_meta_size;
> >       }
> >
> > -     var_meta_size = btf_type_ops(t)->check_meta(env, t, meta_left);
> > -     if (var_meta_size < 0)
> > -             return var_meta_size;
> > -
> >       meta_left -= var_meta_size;
>
> It appears that a smaller change here would achieve same results:
>
>     -        if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX ||
>     +        u32 layout_kind_max = env->btf->hdr.kind_layout_len / sizeof(struct btf_kind_layout);
>     +        if (BTF_INFO_KIND(t->info) > layout_kind_max ||
>                  BTF_INFO_KIND(t->info) == BTF_KIND_UNKN) {
>                      btf_verifier_log(env, "[%u] Invalid kind:%u",
>                                       env->log_type_id, BTF_INFO_KIND(t->info));
>                      return -EINVAL;
>              }
>
>     +        if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX) {
>     +                btf_verifier_log(env, "[%u] unknown but required kind %u",
>     +                                 env->log_type_id,
>     +                                 BTF_INFO_KIND(t->info));
>     +        }
>     +
>              if (!btf_name_offset_valid(env->btf, t->name_off)) {
>                      btf_verifier_log(env, "[%u] Invalid name_offset:%u",
>                                       env->log_type_id, t->name_off);
>
> wdyt?
>
> But tbh, the "unknown but required kind" message seems unnecessary,
>

Hm.. Do I understand that this patch will actually allow uploading BTF
with some kinds that are unknown to the kernel? I don't think we
should allow this. If the kernel sees a kind that it knows nothing
about, it should reject the BTF. libbpf will sanitize such BTF so that
the host kernel never sees unknown/unsupported BTF kind.

I think doing layout info validation is a good thing, I'd keep it, but
having layout information is not a substitute for kernel knowing full
semantics of the kind. Let's not relax kernel-side validation for BTF.

> >
> >       return saved_meta_left - meta_left;
> > @@ -5405,7 +5419,8 @@ static int btf_parse_str_sec(struct btf_verifier_env *env)
> >       start = btf->nohdr_data + hdr->str_off;
> >       end = start + hdr->str_len;
> >
> > -     if (end != btf->data + btf->data_size) {
> > +     if (hdr->hdr_len < sizeof(struct btf_header) &&
> > +         end != btf->data + btf->data_size) {
>
> Given that btf_check_sec_info() checks for overlap and total size,
> is this check needed at all?
>
> >               btf_verifier_log(env, "String section is not at the end");
> >               return -EINVAL;
> >       }
>
> [...]

Re: [PATCH v8 bpf-next 03/10] libbpf: use kind layout to compute an unknown kind size

[Eduard Zingerman]

On Tue, 2025-12-16 at 13:11 -0800, Andrii Nakryiko wrote:
> On Tue, Dec 16, 2025 at 11:58 AM Eduard Zingerman <eddyz87@gmail.com> wrote:
> > 
> > On Tue, 2025-12-16 at 11:42 -0800, Andrii Nakryiko wrote:
> > > On Tue, Dec 16, 2025 at 7:00 AM Alan Maguire <alan.maguire@oracle.com> wrote:
> > > > 
> > > > On 16/12/2025 06:07, Eduard Zingerman wrote:
> > > > > On Mon, 2025-12-15 at 09:17 +0000, Alan Maguire wrote:
> > > > > 
> > > > > [...]
> > > > > 
> > > > > > @@ -395,8 +416,7 @@ static int btf_type_size(const struct btf_type *t)
> > > > > >      case BTF_KIND_DECL_TAG:
> > > > > >              return base_size + sizeof(struct btf_decl_tag);
> > > > > >      default:
> > > > > > -            pr_debug("Unsupported BTF_KIND:%u\n", btf_kind(t));
> > > > > > -            return -EINVAL;
> > > > > > +            return btf_type_size_unknown(btf, t);
> > > > > >      }
> > > > > >  }
> > > > > > 
> > > > > 
> > > > > That's a matter of personal preference, of-course, but it seems to me
> > > > > that using `kind_layouts` table from your next patch for size
> > > > > computation for all kinds would be a bit more elegant.
> > > > > 
> > > > > Also, a question, should BTF validation check sizes for known kinds
> > > > > and reject kind layout sections if those sizes differ from expected?
> > > > > 
> > > > 
> > > > yeah, I'd say we'd need your second suggestion for the first to be safe,
> > > > and it seems worthwhile doing both I think. Thanks!
> > > 
> > > ... but we will just blindly trust layout for unknown kinds, though?
> > > So it's a bit inconsistent. I'd say let's keep it simple and don't
> > > overdo the checking? btf_sanity_check() will validate that all known
> > > kinds are well-formed, isn't that sufficient to ensure that subsequent
> > > use of BTF data in libbpf won't crash? If some tool generated a subtly
> > > invalid layout section which otherwise preserves BTF data
> > > correctness... I don't know, this seems fine. The goal of sanity
> > > checking is just to prevent more checks in all different places that
> > > will subsequently rely on IDs being valid, and stuff like that. If
> > > layout info is wrong for known kinds, so be it, we are not using that
> > > information anyways.
> > 
> > Ignoring layout information for known kinds can lead to weird
> > scenarios: e.g. suppose type size is N, but kind layout specifies that
> > it is M > N, and the tool generating BTF uses M to actually layout the
> > binary data. We are being a bit inconsistent with such encoding.
> 
> Who are "we" here? The tool that emitted incorrect layout information
> -- yep, for sure. (But that shouldn't happen for correct BTF.)
> 
> We do btf_sanity_check() upfront to minimize various sanity checks
> spread out in libbpf code when using BTF data later on. But the goal
> there is not really to check "100% standards conformance" of whatever
> BTF we are working with. Kernel is way more concerned about validity
> and not letting anything unexpected get through, but libbpf is in user
> space and it's a bit different approach there.
> 
> As long as BTF looks structurally sound (btf_sanity_check), it should
> be fine for our needs. If BTF is corrupted or just uses invalid ID, or
> whatnot, it will eventually fail somewhere, most probably. But the
> goal is not to have NULL derefs and stuff like that.

Introducing layout info into format provides an alternative definition
for structural soundness. E.g. some types or vlen elements can have
padding in the end all of a sudden.

Using this info for some types but not the others is inconsistent.
Given that BTF rewrites would only be unsound in presence of unknown
types the whole feature looks questionable to me.

> Basically what I'm trying to say is that if some tool intentionally or
> unintentionally generates invalid layout information, but the rest of
> data is valid, and libbpf doesn't look at layout and otherwise makes
> fine use of BTF. Then that's fine with me, that BTF fulfills its duty
> as far as libbpf is concerned. Libbpf here is just a consumer that
> tries to be as permissive (unlike kernel) as possible, while not
> leading to a process crash.
> 
> With double checking that layout info matches our implicit knowledge
> of type layout we are starting to move into BTF verification a bit,
> IMO. And I think that's misleading and unnecessary.

Re: [PATCH v8 bpf-next 06/10] btf: support kernel parsing of BTF with kind layout

[Eduard Zingerman]

On Tue, 2025-12-16 at 13:21 -0800, Andrii Nakryiko wrote:
> On Mon, Dec 15, 2025 at 10:51 PM Eduard Zingerman <eddyz87@gmail.com> wrote:
> > 
> > On Mon, 2025-12-15 at 09:17 +0000, Alan Maguire wrote:
> > 
> > If strict kind layout checks are the goal, would it make sense to
> > check sizes declared in kind_layout for known types?
> > 
> > [...]
> > 
> > > @@ -5215,23 +5216,36 @@ static s32 btf_check_meta(struct btf_verifier_env *env,
> > >               return -EINVAL;
> > >       }
> > > 
> > > -     if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX ||
> > > -         BTF_INFO_KIND(t->info) == BTF_KIND_UNKN) {
> > > +     if (!btf_name_offset_valid(env->btf, t->name_off)) {
> > > +             btf_verifier_log(env, "[%u] Invalid name_offset:%u",
> > > +                              env->log_type_id, t->name_off);
> > > +             return -EINVAL;
> > > +     }
> > > +
> > > +     if (BTF_INFO_KIND(t->info) == BTF_KIND_UNKN) {
> > >               btf_verifier_log(env, "[%u] Invalid kind:%u",
> > >                                env->log_type_id, BTF_INFO_KIND(t->info));
> > >               return -EINVAL;
> > >       }
> > > 
> > > -     if (!btf_name_offset_valid(env->btf, t->name_off)) {
> > > -             btf_verifier_log(env, "[%u] Invalid name_offset:%u",
> > > -                              env->log_type_id, t->name_off);
> > > +     if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX && env->btf->kind_layout &&
> > > +         ((BTF_INFO_KIND(t->info) + 1) * sizeof(struct btf_kind_layout)) <
> > > +          env->btf->hdr.kind_layout_len) {
> > > +             btf_verifier_log(env, "[%u] unknown but required kind %u",
> > > +                              env->log_type_id,
> > > +                              BTF_INFO_KIND(t->info));
> > >               return -EINVAL;
> > > +     } else {
> > > +             if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX) {
> > > +                     btf_verifier_log(env, "[%u] Invalid kind:%u",
> > > +                                      env->log_type_id, BTF_INFO_KIND(t->info));
> > > +                     return -EINVAL;
> > > +             }
> > > +             var_meta_size = btf_type_ops(t)->check_meta(env, t, meta_left);
> > > +             if (var_meta_size < 0)
> > > +                     return var_meta_size;
> > >       }
> > > 
> > > -     var_meta_size = btf_type_ops(t)->check_meta(env, t, meta_left);
> > > -     if (var_meta_size < 0)
> > > -             return var_meta_size;
> > > -
> > >       meta_left -= var_meta_size;
> > 
> > It appears that a smaller change here would achieve same results:
> > 
> >     -        if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX ||
> >     +        u32 layout_kind_max = env->btf->hdr.kind_layout_len / sizeof(struct btf_kind_layout);
> >     +        if (BTF_INFO_KIND(t->info) > layout_kind_max ||
> >                  BTF_INFO_KIND(t->info) == BTF_KIND_UNKN) {
> >                      btf_verifier_log(env, "[%u] Invalid kind:%u",
> >                                       env->log_type_id, BTF_INFO_KIND(t->info));
> >                      return -EINVAL;
> >              }
> > 
> >     +        if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX) {
> >     +                btf_verifier_log(env, "[%u] unknown but required kind %u",
> >     +                                 env->log_type_id,
> >     +                                 BTF_INFO_KIND(t->info));
> >     +        }
> >     +
> >              if (!btf_name_offset_valid(env->btf, t->name_off)) {
> >                      btf_verifier_log(env, "[%u] Invalid name_offset:%u",
> >                                       env->log_type_id, t->name_off);
> > 
> > wdyt?
> > 
> > But tbh, the "unknown but required kind" message seems unnecessary,
> > 
> 
> Hm.. Do I understand that this patch will actually allow uploading BTF
> with some kinds that are unknown to the kernel? I don't think we
> should allow this. If the kernel sees a kind that it knows nothing
> about, it should reject the BTF. libbpf will sanitize such BTF so that
> the host kernel never sees unknown/unsupported BTF kind.
> 
> I think doing layout info validation is a good thing, I'd keep it, but
> having layout information is not a substitute for kernel knowing full
> semantics of the kind. Let's not relax kernel-side validation for BTF.

I don't think this patch relaxes anything.
It just generates a different error message for kinds that are unknown
to kernel and:
- described in the layout info
- not described in the layout info.
(Personally, I don't think that this differentiation is necessary).

Otherwise the patch does basic sanity checks for layout section itself.

Re: [PATCH v8 bpf-next 06/10] btf: support kernel parsing of BTF with kind layout

[Andrii Nakryiko]

On Tue, Dec 16, 2025 at 1:25 PM Eduard Zingerman <eddyz87@gmail.com> wrote:
>
> On Tue, 2025-12-16 at 13:21 -0800, Andrii Nakryiko wrote:
> > On Mon, Dec 15, 2025 at 10:51 PM Eduard Zingerman <eddyz87@gmail.com> wrote:
> > >
> > > On Mon, 2025-12-15 at 09:17 +0000, Alan Maguire wrote:
> > >
> > > If strict kind layout checks are the goal, would it make sense to
> > > check sizes declared in kind_layout for known types?
> > >
> > > [...]
> > >
> > > > @@ -5215,23 +5216,36 @@ static s32 btf_check_meta(struct btf_verifier_env *env,
> > > >               return -EINVAL;
> > > >       }
> > > >
> > > > -     if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX ||
> > > > -         BTF_INFO_KIND(t->info) == BTF_KIND_UNKN) {
> > > > +     if (!btf_name_offset_valid(env->btf, t->name_off)) {
> > > > +             btf_verifier_log(env, "[%u] Invalid name_offset:%u",
> > > > +                              env->log_type_id, t->name_off);
> > > > +             return -EINVAL;
> > > > +     }
> > > > +
> > > > +     if (BTF_INFO_KIND(t->info) == BTF_KIND_UNKN) {
> > > >               btf_verifier_log(env, "[%u] Invalid kind:%u",
> > > >                                env->log_type_id, BTF_INFO_KIND(t->info));
> > > >               return -EINVAL;
> > > >       }
> > > >
> > > > -     if (!btf_name_offset_valid(env->btf, t->name_off)) {
> > > > -             btf_verifier_log(env, "[%u] Invalid name_offset:%u",
> > > > -                              env->log_type_id, t->name_off);
> > > > +     if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX && env->btf->kind_layout &&
> > > > +         ((BTF_INFO_KIND(t->info) + 1) * sizeof(struct btf_kind_layout)) <
> > > > +          env->btf->hdr.kind_layout_len) {
> > > > +             btf_verifier_log(env, "[%u] unknown but required kind %u",
> > > > +                              env->log_type_id,
> > > > +                              BTF_INFO_KIND(t->info));
> > > >               return -EINVAL;
> > > > +     } else {
> > > > +             if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX) {
> > > > +                     btf_verifier_log(env, "[%u] Invalid kind:%u",
> > > > +                                      env->log_type_id, BTF_INFO_KIND(t->info));
> > > > +                     return -EINVAL;
> > > > +             }
> > > > +             var_meta_size = btf_type_ops(t)->check_meta(env, t, meta_left);
> > > > +             if (var_meta_size < 0)
> > > > +                     return var_meta_size;
> > > >       }
> > > >
> > > > -     var_meta_size = btf_type_ops(t)->check_meta(env, t, meta_left);
> > > > -     if (var_meta_size < 0)
> > > > -             return var_meta_size;
> > > > -
> > > >       meta_left -= var_meta_size;
> > >
> > > It appears that a smaller change here would achieve same results:
> > >
> > >     -        if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX ||
> > >     +        u32 layout_kind_max = env->btf->hdr.kind_layout_len / sizeof(struct btf_kind_layout);
> > >     +        if (BTF_INFO_KIND(t->info) > layout_kind_max ||
> > >                  BTF_INFO_KIND(t->info) == BTF_KIND_UNKN) {
> > >                      btf_verifier_log(env, "[%u] Invalid kind:%u",
> > >                                       env->log_type_id, BTF_INFO_KIND(t->info));
> > >                      return -EINVAL;
> > >              }
> > >
> > >     +        if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX) {
> > >     +                btf_verifier_log(env, "[%u] unknown but required kind %u",
> > >     +                                 env->log_type_id,
> > >     +                                 BTF_INFO_KIND(t->info));

hm... did you mean to have return -EINVAL here? Then it wouldn't let
through anything that's not known to verifier.

> > >     +        }
> > >     +
> > >              if (!btf_name_offset_valid(env->btf, t->name_off)) {
> > >                      btf_verifier_log(env, "[%u] Invalid name_offset:%u",
> > >                                       env->log_type_id, t->name_off);
> > >
> > > wdyt?
> > >
> > > But tbh, the "unknown but required kind" message seems unnecessary,
> > >
> >
> > Hm.. Do I understand that this patch will actually allow uploading BTF
> > with some kinds that are unknown to the kernel? I don't think we
> > should allow this. If the kernel sees a kind that it knows nothing
> > about, it should reject the BTF. libbpf will sanitize such BTF so that
> > the host kernel never sees unknown/unsupported BTF kind.
> >
> > I think doing layout info validation is a good thing, I'd keep it, but
> > having layout information is not a substitute for kernel knowing full
> > semantics of the kind. Let's not relax kernel-side validation for BTF.
>
> I don't think this patch relaxes anything.

See above, I got confused by your proposed refactoring that does
change the behavior (probably missed that return -EINVAL, though).

> It just generates a different error message for kinds that are unknown
> to kernel and:
> - described in the layout info
> - not described in the layout info.
> (Personally, I don't think that this differentiation is necessary).

agreed, there is little point. It's just an unknown kind to the kernel

>
> Otherwise the patch does basic sanity checks for layout section itself.

Re: [PATCH v8 bpf-next 06/10] btf: support kernel parsing of BTF with kind layout

[Eduard Zingerman]

On Tue, 2025-12-16 at 14:09 -0800, Andrii Nakryiko wrote:
> On Tue, Dec 16, 2025 at 1:25 PM Eduard Zingerman <eddyz87@gmail.com> wrote:
> > 
> > On Tue, 2025-12-16 at 13:21 -0800, Andrii Nakryiko wrote:
> > > On Mon, Dec 15, 2025 at 10:51 PM Eduard Zingerman <eddyz87@gmail.com> wrote:
> > > > 
> > > > On Mon, 2025-12-15 at 09:17 +0000, Alan Maguire wrote:
> > > > 
> > > > If strict kind layout checks are the goal, would it make sense to
> > > > check sizes declared in kind_layout for known types?
> > > > 
> > > > [...]
> > > > 
> > > > > @@ -5215,23 +5216,36 @@ static s32 btf_check_meta(struct btf_verifier_env *env,
> > > > >               return -EINVAL;
> > > > >       }
> > > > > 
> > > > > -     if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX ||
> > > > > -         BTF_INFO_KIND(t->info) == BTF_KIND_UNKN) {
> > > > > +     if (!btf_name_offset_valid(env->btf, t->name_off)) {
> > > > > +             btf_verifier_log(env, "[%u] Invalid name_offset:%u",
> > > > > +                              env->log_type_id, t->name_off);
> > > > > +             return -EINVAL;
> > > > > +     }
> > > > > +
> > > > > +     if (BTF_INFO_KIND(t->info) == BTF_KIND_UNKN) {
> > > > >               btf_verifier_log(env, "[%u] Invalid kind:%u",
> > > > >                                env->log_type_id, BTF_INFO_KIND(t->info));
> > > > >               return -EINVAL;
> > > > >       }
> > > > > 
> > > > > -     if (!btf_name_offset_valid(env->btf, t->name_off)) {
> > > > > -             btf_verifier_log(env, "[%u] Invalid name_offset:%u",
> > > > > -                              env->log_type_id, t->name_off);
> > > > > +     if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX && env->btf->kind_layout &&
> > > > > +         ((BTF_INFO_KIND(t->info) + 1) * sizeof(struct btf_kind_layout)) <
> > > > > +          env->btf->hdr.kind_layout_len) {
> > > > > +             btf_verifier_log(env, "[%u] unknown but required kind %u",
> > > > > +                              env->log_type_id,
> > > > > +                              BTF_INFO_KIND(t->info));
> > > > >               return -EINVAL;
> > > > > +     } else {
> > > > > +             if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX) {
> > > > > +                     btf_verifier_log(env, "[%u] Invalid kind:%u",
> > > > > +                                      env->log_type_id, BTF_INFO_KIND(t->info));
> > > > > +                     return -EINVAL;
> > > > > +             }
> > > > > +             var_meta_size = btf_type_ops(t)->check_meta(env, t, meta_left);
> > > > > +             if (var_meta_size < 0)
> > > > > +                     return var_meta_size;
> > > > >       }
> > > > > 
> > > > > -     var_meta_size = btf_type_ops(t)->check_meta(env, t, meta_left);
> > > > > -     if (var_meta_size < 0)
> > > > > -             return var_meta_size;
> > > > > -
> > > > >       meta_left -= var_meta_size;
> > > > 
> > > > It appears that a smaller change here would achieve same results:
> > > > 
> > > >     -        if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX ||
> > > >     +        u32 layout_kind_max = env->btf->hdr.kind_layout_len / sizeof(struct btf_kind_layout);
> > > >     +        if (BTF_INFO_KIND(t->info) > layout_kind_max ||
> > > >                  BTF_INFO_KIND(t->info) == BTF_KIND_UNKN) {
> > > >                      btf_verifier_log(env, "[%u] Invalid kind:%u",
> > > >                                       env->log_type_id, BTF_INFO_KIND(t->info));
> > > >                      return -EINVAL;
> > > >              }
> > > > 
> > > >     +        if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX) {
> > > >     +                btf_verifier_log(env, "[%u] unknown but required kind %u",
> > > >     +                                 env->log_type_id,
> > > >     +                                 BTF_INFO_KIND(t->info));
> 
> hm... did you mean to have return -EINVAL here? Then it wouldn't let
> through anything that's not known to verifier.

Oops, yeap, I meant to return error here.
As Alan's original patch does.
Sorry for confusion.

[...]

Re: [PATCH v8 bpf-next 03/10] libbpf: use kind layout to compute an unknown kind size

[Andrii Nakryiko]

On Tue, Dec 16, 2025 at 1:21 PM Eduard Zingerman <eddyz87@gmail.com> wrote:
>
> On Tue, 2025-12-16 at 13:11 -0800, Andrii Nakryiko wrote:
> > On Tue, Dec 16, 2025 at 11:58 AM Eduard Zingerman <eddyz87@gmail.com> wrote:
> > >
> > > On Tue, 2025-12-16 at 11:42 -0800, Andrii Nakryiko wrote:
> > > > On Tue, Dec 16, 2025 at 7:00 AM Alan Maguire <alan.maguire@oracle.com> wrote:
> > > > >
> > > > > On 16/12/2025 06:07, Eduard Zingerman wrote:
> > > > > > On Mon, 2025-12-15 at 09:17 +0000, Alan Maguire wrote:
> > > > > >
> > > > > > [...]
> > > > > >
> > > > > > > @@ -395,8 +416,7 @@ static int btf_type_size(const struct btf_type *t)
> > > > > > >      case BTF_KIND_DECL_TAG:
> > > > > > >              return base_size + sizeof(struct btf_decl_tag);
> > > > > > >      default:
> > > > > > > -            pr_debug("Unsupported BTF_KIND:%u\n", btf_kind(t));
> > > > > > > -            return -EINVAL;
> > > > > > > +            return btf_type_size_unknown(btf, t);
> > > > > > >      }
> > > > > > >  }
> > > > > > >
> > > > > >
> > > > > > That's a matter of personal preference, of-course, but it seems to me
> > > > > > that using `kind_layouts` table from your next patch for size
> > > > > > computation for all kinds would be a bit more elegant.
> > > > > >
> > > > > > Also, a question, should BTF validation check sizes for known kinds
> > > > > > and reject kind layout sections if those sizes differ from expected?
> > > > > >
> > > > >
> > > > > yeah, I'd say we'd need your second suggestion for the first to be safe,
> > > > > and it seems worthwhile doing both I think. Thanks!
> > > >
> > > > ... but we will just blindly trust layout for unknown kinds, though?
> > > > So it's a bit inconsistent. I'd say let's keep it simple and don't
> > > > overdo the checking? btf_sanity_check() will validate that all known
> > > > kinds are well-formed, isn't that sufficient to ensure that subsequent
> > > > use of BTF data in libbpf won't crash? If some tool generated a subtly
> > > > invalid layout section which otherwise preserves BTF data
> > > > correctness... I don't know, this seems fine. The goal of sanity
> > > > checking is just to prevent more checks in all different places that
> > > > will subsequently rely on IDs being valid, and stuff like that. If
> > > > layout info is wrong for known kinds, so be it, we are not using that
> > > > information anyways.
> > >
> > > Ignoring layout information for known kinds can lead to weird
> > > scenarios: e.g. suppose type size is N, but kind layout specifies that
> > > it is M > N, and the tool generating BTF uses M to actually layout the
> > > binary data. We are being a bit inconsistent with such encoding.
> >
> > Who are "we" here? The tool that emitted incorrect layout information
> > -- yep, for sure. (But that shouldn't happen for correct BTF.)
> >
> > We do btf_sanity_check() upfront to minimize various sanity checks
> > spread out in libbpf code when using BTF data later on. But the goal
> > there is not really to check "100% standards conformance" of whatever
> > BTF we are working with. Kernel is way more concerned about validity
> > and not letting anything unexpected get through, but libbpf is in user
> > space and it's a bit different approach there.
> >
> > As long as BTF looks structurally sound (btf_sanity_check), it should
> > be fine for our needs. If BTF is corrupted or just uses invalid ID, or
> > whatnot, it will eventually fail somewhere, most probably. But the
> > goal is not to have NULL derefs and stuff like that.
>
> Introducing layout info into format provides an alternative definition
> for structural soundness. E.g. some types or vlen elements can have
> padding in the end all of a sudden.
>
> Using this info for some types but not the others is inconsistent.

Ok, so what you are saying is that if there is layout info we should
always use that instead of hard-coded knowledge about kind layout,
right? Ok, I can agree to that, but see note about extensibility
below.

But that's a bit different from validating that the recorded layout
of, say, BTF_KIND_STRUCT is what we expect (sizeof(struct btf_type) +
vlen * sizeof(struct btf_member)). Because if we enforce that, then we
still preclude any extensions to those layouts in the future. And if
we do that, what's the point of looking at layout info for kinds we do
know about?

> Given that BTF rewrites would only be unsound in presence of unknown
> types the whole feature looks questionable to me.

What are those "BTF rewrites" you are referring to? I'm getting a bit
lost in this discussion, tbh.

This feature is designed to allow introducing new (presumably,
optional) kinds and not break older versions of libbpf/bpftool to at
least be able to dump known contents. Does the current implementation
achieve that goal? What other goals do you think this feature should
support?

>
> > Basically what I'm trying to say is that if some tool intentionally or
> > unintentionally generates invalid layout information, but the rest of
> > data is valid, and libbpf doesn't look at layout and otherwise makes
> > fine use of BTF. Then that's fine with me, that BTF fulfills its duty
> > as far as libbpf is concerned. Libbpf here is just a consumer that
> > tries to be as permissive (unlike kernel) as possible, while not
> > leading to a process crash.
> >
> > With double checking that layout info matches our implicit knowledge
> > of type layout we are starting to move into BTF verification a bit,
> > IMO. And I think that's misleading and unnecessary.

Re: [PATCH v8 bpf-next 03/10] libbpf: use kind layout to compute an unknown kind size

[Eduard Zingerman]

On Tue, 2025-12-16 at 14:23 -0800, Andrii Nakryiko wrote:

[...]

> Ok, so what you are saying is that if there is layout info we should
> always use that instead of hard-coded knowledge about kind layout,
> right? Ok, I can agree to that, but see note about extensibility
> below.
>
> But that's a bit different from validating that the recorded layout
> of, say, BTF_KIND_STRUCT is what we expect (sizeof(struct btf_type) +
> vlen * sizeof(struct btf_member)). Because if we enforce that, then we
> still preclude any extensions to those layouts in the future. And if
> we do that, what's the point of looking at layout info for kinds we do
> know about?

If full flexibility is allowed, then all places where e.g. libbpf
iterates params or struct members require an update. That's a big
change.

I suggested checking layout sizes for existing types as a half-measure
allowing to avoid such changes.

> > Given that BTF rewrites would only be unsound in presence of unknown
> > types the whole feature looks questionable to me.
> 
> What are those "BTF rewrites" you are referring to? I'm getting a bit
> lost in this discussion, tbh.

E.g. btf__permute(), as it will not permute all types if some of the
are unknown. Or dedup.

> This feature is designed to allow introducing new (presumably,
> optional) kinds and not break older versions of libbpf/bpftool to at
> least be able to dump known contents. Does the current implementation
> achieve that goal? What other goals do you think this feature should
> support?

I don't think anything other than dump is possible to support.

[...]

Re: [PATCH v8 bpf-next 03/10] libbpf: use kind layout to compute an unknown kind size

[Andrii Nakryiko]

On Tue, Dec 16, 2025 at 2:35 PM Eduard Zingerman <eddyz87@gmail.com> wrote:
>
> On Tue, 2025-12-16 at 14:23 -0800, Andrii Nakryiko wrote:
>
> [...]
>
> > Ok, so what you are saying is that if there is layout info we should
> > always use that instead of hard-coded knowledge about kind layout,
> > right? Ok, I can agree to that, but see note about extensibility
> > below.
> >
> > But that's a bit different from validating that the recorded layout
> > of, say, BTF_KIND_STRUCT is what we expect (sizeof(struct btf_type) +
> > vlen * sizeof(struct btf_member)). Because if we enforce that, then we
> > still preclude any extensions to those layouts in the future. And if
> > we do that, what's the point of looking at layout info for kinds we do
> > know about?
>
> If full flexibility is allowed, then all places where e.g. libbpf
> iterates params or struct members require an update. That's a big
> change.
>
> I suggested checking layout sizes for existing types as a half-measure
> allowing to avoid such changes.

Shouldn't we just say that layout info will never change for the kind?
Whatever fixed + vlen size it starts with, that's set in stone.

>
> > > Given that BTF rewrites would only be unsound in presence of unknown
> > > types the whole feature looks questionable to me.
> >
> > What are those "BTF rewrites" you are referring to? I'm getting a bit
> > lost in this discussion, tbh.
>
> E.g. btf__permute(), as it will not permute all types if some of the
> are unknown. Or dedup.

Yes, agreed, I don't think we should allow modifications like that of
course, who said we should?

>
> > This feature is designed to allow introducing new (presumably,
> > optional) kinds and not break older versions of libbpf/bpftool to at
> > least be able to dump known contents. Does the current implementation
> > achieve that goal? What other goals do you think this feature should
> > support?
>
> I don't think anything other than dump is possible to support.

Ok, then we are on the same page.

One interesting question is what to do about libbpf's BTF
sanitization? Should we still try to replace unknown types with
something that byte-size-wise is compatible? It might not work in all
cases, depending on the semantics of unknown KIND, but it should work
in practice if we are careful about adding new kinds "responsibly".
WDYT?

>
> [...]

Re: [PATCH v8 bpf-next 03/10] libbpf: use kind layout to compute an unknown kind size

[Eduard Zingerman]

On Tue, 2025-12-16 at 15:00 -0800, Andrii Nakryiko wrote:
> On Tue, Dec 16, 2025 at 2:35 PM Eduard Zingerman <eddyz87@gmail.com> wrote:
> > 
> > On Tue, 2025-12-16 at 14:23 -0800, Andrii Nakryiko wrote:
> > 
> > [...]
> > 
> > > Ok, so what you are saying is that if there is layout info we should
> > > always use that instead of hard-coded knowledge about kind layout,
> > > right? Ok, I can agree to that, but see note about extensibility
> > > below.
> > > 
> > > But that's a bit different from validating that the recorded layout
> > > of, say, BTF_KIND_STRUCT is what we expect (sizeof(struct btf_type) +
> > > vlen * sizeof(struct btf_member)). Because if we enforce that, then we
> > > still preclude any extensions to those layouts in the future. And if
> > > we do that, what's the point of looking at layout info for kinds we do
> > > know about?
> > 
> > If full flexibility is allowed, then all places where e.g. libbpf
> > iterates params or struct members require an update. That's a big
> > change.
> > 
> > I suggested checking layout sizes for existing types as a half-measure
> > allowing to avoid such changes.
> 
> Shouldn't we just say that layout info will never change for the kind?
> Whatever fixed + vlen size it starts with, that's set in stone.

If we are not going full flexibility road (a lot of work with unclear purpose),
then fixing the layouts for existing types is what we should do.
I suggested adding layout sizes check in BTF validation phase to
enforce a consistent view at the BTF layout.

> > > > Given that BTF rewrites would only be unsound in presence of unknown
> > > > types the whole feature looks questionable to me.
> > > 
> > > What are those "BTF rewrites" you are referring to? I'm getting a bit
> > > lost in this discussion, tbh.
> > 
> > E.g. btf__permute(), as it will not permute all types if some of the
> > are unknown. Or dedup.
> 
> Yes, agreed, I don't think we should allow modifications like that of
> course, who said we should?

No one says, I suggested adding a check in libbpf,
so that btf_ensure_modifiable() can report an error in such cases.

> > 
> > > This feature is designed to allow introducing new (presumably,
> > > optional) kinds and not break older versions of libbpf/bpftool to at
> > > least be able to dump known contents. Does the current implementation
> > > achieve that goal? What other goals do you think this feature should
> > > support?
> > 
> > I don't think anything other than dump is possible to support.
> 
> Ok, then we are on the same page.
> 
> One interesting question is what to do about libbpf's BTF
> sanitization? Should we still try to replace unknown types with
> something that byte-size-wise is compatible? It might not work in all
> cases, depending on the semantics of unknown KIND, but it should work
> in practice if we are careful about adding new kinds "responsibly".
> WDYT?

The question here is to how to compute the size for the unknown.
It is possible to have a flag specifying if btf_type->size is a true
size. But computation is more sophisticated for e.g. arrays.
On the other hand, if member of some structure has unknown kind,
it can be safely deleted, as struct has size field and offsets for all
members. So, sanitization by deleting types of unknown kind is
possible to some extent.

Re: [PATCH v8 bpf-next 03/10] libbpf: use kind layout to compute an unknown kind size

[Andrii Nakryiko]

On Tue, Dec 16, 2025 at 3:36 PM Eduard Zingerman <eddyz87@gmail.com> wrote:
>
> On Tue, 2025-12-16 at 15:00 -0800, Andrii Nakryiko wrote:
> > On Tue, Dec 16, 2025 at 2:35 PM Eduard Zingerman <eddyz87@gmail.com> wrote:
> > >
> > > On Tue, 2025-12-16 at 14:23 -0800, Andrii Nakryiko wrote:
> > >
> > > [...]
> > >
> > > > Ok, so what you are saying is that if there is layout info we should
> > > > always use that instead of hard-coded knowledge about kind layout,
> > > > right? Ok, I can agree to that, but see note about extensibility
> > > > below.
> > > >
> > > > But that's a bit different from validating that the recorded layout
> > > > of, say, BTF_KIND_STRUCT is what we expect (sizeof(struct btf_type) +
> > > > vlen * sizeof(struct btf_member)). Because if we enforce that, then we
> > > > still preclude any extensions to those layouts in the future. And if
> > > > we do that, what's the point of looking at layout info for kinds we do
> > > > know about?
> > >
> > > If full flexibility is allowed, then all places where e.g. libbpf
> > > iterates params or struct members require an update. That's a big
> > > change.
> > >
> > > I suggested checking layout sizes for existing types as a half-measure
> > > allowing to avoid such changes.
> >
> > Shouldn't we just say that layout info will never change for the kind?
> > Whatever fixed + vlen size it starts with, that's set in stone.
>
> If we are not going full flexibility road (a lot of work with unclear purpose),
> then fixing the layouts for existing types is what we should do.

agreed

> I suggested adding layout sizes check in BTF validation phase to
> enforce a consistent view at the BTF layout.

ok

>
> > > > > Given that BTF rewrites would only be unsound in presence of unknown
> > > > > types the whole feature looks questionable to me.
> > > >
> > > > What are those "BTF rewrites" you are referring to? I'm getting a bit
> > > > lost in this discussion, tbh.
> > >
> > > E.g. btf__permute(), as it will not permute all types if some of the
> > > are unknown. Or dedup.
> >
> > Yes, agreed, I don't think we should allow modifications like that of
> > course, who said we should?
>
> No one says, I suggested adding a check in libbpf,
> so that btf_ensure_modifiable() can report an error in such cases.

yeah, I never disagreed with that. Not sure how this whole thing got
into this discussion, but yes, we are on the same page w.r.t. this

>
> > >
> > > > This feature is designed to allow introducing new (presumably,
> > > > optional) kinds and not break older versions of libbpf/bpftool to at
> > > > least be able to dump known contents. Does the current implementation
> > > > achieve that goal? What other goals do you think this feature should
> > > > support?
> > >
> > > I don't think anything other than dump is possible to support.
> >
> > Ok, then we are on the same page.
> >
> > One interesting question is what to do about libbpf's BTF
> > sanitization? Should we still try to replace unknown types with
> > something that byte-size-wise is compatible? It might not work in all
> > cases, depending on the semantics of unknown KIND, but it should work
> > in practice if we are careful about adding new kinds "responsibly".
> > WDYT?
>
> The question here is to how to compute the size for the unknown.
> It is possible to have a flag specifying if btf_type->size is a true
> size. But computation is more sophisticated for e.g. arrays.
> On the other hand, if member of some structure has unknown kind,
> it can be safely deleted, as struct has size field and offsets for all
> members. So, sanitization by deleting types of unknown kind is
> possible to some extent.

I think it's unlikely we'll add some kind that will be directly
embeddable into struct except for some modifiers. For modifiers (which
I'm arguing we should add a flag stating that this kind is used as a
modifier and its type field is actually a type ID field), we can
replace them either with typedef or const and preserve layout and most
of semantics. And for optional stuff like decl_tag, they are usually
stand-alone pointing to types (rather than having types pointed to
them), so just replacing them with something that is compatible in
terms of byte size in BTF data should be sufficient.

I think BTF sanitization will have to be best effort, but if we keep
sanitization in mind, we can ensure reasonable behavior.

Ultimately, though, you should always strive to use the very latest
libbpf with your BPF object files. So maybe no sanitization of unknown
kinds is the right (and simple) answer here: just update libbpf and it
will take care of sanitization of *known* kinds.

Re: [PATCH v8 bpf-next 03/10] libbpf: use kind layout to compute an unknown kind size

[Eduard Zingerman]

On Tue, 2025-12-16 at 16:30 -0800, Andrii Nakryiko wrote:

[...]

> > > One interesting question is what to do about libbpf's BTF
> > > sanitization? Should we still try to replace unknown types with
> > > something that byte-size-wise is compatible? It might not work in all
> > > cases, depending on the semantics of unknown KIND, but it should work
> > > in practice if we are careful about adding new kinds "responsibly".
> > > WDYT?
> > 
> > The question here is to how to compute the size for the unknown.
> > It is possible to have a flag specifying if btf_type->size is a true
> > size. But computation is more sophisticated for e.g. arrays.
> > On the other hand, if member of some structure has unknown kind,
> > it can be safely deleted, as struct has size field and offsets for all
> > members. So, sanitization by deleting types of unknown kind is
> > possible to some extent.
> 
> I think it's unlikely we'll add some kind that will be directly
> embeddable into struct except for some modifiers. For modifiers (which
> I'm arguing we should add a flag stating that this kind is used as a
> modifier and its type field is actually a type ID field), we can
> replace them either with typedef or const and preserve layout and most
> of semantics. And for optional stuff like decl_tag, they are usually
> stand-alone pointing to types (rather than having types pointed to
> them), so just replacing them with something that is compatible in
> terms of byte size in BTF data should be sufficient.
> 
> I think BTF sanitization will have to be best effort, but if we keep
> sanitization in mind, we can ensure reasonable behavior.
> 
> Ultimately, though, you should always strive to use the very latest
> libbpf with your BPF object files. So maybe no sanitization of unknown
> kinds is the right (and simple) answer here: just update libbpf and it
> will take care of sanitization of *known* kinds.

Idk, this would matter if we decide to change clang to emit new BTF
kinds for BPF programs. Maybe postpone sanitization implementation
until requirements are better understood?

Re: [PATCH v8 bpf-next 01/10] btf: add kind layout encoding to UAPI

[Alan Maguire]

On 16/12/2025 19:23, Andrii Nakryiko wrote:
> On Mon, Dec 15, 2025 at 1:18 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>>
>> BTF kind layouts provide information to parse BTF kinds. By separating
>> parsing BTF from using all the information it provides, we allow BTF
>> to encode new features even if they cannot be used by readers. This
>> will be helpful in particular for cases where older tools are used
>> to parse newer BTF with kinds the older tools do not recognize;
>> the BTF can still be parsed in such cases using kind layout.
>>
>> The intent is to support encoding of kind layouts optionally so that
>> tools like pahole can add this information. For each kind, we record
>>
>> - length of singular element following struct btf_type
>> - length of each of the btf_vlen() elements following
>>
>> The ideas here were discussed at [1], [2]; hence
>>
>> Suggested-by: Andrii Nakryiko <andrii@kernel.org>
>> Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
>>
>> [1] https://lore.kernel.org/bpf/CAEf4BzYjWHRdNNw4B=eOXOs_ONrDwrgX4bn=Nuc1g8JPFC34MA@mail.gmail.com/
>> [2] https://lore.kernel.org/bpf/20230531201936.1992188-1-alan.maguire@oracle.com/
>> ---
>>  include/uapi/linux/btf.h       | 11 +++++++++++
>>  tools/include/uapi/linux/btf.h | 11 +++++++++++
>>  2 files changed, 22 insertions(+)
>>
>> diff --git a/include/uapi/linux/btf.h b/include/uapi/linux/btf.h
>> index 266d4ffa6c07..c1854a1c7b38 100644
>> --- a/include/uapi/linux/btf.h
>> +++ b/include/uapi/linux/btf.h
>> @@ -8,6 +8,15 @@
>>  #define BTF_MAGIC      0xeB9F
>>  #define BTF_VERSION    1
>>
>> +/*
>> + * kind layout section consists of a struct btf_kind_layout for each known
>> + * kind at BTF encoding time.
>> + */
>> +struct btf_kind_layout {
>> +       __u8 info_sz;           /* size of singular element after btf_type */
>> +       __u8 elem_sz;           /* size of each of btf_vlen(t) elements */
> 
> So Eduard pointed out that at some point we discussed having a name of
> a kind (i.e., "struct", "typedef", etc). By now I have no recollection
> what were the arguments, do you remember? I'm not sure how I feel now
> about having extra 4 bytes per kind, but that's not really a lot of
> data (20*4 = 80 bytes added), so might as well add it, I suppose?
> 

Yeah we went back and forth on that; I think it's on balance worthwhile
to be honest; tools can be a bit more expressive about what's missing.

> I think we were also discussing having flags per kind to designate
> some extra semantics, where applicable. Again, don't remember
> arguments for or against, but one case where I think this would be
> very beneficial is when we add something like type_tag, which is
> inevitably used from "normal" struct and will be almost inevitable in
> normal vmlinux BTF. Think about it, we have some field which will be
> CONST -> PTR -> TYPE_TAG -> STRUCT. That TYPE_TAG shouldn't just
> totally break (old) bpftool's dump, as it really can be easily ignored
> **if we know TYPE_TAG can be ignored and it is just a reference
> type**. That reference type means that there is another type pointed
> to using struct btf_type::type field (instead of that field being a
> size).
> 
> So I think it would be nice to encode this as a flag that says a) kind
> can be ignored without compromising type integrity (i.e., memory
> layout is preserved) which will be true for all kinds of modifier
> kinds (const/volatile/restrict/type_tag, even for typedef that should
> be true) and b) kind is reference type, so struct btf_type::type is a
> "pointer" to a valid other underlying type.
> 
> Thoughts?
> 

Again we did go back and forth here but to me there's much more value in
being both able to parse _and_ sanitize BTF, at least for the simple cases.
What we can include are as you say types in the type graph that are optional
reference kinds (like type tag), and kinds that are not implicated in the
known type graph like the location stuff (it only points _to_ known kinds, 
no known kinds will point to location data). So any case where known
types + optional ref types constitute the type graph we are good.
Anything more complex than these would involve having to represent the
layout of type references within unknown kinds (kind of like what we do for 
field iteration) which seems a bit much.

Now one thing that we might want to introduce here is a sanitization-friendly
kind, either re-using BTF_KIND_UNKN or adding a new vlen-supporting kind
which can be used to overwrite kinds we don't want in the sanitized output.
We need this to preserve the type ids for the kernel BTF we sanitize.
I get that it seems weird to add a new incompatibility to handle incompatibility,
but the sooner we do it the better I guess. The reason I suggest it now is we'd
potentially need some more complex sanitization for the location stuff for
cases like large location sections, and it might be cleaner to have a special
"ignore this it's just sanitization info" kind, especially for cases like 
BTF C dump.


>> +};
>> +
>>  struct btf_header {
>>         __u16   magic;
>>         __u8    version;
>> @@ -19,6 +28,8 @@ struct btf_header {
>>         __u32   type_len;       /* length of type section       */
>>         __u32   str_off;        /* offset of string section     */
>>         __u32   str_len;        /* length of string section     */
>> +       __u32   kind_layout_off;/* offset of kind layout section */
>> +       __u32   kind_layout_len;/* length of kind layout section */
> 
> nit: kind_layout is a bit mouthful, have you considered "descr" (for
> description/descriptor) or just "layout" as a name designator?
> 

Yep, layout seems good. Thanks!

Alan

Re: [PATCH v8 bpf-next 02/10] libbpf: Support kind layout section handling in BTF

[Alan Maguire]

On 16/12/2025 19:34, Andrii Nakryiko wrote:
> On Mon, Dec 15, 2025 at 1:18 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>>
>> Support reading in kind layout fixing endian issues on reading;
>> also support writing kind layout section to raw BTF object.
>> There is not yet an API to populate the kind layout with meaningful
>> information.
>>
>> As part of this, we need to consider multiple valid BTF header
>> sizes; the original or the kind layout-extended headers.
>> So to support this, the "struct btf" representation is modified
>> to always allocate a "struct btf_header" and copy the valid
>> portion from the raw data to it; this means we can always safely
>> check fields like btf->hdr->kind_layout_len.
>>
>> Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
>> ---
>>  tools/lib/bpf/btf.c | 260 +++++++++++++++++++++++++++++++-------------
>>  1 file changed, 183 insertions(+), 77 deletions(-)
>>
>> diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
>> index b136572e889a..8835aee6ee84 100644
>> --- a/tools/lib/bpf/btf.c
>> +++ b/tools/lib/bpf/btf.c
>> @@ -40,42 +40,53 @@ struct btf {
>>
>>         /*
>>          * When BTF is loaded from an ELF or raw memory it is stored
>> -        * in a contiguous memory block. The hdr, type_data, and, strs_data
>> +        * in a contiguous memory block. The  type_data, and, strs_data
> 
> nit: two spaces, and so many commas around and ;) let's leave Oxford
> comma, but comma after and is weird
> 
>>          * point inside that memory region to their respective parts of BTF
>>          * representation:
>>          *
>> -        * +--------------------------------+
>> -        * |  Header  |  Types  |  Strings  |
>> -        * +--------------------------------+
>> -        * ^          ^         ^
>> -        * |          |         |
>> -        * hdr        |         |
>> -        * types_data-+         |
>> -        * strs_data------------+
>> +        * +--------------------------------+---------------------+
>> +        * |  Header  |  Types  |  Strings  |Optional kind layout |
> 
> Space missing, boo. Keep diagrams beautiful!..
> 
>> +        * +--------------------------------+---------------------+
>> +        * ^          ^         ^           ^
>> +        * |          |         |           |
>> +        * raw_data   |         |           |
>> +        * types_data-+         |           |
>> +        * strs_data------------+           |
>> +        * kind_layout----------------------+
>> +        *
>> +        * A separate struct btf_header is allocated for btf->hdr,
>> +        * and header information is copied into it.  This allows us
>> +        * to handle header data for various header formats; the original,
>> +        * the extended header with kind layout, etc.
>>          *
>>          * If BTF data is later modified, e.g., due to types added or
>>          * removed, BTF deduplication performed, etc, this contiguous
>> -        * representation is broken up into three independently allocated
>> -        * memory regions to be able to modify them independently.
>> +        * representation is broken up into four independent memory
>> +        * regions.
>> +        *
>>          * raw_data is nulled out at that point, but can be later allocated
>>          * and cached again if user calls btf__raw_data(), at which point
>> -        * raw_data will contain a contiguous copy of header, types, and
>> -        * strings:
>> +        * raw_data will contain a contiguous copy of header, types, strings
>> +        * and optionally kind_layout.  kind_layout optionally points to a
>> +        * kind_layout array - this allows us to encode information about
>> +        * the kinds known at encoding time.  If kind_layout is NULL no
>> +        * kind information is encoded.
>>          *
>> -        * +----------+  +---------+  +-----------+
>> -        * |  Header  |  |  Types  |  |  Strings  |
>> -        * +----------+  +---------+  +-----------+
>> -        * ^             ^            ^
>> -        * |             |            |
>> -        * hdr           |            |
>> -        * types_data----+            |
>> -        * strset__data(strs_set)-----+
>> +        * +----------+  +---------+  +-----------+   +-----------+
>> +        * |  Header  |  |  Types  |  |  Strings  |   |kind_layout|
>> +        * +----------+  +---------+  +-----------+   +-----------+
> 
> nit: spaces (and if we go with "layout" naming, this will be short and
> beautiful " Layout " ;)
> 
>> +        * ^             ^            ^               ^
>> +        * |             |            |               |
>> +        * hdr           |            |               |
>> +        * types_data----+            |               |
>> +        * strset__data(strs_set)-----+               |
>> +        * kind_layout--------------------------------+
> 
> [...]
> 
>> @@ -3888,7 +3989,7 @@ static int btf_dedup_strings(struct btf_dedup *d)
>>
>>         /* replace BTF string data and hash with deduped ones */
>>         strset__free(d->btf->strs_set);
>> -       d->btf->hdr->str_len = strset__data_size(d->strs_set);
>> +       btf_hdr_update_str_len(d->btf, strset__data_size(d->strs_set));
>>         d->btf->strs_set = d->strs_set;
>>         d->strs_set = NULL;
>>         d->btf->strs_deduped = true;
>> @@ -5343,6 +5444,11 @@ static int btf_dedup_compact_types(struct btf_dedup *d)
>>         d->btf->type_offs = new_offs;
>>         d->btf->hdr->str_off = d->btf->hdr->type_len;
>>         d->btf->raw_size = d->btf->hdr->hdr_len + d->btf->hdr->type_len + d->btf->hdr->str_len;
>> +       if (d->btf->kind_layout) {
>> +               d->btf->hdr->kind_layout_off = d->btf->hdr->str_off + roundup(d->btf->hdr->str_len,
>> +                                                                             4);
>> +               d->btf->raw_size = roundup(d->btf->raw_size, 4) + d->btf->hdr->kind_layout_len;
> 
> maybe put layout data after type data, but before strings? rounding up
> string section which is byte-based feels weird. I think old libbpf
> implementations should handle all this well, because btf_header
> explicitly specifies string section offset, no?
>

That sounds good, but I think there are some strictness issues with how we parse
BTF on the kernel side that we may need to think about, especially if we want to
make kind layout always available. In that case we'd need to think how old kernels
built with newer pahole might handle newer headers with layout info.

First in btf_parse_hdr() the kernel rejects BTF with non-zero unsupported fields.
So trying to load vmlinux BTF generated by a pahole that adds layout info will 
fail for such a kernel.

Second when validating section info in btf_check_sec_info() we check for overlaps
between known sections, and we also check for gaps between known sections. Finally we
also check for any additional data other than the known section data.

For layout info stored between type+strings we'd wind up rejecting it for a few reasons:

1. we'd find non-zero data in the header (layout offset/len)
2. we'd find a "gap" between types+strings (the layout data)

Similarly with layout at the end

1. we'd find non-zero data in the header (kind layout offset/len)
2. we'd find unaccounted-for data after the string data (the kind layout data)

So either way we'd wind up with unsupported headers. One approach would be to
do stable backports relaxing these header tests; I think we could relax them to
simply ensure no overlap between sections and that sections don't overrun data
length without risking having unusable BTF. Then a newer BTF header with additional
layout info wouldn't get rejected. What do you think?

Alan

Re: [PATCH v8 bpf-next 01/10] btf: add kind layout encoding to UAPI

[Andrii Nakryiko]

On Fri, Dec 19, 2025 at 5:15 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>
> On 16/12/2025 19:23, Andrii Nakryiko wrote:
> > On Mon, Dec 15, 2025 at 1:18 AM Alan Maguire <alan.maguire@oracle.com> wrote:
> >>
> >> BTF kind layouts provide information to parse BTF kinds. By separating
> >> parsing BTF from using all the information it provides, we allow BTF
> >> to encode new features even if they cannot be used by readers. This
> >> will be helpful in particular for cases where older tools are used
> >> to parse newer BTF with kinds the older tools do not recognize;
> >> the BTF can still be parsed in such cases using kind layout.
> >>
> >> The intent is to support encoding of kind layouts optionally so that
> >> tools like pahole can add this information. For each kind, we record
> >>
> >> - length of singular element following struct btf_type
> >> - length of each of the btf_vlen() elements following
> >>
> >> The ideas here were discussed at [1], [2]; hence
> >>
> >> Suggested-by: Andrii Nakryiko <andrii@kernel.org>
> >> Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
> >>
> >> [1] https://lore.kernel.org/bpf/CAEf4BzYjWHRdNNw4B=eOXOs_ONrDwrgX4bn=Nuc1g8JPFC34MA@mail.gmail.com/
> >> [2] https://lore.kernel.org/bpf/20230531201936.1992188-1-alan.maguire@oracle.com/
> >> ---
> >>  include/uapi/linux/btf.h       | 11 +++++++++++
> >>  tools/include/uapi/linux/btf.h | 11 +++++++++++
> >>  2 files changed, 22 insertions(+)
> >>
> >> diff --git a/include/uapi/linux/btf.h b/include/uapi/linux/btf.h
> >> index 266d4ffa6c07..c1854a1c7b38 100644
> >> --- a/include/uapi/linux/btf.h
> >> +++ b/include/uapi/linux/btf.h
> >> @@ -8,6 +8,15 @@
> >>  #define BTF_MAGIC      0xeB9F
> >>  #define BTF_VERSION    1
> >>
> >> +/*
> >> + * kind layout section consists of a struct btf_kind_layout for each known
> >> + * kind at BTF encoding time.
> >> + */
> >> +struct btf_kind_layout {
> >> +       __u8 info_sz;           /* size of singular element after btf_type */
> >> +       __u8 elem_sz;           /* size of each of btf_vlen(t) elements */
> >
> > So Eduard pointed out that at some point we discussed having a name of
> > a kind (i.e., "struct", "typedef", etc). By now I have no recollection
> > what were the arguments, do you remember? I'm not sure how I feel now
> > about having extra 4 bytes per kind, but that's not really a lot of
> > data (20*4 = 80 bytes added), so might as well add it, I suppose?
> >
>
> Yeah we went back and forth on that; I think it's on balance worthwhile
> to be honest; tools can be a bit more expressive about what's missing.
>
> > I think we were also discussing having flags per kind to designate
> > some extra semantics, where applicable. Again, don't remember
> > arguments for or against, but one case where I think this would be
> > very beneficial is when we add something like type_tag, which is
> > inevitably used from "normal" struct and will be almost inevitable in
> > normal vmlinux BTF. Think about it, we have some field which will be
> > CONST -> PTR -> TYPE_TAG -> STRUCT. That TYPE_TAG shouldn't just
> > totally break (old) bpftool's dump, as it really can be easily ignored
> > **if we know TYPE_TAG can be ignored and it is just a reference
> > type**. That reference type means that there is another type pointed
> > to using struct btf_type::type field (instead of that field being a
> > size).
> >
> > So I think it would be nice to encode this as a flag that says a) kind
> > can be ignored without compromising type integrity (i.e., memory
> > layout is preserved) which will be true for all kinds of modifier
> > kinds (const/volatile/restrict/type_tag, even for typedef that should
> > be true) and b) kind is reference type, so struct btf_type::type is a
> > "pointer" to a valid other underlying type.
> >
> > Thoughts?
> >
>
> Again we did go back and forth here but to me there's much more value in
> being both able to parse _and_ sanitize BTF, at least for the simple cases.
> What we can include are as you say types in the type graph that are optional
> reference kinds (like type tag), and kinds that are not implicated in the
> known type graph like the location stuff (it only points _to_ known kinds,
> no known kinds will point to location data). So any case where known
> types + optional ref types constitute the type graph we are good.
> Anything more complex than these would involve having to represent the
> layout of type references within unknown kinds (kind of like what we do for
> field iteration) which seems a bit much.
>
> Now one thing that we might want to introduce here is a sanitization-friendly
> kind, either re-using BTF_KIND_UNKN or adding a new vlen-supporting kind
> which can be used to overwrite kinds we don't want in the sanitized output.
> We need this to preserve the type ids for the kernel BTF we sanitize.
> I get that it seems weird to add a new incompatibility to handle incompatibility,
> but the sooner we do it the better I guess. The reason I suggest it now is we'd
> potentially need some more complex sanitization for the location stuff for
> cases like large location sections, and it might be cleaner to have a special
> "ignore this it's just sanitization info" kind, especially for cases like
> BTF C dump.

So you mean you'd like some "dummy" BTF kind with 4-byte-per-vlen so
we can "overwrite" any possible unknown BTF kind?.. As you said,
though, this would only work for new kernels, so that's sad... I don't
know, I don't hate the idea, but curious what others think.

Alternatively, we can just try to never add kinds where the vlen
element is not a multiple of 8 or 12. We can then use ENUM
(8-bytes-per-vlen) or ENUM64 (12-bytes-per-vlen) to paper over unknown
types. FUNC_PROTO (8-bytes-per-vlen) and DATASEC (12-bytes-per-vlen)
are other options. We just don't have 4-bytes-per-vlen for the most
universal "filler", unfortunately.

The advantage of the latter is full backwards compatibility with old kernels.

>
>
> >> +};
> >> +
> >>  struct btf_header {
> >>         __u16   magic;
> >>         __u8    version;
> >> @@ -19,6 +28,8 @@ struct btf_header {
> >>         __u32   type_len;       /* length of type section       */
> >>         __u32   str_off;        /* offset of string section     */
> >>         __u32   str_len;        /* length of string section     */
> >> +       __u32   kind_layout_off;/* offset of kind layout section */
> >> +       __u32   kind_layout_len;/* length of kind layout section */
> >
> > nit: kind_layout is a bit mouthful, have you considered "descr" (for
> > description/descriptor) or just "layout" as a name designator?
> >
>
> Yep, layout seems good. Thanks!
>
> Alan

Re: [PATCH v8 bpf-next 02/10] libbpf: Support kind layout section handling in BTF

[Andrii Nakryiko]

On Fri, Dec 19, 2025 at 5:34 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>
> On 16/12/2025 19:34, Andrii Nakryiko wrote:
> > On Mon, Dec 15, 2025 at 1:18 AM Alan Maguire <alan.maguire@oracle.com> wrote:
> >>
> >> Support reading in kind layout fixing endian issues on reading;
> >> also support writing kind layout section to raw BTF object.
> >> There is not yet an API to populate the kind layout with meaningful
> >> information.
> >>
> >> As part of this, we need to consider multiple valid BTF header
> >> sizes; the original or the kind layout-extended headers.
> >> So to support this, the "struct btf" representation is modified
> >> to always allocate a "struct btf_header" and copy the valid
> >> portion from the raw data to it; this means we can always safely
> >> check fields like btf->hdr->kind_layout_len.
> >>
> >> Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
> >> ---
> >>  tools/lib/bpf/btf.c | 260 +++++++++++++++++++++++++++++++-------------
> >>  1 file changed, 183 insertions(+), 77 deletions(-)
> >>
> >> diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> >> index b136572e889a..8835aee6ee84 100644
> >> --- a/tools/lib/bpf/btf.c
> >> +++ b/tools/lib/bpf/btf.c
> >> @@ -40,42 +40,53 @@ struct btf {
> >>
> >>         /*
> >>          * When BTF is loaded from an ELF or raw memory it is stored
> >> -        * in a contiguous memory block. The hdr, type_data, and, strs_data
> >> +        * in a contiguous memory block. The  type_data, and, strs_data
> >
> > nit: two spaces, and so many commas around and ;) let's leave Oxford
> > comma, but comma after and is weird
> >
> >>          * point inside that memory region to their respective parts of BTF
> >>          * representation:
> >>          *
> >> -        * +--------------------------------+
> >> -        * |  Header  |  Types  |  Strings  |
> >> -        * +--------------------------------+
> >> -        * ^          ^         ^
> >> -        * |          |         |
> >> -        * hdr        |         |
> >> -        * types_data-+         |
> >> -        * strs_data------------+
> >> +        * +--------------------------------+---------------------+
> >> +        * |  Header  |  Types  |  Strings  |Optional kind layout |
> >
> > Space missing, boo. Keep diagrams beautiful!..
> >
> >> +        * +--------------------------------+---------------------+
> >> +        * ^          ^         ^           ^
> >> +        * |          |         |           |
> >> +        * raw_data   |         |           |
> >> +        * types_data-+         |           |
> >> +        * strs_data------------+           |
> >> +        * kind_layout----------------------+
> >> +        *
> >> +        * A separate struct btf_header is allocated for btf->hdr,
> >> +        * and header information is copied into it.  This allows us
> >> +        * to handle header data for various header formats; the original,
> >> +        * the extended header with kind layout, etc.
> >>          *
> >>          * If BTF data is later modified, e.g., due to types added or
> >>          * removed, BTF deduplication performed, etc, this contiguous
> >> -        * representation is broken up into three independently allocated
> >> -        * memory regions to be able to modify them independently.
> >> +        * representation is broken up into four independent memory
> >> +        * regions.
> >> +        *
> >>          * raw_data is nulled out at that point, but can be later allocated
> >>          * and cached again if user calls btf__raw_data(), at which point
> >> -        * raw_data will contain a contiguous copy of header, types, and
> >> -        * strings:
> >> +        * raw_data will contain a contiguous copy of header, types, strings
> >> +        * and optionally kind_layout.  kind_layout optionally points to a
> >> +        * kind_layout array - this allows us to encode information about
> >> +        * the kinds known at encoding time.  If kind_layout is NULL no
> >> +        * kind information is encoded.
> >>          *
> >> -        * +----------+  +---------+  +-----------+
> >> -        * |  Header  |  |  Types  |  |  Strings  |
> >> -        * +----------+  +---------+  +-----------+
> >> -        * ^             ^            ^
> >> -        * |             |            |
> >> -        * hdr           |            |
> >> -        * types_data----+            |
> >> -        * strset__data(strs_set)-----+
> >> +        * +----------+  +---------+  +-----------+   +-----------+
> >> +        * |  Header  |  |  Types  |  |  Strings  |   |kind_layout|
> >> +        * +----------+  +---------+  +-----------+   +-----------+
> >
> > nit: spaces (and if we go with "layout" naming, this will be short and
> > beautiful " Layout " ;)
> >
> >> +        * ^             ^            ^               ^
> >> +        * |             |            |               |
> >> +        * hdr           |            |               |
> >> +        * types_data----+            |               |
> >> +        * strset__data(strs_set)-----+               |
> >> +        * kind_layout--------------------------------+
> >
> > [...]
> >
> >> @@ -3888,7 +3989,7 @@ static int btf_dedup_strings(struct btf_dedup *d)
> >>
> >>         /* replace BTF string data and hash with deduped ones */
> >>         strset__free(d->btf->strs_set);
> >> -       d->btf->hdr->str_len = strset__data_size(d->strs_set);
> >> +       btf_hdr_update_str_len(d->btf, strset__data_size(d->strs_set));
> >>         d->btf->strs_set = d->strs_set;
> >>         d->strs_set = NULL;
> >>         d->btf->strs_deduped = true;
> >> @@ -5343,6 +5444,11 @@ static int btf_dedup_compact_types(struct btf_dedup *d)
> >>         d->btf->type_offs = new_offs;
> >>         d->btf->hdr->str_off = d->btf->hdr->type_len;
> >>         d->btf->raw_size = d->btf->hdr->hdr_len + d->btf->hdr->type_len + d->btf->hdr->str_len;
> >> +       if (d->btf->kind_layout) {
> >> +               d->btf->hdr->kind_layout_off = d->btf->hdr->str_off + roundup(d->btf->hdr->str_len,
> >> +                                                                             4);
> >> +               d->btf->raw_size = roundup(d->btf->raw_size, 4) + d->btf->hdr->kind_layout_len;
> >
> > maybe put layout data after type data, but before strings? rounding up
> > string section which is byte-based feels weird. I think old libbpf
> > implementations should handle all this well, because btf_header
> > explicitly specifies string section offset, no?
> >
>
> That sounds good, but I think there are some strictness issues with how we parse
> BTF on the kernel side that we may need to think about, especially if we want to
> make kind layout always available. In that case we'd need to think how old kernels
> built with newer pahole might handle newer headers with layout info.
>
> First in btf_parse_hdr() the kernel rejects BTF with non-zero unsupported fields.
> So trying to load vmlinux BTF generated by a pahole that adds layout info will
> fail for such a kernel.
>
> Second when validating section info in btf_check_sec_info() we check for overlaps
> between known sections, and we also check for gaps between known sections. Finally we
> also check for any additional data other than the known section data.

I thought we don't validate gaps, I missed btf_check_sec_info()
checks, though. Good for kernel, it should be strict.

But it's easy to drop this layout info in libbpf for BTF sanitization,
this shouldn't be a problem. Just shift everything to the left and
adjust strs_off.

>
> For layout info stored between type+strings we'd wind up rejecting it for a few reasons:
>
> 1. we'd find non-zero data in the header (layout offset/len)
> 2. we'd find a "gap" between types+strings (the layout data)
>
> Similarly with layout at the end
>
> 1. we'd find non-zero data in the header (kind layout offset/len)
> 2. we'd find unaccounted-for data after the string data (the kind layout data)
>
> So either way we'd wind up with unsupported headers. One approach would be to
> do stable backports relaxing these header tests; I think we could relax them to
> simply ensure no overlap between sections and that sections don't overrun data
> length without risking having unusable BTF. Then a newer BTF header with additional
> layout info wouldn't get rejected. What do you think?

See above, I don't see why we can't just sanitize BTF and drop layout
parts altogether. They are optional for kernel either way, no harm in
dropping them.

>
> Alan

Re: [PATCH v8 bpf-next 01/10] btf: add kind layout encoding to UAPI

[Alan Maguire]

On 19/12/2025 17:53, Andrii Nakryiko wrote:
> On Fri, Dec 19, 2025 at 5:15 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>>
>> On 16/12/2025 19:23, Andrii Nakryiko wrote:
>>> On Mon, Dec 15, 2025 at 1:18 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>>>>
>>>> BTF kind layouts provide information to parse BTF kinds. By separating
>>>> parsing BTF from using all the information it provides, we allow BTF
>>>> to encode new features even if they cannot be used by readers. This
>>>> will be helpful in particular for cases where older tools are used
>>>> to parse newer BTF with kinds the older tools do not recognize;
>>>> the BTF can still be parsed in such cases using kind layout.
>>>>
>>>> The intent is to support encoding of kind layouts optionally so that
>>>> tools like pahole can add this information. For each kind, we record
>>>>
>>>> - length of singular element following struct btf_type
>>>> - length of each of the btf_vlen() elements following
>>>>
>>>> The ideas here were discussed at [1], [2]; hence
>>>>
>>>> Suggested-by: Andrii Nakryiko <andrii@kernel.org>
>>>> Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
>>>>
>>>> [1] https://lore.kernel.org/bpf/CAEf4BzYjWHRdNNw4B=eOXOs_ONrDwrgX4bn=Nuc1g8JPFC34MA@mail.gmail.com/
>>>> [2] https://lore.kernel.org/bpf/20230531201936.1992188-1-alan.maguire@oracle.com/
>>>> ---
>>>>  include/uapi/linux/btf.h       | 11 +++++++++++
>>>>  tools/include/uapi/linux/btf.h | 11 +++++++++++
>>>>  2 files changed, 22 insertions(+)
>>>>
>>>> diff --git a/include/uapi/linux/btf.h b/include/uapi/linux/btf.h
>>>> index 266d4ffa6c07..c1854a1c7b38 100644
>>>> --- a/include/uapi/linux/btf.h
>>>> +++ b/include/uapi/linux/btf.h
>>>> @@ -8,6 +8,15 @@
>>>>  #define BTF_MAGIC      0xeB9F
>>>>  #define BTF_VERSION    1
>>>>
>>>> +/*
>>>> + * kind layout section consists of a struct btf_kind_layout for each known
>>>> + * kind at BTF encoding time.
>>>> + */
>>>> +struct btf_kind_layout {
>>>> +       __u8 info_sz;           /* size of singular element after btf_type */
>>>> +       __u8 elem_sz;           /* size of each of btf_vlen(t) elements */
>>>
>>> So Eduard pointed out that at some point we discussed having a name of
>>> a kind (i.e., "struct", "typedef", etc). By now I have no recollection
>>> what were the arguments, do you remember? I'm not sure how I feel now
>>> about having extra 4 bytes per kind, but that's not really a lot of
>>> data (20*4 = 80 bytes added), so might as well add it, I suppose?
>>>
>>
>> Yeah we went back and forth on that; I think it's on balance worthwhile
>> to be honest; tools can be a bit more expressive about what's missing.
>>
>>> I think we were also discussing having flags per kind to designate
>>> some extra semantics, where applicable. Again, don't remember
>>> arguments for or against, but one case where I think this would be
>>> very beneficial is when we add something like type_tag, which is
>>> inevitably used from "normal" struct and will be almost inevitable in
>>> normal vmlinux BTF. Think about it, we have some field which will be
>>> CONST -> PTR -> TYPE_TAG -> STRUCT. That TYPE_TAG shouldn't just
>>> totally break (old) bpftool's dump, as it really can be easily ignored
>>> **if we know TYPE_TAG can be ignored and it is just a reference
>>> type**. That reference type means that there is another type pointed
>>> to using struct btf_type::type field (instead of that field being a
>>> size).
>>>
>>> So I think it would be nice to encode this as a flag that says a) kind
>>> can be ignored without compromising type integrity (i.e., memory
>>> layout is preserved) which will be true for all kinds of modifier
>>> kinds (const/volatile/restrict/type_tag, even for typedef that should
>>> be true) and b) kind is reference type, so struct btf_type::type is a
>>> "pointer" to a valid other underlying type.
>>>
>>> Thoughts?
>>>
>>
>> Again we did go back and forth here but to me there's much more value in
>> being both able to parse _and_ sanitize BTF, at least for the simple cases.
>> What we can include are as you say types in the type graph that are optional
>> reference kinds (like type tag), and kinds that are not implicated in the
>> known type graph like the location stuff (it only points _to_ known kinds,
>> no known kinds will point to location data). So any case where known
>> types + optional ref types constitute the type graph we are good.
>> Anything more complex than these would involve having to represent the
>> layout of type references within unknown kinds (kind of like what we do for
>> field iteration) which seems a bit much.
>>
>> Now one thing that we might want to introduce here is a sanitization-friendly
>> kind, either re-using BTF_KIND_UNKN or adding a new vlen-supporting kind
>> which can be used to overwrite kinds we don't want in the sanitized output.
>> We need this to preserve the type ids for the kernel BTF we sanitize.
>> I get that it seems weird to add a new incompatibility to handle incompatibility,
>> but the sooner we do it the better I guess. The reason I suggest it now is we'd
>> potentially need some more complex sanitization for the location stuff for
>> cases like large location sections, and it might be cleaner to have a special
>> "ignore this it's just sanitization info" kind, especially for cases like
>> BTF C dump.
> 
> So you mean you'd like some "dummy" BTF kind with 4-byte-per-vlen so
> we can "overwrite" any possible unknown BTF kind?.. As you said,
> though, this would only work for new kernels, so that's sad... I don't
> know, I don't hate the idea, but curious what others think.
> 
> Alternatively, we can just try to never add kinds where the vlen
> element is not a multiple of 8 or 12. We can then use ENUM
> (8-bytes-per-vlen) or ENUM64 (12-bytes-per-vlen) to paper over unknown
> types. FUNC_PROTO (8-bytes-per-vlen) and DATASEC (12-bytes-per-vlen)
> are other options. We just don't have 4-bytes-per-vlen for the most
> universal "filler", unfortunately.
> 
> The advantage of the latter is full backwards compatibility with old kernels.
>

True. And I guess during sanitization we can just handle intermediate
types in a type graph by adjusting type ids to skip over them, so we
likely have everything we need already. Funnily enough the BTF location
stuff will give us a vlen-specified 4 byte object (specifying the
location parameters associated with an inline), so that will help in
the future for cases where it is recognized but other kinds are not.

Alan

Re: [PATCH v8 bpf-next 02/10] libbpf: Support kind layout section handling in BTF

[Alan Maguire]

On 19/12/2025 17:58, Andrii Nakryiko wrote:
> On Fri, Dec 19, 2025 at 5:34 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>>
>> On 16/12/2025 19:34, Andrii Nakryiko wrote:
>>> On Mon, Dec 15, 2025 at 1:18 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>>>>
>>>> Support reading in kind layout fixing endian issues on reading;
>>>> also support writing kind layout section to raw BTF object.
>>>> There is not yet an API to populate the kind layout with meaningful
>>>> information.
>>>>
>>>> As part of this, we need to consider multiple valid BTF header
>>>> sizes; the original or the kind layout-extended headers.
>>>> So to support this, the "struct btf" representation is modified
>>>> to always allocate a "struct btf_header" and copy the valid
>>>> portion from the raw data to it; this means we can always safely
>>>> check fields like btf->hdr->kind_layout_len.
>>>>
>>>> Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
>>>> ---
>>>>  tools/lib/bpf/btf.c | 260 +++++++++++++++++++++++++++++++-------------
>>>>  1 file changed, 183 insertions(+), 77 deletions(-)
>>>>
>>>> diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
>>>> index b136572e889a..8835aee6ee84 100644
>>>> --- a/tools/lib/bpf/btf.c
>>>> +++ b/tools/lib/bpf/btf.c
>>>> @@ -40,42 +40,53 @@ struct btf {
>>>>
>>>>         /*
>>>>          * When BTF is loaded from an ELF or raw memory it is stored
>>>> -        * in a contiguous memory block. The hdr, type_data, and, strs_data
>>>> +        * in a contiguous memory block. The  type_data, and, strs_data
>>>
>>> nit: two spaces, and so many commas around and ;) let's leave Oxford
>>> comma, but comma after and is weird
>>>
>>>>          * point inside that memory region to their respective parts of BTF
>>>>          * representation:
>>>>          *
>>>> -        * +--------------------------------+
>>>> -        * |  Header  |  Types  |  Strings  |
>>>> -        * +--------------------------------+
>>>> -        * ^          ^         ^
>>>> -        * |          |         |
>>>> -        * hdr        |         |
>>>> -        * types_data-+         |
>>>> -        * strs_data------------+
>>>> +        * +--------------------------------+---------------------+
>>>> +        * |  Header  |  Types  |  Strings  |Optional kind layout |
>>>
>>> Space missing, boo. Keep diagrams beautiful!..
>>>
>>>> +        * +--------------------------------+---------------------+
>>>> +        * ^          ^         ^           ^
>>>> +        * |          |         |           |
>>>> +        * raw_data   |         |           |
>>>> +        * types_data-+         |           |
>>>> +        * strs_data------------+           |
>>>> +        * kind_layout----------------------+
>>>> +        *
>>>> +        * A separate struct btf_header is allocated for btf->hdr,
>>>> +        * and header information is copied into it.  This allows us
>>>> +        * to handle header data for various header formats; the original,
>>>> +        * the extended header with kind layout, etc.
>>>>          *
>>>>          * If BTF data is later modified, e.g., due to types added or
>>>>          * removed, BTF deduplication performed, etc, this contiguous
>>>> -        * representation is broken up into three independently allocated
>>>> -        * memory regions to be able to modify them independently.
>>>> +        * representation is broken up into four independent memory
>>>> +        * regions.
>>>> +        *
>>>>          * raw_data is nulled out at that point, but can be later allocated
>>>>          * and cached again if user calls btf__raw_data(), at which point
>>>> -        * raw_data will contain a contiguous copy of header, types, and
>>>> -        * strings:
>>>> +        * raw_data will contain a contiguous copy of header, types, strings
>>>> +        * and optionally kind_layout.  kind_layout optionally points to a
>>>> +        * kind_layout array - this allows us to encode information about
>>>> +        * the kinds known at encoding time.  If kind_layout is NULL no
>>>> +        * kind information is encoded.
>>>>          *
>>>> -        * +----------+  +---------+  +-----------+
>>>> -        * |  Header  |  |  Types  |  |  Strings  |
>>>> -        * +----------+  +---------+  +-----------+
>>>> -        * ^             ^            ^
>>>> -        * |             |            |
>>>> -        * hdr           |            |
>>>> -        * types_data----+            |
>>>> -        * strset__data(strs_set)-----+
>>>> +        * +----------+  +---------+  +-----------+   +-----------+
>>>> +        * |  Header  |  |  Types  |  |  Strings  |   |kind_layout|
>>>> +        * +----------+  +---------+  +-----------+   +-----------+
>>>
>>> nit: spaces (and if we go with "layout" naming, this will be short and
>>> beautiful " Layout " ;)
>>>
>>>> +        * ^             ^            ^               ^
>>>> +        * |             |            |               |
>>>> +        * hdr           |            |               |
>>>> +        * types_data----+            |               |
>>>> +        * strset__data(strs_set)-----+               |
>>>> +        * kind_layout--------------------------------+
>>>
>>> [...]
>>>
>>>> @@ -3888,7 +3989,7 @@ static int btf_dedup_strings(struct btf_dedup *d)
>>>>
>>>>         /* replace BTF string data and hash with deduped ones */
>>>>         strset__free(d->btf->strs_set);
>>>> -       d->btf->hdr->str_len = strset__data_size(d->strs_set);
>>>> +       btf_hdr_update_str_len(d->btf, strset__data_size(d->strs_set));
>>>>         d->btf->strs_set = d->strs_set;
>>>>         d->strs_set = NULL;
>>>>         d->btf->strs_deduped = true;
>>>> @@ -5343,6 +5444,11 @@ static int btf_dedup_compact_types(struct btf_dedup *d)
>>>>         d->btf->type_offs = new_offs;
>>>>         d->btf->hdr->str_off = d->btf->hdr->type_len;
>>>>         d->btf->raw_size = d->btf->hdr->hdr_len + d->btf->hdr->type_len + d->btf->hdr->str_len;
>>>> +       if (d->btf->kind_layout) {
>>>> +               d->btf->hdr->kind_layout_off = d->btf->hdr->str_off + roundup(d->btf->hdr->str_len,
>>>> +                                                                             4);
>>>> +               d->btf->raw_size = roundup(d->btf->raw_size, 4) + d->btf->hdr->kind_layout_len;
>>>
>>> maybe put layout data after type data, but before strings? rounding up
>>> string section which is byte-based feels weird. I think old libbpf
>>> implementations should handle all this well, because btf_header
>>> explicitly specifies string section offset, no?
>>>
>>
>> That sounds good, but I think there are some strictness issues with how we parse
>> BTF on the kernel side that we may need to think about, especially if we want to
>> make kind layout always available. In that case we'd need to think how old kernels
>> built with newer pahole might handle newer headers with layout info.
>>
>> First in btf_parse_hdr() the kernel rejects BTF with non-zero unsupported fields.
>> So trying to load vmlinux BTF generated by a pahole that adds layout info will
>> fail for such a kernel.
>>
>> Second when validating section info in btf_check_sec_info() we check for overlaps
>> between known sections, and we also check for gaps between known sections. Finally we
>> also check for any additional data other than the known section data.
> 
> I thought we don't validate gaps, I missed btf_check_sec_info()
> checks, though. Good for kernel, it should be strict.
> 
> But it's easy to drop this layout info in libbpf for BTF sanitization,
> this shouldn't be a problem. Just shift everything to the left and
> adjust strs_off.
> 
>>
>> For layout info stored between type+strings we'd wind up rejecting it for a few reasons:
>>
>> 1. we'd find non-zero data in the header (layout offset/len)
>> 2. we'd find a "gap" between types+strings (the layout data)
>>
>> Similarly with layout at the end
>>
>> 1. we'd find non-zero data in the header (kind layout offset/len)
>> 2. we'd find unaccounted-for data after the string data (the kind layout data)
>>
>> So either way we'd wind up with unsupported headers. One approach would be to
>> do stable backports relaxing these header tests; I think we could relax them to
>> simply ensure no overlap between sections and that sections don't overrun data
>> length without risking having unusable BTF. Then a newer BTF header with additional
>> layout info wouldn't get rejected. What do you think?
> 
> See above, I don't see why we can't just sanitize BTF and drop layout
> parts altogether. They are optional for kernel either way, no harm in
> dropping them.
>

The sanitization for user-space consumption is doable alright, I was thinking
of the case where the kernel itself reads in BTF for vmlinux/modules on boot,
and that BTF was generated by newer pahole so has unexpected layout info. 
If we just emitted layout info unconditionally that would mean newer pahole might
generate BTf for a kernel that it could not read. If however we relaxed the
constraints a bit I think we could get the validation to succeed for older
kernels while ignoring the bits of the BTF they don't care about. Fix that would
also potentially future-proof addition of other sections to the BTF header without
requiring options.

Alan

Re: [PATCH v8 bpf-next 01/10] btf: add kind layout encoding to UAPI

[Andrii Nakryiko]

On Fri, Dec 19, 2025 at 10:14 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>
> On 19/12/2025 17:53, Andrii Nakryiko wrote:
> > On Fri, Dec 19, 2025 at 5:15 AM Alan Maguire <alan.maguire@oracle.com> wrote:
> >>
> >> On 16/12/2025 19:23, Andrii Nakryiko wrote:
> >>> On Mon, Dec 15, 2025 at 1:18 AM Alan Maguire <alan.maguire@oracle.com> wrote:
> >>>>
> >>>> BTF kind layouts provide information to parse BTF kinds. By separating
> >>>> parsing BTF from using all the information it provides, we allow BTF
> >>>> to encode new features even if they cannot be used by readers. This
> >>>> will be helpful in particular for cases where older tools are used
> >>>> to parse newer BTF with kinds the older tools do not recognize;
> >>>> the BTF can still be parsed in such cases using kind layout.
> >>>>
> >>>> The intent is to support encoding of kind layouts optionally so that
> >>>> tools like pahole can add this information. For each kind, we record
> >>>>
> >>>> - length of singular element following struct btf_type
> >>>> - length of each of the btf_vlen() elements following
> >>>>
> >>>> The ideas here were discussed at [1], [2]; hence
> >>>>
> >>>> Suggested-by: Andrii Nakryiko <andrii@kernel.org>
> >>>> Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
> >>>>
> >>>> [1] https://lore.kernel.org/bpf/CAEf4BzYjWHRdNNw4B=eOXOs_ONrDwrgX4bn=Nuc1g8JPFC34MA@mail.gmail.com/
> >>>> [2] https://lore.kernel.org/bpf/20230531201936.1992188-1-alan.maguire@oracle.com/
> >>>> ---
> >>>>  include/uapi/linux/btf.h       | 11 +++++++++++
> >>>>  tools/include/uapi/linux/btf.h | 11 +++++++++++
> >>>>  2 files changed, 22 insertions(+)
> >>>>
> >>>> diff --git a/include/uapi/linux/btf.h b/include/uapi/linux/btf.h
> >>>> index 266d4ffa6c07..c1854a1c7b38 100644
> >>>> --- a/include/uapi/linux/btf.h
> >>>> +++ b/include/uapi/linux/btf.h
> >>>> @@ -8,6 +8,15 @@
> >>>>  #define BTF_MAGIC      0xeB9F
> >>>>  #define BTF_VERSION    1
> >>>>
> >>>> +/*
> >>>> + * kind layout section consists of a struct btf_kind_layout for each known
> >>>> + * kind at BTF encoding time.
> >>>> + */
> >>>> +struct btf_kind_layout {
> >>>> +       __u8 info_sz;           /* size of singular element after btf_type */
> >>>> +       __u8 elem_sz;           /* size of each of btf_vlen(t) elements */
> >>>
> >>> So Eduard pointed out that at some point we discussed having a name of
> >>> a kind (i.e., "struct", "typedef", etc). By now I have no recollection
> >>> what were the arguments, do you remember? I'm not sure how I feel now
> >>> about having extra 4 bytes per kind, but that's not really a lot of
> >>> data (20*4 = 80 bytes added), so might as well add it, I suppose?
> >>>
> >>
> >> Yeah we went back and forth on that; I think it's on balance worthwhile
> >> to be honest; tools can be a bit more expressive about what's missing.
> >>
> >>> I think we were also discussing having flags per kind to designate
> >>> some extra semantics, where applicable. Again, don't remember
> >>> arguments for or against, but one case where I think this would be
> >>> very beneficial is when we add something like type_tag, which is
> >>> inevitably used from "normal" struct and will be almost inevitable in
> >>> normal vmlinux BTF. Think about it, we have some field which will be
> >>> CONST -> PTR -> TYPE_TAG -> STRUCT. That TYPE_TAG shouldn't just
> >>> totally break (old) bpftool's dump, as it really can be easily ignored
> >>> **if we know TYPE_TAG can be ignored and it is just a reference
> >>> type**. That reference type means that there is another type pointed
> >>> to using struct btf_type::type field (instead of that field being a
> >>> size).
> >>>
> >>> So I think it would be nice to encode this as a flag that says a) kind
> >>> can be ignored without compromising type integrity (i.e., memory
> >>> layout is preserved) which will be true for all kinds of modifier
> >>> kinds (const/volatile/restrict/type_tag, even for typedef that should
> >>> be true) and b) kind is reference type, so struct btf_type::type is a
> >>> "pointer" to a valid other underlying type.
> >>>
> >>> Thoughts?
> >>>
> >>
> >> Again we did go back and forth here but to me there's much more value in
> >> being both able to parse _and_ sanitize BTF, at least for the simple cases.
> >> What we can include are as you say types in the type graph that are optional
> >> reference kinds (like type tag), and kinds that are not implicated in the
> >> known type graph like the location stuff (it only points _to_ known kinds,
> >> no known kinds will point to location data). So any case where known
> >> types + optional ref types constitute the type graph we are good.
> >> Anything more complex than these would involve having to represent the
> >> layout of type references within unknown kinds (kind of like what we do for
> >> field iteration) which seems a bit much.
> >>
> >> Now one thing that we might want to introduce here is a sanitization-friendly
> >> kind, either re-using BTF_KIND_UNKN or adding a new vlen-supporting kind
> >> which can be used to overwrite kinds we don't want in the sanitized output.
> >> We need this to preserve the type ids for the kernel BTF we sanitize.
> >> I get that it seems weird to add a new incompatibility to handle incompatibility,
> >> but the sooner we do it the better I guess. The reason I suggest it now is we'd
> >> potentially need some more complex sanitization for the location stuff for
> >> cases like large location sections, and it might be cleaner to have a special
> >> "ignore this it's just sanitization info" kind, especially for cases like
> >> BTF C dump.
> >
> > So you mean you'd like some "dummy" BTF kind with 4-byte-per-vlen so
> > we can "overwrite" any possible unknown BTF kind?.. As you said,
> > though, this would only work for new kernels, so that's sad... I don't
> > know, I don't hate the idea, but curious what others think.
> >
> > Alternatively, we can just try to never add kinds where the vlen
> > element is not a multiple of 8 or 12. We can then use ENUM
> > (8-bytes-per-vlen) or ENUM64 (12-bytes-per-vlen) to paper over unknown
> > types. FUNC_PROTO (8-bytes-per-vlen) and DATASEC (12-bytes-per-vlen)
> > are other options. We just don't have 4-bytes-per-vlen for the most
> > universal "filler", unfortunately.
> >
> > The advantage of the latter is full backwards compatibility with old kernels.
> >
>
> True. And I guess during sanitization we can just handle intermediate
> types in a type graph by adjusting type ids to skip over them, so we
> likely have everything we need already. Funnily enough the BTF location
> stuff will give us a vlen-specified 4 byte object (specifying the
> location parameters associated with an inline), so that will help in
> the future for cases where it is recognized but other kinds are not.

So coming back to flags? Let's do two flags: "safe modifier-like
reference kind" (for type_tag-like things where they can be dropped
from the chain of types) and "safe to ignore non-structural type" that
can't be part of any struct/union and are more like decl_tag where
they only reference other types, but can be dropped/replaced with
something? And if kind doesn't have either of those, we won't attempt
to sanitize (and hopefully we won't even have kinds like that, but if
necessary we can add more flags with some other "safe" semantics, if
necessary?)

>
> Alan

Re: [PATCH v8 bpf-next 02/10] libbpf: Support kind layout section handling in BTF

[Andrii Nakryiko]

On Fri, Dec 19, 2025 at 10:19 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>
> On 19/12/2025 17:58, Andrii Nakryiko wrote:
> > On Fri, Dec 19, 2025 at 5:34 AM Alan Maguire <alan.maguire@oracle.com> wrote:
> >>
> >> On 16/12/2025 19:34, Andrii Nakryiko wrote:
> >>> On Mon, Dec 15, 2025 at 1:18 AM Alan Maguire <alan.maguire@oracle.com> wrote:
> >>>>
> >>>> Support reading in kind layout fixing endian issues on reading;
> >>>> also support writing kind layout section to raw BTF object.
> >>>> There is not yet an API to populate the kind layout with meaningful
> >>>> information.
> >>>>
> >>>> As part of this, we need to consider multiple valid BTF header
> >>>> sizes; the original or the kind layout-extended headers.
> >>>> So to support this, the "struct btf" representation is modified
> >>>> to always allocate a "struct btf_header" and copy the valid
> >>>> portion from the raw data to it; this means we can always safely
> >>>> check fields like btf->hdr->kind_layout_len.
> >>>>
> >>>> Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
> >>>> ---
> >>>>  tools/lib/bpf/btf.c | 260 +++++++++++++++++++++++++++++++-------------
> >>>>  1 file changed, 183 insertions(+), 77 deletions(-)
> >>>>
> >>>> diff --git a/tools/lib/bpf/btf.c b/tools/lib/bpf/btf.c
> >>>> index b136572e889a..8835aee6ee84 100644
> >>>> --- a/tools/lib/bpf/btf.c
> >>>> +++ b/tools/lib/bpf/btf.c
> >>>> @@ -40,42 +40,53 @@ struct btf {
> >>>>
> >>>>         /*
> >>>>          * When BTF is loaded from an ELF or raw memory it is stored
> >>>> -        * in a contiguous memory block. The hdr, type_data, and, strs_data
> >>>> +        * in a contiguous memory block. The  type_data, and, strs_data
> >>>
> >>> nit: two spaces, and so many commas around and ;) let's leave Oxford
> >>> comma, but comma after and is weird
> >>>
> >>>>          * point inside that memory region to their respective parts of BTF
> >>>>          * representation:
> >>>>          *
> >>>> -        * +--------------------------------+
> >>>> -        * |  Header  |  Types  |  Strings  |
> >>>> -        * +--------------------------------+
> >>>> -        * ^          ^         ^
> >>>> -        * |          |         |
> >>>> -        * hdr        |         |
> >>>> -        * types_data-+         |
> >>>> -        * strs_data------------+
> >>>> +        * +--------------------------------+---------------------+
> >>>> +        * |  Header  |  Types  |  Strings  |Optional kind layout |
> >>>
> >>> Space missing, boo. Keep diagrams beautiful!..
> >>>
> >>>> +        * +--------------------------------+---------------------+
> >>>> +        * ^          ^         ^           ^
> >>>> +        * |          |         |           |
> >>>> +        * raw_data   |         |           |
> >>>> +        * types_data-+         |           |
> >>>> +        * strs_data------------+           |
> >>>> +        * kind_layout----------------------+
> >>>> +        *
> >>>> +        * A separate struct btf_header is allocated for btf->hdr,
> >>>> +        * and header information is copied into it.  This allows us
> >>>> +        * to handle header data for various header formats; the original,
> >>>> +        * the extended header with kind layout, etc.
> >>>>          *
> >>>>          * If BTF data is later modified, e.g., due to types added or
> >>>>          * removed, BTF deduplication performed, etc, this contiguous
> >>>> -        * representation is broken up into three independently allocated
> >>>> -        * memory regions to be able to modify them independently.
> >>>> +        * representation is broken up into four independent memory
> >>>> +        * regions.
> >>>> +        *
> >>>>          * raw_data is nulled out at that point, but can be later allocated
> >>>>          * and cached again if user calls btf__raw_data(), at which point
> >>>> -        * raw_data will contain a contiguous copy of header, types, and
> >>>> -        * strings:
> >>>> +        * raw_data will contain a contiguous copy of header, types, strings
> >>>> +        * and optionally kind_layout.  kind_layout optionally points to a
> >>>> +        * kind_layout array - this allows us to encode information about
> >>>> +        * the kinds known at encoding time.  If kind_layout is NULL no
> >>>> +        * kind information is encoded.
> >>>>          *
> >>>> -        * +----------+  +---------+  +-----------+
> >>>> -        * |  Header  |  |  Types  |  |  Strings  |
> >>>> -        * +----------+  +---------+  +-----------+
> >>>> -        * ^             ^            ^
> >>>> -        * |             |            |
> >>>> -        * hdr           |            |
> >>>> -        * types_data----+            |
> >>>> -        * strset__data(strs_set)-----+
> >>>> +        * +----------+  +---------+  +-----------+   +-----------+
> >>>> +        * |  Header  |  |  Types  |  |  Strings  |   |kind_layout|
> >>>> +        * +----------+  +---------+  +-----------+   +-----------+
> >>>
> >>> nit: spaces (and if we go with "layout" naming, this will be short and
> >>> beautiful " Layout " ;)
> >>>
> >>>> +        * ^             ^            ^               ^
> >>>> +        * |             |            |               |
> >>>> +        * hdr           |            |               |
> >>>> +        * types_data----+            |               |
> >>>> +        * strset__data(strs_set)-----+               |
> >>>> +        * kind_layout--------------------------------+
> >>>
> >>> [...]
> >>>
> >>>> @@ -3888,7 +3989,7 @@ static int btf_dedup_strings(struct btf_dedup *d)
> >>>>
> >>>>         /* replace BTF string data and hash with deduped ones */
> >>>>         strset__free(d->btf->strs_set);
> >>>> -       d->btf->hdr->str_len = strset__data_size(d->strs_set);
> >>>> +       btf_hdr_update_str_len(d->btf, strset__data_size(d->strs_set));
> >>>>         d->btf->strs_set = d->strs_set;
> >>>>         d->strs_set = NULL;
> >>>>         d->btf->strs_deduped = true;
> >>>> @@ -5343,6 +5444,11 @@ static int btf_dedup_compact_types(struct btf_dedup *d)
> >>>>         d->btf->type_offs = new_offs;
> >>>>         d->btf->hdr->str_off = d->btf->hdr->type_len;
> >>>>         d->btf->raw_size = d->btf->hdr->hdr_len + d->btf->hdr->type_len + d->btf->hdr->str_len;
> >>>> +       if (d->btf->kind_layout) {
> >>>> +               d->btf->hdr->kind_layout_off = d->btf->hdr->str_off + roundup(d->btf->hdr->str_len,
> >>>> +                                                                             4);
> >>>> +               d->btf->raw_size = roundup(d->btf->raw_size, 4) + d->btf->hdr->kind_layout_len;
> >>>
> >>> maybe put layout data after type data, but before strings? rounding up
> >>> string section which is byte-based feels weird. I think old libbpf
> >>> implementations should handle all this well, because btf_header
> >>> explicitly specifies string section offset, no?
> >>>
> >>
> >> That sounds good, but I think there are some strictness issues with how we parse
> >> BTF on the kernel side that we may need to think about, especially if we want to
> >> make kind layout always available. In that case we'd need to think how old kernels
> >> built with newer pahole might handle newer headers with layout info.
> >>
> >> First in btf_parse_hdr() the kernel rejects BTF with non-zero unsupported fields.
> >> So trying to load vmlinux BTF generated by a pahole that adds layout info will
> >> fail for such a kernel.
> >>
> >> Second when validating section info in btf_check_sec_info() we check for overlaps
> >> between known sections, and we also check for gaps between known sections. Finally we
> >> also check for any additional data other than the known section data.
> >
> > I thought we don't validate gaps, I missed btf_check_sec_info()
> > checks, though. Good for kernel, it should be strict.
> >
> > But it's easy to drop this layout info in libbpf for BTF sanitization,
> > this shouldn't be a problem. Just shift everything to the left and
> > adjust strs_off.
> >
> >>
> >> For layout info stored between type+strings we'd wind up rejecting it for a few reasons:
> >>
> >> 1. we'd find non-zero data in the header (layout offset/len)
> >> 2. we'd find a "gap" between types+strings (the layout data)
> >>
> >> Similarly with layout at the end
> >>
> >> 1. we'd find non-zero data in the header (kind layout offset/len)
> >> 2. we'd find unaccounted-for data after the string data (the kind layout data)
> >>
> >> So either way we'd wind up with unsupported headers. One approach would be to
> >> do stable backports relaxing these header tests; I think we could relax them to
> >> simply ensure no overlap between sections and that sections don't overrun data
> >> length without risking having unusable BTF. Then a newer BTF header with additional
> >> layout info wouldn't get rejected. What do you think?
> >
> > See above, I don't see why we can't just sanitize BTF and drop layout
> > parts altogether. They are optional for kernel either way, no harm in
> > dropping them.
> >
>
> The sanitization for user-space consumption is doable alright, I was thinking
> of the case where the kernel itself reads in BTF for vmlinux/modules on boot,
> and that BTF was generated by newer pahole so has unexpected layout info.
> If we just emitted layout info unconditionally that would mean newer pahole might
> generate BTf for a kernel that it could not read. If however we relaxed the
> constraints a bit I think we could get the validation to succeed for older
> kernels while ignoring the bits of the BTF they don't care about. Fix that would
> also potentially future-proof addition of other sections to the BTF header without
> requiring options.

No, let's forget about allowing the kernel to let through some
unrecognized parts of BTF. Pahole will keep introducing feature flags
that we need to enable (like layout stuff, for example), so old
kernels built with new pahole will be just fine. And any
kernel-specific modifications will be moved to resolve_btfids and will
be in-sync with kernel logic. I think we are all good and we don't
have to invent new things on this front, potentially opening us up to
some unforeseen attacks through BTF injection.

>
> Alan

Re: [PATCH v8 bpf-next 01/10] btf: add kind layout encoding to UAPI

[Alan Maguire]

On 19/12/2025 18:19, Andrii Nakryiko wrote:
> On Fri, Dec 19, 2025 at 10:14 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>>
>> On 19/12/2025 17:53, Andrii Nakryiko wrote:
>>> On Fri, Dec 19, 2025 at 5:15 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>>>>
>>>> On 16/12/2025 19:23, Andrii Nakryiko wrote:
>>>>> On Mon, Dec 15, 2025 at 1:18 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>>>>>>
>>>>>> BTF kind layouts provide information to parse BTF kinds. By separating
>>>>>> parsing BTF from using all the information it provides, we allow BTF
>>>>>> to encode new features even if they cannot be used by readers. This
>>>>>> will be helpful in particular for cases where older tools are used
>>>>>> to parse newer BTF with kinds the older tools do not recognize;
>>>>>> the BTF can still be parsed in such cases using kind layout.
>>>>>>
>>>>>> The intent is to support encoding of kind layouts optionally so that
>>>>>> tools like pahole can add this information. For each kind, we record
>>>>>>
>>>>>> - length of singular element following struct btf_type
>>>>>> - length of each of the btf_vlen() elements following
>>>>>>
>>>>>> The ideas here were discussed at [1], [2]; hence
>>>>>>
>>>>>> Suggested-by: Andrii Nakryiko <andrii@kernel.org>
>>>>>> Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
>>>>>>
>>>>>> [1] https://lore.kernel.org/bpf/CAEf4BzYjWHRdNNw4B=eOXOs_ONrDwrgX4bn=Nuc1g8JPFC34MA@mail.gmail.com/
>>>>>> [2] https://lore.kernel.org/bpf/20230531201936.1992188-1-alan.maguire@oracle.com/
>>>>>> ---
>>>>>>  include/uapi/linux/btf.h       | 11 +++++++++++
>>>>>>  tools/include/uapi/linux/btf.h | 11 +++++++++++
>>>>>>  2 files changed, 22 insertions(+)
>>>>>>
>>>>>> diff --git a/include/uapi/linux/btf.h b/include/uapi/linux/btf.h
>>>>>> index 266d4ffa6c07..c1854a1c7b38 100644
>>>>>> --- a/include/uapi/linux/btf.h
>>>>>> +++ b/include/uapi/linux/btf.h
>>>>>> @@ -8,6 +8,15 @@
>>>>>>  #define BTF_MAGIC      0xeB9F
>>>>>>  #define BTF_VERSION    1
>>>>>>
>>>>>> +/*
>>>>>> + * kind layout section consists of a struct btf_kind_layout for each known
>>>>>> + * kind at BTF encoding time.
>>>>>> + */
>>>>>> +struct btf_kind_layout {
>>>>>> +       __u8 info_sz;           /* size of singular element after btf_type */
>>>>>> +       __u8 elem_sz;           /* size of each of btf_vlen(t) elements */
>>>>>
>>>>> So Eduard pointed out that at some point we discussed having a name of
>>>>> a kind (i.e., "struct", "typedef", etc). By now I have no recollection
>>>>> what were the arguments, do you remember? I'm not sure how I feel now
>>>>> about having extra 4 bytes per kind, but that's not really a lot of
>>>>> data (20*4 = 80 bytes added), so might as well add it, I suppose?
>>>>>
>>>>
>>>> Yeah we went back and forth on that; I think it's on balance worthwhile
>>>> to be honest; tools can be a bit more expressive about what's missing.
>>>>
>>>>> I think we were also discussing having flags per kind to designate
>>>>> some extra semantics, where applicable. Again, don't remember
>>>>> arguments for or against, but one case where I think this would be
>>>>> very beneficial is when we add something like type_tag, which is
>>>>> inevitably used from "normal" struct and will be almost inevitable in
>>>>> normal vmlinux BTF. Think about it, we have some field which will be
>>>>> CONST -> PTR -> TYPE_TAG -> STRUCT. That TYPE_TAG shouldn't just
>>>>> totally break (old) bpftool's dump, as it really can be easily ignored
>>>>> **if we know TYPE_TAG can be ignored and it is just a reference
>>>>> type**. That reference type means that there is another type pointed
>>>>> to using struct btf_type::type field (instead of that field being a
>>>>> size).
>>>>>
>>>>> So I think it would be nice to encode this as a flag that says a) kind
>>>>> can be ignored without compromising type integrity (i.e., memory
>>>>> layout is preserved) which will be true for all kinds of modifier
>>>>> kinds (const/volatile/restrict/type_tag, even for typedef that should
>>>>> be true) and b) kind is reference type, so struct btf_type::type is a
>>>>> "pointer" to a valid other underlying type.
>>>>>
>>>>> Thoughts?
>>>>>
>>>>
>>>> Again we did go back and forth here but to me there's much more value in
>>>> being both able to parse _and_ sanitize BTF, at least for the simple cases.
>>>> What we can include are as you say types in the type graph that are optional
>>>> reference kinds (like type tag), and kinds that are not implicated in the
>>>> known type graph like the location stuff (it only points _to_ known kinds,
>>>> no known kinds will point to location data). So any case where known
>>>> types + optional ref types constitute the type graph we are good.
>>>> Anything more complex than these would involve having to represent the
>>>> layout of type references within unknown kinds (kind of like what we do for
>>>> field iteration) which seems a bit much.
>>>>
>>>> Now one thing that we might want to introduce here is a sanitization-friendly
>>>> kind, either re-using BTF_KIND_UNKN or adding a new vlen-supporting kind
>>>> which can be used to overwrite kinds we don't want in the sanitized output.
>>>> We need this to preserve the type ids for the kernel BTF we sanitize.
>>>> I get that it seems weird to add a new incompatibility to handle incompatibility,
>>>> but the sooner we do it the better I guess. The reason I suggest it now is we'd
>>>> potentially need some more complex sanitization for the location stuff for
>>>> cases like large location sections, and it might be cleaner to have a special
>>>> "ignore this it's just sanitization info" kind, especially for cases like
>>>> BTF C dump.
>>>
>>> So you mean you'd like some "dummy" BTF kind with 4-byte-per-vlen so
>>> we can "overwrite" any possible unknown BTF kind?.. As you said,
>>> though, this would only work for new kernels, so that's sad... I don't
>>> know, I don't hate the idea, but curious what others think.
>>>
>>> Alternatively, we can just try to never add kinds where the vlen
>>> element is not a multiple of 8 or 12. We can then use ENUM
>>> (8-bytes-per-vlen) or ENUM64 (12-bytes-per-vlen) to paper over unknown
>>> types. FUNC_PROTO (8-bytes-per-vlen) and DATASEC (12-bytes-per-vlen)
>>> are other options. We just don't have 4-bytes-per-vlen for the most
>>> universal "filler", unfortunately.
>>>
>>> The advantage of the latter is full backwards compatibility with old kernels.
>>>
>>
>> True. And I guess during sanitization we can just handle intermediate
>> types in a type graph by adjusting type ids to skip over them, so we
>> likely have everything we need already. Funnily enough the BTF location
>> stuff will give us a vlen-specified 4 byte object (specifying the
>> location parameters associated with an inline), so that will help in
>> the future for cases where it is recognized but other kinds are not.
> 
> So coming back to flags? Let's do two flags: "safe modifier-like
> reference kind" (for type_tag-like things where they can be dropped
> from the chain of types) and "safe to ignore non-structural type" that
> can't be part of any struct/union and are more like decl_tag where
> they only reference other types, but can be dropped/replaced with
> something? And if kind doesn't have either of those, we won't attempt
> to sanitize (and hopefully we won't even have kinds like that, but if
> necessary we can add more flags with some other "safe" semantics, if
> necessary?)
>

Sounds perfect, thanks!

Re: [PATCH v8 bpf-next 02/10] libbpf: Support kind layout section handling in BTF

[Eduard Zingerman]

On Fri, 2025-12-19 at 10:21 -0800, Andrii Nakryiko wrote:

[...]

> > The sanitization for user-space consumption is doable alright, I was thinking
> > of the case where the kernel itself reads in BTF for vmlinux/modules on boot,
> > and that BTF was generated by newer pahole so has unexpected layout info.
> > If we just emitted layout info unconditionally that would mean newer pahole might
> > generate BTf for a kernel that it could not read. If however we relaxed the
> > constraints a bit I think we could get the validation to succeed for older
> > kernels while ignoring the bits of the BTF they don't care about. Fix that would
> > also potentially future-proof addition of other sections to the BTF header without
> > requiring options.
> 
> No, let's forget about allowing the kernel to let through some
> unrecognized parts of BTF. Pahole will keep introducing feature flags
> that we need to enable (like layout stuff, for example), so old
> kernels built with new pahole will be just fine. And any
> kernel-specific modifications will be moved to resolve_btfids and will
> be in-sync with kernel logic. I think we are all good and we don't
> have to invent new things on this front, potentially opening us up to
> some unforeseen attacks through BTF injection.

That would mean that the flag to generate or not layout information
should remain, right?

Re: [PATCH v8 bpf-next 02/10] libbpf: Support kind layout section handling in BTF

[Andrii Nakryiko]

On Fri, Dec 19, 2025 at 10:36 AM Eduard Zingerman <eddyz87@gmail.com> wrote:
>
> On Fri, 2025-12-19 at 10:21 -0800, Andrii Nakryiko wrote:
>
> [...]
>
> > > The sanitization for user-space consumption is doable alright, I was thinking
> > > of the case where the kernel itself reads in BTF for vmlinux/modules on boot,
> > > and that BTF was generated by newer pahole so has unexpected layout info.
> > > If we just emitted layout info unconditionally that would mean newer pahole might
> > > generate BTf for a kernel that it could not read. If however we relaxed the
> > > constraints a bit I think we could get the validation to succeed for older
> > > kernels while ignoring the bits of the BTF they don't care about. Fix that would
> > > also potentially future-proof addition of other sections to the BTF header without
> > > requiring options.
> >
> > No, let's forget about allowing the kernel to let through some
> > unrecognized parts of BTF. Pahole will keep introducing feature flags
> > that we need to enable (like layout stuff, for example), so old
> > kernels built with new pahole will be just fine. And any
> > kernel-specific modifications will be moved to resolve_btfids and will
> > be in-sync with kernel logic. I think we are all good and we don't
> > have to invent new things on this front, potentially opening us up to
> > some unforeseen attacks through BTF injection.
>
> That would mean that the flag to generate or not layout information
> should remain, right?

Yeah, unfortunately, as there will be no libbpf to sanitize that. But
that brings the layout question for Clang/GCC, are we adding that?
That should be emitted unconditionally, IMO.

Re: [PATCH v8 bpf-next 02/10] libbpf: Support kind layout section handling in BTF

[Eduard Zingerman]

On Fri, 2025-12-19 at 10:41 -0800, Andrii Nakryiko wrote:
> On Fri, Dec 19, 2025 at 10:36 AM Eduard Zingerman <eddyz87@gmail.com> wrote:
> > 
> > On Fri, 2025-12-19 at 10:21 -0800, Andrii Nakryiko wrote:
> > 
> > [...]
> > 
> > > > The sanitization for user-space consumption is doable alright, I was thinking
> > > > of the case where the kernel itself reads in BTF for vmlinux/modules on boot,
> > > > and that BTF was generated by newer pahole so has unexpected layout info.
> > > > If we just emitted layout info unconditionally that would mean newer pahole might
> > > > generate BTf for a kernel that it could not read. If however we relaxed the
> > > > constraints a bit I think we could get the validation to succeed for older
> > > > kernels while ignoring the bits of the BTF they don't care about. Fix that would
> > > > also potentially future-proof addition of other sections to the BTF header without
> > > > requiring options.
> > > 
> > > No, let's forget about allowing the kernel to let through some
> > > unrecognized parts of BTF. Pahole will keep introducing feature flags
> > > that we need to enable (like layout stuff, for example), so old
> > > kernels built with new pahole will be just fine. And any
> > > kernel-specific modifications will be moved to resolve_btfids and will
> > > be in-sync with kernel logic. I think we are all good and we don't
> > > have to invent new things on this front, potentially opening us up to
> > > some unforeseen attacks through BTF injection.
> > 
> > That would mean that the flag to generate or not layout information
> > should remain, right?
> 
> Yeah, unfortunately, as there will be no libbpf to sanitize that. But
> that brings the layout question for Clang/GCC, are we adding that?
> That should be emitted unconditionally, IMO.

I don't think we have layout generation in the current roadmap.
I'll add it to the plan.

Re: [PATCH v8 bpf-next 01/10] btf: add kind layout encoding to UAPI

[Alexei Starovoitov]

On Fri, Dec 19, 2025 at 8:19 AM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Fri, Dec 19, 2025 at 10:14 AM Alan Maguire <alan.maguire@oracle.com> wrote:
> >
> > On 19/12/2025 17:53, Andrii Nakryiko wrote:
> > > On Fri, Dec 19, 2025 at 5:15 AM Alan Maguire <alan.maguire@oracle.com> wrote:
> > >>
> > >> On 16/12/2025 19:23, Andrii Nakryiko wrote:
> > >>> On Mon, Dec 15, 2025 at 1:18 AM Alan Maguire <alan.maguire@oracle.com> wrote:
> > >>>>
> > >>>> BTF kind layouts provide information to parse BTF kinds. By separating
> > >>>> parsing BTF from using all the information it provides, we allow BTF
> > >>>> to encode new features even if they cannot be used by readers. This
> > >>>> will be helpful in particular for cases where older tools are used
> > >>>> to parse newer BTF with kinds the older tools do not recognize;
> > >>>> the BTF can still be parsed in such cases using kind layout.
> > >>>>
> > >>>> The intent is to support encoding of kind layouts optionally so that
> > >>>> tools like pahole can add this information. For each kind, we record
> > >>>>
> > >>>> - length of singular element following struct btf_type
> > >>>> - length of each of the btf_vlen() elements following
> > >>>>
> > >>>> The ideas here were discussed at [1], [2]; hence
> > >>>>
> > >>>> Suggested-by: Andrii Nakryiko <andrii@kernel.org>
> > >>>> Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
> > >>>>
> > >>>> [1] https://lore.kernel.org/bpf/CAEf4BzYjWHRdNNw4B=eOXOs_ONrDwrgX4bn=Nuc1g8JPFC34MA@mail.gmail.com/
> > >>>> [2] https://lore.kernel.org/bpf/20230531201936.1992188-1-alan.maguire@oracle.com/
> > >>>> ---
> > >>>>  include/uapi/linux/btf.h       | 11 +++++++++++
> > >>>>  tools/include/uapi/linux/btf.h | 11 +++++++++++
> > >>>>  2 files changed, 22 insertions(+)
> > >>>>
> > >>>> diff --git a/include/uapi/linux/btf.h b/include/uapi/linux/btf.h
> > >>>> index 266d4ffa6c07..c1854a1c7b38 100644
> > >>>> --- a/include/uapi/linux/btf.h
> > >>>> +++ b/include/uapi/linux/btf.h
> > >>>> @@ -8,6 +8,15 @@
> > >>>>  #define BTF_MAGIC      0xeB9F
> > >>>>  #define BTF_VERSION    1
> > >>>>
> > >>>> +/*
> > >>>> + * kind layout section consists of a struct btf_kind_layout for each known
> > >>>> + * kind at BTF encoding time.
> > >>>> + */
> > >>>> +struct btf_kind_layout {
> > >>>> +       __u8 info_sz;           /* size of singular element after btf_type */
> > >>>> +       __u8 elem_sz;           /* size of each of btf_vlen(t) elements */
> > >>>
> > >>> So Eduard pointed out that at some point we discussed having a name of
> > >>> a kind (i.e., "struct", "typedef", etc). By now I have no recollection
> > >>> what were the arguments, do you remember? I'm not sure how I feel now
> > >>> about having extra 4 bytes per kind, but that's not really a lot of
> > >>> data (20*4 = 80 bytes added), so might as well add it, I suppose?
> > >>>
> > >>
> > >> Yeah we went back and forth on that; I think it's on balance worthwhile
> > >> to be honest; tools can be a bit more expressive about what's missing.
> > >>
> > >>> I think we were also discussing having flags per kind to designate
> > >>> some extra semantics, where applicable. Again, don't remember
> > >>> arguments for or against, but one case where I think this would be
> > >>> very beneficial is when we add something like type_tag, which is
> > >>> inevitably used from "normal" struct and will be almost inevitable in
> > >>> normal vmlinux BTF. Think about it, we have some field which will be
> > >>> CONST -> PTR -> TYPE_TAG -> STRUCT. That TYPE_TAG shouldn't just
> > >>> totally break (old) bpftool's dump, as it really can be easily ignored
> > >>> **if we know TYPE_TAG can be ignored and it is just a reference
> > >>> type**. That reference type means that there is another type pointed
> > >>> to using struct btf_type::type field (instead of that field being a
> > >>> size).
> > >>>
> > >>> So I think it would be nice to encode this as a flag that says a) kind
> > >>> can be ignored without compromising type integrity (i.e., memory
> > >>> layout is preserved) which will be true for all kinds of modifier
> > >>> kinds (const/volatile/restrict/type_tag, even for typedef that should
> > >>> be true) and b) kind is reference type, so struct btf_type::type is a
> > >>> "pointer" to a valid other underlying type.
> > >>>
> > >>> Thoughts?
> > >>>
> > >>
> > >> Again we did go back and forth here but to me there's much more value in
> > >> being both able to parse _and_ sanitize BTF, at least for the simple cases.
> > >> What we can include are as you say types in the type graph that are optional
> > >> reference kinds (like type tag), and kinds that are not implicated in the
> > >> known type graph like the location stuff (it only points _to_ known kinds,
> > >> no known kinds will point to location data). So any case where known
> > >> types + optional ref types constitute the type graph we are good.
> > >> Anything more complex than these would involve having to represent the
> > >> layout of type references within unknown kinds (kind of like what we do for
> > >> field iteration) which seems a bit much.
> > >>
> > >> Now one thing that we might want to introduce here is a sanitization-friendly
> > >> kind, either re-using BTF_KIND_UNKN or adding a new vlen-supporting kind
> > >> which can be used to overwrite kinds we don't want in the sanitized output.
> > >> We need this to preserve the type ids for the kernel BTF we sanitize.
> > >> I get that it seems weird to add a new incompatibility to handle incompatibility,
> > >> but the sooner we do it the better I guess. The reason I suggest it now is we'd
> > >> potentially need some more complex sanitization for the location stuff for
> > >> cases like large location sections, and it might be cleaner to have a special
> > >> "ignore this it's just sanitization info" kind, especially for cases like
> > >> BTF C dump.
> > >
> > > So you mean you'd like some "dummy" BTF kind with 4-byte-per-vlen so
> > > we can "overwrite" any possible unknown BTF kind?.. As you said,
> > > though, this would only work for new kernels, so that's sad... I don't
> > > know, I don't hate the idea, but curious what others think.
> > >
> > > Alternatively, we can just try to never add kinds where the vlen
> > > element is not a multiple of 8 or 12. We can then use ENUM
> > > (8-bytes-per-vlen) or ENUM64 (12-bytes-per-vlen) to paper over unknown
> > > types. FUNC_PROTO (8-bytes-per-vlen) and DATASEC (12-bytes-per-vlen)
> > > are other options. We just don't have 4-bytes-per-vlen for the most
> > > universal "filler", unfortunately.
> > >
> > > The advantage of the latter is full backwards compatibility with old kernels.
> > >
> >
> > True. And I guess during sanitization we can just handle intermediate
> > types in a type graph by adjusting type ids to skip over them, so we
> > likely have everything we need already. Funnily enough the BTF location
> > stuff will give us a vlen-specified 4 byte object (specifying the
> > location parameters associated with an inline), so that will help in
> > the future for cases where it is recognized but other kinds are not.
>
> So coming back to flags? Let's do two flags: "safe modifier-like
> reference kind" (for type_tag-like things where they can be dropped
> from the chain of types) and "safe to ignore non-structural type" that
> can't be part of any struct/union and are more like decl_tag where
> they only reference other types, but can be dropped/replaced with
> something? And if kind doesn't have either of those, we won't attempt
> to sanitize (and hopefully we won't even have kinds like that, but if
> necessary we can add more flags with some other "safe" semantics, if
> necessary?)

Hold on. I'm missing how libbpf will sanitize things for older kernels?
I think new kind just to be ignored is not great.
libbpf can use any existing kind to sanitize and give it some
convention defined name to separate from normal types of the same kind.
We do this for bpf insn with call #magic_helper_number.

Flags sound nice, but libbpf would still need to sanitize for old
kernels. If it needs to sanitize for very old vs recently old things
get more complex.

Re: [PATCH v8 bpf-next 01/10] btf: add kind layout encoding to UAPI

[Alan Maguire]

On 20/12/2025 00:05, Alexei Starovoitov wrote:
> On Fri, Dec 19, 2025 at 8:19 AM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
>>
>> On Fri, Dec 19, 2025 at 10:14 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>>>
>>> On 19/12/2025 17:53, Andrii Nakryiko wrote:
>>>> On Fri, Dec 19, 2025 at 5:15 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>>>>>
>>>>> On 16/12/2025 19:23, Andrii Nakryiko wrote:
>>>>>> On Mon, Dec 15, 2025 at 1:18 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>>>>>>>
>>>>>>> BTF kind layouts provide information to parse BTF kinds. By separating
>>>>>>> parsing BTF from using all the information it provides, we allow BTF
>>>>>>> to encode new features even if they cannot be used by readers. This
>>>>>>> will be helpful in particular for cases where older tools are used
>>>>>>> to parse newer BTF with kinds the older tools do not recognize;
>>>>>>> the BTF can still be parsed in such cases using kind layout.
>>>>>>>
>>>>>>> The intent is to support encoding of kind layouts optionally so that
>>>>>>> tools like pahole can add this information. For each kind, we record
>>>>>>>
>>>>>>> - length of singular element following struct btf_type
>>>>>>> - length of each of the btf_vlen() elements following
>>>>>>>
>>>>>>> The ideas here were discussed at [1], [2]; hence
>>>>>>>
>>>>>>> Suggested-by: Andrii Nakryiko <andrii@kernel.org>
>>>>>>> Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
>>>>>>>
>>>>>>> [1] https://lore.kernel.org/bpf/CAEf4BzYjWHRdNNw4B=eOXOs_ONrDwrgX4bn=Nuc1g8JPFC34MA@mail.gmail.com/
>>>>>>> [2] https://lore.kernel.org/bpf/20230531201936.1992188-1-alan.maguire@oracle.com/
>>>>>>> ---
>>>>>>>  include/uapi/linux/btf.h       | 11 +++++++++++
>>>>>>>  tools/include/uapi/linux/btf.h | 11 +++++++++++
>>>>>>>  2 files changed, 22 insertions(+)
>>>>>>>
>>>>>>> diff --git a/include/uapi/linux/btf.h b/include/uapi/linux/btf.h
>>>>>>> index 266d4ffa6c07..c1854a1c7b38 100644
>>>>>>> --- a/include/uapi/linux/btf.h
>>>>>>> +++ b/include/uapi/linux/btf.h
>>>>>>> @@ -8,6 +8,15 @@
>>>>>>>  #define BTF_MAGIC      0xeB9F
>>>>>>>  #define BTF_VERSION    1
>>>>>>>
>>>>>>> +/*
>>>>>>> + * kind layout section consists of a struct btf_kind_layout for each known
>>>>>>> + * kind at BTF encoding time.
>>>>>>> + */
>>>>>>> +struct btf_kind_layout {
>>>>>>> +       __u8 info_sz;           /* size of singular element after btf_type */
>>>>>>> +       __u8 elem_sz;           /* size of each of btf_vlen(t) elements */
>>>>>>
>>>>>> So Eduard pointed out that at some point we discussed having a name of
>>>>>> a kind (i.e., "struct", "typedef", etc). By now I have no recollection
>>>>>> what were the arguments, do you remember? I'm not sure how I feel now
>>>>>> about having extra 4 bytes per kind, but that's not really a lot of
>>>>>> data (20*4 = 80 bytes added), so might as well add it, I suppose?
>>>>>>
>>>>>
>>>>> Yeah we went back and forth on that; I think it's on balance worthwhile
>>>>> to be honest; tools can be a bit more expressive about what's missing.
>>>>>
>>>>>> I think we were also discussing having flags per kind to designate
>>>>>> some extra semantics, where applicable. Again, don't remember
>>>>>> arguments for or against, but one case where I think this would be
>>>>>> very beneficial is when we add something like type_tag, which is
>>>>>> inevitably used from "normal" struct and will be almost inevitable in
>>>>>> normal vmlinux BTF. Think about it, we have some field which will be
>>>>>> CONST -> PTR -> TYPE_TAG -> STRUCT. That TYPE_TAG shouldn't just
>>>>>> totally break (old) bpftool's dump, as it really can be easily ignored
>>>>>> **if we know TYPE_TAG can be ignored and it is just a reference
>>>>>> type**. That reference type means that there is another type pointed
>>>>>> to using struct btf_type::type field (instead of that field being a
>>>>>> size).
>>>>>>
>>>>>> So I think it would be nice to encode this as a flag that says a) kind
>>>>>> can be ignored without compromising type integrity (i.e., memory
>>>>>> layout is preserved) which will be true for all kinds of modifier
>>>>>> kinds (const/volatile/restrict/type_tag, even for typedef that should
>>>>>> be true) and b) kind is reference type, so struct btf_type::type is a
>>>>>> "pointer" to a valid other underlying type.
>>>>>>
>>>>>> Thoughts?
>>>>>>
>>>>>
>>>>> Again we did go back and forth here but to me there's much more value in
>>>>> being both able to parse _and_ sanitize BTF, at least for the simple cases.
>>>>> What we can include are as you say types in the type graph that are optional
>>>>> reference kinds (like type tag), and kinds that are not implicated in the
>>>>> known type graph like the location stuff (it only points _to_ known kinds,
>>>>> no known kinds will point to location data). So any case where known
>>>>> types + optional ref types constitute the type graph we are good.
>>>>> Anything more complex than these would involve having to represent the
>>>>> layout of type references within unknown kinds (kind of like what we do for
>>>>> field iteration) which seems a bit much.
>>>>>
>>>>> Now one thing that we might want to introduce here is a sanitization-friendly
>>>>> kind, either re-using BTF_KIND_UNKN or adding a new vlen-supporting kind
>>>>> which can be used to overwrite kinds we don't want in the sanitized output.
>>>>> We need this to preserve the type ids for the kernel BTF we sanitize.
>>>>> I get that it seems weird to add a new incompatibility to handle incompatibility,
>>>>> but the sooner we do it the better I guess. The reason I suggest it now is we'd
>>>>> potentially need some more complex sanitization for the location stuff for
>>>>> cases like large location sections, and it might be cleaner to have a special
>>>>> "ignore this it's just sanitization info" kind, especially for cases like
>>>>> BTF C dump.
>>>>
>>>> So you mean you'd like some "dummy" BTF kind with 4-byte-per-vlen so
>>>> we can "overwrite" any possible unknown BTF kind?.. As you said,
>>>> though, this would only work for new kernels, so that's sad... I don't
>>>> know, I don't hate the idea, but curious what others think.
>>>>
>>>> Alternatively, we can just try to never add kinds where the vlen
>>>> element is not a multiple of 8 or 12. We can then use ENUM
>>>> (8-bytes-per-vlen) or ENUM64 (12-bytes-per-vlen) to paper over unknown
>>>> types. FUNC_PROTO (8-bytes-per-vlen) and DATASEC (12-bytes-per-vlen)
>>>> are other options. We just don't have 4-bytes-per-vlen for the most
>>>> universal "filler", unfortunately.
>>>>
>>>> The advantage of the latter is full backwards compatibility with old kernels.
>>>>
>>>
>>> True. And I guess during sanitization we can just handle intermediate
>>> types in a type graph by adjusting type ids to skip over them, so we
>>> likely have everything we need already. Funnily enough the BTF location
>>> stuff will give us a vlen-specified 4 byte object (specifying the
>>> location parameters associated with an inline), so that will help in
>>> the future for cases where it is recognized but other kinds are not.
>>
>> So coming back to flags? Let's do two flags: "safe modifier-like
>> reference kind" (for type_tag-like things where they can be dropped
>> from the chain of types) and "safe to ignore non-structural type" that
>> can't be part of any struct/union and are more like decl_tag where
>> they only reference other types, but can be dropped/replaced with
>> something? And if kind doesn't have either of those, we won't attempt
>> to sanitize (and hopefully we won't even have kinds like that, but if
>> necessary we can add more flags with some other "safe" semantics, if
>> necessary?)
> 
> Hold on. I'm missing how libbpf will sanitize things for older kernels?

The sanitization we can get from layout info is for handling a kernel built with
newer kernel/module BTF. The userspace tooling (libbpf and others) does not fully
understand it due to the presence of new kinds. In such a case layout data gives us 
info to parse it by providing info on kind layout, and libbpf can sanitize it 
to be usable for some cases (where the type graph is not fatally compromised
by the lack of a kind). This will always be somewhat limited, but it
does provide more usability than we have today.
 
Alan

Re: [PATCH v8 bpf-next 01/10] btf: add kind layout encoding to UAPI

[Alexei Starovoitov]

On Sun, Dec 21, 2025 at 10:58 PM Alan Maguire <alan.maguire@oracle.com> wrote:
>
> >
> > Hold on. I'm missing how libbpf will sanitize things for older kernels?
>
> The sanitization we can get from layout info is for handling a kernel built with
> newer kernel/module BTF. The userspace tooling (libbpf and others) does not fully
> understand it due to the presence of new kinds. In such a case layout data gives us
> info to parse it by providing info on kind layout, and libbpf can sanitize it
> to be usable for some cases (where the type graph is not fatally compromised
> by the lack of a kind). This will always be somewhat limited, but it
> does provide more usability than we have today.

I'm even more confused now. libbpf will sanitize BTF for the sake of
user space? That's not something it ever did. libbpf sanitizes BTF
only to
be loaded in the older kernel where the original BTF was
generated for a newer one. There is no reason to mangle BTF right until
the point of loading. Presence of a kind layout helps user space tooling
to print it, but that's not sanitization. The tools will just skip over.

Re: [PATCH v8 bpf-next 01/10] btf: add kind layout encoding to UAPI

[Alan Maguire]

On 22/12/2025 19:03, Alexei Starovoitov wrote:
> On Sun, Dec 21, 2025 at 10:58 PM Alan Maguire <alan.maguire@oracle.com> wrote:
>>
>>>
>>> Hold on. I'm missing how libbpf will sanitize things for older kernels?
>>
>> The sanitization we can get from layout info is for handling a kernel built with
>> newer kernel/module BTF. The userspace tooling (libbpf and others) does not fully
>> understand it due to the presence of new kinds. In such a case layout data gives us
>> info to parse it by providing info on kind layout, and libbpf can sanitize it
>> to be usable for some cases (where the type graph is not fatally compromised
>> by the lack of a kind). This will always be somewhat limited, but it
>> does provide more usability than we have today.
> 
> I'm even more confused now. libbpf will sanitize BTF for the sake of
> user space? That's not something it ever did. libbpf sanitizes BTF
> only to

Right; it's an extension of the sanitization concept from what it does today.
Today we sanitize newer _program_ BTF to ensure it is acceptable to a kernel which 
lacks specific aspects of that BTF; the goal here is to support some simple sanitization
of the newer _kernel_ BTF by libbpf to help tools (that know about kind layout but may lack
latest kind info kernel has) to make that kernel BTF usable. Both address mismatches between 
kernel BTF version and userspace. The sanitization available in this case is quite limited,
but it would work for cases like BTF location info where it's optional and doesn't get
entangled in the type graph. We could call it something else if it would help distinguish
the concepts, but it is a similar sort of activity.


> be loaded in the older kernel where the original BTF was
> generated for a newer one. There is no reason to mangle BTF right until
> the point of loading. Presence of a kind layout helps user space tooling
> to print it, but that's not sanitization. The tools will just skip over.

With the help of flags, we can do a bit more than just have bpftool print types; we can
also support generation of a vmlinux.h in some cases, support some fentry tracing etc.
Anything that requires a close type match or relies on the newer BTF features we do not
support will not work of course, but it still salvages some usability when kernel BTF is
newer.

Re: [PATCH v8 bpf-next 01/10] btf: add kind layout encoding to UAPI

[Andrii Nakryiko]

On Tue, Dec 23, 2025 at 3:09 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>
> On 22/12/2025 19:03, Alexei Starovoitov wrote:
> > On Sun, Dec 21, 2025 at 10:58 PM Alan Maguire <alan.maguire@oracle.com> wrote:
> >>
> >>>
> >>> Hold on. I'm missing how libbpf will sanitize things for older kernels?
> >>
> >> The sanitization we can get from layout info is for handling a kernel built with
> >> newer kernel/module BTF. The userspace tooling (libbpf and others) does not fully
> >> understand it due to the presence of new kinds. In such a case layout data gives us
> >> info to parse it by providing info on kind layout, and libbpf can sanitize it
> >> to be usable for some cases (where the type graph is not fatally compromised
> >> by the lack of a kind). This will always be somewhat limited, but it
> >> does provide more usability than we have today.
> >
> > I'm even more confused now. libbpf will sanitize BTF for the sake of
> > user space? That's not something it ever did. libbpf sanitizes BTF
> > only to
>
> Right; it's an extension of the sanitization concept from what it does today.
> Today we sanitize newer _program_ BTF to ensure it is acceptable to a kernel which
> lacks specific aspects of that BTF; the goal here is to support some simple sanitization
> of the newer _kernel_ BTF by libbpf to help tools (that know about kind layout but may lack
> latest kind info kernel has) to make that kernel BTF usable.

Wait, is that really a goal? I get why Alexei is confused now :)

I think we should stick to libbpf sanitizing only BPF program's BTFs
for the sake of loading it into the kernel. If some user space tool is
trying to work with kernel BTF that has BTF features that tool doesn't
support, then we only have two reasonable options: a) tool just fails
to process that BTF altogether or b) the tool is smart enough to
utilize BTF layout information to know which BTF types it can safely
skip (that's where those flags I argue for would be useful). In both
cases libbpf's btf__parse() will succeed because libbpf can utilize
layout info to construct a lookup table for btf__type_by_id(). And
libbpf doesn't need to do anything beyond that, IMO.

We'll teach bpftool to dump as much of BTF as possible (I mean
`bpftool btf dump file`), so it's possible to get an idea of what part
of BTF is not supported and show those that we know about. We could
teach btf_dump to ignore those types that are "safe modifier-like
reference kind" (as marked with that flag I proposed earlier), so that
`format c` works as well (though I wouldn't recommend using such
output as a proper vmlinux.h, users should update bpftool ASAP for
such use cases).

As far as the kernel is concerned, BTF layout is not used and should
not be used or trusted (it can be "spoofed" by the user). It can
validate it for sanity, but that's pretty much it. Other than that, if
the kernel doesn't *completely* understand every single piece of BTF,
it should reject it (and that's also why libbpf should sanitize BPF
object's BTF, of course).

> Both address mismatches between
> kernel BTF version and userspace. The sanitization available in this case is quite limited,
> but it would work for cases like BTF location info where it's optional and doesn't get
> entangled in the type graph. We could call it something else if it would help distinguish
> the concepts, but it is a similar sort of activity.
>
>
> > be loaded in the older kernel where the original BTF was
> > generated for a newer one. There is no reason to mangle BTF right until
> > the point of loading. Presence of a kind layout helps user space tooling
> > to print it, but that's not sanitization. The tools will just skip over.
>
> With the help of flags, we can do a bit more than just have bpftool print types; we can
> also support generation of a vmlinux.h in some cases, support some fentry tracing etc.
> Anything that requires a close type match or relies on the newer BTF features we do not
> support will not work of course, but it still salvages some usability when kernel BTF is
> newer.

Re: [PATCH v8 bpf-next 01/10] btf: add kind layout encoding to UAPI

[Alexei Starovoitov]

On Mon, Jan 5, 2026 at 4:11 PM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Tue, Dec 23, 2025 at 3:09 AM Alan Maguire <alan.maguire@oracle.com> wrote:
> >
> > On 22/12/2025 19:03, Alexei Starovoitov wrote:
> > > On Sun, Dec 21, 2025 at 10:58 PM Alan Maguire <alan.maguire@oracle.com> wrote:
> > >>
> > >>>
> > >>> Hold on. I'm missing how libbpf will sanitize things for older kernels?
> > >>
> > >> The sanitization we can get from layout info is for handling a kernel built with
> > >> newer kernel/module BTF. The userspace tooling (libbpf and others) does not fully
> > >> understand it due to the presence of new kinds. In such a case layout data gives us
> > >> info to parse it by providing info on kind layout, and libbpf can sanitize it
> > >> to be usable for some cases (where the type graph is not fatally compromised
> > >> by the lack of a kind). This will always be somewhat limited, but it
> > >> does provide more usability than we have today.
> > >
> > > I'm even more confused now. libbpf will sanitize BTF for the sake of
> > > user space? That's not something it ever did. libbpf sanitizes BTF
> > > only to
> >
> > Right; it's an extension of the sanitization concept from what it does today.
> > Today we sanitize newer _program_ BTF to ensure it is acceptable to a kernel which
> > lacks specific aspects of that BTF; the goal here is to support some simple sanitization
> > of the newer _kernel_ BTF by libbpf to help tools (that know about kind layout but may lack
> > latest kind info kernel has) to make that kernel BTF usable.
>
> Wait, is that really a goal? I get why Alexei is confused now :)
>
> I think we should stick to libbpf sanitizing only BPF program's BTFs
> for the sake of loading it into the kernel. If some user space tool is
> trying to work with kernel BTF that has BTF features that tool doesn't
> support, then we only have two reasonable options: a) tool just fails
> to process that BTF altogether or b) the tool is smart enough to
> utilize BTF layout information to know which BTF types it can safely
> skip (that's where those flags I argue for would be useful). In both
> cases libbpf's btf__parse() will succeed because libbpf can utilize
> layout info to construct a lookup table for btf__type_by_id(). And
> libbpf doesn't need to do anything beyond that, IMO.
>
> We'll teach bpftool to dump as much of BTF as possible (I mean
> `bpftool btf dump file`), so it's possible to get an idea of what part
> of BTF is not supported and show those that we know about. We could
> teach btf_dump to ignore those types that are "safe modifier-like
> reference kind" (as marked with that flag I proposed earlier), so that
> `format c` works as well (though I wouldn't recommend using such
> output as a proper vmlinux.h, users should update bpftool ASAP for
> such use cases).
>
> As far as the kernel is concerned, BTF layout is not used and should
> not be used or trusted (it can be "spoofed" by the user). It can
> validate it for sanity, but that's pretty much it. Other than that, if
> the kernel doesn't *completely* understand every single piece of BTF,
> it should reject it (and that's also why libbpf should sanitize BPF
> object's BTF, of course).

+1 to all of the above, except ok-to-skip flag, since I feel
it will cause more bike sheding and arguing whether a particular
new addition to BTF is skippable or not. Like upcoming location info.
Is it skippable? kinda. Or, say, we decide to add vector types to BTF.
Skippable? One might argue that they are while they are mandatory
for some other use case.
Looking at it differently, if the kernel is old and cannot understand that
BTF feature the libbpf has to sanitize it no matter skippable or not.
While from btf__parse() pov it also doesn't matter.
btf_new()->btf_parse_hdr() will remember kind layout,
and btf_parse_type_sec() can construct the index for the whole thing
with layout info,
while btf_validate_type() has to warn about unknown kind regardless
of skippable flag. The tool (bpftool or else) needs to yell when
final vmlinux.h is incomplete. Skipping printing modifier-like decl_tag
is pretty bad for vmlinux.h. It's really not skippable (in my opinion)
though one might argue that they are.

Also let's not call btf__parse() process with layout info
as a sanitization. Sanitization is the word to describe actual replacement
of unrecognized kind with a known kind. Here btf__parse() won't be
mangling it. Who knows, maybe btf_dumper (in bpftool or else)
understands the kind, though libbpf's btf__parse() does not.

Re: [PATCH v8 bpf-next 01/10] btf: add kind layout encoding to UAPI

[Andrii Nakryiko]

On Mon, Jan 5, 2026 at 4:51 PM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Mon, Jan 5, 2026 at 4:11 PM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
> >
> > On Tue, Dec 23, 2025 at 3:09 AM Alan Maguire <alan.maguire@oracle.com> wrote:
> > >
> > > On 22/12/2025 19:03, Alexei Starovoitov wrote:
> > > > On Sun, Dec 21, 2025 at 10:58 PM Alan Maguire <alan.maguire@oracle.com> wrote:
> > > >>
> > > >>>
> > > >>> Hold on. I'm missing how libbpf will sanitize things for older kernels?
> > > >>
> > > >> The sanitization we can get from layout info is for handling a kernel built with
> > > >> newer kernel/module BTF. The userspace tooling (libbpf and others) does not fully
> > > >> understand it due to the presence of new kinds. In such a case layout data gives us
> > > >> info to parse it by providing info on kind layout, and libbpf can sanitize it
> > > >> to be usable for some cases (where the type graph is not fatally compromised
> > > >> by the lack of a kind). This will always be somewhat limited, but it
> > > >> does provide more usability than we have today.
> > > >
> > > > I'm even more confused now. libbpf will sanitize BTF for the sake of
> > > > user space? That's not something it ever did. libbpf sanitizes BTF
> > > > only to
> > >
> > > Right; it's an extension of the sanitization concept from what it does today.
> > > Today we sanitize newer _program_ BTF to ensure it is acceptable to a kernel which
> > > lacks specific aspects of that BTF; the goal here is to support some simple sanitization
> > > of the newer _kernel_ BTF by libbpf to help tools (that know about kind layout but may lack
> > > latest kind info kernel has) to make that kernel BTF usable.
> >
> > Wait, is that really a goal? I get why Alexei is confused now :)
> >
> > I think we should stick to libbpf sanitizing only BPF program's BTFs
> > for the sake of loading it into the kernel. If some user space tool is
> > trying to work with kernel BTF that has BTF features that tool doesn't
> > support, then we only have two reasonable options: a) tool just fails
> > to process that BTF altogether or b) the tool is smart enough to
> > utilize BTF layout information to know which BTF types it can safely
> > skip (that's where those flags I argue for would be useful). In both
> > cases libbpf's btf__parse() will succeed because libbpf can utilize
> > layout info to construct a lookup table for btf__type_by_id(). And
> > libbpf doesn't need to do anything beyond that, IMO.
> >
> > We'll teach bpftool to dump as much of BTF as possible (I mean
> > `bpftool btf dump file`), so it's possible to get an idea of what part
> > of BTF is not supported and show those that we know about. We could
> > teach btf_dump to ignore those types that are "safe modifier-like
> > reference kind" (as marked with that flag I proposed earlier), so that
> > `format c` works as well (though I wouldn't recommend using such
> > output as a proper vmlinux.h, users should update bpftool ASAP for
> > such use cases).
> >
> > As far as the kernel is concerned, BTF layout is not used and should
> > not be used or trusted (it can be "spoofed" by the user). It can
> > validate it for sanity, but that's pretty much it. Other than that, if
> > the kernel doesn't *completely* understand every single piece of BTF,
> > it should reject it (and that's also why libbpf should sanitize BPF
> > object's BTF, of course).
>
> +1 to all of the above, except ok-to-skip flag, since I feel
> it will cause more bike sheding and arguing whether a particular
> new addition to BTF is skippable or not. Like upcoming location info.

I was thinking about something like TYPE_TAG, where it's in the chain
of types and is unavoidable when processing STRUCT and its field.
Having a flag specifying that it's ref-like (so btf_type::type field
points to a valid type ID) would allow it to still make sense of the
entire struct and its fields, though you might be missing some
(presumably) optional and highly-specialized extra annotation.

But it's fine not to add it, just some type graphs will be completely
unprocessable using old tools. Perhaps not such a big deal.

I suspect all the newly added BTF kinds will be of "ok-to-skip" kind,
whether they are more like DECL_TAG (roots pointing to other types) or
TYPE_TAG (in the middle of type chain, being pointed to from STRUCT
fields, PROTO args, etc).

> Is it skippable? kinda. Or, say, we decide to add vector types to BTF.
> Skippable? One might argue that they are while they are mandatory
> for some other use case.
> Looking at it differently, if the kernel is old and cannot understand that
> BTF feature the libbpf has to sanitize it no matter skippable or not.
> While from btf__parse() pov it also doesn't matter.
> btf_new()->btf_parse_hdr() will remember kind layout,
> and btf_parse_type_sec() can construct the index for the whole thing
> with layout info,
> while btf_validate_type() has to warn about unknown kind regardless
> of skippable flag. The tool (bpftool or else) needs to yell when
> final vmlinux.h is incomplete. Skipping printing modifier-like decl_tag
> is pretty bad for vmlinux.h. It's really not skippable (in my opinion)
> though one might argue that they are.

Yeah, I agree about vmlinux.h. One way to enforce this would be to
have btf_dump emit something uncompilable as
"HERE_BE_DRAGONS_SKIPPED_SOMETHING"  as if it was const/volatile
modified.

But yeah, we don't want bikeshedding. It's fine.

>
> Also let's not call btf__parse() process with layout info
> as a sanitization. Sanitization is the word to describe actual replacement
> of unrecognized kind with a known kind. Here btf__parse() won't be
> mangling it. Who knows, maybe btf_dumper (in bpftool or else)
> understands the kind, though libbpf's btf__parse() does not.

Re: [PATCH v8 bpf-next 01/10] btf: add kind layout encoding to UAPI

[Alan Maguire]

On 06/01/2026 01:19, Andrii Nakryiko wrote:
> On Mon, Jan 5, 2026 at 4:51 PM Alexei Starovoitov
> <alexei.starovoitov@gmail.com> wrote:
>>
>> On Mon, Jan 5, 2026 at 4:11 PM Andrii Nakryiko
>> <andrii.nakryiko@gmail.com> wrote:
>>>
>>> On Tue, Dec 23, 2025 at 3:09 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>>>>
>>>> On 22/12/2025 19:03, Alexei Starovoitov wrote:
>>>>> On Sun, Dec 21, 2025 at 10:58 PM Alan Maguire <alan.maguire@oracle.com> wrote:
>>>>>>
>>>>>>>
>>>>>>> Hold on. I'm missing how libbpf will sanitize things for older kernels?
>>>>>>
>>>>>> The sanitization we can get from layout info is for handling a kernel built with
>>>>>> newer kernel/module BTF. The userspace tooling (libbpf and others) does not fully
>>>>>> understand it due to the presence of new kinds. In such a case layout data gives us
>>>>>> info to parse it by providing info on kind layout, and libbpf can sanitize it
>>>>>> to be usable for some cases (where the type graph is not fatally compromised
>>>>>> by the lack of a kind). This will always be somewhat limited, but it
>>>>>> does provide more usability than we have today.
>>>>>
>>>>> I'm even more confused now. libbpf will sanitize BTF for the sake of
>>>>> user space? That's not something it ever did. libbpf sanitizes BTF
>>>>> only to
>>>>
>>>> Right; it's an extension of the sanitization concept from what it does today.
>>>> Today we sanitize newer _program_ BTF to ensure it is acceptable to a kernel which
>>>> lacks specific aspects of that BTF; the goal here is to support some simple sanitization
>>>> of the newer _kernel_ BTF by libbpf to help tools (that know about kind layout but may lack
>>>> latest kind info kernel has) to make that kernel BTF usable.
>>>
>>> Wait, is that really a goal? I get why Alexei is confused now :)
>>>
>>> I think we should stick to libbpf sanitizing only BPF program's BTFs
>>> for the sake of loading it into the kernel. If some user space tool is
>>> trying to work with kernel BTF that has BTF features that tool doesn't
>>> support, then we only have two reasonable options: a) tool just fails
>>> to process that BTF altogether or b) the tool is smart enough to
>>> utilize BTF layout information to know which BTF types it can safely
>>> skip (that's where those flags I argue for would be useful). In both
>>> cases libbpf's btf__parse() will succeed because libbpf can utilize
>>> layout info to construct a lookup table for btf__type_by_id(). And
>>> libbpf doesn't need to do anything beyond that, IMO.
>>>
>>> We'll teach bpftool to dump as much of BTF as possible (I mean
>>> `bpftool btf dump file`), so it's possible to get an idea of what part
>>> of BTF is not supported and show those that we know about. We could
>>> teach btf_dump to ignore those types that are "safe modifier-like
>>> reference kind" (as marked with that flag I proposed earlier), so that
>>> `format c` works as well (though I wouldn't recommend using such
>>> output as a proper vmlinux.h, users should update bpftool ASAP for
>>> such use cases).
>>>
>>> As far as the kernel is concerned, BTF layout is not used and should
>>> not be used or trusted (it can be "spoofed" by the user). It can
>>> validate it for sanity, but that's pretty much it. Other than that, if
>>> the kernel doesn't *completely* understand every single piece of BTF,
>>> it should reject it (and that's also why libbpf should sanitize BPF
>>> object's BTF, of course).
>>
>> +1 to all of the above, except ok-to-skip flag, since I feel
>> it will cause more bike sheding and arguing whether a particular
>> new addition to BTF is skippable or not. Like upcoming location info.
> 
> I was thinking about something like TYPE_TAG, where it's in the chain
> of types and is unavoidable when processing STRUCT and its field.
> Having a flag specifying that it's ref-like (so btf_type::type field
> points to a valid type ID) would allow it to still make sense of the
> entire struct and its fields, though you might be missing some
> (presumably) optional and highly-specialized extra annotation.
> 
> But it's fine not to add it, just some type graphs will be completely
> unprocessable using old tools. Perhaps not such a big deal.
> 
> I suspect all the newly added BTF kinds will be of "ok-to-skip" kind,
> whether they are more like DECL_TAG (roots pointing to other types) or
> TYPE_TAG (in the middle of type chain, being pointed to from STRUCT
> fields, PROTO args, etc).
> 
>> Is it skippable? kinda. Or, say, we decide to add vector types to BTF.
>> Skippable? One might argue that they are while they are mandatory
>> for some other use case.
>> Looking at it differently, if the kernel is old and cannot understand that
>> BTF feature the libbpf has to sanitize it no matter skippable or not.
>> While from btf__parse() pov it also doesn't matter.
>> btf_new()->btf_parse_hdr() will remember kind layout,
>> and btf_parse_type_sec() can construct the index for the whole thing
>> with layout info,
>> while btf_validate_type() has to warn about unknown kind regardless
>> of skippable flag. The tool (bpftool or else) needs to yell when
>> final vmlinux.h is incomplete. Skipping printing modifier-like decl_tag
>> is pretty bad for vmlinux.h. It's really not skippable (in my opinion)
>> though one might argue that they are.
> 
> Yeah, I agree about vmlinux.h. One way to enforce this would be to
> have btf_dump emit something uncompilable as
> "HERE_BE_DRAGONS_SKIPPED_SOMETHING"  as if it was const/volatile
> modified.
> 
> But yeah, we don't want bikeshedding. It's fine.
> 

Ok so is it best to leave out flags entirely then? If so where we
are now is to have each kind layout entry have a string name offset,
a singular element size and a vlen-specified object size. To be
conservative it might make sense to allow 16 bits for each size field,
leaving us with 64 bits per kind, 160 bytes in total for the 20 kinds.
We could cut down further by leaving out kind name strings if needed.

Re: [PATCH v8 bpf-next 01/10] btf: add kind layout encoding to UAPI

[Andrii Nakryiko]

On Thu, Jan 8, 2026 at 10:55 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>
> On 06/01/2026 01:19, Andrii Nakryiko wrote:
> > On Mon, Jan 5, 2026 at 4:51 PM Alexei Starovoitov
> > <alexei.starovoitov@gmail.com> wrote:
> >>
> >> On Mon, Jan 5, 2026 at 4:11 PM Andrii Nakryiko
> >> <andrii.nakryiko@gmail.com> wrote:
> >>>
> >>> On Tue, Dec 23, 2025 at 3:09 AM Alan Maguire <alan.maguire@oracle.com> wrote:
> >>>>
> >>>> On 22/12/2025 19:03, Alexei Starovoitov wrote:
> >>>>> On Sun, Dec 21, 2025 at 10:58 PM Alan Maguire <alan.maguire@oracle.com> wrote:
> >>>>>>
> >>>>>>>
> >>>>>>> Hold on. I'm missing how libbpf will sanitize things for older kernels?
> >>>>>>
> >>>>>> The sanitization we can get from layout info is for handling a kernel built with
> >>>>>> newer kernel/module BTF. The userspace tooling (libbpf and others) does not fully
> >>>>>> understand it due to the presence of new kinds. In such a case layout data gives us
> >>>>>> info to parse it by providing info on kind layout, and libbpf can sanitize it
> >>>>>> to be usable for some cases (where the type graph is not fatally compromised
> >>>>>> by the lack of a kind). This will always be somewhat limited, but it
> >>>>>> does provide more usability than we have today.
> >>>>>
> >>>>> I'm even more confused now. libbpf will sanitize BTF for the sake of
> >>>>> user space? That's not something it ever did. libbpf sanitizes BTF
> >>>>> only to
> >>>>
> >>>> Right; it's an extension of the sanitization concept from what it does today.
> >>>> Today we sanitize newer _program_ BTF to ensure it is acceptable to a kernel which
> >>>> lacks specific aspects of that BTF; the goal here is to support some simple sanitization
> >>>> of the newer _kernel_ BTF by libbpf to help tools (that know about kind layout but may lack
> >>>> latest kind info kernel has) to make that kernel BTF usable.
> >>>
> >>> Wait, is that really a goal? I get why Alexei is confused now :)
> >>>
> >>> I think we should stick to libbpf sanitizing only BPF program's BTFs
> >>> for the sake of loading it into the kernel. If some user space tool is
> >>> trying to work with kernel BTF that has BTF features that tool doesn't
> >>> support, then we only have two reasonable options: a) tool just fails
> >>> to process that BTF altogether or b) the tool is smart enough to
> >>> utilize BTF layout information to know which BTF types it can safely
> >>> skip (that's where those flags I argue for would be useful). In both
> >>> cases libbpf's btf__parse() will succeed because libbpf can utilize
> >>> layout info to construct a lookup table for btf__type_by_id(). And
> >>> libbpf doesn't need to do anything beyond that, IMO.
> >>>
> >>> We'll teach bpftool to dump as much of BTF as possible (I mean
> >>> `bpftool btf dump file`), so it's possible to get an idea of what part
> >>> of BTF is not supported and show those that we know about. We could
> >>> teach btf_dump to ignore those types that are "safe modifier-like
> >>> reference kind" (as marked with that flag I proposed earlier), so that
> >>> `format c` works as well (though I wouldn't recommend using such
> >>> output as a proper vmlinux.h, users should update bpftool ASAP for
> >>> such use cases).
> >>>
> >>> As far as the kernel is concerned, BTF layout is not used and should
> >>> not be used or trusted (it can be "spoofed" by the user). It can
> >>> validate it for sanity, but that's pretty much it. Other than that, if
> >>> the kernel doesn't *completely* understand every single piece of BTF,
> >>> it should reject it (and that's also why libbpf should sanitize BPF
> >>> object's BTF, of course).
> >>
> >> +1 to all of the above, except ok-to-skip flag, since I feel
> >> it will cause more bike sheding and arguing whether a particular
> >> new addition to BTF is skippable or not. Like upcoming location info.
> >
> > I was thinking about something like TYPE_TAG, where it's in the chain
> > of types and is unavoidable when processing STRUCT and its field.
> > Having a flag specifying that it's ref-like (so btf_type::type field
> > points to a valid type ID) would allow it to still make sense of the
> > entire struct and its fields, though you might be missing some
> > (presumably) optional and highly-specialized extra annotation.
> >
> > But it's fine not to add it, just some type graphs will be completely
> > unprocessable using old tools. Perhaps not such a big deal.
> >
> > I suspect all the newly added BTF kinds will be of "ok-to-skip" kind,
> > whether they are more like DECL_TAG (roots pointing to other types) or
> > TYPE_TAG (in the middle of type chain, being pointed to from STRUCT
> > fields, PROTO args, etc).
> >
> >> Is it skippable? kinda. Or, say, we decide to add vector types to BTF.
> >> Skippable? One might argue that they are while they are mandatory
> >> for some other use case.
> >> Looking at it differently, if the kernel is old and cannot understand that
> >> BTF feature the libbpf has to sanitize it no matter skippable or not.
> >> While from btf__parse() pov it also doesn't matter.
> >> btf_new()->btf_parse_hdr() will remember kind layout,
> >> and btf_parse_type_sec() can construct the index for the whole thing
> >> with layout info,
> >> while btf_validate_type() has to warn about unknown kind regardless
> >> of skippable flag. The tool (bpftool or else) needs to yell when
> >> final vmlinux.h is incomplete. Skipping printing modifier-like decl_tag
> >> is pretty bad for vmlinux.h. It's really not skippable (in my opinion)
> >> though one might argue that they are.
> >
> > Yeah, I agree about vmlinux.h. One way to enforce this would be to
> > have btf_dump emit something uncompilable as
> > "HERE_BE_DRAGONS_SKIPPED_SOMETHING"  as if it was const/volatile
> > modified.
> >
> > But yeah, we don't want bikeshedding. It's fine.
> >
>
> Ok so is it best to leave out flags entirely then? If so where we
> are now is to have each kind layout entry have a string name offset,
> a singular element size and a vlen-specified object size. To be
> conservative it might make sense to allow 16 bits for each size field,
> leaving us with 64 bits per kind, 160 bytes in total for the 20 kinds.
> We could cut down further by leaving out kind name strings if needed.

Are we sure we will *never* need flags? I'd probably stick to
single-byte sizes and have 2 bytes reserved for flags or whatever we
might need in the future?

Re: [PATCH v8 bpf-next 01/10] btf: add kind layout encoding to UAPI

[Alexei Starovoitov]

On Thu, Jan 8, 2026 at 5:24 PM Andrii Nakryiko
<andrii.nakryiko@gmail.com> wrote:
>
> On Thu, Jan 8, 2026 at 10:55 AM Alan Maguire <alan.maguire@oracle.com> wrote:
> >
> > On 06/01/2026 01:19, Andrii Nakryiko wrote:
> > > On Mon, Jan 5, 2026 at 4:51 PM Alexei Starovoitov
> > > <alexei.starovoitov@gmail.com> wrote:
> > >>
> > >> On Mon, Jan 5, 2026 at 4:11 PM Andrii Nakryiko
> > >> <andrii.nakryiko@gmail.com> wrote:
> > >>>
> > >>> On Tue, Dec 23, 2025 at 3:09 AM Alan Maguire <alan.maguire@oracle.com> wrote:
> > >>>>
> > >>>> On 22/12/2025 19:03, Alexei Starovoitov wrote:
> > >>>>> On Sun, Dec 21, 2025 at 10:58 PM Alan Maguire <alan.maguire@oracle.com> wrote:
> > >>>>>>
> > >>>>>>>
> > >>>>>>> Hold on. I'm missing how libbpf will sanitize things for older kernels?
> > >>>>>>
> > >>>>>> The sanitization we can get from layout info is for handling a kernel built with
> > >>>>>> newer kernel/module BTF. The userspace tooling (libbpf and others) does not fully
> > >>>>>> understand it due to the presence of new kinds. In such a case layout data gives us
> > >>>>>> info to parse it by providing info on kind layout, and libbpf can sanitize it
> > >>>>>> to be usable for some cases (where the type graph is not fatally compromised
> > >>>>>> by the lack of a kind). This will always be somewhat limited, but it
> > >>>>>> does provide more usability than we have today.
> > >>>>>
> > >>>>> I'm even more confused now. libbpf will sanitize BTF for the sake of
> > >>>>> user space? That's not something it ever did. libbpf sanitizes BTF
> > >>>>> only to
> > >>>>
> > >>>> Right; it's an extension of the sanitization concept from what it does today.
> > >>>> Today we sanitize newer _program_ BTF to ensure it is acceptable to a kernel which
> > >>>> lacks specific aspects of that BTF; the goal here is to support some simple sanitization
> > >>>> of the newer _kernel_ BTF by libbpf to help tools (that know about kind layout but may lack
> > >>>> latest kind info kernel has) to make that kernel BTF usable.
> > >>>
> > >>> Wait, is that really a goal? I get why Alexei is confused now :)
> > >>>
> > >>> I think we should stick to libbpf sanitizing only BPF program's BTFs
> > >>> for the sake of loading it into the kernel. If some user space tool is
> > >>> trying to work with kernel BTF that has BTF features that tool doesn't
> > >>> support, then we only have two reasonable options: a) tool just fails
> > >>> to process that BTF altogether or b) the tool is smart enough to
> > >>> utilize BTF layout information to know which BTF types it can safely
> > >>> skip (that's where those flags I argue for would be useful). In both
> > >>> cases libbpf's btf__parse() will succeed because libbpf can utilize
> > >>> layout info to construct a lookup table for btf__type_by_id(). And
> > >>> libbpf doesn't need to do anything beyond that, IMO.
> > >>>
> > >>> We'll teach bpftool to dump as much of BTF as possible (I mean
> > >>> `bpftool btf dump file`), so it's possible to get an idea of what part
> > >>> of BTF is not supported and show those that we know about. We could
> > >>> teach btf_dump to ignore those types that are "safe modifier-like
> > >>> reference kind" (as marked with that flag I proposed earlier), so that
> > >>> `format c` works as well (though I wouldn't recommend using such
> > >>> output as a proper vmlinux.h, users should update bpftool ASAP for
> > >>> such use cases).
> > >>>
> > >>> As far as the kernel is concerned, BTF layout is not used and should
> > >>> not be used or trusted (it can be "spoofed" by the user). It can
> > >>> validate it for sanity, but that's pretty much it. Other than that, if
> > >>> the kernel doesn't *completely* understand every single piece of BTF,
> > >>> it should reject it (and that's also why libbpf should sanitize BPF
> > >>> object's BTF, of course).
> > >>
> > >> +1 to all of the above, except ok-to-skip flag, since I feel
> > >> it will cause more bike sheding and arguing whether a particular
> > >> new addition to BTF is skippable or not. Like upcoming location info.
> > >
> > > I was thinking about something like TYPE_TAG, where it's in the chain
> > > of types and is unavoidable when processing STRUCT and its field.
> > > Having a flag specifying that it's ref-like (so btf_type::type field
> > > points to a valid type ID) would allow it to still make sense of the
> > > entire struct and its fields, though you might be missing some
> > > (presumably) optional and highly-specialized extra annotation.
> > >
> > > But it's fine not to add it, just some type graphs will be completely
> > > unprocessable using old tools. Perhaps not such a big deal.
> > >
> > > I suspect all the newly added BTF kinds will be of "ok-to-skip" kind,
> > > whether they are more like DECL_TAG (roots pointing to other types) or
> > > TYPE_TAG (in the middle of type chain, being pointed to from STRUCT
> > > fields, PROTO args, etc).
> > >
> > >> Is it skippable? kinda. Or, say, we decide to add vector types to BTF.
> > >> Skippable? One might argue that they are while they are mandatory
> > >> for some other use case.
> > >> Looking at it differently, if the kernel is old and cannot understand that
> > >> BTF feature the libbpf has to sanitize it no matter skippable or not.
> > >> While from btf__parse() pov it also doesn't matter.
> > >> btf_new()->btf_parse_hdr() will remember kind layout,
> > >> and btf_parse_type_sec() can construct the index for the whole thing
> > >> with layout info,
> > >> while btf_validate_type() has to warn about unknown kind regardless
> > >> of skippable flag. The tool (bpftool or else) needs to yell when
> > >> final vmlinux.h is incomplete. Skipping printing modifier-like decl_tag
> > >> is pretty bad for vmlinux.h. It's really not skippable (in my opinion)
> > >> though one might argue that they are.
> > >
> > > Yeah, I agree about vmlinux.h. One way to enforce this would be to
> > > have btf_dump emit something uncompilable as
> > > "HERE_BE_DRAGONS_SKIPPED_SOMETHING"  as if it was const/volatile
> > > modified.
> > >
> > > But yeah, we don't want bikeshedding. It's fine.
> > >
> >
> > Ok so is it best to leave out flags entirely then? If so where we
> > are now is to have each kind layout entry have a string name offset,
> > a singular element size and a vlen-specified object size. To be
> > conservative it might make sense to allow 16 bits for each size field,
> > leaving us with 64 bits per kind, 160 bytes in total for the 20 kinds.
> > We could cut down further by leaving out kind name strings if needed.
>
> Are we sure we will *never* need flags? I'd probably stick to
> single-byte sizes and have 2 bytes reserved for flags or whatever we
> might need in the future?

Just to clarify what I was saying.
I think it's a good thing to have flags space and reserve it.
I just struggle to see the value of 'ok-to-skip' flag.

So 2 bytes of reserved space for flags makes sense to me.

Re: [PATCH v8 bpf-next 01/10] btf: add kind layout encoding to UAPI

[Alan Maguire]

On 09/01/2026 01:40, Alexei Starovoitov wrote:
> On Thu, Jan 8, 2026 at 5:24 PM Andrii Nakryiko
> <andrii.nakryiko@gmail.com> wrote:
>>
>> On Thu, Jan 8, 2026 at 10:55 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>>>
>>> On 06/01/2026 01:19, Andrii Nakryiko wrote:
>>>> On Mon, Jan 5, 2026 at 4:51 PM Alexei Starovoitov
>>>> <alexei.starovoitov@gmail.com> wrote:
>>>>>
>>>>> On Mon, Jan 5, 2026 at 4:11 PM Andrii Nakryiko
>>>>> <andrii.nakryiko@gmail.com> wrote:
>>>>>>
>>>>>> On Tue, Dec 23, 2025 at 3:09 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>>>>>>>
>>>>>>> On 22/12/2025 19:03, Alexei Starovoitov wrote:
>>>>>>>> On Sun, Dec 21, 2025 at 10:58 PM Alan Maguire <alan.maguire@oracle.com> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Hold on. I'm missing how libbpf will sanitize things for older kernels?
>>>>>>>>>
>>>>>>>>> The sanitization we can get from layout info is for handling a kernel built with
>>>>>>>>> newer kernel/module BTF. The userspace tooling (libbpf and others) does not fully
>>>>>>>>> understand it due to the presence of new kinds. In such a case layout data gives us
>>>>>>>>> info to parse it by providing info on kind layout, and libbpf can sanitize it
>>>>>>>>> to be usable for some cases (where the type graph is not fatally compromised
>>>>>>>>> by the lack of a kind). This will always be somewhat limited, but it
>>>>>>>>> does provide more usability than we have today.
>>>>>>>>
>>>>>>>> I'm even more confused now. libbpf will sanitize BTF for the sake of
>>>>>>>> user space? That's not something it ever did. libbpf sanitizes BTF
>>>>>>>> only to
>>>>>>>
>>>>>>> Right; it's an extension of the sanitization concept from what it does today.
>>>>>>> Today we sanitize newer _program_ BTF to ensure it is acceptable to a kernel which
>>>>>>> lacks specific aspects of that BTF; the goal here is to support some simple sanitization
>>>>>>> of the newer _kernel_ BTF by libbpf to help tools (that know about kind layout but may lack
>>>>>>> latest kind info kernel has) to make that kernel BTF usable.
>>>>>>
>>>>>> Wait, is that really a goal? I get why Alexei is confused now :)
>>>>>>
>>>>>> I think we should stick to libbpf sanitizing only BPF program's BTFs
>>>>>> for the sake of loading it into the kernel. If some user space tool is
>>>>>> trying to work with kernel BTF that has BTF features that tool doesn't
>>>>>> support, then we only have two reasonable options: a) tool just fails
>>>>>> to process that BTF altogether or b) the tool is smart enough to
>>>>>> utilize BTF layout information to know which BTF types it can safely
>>>>>> skip (that's where those flags I argue for would be useful). In both
>>>>>> cases libbpf's btf__parse() will succeed because libbpf can utilize
>>>>>> layout info to construct a lookup table for btf__type_by_id(). And
>>>>>> libbpf doesn't need to do anything beyond that, IMO.
>>>>>>
>>>>>> We'll teach bpftool to dump as much of BTF as possible (I mean
>>>>>> `bpftool btf dump file`), so it's possible to get an idea of what part
>>>>>> of BTF is not supported and show those that we know about. We could
>>>>>> teach btf_dump to ignore those types that are "safe modifier-like
>>>>>> reference kind" (as marked with that flag I proposed earlier), so that
>>>>>> `format c` works as well (though I wouldn't recommend using such
>>>>>> output as a proper vmlinux.h, users should update bpftool ASAP for
>>>>>> such use cases).
>>>>>>
>>>>>> As far as the kernel is concerned, BTF layout is not used and should
>>>>>> not be used or trusted (it can be "spoofed" by the user). It can
>>>>>> validate it for sanity, but that's pretty much it. Other than that, if
>>>>>> the kernel doesn't *completely* understand every single piece of BTF,
>>>>>> it should reject it (and that's also why libbpf should sanitize BPF
>>>>>> object's BTF, of course).
>>>>>
>>>>> +1 to all of the above, except ok-to-skip flag, since I feel
>>>>> it will cause more bike sheding and arguing whether a particular
>>>>> new addition to BTF is skippable or not. Like upcoming location info.
>>>>
>>>> I was thinking about something like TYPE_TAG, where it's in the chain
>>>> of types and is unavoidable when processing STRUCT and its field.
>>>> Having a flag specifying that it's ref-like (so btf_type::type field
>>>> points to a valid type ID) would allow it to still make sense of the
>>>> entire struct and its fields, though you might be missing some
>>>> (presumably) optional and highly-specialized extra annotation.
>>>>
>>>> But it's fine not to add it, just some type graphs will be completely
>>>> unprocessable using old tools. Perhaps not such a big deal.
>>>>
>>>> I suspect all the newly added BTF kinds will be of "ok-to-skip" kind,
>>>> whether they are more like DECL_TAG (roots pointing to other types) or
>>>> TYPE_TAG (in the middle of type chain, being pointed to from STRUCT
>>>> fields, PROTO args, etc).
>>>>
>>>>> Is it skippable? kinda. Or, say, we decide to add vector types to BTF.
>>>>> Skippable? One might argue that they are while they are mandatory
>>>>> for some other use case.
>>>>> Looking at it differently, if the kernel is old and cannot understand that
>>>>> BTF feature the libbpf has to sanitize it no matter skippable or not.
>>>>> While from btf__parse() pov it also doesn't matter.
>>>>> btf_new()->btf_parse_hdr() will remember kind layout,
>>>>> and btf_parse_type_sec() can construct the index for the whole thing
>>>>> with layout info,
>>>>> while btf_validate_type() has to warn about unknown kind regardless
>>>>> of skippable flag. The tool (bpftool or else) needs to yell when
>>>>> final vmlinux.h is incomplete. Skipping printing modifier-like decl_tag
>>>>> is pretty bad for vmlinux.h. It's really not skippable (in my opinion)
>>>>> though one might argue that they are.
>>>>
>>>> Yeah, I agree about vmlinux.h. One way to enforce this would be to
>>>> have btf_dump emit something uncompilable as
>>>> "HERE_BE_DRAGONS_SKIPPED_SOMETHING"  as if it was const/volatile
>>>> modified.
>>>>
>>>> But yeah, we don't want bikeshedding. It's fine.
>>>>
>>>
>>> Ok so is it best to leave out flags entirely then? If so where we
>>> are now is to have each kind layout entry have a string name offset,
>>> a singular element size and a vlen-specified object size. To be
>>> conservative it might make sense to allow 16 bits for each size field,
>>> leaving us with 64 bits per kind, 160 bytes in total for the 20 kinds.
>>> We could cut down further by leaving out kind name strings if needed.
>>
>> Are we sure we will *never* need flags? I'd probably stick to
>> single-byte sizes and have 2 bytes reserved for flags or whatever we
>> might need in the future?
> 
> Just to clarify what I was saying.
> I think it's a good thing to have flags space and reserve it.
> I just struggle to see the value of 'ok-to-skip' flag.
> 
> So 2 bytes of reserved space for flags makes sense to me.

Ok sounds good; I think there is still value in having the single flag
that tells us that the type/size field in struct btf_type refers
to a type though, right?

Re: [PATCH v8 bpf-next 01/10] btf: add kind layout encoding to UAPI

[Alexei Starovoitov]

On Fri, Jan 9, 2026 at 5:21 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>
> On 09/01/2026 01:40, Alexei Starovoitov wrote:
> > On Thu, Jan 8, 2026 at 5:24 PM Andrii Nakryiko
> > <andrii.nakryiko@gmail.com> wrote:
> >>
> >> On Thu, Jan 8, 2026 at 10:55 AM Alan Maguire <alan.maguire@oracle.com> wrote:
> >>>
> >>> On 06/01/2026 01:19, Andrii Nakryiko wrote:
> >>>> On Mon, Jan 5, 2026 at 4:51 PM Alexei Starovoitov
> >>>> <alexei.starovoitov@gmail.com> wrote:
> >>>>>
> >>>>> On Mon, Jan 5, 2026 at 4:11 PM Andrii Nakryiko
> >>>>> <andrii.nakryiko@gmail.com> wrote:
> >>>>>>
> >>>>>> On Tue, Dec 23, 2025 at 3:09 AM Alan Maguire <alan.maguire@oracle.com> wrote:
> >>>>>>>
> >>>>>>> On 22/12/2025 19:03, Alexei Starovoitov wrote:
> >>>>>>>> On Sun, Dec 21, 2025 at 10:58 PM Alan Maguire <alan.maguire@oracle.com> wrote:
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Hold on. I'm missing how libbpf will sanitize things for older kernels?
> >>>>>>>>>
> >>>>>>>>> The sanitization we can get from layout info is for handling a kernel built with
> >>>>>>>>> newer kernel/module BTF. The userspace tooling (libbpf and others) does not fully
> >>>>>>>>> understand it due to the presence of new kinds. In such a case layout data gives us
> >>>>>>>>> info to parse it by providing info on kind layout, and libbpf can sanitize it
> >>>>>>>>> to be usable for some cases (where the type graph is not fatally compromised
> >>>>>>>>> by the lack of a kind). This will always be somewhat limited, but it
> >>>>>>>>> does provide more usability than we have today.
> >>>>>>>>
> >>>>>>>> I'm even more confused now. libbpf will sanitize BTF for the sake of
> >>>>>>>> user space? That's not something it ever did. libbpf sanitizes BTF
> >>>>>>>> only to
> >>>>>>>
> >>>>>>> Right; it's an extension of the sanitization concept from what it does today.
> >>>>>>> Today we sanitize newer _program_ BTF to ensure it is acceptable to a kernel which
> >>>>>>> lacks specific aspects of that BTF; the goal here is to support some simple sanitization
> >>>>>>> of the newer _kernel_ BTF by libbpf to help tools (that know about kind layout but may lack
> >>>>>>> latest kind info kernel has) to make that kernel BTF usable.
> >>>>>>
> >>>>>> Wait, is that really a goal? I get why Alexei is confused now :)
> >>>>>>
> >>>>>> I think we should stick to libbpf sanitizing only BPF program's BTFs
> >>>>>> for the sake of loading it into the kernel. If some user space tool is
> >>>>>> trying to work with kernel BTF that has BTF features that tool doesn't
> >>>>>> support, then we only have two reasonable options: a) tool just fails
> >>>>>> to process that BTF altogether or b) the tool is smart enough to
> >>>>>> utilize BTF layout information to know which BTF types it can safely
> >>>>>> skip (that's where those flags I argue for would be useful). In both
> >>>>>> cases libbpf's btf__parse() will succeed because libbpf can utilize
> >>>>>> layout info to construct a lookup table for btf__type_by_id(). And
> >>>>>> libbpf doesn't need to do anything beyond that, IMO.
> >>>>>>
> >>>>>> We'll teach bpftool to dump as much of BTF as possible (I mean
> >>>>>> `bpftool btf dump file`), so it's possible to get an idea of what part
> >>>>>> of BTF is not supported and show those that we know about. We could
> >>>>>> teach btf_dump to ignore those types that are "safe modifier-like
> >>>>>> reference kind" (as marked with that flag I proposed earlier), so that
> >>>>>> `format c` works as well (though I wouldn't recommend using such
> >>>>>> output as a proper vmlinux.h, users should update bpftool ASAP for
> >>>>>> such use cases).
> >>>>>>
> >>>>>> As far as the kernel is concerned, BTF layout is not used and should
> >>>>>> not be used or trusted (it can be "spoofed" by the user). It can
> >>>>>> validate it for sanity, but that's pretty much it. Other than that, if
> >>>>>> the kernel doesn't *completely* understand every single piece of BTF,
> >>>>>> it should reject it (and that's also why libbpf should sanitize BPF
> >>>>>> object's BTF, of course).
> >>>>>
> >>>>> +1 to all of the above, except ok-to-skip flag, since I feel
> >>>>> it will cause more bike sheding and arguing whether a particular
> >>>>> new addition to BTF is skippable or not. Like upcoming location info.
> >>>>
> >>>> I was thinking about something like TYPE_TAG, where it's in the chain
> >>>> of types and is unavoidable when processing STRUCT and its field.
> >>>> Having a flag specifying that it's ref-like (so btf_type::type field
> >>>> points to a valid type ID) would allow it to still make sense of the
> >>>> entire struct and its fields, though you might be missing some
> >>>> (presumably) optional and highly-specialized extra annotation.
> >>>>
> >>>> But it's fine not to add it, just some type graphs will be completely
> >>>> unprocessable using old tools. Perhaps not such a big deal.
> >>>>
> >>>> I suspect all the newly added BTF kinds will be of "ok-to-skip" kind,
> >>>> whether they are more like DECL_TAG (roots pointing to other types) or
> >>>> TYPE_TAG (in the middle of type chain, being pointed to from STRUCT
> >>>> fields, PROTO args, etc).
> >>>>
> >>>>> Is it skippable? kinda. Or, say, we decide to add vector types to BTF.
> >>>>> Skippable? One might argue that they are while they are mandatory
> >>>>> for some other use case.
> >>>>> Looking at it differently, if the kernel is old and cannot understand that
> >>>>> BTF feature the libbpf has to sanitize it no matter skippable or not.
> >>>>> While from btf__parse() pov it also doesn't matter.
> >>>>> btf_new()->btf_parse_hdr() will remember kind layout,
> >>>>> and btf_parse_type_sec() can construct the index for the whole thing
> >>>>> with layout info,
> >>>>> while btf_validate_type() has to warn about unknown kind regardless
> >>>>> of skippable flag. The tool (bpftool or else) needs to yell when
> >>>>> final vmlinux.h is incomplete. Skipping printing modifier-like decl_tag
> >>>>> is pretty bad for vmlinux.h. It's really not skippable (in my opinion)
> >>>>> though one might argue that they are.
> >>>>
> >>>> Yeah, I agree about vmlinux.h. One way to enforce this would be to
> >>>> have btf_dump emit something uncompilable as
> >>>> "HERE_BE_DRAGONS_SKIPPED_SOMETHING"  as if it was const/volatile
> >>>> modified.
> >>>>
> >>>> But yeah, we don't want bikeshedding. It's fine.
> >>>>
> >>>
> >>> Ok so is it best to leave out flags entirely then? If so where we
> >>> are now is to have each kind layout entry have a string name offset,
> >>> a singular element size and a vlen-specified object size. To be
> >>> conservative it might make sense to allow 16 bits for each size field,
> >>> leaving us with 64 bits per kind, 160 bytes in total for the 20 kinds.
> >>> We could cut down further by leaving out kind name strings if needed.
> >>
> >> Are we sure we will *never* need flags? I'd probably stick to
> >> single-byte sizes and have 2 bytes reserved for flags or whatever we
> >> might need in the future?
> >
> > Just to clarify what I was saying.
> > I think it's a good thing to have flags space and reserve it.
> > I just struggle to see the value of 'ok-to-skip' flag.
> >
> > So 2 bytes of reserved space for flags makes sense to me.
>
> Ok sounds good; I think there is still value in having the single flag
> that tells us that the type/size field in struct btf_type refers
> to a type though, right?

What's the value?

I sort-of kinda see small value of a set of flags like:
- this kind is a type (int, ptr, array, struct, func_proto)
- this kind is a modifier of a type (volatile, const, restrict, type_tag)

but then we cannot quite classify var, datasec, decl_tag, func..

So it feels like it's getting into the bikeshed category again.

Re: [PATCH v8 bpf-next 01/10] btf: add kind layout encoding to UAPI

[Alan Maguire]

On 09/01/2026 18:34, Alexei Starovoitov wrote:
> On Fri, Jan 9, 2026 at 5:21 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>>
>> On 09/01/2026 01:40, Alexei Starovoitov wrote:
>>> On Thu, Jan 8, 2026 at 5:24 PM Andrii Nakryiko
>>> <andrii.nakryiko@gmail.com> wrote:
>>>>
>>>> On Thu, Jan 8, 2026 at 10:55 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>>>>>
>>>>> On 06/01/2026 01:19, Andrii Nakryiko wrote:
>>>>>> On Mon, Jan 5, 2026 at 4:51 PM Alexei Starovoitov
>>>>>> <alexei.starovoitov@gmail.com> wrote:
>>>>>>>
>>>>>>> On Mon, Jan 5, 2026 at 4:11 PM Andrii Nakryiko
>>>>>>> <andrii.nakryiko@gmail.com> wrote:
>>>>>>>>
>>>>>>>> On Tue, Dec 23, 2025 at 3:09 AM Alan Maguire <alan.maguire@oracle.com> wrote:
>>>>>>>>>
>>>>>>>>> On 22/12/2025 19:03, Alexei Starovoitov wrote:
>>>>>>>>>> On Sun, Dec 21, 2025 at 10:58 PM Alan Maguire <alan.maguire@oracle.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Hold on. I'm missing how libbpf will sanitize things for older kernels?
>>>>>>>>>>>
>>>>>>>>>>> The sanitization we can get from layout info is for handling a kernel built with
>>>>>>>>>>> newer kernel/module BTF. The userspace tooling (libbpf and others) does not fully
>>>>>>>>>>> understand it due to the presence of new kinds. In such a case layout data gives us
>>>>>>>>>>> info to parse it by providing info on kind layout, and libbpf can sanitize it
>>>>>>>>>>> to be usable for some cases (where the type graph is not fatally compromised
>>>>>>>>>>> by the lack of a kind). This will always be somewhat limited, but it
>>>>>>>>>>> does provide more usability than we have today.
>>>>>>>>>>
>>>>>>>>>> I'm even more confused now. libbpf will sanitize BTF for the sake of
>>>>>>>>>> user space? That's not something it ever did. libbpf sanitizes BTF
>>>>>>>>>> only to
>>>>>>>>>
>>>>>>>>> Right; it's an extension of the sanitization concept from what it does today.
>>>>>>>>> Today we sanitize newer _program_ BTF to ensure it is acceptable to a kernel which
>>>>>>>>> lacks specific aspects of that BTF; the goal here is to support some simple sanitization
>>>>>>>>> of the newer _kernel_ BTF by libbpf to help tools (that know about kind layout but may lack
>>>>>>>>> latest kind info kernel has) to make that kernel BTF usable.
>>>>>>>>
>>>>>>>> Wait, is that really a goal? I get why Alexei is confused now :)
>>>>>>>>
>>>>>>>> I think we should stick to libbpf sanitizing only BPF program's BTFs
>>>>>>>> for the sake of loading it into the kernel. If some user space tool is
>>>>>>>> trying to work with kernel BTF that has BTF features that tool doesn't
>>>>>>>> support, then we only have two reasonable options: a) tool just fails
>>>>>>>> to process that BTF altogether or b) the tool is smart enough to
>>>>>>>> utilize BTF layout information to know which BTF types it can safely
>>>>>>>> skip (that's where those flags I argue for would be useful). In both
>>>>>>>> cases libbpf's btf__parse() will succeed because libbpf can utilize
>>>>>>>> layout info to construct a lookup table for btf__type_by_id(). And
>>>>>>>> libbpf doesn't need to do anything beyond that, IMO.
>>>>>>>>
>>>>>>>> We'll teach bpftool to dump as much of BTF as possible (I mean
>>>>>>>> `bpftool btf dump file`), so it's possible to get an idea of what part
>>>>>>>> of BTF is not supported and show those that we know about. We could
>>>>>>>> teach btf_dump to ignore those types that are "safe modifier-like
>>>>>>>> reference kind" (as marked with that flag I proposed earlier), so that
>>>>>>>> `format c` works as well (though I wouldn't recommend using such
>>>>>>>> output as a proper vmlinux.h, users should update bpftool ASAP for
>>>>>>>> such use cases).
>>>>>>>>
>>>>>>>> As far as the kernel is concerned, BTF layout is not used and should
>>>>>>>> not be used or trusted (it can be "spoofed" by the user). It can
>>>>>>>> validate it for sanity, but that's pretty much it. Other than that, if
>>>>>>>> the kernel doesn't *completely* understand every single piece of BTF,
>>>>>>>> it should reject it (and that's also why libbpf should sanitize BPF
>>>>>>>> object's BTF, of course).
>>>>>>>
>>>>>>> +1 to all of the above, except ok-to-skip flag, since I feel
>>>>>>> it will cause more bike sheding and arguing whether a particular
>>>>>>> new addition to BTF is skippable or not. Like upcoming location info.
>>>>>>
>>>>>> I was thinking about something like TYPE_TAG, where it's in the chain
>>>>>> of types and is unavoidable when processing STRUCT and its field.
>>>>>> Having a flag specifying that it's ref-like (so btf_type::type field
>>>>>> points to a valid type ID) would allow it to still make sense of the
>>>>>> entire struct and its fields, though you might be missing some
>>>>>> (presumably) optional and highly-specialized extra annotation.
>>>>>>
>>>>>> But it's fine not to add it, just some type graphs will be completely
>>>>>> unprocessable using old tools. Perhaps not such a big deal.
>>>>>>
>>>>>> I suspect all the newly added BTF kinds will be of "ok-to-skip" kind,
>>>>>> whether they are more like DECL_TAG (roots pointing to other types) or
>>>>>> TYPE_TAG (in the middle of type chain, being pointed to from STRUCT
>>>>>> fields, PROTO args, etc).
>>>>>>
>>>>>>> Is it skippable? kinda. Or, say, we decide to add vector types to BTF.
>>>>>>> Skippable? One might argue that they are while they are mandatory
>>>>>>> for some other use case.
>>>>>>> Looking at it differently, if the kernel is old and cannot understand that
>>>>>>> BTF feature the libbpf has to sanitize it no matter skippable or not.
>>>>>>> While from btf__parse() pov it also doesn't matter.
>>>>>>> btf_new()->btf_parse_hdr() will remember kind layout,
>>>>>>> and btf_parse_type_sec() can construct the index for the whole thing
>>>>>>> with layout info,
>>>>>>> while btf_validate_type() has to warn about unknown kind regardless
>>>>>>> of skippable flag. The tool (bpftool or else) needs to yell when
>>>>>>> final vmlinux.h is incomplete. Skipping printing modifier-like decl_tag
>>>>>>> is pretty bad for vmlinux.h. It's really not skippable (in my opinion)
>>>>>>> though one might argue that they are.
>>>>>>
>>>>>> Yeah, I agree about vmlinux.h. One way to enforce this would be to
>>>>>> have btf_dump emit something uncompilable as
>>>>>> "HERE_BE_DRAGONS_SKIPPED_SOMETHING"  as if it was const/volatile
>>>>>> modified.
>>>>>>
>>>>>> But yeah, we don't want bikeshedding. It's fine.
>>>>>>
>>>>>
>>>>> Ok so is it best to leave out flags entirely then? If so where we
>>>>> are now is to have each kind layout entry have a string name offset,
>>>>> a singular element size and a vlen-specified object size. To be
>>>>> conservative it might make sense to allow 16 bits for each size field,
>>>>> leaving us with 64 bits per kind, 160 bytes in total for the 20 kinds.
>>>>> We could cut down further by leaving out kind name strings if needed.
>>>>
>>>> Are we sure we will *never* need flags? I'd probably stick to
>>>> single-byte sizes and have 2 bytes reserved for flags or whatever we
>>>> might need in the future?
>>>
>>> Just to clarify what I was saying.
>>> I think it's a good thing to have flags space and reserve it.
>>> I just struggle to see the value of 'ok-to-skip' flag.
>>>
>>> So 2 bytes of reserved space for flags makes sense to me.
>>
>> Ok sounds good; I think there is still value in having the single flag
>> that tells us that the type/size field in struct btf_type refers
>> to a type though, right?
> 
> What's the value?
> 
> I sort-of kinda see small value of a set of flags like:
> - this kind is a type (int, ptr, array, struct, func_proto)
> - this kind is a modifier of a type (volatile, const, restrict, type_tag)
> 
> but then we cannot quite classify var, datasec, decl_tag, func..
> 
> So it feels like it's getting into the bikeshed category again.

I suppose one value would be it would be easier to recognize pure reference kinds
since they'd have such a flag set and have no extra info. But I think at this point
we should just focus on support parsing of unknown BTF kinds via kind layout. I'll add
an empty flags field for now.

Alan