* [PATCH 0/2] kallsyms: show typed function parameters in oops/WARN dumps
@ 2026-03-23 16:48 Sasha Levin
2026-03-23 16:48 ` [PATCH 1/2] kallsyms: show function parameter info " Sasha Levin
` (4 more replies)
0 siblings, 5 replies; 18+ messages in thread
From: Sasha Levin @ 2026-03-23 16:48 UTC (permalink / raw)
To: Andrew Morton, Masahiro Yamada, Nathan Chancellor, Nicolas Schier
Cc: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Peter Zijlstra, Josh Poimboeuf, Petr Mladek,
Alexei Starovoitov, Jonathan Corbet, David Gow, Kees Cook,
Greg KH, Luis Chamberlain, Steven Rostedt, Helge Deller,
Randy Dunlap, Geert Uytterhoeven, Juergen Gross, James Bottomley,
Alexey Dobriyan, Vlastimil Babka, Laurent Pinchart, Petr Pavlu,
x86, linux-kernel, linux-kbuild, linux-doc, linux-modules, bpf,
Sasha Levin
Building on the lineinfo series, this adds typed function parameter
display to oops and WARN dumps. A build-time tool extracts parameter
names and types from DWARF, and the kernel maps pt_regs to the calling
convention at crash time. When BTF is available, struct pointer
parameters are dereferenced and their members displayed.
Example output from a WARN in a function receiving struct new_utsname *
(kernel version info) and struct file * parameters:
------------[ cut here ]------------
WARNING: drivers/tty/sysrq.c:1209 at demo_crash+0xf/0x20 (drivers/tty/sysrq.c:1209)
CPU: 2 UID: 0 PID: 323 Comm: bash
RIP: 0010:demo_crash+0xf/0x20 (drivers/tty/sysrq.c:1209)
...
RDI: ffffffffb8ca8d00
RSI: ffffa0a3c250acc0
...
Function parameters (paraminfo_demo_crash):
uts (struct new_utsname *) = 0xffffffffb8ca8d00
.sysname = "Linux" .nodename = "localhost"
.release = "7.0.0-rc2-00006-g3190..." .version = "#45 SMP PRE"
file (struct file * ) = 0xffffa0a3c250acc0
.f_mode = (fmode_t)67993630 .f_op = (struct file_operations *)0xffffffffb7237620
.f_flags = (unsigned int)32769 .f_cred = (struct cred *)0xffffa0a3c2e06a80
.dentry = (struct dentry *)0xffffa0a3c0978cc0
.prev_pos = (loff_t)-1
Call Trace:
<TASK>
write_sysrq_trigger+0x96/0xb0 (drivers/tty/sysrq.c:1222)
proc_reg_write+0x54/0xa0 (fs/proc/inode.c:330)
vfs_write+0xc9/0x480 (fs/read_write.c:686)
ksys_write+0x6e/0xe0 (fs/read_write.c:738)
do_syscall_64+0xe2/0x570 (arch/x86/entry/syscall_64.c:62)
entry_SYSCALL_64_after_hwframe+0x77/0x7f (arch/x86/entry/entry_64.S:121)
Patch 1 adds the core paraminfo infrastructure (DWARF extraction,
kernel-side lookup, register-to-parameter mapping, ~1-2 MB overhead).
Patch 2 adds optional BTF-based struct rendering, gated behind
CONFIG_KALLSYMS_PARAMINFO_BTF.
Sasha Levin (2):
kallsyms: show function parameter info in oops/WARN dumps
kallsyms: add BTF-based deep parameter rendering in oops dumps
.../admin-guide/kallsyms-lineinfo.rst | 31 +
arch/x86/kernel/dumpstack.c | 6 +-
include/linux/kallsyms.h | 9 +
init/Kconfig | 40 ++
kernel/Makefile | 1 +
kernel/kallsyms.c | 182 ++++++
kernel/kallsyms_internal.h | 6 +
kernel/kallsyms_paraminfo_btf.c | 199 ++++++
lib/Kconfig.debug | 11 +
lib/tests/Makefile | 3 +
lib/tests/paraminfo_kunit.c | 249 ++++++++
scripts/Makefile | 3 +
scripts/empty_paraminfo.S | 18 +
scripts/gen_paraminfo.c | 597 ++++++++++++++++++
scripts/link-vmlinux.sh | 44 +-
15 files changed, 1393 insertions(+), 6 deletions(-)
create mode 100644 kernel/kallsyms_paraminfo_btf.c
create mode 100644 lib/tests/paraminfo_kunit.c
create mode 100644 scripts/empty_paraminfo.S
create mode 100644 scripts/gen_paraminfo.c
--
2.51.0
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH 1/2] kallsyms: show function parameter info in oops/WARN dumps
2026-03-23 16:48 [PATCH 0/2] kallsyms: show typed function parameters in oops/WARN dumps Sasha Levin
@ 2026-03-23 16:48 ` Sasha Levin
2026-03-24 15:03 ` Alexei Starovoitov
2026-03-23 16:48 ` [PATCH 2/2] kallsyms: add BTF-based deep parameter rendering in oops dumps Sasha Levin
` (3 subsequent siblings)
4 siblings, 1 reply; 18+ messages in thread
From: Sasha Levin @ 2026-03-23 16:48 UTC (permalink / raw)
To: Andrew Morton, Masahiro Yamada, Nathan Chancellor, Nicolas Schier
Cc: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Peter Zijlstra, Josh Poimboeuf, Petr Mladek,
Alexei Starovoitov, Jonathan Corbet, David Gow, Kees Cook,
Greg KH, Luis Chamberlain, Steven Rostedt, Helge Deller,
Randy Dunlap, Geert Uytterhoeven, Juergen Gross, James Bottomley,
Alexey Dobriyan, Vlastimil Babka, Laurent Pinchart, Petr Pavlu,
x86, linux-kernel, linux-kbuild, linux-doc, linux-modules, bpf,
Sasha Levin
Embed DWARF-derived function parameter name and type information in the
kernel image so that oops and WARN dumps display the crashing function's
register-passed arguments with their names, types, and values.
A new build-time tool (scripts/gen_paraminfo.c) parses DW_TAG_subprogram
and DW_TAG_formal_parameter entries from DWARF .debug_info, extracting
parameter names and human-readable type strings. The resulting tables are
stored in .rodata using the same two-phase link approach as lineinfo.
At runtime, kallsyms_show_paraminfo() performs a binary search on the
paraminfo tables, maps parameters to x86-64 calling convention registers
(RDI, RSI, RDX, RCX, R8, R9), and prints each parameter's name, type,
and value from pt_regs. If a parameter value matches the page fault
address (CR2), it is highlighted with "<-- fault address".
Integration at show_regs() means this works for both oops and WARN()
automatically, since both paths provide full pt_regs at the exception
point.
Example output:
Function parameters (ext4_readdir):
file (struct file *) = 0xffff888123456000
ctx (struct dir_context *) = 0x0000000000001234 <-- fault address
Gated behind CONFIG_KALLSYMS_PARAMINFO (depends on CONFIG_KALLSYMS_LINEINFO).
Adds approximately 1-2 MB to the kernel image for ~58K functions.
Assisted-by: Claude:claude-opus-4-6
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
.../admin-guide/kallsyms-lineinfo.rst | 31 +
arch/x86/kernel/dumpstack.c | 6 +-
include/linux/kallsyms.h | 9 +
init/Kconfig | 21 +
kernel/kallsyms.c | 168 ++++++
kernel/kallsyms_internal.h | 6 +
lib/Kconfig.debug | 11 +
lib/tests/Makefile | 3 +
lib/tests/paraminfo_kunit.c | 256 ++++++++
scripts/Makefile | 3 +
scripts/empty_paraminfo.S | 18 +
scripts/gen_paraminfo.c | 549 ++++++++++++++++++
scripts/link-vmlinux.sh | 44 +-
13 files changed, 1119 insertions(+), 6 deletions(-)
create mode 100644 lib/tests/paraminfo_kunit.c
create mode 100644 scripts/empty_paraminfo.S
create mode 100644 scripts/gen_paraminfo.c
diff --git a/Documentation/admin-guide/kallsyms-lineinfo.rst b/Documentation/admin-guide/kallsyms-lineinfo.rst
index dd264830c8d5b..26921bb3f7f81 100644
--- a/Documentation/admin-guide/kallsyms-lineinfo.rst
+++ b/Documentation/admin-guide/kallsyms-lineinfo.rst
@@ -83,6 +83,37 @@ compression).
Per-module lineinfo adds approximately 2-3 bytes per DWARF line entry to each
``.ko`` file.
+Function Parameter Info
+======================
+
+``CONFIG_KALLSYMS_PARAMINFO`` extends the debugging information by embedding
+function parameter names and types in the kernel image. When an oops or WARN
+occurs, the faulting function's register-passed arguments are displayed with
+their names, types, and values from the saved registers.
+
+Enable in addition to the lineinfo options::
+
+ CONFIG_KALLSYMS_PARAMINFO=y
+
+Example oops output with paraminfo::
+
+ RIP: 0010:ext4_readdir+0x1a3/0x5b0 (fs/ext4/dir.c:421)
+ ...
+ Function parameters (ext4_readdir):
+ file (struct file *) = 0xffff888123456000
+ ctx (struct dir_context *) = 0x0000000000001234 <-- fault address
+
+The ``<-- fault address`` annotation appears when a parameter value matches
+the page fault address (CR2 on x86), helping quickly identify which argument
+caused the crash.
+
+This feature works for both oops and WARN() on x86-64. Only register-passed
+parameters (up to 6 on x86-64: RDI, RSI, RDX, RCX, R8, R9) are displayed.
+The parameter info is only shown for the faulting/warning frame where full
+register state is available.
+
+The paraminfo tables add approximately 1-2 MiB to the kernel image.
+
Known Limitations
=================
diff --git a/arch/x86/kernel/dumpstack.c b/arch/x86/kernel/dumpstack.c
index b10684dedc589..4e9b5fd58fd1b 100644
--- a/arch/x86/kernel/dumpstack.c
+++ b/arch/x86/kernel/dumpstack.c
@@ -483,8 +483,10 @@ void show_regs(struct pt_regs *regs)
__show_regs(regs, print_kernel_regs, KERN_DEFAULT);
/*
- * When in-kernel, we also print out the stack at the time of the fault..
+ * When in-kernel, show function parameter info and stack trace.
*/
- if (!user_mode(regs))
+ if (!user_mode(regs)) {
+ kallsyms_show_paraminfo(regs);
show_trace_log_lvl(current, regs, NULL, KERN_DEFAULT);
+ }
}
diff --git a/include/linux/kallsyms.h b/include/linux/kallsyms.h
index 7d4c9dca06c87..17c9df520b2b0 100644
--- a/include/linux/kallsyms.h
+++ b/include/linux/kallsyms.h
@@ -104,6 +104,13 @@ int lookup_symbol_name(unsigned long addr, char *symname);
bool kallsyms_lookup_lineinfo(unsigned long addr,
const char **file, unsigned int *line);
+#ifdef CONFIG_KALLSYMS_PARAMINFO
+struct pt_regs;
+void kallsyms_show_paraminfo(struct pt_regs *regs);
+#else
+static inline void kallsyms_show_paraminfo(struct pt_regs *regs) {}
+#endif
+
#else /* !CONFIG_KALLSYMS */
static inline unsigned long kallsyms_lookup_name(const char *name)
@@ -179,6 +186,8 @@ static inline bool kallsyms_lookup_lineinfo(unsigned long addr,
{
return false;
}
+
+static inline void kallsyms_show_paraminfo(struct pt_regs *regs) {}
#endif /*CONFIG_KALLSYMS*/
static inline void print_ip_sym(const char *loglvl, unsigned long ip)
diff --git a/init/Kconfig b/init/Kconfig
index 6e3795b3dbd62..76d0c2da7d612 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -2085,6 +2085,27 @@ config KALLSYMS_LINEINFO_MODULES
If unsure, say N.
+config KALLSYMS_PARAMINFO
+ bool "Show function parameter info in oops/WARN dumps"
+ depends on KALLSYMS_LINEINFO
+ help
+ Embeds function parameter name and type information in the kernel
+ image, extracted from DWARF debug info at build time. When an
+ oops or WARN occurs, the crashing/warning function's register-
+ passed arguments are displayed with their names, types, and
+ values from pt_regs.
+
+ When enabled, oops/WARN dumps include lines like:
+
+ Function parameters (ext4_readdir):
+ file (struct file *) = 0xffff888123456000
+ ctx (struct dir_context *) = 0x0000000000001234
+
+ Requires elfutils (libdw-dev/elfutils-devel) on the build host.
+ Adds approximately 1-2 MB to the kernel image.
+
+ If unsure, say N.
+
# end of the "standard kernel features (expert users)" menu
config ARCH_HAS_MEMBARRIER_CALLBACKS
diff --git a/kernel/kallsyms.c b/kernel/kallsyms.c
index e6f796d43dd70..af8de3d8e3ba3 100644
--- a/kernel/kallsyms.c
+++ b/kernel/kallsyms.c
@@ -501,6 +501,174 @@ bool kallsyms_lookup_lineinfo(unsigned long addr,
return lineinfo_search(&tbl, (unsigned int)raw_offset, file, line);
}
+#ifdef CONFIG_KALLSYMS_PARAMINFO
+
+#include <linux/ptrace.h>
+#ifdef CONFIG_X86
+#include <asm/special_insns.h>
+#endif
+
+#define MAX_PARAMINFO_PARAMS 6
+
+/*
+ * x86-64 calling convention: arguments are passed in registers
+ * RDI, RSI, RDX, RCX, R8, R9 (in that order).
+ */
+#ifdef CONFIG_X86_64
+
+static const char * const paraminfo_reg_names[] = {
+ "RDI", "RSI", "RDX", "RCX", "R8", "R9"
+};
+
+static unsigned long paraminfo_get_reg(const struct pt_regs *regs,
+ unsigned int idx)
+{
+ switch (idx) {
+ case 0: return regs->di;
+ case 1: return regs->si;
+ case 2: return regs->dx;
+ case 3: return regs->cx;
+ case 4: return regs->r8;
+ case 5: return regs->r9;
+ default: return 0;
+ }
+}
+#else
+/* Stub for non-x86-64 architectures */
+static const char * const paraminfo_reg_names[] = {};
+
+static unsigned long paraminfo_get_reg(const struct pt_regs *regs,
+ unsigned int idx)
+{
+ return 0;
+}
+#endif /* CONFIG_X86_64 */
+
+/*
+ * Binary search for the function containing the given offset in
+ * paraminfo_func_addrs[]. Returns the index of the function whose
+ * start address is <= offset, or -1 if not found.
+ */
+static int paraminfo_find_func(unsigned int offset)
+{
+ int lo = 0, hi = paraminfo_num_funcs - 1;
+ int result = -1;
+
+ if (!paraminfo_num_funcs)
+ return -1;
+
+ while (lo <= hi) {
+ int mid = lo + (hi - lo) / 2;
+
+ if (paraminfo_func_addrs[mid] <= offset) {
+ result = mid;
+ lo = mid + 1;
+ } else {
+ hi = mid - 1;
+ }
+ }
+
+ return result;
+}
+
+/*
+ * Show function parameter info for the faulting/warning instruction.
+ *
+ * Called from show_regs() on x86 when CONFIG_KALLSYMS_PARAMINFO is
+ * enabled. Works for both oops (page fault, GPF, etc.) and WARN(),
+ * since both paths provide full pt_regs at the exception point.
+ */
+void kallsyms_show_paraminfo(struct pt_regs *regs)
+{
+ unsigned long long raw_offset;
+ unsigned int offset;
+ int func_idx;
+ const u8 *data;
+ unsigned int num_params, i;
+ unsigned long ip, fault_addr;
+ char sym_name[KSYM_NAME_LEN];
+ unsigned long sym_size, sym_offset;
+
+ if (!regs || !paraminfo_num_funcs)
+ return;
+
+ ip = regs->ip;
+
+ /* Only handle kernel-mode faults */
+ if (user_mode(regs))
+ return;
+
+ if (ip < (unsigned long)_text)
+ return;
+
+ raw_offset = ip - (unsigned long)_text;
+ if (raw_offset > UINT_MAX)
+ return;
+ offset = (unsigned int)raw_offset;
+
+ func_idx = paraminfo_find_func(offset);
+ if (func_idx < 0)
+ return;
+
+ /*
+ * Verify the IP is within a reasonable range of the function
+ * start. paraminfo_func_addrs[] contains function start offsets;
+ * check that we're not too far past the start. Use kallsyms to
+ * verify we're in the right function.
+ */
+ if (!kallsyms_lookup_size_offset(ip, &sym_size, &sym_offset))
+ return;
+
+ /* Decode the function's parameter data */
+ data = paraminfo_func_data + paraminfo_func_offsets[func_idx];
+ num_params = *data++;
+
+ if (num_params == 0 || num_params > MAX_PARAMINFO_PARAMS)
+ return;
+
+ /* Look up function name for the header */
+ if (lookup_symbol_name(ip - sym_offset, sym_name))
+ return;
+
+ /*
+ * Read the fault address for highlighting. On x86, CR2 holds
+ * the page fault linear address. On other architectures this
+ * would need a different mechanism.
+ */
+#ifdef CONFIG_X86
+ fault_addr = read_cr2();
+#else
+ fault_addr = 0;
+#endif
+
+ printk(KERN_DEFAULT "Function parameters (%s):\n", sym_name);
+
+ for (i = 0; i < num_params; i++) {
+ u32 name_off, type_off;
+ const char *pname, *ptype;
+ unsigned long val;
+ bool is_fault_addr;
+
+ memcpy(&name_off, data, sizeof(u32));
+ data += sizeof(u32);
+ memcpy(&type_off, data, sizeof(u32));
+ data += sizeof(u32);
+
+ pname = paraminfo_strings + name_off;
+ ptype = paraminfo_strings + type_off;
+ val = paraminfo_get_reg(regs, i);
+
+ is_fault_addr = fault_addr && (val == fault_addr);
+
+ printk(KERN_DEFAULT " %-8s (%-20s) = 0x%016lx%s\n",
+ pname, ptype, val,
+ is_fault_addr ? " <-- fault address" : "");
+ }
+}
+EXPORT_SYMBOL_GPL(kallsyms_show_paraminfo);
+
+#endif /* CONFIG_KALLSYMS_PARAMINFO */
+
/* Look up a kernel symbol and return it in a text buffer. */
static int __sprint_symbol(char *buffer, unsigned long address,
int symbol_offset, int add_offset, int add_buildid)
diff --git a/kernel/kallsyms_internal.h b/kernel/kallsyms_internal.h
index ffe4c658067ec..7287ee0859515 100644
--- a/kernel/kallsyms_internal.h
+++ b/kernel/kallsyms_internal.h
@@ -26,4 +26,10 @@ extern const u32 lineinfo_file_offsets[];
extern const u32 lineinfo_filenames_size;
extern const char lineinfo_filenames[];
+extern const u32 paraminfo_num_funcs;
+extern const u32 paraminfo_func_addrs[];
+extern const u32 paraminfo_func_offsets[];
+extern const u8 paraminfo_func_data[];
+extern const char paraminfo_strings[];
+
#endif // LINUX_KALLSYMS_INTERNAL_H_
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 688bbcb3eaa62..be8cee0985fbd 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -3058,6 +3058,17 @@ config LINEINFO_KUNIT_TEST
If unsure, say N.
+config PARAMINFO_KUNIT_TEST
+ tristate "KUnit tests for kallsyms paraminfo" if !KUNIT_ALL_TESTS
+ depends on KUNIT && KALLSYMS_PARAMINFO
+ default KUNIT_ALL_TESTS
+ help
+ KUnit tests for the kallsyms function parameter info feature.
+ Verifies that paraminfo tables correctly map functions to their
+ parameter names and types.
+
+ If unsure, say N.
+
config HW_BREAKPOINT_KUNIT_TEST
bool "Test hw_breakpoint constraints accounting" if !KUNIT_ALL_TESTS
depends on HAVE_HW_BREAKPOINT
diff --git a/lib/tests/Makefile b/lib/tests/Makefile
index c6add3b04bbd5..70452942baf45 100644
--- a/lib/tests/Makefile
+++ b/lib/tests/Makefile
@@ -39,6 +39,9 @@ obj-$(CONFIG_LONGEST_SYM_KUNIT_TEST) += longest_symbol_kunit.o
CFLAGS_lineinfo_kunit.o += $(call cc-option,-fno-inline-functions-called-once)
obj-$(CONFIG_LINEINFO_KUNIT_TEST) += lineinfo_kunit.o
+CFLAGS_paraminfo_kunit.o += $(call cc-option,-fno-inline-functions-called-once)
+obj-$(CONFIG_PARAMINFO_KUNIT_TEST) += paraminfo_kunit.o
+
obj-$(CONFIG_MEMCPY_KUNIT_TEST) += memcpy_kunit.o
obj-$(CONFIG_MIN_HEAP_KUNIT_TEST) += min_heap_kunit.o
CFLAGS_overflow_kunit.o = $(call cc-disable-warning, tautological-constant-out-of-range-compare)
diff --git a/lib/tests/paraminfo_kunit.c b/lib/tests/paraminfo_kunit.c
new file mode 100644
index 0000000000000..e09efc4ddeb0e
--- /dev/null
+++ b/lib/tests/paraminfo_kunit.c
@@ -0,0 +1,256 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * KUnit tests for kallsyms paraminfo (CONFIG_KALLSYMS_PARAMINFO).
+ *
+ * Copyright (c) 2026 Sasha Levin <sashal@kernel.org>
+ *
+ * Verifies that the paraminfo tables correctly map function addresses
+ * to their parameter names and types.
+ *
+ * Build with: CONFIG_PARAMINFO_KUNIT_TEST=m (or =y)
+ */
+
+#include <kunit/test.h>
+#include <linux/kallsyms.h>
+#include <linux/module.h>
+#include <linux/string.h>
+#include <linux/slab.h>
+
+/* ---- Test target functions with known signatures ---- */
+
+static noinline int paraminfo_test_two_args(struct kunit *test, int value)
+{
+ /* Prevent optimization */
+ return test ? value + 1 : 0;
+}
+
+static noinline void *paraminfo_test_ptr_arg(void *ptr, unsigned long size)
+{
+ if (ptr && size > 0)
+ return ptr;
+ return NULL;
+}
+
+static noinline long paraminfo_test_many_args(int a, int b, int c,
+ int d, int e, int f)
+{
+ return (long)a + b + c + d + e + f;
+}
+
+static noinline void paraminfo_test_no_args(void)
+{
+ /* Function with no parameters */
+ barrier();
+}
+
+/* ---- Helpers to query paraminfo tables directly ---- */
+
+/*
+ * These access the raw paraminfo tables to verify correctness.
+ * The tables are defined in kernel/kallsyms_internal.h.
+ */
+extern const u32 paraminfo_num_funcs;
+extern const u32 paraminfo_func_addrs[];
+extern const u32 paraminfo_func_offsets[];
+extern const u8 paraminfo_func_data[];
+extern const char paraminfo_strings[];
+
+struct param_result {
+ unsigned int num_params;
+ const char *names[6];
+ const char *types[6];
+};
+
+/*
+ * Look up paraminfo for a given kernel address.
+ * Returns true if found, filling in @result.
+ */
+static bool lookup_paraminfo(unsigned long addr, struct param_result *result)
+{
+ unsigned long long raw_offset;
+ unsigned int offset;
+ int lo, hi, func_idx;
+ const u8 *data;
+ unsigned int i;
+
+ if (!paraminfo_num_funcs)
+ return false;
+
+ if (addr < (unsigned long)_text)
+ return false;
+
+ raw_offset = addr - (unsigned long)_text;
+ if (raw_offset > UINT_MAX)
+ return false;
+ offset = (unsigned int)raw_offset;
+
+ /* Binary search for the function */
+ lo = 0;
+ hi = paraminfo_num_funcs - 1;
+ func_idx = -1;
+ while (lo <= hi) {
+ int mid = lo + (hi - lo) / 2;
+
+ if (paraminfo_func_addrs[mid] <= offset) {
+ func_idx = mid;
+ lo = mid + 1;
+ } else {
+ hi = mid - 1;
+ }
+ }
+
+ if (func_idx < 0)
+ return false;
+
+ /* Verify we're not too far from the function start */
+ if (offset - paraminfo_func_addrs[func_idx] > 0x10000)
+ return false;
+
+ data = paraminfo_func_data + paraminfo_func_offsets[func_idx];
+ result->num_params = *data++;
+
+ if (result->num_params > 6)
+ return false;
+
+ for (i = 0; i < result->num_params; i++) {
+ u32 name_off, type_off;
+
+ memcpy(&name_off, data, sizeof(u32));
+ data += sizeof(u32);
+ memcpy(&type_off, data, sizeof(u32));
+ data += sizeof(u32);
+
+ result->names[i] = paraminfo_strings + name_off;
+ result->types[i] = paraminfo_strings + type_off;
+ }
+
+ return true;
+}
+
+/* ---- Test cases ---- */
+
+static void test_paraminfo_two_args(struct kunit *test)
+{
+ struct param_result res;
+ bool found;
+
+ found = lookup_paraminfo((unsigned long)paraminfo_test_two_args, &res);
+
+ if (!IS_ENABLED(CONFIG_KALLSYMS_PARAMINFO)) {
+ KUNIT_EXPECT_FALSE(test, found);
+ return;
+ }
+
+ KUNIT_ASSERT_TRUE(test, found);
+ KUNIT_EXPECT_EQ(test, res.num_params, 2U);
+ KUNIT_EXPECT_STREQ(test, res.names[0], "test");
+ KUNIT_EXPECT_STREQ(test, res.names[1], "value");
+ KUNIT_EXPECT_TRUE(test, strstr(res.types[1], "int") != NULL);
+}
+
+static void test_paraminfo_ptr_arg(struct kunit *test)
+{
+ struct param_result res;
+ bool found;
+
+ found = lookup_paraminfo((unsigned long)paraminfo_test_ptr_arg, &res);
+
+ if (!IS_ENABLED(CONFIG_KALLSYMS_PARAMINFO)) {
+ KUNIT_EXPECT_FALSE(test, found);
+ return;
+ }
+
+ KUNIT_ASSERT_TRUE(test, found);
+ KUNIT_EXPECT_EQ(test, res.num_params, 2U);
+ KUNIT_EXPECT_STREQ(test, res.names[0], "ptr");
+ KUNIT_EXPECT_STREQ(test, res.names[1], "size");
+ /* First param should be a pointer type */
+ KUNIT_EXPECT_TRUE(test, strstr(res.types[0], "*") != NULL);
+}
+
+static void test_paraminfo_many_args(struct kunit *test)
+{
+ struct param_result res;
+ bool found;
+
+ found = lookup_paraminfo((unsigned long)paraminfo_test_many_args, &res);
+
+ if (!IS_ENABLED(CONFIG_KALLSYMS_PARAMINFO)) {
+ KUNIT_EXPECT_FALSE(test, found);
+ return;
+ }
+
+ KUNIT_ASSERT_TRUE(test, found);
+ KUNIT_EXPECT_EQ(test, res.num_params, 6U);
+ KUNIT_EXPECT_STREQ(test, res.names[0], "a");
+ KUNIT_EXPECT_STREQ(test, res.names[1], "b");
+ KUNIT_EXPECT_STREQ(test, res.names[2], "c");
+ KUNIT_EXPECT_STREQ(test, res.names[3], "d");
+ KUNIT_EXPECT_STREQ(test, res.names[4], "e");
+ KUNIT_EXPECT_STREQ(test, res.names[5], "f");
+}
+
+static void test_paraminfo_no_args(struct kunit *test)
+{
+ struct param_result res;
+ bool found;
+
+ /*
+ * Functions with no parameters should not have entries in the
+ * paraminfo table (they are filtered out at build time).
+ * The lookup may find a nearby function instead, but if it
+ * does find our function exactly, num_params should be 0.
+ */
+ found = lookup_paraminfo((unsigned long)paraminfo_test_no_args, &res);
+
+ if (found) {
+ /* If it matched our exact function, it should have 0 params.
+ * But it may have matched a preceding function instead.
+ */
+ kunit_info(test, "lookup found func with %u params\n",
+ res.num_params);
+ }
+}
+
+static void test_paraminfo_bogus_addr(struct kunit *test)
+{
+ struct param_result res;
+
+ /* Address 0 should not match anything */
+ KUNIT_EXPECT_FALSE(test, lookup_paraminfo(0, &res));
+
+ /* Address in userspace should not match */
+ KUNIT_EXPECT_FALSE(test, lookup_paraminfo(0x1000, &res));
+}
+
+static void test_paraminfo_tables_present(struct kunit *test)
+{
+ if (!IS_ENABLED(CONFIG_KALLSYMS_PARAMINFO)) {
+ kunit_skip(test, "CONFIG_KALLSYMS_PARAMINFO not enabled");
+ return;
+ }
+
+ KUNIT_EXPECT_GT(test, paraminfo_num_funcs, 0U);
+ kunit_info(test, "paraminfo: %u functions in table\n",
+ paraminfo_num_funcs);
+}
+
+static struct kunit_case paraminfo_test_cases[] = {
+ KUNIT_CASE(test_paraminfo_tables_present),
+ KUNIT_CASE(test_paraminfo_two_args),
+ KUNIT_CASE(test_paraminfo_ptr_arg),
+ KUNIT_CASE(test_paraminfo_many_args),
+ KUNIT_CASE(test_paraminfo_no_args),
+ KUNIT_CASE(test_paraminfo_bogus_addr),
+ {}
+};
+
+static struct kunit_suite paraminfo_test_suite = {
+ .name = "paraminfo",
+ .test_cases = paraminfo_test_cases,
+};
+
+kunit_test_suite(paraminfo_test_suite);
+
+MODULE_LICENSE("GPL");
+MODULE_DESCRIPTION("KUnit tests for kallsyms paraminfo");
diff --git a/scripts/Makefile b/scripts/Makefile
index ffe89875b3295..f681b94c6d9e7 100644
--- a/scripts/Makefile
+++ b/scripts/Makefile
@@ -5,6 +5,7 @@
hostprogs-always-$(CONFIG_KALLSYMS) += kallsyms
hostprogs-always-$(CONFIG_KALLSYMS_LINEINFO) += gen_lineinfo
+hostprogs-always-$(CONFIG_KALLSYMS_PARAMINFO) += gen_paraminfo
hostprogs-always-$(BUILD_C_RECORDMCOUNT) += recordmcount
hostprogs-always-$(CONFIG_BUILDTIME_TABLE_SORT) += sorttable
hostprogs-always-$(CONFIG_ASN1) += asn1_compiler
@@ -39,6 +40,8 @@ HOSTCFLAGS_sign-file.o = $(shell $(HOSTPKG_CONFIG) --cflags libcrypto 2> /dev/nu
HOSTLDLIBS_sign-file = $(shell $(HOSTPKG_CONFIG) --libs libcrypto 2> /dev/null || echo -lcrypto)
HOSTCFLAGS_gen_lineinfo.o = $(shell $(HOSTPKG_CONFIG) --cflags libdw 2> /dev/null)
HOSTLDLIBS_gen_lineinfo = $(shell $(HOSTPKG_CONFIG) --libs libdw 2> /dev/null || echo -ldw -lelf -lz)
+HOSTCFLAGS_gen_paraminfo.o = $(shell $(HOSTPKG_CONFIG) --cflags libdw 2> /dev/null)
+HOSTLDLIBS_gen_paraminfo = $(shell $(HOSTPKG_CONFIG) --libs libdw 2> /dev/null || echo -ldw -lelf -lz)
ifdef CONFIG_UNWINDER_ORC
ifeq ($(ARCH),x86_64)
diff --git a/scripts/empty_paraminfo.S b/scripts/empty_paraminfo.S
new file mode 100644
index 0000000000000..a0e39dbfbd3cb
--- /dev/null
+++ b/scripts/empty_paraminfo.S
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Empty paraminfo stub for the initial vmlinux link.
+ * The real paraminfo is generated from .tmp_vmlinux1 by gen_paraminfo.
+ */
+ .section .rodata, "a"
+ .globl paraminfo_num_funcs
+ .balign 4
+paraminfo_num_funcs:
+ .long 0
+ .globl paraminfo_func_addrs
+paraminfo_func_addrs:
+ .globl paraminfo_func_offsets
+paraminfo_func_offsets:
+ .globl paraminfo_func_data
+paraminfo_func_data:
+ .globl paraminfo_strings
+paraminfo_strings:
diff --git a/scripts/gen_paraminfo.c b/scripts/gen_paraminfo.c
new file mode 100644
index 0000000000000..ea1d23f3ddd9a
--- /dev/null
+++ b/scripts/gen_paraminfo.c
@@ -0,0 +1,549 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * gen_paraminfo.c - Generate function parameter info tables from DWARF
+ *
+ * Copyright (C) 2026 Sasha Levin <sashal@kernel.org>
+ *
+ * Reads DWARF .debug_info from a vmlinux ELF file and outputs an assembly
+ * file containing function parameter name/type tables that the kernel uses
+ * to annotate oops/WARN dumps with typed parameter values.
+ *
+ * Requires libdw from elfutils.
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <errno.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <elfutils/libdw.h>
+#include <dwarf.h>
+#include <elf.h>
+#include <gelf.h>
+#include <limits.h>
+
+/* Maximum register-passed parameters on x86-64 */
+#define MAX_PARAMS 6
+
+/* Maximum length for a type name string */
+#define MAX_TYPE_LEN 64
+
+struct param_info {
+ char name[64];
+ char type[MAX_TYPE_LEN];
+};
+
+struct func_entry {
+ unsigned int offset; /* offset from _text */
+ unsigned int num_params;
+ struct param_info params[MAX_PARAMS];
+};
+
+static struct func_entry *funcs;
+static unsigned int num_funcs;
+static unsigned int funcs_capacity;
+
+/*
+ * String table for parameter names and type names.
+ * Deduplicated via a hash table.
+ */
+struct str_entry {
+ char *str;
+ unsigned int offset; /* byte offset in concatenated output */
+};
+
+static struct str_entry *strtab;
+static unsigned int num_strings;
+static unsigned int strtab_capacity;
+static unsigned int strtab_total_size;
+
+#define STR_HASH_BITS 14
+#define STR_HASH_SIZE (1 << STR_HASH_BITS)
+
+struct str_hash_entry {
+ const char *str;
+ unsigned int idx; /* index into strtab[] */
+};
+
+static struct str_hash_entry str_hash[STR_HASH_SIZE];
+
+static unsigned int hash_str(const char *s)
+{
+ unsigned int h = 5381;
+
+ for (; *s; s++)
+ h = h * 33 + (unsigned char)*s;
+ return h & (STR_HASH_SIZE - 1);
+}
+
+static unsigned int find_or_add_string(const char *s)
+{
+ unsigned int h = hash_str(s);
+
+ while (str_hash[h].str) {
+ if (!strcmp(str_hash[h].str, s))
+ return str_hash[h].idx;
+ h = (h + 1) & (STR_HASH_SIZE - 1);
+ }
+
+ if (num_strings >= strtab_capacity) {
+ strtab_capacity = strtab_capacity ? strtab_capacity * 2 : 8192;
+ strtab = realloc(strtab, strtab_capacity * sizeof(*strtab));
+ if (!strtab) {
+ fprintf(stderr, "out of memory\n");
+ exit(1);
+ }
+ }
+
+ strtab[num_strings].str = strdup(s);
+ strtab[num_strings].offset = strtab_total_size;
+ strtab_total_size += strlen(s) + 1;
+
+ str_hash[h].str = strtab[num_strings].str;
+ str_hash[h].idx = num_strings;
+
+ num_strings++;
+ return num_strings - 1;
+}
+
+static void add_func(struct func_entry *f)
+{
+ if (num_funcs >= funcs_capacity) {
+ funcs_capacity = funcs_capacity ? funcs_capacity * 2 : 16384;
+ funcs = realloc(funcs, funcs_capacity * sizeof(*funcs));
+ if (!funcs) {
+ fprintf(stderr, "out of memory\n");
+ exit(1);
+ }
+ }
+ funcs[num_funcs++] = *f;
+}
+
+/*
+ * Build a human-readable type name string from a DWARF type DIE.
+ * Follows the type chain (pointers, const, etc.) to produce strings like:
+ * "struct file *", "const char *", "unsigned long", "void *"
+ */
+static void build_type_name(Dwarf_Die *type_die, char *buf, size_t bufsz)
+{
+ Dwarf_Die child;
+ Dwarf_Attribute attr;
+ const char *name;
+ int tag;
+
+ if (!type_die) {
+ snprintf(buf, bufsz, "void");
+ return;
+ }
+
+ tag = dwarf_tag(type_die);
+
+ switch (tag) {
+ case DW_TAG_base_type:
+ name = dwarf_diename(type_die);
+ snprintf(buf, bufsz, "%s", name ? name : "?");
+ break;
+
+ case DW_TAG_pointer_type:
+ if (dwarf_attr(type_die, DW_AT_type, &attr) &&
+ dwarf_formref_die(&attr, &child)) {
+ build_type_name(&child, buf, bufsz);
+ if (strlen(buf) + 3 < bufsz)
+ strcat(buf, " *");
+ } else {
+ snprintf(buf, bufsz, "void *");
+ }
+ break;
+
+ case DW_TAG_const_type:
+ if (dwarf_attr(type_die, DW_AT_type, &attr) &&
+ dwarf_formref_die(&attr, &child)) {
+ char tmp[MAX_TYPE_LEN - 10];
+
+ build_type_name(&child, tmp, sizeof(tmp));
+ snprintf(buf, bufsz, "const %s", tmp);
+ } else {
+ snprintf(buf, bufsz, "const void");
+ }
+ break;
+
+ case DW_TAG_volatile_type:
+ if (dwarf_attr(type_die, DW_AT_type, &attr) &&
+ dwarf_formref_die(&attr, &child)) {
+ char tmp[MAX_TYPE_LEN - 10];
+
+ build_type_name(&child, tmp, sizeof(tmp));
+ snprintf(buf, bufsz, "volatile %s", tmp);
+ } else {
+ snprintf(buf, bufsz, "volatile void");
+ }
+ break;
+
+ case DW_TAG_restrict_type:
+ if (dwarf_attr(type_die, DW_AT_type, &attr) &&
+ dwarf_formref_die(&attr, &child)) {
+ build_type_name(&child, buf, bufsz);
+ } else {
+ snprintf(buf, bufsz, "void");
+ }
+ break;
+
+ case DW_TAG_typedef:
+ name = dwarf_diename(type_die);
+ if (name) {
+ snprintf(buf, bufsz, "%s", name);
+ } else if (dwarf_attr(type_die, DW_AT_type, &attr) &&
+ dwarf_formref_die(&attr, &child)) {
+ build_type_name(&child, buf, bufsz);
+ } else {
+ snprintf(buf, bufsz, "?");
+ }
+ break;
+
+ case DW_TAG_structure_type:
+ name = dwarf_diename(type_die);
+ snprintf(buf, bufsz, "struct %s", name ? name : "(anon)");
+ break;
+
+ case DW_TAG_union_type:
+ name = dwarf_diename(type_die);
+ snprintf(buf, bufsz, "union %s", name ? name : "(anon)");
+ break;
+
+ case DW_TAG_enumeration_type:
+ name = dwarf_diename(type_die);
+ snprintf(buf, bufsz, "enum %s", name ? name : "(anon)");
+ break;
+
+ case DW_TAG_array_type:
+ if (dwarf_attr(type_die, DW_AT_type, &attr) &&
+ dwarf_formref_die(&attr, &child)) {
+ build_type_name(&child, buf, bufsz);
+ if (strlen(buf) + 3 < bufsz)
+ strcat(buf, "[]");
+ } else {
+ snprintf(buf, bufsz, "?[]");
+ }
+ break;
+
+ case DW_TAG_subroutine_type:
+ snprintf(buf, bufsz, "func_ptr");
+ break;
+
+ default:
+ snprintf(buf, bufsz, "?");
+ break;
+ }
+}
+
+static unsigned long long find_text_addr(Elf *elf)
+{
+ size_t nsyms, i;
+ Elf_Scn *scn = NULL;
+ GElf_Shdr shdr;
+
+ while ((scn = elf_nextscn(elf, scn)) != NULL) {
+ Elf_Data *data;
+
+ if (!gelf_getshdr(scn, &shdr))
+ continue;
+ if (shdr.sh_type != SHT_SYMTAB)
+ continue;
+
+ data = elf_getdata(scn, NULL);
+ if (!data)
+ continue;
+
+ nsyms = shdr.sh_size / shdr.sh_entsize;
+ for (i = 0; i < nsyms; i++) {
+ GElf_Sym sym;
+ const char *name;
+
+ if (!gelf_getsym(data, i, &sym))
+ continue;
+ name = elf_strptr(elf, shdr.sh_link, sym.st_name);
+ if (name && !strcmp(name, "_text"))
+ return sym.st_value;
+ }
+ }
+
+ fprintf(stderr, "Cannot find _text symbol\n");
+ exit(1);
+}
+
+static int compare_funcs(const void *a, const void *b)
+{
+ const struct func_entry *fa = a;
+ const struct func_entry *fb = b;
+
+ if (fa->offset < fb->offset)
+ return -1;
+ if (fa->offset > fb->offset)
+ return 1;
+ return 0;
+}
+
+static void process_dwarf(Dwarf *dwarf, unsigned long long text_addr)
+{
+ Dwarf_Off off = 0, next_off;
+ size_t hdr_size;
+
+ while (dwarf_nextcu(dwarf, off, &next_off, &hdr_size,
+ NULL, NULL, NULL) == 0) {
+ Dwarf_Die cudie, child;
+
+ if (!dwarf_offdie(dwarf, off + hdr_size, &cudie))
+ goto next;
+
+ if (dwarf_child(&cudie, &child) != 0)
+ goto next;
+
+ do {
+ Dwarf_Die param;
+ Dwarf_Attribute attr;
+ Dwarf_Addr low_pc;
+ struct func_entry func;
+ int tag;
+
+ tag = dwarf_tag(&child);
+ if (tag != DW_TAG_subprogram)
+ continue;
+
+ /* Skip declarations (no body) */
+ if (dwarf_attr(&child, DW_AT_declaration, &attr))
+ continue;
+
+ /* Get function start address */
+ if (dwarf_lowpc(&child, &low_pc) != 0)
+ continue;
+
+ if (low_pc < text_addr)
+ continue;
+
+ {
+ unsigned long long raw_offset = low_pc - text_addr;
+
+ if (raw_offset > UINT_MAX)
+ continue;
+ func.offset = (unsigned int)raw_offset;
+ }
+
+ /* Iterate formal parameters */
+ func.num_params = 0;
+ if (dwarf_child(&child, ¶m) == 0) {
+ do {
+ Dwarf_Die type_die;
+ const char *pname;
+
+ if (dwarf_tag(¶m) != DW_TAG_formal_parameter)
+ continue;
+ if (func.num_params >= MAX_PARAMS)
+ break;
+
+ pname = dwarf_diename(¶m);
+ if (!pname)
+ pname = "?";
+
+ snprintf(func.params[func.num_params].name,
+ sizeof(func.params[0].name),
+ "%s", pname);
+
+ /* Resolve type */
+ if (dwarf_attr(¶m, DW_AT_type, &attr) &&
+ dwarf_formref_die(&attr, &type_die)) {
+ build_type_name(&type_die,
+ func.params[func.num_params].type,
+ MAX_TYPE_LEN);
+ } else {
+ snprintf(func.params[func.num_params].type,
+ MAX_TYPE_LEN, "?");
+ }
+
+ func.num_params++;
+ } while (dwarf_siblingof(¶m, ¶m) == 0);
+ }
+
+ /* Skip functions with no parameters */
+ if (func.num_params == 0)
+ continue;
+
+ add_func(&func);
+ } while (dwarf_siblingof(&child, &child) == 0);
+next:
+ off = next_off;
+ }
+}
+
+static void deduplicate(void)
+{
+ unsigned int i, j;
+
+ if (num_funcs < 2)
+ return;
+
+ /* Sort by offset */
+ qsort(funcs, num_funcs, sizeof(*funcs), compare_funcs);
+
+ /* Remove duplicates (same offset — keep first) */
+ j = 0;
+ for (i = 1; i < num_funcs; i++) {
+ if (funcs[i].offset == funcs[j].offset)
+ continue;
+ j++;
+ if (j != i)
+ funcs[j] = funcs[i];
+ }
+ num_funcs = j + 1;
+}
+
+static void print_escaped_asciz(const char *s)
+{
+ printf("\t.asciz \"");
+ for (; *s; s++) {
+ if (*s == '"' || *s == '\\')
+ putchar('\\');
+ putchar(*s);
+ }
+ printf("\"\n");
+}
+
+static void output_assembly(void)
+{
+ unsigned int i, j;
+
+ printf("/* SPDX-License-Identifier: GPL-2.0 */\n");
+ printf("/*\n");
+ printf(" * Automatically generated by scripts/gen_paraminfo\n");
+ printf(" * Do not edit.\n");
+ printf(" */\n\n");
+
+ printf("\t.section .rodata, \"a\"\n\n");
+
+ /* Number of functions */
+ printf("\t.globl paraminfo_num_funcs\n");
+ printf("\t.balign 4\n");
+ printf("paraminfo_num_funcs:\n");
+ printf("\t.long %u\n\n", num_funcs);
+
+ /* Function address offsets (sorted, for binary search) */
+ printf("\t.globl paraminfo_func_addrs\n");
+ printf("\t.balign 4\n");
+ printf("paraminfo_func_addrs:\n");
+ for (i = 0; i < num_funcs; i++)
+ printf("\t.long 0x%x\n", funcs[i].offset);
+ printf("\n");
+
+ /*
+ * Function data offsets — byte offset into paraminfo_func_data
+ * for each function's parameter list.
+ */
+ printf("\t.globl paraminfo_func_offsets\n");
+ printf("\t.balign 4\n");
+ printf("paraminfo_func_offsets:\n");
+ for (i = 0; i < num_funcs; i++)
+ printf("\t.long .Lfunc_%u - paraminfo_func_data\n", i);
+ printf("\n");
+
+ /*
+ * Register strings in the string table and build the func data.
+ * Func data format per function:
+ * u8 num_params
+ * For each param:
+ * u32 name_str_offset
+ * u32 type_str_offset
+ */
+ /* First pass: register all strings */
+ for (i = 0; i < num_funcs; i++) {
+ for (j = 0; j < funcs[i].num_params; j++) {
+ find_or_add_string(funcs[i].params[j].name);
+ find_or_add_string(funcs[i].params[j].type);
+ }
+ }
+
+ /* Function parameter data */
+ printf("\t.globl paraminfo_func_data\n");
+ printf("paraminfo_func_data:\n");
+ for (i = 0; i < num_funcs; i++) {
+ printf(".Lfunc_%u:\n", i);
+ printf("\t.byte %u\n", funcs[i].num_params);
+ for (j = 0; j < funcs[i].num_params; j++) {
+ unsigned int name_idx = find_or_add_string(funcs[i].params[j].name);
+ unsigned int type_idx = find_or_add_string(funcs[i].params[j].type);
+
+ printf("\t.long %u, %u\n",
+ strtab[name_idx].offset,
+ strtab[type_idx].offset);
+ }
+ }
+ printf("\n");
+
+ /* String table */
+ printf("\t.globl paraminfo_strings\n");
+ printf("paraminfo_strings:\n");
+ for (i = 0; i < num_strings; i++)
+ print_escaped_asciz(strtab[i].str);
+ printf("\n");
+}
+
+int main(int argc, char *argv[])
+{
+ int fd;
+ Elf *elf;
+ Dwarf *dwarf;
+ unsigned long long text_addr;
+
+ if (argc != 2) {
+ fprintf(stderr, "Usage: %s <vmlinux ELF>\n", argv[0]);
+ return 1;
+ }
+
+ fd = open(argv[1], O_RDONLY);
+ if (fd < 0) {
+ fprintf(stderr, "Cannot open %s: %s\n", argv[1],
+ strerror(errno));
+ return 1;
+ }
+
+ elf_version(EV_CURRENT);
+ elf = elf_begin(fd, ELF_C_READ, NULL);
+ if (!elf) {
+ fprintf(stderr, "elf_begin failed: %s\n",
+ elf_errmsg(elf_errno()));
+ close(fd);
+ return 1;
+ }
+
+ text_addr = find_text_addr(elf);
+
+ dwarf = dwarf_begin_elf(elf, DWARF_C_READ, NULL);
+ if (!dwarf) {
+ fprintf(stderr, "dwarf_begin_elf failed: %s\n",
+ dwarf_errmsg(dwarf_errno()));
+ fprintf(stderr, "Is %s built with CONFIG_DEBUG_INFO?\n",
+ argv[1]);
+ elf_end(elf);
+ close(fd);
+ return 1;
+ }
+
+ process_dwarf(dwarf, text_addr);
+ deduplicate();
+
+ fprintf(stderr, "paraminfo: %u functions, %u strings\n",
+ num_funcs, num_strings);
+
+ output_assembly();
+
+ dwarf_end(dwarf);
+ elf_end(elf);
+ close(fd);
+
+ /* Cleanup */
+ free(funcs);
+ for (unsigned int i = 0; i < num_strings; i++)
+ free(strtab[i].str);
+ free(strtab);
+
+ return 0;
+}
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index 39ca44fbb259b..d41b31dde4ef9 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -103,7 +103,7 @@ vmlinux_link()
${ld} ${ldflags} -o ${output} \
${wl}--whole-archive ${objs} ${wl}--no-whole-archive \
${wl}--start-group ${libs} ${wl}--end-group \
- ${kallsymso} ${lineinfo_o} ${btf_vmlinux_bin_o} ${arch_vmlinux_o} ${ldlibs}
+ ${kallsymso} ${lineinfo_o} ${paraminfo_o} ${btf_vmlinux_bin_o} ${arch_vmlinux_o} ${ldlibs}
}
# Create ${2}.o file with all symbols from the ${1} object file
@@ -149,6 +149,26 @@ gen_lineinfo()
lineinfo_o=.tmp_lineinfo.o
}
+# Generate paraminfo tables from DWARF debug info in a temporary vmlinux.
+# ${1} - temporary vmlinux with debug info
+# Output: sets paraminfo_o to the generated .o file
+gen_paraminfo()
+{
+ info PARAMINFO .tmp_paraminfo.S
+ if ! scripts/gen_paraminfo "${1}" > .tmp_paraminfo.S; then
+ echo >&2 "Failed to generate paraminfo from ${1}"
+ echo >&2 "Try to disable CONFIG_KALLSYMS_PARAMINFO"
+ exit 1
+ fi
+
+ info AS .tmp_paraminfo.o
+ ${CC} ${NOSTDINC_FLAGS} ${LINUXINCLUDE} ${KBUILD_CPPFLAGS} \
+ ${KBUILD_AFLAGS} ${KBUILD_AFLAGS_KERNEL} \
+ -c -o .tmp_paraminfo.o .tmp_paraminfo.S
+
+ paraminfo_o=.tmp_paraminfo.o
+}
+
# Perform kallsyms for the given temporary vmlinux.
sysmap_and_kallsyms()
{
@@ -176,6 +196,7 @@ cleanup()
{
rm -f .btf.*
rm -f .tmp_lineinfo.*
+ rm -f .tmp_paraminfo.*
rm -f .tmp_vmlinux.nm-sort
rm -f System.map
rm -f vmlinux
@@ -205,6 +226,7 @@ btf_vmlinux_bin_o=
btfids_vmlinux=
kallsymso=
lineinfo_o=
+paraminfo_o=
strip_debug=
generate_map=
@@ -229,12 +251,22 @@ if is_enabled CONFIG_KALLSYMS_LINEINFO; then
lineinfo_o=.tmp_lineinfo.o
fi
+if is_enabled CONFIG_KALLSYMS_PARAMINFO; then
+ # Assemble an empty paraminfo stub for the initial link.
+ # The real paraminfo is generated from .tmp_vmlinux1 by gen_paraminfo.
+ ${CC} ${NOSTDINC_FLAGS} ${LINUXINCLUDE} ${KBUILD_CPPFLAGS} \
+ ${KBUILD_AFLAGS} ${KBUILD_AFLAGS_KERNEL} \
+ -c -o .tmp_paraminfo.o "${srctree}/scripts/empty_paraminfo.S"
+ paraminfo_o=.tmp_paraminfo.o
+fi
+
if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then
- # The kallsyms linking does not need debug symbols, but BTF and
- # lineinfo generation do.
+ # The kallsyms linking does not need debug symbols, but BTF,
+ # lineinfo and paraminfo generation do.
if ! is_enabled CONFIG_DEBUG_INFO_BTF &&
- ! is_enabled CONFIG_KALLSYMS_LINEINFO; then
+ ! is_enabled CONFIG_KALLSYMS_LINEINFO &&
+ ! is_enabled CONFIG_KALLSYMS_PARAMINFO; then
strip_debug=1
fi
@@ -256,6 +288,10 @@ if is_enabled CONFIG_KALLSYMS_LINEINFO; then
gen_lineinfo .tmp_vmlinux1
fi
+if is_enabled CONFIG_KALLSYMS_PARAMINFO; then
+ gen_paraminfo .tmp_vmlinux1
+fi
+
if is_enabled CONFIG_KALLSYMS; then
# kallsyms support
--
2.51.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH 2/2] kallsyms: add BTF-based deep parameter rendering in oops dumps
2026-03-23 16:48 [PATCH 0/2] kallsyms: show typed function parameters in oops/WARN dumps Sasha Levin
2026-03-23 16:48 ` [PATCH 1/2] kallsyms: show function parameter info " Sasha Levin
@ 2026-03-23 16:48 ` Sasha Levin
2026-03-24 15:07 ` Alexei Starovoitov
2026-03-23 18:43 ` [PATCH 0/2] kallsyms: show typed function parameters in oops/WARN dumps Alexey Dobriyan
` (2 subsequent siblings)
4 siblings, 1 reply; 18+ messages in thread
From: Sasha Levin @ 2026-03-23 16:48 UTC (permalink / raw)
To: Andrew Morton, Masahiro Yamada, Nathan Chancellor, Nicolas Schier
Cc: Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Peter Zijlstra, Josh Poimboeuf, Petr Mladek,
Alexei Starovoitov, Jonathan Corbet, David Gow, Kees Cook,
Greg KH, Luis Chamberlain, Steven Rostedt, Helge Deller,
Randy Dunlap, Geert Uytterhoeven, Juergen Gross, James Bottomley,
Alexey Dobriyan, Vlastimil Babka, Laurent Pinchart, Petr Pavlu,
x86, linux-kernel, linux-kbuild, linux-doc, linux-modules, bpf,
Sasha Levin
When CONFIG_KALLSYMS_PARAMINFO_BTF is enabled and a function parameter
is a pointer to a kernel struct, use BTF type information to safely
dereference the pointer and display struct member values in oops/WARN
dumps.
The rendering uses btf_type_snprintf_show() which internally uses
copy_from_kernel_nofault() for safe memory access, making it safe to
call in oops/panic context. Struct members are printed in a compact
two-column layout for readability.
Example output:
Function parameters (paraminfo_demo_crash):
file (struct file *) = 0xffff8bc043cb36c0
.f_mode = (fmode_t)67993630 .f_flags = (unsigned int)32769
.f_mapping = (struct address_space *)0x.. .f_inode = (struct inode *)0x..
.f_cred = (struct cred *)0x... .prev_pos = (loff_t)-1
Gated behind CONFIG_KALLSYMS_PARAMINFO_BTF which depends on both
CONFIG_KALLSYMS_PARAMINFO and CONFIG_DEBUG_INFO_BTF. No additional
kernel image size beyond BTF itself.
Assisted-by: Claude:claude-opus-4-6
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
init/Kconfig | 19 +++
kernel/Makefile | 1 +
kernel/kallsyms.c | 47 ++++--
kernel/kallsyms_paraminfo_btf.c | 267 ++++++++++++++++++++++++++++++++
lib/tests/paraminfo_kunit.c | 13 +-
scripts/gen_paraminfo.c | 80 ++++++++--
6 files changed, 384 insertions(+), 43 deletions(-)
create mode 100644 kernel/kallsyms_paraminfo_btf.c
diff --git a/init/Kconfig b/init/Kconfig
index 76d0c2da7d612..602594c86bf7e 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -2106,6 +2106,25 @@ config KALLSYMS_PARAMINFO
If unsure, say N.
+config KALLSYMS_PARAMINFO_BTF
+ bool "Render struct contents for pointer parameters in oops dumps"
+ depends on KALLSYMS_PARAMINFO && DEBUG_INFO_BTF
+ help
+ When a function parameter is a pointer to a kernel struct and BTF
+ type information is available, dereference the pointer and display
+ key struct members (1 level deep) in oops/WARN dumps.
+
+ When enabled, oops dumps may include additional indented lines
+ showing struct member values in a two-column layout:
+
+ file (struct file *) = 0xffff888123456000
+ .f_flags = (unsigned int)32769 .f_mode = (fmode_t)29
+
+ Requires CONFIG_DEBUG_INFO_BTF.
+ No additional kernel image size beyond BTF itself.
+
+ If unsure, say N.
+
# end of the "standard kernel features (expert users)" menu
config ARCH_HAS_MEMBARRIER_CALLBACKS
diff --git a/kernel/Makefile b/kernel/Makefile
index 6785982013dce..e47d911340cad 100644
--- a/kernel/Makefile
+++ b/kernel/Makefile
@@ -75,6 +75,7 @@ obj-$(CONFIG_UID16) += uid16.o
obj-$(CONFIG_MODULE_SIG_FORMAT) += module_signature.o
obj-$(CONFIG_KALLSYMS) += kallsyms.o
obj-$(CONFIG_KALLSYMS_SELFTEST) += kallsyms_selftest.o
+obj-$(CONFIG_KALLSYMS_PARAMINFO_BTF) += kallsyms_paraminfo_btf.o
obj-$(CONFIG_BSD_PROCESS_ACCT) += acct.o
obj-$(CONFIG_VMCORE_INFO) += vmcore_info.o elfcorehdr.o
obj-$(CONFIG_CRASH_RESERVE) += crash_reserve.o
diff --git a/kernel/kallsyms.c b/kernel/kallsyms.c
index af8de3d8e3ba3..362cfa80b2c08 100644
--- a/kernel/kallsyms.c
+++ b/kernel/kallsyms.c
@@ -510,16 +510,19 @@ bool kallsyms_lookup_lineinfo(unsigned long addr,
#define MAX_PARAMINFO_PARAMS 6
+#ifdef CONFIG_KALLSYMS_PARAMINFO_BTF
+void paraminfo_btf_show_ptr(unsigned long ptr_val, const char *type_str);
+#else
+static inline void paraminfo_btf_show_ptr(unsigned long ptr_val,
+ const char *type_str) {}
+#endif
+
/*
* x86-64 calling convention: arguments are passed in registers
* RDI, RSI, RDX, RCX, R8, R9 (in that order).
*/
#ifdef CONFIG_X86_64
-static const char * const paraminfo_reg_names[] = {
- "RDI", "RSI", "RDX", "RCX", "R8", "R9"
-};
-
static unsigned long paraminfo_get_reg(const struct pt_regs *regs,
unsigned int idx)
{
@@ -534,9 +537,6 @@ static unsigned long paraminfo_get_reg(const struct pt_regs *regs,
}
}
#else
-/* Stub for non-x86-64 architectures */
-static const char * const paraminfo_reg_names[] = {};
-
static unsigned long paraminfo_get_reg(const struct pt_regs *regs,
unsigned int idx)
{
@@ -586,13 +586,13 @@ void kallsyms_show_paraminfo(struct pt_regs *regs)
const u8 *data;
unsigned int num_params, i;
unsigned long ip, fault_addr;
- char sym_name[KSYM_NAME_LEN];
+ char sym_name[128];
unsigned long sym_size, sym_offset;
if (!regs || !paraminfo_num_funcs)
return;
- ip = regs->ip;
+ ip = instruction_pointer(regs);
/* Only handle kernel-mode faults */
if (user_mode(regs))
@@ -611,14 +611,23 @@ void kallsyms_show_paraminfo(struct pt_regs *regs)
return;
/*
- * Verify the IP is within a reasonable range of the function
- * start. paraminfo_func_addrs[] contains function start offsets;
- * check that we're not too far past the start. Use kallsyms to
- * verify we're in the right function.
+ * Verify the paraminfo entry actually matches the function
+ * containing the IP. Without this, if the faulting function
+ * has no paraminfo, the binary search silently returns the
+ * preceding function's entry — showing wrong parameter info.
*/
if (!kallsyms_lookup_size_offset(ip, &sym_size, &sym_offset))
return;
+ {
+ unsigned int func_start_offset;
+
+ func_start_offset = (unsigned int)(ip - sym_offset -
+ (unsigned long)_text);
+ if (paraminfo_func_addrs[func_idx] != func_start_offset)
+ return;
+ }
+
/* Decode the function's parameter data */
data = paraminfo_func_data + paraminfo_func_offsets[func_idx];
num_params = *data++;
@@ -631,9 +640,9 @@ void kallsyms_show_paraminfo(struct pt_regs *regs)
return;
/*
- * Read the fault address for highlighting. On x86, CR2 holds
- * the page fault linear address. On other architectures this
- * would need a different mechanism.
+ * Read CR2 for fault address highlighting. CR2 is only meaningful
+ * for page faults; for GPF, BUG, WARN, etc. it may hold a stale
+ * value. This is best-effort — a false match is harmless.
*/
#ifdef CONFIG_X86
fault_addr = read_cr2();
@@ -660,9 +669,13 @@ void kallsyms_show_paraminfo(struct pt_regs *regs)
is_fault_addr = fault_addr && (val == fault_addr);
- printk(KERN_DEFAULT " %-8s (%-20s) = 0x%016lx%s\n",
+ printk(KERN_DEFAULT " %-8s (%-20s) = 0x%016lx%s\n",
pname, ptype, val,
is_fault_addr ? " <-- fault address" : "");
+
+ /* If this is a pointer to a struct, try BTF deep rendering */
+ if (val && strstr(ptype, "*"))
+ paraminfo_btf_show_ptr(val, ptype);
}
}
EXPORT_SYMBOL_GPL(kallsyms_show_paraminfo);
diff --git a/kernel/kallsyms_paraminfo_btf.c b/kernel/kallsyms_paraminfo_btf.c
new file mode 100644
index 0000000000000..28ce1dd45f7a8
--- /dev/null
+++ b/kernel/kallsyms_paraminfo_btf.c
@@ -0,0 +1,267 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * kallsyms_paraminfo_btf.c - BTF-based deep rendering for paraminfo
+ *
+ * Copyright (C) 2026 Sasha Levin <sashal@kernel.org>
+ *
+ * When CONFIG_KALLSYMS_PARAMINFO_BTF is enabled and a function parameter
+ * is a pointer to a kernel struct, this module uses BTF type information
+ * to safely dereference the pointer and display key struct members in
+ * oops/WARN dumps.
+ */
+
+#include <linux/btf.h>
+#include <linux/bpf.h>
+#include <linux/kallsyms.h>
+#include <linux/kernel.h>
+#include <linux/printk.h>
+#include <linux/string.h>
+#include <linux/uaccess.h>
+
+/* Declared in kernel/kallsyms.c under CONFIG_KALLSYMS_PARAMINFO_BTF */
+void paraminfo_btf_show_ptr(unsigned long ptr_val, const char *type_str);
+
+extern struct btf *btf_vmlinux;
+
+/*
+ * Maximum output buffer for BTF rendering. Large structs (e.g.,
+ * struct dentry) need substantial space. This is declared static
+ * rather than on the stack because 4096 bytes would exceed the
+ * frame size limit. Oops context is effectively single-threaded
+ * (other CPUs are stopped or spinning), so a static buffer is safe.
+ */
+#define BTF_SHOW_BUF_LEN 4096
+
+/*
+ * Parse a type string like "struct file *" to extract the struct name.
+ * Writes into caller-provided @name_buf of size @bufsz.
+ * Returns @name_buf on success, or NULL if not a struct/union pointer type.
+ */
+static const char *extract_struct_name(const char *type_str, bool *is_union,
+ char *name_buf, size_t bufsz)
+{
+ const char *p, *end;
+
+ *is_union = false;
+
+ /* Must end with " *" to be a pointer */
+ end = type_str + strlen(type_str);
+ if (end - type_str < 3 || end[-1] != '*' || end[-2] != ' ')
+ return NULL;
+
+ if (!strncmp(type_str, "struct ", 7)) {
+ p = type_str + 7;
+ } else if (!strncmp(type_str, "union ", 6)) {
+ p = type_str + 6;
+ *is_union = true;
+ } else {
+ return NULL;
+ }
+
+ /* Copy name up to the " *" */
+ {
+ size_t len = (end - 2) - p;
+
+ if (len == 0 || len >= bufsz)
+ return NULL;
+ memcpy(name_buf, p, len);
+ name_buf[len] = '\0';
+ }
+
+ return name_buf;
+}
+
+/*
+ * Show BTF-rendered struct contents for a pointer parameter.
+ * Called from kallsyms_show_paraminfo() when the parameter is a
+ * struct/union pointer.
+ *
+ * Uses btf_type_snprintf_show() which internally uses
+ * copy_from_kernel_nofault() for safe memory access, making it
+ * safe to call in oops/panic context.
+ */
+void paraminfo_btf_show_ptr(unsigned long ptr_val, const char *type_str)
+{
+ static char buf[BTF_SHOW_BUF_LEN];
+ char sname[64];
+ const char *name;
+ bool is_union;
+ s32 type_id;
+ int ret;
+
+ if (!btf_vmlinux || !ptr_val)
+ return;
+
+ /* Only handle kernel pointers */
+ if (ptr_val < PAGE_OFFSET)
+ return;
+
+ name = extract_struct_name(type_str, &is_union, sname, sizeof(sname));
+ if (!name)
+ return;
+
+ type_id = btf_find_by_name_kind(btf_vmlinux, name,
+ is_union ? BTF_KIND_UNION
+ : BTF_KIND_STRUCT);
+ if (type_id < 0)
+ return;
+
+ /*
+ * Render without BTF_SHOW_COMPACT so each member gets its own
+ * line with proper indentation from BTF. Use BTF_SHOW_PTR_RAW
+ * to print real kernel addresses instead of hashed pointers —
+ * this is oops context where address visibility is critical.
+ */
+ ret = btf_type_snprintf_show(btf_vmlinux, type_id, (void *)ptr_val,
+ buf, sizeof(buf), BTF_SHOW_PTR_RAW);
+ if (ret < 0)
+ return;
+
+ buf[sizeof(buf) - 1] = '\0';
+
+ /*
+ * Filter the multi-line BTF output: skip lines that contain
+ * only braces/brackets/whitespace (structural noise), collect
+ * meaningful member lines, and print them two per row.
+ */
+ {
+ /*
+ * Collect filtered lines as pointers into buf[] (which
+ * we NUL-terminate in place). Stack budget: ~64 pointers.
+ */
+#define MAX_BTF_LINES 64
+#define BTF_COL_WIDTH 40
+ char *lines[MAX_BTF_LINES];
+ int nlines = 0;
+ char *line, *next;
+
+ for (line = buf; line && *line; line = next) {
+ char *s, *c;
+ bool has_content = false;
+
+ next = strchr(line, '\n');
+ if (next)
+ *next++ = '\0';
+
+ s = line;
+ while (*s == ' ' || *s == '\t')
+ s++;
+
+ /* Skip structural-only lines: {}[](), */
+ for (c = s; *c; c++) {
+ if (*c != '{' && *c != '}' &&
+ *c != '(' && *c != ')' &&
+ *c != '[' && *c != ']' &&
+ *c != ',' && *c != ' ' &&
+ *c != '\t') {
+ has_content = true;
+ break;
+ }
+ }
+ if (!has_content)
+ continue;
+
+ /* Skip type-only prefix lines */
+ if (*s == '(' && !strchr(s, '=') && !strchr(s, '['))
+ continue;
+
+ /* Trim trailing commas/spaces */
+ {
+ size_t len = strlen(s);
+
+ while (len > 0 && (s[len - 1] == ','
+ || s[len - 1] == ' '))
+ s[--len] = '\0';
+ }
+
+ if (*s && nlines < MAX_BTF_LINES)
+ lines[nlines++] = s;
+ }
+
+ /*
+ * Coalesce char array elements into strings.
+ *
+ * BTF renders char[] as individual elements:
+ * .sysname = (char[])[
+ * 'L'
+ * 'i'
+ * 'n' ...
+ *
+ * Detect lines containing "= (char[])[" or
+ * "= (unsigned char[])[" and collect following
+ * single-quoted-char lines into a readable string
+ * like: .sysname = "Linux"
+ */
+#define MAX_COALESCED 8
+ {
+ static char coalesced[MAX_COALESCED][128];
+ int ci = 0, i;
+
+ for (i = 0; i < nlines && ci < MAX_COALESCED; i++) {
+ char *eq;
+ int spos, j, pfxlen;
+
+ eq = strstr(lines[i], "(char[])[");
+ if (!eq)
+ eq = strstr(lines[i], "(unsigned char[])[");
+ if (!eq)
+ continue;
+
+ /* Extract prefix up to and including '=' */
+ eq = strstr(lines[i], "= ");
+ if (!eq)
+ continue;
+ pfxlen = eq - lines[i] + 2;
+ if (pfxlen > 60)
+ pfxlen = 60;
+
+ memcpy(coalesced[ci], lines[i], pfxlen);
+ coalesced[ci][pfxlen] = '"';
+ spos = pfxlen + 1;
+
+ /* Gather chars from subsequent lines */
+ for (j = i + 1; j < nlines &&
+ spos < (int)sizeof(coalesced[0]) - 2; j++) {
+ char *s = lines[j];
+
+ if (s[0] == '\'' && s[2] == '\'' &&
+ (s[3] == '\0' || s[3] == ',')) {
+ coalesced[ci][spos++] = s[1];
+ lines[j] = "";
+ } else {
+ break;
+ }
+ }
+ coalesced[ci][spos++] = '"';
+ coalesced[ci][spos] = '\0';
+ lines[i] = coalesced[ci];
+ ci++;
+ }
+ }
+#undef MAX_COALESCED
+
+ /* Print in two columns, skipping empty (consumed) lines */
+ {
+ int i, col = 0;
+ char *pending = NULL;
+
+ for (i = 0; i < nlines; i++) {
+ if (!lines[i][0])
+ continue;
+ if (col == 0) {
+ pending = lines[i];
+ col = 1;
+ } else {
+ printk(KERN_DEFAULT " %-*s %s\n",
+ BTF_COL_WIDTH, pending,
+ lines[i]);
+ col = 0;
+ }
+ }
+ if (col == 1)
+ printk(KERN_DEFAULT " %s\n", pending);
+ }
+#undef MAX_BTF_LINES
+#undef BTF_COL_WIDTH
+ }
+}
diff --git a/lib/tests/paraminfo_kunit.c b/lib/tests/paraminfo_kunit.c
index e09efc4ddeb0e..74a4436163a98 100644
--- a/lib/tests/paraminfo_kunit.c
+++ b/lib/tests/paraminfo_kunit.c
@@ -7,7 +7,8 @@
* Verifies that the paraminfo tables correctly map function addresses
* to their parameter names and types.
*
- * Build with: CONFIG_PARAMINFO_KUNIT_TEST=m (or =y)
+ * Build with: CONFIG_PARAMINFO_KUNIT_TEST=y (must be built-in; paraminfo
+ * tables are vmlinux-only, so module test functions won't be found)
*/
#include <kunit/test.h>
@@ -45,15 +46,7 @@ static noinline void paraminfo_test_no_args(void)
/* ---- Helpers to query paraminfo tables directly ---- */
-/*
- * These access the raw paraminfo tables to verify correctness.
- * The tables are defined in kernel/kallsyms_internal.h.
- */
-extern const u32 paraminfo_num_funcs;
-extern const u32 paraminfo_func_addrs[];
-extern const u32 paraminfo_func_offsets[];
-extern const u8 paraminfo_func_data[];
-extern const char paraminfo_strings[];
+#include "../../kernel/kallsyms_internal.h"
struct param_result {
unsigned int num_params;
diff --git a/scripts/gen_paraminfo.c b/scripts/gen_paraminfo.c
index ea1d23f3ddd9a..b64dd1232c77c 100644
--- a/scripts/gen_paraminfo.c
+++ b/scripts/gen_paraminfo.c
@@ -58,7 +58,7 @@ static unsigned int num_strings;
static unsigned int strtab_capacity;
static unsigned int strtab_total_size;
-#define STR_HASH_BITS 14
+#define STR_HASH_BITS 18
#define STR_HASH_SIZE (1 << STR_HASH_BITS)
struct str_hash_entry {
@@ -87,6 +87,13 @@ static unsigned int find_or_add_string(const char *s)
h = (h + 1) & (STR_HASH_SIZE - 1);
}
+ if (num_strings >= STR_HASH_SIZE * 3 / 4) {
+ fprintf(stderr,
+ "gen_paraminfo: string hash table overflow (%u entries)\n",
+ num_strings);
+ exit(1);
+ }
+
if (num_strings >= strtab_capacity) {
strtab_capacity = strtab_capacity ? strtab_capacity * 2 : 8192;
strtab = realloc(strtab, strtab_capacity * sizeof(*strtab));
@@ -97,6 +104,10 @@ static unsigned int find_or_add_string(const char *s)
}
strtab[num_strings].str = strdup(s);
+ if (!strtab[num_strings].str) {
+ fprintf(stderr, "out of memory\n");
+ exit(1);
+ }
strtab[num_strings].offset = strtab_total_size;
strtab_total_size += strlen(s) + 1;
@@ -120,20 +131,32 @@ static void add_func(struct func_entry *f)
funcs[num_funcs++] = *f;
}
+/* Max recursion depth to prevent stack overflow on pathological DWARF */
+#define MAX_TYPE_DEPTH 16
+
+static void __build_type_name(Dwarf_Die *type_die, char *buf, size_t bufsz,
+ int depth);
+
/*
* Build a human-readable type name string from a DWARF type DIE.
* Follows the type chain (pointers, const, etc.) to produce strings like:
* "struct file *", "const char *", "unsigned long", "void *"
*/
static void build_type_name(Dwarf_Die *type_die, char *buf, size_t bufsz)
+{
+ __build_type_name(type_die, buf, bufsz, 0);
+}
+
+static void __build_type_name(Dwarf_Die *type_die, char *buf, size_t bufsz,
+ int depth)
{
Dwarf_Die child;
Dwarf_Attribute attr;
const char *name;
int tag;
- if (!type_die) {
- snprintf(buf, bufsz, "void");
+ if (!type_die || depth > MAX_TYPE_DEPTH) {
+ snprintf(buf, bufsz, depth > MAX_TYPE_DEPTH ? "..." : "void");
return;
}
@@ -148,7 +171,7 @@ static void build_type_name(Dwarf_Die *type_die, char *buf, size_t bufsz)
case DW_TAG_pointer_type:
if (dwarf_attr(type_die, DW_AT_type, &attr) &&
dwarf_formref_die(&attr, &child)) {
- build_type_name(&child, buf, bufsz);
+ __build_type_name(&child, buf, bufsz, depth + 1);
if (strlen(buf) + 3 < bufsz)
strcat(buf, " *");
} else {
@@ -161,7 +184,7 @@ static void build_type_name(Dwarf_Die *type_die, char *buf, size_t bufsz)
dwarf_formref_die(&attr, &child)) {
char tmp[MAX_TYPE_LEN - 10];
- build_type_name(&child, tmp, sizeof(tmp));
+ __build_type_name(&child, tmp, sizeof(tmp), depth + 1);
snprintf(buf, bufsz, "const %s", tmp);
} else {
snprintf(buf, bufsz, "const void");
@@ -173,7 +196,7 @@ static void build_type_name(Dwarf_Die *type_die, char *buf, size_t bufsz)
dwarf_formref_die(&attr, &child)) {
char tmp[MAX_TYPE_LEN - 10];
- build_type_name(&child, tmp, sizeof(tmp));
+ __build_type_name(&child, tmp, sizeof(tmp), depth + 1);
snprintf(buf, bufsz, "volatile %s", tmp);
} else {
snprintf(buf, bufsz, "volatile void");
@@ -183,7 +206,7 @@ static void build_type_name(Dwarf_Die *type_die, char *buf, size_t bufsz)
case DW_TAG_restrict_type:
if (dwarf_attr(type_die, DW_AT_type, &attr) &&
dwarf_formref_die(&attr, &child)) {
- build_type_name(&child, buf, bufsz);
+ __build_type_name(&child, buf, bufsz, depth + 1);
} else {
snprintf(buf, bufsz, "void");
}
@@ -195,7 +218,7 @@ static void build_type_name(Dwarf_Die *type_die, char *buf, size_t bufsz)
snprintf(buf, bufsz, "%s", name);
} else if (dwarf_attr(type_die, DW_AT_type, &attr) &&
dwarf_formref_die(&attr, &child)) {
- build_type_name(&child, buf, bufsz);
+ __build_type_name(&child, buf, bufsz, depth + 1);
} else {
snprintf(buf, bufsz, "?");
}
@@ -219,7 +242,7 @@ static void build_type_name(Dwarf_Die *type_die, char *buf, size_t bufsz)
case DW_TAG_array_type:
if (dwarf_attr(type_die, DW_AT_type, &attr) &&
dwarf_formref_die(&attr, &child)) {
- build_type_name(&child, buf, bufsz);
+ __build_type_name(&child, buf, bufsz, depth + 1);
if (strlen(buf) + 3 < bufsz)
strcat(buf, "[]");
} else {
@@ -231,8 +254,23 @@ static void build_type_name(Dwarf_Die *type_die, char *buf, size_t bufsz)
snprintf(buf, bufsz, "func_ptr");
break;
+ case DW_TAG_unspecified_type:
+ name = dwarf_diename(type_die);
+ snprintf(buf, bufsz, "%s", name ? name : "void");
+ break;
+
default:
- snprintf(buf, bufsz, "?");
+ /*
+ * Unknown tag — try to follow DW_AT_type if present
+ * (handles DW_TAG_atomic_type and others).
+ */
+ if (dwarf_attr(type_die, DW_AT_type, &attr) &&
+ dwarf_formref_die(&attr, &child)) {
+ __build_type_name(&child, buf, bufsz, depth + 1);
+ } else {
+ name = dwarf_diename(type_die);
+ snprintf(buf, bufsz, "%s", name ? name : "?");
+ }
break;
}
}
@@ -311,10 +349,15 @@ static void process_dwarf(Dwarf *dwarf, unsigned long long text_addr)
continue;
/* Skip declarations (no body) */
- if (dwarf_attr(&child, DW_AT_declaration, &attr))
+ if (dwarf_attr_integrate(&child, DW_AT_declaration, &attr))
continue;
- /* Get function start address */
+ /*
+ * Get function start address.
+ * dwarf_lowpc handles DW_AT_low_pc directly, but
+ * for concrete inlined instances the address may be
+ * in a ranges table.
+ */
if (dwarf_lowpc(&child, &low_pc) != 0)
continue;
@@ -341,6 +384,12 @@ static void process_dwarf(Dwarf *dwarf, unsigned long long text_addr)
if (func.num_params >= MAX_PARAMS)
break;
+ /*
+ * Use dwarf_attr_integrate to follow
+ * DW_AT_abstract_origin chains — inlined
+ * or outlined functions may store param
+ * names/types in an abstract instance.
+ */
pname = dwarf_diename(¶m);
if (!pname)
pname = "?";
@@ -349,8 +398,8 @@ static void process_dwarf(Dwarf *dwarf, unsigned long long text_addr)
sizeof(func.params[0].name),
"%s", pname);
- /* Resolve type */
- if (dwarf_attr(¶m, DW_AT_type, &attr) &&
+ /* Resolve type (follow abstract origin) */
+ if (dwarf_attr_integrate(¶m, DW_AT_type, &attr) &&
dwarf_formref_die(&attr, &type_die)) {
build_type_name(&type_die,
func.params[func.num_params].type,
@@ -530,8 +579,7 @@ int main(int argc, char *argv[])
process_dwarf(dwarf, text_addr);
deduplicate();
- fprintf(stderr, "paraminfo: %u functions, %u strings\n",
- num_funcs, num_strings);
+ fprintf(stderr, "paraminfo: %u functions\n", num_funcs);
output_assembly();
--
2.51.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [PATCH 0/2] kallsyms: show typed function parameters in oops/WARN dumps
2026-03-23 16:48 [PATCH 0/2] kallsyms: show typed function parameters in oops/WARN dumps Sasha Levin
2026-03-23 16:48 ` [PATCH 1/2] kallsyms: show function parameter info " Sasha Levin
2026-03-23 16:48 ` [PATCH 2/2] kallsyms: add BTF-based deep parameter rendering in oops dumps Sasha Levin
@ 2026-03-23 18:43 ` Alexey Dobriyan
2026-03-23 22:58 ` Sasha Levin
2026-03-23 22:50 ` Andrew Morton
2026-03-24 8:57 ` Jiri Olsa
4 siblings, 1 reply; 18+ messages in thread
From: Alexey Dobriyan @ 2026-03-23 18:43 UTC (permalink / raw)
To: Sasha Levin
Cc: Andrew Morton, Masahiro Yamada, Nathan Chancellor, Nicolas Schier,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Peter Zijlstra, Josh Poimboeuf, Petr Mladek,
Alexei Starovoitov, Jonathan Corbet, David Gow, Kees Cook,
Greg KH, Luis Chamberlain, Steven Rostedt, Helge Deller,
Randy Dunlap, Geert Uytterhoeven, Juergen Gross, James Bottomley,
Vlastimil Babka, Laurent Pinchart, Petr Pavlu, x86, linux-kernel,
linux-kbuild, linux-doc, linux-modules, bpf
On Mon, Mar 23, 2026 at 12:48:55PM -0400, Sasha Levin wrote:
> Function parameters (paraminfo_demo_crash):
> uts (struct new_utsname *) = 0xffffffffb8ca8d00
> .sysname = "Linux" .nodename = "localhost"
> .release = "7.0.0-rc2-00006-g3190..." .version = "#45 SMP PRE"
> file (struct file * ) = 0xffffa0a3c250acc0
> .f_mode = (fmode_t)67993630 .f_op = (struct file_operations *)0xffffffffb7237620
> .f_flags = (unsigned int)32769 .f_cred = (struct cred *)0xffffa0a3c2e06a80
> .dentry = (struct dentry *)0xffffa0a3c0978cc0
Should this be in crash's format?
struct dentry ffffffffffff0000
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 0/2] kallsyms: show typed function parameters in oops/WARN dumps
2026-03-23 16:48 [PATCH 0/2] kallsyms: show typed function parameters in oops/WARN dumps Sasha Levin
` (2 preceding siblings ...)
2026-03-23 18:43 ` [PATCH 0/2] kallsyms: show typed function parameters in oops/WARN dumps Alexey Dobriyan
@ 2026-03-23 22:50 ` Andrew Morton
2026-03-23 23:08 ` Sasha Levin
2026-03-24 8:57 ` Jiri Olsa
4 siblings, 1 reply; 18+ messages in thread
From: Andrew Morton @ 2026-03-23 22:50 UTC (permalink / raw)
To: Sasha Levin
Cc: Masahiro Yamada, Nathan Chancellor, Nicolas Schier,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Peter Zijlstra, Josh Poimboeuf, Petr Mladek,
Alexei Starovoitov, Jonathan Corbet, David Gow, Kees Cook,
Greg KH, Luis Chamberlain, Steven Rostedt, Helge Deller,
Randy Dunlap, Geert Uytterhoeven, Juergen Gross, James Bottomley,
Alexey Dobriyan, Vlastimil Babka, Laurent Pinchart, Petr Pavlu,
x86, linux-kernel, linux-kbuild, linux-doc, linux-modules, bpf
On Mon, 23 Mar 2026 12:48:55 -0400 Sasha Levin <sashal@kernel.org> wrote:
> Building on the lineinfo series, this adds typed function parameter
> display to oops and WARN dumps. A build-time tool extracts parameter
> names and types from DWARF, and the kernel maps pt_regs to the calling
> convention at crash time. When BTF is available, struct pointer
> parameters are dereferenced and their members displayed.
mm.git is full and I'm seriously looking at loadshedded. Can we please
leave this until next cycle, give your "kallsyms: embed source file:line
info in kernel stack traces", v4 time to settle in?
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 0/2] kallsyms: show typed function parameters in oops/WARN dumps
2026-03-23 18:43 ` [PATCH 0/2] kallsyms: show typed function parameters in oops/WARN dumps Alexey Dobriyan
@ 2026-03-23 22:58 ` Sasha Levin
0 siblings, 0 replies; 18+ messages in thread
From: Sasha Levin @ 2026-03-23 22:58 UTC (permalink / raw)
To: Alexey Dobriyan
Cc: Andrew Morton, Masahiro Yamada, Nathan Chancellor, Nicolas Schier,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Peter Zijlstra, Josh Poimboeuf, Petr Mladek,
Alexei Starovoitov, Jonathan Corbet, David Gow, Kees Cook,
Greg KH, Luis Chamberlain, Steven Rostedt, Helge Deller,
Randy Dunlap, Geert Uytterhoeven, Juergen Gross, James Bottomley,
Vlastimil Babka, Laurent Pinchart, Petr Pavlu, x86, linux-kernel,
linux-kbuild, linux-doc, linux-modules, bpf
On Mon, Mar 23, 2026 at 09:43:59PM +0300, Alexey Dobriyan wrote:
>On Mon, Mar 23, 2026 at 12:48:55PM -0400, Sasha Levin wrote:
>> Function parameters (paraminfo_demo_crash):
>> uts (struct new_utsname *) = 0xffffffffb8ca8d00
>> .sysname = "Linux" .nodename = "localhost"
>> .release = "7.0.0-rc2-00006-g3190..." .version = "#45 SMP PRE"
>> file (struct file * ) = 0xffffa0a3c250acc0
>> .f_mode = (fmode_t)67993630 .f_op = (struct file_operations *)0xffffffffb7237620
>> .f_flags = (unsigned int)32769 .f_cred = (struct cred *)0xffffa0a3c2e06a80
>> .dentry = (struct dentry *)0xffffa0a3c0978cc0
>
>Should this be in crash's format?
>
> struct dentry ffffffffffff0000
The format currently used comes from the kernel's own BTF show infrastructure.
crash's struct dentry ffffffffffff0000 syntax is specific to crash. drgn, GDB,
bpftool, and the kernel's own BTF rendering all use the (struct type *)0xaddr
notation we're already using. Adopting crash's format would make this output
inconsistent with all other BTF-based output in the kernel, and would also lose
the member name context (.dentry).
--
Thanks,
Sasha
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 0/2] kallsyms: show typed function parameters in oops/WARN dumps
2026-03-23 22:50 ` Andrew Morton
@ 2026-03-23 23:08 ` Sasha Levin
0 siblings, 0 replies; 18+ messages in thread
From: Sasha Levin @ 2026-03-23 23:08 UTC (permalink / raw)
To: Andrew Morton
Cc: Masahiro Yamada, Nathan Chancellor, Nicolas Schier,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Peter Zijlstra, Josh Poimboeuf, Petr Mladek,
Alexei Starovoitov, Jonathan Corbet, David Gow, Kees Cook,
Greg KH, Luis Chamberlain, Steven Rostedt, Helge Deller,
Randy Dunlap, Geert Uytterhoeven, Juergen Gross, James Bottomley,
Alexey Dobriyan, Vlastimil Babka, Laurent Pinchart, Petr Pavlu,
x86, linux-kernel, linux-kbuild, linux-doc, linux-modules, bpf
On Mon, Mar 23, 2026 at 03:50:57PM -0700, Andrew Morton wrote:
>On Mon, 23 Mar 2026 12:48:55 -0400 Sasha Levin <sashal@kernel.org> wrote:
>
>> Building on the lineinfo series, this adds typed function parameter
>> display to oops and WARN dumps. A build-time tool extracts parameter
>> names and types from DWARF, and the kernel maps pt_regs to the calling
>> convention at crash time. When BTF is available, struct pointer
>> parameters are dereferenced and their members displayed.
>
>mm.git is full and I'm seriously looking at loadshedded. Can we please
>leave this until next cycle, give your "kallsyms: embed source file:line
>info in kernel stack traces", v4 time to settle in?
Definitely.
If you want to ignore these, and related patches that might appear in the next
couple of weeks, I'll plan to send them to you in an organized series after
-rc1.
--
Thanks,
Sasha
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 0/2] kallsyms: show typed function parameters in oops/WARN dumps
2026-03-23 16:48 [PATCH 0/2] kallsyms: show typed function parameters in oops/WARN dumps Sasha Levin
` (3 preceding siblings ...)
2026-03-23 22:50 ` Andrew Morton
@ 2026-03-24 8:57 ` Jiri Olsa
2026-03-24 11:39 ` Sasha Levin
4 siblings, 1 reply; 18+ messages in thread
From: Jiri Olsa @ 2026-03-24 8:57 UTC (permalink / raw)
To: Sasha Levin
Cc: Andrew Morton, Masahiro Yamada, Nathan Chancellor, Nicolas Schier,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Peter Zijlstra, Josh Poimboeuf, Petr Mladek,
Alexei Starovoitov, Jonathan Corbet, David Gow, Kees Cook,
Greg KH, Luis Chamberlain, Steven Rostedt, Helge Deller,
Randy Dunlap, Geert Uytterhoeven, Juergen Gross, James Bottomley,
Alexey Dobriyan, Vlastimil Babka, Laurent Pinchart, Petr Pavlu,
x86, linux-kernel, linux-kbuild, linux-doc, linux-modules, bpf
On Mon, Mar 23, 2026 at 12:48:55PM -0400, Sasha Levin wrote:
> Building on the lineinfo series, this adds typed function parameter
hi,
could you please specify the exact tree/commit and point to the
series this patchset is based on?
thanks,
jirka
> display to oops and WARN dumps. A build-time tool extracts parameter
> names and types from DWARF, and the kernel maps pt_regs to the calling
> convention at crash time. When BTF is available, struct pointer
> parameters are dereferenced and their members displayed.
>
> Example output from a WARN in a function receiving struct new_utsname *
> (kernel version info) and struct file * parameters:
>
> ------------[ cut here ]------------
> WARNING: drivers/tty/sysrq.c:1209 at demo_crash+0xf/0x20 (drivers/tty/sysrq.c:1209)
> CPU: 2 UID: 0 PID: 323 Comm: bash
> RIP: 0010:demo_crash+0xf/0x20 (drivers/tty/sysrq.c:1209)
> ...
> RDI: ffffffffb8ca8d00
> RSI: ffffa0a3c250acc0
> ...
> Function parameters (paraminfo_demo_crash):
> uts (struct new_utsname *) = 0xffffffffb8ca8d00
> .sysname = "Linux" .nodename = "localhost"
> .release = "7.0.0-rc2-00006-g3190..." .version = "#45 SMP PRE"
> file (struct file * ) = 0xffffa0a3c250acc0
> .f_mode = (fmode_t)67993630 .f_op = (struct file_operations *)0xffffffffb7237620
> .f_flags = (unsigned int)32769 .f_cred = (struct cred *)0xffffa0a3c2e06a80
> .dentry = (struct dentry *)0xffffa0a3c0978cc0
> .prev_pos = (loff_t)-1
> Call Trace:
> <TASK>
> write_sysrq_trigger+0x96/0xb0 (drivers/tty/sysrq.c:1222)
> proc_reg_write+0x54/0xa0 (fs/proc/inode.c:330)
> vfs_write+0xc9/0x480 (fs/read_write.c:686)
> ksys_write+0x6e/0xe0 (fs/read_write.c:738)
> do_syscall_64+0xe2/0x570 (arch/x86/entry/syscall_64.c:62)
> entry_SYSCALL_64_after_hwframe+0x77/0x7f (arch/x86/entry/entry_64.S:121)
>
> Patch 1 adds the core paraminfo infrastructure (DWARF extraction,
> kernel-side lookup, register-to-parameter mapping, ~1-2 MB overhead).
> Patch 2 adds optional BTF-based struct rendering, gated behind
> CONFIG_KALLSYMS_PARAMINFO_BTF.
>
> Sasha Levin (2):
> kallsyms: show function parameter info in oops/WARN dumps
> kallsyms: add BTF-based deep parameter rendering in oops dumps
>
> .../admin-guide/kallsyms-lineinfo.rst | 31 +
> arch/x86/kernel/dumpstack.c | 6 +-
> include/linux/kallsyms.h | 9 +
> init/Kconfig | 40 ++
> kernel/Makefile | 1 +
> kernel/kallsyms.c | 182 ++++++
> kernel/kallsyms_internal.h | 6 +
> kernel/kallsyms_paraminfo_btf.c | 199 ++++++
> lib/Kconfig.debug | 11 +
> lib/tests/Makefile | 3 +
> lib/tests/paraminfo_kunit.c | 249 ++++++++
> scripts/Makefile | 3 +
> scripts/empty_paraminfo.S | 18 +
> scripts/gen_paraminfo.c | 597 ++++++++++++++++++
> scripts/link-vmlinux.sh | 44 +-
> 15 files changed, 1393 insertions(+), 6 deletions(-)
> create mode 100644 kernel/kallsyms_paraminfo_btf.c
> create mode 100644 lib/tests/paraminfo_kunit.c
> create mode 100644 scripts/empty_paraminfo.S
> create mode 100644 scripts/gen_paraminfo.c
>
> --
> 2.51.0
>
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 0/2] kallsyms: show typed function parameters in oops/WARN dumps
2026-03-24 8:57 ` Jiri Olsa
@ 2026-03-24 11:39 ` Sasha Levin
0 siblings, 0 replies; 18+ messages in thread
From: Sasha Levin @ 2026-03-24 11:39 UTC (permalink / raw)
To: Jiri Olsa
Cc: Andrew Morton, Masahiro Yamada, Nathan Chancellor, Nicolas Schier,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Peter Zijlstra, Josh Poimboeuf, Petr Mladek,
Alexei Starovoitov, Jonathan Corbet, David Gow, Kees Cook,
Greg KH, Luis Chamberlain, Steven Rostedt, Helge Deller,
Randy Dunlap, Geert Uytterhoeven, Juergen Gross, James Bottomley,
Alexey Dobriyan, Vlastimil Babka, Laurent Pinchart, Petr Pavlu,
x86, linux-kernel, linux-kbuild, linux-doc, linux-modules, bpf
On Tue, Mar 24, 2026 at 09:57:11AM +0100, Jiri Olsa wrote:
>On Mon, Mar 23, 2026 at 12:48:55PM -0400, Sasha Levin wrote:
>> Building on the lineinfo series, this adds typed function parameter
>
>hi,
>could you please specify the exact tree/commit and point to the
>series this patchset is based on?
Yup, sorry, I should have noted that. This series is based on Andrew Morton's
mm-nonmm-unstable branch (git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
mm-nonmm-unstable) since he took the base lineinfo patches and these build on
code adjacent to it.
There's no direct dependency between the two, but they touch the same parts of
the code.
--
Thanks,
Sasha
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 1/2] kallsyms: show function parameter info in oops/WARN dumps
2026-03-23 16:48 ` [PATCH 1/2] kallsyms: show function parameter info " Sasha Levin
@ 2026-03-24 15:03 ` Alexei Starovoitov
2026-03-24 16:00 ` Sasha Levin
0 siblings, 1 reply; 18+ messages in thread
From: Alexei Starovoitov @ 2026-03-24 15:03 UTC (permalink / raw)
To: Sasha Levin
Cc: Andrew Morton, Masahiro Yamada, Nathan Chancellor, Nicolas Schier,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Peter Zijlstra, Josh Poimboeuf, Petr Mladek,
Alexei Starovoitov, Jonathan Corbet, David Gow, Kees Cook,
Greg KH, Luis Chamberlain, Steven Rostedt, Helge Deller,
Randy Dunlap, Geert Uytterhoeven, Juergen Gross, James Bottomley,
Alexey Dobriyan, Vlastimil Babka, Laurent Pinchart, Petr Pavlu,
X86 ML, LKML, Linux Kbuild mailing list, open list:DOCUMENTATION,
linux-modules, bpf
On Mon, Mar 23, 2026 at 9:49 AM Sasha Levin <sashal@kernel.org> wrote:
>
> Embed DWARF-derived function parameter name and type information in the
> kernel image so that oops and WARN dumps display the crashing function's
> register-passed arguments with their names, types, and values.
>
> A new build-time tool (scripts/gen_paraminfo.c) parses DW_TAG_subprogram
> and DW_TAG_formal_parameter entries from DWARF .debug_info, extracting
> parameter names and human-readable type strings. The resulting tables are
> stored in .rodata using the same two-phase link approach as lineinfo.
>
> At runtime, kallsyms_show_paraminfo() performs a binary search on the
> paraminfo tables, maps parameters to x86-64 calling convention registers
> (RDI, RSI, RDX, RCX, R8, R9), and prints each parameter's name, type,
> and value from pt_regs. If a parameter value matches the page fault
> address (CR2), it is highlighted with "<-- fault address".
>
> Integration at show_regs() means this works for both oops and WARN()
> automatically, since both paths provide full pt_regs at the exception
> point.
>
> Example output:
>
> Function parameters (ext4_readdir):
> file (struct file *) = 0xffff888123456000
> ctx (struct dir_context *) = 0x0000000000001234 <-- fault address
>
> Gated behind CONFIG_KALLSYMS_PARAMINFO (depends on CONFIG_KALLSYMS_LINEINFO).
> Adds approximately 1-2 MB to the kernel image for ~58K functions.
>
> Assisted-by: Claude:claude-opus-4-6
> Signed-off-by: Sasha Levin <sashal@kernel.org>
Nack.
You asked claude to reinvent pahole and BTF and it did it
completely missing years of fine tuning that pahole does.
dwarf cannot be trusted as-is. pahole converts it carefully
by analyzing optimized out arguments and dropping signatures
from BTF that are not accurate. This work is still ongoing.
For example see this set:
https://lore.kernel.org/bpf/20260320190917.1970524-1-yonghong.song@linux.dev/
pahole isn't perfect, but what you attempted to do here
is just broken.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 2/2] kallsyms: add BTF-based deep parameter rendering in oops dumps
2026-03-23 16:48 ` [PATCH 2/2] kallsyms: add BTF-based deep parameter rendering in oops dumps Sasha Levin
@ 2026-03-24 15:07 ` Alexei Starovoitov
0 siblings, 0 replies; 18+ messages in thread
From: Alexei Starovoitov @ 2026-03-24 15:07 UTC (permalink / raw)
To: Sasha Levin
Cc: Andrew Morton, Masahiro Yamada, Nathan Chancellor, Nicolas Schier,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Peter Zijlstra, Josh Poimboeuf, Petr Mladek,
Alexei Starovoitov, Jonathan Corbet, David Gow, Kees Cook,
Greg KH, Luis Chamberlain, Steven Rostedt, Helge Deller,
Randy Dunlap, Geert Uytterhoeven, Juergen Gross, James Bottomley,
Alexey Dobriyan, Vlastimil Babka, Laurent Pinchart, Petr Pavlu,
X86 ML, LKML, Linux Kbuild mailing list, open list:DOCUMENTATION,
linux-modules, bpf
On Mon, Mar 23, 2026 at 9:49 AM Sasha Levin <sashal@kernel.org> wrote:
>
> +static const char *extract_struct_name(const char *type_str, bool *is_union,
> + char *name_buf, size_t bufsz)
> +{
> + const char *p, *end;
> +
> + *is_union = false;
> +
> + /* Must end with " *" to be a pointer */
> + end = type_str + strlen(type_str);
> + if (end - type_str < 3 || end[-1] != '*' || end[-2] != ' ')
> + return NULL;
> +
> + if (!strncmp(type_str, "struct ", 7)) {
> + p = type_str + 7;
> + } else if (!strncmp(type_str, "union ", 6)) {
> + p = type_str + 6;
> + *is_union = true;
> + } else {
> + return NULL;
> + }
> +
> + /* Copy name up to the " *" */
> + {
> + size_t len = (end - 2) - p;
> +
> + if (len == 0 || len >= bufsz)
> + return NULL;
> + memcpy(name_buf, p, len);
> + name_buf[len] = '\0';
> + }
> +
> + return name_buf;
Nack. This is just awful.
You didn't even bother to reformat what claude spat out.
Which means you didn't think it through.
It prints something that looks plausible
which is the opposite of what kernel crash dump should be.
crash output should be accurate and not a guess work.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 1/2] kallsyms: show function parameter info in oops/WARN dumps
2026-03-24 15:03 ` Alexei Starovoitov
@ 2026-03-24 16:00 ` Sasha Levin
2026-03-24 16:04 ` Alexei Starovoitov
2026-03-24 17:34 ` [PATCH 1/2] kallsyms: show function parameter info in oops/WARN dumps Alan Maguire
0 siblings, 2 replies; 18+ messages in thread
From: Sasha Levin @ 2026-03-24 16:00 UTC (permalink / raw)
To: Alexei Starovoitov
Cc: Andrew Morton, Masahiro Yamada, Nathan Chancellor, Nicolas Schier,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Peter Zijlstra, Josh Poimboeuf, Petr Mladek,
Alexei Starovoitov, Jonathan Corbet, David Gow, Kees Cook,
Greg KH, Luis Chamberlain, Steven Rostedt, Helge Deller,
Randy Dunlap, Geert Uytterhoeven, Juergen Gross, James Bottomley,
Alexey Dobriyan, Vlastimil Babka, Laurent Pinchart, Petr Pavlu,
X86 ML, LKML, Linux Kbuild mailing list, open list:DOCUMENTATION,
linux-modules, bpf
On Tue, Mar 24, 2026 at 08:03:30AM -0700, Alexei Starovoitov wrote:
>On Mon, Mar 23, 2026 at 9:49 AM Sasha Levin <sashal@kernel.org> wrote:
>>
>> Embed DWARF-derived function parameter name and type information in the
>> kernel image so that oops and WARN dumps display the crashing function's
>> register-passed arguments with their names, types, and values.
>>
>> A new build-time tool (scripts/gen_paraminfo.c) parses DW_TAG_subprogram
>> and DW_TAG_formal_parameter entries from DWARF .debug_info, extracting
>> parameter names and human-readable type strings. The resulting tables are
>> stored in .rodata using the same two-phase link approach as lineinfo.
>>
>> At runtime, kallsyms_show_paraminfo() performs a binary search on the
>> paraminfo tables, maps parameters to x86-64 calling convention registers
>> (RDI, RSI, RDX, RCX, R8, R9), and prints each parameter's name, type,
>> and value from pt_regs. If a parameter value matches the page fault
>> address (CR2), it is highlighted with "<-- fault address".
>>
>> Integration at show_regs() means this works for both oops and WARN()
>> automatically, since both paths provide full pt_regs at the exception
>> point.
>>
>> Example output:
>>
>> Function parameters (ext4_readdir):
>> file (struct file *) = 0xffff888123456000
>> ctx (struct dir_context *) = 0x0000000000001234 <-- fault address
>>
>> Gated behind CONFIG_KALLSYMS_PARAMINFO (depends on CONFIG_KALLSYMS_LINEINFO).
>> Adds approximately 1-2 MB to the kernel image for ~58K functions.
>>
>> Assisted-by: Claude:claude-opus-4-6
>> Signed-off-by: Sasha Levin <sashal@kernel.org>
>
>Nack.
>
>You asked claude to reinvent pahole and BTF and it did it
>completely missing years of fine tuning that pahole does.
Let's keep this on the technical side please.
>dwarf cannot be trusted as-is. pahole converts it carefully
>by analyzing optimized out arguments and dropping signatures
Fair point about pahole and optimized-out args. The problem is that BTF depends
on BPF_SYSCALL, and the environments I care about can't enable either.
Automotive, robotics, and safety configs all have DWARF and KALLSYMS but no
path to BTF.
>from BTF that are not accurate. This work is still ongoing.
>For example see this set:
>https://lore.kernel.org/bpf/20260320190917.1970524-1-yonghong.song@linux.dev/
Ack. I wasn't familiar with this, and looks like it makes the 2nd patch in this
series unnecessary.
>pahole isn't perfect, but what you attempted to do here
>is just broken.
I hear you that raw DWARF isn't perfect with optimized code, but I'd rather
show best-effort info than nothing. Happy to mark it as such in the output.
Open to suggestions on improving accuracy without the BTF dependency.
--
Thanks,
Sasha
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 1/2] kallsyms: show function parameter info in oops/WARN dumps
2026-03-24 16:00 ` Sasha Levin
@ 2026-03-24 16:04 ` Alexei Starovoitov
2026-03-24 18:44 ` Sasha Levin
2026-03-24 17:34 ` [PATCH 1/2] kallsyms: show function parameter info in oops/WARN dumps Alan Maguire
1 sibling, 1 reply; 18+ messages in thread
From: Alexei Starovoitov @ 2026-03-24 16:04 UTC (permalink / raw)
To: Sasha Levin
Cc: Andrew Morton, Masahiro Yamada, Nathan Chancellor, Nicolas Schier,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Peter Zijlstra, Josh Poimboeuf, Petr Mladek,
Alexei Starovoitov, Jonathan Corbet, David Gow, Kees Cook,
Greg KH, Luis Chamberlain, Steven Rostedt, Helge Deller,
Randy Dunlap, Geert Uytterhoeven, Juergen Gross, James Bottomley,
Alexey Dobriyan, Vlastimil Babka, Laurent Pinchart, Petr Pavlu,
X86 ML, LKML, Linux Kbuild mailing list, open list:DOCUMENTATION,
linux-modules, bpf
On Tue, Mar 24, 2026 at 9:00 AM Sasha Levin <sashal@kernel.org> wrote:
>
> On Tue, Mar 24, 2026 at 08:03:30AM -0700, Alexei Starovoitov wrote:
> >On Mon, Mar 23, 2026 at 9:49 AM Sasha Levin <sashal@kernel.org> wrote:
> >>
> >> Embed DWARF-derived function parameter name and type information in the
> >> kernel image so that oops and WARN dumps display the crashing function's
> >> register-passed arguments with their names, types, and values.
> >>
> >> A new build-time tool (scripts/gen_paraminfo.c) parses DW_TAG_subprogram
> >> and DW_TAG_formal_parameter entries from DWARF .debug_info, extracting
> >> parameter names and human-readable type strings. The resulting tables are
> >> stored in .rodata using the same two-phase link approach as lineinfo.
> >>
> >> At runtime, kallsyms_show_paraminfo() performs a binary search on the
> >> paraminfo tables, maps parameters to x86-64 calling convention registers
> >> (RDI, RSI, RDX, RCX, R8, R9), and prints each parameter's name, type,
> >> and value from pt_regs. If a parameter value matches the page fault
> >> address (CR2), it is highlighted with "<-- fault address".
> >>
> >> Integration at show_regs() means this works for both oops and WARN()
> >> automatically, since both paths provide full pt_regs at the exception
> >> point.
> >>
> >> Example output:
> >>
> >> Function parameters (ext4_readdir):
> >> file (struct file *) = 0xffff888123456000
> >> ctx (struct dir_context *) = 0x0000000000001234 <-- fault address
> >>
> >> Gated behind CONFIG_KALLSYMS_PARAMINFO (depends on CONFIG_KALLSYMS_LINEINFO).
> >> Adds approximately 1-2 MB to the kernel image for ~58K functions.
> >>
> >> Assisted-by: Claude:claude-opus-4-6
> >> Signed-off-by: Sasha Levin <sashal@kernel.org>
> >
> >Nack.
> >
> >You asked claude to reinvent pahole and BTF and it did it
> >completely missing years of fine tuning that pahole does.
>
> Let's keep this on the technical side please.
>
> >dwarf cannot be trusted as-is. pahole converts it carefully
> >by analyzing optimized out arguments and dropping signatures
>
> Fair point about pahole and optimized-out args. The problem is that BTF depends
> on BPF_SYSCALL, and the environments I care about can't enable either.
This is trivially fixable without reinventing pahole.
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 1/2] kallsyms: show function parameter info in oops/WARN dumps
2026-03-24 16:00 ` Sasha Levin
2026-03-24 16:04 ` Alexei Starovoitov
@ 2026-03-24 17:34 ` Alan Maguire
2026-03-24 18:51 ` Sasha Levin
1 sibling, 1 reply; 18+ messages in thread
From: Alan Maguire @ 2026-03-24 17:34 UTC (permalink / raw)
To: Sasha Levin, Alexei Starovoitov
Cc: Andrew Morton, Masahiro Yamada, Nathan Chancellor, Nicolas Schier,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Peter Zijlstra, Josh Poimboeuf, Petr Mladek,
Alexei Starovoitov, Jonathan Corbet, David Gow, Kees Cook,
Greg KH, Luis Chamberlain, Steven Rostedt, Helge Deller,
Randy Dunlap, Geert Uytterhoeven, Juergen Gross, James Bottomley,
Alexey Dobriyan, Vlastimil Babka, Laurent Pinchart, Petr Pavlu,
X86 ML, LKML, Linux Kbuild mailing list, open list:DOCUMENTATION,
linux-modules, bpf
On 24/03/2026 16:00, Sasha Levin wrote:
> On Tue, Mar 24, 2026 at 08:03:30AM -0700, Alexei Starovoitov wrote:
>> On Mon, Mar 23, 2026 at 9:49 AM Sasha Levin <sashal@kernel.org> wrote:
>>>
>>> Embed DWARF-derived function parameter name and type information in the
>>> kernel image so that oops and WARN dumps display the crashing function's
>>> register-passed arguments with their names, types, and values.
>>>
>>> A new build-time tool (scripts/gen_paraminfo.c) parses DW_TAG_subprogram
>>> and DW_TAG_formal_parameter entries from DWARF .debug_info, extracting
>>> parameter names and human-readable type strings. The resulting tables are
>>> stored in .rodata using the same two-phase link approach as lineinfo.
>>>
>>> At runtime, kallsyms_show_paraminfo() performs a binary search on the
>>> paraminfo tables, maps parameters to x86-64 calling convention registers
>>> (RDI, RSI, RDX, RCX, R8, R9), and prints each parameter's name, type,
>>> and value from pt_regs. If a parameter value matches the page fault
>>> address (CR2), it is highlighted with "<-- fault address".
>>>
>>> Integration at show_regs() means this works for both oops and WARN()
>>> automatically, since both paths provide full pt_regs at the exception
>>> point.
>>>
>>> Example output:
>>>
>>> Function parameters (ext4_readdir):
>>> file (struct file *) = 0xffff888123456000
>>> ctx (struct dir_context *) = 0x0000000000001234 <-- fault address
>>>
>>> Gated behind CONFIG_KALLSYMS_PARAMINFO (depends on CONFIG_KALLSYMS_LINEINFO).
>>> Adds approximately 1-2 MB to the kernel image for ~58K functions.
>>>
>>> Assisted-by: Claude:claude-opus-4-6
>>> Signed-off-by: Sasha Levin <sashal@kernel.org>
>>
>> Nack.
>>
>> You asked claude to reinvent pahole and BTF and it did it
>> completely missing years of fine tuning that pahole does.
>
> Let's keep this on the technical side please.
>
>> dwarf cannot be trusted as-is. pahole converts it carefully
>> by analyzing optimized out arguments and dropping signatures
>
> Fair point about pahole and optimized-out args. The problem is that BTF depends
> on BPF_SYSCALL, and the environments I care about can't enable either.
> Automotive, robotics, and safety configs all have DWARF and KALLSYMS but no
> path to BTF.
>
Curious what the blockers are to BTF adoption? Hopefully we can tackle some
of these or get them on a roadmap at least. I know some embedded folks want
vmlinux BTF as a module instead of directly contained in the vmlinux binary
to minimize vmlinux size; is this the problem you run into? Are there other
issues? Any info you could provide would be great as the aim is to make BTF
feasible in as many environments as possible. Thanks!
Alan
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 1/2] kallsyms: show function parameter info in oops/WARN dumps
2026-03-24 16:04 ` Alexei Starovoitov
@ 2026-03-24 18:44 ` Sasha Levin
2026-03-25 1:18 ` [RFC] btf: split core BTF parsing out of BPF subsystem into kernel/btf/ Sasha Levin
0 siblings, 1 reply; 18+ messages in thread
From: Sasha Levin @ 2026-03-24 18:44 UTC (permalink / raw)
To: Alexei Starovoitov
Cc: Andrew Morton, Masahiro Yamada, Nathan Chancellor, Nicolas Schier,
Thomas Gleixner, Ingo Molnar, Borislav Petkov, Dave Hansen,
H. Peter Anvin, Peter Zijlstra, Josh Poimboeuf, Petr Mladek,
Alexei Starovoitov, Jonathan Corbet, David Gow, Kees Cook,
Greg KH, Luis Chamberlain, Steven Rostedt, Helge Deller,
Randy Dunlap, Geert Uytterhoeven, Juergen Gross, James Bottomley,
Alexey Dobriyan, Vlastimil Babka, Laurent Pinchart, Petr Pavlu,
X86 ML, LKML, Linux Kbuild mailing list, open list:DOCUMENTATION,
linux-modules, bpf
On Tue, Mar 24, 2026 at 09:04:03AM -0700, Alexei Starovoitov wrote:
>On Tue, Mar 24, 2026 at 9:00 AM Sasha Levin <sashal@kernel.org> wrote:
>>
>> On Tue, Mar 24, 2026 at 08:03:30AM -0700, Alexei Starovoitov wrote:
>> >On Mon, Mar 23, 2026 at 9:49 AM Sasha Levin <sashal@kernel.org> wrote:
>> >>
>> >> Embed DWARF-derived function parameter name and type information in the
>> >> kernel image so that oops and WARN dumps display the crashing function's
>> >> register-passed arguments with their names, types, and values.
>> >>
>> >> A new build-time tool (scripts/gen_paraminfo.c) parses DW_TAG_subprogram
>> >> and DW_TAG_formal_parameter entries from DWARF .debug_info, extracting
>> >> parameter names and human-readable type strings. The resulting tables are
>> >> stored in .rodata using the same two-phase link approach as lineinfo.
>> >>
>> >> At runtime, kallsyms_show_paraminfo() performs a binary search on the
>> >> paraminfo tables, maps parameters to x86-64 calling convention registers
>> >> (RDI, RSI, RDX, RCX, R8, R9), and prints each parameter's name, type,
>> >> and value from pt_regs. If a parameter value matches the page fault
>> >> address (CR2), it is highlighted with "<-- fault address".
>> >>
>> >> Integration at show_regs() means this works for both oops and WARN()
>> >> automatically, since both paths provide full pt_regs at the exception
>> >> point.
>> >>
>> >> Example output:
>> >>
>> >> Function parameters (ext4_readdir):
>> >> file (struct file *) = 0xffff888123456000
>> >> ctx (struct dir_context *) = 0x0000000000001234 <-- fault address
>> >>
>> >> Gated behind CONFIG_KALLSYMS_PARAMINFO (depends on CONFIG_KALLSYMS_LINEINFO).
>> >> Adds approximately 1-2 MB to the kernel image for ~58K functions.
>> >>
>> >> Assisted-by: Claude:claude-opus-4-6
>> >> Signed-off-by: Sasha Levin <sashal@kernel.org>
>> >
>> >Nack.
>> >
>> >You asked claude to reinvent pahole and BTF and it did it
>> >completely missing years of fine tuning that pahole does.
>>
>> Let's keep this on the technical side please.
>>
>> >dwarf cannot be trusted as-is. pahole converts it carefully
>> >by analyzing optimized out arguments and dropping signatures
>>
>> Fair point about pahole and optimized-out args. The problem is that BTF depends
>> on BPF_SYSCALL, and the environments I care about can't enable either.
>
>This is trivially fixable without reinventing pahole.
Hmm...
Looking at the code, I'd need to:
- Split BTF parsing from kernel/bpf/btf.c to somewhere outside of kernel/bpf/.
- New init path for btf_vmlinux outside BPF verifier.
- Refactor btf_parse_vmlinux() BPF-specific bits.
- Remove BPF_SYSCALL dependency from DEBUG_INFO_BTF.
- Somehow make BTF work with DEBUG_INFO_REDUCED.
I suppose that the first 4 are straightforward, but I don't have an idea about
DEBUG_INFO_REDUCED. Though we can probably tackle it later.
Does that make sense? Did I miss anything?
--
Thanks,
Sasha
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH 1/2] kallsyms: show function parameter info in oops/WARN dumps
2026-03-24 17:34 ` [PATCH 1/2] kallsyms: show function parameter info in oops/WARN dumps Alan Maguire
@ 2026-03-24 18:51 ` Sasha Levin
0 siblings, 0 replies; 18+ messages in thread
From: Sasha Levin @ 2026-03-24 18:51 UTC (permalink / raw)
To: Alan Maguire
Cc: Alexei Starovoitov, Andrew Morton, Masahiro Yamada,
Nathan Chancellor, Nicolas Schier, Thomas Gleixner, Ingo Molnar,
Borislav Petkov, Dave Hansen, H. Peter Anvin, Peter Zijlstra,
Josh Poimboeuf, Petr Mladek, Alexei Starovoitov, Jonathan Corbet,
David Gow, Kees Cook, Greg KH, Luis Chamberlain, Steven Rostedt,
Helge Deller, Randy Dunlap, Geert Uytterhoeven, Juergen Gross,
James Bottomley, Alexey Dobriyan, Vlastimil Babka,
Laurent Pinchart, Petr Pavlu, X86 ML, LKML,
Linux Kbuild mailing list, open list:DOCUMENTATION, linux-modules,
bpf
On Tue, Mar 24, 2026 at 05:34:06PM +0000, Alan Maguire wrote:
>On 24/03/2026 16:00, Sasha Levin wrote:
>> On Tue, Mar 24, 2026 at 08:03:30AM -0700, Alexei Starovoitov wrote:
>>> On Mon, Mar 23, 2026 at 9:49 AM Sasha Levin <sashal@kernel.org> wrote:
>>>>
>>>> Embed DWARF-derived function parameter name and type information in the
>>>> kernel image so that oops and WARN dumps display the crashing function's
>>>> register-passed arguments with their names, types, and values.
>>>>
>>>> A new build-time tool (scripts/gen_paraminfo.c) parses DW_TAG_subprogram
>>>> and DW_TAG_formal_parameter entries from DWARF .debug_info, extracting
>>>> parameter names and human-readable type strings. The resulting tables are
>>>> stored in .rodata using the same two-phase link approach as lineinfo.
>>>>
>>>> At runtime, kallsyms_show_paraminfo() performs a binary search on the
>>>> paraminfo tables, maps parameters to x86-64 calling convention registers
>>>> (RDI, RSI, RDX, RCX, R8, R9), and prints each parameter's name, type,
>>>> and value from pt_regs. If a parameter value matches the page fault
>>>> address (CR2), it is highlighted with "<-- fault address".
>>>>
>>>> Integration at show_regs() means this works for both oops and WARN()
>>>> automatically, since both paths provide full pt_regs at the exception
>>>> point.
>>>>
>>>> Example output:
>>>>
>>>> Function parameters (ext4_readdir):
>>>> file (struct file *) = 0xffff888123456000
>>>> ctx (struct dir_context *) = 0x0000000000001234 <-- fault address
>>>>
>>>> Gated behind CONFIG_KALLSYMS_PARAMINFO (depends on CONFIG_KALLSYMS_LINEINFO).
>>>> Adds approximately 1-2 MB to the kernel image for ~58K functions.
>>>>
>>>> Assisted-by: Claude:claude-opus-4-6
>>>> Signed-off-by: Sasha Levin <sashal@kernel.org>
>>>
>>> Nack.
>>>
>>> You asked claude to reinvent pahole and BTF and it did it
>>> completely missing years of fine tuning that pahole does.
>>
>> Let's keep this on the technical side please.
>>
>>> dwarf cannot be trusted as-is. pahole converts it carefully
>>> by analyzing optimized out arguments and dropping signatures
>>
>> Fair point about pahole and optimized-out args. The problem is that BTF depends
>> on BPF_SYSCALL, and the environments I care about can't enable either.
>> Automotive, robotics, and safety configs all have DWARF and KALLSYMS but no
>> path to BTF.
>>
>
>Curious what the blockers are to BTF adoption? Hopefully we can tackle some
>of these or get them on a roadmap at least. I know some embedded folks want
>vmlinux BTF as a module instead of directly contained in the vmlinux binary
>to minimize vmlinux size; is this the problem you run into? Are there other
>issues? Any info you could provide would be great as the aim is to make BTF
>feasible in as many environments as possible. Thanks!
So the biggest reason I'm aware of is that those systems do not want to enable
BPF, and BTF is hidden behind BPF.
Other than that:
1. Toolchain qualifications for safety uses (we'd need to get pahole safety
certified).
2. On the ecosystem side, from what I saw, BTF isn't part of most BSPs that
vendors produce.
3. I saw concerns in the past about interactions with PREEMPT_RT, but I'm not
sure if it's still a thing.
--
Thanks,
Sasha
^ permalink raw reply [flat|nested] 18+ messages in thread
* [RFC] btf: split core BTF parsing out of BPF subsystem into kernel/btf/
2026-03-24 18:44 ` Sasha Levin
@ 2026-03-25 1:18 ` Sasha Levin
2026-03-25 2:15 ` bot+bpf-ci
0 siblings, 1 reply; 18+ messages in thread
From: Sasha Levin @ 2026-03-25 1:18 UTC (permalink / raw)
To: sashal
Cc: James.Bottomley, adobriyan, akpm, alexei.starovoitov, ast, bp,
bpf, corbet, dave.hansen, davidgow, deller, geert, gregkh, hpa,
jgross, jpoimboe, kees, laurent.pinchart, linux-doc, linux-kbuild,
linux-kernel, linux-modules, masahiroy, mcgrof, mingo, nathan,
nsc, peterz, petr.pavlu, pmladek, rdunlap, rostedt, tglx, vbabka,
x86
Move BTF type format parsing and inspection code out of the BPF
subsystem into its own kernel/btf/ directory, separating core BTF
functionality from BPF-specific extensions.
CONFIG_DEBUG_INFO_BTF currently depends on CONFIG_BPF_SYSCALL, which
prevents embedded, automotive, and safety-critical environments from
using BTF. These platforms often disable BPF for security and size
reasons but would benefit from BTF type information for crash
diagnostics and debugging.
Introduce CONFIG_BTF for the core BTF runtime support (type parsing,
lookup, and display) which can be enabled independently of
CONFIG_BPF_SYSCALL, and remove the BPF_SYSCALL dependency from
CONFIG_DEBUG_INFO_BTF. The code is split into:
kernel/btf/btf.c - Core BTF: parsing, verification, type lookup,
display infrastructure. No BPF dependencies.
kernel/btf/bpf.c - BPF extensions: kfunc registration, struct_ops,
verifier integration, FD interface, module BTF.
kernel/btf/btf.h - Internal shared header.
When CONFIG_BPF_SYSCALL=n, only btf.c is compiled, providing the
minimal BTF API. When CONFIG_BPF_SYSCALL=y, both files are compiled
and bpf.c overrides weak symbols in btf.c for BPF-specific cleanup
(IDR management, RCU-deferred freeing, kfunc/struct_ops tab cleanup).
No functional change when CONFIG_BPF_SYSCALL=y.
Assisted-by: Claude:claude-opus-4-6
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
include/linux/btf.h | 80 +-
kernel/Makefile | 1 +
kernel/bpf/Makefile | 2 +-
kernel/btf/Makefile | 4 +
kernel/{bpf/btf.c => btf/bpf.c} | 6386 ++++---------------------------
kernel/btf/btf.c | 4803 +++++++++++++++++++++++
kernel/btf/btf.h | 135 +
lib/Kconfig.debug | 15 +-
8 files changed, 5796 insertions(+), 5630 deletions(-)
create mode 100644 kernel/btf/Makefile
rename kernel/{bpf/btf.c => btf/bpf.c} (50%)
create mode 100644 kernel/btf/btf.c
create mode 100644 kernel/btf/btf.h
diff --git a/include/linux/btf.h b/include/linux/btf.h
index 48108471c5b13..d4d64878324ff 100644
--- a/include/linux/btf.h
+++ b/include/linux/btf.h
@@ -139,17 +139,11 @@ struct btf_struct_metas {
struct btf_struct_meta types[];
};
-extern const struct file_operations btf_fops;
-
const char *btf_get_name(const struct btf *btf);
void btf_get(struct btf *btf);
void btf_put(struct btf *btf);
const struct btf_header *btf_header(const struct btf *btf);
-int btf_new_fd(const union bpf_attr *attr, bpfptr_t uattr, u32 uattr_sz);
-struct btf *btf_get_by_fd(int fd);
-int btf_get_info_by_fd(const struct btf *btf,
- const union bpf_attr *attr,
- union bpf_attr __user *uattr);
+
/* Figure out the size of a type_id. If type_id is a modifier
* (e.g. const), it will be resolved to find out the type with size.
*
@@ -213,12 +207,9 @@ int btf_type_seq_show_flags(const struct btf *btf, u32 type_id, void *obj,
int btf_type_snprintf_show(const struct btf *btf, u32 type_id, void *obj,
char *buf, int len, u64 flags);
-int btf_get_fd_by_id(u32 id);
-u32 btf_obj_id(const struct btf *btf);
bool btf_is_kernel(const struct btf *btf);
bool btf_is_module(const struct btf *btf);
bool btf_is_vmlinux(const struct btf *btf);
-struct module *btf_try_get_module(const struct btf *btf);
u32 btf_nr_types(const struct btf *btf);
u32 btf_named_start_id(const struct btf *btf, bool own);
struct btf *btf_base_btf(const struct btf *btf);
@@ -228,12 +219,8 @@ bool btf_type_is_primitive(const struct btf_type *t);
bool btf_member_is_reg_int(const struct btf *btf, const struct btf_type *s,
const struct btf_member *m,
u32 expected_offset, u32 expected_size);
-struct btf_record *btf_parse_fields(const struct btf *btf, const struct btf_type *t,
- u32 field_mask, u32 value_size);
-int btf_check_and_fixup_fields(const struct btf *btf, struct btf_record *rec);
bool btf_type_is_void(const struct btf_type *t);
s32 btf_find_by_name_kind(const struct btf *btf, const char *name, u8 kind);
-s32 bpf_find_btf_id(const char *name, u32 kind, struct btf **btf_p);
const struct btf_type *btf_type_skip_modifiers(const struct btf *btf,
u32 id, u32 *res_id);
const struct btf_type *btf_type_resolve_ptr(const struct btf *btf,
@@ -245,6 +232,20 @@ btf_resolve_size(const struct btf *btf, const struct btf_type *type,
u32 *type_size);
const char *btf_type_str(const struct btf_type *t);
+/* BPF-specific declarations */
+extern const struct file_operations btf_fops;
+int btf_new_fd(const union bpf_attr *attr, bpfptr_t uattr, u32 uattr_sz);
+struct btf *btf_get_by_fd(int fd);
+int btf_get_info_by_fd(const struct btf *btf,
+ const union bpf_attr *attr,
+ union bpf_attr __user *uattr);
+int btf_get_fd_by_id(u32 id);
+u32 btf_obj_id(const struct btf *btf);
+struct btf_record *btf_parse_fields(const struct btf *btf, const struct btf_type *t,
+ u32 field_mask, u32 value_size);
+int btf_check_and_fixup_fields(const struct btf *btf, struct btf_record *rec);
+s32 bpf_find_btf_id(const char *name, u32 kind, struct btf **btf_p);
+
#define for_each_member(i, struct_type, member) \
for (i = 0, member = btf_type_member(struct_type); \
i < btf_type_vlen(struct_type); \
@@ -524,9 +525,6 @@ static inline void *btf_id_set8_contains(const struct btf_id_set8 *set, u32 id)
bool btf_param_match_suffix(const struct btf *btf,
const struct btf_param *arg,
const char *suffix);
-int btf_ctx_arg_offset(const struct btf *btf, const struct btf_type *func_proto,
- u32 arg_no);
-u32 btf_ctx_arg_idx(struct btf *btf, const struct btf_type *func_proto, int off);
struct bpf_verifier_log;
@@ -564,16 +562,39 @@ struct btf_field_iter {
int vlen;
};
-#ifdef CONFIG_BPF_SYSCALL
+#ifdef CONFIG_BTF
const struct btf_type *btf_type_by_id(const struct btf *btf, u32 type_id);
void btf_set_base_btf(struct btf *btf, const struct btf *base_btf);
+const char *btf_name_by_offset(const struct btf *btf, u32 offset);
+const char *btf_str_by_offset(const struct btf *btf, u32 offset);
+#else
+static inline const struct btf_type *btf_type_by_id(const struct btf *btf,
+ u32 type_id)
+{
+ return NULL;
+}
+
+static inline void btf_set_base_btf(struct btf *btf, const struct btf *base_btf)
+{
+}
+
+static inline const char *btf_name_by_offset(const struct btf *btf,
+ u32 offset)
+{
+ return NULL;
+}
+#endif /* CONFIG_BTF */
+
+#ifdef CONFIG_BPF_SYSCALL
+struct module *btf_try_get_module(const struct btf *btf);
int btf_relocate(struct btf *btf, const struct btf *base_btf, __u32 **map_ids);
int btf_field_iter_init(struct btf_field_iter *it, struct btf_type *t,
enum btf_field_iter_kind iter_kind);
__u32 *btf_field_iter_next(struct btf_field_iter *it);
-const char *btf_name_by_offset(const struct btf *btf, u32 offset);
-const char *btf_str_by_offset(const struct btf *btf, u32 offset);
+int btf_ctx_arg_offset(const struct btf *btf, const struct btf_type *func_proto,
+ u32 arg_no);
+u32 btf_ctx_arg_idx(struct btf *btf, const struct btf_type *func_proto, int off);
struct btf *btf_parse_vmlinux(void);
struct btf *bpf_prog_get_target_btf(const struct bpf_prog *prog);
u32 *btf_kfunc_flags(const struct btf *btf, u32 kfunc_btf_id, const struct bpf_prog *prog);
@@ -606,17 +627,7 @@ static inline bool btf_type_is_struct_ptr(struct btf *btf, const struct btf_type
return btf_type_is_struct(t);
}
#else
-static inline const struct btf_type *btf_type_by_id(const struct btf *btf,
- u32 type_id)
-{
- return NULL;
-}
-
-static inline void btf_set_base_btf(struct btf *btf, const struct btf *base_btf)
-{
-}
-
-static inline int btf_relocate(void *log, struct btf *btf, const struct btf *base_btf,
+static inline int btf_relocate(struct btf *btf, const struct btf *base_btf,
__u32 **map_ids)
{
return -EOPNOTSUPP;
@@ -633,11 +644,6 @@ static inline __u32 *btf_field_iter_next(struct btf_field_iter *it)
return NULL;
}
-static inline const char *btf_name_by_offset(const struct btf *btf,
- u32 offset)
-{
- return NULL;
-}
static inline u32 *btf_kfunc_id_set_contains(const struct btf *btf,
u32 kfunc_btf_id,
struct bpf_prog *prog)
@@ -683,5 +689,5 @@ static inline int btf_check_iter_arg(struct btf *btf, const struct btf_type *fun
{
return -EOPNOTSUPP;
}
-#endif
+#endif /* CONFIG_BPF_SYSCALL */
#endif
diff --git a/kernel/Makefile b/kernel/Makefile
index 6785982013dce..d09d7f102c335 100644
--- a/kernel/Makefile
+++ b/kernel/Makefile
@@ -121,6 +121,7 @@ obj-$(CONFIG_RETHOOK) += trace/
obj-$(CONFIG_IRQ_WORK) += irq_work.o
obj-$(CONFIG_CPU_PM) += cpu_pm.o
obj-$(CONFIG_BPF) += bpf/
+obj-$(CONFIG_BTF) += btf/
obj-$(CONFIG_KCSAN) += kcsan/
obj-$(CONFIG_SHADOW_CALL_STACK) += scs.o
obj-$(CONFIG_HAVE_STATIC_CALL) += static_call.o
diff --git a/kernel/bpf/Makefile b/kernel/bpf/Makefile
index 79cf22860a99b..37d7ebf4808f3 100644
--- a/kernel/bpf/Makefile
+++ b/kernel/bpf/Makefile
@@ -14,7 +14,7 @@ obj-$(CONFIG_BPF_SYSCALL) += bpf_local_storage.o bpf_task_storage.o
obj-${CONFIG_BPF_LSM} += bpf_inode_storage.o
obj-$(CONFIG_BPF_SYSCALL) += disasm.o mprog.o
obj-$(CONFIG_BPF_JIT) += trampoline.o
-obj-$(CONFIG_BPF_SYSCALL) += btf.o memalloc.o rqspinlock.o stream.o
+obj-$(CONFIG_BPF_SYSCALL) += memalloc.o rqspinlock.o stream.o
ifeq ($(CONFIG_MMU)$(CONFIG_64BIT),yy)
obj-$(CONFIG_BPF_SYSCALL) += arena.o range_tree.o
endif
diff --git a/kernel/btf/Makefile b/kernel/btf/Makefile
new file mode 100644
index 0000000000000..e0365d5c7e9bd
--- /dev/null
+++ b/kernel/btf/Makefile
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: GPL-2.0
+
+obj-$(CONFIG_BTF) += btf.o
+obj-$(CONFIG_BPF_SYSCALL) += bpf.o
diff --git a/kernel/bpf/btf.c b/kernel/btf/bpf.c
similarity index 50%
rename from kernel/bpf/btf.c
rename to kernel/btf/bpf.c
index 71f9143fe90f3..986d29f4bc3ba 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/btf/bpf.c
@@ -1,33 +1,35 @@
// SPDX-License-Identifier: GPL-2.0
/* Copyright (c) 2018 Facebook */
+/* BPF-specific BTF extensions */
#include <uapi/linux/btf.h>
#include <uapi/linux/bpf.h>
-#include <uapi/linux/bpf_perf_event.h>
#include <uapi/linux/types.h>
-#include <linux/seq_file.h>
#include <linux/compiler.h>
#include <linux/ctype.h>
#include <linux/errno.h>
#include <linux/slab.h>
-#include <linux/anon_inodes.h>
#include <linux/file.h>
-#include <linux/uaccess.h>
#include <linux/kernel.h>
-#include <linux/idr.h>
#include <linux/sort.h>
+#include <linux/seq_file.h>
#include <linux/bpf_verifier.h>
#include <linux/btf.h>
#include <linux/btf_ids.h>
+#include <linux/bsearch.h>
+#include <linux/string.h>
+#include <linux/overflow.h>
+#include <linux/kobject.h>
+
+#include <uapi/linux/bpf_perf_event.h>
+#include <linux/anon_inodes.h>
+#include <linux/uaccess.h>
+#include <linux/idr.h>
#include <linux/bpf.h>
#include <linux/bpf_lsm.h>
#include <linux/skmsg.h>
#include <linux/perf_event.h>
-#include <linux/bsearch.h>
-#include <linux/kobject.h>
-#include <linux/string.h>
#include <linux/sysfs.h>
-#include <linux/overflow.h>
#include <net/netfilter/nf_bpf_link.h>
@@ -35,173 +37,7 @@
#include <net/xdp.h>
#include "../tools/lib/bpf/relo_core.h"
-/* BTF (BPF Type Format) is the meta data format which describes
- * the data types of BPF program/map. Hence, it basically focus
- * on the C programming language which the modern BPF is primary
- * using.
- *
- * ELF Section:
- * ~~~~~~~~~~~
- * The BTF data is stored under the ".BTF" ELF section
- *
- * struct btf_type:
- * ~~~~~~~~~~~~~~~
- * Each 'struct btf_type' object describes a C data type.
- * Depending on the type it is describing, a 'struct btf_type'
- * object may be followed by more data. F.e.
- * To describe an array, 'struct btf_type' is followed by
- * 'struct btf_array'.
- *
- * 'struct btf_type' and any extra data following it are
- * 4 bytes aligned.
- *
- * Type section:
- * ~~~~~~~~~~~~~
- * The BTF type section contains a list of 'struct btf_type' objects.
- * Each one describes a C type. Recall from the above section
- * that a 'struct btf_type' object could be immediately followed by extra
- * data in order to describe some particular C types.
- *
- * type_id:
- * ~~~~~~~
- * Each btf_type object is identified by a type_id. The type_id
- * is implicitly implied by the location of the btf_type object in
- * the BTF type section. The first one has type_id 1. The second
- * one has type_id 2...etc. Hence, an earlier btf_type has
- * a smaller type_id.
- *
- * A btf_type object may refer to another btf_type object by using
- * type_id (i.e. the "type" in the "struct btf_type").
- *
- * NOTE that we cannot assume any reference-order.
- * A btf_type object can refer to an earlier btf_type object
- * but it can also refer to a later btf_type object.
- *
- * For example, to describe "const void *". A btf_type
- * object describing "const" may refer to another btf_type
- * object describing "void *". This type-reference is done
- * by specifying type_id:
- *
- * [1] CONST (anon) type_id=2
- * [2] PTR (anon) type_id=0
- *
- * The above is the btf_verifier debug log:
- * - Each line started with "[?]" is a btf_type object
- * - [?] is the type_id of the btf_type object.
- * - CONST/PTR is the BTF_KIND_XXX
- * - "(anon)" is the name of the type. It just
- * happens that CONST and PTR has no name.
- * - type_id=XXX is the 'u32 type' in btf_type
- *
- * NOTE: "void" has type_id 0
- *
- * String section:
- * ~~~~~~~~~~~~~~
- * The BTF string section contains the names used by the type section.
- * Each string is referred by an "offset" from the beginning of the
- * string section.
- *
- * Each string is '\0' terminated.
- *
- * The first character in the string section must be '\0'
- * which is used to mean 'anonymous'. Some btf_type may not
- * have a name.
- */
-
-/* BTF verification:
- *
- * To verify BTF data, two passes are needed.
- *
- * Pass #1
- * ~~~~~~~
- * The first pass is to collect all btf_type objects to
- * an array: "btf->types".
- *
- * Depending on the C type that a btf_type is describing,
- * a btf_type may be followed by extra data. We don't know
- * how many btf_type is there, and more importantly we don't
- * know where each btf_type is located in the type section.
- *
- * Without knowing the location of each type_id, most verifications
- * cannot be done. e.g. an earlier btf_type may refer to a later
- * btf_type (recall the "const void *" above), so we cannot
- * check this type-reference in the first pass.
- *
- * In the first pass, it still does some verifications (e.g.
- * checking the name is a valid offset to the string section).
- *
- * Pass #2
- * ~~~~~~~
- * The main focus is to resolve a btf_type that is referring
- * to another type.
- *
- * We have to ensure the referring type:
- * 1) does exist in the BTF (i.e. in btf->types[])
- * 2) does not cause a loop:
- * struct A {
- * struct B b;
- * };
- *
- * struct B {
- * struct A a;
- * };
- *
- * btf_type_needs_resolve() decides if a btf_type needs
- * to be resolved.
- *
- * The needs_resolve type implements the "resolve()" ops which
- * essentially does a DFS and detects backedge.
- *
- * During resolve (or DFS), different C types have different
- * "RESOLVED" conditions.
- *
- * When resolving a BTF_KIND_STRUCT, we need to resolve all its
- * members because a member is always referring to another
- * type. A struct's member can be treated as "RESOLVED" if
- * it is referring to a BTF_KIND_PTR. Otherwise, the
- * following valid C struct would be rejected:
- *
- * struct A {
- * int m;
- * struct A *a;
- * };
- *
- * When resolving a BTF_KIND_PTR, it needs to keep resolving if
- * it is referring to another BTF_KIND_PTR. Otherwise, we cannot
- * detect a pointer loop, e.g.:
- * BTF_KIND_CONST -> BTF_KIND_PTR -> BTF_KIND_CONST -> BTF_KIND_PTR +
- * ^ |
- * +-----------------------------------------+
- *
- */
-
-#define BITS_PER_U128 (sizeof(u64) * BITS_PER_BYTE * 2)
-#define BITS_PER_BYTE_MASK (BITS_PER_BYTE - 1)
-#define BITS_PER_BYTE_MASKED(bits) ((bits) & BITS_PER_BYTE_MASK)
-#define BITS_ROUNDDOWN_BYTES(bits) ((bits) >> 3)
-#define BITS_ROUNDUP_BYTES(bits) \
- (BITS_ROUNDDOWN_BYTES(bits) + !!BITS_PER_BYTE_MASKED(bits))
-
-#define BTF_INFO_MASK 0x9f00ffff
-#define BTF_INT_MASK 0x0fffffff
-#define BTF_TYPE_ID_VALID(type_id) ((type_id) <= BTF_MAX_TYPE)
-#define BTF_STR_OFFSET_VALID(name_off) ((name_off) <= BTF_MAX_NAME_OFFSET)
-
-/* 16MB for 64k structs and each has 16 members and
- * a few MB spaces for the string section.
- * The hard limit is S32_MAX.
- */
-#define BTF_MAX_SIZE (16 * 1024 * 1024)
-
-#define for_each_member_from(i, from, struct_type, member) \
- for (i = from, member = btf_type_member(struct_type) + from; \
- i < btf_type_vlen(struct_type); \
- i++, member++)
-
-#define for_each_vsi_from(i, from, struct_type, member) \
- for (i = from, member = btf_type_var_secinfo(struct_type) + from; \
- i < btf_type_vlen(struct_type); \
- i++, member++)
+#include "btf.h"
DEFINE_IDR(btf_idr);
DEFINE_SPINLOCK(btf_idr_lock);
@@ -250,431 +86,12 @@ struct btf_struct_ops_tab {
u32 capacity;
struct bpf_struct_ops_desc ops[];
};
-
-struct btf {
- void *data;
- struct btf_type **types;
- u32 *resolved_ids;
- u32 *resolved_sizes;
- const char *strings;
- void *nohdr_data;
- struct btf_header hdr;
- u32 nr_types; /* includes VOID for base BTF */
- u32 named_start_id;
- u32 types_size;
- u32 data_size;
- refcount_t refcnt;
u32 id;
struct rcu_head rcu;
struct btf_kfunc_set_tab *kfunc_set_tab;
struct btf_id_dtor_kfunc_tab *dtor_kfunc_tab;
struct btf_struct_metas *struct_meta_tab;
struct btf_struct_ops_tab *struct_ops_tab;
-
- /* split BTF support */
- struct btf *base_btf;
- u32 start_id; /* first type ID in this BTF (0 for base BTF) */
- u32 start_str_off; /* first string offset (0 for base BTF) */
- char name[MODULE_NAME_LEN];
- bool kernel_btf;
- __u32 *base_id_map; /* map from distilled base BTF -> vmlinux BTF ids */
-};
-
-enum verifier_phase {
- CHECK_META,
- CHECK_TYPE,
-};
-
-struct resolve_vertex {
- const struct btf_type *t;
- u32 type_id;
- u16 next_member;
-};
-
-enum visit_state {
- NOT_VISITED,
- VISITED,
- RESOLVED,
-};
-
-enum resolve_mode {
- RESOLVE_TBD, /* To Be Determined */
- RESOLVE_PTR, /* Resolving for Pointer */
- RESOLVE_STRUCT_OR_ARRAY, /* Resolving for struct/union
- * or array
- */
-};
-
-#define MAX_RESOLVE_DEPTH 32
-
-struct btf_sec_info {
- u32 off;
- u32 len;
-};
-
-struct btf_verifier_env {
- struct btf *btf;
- u8 *visit_states;
- struct resolve_vertex stack[MAX_RESOLVE_DEPTH];
- struct bpf_verifier_log log;
- u32 log_type_id;
- u32 top_stack;
- enum verifier_phase phase;
- enum resolve_mode resolve_mode;
-};
-
-static const char * const btf_kind_str[NR_BTF_KINDS] = {
- [BTF_KIND_UNKN] = "UNKNOWN",
- [BTF_KIND_INT] = "INT",
- [BTF_KIND_PTR] = "PTR",
- [BTF_KIND_ARRAY] = "ARRAY",
- [BTF_KIND_STRUCT] = "STRUCT",
- [BTF_KIND_UNION] = "UNION",
- [BTF_KIND_ENUM] = "ENUM",
- [BTF_KIND_FWD] = "FWD",
- [BTF_KIND_TYPEDEF] = "TYPEDEF",
- [BTF_KIND_VOLATILE] = "VOLATILE",
- [BTF_KIND_CONST] = "CONST",
- [BTF_KIND_RESTRICT] = "RESTRICT",
- [BTF_KIND_FUNC] = "FUNC",
- [BTF_KIND_FUNC_PROTO] = "FUNC_PROTO",
- [BTF_KIND_VAR] = "VAR",
- [BTF_KIND_DATASEC] = "DATASEC",
- [BTF_KIND_FLOAT] = "FLOAT",
- [BTF_KIND_DECL_TAG] = "DECL_TAG",
- [BTF_KIND_TYPE_TAG] = "TYPE_TAG",
- [BTF_KIND_ENUM64] = "ENUM64",
-};
-
-const char *btf_type_str(const struct btf_type *t)
-{
- return btf_kind_str[BTF_INFO_KIND(t->info)];
-}
-
-/* Chunk size we use in safe copy of data to be shown. */
-#define BTF_SHOW_OBJ_SAFE_SIZE 32
-
-/*
- * This is the maximum size of a base type value (equivalent to a
- * 128-bit int); if we are at the end of our safe buffer and have
- * less than 16 bytes space we can't be assured of being able
- * to copy the next type safely, so in such cases we will initiate
- * a new copy.
- */
-#define BTF_SHOW_OBJ_BASE_TYPE_SIZE 16
-
-/* Type name size */
-#define BTF_SHOW_NAME_SIZE 80
-
-/*
- * The suffix of a type that indicates it cannot alias another type when
- * comparing BTF IDs for kfunc invocations.
- */
-#define NOCAST_ALIAS_SUFFIX "___init"
-
-/*
- * Common data to all BTF show operations. Private show functions can add
- * their own data to a structure containing a struct btf_show and consult it
- * in the show callback. See btf_type_show() below.
- *
- * One challenge with showing nested data is we want to skip 0-valued
- * data, but in order to figure out whether a nested object is all zeros
- * we need to walk through it. As a result, we need to make two passes
- * when handling structs, unions and arrays; the first path simply looks
- * for nonzero data, while the second actually does the display. The first
- * pass is signalled by show->state.depth_check being set, and if we
- * encounter a non-zero value we set show->state.depth_to_show to
- * the depth at which we encountered it. When we have completed the
- * first pass, we will know if anything needs to be displayed if
- * depth_to_show > depth. See btf_[struct,array]_show() for the
- * implementation of this.
- *
- * Another problem is we want to ensure the data for display is safe to
- * access. To support this, the anonymous "struct {} obj" tracks the data
- * object and our safe copy of it. We copy portions of the data needed
- * to the object "copy" buffer, but because its size is limited to
- * BTF_SHOW_OBJ_COPY_LEN bytes, multiple copies may be required as we
- * traverse larger objects for display.
- *
- * The various data type show functions all start with a call to
- * btf_show_start_type() which returns a pointer to the safe copy
- * of the data needed (or if BTF_SHOW_UNSAFE is specified, to the
- * raw data itself). btf_show_obj_safe() is responsible for
- * using copy_from_kernel_nofault() to update the safe data if necessary
- * as we traverse the object's data. skbuff-like semantics are
- * used:
- *
- * - obj.head points to the start of the toplevel object for display
- * - obj.size is the size of the toplevel object
- * - obj.data points to the current point in the original data at
- * which our safe data starts. obj.data will advance as we copy
- * portions of the data.
- *
- * In most cases a single copy will suffice, but larger data structures
- * such as "struct task_struct" will require many copies. The logic in
- * btf_show_obj_safe() handles the logic that determines if a new
- * copy_from_kernel_nofault() is needed.
- */
-struct btf_show {
- u64 flags;
- void *target; /* target of show operation (seq file, buffer) */
- __printf(2, 0) void (*showfn)(struct btf_show *show, const char *fmt, va_list args);
- const struct btf *btf;
- /* below are used during iteration */
- struct {
- u8 depth;
- u8 depth_to_show;
- u8 depth_check;
- u8 array_member:1,
- array_terminated:1;
- u16 array_encoding;
- u32 type_id;
- int status; /* non-zero for error */
- const struct btf_type *type;
- const struct btf_member *member;
- char name[BTF_SHOW_NAME_SIZE]; /* space for member name/type */
- } state;
- struct {
- u32 size;
- void *head;
- void *data;
- u8 safe[BTF_SHOW_OBJ_SAFE_SIZE];
- } obj;
-};
-
-struct btf_kind_operations {
- s32 (*check_meta)(struct btf_verifier_env *env,
- const struct btf_type *t,
- u32 meta_left);
- int (*resolve)(struct btf_verifier_env *env,
- const struct resolve_vertex *v);
- int (*check_member)(struct btf_verifier_env *env,
- const struct btf_type *struct_type,
- const struct btf_member *member,
- const struct btf_type *member_type);
- int (*check_kflag_member)(struct btf_verifier_env *env,
- const struct btf_type *struct_type,
- const struct btf_member *member,
- const struct btf_type *member_type);
- void (*log_details)(struct btf_verifier_env *env,
- const struct btf_type *t);
- void (*show)(const struct btf *btf, const struct btf_type *t,
- u32 type_id, void *data, u8 bits_offsets,
- struct btf_show *show);
-};
-
-static const struct btf_kind_operations * const kind_ops[NR_BTF_KINDS];
-static struct btf_type btf_void;
-
-static int btf_resolve(struct btf_verifier_env *env,
- const struct btf_type *t, u32 type_id);
-
-static int btf_func_check(struct btf_verifier_env *env,
- const struct btf_type *t);
-
-static bool btf_type_is_modifier(const struct btf_type *t)
-{
- /* Some of them is not strictly a C modifier
- * but they are grouped into the same bucket
- * for BTF concern:
- * A type (t) that refers to another
- * type through t->type AND its size cannot
- * be determined without following the t->type.
- *
- * ptr does not fall into this bucket
- * because its size is always sizeof(void *).
- */
- switch (BTF_INFO_KIND(t->info)) {
- case BTF_KIND_TYPEDEF:
- case BTF_KIND_VOLATILE:
- case BTF_KIND_CONST:
- case BTF_KIND_RESTRICT:
- case BTF_KIND_TYPE_TAG:
- return true;
- }
-
- return false;
-}
-
-static int btf_start_id(const struct btf *btf)
-{
- return btf->start_id + (btf->base_btf ? 0 : 1);
-}
-
-bool btf_type_is_void(const struct btf_type *t)
-{
- return t == &btf_void;
-}
-
-static bool btf_type_is_datasec(const struct btf_type *t)
-{
- return BTF_INFO_KIND(t->info) == BTF_KIND_DATASEC;
-}
-
-static bool btf_type_is_decl_tag(const struct btf_type *t)
-{
- return BTF_INFO_KIND(t->info) == BTF_KIND_DECL_TAG;
-}
-
-static bool btf_type_nosize(const struct btf_type *t)
-{
- return btf_type_is_void(t) || btf_type_is_fwd(t) ||
- btf_type_is_func(t) || btf_type_is_func_proto(t) ||
- btf_type_is_decl_tag(t);
-}
-
-static bool btf_type_nosize_or_null(const struct btf_type *t)
-{
- return !t || btf_type_nosize(t);
-}
-
-static bool btf_type_is_decl_tag_target(const struct btf_type *t)
-{
- return btf_type_is_func(t) || btf_type_is_struct(t) ||
- btf_type_is_var(t) || btf_type_is_typedef(t);
-}
-
-bool btf_is_vmlinux(const struct btf *btf)
-{
- return btf->kernel_btf && !btf->base_btf;
-}
-
-u32 btf_nr_types(const struct btf *btf)
-{
- u32 total = 0;
-
- while (btf) {
- total += btf->nr_types;
- btf = btf->base_btf;
- }
-
- return total;
-}
-
-/*
- * Note that vmlinux and kernel module BTFs are always sorted
- * during the building phase.
- */
-static void btf_check_sorted(struct btf *btf)
-{
- u32 i, n, named_start_id = 0;
-
- n = btf_nr_types(btf);
- if (btf_is_vmlinux(btf)) {
- for (i = btf_start_id(btf); i < n; i++) {
- const struct btf_type *t = btf_type_by_id(btf, i);
- const char *n = btf_name_by_offset(btf, t->name_off);
-
- if (n[0] != '\0') {
- btf->named_start_id = i;
- return;
- }
- }
- return;
- }
-
- for (i = btf_start_id(btf) + 1; i < n; i++) {
- const struct btf_type *ta = btf_type_by_id(btf, i - 1);
- const struct btf_type *tb = btf_type_by_id(btf, i);
- const char *na = btf_name_by_offset(btf, ta->name_off);
- const char *nb = btf_name_by_offset(btf, tb->name_off);
-
- if (strcmp(na, nb) > 0)
- return;
-
- if (named_start_id == 0 && na[0] != '\0')
- named_start_id = i - 1;
- if (named_start_id == 0 && nb[0] != '\0')
- named_start_id = i;
- }
-
- if (named_start_id)
- btf->named_start_id = named_start_id;
-}
-
-/*
- * btf_named_start_id - Get the named starting ID for the BTF
- * @btf: Pointer to the target BTF object
- * @own: Flag indicating whether to query only the current BTF (true = current BTF only,
- * false = recursively traverse the base BTF chain)
- *
- * Return value rules:
- * 1. For a sorted btf, return its named_start_id
- * 2. Else for a split BTF, return its start_id
- * 3. Else for a base BTF, return 1
- */
-u32 btf_named_start_id(const struct btf *btf, bool own)
-{
- const struct btf *base_btf = btf;
-
- while (!own && base_btf->base_btf)
- base_btf = base_btf->base_btf;
-
- return base_btf->named_start_id ?: (base_btf->start_id ?: 1);
-}
-
-static s32 btf_find_by_name_kind_bsearch(const struct btf *btf, const char *name)
-{
- const struct btf_type *t;
- const char *tname;
- s32 l, r, m;
-
- l = btf_named_start_id(btf, true);
- r = btf_nr_types(btf) - 1;
- while (l <= r) {
- m = l + (r - l) / 2;
- t = btf_type_by_id(btf, m);
- tname = btf_name_by_offset(btf, t->name_off);
- if (strcmp(tname, name) >= 0) {
- if (l == r)
- return r;
- r = m;
- } else {
- l = m + 1;
- }
- }
-
- return btf_nr_types(btf);
-}
-
-s32 btf_find_by_name_kind(const struct btf *btf, const char *name, u8 kind)
-{
- const struct btf *base_btf = btf_base_btf(btf);
- const struct btf_type *t;
- const char *tname;
- s32 id, total;
-
- if (base_btf) {
- id = btf_find_by_name_kind(base_btf, name, kind);
- if (id > 0)
- return id;
- }
-
- total = btf_nr_types(btf);
- if (btf->named_start_id > 0 && name[0]) {
- id = btf_find_by_name_kind_bsearch(btf, name);
- for (; id < total; id++) {
- t = btf_type_by_id(btf, id);
- tname = btf_name_by_offset(btf, t->name_off);
- if (strcmp(tname, name) != 0)
- return -ENOENT;
- if (BTF_INFO_KIND(t->info) == kind)
- return id;
- }
- } else {
- for (id = btf_start_id(btf); id < total; id++) {
- t = btf_type_by_id(btf, id);
- if (BTF_INFO_KIND(t->info) != kind)
- continue;
- tname = btf_name_by_offset(btf, t->name_off);
- if (strcmp(tname, name) == 0)
- return id;
- }
- }
-
- return -ENOENT;
-}
-
s32 bpf_find_btf_id(const char *name, u32 kind, struct btf **btf_p)
{
struct btf *btf;
@@ -719,4964 +136,976 @@ s32 bpf_find_btf_id(const char *name, u32 kind, struct btf **btf_p)
return ret;
}
EXPORT_SYMBOL_GPL(bpf_find_btf_id);
-
-const struct btf_type *btf_type_skip_modifiers(const struct btf *btf,
- u32 id, u32 *res_id)
+static int btf_alloc_id(struct btf *btf)
{
- const struct btf_type *t = btf_type_by_id(btf, id);
+ int id;
- while (btf_type_is_modifier(t)) {
- id = t->type;
- t = btf_type_by_id(btf, t->type);
- }
+ idr_preload(GFP_KERNEL);
+ spin_lock_bh(&btf_idr_lock);
+ id = idr_alloc_cyclic(&btf_idr, btf, 1, INT_MAX, GFP_ATOMIC);
+ if (id > 0)
+ btf->id = id;
+ spin_unlock_bh(&btf_idr_lock);
+ idr_preload_end();
- if (res_id)
- *res_id = id;
+ if (WARN_ON_ONCE(!id))
+ return -ENOSPC;
- return t;
+ return id > 0 ? 0 : id;
}
-const struct btf_type *btf_type_resolve_ptr(const struct btf *btf,
- u32 id, u32 *res_id)
+static void btf_free_id(struct btf *btf)
{
- const struct btf_type *t;
-
- t = btf_type_skip_modifiers(btf, id, NULL);
- if (!btf_type_is_ptr(t))
- return NULL;
+ unsigned long flags;
- return btf_type_skip_modifiers(btf, t->type, res_id);
+ /*
+ * In map-in-map, calling map_delete_elem() on outer
+ * map will call bpf_map_put on the inner map.
+ * It will then eventually call btf_free_id()
+ * on the inner map. Some of the map_delete_elem()
+ * implementation may have irq disabled, so
+ * we need to use the _irqsave() version instead
+ * of the _bh() version.
+ */
+ spin_lock_irqsave(&btf_idr_lock, flags);
+ if (btf->id) {
+ idr_remove(&btf_idr, btf->id);
+ /*
+ * Clear the id here to make this function idempotent, since it will get
+ * called a couple of times for module BTFs: on module unload, and then
+ * the final btf_put(). btf_alloc_id() starts IDs with 1, so we can use
+ * 0 as sentinel value.
+ */
+ WRITE_ONCE(btf->id, 0);
+ }
+ spin_unlock_irqrestore(&btf_idr_lock, flags);
}
-const struct btf_type *btf_type_resolve_func_ptr(const struct btf *btf,
- u32 id, u32 *res_id)
+static void btf_free_kfunc_set_tab(struct btf *btf)
{
- const struct btf_type *ptype;
-
- ptype = btf_type_resolve_ptr(btf, id, res_id);
- if (ptype && btf_type_is_func_proto(ptype))
- return ptype;
+ struct btf_kfunc_set_tab *tab = btf->kfunc_set_tab;
+ int hook;
- return NULL;
+ if (!tab)
+ return;
+ for (hook = 0; hook < ARRAY_SIZE(tab->sets); hook++)
+ kfree(tab->sets[hook]);
+ kfree(tab);
+ btf->kfunc_set_tab = NULL;
}
-/* Types that act only as a source, not sink or intermediate
- * type when resolving.
- */
-static bool btf_type_is_resolve_source_only(const struct btf_type *t)
+static void btf_free_dtor_kfunc_tab(struct btf *btf)
{
- return btf_type_is_var(t) ||
- btf_type_is_decl_tag(t) ||
- btf_type_is_datasec(t);
+ struct btf_id_dtor_kfunc_tab *tab = btf->dtor_kfunc_tab;
+
+ if (!tab)
+ return;
+ kfree(tab);
+ btf->dtor_kfunc_tab = NULL;
}
-/* What types need to be resolved?
- *
- * btf_type_is_modifier() is an obvious one.
- *
- * btf_type_is_struct() because its member refers to
- * another type (through member->type).
- *
- * btf_type_is_var() because the variable refers to
- * another type. btf_type_is_datasec() holds multiple
- * btf_type_is_var() types that need resolving.
- *
- * btf_type_is_array() because its element (array->type)
- * refers to another type. Array can be thought of a
- * special case of struct while array just has the same
- * member-type repeated by array->nelems of times.
- */
-static bool btf_type_needs_resolve(const struct btf_type *t)
+static void btf_struct_metas_free(struct btf_struct_metas *tab)
{
- return btf_type_is_modifier(t) ||
- btf_type_is_ptr(t) ||
- btf_type_is_struct(t) ||
- btf_type_is_array(t) ||
- btf_type_is_var(t) ||
- btf_type_is_func(t) ||
- btf_type_is_decl_tag(t) ||
- btf_type_is_datasec(t);
+ int i;
+
+ if (!tab)
+ return;
+ for (i = 0; i < tab->cnt; i++)
+ btf_record_free(tab->types[i].record);
+ kfree(tab);
}
-/* t->size can be used */
-static bool btf_type_has_size(const struct btf_type *t)
+static void btf_free_struct_meta_tab(struct btf *btf)
{
- switch (BTF_INFO_KIND(t->info)) {
- case BTF_KIND_INT:
- case BTF_KIND_STRUCT:
- case BTF_KIND_UNION:
- case BTF_KIND_ENUM:
- case BTF_KIND_DATASEC:
- case BTF_KIND_FLOAT:
- case BTF_KIND_ENUM64:
- return true;
- }
+ struct btf_struct_metas *tab = btf->struct_meta_tab;
- return false;
+ btf_struct_metas_free(tab);
+ btf->struct_meta_tab = NULL;
}
-static const char *btf_int_encoding_str(u8 encoding)
+static void btf_free_struct_ops_tab(struct btf *btf)
{
- if (encoding == 0)
- return "(none)";
- else if (encoding == BTF_INT_SIGNED)
- return "SIGNED";
- else if (encoding == BTF_INT_CHAR)
- return "CHAR";
- else if (encoding == BTF_INT_BOOL)
- return "BOOL";
- else
- return "UNKN";
-}
+ struct btf_struct_ops_tab *tab = btf->struct_ops_tab;
+ u32 i;
-static u32 btf_type_int(const struct btf_type *t)
-{
- return *(u32 *)(t + 1);
-}
+ if (!tab)
+ return;
-static const struct btf_array *btf_type_array(const struct btf_type *t)
-{
- return (const struct btf_array *)(t + 1);
-}
+ for (i = 0; i < tab->cnt; i++)
+ bpf_struct_ops_desc_release(&tab->ops[i]);
-static const struct btf_enum *btf_type_enum(const struct btf_type *t)
-{
- return (const struct btf_enum *)(t + 1);
+ kfree(tab);
+ btf->struct_ops_tab = NULL;
}
-static const struct btf_var *btf_type_var(const struct btf_type *t)
+void btf_free_bpf_data(struct btf *btf)
{
- return (const struct btf_var *)(t + 1);
+ btf_free_struct_meta_tab(btf);
+ btf_free_dtor_kfunc_tab(btf);
+ btf_free_kfunc_set_tab(btf);
+ btf_free_struct_ops_tab(btf);
}
-static const struct btf_decl_tag *btf_type_decl_tag(const struct btf_type *t)
+static void btf_free_rcu(struct rcu_head *rcu)
{
- return (const struct btf_decl_tag *)(t + 1);
-}
+ struct btf *btf = container_of(rcu, struct btf, rcu);
-static const struct btf_enum64 *btf_type_enum64(const struct btf_type *t)
-{
- return (const struct btf_enum64 *)(t + 1);
+ btf_free(btf);
}
-static const struct btf_kind_operations *btf_type_ops(const struct btf_type *t)
+void btf_put_bpf(struct btf *btf)
{
- return kind_ops[BTF_INFO_KIND(t->info)];
+ btf_free_id(btf);
+ call_rcu(&btf->rcu, btf_free_rcu);
}
+enum {
+ BTF_FIELD_IGNORE = 0,
+ BTF_FIELD_FOUND = 1,
+};
-static bool btf_name_offset_valid(const struct btf *btf, u32 offset)
-{
- if (!BTF_STR_OFFSET_VALID(offset))
- return false;
-
- while (offset < btf->start_str_off)
- btf = btf->base_btf;
-
- offset -= btf->start_str_off;
- return offset < btf->hdr.str_len;
-}
+struct btf_field_info {
+ enum btf_field_type type;
+ u32 off;
+ union {
+ struct {
+ u32 type_id;
+ } kptr;
+ struct {
+ const char *node_name;
+ u32 value_btf_id;
+ } graph_root;
+ };
+};
-static bool __btf_name_char_ok(char c, bool first)
+static int btf_find_struct(const struct btf *btf, const struct btf_type *t,
+ u32 off, int sz, enum btf_field_type field_type,
+ struct btf_field_info *info)
{
- if ((first ? !isalpha(c) :
- !isalnum(c)) &&
- c != '_' &&
- c != '.')
- return false;
- return true;
+ if (!__btf_type_is_struct(t))
+ return BTF_FIELD_IGNORE;
+ if (t->size != sz)
+ return BTF_FIELD_IGNORE;
+ info->type = field_type;
+ info->off = off;
+ return BTF_FIELD_FOUND;
}
-const char *btf_str_by_offset(const struct btf *btf, u32 offset)
+static int btf_find_kptr(const struct btf *btf, const struct btf_type *t,
+ u32 off, int sz, struct btf_field_info *info, u32 field_mask)
{
- while (offset < btf->start_str_off)
- btf = btf->base_btf;
-
- offset -= btf->start_str_off;
- if (offset < btf->hdr.str_len)
- return &btf->strings[offset];
-
- return NULL;
-}
+ enum btf_field_type type;
+ const char *tag_value;
+ bool is_type_tag;
+ u32 res_id;
-static bool btf_name_valid_identifier(const struct btf *btf, u32 offset)
-{
- /* offset must be valid */
- const char *src = btf_str_by_offset(btf, offset);
- const char *src_limit;
+ /* Permit modifiers on the pointer itself */
+ if (btf_type_is_volatile(t))
+ t = btf_type_by_id(btf, t->type);
+ /* For PTR, sz is always == 8 */
+ if (!btf_type_is_ptr(t))
+ return BTF_FIELD_IGNORE;
+ t = btf_type_by_id(btf, t->type);
+ is_type_tag = btf_type_is_type_tag(t) && !btf_type_kflag(t);
+ if (!is_type_tag)
+ return BTF_FIELD_IGNORE;
+ /* Reject extra tags */
+ if (btf_type_is_type_tag(btf_type_by_id(btf, t->type)))
+ return -EINVAL;
+ tag_value = __btf_name_by_offset(btf, t->name_off);
+ if (!strcmp("kptr_untrusted", tag_value))
+ type = BPF_KPTR_UNREF;
+ else if (!strcmp("kptr", tag_value))
+ type = BPF_KPTR_REF;
+ else if (!strcmp("percpu_kptr", tag_value))
+ type = BPF_KPTR_PERCPU;
+ else if (!strcmp("uptr", tag_value))
+ type = BPF_UPTR;
+ else
+ return -EINVAL;
- if (!__btf_name_char_ok(*src, true))
- return false;
+ if (!(type & field_mask))
+ return BTF_FIELD_IGNORE;
- /* set a limit on identifier length */
- src_limit = src + KSYM_NAME_LEN;
- src++;
- while (*src && src < src_limit) {
- if (!__btf_name_char_ok(*src, false))
- return false;
- src++;
- }
+ /* Get the base type */
+ t = btf_type_skip_modifiers(btf, t->type, &res_id);
+ /* Only pointer to struct is allowed */
+ if (!__btf_type_is_struct(t))
+ return -EINVAL;
- return !*src;
+ info->type = type;
+ info->off = off;
+ info->kptr.type_id = res_id;
+ return BTF_FIELD_FOUND;
}
-/* Allow any printable character in DATASEC names */
-static bool btf_name_valid_section(const struct btf *btf, u32 offset)
+int btf_find_next_decl_tag(const struct btf *btf, const struct btf_type *pt,
+ int comp_idx, const char *tag_key, int last_id)
{
- /* offset must be valid */
- const char *src = btf_str_by_offset(btf, offset);
- const char *src_limit;
+ int len = strlen(tag_key);
+ int i, n;
- if (!*src)
- return false;
+ for (i = last_id + 1, n = btf_nr_types(btf); i < n; i++) {
+ const struct btf_type *t = btf_type_by_id(btf, i);
- /* set a limit on identifier length */
- src_limit = src + KSYM_NAME_LEN;
- while (*src && src < src_limit) {
- if (!isprint(*src))
- return false;
- src++;
+ if (!btf_type_is_decl_tag(t))
+ continue;
+ if (pt != btf_type_by_id(btf, t->type))
+ continue;
+ if (btf_type_decl_tag(t)->component_idx != comp_idx)
+ continue;
+ if (strncmp(__btf_name_by_offset(btf, t->name_off), tag_key, len))
+ continue;
+ return i;
}
-
- return !*src;
-}
-
-static const char *__btf_name_by_offset(const struct btf *btf, u32 offset)
-{
- const char *name;
-
- if (!offset)
- return "(anon)";
-
- name = btf_str_by_offset(btf, offset);
- return name ?: "(invalid-name-offset)";
+ return -ENOENT;
}
-const char *btf_name_by_offset(const struct btf *btf, u32 offset)
+const char *btf_find_decl_tag_value(const struct btf *btf, const struct btf_type *pt,
+ int comp_idx, const char *tag_key)
{
- return btf_str_by_offset(btf, offset);
-}
+ const char *value = NULL;
+ const struct btf_type *t;
+ int len, id;
-const struct btf_type *btf_type_by_id(const struct btf *btf, u32 type_id)
-{
- while (type_id < btf->start_id)
- btf = btf->base_btf;
+ id = btf_find_next_decl_tag(btf, pt, comp_idx, tag_key,
+ btf_named_start_id(btf, false) - 1);
+ if (id < 0)
+ return ERR_PTR(id);
- type_id -= btf->start_id;
- if (type_id >= btf->nr_types)
- return NULL;
- return btf->types[type_id];
-}
-EXPORT_SYMBOL_GPL(btf_type_by_id);
-
-/*
- * Check that the type @t is a regular int. This means that @t is not
- * a bit field and it has the same size as either of u8/u16/u32/u64
- * or __int128. If @expected_size is not zero, then size of @t should
- * be the same. A caller should already have checked that the type @t
- * is an integer.
- */
-static bool __btf_type_int_is_regular(const struct btf_type *t, size_t expected_size)
-{
- u32 int_data = btf_type_int(t);
- u8 nr_bits = BTF_INT_BITS(int_data);
- u8 nr_bytes = BITS_ROUNDUP_BYTES(nr_bits);
-
- return BITS_PER_BYTE_MASKED(nr_bits) == 0 &&
- BTF_INT_OFFSET(int_data) == 0 &&
- (nr_bytes <= 16 && is_power_of_2(nr_bytes)) &&
- (expected_size == 0 || nr_bytes == expected_size);
-}
+ t = btf_type_by_id(btf, id);
+ len = strlen(tag_key);
+ value = __btf_name_by_offset(btf, t->name_off) + len;
-static bool btf_type_int_is_regular(const struct btf_type *t)
-{
- return __btf_type_int_is_regular(t, 0);
-}
+ /* Prevent duplicate entries for same type */
+ id = btf_find_next_decl_tag(btf, pt, comp_idx, tag_key, id);
+ if (id >= 0)
+ return ERR_PTR(-EEXIST);
-bool btf_type_is_i32(const struct btf_type *t)
-{
- return btf_type_is_int(t) && __btf_type_int_is_regular(t, 4);
+ return value;
}
-bool btf_type_is_i64(const struct btf_type *t)
+static int
+btf_find_graph_root(const struct btf *btf, const struct btf_type *pt,
+ const struct btf_type *t, int comp_idx, u32 off,
+ int sz, struct btf_field_info *info,
+ enum btf_field_type head_type)
{
- return btf_type_is_int(t) && __btf_type_int_is_regular(t, 8);
-}
+ const char *node_field_name;
+ const char *value_type;
+ s32 id;
-bool btf_type_is_primitive(const struct btf_type *t)
-{
- return (btf_type_is_int(t) && btf_type_int_is_regular(t)) ||
- btf_is_any_enum(t);
+ if (!__btf_type_is_struct(t))
+ return BTF_FIELD_IGNORE;
+ if (t->size != sz)
+ return BTF_FIELD_IGNORE;
+ value_type = btf_find_decl_tag_value(btf, pt, comp_idx, "contains:");
+ if (IS_ERR(value_type))
+ return -EINVAL;
+ node_field_name = strstr(value_type, ":");
+ if (!node_field_name)
+ return -EINVAL;
+ value_type = kstrndup(value_type, node_field_name - value_type,
+ GFP_KERNEL_ACCOUNT | __GFP_NOWARN);
+ if (!value_type)
+ return -ENOMEM;
+ id = btf_find_by_name_kind(btf, value_type, BTF_KIND_STRUCT);
+ kfree(value_type);
+ if (id < 0)
+ return id;
+ node_field_name++;
+ if (str_is_empty(node_field_name))
+ return -EINVAL;
+ info->type = head_type;
+ info->off = off;
+ info->graph_root.value_btf_id = id;
+ info->graph_root.node_name = node_field_name;
+ return BTF_FIELD_FOUND;
}
-/*
- * Check that given struct member is a regular int with expected
- * offset and size.
- */
-bool btf_member_is_reg_int(const struct btf *btf, const struct btf_type *s,
- const struct btf_member *m,
- u32 expected_offset, u32 expected_size)
+static int btf_get_field_type(const struct btf *btf, const struct btf_type *var_type,
+ u32 field_mask, u32 *seen_mask, int *align, int *sz)
{
- const struct btf_type *t;
- u32 id, int_data;
- u8 nr_bits;
+ const struct {
+ enum btf_field_type type;
+ const char *const name;
+ const bool is_unique;
+ } field_types[] = {
+ { BPF_SPIN_LOCK, "bpf_spin_lock", true },
+ { BPF_RES_SPIN_LOCK, "bpf_res_spin_lock", true },
+ { BPF_TIMER, "bpf_timer", true },
+ { BPF_WORKQUEUE, "bpf_wq", true },
+ { BPF_TASK_WORK, "bpf_task_work", true },
+ { BPF_LIST_HEAD, "bpf_list_head", false },
+ { BPF_LIST_NODE, "bpf_list_node", false },
+ { BPF_RB_ROOT, "bpf_rb_root", false },
+ { BPF_RB_NODE, "bpf_rb_node", false },
+ { BPF_REFCOUNT, "bpf_refcount", false },
+ };
+ int type = 0, i;
+ const char *name = __btf_name_by_offset(btf, var_type->name_off);
+ const char *field_type_name;
+ enum btf_field_type field_type;
+ bool is_unique;
- id = m->type;
- t = btf_type_id_size(btf, &id, NULL);
- if (!t || !btf_type_is_int(t))
- return false;
+ for (i = 0; i < ARRAY_SIZE(field_types); ++i) {
+ field_type = field_types[i].type;
+ field_type_name = field_types[i].name;
+ is_unique = field_types[i].is_unique;
+ if (!(field_mask & field_type) || strcmp(name, field_type_name))
+ continue;
+ if (is_unique) {
+ if (*seen_mask & field_type)
+ return -E2BIG;
+ *seen_mask |= field_type;
+ }
+ type = field_type;
+ goto end;
+ }
- int_data = btf_type_int(t);
- nr_bits = BTF_INT_BITS(int_data);
- if (btf_type_kflag(s)) {
- u32 bitfield_size = BTF_MEMBER_BITFIELD_SIZE(m->offset);
- u32 bit_offset = BTF_MEMBER_BIT_OFFSET(m->offset);
-
- /* if kflag set, int should be a regular int and
- * bit offset should be at byte boundary.
- */
- return !bitfield_size &&
- BITS_ROUNDUP_BYTES(bit_offset) == expected_offset &&
- BITS_ROUNDUP_BYTES(nr_bits) == expected_size;
- }
-
- if (BTF_INT_OFFSET(int_data) ||
- BITS_PER_BYTE_MASKED(m->offset) ||
- BITS_ROUNDUP_BYTES(m->offset) != expected_offset ||
- BITS_PER_BYTE_MASKED(nr_bits) ||
- BITS_ROUNDUP_BYTES(nr_bits) != expected_size)
- return false;
-
- return true;
-}
-
-/* Similar to btf_type_skip_modifiers() but does not skip typedefs. */
-static const struct btf_type *btf_type_skip_qualifiers(const struct btf *btf,
- u32 id)
-{
- const struct btf_type *t = btf_type_by_id(btf, id);
-
- while (btf_type_is_modifier(t) &&
- BTF_INFO_KIND(t->info) != BTF_KIND_TYPEDEF) {
- t = btf_type_by_id(btf, t->type);
+ /* Only return BPF_KPTR when all other types with matchable names fail */
+ if (field_mask & (BPF_KPTR | BPF_UPTR) && !__btf_type_is_struct(var_type)) {
+ type = BPF_KPTR_REF;
+ goto end;
}
-
- return t;
+ return 0;
+end:
+ *sz = btf_field_type_size(type);
+ *align = btf_field_type_align(type);
+ return type;
}
-#define BTF_SHOW_MAX_ITER 10
-
-#define BTF_KIND_BIT(kind) (1ULL << kind)
-
-/*
- * Populate show->state.name with type name information.
- * Format of type name is
+/* Repeat a number of fields for a specified number of times.
*
- * [.member_name = ] (type_name)
+ * Copy the fields starting from the first field and repeat them for
+ * repeat_cnt times. The fields are repeated by adding the offset of each
+ * field with
+ * (i + 1) * elem_size
+ * where i is the repeat index and elem_size is the size of an element.
*/
-static const char *btf_show_name(struct btf_show *show)
+static int btf_repeat_fields(struct btf_field_info *info, int info_cnt,
+ u32 field_cnt, u32 repeat_cnt, u32 elem_size)
{
- /* BTF_MAX_ITER array suffixes "[]" */
- const char *array_suffixes = "[][][][][][][][][][]";
- const char *array_suffix = &array_suffixes[strlen(array_suffixes)];
- /* BTF_MAX_ITER pointer suffixes "*" */
- const char *ptr_suffixes = "**********";
- const char *ptr_suffix = &ptr_suffixes[strlen(ptr_suffixes)];
- const char *name = NULL, *prefix = "", *parens = "";
- const struct btf_member *m = show->state.member;
- const struct btf_type *t;
- const struct btf_array *array;
- u32 id = show->state.type_id;
- const char *member = NULL;
- bool show_member = false;
- u64 kinds = 0;
- int i;
-
- show->state.name[0] = '\0';
-
- /*
- * Don't show type name if we're showing an array member;
- * in that case we show the array type so don't need to repeat
- * ourselves for each member.
- */
- if (show->state.array_member)
- return "";
-
- /* Retrieve member name, if any. */
- if (m) {
- member = btf_name_by_offset(show->btf, m->name_off);
- show_member = strlen(member) > 0;
- id = m->type;
- }
-
- /*
- * Start with type_id, as we have resolved the struct btf_type *
- * via btf_modifier_show() past the parent typedef to the child
- * struct, int etc it is defined as. In such cases, the type_id
- * still represents the starting type while the struct btf_type *
- * in our show->state points at the resolved type of the typedef.
- */
- t = btf_type_by_id(show->btf, id);
- if (!t)
- return "";
+ u32 i, j;
+ u32 cur;
- /*
- * The goal here is to build up the right number of pointer and
- * array suffixes while ensuring the type name for a typedef
- * is represented. Along the way we accumulate a list of
- * BTF kinds we have encountered, since these will inform later
- * display; for example, pointer types will not require an
- * opening "{" for struct, we will just display the pointer value.
- *
- * We also want to accumulate the right number of pointer or array
- * indices in the format string while iterating until we get to
- * the typedef/pointee/array member target type.
- *
- * We start by pointing at the end of pointer and array suffix
- * strings; as we accumulate pointers and arrays we move the pointer
- * or array string backwards so it will show the expected number of
- * '*' or '[]' for the type. BTF_SHOW_MAX_ITER of nesting of pointers
- * and/or arrays and typedefs are supported as a precaution.
- *
- * We also want to get typedef name while proceeding to resolve
- * type it points to so that we can add parentheses if it is a
- * "typedef struct" etc.
- */
- for (i = 0; i < BTF_SHOW_MAX_ITER; i++) {
-
- switch (BTF_INFO_KIND(t->info)) {
- case BTF_KIND_TYPEDEF:
- if (!name)
- name = btf_name_by_offset(show->btf,
- t->name_off);
- kinds |= BTF_KIND_BIT(BTF_KIND_TYPEDEF);
- id = t->type;
- break;
- case BTF_KIND_ARRAY:
- kinds |= BTF_KIND_BIT(BTF_KIND_ARRAY);
- parens = "[";
- if (!t)
- return "";
- array = btf_type_array(t);
- if (array_suffix > array_suffixes)
- array_suffix -= 2;
- id = array->type;
- break;
- case BTF_KIND_PTR:
- kinds |= BTF_KIND_BIT(BTF_KIND_PTR);
- if (ptr_suffix > ptr_suffixes)
- ptr_suffix -= 1;
- id = t->type;
+ /* Ensure not repeating fields that should not be repeated. */
+ for (i = 0; i < field_cnt; i++) {
+ switch (info[i].type) {
+ case BPF_KPTR_UNREF:
+ case BPF_KPTR_REF:
+ case BPF_KPTR_PERCPU:
+ case BPF_UPTR:
+ case BPF_LIST_HEAD:
+ case BPF_RB_ROOT:
break;
default:
- id = 0;
- break;
+ return -EINVAL;
}
- if (!id)
- break;
- t = btf_type_skip_qualifiers(show->btf, id);
- }
- /* We may not be able to represent this type; bail to be safe */
- if (i == BTF_SHOW_MAX_ITER)
- return "";
-
- if (!name)
- name = btf_name_by_offset(show->btf, t->name_off);
-
- switch (BTF_INFO_KIND(t->info)) {
- case BTF_KIND_STRUCT:
- case BTF_KIND_UNION:
- prefix = BTF_INFO_KIND(t->info) == BTF_KIND_STRUCT ?
- "struct" : "union";
- /* if it's an array of struct/union, parens is already set */
- if (!(kinds & (BTF_KIND_BIT(BTF_KIND_ARRAY))))
- parens = "{";
- break;
- case BTF_KIND_ENUM:
- case BTF_KIND_ENUM64:
- prefix = "enum";
- break;
- default:
- break;
}
- /* pointer does not require parens */
- if (kinds & BTF_KIND_BIT(BTF_KIND_PTR))
- parens = "";
- /* typedef does not require struct/union/enum prefix */
- if (kinds & BTF_KIND_BIT(BTF_KIND_TYPEDEF))
- prefix = "";
-
- if (!name)
- name = "";
-
- /* Even if we don't want type name info, we want parentheses etc */
- if (show->flags & BTF_SHOW_NONAME)
- snprintf(show->state.name, sizeof(show->state.name), "%s",
- parens);
- else
- snprintf(show->state.name, sizeof(show->state.name),
- "%s%s%s(%s%s%s%s%s%s)%s",
- /* first 3 strings comprise ".member = " */
- show_member ? "." : "",
- show_member ? member : "",
- show_member ? " = " : "",
- /* ...next is our prefix (struct, enum, etc) */
- prefix,
- strlen(prefix) > 0 && strlen(name) > 0 ? " " : "",
- /* ...this is the type name itself */
- name,
- /* ...suffixed by the appropriate '*', '[]' suffixes */
- strlen(ptr_suffix) > 0 ? " " : "", ptr_suffix,
- array_suffix, parens);
-
- return show->state.name;
-}
+ /* The type of struct size or variable size is u32,
+ * so the multiplication will not overflow.
+ */
+ if (field_cnt * (repeat_cnt + 1) > info_cnt)
+ return -E2BIG;
-static const char *__btf_show_indent(struct btf_show *show)
-{
- const char *indents = " ";
- const char *indent = &indents[strlen(indents)];
+ cur = field_cnt;
+ for (i = 0; i < repeat_cnt; i++) {
+ memcpy(&info[cur], &info[0], field_cnt * sizeof(info[0]));
+ for (j = 0; j < field_cnt; j++)
+ info[cur++].off += (i + 1) * elem_size;
+ }
- if ((indent - show->state.depth) >= indents)
- return indent - show->state.depth;
- return indents;
+ return 0;
}
-static const char *btf_show_indent(struct btf_show *show)
-{
- return show->flags & BTF_SHOW_COMPACT ? "" : __btf_show_indent(show);
-}
+static int btf_find_struct_field(const struct btf *btf,
+ const struct btf_type *t, u32 field_mask,
+ struct btf_field_info *info, int info_cnt,
+ u32 level);
-static const char *btf_show_newline(struct btf_show *show)
+/* Find special fields in the struct type of a field.
+ *
+ * This function is used to find fields of special types that is not a
+ * global variable or a direct field of a struct type. It also handles the
+ * repetition if it is the element type of an array.
+ */
+static int btf_find_nested_struct(const struct btf *btf, const struct btf_type *t,
+ u32 off, u32 nelems,
+ u32 field_mask, struct btf_field_info *info,
+ int info_cnt, u32 level)
{
- return show->flags & BTF_SHOW_COMPACT ? "" : "\n";
-}
+ int ret, err, i;
-static const char *btf_show_delim(struct btf_show *show)
-{
- if (show->state.depth == 0)
- return "";
+ level++;
+ if (level >= MAX_RESOLVE_DEPTH)
+ return -E2BIG;
- if ((show->flags & BTF_SHOW_COMPACT) && show->state.type &&
- BTF_INFO_KIND(show->state.type->info) == BTF_KIND_UNION)
- return "|";
+ ret = btf_find_struct_field(btf, t, field_mask, info, info_cnt, level);
- return ",";
-}
+ if (ret <= 0)
+ return ret;
-__printf(2, 3) static void btf_show(struct btf_show *show, const char *fmt, ...)
-{
- va_list args;
+ /* Shift the offsets of the nested struct fields to the offsets
+ * related to the container.
+ */
+ for (i = 0; i < ret; i++)
+ info[i].off += off;
- if (!show->state.depth_check) {
- va_start(args, fmt);
- show->showfn(show, fmt, args);
- va_end(args);
+ if (nelems > 1) {
+ err = btf_repeat_fields(info, info_cnt, ret, nelems - 1, t->size);
+ if (err == 0)
+ ret *= nelems;
+ else
+ ret = err;
}
-}
-
-/* Macros are used here as btf_show_type_value[s]() prepends and appends
- * format specifiers to the format specifier passed in; these do the work of
- * adding indentation, delimiters etc while the caller simply has to specify
- * the type value(s) in the format specifier + value(s).
- */
-#define btf_show_type_value(show, fmt, value) \
- do { \
- if ((value) != (__typeof__(value))0 || \
- (show->flags & BTF_SHOW_ZERO) || \
- show->state.depth == 0) { \
- btf_show(show, "%s%s" fmt "%s%s", \
- btf_show_indent(show), \
- btf_show_name(show), \
- value, btf_show_delim(show), \
- btf_show_newline(show)); \
- if (show->state.depth > show->state.depth_to_show) \
- show->state.depth_to_show = show->state.depth; \
- } \
- } while (0)
-
-#define btf_show_type_values(show, fmt, ...) \
- do { \
- btf_show(show, "%s%s" fmt "%s%s", btf_show_indent(show), \
- btf_show_name(show), \
- __VA_ARGS__, btf_show_delim(show), \
- btf_show_newline(show)); \
- if (show->state.depth > show->state.depth_to_show) \
- show->state.depth_to_show = show->state.depth; \
- } while (0)
-
-/* How much is left to copy to safe buffer after @data? */
-static int btf_show_obj_size_left(struct btf_show *show, void *data)
-{
- return show->obj.head + show->obj.size - data;
-}
-
-/* Is object pointed to by @data of @size already copied to our safe buffer? */
-static bool btf_show_obj_is_safe(struct btf_show *show, void *data, int size)
-{
- return data >= show->obj.data &&
- (data + size) < (show->obj.data + BTF_SHOW_OBJ_SAFE_SIZE);
-}
-/*
- * If object pointed to by @data of @size falls within our safe buffer, return
- * the equivalent pointer to the same safe data. Assumes
- * copy_from_kernel_nofault() has already happened and our safe buffer is
- * populated.
- */
-static void *__btf_show_obj_safe(struct btf_show *show, void *data, int size)
-{
- if (btf_show_obj_is_safe(show, data, size))
- return show->obj.safe + (data - show->obj.data);
- return NULL;
+ return ret;
}
-/*
- * Return a safe-to-access version of data pointed to by @data.
- * We do this by copying the relevant amount of information
- * to the struct btf_show obj.safe buffer using copy_from_kernel_nofault().
- *
- * If BTF_SHOW_UNSAFE is specified, just return data as-is; no
- * safe copy is needed.
- *
- * Otherwise we need to determine if we have the required amount
- * of data (determined by the @data pointer and the size of the
- * largest base type we can encounter (represented by
- * BTF_SHOW_OBJ_BASE_TYPE_SIZE). Having that much data ensures
- * that we will be able to print some of the current object,
- * and if more is needed a copy will be triggered.
- * Some objects such as structs will not fit into the buffer;
- * in such cases additional copies when we iterate over their
- * members may be needed.
- *
- * btf_show_obj_safe() is used to return a safe buffer for
- * btf_show_start_type(); this ensures that as we recurse into
- * nested types we always have safe data for the given type.
- * This approach is somewhat wasteful; it's possible for example
- * that when iterating over a large union we'll end up copying the
- * same data repeatedly, but the goal is safety not performance.
- * We use stack data as opposed to per-CPU buffers because the
- * iteration over a type can take some time, and preemption handling
- * would greatly complicate use of the safe buffer.
- */
-static void *btf_show_obj_safe(struct btf_show *show,
- const struct btf_type *t,
- void *data)
+static int btf_find_field_one(const struct btf *btf,
+ const struct btf_type *var,
+ const struct btf_type *var_type,
+ int var_idx,
+ u32 off, u32 expected_size,
+ u32 field_mask, u32 *seen_mask,
+ struct btf_field_info *info, int info_cnt,
+ u32 level)
{
- const struct btf_type *rt;
- int size_left, size;
- void *safe = NULL;
-
- if (show->flags & BTF_SHOW_UNSAFE)
- return data;
-
- rt = btf_resolve_size(show->btf, t, &size);
- if (IS_ERR(rt)) {
- show->state.status = PTR_ERR(rt);
- return NULL;
- }
+ int ret, align, sz, field_type;
+ struct btf_field_info tmp;
+ const struct btf_array *array;
+ u32 i, nelems = 1;
- /*
- * Is this toplevel object? If so, set total object size and
- * initialize pointers. Otherwise check if we still fall within
- * our safe object data.
+ /* Walk into array types to find the element type and the number of
+ * elements in the (flattened) array.
*/
- if (show->state.depth == 0) {
- show->obj.size = size;
- show->obj.head = data;
- } else {
- /*
- * If the size of the current object is > our remaining
- * safe buffer we _may_ need to do a new copy. However
- * consider the case of a nested struct; it's size pushes
- * us over the safe buffer limit, but showing any individual
- * struct members does not. In such cases, we don't need
- * to initiate a fresh copy yet; however we definitely need
- * at least BTF_SHOW_OBJ_BASE_TYPE_SIZE bytes left
- * in our buffer, regardless of the current object size.
- * The logic here is that as we resolve types we will
- * hit a base type at some point, and we need to be sure
- * the next chunk of data is safely available to display
- * that type info safely. We cannot rely on the size of
- * the current object here because it may be much larger
- * than our current buffer (e.g. task_struct is 8k).
- * All we want to do here is ensure that we can print the
- * next basic type, which we can if either
- * - the current type size is within the safe buffer; or
- * - at least BTF_SHOW_OBJ_BASE_TYPE_SIZE bytes are left in
- * the safe buffer.
- */
- safe = __btf_show_obj_safe(show, data,
- min(size,
- BTF_SHOW_OBJ_BASE_TYPE_SIZE));
+ for (i = 0; i < MAX_RESOLVE_DEPTH && btf_type_is_array(var_type); i++) {
+ array = btf_array(var_type);
+ nelems *= array->nelems;
+ var_type = btf_type_by_id(btf, array->type);
}
+ if (i == MAX_RESOLVE_DEPTH)
+ return -E2BIG;
+ if (nelems == 0)
+ return 0;
- /*
- * We need a new copy to our safe object, either because we haven't
- * yet copied and are initializing safe data, or because the data
- * we want falls outside the boundaries of the safe object.
- */
- if (!safe) {
- size_left = btf_show_obj_size_left(show, data);
- if (size_left > BTF_SHOW_OBJ_SAFE_SIZE)
- size_left = BTF_SHOW_OBJ_SAFE_SIZE;
- show->state.status = copy_from_kernel_nofault(show->obj.safe,
- data, size_left);
- if (!show->state.status) {
- show->obj.data = data;
- safe = show->obj.safe;
- }
+ field_type = btf_get_field_type(btf, var_type,
+ field_mask, seen_mask, &align, &sz);
+ /* Look into variables of struct types */
+ if (!field_type && __btf_type_is_struct(var_type)) {
+ sz = var_type->size;
+ if (expected_size && expected_size != sz * nelems)
+ return 0;
+ ret = btf_find_nested_struct(btf, var_type, off, nelems, field_mask,
+ &info[0], info_cnt, level);
+ return ret;
}
- return safe;
-}
+ if (field_type == 0)
+ return 0;
+ if (field_type < 0)
+ return field_type;
-/*
- * Set the type we are starting to show and return a safe data pointer
- * to be used for showing the associated data.
- */
-static void *btf_show_start_type(struct btf_show *show,
- const struct btf_type *t,
- u32 type_id, void *data)
-{
- show->state.type = t;
- show->state.type_id = type_id;
- show->state.name[0] = '\0';
+ if (expected_size && expected_size != sz * nelems)
+ return 0;
+ if (off % align)
+ return 0;
- return btf_show_obj_safe(show, t, data);
-}
+ switch (field_type) {
+ case BPF_SPIN_LOCK:
+ case BPF_RES_SPIN_LOCK:
+ case BPF_TIMER:
+ case BPF_WORKQUEUE:
+ case BPF_LIST_NODE:
+ case BPF_RB_NODE:
+ case BPF_REFCOUNT:
+ case BPF_TASK_WORK:
+ ret = btf_find_struct(btf, var_type, off, sz, field_type,
+ info_cnt ? &info[0] : &tmp);
+ if (ret < 0)
+ return ret;
+ break;
+ case BPF_KPTR_UNREF:
+ case BPF_KPTR_REF:
+ case BPF_KPTR_PERCPU:
+ case BPF_UPTR:
+ ret = btf_find_kptr(btf, var_type, off, sz,
+ info_cnt ? &info[0] : &tmp, field_mask);
+ if (ret < 0)
+ return ret;
+ break;
+ case BPF_LIST_HEAD:
+ case BPF_RB_ROOT:
+ ret = btf_find_graph_root(btf, var, var_type,
+ var_idx, off, sz,
+ info_cnt ? &info[0] : &tmp,
+ field_type);
+ if (ret < 0)
+ return ret;
+ break;
+ default:
+ return -EFAULT;
+ }
-static void btf_show_end_type(struct btf_show *show)
-{
- show->state.type = NULL;
- show->state.type_id = 0;
- show->state.name[0] = '\0';
+ if (ret == BTF_FIELD_IGNORE)
+ return 0;
+ if (!info_cnt)
+ return -E2BIG;
+ if (nelems > 1) {
+ ret = btf_repeat_fields(info, info_cnt, 1, nelems - 1, sz);
+ if (ret < 0)
+ return ret;
+ }
+ return nelems;
}
-static void *btf_show_start_aggr_type(struct btf_show *show,
- const struct btf_type *t,
- u32 type_id, void *data)
+static int btf_find_struct_field(const struct btf *btf,
+ const struct btf_type *t, u32 field_mask,
+ struct btf_field_info *info, int info_cnt,
+ u32 level)
{
- void *safe_data = btf_show_start_type(show, t, type_id, data);
-
- if (!safe_data)
- return safe_data;
+ int ret, idx = 0;
+ const struct btf_member *member;
+ u32 i, off, seen_mask = 0;
- btf_show(show, "%s%s%s", btf_show_indent(show),
- btf_show_name(show),
- btf_show_newline(show));
- show->state.depth++;
- return safe_data;
-}
+ for_each_member(i, t, member) {
+ const struct btf_type *member_type = btf_type_by_id(btf,
+ member->type);
-static void btf_show_end_aggr_type(struct btf_show *show,
- const char *suffix)
-{
- show->state.depth--;
- btf_show(show, "%s%s%s%s", btf_show_indent(show), suffix,
- btf_show_delim(show), btf_show_newline(show));
- btf_show_end_type(show);
-}
-
-static void btf_show_start_member(struct btf_show *show,
- const struct btf_member *m)
-{
- show->state.member = m;
-}
-
-static void btf_show_start_array_member(struct btf_show *show)
-{
- show->state.array_member = 1;
- btf_show_start_member(show, NULL);
-}
-
-static void btf_show_end_member(struct btf_show *show)
-{
- show->state.member = NULL;
-}
-
-static void btf_show_end_array_member(struct btf_show *show)
-{
- show->state.array_member = 0;
- btf_show_end_member(show);
-}
+ off = __btf_member_bit_offset(t, member);
+ if (off % 8)
+ /* valid C code cannot generate such BTF */
+ return -EINVAL;
+ off /= 8;
-static void *btf_show_start_array_type(struct btf_show *show,
- const struct btf_type *t,
- u32 type_id,
- u16 array_encoding,
- void *data)
-{
- show->state.array_encoding = array_encoding;
- show->state.array_terminated = 0;
- return btf_show_start_aggr_type(show, t, type_id, data);
+ ret = btf_find_field_one(btf, t, member_type, i,
+ off, 0,
+ field_mask, &seen_mask,
+ &info[idx], info_cnt - idx, level);
+ if (ret < 0)
+ return ret;
+ idx += ret;
+ }
+ return idx;
}
-static void btf_show_end_array_type(struct btf_show *show)
+static int btf_find_datasec_var(const struct btf *btf, const struct btf_type *t,
+ u32 field_mask, struct btf_field_info *info,
+ int info_cnt, u32 level)
{
- show->state.array_encoding = 0;
- show->state.array_terminated = 0;
- btf_show_end_aggr_type(show, "]");
-}
+ int ret, idx = 0;
+ const struct btf_var_secinfo *vsi;
+ u32 i, off, seen_mask = 0;
-static void *btf_show_start_struct_type(struct btf_show *show,
- const struct btf_type *t,
- u32 type_id,
- void *data)
-{
- return btf_show_start_aggr_type(show, t, type_id, data);
-}
+ for_each_vsi(i, t, vsi) {
+ const struct btf_type *var = btf_type_by_id(btf, vsi->type);
+ const struct btf_type *var_type = btf_type_by_id(btf, var->type);
-static void btf_show_end_struct_type(struct btf_show *show)
-{
- btf_show_end_aggr_type(show, "}");
+ off = vsi->offset;
+ ret = btf_find_field_one(btf, var, var_type, -1, off, vsi->size,
+ field_mask, &seen_mask,
+ &info[idx], info_cnt - idx,
+ level);
+ if (ret < 0)
+ return ret;
+ idx += ret;
+ }
+ return idx;
}
-__printf(2, 3) static void __btf_verifier_log(struct bpf_verifier_log *log,
- const char *fmt, ...)
+static int btf_find_field(const struct btf *btf, const struct btf_type *t,
+ u32 field_mask, struct btf_field_info *info,
+ int info_cnt)
{
- va_list args;
-
- va_start(args, fmt);
- bpf_verifier_vlog(log, fmt, args);
- va_end(args);
+ if (__btf_type_is_struct(t))
+ return btf_find_struct_field(btf, t, field_mask, info, info_cnt, 0);
+ else if (btf_type_is_datasec(t))
+ return btf_find_datasec_var(btf, t, field_mask, info, info_cnt, 0);
+ return -EINVAL;
}
-__printf(2, 3) static void btf_verifier_log(struct btf_verifier_env *env,
- const char *fmt, ...)
+/* Callers have to ensure the life cycle of btf if it is program BTF */
+static int btf_parse_kptr(const struct btf *btf, struct btf_field *field,
+ struct btf_field_info *info)
{
- struct bpf_verifier_log *log = &env->log;
- va_list args;
-
- if (!bpf_verifier_log_needed(log))
- return;
+ struct module *mod = NULL;
+ const struct btf_type *t;
+ /* If a matching btf type is found in kernel or module BTFs, kptr_ref
+ * is that BTF, otherwise it's program BTF
+ */
+ struct btf *kptr_btf;
+ int ret;
+ s32 id;
- va_start(args, fmt);
- bpf_verifier_vlog(log, fmt, args);
- va_end(args);
-}
+ /* Find type in map BTF, and use it to look up the matching type
+ * in vmlinux or module BTFs, by name and kind.
+ */
+ t = btf_type_by_id(btf, info->kptr.type_id);
+ id = bpf_find_btf_id(__btf_name_by_offset(btf, t->name_off), BTF_INFO_KIND(t->info),
+ &kptr_btf);
+ if (id == -ENOENT) {
+ /* btf_parse_kptr should only be called w/ btf = program BTF */
+ WARN_ON_ONCE(btf_is_kernel(btf));
-__printf(4, 5) static void __btf_verifier_log_type(struct btf_verifier_env *env,
- const struct btf_type *t,
- bool log_details,
- const char *fmt, ...)
-{
- struct bpf_verifier_log *log = &env->log;
- struct btf *btf = env->btf;
- va_list args;
+ /* Type exists only in program BTF. Assume that it's a MEM_ALLOC
+ * kptr allocated via bpf_obj_new
+ */
+ field->kptr.dtor = NULL;
+ id = info->kptr.type_id;
+ kptr_btf = (struct btf *)btf;
+ goto found_dtor;
+ }
+ if (id < 0)
+ return id;
- if (!bpf_verifier_log_needed(log))
- return;
+ /* Find and stash the function pointer for the destruction function that
+ * needs to be eventually invoked from the map free path.
+ */
+ if (info->type == BPF_KPTR_REF) {
+ const struct btf_type *dtor_func;
+ const char *dtor_func_name;
+ unsigned long addr;
+ s32 dtor_btf_id;
- if (log->level == BPF_LOG_KERNEL) {
- /* btf verifier prints all types it is processing via
- * btf_verifier_log_type(..., fmt = NULL).
- * Skip those prints for in-kernel BTF verification.
+ /* This call also serves as a whitelist of allowed objects that
+ * can be used as a referenced pointer and be stored in a map at
+ * the same time.
*/
- if (!fmt)
- return;
-
- /* Skip logging when loading module BTF with mismatches permitted */
- if (env->btf->base_btf && IS_ENABLED(CONFIG_MODULE_ALLOW_BTF_MISMATCH))
- return;
- }
+ dtor_btf_id = btf_find_dtor_kfunc(kptr_btf, id);
+ if (dtor_btf_id < 0) {
+ ret = dtor_btf_id;
+ goto end_btf;
+ }
- __btf_verifier_log(log, "[%u] %s %s%s",
- env->log_type_id,
- btf_type_str(t),
- __btf_name_by_offset(btf, t->name_off),
- log_details ? " " : "");
+ dtor_func = btf_type_by_id(kptr_btf, dtor_btf_id);
+ if (!dtor_func) {
+ ret = -ENOENT;
+ goto end_btf;
+ }
- if (log_details)
- btf_type_ops(t)->log_details(env, t);
+ if (btf_is_module(kptr_btf)) {
+ mod = btf_try_get_module(kptr_btf);
+ if (!mod) {
+ ret = -ENXIO;
+ goto end_btf;
+ }
+ }
- if (fmt && *fmt) {
- __btf_verifier_log(log, " ");
- va_start(args, fmt);
- bpf_verifier_vlog(log, fmt, args);
- va_end(args);
+ /* We already verified dtor_func to be btf_type_is_func
+ * in register_btf_id_dtor_kfuncs.
+ */
+ dtor_func_name = __btf_name_by_offset(kptr_btf, dtor_func->name_off);
+ addr = kallsyms_lookup_name(dtor_func_name);
+ if (!addr) {
+ ret = -EINVAL;
+ goto end_mod;
+ }
+ field->kptr.dtor = (void *)addr;
}
- __btf_verifier_log(log, "\n");
+found_dtor:
+ field->kptr.btf_id = id;
+ field->kptr.btf = kptr_btf;
+ field->kptr.module = mod;
+ return 0;
+end_mod:
+ module_put(mod);
+end_btf:
+ btf_put(kptr_btf);
+ return ret;
}
-#define btf_verifier_log_type(env, t, ...) \
- __btf_verifier_log_type((env), (t), true, __VA_ARGS__)
-#define btf_verifier_log_basic(env, t, ...) \
- __btf_verifier_log_type((env), (t), false, __VA_ARGS__)
-
-__printf(4, 5)
-static void btf_verifier_log_member(struct btf_verifier_env *env,
- const struct btf_type *struct_type,
- const struct btf_member *member,
- const char *fmt, ...)
+static int btf_parse_graph_root(const struct btf *btf,
+ struct btf_field *field,
+ struct btf_field_info *info,
+ const char *node_type_name,
+ size_t node_type_align)
{
- struct bpf_verifier_log *log = &env->log;
- struct btf *btf = env->btf;
- va_list args;
-
- if (!bpf_verifier_log_needed(log))
- return;
-
- if (log->level == BPF_LOG_KERNEL) {
- if (!fmt)
- return;
-
- /* Skip logging when loading module BTF with mismatches permitted */
- if (env->btf->base_btf && IS_ENABLED(CONFIG_MODULE_ALLOW_BTF_MISMATCH))
- return;
- }
+ const struct btf_type *t, *n = NULL;
+ const struct btf_member *member;
+ u32 offset;
+ int i;
- /* The CHECK_META phase already did a btf dump.
- *
- * If member is logged again, it must hit an error in
- * parsing this member. It is useful to print out which
- * struct this member belongs to.
+ t = btf_type_by_id(btf, info->graph_root.value_btf_id);
+ /* We've already checked that value_btf_id is a struct type. We
+ * just need to figure out the offset of the list_node, and
+ * verify its type.
*/
- if (env->phase != CHECK_META)
- btf_verifier_log_type(env, struct_type, NULL);
-
- if (btf_type_kflag(struct_type))
- __btf_verifier_log(log,
- "\t%s type_id=%u bitfield_size=%u bits_offset=%u",
- __btf_name_by_offset(btf, member->name_off),
- member->type,
- BTF_MEMBER_BITFIELD_SIZE(member->offset),
- BTF_MEMBER_BIT_OFFSET(member->offset));
- else
- __btf_verifier_log(log, "\t%s type_id=%u bits_offset=%u",
- __btf_name_by_offset(btf, member->name_off),
- member->type, member->offset);
+ for_each_member(i, t, member) {
+ if (strcmp(info->graph_root.node_name,
+ __btf_name_by_offset(btf, member->name_off)))
+ continue;
+ /* Invalid BTF, two members with same name */
+ if (n)
+ return -EINVAL;
+ n = btf_type_by_id(btf, member->type);
+ if (!__btf_type_is_struct(n))
+ return -EINVAL;
+ if (strcmp(node_type_name, __btf_name_by_offset(btf, n->name_off)))
+ return -EINVAL;
+ offset = __btf_member_bit_offset(n, member);
+ if (offset % 8)
+ return -EINVAL;
+ offset /= 8;
+ if (offset % node_type_align)
+ return -EINVAL;
- if (fmt && *fmt) {
- __btf_verifier_log(log, " ");
- va_start(args, fmt);
- bpf_verifier_vlog(log, fmt, args);
- va_end(args);
+ field->graph_root.btf = (struct btf *)btf;
+ field->graph_root.value_btf_id = info->graph_root.value_btf_id;
+ field->graph_root.node_offset = offset;
}
-
- __btf_verifier_log(log, "\n");
+ if (!n)
+ return -ENOENT;
+ return 0;
}
-__printf(4, 5)
-static void btf_verifier_log_vsi(struct btf_verifier_env *env,
- const struct btf_type *datasec_type,
- const struct btf_var_secinfo *vsi,
- const char *fmt, ...)
+static int btf_parse_list_head(const struct btf *btf, struct btf_field *field,
+ struct btf_field_info *info)
{
- struct bpf_verifier_log *log = &env->log;
- va_list args;
-
- if (!bpf_verifier_log_needed(log))
- return;
- if (log->level == BPF_LOG_KERNEL && !fmt)
- return;
- if (env->phase != CHECK_META)
- btf_verifier_log_type(env, datasec_type, NULL);
-
- __btf_verifier_log(log, "\t type_id=%u offset=%u size=%u",
- vsi->type, vsi->offset, vsi->size);
- if (fmt && *fmt) {
- __btf_verifier_log(log, " ");
- va_start(args, fmt);
- bpf_verifier_vlog(log, fmt, args);
- va_end(args);
- }
-
- __btf_verifier_log(log, "\n");
+ return btf_parse_graph_root(btf, field, info, "bpf_list_node",
+ __alignof__(struct bpf_list_node));
}
-static void btf_verifier_log_hdr(struct btf_verifier_env *env,
- u32 btf_data_size)
+static int btf_parse_rb_root(const struct btf *btf, struct btf_field *field,
+ struct btf_field_info *info)
{
- struct bpf_verifier_log *log = &env->log;
- const struct btf *btf = env->btf;
- const struct btf_header *hdr;
-
- if (!bpf_verifier_log_needed(log))
- return;
-
- if (log->level == BPF_LOG_KERNEL)
- return;
- hdr = &btf->hdr;
- __btf_verifier_log(log, "magic: 0x%x\n", hdr->magic);
- __btf_verifier_log(log, "version: %u\n", hdr->version);
- __btf_verifier_log(log, "flags: 0x%x\n", hdr->flags);
- __btf_verifier_log(log, "hdr_len: %u\n", hdr->hdr_len);
- __btf_verifier_log(log, "type_off: %u\n", hdr->type_off);
- __btf_verifier_log(log, "type_len: %u\n", hdr->type_len);
- __btf_verifier_log(log, "str_off: %u\n", hdr->str_off);
- __btf_verifier_log(log, "str_len: %u\n", hdr->str_len);
- __btf_verifier_log(log, "btf_total_size: %u\n", btf_data_size);
+ return btf_parse_graph_root(btf, field, info, "bpf_rb_node",
+ __alignof__(struct bpf_rb_node));
}
-static int btf_add_type(struct btf_verifier_env *env, struct btf_type *t)
+static int btf_field_cmp(const void *_a, const void *_b, const void *priv)
{
- struct btf *btf = env->btf;
-
- if (btf->types_size == btf->nr_types) {
- /* Expand 'types' array */
-
- struct btf_type **new_types;
- u32 expand_by, new_size;
-
- if (btf->start_id + btf->types_size == BTF_MAX_TYPE) {
- btf_verifier_log(env, "Exceeded max num of types");
- return -E2BIG;
- }
-
- expand_by = max_t(u32, btf->types_size >> 2, 16);
- new_size = min_t(u32, BTF_MAX_TYPE,
- btf->types_size + expand_by);
-
- new_types = kvzalloc_objs(*new_types, new_size,
- GFP_KERNEL | __GFP_NOWARN);
- if (!new_types)
- return -ENOMEM;
-
- if (btf->nr_types == 0) {
- if (!btf->base_btf) {
- /* lazily init VOID type */
- new_types[0] = &btf_void;
- btf->nr_types++;
- }
- } else {
- memcpy(new_types, btf->types,
- sizeof(*btf->types) * btf->nr_types);
- }
-
- kvfree(btf->types);
- btf->types = new_types;
- btf->types_size = new_size;
- }
-
- btf->types[btf->nr_types++] = t;
+ const struct btf_field *a = (const struct btf_field *)_a;
+ const struct btf_field *b = (const struct btf_field *)_b;
+ if (a->offset < b->offset)
+ return -1;
+ else if (a->offset > b->offset)
+ return 1;
return 0;
}
-static int btf_alloc_id(struct btf *btf)
+struct btf_record *btf_parse_fields(const struct btf *btf, const struct btf_type *t,
+ u32 field_mask, u32 value_size)
{
- int id;
+ struct btf_field_info info_arr[BTF_FIELDS_MAX];
+ u32 next_off = 0, field_type_size;
+ struct btf_record *rec;
+ int ret, i, cnt;
- idr_preload(GFP_KERNEL);
- spin_lock_bh(&btf_idr_lock);
- id = idr_alloc_cyclic(&btf_idr, btf, 1, INT_MAX, GFP_ATOMIC);
- if (id > 0)
- btf->id = id;
- spin_unlock_bh(&btf_idr_lock);
- idr_preload_end();
+ ret = btf_find_field(btf, t, field_mask, info_arr, ARRAY_SIZE(info_arr));
+ if (ret < 0)
+ return ERR_PTR(ret);
+ if (!ret)
+ return NULL;
- if (WARN_ON_ONCE(!id))
- return -ENOSPC;
-
- return id > 0 ? 0 : id;
-}
-
-static void btf_free_id(struct btf *btf)
-{
- unsigned long flags;
-
- /*
- * In map-in-map, calling map_delete_elem() on outer
- * map will call bpf_map_put on the inner map.
- * It will then eventually call btf_free_id()
- * on the inner map. Some of the map_delete_elem()
- * implementation may have irq disabled, so
- * we need to use the _irqsave() version instead
- * of the _bh() version.
- */
- spin_lock_irqsave(&btf_idr_lock, flags);
- if (btf->id) {
- idr_remove(&btf_idr, btf->id);
- /*
- * Clear the id here to make this function idempotent, since it will get
- * called a couple of times for module BTFs: on module unload, and then
- * the final btf_put(). btf_alloc_id() starts IDs with 1, so we can use
- * 0 as sentinel value.
- */
- WRITE_ONCE(btf->id, 0);
- }
- spin_unlock_irqrestore(&btf_idr_lock, flags);
-}
-
-static void btf_free_kfunc_set_tab(struct btf *btf)
-{
- struct btf_kfunc_set_tab *tab = btf->kfunc_set_tab;
- int hook;
-
- if (!tab)
- return;
- for (hook = 0; hook < ARRAY_SIZE(tab->sets); hook++)
- kfree(tab->sets[hook]);
- kfree(tab);
- btf->kfunc_set_tab = NULL;
-}
-
-static void btf_free_dtor_kfunc_tab(struct btf *btf)
-{
- struct btf_id_dtor_kfunc_tab *tab = btf->dtor_kfunc_tab;
-
- if (!tab)
- return;
- kfree(tab);
- btf->dtor_kfunc_tab = NULL;
-}
-
-static void btf_struct_metas_free(struct btf_struct_metas *tab)
-{
- int i;
-
- if (!tab)
- return;
- for (i = 0; i < tab->cnt; i++)
- btf_record_free(tab->types[i].record);
- kfree(tab);
-}
-
-static void btf_free_struct_meta_tab(struct btf *btf)
-{
- struct btf_struct_metas *tab = btf->struct_meta_tab;
-
- btf_struct_metas_free(tab);
- btf->struct_meta_tab = NULL;
-}
-
-static void btf_free_struct_ops_tab(struct btf *btf)
-{
- struct btf_struct_ops_tab *tab = btf->struct_ops_tab;
- u32 i;
-
- if (!tab)
- return;
-
- for (i = 0; i < tab->cnt; i++)
- bpf_struct_ops_desc_release(&tab->ops[i]);
-
- kfree(tab);
- btf->struct_ops_tab = NULL;
-}
-
-static void btf_free(struct btf *btf)
-{
- btf_free_struct_meta_tab(btf);
- btf_free_dtor_kfunc_tab(btf);
- btf_free_kfunc_set_tab(btf);
- btf_free_struct_ops_tab(btf);
- kvfree(btf->types);
- kvfree(btf->resolved_sizes);
- kvfree(btf->resolved_ids);
- /* vmlinux does not allocate btf->data, it simply points it at
- * __start_BTF.
+ cnt = ret;
+ /* This needs to be kzalloc to zero out padding and unused fields, see
+ * comment in btf_record_equal.
*/
- if (!btf_is_vmlinux(btf))
- kvfree(btf->data);
- kvfree(btf->base_id_map);
- kfree(btf);
-}
-
-static void btf_free_rcu(struct rcu_head *rcu)
-{
- struct btf *btf = container_of(rcu, struct btf, rcu);
-
- btf_free(btf);
-}
-
-const char *btf_get_name(const struct btf *btf)
-{
- return btf->name;
-}
-
-void btf_get(struct btf *btf)
-{
- refcount_inc(&btf->refcnt);
-}
-
-void btf_put(struct btf *btf)
-{
- if (btf && refcount_dec_and_test(&btf->refcnt)) {
- btf_free_id(btf);
- call_rcu(&btf->rcu, btf_free_rcu);
- }
-}
-
-struct btf *btf_base_btf(const struct btf *btf)
-{
- return btf->base_btf;
-}
-
-const struct btf_header *btf_header(const struct btf *btf)
-{
- return &btf->hdr;
-}
-
-void btf_set_base_btf(struct btf *btf, const struct btf *base_btf)
-{
- btf->base_btf = (struct btf *)base_btf;
- btf->start_id = btf_nr_types(base_btf);
- btf->start_str_off = base_btf->hdr.str_len;
-}
-
-static int env_resolve_init(struct btf_verifier_env *env)
-{
- struct btf *btf = env->btf;
- u32 nr_types = btf->nr_types;
- u32 *resolved_sizes = NULL;
- u32 *resolved_ids = NULL;
- u8 *visit_states = NULL;
-
- resolved_sizes = kvcalloc(nr_types, sizeof(*resolved_sizes),
- GFP_KERNEL | __GFP_NOWARN);
- if (!resolved_sizes)
- goto nomem;
-
- resolved_ids = kvcalloc(nr_types, sizeof(*resolved_ids),
- GFP_KERNEL | __GFP_NOWARN);
- if (!resolved_ids)
- goto nomem;
-
- visit_states = kvcalloc(nr_types, sizeof(*visit_states),
- GFP_KERNEL | __GFP_NOWARN);
- if (!visit_states)
- goto nomem;
-
- btf->resolved_sizes = resolved_sizes;
- btf->resolved_ids = resolved_ids;
- env->visit_states = visit_states;
-
- return 0;
-
-nomem:
- kvfree(resolved_sizes);
- kvfree(resolved_ids);
- kvfree(visit_states);
- return -ENOMEM;
-}
-
-static void btf_verifier_env_free(struct btf_verifier_env *env)
-{
- kvfree(env->visit_states);
- kfree(env);
-}
-
-static bool env_type_is_resolve_sink(const struct btf_verifier_env *env,
- const struct btf_type *next_type)
-{
- switch (env->resolve_mode) {
- case RESOLVE_TBD:
- /* int, enum or void is a sink */
- return !btf_type_needs_resolve(next_type);
- case RESOLVE_PTR:
- /* int, enum, void, struct, array, func or func_proto is a sink
- * for ptr
- */
- return !btf_type_is_modifier(next_type) &&
- !btf_type_is_ptr(next_type);
- case RESOLVE_STRUCT_OR_ARRAY:
- /* int, enum, void, ptr, func or func_proto is a sink
- * for struct and array
- */
- return !btf_type_is_modifier(next_type) &&
- !btf_type_is_array(next_type) &&
- !btf_type_is_struct(next_type);
- default:
- BUG();
- }
-}
-
-static bool env_type_is_resolved(const struct btf_verifier_env *env,
- u32 type_id)
-{
- /* base BTF types should be resolved by now */
- if (type_id < env->btf->start_id)
- return true;
-
- return env->visit_states[type_id - env->btf->start_id] == RESOLVED;
-}
-
-static int env_stack_push(struct btf_verifier_env *env,
- const struct btf_type *t, u32 type_id)
-{
- const struct btf *btf = env->btf;
- struct resolve_vertex *v;
-
- if (env->top_stack == MAX_RESOLVE_DEPTH)
- return -E2BIG;
-
- if (type_id < btf->start_id
- || env->visit_states[type_id - btf->start_id] != NOT_VISITED)
- return -EEXIST;
-
- env->visit_states[type_id - btf->start_id] = VISITED;
-
- v = &env->stack[env->top_stack++];
- v->t = t;
- v->type_id = type_id;
- v->next_member = 0;
-
- if (env->resolve_mode == RESOLVE_TBD) {
- if (btf_type_is_ptr(t))
- env->resolve_mode = RESOLVE_PTR;
- else if (btf_type_is_struct(t) || btf_type_is_array(t))
- env->resolve_mode = RESOLVE_STRUCT_OR_ARRAY;
- }
-
- return 0;
-}
-
-static void env_stack_set_next_member(struct btf_verifier_env *env,
- u16 next_member)
-{
- env->stack[env->top_stack - 1].next_member = next_member;
-}
-
-static void env_stack_pop_resolved(struct btf_verifier_env *env,
- u32 resolved_type_id,
- u32 resolved_size)
-{
- u32 type_id = env->stack[--(env->top_stack)].type_id;
- struct btf *btf = env->btf;
+ rec = kzalloc_flex(*rec, fields, cnt, GFP_KERNEL_ACCOUNT | __GFP_NOWARN);
+ if (!rec)
+ return ERR_PTR(-ENOMEM);
- type_id -= btf->start_id; /* adjust to local type id */
- btf->resolved_sizes[type_id] = resolved_size;
- btf->resolved_ids[type_id] = resolved_type_id;
- env->visit_states[type_id] = RESOLVED;
-}
+ rec->spin_lock_off = -EINVAL;
+ rec->res_spin_lock_off = -EINVAL;
+ rec->timer_off = -EINVAL;
+ rec->wq_off = -EINVAL;
+ rec->refcount_off = -EINVAL;
+ rec->task_work_off = -EINVAL;
+ for (i = 0; i < cnt; i++) {
+ field_type_size = btf_field_type_size(info_arr[i].type);
+ if (info_arr[i].off + field_type_size > value_size) {
+ WARN_ONCE(1, "verifier bug off %d size %d", info_arr[i].off, value_size);
+ ret = -EFAULT;
+ goto end;
+ }
+ if (info_arr[i].off < next_off) {
+ ret = -EEXIST;
+ goto end;
+ }
+ next_off = info_arr[i].off + field_type_size;
-static const struct resolve_vertex *env_stack_peak(struct btf_verifier_env *env)
-{
- return env->top_stack ? &env->stack[env->top_stack - 1] : NULL;
-}
+ rec->field_mask |= info_arr[i].type;
+ rec->fields[i].offset = info_arr[i].off;
+ rec->fields[i].type = info_arr[i].type;
+ rec->fields[i].size = field_type_size;
-/* Resolve the size of a passed-in "type"
- *
- * type: is an array (e.g. u32 array[x][y])
- * return type: type "u32[x][y]", i.e. BTF_KIND_ARRAY,
- * *type_size: (x * y * sizeof(u32)). Hence, *type_size always
- * corresponds to the return type.
- * *elem_type: u32
- * *elem_id: id of u32
- * *total_nelems: (x * y). Hence, individual elem size is
- * (*type_size / *total_nelems)
- * *type_id: id of type if it's changed within the function, 0 if not
- *
- * type: is not an array (e.g. const struct X)
- * return type: type "struct X"
- * *type_size: sizeof(struct X)
- * *elem_type: same as return type ("struct X")
- * *elem_id: 0
- * *total_nelems: 1
- * *type_id: id of type if it's changed within the function, 0 if not
- */
-static const struct btf_type *
-__btf_resolve_size(const struct btf *btf, const struct btf_type *type,
- u32 *type_size, const struct btf_type **elem_type,
- u32 *elem_id, u32 *total_nelems, u32 *type_id)
-{
- const struct btf_type *array_type = NULL;
- const struct btf_array *array = NULL;
- u32 i, size, nelems = 1, id = 0;
-
- for (i = 0; i < MAX_RESOLVE_DEPTH; i++) {
- switch (BTF_INFO_KIND(type->info)) {
- /* type->size can be used */
- case BTF_KIND_INT:
- case BTF_KIND_STRUCT:
- case BTF_KIND_UNION:
- case BTF_KIND_ENUM:
- case BTF_KIND_FLOAT:
- case BTF_KIND_ENUM64:
- size = type->size;
- goto resolved;
-
- case BTF_KIND_PTR:
- size = sizeof(void *);
- goto resolved;
-
- /* Modifiers */
- case BTF_KIND_TYPEDEF:
- case BTF_KIND_VOLATILE:
- case BTF_KIND_CONST:
- case BTF_KIND_RESTRICT:
- case BTF_KIND_TYPE_TAG:
- id = type->type;
- type = btf_type_by_id(btf, type->type);
+ switch (info_arr[i].type) {
+ case BPF_SPIN_LOCK:
+ WARN_ON_ONCE(rec->spin_lock_off >= 0);
+ /* Cache offset for faster lookup at runtime */
+ rec->spin_lock_off = rec->fields[i].offset;
break;
-
- case BTF_KIND_ARRAY:
- if (!array_type)
- array_type = type;
- array = btf_type_array(type);
- if (nelems && array->nelems > U32_MAX / nelems)
- return ERR_PTR(-EINVAL);
- nelems *= array->nelems;
- type = btf_type_by_id(btf, array->type);
+ case BPF_RES_SPIN_LOCK:
+ WARN_ON_ONCE(rec->spin_lock_off >= 0);
+ /* Cache offset for faster lookup at runtime */
+ rec->res_spin_lock_off = rec->fields[i].offset;
+ break;
+ case BPF_TIMER:
+ WARN_ON_ONCE(rec->timer_off >= 0);
+ /* Cache offset for faster lookup at runtime */
+ rec->timer_off = rec->fields[i].offset;
+ break;
+ case BPF_WORKQUEUE:
+ WARN_ON_ONCE(rec->wq_off >= 0);
+ /* Cache offset for faster lookup at runtime */
+ rec->wq_off = rec->fields[i].offset;
+ break;
+ case BPF_TASK_WORK:
+ WARN_ON_ONCE(rec->task_work_off >= 0);
+ rec->task_work_off = rec->fields[i].offset;
+ break;
+ case BPF_REFCOUNT:
+ WARN_ON_ONCE(rec->refcount_off >= 0);
+ /* Cache offset for faster lookup at runtime */
+ rec->refcount_off = rec->fields[i].offset;
+ break;
+ case BPF_KPTR_UNREF:
+ case BPF_KPTR_REF:
+ case BPF_KPTR_PERCPU:
+ case BPF_UPTR:
+ ret = btf_parse_kptr(btf, &rec->fields[i], &info_arr[i]);
+ if (ret < 0)
+ goto end;
+ break;
+ case BPF_LIST_HEAD:
+ ret = btf_parse_list_head(btf, &rec->fields[i], &info_arr[i]);
+ if (ret < 0)
+ goto end;
+ break;
+ case BPF_RB_ROOT:
+ ret = btf_parse_rb_root(btf, &rec->fields[i], &info_arr[i]);
+ if (ret < 0)
+ goto end;
+ break;
+ case BPF_LIST_NODE:
+ case BPF_RB_NODE:
break;
-
- /* type without size */
default:
- return ERR_PTR(-EINVAL);
+ ret = -EFAULT;
+ goto end;
}
+ rec->cnt++;
}
- return ERR_PTR(-EINVAL);
-
-resolved:
- if (nelems && size > U32_MAX / nelems)
- return ERR_PTR(-EINVAL);
-
- *type_size = nelems * size;
- if (total_nelems)
- *total_nelems = nelems;
- if (elem_type)
- *elem_type = type;
- if (elem_id)
- *elem_id = array ? array->type : 0;
- if (type_id && id)
- *type_id = id;
-
- return array_type ? : type;
-}
-
-const struct btf_type *
-btf_resolve_size(const struct btf *btf, const struct btf_type *type,
- u32 *type_size)
-{
- return __btf_resolve_size(btf, type, type_size, NULL, NULL, NULL, NULL);
-}
-
-static u32 btf_resolved_type_id(const struct btf *btf, u32 type_id)
-{
- while (type_id < btf->start_id)
- btf = btf->base_btf;
-
- return btf->resolved_ids[type_id - btf->start_id];
-}
-
-/* The input param "type_id" must point to a needs_resolve type */
-static const struct btf_type *btf_type_id_resolve(const struct btf *btf,
- u32 *type_id)
-{
- *type_id = btf_resolved_type_id(btf, *type_id);
- return btf_type_by_id(btf, *type_id);
-}
-
-static u32 btf_resolved_type_size(const struct btf *btf, u32 type_id)
-{
- while (type_id < btf->start_id)
- btf = btf->base_btf;
-
- return btf->resolved_sizes[type_id - btf->start_id];
-}
-
-const struct btf_type *btf_type_id_size(const struct btf *btf,
- u32 *type_id, u32 *ret_size)
-{
- const struct btf_type *size_type;
- u32 size_type_id = *type_id;
- u32 size = 0;
-
- size_type = btf_type_by_id(btf, size_type_id);
- if (btf_type_nosize_or_null(size_type))
- return NULL;
-
- if (btf_type_has_size(size_type)) {
- size = size_type->size;
- } else if (btf_type_is_array(size_type)) {
- size = btf_resolved_type_size(btf, size_type_id);
- } else if (btf_type_is_ptr(size_type)) {
- size = sizeof(void *);
- } else {
- if (WARN_ON_ONCE(!btf_type_is_modifier(size_type) &&
- !btf_type_is_var(size_type)))
- return NULL;
-
- size_type_id = btf_resolved_type_id(btf, size_type_id);
- size_type = btf_type_by_id(btf, size_type_id);
- if (btf_type_nosize_or_null(size_type))
- return NULL;
- else if (btf_type_has_size(size_type))
- size = size_type->size;
- else if (btf_type_is_array(size_type))
- size = btf_resolved_type_size(btf, size_type_id);
- else if (btf_type_is_ptr(size_type))
- size = sizeof(void *);
- else
- return NULL;
- }
-
- *type_id = size_type_id;
- if (ret_size)
- *ret_size = size;
-
- return size_type;
-}
-
-static int btf_df_check_member(struct btf_verifier_env *env,
- const struct btf_type *struct_type,
- const struct btf_member *member,
- const struct btf_type *member_type)
-{
- btf_verifier_log_basic(env, struct_type,
- "Unsupported check_member");
- return -EINVAL;
-}
-
-static int btf_df_check_kflag_member(struct btf_verifier_env *env,
- const struct btf_type *struct_type,
- const struct btf_member *member,
- const struct btf_type *member_type)
-{
- btf_verifier_log_basic(env, struct_type,
- "Unsupported check_kflag_member");
- return -EINVAL;
-}
-
-/* Used for ptr, array struct/union and float type members.
- * int, enum and modifier types have their specific callback functions.
- */
-static int btf_generic_check_kflag_member(struct btf_verifier_env *env,
- const struct btf_type *struct_type,
- const struct btf_member *member,
- const struct btf_type *member_type)
-{
- if (BTF_MEMBER_BITFIELD_SIZE(member->offset)) {
- btf_verifier_log_member(env, struct_type, member,
- "Invalid member bitfield_size");
- return -EINVAL;
- }
-
- /* bitfield size is 0, so member->offset represents bit offset only.
- * It is safe to call non kflag check_member variants.
- */
- return btf_type_ops(member_type)->check_member(env, struct_type,
- member,
- member_type);
-}
-
-static int btf_df_resolve(struct btf_verifier_env *env,
- const struct resolve_vertex *v)
-{
- btf_verifier_log_basic(env, v->t, "Unsupported resolve");
- return -EINVAL;
-}
-
-static void btf_df_show(const struct btf *btf, const struct btf_type *t,
- u32 type_id, void *data, u8 bits_offsets,
- struct btf_show *show)
-{
- btf_show(show, "<unsupported kind:%u>", BTF_INFO_KIND(t->info));
-}
-
-static int btf_int_check_member(struct btf_verifier_env *env,
- const struct btf_type *struct_type,
- const struct btf_member *member,
- const struct btf_type *member_type)
-{
- u32 int_data = btf_type_int(member_type);
- u32 struct_bits_off = member->offset;
- u32 struct_size = struct_type->size;
- u32 nr_copy_bits;
- u32 bytes_offset;
-
- if (U32_MAX - struct_bits_off < BTF_INT_OFFSET(int_data)) {
- btf_verifier_log_member(env, struct_type, member,
- "bits_offset exceeds U32_MAX");
- return -EINVAL;
- }
-
- struct_bits_off += BTF_INT_OFFSET(int_data);
- bytes_offset = BITS_ROUNDDOWN_BYTES(struct_bits_off);
- nr_copy_bits = BTF_INT_BITS(int_data) +
- BITS_PER_BYTE_MASKED(struct_bits_off);
-
- if (nr_copy_bits > BITS_PER_U128) {
- btf_verifier_log_member(env, struct_type, member,
- "nr_copy_bits exceeds 128");
- return -EINVAL;
- }
-
- if (struct_size < bytes_offset ||
- struct_size - bytes_offset < BITS_ROUNDUP_BYTES(nr_copy_bits)) {
- btf_verifier_log_member(env, struct_type, member,
- "Member exceeds struct_size");
- return -EINVAL;
- }
-
- return 0;
-}
-
-static int btf_int_check_kflag_member(struct btf_verifier_env *env,
- const struct btf_type *struct_type,
- const struct btf_member *member,
- const struct btf_type *member_type)
-{
- u32 struct_bits_off, nr_bits, nr_int_data_bits, bytes_offset;
- u32 int_data = btf_type_int(member_type);
- u32 struct_size = struct_type->size;
- u32 nr_copy_bits;
-
- /* a regular int type is required for the kflag int member */
- if (!btf_type_int_is_regular(member_type)) {
- btf_verifier_log_member(env, struct_type, member,
- "Invalid member base type");
- return -EINVAL;
- }
-
- /* check sanity of bitfield size */
- nr_bits = BTF_MEMBER_BITFIELD_SIZE(member->offset);
- struct_bits_off = BTF_MEMBER_BIT_OFFSET(member->offset);
- nr_int_data_bits = BTF_INT_BITS(int_data);
- if (!nr_bits) {
- /* Not a bitfield member, member offset must be at byte
- * boundary.
- */
- if (BITS_PER_BYTE_MASKED(struct_bits_off)) {
- btf_verifier_log_member(env, struct_type, member,
- "Invalid member offset");
- return -EINVAL;
- }
-
- nr_bits = nr_int_data_bits;
- } else if (nr_bits > nr_int_data_bits) {
- btf_verifier_log_member(env, struct_type, member,
- "Invalid member bitfield_size");
- return -EINVAL;
- }
-
- bytes_offset = BITS_ROUNDDOWN_BYTES(struct_bits_off);
- nr_copy_bits = nr_bits + BITS_PER_BYTE_MASKED(struct_bits_off);
- if (nr_copy_bits > BITS_PER_U128) {
- btf_verifier_log_member(env, struct_type, member,
- "nr_copy_bits exceeds 128");
- return -EINVAL;
- }
-
- if (struct_size < bytes_offset ||
- struct_size - bytes_offset < BITS_ROUNDUP_BYTES(nr_copy_bits)) {
- btf_verifier_log_member(env, struct_type, member,
- "Member exceeds struct_size");
- return -EINVAL;
- }
-
- return 0;
-}
-
-static s32 btf_int_check_meta(struct btf_verifier_env *env,
- const struct btf_type *t,
- u32 meta_left)
-{
- u32 int_data, nr_bits, meta_needed = sizeof(int_data);
- u16 encoding;
-
- if (meta_left < meta_needed) {
- btf_verifier_log_basic(env, t,
- "meta_left:%u meta_needed:%u",
- meta_left, meta_needed);
- return -EINVAL;
- }
-
- if (btf_type_vlen(t)) {
- btf_verifier_log_type(env, t, "vlen != 0");
- return -EINVAL;
- }
-
- if (btf_type_kflag(t)) {
- btf_verifier_log_type(env, t, "Invalid btf_info kind_flag");
- return -EINVAL;
- }
-
- int_data = btf_type_int(t);
- if (int_data & ~BTF_INT_MASK) {
- btf_verifier_log_basic(env, t, "Invalid int_data:%x",
- int_data);
- return -EINVAL;
- }
-
- nr_bits = BTF_INT_BITS(int_data) + BTF_INT_OFFSET(int_data);
-
- if (nr_bits > BITS_PER_U128) {
- btf_verifier_log_type(env, t, "nr_bits exceeds %zu",
- BITS_PER_U128);
- return -EINVAL;
- }
-
- if (BITS_ROUNDUP_BYTES(nr_bits) > t->size) {
- btf_verifier_log_type(env, t, "nr_bits exceeds type_size");
- return -EINVAL;
- }
-
- /*
- * Only one of the encoding bits is allowed and it
- * should be sufficient for the pretty print purpose (i.e. decoding).
- * Multiple bits can be allowed later if it is found
- * to be insufficient.
- */
- encoding = BTF_INT_ENCODING(int_data);
- if (encoding &&
- encoding != BTF_INT_SIGNED &&
- encoding != BTF_INT_CHAR &&
- encoding != BTF_INT_BOOL) {
- btf_verifier_log_type(env, t, "Unsupported encoding");
- return -ENOTSUPP;
- }
-
- btf_verifier_log_type(env, t, NULL);
-
- return meta_needed;
-}
-
-static void btf_int_log(struct btf_verifier_env *env,
- const struct btf_type *t)
-{
- int int_data = btf_type_int(t);
-
- btf_verifier_log(env,
- "size=%u bits_offset=%u nr_bits=%u encoding=%s",
- t->size, BTF_INT_OFFSET(int_data),
- BTF_INT_BITS(int_data),
- btf_int_encoding_str(BTF_INT_ENCODING(int_data)));
-}
-
-static void btf_int128_print(struct btf_show *show, void *data)
-{
- /* data points to a __int128 number.
- * Suppose
- * int128_num = *(__int128 *)data;
- * The below formulas shows what upper_num and lower_num represents:
- * upper_num = int128_num >> 64;
- * lower_num = int128_num & 0xffffffffFFFFFFFFULL;
- */
- u64 upper_num, lower_num;
-
-#ifdef __BIG_ENDIAN_BITFIELD
- upper_num = *(u64 *)data;
- lower_num = *(u64 *)(data + 8);
-#else
- upper_num = *(u64 *)(data + 8);
- lower_num = *(u64 *)data;
-#endif
- if (upper_num == 0)
- btf_show_type_value(show, "0x%llx", lower_num);
- else
- btf_show_type_values(show, "0x%llx%016llx", upper_num,
- lower_num);
-}
-
-static void btf_int128_shift(u64 *print_num, u16 left_shift_bits,
- u16 right_shift_bits)
-{
- u64 upper_num, lower_num;
-
-#ifdef __BIG_ENDIAN_BITFIELD
- upper_num = print_num[0];
- lower_num = print_num[1];
-#else
- upper_num = print_num[1];
- lower_num = print_num[0];
-#endif
-
- /* shake out un-needed bits by shift/or operations */
- if (left_shift_bits >= 64) {
- upper_num = lower_num << (left_shift_bits - 64);
- lower_num = 0;
- } else {
- upper_num = (upper_num << left_shift_bits) |
- (lower_num >> (64 - left_shift_bits));
- lower_num = lower_num << left_shift_bits;
- }
-
- if (right_shift_bits >= 64) {
- lower_num = upper_num >> (right_shift_bits - 64);
- upper_num = 0;
- } else {
- lower_num = (lower_num >> right_shift_bits) |
- (upper_num << (64 - right_shift_bits));
- upper_num = upper_num >> right_shift_bits;
- }
-
-#ifdef __BIG_ENDIAN_BITFIELD
- print_num[0] = upper_num;
- print_num[1] = lower_num;
-#else
- print_num[0] = lower_num;
- print_num[1] = upper_num;
-#endif
-}
-
-static void btf_bitfield_show(void *data, u8 bits_offset,
- u8 nr_bits, struct btf_show *show)
-{
- u16 left_shift_bits, right_shift_bits;
- u8 nr_copy_bytes;
- u8 nr_copy_bits;
- u64 print_num[2] = {};
-
- nr_copy_bits = nr_bits + bits_offset;
- nr_copy_bytes = BITS_ROUNDUP_BYTES(nr_copy_bits);
-
- memcpy(print_num, data, nr_copy_bytes);
-
-#ifdef __BIG_ENDIAN_BITFIELD
- left_shift_bits = bits_offset;
-#else
- left_shift_bits = BITS_PER_U128 - nr_copy_bits;
-#endif
- right_shift_bits = BITS_PER_U128 - nr_bits;
-
- btf_int128_shift(print_num, left_shift_bits, right_shift_bits);
- btf_int128_print(show, print_num);
-}
-
-
-static void btf_int_bits_show(const struct btf *btf,
- const struct btf_type *t,
- void *data, u8 bits_offset,
- struct btf_show *show)
-{
- u32 int_data = btf_type_int(t);
- u8 nr_bits = BTF_INT_BITS(int_data);
- u8 total_bits_offset;
-
- /*
- * bits_offset is at most 7.
- * BTF_INT_OFFSET() cannot exceed 128 bits.
- */
- total_bits_offset = bits_offset + BTF_INT_OFFSET(int_data);
- data += BITS_ROUNDDOWN_BYTES(total_bits_offset);
- bits_offset = BITS_PER_BYTE_MASKED(total_bits_offset);
- btf_bitfield_show(data, bits_offset, nr_bits, show);
-}
-
-static void btf_int_show(const struct btf *btf, const struct btf_type *t,
- u32 type_id, void *data, u8 bits_offset,
- struct btf_show *show)
-{
- u32 int_data = btf_type_int(t);
- u8 encoding = BTF_INT_ENCODING(int_data);
- bool sign = encoding & BTF_INT_SIGNED;
- u8 nr_bits = BTF_INT_BITS(int_data);
- void *safe_data;
-
- safe_data = btf_show_start_type(show, t, type_id, data);
- if (!safe_data)
- return;
-
- if (bits_offset || BTF_INT_OFFSET(int_data) ||
- BITS_PER_BYTE_MASKED(nr_bits)) {
- btf_int_bits_show(btf, t, safe_data, bits_offset, show);
- goto out;
- }
-
- switch (nr_bits) {
- case 128:
- btf_int128_print(show, safe_data);
- break;
- case 64:
- if (sign)
- btf_show_type_value(show, "%lld", *(s64 *)safe_data);
- else
- btf_show_type_value(show, "%llu", *(u64 *)safe_data);
- break;
- case 32:
- if (sign)
- btf_show_type_value(show, "%d", *(s32 *)safe_data);
- else
- btf_show_type_value(show, "%u", *(u32 *)safe_data);
- break;
- case 16:
- if (sign)
- btf_show_type_value(show, "%d", *(s16 *)safe_data);
- else
- btf_show_type_value(show, "%u", *(u16 *)safe_data);
- break;
- case 8:
- if (show->state.array_encoding == BTF_INT_CHAR) {
- /* check for null terminator */
- if (show->state.array_terminated)
- break;
- if (*(char *)data == '\0') {
- show->state.array_terminated = 1;
- break;
- }
- if (isprint(*(char *)data)) {
- btf_show_type_value(show, "'%c'",
- *(char *)safe_data);
- break;
- }
- }
- if (sign)
- btf_show_type_value(show, "%d", *(s8 *)safe_data);
- else
- btf_show_type_value(show, "%u", *(u8 *)safe_data);
- break;
- default:
- btf_int_bits_show(btf, t, safe_data, bits_offset, show);
- break;
- }
-out:
- btf_show_end_type(show);
-}
-
-static const struct btf_kind_operations int_ops = {
- .check_meta = btf_int_check_meta,
- .resolve = btf_df_resolve,
- .check_member = btf_int_check_member,
- .check_kflag_member = btf_int_check_kflag_member,
- .log_details = btf_int_log,
- .show = btf_int_show,
-};
-
-static int btf_modifier_check_member(struct btf_verifier_env *env,
- const struct btf_type *struct_type,
- const struct btf_member *member,
- const struct btf_type *member_type)
-{
- const struct btf_type *resolved_type;
- u32 resolved_type_id = member->type;
- struct btf_member resolved_member;
- struct btf *btf = env->btf;
-
- resolved_type = btf_type_id_size(btf, &resolved_type_id, NULL);
- if (!resolved_type) {
- btf_verifier_log_member(env, struct_type, member,
- "Invalid member");
- return -EINVAL;
- }
-
- resolved_member = *member;
- resolved_member.type = resolved_type_id;
-
- return btf_type_ops(resolved_type)->check_member(env, struct_type,
- &resolved_member,
- resolved_type);
-}
-
-static int btf_modifier_check_kflag_member(struct btf_verifier_env *env,
- const struct btf_type *struct_type,
- const struct btf_member *member,
- const struct btf_type *member_type)
-{
- const struct btf_type *resolved_type;
- u32 resolved_type_id = member->type;
- struct btf_member resolved_member;
- struct btf *btf = env->btf;
-
- resolved_type = btf_type_id_size(btf, &resolved_type_id, NULL);
- if (!resolved_type) {
- btf_verifier_log_member(env, struct_type, member,
- "Invalid member");
- return -EINVAL;
- }
-
- resolved_member = *member;
- resolved_member.type = resolved_type_id;
-
- return btf_type_ops(resolved_type)->check_kflag_member(env, struct_type,
- &resolved_member,
- resolved_type);
-}
-
-static int btf_ptr_check_member(struct btf_verifier_env *env,
- const struct btf_type *struct_type,
- const struct btf_member *member,
- const struct btf_type *member_type)
-{
- u32 struct_size, struct_bits_off, bytes_offset;
-
- struct_size = struct_type->size;
- struct_bits_off = member->offset;
- bytes_offset = BITS_ROUNDDOWN_BYTES(struct_bits_off);
-
- if (BITS_PER_BYTE_MASKED(struct_bits_off)) {
- btf_verifier_log_member(env, struct_type, member,
- "Member is not byte aligned");
- return -EINVAL;
- }
-
- if (struct_size - bytes_offset < sizeof(void *)) {
- btf_verifier_log_member(env, struct_type, member,
- "Member exceeds struct_size");
- return -EINVAL;
- }
-
- return 0;
-}
-
-static int btf_ref_type_check_meta(struct btf_verifier_env *env,
- const struct btf_type *t,
- u32 meta_left)
-{
- const char *value;
-
- if (btf_type_vlen(t)) {
- btf_verifier_log_type(env, t, "vlen != 0");
- return -EINVAL;
- }
-
- if (btf_type_kflag(t) && !btf_type_is_type_tag(t)) {
- btf_verifier_log_type(env, t, "Invalid btf_info kind_flag");
- return -EINVAL;
- }
-
- if (!BTF_TYPE_ID_VALID(t->type)) {
- btf_verifier_log_type(env, t, "Invalid type_id");
- return -EINVAL;
- }
-
- /* typedef/type_tag type must have a valid name, and other ref types,
- * volatile, const, restrict, should have a null name.
- */
- if (BTF_INFO_KIND(t->info) == BTF_KIND_TYPEDEF) {
- if (!t->name_off ||
- !btf_name_valid_identifier(env->btf, t->name_off)) {
- btf_verifier_log_type(env, t, "Invalid name");
- return -EINVAL;
- }
- } else if (BTF_INFO_KIND(t->info) == BTF_KIND_TYPE_TAG) {
- value = btf_name_by_offset(env->btf, t->name_off);
- if (!value || !value[0]) {
- btf_verifier_log_type(env, t, "Invalid name");
- return -EINVAL;
- }
- } else {
- if (t->name_off) {
- btf_verifier_log_type(env, t, "Invalid name");
- return -EINVAL;
- }
- }
-
- btf_verifier_log_type(env, t, NULL);
-
- return 0;
-}
-
-static int btf_modifier_resolve(struct btf_verifier_env *env,
- const struct resolve_vertex *v)
-{
- const struct btf_type *t = v->t;
- const struct btf_type *next_type;
- u32 next_type_id = t->type;
- struct btf *btf = env->btf;
-
- next_type = btf_type_by_id(btf, next_type_id);
- if (!next_type || btf_type_is_resolve_source_only(next_type)) {
- btf_verifier_log_type(env, v->t, "Invalid type_id");
- return -EINVAL;
- }
-
- if (!env_type_is_resolve_sink(env, next_type) &&
- !env_type_is_resolved(env, next_type_id))
- return env_stack_push(env, next_type, next_type_id);
-
- /* Figure out the resolved next_type_id with size.
- * They will be stored in the current modifier's
- * resolved_ids and resolved_sizes such that it can
- * save us a few type-following when we use it later (e.g. in
- * pretty print).
- */
- if (!btf_type_id_size(btf, &next_type_id, NULL)) {
- if (env_type_is_resolved(env, next_type_id))
- next_type = btf_type_id_resolve(btf, &next_type_id);
-
- /* "typedef void new_void", "const void"...etc */
- if (!btf_type_is_void(next_type) &&
- !btf_type_is_fwd(next_type) &&
- !btf_type_is_func_proto(next_type)) {
- btf_verifier_log_type(env, v->t, "Invalid type_id");
- return -EINVAL;
- }
- }
-
- env_stack_pop_resolved(env, next_type_id, 0);
-
- return 0;
-}
-
-static int btf_var_resolve(struct btf_verifier_env *env,
- const struct resolve_vertex *v)
-{
- const struct btf_type *next_type;
- const struct btf_type *t = v->t;
- u32 next_type_id = t->type;
- struct btf *btf = env->btf;
-
- next_type = btf_type_by_id(btf, next_type_id);
- if (!next_type || btf_type_is_resolve_source_only(next_type)) {
- btf_verifier_log_type(env, v->t, "Invalid type_id");
- return -EINVAL;
- }
-
- if (!env_type_is_resolve_sink(env, next_type) &&
- !env_type_is_resolved(env, next_type_id))
- return env_stack_push(env, next_type, next_type_id);
-
- if (btf_type_is_modifier(next_type)) {
- const struct btf_type *resolved_type;
- u32 resolved_type_id;
-
- resolved_type_id = next_type_id;
- resolved_type = btf_type_id_resolve(btf, &resolved_type_id);
-
- if (btf_type_is_ptr(resolved_type) &&
- !env_type_is_resolve_sink(env, resolved_type) &&
- !env_type_is_resolved(env, resolved_type_id))
- return env_stack_push(env, resolved_type,
- resolved_type_id);
- }
-
- /* We must resolve to something concrete at this point, no
- * forward types or similar that would resolve to size of
- * zero is allowed.
- */
- if (!btf_type_id_size(btf, &next_type_id, NULL)) {
- btf_verifier_log_type(env, v->t, "Invalid type_id");
- return -EINVAL;
- }
-
- env_stack_pop_resolved(env, next_type_id, 0);
-
- return 0;
-}
-
-static int btf_ptr_resolve(struct btf_verifier_env *env,
- const struct resolve_vertex *v)
-{
- const struct btf_type *next_type;
- const struct btf_type *t = v->t;
- u32 next_type_id = t->type;
- struct btf *btf = env->btf;
-
- next_type = btf_type_by_id(btf, next_type_id);
- if (!next_type || btf_type_is_resolve_source_only(next_type)) {
- btf_verifier_log_type(env, v->t, "Invalid type_id");
- return -EINVAL;
- }
-
- if (!env_type_is_resolve_sink(env, next_type) &&
- !env_type_is_resolved(env, next_type_id))
- return env_stack_push(env, next_type, next_type_id);
-
- /* If the modifier was RESOLVED during RESOLVE_STRUCT_OR_ARRAY,
- * the modifier may have stopped resolving when it was resolved
- * to a ptr (last-resolved-ptr).
- *
- * We now need to continue from the last-resolved-ptr to
- * ensure the last-resolved-ptr will not referring back to
- * the current ptr (t).
- */
- if (btf_type_is_modifier(next_type)) {
- const struct btf_type *resolved_type;
- u32 resolved_type_id;
-
- resolved_type_id = next_type_id;
- resolved_type = btf_type_id_resolve(btf, &resolved_type_id);
-
- if (btf_type_is_ptr(resolved_type) &&
- !env_type_is_resolve_sink(env, resolved_type) &&
- !env_type_is_resolved(env, resolved_type_id))
- return env_stack_push(env, resolved_type,
- resolved_type_id);
- }
-
- if (!btf_type_id_size(btf, &next_type_id, NULL)) {
- if (env_type_is_resolved(env, next_type_id))
- next_type = btf_type_id_resolve(btf, &next_type_id);
-
- if (!btf_type_is_void(next_type) &&
- !btf_type_is_fwd(next_type) &&
- !btf_type_is_func_proto(next_type)) {
- btf_verifier_log_type(env, v->t, "Invalid type_id");
- return -EINVAL;
- }
- }
-
- env_stack_pop_resolved(env, next_type_id, 0);
-
- return 0;
-}
-
-static void btf_modifier_show(const struct btf *btf,
- const struct btf_type *t,
- u32 type_id, void *data,
- u8 bits_offset, struct btf_show *show)
-{
- if (btf->resolved_ids)
- t = btf_type_id_resolve(btf, &type_id);
- else
- t = btf_type_skip_modifiers(btf, type_id, NULL);
-
- btf_type_ops(t)->show(btf, t, type_id, data, bits_offset, show);
-}
-
-static void btf_var_show(const struct btf *btf, const struct btf_type *t,
- u32 type_id, void *data, u8 bits_offset,
- struct btf_show *show)
-{
- t = btf_type_id_resolve(btf, &type_id);
-
- btf_type_ops(t)->show(btf, t, type_id, data, bits_offset, show);
-}
-
-static void btf_ptr_show(const struct btf *btf, const struct btf_type *t,
- u32 type_id, void *data, u8 bits_offset,
- struct btf_show *show)
-{
- void *safe_data;
-
- safe_data = btf_show_start_type(show, t, type_id, data);
- if (!safe_data)
- return;
-
- /* It is a hashed value unless BTF_SHOW_PTR_RAW is specified */
- if (show->flags & BTF_SHOW_PTR_RAW)
- btf_show_type_value(show, "0x%px", *(void **)safe_data);
- else
- btf_show_type_value(show, "0x%p", *(void **)safe_data);
- btf_show_end_type(show);
-}
-
-static void btf_ref_type_log(struct btf_verifier_env *env,
- const struct btf_type *t)
-{
- btf_verifier_log(env, "type_id=%u", t->type);
-}
-
-static const struct btf_kind_operations modifier_ops = {
- .check_meta = btf_ref_type_check_meta,
- .resolve = btf_modifier_resolve,
- .check_member = btf_modifier_check_member,
- .check_kflag_member = btf_modifier_check_kflag_member,
- .log_details = btf_ref_type_log,
- .show = btf_modifier_show,
-};
-
-static const struct btf_kind_operations ptr_ops = {
- .check_meta = btf_ref_type_check_meta,
- .resolve = btf_ptr_resolve,
- .check_member = btf_ptr_check_member,
- .check_kflag_member = btf_generic_check_kflag_member,
- .log_details = btf_ref_type_log,
- .show = btf_ptr_show,
-};
-
-static s32 btf_fwd_check_meta(struct btf_verifier_env *env,
- const struct btf_type *t,
- u32 meta_left)
-{
- if (btf_type_vlen(t)) {
- btf_verifier_log_type(env, t, "vlen != 0");
- return -EINVAL;
- }
-
- if (t->type) {
- btf_verifier_log_type(env, t, "type != 0");
- return -EINVAL;
- }
-
- /* fwd type must have a valid name */
- if (!t->name_off ||
- !btf_name_valid_identifier(env->btf, t->name_off)) {
- btf_verifier_log_type(env, t, "Invalid name");
- return -EINVAL;
- }
-
- btf_verifier_log_type(env, t, NULL);
-
- return 0;
-}
-
-static void btf_fwd_type_log(struct btf_verifier_env *env,
- const struct btf_type *t)
-{
- btf_verifier_log(env, "%s", btf_type_kflag(t) ? "union" : "struct");
-}
-
-static const struct btf_kind_operations fwd_ops = {
- .check_meta = btf_fwd_check_meta,
- .resolve = btf_df_resolve,
- .check_member = btf_df_check_member,
- .check_kflag_member = btf_df_check_kflag_member,
- .log_details = btf_fwd_type_log,
- .show = btf_df_show,
-};
-
-static int btf_array_check_member(struct btf_verifier_env *env,
- const struct btf_type *struct_type,
- const struct btf_member *member,
- const struct btf_type *member_type)
-{
- u32 struct_bits_off = member->offset;
- u32 struct_size, bytes_offset;
- u32 array_type_id, array_size;
- struct btf *btf = env->btf;
-
- if (BITS_PER_BYTE_MASKED(struct_bits_off)) {
- btf_verifier_log_member(env, struct_type, member,
- "Member is not byte aligned");
- return -EINVAL;
- }
-
- array_type_id = member->type;
- btf_type_id_size(btf, &array_type_id, &array_size);
- struct_size = struct_type->size;
- bytes_offset = BITS_ROUNDDOWN_BYTES(struct_bits_off);
- if (struct_size - bytes_offset < array_size) {
- btf_verifier_log_member(env, struct_type, member,
- "Member exceeds struct_size");
- return -EINVAL;
- }
-
- return 0;
-}
-
-static s32 btf_array_check_meta(struct btf_verifier_env *env,
- const struct btf_type *t,
- u32 meta_left)
-{
- const struct btf_array *array = btf_type_array(t);
- u32 meta_needed = sizeof(*array);
-
- if (meta_left < meta_needed) {
- btf_verifier_log_basic(env, t,
- "meta_left:%u meta_needed:%u",
- meta_left, meta_needed);
- return -EINVAL;
- }
-
- /* array type should not have a name */
- if (t->name_off) {
- btf_verifier_log_type(env, t, "Invalid name");
- return -EINVAL;
- }
-
- if (btf_type_vlen(t)) {
- btf_verifier_log_type(env, t, "vlen != 0");
- return -EINVAL;
- }
-
- if (btf_type_kflag(t)) {
- btf_verifier_log_type(env, t, "Invalid btf_info kind_flag");
- return -EINVAL;
- }
-
- if (t->size) {
- btf_verifier_log_type(env, t, "size != 0");
- return -EINVAL;
- }
-
- /* Array elem type and index type cannot be in type void,
- * so !array->type and !array->index_type are not allowed.
- */
- if (!array->type || !BTF_TYPE_ID_VALID(array->type)) {
- btf_verifier_log_type(env, t, "Invalid elem");
- return -EINVAL;
- }
-
- if (!array->index_type || !BTF_TYPE_ID_VALID(array->index_type)) {
- btf_verifier_log_type(env, t, "Invalid index");
- return -EINVAL;
- }
-
- btf_verifier_log_type(env, t, NULL);
-
- return meta_needed;
-}
-
-static int btf_array_resolve(struct btf_verifier_env *env,
- const struct resolve_vertex *v)
-{
- const struct btf_array *array = btf_type_array(v->t);
- const struct btf_type *elem_type, *index_type;
- u32 elem_type_id, index_type_id;
- struct btf *btf = env->btf;
- u32 elem_size;
-
- /* Check array->index_type */
- index_type_id = array->index_type;
- index_type = btf_type_by_id(btf, index_type_id);
- if (btf_type_nosize_or_null(index_type) ||
- btf_type_is_resolve_source_only(index_type)) {
- btf_verifier_log_type(env, v->t, "Invalid index");
- return -EINVAL;
- }
-
- if (!env_type_is_resolve_sink(env, index_type) &&
- !env_type_is_resolved(env, index_type_id))
- return env_stack_push(env, index_type, index_type_id);
-
- index_type = btf_type_id_size(btf, &index_type_id, NULL);
- if (!index_type || !btf_type_is_int(index_type) ||
- !btf_type_int_is_regular(index_type)) {
- btf_verifier_log_type(env, v->t, "Invalid index");
- return -EINVAL;
- }
-
- /* Check array->type */
- elem_type_id = array->type;
- elem_type = btf_type_by_id(btf, elem_type_id);
- if (btf_type_nosize_or_null(elem_type) ||
- btf_type_is_resolve_source_only(elem_type)) {
- btf_verifier_log_type(env, v->t,
- "Invalid elem");
- return -EINVAL;
- }
-
- if (!env_type_is_resolve_sink(env, elem_type) &&
- !env_type_is_resolved(env, elem_type_id))
- return env_stack_push(env, elem_type, elem_type_id);
-
- elem_type = btf_type_id_size(btf, &elem_type_id, &elem_size);
- if (!elem_type) {
- btf_verifier_log_type(env, v->t, "Invalid elem");
- return -EINVAL;
- }
-
- if (btf_type_is_int(elem_type) && !btf_type_int_is_regular(elem_type)) {
- btf_verifier_log_type(env, v->t, "Invalid array of int");
- return -EINVAL;
- }
-
- if (array->nelems && elem_size > U32_MAX / array->nelems) {
- btf_verifier_log_type(env, v->t,
- "Array size overflows U32_MAX");
- return -EINVAL;
- }
-
- env_stack_pop_resolved(env, elem_type_id, elem_size * array->nelems);
-
- return 0;
-}
-
-static void btf_array_log(struct btf_verifier_env *env,
- const struct btf_type *t)
-{
- const struct btf_array *array = btf_type_array(t);
-
- btf_verifier_log(env, "type_id=%u index_type_id=%u nr_elems=%u",
- array->type, array->index_type, array->nelems);
-}
-
-static void __btf_array_show(const struct btf *btf, const struct btf_type *t,
- u32 type_id, void *data, u8 bits_offset,
- struct btf_show *show)
-{
- const struct btf_array *array = btf_type_array(t);
- const struct btf_kind_operations *elem_ops;
- const struct btf_type *elem_type;
- u32 i, elem_size = 0, elem_type_id;
- u16 encoding = 0;
-
- elem_type_id = array->type;
- elem_type = btf_type_skip_modifiers(btf, elem_type_id, NULL);
- if (elem_type && btf_type_has_size(elem_type))
- elem_size = elem_type->size;
-
- if (elem_type && btf_type_is_int(elem_type)) {
- u32 int_type = btf_type_int(elem_type);
-
- encoding = BTF_INT_ENCODING(int_type);
-
- /*
- * BTF_INT_CHAR encoding never seems to be set for
- * char arrays, so if size is 1 and element is
- * printable as a char, we'll do that.
- */
- if (elem_size == 1)
- encoding = BTF_INT_CHAR;
- }
-
- if (!btf_show_start_array_type(show, t, type_id, encoding, data))
- return;
-
- if (!elem_type)
- goto out;
- elem_ops = btf_type_ops(elem_type);
-
- for (i = 0; i < array->nelems; i++) {
-
- btf_show_start_array_member(show);
-
- elem_ops->show(btf, elem_type, elem_type_id, data,
- bits_offset, show);
- data += elem_size;
-
- btf_show_end_array_member(show);
-
- if (show->state.array_terminated)
- break;
- }
-out:
- btf_show_end_array_type(show);
-}
-
-static void btf_array_show(const struct btf *btf, const struct btf_type *t,
- u32 type_id, void *data, u8 bits_offset,
- struct btf_show *show)
-{
- const struct btf_member *m = show->state.member;
-
- /*
- * First check if any members would be shown (are non-zero).
- * See comments above "struct btf_show" definition for more
- * details on how this works at a high-level.
- */
- if (show->state.depth > 0 && !(show->flags & BTF_SHOW_ZERO)) {
- if (!show->state.depth_check) {
- show->state.depth_check = show->state.depth + 1;
- show->state.depth_to_show = 0;
- }
- __btf_array_show(btf, t, type_id, data, bits_offset, show);
- show->state.member = m;
-
- if (show->state.depth_check != show->state.depth + 1)
- return;
- show->state.depth_check = 0;
-
- if (show->state.depth_to_show <= show->state.depth)
- return;
- /*
- * Reaching here indicates we have recursed and found
- * non-zero array member(s).
- */
- }
- __btf_array_show(btf, t, type_id, data, bits_offset, show);
-}
-
-static const struct btf_kind_operations array_ops = {
- .check_meta = btf_array_check_meta,
- .resolve = btf_array_resolve,
- .check_member = btf_array_check_member,
- .check_kflag_member = btf_generic_check_kflag_member,
- .log_details = btf_array_log,
- .show = btf_array_show,
-};
-
-static int btf_struct_check_member(struct btf_verifier_env *env,
- const struct btf_type *struct_type,
- const struct btf_member *member,
- const struct btf_type *member_type)
-{
- u32 struct_bits_off = member->offset;
- u32 struct_size, bytes_offset;
-
- if (BITS_PER_BYTE_MASKED(struct_bits_off)) {
- btf_verifier_log_member(env, struct_type, member,
- "Member is not byte aligned");
- return -EINVAL;
- }
-
- struct_size = struct_type->size;
- bytes_offset = BITS_ROUNDDOWN_BYTES(struct_bits_off);
- if (struct_size - bytes_offset < member_type->size) {
- btf_verifier_log_member(env, struct_type, member,
- "Member exceeds struct_size");
- return -EINVAL;
- }
-
- return 0;
-}
-
-static s32 btf_struct_check_meta(struct btf_verifier_env *env,
- const struct btf_type *t,
- u32 meta_left)
-{
- bool is_union = BTF_INFO_KIND(t->info) == BTF_KIND_UNION;
- const struct btf_member *member;
- u32 meta_needed, last_offset;
- struct btf *btf = env->btf;
- u32 struct_size = t->size;
- u32 offset;
- u16 i;
-
- meta_needed = btf_type_vlen(t) * sizeof(*member);
- if (meta_left < meta_needed) {
- btf_verifier_log_basic(env, t,
- "meta_left:%u meta_needed:%u",
- meta_left, meta_needed);
- return -EINVAL;
- }
-
- /* struct type either no name or a valid one */
- if (t->name_off &&
- !btf_name_valid_identifier(env->btf, t->name_off)) {
- btf_verifier_log_type(env, t, "Invalid name");
- return -EINVAL;
- }
-
- btf_verifier_log_type(env, t, NULL);
-
- last_offset = 0;
- for_each_member(i, t, member) {
- if (!btf_name_offset_valid(btf, member->name_off)) {
- btf_verifier_log_member(env, t, member,
- "Invalid member name_offset:%u",
- member->name_off);
- return -EINVAL;
- }
-
- /* struct member either no name or a valid one */
- if (member->name_off &&
- !btf_name_valid_identifier(btf, member->name_off)) {
- btf_verifier_log_member(env, t, member, "Invalid name");
- return -EINVAL;
- }
- /* A member cannot be in type void */
- if (!member->type || !BTF_TYPE_ID_VALID(member->type)) {
- btf_verifier_log_member(env, t, member,
- "Invalid type_id");
- return -EINVAL;
- }
-
- offset = __btf_member_bit_offset(t, member);
- if (is_union && offset) {
- btf_verifier_log_member(env, t, member,
- "Invalid member bits_offset");
- return -EINVAL;
- }
-
- /*
- * ">" instead of ">=" because the last member could be
- * "char a[0];"
- */
- if (last_offset > offset) {
- btf_verifier_log_member(env, t, member,
- "Invalid member bits_offset");
- return -EINVAL;
- }
-
- if (BITS_ROUNDUP_BYTES(offset) > struct_size) {
- btf_verifier_log_member(env, t, member,
- "Member bits_offset exceeds its struct size");
- return -EINVAL;
- }
-
- btf_verifier_log_member(env, t, member, NULL);
- last_offset = offset;
- }
-
- return meta_needed;
-}
-
-static int btf_struct_resolve(struct btf_verifier_env *env,
- const struct resolve_vertex *v)
-{
- const struct btf_member *member;
- int err;
- u16 i;
-
- /* Before continue resolving the next_member,
- * ensure the last member is indeed resolved to a
- * type with size info.
- */
- if (v->next_member) {
- const struct btf_type *last_member_type;
- const struct btf_member *last_member;
- u32 last_member_type_id;
-
- last_member = btf_type_member(v->t) + v->next_member - 1;
- last_member_type_id = last_member->type;
- if (WARN_ON_ONCE(!env_type_is_resolved(env,
- last_member_type_id)))
- return -EINVAL;
-
- last_member_type = btf_type_by_id(env->btf,
- last_member_type_id);
- if (btf_type_kflag(v->t))
- err = btf_type_ops(last_member_type)->check_kflag_member(env, v->t,
- last_member,
- last_member_type);
- else
- err = btf_type_ops(last_member_type)->check_member(env, v->t,
- last_member,
- last_member_type);
- if (err)
- return err;
- }
-
- for_each_member_from(i, v->next_member, v->t, member) {
- u32 member_type_id = member->type;
- const struct btf_type *member_type = btf_type_by_id(env->btf,
- member_type_id);
-
- if (btf_type_nosize_or_null(member_type) ||
- btf_type_is_resolve_source_only(member_type)) {
- btf_verifier_log_member(env, v->t, member,
- "Invalid member");
- return -EINVAL;
- }
-
- if (!env_type_is_resolve_sink(env, member_type) &&
- !env_type_is_resolved(env, member_type_id)) {
- env_stack_set_next_member(env, i + 1);
- return env_stack_push(env, member_type, member_type_id);
- }
-
- if (btf_type_kflag(v->t))
- err = btf_type_ops(member_type)->check_kflag_member(env, v->t,
- member,
- member_type);
- else
- err = btf_type_ops(member_type)->check_member(env, v->t,
- member,
- member_type);
- if (err)
- return err;
- }
-
- env_stack_pop_resolved(env, 0, 0);
-
- return 0;
-}
-
-static void btf_struct_log(struct btf_verifier_env *env,
- const struct btf_type *t)
-{
- btf_verifier_log(env, "size=%u vlen=%u", t->size, btf_type_vlen(t));
-}
-
-enum {
- BTF_FIELD_IGNORE = 0,
- BTF_FIELD_FOUND = 1,
-};
-
-struct btf_field_info {
- enum btf_field_type type;
- u32 off;
- union {
- struct {
- u32 type_id;
- } kptr;
- struct {
- const char *node_name;
- u32 value_btf_id;
- } graph_root;
- };
-};
-
-static int btf_find_struct(const struct btf *btf, const struct btf_type *t,
- u32 off, int sz, enum btf_field_type field_type,
- struct btf_field_info *info)
-{
- if (!__btf_type_is_struct(t))
- return BTF_FIELD_IGNORE;
- if (t->size != sz)
- return BTF_FIELD_IGNORE;
- info->type = field_type;
- info->off = off;
- return BTF_FIELD_FOUND;
-}
-
-static int btf_find_kptr(const struct btf *btf, const struct btf_type *t,
- u32 off, int sz, struct btf_field_info *info, u32 field_mask)
-{
- enum btf_field_type type;
- const char *tag_value;
- bool is_type_tag;
- u32 res_id;
-
- /* Permit modifiers on the pointer itself */
- if (btf_type_is_volatile(t))
- t = btf_type_by_id(btf, t->type);
- /* For PTR, sz is always == 8 */
- if (!btf_type_is_ptr(t))
- return BTF_FIELD_IGNORE;
- t = btf_type_by_id(btf, t->type);
- is_type_tag = btf_type_is_type_tag(t) && !btf_type_kflag(t);
- if (!is_type_tag)
- return BTF_FIELD_IGNORE;
- /* Reject extra tags */
- if (btf_type_is_type_tag(btf_type_by_id(btf, t->type)))
- return -EINVAL;
- tag_value = __btf_name_by_offset(btf, t->name_off);
- if (!strcmp("kptr_untrusted", tag_value))
- type = BPF_KPTR_UNREF;
- else if (!strcmp("kptr", tag_value))
- type = BPF_KPTR_REF;
- else if (!strcmp("percpu_kptr", tag_value))
- type = BPF_KPTR_PERCPU;
- else if (!strcmp("uptr", tag_value))
- type = BPF_UPTR;
- else
- return -EINVAL;
-
- if (!(type & field_mask))
- return BTF_FIELD_IGNORE;
-
- /* Get the base type */
- t = btf_type_skip_modifiers(btf, t->type, &res_id);
- /* Only pointer to struct is allowed */
- if (!__btf_type_is_struct(t))
- return -EINVAL;
-
- info->type = type;
- info->off = off;
- info->kptr.type_id = res_id;
- return BTF_FIELD_FOUND;
-}
-
-int btf_find_next_decl_tag(const struct btf *btf, const struct btf_type *pt,
- int comp_idx, const char *tag_key, int last_id)
-{
- int len = strlen(tag_key);
- int i, n;
-
- for (i = last_id + 1, n = btf_nr_types(btf); i < n; i++) {
- const struct btf_type *t = btf_type_by_id(btf, i);
-
- if (!btf_type_is_decl_tag(t))
- continue;
- if (pt != btf_type_by_id(btf, t->type))
- continue;
- if (btf_type_decl_tag(t)->component_idx != comp_idx)
- continue;
- if (strncmp(__btf_name_by_offset(btf, t->name_off), tag_key, len))
- continue;
- return i;
- }
- return -ENOENT;
-}
-
-const char *btf_find_decl_tag_value(const struct btf *btf, const struct btf_type *pt,
- int comp_idx, const char *tag_key)
-{
- const char *value = NULL;
- const struct btf_type *t;
- int len, id;
-
- id = btf_find_next_decl_tag(btf, pt, comp_idx, tag_key,
- btf_named_start_id(btf, false) - 1);
- if (id < 0)
- return ERR_PTR(id);
-
- t = btf_type_by_id(btf, id);
- len = strlen(tag_key);
- value = __btf_name_by_offset(btf, t->name_off) + len;
-
- /* Prevent duplicate entries for same type */
- id = btf_find_next_decl_tag(btf, pt, comp_idx, tag_key, id);
- if (id >= 0)
- return ERR_PTR(-EEXIST);
-
- return value;
-}
-
-static int
-btf_find_graph_root(const struct btf *btf, const struct btf_type *pt,
- const struct btf_type *t, int comp_idx, u32 off,
- int sz, struct btf_field_info *info,
- enum btf_field_type head_type)
-{
- const char *node_field_name;
- const char *value_type;
- s32 id;
-
- if (!__btf_type_is_struct(t))
- return BTF_FIELD_IGNORE;
- if (t->size != sz)
- return BTF_FIELD_IGNORE;
- value_type = btf_find_decl_tag_value(btf, pt, comp_idx, "contains:");
- if (IS_ERR(value_type))
- return -EINVAL;
- node_field_name = strstr(value_type, ":");
- if (!node_field_name)
- return -EINVAL;
- value_type = kstrndup(value_type, node_field_name - value_type,
- GFP_KERNEL_ACCOUNT | __GFP_NOWARN);
- if (!value_type)
- return -ENOMEM;
- id = btf_find_by_name_kind(btf, value_type, BTF_KIND_STRUCT);
- kfree(value_type);
- if (id < 0)
- return id;
- node_field_name++;
- if (str_is_empty(node_field_name))
- return -EINVAL;
- info->type = head_type;
- info->off = off;
- info->graph_root.value_btf_id = id;
- info->graph_root.node_name = node_field_name;
- return BTF_FIELD_FOUND;
-}
-
-static int btf_get_field_type(const struct btf *btf, const struct btf_type *var_type,
- u32 field_mask, u32 *seen_mask, int *align, int *sz)
-{
- const struct {
- enum btf_field_type type;
- const char *const name;
- const bool is_unique;
- } field_types[] = {
- { BPF_SPIN_LOCK, "bpf_spin_lock", true },
- { BPF_RES_SPIN_LOCK, "bpf_res_spin_lock", true },
- { BPF_TIMER, "bpf_timer", true },
- { BPF_WORKQUEUE, "bpf_wq", true },
- { BPF_TASK_WORK, "bpf_task_work", true },
- { BPF_LIST_HEAD, "bpf_list_head", false },
- { BPF_LIST_NODE, "bpf_list_node", false },
- { BPF_RB_ROOT, "bpf_rb_root", false },
- { BPF_RB_NODE, "bpf_rb_node", false },
- { BPF_REFCOUNT, "bpf_refcount", false },
- };
- int type = 0, i;
- const char *name = __btf_name_by_offset(btf, var_type->name_off);
- const char *field_type_name;
- enum btf_field_type field_type;
- bool is_unique;
-
- for (i = 0; i < ARRAY_SIZE(field_types); ++i) {
- field_type = field_types[i].type;
- field_type_name = field_types[i].name;
- is_unique = field_types[i].is_unique;
- if (!(field_mask & field_type) || strcmp(name, field_type_name))
- continue;
- if (is_unique) {
- if (*seen_mask & field_type)
- return -E2BIG;
- *seen_mask |= field_type;
- }
- type = field_type;
- goto end;
- }
-
- /* Only return BPF_KPTR when all other types with matchable names fail */
- if (field_mask & (BPF_KPTR | BPF_UPTR) && !__btf_type_is_struct(var_type)) {
- type = BPF_KPTR_REF;
- goto end;
- }
- return 0;
-end:
- *sz = btf_field_type_size(type);
- *align = btf_field_type_align(type);
- return type;
-}
-
-/* Repeat a number of fields for a specified number of times.
- *
- * Copy the fields starting from the first field and repeat them for
- * repeat_cnt times. The fields are repeated by adding the offset of each
- * field with
- * (i + 1) * elem_size
- * where i is the repeat index and elem_size is the size of an element.
- */
-static int btf_repeat_fields(struct btf_field_info *info, int info_cnt,
- u32 field_cnt, u32 repeat_cnt, u32 elem_size)
-{
- u32 i, j;
- u32 cur;
-
- /* Ensure not repeating fields that should not be repeated. */
- for (i = 0; i < field_cnt; i++) {
- switch (info[i].type) {
- case BPF_KPTR_UNREF:
- case BPF_KPTR_REF:
- case BPF_KPTR_PERCPU:
- case BPF_UPTR:
- case BPF_LIST_HEAD:
- case BPF_RB_ROOT:
- break;
- default:
- return -EINVAL;
- }
- }
-
- /* The type of struct size or variable size is u32,
- * so the multiplication will not overflow.
- */
- if (field_cnt * (repeat_cnt + 1) > info_cnt)
- return -E2BIG;
-
- cur = field_cnt;
- for (i = 0; i < repeat_cnt; i++) {
- memcpy(&info[cur], &info[0], field_cnt * sizeof(info[0]));
- for (j = 0; j < field_cnt; j++)
- info[cur++].off += (i + 1) * elem_size;
- }
-
- return 0;
-}
-
-static int btf_find_struct_field(const struct btf *btf,
- const struct btf_type *t, u32 field_mask,
- struct btf_field_info *info, int info_cnt,
- u32 level);
-
-/* Find special fields in the struct type of a field.
- *
- * This function is used to find fields of special types that is not a
- * global variable or a direct field of a struct type. It also handles the
- * repetition if it is the element type of an array.
- */
-static int btf_find_nested_struct(const struct btf *btf, const struct btf_type *t,
- u32 off, u32 nelems,
- u32 field_mask, struct btf_field_info *info,
- int info_cnt, u32 level)
-{
- int ret, err, i;
-
- level++;
- if (level >= MAX_RESOLVE_DEPTH)
- return -E2BIG;
-
- ret = btf_find_struct_field(btf, t, field_mask, info, info_cnt, level);
-
- if (ret <= 0)
- return ret;
-
- /* Shift the offsets of the nested struct fields to the offsets
- * related to the container.
- */
- for (i = 0; i < ret; i++)
- info[i].off += off;
-
- if (nelems > 1) {
- err = btf_repeat_fields(info, info_cnt, ret, nelems - 1, t->size);
- if (err == 0)
- ret *= nelems;
- else
- ret = err;
- }
-
- return ret;
-}
-
-static int btf_find_field_one(const struct btf *btf,
- const struct btf_type *var,
- const struct btf_type *var_type,
- int var_idx,
- u32 off, u32 expected_size,
- u32 field_mask, u32 *seen_mask,
- struct btf_field_info *info, int info_cnt,
- u32 level)
-{
- int ret, align, sz, field_type;
- struct btf_field_info tmp;
- const struct btf_array *array;
- u32 i, nelems = 1;
-
- /* Walk into array types to find the element type and the number of
- * elements in the (flattened) array.
- */
- for (i = 0; i < MAX_RESOLVE_DEPTH && btf_type_is_array(var_type); i++) {
- array = btf_array(var_type);
- nelems *= array->nelems;
- var_type = btf_type_by_id(btf, array->type);
- }
- if (i == MAX_RESOLVE_DEPTH)
- return -E2BIG;
- if (nelems == 0)
- return 0;
-
- field_type = btf_get_field_type(btf, var_type,
- field_mask, seen_mask, &align, &sz);
- /* Look into variables of struct types */
- if (!field_type && __btf_type_is_struct(var_type)) {
- sz = var_type->size;
- if (expected_size && expected_size != sz * nelems)
- return 0;
- ret = btf_find_nested_struct(btf, var_type, off, nelems, field_mask,
- &info[0], info_cnt, level);
- return ret;
- }
-
- if (field_type == 0)
- return 0;
- if (field_type < 0)
- return field_type;
-
- if (expected_size && expected_size != sz * nelems)
- return 0;
- if (off % align)
- return 0;
-
- switch (field_type) {
- case BPF_SPIN_LOCK:
- case BPF_RES_SPIN_LOCK:
- case BPF_TIMER:
- case BPF_WORKQUEUE:
- case BPF_LIST_NODE:
- case BPF_RB_NODE:
- case BPF_REFCOUNT:
- case BPF_TASK_WORK:
- ret = btf_find_struct(btf, var_type, off, sz, field_type,
- info_cnt ? &info[0] : &tmp);
- if (ret < 0)
- return ret;
- break;
- case BPF_KPTR_UNREF:
- case BPF_KPTR_REF:
- case BPF_KPTR_PERCPU:
- case BPF_UPTR:
- ret = btf_find_kptr(btf, var_type, off, sz,
- info_cnt ? &info[0] : &tmp, field_mask);
- if (ret < 0)
- return ret;
- break;
- case BPF_LIST_HEAD:
- case BPF_RB_ROOT:
- ret = btf_find_graph_root(btf, var, var_type,
- var_idx, off, sz,
- info_cnt ? &info[0] : &tmp,
- field_type);
- if (ret < 0)
- return ret;
- break;
- default:
- return -EFAULT;
- }
-
- if (ret == BTF_FIELD_IGNORE)
- return 0;
- if (!info_cnt)
- return -E2BIG;
- if (nelems > 1) {
- ret = btf_repeat_fields(info, info_cnt, 1, nelems - 1, sz);
- if (ret < 0)
- return ret;
- }
- return nelems;
-}
-
-static int btf_find_struct_field(const struct btf *btf,
- const struct btf_type *t, u32 field_mask,
- struct btf_field_info *info, int info_cnt,
- u32 level)
-{
- int ret, idx = 0;
- const struct btf_member *member;
- u32 i, off, seen_mask = 0;
-
- for_each_member(i, t, member) {
- const struct btf_type *member_type = btf_type_by_id(btf,
- member->type);
-
- off = __btf_member_bit_offset(t, member);
- if (off % 8)
- /* valid C code cannot generate such BTF */
- return -EINVAL;
- off /= 8;
-
- ret = btf_find_field_one(btf, t, member_type, i,
- off, 0,
- field_mask, &seen_mask,
- &info[idx], info_cnt - idx, level);
- if (ret < 0)
- return ret;
- idx += ret;
- }
- return idx;
-}
-
-static int btf_find_datasec_var(const struct btf *btf, const struct btf_type *t,
- u32 field_mask, struct btf_field_info *info,
- int info_cnt, u32 level)
-{
- int ret, idx = 0;
- const struct btf_var_secinfo *vsi;
- u32 i, off, seen_mask = 0;
-
- for_each_vsi(i, t, vsi) {
- const struct btf_type *var = btf_type_by_id(btf, vsi->type);
- const struct btf_type *var_type = btf_type_by_id(btf, var->type);
-
- off = vsi->offset;
- ret = btf_find_field_one(btf, var, var_type, -1, off, vsi->size,
- field_mask, &seen_mask,
- &info[idx], info_cnt - idx,
- level);
- if (ret < 0)
- return ret;
- idx += ret;
- }
- return idx;
-}
-
-static int btf_find_field(const struct btf *btf, const struct btf_type *t,
- u32 field_mask, struct btf_field_info *info,
- int info_cnt)
-{
- if (__btf_type_is_struct(t))
- return btf_find_struct_field(btf, t, field_mask, info, info_cnt, 0);
- else if (btf_type_is_datasec(t))
- return btf_find_datasec_var(btf, t, field_mask, info, info_cnt, 0);
- return -EINVAL;
-}
-
-/* Callers have to ensure the life cycle of btf if it is program BTF */
-static int btf_parse_kptr(const struct btf *btf, struct btf_field *field,
- struct btf_field_info *info)
-{
- struct module *mod = NULL;
- const struct btf_type *t;
- /* If a matching btf type is found in kernel or module BTFs, kptr_ref
- * is that BTF, otherwise it's program BTF
- */
- struct btf *kptr_btf;
- int ret;
- s32 id;
-
- /* Find type in map BTF, and use it to look up the matching type
- * in vmlinux or module BTFs, by name and kind.
- */
- t = btf_type_by_id(btf, info->kptr.type_id);
- id = bpf_find_btf_id(__btf_name_by_offset(btf, t->name_off), BTF_INFO_KIND(t->info),
- &kptr_btf);
- if (id == -ENOENT) {
- /* btf_parse_kptr should only be called w/ btf = program BTF */
- WARN_ON_ONCE(btf_is_kernel(btf));
-
- /* Type exists only in program BTF. Assume that it's a MEM_ALLOC
- * kptr allocated via bpf_obj_new
- */
- field->kptr.dtor = NULL;
- id = info->kptr.type_id;
- kptr_btf = (struct btf *)btf;
- goto found_dtor;
- }
- if (id < 0)
- return id;
-
- /* Find and stash the function pointer for the destruction function that
- * needs to be eventually invoked from the map free path.
- */
- if (info->type == BPF_KPTR_REF) {
- const struct btf_type *dtor_func;
- const char *dtor_func_name;
- unsigned long addr;
- s32 dtor_btf_id;
-
- /* This call also serves as a whitelist of allowed objects that
- * can be used as a referenced pointer and be stored in a map at
- * the same time.
- */
- dtor_btf_id = btf_find_dtor_kfunc(kptr_btf, id);
- if (dtor_btf_id < 0) {
- ret = dtor_btf_id;
- goto end_btf;
- }
-
- dtor_func = btf_type_by_id(kptr_btf, dtor_btf_id);
- if (!dtor_func) {
- ret = -ENOENT;
- goto end_btf;
- }
-
- if (btf_is_module(kptr_btf)) {
- mod = btf_try_get_module(kptr_btf);
- if (!mod) {
- ret = -ENXIO;
- goto end_btf;
- }
- }
-
- /* We already verified dtor_func to be btf_type_is_func
- * in register_btf_id_dtor_kfuncs.
- */
- dtor_func_name = __btf_name_by_offset(kptr_btf, dtor_func->name_off);
- addr = kallsyms_lookup_name(dtor_func_name);
- if (!addr) {
- ret = -EINVAL;
- goto end_mod;
- }
- field->kptr.dtor = (void *)addr;
- }
-
-found_dtor:
- field->kptr.btf_id = id;
- field->kptr.btf = kptr_btf;
- field->kptr.module = mod;
- return 0;
-end_mod:
- module_put(mod);
-end_btf:
- btf_put(kptr_btf);
- return ret;
-}
-
-static int btf_parse_graph_root(const struct btf *btf,
- struct btf_field *field,
- struct btf_field_info *info,
- const char *node_type_name,
- size_t node_type_align)
-{
- const struct btf_type *t, *n = NULL;
- const struct btf_member *member;
- u32 offset;
- int i;
-
- t = btf_type_by_id(btf, info->graph_root.value_btf_id);
- /* We've already checked that value_btf_id is a struct type. We
- * just need to figure out the offset of the list_node, and
- * verify its type.
- */
- for_each_member(i, t, member) {
- if (strcmp(info->graph_root.node_name,
- __btf_name_by_offset(btf, member->name_off)))
- continue;
- /* Invalid BTF, two members with same name */
- if (n)
- return -EINVAL;
- n = btf_type_by_id(btf, member->type);
- if (!__btf_type_is_struct(n))
- return -EINVAL;
- if (strcmp(node_type_name, __btf_name_by_offset(btf, n->name_off)))
- return -EINVAL;
- offset = __btf_member_bit_offset(n, member);
- if (offset % 8)
- return -EINVAL;
- offset /= 8;
- if (offset % node_type_align)
- return -EINVAL;
-
- field->graph_root.btf = (struct btf *)btf;
- field->graph_root.value_btf_id = info->graph_root.value_btf_id;
- field->graph_root.node_offset = offset;
- }
- if (!n)
- return -ENOENT;
- return 0;
-}
-
-static int btf_parse_list_head(const struct btf *btf, struct btf_field *field,
- struct btf_field_info *info)
-{
- return btf_parse_graph_root(btf, field, info, "bpf_list_node",
- __alignof__(struct bpf_list_node));
-}
-
-static int btf_parse_rb_root(const struct btf *btf, struct btf_field *field,
- struct btf_field_info *info)
-{
- return btf_parse_graph_root(btf, field, info, "bpf_rb_node",
- __alignof__(struct bpf_rb_node));
-}
-
-static int btf_field_cmp(const void *_a, const void *_b, const void *priv)
-{
- const struct btf_field *a = (const struct btf_field *)_a;
- const struct btf_field *b = (const struct btf_field *)_b;
-
- if (a->offset < b->offset)
- return -1;
- else if (a->offset > b->offset)
- return 1;
- return 0;
-}
-
-struct btf_record *btf_parse_fields(const struct btf *btf, const struct btf_type *t,
- u32 field_mask, u32 value_size)
-{
- struct btf_field_info info_arr[BTF_FIELDS_MAX];
- u32 next_off = 0, field_type_size;
- struct btf_record *rec;
- int ret, i, cnt;
-
- ret = btf_find_field(btf, t, field_mask, info_arr, ARRAY_SIZE(info_arr));
- if (ret < 0)
- return ERR_PTR(ret);
- if (!ret)
- return NULL;
-
- cnt = ret;
- /* This needs to be kzalloc to zero out padding and unused fields, see
- * comment in btf_record_equal.
- */
- rec = kzalloc_flex(*rec, fields, cnt, GFP_KERNEL_ACCOUNT | __GFP_NOWARN);
- if (!rec)
- return ERR_PTR(-ENOMEM);
-
- rec->spin_lock_off = -EINVAL;
- rec->res_spin_lock_off = -EINVAL;
- rec->timer_off = -EINVAL;
- rec->wq_off = -EINVAL;
- rec->refcount_off = -EINVAL;
- rec->task_work_off = -EINVAL;
- for (i = 0; i < cnt; i++) {
- field_type_size = btf_field_type_size(info_arr[i].type);
- if (info_arr[i].off + field_type_size > value_size) {
- WARN_ONCE(1, "verifier bug off %d size %d", info_arr[i].off, value_size);
- ret = -EFAULT;
- goto end;
- }
- if (info_arr[i].off < next_off) {
- ret = -EEXIST;
- goto end;
- }
- next_off = info_arr[i].off + field_type_size;
-
- rec->field_mask |= info_arr[i].type;
- rec->fields[i].offset = info_arr[i].off;
- rec->fields[i].type = info_arr[i].type;
- rec->fields[i].size = field_type_size;
-
- switch (info_arr[i].type) {
- case BPF_SPIN_LOCK:
- WARN_ON_ONCE(rec->spin_lock_off >= 0);
- /* Cache offset for faster lookup at runtime */
- rec->spin_lock_off = rec->fields[i].offset;
- break;
- case BPF_RES_SPIN_LOCK:
- WARN_ON_ONCE(rec->spin_lock_off >= 0);
- /* Cache offset for faster lookup at runtime */
- rec->res_spin_lock_off = rec->fields[i].offset;
- break;
- case BPF_TIMER:
- WARN_ON_ONCE(rec->timer_off >= 0);
- /* Cache offset for faster lookup at runtime */
- rec->timer_off = rec->fields[i].offset;
- break;
- case BPF_WORKQUEUE:
- WARN_ON_ONCE(rec->wq_off >= 0);
- /* Cache offset for faster lookup at runtime */
- rec->wq_off = rec->fields[i].offset;
- break;
- case BPF_TASK_WORK:
- WARN_ON_ONCE(rec->task_work_off >= 0);
- rec->task_work_off = rec->fields[i].offset;
- break;
- case BPF_REFCOUNT:
- WARN_ON_ONCE(rec->refcount_off >= 0);
- /* Cache offset for faster lookup at runtime */
- rec->refcount_off = rec->fields[i].offset;
- break;
- case BPF_KPTR_UNREF:
- case BPF_KPTR_REF:
- case BPF_KPTR_PERCPU:
- case BPF_UPTR:
- ret = btf_parse_kptr(btf, &rec->fields[i], &info_arr[i]);
- if (ret < 0)
- goto end;
- break;
- case BPF_LIST_HEAD:
- ret = btf_parse_list_head(btf, &rec->fields[i], &info_arr[i]);
- if (ret < 0)
- goto end;
- break;
- case BPF_RB_ROOT:
- ret = btf_parse_rb_root(btf, &rec->fields[i], &info_arr[i]);
- if (ret < 0)
- goto end;
- break;
- case BPF_LIST_NODE:
- case BPF_RB_NODE:
- break;
- default:
- ret = -EFAULT;
- goto end;
- }
- rec->cnt++;
- }
-
- if (rec->spin_lock_off >= 0 && rec->res_spin_lock_off >= 0) {
- ret = -EINVAL;
- goto end;
- }
-
- /* bpf_{list_head, rb_node} require bpf_spin_lock */
- if ((btf_record_has_field(rec, BPF_LIST_HEAD) ||
- btf_record_has_field(rec, BPF_RB_ROOT)) &&
- (rec->spin_lock_off < 0 && rec->res_spin_lock_off < 0)) {
- ret = -EINVAL;
- goto end;
- }
-
- if (rec->refcount_off < 0 &&
- btf_record_has_field(rec, BPF_LIST_NODE) &&
- btf_record_has_field(rec, BPF_RB_NODE)) {
- ret = -EINVAL;
- goto end;
- }
-
- sort_r(rec->fields, rec->cnt, sizeof(struct btf_field), btf_field_cmp,
- NULL, rec);
-
- return rec;
-end:
- btf_record_free(rec);
- return ERR_PTR(ret);
-}
-
-int btf_check_and_fixup_fields(const struct btf *btf, struct btf_record *rec)
-{
- int i;
-
- /* There are three types that signify ownership of some other type:
- * kptr_ref, bpf_list_head, bpf_rb_root.
- * kptr_ref only supports storing kernel types, which can't store
- * references to program allocated local types.
- *
- * Hence we only need to ensure that bpf_{list_head,rb_root} ownership
- * does not form cycles.
- */
- if (IS_ERR_OR_NULL(rec) || !(rec->field_mask & (BPF_GRAPH_ROOT | BPF_UPTR)))
- return 0;
- for (i = 0; i < rec->cnt; i++) {
- struct btf_struct_meta *meta;
- const struct btf_type *t;
- u32 btf_id;
-
- if (rec->fields[i].type == BPF_UPTR) {
- /* The uptr only supports pinning one page and cannot
- * point to a kernel struct
- */
- if (btf_is_kernel(rec->fields[i].kptr.btf))
- return -EINVAL;
- t = btf_type_by_id(rec->fields[i].kptr.btf,
- rec->fields[i].kptr.btf_id);
- if (!t->size)
- return -EINVAL;
- if (t->size > PAGE_SIZE)
- return -E2BIG;
- continue;
- }
-
- if (!(rec->fields[i].type & BPF_GRAPH_ROOT))
- continue;
- btf_id = rec->fields[i].graph_root.value_btf_id;
- meta = btf_find_struct_meta(btf, btf_id);
- if (!meta)
- return -EFAULT;
- rec->fields[i].graph_root.value_rec = meta->record;
-
- /* We need to set value_rec for all root types, but no need
- * to check ownership cycle for a type unless it's also a
- * node type.
- */
- if (!(rec->field_mask & BPF_GRAPH_NODE))
- continue;
-
- /* We need to ensure ownership acyclicity among all types. The
- * proper way to do it would be to topologically sort all BTF
- * IDs based on the ownership edges, since there can be multiple
- * bpf_{list_head,rb_node} in a type. Instead, we use the
- * following resaoning:
- *
- * - A type can only be owned by another type in user BTF if it
- * has a bpf_{list,rb}_node. Let's call these node types.
- * - A type can only _own_ another type in user BTF if it has a
- * bpf_{list_head,rb_root}. Let's call these root types.
- *
- * We ensure that if a type is both a root and node, its
- * element types cannot be root types.
- *
- * To ensure acyclicity:
- *
- * When A is an root type but not a node, its ownership
- * chain can be:
- * A -> B -> C
- * Where:
- * - A is an root, e.g. has bpf_rb_root.
- * - B is both a root and node, e.g. has bpf_rb_node and
- * bpf_list_head.
- * - C is only an root, e.g. has bpf_list_node
- *
- * When A is both a root and node, some other type already
- * owns it in the BTF domain, hence it can not own
- * another root type through any of the ownership edges.
- * A -> B
- * Where:
- * - A is both an root and node.
- * - B is only an node.
- */
- if (meta->record->field_mask & BPF_GRAPH_ROOT)
- return -ELOOP;
- }
- return 0;
-}
-
-static void __btf_struct_show(const struct btf *btf, const struct btf_type *t,
- u32 type_id, void *data, u8 bits_offset,
- struct btf_show *show)
-{
- const struct btf_member *member;
- void *safe_data;
- u32 i;
-
- safe_data = btf_show_start_struct_type(show, t, type_id, data);
- if (!safe_data)
- return;
-
- for_each_member(i, t, member) {
- const struct btf_type *member_type = btf_type_by_id(btf,
- member->type);
- const struct btf_kind_operations *ops;
- u32 member_offset, bitfield_size;
- u32 bytes_offset;
- u8 bits8_offset;
-
- btf_show_start_member(show, member);
-
- member_offset = __btf_member_bit_offset(t, member);
- bitfield_size = __btf_member_bitfield_size(t, member);
- bytes_offset = BITS_ROUNDDOWN_BYTES(member_offset);
- bits8_offset = BITS_PER_BYTE_MASKED(member_offset);
- if (bitfield_size) {
- safe_data = btf_show_start_type(show, member_type,
- member->type,
- data + bytes_offset);
- if (safe_data)
- btf_bitfield_show(safe_data,
- bits8_offset,
- bitfield_size, show);
- btf_show_end_type(show);
- } else {
- ops = btf_type_ops(member_type);
- ops->show(btf, member_type, member->type,
- data + bytes_offset, bits8_offset, show);
- }
-
- btf_show_end_member(show);
- }
-
- btf_show_end_struct_type(show);
-}
-
-static void btf_struct_show(const struct btf *btf, const struct btf_type *t,
- u32 type_id, void *data, u8 bits_offset,
- struct btf_show *show)
-{
- const struct btf_member *m = show->state.member;
-
- /*
- * First check if any members would be shown (are non-zero).
- * See comments above "struct btf_show" definition for more
- * details on how this works at a high-level.
- */
- if (show->state.depth > 0 && !(show->flags & BTF_SHOW_ZERO)) {
- if (!show->state.depth_check) {
- show->state.depth_check = show->state.depth + 1;
- show->state.depth_to_show = 0;
- }
- __btf_struct_show(btf, t, type_id, data, bits_offset, show);
- /* Restore saved member data here */
- show->state.member = m;
- if (show->state.depth_check != show->state.depth + 1)
- return;
- show->state.depth_check = 0;
-
- if (show->state.depth_to_show <= show->state.depth)
- return;
- /*
- * Reaching here indicates we have recursed and found
- * non-zero child values.
- */
- }
-
- __btf_struct_show(btf, t, type_id, data, bits_offset, show);
-}
-
-static const struct btf_kind_operations struct_ops = {
- .check_meta = btf_struct_check_meta,
- .resolve = btf_struct_resolve,
- .check_member = btf_struct_check_member,
- .check_kflag_member = btf_generic_check_kflag_member,
- .log_details = btf_struct_log,
- .show = btf_struct_show,
-};
-
-static int btf_enum_check_member(struct btf_verifier_env *env,
- const struct btf_type *struct_type,
- const struct btf_member *member,
- const struct btf_type *member_type)
-{
- u32 struct_bits_off = member->offset;
- u32 struct_size, bytes_offset;
-
- if (BITS_PER_BYTE_MASKED(struct_bits_off)) {
- btf_verifier_log_member(env, struct_type, member,
- "Member is not byte aligned");
- return -EINVAL;
- }
-
- struct_size = struct_type->size;
- bytes_offset = BITS_ROUNDDOWN_BYTES(struct_bits_off);
- if (struct_size - bytes_offset < member_type->size) {
- btf_verifier_log_member(env, struct_type, member,
- "Member exceeds struct_size");
- return -EINVAL;
- }
-
- return 0;
-}
-
-static int btf_enum_check_kflag_member(struct btf_verifier_env *env,
- const struct btf_type *struct_type,
- const struct btf_member *member,
- const struct btf_type *member_type)
-{
- u32 struct_bits_off, nr_bits, bytes_end, struct_size;
- u32 int_bitsize = sizeof(int) * BITS_PER_BYTE;
-
- struct_bits_off = BTF_MEMBER_BIT_OFFSET(member->offset);
- nr_bits = BTF_MEMBER_BITFIELD_SIZE(member->offset);
- if (!nr_bits) {
- if (BITS_PER_BYTE_MASKED(struct_bits_off)) {
- btf_verifier_log_member(env, struct_type, member,
- "Member is not byte aligned");
- return -EINVAL;
- }
-
- nr_bits = int_bitsize;
- } else if (nr_bits > int_bitsize) {
- btf_verifier_log_member(env, struct_type, member,
- "Invalid member bitfield_size");
- return -EINVAL;
- }
-
- struct_size = struct_type->size;
- bytes_end = BITS_ROUNDUP_BYTES(struct_bits_off + nr_bits);
- if (struct_size < bytes_end) {
- btf_verifier_log_member(env, struct_type, member,
- "Member exceeds struct_size");
- return -EINVAL;
- }
-
- return 0;
-}
-
-static s32 btf_enum_check_meta(struct btf_verifier_env *env,
- const struct btf_type *t,
- u32 meta_left)
-{
- const struct btf_enum *enums = btf_type_enum(t);
- struct btf *btf = env->btf;
- const char *fmt_str;
- u16 i, nr_enums;
- u32 meta_needed;
-
- nr_enums = btf_type_vlen(t);
- meta_needed = nr_enums * sizeof(*enums);
-
- if (meta_left < meta_needed) {
- btf_verifier_log_basic(env, t,
- "meta_left:%u meta_needed:%u",
- meta_left, meta_needed);
- return -EINVAL;
- }
-
- if (t->size > 8 || !is_power_of_2(t->size)) {
- btf_verifier_log_type(env, t, "Unexpected size");
- return -EINVAL;
- }
-
- /* enum type either no name or a valid one */
- if (t->name_off &&
- !btf_name_valid_identifier(env->btf, t->name_off)) {
- btf_verifier_log_type(env, t, "Invalid name");
- return -EINVAL;
- }
-
- btf_verifier_log_type(env, t, NULL);
-
- for (i = 0; i < nr_enums; i++) {
- if (!btf_name_offset_valid(btf, enums[i].name_off)) {
- btf_verifier_log(env, "\tInvalid name_offset:%u",
- enums[i].name_off);
- return -EINVAL;
- }
-
- /* enum member must have a valid name */
- if (!enums[i].name_off ||
- !btf_name_valid_identifier(btf, enums[i].name_off)) {
- btf_verifier_log_type(env, t, "Invalid name");
- return -EINVAL;
- }
-
- if (env->log.level == BPF_LOG_KERNEL)
- continue;
- fmt_str = btf_type_kflag(t) ? "\t%s val=%d\n" : "\t%s val=%u\n";
- btf_verifier_log(env, fmt_str,
- __btf_name_by_offset(btf, enums[i].name_off),
- enums[i].val);
- }
-
- return meta_needed;
-}
-
-static void btf_enum_log(struct btf_verifier_env *env,
- const struct btf_type *t)
-{
- btf_verifier_log(env, "size=%u vlen=%u", t->size, btf_type_vlen(t));
-}
-
-static void btf_enum_show(const struct btf *btf, const struct btf_type *t,
- u32 type_id, void *data, u8 bits_offset,
- struct btf_show *show)
-{
- const struct btf_enum *enums = btf_type_enum(t);
- u32 i, nr_enums = btf_type_vlen(t);
- void *safe_data;
- int v;
-
- safe_data = btf_show_start_type(show, t, type_id, data);
- if (!safe_data)
- return;
-
- v = *(int *)safe_data;
-
- for (i = 0; i < nr_enums; i++) {
- if (v != enums[i].val)
- continue;
-
- btf_show_type_value(show, "%s",
- __btf_name_by_offset(btf,
- enums[i].name_off));
-
- btf_show_end_type(show);
- return;
- }
-
- if (btf_type_kflag(t))
- btf_show_type_value(show, "%d", v);
- else
- btf_show_type_value(show, "%u", v);
- btf_show_end_type(show);
-}
-
-static const struct btf_kind_operations enum_ops = {
- .check_meta = btf_enum_check_meta,
- .resolve = btf_df_resolve,
- .check_member = btf_enum_check_member,
- .check_kflag_member = btf_enum_check_kflag_member,
- .log_details = btf_enum_log,
- .show = btf_enum_show,
-};
-
-static s32 btf_enum64_check_meta(struct btf_verifier_env *env,
- const struct btf_type *t,
- u32 meta_left)
-{
- const struct btf_enum64 *enums = btf_type_enum64(t);
- struct btf *btf = env->btf;
- const char *fmt_str;
- u16 i, nr_enums;
- u32 meta_needed;
-
- nr_enums = btf_type_vlen(t);
- meta_needed = nr_enums * sizeof(*enums);
-
- if (meta_left < meta_needed) {
- btf_verifier_log_basic(env, t,
- "meta_left:%u meta_needed:%u",
- meta_left, meta_needed);
- return -EINVAL;
- }
-
- if (t->size > 8 || !is_power_of_2(t->size)) {
- btf_verifier_log_type(env, t, "Unexpected size");
- return -EINVAL;
- }
-
- /* enum type either no name or a valid one */
- if (t->name_off &&
- !btf_name_valid_identifier(env->btf, t->name_off)) {
- btf_verifier_log_type(env, t, "Invalid name");
- return -EINVAL;
- }
-
- btf_verifier_log_type(env, t, NULL);
-
- for (i = 0; i < nr_enums; i++) {
- if (!btf_name_offset_valid(btf, enums[i].name_off)) {
- btf_verifier_log(env, "\tInvalid name_offset:%u",
- enums[i].name_off);
- return -EINVAL;
- }
-
- /* enum member must have a valid name */
- if (!enums[i].name_off ||
- !btf_name_valid_identifier(btf, enums[i].name_off)) {
- btf_verifier_log_type(env, t, "Invalid name");
- return -EINVAL;
- }
-
- if (env->log.level == BPF_LOG_KERNEL)
- continue;
-
- fmt_str = btf_type_kflag(t) ? "\t%s val=%lld\n" : "\t%s val=%llu\n";
- btf_verifier_log(env, fmt_str,
- __btf_name_by_offset(btf, enums[i].name_off),
- btf_enum64_value(enums + i));
- }
-
- return meta_needed;
-}
-
-static void btf_enum64_show(const struct btf *btf, const struct btf_type *t,
- u32 type_id, void *data, u8 bits_offset,
- struct btf_show *show)
-{
- const struct btf_enum64 *enums = btf_type_enum64(t);
- u32 i, nr_enums = btf_type_vlen(t);
- void *safe_data;
- s64 v;
-
- safe_data = btf_show_start_type(show, t, type_id, data);
- if (!safe_data)
- return;
-
- v = *(u64 *)safe_data;
-
- for (i = 0; i < nr_enums; i++) {
- if (v != btf_enum64_value(enums + i))
- continue;
-
- btf_show_type_value(show, "%s",
- __btf_name_by_offset(btf,
- enums[i].name_off));
-
- btf_show_end_type(show);
- return;
- }
-
- if (btf_type_kflag(t))
- btf_show_type_value(show, "%lld", v);
- else
- btf_show_type_value(show, "%llu", v);
- btf_show_end_type(show);
-}
-
-static const struct btf_kind_operations enum64_ops = {
- .check_meta = btf_enum64_check_meta,
- .resolve = btf_df_resolve,
- .check_member = btf_enum_check_member,
- .check_kflag_member = btf_enum_check_kflag_member,
- .log_details = btf_enum_log,
- .show = btf_enum64_show,
-};
-
-static s32 btf_func_proto_check_meta(struct btf_verifier_env *env,
- const struct btf_type *t,
- u32 meta_left)
-{
- u32 meta_needed = btf_type_vlen(t) * sizeof(struct btf_param);
-
- if (meta_left < meta_needed) {
- btf_verifier_log_basic(env, t,
- "meta_left:%u meta_needed:%u",
- meta_left, meta_needed);
- return -EINVAL;
- }
-
- if (t->name_off) {
- btf_verifier_log_type(env, t, "Invalid name");
- return -EINVAL;
- }
-
- if (btf_type_kflag(t)) {
- btf_verifier_log_type(env, t, "Invalid btf_info kind_flag");
- return -EINVAL;
- }
-
- btf_verifier_log_type(env, t, NULL);
-
- return meta_needed;
-}
-
-static void btf_func_proto_log(struct btf_verifier_env *env,
- const struct btf_type *t)
-{
- const struct btf_param *args = (const struct btf_param *)(t + 1);
- u16 nr_args = btf_type_vlen(t), i;
-
- btf_verifier_log(env, "return=%u args=(", t->type);
- if (!nr_args) {
- btf_verifier_log(env, "void");
- goto done;
- }
-
- if (nr_args == 1 && !args[0].type) {
- /* Only one vararg */
- btf_verifier_log(env, "vararg");
- goto done;
- }
-
- btf_verifier_log(env, "%u %s", args[0].type,
- __btf_name_by_offset(env->btf,
- args[0].name_off));
- for (i = 1; i < nr_args - 1; i++)
- btf_verifier_log(env, ", %u %s", args[i].type,
- __btf_name_by_offset(env->btf,
- args[i].name_off));
-
- if (nr_args > 1) {
- const struct btf_param *last_arg = &args[nr_args - 1];
-
- if (last_arg->type)
- btf_verifier_log(env, ", %u %s", last_arg->type,
- __btf_name_by_offset(env->btf,
- last_arg->name_off));
- else
- btf_verifier_log(env, ", vararg");
- }
-
-done:
- btf_verifier_log(env, ")");
-}
-
-static const struct btf_kind_operations func_proto_ops = {
- .check_meta = btf_func_proto_check_meta,
- .resolve = btf_df_resolve,
- /*
- * BTF_KIND_FUNC_PROTO cannot be directly referred by
- * a struct's member.
- *
- * It should be a function pointer instead.
- * (i.e. struct's member -> BTF_KIND_PTR -> BTF_KIND_FUNC_PROTO)
- *
- * Hence, there is no btf_func_check_member().
- */
- .check_member = btf_df_check_member,
- .check_kflag_member = btf_df_check_kflag_member,
- .log_details = btf_func_proto_log,
- .show = btf_df_show,
-};
-
-static s32 btf_func_check_meta(struct btf_verifier_env *env,
- const struct btf_type *t,
- u32 meta_left)
-{
- if (!t->name_off ||
- !btf_name_valid_identifier(env->btf, t->name_off)) {
- btf_verifier_log_type(env, t, "Invalid name");
- return -EINVAL;
- }
-
- if (btf_type_vlen(t) > BTF_FUNC_GLOBAL) {
- btf_verifier_log_type(env, t, "Invalid func linkage");
- return -EINVAL;
- }
-
- if (btf_type_kflag(t)) {
- btf_verifier_log_type(env, t, "Invalid btf_info kind_flag");
- return -EINVAL;
- }
-
- btf_verifier_log_type(env, t, NULL);
-
- return 0;
-}
-
-static int btf_func_resolve(struct btf_verifier_env *env,
- const struct resolve_vertex *v)
-{
- const struct btf_type *t = v->t;
- u32 next_type_id = t->type;
- int err;
-
- err = btf_func_check(env, t);
- if (err)
- return err;
-
- env_stack_pop_resolved(env, next_type_id, 0);
- return 0;
-}
-
-static const struct btf_kind_operations func_ops = {
- .check_meta = btf_func_check_meta,
- .resolve = btf_func_resolve,
- .check_member = btf_df_check_member,
- .check_kflag_member = btf_df_check_kflag_member,
- .log_details = btf_ref_type_log,
- .show = btf_df_show,
-};
-
-static s32 btf_var_check_meta(struct btf_verifier_env *env,
- const struct btf_type *t,
- u32 meta_left)
-{
- const struct btf_var *var;
- u32 meta_needed = sizeof(*var);
-
- if (meta_left < meta_needed) {
- btf_verifier_log_basic(env, t,
- "meta_left:%u meta_needed:%u",
- meta_left, meta_needed);
- return -EINVAL;
- }
-
- if (btf_type_vlen(t)) {
- btf_verifier_log_type(env, t, "vlen != 0");
- return -EINVAL;
- }
-
- if (btf_type_kflag(t)) {
- btf_verifier_log_type(env, t, "Invalid btf_info kind_flag");
- return -EINVAL;
- }
-
- if (!t->name_off ||
- !btf_name_valid_identifier(env->btf, t->name_off)) {
- btf_verifier_log_type(env, t, "Invalid name");
- return -EINVAL;
- }
-
- /* A var cannot be in type void */
- if (!t->type || !BTF_TYPE_ID_VALID(t->type)) {
- btf_verifier_log_type(env, t, "Invalid type_id");
- return -EINVAL;
- }
-
- var = btf_type_var(t);
- if (var->linkage != BTF_VAR_STATIC &&
- var->linkage != BTF_VAR_GLOBAL_ALLOCATED) {
- btf_verifier_log_type(env, t, "Linkage not supported");
- return -EINVAL;
- }
-
- btf_verifier_log_type(env, t, NULL);
-
- return meta_needed;
-}
-
-static void btf_var_log(struct btf_verifier_env *env, const struct btf_type *t)
-{
- const struct btf_var *var = btf_type_var(t);
-
- btf_verifier_log(env, "type_id=%u linkage=%u", t->type, var->linkage);
-}
-
-static const struct btf_kind_operations var_ops = {
- .check_meta = btf_var_check_meta,
- .resolve = btf_var_resolve,
- .check_member = btf_df_check_member,
- .check_kflag_member = btf_df_check_kflag_member,
- .log_details = btf_var_log,
- .show = btf_var_show,
-};
-
-static s32 btf_datasec_check_meta(struct btf_verifier_env *env,
- const struct btf_type *t,
- u32 meta_left)
-{
- const struct btf_var_secinfo *vsi;
- u64 last_vsi_end_off = 0, sum = 0;
- u32 i, meta_needed;
-
- meta_needed = btf_type_vlen(t) * sizeof(*vsi);
- if (meta_left < meta_needed) {
- btf_verifier_log_basic(env, t,
- "meta_left:%u meta_needed:%u",
- meta_left, meta_needed);
- return -EINVAL;
- }
-
- if (!t->size) {
- btf_verifier_log_type(env, t, "size == 0");
- return -EINVAL;
- }
-
- if (btf_type_kflag(t)) {
- btf_verifier_log_type(env, t, "Invalid btf_info kind_flag");
- return -EINVAL;
- }
-
- if (!t->name_off ||
- !btf_name_valid_section(env->btf, t->name_off)) {
- btf_verifier_log_type(env, t, "Invalid name");
- return -EINVAL;
- }
-
- btf_verifier_log_type(env, t, NULL);
-
- for_each_vsi(i, t, vsi) {
- /* A var cannot be in type void */
- if (!vsi->type || !BTF_TYPE_ID_VALID(vsi->type)) {
- btf_verifier_log_vsi(env, t, vsi,
- "Invalid type_id");
- return -EINVAL;
- }
-
- if (vsi->offset < last_vsi_end_off || vsi->offset >= t->size) {
- btf_verifier_log_vsi(env, t, vsi,
- "Invalid offset");
- return -EINVAL;
- }
-
- if (!vsi->size || vsi->size > t->size) {
- btf_verifier_log_vsi(env, t, vsi,
- "Invalid size");
- return -EINVAL;
- }
-
- last_vsi_end_off = vsi->offset + vsi->size;
- if (last_vsi_end_off > t->size) {
- btf_verifier_log_vsi(env, t, vsi,
- "Invalid offset+size");
- return -EINVAL;
- }
-
- btf_verifier_log_vsi(env, t, vsi, NULL);
- sum += vsi->size;
- }
-
- if (t->size < sum) {
- btf_verifier_log_type(env, t, "Invalid btf_info size");
- return -EINVAL;
- }
-
- return meta_needed;
-}
-
-static int btf_datasec_resolve(struct btf_verifier_env *env,
- const struct resolve_vertex *v)
-{
- const struct btf_var_secinfo *vsi;
- struct btf *btf = env->btf;
- u16 i;
-
- env->resolve_mode = RESOLVE_TBD;
- for_each_vsi_from(i, v->next_member, v->t, vsi) {
- u32 var_type_id = vsi->type, type_id, type_size = 0;
- const struct btf_type *var_type = btf_type_by_id(env->btf,
- var_type_id);
- if (!var_type || !btf_type_is_var(var_type)) {
- btf_verifier_log_vsi(env, v->t, vsi,
- "Not a VAR kind member");
- return -EINVAL;
- }
-
- if (!env_type_is_resolve_sink(env, var_type) &&
- !env_type_is_resolved(env, var_type_id)) {
- env_stack_set_next_member(env, i + 1);
- return env_stack_push(env, var_type, var_type_id);
- }
-
- type_id = var_type->type;
- if (!btf_type_id_size(btf, &type_id, &type_size)) {
- btf_verifier_log_vsi(env, v->t, vsi, "Invalid type");
- return -EINVAL;
- }
-
- if (vsi->size < type_size) {
- btf_verifier_log_vsi(env, v->t, vsi, "Invalid size");
- return -EINVAL;
- }
- }
-
- env_stack_pop_resolved(env, 0, 0);
- return 0;
-}
-
-static void btf_datasec_log(struct btf_verifier_env *env,
- const struct btf_type *t)
-{
- btf_verifier_log(env, "size=%u vlen=%u", t->size, btf_type_vlen(t));
-}
-
-static void btf_datasec_show(const struct btf *btf,
- const struct btf_type *t, u32 type_id,
- void *data, u8 bits_offset,
- struct btf_show *show)
-{
- const struct btf_var_secinfo *vsi;
- const struct btf_type *var;
- u32 i;
-
- if (!btf_show_start_type(show, t, type_id, data))
- return;
-
- btf_show_type_value(show, "section (\"%s\") = {",
- __btf_name_by_offset(btf, t->name_off));
- for_each_vsi(i, t, vsi) {
- var = btf_type_by_id(btf, vsi->type);
- if (i)
- btf_show(show, ",");
- btf_type_ops(var)->show(btf, var, vsi->type,
- data + vsi->offset, bits_offset, show);
- }
- btf_show_end_type(show);
-}
-
-static const struct btf_kind_operations datasec_ops = {
- .check_meta = btf_datasec_check_meta,
- .resolve = btf_datasec_resolve,
- .check_member = btf_df_check_member,
- .check_kflag_member = btf_df_check_kflag_member,
- .log_details = btf_datasec_log,
- .show = btf_datasec_show,
-};
-
-static s32 btf_float_check_meta(struct btf_verifier_env *env,
- const struct btf_type *t,
- u32 meta_left)
-{
- if (btf_type_vlen(t)) {
- btf_verifier_log_type(env, t, "vlen != 0");
- return -EINVAL;
- }
-
- if (btf_type_kflag(t)) {
- btf_verifier_log_type(env, t, "Invalid btf_info kind_flag");
- return -EINVAL;
- }
-
- if (t->size != 2 && t->size != 4 && t->size != 8 && t->size != 12 &&
- t->size != 16) {
- btf_verifier_log_type(env, t, "Invalid type_size");
- return -EINVAL;
- }
-
- btf_verifier_log_type(env, t, NULL);
-
- return 0;
-}
-
-static int btf_float_check_member(struct btf_verifier_env *env,
- const struct btf_type *struct_type,
- const struct btf_member *member,
- const struct btf_type *member_type)
-{
- u64 start_offset_bytes;
- u64 end_offset_bytes;
- u64 misalign_bits;
- u64 align_bytes;
- u64 align_bits;
-
- /* Different architectures have different alignment requirements, so
- * here we check only for the reasonable minimum. This way we ensure
- * that types after CO-RE can pass the kernel BTF verifier.
- */
- align_bytes = min_t(u64, sizeof(void *), member_type->size);
- align_bits = align_bytes * BITS_PER_BYTE;
- div64_u64_rem(member->offset, align_bits, &misalign_bits);
- if (misalign_bits) {
- btf_verifier_log_member(env, struct_type, member,
- "Member is not properly aligned");
- return -EINVAL;
- }
-
- start_offset_bytes = member->offset / BITS_PER_BYTE;
- end_offset_bytes = start_offset_bytes + member_type->size;
- if (end_offset_bytes > struct_type->size) {
- btf_verifier_log_member(env, struct_type, member,
- "Member exceeds struct_size");
- return -EINVAL;
- }
-
- return 0;
-}
-
-static void btf_float_log(struct btf_verifier_env *env,
- const struct btf_type *t)
-{
- btf_verifier_log(env, "size=%u", t->size);
-}
-
-static const struct btf_kind_operations float_ops = {
- .check_meta = btf_float_check_meta,
- .resolve = btf_df_resolve,
- .check_member = btf_float_check_member,
- .check_kflag_member = btf_generic_check_kflag_member,
- .log_details = btf_float_log,
- .show = btf_df_show,
-};
-
-static s32 btf_decl_tag_check_meta(struct btf_verifier_env *env,
- const struct btf_type *t,
- u32 meta_left)
-{
- const struct btf_decl_tag *tag;
- u32 meta_needed = sizeof(*tag);
- s32 component_idx;
- const char *value;
-
- if (meta_left < meta_needed) {
- btf_verifier_log_basic(env, t,
- "meta_left:%u meta_needed:%u",
- meta_left, meta_needed);
- return -EINVAL;
- }
-
- value = btf_name_by_offset(env->btf, t->name_off);
- if (!value || !value[0]) {
- btf_verifier_log_type(env, t, "Invalid value");
- return -EINVAL;
- }
-
- if (btf_type_vlen(t)) {
- btf_verifier_log_type(env, t, "vlen != 0");
- return -EINVAL;
- }
-
- component_idx = btf_type_decl_tag(t)->component_idx;
- if (component_idx < -1) {
- btf_verifier_log_type(env, t, "Invalid component_idx");
- return -EINVAL;
- }
-
- btf_verifier_log_type(env, t, NULL);
-
- return meta_needed;
-}
-
-static int btf_decl_tag_resolve(struct btf_verifier_env *env,
- const struct resolve_vertex *v)
-{
- const struct btf_type *next_type;
- const struct btf_type *t = v->t;
- u32 next_type_id = t->type;
- struct btf *btf = env->btf;
- s32 component_idx;
- u32 vlen;
-
- next_type = btf_type_by_id(btf, next_type_id);
- if (!next_type || !btf_type_is_decl_tag_target(next_type)) {
- btf_verifier_log_type(env, v->t, "Invalid type_id");
- return -EINVAL;
- }
-
- if (!env_type_is_resolve_sink(env, next_type) &&
- !env_type_is_resolved(env, next_type_id))
- return env_stack_push(env, next_type, next_type_id);
-
- component_idx = btf_type_decl_tag(t)->component_idx;
- if (component_idx != -1) {
- if (btf_type_is_var(next_type) || btf_type_is_typedef(next_type)) {
- btf_verifier_log_type(env, v->t, "Invalid component_idx");
- return -EINVAL;
- }
-
- if (btf_type_is_struct(next_type)) {
- vlen = btf_type_vlen(next_type);
- } else {
- /* next_type should be a function */
- next_type = btf_type_by_id(btf, next_type->type);
- vlen = btf_type_vlen(next_type);
- }
-
- if ((u32)component_idx >= vlen) {
- btf_verifier_log_type(env, v->t, "Invalid component_idx");
- return -EINVAL;
- }
- }
-
- env_stack_pop_resolved(env, next_type_id, 0);
-
- return 0;
-}
-
-static void btf_decl_tag_log(struct btf_verifier_env *env, const struct btf_type *t)
-{
- btf_verifier_log(env, "type=%u component_idx=%d", t->type,
- btf_type_decl_tag(t)->component_idx);
-}
-
-static const struct btf_kind_operations decl_tag_ops = {
- .check_meta = btf_decl_tag_check_meta,
- .resolve = btf_decl_tag_resolve,
- .check_member = btf_df_check_member,
- .check_kflag_member = btf_df_check_kflag_member,
- .log_details = btf_decl_tag_log,
- .show = btf_df_show,
-};
-
-static int btf_func_proto_check(struct btf_verifier_env *env,
- const struct btf_type *t)
-{
- const struct btf_type *ret_type;
- const struct btf_param *args;
- const struct btf *btf;
- u16 nr_args, i;
- int err;
-
- btf = env->btf;
- args = (const struct btf_param *)(t + 1);
- nr_args = btf_type_vlen(t);
-
- /* Check func return type which could be "void" (t->type == 0) */
- if (t->type) {
- u32 ret_type_id = t->type;
-
- ret_type = btf_type_by_id(btf, ret_type_id);
- if (!ret_type) {
- btf_verifier_log_type(env, t, "Invalid return type");
- return -EINVAL;
- }
-
- if (btf_type_is_resolve_source_only(ret_type)) {
- btf_verifier_log_type(env, t, "Invalid return type");
- return -EINVAL;
- }
-
- if (btf_type_needs_resolve(ret_type) &&
- !env_type_is_resolved(env, ret_type_id)) {
- err = btf_resolve(env, ret_type, ret_type_id);
- if (err)
- return err;
- }
-
- /* Ensure the return type is a type that has a size */
- if (!btf_type_id_size(btf, &ret_type_id, NULL)) {
- btf_verifier_log_type(env, t, "Invalid return type");
- return -EINVAL;
- }
- }
-
- if (!nr_args)
- return 0;
-
- /* Last func arg type_id could be 0 if it is a vararg */
- if (!args[nr_args - 1].type) {
- if (args[nr_args - 1].name_off) {
- btf_verifier_log_type(env, t, "Invalid arg#%u",
- nr_args);
- return -EINVAL;
- }
- nr_args--;
- }
-
- for (i = 0; i < nr_args; i++) {
- const struct btf_type *arg_type;
- u32 arg_type_id;
-
- arg_type_id = args[i].type;
- arg_type = btf_type_by_id(btf, arg_type_id);
- if (!arg_type) {
- btf_verifier_log_type(env, t, "Invalid arg#%u", i + 1);
- return -EINVAL;
- }
-
- if (btf_type_is_resolve_source_only(arg_type)) {
- btf_verifier_log_type(env, t, "Invalid arg#%u", i + 1);
- return -EINVAL;
- }
-
- if (args[i].name_off &&
- (!btf_name_offset_valid(btf, args[i].name_off) ||
- !btf_name_valid_identifier(btf, args[i].name_off))) {
- btf_verifier_log_type(env, t,
- "Invalid arg#%u", i + 1);
- return -EINVAL;
- }
-
- if (btf_type_needs_resolve(arg_type) &&
- !env_type_is_resolved(env, arg_type_id)) {
- err = btf_resolve(env, arg_type, arg_type_id);
- if (err)
- return err;
- }
-
- if (!btf_type_id_size(btf, &arg_type_id, NULL)) {
- btf_verifier_log_type(env, t, "Invalid arg#%u", i + 1);
- return -EINVAL;
- }
- }
-
- return 0;
-}
-
-static int btf_func_check(struct btf_verifier_env *env,
- const struct btf_type *t)
-{
- const struct btf_type *proto_type;
- const struct btf_param *args;
- const struct btf *btf;
- u16 nr_args, i;
-
- btf = env->btf;
- proto_type = btf_type_by_id(btf, t->type);
-
- if (!proto_type || !btf_type_is_func_proto(proto_type)) {
- btf_verifier_log_type(env, t, "Invalid type_id");
- return -EINVAL;
- }
-
- args = (const struct btf_param *)(proto_type + 1);
- nr_args = btf_type_vlen(proto_type);
- for (i = 0; i < nr_args; i++) {
- if (!args[i].name_off && args[i].type) {
- btf_verifier_log_type(env, t, "Invalid arg#%u", i + 1);
- return -EINVAL;
- }
- }
-
- return 0;
-}
-
-static const struct btf_kind_operations * const kind_ops[NR_BTF_KINDS] = {
- [BTF_KIND_INT] = &int_ops,
- [BTF_KIND_PTR] = &ptr_ops,
- [BTF_KIND_ARRAY] = &array_ops,
- [BTF_KIND_STRUCT] = &struct_ops,
- [BTF_KIND_UNION] = &struct_ops,
- [BTF_KIND_ENUM] = &enum_ops,
- [BTF_KIND_FWD] = &fwd_ops,
- [BTF_KIND_TYPEDEF] = &modifier_ops,
- [BTF_KIND_VOLATILE] = &modifier_ops,
- [BTF_KIND_CONST] = &modifier_ops,
- [BTF_KIND_RESTRICT] = &modifier_ops,
- [BTF_KIND_FUNC] = &func_ops,
- [BTF_KIND_FUNC_PROTO] = &func_proto_ops,
- [BTF_KIND_VAR] = &var_ops,
- [BTF_KIND_DATASEC] = &datasec_ops,
- [BTF_KIND_FLOAT] = &float_ops,
- [BTF_KIND_DECL_TAG] = &decl_tag_ops,
- [BTF_KIND_TYPE_TAG] = &modifier_ops,
- [BTF_KIND_ENUM64] = &enum64_ops,
-};
-
-static s32 btf_check_meta(struct btf_verifier_env *env,
- const struct btf_type *t,
- u32 meta_left)
-{
- u32 saved_meta_left = meta_left;
- s32 var_meta_size;
-
- if (meta_left < sizeof(*t)) {
- btf_verifier_log(env, "[%u] meta_left:%u meta_needed:%zu",
- env->log_type_id, meta_left, sizeof(*t));
- return -EINVAL;
- }
- meta_left -= sizeof(*t);
-
- if (t->info & ~BTF_INFO_MASK) {
- btf_verifier_log(env, "[%u] Invalid btf_info:%x",
- env->log_type_id, t->info);
- return -EINVAL;
- }
-
- if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX ||
- BTF_INFO_KIND(t->info) == BTF_KIND_UNKN) {
- btf_verifier_log(env, "[%u] Invalid kind:%u",
- env->log_type_id, BTF_INFO_KIND(t->info));
- return -EINVAL;
- }
-
- if (!btf_name_offset_valid(env->btf, t->name_off)) {
- btf_verifier_log(env, "[%u] Invalid name_offset:%u",
- env->log_type_id, t->name_off);
- return -EINVAL;
- }
-
- var_meta_size = btf_type_ops(t)->check_meta(env, t, meta_left);
- if (var_meta_size < 0)
- return var_meta_size;
-
- meta_left -= var_meta_size;
-
- return saved_meta_left - meta_left;
-}
-
-static int btf_check_all_metas(struct btf_verifier_env *env)
-{
- struct btf *btf = env->btf;
- struct btf_header *hdr;
- void *cur, *end;
-
- hdr = &btf->hdr;
- cur = btf->nohdr_data + hdr->type_off;
- end = cur + hdr->type_len;
-
- env->log_type_id = btf->base_btf ? btf->start_id : 1;
- while (cur < end) {
- struct btf_type *t = cur;
- s32 meta_size;
-
- meta_size = btf_check_meta(env, t, end - cur);
- if (meta_size < 0)
- return meta_size;
-
- btf_add_type(env, t);
- cur += meta_size;
- env->log_type_id++;
- }
-
- return 0;
-}
-
-static bool btf_resolve_valid(struct btf_verifier_env *env,
- const struct btf_type *t,
- u32 type_id)
-{
- struct btf *btf = env->btf;
-
- if (!env_type_is_resolved(env, type_id))
- return false;
-
- if (btf_type_is_struct(t) || btf_type_is_datasec(t))
- return !btf_resolved_type_id(btf, type_id) &&
- !btf_resolved_type_size(btf, type_id);
-
- if (btf_type_is_decl_tag(t) || btf_type_is_func(t))
- return btf_resolved_type_id(btf, type_id) &&
- !btf_resolved_type_size(btf, type_id);
-
- if (btf_type_is_modifier(t) || btf_type_is_ptr(t) ||
- btf_type_is_var(t)) {
- t = btf_type_id_resolve(btf, &type_id);
- return t &&
- !btf_type_is_modifier(t) &&
- !btf_type_is_var(t) &&
- !btf_type_is_datasec(t);
- }
-
- if (btf_type_is_array(t)) {
- const struct btf_array *array = btf_type_array(t);
- const struct btf_type *elem_type;
- u32 elem_type_id = array->type;
- u32 elem_size;
-
- elem_type = btf_type_id_size(btf, &elem_type_id, &elem_size);
- return elem_type && !btf_type_is_modifier(elem_type) &&
- (array->nelems * elem_size ==
- btf_resolved_type_size(btf, type_id));
- }
-
- return false;
-}
-
-static int btf_resolve(struct btf_verifier_env *env,
- const struct btf_type *t, u32 type_id)
-{
- u32 save_log_type_id = env->log_type_id;
- const struct resolve_vertex *v;
- int err = 0;
-
- env->resolve_mode = RESOLVE_TBD;
- env_stack_push(env, t, type_id);
- while (!err && (v = env_stack_peak(env))) {
- env->log_type_id = v->type_id;
- err = btf_type_ops(v->t)->resolve(env, v);
- }
-
- env->log_type_id = type_id;
- if (err == -E2BIG) {
- btf_verifier_log_type(env, t,
- "Exceeded max resolving depth:%u",
- MAX_RESOLVE_DEPTH);
- } else if (err == -EEXIST) {
- btf_verifier_log_type(env, t, "Loop detected");
- }
-
- /* Final sanity check */
- if (!err && !btf_resolve_valid(env, t, type_id)) {
- btf_verifier_log_type(env, t, "Invalid resolve state");
- err = -EINVAL;
- }
-
- env->log_type_id = save_log_type_id;
- return err;
-}
-
-static int btf_check_all_types(struct btf_verifier_env *env)
-{
- struct btf *btf = env->btf;
- const struct btf_type *t;
- u32 type_id, i;
- int err;
-
- err = env_resolve_init(env);
- if (err)
- return err;
-
- env->phase++;
- for (i = btf->base_btf ? 0 : 1; i < btf->nr_types; i++) {
- type_id = btf->start_id + i;
- t = btf_type_by_id(btf, type_id);
-
- env->log_type_id = type_id;
- if (btf_type_needs_resolve(t) &&
- !env_type_is_resolved(env, type_id)) {
- err = btf_resolve(env, t, type_id);
- if (err)
- return err;
- }
-
- if (btf_type_is_func_proto(t)) {
- err = btf_func_proto_check(env, t);
- if (err)
- return err;
- }
- }
-
- return 0;
-}
-
-static int btf_parse_type_sec(struct btf_verifier_env *env)
-{
- const struct btf_header *hdr = &env->btf->hdr;
- int err;
-
- /* Type section must align to 4 bytes */
- if (hdr->type_off & (sizeof(u32) - 1)) {
- btf_verifier_log(env, "Unaligned type_off");
- return -EINVAL;
- }
-
- if (!env->btf->base_btf && !hdr->type_len) {
- btf_verifier_log(env, "No type found");
- return -EINVAL;
- }
-
- err = btf_check_all_metas(env);
- if (err)
- return err;
-
- return btf_check_all_types(env);
-}
-
-static int btf_parse_str_sec(struct btf_verifier_env *env)
-{
- const struct btf_header *hdr;
- struct btf *btf = env->btf;
- const char *start, *end;
-
- hdr = &btf->hdr;
- start = btf->nohdr_data + hdr->str_off;
- end = start + hdr->str_len;
-
- if (end != btf->data + btf->data_size) {
- btf_verifier_log(env, "String section is not at the end");
- return -EINVAL;
- }
-
- btf->strings = start;
-
- if (btf->base_btf && !hdr->str_len)
- return 0;
- if (!hdr->str_len || hdr->str_len - 1 > BTF_MAX_NAME_OFFSET || end[-1]) {
- btf_verifier_log(env, "Invalid string section");
- return -EINVAL;
- }
- if (!btf->base_btf && start[0]) {
- btf_verifier_log(env, "Invalid string section");
- return -EINVAL;
- }
-
- return 0;
-}
-
-static const size_t btf_sec_info_offset[] = {
- offsetof(struct btf_header, type_off),
- offsetof(struct btf_header, str_off),
-};
-
-static int btf_sec_info_cmp(const void *a, const void *b)
-{
- const struct btf_sec_info *x = a;
- const struct btf_sec_info *y = b;
-
- return (int)(x->off - y->off) ? : (int)(x->len - y->len);
-}
+ if (rec->spin_lock_off >= 0 && rec->res_spin_lock_off >= 0) {
+ ret = -EINVAL;
+ goto end;
+ }
-static int btf_check_sec_info(struct btf_verifier_env *env,
- u32 btf_data_size)
-{
- struct btf_sec_info secs[ARRAY_SIZE(btf_sec_info_offset)];
- u32 total, expected_total, i;
- const struct btf_header *hdr;
- const struct btf *btf;
-
- btf = env->btf;
- hdr = &btf->hdr;
-
- /* Populate the secs from hdr */
- for (i = 0; i < ARRAY_SIZE(btf_sec_info_offset); i++)
- secs[i] = *(struct btf_sec_info *)((void *)hdr +
- btf_sec_info_offset[i]);
-
- sort(secs, ARRAY_SIZE(btf_sec_info_offset),
- sizeof(struct btf_sec_info), btf_sec_info_cmp, NULL);
-
- /* Check for gaps and overlap among sections */
- total = 0;
- expected_total = btf_data_size - hdr->hdr_len;
- for (i = 0; i < ARRAY_SIZE(btf_sec_info_offset); i++) {
- if (expected_total < secs[i].off) {
- btf_verifier_log(env, "Invalid section offset");
- return -EINVAL;
- }
- if (total < secs[i].off) {
- /* gap */
- btf_verifier_log(env, "Unsupported section found");
- return -EINVAL;
- }
- if (total > secs[i].off) {
- btf_verifier_log(env, "Section overlap found");
- return -EINVAL;
- }
- if (expected_total - total < secs[i].len) {
- btf_verifier_log(env,
- "Total section length too long");
- return -EINVAL;
- }
- total += secs[i].len;
+ /* bpf_{list_head, rb_node} require bpf_spin_lock */
+ if ((btf_record_has_field(rec, BPF_LIST_HEAD) ||
+ btf_record_has_field(rec, BPF_RB_ROOT)) &&
+ (rec->spin_lock_off < 0 && rec->res_spin_lock_off < 0)) {
+ ret = -EINVAL;
+ goto end;
}
- /* There is data other than hdr and known sections */
- if (expected_total != total) {
- btf_verifier_log(env, "Unsupported section found");
- return -EINVAL;
+ if (rec->refcount_off < 0 &&
+ btf_record_has_field(rec, BPF_LIST_NODE) &&
+ btf_record_has_field(rec, BPF_RB_NODE)) {
+ ret = -EINVAL;
+ goto end;
}
- return 0;
+ sort_r(rec->fields, rec->cnt, sizeof(struct btf_field), btf_field_cmp,
+ NULL, rec);
+
+ return rec;
+end:
+ btf_record_free(rec);
+ return ERR_PTR(ret);
}
-static int btf_parse_hdr(struct btf_verifier_env *env)
+int btf_check_and_fixup_fields(const struct btf *btf, struct btf_record *rec)
{
- u32 hdr_len, hdr_copy, btf_data_size;
- const struct btf_header *hdr;
- struct btf *btf;
-
- btf = env->btf;
- btf_data_size = btf->data_size;
-
- if (btf_data_size < offsetofend(struct btf_header, hdr_len)) {
- btf_verifier_log(env, "hdr_len not found");
- return -EINVAL;
- }
-
- hdr = btf->data;
- hdr_len = hdr->hdr_len;
- if (btf_data_size < hdr_len) {
- btf_verifier_log(env, "btf_header not found");
- return -EINVAL;
- }
+ int i;
- /* Ensure the unsupported header fields are zero */
- if (hdr_len > sizeof(btf->hdr)) {
- u8 *expected_zero = btf->data + sizeof(btf->hdr);
- u8 *end = btf->data + hdr_len;
+ /* There are three types that signify ownership of some other type:
+ * kptr_ref, bpf_list_head, bpf_rb_root.
+ * kptr_ref only supports storing kernel types, which can't store
+ * references to program allocated local types.
+ *
+ * Hence we only need to ensure that bpf_{list_head,rb_root} ownership
+ * does not form cycles.
+ */
+ if (IS_ERR_OR_NULL(rec) || !(rec->field_mask & (BPF_GRAPH_ROOT | BPF_UPTR)))
+ return 0;
+ for (i = 0; i < rec->cnt; i++) {
+ struct btf_struct_meta *meta;
+ const struct btf_type *t;
+ u32 btf_id;
- for (; expected_zero < end; expected_zero++) {
- if (*expected_zero) {
- btf_verifier_log(env, "Unsupported btf_header");
+ if (rec->fields[i].type == BPF_UPTR) {
+ /* The uptr only supports pinning one page and cannot
+ * point to a kernel struct
+ */
+ if (btf_is_kernel(rec->fields[i].kptr.btf))
+ return -EINVAL;
+ t = btf_type_by_id(rec->fields[i].kptr.btf,
+ rec->fields[i].kptr.btf_id);
+ if (!t->size)
+ return -EINVAL;
+ if (t->size > PAGE_SIZE)
return -E2BIG;
- }
+ continue;
}
- }
-
- hdr_copy = min_t(u32, hdr_len, sizeof(btf->hdr));
- memcpy(&btf->hdr, btf->data, hdr_copy);
- hdr = &btf->hdr;
-
- btf_verifier_log_hdr(env, btf_data_size);
-
- if (hdr->magic != BTF_MAGIC) {
- btf_verifier_log(env, "Invalid magic");
- return -EINVAL;
- }
-
- if (hdr->version != BTF_VERSION) {
- btf_verifier_log(env, "Unsupported version");
- return -ENOTSUPP;
- }
+ if (!(rec->fields[i].type & BPF_GRAPH_ROOT))
+ continue;
+ btf_id = rec->fields[i].graph_root.value_btf_id;
+ meta = btf_find_struct_meta(btf, btf_id);
+ if (!meta)
+ return -EFAULT;
+ rec->fields[i].graph_root.value_rec = meta->record;
- if (hdr->flags) {
- btf_verifier_log(env, "Unsupported flags");
- return -ENOTSUPP;
- }
+ /* We need to set value_rec for all root types, but no need
+ * to check ownership cycle for a type unless it's also a
+ * node type.
+ */
+ if (!(rec->field_mask & BPF_GRAPH_NODE))
+ continue;
- if (!btf->base_btf && btf_data_size == hdr->hdr_len) {
- btf_verifier_log(env, "No data");
- return -EINVAL;
+ /* We need to ensure ownership acyclicity among all types. The
+ * proper way to do it would be to topologically sort all BTF
+ * IDs based on the ownership edges, since there can be multiple
+ * bpf_{list_head,rb_node} in a type. Instead, we use the
+ * following resaoning:
+ *
+ * - A type can only be owned by another type in user BTF if it
+ * has a bpf_{list,rb}_node. Let's call these node types.
+ * - A type can only _own_ another type in user BTF if it has a
+ * bpf_{list_head,rb_root}. Let's call these root types.
+ *
+ * We ensure that if a type is both a root and node, its
+ * element types cannot be root types.
+ *
+ * To ensure acyclicity:
+ *
+ * When A is an root type but not a node, its ownership
+ * chain can be:
+ * A -> B -> C
+ * Where:
+ * - A is an root, e.g. has bpf_rb_root.
+ * - B is both a root and node, e.g. has bpf_rb_node and
+ * bpf_list_head.
+ * - C is only an root, e.g. has bpf_list_node
+ *
+ * When A is both a root and node, some other type already
+ * owns it in the BTF domain, hence it can not own
+ * another root type through any of the ownership edges.
+ * A -> B
+ * Where:
+ * - A is both an root and node.
+ * - B is only an node.
+ */
+ if (meta->record->field_mask & BPF_GRAPH_ROOT)
+ return -ELOOP;
}
-
- return btf_check_sec_info(env, btf_data_size);
+ return 0;
}
-
static const char *alloc_obj_fields[] = {
"bpf_spin_lock",
"bpf_list_head",
@@ -5818,54 +1247,6 @@ struct btf_struct_meta *btf_find_struct_meta(const struct btf *btf, u32 btf_id)
return NULL;
return bsearch(&btf_id, tab->types, tab->cnt, sizeof(tab->types[0]), btf_id_cmp_func);
}
-
-static int btf_check_type_tags(struct btf_verifier_env *env,
- struct btf *btf, int start_id)
-{
- int i, n, good_id = start_id - 1;
- bool in_tags;
-
- n = btf_nr_types(btf);
- for (i = start_id; i < n; i++) {
- const struct btf_type *t;
- int chain_limit = 32;
- u32 cur_id = i;
-
- t = btf_type_by_id(btf, i);
- if (!t)
- return -EINVAL;
- if (!btf_type_is_modifier(t))
- continue;
-
- cond_resched();
-
- in_tags = btf_type_is_type_tag(t);
- while (btf_type_is_modifier(t)) {
- if (!chain_limit--) {
- btf_verifier_log(env, "Max chain length or cycle detected");
- return -ELOOP;
- }
- if (btf_type_is_type_tag(t)) {
- if (!in_tags) {
- btf_verifier_log(env, "Type tags don't precede modifiers");
- return -EINVAL;
- }
- } else if (in_tags) {
- in_tags = false;
- }
- if (cur_id <= good_id)
- break;
- /* Move to next type */
- cur_id = t->type;
- t = btf_type_by_id(btf, cur_id);
- if (!t)
- return -EINVAL;
- }
- good_id = i;
- }
- return 0;
-}
-
static int finalize_log(struct bpf_verifier_log *log, bpfptr_t uattr, u32 uattr_size)
{
u32 log_true_size;
@@ -6313,60 +1694,6 @@ int get_kern_ctx_btf_id(struct bpf_verifier_log *log, enum bpf_prog_type prog_ty
}
BTF_ID_LIST_SINGLE(bpf_ctx_convert_btf_id, struct, bpf_ctx_convert)
-
-static struct btf *btf_parse_base(struct btf_verifier_env *env, const char *name,
- void *data, unsigned int data_size)
-{
- struct btf *btf = NULL;
- int err;
-
- if (!IS_ENABLED(CONFIG_DEBUG_INFO_BTF))
- return ERR_PTR(-ENOENT);
-
- btf = kzalloc_obj(*btf, GFP_KERNEL | __GFP_NOWARN);
- if (!btf) {
- err = -ENOMEM;
- goto errout;
- }
- env->btf = btf;
-
- btf->data = data;
- btf->data_size = data_size;
- btf->kernel_btf = true;
- btf->named_start_id = 0;
- strscpy(btf->name, name);
-
- err = btf_parse_hdr(env);
- if (err)
- goto errout;
-
- btf->nohdr_data = btf->data + btf->hdr.hdr_len;
-
- err = btf_parse_str_sec(env);
- if (err)
- goto errout;
-
- err = btf_check_all_metas(env);
- if (err)
- goto errout;
-
- err = btf_check_type_tags(env, btf, 1);
- if (err)
- goto errout;
-
- btf_check_sorted(btf);
- refcount_set(&btf->refcnt, 1);
-
- return btf;
-
-errout:
- if (btf) {
- kvfree(btf->types);
- kfree(btf);
- }
- return ERR_PTR(err);
-}
-
struct btf *btf_parse_vmlinux(void)
{
struct btf_verifier_env *env = NULL;
@@ -8029,96 +3356,6 @@ int btf_prepare_func_args(struct bpf_verifier_env *env, int subprog)
return 0;
}
-
-static void btf_type_show(const struct btf *btf, u32 type_id, void *obj,
- struct btf_show *show)
-{
- const struct btf_type *t = btf_type_by_id(btf, type_id);
-
- show->btf = btf;
- memset(&show->state, 0, sizeof(show->state));
- memset(&show->obj, 0, sizeof(show->obj));
-
- btf_type_ops(t)->show(btf, t, type_id, obj, 0, show);
-}
-
-__printf(2, 0) static void btf_seq_show(struct btf_show *show, const char *fmt,
- va_list args)
-{
- seq_vprintf((struct seq_file *)show->target, fmt, args);
-}
-
-int btf_type_seq_show_flags(const struct btf *btf, u32 type_id,
- void *obj, struct seq_file *m, u64 flags)
-{
- struct btf_show sseq;
-
- sseq.target = m;
- sseq.showfn = btf_seq_show;
- sseq.flags = flags;
-
- btf_type_show(btf, type_id, obj, &sseq);
-
- return sseq.state.status;
-}
-
-void btf_type_seq_show(const struct btf *btf, u32 type_id, void *obj,
- struct seq_file *m)
-{
- (void) btf_type_seq_show_flags(btf, type_id, obj, m,
- BTF_SHOW_NONAME | BTF_SHOW_COMPACT |
- BTF_SHOW_ZERO | BTF_SHOW_UNSAFE);
-}
-
-struct btf_show_snprintf {
- struct btf_show show;
- int len_left; /* space left in string */
- int len; /* length we would have written */
-};
-
-__printf(2, 0) static void btf_snprintf_show(struct btf_show *show, const char *fmt,
- va_list args)
-{
- struct btf_show_snprintf *ssnprintf = (struct btf_show_snprintf *)show;
- int len;
-
- len = vsnprintf(show->target, ssnprintf->len_left, fmt, args);
-
- if (len < 0) {
- ssnprintf->len_left = 0;
- ssnprintf->len = len;
- } else if (len >= ssnprintf->len_left) {
- /* no space, drive on to get length we would have written */
- ssnprintf->len_left = 0;
- ssnprintf->len += len;
- } else {
- ssnprintf->len_left -= len;
- ssnprintf->len += len;
- show->target += len;
- }
-}
-
-int btf_type_snprintf_show(const struct btf *btf, u32 type_id, void *obj,
- char *buf, int len, u64 flags)
-{
- struct btf_show_snprintf ssnprintf;
-
- ssnprintf.show.target = buf;
- ssnprintf.show.flags = flags;
- ssnprintf.show.showfn = btf_snprintf_show;
- ssnprintf.len_left = len;
- ssnprintf.len = 0;
-
- btf_type_show(btf, type_id, obj, (struct btf_show *)&ssnprintf);
-
- /* If we encountered an error, return it. */
- if (ssnprintf.show.state.status)
- return ssnprintf.show.state.status;
-
- /* Otherwise return length we would have written */
- return ssnprintf.len;
-}
-
#ifdef CONFIG_PROC_FS
static void bpf_btf_show_fdinfo(struct seq_file *m, struct file *filp)
{
@@ -8271,17 +3508,6 @@ u32 btf_obj_id(const struct btf *btf)
{
return READ_ONCE(btf->id);
}
-
-bool btf_is_kernel(const struct btf *btf)
-{
- return btf->kernel_btf;
-}
-
-bool btf_is_module(const struct btf *btf)
-{
- return btf->kernel_btf && strcmp(btf->name, "vmlinux") != 0;
-}
-
enum {
BTF_MODULE_F_LIVE = (1 << 0),
};
@@ -9740,21 +4966,3 @@ int __register_bpf_struct_ops(struct bpf_struct_ops *st_ops)
}
EXPORT_SYMBOL_GPL(__register_bpf_struct_ops);
#endif
-
-bool btf_param_match_suffix(const struct btf *btf,
- const struct btf_param *arg,
- const char *suffix)
-{
- int suffix_len = strlen(suffix), len;
- const char *param_name;
-
- /* In the future, this can be ported to use BTF tagging */
- param_name = btf_name_by_offset(btf, arg->name_off);
- if (str_is_empty(param_name))
- return false;
- len = strlen(param_name);
- if (len <= suffix_len)
- return false;
- param_name += len - suffix_len;
- return !strncmp(param_name, suffix, suffix_len);
-}
diff --git a/kernel/btf/btf.c b/kernel/btf/btf.c
new file mode 100644
index 0000000000000..0eb383926b7d3
--- /dev/null
+++ b/kernel/btf/btf.c
@@ -0,0 +1,4803 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (c) 2018 Facebook */
+
+#include <uapi/linux/btf.h>
+#include <uapi/linux/bpf.h>
+#include <uapi/linux/types.h>
+#include <linux/seq_file.h>
+#include <linux/compiler.h>
+#include <linux/ctype.h>
+#include <linux/errno.h>
+#include <linux/slab.h>
+#include <linux/file.h>
+#include <linux/kernel.h>
+#include <linux/sort.h>
+#include <linux/bpf_verifier.h>
+#include <linux/btf.h>
+#include <linux/btf_ids.h>
+#include <linux/bsearch.h>
+#include <linux/kobject.h>
+#include <linux/string.h>
+#include <linux/overflow.h>
+#include "btf.h"
+
+#ifndef CONFIG_BPF_SYSCALL
+/*
+ * When CONFIG_BPF_SYSCALL is not set, log.c is not compiled, so
+ * bpf_verifier_vlog() is unavailable. Provide a simple implementation
+ * that handles the BPF_LOG_KERNEL case (pr_err), which is the only
+ * log level used by the core BTF verifier for kernel BTF parsing.
+ */
+void bpf_verifier_vlog(struct bpf_verifier_log *log, const char *fmt,
+ va_list args)
+{
+ char buf[256];
+
+ vscnprintf(buf, sizeof(buf), fmt, args);
+ if (log->level == BPF_LOG_KERNEL)
+ pr_err("BPF: %s", buf);
+}
+
+__printf(2, 3) void bpf_log(struct bpf_verifier_log *log,
+ const char *fmt, ...)
+{
+ va_list args;
+
+ if (!bpf_verifier_log_needed(log))
+ return;
+
+ va_start(args, fmt);
+ bpf_verifier_vlog(log, fmt, args);
+ va_end(args);
+}
+#endif /* !CONFIG_BPF_SYSCALL */
+
+/* BTF (BPF Type Format) is the meta data format which describes
+ * the data types of BPF program/map. Hence, it basically focus
+ * on the C programming language which the modern BPF is primary
+ * using.
+ *
+ * ELF Section:
+ * ~~~~~~~~~~~
+ * The BTF data is stored under the ".BTF" ELF section
+ *
+ * struct btf_type:
+ * ~~~~~~~~~~~~~~~
+ * Each 'struct btf_type' object describes a C data type.
+ * Depending on the type it is describing, a 'struct btf_type'
+ * object may be followed by more data. F.e.
+ * To describe an array, 'struct btf_type' is followed by
+ * 'struct btf_array'.
+ *
+ * 'struct btf_type' and any extra data following it are
+ * 4 bytes aligned.
+ *
+ * Type section:
+ * ~~~~~~~~~~~~~
+ * The BTF type section contains a list of 'struct btf_type' objects.
+ * Each one describes a C type. Recall from the above section
+ * that a 'struct btf_type' object could be immediately followed by extra
+ * data in order to describe some particular C types.
+ *
+ * type_id:
+ * ~~~~~~~
+ * Each btf_type object is identified by a type_id. The type_id
+ * is implicitly implied by the location of the btf_type object in
+ * the BTF type section. The first one has type_id 1. The second
+ * one has type_id 2...etc. Hence, an earlier btf_type has
+ * a smaller type_id.
+ *
+ * A btf_type object may refer to another btf_type object by using
+ * type_id (i.e. the "type" in the "struct btf_type").
+ *
+ * NOTE that we cannot assume any reference-order.
+ * A btf_type object can refer to an earlier btf_type object
+ * but it can also refer to a later btf_type object.
+ *
+ * For example, to describe "const void *". A btf_type
+ * object describing "const" may refer to another btf_type
+ * object describing "void *". This type-reference is done
+ * by specifying type_id:
+ *
+ * [1] CONST (anon) type_id=2
+ * [2] PTR (anon) type_id=0
+ *
+ * The above is the btf_verifier debug log:
+ * - Each line started with "[?]" is a btf_type object
+ * - [?] is the type_id of the btf_type object.
+ * - CONST/PTR is the BTF_KIND_XXX
+ * - "(anon)" is the name of the type. It just
+ * happens that CONST and PTR has no name.
+ * - type_id=XXX is the 'u32 type' in btf_type
+ *
+ * NOTE: "void" has type_id 0
+ *
+ * String section:
+ * ~~~~~~~~~~~~~~
+ * The BTF string section contains the names used by the type section.
+ * Each string is referred by an "offset" from the beginning of the
+ * string section.
+ *
+ * Each string is '\0' terminated.
+ *
+ * The first character in the string section must be '\0'
+ * which is used to mean 'anonymous'. Some btf_type may not
+ * have a name.
+ */
+
+/* BTF verification:
+ *
+ * To verify BTF data, two passes are needed.
+ *
+ * Pass #1
+ * ~~~~~~~
+ * The first pass is to collect all btf_type objects to
+ * an array: "btf->types".
+ *
+ * Depending on the C type that a btf_type is describing,
+ * a btf_type may be followed by extra data. We don't know
+ * how many btf_type is there, and more importantly we don't
+ * know where each btf_type is located in the type section.
+ *
+ * Without knowing the location of each type_id, most verifications
+ * cannot be done. e.g. an earlier btf_type may refer to a later
+ * btf_type (recall the "const void *" above), so we cannot
+ * check this type-reference in the first pass.
+ *
+ * In the first pass, it still does some verifications (e.g.
+ * checking the name is a valid offset to the string section).
+ *
+ * Pass #2
+ * ~~~~~~~
+ * The main focus is to resolve a btf_type that is referring
+ * to another type.
+ *
+ * We have to ensure the referring type:
+ * 1) does exist in the BTF (i.e. in btf->types[])
+ * 2) does not cause a loop:
+ * struct A {
+ * struct B b;
+ * };
+ *
+ * struct B {
+ * struct A a;
+ * };
+ *
+ * btf_type_needs_resolve() decides if a btf_type needs
+ * to be resolved.
+ *
+ * The needs_resolve type implements the "resolve()" ops which
+ * essentially does a DFS and detects backedge.
+ *
+ * During resolve (or DFS), different C types have different
+ * "RESOLVED" conditions.
+ *
+ * When resolving a BTF_KIND_STRUCT, we need to resolve all its
+ * members because a member is always referring to another
+ * type. A struct's member can be treated as "RESOLVED" if
+ * it is referring to a BTF_KIND_PTR. Otherwise, the
+ * following valid C struct would be rejected:
+ *
+ * struct A {
+ * int m;
+ * struct A *a;
+ * };
+ *
+ * When resolving a BTF_KIND_PTR, it needs to keep resolving if
+ * it is referring to another BTF_KIND_PTR. Otherwise, we cannot
+ * detect a pointer loop, e.g.:
+ * BTF_KIND_CONST -> BTF_KIND_PTR -> BTF_KIND_CONST -> BTF_KIND_PTR +
+ * ^ |
+ * +-----------------------------------------+
+ *
+ */
+
+#define BITS_PER_U128 (sizeof(u64) * BITS_PER_BYTE * 2)
+
+#define BTF_INFO_MASK 0x9f00ffff
+#define BTF_INT_MASK 0x0fffffff
+#define BTF_TYPE_ID_VALID(type_id) ((type_id) <= BTF_MAX_TYPE)
+#define BTF_STR_OFFSET_VALID(name_off) ((name_off) <= BTF_MAX_NAME_OFFSET)
+
+
+#define for_each_member_from(i, from, struct_type, member) \
+ for (i = from, member = btf_type_member(struct_type) + from; \
+ i < btf_type_vlen(struct_type); \
+ i++, member++)
+
+#define for_each_vsi_from(i, from, struct_type, member) \
+ for (i = from, member = btf_type_var_secinfo(struct_type) + from; \
+ i < btf_type_vlen(struct_type); \
+ i++, member++)
+
+
+
+
+enum visit_state {
+ NOT_VISITED,
+ VISITED,
+ RESOLVED,
+};
+
+struct btf_sec_info {
+ u32 off;
+ u32 len;
+};
+
+const char * const btf_kind_str[NR_BTF_KINDS] = {
+ [BTF_KIND_UNKN] = "UNKNOWN",
+ [BTF_KIND_INT] = "INT",
+ [BTF_KIND_PTR] = "PTR",
+ [BTF_KIND_ARRAY] = "ARRAY",
+ [BTF_KIND_STRUCT] = "STRUCT",
+ [BTF_KIND_UNION] = "UNION",
+ [BTF_KIND_ENUM] = "ENUM",
+ [BTF_KIND_FWD] = "FWD",
+ [BTF_KIND_TYPEDEF] = "TYPEDEF",
+ [BTF_KIND_VOLATILE] = "VOLATILE",
+ [BTF_KIND_CONST] = "CONST",
+ [BTF_KIND_RESTRICT] = "RESTRICT",
+ [BTF_KIND_FUNC] = "FUNC",
+ [BTF_KIND_FUNC_PROTO] = "FUNC_PROTO",
+ [BTF_KIND_VAR] = "VAR",
+ [BTF_KIND_DATASEC] = "DATASEC",
+ [BTF_KIND_FLOAT] = "FLOAT",
+ [BTF_KIND_DECL_TAG] = "DECL_TAG",
+ [BTF_KIND_TYPE_TAG] = "TYPE_TAG",
+ [BTF_KIND_ENUM64] = "ENUM64",
+};
+
+const char *btf_type_str(const struct btf_type *t)
+{
+ return btf_kind_str[BTF_INFO_KIND(t->info)];
+}
+
+/* Chunk size we use in safe copy of data to be shown. */
+#define BTF_SHOW_OBJ_SAFE_SIZE 32
+
+/*
+ * This is the maximum size of a base type value (equivalent to a
+ * 128-bit int); if we are at the end of our safe buffer and have
+ * less than 16 bytes space we can't be assured of being able
+ * to copy the next type safely, so in such cases we will initiate
+ * a new copy.
+ */
+#define BTF_SHOW_OBJ_BASE_TYPE_SIZE 16
+
+/* Type name size */
+#define BTF_SHOW_NAME_SIZE 80
+
+
+/*
+ * Common data to all BTF show operations. Private show functions can add
+ * their own data to a structure containing a struct btf_show and consult it
+ * in the show callback. See btf_type_show() below.
+ *
+ * One challenge with showing nested data is we want to skip 0-valued
+ * data, but in order to figure out whether a nested object is all zeros
+ * we need to walk through it. As a result, we need to make two passes
+ * when handling structs, unions and arrays; the first path simply looks
+ * for nonzero data, while the second actually does the display. The first
+ * pass is signalled by show->state.depth_check being set, and if we
+ * encounter a non-zero value we set show->state.depth_to_show to
+ * the depth at which we encountered it. When we have completed the
+ * first pass, we will know if anything needs to be displayed if
+ * depth_to_show > depth. See btf_[struct,array]_show() for the
+ * implementation of this.
+ *
+ * Another problem is we want to ensure the data for display is safe to
+ * access. To support this, the anonymous "struct {} obj" tracks the data
+ * object and our safe copy of it. We copy portions of the data needed
+ * to the object "copy" buffer, but because its size is limited to
+ * BTF_SHOW_OBJ_COPY_LEN bytes, multiple copies may be required as we
+ * traverse larger objects for display.
+ *
+ * The various data type show functions all start with a call to
+ * btf_show_start_type() which returns a pointer to the safe copy
+ * of the data needed (or if BTF_SHOW_UNSAFE is specified, to the
+ * raw data itself). btf_show_obj_safe() is responsible for
+ * using copy_from_kernel_nofault() to update the safe data if necessary
+ * as we traverse the object's data. skbuff-like semantics are
+ * used:
+ *
+ * - obj.head points to the start of the toplevel object for display
+ * - obj.size is the size of the toplevel object
+ * - obj.data points to the current point in the original data at
+ * which our safe data starts. obj.data will advance as we copy
+ * portions of the data.
+ *
+ * In most cases a single copy will suffice, but larger data structures
+ * such as "struct task_struct" will require many copies. The logic in
+ * btf_show_obj_safe() handles the logic that determines if a new
+ * copy_from_kernel_nofault() is needed.
+ */
+struct btf_show {
+ u64 flags;
+ void *target; /* target of show operation (seq file, buffer) */
+ __printf(2, 0) void (*showfn)(struct btf_show *show, const char *fmt, va_list args);
+ const struct btf *btf;
+ /* below are used during iteration */
+ struct {
+ u8 depth;
+ u8 depth_to_show;
+ u8 depth_check;
+ u8 array_member:1,
+ array_terminated:1;
+ u16 array_encoding;
+ u32 type_id;
+ int status; /* non-zero for error */
+ const struct btf_type *type;
+ const struct btf_member *member;
+ char name[BTF_SHOW_NAME_SIZE]; /* space for member name/type */
+ } state;
+ struct {
+ u32 size;
+ void *head;
+ void *data;
+ u8 safe[BTF_SHOW_OBJ_SAFE_SIZE];
+ } obj;
+};
+
+struct btf_kind_operations {
+ s32 (*check_meta)(struct btf_verifier_env *env,
+ const struct btf_type *t,
+ u32 meta_left);
+ int (*resolve)(struct btf_verifier_env *env,
+ const struct resolve_vertex *v);
+ int (*check_member)(struct btf_verifier_env *env,
+ const struct btf_type *struct_type,
+ const struct btf_member *member,
+ const struct btf_type *member_type);
+ int (*check_kflag_member)(struct btf_verifier_env *env,
+ const struct btf_type *struct_type,
+ const struct btf_member *member,
+ const struct btf_type *member_type);
+ void (*log_details)(struct btf_verifier_env *env,
+ const struct btf_type *t);
+ void (*show)(const struct btf *btf, const struct btf_type *t,
+ u32 type_id, void *data, u8 bits_offsets,
+ struct btf_show *show);
+};
+
+static const struct btf_kind_operations * const kind_ops[NR_BTF_KINDS];
+static struct btf_type btf_void;
+
+static int btf_resolve(struct btf_verifier_env *env,
+ const struct btf_type *t, u32 type_id);
+
+static int btf_func_check(struct btf_verifier_env *env,
+ const struct btf_type *t);
+
+bool btf_type_is_modifier(const struct btf_type *t)
+{
+ /* Some of them is not strictly a C modifier
+ * but they are grouped into the same bucket
+ * for BTF concern:
+ * A type (t) that refers to another
+ * type through t->type AND its size cannot
+ * be determined without following the t->type.
+ *
+ * ptr does not fall into this bucket
+ * because its size is always sizeof(void *).
+ */
+ switch (BTF_INFO_KIND(t->info)) {
+ case BTF_KIND_TYPEDEF:
+ case BTF_KIND_VOLATILE:
+ case BTF_KIND_CONST:
+ case BTF_KIND_RESTRICT:
+ case BTF_KIND_TYPE_TAG:
+ return true;
+ }
+
+ return false;
+}
+
+static int btf_start_id(const struct btf *btf)
+{
+ return btf->start_id + (btf->base_btf ? 0 : 1);
+}
+
+bool btf_type_is_void(const struct btf_type *t)
+{
+ return t == &btf_void;
+}
+
+bool btf_type_is_datasec(const struct btf_type *t)
+{
+ return BTF_INFO_KIND(t->info) == BTF_KIND_DATASEC;
+}
+
+bool btf_type_is_decl_tag(const struct btf_type *t)
+{
+ return BTF_INFO_KIND(t->info) == BTF_KIND_DECL_TAG;
+}
+
+static bool btf_type_nosize(const struct btf_type *t)
+{
+ return btf_type_is_void(t) || btf_type_is_fwd(t) ||
+ btf_type_is_func(t) || btf_type_is_func_proto(t) ||
+ btf_type_is_decl_tag(t);
+}
+
+static bool btf_type_nosize_or_null(const struct btf_type *t)
+{
+ return !t || btf_type_nosize(t);
+}
+
+static bool btf_type_is_decl_tag_target(const struct btf_type *t)
+{
+ return btf_type_is_func(t) || btf_type_is_struct(t) ||
+ btf_type_is_var(t) || btf_type_is_typedef(t);
+}
+
+bool btf_is_vmlinux(const struct btf *btf)
+{
+ return btf->kernel_btf && !btf->base_btf;
+}
+
+u32 btf_nr_types(const struct btf *btf)
+{
+ u32 total = 0;
+
+ while (btf) {
+ total += btf->nr_types;
+ btf = btf->base_btf;
+ }
+
+ return total;
+}
+
+/*
+ * Note that vmlinux and kernel module BTFs are always sorted
+ * during the building phase.
+ */
+void btf_check_sorted(struct btf *btf)
+{
+ u32 i, n, named_start_id = 0;
+
+ n = btf_nr_types(btf);
+ if (btf_is_vmlinux(btf)) {
+ for (i = btf_start_id(btf); i < n; i++) {
+ const struct btf_type *t = btf_type_by_id(btf, i);
+ const char *n = btf_name_by_offset(btf, t->name_off);
+
+ if (n[0] != '\0') {
+ btf->named_start_id = i;
+ return;
+ }
+ }
+ return;
+ }
+
+ for (i = btf_start_id(btf) + 1; i < n; i++) {
+ const struct btf_type *ta = btf_type_by_id(btf, i - 1);
+ const struct btf_type *tb = btf_type_by_id(btf, i);
+ const char *na = btf_name_by_offset(btf, ta->name_off);
+ const char *nb = btf_name_by_offset(btf, tb->name_off);
+
+ if (strcmp(na, nb) > 0)
+ return;
+
+ if (named_start_id == 0 && na[0] != '\0')
+ named_start_id = i - 1;
+ if (named_start_id == 0 && nb[0] != '\0')
+ named_start_id = i;
+ }
+
+ if (named_start_id)
+ btf->named_start_id = named_start_id;
+}
+
+/*
+ * btf_named_start_id - Get the named starting ID for the BTF
+ * @btf: Pointer to the target BTF object
+ * @own: Flag indicating whether to query only the current BTF (true = current BTF only,
+ * false = recursively traverse the base BTF chain)
+ *
+ * Return value rules:
+ * 1. For a sorted btf, return its named_start_id
+ * 2. Else for a split BTF, return its start_id
+ * 3. Else for a base BTF, return 1
+ */
+u32 btf_named_start_id(const struct btf *btf, bool own)
+{
+ const struct btf *base_btf = btf;
+
+ while (!own && base_btf->base_btf)
+ base_btf = base_btf->base_btf;
+
+ return base_btf->named_start_id ?: (base_btf->start_id ?: 1);
+}
+
+static s32 btf_find_by_name_kind_bsearch(const struct btf *btf, const char *name)
+{
+ const struct btf_type *t;
+ const char *tname;
+ s32 l, r, m;
+
+ l = btf_named_start_id(btf, true);
+ r = btf_nr_types(btf) - 1;
+ while (l <= r) {
+ m = l + (r - l) / 2;
+ t = btf_type_by_id(btf, m);
+ tname = btf_name_by_offset(btf, t->name_off);
+ if (strcmp(tname, name) >= 0) {
+ if (l == r)
+ return r;
+ r = m;
+ } else {
+ l = m + 1;
+ }
+ }
+
+ return btf_nr_types(btf);
+}
+
+s32 btf_find_by_name_kind(const struct btf *btf, const char *name, u8 kind)
+{
+ const struct btf *base_btf = btf_base_btf(btf);
+ const struct btf_type *t;
+ const char *tname;
+ s32 id, total;
+
+ if (base_btf) {
+ id = btf_find_by_name_kind(base_btf, name, kind);
+ if (id > 0)
+ return id;
+ }
+
+ total = btf_nr_types(btf);
+ if (btf->named_start_id > 0 && name[0]) {
+ id = btf_find_by_name_kind_bsearch(btf, name);
+ for (; id < total; id++) {
+ t = btf_type_by_id(btf, id);
+ tname = btf_name_by_offset(btf, t->name_off);
+ if (strcmp(tname, name) != 0)
+ return -ENOENT;
+ if (BTF_INFO_KIND(t->info) == kind)
+ return id;
+ }
+ } else {
+ for (id = btf_start_id(btf); id < total; id++) {
+ t = btf_type_by_id(btf, id);
+ if (BTF_INFO_KIND(t->info) != kind)
+ continue;
+ tname = btf_name_by_offset(btf, t->name_off);
+ if (strcmp(tname, name) == 0)
+ return id;
+ }
+ }
+
+ return -ENOENT;
+}
+
+
+const struct btf_type *btf_type_skip_modifiers(const struct btf *btf,
+ u32 id, u32 *res_id)
+{
+ const struct btf_type *t = btf_type_by_id(btf, id);
+
+ while (btf_type_is_modifier(t)) {
+ id = t->type;
+ t = btf_type_by_id(btf, t->type);
+ }
+
+ if (res_id)
+ *res_id = id;
+
+ return t;
+}
+
+const struct btf_type *btf_type_resolve_ptr(const struct btf *btf,
+ u32 id, u32 *res_id)
+{
+ const struct btf_type *t;
+
+ t = btf_type_skip_modifiers(btf, id, NULL);
+ if (!btf_type_is_ptr(t))
+ return NULL;
+
+ return btf_type_skip_modifiers(btf, t->type, res_id);
+}
+
+const struct btf_type *btf_type_resolve_func_ptr(const struct btf *btf,
+ u32 id, u32 *res_id)
+{
+ const struct btf_type *ptype;
+
+ ptype = btf_type_resolve_ptr(btf, id, res_id);
+ if (ptype && btf_type_is_func_proto(ptype))
+ return ptype;
+
+ return NULL;
+}
+
+/* Types that act only as a source, not sink or intermediate
+ * type when resolving.
+ */
+static bool btf_type_is_resolve_source_only(const struct btf_type *t)
+{
+ return btf_type_is_var(t) ||
+ btf_type_is_decl_tag(t) ||
+ btf_type_is_datasec(t);
+}
+
+/* What types need to be resolved?
+ *
+ * btf_type_is_modifier() is an obvious one.
+ *
+ * btf_type_is_struct() because its member refers to
+ * another type (through member->type).
+ *
+ * btf_type_is_var() because the variable refers to
+ * another type. btf_type_is_datasec() holds multiple
+ * btf_type_is_var() types that need resolving.
+ *
+ * btf_type_is_array() because its element (array->type)
+ * refers to another type. Array can be thought of a
+ * special case of struct while array just has the same
+ * member-type repeated by array->nelems of times.
+ */
+static bool btf_type_needs_resolve(const struct btf_type *t)
+{
+ return btf_type_is_modifier(t) ||
+ btf_type_is_ptr(t) ||
+ btf_type_is_struct(t) ||
+ btf_type_is_array(t) ||
+ btf_type_is_var(t) ||
+ btf_type_is_func(t) ||
+ btf_type_is_decl_tag(t) ||
+ btf_type_is_datasec(t);
+}
+
+/* t->size can be used */
+bool btf_type_has_size(const struct btf_type *t)
+{
+ switch (BTF_INFO_KIND(t->info)) {
+ case BTF_KIND_INT:
+ case BTF_KIND_STRUCT:
+ case BTF_KIND_UNION:
+ case BTF_KIND_ENUM:
+ case BTF_KIND_DATASEC:
+ case BTF_KIND_FLOAT:
+ case BTF_KIND_ENUM64:
+ return true;
+ }
+
+ return false;
+}
+
+static const char *btf_int_encoding_str(u8 encoding)
+{
+ if (encoding == 0)
+ return "(none)";
+ else if (encoding == BTF_INT_SIGNED)
+ return "SIGNED";
+ else if (encoding == BTF_INT_CHAR)
+ return "CHAR";
+ else if (encoding == BTF_INT_BOOL)
+ return "BOOL";
+ else
+ return "UNKN";
+}
+
+static u32 btf_type_int(const struct btf_type *t)
+{
+ return *(u32 *)(t + 1);
+}
+
+static const struct btf_array *btf_type_array(const struct btf_type *t)
+{
+ return (const struct btf_array *)(t + 1);
+}
+
+static const struct btf_enum *btf_type_enum(const struct btf_type *t)
+{
+ return (const struct btf_enum *)(t + 1);
+}
+
+static const struct btf_var *btf_type_var(const struct btf_type *t)
+{
+ return (const struct btf_var *)(t + 1);
+}
+
+const struct btf_decl_tag *btf_type_decl_tag(const struct btf_type *t)
+{
+ return (const struct btf_decl_tag *)(t + 1);
+}
+
+static const struct btf_enum64 *btf_type_enum64(const struct btf_type *t)
+{
+ return (const struct btf_enum64 *)(t + 1);
+}
+
+static const struct btf_kind_operations *btf_type_ops(const struct btf_type *t)
+{
+ return kind_ops[BTF_INFO_KIND(t->info)];
+}
+
+static bool btf_name_offset_valid(const struct btf *btf, u32 offset)
+{
+ if (!BTF_STR_OFFSET_VALID(offset))
+ return false;
+
+ while (offset < btf->start_str_off)
+ btf = btf->base_btf;
+
+ offset -= btf->start_str_off;
+ return offset < btf->hdr.str_len;
+}
+
+static bool __btf_name_char_ok(char c, bool first)
+{
+ if ((first ? !isalpha(c) :
+ !isalnum(c)) &&
+ c != '_' &&
+ c != '.')
+ return false;
+ return true;
+}
+
+const char *btf_str_by_offset(const struct btf *btf, u32 offset)
+{
+ while (offset < btf->start_str_off)
+ btf = btf->base_btf;
+
+ offset -= btf->start_str_off;
+ if (offset < btf->hdr.str_len)
+ return &btf->strings[offset];
+
+ return NULL;
+}
+
+static bool btf_name_valid_identifier(const struct btf *btf, u32 offset)
+{
+ /* offset must be valid */
+ const char *src = btf_str_by_offset(btf, offset);
+ const char *src_limit;
+
+ if (!__btf_name_char_ok(*src, true))
+ return false;
+
+ /* set a limit on identifier length */
+ src_limit = src + KSYM_NAME_LEN;
+ src++;
+ while (*src && src < src_limit) {
+ if (!__btf_name_char_ok(*src, false))
+ return false;
+ src++;
+ }
+
+ return !*src;
+}
+
+/* Allow any printable character in DATASEC names */
+static bool btf_name_valid_section(const struct btf *btf, u32 offset)
+{
+ /* offset must be valid */
+ const char *src = btf_str_by_offset(btf, offset);
+ const char *src_limit;
+
+ if (!*src)
+ return false;
+
+ /* set a limit on identifier length */
+ src_limit = src + KSYM_NAME_LEN;
+ while (*src && src < src_limit) {
+ if (!isprint(*src))
+ return false;
+ src++;
+ }
+
+ return !*src;
+}
+
+const char *__btf_name_by_offset(const struct btf *btf, u32 offset)
+{
+ const char *name;
+
+ if (!offset)
+ return "(anon)";
+
+ name = btf_str_by_offset(btf, offset);
+ return name ?: "(invalid-name-offset)";
+}
+
+const char *btf_name_by_offset(const struct btf *btf, u32 offset)
+{
+ return btf_str_by_offset(btf, offset);
+}
+
+const struct btf_type *btf_type_by_id(const struct btf *btf, u32 type_id)
+{
+ while (type_id < btf->start_id)
+ btf = btf->base_btf;
+
+ type_id -= btf->start_id;
+ if (type_id >= btf->nr_types)
+ return NULL;
+ return btf->types[type_id];
+}
+EXPORT_SYMBOL_GPL(btf_type_by_id);
+
+/*
+ * Check that the type @t is a regular int. This means that @t is not
+ * a bit field and it has the same size as either of u8/u16/u32/u64
+ * or __int128. If @expected_size is not zero, then size of @t should
+ * be the same. A caller should already have checked that the type @t
+ * is an integer.
+ */
+static bool __btf_type_int_is_regular(const struct btf_type *t, size_t expected_size)
+{
+ u32 int_data = btf_type_int(t);
+ u8 nr_bits = BTF_INT_BITS(int_data);
+ u8 nr_bytes = BITS_ROUNDUP_BYTES(nr_bits);
+
+ return BITS_PER_BYTE_MASKED(nr_bits) == 0 &&
+ BTF_INT_OFFSET(int_data) == 0 &&
+ (nr_bytes <= 16 && is_power_of_2(nr_bytes)) &&
+ (expected_size == 0 || nr_bytes == expected_size);
+}
+
+static bool btf_type_int_is_regular(const struct btf_type *t)
+{
+ return __btf_type_int_is_regular(t, 0);
+}
+
+bool btf_type_is_i32(const struct btf_type *t)
+{
+ return btf_type_is_int(t) && __btf_type_int_is_regular(t, 4);
+}
+
+bool btf_type_is_i64(const struct btf_type *t)
+{
+ return btf_type_is_int(t) && __btf_type_int_is_regular(t, 8);
+}
+
+bool btf_type_is_primitive(const struct btf_type *t)
+{
+ return (btf_type_is_int(t) && btf_type_int_is_regular(t)) ||
+ btf_is_any_enum(t);
+}
+
+/*
+ * Check that given struct member is a regular int with expected
+ * offset and size.
+ */
+bool btf_member_is_reg_int(const struct btf *btf, const struct btf_type *s,
+ const struct btf_member *m,
+ u32 expected_offset, u32 expected_size)
+{
+ const struct btf_type *t;
+ u32 id, int_data;
+ u8 nr_bits;
+
+ id = m->type;
+ t = btf_type_id_size(btf, &id, NULL);
+ if (!t || !btf_type_is_int(t))
+ return false;
+
+ int_data = btf_type_int(t);
+ nr_bits = BTF_INT_BITS(int_data);
+ if (btf_type_kflag(s)) {
+ u32 bitfield_size = BTF_MEMBER_BITFIELD_SIZE(m->offset);
+ u32 bit_offset = BTF_MEMBER_BIT_OFFSET(m->offset);
+
+ /* if kflag set, int should be a regular int and
+ * bit offset should be at byte boundary.
+ */
+ return !bitfield_size &&
+ BITS_ROUNDUP_BYTES(bit_offset) == expected_offset &&
+ BITS_ROUNDUP_BYTES(nr_bits) == expected_size;
+ }
+
+ if (BTF_INT_OFFSET(int_data) ||
+ BITS_PER_BYTE_MASKED(m->offset) ||
+ BITS_ROUNDUP_BYTES(m->offset) != expected_offset ||
+ BITS_PER_BYTE_MASKED(nr_bits) ||
+ BITS_ROUNDUP_BYTES(nr_bits) != expected_size)
+ return false;
+
+ return true;
+}
+
+/* Similar to btf_type_skip_modifiers() but does not skip typedefs. */
+static const struct btf_type *btf_type_skip_qualifiers(const struct btf *btf,
+ u32 id)
+{
+ const struct btf_type *t = btf_type_by_id(btf, id);
+
+ while (btf_type_is_modifier(t) &&
+ BTF_INFO_KIND(t->info) != BTF_KIND_TYPEDEF) {
+ t = btf_type_by_id(btf, t->type);
+ }
+
+ return t;
+}
+
+#define BTF_SHOW_MAX_ITER 10
+
+#define BTF_KIND_BIT(kind) (1ULL << kind)
+
+/*
+ * Populate show->state.name with type name information.
+ * Format of type name is
+ *
+ * [.member_name = ] (type_name)
+ */
+static const char *btf_show_name(struct btf_show *show)
+{
+ /* BTF_MAX_ITER array suffixes "[]" */
+ const char *array_suffixes = "[][][][][][][][][][]";
+ const char *array_suffix = &array_suffixes[strlen(array_suffixes)];
+ /* BTF_MAX_ITER pointer suffixes "*" */
+ const char *ptr_suffixes = "**********";
+ const char *ptr_suffix = &ptr_suffixes[strlen(ptr_suffixes)];
+ const char *name = NULL, *prefix = "", *parens = "";
+ const struct btf_member *m = show->state.member;
+ const struct btf_type *t;
+ const struct btf_array *array;
+ u32 id = show->state.type_id;
+ const char *member = NULL;
+ bool show_member = false;
+ u64 kinds = 0;
+ int i;
+
+ show->state.name[0] = '\0';
+
+ /*
+ * Don't show type name if we're showing an array member;
+ * in that case we show the array type so don't need to repeat
+ * ourselves for each member.
+ */
+ if (show->state.array_member)
+ return "";
+
+ /* Retrieve member name, if any. */
+ if (m) {
+ member = btf_name_by_offset(show->btf, m->name_off);
+ show_member = strlen(member) > 0;
+ id = m->type;
+ }
+
+ /*
+ * Start with type_id, as we have resolved the struct btf_type *
+ * via btf_modifier_show() past the parent typedef to the child
+ * struct, int etc it is defined as. In such cases, the type_id
+ * still represents the starting type while the struct btf_type *
+ * in our show->state points at the resolved type of the typedef.
+ */
+ t = btf_type_by_id(show->btf, id);
+ if (!t)
+ return "";
+
+ /*
+ * The goal here is to build up the right number of pointer and
+ * array suffixes while ensuring the type name for a typedef
+ * is represented. Along the way we accumulate a list of
+ * BTF kinds we have encountered, since these will inform later
+ * display; for example, pointer types will not require an
+ * opening "{" for struct, we will just display the pointer value.
+ *
+ * We also want to accumulate the right number of pointer or array
+ * indices in the format string while iterating until we get to
+ * the typedef/pointee/array member target type.
+ *
+ * We start by pointing at the end of pointer and array suffix
+ * strings; as we accumulate pointers and arrays we move the pointer
+ * or array string backwards so it will show the expected number of
+ * '*' or '[]' for the type. BTF_SHOW_MAX_ITER of nesting of pointers
+ * and/or arrays and typedefs are supported as a precaution.
+ *
+ * We also want to get typedef name while proceeding to resolve
+ * type it points to so that we can add parentheses if it is a
+ * "typedef struct" etc.
+ */
+ for (i = 0; i < BTF_SHOW_MAX_ITER; i++) {
+
+ switch (BTF_INFO_KIND(t->info)) {
+ case BTF_KIND_TYPEDEF:
+ if (!name)
+ name = btf_name_by_offset(show->btf,
+ t->name_off);
+ kinds |= BTF_KIND_BIT(BTF_KIND_TYPEDEF);
+ id = t->type;
+ break;
+ case BTF_KIND_ARRAY:
+ kinds |= BTF_KIND_BIT(BTF_KIND_ARRAY);
+ parens = "[";
+ if (!t)
+ return "";
+ array = btf_type_array(t);
+ if (array_suffix > array_suffixes)
+ array_suffix -= 2;
+ id = array->type;
+ break;
+ case BTF_KIND_PTR:
+ kinds |= BTF_KIND_BIT(BTF_KIND_PTR);
+ if (ptr_suffix > ptr_suffixes)
+ ptr_suffix -= 1;
+ id = t->type;
+ break;
+ default:
+ id = 0;
+ break;
+ }
+ if (!id)
+ break;
+ t = btf_type_skip_qualifiers(show->btf, id);
+ }
+ /* We may not be able to represent this type; bail to be safe */
+ if (i == BTF_SHOW_MAX_ITER)
+ return "";
+
+ if (!name)
+ name = btf_name_by_offset(show->btf, t->name_off);
+
+ switch (BTF_INFO_KIND(t->info)) {
+ case BTF_KIND_STRUCT:
+ case BTF_KIND_UNION:
+ prefix = BTF_INFO_KIND(t->info) == BTF_KIND_STRUCT ?
+ "struct" : "union";
+ /* if it's an array of struct/union, parens is already set */
+ if (!(kinds & (BTF_KIND_BIT(BTF_KIND_ARRAY))))
+ parens = "{";
+ break;
+ case BTF_KIND_ENUM:
+ case BTF_KIND_ENUM64:
+ prefix = "enum";
+ break;
+ default:
+ break;
+ }
+
+ /* pointer does not require parens */
+ if (kinds & BTF_KIND_BIT(BTF_KIND_PTR))
+ parens = "";
+ /* typedef does not require struct/union/enum prefix */
+ if (kinds & BTF_KIND_BIT(BTF_KIND_TYPEDEF))
+ prefix = "";
+
+ if (!name)
+ name = "";
+
+ /* Even if we don't want type name info, we want parentheses etc */
+ if (show->flags & BTF_SHOW_NONAME)
+ snprintf(show->state.name, sizeof(show->state.name), "%s",
+ parens);
+ else
+ snprintf(show->state.name, sizeof(show->state.name),
+ "%s%s%s(%s%s%s%s%s%s)%s",
+ /* first 3 strings comprise ".member = " */
+ show_member ? "." : "",
+ show_member ? member : "",
+ show_member ? " = " : "",
+ /* ...next is our prefix (struct, enum, etc) */
+ prefix,
+ strlen(prefix) > 0 && strlen(name) > 0 ? " " : "",
+ /* ...this is the type name itself */
+ name,
+ /* ...suffixed by the appropriate '*', '[]' suffixes */
+ strlen(ptr_suffix) > 0 ? " " : "", ptr_suffix,
+ array_suffix, parens);
+
+ return show->state.name;
+}
+
+static const char *__btf_show_indent(struct btf_show *show)
+{
+ const char *indents = " ";
+ const char *indent = &indents[strlen(indents)];
+
+ if ((indent - show->state.depth) >= indents)
+ return indent - show->state.depth;
+ return indents;
+}
+
+static const char *btf_show_indent(struct btf_show *show)
+{
+ return show->flags & BTF_SHOW_COMPACT ? "" : __btf_show_indent(show);
+}
+
+static const char *btf_show_newline(struct btf_show *show)
+{
+ return show->flags & BTF_SHOW_COMPACT ? "" : "\n";
+}
+
+static const char *btf_show_delim(struct btf_show *show)
+{
+ if (show->state.depth == 0)
+ return "";
+
+ if ((show->flags & BTF_SHOW_COMPACT) && show->state.type &&
+ BTF_INFO_KIND(show->state.type->info) == BTF_KIND_UNION)
+ return "|";
+
+ return ",";
+}
+
+__printf(2, 3) static void btf_show(struct btf_show *show, const char *fmt, ...)
+{
+ va_list args;
+
+ if (!show->state.depth_check) {
+ va_start(args, fmt);
+ show->showfn(show, fmt, args);
+ va_end(args);
+ }
+}
+
+/* Macros are used here as btf_show_type_value[s]() prepends and appends
+ * format specifiers to the format specifier passed in; these do the work of
+ * adding indentation, delimiters etc while the caller simply has to specify
+ * the type value(s) in the format specifier + value(s).
+ */
+#define btf_show_type_value(show, fmt, value) \
+ do { \
+ if ((value) != (__typeof__(value))0 || \
+ (show->flags & BTF_SHOW_ZERO) || \
+ show->state.depth == 0) { \
+ btf_show(show, "%s%s" fmt "%s%s", \
+ btf_show_indent(show), \
+ btf_show_name(show), \
+ value, btf_show_delim(show), \
+ btf_show_newline(show)); \
+ if (show->state.depth > show->state.depth_to_show) \
+ show->state.depth_to_show = show->state.depth; \
+ } \
+ } while (0)
+
+#define btf_show_type_values(show, fmt, ...) \
+ do { \
+ btf_show(show, "%s%s" fmt "%s%s", btf_show_indent(show), \
+ btf_show_name(show), \
+ __VA_ARGS__, btf_show_delim(show), \
+ btf_show_newline(show)); \
+ if (show->state.depth > show->state.depth_to_show) \
+ show->state.depth_to_show = show->state.depth; \
+ } while (0)
+
+/* How much is left to copy to safe buffer after @data? */
+static int btf_show_obj_size_left(struct btf_show *show, void *data)
+{
+ return show->obj.head + show->obj.size - data;
+}
+
+/* Is object pointed to by @data of @size already copied to our safe buffer? */
+static bool btf_show_obj_is_safe(struct btf_show *show, void *data, int size)
+{
+ return data >= show->obj.data &&
+ (data + size) < (show->obj.data + BTF_SHOW_OBJ_SAFE_SIZE);
+}
+
+/*
+ * If object pointed to by @data of @size falls within our safe buffer, return
+ * the equivalent pointer to the same safe data. Assumes
+ * copy_from_kernel_nofault() has already happened and our safe buffer is
+ * populated.
+ */
+static void *__btf_show_obj_safe(struct btf_show *show, void *data, int size)
+{
+ if (btf_show_obj_is_safe(show, data, size))
+ return show->obj.safe + (data - show->obj.data);
+ return NULL;
+}
+
+/*
+ * Return a safe-to-access version of data pointed to by @data.
+ * We do this by copying the relevant amount of information
+ * to the struct btf_show obj.safe buffer using copy_from_kernel_nofault().
+ *
+ * If BTF_SHOW_UNSAFE is specified, just return data as-is; no
+ * safe copy is needed.
+ *
+ * Otherwise we need to determine if we have the required amount
+ * of data (determined by the @data pointer and the size of the
+ * largest base type we can encounter (represented by
+ * BTF_SHOW_OBJ_BASE_TYPE_SIZE). Having that much data ensures
+ * that we will be able to print some of the current object,
+ * and if more is needed a copy will be triggered.
+ * Some objects such as structs will not fit into the buffer;
+ * in such cases additional copies when we iterate over their
+ * members may be needed.
+ *
+ * btf_show_obj_safe() is used to return a safe buffer for
+ * btf_show_start_type(); this ensures that as we recurse into
+ * nested types we always have safe data for the given type.
+ * This approach is somewhat wasteful; it's possible for example
+ * that when iterating over a large union we'll end up copying the
+ * same data repeatedly, but the goal is safety not performance.
+ * We use stack data as opposed to per-CPU buffers because the
+ * iteration over a type can take some time, and preemption handling
+ * would greatly complicate use of the safe buffer.
+ */
+static void *btf_show_obj_safe(struct btf_show *show,
+ const struct btf_type *t,
+ void *data)
+{
+ const struct btf_type *rt;
+ int size_left, size;
+ void *safe = NULL;
+
+ if (show->flags & BTF_SHOW_UNSAFE)
+ return data;
+
+ rt = btf_resolve_size(show->btf, t, &size);
+ if (IS_ERR(rt)) {
+ show->state.status = PTR_ERR(rt);
+ return NULL;
+ }
+
+ /*
+ * Is this toplevel object? If so, set total object size and
+ * initialize pointers. Otherwise check if we still fall within
+ * our safe object data.
+ */
+ if (show->state.depth == 0) {
+ show->obj.size = size;
+ show->obj.head = data;
+ } else {
+ /*
+ * If the size of the current object is > our remaining
+ * safe buffer we _may_ need to do a new copy. However
+ * consider the case of a nested struct; it's size pushes
+ * us over the safe buffer limit, but showing any individual
+ * struct members does not. In such cases, we don't need
+ * to initiate a fresh copy yet; however we definitely need
+ * at least BTF_SHOW_OBJ_BASE_TYPE_SIZE bytes left
+ * in our buffer, regardless of the current object size.
+ * The logic here is that as we resolve types we will
+ * hit a base type at some point, and we need to be sure
+ * the next chunk of data is safely available to display
+ * that type info safely. We cannot rely on the size of
+ * the current object here because it may be much larger
+ * than our current buffer (e.g. task_struct is 8k).
+ * All we want to do here is ensure that we can print the
+ * next basic type, which we can if either
+ * - the current type size is within the safe buffer; or
+ * - at least BTF_SHOW_OBJ_BASE_TYPE_SIZE bytes are left in
+ * the safe buffer.
+ */
+ safe = __btf_show_obj_safe(show, data,
+ min(size,
+ BTF_SHOW_OBJ_BASE_TYPE_SIZE));
+ }
+
+ /*
+ * We need a new copy to our safe object, either because we haven't
+ * yet copied and are initializing safe data, or because the data
+ * we want falls outside the boundaries of the safe object.
+ */
+ if (!safe) {
+ size_left = btf_show_obj_size_left(show, data);
+ if (size_left > BTF_SHOW_OBJ_SAFE_SIZE)
+ size_left = BTF_SHOW_OBJ_SAFE_SIZE;
+ show->state.status = copy_from_kernel_nofault(show->obj.safe,
+ data, size_left);
+ if (!show->state.status) {
+ show->obj.data = data;
+ safe = show->obj.safe;
+ }
+ }
+
+ return safe;
+}
+
+/*
+ * Set the type we are starting to show and return a safe data pointer
+ * to be used for showing the associated data.
+ */
+static void *btf_show_start_type(struct btf_show *show,
+ const struct btf_type *t,
+ u32 type_id, void *data)
+{
+ show->state.type = t;
+ show->state.type_id = type_id;
+ show->state.name[0] = '\0';
+
+ return btf_show_obj_safe(show, t, data);
+}
+
+static void btf_show_end_type(struct btf_show *show)
+{
+ show->state.type = NULL;
+ show->state.type_id = 0;
+ show->state.name[0] = '\0';
+}
+
+static void *btf_show_start_aggr_type(struct btf_show *show,
+ const struct btf_type *t,
+ u32 type_id, void *data)
+{
+ void *safe_data = btf_show_start_type(show, t, type_id, data);
+
+ if (!safe_data)
+ return safe_data;
+
+ btf_show(show, "%s%s%s", btf_show_indent(show),
+ btf_show_name(show),
+ btf_show_newline(show));
+ show->state.depth++;
+ return safe_data;
+}
+
+static void btf_show_end_aggr_type(struct btf_show *show,
+ const char *suffix)
+{
+ show->state.depth--;
+ btf_show(show, "%s%s%s%s", btf_show_indent(show), suffix,
+ btf_show_delim(show), btf_show_newline(show));
+ btf_show_end_type(show);
+}
+
+static void btf_show_start_member(struct btf_show *show,
+ const struct btf_member *m)
+{
+ show->state.member = m;
+}
+
+static void btf_show_start_array_member(struct btf_show *show)
+{
+ show->state.array_member = 1;
+ btf_show_start_member(show, NULL);
+}
+
+static void btf_show_end_member(struct btf_show *show)
+{
+ show->state.member = NULL;
+}
+
+static void btf_show_end_array_member(struct btf_show *show)
+{
+ show->state.array_member = 0;
+ btf_show_end_member(show);
+}
+
+static void *btf_show_start_array_type(struct btf_show *show,
+ const struct btf_type *t,
+ u32 type_id,
+ u16 array_encoding,
+ void *data)
+{
+ show->state.array_encoding = array_encoding;
+ show->state.array_terminated = 0;
+ return btf_show_start_aggr_type(show, t, type_id, data);
+}
+
+static void btf_show_end_array_type(struct btf_show *show)
+{
+ show->state.array_encoding = 0;
+ show->state.array_terminated = 0;
+ btf_show_end_aggr_type(show, "]");
+}
+
+static void *btf_show_start_struct_type(struct btf_show *show,
+ const struct btf_type *t,
+ u32 type_id,
+ void *data)
+{
+ return btf_show_start_aggr_type(show, t, type_id, data);
+}
+
+static void btf_show_end_struct_type(struct btf_show *show)
+{
+ btf_show_end_aggr_type(show, "}");
+}
+
+__printf(2, 3) static void __btf_verifier_log(struct bpf_verifier_log *log,
+ const char *fmt, ...)
+{
+ va_list args;
+
+ va_start(args, fmt);
+ bpf_verifier_vlog(log, fmt, args);
+ va_end(args);
+}
+
+__printf(2, 3) static void btf_verifier_log(struct btf_verifier_env *env,
+ const char *fmt, ...)
+{
+ struct bpf_verifier_log *log = &env->log;
+ va_list args;
+
+ if (!bpf_verifier_log_needed(log))
+ return;
+
+ va_start(args, fmt);
+ bpf_verifier_vlog(log, fmt, args);
+ va_end(args);
+}
+
+__printf(4, 5) static void __btf_verifier_log_type(struct btf_verifier_env *env,
+ const struct btf_type *t,
+ bool log_details,
+ const char *fmt, ...)
+{
+ struct bpf_verifier_log *log = &env->log;
+ struct btf *btf = env->btf;
+ va_list args;
+
+ if (!bpf_verifier_log_needed(log))
+ return;
+
+ if (log->level == BPF_LOG_KERNEL) {
+ /* btf verifier prints all types it is processing via
+ * btf_verifier_log_type(..., fmt = NULL).
+ * Skip those prints for in-kernel BTF verification.
+ */
+ if (!fmt)
+ return;
+
+ /* Skip logging when loading module BTF with mismatches permitted */
+ if (env->btf->base_btf && IS_ENABLED(CONFIG_MODULE_ALLOW_BTF_MISMATCH))
+ return;
+ }
+
+ __btf_verifier_log(log, "[%u] %s %s%s",
+ env->log_type_id,
+ btf_type_str(t),
+ __btf_name_by_offset(btf, t->name_off),
+ log_details ? " " : "");
+
+ if (log_details)
+ btf_type_ops(t)->log_details(env, t);
+
+ if (fmt && *fmt) {
+ __btf_verifier_log(log, " ");
+ va_start(args, fmt);
+ bpf_verifier_vlog(log, fmt, args);
+ va_end(args);
+ }
+
+ __btf_verifier_log(log, "\n");
+}
+
+#define btf_verifier_log_type(env, t, ...) \
+ __btf_verifier_log_type((env), (t), true, __VA_ARGS__)
+#define btf_verifier_log_basic(env, t, ...) \
+ __btf_verifier_log_type((env), (t), false, __VA_ARGS__)
+
+__printf(4, 5)
+static void btf_verifier_log_member(struct btf_verifier_env *env,
+ const struct btf_type *struct_type,
+ const struct btf_member *member,
+ const char *fmt, ...)
+{
+ struct bpf_verifier_log *log = &env->log;
+ struct btf *btf = env->btf;
+ va_list args;
+
+ if (!bpf_verifier_log_needed(log))
+ return;
+
+ if (log->level == BPF_LOG_KERNEL) {
+ if (!fmt)
+ return;
+
+ /* Skip logging when loading module BTF with mismatches permitted */
+ if (env->btf->base_btf && IS_ENABLED(CONFIG_MODULE_ALLOW_BTF_MISMATCH))
+ return;
+ }
+
+ /* The CHECK_META phase already did a btf dump.
+ *
+ * If member is logged again, it must hit an error in
+ * parsing this member. It is useful to print out which
+ * struct this member belongs to.
+ */
+ if (env->phase != CHECK_META)
+ btf_verifier_log_type(env, struct_type, NULL);
+
+ if (btf_type_kflag(struct_type))
+ __btf_verifier_log(log,
+ "\t%s type_id=%u bitfield_size=%u bits_offset=%u",
+ __btf_name_by_offset(btf, member->name_off),
+ member->type,
+ BTF_MEMBER_BITFIELD_SIZE(member->offset),
+ BTF_MEMBER_BIT_OFFSET(member->offset));
+ else
+ __btf_verifier_log(log, "\t%s type_id=%u bits_offset=%u",
+ __btf_name_by_offset(btf, member->name_off),
+ member->type, member->offset);
+
+ if (fmt && *fmt) {
+ __btf_verifier_log(log, " ");
+ va_start(args, fmt);
+ bpf_verifier_vlog(log, fmt, args);
+ va_end(args);
+ }
+
+ __btf_verifier_log(log, "\n");
+}
+
+__printf(4, 5)
+static void btf_verifier_log_vsi(struct btf_verifier_env *env,
+ const struct btf_type *datasec_type,
+ const struct btf_var_secinfo *vsi,
+ const char *fmt, ...)
+{
+ struct bpf_verifier_log *log = &env->log;
+ va_list args;
+
+ if (!bpf_verifier_log_needed(log))
+ return;
+ if (log->level == BPF_LOG_KERNEL && !fmt)
+ return;
+ if (env->phase != CHECK_META)
+ btf_verifier_log_type(env, datasec_type, NULL);
+
+ __btf_verifier_log(log, "\t type_id=%u offset=%u size=%u",
+ vsi->type, vsi->offset, vsi->size);
+ if (fmt && *fmt) {
+ __btf_verifier_log(log, " ");
+ va_start(args, fmt);
+ bpf_verifier_vlog(log, fmt, args);
+ va_end(args);
+ }
+
+ __btf_verifier_log(log, "\n");
+}
+
+static void btf_verifier_log_hdr(struct btf_verifier_env *env,
+ u32 btf_data_size)
+{
+ struct bpf_verifier_log *log = &env->log;
+ const struct btf *btf = env->btf;
+ const struct btf_header *hdr;
+
+ if (!bpf_verifier_log_needed(log))
+ return;
+
+ if (log->level == BPF_LOG_KERNEL)
+ return;
+ hdr = &btf->hdr;
+ __btf_verifier_log(log, "magic: 0x%x\n", hdr->magic);
+ __btf_verifier_log(log, "version: %u\n", hdr->version);
+ __btf_verifier_log(log, "flags: 0x%x\n", hdr->flags);
+ __btf_verifier_log(log, "hdr_len: %u\n", hdr->hdr_len);
+ __btf_verifier_log(log, "type_off: %u\n", hdr->type_off);
+ __btf_verifier_log(log, "type_len: %u\n", hdr->type_len);
+ __btf_verifier_log(log, "str_off: %u\n", hdr->str_off);
+ __btf_verifier_log(log, "str_len: %u\n", hdr->str_len);
+ __btf_verifier_log(log, "btf_total_size: %u\n", btf_data_size);
+}
+
+static int btf_add_type(struct btf_verifier_env *env, struct btf_type *t)
+{
+ struct btf *btf = env->btf;
+
+ if (btf->types_size == btf->nr_types) {
+ /* Expand 'types' array */
+
+ struct btf_type **new_types;
+ u32 expand_by, new_size;
+
+ if (btf->start_id + btf->types_size == BTF_MAX_TYPE) {
+ btf_verifier_log(env, "Exceeded max num of types");
+ return -E2BIG;
+ }
+
+ expand_by = max_t(u32, btf->types_size >> 2, 16);
+ new_size = min_t(u32, BTF_MAX_TYPE,
+ btf->types_size + expand_by);
+
+ new_types = kvzalloc_objs(*new_types, new_size,
+ GFP_KERNEL | __GFP_NOWARN);
+ if (!new_types)
+ return -ENOMEM;
+
+ if (btf->nr_types == 0) {
+ if (!btf->base_btf) {
+ /* lazily init VOID type */
+ new_types[0] = &btf_void;
+ btf->nr_types++;
+ }
+ } else {
+ memcpy(new_types, btf->types,
+ sizeof(*btf->types) * btf->nr_types);
+ }
+
+ kvfree(btf->types);
+ btf->types = new_types;
+ btf->types_size = new_size;
+ }
+
+ btf->types[btf->nr_types++] = t;
+
+ return 0;
+}
+
+
+void __weak btf_free_bpf_data(struct btf *btf)
+{
+}
+
+void btf_free(struct btf *btf)
+{
+ btf_free_bpf_data(btf);
+ kvfree(btf->types);
+ kvfree(btf->resolved_sizes);
+ kvfree(btf->resolved_ids);
+ /* vmlinux does not allocate btf->data, it simply points it at
+ * __start_BTF.
+ */
+ if (!btf_is_vmlinux(btf))
+ kvfree(btf->data);
+ kvfree(btf->base_id_map);
+ kfree(btf);
+}
+
+
+const char *btf_get_name(const struct btf *btf)
+{
+ return btf->name;
+}
+
+void btf_get(struct btf *btf)
+{
+ refcount_inc(&btf->refcnt);
+}
+
+void __weak btf_put_bpf(struct btf *btf)
+{
+ btf_free(btf);
+}
+
+void btf_put(struct btf *btf)
+{
+ if (btf && refcount_dec_and_test(&btf->refcnt)) {
+ btf_put_bpf(btf);
+ }
+}
+
+struct btf *btf_base_btf(const struct btf *btf)
+{
+ return btf->base_btf;
+}
+
+const struct btf_header *btf_header(const struct btf *btf)
+{
+ return &btf->hdr;
+}
+
+void btf_set_base_btf(struct btf *btf, const struct btf *base_btf)
+{
+ btf->base_btf = (struct btf *)base_btf;
+ btf->start_id = btf_nr_types(base_btf);
+ btf->start_str_off = base_btf->hdr.str_len;
+}
+
+static int env_resolve_init(struct btf_verifier_env *env)
+{
+ struct btf *btf = env->btf;
+ u32 nr_types = btf->nr_types;
+ u32 *resolved_sizes = NULL;
+ u32 *resolved_ids = NULL;
+ u8 *visit_states = NULL;
+
+ resolved_sizes = kvcalloc(nr_types, sizeof(*resolved_sizes),
+ GFP_KERNEL | __GFP_NOWARN);
+ if (!resolved_sizes)
+ goto nomem;
+
+ resolved_ids = kvcalloc(nr_types, sizeof(*resolved_ids),
+ GFP_KERNEL | __GFP_NOWARN);
+ if (!resolved_ids)
+ goto nomem;
+
+ visit_states = kvcalloc(nr_types, sizeof(*visit_states),
+ GFP_KERNEL | __GFP_NOWARN);
+ if (!visit_states)
+ goto nomem;
+
+ btf->resolved_sizes = resolved_sizes;
+ btf->resolved_ids = resolved_ids;
+ env->visit_states = visit_states;
+
+ return 0;
+
+nomem:
+ kvfree(resolved_sizes);
+ kvfree(resolved_ids);
+ kvfree(visit_states);
+ return -ENOMEM;
+}
+
+void btf_verifier_env_free(struct btf_verifier_env *env)
+{
+ kvfree(env->visit_states);
+ kfree(env);
+}
+
+static bool env_type_is_resolve_sink(const struct btf_verifier_env *env,
+ const struct btf_type *next_type)
+{
+ switch (env->resolve_mode) {
+ case RESOLVE_TBD:
+ /* int, enum or void is a sink */
+ return !btf_type_needs_resolve(next_type);
+ case RESOLVE_PTR:
+ /* int, enum, void, struct, array, func or func_proto is a sink
+ * for ptr
+ */
+ return !btf_type_is_modifier(next_type) &&
+ !btf_type_is_ptr(next_type);
+ case RESOLVE_STRUCT_OR_ARRAY:
+ /* int, enum, void, ptr, func or func_proto is a sink
+ * for struct and array
+ */
+ return !btf_type_is_modifier(next_type) &&
+ !btf_type_is_array(next_type) &&
+ !btf_type_is_struct(next_type);
+ default:
+ BUG();
+ }
+}
+
+static bool env_type_is_resolved(const struct btf_verifier_env *env,
+ u32 type_id)
+{
+ /* base BTF types should be resolved by now */
+ if (type_id < env->btf->start_id)
+ return true;
+
+ return env->visit_states[type_id - env->btf->start_id] == RESOLVED;
+}
+
+static int env_stack_push(struct btf_verifier_env *env,
+ const struct btf_type *t, u32 type_id)
+{
+ const struct btf *btf = env->btf;
+ struct resolve_vertex *v;
+
+ if (env->top_stack == MAX_RESOLVE_DEPTH)
+ return -E2BIG;
+
+ if (type_id < btf->start_id
+ || env->visit_states[type_id - btf->start_id] != NOT_VISITED)
+ return -EEXIST;
+
+ env->visit_states[type_id - btf->start_id] = VISITED;
+
+ v = &env->stack[env->top_stack++];
+ v->t = t;
+ v->type_id = type_id;
+ v->next_member = 0;
+
+ if (env->resolve_mode == RESOLVE_TBD) {
+ if (btf_type_is_ptr(t))
+ env->resolve_mode = RESOLVE_PTR;
+ else if (btf_type_is_struct(t) || btf_type_is_array(t))
+ env->resolve_mode = RESOLVE_STRUCT_OR_ARRAY;
+ }
+
+ return 0;
+}
+
+static void env_stack_set_next_member(struct btf_verifier_env *env,
+ u16 next_member)
+{
+ env->stack[env->top_stack - 1].next_member = next_member;
+}
+
+static void env_stack_pop_resolved(struct btf_verifier_env *env,
+ u32 resolved_type_id,
+ u32 resolved_size)
+{
+ u32 type_id = env->stack[--(env->top_stack)].type_id;
+ struct btf *btf = env->btf;
+
+ type_id -= btf->start_id; /* adjust to local type id */
+ btf->resolved_sizes[type_id] = resolved_size;
+ btf->resolved_ids[type_id] = resolved_type_id;
+ env->visit_states[type_id] = RESOLVED;
+}
+
+static const struct resolve_vertex *env_stack_peak(struct btf_verifier_env *env)
+{
+ return env->top_stack ? &env->stack[env->top_stack - 1] : NULL;
+}
+
+/* Resolve the size of a passed-in "type"
+ *
+ * type: is an array (e.g. u32 array[x][y])
+ * return type: type "u32[x][y]", i.e. BTF_KIND_ARRAY,
+ * *type_size: (x * y * sizeof(u32)). Hence, *type_size always
+ * corresponds to the return type.
+ * *elem_type: u32
+ * *elem_id: id of u32
+ * *total_nelems: (x * y). Hence, individual elem size is
+ * (*type_size / *total_nelems)
+ * *type_id: id of type if it's changed within the function, 0 if not
+ *
+ * type: is not an array (e.g. const struct X)
+ * return type: type "struct X"
+ * *type_size: sizeof(struct X)
+ * *elem_type: same as return type ("struct X")
+ * *elem_id: 0
+ * *total_nelems: 1
+ * *type_id: id of type if it's changed within the function, 0 if not
+ */
+const struct btf_type *
+__btf_resolve_size(const struct btf *btf, const struct btf_type *type,
+ u32 *type_size, const struct btf_type **elem_type,
+ u32 *elem_id, u32 *total_nelems, u32 *type_id)
+{
+ const struct btf_type *array_type = NULL;
+ const struct btf_array *array = NULL;
+ u32 i, size, nelems = 1, id = 0;
+
+ for (i = 0; i < MAX_RESOLVE_DEPTH; i++) {
+ switch (BTF_INFO_KIND(type->info)) {
+ /* type->size can be used */
+ case BTF_KIND_INT:
+ case BTF_KIND_STRUCT:
+ case BTF_KIND_UNION:
+ case BTF_KIND_ENUM:
+ case BTF_KIND_FLOAT:
+ case BTF_KIND_ENUM64:
+ size = type->size;
+ goto resolved;
+
+ case BTF_KIND_PTR:
+ size = sizeof(void *);
+ goto resolved;
+
+ /* Modifiers */
+ case BTF_KIND_TYPEDEF:
+ case BTF_KIND_VOLATILE:
+ case BTF_KIND_CONST:
+ case BTF_KIND_RESTRICT:
+ case BTF_KIND_TYPE_TAG:
+ id = type->type;
+ type = btf_type_by_id(btf, type->type);
+ break;
+
+ case BTF_KIND_ARRAY:
+ if (!array_type)
+ array_type = type;
+ array = btf_type_array(type);
+ if (nelems && array->nelems > U32_MAX / nelems)
+ return ERR_PTR(-EINVAL);
+ nelems *= array->nelems;
+ type = btf_type_by_id(btf, array->type);
+ break;
+
+ /* type without size */
+ default:
+ return ERR_PTR(-EINVAL);
+ }
+ }
+
+ return ERR_PTR(-EINVAL);
+
+resolved:
+ if (nelems && size > U32_MAX / nelems)
+ return ERR_PTR(-EINVAL);
+
+ *type_size = nelems * size;
+ if (total_nelems)
+ *total_nelems = nelems;
+ if (elem_type)
+ *elem_type = type;
+ if (elem_id)
+ *elem_id = array ? array->type : 0;
+ if (type_id && id)
+ *type_id = id;
+
+ return array_type ? : type;
+}
+
+const struct btf_type *
+btf_resolve_size(const struct btf *btf, const struct btf_type *type,
+ u32 *type_size)
+{
+ return __btf_resolve_size(btf, type, type_size, NULL, NULL, NULL, NULL);
+}
+
+static u32 btf_resolved_type_id(const struct btf *btf, u32 type_id)
+{
+ while (type_id < btf->start_id)
+ btf = btf->base_btf;
+
+ return btf->resolved_ids[type_id - btf->start_id];
+}
+
+/* The input param "type_id" must point to a needs_resolve type */
+static const struct btf_type *btf_type_id_resolve(const struct btf *btf,
+ u32 *type_id)
+{
+ *type_id = btf_resolved_type_id(btf, *type_id);
+ return btf_type_by_id(btf, *type_id);
+}
+
+static u32 btf_resolved_type_size(const struct btf *btf, u32 type_id)
+{
+ while (type_id < btf->start_id)
+ btf = btf->base_btf;
+
+ return btf->resolved_sizes[type_id - btf->start_id];
+}
+
+const struct btf_type *btf_type_id_size(const struct btf *btf,
+ u32 *type_id, u32 *ret_size)
+{
+ const struct btf_type *size_type;
+ u32 size_type_id = *type_id;
+ u32 size = 0;
+
+ size_type = btf_type_by_id(btf, size_type_id);
+ if (btf_type_nosize_or_null(size_type))
+ return NULL;
+
+ if (btf_type_has_size(size_type)) {
+ size = size_type->size;
+ } else if (btf_type_is_array(size_type)) {
+ size = btf_resolved_type_size(btf, size_type_id);
+ } else if (btf_type_is_ptr(size_type)) {
+ size = sizeof(void *);
+ } else {
+ if (WARN_ON_ONCE(!btf_type_is_modifier(size_type) &&
+ !btf_type_is_var(size_type)))
+ return NULL;
+
+ size_type_id = btf_resolved_type_id(btf, size_type_id);
+ size_type = btf_type_by_id(btf, size_type_id);
+ if (btf_type_nosize_or_null(size_type))
+ return NULL;
+ else if (btf_type_has_size(size_type))
+ size = size_type->size;
+ else if (btf_type_is_array(size_type))
+ size = btf_resolved_type_size(btf, size_type_id);
+ else if (btf_type_is_ptr(size_type))
+ size = sizeof(void *);
+ else
+ return NULL;
+ }
+
+ *type_id = size_type_id;
+ if (ret_size)
+ *ret_size = size;
+
+ return size_type;
+}
+
+static int btf_df_check_member(struct btf_verifier_env *env,
+ const struct btf_type *struct_type,
+ const struct btf_member *member,
+ const struct btf_type *member_type)
+{
+ btf_verifier_log_basic(env, struct_type,
+ "Unsupported check_member");
+ return -EINVAL;
+}
+
+static int btf_df_check_kflag_member(struct btf_verifier_env *env,
+ const struct btf_type *struct_type,
+ const struct btf_member *member,
+ const struct btf_type *member_type)
+{
+ btf_verifier_log_basic(env, struct_type,
+ "Unsupported check_kflag_member");
+ return -EINVAL;
+}
+
+/* Used for ptr, array struct/union and float type members.
+ * int, enum and modifier types have their specific callback functions.
+ */
+static int btf_generic_check_kflag_member(struct btf_verifier_env *env,
+ const struct btf_type *struct_type,
+ const struct btf_member *member,
+ const struct btf_type *member_type)
+{
+ if (BTF_MEMBER_BITFIELD_SIZE(member->offset)) {
+ btf_verifier_log_member(env, struct_type, member,
+ "Invalid member bitfield_size");
+ return -EINVAL;
+ }
+
+ /* bitfield size is 0, so member->offset represents bit offset only.
+ * It is safe to call non kflag check_member variants.
+ */
+ return btf_type_ops(member_type)->check_member(env, struct_type,
+ member,
+ member_type);
+}
+
+static int btf_df_resolve(struct btf_verifier_env *env,
+ const struct resolve_vertex *v)
+{
+ btf_verifier_log_basic(env, v->t, "Unsupported resolve");
+ return -EINVAL;
+}
+
+static void btf_df_show(const struct btf *btf, const struct btf_type *t,
+ u32 type_id, void *data, u8 bits_offsets,
+ struct btf_show *show)
+{
+ btf_show(show, "<unsupported kind:%u>", BTF_INFO_KIND(t->info));
+}
+
+static int btf_int_check_member(struct btf_verifier_env *env,
+ const struct btf_type *struct_type,
+ const struct btf_member *member,
+ const struct btf_type *member_type)
+{
+ u32 int_data = btf_type_int(member_type);
+ u32 struct_bits_off = member->offset;
+ u32 struct_size = struct_type->size;
+ u32 nr_copy_bits;
+ u32 bytes_offset;
+
+ if (U32_MAX - struct_bits_off < BTF_INT_OFFSET(int_data)) {
+ btf_verifier_log_member(env, struct_type, member,
+ "bits_offset exceeds U32_MAX");
+ return -EINVAL;
+ }
+
+ struct_bits_off += BTF_INT_OFFSET(int_data);
+ bytes_offset = BITS_ROUNDDOWN_BYTES(struct_bits_off);
+ nr_copy_bits = BTF_INT_BITS(int_data) +
+ BITS_PER_BYTE_MASKED(struct_bits_off);
+
+ if (nr_copy_bits > BITS_PER_U128) {
+ btf_verifier_log_member(env, struct_type, member,
+ "nr_copy_bits exceeds 128");
+ return -EINVAL;
+ }
+
+ if (struct_size < bytes_offset ||
+ struct_size - bytes_offset < BITS_ROUNDUP_BYTES(nr_copy_bits)) {
+ btf_verifier_log_member(env, struct_type, member,
+ "Member exceeds struct_size");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int btf_int_check_kflag_member(struct btf_verifier_env *env,
+ const struct btf_type *struct_type,
+ const struct btf_member *member,
+ const struct btf_type *member_type)
+{
+ u32 struct_bits_off, nr_bits, nr_int_data_bits, bytes_offset;
+ u32 int_data = btf_type_int(member_type);
+ u32 struct_size = struct_type->size;
+ u32 nr_copy_bits;
+
+ /* a regular int type is required for the kflag int member */
+ if (!btf_type_int_is_regular(member_type)) {
+ btf_verifier_log_member(env, struct_type, member,
+ "Invalid member base type");
+ return -EINVAL;
+ }
+
+ /* check sanity of bitfield size */
+ nr_bits = BTF_MEMBER_BITFIELD_SIZE(member->offset);
+ struct_bits_off = BTF_MEMBER_BIT_OFFSET(member->offset);
+ nr_int_data_bits = BTF_INT_BITS(int_data);
+ if (!nr_bits) {
+ /* Not a bitfield member, member offset must be at byte
+ * boundary.
+ */
+ if (BITS_PER_BYTE_MASKED(struct_bits_off)) {
+ btf_verifier_log_member(env, struct_type, member,
+ "Invalid member offset");
+ return -EINVAL;
+ }
+
+ nr_bits = nr_int_data_bits;
+ } else if (nr_bits > nr_int_data_bits) {
+ btf_verifier_log_member(env, struct_type, member,
+ "Invalid member bitfield_size");
+ return -EINVAL;
+ }
+
+ bytes_offset = BITS_ROUNDDOWN_BYTES(struct_bits_off);
+ nr_copy_bits = nr_bits + BITS_PER_BYTE_MASKED(struct_bits_off);
+ if (nr_copy_bits > BITS_PER_U128) {
+ btf_verifier_log_member(env, struct_type, member,
+ "nr_copy_bits exceeds 128");
+ return -EINVAL;
+ }
+
+ if (struct_size < bytes_offset ||
+ struct_size - bytes_offset < BITS_ROUNDUP_BYTES(nr_copy_bits)) {
+ btf_verifier_log_member(env, struct_type, member,
+ "Member exceeds struct_size");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static s32 btf_int_check_meta(struct btf_verifier_env *env,
+ const struct btf_type *t,
+ u32 meta_left)
+{
+ u32 int_data, nr_bits, meta_needed = sizeof(int_data);
+ u16 encoding;
+
+ if (meta_left < meta_needed) {
+ btf_verifier_log_basic(env, t,
+ "meta_left:%u meta_needed:%u",
+ meta_left, meta_needed);
+ return -EINVAL;
+ }
+
+ if (btf_type_vlen(t)) {
+ btf_verifier_log_type(env, t, "vlen != 0");
+ return -EINVAL;
+ }
+
+ if (btf_type_kflag(t)) {
+ btf_verifier_log_type(env, t, "Invalid btf_info kind_flag");
+ return -EINVAL;
+ }
+
+ int_data = btf_type_int(t);
+ if (int_data & ~BTF_INT_MASK) {
+ btf_verifier_log_basic(env, t, "Invalid int_data:%x",
+ int_data);
+ return -EINVAL;
+ }
+
+ nr_bits = BTF_INT_BITS(int_data) + BTF_INT_OFFSET(int_data);
+
+ if (nr_bits > BITS_PER_U128) {
+ btf_verifier_log_type(env, t, "nr_bits exceeds %zu",
+ BITS_PER_U128);
+ return -EINVAL;
+ }
+
+ if (BITS_ROUNDUP_BYTES(nr_bits) > t->size) {
+ btf_verifier_log_type(env, t, "nr_bits exceeds type_size");
+ return -EINVAL;
+ }
+
+ /*
+ * Only one of the encoding bits is allowed and it
+ * should be sufficient for the pretty print purpose (i.e. decoding).
+ * Multiple bits can be allowed later if it is found
+ * to be insufficient.
+ */
+ encoding = BTF_INT_ENCODING(int_data);
+ if (encoding &&
+ encoding != BTF_INT_SIGNED &&
+ encoding != BTF_INT_CHAR &&
+ encoding != BTF_INT_BOOL) {
+ btf_verifier_log_type(env, t, "Unsupported encoding");
+ return -ENOTSUPP;
+ }
+
+ btf_verifier_log_type(env, t, NULL);
+
+ return meta_needed;
+}
+
+static void btf_int_log(struct btf_verifier_env *env,
+ const struct btf_type *t)
+{
+ int int_data = btf_type_int(t);
+
+ btf_verifier_log(env,
+ "size=%u bits_offset=%u nr_bits=%u encoding=%s",
+ t->size, BTF_INT_OFFSET(int_data),
+ BTF_INT_BITS(int_data),
+ btf_int_encoding_str(BTF_INT_ENCODING(int_data)));
+}
+
+static void btf_int128_print(struct btf_show *show, void *data)
+{
+ /* data points to a __int128 number.
+ * Suppose
+ * int128_num = *(__int128 *)data;
+ * The below formulas shows what upper_num and lower_num represents:
+ * upper_num = int128_num >> 64;
+ * lower_num = int128_num & 0xffffffffFFFFFFFFULL;
+ */
+ u64 upper_num, lower_num;
+
+#ifdef __BIG_ENDIAN_BITFIELD
+ upper_num = *(u64 *)data;
+ lower_num = *(u64 *)(data + 8);
+#else
+ upper_num = *(u64 *)(data + 8);
+ lower_num = *(u64 *)data;
+#endif
+ if (upper_num == 0)
+ btf_show_type_value(show, "0x%llx", lower_num);
+ else
+ btf_show_type_values(show, "0x%llx%016llx", upper_num,
+ lower_num);
+}
+
+static void btf_int128_shift(u64 *print_num, u16 left_shift_bits,
+ u16 right_shift_bits)
+{
+ u64 upper_num, lower_num;
+
+#ifdef __BIG_ENDIAN_BITFIELD
+ upper_num = print_num[0];
+ lower_num = print_num[1];
+#else
+ upper_num = print_num[1];
+ lower_num = print_num[0];
+#endif
+
+ /* shake out un-needed bits by shift/or operations */
+ if (left_shift_bits >= 64) {
+ upper_num = lower_num << (left_shift_bits - 64);
+ lower_num = 0;
+ } else {
+ upper_num = (upper_num << left_shift_bits) |
+ (lower_num >> (64 - left_shift_bits));
+ lower_num = lower_num << left_shift_bits;
+ }
+
+ if (right_shift_bits >= 64) {
+ lower_num = upper_num >> (right_shift_bits - 64);
+ upper_num = 0;
+ } else {
+ lower_num = (lower_num >> right_shift_bits) |
+ (upper_num << (64 - right_shift_bits));
+ upper_num = upper_num >> right_shift_bits;
+ }
+
+#ifdef __BIG_ENDIAN_BITFIELD
+ print_num[0] = upper_num;
+ print_num[1] = lower_num;
+#else
+ print_num[0] = lower_num;
+ print_num[1] = upper_num;
+#endif
+}
+
+static void btf_bitfield_show(void *data, u8 bits_offset,
+ u8 nr_bits, struct btf_show *show)
+{
+ u16 left_shift_bits, right_shift_bits;
+ u8 nr_copy_bytes;
+ u8 nr_copy_bits;
+ u64 print_num[2] = {};
+
+ nr_copy_bits = nr_bits + bits_offset;
+ nr_copy_bytes = BITS_ROUNDUP_BYTES(nr_copy_bits);
+
+ memcpy(print_num, data, nr_copy_bytes);
+
+#ifdef __BIG_ENDIAN_BITFIELD
+ left_shift_bits = bits_offset;
+#else
+ left_shift_bits = BITS_PER_U128 - nr_copy_bits;
+#endif
+ right_shift_bits = BITS_PER_U128 - nr_bits;
+
+ btf_int128_shift(print_num, left_shift_bits, right_shift_bits);
+ btf_int128_print(show, print_num);
+}
+
+
+static void btf_int_bits_show(const struct btf *btf,
+ const struct btf_type *t,
+ void *data, u8 bits_offset,
+ struct btf_show *show)
+{
+ u32 int_data = btf_type_int(t);
+ u8 nr_bits = BTF_INT_BITS(int_data);
+ u8 total_bits_offset;
+
+ /*
+ * bits_offset is at most 7.
+ * BTF_INT_OFFSET() cannot exceed 128 bits.
+ */
+ total_bits_offset = bits_offset + BTF_INT_OFFSET(int_data);
+ data += BITS_ROUNDDOWN_BYTES(total_bits_offset);
+ bits_offset = BITS_PER_BYTE_MASKED(total_bits_offset);
+ btf_bitfield_show(data, bits_offset, nr_bits, show);
+}
+
+static void btf_int_show(const struct btf *btf, const struct btf_type *t,
+ u32 type_id, void *data, u8 bits_offset,
+ struct btf_show *show)
+{
+ u32 int_data = btf_type_int(t);
+ u8 encoding = BTF_INT_ENCODING(int_data);
+ bool sign = encoding & BTF_INT_SIGNED;
+ u8 nr_bits = BTF_INT_BITS(int_data);
+ void *safe_data;
+
+ safe_data = btf_show_start_type(show, t, type_id, data);
+ if (!safe_data)
+ return;
+
+ if (bits_offset || BTF_INT_OFFSET(int_data) ||
+ BITS_PER_BYTE_MASKED(nr_bits)) {
+ btf_int_bits_show(btf, t, safe_data, bits_offset, show);
+ goto out;
+ }
+
+ switch (nr_bits) {
+ case 128:
+ btf_int128_print(show, safe_data);
+ break;
+ case 64:
+ if (sign)
+ btf_show_type_value(show, "%lld", *(s64 *)safe_data);
+ else
+ btf_show_type_value(show, "%llu", *(u64 *)safe_data);
+ break;
+ case 32:
+ if (sign)
+ btf_show_type_value(show, "%d", *(s32 *)safe_data);
+ else
+ btf_show_type_value(show, "%u", *(u32 *)safe_data);
+ break;
+ case 16:
+ if (sign)
+ btf_show_type_value(show, "%d", *(s16 *)safe_data);
+ else
+ btf_show_type_value(show, "%u", *(u16 *)safe_data);
+ break;
+ case 8:
+ if (show->state.array_encoding == BTF_INT_CHAR) {
+ /* check for null terminator */
+ if (show->state.array_terminated)
+ break;
+ if (*(char *)data == '\0') {
+ show->state.array_terminated = 1;
+ break;
+ }
+ if (isprint(*(char *)data)) {
+ btf_show_type_value(show, "'%c'",
+ *(char *)safe_data);
+ break;
+ }
+ }
+ if (sign)
+ btf_show_type_value(show, "%d", *(s8 *)safe_data);
+ else
+ btf_show_type_value(show, "%u", *(u8 *)safe_data);
+ break;
+ default:
+ btf_int_bits_show(btf, t, safe_data, bits_offset, show);
+ break;
+ }
+out:
+ btf_show_end_type(show);
+}
+
+static const struct btf_kind_operations int_ops = {
+ .check_meta = btf_int_check_meta,
+ .resolve = btf_df_resolve,
+ .check_member = btf_int_check_member,
+ .check_kflag_member = btf_int_check_kflag_member,
+ .log_details = btf_int_log,
+ .show = btf_int_show,
+};
+
+static int btf_modifier_check_member(struct btf_verifier_env *env,
+ const struct btf_type *struct_type,
+ const struct btf_member *member,
+ const struct btf_type *member_type)
+{
+ const struct btf_type *resolved_type;
+ u32 resolved_type_id = member->type;
+ struct btf_member resolved_member;
+ struct btf *btf = env->btf;
+
+ resolved_type = btf_type_id_size(btf, &resolved_type_id, NULL);
+ if (!resolved_type) {
+ btf_verifier_log_member(env, struct_type, member,
+ "Invalid member");
+ return -EINVAL;
+ }
+
+ resolved_member = *member;
+ resolved_member.type = resolved_type_id;
+
+ return btf_type_ops(resolved_type)->check_member(env, struct_type,
+ &resolved_member,
+ resolved_type);
+}
+
+static int btf_modifier_check_kflag_member(struct btf_verifier_env *env,
+ const struct btf_type *struct_type,
+ const struct btf_member *member,
+ const struct btf_type *member_type)
+{
+ const struct btf_type *resolved_type;
+ u32 resolved_type_id = member->type;
+ struct btf_member resolved_member;
+ struct btf *btf = env->btf;
+
+ resolved_type = btf_type_id_size(btf, &resolved_type_id, NULL);
+ if (!resolved_type) {
+ btf_verifier_log_member(env, struct_type, member,
+ "Invalid member");
+ return -EINVAL;
+ }
+
+ resolved_member = *member;
+ resolved_member.type = resolved_type_id;
+
+ return btf_type_ops(resolved_type)->check_kflag_member(env, struct_type,
+ &resolved_member,
+ resolved_type);
+}
+
+static int btf_ptr_check_member(struct btf_verifier_env *env,
+ const struct btf_type *struct_type,
+ const struct btf_member *member,
+ const struct btf_type *member_type)
+{
+ u32 struct_size, struct_bits_off, bytes_offset;
+
+ struct_size = struct_type->size;
+ struct_bits_off = member->offset;
+ bytes_offset = BITS_ROUNDDOWN_BYTES(struct_bits_off);
+
+ if (BITS_PER_BYTE_MASKED(struct_bits_off)) {
+ btf_verifier_log_member(env, struct_type, member,
+ "Member is not byte aligned");
+ return -EINVAL;
+ }
+
+ if (struct_size - bytes_offset < sizeof(void *)) {
+ btf_verifier_log_member(env, struct_type, member,
+ "Member exceeds struct_size");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int btf_ref_type_check_meta(struct btf_verifier_env *env,
+ const struct btf_type *t,
+ u32 meta_left)
+{
+ const char *value;
+
+ if (btf_type_vlen(t)) {
+ btf_verifier_log_type(env, t, "vlen != 0");
+ return -EINVAL;
+ }
+
+ if (btf_type_kflag(t) && !btf_type_is_type_tag(t)) {
+ btf_verifier_log_type(env, t, "Invalid btf_info kind_flag");
+ return -EINVAL;
+ }
+
+ if (!BTF_TYPE_ID_VALID(t->type)) {
+ btf_verifier_log_type(env, t, "Invalid type_id");
+ return -EINVAL;
+ }
+
+ /* typedef/type_tag type must have a valid name, and other ref types,
+ * volatile, const, restrict, should have a null name.
+ */
+ if (BTF_INFO_KIND(t->info) == BTF_KIND_TYPEDEF) {
+ if (!t->name_off ||
+ !btf_name_valid_identifier(env->btf, t->name_off)) {
+ btf_verifier_log_type(env, t, "Invalid name");
+ return -EINVAL;
+ }
+ } else if (BTF_INFO_KIND(t->info) == BTF_KIND_TYPE_TAG) {
+ value = btf_name_by_offset(env->btf, t->name_off);
+ if (!value || !value[0]) {
+ btf_verifier_log_type(env, t, "Invalid name");
+ return -EINVAL;
+ }
+ } else {
+ if (t->name_off) {
+ btf_verifier_log_type(env, t, "Invalid name");
+ return -EINVAL;
+ }
+ }
+
+ btf_verifier_log_type(env, t, NULL);
+
+ return 0;
+}
+
+static int btf_modifier_resolve(struct btf_verifier_env *env,
+ const struct resolve_vertex *v)
+{
+ const struct btf_type *t = v->t;
+ const struct btf_type *next_type;
+ u32 next_type_id = t->type;
+ struct btf *btf = env->btf;
+
+ next_type = btf_type_by_id(btf, next_type_id);
+ if (!next_type || btf_type_is_resolve_source_only(next_type)) {
+ btf_verifier_log_type(env, v->t, "Invalid type_id");
+ return -EINVAL;
+ }
+
+ if (!env_type_is_resolve_sink(env, next_type) &&
+ !env_type_is_resolved(env, next_type_id))
+ return env_stack_push(env, next_type, next_type_id);
+
+ /* Figure out the resolved next_type_id with size.
+ * They will be stored in the current modifier's
+ * resolved_ids and resolved_sizes such that it can
+ * save us a few type-following when we use it later (e.g. in
+ * pretty print).
+ */
+ if (!btf_type_id_size(btf, &next_type_id, NULL)) {
+ if (env_type_is_resolved(env, next_type_id))
+ next_type = btf_type_id_resolve(btf, &next_type_id);
+
+ /* "typedef void new_void", "const void"...etc */
+ if (!btf_type_is_void(next_type) &&
+ !btf_type_is_fwd(next_type) &&
+ !btf_type_is_func_proto(next_type)) {
+ btf_verifier_log_type(env, v->t, "Invalid type_id");
+ return -EINVAL;
+ }
+ }
+
+ env_stack_pop_resolved(env, next_type_id, 0);
+
+ return 0;
+}
+
+static int btf_var_resolve(struct btf_verifier_env *env,
+ const struct resolve_vertex *v)
+{
+ const struct btf_type *next_type;
+ const struct btf_type *t = v->t;
+ u32 next_type_id = t->type;
+ struct btf *btf = env->btf;
+
+ next_type = btf_type_by_id(btf, next_type_id);
+ if (!next_type || btf_type_is_resolve_source_only(next_type)) {
+ btf_verifier_log_type(env, v->t, "Invalid type_id");
+ return -EINVAL;
+ }
+
+ if (!env_type_is_resolve_sink(env, next_type) &&
+ !env_type_is_resolved(env, next_type_id))
+ return env_stack_push(env, next_type, next_type_id);
+
+ if (btf_type_is_modifier(next_type)) {
+ const struct btf_type *resolved_type;
+ u32 resolved_type_id;
+
+ resolved_type_id = next_type_id;
+ resolved_type = btf_type_id_resolve(btf, &resolved_type_id);
+
+ if (btf_type_is_ptr(resolved_type) &&
+ !env_type_is_resolve_sink(env, resolved_type) &&
+ !env_type_is_resolved(env, resolved_type_id))
+ return env_stack_push(env, resolved_type,
+ resolved_type_id);
+ }
+
+ /* We must resolve to something concrete at this point, no
+ * forward types or similar that would resolve to size of
+ * zero is allowed.
+ */
+ if (!btf_type_id_size(btf, &next_type_id, NULL)) {
+ btf_verifier_log_type(env, v->t, "Invalid type_id");
+ return -EINVAL;
+ }
+
+ env_stack_pop_resolved(env, next_type_id, 0);
+
+ return 0;
+}
+
+static int btf_ptr_resolve(struct btf_verifier_env *env,
+ const struct resolve_vertex *v)
+{
+ const struct btf_type *next_type;
+ const struct btf_type *t = v->t;
+ u32 next_type_id = t->type;
+ struct btf *btf = env->btf;
+
+ next_type = btf_type_by_id(btf, next_type_id);
+ if (!next_type || btf_type_is_resolve_source_only(next_type)) {
+ btf_verifier_log_type(env, v->t, "Invalid type_id");
+ return -EINVAL;
+ }
+
+ if (!env_type_is_resolve_sink(env, next_type) &&
+ !env_type_is_resolved(env, next_type_id))
+ return env_stack_push(env, next_type, next_type_id);
+
+ /* If the modifier was RESOLVED during RESOLVE_STRUCT_OR_ARRAY,
+ * the modifier may have stopped resolving when it was resolved
+ * to a ptr (last-resolved-ptr).
+ *
+ * We now need to continue from the last-resolved-ptr to
+ * ensure the last-resolved-ptr will not referring back to
+ * the current ptr (t).
+ */
+ if (btf_type_is_modifier(next_type)) {
+ const struct btf_type *resolved_type;
+ u32 resolved_type_id;
+
+ resolved_type_id = next_type_id;
+ resolved_type = btf_type_id_resolve(btf, &resolved_type_id);
+
+ if (btf_type_is_ptr(resolved_type) &&
+ !env_type_is_resolve_sink(env, resolved_type) &&
+ !env_type_is_resolved(env, resolved_type_id))
+ return env_stack_push(env, resolved_type,
+ resolved_type_id);
+ }
+
+ if (!btf_type_id_size(btf, &next_type_id, NULL)) {
+ if (env_type_is_resolved(env, next_type_id))
+ next_type = btf_type_id_resolve(btf, &next_type_id);
+
+ if (!btf_type_is_void(next_type) &&
+ !btf_type_is_fwd(next_type) &&
+ !btf_type_is_func_proto(next_type)) {
+ btf_verifier_log_type(env, v->t, "Invalid type_id");
+ return -EINVAL;
+ }
+ }
+
+ env_stack_pop_resolved(env, next_type_id, 0);
+
+ return 0;
+}
+
+static void btf_modifier_show(const struct btf *btf,
+ const struct btf_type *t,
+ u32 type_id, void *data,
+ u8 bits_offset, struct btf_show *show)
+{
+ if (btf->resolved_ids)
+ t = btf_type_id_resolve(btf, &type_id);
+ else
+ t = btf_type_skip_modifiers(btf, type_id, NULL);
+
+ btf_type_ops(t)->show(btf, t, type_id, data, bits_offset, show);
+}
+
+static void btf_var_show(const struct btf *btf, const struct btf_type *t,
+ u32 type_id, void *data, u8 bits_offset,
+ struct btf_show *show)
+{
+ t = btf_type_id_resolve(btf, &type_id);
+
+ btf_type_ops(t)->show(btf, t, type_id, data, bits_offset, show);
+}
+
+static void btf_ptr_show(const struct btf *btf, const struct btf_type *t,
+ u32 type_id, void *data, u8 bits_offset,
+ struct btf_show *show)
+{
+ void *safe_data;
+
+ safe_data = btf_show_start_type(show, t, type_id, data);
+ if (!safe_data)
+ return;
+
+ /* It is a hashed value unless BTF_SHOW_PTR_RAW is specified */
+ if (show->flags & BTF_SHOW_PTR_RAW)
+ btf_show_type_value(show, "0x%px", *(void **)safe_data);
+ else
+ btf_show_type_value(show, "0x%p", *(void **)safe_data);
+ btf_show_end_type(show);
+}
+
+static void btf_ref_type_log(struct btf_verifier_env *env,
+ const struct btf_type *t)
+{
+ btf_verifier_log(env, "type_id=%u", t->type);
+}
+
+static const struct btf_kind_operations modifier_ops = {
+ .check_meta = btf_ref_type_check_meta,
+ .resolve = btf_modifier_resolve,
+ .check_member = btf_modifier_check_member,
+ .check_kflag_member = btf_modifier_check_kflag_member,
+ .log_details = btf_ref_type_log,
+ .show = btf_modifier_show,
+};
+
+static const struct btf_kind_operations ptr_ops = {
+ .check_meta = btf_ref_type_check_meta,
+ .resolve = btf_ptr_resolve,
+ .check_member = btf_ptr_check_member,
+ .check_kflag_member = btf_generic_check_kflag_member,
+ .log_details = btf_ref_type_log,
+ .show = btf_ptr_show,
+};
+
+static s32 btf_fwd_check_meta(struct btf_verifier_env *env,
+ const struct btf_type *t,
+ u32 meta_left)
+{
+ if (btf_type_vlen(t)) {
+ btf_verifier_log_type(env, t, "vlen != 0");
+ return -EINVAL;
+ }
+
+ if (t->type) {
+ btf_verifier_log_type(env, t, "type != 0");
+ return -EINVAL;
+ }
+
+ /* fwd type must have a valid name */
+ if (!t->name_off ||
+ !btf_name_valid_identifier(env->btf, t->name_off)) {
+ btf_verifier_log_type(env, t, "Invalid name");
+ return -EINVAL;
+ }
+
+ btf_verifier_log_type(env, t, NULL);
+
+ return 0;
+}
+
+static void btf_fwd_type_log(struct btf_verifier_env *env,
+ const struct btf_type *t)
+{
+ btf_verifier_log(env, "%s", btf_type_kflag(t) ? "union" : "struct");
+}
+
+static const struct btf_kind_operations fwd_ops = {
+ .check_meta = btf_fwd_check_meta,
+ .resolve = btf_df_resolve,
+ .check_member = btf_df_check_member,
+ .check_kflag_member = btf_df_check_kflag_member,
+ .log_details = btf_fwd_type_log,
+ .show = btf_df_show,
+};
+
+static int btf_array_check_member(struct btf_verifier_env *env,
+ const struct btf_type *struct_type,
+ const struct btf_member *member,
+ const struct btf_type *member_type)
+{
+ u32 struct_bits_off = member->offset;
+ u32 struct_size, bytes_offset;
+ u32 array_type_id, array_size;
+ struct btf *btf = env->btf;
+
+ if (BITS_PER_BYTE_MASKED(struct_bits_off)) {
+ btf_verifier_log_member(env, struct_type, member,
+ "Member is not byte aligned");
+ return -EINVAL;
+ }
+
+ array_type_id = member->type;
+ btf_type_id_size(btf, &array_type_id, &array_size);
+ struct_size = struct_type->size;
+ bytes_offset = BITS_ROUNDDOWN_BYTES(struct_bits_off);
+ if (struct_size - bytes_offset < array_size) {
+ btf_verifier_log_member(env, struct_type, member,
+ "Member exceeds struct_size");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static s32 btf_array_check_meta(struct btf_verifier_env *env,
+ const struct btf_type *t,
+ u32 meta_left)
+{
+ const struct btf_array *array = btf_type_array(t);
+ u32 meta_needed = sizeof(*array);
+
+ if (meta_left < meta_needed) {
+ btf_verifier_log_basic(env, t,
+ "meta_left:%u meta_needed:%u",
+ meta_left, meta_needed);
+ return -EINVAL;
+ }
+
+ /* array type should not have a name */
+ if (t->name_off) {
+ btf_verifier_log_type(env, t, "Invalid name");
+ return -EINVAL;
+ }
+
+ if (btf_type_vlen(t)) {
+ btf_verifier_log_type(env, t, "vlen != 0");
+ return -EINVAL;
+ }
+
+ if (btf_type_kflag(t)) {
+ btf_verifier_log_type(env, t, "Invalid btf_info kind_flag");
+ return -EINVAL;
+ }
+
+ if (t->size) {
+ btf_verifier_log_type(env, t, "size != 0");
+ return -EINVAL;
+ }
+
+ /* Array elem type and index type cannot be in type void,
+ * so !array->type and !array->index_type are not allowed.
+ */
+ if (!array->type || !BTF_TYPE_ID_VALID(array->type)) {
+ btf_verifier_log_type(env, t, "Invalid elem");
+ return -EINVAL;
+ }
+
+ if (!array->index_type || !BTF_TYPE_ID_VALID(array->index_type)) {
+ btf_verifier_log_type(env, t, "Invalid index");
+ return -EINVAL;
+ }
+
+ btf_verifier_log_type(env, t, NULL);
+
+ return meta_needed;
+}
+
+static int btf_array_resolve(struct btf_verifier_env *env,
+ const struct resolve_vertex *v)
+{
+ const struct btf_array *array = btf_type_array(v->t);
+ const struct btf_type *elem_type, *index_type;
+ u32 elem_type_id, index_type_id;
+ struct btf *btf = env->btf;
+ u32 elem_size;
+
+ /* Check array->index_type */
+ index_type_id = array->index_type;
+ index_type = btf_type_by_id(btf, index_type_id);
+ if (btf_type_nosize_or_null(index_type) ||
+ btf_type_is_resolve_source_only(index_type)) {
+ btf_verifier_log_type(env, v->t, "Invalid index");
+ return -EINVAL;
+ }
+
+ if (!env_type_is_resolve_sink(env, index_type) &&
+ !env_type_is_resolved(env, index_type_id))
+ return env_stack_push(env, index_type, index_type_id);
+
+ index_type = btf_type_id_size(btf, &index_type_id, NULL);
+ if (!index_type || !btf_type_is_int(index_type) ||
+ !btf_type_int_is_regular(index_type)) {
+ btf_verifier_log_type(env, v->t, "Invalid index");
+ return -EINVAL;
+ }
+
+ /* Check array->type */
+ elem_type_id = array->type;
+ elem_type = btf_type_by_id(btf, elem_type_id);
+ if (btf_type_nosize_or_null(elem_type) ||
+ btf_type_is_resolve_source_only(elem_type)) {
+ btf_verifier_log_type(env, v->t,
+ "Invalid elem");
+ return -EINVAL;
+ }
+
+ if (!env_type_is_resolve_sink(env, elem_type) &&
+ !env_type_is_resolved(env, elem_type_id))
+ return env_stack_push(env, elem_type, elem_type_id);
+
+ elem_type = btf_type_id_size(btf, &elem_type_id, &elem_size);
+ if (!elem_type) {
+ btf_verifier_log_type(env, v->t, "Invalid elem");
+ return -EINVAL;
+ }
+
+ if (btf_type_is_int(elem_type) && !btf_type_int_is_regular(elem_type)) {
+ btf_verifier_log_type(env, v->t, "Invalid array of int");
+ return -EINVAL;
+ }
+
+ if (array->nelems && elem_size > U32_MAX / array->nelems) {
+ btf_verifier_log_type(env, v->t,
+ "Array size overflows U32_MAX");
+ return -EINVAL;
+ }
+
+ env_stack_pop_resolved(env, elem_type_id, elem_size * array->nelems);
+
+ return 0;
+}
+
+static void btf_array_log(struct btf_verifier_env *env,
+ const struct btf_type *t)
+{
+ const struct btf_array *array = btf_type_array(t);
+
+ btf_verifier_log(env, "type_id=%u index_type_id=%u nr_elems=%u",
+ array->type, array->index_type, array->nelems);
+}
+
+static void __btf_array_show(const struct btf *btf, const struct btf_type *t,
+ u32 type_id, void *data, u8 bits_offset,
+ struct btf_show *show)
+{
+ const struct btf_array *array = btf_type_array(t);
+ const struct btf_kind_operations *elem_ops;
+ const struct btf_type *elem_type;
+ u32 i, elem_size = 0, elem_type_id;
+ u16 encoding = 0;
+
+ elem_type_id = array->type;
+ elem_type = btf_type_skip_modifiers(btf, elem_type_id, NULL);
+ if (elem_type && btf_type_has_size(elem_type))
+ elem_size = elem_type->size;
+
+ if (elem_type && btf_type_is_int(elem_type)) {
+ u32 int_type = btf_type_int(elem_type);
+
+ encoding = BTF_INT_ENCODING(int_type);
+
+ /*
+ * BTF_INT_CHAR encoding never seems to be set for
+ * char arrays, so if size is 1 and element is
+ * printable as a char, we'll do that.
+ */
+ if (elem_size == 1)
+ encoding = BTF_INT_CHAR;
+ }
+
+ if (!btf_show_start_array_type(show, t, type_id, encoding, data))
+ return;
+
+ if (!elem_type)
+ goto out;
+ elem_ops = btf_type_ops(elem_type);
+
+ for (i = 0; i < array->nelems; i++) {
+
+ btf_show_start_array_member(show);
+
+ elem_ops->show(btf, elem_type, elem_type_id, data,
+ bits_offset, show);
+ data += elem_size;
+
+ btf_show_end_array_member(show);
+
+ if (show->state.array_terminated)
+ break;
+ }
+out:
+ btf_show_end_array_type(show);
+}
+
+static void btf_array_show(const struct btf *btf, const struct btf_type *t,
+ u32 type_id, void *data, u8 bits_offset,
+ struct btf_show *show)
+{
+ const struct btf_member *m = show->state.member;
+
+ /*
+ * First check if any members would be shown (are non-zero).
+ * See comments above "struct btf_show" definition for more
+ * details on how this works at a high-level.
+ */
+ if (show->state.depth > 0 && !(show->flags & BTF_SHOW_ZERO)) {
+ if (!show->state.depth_check) {
+ show->state.depth_check = show->state.depth + 1;
+ show->state.depth_to_show = 0;
+ }
+ __btf_array_show(btf, t, type_id, data, bits_offset, show);
+ show->state.member = m;
+
+ if (show->state.depth_check != show->state.depth + 1)
+ return;
+ show->state.depth_check = 0;
+
+ if (show->state.depth_to_show <= show->state.depth)
+ return;
+ /*
+ * Reaching here indicates we have recursed and found
+ * non-zero array member(s).
+ */
+ }
+ __btf_array_show(btf, t, type_id, data, bits_offset, show);
+}
+
+static const struct btf_kind_operations array_ops = {
+ .check_meta = btf_array_check_meta,
+ .resolve = btf_array_resolve,
+ .check_member = btf_array_check_member,
+ .check_kflag_member = btf_generic_check_kflag_member,
+ .log_details = btf_array_log,
+ .show = btf_array_show,
+};
+
+static int btf_struct_check_member(struct btf_verifier_env *env,
+ const struct btf_type *struct_type,
+ const struct btf_member *member,
+ const struct btf_type *member_type)
+{
+ u32 struct_bits_off = member->offset;
+ u32 struct_size, bytes_offset;
+
+ if (BITS_PER_BYTE_MASKED(struct_bits_off)) {
+ btf_verifier_log_member(env, struct_type, member,
+ "Member is not byte aligned");
+ return -EINVAL;
+ }
+
+ struct_size = struct_type->size;
+ bytes_offset = BITS_ROUNDDOWN_BYTES(struct_bits_off);
+ if (struct_size - bytes_offset < member_type->size) {
+ btf_verifier_log_member(env, struct_type, member,
+ "Member exceeds struct_size");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static s32 btf_struct_check_meta(struct btf_verifier_env *env,
+ const struct btf_type *t,
+ u32 meta_left)
+{
+ bool is_union = BTF_INFO_KIND(t->info) == BTF_KIND_UNION;
+ const struct btf_member *member;
+ u32 meta_needed, last_offset;
+ struct btf *btf = env->btf;
+ u32 struct_size = t->size;
+ u32 offset;
+ u16 i;
+
+ meta_needed = btf_type_vlen(t) * sizeof(*member);
+ if (meta_left < meta_needed) {
+ btf_verifier_log_basic(env, t,
+ "meta_left:%u meta_needed:%u",
+ meta_left, meta_needed);
+ return -EINVAL;
+ }
+
+ /* struct type either no name or a valid one */
+ if (t->name_off &&
+ !btf_name_valid_identifier(env->btf, t->name_off)) {
+ btf_verifier_log_type(env, t, "Invalid name");
+ return -EINVAL;
+ }
+
+ btf_verifier_log_type(env, t, NULL);
+
+ last_offset = 0;
+ for_each_member(i, t, member) {
+ if (!btf_name_offset_valid(btf, member->name_off)) {
+ btf_verifier_log_member(env, t, member,
+ "Invalid member name_offset:%u",
+ member->name_off);
+ return -EINVAL;
+ }
+
+ /* struct member either no name or a valid one */
+ if (member->name_off &&
+ !btf_name_valid_identifier(btf, member->name_off)) {
+ btf_verifier_log_member(env, t, member, "Invalid name");
+ return -EINVAL;
+ }
+ /* A member cannot be in type void */
+ if (!member->type || !BTF_TYPE_ID_VALID(member->type)) {
+ btf_verifier_log_member(env, t, member,
+ "Invalid type_id");
+ return -EINVAL;
+ }
+
+ offset = __btf_member_bit_offset(t, member);
+ if (is_union && offset) {
+ btf_verifier_log_member(env, t, member,
+ "Invalid member bits_offset");
+ return -EINVAL;
+ }
+
+ /*
+ * ">" instead of ">=" because the last member could be
+ * "char a[0];"
+ */
+ if (last_offset > offset) {
+ btf_verifier_log_member(env, t, member,
+ "Invalid member bits_offset");
+ return -EINVAL;
+ }
+
+ if (BITS_ROUNDUP_BYTES(offset) > struct_size) {
+ btf_verifier_log_member(env, t, member,
+ "Member bits_offset exceeds its struct size");
+ return -EINVAL;
+ }
+
+ btf_verifier_log_member(env, t, member, NULL);
+ last_offset = offset;
+ }
+
+ return meta_needed;
+}
+
+static int btf_struct_resolve(struct btf_verifier_env *env,
+ const struct resolve_vertex *v)
+{
+ const struct btf_member *member;
+ int err;
+ u16 i;
+
+ /* Before continue resolving the next_member,
+ * ensure the last member is indeed resolved to a
+ * type with size info.
+ */
+ if (v->next_member) {
+ const struct btf_type *last_member_type;
+ const struct btf_member *last_member;
+ u32 last_member_type_id;
+
+ last_member = btf_type_member(v->t) + v->next_member - 1;
+ last_member_type_id = last_member->type;
+ if (WARN_ON_ONCE(!env_type_is_resolved(env,
+ last_member_type_id)))
+ return -EINVAL;
+
+ last_member_type = btf_type_by_id(env->btf,
+ last_member_type_id);
+ if (btf_type_kflag(v->t))
+ err = btf_type_ops(last_member_type)->check_kflag_member(env, v->t,
+ last_member,
+ last_member_type);
+ else
+ err = btf_type_ops(last_member_type)->check_member(env, v->t,
+ last_member,
+ last_member_type);
+ if (err)
+ return err;
+ }
+
+ for_each_member_from(i, v->next_member, v->t, member) {
+ u32 member_type_id = member->type;
+ const struct btf_type *member_type = btf_type_by_id(env->btf,
+ member_type_id);
+
+ if (btf_type_nosize_or_null(member_type) ||
+ btf_type_is_resolve_source_only(member_type)) {
+ btf_verifier_log_member(env, v->t, member,
+ "Invalid member");
+ return -EINVAL;
+ }
+
+ if (!env_type_is_resolve_sink(env, member_type) &&
+ !env_type_is_resolved(env, member_type_id)) {
+ env_stack_set_next_member(env, i + 1);
+ return env_stack_push(env, member_type, member_type_id);
+ }
+
+ if (btf_type_kflag(v->t))
+ err = btf_type_ops(member_type)->check_kflag_member(env, v->t,
+ member,
+ member_type);
+ else
+ err = btf_type_ops(member_type)->check_member(env, v->t,
+ member,
+ member_type);
+ if (err)
+ return err;
+ }
+
+ env_stack_pop_resolved(env, 0, 0);
+
+ return 0;
+}
+
+static void btf_struct_log(struct btf_verifier_env *env,
+ const struct btf_type *t)
+{
+ btf_verifier_log(env, "size=%u vlen=%u", t->size, btf_type_vlen(t));
+}
+
+
+static void __btf_struct_show(const struct btf *btf, const struct btf_type *t,
+ u32 type_id, void *data, u8 bits_offset,
+ struct btf_show *show)
+{
+ const struct btf_member *member;
+ void *safe_data;
+ u32 i;
+
+ safe_data = btf_show_start_struct_type(show, t, type_id, data);
+ if (!safe_data)
+ return;
+
+ for_each_member(i, t, member) {
+ const struct btf_type *member_type = btf_type_by_id(btf,
+ member->type);
+ const struct btf_kind_operations *ops;
+ u32 member_offset, bitfield_size;
+ u32 bytes_offset;
+ u8 bits8_offset;
+
+ btf_show_start_member(show, member);
+
+ member_offset = __btf_member_bit_offset(t, member);
+ bitfield_size = __btf_member_bitfield_size(t, member);
+ bytes_offset = BITS_ROUNDDOWN_BYTES(member_offset);
+ bits8_offset = BITS_PER_BYTE_MASKED(member_offset);
+ if (bitfield_size) {
+ safe_data = btf_show_start_type(show, member_type,
+ member->type,
+ data + bytes_offset);
+ if (safe_data)
+ btf_bitfield_show(safe_data,
+ bits8_offset,
+ bitfield_size, show);
+ btf_show_end_type(show);
+ } else {
+ ops = btf_type_ops(member_type);
+ ops->show(btf, member_type, member->type,
+ data + bytes_offset, bits8_offset, show);
+ }
+
+ btf_show_end_member(show);
+ }
+
+ btf_show_end_struct_type(show);
+}
+
+static void btf_struct_show(const struct btf *btf, const struct btf_type *t,
+ u32 type_id, void *data, u8 bits_offset,
+ struct btf_show *show)
+{
+ const struct btf_member *m = show->state.member;
+
+ /*
+ * First check if any members would be shown (are non-zero).
+ * See comments above "struct btf_show" definition for more
+ * details on how this works at a high-level.
+ */
+ if (show->state.depth > 0 && !(show->flags & BTF_SHOW_ZERO)) {
+ if (!show->state.depth_check) {
+ show->state.depth_check = show->state.depth + 1;
+ show->state.depth_to_show = 0;
+ }
+ __btf_struct_show(btf, t, type_id, data, bits_offset, show);
+ /* Restore saved member data here */
+ show->state.member = m;
+ if (show->state.depth_check != show->state.depth + 1)
+ return;
+ show->state.depth_check = 0;
+
+ if (show->state.depth_to_show <= show->state.depth)
+ return;
+ /*
+ * Reaching here indicates we have recursed and found
+ * non-zero child values.
+ */
+ }
+
+ __btf_struct_show(btf, t, type_id, data, bits_offset, show);
+}
+
+static const struct btf_kind_operations struct_ops = {
+ .check_meta = btf_struct_check_meta,
+ .resolve = btf_struct_resolve,
+ .check_member = btf_struct_check_member,
+ .check_kflag_member = btf_generic_check_kflag_member,
+ .log_details = btf_struct_log,
+ .show = btf_struct_show,
+};
+
+static int btf_enum_check_member(struct btf_verifier_env *env,
+ const struct btf_type *struct_type,
+ const struct btf_member *member,
+ const struct btf_type *member_type)
+{
+ u32 struct_bits_off = member->offset;
+ u32 struct_size, bytes_offset;
+
+ if (BITS_PER_BYTE_MASKED(struct_bits_off)) {
+ btf_verifier_log_member(env, struct_type, member,
+ "Member is not byte aligned");
+ return -EINVAL;
+ }
+
+ struct_size = struct_type->size;
+ bytes_offset = BITS_ROUNDDOWN_BYTES(struct_bits_off);
+ if (struct_size - bytes_offset < member_type->size) {
+ btf_verifier_log_member(env, struct_type, member,
+ "Member exceeds struct_size");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int btf_enum_check_kflag_member(struct btf_verifier_env *env,
+ const struct btf_type *struct_type,
+ const struct btf_member *member,
+ const struct btf_type *member_type)
+{
+ u32 struct_bits_off, nr_bits, bytes_end, struct_size;
+ u32 int_bitsize = sizeof(int) * BITS_PER_BYTE;
+
+ struct_bits_off = BTF_MEMBER_BIT_OFFSET(member->offset);
+ nr_bits = BTF_MEMBER_BITFIELD_SIZE(member->offset);
+ if (!nr_bits) {
+ if (BITS_PER_BYTE_MASKED(struct_bits_off)) {
+ btf_verifier_log_member(env, struct_type, member,
+ "Member is not byte aligned");
+ return -EINVAL;
+ }
+
+ nr_bits = int_bitsize;
+ } else if (nr_bits > int_bitsize) {
+ btf_verifier_log_member(env, struct_type, member,
+ "Invalid member bitfield_size");
+ return -EINVAL;
+ }
+
+ struct_size = struct_type->size;
+ bytes_end = BITS_ROUNDUP_BYTES(struct_bits_off + nr_bits);
+ if (struct_size < bytes_end) {
+ btf_verifier_log_member(env, struct_type, member,
+ "Member exceeds struct_size");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static s32 btf_enum_check_meta(struct btf_verifier_env *env,
+ const struct btf_type *t,
+ u32 meta_left)
+{
+ const struct btf_enum *enums = btf_type_enum(t);
+ struct btf *btf = env->btf;
+ const char *fmt_str;
+ u16 i, nr_enums;
+ u32 meta_needed;
+
+ nr_enums = btf_type_vlen(t);
+ meta_needed = nr_enums * sizeof(*enums);
+
+ if (meta_left < meta_needed) {
+ btf_verifier_log_basic(env, t,
+ "meta_left:%u meta_needed:%u",
+ meta_left, meta_needed);
+ return -EINVAL;
+ }
+
+ if (t->size > 8 || !is_power_of_2(t->size)) {
+ btf_verifier_log_type(env, t, "Unexpected size");
+ return -EINVAL;
+ }
+
+ /* enum type either no name or a valid one */
+ if (t->name_off &&
+ !btf_name_valid_identifier(env->btf, t->name_off)) {
+ btf_verifier_log_type(env, t, "Invalid name");
+ return -EINVAL;
+ }
+
+ btf_verifier_log_type(env, t, NULL);
+
+ for (i = 0; i < nr_enums; i++) {
+ if (!btf_name_offset_valid(btf, enums[i].name_off)) {
+ btf_verifier_log(env, "\tInvalid name_offset:%u",
+ enums[i].name_off);
+ return -EINVAL;
+ }
+
+ /* enum member must have a valid name */
+ if (!enums[i].name_off ||
+ !btf_name_valid_identifier(btf, enums[i].name_off)) {
+ btf_verifier_log_type(env, t, "Invalid name");
+ return -EINVAL;
+ }
+
+ if (env->log.level == BPF_LOG_KERNEL)
+ continue;
+ fmt_str = btf_type_kflag(t) ? "\t%s val=%d\n" : "\t%s val=%u\n";
+ btf_verifier_log(env, fmt_str,
+ __btf_name_by_offset(btf, enums[i].name_off),
+ enums[i].val);
+ }
+
+ return meta_needed;
+}
+
+static void btf_enum_log(struct btf_verifier_env *env,
+ const struct btf_type *t)
+{
+ btf_verifier_log(env, "size=%u vlen=%u", t->size, btf_type_vlen(t));
+}
+
+static void btf_enum_show(const struct btf *btf, const struct btf_type *t,
+ u32 type_id, void *data, u8 bits_offset,
+ struct btf_show *show)
+{
+ const struct btf_enum *enums = btf_type_enum(t);
+ u32 i, nr_enums = btf_type_vlen(t);
+ void *safe_data;
+ int v;
+
+ safe_data = btf_show_start_type(show, t, type_id, data);
+ if (!safe_data)
+ return;
+
+ v = *(int *)safe_data;
+
+ for (i = 0; i < nr_enums; i++) {
+ if (v != enums[i].val)
+ continue;
+
+ btf_show_type_value(show, "%s",
+ __btf_name_by_offset(btf,
+ enums[i].name_off));
+
+ btf_show_end_type(show);
+ return;
+ }
+
+ if (btf_type_kflag(t))
+ btf_show_type_value(show, "%d", v);
+ else
+ btf_show_type_value(show, "%u", v);
+ btf_show_end_type(show);
+}
+
+static const struct btf_kind_operations enum_ops = {
+ .check_meta = btf_enum_check_meta,
+ .resolve = btf_df_resolve,
+ .check_member = btf_enum_check_member,
+ .check_kflag_member = btf_enum_check_kflag_member,
+ .log_details = btf_enum_log,
+ .show = btf_enum_show,
+};
+
+static s32 btf_enum64_check_meta(struct btf_verifier_env *env,
+ const struct btf_type *t,
+ u32 meta_left)
+{
+ const struct btf_enum64 *enums = btf_type_enum64(t);
+ struct btf *btf = env->btf;
+ const char *fmt_str;
+ u16 i, nr_enums;
+ u32 meta_needed;
+
+ nr_enums = btf_type_vlen(t);
+ meta_needed = nr_enums * sizeof(*enums);
+
+ if (meta_left < meta_needed) {
+ btf_verifier_log_basic(env, t,
+ "meta_left:%u meta_needed:%u",
+ meta_left, meta_needed);
+ return -EINVAL;
+ }
+
+ if (t->size > 8 || !is_power_of_2(t->size)) {
+ btf_verifier_log_type(env, t, "Unexpected size");
+ return -EINVAL;
+ }
+
+ /* enum type either no name or a valid one */
+ if (t->name_off &&
+ !btf_name_valid_identifier(env->btf, t->name_off)) {
+ btf_verifier_log_type(env, t, "Invalid name");
+ return -EINVAL;
+ }
+
+ btf_verifier_log_type(env, t, NULL);
+
+ for (i = 0; i < nr_enums; i++) {
+ if (!btf_name_offset_valid(btf, enums[i].name_off)) {
+ btf_verifier_log(env, "\tInvalid name_offset:%u",
+ enums[i].name_off);
+ return -EINVAL;
+ }
+
+ /* enum member must have a valid name */
+ if (!enums[i].name_off ||
+ !btf_name_valid_identifier(btf, enums[i].name_off)) {
+ btf_verifier_log_type(env, t, "Invalid name");
+ return -EINVAL;
+ }
+
+ if (env->log.level == BPF_LOG_KERNEL)
+ continue;
+
+ fmt_str = btf_type_kflag(t) ? "\t%s val=%lld\n" : "\t%s val=%llu\n";
+ btf_verifier_log(env, fmt_str,
+ __btf_name_by_offset(btf, enums[i].name_off),
+ btf_enum64_value(enums + i));
+ }
+
+ return meta_needed;
+}
+
+static void btf_enum64_show(const struct btf *btf, const struct btf_type *t,
+ u32 type_id, void *data, u8 bits_offset,
+ struct btf_show *show)
+{
+ const struct btf_enum64 *enums = btf_type_enum64(t);
+ u32 i, nr_enums = btf_type_vlen(t);
+ void *safe_data;
+ s64 v;
+
+ safe_data = btf_show_start_type(show, t, type_id, data);
+ if (!safe_data)
+ return;
+
+ v = *(u64 *)safe_data;
+
+ for (i = 0; i < nr_enums; i++) {
+ if (v != btf_enum64_value(enums + i))
+ continue;
+
+ btf_show_type_value(show, "%s",
+ __btf_name_by_offset(btf,
+ enums[i].name_off));
+
+ btf_show_end_type(show);
+ return;
+ }
+
+ if (btf_type_kflag(t))
+ btf_show_type_value(show, "%lld", v);
+ else
+ btf_show_type_value(show, "%llu", v);
+ btf_show_end_type(show);
+}
+
+static const struct btf_kind_operations enum64_ops = {
+ .check_meta = btf_enum64_check_meta,
+ .resolve = btf_df_resolve,
+ .check_member = btf_enum_check_member,
+ .check_kflag_member = btf_enum_check_kflag_member,
+ .log_details = btf_enum_log,
+ .show = btf_enum64_show,
+};
+
+static s32 btf_func_proto_check_meta(struct btf_verifier_env *env,
+ const struct btf_type *t,
+ u32 meta_left)
+{
+ u32 meta_needed = btf_type_vlen(t) * sizeof(struct btf_param);
+
+ if (meta_left < meta_needed) {
+ btf_verifier_log_basic(env, t,
+ "meta_left:%u meta_needed:%u",
+ meta_left, meta_needed);
+ return -EINVAL;
+ }
+
+ if (t->name_off) {
+ btf_verifier_log_type(env, t, "Invalid name");
+ return -EINVAL;
+ }
+
+ if (btf_type_kflag(t)) {
+ btf_verifier_log_type(env, t, "Invalid btf_info kind_flag");
+ return -EINVAL;
+ }
+
+ btf_verifier_log_type(env, t, NULL);
+
+ return meta_needed;
+}
+
+static void btf_func_proto_log(struct btf_verifier_env *env,
+ const struct btf_type *t)
+{
+ const struct btf_param *args = (const struct btf_param *)(t + 1);
+ u16 nr_args = btf_type_vlen(t), i;
+
+ btf_verifier_log(env, "return=%u args=(", t->type);
+ if (!nr_args) {
+ btf_verifier_log(env, "void");
+ goto done;
+ }
+
+ if (nr_args == 1 && !args[0].type) {
+ /* Only one vararg */
+ btf_verifier_log(env, "vararg");
+ goto done;
+ }
+
+ btf_verifier_log(env, "%u %s", args[0].type,
+ __btf_name_by_offset(env->btf,
+ args[0].name_off));
+ for (i = 1; i < nr_args - 1; i++)
+ btf_verifier_log(env, ", %u %s", args[i].type,
+ __btf_name_by_offset(env->btf,
+ args[i].name_off));
+
+ if (nr_args > 1) {
+ const struct btf_param *last_arg = &args[nr_args - 1];
+
+ if (last_arg->type)
+ btf_verifier_log(env, ", %u %s", last_arg->type,
+ __btf_name_by_offset(env->btf,
+ last_arg->name_off));
+ else
+ btf_verifier_log(env, ", vararg");
+ }
+
+done:
+ btf_verifier_log(env, ")");
+}
+
+static const struct btf_kind_operations func_proto_ops = {
+ .check_meta = btf_func_proto_check_meta,
+ .resolve = btf_df_resolve,
+ /*
+ * BTF_KIND_FUNC_PROTO cannot be directly referred by
+ * a struct's member.
+ *
+ * It should be a function pointer instead.
+ * (i.e. struct's member -> BTF_KIND_PTR -> BTF_KIND_FUNC_PROTO)
+ *
+ * Hence, there is no btf_func_check_member().
+ */
+ .check_member = btf_df_check_member,
+ .check_kflag_member = btf_df_check_kflag_member,
+ .log_details = btf_func_proto_log,
+ .show = btf_df_show,
+};
+
+static s32 btf_func_check_meta(struct btf_verifier_env *env,
+ const struct btf_type *t,
+ u32 meta_left)
+{
+ if (!t->name_off ||
+ !btf_name_valid_identifier(env->btf, t->name_off)) {
+ btf_verifier_log_type(env, t, "Invalid name");
+ return -EINVAL;
+ }
+
+ if (btf_type_vlen(t) > BTF_FUNC_GLOBAL) {
+ btf_verifier_log_type(env, t, "Invalid func linkage");
+ return -EINVAL;
+ }
+
+ if (btf_type_kflag(t)) {
+ btf_verifier_log_type(env, t, "Invalid btf_info kind_flag");
+ return -EINVAL;
+ }
+
+ btf_verifier_log_type(env, t, NULL);
+
+ return 0;
+}
+
+static int btf_func_resolve(struct btf_verifier_env *env,
+ const struct resolve_vertex *v)
+{
+ const struct btf_type *t = v->t;
+ u32 next_type_id = t->type;
+ int err;
+
+ err = btf_func_check(env, t);
+ if (err)
+ return err;
+
+ env_stack_pop_resolved(env, next_type_id, 0);
+ return 0;
+}
+
+static const struct btf_kind_operations func_ops = {
+ .check_meta = btf_func_check_meta,
+ .resolve = btf_func_resolve,
+ .check_member = btf_df_check_member,
+ .check_kflag_member = btf_df_check_kflag_member,
+ .log_details = btf_ref_type_log,
+ .show = btf_df_show,
+};
+
+static s32 btf_var_check_meta(struct btf_verifier_env *env,
+ const struct btf_type *t,
+ u32 meta_left)
+{
+ const struct btf_var *var;
+ u32 meta_needed = sizeof(*var);
+
+ if (meta_left < meta_needed) {
+ btf_verifier_log_basic(env, t,
+ "meta_left:%u meta_needed:%u",
+ meta_left, meta_needed);
+ return -EINVAL;
+ }
+
+ if (btf_type_vlen(t)) {
+ btf_verifier_log_type(env, t, "vlen != 0");
+ return -EINVAL;
+ }
+
+ if (btf_type_kflag(t)) {
+ btf_verifier_log_type(env, t, "Invalid btf_info kind_flag");
+ return -EINVAL;
+ }
+
+ if (!t->name_off ||
+ !btf_name_valid_identifier(env->btf, t->name_off)) {
+ btf_verifier_log_type(env, t, "Invalid name");
+ return -EINVAL;
+ }
+
+ /* A var cannot be in type void */
+ if (!t->type || !BTF_TYPE_ID_VALID(t->type)) {
+ btf_verifier_log_type(env, t, "Invalid type_id");
+ return -EINVAL;
+ }
+
+ var = btf_type_var(t);
+ if (var->linkage != BTF_VAR_STATIC &&
+ var->linkage != BTF_VAR_GLOBAL_ALLOCATED) {
+ btf_verifier_log_type(env, t, "Linkage not supported");
+ return -EINVAL;
+ }
+
+ btf_verifier_log_type(env, t, NULL);
+
+ return meta_needed;
+}
+
+static void btf_var_log(struct btf_verifier_env *env, const struct btf_type *t)
+{
+ const struct btf_var *var = btf_type_var(t);
+
+ btf_verifier_log(env, "type_id=%u linkage=%u", t->type, var->linkage);
+}
+
+static const struct btf_kind_operations var_ops = {
+ .check_meta = btf_var_check_meta,
+ .resolve = btf_var_resolve,
+ .check_member = btf_df_check_member,
+ .check_kflag_member = btf_df_check_kflag_member,
+ .log_details = btf_var_log,
+ .show = btf_var_show,
+};
+
+static s32 btf_datasec_check_meta(struct btf_verifier_env *env,
+ const struct btf_type *t,
+ u32 meta_left)
+{
+ const struct btf_var_secinfo *vsi;
+ u64 last_vsi_end_off = 0, sum = 0;
+ u32 i, meta_needed;
+
+ meta_needed = btf_type_vlen(t) * sizeof(*vsi);
+ if (meta_left < meta_needed) {
+ btf_verifier_log_basic(env, t,
+ "meta_left:%u meta_needed:%u",
+ meta_left, meta_needed);
+ return -EINVAL;
+ }
+
+ if (!t->size) {
+ btf_verifier_log_type(env, t, "size == 0");
+ return -EINVAL;
+ }
+
+ if (btf_type_kflag(t)) {
+ btf_verifier_log_type(env, t, "Invalid btf_info kind_flag");
+ return -EINVAL;
+ }
+
+ if (!t->name_off ||
+ !btf_name_valid_section(env->btf, t->name_off)) {
+ btf_verifier_log_type(env, t, "Invalid name");
+ return -EINVAL;
+ }
+
+ btf_verifier_log_type(env, t, NULL);
+
+ for_each_vsi(i, t, vsi) {
+ /* A var cannot be in type void */
+ if (!vsi->type || !BTF_TYPE_ID_VALID(vsi->type)) {
+ btf_verifier_log_vsi(env, t, vsi,
+ "Invalid type_id");
+ return -EINVAL;
+ }
+
+ if (vsi->offset < last_vsi_end_off || vsi->offset >= t->size) {
+ btf_verifier_log_vsi(env, t, vsi,
+ "Invalid offset");
+ return -EINVAL;
+ }
+
+ if (!vsi->size || vsi->size > t->size) {
+ btf_verifier_log_vsi(env, t, vsi,
+ "Invalid size");
+ return -EINVAL;
+ }
+
+ last_vsi_end_off = vsi->offset + vsi->size;
+ if (last_vsi_end_off > t->size) {
+ btf_verifier_log_vsi(env, t, vsi,
+ "Invalid offset+size");
+ return -EINVAL;
+ }
+
+ btf_verifier_log_vsi(env, t, vsi, NULL);
+ sum += vsi->size;
+ }
+
+ if (t->size < sum) {
+ btf_verifier_log_type(env, t, "Invalid btf_info size");
+ return -EINVAL;
+ }
+
+ return meta_needed;
+}
+
+static int btf_datasec_resolve(struct btf_verifier_env *env,
+ const struct resolve_vertex *v)
+{
+ const struct btf_var_secinfo *vsi;
+ struct btf *btf = env->btf;
+ u16 i;
+
+ env->resolve_mode = RESOLVE_TBD;
+ for_each_vsi_from(i, v->next_member, v->t, vsi) {
+ u32 var_type_id = vsi->type, type_id, type_size = 0;
+ const struct btf_type *var_type = btf_type_by_id(env->btf,
+ var_type_id);
+ if (!var_type || !btf_type_is_var(var_type)) {
+ btf_verifier_log_vsi(env, v->t, vsi,
+ "Not a VAR kind member");
+ return -EINVAL;
+ }
+
+ if (!env_type_is_resolve_sink(env, var_type) &&
+ !env_type_is_resolved(env, var_type_id)) {
+ env_stack_set_next_member(env, i + 1);
+ return env_stack_push(env, var_type, var_type_id);
+ }
+
+ type_id = var_type->type;
+ if (!btf_type_id_size(btf, &type_id, &type_size)) {
+ btf_verifier_log_vsi(env, v->t, vsi, "Invalid type");
+ return -EINVAL;
+ }
+
+ if (vsi->size < type_size) {
+ btf_verifier_log_vsi(env, v->t, vsi, "Invalid size");
+ return -EINVAL;
+ }
+ }
+
+ env_stack_pop_resolved(env, 0, 0);
+ return 0;
+}
+
+static void btf_datasec_log(struct btf_verifier_env *env,
+ const struct btf_type *t)
+{
+ btf_verifier_log(env, "size=%u vlen=%u", t->size, btf_type_vlen(t));
+}
+
+static void btf_datasec_show(const struct btf *btf,
+ const struct btf_type *t, u32 type_id,
+ void *data, u8 bits_offset,
+ struct btf_show *show)
+{
+ const struct btf_var_secinfo *vsi;
+ const struct btf_type *var;
+ u32 i;
+
+ if (!btf_show_start_type(show, t, type_id, data))
+ return;
+
+ btf_show_type_value(show, "section (\"%s\") = {",
+ __btf_name_by_offset(btf, t->name_off));
+ for_each_vsi(i, t, vsi) {
+ var = btf_type_by_id(btf, vsi->type);
+ if (i)
+ btf_show(show, ",");
+ btf_type_ops(var)->show(btf, var, vsi->type,
+ data + vsi->offset, bits_offset, show);
+ }
+ btf_show_end_type(show);
+}
+
+static const struct btf_kind_operations datasec_ops = {
+ .check_meta = btf_datasec_check_meta,
+ .resolve = btf_datasec_resolve,
+ .check_member = btf_df_check_member,
+ .check_kflag_member = btf_df_check_kflag_member,
+ .log_details = btf_datasec_log,
+ .show = btf_datasec_show,
+};
+
+static s32 btf_float_check_meta(struct btf_verifier_env *env,
+ const struct btf_type *t,
+ u32 meta_left)
+{
+ if (btf_type_vlen(t)) {
+ btf_verifier_log_type(env, t, "vlen != 0");
+ return -EINVAL;
+ }
+
+ if (btf_type_kflag(t)) {
+ btf_verifier_log_type(env, t, "Invalid btf_info kind_flag");
+ return -EINVAL;
+ }
+
+ if (t->size != 2 && t->size != 4 && t->size != 8 && t->size != 12 &&
+ t->size != 16) {
+ btf_verifier_log_type(env, t, "Invalid type_size");
+ return -EINVAL;
+ }
+
+ btf_verifier_log_type(env, t, NULL);
+
+ return 0;
+}
+
+static int btf_float_check_member(struct btf_verifier_env *env,
+ const struct btf_type *struct_type,
+ const struct btf_member *member,
+ const struct btf_type *member_type)
+{
+ u64 start_offset_bytes;
+ u64 end_offset_bytes;
+ u64 misalign_bits;
+ u64 align_bytes;
+ u64 align_bits;
+
+ /* Different architectures have different alignment requirements, so
+ * here we check only for the reasonable minimum. This way we ensure
+ * that types after CO-RE can pass the kernel BTF verifier.
+ */
+ align_bytes = min_t(u64, sizeof(void *), member_type->size);
+ align_bits = align_bytes * BITS_PER_BYTE;
+ div64_u64_rem(member->offset, align_bits, &misalign_bits);
+ if (misalign_bits) {
+ btf_verifier_log_member(env, struct_type, member,
+ "Member is not properly aligned");
+ return -EINVAL;
+ }
+
+ start_offset_bytes = member->offset / BITS_PER_BYTE;
+ end_offset_bytes = start_offset_bytes + member_type->size;
+ if (end_offset_bytes > struct_type->size) {
+ btf_verifier_log_member(env, struct_type, member,
+ "Member exceeds struct_size");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static void btf_float_log(struct btf_verifier_env *env,
+ const struct btf_type *t)
+{
+ btf_verifier_log(env, "size=%u", t->size);
+}
+
+static const struct btf_kind_operations float_ops = {
+ .check_meta = btf_float_check_meta,
+ .resolve = btf_df_resolve,
+ .check_member = btf_float_check_member,
+ .check_kflag_member = btf_generic_check_kflag_member,
+ .log_details = btf_float_log,
+ .show = btf_df_show,
+};
+
+static s32 btf_decl_tag_check_meta(struct btf_verifier_env *env,
+ const struct btf_type *t,
+ u32 meta_left)
+{
+ const struct btf_decl_tag *tag;
+ u32 meta_needed = sizeof(*tag);
+ s32 component_idx;
+ const char *value;
+
+ if (meta_left < meta_needed) {
+ btf_verifier_log_basic(env, t,
+ "meta_left:%u meta_needed:%u",
+ meta_left, meta_needed);
+ return -EINVAL;
+ }
+
+ value = btf_name_by_offset(env->btf, t->name_off);
+ if (!value || !value[0]) {
+ btf_verifier_log_type(env, t, "Invalid value");
+ return -EINVAL;
+ }
+
+ if (btf_type_vlen(t)) {
+ btf_verifier_log_type(env, t, "vlen != 0");
+ return -EINVAL;
+ }
+
+ component_idx = btf_type_decl_tag(t)->component_idx;
+ if (component_idx < -1) {
+ btf_verifier_log_type(env, t, "Invalid component_idx");
+ return -EINVAL;
+ }
+
+ btf_verifier_log_type(env, t, NULL);
+
+ return meta_needed;
+}
+
+static int btf_decl_tag_resolve(struct btf_verifier_env *env,
+ const struct resolve_vertex *v)
+{
+ const struct btf_type *next_type;
+ const struct btf_type *t = v->t;
+ u32 next_type_id = t->type;
+ struct btf *btf = env->btf;
+ s32 component_idx;
+ u32 vlen;
+
+ next_type = btf_type_by_id(btf, next_type_id);
+ if (!next_type || !btf_type_is_decl_tag_target(next_type)) {
+ btf_verifier_log_type(env, v->t, "Invalid type_id");
+ return -EINVAL;
+ }
+
+ if (!env_type_is_resolve_sink(env, next_type) &&
+ !env_type_is_resolved(env, next_type_id))
+ return env_stack_push(env, next_type, next_type_id);
+
+ component_idx = btf_type_decl_tag(t)->component_idx;
+ if (component_idx != -1) {
+ if (btf_type_is_var(next_type) || btf_type_is_typedef(next_type)) {
+ btf_verifier_log_type(env, v->t, "Invalid component_idx");
+ return -EINVAL;
+ }
+
+ if (btf_type_is_struct(next_type)) {
+ vlen = btf_type_vlen(next_type);
+ } else {
+ /* next_type should be a function */
+ next_type = btf_type_by_id(btf, next_type->type);
+ vlen = btf_type_vlen(next_type);
+ }
+
+ if ((u32)component_idx >= vlen) {
+ btf_verifier_log_type(env, v->t, "Invalid component_idx");
+ return -EINVAL;
+ }
+ }
+
+ env_stack_pop_resolved(env, next_type_id, 0);
+
+ return 0;
+}
+
+static void btf_decl_tag_log(struct btf_verifier_env *env, const struct btf_type *t)
+{
+ btf_verifier_log(env, "type=%u component_idx=%d", t->type,
+ btf_type_decl_tag(t)->component_idx);
+}
+
+static const struct btf_kind_operations decl_tag_ops = {
+ .check_meta = btf_decl_tag_check_meta,
+ .resolve = btf_decl_tag_resolve,
+ .check_member = btf_df_check_member,
+ .check_kflag_member = btf_df_check_kflag_member,
+ .log_details = btf_decl_tag_log,
+ .show = btf_df_show,
+};
+
+static int btf_func_proto_check(struct btf_verifier_env *env,
+ const struct btf_type *t)
+{
+ const struct btf_type *ret_type;
+ const struct btf_param *args;
+ const struct btf *btf;
+ u16 nr_args, i;
+ int err;
+
+ btf = env->btf;
+ args = (const struct btf_param *)(t + 1);
+ nr_args = btf_type_vlen(t);
+
+ /* Check func return type which could be "void" (t->type == 0) */
+ if (t->type) {
+ u32 ret_type_id = t->type;
+
+ ret_type = btf_type_by_id(btf, ret_type_id);
+ if (!ret_type) {
+ btf_verifier_log_type(env, t, "Invalid return type");
+ return -EINVAL;
+ }
+
+ if (btf_type_is_resolve_source_only(ret_type)) {
+ btf_verifier_log_type(env, t, "Invalid return type");
+ return -EINVAL;
+ }
+
+ if (btf_type_needs_resolve(ret_type) &&
+ !env_type_is_resolved(env, ret_type_id)) {
+ err = btf_resolve(env, ret_type, ret_type_id);
+ if (err)
+ return err;
+ }
+
+ /* Ensure the return type is a type that has a size */
+ if (!btf_type_id_size(btf, &ret_type_id, NULL)) {
+ btf_verifier_log_type(env, t, "Invalid return type");
+ return -EINVAL;
+ }
+ }
+
+ if (!nr_args)
+ return 0;
+
+ /* Last func arg type_id could be 0 if it is a vararg */
+ if (!args[nr_args - 1].type) {
+ if (args[nr_args - 1].name_off) {
+ btf_verifier_log_type(env, t, "Invalid arg#%u",
+ nr_args);
+ return -EINVAL;
+ }
+ nr_args--;
+ }
+
+ for (i = 0; i < nr_args; i++) {
+ const struct btf_type *arg_type;
+ u32 arg_type_id;
+
+ arg_type_id = args[i].type;
+ arg_type = btf_type_by_id(btf, arg_type_id);
+ if (!arg_type) {
+ btf_verifier_log_type(env, t, "Invalid arg#%u", i + 1);
+ return -EINVAL;
+ }
+
+ if (btf_type_is_resolve_source_only(arg_type)) {
+ btf_verifier_log_type(env, t, "Invalid arg#%u", i + 1);
+ return -EINVAL;
+ }
+
+ if (args[i].name_off &&
+ (!btf_name_offset_valid(btf, args[i].name_off) ||
+ !btf_name_valid_identifier(btf, args[i].name_off))) {
+ btf_verifier_log_type(env, t,
+ "Invalid arg#%u", i + 1);
+ return -EINVAL;
+ }
+
+ if (btf_type_needs_resolve(arg_type) &&
+ !env_type_is_resolved(env, arg_type_id)) {
+ err = btf_resolve(env, arg_type, arg_type_id);
+ if (err)
+ return err;
+ }
+
+ if (!btf_type_id_size(btf, &arg_type_id, NULL)) {
+ btf_verifier_log_type(env, t, "Invalid arg#%u", i + 1);
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
+static int btf_func_check(struct btf_verifier_env *env,
+ const struct btf_type *t)
+{
+ const struct btf_type *proto_type;
+ const struct btf_param *args;
+ const struct btf *btf;
+ u16 nr_args, i;
+
+ btf = env->btf;
+ proto_type = btf_type_by_id(btf, t->type);
+
+ if (!proto_type || !btf_type_is_func_proto(proto_type)) {
+ btf_verifier_log_type(env, t, "Invalid type_id");
+ return -EINVAL;
+ }
+
+ args = (const struct btf_param *)(proto_type + 1);
+ nr_args = btf_type_vlen(proto_type);
+ for (i = 0; i < nr_args; i++) {
+ if (!args[i].name_off && args[i].type) {
+ btf_verifier_log_type(env, t, "Invalid arg#%u", i + 1);
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
+static const struct btf_kind_operations * const kind_ops[NR_BTF_KINDS] = {
+ [BTF_KIND_INT] = &int_ops,
+ [BTF_KIND_PTR] = &ptr_ops,
+ [BTF_KIND_ARRAY] = &array_ops,
+ [BTF_KIND_STRUCT] = &struct_ops,
+ [BTF_KIND_UNION] = &struct_ops,
+ [BTF_KIND_ENUM] = &enum_ops,
+ [BTF_KIND_FWD] = &fwd_ops,
+ [BTF_KIND_TYPEDEF] = &modifier_ops,
+ [BTF_KIND_VOLATILE] = &modifier_ops,
+ [BTF_KIND_CONST] = &modifier_ops,
+ [BTF_KIND_RESTRICT] = &modifier_ops,
+ [BTF_KIND_FUNC] = &func_ops,
+ [BTF_KIND_FUNC_PROTO] = &func_proto_ops,
+ [BTF_KIND_VAR] = &var_ops,
+ [BTF_KIND_DATASEC] = &datasec_ops,
+ [BTF_KIND_FLOAT] = &float_ops,
+ [BTF_KIND_DECL_TAG] = &decl_tag_ops,
+ [BTF_KIND_TYPE_TAG] = &modifier_ops,
+ [BTF_KIND_ENUM64] = &enum64_ops,
+};
+
+static s32 btf_check_meta(struct btf_verifier_env *env,
+ const struct btf_type *t,
+ u32 meta_left)
+{
+ u32 saved_meta_left = meta_left;
+ s32 var_meta_size;
+
+ if (meta_left < sizeof(*t)) {
+ btf_verifier_log(env, "[%u] meta_left:%u meta_needed:%zu",
+ env->log_type_id, meta_left, sizeof(*t));
+ return -EINVAL;
+ }
+ meta_left -= sizeof(*t);
+
+ if (t->info & ~BTF_INFO_MASK) {
+ btf_verifier_log(env, "[%u] Invalid btf_info:%x",
+ env->log_type_id, t->info);
+ return -EINVAL;
+ }
+
+ if (BTF_INFO_KIND(t->info) > BTF_KIND_MAX ||
+ BTF_INFO_KIND(t->info) == BTF_KIND_UNKN) {
+ btf_verifier_log(env, "[%u] Invalid kind:%u",
+ env->log_type_id, BTF_INFO_KIND(t->info));
+ return -EINVAL;
+ }
+
+ if (!btf_name_offset_valid(env->btf, t->name_off)) {
+ btf_verifier_log(env, "[%u] Invalid name_offset:%u",
+ env->log_type_id, t->name_off);
+ return -EINVAL;
+ }
+
+ var_meta_size = btf_type_ops(t)->check_meta(env, t, meta_left);
+ if (var_meta_size < 0)
+ return var_meta_size;
+
+ meta_left -= var_meta_size;
+
+ return saved_meta_left - meta_left;
+}
+
+int btf_check_all_metas(struct btf_verifier_env *env)
+{
+ struct btf *btf = env->btf;
+ struct btf_header *hdr;
+ void *cur, *end;
+
+ hdr = &btf->hdr;
+ cur = btf->nohdr_data + hdr->type_off;
+ end = cur + hdr->type_len;
+
+ env->log_type_id = btf->base_btf ? btf->start_id : 1;
+ while (cur < end) {
+ struct btf_type *t = cur;
+ s32 meta_size;
+
+ meta_size = btf_check_meta(env, t, end - cur);
+ if (meta_size < 0)
+ return meta_size;
+
+ btf_add_type(env, t);
+ cur += meta_size;
+ env->log_type_id++;
+ }
+
+ return 0;
+}
+
+static bool btf_resolve_valid(struct btf_verifier_env *env,
+ const struct btf_type *t,
+ u32 type_id)
+{
+ struct btf *btf = env->btf;
+
+ if (!env_type_is_resolved(env, type_id))
+ return false;
+
+ if (btf_type_is_struct(t) || btf_type_is_datasec(t))
+ return !btf_resolved_type_id(btf, type_id) &&
+ !btf_resolved_type_size(btf, type_id);
+
+ if (btf_type_is_decl_tag(t) || btf_type_is_func(t))
+ return btf_resolved_type_id(btf, type_id) &&
+ !btf_resolved_type_size(btf, type_id);
+
+ if (btf_type_is_modifier(t) || btf_type_is_ptr(t) ||
+ btf_type_is_var(t)) {
+ t = btf_type_id_resolve(btf, &type_id);
+ return t &&
+ !btf_type_is_modifier(t) &&
+ !btf_type_is_var(t) &&
+ !btf_type_is_datasec(t);
+ }
+
+ if (btf_type_is_array(t)) {
+ const struct btf_array *array = btf_type_array(t);
+ const struct btf_type *elem_type;
+ u32 elem_type_id = array->type;
+ u32 elem_size;
+
+ elem_type = btf_type_id_size(btf, &elem_type_id, &elem_size);
+ return elem_type && !btf_type_is_modifier(elem_type) &&
+ (array->nelems * elem_size ==
+ btf_resolved_type_size(btf, type_id));
+ }
+
+ return false;
+}
+
+static int btf_resolve(struct btf_verifier_env *env,
+ const struct btf_type *t, u32 type_id)
+{
+ u32 save_log_type_id = env->log_type_id;
+ const struct resolve_vertex *v;
+ int err = 0;
+
+ env->resolve_mode = RESOLVE_TBD;
+ env_stack_push(env, t, type_id);
+ while (!err && (v = env_stack_peak(env))) {
+ env->log_type_id = v->type_id;
+ err = btf_type_ops(v->t)->resolve(env, v);
+ }
+
+ env->log_type_id = type_id;
+ if (err == -E2BIG) {
+ btf_verifier_log_type(env, t,
+ "Exceeded max resolving depth:%u",
+ MAX_RESOLVE_DEPTH);
+ } else if (err == -EEXIST) {
+ btf_verifier_log_type(env, t, "Loop detected");
+ }
+
+ /* Final sanity check */
+ if (!err && !btf_resolve_valid(env, t, type_id)) {
+ btf_verifier_log_type(env, t, "Invalid resolve state");
+ err = -EINVAL;
+ }
+
+ env->log_type_id = save_log_type_id;
+ return err;
+}
+
+static int btf_check_all_types(struct btf_verifier_env *env)
+{
+ struct btf *btf = env->btf;
+ const struct btf_type *t;
+ u32 type_id, i;
+ int err;
+
+ err = env_resolve_init(env);
+ if (err)
+ return err;
+
+ env->phase++;
+ for (i = btf->base_btf ? 0 : 1; i < btf->nr_types; i++) {
+ type_id = btf->start_id + i;
+ t = btf_type_by_id(btf, type_id);
+
+ env->log_type_id = type_id;
+ if (btf_type_needs_resolve(t) &&
+ !env_type_is_resolved(env, type_id)) {
+ err = btf_resolve(env, t, type_id);
+ if (err)
+ return err;
+ }
+
+ if (btf_type_is_func_proto(t)) {
+ err = btf_func_proto_check(env, t);
+ if (err)
+ return err;
+ }
+ }
+
+ return 0;
+}
+
+int btf_parse_type_sec(struct btf_verifier_env *env)
+{
+ const struct btf_header *hdr = &env->btf->hdr;
+ int err;
+
+ /* Type section must align to 4 bytes */
+ if (hdr->type_off & (sizeof(u32) - 1)) {
+ btf_verifier_log(env, "Unaligned type_off");
+ return -EINVAL;
+ }
+
+ if (!env->btf->base_btf && !hdr->type_len) {
+ btf_verifier_log(env, "No type found");
+ return -EINVAL;
+ }
+
+ err = btf_check_all_metas(env);
+ if (err)
+ return err;
+
+ return btf_check_all_types(env);
+}
+
+int btf_parse_str_sec(struct btf_verifier_env *env)
+{
+ const struct btf_header *hdr;
+ struct btf *btf = env->btf;
+ const char *start, *end;
+
+ hdr = &btf->hdr;
+ start = btf->nohdr_data + hdr->str_off;
+ end = start + hdr->str_len;
+
+ if (end != btf->data + btf->data_size) {
+ btf_verifier_log(env, "String section is not at the end");
+ return -EINVAL;
+ }
+
+ btf->strings = start;
+
+ if (btf->base_btf && !hdr->str_len)
+ return 0;
+ if (!hdr->str_len || hdr->str_len - 1 > BTF_MAX_NAME_OFFSET || end[-1]) {
+ btf_verifier_log(env, "Invalid string section");
+ return -EINVAL;
+ }
+ if (!btf->base_btf && start[0]) {
+ btf_verifier_log(env, "Invalid string section");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static const size_t btf_sec_info_offset[] = {
+ offsetof(struct btf_header, type_off),
+ offsetof(struct btf_header, str_off),
+};
+
+static int btf_sec_info_cmp(const void *a, const void *b)
+{
+ const struct btf_sec_info *x = a;
+ const struct btf_sec_info *y = b;
+
+ return (int)(x->off - y->off) ? : (int)(x->len - y->len);
+}
+
+static int btf_check_sec_info(struct btf_verifier_env *env,
+ u32 btf_data_size)
+{
+ struct btf_sec_info secs[ARRAY_SIZE(btf_sec_info_offset)];
+ u32 total, expected_total, i;
+ const struct btf_header *hdr;
+ const struct btf *btf;
+
+ btf = env->btf;
+ hdr = &btf->hdr;
+
+ /* Populate the secs from hdr */
+ for (i = 0; i < ARRAY_SIZE(btf_sec_info_offset); i++)
+ secs[i] = *(struct btf_sec_info *)((void *)hdr +
+ btf_sec_info_offset[i]);
+
+ sort(secs, ARRAY_SIZE(btf_sec_info_offset),
+ sizeof(struct btf_sec_info), btf_sec_info_cmp, NULL);
+
+ /* Check for gaps and overlap among sections */
+ total = 0;
+ expected_total = btf_data_size - hdr->hdr_len;
+ for (i = 0; i < ARRAY_SIZE(btf_sec_info_offset); i++) {
+ if (expected_total < secs[i].off) {
+ btf_verifier_log(env, "Invalid section offset");
+ return -EINVAL;
+ }
+ if (total < secs[i].off) {
+ /* gap */
+ btf_verifier_log(env, "Unsupported section found");
+ return -EINVAL;
+ }
+ if (total > secs[i].off) {
+ btf_verifier_log(env, "Section overlap found");
+ return -EINVAL;
+ }
+ if (expected_total - total < secs[i].len) {
+ btf_verifier_log(env,
+ "Total section length too long");
+ return -EINVAL;
+ }
+ total += secs[i].len;
+ }
+
+ /* There is data other than hdr and known sections */
+ if (expected_total != total) {
+ btf_verifier_log(env, "Unsupported section found");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+int btf_parse_hdr(struct btf_verifier_env *env)
+{
+ u32 hdr_len, hdr_copy, btf_data_size;
+ const struct btf_header *hdr;
+ struct btf *btf;
+
+ btf = env->btf;
+ btf_data_size = btf->data_size;
+
+ if (btf_data_size < offsetofend(struct btf_header, hdr_len)) {
+ btf_verifier_log(env, "hdr_len not found");
+ return -EINVAL;
+ }
+
+ hdr = btf->data;
+ hdr_len = hdr->hdr_len;
+ if (btf_data_size < hdr_len) {
+ btf_verifier_log(env, "btf_header not found");
+ return -EINVAL;
+ }
+
+ /* Ensure the unsupported header fields are zero */
+ if (hdr_len > sizeof(btf->hdr)) {
+ u8 *expected_zero = btf->data + sizeof(btf->hdr);
+ u8 *end = btf->data + hdr_len;
+
+ for (; expected_zero < end; expected_zero++) {
+ if (*expected_zero) {
+ btf_verifier_log(env, "Unsupported btf_header");
+ return -E2BIG;
+ }
+ }
+ }
+
+ hdr_copy = min_t(u32, hdr_len, sizeof(btf->hdr));
+ memcpy(&btf->hdr, btf->data, hdr_copy);
+
+ hdr = &btf->hdr;
+
+ btf_verifier_log_hdr(env, btf_data_size);
+
+ if (hdr->magic != BTF_MAGIC) {
+ btf_verifier_log(env, "Invalid magic");
+ return -EINVAL;
+ }
+
+ if (hdr->version != BTF_VERSION) {
+ btf_verifier_log(env, "Unsupported version");
+ return -ENOTSUPP;
+ }
+
+ if (hdr->flags) {
+ btf_verifier_log(env, "Unsupported flags");
+ return -ENOTSUPP;
+ }
+
+ if (!btf->base_btf && btf_data_size == hdr->hdr_len) {
+ btf_verifier_log(env, "No data");
+ return -EINVAL;
+ }
+
+ return btf_check_sec_info(env, btf_data_size);
+}
+
+
+int btf_check_type_tags(struct btf_verifier_env *env,
+ struct btf *btf, int start_id)
+{
+ int i, n, good_id = start_id - 1;
+ bool in_tags;
+
+ n = btf_nr_types(btf);
+ for (i = start_id; i < n; i++) {
+ const struct btf_type *t;
+ int chain_limit = 32;
+ u32 cur_id = i;
+
+ t = btf_type_by_id(btf, i);
+ if (!t)
+ return -EINVAL;
+ if (!btf_type_is_modifier(t))
+ continue;
+
+ cond_resched();
+
+ in_tags = btf_type_is_type_tag(t);
+ while (btf_type_is_modifier(t)) {
+ if (!chain_limit--) {
+ btf_verifier_log(env, "Max chain length or cycle detected");
+ return -ELOOP;
+ }
+ if (btf_type_is_type_tag(t)) {
+ if (!in_tags) {
+ btf_verifier_log(env, "Type tags don't precede modifiers");
+ return -EINVAL;
+ }
+ } else if (in_tags) {
+ in_tags = false;
+ }
+ if (cur_id <= good_id)
+ break;
+ /* Move to next type */
+ cur_id = t->type;
+ t = btf_type_by_id(btf, cur_id);
+ if (!t)
+ return -EINVAL;
+ }
+ good_id = i;
+ }
+ return 0;
+}
+
+
+struct btf *
+btf_parse_base(struct btf_verifier_env *env, const char *name,
+ void *data, unsigned int data_size)
+{
+ struct btf *btf = NULL;
+ int err;
+
+ if (!IS_ENABLED(CONFIG_DEBUG_INFO_BTF))
+ return ERR_PTR(-ENOENT);
+
+ btf = kzalloc_obj(*btf, GFP_KERNEL | __GFP_NOWARN);
+ if (!btf) {
+ err = -ENOMEM;
+ goto errout;
+ }
+ env->btf = btf;
+
+ btf->data = data;
+ btf->data_size = data_size;
+ btf->kernel_btf = true;
+ btf->named_start_id = 0;
+ strscpy(btf->name, name);
+
+ err = btf_parse_hdr(env);
+ if (err)
+ goto errout;
+
+ btf->nohdr_data = btf->data + btf->hdr.hdr_len;
+
+ err = btf_parse_str_sec(env);
+ if (err)
+ goto errout;
+
+ err = btf_check_all_metas(env);
+ if (err)
+ goto errout;
+
+ err = btf_check_type_tags(env, btf, 1);
+ if (err)
+ goto errout;
+
+ btf_check_sorted(btf);
+ refcount_set(&btf->refcnt, 1);
+
+ return btf;
+
+errout:
+ if (btf) {
+ kvfree(btf->types);
+ kfree(btf);
+ }
+ return ERR_PTR(err);
+}
+
+
+static void btf_type_show(const struct btf *btf, u32 type_id, void *obj,
+ struct btf_show *show)
+{
+ const struct btf_type *t = btf_type_by_id(btf, type_id);
+
+ show->btf = btf;
+ memset(&show->state, 0, sizeof(show->state));
+ memset(&show->obj, 0, sizeof(show->obj));
+
+ btf_type_ops(t)->show(btf, t, type_id, obj, 0, show);
+}
+
+__printf(2, 0) static void btf_seq_show(struct btf_show *show, const char *fmt,
+ va_list args)
+{
+ seq_vprintf((struct seq_file *)show->target, fmt, args);
+}
+
+int btf_type_seq_show_flags(const struct btf *btf, u32 type_id,
+ void *obj, struct seq_file *m, u64 flags)
+{
+ struct btf_show sseq;
+
+ sseq.target = m;
+ sseq.showfn = btf_seq_show;
+ sseq.flags = flags;
+
+ btf_type_show(btf, type_id, obj, &sseq);
+
+ return sseq.state.status;
+}
+
+void btf_type_seq_show(const struct btf *btf, u32 type_id, void *obj,
+ struct seq_file *m)
+{
+ (void) btf_type_seq_show_flags(btf, type_id, obj, m,
+ BTF_SHOW_NONAME | BTF_SHOW_COMPACT |
+ BTF_SHOW_ZERO | BTF_SHOW_UNSAFE);
+}
+
+struct btf_show_snprintf {
+ struct btf_show show;
+ int len_left; /* space left in string */
+ int len; /* length we would have written */
+};
+
+__printf(2, 0) static void btf_snprintf_show(struct btf_show *show, const char *fmt,
+ va_list args)
+{
+ struct btf_show_snprintf *ssnprintf = (struct btf_show_snprintf *)show;
+ int len;
+
+ len = vsnprintf(show->target, ssnprintf->len_left, fmt, args);
+
+ if (len < 0) {
+ ssnprintf->len_left = 0;
+ ssnprintf->len = len;
+ } else if (len >= ssnprintf->len_left) {
+ /* no space, drive on to get length we would have written */
+ ssnprintf->len_left = 0;
+ ssnprintf->len += len;
+ } else {
+ ssnprintf->len_left -= len;
+ ssnprintf->len += len;
+ show->target += len;
+ }
+}
+
+int btf_type_snprintf_show(const struct btf *btf, u32 type_id, void *obj,
+ char *buf, int len, u64 flags)
+{
+ struct btf_show_snprintf ssnprintf;
+
+ ssnprintf.show.target = buf;
+ ssnprintf.show.flags = flags;
+ ssnprintf.show.showfn = btf_snprintf_show;
+ ssnprintf.len_left = len;
+ ssnprintf.len = 0;
+
+ btf_type_show(btf, type_id, obj, (struct btf_show *)&ssnprintf);
+
+ /* If we encountered an error, return it. */
+ if (ssnprintf.show.state.status)
+ return ssnprintf.show.state.status;
+
+ /* Otherwise return length we would have written */
+ return ssnprintf.len;
+}
+
+
+bool btf_is_kernel(const struct btf *btf)
+{
+ return btf->kernel_btf;
+}
+
+bool btf_is_module(const struct btf *btf)
+{
+ return btf->kernel_btf && strcmp(btf->name, "vmlinux") != 0;
+}
+
+
+bool btf_param_match_suffix(const struct btf *btf,
+ const struct btf_param *arg,
+ const char *suffix)
+{
+ int suffix_len = strlen(suffix), len;
+ const char *param_name;
+
+ /* In the future, this can be ported to use BTF tagging */
+ param_name = btf_name_by_offset(btf, arg->name_off);
+ if (str_is_empty(param_name))
+ return false;
+ len = strlen(param_name);
+ if (len <= suffix_len)
+ return false;
+ param_name += len - suffix_len;
+ return !strncmp(param_name, suffix, suffix_len);
+}
diff --git a/kernel/btf/btf.h b/kernel/btf/btf.h
new file mode 100644
index 0000000000000..f66d9300adb31
--- /dev/null
+++ b/kernel/btf/btf.h
@@ -0,0 +1,135 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright (c) 2018 Facebook */
+/* Internal BTF header for kernel/btf/ */
+
+#ifndef _KERNEL_BTF_BTF_H
+#define _KERNEL_BTF_BTF_H
+
+#include <linux/types.h>
+#include <linux/btf.h>
+#include <linux/bpf_verifier.h>
+#include <linux/module.h>
+#include <linux/rculist.h>
+
+/* Forward declarations */
+struct btf_show;
+struct btf_struct_metas;
+
+/* btf_verifier_env and its dependencies (needed by both btf.c and bpf.c) */
+enum verifier_phase {
+ CHECK_META,
+ CHECK_TYPE,
+};
+
+struct resolve_vertex {
+ const struct btf_type *t;
+ u32 type_id;
+ u16 next_member;
+};
+
+enum resolve_mode {
+ RESOLVE_TBD, /* To Be Determined */
+ RESOLVE_PTR, /* Resolving for Pointer */
+ RESOLVE_STRUCT_OR_ARRAY, /* Resolving for struct/union
+ * or array
+ */
+};
+
+#define MAX_RESOLVE_DEPTH 32
+
+struct btf_verifier_env {
+ struct btf *btf;
+ u8 *visit_states;
+ struct resolve_vertex stack[MAX_RESOLVE_DEPTH];
+ struct bpf_verifier_log log;
+ u32 log_type_id;
+ u32 top_stack;
+ enum verifier_phase phase;
+ enum resolve_mode resolve_mode;
+};
+
+#ifdef CONFIG_BPF_SYSCALL
+struct btf_kfunc_set_tab;
+struct btf_id_dtor_kfunc_tab;
+struct btf_struct_ops_tab;
+#endif
+
+struct btf {
+ void *data;
+ struct btf_type **types;
+ u32 *resolved_ids;
+ u32 *resolved_sizes;
+ const char *strings;
+ void *nohdr_data;
+ struct btf_header hdr;
+ u32 nr_types; /* includes VOID for base BTF */
+ u32 named_start_id;
+ u32 types_size;
+ u32 data_size;
+ refcount_t refcnt;
+#ifdef CONFIG_BPF_SYSCALL
+ u32 id;
+ struct rcu_head rcu;
+ struct btf_kfunc_set_tab *kfunc_set_tab;
+ struct btf_id_dtor_kfunc_tab *dtor_kfunc_tab;
+ struct btf_struct_metas *struct_meta_tab;
+ struct btf_struct_ops_tab *struct_ops_tab;
+#endif
+
+ /* split BTF support */
+ struct btf *base_btf;
+ u32 start_id; /* first type ID in this BTF (0 for base BTF) */
+ u32 start_str_off; /* first string offset (0 for base BTF) */
+ char name[MODULE_NAME_LEN];
+ bool kernel_btf;
+ __u32 *base_id_map; /* map from distilled base BTF -> vmlinux BTF ids */
+};
+
+#define BITS_PER_BYTE_MASK (BITS_PER_BYTE - 1)
+#define BITS_PER_BYTE_MASKED(bits) ((bits) & BITS_PER_BYTE_MASK)
+#define BITS_ROUNDDOWN_BYTES(bits) ((bits) >> 3)
+#define BITS_ROUNDUP_BYTES(bits) \
+ (BITS_ROUNDDOWN_BYTES(bits) + !!BITS_PER_BYTE_MASKED(bits))
+
+/* 16MB for 64k structs and each has 16 members and
+ * a few MB spaces for the string section.
+ * The hard limit is S32_MAX.
+ */
+#define BTF_MAX_SIZE (16 * 1024 * 1024)
+
+/*
+ * The suffix of a type that indicates it cannot alias another type when
+ * comparing BTF IDs for kfunc invocations.
+ */
+#define NOCAST_ALIAS_SUFFIX "___init"
+
+/* Core BTF functions needed by bpf.c */
+const char *__btf_name_by_offset(const struct btf *btf, u32 offset);
+const struct btf_type *
+__btf_resolve_size(const struct btf *btf, const struct btf_type *type,
+ u32 *type_size, const struct btf_type **elem_type,
+ u32 *elem_id, u32 *total_nelems, u32 *type_id);
+int btf_check_all_metas(struct btf_verifier_env *env);
+void btf_check_sorted(struct btf *btf);
+int btf_check_type_tags(struct btf_verifier_env *env,
+ struct btf *btf, int start_id);
+void btf_free(struct btf *btf);
+struct btf *btf_parse_base(struct btf_verifier_env *env, const char *name,
+ void *data, unsigned int data_size);
+int btf_parse_hdr(struct btf_verifier_env *env);
+int btf_parse_str_sec(struct btf_verifier_env *env);
+int btf_parse_type_sec(struct btf_verifier_env *env);
+const struct btf_decl_tag *btf_type_decl_tag(const struct btf_type *t);
+bool btf_type_has_size(const struct btf_type *t);
+bool btf_type_is_datasec(const struct btf_type *t);
+bool btf_type_is_decl_tag(const struct btf_type *t);
+bool btf_type_is_modifier(const struct btf_type *t);
+void btf_verifier_env_free(struct btf_verifier_env *env);
+
+extern const char * const btf_kind_str[NR_BTF_KINDS];
+
+/* Weak symbols - overridden by bpf.c when CONFIG_BPF_SYSCALL=y */
+void btf_free_bpf_data(struct btf *btf);
+void btf_put_bpf(struct btf *btf);
+
+#endif /* _KERNEL_BTF_BTF_H */
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 93f356d2b3d95..0009a7ac78462 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -395,14 +395,23 @@ config DEBUG_INFO_SPLIT
to know about the .dwo files and include them.
Incompatible with older versions of ccache.
-config DEBUG_INFO_BTF
- bool "Generate BTF type information"
+config BTF
+ bool "BTF type information support"
depends on !DEBUG_INFO_SPLIT && !DEBUG_INFO_REDUCED
depends on !GCC_PLUGIN_RANDSTRUCT || COMPILE_TEST
- depends on BPF_SYSCALL
depends on PAHOLE_VERSION >= 122
# pahole uses elfutils, which does not have support for Hexagon relocations
depends on !HEXAGON
+ help
+ Enable the core BTF (BPF Type Format) infrastructure. This provides
+ BTF parsing, type lookup, and display functionality used by BPF and
+ other kernel subsystems.
+
+ This option is typically selected automatically by DEBUG_INFO_BTF.
+
+config DEBUG_INFO_BTF
+ bool "Generate BTF type information"
+ select BTF
help
Generate deduplicated BTF type information from DWARF debug info.
Turning this on requires pahole v1.22 or later, which will convert
--
2.51.0
^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [RFC] btf: split core BTF parsing out of BPF subsystem into kernel/btf/
2026-03-25 1:18 ` [RFC] btf: split core BTF parsing out of BPF subsystem into kernel/btf/ Sasha Levin
@ 2026-03-25 2:15 ` bot+bpf-ci
0 siblings, 0 replies; 18+ messages in thread
From: bot+bpf-ci @ 2026-03-25 2:15 UTC (permalink / raw)
To: sashal, sashal
Cc: James.Bottomley, adobriyan, akpm, alexei.starovoitov, ast, bp,
bpf, corbet, dave.hansen, davidgow, deller, geert, gregkh, hpa,
jgross, jpoimboe, kees, laurent.pinchart, linux-doc, linux-kbuild,
linux-kernel, linux-modules, masahiroy, mcgrof, mingo, nathan,
nsc, peterz, petr.pavlu, pmladek, rdunlap, rostedt, tglx, vbabka,
x86, ast, andrii, daniel, martin.lau, eddyz87, yonghong.song, clm,
ihor.solodrai
[-- Attachment #1: Type: text/plain, Size: 2607 bytes --]
> diff --git a/kernel/btf/bpf.c b/kernel/btf/bpf.c
> new file mode 100644
> index 0000000000000..ef973771bb2ab
> --- /dev/null
> +++ b/kernel/btf/bpf.c
[ ... ]
> +struct btf_struct_ops_tab {
> + u32 cnt;
> + u32 capacity;
> + struct bpf_struct_ops_desc ops[];
> +};
> + u32 id;
> + struct rcu_head rcu;
> + struct btf_kfunc_set_tab *kfunc_set_tab;
> + struct btf_id_dtor_kfunc_tab *dtor_kfunc_tab;
> + struct btf_struct_metas *struct_meta_tab;
> + struct btf_struct_ops_tab *struct_ops_tab;
> +s32 bpf_find_btf_id(const char *name, u32 kind, struct btf **btf_p)
These six declarations between the closing brace of struct
btf_struct_ops_tab and bpf_find_btf_id() appear to be orphaned
fields from the original struct btf definition that were left
behind during the split.
They are parsed as file-scope global variable definitions. With
-fno-common, the name "id" in particular is likely to collide at
link time. Were these meant to be removed here, given that they
already appear correctly inside struct btf in kernel/btf/btf.h
under the CONFIG_BPF_SYSCALL guard?
> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> index 93f356d2b3d95..0009a7ac78462 100644
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -395,14 +395,23 @@ config DEBUG_INFO_SPLIT
[ ... ]
> +config BTF
> + bool "BTF type information support"
> + depends on !DEBUG_INFO_SPLIT && !DEBUG_INFO_REDUCED
> + depends on !GCC_PLUGIN_RANDSTRUCT || COMPILE_TEST
> + depends on PAHOLE_VERSION >= 122
> + # pahole uses elfutils, which does not have support for Hexagon relocations
> + depends on !HEXAGON
[ ... ]
> +config DEBUG_INFO_BTF
> + bool "Generate BTF type information"
> + select BTF
The old DEBUG_INFO_BTF had its own depends on lines:
depends on !DEBUG_INFO_SPLIT && !DEBUG_INFO_REDUCED
depends on !GCC_PLUGIN_RANDSTRUCT || COMPILE_TEST
depends on BPF_SYSCALL
depends on PAHOLE_VERSION >= 122
depends on !HEXAGON
Those constraints now live only on CONFIG_BTF, and DEBUG_INFO_BTF
uses "select BTF" with no depends of its own. Since select bypasses
the selected symbol's depends-on checks, DEBUG_INFO_BTF is now
visible and selectable even when BTF's dependencies are not met
(e.g. on Hexagon, or with PAHOLE_VERSION < 122).
Should DEBUG_INFO_BTF use "depends on BTF" instead of (or in
addition to) "select BTF", so that it inherits the same
constraints?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/23520643445
^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2026-03-25 2:15 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-03-23 16:48 [PATCH 0/2] kallsyms: show typed function parameters in oops/WARN dumps Sasha Levin
2026-03-23 16:48 ` [PATCH 1/2] kallsyms: show function parameter info " Sasha Levin
2026-03-24 15:03 ` Alexei Starovoitov
2026-03-24 16:00 ` Sasha Levin
2026-03-24 16:04 ` Alexei Starovoitov
2026-03-24 18:44 ` Sasha Levin
2026-03-25 1:18 ` [RFC] btf: split core BTF parsing out of BPF subsystem into kernel/btf/ Sasha Levin
2026-03-25 2:15 ` bot+bpf-ci
2026-03-24 17:34 ` [PATCH 1/2] kallsyms: show function parameter info in oops/WARN dumps Alan Maguire
2026-03-24 18:51 ` Sasha Levin
2026-03-23 16:48 ` [PATCH 2/2] kallsyms: add BTF-based deep parameter rendering in oops dumps Sasha Levin
2026-03-24 15:07 ` Alexei Starovoitov
2026-03-23 18:43 ` [PATCH 0/2] kallsyms: show typed function parameters in oops/WARN dumps Alexey Dobriyan
2026-03-23 22:58 ` Sasha Levin
2026-03-23 22:50 ` Andrew Morton
2026-03-23 23:08 ` Sasha Levin
2026-03-24 8:57 ` Jiri Olsa
2026-03-24 11:39 ` Sasha Levin
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox