All of lore.kernel.org
 help / color / mirror / Atom feed
From: Arnaldo Carvalho de Melo <acme@kernel.org>
To: Alan Maguire <alan.maguire@oracle.com>
Cc: Jiri Olsa <jolsa@kernel.org>,
	Clark Williams <williams@redhat.com>,
	dwarves@vger.kernel.org, bpf@vger.kernel.org,
	Andrii Nakryiko <andrii@kernel.org>,
	Yonghong Song <yonghong.song@linux.dev>,
	Arnaldo Carvalho de Melo <acme@redhat.com>
Subject: [PATCH 30/31] btf_loader: Fix multi-dimensional array loading
Date: Wed, 29 Jul 2026 16:07:30 -0300	[thread overview]
Message-ID: <20260729190733.72876-31-acme@kernel.org> (raw)
In-Reply-To: <20260729190733.72876-1-acme@kernel.org>

From: Arnaldo Carvalho de Melo <acme@redhat.com>

create_new_array() had a FIXME — it loaded every BTF_KIND_ARRAY as a
1D array (dimensions=1, nr_entries=[nelems]) and lost the chaining
that encodes multi-dim arrays.

BTF chains dimensions through the element type field:
  inner: ARRAY { type=int,   nelems=4 }  ← int[4]
  outer: ARRAY { type=inner, nelems=3 }  ← int[3][4]

When the outer node is loaded, the inner has already been loaded (BTF
types are processed in id order, and inner always has a lower id).
Detect this by checking whether cu__type(cu, ap->type) is a
DW_TAG_array_type, and if so, absorb its dimensions:

  array->dimensions   = inner->dimensions + 1
  array->nr_entries[] = [ap->nelems, inner->nr_entries...]
  array->tag.type     = inner->tag.type   ← skip to base element type

Note: cu__type() (types_table) must be used, not cu__tag() (tags_table);
DW_TAG_array_type entries are stored in the types table.

Before: int a[3][4] round-tripped as int a[4][3] (reversed dimensions,
        because each BTF_KIND_ARRAY was 1D and the display chain printed
        dimensions from innermost to outermost)
After:  int a[3][4] and int a[2][3][4] round-trip correctly via BTF

Fixes: 472256d3c57bef53 ("btf_loader: Introduce a loader for the BTF format")
Assisted-by: Claude:claude-sonnet-4-6
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 btf_loader.c        |  46 ++++++++++---
 tests/btf_arrays.sh | 158 ++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 194 insertions(+), 10 deletions(-)
 create mode 100755 tests/btf_arrays.sh

diff --git a/btf_loader.c b/btf_loader.c
index 5857c3b6d2fc4b2b..66ffd1a0b5ad73a3 100644
--- a/btf_loader.c
+++ b/btf_loader.c
@@ -200,20 +200,46 @@ static int create_new_array(struct cu *cu, const struct btf_type *tp, uint32_t i
 	if (array == NULL)
 		return -ENOMEM;
 
-	/* FIXME: where to get the number of dimensions?
-	 * it it flattened? */
-	array->dimensions = 1;
-	array->nr_entries = malloc(sizeof(uint32_t));
+	/*
+	 * BTF encodes each dimension of a multi-dimensional array as a
+	 * separate BTF_KIND_ARRAY node chained via the element type field.
+	 * For example, int a[3][4] becomes:
+	 *   inner: BTF_KIND_ARRAY { type=int,   nelems=4 }
+	 *   outer: BTF_KIND_ARRAY { type=inner, nelems=3 }
+	 *
+	 * Reconstruct the pahole multi-dim representation (one array_type
+	 * with dimensions[] and nr_entries[]) by absorbing any inner
+	 * BTF_KIND_ARRAY that was already loaded.
+	 */
+	struct tag *elem_tag = cu__type(cu, ap->type);
 
-	if (array->nr_entries == NULL) {
-		free(array);
-		return -ENOMEM;
+	if (elem_tag && elem_tag->tag == DW_TAG_array_type &&
+	    tag__array_type(elem_tag)->dimensions < UINT8_MAX) {
+		struct array_type *inner = tag__array_type(elem_tag);
+
+		array->dimensions = inner->dimensions + 1;
+		array->nr_entries = malloc(array->dimensions * sizeof(uint32_t));
+		if (array->nr_entries == NULL) {
+			free(array);
+			return -ENOMEM;
+		}
+		array->nr_entries[0] = ap->nelems;
+		memcpy(&array->nr_entries[1], inner->nr_entries,
+		       inner->dimensions * sizeof(uint32_t));
+		/* point directly to the base element type, skipping the inner array node */
+		array->tag.type = inner->tag.type;
+	} else {
+		array->dimensions = 1;
+		array->nr_entries = malloc(sizeof(uint32_t));
+		if (array->nr_entries == NULL) {
+			free(array);
+			return -ENOMEM;
+		}
+		array->nr_entries[0] = ap->nelems;
+		array->tag.type = ap->type;
 	}
 
-	array->nr_entries[0] = ap->nelems;
 	array->tag.tag = DW_TAG_array_type;
-	array->tag.type = ap->type;
-
 	cu__add_tag_with_id(cu, &array->tag, id);
 
 	return 0;
diff --git a/tests/btf_arrays.sh b/tests/btf_arrays.sh
new file mode 100755
index 0000000000000000..42979156b24c041c
--- /dev/null
+++ b/tests/btf_arrays.sh
@@ -0,0 +1,158 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0-only
+# Copyright © 2026 Red Hat Inc, Arnaldo Carvalho de Melo <acme@redhat.com>
+#
+# Test BTF multi-dimensional array encoding and loading round-trip.
+#
+# BTF represents each dimension of a multi-dimensional array as a separate
+# chained BTF_KIND_ARRAY node.  For int a[3][4]:
+#   inner: ARRAY { type=int,   nelems=4 }
+#   outer: ARRAY { type=inner, nelems=3 }
+#
+# The encoder (btf_encoder.c) must emit one BTF_KIND_ARRAY per dimension,
+# from innermost to outermost.  The loader (btf_loader.c) must collapse
+# the chain back into pahole's flat multi-dim array_type.
+#
+# Tests:
+#  1. BTF dump has the right number of chained ARRAY nodes for each dim
+#  2. pahole -F btf output matches DWARF output for all array members
+#  3. Array sizes are preserved across the encode→load round-trip
+
+. "$(dirname "$0")/test_lib.sh"
+
+outdir=$(make_tmpdir)
+trap cleanup EXIT
+
+title_log "BTF multi-dimensional array encoding and round-trip."
+
+CC=${CC:-gcc}
+if ! command -v ${CC%% *} > /dev/null 2>&1; then
+	info_log "skip: $CC not available"
+	test_skip
+fi
+
+if ! command -v bpftool > /dev/null 2>&1; then
+	info_log "skip: bpftool not available"
+	test_skip
+fi
+
+src=$(make_tmpsrc)
+obj=$(make_tmpobj)
+
+cat > "$src" << 'EOF'
+struct multidim {
+	int a1d[5];
+	int a2d[3][4];
+	int a3d[2][3][4];
+	char s[8];
+};
+
+struct multidim g;
+EOF
+
+$CC -g -c -o "$obj" "$src" 2>/dev/null
+if [ $? -ne 0 ]; then
+	error_log "FAIL: compilation failed"
+	test_fail
+fi
+
+# --- BTF encoding ---
+
+btf="$outdir/arrays.btf"
+pahole --btf_encode_detached="$btf" "$obj" 2>/dev/null
+if [ $? -ne 0 ] || [ ! -s "$btf" ]; then
+	error_log "FAIL: pahole --btf_encode_detached failed"
+	test_fail
+fi
+
+dump=$(bpftool btf dump file "$btf" 2>/dev/null)
+if [ -z "$dump" ]; then
+	error_log "FAIL: bpftool btf dump produced no output"
+	test_fail
+fi
+
+# --- Check 1: chained ARRAY nodes for int[3][4] ---
+# After btf__dedup the chain int[4] → int[3][4] must be present.
+
+inner4_id=$(echo "$dump" | grep "ARRAY.*nr_elems=4" | head -1 | sed 's/\[\([0-9]*\)\].*/\1/')
+if [ -z "$inner4_id" ]; then
+	error_log "FAIL: no ARRAY with nr_elems=4 found for int[3][4] inner dim"
+	test_fail
+fi
+info_log "inner ARRAY nr_elems=4 at type_id=$inner4_id: ok"
+
+# There must be an ARRAY referencing inner4_id with nr_elems=3 (outer of int[3][4])
+if ! echo "$dump" | grep -q "ARRAY.*type_id=${inner4_id}.*nr_elems=3"; then
+	error_log "FAIL: no outer ARRAY(type=${inner4_id}, nr_elems=3) for int[3][4]"
+	test_fail
+fi
+info_log "outer ARRAY nr_elems=3 referencing inner: ok"
+
+# --- Check 2: struct member type_ids must differ between a2d and a3d ---
+# If they were the same, a3d would not add the extra outer dimension.
+
+a2d_type=$(echo "$dump" | grep "'a2d'" | sed "s/.*type_id=\([0-9]*\).*/\1/")
+a3d_type=$(echo "$dump" | grep "'a3d'" | sed "s/.*type_id=\([0-9]*\).*/\1/")
+
+if [ -z "$a2d_type" ] || [ -z "$a3d_type" ]; then
+	error_log "FAIL: could not extract type_ids for a2d ($a2d_type) or a3d ($a3d_type)"
+	test_fail
+fi
+if [ "$a2d_type" = "$a3d_type" ]; then
+	error_log "FAIL: a2d and a3d share the same type_id ($a2d_type); a3d not encoded with extra dim"
+	test_fail
+fi
+info_log "a2d type_id=$a2d_type, a3d type_id=$a3d_type (distinct): ok"
+
+# --- Check 3: round-trip via pahole -F btf ---
+# In-place encoding then reload via BTF frontend must match DWARF output.
+
+pahole -J "$obj" 2>/dev/null
+if [ $? -ne 0 ]; then
+	error_log "FAIL: pahole -J in-place encoding failed"
+	test_fail
+fi
+
+btf_out=$(pahole -F btf -C multidim "$obj" 2>/dev/null)
+
+if [ -z "$btf_out" ]; then
+	error_log "FAIL: pahole -F btf -C multidim produced no output"
+	test_fail
+fi
+
+# 1D array preserved
+if ! echo "$btf_out" | grep -q "a1d\[5\]"; then
+	error_log "FAIL: round-trip: a1d[5] not found in BTF output"
+	test_fail
+fi
+info_log "a1d[5] preserved: ok"
+
+# 2D array — exact dimension order must match DWARF
+if ! echo "$btf_out" | grep -q "a2d\[3\]\[4\]"; then
+	error_log "FAIL: round-trip: a2d[3][4] not found in BTF output"
+	test_fail
+fi
+info_log "a2d[3][4] preserved: ok"
+
+# 3D array — exact dimension order must match DWARF
+if ! echo "$btf_out" | grep -q "a3d\[2\]\[3\]\[4\]"; then
+	error_log "FAIL: round-trip: a3d[2][3][4] not found in BTF output"
+	test_fail
+fi
+info_log "a3d[2][3][4] preserved: ok"
+
+# char array preserved
+if ! echo "$btf_out" | grep -q "s\[8\]"; then
+	error_log "FAIL: round-trip: s[8] not found in BTF output"
+	test_fail
+fi
+info_log "s[8] preserved: ok"
+
+# --- Check 4: struct size preserved ---
+if ! echo "$btf_out" | grep -q "size: 172"; then
+	error_log "FAIL: struct multidim size=172 not preserved after BTF round-trip"
+	test_fail
+fi
+info_log "struct multidim size=172 preserved: ok"
+
+test_pass
-- 
2.55.0


  parent reply	other threads:[~2026-07-29 19:08 UTC|newest]

Thread overview: 32+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-07-29 19:07 [PATCHES 00/31] pahole: Bug fixes and small improvements Arnaldo Carvalho de Melo
2026-07-29 19:07 ` [PATCH 01/31] cmake: Update minimum required version from 3.5 to 3.10 Arnaldo Carvalho de Melo
2026-07-29 19:07 ` [PATCH 02/31] Fix -Wsign-compare warnings across the codebase Arnaldo Carvalho de Melo
2026-07-29 19:07 ` [PATCH 03/31] btf_encoder: Fix interior pointer free and missing NULL check Arnaldo Carvalho de Melo
2026-07-29 19:07 ` [PATCH 04/31] dwarves: Fix missing list head initialization in type__clone_members Arnaldo Carvalho de Melo
2026-07-29 19:07 ` [PATCH 05/31] pahole: Fix instance memory leak on early returns in prototype__stdio_fprintf_value Arnaldo Carvalho de Melo
2026-07-29 19:07 ` [PATCH 06/31] ctf_loader, libctf: Fix error path resource leaks Arnaldo Carvalho de Melo
2026-07-29 19:07 ` [PATCH 07/31] dwarves: Don't search for holes before member byte sizes are cached Arnaldo Carvalho de Melo
2026-07-29 19:07 ` [PATCH 08/31] dwarf_loader: Allocate type_dcu via dwarf_cu__new to fix dangling stack pointer Arnaldo Carvalho de Melo
2026-07-29 19:07 ` [PATCH 09/31] dwarf_loader: Fix annotation failure leaks in variable and typedef creation Arnaldo Carvalho de Melo
2026-07-29 19:07 ` [PATCH 10/31] btf_encoder: Use btf_encoder__tag_type() for all type ID computations Arnaldo Carvalho de Melo
2026-07-29 19:07 ` [PATCH 11/31] pahole: Fix --errno typo that decrements instead of negating Arnaldo Carvalho de Melo
2026-07-29 19:07 ` [PATCH 12/31] dwarves: Fix heap buffer overflow in languages__parse realloc Arnaldo Carvalho de Melo
2026-07-29 19:07 ` [PATCH 13/31] btf_encoder: Fix early cleanup crashes in btf_encoder__new/delete Arnaldo Carvalho de Melo
2026-07-29 19:07 ` [PATCH 14/31] btf_encoder, libctf: Add elf_strptr NULL checks and fix kfunc bounds Arnaldo Carvalho de Melo
2026-07-29 19:07 ` [PATCH 15/31] dwarf_loader: Fix --fixup_silly_bitfields condition check Arnaldo Carvalho de Melo
2026-07-29 19:07 ` [PATCH 16/31] dwarf_loader: Skip libdw__lock when elfutils is built thread-safe Arnaldo Carvalho de Melo
2026-07-29 19:07 ` [PATCH 17/31] dwarf_loader: Fix data race in tag__init() decl_file string cache Arnaldo Carvalho de Melo
2026-07-29 19:07 ` [PATCH 18/31] pahole: Fix parse_btf_features("all") being a silent no-op Arnaldo Carvalho de Melo
2026-07-29 19:07 ` [PATCH 19/31] dwarves: Fix variable shadowing in __cus__find_struct_by_name() Arnaldo Carvalho de Melo
2026-07-29 19:07 ` [PATCH 20/31] dutil: Add exec_objcopy() shell-injection-safe helper Arnaldo Carvalho de Melo
2026-07-29 19:07 ` [PATCH 21/31] btf_encoder: Fall back to objcopy when llvm-objcopy is not available Arnaldo Carvalho de Melo
2026-07-29 19:07 ` [PATCH 22/31] dwarf_loader, btf_loader: Replace stale FIXME/XXX comments with explanations Arnaldo Carvalho de Melo
2026-07-29 19:07 ` [PATCH 23/31] pahole: Skip inline expansions during BTF encoding Arnaldo Carvalho de Melo
2026-07-29 19:07 ` [PATCH 24/31] pahole: Use fseek for seekable files in --prettify and --seek_bytes Arnaldo Carvalho de Melo
2026-07-29 19:07 ` [PATCH 25/31] pahole: Guard pipe_seek() against negative offsets Arnaldo Carvalho de Melo
2026-07-29 19:07 ` [PATCH 26/31] gobuffer: Remove 5 dead functions found via coverage analysis Arnaldo Carvalho de Melo
2026-07-29 19:07 ` [PATCH 27/31] dwarves: Remove 6 " Arnaldo Carvalho de Melo
2026-07-29 19:07 ` [PATCH 28/31] pfunct, dwarves_fprintf: Mark file-local functions as static Arnaldo Carvalho de Melo
2026-07-29 19:07 ` [PATCH 29/31] btf_encoder: Fix multi-dimensional array encoding Arnaldo Carvalho de Melo
2026-07-29 19:07 ` Arnaldo Carvalho de Melo [this message]
2026-07-29 19:07 ` [PATCH 31/31] btfdiff: Remove --flat_arrays now that pahole encodes multi dim arrays in BTF Arnaldo Carvalho de Melo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260729190733.72876-31-acme@kernel.org \
    --to=acme@kernel.org \
    --cc=acme@redhat.com \
    --cc=alan.maguire@oracle.com \
    --cc=andrii@kernel.org \
    --cc=bpf@vger.kernel.org \
    --cc=dwarves@vger.kernel.org \
    --cc=jolsa@kernel.org \
    --cc=williams@redhat.com \
    --cc=yonghong.song@linux.dev \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.