[PATCH bpf-next 0/2] bpftool: Batch bounded hash map dumps

[PATCH bpf-next 0/2] bpftool: Batch bounded hash map dumps

[Tianyi Chen]

This series uses lookup batches for ordinary hash maps whose maximum
key/value storage fits within a 4 MiB budget. It retains the existing
formatting and falls back to individual lookups if the initial batch
operation is unsupported. Other map types keep individual lookups.

Hash batch lookups need room for a complete bucket, so ENOSPC grows the
buffer without advancing the cursor. The memory eligibility check makes
that growth bounded even after output has begun. Non-ENOENT errors do
not expose untrusted counts or trigger a duplicate dump restart.

Related request:
https://github.com/libbpf/bpftool/issues/63

The series is based on bpf-next and does not depend on my recursive
map-dump series or the flags/ring-buffer series sent alongside it.

Validation on x86-64, Linux 7.3.0-rc1 in KVM, LLVM 20:
- Full bpftool build and focused BPF selftests build.
- All 11 bpftool_map_batch subtests passed against a bpftool built with
  only this series: complete unordered plain, JSON and pretty JSON
  contents, batch boundaries, short keys, odd-sized values and BTF.
- strace fault injection checked initial EINVAL/EOPNOTSUPP/ENOTSUPP
  fallback; ENOSPC growth initially and after progress; fatal EFAULT,
  ENOMEM and post-progress EIO/EINVAL; no duplicates and valid JSON.
- A map exceeding the memory eligibility budget used individual lookups.
- On the same static 100,000-entry hash map, BPF syscall counts dropped
  from 200,004 to 395. Five untraced runs had median elapsed times of
  0.774 s before and 0.734 s after; text formatting still dominates.

Fault injection simulates error returns; it is not a real collision
stress test. Only focused selftests were run. Strict checkpatch has no
errors or checks; the new test file is covered by existing MAINTAINERS
patterns.


Integration check: these three independent series also applied and built
together with my previously posted recursive map-dump v2 series. The
combined 64 subtests passed with no skips or failures in the same guest.

Tianyi Chen (2):
  bpftool: Use batch lookups for bounded hash map dumps
  selftests/bpf: Check bpftool batch map dump contents

 tools/bpf/bpftool/map.c                       | 113 ++++++++++-
 .../bpf/prog_tests/bpftool_map_batch.c        | 186 ++++++++++++++++++
 2 files changed, 291 insertions(+), 8 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/bpftool_map_batch.c

-- 
2.55.0

[PATCH bpf-next 1/2] bpftool: Use batch lookups for bounded hash map dumps

[Tianyi Chen]

Use BPF_MAP_LOOKUP_BATCH when dumping hash maps to reduce the number
of BPF syscalls. Share element formatting with individual lookups to
preserve plain, JSON and BTF output.

Start with up to 256 entries and grow on ENOSPC without advancing the
input cursor. Restrict the optimization to maps whose maximum key and
value storage fits in 4 MiB, so even a worst-case bucket can fit without
restarting a partially printed dump. Preserve aligned element buffers
for formatting keys and values with odd sizes.

Fall back to individual lookups only if the initial batch operation is
unsupported. Process the final partial batch on ENOENT, but never use
count or output buffers after other errors. Report errors after batch
traversal starts without restarting and duplicating output.

Assisted-by: Codex:GPT-6
Signed-off-by: Tianyi Chen <hi@tychen.cc>
---
 tools/bpf/bpftool/map.c | 113 +++++++++++++++++++++++++++++++++++++---
 1 file changed, 105 insertions(+), 8 deletions(-)

diff --git a/tools/bpf/bpftool/map.c b/tools/bpf/bpftool/map.c
index 684a8fb7241..1eccdc0d1e9 100644
--- a/tools/bpf/bpftool/map.c
+++ b/tools/bpf/bpftool/map.c
@@ -740,15 +740,10 @@ static int do_show(int argc, char **argv)
 	return errno == ENOENT ? 0 : -1;
 }
 
-static int dump_map_elem(int fd, void *key, void *value,
-			 struct bpf_map_info *map_info, struct btf *btf,
-			 json_writer_t *btf_wtr)
+static void print_map_elem(void *key, void *value,
+			   struct bpf_map_info *map_info, struct btf *btf,
+			   json_writer_t *btf_wtr)
 {
-	if (bpf_map_lookup_elem(fd, key, value)) {
-		print_entry_error(map_info, key, errno);
-		return -1;
-	}
-
 	if (json_output) {
 		print_entry_json(map_info, key, value, btf);
 	} else if (btf) {
@@ -762,10 +757,108 @@ static int dump_map_elem(int fd, void *key, void *value,
 	} else {
 		print_entry_plain(map_info, key, value);
 	}
+}
+
+static int dump_map_elem(int fd, void *key, void *value,
+			 struct bpf_map_info *map_info, struct btf *btf,
+			 json_writer_t *btf_wtr)
+{
+	if (bpf_map_lookup_elem(fd, key, value)) {
+		print_entry_error(map_info, key, errno);
+		return -1;
+	}
 
+	print_map_elem(key, value, map_info, btf, btf_wtr);
 	return 0;
 }
 
+#define MAP_DUMP_BATCH_SIZE 256U
+#define MAP_DUMP_BATCH_MAX_BYTES (4 * 1024 * 1024)
+
+/* Return 1 to use individual lookups, but only before batch traversal starts. */
+static int dump_map_batch(int fd, void *key, void *value,
+			  struct bpf_map_info *info, struct btf *btf,
+			  json_writer_t *wtr, unsigned int *num_elems)
+{
+	__u32 capacity, count, batch = 0, next_batch = 0, i;
+	void *keys = NULL, *values = NULL, *buf;
+	bool first = true, can_fallback = true;
+	int err;
+
+	/* Hash lookup batches must accommodate a whole bucket. Restrict the
+	 * optimization to maps whose worst-case bucket fits the memory budget,
+	 * so a later ENOSPC never forces a restart after printing some entries.
+	 * Division also bounds the allocation multiplications on 32-bit hosts.
+	 */
+	if (info->type != BPF_MAP_TYPE_HASH || !info->max_entries ||
+	    (__u64)info->key_size + info->value_size >
+	    MAP_DUMP_BATCH_MAX_BYTES / info->max_entries)
+		return 1;
+
+	capacity = min(info->max_entries, MAP_DUMP_BATCH_SIZE);
+resize:
+	buf = realloc(keys, (size_t)capacity * info->key_size);
+	if (!buf) {
+		err = ENOMEM;
+		goto error;
+	}
+	keys = buf;
+	buf = realloc(values, (size_t)capacity * info->value_size);
+	if (!buf) {
+		err = ENOMEM;
+		goto error;
+	}
+	values = buf;
+
+	while (true) {
+		count = capacity;
+		err = bpf_map_lookup_batch(fd, first ? NULL : &batch,
+					   &next_batch, keys, values, &count, NULL);
+		err = err ? errno : 0;
+		/* Older kernels reject the command before updating count. Do not
+		 * inspect the buffers on these errors, or fall back after progress.
+		 */
+		if (can_fallback && (err == EINVAL || err == EOPNOTSUPP ||
+				     err == 524 /* ENOTSUPP */)) {
+			err = 1;
+			goto out;
+		}
+		can_fallback = false;
+		if (err == ENOSPC) {
+			if (capacity == info->max_entries)
+				goto error;
+			capacity += min(capacity, info->max_entries - capacity);
+			/* Preserve the input cursor: the oversized bucket was not read. */
+			goto resize;
+		}
+		/* In particular, EFAULT can leave count and the buffers invalid. */
+		if (err && err != ENOENT)
+			goto error;
+		for (i = 0; i < count; i++) {
+			/* Keep the alignment provided by individual lookups, including
+			 * for BTF types whose map key/value size is not aligned.
+			 */
+			memcpy(key, keys + (size_t)i * info->key_size, info->key_size);
+			memcpy(value, values + (size_t)i * info->value_size, info->value_size);
+			print_map_elem(key, value, info, btf, wtr);
+			(*num_elems)++;
+		}
+		if (err == ENOENT) {
+			err = 0;
+			goto out;
+		}
+		first = false;
+		batch = next_batch;
+	}
+error:
+	p_err("can't lookup map batch: %s", strerror(err));
+	err = -1;
+out:
+	free(keys);
+	free(values);
+	return err;
+}
+
 static int maps_have_btf(int *fds, int nb_fds)
 {
 	struct bpf_map_info info = {};
@@ -869,6 +962,9 @@ map_dump(int fd, struct bpf_map_info *info, json_writer_t *wtr,
 		p_info("Warning: cannot read values from %s map with value_size != 8",
 		       map_type_str);
 	}
+	err = dump_map_batch(fd, key, value, info, btf, wtr, &num_elems);
+	if (err != 1)
+		goto end_dump;
 	while (true) {
 		err = bpf_map_get_next_key(fd, prev_key, key);
 		if (err) {
@@ -881,6 +977,7 @@ map_dump(int fd, struct bpf_map_info *info, json_writer_t *wtr,
 		prev_key = key;
 	}
 
+end_dump:
 	if (wtr) {
 		jsonw_end_array(wtr);	/* elements */
 		if (show_header)
-- 
2.55.0

[PATCH bpf-next 2/2] selftests/bpf: Check bpftool batch map dump contents

[Tianyi Chen]

Exercise hash map dumps around the initial batch size and across
multiple batches, including empty and single-entry maps. Compare each
complete unordered key/value set with the input data in plain, JSON
and pretty JSON output.

Cover one-byte keys, three-byte values and BTF-formatted maps to catch
cursor sizing, buffer alignment and formatting regressions.

Assisted-by: Codex:GPT-6
Signed-off-by: Tianyi Chen <hi@tychen.cc>
---
 .../bpf/prog_tests/bpftool_map_batch.c        | 186 ++++++++++++++++++
 1 file changed, 186 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/bpftool_map_batch.c

diff --git a/tools/testing/selftests/bpf/prog_tests/bpftool_map_batch.c b/tools/testing/selftests/bpf/prog_tests/bpftool_map_batch.c
new file mode 100644
index 00000000000..139d13a49dc
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/bpftool_map_batch.c
@@ -0,0 +1,186 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <test_progs.h>
+#include <bpftool_helpers.h>
+#include <bpf/btf.h>
+#include <ctype.h>
+
+#define MAX_ENTRIES 1025
+#define RECORD_SIZE 256
+#define OUTPUT_SIZE (MAX_ENTRIES * RECORD_SIZE + 1024)
+
+struct dump_case {
+	const char *name;
+	unsigned int count;
+	unsigned int key_size;
+	unsigned int value_size;
+	bool btf;
+};
+
+static void hex_bytes(char *out, const void *data, unsigned int size, bool json)
+{
+	const unsigned char *bytes = data;
+	unsigned int i;
+
+	if (json)
+		*out++ = '[';
+	for (i = 0; i < size; i++) {
+		if (json && i)
+			*out++ = ',';
+		out += sprintf(out, json ? "\"0x%02x\"" : "%02x", bytes[i]);
+	}
+	if (json)
+		*out++ = ']';
+	*out = '\0';
+}
+
+static void expected_record(char *record, const struct dump_case *test,
+			    unsigned int index, bool json)
+{
+	__u32 key = index, value = index * 37 + 11;
+	unsigned char short_key = index;
+	char key_hex[64], value_hex[64], formatted[96];
+
+	hex_bytes(key_hex, test->key_size == 1 ? (void *)&short_key : &key,
+		  test->key_size, json);
+	hex_bytes(value_hex, &value, test->value_size, json);
+	snprintf(formatted, sizeof(formatted), "{\"key\":%u,\"value\":%u}",
+		 key, value);
+	if (json && test->btf)
+		snprintf(record, RECORD_SIZE,
+			 "{\"key\":%s,\"value\":%s,\"formatted\":%s}",
+			 key_hex, value_hex, formatted);
+	else if (json)
+		snprintf(record, RECORD_SIZE, "{\"key\":%s,\"value\":%s}",
+			 key_hex, value_hex);
+	else if (test->btf)
+		snprintf(record, RECORD_SIZE, "%s", formatted);
+	else
+		snprintf(record, RECORD_SIZE, "key:%svalue:%s", key_hex, value_hex);
+}
+
+static void check_dump(const struct dump_case *test, __u32 id, bool json, bool pretty)
+{
+	bool array = json || test->btf;
+	char command[MAX_BPFTOOL_CMD_LEN], expected[RECORD_SIZE], footer[64];
+	bool seen[MAX_ENTRIES] = {};
+	char *output, *src, *dst, *cursor;
+	unsigned int i, n;
+	int err;
+
+	output = calloc(1, OUTPUT_SIZE);
+	if (!ASSERT_OK_PTR(output, "alloc_output"))
+		return;
+	snprintf(command, sizeof(command), "%smap dump id %u",
+		 pretty ? "-p " : json ? "-j " : "", id);
+	err = get_bpftool_command_output(command, output, OUTPUT_SIZE);
+	if (!ASSERT_OK(err, "map_dump"))
+		goto out;
+	/* Ignore presentation whitespace, but compare complete records and all
+	 * punctuation. Expected contents come only from the input data, never
+	 * from another map walk or bpftool invocation.
+	 */
+	for (src = output, dst = output; *src; src++)
+		if (!isspace((unsigned char)*src))
+			*dst++ = *src;
+	*dst = '\0';
+	cursor = output;
+	if (array) {
+		if (!ASSERT_EQ(*cursor, '[', "array_start"))
+			goto out;
+		cursor++;
+	}
+	for (n = 0; n < test->count; n++) {
+		if (array && n) {
+			if (!ASSERT_EQ(*cursor, ',', "record_separator"))
+				goto out;
+			cursor++;
+		}
+		for (i = 0; i < test->count; i++) {
+			if (seen[i])
+				continue;
+			expected_record(expected, test, i, json);
+			if (!strncmp(cursor, expected, strlen(expected)))
+				break;
+		}
+		if (!ASSERT_LT(i, test->count, "unique_expected_record"))
+			goto out;
+		seen[i] = true;
+		cursor += strlen(expected);
+	}
+	if (array) {
+		ASSERT_STREQ(cursor, "]", "array_end_and_count");
+	} else {
+		snprintf(footer, sizeof(footer), "Found%uelement%s", test->count,
+			 test->count == 1 ? "" : "s");
+		ASSERT_STREQ(cursor, footer, "plain_count");
+	}
+out:
+	free(output);
+}
+
+static void run_dump_case(const struct dump_case *test)
+{
+	LIBBPF_OPTS(bpf_map_create_opts, opts);
+	struct bpf_map_info info = {};
+	__u32 info_len = sizeof(info);
+	struct btf *btf = NULL;
+	unsigned int i;
+	int fd = -1;
+
+	if (test->btf) {
+		btf = btf__new_empty();
+		if (!ASSERT_OK_PTR(btf, "btf_new"))
+			return;
+		if (!ASSERT_EQ(btf__add_int(btf, "unsigned int", 4, 0), 1,
+			       "btf_int") ||
+		    !ASSERT_OK(btf__load_into_kernel(btf), "btf_load"))
+			goto out;
+		opts.btf_fd = btf__fd(btf);
+		opts.btf_key_type_id = 1;
+		opts.btf_value_type_id = 1;
+	}
+	fd = bpf_map_create(BPF_MAP_TYPE_HASH, "dump_batch", test->key_size,
+			    test->value_size, test->count ?: 1, &opts);
+	if (!ASSERT_OK_FD(fd, "map_create"))
+		goto out;
+	for (i = 0; i < test->count; i++) {
+		__u32 key = i, value = i * 37 + 11;
+		unsigned char short_key = i;
+		void *key_ptr = test->key_size == 1 ? (void *)&short_key : &key;
+
+		if (!ASSERT_OK(bpf_map_update_elem(fd, key_ptr, &value, BPF_ANY),
+			       "map_update"))
+			goto out;
+	}
+	if (!ASSERT_OK(bpf_map_get_info_by_fd(fd, &info, &info_len), "map_info"))
+		goto out;
+	check_dump(test, info.id, false, false);
+	check_dump(test, info.id, true, false);
+	check_dump(test, info.id, true, true);
+out:
+	if (fd >= 0)
+		close(fd);
+	btf__free(btf);
+}
+
+void test_bpftool_map_batch(void)
+{
+	static const struct dump_case cases[] = {
+		{ "empty", 0, 4, 4 },
+		{ "single", 1, 4, 4 },
+		{ "below_batch", 255, 4, 4 },
+		{ "exact_batch", 256, 4, 4 },
+		{ "above_batch", 257, 4, 4 },
+		{ "multiple_batches", 1025, 4, 4 },
+		{ "one_byte_key", 256, 1, 4 },
+		{ "odd_value_size", 257, 4, 3 },
+		{ "btf_empty", 0, 4, 4, true },
+		{ "btf_single", 1, 4, 4, true },
+		{ "btf_multiple_batches", 1025, 4, 4, true },
+	};
+	unsigned int i;
+
+	for (i = 0; i < ARRAY_SIZE(cases); i++)
+		if (test__start_subtest(cases[i].name))
+			run_dump_case(&cases[i]);
+}
-- 
2.55.0

Re: [PATCH bpf-next 2/2] selftests/bpf: Check bpftool batch map dump contents

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Multi-line comment does not follow the required BPF subsystem style.
--

commit c56ab936d02afbcb53a4b6f2e304647318354a05
Author: Tianyi Chen <hi@tychen.cc>

selftests/bpf: Check bpftool batch map dump contents

This commit adds selftests to exercise bpftool hash map dumps using batches.
It compares the unordered key/value results against expected inputs in plain,
JSON, and pretty JSON formats.

> diff --git a/tools/testing/selftests/bpf/prog_tests/bpftool_map_batch.c b/tools/testing/selftests/bpf/prog_tests/bpftool_map_batch.c
> new file mode 100644
> index 0000000000000..139d13a49dcef
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/prog_tests/bpftool_map_batch.c

[ ... ]

> +static void check_dump(const struct dump_case *test, __u32 id, bool json, bool pretty)
> +{

[ ... ]

> +	err = get_bpftool_command_output(command, output, OUTPUT_SIZE);
> +	if (!ASSERT_OK(err, "map_dump"))
> +		goto out;
> +	/* Ignore presentation whitespace, but compare complete records and all
> +	 * punctuation. Expected contents come only from the input data, never
> +	 * from another map walk or bpftool invocation.
> +	 */

[Severity: Low]
Could we update this multi-line comment in check_dump() to follow the
preferred BPF subsystem style where the opening /* is placed on its own line?

> +	for (src = output, dst = output; *src; src++)
> +		if (!isspace((unsigned char)*src))
> +			*dst++ = *src;
> +	*dst = '\0';

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260906170819.1212661-1-hi@tychen.cc?part=2

Re: [PATCH bpf-next 1/2] bpftool: Use batch lookups for bounded hash map dumps

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Multi-line comments do not follow the mandatory BPF subsystem style.
--

commit a1cb333fccdbef3bcdfaf55fc0e0a282326e27a3
Author: Tianyi Chen <hi@tychen.cc>

bpftool: Use batch lookups for bounded hash map dumps

This patch updates bpftool to use BPF_MAP_LOOKUP_BATCH when dumping bounded
hash maps to reduce BPF syscalls. It restricts this optimization to maps
that fit within a 4 MiB memory budget and handles fallbacks.

> diff --git a/tools/bpf/bpftool/map.c b/tools/bpf/bpftool/map.c
> index 684a8fb724146..1eccdc0d1e963 100644
> --- a/tools/bpf/bpftool/map.c
> +++ b/tools/bpf/bpftool/map.c
[ ... ]
> @@ -762,10 +757,108 @@ static int dump_map_elem(int fd, void *key, void *value,
[ ... ]
> +/* Return 1 to use individual lookups, but only before batch traversal starts. */
> +static int dump_map_batch(int fd, void *key, void *value,
> +			  struct bpf_map_info *info, struct btf *btf,
> +			  json_writer_t *wtr, unsigned int *num_elems)
> +{
> +	__u32 capacity, count, batch = 0, next_batch = 0, i;
> +	void *keys = NULL, *values = NULL, *buf;
> +	bool first = true, can_fallback = true;
> +	int err;
> +
> +	/* Hash lookup batches must accommodate a whole bucket. Restrict the
> +	 * optimization to maps whose worst-case bucket fits the memory budget,
> +	 * so a later ENOSPC never forces a restart after printing some entries.
> +	 * Division also bounds the allocation multiplications on 32-bit hosts.
> +	 */

[Severity: Low]
Should the opening /* be on its own line to follow the BPF subsystem
multi-line comment style?

> +	if (info->type != BPF_MAP_TYPE_HASH || !info->max_entries ||
> +	    (__u64)info->key_size + info->value_size >
> +	    MAP_DUMP_BATCH_MAX_BYTES / info->max_entries)
> +		return 1;
[ ... ]
> +	while (true) {
> +		count = capacity;
> +		err = bpf_map_lookup_batch(fd, first ? NULL : &batch,
> +					   &next_batch, keys, values, &count, NULL);
> +		err = err ? errno : 0;
> +		/* Older kernels reject the command before updating count. Do not
> +		 * inspect the buffers on these errors, or fall back after progress.
> +		 */

[Severity: Low]
Does this comment also need the opening /* on its own line?

> +		if (can_fallback && (err == EINVAL || err == EOPNOTSUPP ||
> +				     err == 524 /* ENOTSUPP */)) {
> +			err = 1;
> +			goto out;
> +		}
[ ... ]
> +		for (i = 0; i < count; i++) {
> +			/* Keep the alignment provided by individual lookups, including
> +			 * for BTF types whose map key/value size is not aligned.
> +			 */

[Severity: Low]
Here is a third instance. Should this opening /* be on its own line as well?

> +			memcpy(key, keys + (size_t)i * info->key_size, info->key_size);
> +			memcpy(value, values + (size_t)i * info->value_size, info->value_size);
> +			print_map_elem(key, value, info, btf, wtr);
> +			(*num_elems)++;
> +		}

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260906170819.1212661-1-hi@tychen.cc?part=1

Re: [PATCH bpf-next 2/2] selftests/bpf: Check bpftool batch map dump contents

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 5292 bytes --]

> selftests/bpf: Check bpftool batch map dump contents
>
> Exercise hash map dumps around the initial batch size and across
> multiple batches, including empty and single-entry maps. Compare each
> complete unordered key/value set with the input data in plain, JSON
> and pretty JSON output.
>
> Cover one-byte keys, three-byte values and BTF-formatted maps to catch
> cursor sizing, buffer alignment and formatting regressions.

The changelog, the file name and the subtest names all describe a batched
map-dump implementation that does not exist in the code under test.

bpftool map dump is implemented by map_dump() in tools/bpf/bpftool/map.c,
which walks the map one element at a time:

    while (true) {
        err = bpf_map_get_next_key(fd, prev_key, key);
        if (err) { ... break; }
        if (!dump_map_elem(fd, key, value, info, btf, wtr))
            num_elems++;
        prev_key = key;
    }

and dump_map_elem() calls bpf_map_lookup_elem() per key.

grep -rn 'batch\|BPF_MAP_LOOKUP_BATCH\|bpf_map_lookup_batch'
tools/bpf/bpftool/ returns only the unrelated bpftool batch file FILE
subcommand in main.c. There is no bpf_map_lookup_batch() call, no batch
cursor (in_batch/out_batch), no per-batch buffer and no batch-size
constant anywhere in bpftool.

What does this mean for the test?

- "around the initial batch size", "across multiple batches" and "cursor
  sizing, buffer alignment" describe nothing in the tested code. The
  counts 255/256/257/1025 and the subtest names below_batch, exact_batch,
  above_batch, multiple_batches are arbitrary; 256 is not a boundary of
  any code path.

- The file name bpftool_map_batch.c and the test name bpftool_map_batch
  collide with bpftool's real, unrelated batch subcommand (bpftool batch
  file FILE, main.c:334 do_batch()), so a reader looking for coverage of
  that subcommand will find a map-dump test instead.

- The series range 407ec415f849..1dcddb9351e8 contains no commit
  converting map dump to the batch API (the only other change is
  map_perf_ring.c ring-buffer support in c23648f76644), so this is not
  infrastructure landing ahead of its user.

What the test actually does is valid and useful - it verifies the exact
plain/JSON/pretty-JSON rendering of map dump for various key/value sizes
and element counts. Should the changelog and naming describe that (e.g.
"check map dump output contents") instead of claiming batch-boundary
coverage, so a future maintainer doesn't believe the batch paths are
covered when they are not?

> diff --git a/tools/testing/selftests/bpf/prog_tests/bpftool_map_batch.c b/tools/testing/selftests/bpf/prog_tests/bpftool_map_batch.c
> new file mode 100644
> index 000000000000..139d13a49dce
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/prog_tests/bpftool_map_batch.c

[ ... ]

> @@ -0,0 +1,186 @@

[ ... ]

> +void test_bpftool_map_batch(void)
> +{
> +	static const struct dump_case cases[] = {
> +		{ "empty", 0, 4, 4 },
> +		{ "single", 1, 4, 4 },
> +		{ "below_batch", 255, 4, 4 },
> +		{ "exact_batch", 256, 4, 4 },
> +		{ "above_batch", 257, 4, 4 },
> +		{ "multiple_batches", 1025, 4, 4 },
> +		{ "one_byte_key", 256, 1, 4 },
> +		{ "odd_value_size", 257, 4, 3 },
> +		{ "btf_empty", 0, 4, 4, true },
> +		{ "btf_single", 1, 4, 4, true },
> +		{ "btf_multiple_batches", 1025, 4, 4, true },
> +	};

A subsystem pattern flags this as potentially concerning: the new test
file is structured around an internal batching strategy rather than the
interface it actually checks. Four of the eleven cases exist only to
straddle a supposed 256-element batch boundary (below_batch, exact_batch,
above_batch, multiple_batches), and the changelog names "cursor sizing"
and "buffer alignment" as the regressions being guarded.

map_dump() in tools/bpf/bpftool/map.c uses bpf_map_get_next_key() +
bpf_map_lookup_elem() per element, so no such boundary exists and these
cases are indistinguishable from any other element count. The
interface-level part of the test (exact plain/JSON/pretty-JSON record
text, element count footer, BTF formatted field) is sound and is what the
case table should be organised around.

The guide directive notes: a new selftest file added for a narrow variant
(a specific size, a specific internal code path) of behavior an existing
test in the same directory already covers more generally should be
reported as a concern - asking whether it should be a new case in the
existing test instead. Tests written against specific implementation
details (an internal batching strategy, a particular code path taken to
reach a result) break or need a rewrite whenever that implementation
changes.

That said, the assertions themselves are written against observable
bpftool output, not against internal state, so the test will not break if
bpftool's enumeration strategy changes - only the case names and the
changelog would become misleading. Adding extra element counts is also
harmless coverage.

Does the case table organization based on a non-existent batch boundary
make the test harder to maintain or extend?


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34048930508

Re: [PATCH bpf-next 1/2] bpftool: Use batch lookups for bounded hash map dumps

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 5757 bytes --]

> diff --git a/tools/bpf/bpftool/map.c b/tools/bpf/bpftool/map.c
> index 684a8fb72414..1eccdc0d1e96 100644
> --- a/tools/bpf/bpftool/map.c
> +++ b/tools/bpf/bpftool/map.c

[ ... ]

> @@ -762,10 +757,108 @@ static int dump_map_elem(int fd, void *key, void *value,
>  	} else {
>  		print_entry_plain(map_info, key, value);
>  	}
> +}
> +
> +static int dump_map_elem(int fd, void *key, void *value,
> +			 struct bpf_map_info *map_info, struct btf *btf,
> +			 json_writer_t *btf_wtr)
> +{
> +	if (bpf_map_lookup_elem(fd, key, value)) {
> +		print_entry_error(map_info, key, errno);
> +		return -1;
> +	}
>
> +	print_map_elem(key, value, map_info, btf, btf_wtr);
>  	return 0;
>  }
>
> +#define MAP_DUMP_BATCH_SIZE 256U
> +#define MAP_DUMP_BATCH_MAX_BYTES (4 * 1024 * 1024)
> +
> +/* Return 1 to use individual lookups, but only before batch traversal starts. */
> +static int dump_map_batch(int fd, void *key, void *value,
> +			  struct bpf_map_info *info, struct btf *btf,
> +			  json_writer_t *wtr, unsigned int *num_elems)
> +{
> +	__u32 capacity, count, batch = 0, next_batch = 0, i;
> +	void *keys = NULL, *values = NULL, *buf;
> +	bool first = true, can_fallback = true;
> +	int err;
> +
> +	/* Hash lookup batches must accommodate a whole bucket. Restrict the
> +	 * optimization to maps whose worst-case bucket fits the memory budget,
> +	 * so a later ENOSPC never forces a restart after printing some entries.
> +	 * Division also bounds the allocation multiplications on 32-bit hosts.
> +	 */
> +	if (info->type != BPF_MAP_TYPE_HASH || !info->max_entries ||
> +	    (__u64)info->key_size + info->value_size >
> +	    MAP_DUMP_BATCH_MAX_BYTES / info->max_entries)
> +		return 1;
> +
> +	capacity = min(info->max_entries, MAP_DUMP_BATCH_SIZE);
> +resize:
> +	buf = realloc(keys, (size_t)capacity * info->key_size);
> +	if (!buf) {
> +		err = ENOMEM;
> +		goto error;
> +	}
> +	keys = buf;
> +	buf = realloc(values, (size_t)capacity * info->value_size);
> +	if (!buf) {
> +		err = ENOMEM;
> +		goto error;
> +	}
> +	values = buf;
> +
> +	while (true) {
> +		count = capacity;
> +		err = bpf_map_lookup_batch(fd, first ? NULL : &batch,
> +					   &next_batch, keys, values, &count, NULL);
> +		err = err ? errno : 0;
> +		/* Older kernels reject the command before updating count. Do not
> +		 * inspect the buffers on these errors, or fall back after progress.
> +		 */
> +		if (can_fallback && (err == EINVAL || err == EOPNOTSUPP ||
> +				     err == 524 /* ENOTSUPP */)) {
> +			err = 1;
> +			goto out;
> +		}
> +		can_fallback = false;
> +		if (err == ENOSPC) {
> +			if (capacity == info->max_entries)
> +				goto error;
> +			capacity += min(capacity, info->max_entries - capacity);
> +			/* Preserve the input cursor: the oversized bucket was not read. */
> +			goto resize;
> +		}
> +		/* In particular, EFAULT can leave count and the buffers invalid. */
> +		if (err && err != ENOENT)
> +			goto error;
> +		for (i = 0; i < count; i++) {
> +			/* Keep the alignment provided by individual lookups, including
> +			 * for BTF types whose map key/value size is not aligned.
> +			 */
> +			memcpy(key, keys + (size_t)i * info->key_size, info->key_size);
> +			memcpy(value, values + (size_t)i * info->value_size, info->value_size);
> +			print_map_elem(key, value, info, btf, wtr);
> +			(*num_elems)++;
> +		}
> +		if (err == ENOENT) {
> +			err = 0;
> +			goto out;
> +		}
> +		first = false;
> +		batch = next_batch;
> +	}
> +error:
> +	p_err("can't lookup map batch: %s", strerror(err));
> +	err = -1;

Could this p_err() call produce malformed JSON output when json_output is
set?

Looking at the call chain from do_dump():

do_dump()
  assigns wtr = json_wtr when json_output is set
  calls map_dump()
    opens the elements array with jsonw_start_array(wtr)
    calls dump_map_batch()
      can reach p_err() here while the array is still open
    closes the array at end_dump: with jsonw_end_array(wtr)

When p_err() runs with json_output set, it emits:

tools/bpf/bpftool/common.c:p_err() {
    if (json_output) {
        jsonw_start_object(json_wtr);
        jsonw_name(json_wtr, "error");
        jsonw_vprintf_enquote(json_wtr, fmt, ap);
        jsonw_end_object(json_wtr);
    }
}

So the error object would be inserted as a member of the elements array,
giving consumers an object with no .key or .value fields mixed in with
normal element objects.

This differs from the existing in-array error handling. The pre-existing
print_entry_error() deliberately emits a well-formed element:

{"key": [...], "value": {"error": "..."}}

And p_info() returns early when json_output is set, which is why the
existing REUSEPORT_SOCKARRAY warning inside the array is safe.

Would it be better to emit the error after jsonw_end_array() to keep the
array schema homogeneous, or to use the print_entry_error() format with
a key?

> +out:
> +	free(keys);
> +	free(values);
> +	return err;
> +}

[ ... ]

> @@ -869,6 +962,9 @@ map_dump(int fd, struct bpf_map_info *info, json_writer_t *wtr,
>  		p_info("Warning: cannot read values from %s map with value_size != 8",
>  		       map_type_str);
>  	}
> +	err = dump_map_batch(fd, key, value, info, btf, wtr, &num_elems);
> +	if (err != 1)
> +		goto end_dump;
>  	while (true) {
>  		err = bpf_map_get_next_key(fd, prev_key, key);
>  		if (err) {
> @@ -881,6 +977,7 @@ map_dump(int fd, struct bpf_map_info *info, json_writer_t *wtr,
>  		prev_key = key;
>  	}
>
> +end_dump:
>  	if (wtr) {
>  		jsonw_end_array(wtr);	/* elements */
>  		if (show_header)


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34048943300

[PATCH bpf-next v2 0/2] bpftool: Batch bounded hash map dumps

[Tianyi Chen]

Use lookup batches for ordinary hash maps whose maximum key/value
storage fits within 4 MiB. Patch 1 implements batch traversal; patch 2
checks complete output contents around its batch boundary and for short
keys, odd-sized values and BTF formatting.

Changes in v2:
- Send fatal batch diagnostics to stderr. JSON elements arrays retain
  only map-entry objects, including after a partially completed dump.
- Use the BPF multiline comment style in the implementation and test.

Validation:
- All 11 batch dump subtests passed in the integration run below.
- Reproduced the v1 generic error object inside the elements array.
- Injected initial EFAULT/ENOMEM and post-progress EIO/EINVAL with
  strace for -j and -p, both with and without -r. All 16 cases returned
  failure, emitted the diagnostic on stderr, and kept a complete JSON
  document containing only the expected zero or 256 distinct entries.
- Initial EINVAL/EOPNOTSUPP/ENOTSUPP fallback and ENOSPC growth before
  and after progress still returned all 1,025 entries without duplicates.

The injected syscall errors are not a real hash collision stress test.
The batching algorithm and eligibility bound are unchanged from v1.

v1: https://lore.kernel.org/r/20260906170819.1212661-1-hi@tychen.cc
Request: https://github.com/libbpf/bpftool/issues/63

Integration validation: the flags, ring-buffer and batch v2 series plus
recursive dump v3 built together with LLVM 20. All 69 focused BPF subtests
passed in an x86-64 KVM guest running Linux 7.3.0-rc1, with no skips or
failures. The bpftool-map man page also built successfully. This is a
focused run, not the full BPF selftest suite.


Tianyi Chen (2):
  bpftool: Use batch lookups for bounded hash map dumps
  selftests/bpf: Check bpftool batch map dump contents

 tools/bpf/bpftool/map.c                       | 116 ++++++++++-
 .../bpf/prog_tests/bpftool_map_batch.c        | 187 ++++++++++++++++++
 2 files changed, 295 insertions(+), 8 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/bpftool_map_batch.c

-- 
2.55.0

[PATCH bpf-next v2 1/2] bpftool: Use batch lookups for bounded hash map dumps

[Tianyi Chen]

Use BPF_MAP_LOOKUP_BATCH when dumping hash maps to reduce the number
of BPF syscalls. Share element formatting with individual lookups to
preserve plain, JSON and BTF output.

Start with up to 256 entries and grow on ENOSPC without advancing the
input cursor. Restrict the optimization to maps whose maximum key and
value storage fits in 4 MiB, so even a worst-case bucket can fit without
restarting a partially printed dump. Preserve aligned element buffers
for formatting keys and values with odd sizes.

Fall back to individual lookups only if the initial batch operation is
unsupported. Process the final partial batch on ENOENT, but never use
count or output buffers after other errors. Report errors after batch
traversal starts without restarting and duplicating output.

Send batch failure diagnostics to stderr so JSON element arrays contain
only map entries.

Link: https://github.com/libbpf/bpftool/issues/63

Assisted-by: Codex:GPT-6
Signed-off-by: Tianyi Chen <hi@tychen.cc>
---
 tools/bpf/bpftool/map.c | 116 +++++++++++++++++++++++++++++++++++++---
 1 file changed, 108 insertions(+), 8 deletions(-)

diff --git a/tools/bpf/bpftool/map.c b/tools/bpf/bpftool/map.c
index 684a8fb7241..ea6f1389245 100644
--- a/tools/bpf/bpftool/map.c
+++ b/tools/bpf/bpftool/map.c
@@ -740,15 +740,10 @@ static int do_show(int argc, char **argv)
 	return errno == ENOENT ? 0 : -1;
 }
 
-static int dump_map_elem(int fd, void *key, void *value,
-			 struct bpf_map_info *map_info, struct btf *btf,
-			 json_writer_t *btf_wtr)
+static void print_map_elem(void *key, void *value,
+			   struct bpf_map_info *map_info, struct btf *btf,
+			   json_writer_t *btf_wtr)
 {
-	if (bpf_map_lookup_elem(fd, key, value)) {
-		print_entry_error(map_info, key, errno);
-		return -1;
-	}
-
 	if (json_output) {
 		print_entry_json(map_info, key, value, btf);
 	} else if (btf) {
@@ -762,10 +757,111 @@ static int dump_map_elem(int fd, void *key, void *value,
 	} else {
 		print_entry_plain(map_info, key, value);
 	}
+}
+
+static int dump_map_elem(int fd, void *key, void *value,
+			 struct bpf_map_info *map_info, struct btf *btf,
+			 json_writer_t *btf_wtr)
+{
+	if (bpf_map_lookup_elem(fd, key, value)) {
+		print_entry_error(map_info, key, errno);
+		return -1;
+	}
 
+	print_map_elem(key, value, map_info, btf, btf_wtr);
 	return 0;
 }
 
+#define MAP_DUMP_BATCH_SIZE 256U
+#define MAP_DUMP_BATCH_MAX_BYTES (4 * 1024 * 1024)
+
+/* Return 1 to use individual lookups, but only before batch traversal starts. */
+static int dump_map_batch(int fd, void *key, void *value,
+			  struct bpf_map_info *info, struct btf *btf,
+			  json_writer_t *wtr, unsigned int *num_elems)
+{
+	__u32 capacity, count, batch = 0, next_batch = 0, i;
+	void *keys = NULL, *values = NULL, *buf;
+	bool first = true, can_fallback = true;
+	int err;
+
+	/*
+	 * Hash lookup batches must accommodate a whole bucket. Restrict the
+	 * optimization to maps whose worst-case bucket fits the memory budget,
+	 * so a later ENOSPC never forces a restart after printing some entries.
+	 * Division also bounds the allocation multiplications on 32-bit hosts.
+	 */
+	if (info->type != BPF_MAP_TYPE_HASH || !info->max_entries ||
+	    (__u64)info->key_size + info->value_size >
+	    MAP_DUMP_BATCH_MAX_BYTES / info->max_entries)
+		return 1;
+
+	capacity = min(info->max_entries, MAP_DUMP_BATCH_SIZE);
+resize:
+	buf = realloc(keys, (size_t)capacity * info->key_size);
+	if (!buf) {
+		err = ENOMEM;
+		goto error;
+	}
+	keys = buf;
+	buf = realloc(values, (size_t)capacity * info->value_size);
+	if (!buf) {
+		err = ENOMEM;
+		goto error;
+	}
+	values = buf;
+
+	while (true) {
+		count = capacity;
+		err = bpf_map_lookup_batch(fd, first ? NULL : &batch,
+					   &next_batch, keys, values, &count, NULL);
+		err = err ? errno : 0;
+		/*
+		 * Older kernels reject the command before updating count. Do not
+		 * inspect the buffers on these errors, or fall back after progress.
+		 */
+		if (can_fallback && (err == EINVAL || err == EOPNOTSUPP ||
+				     err == 524 /* ENOTSUPP */)) {
+			err = 1;
+			goto out;
+		}
+		can_fallback = false;
+		if (err == ENOSPC) {
+			if (capacity == info->max_entries)
+				goto error;
+			capacity += min(capacity, info->max_entries - capacity);
+			/* Preserve the input cursor: the oversized bucket was not read. */
+			goto resize;
+		}
+		/* In particular, EFAULT can leave count and the buffers invalid. */
+		if (err && err != ENOENT)
+			goto error;
+		for (i = 0; i < count; i++) {
+			/*
+			 * Keep the alignment provided by individual lookups, including
+			 * for BTF types whose map key/value size is not aligned.
+			 */
+			memcpy(key, keys + (size_t)i * info->key_size, info->key_size);
+			memcpy(value, values + (size_t)i * info->value_size, info->value_size);
+			print_map_elem(key, value, info, btf, wtr);
+			(*num_elems)++;
+		}
+		if (err == ENOENT) {
+			err = 0;
+			goto out;
+		}
+		first = false;
+		batch = next_batch;
+	}
+error:
+	fprintf(stderr, "Error: can't lookup map batch: %s\n", strerror(err));
+	err = -1;
+out:
+	free(keys);
+	free(values);
+	return err;
+}
+
 static int maps_have_btf(int *fds, int nb_fds)
 {
 	struct bpf_map_info info = {};
@@ -869,6 +965,9 @@ map_dump(int fd, struct bpf_map_info *info, json_writer_t *wtr,
 		p_info("Warning: cannot read values from %s map with value_size != 8",
 		       map_type_str);
 	}
+	err = dump_map_batch(fd, key, value, info, btf, wtr, &num_elems);
+	if (err != 1)
+		goto end_dump;
 	while (true) {
 		err = bpf_map_get_next_key(fd, prev_key, key);
 		if (err) {
@@ -881,6 +980,7 @@ map_dump(int fd, struct bpf_map_info *info, json_writer_t *wtr,
 		prev_key = key;
 	}
 
+end_dump:
 	if (wtr) {
 		jsonw_end_array(wtr);	/* elements */
 		if (show_header)
-- 
2.55.0

[PATCH bpf-next v2 2/2] selftests/bpf: Check bpftool batch map dump contents

[Tianyi Chen]

Exercise hash map dumps around the initial batch size and across
multiple batches, including empty and single-entry maps. Compare each
complete unordered key/value set with the input data in plain, JSON
and pretty JSON output.

Cover one-byte keys, three-byte values and BTF-formatted maps to catch
cursor sizing, buffer alignment and formatting regressions.

Assisted-by: Codex:GPT-6
Signed-off-by: Tianyi Chen <hi@tychen.cc>
---
 .../bpf/prog_tests/bpftool_map_batch.c        | 187 ++++++++++++++++++
 1 file changed, 187 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/bpftool_map_batch.c

diff --git a/tools/testing/selftests/bpf/prog_tests/bpftool_map_batch.c b/tools/testing/selftests/bpf/prog_tests/bpftool_map_batch.c
new file mode 100644
index 00000000000..b4216ed778e
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/bpftool_map_batch.c
@@ -0,0 +1,187 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <test_progs.h>
+#include <bpftool_helpers.h>
+#include <bpf/btf.h>
+#include <ctype.h>
+
+#define MAX_ENTRIES 1025
+#define RECORD_SIZE 256
+#define OUTPUT_SIZE (MAX_ENTRIES * RECORD_SIZE + 1024)
+
+struct dump_case {
+	const char *name;
+	unsigned int count;
+	unsigned int key_size;
+	unsigned int value_size;
+	bool btf;
+};
+
+static void hex_bytes(char *out, const void *data, unsigned int size, bool json)
+{
+	const unsigned char *bytes = data;
+	unsigned int i;
+
+	if (json)
+		*out++ = '[';
+	for (i = 0; i < size; i++) {
+		if (json && i)
+			*out++ = ',';
+		out += sprintf(out, json ? "\"0x%02x\"" : "%02x", bytes[i]);
+	}
+	if (json)
+		*out++ = ']';
+	*out = '\0';
+}
+
+static void expected_record(char *record, const struct dump_case *test,
+			    unsigned int index, bool json)
+{
+	__u32 key = index, value = index * 37 + 11;
+	unsigned char short_key = index;
+	char key_hex[64], value_hex[64], formatted[96];
+
+	hex_bytes(key_hex, test->key_size == 1 ? (void *)&short_key : &key,
+		  test->key_size, json);
+	hex_bytes(value_hex, &value, test->value_size, json);
+	snprintf(formatted, sizeof(formatted), "{\"key\":%u,\"value\":%u}",
+		 key, value);
+	if (json && test->btf)
+		snprintf(record, RECORD_SIZE,
+			 "{\"key\":%s,\"value\":%s,\"formatted\":%s}",
+			 key_hex, value_hex, formatted);
+	else if (json)
+		snprintf(record, RECORD_SIZE, "{\"key\":%s,\"value\":%s}",
+			 key_hex, value_hex);
+	else if (test->btf)
+		snprintf(record, RECORD_SIZE, "%s", formatted);
+	else
+		snprintf(record, RECORD_SIZE, "key:%svalue:%s", key_hex, value_hex);
+}
+
+static void check_dump(const struct dump_case *test, __u32 id, bool json, bool pretty)
+{
+	bool array = json || test->btf;
+	char command[MAX_BPFTOOL_CMD_LEN], expected[RECORD_SIZE], footer[64];
+	bool seen[MAX_ENTRIES] = {};
+	char *output, *src, *dst, *cursor;
+	unsigned int i, n;
+	int err;
+
+	output = calloc(1, OUTPUT_SIZE);
+	if (!ASSERT_OK_PTR(output, "alloc_output"))
+		return;
+	snprintf(command, sizeof(command), "%smap dump id %u",
+		 pretty ? "-p " : json ? "-j " : "", id);
+	err = get_bpftool_command_output(command, output, OUTPUT_SIZE);
+	if (!ASSERT_OK(err, "map_dump"))
+		goto out;
+	/*
+	 * Ignore presentation whitespace, but compare complete records and all
+	 * punctuation. Expected contents come only from the input data, never
+	 * from another map walk or bpftool invocation.
+	 */
+	for (src = output, dst = output; *src; src++)
+		if (!isspace((unsigned char)*src))
+			*dst++ = *src;
+	*dst = '\0';
+	cursor = output;
+	if (array) {
+		if (!ASSERT_EQ(*cursor, '[', "array_start"))
+			goto out;
+		cursor++;
+	}
+	for (n = 0; n < test->count; n++) {
+		if (array && n) {
+			if (!ASSERT_EQ(*cursor, ',', "record_separator"))
+				goto out;
+			cursor++;
+		}
+		for (i = 0; i < test->count; i++) {
+			if (seen[i])
+				continue;
+			expected_record(expected, test, i, json);
+			if (!strncmp(cursor, expected, strlen(expected)))
+				break;
+		}
+		if (!ASSERT_LT(i, test->count, "unique_expected_record"))
+			goto out;
+		seen[i] = true;
+		cursor += strlen(expected);
+	}
+	if (array) {
+		ASSERT_STREQ(cursor, "]", "array_end_and_count");
+	} else {
+		snprintf(footer, sizeof(footer), "Found%uelement%s", test->count,
+			 test->count == 1 ? "" : "s");
+		ASSERT_STREQ(cursor, footer, "plain_count");
+	}
+out:
+	free(output);
+}
+
+static void run_dump_case(const struct dump_case *test)
+{
+	LIBBPF_OPTS(bpf_map_create_opts, opts);
+	struct bpf_map_info info = {};
+	__u32 info_len = sizeof(info);
+	struct btf *btf = NULL;
+	unsigned int i;
+	int fd = -1;
+
+	if (test->btf) {
+		btf = btf__new_empty();
+		if (!ASSERT_OK_PTR(btf, "btf_new"))
+			return;
+		if (!ASSERT_EQ(btf__add_int(btf, "unsigned int", 4, 0), 1,
+			       "btf_int") ||
+		    !ASSERT_OK(btf__load_into_kernel(btf), "btf_load"))
+			goto out;
+		opts.btf_fd = btf__fd(btf);
+		opts.btf_key_type_id = 1;
+		opts.btf_value_type_id = 1;
+	}
+	fd = bpf_map_create(BPF_MAP_TYPE_HASH, "dump_batch", test->key_size,
+			    test->value_size, test->count ?: 1, &opts);
+	if (!ASSERT_OK_FD(fd, "map_create"))
+		goto out;
+	for (i = 0; i < test->count; i++) {
+		__u32 key = i, value = i * 37 + 11;
+		unsigned char short_key = i;
+		void *key_ptr = test->key_size == 1 ? (void *)&short_key : &key;
+
+		if (!ASSERT_OK(bpf_map_update_elem(fd, key_ptr, &value, BPF_ANY),
+			       "map_update"))
+			goto out;
+	}
+	if (!ASSERT_OK(bpf_map_get_info_by_fd(fd, &info, &info_len), "map_info"))
+		goto out;
+	check_dump(test, info.id, false, false);
+	check_dump(test, info.id, true, false);
+	check_dump(test, info.id, true, true);
+out:
+	if (fd >= 0)
+		close(fd);
+	btf__free(btf);
+}
+
+void test_bpftool_map_batch(void)
+{
+	static const struct dump_case cases[] = {
+		{ "empty", 0, 4, 4 },
+		{ "single", 1, 4, 4 },
+		{ "below_batch", 255, 4, 4 },
+		{ "exact_batch", 256, 4, 4 },
+		{ "above_batch", 257, 4, 4 },
+		{ "multiple_batches", 1025, 4, 4 },
+		{ "one_byte_key", 256, 1, 4 },
+		{ "odd_value_size", 257, 4, 3 },
+		{ "btf_empty", 0, 4, 4, true },
+		{ "btf_single", 1, 4, 4, true },
+		{ "btf_multiple_batches", 1025, 4, 4, true },
+	};
+	unsigned int i;
+
+	for (i = 0; i < ARRAY_SIZE(cases); i++)
+		if (test__start_subtest(cases[i].name))
+			run_dump_case(&cases[i]);
+}
-- 
2.55.0

Re: [PATCH bpf-next 2/2] selftests/bpf: Check bpftool batch map dump contents

[Tianyi Chen]

Thank you for the review. The batch implementation is patch 1 of the
same series:
https://lore.kernel.org/r/20260906170819.1212661-2-hi@tychen.cc

Its dump_map_batch() calls bpf_map_lookup_batch(), starts with up to 256
entries and grows the buffer on ENOSPC. Patch 2 tests that implementation.
The range described in the review appears to include the separate ring
buffer patch instead of this batch implementation.

The 255/256/257 cases therefore do straddle an implemented boundary.
Their assertions compare complete observable key/value output and do not
require a particular internal traversal order. I also built and tested
the v1 batch series independently and confirmed the batch syscalls.

The updated series is here:
https://lore.kernel.org/r/178874362960.1415955.9584865244247659247.batch-v2-0@tychen.cc

It also addresses the separate JSON error-reporting and comment-style
feedback. Fatal batch diagnostics now go to stderr. Fault injection
checked initial EFAULT/ENOMEM and post-progress EIO/EINVAL in compact and
pretty JSON, both with and without recursive dumping: documents remained
complete and contained only expected map entries, with a nonzero status.
Unsupported-command fallback and ENOSPC growth retained all 1,025 entries.

Thanks,
Tianyi

Re: [PATCH bpf-next v2 2/2] selftests/bpf: Check bpftool batch map dump contents

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 1685 bytes --]

> diff --git a/tools/testing/selftests/bpf/prog_tests/bpftool_map_batch.c b/tools/testing/selftests/bpf/prog_tests/bpftool_map_batch.c
> new file mode 100644
> index 000000000000..b4216ed778ef
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/prog_tests/bpftool_map_batch.c

[ ... ]

> +static void expected_record(char *record, const struct dump_case *test,
> +			    unsigned int index, bool json)
> +{
> +	__u32 key = index, value = index * 37 + 11;
> +	unsigned char short_key = index;
> +	char key_hex[64], value_hex[64], formatted[96];
> +
> +	hex_bytes(key_hex, test->key_size == 1 ? (void *)&short_key : &key,
> +		  test->key_size, json);
> +	hex_bytes(value_hex, &value, test->value_size, json);

[ ... ]

> +static void run_dump_case(const struct dump_case *test)
> +{

[ ... ]

> +	for (i = 0; i < test->count; i++) {
> +		__u32 key = i, value = i * 37 + 11;
> +		unsigned char short_key = i;
> +		void *key_ptr = test->key_size == 1 ? (void *)&short_key : &key;
> +
> +		if (!ASSERT_OK(bpf_map_update_elem(fd, key_ptr, &value, BPF_ANY),
> +			       "map_update"))
> +			goto out;
> +	}

This isn't a bug, but the key/value derivation appears in both
expected_record() and run_dump_case() with the same formula (index * 37
+ 11) and the same short_key narrowing logic. Would it be worth having
a single small helper that derives the key pointer and value for an
index, so the insert side and expectation side cannot drift apart?


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34074207177

Re: [PATCH bpf-next v2 1/2] bpftool: Use batch lookups for bounded hash map dumps

[bot+bpf-ci]

[-- Attachment #1: Type: text/plain, Size: 3511 bytes --]

> bpftool: Use batch lookups for bounded hash map dumps
>
> Use BPF_MAP_LOOKUP_BATCH when dumping hash maps to reduce the number
> of BPF syscalls. Share element formatting with individual lookups to
> preserve plain, JSON and BTF output.
>
> Start with up to 256 entries and grow on ENOSPC without advancing the
> input cursor. Restrict the optimization to maps whose maximum key and
> value storage fits in 4 MiB, so even a worst-case bucket can fit without
> restarting a partially printed dump. Preserve aligned element buffers
> for formatting keys and values with odd sizes.
>
> Fall back to individual lookups only if the initial batch operation is
> unsupported. Process the final partial batch on ENOENT, but never use
> count or output buffers after other errors. Report errors after batch
> traversal starts without restarting and duplicating output.
>
> Send batch failure diagnostics to stderr so JSON element arrays contain
> only map entries.

This isn't a bug, but would it help to include performance numbers in the
commit message? The stated motivation is reducing the number of BPF
syscalls, but the last three paragraphs describe what each part of
dump_map_batch() does rather than showing how much the change gains. For
a patch whose purpose is syscall reduction, showing before and after
syscall counts or wall time for a representative dump would clarify the
benefit.

> diff --git a/tools/bpf/bpftool/map.c b/tools/bpf/bpftool/map.c
> index 684a8fb72414..ea6f1389245f 100644
> --- a/tools/bpf/bpftool/map.c
> +++ b/tools/bpf/bpftool/map.c

[ ... ]

> +#define MAP_DUMP_BATCH_SIZE 256U
> +#define MAP_DUMP_BATCH_MAX_BYTES (4 * 1024 * 1024)
> +
> +/* Return 1 to use individual lookups, but only before batch traversal starts. */
> +static int dump_map_batch(int fd, void *key, void *value,
> +			  struct bpf_map_info *info, struct btf *btf,
> +			  json_writer_t *wtr, unsigned int *num_elems)
> +{
> +	__u32 capacity, count, batch = 0, next_batch = 0, i;
> +	void *keys = NULL, *values = NULL, *buf;
> +	bool first = true, can_fallback = true;
> +	int err;
> +
> +	/*
> +	 * Hash lookup batches must accommodate a whole bucket. Restrict the
> +	 * optimization to maps whose worst-case bucket fits the memory budget,
> +	 * so a later ENOSPC never forces a restart after printing some entries.
> +	 * Division also bounds the allocation multiplications on 32-bit hosts.
> +	 */
> +	if (info->type != BPF_MAP_TYPE_HASH || !info->max_entries ||
> +	    (__u64)info->key_size + info->value_size >
> +	    MAP_DUMP_BATCH_MAX_BYTES / info->max_entries)
> +		return 1;

This isn't a bug, but does the eligibility test have the intended effect
for large maps? The check disables batching when (key_size + value_size)
exceeds MAP_DUMP_BATCH_MAX_BYTES / max_entries. For a hash map with 1
million entries and key_size = value_size = 4, that division yields 4, so
the test rejects it and forces individual lookups. The batching code
allocates starting at MAP_DUMP_BATCH_SIZE (256) entries and grows on
ENOSPC, so the 4 MiB budget could instead cap how far capacity is allowed
to grow rather than disqualifying the map entirely. That way large hash
maps would still benefit from batching, and only a genuinely oversized
bucket would hit the fallback path.


---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/34074207177

Re: [PATCH bpf-next v2 2/2] selftests/bpf: Check bpftool batch map dump contents

[Tianyi Chen]

Thanks. A helper could share the deterministic key/value derivation,
but these are two short, explicit calculations in a small selftest.
The one-byte key setup is also visible next to each use, and the test
already compares the inserted contents with the complete dump output.

I am leaving this unchanged: extracting the setup would be a readability
choice, rather than a correction to the test or its coverage.

Re: [PATCH bpf-next v2 1/2] bpftool: Use batch lookups for bounded hash map dumps

[Tianyi Chen]

Thanks for the review.

I have added the existing measurement to the local commit message for
the next full revision. For a 100,000-entry hash map in an x86-64 KVM
guest, BPF syscall counts decreased from 200,004 to 395. The median of
five untraced runs decreased from 0.774013 s to 0.733844 s, about 5.2% in
this measurement. Syscalls were counted separately with strace. The v2
change affects error reporting, not this traversal algorithm or bound.

The 4 MiB eligibility check is intentional. Hash batch lookup must fit
an entire bucket; ENOSPC leaves that bucket at the current cursor so it
can be retried with a larger buffer. The bound guarantees that growth
can accommodate even a worst-case bucket within the memory budget.

Capping growth for otherwise unbounded maps leaves a problem when a
later bucket is too large, after some entries have already been printed.
The individual-lookup fallback starts from NULL and would print those
entries again. The opaque batch cursor is a bucket cursor, not a key
from which bpf_map_get_next_key() can resume. Returning an error instead
would newly fail a dump that individual lookups could have completed.

Thus the example with one million 4-byte keys and 4-byte values does
intentionally use individual lookups. I am keeping the conservative
bound to preserve the output behavior and memory limit. Broadening it
would need an explicit solution for the post-output oversized-bucket
case, beyond just capping the allocation.