Linux Documentation
 help / color / mirror / Atom feed
From: Li Pengfei <ljdlns1987@gmail.com>
To: rostedt@goodmis.org, mhiramat@kernel.org
Cc: mathieu.desnoyers@efficios.com, mark.rutland@arm.com,
	corbet@lwn.net, skhan@linuxfoundation.org, lkp@intel.com,
	linux-trace-kernel@vger.kernel.org, linux-kernel@vger.kernel.org,
	linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org,
	zhangbo56@xiaomi.com, lipengfei28@xiaomi.com
Subject: [RFC PATCH v7 07/10] tools/tracing: add a parser for the stackmap binary export
Date: Sat, 12 Sep 2026 16:37:50 +0800	[thread overview]
Message-ID: <20260912083753.3426176-8-lipengfei28@xiaomi.com> (raw)
In-Reply-To: <20260912083753.3426176-1-lipengfei28@xiaomi.com>

From: Pengfei Li <lipengfei28@xiaomi.com>

Add stackmap_dump.py to decode native-endian version 1 stack_map_bin
streams from a file or stdin and print text or JSON.

Parse incrementally with bounded reads. Validate the header count,
require stack depths from 1 through 64, reject trailing bytes, and
report malformed input without a traceback.

--top rejects negative values, keeps zero as unlimited, and selects a
stable top N with an O(N) heap instead of materializing and sorting the
whole input. Normal text output is emitted record by record.

JSON output is staged in a SpooledTemporaryFile so malformed later
records or symbolization failures leave stdout empty. The spool keeps up
to 1 MiB in memory and then spills to disk.

With --vmlinux, addr2line receives addresses on stdin in batches bounded
by both 128 records and 128 unique addresses. This avoids ARG_MAX and
bounds retained records even when addresses repeat. Document that
KASLR-slid core addresses and module addresses are unsupported.

Install the script from the tools/tracing install target and add parser,
CLI, streaming, memory-bound and error-path tests. Report totals as
stack records rather than strict unique stacks.

Signed-off-by: Pengfei Li <lipengfei28@xiaomi.com>
---
 tools/tracing/Makefile                    |  17 +-
 tools/tracing/stackmap_dump.py            | 354 +++++++++++++++
 tools/tracing/tests/test_stackmap_dump.py | 498 ++++++++++++++++++++++
 3 files changed, 866 insertions(+), 3 deletions(-)
 create mode 100755 tools/tracing/stackmap_dump.py
 create mode 100644 tools/tracing/tests/test_stackmap_dump.py

diff --git a/tools/tracing/Makefile b/tools/tracing/Makefile
index 95e485f12d97..8c2be033e360 100644
--- a/tools/tracing/Makefile
+++ b/tools/tracing/Makefile
@@ -1,11 +1,22 @@
 # SPDX-License-Identifier: GPL-2.0
 include ../scripts/Makefile.include
 
+INSTALL ?= install
+BINDIR ?= /usr/bin
+PYTHON ?= python3
+
 all: latency rtla
 
+check:
+	$(PYTHON) -m unittest discover -s tests -v
+
 clean: latency_clean rtla_clean
 
-install: latency_install rtla_install
+install: latency_install rtla_install stackmap_install
+
+stackmap_install:
+	$(call QUIET_INSTALL,stackmap_dump.py)$(INSTALL) -D -m 755 stackmap_dump.py \
+		$(DESTDIR)$(BINDIR)/stackmap_dump.py
 
 latency:
 	$(call descend,latency)
@@ -25,5 +36,5 @@ rtla_install:
 rtla_clean:
 	$(call descend,rtla,clean)
 
-.PHONY: all install clean latency latency_install latency_clean \
-	rtla rtla_install rtla_clean
+.PHONY: all check install clean stackmap_install latency latency_install \
+	latency_clean rtla rtla_install rtla_clean
diff --git a/tools/tracing/stackmap_dump.py b/tools/tracing/stackmap_dump.py
new file mode 100755
index 000000000000..cf461b49cee7
--- /dev/null
+++ b/tools/tracing/stackmap_dump.py
@@ -0,0 +1,354 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+"""
+stackmap_dump.py - Parse and display ftrace stack_map_bin binary export.
+
+Usage:
+    # Read straight from the device over a pipe
+    adb shell cat /sys/kernel/debug/tracing/stack_map_bin | stackmap_dump.py
+
+    # Or pull the file first and parse it
+    adb pull /sys/kernel/debug/tracing/stack_map_bin /tmp/stack_map.bin
+    python3 stackmap_dump.py /tmp/stack_map.bin
+
+    # With vmlinux when addresses already match its link-time range
+    # (for example, a nokaslr core kernel; modules are not supported)
+    python3 stackmap_dump.py /tmp/stack_map.bin --vmlinux vmlinux
+
+    # JSON output for tooling
+    python3 stackmap_dump.py /tmp/stack_map.bin --json
+"""
+
+import argparse
+import heapq
+import io
+import json
+import struct
+import subprocess
+import sys
+import tempfile
+
+MAGIC = 0x46534D42  # 'FSMB'
+HEADER_SIZE = 16  # 4 x u32
+ENTRY_SIZE = 16   # 4 x u32
+MAX_STACK_DEPTH_V1 = 64
+MAX_STACKS_V1 = 1 << 18
+SYMBOL_BATCH_RECORDS = 128
+SYMBOL_BATCH_ADDRESSES = 128
+SYMBOL_MAX_BATCHES = 64
+SYMBOL_MAX_ADDRESSES = 8192
+JSON_SPOOL_MEMORY_LIMIT = 1 << 20
+COPY_BUFFER_SIZE = 64 << 10
+
+# __ftrace_trace_stack() replaces trampoline addresses with this marker
+# before storing the stack, so the binary export carries it verbatim.
+FTRACE_TRAMPOLINE_MARKER = 0x7fffffff
+TRAMPOLINE_LABEL = '[FTRACE TRAMPOLINE]'
+
+
+def detect_endianness(data):
+    """Detect byte order from magic number in header."""
+    if len(data) < 4:
+        raise ValueError("File too small")
+    magic_le = struct.unpack_from('<I', data, 0)[0]
+    if magic_le == MAGIC:
+        return '<'
+    magic_be = struct.unpack_from('>I', data, 0)[0]
+    if magic_be == MAGIC:
+        return '>'
+    raise ValueError(f"Bad magic: 0x{magic_le:08x} (neither LE nor BE)")
+
+
+def batch_addr2line(vmlinux, addrs):
+    """Resolve multiple addresses in one addr2line invocation."""
+    if not addrs:
+        return {}
+    try:
+        # Use stdin to avoid ARG_MAX with large address sets.
+        stdin = '\n'.join(hex(a) for a in addrs) + '\n'
+        result = subprocess.run(
+            ['addr2line', '-f', '-e', vmlinux],
+            input=stdin, capture_output=True, text=True, timeout=60
+        )
+    except subprocess.TimeoutExpired as error:
+        raise RuntimeError("addr2line failed: timed out") from error
+    except FileNotFoundError as error:
+        raise RuntimeError("addr2line failed: executable not found") from error
+    except OSError as error:
+        detail = error.strerror or str(error)
+        raise RuntimeError(f"addr2line failed: {detail}") from error
+
+    if result.returncode:
+        detail = result.stderr.strip() or f"exit status {result.returncode}"
+        raise RuntimeError(f"addr2line failed: {detail}")
+
+    lines = result.stdout.splitlines()
+    expected_lines = len(addrs) * 2
+    if (len(lines) != expected_lines or
+            any(not line for line in lines)):
+        raise RuntimeError(
+            "addr2line returned malformed output: "
+            f"expected {expected_lines} non-empty lines, got {len(lines)}")
+
+    symbols = {}
+    for i, addr in enumerate(addrs):
+        function = lines[i * 2]
+        if function != '??':
+            symbols[addr] = function
+    return symbols
+
+
+def read_exact(stream, size, error_message):
+    """Read exactly size bytes from a stream that may return short reads."""
+    chunks = []
+    remaining = size
+    while remaining:
+        chunk = stream.read(remaining)
+        if not chunk:
+            raise ValueError(error_message)
+        chunks.append(chunk)
+        remaining -= len(chunk)
+    return b''.join(chunks)
+
+
+def parse_stackmap_stream(stream):
+    """Yield (stack_id, ref_count, ips) tuples from a binary stream."""
+    header = read_exact(stream, HEADER_SIZE, "File too small for header")
+    endian = detect_endianness(header)
+    header_fmt = f'{endian}IIII'
+    entry_fmt = f'{endian}IIII'
+
+    _magic, version, nr_stacks, _reserved = struct.unpack(header_fmt, header)
+    if version != 1:
+        raise ValueError(f"Unsupported version: {version}")
+    if nr_stacks > MAX_STACKS_V1:
+        raise ValueError(
+            f"Invalid stack count {nr_stacks}: version 1 maximum is "
+            f"{MAX_STACKS_V1}")
+
+    for _ in range(nr_stacks):
+        entry = read_exact(stream, ENTRY_SIZE,
+                           "Truncated stack entry header")
+        stack_id, nr, ref_count, _reserved = struct.unpack(entry_fmt, entry)
+        if nr == 0:
+            raise ValueError(
+                "Invalid stack depth 0: version 1 minimum is 1")
+        if nr > MAX_STACK_DEPTH_V1:
+            raise ValueError(
+                f"Invalid stack depth {nr}: version 1 maximum is "
+                f"{MAX_STACK_DEPTH_V1}")
+
+        ips_data = read_exact(
+            stream, nr * 8,
+            f"Truncated stack IP data for stack_id {stack_id}")
+        ips = struct.unpack(f'{endian}{nr}Q', ips_data)
+        yield stack_id, ref_count, list(ips)
+
+    if stream.read(1):
+        raise ValueError(f"Trailing data after {nr_stacks} stack records")
+
+
+def parse_stackmap_bin(data):
+    """Parse in-memory binary data using the streaming parser."""
+    return list(parse_stackmap_stream(io.BytesIO(data)))
+
+
+def non_negative_int(value):
+    number = int(value)
+    if number < 0:
+        raise argparse.ArgumentTypeError("--top must be non-negative")
+    return number
+
+
+def select_top(records, limit):
+    """Select the largest ref_counts stably using O(limit) space."""
+    heap = []
+    for order, record in enumerate(records):
+        item = (record[1], -order, order, record)
+        if len(heap) < limit:
+            heapq.heappush(heap, item)
+        elif item[:2] > heap[0][:2]:
+            heapq.heapreplace(heap, item)
+
+    heap.sort(key=lambda item: (-item[0], item[2]))
+    return [item[3] for item in heap]
+
+
+def iter_symbol_batches(records):
+    """Group records without exceeding the unique-address limit."""
+    batch = []
+    addresses = set()
+    for record in records:
+        _stack_id, _ref_count, ips = record
+        record_addresses = {
+            ip for ip in ips if ip != FTRACE_TRAMPOLINE_MARKER
+        }
+        if (batch and
+                (len(batch) >= SYMBOL_BATCH_RECORDS or
+                 len(addresses | record_addresses) >
+                 SYMBOL_BATCH_ADDRESSES)):
+            yield batch, addresses
+            batch.clear()
+            addresses.clear()
+        batch.append(record)
+        addresses.update(record_addresses)
+    if batch:
+        yield batch, addresses
+
+
+class RecordRenderer:
+    """Write records incrementally without changing text or JSON formats."""
+
+    def __init__(self, output, json_output, include_symbols):
+        self.output = output
+        self.json_output = json_output
+        self.include_symbols = include_symbols
+        self.count = 0
+
+    @staticmethod
+    def terminal_safe(text):
+        """Escape terminal controls while preserving printable Unicode."""
+        escaped = []
+        for character in text:
+            codepoint = ord(character)
+            if character.isprintable():
+                escaped.append(character)
+            elif codepoint <= 0xff:
+                escaped.append(f'\\x{codepoint:02x}')
+            elif codepoint <= 0xffff:
+                escaped.append(f'\\u{codepoint:04x}')
+            else:
+                escaped.append(f'\\U{codepoint:08x}')
+        return ''.join(escaped)
+
+    @staticmethod
+    def render_ip(ip, symbols):
+        if ip == FTRACE_TRAMPOLINE_MARKER:
+            return TRAMPOLINE_LABEL
+        return symbols.get(ip, f'0x{ip:x}')
+
+    def emit(self, record, symbols):
+        stack_id, ref_count, ips = record
+        if self.json_output:
+            entry = {
+                'stack_id': stack_id,
+                'ref_count': ref_count,
+                'ips': [f'0x{ip:x}' for ip in ips]
+            }
+            if self.include_symbols:
+                entry['symbols'] = [self.render_ip(ip, symbols)
+                                    for ip in ips]
+            encoded = json.dumps(entry, indent=2)
+            indented = '\n'.join(f'  {line}' for line in encoded.splitlines())
+            if self.count == 0:
+                self.output.write('[\n')
+            else:
+                self.output.write(',\n')
+            self.output.write(indented)
+        else:
+            self.output.write(
+                f"stack_id {stack_id} [ref {ref_count}, depth {len(ips)}]\n")
+            for index, ip in enumerate(ips):
+                if ip == FTRACE_TRAMPOLINE_MARKER:
+                    self.output.write(
+                        f"  [{index}] {TRAMPOLINE_LABEL}\n")
+                    continue
+                symbol = symbols.get(ip, '')
+                if symbol:
+                    symbol = f' {self.terminal_safe(symbol)}'
+                self.output.write(f"  [{index}] 0x{ip:x}{symbol}\n")
+            self.output.write('\n')
+        self.count += 1
+
+    def finish(self):
+        if not self.json_output:
+            return
+        if self.count:
+            self.output.write('\n]\n')
+        else:
+            self.output.write('[]\n')
+
+
+def render_records(records, args, output):
+    renderer = RecordRenderer(output, args.json, bool(args.vmlinux))
+    if args.vmlinux:
+        batch_count = 0
+        address_count = 0
+        for batch, addresses in iter_symbol_batches(records):
+            batch_count += 1
+            address_count += len(addresses)
+            if (batch_count > SYMBOL_MAX_BATCHES or
+                    address_count > SYMBOL_MAX_ADDRESSES):
+                raise RuntimeError(
+                    "symbolization work limit exceeded: "
+                    f"maximum {SYMBOL_MAX_BATCHES} batches and "
+                    f"{SYMBOL_MAX_ADDRESSES} addresses")
+            symbols = batch_addr2line(args.vmlinux, list(addresses))
+            for record in batch:
+                renderer.emit(record, symbols)
+    else:
+        for record in records:
+            renderer.emit(record, {})
+    renderer.finish()
+    return renderer.count
+
+
+def main(argv=None, input_file=None, output=None, error=None):
+    parser = argparse.ArgumentParser(description='Parse ftrace stack_map_bin')
+    parser.add_argument('file', nargs='?', default='-',
+                        help="Path to stack_map_bin file, or '-' for stdin "
+                             "(the default)")
+    parser.add_argument(
+        '--vmlinux',
+        help=('Path to vmlinux for symbol resolution; addresses must already '
+              'match vmlinux (for example, nokaslr). KASLR runtime addresses '
+              'and module addresses are not supported. Symbolization is '
+              'limited to 8192 addresses in 64 batches'))
+    parser.add_argument('--json', action='store_true', help='JSON output')
+    parser.add_argument('--top', type=non_negative_int, default=0,
+                        help='Show only top N stacks by ref_count; 0 is unlimited')
+    args = parser.parse_args(argv)
+
+    output = output if output is not None else sys.stdout
+    error = error if error is not None else sys.stderr
+    stream = input_file
+    close_stream = False
+
+    try:
+        # stdin accepts a stack_map_bin stream directly from a pipe.
+        if stream is None:
+            if args.file == '-':
+                stream = sys.stdin.buffer
+            else:
+                stream = open(args.file, 'rb')
+                close_stream = True
+
+        records = parse_stackmap_stream(stream)
+        if args.top:
+            records = select_top(records, args.top)
+        if args.json:
+            with tempfile.SpooledTemporaryFile(
+                    max_size=JSON_SPOOL_MEMORY_LIMIT, mode='w+',
+                    encoding='utf-8') as staged_output:
+                count = render_records(records, args, staged_output)
+                staged_output.seek(0)
+                while True:
+                    chunk = staged_output.read(COPY_BUFFER_SIZE)
+                    if not chunk:
+                        break
+                    output.write(chunk)
+        else:
+            count = render_records(records, args, output)
+    except (ValueError, OSError, RuntimeError) as exception:
+        print(f"error: {exception}", file=error)
+        return 1
+    finally:
+        if close_stream:
+            stream.close()
+
+    print(f"Total: {count} stack records", file=error)
+    return 0
+
+
+if __name__ == '__main__':
+    sys.exit(main())
diff --git a/tools/tracing/tests/test_stackmap_dump.py b/tools/tracing/tests/test_stackmap_dump.py
new file mode 100644
index 000000000000..03769cbc190e
--- /dev/null
+++ b/tools/tracing/tests/test_stackmap_dump.py
@@ -0,0 +1,498 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: GPL-2.0
+
+import importlib.util
+import io
+import json
+import os
+from pathlib import Path
+import struct
+import subprocess
+import sys
+import tempfile
+import unittest
+from unittest import mock
+
+SCRIPT = Path(__file__).resolve().parents[1] / "stackmap_dump.py"
+SPEC = importlib.util.spec_from_file_location("stackmap_dump", SCRIPT)
+stackmap_dump = importlib.util.module_from_spec(SPEC)
+SPEC.loader.exec_module(stackmap_dump)
+
+
+def image(endian="<", version=1, entries=()):
+    data = bytearray(struct.pack(f"{endian}IIII", stackmap_dump.MAGIC,
+                                 version, len(entries), 0))
+    for stack_id, ref_count, ips in entries:
+        data += struct.pack(f"{endian}IIII", stack_id, len(ips),
+                            ref_count, 0)
+        data += struct.pack(f"{endian}{len(ips)}Q", *ips)
+    return bytes(data)
+
+
+class ShortReadInput(io.BytesIO):
+    def read(self, size=-1):
+        if size < 0:
+            raise AssertionError("unbounded read() is not allowed")
+        if size > stackmap_dump.MAX_STACK_DEPTH_V1 * 8:
+            raise AssertionError("read() exceeds one version 1 stack")
+        return super().read(min(size, 3))
+
+
+class StreamingGuardInput(io.BytesIO):
+    def __init__(self, data, first_record_end, output):
+        super().__init__(data)
+        self.first_record_end = first_record_end
+        self.output = output
+
+    def read(self, size=-1):
+        if (self.tell() >= self.first_record_end and
+                "stack_id 1" not in self.output.getvalue()):
+            raise AssertionError("next record read before current output")
+        return super().read(size)
+
+
+class TrackedRecord:
+    live_count = 0
+    max_live_count = 0
+
+    def __init__(self, stack_id, ip=None):
+        self.stack_id = stack_id
+        self.ip = 0x1000 + stack_id if ip is None else ip
+        type(self).live_count += 1
+        type(self).max_live_count = max(type(self).max_live_count,
+                                        type(self).live_count)
+
+    def __iter__(self):
+        return iter((self.stack_id, 1, [self.ip]))
+
+    def __del__(self):
+        type(self).live_count -= 1
+
+
+class StackmapParserTest(unittest.TestCase):
+    def test_little_endian_entry(self):
+        raw = image(entries=[(7, 3, [0x1234, 0x7fffffff])])
+        self.assertEqual(list(stackmap_dump.parse_stackmap_bin(raw)),
+                         [(7, 3, [0x1234, 0x7fffffff])])
+
+    def test_big_endian_entry(self):
+        raw = image(">", entries=[(9, 2, [0xabcdef])])
+        self.assertEqual(list(stackmap_dump.parse_stackmap_bin(raw)),
+                         [(9, 2, [0xabcdef])])
+
+    def test_bad_magic(self):
+        with self.assertRaisesRegex(ValueError, "Bad magic"):
+            list(stackmap_dump.parse_stackmap_bin(b"BAD!" + bytes(12)))
+
+    def test_unknown_version(self):
+        with self.assertRaisesRegex(ValueError, "Unsupported version"):
+            list(stackmap_dump.parse_stackmap_bin(image(version=2)))
+
+    def test_truncated_header(self):
+        with self.assertRaisesRegex(ValueError, "too small for header"):
+            list(stackmap_dump.parse_stackmap_bin(bytes(15)))
+
+    def test_truncated_entry_header(self):
+        raw = struct.pack("<IIII", stackmap_dump.MAGIC, 1, 1, 0)
+        with self.assertRaisesRegex(ValueError, "Truncated stack entry header"):
+            list(stackmap_dump.parse_stackmap_bin(raw))
+
+    def test_truncated_ip_data(self):
+        raw = image(entries=[(1, 1, [0x1234])])[:-1]
+        with self.assertRaisesRegex(ValueError, "Truncated stack IP data"):
+            list(stackmap_dump.parse_stackmap_bin(raw))
+
+    def test_rejects_records_beyond_header_count(self):
+        raw = image(entries=[(1, 1, [0x1234])])
+        raw = bytearray(raw)
+        struct.pack_into("<I", raw, 8, 0)
+        with self.assertRaisesRegex(ValueError, "Trailing data"):
+            stackmap_dump.parse_stackmap_bin(raw)
+
+    def test_cli_reports_malformed_input_without_traceback(self):
+        result = subprocess.run([sys.executable, str(SCRIPT), "-"],
+                                input=b"bad", capture_output=True)
+        stderr = result.stderr.decode()
+        self.assertNotEqual(result.returncode, 0)
+        self.assertIn("error: File too small for header", stderr)
+        self.assertNotIn("Traceback", stderr)
+
+    def test_cli_reports_missing_file_without_traceback(self):
+        missing = SCRIPT.parent / "does-not-exist.stackmap"
+        result = subprocess.run([sys.executable, str(SCRIPT), str(missing)],
+                                capture_output=True, text=True)
+        self.assertNotEqual(result.returncode, 0)
+        self.assertIn("error:", result.stderr)
+        self.assertNotIn("Traceback", result.stderr)
+
+    def test_cli_reports_addr2line_failure(self):
+        raw = image(entries=[(1, 2, [0x1234])])
+        with tempfile.TemporaryDirectory() as temp_dir:
+            addr2line = Path(temp_dir) / "addr2line"
+            addr2line.write_text(
+                "#!/bin/sh\necho controlled addr2line failure >&2\nexit 23\n",
+                encoding="utf-8")
+            addr2line.chmod(0o755)
+            env = os.environ.copy()
+            env["PATH"] = temp_dir
+            result = subprocess.run(
+                [sys.executable, str(SCRIPT), "-", "--vmlinux", "vmlinux"],
+                input=raw, capture_output=True, env=env)
+        stderr = result.stderr.decode()
+        self.assertNotEqual(result.returncode, 0)
+        self.assertIn(
+            "error: addr2line failed: controlled addr2line failure", stderr)
+        self.assertNotIn("Traceback", stderr)
+
+    def test_cli_reports_truncated_addr2line_output(self):
+        raw = image(entries=[(1, 2, [0x1234])])
+        with tempfile.TemporaryDirectory() as temp_dir:
+            addr2line = Path(temp_dir) / "addr2line"
+            addr2line.write_text(
+                "#!/bin/sh\nprintf 'resolved_function\\n'\n",
+                encoding="utf-8")
+            addr2line.chmod(0o755)
+            env = os.environ.copy()
+            env["PATH"] = temp_dir
+            result = subprocess.run(
+                [sys.executable, str(SCRIPT), "-", "--vmlinux", "vmlinux"],
+                input=raw, capture_output=True, env=env)
+        stderr = result.stderr.decode()
+        self.assertNotEqual(result.returncode, 0)
+        self.assertIn("error: addr2line returned malformed output", stderr)
+        self.assertNotIn("Traceback", stderr)
+
+    def test_cli_accepts_unresolved_addr2line_output(self):
+        raw = image(entries=[(1, 2, [0x1234])])
+        with tempfile.TemporaryDirectory() as temp_dir:
+            addr2line = Path(temp_dir) / "addr2line"
+            addr2line.write_text(
+                "#!/bin/sh\nprintf '??\\n??:0\\n'\n",
+                encoding="utf-8")
+            addr2line.chmod(0o755)
+            env = os.environ.copy()
+            env["PATH"] = temp_dir
+            result = subprocess.run(
+                [sys.executable, str(SCRIPT), "-", "--vmlinux", "vmlinux"],
+                input=raw, capture_output=True, env=env)
+        self.assertEqual(result.returncode, 0, result.stderr.decode())
+        self.assertIn(b"[0] 0x1234", result.stdout)
+
+    def test_cli_rejects_trailing_data(self):
+        raw = image(entries=[(1, 1, [0x1234])]) + b"junk"
+        result = subprocess.run([sys.executable, str(SCRIPT), "-"],
+                                input=raw, capture_output=True)
+        self.assertNotEqual(result.returncode, 0)
+        self.assertIn(b"error: Trailing data", result.stderr)
+
+    def test_cli_accepts_valid_input(self):
+        raw = image(entries=[(1, 2, [0x1234])])
+        result = subprocess.run([sys.executable, str(SCRIPT), "-", "--json"],
+                                input=raw, capture_output=True)
+        self.assertEqual(result.returncode, 0, result.stderr.decode())
+        self.assertIn(b'"stack_id": 1', result.stdout)
+
+    def test_stream_parser_handles_short_reads(self):
+        raw = image(entries=[(7, 3, [0x1234, 0x5678])])
+        self.assertEqual(list(stackmap_dump.parse_stackmap_stream(
+            ShortReadInput(raw))), [(7, 3, [0x1234, 0x5678])])
+
+    def test_rejects_stack_depth_above_version_1_limit(self):
+        raw = (struct.pack("<IIII", stackmap_dump.MAGIC, 1, 1, 0) +
+               struct.pack("<IIII", 7, 65, 1, 0))
+        with self.assertRaisesRegex(ValueError, "stack depth.*64"):
+            list(stackmap_dump.parse_stackmap_stream(io.BytesIO(raw)))
+
+    def test_accepts_stack_depth_at_version_1_limit(self):
+        ips = list(range(stackmap_dump.MAX_STACK_DEPTH_V1))
+        raw = image(entries=[(7, 1, ips)])
+        self.assertEqual(stackmap_dump.parse_stackmap_bin(raw), [(7, 1, ips)])
+
+    def test_rejects_stack_count_above_version_1_limit(self):
+        raw = struct.pack("<IIII", stackmap_dump.MAGIC, 1,
+                          (1 << 18) + 1, 0)
+        with self.assertRaisesRegex(ValueError, "stack count.*262144"):
+            list(stackmap_dump.parse_stackmap_stream(io.BytesIO(raw)))
+
+    def test_accepts_stack_count_at_version_1_limit(self):
+        raw = struct.pack("<IIII", stackmap_dump.MAGIC, 1, 1 << 18, 0)
+        with self.assertRaisesRegex(ValueError,
+                                    "Truncated stack entry header"):
+            list(stackmap_dump.parse_stackmap_stream(io.BytesIO(raw)))
+
+    def test_cli_rejects_negative_top(self):
+        raw = image(entries=[(1, 2, [0x1234])])
+        result = subprocess.run(
+            [sys.executable, str(SCRIPT), "-", "--top", "-1"],
+            input=raw, capture_output=True)
+        self.assertNotEqual(result.returncode, 0)
+        self.assertIn(b"--top must be non-negative", result.stderr)
+
+    def test_cli_top_zero_remains_unlimited(self):
+        raw = image(entries=[(1, 2, [0x1000]), (2, 1, [0x2000])])
+        result = subprocess.run(
+            [sys.executable, str(SCRIPT), "-", "--json", "--top", "0"],
+            input=raw, capture_output=True)
+        self.assertEqual(result.returncode, 0, result.stderr.decode())
+        self.assertEqual([entry["stack_id"] for entry in
+                          json.loads(result.stdout)], [1, 2])
+
+    def test_cli_top_n_is_descending_and_stable_for_ties(self):
+        raw = image(entries=[(1, 3, [0x1000]),
+                             (2, 9, [0x2000]),
+                             (3, 9, [0x3000]),
+                             (4, 9, [0x4000]),
+                             (5, 1, [0x5000])])
+        result = subprocess.run(
+            [sys.executable, str(SCRIPT), "-", "--json", "--top", "2"],
+            input=raw, capture_output=True)
+        self.assertEqual(result.returncode, 0, result.stderr.decode())
+        self.assertEqual([(entry["stack_id"], entry["ref_count"])
+                          for entry in json.loads(result.stdout)],
+                         [(2, 9), (3, 9)])
+
+    def test_rejects_zero_stack_depth(self):
+        raw = (struct.pack("<IIII", stackmap_dump.MAGIC, 1, 1, 0) +
+               struct.pack("<IIII", 7, 0, 1, 0))
+        with self.assertRaisesRegex(ValueError, "stack depth.*minimum is 1"):
+            list(stackmap_dump.parse_stackmap_stream(io.BytesIO(raw)))
+
+    def test_cli_empty_text_output_format(self):
+        result = subprocess.run([sys.executable, str(SCRIPT), "-"],
+                                input=image(), capture_output=True)
+        self.assertEqual(result.returncode, 0, result.stderr.decode())
+        self.assertEqual(result.stdout, b"")
+        self.assertEqual(result.stderr, b"Total: 0 stack records\n")
+
+    def test_cli_empty_json_output_format(self):
+        result = subprocess.run([sys.executable, str(SCRIPT), "-", "--json"],
+                                input=image(), capture_output=True)
+        self.assertEqual(result.returncode, 0, result.stderr.decode())
+        self.assertEqual(result.stdout, b"[]\n")
+        self.assertEqual(result.stderr, b"Total: 0 stack records\n")
+
+    def test_cli_multi_record_text_output_format(self):
+        raw = image(entries=[(7, 3, [0x1234,
+                                      stackmap_dump.FTRACE_TRAMPOLINE_MARKER]),
+                             (9, 1, [0xabcd])])
+        result = subprocess.run([sys.executable, str(SCRIPT), "-"],
+                                input=raw, capture_output=True)
+        self.assertEqual(result.returncode, 0, result.stderr.decode())
+        self.assertEqual(
+            result.stdout.decode(),
+            "stack_id 7 [ref 3, depth 2]\n"
+            "  [0] 0x1234\n"
+            "  [1] [FTRACE TRAMPOLINE]\n\n"
+            "stack_id 9 [ref 1, depth 1]\n"
+            "  [0] 0xabcd\n\n")
+        self.assertEqual(result.stderr, b"Total: 2 stack records\n")
+
+    def test_cli_multi_record_json_output_format(self):
+        raw = image(entries=[(7, 3, [0x1234]), (9, 1, [0xabcd])])
+        result = subprocess.run([sys.executable, str(SCRIPT), "-", "--json"],
+                                input=raw, capture_output=True)
+        self.assertEqual(result.returncode, 0, result.stderr.decode())
+        expected = json.dumps([
+            {"stack_id": 7, "ref_count": 3, "ips": ["0x1234"]},
+            {"stack_id": 9, "ref_count": 1, "ips": ["0xabcd"]},
+        ], indent=2) + "\n"
+        self.assertEqual(result.stdout.decode(), expected)
+        self.assertEqual(result.stderr, b"Total: 2 stack records\n")
+
+    def test_vmlinux_text_output_format(self):
+        raw = image(entries=[(7, 3, [
+            0x1000, 0x2000, stackmap_dump.FTRACE_TRAMPOLINE_MARKER])])
+        output = io.StringIO()
+        error = io.StringIO()
+        with mock.patch.object(stackmap_dump, "batch_addr2line",
+                               return_value={0x1000: "resolved"}):
+            result = stackmap_dump.main(
+                ["--vmlinux", "vmlinux"], input_file=io.BytesIO(raw),
+                output=output, error=error)
+
+        self.assertEqual(result, 0, error.getvalue())
+        self.assertEqual(
+            output.getvalue(),
+            "stack_id 7 [ref 3, depth 3]\n"
+            "  [0] 0x1000 resolved\n"
+            "  [1] 0x2000\n"
+            "  [2] [FTRACE TRAMPOLINE]\n\n")
+        self.assertEqual(error.getvalue(), "Total: 1 stack records\n")
+
+    def test_vmlinux_json_output_format(self):
+        raw = image(entries=[(7, 3, [
+            0x1000, 0x2000, stackmap_dump.FTRACE_TRAMPOLINE_MARKER])])
+        output = io.StringIO()
+        error = io.StringIO()
+        with mock.patch.object(stackmap_dump, "batch_addr2line",
+                               return_value={0x1000: "resolved"}):
+            result = stackmap_dump.main(
+                ["--json", "--vmlinux", "vmlinux"],
+                input_file=io.BytesIO(raw), output=output, error=error)
+
+        expected = json.dumps([{
+            "stack_id": 7,
+            "ref_count": 3,
+            "ips": ["0x1000", "0x2000", "0x7fffffff"],
+            "symbols": ["resolved", "0x2000", "[FTRACE TRAMPOLINE]"],
+        }], indent=2) + "\n"
+        self.assertEqual(result, 0, error.getvalue())
+        self.assertEqual(output.getvalue(), expected)
+        self.assertEqual(error.getvalue(), "Total: 1 stack records\n")
+
+    def test_json_malformed_later_record_leaves_stdout_empty(self):
+        raw = image(entries=[(1, 2, [0x1000]), (2, 1, [0x2000])])[:-1]
+        output = io.StringIO()
+        error = io.StringIO()
+
+        result = stackmap_dump.main(["--json"], input_file=io.BytesIO(raw),
+                                    output=output, error=error)
+
+        self.assertEqual(result, 1)
+        self.assertEqual(output.getvalue(), "")
+        self.assertIn("error: Truncated stack IP data", error.getvalue())
+
+    def test_json_later_addr2line_failure_leaves_stdout_empty(self):
+        raw = image(entries=[(stack_id, 1, [0x1000 + stack_id])
+                             for stack_id in range(129)])
+        output = io.StringIO()
+        error = io.StringIO()
+        with mock.patch.object(
+                stackmap_dump, "batch_addr2line",
+                side_effect=[{}, RuntimeError("addr2line failed: later batch")]):
+            result = stackmap_dump.main(
+                ["--json", "--vmlinux", "vmlinux"],
+                input_file=io.BytesIO(raw), output=output, error=error)
+
+        self.assertEqual(result, 1)
+        self.assertEqual(output.getvalue(), "")
+        self.assertEqual(error.getvalue(),
+                         "error: addr2line failed: later batch\n")
+
+    def test_text_streams_before_reading_the_next_record(self):
+        raw = image(entries=[(1, 2, [0x1000]), (2, 1, [0x2000])])
+        output = io.StringIO()
+        first_record_end = (stackmap_dump.HEADER_SIZE +
+                            stackmap_dump.ENTRY_SIZE + 8)
+        input_file = StreamingGuardInput(raw, first_record_end, output)
+        error = io.StringIO()
+
+        result = stackmap_dump.main([], input_file=input_file,
+                                    output=output, error=error)
+
+        self.assertEqual(result, 0, error.getvalue())
+        self.assertIn("stack_id 2", output.getvalue())
+
+    def test_json_input_uses_only_bounded_reads(self):
+        raw = image(entries=[(1, 2, [0x1000]), (2, 1, [0x2000])])
+        output = io.StringIO()
+        error = io.StringIO()
+
+        result = stackmap_dump.main(
+            ["--json"], input_file=ShortReadInput(raw),
+            output=output, error=error)
+
+        self.assertEqual(result, 0, error.getvalue())
+        self.assertEqual([entry["stack_id"]
+                          for entry in json.loads(output.getvalue())], [1, 2])
+
+    def test_json_does_not_materialize_all_records(self):
+        TrackedRecord.live_count = 0
+        TrackedRecord.max_live_count = 0
+
+        def records(_stream):
+            for stack_id in range(1000):
+                yield TrackedRecord(stack_id)
+
+        output = io.StringIO()
+        with mock.patch.object(stackmap_dump, "parse_stackmap_stream",
+                               side_effect=records):
+            result = stackmap_dump.main(
+                ["--json"], input_file=io.BytesIO(), output=output,
+                error=io.StringIO())
+
+        self.assertEqual(result, 0)
+        self.assertLessEqual(TrackedRecord.max_live_count, 2)
+        self.assertEqual(len(json.loads(output.getvalue())), 1000)
+
+    def test_vmlinux_symbolization_limits_unique_addresses_per_call(self):
+        raw = image(entries=[
+            (stack_id, 1, [0x10000 + stack_id * 4 + frame
+                           for frame in range(4)])
+            for stack_id in range(50)])
+        calls = []
+
+        def resolve(_vmlinux, addrs):
+            calls.append(list(addrs))
+            return {}
+
+        with mock.patch.object(stackmap_dump, "batch_addr2line",
+                               side_effect=resolve):
+            result = stackmap_dump.main(
+                ["--vmlinux", "vmlinux"], input_file=io.BytesIO(raw),
+                output=io.StringIO(), error=io.StringIO())
+
+        self.assertEqual(result, 0)
+        self.assertLessEqual(max(map(len, calls)), 128)
+        self.assertEqual(stackmap_dump.SYMBOL_BATCH_ADDRESSES, 128)
+
+    def test_vmlinux_duplicate_addresses_keep_record_batches_bounded(self):
+        TrackedRecord.live_count = 0
+        TrackedRecord.max_live_count = 0
+
+        def records(_stream):
+            for stack_id in range(1000):
+                yield TrackedRecord(stack_id, ip=0x1000)
+
+        with mock.patch.object(stackmap_dump, "parse_stackmap_stream",
+                               side_effect=records), mock.patch.object(
+                                   stackmap_dump, "batch_addr2line",
+                                   return_value={}):
+            result = stackmap_dump.main(
+                ["--json", "--vmlinux", "vmlinux"],
+                input_file=io.BytesIO(), output=io.StringIO(),
+                error=io.StringIO())
+
+        self.assertEqual(result, 0)
+        self.assertLessEqual(TrackedRecord.max_live_count, 130)
+
+    def test_vmlinux_rejects_aggregate_symbolization_work_above_limit(self):
+        raw = image(entries=[(stack_id, 1, [0x10000 + stack_id])
+                             for stack_id in range(8193)])
+        error = io.StringIO()
+        with mock.patch.object(stackmap_dump, "batch_addr2line",
+                               return_value={}):
+            result = stackmap_dump.main(
+                ["--vmlinux", "vmlinux"], input_file=io.BytesIO(raw),
+                output=io.StringIO(), error=error)
+
+        self.assertNotEqual(result, 0)
+        self.assertIn("symbolization work limit", error.getvalue())
+
+    def test_vmlinux_text_output_escapes_terminal_control_characters(self):
+        raw = image(entries=[(7, 3, [0x1000])])
+        output = io.StringIO()
+        with mock.patch.object(
+                stackmap_dump, "batch_addr2line",
+                return_value={0x1000: "name\x1b[2J\rspoof\tend"}):
+            result = stackmap_dump.main(
+                ["--vmlinux", "vmlinux"], input_file=io.BytesIO(raw),
+                output=output, error=io.StringIO())
+
+        self.assertEqual(result, 0)
+        self.assertIn(r"name\x1b[2J\x0dspoof\x09end", output.getvalue())
+        self.assertNotIn("\x1b", output.getvalue())
+        self.assertNotIn("\r", output.getvalue())
+        self.assertNotIn("\t", output.getvalue())
+
+    def test_help_states_vmlinux_kaslr_and_work_limits(self):
+        result = subprocess.run([sys.executable, str(SCRIPT), "--help"],
+                                capture_output=True, text=True, check=True)
+        self.assertIn("KASLR", result.stdout)
+        self.assertIn("module", result.stdout)
+        self.assertIn("8192", result.stdout)
+        self.assertIn("64 batches", result.stdout)
+
+
+if __name__ == "__main__":
+    unittest.main()
-- 
2.34.1


  parent reply	other threads:[~2026-09-12  8:39 UTC|newest]

Thread overview: 11+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-09-12  8:37 [RFC PATCH v7 00/10] trace: stack trace deduplication for ftrace ring buffer Li Pengfei
2026-09-12  8:37 ` [RFC PATCH v7 01/10] trace: add lock-free stackmap for stack trace deduplication Li Pengfei
2026-09-12  8:37 ` [RFC PATCH v7 02/10] trace: use the stackmap from the ftrace stack recording path Li Pengfei
2026-09-12  8:37 ` [RFC PATCH v7 03/10] trace: add stackmap statistics interface Li Pengfei
2026-09-12  8:37 ` [RFC PATCH v7 04/10] trace: add stackmap binary export Li Pengfei
2026-09-12  8:37 ` [RFC PATCH v7 05/10] trace: make the stackmap capacity settable on the kernel command line Li Pengfei
2026-09-12  8:37 ` [RFC PATCH v7 06/10] Documentation: tracing: document the ftrace stackmap Li Pengfei
2026-09-12  8:37 ` Li Pengfei [this message]
2026-09-12  8:37 ` [RFC PATCH v7 08/10] selftests/ftrace: add a stackmap basic functionality test Li Pengfei
2026-09-12  8:37 ` [RFC PATCH v7 09/10] selftests/ftrace: add a stackmap reset and binary ABI test Li Pengfei
2026-09-12  8:37 ` [RFC PATCH v7 10/10] selftests/ftrace: add a stackmap instance gating test Li Pengfei

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260912083753.3426176-8-lipengfei28@xiaomi.com \
    --to=ljdlns1987@gmail.com \
    --cc=corbet@lwn.net \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-kselftest@vger.kernel.org \
    --cc=linux-trace-kernel@vger.kernel.org \
    --cc=lipengfei28@xiaomi.com \
    --cc=lkp@intel.com \
    --cc=mark.rutland@arm.com \
    --cc=mathieu.desnoyers@efficios.com \
    --cc=mhiramat@kernel.org \
    --cc=rostedt@goodmis.org \
    --cc=skhan@linuxfoundation.org \
    --cc=zhangbo56@xiaomi.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox