From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 19B3BFF885A for ; Sat, 25 Apr 2026 22:52:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:Cc:To:From: Subject:Message-ID:References:Mime-Version:In-Reply-To:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=NR1gzF3F6kI5f0yQJQV926hkNwQ4+t7RNPW4Qp/LJSM=; b=vgUiDxC67pZjp3qWyK9Ha9rexM 6sQeVQg+6C/RPTUiONel3CDag6uIfKRbzD/GGhkci3XOuNUdhVajUyuDhcv6PZ8xiZtC/Lo5c+VwB AOx/JuaT8ijceU3PCsnptP1atiKeqjPi8O0P1ZMXDvgcgq3CtqwrAoJFNoHlcM9aUVb+IA7XzxRy0 TmqDXEI4W/fvQaGEPXNH1OMoEghJ6w3X1kZBUIGyHV/VO5AtBPE4JoUMC+mw3gaShjDq6OZXPOg7R FZ4KU3pW4DUex72PZfQYKRyL2QPpcR598jqERe903ud1Ul9ngzm43bvU8y6WSzClVGUNG0vtjuOYN SbLLEGFw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.2 #2 (Red Hat Linux)) id 1wGlqv-0000000EzcR-2VXF; Sat, 25 Apr 2026 22:51:57 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.98.2 #2 (Red Hat Linux)) id 1wGlqN-0000000EywL-2t7F for linux-arm-kernel@bombadil.infradead.org; Sat, 25 Apr 2026 22:51:24 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:Cc:To:From:Subject: Message-ID:References:Mime-Version:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=NR1gzF3F6kI5f0yQJQV926hkNwQ4+t7RNPW4Qp/LJSM=; b=PBD8AMA/vjXvBOPod81uNo8gYi PNNtgi0W9odGYnFc8H1LoNzcyuTSkRzZmj2RhAJlUZ2IWGATJ+d5GDhCfdwR4j0ZcQwxbT45QfZM7 r+3/wNDpYj1ppTqNcE2niJbDlSKVReqzU4SZSulmmGWtRB+xRGnryqLLvJao5rSuQI7+LS8wd/DDL o9xOXUMIPUmJUoP4qHd9KQNPzcBmN3/+GOYJtImr1x3tc61XGD8+yuJEEcVmM28Wl8h5uBovzm+TJ MR3iDDWYOvuphaUxgcTno0jb8YAQKo/uuPGehYyYto4zVya4OHqk/ckn1BPrik3ypoGINnZpRSwod YFZ6JnDA==; Received: from mail-dy1-x134a.google.com ([2607:f8b0:4864:20::134a]) by desiato.infradead.org with esmtps (Exim 4.98.2 #2 (Red Hat Linux)) id 1wGlqK-0000000GMfR-119z for linux-arm-kernel@lists.infradead.org; Sat, 25 Apr 2026 22:51:22 +0000 Received: by mail-dy1-x134a.google.com with SMTP id 5a478bee46e88-2da19227bc1so20923538eec.1 for ; Sat, 25 Apr 2026 15:51:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1777157478; x=1777762278; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=NR1gzF3F6kI5f0yQJQV926hkNwQ4+t7RNPW4Qp/LJSM=; b=maPnSTzHGt0Z1Xp5YVX854qAaAAa9pJ3sr6L3fOBY6mOwGBZQ0uBMfBDChAeSACfCd pYOcGQRZNYssWEb/R+nz+4aZLCxDx1zGKsvpdl05vHnEyoC+NFD/gIAzIeSXXc583/nl rG0NiOgRoZtAfpRhkmnI5WtUztA0UbWBSw2kfQMiLU26jfZNJJVgnVVDv2lWBnlyY6Yj iZ1rffzqlwo7BJPcsRebun8k89lEYEx8pEsyJ9Vcvr5bzfaadlFxTwGUZ0Dao2A0IrRY oSbkEsnsFtEKN4LiHInGK01dp5kxOupW38Cfos//HcrbOCZOxV8ncXtcGxo4x3iz9kpH qZUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777157478; x=1777762278; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=NR1gzF3F6kI5f0yQJQV926hkNwQ4+t7RNPW4Qp/LJSM=; b=dDI/ypT/CSxXZWwAZ/NbsONGuJpiYY3YXvzN9PbZu1eJ9+b+igMNi/ww7PcfSKusIp PAdEpibAUx7RgTlJwZRbl/NIdvDJ+rDru8/lLtNLq4WW04lyNfnwc9XHev0sgtjYzeZX XiSzZ/dTRfCIAirtb0vobHIFnM+60silsMw6GY43Lm34rFdKpDtB91Y7d9T3KweDUayD UlB1xZKIa9+f2nMOEqW8fvwbnl/rpSRq052hOQbiPChMKcn8JEjaZuObg0dCSDg6U+AO 0/v/1Enx0pcdpkMIbd0Esf4W7kUESmwW1uRINCdWQ0cgRnHlHGJozGUzkKgnowGyx4c5 pH7w== X-Forwarded-Encrypted: i=1; AFNElJ9yemewK7TL6bQRmPhsLICJ6kjS0Ar4xyO/56QazdGWPZbMc3STrctX3xD7Hu4B2aLWpxM/ug8I9J8HvTcxi7J8@lists.infradead.org X-Gm-Message-State: AOJu0YwzoqsPQgul0aryoxOAqZ96OH6ptCx+ZPsdop+4I1pHbfvHHmJp eiAB7yhMJupkzjnu1pkgtTM50PJTuETxZsxxCJKdv2qpGHA2JPhhb6GX/POzZUJ+Eqqk3w0b6op KKogkVCsFUw== X-Received: from dycnr28-n1.prod.google.com ([2002:a05:7300:e9dc:10b0:2d5:d26c:d4bc]) (user=irogers job=prod-delivery.src-stubby-dispatcher) by 2002:a05:7301:129b:b0:2ea:ed7c:912f with SMTP id 5a478bee46e88-2eaed7c9a68mr2186737eec.27.1777157477301; Sat, 25 Apr 2026 15:51:17 -0700 (PDT) Date: Sat, 25 Apr 2026 15:49:23 -0700 In-Reply-To: <20260425224951.174663-1-irogers@google.com> Mime-Version: 1.0 References: <20260425174858.3922152-1-irogers@google.com> <20260425224951.174663-1-irogers@google.com> X-Mailer: git-send-email 2.54.0.545.g6539524ca2-goog Message-ID: <20260425224951.174663-32-irogers@google.com> Subject: [PATCH v7 31/59] perf gecko: Port gecko to use python module From: Ian Rogers To: acme@kernel.org, adrian.hunter@intel.com, james.clark@linaro.org, leo.yan@linux.dev, namhyung@kernel.org, tmricht@linux.ibm.com Cc: alice.mei.rogers@gmail.com, dapeng1.mi@linux.intel.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, mingo@redhat.com, peterz@infradead.org, Ian Rogers Content-Type: text/plain; charset="UTF-8" X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20260425_235120_474059_4B6E06F9 X-CRM114-Status: GOOD ( 20.22 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add a port of the gecko script that uses the perf python module directly. This approach is significantly faster than using perf script callbacks as it avoids creating intermediate dictionaries for all event fields. Assisted-by: Gemini:gemini-3.1-pro-preview Signed-off-by: Ian Rogers --- v2: 1. Improved Portability: Replaced the non-portable uname -op call with platform.system() and platform.machine() , preventing potential crashes on non-Linux platforms like macOS or BSD. 2. Robust Fallbacks: Fixed getattr calls for symbol and dso to explicitly handle None values, preventing literal "None (in None)" strings in the output when resolution fails. 3. Network Security: Bound the HTTP server to 127.0.0.1 (localhost) instead of 0.0.0.0 (all interfaces), ensuring the current directory is not exposed to the local network. 4. Avoided Port Conflicts: Switched from hardcoded port 8000 to port 0, allowing the operating system to automatically select an available free port. 5. Fixed Race Condition: Moved HTTPServer creation to the main thread, ensuring the server is bound and listening before the browser is launched to fetch the file. 6. Browser Spec Compliance: Used 127.0.0.1 instead of localhost in the generated URL to ensure modern browsers treat the connection as a secure origin, avoiding mixed content blocks. v6: - Fixed CWD exposure and symlink attack risks by using a secure temporary directory for the HTTP server. --- tools/perf/python/gecko.py | 385 +++++++++++++++++++++++++++++++++++++ 1 file changed, 385 insertions(+) create mode 100755 tools/perf/python/gecko.py diff --git a/tools/perf/python/gecko.py b/tools/perf/python/gecko.py new file mode 100755 index 000000000000..1f152e1eca52 --- /dev/null +++ b/tools/perf/python/gecko.py @@ -0,0 +1,385 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0 +""" +gecko.py - Convert perf record output to Firefox's gecko profile format +""" + +import argparse +import functools +import json +import os +import platform +import sys +import tempfile +import threading +import urllib.parse +import webbrowser +from dataclasses import dataclass, field +from http.server import HTTPServer, SimpleHTTPRequestHandler +from typing import Dict, List, NamedTuple, Optional, Tuple + +import perf + + +# https://github.com/firefox-devtools/profiler/blob/53970305b51b9b472e26d7457fee1d66cd4e2737/src/types/gecko-profile.js#L156 +class Frame(NamedTuple): + """A single stack frame in the gecko profile format.""" + string_id: int + relevantForJS: bool + innerWindowID: int + implementation: None + optimizations: None + line: None + column: None + category: int + subcategory: Optional[int] + + +# https://github.com/firefox-devtools/profiler/blob/53970305b51b9b472e26d7457fee1d66cd4e2737/src/types/gecko-profile.js#L216 +class Stack(NamedTuple): + """A single stack in the gecko profile format.""" + prefix_id: Optional[int] + frame_id: int + + +# https://github.com/firefox-devtools/profiler/blob/53970305b51b9b472e26d7457fee1d66cd4e2737/src/types/gecko-profile.js#L90 +class Sample(NamedTuple): + """A single sample in the gecko profile format.""" + stack_id: Optional[int] + time_ms: float + responsiveness: int + + +@dataclass +class Tables: + """Interned tables for the gecko profile format.""" + frame_table: List[Frame] = field(default_factory=list) + string_table: List[str] = field(default_factory=list) + string_map: Dict[str, int] = field(default_factory=dict) + stack_table: List[Stack] = field(default_factory=list) + stack_map: Dict[Tuple[Optional[int], int], int] = field(default_factory=dict) + frame_map: Dict[str, int] = field(default_factory=dict) + + +@dataclass +class Thread: + """A builder for a profile of the thread.""" + comm: str + pid: int + tid: int + user_category: int + kernel_category: int + samples: List[Sample] = field(default_factory=list) + tables: Tables = field(default_factory=Tables) + + def _intern_stack(self, frame_id: int, prefix_id: Optional[int]) -> int: + """Gets a matching stack, or saves the new stack. Returns a Stack ID.""" + key = (prefix_id, frame_id) + stack_id = self.tables.stack_map.get(key) + if stack_id is None: + stack_id = len(self.tables.stack_table) + self.tables.stack_table.append(Stack(prefix_id=prefix_id, frame_id=frame_id)) + self.tables.stack_map[key] = stack_id + return stack_id + + def _intern_string(self, string: str) -> int: + """Gets a matching string, or saves the new string. Returns a String ID.""" + string_id = self.tables.string_map.get(string) + if string_id is not None: + return string_id + string_id = len(self.tables.string_table) + self.tables.string_table.append(string) + self.tables.string_map[string] = string_id + return string_id + + def _intern_frame(self, frame_str: str) -> int: + """Gets a matching stack frame, or saves the new frame. Returns a Frame ID.""" + frame_id = self.tables.frame_map.get(frame_str) + if frame_id is not None: + return frame_id + frame_id = len(self.tables.frame_table) + self.tables.frame_map[frame_str] = frame_id + string_id = self._intern_string(frame_str) + + category = self.user_category + if (frame_str.find('kallsyms') != -1 or + frame_str.find('/vmlinux') != -1 or + frame_str.endswith('.ko)')): + category = self.kernel_category + + self.tables.frame_table.append(Frame( + string_id=string_id, + relevantForJS=False, + innerWindowID=0, + implementation=None, + optimizations=None, + line=None, + column=None, + category=category, + subcategory=None, + )) + return frame_id + + def add_sample(self, comm: str, stack: List[str], time_ms: float) -> None: + """Add a timestamped stack trace sample to the thread builder.""" + if self.comm != comm: + self.comm = comm + + prefix_stack_id: Optional[int] = None + for frame in stack: + frame_id = self._intern_frame(frame) + prefix_stack_id = self._intern_stack(frame_id, prefix_stack_id) + + if prefix_stack_id is not None: + self.samples.append(Sample(stack_id=prefix_stack_id, + time_ms=time_ms, + responsiveness=0)) + + def to_json_dict(self) -> Dict: + """Converts current Thread to GeckoThread JSON format.""" + return { + "tid": self.tid, + "pid": self.pid, + "name": self.comm, + "markers": { + "schema": { + "name": 0, + "startTime": 1, + "endTime": 2, + "phase": 3, + "category": 4, + "data": 5, + }, + "data": [], + }, + "samples": { + "schema": { + "stack": 0, + "time": 1, + "responsiveness": 2, + }, + "data": self.samples + }, + "frameTable": { + "schema": { + "location": 0, + "relevantForJS": 1, + "innerWindowID": 2, + "implementation": 3, + "optimizations": 4, + "line": 5, + "column": 6, + "category": 7, + "subcategory": 8, + }, + "data": self.tables.frame_table, + }, + "stackTable": { + "schema": { + "prefix": 0, + "frame": 1, + }, + "data": self.tables.stack_table, + }, + "stringTable": self.tables.string_table, + "registerTime": 0, + "unregisterTime": None, + "processType": "default", + } + + +class CORSRequestHandler(SimpleHTTPRequestHandler): + """Enable CORS for requests from profiler.firefox.com.""" + def end_headers(self): + self.send_header('Access-Control-Allow-Origin', 'https://profiler.firefox.com') + super().end_headers() + + +@dataclass +class CategoryData: + """Category configuration for the gecko profile.""" + user_index: int = 0 + kernel_index: int = 1 + categories: List[Dict] = field(default_factory=list) + + +class GeckoCLI: + """Command-line interface for converting perf data to Gecko format.""" + def __init__(self, args): + self.args = args + self.tid_to_thread: Dict[int, Thread] = {} + self.start_time_ms: Optional[float] = None + self.session = None + self.product = f"{platform.system()} {platform.machine()}" + self.cat_data = CategoryData( + categories=[ + { + "name": 'User', + "color": args.user_color, + "subcategories": ['Other'] + }, + { + "name": 'Kernel', + "color": args.kernel_color, + "subcategories": ['Other'] + }, + ] + ) + + def process_event(self, sample) -> None: + """Process a single perf sample event.""" + if self.args.event_name and self.args.event_name not in str(sample.evsel): + return + + # sample_time is in nanoseconds. Gecko wants milliseconds. + time_ms = sample.sample_time / 1000000.0 + pid = sample.sample_pid + tid = sample.sample_tid + + if self.start_time_ms is None: + self.start_time_ms = time_ms + + try: + thread_info = self.session.find_thread(tid) + comm = thread_info.comm() + except Exception: + comm = "[unknown]" + + stack = [] + callchain = sample.callchain + if callchain: + for entry in callchain: + symbol = entry.symbol or "[unknown]" + dso = entry.dso or "[unknown]" + stack.append(f"{symbol} (in {dso})") + # Reverse because Gecko wants root first. + stack.reverse() + else: + # Fallback if no callchain is present + try: + # If the perf module exposes symbol/dso directly on sample + # when callchain is missing, we use them. + symbol = getattr(sample, 'symbol', '[unknown]') or '[unknown]' + dso = getattr(sample, 'dso', '[unknown]') or '[unknown]' + stack.append(f"{symbol} (in {dso})") + except AttributeError: + stack.append("[unknown] (in [unknown])") + + thread = self.tid_to_thread.get(tid) + if thread is None: + thread = Thread(comm=comm, pid=pid, tid=tid, + user_category=self.cat_data.user_index, + kernel_category=self.cat_data.kernel_index) + self.tid_to_thread[tid] = thread + thread.add_sample(comm=comm, stack=stack, time_ms=time_ms) + + def run(self) -> None: + """Run the conversion process.""" + input_file = self.args.input or "perf.data" + if not os.path.exists(input_file): + print(f"Error: {input_file} not found.", file=sys.stderr) + sys.exit(1) + + try: + self.session = perf.session(perf.data(input_file), sample=self.process_event) + except Exception as e: + print(f"Error opening session: {e}", file=sys.stderr) + sys.exit(1) + + self.session.process_events() + + threads = [t.to_json_dict() for t in self.tid_to_thread.values()] + + gecko_profile = { + "meta": { + "interval": 1, + "processType": 0, + "product": self.product, + "stackwalk": 1, + "debug": 0, + "gcpoison": 0, + "asyncstack": 1, + "startTime": self.start_time_ms, + "shutdownTime": None, + "version": 24, + "presymbolicated": True, + "categories": self.cat_data.categories, + "markerSchema": [], + }, + "libs": [], + "threads": threads, + "processes": [], + "pausedRanges": [], + } + + output_file = self.args.save_only + if output_file is None: + self._write_and_launch(gecko_profile) + else: + print(f'[ perf gecko: Captured and wrote into {output_file} ]') + with open(output_file, 'w', encoding='utf-8') as f: + json.dump(gecko_profile, f, indent=2) + + def _write_and_launch(self, profile: Dict) -> None: + """Write the profile to a file and launch the Firefox profiler.""" + print("Starting Firefox Profiler on your default browser...") + + with tempfile.TemporaryDirectory() as tmp_dir_name: + filename = os.path.join(tmp_dir_name, 'gecko_profile.json') + + with open(filename, 'w', encoding='utf-8') as f: + json.dump(profile, f, indent=2) + + handler = functools.partial(CORSRequestHandler, directory=tmp_dir_name) + try: + httpd = HTTPServer(('127.0.0.1', 0), handler) + except OSError as e: + print(f"Error starting HTTP server: {e}", file=sys.stderr) + sys.exit(1) + + port = httpd.server_port + + def start_server(): + httpd.serve_forever() + + thread = threading.Thread(target=start_server, daemon=True) + thread.start() + + # Open the browser + safe_string = urllib.parse.quote_plus(f'http://127.0.0.1:{port}/gecko_profile.json') + url = f'https://profiler.firefox.com/from-url/{safe_string}' + webbrowser.open(url) + + print(f'[ perf gecko: Captured and wrote into {filename} ]') + print("Press Ctrl+C to stop the local server.") + try: + # Keep the main thread alive so the daemon thread can serve requests + stop_event = threading.Event() + while True: + stop_event.wait(1) + except KeyboardInterrupt: + print("\nStopping server...") + httpd.shutdown() + + +if __name__ == "__main__": + parser = argparse.ArgumentParser( + description="Convert perf.data to Firefox's Gecko Profile format" + ) + parser.add_argument('--user-color', default='yellow', + help='Color for the User category', + choices=['yellow', 'blue', 'purple', 'green', 'orange', 'red', + 'grey', 'magenta']) + parser.add_argument('--kernel-color', default='orange', + help='Color for the Kernel category', + choices=['yellow', 'blue', 'purple', 'green', 'orange', 'red', + 'grey', 'magenta']) + parser.add_argument('--save-only', + help='Save the output to a file instead of opening Firefox\'s profiler') + parser.add_argument("-i", "--input", help="input perf.data file") + parser.add_argument("-e", "--event", default="", dest="event_name", type=str, + help="specify the event to generate gecko profile for") + + cli_args = parser.parse_args() + cli = GeckoCLI(cli_args) + cli.run() -- 2.54.0.545.g6539524ca2-goog