From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 70BBC22AE7A; Thu, 20 Nov 2025 23:48:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763682489; cv=none; b=jKoLeFIx2go11KEyVqUa6C60SGviFGUjRw1dBTlUvfx1ILG597qr0OKS6hx+xWo/lJOO2D1TZGu4frB8D6+9pFcZdAuh4ybIAKqkHeoG808zx2AIUUFb51bVtoVI/GxE7zQ+Gue9epagnnBOLzxxRJOgXtZvIfrbfXnlCf2HkJo= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763682489; c=relaxed/simple; bh=VCitTLNl4pF3HT4SSFNsCGSBjZfQ7YTxcCLL1JOLWjg=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=pEP0/D6HpPqwHB7kwMmLzQLHEZI2fkoEbLKLYAxQIRtmaakDyIvdeCSKTnfQyrClZa8aQ8mrShEdn0hcS/Y9Sj1u4FLBW2o9o/cybJfGUGnIXIOZdQJfNrniaKWbVsJEhvZnAfCGGzpomt3g3Pk1CWzmYDqGgFwZDFRIg6mFYWI= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ks5lRGic; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ks5lRGic" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 51721C4CEF1; Thu, 20 Nov 2025 23:48:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1763682487; bh=VCitTLNl4pF3HT4SSFNsCGSBjZfQ7YTxcCLL1JOLWjg=; h=From:To:Cc:Subject:Date:From; b=ks5lRGicUYP9mNqtczq6x91MbGXtoBz+8HdIQ+0C89uk5+yFL5t/Gdd+6iW6Iw0y+ 7qVUGFQXeSSd9YDPoeoqwwNcsF66GbsKmXuUIZLpLFhZGJEVuRIgS8uTnjsFqTvYjC nr8/FuCAYVNUwM+2qt47EoUECZRuHeCtTxkskG/3yIl3xTz4DB27GnGrPSHrBVw+eA UkoshMBsJS/6QuQNeAvVam20rMkwRS9fH04h4V+71GHPf/SXx1xrPHCZcteDEN7Bkm x2n3PJSo+eAbE2QTWLYRqkKruPlg/Bt4rlrQHPw3tfJgYCkF6rIOzYOtcHaT4xu3Rc 4il2nfceedllw== From: Namhyung Kim To: Arnaldo Carvalho de Melo , Ian Rogers , James Clark Cc: Jiri Olsa , Adrian Hunter , Peter Zijlstra , Ingo Molnar , LKML , linux-perf-users@vger.kernel.org, Steven Rostedt , Josh Poimboeuf , Indu Bhagat , Jens Remus , Mathieu Desnoyers , linux-trace-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCHSET v6 0/6] perf tools: Add deferred callchain support Date: Thu, 20 Nov 2025 15:47:58 -0800 Message-ID: <20251120234804.156340-1-namhyung@kernel.org> X-Mailer: git-send-email 2.52.0.rc2.455.g230fcf2819-goog Precedence: bulk X-Mailing-List: linux-perf-users@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Hello, This is a new version of deferred callchain support as the kernel part is merged to the tip tree. Actually this is based on Steve's work (v16). https://lore.kernel.org/r/20250908175319.841517121@kernel.org v6 changes) * always copy the events (Ian) * add flush in the pipe mode v5: https://lore.kernel.org/r/20251120021046.94490-1-namhyung@kernel.org/ * update delegate tools (Ian) * copy and flush remaining samples (Ian) * add Ian's Reviewed-by tags v4: https://lore.kernel.org/r/20251115234106.348571-1-namhyung@kernel.org * add --call-graph fp,defer option (Ian, Steve) * add more comment on the cookie (Ian) * display cookie part in the deferred callchain (Ian) v3: https://lore.kernel.org/r/20251114070018.160330-1-namhyung@kernel.org * handle new attr.defer_output to generate deferred callchains * fix crash when cookies don't match (Steven) * disable merging for perf inject * fix missing feature detection bug * symbolize merged callchains properly Here's an example session. $ perf record --call-graph fp,defer pwd /home/namhyung/project/linux [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.010 MB perf.data (29 samples) ] $ perf evlist -v cpu/cycles/P: type: 0 (PERF_TYPE_HARDWARE), size: 136, config: 0 (PERF_COUNT_HW_CPU_CYCLES), { sample_period, sample_freq }: 4000, sample_type: IP|TID|TIME|CALLCHAIN|PERIOD, read_format: ID|LOST, disabled: 1, inherit: 1, mmap: 1, comm: 1, freq: 1, enable_on_exec: 1, task: 1, sample_id_all: 1, mmap2: 1, comm_exec: 1, ksymbol: 1, bpf_event: 1, build_id: 1, defer_callchain: 1, defer_output: 1 $ perf script ... pwd 2312 121.163435: 249113 cpu/cycles/P: ffffffff845b78d8 __build_id_parse.isra.0+0x218 ([kernel.kallsyms]) ffffffff83bb5bf6 perf_event_mmap+0x2e6 ([kernel.kallsyms]) ffffffff83c31959 mprotect_fixup+0x1e9 ([kernel.kallsyms]) ffffffff83c31dc5 do_mprotect_pkey+0x2b5 ([kernel.kallsyms]) ffffffff83c3206f __x64_sys_mprotect+0x1f ([kernel.kallsyms]) ffffffff845e6692 do_syscall_64+0x62 ([kernel.kallsyms]) ffffffff8360012f entry_SYSCALL_64_after_hwframe+0x76 ([kernel.kallsyms]) 7f18fe337fa7 mprotect+0x7 (/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2) 7f18fe330e0f _dl_sysdep_start+0x7f (/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2) 7f18fe331448 _dl_start_user+0x0 (/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2) ... $ perf script --no-merge-callchains ... pwd 2312 121.163435: 249113 cpu/cycles/P: ffffffff845b78d8 __build_id_parse.isra.0+0x218 ([kernel.kallsyms]) ffffffff83bb5bf6 perf_event_mmap+0x2e6 ([kernel.kallsyms]) ffffffff83c31959 mprotect_fixup+0x1e9 ([kernel.kallsyms]) ffffffff83c31dc5 do_mprotect_pkey+0x2b5 ([kernel.kallsyms]) ffffffff83c3206f __x64_sys_mprotect+0x1f ([kernel.kallsyms]) ffffffff845e6692 do_syscall_64+0x62 ([kernel.kallsyms]) ffffffff8360012f entry_SYSCALL_64_after_hwframe+0x76 ([kernel.kallsyms]) b00000006 (cookie) ([unknown]) pwd 2312 121.163447: DEFERRED CALLCHAIN [cookie: b00000006] 7f18fe337fa7 mprotect+0x7 (/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2) 7f18fe330e0f _dl_sysdep_start+0x7f (/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2) 7f18fe331448 _dl_start_user+0x0 (/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2) ... The code is available at 'perf/defer-callchain-v6' branch in git.kernel.org/pub/scm/linux/kernel/git/namhyung/linux-perf.git Thanks, Namhyung Namhyung Kim (6): tools headers UAPI: Sync linux/perf_event.h for deferred callchains perf tools: Minimal DEFERRED_CALLCHAIN support perf record: Add --call-graph fp,defer option for deferred callchains perf script: Display PERF_RECORD_CALLCHAIN_DEFERRED perf tools: Merge deferred user callchains perf tools: Flush remaining samples w/o deferred callchains tools/include/uapi/linux/perf_event.h | 21 +++- tools/lib/perf/include/perf/event.h | 13 ++ tools/perf/Documentation/perf-config.txt | 3 + tools/perf/Documentation/perf-record.txt | 4 + tools/perf/Documentation/perf-script.txt | 5 + tools/perf/builtin-inject.c | 1 + tools/perf/builtin-report.c | 1 + tools/perf/builtin-script.c | 93 ++++++++++++++ tools/perf/util/callchain.c | 45 ++++++- tools/perf/util/callchain.h | 4 + tools/perf/util/event.c | 1 + tools/perf/util/evlist.c | 1 + tools/perf/util/evlist.h | 2 + tools/perf/util/evsel.c | 50 +++++++- tools/perf/util/evsel.h | 1 + tools/perf/util/evsel_fprintf.c | 5 +- tools/perf/util/machine.c | 1 + tools/perf/util/perf_event_attr_fprintf.c | 2 + tools/perf/util/sample.h | 2 + tools/perf/util/session.c | 147 ++++++++++++++++++++++ tools/perf/util/tool.c | 5 + tools/perf/util/tool.h | 4 +- 22 files changed, 403 insertions(+), 8 deletions(-) -- 2.52.0.rc2.455.g230fcf2819-goog