From: Valentin Schneider <vschneid@redhat.com>
To: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org
Cc: Steven Rostedt <rostedt@goodmis.org>,
Masami Hiramatsu <mhiramat@kernel.org>,
Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
Tomas Glozar <tglozar@redhat.com>,
Costa Shulyupin <costa.shul@redhat.com>,
Crystal Wood <crwood@redhat.com>,
Ivan Pravdin <ipravdin.official@gmail.com>
Subject: [RFC PATCH 0/2] tracing/osnoise: Track IPIs
Date: Wed, 10 Jun 2026 15:04:50 +0200 [thread overview]
Message-ID: <20260610130457.1304245-1-vschneid@redhat.com> (raw)
Hi folks,
So I've seen a few times now reports of latency spikes caused by IPIs, usually
because of isolation misconfiguration, but only detected at the tail of end
e.g. a 24h timerlat run.
It's not because those IPIs are rare, but rather that they don't by themselves
cause a monitered CPU to reach the latency threshold, it's usually a combined
interference that gets us there.
I'd like to make it easier to detect such misconfigurations and thus IPIs
hitting supposedly-isolated CPUs. I initially kludged a timerlat option to stop
tracing as soon as an IPI was sent to a monitored CPU, regardless of the latency
threshold. It sort of did the trick, but Tomáš convinced me timerlat wasn't
really the place for that.
So here's IPI tracking added to osnoise. Two things worth pointing out:
o This only adds IPI count tracking, nothing about noise duration - this is
already tracked as part of the IRQ noise.
o This modifies the osnoise Ftrace entry, I have no idea how acceptable this is,
although the only real consumer of these should be rtla...
Tested with:
$ rtla osnoise top -d 5s
$ trace-cmd record -p osnoise hackbench -l 10000
Cheers,
Valentin
Valentin Schneider (2):
tracing/osnoise: Sample IPI counts
rtla/osnoise: Report IPI count in osnoise top
include/trace/events/osnoise.h | 1 +
kernel/trace/trace_entries.h | 6 ++-
kernel/trace/trace_osnoise.c | 80 ++++++++++++++++++++++++++--
tools/tracing/rtla/src/osnoise_top.c | 9 +++-
4 files changed, 88 insertions(+), 8 deletions(-)
--
2.54.0
next reply other threads:[~2026-06-10 13:05 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-06-10 13:04 Valentin Schneider [this message]
2026-06-10 13:04 ` [RFC PATCH 1/2] tracing/osnoise: Sample IPI counts Valentin Schneider
2026-06-10 19:51 ` Crystal Wood
2026-06-11 8:59 ` Tomas Glozar
2026-06-11 10:30 ` Valentin Schneider
2026-06-11 11:55 ` Tomas Glozar
2026-06-12 8:53 ` Valentin Schneider
2026-06-11 20:49 ` Crystal Wood
2026-06-11 10:21 ` Valentin Schneider
2026-06-10 13:04 ` [RFC PATCH 2/2] rtla/osnoise: Report IPI count in osnoise top Valentin Schneider
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260610130457.1304245-1-vschneid@redhat.com \
--to=vschneid@redhat.com \
--cc=costa.shul@redhat.com \
--cc=crwood@redhat.com \
--cc=ipravdin.official@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-trace-kernel@vger.kernel.org \
--cc=mathieu.desnoyers@efficios.com \
--cc=mhiramat@kernel.org \
--cc=rostedt@goodmis.org \
--cc=tglozar@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox