Linux Perf Users
 help / color / mirror / Atom feed
From: Adrian Hunter <adrian.hunter@intel.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@redhat.com>,
	Arnaldo Carvalho de Melo <acme@kernel.org>,
	Namhyung Kim <namhyung@kernel.org>,
	Mark Rutland <mark.rutland@arm.com>,
	Alexander Shishkin <alexander.shishkin@linux.intel.com>,
	Jiri Olsa <jolsa@kernel.org>, "Ian Rogers" <irogers@google.com>,
	James Clark <james.clark@linaro.org>,
	"Thomas Gleixner" <tglx@kernel.org>,
	Borislav Petkov <bp@alien8.de>,
	Dave Hansen <dave.hansen@linux.intel.com>, <x86@kernel.org>,
	"H. Peter Anvin" <hpa@zytor.com>,
	"Rafael J. Wysocki" <rafael@kernel.org>,
	Pavel Machek <pavel@kernel.org>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	"Danilo Krummrich" <dakr@kernel.org>,
	Johan Hovold <johan@kernel.org>,
	"Sebastian Andrzej Siewior" <bigeasy@linutronix.de>,
	Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
	Jiayuan Chen <jiayuan.chen@linux.dev>,
	"Steven Rostedt (Google)" <rostedt@goodmis.org>,
	"Vineeth Pillai (Google)" <vineeth@bitbyteword.org>,
	Frederic Weisbecker <frederic@kernel.org>,
	"Andi Kleen" <ak@linux.intel.com>,
	<linux-perf-users@vger.kernel.org>,
	<linux-kernel@vger.kernel.org>, <linux-pm@vger.kernel.org>,
	<driver-core@lists.linux.dev>
Subject: Re: [PATCH 4/6] perf/x86/intel/pt: Clear TraceEn in hardware in cpu_emergency_stop_pt()
Date: Mon, 2 Nov 2026 20:41:01 +0200	[thread overview]
Message-ID: <0cdd28de-cbdb-4194-90c7-58b07c45dffc@intel.com> (raw)
In-Reply-To: <20261102100750.GU1779021@noisy.programming.kicks-ass.net>

On 02/11/2026 12:07, Peter Zijlstra wrote:
> On Mon, Nov 02, 2026 at 09:40:37AM +0200, Adrian Hunter wrote:
>> cpu_emergency_stop_pt() stops tracing through pt_event_stop(), which
>> assumes the software state accurately reflects whether tracing is
>> enabled.  That assumption can fail when a crash NMI interrupts
>> pt_config_start().
>>
>> pt_config_start() sets TraceEn in IA32_RTIT_CTL before updating
>> event->hw.aux_config.  If a crash NMI arrives in that window,
>> pt_config_stop() sees TraceEn clear in aux_config and skips the MSR
>> write needed to disable tracing.  Tracing remains active in hardware.
>>
>> Alternatively, an earlier NMI in the same window can stop the event
>> without restarting it, for example a group throttle calling
>> pt_event_stop(event, 0), or a PT PMI for which perf_aux_output_begin()
>> fails.  pt_config_stop() again skips the write, so the event is stopped
>> only in software: TraceEn remains set in IA32_RTIT_CTL, and
>> pt->handle.event becomes NULL.  When pt_config_start() continues, it
>> sets TraceEn in aux_config, so tracing continues until the event is
>> next stopped.  If a crash occurs before then, cpu_emergency_stop_pt()
>> sees no event and does nothing.
>>
>> In both cases tracing can remain active into the kdump kernel.  Moreover,
>> pt_event_stop() may access PT MSRs while tracing is still enabled and
>> can trigger a #GP by writing IA32_RTIT_STATUS with TraceEn set.
>>
>> Fix this by making cpu_emergency_stop_pt() disable tracing directly from
>> the hardware state.  Set pt_shutdown and invoke pt_shutdown_cpu(), which
>> reads IA32_RTIT_CTL and clears TraceEn whenever it is set, independent
>> of perf's software state.  The shutdown flag prevents any path from
>> re-enabling tracing afterwards, including intel_pt_handle_vmx(0) during
>> the emergency VMXOFF sequence on other CPUs.
>>
>> Continue to call pt_event_stop() afterwards so that AUX state, including
>> the buffer head, is updated for vmcore collection.  Its PT MSR accesses
>> now occur with tracing disabled.
>>
>> Do this only if the driver initialized successfully.  In particular, if
>> pt_init() finds tracing already enabled at boot, it leaves Intel PT
>> alone, and the emergency path must do the same.
>>
>> The race is reachable only through restart paths that call
>> pt_config_start() with TraceEn clear in the software copy.  These were
>> introduced by the commits with the Fixes tags below.  The restart path
>> in the PT PMI handler runs in NMI context, so only a machine check can
>> interrupt it.
>>
>> Fixes: 8e105a1fc2a0 ("perf/x86/intel/pt: Factor out pt_config_start()")
>> Fixes: 25e8920b301c ("perf/x86/intel/pt: Add sampling support")
>> Fixes: 08c7454ceb94 ("perf/x86/intel/pt: Add support for pause / resume")
>> Fixes: 2e17bf3a469a ("perf/x86/intel/pt: Fix stop/start with no update")
>> Signed-off-by: Adrian Hunter <adrian.hunter@intel.com>
>> Cc: stable@vger.kernel.org
>> ---
>>  arch/x86/events/intel/pt.c | 9 +++++++++
>>  1 file changed, 9 insertions(+)
>>
>> diff --git a/arch/x86/events/intel/pt.c b/arch/x86/events/intel/pt.c
>> index 7453e9880b25..8d71c00f9ff6 100644
>> --- a/arch/x86/events/intel/pt.c
>> +++ b/arch/x86/events/intel/pt.c
>> @@ -38,6 +38,8 @@ static struct pt_pmu pt_pmu;
>>  /* Set at shutdown, after which TraceEn must not be set */
>>  static bool pt_shutdown;
>>  
>> +static bool pt_initialized __ro_after_init;
>> +
>>  /*
>>   * Capabilities of Intel PT hardware, such as number of address bits or
>>   * supported output schemes, are cached and exported to userspace as "caps"
>> @@ -1900,6 +1902,12 @@ void cpu_emergency_stop_pt(void)
>>  {
>>  	struct pt *pt = this_cpu_ptr(&pt_ctx);
>>  
>> +	if (!pt_initialized)
>> +		return;
>> +
>> +	WRITE_ONCE(pt_shutdown, true);
>> +	pt_shutdown_cpu(NULL);
>> +
>>  	if (pt->handle.event)
>>  		pt_event_stop(pt->handle.event, PERF_EF_UPDATE);
> 
> So in general pmu::stop() is reachable from PTI/NMI context, through
> things like throttling. So arguably it not functioning correctly when
> interleaving with pmu::start() is a problem.
> 

Yes, and it looks like there is a separate fix for the 1st
scenario described in the commit message: write aux_config both
before and after the WRMSR.  I will add that fix next time around,
and re-evaluate how much this patch is still needed.


  reply	other threads:[~2026-11-02 18:41 UTC|newest]

Thread overview: 30+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-02  7:40 [PATCH 0/6] perf/x86/intel/pt: Fix tracing across kexec, kdump and suspend Adrian Hunter
2026-11-02  7:40 ` [PATCH 1/6] irq_work: Re-raise pending irq_work after system resume Adrian Hunter
2026-11-02  7:51   ` sashiko-bot
2026-11-02  9:06   ` Peter Zijlstra
2026-11-02 10:54     ` Adrian Hunter
2026-11-02 16:39       ` Sebastian Andrzej Siewior
2026-11-02  7:40 ` [PATCH 2/6] perf/x86/intel/pt: Stop tracing if VMXON has not yet happened Adrian Hunter
2026-11-02  7:59   ` sashiko-bot
2026-11-02  9:57   ` Peter Zijlstra
2026-11-02 11:04     ` Adrian Hunter
2026-11-02 11:07       ` Peter Zijlstra
2026-11-02  7:40 ` [PATCH 3/6] perf/x86/intel/pt: Stop tracing at syscore shutdown Adrian Hunter
2026-11-02  8:05   ` sashiko-bot
2026-11-02 10:04   ` Peter Zijlstra
2026-11-02 10:23     ` Adrian Hunter
2026-11-02 11:01       ` Peter Zijlstra
2026-11-02 16:15   ` Andi Kleen
2026-11-02 16:27     ` Adrian Hunter
2026-11-02 17:45       ` Andi Kleen
2026-11-02 18:13         ` Adrian Hunter
2026-11-02  7:40 ` [PATCH 4/6] perf/x86/intel/pt: Clear TraceEn in hardware in cpu_emergency_stop_pt() Adrian Hunter
2026-11-02  8:21   ` sashiko-bot
2026-11-02 10:07   ` Peter Zijlstra
2026-11-02 18:41     ` Adrian Hunter [this message]
2026-11-02  7:40 ` [PATCH 5/6] perf/x86/intel/pt: Reset stale MSR state on CPU start and resume Adrian Hunter
2026-11-02  8:31   ` sashiko-bot
2026-11-02  7:40 ` [PATCH 6/6] perf/x86/intel/pt: Stop tracing across system suspend Adrian Hunter
2026-11-02  8:59   ` sashiko-bot
2026-11-02 10:10 ` [PATCH 0/6] perf/x86/intel/pt: Fix tracing across kexec, kdump and suspend Peter Zijlstra
2026-11-02 12:22   ` Adrian Hunter

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=0cdd28de-cbdb-4194-90c7-58b07c45dffc@intel.com \
    --to=adrian.hunter@intel.com \
    --cc=acme@kernel.org \
    --cc=ak@linux.intel.com \
    --cc=alexander.shishkin@linux.intel.com \
    --cc=bigeasy@linutronix.de \
    --cc=bp@alien8.de \
    --cc=dakr@kernel.org \
    --cc=dave.hansen@linux.intel.com \
    --cc=driver-core@lists.linux.dev \
    --cc=frederic@kernel.org \
    --cc=gregkh@linuxfoundation.org \
    --cc=hpa@zytor.com \
    --cc=irogers@google.com \
    --cc=james.clark@linaro.org \
    --cc=jiayuan.chen@linux.dev \
    --cc=johan@kernel.org \
    --cc=jolsa@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-perf-users@vger.kernel.org \
    --cc=linux-pm@vger.kernel.org \
    --cc=mark.rutland@arm.com \
    --cc=mathieu.desnoyers@efficios.com \
    --cc=mingo@redhat.com \
    --cc=namhyung@kernel.org \
    --cc=pavel@kernel.org \
    --cc=peterz@infradead.org \
    --cc=rafael@kernel.org \
    --cc=rostedt@goodmis.org \
    --cc=tglx@kernel.org \
    --cc=vineeth@bitbyteword.org \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox