Linux Perf Users
 help / color / mirror / Atom feed
* [PATCH v13 00/11] riscv: add SBI Supervisor Software Events support
@ 2026-11-19 12:08 Zhanpeng Zhang
  2026-11-19 12:08 ` [PATCH v13 01/11] riscv: add SBI SSE extension definitions Zhanpeng Zhang
                   ` (10 more replies)
  0 siblings, 11 replies; 22+ messages in thread
From: Zhanpeng Zhang @ 2026-11-19 12:08 UTC (permalink / raw)
  To: pjw, palmer, aou, alex, himanshu.chauhan, conor, anup.patel
  Cc: clement.leger, cuiyunhui, atish.patra, peterz, mingo, acme,
	namhyung, mark.rutland, alexander.shishkin, jolsa, irogers,
	adrian.hunter, james.clark, will, tglx, corbet, rdunlap, shuah,
	yuanzhu, yang.susheng, linux-riscv, linux-kernel,
	linux-perf-users, linux-doc, linux-kselftest,
	zhangzhanpeng.jasper

This revision rebases v12 [7] onto riscv/for-next at 642fb67393c1
without changing the implementation. Clément Léger joins Himanshu
Chauhan and Zhanpeng Zhang as a co-maintainer of Linux SSE.

Why SSE
=======

RISC-V does not define an ordinary supervisor-mode non-maskable interrupt
(NMI). When Linux clears SIE, a normal supervisor interrupt remains pending
until interrupts are enabled again. That is appropriate for ordinary
interrupt handling, but leaves a blind spot for profiling and delays
notifications that need attention inside an interrupt-disabled region.

The SBI Supervisor Software Events (SSE) extension [2] provides a
firmware-to-supervisor event delivery interface that does not depend on
SIE. Linux registers an entry point, and firmware can transfer control to
it while ordinary supervisor interrupts are masked. After the handler
completes the event, firmware resumes the interrupted execution.

PMU overflow is the first client in this series. The same framework is
intended to support other clients, including high-priority RAS reports;
the RAS client implementation is separate from this series.

Delivery and implementation
===========================

For a hardware-triggered event, the flow is:

                   Linux kernel                  Firmware         Hardware
            interrupted    SSE handler            OpenSBI          source
              context           |                    |                |
                 |              |                    |                |
  1. setup       |              |-----register------->                |
                 |              |------enable-------->                |
  2. trigger     |              |                    <-----event------|
                 |              |          +-------------------+      |
  3. save        |              |          |   context save    |      |
                 |              |          +-------------------+      |
                 |              |                    |                |
                 |              |          +-------------------+      |
  4. inject      |              |          |   handler setup   |      |
                 |              |          +-------------------+      |
                 |              <-------mret---------|                |
                 |    +-------------------+          |                |
  5. handle      |    |  event handling   |          |                |
                 |    +-------------------+          |                |
  6. complete    |              |-----complete------->                |
                 |              |          +-------------------+      |
  7. restore     |              |          |  context restore  |      |
                 |              |          +-------------------+      |
  8. resume      <---------------mret----------------|                |
                 |              |                    |                |

Software-injected events do not need the hardware trigger. The diagram
shows a typical OpenSBI implementation of the firmware interface.

Linux enters on a dedicated event stack and establishes NMI context.
The entry wrapper reconstructs the interrupted pt_regs for the callback,
rather than treating the synthetic handler context as the sampled task.
It also preserves Linux-owned trap state, task stack metadata and the
temporary register slots used by ordinary exception entry.

Handlers must not sleep or rely on locks that the interrupted code may
hold. A nofault user access may still raise a synchronous exception: that
exception must return through its exception-table fixup without paging
memory in or opening an interrupt window. Preserving the interrupted
entry state across such nested exceptions is therefore essential.

The firmware driver provides a common client API for local and global
events. It tracks ownership across partial setup and teardown failures,
keeps callback resources alive while firmware may still reference them,
and integrates event handling with CPU hotplug, shutdown and kexec.

PMU overflow and perf
=====================

Ordinary PMU interrupts cannot sample an interrupt-disabled critical
section until Linux unmasks interrupts. The sample then often lands at
that boundary, obscuring the work that preceded it. SSE removes this
SIE-induced delivery delay. It does not promise zero skid or precise-event
sampling, but lets perf observe execution inside those critical sections.

No new perf command or perf.data format is required. The PMU driver
selects its delivery mechanism during setup. With PMU-SSE configured,
explicitly unsupported firmware or events can use ordinary interrupts;
other setup errors do not silently select that route. There is no runtime
SSE-to-IRQ transition after SSE has become active.

Kernel and user callchains use the interrupted context. Kernel unwinding
falls back to the interrupted PC where stack bounds cannot be established.
For DWARF sampling, perf copies the raw user stack with page faults
disabled and truncates it at an inaccessible address. This series uses inatomic
uaccess and exception-table fixup instead of fast-only GUP. The additional
address-space checks reject a missing mm or mismatched active page-table
root; they are not a complete equivalent of x86's nmi_uaccess_okay().

The PMU path retains throttling and stopped-event semantics. It restores
counters and userpage state before enabling an event after CPU power
management, and stops affected counters if an SSE operation fails.

Hardware observations
=====================

The following measurements were reported with v10. They illustrate the
delivery mechanism, not a new benchmark of this revision. The IRQ
and SSE comparisons used the same kernel source and perf binary on a
RISC-V server platform.

For perf bench sched pipe, the mean of three runs of three million
single-CPU operations was:

  rate       IRQ ops/s   SSE ops/s   delta   IRQ samples/s   SSE samples/s
  -F 99        337,707     339,555   +0.55%            98.0            98.6
  -F 999       338,352     338,289   -0.02%           995.7           998.0
  -F 5000      329,002     333,034   +1.23%          5001.7          5001.6

These runs had no lost samples. Both routes reached the requested rates,
with workload throughput differing by at most 1.23% in this comparison.

The same workload makes the irq-off sampling bias visible. In the -F 999
IRQ profile, finish_task_switch() and _raw_spin_unlock_irqrestore()
accounted for 54.99% of samples. Sampling via SSE instead exposed work
inside the critical sections, including scheduling, locking and address
space switching. Leading entries from the two reports were:

  Ordinary PMU IRQ
  overhead   symbol
    36.63%   finish_task_switch.isra.0
    18.36%   _raw_spin_unlock_irqrestore
     7.66%   __internal_syscall_cancel
     7.55%   do_trap_ecall_u
     4.19%   mutex_lock
     3.64%   mutex_unlock
     3.06%   exit_to_user_mode_loop

  PMU-SSE
  overhead   symbol
     5.48%   __kprobes_text_end
     5.29%   __schedule
     5.10%   ret_from_exception
     4.71%   do_raw_spin_lock
     4.01%   do_trap_ecall_u
     3.99%   mutex_lock
     3.66%   switch_mm
     3.43%   mutex_unlock
     3.29%   exit_to_user_mode_loop
     3.28%   psi_group_change

Symbols are reproduced as reported by perf. This is a change in what the
profile can observe, not evidence that SSE made those functions faster.

Entry-state fix and validation
==============================

Himanshu identified that TASK_TI_A0/A1/A2 were not preserved across SSE
handling [3]. Ordinary RV64 exception entry temporarily stores registers
in these slots. If SSE interrupts that window and its handler takes a
nested exception, the nested entry can overwrite slots still needed by
the interrupted entry path. Patch 2 preserves and restores them alongside
the task's kernel and user stack pointers.

Hardware A/B testing confirmed that preserving these scratch slots fixes
the reproduced corruption, allowing the temporary GUP workaround to be
replaced with nofault uaccess.

Guest SSE delivery, guest callchains and hibernation are outside the scope
of this series.

Firmware requirements
=====================

Firmware must implement SSE and the local PMU event. PMU testing also
requires the OpenSBI fixes that preserve pending overflows across event
enablement [4] and clear stale SPV for non-virtualized events [5].

Changes in v13
==============

- Rebase onto riscv/for-next (642fb67393c1).
- P9: Add Clément Léger as an SSE co-maintainer.

The restriction on normal kexec from a crash kernel (patch 3) stays in
place because firmware may still hold the crashed kernel's SSE
registrations; narrowing the check is left to a follow-up.

Changes in v12
==============

Relative to Anup's v11 [1], on the same a5ae1bf65cee base:

- P2: Preserve TASK_TI_A0/A1/A2, fix per-CPU initialization preemption,
  and address the kernel test robot's unused-function warning.
- P3/P4: Clarify the kexec restriction and retain v10 RESEND's CPU-hotplug
  preemption guards.
- P5/P6: Order RV32 vmalloc synchronization before nofault fixup and add
  NMI-safe lockdep irqflags tracking.
- P7: Replace GUP with nofault uaccess.
- P8: Fix early-error counter STOP, ordinary IRQ CPU-PM ordering and IRQ
  cleanup tracking; carry Conor's Acked-by [6].
- P9: Split MAINTAINERS updates out; keep both selftests last.

Acknowledgements
================

The original feature patches were developed by Clément Léger and
Himanshu Chauhan. Thanks to Himanshu for identifying the missing scratch
slot preservation and reviewing the entry and lifecycle code; to Susheng
Yang for reporting the callchain failure and providing a reproducer; to
Conor Dooley for review and acknowledgements; and to Anup Patel, Paul
Walmsley and the kernel test robot for integration work and feedback.

[1] https://lore.kernel.org/r/20261028165531.940336-1-anup.patel@oss.qualcomm.com/
[2] https://docs.riscv.org/reference/sbi/ext-sse.html
[3] https://lore.kernel.org/r/as79MHtNYd-VbUug@hu-himchau-blr.qualcomm.com/
[4] https://github.com/riscv-software-src/opensbi/commit/f30a54f3b3a091c225a00476f4039bf399badd1f
[5] https://github.com/riscv-software-src/opensbi/commit/35511bc6ee1c9c17b6a89b44c52e2044bb51b979
[6] https://lore.kernel.org/r/20261002-reach-cloning-f1594729dbe3@wendy/
[7] https://lore.kernel.org/r/20261029074853.35467-1-zhangzhanpeng.jasper@bytedance.com/

Clément Léger (5):
  riscv: add SBI SSE extension definitions
  riscv: add support for SBI Supervisor Software Events extension
  drivers: firmware: add riscv SSE support
  perf: RISC-V: add support for SSE event
  selftests/riscv: add SSE test module

Zhanpeng Zhang (6):
  riscv: sse: mask events during shutdown and kexec
  riscv: mm: avoid enabling interrupts for nofault page faults
  riscv: enable NMI-safe lockdep irqflags tracking
  perf: RISC-V: support callchains with SSE delivery
  MAINTAINERS: add RISC-V SSE maintainers and file coverage
  selftests/riscv: add perf user-stack SSE copy regression test

 Documentation/arch/riscv/index.rst            |    1 +
 Documentation/arch/riscv/pmu-sse.rst          |   62 +
 MAINTAINERS                                   |   23 +
 arch/riscv/Kconfig                            |    1 +
 arch/riscv/include/asm/asm.h                  |   14 +-
 arch/riscv/include/asm/perf_event.h           |   10 +
 arch/riscv/include/asm/sbi.h                  |   63 +
 arch/riscv/include/asm/scs.h                  |    7 +
 arch/riscv/include/asm/sse.h                  |   82 ++
 arch/riscv/include/asm/thread_info.h          |    1 +
 arch/riscv/kernel/Makefile                    |    1 +
 arch/riscv/kernel/asm-offsets.c               |   14 +
 arch/riscv/kernel/entry.S                     |   14 +
 arch/riscv/kernel/machine_kexec.c             |   11 +
 arch/riscv/kernel/perf_callchain.c            |  140 ++
 arch/riscv/kernel/reset.c                     |   18 +
 arch/riscv/kernel/sbi_sse.c                   |  252 ++++
 arch/riscv/kernel/sbi_sse_entry.S             |  237 ++++
 arch/riscv/kernel/smp.c                       |   17 +
 arch/riscv/mm/fault.c                         |   32 +-
 drivers/firmware/Kconfig                      |    1 +
 drivers/firmware/Makefile                     |    1 +
 drivers/firmware/riscv/Kconfig                |   18 +
 drivers/firmware/riscv/Makefile               |    3 +
 drivers/firmware/riscv/riscv_sbi_sse.c        | 1228 +++++++++++++++++
 drivers/perf/Kconfig                          |   11 +
 drivers/perf/riscv_pmu.c                      |   14 +-
 drivers/perf/riscv_pmu_sbi.c                  |  561 ++++++--
 include/linux/cpuhotplug.h                    |    1 +
 include/linux/perf/riscv_pmu.h                |   20 +-
 include/linux/riscv_sbi_sse.h                 |   95 ++
 tools/testing/selftests/riscv/Makefile        |    2 +-
 tools/testing/selftests/riscv/sse/Makefile    |   10 +
 .../selftests/riscv/sse/module/Makefile       |   22 +
 .../riscv/sse/module/riscv_sse_test.c         | 1154 ++++++++++++++++
 .../selftests/riscv/sse/run_sse_test.sh       |   59 +
 .../selftests/riscv/sse/sse_perf_ustack.c     |  564 ++++++++
 37 files changed, 4657 insertions(+), 107 deletions(-)
 create mode 100644 Documentation/arch/riscv/pmu-sse.rst
 create mode 100644 arch/riscv/include/asm/sse.h
 create mode 100644 arch/riscv/kernel/sbi_sse.c
 create mode 100644 arch/riscv/kernel/sbi_sse_entry.S
 create mode 100644 drivers/firmware/riscv/Kconfig
 create mode 100644 drivers/firmware/riscv/Makefile
 create mode 100644 drivers/firmware/riscv/riscv_sbi_sse.c
 create mode 100644 include/linux/riscv_sbi_sse.h
 create mode 100644 tools/testing/selftests/riscv/sse/Makefile
 create mode 100644 tools/testing/selftests/riscv/sse/module/Makefile
 create mode 100644 tools/testing/selftests/riscv/sse/module/riscv_sse_test.c
 create mode 100644 tools/testing/selftests/riscv/sse/run_sse_test.sh
 create mode 100644 tools/testing/selftests/riscv/sse/sse_perf_ustack.c


base-commit: 642fb67393c148f45a635b3b5afefdbd27632ed0
-- 
2.50.1 (Apple Git-155)


^ permalink raw reply	[flat|nested] 22+ messages in thread

* [PATCH v13 01/11] riscv: add SBI SSE extension definitions
  2026-11-19 12:08 [PATCH v13 00/11] riscv: add SBI Supervisor Software Events support Zhanpeng Zhang
@ 2026-11-19 12:08 ` Zhanpeng Zhang
  2026-11-19 12:14   ` sashiko-bot
  2026-11-19 12:08 ` [PATCH v13 02/11] riscv: add support for SBI Supervisor Software Events extension Zhanpeng Zhang
                   ` (9 subsequent siblings)
  10 siblings, 1 reply; 22+ messages in thread
From: Zhanpeng Zhang @ 2026-11-19 12:08 UTC (permalink / raw)
  To: pjw, palmer, aou, alex, himanshu.chauhan, conor, anup.patel
  Cc: clement.leger, cuiyunhui, atish.patra, peterz, mingo, acme,
	namhyung, mark.rutland, alexander.shishkin, jolsa, irogers,
	adrian.hunter, james.clark, will, tglx, corbet, rdunlap, shuah,
	yuanzhu, yang.susheng, linux-riscv, linux-kernel,
	linux-perf-users, linux-doc, linux-kselftest,
	zhangzhanpeng.jasper

From: Clément Léger <cleger@rivosinc.com>

Add definitions for the SBI Supervisor Software Events extension [1].
This extension enables the SBI to inject events into supervisor software
much like ARM SDEI.

[1] https://lists.riscv.org/g/tech-prs/message/515

Signed-off-by: Clément Léger <cleger@rivosinc.com>
Co-developed-by: Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>
Signed-off-by: Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>
Co-developed-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
Signed-off-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
---
 arch/riscv/include/asm/sbi.h | 63 ++++++++++++++++++++++++++++++++++++
 1 file changed, 63 insertions(+)

diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
index fd6266a7b339..eb22d79cea61 100644
--- a/arch/riscv/include/asm/sbi.h
+++ b/arch/riscv/include/asm/sbi.h
@@ -40,6 +40,7 @@ enum sbi_ext_id {
 	SBI_EXT_FWFT = 0x46574654,
 	SBI_EXT_MPXY = 0x4D505859,
 	SBI_EXT_DBTR = 0x44425452,
+	SBI_EXT_SSE = 0x535345,
 
 	/* Experimentals extensions must lie within this range */
 	SBI_EXT_EXPERIMENTAL_START = 0x08000000,
@@ -508,6 +509,68 @@ enum sbi_mpxy_rpmi_attribute_id {
 #define SBI_MPXY_CHAN_CAP_SEND_WITHOUT_RESP	BIT(4)
 #define SBI_MPXY_CHAN_CAP_GET_NOTIFICATIONS	BIT(5)
 
+enum sbi_ext_sse_fid {
+	SBI_SSE_EVENT_ATTR_READ = 0,
+	SBI_SSE_EVENT_ATTR_WRITE,
+	SBI_SSE_EVENT_REGISTER,
+	SBI_SSE_EVENT_UNREGISTER,
+	SBI_SSE_EVENT_ENABLE,
+	SBI_SSE_EVENT_DISABLE,
+	SBI_SSE_EVENT_COMPLETE,
+	SBI_SSE_EVENT_INJECT,
+	SBI_SSE_HART_UNMASK,
+	SBI_SSE_HART_MASK,
+};
+
+enum sbi_sse_state {
+	SBI_SSE_STATE_UNUSED     = 0,
+	SBI_SSE_STATE_REGISTERED = 1,
+	SBI_SSE_STATE_ENABLED    = 2,
+	SBI_SSE_STATE_RUNNING    = 3,
+};
+
+/* SBI SSE Event Attributes. */
+enum sbi_sse_attr_id {
+	SBI_SSE_ATTR_STATUS		= 0x00000000,
+	SBI_SSE_ATTR_PRIO		= 0x00000001,
+	SBI_SSE_ATTR_CONFIG		= 0x00000002,
+	SBI_SSE_ATTR_PREFERRED_HART	= 0x00000003,
+	SBI_SSE_ATTR_ENTRY_PC		= 0x00000004,
+	SBI_SSE_ATTR_ENTRY_ARG		= 0x00000005,
+	SBI_SSE_ATTR_INTERRUPTED_SEPC	= 0x00000006,
+	SBI_SSE_ATTR_INTERRUPTED_FLAGS	= 0x00000007,
+	SBI_SSE_ATTR_INTERRUPTED_A6	= 0x00000008,
+	SBI_SSE_ATTR_INTERRUPTED_A7	= 0x00000009,
+
+	SBI_SSE_ATTR_MAX		= 0x0000000A
+};
+
+#define SBI_SSE_ATTR_STATUS_STATE_OFFSET	0
+#define SBI_SSE_ATTR_STATUS_STATE_MASK		0x3
+#define SBI_SSE_ATTR_STATUS_PENDING_OFFSET	2
+#define SBI_SSE_ATTR_STATUS_INJECT_OFFSET	3
+
+#define SBI_SSE_ATTR_CONFIG_ONESHOT	BIT(0)
+
+#define SBI_SSE_ATTR_INTERRUPTED_FLAGS_SSTATUS_SPP	BIT(0)
+#define SBI_SSE_ATTR_INTERRUPTED_FLAGS_SSTATUS_SPIE	BIT(1)
+#define SBI_SSE_ATTR_INTERRUPTED_FLAGS_HSTATUS_SPV	BIT(2)
+#define SBI_SSE_ATTR_INTERRUPTED_FLAGS_HSTATUS_SPVP	BIT(3)
+#define SBI_SSE_ATTR_INTERRUPTED_FLAGS_SSTATUS_SPELP	BIT(4)
+#define SBI_SSE_ATTR_INTERRUPTED_FLAGS_SSTATUS_SDT	BIT(5)
+
+#define SBI_SSE_EVENT_LOCAL_HIGH_PRIO_RAS	0x00000000
+#define SBI_SSE_EVENT_LOCAL_DOUBLE_TRAP		0x00000001
+#define SBI_SSE_EVENT_GLOBAL_HIGH_PRIO_RAS	0x00008000
+#define SBI_SSE_EVENT_LOCAL_PMU_OVERFLOW	0x00010000
+#define SBI_SSE_EVENT_LOCAL_LOW_PRIO_RAS	0x00100000
+#define SBI_SSE_EVENT_GLOBAL_LOW_PRIO_RAS	0x00108000
+#define SBI_SSE_EVENT_LOCAL_SOFTWARE_INJECTED	0xffff0000
+#define SBI_SSE_EVENT_GLOBAL_SOFTWARE_INJECTED	0xffff8000
+
+#define SBI_SSE_EVENT_PLATFORM		BIT(14)
+#define SBI_SSE_EVENT_GLOBAL		BIT(15)
+
 /* SBI debug triggers function IDs */
 enum sbi_ext_dbtr_fid {
 	SBI_EXT_DBTR_NUM_TRIGGERS = 0,
-- 
2.50.1 (Apple Git-155)


^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH v13 02/11] riscv: add support for SBI Supervisor Software Events extension
  2026-11-19 12:08 [PATCH v13 00/11] riscv: add SBI Supervisor Software Events support Zhanpeng Zhang
  2026-11-19 12:08 ` [PATCH v13 01/11] riscv: add SBI SSE extension definitions Zhanpeng Zhang
@ 2026-11-19 12:08 ` Zhanpeng Zhang
  2026-11-19 12:36   ` sashiko-bot
  2026-11-19 12:08 ` [PATCH v13 03/11] riscv: sse: mask events during shutdown and kexec Zhanpeng Zhang
                   ` (8 subsequent siblings)
  10 siblings, 1 reply; 22+ messages in thread
From: Zhanpeng Zhang @ 2026-11-19 12:08 UTC (permalink / raw)
  To: pjw, palmer, aou, alex, himanshu.chauhan, conor, anup.patel
  Cc: clement.leger, cuiyunhui, atish.patra, peterz, mingo, acme,
	namhyung, mark.rutland, alexander.shishkin, jolsa, irogers,
	adrian.hunter, james.clark, will, tglx, corbet, rdunlap, shuah,
	yuanzhu, yang.susheng, linux-riscv, linux-kernel,
	linux-perf-users, linux-doc, linux-kselftest,
	zhangzhanpeng.jasper

From: Clément Léger <cleger@rivosinc.com>

The SBI SSE extension allows firmware to notify supervisor software of
events that must be delivered independently of normal S-mode interrupts.
Firmware saves the minimal state required to enter the supervisor handler,
and Linux builds the synthetic handler context around it.

SSE can arrive while Linux is already in an exception entry path. At that
point sscratch and tp may be in the middle of the normal trap-entry
exchange, so they cannot always identify current. Store current in a
per-CPU slot and use the hart ID passed by firmware to recover it.

Give each event, including each CPU instance of a local event, a dedicated
stack and shadow call stack. Synchronize vmapped stack ranges before
unmasking events so that the handler cannot take a vmalloc fault while
running in an NMI-like context.

The handler is a synthetic supervisor episode, but completion must resume
the context interrupted by the SSE. Preserve stvec and, when the hypervisor
extension is present, hstatus across the handler. Read the interrupted a6
and a7 values from the SSE attributes, construct pt_regs for the
interrupted context, and write back any changes made by the handler.

Nested exceptions on the SSE event stack temporarily replace both
TASK_TI_KERNEL_SP and TASK_TI_USER_SP. Preserve their original values
across the handler and restore them before completing the event, so a
nested exception cannot leave the interrupted task referring to the event
stack.

On RV64 with an MMU, early trap entry also uses thread_info.a0/a1/a2
as scratch slots before saving the full register frame. An SSE handler
can interrupt that window and take a nested fault which overwrites the
same slots. Preserve them in already-saved callee-saved registers and
restore them before resuming the interrupted entry path.

Disable preemption while initializing the per-CPU entry-task slot.
CPU hotplug callbacks run in a preemptible thread, and this write
happens before the guards around the local SBI operations.

Compile the stack TLB helper only for VMAP_STACK or SHADOW_CALL_STACK,
its only users, to avoid an unused-function warning in other builds.

Keep an explicit EVENT_REGISTER not-supported result distinct from other
firmware failures. This lets clients select another delivery mechanism
only when firmware has positively rejected the requested event.

Link: https://lore.kernel.org/r/as79MHtNYd-VbUug@hu-himchau-blr.qualcomm.com/
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202610020009.P3ILnDi1-lkp@intel.com/
Signed-off-by: Clément Léger <cleger@rivosinc.com>
Co-developed-by: Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>
Signed-off-by: Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>
Co-developed-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
Signed-off-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
---
 arch/riscv/include/asm/asm.h         |  14 +-
 arch/riscv/include/asm/scs.h         |   7 +
 arch/riscv/include/asm/sse.h         |  82 +++++++++
 arch/riscv/include/asm/thread_info.h |   1 +
 arch/riscv/kernel/Makefile           |   1 +
 arch/riscv/kernel/asm-offsets.c      |  14 ++
 arch/riscv/kernel/entry.S            |  14 ++
 arch/riscv/kernel/sbi_sse.c          | 251 +++++++++++++++++++++++++++
 arch/riscv/kernel/sbi_sse_entry.S    | 237 +++++++++++++++++++++++++
 9 files changed, 618 insertions(+), 3 deletions(-)
 create mode 100644 arch/riscv/include/asm/sse.h
 create mode 100644 arch/riscv/kernel/sbi_sse.c
 create mode 100644 arch/riscv/kernel/sbi_sse_entry.S

diff --git a/arch/riscv/include/asm/asm.h b/arch/riscv/include/asm/asm.h
index b8bf842d4c13..e1196caea02d 100644
--- a/arch/riscv/include/asm/asm.h
+++ b/arch/riscv/include/asm/asm.h
@@ -91,16 +91,24 @@
 .endm
 
 #ifdef CONFIG_SMP
-.macro asm_per_cpu dst sym tmp
-	lw    \tmp, TASK_TI_CPU_NUM(tp)
-	slli  \tmp, \tmp, RISCV_LGPTR
+.macro asm_per_cpu_with_cpu dst sym tmp cpu
+	slli  \tmp, \cpu, RISCV_LGPTR
 	la    \dst, __per_cpu_offset
 	add   \dst, \dst, \tmp
 	REG_L \tmp, 0(\dst)
 	la    \dst, \sym
 	add   \dst, \dst, \tmp
 .endm
+
+.macro asm_per_cpu dst sym tmp
+	lw \tmp, TASK_TI_CPU_NUM(tp)
+	asm_per_cpu_with_cpu \dst \sym \tmp \tmp
+.endm
 #else /* CONFIG_SMP */
+.macro asm_per_cpu_with_cpu dst sym tmp cpu
+	la    \dst, \sym
+.endm
+
 .macro asm_per_cpu dst sym tmp
 	la    \dst, \sym
 .endm
diff --git a/arch/riscv/include/asm/scs.h b/arch/riscv/include/asm/scs.h
index 49e40eef4ed7..af16b1cb974e 100644
--- a/arch/riscv/include/asm/scs.h
+++ b/arch/riscv/include/asm/scs.h
@@ -18,6 +18,11 @@
 	load_per_cpu gp, irq_shadow_call_stack_ptr, \tmp
 .endm
 
+/* Load the per-CPU IRQ shadow call stack to gp. */
+.macro scs_load_sse_stack reg_evt
+	REG_L gp, SSE_REG_EVT_SHADOW_STACK(\reg_evt)
+.endm
+
 /* Load task_scs_sp(current) to gp. */
 .macro scs_load_current
 	REG_L	gp, TASK_TI_SCS_SP(tp)
@@ -49,6 +54,8 @@
 .endm
 .macro scs_load_irq_stack tmp
 .endm
+.macro scs_load_sse_stack reg_evt
+.endm
 .macro scs_load_current
 .endm
 .macro scs_load_current_if_from_user status, tmp
diff --git a/arch/riscv/include/asm/sse.h b/arch/riscv/include/asm/sse.h
new file mode 100644
index 000000000000..cbd8618c6e00
--- /dev/null
+++ b/arch/riscv/include/asm/sse.h
@@ -0,0 +1,82 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2024 Rivos Inc.
+ */
+#ifndef __ASM_SSE_H
+#define __ASM_SSE_H
+
+#include <linux/printk.h>
+#include <linux/types.h>
+
+#include <asm/sbi.h>
+
+static inline bool riscv_sse_available(void)
+{
+#ifdef CONFIG_RISCV_SBI
+	return sbi_probe_extension(SBI_EXT_SSE) > 0;
+#else
+	return false;
+#endif
+}
+
+static inline void riscv_sse_mask_current_hart(void)
+{
+#ifdef CONFIG_RISCV_SBI
+	struct sbiret ret;
+
+	if (!riscv_sse_available())
+		return;
+
+	ret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_HART_MASK, 0, 0, 0, 0, 0, 0);
+	if (ret.error && ret.error != SBI_ERR_ALREADY_STOPPED)
+		pr_emerg("SSE hart mask failed: %ld\n", ret.error);
+#endif
+}
+
+#ifdef CONFIG_RISCV_SBI_SSE
+
+struct sse_event_interrupted_state {
+	unsigned long a6;
+	unsigned long a7;
+};
+
+struct sse_event_arch_data {
+	void *stack;
+	void *shadow_stack;
+	unsigned long tmp;
+	struct sse_event_interrupted_state *interrupted;
+	phys_addr_t interrupted_phys;
+	u32 evt_id;
+	unsigned long hart_id;
+	unsigned int cpu_id;
+};
+
+struct riscv_sse_interrupted_context {
+	struct pt_regs *regs;
+	unsigned long hstatus;
+};
+
+static inline bool sse_event_is_global(u32 evt)
+{
+	return !!(evt & SBI_SSE_EVENT_GLOBAL);
+}
+
+void arch_sse_event_update_cpu(struct sse_event_arch_data *arch_evt, int cpu);
+int arch_sse_init_event(struct sse_event_arch_data *arch_evt, u32 evt_id,
+			int cpu);
+void arch_sse_free_event(struct sse_event_arch_data *arch_evt);
+int arch_sse_register_event(struct sse_event_arch_data *arch_evt);
+void arch_sse_init_cpu(void);
+
+void sse_handle_event(struct sse_event_arch_data *arch_evt,
+		      struct pt_regs *regs);
+asmlinkage void handle_sse(void);
+asmlinkage void noinstr do_sse(struct sse_event_arch_data *arch_evt,
+			       struct pt_regs *regs, unsigned long hstatus);
+
+const struct riscv_sse_interrupted_context *
+riscv_sse_get_interrupted_context(void);
+
+#endif
+
+#endif
diff --git a/arch/riscv/include/asm/thread_info.h b/arch/riscv/include/asm/thread_info.h
index 55019fdfa9ec..d14b45610c73 100644
--- a/arch/riscv/include/asm/thread_info.h
+++ b/arch/riscv/include/asm/thread_info.h
@@ -36,6 +36,7 @@
 #define OVERFLOW_STACK_SIZE     SZ_4K
 
 #define IRQ_STACK_SIZE		THREAD_SIZE
+#define SSE_STACK_SIZE		THREAD_SIZE
 
 #ifndef __ASSEMBLER__
 
diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile
index f54c336bf547..0728362ba0ae 100644
--- a/arch/riscv/kernel/Makefile
+++ b/arch/riscv/kernel/Makefile
@@ -101,6 +101,7 @@ obj-$(CONFIG_DYNAMIC_FTRACE)	+= mcount-dyn.o
 obj-$(CONFIG_PERF_EVENTS)	+= perf_callchain.o
 obj-$(CONFIG_HAVE_PERF_REGS)	+= perf_regs.o
 obj-$(CONFIG_RISCV_SBI)		+= sbi.o sbi_ecall.o
+obj-$(CONFIG_RISCV_SBI_SSE)	+= sbi_sse.o sbi_sse_entry.o
 ifeq ($(CONFIG_RISCV_SBI), y)
 obj-$(CONFIG_SMP)		+= sbi-ipi.o
 obj-$(CONFIG_SMP) += cpu_ops_sbi.o
diff --git a/arch/riscv/kernel/asm-offsets.c b/arch/riscv/kernel/asm-offsets.c
index a75f0cfea1e9..15363703cdd6 100644
--- a/arch/riscv/kernel/asm-offsets.c
+++ b/arch/riscv/kernel/asm-offsets.c
@@ -15,6 +15,8 @@
 #include <asm/ptrace.h>
 #include <asm/cpu_ops_sbi.h>
 #include <asm/stacktrace.h>
+#include <asm/sbi.h>
+#include <asm/sse.h>
 #include <asm/suspend.h>
 
 void asm_offsets(void);
@@ -533,6 +535,18 @@ void asm_offsets(void)
 	DEFINE(FREGS_A6,	    offsetof(struct __arch_ftrace_regs, a6));
 	DEFINE(FREGS_A7,	    offsetof(struct __arch_ftrace_regs, a7));
 #endif
+
+#ifdef CONFIG_RISCV_SBI_SSE
+	OFFSET(SSE_REG_EVT_STACK, sse_event_arch_data, stack);
+	OFFSET(SSE_REG_EVT_SHADOW_STACK, sse_event_arch_data, shadow_stack);
+	OFFSET(SSE_REG_EVT_TMP, sse_event_arch_data, tmp);
+	OFFSET(SSE_REG_HART_ID, sse_event_arch_data, hart_id);
+	OFFSET(SSE_REG_CPU_ID, sse_event_arch_data, cpu_id);
+
+	DEFINE(SBI_EXT_SSE, SBI_EXT_SSE);
+	DEFINE(SBI_SSE_EVENT_COMPLETE, SBI_SSE_EVENT_COMPLETE);
+	DEFINE(ASM_NR_CPUS, CONFIG_NR_CPUS);
+#endif
 #ifdef CONFIG_RISCV_SBI
 	DEFINE(SBI_EXT_FWFT, SBI_EXT_FWFT);
 	DEFINE(SBI_EXT_FWFT_SET, SBI_EXT_FWFT_SET);
diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
index 628b85e3915a..01d024789b0e 100644
--- a/arch/riscv/kernel/entry.S
+++ b/arch/riscv/kernel/entry.S
@@ -424,6 +424,15 @@ SYM_FUNC_END(call_on_irq_stack)
  * arguments are passed to schedule_tail.
  */
 SYM_FUNC_START(__switch_to)
+#ifdef CONFIG_RISCV_SBI_SSE
+	/*
+	 * Mark the interval where tp changes from prev to next. SSE entry uses
+	 * the interrupted tp while this per-CPU pointer is NULL.
+	 */
+	asm_per_cpu t0, __sbi_sse_entry_task, t1
+	REG_S zero, 0(t0)
+#endif
+
 	/* Save context into prev->thread */
 	li    a4,  TASK_THREAD_RA
 	add   a3, a0, a4
@@ -470,6 +479,11 @@ SYM_FUNC_START(__switch_to)
 	REG_L s11, TASK_THREAD_S11_RA(a4)
 	/* The offset of thread_info in task_struct is zero. */
 	move tp, a1
+#ifdef CONFIG_RISCV_SBI_SSE
+	/* Publish next only after tp contains its task_struct pointer. */
+	asm_per_cpu t0, __sbi_sse_entry_task, t1
+	REG_S tp, 0(t0)
+#endif
 	/* Switch to the next shadow call stack */
 	scs_load_current
 	ret
diff --git a/arch/riscv/kernel/sbi_sse.c b/arch/riscv/kernel/sbi_sse.c
new file mode 100644
index 000000000000..032fcc07abcf
--- /dev/null
+++ b/arch/riscv/kernel/sbi_sse.c
@@ -0,0 +1,251 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2025 Rivos Inc.
+ */
+#include <linux/nmi.h>
+#include <linux/sched.h>
+#include <linux/scs.h>
+#include <linux/bitfield.h>
+#include <linux/percpu-defs.h>
+#include <linux/preempt.h>
+#include <linux/string.h>
+
+#include <asm/asm-prototypes.h>
+#include <asm/switch_to.h>
+#include <asm/irq_stack.h>
+#include <asm/sbi.h>
+#include <asm/sse.h>
+#include <asm/tlbflush.h>
+
+DEFINE_PER_CPU(struct task_struct *, __sbi_sse_entry_task);
+static DEFINE_PER_CPU(struct riscv_sse_interrupted_context *,
+			      riscv_sse_interrupted_context);
+
+const struct riscv_sse_interrupted_context *
+riscv_sse_get_interrupted_context(void)
+{
+	return this_cpu_read(riscv_sse_interrupted_context);
+}
+
+void __weak sse_handle_event(struct sse_event_arch_data *arch_evt, struct pt_regs *regs)
+{
+}
+
+void noinstr do_sse(struct sse_event_arch_data *arch_evt,
+		    struct pt_regs *regs, unsigned long hstatus)
+{
+	struct riscv_sse_interrupted_context context = { regs, hstatus };
+	struct riscv_sse_interrupted_context *previous;
+	struct sbiret sret;
+
+	nmi_enter();
+	instrumentation_begin();
+
+	/* Retrieve missing GPRs from SBI */
+	sret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_EVENT_ATTR_READ, arch_evt->evt_id,
+			 SBI_SSE_ATTR_INTERRUPTED_A6,
+			 (SBI_SSE_ATTR_INTERRUPTED_A7 -
+			  SBI_SSE_ATTR_INTERRUPTED_A6) + 1,
+			 (unsigned long)arch_evt->interrupted_phys, 0, 0);
+	if (sret.error) {
+		pr_warn("Failed to read interrupted registers for event %x: %ld\n",
+			arch_evt->evt_id, sret.error);
+		/* Let the client quiesce its source without using incomplete regs. */
+		sse_handle_event(arch_evt, NULL);
+		goto out;
+	}
+
+	memcpy(&regs->a6, arch_evt->interrupted,
+	       sizeof(*arch_evt->interrupted));
+
+	/* Make the interrupted frame visible while clients handle this event. */
+	previous = this_cpu_read(riscv_sse_interrupted_context);
+	this_cpu_write(riscv_sse_interrupted_context, &context);
+	sse_handle_event(arch_evt, regs);
+	this_cpu_write(riscv_sse_interrupted_context, previous);
+
+	if (memcmp(&regs->a6, arch_evt->interrupted,
+		   sizeof(*arch_evt->interrupted))) {
+		memcpy(arch_evt->interrupted, &regs->a6,
+		       sizeof(*arch_evt->interrupted));
+		sret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_EVENT_ATTR_WRITE,
+				 arch_evt->evt_id, SBI_SSE_ATTR_INTERRUPTED_A6,
+				 (SBI_SSE_ATTR_INTERRUPTED_A7 -
+				  SBI_SSE_ATTR_INTERRUPTED_A6) + 1,
+				 (unsigned long)arch_evt->interrupted_phys, 0, 0);
+		/*
+		 * If writeback fails, COMPLETE resumes with firmware's original
+		 * a6/a7 rather than treating the shared buffer as committed.
+		 */
+		if (sret.error)
+			pr_warn("Failed to write interrupted registers for event %x: %ld\n",
+				arch_evt->evt_id, sret.error);
+	}
+
+out:
+	instrumentation_end();
+	nmi_exit();
+}
+
+static void *alloc_to_stack_pointer(void *alloc)
+{
+	return alloc ? alloc + SSE_STACK_SIZE : NULL;
+}
+
+static void *stack_pointer_to_alloc(void *stack)
+{
+	return stack ? stack - SSE_STACK_SIZE : NULL;
+}
+
+#if defined(CONFIG_VMAP_STACK) || defined(CONFIG_SHADOW_CALL_STACK)
+static void arch_sse_flush_tlb_range(struct sse_event_arch_data *arch_evt,
+				     unsigned long start, unsigned long size)
+{
+	unsigned long end = start + size;
+
+	if (sse_event_is_global(arch_evt->evt_id))
+		flush_tlb_kernel_range(start, end);
+	else
+		local_flush_tlb_kernel_range(start, end);
+}
+#endif
+
+static void arch_sse_shadow_stack_cpu_sync(struct sse_event_arch_data *arch_evt)
+{
+#ifdef CONFIG_SHADOW_CALL_STACK
+	if (arch_evt->shadow_stack)
+		arch_sse_flush_tlb_range(arch_evt,
+					 (unsigned long)arch_evt->shadow_stack,
+					 SCS_SIZE);
+#endif
+}
+
+#ifdef CONFIG_VMAP_STACK
+static void *sse_stack_alloc(unsigned int cpu)
+{
+	void *stack = arch_alloc_vmap_stack(SSE_STACK_SIZE, cpu_to_node(cpu));
+
+	return alloc_to_stack_pointer(stack);
+}
+
+static void sse_stack_free(void *stack)
+{
+	vfree(stack_pointer_to_alloc(stack));
+}
+
+static void arch_sse_stack_cpu_sync(struct sse_event_arch_data *arch_evt)
+{
+	void *p_stack = arch_evt->stack;
+	unsigned long stack = (unsigned long)stack_pointer_to_alloc(p_stack);
+
+	/*
+	 * Flush the tlb to avoid taking any exception when accessing the
+	 * vmapped stack inside the SSE handler
+	 */
+	arch_sse_flush_tlb_range(arch_evt, stack, SSE_STACK_SIZE);
+
+	arch_sse_shadow_stack_cpu_sync(arch_evt);
+}
+#else /* CONFIG_VMAP_STACK */
+static void *sse_stack_alloc(unsigned int cpu)
+{
+	void *stack = kmalloc(SSE_STACK_SIZE, GFP_KERNEL);
+
+	return alloc_to_stack_pointer(stack);
+}
+
+static void sse_stack_free(void *stack)
+{
+	kfree(stack_pointer_to_alloc(stack));
+}
+
+static void arch_sse_stack_cpu_sync(struct sse_event_arch_data *arch_evt)
+{
+	arch_sse_shadow_stack_cpu_sync(arch_evt);
+}
+#endif /* CONFIG_VMAP_STACK */
+
+static int sse_init_scs(int cpu, struct sse_event_arch_data *arch_evt)
+{
+	void *stack;
+
+	if (!scs_is_enabled())
+		return 0;
+
+	stack = scs_alloc(cpu_to_node(cpu));
+	if (!stack)
+		return -ENOMEM;
+
+	arch_evt->shadow_stack = stack;
+
+	return 0;
+}
+
+void arch_sse_event_update_cpu(struct sse_event_arch_data *arch_evt, int cpu)
+{
+	arch_evt->cpu_id = cpu;
+	arch_evt->hart_id = cpuid_to_hartid_map(cpu);
+}
+
+void arch_sse_init_cpu(void)
+{
+	preempt_disable();
+	this_cpu_write(__sbi_sse_entry_task, current);
+	preempt_enable();
+}
+
+int arch_sse_init_event(struct sse_event_arch_data *arch_evt, u32 evt_id,
+			int cpu)
+{
+	void *stack;
+
+	arch_evt->interrupted = kmalloc_obj(*arch_evt->interrupted, GFP_KERNEL);
+	if (!arch_evt->interrupted)
+		return -ENOMEM;
+
+	arch_evt->evt_id = evt_id;
+	stack = sse_stack_alloc(cpu);
+	if (!stack)
+		goto err_free_interrupted;
+
+	arch_evt->stack = stack;
+
+	if (sse_init_scs(cpu, arch_evt)) {
+		sse_stack_free(arch_evt->stack);
+		goto err_free_interrupted;
+	}
+
+	/* kmalloc keeps the two adjacent SBI attribute words contiguous. */
+	arch_evt->interrupted_phys = virt_to_phys(arch_evt->interrupted);
+
+	arch_sse_event_update_cpu(arch_evt, cpu);
+
+	return 0;
+
+err_free_interrupted:
+	kfree(arch_evt->interrupted);
+	arch_evt->interrupted = NULL;
+	return -ENOMEM;
+}
+
+void arch_sse_free_event(struct sse_event_arch_data *arch_evt)
+{
+	scs_free(arch_evt->shadow_stack);
+	sse_stack_free(arch_evt->stack);
+	kfree(arch_evt->interrupted);
+}
+
+int arch_sse_register_event(struct sse_event_arch_data *arch_evt)
+{
+	struct sbiret sret;
+
+	arch_sse_stack_cpu_sync(arch_evt);
+
+	sret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_EVENT_REGISTER, arch_evt->evt_id,
+			 (unsigned long)handle_sse, (unsigned long)arch_evt, 0,
+			 0, 0);
+	if (sret.error == SBI_ERR_NOT_SUPPORTED)
+		return -EOPNOTSUPP;
+
+	return sbi_err_map_linux_errno(sret.error);
+}
diff --git a/arch/riscv/kernel/sbi_sse_entry.S b/arch/riscv/kernel/sbi_sse_entry.S
new file mode 100644
index 000000000000..1e38fbb1c2a4
--- /dev/null
+++ b/arch/riscv/kernel/sbi_sse_entry.S
@@ -0,0 +1,237 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (C) 2025 Rivos Inc.
+ */
+
+#include <linux/init.h>
+#include <linux/linkage.h>
+
+#include <asm/alternative-macros.h>
+#include <asm/asm.h>
+#include <asm/csr.h>
+#include <asm/hwcap.h>
+#include <asm/scs.h>
+
+/* When entering handle_sse, the following registers are set:
+ * a6: contains the hartid
+ * a7: contains a sse_event_arch_data struct pointer
+ */
+SYM_CODE_START(handle_sse)
+	/* Save stack temporarily */
+	REG_S sp, SSE_REG_EVT_TMP(a7)
+	/* Set entry stack */
+	REG_L sp, SSE_REG_EVT_STACK(a7)
+
+	addi sp, sp, -(PT_SIZE_ON_STACK)
+	REG_S ra, PT_RA(sp)
+	REG_S s0, PT_S0(sp)
+	REG_S s1, PT_S1(sp)
+	REG_S s2, PT_S2(sp)
+	REG_S s3, PT_S3(sp)
+	REG_S s4, PT_S4(sp)
+	REG_S s5, PT_S5(sp)
+	REG_S s6, PT_S6(sp)
+	REG_S s7, PT_S7(sp)
+	REG_S s8, PT_S8(sp)
+	REG_S s9, PT_S9(sp)
+	REG_S s10, PT_S10(sp)
+	REG_S s11, PT_S11(sp)
+	REG_S tp, PT_TP(sp)
+	REG_S t0, PT_T0(sp)
+	REG_S t1, PT_T1(sp)
+	REG_S t2, PT_T2(sp)
+	REG_S t3, PT_T3(sp)
+	REG_S t4, PT_T4(sp)
+	REG_S t5, PT_T5(sp)
+	REG_S t6, PT_T6(sp)
+	REG_S gp, PT_GP(sp)
+	REG_S a0, PT_A0(sp)
+	REG_S a1, PT_A1(sp)
+	REG_S a2, PT_A2(sp)
+	REG_S a3, PT_A3(sp)
+	REG_S a4, PT_A4(sp)
+	REG_S a5, PT_A5(sp)
+
+	/* Retrieve entry sp */
+	REG_L a4, SSE_REG_EVT_TMP(a7)
+	/* Save CSRs */
+	csrr a0, CSR_EPC
+	csrr a1, CSR_SSTATUS
+	csrr a2, CSR_STVAL
+	csrr a3, CSR_SCAUSE
+
+	REG_S a0, PT_EPC(sp)
+	REG_S a1, PT_STATUS(sp)
+	REG_S a2, PT_BADADDR(sp)
+	REG_S a3, PT_CAUSE(sp)
+	REG_S a4, PT_SP(sp)
+
+	/* Disable user memory access and floating/vector computing */
+	li t0, SR_SUM | SR_FS_VS
+	csrc CSR_STATUS, t0
+
+	load_global_pointer
+	scs_load_sse_stack a7
+
+#ifdef CONFIG_SMP
+	REG_L t4, SSE_REG_HART_ID(a7)
+	lw t3, SSE_REG_CPU_ID(a7)
+
+	bne t4, a6, .Lfind_hart_id_slowpath
+
+.Lcpu_id_found:
+#else
+	mv t3, zero
+#endif
+
+	asm_per_cpu_with_cpu t2 __sbi_sse_entry_task t1 t3
+	REG_L tp, 0(t2)
+	bnez tp, .Lcurrent_task_found
+
+	/* __switch_to() marks its transition window with a NULL entry task. */
+	REG_L tp, PT_TP(sp)
+
+.Lcurrent_task_found:
+	/* Nested exceptions temporarily replace these with the SSE stack. */
+	REG_L s6, TASK_TI_KERNEL_SP(tp)
+	REG_L s7, TASK_TI_USER_SP(tp)
+#if defined(CONFIG_64BIT) && defined(CONFIG_MMU)
+	/* A nested fault must not overwrite an interrupted trap's scratch slots. */
+	REG_L s8, TASK_TI_A0(tp)
+	REG_L s9, TASK_TI_A1(tp)
+	REG_L s10, TASK_TI_A2(tp)
+#endif
+
+	mv a1, sp /* pt_regs on stack */
+
+	/*
+	 * Run the SSE handler with the normal exception vector, but restore the
+	 * interrupted stvec before completing the event. SSE can arrive while
+	 * the kernel is using a temporary trap vector in a sensitive entry path.
+	 */
+	csrr s3, CSR_STVEC
+	la t0, handle_exception
+	csrw CSR_STVEC, t0
+
+	/*
+	 * Preserve the full HS-mode virtualization state across the handler.
+	 * hstatus is live supervisor state rather than an SSE interrupted
+	 * attribute, and OpenSBI consumes hstatus.SPV during event completion.
+	 * Saving the whole CSR keeps the handler episode transparent to KVM and
+	 * avoids having to infer which hstatus bits may matter to a guest resume.
+	 */
+	li s5, 0
+	ALTERNATIVE("nop", "csrr s5, hstatus", 0, RISCV_ISA_EXT_H, 1)
+
+	/*
+	 * Save sscratch for restoration since we might have interrupted the
+	 * kernel in early exception path and thus, we don't know the content of
+	 * sscratch.
+	 */
+	csrrw s4, CSR_SSCRATCH, x0
+
+	mv a0, a7
+	mv a2, s5
+
+	call do_sse
+
+	/* Leave no reference to the event stack in the interrupted task. */
+	REG_S s7, TASK_TI_USER_SP(tp)
+	REG_S s6, TASK_TI_KERNEL_SP(tp)
+#if defined(CONFIG_64BIT) && defined(CONFIG_MMU)
+	REG_S s8, TASK_TI_A0(tp)
+	REG_S s9, TASK_TI_A1(tp)
+	REG_S s10, TASK_TI_A2(tp)
+#endif
+
+	csrw CSR_SSCRATCH, s4
+	ALTERNATIVE("nop", "csrw hstatus, s5", 0, RISCV_ISA_EXT_H, 1)
+	csrw CSR_STVEC, s3
+
+	REG_L a0, PT_STATUS(sp)
+	REG_L a1, PT_EPC(sp)
+	REG_L a2, PT_BADADDR(sp)
+	REG_L a3, PT_CAUSE(sp)
+	csrw CSR_SSTATUS, a0
+	csrw CSR_EPC, a1
+	csrw CSR_STVAL, a2
+	csrw CSR_SCAUSE, a3
+
+	REG_L ra, PT_RA(sp)
+	REG_L s0, PT_S0(sp)
+	REG_L s1, PT_S1(sp)
+	REG_L s2, PT_S2(sp)
+	REG_L s3, PT_S3(sp)
+	REG_L s4, PT_S4(sp)
+	REG_L s5, PT_S5(sp)
+	REG_L s6, PT_S6(sp)
+	REG_L s7, PT_S7(sp)
+	REG_L s8, PT_S8(sp)
+	REG_L s9, PT_S9(sp)
+	REG_L s10, PT_S10(sp)
+	REG_L s11, PT_S11(sp)
+	REG_L tp, PT_TP(sp)
+	REG_L t0, PT_T0(sp)
+	REG_L t1, PT_T1(sp)
+	REG_L t2, PT_T2(sp)
+	REG_L t3, PT_T3(sp)
+	REG_L t4, PT_T4(sp)
+	REG_L t5, PT_T5(sp)
+	REG_L t6, PT_T6(sp)
+	REG_L gp, PT_GP(sp)
+	REG_L a0, PT_A0(sp)
+	REG_L a1, PT_A1(sp)
+	REG_L a2, PT_A2(sp)
+	REG_L a3, PT_A3(sp)
+	REG_L a4, PT_A4(sp)
+	REG_L a5, PT_A5(sp)
+
+	REG_L sp, PT_SP(sp)
+
+	li a7, SBI_EXT_SSE
+	li a6, SBI_SSE_EVENT_COMPLETE
+	ecall
+
+	/*
+	 * COMPLETE must resume the interrupted context and never return. Trap
+	 * through the normal vector instead of falling into adjacent assembly.
+	 */
+	la t0, handle_exception
+	csrw CSR_STVEC, t0
+	ebreak
+	/* The fatal trap must not return; execution should never reach here. */
+
+#ifdef CONFIG_SMP
+.Lfind_hart_id_slowpath:
+
+	/* Restore current task struct from __sbi_sse_entry_task */
+	li t1, ASM_NR_CPUS
+	/* Slowpath to find the CPU id associated to the hart id */
+	la t0, __cpuid_to_hartid_map
+	li t3, 0
+
+.Lhart_id_loop:
+	REG_L t2, 0(t0)
+	beq t2, a6, .Lcpu_id_found
+
+	/* Increment pointer and CPU number */
+	addi t3, t3, 1
+	addi t0, t0, RISCV_SZPTR
+	bltu t3, t1, .Lhart_id_loop
+
+	/*
+	 * This should never happen since we expect the hart_id to match one
+	 * of our CPU, but better be safe than sorry
+	 */
+	la tp, init_task
+	la a0, sse_hart_id_panic_string
+	la t0, panic
+	jalr t0
+#endif
+
+SYM_CODE_END(handle_sse)
+ASM_NOKPROBE(handle_sse)
+
+SYM_DATA_START_LOCAL(sse_hart_id_panic_string)
+    .ascii "Unable to match hart_id with cpu\0"
+SYM_DATA_END(sse_hart_id_panic_string)
-- 
2.50.1 (Apple Git-155)


^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH v13 03/11] riscv: sse: mask events during shutdown and kexec
  2026-11-19 12:08 [PATCH v13 00/11] riscv: add SBI Supervisor Software Events support Zhanpeng Zhang
  2026-11-19 12:08 ` [PATCH v13 01/11] riscv: add SBI SSE extension definitions Zhanpeng Zhang
  2026-11-19 12:08 ` [PATCH v13 02/11] riscv: add support for SBI Supervisor Software Events extension Zhanpeng Zhang
@ 2026-11-19 12:08 ` Zhanpeng Zhang
  2026-11-19 12:38   ` sashiko-bot
  2026-11-19 12:08 ` [PATCH v13 04/11] drivers: firmware: add riscv SSE support Zhanpeng Zhang
                   ` (7 subsequent siblings)
  10 siblings, 1 reply; 22+ messages in thread
From: Zhanpeng Zhang @ 2026-11-19 12:08 UTC (permalink / raw)
  To: pjw, palmer, aou, alex, himanshu.chauhan, conor, anup.patel
  Cc: clement.leger, cuiyunhui, atish.patra, peterz, mingo, acme,
	namhyung, mark.rutland, alexander.shishkin, jolsa, irogers,
	adrian.hunter, james.clark, will, tglx, corbet, rdunlap, shuah,
	yuanzhu, yang.susheng, linux-riscv, linux-kernel,
	linux-perf-users, linux-doc, linux-kselftest,
	zhangzhanpeng.jasper

SSE delivery is independent of normal S-mode interrupts. Firmware may also
retain an event registration until Linux explicitly unregisters it. A hart
must therefore stop accepting SSE events before Linux stops servicing the
registered handler.

Mask SSE on the local hart before panic stop, CPU stop, restart, poweroff,
and crash shutdown paths. This prevents firmware from entering Linux-owned
handler state after the corresponding CPU or kernel context is no longer
valid.

A crash kernel cannot identify or take ownership of registrations
inherited from the crashed kernel. Reject a later normal kexec while such
SSE state may still exist, rather than transferring unknown firmware state
to another kernel.

The restriction is deliberately based on firmware SSE availability, not
CONFIG_RISCV_SBI_SSE or registrations made by the crash kernel. Disabling
SSE in the crash kernel does not establish that registrations left by the
crashed kernel are absent. Consequently, normal kexec from a crash kernel
is also rejected when firmware advertises SSE but no registrations were
actually left behind. This conservative restriction remains until a safe
way to establish or clear inherited firmware state is available.

Link: https://lore.kernel.org/r/as7-3PbWDxygadnJ@hu-himchau-blr.qualcomm.com/
Reviewed-by: Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>
Signed-off-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
---
 arch/riscv/kernel/machine_kexec.c | 11 +++++++++++
 arch/riscv/kernel/reset.c         | 18 ++++++++++++++++++
 arch/riscv/kernel/smp.c           | 17 +++++++++++++++++
 3 files changed, 46 insertions(+)

diff --git a/arch/riscv/kernel/machine_kexec.c b/arch/riscv/kernel/machine_kexec.c
index 738df176ff6f..24e7affae70b 100644
--- a/arch/riscv/kernel/machine_kexec.c
+++ b/arch/riscv/kernel/machine_kexec.c
@@ -14,10 +14,13 @@
 #include <asm/set_memory.h>	/* For set_memory_x() */
 #include <linux/compiler.h>	/* For unreachable() */
 #include <linux/cpu.h>		/* For cpu_down() */
+#include <linux/crash_dump.h>
 #include <linux/reboot.h>
 #include <linux/interrupt.h>
 #include <linux/irq.h>
 
+#include <asm/sse.h>
+
 /*
  * machine_kexec_prepare - Initialize kexec
  *
@@ -36,6 +39,13 @@ machine_kexec_prepare(struct kimage *image)
 	unsigned int control_code_buffer_sz = 0;
 	int i = 0;
 
+	/* A crash kernel cannot tear down registrations inherited from firmware. */
+	if (is_kdump_kernel() && image->type != KEXEC_TYPE_CRASH &&
+	    riscv_sse_available()) {
+		pr_err("Normal kexec from a crash kernel is unsupported with SSE\n");
+		return -EOPNOTSUPP;
+	}
+
 	/* Find the Flattened Device Tree and save its physical address */
 	for (i = 0; i < image->nr_segments; i++) {
 		if (image->segment[i].memsz <= sizeof(fdt))
@@ -127,6 +137,7 @@ void
 machine_crash_shutdown(struct pt_regs *regs)
 {
 	local_irq_disable();
+	riscv_sse_mask_current_hart();
 
 	/* shutdown non-crashing cpus */
 	crash_smp_send_stop();
diff --git a/arch/riscv/kernel/reset.c b/arch/riscv/kernel/reset.c
index 14eb08a6db85..fdab37e7ae52 100644
--- a/arch/riscv/kernel/reset.c
+++ b/arch/riscv/kernel/reset.c
@@ -6,6 +6,20 @@
 #include <linux/efi.h>
 #include <linux/reboot.h>
 #include <linux/pm.h>
+#include <linux/smp.h>
+
+#include <asm/sse.h>
+
+#ifndef CONFIG_SMP
+void __noreturn panic_smp_self_stop(void)
+{
+	riscv_sse_mask_current_hart();
+	local_irq_disable();
+
+	for (;;)
+		cpu_relax();
+}
+#endif
 
 static void __noreturn default_power_off(void)
 {
@@ -18,6 +32,8 @@ EXPORT_SYMBOL(pm_power_off);
 
 void machine_restart(char *cmd)
 {
+	riscv_sse_mask_current_hart();
+
 	/*
 	 * UpdateCapsule() depends on the system being reset via ResetSystem().
 	 */
@@ -30,12 +46,14 @@ void machine_restart(char *cmd)
 
 void machine_halt(void)
 {
+	riscv_sse_mask_current_hart();
 	do_kernel_power_off();
 	default_power_off();
 }
 
 void machine_power_off(void)
 {
+	riscv_sse_mask_current_hart();
 	do_kernel_power_off();
 	default_power_off();
 }
diff --git a/arch/riscv/kernel/smp.c b/arch/riscv/kernel/smp.c
index 07027cfeba4b..5b05243e1deb 100644
--- a/arch/riscv/kernel/smp.c
+++ b/arch/riscv/kernel/smp.c
@@ -22,10 +22,13 @@
 #include <linux/irq.h>
 #include <linux/irq_work.h>
 #include <linux/nmi.h>
+#include <linux/panic.h>
 
 #include <asm/tlbflush.h>
 #include <asm/cacheflush.h>
 #include <asm/cpu_ops.h>
+#include <asm/sbi.h>
+#include <asm/sse.h>
 
 static const char * const ipi_names[] = {
 	[IPI_RESCHEDULE]	= "Rescheduling interrupts",
@@ -66,8 +69,18 @@ int riscv_hartid_to_cpuid(unsigned long hartid)
 	return -ENOENT;
 }
 
+void __noreturn panic_smp_self_stop(void)
+{
+	riscv_sse_mask_current_hart();
+	local_irq_disable();
+
+	for (;;)
+		cpu_relax();
+}
+
 static void ipi_stop(void)
 {
+	riscv_sse_mask_current_hart();
 	set_cpu_online(smp_processor_id(), false);
 	while (1)
 		wait_for_interrupt();
@@ -78,6 +91,7 @@ static atomic_t waiting_for_crash_ipi = ATOMIC_INIT(0);
 
 static inline void ipi_cpu_crash_stop(unsigned int cpu, struct pt_regs *regs)
 {
+	riscv_sse_mask_current_hart();
 	crash_save_cpu(regs, cpu);
 
 	atomic_dec(&waiting_for_crash_ipi);
@@ -241,6 +255,8 @@ void smp_send_stop(void)
 {
 	unsigned long timeout;
 
+	riscv_sse_mask_current_hart();
+
 	if (num_online_cpus() > 1) {
 		cpumask_t mask;
 
@@ -288,6 +304,7 @@ void crash_smp_send_stop(void)
 		return;
 
 	cpus_stopped = 1;
+	riscv_sse_mask_current_hart();
 
 	/*
 	 * If this cpu is the only one alive at this point in time, online or
-- 
2.50.1 (Apple Git-155)


^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH v13 04/11] drivers: firmware: add riscv SSE support
  2026-11-19 12:08 [PATCH v13 00/11] riscv: add SBI Supervisor Software Events support Zhanpeng Zhang
                   ` (2 preceding siblings ...)
  2026-11-19 12:08 ` [PATCH v13 03/11] riscv: sse: mask events during shutdown and kexec Zhanpeng Zhang
@ 2026-11-19 12:08 ` Zhanpeng Zhang
  2026-11-19 12:38   ` sashiko-bot
  2026-11-19 12:08 ` [PATCH v13 05/11] riscv: mm: avoid enabling interrupts for nofault page faults Zhanpeng Zhang
                   ` (6 subsequent siblings)
  10 siblings, 1 reply; 22+ messages in thread
From: Zhanpeng Zhang @ 2026-11-19 12:08 UTC (permalink / raw)
  To: pjw, palmer, aou, alex, himanshu.chauhan, conor, anup.patel
  Cc: clement.leger, cuiyunhui, atish.patra, peterz, mingo, acme,
	namhyung, mark.rutland, alexander.shishkin, jolsa, irogers,
	adrian.hunter, james.clark, will, tglx, corbet, rdunlap, shuah,
	yuanzhu, yang.susheng, linux-riscv, linux-kernel,
	linux-perf-users, linux-doc, linux-kselftest,
	zhangzhanpeng.jasper

From: Clément Léger <cleger@rivosinc.com>

Add a driver-level interface for RISC-V SSE. Linux clients can register
handlers, select a target CPU for global events, and enable, disable or
unregister events. The architecture entry wrapper completes an event after
the registered handler returns. PMU and GHES drivers can use this
interface.

Represent global events with one firmware registration and local events
with one registration per possible CPU. Keep registration and enable state
stable across CPU hotplug, and validate firmware-provided hart IDs before
converting them to Linux CPU IDs. Use phys_addr_t for attribute buffers to
match the physical addresses passed to firmware. Require MMU support
because the current event-stack implementation relies on vmapped memory and
TLB synchronization.

Serialize client list updates with the SSE mutex and the CPU read lock.
Normal CPU hotplug callbacks provide the matching write-side exclusion, and
CPUHP state removal holds the SSE mutex. These rules avoid holding an
additional spinlock across firmware calls. Local event operations select a
per-CPU registration, so require callers to remain on the current CPU and
use lockdep assertions to verify that contract. Because CPU hotplug
callbacks run in a preemptible thread, disable preemption around the
local operations they perform on the offlining or onlining hart.

Propagate firmware failures from register, disable and unregister
operations. Update Linux state only after successful firmware operations,
and release an event only after all registrations are gone. Preserve the
difference between SBI_ERR_NOT_SUPPORTED and a generic SBI failure so
clients only select a fallback after firmware explicitly rejects an SSE
operation.

If setup fails partway through a local event, roll back only the CPU
instances changed by that invocation. Keep non-fallback errors across CPUs.
This lets a client distinguish an event rejected as unsupported by every
failing hart from an unknown firmware failure.

A failed registration rollback can leave firmware state without a client
handle. Retain these events on a driver-owned cleanup list with a no-op
handler. This lets CPU hotplug and shutdown retry cleanup without relying
on client callback lifetime.

Mask SSE before CPU teardown. If a normal CPU-offline teardown fails after
partially changing local events, reconstruct their requested registration
and enable state. Unmask the hart, then return the original error to abort
the offline operation. Shutdown and CPUHP state removal cannot fail, so
record incomplete cleanup and refuse an unsafe normal kexec instead.

Close enable admission before reboot or kexec teardown. Protect the
shutdown check and firmware enable operation with RCU, then drain existing
enable calls before CPUHP removes registrations. This prevents a client
from re-enabling an event between the teardown disable and unregister
operations.

A crash kernel cannot identify registrations inherited from the crashed
kernel. Leave SSE masked, report the retained firmware state separately
from an unavailable extension, and do not let clients select a delivery
path that may still be owned by the old kernel.

Signed-off-by: Clément Léger <cleger@rivosinc.com>
Co-developed-by: Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>
Signed-off-by: Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>
Co-developed-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
Signed-off-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
Acked-by: Conor Dooley <conor.dooley@microchip.com>
---
 drivers/firmware/Kconfig               |    1 +
 drivers/firmware/Makefile              |    1 +
 drivers/firmware/riscv/Kconfig         |   18 +
 drivers/firmware/riscv/Makefile        |    3 +
 drivers/firmware/riscv/riscv_sbi_sse.c | 1219 ++++++++++++++++++++++++
 include/linux/cpuhotplug.h             |    1 +
 include/linux/riscv_sbi_sse.h          |   89 ++
 7 files changed, 1332 insertions(+)
 create mode 100644 drivers/firmware/riscv/Kconfig
 create mode 100644 drivers/firmware/riscv/Makefile
 create mode 100644 drivers/firmware/riscv/riscv_sbi_sse.c
 create mode 100644 include/linux/riscv_sbi_sse.h

diff --git a/drivers/firmware/Kconfig b/drivers/firmware/Kconfig
index 43e2e2a64594..d5b31467e074 100644
--- a/drivers/firmware/Kconfig
+++ b/drivers/firmware/Kconfig
@@ -310,6 +310,7 @@ source "drivers/firmware/meson/Kconfig"
 source "drivers/firmware/microchip/Kconfig"
 source "drivers/firmware/psci/Kconfig"
 source "drivers/firmware/qcom/Kconfig"
+source "drivers/firmware/riscv/Kconfig"
 source "drivers/firmware/samsung/Kconfig"
 source "drivers/firmware/smccc/Kconfig"
 source "drivers/firmware/tegra/Kconfig"
diff --git a/drivers/firmware/Makefile b/drivers/firmware/Makefile
index 50b338313152..5c42a0958247 100644
--- a/drivers/firmware/Makefile
+++ b/drivers/firmware/Makefile
@@ -37,6 +37,7 @@ obj-y				+= efi/
 obj-y				+= imx/
 obj-y				+= psci/
 obj-y				+= qcom/
+obj-y				+= riscv/
 obj-y				+= samsung/
 obj-y				+= smccc/
 obj-y				+= tegra/
diff --git a/drivers/firmware/riscv/Kconfig b/drivers/firmware/riscv/Kconfig
new file mode 100644
index 000000000000..d15ff84e1258
--- /dev/null
+++ b/drivers/firmware/riscv/Kconfig
@@ -0,0 +1,18 @@
+# SPDX-License-Identifier: GPL-2.0-only
+menu "Risc-V Specific firmware drivers"
+depends on RISCV
+
+config RISCV_SBI_SSE
+	bool "Enable SBI Supervisor Software Events support"
+	depends on RISCV_SBI && MMU && !HIBERNATION
+	default y
+	help
+	  The Supervisor Software Events support allows the SBI to deliver
+	  NMI-like notifications to the supervisor mode software. When enabled,
+	  this option provides support to register callbacks on specific SSE
+	  events.
+
+	  Hibernation is not supported because firmware registrations do not
+	  survive restoring a kernel image.
+
+endmenu
diff --git a/drivers/firmware/riscv/Makefile b/drivers/firmware/riscv/Makefile
new file mode 100644
index 000000000000..c8795d4bbb2e
--- /dev/null
+++ b/drivers/firmware/riscv/Makefile
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0
+
+obj-$(CONFIG_RISCV_SBI_SSE)		+= riscv_sbi_sse.o
diff --git a/drivers/firmware/riscv/riscv_sbi_sse.c b/drivers/firmware/riscv/riscv_sbi_sse.c
new file mode 100644
index 000000000000..0cb738bab6c3
--- /dev/null
+++ b/drivers/firmware/riscv/riscv_sbi_sse.c
@@ -0,0 +1,1219 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2025 Rivos Inc.
+ */
+
+#define pr_fmt(fmt) "sse: " fmt
+
+#include <linux/atomic.h>
+#include <linux/cpu.h>
+#include <linux/cpuhotplug.h>
+#include <linux/cpu_pm.h>
+#include <linux/crash_dump.h>
+#include <linux/hardirq.h>
+#include <linux/kexec.h>
+#include <linux/list.h>
+#include <linux/panic_notifier.h>
+#include <linux/percpu-defs.h>
+#include <linux/rcupdate.h>
+#include <linux/reboot.h>
+#include <linux/riscv_sbi_sse.h>
+#include <linux/slab.h>
+#include <linux/smp.h>
+
+#include <asm/sbi.h>
+#include <asm/sse.h>
+
+struct sse_event {
+	struct list_head list;
+	u32 evt_id;
+	u32 priority;
+	sse_event_handler_fn __rcu *handler;
+	void *handler_arg;
+	/* Only valid for global events */
+	unsigned int cpu;
+	/*
+	 * Desired state requested by the client. Firmware state is tracked per
+	 * instance because a failed transition can leave a partial result.
+	 */
+	bool enable_requested;
+	/* Registration failed, but firmware state still needs driver cleanup. */
+	bool cleanup_pending;
+
+	union {
+		struct sse_registered_event *global;
+		struct sse_registered_event __percpu *local;
+	};
+};
+
+static bool sse_available __ro_after_init;
+static bool sse_fw_state_retained __ro_after_init;
+static bool sse_shutting_down;
+static atomic_t sse_teardown_failed = ATOMIC_INIT(0);
+/*
+ * Client-side updates hold sse_mutex and the CPU read lock. Normal CPU hotplug
+ * callbacks exclude them through the CPUHP write side. Initialization replay
+ * runs before clients can register, and state removal holds sse_mutex.
+ */
+static LIST_HEAD(events);
+static DEFINE_MUTEX(sse_mutex);
+
+/*
+ * A registration rollback can fail before the client receives an event
+ * handle. Keep the retained event independent of client-owned callback text
+ * and data while the driver retries firmware cleanup.
+ */
+static int sse_cleanup_event_handler(u32 evt, void *arg, struct pt_regs *regs)
+{
+	return 0;
+}
+
+struct sse_registered_event {
+	struct sse_event_arch_data arch;
+	struct sse_event *event;
+	unsigned long attr;
+	/*
+	 * Actual firmware state for one global or per-CPU instance. A retry can
+	 * then skip instances that already completed a partial transition.
+	 */
+	bool is_registered;
+	bool is_enabled;
+};
+
+void sse_handle_event(struct sse_event_arch_data *arch_event,
+		      struct pt_regs *regs)
+{
+	sse_event_handler_fn *handler;
+	int ret;
+	struct sse_registered_event *reg_evt =
+		container_of(arch_event, struct sse_registered_event, arch);
+	struct sse_event *evt = reg_evt->event;
+
+	rcu_read_lock();
+	handler = rcu_dereference(evt->handler);
+	ret = handler(evt->evt_id, evt->handler_arg, regs);
+	rcu_read_unlock();
+	if (ret)
+		pr_warn("event %x handler failed with error %d\n", evt->evt_id, ret);
+}
+
+static struct sse_event *sse_event_get(u32 evt)
+{
+	struct sse_event *event;
+
+	lockdep_assert_held(&sse_mutex);
+
+	list_for_each_entry(event, &events, list) {
+		if (event->evt_id == evt)
+			return event;
+	}
+
+	return NULL;
+}
+
+static phys_addr_t sse_event_get_attr_phys(struct sse_registered_event *reg_evt)
+{
+	phys_addr_t phys;
+	void *addr = &reg_evt->attr;
+
+	if (sse_event_is_global(reg_evt->event->evt_id))
+		phys = virt_to_phys(addr);
+	else
+		phys = per_cpu_ptr_to_phys(addr);
+
+	return phys;
+}
+
+static struct sse_registered_event *sse_get_reg_evt(struct sse_event *event)
+{
+	if (sse_event_is_global(event->evt_id))
+		return event->global;
+	else
+		return per_cpu_ptr(event->local, smp_processor_id());
+}
+
+static int sse_err_map_linux_errno(long err)
+{
+	if (err == SBI_ERR_NOT_SUPPORTED)
+		return -EOPNOTSUPP;
+
+	return sbi_err_map_linux_errno(err);
+}
+
+static int sse_sbi_event_func(struct sse_event *event, unsigned long func)
+{
+	struct sbiret ret;
+	u32 evt = event->evt_id;
+	struct sse_registered_event *reg_evt = sse_get_reg_evt(event);
+
+	ret = sbi_ecall(SBI_EXT_SSE, func, evt, 0, 0, 0, 0, 0);
+	if (ret.error) {
+		pr_warn("Failed to execute func %lx, event %x, error %ld\n",
+			func, evt, ret.error);
+		return sse_err_map_linux_errno(ret.error);
+	}
+
+	if (func == SBI_SSE_EVENT_DISABLE)
+		reg_evt->is_enabled = false;
+	else if (func == SBI_SSE_EVENT_ENABLE)
+		reg_evt->is_enabled = true;
+
+	return 0;
+}
+
+int sse_event_disable_local(struct sse_event *event)
+{
+	if (!sse_event_is_global(event->evt_id))
+		lockdep_assert_preemption_disabled();
+
+	if (!sse_get_reg_evt(event)->is_enabled)
+		return 0;
+
+	return sse_sbi_event_func(event, SBI_SSE_EVENT_DISABLE);
+}
+EXPORT_SYMBOL_GPL(sse_event_disable_local);
+
+int sse_event_enable_local(struct sse_event *event)
+{
+	struct sse_registered_event *reg_evt = sse_get_reg_evt(event);
+	int ret;
+
+	if (!sse_event_is_global(event->evt_id))
+		lockdep_assert_preemption_disabled();
+
+	rcu_read_lock();
+	if (READ_ONCE(sse_shutting_down)) {
+		ret = -ESHUTDOWN;
+		goto out;
+	}
+
+	if (!reg_evt->is_registered) {
+		ret = -EINVAL;
+		goto out;
+	}
+
+	if (reg_evt->is_enabled) {
+		ret = 0;
+		goto out;
+	}
+
+	ret = sse_sbi_event_func(event, SBI_SSE_EVENT_ENABLE);
+out:
+	rcu_read_unlock();
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(sse_event_enable_local);
+
+static int sse_event_attr_get_no_lock(struct sse_registered_event *reg_evt,
+				      unsigned long attr_id, unsigned long *val)
+{
+	struct sbiret sret;
+	u32 evt = reg_evt->event->evt_id;
+	phys_addr_t phys;
+
+	phys = sse_event_get_attr_phys(reg_evt);
+
+	sret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_EVENT_ATTR_READ, evt, attr_id, 1,
+			 (unsigned long)phys, 0, 0);
+	if (sret.error) {
+		pr_debug("Failed to get event %x attr %lx, error %ld\n", evt,
+			 attr_id, sret.error);
+		return sse_err_map_linux_errno(sret.error);
+	}
+
+	*val = reg_evt->attr;
+
+	return 0;
+}
+
+static int sse_event_attr_set_nolock(struct sse_registered_event *reg_evt,
+				     unsigned long attr_id, unsigned long val)
+{
+	struct sbiret sret;
+	u32 evt = reg_evt->event->evt_id;
+	phys_addr_t phys;
+
+	reg_evt->attr = val;
+	phys = sse_event_get_attr_phys(reg_evt);
+
+	sret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_EVENT_ATTR_WRITE, evt, attr_id, 1,
+			 (unsigned long)phys, 0, 0);
+	if (sret.error)
+		pr_debug("Failed to set event %x attr %lx, error %ld\n", evt,
+			 attr_id, sret.error);
+
+	return sse_err_map_linux_errno(sret.error);
+}
+
+static void sse_global_event_update_cpu(struct sse_event *event,
+					unsigned int cpu)
+{
+	struct sse_registered_event *reg_evt = event->global;
+
+	event->cpu = cpu;
+	arch_sse_event_update_cpu(&reg_evt->arch, cpu);
+}
+
+static int sse_event_set_target_cpu_nolock(struct sse_event *event,
+					   unsigned int cpu)
+{
+	unsigned long hart_id, old_hart_id;
+	struct sse_registered_event *reg_evt = event->global;
+	u32 evt = event->evt_id;
+	unsigned int old_cpu;
+	bool was_enabled;
+	int ret;
+
+	if (!sse_event_is_global(evt))
+		return -EINVAL;
+
+	if (cpu >= nr_cpu_ids || !cpu_online(cpu))
+		return -EINVAL;
+	hart_id = cpuid_to_hartid_map(cpu);
+	old_cpu = event->cpu;
+	old_hart_id = cpuid_to_hartid_map(old_cpu);
+
+	was_enabled = reg_evt->is_enabled;
+	if (was_enabled) {
+		ret = sse_event_disable_local(event);
+		if (ret)
+			return ret;
+	}
+
+	ret = sse_event_attr_set_nolock(reg_evt, SBI_SSE_ATTR_PREFERRED_HART,
+					hart_id);
+	if (ret == 0)
+		sse_global_event_update_cpu(event, cpu);
+
+	if (was_enabled) {
+		int enable_ret;
+
+		enable_ret = sse_event_enable_local(event);
+		if (enable_ret) {
+			int rollback_ret;
+
+			/*
+			 * The preferred hart was already changed. Restore both the
+			 * firmware attribute and Linux's cached target before reporting
+			 * the failed migration.
+			 */
+			rollback_ret = sse_event_attr_set_nolock(reg_evt,
+								 SBI_SSE_ATTR_PREFERRED_HART,
+								 old_hart_id);
+			if (!rollback_ret) {
+				sse_global_event_update_cpu(event, old_cpu);
+				rollback_ret = sse_event_enable_local(event);
+			}
+			if (!rollback_ret)
+				return enable_ret;
+
+			pr_warn("Failed to restore global event %x to CPU %u: %d\n",
+				evt, old_cpu, rollback_ret);
+			event->enable_requested = false;
+			return enable_ret;
+		}
+	}
+
+	return ret;
+}
+
+int sse_event_set_target_cpu(struct sse_event *event, unsigned int cpu)
+{
+	int ret;
+
+	if (cpu >= nr_cpu_ids)
+		return -EINVAL;
+
+	scoped_guard(mutex, &sse_mutex) {
+		if (READ_ONCE(sse_shutting_down))
+			return -ESHUTDOWN;
+
+		scoped_guard(cpus_read_lock) {
+			if (!cpu_online(cpu))
+				return -EINVAL;
+
+			ret = sse_event_set_target_cpu_nolock(event, cpu);
+		}
+	}
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(sse_event_set_target_cpu);
+
+static int sse_event_init_registered(unsigned int cpu,
+				     struct sse_registered_event *reg_evt,
+				     struct sse_event *event)
+{
+	reg_evt->event = event;
+	reg_evt->is_registered = false;
+	reg_evt->is_enabled = false;
+
+	return arch_sse_init_event(&reg_evt->arch, event->evt_id, cpu);
+}
+
+static void sse_event_free_registered(struct sse_registered_event *reg_evt)
+{
+	arch_sse_free_event(&reg_evt->arch);
+}
+
+static int sse_event_alloc_global(struct sse_event *event)
+{
+	unsigned int cpu;
+	int err;
+	struct sse_registered_event *reg_evt;
+
+	reg_evt = kzalloc_obj(*reg_evt, GFP_KERNEL);
+	if (!reg_evt)
+		return -ENOMEM;
+
+	event->global = reg_evt;
+	cpu = cpumask_first(cpu_possible_mask);
+	if (cpu >= nr_cpu_ids) {
+		kfree(reg_evt);
+		return -ENODEV;
+	}
+
+	err = sse_event_init_registered(cpu, reg_evt, event);
+	if (err)
+		kfree(reg_evt);
+
+	return err;
+}
+
+static int sse_event_alloc_local(struct sse_event *event)
+{
+	int err;
+	unsigned int cpu, err_cpu;
+	struct sse_registered_event *reg_evt;
+	struct sse_registered_event __percpu *reg_evts;
+
+	reg_evts = alloc_percpu(struct sse_registered_event);
+	if (!reg_evts)
+		return -ENOMEM;
+
+	event->local = reg_evts;
+
+	for_each_possible_cpu(cpu) {
+		reg_evt = per_cpu_ptr(reg_evts, cpu);
+		err = sse_event_init_registered(cpu, reg_evt, event);
+		if (err) {
+			err_cpu = cpu;
+			goto err_free_per_cpu;
+		}
+	}
+
+	return 0;
+
+err_free_per_cpu:
+	for_each_possible_cpu(cpu) {
+		if (cpu == err_cpu)
+			break;
+		reg_evt = per_cpu_ptr(reg_evts, cpu);
+		sse_event_free_registered(reg_evt);
+	}
+
+	free_percpu(reg_evts);
+
+	return err;
+}
+
+static struct sse_event *sse_event_alloc(u32 evt, u32 priority,
+					 sse_event_handler_fn *handler,
+					 void *arg)
+{
+	int err;
+	struct sse_event *event;
+
+	event = kzalloc_obj(*event, GFP_KERNEL);
+	if (!event)
+		return ERR_PTR(-ENOMEM);
+
+	event->evt_id = evt;
+	event->priority = priority;
+	event->handler_arg = arg;
+	RCU_INIT_POINTER(event->handler, handler);
+
+	if (sse_event_is_global(evt))
+		err = sse_event_alloc_global(event);
+	else
+		err = sse_event_alloc_local(event);
+
+	if (err) {
+		kfree(event);
+		return ERR_PTR(err);
+	}
+
+	return event;
+}
+
+static int sse_sbi_register_event(struct sse_event *event,
+				  struct sse_registered_event *reg_evt)
+{
+	int ret;
+
+	if (reg_evt->is_registered)
+		return 0;
+
+	ret = sse_event_attr_set_nolock(reg_evt, SBI_SSE_ATTR_PRIO,
+					event->priority);
+	if (ret)
+		return ret;
+
+	ret = arch_sse_register_event(&reg_evt->arch);
+	if (!ret)
+		reg_evt->is_registered = true;
+
+	return ret;
+}
+
+static int sse_event_register_local(struct sse_event *event)
+{
+	int ret;
+	struct sse_registered_event *reg_evt;
+
+	reg_evt = per_cpu_ptr(event->local, smp_processor_id());
+	ret = sse_sbi_register_event(event, reg_evt);
+	if (ret)
+		pr_debug("Failed to register event %x: err %d\n", event->evt_id,
+			 ret);
+
+	return ret;
+}
+
+static int sse_sbi_unregister_event(struct sse_event *event)
+{
+	struct sse_registered_event *reg_evt = sse_get_reg_evt(event);
+	int ret;
+
+	if (!reg_evt->is_registered)
+		return 0;
+
+	ret = sse_sbi_event_func(event, SBI_SSE_EVENT_UNREGISTER);
+	if (!ret) {
+		reg_evt->is_registered = false;
+		reg_evt->is_enabled = false;
+	}
+
+	return ret;
+}
+
+struct sse_per_cpu_evt {
+	struct sse_event *event;
+	unsigned long func;
+	atomic_t first_error;
+	atomic_t nonfallback_error;
+	cpumask_t changed;
+};
+
+static void sse_event_per_cpu_func(void *info)
+{
+	struct sse_per_cpu_evt *cpu_evt = info;
+	struct sse_registered_event *reg_evt;
+	bool changed;
+	int ret;
+
+	reg_evt = sse_get_reg_evt(cpu_evt->event);
+
+	if (cpu_evt->func == SBI_SSE_EVENT_REGISTER) {
+		changed = !reg_evt->is_registered;
+		ret = sse_event_register_local(cpu_evt->event);
+	} else if (cpu_evt->func == SBI_SSE_EVENT_UNREGISTER) {
+		changed = reg_evt->is_registered;
+		ret = sse_sbi_unregister_event(cpu_evt->event);
+	} else if (cpu_evt->func == SBI_SSE_EVENT_ENABLE) {
+		changed = !reg_evt->is_enabled;
+		ret = sse_event_enable_local(cpu_evt->event);
+	} else if (cpu_evt->func == SBI_SSE_EVENT_DISABLE) {
+		changed = reg_evt->is_enabled;
+		ret = sse_event_disable_local(cpu_evt->event);
+	} else {
+		changed = false;
+		ret = -EINVAL;
+	}
+
+	if (ret) {
+		atomic_cmpxchg(&cpu_evt->first_error, 0, ret);
+		if (ret != -EOPNOTSUPP)
+			atomic_cmpxchg(&cpu_evt->nonfallback_error, 0, ret);
+	} else if (changed) {
+		cpumask_set_cpu(smp_processor_id(), &cpu_evt->changed);
+	}
+}
+
+static bool sse_event_is_registered(struct sse_event *event)
+{
+	unsigned int cpu;
+
+	if (sse_event_is_global(event->evt_id))
+		return event->global->is_registered;
+
+	for_each_possible_cpu(cpu) {
+		if (per_cpu_ptr(event->local, cpu)->is_registered)
+			return true;
+	}
+
+	return false;
+}
+
+static bool sse_event_is_enabled(struct sse_event *event)
+{
+	unsigned int cpu;
+
+	if (sse_event_is_global(event->evt_id))
+		return event->global->is_enabled;
+
+	for_each_possible_cpu(cpu) {
+		if (per_cpu_ptr(event->local, cpu)->is_enabled)
+			return true;
+	}
+
+	return false;
+}
+
+static void sse_event_free(struct sse_event *event)
+{
+	unsigned int cpu;
+	struct sse_registered_event *reg_evt;
+
+	if (WARN_ON_ONCE(sse_event_is_registered(event)))
+		return;
+
+	if (sse_event_is_global(event->evt_id)) {
+		sse_event_free_registered(event->global);
+		kfree(event->global);
+	} else {
+		for_each_possible_cpu(cpu) {
+			reg_evt = per_cpu_ptr(event->local, cpu);
+			sse_event_free_registered(reg_evt);
+		}
+		free_percpu(event->local);
+	}
+
+	kfree(event);
+}
+
+static struct sse_event *sse_register_failed(struct sse_event *event, int ret)
+{
+	/*
+	 * Keep failed rollback state visible to CPU hotplug and shutdown. The
+	 * core-owned handler also makes the retained registration independent of
+	 * the client whose registration request failed.
+	 */
+	if (sse_event_is_registered(event)) {
+		event->cleanup_pending = true;
+		rcu_assign_pointer(event->handler, sse_cleanup_event_handler);
+		synchronize_rcu();
+		list_add(&event->list, &events);
+		pr_err("Event %x remains registered after rollback; cleanup retained\n",
+		       event->evt_id);
+		ret = -EUCLEAN;
+	} else {
+		sse_event_free(event);
+	}
+
+	return ERR_PTR(ret);
+}
+
+void sse_event_cleanup(struct sse_event *event)
+{
+	guard(mutex)(&sse_mutex);
+	guard(cpus_read_lock)();
+
+	if (event->cleanup_pending)
+		return;
+
+	/*
+	 * Firmware may still enter the old callback after disable or unregister
+	 * fails. Publish a core-owned callback, then wait before the client frees
+	 * its callback data.
+	 */
+	event->cleanup_pending = true;
+	rcu_assign_pointer(event->handler, sse_cleanup_event_handler);
+	synchronize_rcu();
+}
+EXPORT_SYMBOL_GPL(sse_event_cleanup);
+
+static void sse_release_cleanup_event(struct sse_event *event)
+{
+	if (!event->cleanup_pending || sse_event_is_registered(event))
+		return;
+
+	list_del(&event->list);
+	sse_event_free(event);
+}
+
+static int sse_event_setup_all_cpus(struct sse_event *event,
+				    unsigned long func,
+				    unsigned long rollback_func)
+{
+	struct sse_per_cpu_evt cpu_evt;
+	int rollback_ret;
+	int ret;
+
+	cpu_evt.event = event;
+	atomic_set(&cpu_evt.first_error, 0);
+	atomic_set(&cpu_evt.nonfallback_error, 0);
+	cpumask_clear(&cpu_evt.changed);
+	cpu_evt.func = func;
+	on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1);
+	/* IRQ fallback is safe only if every failing CPU reports unsupported. */
+	ret = atomic_read(&cpu_evt.nonfallback_error);
+	if (!ret)
+		ret = atomic_read(&cpu_evt.first_error);
+	/*
+	 * A previous attempt may already have changed some CPUs. Roll back only
+	 * instances changed by this invocation.
+	 */
+	if (ret) {
+		cpu_evt.func = rollback_func;
+		atomic_set(&cpu_evt.first_error, 0);
+		atomic_set(&cpu_evt.nonfallback_error, 0);
+		on_each_cpu_mask(&cpu_evt.changed, sse_event_per_cpu_func, &cpu_evt, 1);
+
+		rollback_ret = atomic_read(&cpu_evt.nonfallback_error);
+		if (!rollback_ret)
+			rollback_ret = atomic_read(&cpu_evt.first_error);
+
+		/* A rollback failure leaves the firmware state uncertain. */
+		return rollback_ret ?: ret;
+	}
+
+	return 0;
+}
+
+static int sse_event_teardown_all_cpus(struct sse_event *event,
+				       unsigned long func)
+{
+	struct sse_per_cpu_evt cpu_evt;
+
+	cpu_evt.event = event;
+	atomic_set(&cpu_evt.first_error, 0);
+	atomic_set(&cpu_evt.nonfallback_error, 0);
+	cpumask_clear(&cpu_evt.changed);
+	cpu_evt.func = func;
+	on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1);
+
+	return atomic_read(&cpu_evt.first_error);
+}
+
+int sse_event_enable(struct sse_event *event)
+{
+	int ret = 0;
+
+	scoped_guard(mutex, &sse_mutex) {
+		if (READ_ONCE(sse_shutting_down))
+			return -ESHUTDOWN;
+
+		scoped_guard(cpus_read_lock) {
+			if (sse_event_is_global(event->evt_id)) {
+				ret = sse_event_enable_local(event);
+			} else {
+				ret = sse_event_setup_all_cpus(event,
+							       SBI_SSE_EVENT_ENABLE,
+							       SBI_SSE_EVENT_DISABLE);
+			}
+			event->enable_requested = !ret;
+		}
+	}
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(sse_event_enable);
+
+static int sse_events_mask(void)
+{
+	struct sbiret ret;
+
+	ret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_HART_MASK, 0, 0, 0, 0, 0, 0);
+	if (ret.error == SBI_ERR_ALREADY_STOPPED)
+		return 0;
+
+	return sse_err_map_linux_errno(ret.error);
+}
+
+static int sse_events_unmask(void)
+{
+	struct sbiret ret;
+
+	ret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_HART_UNMASK, 0, 0, 0, 0, 0, 0);
+	if (ret.error == SBI_ERR_ALREADY_STARTED)
+		return 0;
+
+	return sse_err_map_linux_errno(ret.error);
+}
+
+static int sse_event_disable_nolock(struct sse_event *event)
+{
+	if (sse_event_is_global(event->evt_id))
+		return sse_event_disable_local(event);
+
+	return sse_event_teardown_all_cpus(event, SBI_SSE_EVENT_DISABLE);
+}
+
+int sse_event_disable(struct sse_event *event)
+{
+	int ret = 0;
+
+	scoped_guard(mutex, &sse_mutex) {
+		if (READ_ONCE(sse_shutting_down))
+			return -ESHUTDOWN;
+
+		scoped_guard(cpus_read_lock) {
+			if (!event->enable_requested && !sse_event_is_enabled(event))
+				return 0;
+
+			event->enable_requested = false;
+			ret = sse_event_disable_nolock(event);
+			if (!ret && sse_event_is_enabled(event))
+				ret = -EIO;
+		}
+	}
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(sse_event_disable);
+
+struct sse_event *sse_event_register(u32 evt, u32 priority,
+				     sse_event_handler_fn *handler, void *arg)
+{
+	struct sse_event *event;
+	int cpu;
+	int ret = 0;
+
+	if (sse_fw_state_retained)
+		return ERR_PTR(-EUCLEAN);
+	if (!sse_available)
+		return ERR_PTR(-EOPNOTSUPP);
+
+	guard(mutex)(&sse_mutex);
+	if (READ_ONCE(sse_shutting_down))
+		return ERR_PTR(-ESHUTDOWN);
+
+	guard(cpus_read_lock)();
+
+	if (sse_event_get(evt))
+		return ERR_PTR(-EEXIST);
+
+	event = sse_event_alloc(evt, priority, handler, arg);
+	if (IS_ERR(event))
+		return event;
+
+	if (sse_event_is_global(evt)) {
+		unsigned long preferred_hart;
+
+		ret = sse_event_attr_get_no_lock(event->global,
+						 SBI_SSE_ATTR_PREFERRED_HART,
+						 &preferred_hart);
+		if (ret)
+			return sse_register_failed(event, ret);
+
+		cpu = riscv_hartid_to_cpuid(preferred_hart);
+		if (cpu < 0 || !cpu_online(cpu)) {
+			cpu = cpumask_first(cpu_online_mask);
+			if (cpu >= nr_cpu_ids)
+				return sse_register_failed(event, -ENODEV);
+
+			ret = sse_event_set_target_cpu_nolock(event, cpu);
+			if (ret)
+				return sse_register_failed(event, ret);
+		} else {
+			sse_global_event_update_cpu(event, cpu);
+		}
+
+		ret = sse_sbi_register_event(event, event->global);
+		if (ret)
+			return sse_register_failed(event, ret);
+	} else {
+		ret = sse_event_setup_all_cpus(event, SBI_SSE_EVENT_REGISTER,
+					       SBI_SSE_EVENT_UNREGISTER);
+		if (ret)
+			return sse_register_failed(event, ret);
+	}
+
+	list_add(&event->list, &events);
+
+	return event;
+}
+EXPORT_SYMBOL_GPL(sse_event_register);
+
+static int sse_event_unregister_nolock(struct sse_event *event)
+{
+	if (sse_event_is_global(event->evt_id))
+		return sse_sbi_unregister_event(event);
+
+	return sse_event_teardown_all_cpus(event, SBI_SSE_EVENT_UNREGISTER);
+}
+
+int sse_event_unregister(struct sse_event *event)
+{
+	int ret = 0;
+
+	scoped_guard(mutex, &sse_mutex) {
+		if (READ_ONCE(sse_shutting_down))
+			return -ESHUTDOWN;
+
+		scoped_guard(cpus_read_lock) {
+			ret = sse_event_unregister_nolock(event);
+			if (ret)
+				return ret;
+			if (sse_event_is_registered(event))
+				return -EBUSY;
+
+			list_del(&event->list);
+
+			sse_event_free(event);
+		}
+	}
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(sse_event_unregister);
+
+static int sse_teardown_event(struct sse_event *event, unsigned int cpu);
+
+static int sse_cpu_online(unsigned int cpu)
+{
+	int ret, rollback_ret;
+	struct sse_event *event, *tmp;
+	struct sse_registered_event *reg_evt;
+
+	arch_sse_init_cpu();
+
+	list_for_each_entry_safe(event, tmp, &events, list) {
+		if (sse_event_is_global(event->evt_id))
+			continue;
+		if (event->cleanup_pending) {
+			scoped_guard(preempt)
+				ret = sse_teardown_event(event, cpu);
+			if (ret)
+				goto rollback;
+			sse_release_cleanup_event(event);
+			continue;
+		}
+
+		/*
+		 * The local SBI helpers act on the current hart and assert that
+		 * preemption is disabled. Unlike the client-facing paths, which
+		 * reach them through an IPI, the CPUHP thread runs preemptible.
+		 */
+		scoped_guard(preempt) {
+			ret = sse_event_register_local(event);
+			if (!ret)
+				ret = event->enable_requested ?
+					sse_event_enable_local(event) :
+					sse_event_disable_local(event);
+		}
+		if (ret)
+			goto rollback;
+	}
+
+	/* Only unmask after every cached per-CPU state is reconstructed. */
+	ret = sse_events_unmask();
+	if (!ret)
+		return 0;
+
+rollback:
+	/* A failed startup callback is not followed by this state's teardown. */
+	list_for_each_entry_safe(event, tmp, &events, list) {
+		if (sse_event_is_global(event->evt_id))
+			continue;
+
+		scoped_guard(preempt) {
+			reg_evt = sse_get_reg_evt(event);
+			rollback_ret = reg_evt->is_enabled ?
+				sse_event_disable_local(event) : 0;
+			if (!rollback_ret)
+				rollback_ret = reg_evt->is_registered ?
+					sse_sbi_unregister_event(event) : 0;
+		}
+		if (rollback_ret) {
+			pr_warn("Failed to roll back event %x on CPU %u: %d\n",
+				event->evt_id, cpu, rollback_ret);
+			atomic_set(&sse_teardown_failed, 1);
+			continue;
+		}
+		sse_release_cleanup_event(event);
+	}
+
+	return ret;
+}
+
+static int sse_teardown_event(struct sse_event *event, unsigned int cpu)
+{
+	struct sse_registered_event *reg_evt = sse_get_reg_evt(event);
+	int ret;
+
+	if (reg_evt->is_enabled) {
+		ret = sse_event_disable_local(event);
+		if (ret) {
+			pr_warn("Failed to disable event %x on CPU %u: %d\n",
+				event->evt_id, cpu, ret);
+			return ret;
+		}
+	}
+
+	ret = sse_sbi_unregister_event(event);
+	if (ret)
+		pr_warn("Failed to unregister event %x on CPU %u: %d\n",
+			event->evt_id, cpu, ret);
+
+	return ret;
+}
+
+static int sse_restore_local_events(unsigned int cpu)
+{
+	struct sse_event *event;
+	int first_error = 0;
+	int ret;
+
+	list_for_each_entry(event, &events, list) {
+		if (sse_event_is_global(event->evt_id) || event->cleanup_pending)
+			continue;
+
+		scoped_guard(preempt) {
+			ret = sse_event_register_local(event);
+			if (!ret)
+				ret = event->enable_requested ?
+					sse_event_enable_local(event) :
+					sse_event_disable_local(event);
+		}
+		if (ret) {
+			pr_warn("Failed to restore event %x on CPU %u: %d\n",
+				event->evt_id, cpu, ret);
+			if (!first_error)
+				first_error = ret;
+		}
+	}
+
+	return first_error;
+}
+
+static int sse_cpu_teardown(unsigned int cpu)
+{
+	/* Only a regular CPU-offline callback may abort the CPUHP operation. */
+	bool regular_offline = READ_ONCE(sse_available) &&
+			       !READ_ONCE(sse_shutting_down);
+	unsigned int next_cpu;
+	struct sse_event *event, *tmp;
+	int first_error = 0;
+	int ret;
+
+	/* Do not dismantle CPU state while firmware can still deliver SSE. */
+	ret = sse_events_mask();
+	if (ret) {
+		pr_warn("Failed to mask SSE on CPU %u during teardown: %d\n",
+			cpu, ret);
+		if (READ_ONCE(sse_shutting_down))
+			atomic_set(&sse_teardown_failed, 1);
+		/* CPUHP installation rollback and state removal cannot fail. */
+		return regular_offline ? ret : 0;
+	}
+
+	list_for_each_entry_safe(event, tmp, &events, list) {
+		if (sse_event_is_global(event->evt_id))
+			continue;
+
+		scoped_guard(preempt)
+			ret = sse_teardown_event(event, cpu);
+		if (ret && !first_error)
+			first_error = ret;
+		sse_release_cleanup_event(event);
+		if (ret && regular_offline)
+			goto restore_cpu;
+	}
+
+	list_for_each_entry_safe(event, tmp, &events, list) {
+		if (!sse_event_is_global(event->evt_id) || event->cpu != cpu)
+			continue;
+
+		/*
+		 * cpuhp_remove_state() invokes teardown while every CPU remains
+		 * online. Do not migrate to a CPU whose callback may have run.
+		 */
+		if (READ_ONCE(sse_shutting_down) || event->cleanup_pending) {
+			ret = sse_teardown_event(event, cpu);
+		} else {
+			next_cpu = cpumask_any_but(cpu_online_mask, cpu);
+			if (next_cpu >= nr_cpu_ids) {
+				ret = sse_teardown_event(event, cpu);
+			} else {
+				ret = sse_event_set_target_cpu_nolock(event, next_cpu);
+				if (ret)
+					pr_warn("Failed to migrate global event %x from CPU %u: %d\n",
+						event->evt_id, cpu, ret);
+			}
+		}
+
+		if (ret && !first_error)
+			first_error = ret;
+		sse_release_cleanup_event(event);
+		if (ret && regular_offline)
+			goto restore_cpu;
+	}
+
+	if (first_error) {
+		/* An offline CPU is not revisited when this CPUHP state is removed. */
+		atomic_set(&sse_teardown_failed, 1);
+	}
+
+	return 0;
+
+restore_cpu:
+	/*
+	 * CPUHP leaves this CPU online when teardown returns an error. Restore
+	 * every client-owned local event before making SSE delivery visible again.
+	 */
+	ret = sse_restore_local_events(cpu);
+	if (!ret)
+		ret = sse_events_unmask();
+	if (ret) {
+		atomic_set(&sse_teardown_failed, 1);
+		pr_warn("Failed to restore SSE after aborting CPU %u offline: %d\n",
+			cpu, ret);
+	} else {
+		pr_warn("Aborted CPU %u offline after SSE teardown failed: %d\n",
+			cpu, first_error);
+	}
+
+	return first_error;
+}
+
+static int sse_pm_notifier(struct notifier_block *nb, unsigned long action,
+			   void *data)
+{
+	int ret;
+
+	WARN_ON_ONCE(preemptible());
+
+	switch (action) {
+	case CPU_PM_ENTER:
+		ret = sse_events_mask();
+		break;
+	case CPU_PM_EXIT:
+	case CPU_PM_ENTER_FAILED:
+		if (READ_ONCE(sse_shutting_down))
+			return NOTIFY_OK;
+		ret = sse_events_unmask();
+		break;
+	default:
+		return NOTIFY_DONE;
+	}
+
+	if (ret)
+		return notifier_from_errno(ret);
+
+	return NOTIFY_OK;
+}
+
+static struct notifier_block sse_pm_nb = {
+	.notifier_call = sse_pm_notifier,
+};
+
+static int sse_panic_notifier(struct notifier_block *nb, unsigned long action,
+			      void *data)
+{
+	riscv_sse_mask_current_hart();
+
+	return NOTIFY_OK;
+}
+
+static struct notifier_block sse_panic_nb = {
+	.notifier_call = sse_panic_notifier,
+	.priority = INT_MAX,
+};
+
+/*
+ * Mask all CPUs and unregister all events on reboot or kexec.
+ */
+static int sse_reboot_notifier(struct notifier_block *nb, unsigned long action,
+			       void *data)
+{
+	int ret;
+
+	scoped_guard(mutex, &sse_mutex) {
+		if (!sse_shutting_down) {
+			WRITE_ONCE(sse_shutting_down, true);
+			ret = cpu_pm_unregister_notifier(&sse_pm_nb);
+			if (ret) {
+				pr_warn("Failed to unregister CPU PM notifier: %d\n",
+					ret);
+				atomic_set(&sse_teardown_failed, 1);
+			}
+			/* Drain CPU PM callbacks and client enables before teardown. */
+			synchronize_rcu();
+			cpuhp_remove_state(CPUHP_AP_RISCV_SSE_ONLINE);
+		}
+	}
+
+	/* Normal kexec preserves firmware state but discards old kernel memory. */
+	if (kexec_in_progress && atomic_read(&sse_teardown_failed))
+		panic("SSE teardown failed; refusing unsafe kexec");
+
+	return NOTIFY_OK;
+}
+
+static struct notifier_block sse_reboot_nb = {
+	.notifier_call = sse_reboot_notifier,
+};
+
+static int __init sse_init(void)
+{
+	int ret;
+
+	/*
+	 * A kdump kernel cannot identify registrations left by the crashed
+	 * kernel. Keep them masked by not initializing SSE again.
+	 */
+	if (is_kdump_kernel() && riscv_sse_available()) {
+		sse_fw_state_retained = true;
+		pr_info("SSE remains disabled in the crash kernel\n");
+		return -EOPNOTSUPP;
+	}
+
+	if (sbi_probe_extension(SBI_EXT_SSE) <= 0) {
+		pr_info("Missing SBI SSE extension\n");
+		return -EOPNOTSUPP;
+	}
+	pr_info("SBI SSE extension detected\n");
+
+	ret = cpu_pm_register_notifier(&sse_pm_nb);
+	if (ret) {
+		pr_warn("Failed to register CPU PM notifier...\n");
+		return ret;
+	}
+
+	ret = register_reboot_notifier(&sse_reboot_nb);
+	if (ret) {
+		pr_warn("Failed to register reboot notifier...\n");
+		goto remove_cpupm;
+	}
+
+	ret = atomic_notifier_chain_register(&panic_notifier_list, &sse_panic_nb);
+	if (ret) {
+		pr_warn("Failed to register panic notifier...\n");
+		goto remove_reboot;
+	}
+
+	/* Tear down perf events before dismantling their SSE delivery path. */
+	ret = cpuhp_setup_state(CPUHP_AP_RISCV_SSE_ONLINE, "riscv/sse:online",
+				sse_cpu_online, sse_cpu_teardown);
+	if (ret < 0)
+		goto remove_panic;
+
+	sse_available = true;
+
+	return 0;
+
+remove_panic:
+	atomic_notifier_chain_unregister(&panic_notifier_list, &sse_panic_nb);
+
+remove_reboot:
+	unregister_reboot_notifier(&sse_reboot_nb);
+
+remove_cpupm:
+	cpu_pm_unregister_notifier(&sse_pm_nb);
+
+	return ret;
+}
+arch_initcall(sse_init);
diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
index 1b6dba159bb2..c88025724a36 100644
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -200,6 +200,7 @@ enum cpuhp_state {
 	CPUHP_AP_ARM_MVEBU_SYNC_CLOCKS,
 	CPUHP_AP_ARM_CORESIGHT_ONLINE,
 	CPUHP_AP_X86_INTEL_EPB_ONLINE,
+	CPUHP_AP_RISCV_SSE_ONLINE,
 	CPUHP_AP_PERF_ONLINE,
 	CPUHP_AP_PERF_X86_ONLINE,
 	CPUHP_AP_PERF_X86_UNCORE_ONLINE,
diff --git a/include/linux/riscv_sbi_sse.h b/include/linux/riscv_sbi_sse.h
new file mode 100644
index 000000000000..774a782e556d
--- /dev/null
+++ b/include/linux/riscv_sbi_sse.h
@@ -0,0 +1,89 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (C) 2025 Rivos Inc.
+ */
+
+#ifndef __LINUX_RISCV_SBI_SSE_H
+#define __LINUX_RISCV_SBI_SSE_H
+
+#include <linux/err.h>
+#include <linux/errno.h>
+#include <linux/types.h>
+#include <linux/linkage.h>
+
+struct sse_event;
+struct pt_regs;
+
+typedef int (sse_event_handler_fn)(u32 event_num, void *arg,
+				   struct pt_regs *regs);
+
+#ifdef CONFIG_RISCV_SBI_SSE
+
+/*
+ * The callback and its argument must remain valid until unregister succeeds.
+ * The callback runs in NMI context and must not sleep.
+ * regs is NULL if firmware cannot provide a complete interrupted context.
+ */
+struct sse_event *sse_event_register(u32 event_num, u32 priority,
+				     sse_event_handler_fn *handler, void *arg);
+
+int sse_event_unregister(struct sse_event *evt);
+
+/*
+ * Transfer a retained event to the SSE core for deferred cleanup. The caller
+ * must not access the event or its callback data after this function returns.
+ */
+void sse_event_cleanup(struct sse_event *evt);
+
+int sse_event_set_target_cpu(struct sse_event *sse_evt, unsigned int cpu);
+
+int sse_event_enable(struct sse_event *sse_evt);
+
+int sse_event_disable(struct sse_event *sse_evt);
+
+/* Local events require the caller to remain on the current CPU. */
+int sse_event_enable_local(struct sse_event *sse_evt);
+int sse_event_disable_local(struct sse_event *sse_evt);
+
+#else
+static inline struct sse_event *sse_event_register(u32 event_num, u32 priority,
+						   sse_event_handler_fn *handler,
+						   void *arg)
+{
+	return ERR_PTR(-EOPNOTSUPP);
+}
+
+static inline int sse_event_unregister(struct sse_event *evt)
+{
+	return -EOPNOTSUPP;
+}
+
+static inline void sse_event_cleanup(struct sse_event *evt) { }
+
+static inline int sse_event_set_target_cpu(struct sse_event *sse_evt,
+					   unsigned int cpu)
+{
+	return -EOPNOTSUPP;
+}
+
+static inline int sse_event_enable(struct sse_event *sse_evt)
+{
+	return -EOPNOTSUPP;
+}
+
+static inline int sse_event_disable(struct sse_event *sse_evt)
+{
+	return -EOPNOTSUPP;
+}
+
+static inline int sse_event_enable_local(struct sse_event *sse_evt)
+{
+	return -EOPNOTSUPP;
+}
+
+static inline int sse_event_disable_local(struct sse_event *sse_evt)
+{
+	return -EOPNOTSUPP;
+}
+#endif
+#endif /* __LINUX_RISCV_SBI_SSE_H */
-- 
2.50.1 (Apple Git-155)


^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH v13 05/11] riscv: mm: avoid enabling interrupts for nofault page faults
  2026-11-19 12:08 [PATCH v13 00/11] riscv: add SBI Supervisor Software Events support Zhanpeng Zhang
                   ` (3 preceding siblings ...)
  2026-11-19 12:08 ` [PATCH v13 04/11] drivers: firmware: add riscv SSE support Zhanpeng Zhang
@ 2026-11-19 12:08 ` Zhanpeng Zhang
  2026-11-19 12:27   ` sashiko-bot
  2026-11-19 12:08 ` [PATCH v13 06/11] riscv: enable NMI-safe lockdep irqflags tracking Zhanpeng Zhang
                   ` (5 subsequent siblings)
  10 siblings, 1 reply; 22+ messages in thread
From: Zhanpeng Zhang @ 2026-11-19 12:08 UTC (permalink / raw)
  To: pjw, palmer, aou, alex, himanshu.chauhan, conor, anup.patel
  Cc: clement.leger, cuiyunhui, atish.patra, peterz, mingo, acme,
	namhyung, mark.rutland, alexander.shishkin, jolsa, irogers,
	adrian.hunter, james.clark, will, tglx, corbet, rdunlap, shuah,
	yuanzhu, yang.susheng, linux-riscv, linux-kernel,
	linux-perf-users, linux-doc, linux-kselftest,
	zhangzhanpeng.jasper

RISC-V enables interrupts in handle_page_fault() before checking whether
the fault occurred with fault handling disabled. A nofault access from an
atomic context can therefore run tracepoints and open an interrupt window
before reaching the exception-table fixup.

Handle an exception-table entry before entering the generic fault path when
fault handling is disabled. Also keep interrupts disabled until such a
fault has been resolved.

On 32-bit RISC-V the vmalloc mapping must still be synchronised into the
current page table before that fixup runs: otherwise a nofault access to a
not-yet-synced vmalloc address would take the exception-table fixup and
fail spuriously instead of being resolved. Order the vmalloc
synchronisation ahead of the nofault fixup. That synchronisation only
copies an already-established kernel mapping, takes no locks and waits for
no I/O, so it is safe in the nofault path; if the mapping does not exist it
falls through to the existing no_context() error handling. This early
return skips the page-fault tracepoints, as before.

This makes RISC-V consistent with the expectation that an in-atomic
nofault access does not enter the normal fault-handling path. It also
removes one source of re-entry when perf sampling is delivered through an
SBI Supervisor Software Event (SSE).

Signed-off-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
---
 arch/riscv/mm/fault.c | 32 ++++++++++++++++++++++++--------
 1 file changed, 24 insertions(+), 8 deletions(-)

diff --git a/arch/riscv/mm/fault.c b/arch/riscv/mm/fault.c
index 04ed6f8acae4..3934248f7a27 100644
--- a/arch/riscv/mm/fault.c
+++ b/arch/riscv/mm/fault.c
@@ -294,19 +294,23 @@ void handle_page_fault(struct pt_regs *regs)
 	if (kprobe_page_fault(regs, cause))
 		return;
 
-	if (user_mode(regs))
-		trace_page_fault_user(addr, regs, cause);
-	else
-		trace_page_fault_kernel(addr, regs, cause);
-
 	/*
 	 * Fault-in kernel-space virtual memory on-demand.
 	 * The 'reference' page table is init_mm.pgd.
 	 *
+	 * This only synchronises an already-established kernel vmalloc mapping
+	 * into the current page table; it neither allocates memory nor waits for
+	 * I/O, so it is safe in the nofault path. It must run before the
+	 * exception-table fixup below: otherwise a nofault access to a
+	 * not-yet-synced vmalloc address would take the fixup and fail
+	 * spuriously instead of being resolved. If the mapping does not exist,
+	 * vmalloc_fault() falls through to no_context() error handling.
+	 *
 	 * NOTE! We MUST NOT take any locks for this case. We may
 	 * be in an interrupt or a critical region, and should
 	 * only copy the information from the master page table,
-	 * nothing more.
+	 * nothing more. This path returns early and therefore skips the
+	 * trace_page_fault_*() tracepoints below.
 	 */
 	if ((!IS_ENABLED(CONFIG_MMU) || !IS_ENABLED(CONFIG_64BIT)) &&
 	    unlikely(addr >= VMALLOC_START && addr < VMALLOC_END)) {
@@ -314,8 +318,20 @@ void handle_page_fault(struct pt_regs *regs)
 		return;
 	}
 
-	/* Enable interrupts if they were enabled in the parent context. */
-	if (!regs_irqs_disabled(regs))
+	/*
+	 * Nofault accesses must be resolved through the exception table before
+	 * entering the generic fault path or enabling interrupts.
+	 */
+	if (unlikely(faulthandler_disabled()) && fixup_exception(regs))
+		return;
+
+	if (user_mode(regs))
+		trace_page_fault_user(addr, regs, cause);
+	else
+		trace_page_fault_kernel(addr, regs, cause);
+
+	/* Do not open an interrupt window before a nofault fixup completes. */
+	if (!regs_irqs_disabled(regs) && !faulthandler_disabled())
 		local_irq_enable();
 
 	/*
-- 
2.50.1 (Apple Git-155)


^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH v13 06/11] riscv: enable NMI-safe lockdep irqflags tracking
  2026-11-19 12:08 [PATCH v13 00/11] riscv: add SBI Supervisor Software Events support Zhanpeng Zhang
                   ` (4 preceding siblings ...)
  2026-11-19 12:08 ` [PATCH v13 05/11] riscv: mm: avoid enabling interrupts for nofault page faults Zhanpeng Zhang
@ 2026-11-19 12:08 ` Zhanpeng Zhang
  2026-11-19 12:21   ` sashiko-bot
  2026-11-19 12:08 ` [PATCH v13 07/11] perf: RISC-V: support callchains with SSE delivery Zhanpeng Zhang
                   ` (4 subsequent siblings)
  10 siblings, 1 reply; 22+ messages in thread
From: Zhanpeng Zhang @ 2026-11-19 12:08 UTC (permalink / raw)
  To: pjw, palmer, aou, alex, himanshu.chauhan, conor, anup.patel
  Cc: clement.leger, cuiyunhui, atish.patra, peterz, mingo, acme,
	namhyung, mark.rutland, alexander.shishkin, jolsa, irogers,
	adrian.hunter, james.clark, will, tglx, corbet, rdunlap, shuah,
	yuanzhu, yang.susheng, linux-riscv, linux-kernel,
	linux-perf-users, linux-doc, linux-kselftest,
	zhangzhanpeng.jasper

An SSE handler runs with interrupts masked, but can interrupt code whose
lockdep hardirq state is enabled. Bare nmi_enter()/nmi_exit() do not
update that state. Consequently, lockdep_assert_irqs_disabled() in the
perf overflow path can report a warning despite interrupts being masked.

Select TRACE_IRQFLAGS_NMI_SUPPORT so lockdep can update its hardirq state
inside an NMI. Pair this with irqentry_nmi_enter()/irqentry_nmi_exit() in
the SSE wrapper: these helpers save the previous tracked state, mark
interrupts disabled on entry and restore the tracked state on exit.
Selecting the configuration option alone does not perform those updates.
This follows the entry helpers used by the other RISC-V NMI-like traps.

Signed-off-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
---
 arch/riscv/Kconfig          | 1 +
 arch/riscv/kernel/sbi_sse.c | 7 ++++---
 2 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index d2db8f40c616..5501c02a07eb 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -225,6 +225,7 @@ config RISCV
 	select SYSCTL_EXCEPTION_TRACE
 	select THREAD_INFO_IN_TASK
 	select TRACE_IRQFLAGS_SUPPORT
+	select TRACE_IRQFLAGS_NMI_SUPPORT
 	select UACCESS_MEMCPY if !MMU
 	select VDSO_DATASTORE if MMU
 	select VDSO_GETRANDOM if MMU && 64BIT
diff --git a/arch/riscv/kernel/sbi_sse.c b/arch/riscv/kernel/sbi_sse.c
index 032fcc07abcf..59e47a39589d 100644
--- a/arch/riscv/kernel/sbi_sse.c
+++ b/arch/riscv/kernel/sbi_sse.c
@@ -2,7 +2,7 @@
 /*
  * Copyright (C) 2025 Rivos Inc.
  */
-#include <linux/nmi.h>
+#include <linux/entry-common.h>
 #include <linux/sched.h>
 #include <linux/scs.h>
 #include <linux/bitfield.h>
@@ -36,9 +36,10 @@ void noinstr do_sse(struct sse_event_arch_data *arch_evt,
 {
 	struct riscv_sse_interrupted_context context = { regs, hstatus };
 	struct riscv_sse_interrupted_context *previous;
+	irqentry_state_t state;
 	struct sbiret sret;
 
-	nmi_enter();
+	state = irqentry_nmi_enter(regs);
 	instrumentation_begin();
 
 	/* Retrieve missing GPRs from SBI */
@@ -84,7 +85,7 @@ void noinstr do_sse(struct sse_event_arch_data *arch_evt,
 
 out:
 	instrumentation_end();
-	nmi_exit();
+	irqentry_nmi_exit(regs, state);
 }
 
 static void *alloc_to_stack_pointer(void *alloc)
-- 
2.50.1 (Apple Git-155)


^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH v13 07/11] perf: RISC-V: support callchains with SSE delivery
  2026-11-19 12:08 [PATCH v13 00/11] riscv: add SBI Supervisor Software Events support Zhanpeng Zhang
                   ` (5 preceding siblings ...)
  2026-11-19 12:08 ` [PATCH v13 06/11] riscv: enable NMI-safe lockdep irqflags tracking Zhanpeng Zhang
@ 2026-11-19 12:08 ` Zhanpeng Zhang
  2026-11-19 12:45   ` sashiko-bot
  2026-11-19 12:08 ` [PATCH v13 08/11] perf: RISC-V: add support for SSE event Zhanpeng Zhang
                   ` (3 subsequent siblings)
  10 siblings, 1 reply; 22+ messages in thread
From: Zhanpeng Zhang @ 2026-11-19 12:08 UTC (permalink / raw)
  To: pjw, palmer, aou, alex, himanshu.chauhan, conor, anup.patel
  Cc: clement.leger, cuiyunhui, atish.patra, peterz, mingo, acme,
	namhyung, mark.rutland, alexander.shishkin, jolsa, irogers,
	adrian.hunter, james.clark, will, tglx, corbet, rdunlap, shuah,
	yuanzhu, yang.susheng, linux-riscv, linux-kernel,
	linux-perf-users, linux-doc, linux-kselftest,
	zhangzhanpeng.jasper

PMU overflow delivery through SSE enters Linux with a synthetic supervisor
context on a dedicated event stack. The perf unwinder must use the context
interrupted by the overflow rather than treating the SSE handler frame as
the sampled frame.

Use the interrupted pt_regs published by the RISC-V SSE entry path. Walk a
kernel callchain only when the interrupted PC, SP, and frame pointer are
consistent with the current task stack. For sensitive entry windows and
IRQ stacks whose bounds cannot be proven, retain the interrupted PC without
following an unsafe frame chain.

User callchains continue through the existing nofault RISC-V user unwinder.
If hstatus.SPV says the interrupted context was a guest, do not interpret
the guest stack through the host address space.

DWARF callchains additionally copy a raw user stack. Provide an SSE-aware
riscv_perf_out_copy_user() following x86's copy_from_user_nmi().
The copy runs under pagefault_disable(), so a non-resident source
page takes an exception-table handled fault that returns the number
of bytes not copied
instead of entering handle_mm_fault() or opening an interrupt window, and
perf truncates the user-stack dump accordingly. The SSE entry saves and
restores the interrupted task's stack metadata across that nested trap.

In SSE context also check access_ok(), require a user mm, and compare
the active page-table root with current->mm before copying. This rejects
a mismatched root, but does not exclude stale translations in the window
between a satp update and the following TLB flush. It is not a complete
equivalent of x86's nmi_uaccess_okay(). Outside SSE, retain the existing
inatomic copy with page faults disabled.

Susheng Yang reported this failure with perf callchain workloads.

Link: https://lore.kernel.org/r/as79MHtNYd-VbUug@hu-himchau-blr.qualcomm.com/
Reported-by: Susheng Yang <yang.susheng@zte.com.cn>
Signed-off-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
---
 arch/riscv/include/asm/perf_event.h |  10 ++
 arch/riscv/kernel/perf_callchain.c  | 140 ++++++++++++++++++++++++++++
 2 files changed, 150 insertions(+)

diff --git a/arch/riscv/include/asm/perf_event.h b/arch/riscv/include/asm/perf_event.h
index bcc928fd3785..ddc404794751 100644
--- a/arch/riscv/include/asm/perf_event.h
+++ b/arch/riscv/include/asm/perf_event.h
@@ -18,6 +18,16 @@
 	(regs)->sp = current_stack_pointer; \
 	(regs)->status = SR_PP; \
 }
+
+#ifdef CONFIG_RISCV_SBI_SSE
+/*
+ * Raw user-stack sampling can run in the NMI-like SSE context. Route it
+ * through an implementation that does not fault on a non-resident page.
+ */
+unsigned long riscv_perf_out_copy_user(void *dst, const void *src,
+				       unsigned long n);
+#define arch_perf_out_copy_user riscv_perf_out_copy_user
+#endif
 #endif
 
 #endif /* _ASM_RISCV_PERF_EVENT_H */
diff --git a/arch/riscv/kernel/perf_callchain.c b/arch/riscv/kernel/perf_callchain.c
index b465bc9eb870..216f5bd14c00 100644
--- a/arch/riscv/kernel/perf_callchain.c
+++ b/arch/riscv/kernel/perf_callchain.c
@@ -1,9 +1,15 @@
 // SPDX-License-Identifier: GPL-2.0
 /* Copyright (C) 2019 Hangzhou C-SKY Microsystems co.,ltd. */
 
+#include <linux/kallsyms.h>
+#include <linux/mm.h>
 #include <linux/perf_event.h>
+#include <linux/sched/task_stack.h>
+#include <linux/thread_info.h>
 #include <linux/uaccess.h>
 
+#include <asm/csr.h>
+#include <asm/sse.h>
 #include <asm/stacktrace.h>
 
 static bool fill_callchain(void *entry, unsigned long pc)
@@ -11,6 +17,126 @@ static bool fill_callchain(void *entry, unsigned long pc)
 	return perf_callchain_store(entry, pc) == 0;
 }
 
+#ifdef CONFIG_RISCV_SBI_SSE
+static bool sse_addr_on_task_stack(unsigned long addr, unsigned long size)
+{
+	unsigned long end = addr + size;
+	unsigned long stack;
+
+	if (end < addr)
+		return false;
+
+	stack = (unsigned long)task_stack_page(current);
+	if (addr >= stack && end <= stack + THREAD_SIZE)
+		return true;
+
+	return false;
+}
+
+static bool sse_kernel_regs_safe(struct pt_regs *regs)
+{
+	unsigned long fp = frame_pointer(regs);
+	unsigned long pc = instruction_pointer(regs);
+	unsigned long sp = user_stack_pointer(regs);
+
+	if (!__kernel_text_address(pc))
+		return false;
+	if (!sse_addr_on_task_stack(sp, sizeof(unsigned long)))
+		return false;
+	if (fp < sizeof(struct stackframe))
+		return false;
+
+	return sse_addr_on_task_stack(fp - sizeof(struct stackframe),
+				      sizeof(struct stackframe));
+}
+
+static bool sse_callchain_is_guest(const struct riscv_sse_interrupted_context *context)
+{
+	return context && (context->hstatus & HSTATUS_SPV);
+}
+
+static bool sse_callchain_kernel(struct perf_callchain_entry_ctx *entry,
+				 struct pt_regs *regs)
+{
+	const struct riscv_sse_interrupted_context *context;
+	unsigned long pc;
+
+	context = riscv_sse_get_interrupted_context();
+	if (!context || context->regs != regs)
+		return false;
+
+	/* A guest stack cannot be walked using the host kernel address space. */
+	if (sse_callchain_is_guest(context))
+		return true;
+
+	if (user_mode(regs))
+		return true;
+
+	if (sse_kernel_regs_safe(regs)) {
+		walk_stackframe(NULL, regs, fill_callchain, entry);
+		return true;
+	}
+
+	/*
+	 * Keep the sample useful for sensitive entry paths and IRQ stacks. The
+	 * generic walker does not take explicit IRQ stack bounds, and its
+	 * THREAD_SIZE alignment assumption fails for non-vmapped IRQ stacks.
+	 * Conservatively avoid walking IRQ stacks in every configuration.
+	 */
+	pc = instruction_pointer(regs);
+	if (__kernel_text_address(pc))
+		perf_callchain_store(entry, pc);
+
+	return true;
+}
+
+unsigned long riscv_perf_out_copy_user(void *dst, const void *src,
+				       unsigned long n)
+{
+	unsigned long ret;
+
+	/*
+	 * Raw user-stack sampling can run in the NMI-like SSE context. Mirror
+	 * x86's copy_from_user_nmi(): the copy may take a page fault, but it
+	 * must not fault a non-resident page in. pagefault_disable() keeps the
+	 * RISC-V fault handler on the nofault exception-table fixup (no
+	 * handle_mm_fault(), no interrupt window), and the SSE entry saves and
+	 * restores the interrupted task's stack metadata across that nested
+	 * trap. A fault therefore returns the number of bytes not copied, and
+	 * perf truncates the user-stack dump accordingly.
+	 *
+	 * In SSE context also approximate x86's nmi_uaccess_okay(): only sample
+	 * when the active page table root still matches current->mm, so we do
+	 * not walk an unrelated address space after interrupting a KVM guest, a
+	 * kernel thread, or a task whose mm is being swapped. This root check
+	 * does not close the window inside switch_mm() between writing satp and
+	 * flushing the local TLB (RISC-V has no LOADED_MM_SWITCHING marker like
+	 * x86); that is a window between instructions, not within one. In that
+	 * window a stale TLB entry may still resolve to the outgoing mapping, and
+	 * with ASID reuse the residue can even belong to an unrelated address
+	 * space. The nofault fixup only bounds faulting accesses; it cannot
+	 * detect a successful hit on a stale translation. This is therefore a
+	 * bounded sample-quality gap (a mis-attributed or truncated user-stack
+	 * dump), not a memory-safety hazard: the copy target is the perf ring
+	 * buffer and the length is bounded, so a stale read cannot corrupt an
+	 * unrelated kernel object.
+	 */
+	if (riscv_sse_get_interrupted_context()) {
+		if (!access_ok(src, n))
+			return n;
+		if (!current->mm ||
+		    (csr_read(CSR_SATP) & SATP_PPN) != virt_to_pfn(current->mm->pgd))
+			return n;
+	}
+
+	pagefault_disable();
+	ret = __copy_from_user_inatomic(dst, src, n);
+	pagefault_enable();
+
+	return ret;
+}
+#endif
+
 /*
  * This will be called when the target is in user mode
  * This function will only be called when we use
@@ -28,6 +154,15 @@ static bool fill_callchain(void *entry, unsigned long pc)
 void perf_callchain_user(struct perf_callchain_entry_ctx *entry,
 			 struct pt_regs *regs)
 {
+#ifdef CONFIG_RISCV_SBI_SSE
+	const struct riscv_sse_interrupted_context *context;
+
+	context = riscv_sse_get_interrupted_context();
+	/* A guest stack cannot be walked using the host address space. */
+	if (sse_callchain_is_guest(context))
+		return;
+#endif
+
 	if (perf_guest_state()) {
 		/* TODO: We don't support guest os callchain now */
 		return;
@@ -39,6 +174,11 @@ void perf_callchain_user(struct perf_callchain_entry_ctx *entry,
 void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry,
 			   struct pt_regs *regs)
 {
+#ifdef CONFIG_RISCV_SBI_SSE
+	if (sse_callchain_kernel(entry, regs))
+		return;
+#endif
+
 	if (perf_guest_state()) {
 		/* TODO: We don't support guest os callchain now */
 		return;
-- 
2.50.1 (Apple Git-155)


^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH v13 08/11] perf: RISC-V: add support for SSE event
  2026-11-19 12:08 [PATCH v13 00/11] riscv: add SBI Supervisor Software Events support Zhanpeng Zhang
                   ` (6 preceding siblings ...)
  2026-11-19 12:08 ` [PATCH v13 07/11] perf: RISC-V: support callchains with SSE delivery Zhanpeng Zhang
@ 2026-11-19 12:08 ` Zhanpeng Zhang
  2026-11-19 12:34   ` sashiko-bot
  2026-11-19 12:08 ` [PATCH v13 09/11] MAINTAINERS: add RISC-V SSE maintainers and file coverage Zhanpeng Zhang
                   ` (2 subsequent siblings)
  10 siblings, 1 reply; 22+ messages in thread
From: Zhanpeng Zhang @ 2026-11-19 12:08 UTC (permalink / raw)
  To: pjw, palmer, aou, alex, himanshu.chauhan, conor, anup.patel
  Cc: clement.leger, cuiyunhui, atish.patra, peterz, mingo, acme,
	namhyung, mark.rutland, alexander.shishkin, jolsa, irogers,
	adrian.hunter, james.clark, will, tglx, corbet, rdunlap, shuah,
	yuanzhu, yang.susheng, linux-riscv, linux-kernel,
	linux-perf-users, linux-doc, linux-kselftest,
	zhangzhanpeng.jasper

From: Clément Léger <cleger@rivosinc.com>

Register a handler for the local PMU overflow SSE event so that RISC-V
perf can receive overflows even when normal S-mode interrupts are masked.
Reuse the existing overflow handler and pass it the interrupted pt_regs
rebuilt by the architecture SSE entry path.

Select the delivery mechanism once during PMU probe. Prefer SSE when its
event can be registered and enabled. If the extension or PMU event is
explicitly unsupported, use the ordinary PMU interrupt. Do not enable the
interrupt after any other SSE setup failure or when a crash kernel may
have inherited firmware state. Install the PMU enable and disable
callbacks only after SSE delivery becomes active.

Keep the local PMU SSE event disabled across CPU power management. On
entry, the generic SSE notifier masks the hart before the lower-priority
PMU notifier disables the event and stops the counters. On exit, the SSE
notifier first unmasks the hart while the event remains disabled. The PMU
notifier then restores counters and event userpage state before enabling
the event. An unmask failure stops the notifier chain and leaves the
counters stopped. Ordinary PMU interrupts retain their existing notifier
ordering.

An SSE overflow can arrive as soon as the event is enabled during probe.
Publish the counter mask before SSE setup, so an early handler can stop
the counter source even before perf starts admitting normal samples.
After a real overflow, restart only counters whose perf state is still
running. Honor a non-zero return from perf_event_overflow() and leave
throttled events stopped.

Guest attribution is not part of this version. Detect an interrupted guest
from hstatus.SPV and skip its sample while still updating the period and
counter state, rather than exposing guest state as a host sample.

The perf PMU callbacks cannot return errors. If an SSE transition or
interrupted-context read fails, latch the failure per CPU and stop its
mapped events through the normal perf state transitions. Do not reset the
firmware counter mapping behind perf, restart a failed event, or attempt a
runtime switch to IRQ delivery.

Track ordinary IRQ ownership only when request_percpu_irq() succeeds.
Successful SSE setup does not acquire a per-CPU IRQ to release during
probe error unwinding.

When an SSE arrives without a usable perf event or before callback
admission, stop the counter sources individually. A batch STOP can
return SBI_ERR_ALREADY_STOPPED before stopping other counters.
Treat that result as harmless for a single counter, report other
errors, and continue attempting to stop the remaining sources.

During cleanup, close callback admission and synchronously drain each CPU
before disabling and unregistering the SSE event. If firmware refuses the
cleanup, stop the counter source and transfer the event to the SSE core so
later CPU hotplug or shutdown processing can retry without using freed PMU
callback state.

Signed-off-by: Clément Léger <cleger@rivosinc.com>
Co-developed-by: Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>
Signed-off-by: Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>
Co-developed-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
Signed-off-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
Acked-by: Conor Dooley <conor.dooley@microchip.com>
Link: https://lore.kernel.org/r/20261002-reach-cloning-f1594729dbe3@wendy/
---
 Documentation/arch/riscv/index.rst     |   1 +
 Documentation/arch/riscv/pmu-sse.rst   |  62 +++
 drivers/firmware/riscv/riscv_sbi_sse.c |   9 +
 drivers/perf/Kconfig                   |  11 +
 drivers/perf/riscv_pmu.c               |  14 +-
 drivers/perf/riscv_pmu_sbi.c           | 561 +++++++++++++++++++++----
 include/linux/perf/riscv_pmu.h         |  20 +-
 include/linux/riscv_sbi_sse.h          |   6 +
 8 files changed, 589 insertions(+), 95 deletions(-)
 create mode 100644 Documentation/arch/riscv/pmu-sse.rst

diff --git a/Documentation/arch/riscv/index.rst b/Documentation/arch/riscv/index.rst
index ac535c52d509..5cb909c83a88 100644
--- a/Documentation/arch/riscv/index.rst
+++ b/Documentation/arch/riscv/index.rst
@@ -11,6 +11,7 @@ RISC-V architecture
     vm-layout
     hwprobe
     patch-acceptance
+    pmu-sse
     uabi
     vector
     cmodx
diff --git a/Documentation/arch/riscv/pmu-sse.rst b/Documentation/arch/riscv/pmu-sse.rst
new file mode 100644
index 000000000000..98b55cc4bce6
--- /dev/null
+++ b/Documentation/arch/riscv/pmu-sse.rst
@@ -0,0 +1,62 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+========================================
+RISC-V PMU overflow delivery through SSE
+========================================
+
+When ``CONFIG_RISCV_PMU_SBI_SSE`` is enabled and firmware provides the local
+PMU overflow event, the RISC-V SBI PMU driver uses Supervisor Software Events
+(SSE) to deliver counter overflows.
+
+Delivery selection
+==================
+
+The delivery mechanism is selected once while the PMU device is probed.  The
+driver first tries to register and enable the local PMU overflow SSE event.  It
+tries the ordinary PMU interrupt only when the SSE extension or the local PMU
+event is explicitly unsupported.  It does not change the delivery mechanism
+after the PMU has been registered.
+
+Other SSE setup errors do not prove that firmware released the overflow route,
+so the driver does not enable the ordinary PMU interrupt.  The PMU remains
+available for counting but not sampling.  A retained event handle does not
+mean that SSE delivery is active: the PMU enable and disable callbacks are
+installed only after setup succeeds.  Retained state prevents IRQ fallback
+and remains subject to SSE cleanup.
+
+Interrupted context
+===================
+
+SSE enters Linux through a supervisor handler context constructed by firmware.
+The entry contract preserves the interrupted GPRs except ``a6`` and ``a7``,
+which Linux reads from the interrupted-register event attributes.  Linux then
+reconstructs the interrupted ``pt_regs`` and publishes it while dispatching the
+event.
+
+Perf uses that context for register samples and callchains.  Kernel stack
+walking verifies that the interrupted frame belongs to the current task stack
+before dereferencing it.  For sensitive entry paths and IRQ stacks, Linux
+records the interrupted PC without walking a stack whose bounds cannot be
+proved.  User callchains use the existing no-fault user unwinder.  A DWARF
+raw user-stack copy uses ``__copy_from_user_inatomic()`` with page-fault
+handling disabled.  A non-resident source page may still cause a synchronous
+exception, which the exception-table fixup handles without entering normal
+page-fault handling or enabling interrupts.  Perf truncates the dump by the
+number of bytes not copied.  SSE entry preserves the task's stack metadata
+and early trap-entry scratch slots across nested exceptions.
+
+The SSE copy checks the user address range and requires the active page-table
+root to match ``current->mm``.  This check does not detect stale TLB entries
+in the address-space switch window; samples from that window can still be
+misattributed.
+
+CPU power management
+====================
+
+The PMU and SSE CPU power-management callbacks are ordered according to the
+selected delivery mechanism.  With SSE delivery, SSE events are masked before
+the lower-priority PMU callback disables the local PMU event and stops the
+counters on entry.  On exit, the hart is unmasked while the local PMU event is
+still disabled.  The PMU callback then restores the counters and enables the
+event, so an unmask failure leaves the counters stopped.  The ordinary
+interrupt path retains the existing PMU ordering.
diff --git a/drivers/firmware/riscv/riscv_sbi_sse.c b/drivers/firmware/riscv/riscv_sbi_sse.c
index 0cb738bab6c3..69ce98380331 100644
--- a/drivers/firmware/riscv/riscv_sbi_sse.c
+++ b/drivers/firmware/riscv/riscv_sbi_sse.c
@@ -161,6 +161,15 @@ static int sse_sbi_event_func(struct sse_event *event, unsigned long func)
 	return 0;
 }
 
+bool sse_event_is_enabled_local(struct sse_event *event)
+{
+	if (!sse_event_is_global(event->evt_id))
+		lockdep_assert_preemption_disabled();
+
+	return sse_get_reg_evt(event)->is_enabled;
+}
+EXPORT_SYMBOL_GPL(sse_event_is_enabled_local);
+
 int sse_event_disable_local(struct sse_event *event)
 {
 	if (!sse_event_is_global(event->evt_id))
diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig
index 245e7bb763b9..10b047856121 100644
--- a/drivers/perf/Kconfig
+++ b/drivers/perf/Kconfig
@@ -105,6 +105,17 @@ config RISCV_PMU_SBI
 	  full perf feature support i.e. counter overflow, privilege mode
 	  filtering, counter configuration.
 
+config RISCV_PMU_SBI_SSE
+	depends on RISCV_PMU_SBI && RISCV_SBI_SSE
+	bool "RISC-V PMU SSE events"
+	default n
+	help
+	  Say y if you want to use SSE events to deliver PMU interrupts. This
+	  provides a way to profile the kernel at any level by using NMI-like
+	  SSE events. SSE events being really intrusive, this option allows
+	  to select it only if needed. See Documentation/arch/riscv/pmu-sse.rst
+	  for interrupted-context limitations.
+
 config STARFIVE_STARLINK_PMU
 	depends on ARCH_STARFIVE || COMPILE_TEST
 	depends on 64BIT
diff --git a/drivers/perf/riscv_pmu.c b/drivers/perf/riscv_pmu.c
index 8e3cd0f35336..1f65c640b693 100644
--- a/drivers/perf/riscv_pmu.c
+++ b/drivers/perf/riscv_pmu.c
@@ -13,6 +13,7 @@
 #include <linux/irqdesc.h>
 #include <linux/perf/riscv_pmu.h>
 #include <linux/printk.h>
+#include <linux/riscv_sbi_sse.h>
 #include <linux/smp.h>
 #include <linux/sched_clock.h>
 
@@ -247,6 +248,11 @@ void riscv_pmu_start(struct perf_event *event, int flags)
 	if (flags & PERF_EF_RELOAD)
 		WARN_ON_ONCE(!(event->hw.state & PERF_HES_UPTODATE));
 
+#ifdef CONFIG_RISCV_PMU_SBI_SSE
+	if (unlikely(this_cpu_ptr(rvpmu->hw_events)->sse_failed))
+		return;
+#endif
+
 	hwc->state = 0;
 	riscv_pmu_event_set_period(event);
 	init_val = local64_read(&hwc->prev_count) & max_period;
@@ -306,6 +312,7 @@ static int riscv_pmu_event_init(struct perf_event *event)
 	struct hw_perf_event *hwc = &event->hw;
 	struct riscv_pmu *rvpmu = to_riscv_pmu(event->pmu);
 	int mapped_event;
+	int ret;
 	u64 event_config = 0;
 	uint64_t cmask;
 
@@ -331,8 +338,11 @@ static int riscv_pmu_event_init(struct perf_event *event)
 	hwc->idx = -1;
 	hwc->event_base = mapped_event;
 
-	if (rvpmu->event_init)
-		rvpmu->event_init(event);
+	if (rvpmu->event_init) {
+		ret = rvpmu->event_init(event);
+		if (ret)
+			return ret;
+	}
 
 	if (!is_sampling_event(event)) {
 		/*
diff --git a/drivers/perf/riscv_pmu_sbi.c b/drivers/perf/riscv_pmu_sbi.c
index 21a2078eab98..a984ac57a7b6 100644
--- a/drivers/perf/riscv_pmu_sbi.c
+++ b/drivers/perf/riscv_pmu_sbi.c
@@ -16,13 +16,16 @@
 #include <linux/irqdomain.h>
 #include <linux/of_irq.h>
 #include <linux/of.h>
+#include <linux/riscv_sbi_sse.h>
 #include <linux/cpu_pm.h>
+#include <linux/crash_dump.h>
 #include <linux/sched/clock.h>
 #include <linux/soc/andes/irq.h>
 #include <linux/workqueue.h>
 
 #include <asm/errata_list.h>
 #include <asm/sbi.h>
+#include <asm/sse.h>
 #include <asm/cpufeature.h>
 #include <asm/vendor_extensions.h>
 #include <asm/vendor_extensions/andes.h>
@@ -95,7 +98,6 @@ static bool riscv_pmu_use_irq;
 static unsigned int riscv_pmu_irq_num;
 static unsigned int riscv_pmu_irq_mask;
 static unsigned int riscv_pmu_irq;
-
 /* Cache the available counters in a bitmask */
 static DECLARE_BITMAP(cmask, RISCV_MAX_COUNTERS);
 
@@ -927,7 +929,7 @@ static int pmu_sbi_get_ctrinfo(int nctr, unsigned long *mask)
 	return 0;
 }
 
-static inline void pmu_sbi_stop_all(struct riscv_pmu *pmu)
+static inline void pmu_sbi_stop_all_mask(const unsigned long *ctr_mask)
 {
 	int i;
 
@@ -936,14 +938,42 @@ static inline void pmu_sbi_stop_all(struct riscv_pmu *pmu)
 	 * which may include counters that are not enabled yet.
 	 */
 	for (i = 0; i < BITS_TO_LONGS(RISCV_MAX_COUNTERS); i++) {
-		if (!pmu->cmask[i])
+		if (!ctr_mask[i])
 			continue;
 		sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_STOP,
-			  i * BITS_PER_LONG, pmu->cmask[i],
+			  i * BITS_PER_LONG, ctr_mask[i],
 			  SBI_PMU_STOP_FLAG_RESET);
 	}
 }
 
+static void pmu_sbi_sse_stop_all(const unsigned long *ctr_mask)
+{
+	struct sbiret ret;
+	int i;
+
+	/*
+	 * Stop each counter separately: an already-stopped counter must not
+	 * prevent firmware from stopping a later, overflowing counter.
+	 */
+	for_each_set_bit(i, ctr_mask, RISCV_MAX_COUNTERS) {
+		ret = sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_STOP,
+				i, 1, SBI_PMU_STOP_FLAG_RESET);
+		if (ret.error && ret.error != SBI_ERR_ALREADY_STOPPED)
+			pr_warn_ratelimited("failed to stop counter %d: %ld\n",
+					    i, ret.error);
+	}
+}
+
+static inline void pmu_sbi_stop_all(struct riscv_pmu *pmu)
+{
+	pmu_sbi_stop_all_mask(pmu->cmask);
+}
+
+static void pmu_sbi_stop_all_cpu(void *info)
+{
+	pmu_sbi_stop_all(info);
+}
+
 static inline void pmu_sbi_stop_hw_ctrs(struct riscv_pmu *pmu)
 {
 	struct cpu_hw_events *cpu_hw_evt = this_cpu_ptr(pmu->hw_events);
@@ -991,7 +1021,7 @@ static inline void pmu_sbi_stop_hw_ctrs(struct riscv_pmu *pmu)
 static inline void pmu_sbi_start_ovf_ctrs_sbi(struct cpu_hw_events *cpu_hw_evt,
 					      u64 ctr_ovf_mask)
 {
-	int idx = 0, i;
+	int idx, i;
 	struct perf_event *event;
 	unsigned long flag = SBI_PMU_START_FLAG_SET_INIT_VALUE;
 	unsigned long ctr_start_mask = 0;
@@ -1000,7 +1030,19 @@ static inline void pmu_sbi_start_ovf_ctrs_sbi(struct cpu_hw_events *cpu_hw_evt,
 	u64 init_val = 0;
 
 	for (i = 0; i < BITS_TO_LONGS(RISCV_MAX_COUNTERS); i++) {
-		ctr_start_mask = cpu_hw_evt->used_hw_ctrs[i] & ~ctr_ovf_mask;
+		unsigned long ctr_ovf_mask_word;
+		int lidx;
+
+		ctr_ovf_mask_word = ctr_ovf_mask >> (i * BITS_PER_LONG);
+		ctr_start_mask = 0;
+		for_each_set_bit(idx, &cpu_hw_evt->used_hw_ctrs[i], BITS_PER_LONG) {
+			lidx = idx + i * BITS_PER_LONG;
+			event = cpu_hw_evt->events[lidx];
+			if (event && !(event->hw.state & PERF_HES_STOPPED))
+				ctr_start_mask |= BIT(idx);
+		}
+		ctr_start_mask &= ~ctr_ovf_mask_word;
+
 		/* Start all the counters that did not overflow in a single shot */
 		if (ctr_start_mask) {
 			sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_START, i * BITS_PER_LONG,
@@ -1009,23 +1051,25 @@ static inline void pmu_sbi_start_ovf_ctrs_sbi(struct cpu_hw_events *cpu_hw_evt,
 	}
 
 	/* Reinitialize and start all the counter that overflowed */
-	while (ctr_ovf_mask) {
-		if (ctr_ovf_mask & 0x01) {
-			event = cpu_hw_evt->events[idx];
-			hwc = &event->hw;
-			max_period = riscv_pmu_ctr_get_width_mask(event);
-			init_val = local64_read(&hwc->prev_count) & max_period;
+	for (idx = 0; idx < RISCV_MAX_COUNTERS; idx++) {
+		if (!(ctr_ovf_mask & BIT_ULL(idx)))
+			continue;
+
+		event = cpu_hw_evt->events[idx];
+		if (!event || event->hw.state & PERF_HES_STOPPED)
+			continue;
+
+		hwc = &event->hw;
+		max_period = riscv_pmu_ctr_get_width_mask(event);
+		init_val = local64_read(&hwc->prev_count) & max_period;
 #if defined(CONFIG_32BIT)
-			sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_START, idx,
-				  1, flag, init_val, init_val >> 32);
+		sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_START, idx, 1,
+			  flag, init_val, init_val >> 32);
 #else
-			sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_START, idx,
-				  1, flag, init_val);
+		sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_START, idx, 1,
+			  flag, init_val);
 #endif
-			perf_event_update_userpage(event);
-		}
-		ctr_ovf_mask = ctr_ovf_mask >> 1;
-		idx++;
+		perf_event_update_userpage(event);
 	}
 }
 
@@ -1054,32 +1098,111 @@ static inline void pmu_sbi_start_ovf_ctrs_snapshot(struct cpu_hw_events *cpu_hw_
 	}
 
 	for (i = 0; i < BITS_TO_LONGS(RISCV_MAX_COUNTERS); i++) {
+		unsigned long ctr_start_mask = 0;
+		int lidx;
+
 		/* Restore the counter values to relative indices for used hw counters */
-		for_each_set_bit(idx, &cpu_hw_evt->used_hw_ctrs[i], BITS_PER_LONG)
-			sdata->ctr_values[idx] =
-					cpu_hw_evt->snapshot_cval_shcopy[idx + i * BITS_PER_LONG];
+		for_each_set_bit(idx, &cpu_hw_evt->used_hw_ctrs[i], BITS_PER_LONG) {
+			lidx = idx + i * BITS_PER_LONG;
+			event = cpu_hw_evt->events[lidx];
+			if (event && !(event->hw.state & PERF_HES_STOPPED))
+				ctr_start_mask |= BIT(idx);
+
+			sdata->ctr_values[idx] = cpu_hw_evt->snapshot_cval_shcopy[lidx];
+		}
+
 		/* Start all the counters in a single shot */
-		sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_START,
-			  idx * BITS_PER_LONG,
-			  cpu_hw_evt->used_hw_ctrs[i], flag);
+		if (ctr_start_mask)
+			sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_START,
+				  i * BITS_PER_LONG, ctr_start_mask, flag);
 	}
 }
 
+static bool pmu_sbi_sse_failed(struct cpu_hw_events *cpu_hw_evt)
+{
+#ifdef CONFIG_RISCV_PMU_SBI_SSE
+	return READ_ONCE(cpu_hw_evt->sse_failed);
+#else
+	return false;
+#endif
+}
+
 static void pmu_sbi_start_overflow_mask(struct riscv_pmu *pmu,
 					u64 ctr_ovf_mask)
 {
 	struct cpu_hw_events *cpu_hw_evt = this_cpu_ptr(pmu->hw_events);
 
+	if (unlikely(pmu_sbi_sse_failed(cpu_hw_evt)))
+		return;
+
 	if (sbi_pmu_snapshot_available())
 		pmu_sbi_start_ovf_ctrs_snapshot(cpu_hw_evt, ctr_ovf_mask);
 	else
 		pmu_sbi_start_ovf_ctrs_sbi(cpu_hw_evt, ctr_ovf_mask);
 }
 
-static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
+#ifdef CONFIG_RISCV_PMU_SBI_SSE
+/*
+ * A local SSE delivery failure makes the current PMU state unsafe to resume.
+ * Latch the failure before stopping mapped events so the SSE transition and
+ * overflow restart paths cannot undo the fail-safe while they are quiesced.
+ */
+static void pmu_sbi_fail_sse(struct riscv_pmu *pmu, const char *op, int ret)
+{
+	struct cpu_hw_events *cpu_hw_evt = this_cpu_ptr(pmu->hw_events);
+	struct perf_event *event;
+	int idx;
+
+	if (READ_ONCE(cpu_hw_evt->sse_failed))
+		return;
+
+	WRITE_ONCE(cpu_hw_evt->sse_failed, true);
+	pr_err_ratelimited("failed to %s local PMU SSE event: %d; stopping counters\n",
+			   op, ret);
+
+	for (idx = 0; idx < RISCV_MAX_COUNTERS; idx++) {
+		event = cpu_hw_evt->events[idx];
+		if (event)
+			riscv_pmu_stop(event, PERF_EF_UPDATE);
+	}
+}
+
+static void pmu_sbi_sse_disable(struct pmu *pmu)
+{
+	struct riscv_pmu *rvpmu = to_riscv_pmu(pmu);
+	struct cpu_hw_events *cpu_hw_evt = this_cpu_ptr(rvpmu->hw_events);
+	int ret;
+
+	if (!READ_ONCE(rvpmu->sse_active) ||
+	    READ_ONCE(cpu_hw_evt->sse_failed))
+		return;
+
+	ret = sse_event_disable_local(rvpmu->sse_evt);
+	if (unlikely(ret))
+		pmu_sbi_fail_sse(rvpmu, "disable", ret);
+}
+
+static void pmu_sbi_sse_enable(struct pmu *pmu)
+{
+	struct riscv_pmu *rvpmu = to_riscv_pmu(pmu);
+	struct cpu_hw_events *cpu_hw_evt = this_cpu_ptr(rvpmu->hw_events);
+	int ret;
+
+	if (!READ_ONCE(rvpmu->sse_active) ||
+	    READ_ONCE(cpu_hw_evt->sse_failed))
+		return;
+
+	ret = sse_event_enable_local(rvpmu->sse_evt);
+	if (unlikely(ret))
+		pmu_sbi_fail_sse(rvpmu, "enable", ret);
+}
+#endif
+
+static irqreturn_t pmu_sbi_ovf_handler(struct cpu_hw_events *cpu_hw_evt,
+				       struct pt_regs *regs, bool from_sse,
+				       bool from_guest)
 {
 	struct perf_sample_data data;
-	struct pt_regs *regs;
 	struct hw_perf_event *hw_evt;
 	union sbi_pmu_ctr_info *info;
 	int lidx, hidx, fidx;
@@ -1087,7 +1210,6 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
 	struct perf_event *event;
 	u64 overflow;
 	u64 overflowed_ctrs = 0;
-	struct cpu_hw_events *cpu_hw_evt = dev;
 	u64 start_clock = sched_clock();
 	struct riscv_pmu_snapshot_data *sdata;
 
@@ -1096,21 +1218,32 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
 
 	sdata = cpu_hw_evt->snapshot_addr;
 
-	/* Firmware counter don't support overflow yet */
+	/*
+	 * SSE can arrive before perf installs an event. The early exits below
+	 * must stop the PMU source before firmware completes the SSE.
+	 */
 	fidx = find_first_bit(cpu_hw_evt->used_hw_ctrs, RISCV_MAX_COUNTERS);
 	if (fidx == RISCV_MAX_COUNTERS) {
-		csr_clear(CSR_SIP, BIT(riscv_pmu_irq_num));
+		if (from_sse)
+			pmu_sbi_sse_stop_all(cmask);
+		else
+			csr_clear(CSR_SIP, BIT(riscv_pmu_irq_num));
 		return IRQ_NONE;
 	}
 
 	event = cpu_hw_evt->events[fidx];
 	if (!event) {
-		ALT_SBI_PMU_OVF_CLEAR_PENDING(riscv_pmu_irq_mask);
+		if (from_sse)
+			pmu_sbi_sse_stop_all(cmask);
+		else
+			ALT_SBI_PMU_OVF_CLEAR_PENDING(riscv_pmu_irq_mask);
 		return IRQ_NONE;
 	}
 
 	pmu = to_riscv_pmu(event->pmu);
 	pmu_sbi_stop_hw_ctrs(pmu);
+	if (unlikely(pmu_sbi_sse_failed(cpu_hw_evt)))
+		return IRQ_NONE;
 
 	/* Overflow status register should only be read after counter are stopped */
 	if (sbi_pmu_snapshot_available())
@@ -1120,15 +1253,17 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
 
 	/*
 	 * Overflow interrupt pending bit should only be cleared after stopping
-	 * all the counters to avoid any race condition.
+	 * all the counters to avoid any race condition. When using SSE,
+	 * interrupt is cleared when stopping counters.
 	 */
-	ALT_SBI_PMU_OVF_CLEAR_PENDING(riscv_pmu_irq_mask);
+	if (!from_sse)
+		ALT_SBI_PMU_OVF_CLEAR_PENDING(riscv_pmu_irq_mask);
 
 	/* No overflow bit is set */
-	if (!overflow)
+	if (!overflow) {
+		pmu_sbi_start_overflow_mask(pmu, 0);
 		return IRQ_NONE;
-
-	regs = get_irq_regs();
+	}
 
 	for_each_set_bit(lidx, cpu_hw_evt->used_hw_ctrs, RISCV_MAX_COUNTERS) {
 		struct perf_event *event = cpu_hw_evt->events[lidx];
@@ -1153,6 +1288,12 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
 		if (!(overflow & BIT_ULL(hidx)))
 			continue;
 
+#ifdef CONFIG_CPU_PM
+		/* Do not let CPU-PM resume override this overflow decision. */
+		if (from_sse)
+			clear_bit(lidx, cpu_hw_evt->pm_resume_hw_ctrs);
+#endif
+
 		/*
 		 * Keep a track of overflowed counters so that they can be started
 		 * with updated initial value.
@@ -1165,6 +1306,14 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
 		hw_evt->state |= PERF_HES_UPTODATE;
 		perf_sample_data_init(&data, 0, hw_evt->last_period);
 		if (riscv_pmu_event_set_period(event)) {
+			int overflow_ret;
+
+			/* Guest attribution and guest stack sampling are not supported yet. */
+			if (from_guest) {
+				hw_evt->state = 0;
+				continue;
+			}
+
 			/*
 			 * Unlike other ISAs, RISC-V don't have to disable interrupts
 			 * to avoid throttling here. As per the specification, the
@@ -1173,128 +1322,291 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
 			 * TODO: We will need to stop the guest counters once
 			 * virtualization support is added.
 			 */
-			perf_event_overflow(event, &data, regs);
+			overflow_ret = perf_event_overflow(event, &data, regs);
+			if (!overflow_ret)
+				hw_evt->state = 0;
+		} else {
+			hw_evt->state = 0;
 		}
-		/* Reset the state as we are going to start the counter after the loop */
-		hw_evt->state = 0;
 	}
 
 	pmu_sbi_start_overflow_mask(pmu, overflowed_ctrs);
+
 	perf_sample_event_took(sched_clock() - start_clock);
 
 	return IRQ_HANDLED;
 }
 
-static int pmu_sbi_starting_cpu(unsigned int cpu, struct hlist_node *node)
+static irqreturn_t pmu_sbi_ovf_irq_handler(int irq, void *dev)
 {
-	struct riscv_pmu *pmu = hlist_entry_safe(node, struct riscv_pmu, node);
-	struct cpu_hw_events *cpu_hw_evt = this_cpu_ptr(pmu->hw_events);
-
-	/*
-	 * We keep enabling userspace access to CYCLE, TIME and INSTRET via the
-	 * legacy option but that will be removed in the future.
-	 */
-	if (sysctl_perf_user_access == SYSCTL_LEGACY)
-		csr_write(CSR_SCOUNTEREN, 0x7);
-	else
-		csr_write(CSR_SCOUNTEREN, 0x2);
+	return pmu_sbi_ovf_handler(dev, get_irq_regs(), false, false);
+}
 
-	/* Stop all the counters so that they can be enabled from perf */
-	pmu_sbi_stop_all(pmu);
+#ifdef CONFIG_RISCV_PMU_SBI_SSE
+static int pmu_sbi_ovf_sse_handler(u32 evt, void *arg, struct pt_regs *regs)
+{
+	const struct riscv_sse_interrupted_context *context;
+	struct riscv_pmu *pmu = arg;
+	struct cpu_hw_events *hw_event = raw_cpu_ptr(pmu->hw_events);
+	bool from_guest;
+
+	if (unlikely(!READ_ONCE(pmu->sse_active))) {
+		pmu_sbi_sse_stop_all(pmu->cmask);
+		return -EIO;
+	}
 
-	if (riscv_pmu_use_irq) {
-		cpu_hw_evt->irq = riscv_pmu_irq;
-		ALT_SBI_PMU_OVF_CLEAR_PENDING(riscv_pmu_irq_mask);
-		enable_percpu_irq(riscv_pmu_irq, IRQ_TYPE_NONE);
+	if (unlikely(!regs)) {
+		pmu_sbi_fail_sse(pmu, "read interrupted context for", -EIO);
+		return -EIO;
 	}
 
-	if (sbi_pmu_snapshot_available())
-		return pmu_sbi_snapshot_setup(pmu, cpu);
+	context = riscv_sse_get_interrupted_context();
+	from_guest = context && context->regs == regs &&
+		     (context->hstatus & HSTATUS_SPV);
+	pmu_sbi_ovf_handler(hw_event, regs, true, from_guest);
 
 	return 0;
 }
 
-static int pmu_sbi_dying_cpu(unsigned int cpu, struct hlist_node *node)
+static int pmu_sbi_setup_sse(struct riscv_pmu *pmu)
 {
-	if (riscv_pmu_use_irq) {
-		disable_percpu_irq(riscv_pmu_irq);
-	}
+	int ret;
+	struct sse_event *evt;
 
-	/* Disable all counters access for user mode now */
-	csr_write(CSR_SCOUNTEREN, 0x0);
+	evt = sse_event_register(SBI_SSE_EVENT_LOCAL_PMU_OVERFLOW, 0,
+				 pmu_sbi_ovf_sse_handler, pmu);
+	if (IS_ERR(evt))
+		return PTR_ERR(evt);
+	pmu->sse_evt = evt;
 
-	if (sbi_pmu_snapshot_available())
-		return pmu_sbi_snapshot_disable();
+	ret = sse_event_enable(evt);
+	if (ret) {
+		int cleanup_ret;
 
-	return 0;
+		cleanup_ret = sse_event_disable(evt);
+		if (cleanup_ret) {
+			pr_warn("failed to disable SSE event after setup error: %d\n",
+				cleanup_ret);
+			return cleanup_ret;
+		}
+
+		cleanup_ret = sse_event_unregister(evt);
+
+		if (cleanup_ret) {
+			pr_warn("failed to unregister SSE event: %d\n",
+				cleanup_ret);
+		} else {
+			pmu->sse_evt = NULL;
+		}
+		return cleanup_ret ?: ret;
+	}
+
+	WRITE_ONCE(pmu->sse_active, true);
+	pr_info("using SSE for PMU event delivery\n");
+
+	return ret;
 }
 
-static int pmu_sbi_setup_irqs(struct riscv_pmu *pmu, struct platform_device *pdev)
+static void pmu_sbi_cleanup_sse(struct riscv_pmu *pmu)
 {
+	struct sse_event *sse_evt;
 	int ret;
-	struct cpu_hw_events __percpu *hw_events = pmu->hw_events;
-	struct irq_domain *domain = NULL;
 
+	sse_evt = pmu->sse_evt;
+	if (!sse_evt)
+		return;
+	/*
+	 * Close callback admission before draining each CPU. PMU callbacks run
+	 * with local interrupts disabled, so the synchronous IPI cannot complete
+	 * until a callback that observed the old state has returned.
+	 */
+	WRITE_ONCE(pmu->sse_active, false);
+	on_each_cpu(pmu_sbi_stop_all_cpu, pmu, 1);
+
+	ret = sse_event_disable(sse_evt);
+	if (ret) {
+		pr_warn("failed to disable SSE event: %d\n", ret);
+		goto retain;
+	}
+
+	ret = sse_event_unregister(sse_evt);
+	if (ret) {
+		pr_warn("failed to unregister SSE event: %d\n", ret);
+		goto retain;
+	}
+
+	pmu->sse_evt = NULL;
+	return;
+
+retain:
+	sse_event_cleanup(sse_evt);
+	pmu->sse_evt = NULL;
+}
+
+static bool pmu_sbi_sse_state_retained(struct riscv_pmu *pmu)
+{
+	return pmu->sse_evt;
+}
+#else
+static int pmu_sbi_setup_sse(struct riscv_pmu *pmu)
+{
+	return -EOPNOTSUPP;
+}
+
+static void pmu_sbi_cleanup_sse(struct riscv_pmu *pmu) {}
+
+static bool pmu_sbi_sse_state_retained(struct riscv_pmu *pmu)
+{
+	return false;
+}
+#endif
+
+static bool pmu_sbi_select_irq(void)
+{
 	if (riscv_isa_extension_available(NULL, SSCOFPMF)) {
 		riscv_pmu_irq_num = RV_IRQ_PMU;
-		riscv_pmu_use_irq = true;
+		return true;
 	} else if (IS_ENABLED(CONFIG_ERRATA_THEAD_PMU) &&
 		   riscv_cached_mvendorid(0) == THEAD_VENDOR_ID &&
 		   riscv_cached_marchid(0) == 0 &&
 		   riscv_cached_mimpid(0) == 0) {
 		riscv_pmu_irq_num = THEAD_C9XX_RV_IRQ_PMU;
-		riscv_pmu_use_irq = true;
+		return true;
 	} else if (riscv_has_vendor_extension_unlikely(ANDES_VENDOR_ID,
 						       RISCV_ISA_VENDOR_EXT_XANDESPMU) &&
 		   IS_ENABLED(CONFIG_ANDES_CUSTOM_PMU)) {
 		riscv_pmu_irq_num = ANDES_SLI_CAUSE_BASE + ANDES_RV_IRQ_PMOVI;
-		riscv_pmu_use_irq = true;
+		return true;
 	}
 
-	riscv_pmu_irq_mask = BIT(riscv_pmu_irq_num % BITS_PER_LONG);
+	return false;
+}
 
-	if (!riscv_pmu_use_irq)
+static int pmu_sbi_setup_irq(struct riscv_pmu *pmu)
+{
+	struct cpu_hw_events __percpu *hw_events = pmu->hw_events;
+	struct irq_domain *domain;
+	int ret;
+
+	if (!pmu_sbi_select_irq())
 		return -EOPNOTSUPP;
 
+	riscv_pmu_irq_mask = BIT(riscv_pmu_irq_num % BITS_PER_LONG);
+
 	domain = irq_find_matching_fwnode(riscv_get_intc_hwnode(),
 					  DOMAIN_BUS_ANY);
 	if (!domain) {
 		pr_err("Failed to find INTC IRQ root domain\n");
-		ret = -ENODEV;
-		goto err;
+		return -ENODEV;
 	}
 
 	riscv_pmu_irq = irq_create_mapping(domain, riscv_pmu_irq_num);
 	if (!riscv_pmu_irq) {
 		pr_err("Failed to map PMU interrupt for node\n");
-		ret = -ENODEV;
-		goto err;
+		return -ENODEV;
 	}
 
-	ret = request_percpu_irq(riscv_pmu_irq, pmu_sbi_ovf_handler, "riscv-pmu", hw_events);
+	ret = request_percpu_irq(riscv_pmu_irq, pmu_sbi_ovf_irq_handler,
+				 "riscv-pmu", hw_events);
 	if (ret) {
 		pr_err("registering percpu irq failed [%d]\n", ret);
 		irq_dispose_mapping(riscv_pmu_irq);
 		riscv_pmu_irq = 0;
-		goto err;
+		return ret;
 	}
 
 	return 0;
-err:
-	riscv_pmu_use_irq = false;
-	return ret;
+}
+
+static int pmu_sbi_starting_cpu(unsigned int cpu, struct hlist_node *node)
+{
+	struct riscv_pmu *pmu = hlist_entry_safe(node, struct riscv_pmu, node);
+	struct cpu_hw_events *cpu_hw_evt = this_cpu_ptr(pmu->hw_events);
+
+	/*
+	 * We keep enabling userspace access to CYCLE, TIME and INSTRET via the
+	 * legacy option but that will be removed in the future.
+	 */
+	if (sysctl_perf_user_access == SYSCTL_LEGACY)
+		csr_write(CSR_SCOUNTEREN, 0x7);
+	else
+		csr_write(CSR_SCOUNTEREN, 0x2);
+
+	/* Stop all the counters so that they can be enabled from perf */
+	pmu_sbi_stop_all(pmu);
+
+	if (riscv_pmu_use_irq) {
+		cpu_hw_evt->irq = riscv_pmu_irq;
+		ALT_SBI_PMU_OVF_CLEAR_PENDING(riscv_pmu_irq_mask);
+		enable_percpu_irq(riscv_pmu_irq, IRQ_TYPE_NONE);
+	}
+
+	if (sbi_pmu_snapshot_available())
+		return pmu_sbi_snapshot_setup(pmu, cpu);
+
+	return 0;
+}
+
+static int pmu_sbi_dying_cpu(unsigned int cpu, struct hlist_node *node)
+{
+	if (riscv_pmu_use_irq)
+		disable_percpu_irq(riscv_pmu_irq);
+
+	/* Disable all counters access for user mode now */
+	csr_write(CSR_SCOUNTEREN, 0x0);
+
+	if (sbi_pmu_snapshot_available())
+		return pmu_sbi_snapshot_disable();
+
+	return 0;
+}
+
+static int pmu_sbi_setup_irqs(struct riscv_pmu *pmu, struct platform_device *pdev)
+{
+	int irq_ret;
+	int sse_ret;
+
+	/* Do not claim an IRQ route while a crash kernel leaves SSE state intact. */
+	if (is_kdump_kernel() && riscv_sse_available()) {
+		pr_warn("PMU delivery unavailable with retained crash-kernel SSE state\n");
+		return -EUCLEAN;
+	}
+
+	sse_ret = pmu_sbi_setup_sse(pmu);
+	if (!sse_ret) {
+		riscv_pmu_use_irq = false;
+		return 0;
+	}
+	if (pmu_sbi_sse_state_retained(pmu)) {
+		pr_err("PMU-SSE setup failed with firmware state retained: %d\n",
+		       sse_ret);
+		return -EUCLEAN;
+	}
+	/* Only an explicitly unsupported SSE path proves IRQ fallback is safe. */
+	if (sse_ret != -EOPNOTSUPP) {
+		pr_err("PMU-SSE setup failed: %d\n", sse_ret);
+		return sse_ret;
+	}
+
+	irq_ret = pmu_sbi_setup_irq(pmu);
+	if (!irq_ret) {
+		riscv_pmu_use_irq = true;
+		return 0;
+	}
+	return irq_ret;
 }
 
 #ifdef CONFIG_CPU_PM
-static int riscv_pm_pmu_notify(struct notifier_block *b, unsigned long cmd,
-				void *v)
+static int riscv_pm_pmu_update(struct riscv_pmu *rvpmu, unsigned long cmd)
 {
-	struct riscv_pmu *rvpmu = container_of(b, struct riscv_pmu, riscv_pm_nb);
 	struct cpu_hw_events *cpuc = this_cpu_ptr(rvpmu->hw_events);
 	bool enabled = !bitmap_empty(cpuc->used_hw_ctrs, RISCV_MAX_COUNTERS);
 	struct perf_event *event;
 	int idx;
 
+	if (cmd == CPU_PM_ENTER)
+		bitmap_zero(cpuc->pm_resume_hw_ctrs, RISCV_MAX_COUNTERS);
+
 	if (!enabled)
 		return NOTIFY_OK;
 
@@ -1305,6 +1617,8 @@ static int riscv_pm_pmu_notify(struct notifier_block *b, unsigned long cmd,
 
 		switch (cmd) {
 		case CPU_PM_ENTER:
+			if (!(event->hw.state & PERF_HES_STOPPED))
+				set_bit(idx, cpuc->pm_resume_hw_ctrs);
 			/*
 			 * Stop and update the counter
 			 */
@@ -1312,6 +1626,8 @@ static int riscv_pm_pmu_notify(struct notifier_block *b, unsigned long cmd,
 			break;
 		case CPU_PM_EXIT:
 		case CPU_PM_ENTER_FAILED:
+			if (!test_and_clear_bit(idx, cpuc->pm_resume_hw_ctrs))
+				break;
 			/*
 			 * Restore and enable the counter.
 			 */
@@ -1325,9 +1641,59 @@ static int riscv_pm_pmu_notify(struct notifier_block *b, unsigned long cmd,
 	return NOTIFY_OK;
 }
 
+static int riscv_pm_pmu_notify(struct notifier_block *b,
+			       unsigned long cmd, void *v)
+{
+	struct riscv_pmu *rvpmu = container_of(b, struct riscv_pmu,
+						 riscv_pm_nb);
+
+#ifdef CONFIG_RISCV_PMU_SBI_SSE
+	struct cpu_hw_events *cpuc = this_cpu_ptr(rvpmu->hw_events);
+	int ret;
+
+	if (!riscv_pmu_use_irq && READ_ONCE(rvpmu->sse_active)) {
+		switch (cmd) {
+		case CPU_PM_ENTER:
+			cpuc->pm_resume_sse =
+				sse_event_is_enabled_local(rvpmu->sse_evt);
+			if (cpuc->pm_resume_sse) {
+				ret = sse_event_disable_local(rvpmu->sse_evt);
+				if (ret) {
+					cpuc->pm_resume_sse = false;
+					pmu_sbi_fail_sse(rvpmu, "disable for CPU PM",
+							 ret);
+					return notifier_from_errno(ret);
+				}
+			}
+			break;
+		case CPU_PM_EXIT:
+		case CPU_PM_ENTER_FAILED:
+			ret = riscv_pm_pmu_update(rvpmu, cmd);
+			if (!cpuc->pm_resume_sse)
+				return ret;
+
+			cpuc->pm_resume_sse = false;
+			ret = sse_event_enable_local(rvpmu->sse_evt);
+			if (ret) {
+				pmu_sbi_fail_sse(rvpmu, "enable after CPU PM", ret);
+				return notifier_from_errno(ret);
+			}
+			return NOTIFY_OK;
+		default:
+			break;
+		}
+	}
+#endif
+
+	return riscv_pm_pmu_update(rvpmu, cmd);
+}
+
 static int riscv_pm_pmu_register(struct riscv_pmu *pmu)
 {
 	pmu->riscv_pm_nb.notifier_call = riscv_pm_pmu_notify;
+	/* Keep PMU-SSE disabled until counters and userpage state are restored. */
+	pmu->riscv_pm_nb.priority = riscv_pmu_use_irq ? 0 : -1;
+
 	return cpu_pm_register_notifier(&pmu->riscv_pm_nb);
 }
 
@@ -1342,6 +1708,8 @@ static inline void riscv_pm_pmu_unregister(struct riscv_pmu *pmu) { }
 
 static void riscv_pmu_destroy(struct riscv_pmu *pmu)
 {
+	pmu_sbi_cleanup_sse(pmu);
+
 	if (sbi_v2_available) {
 		if (sbi_pmu_snapshot_available()) {
 			pmu_sbi_snapshot_disable();
@@ -1353,7 +1721,7 @@ static void riscv_pmu_destroy(struct riscv_pmu *pmu)
 		cpuhp_state_remove_instance(CPUHP_AP_PERF_RISCV_STARTING, &pmu->node);
 }
 
-static void pmu_sbi_event_init(struct perf_event *event)
+static int pmu_sbi_event_init(struct perf_event *event)
 {
 	/*
 	 * The permissions are set at event_init so that we do not depend
@@ -1365,6 +1733,8 @@ static void pmu_sbi_event_init(struct perf_event *event)
 		event->hw.flags |= PERF_EVENT_FLAG_USER_ACCESS;
 	else
 		event->hw.flags |= PERF_EVENT_FLAG_LEGACY;
+
+	return 0;
 }
 
 static void pmu_sbi_event_mapped(struct perf_event *event, struct mm_struct *mm)
@@ -1494,6 +1864,7 @@ static int pmu_sbi_device_probe(struct platform_device *pdev)
 	/* cache all the information about counters now */
 	if (pmu_sbi_get_ctrinfo(num_counters, cmask))
 		goto out_free;
+	bitmap_copy(pmu->cmask, cmask, RISCV_MAX_COUNTERS);
 
 	ret = pmu_sbi_setup_irqs(pmu, pdev);
 	if (ret < 0) {
@@ -1501,11 +1872,17 @@ static int pmu_sbi_device_probe(struct platform_device *pdev)
 		pmu->pmu.capabilities |= PERF_PMU_CAP_NO_INTERRUPT;
 		pmu->pmu.capabilities |= PERF_PMU_CAP_NO_EXCLUDE;
 	}
-	irq_requested = (ret == 0);
+	irq_requested = !ret && riscv_pmu_use_irq;
+
+#ifdef CONFIG_RISCV_PMU_SBI_SSE
+	if (pmu->sse_active) {
+		pmu->pmu.pmu_enable = pmu_sbi_sse_enable;
+		pmu->pmu.pmu_disable = pmu_sbi_sse_disable;
+	}
+#endif
 
 	pmu->pmu.attr_groups = riscv_pmu_attr_groups;
 	pmu->pmu.parent = &pdev->dev;
-	bitmap_copy(pmu->cmask, cmask, RISCV_MAX_COUNTERS);
 	pmu->ctr_start = pmu_sbi_ctr_start;
 	pmu->ctr_stop = pmu_sbi_ctr_stop;
 	pmu->event_map = pmu_sbi_event_map;
diff --git a/include/linux/perf/riscv_pmu.h b/include/linux/perf/riscv_pmu.h
index ecaa40370830..b06abc705100 100644
--- a/include/linux/perf/riscv_pmu.h
+++ b/include/linux/perf/riscv_pmu.h
@@ -28,6 +28,8 @@
 
 #define RISCV_PMU_CONFIG1_GUEST_EVENTS 0x1
 
+struct sse_event;
+
 struct cpu_hw_events {
 	/* currently enabled events */
 	int			n_events;
@@ -39,6 +41,18 @@ struct cpu_hw_events {
 	DECLARE_BITMAP(used_hw_ctrs, RISCV_MAX_COUNTERS);
 	/* currently enabled firmware counters */
 	DECLARE_BITMAP(used_fw_ctrs, RISCV_MAX_COUNTERS);
+#ifdef CONFIG_RISCV_PMU_SBI_SSE
+	/* Keep counters stopped after an unrecoverable SSE transition failure. */
+	bool sse_failed;
+#endif
+#ifdef CONFIG_CPU_PM
+	/* Counters stopped by CPU PM and still waiting to be restored. */
+	DECLARE_BITMAP(pm_resume_hw_ctrs, RISCV_MAX_COUNTERS);
+#ifdef CONFIG_RISCV_PMU_SBI_SSE
+	/* Restore the local PMU SSE event after counters and userpage state. */
+	bool pm_resume_sse;
+#endif
+#endif
 	/* The virtual address of the shared memory where counter snapshot will be taken */
 	void *snapshot_addr;
 	/* The physical address of the shared memory where counter snapshot will be taken */
@@ -54,6 +68,10 @@ struct riscv_pmu {
 	char		*name;
 
 	irqreturn_t	(*handle_irq)(int irq_num, void *dev);
+#ifdef CONFIG_RISCV_PMU_SBI_SSE
+	struct sse_event	*sse_evt;
+	bool			sse_active;
+#endif
 
 	DECLARE_BITMAP(cmask, RISCV_MAX_COUNTERS);
 	u64		(*ctr_read)(struct perf_event *event);
@@ -63,7 +81,7 @@ struct riscv_pmu {
 	void		(*ctr_start)(struct perf_event *event, u64 init_val);
 	void		(*ctr_stop)(struct perf_event *event, unsigned long flag);
 	int		(*event_map)(struct perf_event *event, u64 *config);
-	void		(*event_init)(struct perf_event *event);
+	int		(*event_init)(struct perf_event *event);
 	void		(*event_mapped)(struct perf_event *event, struct mm_struct *mm);
 	void		(*event_unmapped)(struct perf_event *event, struct mm_struct *mm);
 	uint8_t		(*csr_index)(struct perf_event *event);
diff --git a/include/linux/riscv_sbi_sse.h b/include/linux/riscv_sbi_sse.h
index 774a782e556d..f6b3881c8884 100644
--- a/include/linux/riscv_sbi_sse.h
+++ b/include/linux/riscv_sbi_sse.h
@@ -42,6 +42,7 @@ int sse_event_enable(struct sse_event *sse_evt);
 int sse_event_disable(struct sse_event *sse_evt);
 
 /* Local events require the caller to remain on the current CPU. */
+bool sse_event_is_enabled_local(struct sse_event *sse_evt);
 int sse_event_enable_local(struct sse_event *sse_evt);
 int sse_event_disable_local(struct sse_event *sse_evt);
 
@@ -76,6 +77,11 @@ static inline int sse_event_disable(struct sse_event *sse_evt)
 	return -EOPNOTSUPP;
 }
 
+static inline bool sse_event_is_enabled_local(struct sse_event *sse_evt)
+{
+	return false;
+}
+
 static inline int sse_event_enable_local(struct sse_event *sse_evt)
 {
 	return -EOPNOTSUPP;
-- 
2.50.1 (Apple Git-155)


^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH v13 09/11] MAINTAINERS: add RISC-V SSE maintainers and file coverage
  2026-11-19 12:08 [PATCH v13 00/11] riscv: add SBI Supervisor Software Events support Zhanpeng Zhang
                   ` (7 preceding siblings ...)
  2026-11-19 12:08 ` [PATCH v13 08/11] perf: RISC-V: add support for SSE event Zhanpeng Zhang
@ 2026-11-19 12:08 ` Zhanpeng Zhang
  2026-11-19 12:08 ` [PATCH v13 10/11] selftests/riscv: add SSE test module Zhanpeng Zhang
  2026-11-19 12:08 ` [PATCH v13 11/11] selftests/riscv: add perf user-stack SSE copy regression test Zhanpeng Zhang
  10 siblings, 0 replies; 22+ messages in thread
From: Zhanpeng Zhang @ 2026-11-19 12:08 UTC (permalink / raw)
  To: pjw, palmer, aou, alex, himanshu.chauhan, conor, anup.patel
  Cc: clement.leger, cuiyunhui, atish.patra, peterz, mingo, acme,
	namhyung, mark.rutland, alexander.shishkin, jolsa, irogers,
	adrian.hunter, james.clark, will, tglx, corbet, rdunlap, shuah,
	yuanzhu, yang.susheng, linux-riscv, linux-kernel,
	linux-perf-users, linux-doc, linux-kselftest,
	zhangzhanpeng.jasper

List Zhanpeng Zhang, Himanshu Chauhan and Clément Léger as co-maintainers
for RISC-V Supervisor Software Events, with Yunhui Cui as reviewer.
Cover the architecture entry, firmware driver, PMU documentation and
selftests in a separate patch, as requested by Himanshu. Also add
coverage for RISC-V firmware drivers and the architecture selftests.

Link: https://lore.kernel.org/r/as79MHtNYd-VbUug@hu-himchau-blr.qualcomm.com/
Signed-off-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
---
 MAINTAINERS | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 93abbb315065..c2d283895ef4 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -24060,6 +24060,7 @@ C:	irc://irc.libera.chat/riscv
 P:	Documentation/arch/riscv/patch-acceptance.rst
 T:	git git://git.kernel.org/pub/scm/linux/kernel/git/riscv/linux.git
 F:	arch/riscv/
+F:	tools/testing/selftests/riscv/
 N:	riscv
 K:	riscv
 
@@ -24072,6 +24073,13 @@ T:	git git://git.kernel.org/pub/scm/linux/kernel/git/iommu/linux.git
 F:	Documentation/devicetree/bindings/iommu/riscv,iommu.yaml
 F:	drivers/iommu/riscv/
 
+RISC-V FIRMWARE DRIVERS
+M:	Conor Dooley <conor@kernel.org>
+L:	linux-riscv@lists.infradead.org
+S:	Maintained
+T:	git git://git.kernel.org/pub/scm/linux/kernel/git/conor/linux.git
+F:	drivers/firmware/riscv/
+
 RISC-V MICROCHIP SUPPORT
 M:	Conor Dooley <conor.dooley@microchip.com>
 R:	Valentina Fernandez <valentina.fernandezalanis@microchip.com>
@@ -24128,6 +24136,7 @@ M:	Atish Patra <atish.patra@linux.dev>
 R:	Anup Patel <anup@brainfault.org>
 L:	linux-riscv@lists.infradead.org
 S:	Supported
+F:	Documentation/arch/riscv/pmu-sse.rst
 F:	drivers/perf/riscv_pmu.c
 F:	drivers/perf/riscv_pmu_legacy.c
 F:	drivers/perf/riscv_pmu_sbi.c
@@ -24176,6 +24185,20 @@ F:	arch/riscv/boot/dts/spacemit/
 N:	spacemit
 K:	spacemit
 
+RISC-V SUPERVISOR SOFTWARE EVENTS
+M:	Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
+M:	Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>
+M:	Clément Léger <clement.leger@linux.dev>
+R:	Yunhui Cui <cuiyunhui@bytedance.com>
+L:	linux-riscv@lists.infradead.org
+S:	Maintained
+F:	arch/riscv/include/asm/sse.h
+F:	arch/riscv/kernel/sbi_sse.c
+F:	arch/riscv/kernel/sbi_sse_entry.S
+F:	drivers/firmware/riscv/riscv_sbi_sse.c
+F:	include/linux/riscv_sbi_sse.h
+F:	tools/testing/selftests/riscv/sse/
+
 RISC-V TENSTORRENT SoC SUPPORT
 M:	Drew Fustini <fustini@kernel.org>
 M:	Joel Stanley <jms@oss.tenstorrent.com>
-- 
2.50.1 (Apple Git-155)


^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH v13 10/11] selftests/riscv: add SSE test module
  2026-11-19 12:08 [PATCH v13 00/11] riscv: add SBI Supervisor Software Events support Zhanpeng Zhang
                   ` (8 preceding siblings ...)
  2026-11-19 12:08 ` [PATCH v13 09/11] MAINTAINERS: add RISC-V SSE maintainers and file coverage Zhanpeng Zhang
@ 2026-11-19 12:08 ` Zhanpeng Zhang
  2026-11-19 12:37   ` sashiko-bot
  2026-11-19 12:08 ` [PATCH v13 11/11] selftests/riscv: add perf user-stack SSE copy regression test Zhanpeng Zhang
  10 siblings, 1 reply; 22+ messages in thread
From: Zhanpeng Zhang @ 2026-11-19 12:08 UTC (permalink / raw)
  To: pjw, palmer, aou, alex, himanshu.chauhan, conor, anup.patel
  Cc: clement.leger, cuiyunhui, atish.patra, peterz, mingo, acme,
	namhyung, mark.rutland, alexander.shishkin, jolsa, irogers,
	adrian.hunter, james.clark, will, tglx, corbet, rdunlap, shuah,
	yuanzhu, yang.susheng, linux-riscv, linux-kernel,
	linux-perf-users, linux-doc, linux-kselftest,
	zhangzhanpeng.jasper

From: Clément Léger <cleger@rivosinc.com>

Add an SSE selftest module and runner. Loading the module executes
smoke tests for the SSE framework, and the runner reports any error
emitted by the module.

Add stress={0,1,2} modes to exercise repeated handler entry and
completion, single and multiple read-only SBI calls from a handler, and
self re-injection. Check the SBI return values so handler execution alone
cannot produce a false pass, and avoid touching an unreserved PMU counter
owned by another user.

Harden the test against false failures and leaks by using TEST_PROGS for
the runner, using phys_addr_t for SBI attribute buffers, adding timeouts
to busy waits, pinning each priority chain with migrate_disable(), and
holding the CPU read lock while a fast-test target is selected, injected,
and completed. Unregister all registered events on error, check teardown
failures, and let kthread_stop() drive monitor-thread exit.

Track handler progress across fast, priority, and stress paths. If
firmware reports injectable events but the test cannot acquire or handle
any of them, report SKIP instead of silently passing a capability-only
run.

Signed-off-by: Clément Léger <cleger@rivosinc.com>
Co-developed-by: Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>
Signed-off-by: Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>
Co-developed-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
Signed-off-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
---
 tools/testing/selftests/riscv/Makefile        |    2 +-
 tools/testing/selftests/riscv/sse/Makefile    |    5 +
 .../selftests/riscv/sse/module/Makefile       |   22 +
 .../riscv/sse/module/riscv_sse_test.c         | 1154 +++++++++++++++++
 .../selftests/riscv/sse/run_sse_test.sh       |   59 +
 5 files changed, 1241 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/riscv/sse/Makefile
 create mode 100644 tools/testing/selftests/riscv/sse/module/Makefile
 create mode 100644 tools/testing/selftests/riscv/sse/module/riscv_sse_test.c
 create mode 100644 tools/testing/selftests/riscv/sse/run_sse_test.sh

diff --git a/tools/testing/selftests/riscv/Makefile b/tools/testing/selftests/riscv/Makefile
index 5671b4405a12..43c7c8f97676 100644
--- a/tools/testing/selftests/riscv/Makefile
+++ b/tools/testing/selftests/riscv/Makefile
@@ -5,7 +5,7 @@
 ARCH ?= $(shell uname -m 2>/dev/null || echo not)
 
 ifneq (,$(filter $(ARCH),riscv))
-RISCV_SUBTARGETS ?= abi hwprobe mm sigreturn vector cfi
+RISCV_SUBTARGETS ?= abi hwprobe mm sigreturn vector cfi sse
 else
 RISCV_SUBTARGETS :=
 endif
diff --git a/tools/testing/selftests/riscv/sse/Makefile b/tools/testing/selftests/riscv/sse/Makefile
new file mode 100644
index 000000000000..7e2677fdce09
--- /dev/null
+++ b/tools/testing/selftests/riscv/sse/Makefile
@@ -0,0 +1,5 @@
+TEST_GEN_MODS_DIR := module
+
+TEST_PROGS := run_sse_test.sh
+
+include ../../lib.mk
diff --git a/tools/testing/selftests/riscv/sse/module/Makefile b/tools/testing/selftests/riscv/sse/module/Makefile
new file mode 100644
index 000000000000..eac4b1c6228b
--- /dev/null
+++ b/tools/testing/selftests/riscv/sse/module/Makefile
@@ -0,0 +1,22 @@
+ifneq ($(CONFIG_RISCV_SBI_SSE),)
+obj-m += riscv_sse_test.o
+endif
+
+ifndef KERNELRELEASE
+
+TESTMODS_DIR := $(realpath $(dir $(abspath $(lastword $(MAKEFILE_LIST)))))
+KDIR ?= /lib/modules/$(shell uname -r)/build
+
+# Ensure that KDIR exists, otherwise skip the compilation
+modules:
+ifneq ("$(wildcard $(KDIR))", "")
+	$(Q)$(MAKE) -C $(KDIR) modules KBUILD_EXTMOD=$(TESTMODS_DIR)
+endif
+
+# Ensure that KDIR exists, otherwise skip the clean target
+clean:
+ifneq ("$(wildcard $(KDIR))", "")
+	$(Q)$(MAKE) -C $(KDIR) clean KBUILD_EXTMOD=$(TESTMODS_DIR)
+endif
+
+endif
diff --git a/tools/testing/selftests/riscv/sse/module/riscv_sse_test.c b/tools/testing/selftests/riscv/sse/module/riscv_sse_test.c
new file mode 100644
index 000000000000..cc5c2e46f2fd
--- /dev/null
+++ b/tools/testing/selftests/riscv/sse/module/riscv_sse_test.c
@@ -0,0 +1,1154 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2025 Rivos Inc.
+ */
+
+#define pr_fmt(fmt) "riscv_sse_test: " fmt
+
+#include <linux/array_size.h>
+#include <linux/atomic.h>
+#include <linux/completion.h>
+#include <linux/cpu.h>
+#include <linux/cpuhplock.h>
+#include <linux/delay.h>
+#include <linux/err.h>
+#include <linux/hrtimer.h>
+#include <linux/io.h>
+#include <linux/jiffies.h>
+#include <linux/kthread.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/riscv_sbi_sse.h>
+#include <linux/sched.h>
+#include <linux/slab.h>
+#include <linux/smp.h>
+#include <linux/workqueue.h>
+
+#include <asm/sbi.h>
+#include <asm/sse.h>
+
+#define RUN_LOOP_COUNT		1000
+#define SSE_FAILED_PREFIX	"FAILED: "
+#define SSE_SKIP_PREFIX		"SKIP: "
+#define STRESS_DURATION_MS	3000
+#define STRESS_INJECT_NS	10000
+#define STRESS_REINJECT_DEPTH	10
+#define sse_err(...)		pr_err(SSE_FAILED_PREFIX __VA_ARGS__)
+#define sse_skip(...)		pr_info(SSE_SKIP_PREFIX __VA_ARGS__)
+
+enum sse_stress_mode {
+	SSE_STRESS_OFF,
+	SSE_STRESS_AFTER_SMOKE,
+	SSE_STRESS_ONLY,
+};
+
+static int stress;
+module_param(stress, int, 0444);
+MODULE_PARM_DESC(stress, "Stress mode: 0=off, 1=after smoke, 2=stress only");
+
+static char *run_id = "unknown";
+module_param(run_id, charp, 0444);
+MODULE_PARM_DESC(run_id, "Unique identifier used to delimit one test run");
+
+/* Do not report PASS for a capability-only run that handled no event. */
+static atomic_t sse_test_handler_count = ATOMIC_INIT(0);
+static bool sse_stress_event_can_inject;
+
+struct sse_event_desc {
+	u32 evt_id;
+	const char *name;
+	bool can_inject;
+};
+
+static struct sse_event_desc sse_event_descs[] = {
+	{
+		.evt_id = SBI_SSE_EVENT_LOCAL_HIGH_PRIO_RAS,
+		.name = "local_high_prio_ras",
+	},
+	{
+		.evt_id = SBI_SSE_EVENT_LOCAL_DOUBLE_TRAP,
+		.name = "local_double_trap",
+	},
+	{
+		.evt_id = SBI_SSE_EVENT_GLOBAL_HIGH_PRIO_RAS,
+		.name = "global_high_prio_ras",
+	},
+	{
+		.evt_id = SBI_SSE_EVENT_LOCAL_PMU_OVERFLOW,
+		.name = "local_pmu_overflow",
+	},
+	{
+		.evt_id = SBI_SSE_EVENT_LOCAL_LOW_PRIO_RAS,
+		.name = "local_low_prio_ras",
+	},
+	{
+		.evt_id = SBI_SSE_EVENT_GLOBAL_LOW_PRIO_RAS,
+		.name = "global_low_prio_ras",
+	},
+	{
+		.evt_id = SBI_SSE_EVENT_LOCAL_SOFTWARE_INJECTED,
+		.name = "local_software_injected",
+	},
+	{
+		.evt_id = SBI_SSE_EVENT_GLOBAL_SOFTWARE_INJECTED,
+		.name = "global_software_injected",
+	}
+};
+
+static DEFINE_MUTEX(sse_test_cleanup_lock);
+/* Firmware permits only one registration for each event ID. */
+static struct sse_event *sse_test_cleanup_events[ARRAY_SIZE(sse_event_descs)];
+
+static void sse_test_cleanup_workfn(struct work_struct *work);
+static DECLARE_DELAYED_WORK(sse_test_cleanup_work, sse_test_cleanup_workfn);
+
+static void sse_test_queue_cleanup(struct sse_event *event)
+{
+	int i, free_slot = -1;
+
+	mutex_lock(&sse_test_cleanup_lock);
+	for (i = 0; i < ARRAY_SIZE(sse_test_cleanup_events); i++) {
+		if (sse_test_cleanup_events[i] == event)
+			goto out_schedule;
+		if (!sse_test_cleanup_events[i] && free_slot < 0)
+			free_slot = i;
+	}
+
+	if (WARN_ON_ONCE(free_slot < 0))
+		goto out_unlock;
+
+	sse_test_cleanup_events[free_slot] = event;
+
+out_schedule:
+	mod_delayed_work(system_wq, &sse_test_cleanup_work,
+			 msecs_to_jiffies(100));
+out_unlock:
+	mutex_unlock(&sse_test_cleanup_lock);
+}
+
+static struct sse_event_desc *sse_get_evt_desc(u32 evt)
+{
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(sse_event_descs); i++) {
+		if (sse_event_descs[i].evt_id == evt)
+			return &sse_event_descs[i];
+	}
+
+	return NULL;
+}
+
+static const char *sse_evt_name(u32 evt)
+{
+	struct sse_event_desc *desc = sse_get_evt_desc(evt);
+
+	return desc ? desc->name : NULL;
+}
+
+static bool sse_test_can_inject_event(u32 evt)
+{
+	struct sse_event_desc *desc = sse_get_evt_desc(evt);
+
+	return desc ? desc->can_inject : false;
+}
+
+/*
+ * Firmware can invoke the callback until unregister succeeds. Pin the module
+ * so its handler text cannot disappear first.
+ */
+static struct sse_event *sse_test_event_register(u32 evt, u32 priority,
+						 sse_event_handler_fn *handler,
+						 void *arg)
+{
+	struct sse_event *event;
+
+	event = sse_event_register(evt, priority, handler, arg);
+	if (!IS_ERR(event))
+		__module_get(THIS_MODULE);
+
+	return event;
+}
+
+static int sse_test_event_unregister(struct sse_event *event)
+{
+	int ret;
+
+	ret = sse_event_unregister(event);
+	if (!ret)
+		module_put(THIS_MODULE);
+	else
+		sse_test_queue_cleanup(event);
+
+	return ret;
+}
+
+static void sse_test_cleanup_workfn(struct work_struct *work)
+{
+	bool retry = false;
+	int i, ret;
+
+	mutex_lock(&sse_test_cleanup_lock);
+	for (i = 0; i < ARRAY_SIZE(sse_test_cleanup_events); i++) {
+		struct sse_event *event = sse_test_cleanup_events[i];
+
+		if (!event)
+			continue;
+
+		ret = sse_event_disable(event);
+		if (!ret)
+			ret = sse_event_unregister(event);
+		if (ret) {
+			retry = true;
+			continue;
+		}
+
+		sse_test_cleanup_events[i] = NULL;
+		module_put(THIS_MODULE);
+	}
+	mutex_unlock(&sse_test_cleanup_lock);
+
+	if (retry)
+		mod_delayed_work(system_wq, &sse_test_cleanup_work,
+				 msecs_to_jiffies(100));
+}
+
+static struct sbiret sbi_sse_ecall(int fid, unsigned long arg0, unsigned long arg1)
+{
+	return sbi_ecall(SBI_EXT_SSE, fid, arg0, arg1, 0, 0, 0, 0);
+}
+
+static int sse_event_attr_read(u32 evt, unsigned long attr_id,
+			       unsigned long *attr_buf)
+{
+	struct sbiret sret;
+	phys_addr_t phys;
+
+	phys = virt_to_phys(attr_buf);
+
+	sret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_EVENT_ATTR_READ, evt, attr_id, 1,
+			 (unsigned long)phys, 0, 0);
+	if (sret.error)
+		return sbi_err_map_linux_errno(sret.error);
+
+	return 0;
+}
+
+static int sse_event_attr_get(u32 evt, unsigned long attr_id,
+			      unsigned long *val)
+{
+	unsigned long *attr_buf;
+	int ret;
+
+	attr_buf = kmalloc_obj(*attr_buf, GFP_KERNEL);
+	if (!attr_buf)
+		return -ENOMEM;
+
+	ret = sse_event_attr_read(evt, attr_id, attr_buf);
+	if (!ret)
+		*val = *attr_buf;
+	kfree(attr_buf);
+
+	return ret;
+}
+
+static int sse_test_signal(u32 evt, unsigned int cpu)
+{
+	unsigned long hart_id = cpuid_to_hartid_map(cpu);
+	struct sbiret ret;
+
+	ret = sbi_sse_ecall(SBI_SSE_EVENT_INJECT, evt, hart_id);
+	if (ret.error) {
+		sse_err("Failed to signal event %x, error %ld\n", evt, ret.error);
+		return sbi_err_map_linux_errno(ret.error);
+	}
+
+	return 0;
+}
+
+static int sse_test_wait_not_running(u32 evt)
+{
+	unsigned long timeout = jiffies + HZ;
+	unsigned long status;
+	int ret;
+
+	do {
+		ret = sse_event_attr_get(evt, SBI_SSE_ATTR_STATUS, &status);
+		if (ret) {
+			sse_err("Failed to get status for evt %x, error %d\n", evt, ret);
+			return ret;
+		}
+		status &= SBI_SSE_ATTR_STATUS_STATE_MASK;
+		cpu_relax();
+	} while (status == SBI_SSE_STATE_RUNNING && time_before(jiffies, timeout));
+
+	if (status == SBI_SSE_STATE_RUNNING) {
+		sse_err("Timed out waiting for event %x to leave RUNNING state\n", evt);
+		return -ETIMEDOUT;
+	}
+
+	return 0;
+}
+
+struct sse_test_wait_status {
+	u32 evt;
+	unsigned long *attr_buf;
+	unsigned long status;
+	int ret;
+};
+
+static void sse_test_read_status_local(void *info)
+{
+	struct sse_test_wait_status *wait = info;
+
+	wait->ret = sse_event_attr_read(wait->evt, SBI_SSE_ATTR_STATUS,
+					wait->attr_buf);
+	if (!wait->ret)
+		wait->status = *wait->attr_buf & SBI_SSE_ATTR_STATUS_STATE_MASK;
+}
+
+static int sse_test_wait_not_running_on_cpu(u32 evt, unsigned int cpu)
+{
+	struct sse_test_wait_status wait = { .evt = evt };
+	unsigned long timeout;
+	int ret = 0;
+
+	if (sse_event_is_global(evt))
+		return sse_test_wait_not_running(evt);
+
+	wait.attr_buf = kmalloc_obj(*wait.attr_buf, GFP_KERNEL);
+	if (!wait.attr_buf)
+		return -ENOMEM;
+
+	timeout = jiffies + HZ;
+	do {
+		ret = smp_call_function_single(cpu, sse_test_read_status_local,
+					       &wait, true);
+		if (ret || wait.ret) {
+			ret = ret ?: wait.ret;
+			break;
+		}
+		if (wait.status != SBI_SSE_STATE_RUNNING)
+			break;
+		usleep_range(100, 200);
+	} while (time_before(jiffies, timeout));
+
+	if (!ret && wait.status == SBI_SSE_STATE_RUNNING) {
+		sse_err("Timed out waiting for event %x on CPU %u\n", evt, cpu);
+		ret = -ETIMEDOUT;
+	}
+
+	kfree(wait.attr_buf);
+
+	return ret;
+}
+
+static int sse_test_inject_event(struct sse_event *event, u32 evt, unsigned int cpu)
+{
+	int res;
+
+	if (sse_event_is_global(evt)) {
+		/*
+		 * Due to the fact the completion might happen faster than
+		 * the call to SBI_SSE_COMPLETE in the handler, if the event was
+		 * running on another CPU, we need to wait for the event status
+		 * to be !RUNNING.
+		 */
+		res = sse_test_wait_not_running(evt);
+		if (res)
+			return res;
+
+		res = sse_event_set_target_cpu(event, cpu);
+		if (res) {
+			sse_err("Failed to set cpu for evt %x, error %d\n", evt, res);
+			return res;
+		}
+	}
+
+	return sse_test_signal(evt, cpu);
+}
+
+struct fast_test_arg {
+	u32 evt;
+	int cpu;
+	bool args_ready;
+	bool completion;
+};
+
+/* A failed unregister may leave firmware holding this handler argument. */
+static struct fast_test_arg fast_test_arg;
+
+static int sse_test_handler(u32 evt, void *arg, struct pt_regs *regs)
+{
+	int ret = 0;
+	struct fast_test_arg *targ = arg;
+	u32 test_evt;
+	int cpu;
+
+	atomic_inc(&sse_test_handler_count);
+
+	/* Pairs with the argument publication in sse_run_fast_test_cpu(). */
+	if (!smp_load_acquire(&targ->args_ready)) {
+		sse_err("Received SSE event %x before its test arguments were published\n",
+			evt);
+		ret = -EINVAL;
+		goto complete;
+	}
+
+	test_evt = READ_ONCE(targ->evt);
+	cpu = READ_ONCE(targ->cpu);
+
+	if (evt != test_evt) {
+		sse_err("Received SSE event id %x instead of %x\n", test_evt, evt);
+		ret = -EINVAL;
+	}
+
+	if (!sse_event_is_global(evt) && cpu != smp_processor_id()) {
+		sse_err("Received SSE event %d on CPU %d instead of %d\n", evt, smp_processor_id(),
+			cpu);
+		ret = -EINVAL;
+	}
+
+complete:
+	WRITE_ONCE(targ->args_ready, false);
+	/* Publish handler-side checks before waking the waiting CPU. */
+	smp_store_release(&targ->completion, true);
+
+	return ret;
+}
+
+static int sse_run_fast_test_cpu(struct fast_test_arg *test_arg,
+				 struct sse_event *event, u32 evt, int cpu)
+{
+	unsigned long timeout;
+	int ret;
+
+	WRITE_ONCE(test_arg->completion, false);
+	WRITE_ONCE(test_arg->args_ready, false);
+	WRITE_ONCE(test_arg->evt, evt);
+	WRITE_ONCE(test_arg->cpu, cpu);
+	/* Publish all arguments before firmware can inject on another hart. */
+	smp_store_release(&test_arg->args_ready, true);
+
+	ret = sse_test_inject_event(event, evt, cpu);
+	if (ret) {
+		sse_err("event %s injection failed, err %d\n",
+			sse_evt_name(evt), ret);
+		return ret;
+	}
+
+	timeout = jiffies + HZ / 100;
+	/* We can not use <linux/completion.h> since they are not NMI safe */
+	/* Pairs with the handler's completion publication. */
+	while (!smp_load_acquire(&test_arg->completion) &&
+	       time_before(jiffies, timeout))
+		cpu_relax();
+	/* Acquire the handler's checks even if the loop observed a timeout. */
+	if (!smp_load_acquire(&test_arg->completion)) {
+		sse_err("Failed to wait for event %s completion on CPU %d\n",
+			sse_evt_name(evt), cpu);
+		return -ETIMEDOUT;
+	}
+
+	return sse_test_wait_not_running_on_cpu(evt, cpu);
+}
+
+static void sse_run_fast_test(struct fast_test_arg *test_arg,
+			      struct sse_event *event, u32 evt)
+{
+	int cpu;
+
+	if (sse_event_is_global(evt)) {
+		/* Keep the selected target online through injection and completion. */
+		cpu_hotplug_disable();
+		for_each_online_cpu(cpu) {
+			if (sse_run_fast_test_cpu(test_arg, event, evt, cpu))
+				break;
+		}
+		cpu_hotplug_enable();
+		return;
+	}
+
+	guard(cpus_read_lock)();
+	for_each_online_cpu(cpu) {
+		if (sse_run_fast_test_cpu(test_arg, event, evt, cpu))
+			return;
+	}
+}
+
+static void sse_test_injection_fast(void)
+{
+	int i, ret = 0, j;
+	u32 evt;
+	struct sse_event *event;
+
+	pr_info("Starting SSE test (fast)\n");
+
+	for (i = 0; i < ARRAY_SIZE(sse_event_descs); i++) {
+		evt = sse_event_descs[i].evt_id;
+		WRITE_ONCE(fast_test_arg.evt, evt);
+		WRITE_ONCE(fast_test_arg.cpu, -1);
+		WRITE_ONCE(fast_test_arg.args_ready, false);
+		WRITE_ONCE(fast_test_arg.completion, false);
+
+		if (!sse_event_descs[i].can_inject)
+			continue;
+
+		event = sse_test_event_register(evt, 0, sse_test_handler,
+						(void *)&fast_test_arg);
+		if (IS_ERR(event)) {
+			if (PTR_ERR(event) == -EEXIST) {
+				pr_info("Event %s already registered, skipping\n",
+					sse_evt_name(evt));
+				continue;
+			}
+			sse_err("Failed to register event %s, err %ld\n", sse_evt_name(evt),
+				PTR_ERR(event));
+			continue;
+		}
+
+		ret = sse_event_enable(event);
+		if (ret) {
+			sse_err("Failed to enable event %s, err %d\n", sse_evt_name(evt), ret);
+			goto err_disable;
+		}
+
+		pr_info("Starting testing event %s\n", sse_evt_name(evt));
+
+		for (j = 0; j < RUN_LOOP_COUNT; j++)
+			sse_run_fast_test(&fast_test_arg, event, evt);
+		pr_info("Finished testing event %s\n", sse_evt_name(evt));
+
+err_disable:
+		ret = sse_event_disable(event);
+		if (ret)
+			sse_err("Failed to disable event %s, err %d\n",
+				sse_evt_name(evt), ret);
+		ret = sse_test_event_unregister(event);
+		if (ret) {
+			sse_err("Failed to unregister event %s, err %d\n",
+				sse_evt_name(evt), ret);
+			return;
+		}
+	}
+	pr_info("Finished SSE test (fast)\n");
+}
+
+struct priority_test_arg {
+	unsigned long evt;
+	struct sse_event *event;
+	bool called;
+	bool enable_attempted;
+	u32 prio;
+	struct priority_test_arg *next_evt_arg;
+	void (*check_func)(struct priority_test_arg *arg);
+};
+
+/* A failed unregister may leave firmware holding these handler arguments. */
+static struct priority_test_arg default_hi_prio_args[] = {
+	{ .evt = SBI_SSE_EVENT_GLOBAL_SOFTWARE_INJECTED },
+	{ .evt = SBI_SSE_EVENT_LOCAL_SOFTWARE_INJECTED },
+	{ .evt = SBI_SSE_EVENT_GLOBAL_LOW_PRIO_RAS },
+	{ .evt = SBI_SSE_EVENT_LOCAL_LOW_PRIO_RAS },
+	{ .evt = SBI_SSE_EVENT_LOCAL_PMU_OVERFLOW },
+	{ .evt = SBI_SSE_EVENT_GLOBAL_HIGH_PRIO_RAS },
+	{ .evt = SBI_SSE_EVENT_LOCAL_DOUBLE_TRAP },
+	{ .evt = SBI_SSE_EVENT_LOCAL_HIGH_PRIO_RAS },
+};
+
+static struct priority_test_arg default_low_prio_args[] = {
+	{ .evt = SBI_SSE_EVENT_LOCAL_HIGH_PRIO_RAS },
+	{ .evt = SBI_SSE_EVENT_LOCAL_DOUBLE_TRAP },
+	{ .evt = SBI_SSE_EVENT_GLOBAL_HIGH_PRIO_RAS },
+	{ .evt = SBI_SSE_EVENT_LOCAL_PMU_OVERFLOW },
+	{ .evt = SBI_SSE_EVENT_LOCAL_LOW_PRIO_RAS },
+	{ .evt = SBI_SSE_EVENT_GLOBAL_LOW_PRIO_RAS },
+	{ .evt = SBI_SSE_EVENT_LOCAL_SOFTWARE_INJECTED },
+	{ .evt = SBI_SSE_EVENT_GLOBAL_SOFTWARE_INJECTED },
+};
+
+static struct priority_test_arg set_prio_args[] = {
+	{ .evt = SBI_SSE_EVENT_GLOBAL_SOFTWARE_INJECTED, .prio = 5 },
+	{ .evt = SBI_SSE_EVENT_LOCAL_SOFTWARE_INJECTED, .prio = 10 },
+	{ .evt = SBI_SSE_EVENT_GLOBAL_LOW_PRIO_RAS, .prio = 15 },
+	{ .evt = SBI_SSE_EVENT_LOCAL_LOW_PRIO_RAS, .prio = 20 },
+	{ .evt = SBI_SSE_EVENT_LOCAL_PMU_OVERFLOW, .prio = 25 },
+	{ .evt = SBI_SSE_EVENT_GLOBAL_HIGH_PRIO_RAS, .prio = 30 },
+	{ .evt = SBI_SSE_EVENT_LOCAL_DOUBLE_TRAP, .prio = 35 },
+	{ .evt = SBI_SSE_EVENT_LOCAL_HIGH_PRIO_RAS, .prio = 40 },
+};
+
+static struct priority_test_arg same_prio_args[] = {
+	{ .evt = SBI_SSE_EVENT_LOCAL_PMU_OVERFLOW, .prio = 0 },
+	{ .evt = SBI_SSE_EVENT_LOCAL_HIGH_PRIO_RAS, .prio = 10 },
+	{ .evt = SBI_SSE_EVENT_LOCAL_SOFTWARE_INJECTED, .prio = 10 },
+	{ .evt = SBI_SSE_EVENT_GLOBAL_SOFTWARE_INJECTED, .prio = 10 },
+	{ .evt = SBI_SSE_EVENT_GLOBAL_HIGH_PRIO_RAS, .prio = 20 },
+};
+
+static int sse_hi_priority_test_handler(u32 evt, void *arg,
+					struct pt_regs *regs)
+{
+	struct priority_test_arg *targ = arg;
+	struct priority_test_arg *next = READ_ONCE(targ->next_evt_arg);
+
+	atomic_inc(&sse_test_handler_count);
+	WRITE_ONCE(targ->called, 1);
+
+	if (next) {
+		sse_test_signal(next->evt, smp_processor_id());
+		if (!READ_ONCE(next->called)) {
+			sse_err("Higher priority event %s was not handled %s\n",
+				sse_evt_name(next->evt), sse_evt_name(evt));
+		}
+	}
+
+	return 0;
+}
+
+static int sse_low_priority_test_handler(u32 evt, void *arg, struct pt_regs *regs)
+{
+	struct priority_test_arg *targ = arg;
+	struct priority_test_arg *next = READ_ONCE(targ->next_evt_arg);
+
+	atomic_inc(&sse_test_handler_count);
+	WRITE_ONCE(targ->called, 1);
+
+	if (next) {
+		sse_test_signal(next->evt, smp_processor_id());
+		if (READ_ONCE(next->called)) {
+			sse_err("Lower priority event %s was handle before %s\n",
+				sse_evt_name(next->evt), sse_evt_name(evt));
+		}
+	}
+
+	return 0;
+}
+
+static void sse_test_injection_priority_arg(struct priority_test_arg *args, unsigned int args_size,
+					    sse_event_handler_fn handler, const char *test_name)
+{
+	unsigned int i;
+	unsigned long timeout;
+	int ret;
+	int target_cpu;
+	struct sse_event *event;
+	struct priority_test_arg *arg, *first_arg = NULL, *prev_arg = NULL;
+
+	pr_info("Starting SSE priority test (%s)\n", test_name);
+	/* Keep the complete priority chain on one CPU. */
+	migrate_disable();
+	target_cpu = smp_processor_id();
+
+	for (i = 0; i < args_size; i++) {
+		arg = &args[i];
+
+		if (!sse_test_can_inject_event(arg->evt))
+			continue;
+
+		WRITE_ONCE(arg->called, false);
+		WRITE_ONCE(arg->next_evt_arg, NULL);
+		WRITE_ONCE(arg->event, NULL);
+		WRITE_ONCE(arg->enable_attempted, false);
+
+		event = sse_test_event_register(arg->evt, arg->prio, handler,
+						(void *)arg);
+		if (IS_ERR(event)) {
+			if (PTR_ERR(event) == -EEXIST) {
+				pr_info("Event %s already registered, skipping\n",
+					sse_evt_name(arg->evt));
+				continue;
+			}
+			sse_err("Failed to register event %s, err %ld\n", sse_evt_name(arg->evt),
+				PTR_ERR(event));
+			goto release_events;
+		}
+		arg->event = event;
+
+		if (sse_event_is_global(arg->evt)) {
+			/* Keep the chain on one stable CPU. */
+			ret = sse_event_set_target_cpu(event, target_cpu);
+			if (ret) {
+				sse_err("Failed to set event %s target CPU, err %d\n",
+					sse_evt_name(arg->evt), ret);
+				goto release_events;
+			}
+		}
+
+		WRITE_ONCE(arg->enable_attempted, true);
+		ret = sse_event_enable(event);
+		if (ret) {
+			sse_err("Failed to enable event %s, err %d\n", sse_evt_name(arg->evt), ret);
+			goto release_events;
+		}
+
+		if (prev_arg)
+			WRITE_ONCE(prev_arg->next_evt_arg, arg);
+
+		prev_arg = arg;
+
+		if (!first_arg)
+			first_arg = arg;
+	}
+
+	if (!first_arg) {
+		pr_info("No injectable event available for %s priority test\n",
+			test_name);
+		goto out;
+	}
+
+	/* Inject first event, handler should trigger the others in chain. */
+	ret = sse_test_inject_event(first_arg->event, first_arg->evt, target_cpu);
+	if (ret) {
+		sse_err("SSE event %s injection failed\n", sse_evt_name(first_arg->evt));
+		goto release_events;
+	}
+
+	/* Lower-priority events run after the handler that injected them completes. */
+	arg = first_arg;
+	while (arg) {
+		timeout = jiffies + HZ;
+		while (!READ_ONCE(arg->called) && time_before(jiffies, timeout))
+			cpu_relax();
+
+		if (!READ_ONCE(arg->called)) {
+			sse_err("Event %s handler was not called\n",
+				sse_evt_name(arg->evt));
+			ret = -EINVAL;
+		}
+
+		event = arg->event;
+		arg = READ_ONCE(arg->next_evt_arg);
+	}
+
+release_events:
+
+	for (i = 0; i < args_size; i++) {
+		arg = &args[i];
+		event = arg->event;
+		if (!event)
+			continue;
+
+		ret = sse_test_wait_not_running_on_cpu(arg->evt, target_cpu);
+		if (ret)
+			sse_err("Event %s did not complete, err %d\n",
+				sse_evt_name(arg->evt), ret);
+
+		if (arg->enable_attempted) {
+			ret = sse_event_disable(event);
+			if (ret) {
+				sse_err("Failed to disable event %s, err %d\n",
+					sse_evt_name(arg->evt), ret);
+				sse_test_queue_cleanup(event);
+				WRITE_ONCE(arg->event, NULL);
+				WRITE_ONCE(arg->enable_attempted, false);
+				continue;
+			}
+		}
+
+		ret = sse_test_event_unregister(event);
+		if (ret) {
+			sse_err("Failed to unregister event %s, err %d\n",
+				sse_evt_name(arg->evt), ret);
+			continue;
+		}
+
+		WRITE_ONCE(arg->event, NULL);
+		WRITE_ONCE(arg->enable_attempted, false);
+	}
+
+	pr_info("Finished SSE priority test (%s)\n", test_name);
+out:
+	migrate_enable();
+}
+
+static void sse_test_injection_priority(void)
+{
+	sse_test_injection_priority_arg(default_hi_prio_args, ARRAY_SIZE(default_hi_prio_args),
+					sse_hi_priority_test_handler, "high");
+
+	sse_test_injection_priority_arg(default_low_prio_args, ARRAY_SIZE(default_low_prio_args),
+					sse_low_priority_test_handler, "low");
+
+	sse_test_injection_priority_arg(set_prio_args, ARRAY_SIZE(set_prio_args),
+					sse_low_priority_test_handler, "set");
+
+	sse_test_injection_priority_arg(same_prio_args, ARRAY_SIZE(same_prio_args),
+					sse_low_priority_test_handler, "same_prio_args");
+}
+
+static int sse_get_inject_status(u32 evt, bool *can_inject)
+{
+	int ret;
+	unsigned long val;
+
+	/* Check if injection is supported */
+	ret = sse_event_attr_get(evt, SBI_SSE_ATTR_STATUS, &val);
+	if (ret == sbi_err_map_linux_errno(SBI_ERR_NOT_SUPPORTED) ||
+	    ret == sbi_err_map_linux_errno(SBI_ERR_INVALID_PARAM)) {
+		*can_inject = false;
+		return 0;
+	}
+	if (ret)
+		return ret;
+
+	*can_inject = !!(val & BIT(SBI_SSE_ATTR_STATUS_INJECT_OFFSET));
+
+	return 0;
+}
+
+static int sse_init_events(void)
+{
+	int i, injectable = 0, ret;
+
+	for (i = 0; i < ARRAY_SIZE(sse_event_descs); i++) {
+		struct sse_event_desc *desc = &sse_event_descs[i];
+
+		ret = sse_get_inject_status(desc->evt_id, &desc->can_inject);
+		if (ret) {
+			sse_err("Failed to read injection status for %s, err %d\n",
+				desc->name, ret);
+			return ret;
+		}
+
+		if (desc->can_inject)
+			injectable++;
+		else
+			pr_info("Can not inject event %s, tests using this event will be skipped\n",
+				desc->name);
+
+		if (desc->evt_id == SBI_SSE_EVENT_LOCAL_SOFTWARE_INJECTED)
+			sse_stress_event_can_inject = desc->can_inject;
+	}
+
+	return injectable;
+}
+
+struct stress_test_ctx {
+	struct sse_event *event;
+	struct hrtimer timer;
+	struct hrtimer stop_timer;
+	struct task_struct *monitor_task;
+	wait_queue_head_t wait_q;
+	atomic_t inject_count;
+	atomic_t handler_count;
+	atomic_t handler_errors;
+	u32 evt_id;
+	int layer;
+	bool running;
+	bool test_done;
+};
+
+static struct stress_test_ctx stress_ctx;
+static DEFINE_PER_CPU(int, stress_reinject_cpu_depth);
+
+static int stress_handler_empty(u32 evt, void *arg, struct pt_regs *regs)
+{
+	struct stress_test_ctx *ctx = arg;
+
+	atomic_inc(&sse_test_handler_count);
+	atomic_inc(&ctx->handler_count);
+
+	return 0;
+}
+
+static int stress_handler_ecall(u32 evt, void *arg, struct pt_regs *regs)
+{
+	struct stress_test_ctx *ctx = arg;
+	struct sbiret ret;
+
+	ret = sbi_ecall(SBI_EXT_BASE, SBI_EXT_BASE_GET_SPEC_VERSION,
+			0, 0, 0, 0, 0, 0);
+	if (ret.error)
+		atomic_inc(&ctx->handler_errors);
+	atomic_inc(&sse_test_handler_count);
+	atomic_inc(&ctx->handler_count);
+
+	return 0;
+}
+
+static int stress_handler_multi_ecall(u32 evt, void *arg, struct pt_regs *regs)
+{
+	struct stress_test_ctx *ctx = arg;
+	struct sbiret ret;
+
+	ret = sbi_ecall(SBI_EXT_BASE, SBI_EXT_BASE_GET_SPEC_VERSION,
+			0, 0, 0, 0, 0, 0);
+	if (ret.error)
+		atomic_inc(&ctx->handler_errors);
+	ret = sbi_ecall(SBI_EXT_BASE, SBI_EXT_BASE_GET_IMP_ID,
+			0, 0, 0, 0, 0, 0);
+	if (ret.error)
+		atomic_inc(&ctx->handler_errors);
+	ret = sbi_ecall(SBI_EXT_BASE, SBI_EXT_BASE_GET_IMP_VERSION,
+			0, 0, 0, 0, 0, 0);
+	if (ret.error)
+		atomic_inc(&ctx->handler_errors);
+	atomic_inc(&sse_test_handler_count);
+	atomic_inc(&ctx->handler_count);
+
+	return 0;
+}
+
+static int stress_handler_reinject(u32 evt, void *arg, struct pt_regs *regs)
+{
+	struct stress_test_ctx *ctx = arg;
+	int *depth = this_cpu_ptr(&stress_reinject_cpu_depth);
+
+	(*depth)++;
+	if (*depth < STRESS_REINJECT_DEPTH)
+		sse_test_signal(evt, smp_processor_id());
+	else
+		*depth = 0;
+
+	atomic_inc(&sse_test_handler_count);
+	atomic_inc(&ctx->handler_count);
+
+	return 0;
+}
+
+static sse_event_handler_fn *stress_handlers[] = {
+	stress_handler_empty,
+	stress_handler_ecall,
+	stress_handler_multi_ecall,
+	stress_handler_reinject,
+};
+
+static const char * const stress_layer_names[] = {
+	"empty handler",
+	"single SBI ecall in handler",
+	"multiple SBI ecalls in handler",
+	"self re-inject",
+};
+
+static enum hrtimer_restart stress_timer_callback(struct hrtimer *timer)
+{
+	struct stress_test_ctx *ctx = container_of(timer, struct stress_test_ctx, timer);
+
+	if (!READ_ONCE(ctx->running))
+		return HRTIMER_NORESTART;
+
+	if (!sse_test_signal(ctx->evt_id, smp_processor_id()))
+		atomic_inc(&ctx->inject_count);
+	hrtimer_forward_now(timer, ns_to_ktime(STRESS_INJECT_NS));
+
+	return HRTIMER_RESTART;
+}
+
+static enum hrtimer_restart stress_stop_timer_callback(struct hrtimer *timer)
+{
+	struct stress_test_ctx *ctx;
+
+	ctx = container_of(timer, struct stress_test_ctx, stop_timer);
+	WRITE_ONCE(ctx->test_done, true);
+	wake_up(&ctx->wait_q);
+
+	return HRTIMER_NORESTART;
+}
+
+static int stress_monitor_thread(void *data)
+{
+	struct stress_test_ctx *ctx = data;
+	unsigned long last_inject = 0, last_handler = 0;
+
+	while (!kthread_should_stop()) {
+		unsigned long inject = atomic_read(&ctx->inject_count);
+		unsigned long handler = atomic_read(&ctx->handler_count);
+
+		pr_info("stress layer %d: inject=%lu (+%lu), handler=%lu (+%lu)\n",
+			ctx->layer, inject, inject - last_inject,
+			handler, handler - last_handler);
+
+		last_inject = inject;
+		last_handler = handler;
+
+		schedule_timeout_interruptible(HZ);
+	}
+
+	return 0;
+}
+
+static int sse_stress_test_layer(int layer)
+{
+	struct sse_event *event;
+	int inject_count, handler_count;
+	int ret, target_cpu, unregister_ret;
+
+	if (layer < 0 || layer >= ARRAY_SIZE(stress_handlers))
+		return -EINVAL;
+
+	pr_info("Starting SSE stress layer %d (%s)\n",
+		layer, stress_layer_names[layer]);
+
+	memset(&stress_ctx, 0, sizeof(stress_ctx));
+	stress_ctx.evt_id = SBI_SSE_EVENT_LOCAL_SOFTWARE_INJECTED;
+	stress_ctx.layer = layer;
+	WRITE_ONCE(stress_ctx.running, true);
+	atomic_set(&stress_ctx.inject_count, 0);
+	atomic_set(&stress_ctx.handler_count, 0);
+	atomic_set(&stress_ctx.handler_errors, 0);
+	init_waitqueue_head(&stress_ctx.wait_q);
+
+	event = sse_test_event_register(stress_ctx.evt_id, 0,
+					stress_handlers[layer], &stress_ctx);
+	if (IS_ERR(event)) {
+		sse_err("Failed to register stress event, err %ld\n",
+			PTR_ERR(event));
+		return PTR_ERR(event);
+	}
+
+	stress_ctx.event = event;
+
+	ret = sse_event_enable(event);
+	if (ret) {
+		sse_err("Failed to enable stress event, err %d\n", ret);
+		goto err_disable;
+	}
+
+	stress_ctx.monitor_task = kthread_run(stress_monitor_thread,
+					      &stress_ctx, "sse_stress_mon");
+	if (IS_ERR(stress_ctx.monitor_task)) {
+		ret = PTR_ERR(stress_ctx.monitor_task);
+		sse_err("Failed to create stress monitor thread, err %d\n", ret);
+		goto err_disable;
+	}
+
+	/* Keep the pinned timer and its local event on the selected CPU. */
+	cpus_read_lock();
+	migrate_disable();
+	target_cpu = smp_processor_id();
+	hrtimer_setup(&stress_ctx.timer, stress_timer_callback,
+		      CLOCK_MONOTONIC, HRTIMER_MODE_PINNED);
+	hrtimer_start(&stress_ctx.timer, ns_to_ktime(STRESS_INJECT_NS),
+		      HRTIMER_MODE_REL_PINNED);
+	migrate_enable();
+
+	hrtimer_setup(&stress_ctx.stop_timer, stress_stop_timer_callback,
+		      CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+	hrtimer_start(&stress_ctx.stop_timer, ms_to_ktime(STRESS_DURATION_MS),
+		      HRTIMER_MODE_REL);
+
+	wait_event(stress_ctx.wait_q, READ_ONCE(stress_ctx.test_done));
+
+	WRITE_ONCE(stress_ctx.running, false);
+	hrtimer_cancel(&stress_ctx.timer);
+	hrtimer_cancel(&stress_ctx.stop_timer);
+	kthread_stop(stress_ctx.monitor_task);
+
+	pr_info("Finished SSE stress layer %d (%s): inject=%d, handler=%d\n",
+		layer, stress_layer_names[layer],
+		atomic_read(&stress_ctx.inject_count),
+		atomic_read(&stress_ctx.handler_count));
+
+	inject_count = atomic_read(&stress_ctx.inject_count);
+	handler_count = atomic_read(&stress_ctx.handler_count);
+	if (!inject_count || !handler_count) {
+		sse_err("Stress layer %d made no progress: inject=%d, handler=%d\n",
+			layer, inject_count, handler_count);
+		ret = -EIO;
+	}
+	if (atomic_read(&stress_ctx.handler_errors)) {
+		sse_err("Stress layer %d observed %d SBI call errors\n", layer,
+			atomic_read(&stress_ctx.handler_errors));
+		ret = -EIO;
+	}
+	if (sse_test_wait_not_running_on_cpu(stress_ctx.evt_id, target_cpu)) {
+		sse_err("Stress event did not complete on CPU %d\n", target_cpu);
+		if (!ret)
+			ret = -ETIMEDOUT;
+	}
+	cpus_read_unlock();
+
+err_disable:
+	if (sse_event_disable(event)) {
+		sse_err("Failed to disable stress event\n");
+		if (!ret)
+			ret = -EIO;
+	}
+	unregister_ret = sse_test_event_unregister(event);
+	if (unregister_ret) {
+		sse_err("Failed to unregister stress event\n");
+		if (!ret)
+			ret = unregister_ret;
+	}
+	stress_ctx.event = NULL;
+
+	return ret;
+}
+
+static void sse_stress_test_all_layers(void)
+{
+	int i, ret;
+
+	pr_info("Starting SSE stress tests: duration=%d ms, interval=%d ns\n",
+		STRESS_DURATION_MS, STRESS_INJECT_NS);
+
+	for (i = 0; i < ARRAY_SIZE(stress_handlers); i++) {
+		ret = sse_stress_test_layer(i);
+		if (ret) {
+			sse_err("Stress layer %d failed, err %d\n", i, ret);
+			break;
+		}
+
+		msleep(100);
+	}
+
+	pr_info("Finished SSE stress tests\n");
+}
+
+static int __init sse_test_init(void)
+{
+	int ret;
+
+	pr_info("RUN %s BEGIN\n", run_id);
+	atomic_set(&sse_test_handler_count, 0);
+
+	if (stress < SSE_STRESS_OFF || stress > SSE_STRESS_ONLY) {
+		sse_err("Invalid stress mode %d\n", stress);
+		pr_info("RUN %s END\n", run_id);
+		return -EINVAL;
+	}
+
+	ret = sse_init_events();
+	if (ret < 0) {
+		pr_info("RUN %s END\n", run_id);
+		return ret;
+	}
+	if (!ret) {
+		sse_skip("No injectable SSE event is available\n");
+		pr_info("RUN %s END\n", run_id);
+		return 0;
+	}
+	if (stress == SSE_STRESS_ONLY && !sse_stress_event_can_inject) {
+		sse_skip("Local software-injected event is unavailable for stress\n");
+		pr_info("RUN %s END\n", run_id);
+		return 0;
+	}
+
+	if (stress != SSE_STRESS_ONLY) {
+		sse_test_injection_fast();
+		sse_test_injection_priority();
+	}
+
+	if (stress == SSE_STRESS_AFTER_SMOKE && !sse_stress_event_can_inject)
+		sse_skip("Local software-injected event is unavailable for stress\n");
+	else if (stress != SSE_STRESS_OFF)
+		sse_stress_test_all_layers();
+	if (!atomic_read(&sse_test_handler_count))
+		sse_skip("No SSE event was handled\n");
+
+	pr_info("RUN %s END\n", run_id);
+
+	return 0;
+}
+
+static void __exit sse_test_exit(void)
+{
+	cancel_delayed_work_sync(&sse_test_cleanup_work);
+}
+
+module_init(sse_test_init);
+module_exit(sse_test_exit);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Clément Léger <cleger@rivosinc.com>");
+MODULE_DESCRIPTION("Test module for SSE");
diff --git a/tools/testing/selftests/riscv/sse/run_sse_test.sh b/tools/testing/selftests/riscv/sse/run_sse_test.sh
new file mode 100644
index 000000000000..e70a2fd14b05
--- /dev/null
+++ b/tools/testing/selftests/riscv/sse/run_sse_test.sh
@@ -0,0 +1,59 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (C) 2025 Rivos Inc.
+
+MODULE_NAME=riscv_sse_test
+DRIVER="./module/${MODULE_NAME}.ko"
+ksft_skip=4
+
+check_test_requirements()
+{
+	uid=$(id -u)
+	if [ $uid -ne 0 ]; then
+		echo "$0: Must be run as root"
+		exit $ksft_skip
+	fi
+
+	if ! which insmod > /dev/null 2>&1; then
+		echo "$0: You need insmod installed"
+		exit $ksft_skip
+	fi
+
+	if [ ! -f "$DRIVER" ]; then
+		echo "$0: SSE is disabled or ${MODULE_NAME} is not built"
+		exit $ksft_skip
+	fi
+}
+
+check_test_requirements
+run_id="$$-$(date +%s)"
+
+if ! insmod "$DRIVER" run_id="$run_id" "$@" > /dev/null 2>&1; then
+	echo "${MODULE_NAME}: failed to load, please check dmesg"
+	exit 1
+fi
+
+if ! rmmod "$MODULE_NAME"; then
+	echo "${MODULE_NAME}: failed to unload, please check dmesg"
+	exit 1
+fi
+
+run_log=$(dmesg | sed -n \
+	"/${MODULE_NAME}: RUN ${run_id} BEGIN/,/${MODULE_NAME}: RUN ${run_id} END/p")
+if [ -z "$run_log" ]; then
+	echo "${MODULE_NAME}: unable to find log for run ${run_id}"
+	exit 1
+fi
+
+if echo "$run_log" | grep -q "${MODULE_NAME}: FAILED:"; then
+	echo "${MODULE_NAME} failed, please check dmesg"
+	exit 1
+fi
+
+if echo "$run_log" | grep -q "${MODULE_NAME}: SKIP:"; then
+	echo "${MODULE_NAME}: no injectable SSE event"
+	exit $ksft_skip
+fi
+
+exit 0
-- 
2.50.1 (Apple Git-155)


^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH v13 11/11] selftests/riscv: add perf user-stack SSE copy regression test
  2026-11-19 12:08 [PATCH v13 00/11] riscv: add SBI Supervisor Software Events support Zhanpeng Zhang
                   ` (9 preceding siblings ...)
  2026-11-19 12:08 ` [PATCH v13 10/11] selftests/riscv: add SSE test module Zhanpeng Zhang
@ 2026-11-19 12:08 ` Zhanpeng Zhang
  2026-11-19 12:34   ` sashiko-bot
  10 siblings, 1 reply; 22+ messages in thread
From: Zhanpeng Zhang @ 2026-11-19 12:08 UTC (permalink / raw)
  To: pjw, palmer, aou, alex, himanshu.chauhan, conor, anup.patel
  Cc: clement.leger, cuiyunhui, atish.patra, peterz, mingo, acme,
	namhyung, mark.rutland, alexander.shishkin, jolsa, irogers,
	adrian.hunter, james.clark, will, tglx, corbet, rdunlap, shuah,
	yuanzhu, yang.susheng, linux-riscv, linux-kernel,
	linux-perf-users, linux-doc, linux-kselftest,
	zhangzhanpeng.jasper

On RISC-V, PMU overflow interrupts can be delivered through the SBI
Supervisor Software Events (SSE) mechanism. A perf event that samples
the raw user stack (PERF_SAMPLE_STACK_USER, as perf record
--call-graph dwarf does) then copies a chunk of the interrupted task's
user stack from an NMI-like context. Non-resident pages must truncate
that copy through exception-table fixup without entering normal
page-fault handling or disturbing the interrupted context. Exercise
this boundary and the perf record layout under SSE sampling.

The existing SSE selftest module exercises the framework (register,
enable, inject, complete, priorities, stress) but never drives the perf
user-stack copy that motivated the no-fault path. Add a userspace test
that closes that gap:

  - Sample a child whose stack ends at a PROT_NONE page using
    PERF_SAMPLE_STACK_USER. Check each sample record for a bounded dump
    size and require at least one truncated copy at that boundary.
  - Drive a multi-CPU unix-socket + deep-recursion workload under
    high-frequency raw user-stack sampling to exercise repeated entry,
    copying and completion under load.

The test reports SKIP when hardware PMU sampling is unavailable or
perf_event_paranoid forbids it, so it is safe to run unprivileged or in
constrained environments. It is placed under the RISC-V SSE selftests
because SSE delivery is the RISC-V-specific condition it protects, and
is wired into the sse subtarget Makefile alongside the module runner.

Signed-off-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
---
 tools/testing/selftests/riscv/sse/Makefile    |   5 +
 .../selftests/riscv/sse/sse_perf_ustack.c     | 564 ++++++++++++++++++
 2 files changed, 569 insertions(+)
 create mode 100644 tools/testing/selftests/riscv/sse/sse_perf_ustack.c

diff --git a/tools/testing/selftests/riscv/sse/Makefile b/tools/testing/selftests/riscv/sse/Makefile
index 7e2677fdce09..646b418de9ff 100644
--- a/tools/testing/selftests/riscv/sse/Makefile
+++ b/tools/testing/selftests/riscv/sse/Makefile
@@ -1,5 +1,10 @@
+CFLAGS += -I$(top_srcdir)/tools/testing/selftests
+LDLIBS += -lpthread
+
 TEST_GEN_MODS_DIR := module
 
+TEST_GEN_PROGS := sse_perf_ustack
+
 TEST_PROGS := run_sse_test.sh
 
 include ../../lib.mk
diff --git a/tools/testing/selftests/riscv/sse/sse_perf_ustack.c b/tools/testing/selftests/riscv/sse/sse_perf_ustack.c
new file mode 100644
index 000000000000..9535b6d7ba4c
--- /dev/null
+++ b/tools/testing/selftests/riscv/sse/sse_perf_ustack.c
@@ -0,0 +1,564 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Regression test for the RISC-V perf user-stack copy taken in SSE
+ * (NMI-like) context.
+ *
+ * On RISC-V, PMU overflow interrupts can be delivered through the SBI
+ * Supervisor Software Events (SSE) mechanism. A perf event that samples the
+ * raw user stack (PERF_SAMPLE_STACK_USER, as perf record --call-graph dwarf
+ * does) then copies a large chunk of the interrupted task's user stack from
+ * that context. If that copy is allowed to take a nested page fault it can
+ * corrupt the interrupted task's kernel state and hang or crash the machine
+ * under load.
+ *
+ * This test exercises that exact path:
+ *   - It opens a sampling hardware PMU event with PERF_SAMPLE_STACK_USER.
+ *   - It samples a child running on a controlled user stack followed by an
+ *     inaccessible page, so the copy must truncate at that page boundary.
+ *   - It checks that every user-stack sample record is well formed and that
+ *     the dumped size never exceeds the requested size (i.e. the copy stops
+ *     cleanly rather than faulting on).
+ *   - It then drives a multi-threaded unix-socket + deep-recursion workload
+ *     under high-frequency per-CPU sampling and requires every active sampler
+ *     to make progress without taking the machine down.
+ *
+ * The test is architecture independent in what it drives; it is placed under
+ * the RISC-V SSE selftests because SSE delivery is the RISC-V-specific
+ * condition it is meant to protect.
+ */
+#define _GNU_SOURCE
+
+#include <errno.h>
+#include <poll.h>
+#include <sched.h>
+#include <signal.h>
+#include <stdbool.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <time.h>
+#include <unistd.h>
+
+#include <linux/perf_event.h>
+#include <pthread.h>
+#include <sys/ioctl.h>
+#include <sys/mman.h>
+#include <sys/socket.h>
+#include <sys/syscall.h>
+#include <sys/wait.h>
+
+#include "../../kselftest.h"
+
+#ifndef noinline
+#define noinline __attribute__((noinline))
+#endif
+
+#define STACK_DUMP_SIZE		8192		/* 8 KiB, 8-byte aligned */
+#define RB_DATA_PAGES		64		/* power of two */
+#define SELF_SAMPLE_FREQ	4000
+#define STRESS_SAMPLE_FREQ	5000
+#define STRESS_SECONDS		5
+#define RECURSE_DEPTH		512
+#define TRUNCATION_RUN_MS	250
+
+static long page_size;
+
+static int perf_event_open(struct perf_event_attr *attr, pid_t pid, int cpu,
+			   int group_fd, unsigned long flags)
+{
+	return syscall(__NR_perf_event_open, attr, pid, cpu, group_fd, flags);
+}
+
+/* Prevent the compiler from optimizing away a stack buffer. */
+static void keep_alive(void *p)
+{
+	__asm__ __volatile__("" : : "r"(p) : "memory");
+}
+
+/*
+ * Consume a deep user stack and keep it live, so a raw user-stack sample has
+ * many pages to copy. Returns a value derived from the stack so the compiler
+ * cannot elide the frames.
+ */
+static noinline unsigned long burn_stack(int depth, unsigned long *sink)
+{
+	unsigned long frame[32];
+	unsigned int i;
+
+	for (i = 0; i < ARRAY_SIZE(frame); i++)
+		frame[i] = (unsigned long)depth * i + *sink;
+
+	if (depth > 0)
+		frame[depth & 31] += burn_stack(depth - 1, sink);
+
+	for (i = 0; i < ARRAY_SIZE(frame); i++)
+		*sink += frame[i];
+
+	keep_alive(frame);
+	return *sink;
+}
+
+static struct perf_event_attr sampling_attr(unsigned long freq)
+{
+	struct perf_event_attr attr = {
+		.type		= PERF_TYPE_HARDWARE,
+		.size		= sizeof(attr),
+		.config		= PERF_COUNT_HW_INSTRUCTIONS,
+		.sample_type	= PERF_SAMPLE_STACK_USER,
+		.sample_stack_user = STACK_DUMP_SIZE,
+		.freq		= 1,
+		.sample_freq	= freq,
+		.disabled	= 1,
+		.exclude_kernel	= 1,
+		.exclude_hv	= 1,
+	};
+
+	return attr;
+}
+
+static bool open_skip_reason(int err, const char **why)
+{
+	switch (err) {
+	case EACCES:
+	case EPERM:
+		*why = "insufficient privilege for PMU sampling (perf_event_paranoid)";
+		return true;
+	case ENOENT:
+	case ENODEV:
+	case EOPNOTSUPP:
+		*why = "hardware PMU sampling event not available";
+		return true;
+	default:
+		return false;
+	}
+}
+
+static bool pmu_sse_route_testable(const char **why)
+{
+	char *line = NULL;
+	size_t line_size = 0;
+	FILE *interrupts;
+	bool testable = true;
+
+	interrupts = fopen("/proc/interrupts", "re");
+	if (!interrupts) {
+		*why = "cannot inspect the active PMU delivery route";
+		return false;
+	}
+
+	/* The SBI PMU driver registers this name only for ordinary IRQ delivery. */
+	while (getline(&line, &line_size, interrupts) >= 0) {
+		if (strstr(line, "riscv-pmu")) {
+			*why = "ordinary RISC-V PMU IRQ delivery is active";
+			testable = false;
+			break;
+		}
+	}
+
+	free(line);
+	fclose(interrupts);
+	return testable;
+}
+
+static bool ring_copy_from(void *dst, const void *rb, size_t rb_bytes,
+			   uint64_t pos, size_t size)
+{
+	size_t offset = pos % rb_bytes;
+	size_t first;
+
+	if (size > rb_bytes)
+		return false;
+
+	first = size < rb_bytes - offset ? size : rb_bytes - offset;
+	memcpy(dst, (const char *)rb + offset, first);
+	if (first != size)
+		memcpy((char *)dst + first, rb, size - first);
+
+	return true;
+}
+
+static int truncation_child(void *arg)
+{
+	int ready_fd = (intptr_t)arg;
+	char ready = 1;
+
+	if (write(ready_fd, &ready, sizeof(ready)) != 1)
+		return 1;
+
+	for (;;)
+		__asm__ __volatile__("" : : : "memory");
+}
+
+static pid_t start_truncation_child(void **stack_mapping)
+{
+	struct pollfd pfd = { .events = POLLIN };
+	size_t mapping_size = 2 * page_size;
+	char ready;
+	void *stack;
+	pid_t pid;
+	int pipefd[2];
+	int saved_errno;
+
+	stack = mmap(NULL, mapping_size, PROT_NONE,
+		     MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
+	if (stack == MAP_FAILED)
+		return -1;
+	if (mprotect(stack, page_size, PROT_READ | PROT_WRITE))
+		goto err_unmap;
+	if (pipe(pipefd))
+		goto err_unmap;
+
+	/* clone() starts the child below the inaccessible second page. */
+	pid = clone(truncation_child, (char *)stack + page_size, SIGCHLD,
+		    (void *)(intptr_t)pipefd[1]);
+	if (pid < 0)
+		goto err_pipe;
+
+	close(pipefd[1]);
+	pfd.fd = pipefd[0];
+	if (poll(&pfd, 1, 1000) != 1 ||
+	    read(pipefd[0], &ready, sizeof(ready)) != sizeof(ready)) {
+		saved_errno = ETIMEDOUT;
+		kill(pid, SIGKILL);
+		waitpid(pid, NULL, 0);
+		close(pipefd[0]);
+		errno = saved_errno;
+		goto err_unmap;
+	}
+	close(pipefd[0]);
+
+	*stack_mapping = stack;
+	return pid;
+
+err_pipe:
+	saved_errno = errno;
+	close(pipefd[0]);
+	close(pipefd[1]);
+	errno = saved_errno;
+err_unmap:
+	saved_errno = errno;
+	munmap(stack, mapping_size);
+	errno = saved_errno;
+	return -1;
+}
+
+static void stop_truncation_child(pid_t pid, void *stack_mapping)
+{
+	kill(pid, SIGKILL);
+	while (waitpid(pid, NULL, 0) < 0 && errno == EINTR)
+		;
+	munmap(stack_mapping, 2 * page_size);
+}
+
+/*
+ * Subtest 1: sample a child whose stack is followed by an inaccessible page.
+ * Every record must be well formed and at least one stack copy must truncate
+ * at the controlled page boundary rather than fault or overrun.
+ */
+static void test_ustack_records_wellformed(void)
+{
+	struct perf_event_attr attr = sampling_attr(SELF_SAMPLE_FREQ);
+	size_t rb_bytes = (size_t)RB_DATA_PAGES * page_size;
+	struct perf_event_mmap_page *meta;
+	unsigned long samples = 0, truncated = 0;
+	void *child_stack;
+	const char *why;
+	void *rb;
+	pid_t child;
+	int fd;
+
+	child = start_truncation_child(&child_stack);
+	if (child < 0) {
+		ksft_test_result_fail("ustack records: create guarded stack child: %s\n",
+				      strerror(errno));
+		return;
+	}
+
+	fd = perf_event_open(&attr, child, -1, -1, PERF_FLAG_FD_CLOEXEC);
+	if (fd < 0) {
+		if (open_skip_reason(errno, &why))
+			ksft_test_result_skip("ustack records: %s\n", why);
+		else
+			ksft_test_result_fail("ustack records: perf_event_open: %s\n",
+					      strerror(errno));
+		goto out_child;
+	}
+
+	meta = mmap(NULL, page_size + rb_bytes, PROT_READ | PROT_WRITE,
+		    MAP_SHARED, fd, 0);
+	if (meta == MAP_FAILED) {
+		ksft_test_result_fail("ustack records: mmap ring buffer: %s\n",
+				      strerror(errno));
+		close(fd);
+		goto out_child;
+	}
+	rb = (char *)meta + page_size;
+
+	ioctl(fd, PERF_EVENT_IOC_RESET, 0);
+	ioctl(fd, PERF_EVENT_IOC_ENABLE, 0);
+	usleep(TRUNCATION_RUN_MS * 1000);
+	ioctl(fd, PERF_EVENT_IOC_DISABLE, 0);
+
+	/* Drain the ring buffer and validate every SAMPLE record. */
+	{
+		uint64_t head = __atomic_load_n(&meta->data_head, __ATOMIC_ACQUIRE);
+		uint64_t tail = meta->data_tail;
+		bool ok = true;
+
+		if (head < tail || head - tail > rb_bytes)
+			ok = false;
+
+		while (ok && tail < head) {
+			struct perf_event_header hdr;
+			uint64_t available = head - tail;
+
+			if (available < sizeof(hdr) ||
+			    !ring_copy_from(&hdr, rb, rb_bytes, tail, sizeof(hdr)) ||
+			    hdr.size < sizeof(hdr) || hdr.size > available ||
+			    hdr.size > rb_bytes) {
+				ok = false;
+				break;
+			}
+
+			if (hdr.type == PERF_RECORD_SAMPLE) {
+				uint64_t dump_size, dyn_size;
+				size_t cursor = sizeof(hdr);
+
+				if (sizeof(dump_size) > hdr.size - cursor ||
+				    !ring_copy_from(&dump_size, rb, rb_bytes,
+						    tail + cursor, sizeof(dump_size)) ||
+				    dump_size > STACK_DUMP_SIZE) {
+					ok = false;
+					break;
+				}
+				cursor += sizeof(dump_size);
+				samples++;
+				if (dump_size) {
+					/* data blob then trailing dynamic size */
+					if (dump_size > hdr.size - cursor) {
+						ok = false;
+						break;
+					}
+					cursor += dump_size;
+					if (sizeof(dyn_size) > hdr.size - cursor ||
+					    !ring_copy_from(&dyn_size, rb, rb_bytes,
+							    tail + cursor,
+							    sizeof(dyn_size))) {
+						ok = false;
+						break;
+					}
+					if (dyn_size > dump_size) {
+						ok = false;
+						break;
+					}
+					if (dyn_size < dump_size)
+						truncated++;
+				}
+			}
+			tail += hdr.size;
+		}
+		__atomic_store_n(&meta->data_tail, head, __ATOMIC_RELEASE);
+
+		if (!ok)
+			ksft_test_result_fail("ustack records: malformed sample record\n");
+		else if (samples == 0)
+			ksft_test_result_skip("ustack records: no samples collected\n");
+		else if (truncated == 0)
+			ksft_test_result_fail("ustack records: no guarded-stack truncation\n");
+		else
+			ksft_test_result_pass("ustack records: %lu samples, %lu truncated\n",
+					      samples, truncated);
+	}
+
+	munmap(meta, page_size + rb_bytes);
+	close(fd);
+out_child:
+	stop_truncation_child(child, child_stack);
+}
+
+/* ---- Subtest 2: multi-threaded per-CPU sampling stress ---- */
+
+struct stress_thread {
+	pthread_t tid;
+	int cpu;
+	int *stop;
+	int fd;
+	void *rb;
+	size_t rb_bytes;
+};
+
+static void *stress_worker(void *arg)
+{
+	struct stress_thread *st = arg;
+	unsigned long sink = 1;
+	int sv[2];
+	char buf[64];
+
+	if (socketpair(AF_UNIX, SOCK_STREAM, 0, sv) == 0) {
+		while (!__atomic_load_n(st->stop, __ATOMIC_RELAXED)) {
+			/* unix-socket ping-pong: takes the socket locks the
+			 * original bug corrupted, while sampling nests.
+			 */
+			if (write(sv[0], buf, sizeof(buf)) > 0)
+				(void)read(sv[1], buf, sizeof(buf));
+			burn_stack(RECURSE_DEPTH, &sink);
+			/* Periodically consume the ring buffer so sampling
+			 * keeps delivering rather than filling up and stopping.
+			 */
+			if (st->rb) {
+				struct perf_event_mmap_page *m = st->rb;
+				uint64_t h = __atomic_load_n(&m->data_head,
+							     __ATOMIC_ACQUIRE);
+				__atomic_store_n(&m->data_tail, h,
+						 __ATOMIC_RELEASE);
+			}
+		}
+		close(sv[0]);
+		close(sv[1]);
+	}
+
+	return (void *)sink;
+}
+
+static void test_sse_stress_no_crash(void)
+{
+	struct perf_event_attr attr = sampling_attr(STRESS_SAMPLE_FREQ);
+	size_t rb_bytes = (size_t)RB_DATA_PAGES * page_size;
+	struct stress_thread *threads;
+	cpu_set_t available;
+	long progressed = 0;
+	int stop = 0;
+	const char *why = NULL;
+	long started = 0;
+	long nproc;
+	long slot;
+	int cpu;
+
+	if (sched_getaffinity(0, sizeof(available), &available)) {
+		ksft_test_result_fail("sse stress: sched_getaffinity: %s\n",
+				      strerror(errno));
+		return;
+	}
+	nproc = CPU_COUNT(&available);
+	if (nproc < 1) {
+		ksft_test_result_skip("sse stress: no available CPUs\n");
+		return;
+	}
+
+	threads = calloc(nproc, sizeof(*threads));
+	if (!threads) {
+		ksft_test_result_fail("sse stress: out of memory\n");
+		return;
+	}
+
+	slot = 0;
+	for (cpu = 0; cpu < CPU_SETSIZE; cpu++) {
+		struct stress_thread *st;
+		pthread_attr_t thread_attr;
+		cpu_set_t set;
+		void *map;
+		int ret;
+
+		if (!CPU_ISSET(cpu, &available))
+			continue;
+		st = &threads[slot++];
+
+		st->fd = -1;
+		st->cpu = cpu;
+		st->stop = &stop;
+		st->rb_bytes = rb_bytes;
+		st->fd = perf_event_open(&attr, -1, cpu, -1,
+					 PERF_FLAG_FD_CLOEXEC);
+		if (st->fd < 0) {
+			if (!started && open_skip_reason(errno, &why))
+				break;
+			continue;
+		}
+
+		map = mmap(NULL, page_size + rb_bytes, PROT_READ | PROT_WRITE,
+			   MAP_SHARED, st->fd, 0);
+		if (map == MAP_FAILED) {
+			close(st->fd);
+			st->fd = -1;
+			continue;
+		}
+		st->rb = map;
+
+		CPU_ZERO(&set);
+		CPU_SET(cpu, &set);
+		pthread_attr_init(&thread_attr);
+		ret = pthread_attr_setaffinity_np(&thread_attr, sizeof(set), &set);
+		if (!ret)
+			ret = pthread_create(&st->tid, &thread_attr,
+					     stress_worker, st);
+		pthread_attr_destroy(&thread_attr);
+
+		if (ret) {
+			munmap(st->rb, page_size + rb_bytes);
+			close(st->fd);
+			st->rb = NULL;
+			st->fd = -1;
+			continue;
+		}
+
+		ioctl(st->fd, PERF_EVENT_IOC_RESET, 0);
+		ioctl(st->fd, PERF_EVENT_IOC_ENABLE, 0);
+		started++;
+	}
+
+	if (started == 0) {
+		free(threads);
+		if (why)
+			ksft_test_result_skip("sse stress: %s\n", why);
+		else
+			ksft_test_result_skip("sse stress: could not start any sampler\n");
+		return;
+	}
+
+	sleep(STRESS_SECONDS);
+	__atomic_store_n(&stop, 1, __ATOMIC_RELAXED);
+
+	for (slot = 0; slot < nproc; slot++) {
+		struct stress_thread *st = &threads[slot];
+		struct perf_event_mmap_page *meta;
+
+		if (st->fd < 0)
+			continue;
+		pthread_join(st->tid, NULL);
+		ioctl(st->fd, PERF_EVENT_IOC_DISABLE, 0);
+		meta = st->rb;
+		if (__atomic_load_n(&meta->data_head, __ATOMIC_ACQUIRE))
+			progressed++;
+		munmap(st->rb, page_size + rb_bytes);
+		close(st->fd);
+	}
+
+	free(threads);
+	if (progressed != started)
+		ksft_test_result_fail("sse stress: %ld/%ld samplers made progress\n",
+				      progressed, started);
+	else
+		ksft_test_result_pass("sse stress: %ld samplers x %ds made progress\n",
+				      started, STRESS_SECONDS);
+}
+
+int main(void)
+{
+	const char *why;
+
+	page_size = sysconf(_SC_PAGESIZE);
+
+	ksft_print_header();
+	ksft_set_plan(2);
+	if (!pmu_sse_route_testable(&why)) {
+		ksft_test_result_skip("ustack records: %s\n", why);
+		ksft_test_result_skip("sse stress: %s\n", why);
+		ksft_finished();
+	}
+
+	test_ustack_records_wellformed();
+	test_sse_stress_no_crash();
+
+	ksft_finished();
+}
-- 
2.50.1 (Apple Git-155)


^ permalink raw reply related	[flat|nested] 22+ messages in thread

* Re: [PATCH v13 01/11] riscv: add SBI SSE extension definitions
  2026-11-19 12:08 ` [PATCH v13 01/11] riscv: add SBI SSE extension definitions Zhanpeng Zhang
@ 2026-11-19 12:14   ` sashiko-bot
  0 siblings, 0 replies; 22+ messages in thread
From: sashiko-bot @ 2026-11-19 12:14 UTC (permalink / raw)
  To: Zhanpeng Zhang; +Cc: linux-perf-users

> From: Clément Léger <cleger@rivosinc.com>
> 
> Add definitions for the SBI Supervisor Software Events extension [1].
> This extension enables the SBI to inject events into supervisor software
> much like ARM SDEI.
> 
> [1] https://lists.riscv.org/g/tech-prs/message/515
> 
> Signed-off-by: Clément Léger <cleger@rivosinc.com>
> Co-developed-by: Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>
> Signed-off-by: Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>
> Co-developed-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
> Signed-off-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261119120824.13969-13-zhangzhanpeng.jasper@bytedance.com?part=1


^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [PATCH v13 06/11] riscv: enable NMI-safe lockdep irqflags tracking
  2026-11-19 12:08 ` [PATCH v13 06/11] riscv: enable NMI-safe lockdep irqflags tracking Zhanpeng Zhang
@ 2026-11-19 12:21   ` sashiko-bot
  0 siblings, 0 replies; 22+ messages in thread
From: sashiko-bot @ 2026-11-19 12:21 UTC (permalink / raw)
  To: Zhanpeng Zhang; +Cc: linux-perf-users

> An SSE handler runs with interrupts masked, but can interrupt code whose
> lockdep hardirq state is enabled. Bare nmi_enter()/nmi_exit() do not
> update that state. Consequently, lockdep_assert_irqs_disabled() in the
> perf overflow path can report a warning despite interrupts being masked.
> 
> Select TRACE_IRQFLAGS_NMI_SUPPORT so lockdep can update its hardirq state
> inside an NMI. Pair this with irqentry_nmi_enter()/irqentry_nmi_exit() in
> the SSE wrapper: these helpers save the previous tracked state, mark
> interrupts disabled on entry and restore the tracked state on exit.
> Selecting the configuration option alone does not perform those updates.
> This follows the entry helpers used by the other RISC-V NMI-like traps.
> 
> Signed-off-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261119120824.13969-13-zhangzhanpeng.jasper@bytedance.com?part=6


^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [PATCH v13 05/11] riscv: mm: avoid enabling interrupts for nofault page faults
  2026-11-19 12:08 ` [PATCH v13 05/11] riscv: mm: avoid enabling interrupts for nofault page faults Zhanpeng Zhang
@ 2026-11-19 12:27   ` sashiko-bot
  0 siblings, 0 replies; 22+ messages in thread
From: sashiko-bot @ 2026-11-19 12:27 UTC (permalink / raw)
  To: Zhanpeng Zhang; +Cc: linux-perf-users

> RISC-V enables interrupts in handle_page_fault() before checking whether
> the fault occurred with fault handling disabled. A nofault access from an
> atomic context can therefore run tracepoints and open an interrupt window
> before reaching the exception-table fixup.
> 
> Handle an exception-table entry before entering the generic fault path when
> fault handling is disabled. Also keep interrupts disabled until such a
> fault has been resolved.
> 
> On 32-bit RISC-V the vmalloc mapping must still be synchronised into the
> current page table before that fixup runs: otherwise a nofault access to a
> not-yet-synced vmalloc address would take the exception-table fixup and
> fail spuriously instead of being resolved. Order the vmalloc
> synchronisation ahead of the nofault fixup. That synchronisation only
> copies an already-established kernel mapping, takes no locks and waits for
> no I/O, so it is safe in the nofault path; if the mapping does not exist it
> falls through to the existing no_context() error handling. This early
> return skips the page-fault tracepoints, as before.
> 
> This makes RISC-V consistent with the expectation that an in-atomic
> nofault access does not enter the normal fault-handling path. It also
> removes one source of re-entry when perf sampling is delivered through an
> SBI Supervisor Software Event (SSE).
> 
> Signed-off-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261119120824.13969-13-zhangzhanpeng.jasper@bytedance.com?part=5


^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [PATCH v13 08/11] perf: RISC-V: add support for SSE event
  2026-11-19 12:08 ` [PATCH v13 08/11] perf: RISC-V: add support for SSE event Zhanpeng Zhang
@ 2026-11-19 12:34   ` sashiko-bot
  0 siblings, 0 replies; 22+ messages in thread
From: sashiko-bot @ 2026-11-19 12:34 UTC (permalink / raw)
  To: Zhanpeng Zhang; +Cc: linux-perf-users

Thank you for your contribution! Sashiko AI review found 10 potential
issues to consider:
- [High] perf: riscv: unstopped counters in pmu_sbi_ovf_sse_handler()
  when regs is NULL
- [High] perf: riscv: unhandled PERF_HES_STOPPED counters in overflow
  and stop paths
- [Medium] perf: riscv: missing sse_event_cleanup() when
  pmu_sbi_setup_sse() fails
- [Medium] perf: riscv: batch counter stop in pmu_sbi_cleanup_sse()
  skips active counters
- [Medium] perf: riscv: premature hw_evt->state reset in
  pmu_sbi_ovf_handler() loop
- [Medium] perf: riscv: missing perf_event_update_userpage() in
  pmu_sbi_fail_sse()
- [Medium] perf: riscv: pmu_sbi_sse_enable() latches sse_failed on
  hotplug and reboot
- [Medium] perf: riscv: riscv_pm_pmu_notify() ignores latched
  cpuc->sse_failed state
- [Medium] perf: riscv: lost PMU SSE resume state on CPU_PM_ENTER after
  unmask failure
- [Low] perf: riscv: unused pmu_sbi_stop_all_cpu() when
  CONFIG_RISCV_PMU_SBI_SSE=n
--

commit f0818fdfc418d31969bed16ca22047c7d7a80ce8
Author: Clément Léger <cleger@rivosinc.com>

perf: RISC-V: add support for SSE event

This commit adds support for delivering RISC-V SBI PMU counter overflow
events via Supervisor Software Events (SSE), including probe-time delivery
selection, CPU power-management coordination, and error handling.

Link: https://lore.kernel.org/r/20261002-reach-cloning-f1594729dbe3@wendy/

> diff --git a/drivers/perf/riscv_pmu_sbi.c b/drivers/perf/riscv_pmu_sbi.c
> index 21a2078eab98a..a984ac57a7b6e 100644
> --- a/drivers/perf/riscv_pmu_sbi.c
> +++ b/drivers/perf/riscv_pmu_sbi.c

[ ... ]

> @@ -936,14 +938,42 @@ static inline void pmu_sbi_stop_all(struct riscv_pmu *pmu)
>  	 * which may include counters that are not enabled yet.
>  	 */
>  	for (i = 0; i < BITS_TO_LONGS(RISCV_MAX_COUNTERS); i++) {
> -		if (!pmu->cmask[i])
> +		if (!ctr_mask[i])
>  			continue;
>  		sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_STOP,
> -			  i * BITS_PER_LONG, pmu->cmask[i],
> +			  i * BITS_PER_LONG, ctr_mask[i],
>  			  SBI_PMU_STOP_FLAG_RESET);
>  	}
>  }
>  
> +static void pmu_sbi_sse_stop_all(const unsigned long *ctr_mask)
> +{
> +	struct sbiret ret;
> +	int i;
> +
> +	/*
> +	 * Stop each counter separately: an already-stopped counter must not
> +	 * prevent firmware from stopping a later, overflowing counter.
> +	 */
> +	for_each_set_bit(i, ctr_mask, RISCV_MAX_COUNTERS) {
> +		ret = sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_STOP,
> +				i, 1, SBI_PMU_STOP_FLAG_RESET);
> +		if (ret.error && ret.error != SBI_ERR_ALREADY_STOPPED)
> +			pr_warn_ratelimited("failed to stop counter %d: %ld\n",
> +					    i, ret.error);
> +	}
> +}
> +
> +static inline void pmu_sbi_stop_all(struct riscv_pmu *pmu)
> +{
> +	pmu_sbi_stop_all_mask(pmu->cmask);
> +}
> +
> +static void pmu_sbi_stop_all_cpu(void *info)
> +{
> +	pmu_sbi_stop_all(info);
> +}

[Severity: Low]
Will pmu_sbi_stop_all_cpu() trigger an unused-function warning when
CONFIG_RISCV_PMU_SBI_SSE is disabled, since its only caller is
pmu_sbi_cleanup_sse() inside #ifdef CONFIG_RISCV_PMU_SBI_SSE?

[ ... ]

> @@ -1054,32 +1098,111 @@ static inline void pmu_sbi_start_ovf_ctrs_snapshot(struct cpu_hw_events *cpu_hw_

[ ... ]

> +#ifdef CONFIG_RISCV_PMU_SBI_SSE
> +/*
> + * A local SSE delivery failure makes the current PMU state unsafe to resume.
> + * Latch the failure before stopping mapped events so the SSE transition and
> + * overflow restart paths cannot undo the fail-safe while they are quiesced.
> + */
> +static void pmu_sbi_fail_sse(struct riscv_pmu *pmu, const char *op, int ret)
> +{
> +	struct cpu_hw_events *cpu_hw_evt = this_cpu_ptr(pmu->hw_events);
> +	struct perf_event *event;
> +	int idx;
> +
> +	if (READ_ONCE(cpu_hw_evt->sse_failed))
> +		return;
> +
> +	WRITE_ONCE(cpu_hw_evt->sse_failed, true);
> +	pr_err_ratelimited("failed to %s local PMU SSE event: %d; stopping counters\n",
> +			   op, ret);
> +
> +	for (idx = 0; idx < RISCV_MAX_COUNTERS; idx++) {\n> +		event = cpu_hw_evt->events[idx];
> +		if (event)
> +			riscv_pmu_stop(event, PERF_EF_UPDATE);
> +	}
> +}

[Severity: Medium]
Should pmu_sbi_fail_sse() also call perf_event_update_userpage(event) after
riscv_pmu_stop()?

riscv_pmu_stop() calls pmu_sbi_ctr_stop() -> pmu_sbi_reset_scounteren(),
which clears the counter's bit in CSR_SCOUNTEREN and sets PERF_HES_STOPPED,
but does not update the perf mmap page.  If userpg->index remains non-zero
(for example when sse_event_enable_local() fails in pmu_sbi_sse_enable() or
after riscv_pm_pmu_update() on CPU_PM_EXIT), userspace direct counter reads
will execute csrr on the disabled CSR and trap with SIGILL.

> +
> +static void pmu_sbi_sse_disable(struct pmu *pmu)
> +{
> +	struct riscv_pmu *rvpmu = to_riscv_pmu(pmu);
> +	struct cpu_hw_events *cpu_hw_evt = this_cpu_ptr(rvpmu->hw_events);
> +	int ret;
> +
> +	if (!READ_ONCE(rvpmu->sse_active) ||
> +	    READ_ONCE(cpu_hw_evt->sse_failed))
> +		return;
> +
> +	ret = sse_event_disable_local(rvpmu->sse_evt);
> +	if (unlikely(ret))
> +		pmu_sbi_fail_sse(rvpmu, "disable", ret);
> +}
> +
> +static void pmu_sbi_sse_enable(struct pmu *pmu)
> +{
> +	struct riscv_pmu *rvpmu = to_riscv_pmu(pmu);
> +	struct cpu_hw_events *cpu_hw_evt = this_cpu_ptr(rvpmu->hw_events);
> +	int ret;
> +
> +	if (!READ_ONCE(rvpmu->sse_active) ||
> +	    READ_ONCE(cpu_hw_evt->sse_failed))
> +		return;
> +
> +	ret = sse_event_enable_local(rvpmu->sse_evt);
> +	if (unlikely(ret))
> +		pmu_sbi_fail_sse(rvpmu, "enable", ret);
> +}
> +#endif

[Severity: Medium]
Can pmu_sbi_sse_enable() permanently latch cpu_hw_evt->sse_failed during
CPU hotplug or system reboot?

CPUHP_AP_RISCV_SSE_ONLINE runs in the threaded ONLINE section after
CPUHP_AP_ONLINE_IDLE, CPUHP_AP_SCHED_WAIT_EMPTY, and
CPUHP_AP_SMPBOOT_THREADS.  If a task carrying a hardware perf event
context-switches on a CPU after sse_cpu_teardown() on cpu_down or before
sse_cpu_online() on cpu_up, perf_pmu_enable() -> pmu_sbi_sse_enable() calls
sse_event_enable_local() while reg_evt->is_registered is false, which
returns -EINVAL and permanently disables PMU counters on that CPU via
pmu_sbi_fail_sse().  Similarly, during reboot sse_reboot_nb (priority 0)
sets sse_shutting_down before perf_reboot_notifier (priority INT_MIN) stops
events, causing sse_event_enable_local() to return -ESHUTDOWN and hit
pmu_sbi_fail_sse().

[ ... ]

> @@ -1165,6 +1306,14 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
>  		hw_evt->state |= PERF_HES_UPTODATE;
>  		perf_sample_data_init(&data, 0, hw_evt->last_period);
>  		if (riscv_pmu_event_set_period(event)) {
> +			int overflow_ret;
> +
> +			/* Guest attribution and guest stack sampling are not supported yet. */
> +			if (from_guest) {
> +				hw_evt->state = 0;
> +				continue;
> +			}
> +
>  			/*
>  			 * Unlike other ISAs, RISC-V don't have to disable interrupts
>  			 * to avoid throttling here. As per the specification, the
> @@ -1173,128 +1322,291 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
>  			 * TODO: We will need to stop the guest counters once
>  			 * virtualization support is added.
>  			 */
> -			perf_event_overflow(event, &data, regs);
> +			overflow_ret = perf_event_overflow(event, &data, regs);
> +			if (!overflow_ret)
> +				hw_evt->state = 0;
> +		} else {
> +			hw_evt->state = 0;
>  		}
> -		/* Reset the state as we are going to start the counter after the loop */
> -		hw_evt->state = 0;
>  	}
>  
>  	pmu_sbi_start_overflow_mask(pmu, overflowed_ctrs);

[Severity: High]
When perf_event_overflow() returns non-zero and leaves a throttled counter
stopped with PERF_HES_STOPPED while its bit remains set in
cpu_hw_evt->used_hw_ctrs, how does a subsequent overflow on another counter
handle that stopped counter?

First, pmu_sbi_stop_hw_ctrs() passes the unfiltered cpu_hw_evt->used_hw_ctrs[i]
to a batch SBI_EXT_PMU_COUNTER_STOP call.  Because the throttled counter is
already stopped in firmware, the batch call can abort early with
SBI_ERR_ALREADY_STOPPED without stopping higher-indexed overflowing
counters, or return ret.error == SBI_ERR_ALREADY_STOPPED and cause
pmu_sbi_stop_hw_ctrs() to skip saving sdata->ctr_values[] and overwrite the
shared snapshot buffer with zeroes from snapshot_cval_shcopy[].

Second, because the throttled counter was never restarted, its hardware OF
bit in CSR_SCOUNTOVF remains set while riscv_pmu_event_set_period() already
reloaded hw_evt->period_left > 0.  On the next overflow of another counter,
the loop in pmu_sbi_ovf_handler() does not skip events with
PERF_HES_STOPPED set; it revisits the throttled counter, sees
riscv_pmu_event_set_period(event) return 0, and sets hw_evt->state = 0 in
the else branch (or the from_guest branch), causing
pmu_sbi_start_overflow_mask() to restart the throttled counter.

[Severity: Medium]
Can clearing hw_evt->state = 0 inside the pmu_sbi_ovf_handler() loop before
pmu_sbi_start_overflow_mask() reprograms the counters corrupt group
throttling or frequency-mode throttling?

If two events in the same group overflow together and the first does not
throttle (setting its hw_evt->state = 0) while the second throttles the
group, perf_event_throttle_group() calls riscv_pmu_stop() on the first
event before its hardware counter is reprogrammed, running
riscv_pmu_event_update() a second time against the new hw_evt->prev_count.
Also, for attr.freq events, __perf_event_account_interrupt() calls
perf_adjust_period() after perf_event_throttle_group(); when period_left >
8 * sample_period, perf_adjust_period() calls event->pmu->start()
(riscv_pmu_start()), clearing hwc->state = 0 and starting the counter even
though perf_event_overflow() returns 1.

>  
>  	perf_sample_event_took(sched_clock() - start_clock);
>  
>  	return IRQ_HANDLED;
>  }
>  
> -static int pmu_sbi_starting_cpu(unsigned int cpu, struct hlist_node *node)
> +static irqreturn_t pmu_sbi_ovf_irq_handler(int irq, void *dev)
>  {
> -	struct riscv_pmu *pmu = hlist_entry_safe(node, struct riscv_pmu, node);
> -	struct cpu_hw_events *cpu_hw_evt = this_cpu_ptr(pmu->hw_events);
> -
> -	/*
> -	 * We keep enabling userspace access to CYCLE, TIME and INSTRET via the
> -	 * legacy option but that will be removed in the future.
> -	 */
> -	if (sysctl_perf_user_access == SYSCTL_LEGACY)
> -		csr_write(CSR_SCOUNTEREN, 0x7);
> -	else
> -		csr_write(CSR_SCOUNTEREN, 0x2);
> +	return pmu_sbi_ovf_handler(dev, get_irq_regs(), false, false);
> +}
>  
> -	/* Stop all the counters so that they can be enabled from perf */
> -	pmu_sbi_stop_all(pmu);
> +#ifdef CONFIG_RISCV_PMU_SBI_SSE
> +static int pmu_sbi_ovf_sse_handler(u32 evt, void *arg, struct pt_regs *regs)
> +{
> +	const struct riscv_sse_interrupted_context *context;
> +	struct riscv_pmu *pmu = arg;
> +	struct cpu_hw_events *hw_event = raw_cpu_ptr(pmu->hw_events);
> +	bool from_guest;
> +
> +	if (unlikely(!READ_ONCE(pmu->sse_active))) {
> +		pmu_sbi_sse_stop_all(pmu->cmask);
> +		return -EIO;
> +	}
>  
> -	if (riscv_pmu_use_irq) {
> -		cpu_hw_evt->irq = riscv_pmu_irq;
> -		ALT_SBI_PMU_OVF_CLEAR_PENDING(riscv_pmu_irq_mask);
> -		enable_percpu_irq(riscv_pmu_irq, IRQ_TYPE_NONE);
> +	if (unlikely(!regs)) {
> +		pmu_sbi_fail_sse(pmu, "read interrupted context for", -EIO);
> +		return -EIO;
>  	}

[Severity: High]
Does the !regs path in pmu_sbi_ovf_sse_handler() leave the overflowing
counter running when no perf event is mapped or when cpu_hw_evt->sse_failed
is already true?

Unlike pmu_sbi_ovf_handler(), which stops all counters via
pmu_sbi_sse_stop_all(cmask) when no event is installed and calls
pmu_sbi_stop_hw_ctrs(pmu) before checking pmu_sbi_sse_failed(),
pmu_sbi_fail_sse() only stops non-NULL entries in cpu_hw_evt->events[] and
returns immediately if cpu_hw_evt->sse_failed was already latched (for
example after a failed sse_event_disable_local()).  If an early or unmapped
counter overflows after pmu_sbi_setup_sse() enables the event during probe,
or an overflow arrives after sse_failed is set, returning with the counter
still overflowing will cause firmware to immediately re-deliver the NMI-like
SSE event on SBI_SSE_EVENT_COMPLETE.

[ ... ]

> +static int pmu_sbi_setup_sse(struct riscv_pmu *pmu)
> +{
> +	int ret;
> +	struct sse_event *evt;
>  
> -	/* Disable all counters access for user mode now */
> -	csr_write(CSR_SCOUNTEREN, 0x0);
> +	evt = sse_event_register(SBI_SSE_EVENT_LOCAL_PMU_OVERFLOW, 0,
> +				 pmu_sbi_ovf_sse_handler, pmu);
> +	if (IS_ERR(evt))
> +		return PTR_ERR(evt);
> +	pmu->sse_evt = evt;
>  
> -	if (sbi_pmu_snapshot_available())
> -		return pmu_sbi_snapshot_disable();
> +	ret = sse_event_enable(evt);
> +	if (ret) {
> +		int cleanup_ret;
>  
> -	return 0;
> +		cleanup_ret = sse_event_disable(evt);
> +		if (cleanup_ret) {
> +			pr_warn("failed to disable SSE event after setup error: %d\n",
> +				cleanup_ret);
> +			return cleanup_ret;
> +		}
> +
> +		cleanup_ret = sse_event_unregister(evt);
> +
> +		if (cleanup_ret) {
> +			pr_warn("failed to unregister SSE event: %d\n",
> +				cleanup_ret);
> +		} else {
> +			pmu->sse_evt = NULL;
> +		}
> +		return cleanup_ret ?: ret;
> +	}

[Severity: Medium]
If sse_event_enable() fails in pmu_sbi_setup_sse() and the rollback in
sse_event_disable() or sse_event_unregister() also fails, what transfers the
retained pmu->sse_evt to sse_event_cleanup()?

When pmu_sbi_setup_irqs() returns -EUCLEAN, pmu_sbi_device_probe() treats
the error as non-fatal and continues registering the PMU in counting-only
mode without calling riscv_pmu_destroy() -> pmu_sbi_cleanup_sse().  Because
sse_event_cleanup() is never called on pmu->sse_evt, evt->cleanup_pending
remains false, so the SSE core continues treating the event as active across
CPU hotplug and any counter wrap while pmu->sse_active is false will invoke
pmu_sbi_ovf_sse_handler() -> pmu_sbi_sse_stop_all(pmu->cmask), resetting
active counting counters.

> +
> +	WRITE_ONCE(pmu->sse_active, true);
> +	pr_info("using SSE for PMU event delivery\n");
> +
> +	return ret;
>  }
>  
> -static int pmu_sbi_setup_irqs(struct riscv_pmu *pmu, struct platform_device *pdev)
> +static void pmu_sbi_cleanup_sse(struct riscv_pmu *pmu)
>  {
> +	struct sse_event *sse_evt;
>  	int ret;
> -	struct cpu_hw_events __percpu *hw_events = pmu->hw_events;
> -	struct irq_domain *domain = NULL;
>  
> +	sse_evt = pmu->sse_evt;
> +	if (!sse_evt)
> +		return;
> +	/*
> +	 * Close callback admission before draining each CPU. PMU callbacks run
> +	 * with local interrupts disabled, so the synchronous IPI cannot complete
> +	 * until a callback that observed the old state has returned.
> +	 */
> +	WRITE_ONCE(pmu->sse_active, false);
> +	on_each_cpu(pmu_sbi_stop_all_cpu, pmu, 1);

[Severity: Medium]
Should pmu_sbi_stop_all_cpu() call pmu_sbi_sse_stop_all(pmu->cmask) instead
of the batch pmu_sbi_stop_all(pmu)?

pmu_sbi_stop_all() issues a batch SBI_EXT_PMU_COUNTER_STOP across
pmu->cmask, which can stop at an already-stopped lower-indexed counter and
return SBI_ERR_ALREADY_STOPPED before stopping higher-indexed active
counters.  If sse_event_disable() or sse_event_unregister() then fails and
branches to retain, sse_event_cleanup() replaces the handler with the no-op
sse_cleanup_event_handler(), leaving no handler to stop the counter source.

[ ... ]

> @@ -1289,14 +1601,17 @@ static int pmu_sbi_setup_irqs(struct riscv_pmu *pmu, struct platform_device *pde
>  }
>  
>  #ifdef CONFIG_CPU_PM
> -static int riscv_pm_pmu_notify(struct notifier_block *b, unsigned long cmd,
> -				void *v)
> +static int riscv_pm_pmu_update(struct riscv_pmu *rvpmu, unsigned long cmd)
>  {
> -	struct riscv_pmu *rvpmu = container_of(b, struct riscv_pmu, riscv_pm_nb);
>  	struct cpu_hw_events *cpuc = this_cpu_ptr(rvpmu->hw_events);
>  	bool enabled = !bitmap_empty(cpuc->used_hw_ctrs, RISCV_MAX_COUNTERS);
>  	struct perf_event *event;
>  	int idx;
>  
> +	if (cmd == CPU_PM_ENTER)
> +		bitmap_zero(cpuc->pm_resume_hw_ctrs, RISCV_MAX_COUNTERS);
> +
>  	if (!enabled)
>  		return NOTIFY_OK;
>  
> @@ -1305,6 +1617,8 @@ static int riscv_pm_pmu_notify(struct notifier_block *b, unsigned long cmd,
>  
>  		switch (cmd) {
>  		case CPU_PM_ENTER:
> +			if (!(event->hw.state & PERF_HES_STOPPED))
> +				set_bit(idx, cpuc->pm_resume_hw_ctrs);
>  			/*
>  			 * Stop and update the counter
>  			 */
> @@ -1312,6 +1626,8 @@ static int riscv_pm_pmu_notify(struct notifier_block *b, unsigned long cmd,
>  			break;
>  		case CPU_PM_EXIT:
>  		case CPU_PM_ENTER_FAILED:
> +			if (!test_and_clear_bit(idx, cpuc->pm_resume_hw_ctrs))
> +				break;
>  			/*
>  			 * Restore and enable the counter.
>  			 */
> @@ -1325,9 +1641,59 @@ static int riscv_pm_pmu_notify(struct notifier_block *b, unsigned long cmd,
>  	return NOTIFY_OK;
>  }
>  
> +static int riscv_pm_pmu_notify(struct notifier_block *b,
> +			       unsigned long cmd, void *v)
> +{
> +	struct riscv_pmu *rvpmu = container_of(b, struct riscv_pmu,
> +						 riscv_pm_nb);
> +
> +#ifdef CONFIG_RISCV_PMU_SBI_SSE
> +	struct cpu_hw_events *cpuc = this_cpu_ptr(rvpmu->hw_events);
> +	int ret;
> +
> +	if (!riscv_pmu_use_irq && READ_ONCE(rvpmu->sse_active)) {
> +		switch (cmd) {
> +		case CPU_PM_ENTER:
> +			cpuc->pm_resume_sse =
> +				sse_event_is_enabled_local(rvpmu->sse_evt);
> +			if (cpuc->pm_resume_sse) {
> +				ret = sse_event_disable_local(rvpmu->sse_evt);
> +				if (ret) {
> +					cpuc->pm_resume_sse = false;
> +					pmu_sbi_fail_sse(rvpmu, "disable for CPU PM",
> +							 ret);
> +					return notifier_from_errno(ret);
> +				}
> +			}
> +			break;
> +		case CPU_PM_EXIT:
> +		case CPU_PM_ENTER_FAILED:
> +			ret = riscv_pm_pmu_update(rvpmu, cmd);
> +			if (!cpuc->pm_resume_sse)
> +				return ret;
> +
> +			cpuc->pm_resume_sse = false;
> +			ret = sse_event_enable_local(rvpmu->sse_evt);
> +			if (ret) {
> +				pmu_sbi_fail_sse(rvpmu, "enable after CPU PM", ret);
> +				return notifier_from_errno(ret);
> +			}
> +			return NOTIFY_OK;
> +		default:
> +			break;
> +		}
> +	}
> +#endif

[Severity: Medium]
Should riscv_pm_pmu_notify() check READ_ONCE(cpuc->sse_failed) before
performing local SSE transitions?

If pmu_sbi_fail_sse() is triggered while the local SSE event is enabled (for
example when regs is NULL in pmu_sbi_ovf_sse_handler() or when
sse_event_disable_local() fails in pmu_sbi_sse_disable()), reg_evt->is_enabled
remains true.  On the next CPU_PM_ENTER, riscv_pm_pmu_notify() sees
sse_event_is_enabled_local() == true and tries to disable the event: if that
succeeds, CPU_PM_EXIT re-enables the failed SSE event in firmware; if it
keeps failing, every CPU_PM_ENTER returns notifier_from_errno(ret) and
aborts CPU idle entry.

[Severity: Medium]
What happens on the next CPU_PM_ENTER if sse_events_unmask() fails during
CPU_PM_EXIT?

When sse_events_unmask() fails on CPU_PM_EXIT, sse_pm_notifier() returns
notifier_from_errno(ret) and stops the notifier chain before the
lower-priority riscv_pm_pmu_notify() runs, leaving rvpmu->sse_evt disabled
and the counters stopped with PERF_HES_STOPPED (without setting
cpuc->sse_failed).  On the next CPU_PM_ENTER, riscv_pm_pmu_notify()
overwrites cpuc->pm_resume_sse with false and riscv_pm_pmu_update() zeroes
cpuc->pm_resume_hw_ctrs while skipping the stopped counters, so subsequent
CPU_PM_EXIT calls never restore the counters or re-enable the SSE event.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261119120824.13969-13-zhangzhanpeng.jasper@bytedance.com?part=8

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [PATCH v13 11/11] selftests/riscv: add perf user-stack SSE copy regression test
  2026-11-19 12:08 ` [PATCH v13 11/11] selftests/riscv: add perf user-stack SSE copy regression test Zhanpeng Zhang
@ 2026-11-19 12:34   ` sashiko-bot
  0 siblings, 0 replies; 22+ messages in thread
From: sashiko-bot @ 2026-11-19 12:34 UTC (permalink / raw)
  To: Zhanpeng Zhang; +Cc: linux-perf-users

Thank you for your contribution! Sashiko AI review found 5 potential
issues to consider:
- [Medium] selftests/riscv: perf_event_open() EINVAL fails instead of
  skipping
- [Medium] selftests/riscv: test_ustack_records_wellformed() accepts
  0-byte copy
- [Medium] selftests/riscv: burn_stack() overflows default musl pthread
  stack
- [Low] selftests/riscv: false skip on setup failure in
  test_sse_stress_no_crash()
- [Low] selftests/riscv: unused stress_thread fields and missing
  .gitignore entry
--

commit e1bf6fbd4261f2eb37c1aa64007c8a76c7256f6e
Author: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>

selftests/riscv: add perf user-stack SSE copy regression test

This commit adds a userspace kselftest under tools/testing/selftests/riscv/sse
to exercise PERF_SAMPLE_STACK_USER user-stack copying from RISC-V SBI
Supervisor Software Events (SSE) context across a guard-page boundary and
under multi-CPU stress.

> diff --git a/tools/testing/selftests/riscv/sse/sse_perf_ustack.c b/tools/testing/selftests/riscv/sse/sse_perf_ustack.c
> new file mode 100644
> index 0000000000000..9535b6d7ba4c1
> --- /dev/null
> +++ b/tools/testing/selftests/riscv/sse/sse_perf_ustack.c
> @@ -0,0 +1,564 @@

[ ... ]

> +static struct perf_event_attr sampling_attr(unsigned long freq)
> +{
> +	struct perf_event_attr attr = {
> +		.type		= PERF_TYPE_HARDWARE,
> +		.size		= sizeof(attr),
> +		.config		= PERF_COUNT_HW_INSTRUCTIONS,
> +		.sample_type	= PERF_SAMPLE_STACK_USER,
> +		.sample_stack_user = STACK_DUMP_SIZE,
> +		.freq		= 1,
> +		.sample_freq	= freq,
> +		.disabled	= 1,
> +		.exclude_kernel	= 1,
> +		.exclude_hv	= 1,
> +	};
> +
> +	return attr;
> +}
> +
> +static bool open_skip_reason(int err, const char **why)
> +{
> +	switch (err) {
> +	case EACCES:
> +	case EPERM:
> +		*why = "insufficient privilege for PMU sampling (perf_event_paranoid)";
> +		return true;
> +	case ENOENT:
> +	case ENODEV:
> +	case EOPNOTSUPP:
> +		*why = "hardware PMU sampling event not available";
> +		return true;
> +	default:
> +		return false;
> +	}
> +}

[Severity: Medium]
Does perf_event_open() fail with EINVAL instead of skipping on RISC-V
systems without PMU overflow support?

When neither SBI SSE nor Sscofpmf IRQ overflow delivery is available,
pmu_sbi_device_probe() and riscv_pmu_legacy set both
PERF_PMU_CAP_NO_INTERRUPT and PERF_PMU_CAP_NO_EXCLUDE without registering a
riscv-pmu entry in /proc/interrupts.  Because sampling_attr() sets
exclude_kernel and exclude_hv, perf_try_init_event() rejects the event with
-EINVAL on PERF_PMU_CAP_NO_EXCLUDE before perf_event_open() reaches the
PERF_PMU_CAP_NO_INTERRUPT check that returns -EOPNOTSUPP, causing
test_ustack_records_wellformed() to report FAIL instead of SKIP.

[ ... ]

> +static int truncation_child(void *arg)
> +{
> +	int ready_fd = (intptr_t)arg;
> +	char ready = 1;
> +
> +	if (write(ready_fd, &ready, sizeof(ready)) != 1)
> +		return 1;
> +
> +	for (;;)
> +		__asm__ __volatile__("" : : : "memory");
> +}

[ ... ]

> +					if (dyn_size > dump_size) {
> +						ok = false;
> +						break;
> +					}
> +					if (dyn_size < dump_size)
> +						truncated++;

[Severity: Medium]
Does test_ustack_records_wellformed() copy 0 bytes on every sample and count
dyn_size == 0 as a valid guard-page truncation?

start_truncation_child() passes stack + page_size (immediately below the
PROT_NONE page) to clone(), and truncation_child() only allocates ready_fd
and ready before spinning, leaving sp within 16 to 48 bytes of the guard
page.  On RV64, __copy_from_user_inatomic() enters .Lword_copy in
arch/riscv/lib/uaccess.S and issues eight unrolled REG_L loads (64 bytes)
before storing anything or advancing the destination pointer, so the copy
faults during the first burst and returns a remaining count equal to
dump_size (dyn_size == 0).  Since any sample with dyn_size < dump_size
increments truncated, the subtest passes without verifying that a non-zero
partial stack copy occurred across the boundary.

Giving truncation_child() a live stack frame larger than 64 bytes and
requiring 0 < dyn_size && dyn_size < dump_size would ensure a partial copy
is actually tested.

[ ... ]

> +struct stress_thread {
> +	pthread_t tid;
> +	int cpu;
> +	int *stop;
> +	int fd;
> +	void *rb;
> +	size_t rb_bytes;
> +};

[ ... ]

> +		st->fd = -1;
> +		st->cpu = cpu;
> +		st->stop = &stop;
> +		st->rb_bytes = rb_bytes;

[Severity: Low]
This isn't a bug, but stress_thread::cpu and stress_thread::rb_bytes are
written here and never read anywhere.  Also, tools/testing/selftests/riscv/sse
is missing a .gitignore entry for the new sse_perf_ustack binary added to
TEST_GEN_PROGS.

[ ... ]

> +		CPU_ZERO(&set);
> +		CPU_SET(cpu, &set);
> +		pthread_attr_init(&thread_attr);
> +		ret = pthread_attr_setaffinity_np(&thread_attr, sizeof(set), &set);
> +		if (!ret)
> +			ret = pthread_create(&st->tid, &thread_attr,
> +					     stress_worker, st);
> +		pthread_attr_destroy(&thread_attr);

[Severity: Medium]
Can stress_worker() overflow the default pthread stack on musl libc?

burn_stack() is noinline and keeps unsigned long frame[32] (256 bytes on
64-bit RISC-V) live across RECURSE_DEPTH (512) recursive calls, consuming
about 145 KiB of stack.  Because thread_attr uses the default stack size
 without pthread_attr_setstacksize(), this exceeds musl's 128 KiB default
pthread stack and crashes the test with SIGSEGV.

[ ... ]

> +	if (started == 0) {
> +		free(threads);
> +		if (why)
> +			ksft_test_result_skip("sse stress: %s\n", why);
> +		else
> +			ksft_test_result_skip("sse stress: could not start any sampler\n");
> +		return;
> +	}

[Severity: Low]
Should the else branch report a failure or error rather than
ksft_test_result_skip() when started == 0 and why is NULL?

open_skip_reason() sets why when perf_event_open() fails due to missing
privileges or unavailable PMU support, so reaching started == 0 with
why == NULL means every sampler failed due to an unexpected
perf_event_open(), mmap(), or pthread creation error.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261119120824.13969-13-zhangzhanpeng.jasper@bytedance.com?part=11

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [PATCH v13 02/11] riscv: add support for SBI Supervisor Software Events extension
  2026-11-19 12:08 ` [PATCH v13 02/11] riscv: add support for SBI Supervisor Software Events extension Zhanpeng Zhang
@ 2026-11-19 12:36   ` sashiko-bot
  0 siblings, 0 replies; 22+ messages in thread
From: sashiko-bot @ 2026-11-19 12:36 UTC (permalink / raw)
  To: Zhanpeng Zhang; +Cc: linux-perf-users

Thank you for your contribution! Sashiko AI review found 2 potential
issues to consider:
- [High] riscv: sbi_sse: field-spanning memcpy() on regs->a6 in do_sse()
- [High] riscv: sse: NULL tp dereference in handle_sse() during
  __switch_to()
--

commit 498ea890d0b707a9ef408aa0d005f9d3ac0e2ea3
Author: Clément Léger <cleger@rivosinc.com>

riscv: add support for SBI Supervisor Software Events extension

This commit adds architectural support for the RISC-V SBI Supervisor
Software Events (SSE) extension, including dedicated per-event stacks,
entry/exit assembly, and context preservation across SSE handlers.

Link: https://lore.kernel.org/r/as79MHtNYd-VbUug@hu-himchau-blr.qualcomm.com/

> diff --git a/arch/riscv/kernel/sbi_sse.c b/arch/riscv/kernel/sbi_sse.c
> new file mode 100644
> index 0000000000000..032fcc07abcf7
> --- /dev/null
> +++ b/arch/riscv/kernel/sbi_sse.c
> @@ -0,0 +1,251 @@

[ ... ]

> +void noinstr do_sse(struct sse_event_arch_data *arch_evt,
> +		    struct pt_regs *regs, unsigned long hstatus)
> +{
> +	struct riscv_sse_interrupted_context context = { regs, hstatus };
> +	struct riscv_sse_interrupted_context *previous;
> +	struct sbiret sret;
> +
> +	nmi_enter();
> +	instrumentation_begin();
> +
> +	/* Retrieve missing GPRs from SBI */
> +	sret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_EVENT_ATTR_READ, arch_evt->evt_id,
> +			 SBI_SSE_ATTR_INTERRUPTED_A6,
> +			 (SBI_SSE_ATTR_INTERRUPTED_A7 -
> +			  SBI_SSE_ATTR_INTERRUPTED_A6) + 1,
> +			 (unsigned long)arch_evt->interrupted_phys, 0, 0);
> +	if (sret.error) {
> +		pr_warn("Failed to read interrupted registers for event %x: %ld\n",
> +			arch_evt->evt_id, sret.error);
> +		/* Let the client quiesce its source without using incomplete regs. */
> +		sse_handle_event(arch_evt, NULL);
> +		goto out;
> +	}
> +
> +	memcpy(&regs->a6, arch_evt->interrupted,
> +	       sizeof(*arch_evt->interrupted));
> +
> +	/* Make the interrupted frame visible while clients handle this event. */
> +	previous = this_cpu_read(riscv_sse_interrupted_context);
> +	this_cpu_write(riscv_sse_interrupted_context, &context);
> +	sse_handle_event(arch_evt, regs);
> +	this_cpu_write(riscv_sse_interrupted_context, previous);
> +
> +	if (memcmp(&regs->a6, arch_evt->interrupted,
> +		   sizeof(*arch_evt->interrupted))) {
> +		memcpy(arch_evt->interrupted, &regs->a6,
> +		       sizeof(*arch_evt->interrupted));

[Severity: High]
Does copying sizeof(*arch_evt->interrupted) bytes across &regs->a6 trip
CONFIG_FORTIFY_SOURCE?

In struct pt_regs, a6 and a7 are individual unsigned long fields rather
than a 2-register struct_group, so __builtin_object_size(&regs->a6, 1) is
sizeof(unsigned long) while sizeof(*arch_evt->interrupted) is
2 * sizeof(unsigned long).  With CONFIG_FORTIFY_SOURCE=y, the first
memcpy() fails the build with __write_overflow_field() under
CONFIG_WERROR=y (and the second triggers __read_overflow2_field() under
W=1), or triggers a runtime fortify WARN_ONCE() inside NMI context when
an SSE event arrives.  Copying and comparing a6 and a7 directly by field
avoids the field-spanning memcpy().

> diff --git a/arch/riscv/kernel/sbi_sse_entry.S b/arch/riscv/kernel/sbi_sse_entry.S
> new file mode 100644
> index 0000000000000..1e38fbb1c2a42
> --- /dev/null
> +++ b/arch/riscv/kernel/sbi_sse_entry.S
> @@ -0,0 +1,237 @@

[ ... ]

> +	asm_per_cpu_with_cpu t2 __sbi_sse_entry_task t1 t3
> +	REG_L tp, 0(t2)
> +	bnez tp, .Lcurrent_task_found
> +
> +	/* __switch_to() marks its transition window with a NULL entry task. */
> +	REG_L tp, PT_TP(sp)
> +
> +.Lcurrent_task_found:
> +	/* Nested exceptions temporarily replace these with the SSE stack. */
> +	REG_L s6, TASK_TI_KERNEL_SP(tp)

[Severity: High]
Can a nested higher-priority SSE event see tp == NULL here when an SSE
interrupts __switch_to()?

__switch_to() zeroes __sbi_sse_entry_task while switching from prev to
next, relying on handle_sse() recovering current from the interrupted tp
register saved in PT_TP(sp).  However, REG_L tp, 0(t2) loads NULL directly
into the live tp register before falling back to PT_TP(sp), and leaves
__sbi_sse_entry_task NULL while do_sse() runs:

__switch_to()
  __sbi_sse_entry_task = NULL
  <low-priority SSE>
    handle_sse()
      REG_S tp, PT_TP(sp)
      REG_L tp, 0(t2)            # tp becomes 0
      <high-priority SSE>
        handle_sse()
          REG_S tp, PT_TP(sp)    # saves 0
          REG_L tp, 0(t2)        # still 0
          REG_L tp, PT_TP(sp)    # reloads 0
          REG_L s6, TASK_TI_KERNEL_SP(tp)  <-- NULL pointer dereference

The same NULL tp window also occurs if do_sse() takes a nested synchronous
exception while __sbi_sse_entry_task is still NULL, because
handle_exception() swaps tp with CSR_SCRATCH (which handle_sse() cleared
to 0) for a few instructions before restoring tp.

Loading __sbi_sse_entry_task into a temporary register before overwriting
tp, or updating __sbi_sse_entry_task in lockstep with move tp, a1 in
__switch_to(), avoids exposing a NULL tp to nested SSE events.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261119120824.13969-13-zhangzhanpeng.jasper@bytedance.com?part=2

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [PATCH v13 10/11] selftests/riscv: add SSE test module
  2026-11-19 12:08 ` [PATCH v13 10/11] selftests/riscv: add SSE test module Zhanpeng Zhang
@ 2026-11-19 12:37   ` sashiko-bot
  0 siblings, 0 replies; 22+ messages in thread
From: sashiko-bot @ 2026-11-19 12:37 UTC (permalink / raw)
  To: Zhanpeng Zhang; +Cc: linux-perf-users

Thank you for your contribution! Sashiko AI review found 7 potential
issues to consider:
- [Medium] selftests/riscv: sse: deadlock between migrate_disable() and
  cpus_read_lock()
- [Medium] selftests/riscv: sse: fast test ignores errors and races on
  1-jiffy timeout
- [Medium] selftests/riscv: sse: stress test treats -EEXIST as failure
  instead of skip
- [Medium] selftests/riscv: sse module Makefile defaults KDIR to host
  /lib/modules
- [Low] selftests/riscv: sse: stale pointers left in priority_test_arg
  on teardown
- [Low] selftests/riscv: sse: swapped log args in sse_test_handler() and
  dead field
- [Low] selftests/riscv: run_sse_test.sh added with non-executable mode
  0644
--

commit e938d689e90e8ec77bc093b9edf15fcf64552952
Author: Clément Léger <cleger@rivosinc.com>

selftests/riscv: add SSE test module

This commit adds a kernel module and kselftest runner script to test the
RISC-V SBI Supervisor Software Events (SSE) framework, including fast
injection, event priority chaining, and stress modes.

> diff --git a/tools/testing/selftests/riscv/sse/module/Makefile b/tools/testing/selftests/riscv/sse/module/Makefile
> new file mode 100644
> index 0000000000000..eac4b1c6228bb
> --- /dev/null
> +++ b/tools/testing/selftests/riscv/sse/module/Makefile
> @@ -0,0 +1,22 @@
> +ifneq ($(CONFIG_RISCV_SBI_SSE),)
> +obj-m += riscv_sse_test.o
> +endif
> +
> +ifndef KERNELRELEASE
> +
> +TESTMODS_DIR := $(realpath $(dir $(abspath $(lastword $(MAKEFILE_LIST)))))
> +KDIR ?= /lib/modules/$(shell uname -r)/build
> +
> +# Ensure that KDIR exists, otherwise skip the compilation
> +modules:
> +ifneq ("$(wildcard $(KDIR))", "")
> +	$(Q)$(MAKE) -C $(KDIR) modules KBUILD_EXTMOD=$(TESTMODS_DIR)
> +endif

[Severity: Medium]
Does defaulting KDIR to /lib/modules/$(shell uname -r)/build break
cross-compilation and in-tree or O= builds?

When cross-compiling with ARCH=riscv on a non-RISC-V host where
/lib/modules/$(uname -r)/build exists, Kbuild runs against the host kernel
headers with ARCH=riscv and fails.  When /lib/modules/$(uname -r)/build is
absent or points to a running kernel without CONFIG_RISCV_SBI_SSE,
riscv_sse_test.ko is silently skipped instead of building against the
target kernel tree.  Other selftest modules (such as
tools/testing/selftests/mm/page_frag/Makefile) resolve KDIR from O,
KBUILD_OUTPUT, or $(abs_srctree).

> diff --git a/tools/testing/selftests/riscv/sse/module/riscv_sse_test.c b/tools/testing/selftests/riscv/sse/module/riscv_sse_test.c
> new file mode 100644
> index 0000000000000..cc5c2e46f2fdb
> --- /dev/null
> +++ b/tools/testing/selftests/riscv/sse/module/riscv_sse_test.c
[ ... ]
> +static int sse_test_handler(u32 evt, void *arg, struct pt_regs *regs)
> +{
> +	int ret = 0;
> +	struct fast_test_arg *targ = arg;
> +	u32 test_evt;
> +	int cpu;
> +
> +	atomic_inc(&sse_test_handler_count);
> +
> +	/* Pairs with the argument publication in sse_run_fast_test_cpu(). */
> +	if (!smp_load_acquire(&targ->args_ready)) {
> +		sse_err("Received SSE event %x before its test arguments were published\n",
> +			evt);
> +		ret = -EINVAL;
> +		goto complete;
> +	}
> +
> +	test_evt = READ_ONCE(targ->evt);
> +	cpu = READ_ONCE(targ->cpu);
> +
> +	if (evt != test_evt) {
> +		sse_err("Received SSE event id %x instead of %x\n", test_evt, evt);
> +		ret = -EINVAL;
> +	}

[Severity: Low]
Are test_evt and evt swapped in this sse_err() call?  evt is the received
event ID and test_evt is the expected one.

Also, struct priority_test_arg below defines an unused check_func member.

[ ... ]

> +static int sse_run_fast_test_cpu(struct fast_test_arg *test_arg,
> +				 struct sse_event *event, u32 evt, int cpu)
> +{
> +	unsigned long timeout;
> +	int ret;
> +
> +	WRITE_ONCE(test_arg->completion, false);
> +	WRITE_ONCE(test_arg->args_ready, false);
> +	WRITE_ONCE(test_arg->evt, evt);
> +	WRITE_ONCE(test_arg->cpu, cpu);
> +	/* Publish all arguments before firmware can inject on another hart. */
> +	smp_store_release(&test_arg->args_ready, true);
> +
> +	ret = sse_test_inject_event(event, evt, cpu);
> +	if (ret) {
> +		sse_err("event %s injection failed, err %d\n",
> +			sse_evt_name(evt), ret);
> +		return ret;
> +	}
> +
> +	timeout = jiffies + HZ / 100;
> +	/* We can not use <linux/completion.h> since they are not NMI safe */
> +	/* Pairs with the handler's completion publication. */
> +	while (!smp_load_acquire(&test_arg->completion) &&
> +	       time_before(jiffies, timeout))
> +		cpu_relax();
> +	/* Acquire the handler's checks even if the loop observed a timeout. */
> +	if (!smp_load_acquire(&test_arg->completion)) {
> +		sse_err("Failed to wait for event %s completion on CPU %d\n",
> +			sse_evt_name(evt), cpu);
> +		return -ETIMEDOUT;
> +	}
> +
> +	return sse_test_wait_not_running_on_cpu(evt, cpu);
> +}
> +
> +static void sse_run_fast_test(struct fast_test_arg *test_arg,
> +			      struct sse_event *event, u32 evt)
> +{
> +	int cpu;
> +
> +	if (sse_event_is_global(evt)) {
> +		/* Keep the selected target online through injection and completion. */
> +		cpu_hotplug_disable();
> +		for_each_online_cpu(cpu) {
> +			if (sse_run_fast_test_cpu(test_arg, event, evt, cpu))
> +				break;
> +		}
> +		cpu_hotplug_enable();
> +		return;
> +	}
> +
> +	guard(cpus_read_lock)();
> +	for_each_online_cpu(cpu) {
> +		if (sse_run_fast_test_cpu(test_arg, event, evt, cpu))
> +			return;
> +	}
> +}
> +
> +static void sse_test_injection_fast(void)
> +{
[ ... ]
> +		for (j = 0; j < RUN_LOOP_COUNT; j++)
> +			sse_run_fast_test(&fast_test_arg, event, evt);

[Severity: Medium]
Can a timeout in sse_run_fast_test_cpu() race with an in-flight handler on
fast_test_arg across the remaining RUN_LOOP_COUNT iterations?

With CONFIG_HZ=100, HZ / 100 is 1 jiffy, which can expire on the next
timer tick while waiting for a remote CPU.  On timeout,
sse_run_fast_test_cpu() returns without calling
sse_test_wait_not_running_on_cpu(), and sse_run_fast_test() discards the
error so sse_test_injection_fast() immediately starts the next iteration
and overwrites the shared static fast_test_arg while the delayed handler
may still be running (or stalls module load if all 1,000 iterations time
out).  In addition, non-timeout errors in
sse_test_wait_not_running_on_cpu() (-ENOMEM or smp_call_function_single()
and sse_event_attr_read() failures) return without logging sse_err(), so
run_sse_test.sh will not detect them.

[ ... ]

> +static void sse_test_injection_priority_arg(struct priority_test_arg *args, unsigned int args_size,
> +					    sse_event_handler_fn handler, const char *test_name)
> +{
> +	unsigned int i;
> +	unsigned long timeout;
> +	int ret;
> +	int target_cpu;
> +	struct sse_event *event;
> +	struct priority_test_arg *arg, *first_arg = NULL, *prev_arg = NULL;
> +
> +	pr_info("Starting SSE priority test (%s)\n", test_name);
> +	/* Keep the complete priority chain on one CPU. */
> +	migrate_disable();
> +	target_cpu = smp_processor_id();
> +
> +	for (i = 0; i < args_size; i++) {
> +		arg = &args[i];
> +
> +		if (!sse_test_can_inject_event(arg->evt))
> +			continue;
> +
> +		WRITE_ONCE(arg->called, false);
> +		WRITE_ONCE(arg->next_evt_arg, NULL);
> +		WRITE_ONCE(arg->event, NULL);
> +		WRITE_ONCE(arg->enable_attempted, false);
> +
> +		event = sse_test_event_register(arg->evt, arg->prio, handler,
> +						(void *)arg);

[Severity: Medium]
Can calling sse_test_event_register() and the other SSE helpers inside
migrate_disable() deadlock with CPU hotplug?

migrate_disable() pins the task to target_cpu without blocking CPU
hotplug, while sse_event_register(), sse_event_set_target_cpu(),
sse_event_enable(), sse_event_disable(), and sse_event_unregister() each
acquire cpus_read_lock() internally:

CPU0 (target_cpu)                 CPU1
-----------------                 ----
sse_test_injection_priority_arg()
  migrate_disable()
                                  _cpu_down(target_cpu)
                                    cpus_write_lock()
                                    ...
                                    sched_cpu_wait_empty()
                                      balance_hotplug_wait()
  sse_test_event_register()
    sse_event_register()
      mutex_lock(&sse_mutex)
      cpus_read_lock()

 *** DEADLOCK ***

CPU1 holds cpus_write_lock() and waits in balance_hotplug_wait() for
pinned tasks on target_cpu to call migrate_enable(), while CPU0 is pinned
to target_cpu and blocks in cpus_read_lock() (also holding sse_mutex).
Surrounding the migrate_disable() section with cpu_hotplug_disable() and
cpu_hotplug_enable() avoids the lock inversion and prevents
sse_cpu_teardown() from tearing down SSE on target_cpu mid-test.

[ ... ]

> +release_events:
> +
> +	for (i = 0; i < args_size; i++) {
> +		arg = &args[i];
> +		event = arg->event;
> +		if (!event)
> +			continue;
> +
> +		ret = sse_test_wait_not_running_on_cpu(arg->evt, target_cpu);
> +		if (ret)
> +			sse_err("Event %s did not complete, err %d\n",
> +				sse_evt_name(arg->evt), ret);
> +
> +		if (arg->enable_attempted) {
> +			ret = sse_event_disable(event);
> +			if (ret) {
> +				sse_err("Failed to disable event %s, err %d\n",
> +					sse_evt_name(arg->evt), ret);
> +				sse_test_queue_cleanup(event);
> +				WRITE_ONCE(arg->event, NULL);
> +				WRITE_ONCE(arg->enable_attempted, false);
> +				continue;
> +			}
> +		}
> +
> +		ret = sse_test_event_unregister(event);
> +		if (ret) {
> +			sse_err("Failed to unregister event %s, err %d\n",
> +				sse_evt_name(arg->evt), ret);
> +			continue;
> +		}
> +
> +		WRITE_ONCE(arg->event, NULL);
> +		WRITE_ONCE(arg->enable_attempted, false);
> +	}

[Severity: Low]
Does release_events leave stale pointers in priority_test_arg when disable
or unregister fails?

When sse_test_event_unregister() fails, it queues event for deferred
freeing via sse_test_queue_cleanup(), but the error branch jumps to the
next iteration before clearing arg->event and arg->enable_attempted.
Also, release_events never clears arg->next_evt_arg, so if an event fails
to disable or unregister and fires late after its successor in args[] has
been unregistered, the handler will read a non-NULL targ->next_evt_arg and
signal the unregistered successor event.

[ ... ]

> +static int sse_stress_test_layer(int layer)
> +{
[ ... ]
> +	event = sse_test_event_register(stress_ctx.evt_id, 0,
> +					stress_handlers[layer], &stress_ctx);
> +	if (IS_ERR(event)) {
> +		sse_err("Failed to register stress event, err %ld\n",
> +			PTR_ERR(event));
> +		return PTR_ERR(event);
> +	}
[ ... ]
> +static void sse_stress_test_all_layers(void)
> +{
> +	int i, ret;
> +
> +	pr_info("Starting SSE stress tests: duration=%d ms, interval=%d ns\n",
> +		STRESS_DURATION_MS, STRESS_INJECT_NS);
> +
> +	for (i = 0; i < ARRAY_SIZE(stress_handlers); i++) {
> +		ret = sse_stress_test_layer(i);
> +		if (ret) {
> +			sse_err("Stress layer %d failed, err %d\n", i, ret);
> +			break;
> +		}

[Severity: Medium]
Should sse_stress_test_layer() skip rather than fail when
sse_test_event_register() returns -EEXIST?

Unlike sse_test_injection_fast() and sse_test_injection_priority_arg(),
sse_stress_test_layer() and sse_stress_test_all_layers() log sse_err() on
-EEXIST.  Because run_sse_test.sh checks for FAILED: before SKIP:,
running with stress=1 or stress=2 when
SBI_SSE_EVENT_LOCAL_SOFTWARE_INJECTED is already registered exits with
failure (1) instead of ksft_skip (4).

> diff --git a/tools/testing/selftests/riscv/sse/run_sse_test.sh b/tools/testing/selftests/riscv/sse/run_sse_test.sh
> new file mode 100644
> index 0000000000000..e70a2fd14b05e
> --- /dev/null
> +++ b/tools/testing/selftests/riscv/sse/run_sse_test.sh
> @@ -0,0 +1,59 @@
> +#!/bin/bash
> +# SPDX-License-Identifier: GPL-2.0

[Severity: Low]
Should run_sse_test.sh have mode 0755 instead of 0644?

Because the script is not executable, invoking ./run_sse_test.sh directly
fails with permission denied, and tools/testing/selftests/kselftest/runner.sh
logs a warning and takes its non-executable fallback path, which drops
kselftest_cmd_args instead of forwarding module arguments such as stress=1
to insmod.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261119120824.13969-13-zhangzhanpeng.jasper@bytedance.com?part=10

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [PATCH v13 04/11] drivers: firmware: add riscv SSE support
  2026-11-19 12:08 ` [PATCH v13 04/11] drivers: firmware: add riscv SSE support Zhanpeng Zhang
@ 2026-11-19 12:38   ` sashiko-bot
  0 siblings, 0 replies; 22+ messages in thread
From: sashiko-bot @ 2026-11-19 12:38 UTC (permalink / raw)
  To: Zhanpeng Zhang; +Cc: linux-perf-users

Thank you for your contribution! Sashiko AI review found 8 potential
issues to consider:
- [High] firmware: riscv_sbi_sse: sse_pm_notifier() unmasks SSE on
  unready or offlining CPUs
- [High] firmware: riscv_sbi_sse: sse_event_cleanup() leaks retained SSE
  events
- [High] firmware: riscv_sbi_sse: missing IRQ and teardown sync in
  sse_event_*_local()
- [High] firmware: riscv_sbi_sse: sse_cpu_teardown() mishandles CPU-up
  rollback
- [Medium] firmware: riscv_sbi_sse: missing hart and CPU bounds check in
  sse_event_register()
- [Medium] firmware: riscv_sbi_sse: sse_event_setup_all_cpus() can
  return -EOPNOTSUPP with enabled state
- [Medium] firmware: riscv_sbi_sse: sse_reboot_notifier() races with CPU
  hotplug callbacks
- [Medium] firmware: riscv_sbi_sse: broken rollback in
  sse_event_set_target_cpu_nolock()
--

commit 62013a3fca4c71adabe66ac29e99aa0bab92f5a1
Author: Clément Léger <cleger@rivosinc.com>

drivers: firmware: add riscv SSE support

This commit adds a driver-level interface for RISC-V Supervisor Software
Events (SSE) to manage global and per-CPU event registration, enabling,
disabling, and CPU hotplug and reboot teardown.

> diff --git a/drivers/firmware/riscv/riscv_sbi_sse.c b/drivers/firmware/riscv/riscv_sbi_sse.c
> new file mode 100644
> index 0000000000000..0cb738bab6c39
> --- /dev/null
> +++ b/drivers/firmware/riscv/riscv_sbi_sse.c
> @@ -0,0 +1,1219 @@

[ ... ]

> +static int sse_sbi_event_func(struct sse_event *event, unsigned long func)
> +{
> +	struct sbiret ret;
> +	u32 evt = event->evt_id;
> +	struct sse_registered_event *reg_evt = sse_get_reg_evt(event);
> +
> +	ret = sbi_ecall(SBI_EXT_SSE, func, evt, 0, 0, 0, 0, 0);
> +	if (ret.error) {
> +		pr_warn("Failed to execute func %lx, event %x, error %ld\n",
> +			func, evt, ret.error);
> +		return sse_err_map_linux_errno(ret.error);
> +	}
> +
> +	if (func == SBI_SSE_EVENT_DISABLE)
> +		reg_evt->is_enabled = false;
> +	else if (func == SBI_SSE_EVENT_ENABLE)
> +		reg_evt->is_enabled = true;
> +
> +	return 0;
> +}
> +
> +int sse_event_disable_local(struct sse_event *event)
> +{
> +	if (!sse_event_is_global(event->evt_id))
> +		lockdep_assert_preemption_disabled();
> +
> +	if (!sse_get_reg_evt(event)->is_enabled)
> +		return 0;
> +
> +	return sse_sbi_event_func(event, SBI_SSE_EVENT_DISABLE);
> +}
> +EXPORT_SYMBOL_GPL(sse_event_disable_local);
> +
> +int sse_event_enable_local(struct sse_event *event)
> +{
> +	struct sse_registered_event *reg_evt = sse_get_reg_evt(event);
> +	int ret;
> +
> +	if (!sse_event_is_global(event->evt_id))
> +		lockdep_assert_preemption_disabled();
> +
> +	rcu_read_lock();
> +	if (READ_ONCE(sse_shutting_down)) {
> +		ret = -ESHUTDOWN;
> +		goto out;
> +	}
> +
> +	if (!reg_evt->is_registered) {
> +		ret = -EINVAL;
> +		goto out;
> +	}
> +
> +	if (reg_evt->is_enabled) {
> +		ret = 0;
> +		goto out;
> +	}
> +
> +	ret = sse_sbi_event_func(event, SBI_SSE_EVENT_ENABLE);
> +out:
> +	rcu_read_unlock();
> +
> +	return ret;
> +}
> +EXPORT_SYMBOL_GPL(sse_event_enable_local);

[Severity: High]
Can a hardirq interrupting sse_event_disable_local(),
sse_event_enable_local(), or sse_teardown_event() observe stale
reg_evt->is_enabled state and issue conflicting SBI calls?

sse_event_disable_local(), sse_event_enable_local(), and
sse_cpu_teardown() run with local interrupts enabled, and
sse_sbi_event_func() updates reg_evt->is_enabled only after sbi_ecall()
returns:

sse_cpu_teardown()
  sse_teardown_event()
    if (reg_evt->is_enabled)
      sse_event_disable_local()
        <hardirq>
          perf_mux_hrtimer_handler()
            ...
              sse_event_disable_local()
                sbi_ecall(..., SBI_SSE_EVENT_DISABLE, ...)
                reg_evt->is_enabled = false;
        sse_sbi_event_func(..., SBI_SSE_EVENT_DISABLE)
          sbi_ecall(..., SBI_SSE_EVENT_DISABLE, ...)  <-- returns -EINVAL

During reboot or kexec, sse_reboot_nb runs at default priority 0 while
CPUHP_AP_PERF_ONLINE and perf timer interrupts are still active.  When the
second SBI_SSE_EVENT_DISABLE fails with -EINVAL, sse_teardown_event()
aborts before unregistering the event and sets sse_teardown_failed,
causing sse_reboot_notifier() to panic on kexec.

The same window exists when an interrupt nests inside normal
sse_event_disable_local() or sse_event_enable_local() calls, or when an
interrupt calls sse_event_enable_local() after sse_teardown_event() (on
normal CPU offline where sse_shutting_down is false) or
sse_event_set_target_cpu_nolock() disables an event while
reg_evt->is_registered is still true.

Also, sse_event_enable_local() calls sse_get_reg_evt(event) (which calls
smp_processor_id() for local events) before
lockdep_assert_preemption_disabled(), and sse_restore_local_events()
restores local events from event->enable_requested rather than the
per-CPU reg_evt->is_enabled state that was active before
sse_cpu_teardown() began.

[ ... ]

> +static int sse_event_set_target_cpu_nolock(struct sse_event *event,
> +					   unsigned int cpu)
> +{
> +	unsigned long hart_id, old_hart_id;
> +	struct sse_registered_event *reg_evt = event->global;
> +	u32 evt = event->evt_id;
> +	unsigned int old_cpu;
> +	bool was_enabled;
> +	int ret;
> +
> +	if (!sse_event_is_global(evt))
> +		return -EINVAL;
> +
> +	if (cpu >= nr_cpu_ids || !cpu_online(cpu))
> +		return -EINVAL;
> +	hart_id = cpuid_to_hartid_map(cpu);
> +	old_cpu = event->cpu;
> +	old_hart_id = cpuid_to_hartid_map(old_cpu);
> +
> +	was_enabled = reg_evt->is_enabled;
> +	if (was_enabled) {
> +		ret = sse_event_disable_local(event);
> +		if (ret)
> +			return ret;
> +	}
> +
> +	ret = sse_event_attr_set_nolock(reg_evt, SBI_SSE_ATTR_PREFERRED_HART,
> +					hart_id);
> +	if (ret == 0)
> +		sse_global_event_update_cpu(event, cpu);
> +
> +	if (was_enabled) {
> +		int enable_ret;
> +
> +		enable_ret = sse_event_enable_local(event);
> +		if (enable_ret) {
> +			int rollback_ret;
> +
> +			/*
> +			 * The preferred hart was already changed. Restore both the
> +			 * firmware attribute and Linux's cached target before reporting
> +			 * the failed migration.
> +			 */
> +			rollback_ret = sse_event_attr_set_nolock(reg_evt,
> +								 SBI_SSE_ATTR_PREFERRED_HART,
> +								 old_hart_id);
> +			if (!rollback_ret) {
> +				sse_global_event_update_cpu(event, old_cpu);
> +				rollback_ret = sse_event_enable_local(event);
> +			}
> +			if (!rollback_ret)
> +				return enable_ret;
> +
> +			pr_warn("Failed to restore global event %x to CPU %u: %d\n",
> +				evt, old_cpu, rollback_ret);
> +			event->enable_requested = false;
> +			return enable_ret;
> +		}
> +	}
> +
> +	return ret;
> +}

[Severity: Medium]
Does the rollback in sse_event_set_target_cpu_nolock() misbehave when
setting SBI_SSE_ATTR_PREFERRED_HART or rolling it back fails?

If sse_event_attr_set_nolock() fails (ret != 0) on an enabled event and
re-enabling the event also fails (enable_ret != 0), the code enters the
rollback block even though the preferred hart was never changed,
redundantly writing old_hart_id and returning enable_ret instead of ret.

Conversely, if setting the new preferred hart succeeds (ret == 0),
sse_global_event_update_cpu(event, cpu) updates event->cpu to the new
CPU.  If both sse_event_enable_local() and the rollback
sse_event_attr_set_nolock(..., old_hart_id) then fail, event->cpu remains
set to the new CPU while the function returns enable_ret, causing
sse_cpu_teardown() to abort the CPU offline even though the event has
already been moved to next_cpu and disabled.

[ ... ]

> +void sse_event_cleanup(struct sse_event *event)
> +{
> +	guard(mutex)(&sse_mutex);
> +	guard(cpus_read_lock)();
> +
> +	if (event->cleanup_pending)
> +		return;
> +
> +	/*
> +	 * Firmware may still enter the old callback after disable or unregister
> +	 * fails. Publish a core-owned callback, then wait before the client frees
> +	 * its callback data.
> +	 */
> +	event->cleanup_pending = true;
> +	rcu_assign_pointer(event->handler, sse_cleanup_event_handler);
> +	synchronize_rcu();
> +}
> +EXPORT_SYMBOL_GPL(sse_event_cleanup);
> +
> +static void sse_release_cleanup_event(struct sse_event *event)
> +{
> +	if (!event->cleanup_pending || sse_event_is_registered(event))
> +		return;
> +
> +	list_del(&event->list);
> +	sse_event_free(event);
> +}
> +
> +static int sse_event_setup_all_cpus(struct sse_event *event,
> +				    unsigned long func,
> +				    unsigned long rollback_func)
> +{
> +	struct sse_per_cpu_evt cpu_evt;
> +	int rollback_ret;
> +	int ret;
> +
> +	cpu_evt.event = event;
> +	atomic_set(&cpu_evt.first_error, 0);
> +	atomic_set(&cpu_evt.nonfallback_error, 0);
> +	cpumask_clear(&cpu_evt.changed);
> +	cpu_evt.func = func;
> +	on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1);
> +	/* IRQ fallback is safe only if every failing CPU reports unsupported. */
> +	ret = atomic_read(&cpu_evt.nonfallback_error);
> +	if (!ret)
> +		ret = atomic_read(&cpu_evt.first_error);
> +	/*
> +	 * A previous attempt may already have changed some CPUs. Roll back only
> +	 * instances changed by this invocation.
> +	 */
> +	if (ret) {
> +		cpu_evt.func = rollback_func;
> +		atomic_set(&cpu_evt.first_error, 0);
> +		atomic_set(&cpu_evt.nonfallback_error, 0);
> +		on_each_cpu_mask(&cpu_evt.changed, sse_event_per_cpu_func, &cpu_evt, 1);
> +
> +		rollback_ret = atomic_read(&cpu_evt.nonfallback_error);
> +		if (!rollback_ret)
> +			rollback_ret = atomic_read(&cpu_evt.first_error);
> +
> +		/* A rollback failure leaves the firmware state uncertain. */
> +		return rollback_ret ?: ret;
> +	}
> +
> +	return 0;
> +}

[Severity: High]
Does sse_event_cleanup() leak struct sse_event and leave its evt_id
permanently blocked on the events list?

sse_event_cleanup() sets event->cleanup_pending = true and swaps the
handler, but never tears down remaining registrations or calls
sse_release_cleanup_event(event).

If all registrations were already removed before cleanup_pending was set
(for example, when sse_reboot_notifier() tore down all events and a
client's later sse_event_unregister() returned -ESHUTDOWN, or when the
failing CPU went offline in the window between sse_event_unregister()
dropping sse_mutex and sse_event_cleanup() acquiring it),
sse_release_cleanup_event() during sse_cpu_teardown() saw
!event->cleanup_pending and did nothing.  Once sse_event_cleanup() sets
cleanup_pending = true, nothing calls sse_release_cleanup_event(event)
again, leaking event and causing future sse_event_register() calls for
that evt_id to fail with -EEXIST.

Similarly, if a client calls sse_event_cleanup(event) directly after
sse_event_disable(event) fails on one CPU without calling
sse_event_unregister(event), sse_event_cleanup() neither clears
event->enable_requested nor unregisters the event on the online CPUs
where disabling succeeded, leaving those CPUs registered and preventing
sse_release_cleanup_event() from ever freeing the event.

[Severity: Medium]
Can sse_event_setup_all_cpus() return -EOPNOTSUPP to sse_event_enable()
while leaving the event enabled on some CPUs?

If the forward enable pass fails with a non-fallback error (such as -EIO)
after enabling the event on CPUs in cpu_evt.changed, and the rollback
disable pass fails with -EOPNOTSUPP, "return rollback_ret ?: ret;"
overwrites ret with -EOPNOTSUPP.  Likewise, if any CPU was already
enabled before sse_event_enable() ran (so changed was false on that CPU)
and another CPU fails with -EOPNOTSUPP, only cpu_evt.changed is rolled
back.  Unlike sse_event_register(), sse_event_enable() does not check
sse_event_is_enabled(event), so it returns -EOPNOTSUPP and allows the
client to select an IRQ fallback while firmware still has SSE enabled.

[ ... ]

> +	if (sse_event_is_global(evt)) {
> +		unsigned long preferred_hart;
> +
> +		ret = sse_event_attr_get_no_lock(event->global,
> +						 SBI_SSE_ATTR_PREFERRED_HART,
> +						 &preferred_hart);
> +		if (ret)
> +			return sse_register_failed(event, ret);
> +
> +		cpu = riscv_hartid_to_cpuid(preferred_hart);
> +		if (cpu < 0 || !cpu_online(cpu)) {
> +			cpu = cpumask_first(cpu_online_mask);

[Severity: Medium]
Can cpu_online(cpu) read past nr_cpu_ids when firmware reports an invalid
or unpopulated preferred_hart?

In arch/riscv/kernel/smp.c, __cpuid_to_hartid_map[NR_CPUS] initializes
all NR_CPUS slots to INVALID_HARTID, and riscv_hartid_to_cpuid() scans up
to NR_CPUS:

arch/riscv/kernel/smp.c:riscv_hartid_to_cpuid() {
	for (i = 0; i < NR_CPUS; i++)
		if (cpuid_to_hartid_map(i) == hartid)
			return i;
}

When nr_cpu_ids < NR_CPUS and firmware returns preferred_hart ==
INVALID_HARTID (or a hart mapped at index >= nr_cpu_ids),
riscv_hartid_to_cpuid() returns an index >= nr_cpu_ids.  Passing that cpu
to cpu_online(cpu) without checking preferred_hart == INVALID_HARTID or
cpu >= nr_cpu_ids (unlike sse_event_set_target_cpu_nolock()) reads past
nr_cpu_ids in cpu_online_mask and triggers the WARN_ON_ONCE() in
cpumask_check() when CONFIG_DEBUG_PER_CPU_MAPS is enabled.

[ ... ]

> +static int sse_cpu_teardown(unsigned int cpu)
> +{
> +	/* Only a regular CPU-offline callback may abort the CPUHP operation. */
> +	bool regular_offline = READ_ONCE(sse_available) &&
> +			       !READ_ONCE(sse_shutting_down);
> +	unsigned int next_cpu;
> +	struct sse_event *event, *tmp;
> +	int first_error = 0;
> +	int ret;
> +
> +	/* Do not dismantle CPU state while firmware can still deliver SSE. */
> +	ret = sse_events_mask();
> +	if (ret) {
> +		pr_warn("Failed to mask SSE on CPU %u during teardown: %d\n",
> +			cpu, ret);
> +		if (READ_ONCE(sse_shutting_down))
> +			atomic_set(&sse_teardown_failed, 1);
> +		/* CPUHP installation rollback and state removal cannot fail. */
> +		return regular_offline ? ret : 0;
> +	}
> +
> +	list_for_each_entry_safe(event, tmp, &events, list) {
> +		if (sse_event_is_global(event->evt_id))
> +			continue;
> +
> +		scoped_guard(preempt)
> +			ret = sse_teardown_event(event, cpu);
> +		if (ret && !first_error)
> +			first_error = ret;
> +		sse_release_cleanup_event(event);
> +		if (ret && regular_offline)
> +			goto restore_cpu;
> +	}
> +
> +	list_for_each_entry_safe(event, tmp, &events, list) {
> +		if (!sse_event_is_global(event->evt_id) || event->cpu != cpu)
> +			continue;
> +
> +		/*
> +		 * cpuhp_remove_state() invokes teardown while every CPU remains
> +		 * online. Do not migrate to a CPU whose callback may have run.
> +		 */
> +		if (READ_ONCE(sse_shutting_down) || event->cleanup_pending) {
> +			ret = sse_teardown_event(event, cpu);
> +		} else {
> +			next_cpu = cpumask_any_but(cpu_online_mask, cpu);
> +			if (next_cpu >= nr_cpu_ids) {
> +				ret = sse_teardown_event(event, cpu);
> +			} else {
> +				ret = sse_event_set_target_cpu_nolock(event, next_cpu);
> +				if (ret)
> +					pr_warn("Failed to migrate global event %x from CPU %u: %d\n",
> +						event->evt_id, cpu, ret);
> +			}
> +		}
> +
> +		if (ret && !first_error)
> +			first_error = ret;
> +		sse_release_cleanup_event(event);
> +		if (ret && regular_offline)
> +			goto restore_cpu;
> +	}
> +
> +	if (first_error) {
> +		/* An offline CPU is not revisited when this CPUHP state is removed. */
> +		atomic_set(&sse_teardown_failed, 1);
> +	}
> +
> +	return 0;
> +
> +restore_cpu:
> +	/*
> +	 * CPUHP leaves this CPU online when teardown returns an error. Restore
> +	 * every client-owned local event before making SSE delivery visible again.
> +	 */
> +	ret = sse_restore_local_events(cpu);
> +	if (!ret)
> +		ret = sse_events_unmask();
> +	if (ret) {
> +		atomic_set(&sse_teardown_failed, 1);
> +		pr_warn("Failed to restore SSE after aborting CPU %u offline: %d\n",
> +			cpu, ret);
> +	} else {
> +		pr_warn("Aborted CPU %u offline after SSE teardown failed: %d\n",
> +			cpu, first_error);
> +	}
> +
> +	return first_error;
> +}

[Severity: High]
What happens if sse_cpu_teardown() is invoked during a post-boot
_cpu_up() rollback after a later CPUHP state fails?

Because sse_available is true and sse_shutting_down is false after boot,
regular_offline evaluates to true even when sse_cpu_teardown() is rolling
back a failed CPU bringup.  If sse_teardown_event() or migration fails,
sse_cpu_teardown() jumps to restore_cpu, re-registers and re-enables
local events, unmasks SSE on the hart, leaves sse_teardown_failed unset
when restore succeeds, and returns first_error (triggering
WARN_ON_ONCE(st->rollback) in cpuhp_thread_fun()), while
cpuhp_up_callbacks() proceeds to take the CPU down via takedown_cpu()
with SSE still unmasked and events active in firmware.  Likewise, if
sse_events_mask() fails during a _cpu_up() rollback, sse_cpu_teardown()
returns a non-zero error without setting sse_teardown_failed.

> +
> +static int sse_pm_notifier(struct notifier_block *nb, unsigned long action,
> +			   void *data)
> +{
> +	int ret;
> +
> +	WARN_ON_ONCE(preemptible());
> +
> +	switch (action) {
> +	case CPU_PM_ENTER:
> +		ret = sse_events_mask();
> +		break;
> +	case CPU_PM_EXIT:
> +	case CPU_PM_ENTER_FAILED:
> +		if (READ_ONCE(sse_shutting_down))
> +			return NOTIFY_OK;
> +		ret = sse_events_unmask();
> +		break;
> +	default:
> +		return NOTIFY_DONE;
> +	}
> +
> +	if (ret)
> +		return notifier_from_errno(ret);
> +
> +	return NOTIFY_OK;
> +}

[Severity: High]
Can sse_pm_notifier() unmask SSE on a CPU where SSE has already been torn
down, has not been initialized yet, or was intentionally left masked?

CPU_PM_EXIT and CPU_PM_ENTER_FAILED only check sse_shutting_down before
calling sse_events_unmask(), without tracking whether SSE is currently
online and unmasked on this CPU.

During normal CPU hot-unplug, sse_cpu_teardown() masks SSE and tears down
events at CPUHP_AP_RISCV_SSE_ONLINE, after which the cpuhp thread parks
at CPUHP_AP_ONLINE_IDLE and the offlining CPU enters do_idle() ->
cpuidle_idle_call() while waiting for takedown_cpu() at
CPUHP_TEARDOWN_CPU.  Exiting a CPU_PM idle state in that window calls
sse_events_unmask(), re-enabling SSE delivery on the dying hart after its
events have been torn down.

The same unconditional unmask can happen on a CPU exiting a CPU_PM idle
state before CPUHP_AP_RISCV_SSE_ONLINE has run arch_sse_init_cpu() (during
sse_init() where cpu_pm_register_notifier() runs before
cpuhp_setup_state(), or during _cpu_up()), or after restore_cpu in
sse_cpu_teardown() intentionally left SSE masked because
sse_restore_local_events() failed.

[ ... ]

> +static int sse_reboot_notifier(struct notifier_block *nb, unsigned long action,
> +			       void *data)
> +{
> +	int ret;
> +
> +	scoped_guard(mutex, &sse_mutex) {
> +		if (!sse_shutting_down) {
> +			WRITE_ONCE(sse_shutting_down, true);
> +			ret = cpu_pm_unregister_notifier(&sse_pm_nb);
> +			if (ret) {
> +				pr_warn("Failed to unregister CPU PM notifier: %d\n",
> +					ret);
> +				atomic_set(&sse_teardown_failed, 1);
> +			}
> +			/* Drain CPU PM callbacks and client enables before teardown. */
> +			synchronize_rcu();
> +			cpuhp_remove_state(CPUHP_AP_RISCV_SSE_ONLINE);
> +		}
> +	}
> +
> +	/* Normal kexec preserves firmware state but discards old kernel memory. */
> +	if (kexec_in_progress && atomic_read(&sse_teardown_failed))
> +		panic("SSE teardown failed; refusing unsafe kexec");
> +
> +	return NOTIFY_OK;
> +}

[Severity: Medium]
Can sse_reboot_notifier() setting sse_shutting_down without
cpus_read_lock() race with an in-progress sse_cpu_teardown() and trigger
the kexec panic?

Normal CPU hotplug callbacks run under cpus_write_lock() without holding
sse_mutex, while reboot notifiers run before migrate_to_reboot_cpu()
disables CPU hotplug:

CPU0                                     CPU1
_cpu_down()
  sse_cpu_teardown()
    regular_offline = true;
                                         sse_reboot_notifier()
                                           mutex_lock(&sse_mutex)
                                           WRITE_ONCE(sse_shutting_down, true)
    sse_event_set_target_cpu_nolock()
      sse_event_enable_local()
        if (READ_ONCE(sse_shutting_down))
          return -ESHUTDOWN;
    goto restore_cpu;
    sse_restore_local_events()
      sse_event_enable_local()
        return -ESHUTDOWN;
    atomic_set(&sse_teardown_failed, 1)
                                           cpuhp_remove_state(...)
                                           if (kexec_in_progress &&
                                               atomic_read(&sse_teardown_failed))
                                             panic(...)  <-- false-positive panic

Holding cpus_read_lock() across the sse_shutting_down transition (or
using __cpuhp_remove_state_cpuslocked() under cpus_read_lock()) would
serialize sse_reboot_notifier() against active CPU hotplug callbacks.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261119120824.13969-13-zhangzhanpeng.jasper@bytedance.com?part=4

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [PATCH v13 03/11] riscv: sse: mask events during shutdown and kexec
  2026-11-19 12:08 ` [PATCH v13 03/11] riscv: sse: mask events during shutdown and kexec Zhanpeng Zhang
@ 2026-11-19 12:38   ` sashiko-bot
  0 siblings, 0 replies; 22+ messages in thread
From: sashiko-bot @ 2026-11-19 12:38 UTC (permalink / raw)
  To: Zhanpeng Zhang; +Cc: linux-perf-users

Thank you for your contribution! Sashiko AI review found 2 potential
issues to consider:
- [Medium] riscv: sse: unreachable !CONFIG_SMP panic_smp_self_stop() in
  reset.c
- [Medium] riscv: sse: unmasked SSE on other harts and preemptible
  machine_restart()
--

commit 39abe8f5f293bf4034ca8ab03a15f0ede47b20ab
Author: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>

riscv: sse: mask events during shutdown and kexec

This commit masks SBI Supervisor Software Events (SSE) on the local hart
during panic, CPU stop, restart, poweroff, and crash shutdown paths, and
rejects normal kexec from a crash kernel when firmware supports SSE.

Link: https://lore.kernel.org/r/as7-3PbWDxygadnJ@hu-himchau-blr.qualcomm.com/

> diff --git a/arch/riscv/kernel/reset.c b/arch/riscv/kernel/reset.c
> index 14eb08a6db855..fdab37e7ae52d 100644
> --- a/arch/riscv/kernel/reset.c
> +++ b/arch/riscv/kernel/reset.c
> @@ -6,6 +6,20 @@
>  #include <linux/efi.h>
>  #include <linux/reboot.h>
>  #include <linux/pm.h>
> +#include <linux/smp.h>
> +
> +#include <asm/sse.h>
> +
> +#ifndef CONFIG_SMP
> +void __noreturn panic_smp_self_stop(void)
> +{
> +	riscv_sse_mask_current_hart();
> +	local_irq_disable();
> +
> +	for (;;)
> +		cpu_relax();
> +}
> +#endif

[Severity: Medium]
Is this !CONFIG_SMP definition of panic_smp_self_stop() reachable, and does
a uniprocessor panic ever mask SSE on the panicking hart?

All callers of panic_smp_self_stop() in kernel/panic.c are gated by
panic_try_force_cpu() or panic_on_other_cpu().  When !CONFIG_SMP,
panic_try_force_cpu() is a stub returning false, and raw_smp_processor_id()
is always 0 while panic_cpu can only be PANIC_CPU_INVALID or 0, so
panic_on_other_cpu() is also always false.

Meanwhile, the panicking CPU calls panic_other_cpus_shutdown() ->
smp_send_stop() / crash_smp_send_stop(), both of which compile to the empty
smp_send_stop() stub in include/linux/smp.h when !CONFIG_SMP, leaving SSE
unmasked during vpanic().

>  static void __noreturn default_power_off(void)
>  {
> @@ -18,6 +32,8 @@ EXPORT_SYMBOL(pm_power_off);
>  
>  void machine_restart(char *cmd)
>  {
> +	riscv_sse_mask_current_hart();
> +
>  	/*
>  	 * UpdateCapsule() depends on the system being reset via ResetSystem().
>  	 */
> @@ -30,12 +46,14 @@ void machine_restart(char *cmd)
>  
>  void machine_halt(void)
>  {
> +	riscv_sse_mask_current_hart();
>  	do_kernel_power_off();
>  	default_power_off();
>  }
>  
>  void machine_power_off(void)
>  {
> +	riscv_sse_mask_current_hart();
>  	do_kernel_power_off();
>  	default_power_off();
>  }

[Severity: Medium]
Can secondary harts—or the calling task itself after migration—still receive
SSE events during machine_restart(), machine_halt(), and
machine_power_off()?

Unlike arm64, these functions do not call local_irq_disable() or
smp_send_stop().  On paths that bypass reboot_notifier_list and
migrate_to_reboot_cpu(), such as emergency_restart() ->
machine_emergency_restart() -> machine_restart(), secondary harts never
receive IPI_CPU_STOP and stay online with SSE unmasked.  Because preemption
is not disabled before riscv_sse_mask_current_hart(), a preemptible
emergency_restart() caller (for example from a workqueue) can also migrate
to another unmasked hart after masking the previous one.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261119120824.13969-13-zhangzhanpeng.jasper@bytedance.com?part=3

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [PATCH v13 07/11] perf: RISC-V: support callchains with SSE delivery
  2026-11-19 12:08 ` [PATCH v13 07/11] perf: RISC-V: support callchains with SSE delivery Zhanpeng Zhang
@ 2026-11-19 12:45   ` sashiko-bot
  0 siblings, 0 replies; 22+ messages in thread
From: sashiko-bot @ 2026-11-19 12:45 UTC (permalink / raw)
  To: Zhanpeng Zhang; +Cc: linux-perf-users

Thank you for your contribution! Sashiko AI review found 5 potential
issues to consider:
- [High] perf: riscv: missing sse_callchain_is_guest() check in
  riscv_perf_out_copy_user()
- [High] perf: riscv: missing current->mm and CSR_SATP checks in
  perf_callchain_user()
- [High] perf: riscv: unsafe entry unwind and stack OOB read in
  sse_kernel_regs_safe()
- [High] perf: riscv: NULL task pointer passed to walk_stackframe()
  crashes in fgraph
- [High] perf: riscv: NULL current->mm dereference in
  riscv_perf_out_copy_user()
--

commit 6c97040a7b3339404ccd87297e5e59c6632d3219
Author: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>

perf: RISC-V: support callchains with SSE delivery

This commit adds perf kernel, frame-pointer user, and DWARF user-stack
callchain sampling support on RISC-V when PMU overflow events are
delivered via SBI Supervisor Software Events (SSE).

Link: https://lore.kernel.org/r/as79MHtNYd-VbUug@hu-himchau-blr.qualcomm.com/

> diff --git a/arch/riscv/kernel/perf_callchain.c b/arch/riscv/kernel/perf_callchain.c
> index b465bc9eb870e..216f5bd14c001 100644
> --- a/arch/riscv/kernel/perf_callchain.c
> +++ b/arch/riscv/kernel/perf_callchain.c

[ ... ]

> +static bool sse_kernel_regs_safe(struct pt_regs *regs)
> +{
> +	unsigned long fp = frame_pointer(regs);
> +	unsigned long pc = instruction_pointer(regs);
> +	unsigned long sp = user_stack_pointer(regs);
> +
> +	if (!__kernel_text_address(pc))
> +		return false;
> +	if (!sse_addr_on_task_stack(sp, sizeof(unsigned long)))
> +		return false;
> +	if (fp < sizeof(struct stackframe))
> +		return false;
> +
> +	return sse_addr_on_task_stack(fp - sizeof(struct stackframe),
> +				      sizeof(struct stackframe));
> +}

[Severity: High]
Can sse_kernel_regs_safe() accept an entry/exit frame where s0 is not a
valid frame pointer and cause walk_stackframe() to read past the top of
the task stack?

In ret_from_exception, sp points to task_pt_regs(current) (stack +
THREAD_SIZE - PT_SIZE_ON_STACK) and s0 (fp) is set to stack +
THREAD_SIZE:

arch/riscv/kernel/entry.S:ret_from_exception {
    ...
	/* Save unwound kernel stack pointer in thread_info */
	addi s0, sp, PT_SIZE_ON_STACK
	REG_S s0, TASK_TI_KERNEL_SP(tp)
    ...
}

Because sse_addr_on_task_stack() allows fp - sizeof(struct stackframe)
all the way up to stack + THREAD_SIZE, an SSE interrupting this window
passes sse_kernel_regs_safe().  Under CONFIG_FRAME_POINTER,
walk_stackframe() then reads frame = (struct stackframe *)fp - 1 (the
last two words of task_pt_regs(current), cause and orig_a0), advances sp
= fp = stack + THREAD_SIZE, and sets pc = frame->ra (pt_regs->orig_a0).
If orig_a0 falls in [handle_exception, ret_from_exception_end),
walk_stackframe() dereferences ((struct pt_regs *)sp)->epc at stack +
THREAD_SIZE, hitting the unmapped CONFIG_VMAP_STACK guard page.

Entry and exit windows in [handle_exception, ret_from_exception_end) also
leave s0 holding the user s0 register or sp + PT_SIZE_ON_STACK while sp
is already on the kernel stack.  Rejecting PCs in [handle_exception,
ret_from_exception_end) and requiring fp <= (unsigned
long)task_pt_regs(current) would avoid unwinding these windows.

> +
> +static bool sse_callchain_is_guest(const struct riscv_sse_interrupted_context *context)
> +{
> +	return context && (context->hstatus & HSTATUS_SPV);
> +}
> +
> +static bool sse_callchain_kernel(struct perf_callchain_entry_ctx *entry,
> +				 struct pt_regs *regs)
> +{
> +	const struct riscv_sse_interrupted_context *context;
> +	unsigned long pc;
> +
> +	context = riscv_sse_get_interrupted_context();
> +	if (!context || context->regs != regs)
> +		return false;
> +
> +	/* A guest stack cannot be walked using the host address space. */
> +	if (sse_callchain_is_guest(context))
> +		return true;
> +
> +	if (user_mode(regs))
> +		return true;
> +
> +	if (sse_kernel_regs_safe(regs)) {
> +		walk_stackframe(NULL, regs, fill_callchain, entry);
> +		return true;
> +	}

[Severity: High]
Does passing NULL as the task argument to walk_stackframe() here (and in
perf_callchain_kernel() below) cause a NULL pointer dereference when
function_graph tracing is active?

When regs is non-NULL, walk_stackframe() takes the if (regs) branch and
leaves task as NULL, then passes task to ftrace_graph_ret_addr():

arch/riscv/kernel/stacktrace.c:walk_stackframe() {
    ...
			fp = READ_ONCE_TASK_STACK(task, frame->fp);
			pc = READ_ONCE_TASK_STACK(task, frame->ra);
			pc = ftrace_graph_ret_addr(task, &graph_idx, pc,
						   &frame->ra);
    ...
}

When an unwound frame's return address is return_to_handler,
ftrace_graph_ret_addr() evaluates task->curr_ret_stack without a NULL
check, panicking in SSE context.  Passing current (or defaulting task to
current in walk_stackframe() when task is NULL) avoids this.

[ ... ]

> +unsigned long riscv_perf_out_copy_user(void *dst, const void *src,
> +				       unsigned long n)
> +{
> +	unsigned long ret;

[ ... ]

> +	if (riscv_sse_get_interrupted_context()) {
> +		if (!access_ok(src, n))
> +			return n;
> +		if (!current->mm ||
> +		    (csr_read(CSR_SATP) & SATP_PPN) != virt_to_pfn(current->mm->pgd))
> +			return n;
> +	}

[Severity: High]
Should riscv_perf_out_copy_user() also reject interrupted KVM guest
contexts with sse_callchain_is_guest(context), the same way
sse_callchain_kernel() and perf_callchain_user() do?

On RISC-V with the Hypervisor extension, guest stage-1 and stage-2
translations use CSR_VSATP and CSR_HGATP while the host HS-mode CSR_SATP
stays loaded with the host VMM task's current->mm->pgd across
__kvm_riscv_switch_to().  When an SSE PMU overflow traps from a running
guest into HS-mode, current->mm is non-NULL and (csr_read(CSR_SATP) &
SATP_PPN) == virt_to_pfn(current->mm->pgd) still holds, so
riscv_perf_out_copy_user() copies from the host VMM user address space at
the guest's sp into the perf ring buffer.

> +
> +	pagefault_disable();
> +	ret = __copy_from_user_inatomic(dst, src, n);
> +	pagefault_enable();
> +
> +	return ret;
> +}
> +#endif

[Severity: High]
Can calling access_ok() and __copy_from_user_inatomic() before verifying
current->mm != NULL dereference a NULL current->mm under
CONFIG_RISCV_ISA_SUPM?

On RISC-V, both access_ok() and raw_copy_from_user() expand
untagged_addr(), which passes current->mm to __untagged_addr_remote() and
dereferences mm->context.pmlen when RISCV_ISA_EXT_SUPM is active.  If a
perf sample hits a user task in do_exit() after exit_mm() has cleared
current->mm (where perf_output_sample_ustack() still invokes
__output_copy_user() with size 0), riscv_perf_out_copy_user() calls
access_ok() before checking !current->mm in the SSE path, and never
checks !current->mm in the non-SSE path.

> +
>  /*
>   * This will be called when the target is in user mode
>   * This function will only be called when we use
> @@ -28,6 +154,15 @@ static bool fill_callchain(void *entry, unsigned long pc)
>  void perf_callchain_user(struct perf_callchain_entry_ctx *entry,
>  			 struct pt_regs *regs)
>  {
> +#ifdef CONFIG_RISCV_SBI_SSE
> +	const struct riscv_sse_interrupted_context *context;
> +
> +	context = riscv_sse_get_interrupted_context();
> +	/* A guest stack cannot be walked using the host address space. */
> +	if (sse_callchain_is_guest(context))
> +		return;
> +#endif

[Severity: High]
When context is non-NULL, does perf_callchain_user() also need the
!current->mm and (csr_read(CSR_SATP) & SATP_PPN) !=
virt_to_pfn(current->mm->pgd) check from riscv_perf_out_copy_user()?

When an SSE event interrupts the kernel while current is a user task,
get_perf_callchain() substitutes task_pt_regs(current) and calls
perf_callchain_user().  If the SSE arrived during an address-space
transition such as context_switch() (after switch_mm_irqs_off() writes
next->mm->pgd to CSR_SATP before __switch_to() updates current) or
efi_virtmap_load(), arch_stack_walk_user() -> unwind_user_frame() copies
user stack frames from the wrong page tables, and if current->mm is NULL
(reachable via bpf_get_stackid(BPF_F_USER_STACK) in do_exit() after
exit_mm()), access_ok() in unwind_user_frame() dereferences NULL
current->mm->context.pmlen under CONFIG_RISCV_ISA_SUPM.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261119120824.13969-13-zhangzhanpeng.jasper@bytedance.com?part=7

^ permalink raw reply	[flat|nested] 22+ messages in thread

end of thread, other threads:[~2026-11-19 12:45 UTC | newest]

Thread overview: 22+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-19 12:08 [PATCH v13 00/11] riscv: add SBI Supervisor Software Events support Zhanpeng Zhang
2026-11-19 12:08 ` [PATCH v13 01/11] riscv: add SBI SSE extension definitions Zhanpeng Zhang
2026-11-19 12:14   ` sashiko-bot
2026-11-19 12:08 ` [PATCH v13 02/11] riscv: add support for SBI Supervisor Software Events extension Zhanpeng Zhang
2026-11-19 12:36   ` sashiko-bot
2026-11-19 12:08 ` [PATCH v13 03/11] riscv: sse: mask events during shutdown and kexec Zhanpeng Zhang
2026-11-19 12:38   ` sashiko-bot
2026-11-19 12:08 ` [PATCH v13 04/11] drivers: firmware: add riscv SSE support Zhanpeng Zhang
2026-11-19 12:38   ` sashiko-bot
2026-11-19 12:08 ` [PATCH v13 05/11] riscv: mm: avoid enabling interrupts for nofault page faults Zhanpeng Zhang
2026-11-19 12:27   ` sashiko-bot
2026-11-19 12:08 ` [PATCH v13 06/11] riscv: enable NMI-safe lockdep irqflags tracking Zhanpeng Zhang
2026-11-19 12:21   ` sashiko-bot
2026-11-19 12:08 ` [PATCH v13 07/11] perf: RISC-V: support callchains with SSE delivery Zhanpeng Zhang
2026-11-19 12:45   ` sashiko-bot
2026-11-19 12:08 ` [PATCH v13 08/11] perf: RISC-V: add support for SSE event Zhanpeng Zhang
2026-11-19 12:34   ` sashiko-bot
2026-11-19 12:08 ` [PATCH v13 09/11] MAINTAINERS: add RISC-V SSE maintainers and file coverage Zhanpeng Zhang
2026-11-19 12:08 ` [PATCH v13 10/11] selftests/riscv: add SSE test module Zhanpeng Zhang
2026-11-19 12:37   ` sashiko-bot
2026-11-19 12:08 ` [PATCH v13 11/11] selftests/riscv: add perf user-stack SSE copy regression test Zhanpeng Zhang
2026-11-19 12:34   ` sashiko-bot

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox