[PATCH v12 00/11] riscv: add SBI Supervisor Software Events support

[PATCH v12 00/11] riscv: add SBI Supervisor Software Events support

[Zhanpeng Zhang]

This revision follows Anup's v11 [1] and uses the same riscv/for-next
base, a5ae1bf65cee. It incorporates the entry-state fix identified by
Himanshu, replaces the temporary GUP workaround for perf user-stack
sampling, and brings together the outstanding preemption, fault-entry,
lockdep and PMU error-handling changes. Thanks to Anup for the rebase
and PMU testing, and to Paul for coordinating integration.

Why SSE
=======

RISC-V does not define an ordinary supervisor-mode non-maskable interrupt
(NMI). When Linux clears SIE, a normal supervisor interrupt remains pending
until interrupts are enabled again. That is appropriate for ordinary
interrupt handling, but leaves a blind spot for profiling and delays
notifications that need attention inside an interrupt-disabled region.

The SBI Supervisor Software Events (SSE) extension [2] provides a
firmware-to-supervisor event delivery interface that does not depend on
SIE. Linux registers an entry point, and firmware can transfer control to
it while ordinary supervisor interrupts are masked. After the handler
completes the event, firmware resumes the interrupted execution.

PMU overflow is the first client in this series. The same framework is
intended to support other clients, including high-priority RAS reports;
the RAS client implementation is separate from this series.

Delivery and implementation
===========================

For a hardware-triggered event, the flow is:

                   Linux kernel                  Firmware         Hardware
            interrupted    SSE handler            OpenSBI          source
              context           |                    |                |
                 |              |                    |                |
  1. setup       |              |-----register------->                |
                 |              |------enable-------->                |
  2. trigger     |              |                    <-----event------|
                 |              |          +-------------------+      |
  3. save        |              |          |   context save    |      |
                 |              |          +-------------------+      |
                 |              |                    |                |
                 |              |          +-------------------+      |
  4. inject      |              |          |   handler setup   |      |
                 |              |          +-------------------+      |
                 |              <-------mret---------|                |
                 |    +-------------------+          |                |
  5. handle      |    |  event handling   |          |                |
                 |    +-------------------+          |                |
  6. complete    |              |-----complete------->                |
                 |              |          +-------------------+      |
  7. restore     |              |          |  context restore  |      |
                 |              |          +-------------------+      |
  8. resume      <---------------mret----------------|                |
                 |              |                    |                |

Software-injected events do not need the hardware trigger. The diagram
shows a typical OpenSBI implementation of the firmware interface.

Linux enters on a dedicated event stack and establishes NMI context.
The entry wrapper reconstructs the interrupted pt_regs for the callback,
rather than treating the synthetic handler context as the sampled task.
It also preserves Linux-owned trap state, task stack metadata and the
temporary register slots used by ordinary exception entry.

Handlers must not sleep or rely on locks that the interrupted code may
hold. A nofault user access may still raise a synchronous exception: that
exception must return through its exception-table fixup without paging
memory in or opening an interrupt window. Preserving the interrupted
entry state across such nested exceptions is therefore essential.

The firmware driver provides a common client API for local and global
events. It tracks ownership across partial setup and teardown failures,
keeps callback resources alive while firmware may still reference them,
and integrates event handling with CPU hotplug, shutdown and kexec.

PMU overflow and perf
=====================

Ordinary PMU interrupts cannot sample an interrupt-disabled critical
section until Linux unmasks interrupts. The sample then often lands at
that boundary, obscuring the work that preceded it. SSE removes this
SIE-induced delivery delay. It does not promise zero skid or precise-event
sampling, but lets perf observe execution inside those critical sections.

No new perf command or perf.data format is required. The PMU driver
selects its delivery mechanism during setup. With PMU-SSE configured,
explicitly unsupported firmware or events can use ordinary interrupts;
other setup errors do not silently select that route. There is no runtime
SSE-to-IRQ transition after SSE has become active.

Kernel and user callchains use the interrupted context. Kernel unwinding
falls back to the interrupted PC where stack bounds cannot be established.
For DWARF sampling, perf copies the raw user stack with page faults
disabled and truncates it at an inaccessible address. V12 uses inatomic
uaccess and exception-table fixup instead of fast-only GUP. The additional
address-space checks reject a missing mm or mismatched active page-table
root; they are not a complete equivalent of x86's nmi_uaccess_okay().

The PMU path retains throttling and stopped-event semantics. It restores
counters and userpage state before enabling an event after CPU power
management, and stops affected counters if an SSE operation fails.

Hardware observations
=====================

The following measurements were reported with v10. They illustrate the
delivery mechanism, not a new benchmark of this v12 source tree. The IRQ
and SSE comparisons used the same kernel source and perf binary on a
RISC-V server platform.

For perf bench sched pipe, the mean of three runs of three million
single-CPU operations was:

  rate       IRQ ops/s   SSE ops/s   delta   IRQ samples/s   SSE samples/s
  -F 99        337,707     339,555   +0.55%            98.0            98.6
  -F 999       338,352     338,289   -0.02%           995.7           998.0
  -F 5000      329,002     333,034   +1.23%          5001.7          5001.6

These runs had no lost samples. Both routes reached the requested rates,
with workload throughput differing by at most 1.23% in this comparison.

The same workload makes the irq-off sampling bias visible. In the -F 999
IRQ profile, finish_task_switch() and _raw_spin_unlock_irqrestore()
accounted for 54.99% of samples. Sampling via SSE instead exposed work
inside the critical sections, including scheduling, locking and address
space switching. Leading entries from the two reports were:

  Ordinary PMU IRQ
  overhead   symbol
    36.63%   finish_task_switch.isra.0
    18.36%   _raw_spin_unlock_irqrestore
     7.66%   __internal_syscall_cancel
     7.55%   do_trap_ecall_u
     4.19%   mutex_lock
     3.64%   mutex_unlock
     3.06%   exit_to_user_mode_loop

  PMU-SSE
  overhead   symbol
     5.48%   __kprobes_text_end
     5.29%   __schedule
     5.10%   ret_from_exception
     4.71%   do_raw_spin_lock
     4.01%   do_trap_ecall_u
     3.99%   mutex_lock
     3.66%   switch_mm
     3.43%   mutex_unlock
     3.29%   exit_to_user_mode_loop
     3.28%   psi_group_change

Symbols are reproduced as reported by perf. This is a change in what the
profile can observe, not evidence that SSE made those functions faster.

Entry-state fix and validation
==============================

Himanshu identified that TASK_TI_A0/A1/A2 were not preserved across SSE
handling [3]. Ordinary RV64 exception entry temporarily stores registers
in these slots. If SSE interrupts that window and its handler takes a
nested exception, the nested entry can overwrite slots still needed by
the interrupted entry path. Patch 2 preserves and restores them alongside
the task's kernel and user stack pointers.

Hardware A/B testing confirmed that preserving these scratch slots fixes
the reproduced corruption, allowing the temporary GUP workaround to be
replaced with nofault uaccess.

Guest SSE delivery, guest callchains and hibernation are outside the scope
of this series.

Firmware requirements
=====================

Firmware must implement SSE and the local PMU event. PMU testing also
requires the OpenSBI fixes that preserve pending overflows across event
enablement [4] and clear stale SPV for non-virtualized events [5].

Changes in v12
==============

Relative to Anup's v11 [1], on the same a5ae1bf65cee base:

- P2: Preserve TASK_TI_A0/A1/A2, fix per-CPU initialization preemption,
  and address the kernel test robot's unused-function warning.
- P3/P4: Clarify the kexec restriction and retain v10 RESEND's CPU-hotplug
  preemption guards.
- P5/P6: Order RV32 vmalloc synchronization before nofault fixup and add
  NMI-safe lockdep irqflags tracking.
- P7: Replace GUP with nofault uaccess.
- P8: Fix early-error counter STOP, ordinary IRQ CPU-PM ordering and IRQ
  cleanup tracking; carry Conor's Acked-by [6].
- P9: Split MAINTAINERS updates out; keep both selftests last.

Acknowledgements
================

The original feature patches were developed by Clément Léger and
Himanshu Chauhan. Thanks to Himanshu for identifying the missing scratch
slot preservation and reviewing the entry and lifecycle code; to Susheng
Yang for reporting the callchain failure and providing a reproducer; to
Conor Dooley for review and acknowledgements; and to Anup Patel, Paul
Walmsley and the kernel test robot for integration work and feedback.

[1] https://lore.kernel.org/r/20261028165531.940336-1-anup.patel@oss.qualcomm.com/
[2] https://docs.riscv.org/reference/sbi/ext-sse.html
[3] https://lore.kernel.org/r/as79MHtNYd-VbUug@hu-himchau-blr.qualcomm.com/
[4] https://github.com/riscv-software-src/opensbi/commit/f30a54f3b3a091c225a00476f4039bf399badd1f
[5] https://github.com/riscv-software-src/opensbi/commit/35511bc6ee1c9c17b6a89b44c52e2044bb51b979
[6] https://lore.kernel.org/r/20261002-reach-cloning-f1594729dbe3@wendy/

Clément Léger (5):
  riscv: add SBI SSE extension definitions
  riscv: add support for SBI Supervisor Software Events extension
  drivers: firmware: add riscv SSE support
  perf: RISC-V: add support for SSE event
  selftests/riscv: add SSE test module

Zhanpeng Zhang (6):
  riscv: sse: mask events during shutdown and kexec
  riscv: mm: avoid enabling interrupts for nofault page faults
  riscv: enable NMI-safe lockdep irqflags tracking
  perf: RISC-V: support callchains with SSE delivery
  MAINTAINERS: add RISC-V SSE maintainers and file coverage
  selftests/riscv: add perf user-stack SSE copy regression test

 Documentation/arch/riscv/index.rst                 |    1 +
 Documentation/arch/riscv/pmu-sse.rst               |   62 +
 MAINTAINERS                                        |   22 +
 arch/riscv/Kconfig                                 |    1 +
 arch/riscv/include/asm/asm.h                       |   14 +-
 arch/riscv/include/asm/perf_event.h                |   10 +
 arch/riscv/include/asm/sbi.h                       |   63 +
 arch/riscv/include/asm/scs.h                       |    7 +
 arch/riscv/include/asm/sse.h                       |   82 ++
 arch/riscv/include/asm/thread_info.h               |    1 +
 arch/riscv/kernel/Makefile                         |    1 +
 arch/riscv/kernel/asm-offsets.c                    |   14 +
 arch/riscv/kernel/entry.S                          |   14 +
 arch/riscv/kernel/machine_kexec.c                  |   11 +
 arch/riscv/kernel/perf_callchain.c                 |  140 +++
 arch/riscv/kernel/reset.c                          |   18 +
 arch/riscv/kernel/sbi_sse.c                        |  252 ++++
 arch/riscv/kernel/sbi_sse_entry.S                  |  237 ++++
 arch/riscv/kernel/smp.c                            |   17 +
 arch/riscv/mm/fault.c                              |   32 +-
 drivers/firmware/Kconfig                           |    1 +
 drivers/firmware/Makefile                          |    1 +
 drivers/firmware/riscv/Kconfig                     |   18 +
 drivers/firmware/riscv/Makefile                    |    3 +
 drivers/firmware/riscv/riscv_sbi_sse.c             | 1228 ++++++++++++++++++++
 drivers/perf/Kconfig                               |   11 +
 drivers/perf/riscv_pmu.c                           |   14 +-
 drivers/perf/riscv_pmu_sbi.c                       |  561 +++++++--
 include/linux/cpuhotplug.h                         |    1 +
 include/linux/perf/riscv_pmu.h                     |   20 +-
 include/linux/riscv_sbi_sse.h                      |   95 ++
 tools/testing/selftests/riscv/Makefile             |    2 +-
 tools/testing/selftests/riscv/sse/Makefile         |   10 +
 tools/testing/selftests/riscv/sse/module/Makefile  |   22 +
 .../selftests/riscv/sse/module/riscv_sse_test.c    | 1154 ++++++++++++++++++
 tools/testing/selftests/riscv/sse/run_sse_test.sh  |   59 +
 .../testing/selftests/riscv/sse/sse_perf_ustack.c  |  564 +++++++++
 37 files changed, 4656 insertions(+), 107 deletions(-)

base-commit: a5ae1bf65cee84bcc121d659e7e22a767853f5f5
-- 
2.50.1

[PATCH v12 01/11] riscv: add SBI SSE extension definitions

[Zhanpeng Zhang]

From: Clément Léger <cleger@rivosinc.com>

Add definitions for the SBI Supervisor Software Events extension [1].
This extension enables the SBI to inject events into supervisor software
much like ARM SDEI.

[1] https://lists.riscv.org/g/tech-prs/message/515

Signed-off-by: Clément Léger <cleger@rivosinc.com>
Co-developed-by: Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>
Signed-off-by: Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>
Co-developed-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
Signed-off-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
---
 arch/riscv/include/asm/sbi.h | 63 ++++++++++++++++++++++++++++++++++++
 1 file changed, 63 insertions(+)

diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
index fd6266a7b33..eb22d79cea6 100644
--- a/arch/riscv/include/asm/sbi.h
+++ b/arch/riscv/include/asm/sbi.h
@@ -40,6 +40,7 @@ enum sbi_ext_id {
 	SBI_EXT_FWFT = 0x46574654,
 	SBI_EXT_MPXY = 0x4D505859,
 	SBI_EXT_DBTR = 0x44425452,
+	SBI_EXT_SSE = 0x535345,
 
 	/* Experimentals extensions must lie within this range */
 	SBI_EXT_EXPERIMENTAL_START = 0x08000000,
@@ -508,6 +509,68 @@ enum sbi_mpxy_rpmi_attribute_id {
 #define SBI_MPXY_CHAN_CAP_SEND_WITHOUT_RESP	BIT(4)
 #define SBI_MPXY_CHAN_CAP_GET_NOTIFICATIONS	BIT(5)
 
+enum sbi_ext_sse_fid {
+	SBI_SSE_EVENT_ATTR_READ = 0,
+	SBI_SSE_EVENT_ATTR_WRITE,
+	SBI_SSE_EVENT_REGISTER,
+	SBI_SSE_EVENT_UNREGISTER,
+	SBI_SSE_EVENT_ENABLE,
+	SBI_SSE_EVENT_DISABLE,
+	SBI_SSE_EVENT_COMPLETE,
+	SBI_SSE_EVENT_INJECT,
+	SBI_SSE_HART_UNMASK,
+	SBI_SSE_HART_MASK,
+};
+
+enum sbi_sse_state {
+	SBI_SSE_STATE_UNUSED     = 0,
+	SBI_SSE_STATE_REGISTERED = 1,
+	SBI_SSE_STATE_ENABLED    = 2,
+	SBI_SSE_STATE_RUNNING    = 3,
+};
+
+/* SBI SSE Event Attributes. */
+enum sbi_sse_attr_id {
+	SBI_SSE_ATTR_STATUS		= 0x00000000,
+	SBI_SSE_ATTR_PRIO		= 0x00000001,
+	SBI_SSE_ATTR_CONFIG		= 0x00000002,
+	SBI_SSE_ATTR_PREFERRED_HART	= 0x00000003,
+	SBI_SSE_ATTR_ENTRY_PC		= 0x00000004,
+	SBI_SSE_ATTR_ENTRY_ARG		= 0x00000005,
+	SBI_SSE_ATTR_INTERRUPTED_SEPC	= 0x00000006,
+	SBI_SSE_ATTR_INTERRUPTED_FLAGS	= 0x00000007,
+	SBI_SSE_ATTR_INTERRUPTED_A6	= 0x00000008,
+	SBI_SSE_ATTR_INTERRUPTED_A7	= 0x00000009,
+
+	SBI_SSE_ATTR_MAX		= 0x0000000A
+};
+
+#define SBI_SSE_ATTR_STATUS_STATE_OFFSET	0
+#define SBI_SSE_ATTR_STATUS_STATE_MASK		0x3
+#define SBI_SSE_ATTR_STATUS_PENDING_OFFSET	2
+#define SBI_SSE_ATTR_STATUS_INJECT_OFFSET	3
+
+#define SBI_SSE_ATTR_CONFIG_ONESHOT	BIT(0)
+
+#define SBI_SSE_ATTR_INTERRUPTED_FLAGS_SSTATUS_SPP	BIT(0)
+#define SBI_SSE_ATTR_INTERRUPTED_FLAGS_SSTATUS_SPIE	BIT(1)
+#define SBI_SSE_ATTR_INTERRUPTED_FLAGS_HSTATUS_SPV	BIT(2)
+#define SBI_SSE_ATTR_INTERRUPTED_FLAGS_HSTATUS_SPVP	BIT(3)
+#define SBI_SSE_ATTR_INTERRUPTED_FLAGS_SSTATUS_SPELP	BIT(4)
+#define SBI_SSE_ATTR_INTERRUPTED_FLAGS_SSTATUS_SDT	BIT(5)
+
+#define SBI_SSE_EVENT_LOCAL_HIGH_PRIO_RAS	0x00000000
+#define SBI_SSE_EVENT_LOCAL_DOUBLE_TRAP		0x00000001
+#define SBI_SSE_EVENT_GLOBAL_HIGH_PRIO_RAS	0x00008000
+#define SBI_SSE_EVENT_LOCAL_PMU_OVERFLOW	0x00010000
+#define SBI_SSE_EVENT_LOCAL_LOW_PRIO_RAS	0x00100000
+#define SBI_SSE_EVENT_GLOBAL_LOW_PRIO_RAS	0x00108000
+#define SBI_SSE_EVENT_LOCAL_SOFTWARE_INJECTED	0xffff0000
+#define SBI_SSE_EVENT_GLOBAL_SOFTWARE_INJECTED	0xffff8000
+
+#define SBI_SSE_EVENT_PLATFORM		BIT(14)
+#define SBI_SSE_EVENT_GLOBAL		BIT(15)
+
 /* SBI debug triggers function IDs */
 enum sbi_ext_dbtr_fid {
 	SBI_EXT_DBTR_NUM_TRIGGERS = 0,
-- 
2.50.1 (Apple Git-155)

Re: [PATCH v12 01/11] riscv: add SBI SSE extension definitions

[sashiko-bot]

> From: Clément Léger <cleger@rivosinc.com>
> 
> Add definitions for the SBI Supervisor Software Events extension [1].
> This extension enables the SBI to inject events into supervisor software
> much like ARM SDEI.
> 
> [1] https://lists.riscv.org/g/tech-prs/message/515
> 
> Signed-off-by: Clément Léger <cleger@rivosinc.com>
> Co-developed-by: Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>
> Signed-off-by: Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>
> Co-developed-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
> Signed-off-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261029074853.35467-1-zhangzhanpeng.jasper@bytedance.com?part=1

[PATCH v12 02/11] riscv: add support for SBI Supervisor Software Events extension

[Zhanpeng Zhang]

From: Clément Léger <cleger@rivosinc.com>

The SBI SSE extension allows firmware to notify supervisor software of
events that must be delivered independently of normal S-mode interrupts.
Firmware saves the minimal state required to enter the supervisor handler,
and Linux builds the synthetic handler context around it.

SSE can arrive while Linux is already in an exception entry path. At that
point sscratch and tp may be in the middle of the normal trap-entry
exchange, so they cannot always identify current. Store current in a
per-CPU slot and use the hart ID passed by firmware to recover it.

Give each event, including each CPU instance of a local event, a dedicated
stack and shadow call stack. Synchronize vmapped stack ranges before
unmasking events so that the handler cannot take a vmalloc fault while
running in an NMI-like context.

The handler is a synthetic supervisor episode, but completion must resume
the context interrupted by the SSE. Preserve stvec and, when the hypervisor
extension is present, hstatus across the handler. Read the interrupted a6
and a7 values from the SSE attributes, construct pt_regs for the
interrupted context, and write back any changes made by the handler.

Nested exceptions on the SSE event stack temporarily replace both
TASK_TI_KERNEL_SP and TASK_TI_USER_SP. Preserve their original values
across the handler and restore them before completing the event, so a
nested exception cannot leave the interrupted task referring to the event
stack.

On RV64 with an MMU, early trap entry also uses thread_info.a0/a1/a2
as scratch slots before saving the full register frame. An SSE handler
can interrupt that window and take a nested fault which overwrites the
same slots. Preserve them in already-saved callee-saved registers and
restore them before resuming the interrupted entry path.

Disable preemption while initializing the per-CPU entry-task slot.
CPU hotplug callbacks run in a preemptible thread, and this write
happens before the guards around the local SBI operations.

Compile the stack TLB helper only for VMAP_STACK or SHADOW_CALL_STACK,
its only users, to avoid an unused-function warning in other builds.

Keep an explicit EVENT_REGISTER not-supported result distinct from other
firmware failures. This lets clients select another delivery mechanism
only when firmware has positively rejected the requested event.

Link: https://lore.kernel.org/r/as79MHtNYd-VbUug@hu-himchau-blr.qualcomm.com/
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202610020009.P3ILnDi1-lkp@intel.com/
Signed-off-by: Clément Léger <cleger@rivosinc.com>
Co-developed-by: Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>
Signed-off-by: Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>
Co-developed-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
Signed-off-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
---
 arch/riscv/include/asm/asm.h         |  14 +-
 arch/riscv/include/asm/scs.h         |   7 +
 arch/riscv/include/asm/sse.h         |  82 +++++++++
 arch/riscv/include/asm/thread_info.h |   1 +
 arch/riscv/kernel/Makefile           |   1 +
 arch/riscv/kernel/asm-offsets.c      |  14 ++
 arch/riscv/kernel/entry.S            |  14 ++
 arch/riscv/kernel/sbi_sse.c          | 251 +++++++++++++++++++++++++++
 arch/riscv/kernel/sbi_sse_entry.S    | 237 +++++++++++++++++++++++++
 9 files changed, 618 insertions(+), 3 deletions(-)
 create mode 100644 arch/riscv/include/asm/sse.h
 create mode 100644 arch/riscv/kernel/sbi_sse.c
 create mode 100644 arch/riscv/kernel/sbi_sse_entry.S

diff --git a/arch/riscv/include/asm/asm.h b/arch/riscv/include/asm/asm.h
index b8bf842d4c1..e1196caea02 100644
--- a/arch/riscv/include/asm/asm.h
+++ b/arch/riscv/include/asm/asm.h
@@ -91,16 +91,24 @@
 .endm
 
 #ifdef CONFIG_SMP
-.macro asm_per_cpu dst sym tmp
-	lw    \tmp, TASK_TI_CPU_NUM(tp)
-	slli  \tmp, \tmp, RISCV_LGPTR
+.macro asm_per_cpu_with_cpu dst sym tmp cpu
+	slli  \tmp, \cpu, RISCV_LGPTR
 	la    \dst, __per_cpu_offset
 	add   \dst, \dst, \tmp
 	REG_L \tmp, 0(\dst)
 	la    \dst, \sym
 	add   \dst, \dst, \tmp
 .endm
+
+.macro asm_per_cpu dst sym tmp
+	lw \tmp, TASK_TI_CPU_NUM(tp)
+	asm_per_cpu_with_cpu \dst \sym \tmp \tmp
+.endm
 #else /* CONFIG_SMP */
+.macro asm_per_cpu_with_cpu dst sym tmp cpu
+	la    \dst, \sym
+.endm
+
 .macro asm_per_cpu dst sym tmp
 	la    \dst, \sym
 .endm
diff --git a/arch/riscv/include/asm/scs.h b/arch/riscv/include/asm/scs.h
index 023a412fe38..0d70a35bc01 100644
--- a/arch/riscv/include/asm/scs.h
+++ b/arch/riscv/include/asm/scs.h
@@ -17,6 +17,11 @@
 	load_per_cpu gp, irq_shadow_call_stack_ptr, \tmp
 .endm
 
+/* Load the per-CPU IRQ shadow call stack to gp. */
+.macro scs_load_sse_stack reg_evt
+	REG_L gp, SSE_REG_EVT_SHADOW_STACK(\reg_evt)
+.endm
+
 /* Load task_scs_sp(current) to gp. */
 .macro scs_load_current
 	REG_L	gp, TASK_TI_SCS_SP(tp)
@@ -40,6 +45,8 @@
 .endm
 .macro scs_load_irq_stack tmp
 .endm
+.macro scs_load_sse_stack reg_evt
+.endm
 .macro scs_load_current
 .endm
 .macro scs_load_current_if_task_changed prev
diff --git a/arch/riscv/include/asm/sse.h b/arch/riscv/include/asm/sse.h
new file mode 100644
index 00000000000..cbd8618c6e0
--- /dev/null
+++ b/arch/riscv/include/asm/sse.h
@@ -0,0 +1,82 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2024 Rivos Inc.
+ */
+#ifndef __ASM_SSE_H
+#define __ASM_SSE_H
+
+#include <linux/printk.h>
+#include <linux/types.h>
+
+#include <asm/sbi.h>
+
+static inline bool riscv_sse_available(void)
+{
+#ifdef CONFIG_RISCV_SBI
+	return sbi_probe_extension(SBI_EXT_SSE) > 0;
+#else
+	return false;
+#endif
+}
+
+static inline void riscv_sse_mask_current_hart(void)
+{
+#ifdef CONFIG_RISCV_SBI
+	struct sbiret ret;
+
+	if (!riscv_sse_available())
+		return;
+
+	ret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_HART_MASK, 0, 0, 0, 0, 0, 0);
+	if (ret.error && ret.error != SBI_ERR_ALREADY_STOPPED)
+		pr_emerg("SSE hart mask failed: %ld\n", ret.error);
+#endif
+}
+
+#ifdef CONFIG_RISCV_SBI_SSE
+
+struct sse_event_interrupted_state {
+	unsigned long a6;
+	unsigned long a7;
+};
+
+struct sse_event_arch_data {
+	void *stack;
+	void *shadow_stack;
+	unsigned long tmp;
+	struct sse_event_interrupted_state *interrupted;
+	phys_addr_t interrupted_phys;
+	u32 evt_id;
+	unsigned long hart_id;
+	unsigned int cpu_id;
+};
+
+struct riscv_sse_interrupted_context {
+	struct pt_regs *regs;
+	unsigned long hstatus;
+};
+
+static inline bool sse_event_is_global(u32 evt)
+{
+	return !!(evt & SBI_SSE_EVENT_GLOBAL);
+}
+
+void arch_sse_event_update_cpu(struct sse_event_arch_data *arch_evt, int cpu);
+int arch_sse_init_event(struct sse_event_arch_data *arch_evt, u32 evt_id,
+			int cpu);
+void arch_sse_free_event(struct sse_event_arch_data *arch_evt);
+int arch_sse_register_event(struct sse_event_arch_data *arch_evt);
+void arch_sse_init_cpu(void);
+
+void sse_handle_event(struct sse_event_arch_data *arch_evt,
+		      struct pt_regs *regs);
+asmlinkage void handle_sse(void);
+asmlinkage void noinstr do_sse(struct sse_event_arch_data *arch_evt,
+			       struct pt_regs *regs, unsigned long hstatus);
+
+const struct riscv_sse_interrupted_context *
+riscv_sse_get_interrupted_context(void);
+
+#endif
+
+#endif
diff --git a/arch/riscv/include/asm/thread_info.h b/arch/riscv/include/asm/thread_info.h
index 55019fdfa9e..d14b45610c7 100644
--- a/arch/riscv/include/asm/thread_info.h
+++ b/arch/riscv/include/asm/thread_info.h
@@ -36,6 +36,7 @@
 #define OVERFLOW_STACK_SIZE     SZ_4K
 
 #define IRQ_STACK_SIZE		THREAD_SIZE
+#define SSE_STACK_SIZE		THREAD_SIZE
 
 #ifndef __ASSEMBLER__
 
diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile
index f54c336bf54..0728362ba0a 100644
--- a/arch/riscv/kernel/Makefile
+++ b/arch/riscv/kernel/Makefile
@@ -101,6 +101,7 @@ obj-$(CONFIG_DYNAMIC_FTRACE)	+= mcount-dyn.o
 obj-$(CONFIG_PERF_EVENTS)	+= perf_callchain.o
 obj-$(CONFIG_HAVE_PERF_REGS)	+= perf_regs.o
 obj-$(CONFIG_RISCV_SBI)		+= sbi.o sbi_ecall.o
+obj-$(CONFIG_RISCV_SBI_SSE)	+= sbi_sse.o sbi_sse_entry.o
 ifeq ($(CONFIG_RISCV_SBI), y)
 obj-$(CONFIG_SMP)		+= sbi-ipi.o
 obj-$(CONFIG_SMP) += cpu_ops_sbi.o
diff --git a/arch/riscv/kernel/asm-offsets.c b/arch/riscv/kernel/asm-offsets.c
index a75f0cfea1e..15363703cdd 100644
--- a/arch/riscv/kernel/asm-offsets.c
+++ b/arch/riscv/kernel/asm-offsets.c
@@ -15,6 +15,8 @@
 #include <asm/ptrace.h>
 #include <asm/cpu_ops_sbi.h>
 #include <asm/stacktrace.h>
+#include <asm/sbi.h>
+#include <asm/sse.h>
 #include <asm/suspend.h>
 
 void asm_offsets(void);
@@ -533,6 +535,18 @@ void asm_offsets(void)
 	DEFINE(FREGS_A6,	    offsetof(struct __arch_ftrace_regs, a6));
 	DEFINE(FREGS_A7,	    offsetof(struct __arch_ftrace_regs, a7));
 #endif
+
+#ifdef CONFIG_RISCV_SBI_SSE
+	OFFSET(SSE_REG_EVT_STACK, sse_event_arch_data, stack);
+	OFFSET(SSE_REG_EVT_SHADOW_STACK, sse_event_arch_data, shadow_stack);
+	OFFSET(SSE_REG_EVT_TMP, sse_event_arch_data, tmp);
+	OFFSET(SSE_REG_HART_ID, sse_event_arch_data, hart_id);
+	OFFSET(SSE_REG_CPU_ID, sse_event_arch_data, cpu_id);
+
+	DEFINE(SBI_EXT_SSE, SBI_EXT_SSE);
+	DEFINE(SBI_SSE_EVENT_COMPLETE, SBI_SSE_EVENT_COMPLETE);
+	DEFINE(ASM_NR_CPUS, CONFIG_NR_CPUS);
+#endif
 #ifdef CONFIG_RISCV_SBI
 	DEFINE(SBI_EXT_FWFT, SBI_EXT_FWFT);
 	DEFINE(SBI_EXT_FWFT_SET, SBI_EXT_FWFT_SET);
diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
index d799c4e56f8..0b79fa7241e 100644
--- a/arch/riscv/kernel/entry.S
+++ b/arch/riscv/kernel/entry.S
@@ -424,6 +424,15 @@ SYM_FUNC_END(call_on_irq_stack)
  * arguments are passed to schedule_tail.
  */
 SYM_FUNC_START(__switch_to)
+#ifdef CONFIG_RISCV_SBI_SSE
+	/*
+	 * Mark the interval where tp changes from prev to next. SSE entry uses
+	 * the interrupted tp while this per-CPU pointer is NULL.
+	 */
+	asm_per_cpu t0, __sbi_sse_entry_task, t1
+	REG_S zero, 0(t0)
+#endif
+
 	/* Save context into prev->thread */
 	li    a4,  TASK_THREAD_RA
 	add   a3, a0, a4
@@ -470,6 +479,11 @@ SYM_FUNC_START(__switch_to)
 	REG_L s11, TASK_THREAD_S11_RA(a4)
 	/* The offset of thread_info in task_struct is zero. */
 	move tp, a1
+#ifdef CONFIG_RISCV_SBI_SSE
+	/* Publish next only after tp contains its task_struct pointer. */
+	asm_per_cpu t0, __sbi_sse_entry_task, t1
+	REG_S tp, 0(t0)
+#endif
 	/* Switch to the next shadow call stack */
 	scs_load_current
 	ret
diff --git a/arch/riscv/kernel/sbi_sse.c b/arch/riscv/kernel/sbi_sse.c
new file mode 100644
index 00000000000..032fcc07abc
--- /dev/null
+++ b/arch/riscv/kernel/sbi_sse.c
@@ -0,0 +1,251 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2025 Rivos Inc.
+ */
+#include <linux/nmi.h>
+#include <linux/sched.h>
+#include <linux/scs.h>
+#include <linux/bitfield.h>
+#include <linux/percpu-defs.h>
+#include <linux/preempt.h>
+#include <linux/string.h>
+
+#include <asm/asm-prototypes.h>
+#include <asm/switch_to.h>
+#include <asm/irq_stack.h>
+#include <asm/sbi.h>
+#include <asm/sse.h>
+#include <asm/tlbflush.h>
+
+DEFINE_PER_CPU(struct task_struct *, __sbi_sse_entry_task);
+static DEFINE_PER_CPU(struct riscv_sse_interrupted_context *,
+			      riscv_sse_interrupted_context);
+
+const struct riscv_sse_interrupted_context *
+riscv_sse_get_interrupted_context(void)
+{
+	return this_cpu_read(riscv_sse_interrupted_context);
+}
+
+void __weak sse_handle_event(struct sse_event_arch_data *arch_evt, struct pt_regs *regs)
+{
+}
+
+void noinstr do_sse(struct sse_event_arch_data *arch_evt,
+		    struct pt_regs *regs, unsigned long hstatus)
+{
+	struct riscv_sse_interrupted_context context = { regs, hstatus };
+	struct riscv_sse_interrupted_context *previous;
+	struct sbiret sret;
+
+	nmi_enter();
+	instrumentation_begin();
+
+	/* Retrieve missing GPRs from SBI */
+	sret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_EVENT_ATTR_READ, arch_evt->evt_id,
+			 SBI_SSE_ATTR_INTERRUPTED_A6,
+			 (SBI_SSE_ATTR_INTERRUPTED_A7 -
+			  SBI_SSE_ATTR_INTERRUPTED_A6) + 1,
+			 (unsigned long)arch_evt->interrupted_phys, 0, 0);
+	if (sret.error) {
+		pr_warn("Failed to read interrupted registers for event %x: %ld\n",
+			arch_evt->evt_id, sret.error);
+		/* Let the client quiesce its source without using incomplete regs. */
+		sse_handle_event(arch_evt, NULL);
+		goto out;
+	}
+
+	memcpy(&regs->a6, arch_evt->interrupted,
+	       sizeof(*arch_evt->interrupted));
+
+	/* Make the interrupted frame visible while clients handle this event. */
+	previous = this_cpu_read(riscv_sse_interrupted_context);
+	this_cpu_write(riscv_sse_interrupted_context, &context);
+	sse_handle_event(arch_evt, regs);
+	this_cpu_write(riscv_sse_interrupted_context, previous);
+
+	if (memcmp(&regs->a6, arch_evt->interrupted,
+		   sizeof(*arch_evt->interrupted))) {
+		memcpy(arch_evt->interrupted, &regs->a6,
+		       sizeof(*arch_evt->interrupted));
+		sret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_EVENT_ATTR_WRITE,
+				 arch_evt->evt_id, SBI_SSE_ATTR_INTERRUPTED_A6,
+				 (SBI_SSE_ATTR_INTERRUPTED_A7 -
+				  SBI_SSE_ATTR_INTERRUPTED_A6) + 1,
+				 (unsigned long)arch_evt->interrupted_phys, 0, 0);
+		/*
+		 * If writeback fails, COMPLETE resumes with firmware's original
+		 * a6/a7 rather than treating the shared buffer as committed.
+		 */
+		if (sret.error)
+			pr_warn("Failed to write interrupted registers for event %x: %ld\n",
+				arch_evt->evt_id, sret.error);
+	}
+
+out:
+	instrumentation_end();
+	nmi_exit();
+}
+
+static void *alloc_to_stack_pointer(void *alloc)
+{
+	return alloc ? alloc + SSE_STACK_SIZE : NULL;
+}
+
+static void *stack_pointer_to_alloc(void *stack)
+{
+	return stack ? stack - SSE_STACK_SIZE : NULL;
+}
+
+#if defined(CONFIG_VMAP_STACK) || defined(CONFIG_SHADOW_CALL_STACK)
+static void arch_sse_flush_tlb_range(struct sse_event_arch_data *arch_evt,
+				     unsigned long start, unsigned long size)
+{
+	unsigned long end = start + size;
+
+	if (sse_event_is_global(arch_evt->evt_id))
+		flush_tlb_kernel_range(start, end);
+	else
+		local_flush_tlb_kernel_range(start, end);
+}
+#endif
+
+static void arch_sse_shadow_stack_cpu_sync(struct sse_event_arch_data *arch_evt)
+{
+#ifdef CONFIG_SHADOW_CALL_STACK
+	if (arch_evt->shadow_stack)
+		arch_sse_flush_tlb_range(arch_evt,
+					 (unsigned long)arch_evt->shadow_stack,
+					 SCS_SIZE);
+#endif
+}
+
+#ifdef CONFIG_VMAP_STACK
+static void *sse_stack_alloc(unsigned int cpu)
+{
+	void *stack = arch_alloc_vmap_stack(SSE_STACK_SIZE, cpu_to_node(cpu));
+
+	return alloc_to_stack_pointer(stack);
+}
+
+static void sse_stack_free(void *stack)
+{
+	vfree(stack_pointer_to_alloc(stack));
+}
+
+static void arch_sse_stack_cpu_sync(struct sse_event_arch_data *arch_evt)
+{
+	void *p_stack = arch_evt->stack;
+	unsigned long stack = (unsigned long)stack_pointer_to_alloc(p_stack);
+
+	/*
+	 * Flush the tlb to avoid taking any exception when accessing the
+	 * vmapped stack inside the SSE handler
+	 */
+	arch_sse_flush_tlb_range(arch_evt, stack, SSE_STACK_SIZE);
+
+	arch_sse_shadow_stack_cpu_sync(arch_evt);
+}
+#else /* CONFIG_VMAP_STACK */
+static void *sse_stack_alloc(unsigned int cpu)
+{
+	void *stack = kmalloc(SSE_STACK_SIZE, GFP_KERNEL);
+
+	return alloc_to_stack_pointer(stack);
+}
+
+static void sse_stack_free(void *stack)
+{
+	kfree(stack_pointer_to_alloc(stack));
+}
+
+static void arch_sse_stack_cpu_sync(struct sse_event_arch_data *arch_evt)
+{
+	arch_sse_shadow_stack_cpu_sync(arch_evt);
+}
+#endif /* CONFIG_VMAP_STACK */
+
+static int sse_init_scs(int cpu, struct sse_event_arch_data *arch_evt)
+{
+	void *stack;
+
+	if (!scs_is_enabled())
+		return 0;
+
+	stack = scs_alloc(cpu_to_node(cpu));
+	if (!stack)
+		return -ENOMEM;
+
+	arch_evt->shadow_stack = stack;
+
+	return 0;
+}
+
+void arch_sse_event_update_cpu(struct sse_event_arch_data *arch_evt, int cpu)
+{
+	arch_evt->cpu_id = cpu;
+	arch_evt->hart_id = cpuid_to_hartid_map(cpu);
+}
+
+void arch_sse_init_cpu(void)
+{
+	preempt_disable();
+	this_cpu_write(__sbi_sse_entry_task, current);
+	preempt_enable();
+}
+
+int arch_sse_init_event(struct sse_event_arch_data *arch_evt, u32 evt_id,
+			int cpu)
+{
+	void *stack;
+
+	arch_evt->interrupted = kmalloc_obj(*arch_evt->interrupted, GFP_KERNEL);
+	if (!arch_evt->interrupted)
+		return -ENOMEM;
+
+	arch_evt->evt_id = evt_id;
+	stack = sse_stack_alloc(cpu);
+	if (!stack)
+		goto err_free_interrupted;
+
+	arch_evt->stack = stack;
+
+	if (sse_init_scs(cpu, arch_evt)) {
+		sse_stack_free(arch_evt->stack);
+		goto err_free_interrupted;
+	}
+
+	/* kmalloc keeps the two adjacent SBI attribute words contiguous. */
+	arch_evt->interrupted_phys = virt_to_phys(arch_evt->interrupted);
+
+	arch_sse_event_update_cpu(arch_evt, cpu);
+
+	return 0;
+
+err_free_interrupted:
+	kfree(arch_evt->interrupted);
+	arch_evt->interrupted = NULL;
+	return -ENOMEM;
+}
+
+void arch_sse_free_event(struct sse_event_arch_data *arch_evt)
+{
+	scs_free(arch_evt->shadow_stack);
+	sse_stack_free(arch_evt->stack);
+	kfree(arch_evt->interrupted);
+}
+
+int arch_sse_register_event(struct sse_event_arch_data *arch_evt)
+{
+	struct sbiret sret;
+
+	arch_sse_stack_cpu_sync(arch_evt);
+
+	sret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_EVENT_REGISTER, arch_evt->evt_id,
+			 (unsigned long)handle_sse, (unsigned long)arch_evt, 0,
+			 0, 0);
+	if (sret.error == SBI_ERR_NOT_SUPPORTED)
+		return -EOPNOTSUPP;
+
+	return sbi_err_map_linux_errno(sret.error);
+}
diff --git a/arch/riscv/kernel/sbi_sse_entry.S b/arch/riscv/kernel/sbi_sse_entry.S
new file mode 100644
index 00000000000..1e38fbb1c2a
--- /dev/null
+++ b/arch/riscv/kernel/sbi_sse_entry.S
@@ -0,0 +1,237 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (C) 2025 Rivos Inc.
+ */
+
+#include <linux/init.h>
+#include <linux/linkage.h>
+
+#include <asm/alternative-macros.h>
+#include <asm/asm.h>
+#include <asm/csr.h>
+#include <asm/hwcap.h>
+#include <asm/scs.h>
+
+/* When entering handle_sse, the following registers are set:
+ * a6: contains the hartid
+ * a7: contains a sse_event_arch_data struct pointer
+ */
+SYM_CODE_START(handle_sse)
+	/* Save stack temporarily */
+	REG_S sp, SSE_REG_EVT_TMP(a7)
+	/* Set entry stack */
+	REG_L sp, SSE_REG_EVT_STACK(a7)
+
+	addi sp, sp, -(PT_SIZE_ON_STACK)
+	REG_S ra, PT_RA(sp)
+	REG_S s0, PT_S0(sp)
+	REG_S s1, PT_S1(sp)
+	REG_S s2, PT_S2(sp)
+	REG_S s3, PT_S3(sp)
+	REG_S s4, PT_S4(sp)
+	REG_S s5, PT_S5(sp)
+	REG_S s6, PT_S6(sp)
+	REG_S s7, PT_S7(sp)
+	REG_S s8, PT_S8(sp)
+	REG_S s9, PT_S9(sp)
+	REG_S s10, PT_S10(sp)
+	REG_S s11, PT_S11(sp)
+	REG_S tp, PT_TP(sp)
+	REG_S t0, PT_T0(sp)
+	REG_S t1, PT_T1(sp)
+	REG_S t2, PT_T2(sp)
+	REG_S t3, PT_T3(sp)
+	REG_S t4, PT_T4(sp)
+	REG_S t5, PT_T5(sp)
+	REG_S t6, PT_T6(sp)
+	REG_S gp, PT_GP(sp)
+	REG_S a0, PT_A0(sp)
+	REG_S a1, PT_A1(sp)
+	REG_S a2, PT_A2(sp)
+	REG_S a3, PT_A3(sp)
+	REG_S a4, PT_A4(sp)
+	REG_S a5, PT_A5(sp)
+
+	/* Retrieve entry sp */
+	REG_L a4, SSE_REG_EVT_TMP(a7)
+	/* Save CSRs */
+	csrr a0, CSR_EPC
+	csrr a1, CSR_SSTATUS
+	csrr a2, CSR_STVAL
+	csrr a3, CSR_SCAUSE
+
+	REG_S a0, PT_EPC(sp)
+	REG_S a1, PT_STATUS(sp)
+	REG_S a2, PT_BADADDR(sp)
+	REG_S a3, PT_CAUSE(sp)
+	REG_S a4, PT_SP(sp)
+
+	/* Disable user memory access and floating/vector computing */
+	li t0, SR_SUM | SR_FS_VS
+	csrc CSR_STATUS, t0
+
+	load_global_pointer
+	scs_load_sse_stack a7
+
+#ifdef CONFIG_SMP
+	REG_L t4, SSE_REG_HART_ID(a7)
+	lw t3, SSE_REG_CPU_ID(a7)
+
+	bne t4, a6, .Lfind_hart_id_slowpath
+
+.Lcpu_id_found:
+#else
+	mv t3, zero
+#endif
+
+	asm_per_cpu_with_cpu t2 __sbi_sse_entry_task t1 t3
+	REG_L tp, 0(t2)
+	bnez tp, .Lcurrent_task_found
+
+	/* __switch_to() marks its transition window with a NULL entry task. */
+	REG_L tp, PT_TP(sp)
+
+.Lcurrent_task_found:
+	/* Nested exceptions temporarily replace these with the SSE stack. */
+	REG_L s6, TASK_TI_KERNEL_SP(tp)
+	REG_L s7, TASK_TI_USER_SP(tp)
+#if defined(CONFIG_64BIT) && defined(CONFIG_MMU)
+	/* A nested fault must not overwrite an interrupted trap's scratch slots. */
+	REG_L s8, TASK_TI_A0(tp)
+	REG_L s9, TASK_TI_A1(tp)
+	REG_L s10, TASK_TI_A2(tp)
+#endif
+
+	mv a1, sp /* pt_regs on stack */
+
+	/*
+	 * Run the SSE handler with the normal exception vector, but restore the
+	 * interrupted stvec before completing the event. SSE can arrive while
+	 * the kernel is using a temporary trap vector in a sensitive entry path.
+	 */
+	csrr s3, CSR_STVEC
+	la t0, handle_exception
+	csrw CSR_STVEC, t0
+
+	/*
+	 * Preserve the full HS-mode virtualization state across the handler.
+	 * hstatus is live supervisor state rather than an SSE interrupted
+	 * attribute, and OpenSBI consumes hstatus.SPV during event completion.
+	 * Saving the whole CSR keeps the handler episode transparent to KVM and
+	 * avoids having to infer which hstatus bits may matter to a guest resume.
+	 */
+	li s5, 0
+	ALTERNATIVE("nop", "csrr s5, hstatus", 0, RISCV_ISA_EXT_H, 1)
+
+	/*
+	 * Save sscratch for restoration since we might have interrupted the
+	 * kernel in early exception path and thus, we don't know the content of
+	 * sscratch.
+	 */
+	csrrw s4, CSR_SSCRATCH, x0
+
+	mv a0, a7
+	mv a2, s5
+
+	call do_sse
+
+	/* Leave no reference to the event stack in the interrupted task. */
+	REG_S s7, TASK_TI_USER_SP(tp)
+	REG_S s6, TASK_TI_KERNEL_SP(tp)
+#if defined(CONFIG_64BIT) && defined(CONFIG_MMU)
+	REG_S s8, TASK_TI_A0(tp)
+	REG_S s9, TASK_TI_A1(tp)
+	REG_S s10, TASK_TI_A2(tp)
+#endif
+
+	csrw CSR_SSCRATCH, s4
+	ALTERNATIVE("nop", "csrw hstatus, s5", 0, RISCV_ISA_EXT_H, 1)
+	csrw CSR_STVEC, s3
+
+	REG_L a0, PT_STATUS(sp)
+	REG_L a1, PT_EPC(sp)
+	REG_L a2, PT_BADADDR(sp)
+	REG_L a3, PT_CAUSE(sp)
+	csrw CSR_SSTATUS, a0
+	csrw CSR_EPC, a1
+	csrw CSR_STVAL, a2
+	csrw CSR_SCAUSE, a3
+
+	REG_L ra, PT_RA(sp)
+	REG_L s0, PT_S0(sp)
+	REG_L s1, PT_S1(sp)
+	REG_L s2, PT_S2(sp)
+	REG_L s3, PT_S3(sp)
+	REG_L s4, PT_S4(sp)
+	REG_L s5, PT_S5(sp)
+	REG_L s6, PT_S6(sp)
+	REG_L s7, PT_S7(sp)
+	REG_L s8, PT_S8(sp)
+	REG_L s9, PT_S9(sp)
+	REG_L s10, PT_S10(sp)
+	REG_L s11, PT_S11(sp)
+	REG_L tp, PT_TP(sp)
+	REG_L t0, PT_T0(sp)
+	REG_L t1, PT_T1(sp)
+	REG_L t2, PT_T2(sp)
+	REG_L t3, PT_T3(sp)
+	REG_L t4, PT_T4(sp)
+	REG_L t5, PT_T5(sp)
+	REG_L t6, PT_T6(sp)
+	REG_L gp, PT_GP(sp)
+	REG_L a0, PT_A0(sp)
+	REG_L a1, PT_A1(sp)
+	REG_L a2, PT_A2(sp)
+	REG_L a3, PT_A3(sp)
+	REG_L a4, PT_A4(sp)
+	REG_L a5, PT_A5(sp)
+
+	REG_L sp, PT_SP(sp)
+
+	li a7, SBI_EXT_SSE
+	li a6, SBI_SSE_EVENT_COMPLETE
+	ecall
+
+	/*
+	 * COMPLETE must resume the interrupted context and never return. Trap
+	 * through the normal vector instead of falling into adjacent assembly.
+	 */
+	la t0, handle_exception
+	csrw CSR_STVEC, t0
+	ebreak
+	/* The fatal trap must not return; execution should never reach here. */
+
+#ifdef CONFIG_SMP
+.Lfind_hart_id_slowpath:
+
+	/* Restore current task struct from __sbi_sse_entry_task */
+	li t1, ASM_NR_CPUS
+	/* Slowpath to find the CPU id associated to the hart id */
+	la t0, __cpuid_to_hartid_map
+	li t3, 0
+
+.Lhart_id_loop:
+	REG_L t2, 0(t0)
+	beq t2, a6, .Lcpu_id_found
+
+	/* Increment pointer and CPU number */
+	addi t3, t3, 1
+	addi t0, t0, RISCV_SZPTR
+	bltu t3, t1, .Lhart_id_loop
+
+	/*
+	 * This should never happen since we expect the hart_id to match one
+	 * of our CPU, but better be safe than sorry
+	 */
+	la tp, init_task
+	la a0, sse_hart_id_panic_string
+	la t0, panic
+	jalr t0
+#endif
+
+SYM_CODE_END(handle_sse)
+ASM_NOKPROBE(handle_sse)
+
+SYM_DATA_START_LOCAL(sse_hart_id_panic_string)
+    .ascii "Unable to match hart_id with cpu\0"
+SYM_DATA_END(sse_hart_id_panic_string)
-- 
2.50.1 (Apple Git-155)

Re: [PATCH v12 02/11] riscv: add support for SBI Supervisor Software Events extension

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] riscv: sbi_sse: field-spanning memcpy() on regs->a6 in do_sse()
- [High] riscv: sse: avoid clobbering tp with NULL in handle_sse() during __switch_to()
--

commit bce7a5b04f20b2889a18e324f4cb5a0c7506b960
Author: Clément Léger <cleger@rivosinc.com>

riscv: add support for SBI Supervisor Software Events extension

This commit adds architectural support for the RISC-V SBI Supervisor
Software Events (SSE) extension, including dedicated per-event stacks,
shadow call stacks, and low-level entry and exit assembly handling.

Link: https://lore.kernel.org/r/as79MHtNYd-VbUug@hu-himchau-blr.qualcomm.com/

> diff --git a/arch/riscv/kernel/sbi_sse.c b/arch/riscv/kernel/sbi_sse.c
> new file mode 100644
> index 0000000000000..032fcc07abcf7
> --- /dev/null
> +++ b/arch/riscv/kernel/sbi_sse.c
> @@ -0,0 +1,251 @@

[ ... ]

> +void noinstr do_sse(struct sse_event_arch_data *arch_evt,
> +		    struct pt_regs *regs, unsigned long hstatus)
> +{
> +	struct riscv_sse_interrupted_context context = { regs, hstatus };
> +	struct riscv_sse_interrupted_context *previous;
> +	struct sbiret sret;
> +
> +	nmi_enter();
> +	instrumentation_begin();
> +
> +	/* Retrieve missing GPRs from SBI */
> +	sret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_EVENT_ATTR_READ, arch_evt->evt_id,
> +			 SBI_SSE_ATTR_INTERRUPTED_A6,
> +			 (SBI_SSE_ATTR_INTERRUPTED_A7 -
> +			  SBI_SSE_ATTR_INTERRUPTED_A6) + 1,
> +			 (unsigned long)arch_evt->interrupted_phys, 0, 0);
> +	if (sret.error) {
> +		pr_warn("Failed to read interrupted registers for event %x: %ld\n",
> +			arch_evt->evt_id, sret.error);
> +		/* Let the client quiesce its source without using incomplete regs. */
> +		sse_handle_event(arch_evt, NULL);
> +		goto out;
> +	}
> +
> +	memcpy(&regs->a6, arch_evt->interrupted,
> +	       sizeof(*arch_evt->interrupted));
> +
> +	/* Make the interrupted frame visible while clients handle this event. */
> +	previous = this_cpu_read(riscv_sse_interrupted_context);
> +	this_cpu_write(riscv_sse_interrupted_context, &context);
> +	sse_handle_event(arch_evt, regs);
> +	this_cpu_write(riscv_sse_interrupted_context, previous);
> +
> +	if (memcmp(&regs->a6, arch_evt->interrupted,
> +		   sizeof(*arch_evt->interrupted))) {
> +		memcpy(arch_evt->interrupted, &regs->a6,
> +		       sizeof(*arch_evt->interrupted));

[Severity: High]
Does passing &regs->a6 to memcpy() and memcmp() with
sizeof(*arch_evt->interrupted) trigger a field-spanning overflow under
CONFIG_FORTIFY_SOURCE=y?

In struct pt_regs (arch/riscv/include/asm/ptrace.h), a6 and a7 are separate
unsigned long members:

	struct_group(a_regs,
		unsigned long a0;
		...
		unsigned long a6;
		unsigned long a7;
	);

Because &regs->a6 points to a single unsigned long field while
sizeof(*arch_evt->interrupted) spans two unsigned longs (a6 and a7),
fortify_memcpy_chk() sees __builtin_object_size(&regs->a6, 1) as smaller than
the copy size. This triggers a compile-time __write_overflow_field() error
when CONFIG_FORTIFY_SOURCE=y and CONFIG_WERROR=y are enabled (as well as
__read_overflow2_field() under W=1), or a runtime fortify_warn_once() inside
NMI context when an SSE event is delivered.

Could do_sse() read and write regs->a6 and regs->a7 directly (or group them
with struct_group()) instead of using memcpy() and memcmp() across the field
boundary?

[ ... ]

> diff --git a/arch/riscv/kernel/sbi_sse_entry.S b/arch/riscv/kernel/sbi_sse_entry.S
> new file mode 100644
> index 0000000000000..1e38fbb1c2a42
> --- /dev/null
> +++ b/arch/riscv/kernel/sbi_sse_entry.S
> @@ -0,0 +1,237 @@

[ ... ]

> +	asm_per_cpu_with_cpu t2 __sbi_sse_entry_task t1 t3
> +	REG_L tp, 0(t2)
> +	bnez tp, .Lcurrent_task_found
> +
> +	/* __switch_to() marks its transition window with a NULL entry task. */
> +	REG_L tp, PT_TP(sp)
> +
> +.Lcurrent_task_found:
> +	/* Nested exceptions temporarily replace these with the SSE stack. */
> +	REG_L s6, TASK_TI_KERNEL_SP(tp)

[Severity: High]
Can a higher-priority nested SSE event observe a NULL tp here if the first
event interrupted __switch_to()?

In __switch_to() (arch/riscv/kernel/entry.S), __sbi_sse_entry_task is cleared
to zero during the task switch window, relying on the interrupted tp register
remaining valid:

	asm_per_cpu t0, __sbi_sse_entry_task, t1
	REG_S zero, 0(t0)

When a lower-priority SSE event arrives during this window, handle_sse saves
the valid tp into PT_TP(sp) on its stack and then executes:

	REG_L tp, 0(t2)
	bnez tp, .Lcurrent_task_found
	REG_L tp, PT_TP(sp)

Because REG_L tp, 0(t2) loads NULL directly into tp before checking for zero,
both __sbi_sse_entry_task and the live tp register are simultaneously NULL
between those two loads. If a higher-priority SSE event preempts handle_sse
in that window, the nested handle_sse saves tp (now NULL) into its own
PT_TP(sp), loads NULL from __sbi_sse_entry_task, reloads NULL from PT_TP(sp),
and dereferences NULL at REG_L s6, TASK_TI_KERNEL_SP(tp).

Would loading __sbi_sse_entry_task into a temporary register first and only
moving it into tp when non-zero avoid clobbering the live tp register?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261029074853.35467-1-zhangzhanpeng.jasper@bytedance.com?part=2

Re: [PATCH v12 02/11] riscv: add support for SBI Supervisor Software Events extension

[Himanshu Chauhan]

Hi,

On Thu, Oct 29, 2026 at 03:48:44PM +0800, Zhanpeng Zhang wrote:
> From: Clément Léger <cleger@rivosinc.com>
> 
> The SBI SSE extension allows firmware to notify supervisor software of
> events that must be delivered independently of normal S-mode interrupts.
> Firmware saves the minimal state required to enter the supervisor handler,
> and Linux builds the synthetic handler context around it.
> 
> SSE can arrive while Linux is already in an exception entry path. At that
> point sscratch and tp may be in the middle of the normal trap-entry
> exchange, so they cannot always identify current. Store current in a
> per-CPU slot and use the hart ID passed by firmware to recover it.
> 
> Give each event, including each CPU instance of a local event, a dedicated
> stack and shadow call stack. Synchronize vmapped stack ranges before
> unmasking events so that the handler cannot take a vmalloc fault while
> running in an NMI-like context.
> 
> The handler is a synthetic supervisor episode, but completion must resume
> the context interrupted by the SSE. Preserve stvec and, when the hypervisor
> extension is present, hstatus across the handler. Read the interrupted a6
> and a7 values from the SSE attributes, construct pt_regs for the
> interrupted context, and write back any changes made by the handler.
> 
> Nested exceptions on the SSE event stack temporarily replace both
> TASK_TI_KERNEL_SP and TASK_TI_USER_SP. Preserve their original values
> across the handler and restore them before completing the event, so a
> nested exception cannot leave the interrupted task referring to the event
> stack.
> 
> On RV64 with an MMU, early trap entry also uses thread_info.a0/a1/a2
> as scratch slots before saving the full register frame. An SSE handler
> can interrupt that window and take a nested fault which overwrites the
> same slots. Preserve them in already-saved callee-saved registers and
> restore them before resuming the interrupted entry path.
> 
> Disable preemption while initializing the per-CPU entry-task slot.
> CPU hotplug callbacks run in a preemptible thread, and this write
> happens before the guards around the local SBI operations.
> 
> Compile the stack TLB helper only for VMAP_STACK or SHADOW_CALL_STACK,
> its only users, to avoid an unused-function warning in other builds.
> 
> Keep an explicit EVENT_REGISTER not-supported result distinct from other
> firmware failures. This lets clients select another delivery mechanism
> only when firmware has positively rejected the requested event.
> 
> Link: https://lore.kernel.org/r/as79MHtNYd-VbUug@hu-himchau-blr.qualcomm.com/
> Reported-by: kernel test robot <lkp@intel.com>
> Closes: https://lore.kernel.org/oe-kbuild-all/202610020009.P3ILnDi1-lkp@intel.com/
> Signed-off-by: Clément Léger <cleger@rivosinc.com>
> Co-developed-by: Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>
> Signed-off-by: Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>
> Co-developed-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
> Signed-off-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
> ---
>  arch/riscv/include/asm/asm.h         |  14 +-
>  arch/riscv/include/asm/scs.h         |   7 +
>  arch/riscv/include/asm/sse.h         |  82 +++++++++
>  arch/riscv/include/asm/thread_info.h |   1 +
>  arch/riscv/kernel/Makefile           |   1 +
>  arch/riscv/kernel/asm-offsets.c      |  14 ++
>  arch/riscv/kernel/entry.S            |  14 ++
>  arch/riscv/kernel/sbi_sse.c          | 251 +++++++++++++++++++++++++++
>  arch/riscv/kernel/sbi_sse_entry.S    | 237 +++++++++++++++++++++++++
>  9 files changed, 618 insertions(+), 3 deletions(-)
>  create mode 100644 arch/riscv/include/asm/sse.h
>  create mode 100644 arch/riscv/kernel/sbi_sse.c
>  create mode 100644 arch/riscv/kernel/sbi_sse_entry.S
> 
> diff --git a/arch/riscv/include/asm/asm.h b/arch/riscv/include/asm/asm.h
> index b8bf842d4c1..e1196caea02 100644
> --- a/arch/riscv/include/asm/asm.h
> +++ b/arch/riscv/include/asm/asm.h
> @@ -91,16 +91,24 @@
>  .endm
>  
>  #ifdef CONFIG_SMP
> -.macro asm_per_cpu dst sym tmp
> -	lw    \tmp, TASK_TI_CPU_NUM(tp)
> -	slli  \tmp, \tmp, RISCV_LGPTR
> +.macro asm_per_cpu_with_cpu dst sym tmp cpu
> +	slli  \tmp, \cpu, RISCV_LGPTR
>  	la    \dst, __per_cpu_offset
>  	add   \dst, \dst, \tmp
>  	REG_L \tmp, 0(\dst)
>  	la    \dst, \sym
>  	add   \dst, \dst, \tmp
>  .endm
> +
> +.macro asm_per_cpu dst sym tmp
> +	lw \tmp, TASK_TI_CPU_NUM(tp)
> +	asm_per_cpu_with_cpu \dst \sym \tmp \tmp
> +.endm
>  #else /* CONFIG_SMP */
> +.macro asm_per_cpu_with_cpu dst sym tmp cpu
> +	la    \dst, \sym
> +.endm
> +
>  .macro asm_per_cpu dst sym tmp
>  	la    \dst, \sym
>  .endm
> diff --git a/arch/riscv/include/asm/scs.h b/arch/riscv/include/asm/scs.h
> index 023a412fe38..0d70a35bc01 100644
> --- a/arch/riscv/include/asm/scs.h
> +++ b/arch/riscv/include/asm/scs.h

This patch doesn't apply on v7.3-rc7. Did you rebase on it?
b4 shazam failed at this patch in scs.h file.

Please check.

Regards
Himanshu

> @@ -17,6 +17,11 @@
>  	load_per_cpu gp, irq_shadow_call_stack_ptr, \tmp
>  .endm
>  
> +/* Load the per-CPU IRQ shadow call stack to gp. */
> +.macro scs_load_sse_stack reg_evt
> +	REG_L gp, SSE_REG_EVT_SHADOW_STACK(\reg_evt)
> +.endm
> +
>  /* Load task_scs_sp(current) to gp. */
>  .macro scs_load_current
>  	REG_L	gp, TASK_TI_SCS_SP(tp)
> @@ -40,6 +45,8 @@
>  .endm
>  .macro scs_load_irq_stack tmp
>  .endm
> +.macro scs_load_sse_stack reg_evt
> +.endm
>  .macro scs_load_current
>  .endm
>  .macro scs_load_current_if_task_changed prev
> diff --git a/arch/riscv/include/asm/sse.h b/arch/riscv/include/asm/sse.h
> new file mode 100644
> index 00000000000..cbd8618c6e0
> --- /dev/null
> +++ b/arch/riscv/include/asm/sse.h
> @@ -0,0 +1,82 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * Copyright (C) 2024 Rivos Inc.
> + */
> +#ifndef __ASM_SSE_H
> +#define __ASM_SSE_H
> +
> +#include <linux/printk.h>
> +#include <linux/types.h>
> +
> +#include <asm/sbi.h>
> +
> +static inline bool riscv_sse_available(void)
> +{
> +#ifdef CONFIG_RISCV_SBI
> +	return sbi_probe_extension(SBI_EXT_SSE) > 0;
> +#else
> +	return false;
> +#endif
> +}
> +
> +static inline void riscv_sse_mask_current_hart(void)
> +{
> +#ifdef CONFIG_RISCV_SBI
> +	struct sbiret ret;
> +
> +	if (!riscv_sse_available())
> +		return;
> +
> +	ret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_HART_MASK, 0, 0, 0, 0, 0, 0);
> +	if (ret.error && ret.error != SBI_ERR_ALREADY_STOPPED)
> +		pr_emerg("SSE hart mask failed: %ld\n", ret.error);
> +#endif
> +}
> +
> +#ifdef CONFIG_RISCV_SBI_SSE
> +
> +struct sse_event_interrupted_state {
> +	unsigned long a6;
> +	unsigned long a7;
> +};
> +
> +struct sse_event_arch_data {
> +	void *stack;
> +	void *shadow_stack;
> +	unsigned long tmp;
> +	struct sse_event_interrupted_state *interrupted;
> +	phys_addr_t interrupted_phys;
> +	u32 evt_id;
> +	unsigned long hart_id;
> +	unsigned int cpu_id;
> +};
> +
> +struct riscv_sse_interrupted_context {
> +	struct pt_regs *regs;
> +	unsigned long hstatus;
> +};
> +
> +static inline bool sse_event_is_global(u32 evt)
> +{
> +	return !!(evt & SBI_SSE_EVENT_GLOBAL);
> +}
> +
> +void arch_sse_event_update_cpu(struct sse_event_arch_data *arch_evt, int cpu);
> +int arch_sse_init_event(struct sse_event_arch_data *arch_evt, u32 evt_id,
> +			int cpu);
> +void arch_sse_free_event(struct sse_event_arch_data *arch_evt);
> +int arch_sse_register_event(struct sse_event_arch_data *arch_evt);
> +void arch_sse_init_cpu(void);
> +
> +void sse_handle_event(struct sse_event_arch_data *arch_evt,
> +		      struct pt_regs *regs);
> +asmlinkage void handle_sse(void);
> +asmlinkage void noinstr do_sse(struct sse_event_arch_data *arch_evt,
> +			       struct pt_regs *regs, unsigned long hstatus);
> +
> +const struct riscv_sse_interrupted_context *
> +riscv_sse_get_interrupted_context(void);
> +
> +#endif
> +
> +#endif
> diff --git a/arch/riscv/include/asm/thread_info.h b/arch/riscv/include/asm/thread_info.h
> index 55019fdfa9e..d14b45610c7 100644
> --- a/arch/riscv/include/asm/thread_info.h
> +++ b/arch/riscv/include/asm/thread_info.h
> @@ -36,6 +36,7 @@
>  #define OVERFLOW_STACK_SIZE     SZ_4K
>  
>  #define IRQ_STACK_SIZE		THREAD_SIZE
> +#define SSE_STACK_SIZE		THREAD_SIZE
>  
>  #ifndef __ASSEMBLER__
>  
> diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile
> index f54c336bf54..0728362ba0a 100644
> --- a/arch/riscv/kernel/Makefile
> +++ b/arch/riscv/kernel/Makefile
> @@ -101,6 +101,7 @@ obj-$(CONFIG_DYNAMIC_FTRACE)	+= mcount-dyn.o
>  obj-$(CONFIG_PERF_EVENTS)	+= perf_callchain.o
>  obj-$(CONFIG_HAVE_PERF_REGS)	+= perf_regs.o
>  obj-$(CONFIG_RISCV_SBI)		+= sbi.o sbi_ecall.o
> +obj-$(CONFIG_RISCV_SBI_SSE)	+= sbi_sse.o sbi_sse_entry.o
>  ifeq ($(CONFIG_RISCV_SBI), y)
>  obj-$(CONFIG_SMP)		+= sbi-ipi.o
>  obj-$(CONFIG_SMP) += cpu_ops_sbi.o
> diff --git a/arch/riscv/kernel/asm-offsets.c b/arch/riscv/kernel/asm-offsets.c
> index a75f0cfea1e..15363703cdd 100644
> --- a/arch/riscv/kernel/asm-offsets.c
> +++ b/arch/riscv/kernel/asm-offsets.c
> @@ -15,6 +15,8 @@
>  #include <asm/ptrace.h>
>  #include <asm/cpu_ops_sbi.h>
>  #include <asm/stacktrace.h>
> +#include <asm/sbi.h>
> +#include <asm/sse.h>
>  #include <asm/suspend.h>
>  
>  void asm_offsets(void);
> @@ -533,6 +535,18 @@ void asm_offsets(void)
>  	DEFINE(FREGS_A6,	    offsetof(struct __arch_ftrace_regs, a6));
>  	DEFINE(FREGS_A7,	    offsetof(struct __arch_ftrace_regs, a7));
>  #endif
> +
> +#ifdef CONFIG_RISCV_SBI_SSE
> +	OFFSET(SSE_REG_EVT_STACK, sse_event_arch_data, stack);
> +	OFFSET(SSE_REG_EVT_SHADOW_STACK, sse_event_arch_data, shadow_stack);
> +	OFFSET(SSE_REG_EVT_TMP, sse_event_arch_data, tmp);
> +	OFFSET(SSE_REG_HART_ID, sse_event_arch_data, hart_id);
> +	OFFSET(SSE_REG_CPU_ID, sse_event_arch_data, cpu_id);
> +
> +	DEFINE(SBI_EXT_SSE, SBI_EXT_SSE);
> +	DEFINE(SBI_SSE_EVENT_COMPLETE, SBI_SSE_EVENT_COMPLETE);
> +	DEFINE(ASM_NR_CPUS, CONFIG_NR_CPUS);
> +#endif
>  #ifdef CONFIG_RISCV_SBI
>  	DEFINE(SBI_EXT_FWFT, SBI_EXT_FWFT);
>  	DEFINE(SBI_EXT_FWFT_SET, SBI_EXT_FWFT_SET);
> diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
> index d799c4e56f8..0b79fa7241e 100644
> --- a/arch/riscv/kernel/entry.S
> +++ b/arch/riscv/kernel/entry.S
> @@ -424,6 +424,15 @@ SYM_FUNC_END(call_on_irq_stack)
>   * arguments are passed to schedule_tail.
>   */
>  SYM_FUNC_START(__switch_to)
> +#ifdef CONFIG_RISCV_SBI_SSE
> +	/*
> +	 * Mark the interval where tp changes from prev to next. SSE entry uses
> +	 * the interrupted tp while this per-CPU pointer is NULL.
> +	 */
> +	asm_per_cpu t0, __sbi_sse_entry_task, t1
> +	REG_S zero, 0(t0)
> +#endif
> +
>  	/* Save context into prev->thread */
>  	li    a4,  TASK_THREAD_RA
>  	add   a3, a0, a4
> @@ -470,6 +479,11 @@ SYM_FUNC_START(__switch_to)
>  	REG_L s11, TASK_THREAD_S11_RA(a4)
>  	/* The offset of thread_info in task_struct is zero. */
>  	move tp, a1
> +#ifdef CONFIG_RISCV_SBI_SSE
> +	/* Publish next only after tp contains its task_struct pointer. */
> +	asm_per_cpu t0, __sbi_sse_entry_task, t1
> +	REG_S tp, 0(t0)
> +#endif
>  	/* Switch to the next shadow call stack */
>  	scs_load_current
>  	ret
> diff --git a/arch/riscv/kernel/sbi_sse.c b/arch/riscv/kernel/sbi_sse.c
> new file mode 100644
> index 00000000000..032fcc07abc
> --- /dev/null
> +++ b/arch/riscv/kernel/sbi_sse.c
> @@ -0,0 +1,251 @@
> +// SPDX-License-Identifier: GPL-2.0-or-later
> +/*
> + * Copyright (C) 2025 Rivos Inc.
> + */
> +#include <linux/nmi.h>
> +#include <linux/sched.h>
> +#include <linux/scs.h>
> +#include <linux/bitfield.h>
> +#include <linux/percpu-defs.h>
> +#include <linux/preempt.h>
> +#include <linux/string.h>
> +
> +#include <asm/asm-prototypes.h>
> +#include <asm/switch_to.h>
> +#include <asm/irq_stack.h>
> +#include <asm/sbi.h>
> +#include <asm/sse.h>
> +#include <asm/tlbflush.h>
> +
> +DEFINE_PER_CPU(struct task_struct *, __sbi_sse_entry_task);
> +static DEFINE_PER_CPU(struct riscv_sse_interrupted_context *,
> +			      riscv_sse_interrupted_context);
> +
> +const struct riscv_sse_interrupted_context *
> +riscv_sse_get_interrupted_context(void)
> +{
> +	return this_cpu_read(riscv_sse_interrupted_context);
> +}
> +
> +void __weak sse_handle_event(struct sse_event_arch_data *arch_evt, struct pt_regs *regs)
> +{
> +}
> +
> +void noinstr do_sse(struct sse_event_arch_data *arch_evt,
> +		    struct pt_regs *regs, unsigned long hstatus)
> +{
> +	struct riscv_sse_interrupted_context context = { regs, hstatus };
> +	struct riscv_sse_interrupted_context *previous;
> +	struct sbiret sret;
> +
> +	nmi_enter();
> +	instrumentation_begin();
> +
> +	/* Retrieve missing GPRs from SBI */
> +	sret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_EVENT_ATTR_READ, arch_evt->evt_id,
> +			 SBI_SSE_ATTR_INTERRUPTED_A6,
> +			 (SBI_SSE_ATTR_INTERRUPTED_A7 -
> +			  SBI_SSE_ATTR_INTERRUPTED_A6) + 1,
> +			 (unsigned long)arch_evt->interrupted_phys, 0, 0);
> +	if (sret.error) {
> +		pr_warn("Failed to read interrupted registers for event %x: %ld\n",
> +			arch_evt->evt_id, sret.error);
> +		/* Let the client quiesce its source without using incomplete regs. */
> +		sse_handle_event(arch_evt, NULL);
> +		goto out;
> +	}
> +
> +	memcpy(&regs->a6, arch_evt->interrupted,
> +	       sizeof(*arch_evt->interrupted));
> +
> +	/* Make the interrupted frame visible while clients handle this event. */
> +	previous = this_cpu_read(riscv_sse_interrupted_context);
> +	this_cpu_write(riscv_sse_interrupted_context, &context);
> +	sse_handle_event(arch_evt, regs);
> +	this_cpu_write(riscv_sse_interrupted_context, previous);
> +
> +	if (memcmp(&regs->a6, arch_evt->interrupted,
> +		   sizeof(*arch_evt->interrupted))) {
> +		memcpy(arch_evt->interrupted, &regs->a6,
> +		       sizeof(*arch_evt->interrupted));
> +		sret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_EVENT_ATTR_WRITE,
> +				 arch_evt->evt_id, SBI_SSE_ATTR_INTERRUPTED_A6,
> +				 (SBI_SSE_ATTR_INTERRUPTED_A7 -
> +				  SBI_SSE_ATTR_INTERRUPTED_A6) + 1,
> +				 (unsigned long)arch_evt->interrupted_phys, 0, 0);
> +		/*
> +		 * If writeback fails, COMPLETE resumes with firmware's original
> +		 * a6/a7 rather than treating the shared buffer as committed.
> +		 */
> +		if (sret.error)
> +			pr_warn("Failed to write interrupted registers for event %x: %ld\n",
> +				arch_evt->evt_id, sret.error);
> +	}
> +
> +out:
> +	instrumentation_end();
> +	nmi_exit();
> +}
> +
> +static void *alloc_to_stack_pointer(void *alloc)
> +{
> +	return alloc ? alloc + SSE_STACK_SIZE : NULL;
> +}
> +
> +static void *stack_pointer_to_alloc(void *stack)
> +{
> +	return stack ? stack - SSE_STACK_SIZE : NULL;
> +}
> +
> +#if defined(CONFIG_VMAP_STACK) || defined(CONFIG_SHADOW_CALL_STACK)
> +static void arch_sse_flush_tlb_range(struct sse_event_arch_data *arch_evt,
> +				     unsigned long start, unsigned long size)
> +{
> +	unsigned long end = start + size;
> +
> +	if (sse_event_is_global(arch_evt->evt_id))
> +		flush_tlb_kernel_range(start, end);
> +	else
> +		local_flush_tlb_kernel_range(start, end);
> +}
> +#endif
> +
> +static void arch_sse_shadow_stack_cpu_sync(struct sse_event_arch_data *arch_evt)
> +{
> +#ifdef CONFIG_SHADOW_CALL_STACK
> +	if (arch_evt->shadow_stack)
> +		arch_sse_flush_tlb_range(arch_evt,
> +					 (unsigned long)arch_evt->shadow_stack,
> +					 SCS_SIZE);
> +#endif
> +}
> +
> +#ifdef CONFIG_VMAP_STACK
> +static void *sse_stack_alloc(unsigned int cpu)
> +{
> +	void *stack = arch_alloc_vmap_stack(SSE_STACK_SIZE, cpu_to_node(cpu));
> +
> +	return alloc_to_stack_pointer(stack);
> +}
> +
> +static void sse_stack_free(void *stack)
> +{
> +	vfree(stack_pointer_to_alloc(stack));
> +}
> +
> +static void arch_sse_stack_cpu_sync(struct sse_event_arch_data *arch_evt)
> +{
> +	void *p_stack = arch_evt->stack;
> +	unsigned long stack = (unsigned long)stack_pointer_to_alloc(p_stack);
> +
> +	/*
> +	 * Flush the tlb to avoid taking any exception when accessing the
> +	 * vmapped stack inside the SSE handler
> +	 */
> +	arch_sse_flush_tlb_range(arch_evt, stack, SSE_STACK_SIZE);
> +
> +	arch_sse_shadow_stack_cpu_sync(arch_evt);
> +}
> +#else /* CONFIG_VMAP_STACK */
> +static void *sse_stack_alloc(unsigned int cpu)
> +{
> +	void *stack = kmalloc(SSE_STACK_SIZE, GFP_KERNEL);
> +
> +	return alloc_to_stack_pointer(stack);
> +}
> +
> +static void sse_stack_free(void *stack)
> +{
> +	kfree(stack_pointer_to_alloc(stack));
> +}
> +
> +static void arch_sse_stack_cpu_sync(struct sse_event_arch_data *arch_evt)
> +{
> +	arch_sse_shadow_stack_cpu_sync(arch_evt);
> +}
> +#endif /* CONFIG_VMAP_STACK */
> +
> +static int sse_init_scs(int cpu, struct sse_event_arch_data *arch_evt)
> +{
> +	void *stack;
> +
> +	if (!scs_is_enabled())
> +		return 0;
> +
> +	stack = scs_alloc(cpu_to_node(cpu));
> +	if (!stack)
> +		return -ENOMEM;
> +
> +	arch_evt->shadow_stack = stack;
> +
> +	return 0;
> +}
> +
> +void arch_sse_event_update_cpu(struct sse_event_arch_data *arch_evt, int cpu)
> +{
> +	arch_evt->cpu_id = cpu;
> +	arch_evt->hart_id = cpuid_to_hartid_map(cpu);
> +}
> +
> +void arch_sse_init_cpu(void)
> +{
> +	preempt_disable();
> +	this_cpu_write(__sbi_sse_entry_task, current);
> +	preempt_enable();
> +}
> +
> +int arch_sse_init_event(struct sse_event_arch_data *arch_evt, u32 evt_id,
> +			int cpu)
> +{
> +	void *stack;
> +
> +	arch_evt->interrupted = kmalloc_obj(*arch_evt->interrupted, GFP_KERNEL);
> +	if (!arch_evt->interrupted)
> +		return -ENOMEM;
> +
> +	arch_evt->evt_id = evt_id;
> +	stack = sse_stack_alloc(cpu);
> +	if (!stack)
> +		goto err_free_interrupted;
> +
> +	arch_evt->stack = stack;
> +
> +	if (sse_init_scs(cpu, arch_evt)) {
> +		sse_stack_free(arch_evt->stack);
> +		goto err_free_interrupted;
> +	}
> +
> +	/* kmalloc keeps the two adjacent SBI attribute words contiguous. */
> +	arch_evt->interrupted_phys = virt_to_phys(arch_evt->interrupted);
> +
> +	arch_sse_event_update_cpu(arch_evt, cpu);
> +
> +	return 0;
> +
> +err_free_interrupted:
> +	kfree(arch_evt->interrupted);
> +	arch_evt->interrupted = NULL;
> +	return -ENOMEM;
> +}
> +
> +void arch_sse_free_event(struct sse_event_arch_data *arch_evt)
> +{
> +	scs_free(arch_evt->shadow_stack);
> +	sse_stack_free(arch_evt->stack);
> +	kfree(arch_evt->interrupted);
> +}
> +
> +int arch_sse_register_event(struct sse_event_arch_data *arch_evt)
> +{
> +	struct sbiret sret;
> +
> +	arch_sse_stack_cpu_sync(arch_evt);
> +
> +	sret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_EVENT_REGISTER, arch_evt->evt_id,
> +			 (unsigned long)handle_sse, (unsigned long)arch_evt, 0,
> +			 0, 0);
> +	if (sret.error == SBI_ERR_NOT_SUPPORTED)
> +		return -EOPNOTSUPP;
> +
> +	return sbi_err_map_linux_errno(sret.error);
> +}
> diff --git a/arch/riscv/kernel/sbi_sse_entry.S b/arch/riscv/kernel/sbi_sse_entry.S
> new file mode 100644
> index 00000000000..1e38fbb1c2a
> --- /dev/null
> +++ b/arch/riscv/kernel/sbi_sse_entry.S
> @@ -0,0 +1,237 @@
> +/* SPDX-License-Identifier: GPL-2.0-or-later */
> +/*
> + * Copyright (C) 2025 Rivos Inc.
> + */
> +
> +#include <linux/init.h>
> +#include <linux/linkage.h>
> +
> +#include <asm/alternative-macros.h>
> +#include <asm/asm.h>
> +#include <asm/csr.h>
> +#include <asm/hwcap.h>
> +#include <asm/scs.h>
> +
> +/* When entering handle_sse, the following registers are set:
> + * a6: contains the hartid
> + * a7: contains a sse_event_arch_data struct pointer
> + */
> +SYM_CODE_START(handle_sse)
> +	/* Save stack temporarily */
> +	REG_S sp, SSE_REG_EVT_TMP(a7)
> +	/* Set entry stack */
> +	REG_L sp, SSE_REG_EVT_STACK(a7)
> +
> +	addi sp, sp, -(PT_SIZE_ON_STACK)
> +	REG_S ra, PT_RA(sp)
> +	REG_S s0, PT_S0(sp)
> +	REG_S s1, PT_S1(sp)
> +	REG_S s2, PT_S2(sp)
> +	REG_S s3, PT_S3(sp)
> +	REG_S s4, PT_S4(sp)
> +	REG_S s5, PT_S5(sp)
> +	REG_S s6, PT_S6(sp)
> +	REG_S s7, PT_S7(sp)
> +	REG_S s8, PT_S8(sp)
> +	REG_S s9, PT_S9(sp)
> +	REG_S s10, PT_S10(sp)
> +	REG_S s11, PT_S11(sp)
> +	REG_S tp, PT_TP(sp)
> +	REG_S t0, PT_T0(sp)
> +	REG_S t1, PT_T1(sp)
> +	REG_S t2, PT_T2(sp)
> +	REG_S t3, PT_T3(sp)
> +	REG_S t4, PT_T4(sp)
> +	REG_S t5, PT_T5(sp)
> +	REG_S t6, PT_T6(sp)
> +	REG_S gp, PT_GP(sp)
> +	REG_S a0, PT_A0(sp)
> +	REG_S a1, PT_A1(sp)
> +	REG_S a2, PT_A2(sp)
> +	REG_S a3, PT_A3(sp)
> +	REG_S a4, PT_A4(sp)
> +	REG_S a5, PT_A5(sp)
> +
> +	/* Retrieve entry sp */
> +	REG_L a4, SSE_REG_EVT_TMP(a7)
> +	/* Save CSRs */
> +	csrr a0, CSR_EPC
> +	csrr a1, CSR_SSTATUS
> +	csrr a2, CSR_STVAL
> +	csrr a3, CSR_SCAUSE
> +
> +	REG_S a0, PT_EPC(sp)
> +	REG_S a1, PT_STATUS(sp)
> +	REG_S a2, PT_BADADDR(sp)
> +	REG_S a3, PT_CAUSE(sp)
> +	REG_S a4, PT_SP(sp)
> +
> +	/* Disable user memory access and floating/vector computing */
> +	li t0, SR_SUM | SR_FS_VS
> +	csrc CSR_STATUS, t0
> +
> +	load_global_pointer
> +	scs_load_sse_stack a7
> +
> +#ifdef CONFIG_SMP
> +	REG_L t4, SSE_REG_HART_ID(a7)
> +	lw t3, SSE_REG_CPU_ID(a7)
> +
> +	bne t4, a6, .Lfind_hart_id_slowpath
> +
> +.Lcpu_id_found:
> +#else
> +	mv t3, zero
> +#endif
> +
> +	asm_per_cpu_with_cpu t2 __sbi_sse_entry_task t1 t3
> +	REG_L tp, 0(t2)
> +	bnez tp, .Lcurrent_task_found
> +
> +	/* __switch_to() marks its transition window with a NULL entry task. */
> +	REG_L tp, PT_TP(sp)
> +
> +.Lcurrent_task_found:
> +	/* Nested exceptions temporarily replace these with the SSE stack. */
> +	REG_L s6, TASK_TI_KERNEL_SP(tp)
> +	REG_L s7, TASK_TI_USER_SP(tp)
> +#if defined(CONFIG_64BIT) && defined(CONFIG_MMU)
> +	/* A nested fault must not overwrite an interrupted trap's scratch slots. */
> +	REG_L s8, TASK_TI_A0(tp)
> +	REG_L s9, TASK_TI_A1(tp)
> +	REG_L s10, TASK_TI_A2(tp)
> +#endif
> +
> +	mv a1, sp /* pt_regs on stack */
> +
> +	/*
> +	 * Run the SSE handler with the normal exception vector, but restore the
> +	 * interrupted stvec before completing the event. SSE can arrive while
> +	 * the kernel is using a temporary trap vector in a sensitive entry path.
> +	 */
> +	csrr s3, CSR_STVEC
> +	la t0, handle_exception
> +	csrw CSR_STVEC, t0
> +
> +	/*
> +	 * Preserve the full HS-mode virtualization state across the handler.
> +	 * hstatus is live supervisor state rather than an SSE interrupted
> +	 * attribute, and OpenSBI consumes hstatus.SPV during event completion.
> +	 * Saving the whole CSR keeps the handler episode transparent to KVM and
> +	 * avoids having to infer which hstatus bits may matter to a guest resume.
> +	 */
> +	li s5, 0
> +	ALTERNATIVE("nop", "csrr s5, hstatus", 0, RISCV_ISA_EXT_H, 1)
> +
> +	/*
> +	 * Save sscratch for restoration since we might have interrupted the
> +	 * kernel in early exception path and thus, we don't know the content of
> +	 * sscratch.
> +	 */
> +	csrrw s4, CSR_SSCRATCH, x0
> +
> +	mv a0, a7
> +	mv a2, s5
> +
> +	call do_sse
> +
> +	/* Leave no reference to the event stack in the interrupted task. */
> +	REG_S s7, TASK_TI_USER_SP(tp)
> +	REG_S s6, TASK_TI_KERNEL_SP(tp)
> +#if defined(CONFIG_64BIT) && defined(CONFIG_MMU)
> +	REG_S s8, TASK_TI_A0(tp)
> +	REG_S s9, TASK_TI_A1(tp)
> +	REG_S s10, TASK_TI_A2(tp)
> +#endif
> +
> +	csrw CSR_SSCRATCH, s4
> +	ALTERNATIVE("nop", "csrw hstatus, s5", 0, RISCV_ISA_EXT_H, 1)
> +	csrw CSR_STVEC, s3
> +
> +	REG_L a0, PT_STATUS(sp)
> +	REG_L a1, PT_EPC(sp)
> +	REG_L a2, PT_BADADDR(sp)
> +	REG_L a3, PT_CAUSE(sp)
> +	csrw CSR_SSTATUS, a0
> +	csrw CSR_EPC, a1
> +	csrw CSR_STVAL, a2
> +	csrw CSR_SCAUSE, a3
> +
> +	REG_L ra, PT_RA(sp)
> +	REG_L s0, PT_S0(sp)
> +	REG_L s1, PT_S1(sp)
> +	REG_L s2, PT_S2(sp)
> +	REG_L s3, PT_S3(sp)
> +	REG_L s4, PT_S4(sp)
> +	REG_L s5, PT_S5(sp)
> +	REG_L s6, PT_S6(sp)
> +	REG_L s7, PT_S7(sp)
> +	REG_L s8, PT_S8(sp)
> +	REG_L s9, PT_S9(sp)
> +	REG_L s10, PT_S10(sp)
> +	REG_L s11, PT_S11(sp)
> +	REG_L tp, PT_TP(sp)
> +	REG_L t0, PT_T0(sp)
> +	REG_L t1, PT_T1(sp)
> +	REG_L t2, PT_T2(sp)
> +	REG_L t3, PT_T3(sp)
> +	REG_L t4, PT_T4(sp)
> +	REG_L t5, PT_T5(sp)
> +	REG_L t6, PT_T6(sp)
> +	REG_L gp, PT_GP(sp)
> +	REG_L a0, PT_A0(sp)
> +	REG_L a1, PT_A1(sp)
> +	REG_L a2, PT_A2(sp)
> +	REG_L a3, PT_A3(sp)
> +	REG_L a4, PT_A4(sp)
> +	REG_L a5, PT_A5(sp)
> +
> +	REG_L sp, PT_SP(sp)
> +
> +	li a7, SBI_EXT_SSE
> +	li a6, SBI_SSE_EVENT_COMPLETE
> +	ecall
> +
> +	/*
> +	 * COMPLETE must resume the interrupted context and never return. Trap
> +	 * through the normal vector instead of falling into adjacent assembly.
> +	 */
> +	la t0, handle_exception
> +	csrw CSR_STVEC, t0
> +	ebreak
> +	/* The fatal trap must not return; execution should never reach here. */
> +
> +#ifdef CONFIG_SMP
> +.Lfind_hart_id_slowpath:
> +
> +	/* Restore current task struct from __sbi_sse_entry_task */
> +	li t1, ASM_NR_CPUS
> +	/* Slowpath to find the CPU id associated to the hart id */
> +	la t0, __cpuid_to_hartid_map
> +	li t3, 0
> +
> +.Lhart_id_loop:
> +	REG_L t2, 0(t0)
> +	beq t2, a6, .Lcpu_id_found
> +
> +	/* Increment pointer and CPU number */
> +	addi t3, t3, 1
> +	addi t0, t0, RISCV_SZPTR
> +	bltu t3, t1, .Lhart_id_loop
> +
> +	/*
> +	 * This should never happen since we expect the hart_id to match one
> +	 * of our CPU, but better be safe than sorry
> +	 */
> +	la tp, init_task
> +	la a0, sse_hart_id_panic_string
> +	la t0, panic
> +	jalr t0
> +#endif
> +
> +SYM_CODE_END(handle_sse)
> +ASM_NOKPROBE(handle_sse)
> +
> +SYM_DATA_START_LOCAL(sse_hart_id_panic_string)
> +    .ascii "Unable to match hart_id with cpu\0"
> +SYM_DATA_END(sse_hart_id_panic_string)
> -- 
> 2.50.1 (Apple Git-155)
> 

Re: [PATCH v12 02/11] riscv: add support for SBI Supervisor Software Events extension

[Zhanpeng Zhang]

Hi,

On 10/29/26 9:48 PM, Himanshu Chauhan wrote:
> Hi,
>
> On Thu, Oct 29, 2026 at 03:48:44PM +0800, Zhanpeng Zhang wrote:
>> From: Clément Léger <cleger@rivosinc.com>
>>
>> The SBI SSE extension allows firmware to notify supervisor software of
>> events that must be delivered independently of normal S-mode interrupts.
>> Firmware saves the minimal state required to enter the supervisor handler,
>> and Linux builds the synthetic handler context around it.
>>
>> SSE can arrive while Linux is already in an exception entry path. At that
>> point sscratch and tp may be in the middle of the normal trap-entry
>> exchange, so they cannot always identify current. Store current in a
>> per-CPU slot and use the hart ID passed by firmware to recover it.
>>
>> Give each event, including each CPU instance of a local event, a dedicated
>> stack and shadow call stack. Synchronize vmapped stack ranges before
>> unmasking events so that the handler cannot take a vmalloc fault while
>> running in an NMI-like context.
>>
>> The handler is a synthetic supervisor episode, but completion must resume
>> the context interrupted by the SSE. Preserve stvec and, when the hypervisor
>> extension is present, hstatus across the handler. Read the interrupted a6
>> and a7 values from the SSE attributes, construct pt_regs for the
>> interrupted context, and write back any changes made by the handler.
>>
>> Nested exceptions on the SSE event stack temporarily replace both
>> TASK_TI_KERNEL_SP and TASK_TI_USER_SP. Preserve their original values
>> across the handler and restore them before completing the event, so a
>> nested exception cannot leave the interrupted task referring to the event
>> stack.
>>
>> On RV64 with an MMU, early trap entry also uses thread_info.a0/a1/a2
>> as scratch slots before saving the full register frame. An SSE handler
>> can interrupt that window and take a nested fault which overwrites the
>> same slots. Preserve them in already-saved callee-saved registers and
>> restore them before resuming the interrupted entry path.
>>
>> Disable preemption while initializing the per-CPU entry-task slot.
>> CPU hotplug callbacks run in a preemptible thread, and this write
>> happens before the guards around the local SBI operations.
>>
>> Compile the stack TLB helper only for VMAP_STACK or SHADOW_CALL_STACK,
>> its only users, to avoid an unused-function warning in other builds.
>>
>> Keep an explicit EVENT_REGISTER not-supported result distinct from other
>> firmware failures. This lets clients select another delivery mechanism
>> only when firmware has positively rejected the requested event.
>>
>> Link: https://lore.kernel.org/r/as79MHtNYd-VbUug@hu-himchau-blr.qualcomm.com/
>> Reported-by: kernel test robot <lkp@intel.com>
>> Closes: https://lore.kernel.org/oe-kbuild-all/202610020009.P3ILnDi1-lkp@intel.com/
>> Signed-off-by: Clément Léger <cleger@rivosinc.com>
>> Co-developed-by: Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>
>> Signed-off-by: Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>
>> Co-developed-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
>> Signed-off-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
>> ---
>>   arch/riscv/include/asm/asm.h         |  14 +-
>>   arch/riscv/include/asm/scs.h         |   7 +
>>   arch/riscv/include/asm/sse.h         |  82 +++++++++
>>   arch/riscv/include/asm/thread_info.h |   1 +
>>   arch/riscv/kernel/Makefile           |   1 +
>>   arch/riscv/kernel/asm-offsets.c      |  14 ++
>>   arch/riscv/kernel/entry.S            |  14 ++
>>   arch/riscv/kernel/sbi_sse.c          | 251 +++++++++++++++++++++++++++
>>   arch/riscv/kernel/sbi_sse_entry.S    | 237 +++++++++++++++++++++++++
>>   9 files changed, 618 insertions(+), 3 deletions(-)
>>   create mode 100644 arch/riscv/include/asm/sse.h
>>   create mode 100644 arch/riscv/kernel/sbi_sse.c
>>   create mode 100644 arch/riscv/kernel/sbi_sse_entry.S
>>
>> diff --git a/arch/riscv/include/asm/asm.h b/arch/riscv/include/asm/asm.h
>> index b8bf842d4c1..e1196caea02 100644
>> --- a/arch/riscv/include/asm/asm.h
>> +++ b/arch/riscv/include/asm/asm.h
>> @@ -91,16 +91,24 @@
>>   .endm
>>
>>   #ifdef CONFIG_SMP
>> -.macro asm_per_cpu dst sym tmp
>> -	lw    \tmp, TASK_TI_CPU_NUM(tp)
>> -	slli  \tmp, \tmp, RISCV_LGPTR
>> +.macro asm_per_cpu_with_cpu dst sym tmp cpu
>> +	slli  \tmp, \cpu, RISCV_LGPTR
>>   	la    \dst, __per_cpu_offset
>>   	add   \dst, \dst, \tmp
>>   	REG_L \tmp, 0(\dst)
>>   	la    \dst, \sym
>>   	add   \dst, \dst, \tmp
>>   .endm
>> +
>> +.macro asm_per_cpu dst sym tmp
>> +	lw \tmp, TASK_TI_CPU_NUM(tp)
>> +	asm_per_cpu_with_cpu \dst \sym \tmp \tmp
>> +.endm
>>   #else /* CONFIG_SMP */
>> +.macro asm_per_cpu_with_cpu dst sym tmp cpu
>> +	la    \dst, \sym
>> +.endm
>> +
>>   .macro asm_per_cpu dst sym tmp
>>   	la    \dst, \sym
>>   .endm
>> diff --git a/arch/riscv/include/asm/scs.h b/arch/riscv/include/asm/scs.h
>> index 023a412fe38..0d70a35bc01 100644
>> --- a/arch/riscv/include/asm/scs.h
>> +++ b/arch/riscv/include/asm/scs.h
>
> This patch doesn't apply on v7.3-rc7. Did you rebase on it?
> b4 shazam failed at this patch in scs.h file.
>
> Please check.
>
> Regards
> Himanshu
>

V12 uses a5ae1bf65cee, the same base as Anup's v11 and still the current
riscv/for-next tip, as Paul requested.

rc7 has an additional scs.h change that causes this conflict. Could you
please apply the series on a5ae1bf65cee instead?

>> [...]
Thanks,
Zhanpeng

[PATCH v12 03/11] riscv: sse: mask events during shutdown and kexec

[Zhanpeng Zhang]

SSE delivery is independent of normal S-mode interrupts. Firmware may also
retain an event registration until Linux explicitly unregisters it. A hart
must therefore stop accepting SSE events before Linux stops servicing the
registered handler.

Mask SSE on the local hart before panic stop, CPU stop, restart, poweroff,
and crash shutdown paths. This prevents firmware from entering Linux-owned
handler state after the corresponding CPU or kernel context is no longer
valid.

A crash kernel cannot identify or take ownership of registrations
inherited from the crashed kernel. Reject a later normal kexec while such
SSE state may still exist, rather than transferring unknown firmware state
to another kernel.

The restriction is deliberately based on firmware SSE availability, not
CONFIG_RISCV_SBI_SSE or registrations made by the crash kernel. Disabling
SSE in the crash kernel does not establish that registrations left by the
crashed kernel are absent. Consequently, normal kexec from a crash kernel
is also rejected when firmware advertises SSE but no registrations were
actually left behind. This conservative restriction remains until a safe
way to establish or clear inherited firmware state is available.

Link: https://lore.kernel.org/r/as7-3PbWDxygadnJ@hu-himchau-blr.qualcomm.com/
Reviewed-by: Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>
Signed-off-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
---
 arch/riscv/kernel/machine_kexec.c | 11 +++++++++++
 arch/riscv/kernel/reset.c         | 18 ++++++++++++++++++
 arch/riscv/kernel/smp.c           | 17 +++++++++++++++++
 3 files changed, 46 insertions(+)

diff --git a/arch/riscv/kernel/machine_kexec.c b/arch/riscv/kernel/machine_kexec.c
index 738df176ff6..24e7affae70 100644
--- a/arch/riscv/kernel/machine_kexec.c
+++ b/arch/riscv/kernel/machine_kexec.c
@@ -14,10 +14,13 @@
 #include <asm/set_memory.h>	/* For set_memory_x() */
 #include <linux/compiler.h>	/* For unreachable() */
 #include <linux/cpu.h>		/* For cpu_down() */
+#include <linux/crash_dump.h>
 #include <linux/reboot.h>
 #include <linux/interrupt.h>
 #include <linux/irq.h>
 
+#include <asm/sse.h>
+
 /*
  * machine_kexec_prepare - Initialize kexec
  *
@@ -36,6 +39,13 @@ machine_kexec_prepare(struct kimage *image)
 	unsigned int control_code_buffer_sz = 0;
 	int i = 0;
 
+	/* A crash kernel cannot tear down registrations inherited from firmware. */
+	if (is_kdump_kernel() && image->type != KEXEC_TYPE_CRASH &&
+	    riscv_sse_available()) {
+		pr_err("Normal kexec from a crash kernel is unsupported with SSE\n");
+		return -EOPNOTSUPP;
+	}
+
 	/* Find the Flattened Device Tree and save its physical address */
 	for (i = 0; i < image->nr_segments; i++) {
 		if (image->segment[i].memsz <= sizeof(fdt))
@@ -127,6 +137,7 @@ void
 machine_crash_shutdown(struct pt_regs *regs)
 {
 	local_irq_disable();
+	riscv_sse_mask_current_hart();
 
 	/* shutdown non-crashing cpus */
 	crash_smp_send_stop();
diff --git a/arch/riscv/kernel/reset.c b/arch/riscv/kernel/reset.c
index 14eb08a6db8..fdab37e7ae5 100644
--- a/arch/riscv/kernel/reset.c
+++ b/arch/riscv/kernel/reset.c
@@ -6,6 +6,20 @@
 #include <linux/efi.h>
 #include <linux/reboot.h>
 #include <linux/pm.h>
+#include <linux/smp.h>
+
+#include <asm/sse.h>
+
+#ifndef CONFIG_SMP
+void __noreturn panic_smp_self_stop(void)
+{
+	riscv_sse_mask_current_hart();
+	local_irq_disable();
+
+	for (;;)
+		cpu_relax();
+}
+#endif
 
 static void __noreturn default_power_off(void)
 {
@@ -18,6 +32,8 @@ EXPORT_SYMBOL(pm_power_off);
 
 void machine_restart(char *cmd)
 {
+	riscv_sse_mask_current_hart();
+
 	/*
 	 * UpdateCapsule() depends on the system being reset via ResetSystem().
 	 */
@@ -30,12 +46,14 @@ void machine_restart(char *cmd)
 
 void machine_halt(void)
 {
+	riscv_sse_mask_current_hart();
 	do_kernel_power_off();
 	default_power_off();
 }
 
 void machine_power_off(void)
 {
+	riscv_sse_mask_current_hart();
 	do_kernel_power_off();
 	default_power_off();
 }
diff --git a/arch/riscv/kernel/smp.c b/arch/riscv/kernel/smp.c
index eddbc3ad06c..9ad38df4827 100644
--- a/arch/riscv/kernel/smp.c
+++ b/arch/riscv/kernel/smp.c
@@ -22,10 +22,13 @@
 #include <linux/irq.h>
 #include <linux/irq_work.h>
 #include <linux/nmi.h>
+#include <linux/panic.h>
 
 #include <asm/tlbflush.h>
 #include <asm/cacheflush.h>
 #include <asm/cpu_ops.h>
+#include <asm/sbi.h>
+#include <asm/sse.h>
 
 enum ipi_message_type {
 	IPI_RESCHEDULE,
@@ -78,8 +81,18 @@ int riscv_hartid_to_cpuid(unsigned long hartid)
 	return -ENOENT;
 }
 
+void __noreturn panic_smp_self_stop(void)
+{
+	riscv_sse_mask_current_hart();
+	local_irq_disable();
+
+	for (;;)
+		cpu_relax();
+}
+
 static void ipi_stop(void)
 {
+	riscv_sse_mask_current_hart();
 	set_cpu_online(smp_processor_id(), false);
 	while (1)
 		wait_for_interrupt();
@@ -90,6 +103,7 @@ static atomic_t waiting_for_crash_ipi = ATOMIC_INIT(0);
 
 static inline void ipi_cpu_crash_stop(unsigned int cpu, struct pt_regs *regs)
 {
+	riscv_sse_mask_current_hart();
 	crash_save_cpu(regs, cpu);
 
 	atomic_dec(&waiting_for_crash_ipi);
@@ -253,6 +267,8 @@ void smp_send_stop(void)
 {
 	unsigned long timeout;
 
+	riscv_sse_mask_current_hart();
+
 	if (num_online_cpus() > 1) {
 		cpumask_t mask;
 
@@ -300,6 +316,7 @@ void crash_smp_send_stop(void)
 		return;
 
 	cpus_stopped = 1;
+	riscv_sse_mask_current_hart();
 
 	/*
 	 * If this cpu is the only one alive at this point in time, online or
-- 
2.50.1 (Apple Git-155)

Re: [PATCH v12 03/11] riscv: sse: mask events during shutdown and kexec

[Himanshu Chauhan]

On Thu, Oct 29, 2026 at 03:48:45PM +0800, Zhanpeng Zhang wrote:
> SSE delivery is independent of normal S-mode interrupts. Firmware may also
> retain an event registration until Linux explicitly unregisters it. A hart
> must therefore stop accepting SSE events before Linux stops servicing the
> registered handler.
> 
> Mask SSE on the local hart before panic stop, CPU stop, restart, poweroff,
> and crash shutdown paths. This prevents firmware from entering Linux-owned
> handler state after the corresponding CPU or kernel context is no longer
> valid.
> 
> A crash kernel cannot identify or take ownership of registrations
> inherited from the crashed kernel. Reject a later normal kexec while such
> SSE state may still exist, rather than transferring unknown firmware state
> to another kernel.
> 
> The restriction is deliberately based on firmware SSE availability, not
> CONFIG_RISCV_SBI_SSE or registrations made by the crash kernel. Disabling
> SSE in the crash kernel does not establish that registrations left by the
> crashed kernel are absent. Consequently, normal kexec from a crash kernel
> is also rejected when firmware advertises SSE but no registrations were
> actually left behind. This conservative restriction remains until a safe
> way to establish or clear inherited firmware state is available.
> 
> Link: https://lore.kernel.org/r/as7-3PbWDxygadnJ@hu-himchau-blr.qualcomm.com/
> Reviewed-by: Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>
> Signed-off-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
> ---
>  arch/riscv/kernel/machine_kexec.c | 11 +++++++++++
>  arch/riscv/kernel/reset.c         | 18 ++++++++++++++++++
>  arch/riscv/kernel/smp.c           | 17 +++++++++++++++++
>  3 files changed, 46 insertions(+)
> 
> diff --git a/arch/riscv/kernel/machine_kexec.c b/arch/riscv/kernel/machine_kexec.c
> index 738df176ff6..24e7affae70 100644
> --- a/arch/riscv/kernel/machine_kexec.c
> +++ b/arch/riscv/kernel/machine_kexec.c
> @@ -14,10 +14,13 @@
>  #include <asm/set_memory.h>	/* For set_memory_x() */
>  #include <linux/compiler.h>	/* For unreachable() */
>  #include <linux/cpu.h>		/* For cpu_down() */
> +#include <linux/crash_dump.h>
>  #include <linux/reboot.h>
>  #include <linux/interrupt.h>
>  #include <linux/irq.h>
>  
> +#include <asm/sse.h>
> +
>  /*
>   * machine_kexec_prepare - Initialize kexec
>   *
> @@ -36,6 +39,13 @@ machine_kexec_prepare(struct kimage *image)
>  	unsigned int control_code_buffer_sz = 0;
>  	int i = 0;
>  
> +	/* A crash kernel cannot tear down registrations inherited from firmware. */
> +	if (is_kdump_kernel() && image->type != KEXEC_TYPE_CRASH &&
> +	    riscv_sse_available()) {
> +		pr_err("Normal kexec from a crash kernel is unsupported with SSE\n");
> +		return -EOPNOTSUPP;

I think I had pointed this out in v10 also. This rejects a normal kexec from any kdump kernel whenever firmware merely advertises SSE, regardless of CONFIG_RISCV_SBI_SSE or whether anything is registered. Can you check if its breaking kexec() with SSE just being advertised?

Regards
Himanshu

> +	}
> +
>  	/* Find the Flattened Device Tree and save its physical address */
>  	for (i = 0; i < image->nr_segments; i++) {
>  		if (image->segment[i].memsz <= sizeof(fdt))
> @@ -127,6 +137,7 @@ void
>  machine_crash_shutdown(struct pt_regs *regs)
>  {
>  	local_irq_disable();
> +	riscv_sse_mask_current_hart();
>  
>  	/* shutdown non-crashing cpus */
>  	crash_smp_send_stop();
> diff --git a/arch/riscv/kernel/reset.c b/arch/riscv/kernel/reset.c
> index 14eb08a6db8..fdab37e7ae5 100644
> --- a/arch/riscv/kernel/reset.c
> +++ b/arch/riscv/kernel/reset.c
> @@ -6,6 +6,20 @@
>  #include <linux/efi.h>
>  #include <linux/reboot.h>
>  #include <linux/pm.h>
> +#include <linux/smp.h>
> +
> +#include <asm/sse.h>
> +
> +#ifndef CONFIG_SMP
> +void __noreturn panic_smp_self_stop(void)
> +{
> +	riscv_sse_mask_current_hart();
> +	local_irq_disable();
> +
> +	for (;;)
> +		cpu_relax();
> +}
> +#endif
>  
>  static void __noreturn default_power_off(void)
>  {
> @@ -18,6 +32,8 @@ EXPORT_SYMBOL(pm_power_off);
>  
>  void machine_restart(char *cmd)
>  {
> +	riscv_sse_mask_current_hart();
> +
>  	/*
>  	 * UpdateCapsule() depends on the system being reset via ResetSystem().
>  	 */
> @@ -30,12 +46,14 @@ void machine_restart(char *cmd)
>  
>  void machine_halt(void)
>  {
> +	riscv_sse_mask_current_hart();
>  	do_kernel_power_off();
>  	default_power_off();
>  }
>  
>  void machine_power_off(void)
>  {
> +	riscv_sse_mask_current_hart();
>  	do_kernel_power_off();
>  	default_power_off();
>  }
> diff --git a/arch/riscv/kernel/smp.c b/arch/riscv/kernel/smp.c
> index eddbc3ad06c..9ad38df4827 100644
> --- a/arch/riscv/kernel/smp.c
> +++ b/arch/riscv/kernel/smp.c
> @@ -22,10 +22,13 @@
>  #include <linux/irq.h>
>  #include <linux/irq_work.h>
>  #include <linux/nmi.h>
> +#include <linux/panic.h>
>  
>  #include <asm/tlbflush.h>
>  #include <asm/cacheflush.h>
>  #include <asm/cpu_ops.h>
> +#include <asm/sbi.h>
> +#include <asm/sse.h>
>  
>  enum ipi_message_type {
>  	IPI_RESCHEDULE,
> @@ -78,8 +81,18 @@ int riscv_hartid_to_cpuid(unsigned long hartid)
>  	return -ENOENT;
>  }
>  
> +void __noreturn panic_smp_self_stop(void)
> +{
> +	riscv_sse_mask_current_hart();
> +	local_irq_disable();
> +
> +	for (;;)
> +		cpu_relax();
> +}
> +
>  static void ipi_stop(void)
>  {
> +	riscv_sse_mask_current_hart();
>  	set_cpu_online(smp_processor_id(), false);
>  	while (1)
>  		wait_for_interrupt();
> @@ -90,6 +103,7 @@ static atomic_t waiting_for_crash_ipi = ATOMIC_INIT(0);
>  
>  static inline void ipi_cpu_crash_stop(unsigned int cpu, struct pt_regs *regs)
>  {
> +	riscv_sse_mask_current_hart();
>  	crash_save_cpu(regs, cpu);
>  
>  	atomic_dec(&waiting_for_crash_ipi);
> @@ -253,6 +267,8 @@ void smp_send_stop(void)
>  {
>  	unsigned long timeout;
>  
> +	riscv_sse_mask_current_hart();
> +
>  	if (num_online_cpus() > 1) {
>  		cpumask_t mask;
>  
> @@ -300,6 +316,7 @@ void crash_smp_send_stop(void)
>  		return;
>  
>  	cpus_stopped = 1;
> +	riscv_sse_mask_current_hart();
>  
>  	/*
>  	 * If this cpu is the only one alive at this point in time, online or
> -- 
> 2.50.1 (Apple Git-155)
> 

Re: [PATCH v12 03/11] riscv: sse: mask events during shutdown and kexec

[Zhanpeng Zhang]

Hi Himanshu,

On 10/29/26 7:49 PM, Himanshu Chauhan wrote:
> On Thu, Oct 29, 2026 at 03:48:45PM +0800, Zhanpeng Zhang wrote:
>> SSE delivery is independent of normal S-mode interrupts. Firmware may also
>> retain an event registration until Linux explicitly unregisters it. A hart
>> must therefore stop accepting SSE events before Linux stops servicing the
>> registered handler.
>>
>> Mask SSE on the local hart before panic stop, CPU stop, restart, poweroff,
>> and crash shutdown paths. This prevents firmware from entering Linux-owned
>> handler state after the corresponding CPU or kernel context is no longer
>> valid.
>>
>> A crash kernel cannot identify or take ownership of registrations
>> inherited from the crashed kernel. Reject a later normal kexec while such
>> SSE state may still exist, rather than transferring unknown firmware state
>> to another kernel.
>>
>> The restriction is deliberately based on firmware SSE availability, not
>> CONFIG_RISCV_SBI_SSE or registrations made by the crash kernel. Disabling
>> SSE in the crash kernel does not establish that registrations left by the
>> crashed kernel are absent. Consequently, normal kexec from a crash kernel
>> is also rejected when firmware advertises SSE but no registrations were
>> actually left behind. This conservative restriction remains until a safe
>> way to establish or clear inherited firmware state is available.
>>
>> Link: https://lore.kernel.org/r/as7-3PbWDxygadnJ@hu-himchau-blr.qualcomm.com/
>> Reviewed-by: Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>
>> Signed-off-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
>> ---
>>   arch/riscv/kernel/machine_kexec.c | 11 +++++++++++
>>   arch/riscv/kernel/reset.c         | 18 ++++++++++++++++++
>>   arch/riscv/kernel/smp.c           | 17 +++++++++++++++++
>>   3 files changed, 46 insertions(+)
>>
>> diff --git a/arch/riscv/kernel/machine_kexec.c b/arch/riscv/kernel/machine_kexec.c
>> index 738df176ff6..24e7affae70 100644
>> --- a/arch/riscv/kernel/machine_kexec.c
>> +++ b/arch/riscv/kernel/machine_kexec.c
>> @@ -14,10 +14,13 @@
>>   #include <asm/set_memory.h>	/* For set_memory_x() */
>>   #include <linux/compiler.h>	/* For unreachable() */
>>   #include <linux/cpu.h>		/* For cpu_down() */
>> +#include <linux/crash_dump.h>
>>   #include <linux/reboot.h>
>>   #include <linux/interrupt.h>
>>   #include <linux/irq.h>
>>
>> +#include <asm/sse.h>
>> +
>>   /*
>>    * machine_kexec_prepare - Initialize kexec
>>    *
>> @@ -36,6 +39,13 @@ machine_kexec_prepare(struct kimage *image)
>>   	unsigned int control_code_buffer_sz = 0;
>>   	int i = 0;
>>
>> +	/* A crash kernel cannot tear down registrations inherited from firmware. */
>> +	if (is_kdump_kernel() && image->type != KEXEC_TYPE_CRASH &&
>> +	    riscv_sse_available()) {
>> +		pr_err("Normal kexec from a crash kernel is unsupported with SSE\n");
>> +		return -EOPNOTSUPP;
>
> I think I had pointed this out in v10 also. This rejects a normal kexec from any kdump kernel whenever firmware merely advertises SSE, regardless of CONFIG_RISCV_SBI_SSE or whether anything is registered. Can you check if its breaking kexec() with SSE just being advertised?
>
> Regards
> Himanshu

Thanks for following up. Yes, the current check rejects that case based
on firmware advertising SSE alone. I noted this restriction in the
commit message, but it still needs to be narrowed.

With the merge window closing, I kept that change out of v12 and will
address it in a separate follow-up patch like we discussed.

Thanks,
Zhanpeng

Re: [PATCH v12 03/11] riscv: sse: mask events during shutdown and kexec

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] riscv: sse: unmasked SSE on other harts and preemptible machine_restart()
- [Medium] riscv: sse: unreachable !CONFIG_SMP panic_smp_self_stop() in reset.c
--

commit 62f5a8c35f215daa60d30a9c5320a99ed72eb26e
Author: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>

riscv: sse: mask events during shutdown and kexec

This commit masks SBI Supervisor Software Events (SSE) on the local hart
during panic stop, CPU stop, restart, poweroff, and crash shutdown paths,
and rejects normal kexec from a crash kernel when firmware advertises SSE.

Link: https://lore.kernel.org/r/as7-3PbWDxygadnJ@hu-himchau-blr.qualcomm.com/

> diff --git a/arch/riscv/kernel/reset.c b/arch/riscv/kernel/reset.c
> index 14eb08a6db855..fdab37e7ae52d 100644
> --- a/arch/riscv/kernel/reset.c
> +++ b/arch/riscv/kernel/reset.c
> @@ -6,6 +6,20 @@
>  #include <linux/efi.h>
>  #include <linux/reboot.h>
>  #include <linux/pm.h>
> +#include <linux/smp.h>
> +
> +#include <asm/sse.h>
> +
> +#ifndef CONFIG_SMP
> +void __noreturn panic_smp_self_stop(void)
> +{
> +	riscv_sse_mask_current_hart();
> +	local_irq_disable();
> +
> +	for (;;)
> +		cpu_relax();
> +}
> +#endif

[Severity: Medium]
Can this !CONFIG_SMP implementation of panic_smp_self_stop() in
arch/riscv/kernel/reset.c ever be reached, and does a panic under !CONFIG_SMP
leave SSE unmasked on the panicking hart?

In kernel/panic.c, panic_smp_self_stop() is only called from vpanic() and
nmi_panic() -> nmi_panic_self_stop():

kernel/panic.c:vpanic() {
	if (panic_try_force_cpu(fmt, args)) {
		set_cpu_online(smp_processor_id(), false);
		panic_smp_self_stop();
	}
	...
	if (panic_try_start()) {
		/* go ahead */
	} else if (panic_on_other_cpu())
		panic_smp_self_stop();
}

When !CONFIG_SMP, panic_try_force_cpu() returns false directly. Furthermore,
panic_on_other_cpu() checks:

kernel/panic.c:panic_on_other_cpu() {
	return (panic_in_progress() && !panic_on_this_cpu());
}

Because raw_smp_processor_id() is 0 on !CONFIG_SMP and panic_try_start() only
ever sets panic_cpu to 0, panic_on_this_cpu() is always true whenever a panic
is in progress, making panic_on_other_cpu() always false.

Meanwhile, on CONFIG_SMP kernels, the panicking CPU masks SSE in
panic_other_cpus_shutdown() via smp_send_stop() or crash_smp_send_stop() in
arch/riscv/kernel/smp.c:

kernel/panic.c:panic_other_cpus_shutdown() {
	if (!crash_kexec)
		smp_send_stop();
	else
		crash_smp_send_stop();
}

On !CONFIG_SMP kernels, arch/riscv/kernel/smp.c is not built and both
smp_send_stop() and the weak crash_smp_send_stop() reduce to the no-op
smp_send_stop() in include/linux/smp.h. Without the panic notifier from the
follow-up commit "drivers: firmware: add riscv SSE support" (for example, when
CONFIG_RISCV_SBI_SSE is disabled, in a kdump kernel, or before arch_initcall
runs sse_init()), nothing masks SSE on the panicking hart under !CONFIG_SMP.

> @@ -18,6 +32,8 @@ EXPORT_SYMBOL(pm_power_off);
>  
>  void machine_restart(char *cmd)
>  {
> +	riscv_sse_mask_current_hart();
> +
>  	/*
>  	 * UpdateCapsule() depends on the system being reset via ResetSystem().
>  	 */
> @@ -30,12 +46,14 @@ void machine_restart(char *cmd)
>  
>  void machine_halt(void)
>  {
> +	riscv_sse_mask_current_hart();
>  	do_kernel_power_off();
>  	default_power_off();
>  }
>  
>  void machine_power_off(void)
>  {
> +	riscv_sse_mask_current_hart();
>  	do_kernel_power_off();
>  	default_power_off();
>  }

[Severity: High]
Should machine_restart(), machine_halt(), and machine_power_off() in
arch/riscv/kernel/reset.c also disable local interrupts and call
smp_send_stop() before masking SSE?

In arch/riscv/kernel/reset.c, riscv_sse_mask_current_hart() only masks SSE on
the local hart via SBI_SSE_HART_MASK. Although this patch adds
riscv_sse_mask_current_hart() to ipi_stop() and smp_send_stop() in
arch/riscv/kernel/smp.c, none of machine_restart(), machine_halt(), or
machine_power_off() call local_irq_disable() or smp_send_stop().

While the follow-up commit "drivers: firmware: add riscv SSE support" adds
sse_reboot_notifier() on reboot_notifier_list to tear down and mask SSE via
cpuhp_remove_state(CPUHP_AP_RISCV_SSE_ONLINE) during normal kernel_restart(),
kernel_halt(), and kernel_power_off(), emergency_restart() invokes
machine_emergency_restart() -> machine_restart(NULL) directly without running
reboot_notifier_list or migrate_to_reboot_cpu():

emergency_restart()
  -> machine_emergency_restart()
    -> machine_restart(NULL)

During emergency_restart() (for example from sysrq_handle_reboot(),
hw_failure_emergency_action_func(), watchdog handlers, or driver recovery
workqueues), or during normal restart/halt/poweroff if sse_cpu_teardown()
fails on a CPU, secondary CPUs never receive IPI_CPU_STOP and remain online
with SSE unmasked.

In addition, because machine_restart() does not disable interrupts or
preemption before calling riscv_sse_mask_current_hart(), callers with
preemption enabled can be migrated to another unmasked hart after
riscv_sse_mask_current_hart(), or preempted into idle if do_kernel_restart()
falls through to the infinite loop, where sse_pm_notifier() (CPU_PM_EXIT) can
unmask SSE on the local hart because sse_shutting_down remains false.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261029074853.35467-1-zhangzhanpeng.jasper@bytedance.com?part=3

[PATCH v12 04/11] drivers: firmware: add riscv SSE support

[Zhanpeng Zhang]

From: Clément Léger <cleger@rivosinc.com>

Add a driver-level interface for RISC-V SSE. Linux clients can register
handlers, select a target CPU for global events, and enable, disable or
unregister events. The architecture entry wrapper completes an event after
the registered handler returns. PMU and GHES drivers can use this
interface.

Represent global events with one firmware registration and local events
with one registration per possible CPU. Keep registration and enable state
stable across CPU hotplug, and validate firmware-provided hart IDs before
converting them to Linux CPU IDs. Use phys_addr_t for attribute buffers to
match the physical addresses passed to firmware. Require MMU support
because the current event-stack implementation relies on vmapped memory and
TLB synchronization.

Serialize client list updates with the SSE mutex and the CPU read lock.
Normal CPU hotplug callbacks provide the matching write-side exclusion, and
CPUHP state removal holds the SSE mutex. These rules avoid holding an
additional spinlock across firmware calls. Local event operations select a
per-CPU registration, so require callers to remain on the current CPU and
use lockdep assertions to verify that contract. Because CPU hotplug
callbacks run in a preemptible thread, disable preemption around the
local operations they perform on the offlining or onlining hart.

Propagate firmware failures from register, disable and unregister
operations. Update Linux state only after successful firmware operations,
and release an event only after all registrations are gone. Preserve the
difference between SBI_ERR_NOT_SUPPORTED and a generic SBI failure so
clients only select a fallback after firmware explicitly rejects an SSE
operation.

If setup fails partway through a local event, roll back only the CPU
instances changed by that invocation. Keep non-fallback errors across CPUs.
This lets a client distinguish an event rejected as unsupported by every
failing hart from an unknown firmware failure.

A failed registration rollback can leave firmware state without a client
handle. Retain these events on a driver-owned cleanup list with a no-op
handler. This lets CPU hotplug and shutdown retry cleanup without relying
on client callback lifetime.

Mask SSE before CPU teardown. If a normal CPU-offline teardown fails after
partially changing local events, reconstruct their requested registration
and enable state. Unmask the hart, then return the original error to abort
the offline operation. Shutdown and CPUHP state removal cannot fail, so
record incomplete cleanup and refuse an unsafe normal kexec instead.

Close enable admission before reboot or kexec teardown. Protect the
shutdown check and firmware enable operation with RCU, then drain existing
enable calls before CPUHP removes registrations. This prevents a client
from re-enabling an event between the teardown disable and unregister
operations.

A crash kernel cannot identify registrations inherited from the crashed
kernel. Leave SSE masked, report the retained firmware state separately
from an unavailable extension, and do not let clients select a delivery
path that may still be owned by the old kernel.

Signed-off-by: Clément Léger <cleger@rivosinc.com>
Co-developed-by: Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>
Signed-off-by: Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>
Co-developed-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
Signed-off-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
Acked-by: Conor Dooley <conor.dooley@microchip.com>
---
 drivers/firmware/Kconfig               |    1 +
 drivers/firmware/Makefile              |    1 +
 drivers/firmware/riscv/Kconfig         |   18 +
 drivers/firmware/riscv/Makefile        |    3 +
 drivers/firmware/riscv/riscv_sbi_sse.c | 1219 ++++++++++++++++++++++++
 include/linux/cpuhotplug.h             |    1 +
 include/linux/riscv_sbi_sse.h          |   89 ++
 7 files changed, 1332 insertions(+)
 create mode 100644 drivers/firmware/riscv/Kconfig
 create mode 100644 drivers/firmware/riscv/Makefile
 create mode 100644 drivers/firmware/riscv/riscv_sbi_sse.c
 create mode 100644 include/linux/riscv_sbi_sse.h

diff --git a/drivers/firmware/Kconfig b/drivers/firmware/Kconfig
index b7cc11e4fbf..cf0212576df 100644
--- a/drivers/firmware/Kconfig
+++ b/drivers/firmware/Kconfig
@@ -306,6 +306,7 @@ source "drivers/firmware/meson/Kconfig"
 source "drivers/firmware/microchip/Kconfig"
 source "drivers/firmware/psci/Kconfig"
 source "drivers/firmware/qcom/Kconfig"
+source "drivers/firmware/riscv/Kconfig"
 source "drivers/firmware/samsung/Kconfig"
 source "drivers/firmware/smccc/Kconfig"
 source "drivers/firmware/tegra/Kconfig"
diff --git a/drivers/firmware/Makefile b/drivers/firmware/Makefile
index be46f1e1dc7..879953c316d 100644
--- a/drivers/firmware/Makefile
+++ b/drivers/firmware/Makefile
@@ -35,6 +35,7 @@ obj-y				+= efi/
 obj-y				+= imx/
 obj-y				+= psci/
 obj-y				+= qcom/
+obj-y				+= riscv/
 obj-y				+= samsung/
 obj-y				+= smccc/
 obj-y				+= tegra/
diff --git a/drivers/firmware/riscv/Kconfig b/drivers/firmware/riscv/Kconfig
new file mode 100644
index 00000000000..d15ff84e125
--- /dev/null
+++ b/drivers/firmware/riscv/Kconfig
@@ -0,0 +1,18 @@
+# SPDX-License-Identifier: GPL-2.0-only
+menu "Risc-V Specific firmware drivers"
+depends on RISCV
+
+config RISCV_SBI_SSE
+	bool "Enable SBI Supervisor Software Events support"
+	depends on RISCV_SBI && MMU && !HIBERNATION
+	default y
+	help
+	  The Supervisor Software Events support allows the SBI to deliver
+	  NMI-like notifications to the supervisor mode software. When enabled,
+	  this option provides support to register callbacks on specific SSE
+	  events.
+
+	  Hibernation is not supported because firmware registrations do not
+	  survive restoring a kernel image.
+
+endmenu
diff --git a/drivers/firmware/riscv/Makefile b/drivers/firmware/riscv/Makefile
new file mode 100644
index 00000000000..c8795d4bbb2
--- /dev/null
+++ b/drivers/firmware/riscv/Makefile
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0
+
+obj-$(CONFIG_RISCV_SBI_SSE)		+= riscv_sbi_sse.o
diff --git a/drivers/firmware/riscv/riscv_sbi_sse.c b/drivers/firmware/riscv/riscv_sbi_sse.c
new file mode 100644
index 00000000000..0cb738bab6c
--- /dev/null
+++ b/drivers/firmware/riscv/riscv_sbi_sse.c
@@ -0,0 +1,1219 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2025 Rivos Inc.
+ */
+
+#define pr_fmt(fmt) "sse: " fmt
+
+#include <linux/atomic.h>
+#include <linux/cpu.h>
+#include <linux/cpuhotplug.h>
+#include <linux/cpu_pm.h>
+#include <linux/crash_dump.h>
+#include <linux/hardirq.h>
+#include <linux/kexec.h>
+#include <linux/list.h>
+#include <linux/panic_notifier.h>
+#include <linux/percpu-defs.h>
+#include <linux/rcupdate.h>
+#include <linux/reboot.h>
+#include <linux/riscv_sbi_sse.h>
+#include <linux/slab.h>
+#include <linux/smp.h>
+
+#include <asm/sbi.h>
+#include <asm/sse.h>
+
+struct sse_event {
+	struct list_head list;
+	u32 evt_id;
+	u32 priority;
+	sse_event_handler_fn __rcu *handler;
+	void *handler_arg;
+	/* Only valid for global events */
+	unsigned int cpu;
+	/*
+	 * Desired state requested by the client. Firmware state is tracked per
+	 * instance because a failed transition can leave a partial result.
+	 */
+	bool enable_requested;
+	/* Registration failed, but firmware state still needs driver cleanup. */
+	bool cleanup_pending;
+
+	union {
+		struct sse_registered_event *global;
+		struct sse_registered_event __percpu *local;
+	};
+};
+
+static bool sse_available __ro_after_init;
+static bool sse_fw_state_retained __ro_after_init;
+static bool sse_shutting_down;
+static atomic_t sse_teardown_failed = ATOMIC_INIT(0);
+/*
+ * Client-side updates hold sse_mutex and the CPU read lock. Normal CPU hotplug
+ * callbacks exclude them through the CPUHP write side. Initialization replay
+ * runs before clients can register, and state removal holds sse_mutex.
+ */
+static LIST_HEAD(events);
+static DEFINE_MUTEX(sse_mutex);
+
+/*
+ * A registration rollback can fail before the client receives an event
+ * handle. Keep the retained event independent of client-owned callback text
+ * and data while the driver retries firmware cleanup.
+ */
+static int sse_cleanup_event_handler(u32 evt, void *arg, struct pt_regs *regs)
+{
+	return 0;
+}
+
+struct sse_registered_event {
+	struct sse_event_arch_data arch;
+	struct sse_event *event;
+	unsigned long attr;
+	/*
+	 * Actual firmware state for one global or per-CPU instance. A retry can
+	 * then skip instances that already completed a partial transition.
+	 */
+	bool is_registered;
+	bool is_enabled;
+};
+
+void sse_handle_event(struct sse_event_arch_data *arch_event,
+		      struct pt_regs *regs)
+{
+	sse_event_handler_fn *handler;
+	int ret;
+	struct sse_registered_event *reg_evt =
+		container_of(arch_event, struct sse_registered_event, arch);
+	struct sse_event *evt = reg_evt->event;
+
+	rcu_read_lock();
+	handler = rcu_dereference(evt->handler);
+	ret = handler(evt->evt_id, evt->handler_arg, regs);
+	rcu_read_unlock();
+	if (ret)
+		pr_warn("event %x handler failed with error %d\n", evt->evt_id, ret);
+}
+
+static struct sse_event *sse_event_get(u32 evt)
+{
+	struct sse_event *event;
+
+	lockdep_assert_held(&sse_mutex);
+
+	list_for_each_entry(event, &events, list) {
+		if (event->evt_id == evt)
+			return event;
+	}
+
+	return NULL;
+}
+
+static phys_addr_t sse_event_get_attr_phys(struct sse_registered_event *reg_evt)
+{
+	phys_addr_t phys;
+	void *addr = &reg_evt->attr;
+
+	if (sse_event_is_global(reg_evt->event->evt_id))
+		phys = virt_to_phys(addr);
+	else
+		phys = per_cpu_ptr_to_phys(addr);
+
+	return phys;
+}
+
+static struct sse_registered_event *sse_get_reg_evt(struct sse_event *event)
+{
+	if (sse_event_is_global(event->evt_id))
+		return event->global;
+	else
+		return per_cpu_ptr(event->local, smp_processor_id());
+}
+
+static int sse_err_map_linux_errno(long err)
+{
+	if (err == SBI_ERR_NOT_SUPPORTED)
+		return -EOPNOTSUPP;
+
+	return sbi_err_map_linux_errno(err);
+}
+
+static int sse_sbi_event_func(struct sse_event *event, unsigned long func)
+{
+	struct sbiret ret;
+	u32 evt = event->evt_id;
+	struct sse_registered_event *reg_evt = sse_get_reg_evt(event);
+
+	ret = sbi_ecall(SBI_EXT_SSE, func, evt, 0, 0, 0, 0, 0);
+	if (ret.error) {
+		pr_warn("Failed to execute func %lx, event %x, error %ld\n",
+			func, evt, ret.error);
+		return sse_err_map_linux_errno(ret.error);
+	}
+
+	if (func == SBI_SSE_EVENT_DISABLE)
+		reg_evt->is_enabled = false;
+	else if (func == SBI_SSE_EVENT_ENABLE)
+		reg_evt->is_enabled = true;
+
+	return 0;
+}
+
+int sse_event_disable_local(struct sse_event *event)
+{
+	if (!sse_event_is_global(event->evt_id))
+		lockdep_assert_preemption_disabled();
+
+	if (!sse_get_reg_evt(event)->is_enabled)
+		return 0;
+
+	return sse_sbi_event_func(event, SBI_SSE_EVENT_DISABLE);
+}
+EXPORT_SYMBOL_GPL(sse_event_disable_local);
+
+int sse_event_enable_local(struct sse_event *event)
+{
+	struct sse_registered_event *reg_evt = sse_get_reg_evt(event);
+	int ret;
+
+	if (!sse_event_is_global(event->evt_id))
+		lockdep_assert_preemption_disabled();
+
+	rcu_read_lock();
+	if (READ_ONCE(sse_shutting_down)) {
+		ret = -ESHUTDOWN;
+		goto out;
+	}
+
+	if (!reg_evt->is_registered) {
+		ret = -EINVAL;
+		goto out;
+	}
+
+	if (reg_evt->is_enabled) {
+		ret = 0;
+		goto out;
+	}
+
+	ret = sse_sbi_event_func(event, SBI_SSE_EVENT_ENABLE);
+out:
+	rcu_read_unlock();
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(sse_event_enable_local);
+
+static int sse_event_attr_get_no_lock(struct sse_registered_event *reg_evt,
+				      unsigned long attr_id, unsigned long *val)
+{
+	struct sbiret sret;
+	u32 evt = reg_evt->event->evt_id;
+	phys_addr_t phys;
+
+	phys = sse_event_get_attr_phys(reg_evt);
+
+	sret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_EVENT_ATTR_READ, evt, attr_id, 1,
+			 (unsigned long)phys, 0, 0);
+	if (sret.error) {
+		pr_debug("Failed to get event %x attr %lx, error %ld\n", evt,
+			 attr_id, sret.error);
+		return sse_err_map_linux_errno(sret.error);
+	}
+
+	*val = reg_evt->attr;
+
+	return 0;
+}
+
+static int sse_event_attr_set_nolock(struct sse_registered_event *reg_evt,
+				     unsigned long attr_id, unsigned long val)
+{
+	struct sbiret sret;
+	u32 evt = reg_evt->event->evt_id;
+	phys_addr_t phys;
+
+	reg_evt->attr = val;
+	phys = sse_event_get_attr_phys(reg_evt);
+
+	sret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_EVENT_ATTR_WRITE, evt, attr_id, 1,
+			 (unsigned long)phys, 0, 0);
+	if (sret.error)
+		pr_debug("Failed to set event %x attr %lx, error %ld\n", evt,
+			 attr_id, sret.error);
+
+	return sse_err_map_linux_errno(sret.error);
+}
+
+static void sse_global_event_update_cpu(struct sse_event *event,
+					unsigned int cpu)
+{
+	struct sse_registered_event *reg_evt = event->global;
+
+	event->cpu = cpu;
+	arch_sse_event_update_cpu(&reg_evt->arch, cpu);
+}
+
+static int sse_event_set_target_cpu_nolock(struct sse_event *event,
+					   unsigned int cpu)
+{
+	unsigned long hart_id, old_hart_id;
+	struct sse_registered_event *reg_evt = event->global;
+	u32 evt = event->evt_id;
+	unsigned int old_cpu;
+	bool was_enabled;
+	int ret;
+
+	if (!sse_event_is_global(evt))
+		return -EINVAL;
+
+	if (cpu >= nr_cpu_ids || !cpu_online(cpu))
+		return -EINVAL;
+	hart_id = cpuid_to_hartid_map(cpu);
+	old_cpu = event->cpu;
+	old_hart_id = cpuid_to_hartid_map(old_cpu);
+
+	was_enabled = reg_evt->is_enabled;
+	if (was_enabled) {
+		ret = sse_event_disable_local(event);
+		if (ret)
+			return ret;
+	}
+
+	ret = sse_event_attr_set_nolock(reg_evt, SBI_SSE_ATTR_PREFERRED_HART,
+					hart_id);
+	if (ret == 0)
+		sse_global_event_update_cpu(event, cpu);
+
+	if (was_enabled) {
+		int enable_ret;
+
+		enable_ret = sse_event_enable_local(event);
+		if (enable_ret) {
+			int rollback_ret;
+
+			/*
+			 * The preferred hart was already changed. Restore both the
+			 * firmware attribute and Linux's cached target before reporting
+			 * the failed migration.
+			 */
+			rollback_ret = sse_event_attr_set_nolock(reg_evt,
+								 SBI_SSE_ATTR_PREFERRED_HART,
+								 old_hart_id);
+			if (!rollback_ret) {
+				sse_global_event_update_cpu(event, old_cpu);
+				rollback_ret = sse_event_enable_local(event);
+			}
+			if (!rollback_ret)
+				return enable_ret;
+
+			pr_warn("Failed to restore global event %x to CPU %u: %d\n",
+				evt, old_cpu, rollback_ret);
+			event->enable_requested = false;
+			return enable_ret;
+		}
+	}
+
+	return ret;
+}
+
+int sse_event_set_target_cpu(struct sse_event *event, unsigned int cpu)
+{
+	int ret;
+
+	if (cpu >= nr_cpu_ids)
+		return -EINVAL;
+
+	scoped_guard(mutex, &sse_mutex) {
+		if (READ_ONCE(sse_shutting_down))
+			return -ESHUTDOWN;
+
+		scoped_guard(cpus_read_lock) {
+			if (!cpu_online(cpu))
+				return -EINVAL;
+
+			ret = sse_event_set_target_cpu_nolock(event, cpu);
+		}
+	}
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(sse_event_set_target_cpu);
+
+static int sse_event_init_registered(unsigned int cpu,
+				     struct sse_registered_event *reg_evt,
+				     struct sse_event *event)
+{
+	reg_evt->event = event;
+	reg_evt->is_registered = false;
+	reg_evt->is_enabled = false;
+
+	return arch_sse_init_event(&reg_evt->arch, event->evt_id, cpu);
+}
+
+static void sse_event_free_registered(struct sse_registered_event *reg_evt)
+{
+	arch_sse_free_event(&reg_evt->arch);
+}
+
+static int sse_event_alloc_global(struct sse_event *event)
+{
+	unsigned int cpu;
+	int err;
+	struct sse_registered_event *reg_evt;
+
+	reg_evt = kzalloc_obj(*reg_evt, GFP_KERNEL);
+	if (!reg_evt)
+		return -ENOMEM;
+
+	event->global = reg_evt;
+	cpu = cpumask_first(cpu_possible_mask);
+	if (cpu >= nr_cpu_ids) {
+		kfree(reg_evt);
+		return -ENODEV;
+	}
+
+	err = sse_event_init_registered(cpu, reg_evt, event);
+	if (err)
+		kfree(reg_evt);
+
+	return err;
+}
+
+static int sse_event_alloc_local(struct sse_event *event)
+{
+	int err;
+	unsigned int cpu, err_cpu;
+	struct sse_registered_event *reg_evt;
+	struct sse_registered_event __percpu *reg_evts;
+
+	reg_evts = alloc_percpu(struct sse_registered_event);
+	if (!reg_evts)
+		return -ENOMEM;
+
+	event->local = reg_evts;
+
+	for_each_possible_cpu(cpu) {
+		reg_evt = per_cpu_ptr(reg_evts, cpu);
+		err = sse_event_init_registered(cpu, reg_evt, event);
+		if (err) {
+			err_cpu = cpu;
+			goto err_free_per_cpu;
+		}
+	}
+
+	return 0;
+
+err_free_per_cpu:
+	for_each_possible_cpu(cpu) {
+		if (cpu == err_cpu)
+			break;
+		reg_evt = per_cpu_ptr(reg_evts, cpu);
+		sse_event_free_registered(reg_evt);
+	}
+
+	free_percpu(reg_evts);
+
+	return err;
+}
+
+static struct sse_event *sse_event_alloc(u32 evt, u32 priority,
+					 sse_event_handler_fn *handler,
+					 void *arg)
+{
+	int err;
+	struct sse_event *event;
+
+	event = kzalloc_obj(*event, GFP_KERNEL);
+	if (!event)
+		return ERR_PTR(-ENOMEM);
+
+	event->evt_id = evt;
+	event->priority = priority;
+	event->handler_arg = arg;
+	RCU_INIT_POINTER(event->handler, handler);
+
+	if (sse_event_is_global(evt))
+		err = sse_event_alloc_global(event);
+	else
+		err = sse_event_alloc_local(event);
+
+	if (err) {
+		kfree(event);
+		return ERR_PTR(err);
+	}
+
+	return event;
+}
+
+static int sse_sbi_register_event(struct sse_event *event,
+				  struct sse_registered_event *reg_evt)
+{
+	int ret;
+
+	if (reg_evt->is_registered)
+		return 0;
+
+	ret = sse_event_attr_set_nolock(reg_evt, SBI_SSE_ATTR_PRIO,
+					event->priority);
+	if (ret)
+		return ret;
+
+	ret = arch_sse_register_event(&reg_evt->arch);
+	if (!ret)
+		reg_evt->is_registered = true;
+
+	return ret;
+}
+
+static int sse_event_register_local(struct sse_event *event)
+{
+	int ret;
+	struct sse_registered_event *reg_evt;
+
+	reg_evt = per_cpu_ptr(event->local, smp_processor_id());
+	ret = sse_sbi_register_event(event, reg_evt);
+	if (ret)
+		pr_debug("Failed to register event %x: err %d\n", event->evt_id,
+			 ret);
+
+	return ret;
+}
+
+static int sse_sbi_unregister_event(struct sse_event *event)
+{
+	struct sse_registered_event *reg_evt = sse_get_reg_evt(event);
+	int ret;
+
+	if (!reg_evt->is_registered)
+		return 0;
+
+	ret = sse_sbi_event_func(event, SBI_SSE_EVENT_UNREGISTER);
+	if (!ret) {
+		reg_evt->is_registered = false;
+		reg_evt->is_enabled = false;
+	}
+
+	return ret;
+}
+
+struct sse_per_cpu_evt {
+	struct sse_event *event;
+	unsigned long func;
+	atomic_t first_error;
+	atomic_t nonfallback_error;
+	cpumask_t changed;
+};
+
+static void sse_event_per_cpu_func(void *info)
+{
+	struct sse_per_cpu_evt *cpu_evt = info;
+	struct sse_registered_event *reg_evt;
+	bool changed;
+	int ret;
+
+	reg_evt = sse_get_reg_evt(cpu_evt->event);
+
+	if (cpu_evt->func == SBI_SSE_EVENT_REGISTER) {
+		changed = !reg_evt->is_registered;
+		ret = sse_event_register_local(cpu_evt->event);
+	} else if (cpu_evt->func == SBI_SSE_EVENT_UNREGISTER) {
+		changed = reg_evt->is_registered;
+		ret = sse_sbi_unregister_event(cpu_evt->event);
+	} else if (cpu_evt->func == SBI_SSE_EVENT_ENABLE) {
+		changed = !reg_evt->is_enabled;
+		ret = sse_event_enable_local(cpu_evt->event);
+	} else if (cpu_evt->func == SBI_SSE_EVENT_DISABLE) {
+		changed = reg_evt->is_enabled;
+		ret = sse_event_disable_local(cpu_evt->event);
+	} else {
+		changed = false;
+		ret = -EINVAL;
+	}
+
+	if (ret) {
+		atomic_cmpxchg(&cpu_evt->first_error, 0, ret);
+		if (ret != -EOPNOTSUPP)
+			atomic_cmpxchg(&cpu_evt->nonfallback_error, 0, ret);
+	} else if (changed) {
+		cpumask_set_cpu(smp_processor_id(), &cpu_evt->changed);
+	}
+}
+
+static bool sse_event_is_registered(struct sse_event *event)
+{
+	unsigned int cpu;
+
+	if (sse_event_is_global(event->evt_id))
+		return event->global->is_registered;
+
+	for_each_possible_cpu(cpu) {
+		if (per_cpu_ptr(event->local, cpu)->is_registered)
+			return true;
+	}
+
+	return false;
+}
+
+static bool sse_event_is_enabled(struct sse_event *event)
+{
+	unsigned int cpu;
+
+	if (sse_event_is_global(event->evt_id))
+		return event->global->is_enabled;
+
+	for_each_possible_cpu(cpu) {
+		if (per_cpu_ptr(event->local, cpu)->is_enabled)
+			return true;
+	}
+
+	return false;
+}
+
+static void sse_event_free(struct sse_event *event)
+{
+	unsigned int cpu;
+	struct sse_registered_event *reg_evt;
+
+	if (WARN_ON_ONCE(sse_event_is_registered(event)))
+		return;
+
+	if (sse_event_is_global(event->evt_id)) {
+		sse_event_free_registered(event->global);
+		kfree(event->global);
+	} else {
+		for_each_possible_cpu(cpu) {
+			reg_evt = per_cpu_ptr(event->local, cpu);
+			sse_event_free_registered(reg_evt);
+		}
+		free_percpu(event->local);
+	}
+
+	kfree(event);
+}
+
+static struct sse_event *sse_register_failed(struct sse_event *event, int ret)
+{
+	/*
+	 * Keep failed rollback state visible to CPU hotplug and shutdown. The
+	 * core-owned handler also makes the retained registration independent of
+	 * the client whose registration request failed.
+	 */
+	if (sse_event_is_registered(event)) {
+		event->cleanup_pending = true;
+		rcu_assign_pointer(event->handler, sse_cleanup_event_handler);
+		synchronize_rcu();
+		list_add(&event->list, &events);
+		pr_err("Event %x remains registered after rollback; cleanup retained\n",
+		       event->evt_id);
+		ret = -EUCLEAN;
+	} else {
+		sse_event_free(event);
+	}
+
+	return ERR_PTR(ret);
+}
+
+void sse_event_cleanup(struct sse_event *event)
+{
+	guard(mutex)(&sse_mutex);
+	guard(cpus_read_lock)();
+
+	if (event->cleanup_pending)
+		return;
+
+	/*
+	 * Firmware may still enter the old callback after disable or unregister
+	 * fails. Publish a core-owned callback, then wait before the client frees
+	 * its callback data.
+	 */
+	event->cleanup_pending = true;
+	rcu_assign_pointer(event->handler, sse_cleanup_event_handler);
+	synchronize_rcu();
+}
+EXPORT_SYMBOL_GPL(sse_event_cleanup);
+
+static void sse_release_cleanup_event(struct sse_event *event)
+{
+	if (!event->cleanup_pending || sse_event_is_registered(event))
+		return;
+
+	list_del(&event->list);
+	sse_event_free(event);
+}
+
+static int sse_event_setup_all_cpus(struct sse_event *event,
+				    unsigned long func,
+				    unsigned long rollback_func)
+{
+	struct sse_per_cpu_evt cpu_evt;
+	int rollback_ret;
+	int ret;
+
+	cpu_evt.event = event;
+	atomic_set(&cpu_evt.first_error, 0);
+	atomic_set(&cpu_evt.nonfallback_error, 0);
+	cpumask_clear(&cpu_evt.changed);
+	cpu_evt.func = func;
+	on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1);
+	/* IRQ fallback is safe only if every failing CPU reports unsupported. */
+	ret = atomic_read(&cpu_evt.nonfallback_error);
+	if (!ret)
+		ret = atomic_read(&cpu_evt.first_error);
+	/*
+	 * A previous attempt may already have changed some CPUs. Roll back only
+	 * instances changed by this invocation.
+	 */
+	if (ret) {
+		cpu_evt.func = rollback_func;
+		atomic_set(&cpu_evt.first_error, 0);
+		atomic_set(&cpu_evt.nonfallback_error, 0);
+		on_each_cpu_mask(&cpu_evt.changed, sse_event_per_cpu_func, &cpu_evt, 1);
+
+		rollback_ret = atomic_read(&cpu_evt.nonfallback_error);
+		if (!rollback_ret)
+			rollback_ret = atomic_read(&cpu_evt.first_error);
+
+		/* A rollback failure leaves the firmware state uncertain. */
+		return rollback_ret ?: ret;
+	}
+
+	return 0;
+}
+
+static int sse_event_teardown_all_cpus(struct sse_event *event,
+				       unsigned long func)
+{
+	struct sse_per_cpu_evt cpu_evt;
+
+	cpu_evt.event = event;
+	atomic_set(&cpu_evt.first_error, 0);
+	atomic_set(&cpu_evt.nonfallback_error, 0);
+	cpumask_clear(&cpu_evt.changed);
+	cpu_evt.func = func;
+	on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1);
+
+	return atomic_read(&cpu_evt.first_error);
+}
+
+int sse_event_enable(struct sse_event *event)
+{
+	int ret = 0;
+
+	scoped_guard(mutex, &sse_mutex) {
+		if (READ_ONCE(sse_shutting_down))
+			return -ESHUTDOWN;
+
+		scoped_guard(cpus_read_lock) {
+			if (sse_event_is_global(event->evt_id)) {
+				ret = sse_event_enable_local(event);
+			} else {
+				ret = sse_event_setup_all_cpus(event,
+							       SBI_SSE_EVENT_ENABLE,
+							       SBI_SSE_EVENT_DISABLE);
+			}
+			event->enable_requested = !ret;
+		}
+	}
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(sse_event_enable);
+
+static int sse_events_mask(void)
+{
+	struct sbiret ret;
+
+	ret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_HART_MASK, 0, 0, 0, 0, 0, 0);
+	if (ret.error == SBI_ERR_ALREADY_STOPPED)
+		return 0;
+
+	return sse_err_map_linux_errno(ret.error);
+}
+
+static int sse_events_unmask(void)
+{
+	struct sbiret ret;
+
+	ret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_HART_UNMASK, 0, 0, 0, 0, 0, 0);
+	if (ret.error == SBI_ERR_ALREADY_STARTED)
+		return 0;
+
+	return sse_err_map_linux_errno(ret.error);
+}
+
+static int sse_event_disable_nolock(struct sse_event *event)
+{
+	if (sse_event_is_global(event->evt_id))
+		return sse_event_disable_local(event);
+
+	return sse_event_teardown_all_cpus(event, SBI_SSE_EVENT_DISABLE);
+}
+
+int sse_event_disable(struct sse_event *event)
+{
+	int ret = 0;
+
+	scoped_guard(mutex, &sse_mutex) {
+		if (READ_ONCE(sse_shutting_down))
+			return -ESHUTDOWN;
+
+		scoped_guard(cpus_read_lock) {
+			if (!event->enable_requested && !sse_event_is_enabled(event))
+				return 0;
+
+			event->enable_requested = false;
+			ret = sse_event_disable_nolock(event);
+			if (!ret && sse_event_is_enabled(event))
+				ret = -EIO;
+		}
+	}
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(sse_event_disable);
+
+struct sse_event *sse_event_register(u32 evt, u32 priority,
+				     sse_event_handler_fn *handler, void *arg)
+{
+	struct sse_event *event;
+	int cpu;
+	int ret = 0;
+
+	if (sse_fw_state_retained)
+		return ERR_PTR(-EUCLEAN);
+	if (!sse_available)
+		return ERR_PTR(-EOPNOTSUPP);
+
+	guard(mutex)(&sse_mutex);
+	if (READ_ONCE(sse_shutting_down))
+		return ERR_PTR(-ESHUTDOWN);
+
+	guard(cpus_read_lock)();
+
+	if (sse_event_get(evt))
+		return ERR_PTR(-EEXIST);
+
+	event = sse_event_alloc(evt, priority, handler, arg);
+	if (IS_ERR(event))
+		return event;
+
+	if (sse_event_is_global(evt)) {
+		unsigned long preferred_hart;
+
+		ret = sse_event_attr_get_no_lock(event->global,
+						 SBI_SSE_ATTR_PREFERRED_HART,
+						 &preferred_hart);
+		if (ret)
+			return sse_register_failed(event, ret);
+
+		cpu = riscv_hartid_to_cpuid(preferred_hart);
+		if (cpu < 0 || !cpu_online(cpu)) {
+			cpu = cpumask_first(cpu_online_mask);
+			if (cpu >= nr_cpu_ids)
+				return sse_register_failed(event, -ENODEV);
+
+			ret = sse_event_set_target_cpu_nolock(event, cpu);
+			if (ret)
+				return sse_register_failed(event, ret);
+		} else {
+			sse_global_event_update_cpu(event, cpu);
+		}
+
+		ret = sse_sbi_register_event(event, event->global);
+		if (ret)
+			return sse_register_failed(event, ret);
+	} else {
+		ret = sse_event_setup_all_cpus(event, SBI_SSE_EVENT_REGISTER,
+					       SBI_SSE_EVENT_UNREGISTER);
+		if (ret)
+			return sse_register_failed(event, ret);
+	}
+
+	list_add(&event->list, &events);
+
+	return event;
+}
+EXPORT_SYMBOL_GPL(sse_event_register);
+
+static int sse_event_unregister_nolock(struct sse_event *event)
+{
+	if (sse_event_is_global(event->evt_id))
+		return sse_sbi_unregister_event(event);
+
+	return sse_event_teardown_all_cpus(event, SBI_SSE_EVENT_UNREGISTER);
+}
+
+int sse_event_unregister(struct sse_event *event)
+{
+	int ret = 0;
+
+	scoped_guard(mutex, &sse_mutex) {
+		if (READ_ONCE(sse_shutting_down))
+			return -ESHUTDOWN;
+
+		scoped_guard(cpus_read_lock) {
+			ret = sse_event_unregister_nolock(event);
+			if (ret)
+				return ret;
+			if (sse_event_is_registered(event))
+				return -EBUSY;
+
+			list_del(&event->list);
+
+			sse_event_free(event);
+		}
+	}
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(sse_event_unregister);
+
+static int sse_teardown_event(struct sse_event *event, unsigned int cpu);
+
+static int sse_cpu_online(unsigned int cpu)
+{
+	int ret, rollback_ret;
+	struct sse_event *event, *tmp;
+	struct sse_registered_event *reg_evt;
+
+	arch_sse_init_cpu();
+
+	list_for_each_entry_safe(event, tmp, &events, list) {
+		if (sse_event_is_global(event->evt_id))
+			continue;
+		if (event->cleanup_pending) {
+			scoped_guard(preempt)
+				ret = sse_teardown_event(event, cpu);
+			if (ret)
+				goto rollback;
+			sse_release_cleanup_event(event);
+			continue;
+		}
+
+		/*
+		 * The local SBI helpers act on the current hart and assert that
+		 * preemption is disabled. Unlike the client-facing paths, which
+		 * reach them through an IPI, the CPUHP thread runs preemptible.
+		 */
+		scoped_guard(preempt) {
+			ret = sse_event_register_local(event);
+			if (!ret)
+				ret = event->enable_requested ?
+					sse_event_enable_local(event) :
+					sse_event_disable_local(event);
+		}
+		if (ret)
+			goto rollback;
+	}
+
+	/* Only unmask after every cached per-CPU state is reconstructed. */
+	ret = sse_events_unmask();
+	if (!ret)
+		return 0;
+
+rollback:
+	/* A failed startup callback is not followed by this state's teardown. */
+	list_for_each_entry_safe(event, tmp, &events, list) {
+		if (sse_event_is_global(event->evt_id))
+			continue;
+
+		scoped_guard(preempt) {
+			reg_evt = sse_get_reg_evt(event);
+			rollback_ret = reg_evt->is_enabled ?
+				sse_event_disable_local(event) : 0;
+			if (!rollback_ret)
+				rollback_ret = reg_evt->is_registered ?
+					sse_sbi_unregister_event(event) : 0;
+		}
+		if (rollback_ret) {
+			pr_warn("Failed to roll back event %x on CPU %u: %d\n",
+				event->evt_id, cpu, rollback_ret);
+			atomic_set(&sse_teardown_failed, 1);
+			continue;
+		}
+		sse_release_cleanup_event(event);
+	}
+
+	return ret;
+}
+
+static int sse_teardown_event(struct sse_event *event, unsigned int cpu)
+{
+	struct sse_registered_event *reg_evt = sse_get_reg_evt(event);
+	int ret;
+
+	if (reg_evt->is_enabled) {
+		ret = sse_event_disable_local(event);
+		if (ret) {
+			pr_warn("Failed to disable event %x on CPU %u: %d\n",
+				event->evt_id, cpu, ret);
+			return ret;
+		}
+	}
+
+	ret = sse_sbi_unregister_event(event);
+	if (ret)
+		pr_warn("Failed to unregister event %x on CPU %u: %d\n",
+			event->evt_id, cpu, ret);
+
+	return ret;
+}
+
+static int sse_restore_local_events(unsigned int cpu)
+{
+	struct sse_event *event;
+	int first_error = 0;
+	int ret;
+
+	list_for_each_entry(event, &events, list) {
+		if (sse_event_is_global(event->evt_id) || event->cleanup_pending)
+			continue;
+
+		scoped_guard(preempt) {
+			ret = sse_event_register_local(event);
+			if (!ret)
+				ret = event->enable_requested ?
+					sse_event_enable_local(event) :
+					sse_event_disable_local(event);
+		}
+		if (ret) {
+			pr_warn("Failed to restore event %x on CPU %u: %d\n",
+				event->evt_id, cpu, ret);
+			if (!first_error)
+				first_error = ret;
+		}
+	}
+
+	return first_error;
+}
+
+static int sse_cpu_teardown(unsigned int cpu)
+{
+	/* Only a regular CPU-offline callback may abort the CPUHP operation. */
+	bool regular_offline = READ_ONCE(sse_available) &&
+			       !READ_ONCE(sse_shutting_down);
+	unsigned int next_cpu;
+	struct sse_event *event, *tmp;
+	int first_error = 0;
+	int ret;
+
+	/* Do not dismantle CPU state while firmware can still deliver SSE. */
+	ret = sse_events_mask();
+	if (ret) {
+		pr_warn("Failed to mask SSE on CPU %u during teardown: %d\n",
+			cpu, ret);
+		if (READ_ONCE(sse_shutting_down))
+			atomic_set(&sse_teardown_failed, 1);
+		/* CPUHP installation rollback and state removal cannot fail. */
+		return regular_offline ? ret : 0;
+	}
+
+	list_for_each_entry_safe(event, tmp, &events, list) {
+		if (sse_event_is_global(event->evt_id))
+			continue;
+
+		scoped_guard(preempt)
+			ret = sse_teardown_event(event, cpu);
+		if (ret && !first_error)
+			first_error = ret;
+		sse_release_cleanup_event(event);
+		if (ret && regular_offline)
+			goto restore_cpu;
+	}
+
+	list_for_each_entry_safe(event, tmp, &events, list) {
+		if (!sse_event_is_global(event->evt_id) || event->cpu != cpu)
+			continue;
+
+		/*
+		 * cpuhp_remove_state() invokes teardown while every CPU remains
+		 * online. Do not migrate to a CPU whose callback may have run.
+		 */
+		if (READ_ONCE(sse_shutting_down) || event->cleanup_pending) {
+			ret = sse_teardown_event(event, cpu);
+		} else {
+			next_cpu = cpumask_any_but(cpu_online_mask, cpu);
+			if (next_cpu >= nr_cpu_ids) {
+				ret = sse_teardown_event(event, cpu);
+			} else {
+				ret = sse_event_set_target_cpu_nolock(event, next_cpu);
+				if (ret)
+					pr_warn("Failed to migrate global event %x from CPU %u: %d\n",
+						event->evt_id, cpu, ret);
+			}
+		}
+
+		if (ret && !first_error)
+			first_error = ret;
+		sse_release_cleanup_event(event);
+		if (ret && regular_offline)
+			goto restore_cpu;
+	}
+
+	if (first_error) {
+		/* An offline CPU is not revisited when this CPUHP state is removed. */
+		atomic_set(&sse_teardown_failed, 1);
+	}
+
+	return 0;
+
+restore_cpu:
+	/*
+	 * CPUHP leaves this CPU online when teardown returns an error. Restore
+	 * every client-owned local event before making SSE delivery visible again.
+	 */
+	ret = sse_restore_local_events(cpu);
+	if (!ret)
+		ret = sse_events_unmask();
+	if (ret) {
+		atomic_set(&sse_teardown_failed, 1);
+		pr_warn("Failed to restore SSE after aborting CPU %u offline: %d\n",
+			cpu, ret);
+	} else {
+		pr_warn("Aborted CPU %u offline after SSE teardown failed: %d\n",
+			cpu, first_error);
+	}
+
+	return first_error;
+}
+
+static int sse_pm_notifier(struct notifier_block *nb, unsigned long action,
+			   void *data)
+{
+	int ret;
+
+	WARN_ON_ONCE(preemptible());
+
+	switch (action) {
+	case CPU_PM_ENTER:
+		ret = sse_events_mask();
+		break;
+	case CPU_PM_EXIT:
+	case CPU_PM_ENTER_FAILED:
+		if (READ_ONCE(sse_shutting_down))
+			return NOTIFY_OK;
+		ret = sse_events_unmask();
+		break;
+	default:
+		return NOTIFY_DONE;
+	}
+
+	if (ret)
+		return notifier_from_errno(ret);
+
+	return NOTIFY_OK;
+}
+
+static struct notifier_block sse_pm_nb = {
+	.notifier_call = sse_pm_notifier,
+};
+
+static int sse_panic_notifier(struct notifier_block *nb, unsigned long action,
+			      void *data)
+{
+	riscv_sse_mask_current_hart();
+
+	return NOTIFY_OK;
+}
+
+static struct notifier_block sse_panic_nb = {
+	.notifier_call = sse_panic_notifier,
+	.priority = INT_MAX,
+};
+
+/*
+ * Mask all CPUs and unregister all events on reboot or kexec.
+ */
+static int sse_reboot_notifier(struct notifier_block *nb, unsigned long action,
+			       void *data)
+{
+	int ret;
+
+	scoped_guard(mutex, &sse_mutex) {
+		if (!sse_shutting_down) {
+			WRITE_ONCE(sse_shutting_down, true);
+			ret = cpu_pm_unregister_notifier(&sse_pm_nb);
+			if (ret) {
+				pr_warn("Failed to unregister CPU PM notifier: %d\n",
+					ret);
+				atomic_set(&sse_teardown_failed, 1);
+			}
+			/* Drain CPU PM callbacks and client enables before teardown. */
+			synchronize_rcu();
+			cpuhp_remove_state(CPUHP_AP_RISCV_SSE_ONLINE);
+		}
+	}
+
+	/* Normal kexec preserves firmware state but discards old kernel memory. */
+	if (kexec_in_progress && atomic_read(&sse_teardown_failed))
+		panic("SSE teardown failed; refusing unsafe kexec");
+
+	return NOTIFY_OK;
+}
+
+static struct notifier_block sse_reboot_nb = {
+	.notifier_call = sse_reboot_notifier,
+};
+
+static int __init sse_init(void)
+{
+	int ret;
+
+	/*
+	 * A kdump kernel cannot identify registrations left by the crashed
+	 * kernel. Keep them masked by not initializing SSE again.
+	 */
+	if (is_kdump_kernel() && riscv_sse_available()) {
+		sse_fw_state_retained = true;
+		pr_info("SSE remains disabled in the crash kernel\n");
+		return -EOPNOTSUPP;
+	}
+
+	if (sbi_probe_extension(SBI_EXT_SSE) <= 0) {
+		pr_info("Missing SBI SSE extension\n");
+		return -EOPNOTSUPP;
+	}
+	pr_info("SBI SSE extension detected\n");
+
+	ret = cpu_pm_register_notifier(&sse_pm_nb);
+	if (ret) {
+		pr_warn("Failed to register CPU PM notifier...\n");
+		return ret;
+	}
+
+	ret = register_reboot_notifier(&sse_reboot_nb);
+	if (ret) {
+		pr_warn("Failed to register reboot notifier...\n");
+		goto remove_cpupm;
+	}
+
+	ret = atomic_notifier_chain_register(&panic_notifier_list, &sse_panic_nb);
+	if (ret) {
+		pr_warn("Failed to register panic notifier...\n");
+		goto remove_reboot;
+	}
+
+	/* Tear down perf events before dismantling their SSE delivery path. */
+	ret = cpuhp_setup_state(CPUHP_AP_RISCV_SSE_ONLINE, "riscv/sse:online",
+				sse_cpu_online, sse_cpu_teardown);
+	if (ret < 0)
+		goto remove_panic;
+
+	sse_available = true;
+
+	return 0;
+
+remove_panic:
+	atomic_notifier_chain_unregister(&panic_notifier_list, &sse_panic_nb);
+
+remove_reboot:
+	unregister_reboot_notifier(&sse_reboot_nb);
+
+remove_cpupm:
+	cpu_pm_unregister_notifier(&sse_pm_nb);
+
+	return ret;
+}
+arch_initcall(sse_init);
diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
index feb32949aee..d13af475508 100644
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -200,6 +200,7 @@ enum cpuhp_state {
 	CPUHP_AP_ARM_MVEBU_SYNC_CLOCKS,
 	CPUHP_AP_ARM_CORESIGHT_ONLINE,
 	CPUHP_AP_X86_INTEL_EPB_ONLINE,
+	CPUHP_AP_RISCV_SSE_ONLINE,
 	CPUHP_AP_PERF_ONLINE,
 	CPUHP_AP_PERF_X86_ONLINE,
 	CPUHP_AP_PERF_X86_UNCORE_ONLINE,
diff --git a/include/linux/riscv_sbi_sse.h b/include/linux/riscv_sbi_sse.h
new file mode 100644
index 00000000000..774a782e556
--- /dev/null
+++ b/include/linux/riscv_sbi_sse.h
@@ -0,0 +1,89 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright (C) 2025 Rivos Inc.
+ */
+
+#ifndef __LINUX_RISCV_SBI_SSE_H
+#define __LINUX_RISCV_SBI_SSE_H
+
+#include <linux/err.h>
+#include <linux/errno.h>
+#include <linux/types.h>
+#include <linux/linkage.h>
+
+struct sse_event;
+struct pt_regs;
+
+typedef int (sse_event_handler_fn)(u32 event_num, void *arg,
+				   struct pt_regs *regs);
+
+#ifdef CONFIG_RISCV_SBI_SSE
+
+/*
+ * The callback and its argument must remain valid until unregister succeeds.
+ * The callback runs in NMI context and must not sleep.
+ * regs is NULL if firmware cannot provide a complete interrupted context.
+ */
+struct sse_event *sse_event_register(u32 event_num, u32 priority,
+				     sse_event_handler_fn *handler, void *arg);
+
+int sse_event_unregister(struct sse_event *evt);
+
+/*
+ * Transfer a retained event to the SSE core for deferred cleanup. The caller
+ * must not access the event or its callback data after this function returns.
+ */
+void sse_event_cleanup(struct sse_event *evt);
+
+int sse_event_set_target_cpu(struct sse_event *sse_evt, unsigned int cpu);
+
+int sse_event_enable(struct sse_event *sse_evt);
+
+int sse_event_disable(struct sse_event *sse_evt);
+
+/* Local events require the caller to remain on the current CPU. */
+int sse_event_enable_local(struct sse_event *sse_evt);
+int sse_event_disable_local(struct sse_event *sse_evt);
+
+#else
+static inline struct sse_event *sse_event_register(u32 event_num, u32 priority,
+						   sse_event_handler_fn *handler,
+						   void *arg)
+{
+	return ERR_PTR(-EOPNOTSUPP);
+}
+
+static inline int sse_event_unregister(struct sse_event *evt)
+{
+	return -EOPNOTSUPP;
+}
+
+static inline void sse_event_cleanup(struct sse_event *evt) { }
+
+static inline int sse_event_set_target_cpu(struct sse_event *sse_evt,
+					   unsigned int cpu)
+{
+	return -EOPNOTSUPP;
+}
+
+static inline int sse_event_enable(struct sse_event *sse_evt)
+{
+	return -EOPNOTSUPP;
+}
+
+static inline int sse_event_disable(struct sse_event *sse_evt)
+{
+	return -EOPNOTSUPP;
+}
+
+static inline int sse_event_enable_local(struct sse_event *sse_evt)
+{
+	return -EOPNOTSUPP;
+}
+
+static inline int sse_event_disable_local(struct sse_event *sse_evt)
+{
+	return -EOPNOTSUPP;
+}
+#endif
+#endif /* __LINUX_RISCV_SBI_SSE_H */
-- 
2.50.1 (Apple Git-155)

Re: [PATCH v12 04/11] drivers: firmware: add riscv SSE support

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 6 potential issue(s) to consider:
- [High] firmware: riscv_sbi_sse: sse_pm_notifier() unmasks SSE on unready CPUs
- [High] firmware: riscv_sbi_sse: sse_event_cleanup() leaks retained SSE events
- [High] firmware: riscv_sbi_sse: sse_cpu_teardown() mishandles CPU-up rollback
- [High] firmware: riscv_sbi_sse: missing synchronization in sse_event_*_local()
- [Medium] firmware: riscv_sbi_sse: invalid hart/CPU check in sse_event_register()
- [Medium] firmware: riscv_sbi_sse: bad rollback in sse_event_set_target_cpu_nolock()
--

commit a43ce7f3ee4ccdf7ca9265079df6d0b217323b41
Author: Clément Léger <cleger@rivosinc.com>

drivers: firmware: add riscv SSE support

This commit adds a driver-level interface for RISC-V Supervisor Software
Events (SSE), allowing kernel clients to register handlers, manage target
CPUs for global events, and enable, disable, or unregister local and global
events. It also coordinates per-CPU SSE state across CPU hotplug, idle power
management, panic, and reboot/kexec transitions.

> diff --git a/drivers/firmware/riscv/riscv_sbi_sse.c b/drivers/firmware/riscv/riscv_sbi_sse.c
> new file mode 100644
> index 0000000000000..0cb738bab6c39
> --- /dev/null
> +++ b/drivers/firmware/riscv/riscv_sbi_sse.c
[ ... ]
> +static int sse_sbi_event_func(struct sse_event *event, unsigned long func)
> +{
> +	struct sbiret ret;
> +	u32 evt = event->evt_id;
> +	struct sse_registered_event *reg_evt = sse_get_reg_evt(event);
> +
> +	ret = sbi_ecall(SBI_EXT_SSE, func, evt, 0, 0, 0, 0, 0);
> +	if (ret.error) {
> +		pr_warn("Failed to execute func %lx, event %x, error %ld\n",
> +			func, evt, ret.error);
> +		return sse_err_map_linux_errno(ret.error);
> +	}
> +
> +	if (func == SBI_SSE_EVENT_DISABLE)
> +		reg_evt->is_enabled = false;
> +	else if (func == SBI_SSE_EVENT_ENABLE)
> +		reg_evt->is_enabled = true;
> +
> +	return 0;
> +}
> +
> +int sse_event_disable_local(struct sse_event *event)
> +{
> +	if (!sse_event_is_global(event->evt_id))
> +		lockdep_assert_preemption_disabled();
> +
> +	if (!sse_get_reg_evt(event)->is_enabled)
> +		return 0;
> +
> +	return sse_sbi_event_func(event, SBI_SSE_EVENT_DISABLE);
> +}
> +EXPORT_SYMBOL_GPL(sse_event_disable_local);
> +
> +int sse_event_enable_local(struct sse_event *event)
> +{
> +	struct sse_registered_event *reg_evt = sse_get_reg_evt(event);
> +	int ret;
> +
> +	if (!sse_event_is_global(event->evt_id))
> +		lockdep_assert_preemption_disabled();
> +
> +	rcu_read_lock();
> +	if (READ_ONCE(sse_shutting_down)) {
> +		ret = -ESHUTDOWN;
> +		goto out;
> +	}
> +
> +	if (!reg_evt->is_registered) {
> +		ret = -EINVAL;
> +		goto out;
> +	}
> +
> +	if (reg_evt->is_enabled) {
> +		ret = 0;
> +		goto out;
> +	}
> +
> +	ret = sse_sbi_event_func(event, SBI_SSE_EVENT_ENABLE);
> +out:
> +	rcu_read_unlock();
> +
> +	return ret;
> +}
> +EXPORT_SYMBOL_GPL(sse_event_enable_local);

[Severity: High]
Can sse_event_enable_local() and sse_event_disable_local() race with hardirqs,
CPU hotplug teardown, or reboot teardown and trigger duplicate or invalid-state
SBI calls?

There appear to be several related synchronization gaps here:

1. In sse_sbi_event_func(), reg_evt->is_enabled is updated only after
sbi_ecall() returns, and neither sse_event_disable_local() nor
sse_event_enable_local() masks interrupts. If a hardirq (such as
perf_event_task_tick(), perf_mux_hrtimer_handler(), or a perf IPI calling
perf_pmu_disable() -> pmu_sbi_sse_disable() -> sse_event_disable_local())
fires after sbi_ecall(SBI_EXT_SSE, SBI_SSE_EVENT_DISABLE) completes in
firmware but before reg_evt->is_enabled = false is written, the IRQ handler
observes stale reg_evt->is_enabled == true and issues a second
SBI_SSE_EVENT_DISABLE call that fails with SBI_ERR_INVALID_STATE (-EINVAL).
If the IRQ handler then calls sse_event_enable_local(), it also sees stale
reg_evt->is_enabled == true and returns 0 without re-enabling the event in
firmware.

2. During reboot or kexec, sse_reboot_notifier() runs at default priority 0
(before perf_reboot_notifier at INT_MIN) and calls
cpuhp_remove_state(CPUHP_AP_RISCV_SSE_ONLINE) while higher states such as
CPUHP_AP_PERF_ONLINE are still active. In sse_cpu_teardown(),
sse_events_mask() only masks SSE delivery, and scoped_guard(preempt) leaves
hardirqs enabled for local events (and is not used for global events).
Because sse_event_disable_local() does not check sse_shutting_down, a racing
timer IRQ or IPI calling sse_event_disable_local() concurrently with
sse_teardown_event() can cause one of the two SBI_SSE_EVENT_DISABLE calls to
fail with -EINVAL. That causes sse_teardown_event() to abort before
sse_sbi_unregister_event(), sets sse_teardown_failed, and triggers
panic("SSE teardown failed; refusing unsafe kexec") during kexec.

3. During normal CPU offline in sse_cpu_teardown() (where sse_shutting_down
is false) or global event migration in sse_event_set_target_cpu_nolock(), the
event is disabled while reg_evt->is_registered remains true. A concurrent
sse_event_enable_local() from an IRQ on the same CPU (or from another CPU for
a global event, since sse_event_enable_local() and sse_event_disable_local()
do not take sse_mutex or cpus_read_lock()) can re-enable the event in
firmware before SBI_SSE_EVENT_UNREGISTER or SBI_SSE_EVENT_ATTR_WRITE runs,
causing firmware to reject the operation with -EINVAL. Also,
sse_event_enable_local() calls sse_get_reg_evt(event) (which uses
smp_processor_id() for local events) before
lockdep_assert_preemption_disabled().

4. If CPU offline aborts and jumps to restore_cpu in sse_cpu_teardown(),
sse_restore_local_events() restores local events using
event->enable_requested, which is only updated by sse_event_enable() and
sse_event_disable(), losing per-CPU enable state set via
sse_event_enable_local() or sse_event_disable_local().

[ ... ]
> +static int sse_event_set_target_cpu_nolock(struct sse_event *event,
> +					   unsigned int cpu)
> +{
> +	unsigned long hart_id, old_hart_id;
> +	struct sse_registered_event *reg_evt = event->global;
> +	u32 evt = event->evt_id;
> +	unsigned int old_cpu;
> +	bool was_enabled;
> +	int ret;
> +
> +	if (!sse_event_is_global(evt))
> +		return -EINVAL;
> +
> +	if (cpu >= nr_cpu_ids || !cpu_online(cpu))
> +		return -EINVAL;
> +	hart_id = cpuid_to_hartid_map(cpu);
> +	old_cpu = event->cpu;
> +	old_hart_id = cpuid_to_hartid_map(old_cpu);
> +
> +	was_enabled = reg_evt->is_enabled;
> +	if (was_enabled) {
> +		ret = sse_event_disable_local(event);
> +		if (ret)
> +			return ret;
> +	}
> +
> +	ret = sse_event_attr_set_nolock(reg_evt, SBI_SSE_ATTR_PREFERRED_HART,
> +					hart_id);
> +	if (ret == 0)
> +		sse_global_event_update_cpu(event, cpu);
> +
> +	if (was_enabled) {
> +		int enable_ret;
> +
> +		enable_ret = sse_event_enable_local(event);
> +		if (enable_ret) {
> +			int rollback_ret;
> +
> +			/*
> +			 * The preferred hart was already changed. Restore both the
> +			 * firmware attribute and Linux's cached target before reporting
> +			 * the failed migration.
> +			 */
> +			rollback_ret = sse_event_attr_set_nolock(reg_evt,
> +								 SBI_SSE_ATTR_PREFERRED_HART,
> +								 old_hart_id);
> +			if (!rollback_ret) {
> +				sse_global_event_update_cpu(event, old_cpu);
> +				rollback_ret = sse_event_enable_local(event);
> +			}
> +			if (!rollback_ret)
> +				return enable_ret;
> +
> +			pr_warn("Failed to restore global event %x to CPU %u: %d\n",
> +				evt, old_cpu, rollback_ret);
> +			event->enable_requested = false;
> +			return enable_ret;
> +		}
> +	}
> +
> +	return ret;
> +}

[Severity: Medium]
In sse_event_set_target_cpu_nolock(), what happens if
sse_event_attr_set_nolock(reg_evt, SBI_SSE_ATTR_PREFERRED_HART, hart_id)
fails (ret != 0) when was_enabled is true, and the subsequent
sse_event_enable_local(event) on old_cpu also fails?

Because the if (enable_ret) block does not check whether ret == 0, it will
assume the preferred hart was changed, write old_hart_id to
SBI_SSE_ATTR_PREFERRED_HART again, call sse_event_enable_local(event) a
second time, and return enable_ret instead of the original attribute-write
error in ret.

[ ... ]
> +void sse_event_cleanup(struct sse_event *event)
> +{
> +	guard(mutex)(&sse_mutex);
> +	guard(cpus_read_lock)();
> +
> +	if (event->cleanup_pending)
> +		return;
> +
> +	/*
> +	 * Firmware may still enter the old callback after disable or unregister
> +	 * fails. Publish a core-owned callback, then wait before the client frees
> +	 * its callback data.
> +	 */
> +	event->cleanup_pending = true;
> +	rcu_assign_pointer(event->handler, sse_cleanup_event_handler);
> +	synchronize_rcu();
> +}
> +EXPORT_SYMBOL_GPL(sse_event_cleanup);
> +
> +static void sse_release_cleanup_event(struct sse_event *event)
> +{
> +	if (!event->cleanup_pending || sse_event_is_registered(event))
> +		return;
> +
> +	list_del(&event->list);
> +	sse_event_free(event);
> +}

[Severity: High]
Does sse_event_cleanup() leak struct sse_event (along with its global or
per-CPU registrations, vmap stacks, shadow call stacks, and interrupted
buffers) and leave the event ID permanently blocked in the events list with
-EEXIST?

In sse_event_cleanup(), event->cleanup_pending is set to true and the handler
is replaced with sse_cleanup_event_handler, but sse_release_cleanup_event() is
never called, nor does sse_event_cleanup() clear event->enable_requested or
attempt to disable and unregister remaining online instances.

This leaves the event stranded in two scenarios:

1. If all firmware registrations are already gone by the time
sse_event_cleanup() acquires sse_mutex, sse_event_is_registered(event) is
already false. This can happen if sse_event_unregister() fails on one CPU and
releases sse_mutex before returning the error, and that CPU goes offline via
sse_cpu_teardown() before the client calls sse_event_cleanup() (when
sse_cpu_teardown() ran, cleanup_pending was still false, so
sse_release_cleanup_event() was a no-op). It also happens during reboot/kexec
if sse_reboot_notifier() runs cpuhp_remove_state(CPUHP_AP_RISCV_SSE_ONLINE)
first and a client shutdown callback then receives -ESHUTDOWN from
sse_event_disable() or sse_event_unregister() and calls sse_event_cleanup().
Once sse_event_cleanup() sets cleanup_pending = true on an already
unregistered event without calling sse_release_cleanup_event(event), nothing
may ever revisit it (especially for global events, which sse_cpu_online()
skips), leaking the event and causing future sse_event_register() calls for
that evt_id to fail with -EEXIST.

2. If a client calls sse_event_cleanup() directly after sse_event_disable()
fails on one CPU without calling sse_event_unregister(), sse_event_cleanup()
does not clear event->enable_requested or unregister the event on the online
CPUs where disabling succeeded. Those online CPUs remain registered in
firmware, preventing sse_release_cleanup_event() from ever freeing the event.

[ ... ]
> +struct sse_event *sse_event_register(u32 evt, u32 priority,
> +				     sse_event_handler_fn *handler, void *arg)
> +{
[ ... ]
> +	if (sse_event_is_global(evt)) {
> +		unsigned long preferred_hart;
> +
> +		ret = sse_event_attr_get_no_lock(event->global,
> +						 SBI_SSE_ATTR_PREFERRED_HART,
> +						 &preferred_hart);
> +		if (ret)
> +			return sse_register_failed(event, ret);
> +
> +		cpu = riscv_hartid_to_cpuid(preferred_hart);
> +		if (cpu < 0 || !cpu_online(cpu)) {
> +			cpu = cpumask_first(cpu_online_mask);

[Severity: Medium]
What happens in sse_event_register() if firmware reports preferred_hart ==
INVALID_HARTID when nr_cpu_ids < NR_CPUS?

In arch/riscv/kernel/smp.c, unpopulated entries in __cpuid_to_hartid_map[NR_CPUS]
are initialized to INVALID_HARTID, and riscv_hartid_to_cpuid() loops up to
NR_CPUS:

arch/riscv/kernel/smp.c:riscv_hartid_to_cpuid() {
	for (i = 0; i < NR_CPUS; i++)
		if (cpuid_to_hartid_map(i) == hartid)
			return i;
}

When preferred_hart is INVALID_HARTID, riscv_hartid_to_cpuid() returns the
first unpopulated index (cpu >= nr_cpu_ids) rather than a negative error.
Passing cpu >= nr_cpu_ids to cpu_online(cpu) without first checking
preferred_hart == INVALID_HARTID or cpu >= nr_cpu_ids accesses past
nr_cpu_ids in cpu_online_mask and triggers WARN_ON_ONCE(cpu >=
small_cpumask_bits) in cpumask_check() when CONFIG_DEBUG_PER_CPU_MAPS is
enabled.

[ ... ]
> +static int sse_cpu_teardown(unsigned int cpu)
> +{
> +	/* Only a regular CPU-offline callback may abort the CPUHP operation. */
> +	bool regular_offline = READ_ONCE(sse_available) &&
> +			       !READ_ONCE(sse_shutting_down);
> +	unsigned int next_cpu;
> +	struct sse_event *event, *tmp;
> +	int first_error = 0;
> +	int ret;
> +
> +	/* Do not dismantle CPU state while firmware can still deliver SSE. */
> +	ret = sse_events_mask();
> +	if (ret) {
> +		pr_warn("Failed to mask SSE on CPU %u during teardown: %d\n",
> +			cpu, ret);
> +		if (READ_ONCE(sse_shutting_down))
> +			atomic_set(&sse_teardown_failed, 1);
> +		/* CPUHP installation rollback and state removal cannot fail. */
> +		return regular_offline ? ret : 0;
> +	}
> +
> +	list_for_each_entry_safe(event, tmp, &events, list) {
> +		if (sse_event_is_global(event->evt_id))
> +			continue;
> +
> +		scoped_guard(preempt)
> +			ret = sse_teardown_event(event, cpu);
> +		if (ret && !first_error)
> +			first_error = ret;
> +		sse_release_cleanup_event(event);
> +		if (ret && regular_offline)
> +			goto restore_cpu;
> +	}
> +
> +	list_for_each_entry_safe(event, tmp, &events, list) {
> +		if (!sse_event_is_global(event->evt_id) || event->cpu != cpu)
> +			continue;
> +
> +		/*
> +		 * cpuhp_remove_state() invokes teardown while every CPU remains
> +		 * online. Do not migrate to a CPU whose callback may have run.
> +		 */
> +		if (READ_ONCE(sse_shutting_down) || event->cleanup_pending) {
> +			ret = sse_teardown_event(event, cpu);
> +		} else {
> +			next_cpu = cpumask_any_but(cpu_online_mask, cpu);
> +			if (next_cpu >= nr_cpu_ids) {
> +				ret = sse_teardown_event(event, cpu);
> +			} else {
> +				ret = sse_event_set_target_cpu_nolock(event, next_cpu);
> +				if (ret)
> +					pr_warn("Failed to migrate global event %x from CPU %u: %d\n",
> +						event->evt_id, cpu, ret);
> +			}
> +		}
> +
> +		if (ret && !first_error)
> +			first_error = ret;
> +		sse_release_cleanup_event(event);
> +		if (ret && regular_offline)
> +			goto restore_cpu;
> +	}
> +
> +	if (first_error) {
> +		/* An offline CPU is not revisited when this CPUHP state is removed. */
> +		atomic_set(&sse_teardown_failed, 1);
> +	}
> +
> +	return 0;
> +
> +restore_cpu:
> +	/*
> +	 * CPUHP leaves this CPU online when teardown returns an error. Restore
> +	 * every client-owned local event before making SSE delivery visible again.
> +	 */
> +	ret = sse_restore_local_events(cpu);
> +	if (!ret)
> +		ret = sse_events_unmask();
> +	if (ret) {
> +		atomic_set(&sse_teardown_failed, 1);
> +		pr_warn("Failed to restore SSE after aborting CPU %u offline: %d\n",
> +			cpu, ret);
> +	} else {
> +		pr_warn("Aborted CPU %u offline after SSE teardown failed: %d\n",
> +			cpu, first_error);
> +	}
> +
> +	return first_error;
> +}

[Severity: High]
Does sse_cpu_teardown() handle a post-boot _cpu_up() rollback when a CPUHP
state after CPUHP_AP_RISCV_SSE_ONLINE fails?

Once sse_init() sets sse_available = true, if a CPU is being brought online
via _cpu_up() and sse_cpu_online() succeeds at CPUHP_AP_RISCV_SSE_ONLINE, but
a later startup state (such as CPUHP_AP_PERF_ONLINE) fails, CPUHP sets
st->rollback = true and invokes teardown callbacks on the AP to roll back
towards offline.

During that _cpu_up() rollback, regular_offline is true in
sse_cpu_teardown(). If sse_events_mask() fails, sse_cpu_teardown() returns ret
without setting sse_teardown_failed. If any sse_teardown_event() call fails,
it jumps to restore_cpu, which calls sse_restore_local_events(cpu) and
sse_events_unmask(), leaves sse_teardown_failed unset if the restore succeeds,
and returns first_error (non-zero).

In kernel/cpu.c:cpuhp_thread_fun(), returning non-zero from a teardown
callback during rollback triggers WARN_ON_ONCE(st->rollback) and halts
rollback of earlier AP states, while cpuhp_up_callbacks() still proceeds to
takedown_cpu() and stops the CPU with SSE unmasked, local events still
registered and enabled in firmware, and sse_teardown_failed left at 0.

> +
> +static int sse_pm_notifier(struct notifier_block *nb, unsigned long action,
> +			   void *data)
> +{
> +	int ret;
> +
> +	WARN_ON_ONCE(preemptible());
> +
> +	switch (action) {
> +	case CPU_PM_ENTER:\n> +		ret = sse_events_mask();
> +		break;
> +	case CPU_PM_EXIT:
> +	case CPU_PM_ENTER_FAILED:
> +		if (READ_ONCE(sse_shutting_down))
> +			return NOTIFY_OK;
> +		ret = sse_events_unmask();
> +		break;
> +	default:
> +		return NOTIFY_DONE;
> +	}
> +
> +	if (ret)
> +		return notifier_from_errno(ret);
> +
> +	return NOTIFY_OK;
> +}
[ ... ]
> +static int __init sse_init(void)
> +{
[ ... ]
> +	ret = cpu_pm_register_notifier(&sse_pm_nb);
> +	if (ret) {
> +		pr_warn("Failed to register CPU PM notifier...\n");
> +		return ret;
> +	}
> +
> +	ret = register_reboot_notifier(&sse_reboot_nb);
> +	if (ret) {
> +		pr_warn("Failed to register reboot notifier...\n");
> +		goto remove_cpupm;
> +	}
> +
> +	ret = atomic_notifier_chain_register(&panic_notifier_list, &sse_panic_nb);
> +	if (ret) {
> +		pr_warn("Failed to register panic notifier...\n");
> +		goto remove_reboot;
> +	}
> +
> +	/* Tear down perf events before dismantling their SSE delivery path. */
> +	ret = cpuhp_setup_state(CPUHP_AP_RISCV_SSE_ONLINE, "riscv/sse:online",
> +				sse_cpu_online, sse_cpu_teardown);

[Severity: High]
Can sse_pm_notifier() unmask SSE on a CPU before sse_cpu_online() has
initialized its per-CPU SSE state, or after sse_cpu_teardown() has already
dismantled it?

In sse_pm_notifier(), CPU_PM_EXIT and CPU_PM_ENTER_FAILED unconditionally call
sse_events_unmask() whenever sse_shutting_down is false, without checking
whether SSE is currently online on this CPU:

1. In sse_init(), cpu_pm_register_notifier(&sse_pm_nb) is registered before
cpuhp_setup_state(CPUHP_AP_RISCV_SSE_ONLINE, ...). Because sbi_cpuidle_init()
is also an arch_initcall and drivers/cpuidle/ is linked before
drivers/firmware/, cpuidle can already be active on online CPUs and call
cpu_pm_exit() -> sse_pm_notifier() -> sse_events_unmask() before
sse_cpu_online() runs arch_sse_init_cpu() to set __sbi_sse_entry_task.

2. During secondary CPU bringup, the AP enables interrupts in smp_callin() and
enters the idle loop at CPUHP_AP_ONLINE_IDLE (cpu_startup_entry() -> do_idle()
-> cpuidle_idle_call() -> cpu_pm_enter() / cpu_pm_exit()) before the cpuhp/%u
thread reaches CPUHP_AP_RISCV_SSE_ONLINE (or after sse_cpu_online() fails and
rolls back), unmasking SSE on an uninitialized hart.

3. During normal CPU offline (_cpu_down()), sse_cpu_teardown() runs at
CPUHP_AP_RISCV_SSE_ONLINE on the cpuhp/%u thread and masks SSE. The cpuhp/%u
thread then parks at CPUHP_AP_ONLINE_IDLE while the offlining CPU remains in
cpu_online_mask and runs do_idle() -> cpuidle_idle_call() waiting for the
control CPU to invoke takedown_cpu() -> stop_machine_cpuslocked(take_cpu_down,
...) at CPUHP_TEARDOWN_CPU. When the CPU wakes from cpuidle (for example, on
the stop_machine IPI), CPU_PM_EXIT in sse_pm_notifier() calls
sse_events_unmask() and re-enables SSE on the dying hart after teardown.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261029074853.35467-1-zhangzhanpeng.jasper@bytedance.com?part=4

[PATCH v12 05/11] riscv: mm: avoid enabling interrupts for nofault page faults

[Zhanpeng Zhang]

RISC-V enables interrupts in handle_page_fault() before checking whether
the fault occurred with fault handling disabled. A nofault access from an
atomic context can therefore run tracepoints and open an interrupt window
before reaching the exception-table fixup.

Handle an exception-table entry before entering the generic fault path when
fault handling is disabled. Also keep interrupts disabled until such a
fault has been resolved.

On 32-bit RISC-V the vmalloc mapping must still be synchronised into the
current page table before that fixup runs: otherwise a nofault access to a
not-yet-synced vmalloc address would take the exception-table fixup and
fail spuriously instead of being resolved. Order the vmalloc
synchronisation ahead of the nofault fixup. That synchronisation only
copies an already-established kernel mapping, takes no locks and waits for
no I/O, so it is safe in the nofault path; if the mapping does not exist it
falls through to the existing no_context() error handling. This early
return skips the page-fault tracepoints, as before.

This makes RISC-V consistent with the expectation that an in-atomic
nofault access does not enter the normal fault-handling path. It also
removes one source of re-entry when perf sampling is delivered through an
SBI Supervisor Software Event (SSE).

Signed-off-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
---
 arch/riscv/mm/fault.c | 32 ++++++++++++++++++++++++--------
 1 file changed, 24 insertions(+), 8 deletions(-)

diff --git a/arch/riscv/mm/fault.c b/arch/riscv/mm/fault.c
index 04ed6f8acae..3934248f7a2 100644
--- a/arch/riscv/mm/fault.c
+++ b/arch/riscv/mm/fault.c
@@ -294,19 +294,23 @@ void handle_page_fault(struct pt_regs *regs)
 	if (kprobe_page_fault(regs, cause))
 		return;
 
-	if (user_mode(regs))
-		trace_page_fault_user(addr, regs, cause);
-	else
-		trace_page_fault_kernel(addr, regs, cause);
-
 	/*
 	 * Fault-in kernel-space virtual memory on-demand.
 	 * The 'reference' page table is init_mm.pgd.
 	 *
+	 * This only synchronises an already-established kernel vmalloc mapping
+	 * into the current page table; it neither allocates memory nor waits for
+	 * I/O, so it is safe in the nofault path. It must run before the
+	 * exception-table fixup below: otherwise a nofault access to a
+	 * not-yet-synced vmalloc address would take the fixup and fail
+	 * spuriously instead of being resolved. If the mapping does not exist,
+	 * vmalloc_fault() falls through to no_context() error handling.
+	 *
 	 * NOTE! We MUST NOT take any locks for this case. We may
 	 * be in an interrupt or a critical region, and should
 	 * only copy the information from the master page table,
-	 * nothing more.
+	 * nothing more. This path returns early and therefore skips the
+	 * trace_page_fault_*() tracepoints below.
 	 */
 	if ((!IS_ENABLED(CONFIG_MMU) || !IS_ENABLED(CONFIG_64BIT)) &&
 	    unlikely(addr >= VMALLOC_START && addr < VMALLOC_END)) {
@@ -314,8 +318,20 @@ void handle_page_fault(struct pt_regs *regs)
 		return;
 	}
 
-	/* Enable interrupts if they were enabled in the parent context. */
-	if (!regs_irqs_disabled(regs))
+	/*
+	 * Nofault accesses must be resolved through the exception table before
+	 * entering the generic fault path or enabling interrupts.
+	 */
+	if (unlikely(faulthandler_disabled()) && fixup_exception(regs))
+		return;
+
+	if (user_mode(regs))
+		trace_page_fault_user(addr, regs, cause);
+	else
+		trace_page_fault_kernel(addr, regs, cause);
+
+	/* Do not open an interrupt window before a nofault fixup completes. */
+	if (!regs_irqs_disabled(regs) && !faulthandler_disabled())
 		local_irq_enable();
 
 	/*
-- 
2.50.1 (Apple Git-155)

Re: [PATCH v12 05/11] riscv: mm: avoid enabling interrupts for nofault page faults

[sashiko-bot]

> RISC-V enables interrupts in handle_page_fault() before checking whether
> the fault occurred with fault handling disabled. A nofault access from an
> atomic context can therefore run tracepoints and open an interrupt window
> before reaching the exception-table fixup.
> 
> Handle an exception-table entry before entering the generic fault path when
> fault handling is disabled. Also keep interrupts disabled until such a
> fault has been resolved.
> 
> On 32-bit RISC-V the vmalloc mapping must still be synchronised into the
> current page table before that fixup runs: otherwise a nofault access to a
> not-yet-synced vmalloc address would take the exception-table fixup and
> fail spuriously instead of being resolved. Order the vmalloc
> synchronisation ahead of the nofault fixup. That synchronisation only
> copies an already-established kernel mapping, takes no locks and waits for
> no I/O, so it is safe in the nofault path; if the mapping does not exist it
> falls through to the existing no_context() error handling. This early
> return skips the page-fault tracepoints, as before.
> 
> This makes RISC-V consistent with the expectation that an in-atomic
> nofault access does not enter the normal fault-handling path. It also
> removes one source of re-entry when perf sampling is delivered through an
> SBI Supervisor Software Event (SSE).
> 
> Signed-off-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261029074853.35467-1-zhangzhanpeng.jasper@bytedance.com?part=5

[PATCH v12 06/11] riscv: enable NMI-safe lockdep irqflags tracking

[Zhanpeng Zhang]

An SSE handler runs with interrupts masked, but can interrupt code whose
lockdep hardirq state is enabled. Bare nmi_enter()/nmi_exit() do not
update that state. Consequently, lockdep_assert_irqs_disabled() in the
perf overflow path can report a warning despite interrupts being masked.

Select TRACE_IRQFLAGS_NMI_SUPPORT so lockdep can update its hardirq state
inside an NMI. Pair this with irqentry_nmi_enter()/irqentry_nmi_exit() in
the SSE wrapper: these helpers save the previous tracked state, mark
interrupts disabled on entry and restore the tracked state on exit.
Selecting the configuration option alone does not perform those updates.
This follows the entry helpers used by the other RISC-V NMI-like traps.

Signed-off-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
---
 arch/riscv/Kconfig          | 1 +
 arch/riscv/kernel/sbi_sse.c | 7 ++++---
 2 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index 467a877445a..dea30784fc7 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -228,6 +228,7 @@ config RISCV
 	select SYSCTL_EXCEPTION_TRACE
 	select THREAD_INFO_IN_TASK
 	select TRACE_IRQFLAGS_SUPPORT
+	select TRACE_IRQFLAGS_NMI_SUPPORT
 	select UACCESS_MEMCPY if !MMU
 	select VDSO_DATASTORE if MMU
 	select VDSO_GETRANDOM if MMU && 64BIT
diff --git a/arch/riscv/kernel/sbi_sse.c b/arch/riscv/kernel/sbi_sse.c
index 032fcc07abc..59e47a39589 100644
--- a/arch/riscv/kernel/sbi_sse.c
+++ b/arch/riscv/kernel/sbi_sse.c
@@ -2,7 +2,7 @@
 /*
  * Copyright (C) 2025 Rivos Inc.
  */
-#include <linux/nmi.h>
+#include <linux/entry-common.h>
 #include <linux/sched.h>
 #include <linux/scs.h>
 #include <linux/bitfield.h>
@@ -36,9 +36,10 @@ void noinstr do_sse(struct sse_event_arch_data *arch_evt,
 {
 	struct riscv_sse_interrupted_context context = { regs, hstatus };
 	struct riscv_sse_interrupted_context *previous;
+	irqentry_state_t state;
 	struct sbiret sret;
 
-	nmi_enter();
+	state = irqentry_nmi_enter(regs);
 	instrumentation_begin();
 
 	/* Retrieve missing GPRs from SBI */
@@ -84,7 +85,7 @@ void noinstr do_sse(struct sse_event_arch_data *arch_evt,
 
 out:
 	instrumentation_end();
-	nmi_exit();
+	irqentry_nmi_exit(regs, state);
 }
 
 static void *alloc_to_stack_pointer(void *alloc)
-- 
2.50.1 (Apple Git-155)

Re: [PATCH v12 06/11] riscv: enable NMI-safe lockdep irqflags tracking

[sashiko-bot]

> An SSE handler runs with interrupts masked, but can interrupt code whose
> lockdep hardirq state is enabled. Bare nmi_enter()/nmi_exit() do not
> update that state. Consequently, lockdep_assert_irqs_disabled() in the
> perf overflow path can report a warning despite interrupts being masked.
> 
> Select TRACE_IRQFLAGS_NMI_SUPPORT so lockdep can update its hardirq state
> inside an NMI. Pair this with irqentry_nmi_enter()/irqentry_nmi_exit() in
> the SSE wrapper: these helpers save the previous tracked state, mark
> interrupts disabled on entry and restore the tracked state on exit.
> Selecting the configuration option alone does not perform those updates.
> This follows the entry helpers used by the other RISC-V NMI-like traps.
> 
> Signed-off-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261029074853.35467-1-zhangzhanpeng.jasper@bytedance.com?part=6

[PATCH v12 07/11] perf: RISC-V: support callchains with SSE delivery

[Zhanpeng Zhang]

PMU overflow delivery through SSE enters Linux with a synthetic supervisor
context on a dedicated event stack. The perf unwinder must use the context
interrupted by the overflow rather than treating the SSE handler frame as
the sampled frame.

Use the interrupted pt_regs published by the RISC-V SSE entry path. Walk a
kernel callchain only when the interrupted PC, SP, and frame pointer are
consistent with the current task stack. For sensitive entry windows and
IRQ stacks whose bounds cannot be proven, retain the interrupted PC without
following an unsafe frame chain.

User callchains continue through the existing nofault RISC-V user unwinder.
If hstatus.SPV says the interrupted context was a guest, do not interpret
the guest stack through the host address space.

DWARF callchains additionally copy a raw user stack. Provide an SSE-aware
riscv_perf_out_copy_user() following x86's copy_from_user_nmi().
The copy runs under pagefault_disable(), so a non-resident source
page takes an exception-table handled fault that returns the number
of bytes not copied
instead of entering handle_mm_fault() or opening an interrupt window, and
perf truncates the user-stack dump accordingly. The SSE entry saves and
restores the interrupted task's stack metadata across that nested trap.

In SSE context also check access_ok(), require a user mm, and compare
the active page-table root with current->mm before copying. This rejects
a mismatched root, but does not exclude stale translations in the window
between a satp update and the following TLB flush. It is not a complete
equivalent of x86's nmi_uaccess_okay(). Outside SSE, retain the existing
inatomic copy with page faults disabled.

Susheng Yang reported this failure with perf callchain workloads.

Link: https://lore.kernel.org/r/as79MHtNYd-VbUug@hu-himchau-blr.qualcomm.com/
Reported-by: Susheng Yang <yang.susheng@zte.com.cn>
Signed-off-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
---
 arch/riscv/include/asm/perf_event.h |  10 ++
 arch/riscv/kernel/perf_callchain.c  | 140 ++++++++++++++++++++++++++++
 2 files changed, 150 insertions(+)

diff --git a/arch/riscv/include/asm/perf_event.h b/arch/riscv/include/asm/perf_event.h
index bcc928fd378..ddc40479475 100644
--- a/arch/riscv/include/asm/perf_event.h
+++ b/arch/riscv/include/asm/perf_event.h
@@ -18,6 +18,16 @@
 	(regs)->sp = current_stack_pointer; \
 	(regs)->status = SR_PP; \
 }
+
+#ifdef CONFIG_RISCV_SBI_SSE
+/*
+ * Raw user-stack sampling can run in the NMI-like SSE context. Route it
+ * through an implementation that does not fault on a non-resident page.
+ */
+unsigned long riscv_perf_out_copy_user(void *dst, const void *src,
+				       unsigned long n);
+#define arch_perf_out_copy_user riscv_perf_out_copy_user
+#endif
 #endif
 
 #endif /* _ASM_RISCV_PERF_EVENT_H */
diff --git a/arch/riscv/kernel/perf_callchain.c b/arch/riscv/kernel/perf_callchain.c
index b465bc9eb87..216f5bd14c0 100644
--- a/arch/riscv/kernel/perf_callchain.c
+++ b/arch/riscv/kernel/perf_callchain.c
@@ -1,9 +1,15 @@
 // SPDX-License-Identifier: GPL-2.0
 /* Copyright (C) 2019 Hangzhou C-SKY Microsystems co.,ltd. */
 
+#include <linux/kallsyms.h>
+#include <linux/mm.h>
 #include <linux/perf_event.h>
+#include <linux/sched/task_stack.h>
+#include <linux/thread_info.h>
 #include <linux/uaccess.h>
 
+#include <asm/csr.h>
+#include <asm/sse.h>
 #include <asm/stacktrace.h>
 
 static bool fill_callchain(void *entry, unsigned long pc)
@@ -11,6 +17,126 @@ static bool fill_callchain(void *entry, unsigned long pc)
 	return perf_callchain_store(entry, pc) == 0;
 }
 
+#ifdef CONFIG_RISCV_SBI_SSE
+static bool sse_addr_on_task_stack(unsigned long addr, unsigned long size)
+{
+	unsigned long end = addr + size;
+	unsigned long stack;
+
+	if (end < addr)
+		return false;
+
+	stack = (unsigned long)task_stack_page(current);
+	if (addr >= stack && end <= stack + THREAD_SIZE)
+		return true;
+
+	return false;
+}
+
+static bool sse_kernel_regs_safe(struct pt_regs *regs)
+{
+	unsigned long fp = frame_pointer(regs);
+	unsigned long pc = instruction_pointer(regs);
+	unsigned long sp = user_stack_pointer(regs);
+
+	if (!__kernel_text_address(pc))
+		return false;
+	if (!sse_addr_on_task_stack(sp, sizeof(unsigned long)))
+		return false;
+	if (fp < sizeof(struct stackframe))
+		return false;
+
+	return sse_addr_on_task_stack(fp - sizeof(struct stackframe),
+				      sizeof(struct stackframe));
+}
+
+static bool sse_callchain_is_guest(const struct riscv_sse_interrupted_context *context)
+{
+	return context && (context->hstatus & HSTATUS_SPV);
+}
+
+static bool sse_callchain_kernel(struct perf_callchain_entry_ctx *entry,
+				 struct pt_regs *regs)
+{
+	const struct riscv_sse_interrupted_context *context;
+	unsigned long pc;
+
+	context = riscv_sse_get_interrupted_context();
+	if (!context || context->regs != regs)
+		return false;
+
+	/* A guest stack cannot be walked using the host kernel address space. */
+	if (sse_callchain_is_guest(context))
+		return true;
+
+	if (user_mode(regs))
+		return true;
+
+	if (sse_kernel_regs_safe(regs)) {
+		walk_stackframe(NULL, regs, fill_callchain, entry);
+		return true;
+	}
+
+	/*
+	 * Keep the sample useful for sensitive entry paths and IRQ stacks. The
+	 * generic walker does not take explicit IRQ stack bounds, and its
+	 * THREAD_SIZE alignment assumption fails for non-vmapped IRQ stacks.
+	 * Conservatively avoid walking IRQ stacks in every configuration.
+	 */
+	pc = instruction_pointer(regs);
+	if (__kernel_text_address(pc))
+		perf_callchain_store(entry, pc);
+
+	return true;
+}
+
+unsigned long riscv_perf_out_copy_user(void *dst, const void *src,
+				       unsigned long n)
+{
+	unsigned long ret;
+
+	/*
+	 * Raw user-stack sampling can run in the NMI-like SSE context. Mirror
+	 * x86's copy_from_user_nmi(): the copy may take a page fault, but it
+	 * must not fault a non-resident page in. pagefault_disable() keeps the
+	 * RISC-V fault handler on the nofault exception-table fixup (no
+	 * handle_mm_fault(), no interrupt window), and the SSE entry saves and
+	 * restores the interrupted task's stack metadata across that nested
+	 * trap. A fault therefore returns the number of bytes not copied, and
+	 * perf truncates the user-stack dump accordingly.
+	 *
+	 * In SSE context also approximate x86's nmi_uaccess_okay(): only sample
+	 * when the active page table root still matches current->mm, so we do
+	 * not walk an unrelated address space after interrupting a KVM guest, a
+	 * kernel thread, or a task whose mm is being swapped. This root check
+	 * does not close the window inside switch_mm() between writing satp and
+	 * flushing the local TLB (RISC-V has no LOADED_MM_SWITCHING marker like
+	 * x86); that is a window between instructions, not within one. In that
+	 * window a stale TLB entry may still resolve to the outgoing mapping, and
+	 * with ASID reuse the residue can even belong to an unrelated address
+	 * space. The nofault fixup only bounds faulting accesses; it cannot
+	 * detect a successful hit on a stale translation. This is therefore a
+	 * bounded sample-quality gap (a mis-attributed or truncated user-stack
+	 * dump), not a memory-safety hazard: the copy target is the perf ring
+	 * buffer and the length is bounded, so a stale read cannot corrupt an
+	 * unrelated kernel object.
+	 */
+	if (riscv_sse_get_interrupted_context()) {
+		if (!access_ok(src, n))
+			return n;
+		if (!current->mm ||
+		    (csr_read(CSR_SATP) & SATP_PPN) != virt_to_pfn(current->mm->pgd))
+			return n;
+	}
+
+	pagefault_disable();
+	ret = __copy_from_user_inatomic(dst, src, n);
+	pagefault_enable();
+
+	return ret;
+}
+#endif
+
 /*
  * This will be called when the target is in user mode
  * This function will only be called when we use
@@ -28,6 +154,15 @@ static bool fill_callchain(void *entry, unsigned long pc)
 void perf_callchain_user(struct perf_callchain_entry_ctx *entry,
 			 struct pt_regs *regs)
 {
+#ifdef CONFIG_RISCV_SBI_SSE
+	const struct riscv_sse_interrupted_context *context;
+
+	context = riscv_sse_get_interrupted_context();
+	/* A guest stack cannot be walked using the host address space. */
+	if (sse_callchain_is_guest(context))
+		return;
+#endif
+
 	if (perf_guest_state()) {
 		/* TODO: We don't support guest os callchain now */
 		return;
@@ -39,6 +174,11 @@ void perf_callchain_user(struct perf_callchain_entry_ctx *entry,
 void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry,
 			   struct pt_regs *regs)
 {
+#ifdef CONFIG_RISCV_SBI_SSE
+	if (sse_callchain_kernel(entry, regs))
+		return;
+#endif
+
 	if (perf_guest_state()) {
 		/* TODO: We don't support guest os callchain now */
 		return;
-- 
2.50.1 (Apple Git-155)

Re: [PATCH v12 07/11] perf: RISC-V: support callchains with SSE delivery

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:
- [High] perf: riscv: missing guest check in riscv_perf_out_copy_user()
- [High] perf: riscv: missing CSR_SATP check in perf_callchain_user()
- [High] perf: riscv: unsafe entry unwind and OOB read in sse_kernel_regs_safe()
- [High] perf: riscv: NULL task dereference via walk_stackframe() in fgraph
- [High] perf: riscv: NULL mm dereference in riscv_perf_out_copy_user()
--

commit 1d4832e59d0c42c6027b431628d4fdd87c2373eb
Author: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>

perf: RISC-V: support callchains with SSE delivery

This commit adds callchain and raw user-stack sampling support on RISC-V when
PMU overflow events are delivered via SBI Supervisor Software Events (SSE). It
validates the interrupted context before walking kernel or user stacks and
provides an SSE-aware riscv_perf_out_copy_user() helper.

Link: https://lore.kernel.org/r/as79MHtNYd-VbUug@hu-himchau-blr.qualcomm.com/

> diff --git a/arch/riscv/kernel/perf_callchain.c b/arch/riscv/kernel/perf_callchain.c
> index b465bc9eb870e..216f5bd14c001 100644
> --- a/arch/riscv/kernel/perf_callchain.c
> +++ b/arch/riscv/kernel/perf_callchain.c
> @@ -11,6 +17,126 @@ static bool fill_callchain(void *entry, unsigned long pc)
>  	return perf_callchain_store(entry, pc) == 0;
>  }
>  
> +#ifdef CONFIG_RISCV_SBI_SSE
> +static bool sse_addr_on_task_stack(unsigned long addr, unsigned long size)
> +{
> +	unsigned long end = addr + size;
> +	unsigned long stack;
> +
> +	if (end < addr)
> +		return false;
> +
> +	stack = (unsigned long)task_stack_page(current);
> +	if (addr >= stack && end <= stack + THREAD_SIZE)
> +		return true;
> +
> +	return false;
> +}
> +
> +static bool sse_kernel_regs_safe(struct pt_regs *regs)
> +{
> +	unsigned long fp = frame_pointer(regs);
> +	unsigned long pc = instruction_pointer(regs);
> +	unsigned long sp = user_stack_pointer(regs);
> +
> +	if (!__kernel_text_address(pc))
> +		return false;
> +	if (!sse_addr_on_task_stack(sp, sizeof(unsigned long)))
> +		return false;
> +	if (fp < sizeof(struct stackframe))
> +		return false;
> +
> +	return sse_addr_on_task_stack(fp - sizeof(struct stackframe),
> +				      sizeof(struct stackframe));
> +}

[Severity: High]
Can sse_kernel_regs_safe() return true in sensitive entry windows where s0 is
not a valid kernel frame pointer, or when fp equals stack + THREAD_SIZE,
causing walk_stackframe() to read out of bounds past the end of the task
stack?

In arch/riscv/kernel/entry.S, once sp points to the kernel task stack, there
are three windows where s0 does not point to a valid struct stackframe:

1. In ret_from_exception when returning to userspace:

   addi s0, sp, PT_SIZE_ON_STACK
   REG_S s0, TASK_TI_KERNEL_SP(tp)

Here s0 (fp) is set to stack + THREAD_SIZE while sp is on the task stack.
Because [fp - sizeof(struct stackframe), fp] lies within
[stack, stack + THREAD_SIZE], sse_kernel_regs_safe() returns true.

In arch/riscv/kernel/stacktrace.c:walk_stackframe():

   frame = (struct stackframe *)fp - 1;
   sp = fp;
   ...
   if (pc >= (unsigned long)handle_exception &&
       pc < (unsigned long)&ret_from_exception_end) {
           if (unlikely(!fn(arg, pc)))
                   break;

           pc = ((struct pt_regs *)sp)->epc;
           fp = ((struct pt_regs *)sp)->s0;
   }

When fp is stack + THREAD_SIZE, frame aliases the end of task_pt_regs(current)
(pt_regs->cause and pt_regs->orig_a0), and sp becomes stack + THREAD_SIZE. If
frame->ra (pt_regs->orig_a0, which is user-controlled a0 on a syscall) falls
in [handle_exception, ret_from_exception_end), walk_stackframe() dereferences
((struct pt_regs *)sp)->epc at stack + THREAD_SIZE, reading out of bounds into
the unmapped VMAP_STACK guard page in SSE context.

2. In handle_exception and ret_from_exception right after switching sp to the
kernel stack (or before restoring user sp), s0 still holds the untrusted
userspace s0 value. If userspace sets s0 to an address within its kernel task
stack range, sse_kernel_regs_safe() succeeds and walk_stackframe() unwinds
from the user-chosen kernel stack address.

3. In handle_exception on a trap from kernel mode, REG_L s0,
TASK_TI_USER_SP(tp) sets s0 to sp + PT_SIZE_ON_STACK before s0 is saved,
causing walk_stackframe() to interpret the top of the incomplete pt_regs as a
stackframe.

Should sse_kernel_regs_safe() reject PCs in
[handle_exception, ret_from_exception_end) and require fp to be strictly below
(unsigned long)task_pt_regs(current)?

> +
> +static bool sse_callchain_is_guest(const struct riscv_sse_interrupted_context *context)
> +{
> +	return context && (context->hstatus & HSTATUS_SPV);
> +}
> +
> +static bool sse_callchain_kernel(struct perf_callchain_entry_ctx *entry,
> +				 struct pt_regs *regs)
> +{
> +	const struct riscv_sse_interrupted_context *context;
> +	unsigned long pc;
> +
> +	context = riscv_sse_get_interrupted_context();
> +	if (!context || context->regs != regs)
> +		return false;
> +
> +	/* A guest stack cannot be walked using the host kernel address space. */
> +	if (sse_callchain_is_guest(context))
> +		return true;
> +
> +	if (user_mode(regs))
> +		return true;
> +
> +	if (sse_kernel_regs_safe(regs)) {
> +		walk_stackframe(NULL, regs, fill_callchain, entry);
> +		return true;
> +	}

[Severity: High]
Does passing NULL as the task argument to walk_stackframe() in
sse_callchain_kernel() (and in perf_callchain_kernel()) cause a NULL pointer
dereference when function_graph tracing is active?

In arch/riscv/kernel/stacktrace.c:walk_stackframe(), when regs is non-NULL,
task is not initialized to current and remains NULL:

   if (regs) {
           fp = frame_pointer(regs);
           sp = user_stack_pointer(regs);
           pc = instruction_pointer(regs);
   } else if (task == NULL || task == current) {
           ...
   }
   ...
   fp = READ_ONCE_TASK_STACK(task, frame->fp);
   pc = READ_ONCE_TASK_STACK(task, frame->ra);
   pc = ftrace_graph_ret_addr(task, &graph_idx, pc,
                              &frame->ra);

When a stack frame's return address is hooked by function_graph
(ret == return_handler), kernel/trace/fgraph.c:ftrace_graph_ret_addr()
evaluates:

   i = *idx ? : task->curr_ret_stack;

With *idx == 0 and task == NULL, this dereferences a NULL task pointer in SSE
context. Should walk_stackframe() be called with current instead of NULL?

[ ... ]

> +unsigned long riscv_perf_out_copy_user(void *dst, const void *src,
> +				       unsigned long n)
> +{
> +	unsigned long ret;
> +

[ ... ]

> +	if (riscv_sse_get_interrupted_context()) {
> +		if (!access_ok(src, n))
> +			return n;
> +		if (!current->mm ||
> +		    (csr_read(CSR_SATP) & SATP_PPN) != virt_to_pfn(current->mm->pgd))
> +			return n;
> +	}
> +
> +	pagefault_disable();
> +	ret = __copy_from_user_inatomic(dst, src, n);
> +	pagefault_enable();
> +
> +	return ret;
> +}
> +#endif

[Severity: High]
What prevents riscv_perf_out_copy_user() from copying host VMM user memory
when an SSE interrupts a running KVM guest?

The comment notes that comparing CSR_SATP against current->mm->pgd avoids
walking an unrelated address space after interrupting a KVM guest. However, on
RISC-V with the H-extension, arch/riscv/kvm/vcpu.c:kvm_riscv_vcpu_enter_exit()
switches hstatus, vsatp, and hgatp while leaving the host HS-mode CSR_SATP
loaded with the host VMM process's current->mm->pgd.

When an SSE with PERF_SAMPLE_STACK_USER interrupts a guest in VU-mode
(SR_PP == 0) or VS-mode (SR_PP == 1),
perf_prepare_sample()->perf_sample_regs_user() populates data->regs_user.regs
(using the guest's regs->sp for VU-mode, or task_pt_regs(current) for
VS-mode), and perf_output_sample_ustack() calls riscv_perf_out_copy_user().
Because CSR_SATP still matches virt_to_pfn(current->mm->pgd), the check
passes and __copy_from_user_inatomic() reads host VMM user memory at the
guest-controlled stack pointer into the perf sample.

Should riscv_perf_out_copy_user() check sse_callchain_is_guest(context) just
like sse_callchain_kernel() and perf_callchain_user() do?

[Severity: High]
Additionally, in riscv_perf_out_copy_user(), can calling access_ok(src, n)
before checking !current->mm (or calling __copy_from_user_inatomic() outside
SSE context when current->mm is NULL) dereference a NULL current->mm when
CONFIG_RISCV_ISA_SUPM is enabled?

In arch/riscv/include/asm/uaccess.h, access_ok() and raw_copy_from_user()
invoke untagged_addr(addr), which passes current->mm to
__untagged_addr_remote():

   static inline unsigned long __untagged_addr_remote(struct mm_struct *mm, unsigned long addr)
   {
           if (riscv_has_extension_unlikely(RISCV_ISA_EXT_SUPM)) {
                   u8 pmlen = mm->context.pmlen;

If a user task is sampled in do_exit() after exit_mm() has set
current->mm = NULL while is_user_task(current) is still true,
perf_sample_regs_user() sets regs_user->regs = task_pt_regs(current) and
perf_output_sample_ustack()->__output_copy_user() calls
riscv_perf_out_copy_user(dst, src, 0), dereferencing current->mm->context.pmlen
before reaching the !current->mm check.

[ ... ]

> @@ -28,6 +154,15 @@ static bool fill_callchain(void *entry, unsigned long pc)
>  void perf_callchain_user(struct perf_callchain_entry_ctx *entry,
>  			 struct pt_regs *regs)
>  {
> +#ifdef CONFIG_RISCV_SBI_SSE
> +	const struct riscv_sse_interrupted_context *context;
> +
> +	context = riscv_sse_get_interrupted_context();
> +	/* A guest stack cannot be walked using the host address space. */
> +	if (sse_callchain_is_guest(context))
> +		return;
> +#endif
> +
>  	if (perf_guest_state()) {
>  		/* TODO: We don't support guest os callchain now */
>  		return;

[Severity: High]
When perf_callchain_user() runs in SSE context, does it also need the
!current->mm and CSR_SATP page-table root check that riscv_perf_out_copy_user()
performs?

When an SSE interrupts a user task executing in kernel mode
(!user_mode(regs) && is_user_task(current)),
kernel/events/callchain.c:get_perf_callchain() substitutes
regs = task_pt_regs(current) and calls
perf_callchain_user()->arch_stack_walk_user()->unwind_user_frame(), which calls
access_ok() and __copy_from_user_inatomic().

Because SSE can interrupt kernel regions where IRQs are disabled, CSR_SATP or
current->mm may not match the current user task: for example, in
context_switch() after switch_mm_irqs_off() loads next->mm->pgd into CSR_SATP
before __switch_to() updates tp and __sbi_sse_entry_task, during
efi_virtmap_load() when CSR_SATP points to efi_mm.pgd, or in do_exit() after
exit_mm() sets current->mm to NULL. Without checking !current->mm and CSR_SATP
in perf_callchain_user(), unwind_user_frame() can walk the wrong address space
or dereference a NULL current->mm in access_ok().

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261029074853.35467-1-zhangzhanpeng.jasper@bytedance.com?part=7

[PATCH v12 08/11] perf: RISC-V: add support for SSE event

[Zhanpeng Zhang]

From: Clément Léger <cleger@rivosinc.com>

Register a handler for the local PMU overflow SSE event so that RISC-V
perf can receive overflows even when normal S-mode interrupts are masked.
Reuse the existing overflow handler and pass it the interrupted pt_regs
rebuilt by the architecture SSE entry path.

Select the delivery mechanism once during PMU probe. Prefer SSE when its
event can be registered and enabled. If the extension or PMU event is
explicitly unsupported, use the ordinary PMU interrupt. Do not enable the
interrupt after any other SSE setup failure or when a crash kernel may
have inherited firmware state. Install the PMU enable and disable
callbacks only after SSE delivery becomes active.

Keep the local PMU SSE event disabled across CPU power management. On
entry, the generic SSE notifier masks the hart before the lower-priority
PMU notifier disables the event and stops the counters. On exit, the SSE
notifier first unmasks the hart while the event remains disabled. The PMU
notifier then restores counters and event userpage state before enabling
the event. An unmask failure stops the notifier chain and leaves the
counters stopped. Ordinary PMU interrupts retain their existing notifier
ordering.

An SSE overflow can arrive as soon as the event is enabled during probe.
Publish the counter mask before SSE setup, so an early handler can stop
the counter source even before perf starts admitting normal samples.
After a real overflow, restart only counters whose perf state is still
running. Honor a non-zero return from perf_event_overflow() and leave
throttled events stopped.

Guest attribution is not part of this version. Detect an interrupted guest
from hstatus.SPV and skip its sample while still updating the period and
counter state, rather than exposing guest state as a host sample.

The perf PMU callbacks cannot return errors. If an SSE transition or
interrupted-context read fails, latch the failure per CPU and stop its
mapped events through the normal perf state transitions. Do not reset the
firmware counter mapping behind perf, restart a failed event, or attempt a
runtime switch to IRQ delivery.

Track ordinary IRQ ownership only when request_percpu_irq() succeeds.
Successful SSE setup does not acquire a per-CPU IRQ to release during
probe error unwinding.

When an SSE arrives without a usable perf event or before callback
admission, stop the counter sources individually. A batch STOP can
return SBI_ERR_ALREADY_STOPPED before stopping other counters.
Treat that result as harmless for a single counter, report other
errors, and continue attempting to stop the remaining sources.

During cleanup, close callback admission and synchronously drain each CPU
before disabling and unregistering the SSE event. If firmware refuses the
cleanup, stop the counter source and transfer the event to the SSE core so
later CPU hotplug or shutdown processing can retry without using freed PMU
callback state.

Signed-off-by: Clément Léger <cleger@rivosinc.com>
Co-developed-by: Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>
Signed-off-by: Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>
Co-developed-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
Signed-off-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
Acked-by: Conor Dooley <conor.dooley@microchip.com>
Link: https://lore.kernel.org/r/20261002-reach-cloning-f1594729dbe3@wendy/
---
 Documentation/arch/riscv/index.rst     |   1 +
 Documentation/arch/riscv/pmu-sse.rst   |  62 +++
 drivers/firmware/riscv/riscv_sbi_sse.c |   9 +
 drivers/perf/Kconfig                   |  11 +
 drivers/perf/riscv_pmu.c               |  14 +-
 drivers/perf/riscv_pmu_sbi.c           | 561 +++++++++++++++++++++----
 include/linux/perf/riscv_pmu.h         |  20 +-
 include/linux/riscv_sbi_sse.h          |   6 +
 8 files changed, 589 insertions(+), 95 deletions(-)
 create mode 100644 Documentation/arch/riscv/pmu-sse.rst

diff --git a/Documentation/arch/riscv/index.rst b/Documentation/arch/riscv/index.rst
index ac535c52d50..5cb909c83a8 100644
--- a/Documentation/arch/riscv/index.rst
+++ b/Documentation/arch/riscv/index.rst
@@ -11,6 +11,7 @@ RISC-V architecture
     vm-layout
     hwprobe
     patch-acceptance
+    pmu-sse
     uabi
     vector
     cmodx
diff --git a/Documentation/arch/riscv/pmu-sse.rst b/Documentation/arch/riscv/pmu-sse.rst
new file mode 100644
index 00000000000..98b55cc4bce
--- /dev/null
+++ b/Documentation/arch/riscv/pmu-sse.rst
@@ -0,0 +1,62 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+========================================
+RISC-V PMU overflow delivery through SSE
+========================================
+
+When ``CONFIG_RISCV_PMU_SBI_SSE`` is enabled and firmware provides the local
+PMU overflow event, the RISC-V SBI PMU driver uses Supervisor Software Events
+(SSE) to deliver counter overflows.
+
+Delivery selection
+==================
+
+The delivery mechanism is selected once while the PMU device is probed.  The
+driver first tries to register and enable the local PMU overflow SSE event.  It
+tries the ordinary PMU interrupt only when the SSE extension or the local PMU
+event is explicitly unsupported.  It does not change the delivery mechanism
+after the PMU has been registered.
+
+Other SSE setup errors do not prove that firmware released the overflow route,
+so the driver does not enable the ordinary PMU interrupt.  The PMU remains
+available for counting but not sampling.  A retained event handle does not
+mean that SSE delivery is active: the PMU enable and disable callbacks are
+installed only after setup succeeds.  Retained state prevents IRQ fallback
+and remains subject to SSE cleanup.
+
+Interrupted context
+===================
+
+SSE enters Linux through a supervisor handler context constructed by firmware.
+The entry contract preserves the interrupted GPRs except ``a6`` and ``a7``,
+which Linux reads from the interrupted-register event attributes.  Linux then
+reconstructs the interrupted ``pt_regs`` and publishes it while dispatching the
+event.
+
+Perf uses that context for register samples and callchains.  Kernel stack
+walking verifies that the interrupted frame belongs to the current task stack
+before dereferencing it.  For sensitive entry paths and IRQ stacks, Linux
+records the interrupted PC without walking a stack whose bounds cannot be
+proved.  User callchains use the existing no-fault user unwinder.  A DWARF
+raw user-stack copy uses ``__copy_from_user_inatomic()`` with page-fault
+handling disabled.  A non-resident source page may still cause a synchronous
+exception, which the exception-table fixup handles without entering normal
+page-fault handling or enabling interrupts.  Perf truncates the dump by the
+number of bytes not copied.  SSE entry preserves the task's stack metadata
+and early trap-entry scratch slots across nested exceptions.
+
+The SSE copy checks the user address range and requires the active page-table
+root to match ``current->mm``.  This check does not detect stale TLB entries
+in the address-space switch window; samples from that window can still be
+misattributed.
+
+CPU power management
+====================
+
+The PMU and SSE CPU power-management callbacks are ordered according to the
+selected delivery mechanism.  With SSE delivery, SSE events are masked before
+the lower-priority PMU callback disables the local PMU event and stops the
+counters on entry.  On exit, the hart is unmasked while the local PMU event is
+still disabled.  The PMU callback then restores the counters and enables the
+event, so an unmask failure leaves the counters stopped.  The ordinary
+interrupt path retains the existing PMU ordering.
diff --git a/drivers/firmware/riscv/riscv_sbi_sse.c b/drivers/firmware/riscv/riscv_sbi_sse.c
index 0cb738bab6c..69ce9838033 100644
--- a/drivers/firmware/riscv/riscv_sbi_sse.c
+++ b/drivers/firmware/riscv/riscv_sbi_sse.c
@@ -161,6 +161,15 @@ static int sse_sbi_event_func(struct sse_event *event, unsigned long func)
 	return 0;
 }
 
+bool sse_event_is_enabled_local(struct sse_event *event)
+{
+	if (!sse_event_is_global(event->evt_id))
+		lockdep_assert_preemption_disabled();
+
+	return sse_get_reg_evt(event)->is_enabled;
+}
+EXPORT_SYMBOL_GPL(sse_event_is_enabled_local);
+
 int sse_event_disable_local(struct sse_event *event)
 {
 	if (!sse_event_is_global(event->evt_id))
diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig
index 245e7bb763b..10b04785612 100644
--- a/drivers/perf/Kconfig
+++ b/drivers/perf/Kconfig
@@ -105,6 +105,17 @@ config RISCV_PMU_SBI
 	  full perf feature support i.e. counter overflow, privilege mode
 	  filtering, counter configuration.
 
+config RISCV_PMU_SBI_SSE
+	depends on RISCV_PMU_SBI && RISCV_SBI_SSE
+	bool "RISC-V PMU SSE events"
+	default n
+	help
+	  Say y if you want to use SSE events to deliver PMU interrupts. This
+	  provides a way to profile the kernel at any level by using NMI-like
+	  SSE events. SSE events being really intrusive, this option allows
+	  to select it only if needed. See Documentation/arch/riscv/pmu-sse.rst
+	  for interrupted-context limitations.
+
 config STARFIVE_STARLINK_PMU
 	depends on ARCH_STARFIVE || COMPILE_TEST
 	depends on 64BIT
diff --git a/drivers/perf/riscv_pmu.c b/drivers/perf/riscv_pmu.c
index 8e3cd0f3533..1f65c640b69 100644
--- a/drivers/perf/riscv_pmu.c
+++ b/drivers/perf/riscv_pmu.c
@@ -13,6 +13,7 @@
 #include <linux/irqdesc.h>
 #include <linux/perf/riscv_pmu.h>
 #include <linux/printk.h>
+#include <linux/riscv_sbi_sse.h>
 #include <linux/smp.h>
 #include <linux/sched_clock.h>
 
@@ -247,6 +248,11 @@ void riscv_pmu_start(struct perf_event *event, int flags)
 	if (flags & PERF_EF_RELOAD)
 		WARN_ON_ONCE(!(event->hw.state & PERF_HES_UPTODATE));
 
+#ifdef CONFIG_RISCV_PMU_SBI_SSE
+	if (unlikely(this_cpu_ptr(rvpmu->hw_events)->sse_failed))
+		return;
+#endif
+
 	hwc->state = 0;
 	riscv_pmu_event_set_period(event);
 	init_val = local64_read(&hwc->prev_count) & max_period;
@@ -306,6 +312,7 @@ static int riscv_pmu_event_init(struct perf_event *event)
 	struct hw_perf_event *hwc = &event->hw;
 	struct riscv_pmu *rvpmu = to_riscv_pmu(event->pmu);
 	int mapped_event;
+	int ret;
 	u64 event_config = 0;
 	uint64_t cmask;
 
@@ -331,8 +338,11 @@ static int riscv_pmu_event_init(struct perf_event *event)
 	hwc->idx = -1;
 	hwc->event_base = mapped_event;
 
-	if (rvpmu->event_init)
-		rvpmu->event_init(event);
+	if (rvpmu->event_init) {
+		ret = rvpmu->event_init(event);
+		if (ret)
+			return ret;
+	}
 
 	if (!is_sampling_event(event)) {
 		/*
diff --git a/drivers/perf/riscv_pmu_sbi.c b/drivers/perf/riscv_pmu_sbi.c
index 21a2078eab9..a984ac57a7b 100644
--- a/drivers/perf/riscv_pmu_sbi.c
+++ b/drivers/perf/riscv_pmu_sbi.c
@@ -16,13 +16,16 @@
 #include <linux/irqdomain.h>
 #include <linux/of_irq.h>
 #include <linux/of.h>
+#include <linux/riscv_sbi_sse.h>
 #include <linux/cpu_pm.h>
+#include <linux/crash_dump.h>
 #include <linux/sched/clock.h>
 #include <linux/soc/andes/irq.h>
 #include <linux/workqueue.h>
 
 #include <asm/errata_list.h>
 #include <asm/sbi.h>
+#include <asm/sse.h>
 #include <asm/cpufeature.h>
 #include <asm/vendor_extensions.h>
 #include <asm/vendor_extensions/andes.h>
@@ -95,7 +98,6 @@ static bool riscv_pmu_use_irq;
 static unsigned int riscv_pmu_irq_num;
 static unsigned int riscv_pmu_irq_mask;
 static unsigned int riscv_pmu_irq;
-
 /* Cache the available counters in a bitmask */
 static DECLARE_BITMAP(cmask, RISCV_MAX_COUNTERS);
 
@@ -927,7 +929,7 @@ static int pmu_sbi_get_ctrinfo(int nctr, unsigned long *mask)
 	return 0;
 }
 
-static inline void pmu_sbi_stop_all(struct riscv_pmu *pmu)
+static inline void pmu_sbi_stop_all_mask(const unsigned long *ctr_mask)
 {
 	int i;
 
@@ -936,14 +938,42 @@ static inline void pmu_sbi_stop_all(struct riscv_pmu *pmu)
 	 * which may include counters that are not enabled yet.
 	 */
 	for (i = 0; i < BITS_TO_LONGS(RISCV_MAX_COUNTERS); i++) {
-		if (!pmu->cmask[i])
+		if (!ctr_mask[i])
 			continue;
 		sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_STOP,
-			  i * BITS_PER_LONG, pmu->cmask[i],
+			  i * BITS_PER_LONG, ctr_mask[i],
 			  SBI_PMU_STOP_FLAG_RESET);
 	}
 }
 
+static void pmu_sbi_sse_stop_all(const unsigned long *ctr_mask)
+{
+	struct sbiret ret;
+	int i;
+
+	/*
+	 * Stop each counter separately: an already-stopped counter must not
+	 * prevent firmware from stopping a later, overflowing counter.
+	 */
+	for_each_set_bit(i, ctr_mask, RISCV_MAX_COUNTERS) {
+		ret = sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_STOP,
+				i, 1, SBI_PMU_STOP_FLAG_RESET);
+		if (ret.error && ret.error != SBI_ERR_ALREADY_STOPPED)
+			pr_warn_ratelimited("failed to stop counter %d: %ld\n",
+					    i, ret.error);
+	}
+}
+
+static inline void pmu_sbi_stop_all(struct riscv_pmu *pmu)
+{
+	pmu_sbi_stop_all_mask(pmu->cmask);
+}
+
+static void pmu_sbi_stop_all_cpu(void *info)
+{
+	pmu_sbi_stop_all(info);
+}
+
 static inline void pmu_sbi_stop_hw_ctrs(struct riscv_pmu *pmu)
 {
 	struct cpu_hw_events *cpu_hw_evt = this_cpu_ptr(pmu->hw_events);
@@ -991,7 +1021,7 @@ static inline void pmu_sbi_stop_hw_ctrs(struct riscv_pmu *pmu)
 static inline void pmu_sbi_start_ovf_ctrs_sbi(struct cpu_hw_events *cpu_hw_evt,
 					      u64 ctr_ovf_mask)
 {
-	int idx = 0, i;
+	int idx, i;
 	struct perf_event *event;
 	unsigned long flag = SBI_PMU_START_FLAG_SET_INIT_VALUE;
 	unsigned long ctr_start_mask = 0;
@@ -1000,7 +1030,19 @@ static inline void pmu_sbi_start_ovf_ctrs_sbi(struct cpu_hw_events *cpu_hw_evt,
 	u64 init_val = 0;
 
 	for (i = 0; i < BITS_TO_LONGS(RISCV_MAX_COUNTERS); i++) {
-		ctr_start_mask = cpu_hw_evt->used_hw_ctrs[i] & ~ctr_ovf_mask;
+		unsigned long ctr_ovf_mask_word;
+		int lidx;
+
+		ctr_ovf_mask_word = ctr_ovf_mask >> (i * BITS_PER_LONG);
+		ctr_start_mask = 0;
+		for_each_set_bit(idx, &cpu_hw_evt->used_hw_ctrs[i], BITS_PER_LONG) {
+			lidx = idx + i * BITS_PER_LONG;
+			event = cpu_hw_evt->events[lidx];
+			if (event && !(event->hw.state & PERF_HES_STOPPED))
+				ctr_start_mask |= BIT(idx);
+		}
+		ctr_start_mask &= ~ctr_ovf_mask_word;
+
 		/* Start all the counters that did not overflow in a single shot */
 		if (ctr_start_mask) {
 			sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_START, i * BITS_PER_LONG,
@@ -1009,23 +1051,25 @@ static inline void pmu_sbi_start_ovf_ctrs_sbi(struct cpu_hw_events *cpu_hw_evt,
 	}
 
 	/* Reinitialize and start all the counter that overflowed */
-	while (ctr_ovf_mask) {
-		if (ctr_ovf_mask & 0x01) {
-			event = cpu_hw_evt->events[idx];
-			hwc = &event->hw;
-			max_period = riscv_pmu_ctr_get_width_mask(event);
-			init_val = local64_read(&hwc->prev_count) & max_period;
+	for (idx = 0; idx < RISCV_MAX_COUNTERS; idx++) {
+		if (!(ctr_ovf_mask & BIT_ULL(idx)))
+			continue;
+
+		event = cpu_hw_evt->events[idx];
+		if (!event || event->hw.state & PERF_HES_STOPPED)
+			continue;
+
+		hwc = &event->hw;
+		max_period = riscv_pmu_ctr_get_width_mask(event);
+		init_val = local64_read(&hwc->prev_count) & max_period;
 #if defined(CONFIG_32BIT)
-			sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_START, idx,
-				  1, flag, init_val, init_val >> 32);
+		sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_START, idx, 1,
+			  flag, init_val, init_val >> 32);
 #else
-			sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_START, idx,
-				  1, flag, init_val);
+		sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_START, idx, 1,
+			  flag, init_val);
 #endif
-			perf_event_update_userpage(event);
-		}
-		ctr_ovf_mask = ctr_ovf_mask >> 1;
-		idx++;
+		perf_event_update_userpage(event);
 	}
 }
 
@@ -1054,32 +1098,111 @@ static inline void pmu_sbi_start_ovf_ctrs_snapshot(struct cpu_hw_events *cpu_hw_
 	}
 
 	for (i = 0; i < BITS_TO_LONGS(RISCV_MAX_COUNTERS); i++) {
+		unsigned long ctr_start_mask = 0;
+		int lidx;
+
 		/* Restore the counter values to relative indices for used hw counters */
-		for_each_set_bit(idx, &cpu_hw_evt->used_hw_ctrs[i], BITS_PER_LONG)
-			sdata->ctr_values[idx] =
-					cpu_hw_evt->snapshot_cval_shcopy[idx + i * BITS_PER_LONG];
+		for_each_set_bit(idx, &cpu_hw_evt->used_hw_ctrs[i], BITS_PER_LONG) {
+			lidx = idx + i * BITS_PER_LONG;
+			event = cpu_hw_evt->events[lidx];
+			if (event && !(event->hw.state & PERF_HES_STOPPED))
+				ctr_start_mask |= BIT(idx);
+
+			sdata->ctr_values[idx] = cpu_hw_evt->snapshot_cval_shcopy[lidx];
+		}
+
 		/* Start all the counters in a single shot */
-		sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_START,
-			  idx * BITS_PER_LONG,
-			  cpu_hw_evt->used_hw_ctrs[i], flag);
+		if (ctr_start_mask)
+			sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_START,
+				  i * BITS_PER_LONG, ctr_start_mask, flag);
 	}
 }
 
+static bool pmu_sbi_sse_failed(struct cpu_hw_events *cpu_hw_evt)
+{
+#ifdef CONFIG_RISCV_PMU_SBI_SSE
+	return READ_ONCE(cpu_hw_evt->sse_failed);
+#else
+	return false;
+#endif
+}
+
 static void pmu_sbi_start_overflow_mask(struct riscv_pmu *pmu,
 					u64 ctr_ovf_mask)
 {
 	struct cpu_hw_events *cpu_hw_evt = this_cpu_ptr(pmu->hw_events);
 
+	if (unlikely(pmu_sbi_sse_failed(cpu_hw_evt)))
+		return;
+
 	if (sbi_pmu_snapshot_available())
 		pmu_sbi_start_ovf_ctrs_snapshot(cpu_hw_evt, ctr_ovf_mask);
 	else
 		pmu_sbi_start_ovf_ctrs_sbi(cpu_hw_evt, ctr_ovf_mask);
 }
 
-static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
+#ifdef CONFIG_RISCV_PMU_SBI_SSE
+/*
+ * A local SSE delivery failure makes the current PMU state unsafe to resume.
+ * Latch the failure before stopping mapped events so the SSE transition and
+ * overflow restart paths cannot undo the fail-safe while they are quiesced.
+ */
+static void pmu_sbi_fail_sse(struct riscv_pmu *pmu, const char *op, int ret)
+{
+	struct cpu_hw_events *cpu_hw_evt = this_cpu_ptr(pmu->hw_events);
+	struct perf_event *event;
+	int idx;
+
+	if (READ_ONCE(cpu_hw_evt->sse_failed))
+		return;
+
+	WRITE_ONCE(cpu_hw_evt->sse_failed, true);
+	pr_err_ratelimited("failed to %s local PMU SSE event: %d; stopping counters\n",
+			   op, ret);
+
+	for (idx = 0; idx < RISCV_MAX_COUNTERS; idx++) {
+		event = cpu_hw_evt->events[idx];
+		if (event)
+			riscv_pmu_stop(event, PERF_EF_UPDATE);
+	}
+}
+
+static void pmu_sbi_sse_disable(struct pmu *pmu)
+{
+	struct riscv_pmu *rvpmu = to_riscv_pmu(pmu);
+	struct cpu_hw_events *cpu_hw_evt = this_cpu_ptr(rvpmu->hw_events);
+	int ret;
+
+	if (!READ_ONCE(rvpmu->sse_active) ||
+	    READ_ONCE(cpu_hw_evt->sse_failed))
+		return;
+
+	ret = sse_event_disable_local(rvpmu->sse_evt);
+	if (unlikely(ret))
+		pmu_sbi_fail_sse(rvpmu, "disable", ret);
+}
+
+static void pmu_sbi_sse_enable(struct pmu *pmu)
+{
+	struct riscv_pmu *rvpmu = to_riscv_pmu(pmu);
+	struct cpu_hw_events *cpu_hw_evt = this_cpu_ptr(rvpmu->hw_events);
+	int ret;
+
+	if (!READ_ONCE(rvpmu->sse_active) ||
+	    READ_ONCE(cpu_hw_evt->sse_failed))
+		return;
+
+	ret = sse_event_enable_local(rvpmu->sse_evt);
+	if (unlikely(ret))
+		pmu_sbi_fail_sse(rvpmu, "enable", ret);
+}
+#endif
+
+static irqreturn_t pmu_sbi_ovf_handler(struct cpu_hw_events *cpu_hw_evt,
+				       struct pt_regs *regs, bool from_sse,
+				       bool from_guest)
 {
 	struct perf_sample_data data;
-	struct pt_regs *regs;
 	struct hw_perf_event *hw_evt;
 	union sbi_pmu_ctr_info *info;
 	int lidx, hidx, fidx;
@@ -1087,7 +1210,6 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
 	struct perf_event *event;
 	u64 overflow;
 	u64 overflowed_ctrs = 0;
-	struct cpu_hw_events *cpu_hw_evt = dev;
 	u64 start_clock = sched_clock();
 	struct riscv_pmu_snapshot_data *sdata;
 
@@ -1096,21 +1218,32 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
 
 	sdata = cpu_hw_evt->snapshot_addr;
 
-	/* Firmware counter don't support overflow yet */
+	/*
+	 * SSE can arrive before perf installs an event. The early exits below
+	 * must stop the PMU source before firmware completes the SSE.
+	 */
 	fidx = find_first_bit(cpu_hw_evt->used_hw_ctrs, RISCV_MAX_COUNTERS);
 	if (fidx == RISCV_MAX_COUNTERS) {
-		csr_clear(CSR_SIP, BIT(riscv_pmu_irq_num));
+		if (from_sse)
+			pmu_sbi_sse_stop_all(cmask);
+		else
+			csr_clear(CSR_SIP, BIT(riscv_pmu_irq_num));
 		return IRQ_NONE;
 	}
 
 	event = cpu_hw_evt->events[fidx];
 	if (!event) {
-		ALT_SBI_PMU_OVF_CLEAR_PENDING(riscv_pmu_irq_mask);
+		if (from_sse)
+			pmu_sbi_sse_stop_all(cmask);
+		else
+			ALT_SBI_PMU_OVF_CLEAR_PENDING(riscv_pmu_irq_mask);
 		return IRQ_NONE;
 	}
 
 	pmu = to_riscv_pmu(event->pmu);
 	pmu_sbi_stop_hw_ctrs(pmu);
+	if (unlikely(pmu_sbi_sse_failed(cpu_hw_evt)))
+		return IRQ_NONE;
 
 	/* Overflow status register should only be read after counter are stopped */
 	if (sbi_pmu_snapshot_available())
@@ -1120,15 +1253,17 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
 
 	/*
 	 * Overflow interrupt pending bit should only be cleared after stopping
-	 * all the counters to avoid any race condition.
+	 * all the counters to avoid any race condition. When using SSE,
+	 * interrupt is cleared when stopping counters.
 	 */
-	ALT_SBI_PMU_OVF_CLEAR_PENDING(riscv_pmu_irq_mask);
+	if (!from_sse)
+		ALT_SBI_PMU_OVF_CLEAR_PENDING(riscv_pmu_irq_mask);
 
 	/* No overflow bit is set */
-	if (!overflow)
+	if (!overflow) {
+		pmu_sbi_start_overflow_mask(pmu, 0);
 		return IRQ_NONE;
-
-	regs = get_irq_regs();
+	}
 
 	for_each_set_bit(lidx, cpu_hw_evt->used_hw_ctrs, RISCV_MAX_COUNTERS) {
 		struct perf_event *event = cpu_hw_evt->events[lidx];
@@ -1153,6 +1288,12 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
 		if (!(overflow & BIT_ULL(hidx)))
 			continue;
 
+#ifdef CONFIG_CPU_PM
+		/* Do not let CPU-PM resume override this overflow decision. */
+		if (from_sse)
+			clear_bit(lidx, cpu_hw_evt->pm_resume_hw_ctrs);
+#endif
+
 		/*
 		 * Keep a track of overflowed counters so that they can be started
 		 * with updated initial value.
@@ -1165,6 +1306,14 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
 		hw_evt->state |= PERF_HES_UPTODATE;
 		perf_sample_data_init(&data, 0, hw_evt->last_period);
 		if (riscv_pmu_event_set_period(event)) {
+			int overflow_ret;
+
+			/* Guest attribution and guest stack sampling are not supported yet. */
+			if (from_guest) {
+				hw_evt->state = 0;
+				continue;
+			}
+
 			/*
 			 * Unlike other ISAs, RISC-V don't have to disable interrupts
 			 * to avoid throttling here. As per the specification, the
@@ -1173,128 +1322,291 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
 			 * TODO: We will need to stop the guest counters once
 			 * virtualization support is added.
 			 */
-			perf_event_overflow(event, &data, regs);
+			overflow_ret = perf_event_overflow(event, &data, regs);
+			if (!overflow_ret)
+				hw_evt->state = 0;
+		} else {
+			hw_evt->state = 0;
 		}
-		/* Reset the state as we are going to start the counter after the loop */
-		hw_evt->state = 0;
 	}
 
 	pmu_sbi_start_overflow_mask(pmu, overflowed_ctrs);
+
 	perf_sample_event_took(sched_clock() - start_clock);
 
 	return IRQ_HANDLED;
 }
 
-static int pmu_sbi_starting_cpu(unsigned int cpu, struct hlist_node *node)
+static irqreturn_t pmu_sbi_ovf_irq_handler(int irq, void *dev)
 {
-	struct riscv_pmu *pmu = hlist_entry_safe(node, struct riscv_pmu, node);
-	struct cpu_hw_events *cpu_hw_evt = this_cpu_ptr(pmu->hw_events);
-
-	/*
-	 * We keep enabling userspace access to CYCLE, TIME and INSTRET via the
-	 * legacy option but that will be removed in the future.
-	 */
-	if (sysctl_perf_user_access == SYSCTL_LEGACY)
-		csr_write(CSR_SCOUNTEREN, 0x7);
-	else
-		csr_write(CSR_SCOUNTEREN, 0x2);
+	return pmu_sbi_ovf_handler(dev, get_irq_regs(), false, false);
+}
 
-	/* Stop all the counters so that they can be enabled from perf */
-	pmu_sbi_stop_all(pmu);
+#ifdef CONFIG_RISCV_PMU_SBI_SSE
+static int pmu_sbi_ovf_sse_handler(u32 evt, void *arg, struct pt_regs *regs)
+{
+	const struct riscv_sse_interrupted_context *context;
+	struct riscv_pmu *pmu = arg;
+	struct cpu_hw_events *hw_event = raw_cpu_ptr(pmu->hw_events);
+	bool from_guest;
+
+	if (unlikely(!READ_ONCE(pmu->sse_active))) {
+		pmu_sbi_sse_stop_all(pmu->cmask);
+		return -EIO;
+	}
 
-	if (riscv_pmu_use_irq) {
-		cpu_hw_evt->irq = riscv_pmu_irq;
-		ALT_SBI_PMU_OVF_CLEAR_PENDING(riscv_pmu_irq_mask);
-		enable_percpu_irq(riscv_pmu_irq, IRQ_TYPE_NONE);
+	if (unlikely(!regs)) {
+		pmu_sbi_fail_sse(pmu, "read interrupted context for", -EIO);
+		return -EIO;
 	}
 
-	if (sbi_pmu_snapshot_available())
-		return pmu_sbi_snapshot_setup(pmu, cpu);
+	context = riscv_sse_get_interrupted_context();
+	from_guest = context && context->regs == regs &&
+		     (context->hstatus & HSTATUS_SPV);
+	pmu_sbi_ovf_handler(hw_event, regs, true, from_guest);
 
 	return 0;
 }
 
-static int pmu_sbi_dying_cpu(unsigned int cpu, struct hlist_node *node)
+static int pmu_sbi_setup_sse(struct riscv_pmu *pmu)
 {
-	if (riscv_pmu_use_irq) {
-		disable_percpu_irq(riscv_pmu_irq);
-	}
+	int ret;
+	struct sse_event *evt;
 
-	/* Disable all counters access for user mode now */
-	csr_write(CSR_SCOUNTEREN, 0x0);
+	evt = sse_event_register(SBI_SSE_EVENT_LOCAL_PMU_OVERFLOW, 0,
+				 pmu_sbi_ovf_sse_handler, pmu);
+	if (IS_ERR(evt))
+		return PTR_ERR(evt);
+	pmu->sse_evt = evt;
 
-	if (sbi_pmu_snapshot_available())
-		return pmu_sbi_snapshot_disable();
+	ret = sse_event_enable(evt);
+	if (ret) {
+		int cleanup_ret;
 
-	return 0;
+		cleanup_ret = sse_event_disable(evt);
+		if (cleanup_ret) {
+			pr_warn("failed to disable SSE event after setup error: %d\n",
+				cleanup_ret);
+			return cleanup_ret;
+		}
+
+		cleanup_ret = sse_event_unregister(evt);
+
+		if (cleanup_ret) {
+			pr_warn("failed to unregister SSE event: %d\n",
+				cleanup_ret);
+		} else {
+			pmu->sse_evt = NULL;
+		}
+		return cleanup_ret ?: ret;
+	}
+
+	WRITE_ONCE(pmu->sse_active, true);
+	pr_info("using SSE for PMU event delivery\n");
+
+	return ret;
 }
 
-static int pmu_sbi_setup_irqs(struct riscv_pmu *pmu, struct platform_device *pdev)
+static void pmu_sbi_cleanup_sse(struct riscv_pmu *pmu)
 {
+	struct sse_event *sse_evt;
 	int ret;
-	struct cpu_hw_events __percpu *hw_events = pmu->hw_events;
-	struct irq_domain *domain = NULL;
 
+	sse_evt = pmu->sse_evt;
+	if (!sse_evt)
+		return;
+	/*
+	 * Close callback admission before draining each CPU. PMU callbacks run
+	 * with local interrupts disabled, so the synchronous IPI cannot complete
+	 * until a callback that observed the old state has returned.
+	 */
+	WRITE_ONCE(pmu->sse_active, false);
+	on_each_cpu(pmu_sbi_stop_all_cpu, pmu, 1);
+
+	ret = sse_event_disable(sse_evt);
+	if (ret) {
+		pr_warn("failed to disable SSE event: %d\n", ret);
+		goto retain;
+	}
+
+	ret = sse_event_unregister(sse_evt);
+	if (ret) {
+		pr_warn("failed to unregister SSE event: %d\n", ret);
+		goto retain;
+	}
+
+	pmu->sse_evt = NULL;
+	return;
+
+retain:
+	sse_event_cleanup(sse_evt);
+	pmu->sse_evt = NULL;
+}
+
+static bool pmu_sbi_sse_state_retained(struct riscv_pmu *pmu)
+{
+	return pmu->sse_evt;
+}
+#else
+static int pmu_sbi_setup_sse(struct riscv_pmu *pmu)
+{
+	return -EOPNOTSUPP;
+}
+
+static void pmu_sbi_cleanup_sse(struct riscv_pmu *pmu) {}
+
+static bool pmu_sbi_sse_state_retained(struct riscv_pmu *pmu)
+{
+	return false;
+}
+#endif
+
+static bool pmu_sbi_select_irq(void)
+{
 	if (riscv_isa_extension_available(NULL, SSCOFPMF)) {
 		riscv_pmu_irq_num = RV_IRQ_PMU;
-		riscv_pmu_use_irq = true;
+		return true;
 	} else if (IS_ENABLED(CONFIG_ERRATA_THEAD_PMU) &&
 		   riscv_cached_mvendorid(0) == THEAD_VENDOR_ID &&
 		   riscv_cached_marchid(0) == 0 &&
 		   riscv_cached_mimpid(0) == 0) {
 		riscv_pmu_irq_num = THEAD_C9XX_RV_IRQ_PMU;
-		riscv_pmu_use_irq = true;
+		return true;
 	} else if (riscv_has_vendor_extension_unlikely(ANDES_VENDOR_ID,
 						       RISCV_ISA_VENDOR_EXT_XANDESPMU) &&
 		   IS_ENABLED(CONFIG_ANDES_CUSTOM_PMU)) {
 		riscv_pmu_irq_num = ANDES_SLI_CAUSE_BASE + ANDES_RV_IRQ_PMOVI;
-		riscv_pmu_use_irq = true;
+		return true;
 	}
 
-	riscv_pmu_irq_mask = BIT(riscv_pmu_irq_num % BITS_PER_LONG);
+	return false;
+}
 
-	if (!riscv_pmu_use_irq)
+static int pmu_sbi_setup_irq(struct riscv_pmu *pmu)
+{
+	struct cpu_hw_events __percpu *hw_events = pmu->hw_events;
+	struct irq_domain *domain;
+	int ret;
+
+	if (!pmu_sbi_select_irq())
 		return -EOPNOTSUPP;
 
+	riscv_pmu_irq_mask = BIT(riscv_pmu_irq_num % BITS_PER_LONG);
+
 	domain = irq_find_matching_fwnode(riscv_get_intc_hwnode(),
 					  DOMAIN_BUS_ANY);
 	if (!domain) {
 		pr_err("Failed to find INTC IRQ root domain\n");
-		ret = -ENODEV;
-		goto err;
+		return -ENODEV;
 	}
 
 	riscv_pmu_irq = irq_create_mapping(domain, riscv_pmu_irq_num);
 	if (!riscv_pmu_irq) {
 		pr_err("Failed to map PMU interrupt for node\n");
-		ret = -ENODEV;
-		goto err;
+		return -ENODEV;
 	}
 
-	ret = request_percpu_irq(riscv_pmu_irq, pmu_sbi_ovf_handler, "riscv-pmu", hw_events);
+	ret = request_percpu_irq(riscv_pmu_irq, pmu_sbi_ovf_irq_handler,
+				 "riscv-pmu", hw_events);
 	if (ret) {
 		pr_err("registering percpu irq failed [%d]\n", ret);
 		irq_dispose_mapping(riscv_pmu_irq);
 		riscv_pmu_irq = 0;
-		goto err;
+		return ret;
 	}
 
 	return 0;
-err:
-	riscv_pmu_use_irq = false;
-	return ret;
+}
+
+static int pmu_sbi_starting_cpu(unsigned int cpu, struct hlist_node *node)
+{
+	struct riscv_pmu *pmu = hlist_entry_safe(node, struct riscv_pmu, node);
+	struct cpu_hw_events *cpu_hw_evt = this_cpu_ptr(pmu->hw_events);
+
+	/*
+	 * We keep enabling userspace access to CYCLE, TIME and INSTRET via the
+	 * legacy option but that will be removed in the future.
+	 */
+	if (sysctl_perf_user_access == SYSCTL_LEGACY)
+		csr_write(CSR_SCOUNTEREN, 0x7);
+	else
+		csr_write(CSR_SCOUNTEREN, 0x2);
+
+	/* Stop all the counters so that they can be enabled from perf */
+	pmu_sbi_stop_all(pmu);
+
+	if (riscv_pmu_use_irq) {
+		cpu_hw_evt->irq = riscv_pmu_irq;
+		ALT_SBI_PMU_OVF_CLEAR_PENDING(riscv_pmu_irq_mask);
+		enable_percpu_irq(riscv_pmu_irq, IRQ_TYPE_NONE);
+	}
+
+	if (sbi_pmu_snapshot_available())
+		return pmu_sbi_snapshot_setup(pmu, cpu);
+
+	return 0;
+}
+
+static int pmu_sbi_dying_cpu(unsigned int cpu, struct hlist_node *node)
+{
+	if (riscv_pmu_use_irq)
+		disable_percpu_irq(riscv_pmu_irq);
+
+	/* Disable all counters access for user mode now */
+	csr_write(CSR_SCOUNTEREN, 0x0);
+
+	if (sbi_pmu_snapshot_available())
+		return pmu_sbi_snapshot_disable();
+
+	return 0;
+}
+
+static int pmu_sbi_setup_irqs(struct riscv_pmu *pmu, struct platform_device *pdev)
+{
+	int irq_ret;
+	int sse_ret;
+
+	/* Do not claim an IRQ route while a crash kernel leaves SSE state intact. */
+	if (is_kdump_kernel() && riscv_sse_available()) {
+		pr_warn("PMU delivery unavailable with retained crash-kernel SSE state\n");
+		return -EUCLEAN;
+	}
+
+	sse_ret = pmu_sbi_setup_sse(pmu);
+	if (!sse_ret) {
+		riscv_pmu_use_irq = false;
+		return 0;
+	}
+	if (pmu_sbi_sse_state_retained(pmu)) {
+		pr_err("PMU-SSE setup failed with firmware state retained: %d\n",
+		       sse_ret);
+		return -EUCLEAN;
+	}
+	/* Only an explicitly unsupported SSE path proves IRQ fallback is safe. */
+	if (sse_ret != -EOPNOTSUPP) {
+		pr_err("PMU-SSE setup failed: %d\n", sse_ret);
+		return sse_ret;
+	}
+
+	irq_ret = pmu_sbi_setup_irq(pmu);
+	if (!irq_ret) {
+		riscv_pmu_use_irq = true;
+		return 0;
+	}
+	return irq_ret;
 }
 
 #ifdef CONFIG_CPU_PM
-static int riscv_pm_pmu_notify(struct notifier_block *b, unsigned long cmd,
-				void *v)
+static int riscv_pm_pmu_update(struct riscv_pmu *rvpmu, unsigned long cmd)
 {
-	struct riscv_pmu *rvpmu = container_of(b, struct riscv_pmu, riscv_pm_nb);
 	struct cpu_hw_events *cpuc = this_cpu_ptr(rvpmu->hw_events);
 	bool enabled = !bitmap_empty(cpuc->used_hw_ctrs, RISCV_MAX_COUNTERS);
 	struct perf_event *event;
 	int idx;
 
+	if (cmd == CPU_PM_ENTER)
+		bitmap_zero(cpuc->pm_resume_hw_ctrs, RISCV_MAX_COUNTERS);
+
 	if (!enabled)
 		return NOTIFY_OK;
 
@@ -1305,6 +1617,8 @@ static int riscv_pm_pmu_notify(struct notifier_block *b, unsigned long cmd,
 
 		switch (cmd) {
 		case CPU_PM_ENTER:
+			if (!(event->hw.state & PERF_HES_STOPPED))
+				set_bit(idx, cpuc->pm_resume_hw_ctrs);
 			/*
 			 * Stop and update the counter
 			 */
@@ -1312,6 +1626,8 @@ static int riscv_pm_pmu_notify(struct notifier_block *b, unsigned long cmd,
 			break;
 		case CPU_PM_EXIT:
 		case CPU_PM_ENTER_FAILED:
+			if (!test_and_clear_bit(idx, cpuc->pm_resume_hw_ctrs))
+				break;
 			/*
 			 * Restore and enable the counter.
 			 */
@@ -1325,9 +1641,59 @@ static int riscv_pm_pmu_notify(struct notifier_block *b, unsigned long cmd,
 	return NOTIFY_OK;
 }
 
+static int riscv_pm_pmu_notify(struct notifier_block *b,
+			       unsigned long cmd, void *v)
+{
+	struct riscv_pmu *rvpmu = container_of(b, struct riscv_pmu,
+						 riscv_pm_nb);
+
+#ifdef CONFIG_RISCV_PMU_SBI_SSE
+	struct cpu_hw_events *cpuc = this_cpu_ptr(rvpmu->hw_events);
+	int ret;
+
+	if (!riscv_pmu_use_irq && READ_ONCE(rvpmu->sse_active)) {
+		switch (cmd) {
+		case CPU_PM_ENTER:
+			cpuc->pm_resume_sse =
+				sse_event_is_enabled_local(rvpmu->sse_evt);
+			if (cpuc->pm_resume_sse) {
+				ret = sse_event_disable_local(rvpmu->sse_evt);
+				if (ret) {
+					cpuc->pm_resume_sse = false;
+					pmu_sbi_fail_sse(rvpmu, "disable for CPU PM",
+							 ret);
+					return notifier_from_errno(ret);
+				}
+			}
+			break;
+		case CPU_PM_EXIT:
+		case CPU_PM_ENTER_FAILED:
+			ret = riscv_pm_pmu_update(rvpmu, cmd);
+			if (!cpuc->pm_resume_sse)
+				return ret;
+
+			cpuc->pm_resume_sse = false;
+			ret = sse_event_enable_local(rvpmu->sse_evt);
+			if (ret) {
+				pmu_sbi_fail_sse(rvpmu, "enable after CPU PM", ret);
+				return notifier_from_errno(ret);
+			}
+			return NOTIFY_OK;
+		default:
+			break;
+		}
+	}
+#endif
+
+	return riscv_pm_pmu_update(rvpmu, cmd);
+}
+
 static int riscv_pm_pmu_register(struct riscv_pmu *pmu)
 {
 	pmu->riscv_pm_nb.notifier_call = riscv_pm_pmu_notify;
+	/* Keep PMU-SSE disabled until counters and userpage state are restored. */
+	pmu->riscv_pm_nb.priority = riscv_pmu_use_irq ? 0 : -1;
+
 	return cpu_pm_register_notifier(&pmu->riscv_pm_nb);
 }
 
@@ -1342,6 +1708,8 @@ static inline void riscv_pm_pmu_unregister(struct riscv_pmu *pmu) { }
 
 static void riscv_pmu_destroy(struct riscv_pmu *pmu)
 {
+	pmu_sbi_cleanup_sse(pmu);
+
 	if (sbi_v2_available) {
 		if (sbi_pmu_snapshot_available()) {
 			pmu_sbi_snapshot_disable();
@@ -1353,7 +1721,7 @@ static void riscv_pmu_destroy(struct riscv_pmu *pmu)
 		cpuhp_state_remove_instance(CPUHP_AP_PERF_RISCV_STARTING, &pmu->node);
 }
 
-static void pmu_sbi_event_init(struct perf_event *event)
+static int pmu_sbi_event_init(struct perf_event *event)
 {
 	/*
 	 * The permissions are set at event_init so that we do not depend
@@ -1365,6 +1733,8 @@ static void pmu_sbi_event_init(struct perf_event *event)
 		event->hw.flags |= PERF_EVENT_FLAG_USER_ACCESS;
 	else
 		event->hw.flags |= PERF_EVENT_FLAG_LEGACY;
+
+	return 0;
 }
 
 static void pmu_sbi_event_mapped(struct perf_event *event, struct mm_struct *mm)
@@ -1494,6 +1864,7 @@ static int pmu_sbi_device_probe(struct platform_device *pdev)
 	/* cache all the information about counters now */
 	if (pmu_sbi_get_ctrinfo(num_counters, cmask))
 		goto out_free;
+	bitmap_copy(pmu->cmask, cmask, RISCV_MAX_COUNTERS);
 
 	ret = pmu_sbi_setup_irqs(pmu, pdev);
 	if (ret < 0) {
@@ -1501,11 +1872,17 @@ static int pmu_sbi_device_probe(struct platform_device *pdev)
 		pmu->pmu.capabilities |= PERF_PMU_CAP_NO_INTERRUPT;
 		pmu->pmu.capabilities |= PERF_PMU_CAP_NO_EXCLUDE;
 	}
-	irq_requested = (ret == 0);
+	irq_requested = !ret && riscv_pmu_use_irq;
+
+#ifdef CONFIG_RISCV_PMU_SBI_SSE
+	if (pmu->sse_active) {
+		pmu->pmu.pmu_enable = pmu_sbi_sse_enable;
+		pmu->pmu.pmu_disable = pmu_sbi_sse_disable;
+	}
+#endif
 
 	pmu->pmu.attr_groups = riscv_pmu_attr_groups;
 	pmu->pmu.parent = &pdev->dev;
-	bitmap_copy(pmu->cmask, cmask, RISCV_MAX_COUNTERS);
 	pmu->ctr_start = pmu_sbi_ctr_start;
 	pmu->ctr_stop = pmu_sbi_ctr_stop;
 	pmu->event_map = pmu_sbi_event_map;
diff --git a/include/linux/perf/riscv_pmu.h b/include/linux/perf/riscv_pmu.h
index ecaa4037083..b06abc70510 100644
--- a/include/linux/perf/riscv_pmu.h
+++ b/include/linux/perf/riscv_pmu.h
@@ -28,6 +28,8 @@
 
 #define RISCV_PMU_CONFIG1_GUEST_EVENTS 0x1
 
+struct sse_event;
+
 struct cpu_hw_events {
 	/* currently enabled events */
 	int			n_events;
@@ -39,6 +41,18 @@ struct cpu_hw_events {
 	DECLARE_BITMAP(used_hw_ctrs, RISCV_MAX_COUNTERS);
 	/* currently enabled firmware counters */
 	DECLARE_BITMAP(used_fw_ctrs, RISCV_MAX_COUNTERS);
+#ifdef CONFIG_RISCV_PMU_SBI_SSE
+	/* Keep counters stopped after an unrecoverable SSE transition failure. */
+	bool sse_failed;
+#endif
+#ifdef CONFIG_CPU_PM
+	/* Counters stopped by CPU PM and still waiting to be restored. */
+	DECLARE_BITMAP(pm_resume_hw_ctrs, RISCV_MAX_COUNTERS);
+#ifdef CONFIG_RISCV_PMU_SBI_SSE
+	/* Restore the local PMU SSE event after counters and userpage state. */
+	bool pm_resume_sse;
+#endif
+#endif
 	/* The virtual address of the shared memory where counter snapshot will be taken */
 	void *snapshot_addr;
 	/* The physical address of the shared memory where counter snapshot will be taken */
@@ -54,6 +68,10 @@ struct riscv_pmu {
 	char		*name;
 
 	irqreturn_t	(*handle_irq)(int irq_num, void *dev);
+#ifdef CONFIG_RISCV_PMU_SBI_SSE
+	struct sse_event	*sse_evt;
+	bool			sse_active;
+#endif
 
 	DECLARE_BITMAP(cmask, RISCV_MAX_COUNTERS);
 	u64		(*ctr_read)(struct perf_event *event);
@@ -63,7 +81,7 @@ struct riscv_pmu {
 	void		(*ctr_start)(struct perf_event *event, u64 init_val);
 	void		(*ctr_stop)(struct perf_event *event, unsigned long flag);
 	int		(*event_map)(struct perf_event *event, u64 *config);
-	void		(*event_init)(struct perf_event *event);
+	int		(*event_init)(struct perf_event *event);
 	void		(*event_mapped)(struct perf_event *event, struct mm_struct *mm);
 	void		(*event_unmapped)(struct perf_event *event, struct mm_struct *mm);
 	uint8_t		(*csr_index)(struct perf_event *event);
diff --git a/include/linux/riscv_sbi_sse.h b/include/linux/riscv_sbi_sse.h
index 774a782e556..f6b3881c888 100644
--- a/include/linux/riscv_sbi_sse.h
+++ b/include/linux/riscv_sbi_sse.h
@@ -42,6 +42,7 @@ int sse_event_enable(struct sse_event *sse_evt);
 int sse_event_disable(struct sse_event *sse_evt);
 
 /* Local events require the caller to remain on the current CPU. */
+bool sse_event_is_enabled_local(struct sse_event *sse_evt);
 int sse_event_enable_local(struct sse_event *sse_evt);
 int sse_event_disable_local(struct sse_event *sse_evt);
 
@@ -76,6 +77,11 @@ static inline int sse_event_disable(struct sse_event *sse_evt)
 	return -EOPNOTSUPP;
 }
 
+static inline bool sse_event_is_enabled_local(struct sse_event *sse_evt)
+{
+	return false;
+}
+
 static inline int sse_event_enable_local(struct sse_event *sse_evt)
 {
 	return -EOPNOTSUPP;
-- 
2.50.1 (Apple Git-155)

Re: [PATCH v12 08/11] perf: RISC-V: add support for SSE event

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 8 potential issue(s) to consider:
- [High] perf: RISC-V: unhandled PERF_HES_STOPPED counters in pmu_sbi_ovf_handler()
- [High] perf: RISC-V: unstopped PMU counters in pmu_sbi_ovf_sse_handler() when regs is NULL
- [Medium] perf: riscv: missing sse_event_cleanup() when pmu_sbi_setup_sse() fails
- [Medium] perf: RISC-V: batch counter stop in pmu_sbi_cleanup_sse()
- [Medium] perf: RISC-V: pmu_sbi_fail_sse() clears SCOUNTEREN without updating userpage
- [Medium] perf: RISC-V: pmu_sbi_sse_enable() latches sse_failed on hotplug and reboot
- [Medium] perf: RISC-V: riscv_pm_pmu_notify() ignores latched cpuc->sse_failed state
- [Medium] perf: riscv: lost PMU SSE resume state after CPU_PM_EXIT unmask failure
--

commit 97de0c9d3e465757bf9e40945610083784df9d4b
Author: Clément Léger <cleger@rivosinc.com>

perf: RISC-V: add support for SSE event

This commit adds support for delivering RISC-V SBI PMU counter overflow
events via Supervisor Software Events (SSE), including probe-time delivery
selection, overflow handling, error latching, and CPU power management
integration.

Link: https://lore.kernel.org/r/20261002-reach-cloning-f1594729dbe3@wendy/

> diff --git a/drivers/perf/riscv_pmu_sbi.c b/drivers/perf/riscv_pmu_sbi.c
> index 21a2078eab98a..a984ac57a7b6e 100644
> --- a/drivers/perf/riscv_pmu_sbi.c
> +++ b/drivers/perf/riscv_pmu_sbi.c

[ ... ]

> @@ -936,14 +938,42 @@ static inline void pmu_sbi_stop_all(struct riscv_pmu *pmu)
>  	 * which may include counters that are not enabled yet.
>  	 */
>  	for (i = 0; i < BITS_TO_LONGS(RISCV_MAX_COUNTERS); i++) {
> -		if (!pmu->cmask[i])
> +		if (!ctr_mask[i])
>  			continue;
>  		sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_STOP,
> -			  i * BITS_PER_LONG, pmu->cmask[i],
> +			  i * BITS_PER_LONG, ctr_mask[i],
>  			  SBI_PMU_STOP_FLAG_RESET);
>  	}
>  }
>  
> +static void pmu_sbi_sse_stop_all(const unsigned long *ctr_mask)
> +{
> +	struct sbiret ret;
> +	int i;
> +
> +	/*
> +	 * Stop each counter separately: an already-stopped counter must not
> +	 * prevent firmware from stopping a later, overflowing counter.
> +	 */
> +	for_each_set_bit(i, ctr_mask, RISCV_MAX_COUNTERS) {
> +		ret = sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_STOP,
> +				i, 1, SBI_PMU_STOP_FLAG_RESET);
> +		if (ret.error && ret.error != SBI_ERR_ALREADY_STOPPED)
> +			pr_warn_ratelimited("failed to stop counter %d: %ld\n",
> +					    i, ret.error);
> +	}
> +}
> +
> +static inline void pmu_sbi_stop_all(struct riscv_pmu *pmu)
> +{
> +	pmu_sbi_stop_all_mask(pmu->cmask);
> +}
> +
> +static void pmu_sbi_stop_all_cpu(void *info)
> +{
> +	pmu_sbi_stop_all(info);
> +}

[ ... ]

> @@ -1054,32 +1098,111 @@ static inline void pmu_sbi_start_ovf_ctrs_snapshot(struct cpu_hw_events *cpu_hw_

[ ... ]

> +#ifdef CONFIG_RISCV_PMU_SBI_SSE
> +/*
> + * A local SSE delivery failure makes the current PMU state unsafe to resume.
> + * Latch the failure before stopping mapped events so the SSE transition and
> + * overflow restart paths cannot undo the fail-safe while they are quiesced.
> + */
> +static void pmu_sbi_fail_sse(struct riscv_pmu *pmu, const char *op, int ret)
> +{
> +	struct cpu_hw_events *cpu_hw_evt = this_cpu_ptr(pmu->hw_events);
> +	struct perf_event *event;
> +	int idx;
> +
> +	if (READ_ONCE(cpu_hw_evt->sse_failed))
> +		return;
> +
> +	WRITE_ONCE(cpu_hw_evt->sse_failed, true);
> +	pr_err_ratelimited("failed to %s local PMU SSE event: %d; stopping counters\n",
> +			   op, ret);
> +
> +	for (idx = 0; idx < RISCV_MAX_COUNTERS; idx++) {
> +		event = cpu_hw_evt->events[idx];
> +		if (event)
> +			riscv_pmu_stop(event, PERF_EF_UPDATE);
> +	}
> +}

[Severity: Medium]
When pmu_sbi_fail_sse() stops mapped events via riscv_pmu_stop(), can a
userspace task with a direct-mapped counter page subsequently fault with
SIGILL?

When an event is started in riscv_pmu_start() (for example during
event_sched_in() or riscv_pm_pmu_update() on CPU_PM_EXIT), pmu_sbi_ctr_start()
sets the counter bit in CSR_SCOUNTEREN and perf_event_update_userpage() sets
a non-zero userpg->index.

If a subsequent SSE call fails and invokes pmu_sbi_fail_sse(), it calls
riscv_pmu_stop() -> pmu_sbi_ctr_stop() -> pmu_sbi_reset_scounteren(), which
clears the counter bit in CSR_SCOUNTEREN and sets PERF_HES_STOPPED:

drivers/perf/riscv_pmu_sbi.c:pmu_sbi_ctr_stop() {
    ...
	if ((hwc->flags & PERF_EVENT_FLAG_USER_ACCESS) &&
	    (hwc->flags & PERF_EVENT_FLAG_USER_READ_CNT))
		pmu_sbi_reset_scounteren((void *)event);
    ...
}

Because neither riscv_pmu_stop() nor pmu_sbi_fail_sse() calls
perf_event_update_userpage(event), userpg->index remains non-zero while
CSR_SCOUNTEREN has the counter bit cleared, and cpu_hw_evt->sse_failed
prevents riscv_pmu_start() from restarting the event. Should
pmu_sbi_fail_sse() call perf_event_update_userpage(event) after
riscv_pmu_stop() so that userpg->index is cleared and userpg->offset is
updated?

> +
> +static void pmu_sbi_sse_disable(struct pmu *pmu)
> +{
> +	struct riscv_pmu *rvpmu = to_riscv_pmu(pmu);
> +	struct cpu_hw_events *cpu_hw_evt = this_cpu_ptr(rvpmu->hw_events);
> +	int ret;
> +
> +	if (!READ_ONCE(rvpmu->sse_active) ||
> +	    READ_ONCE(cpu_hw_evt->sse_failed))
> +		return;
> +
> +	ret = sse_event_disable_local(rvpmu->sse_evt);
> +	if (unlikely(ret))
> +		pmu_sbi_fail_sse(rvpmu, "disable", ret);
> +}
> +
> +static void pmu_sbi_sse_enable(struct pmu *pmu)
> +{
> +	struct riscv_pmu *rvpmu = to_riscv_pmu(pmu);
> +	struct cpu_hw_events *cpu_hw_evt = this_cpu_ptr(rvpmu->hw_events);
> +	int ret;
> +
> +	if (!READ_ONCE(rvpmu->sse_active) ||
> +	    READ_ONCE(cpu_hw_evt->sse_failed))
> +		return;
> +
> +	ret = sse_event_enable_local(rvpmu->sse_evt);
> +	if (unlikely(ret))
> +		pmu_sbi_fail_sse(rvpmu, "enable", ret);
> +}

[Severity: Medium]
Can pmu_sbi_sse_enable() permanently latch cpu_hw_evt->sse_failed during CPU
hotplug or system reboot?

In drivers/firmware/riscv/riscv_sbi_sse.c, sse_event_enable_local() returns
-ESHUTDOWN once sse_reboot_notifier sets sse_shutting_down, and returns
-EINVAL when reg_evt->is_registered is false on the local CPU:

drivers/firmware/riscv/riscv_sbi_sse.c:sse_event_enable_local() {
    ...
	if (READ_ONCE(sse_shutting_down)) {
		ret = -ESHUTDOWN;
		goto out;
	}

	if (!reg_evt->is_registered) {
		ret = -EINVAL;
		goto out;
	}
    ...
}

During CPU hotplug, CPUHP_AP_RISCV_SSE_ONLINE is in the threaded online
section after CPUHP_AP_ONLINE_IDLE, CPUHP_AP_SCHED_WAIT_EMPTY, and
CPUHP_AP_SMPBOOT_THREADS. On cpu_down, sse_cpu_teardown() sets
reg_evt->is_registered = false before sched_cpu_wait_empty() and
smpboot_park_threads() finish running tasks on that CPU; on cpu_up,
cpuhp/%u and unparked smpboot threads run before sse_cpu_online() registers
the local SSE event. Any context switch of a task with a hardware perf event
in that window calls perf_event_context_sched_out() or
perf_event_context_sched_in() -> perf_pmu_enable() -> pmu_sbi_sse_enable()
-> sse_event_enable_local(), which returns -EINVAL and causes
pmu_sbi_fail_sse() to latch cpu_hw_evt->sse_failed = true (which is never
cleared on a subsequent cpu_up).

Similarly, on reboot or kexec, sse_reboot_nb (priority 0) runs before
perf_reboot_notifier (priority INT_MIN), so perf_event_exit_cpu() triggers
pmu_sbi_sse_enable() after sse_shutting_down is true, failing with
-ESHUTDOWN.

Should pmu_sbi_sse_enable() avoid latching sse_failed when the local SSE
event is not currently registered during hotplug or when SSE is shutting
down?

[ ... ]

> @@ -1153,6 +1288,12 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
>  		if (!(overflow & BIT_ULL(hidx)))
>  			continue;
>  
> +#ifdef CONFIG_CPU_PM
> +		/* Do not let CPU-PM resume override this overflow decision. */
> +		if (from_sse)
> +			clear_bit(lidx, cpu_hw_evt->pm_resume_hw_ctrs);
> +#endif
> +
>  		/*
>  		 * Keep a track of overflowed counters so that they can be started
>  		 * with updated initial value.
>  		 */
> @@ -1165,6 +1306,14 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
>  		hw_evt->state |= PERF_HES_UPTODATE;
>  		perf_sample_data_init(&data, 0, hw_evt->last_period);
>  		if (riscv_pmu_event_set_period(event)) {
> +			int overflow_ret;
> +
> +			/* Guest attribution and guest stack sampling are not supported yet. */
> +			if (from_guest) {
> +				hw_evt->state = 0;
> +				continue;
> +			}
> +
>  			/*
>  			 * Unlike other ISAs, RISC-V don't have to disable interrupts
>  			 * to avoid throttling here. As per the specification, the
> @@ -1173,128 +1322,291 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
>  			 * TODO: We will need to stop the guest counters once
>  			 * virtualization support is added.
>  			 */
> -			perf_event_overflow(event, &data, regs);
> +			overflow_ret = perf_event_overflow(event, &data, regs);
> +			if (!overflow_ret)
> +				hw_evt->state = 0;
> +		} else {
> +			hw_evt->state = 0;
>  		}
> -		/* Reset the state as we are going to start the counter after the loop */
> -		hw_evt->state = 0;
>  	}

[Severity: High]
When perf_event_overflow() returns non-zero and leaves an event with
PERF_HES_STOPPED set while its bit remains set in cpu_hw_evt->used_hw_ctrs,
what happens when a second active counter on the same CPU overflows?

First, pmu_sbi_ovf_handler() calls pmu_sbi_stop_hw_ctrs(), which passes the
unfiltered cpu_hw_evt->used_hw_ctrs[i] mask to a batch
SBI_EXT_PMU_COUNTER_STOP ecall:

drivers/perf/riscv_pmu_sbi.c:pmu_sbi_stop_hw_ctrs() {
    ...
	for (i = 0; i < BITS_TO_LONGS(RISCV_MAX_COUNTERS); i++) {
		/* No need to check the error here as we can't do anything about the error */
		ret = sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_STOP, i * BITS_PER_LONG,
				cpu_hw_evt->used_hw_ctrs[i], flag);
		if (!ret.error && sbi_pmu_snapshot_available()) {
			/* Save the counter values to avoid clobbering */
			for_each_set_bit(idx, &cpu_hw_evt->used_hw_ctrs[i], BITS_PER_LONG)
				cpu_hw_evt->snapshot_cval_shcopy[i * BITS_PER_LONG + idx] =
							sdata->ctr_values[idx];
    ...
}

Because the throttled counter is already stopped in firmware, the batch stop
can return SBI_ERR_ALREADY_STOPPED (or stop early before higher-indexed
counters). When sbi_pmu_snapshot_available() is true, ret.error != 0 skips
copying sdata->ctr_values into snapshot_cval_shcopy after memset() zeroed
snapshot_cval_shcopy, and then overwrites sdata->ctr_values with 0 for all
used_hw_ctrs.

Second, in the overflow loop of pmu_sbi_ovf_handler(), events that already
have PERF_HES_STOPPED set are not skipped:

drivers/perf/riscv_pmu_sbi.c:pmu_sbi_ovf_handler() {
    ...
	for_each_set_bit(lidx, cpu_hw_evt->used_hw_ctrs, RISCV_MAX_COUNTERS) {
		struct perf_event *event = cpu_hw_evt->events[lidx];

		/* Skip if invalid event or user did not request a sampling */
		if (!event || !is_sampling_event(event))
			continue;
    ...
}

Without CSR_SCOUNTOVF (MHPMEVENT_OF) having been cleared by a counter start,
overflow & BIT_ULL(hidx) remains set for the previously stopped counter. On
the subsequent overflow of another counter, pmu_sbi_ovf_handler() visits the
throttled event again: riscv_pmu_event_update() is a no-op because
PERF_HES_UPTODATE is still set, and riscv_pmu_event_set_period() returns 0
because period_left was already made positive on the initial overflow. That
takes the else branch (or the from_guest branch) and clears hw_evt->state =
0, causing pmu_sbi_start_overflow_mask() to prematurely restart the throttled
counter.

Should pmu_sbi_stop_hw_ctrs() and the loop in pmu_sbi_ovf_handler() skip
counters whose event already has PERF_HES_STOPPED set?

>  
>  	pmu_sbi_start_overflow_mask(pmu, overflowed_ctrs);
> +
>  	perf_sample_event_took(sched_clock() - start_clock);
>  
>  	return IRQ_HANDLED;
>  }
>  
> -static int pmu_sbi_starting_cpu(unsigned int cpu, struct hlist_node *node)
> +static irqreturn_t pmu_sbi_ovf_irq_handler(int irq, void *dev)
>  {
> -	struct riscv_pmu *pmu = hlist_entry_safe(node, struct riscv_pmu, node);
> -	struct cpu_hw_events *cpu_hw_evt = this_cpu_ptr(pmu->hw_events);
> -
> -	/*
> -	 * We keep enabling userspace access to CYCLE, TIME and INSTRET via the
> -	 * legacy option but that will be removed in the future.
> -	 */
> -	if (sysctl_perf_user_access == SYSCTL_LEGACY)
> -		csr_write(CSR_SCOUNTEREN, 0x7);
> -	else
> -		csr_write(CSR_SCOUNTEREN, 0x2);
> +	return pmu_sbi_ovf_handler(dev, get_irq_regs(), false, false);
> +}
>  
> -	/* Stop all the counters so that they can be enabled from perf */
> -	pmu_sbi_stop_all(pmu);
> +#ifdef CONFIG_RISCV_PMU_SBI_SSE
> +static int pmu_sbi_ovf_sse_handler(u32 evt, void *arg, struct pt_regs *regs)
> +{
> +	const struct riscv_sse_interrupted_context *context;
> +	struct riscv_pmu *pmu = arg;
> +	struct cpu_hw_events *hw_event = raw_cpu_ptr(pmu->hw_events);
> +	bool from_guest;
> +
> +	if (unlikely(!READ_ONCE(pmu->sse_active))) {
> +		pmu_sbi_sse_stop_all(pmu->cmask);
> +		return -EIO;
> +	}
>  
> -	if (riscv_pmu_use_irq) {
> -		cpu_hw_evt->irq = riscv_pmu_irq;
> -		ALT_SBI_PMU_OVF_CLEAR_PENDING(riscv_pmu_irq_mask);
> -		enable_percpu_irq(riscv_pmu_irq, IRQ_TYPE_NONE);
> +	if (unlikely(!regs)) {
> +		pmu_sbi_fail_sse(pmu, "read interrupted context for", -EIO);
> +		return -EIO;
>  	}

[Severity: High]
Can the !regs path in pmu_sbi_ovf_sse_handler() return without stopping the
overflowing counter source, leading to an infinite SSE re-delivery loop when
firmware executes SBI_SSE_EVENT_COMPLETE?

When regs is NULL, pmu_sbi_ovf_sse_handler() bypasses pmu_sbi_ovf_handler()
and relies only on pmu_sbi_fail_sse(). However, there are two cases where
pmu_sbi_fail_sse() does not stop the counter source while
pmu_sbi_ovf_handler() would have:

1. An SSE overflow arrives after pmu_sbi_setup_sse() sets pmu->sse_active =
   true (before pmu_sbi_starting_cpu() stops counters or before perf installs
   an event in cpu_hw_evt->events[]). In pmu_sbi_ovf_handler(), empty
   used_hw_ctrs or a NULL event calls pmu_sbi_sse_stop_all(cmask), whereas
   pmu_sbi_fail_sse() only loops over non-NULL cpu_hw_evt->events[idx] and
   stops nothing.

2. In pmu_sbi_fail_sse(), cpu_hw_evt->sse_failed is set to true before
   pr_err_ratelimited() and before the loop that stops mapped events (for
   example, when sse_event_disable_local() fails in pmu_sbi_sse_disable()). If
   an SSE overflow preempts that path with regs == NULL, pmu_sbi_fail_sse()
   sees READ_ONCE(cpu_hw_evt->sse_failed) == true and returns immediately
   without stopping the overflowing counter, whereas pmu_sbi_ovf_handler()
   calls pmu_sbi_stop_hw_ctrs(pmu) before checking pmu_sbi_sse_failed().

Should the !regs path (or pmu_sbi_fail_sse()) also ensure that unmapped or
still-running hardware counters are stopped before returning from the SSE
handler?

>  
> -	if (sbi_pmu_snapshot_available())
> -		return pmu_sbi_snapshot_setup(pmu, cpu);
> +	context = riscv_sse_get_interrupted_context();
> +	from_guest = context && context->regs == regs &&
> +		     (context->hstatus & HSTATUS_SPV);
> +	pmu_sbi_ovf_handler(hw_event, regs, true, from_guest);
>  
>  	return 0;
>  }
>  
> -static int pmu_sbi_dying_cpu(unsigned int cpu, struct hlist_node *node)
> +static int pmu_sbi_setup_sse(struct riscv_pmu *pmu)
>  {
> -	if (riscv_pmu_use_irq) {
> -		disable_percpu_irq(riscv_pmu_irq);
> -	}
> +	int ret;
> +	struct sse_event *evt;
>  
> -	/* Disable all counters access for user mode now */
> -	csr_write(CSR_SCOUNTEREN, 0x0);
> +	evt = sse_event_register(SBI_SSE_EVENT_LOCAL_PMU_OVERFLOW, 0,
> +				 pmu_sbi_ovf_sse_handler, pmu);
> +	if (IS_ERR(evt))
> +		return PTR_ERR(evt);
> +	pmu->sse_evt = evt;
>  
> -	if (sbi_pmu_snapshot_available())
> -		return pmu_sbi_snapshot_disable();
> +	ret = sse_event_enable(evt);
> +	if (ret) {
> +		int cleanup_ret;
>  
> -	return 0;
> +		cleanup_ret = sse_event_disable(evt);
> +		if (cleanup_ret) {
> +			pr_warn("failed to disable SSE event after setup error: %d\n",
> +				cleanup_ret);
> +			return cleanup_ret;
> +		}
> +
> +		cleanup_ret = sse_event_unregister(evt);
> +
> +		if (cleanup_ret) {
> +			pr_warn("failed to unregister SSE event: %d\n",
> +				cleanup_ret);
> +		} else {
> +			pmu->sse_evt = NULL;
> +		}
> +		return cleanup_ret ?: ret;
> +	}

[Severity: Medium]
If sse_event_enable(evt) fails in pmu_sbi_setup_sse() and then
sse_event_disable(evt) or sse_event_unregister(evt) also fails, should this
error path hand evt over to sse_event_cleanup(evt)?

When pmu->sse_evt is retained here, pmu_sbi_setup_irqs() sees
pmu_sbi_sse_state_retained(pmu) and returns -EUCLEAN, and
pmu_sbi_device_probe() continues in counting-only mode:

drivers/perf/riscv_pmu_sbi.c:pmu_sbi_device_probe() {
    ...
	ret = pmu_sbi_setup_irqs(pmu, pdev);
	if (ret < 0) {
		pr_info("Perf sampling/filtering is not supported as sscof extension is not available\n");
		pmu->pmu.capabilities |= PERF_PMU_CAP_NO_INTERRUPT;
		pmu->pmu.capabilities |= PERF_PMU_CAP_NO_EXCLUDE;
	}
    ...
}

Because probe succeeds without calling riscv_pmu_destroy() ->
pmu_sbi_cleanup_sse(), sse_event_cleanup(evt) is never called on the retained
event. With evt->cleanup_pending left false, the SSE core's hotplug and PM
callbacks (sse_cpu_online() and sse_restore_local_events()) treat evt as an
active registration and re-register it on CPUs instead of cleaning it up, and
if the event remained enabled on any CPU, a counting-event wrap will invoke
pmu_sbi_ovf_sse_handler() with pmu->sse_active == false and reset active
counters via pmu_sbi_sse_stop_all(pmu->cmask).

> +
> +	WRITE_ONCE(pmu->sse_active, true);
> +	pr_info("using SSE for PMU event delivery\n");
> +
> +	return ret;
>  }
>  
> -static int pmu_sbi_setup_irqs(struct riscv_pmu *pmu, struct platform_device *pdev)
> +static void pmu_sbi_cleanup_sse(struct riscv_pmu *pmu)
>  {
> +	struct sse_event *sse_evt;
>  	int ret;
> -	struct cpu_hw_events __percpu *hw_events = pmu->hw_events;
> -	struct irq_domain *domain = NULL;
>  
> +	sse_evt = pmu->sse_evt;
> +	if (!sse_evt)
> +		return;
> +	/*
> +	 * Close callback admission before draining each CPU. PMU callbacks run
> +	 * with local interrupts disabled, so the synchronous IPI cannot complete
> +	 * until a callback that observed the old state has returned.
> +	 */
> +	WRITE_ONCE(pmu->sse_active, false);
> +	on_each_cpu(pmu_sbi_stop_all_cpu, pmu, 1);

[Severity: Medium]
Should pmu_sbi_cleanup_sse() stop counters on each CPU using
pmu_sbi_sse_stop_all(pmu->cmask) instead of pmu_sbi_stop_all_cpu()?

pmu_sbi_stop_all_cpu() calls pmu_sbi_stop_all() ->
pmu_sbi_stop_all_mask(pmu->cmask), which issues a batch
SBI_EXT_PMU_COUNTER_STOP per word. As noted in the commit message, a batch
stop can return SBI_ERR_ALREADY_STOPPED when encountering an already-stopped
counter before stopping higher-numbered counters in pmu->cmask. If
sse_event_disable() or sse_event_unregister() subsequently fails and jumps to
retain, sse_event_cleanup(sse_evt) replaces the PMU SSE handler with the
no-op cleanup handler while a higher-numbered counter could still be running.

[ ... ]

> @@ -1325,9 +1641,59 @@ static int riscv_pm_pmu_notify(struct notifier_block *b, unsigned long cmd,
>  	return NOTIFY_OK;
>  }
>  
> +static int riscv_pm_pmu_notify(struct notifier_block *b,
> +			       unsigned long cmd, void *v)
> +{
> +	struct riscv_pmu *rvpmu = container_of(b, struct riscv_pmu,
> +						 riscv_pm_nb);
> +
> +#ifdef CONFIG_RISCV_PMU_SBI_SSE
> +	struct cpu_hw_events *cpuc = this_cpu_ptr(rvpmu->hw_events);
> +	int ret;
> +
> +	if (!riscv_pmu_use_irq && READ_ONCE(rvpmu->sse_active)) {
> +		switch (cmd) {
> +		case CPU_PM_ENTER:
> +			cpuc->pm_resume_sse =
> +				sse_event_is_enabled_local(rvpmu->sse_evt);
> +			if (cpuc->pm_resume_sse) {
> +				ret = sse_event_disable_local(rvpmu->sse_evt);
> +				if (ret) {
> +					cpuc->pm_resume_sse = false;
> +					pmu_sbi_fail_sse(rvpmu, "disable for CPU PM",
> +							 ret);
> +					return notifier_from_errno(ret);
> +				}
> +			}
> +			break;
> +		case CPU_PM_EXIT:
> +		case CPU_PM_ENTER_FAILED:
> +			ret = riscv_pm_pmu_update(rvpmu, cmd);
> +			if (!cpuc->pm_resume_sse)
> +				return ret;
> +
> +			cpuc->pm_resume_sse = false;
> +			ret = sse_event_enable_local(rvpmu->sse_evt);
> +			if (ret) {
> +				pmu_sbi_fail_sse(rvpmu, "enable after CPU PM", ret);
> +				return notifier_from_errno(ret);
> +			}
> +			return NOTIFY_OK;
> +		default:
> +			break;
> +		}
> +	}
> +#endif

[Severity: Medium]
Should riscv_pm_pmu_notify() check READ_ONCE(cpuc->sse_failed) before running
the local SSE disable and enable transitions?

If cpuc->sse_failed was latched while the local SSE event remained enabled
(for instance when pmu_sbi_ovf_sse_handler() failed with !regs, or when
sse_event_disable_local() failed in pmu_sbi_sse_disable()), a later
CPU_PM_ENTER will see sse_event_is_enabled_local(rvpmu->sse_evt) == true. If
sse_event_disable_local() then succeeds on CPU_PM_ENTER, cpuc->pm_resume_sse
is set to true and CPU_PM_EXIT will re-enable the local SSE event via
sse_event_enable_local(rvpmu->sse_evt) even though cpuc->sse_failed is latched
(or if sse_event_disable_local() keeps failing, every CPU_PM_ENTER will
return notifier_from_errno(ret) and abort idle entry).

[Severity: Medium]
Also in riscv_pm_pmu_notify() and riscv_pm_pmu_update(), what happens on the
next CPU_PM_ENTER if sse_events_unmask() failed during the preceding
CPU_PM_EXIT?

On CPU_PM_EXIT, sse_pm_notifier() (priority 0) runs before
riscv_pm_pmu_notify() (priority -1). If sse_events_unmask() fails,
sse_pm_notifier() returns notifier_from_errno(ret) with NOTIFY_STOP_MASK,
stopping the notifier chain before riscv_pm_pmu_notify() runs. At that point,
rvpmu->sse_evt is still disabled (is_enabled == false) and the counters
stopped during the prior CPU_PM_ENTER still have PERF_HES_STOPPED set.

When the CPU enters idle again on the next CPU_PM_ENTER,
riscv_pm_pmu_notify() overwrites cpuc->pm_resume_sse with
sse_event_is_enabled_local(rvpmu->sse_evt) (which is now false), and
riscv_pm_pmu_update() zeroes cpuc->pm_resume_hw_ctrs and only records
counters without PERF_HES_STOPPED:

drivers/perf/riscv_pmu_sbi.c:riscv_pm_pmu_update() {
    ...
	if (cmd == CPU_PM_ENTER)
		bitmap_zero(cpuc->pm_resume_hw_ctrs, RISCV_MAX_COUNTERS);
    ...
		case CPU_PM_ENTER:
			if (!(event->hw.state & PERF_HES_STOPPED))
				set_bit(idx, cpuc->pm_resume_hw_ctrs);
    ...
}

Won't that lose both cpuc->pm_resume_sse and cpuc->pm_resume_hw_ctrs, leaving
the counters stopped and the local PMU SSE event disabled on subsequent
CPU_PM_EXIT notifications?

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261029074853.35467-1-zhangzhanpeng.jasper@bytedance.com?part=8

Re: [PATCH v12 00/11] riscv: add SBI Supervisor Software Events support

[Himanshu Chauhan]

Hi Zhanpeng,

> [PATCH v12 00/11] riscv: add SBI Supervisor Software Events support

I See only 8 patches out of 11.

Regards
Himanshu

Re: [External] Re: [PATCH v12 00/11] riscv: add SBI Supervisor Software Events support

[Zhanpeng Zhang]

Hi Himanshu,

On 10/29/26 7:38 PM, Himanshu Chauhan wrote:
> Hi Zhanpeng,
>
>> [PATCH v12 00/11] riscv: add SBI Supervisor Software Events support
>
> I See only 8 patches out of 11.
Oops, I ran into a Gmail smtp system error. I've resent the last 3 patches.

Thanks,
Zhanpeng

[PATCH v12 09/11] MAINTAINERS: add RISC-V SSE maintainers and file coverage

[Zhanpeng Zhang]

Add the RISC-V SSE maintainers and cover the architecture entry,
firmware driver, PMU documentation and selftests in a separate patch,
as requested by Himanshu Chauhan. Also add coverage for RISC-V firmware
drivers and the architecture selftests.

Link: https://lore.kernel.org/r/as79MHtNYd-VbUug@hu-himchau-blr.qualcomm.com/

Signed-off-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
---
 MAINTAINERS | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 65e8a4b5c90..f275e9d4847 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -23506,6 +23506,7 @@ C:	irc://irc.libera.chat/riscv
 P:	Documentation/arch/riscv/patch-acceptance.rst
 T:	git git://git.kernel.org/pub/scm/linux/kernel/git/riscv/linux.git
 F:	arch/riscv/
+F:	tools/testing/selftests/riscv/
 N:	riscv
 K:	riscv
 
@@ -23518,6 +23519,13 @@ T:	git git://git.kernel.org/pub/scm/linux/kernel/git/iommu/linux.git
 F:	Documentation/devicetree/bindings/iommu/riscv,iommu.yaml
 F:	drivers/iommu/riscv/
 
+RISC-V FIRMWARE DRIVERS
+M:	Conor Dooley <conor@kernel.org>
+L:	linux-riscv@lists.infradead.org
+S:	Maintained
+T:	git git://git.kernel.org/pub/scm/linux/kernel/git/conor/linux.git
+F:	drivers/firmware/riscv/
+
 RISC-V MICROCHIP SUPPORT
 M:	Conor Dooley <conor.dooley@microchip.com>
 M:	Daire McNamara <daire.mcnamara@microchip.com>
@@ -23573,6 +23581,7 @@ M:	Atish Patra <atish.patra@linux.dev>
 R:	Anup Patel <anup@brainfault.org>
 L:	linux-riscv@lists.infradead.org
 S:	Supported
+F:	Documentation/arch/riscv/pmu-sse.rst
 F:	drivers/perf/riscv_pmu.c
 F:	drivers/perf/riscv_pmu_legacy.c
 F:	drivers/perf/riscv_pmu_sbi.c
@@ -23612,6 +23621,19 @@ F:	arch/riscv/boot/dts/spacemit/
 N:	spacemit
 K:	spacemit
 
+RISC-V SUPERVISOR SOFTWARE EVENTS
+M:	Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
+M:	Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>
+R:	Yunhui Cui <cuiyunhui@bytedance.com>
+L:	linux-riscv@lists.infradead.org
+S:	Maintained
+F:	arch/riscv/include/asm/sse.h
+F:	arch/riscv/kernel/sbi_sse.c
+F:	arch/riscv/kernel/sbi_sse_entry.S
+F:	drivers/firmware/riscv/riscv_sbi_sse.c
+F:	include/linux/riscv_sbi_sse.h
+F:	tools/testing/selftests/riscv/sse/
+
 RISC-V TENSTORRENT SoC SUPPORT
 M:	Drew Fustini <dfustini@oss.tenstorrent.com>
 M:	Joel Stanley <jms@oss.tenstorrent.com>
-- 
2.50.1 (Apple Git-155)

[PATCH v12 10/11] selftests/riscv: add SSE test module

[Zhanpeng Zhang]

From: Clément Léger <cleger@rivosinc.com>

Add an SSE selftest module and runner. Loading the module executes
smoke tests for the SSE framework, and the runner reports any error
emitted by the module.

Add stress={0,1,2} modes to exercise repeated handler entry and
completion, single and multiple read-only SBI calls from a handler, and
self re-injection. Check the SBI return values so handler execution alone
cannot produce a false pass, and avoid touching an unreserved PMU counter
owned by another user.

Harden the test against false failures and leaks by using TEST_PROGS for
the runner, using phys_addr_t for SBI attribute buffers, adding timeouts
to busy waits, pinning each priority chain with migrate_disable(), and
holding the CPU read lock while a fast-test target is selected, injected,
and completed. Unregister all registered events on error, check teardown
failures, and let kthread_stop() drive monitor-thread exit.

Track handler progress across fast, priority, and stress paths. If
firmware reports injectable events but the test cannot acquire or handle
any of them, report SKIP instead of silently passing a capability-only
run.

Signed-off-by: Clément Léger <cleger@rivosinc.com>
Co-developed-by: Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>
Signed-off-by: Himanshu Chauhan <himanshu.chauhan@oss.qualcomm.com>
Co-developed-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
Signed-off-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
---
 tools/testing/selftests/riscv/Makefile        |    2 +-
 tools/testing/selftests/riscv/sse/Makefile    |    5 +
 .../selftests/riscv/sse/module/Makefile       |   22 +
 .../riscv/sse/module/riscv_sse_test.c         | 1154 +++++++++++++++++
 .../selftests/riscv/sse/run_sse_test.sh       |   59 +
 5 files changed, 1241 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/riscv/sse/Makefile
 create mode 100644 tools/testing/selftests/riscv/sse/module/Makefile
 create mode 100644 tools/testing/selftests/riscv/sse/module/riscv_sse_test.c
 create mode 100644 tools/testing/selftests/riscv/sse/run_sse_test.sh

diff --git a/tools/testing/selftests/riscv/Makefile b/tools/testing/selftests/riscv/Makefile
index 5671b4405a1..43c7c8f9767 100644
--- a/tools/testing/selftests/riscv/Makefile
+++ b/tools/testing/selftests/riscv/Makefile
@@ -5,7 +5,7 @@
 ARCH ?= $(shell uname -m 2>/dev/null || echo not)
 
 ifneq (,$(filter $(ARCH),riscv))
-RISCV_SUBTARGETS ?= abi hwprobe mm sigreturn vector cfi
+RISCV_SUBTARGETS ?= abi hwprobe mm sigreturn vector cfi sse
 else
 RISCV_SUBTARGETS :=
 endif
diff --git a/tools/testing/selftests/riscv/sse/Makefile b/tools/testing/selftests/riscv/sse/Makefile
new file mode 100644
index 00000000000..7e2677fdce0
--- /dev/null
+++ b/tools/testing/selftests/riscv/sse/Makefile
@@ -0,0 +1,5 @@
+TEST_GEN_MODS_DIR := module
+
+TEST_PROGS := run_sse_test.sh
+
+include ../../lib.mk
diff --git a/tools/testing/selftests/riscv/sse/module/Makefile b/tools/testing/selftests/riscv/sse/module/Makefile
new file mode 100644
index 00000000000..eac4b1c6228
--- /dev/null
+++ b/tools/testing/selftests/riscv/sse/module/Makefile
@@ -0,0 +1,22 @@
+ifneq ($(CONFIG_RISCV_SBI_SSE),)
+obj-m += riscv_sse_test.o
+endif
+
+ifndef KERNELRELEASE
+
+TESTMODS_DIR := $(realpath $(dir $(abspath $(lastword $(MAKEFILE_LIST)))))
+KDIR ?= /lib/modules/$(shell uname -r)/build
+
+# Ensure that KDIR exists, otherwise skip the compilation
+modules:
+ifneq ("$(wildcard $(KDIR))", "")
+	$(Q)$(MAKE) -C $(KDIR) modules KBUILD_EXTMOD=$(TESTMODS_DIR)
+endif
+
+# Ensure that KDIR exists, otherwise skip the clean target
+clean:
+ifneq ("$(wildcard $(KDIR))", "")
+	$(Q)$(MAKE) -C $(KDIR) clean KBUILD_EXTMOD=$(TESTMODS_DIR)
+endif
+
+endif
diff --git a/tools/testing/selftests/riscv/sse/module/riscv_sse_test.c b/tools/testing/selftests/riscv/sse/module/riscv_sse_test.c
new file mode 100644
index 00000000000..cc5c2e46f2f
--- /dev/null
+++ b/tools/testing/selftests/riscv/sse/module/riscv_sse_test.c
@@ -0,0 +1,1154 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2025 Rivos Inc.
+ */
+
+#define pr_fmt(fmt) "riscv_sse_test: " fmt
+
+#include <linux/array_size.h>
+#include <linux/atomic.h>
+#include <linux/completion.h>
+#include <linux/cpu.h>
+#include <linux/cpuhplock.h>
+#include <linux/delay.h>
+#include <linux/err.h>
+#include <linux/hrtimer.h>
+#include <linux/io.h>
+#include <linux/jiffies.h>
+#include <linux/kthread.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/riscv_sbi_sse.h>
+#include <linux/sched.h>
+#include <linux/slab.h>
+#include <linux/smp.h>
+#include <linux/workqueue.h>
+
+#include <asm/sbi.h>
+#include <asm/sse.h>
+
+#define RUN_LOOP_COUNT		1000
+#define SSE_FAILED_PREFIX	"FAILED: "
+#define SSE_SKIP_PREFIX		"SKIP: "
+#define STRESS_DURATION_MS	3000
+#define STRESS_INJECT_NS	10000
+#define STRESS_REINJECT_DEPTH	10
+#define sse_err(...)		pr_err(SSE_FAILED_PREFIX __VA_ARGS__)
+#define sse_skip(...)		pr_info(SSE_SKIP_PREFIX __VA_ARGS__)
+
+enum sse_stress_mode {
+	SSE_STRESS_OFF,
+	SSE_STRESS_AFTER_SMOKE,
+	SSE_STRESS_ONLY,
+};
+
+static int stress;
+module_param(stress, int, 0444);
+MODULE_PARM_DESC(stress, "Stress mode: 0=off, 1=after smoke, 2=stress only");
+
+static char *run_id = "unknown";
+module_param(run_id, charp, 0444);
+MODULE_PARM_DESC(run_id, "Unique identifier used to delimit one test run");
+
+/* Do not report PASS for a capability-only run that handled no event. */
+static atomic_t sse_test_handler_count = ATOMIC_INIT(0);
+static bool sse_stress_event_can_inject;
+
+struct sse_event_desc {
+	u32 evt_id;
+	const char *name;
+	bool can_inject;
+};
+
+static struct sse_event_desc sse_event_descs[] = {
+	{
+		.evt_id = SBI_SSE_EVENT_LOCAL_HIGH_PRIO_RAS,
+		.name = "local_high_prio_ras",
+	},
+	{
+		.evt_id = SBI_SSE_EVENT_LOCAL_DOUBLE_TRAP,
+		.name = "local_double_trap",
+	},
+	{
+		.evt_id = SBI_SSE_EVENT_GLOBAL_HIGH_PRIO_RAS,
+		.name = "global_high_prio_ras",
+	},
+	{
+		.evt_id = SBI_SSE_EVENT_LOCAL_PMU_OVERFLOW,
+		.name = "local_pmu_overflow",
+	},
+	{
+		.evt_id = SBI_SSE_EVENT_LOCAL_LOW_PRIO_RAS,
+		.name = "local_low_prio_ras",
+	},
+	{
+		.evt_id = SBI_SSE_EVENT_GLOBAL_LOW_PRIO_RAS,
+		.name = "global_low_prio_ras",
+	},
+	{
+		.evt_id = SBI_SSE_EVENT_LOCAL_SOFTWARE_INJECTED,
+		.name = "local_software_injected",
+	},
+	{
+		.evt_id = SBI_SSE_EVENT_GLOBAL_SOFTWARE_INJECTED,
+		.name = "global_software_injected",
+	}
+};
+
+static DEFINE_MUTEX(sse_test_cleanup_lock);
+/* Firmware permits only one registration for each event ID. */
+static struct sse_event *sse_test_cleanup_events[ARRAY_SIZE(sse_event_descs)];
+
+static void sse_test_cleanup_workfn(struct work_struct *work);
+static DECLARE_DELAYED_WORK(sse_test_cleanup_work, sse_test_cleanup_workfn);
+
+static void sse_test_queue_cleanup(struct sse_event *event)
+{
+	int i, free_slot = -1;
+
+	mutex_lock(&sse_test_cleanup_lock);
+	for (i = 0; i < ARRAY_SIZE(sse_test_cleanup_events); i++) {
+		if (sse_test_cleanup_events[i] == event)
+			goto out_schedule;
+		if (!sse_test_cleanup_events[i] && free_slot < 0)
+			free_slot = i;
+	}
+
+	if (WARN_ON_ONCE(free_slot < 0))
+		goto out_unlock;
+
+	sse_test_cleanup_events[free_slot] = event;
+
+out_schedule:
+	mod_delayed_work(system_wq, &sse_test_cleanup_work,
+			 msecs_to_jiffies(100));
+out_unlock:
+	mutex_unlock(&sse_test_cleanup_lock);
+}
+
+static struct sse_event_desc *sse_get_evt_desc(u32 evt)
+{
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(sse_event_descs); i++) {
+		if (sse_event_descs[i].evt_id == evt)
+			return &sse_event_descs[i];
+	}
+
+	return NULL;
+}
+
+static const char *sse_evt_name(u32 evt)
+{
+	struct sse_event_desc *desc = sse_get_evt_desc(evt);
+
+	return desc ? desc->name : NULL;
+}
+
+static bool sse_test_can_inject_event(u32 evt)
+{
+	struct sse_event_desc *desc = sse_get_evt_desc(evt);
+
+	return desc ? desc->can_inject : false;
+}
+
+/*
+ * Firmware can invoke the callback until unregister succeeds. Pin the module
+ * so its handler text cannot disappear first.
+ */
+static struct sse_event *sse_test_event_register(u32 evt, u32 priority,
+						 sse_event_handler_fn *handler,
+						 void *arg)
+{
+	struct sse_event *event;
+
+	event = sse_event_register(evt, priority, handler, arg);
+	if (!IS_ERR(event))
+		__module_get(THIS_MODULE);
+
+	return event;
+}
+
+static int sse_test_event_unregister(struct sse_event *event)
+{
+	int ret;
+
+	ret = sse_event_unregister(event);
+	if (!ret)
+		module_put(THIS_MODULE);
+	else
+		sse_test_queue_cleanup(event);
+
+	return ret;
+}
+
+static void sse_test_cleanup_workfn(struct work_struct *work)
+{
+	bool retry = false;
+	int i, ret;
+
+	mutex_lock(&sse_test_cleanup_lock);
+	for (i = 0; i < ARRAY_SIZE(sse_test_cleanup_events); i++) {
+		struct sse_event *event = sse_test_cleanup_events[i];
+
+		if (!event)
+			continue;
+
+		ret = sse_event_disable(event);
+		if (!ret)
+			ret = sse_event_unregister(event);
+		if (ret) {
+			retry = true;
+			continue;
+		}
+
+		sse_test_cleanup_events[i] = NULL;
+		module_put(THIS_MODULE);
+	}
+	mutex_unlock(&sse_test_cleanup_lock);
+
+	if (retry)
+		mod_delayed_work(system_wq, &sse_test_cleanup_work,
+				 msecs_to_jiffies(100));
+}
+
+static struct sbiret sbi_sse_ecall(int fid, unsigned long arg0, unsigned long arg1)
+{
+	return sbi_ecall(SBI_EXT_SSE, fid, arg0, arg1, 0, 0, 0, 0);
+}
+
+static int sse_event_attr_read(u32 evt, unsigned long attr_id,
+			       unsigned long *attr_buf)
+{
+	struct sbiret sret;
+	phys_addr_t phys;
+
+	phys = virt_to_phys(attr_buf);
+
+	sret = sbi_ecall(SBI_EXT_SSE, SBI_SSE_EVENT_ATTR_READ, evt, attr_id, 1,
+			 (unsigned long)phys, 0, 0);
+	if (sret.error)
+		return sbi_err_map_linux_errno(sret.error);
+
+	return 0;
+}
+
+static int sse_event_attr_get(u32 evt, unsigned long attr_id,
+			      unsigned long *val)
+{
+	unsigned long *attr_buf;
+	int ret;
+
+	attr_buf = kmalloc_obj(*attr_buf, GFP_KERNEL);
+	if (!attr_buf)
+		return -ENOMEM;
+
+	ret = sse_event_attr_read(evt, attr_id, attr_buf);
+	if (!ret)
+		*val = *attr_buf;
+	kfree(attr_buf);
+
+	return ret;
+}
+
+static int sse_test_signal(u32 evt, unsigned int cpu)
+{
+	unsigned long hart_id = cpuid_to_hartid_map(cpu);
+	struct sbiret ret;
+
+	ret = sbi_sse_ecall(SBI_SSE_EVENT_INJECT, evt, hart_id);
+	if (ret.error) {
+		sse_err("Failed to signal event %x, error %ld\n", evt, ret.error);
+		return sbi_err_map_linux_errno(ret.error);
+	}
+
+	return 0;
+}
+
+static int sse_test_wait_not_running(u32 evt)
+{
+	unsigned long timeout = jiffies + HZ;
+	unsigned long status;
+	int ret;
+
+	do {
+		ret = sse_event_attr_get(evt, SBI_SSE_ATTR_STATUS, &status);
+		if (ret) {
+			sse_err("Failed to get status for evt %x, error %d\n", evt, ret);
+			return ret;
+		}
+		status &= SBI_SSE_ATTR_STATUS_STATE_MASK;
+		cpu_relax();
+	} while (status == SBI_SSE_STATE_RUNNING && time_before(jiffies, timeout));
+
+	if (status == SBI_SSE_STATE_RUNNING) {
+		sse_err("Timed out waiting for event %x to leave RUNNING state\n", evt);
+		return -ETIMEDOUT;
+	}
+
+	return 0;
+}
+
+struct sse_test_wait_status {
+	u32 evt;
+	unsigned long *attr_buf;
+	unsigned long status;
+	int ret;
+};
+
+static void sse_test_read_status_local(void *info)
+{
+	struct sse_test_wait_status *wait = info;
+
+	wait->ret = sse_event_attr_read(wait->evt, SBI_SSE_ATTR_STATUS,
+					wait->attr_buf);
+	if (!wait->ret)
+		wait->status = *wait->attr_buf & SBI_SSE_ATTR_STATUS_STATE_MASK;
+}
+
+static int sse_test_wait_not_running_on_cpu(u32 evt, unsigned int cpu)
+{
+	struct sse_test_wait_status wait = { .evt = evt };
+	unsigned long timeout;
+	int ret = 0;
+
+	if (sse_event_is_global(evt))
+		return sse_test_wait_not_running(evt);
+
+	wait.attr_buf = kmalloc_obj(*wait.attr_buf, GFP_KERNEL);
+	if (!wait.attr_buf)
+		return -ENOMEM;
+
+	timeout = jiffies + HZ;
+	do {
+		ret = smp_call_function_single(cpu, sse_test_read_status_local,
+					       &wait, true);
+		if (ret || wait.ret) {
+			ret = ret ?: wait.ret;
+			break;
+		}
+		if (wait.status != SBI_SSE_STATE_RUNNING)
+			break;
+		usleep_range(100, 200);
+	} while (time_before(jiffies, timeout));
+
+	if (!ret && wait.status == SBI_SSE_STATE_RUNNING) {
+		sse_err("Timed out waiting for event %x on CPU %u\n", evt, cpu);
+		ret = -ETIMEDOUT;
+	}
+
+	kfree(wait.attr_buf);
+
+	return ret;
+}
+
+static int sse_test_inject_event(struct sse_event *event, u32 evt, unsigned int cpu)
+{
+	int res;
+
+	if (sse_event_is_global(evt)) {
+		/*
+		 * Due to the fact the completion might happen faster than
+		 * the call to SBI_SSE_COMPLETE in the handler, if the event was
+		 * running on another CPU, we need to wait for the event status
+		 * to be !RUNNING.
+		 */
+		res = sse_test_wait_not_running(evt);
+		if (res)
+			return res;
+
+		res = sse_event_set_target_cpu(event, cpu);
+		if (res) {
+			sse_err("Failed to set cpu for evt %x, error %d\n", evt, res);
+			return res;
+		}
+	}
+
+	return sse_test_signal(evt, cpu);
+}
+
+struct fast_test_arg {
+	u32 evt;
+	int cpu;
+	bool args_ready;
+	bool completion;
+};
+
+/* A failed unregister may leave firmware holding this handler argument. */
+static struct fast_test_arg fast_test_arg;
+
+static int sse_test_handler(u32 evt, void *arg, struct pt_regs *regs)
+{
+	int ret = 0;
+	struct fast_test_arg *targ = arg;
+	u32 test_evt;
+	int cpu;
+
+	atomic_inc(&sse_test_handler_count);
+
+	/* Pairs with the argument publication in sse_run_fast_test_cpu(). */
+	if (!smp_load_acquire(&targ->args_ready)) {
+		sse_err("Received SSE event %x before its test arguments were published\n",
+			evt);
+		ret = -EINVAL;
+		goto complete;
+	}
+
+	test_evt = READ_ONCE(targ->evt);
+	cpu = READ_ONCE(targ->cpu);
+
+	if (evt != test_evt) {
+		sse_err("Received SSE event id %x instead of %x\n", test_evt, evt);
+		ret = -EINVAL;
+	}
+
+	if (!sse_event_is_global(evt) && cpu != smp_processor_id()) {
+		sse_err("Received SSE event %d on CPU %d instead of %d\n", evt, smp_processor_id(),
+			cpu);
+		ret = -EINVAL;
+	}
+
+complete:
+	WRITE_ONCE(targ->args_ready, false);
+	/* Publish handler-side checks before waking the waiting CPU. */
+	smp_store_release(&targ->completion, true);
+
+	return ret;
+}
+
+static int sse_run_fast_test_cpu(struct fast_test_arg *test_arg,
+				 struct sse_event *event, u32 evt, int cpu)
+{
+	unsigned long timeout;
+	int ret;
+
+	WRITE_ONCE(test_arg->completion, false);
+	WRITE_ONCE(test_arg->args_ready, false);
+	WRITE_ONCE(test_arg->evt, evt);
+	WRITE_ONCE(test_arg->cpu, cpu);
+	/* Publish all arguments before firmware can inject on another hart. */
+	smp_store_release(&test_arg->args_ready, true);
+
+	ret = sse_test_inject_event(event, evt, cpu);
+	if (ret) {
+		sse_err("event %s injection failed, err %d\n",
+			sse_evt_name(evt), ret);
+		return ret;
+	}
+
+	timeout = jiffies + HZ / 100;
+	/* We can not use <linux/completion.h> since they are not NMI safe */
+	/* Pairs with the handler's completion publication. */
+	while (!smp_load_acquire(&test_arg->completion) &&
+	       time_before(jiffies, timeout))
+		cpu_relax();
+	/* Acquire the handler's checks even if the loop observed a timeout. */
+	if (!smp_load_acquire(&test_arg->completion)) {
+		sse_err("Failed to wait for event %s completion on CPU %d\n",
+			sse_evt_name(evt), cpu);
+		return -ETIMEDOUT;
+	}
+
+	return sse_test_wait_not_running_on_cpu(evt, cpu);
+}
+
+static void sse_run_fast_test(struct fast_test_arg *test_arg,
+			      struct sse_event *event, u32 evt)
+{
+	int cpu;
+
+	if (sse_event_is_global(evt)) {
+		/* Keep the selected target online through injection and completion. */
+		cpu_hotplug_disable();
+		for_each_online_cpu(cpu) {
+			if (sse_run_fast_test_cpu(test_arg, event, evt, cpu))
+				break;
+		}
+		cpu_hotplug_enable();
+		return;
+	}
+
+	guard(cpus_read_lock)();
+	for_each_online_cpu(cpu) {
+		if (sse_run_fast_test_cpu(test_arg, event, evt, cpu))
+			return;
+	}
+}
+
+static void sse_test_injection_fast(void)
+{
+	int i, ret = 0, j;
+	u32 evt;
+	struct sse_event *event;
+
+	pr_info("Starting SSE test (fast)\n");
+
+	for (i = 0; i < ARRAY_SIZE(sse_event_descs); i++) {
+		evt = sse_event_descs[i].evt_id;
+		WRITE_ONCE(fast_test_arg.evt, evt);
+		WRITE_ONCE(fast_test_arg.cpu, -1);
+		WRITE_ONCE(fast_test_arg.args_ready, false);
+		WRITE_ONCE(fast_test_arg.completion, false);
+
+		if (!sse_event_descs[i].can_inject)
+			continue;
+
+		event = sse_test_event_register(evt, 0, sse_test_handler,
+						(void *)&fast_test_arg);
+		if (IS_ERR(event)) {
+			if (PTR_ERR(event) == -EEXIST) {
+				pr_info("Event %s already registered, skipping\n",
+					sse_evt_name(evt));
+				continue;
+			}
+			sse_err("Failed to register event %s, err %ld\n", sse_evt_name(evt),
+				PTR_ERR(event));
+			continue;
+		}
+
+		ret = sse_event_enable(event);
+		if (ret) {
+			sse_err("Failed to enable event %s, err %d\n", sse_evt_name(evt), ret);
+			goto err_disable;
+		}
+
+		pr_info("Starting testing event %s\n", sse_evt_name(evt));
+
+		for (j = 0; j < RUN_LOOP_COUNT; j++)
+			sse_run_fast_test(&fast_test_arg, event, evt);
+		pr_info("Finished testing event %s\n", sse_evt_name(evt));
+
+err_disable:
+		ret = sse_event_disable(event);
+		if (ret)
+			sse_err("Failed to disable event %s, err %d\n",
+				sse_evt_name(evt), ret);
+		ret = sse_test_event_unregister(event);
+		if (ret) {
+			sse_err("Failed to unregister event %s, err %d\n",
+				sse_evt_name(evt), ret);
+			return;
+		}
+	}
+	pr_info("Finished SSE test (fast)\n");
+}
+
+struct priority_test_arg {
+	unsigned long evt;
+	struct sse_event *event;
+	bool called;
+	bool enable_attempted;
+	u32 prio;
+	struct priority_test_arg *next_evt_arg;
+	void (*check_func)(struct priority_test_arg *arg);
+};
+
+/* A failed unregister may leave firmware holding these handler arguments. */
+static struct priority_test_arg default_hi_prio_args[] = {
+	{ .evt = SBI_SSE_EVENT_GLOBAL_SOFTWARE_INJECTED },
+	{ .evt = SBI_SSE_EVENT_LOCAL_SOFTWARE_INJECTED },
+	{ .evt = SBI_SSE_EVENT_GLOBAL_LOW_PRIO_RAS },
+	{ .evt = SBI_SSE_EVENT_LOCAL_LOW_PRIO_RAS },
+	{ .evt = SBI_SSE_EVENT_LOCAL_PMU_OVERFLOW },
+	{ .evt = SBI_SSE_EVENT_GLOBAL_HIGH_PRIO_RAS },
+	{ .evt = SBI_SSE_EVENT_LOCAL_DOUBLE_TRAP },
+	{ .evt = SBI_SSE_EVENT_LOCAL_HIGH_PRIO_RAS },
+};
+
+static struct priority_test_arg default_low_prio_args[] = {
+	{ .evt = SBI_SSE_EVENT_LOCAL_HIGH_PRIO_RAS },
+	{ .evt = SBI_SSE_EVENT_LOCAL_DOUBLE_TRAP },
+	{ .evt = SBI_SSE_EVENT_GLOBAL_HIGH_PRIO_RAS },
+	{ .evt = SBI_SSE_EVENT_LOCAL_PMU_OVERFLOW },
+	{ .evt = SBI_SSE_EVENT_LOCAL_LOW_PRIO_RAS },
+	{ .evt = SBI_SSE_EVENT_GLOBAL_LOW_PRIO_RAS },
+	{ .evt = SBI_SSE_EVENT_LOCAL_SOFTWARE_INJECTED },
+	{ .evt = SBI_SSE_EVENT_GLOBAL_SOFTWARE_INJECTED },
+};
+
+static struct priority_test_arg set_prio_args[] = {
+	{ .evt = SBI_SSE_EVENT_GLOBAL_SOFTWARE_INJECTED, .prio = 5 },
+	{ .evt = SBI_SSE_EVENT_LOCAL_SOFTWARE_INJECTED, .prio = 10 },
+	{ .evt = SBI_SSE_EVENT_GLOBAL_LOW_PRIO_RAS, .prio = 15 },
+	{ .evt = SBI_SSE_EVENT_LOCAL_LOW_PRIO_RAS, .prio = 20 },
+	{ .evt = SBI_SSE_EVENT_LOCAL_PMU_OVERFLOW, .prio = 25 },
+	{ .evt = SBI_SSE_EVENT_GLOBAL_HIGH_PRIO_RAS, .prio = 30 },
+	{ .evt = SBI_SSE_EVENT_LOCAL_DOUBLE_TRAP, .prio = 35 },
+	{ .evt = SBI_SSE_EVENT_LOCAL_HIGH_PRIO_RAS, .prio = 40 },
+};
+
+static struct priority_test_arg same_prio_args[] = {
+	{ .evt = SBI_SSE_EVENT_LOCAL_PMU_OVERFLOW, .prio = 0 },
+	{ .evt = SBI_SSE_EVENT_LOCAL_HIGH_PRIO_RAS, .prio = 10 },
+	{ .evt = SBI_SSE_EVENT_LOCAL_SOFTWARE_INJECTED, .prio = 10 },
+	{ .evt = SBI_SSE_EVENT_GLOBAL_SOFTWARE_INJECTED, .prio = 10 },
+	{ .evt = SBI_SSE_EVENT_GLOBAL_HIGH_PRIO_RAS, .prio = 20 },
+};
+
+static int sse_hi_priority_test_handler(u32 evt, void *arg,
+					struct pt_regs *regs)
+{
+	struct priority_test_arg *targ = arg;
+	struct priority_test_arg *next = READ_ONCE(targ->next_evt_arg);
+
+	atomic_inc(&sse_test_handler_count);
+	WRITE_ONCE(targ->called, 1);
+
+	if (next) {
+		sse_test_signal(next->evt, smp_processor_id());
+		if (!READ_ONCE(next->called)) {
+			sse_err("Higher priority event %s was not handled %s\n",
+				sse_evt_name(next->evt), sse_evt_name(evt));
+		}
+	}
+
+	return 0;
+}
+
+static int sse_low_priority_test_handler(u32 evt, void *arg, struct pt_regs *regs)
+{
+	struct priority_test_arg *targ = arg;
+	struct priority_test_arg *next = READ_ONCE(targ->next_evt_arg);
+
+	atomic_inc(&sse_test_handler_count);
+	WRITE_ONCE(targ->called, 1);
+
+	if (next) {
+		sse_test_signal(next->evt, smp_processor_id());
+		if (READ_ONCE(next->called)) {
+			sse_err("Lower priority event %s was handle before %s\n",
+				sse_evt_name(next->evt), sse_evt_name(evt));
+		}
+	}
+
+	return 0;
+}
+
+static void sse_test_injection_priority_arg(struct priority_test_arg *args, unsigned int args_size,
+					    sse_event_handler_fn handler, const char *test_name)
+{
+	unsigned int i;
+	unsigned long timeout;
+	int ret;
+	int target_cpu;
+	struct sse_event *event;
+	struct priority_test_arg *arg, *first_arg = NULL, *prev_arg = NULL;
+
+	pr_info("Starting SSE priority test (%s)\n", test_name);
+	/* Keep the complete priority chain on one CPU. */
+	migrate_disable();
+	target_cpu = smp_processor_id();
+
+	for (i = 0; i < args_size; i++) {
+		arg = &args[i];
+
+		if (!sse_test_can_inject_event(arg->evt))
+			continue;
+
+		WRITE_ONCE(arg->called, false);
+		WRITE_ONCE(arg->next_evt_arg, NULL);
+		WRITE_ONCE(arg->event, NULL);
+		WRITE_ONCE(arg->enable_attempted, false);
+
+		event = sse_test_event_register(arg->evt, arg->prio, handler,
+						(void *)arg);
+		if (IS_ERR(event)) {
+			if (PTR_ERR(event) == -EEXIST) {
+				pr_info("Event %s already registered, skipping\n",
+					sse_evt_name(arg->evt));
+				continue;
+			}
+			sse_err("Failed to register event %s, err %ld\n", sse_evt_name(arg->evt),
+				PTR_ERR(event));
+			goto release_events;
+		}
+		arg->event = event;
+
+		if (sse_event_is_global(arg->evt)) {
+			/* Keep the chain on one stable CPU. */
+			ret = sse_event_set_target_cpu(event, target_cpu);
+			if (ret) {
+				sse_err("Failed to set event %s target CPU, err %d\n",
+					sse_evt_name(arg->evt), ret);
+				goto release_events;
+			}
+		}
+
+		WRITE_ONCE(arg->enable_attempted, true);
+		ret = sse_event_enable(event);
+		if (ret) {
+			sse_err("Failed to enable event %s, err %d\n", sse_evt_name(arg->evt), ret);
+			goto release_events;
+		}
+
+		if (prev_arg)
+			WRITE_ONCE(prev_arg->next_evt_arg, arg);
+
+		prev_arg = arg;
+
+		if (!first_arg)
+			first_arg = arg;
+	}
+
+	if (!first_arg) {
+		pr_info("No injectable event available for %s priority test\n",
+			test_name);
+		goto out;
+	}
+
+	/* Inject first event, handler should trigger the others in chain. */
+	ret = sse_test_inject_event(first_arg->event, first_arg->evt, target_cpu);
+	if (ret) {
+		sse_err("SSE event %s injection failed\n", sse_evt_name(first_arg->evt));
+		goto release_events;
+	}
+
+	/* Lower-priority events run after the handler that injected them completes. */
+	arg = first_arg;
+	while (arg) {
+		timeout = jiffies + HZ;
+		while (!READ_ONCE(arg->called) && time_before(jiffies, timeout))
+			cpu_relax();
+
+		if (!READ_ONCE(arg->called)) {
+			sse_err("Event %s handler was not called\n",
+				sse_evt_name(arg->evt));
+			ret = -EINVAL;
+		}
+
+		event = arg->event;
+		arg = READ_ONCE(arg->next_evt_arg);
+	}
+
+release_events:
+
+	for (i = 0; i < args_size; i++) {
+		arg = &args[i];
+		event = arg->event;
+		if (!event)
+			continue;
+
+		ret = sse_test_wait_not_running_on_cpu(arg->evt, target_cpu);
+		if (ret)
+			sse_err("Event %s did not complete, err %d\n",
+				sse_evt_name(arg->evt), ret);
+
+		if (arg->enable_attempted) {
+			ret = sse_event_disable(event);
+			if (ret) {
+				sse_err("Failed to disable event %s, err %d\n",
+					sse_evt_name(arg->evt), ret);
+				sse_test_queue_cleanup(event);
+				WRITE_ONCE(arg->event, NULL);
+				WRITE_ONCE(arg->enable_attempted, false);
+				continue;
+			}
+		}
+
+		ret = sse_test_event_unregister(event);
+		if (ret) {
+			sse_err("Failed to unregister event %s, err %d\n",
+				sse_evt_name(arg->evt), ret);
+			continue;
+		}
+
+		WRITE_ONCE(arg->event, NULL);
+		WRITE_ONCE(arg->enable_attempted, false);
+	}
+
+	pr_info("Finished SSE priority test (%s)\n", test_name);
+out:
+	migrate_enable();
+}
+
+static void sse_test_injection_priority(void)
+{
+	sse_test_injection_priority_arg(default_hi_prio_args, ARRAY_SIZE(default_hi_prio_args),
+					sse_hi_priority_test_handler, "high");
+
+	sse_test_injection_priority_arg(default_low_prio_args, ARRAY_SIZE(default_low_prio_args),
+					sse_low_priority_test_handler, "low");
+
+	sse_test_injection_priority_arg(set_prio_args, ARRAY_SIZE(set_prio_args),
+					sse_low_priority_test_handler, "set");
+
+	sse_test_injection_priority_arg(same_prio_args, ARRAY_SIZE(same_prio_args),
+					sse_low_priority_test_handler, "same_prio_args");
+}
+
+static int sse_get_inject_status(u32 evt, bool *can_inject)
+{
+	int ret;
+	unsigned long val;
+
+	/* Check if injection is supported */
+	ret = sse_event_attr_get(evt, SBI_SSE_ATTR_STATUS, &val);
+	if (ret == sbi_err_map_linux_errno(SBI_ERR_NOT_SUPPORTED) ||
+	    ret == sbi_err_map_linux_errno(SBI_ERR_INVALID_PARAM)) {
+		*can_inject = false;
+		return 0;
+	}
+	if (ret)
+		return ret;
+
+	*can_inject = !!(val & BIT(SBI_SSE_ATTR_STATUS_INJECT_OFFSET));
+
+	return 0;
+}
+
+static int sse_init_events(void)
+{
+	int i, injectable = 0, ret;
+
+	for (i = 0; i < ARRAY_SIZE(sse_event_descs); i++) {
+		struct sse_event_desc *desc = &sse_event_descs[i];
+
+		ret = sse_get_inject_status(desc->evt_id, &desc->can_inject);
+		if (ret) {
+			sse_err("Failed to read injection status for %s, err %d\n",
+				desc->name, ret);
+			return ret;
+		}
+
+		if (desc->can_inject)
+			injectable++;
+		else
+			pr_info("Can not inject event %s, tests using this event will be skipped\n",
+				desc->name);
+
+		if (desc->evt_id == SBI_SSE_EVENT_LOCAL_SOFTWARE_INJECTED)
+			sse_stress_event_can_inject = desc->can_inject;
+	}
+
+	return injectable;
+}
+
+struct stress_test_ctx {
+	struct sse_event *event;
+	struct hrtimer timer;
+	struct hrtimer stop_timer;
+	struct task_struct *monitor_task;
+	wait_queue_head_t wait_q;
+	atomic_t inject_count;
+	atomic_t handler_count;
+	atomic_t handler_errors;
+	u32 evt_id;
+	int layer;
+	bool running;
+	bool test_done;
+};
+
+static struct stress_test_ctx stress_ctx;
+static DEFINE_PER_CPU(int, stress_reinject_cpu_depth);
+
+static int stress_handler_empty(u32 evt, void *arg, struct pt_regs *regs)
+{
+	struct stress_test_ctx *ctx = arg;
+
+	atomic_inc(&sse_test_handler_count);
+	atomic_inc(&ctx->handler_count);
+
+	return 0;
+}
+
+static int stress_handler_ecall(u32 evt, void *arg, struct pt_regs *regs)
+{
+	struct stress_test_ctx *ctx = arg;
+	struct sbiret ret;
+
+	ret = sbi_ecall(SBI_EXT_BASE, SBI_EXT_BASE_GET_SPEC_VERSION,
+			0, 0, 0, 0, 0, 0);
+	if (ret.error)
+		atomic_inc(&ctx->handler_errors);
+	atomic_inc(&sse_test_handler_count);
+	atomic_inc(&ctx->handler_count);
+
+	return 0;
+}
+
+static int stress_handler_multi_ecall(u32 evt, void *arg, struct pt_regs *regs)
+{
+	struct stress_test_ctx *ctx = arg;
+	struct sbiret ret;
+
+	ret = sbi_ecall(SBI_EXT_BASE, SBI_EXT_BASE_GET_SPEC_VERSION,
+			0, 0, 0, 0, 0, 0);
+	if (ret.error)
+		atomic_inc(&ctx->handler_errors);
+	ret = sbi_ecall(SBI_EXT_BASE, SBI_EXT_BASE_GET_IMP_ID,
+			0, 0, 0, 0, 0, 0);
+	if (ret.error)
+		atomic_inc(&ctx->handler_errors);
+	ret = sbi_ecall(SBI_EXT_BASE, SBI_EXT_BASE_GET_IMP_VERSION,
+			0, 0, 0, 0, 0, 0);
+	if (ret.error)
+		atomic_inc(&ctx->handler_errors);
+	atomic_inc(&sse_test_handler_count);
+	atomic_inc(&ctx->handler_count);
+
+	return 0;
+}
+
+static int stress_handler_reinject(u32 evt, void *arg, struct pt_regs *regs)
+{
+	struct stress_test_ctx *ctx = arg;
+	int *depth = this_cpu_ptr(&stress_reinject_cpu_depth);
+
+	(*depth)++;
+	if (*depth < STRESS_REINJECT_DEPTH)
+		sse_test_signal(evt, smp_processor_id());
+	else
+		*depth = 0;
+
+	atomic_inc(&sse_test_handler_count);
+	atomic_inc(&ctx->handler_count);
+
+	return 0;
+}
+
+static sse_event_handler_fn *stress_handlers[] = {
+	stress_handler_empty,
+	stress_handler_ecall,
+	stress_handler_multi_ecall,
+	stress_handler_reinject,
+};
+
+static const char * const stress_layer_names[] = {
+	"empty handler",
+	"single SBI ecall in handler",
+	"multiple SBI ecalls in handler",
+	"self re-inject",
+};
+
+static enum hrtimer_restart stress_timer_callback(struct hrtimer *timer)
+{
+	struct stress_test_ctx *ctx = container_of(timer, struct stress_test_ctx, timer);
+
+	if (!READ_ONCE(ctx->running))
+		return HRTIMER_NORESTART;
+
+	if (!sse_test_signal(ctx->evt_id, smp_processor_id()))
+		atomic_inc(&ctx->inject_count);
+	hrtimer_forward_now(timer, ns_to_ktime(STRESS_INJECT_NS));
+
+	return HRTIMER_RESTART;
+}
+
+static enum hrtimer_restart stress_stop_timer_callback(struct hrtimer *timer)
+{
+	struct stress_test_ctx *ctx;
+
+	ctx = container_of(timer, struct stress_test_ctx, stop_timer);
+	WRITE_ONCE(ctx->test_done, true);
+	wake_up(&ctx->wait_q);
+
+	return HRTIMER_NORESTART;
+}
+
+static int stress_monitor_thread(void *data)
+{
+	struct stress_test_ctx *ctx = data;
+	unsigned long last_inject = 0, last_handler = 0;
+
+	while (!kthread_should_stop()) {
+		unsigned long inject = atomic_read(&ctx->inject_count);
+		unsigned long handler = atomic_read(&ctx->handler_count);
+
+		pr_info("stress layer %d: inject=%lu (+%lu), handler=%lu (+%lu)\n",
+			ctx->layer, inject, inject - last_inject,
+			handler, handler - last_handler);
+
+		last_inject = inject;
+		last_handler = handler;
+
+		schedule_timeout_interruptible(HZ);
+	}
+
+	return 0;
+}
+
+static int sse_stress_test_layer(int layer)
+{
+	struct sse_event *event;
+	int inject_count, handler_count;
+	int ret, target_cpu, unregister_ret;
+
+	if (layer < 0 || layer >= ARRAY_SIZE(stress_handlers))
+		return -EINVAL;
+
+	pr_info("Starting SSE stress layer %d (%s)\n",
+		layer, stress_layer_names[layer]);
+
+	memset(&stress_ctx, 0, sizeof(stress_ctx));
+	stress_ctx.evt_id = SBI_SSE_EVENT_LOCAL_SOFTWARE_INJECTED;
+	stress_ctx.layer = layer;
+	WRITE_ONCE(stress_ctx.running, true);
+	atomic_set(&stress_ctx.inject_count, 0);
+	atomic_set(&stress_ctx.handler_count, 0);
+	atomic_set(&stress_ctx.handler_errors, 0);
+	init_waitqueue_head(&stress_ctx.wait_q);
+
+	event = sse_test_event_register(stress_ctx.evt_id, 0,
+					stress_handlers[layer], &stress_ctx);
+	if (IS_ERR(event)) {
+		sse_err("Failed to register stress event, err %ld\n",
+			PTR_ERR(event));
+		return PTR_ERR(event);
+	}
+
+	stress_ctx.event = event;
+
+	ret = sse_event_enable(event);
+	if (ret) {
+		sse_err("Failed to enable stress event, err %d\n", ret);
+		goto err_disable;
+	}
+
+	stress_ctx.monitor_task = kthread_run(stress_monitor_thread,
+					      &stress_ctx, "sse_stress_mon");
+	if (IS_ERR(stress_ctx.monitor_task)) {
+		ret = PTR_ERR(stress_ctx.monitor_task);
+		sse_err("Failed to create stress monitor thread, err %d\n", ret);
+		goto err_disable;
+	}
+
+	/* Keep the pinned timer and its local event on the selected CPU. */
+	cpus_read_lock();
+	migrate_disable();
+	target_cpu = smp_processor_id();
+	hrtimer_setup(&stress_ctx.timer, stress_timer_callback,
+		      CLOCK_MONOTONIC, HRTIMER_MODE_PINNED);
+	hrtimer_start(&stress_ctx.timer, ns_to_ktime(STRESS_INJECT_NS),
+		      HRTIMER_MODE_REL_PINNED);
+	migrate_enable();
+
+	hrtimer_setup(&stress_ctx.stop_timer, stress_stop_timer_callback,
+		      CLOCK_MONOTONIC, HRTIMER_MODE_REL);
+	hrtimer_start(&stress_ctx.stop_timer, ms_to_ktime(STRESS_DURATION_MS),
+		      HRTIMER_MODE_REL);
+
+	wait_event(stress_ctx.wait_q, READ_ONCE(stress_ctx.test_done));
+
+	WRITE_ONCE(stress_ctx.running, false);
+	hrtimer_cancel(&stress_ctx.timer);
+	hrtimer_cancel(&stress_ctx.stop_timer);
+	kthread_stop(stress_ctx.monitor_task);
+
+	pr_info("Finished SSE stress layer %d (%s): inject=%d, handler=%d\n",
+		layer, stress_layer_names[layer],
+		atomic_read(&stress_ctx.inject_count),
+		atomic_read(&stress_ctx.handler_count));
+
+	inject_count = atomic_read(&stress_ctx.inject_count);
+	handler_count = atomic_read(&stress_ctx.handler_count);
+	if (!inject_count || !handler_count) {
+		sse_err("Stress layer %d made no progress: inject=%d, handler=%d\n",
+			layer, inject_count, handler_count);
+		ret = -EIO;
+	}
+	if (atomic_read(&stress_ctx.handler_errors)) {
+		sse_err("Stress layer %d observed %d SBI call errors\n", layer,
+			atomic_read(&stress_ctx.handler_errors));
+		ret = -EIO;
+	}
+	if (sse_test_wait_not_running_on_cpu(stress_ctx.evt_id, target_cpu)) {
+		sse_err("Stress event did not complete on CPU %d\n", target_cpu);
+		if (!ret)
+			ret = -ETIMEDOUT;
+	}
+	cpus_read_unlock();
+
+err_disable:
+	if (sse_event_disable(event)) {
+		sse_err("Failed to disable stress event\n");
+		if (!ret)
+			ret = -EIO;
+	}
+	unregister_ret = sse_test_event_unregister(event);
+	if (unregister_ret) {
+		sse_err("Failed to unregister stress event\n");
+		if (!ret)
+			ret = unregister_ret;
+	}
+	stress_ctx.event = NULL;
+
+	return ret;
+}
+
+static void sse_stress_test_all_layers(void)
+{
+	int i, ret;
+
+	pr_info("Starting SSE stress tests: duration=%d ms, interval=%d ns\n",
+		STRESS_DURATION_MS, STRESS_INJECT_NS);
+
+	for (i = 0; i < ARRAY_SIZE(stress_handlers); i++) {
+		ret = sse_stress_test_layer(i);
+		if (ret) {
+			sse_err("Stress layer %d failed, err %d\n", i, ret);
+			break;
+		}
+
+		msleep(100);
+	}
+
+	pr_info("Finished SSE stress tests\n");
+}
+
+static int __init sse_test_init(void)
+{
+	int ret;
+
+	pr_info("RUN %s BEGIN\n", run_id);
+	atomic_set(&sse_test_handler_count, 0);
+
+	if (stress < SSE_STRESS_OFF || stress > SSE_STRESS_ONLY) {
+		sse_err("Invalid stress mode %d\n", stress);
+		pr_info("RUN %s END\n", run_id);
+		return -EINVAL;
+	}
+
+	ret = sse_init_events();
+	if (ret < 0) {
+		pr_info("RUN %s END\n", run_id);
+		return ret;
+	}
+	if (!ret) {
+		sse_skip("No injectable SSE event is available\n");
+		pr_info("RUN %s END\n", run_id);
+		return 0;
+	}
+	if (stress == SSE_STRESS_ONLY && !sse_stress_event_can_inject) {
+		sse_skip("Local software-injected event is unavailable for stress\n");
+		pr_info("RUN %s END\n", run_id);
+		return 0;
+	}
+
+	if (stress != SSE_STRESS_ONLY) {
+		sse_test_injection_fast();
+		sse_test_injection_priority();
+	}
+
+	if (stress == SSE_STRESS_AFTER_SMOKE && !sse_stress_event_can_inject)
+		sse_skip("Local software-injected event is unavailable for stress\n");
+	else if (stress != SSE_STRESS_OFF)
+		sse_stress_test_all_layers();
+	if (!atomic_read(&sse_test_handler_count))
+		sse_skip("No SSE event was handled\n");
+
+	pr_info("RUN %s END\n", run_id);
+
+	return 0;
+}
+
+static void __exit sse_test_exit(void)
+{
+	cancel_delayed_work_sync(&sse_test_cleanup_work);
+}
+
+module_init(sse_test_init);
+module_exit(sse_test_exit);
+
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Clément Léger <cleger@rivosinc.com>");
+MODULE_DESCRIPTION("Test module for SSE");
diff --git a/tools/testing/selftests/riscv/sse/run_sse_test.sh b/tools/testing/selftests/riscv/sse/run_sse_test.sh
new file mode 100644
index 00000000000..e70a2fd14b0
--- /dev/null
+++ b/tools/testing/selftests/riscv/sse/run_sse_test.sh
@@ -0,0 +1,59 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#
+# Copyright (C) 2025 Rivos Inc.
+
+MODULE_NAME=riscv_sse_test
+DRIVER="./module/${MODULE_NAME}.ko"
+ksft_skip=4
+
+check_test_requirements()
+{
+	uid=$(id -u)
+	if [ $uid -ne 0 ]; then
+		echo "$0: Must be run as root"
+		exit $ksft_skip
+	fi
+
+	if ! which insmod > /dev/null 2>&1; then
+		echo "$0: You need insmod installed"
+		exit $ksft_skip
+	fi
+
+	if [ ! -f "$DRIVER" ]; then
+		echo "$0: SSE is disabled or ${MODULE_NAME} is not built"
+		exit $ksft_skip
+	fi
+}
+
+check_test_requirements
+run_id="$$-$(date +%s)"
+
+if ! insmod "$DRIVER" run_id="$run_id" "$@" > /dev/null 2>&1; then
+	echo "${MODULE_NAME}: failed to load, please check dmesg"
+	exit 1
+fi
+
+if ! rmmod "$MODULE_NAME"; then
+	echo "${MODULE_NAME}: failed to unload, please check dmesg"
+	exit 1
+fi
+
+run_log=$(dmesg | sed -n \
+	"/${MODULE_NAME}: RUN ${run_id} BEGIN/,/${MODULE_NAME}: RUN ${run_id} END/p")
+if [ -z "$run_log" ]; then
+	echo "${MODULE_NAME}: unable to find log for run ${run_id}"
+	exit 1
+fi
+
+if echo "$run_log" | grep -q "${MODULE_NAME}: FAILED:"; then
+	echo "${MODULE_NAME} failed, please check dmesg"
+	exit 1
+fi
+
+if echo "$run_log" | grep -q "${MODULE_NAME}: SKIP:"; then
+	echo "${MODULE_NAME}: no injectable SSE event"
+	exit $ksft_skip
+fi
+
+exit 0
-- 
2.50.1 (Apple Git-155)

Re: [PATCH v12 10/11] selftests/riscv: add SSE test module

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 6 potential issue(s) to consider:
- [High] selftests/riscv: sse: deadlock between migrate_disable() and cpus_read_lock()
- [Medium] selftests/riscv: sse: fast test ignores errors and races on 1-jiffy timeout
- [Medium] selftests/riscv: sse: make install fails when riscv_sse_test.ko is not built
- [Medium] selftests/riscv: sse: run_sse_test.sh added without executable permissions
- [Medium] selftests/riscv: sse: stress test treats -EEXIST as failure instead of skip
- [Low] selftests/riscv: sse: swapped event ID arguments in sse_test_handler()
--

commit f2ce096e05cb9e91fc9386faf3d3f4d77029d337
Author: Clément Léger <cleger@rivosinc.com>

selftests/riscv: add SSE test module

This commit adds a RISC-V SSE selftest kernel module and a shell runner
script to exercise event injection, priority ordering, and stress scenarios
with SBI calls and self re-injection.

> diff --git a/tools/testing/selftests/riscv/sse/Makefile b/tools/testing/selftests/riscv/sse/Makefile
> new file mode 100644
> index 0000000000000..7e2677fdce09a
> --- /dev/null
> +++ b/tools/testing/selftests/riscv/sse/Makefile
> @@ -0,0 +1,5 @@
> +TEST_GEN_MODS_DIR := module
> +
> +TEST_PROGS := run_sse_test.sh
> +
> +include ../../lib.mk
> diff --git a/tools/testing/selftests/riscv/sse/module/Makefile b/tools/testing/selftests/riscv/sse/module/Makefile
> new file mode 100644
> index 0000000000000..eac4b1c6228bb
> --- /dev/null
> +++ b/tools/testing/selftests/riscv/sse/module/Makefile
> @@ -0,0 +1,22 @@
> +ifneq ($(CONFIG_RISCV_SBI_SSE),)
> +obj-m += riscv_sse_test.o
> +endif
> +
> +ifndef KERNELRELEASE
> +
> +TESTMODS_DIR := $(realpath $(dir $(abspath $(lastword $(MAKEFILE_LIST)))))
> +KDIR ?= /lib/modules/$(shell uname -r)/build
> +
> +# Ensure that KDIR exists, otherwise skip the compilation
> +modules:
> +ifneq ("$(wildcard $(KDIR))", "")
> +	$(Q)$(MAKE) -C $(KDIR) modules KBUILD_EXTMOD=$(TESTMODS_DIR)
> +endif

[Severity: Medium]
Does make install fail for the riscv selftests when riscv_sse_test.ko is not
built?

In tools/testing/selftests/riscv/sse/Makefile, TEST_GEN_MODS_DIR is set
unconditionally to module. However, in
tools/testing/selftests/riscv/sse/module/Makefile, building riscv_sse_test.o
is skipped when CONFIG_RISCV_SBI_SSE is disabled or when KDIR (which defaults
to the build host's /lib/modules/$(shell uname -r)/build) does not exist.

When no .ko is produced in module/, INSTALL_MODS_RULE in
tools/testing/selftests/lib.mk runs:

    rsync -a --copy-unsafe-links module/*.ko $(INSTALL_PATH)/module

which fails with exit status 23 because module/*.ko matches no files. Since
sse is the final target in RISCV_SUBTARGETS in
tools/testing/selftests/riscv/Makefile, this causes the top-level riscv
selftests install target to fail.

[ ... ]

> diff --git a/tools/testing/selftests/riscv/sse/module/riscv_sse_test.c b/tools/testing/selftests/riscv/sse/module/riscv_sse_test.c
> new file mode 100644
> index 0000000000000..cc5c2e46f2fdb
> --- /dev/null
> +++ b/tools/testing/selftests/riscv/sse/module/riscv_sse_test.c

[ ... ]

> +static int sse_test_wait_not_running_on_cpu(u32 evt, unsigned int cpu)
> +{
> +	struct sse_test_wait_status wait = { .evt = evt };
> +	unsigned long timeout;
> +	int ret = 0;
> +
> +	if (sse_event_is_global(evt))
> +		return sse_test_wait_not_running(evt);
> +
> +	wait.attr_buf = kmalloc_obj(*wait.attr_buf, GFP_KERNEL);
> +	if (!wait.attr_buf)
> +		return -ENOMEM;
> +
> +	timeout = jiffies + HZ;
> +	do {
> +		ret = smp_call_function_single(cpu, sse_test_read_status_local,
> +					       &wait, true);
> +		if (ret || wait.ret) {
> +			ret = ret ?: wait.ret;
> +			break;
> +		}
> +		if (wait.status != SBI_SSE_STATE_RUNNING)
> +			break;
> +		usleep_range(100, 200);
> +	} while (time_before(jiffies, timeout));
> +
> +	if (!ret && wait.status == SBI_SSE_STATE_RUNNING) {
> +		sse_err("Timed out waiting for event %x on CPU %u\n", evt, cpu);
> +		ret = -ETIMEDOUT;
> +	}
> +
> +	kfree(wait.attr_buf);
> +
> +	return ret;
> +}

[ ... ]

> +static int sse_test_handler(u32 evt, void *arg, struct pt_regs *regs)
> +{
> +	int ret = 0;
> +	struct fast_test_arg *targ = arg;
> +	u32 test_evt;
> +	int cpu;
> +
> +	atomic_inc(&sse_test_handler_count);
> +
> +	/* Pairs with the argument publication in sse_run_fast_test_cpu(). */
> +	if (!smp_load_acquire(&targ->args_ready)) {
> +		sse_err("Received SSE event %x before its test arguments were published\n",
> +			evt);
> +		ret = -EINVAL;
> +		goto complete;
> +	}
> +
> +	test_evt = READ_ONCE(targ->evt);
> +	cpu = READ_ONCE(targ->cpu);
> +
> +	if (evt != test_evt) {
> +		sse_err("Received SSE event id %x instead of %x\n", test_evt, evt);
> +		ret = -EINVAL;
> +	}

[Severity: Low]
Are test_evt and evt swapped in the sse_err() format arguments in
sse_test_handler()? evt is the received event ID and test_evt is the
expected one.

> +
> +	if (!sse_event_is_global(evt) && cpu != smp_processor_id()) {
> +		sse_err("Received SSE event %d on CPU %d instead of %d\n", evt, smp_processor_id(),
> +			cpu);
> +		ret = -EINVAL;
> +	}
> +
> +complete:
> +	WRITE_ONCE(targ->args_ready, false);
> +	/* Publish handler-side checks before waking the waiting CPU. */
> +	smp_store_release(&targ->completion, true);
> +
> +	return ret;
> +}
> +
> +static int sse_run_fast_test_cpu(struct fast_test_arg *test_arg,
> +				 struct sse_event *event, u32 evt, int cpu)
> +{
> +	unsigned long timeout;
> +	int ret;
> +
> +	WRITE_ONCE(test_arg->completion, false);
> +	WRITE_ONCE(test_arg->args_ready, false);
> +	WRITE_ONCE(test_arg->evt, evt);
> +	WRITE_ONCE(test_arg->cpu, cpu);
> +	/* Publish all arguments before firmware can inject on another hart. */
> +	smp_store_release(&test_arg->args_ready, true);
> +
> +	ret = sse_test_inject_event(event, evt, cpu);
> +	if (ret) {
> +		sse_err("event %s injection failed, err %d\n",
> +			sse_evt_name(evt), ret);
> +		return ret;
> +	}
> +
> +	timeout = jiffies + HZ / 100;
> +	/* We can not use <linux/completion.h> since they are not NMI safe */
> +	/* Pairs with the handler's completion publication. */
> +	while (!smp_load_acquire(&test_arg->completion) &&
> +	       time_before(jiffies, timeout))
> +		cpu_relax();
> +	/* Acquire the handler's checks even if the loop observed a timeout. */
> +	if (!smp_load_acquire(&test_arg->completion)) {
> +		sse_err("Failed to wait for event %s completion on CPU %d\n",
> +			sse_evt_name(evt), cpu);
> +		return -ETIMEDOUT;
> +	}
> +
> +	return sse_test_wait_not_running_on_cpu(evt, cpu);
> +}
> +
> +static void sse_run_fast_test(struct fast_test_arg *test_arg,
> +			      struct sse_event *event, u32 evt)
> +{
> +	int cpu;
> +
> +	if (sse_event_is_global(evt)) {
> +		/* Keep the selected target online through injection and completion. */
> +		cpu_hotplug_disable();
> +		for_each_online_cpu(cpu) {
> +			if (sse_run_fast_test_cpu(test_arg, event, evt, cpu))
> +				break;
> +		}
> +		cpu_hotplug_enable();
> +		return;
> +	}
> +
> +	guard(cpus_read_lock)();
> +	for_each_online_cpu(cpu) {
> +		if (sse_run_fast_test_cpu(test_arg, event, evt, cpu))
> +			return;
> +	}
> +}
> +
> +static void sse_test_injection_fast(void)
> +{
> +	int i, ret = 0, j;
> +	u32 evt;
> +	struct sse_event *event;

[ ... ]

> +		for (j = 0; j < RUN_LOOP_COUNT; j++)
> +			sse_run_fast_test(&fast_test_arg, event, evt);
> +		pr_info("Finished testing event %s\n", sse_evt_name(evt));

[Severity: Medium]
What happens in sse_test_injection_fast() when sse_run_fast_test_cpu()
times out or returns an error?

In sse_run_fast_test_cpu(), timeout = jiffies + HZ / 100 is only 1 jiffy on
CONFIG_HZ_100=y kernels (which can expire on the very next timer tick) and
2 jiffies on CONFIG_HZ_250=y kernels. If this wait times out,
sse_run_fast_test_cpu() returns -ETIMEDOUT without calling
sse_test_wait_not_running_on_cpu(evt, cpu) to wait for the event to leave
SBI_SSE_STATE_RUNNING.

Because sse_run_fast_test() returns void, sse_test_injection_fast() continues
running all RUN_LOOP_COUNT (1000) iterations even after a failure. On the
next iteration, sse_run_fast_test_cpu() immediately rewrites the shared
static fast_test_arg while the timed-out handler on the target CPU may still
be executing and accessing fast_test_arg. If sse_test_wait_not_running()
times out (1 second per iteration), running all 1000 iterations can also
stall module load for up to 1000 seconds.

Also, if sse_test_wait_not_running_on_cpu() fails with a non-timeout error
(such as -ENOMEM or an error from smp_call_function_single() or
sse_event_attr_read()), neither sse_test_wait_not_running_on_cpu() nor
sse_run_fast_test_cpu() calls sse_err(), so the error will not be logged as
a failure in dmesg. Should sse_run_fast_test() propagate errors to break out
of the RUN_LOOP_COUNT loop, use msecs_to_jiffies() for the completion
timeout, and ensure non-timeout errors are reported?

[ ... ]

> +static void sse_test_injection_priority_arg(struct priority_test_arg *args, unsigned int args_size,
> +					    sse_event_handler_fn handler, const char *test_name)
> +{
> +	unsigned int i;
> +	unsigned long timeout;
> +	int ret;
> +	int target_cpu;
> +	struct sse_event *event;
> +	struct priority_test_arg *arg, *first_arg = NULL, *prev_arg = NULL;
> +
> +	pr_info("Starting SSE priority test (%s)\n", test_name);
> +	/* Keep the complete priority chain on one CPU. */
> +	migrate_disable();
> +	target_cpu = smp_processor_id();
> +
> +	for (i = 0; i < args_size; i++) {
> +		arg = &args[i];
> +
> +		if (!sse_test_can_inject_event(arg->evt))
> +			continue;
> +
> +		WRITE_ONCE(arg->called, false);
> +		WRITE_ONCE(arg->next_evt_arg, NULL);
> +		WRITE_ONCE(arg->event, NULL);
> +		WRITE_ONCE(arg->enable_attempted, false);
> +
> +		event = sse_test_event_register(arg->evt, arg->prio, handler,
> +						(void *)arg);

[ ... ]

> +release_events:
> +
> +	for (i = 0; i < args_size; i++) {
> +		arg = &args[i];
> +		event = arg->event;
> +		if (!event)
> +			continue;
> +
> +		ret = sse_test_wait_not_running_on_cpu(arg->evt, target_cpu);
> +		if (ret)
> +			sse_err("Event %s did not complete, err %d\n",
> +				sse_evt_name(arg->evt), ret);
> +
> +		if (arg->enable_attempted) {
> +			ret = sse_event_disable(event);
> +			if (ret) {
> +				sse_err("Failed to disable event %s, err %d\n",
> +					sse_evt_name(arg->evt), ret);
> +				sse_test_queue_cleanup(event);
> +				WRITE_ONCE(arg->event, NULL);
> +				WRITE_ONCE(arg->enable_attempted, false);
> +				continue;
> +			}
> +		}
> +
> +		ret = sse_test_event_unregister(event);
> +		if (ret) {
> +			sse_err("Failed to unregister event %s, err %d\n",
> +				sse_evt_name(arg->evt), ret);
> +			continue;
> +		}
> +
> +		WRITE_ONCE(arg->event, NULL);
> +		WRITE_ONCE(arg->enable_attempted, false);
> +	}
> +
> +	pr_info("Finished SSE priority test (%s)\n", test_name);
> +out:
> +	migrate_enable();
> +}

[Severity: High]
Can calling sse_event_register(), sse_event_set_target_cpu(),
sse_event_enable(), sse_event_disable(), and sse_event_unregister() inside a
migrate_disable() section in sse_test_injection_priority_arg() deadlock with
CPU hotplug?

In drivers/firmware/riscv/riscv_sbi_sse.c, each of those SSE APIs acquires
sse_mutex and cpus_read_lock():

    scoped_guard(mutex, &sse_mutex) {
        ...
        scoped_guard(cpus_read_lock) {
            ...
        }
    }

In sse_test_injection_priority_arg(), migrate_disable() pins the task to
target_cpu without holding cpus_read_lock() or cpu_hotplug_disable() across
the test. If a concurrent _cpu_down(target_cpu) starts between two SSE API
calls:

1. The hotplug thread acquires cpus_write_lock(), runs sse_cpu_teardown() on
   target_cpu (disabling/unregistering local events and migrating global
   events off target_cpu), and then waits in sched_cpu_wait_empty() ->
   balance_hotplug_wait() for !rq_has_pinned_tasks(rq) on target_cpu.

2. In sse_test_injection_priority_arg(), the next call to sse_event_*() tries
   to acquire cpus_read_lock() while still holding migrate_disable() on
   target_cpu, blocking behind cpus_write_lock() while keeping
   rq->nr_pinned > 0.

Should sse_test_injection_priority_arg() acquire cpu_hotplug_disable()
before migrate_disable() (and release it after migrate_enable()) so that
target_cpu cannot be offlined while the task is pinned and acquiring
cpus_read_lock()?

[ ... ]

> +static int sse_stress_test_layer(int layer)
> +{
> +	struct sse_event *event;
> +	int inject_count, handler_count;
> +	int ret, target_cpu, unregister_ret;

[ ... ]

> +	event = sse_test_event_register(stress_ctx.evt_id, 0,
> +					stress_handlers[layer], &stress_ctx);
> +	if (IS_ERR(event)) {
> +		sse_err("Failed to register stress event, err %ld\n",
> +			PTR_ERR(event));
> +		return PTR_ERR(event);
> +	}

[ ... ]

> +static void sse_stress_test_all_layers(void)
> +{
> +	int i, ret;
> +
> +	pr_info("Starting SSE stress tests: duration=%d ms, interval=%d ns\n",
> +		STRESS_DURATION_MS, STRESS_INJECT_NS);
> +
> +	for (i = 0; i < ARRAY_SIZE(stress_handlers); i++) {
> +		ret = sse_stress_test_layer(i);
> +		if (ret) {
> +			sse_err("Stress layer %d failed, err %d\n", i, ret);
> +			break;
> +		}
> +
> +		msleep(100);
> +	}
> +
> +	pr_info("Finished SSE stress tests\n");
> +}

[Severity: Medium]
Should sse_stress_test_layer() skip rather than fail when
sse_test_event_register() returns -EEXIST?

The commit message notes that if firmware reports injectable events but the
test cannot acquire or handle any of them, it should report SKIP instead of
passing or failing. Both sse_test_injection_fast() and
sse_test_injection_priority_arg() check for PTR_ERR(event) == -EEXIST and
skip with pr_info().

However, in sse_stress_test_layer(), -EEXIST is logged with sse_err() (which
emits the FAILED: prefix) and returned to sse_stress_test_all_layers(),
which logs another sse_err(). Because run_sse_test.sh checks for FAILED:
before SKIP:, an already-registered local software-injected event causes the
test run to fail instead of skipping.

[ ... ]

> diff --git a/tools/testing/selftests/riscv/sse/run_sse_test.sh b/tools/testing/selftests/riscv/sse/run_sse_test.sh
> new file mode 100644
> index 0000000000000..e70a2fd14b05e
> --- /dev/null
> +++ b/tools/testing/selftests/riscv/sse/run_sse_test.sh
> @@ -0,0 +1,59 @@
> +#!/bin/bash
> +# SPDX-License-Identifier: GPL-2.0

[Severity: Medium]
Should tools/testing/selftests/riscv/sse/run_sse_test.sh be committed with
mode 100755 instead of 100644?

Because run_sse_test.sh is listed in TEST_PROGS without executable
permissions, running it directly fails with permission denied, and
tools/testing/selftests/kselftest/runner.sh (in run_one()) warns that the
file is not executable and falls back to invoking the interpreter without
forwarding $kselftest_cmd_args to the script.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261029074853.35467-1-zhangzhanpeng.jasper@bytedance.com?part=10

[PATCH v12 11/11] selftests/riscv: add perf user-stack SSE copy regression test

[Zhanpeng Zhang]

On RISC-V, PMU overflow interrupts can be delivered through the SBI
Supervisor Software Events (SSE) mechanism. A perf event that samples
the raw user stack (PERF_SAMPLE_STACK_USER, as perf record
--call-graph dwarf does) then copies a chunk of the interrupted task's
user stack from an NMI-like context. Non-resident pages must truncate
that copy through exception-table fixup without entering normal
page-fault handling or disturbing the interrupted context. Exercise
this boundary and the perf record layout under SSE sampling.

The existing SSE selftest module exercises the framework (register,
enable, inject, complete, priorities, stress) but never drives the perf
user-stack copy that motivated the no-fault path. Add a userspace test
that closes that gap:

  - Sample a child whose stack ends at a PROT_NONE page using
    PERF_SAMPLE_STACK_USER. Check each sample record for a bounded dump
    size and require at least one truncated copy at that boundary.
  - Drive a multi-CPU unix-socket + deep-recursion workload under
    high-frequency raw user-stack sampling to exercise repeated entry,
    copying and completion under load.

The test reports SKIP when hardware PMU sampling is unavailable or
perf_event_paranoid forbids it, so it is safe to run unprivileged or in
constrained environments. It is placed under the RISC-V SSE selftests
because SSE delivery is the RISC-V-specific condition it protects, and
is wired into the sse subtarget Makefile alongside the module runner.

Signed-off-by: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>
---
 tools/testing/selftests/riscv/sse/Makefile    |   5 +
 .../selftests/riscv/sse/sse_perf_ustack.c     | 564 ++++++++++++++++++
 2 files changed, 569 insertions(+)
 create mode 100644 tools/testing/selftests/riscv/sse/sse_perf_ustack.c

diff --git a/tools/testing/selftests/riscv/sse/Makefile b/tools/testing/selftests/riscv/sse/Makefile
index 7e2677fdce0..646b418de9f 100644
--- a/tools/testing/selftests/riscv/sse/Makefile
+++ b/tools/testing/selftests/riscv/sse/Makefile
@@ -1,5 +1,10 @@
+CFLAGS += -I$(top_srcdir)/tools/testing/selftests
+LDLIBS += -lpthread
+
 TEST_GEN_MODS_DIR := module
 
+TEST_GEN_PROGS := sse_perf_ustack
+
 TEST_PROGS := run_sse_test.sh
 
 include ../../lib.mk
diff --git a/tools/testing/selftests/riscv/sse/sse_perf_ustack.c b/tools/testing/selftests/riscv/sse/sse_perf_ustack.c
new file mode 100644
index 00000000000..9535b6d7ba4
--- /dev/null
+++ b/tools/testing/selftests/riscv/sse/sse_perf_ustack.c
@@ -0,0 +1,564 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Regression test for the RISC-V perf user-stack copy taken in SSE
+ * (NMI-like) context.
+ *
+ * On RISC-V, PMU overflow interrupts can be delivered through the SBI
+ * Supervisor Software Events (SSE) mechanism. A perf event that samples the
+ * raw user stack (PERF_SAMPLE_STACK_USER, as perf record --call-graph dwarf
+ * does) then copies a large chunk of the interrupted task's user stack from
+ * that context. If that copy is allowed to take a nested page fault it can
+ * corrupt the interrupted task's kernel state and hang or crash the machine
+ * under load.
+ *
+ * This test exercises that exact path:
+ *   - It opens a sampling hardware PMU event with PERF_SAMPLE_STACK_USER.
+ *   - It samples a child running on a controlled user stack followed by an
+ *     inaccessible page, so the copy must truncate at that page boundary.
+ *   - It checks that every user-stack sample record is well formed and that
+ *     the dumped size never exceeds the requested size (i.e. the copy stops
+ *     cleanly rather than faulting on).
+ *   - It then drives a multi-threaded unix-socket + deep-recursion workload
+ *     under high-frequency per-CPU sampling and requires every active sampler
+ *     to make progress without taking the machine down.
+ *
+ * The test is architecture independent in what it drives; it is placed under
+ * the RISC-V SSE selftests because SSE delivery is the RISC-V-specific
+ * condition it is meant to protect.
+ */
+#define _GNU_SOURCE
+
+#include <errno.h>
+#include <poll.h>
+#include <sched.h>
+#include <signal.h>
+#include <stdbool.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <time.h>
+#include <unistd.h>
+
+#include <linux/perf_event.h>
+#include <pthread.h>
+#include <sys/ioctl.h>
+#include <sys/mman.h>
+#include <sys/socket.h>
+#include <sys/syscall.h>
+#include <sys/wait.h>
+
+#include "../../kselftest.h"
+
+#ifndef noinline
+#define noinline __attribute__((noinline))
+#endif
+
+#define STACK_DUMP_SIZE		8192		/* 8 KiB, 8-byte aligned */
+#define RB_DATA_PAGES		64		/* power of two */
+#define SELF_SAMPLE_FREQ	4000
+#define STRESS_SAMPLE_FREQ	5000
+#define STRESS_SECONDS		5
+#define RECURSE_DEPTH		512
+#define TRUNCATION_RUN_MS	250
+
+static long page_size;
+
+static int perf_event_open(struct perf_event_attr *attr, pid_t pid, int cpu,
+			   int group_fd, unsigned long flags)
+{
+	return syscall(__NR_perf_event_open, attr, pid, cpu, group_fd, flags);
+}
+
+/* Prevent the compiler from optimizing away a stack buffer. */
+static void keep_alive(void *p)
+{
+	__asm__ __volatile__("" : : "r"(p) : "memory");
+}
+
+/*
+ * Consume a deep user stack and keep it live, so a raw user-stack sample has
+ * many pages to copy. Returns a value derived from the stack so the compiler
+ * cannot elide the frames.
+ */
+static noinline unsigned long burn_stack(int depth, unsigned long *sink)
+{
+	unsigned long frame[32];
+	unsigned int i;
+
+	for (i = 0; i < ARRAY_SIZE(frame); i++)
+		frame[i] = (unsigned long)depth * i + *sink;
+
+	if (depth > 0)
+		frame[depth & 31] += burn_stack(depth - 1, sink);
+
+	for (i = 0; i < ARRAY_SIZE(frame); i++)
+		*sink += frame[i];
+
+	keep_alive(frame);
+	return *sink;
+}
+
+static struct perf_event_attr sampling_attr(unsigned long freq)
+{
+	struct perf_event_attr attr = {
+		.type		= PERF_TYPE_HARDWARE,
+		.size		= sizeof(attr),
+		.config		= PERF_COUNT_HW_INSTRUCTIONS,
+		.sample_type	= PERF_SAMPLE_STACK_USER,
+		.sample_stack_user = STACK_DUMP_SIZE,
+		.freq		= 1,
+		.sample_freq	= freq,
+		.disabled	= 1,
+		.exclude_kernel	= 1,
+		.exclude_hv	= 1,
+	};
+
+	return attr;
+}
+
+static bool open_skip_reason(int err, const char **why)
+{
+	switch (err) {
+	case EACCES:
+	case EPERM:
+		*why = "insufficient privilege for PMU sampling (perf_event_paranoid)";
+		return true;
+	case ENOENT:
+	case ENODEV:
+	case EOPNOTSUPP:
+		*why = "hardware PMU sampling event not available";
+		return true;
+	default:
+		return false;
+	}
+}
+
+static bool pmu_sse_route_testable(const char **why)
+{
+	char *line = NULL;
+	size_t line_size = 0;
+	FILE *interrupts;
+	bool testable = true;
+
+	interrupts = fopen("/proc/interrupts", "re");
+	if (!interrupts) {
+		*why = "cannot inspect the active PMU delivery route";
+		return false;
+	}
+
+	/* The SBI PMU driver registers this name only for ordinary IRQ delivery. */
+	while (getline(&line, &line_size, interrupts) >= 0) {
+		if (strstr(line, "riscv-pmu")) {
+			*why = "ordinary RISC-V PMU IRQ delivery is active";
+			testable = false;
+			break;
+		}
+	}
+
+	free(line);
+	fclose(interrupts);
+	return testable;
+}
+
+static bool ring_copy_from(void *dst, const void *rb, size_t rb_bytes,
+			   uint64_t pos, size_t size)
+{
+	size_t offset = pos % rb_bytes;
+	size_t first;
+
+	if (size > rb_bytes)
+		return false;
+
+	first = size < rb_bytes - offset ? size : rb_bytes - offset;
+	memcpy(dst, (const char *)rb + offset, first);
+	if (first != size)
+		memcpy((char *)dst + first, rb, size - first);
+
+	return true;
+}
+
+static int truncation_child(void *arg)
+{
+	int ready_fd = (intptr_t)arg;
+	char ready = 1;
+
+	if (write(ready_fd, &ready, sizeof(ready)) != 1)
+		return 1;
+
+	for (;;)
+		__asm__ __volatile__("" : : : "memory");
+}
+
+static pid_t start_truncation_child(void **stack_mapping)
+{
+	struct pollfd pfd = { .events = POLLIN };
+	size_t mapping_size = 2 * page_size;
+	char ready;
+	void *stack;
+	pid_t pid;
+	int pipefd[2];
+	int saved_errno;
+
+	stack = mmap(NULL, mapping_size, PROT_NONE,
+		     MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
+	if (stack == MAP_FAILED)
+		return -1;
+	if (mprotect(stack, page_size, PROT_READ | PROT_WRITE))
+		goto err_unmap;
+	if (pipe(pipefd))
+		goto err_unmap;
+
+	/* clone() starts the child below the inaccessible second page. */
+	pid = clone(truncation_child, (char *)stack + page_size, SIGCHLD,
+		    (void *)(intptr_t)pipefd[1]);
+	if (pid < 0)
+		goto err_pipe;
+
+	close(pipefd[1]);
+	pfd.fd = pipefd[0];
+	if (poll(&pfd, 1, 1000) != 1 ||
+	    read(pipefd[0], &ready, sizeof(ready)) != sizeof(ready)) {
+		saved_errno = ETIMEDOUT;
+		kill(pid, SIGKILL);
+		waitpid(pid, NULL, 0);
+		close(pipefd[0]);
+		errno = saved_errno;
+		goto err_unmap;
+	}
+	close(pipefd[0]);
+
+	*stack_mapping = stack;
+	return pid;
+
+err_pipe:
+	saved_errno = errno;
+	close(pipefd[0]);
+	close(pipefd[1]);
+	errno = saved_errno;
+err_unmap:
+	saved_errno = errno;
+	munmap(stack, mapping_size);
+	errno = saved_errno;
+	return -1;
+}
+
+static void stop_truncation_child(pid_t pid, void *stack_mapping)
+{
+	kill(pid, SIGKILL);
+	while (waitpid(pid, NULL, 0) < 0 && errno == EINTR)
+		;
+	munmap(stack_mapping, 2 * page_size);
+}
+
+/*
+ * Subtest 1: sample a child whose stack is followed by an inaccessible page.
+ * Every record must be well formed and at least one stack copy must truncate
+ * at the controlled page boundary rather than fault or overrun.
+ */
+static void test_ustack_records_wellformed(void)
+{
+	struct perf_event_attr attr = sampling_attr(SELF_SAMPLE_FREQ);
+	size_t rb_bytes = (size_t)RB_DATA_PAGES * page_size;
+	struct perf_event_mmap_page *meta;
+	unsigned long samples = 0, truncated = 0;
+	void *child_stack;
+	const char *why;
+	void *rb;
+	pid_t child;
+	int fd;
+
+	child = start_truncation_child(&child_stack);
+	if (child < 0) {
+		ksft_test_result_fail("ustack records: create guarded stack child: %s\n",
+				      strerror(errno));
+		return;
+	}
+
+	fd = perf_event_open(&attr, child, -1, -1, PERF_FLAG_FD_CLOEXEC);
+	if (fd < 0) {
+		if (open_skip_reason(errno, &why))
+			ksft_test_result_skip("ustack records: %s\n", why);
+		else
+			ksft_test_result_fail("ustack records: perf_event_open: %s\n",
+					      strerror(errno));
+		goto out_child;
+	}
+
+	meta = mmap(NULL, page_size + rb_bytes, PROT_READ | PROT_WRITE,
+		    MAP_SHARED, fd, 0);
+	if (meta == MAP_FAILED) {
+		ksft_test_result_fail("ustack records: mmap ring buffer: %s\n",
+				      strerror(errno));
+		close(fd);
+		goto out_child;
+	}
+	rb = (char *)meta + page_size;
+
+	ioctl(fd, PERF_EVENT_IOC_RESET, 0);
+	ioctl(fd, PERF_EVENT_IOC_ENABLE, 0);
+	usleep(TRUNCATION_RUN_MS * 1000);
+	ioctl(fd, PERF_EVENT_IOC_DISABLE, 0);
+
+	/* Drain the ring buffer and validate every SAMPLE record. */
+	{
+		uint64_t head = __atomic_load_n(&meta->data_head, __ATOMIC_ACQUIRE);
+		uint64_t tail = meta->data_tail;
+		bool ok = true;
+
+		if (head < tail || head - tail > rb_bytes)
+			ok = false;
+
+		while (ok && tail < head) {
+			struct perf_event_header hdr;
+			uint64_t available = head - tail;
+
+			if (available < sizeof(hdr) ||
+			    !ring_copy_from(&hdr, rb, rb_bytes, tail, sizeof(hdr)) ||
+			    hdr.size < sizeof(hdr) || hdr.size > available ||
+			    hdr.size > rb_bytes) {
+				ok = false;
+				break;
+			}
+
+			if (hdr.type == PERF_RECORD_SAMPLE) {
+				uint64_t dump_size, dyn_size;
+				size_t cursor = sizeof(hdr);
+
+				if (sizeof(dump_size) > hdr.size - cursor ||
+				    !ring_copy_from(&dump_size, rb, rb_bytes,
+						    tail + cursor, sizeof(dump_size)) ||
+				    dump_size > STACK_DUMP_SIZE) {
+					ok = false;
+					break;
+				}
+				cursor += sizeof(dump_size);
+				samples++;
+				if (dump_size) {
+					/* data blob then trailing dynamic size */
+					if (dump_size > hdr.size - cursor) {
+						ok = false;
+						break;
+					}
+					cursor += dump_size;
+					if (sizeof(dyn_size) > hdr.size - cursor ||
+					    !ring_copy_from(&dyn_size, rb, rb_bytes,
+							    tail + cursor,
+							    sizeof(dyn_size))) {
+						ok = false;
+						break;
+					}
+					if (dyn_size > dump_size) {
+						ok = false;
+						break;
+					}
+					if (dyn_size < dump_size)
+						truncated++;
+				}
+			}
+			tail += hdr.size;
+		}
+		__atomic_store_n(&meta->data_tail, head, __ATOMIC_RELEASE);
+
+		if (!ok)
+			ksft_test_result_fail("ustack records: malformed sample record\n");
+		else if (samples == 0)
+			ksft_test_result_skip("ustack records: no samples collected\n");
+		else if (truncated == 0)
+			ksft_test_result_fail("ustack records: no guarded-stack truncation\n");
+		else
+			ksft_test_result_pass("ustack records: %lu samples, %lu truncated\n",
+					      samples, truncated);
+	}
+
+	munmap(meta, page_size + rb_bytes);
+	close(fd);
+out_child:
+	stop_truncation_child(child, child_stack);
+}
+
+/* ---- Subtest 2: multi-threaded per-CPU sampling stress ---- */
+
+struct stress_thread {
+	pthread_t tid;
+	int cpu;
+	int *stop;
+	int fd;
+	void *rb;
+	size_t rb_bytes;
+};
+
+static void *stress_worker(void *arg)
+{
+	struct stress_thread *st = arg;
+	unsigned long sink = 1;
+	int sv[2];
+	char buf[64];
+
+	if (socketpair(AF_UNIX, SOCK_STREAM, 0, sv) == 0) {
+		while (!__atomic_load_n(st->stop, __ATOMIC_RELAXED)) {
+			/* unix-socket ping-pong: takes the socket locks the
+			 * original bug corrupted, while sampling nests.
+			 */
+			if (write(sv[0], buf, sizeof(buf)) > 0)
+				(void)read(sv[1], buf, sizeof(buf));
+			burn_stack(RECURSE_DEPTH, &sink);
+			/* Periodically consume the ring buffer so sampling
+			 * keeps delivering rather than filling up and stopping.
+			 */
+			if (st->rb) {
+				struct perf_event_mmap_page *m = st->rb;
+				uint64_t h = __atomic_load_n(&m->data_head,
+							     __ATOMIC_ACQUIRE);
+				__atomic_store_n(&m->data_tail, h,
+						 __ATOMIC_RELEASE);
+			}
+		}
+		close(sv[0]);
+		close(sv[1]);
+	}
+
+	return (void *)sink;
+}
+
+static void test_sse_stress_no_crash(void)
+{
+	struct perf_event_attr attr = sampling_attr(STRESS_SAMPLE_FREQ);
+	size_t rb_bytes = (size_t)RB_DATA_PAGES * page_size;
+	struct stress_thread *threads;
+	cpu_set_t available;
+	long progressed = 0;
+	int stop = 0;
+	const char *why = NULL;
+	long started = 0;
+	long nproc;
+	long slot;
+	int cpu;
+
+	if (sched_getaffinity(0, sizeof(available), &available)) {
+		ksft_test_result_fail("sse stress: sched_getaffinity: %s\n",
+				      strerror(errno));
+		return;
+	}
+	nproc = CPU_COUNT(&available);
+	if (nproc < 1) {
+		ksft_test_result_skip("sse stress: no available CPUs\n");
+		return;
+	}
+
+	threads = calloc(nproc, sizeof(*threads));
+	if (!threads) {
+		ksft_test_result_fail("sse stress: out of memory\n");
+		return;
+	}
+
+	slot = 0;
+	for (cpu = 0; cpu < CPU_SETSIZE; cpu++) {
+		struct stress_thread *st;
+		pthread_attr_t thread_attr;
+		cpu_set_t set;
+		void *map;
+		int ret;
+
+		if (!CPU_ISSET(cpu, &available))
+			continue;
+		st = &threads[slot++];
+
+		st->fd = -1;
+		st->cpu = cpu;
+		st->stop = &stop;
+		st->rb_bytes = rb_bytes;
+		st->fd = perf_event_open(&attr, -1, cpu, -1,
+					 PERF_FLAG_FD_CLOEXEC);
+		if (st->fd < 0) {
+			if (!started && open_skip_reason(errno, &why))
+				break;
+			continue;
+		}
+
+		map = mmap(NULL, page_size + rb_bytes, PROT_READ | PROT_WRITE,
+			   MAP_SHARED, st->fd, 0);
+		if (map == MAP_FAILED) {
+			close(st->fd);
+			st->fd = -1;
+			continue;
+		}
+		st->rb = map;
+
+		CPU_ZERO(&set);
+		CPU_SET(cpu, &set);
+		pthread_attr_init(&thread_attr);
+		ret = pthread_attr_setaffinity_np(&thread_attr, sizeof(set), &set);
+		if (!ret)
+			ret = pthread_create(&st->tid, &thread_attr,
+					     stress_worker, st);
+		pthread_attr_destroy(&thread_attr);
+
+		if (ret) {
+			munmap(st->rb, page_size + rb_bytes);
+			close(st->fd);
+			st->rb = NULL;
+			st->fd = -1;
+			continue;
+		}
+
+		ioctl(st->fd, PERF_EVENT_IOC_RESET, 0);
+		ioctl(st->fd, PERF_EVENT_IOC_ENABLE, 0);
+		started++;
+	}
+
+	if (started == 0) {
+		free(threads);
+		if (why)
+			ksft_test_result_skip("sse stress: %s\n", why);
+		else
+			ksft_test_result_skip("sse stress: could not start any sampler\n");
+		return;
+	}
+
+	sleep(STRESS_SECONDS);
+	__atomic_store_n(&stop, 1, __ATOMIC_RELAXED);
+
+	for (slot = 0; slot < nproc; slot++) {
+		struct stress_thread *st = &threads[slot];
+		struct perf_event_mmap_page *meta;
+
+		if (st->fd < 0)
+			continue;
+		pthread_join(st->tid, NULL);
+		ioctl(st->fd, PERF_EVENT_IOC_DISABLE, 0);
+		meta = st->rb;
+		if (__atomic_load_n(&meta->data_head, __ATOMIC_ACQUIRE))
+			progressed++;
+		munmap(st->rb, page_size + rb_bytes);
+		close(st->fd);
+	}
+
+	free(threads);
+	if (progressed != started)
+		ksft_test_result_fail("sse stress: %ld/%ld samplers made progress\n",
+				      progressed, started);
+	else
+		ksft_test_result_pass("sse stress: %ld samplers x %ds made progress\n",
+				      started, STRESS_SECONDS);
+}
+
+int main(void)
+{
+	const char *why;
+
+	page_size = sysconf(_SC_PAGESIZE);
+
+	ksft_print_header();
+	ksft_set_plan(2);
+	if (!pmu_sse_route_testable(&why)) {
+		ksft_test_result_skip("ustack records: %s\n", why);
+		ksft_test_result_skip("sse stress: %s\n", why);
+		ksft_finished();
+	}
+
+	test_ustack_records_wellformed();
+	test_sse_stress_no_crash();
+
+	ksft_finished();
+}
-- 
2.50.1 (Apple Git-155)

Re: [PATCH v12 11/11] selftests/riscv: add perf user-stack SSE copy regression test

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [Medium] selftests/riscv: sse_perf_ustack fails with EINVAL instead of skipping without PMU IRQ/SSE
- [Medium] selftests/riscv: stress_worker() overflows default pthread stack on musl in burn_stack()
- [Medium] selftests/riscv: sse_perf_ustack accepts 0-byte ustack copy as truncated
--

commit 5a3dc623890401f5832b1d92fb8db20380606649
Author: Zhanpeng Zhang <zhangzhanpeng.jasper@bytedance.com>

selftests/riscv: add perf user-stack SSE copy regression test

This commit adds a RISC-V SSE selftest exercising PERF_SAMPLE_STACK_USER
raw user-stack copying under PMU overflow sampling. It checks page-boundary
truncation against a PROT_NONE guard page and runs a multi-CPU stress
workload combining unix sockets and deep stack recursion.

> diff --git a/tools/testing/selftests/riscv/sse/sse_perf_ustack.c b/tools/testing/selftests/riscv/sse/sse_perf_ustack.c
> new file mode 100644
> index 0000000000000..9535b6d7ba4c1
> --- /dev/null
> +++ b/tools/testing/selftests/riscv/sse/sse_perf_ustack.c
[ ... ]
> +static struct perf_event_attr sampling_attr(unsigned long freq)
> +{
> +	struct perf_event_attr attr = {
> +		.type		= PERF_TYPE_HARDWARE,
> +		.size		= sizeof(attr),
> +		.config		= PERF_COUNT_HW_INSTRUCTIONS,
> +		.sample_type	= PERF_SAMPLE_STACK_USER,
> +		.sample_stack_user = STACK_DUMP_SIZE,
> +		.freq		= 1,
> +		.sample_freq	= freq,
> +		.disabled	= 1,
> +		.exclude_kernel	= 1,
> +		.exclude_hv	= 1,
> +	};
> +
> +	return attr;
> +}
> +
> +static bool open_skip_reason(int err, const char **why)
> +{
> +	switch (err) {
> +	case EACCES:
> +	case EPERM:
> +		*why = "insufficient privilege for PMU sampling (perf_event_paranoid)";
> +		return true;
> +	case ENOENT:
> +	case ENODEV:
> +	case EOPNOTSUPP:
> +		*why = "hardware PMU sampling event not available";
> +		return true;
> +	default:
> +		return false;
> +	}
> +}

[Severity: Medium]
When running on a RISC-V system where neither SBI SSE nor Sscofpmf IRQ
overflow delivery is supported, will test_ustack_records_wellformed() fail
with EINVAL instead of skipping?

In that configuration, no riscv-pmu interrupt is registered in
/proc/interrupts, so pmu_sse_route_testable() returns true, and
pmu_sbi_device_probe() sets both PERF_PMU_CAP_NO_INTERRUPT and
PERF_PMU_CAP_NO_EXCLUDE:

drivers/perf/riscv_pmu_sbi.c:pmu_sbi_device_probe() {
    ...
	ret = pmu_sbi_setup_irqs(pmu, pdev);
	if (ret < 0) {
		pr_info("Perf sampling/filtering is not supported as sscof extension is not available\n");
		pmu->pmu.capabilities |= PERF_PMU_CAP_NO_INTERRUPT;
		pmu->pmu.capabilities |= PERF_PMU_CAP_NO_EXCLUDE;
	}
    ...
}

Because sampling_attr() sets exclude_kernel and exclude_hv,
perf_event_open() -> perf_event_alloc() -> perf_init_event() ->
perf_try_init_event() rejects the event with -EINVAL before
perf_event_open() reaches the PERF_PMU_CAP_NO_INTERRUPT check that returns
-EOPNOTSUPP:

kernel/events/core.c:perf_try_init_event() {
    ...
	if (pmu->capabilities & PERF_PMU_CAP_NO_EXCLUDE &&
	    event_has_any_exclude_flag(event)) {
		ret = -EINVAL;
		goto err_destroy;
	}
    ...
}

Since open_skip_reason() does not match EINVAL, test_ustack_records_wellformed()
calls ksft_test_result_fail() rather than ksft_test_result_skip().

[ ... ]
> +static int truncation_child(void *arg)
> +{
> +	int ready_fd = (intptr_t)arg;
> +	char ready = 1;
> +
> +	if (write(ready_fd, &ready, sizeof(ready)) != 1)
> +		return 1;
> +
> +	for (;;)
> +		__asm__ __volatile__("" : : : "memory");
> +}
[ ... ]
> +static void test_ustack_records_wellformed(void)
> +{
[ ... ]
> +					if (dyn_size > dump_size) {
> +						ok = false;
> +						break;
> +					}
> +					if (dyn_size < dump_size)
> +						truncated++;
> +				}
> +			}

[Severity: Medium]
Can test_ustack_records_wellformed() pass when every sample copies 0 bytes
(dyn_size == 0) instead of performing a partial copy up to the guard page?

start_truncation_child() passes stack + page_size to clone(), where the page
at stack + page_size is mapped PROT_NONE. Inside truncation_child(), the
stack frame only holds ready_fd and ready before entering the spin loop,
leaving the user stack pointer within 16 to 48 bytes of the PROT_NONE page
boundary.

When perf_output_sample_ustack() copies the user stack via
__output_copy_user() -> riscv_perf_out_copy_user() ->
__copy_from_user_inatomic(), fallback_scalar_usercopy_sum_enabled() enters
.Lword_copy and issues 8 unrolled word loads (64 bytes on RV64) before
storing any bytes or advancing dst (a0):

arch/riscv/lib/uaccess.S:fallback_scalar_usercopy_sum_enabled() {
    ...
2:
	fixup REG_L   a4,        0(a1), 10f
	fixup REG_L   a5,    SZREG(a1), 10f
	fixup REG_L   a6,  2*SZREG(a1), 10f
	fixup REG_L   a7,  3*SZREG(a1), 10f
	fixup REG_L   t1,  4*SZREG(a1), 10f
	fixup REG_L   t2,  5*SZREG(a1), 10f
	fixup REG_L   t3,  6*SZREG(a1), 10f
	fixup REG_L   t4,  7*SZREG(a1), 10f
    ...
}

Because the stack pointer is closer than 64 bytes to the PROT_NONE page,
one of the initial loads faults and jumps to the fixup label before a0
advances, returning the full requested size as uncopied and producing
dyn_size == 0.

Since test_ustack_records_wellformed() only checks dyn_size < dump_size,
dyn_size == 0 increments truncated and passes the test without verifying
that a non-zero partial stack copy succeeded up to the guard page. Should
truncation_child() keep a live stack frame larger than 64 bytes and require
0 < dyn_size && dyn_size < dump_size?

[ ... ]
> +static void test_sse_stress_no_crash(void)
> +{
[ ... ]
> +		CPU_ZERO(&set);
> +		CPU_SET(cpu, &set);
> +		pthread_attr_init(&thread_attr);
> +		ret = pthread_attr_setaffinity_np(&thread_attr, sizeof(set), &set);
> +		if (!ret)
> +			ret = pthread_create(&st->tid, &thread_attr,
> +					     stress_worker, st);
> +		pthread_attr_destroy(&thread_attr);

[Severity: Medium]
Does stress_worker() overflow the default pthread stack on musl libc when
calling burn_stack(RECURSE_DEPTH, &sink)?

In burn_stack(), each of the 513 active frames (from RECURSE_DEPTH = 512
down to 0) allocates unsigned long frame[32] (256 bytes on RV64) plus
saved registers and keeps the frame live across the recursive call via
keep_alive(frame):

static noinline unsigned long burn_stack(int depth, unsigned long *sink)
{
	unsigned long frame[32];
	unsigned int i;

	for (i = 0; i < ARRAY_SIZE(frame); i++)
		frame[i] = (unsigned long)depth * i + *sink;

	if (depth > 0)
		frame[depth & 31] += burn_stack(depth - 1, sink);

This consumes roughly 148 KiB of stack space, which exceeds musl libc's
default pthread stack size of 128 KiB unless pthread_attr_setstacksize() is
used to request a larger stack for thread_attr.

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261029074853.35467-1-zhangzhanpeng.jasper@bytedance.com?part=11