* [RFC PATCH 0/3] riscv: add support for SBI Supervisor Software Events
@ 2023-10-26 14:31 ` Clément Léger
0 siblings, 0 replies; 25+ messages in thread
From: Clément Léger @ 2023-10-26 14:31 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Albert Ou, Atish Patra, Anup Patel,
linux-riscv, linux-kernel, linux-arm-kernel
Cc: Clément Léger, Himanshu Chauhan, Xu Lu
The SBI Supervisor Software Events (SSE) extensions provides a mechanism
to inject software events from an SBI implementation to supervisor
software such that it preempts all other supervisor level traps and
interrupts [1].
Various events are defined and can be send asynchronously to supervisor
software (RAS, PMU, DEBUG, Asynchronous page fault) from SBI as well
as platform specific events. Events can be either local (per-hart) or
global. Events can be nested on top of each other based on priority and
can interrupt the kernel at any time.
First patch adds the SSE definitions. Second one adds support for SSE
itself. Implementation is split between arch specific code and generic
part (similarly to what is done for ARM SDEI). Finally, the last patch
add support fro SSE event in the SBI PMU driver. If the SSE event is
available from the SBI then, it will be used instead of the normal
interrupt.
Amongst the specific points that needs to be handle is the interruption
at any point of the kernel execution and more specifically during
exception handling. Due to the fact that the exception entry
implementation uses the SCRATCH CSR as both the current task struct and
as the temporary register to switch the stack and save register, it is
difficult to reliably get the current task struct if we get interrupted
at this specific moment. A fixup-like mechanism allows to mark the
location of the current task struct depending on the entry level
(user/kernel) and the location. This is then used in the SSE assembly to
determine where is located the current task_struct.
Contrary to pseudo NMI [2], SSE does not modifies the way interrupts are
handled and does not adds any overhead to existing code. Moreover, it
provides "true" NMI-like interrupts which can interrupt the kernel at
any time (even in exception handling). This is particularly crucial for
RAS errors which needs to be handled as fast as possible to avoid any
fault propagation. Additionally, SSE event handling is faster that the
standard IRQ handling path with almost half executed instruction (700 vs
1590). Some complementary tests/perf measurements will be done.
For testing purpose, one can use the provided SBI implementation at [3].
This series also needs patch [4] to fix a bug in the PMU driver.
Link: https://lists.riscv.org/g/tech-prs/message/515 [1]
Link: https://lore.kernel.org/lkml/20231023082911.23242-10-luxu.kernel@bytedance.com/T/ [2]
Link: https://github.com/rivosinc/opensbi/tree/dev/cleger/sse [3]
Link: https://lore.kernel.org/linux-arm-kernel/20231026084010.11888-1-alexghiti@rivosinc.com/ [4]
---
Clément Léger (3):
riscv: add SBI SSE extension definitions
riscv: add support for SBI Supervisor Software Events extension
perf: RISC-V: add support for SSE event
arch/riscv/include/asm/asm-prototypes.h | 5 +
arch/riscv/include/asm/sbi.h | 40 ++
arch/riscv/include/asm/sse.h | 94 +++++
arch/riscv/kernel/Makefile | 1 +
arch/riscv/kernel/asm-offsets.c | 17 +
arch/riscv/kernel/entry.S | 156 ++++++++
arch/riscv/kernel/sbi.c | 4 +
arch/riscv/kernel/sse.c | 97 +++++
arch/riscv/kernel/stacktrace.c | 13 +
arch/riscv/kernel/vmlinux.lds.S | 6 +
drivers/firmware/Kconfig | 10 +
drivers/firmware/Makefile | 1 +
drivers/firmware/riscv_sse.c | 496 ++++++++++++++++++++++++
drivers/perf/riscv_pmu_sbi.c | 51 ++-
include/linux/riscv_sse.h | 56 +++
15 files changed, 1038 insertions(+), 9 deletions(-)
create mode 100644 arch/riscv/include/asm/sse.h
create mode 100644 arch/riscv/kernel/sse.c
create mode 100644 drivers/firmware/riscv_sse.c
create mode 100644 include/linux/riscv_sse.h
--
2.42.0
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 25+ messages in thread
* [RFC PATCH 0/3] riscv: add support for SBI Supervisor Software Events
@ 2023-10-26 14:31 ` Clément Léger
0 siblings, 0 replies; 25+ messages in thread
From: Clément Léger @ 2023-10-26 14:31 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Albert Ou, Atish Patra, Anup Patel,
linux-riscv, linux-kernel, linux-arm-kernel
Cc: Clément Léger, Himanshu Chauhan, Xu Lu
The SBI Supervisor Software Events (SSE) extensions provides a mechanism
to inject software events from an SBI implementation to supervisor
software such that it preempts all other supervisor level traps and
interrupts [1].
Various events are defined and can be send asynchronously to supervisor
software (RAS, PMU, DEBUG, Asynchronous page fault) from SBI as well
as platform specific events. Events can be either local (per-hart) or
global. Events can be nested on top of each other based on priority and
can interrupt the kernel at any time.
First patch adds the SSE definitions. Second one adds support for SSE
itself. Implementation is split between arch specific code and generic
part (similarly to what is done for ARM SDEI). Finally, the last patch
add support fro SSE event in the SBI PMU driver. If the SSE event is
available from the SBI then, it will be used instead of the normal
interrupt.
Amongst the specific points that needs to be handle is the interruption
at any point of the kernel execution and more specifically during
exception handling. Due to the fact that the exception entry
implementation uses the SCRATCH CSR as both the current task struct and
as the temporary register to switch the stack and save register, it is
difficult to reliably get the current task struct if we get interrupted
at this specific moment. A fixup-like mechanism allows to mark the
location of the current task struct depending on the entry level
(user/kernel) and the location. This is then used in the SSE assembly to
determine where is located the current task_struct.
Contrary to pseudo NMI [2], SSE does not modifies the way interrupts are
handled and does not adds any overhead to existing code. Moreover, it
provides "true" NMI-like interrupts which can interrupt the kernel at
any time (even in exception handling). This is particularly crucial for
RAS errors which needs to be handled as fast as possible to avoid any
fault propagation. Additionally, SSE event handling is faster that the
standard IRQ handling path with almost half executed instruction (700 vs
1590). Some complementary tests/perf measurements will be done.
For testing purpose, one can use the provided SBI implementation at [3].
This series also needs patch [4] to fix a bug in the PMU driver.
Link: https://lists.riscv.org/g/tech-prs/message/515 [1]
Link: https://lore.kernel.org/lkml/20231023082911.23242-10-luxu.kernel@bytedance.com/T/ [2]
Link: https://github.com/rivosinc/opensbi/tree/dev/cleger/sse [3]
Link: https://lore.kernel.org/linux-arm-kernel/20231026084010.11888-1-alexghiti@rivosinc.com/ [4]
---
Clément Léger (3):
riscv: add SBI SSE extension definitions
riscv: add support for SBI Supervisor Software Events extension
perf: RISC-V: add support for SSE event
arch/riscv/include/asm/asm-prototypes.h | 5 +
arch/riscv/include/asm/sbi.h | 40 ++
arch/riscv/include/asm/sse.h | 94 +++++
arch/riscv/kernel/Makefile | 1 +
arch/riscv/kernel/asm-offsets.c | 17 +
arch/riscv/kernel/entry.S | 156 ++++++++
arch/riscv/kernel/sbi.c | 4 +
arch/riscv/kernel/sse.c | 97 +++++
arch/riscv/kernel/stacktrace.c | 13 +
arch/riscv/kernel/vmlinux.lds.S | 6 +
drivers/firmware/Kconfig | 10 +
drivers/firmware/Makefile | 1 +
drivers/firmware/riscv_sse.c | 496 ++++++++++++++++++++++++
drivers/perf/riscv_pmu_sbi.c | 51 ++-
include/linux/riscv_sse.h | 56 +++
15 files changed, 1038 insertions(+), 9 deletions(-)
create mode 100644 arch/riscv/include/asm/sse.h
create mode 100644 arch/riscv/kernel/sse.c
create mode 100644 drivers/firmware/riscv_sse.c
create mode 100644 include/linux/riscv_sse.h
--
2.42.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 25+ messages in thread
* [RFC PATCH 0/3] riscv: add support for SBI Supervisor Software Events
@ 2023-10-26 14:31 ` Clément Léger
0 siblings, 0 replies; 25+ messages in thread
From: Clément Léger @ 2023-10-26 14:31 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Albert Ou, Atish Patra, Anup Patel,
linux-riscv, linux-kernel, linux-arm-kernel
Cc: Clément Léger, Himanshu Chauhan, Xu Lu
The SBI Supervisor Software Events (SSE) extensions provides a mechanism
to inject software events from an SBI implementation to supervisor
software such that it preempts all other supervisor level traps and
interrupts [1].
Various events are defined and can be send asynchronously to supervisor
software (RAS, PMU, DEBUG, Asynchronous page fault) from SBI as well
as platform specific events. Events can be either local (per-hart) or
global. Events can be nested on top of each other based on priority and
can interrupt the kernel at any time.
First patch adds the SSE definitions. Second one adds support for SSE
itself. Implementation is split between arch specific code and generic
part (similarly to what is done for ARM SDEI). Finally, the last patch
add support fro SSE event in the SBI PMU driver. If the SSE event is
available from the SBI then, it will be used instead of the normal
interrupt.
Amongst the specific points that needs to be handle is the interruption
at any point of the kernel execution and more specifically during
exception handling. Due to the fact that the exception entry
implementation uses the SCRATCH CSR as both the current task struct and
as the temporary register to switch the stack and save register, it is
difficult to reliably get the current task struct if we get interrupted
at this specific moment. A fixup-like mechanism allows to mark the
location of the current task struct depending on the entry level
(user/kernel) and the location. This is then used in the SSE assembly to
determine where is located the current task_struct.
Contrary to pseudo NMI [2], SSE does not modifies the way interrupts are
handled and does not adds any overhead to existing code. Moreover, it
provides "true" NMI-like interrupts which can interrupt the kernel at
any time (even in exception handling). This is particularly crucial for
RAS errors which needs to be handled as fast as possible to avoid any
fault propagation. Additionally, SSE event handling is faster that the
standard IRQ handling path with almost half executed instruction (700 vs
1590). Some complementary tests/perf measurements will be done.
For testing purpose, one can use the provided SBI implementation at [3].
This series also needs patch [4] to fix a bug in the PMU driver.
Link: https://lists.riscv.org/g/tech-prs/message/515 [1]
Link: https://lore.kernel.org/lkml/20231023082911.23242-10-luxu.kernel@bytedance.com/T/ [2]
Link: https://github.com/rivosinc/opensbi/tree/dev/cleger/sse [3]
Link: https://lore.kernel.org/linux-arm-kernel/20231026084010.11888-1-alexghiti@rivosinc.com/ [4]
---
Clément Léger (3):
riscv: add SBI SSE extension definitions
riscv: add support for SBI Supervisor Software Events extension
perf: RISC-V: add support for SSE event
arch/riscv/include/asm/asm-prototypes.h | 5 +
arch/riscv/include/asm/sbi.h | 40 ++
arch/riscv/include/asm/sse.h | 94 +++++
arch/riscv/kernel/Makefile | 1 +
arch/riscv/kernel/asm-offsets.c | 17 +
arch/riscv/kernel/entry.S | 156 ++++++++
arch/riscv/kernel/sbi.c | 4 +
arch/riscv/kernel/sse.c | 97 +++++
arch/riscv/kernel/stacktrace.c | 13 +
arch/riscv/kernel/vmlinux.lds.S | 6 +
drivers/firmware/Kconfig | 10 +
drivers/firmware/Makefile | 1 +
drivers/firmware/riscv_sse.c | 496 ++++++++++++++++++++++++
drivers/perf/riscv_pmu_sbi.c | 51 ++-
include/linux/riscv_sse.h | 56 +++
15 files changed, 1038 insertions(+), 9 deletions(-)
create mode 100644 arch/riscv/include/asm/sse.h
create mode 100644 arch/riscv/kernel/sse.c
create mode 100644 drivers/firmware/riscv_sse.c
create mode 100644 include/linux/riscv_sse.h
--
2.42.0
^ permalink raw reply [flat|nested] 25+ messages in thread
* [RFC PATCH 1/3] riscv: add SBI SSE extension definitions
2023-10-26 14:31 ` Clément Léger
(?)
@ 2023-10-26 14:31 ` Clément Léger
-1 siblings, 0 replies; 25+ messages in thread
From: Clément Léger @ 2023-10-26 14:31 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Albert Ou, Atish Patra, Anup Patel,
linux-riscv, linux-kernel, linux-arm-kernel
Cc: Clément Léger, Himanshu Chauhan, Xu Lu
Add needed definitions for SBI Supervisor Software Events extension [1].
This extension enables the SBI to inject events into supervisor software
much like ARM SDEI.
[1] https://lists.riscv.org/g/tech-prs/message/515
Signed-off-by: Clément Léger <cleger@rivosinc.com>
---
arch/riscv/include/asm/sbi.h | 38 ++++++++++++++++++++++++++++++++++++
arch/riscv/kernel/sbi.c | 4 ++++
2 files changed, 42 insertions(+)
diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
index 5b4a1bf5f439..2e99cafe7fed 100644
--- a/arch/riscv/include/asm/sbi.h
+++ b/arch/riscv/include/asm/sbi.h
@@ -30,6 +30,7 @@ enum sbi_ext_id {
SBI_EXT_HSM = 0x48534D,
SBI_EXT_SRST = 0x53525354,
SBI_EXT_PMU = 0x504D55,
+ SBI_EXT_SSE = 0x535345,
/* Experimentals extensions must lie within this range */
SBI_EXT_EXPERIMENTAL_START = 0x08000000,
@@ -236,6 +237,40 @@ enum sbi_pmu_ctr_type {
/* Flags defined for counter stop function */
#define SBI_PMU_STOP_FLAG_RESET (1 << 0)
+enum sbi_ext_sse_fid {
+ SBI_SSE_EVENT_ATTR_GET = 0,
+ SBI_SSE_EVENT_ATTR_SET,
+ SBI_SSE_EVENT_REGISTER,
+ SBI_SSE_EVENT_UNREGISTER,
+ SBI_SSE_EVENT_ENABLE,
+ SBI_SSE_EVENT_DISABLE,
+ SBI_SSE_EVENT_COMPLETE,
+ SBI_SSE_EVENT_SIGNAL,
+};
+
+#define SBI_SSE_EVENT_LOCAL_RAS 0x00000000
+#define SBI_SSE_EVENT_GLOBAL_RAS 0x00008000
+#define SBI_SSE_EVENT_LOCAL_ASYNC_PF 0x00010000
+#define SBI_SSE_EVENT_LOCAL_PMU 0x00010001
+#define SBI_SSE_EVENT_LOCAL_DEBUG 0xffff3fff
+#define SBI_SSE_EVENT_GLOBAL_DEBUG 0xffffbfff
+
+#define SBI_SSE_EVENT_GLOBAL (1 << 15)
+#define SBI_SSE_EVENT_PLATFORM (1 << 14)
+
+enum sbi_sse_event_attr {
+ SBI_SSE_EVENT_ATTR_STATE = 0,
+ SBI_SSE_EVENT_ATTR_PRIORITY,
+ SBI_SSE_EVENT_ATTR_INJECTION,
+ SBI_SSE_EVENT_ATTR_HART_ID,
+ SBI_SSE_EVENT_ATTR_RAW_PENDING_STATUS,
+};
+
+enum sbi_sse_event_handler_sts {
+ SBI_SSE_HANDLER_SUCCESS = 0,
+ SBI_SSE_HANDLER_FAILED,
+};
+
#define SBI_SPEC_VERSION_DEFAULT 0x1
#define SBI_SPEC_VERSION_MAJOR_SHIFT 24
#define SBI_SPEC_VERSION_MAJOR_MASK 0x7f
@@ -251,6 +286,9 @@ enum sbi_pmu_ctr_type {
#define SBI_ERR_ALREADY_AVAILABLE -6
#define SBI_ERR_ALREADY_STARTED -7
#define SBI_ERR_ALREADY_STOPPED -8
+#define SBI_ERR_INVALID_STATE -10
+#define SBI_ERR_BAD_RANGE -11
+#define SBI_ERR_BUSY -12
extern unsigned long sbi_spec_version;
struct sbiret {
diff --git a/arch/riscv/kernel/sbi.c b/arch/riscv/kernel/sbi.c
index c672c8ba9a2a..13b63b383d4e 100644
--- a/arch/riscv/kernel/sbi.c
+++ b/arch/riscv/kernel/sbi.c
@@ -56,9 +56,13 @@ int sbi_err_map_linux_errno(int err)
case SBI_ERR_DENIED:
return -EPERM;
case SBI_ERR_INVALID_PARAM:
+ case SBI_ERR_BAD_RANGE:
+ case SBI_ERR_INVALID_STATE:
return -EINVAL;
case SBI_ERR_INVALID_ADDRESS:
return -EFAULT;
+ case SBI_ERR_BUSY:
+ return -EBUSY;
case SBI_ERR_NOT_SUPPORTED:
case SBI_ERR_FAILURE:
default:
--
2.42.0
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [RFC PATCH 1/3] riscv: add SBI SSE extension definitions
@ 2023-10-26 14:31 ` Clément Léger
0 siblings, 0 replies; 25+ messages in thread
From: Clément Léger @ 2023-10-26 14:31 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Albert Ou, Atish Patra, Anup Patel,
linux-riscv, linux-kernel, linux-arm-kernel
Cc: Clément Léger, Himanshu Chauhan, Xu Lu
Add needed definitions for SBI Supervisor Software Events extension [1].
This extension enables the SBI to inject events into supervisor software
much like ARM SDEI.
[1] https://lists.riscv.org/g/tech-prs/message/515
Signed-off-by: Clément Léger <cleger@rivosinc.com>
---
arch/riscv/include/asm/sbi.h | 38 ++++++++++++++++++++++++++++++++++++
arch/riscv/kernel/sbi.c | 4 ++++
2 files changed, 42 insertions(+)
diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
index 5b4a1bf5f439..2e99cafe7fed 100644
--- a/arch/riscv/include/asm/sbi.h
+++ b/arch/riscv/include/asm/sbi.h
@@ -30,6 +30,7 @@ enum sbi_ext_id {
SBI_EXT_HSM = 0x48534D,
SBI_EXT_SRST = 0x53525354,
SBI_EXT_PMU = 0x504D55,
+ SBI_EXT_SSE = 0x535345,
/* Experimentals extensions must lie within this range */
SBI_EXT_EXPERIMENTAL_START = 0x08000000,
@@ -236,6 +237,40 @@ enum sbi_pmu_ctr_type {
/* Flags defined for counter stop function */
#define SBI_PMU_STOP_FLAG_RESET (1 << 0)
+enum sbi_ext_sse_fid {
+ SBI_SSE_EVENT_ATTR_GET = 0,
+ SBI_SSE_EVENT_ATTR_SET,
+ SBI_SSE_EVENT_REGISTER,
+ SBI_SSE_EVENT_UNREGISTER,
+ SBI_SSE_EVENT_ENABLE,
+ SBI_SSE_EVENT_DISABLE,
+ SBI_SSE_EVENT_COMPLETE,
+ SBI_SSE_EVENT_SIGNAL,
+};
+
+#define SBI_SSE_EVENT_LOCAL_RAS 0x00000000
+#define SBI_SSE_EVENT_GLOBAL_RAS 0x00008000
+#define SBI_SSE_EVENT_LOCAL_ASYNC_PF 0x00010000
+#define SBI_SSE_EVENT_LOCAL_PMU 0x00010001
+#define SBI_SSE_EVENT_LOCAL_DEBUG 0xffff3fff
+#define SBI_SSE_EVENT_GLOBAL_DEBUG 0xffffbfff
+
+#define SBI_SSE_EVENT_GLOBAL (1 << 15)
+#define SBI_SSE_EVENT_PLATFORM (1 << 14)
+
+enum sbi_sse_event_attr {
+ SBI_SSE_EVENT_ATTR_STATE = 0,
+ SBI_SSE_EVENT_ATTR_PRIORITY,
+ SBI_SSE_EVENT_ATTR_INJECTION,
+ SBI_SSE_EVENT_ATTR_HART_ID,
+ SBI_SSE_EVENT_ATTR_RAW_PENDING_STATUS,
+};
+
+enum sbi_sse_event_handler_sts {
+ SBI_SSE_HANDLER_SUCCESS = 0,
+ SBI_SSE_HANDLER_FAILED,
+};
+
#define SBI_SPEC_VERSION_DEFAULT 0x1
#define SBI_SPEC_VERSION_MAJOR_SHIFT 24
#define SBI_SPEC_VERSION_MAJOR_MASK 0x7f
@@ -251,6 +286,9 @@ enum sbi_pmu_ctr_type {
#define SBI_ERR_ALREADY_AVAILABLE -6
#define SBI_ERR_ALREADY_STARTED -7
#define SBI_ERR_ALREADY_STOPPED -8
+#define SBI_ERR_INVALID_STATE -10
+#define SBI_ERR_BAD_RANGE -11
+#define SBI_ERR_BUSY -12
extern unsigned long sbi_spec_version;
struct sbiret {
diff --git a/arch/riscv/kernel/sbi.c b/arch/riscv/kernel/sbi.c
index c672c8ba9a2a..13b63b383d4e 100644
--- a/arch/riscv/kernel/sbi.c
+++ b/arch/riscv/kernel/sbi.c
@@ -56,9 +56,13 @@ int sbi_err_map_linux_errno(int err)
case SBI_ERR_DENIED:
return -EPERM;
case SBI_ERR_INVALID_PARAM:
+ case SBI_ERR_BAD_RANGE:
+ case SBI_ERR_INVALID_STATE:
return -EINVAL;
case SBI_ERR_INVALID_ADDRESS:
return -EFAULT;
+ case SBI_ERR_BUSY:
+ return -EBUSY;
case SBI_ERR_NOT_SUPPORTED:
case SBI_ERR_FAILURE:
default:
--
2.42.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [RFC PATCH 1/3] riscv: add SBI SSE extension definitions
@ 2023-10-26 14:31 ` Clément Léger
0 siblings, 0 replies; 25+ messages in thread
From: Clément Léger @ 2023-10-26 14:31 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Albert Ou, Atish Patra, Anup Patel,
linux-riscv, linux-kernel, linux-arm-kernel
Cc: Clément Léger, Himanshu Chauhan, Xu Lu
Add needed definitions for SBI Supervisor Software Events extension [1].
This extension enables the SBI to inject events into supervisor software
much like ARM SDEI.
[1] https://lists.riscv.org/g/tech-prs/message/515
Signed-off-by: Clément Léger <cleger@rivosinc.com>
---
arch/riscv/include/asm/sbi.h | 38 ++++++++++++++++++++++++++++++++++++
arch/riscv/kernel/sbi.c | 4 ++++
2 files changed, 42 insertions(+)
diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
index 5b4a1bf5f439..2e99cafe7fed 100644
--- a/arch/riscv/include/asm/sbi.h
+++ b/arch/riscv/include/asm/sbi.h
@@ -30,6 +30,7 @@ enum sbi_ext_id {
SBI_EXT_HSM = 0x48534D,
SBI_EXT_SRST = 0x53525354,
SBI_EXT_PMU = 0x504D55,
+ SBI_EXT_SSE = 0x535345,
/* Experimentals extensions must lie within this range */
SBI_EXT_EXPERIMENTAL_START = 0x08000000,
@@ -236,6 +237,40 @@ enum sbi_pmu_ctr_type {
/* Flags defined for counter stop function */
#define SBI_PMU_STOP_FLAG_RESET (1 << 0)
+enum sbi_ext_sse_fid {
+ SBI_SSE_EVENT_ATTR_GET = 0,
+ SBI_SSE_EVENT_ATTR_SET,
+ SBI_SSE_EVENT_REGISTER,
+ SBI_SSE_EVENT_UNREGISTER,
+ SBI_SSE_EVENT_ENABLE,
+ SBI_SSE_EVENT_DISABLE,
+ SBI_SSE_EVENT_COMPLETE,
+ SBI_SSE_EVENT_SIGNAL,
+};
+
+#define SBI_SSE_EVENT_LOCAL_RAS 0x00000000
+#define SBI_SSE_EVENT_GLOBAL_RAS 0x00008000
+#define SBI_SSE_EVENT_LOCAL_ASYNC_PF 0x00010000
+#define SBI_SSE_EVENT_LOCAL_PMU 0x00010001
+#define SBI_SSE_EVENT_LOCAL_DEBUG 0xffff3fff
+#define SBI_SSE_EVENT_GLOBAL_DEBUG 0xffffbfff
+
+#define SBI_SSE_EVENT_GLOBAL (1 << 15)
+#define SBI_SSE_EVENT_PLATFORM (1 << 14)
+
+enum sbi_sse_event_attr {
+ SBI_SSE_EVENT_ATTR_STATE = 0,
+ SBI_SSE_EVENT_ATTR_PRIORITY,
+ SBI_SSE_EVENT_ATTR_INJECTION,
+ SBI_SSE_EVENT_ATTR_HART_ID,
+ SBI_SSE_EVENT_ATTR_RAW_PENDING_STATUS,
+};
+
+enum sbi_sse_event_handler_sts {
+ SBI_SSE_HANDLER_SUCCESS = 0,
+ SBI_SSE_HANDLER_FAILED,
+};
+
#define SBI_SPEC_VERSION_DEFAULT 0x1
#define SBI_SPEC_VERSION_MAJOR_SHIFT 24
#define SBI_SPEC_VERSION_MAJOR_MASK 0x7f
@@ -251,6 +286,9 @@ enum sbi_pmu_ctr_type {
#define SBI_ERR_ALREADY_AVAILABLE -6
#define SBI_ERR_ALREADY_STARTED -7
#define SBI_ERR_ALREADY_STOPPED -8
+#define SBI_ERR_INVALID_STATE -10
+#define SBI_ERR_BAD_RANGE -11
+#define SBI_ERR_BUSY -12
extern unsigned long sbi_spec_version;
struct sbiret {
diff --git a/arch/riscv/kernel/sbi.c b/arch/riscv/kernel/sbi.c
index c672c8ba9a2a..13b63b383d4e 100644
--- a/arch/riscv/kernel/sbi.c
+++ b/arch/riscv/kernel/sbi.c
@@ -56,9 +56,13 @@ int sbi_err_map_linux_errno(int err)
case SBI_ERR_DENIED:
return -EPERM;
case SBI_ERR_INVALID_PARAM:
+ case SBI_ERR_BAD_RANGE:
+ case SBI_ERR_INVALID_STATE:
return -EINVAL;
case SBI_ERR_INVALID_ADDRESS:
return -EFAULT;
+ case SBI_ERR_BUSY:
+ return -EBUSY;
case SBI_ERR_NOT_SUPPORTED:
case SBI_ERR_FAILURE:
default:
--
2.42.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [RFC PATCH 2/3] riscv: add support for SBI Supervisor Software Events extension
2023-10-26 14:31 ` Clément Léger
(?)
@ 2023-10-26 14:31 ` Clément Léger
-1 siblings, 0 replies; 25+ messages in thread
From: Clément Léger @ 2023-10-26 14:31 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Albert Ou, Atish Patra, Anup Patel,
linux-riscv, linux-kernel, linux-arm-kernel
Cc: Clément Léger, Himanshu Chauhan, Xu Lu
The SBI SSE extension allows the supervisor software to be notified by
the SBI of specific events that are not maskable. The context switch is
handled entirely by the firmware which will save all registers into a
supervisor provided context and restore a set of registers from this
same context. This context contains all the RV GP registers and thus
will also switch the stack.
Since each events (and each CPU for local events) have their own
context (and thus their own set of register), allocate a stack for each
event (and for each cpu for local events). This can be modified if
needed to use a single stack per cpu for SSE event but will require a
bit more additional plumbing.
When completing the event, if we were coming from kernel with interrupts
disabled, simply return there. If coming from userspace or kernel with
interrupts enabled, simulate an interrupt exception to allow delivery of
signal to user task. For instance this can happen, when a RAS event has
been generated and a SIGBUS has been sent to a task.
Signed-off-by: Clément Léger <cleger@rivosinc.com>
---
arch/riscv/include/asm/asm-prototypes.h | 5 +
arch/riscv/include/asm/sse.h | 94 +++++
arch/riscv/kernel/Makefile | 1 +
arch/riscv/kernel/asm-offsets.c | 17 +
arch/riscv/kernel/entry.S | 156 ++++++++
arch/riscv/kernel/sse.c | 97 +++++
arch/riscv/kernel/stacktrace.c | 13 +
arch/riscv/kernel/vmlinux.lds.S | 6 +
drivers/firmware/Kconfig | 10 +
drivers/firmware/Makefile | 1 +
drivers/firmware/riscv_sse.c | 496 ++++++++++++++++++++++++
include/linux/riscv_sse.h | 56 +++
12 files changed, 952 insertions(+)
create mode 100644 arch/riscv/include/asm/sse.h
create mode 100644 arch/riscv/kernel/sse.c
create mode 100644 drivers/firmware/riscv_sse.c
create mode 100644 include/linux/riscv_sse.h
diff --git a/arch/riscv/include/asm/asm-prototypes.h b/arch/riscv/include/asm/asm-prototypes.h
index 61ba8ed43d8f..c622ed1db33c 100644
--- a/arch/riscv/include/asm/asm-prototypes.h
+++ b/arch/riscv/include/asm/asm-prototypes.h
@@ -3,6 +3,8 @@
#define _ASM_RISCV_PROTOTYPES_H
#include <linux/ftrace.h>
+#include <linux/riscv_sse.h>
+#include <asm/sse.h>
#include <asm-generic/asm-prototypes.h>
long long __lshrti3(long long a, int b);
@@ -29,5 +31,8 @@ asmlinkage unsigned long get_overflow_stack(void);
asmlinkage void handle_bad_stack(struct pt_regs *regs);
asmlinkage void do_page_fault(struct pt_regs *regs);
asmlinkage void do_irq(struct pt_regs *regs);
+asmlinkage unsigned long do_sse(unsigned long evt,
+ struct sse_interrupted_state *i_state,
+ sse_event_handler *handler, void *arg);
#endif /* _ASM_RISCV_PROTOTYPES_H */
diff --git a/arch/riscv/include/asm/sse.h b/arch/riscv/include/asm/sse.h
new file mode 100644
index 000000000000..e663224296df
--- /dev/null
+++ b/arch/riscv/include/asm/sse.h
@@ -0,0 +1,94 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2023 Rivos Inc.
+ */
+#ifndef __ASM_SSE_H
+#define __ASM_SSE_H
+
+#include <linux/riscv_sse.h>
+
+
+struct sse_interrupted_state {
+ unsigned long pc;
+ unsigned long ra;
+ unsigned long sp;
+ unsigned long gp;
+ unsigned long tp;
+ unsigned long t0;
+ unsigned long t1;
+ unsigned long t2;
+ unsigned long s0;
+ unsigned long s1;
+ unsigned long a0;
+ unsigned long a1;
+ unsigned long a2;
+ unsigned long a3;
+ unsigned long a4;
+ unsigned long a5;
+ unsigned long a6;
+ unsigned long a7;
+ unsigned long s2;
+ unsigned long s3;
+ unsigned long s4;
+ unsigned long s5;
+ unsigned long s6;
+ unsigned long s7;
+ unsigned long s8;
+ unsigned long s9;
+ unsigned long s10;
+ unsigned long s11;
+ unsigned long t3;
+ unsigned long t4;
+ unsigned long t5;
+ unsigned long t6;
+ unsigned long exec_mode;
+};
+
+struct sse_entry_state {
+ unsigned long pc;
+ unsigned long ra;
+ unsigned long sp;
+ unsigned long gp;
+ unsigned long tp;
+ unsigned long t0;
+ unsigned long t1;
+ unsigned long t2;
+ unsigned long s0;
+ unsigned long s1;
+ unsigned long a0;
+ unsigned long a1;
+ unsigned long a2;
+ unsigned long a3;
+ unsigned long a4;
+ unsigned long a5;
+ unsigned long a6;
+ unsigned long a7;
+ unsigned long s2;
+ unsigned long s3;
+ unsigned long s4;
+ unsigned long s5;
+ unsigned long s6;
+ unsigned long s7;
+ unsigned long s8;
+ unsigned long s9;
+ unsigned long s10;
+ unsigned long s11;
+ unsigned long t3;
+ unsigned long t4;
+ unsigned long t5;
+ unsigned long t6;
+};
+
+struct sse_handler_context {
+ struct sse_entry_state e_state;
+ struct sse_interrupted_state i_state;
+};
+
+unsigned long *sse_stack_alloc(unsigned int cpu, unsigned int size);
+void sse_stack_free(unsigned long *stack);
+
+struct sse_handler_context;
+void sse_handler_context_init(struct sse_handler_context *ctx, void *stack,
+ u32 evt, sse_event_handler *handler, void *arg);
+
+#endif
diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile
index 95cf25d48405..571b2aaf669b 100644
--- a/arch/riscv/kernel/Makefile
+++ b/arch/riscv/kernel/Makefile
@@ -79,6 +79,7 @@ obj-$(CONFIG_DYNAMIC_FTRACE) += mcount-dyn.o
obj-$(CONFIG_PERF_EVENTS) += perf_callchain.o
obj-$(CONFIG_HAVE_PERF_REGS) += perf_regs.o
obj-$(CONFIG_RISCV_SBI) += sbi.o
+obj-$(CONFIG_RISCV_SSE) += sse.o
ifeq ($(CONFIG_RISCV_SBI), y)
obj-$(CONFIG_SMP) += sbi-ipi.o
obj-$(CONFIG_SMP) += cpu_ops_sbi.o
diff --git a/arch/riscv/kernel/asm-offsets.c b/arch/riscv/kernel/asm-offsets.c
index d6a75aac1d27..1776448e67d2 100644
--- a/arch/riscv/kernel/asm-offsets.c
+++ b/arch/riscv/kernel/asm-offsets.c
@@ -14,7 +14,10 @@
#include <asm/thread_info.h>
#include <asm/ptrace.h>
#include <asm/cpu_ops_sbi.h>
+#include <asm/sbi.h>
+#include <asm/sse.h>
#include <asm/suspend.h>
+#include <asm/stacktrace.h>
void asm_offsets(void);
@@ -479,4 +482,18 @@ void asm_offsets(void)
OFFSET(KERNEL_MAP_VIRT_ADDR, kernel_mapping, virt_addr);
OFFSET(SBI_HART_BOOT_TASK_PTR_OFFSET, sbi_hart_boot_data, task_ptr);
OFFSET(SBI_HART_BOOT_STACK_PTR_OFFSET, sbi_hart_boot_data, stack_ptr);
+
+ OFFSET(SSE_INTERRUPTED_EXEC_MODE, sse_interrupted_state, exec_mode);
+ OFFSET(SSE_INTERRUPTED_S0, sse_interrupted_state, s0);
+ OFFSET(SSE_INTERRUPTED_TP, sse_interrupted_state, tp);
+ OFFSET(SSE_INTERRUPTED_PC, sse_interrupted_state, pc);
+
+ DEFINE(STACKFRAME_SIZE_ON_STACK, ALIGN(sizeof(struct stackframe),
+ STACK_ALIGN));
+ OFFSET(STACKFRAME_FP, stackframe, fp);
+ OFFSET(STACKFRAME_RA, stackframe, ra);
+ DEFINE(SBI_EXT_SSE, SBI_EXT_SSE);
+ DEFINE(SBI_SSE_EVENT_COMPLETE, SBI_SSE_EVENT_COMPLETE);
+
+ DEFINE(ASM_PAGE_OFFSET, CONFIG_PAGE_OFFSET);
}
diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
index 143a2bb3e697..4adf7b21a568 100644
--- a/arch/riscv/kernel/entry.S
+++ b/arch/riscv/kernel/entry.S
@@ -14,17 +14,49 @@
#include <asm/asm-offsets.h>
#include <asm/errata_list.h>
+#ifdef CONFIG_RISCV_SSE
+#define IN_TP 0
+#define IN_SSCRATCH 1
+#define PRV_U_TASK_LOC(loc) (loc << 0)
+#define PRV_S_TASK_LOC(loc) (loc << 1)
+#define TASK_LOC(s_loc, u_loc) (PRV_U_TASK_LOC(u_loc) | PRV_S_TASK_LOC(s_loc))
+#define TASK_LOC_ENTRY_SIZE (RISCV_SZPTR + 1)
+
+/* __SSE_TASK_LOC - Annotate e'xception code with the location of current task
+ * struct when coming from S/U mode
+ *
+ * When entering handle_exception(), the current task struct is located either
+ * in tp or sscratch depending on interrupted mode. Since SSE handlers can be
+ * triggered at any time during the execution of the kernel, we need to be able
+ * to retrieve the current task struct even if in handle_exception. This macro
+ * create an entry into the __task_loc section that holds the location of
+ * the current task struct for the subsequent assembly located after that up to
+ * _ret_from_exception_end or the next __TASK_LOC.
+ */
+#define __SSE_TASK_LOC(s_loc, u_loc) \
+ .pushsection __task_loc,"a"; \
+ RISCV_PTR 99f; \
+ .byte TASK_LOC(s_loc, u_loc); \
+ .popsection; \
+ 99:
+#else
+#define __SSE_TASK_LOC(s_loc, u_loc)
+#endif
+
SYM_CODE_START(handle_exception)
/*
* If coming from userspace, preserve the user thread pointer and load
* the kernel thread pointer. If we came from the kernel, the scratch
* register will contain 0, and we should continue on the current TP.
*/
+__SSE_TASK_LOC(IN_TP, IN_SSCRATCH)
csrrw tp, CSR_SCRATCH, tp
+__SSE_TASK_LOC(IN_SSCRATCH, IN_TP)
bnez tp, _save_context
_restore_kernel_tpsp:
csrr tp, CSR_SCRATCH
+__SSE_TASK_LOC(IN_TP, IN_TP)
REG_S sp, TASK_TI_KERNEL_SP(tp)
#ifdef CONFIG_VMAP_STACK
@@ -151,6 +183,7 @@ SYM_CODE_START_NOALIGN(ret_from_exception)
REG_SC x0, a2, PT_EPC(sp)
csrw CSR_STATUS, a0
+__SSE_TASK_LOC(IN_TP, IN_SSCRATCH)
csrw CSR_EPC, a2
REG_L x1, PT_RA(sp)
@@ -166,6 +199,7 @@ SYM_CODE_START_NOALIGN(ret_from_exception)
#else
sret
#endif
+_ret_from_exception_end:
SYM_CODE_END(ret_from_exception)
#ifdef CONFIG_VMAP_STACK
@@ -268,6 +302,125 @@ SYM_CODE_START(ret_from_fork)
tail syscall_exit_to_user_mode
SYM_CODE_END(ret_from_fork)
+#ifdef CONFIG_RISCV_SSE
+SYM_CODE_START(handle_sse)
+ move s3, a0
+ /*
+ * Depending on where the sse event interrupted the kernel execution,
+ * sscratch content might be 0 or not even if kernel was interrupted.
+ * This might happen if the sse event was triggered while in
+ * handle_exception() right after entry. In that case, sscratch might
+ * contain 0 if coming from kernel. In order to handle that easily,
+ * simply save sscratch content and restore it when returning from sse
+ * event.
+ */
+ csrr s4, CSR_SSCRATCH
+ /*
+ * Save CAUSE and TVAL in case of nested exceptions, EPC/STATUS are
+ * already saved/restored by handle_exception() for nested exceptions.
+ */
+ csrr s5, CSR_SCAUSE
+ csrr s6, CSR_STVAL
+ REG_L t0, SSE_INTERRUPTED_EXEC_MODE(a1)
+ andi t1, t0, 0x1
+
+ /* Userspace was interrupted, simply restore TP from scratch */
+ bnez t1, _sse_from_kernel
+ move tp, s4
+ j _call_do_sse
+
+_sse_from_kernel:
+ REG_L t0, SSE_INTERRUPTED_PC(a1)
+#ifdef CONFIG_FRAME_POINTER
+ /*
+ * Else, kernel was interrupted and we will create a correct stack frame
+ * from interrupted context.
+ */
+ addi sp, sp, -(STACKFRAME_SIZE_ON_STACK)
+ REG_L t2, SSE_INTERRUPTED_S0(a1)
+ REG_S t2, STACKFRAME_FP(sp)
+ REG_S t0, STACKFRAME_RA(sp)
+ addi s0, sp, STACKFRAME_SIZE_ON_STACK
+#endif
+ REG_L tp, SSE_INTERRUPTED_TP(a1)
+
+ /*
+ * If interrupting the kernel during exception handling
+ * (see handle_exception), then, we might have tp either in SSCRATCH or
+ * in tp, this part is non regular and requires some more work to
+ * determine were is located the current task.
+ */
+ la t1, handle_exception
+ la t2, _ret_from_exception_end
+ bltu t0, t1, _call_do_sse
+ bltu t0, t2, _sse_exception_slow_path
+
+_call_do_sse:
+
+ /* TODO: remove this sanity check once sure everything works ! */
+ li t1, ASM_PAGE_OFFSET
+ bgeu tp, t1, _tp_ok
+ la tp, init_task
+ la a0, sse_panic_string
+ la t0, panic
+ jalr t0
+_tp_ok:
+
+ csrw CSR_SCRATCH, x0
+ la ra, ret_from_sse
+ tail do_sse
+
+.global ret_from_sse
+ret_from_sse:
+ /* Restore saved CSRs */
+ csrw CSR_SSCRATCH, s4
+ csrw CSR_SCAUSE, s5
+ csrw CSR_STVAL, s6
+ li a2, 0
+ move a1, a0
+ move a0, s3
+ li a7, SBI_EXT_SSE
+ li a6, SBI_SSE_EVENT_COMPLETE
+ ecall
+
+/*
+ * t0 contains interrupted pc
+ * t1 contains handle_exception address
+ * When called, t0 must be in [handle_exception, _ret_from_exception_end[
+ */
+_sse_exception_slow_path:
+ la t3, __stop___task_loc
+ add t3, t3, -TASK_LOC_ENTRY_SIZE
+ la t4, __start___task_loc
+ /*
+ * Reverse iterate the task location section to find where is located
+ * the task struct
+ */
+1:
+ REG_L t2, 0(t3)
+ bgeu t0, t2, 2f
+ add t3, t3, -TASK_LOC_ENTRY_SIZE
+ bgeu t3, t4, 1b
+
+2:
+ lbu t2, RISCV_SZPTR(t3)
+
+ /* Get the value of SR_SPP */
+ csrr t1, CSR_SSTATUS
+ andi t1, t1, SR_SPP
+ snez t1, t1
+
+ srl t2, t2, t1
+ andi t2, t2, 1
+ beqz t2, _call_do_sse
+
+_restore_tp_from_sscratch:
+ csrr tp, CSR_SCRATCH
+ j _call_do_sse
+
+SYM_CODE_END(handle_sse)
+#endif
+
/*
* Integer register context switch
* The callee-saved registers must be saved and restored.
@@ -351,3 +504,6 @@ SYM_CODE_START(__user_rt_sigreturn)
ecall
SYM_CODE_END(__user_rt_sigreturn)
#endif
+
+sse_panic_string:
+ .ascii "SSE TP is invalid, last state %px"
diff --git a/arch/riscv/kernel/sse.c b/arch/riscv/kernel/sse.c
new file mode 100644
index 000000000000..69c2607d9dbf
--- /dev/null
+++ b/arch/riscv/kernel/sse.c
@@ -0,0 +1,97 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2023 Rivos Inc.
+ */
+#include <linux/nmi.h>
+#include <linux/bitfield.h>
+#include <linux/riscv_sse.h>
+
+#include <asm/irq_stack.h>
+#include <asm/sbi.h>
+#include <asm/sse.h>
+
+#define SSE_PRIVILEGE_MODE_BIT BIT(0)
+#define SSE_SPIE_BIT BIT(2)
+
+#define sse_privilege_mode(exec_mode) FIELD_GET(SSE_PRIVILEGE_MODE_BIT, exec_mode)
+#define sse_spie(exec_mode) FIELD_GET(SSE_SPIE_BIT, exec_mode)
+
+register unsigned long gp_in_global __asm__("gp");
+
+extern asmlinkage void handle_exception(void);
+extern asmlinkage void handle_sse(unsigned long evt,
+ struct sse_interrupted_state *i_state,
+ sse_event_handler *handler, void *arg);
+
+static void sse_get_pt_regs(struct sse_interrupted_state *i_state,
+ struct pt_regs *regs)
+{
+ unsigned long sstatus = csr_read(CSR_SSTATUS);
+
+ memcpy(regs, i_state, offsetof(struct sse_interrupted_state, exec_mode));
+ regs->status = sstatus & ~SR_SPP;
+ regs->status |= FIELD_PREP(SR_SPP, sse_privilege_mode(i_state->exec_mode));
+}
+
+unsigned long do_sse(unsigned long evt, struct sse_interrupted_state *i_state,
+ sse_event_handler *handler, void *arg)
+{
+ int ret;
+ struct pt_regs regs;
+
+ nmi_enter();
+
+ sse_get_pt_regs(i_state, ®s);
+ ret = handler(evt, arg, ®s);
+ if (ret)
+ pr_warn("event %lx handler failed with error %d\n", evt, ret);
+
+ /* The SSE delivery path does not uses the "standard" exception path and
+ * thus does not process any pending signal/softirqs. Some drivers might
+ * enqueue pending work that needs to be handled as soon as possible.
+ * For that purpose, set the software interrupt pending bit
+ */
+ csr_set(CSR_IP, IE_SIE);
+
+ nmi_exit();
+
+ return ret ? SBI_SSE_HANDLER_FAILED : SBI_SSE_HANDLER_SUCCESS;
+}
+
+#ifdef CONFIG_VMAP_STACK
+unsigned long *sse_stack_alloc(unsigned int cpu, unsigned int size)
+{
+ return arch_alloc_vmap_stack(size, cpu_to_node(cpu));
+}
+
+void sse_stack_free(unsigned long *stack)
+{
+ vfree(stack);
+}
+#else /* CONFIG_VMAP_STACK */
+
+unsigned long *sse_stack_alloc(unsigned int cpu, unsigned int size)
+{
+ return kmalloc(size, GFP_KERNEL);
+}
+
+void sse_stack_free(unsigned long *stack)
+{
+ kfree(stack);
+}
+
+#endif /* CONFIG_VMAP_STACK */
+
+void sse_handler_context_init(struct sse_handler_context *ctx, void *stack,
+ u32 evt, sse_event_handler *handler, void *arg)
+{
+ ctx->e_state.pc = (unsigned long)handle_sse;
+ ctx->e_state.gp = gp_in_global;
+ ctx->e_state.sp = (unsigned long)stack + THREAD_SIZE;
+
+ /* This must match handle_sse expected parameter order */
+ ctx->e_state.a0 = evt;
+ ctx->e_state.a1 = (unsigned long)&ctx->i_state;
+ ctx->e_state.a2 = (unsigned long)handler;
+ ctx->e_state.a3 = (unsigned long)arg;
+}
diff --git a/arch/riscv/kernel/stacktrace.c b/arch/riscv/kernel/stacktrace.c
index 64a9c093aef9..a3d08792e3b5 100644
--- a/arch/riscv/kernel/stacktrace.c
+++ b/arch/riscv/kernel/stacktrace.c
@@ -17,6 +17,7 @@
#ifdef CONFIG_FRAME_POINTER
extern asmlinkage void ret_from_exception(void);
+extern asmlinkage void ret_from_sse(void);
void notrace walk_stackframe(struct task_struct *task, struct pt_regs *regs,
bool (*fn)(void *, unsigned long), void *arg)
@@ -68,6 +69,18 @@ void notrace walk_stackframe(struct task_struct *task, struct pt_regs *regs,
pc = ((struct pt_regs *)sp)->epc;
fp = ((struct pt_regs *)sp)->s0;
+ } else if (pc == (unsigned long)ret_from_sse) {
+ if (unlikely(!fn(arg, pc)))
+ break;
+ /* We don't have pt_regs when handling SSE
+ * events but we build a custom stackframe,
+ * moreover, the stack changes across boundaries
+ * so update it to avoid failing the checks above
+ */
+ frame = (struct stackframe *)fp - 1;
+ fp = frame->fp;
+ pc = frame->ra;
+ sp = fp - sizeof(struct stackframe);
}
}
diff --git a/arch/riscv/kernel/vmlinux.lds.S b/arch/riscv/kernel/vmlinux.lds.S
index 492dd4b8f3d6..f7e9cde2e832 100644
--- a/arch/riscv/kernel/vmlinux.lds.S
+++ b/arch/riscv/kernel/vmlinux.lds.S
@@ -120,6 +120,12 @@ SECTIONS
/* Start of data section */
_sdata = .;
RO_DATA(SECTION_ALIGN)
+
+#ifdef CONFIG_RISCV_SSE
+ __task_loc : {
+ BOUNDED_SECTION_BY(__task_loc, ___task_loc)
+ }
+#endif
.srodata : {
*(.srodata*)
}
diff --git a/drivers/firmware/Kconfig b/drivers/firmware/Kconfig
index b59e3041fd62..1749595b4150 100644
--- a/drivers/firmware/Kconfig
+++ b/drivers/firmware/Kconfig
@@ -226,6 +226,16 @@ config QCOM_SCM_DOWNLOAD_MODE_DEFAULT
Say Y here to enable "download mode" by default.
+config RISCV_SSE
+ bool
+ depends on RISCV_SBI
+ default y
+ help
+ The Supervisor Software Events support allow the SBI to deliver
+ NMI-like notifications to the supervisor mode software. When enable,
+ this option provides support to register callbacks on specific SSE
+ events.
+
config SYSFB
bool
select BOOT_VESA_SUPPORT
diff --git a/drivers/firmware/Makefile b/drivers/firmware/Makefile
index 28fcddcd688f..b0fcb0a74642 100644
--- a/drivers/firmware/Makefile
+++ b/drivers/firmware/Makefile
@@ -20,6 +20,7 @@ obj-$(CONFIG_RASPBERRYPI_FIRMWARE) += raspberrypi.o
obj-$(CONFIG_FW_CFG_SYSFS) += qemu_fw_cfg.o
obj-$(CONFIG_QCOM_SCM) += qcom-scm.o
qcom-scm-objs += qcom_scm.o qcom_scm-smc.o qcom_scm-legacy.o
+obj-$(CONFIG_RISCV_SSE) += riscv_sse.o
obj-$(CONFIG_SYSFB) += sysfb.o
obj-$(CONFIG_SYSFB_SIMPLEFB) += sysfb_simplefb.o
obj-$(CONFIG_TI_SCI_PROTOCOL) += ti_sci.o
diff --git a/drivers/firmware/riscv_sse.c b/drivers/firmware/riscv_sse.c
new file mode 100644
index 000000000000..5447ed0d3ae4
--- /dev/null
+++ b/drivers/firmware/riscv_sse.c
@@ -0,0 +1,496 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2023 Rivos Inc.
+ */
+
+#define pr_fmt(fmt) "sse: " fmt
+
+#include <linux/cpu.h>
+#include <linux/cpuhotplug.h>
+#include <linux/hardirq.h>
+#include <linux/list.h>
+#include <linux/percpu-defs.h>
+#include <linux/riscv_sse.h>
+#include <linux/slab.h>
+
+#include <asm/sbi.h>
+#include <asm/sse.h>
+
+struct sse_registered_event {
+ struct sse_handler_context ctx;
+ unsigned long *stack;
+};
+
+struct sse_event {
+ struct list_head list;
+ u32 evt;
+ u32 priority;
+ unsigned int cpu;
+
+ union {
+ struct sse_registered_event *global;
+ struct sse_registered_event __percpu *local;
+ };
+};
+
+static bool sse_available;
+static DEFINE_SPINLOCK(events_list_lock);
+static LIST_HEAD(events);
+static DEFINE_MUTEX(sse_mutex);
+
+static struct sbiret sbi_sse_ecall(int fid, unsigned long arg0,
+ unsigned long arg1, unsigned long arg2)
+{
+ return sbi_ecall(SBI_EXT_SSE, fid, arg0, arg1, arg2, 0, 0, 0);
+}
+
+static bool sse_event_is_global(u32 evt)
+{
+ return !!(evt & SBI_SSE_EVENT_GLOBAL);
+}
+
+static
+struct sse_event *sse_event_get(u32 evt)
+{
+ struct sse_event *sse_evt = NULL, *tmp;
+
+ spin_lock(&events_list_lock);
+ list_for_each_entry(tmp, &events, list) {
+ if (tmp->evt == evt) {
+ sse_evt = tmp;
+ break;
+ }
+ }
+ spin_unlock(&events_list_lock);
+
+ return sse_evt;
+}
+
+static int sse_event_set_target_cpu_nolock(struct sse_event *event, unsigned int cpu)
+{
+ unsigned int hart_id = cpuid_to_hartid_map(cpu);
+ u32 evt = event->evt;
+ struct sbiret sret;
+
+ if (!sse_event_is_global(evt))
+ return -EINVAL;
+
+ do {
+ sret = sbi_sse_ecall(SBI_SSE_EVENT_ATTR_SET, evt,
+ SBI_SSE_EVENT_ATTR_HART_ID, hart_id);
+ if (sret.error && sret.error != SBI_ERR_BUSY) {
+ pr_err("Failed to set event %x hart id, error %ld\n", evt,
+ sret.error);
+ return sbi_err_map_linux_errno(sret.error);
+ }
+ } while (sret.error);
+
+ event->cpu = cpu;
+
+ return 0;
+}
+
+
+int sse_event_set_target_cpu(struct sse_event *event, unsigned int cpu)
+{
+ int ret;
+
+ mutex_lock(&sse_mutex);
+ ret = sse_event_set_target_cpu_nolock(event, cpu);
+ mutex_unlock(&sse_mutex);
+
+ return ret;
+}
+
+static int sse_event_init_registered(unsigned int cpu, u32 evt,
+ struct sse_registered_event *reg_evt,
+ sse_event_handler *handler, void *arg)
+{
+ unsigned long *stack;
+
+ stack = sse_stack_alloc(cpu, THREAD_SIZE);
+ if (!stack)
+ return -ENOMEM;
+
+ reg_evt->stack = stack;
+
+ sse_handler_context_init(®_evt->ctx, stack, evt, handler, arg);
+
+ return 0;
+}
+
+static struct sse_event *sse_event_alloc(u32 evt,
+ u32 priority,
+ sse_event_handler *handler, void *arg)
+{
+ int err;
+ unsigned int cpu;
+ struct sse_event *event;
+ struct sse_registered_event __percpu *reg_evts;
+ struct sse_registered_event *reg_evt;
+
+ event = kzalloc(sizeof(*event), GFP_KERNEL);
+ if (!event)
+ return ERR_PTR(-ENOMEM);
+
+ event->evt = evt;
+ event->priority = priority;
+
+ if (sse_event_is_global(evt)) {
+ reg_evt = kzalloc(sizeof(*reg_evt), GFP_KERNEL);
+ if (!reg_evt) {
+ err = -ENOMEM;
+ goto err_alloc_reg_evt;
+ }
+
+ event->global = reg_evt;
+ err = sse_event_init_registered(smp_processor_id(), evt,
+ reg_evt, handler, arg);
+ if (err)
+ goto err_alloc_reg_evt;
+
+
+ } else {
+ reg_evts = alloc_percpu(struct sse_registered_event);
+ if (!reg_evts) {
+ err = -ENOMEM;
+ goto err_alloc_reg_evt;
+ }
+
+ event->local = reg_evts;
+
+ for_each_possible_cpu(cpu) {
+ reg_evt = per_cpu_ptr(reg_evts, cpu);
+
+ err = sse_event_init_registered(cpu, evt, reg_evt,
+ handler, arg);
+ if (err)
+ goto err_alloc_reg_evt;
+
+ }
+ }
+
+ return event;
+
+err_alloc_reg_evt:
+ kfree(event);
+
+ return ERR_PTR(err);
+}
+
+static int sse_sbi_register_event(struct sse_event *event,
+ struct sse_registered_event *reg_evt)
+{
+ struct sbiret ret;
+ phys_addr_t phys;
+ u32 evt = event->evt;
+
+ ret = sbi_sse_ecall(SBI_SSE_EVENT_ATTR_SET, evt,
+ SBI_SSE_EVENT_ATTR_PRIORITY, event->priority);
+ if (ret.error) {
+ pr_err("Failed to set event %x priority, error %ld\n", evt,
+ ret.error);
+ return sbi_err_map_linux_errno(ret.error);
+ }
+
+ if (sse_event_is_global(event->evt))
+ phys = virt_to_phys(®_evt->ctx);
+ else
+ phys = per_cpu_ptr_to_phys(®_evt->ctx);
+
+ ret = sbi_sse_ecall(SBI_SSE_EVENT_REGISTER, evt, phys, 0);
+ if (ret.error)
+ pr_err("Failed to register event %d, error %ld\n", evt,
+ ret.error);
+
+ return sbi_err_map_linux_errno(ret.error);
+}
+
+static int sse_sbi_event_func(struct sse_event *event, unsigned long func)
+{
+ struct sbiret ret;
+ u32 evt = event->evt;
+
+ ret = sbi_sse_ecall(func, evt, 0, 0);
+ if (ret.error)
+ pr_err("Failed to execute func %lx, event %d, error %ld\n", func,
+ evt, ret.error);
+
+ return sbi_err_map_linux_errno(ret.error);
+}
+
+static int sse_event_register_local(struct sse_event *event)
+{
+ int ret;
+ struct sse_registered_event *reg_evt = per_cpu_ptr(event->local,
+ smp_processor_id());
+
+ ret = sse_sbi_register_event(event, reg_evt);
+ if (ret)
+ pr_err("Failed to register event %x: err %d\n", event->evt,
+ ret);
+
+ return ret;
+}
+
+static int sse_sbi_disable_event(struct sse_event *event)
+{
+ return sse_sbi_event_func(event, SBI_SSE_EVENT_DISABLE);
+}
+
+static int sse_sbi_enable_event(struct sse_event *event)
+{
+ return sse_sbi_event_func(event, SBI_SSE_EVENT_ENABLE);
+}
+
+static int sse_sbi_unregister_event(struct sse_event *event)
+{
+ return sse_sbi_event_func(event, SBI_SSE_EVENT_UNREGISTER);
+}
+
+struct sse_per_cpu_evt {
+ struct sse_event *event;
+ unsigned long func;
+ int error;
+};
+
+static void sse_event_per_cpu_func(void *info)
+{
+ int ret;
+ struct sse_per_cpu_evt *cpu_evt = info;
+
+ if (cpu_evt->func == SBI_SSE_EVENT_REGISTER)
+ ret = sse_event_register_local(cpu_evt->event);
+ else
+ ret = sse_sbi_event_func(cpu_evt->event, cpu_evt->func);
+
+ if (ret)
+ WRITE_ONCE(cpu_evt->error, 1);
+}
+
+static void sse_event_free(struct sse_event *event)
+{
+ unsigned int cpu;
+ struct sse_registered_event *reg_evt;
+
+ if (sse_event_is_global(event->evt)) {
+ sse_stack_free(event->global->stack);
+ } else {
+ for_each_possible_cpu(cpu) {
+ reg_evt = per_cpu_ptr(event->local, cpu);
+ sse_stack_free(reg_evt->stack);
+ }
+ free_percpu(event->local);
+ }
+
+ kfree(event);
+}
+
+int sse_event_enable(struct sse_event *event)
+{
+ int ret = 0;
+ struct sse_per_cpu_evt cpu_evt;
+
+ mutex_lock(&sse_mutex);
+
+ cpus_read_lock();
+ if (sse_event_is_global(event->evt)) {
+ /* Global events can only be unregister from target hart */
+ ret = sse_event_set_target_cpu_nolock(event, smp_processor_id());
+ if (ret)
+ goto out;
+
+ ret = sse_sbi_enable_event(event);
+ if (ret)
+ goto out;
+
+ } else {
+ cpu_evt.event = event;
+ cpu_evt.error = 0;
+ cpu_evt.func = SBI_SSE_EVENT_ENABLE;
+ on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1);
+ if (READ_ONCE(cpu_evt.error)) {
+ cpu_evt.func = SBI_SSE_EVENT_DISABLE;
+ on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1);
+ goto out;
+ }
+ }
+ cpus_read_unlock();
+
+out:
+ mutex_unlock(&sse_mutex);
+
+ return ret;
+}
+
+void sse_event_disable(struct sse_event *event)
+{
+ struct sse_per_cpu_evt cpu_evt;
+
+ mutex_lock(&sse_mutex);
+
+ if (sse_event_is_global(event->evt)) {
+ sse_sbi_disable_event(event);
+ } else {
+ cpu_evt.event = event;
+ cpu_evt.func = SBI_SSE_EVENT_DISABLE;
+ on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1);
+ }
+
+ mutex_unlock(&sse_mutex);
+}
+
+struct sse_event *sse_event_register(u32 evt, u32 priority,
+ sse_event_handler *handler, void *arg)
+{
+ struct sse_per_cpu_evt cpu_evt;
+ struct sse_event *event;
+ int ret = 0;
+
+ mutex_lock(&sse_mutex);
+ if (sse_event_get(evt)) {
+ pr_err("Event %x already registered\n", evt);
+ ret = -EEXIST;
+ goto out_unlock;
+ }
+
+ event = sse_event_alloc(evt, priority, handler, arg);
+ if (IS_ERR(event)) {
+ ret = PTR_ERR(event);
+ goto out_unlock;
+ }
+
+ cpus_read_lock();
+ if (sse_event_is_global(evt)) {
+ /* SSE spec mandates that the CPU registering the global event be the
+ * one set as the target hart, plus we don't know initial value
+ */
+ ret = sse_event_set_target_cpu_nolock(event, smp_processor_id());
+ if (ret)
+ goto err_event_free;
+
+ ret = sse_sbi_register_event(event, event->global);
+ if (ret)
+ goto err_event_free;
+
+ } else {
+ cpu_evt.event = event;
+ cpu_evt.error = 0;
+ cpu_evt.func = SBI_SSE_EVENT_REGISTER;
+ on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1);
+ if (READ_ONCE(cpu_evt.error)) {
+ cpu_evt.func = SBI_SSE_EVENT_UNREGISTER;
+ on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1);
+ goto err_event_free;
+ }
+ }
+ cpus_read_unlock();
+
+ spin_lock(&events_list_lock);
+ list_add(&event->list, &events);
+ spin_unlock(&events_list_lock);
+
+ mutex_unlock(&sse_mutex);
+
+ return event;
+
+err_event_free:
+ cpus_read_unlock();
+ sse_event_free(event);
+out_unlock:
+ mutex_unlock(&sse_mutex);
+
+ return ERR_PTR(ret);
+}
+
+void sse_event_unregister(struct sse_event *event)
+{
+ int ret;
+ struct sse_per_cpu_evt cpu_evt;
+
+ mutex_lock(&sse_mutex);
+
+ if (sse_event_is_global(event->evt)) {
+ /* Global events can only be unregister from target hart */
+ ret = sse_event_set_target_cpu_nolock(event, smp_processor_id());
+ WARN_ON(ret);
+ sse_sbi_unregister_event(event);
+ } else {
+ cpu_evt.event = event;
+ cpu_evt.func = SBI_SSE_EVENT_UNREGISTER;
+ on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1);
+ }
+
+ spin_lock(&events_list_lock);
+ list_del(&event->list);
+ spin_unlock(&events_list_lock);
+
+ sse_event_free(event);
+
+ mutex_unlock(&sse_mutex);
+}
+
+static int sse_cpu_online(unsigned int cpu)
+{
+ struct sse_event *sse_evt;
+
+ spin_lock(&events_list_lock);
+ list_for_each_entry(sse_evt, &events, list) {
+ if (sse_event_is_global(sse_evt->evt))
+ continue;
+
+ sse_event_register_local(sse_evt);
+ }
+
+ spin_unlock(&events_list_lock);
+
+ return 0;
+}
+
+static int sse_cpu_teardown(unsigned int cpu)
+{
+ unsigned int next_cpu;
+ struct sse_event *sse_evt;
+
+ spin_lock(&events_list_lock);
+ list_for_each_entry(sse_evt, &events, list) {
+ if (!sse_event_is_global(sse_evt->evt)) {
+ sse_sbi_unregister_event(sse_evt);
+ continue;
+ }
+
+ if (sse_evt->cpu != smp_processor_id())
+ continue;
+
+ /* Update destination hart */
+ next_cpu = cpumask_any_but(cpu_online_mask, cpu);
+ sse_event_set_target_cpu(sse_evt, next_cpu);
+ }
+ spin_unlock(&events_list_lock);
+
+ return 0;
+}
+
+static int __init sse_init(void)
+{
+ int cpu, ret;
+
+ if (sbi_probe_extension(SBI_EXT_SSE) <= 0) {
+ pr_err("Missing SBI SSE extension\n");
+ return -EOPNOTSUPP;
+ }
+ pr_info("SBI SSE extension detected\n");
+
+ for_each_possible_cpu(cpu)
+ INIT_LIST_HEAD(&events);
+
+ ret = cpuhp_setup_state_nocalls(CPUHP_AP_ONLINE_DYN, "riscv/sse:online",
+ sse_cpu_online, sse_cpu_teardown);
+ if (ret < 0)
+ return ret;
+
+ sse_available = true;
+
+ return 0;
+}
+device_initcall(sse_init);
diff --git a/include/linux/riscv_sse.h b/include/linux/riscv_sse.h
new file mode 100644
index 000000000000..3ca145368c50
--- /dev/null
+++ b/include/linux/riscv_sse.h
@@ -0,0 +1,56 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2023 Rivos Inc.
+ */
+
+#ifndef __LINUX_RISCV_SSE_H
+#define __LINUX_RISCV_SSE_H
+
+#include <linux/types.h>
+#include <linux/linkage.h>
+
+struct sse_interrupted_state;
+struct sse_event;
+struct pt_regs;
+
+typedef int (sse_event_handler)(u32 event_num, void *arg, struct pt_regs *regs);
+
+#ifdef CONFIG_RISCV_SSE
+
+struct sse_event *sse_event_register(u32 event_num, u32 priority,
+ sse_event_handler *handler, void *arg);
+
+void sse_event_unregister(struct sse_event *evt);
+
+int sse_event_set_target_cpu(struct sse_event *sse_evt, unsigned int cpu);
+
+int sse_event_enable(struct sse_event *sse_evt);
+
+void sse_event_disable(struct sse_event *sse_evt);
+
+#else
+static inline int sse_event_register(struct sse_event *evt, u32 priority,
+ sse_event_handler *handler, void *arg)
+{
+ return -EOPNOTSUPP;
+}
+
+static inline void sse_event_unregister(struct sse_event *evt) {}
+
+static inline int sse_event_set_target_cpu(struct sse_event *sse_evt,
+ unsigned int cpu)
+{
+ return -EOPNOTSUPP;
+}
+
+static inline int sse_event_enable(struct sse_event *sse_evt)
+{
+ return -EOPNOTSUPP;
+}
+
+static inline void sse_event_disable(struct sse_event *sse_evt) {}
+
+
+#endif
+
+#endif /* __LINUX_RISCV_SSE_H */
--
2.42.0
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [RFC PATCH 2/3] riscv: add support for SBI Supervisor Software Events extension
@ 2023-10-26 14:31 ` Clément Léger
0 siblings, 0 replies; 25+ messages in thread
From: Clément Léger @ 2023-10-26 14:31 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Albert Ou, Atish Patra, Anup Patel,
linux-riscv, linux-kernel, linux-arm-kernel
Cc: Clément Léger, Himanshu Chauhan, Xu Lu
The SBI SSE extension allows the supervisor software to be notified by
the SBI of specific events that are not maskable. The context switch is
handled entirely by the firmware which will save all registers into a
supervisor provided context and restore a set of registers from this
same context. This context contains all the RV GP registers and thus
will also switch the stack.
Since each events (and each CPU for local events) have their own
context (and thus their own set of register), allocate a stack for each
event (and for each cpu for local events). This can be modified if
needed to use a single stack per cpu for SSE event but will require a
bit more additional plumbing.
When completing the event, if we were coming from kernel with interrupts
disabled, simply return there. If coming from userspace or kernel with
interrupts enabled, simulate an interrupt exception to allow delivery of
signal to user task. For instance this can happen, when a RAS event has
been generated and a SIGBUS has been sent to a task.
Signed-off-by: Clément Léger <cleger@rivosinc.com>
---
arch/riscv/include/asm/asm-prototypes.h | 5 +
arch/riscv/include/asm/sse.h | 94 +++++
arch/riscv/kernel/Makefile | 1 +
arch/riscv/kernel/asm-offsets.c | 17 +
arch/riscv/kernel/entry.S | 156 ++++++++
arch/riscv/kernel/sse.c | 97 +++++
arch/riscv/kernel/stacktrace.c | 13 +
arch/riscv/kernel/vmlinux.lds.S | 6 +
drivers/firmware/Kconfig | 10 +
drivers/firmware/Makefile | 1 +
drivers/firmware/riscv_sse.c | 496 ++++++++++++++++++++++++
include/linux/riscv_sse.h | 56 +++
12 files changed, 952 insertions(+)
create mode 100644 arch/riscv/include/asm/sse.h
create mode 100644 arch/riscv/kernel/sse.c
create mode 100644 drivers/firmware/riscv_sse.c
create mode 100644 include/linux/riscv_sse.h
diff --git a/arch/riscv/include/asm/asm-prototypes.h b/arch/riscv/include/asm/asm-prototypes.h
index 61ba8ed43d8f..c622ed1db33c 100644
--- a/arch/riscv/include/asm/asm-prototypes.h
+++ b/arch/riscv/include/asm/asm-prototypes.h
@@ -3,6 +3,8 @@
#define _ASM_RISCV_PROTOTYPES_H
#include <linux/ftrace.h>
+#include <linux/riscv_sse.h>
+#include <asm/sse.h>
#include <asm-generic/asm-prototypes.h>
long long __lshrti3(long long a, int b);
@@ -29,5 +31,8 @@ asmlinkage unsigned long get_overflow_stack(void);
asmlinkage void handle_bad_stack(struct pt_regs *regs);
asmlinkage void do_page_fault(struct pt_regs *regs);
asmlinkage void do_irq(struct pt_regs *regs);
+asmlinkage unsigned long do_sse(unsigned long evt,
+ struct sse_interrupted_state *i_state,
+ sse_event_handler *handler, void *arg);
#endif /* _ASM_RISCV_PROTOTYPES_H */
diff --git a/arch/riscv/include/asm/sse.h b/arch/riscv/include/asm/sse.h
new file mode 100644
index 000000000000..e663224296df
--- /dev/null
+++ b/arch/riscv/include/asm/sse.h
@@ -0,0 +1,94 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2023 Rivos Inc.
+ */
+#ifndef __ASM_SSE_H
+#define __ASM_SSE_H
+
+#include <linux/riscv_sse.h>
+
+
+struct sse_interrupted_state {
+ unsigned long pc;
+ unsigned long ra;
+ unsigned long sp;
+ unsigned long gp;
+ unsigned long tp;
+ unsigned long t0;
+ unsigned long t1;
+ unsigned long t2;
+ unsigned long s0;
+ unsigned long s1;
+ unsigned long a0;
+ unsigned long a1;
+ unsigned long a2;
+ unsigned long a3;
+ unsigned long a4;
+ unsigned long a5;
+ unsigned long a6;
+ unsigned long a7;
+ unsigned long s2;
+ unsigned long s3;
+ unsigned long s4;
+ unsigned long s5;
+ unsigned long s6;
+ unsigned long s7;
+ unsigned long s8;
+ unsigned long s9;
+ unsigned long s10;
+ unsigned long s11;
+ unsigned long t3;
+ unsigned long t4;
+ unsigned long t5;
+ unsigned long t6;
+ unsigned long exec_mode;
+};
+
+struct sse_entry_state {
+ unsigned long pc;
+ unsigned long ra;
+ unsigned long sp;
+ unsigned long gp;
+ unsigned long tp;
+ unsigned long t0;
+ unsigned long t1;
+ unsigned long t2;
+ unsigned long s0;
+ unsigned long s1;
+ unsigned long a0;
+ unsigned long a1;
+ unsigned long a2;
+ unsigned long a3;
+ unsigned long a4;
+ unsigned long a5;
+ unsigned long a6;
+ unsigned long a7;
+ unsigned long s2;
+ unsigned long s3;
+ unsigned long s4;
+ unsigned long s5;
+ unsigned long s6;
+ unsigned long s7;
+ unsigned long s8;
+ unsigned long s9;
+ unsigned long s10;
+ unsigned long s11;
+ unsigned long t3;
+ unsigned long t4;
+ unsigned long t5;
+ unsigned long t6;
+};
+
+struct sse_handler_context {
+ struct sse_entry_state e_state;
+ struct sse_interrupted_state i_state;
+};
+
+unsigned long *sse_stack_alloc(unsigned int cpu, unsigned int size);
+void sse_stack_free(unsigned long *stack);
+
+struct sse_handler_context;
+void sse_handler_context_init(struct sse_handler_context *ctx, void *stack,
+ u32 evt, sse_event_handler *handler, void *arg);
+
+#endif
diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile
index 95cf25d48405..571b2aaf669b 100644
--- a/arch/riscv/kernel/Makefile
+++ b/arch/riscv/kernel/Makefile
@@ -79,6 +79,7 @@ obj-$(CONFIG_DYNAMIC_FTRACE) += mcount-dyn.o
obj-$(CONFIG_PERF_EVENTS) += perf_callchain.o
obj-$(CONFIG_HAVE_PERF_REGS) += perf_regs.o
obj-$(CONFIG_RISCV_SBI) += sbi.o
+obj-$(CONFIG_RISCV_SSE) += sse.o
ifeq ($(CONFIG_RISCV_SBI), y)
obj-$(CONFIG_SMP) += sbi-ipi.o
obj-$(CONFIG_SMP) += cpu_ops_sbi.o
diff --git a/arch/riscv/kernel/asm-offsets.c b/arch/riscv/kernel/asm-offsets.c
index d6a75aac1d27..1776448e67d2 100644
--- a/arch/riscv/kernel/asm-offsets.c
+++ b/arch/riscv/kernel/asm-offsets.c
@@ -14,7 +14,10 @@
#include <asm/thread_info.h>
#include <asm/ptrace.h>
#include <asm/cpu_ops_sbi.h>
+#include <asm/sbi.h>
+#include <asm/sse.h>
#include <asm/suspend.h>
+#include <asm/stacktrace.h>
void asm_offsets(void);
@@ -479,4 +482,18 @@ void asm_offsets(void)
OFFSET(KERNEL_MAP_VIRT_ADDR, kernel_mapping, virt_addr);
OFFSET(SBI_HART_BOOT_TASK_PTR_OFFSET, sbi_hart_boot_data, task_ptr);
OFFSET(SBI_HART_BOOT_STACK_PTR_OFFSET, sbi_hart_boot_data, stack_ptr);
+
+ OFFSET(SSE_INTERRUPTED_EXEC_MODE, sse_interrupted_state, exec_mode);
+ OFFSET(SSE_INTERRUPTED_S0, sse_interrupted_state, s0);
+ OFFSET(SSE_INTERRUPTED_TP, sse_interrupted_state, tp);
+ OFFSET(SSE_INTERRUPTED_PC, sse_interrupted_state, pc);
+
+ DEFINE(STACKFRAME_SIZE_ON_STACK, ALIGN(sizeof(struct stackframe),
+ STACK_ALIGN));
+ OFFSET(STACKFRAME_FP, stackframe, fp);
+ OFFSET(STACKFRAME_RA, stackframe, ra);
+ DEFINE(SBI_EXT_SSE, SBI_EXT_SSE);
+ DEFINE(SBI_SSE_EVENT_COMPLETE, SBI_SSE_EVENT_COMPLETE);
+
+ DEFINE(ASM_PAGE_OFFSET, CONFIG_PAGE_OFFSET);
}
diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
index 143a2bb3e697..4adf7b21a568 100644
--- a/arch/riscv/kernel/entry.S
+++ b/arch/riscv/kernel/entry.S
@@ -14,17 +14,49 @@
#include <asm/asm-offsets.h>
#include <asm/errata_list.h>
+#ifdef CONFIG_RISCV_SSE
+#define IN_TP 0
+#define IN_SSCRATCH 1
+#define PRV_U_TASK_LOC(loc) (loc << 0)
+#define PRV_S_TASK_LOC(loc) (loc << 1)
+#define TASK_LOC(s_loc, u_loc) (PRV_U_TASK_LOC(u_loc) | PRV_S_TASK_LOC(s_loc))
+#define TASK_LOC_ENTRY_SIZE (RISCV_SZPTR + 1)
+
+/* __SSE_TASK_LOC - Annotate e'xception code with the location of current task
+ * struct when coming from S/U mode
+ *
+ * When entering handle_exception(), the current task struct is located either
+ * in tp or sscratch depending on interrupted mode. Since SSE handlers can be
+ * triggered at any time during the execution of the kernel, we need to be able
+ * to retrieve the current task struct even if in handle_exception. This macro
+ * create an entry into the __task_loc section that holds the location of
+ * the current task struct for the subsequent assembly located after that up to
+ * _ret_from_exception_end or the next __TASK_LOC.
+ */
+#define __SSE_TASK_LOC(s_loc, u_loc) \
+ .pushsection __task_loc,"a"; \
+ RISCV_PTR 99f; \
+ .byte TASK_LOC(s_loc, u_loc); \
+ .popsection; \
+ 99:
+#else
+#define __SSE_TASK_LOC(s_loc, u_loc)
+#endif
+
SYM_CODE_START(handle_exception)
/*
* If coming from userspace, preserve the user thread pointer and load
* the kernel thread pointer. If we came from the kernel, the scratch
* register will contain 0, and we should continue on the current TP.
*/
+__SSE_TASK_LOC(IN_TP, IN_SSCRATCH)
csrrw tp, CSR_SCRATCH, tp
+__SSE_TASK_LOC(IN_SSCRATCH, IN_TP)
bnez tp, _save_context
_restore_kernel_tpsp:
csrr tp, CSR_SCRATCH
+__SSE_TASK_LOC(IN_TP, IN_TP)
REG_S sp, TASK_TI_KERNEL_SP(tp)
#ifdef CONFIG_VMAP_STACK
@@ -151,6 +183,7 @@ SYM_CODE_START_NOALIGN(ret_from_exception)
REG_SC x0, a2, PT_EPC(sp)
csrw CSR_STATUS, a0
+__SSE_TASK_LOC(IN_TP, IN_SSCRATCH)
csrw CSR_EPC, a2
REG_L x1, PT_RA(sp)
@@ -166,6 +199,7 @@ SYM_CODE_START_NOALIGN(ret_from_exception)
#else
sret
#endif
+_ret_from_exception_end:
SYM_CODE_END(ret_from_exception)
#ifdef CONFIG_VMAP_STACK
@@ -268,6 +302,125 @@ SYM_CODE_START(ret_from_fork)
tail syscall_exit_to_user_mode
SYM_CODE_END(ret_from_fork)
+#ifdef CONFIG_RISCV_SSE
+SYM_CODE_START(handle_sse)
+ move s3, a0
+ /*
+ * Depending on where the sse event interrupted the kernel execution,
+ * sscratch content might be 0 or not even if kernel was interrupted.
+ * This might happen if the sse event was triggered while in
+ * handle_exception() right after entry. In that case, sscratch might
+ * contain 0 if coming from kernel. In order to handle that easily,
+ * simply save sscratch content and restore it when returning from sse
+ * event.
+ */
+ csrr s4, CSR_SSCRATCH
+ /*
+ * Save CAUSE and TVAL in case of nested exceptions, EPC/STATUS are
+ * already saved/restored by handle_exception() for nested exceptions.
+ */
+ csrr s5, CSR_SCAUSE
+ csrr s6, CSR_STVAL
+ REG_L t0, SSE_INTERRUPTED_EXEC_MODE(a1)
+ andi t1, t0, 0x1
+
+ /* Userspace was interrupted, simply restore TP from scratch */
+ bnez t1, _sse_from_kernel
+ move tp, s4
+ j _call_do_sse
+
+_sse_from_kernel:
+ REG_L t0, SSE_INTERRUPTED_PC(a1)
+#ifdef CONFIG_FRAME_POINTER
+ /*
+ * Else, kernel was interrupted and we will create a correct stack frame
+ * from interrupted context.
+ */
+ addi sp, sp, -(STACKFRAME_SIZE_ON_STACK)
+ REG_L t2, SSE_INTERRUPTED_S0(a1)
+ REG_S t2, STACKFRAME_FP(sp)
+ REG_S t0, STACKFRAME_RA(sp)
+ addi s0, sp, STACKFRAME_SIZE_ON_STACK
+#endif
+ REG_L tp, SSE_INTERRUPTED_TP(a1)
+
+ /*
+ * If interrupting the kernel during exception handling
+ * (see handle_exception), then, we might have tp either in SSCRATCH or
+ * in tp, this part is non regular and requires some more work to
+ * determine were is located the current task.
+ */
+ la t1, handle_exception
+ la t2, _ret_from_exception_end
+ bltu t0, t1, _call_do_sse
+ bltu t0, t2, _sse_exception_slow_path
+
+_call_do_sse:
+
+ /* TODO: remove this sanity check once sure everything works ! */
+ li t1, ASM_PAGE_OFFSET
+ bgeu tp, t1, _tp_ok
+ la tp, init_task
+ la a0, sse_panic_string
+ la t0, panic
+ jalr t0
+_tp_ok:
+
+ csrw CSR_SCRATCH, x0
+ la ra, ret_from_sse
+ tail do_sse
+
+.global ret_from_sse
+ret_from_sse:
+ /* Restore saved CSRs */
+ csrw CSR_SSCRATCH, s4
+ csrw CSR_SCAUSE, s5
+ csrw CSR_STVAL, s6
+ li a2, 0
+ move a1, a0
+ move a0, s3
+ li a7, SBI_EXT_SSE
+ li a6, SBI_SSE_EVENT_COMPLETE
+ ecall
+
+/*
+ * t0 contains interrupted pc
+ * t1 contains handle_exception address
+ * When called, t0 must be in [handle_exception, _ret_from_exception_end[
+ */
+_sse_exception_slow_path:
+ la t3, __stop___task_loc
+ add t3, t3, -TASK_LOC_ENTRY_SIZE
+ la t4, __start___task_loc
+ /*
+ * Reverse iterate the task location section to find where is located
+ * the task struct
+ */
+1:
+ REG_L t2, 0(t3)
+ bgeu t0, t2, 2f
+ add t3, t3, -TASK_LOC_ENTRY_SIZE
+ bgeu t3, t4, 1b
+
+2:
+ lbu t2, RISCV_SZPTR(t3)
+
+ /* Get the value of SR_SPP */
+ csrr t1, CSR_SSTATUS
+ andi t1, t1, SR_SPP
+ snez t1, t1
+
+ srl t2, t2, t1
+ andi t2, t2, 1
+ beqz t2, _call_do_sse
+
+_restore_tp_from_sscratch:
+ csrr tp, CSR_SCRATCH
+ j _call_do_sse
+
+SYM_CODE_END(handle_sse)
+#endif
+
/*
* Integer register context switch
* The callee-saved registers must be saved and restored.
@@ -351,3 +504,6 @@ SYM_CODE_START(__user_rt_sigreturn)
ecall
SYM_CODE_END(__user_rt_sigreturn)
#endif
+
+sse_panic_string:
+ .ascii "SSE TP is invalid, last state %px"
diff --git a/arch/riscv/kernel/sse.c b/arch/riscv/kernel/sse.c
new file mode 100644
index 000000000000..69c2607d9dbf
--- /dev/null
+++ b/arch/riscv/kernel/sse.c
@@ -0,0 +1,97 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2023 Rivos Inc.
+ */
+#include <linux/nmi.h>
+#include <linux/bitfield.h>
+#include <linux/riscv_sse.h>
+
+#include <asm/irq_stack.h>
+#include <asm/sbi.h>
+#include <asm/sse.h>
+
+#define SSE_PRIVILEGE_MODE_BIT BIT(0)
+#define SSE_SPIE_BIT BIT(2)
+
+#define sse_privilege_mode(exec_mode) FIELD_GET(SSE_PRIVILEGE_MODE_BIT, exec_mode)
+#define sse_spie(exec_mode) FIELD_GET(SSE_SPIE_BIT, exec_mode)
+
+register unsigned long gp_in_global __asm__("gp");
+
+extern asmlinkage void handle_exception(void);
+extern asmlinkage void handle_sse(unsigned long evt,
+ struct sse_interrupted_state *i_state,
+ sse_event_handler *handler, void *arg);
+
+static void sse_get_pt_regs(struct sse_interrupted_state *i_state,
+ struct pt_regs *regs)
+{
+ unsigned long sstatus = csr_read(CSR_SSTATUS);
+
+ memcpy(regs, i_state, offsetof(struct sse_interrupted_state, exec_mode));
+ regs->status = sstatus & ~SR_SPP;
+ regs->status |= FIELD_PREP(SR_SPP, sse_privilege_mode(i_state->exec_mode));
+}
+
+unsigned long do_sse(unsigned long evt, struct sse_interrupted_state *i_state,
+ sse_event_handler *handler, void *arg)
+{
+ int ret;
+ struct pt_regs regs;
+
+ nmi_enter();
+
+ sse_get_pt_regs(i_state, ®s);
+ ret = handler(evt, arg, ®s);
+ if (ret)
+ pr_warn("event %lx handler failed with error %d\n", evt, ret);
+
+ /* The SSE delivery path does not uses the "standard" exception path and
+ * thus does not process any pending signal/softirqs. Some drivers might
+ * enqueue pending work that needs to be handled as soon as possible.
+ * For that purpose, set the software interrupt pending bit
+ */
+ csr_set(CSR_IP, IE_SIE);
+
+ nmi_exit();
+
+ return ret ? SBI_SSE_HANDLER_FAILED : SBI_SSE_HANDLER_SUCCESS;
+}
+
+#ifdef CONFIG_VMAP_STACK
+unsigned long *sse_stack_alloc(unsigned int cpu, unsigned int size)
+{
+ return arch_alloc_vmap_stack(size, cpu_to_node(cpu));
+}
+
+void sse_stack_free(unsigned long *stack)
+{
+ vfree(stack);
+}
+#else /* CONFIG_VMAP_STACK */
+
+unsigned long *sse_stack_alloc(unsigned int cpu, unsigned int size)
+{
+ return kmalloc(size, GFP_KERNEL);
+}
+
+void sse_stack_free(unsigned long *stack)
+{
+ kfree(stack);
+}
+
+#endif /* CONFIG_VMAP_STACK */
+
+void sse_handler_context_init(struct sse_handler_context *ctx, void *stack,
+ u32 evt, sse_event_handler *handler, void *arg)
+{
+ ctx->e_state.pc = (unsigned long)handle_sse;
+ ctx->e_state.gp = gp_in_global;
+ ctx->e_state.sp = (unsigned long)stack + THREAD_SIZE;
+
+ /* This must match handle_sse expected parameter order */
+ ctx->e_state.a0 = evt;
+ ctx->e_state.a1 = (unsigned long)&ctx->i_state;
+ ctx->e_state.a2 = (unsigned long)handler;
+ ctx->e_state.a3 = (unsigned long)arg;
+}
diff --git a/arch/riscv/kernel/stacktrace.c b/arch/riscv/kernel/stacktrace.c
index 64a9c093aef9..a3d08792e3b5 100644
--- a/arch/riscv/kernel/stacktrace.c
+++ b/arch/riscv/kernel/stacktrace.c
@@ -17,6 +17,7 @@
#ifdef CONFIG_FRAME_POINTER
extern asmlinkage void ret_from_exception(void);
+extern asmlinkage void ret_from_sse(void);
void notrace walk_stackframe(struct task_struct *task, struct pt_regs *regs,
bool (*fn)(void *, unsigned long), void *arg)
@@ -68,6 +69,18 @@ void notrace walk_stackframe(struct task_struct *task, struct pt_regs *regs,
pc = ((struct pt_regs *)sp)->epc;
fp = ((struct pt_regs *)sp)->s0;
+ } else if (pc == (unsigned long)ret_from_sse) {
+ if (unlikely(!fn(arg, pc)))
+ break;
+ /* We don't have pt_regs when handling SSE
+ * events but we build a custom stackframe,
+ * moreover, the stack changes across boundaries
+ * so update it to avoid failing the checks above
+ */
+ frame = (struct stackframe *)fp - 1;
+ fp = frame->fp;
+ pc = frame->ra;
+ sp = fp - sizeof(struct stackframe);
}
}
diff --git a/arch/riscv/kernel/vmlinux.lds.S b/arch/riscv/kernel/vmlinux.lds.S
index 492dd4b8f3d6..f7e9cde2e832 100644
--- a/arch/riscv/kernel/vmlinux.lds.S
+++ b/arch/riscv/kernel/vmlinux.lds.S
@@ -120,6 +120,12 @@ SECTIONS
/* Start of data section */
_sdata = .;
RO_DATA(SECTION_ALIGN)
+
+#ifdef CONFIG_RISCV_SSE
+ __task_loc : {
+ BOUNDED_SECTION_BY(__task_loc, ___task_loc)
+ }
+#endif
.srodata : {
*(.srodata*)
}
diff --git a/drivers/firmware/Kconfig b/drivers/firmware/Kconfig
index b59e3041fd62..1749595b4150 100644
--- a/drivers/firmware/Kconfig
+++ b/drivers/firmware/Kconfig
@@ -226,6 +226,16 @@ config QCOM_SCM_DOWNLOAD_MODE_DEFAULT
Say Y here to enable "download mode" by default.
+config RISCV_SSE
+ bool
+ depends on RISCV_SBI
+ default y
+ help
+ The Supervisor Software Events support allow the SBI to deliver
+ NMI-like notifications to the supervisor mode software. When enable,
+ this option provides support to register callbacks on specific SSE
+ events.
+
config SYSFB
bool
select BOOT_VESA_SUPPORT
diff --git a/drivers/firmware/Makefile b/drivers/firmware/Makefile
index 28fcddcd688f..b0fcb0a74642 100644
--- a/drivers/firmware/Makefile
+++ b/drivers/firmware/Makefile
@@ -20,6 +20,7 @@ obj-$(CONFIG_RASPBERRYPI_FIRMWARE) += raspberrypi.o
obj-$(CONFIG_FW_CFG_SYSFS) += qemu_fw_cfg.o
obj-$(CONFIG_QCOM_SCM) += qcom-scm.o
qcom-scm-objs += qcom_scm.o qcom_scm-smc.o qcom_scm-legacy.o
+obj-$(CONFIG_RISCV_SSE) += riscv_sse.o
obj-$(CONFIG_SYSFB) += sysfb.o
obj-$(CONFIG_SYSFB_SIMPLEFB) += sysfb_simplefb.o
obj-$(CONFIG_TI_SCI_PROTOCOL) += ti_sci.o
diff --git a/drivers/firmware/riscv_sse.c b/drivers/firmware/riscv_sse.c
new file mode 100644
index 000000000000..5447ed0d3ae4
--- /dev/null
+++ b/drivers/firmware/riscv_sse.c
@@ -0,0 +1,496 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2023 Rivos Inc.
+ */
+
+#define pr_fmt(fmt) "sse: " fmt
+
+#include <linux/cpu.h>
+#include <linux/cpuhotplug.h>
+#include <linux/hardirq.h>
+#include <linux/list.h>
+#include <linux/percpu-defs.h>
+#include <linux/riscv_sse.h>
+#include <linux/slab.h>
+
+#include <asm/sbi.h>
+#include <asm/sse.h>
+
+struct sse_registered_event {
+ struct sse_handler_context ctx;
+ unsigned long *stack;
+};
+
+struct sse_event {
+ struct list_head list;
+ u32 evt;
+ u32 priority;
+ unsigned int cpu;
+
+ union {
+ struct sse_registered_event *global;
+ struct sse_registered_event __percpu *local;
+ };
+};
+
+static bool sse_available;
+static DEFINE_SPINLOCK(events_list_lock);
+static LIST_HEAD(events);
+static DEFINE_MUTEX(sse_mutex);
+
+static struct sbiret sbi_sse_ecall(int fid, unsigned long arg0,
+ unsigned long arg1, unsigned long arg2)
+{
+ return sbi_ecall(SBI_EXT_SSE, fid, arg0, arg1, arg2, 0, 0, 0);
+}
+
+static bool sse_event_is_global(u32 evt)
+{
+ return !!(evt & SBI_SSE_EVENT_GLOBAL);
+}
+
+static
+struct sse_event *sse_event_get(u32 evt)
+{
+ struct sse_event *sse_evt = NULL, *tmp;
+
+ spin_lock(&events_list_lock);
+ list_for_each_entry(tmp, &events, list) {
+ if (tmp->evt == evt) {
+ sse_evt = tmp;
+ break;
+ }
+ }
+ spin_unlock(&events_list_lock);
+
+ return sse_evt;
+}
+
+static int sse_event_set_target_cpu_nolock(struct sse_event *event, unsigned int cpu)
+{
+ unsigned int hart_id = cpuid_to_hartid_map(cpu);
+ u32 evt = event->evt;
+ struct sbiret sret;
+
+ if (!sse_event_is_global(evt))
+ return -EINVAL;
+
+ do {
+ sret = sbi_sse_ecall(SBI_SSE_EVENT_ATTR_SET, evt,
+ SBI_SSE_EVENT_ATTR_HART_ID, hart_id);
+ if (sret.error && sret.error != SBI_ERR_BUSY) {
+ pr_err("Failed to set event %x hart id, error %ld\n", evt,
+ sret.error);
+ return sbi_err_map_linux_errno(sret.error);
+ }
+ } while (sret.error);
+
+ event->cpu = cpu;
+
+ return 0;
+}
+
+
+int sse_event_set_target_cpu(struct sse_event *event, unsigned int cpu)
+{
+ int ret;
+
+ mutex_lock(&sse_mutex);
+ ret = sse_event_set_target_cpu_nolock(event, cpu);
+ mutex_unlock(&sse_mutex);
+
+ return ret;
+}
+
+static int sse_event_init_registered(unsigned int cpu, u32 evt,
+ struct sse_registered_event *reg_evt,
+ sse_event_handler *handler, void *arg)
+{
+ unsigned long *stack;
+
+ stack = sse_stack_alloc(cpu, THREAD_SIZE);
+ if (!stack)
+ return -ENOMEM;
+
+ reg_evt->stack = stack;
+
+ sse_handler_context_init(®_evt->ctx, stack, evt, handler, arg);
+
+ return 0;
+}
+
+static struct sse_event *sse_event_alloc(u32 evt,
+ u32 priority,
+ sse_event_handler *handler, void *arg)
+{
+ int err;
+ unsigned int cpu;
+ struct sse_event *event;
+ struct sse_registered_event __percpu *reg_evts;
+ struct sse_registered_event *reg_evt;
+
+ event = kzalloc(sizeof(*event), GFP_KERNEL);
+ if (!event)
+ return ERR_PTR(-ENOMEM);
+
+ event->evt = evt;
+ event->priority = priority;
+
+ if (sse_event_is_global(evt)) {
+ reg_evt = kzalloc(sizeof(*reg_evt), GFP_KERNEL);
+ if (!reg_evt) {
+ err = -ENOMEM;
+ goto err_alloc_reg_evt;
+ }
+
+ event->global = reg_evt;
+ err = sse_event_init_registered(smp_processor_id(), evt,
+ reg_evt, handler, arg);
+ if (err)
+ goto err_alloc_reg_evt;
+
+
+ } else {
+ reg_evts = alloc_percpu(struct sse_registered_event);
+ if (!reg_evts) {
+ err = -ENOMEM;
+ goto err_alloc_reg_evt;
+ }
+
+ event->local = reg_evts;
+
+ for_each_possible_cpu(cpu) {
+ reg_evt = per_cpu_ptr(reg_evts, cpu);
+
+ err = sse_event_init_registered(cpu, evt, reg_evt,
+ handler, arg);
+ if (err)
+ goto err_alloc_reg_evt;
+
+ }
+ }
+
+ return event;
+
+err_alloc_reg_evt:
+ kfree(event);
+
+ return ERR_PTR(err);
+}
+
+static int sse_sbi_register_event(struct sse_event *event,
+ struct sse_registered_event *reg_evt)
+{
+ struct sbiret ret;
+ phys_addr_t phys;
+ u32 evt = event->evt;
+
+ ret = sbi_sse_ecall(SBI_SSE_EVENT_ATTR_SET, evt,
+ SBI_SSE_EVENT_ATTR_PRIORITY, event->priority);
+ if (ret.error) {
+ pr_err("Failed to set event %x priority, error %ld\n", evt,
+ ret.error);
+ return sbi_err_map_linux_errno(ret.error);
+ }
+
+ if (sse_event_is_global(event->evt))
+ phys = virt_to_phys(®_evt->ctx);
+ else
+ phys = per_cpu_ptr_to_phys(®_evt->ctx);
+
+ ret = sbi_sse_ecall(SBI_SSE_EVENT_REGISTER, evt, phys, 0);
+ if (ret.error)
+ pr_err("Failed to register event %d, error %ld\n", evt,
+ ret.error);
+
+ return sbi_err_map_linux_errno(ret.error);
+}
+
+static int sse_sbi_event_func(struct sse_event *event, unsigned long func)
+{
+ struct sbiret ret;
+ u32 evt = event->evt;
+
+ ret = sbi_sse_ecall(func, evt, 0, 0);
+ if (ret.error)
+ pr_err("Failed to execute func %lx, event %d, error %ld\n", func,
+ evt, ret.error);
+
+ return sbi_err_map_linux_errno(ret.error);
+}
+
+static int sse_event_register_local(struct sse_event *event)
+{
+ int ret;
+ struct sse_registered_event *reg_evt = per_cpu_ptr(event->local,
+ smp_processor_id());
+
+ ret = sse_sbi_register_event(event, reg_evt);
+ if (ret)
+ pr_err("Failed to register event %x: err %d\n", event->evt,
+ ret);
+
+ return ret;
+}
+
+static int sse_sbi_disable_event(struct sse_event *event)
+{
+ return sse_sbi_event_func(event, SBI_SSE_EVENT_DISABLE);
+}
+
+static int sse_sbi_enable_event(struct sse_event *event)
+{
+ return sse_sbi_event_func(event, SBI_SSE_EVENT_ENABLE);
+}
+
+static int sse_sbi_unregister_event(struct sse_event *event)
+{
+ return sse_sbi_event_func(event, SBI_SSE_EVENT_UNREGISTER);
+}
+
+struct sse_per_cpu_evt {
+ struct sse_event *event;
+ unsigned long func;
+ int error;
+};
+
+static void sse_event_per_cpu_func(void *info)
+{
+ int ret;
+ struct sse_per_cpu_evt *cpu_evt = info;
+
+ if (cpu_evt->func == SBI_SSE_EVENT_REGISTER)
+ ret = sse_event_register_local(cpu_evt->event);
+ else
+ ret = sse_sbi_event_func(cpu_evt->event, cpu_evt->func);
+
+ if (ret)
+ WRITE_ONCE(cpu_evt->error, 1);
+}
+
+static void sse_event_free(struct sse_event *event)
+{
+ unsigned int cpu;
+ struct sse_registered_event *reg_evt;
+
+ if (sse_event_is_global(event->evt)) {
+ sse_stack_free(event->global->stack);
+ } else {
+ for_each_possible_cpu(cpu) {
+ reg_evt = per_cpu_ptr(event->local, cpu);
+ sse_stack_free(reg_evt->stack);
+ }
+ free_percpu(event->local);
+ }
+
+ kfree(event);
+}
+
+int sse_event_enable(struct sse_event *event)
+{
+ int ret = 0;
+ struct sse_per_cpu_evt cpu_evt;
+
+ mutex_lock(&sse_mutex);
+
+ cpus_read_lock();
+ if (sse_event_is_global(event->evt)) {
+ /* Global events can only be unregister from target hart */
+ ret = sse_event_set_target_cpu_nolock(event, smp_processor_id());
+ if (ret)
+ goto out;
+
+ ret = sse_sbi_enable_event(event);
+ if (ret)
+ goto out;
+
+ } else {
+ cpu_evt.event = event;
+ cpu_evt.error = 0;
+ cpu_evt.func = SBI_SSE_EVENT_ENABLE;
+ on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1);
+ if (READ_ONCE(cpu_evt.error)) {
+ cpu_evt.func = SBI_SSE_EVENT_DISABLE;
+ on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1);
+ goto out;
+ }
+ }
+ cpus_read_unlock();
+
+out:
+ mutex_unlock(&sse_mutex);
+
+ return ret;
+}
+
+void sse_event_disable(struct sse_event *event)
+{
+ struct sse_per_cpu_evt cpu_evt;
+
+ mutex_lock(&sse_mutex);
+
+ if (sse_event_is_global(event->evt)) {
+ sse_sbi_disable_event(event);
+ } else {
+ cpu_evt.event = event;
+ cpu_evt.func = SBI_SSE_EVENT_DISABLE;
+ on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1);
+ }
+
+ mutex_unlock(&sse_mutex);
+}
+
+struct sse_event *sse_event_register(u32 evt, u32 priority,
+ sse_event_handler *handler, void *arg)
+{
+ struct sse_per_cpu_evt cpu_evt;
+ struct sse_event *event;
+ int ret = 0;
+
+ mutex_lock(&sse_mutex);
+ if (sse_event_get(evt)) {
+ pr_err("Event %x already registered\n", evt);
+ ret = -EEXIST;
+ goto out_unlock;
+ }
+
+ event = sse_event_alloc(evt, priority, handler, arg);
+ if (IS_ERR(event)) {
+ ret = PTR_ERR(event);
+ goto out_unlock;
+ }
+
+ cpus_read_lock();
+ if (sse_event_is_global(evt)) {
+ /* SSE spec mandates that the CPU registering the global event be the
+ * one set as the target hart, plus we don't know initial value
+ */
+ ret = sse_event_set_target_cpu_nolock(event, smp_processor_id());
+ if (ret)
+ goto err_event_free;
+
+ ret = sse_sbi_register_event(event, event->global);
+ if (ret)
+ goto err_event_free;
+
+ } else {
+ cpu_evt.event = event;
+ cpu_evt.error = 0;
+ cpu_evt.func = SBI_SSE_EVENT_REGISTER;
+ on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1);
+ if (READ_ONCE(cpu_evt.error)) {
+ cpu_evt.func = SBI_SSE_EVENT_UNREGISTER;
+ on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1);
+ goto err_event_free;
+ }
+ }
+ cpus_read_unlock();
+
+ spin_lock(&events_list_lock);
+ list_add(&event->list, &events);
+ spin_unlock(&events_list_lock);
+
+ mutex_unlock(&sse_mutex);
+
+ return event;
+
+err_event_free:
+ cpus_read_unlock();
+ sse_event_free(event);
+out_unlock:
+ mutex_unlock(&sse_mutex);
+
+ return ERR_PTR(ret);
+}
+
+void sse_event_unregister(struct sse_event *event)
+{
+ int ret;
+ struct sse_per_cpu_evt cpu_evt;
+
+ mutex_lock(&sse_mutex);
+
+ if (sse_event_is_global(event->evt)) {
+ /* Global events can only be unregister from target hart */
+ ret = sse_event_set_target_cpu_nolock(event, smp_processor_id());
+ WARN_ON(ret);
+ sse_sbi_unregister_event(event);
+ } else {
+ cpu_evt.event = event;
+ cpu_evt.func = SBI_SSE_EVENT_UNREGISTER;
+ on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1);
+ }
+
+ spin_lock(&events_list_lock);
+ list_del(&event->list);
+ spin_unlock(&events_list_lock);
+
+ sse_event_free(event);
+
+ mutex_unlock(&sse_mutex);
+}
+
+static int sse_cpu_online(unsigned int cpu)
+{
+ struct sse_event *sse_evt;
+
+ spin_lock(&events_list_lock);
+ list_for_each_entry(sse_evt, &events, list) {
+ if (sse_event_is_global(sse_evt->evt))
+ continue;
+
+ sse_event_register_local(sse_evt);
+ }
+
+ spin_unlock(&events_list_lock);
+
+ return 0;
+}
+
+static int sse_cpu_teardown(unsigned int cpu)
+{
+ unsigned int next_cpu;
+ struct sse_event *sse_evt;
+
+ spin_lock(&events_list_lock);
+ list_for_each_entry(sse_evt, &events, list) {
+ if (!sse_event_is_global(sse_evt->evt)) {
+ sse_sbi_unregister_event(sse_evt);
+ continue;
+ }
+
+ if (sse_evt->cpu != smp_processor_id())
+ continue;
+
+ /* Update destination hart */
+ next_cpu = cpumask_any_but(cpu_online_mask, cpu);
+ sse_event_set_target_cpu(sse_evt, next_cpu);
+ }
+ spin_unlock(&events_list_lock);
+
+ return 0;
+}
+
+static int __init sse_init(void)
+{
+ int cpu, ret;
+
+ if (sbi_probe_extension(SBI_EXT_SSE) <= 0) {
+ pr_err("Missing SBI SSE extension\n");
+ return -EOPNOTSUPP;
+ }
+ pr_info("SBI SSE extension detected\n");
+
+ for_each_possible_cpu(cpu)
+ INIT_LIST_HEAD(&events);
+
+ ret = cpuhp_setup_state_nocalls(CPUHP_AP_ONLINE_DYN, "riscv/sse:online",
+ sse_cpu_online, sse_cpu_teardown);
+ if (ret < 0)
+ return ret;
+
+ sse_available = true;
+
+ return 0;
+}
+device_initcall(sse_init);
diff --git a/include/linux/riscv_sse.h b/include/linux/riscv_sse.h
new file mode 100644
index 000000000000..3ca145368c50
--- /dev/null
+++ b/include/linux/riscv_sse.h
@@ -0,0 +1,56 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2023 Rivos Inc.
+ */
+
+#ifndef __LINUX_RISCV_SSE_H
+#define __LINUX_RISCV_SSE_H
+
+#include <linux/types.h>
+#include <linux/linkage.h>
+
+struct sse_interrupted_state;
+struct sse_event;
+struct pt_regs;
+
+typedef int (sse_event_handler)(u32 event_num, void *arg, struct pt_regs *regs);
+
+#ifdef CONFIG_RISCV_SSE
+
+struct sse_event *sse_event_register(u32 event_num, u32 priority,
+ sse_event_handler *handler, void *arg);
+
+void sse_event_unregister(struct sse_event *evt);
+
+int sse_event_set_target_cpu(struct sse_event *sse_evt, unsigned int cpu);
+
+int sse_event_enable(struct sse_event *sse_evt);
+
+void sse_event_disable(struct sse_event *sse_evt);
+
+#else
+static inline int sse_event_register(struct sse_event *evt, u32 priority,
+ sse_event_handler *handler, void *arg)
+{
+ return -EOPNOTSUPP;
+}
+
+static inline void sse_event_unregister(struct sse_event *evt) {}
+
+static inline int sse_event_set_target_cpu(struct sse_event *sse_evt,
+ unsigned int cpu)
+{
+ return -EOPNOTSUPP;
+}
+
+static inline int sse_event_enable(struct sse_event *sse_evt)
+{
+ return -EOPNOTSUPP;
+}
+
+static inline void sse_event_disable(struct sse_event *sse_evt) {}
+
+
+#endif
+
+#endif /* __LINUX_RISCV_SSE_H */
--
2.42.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [RFC PATCH 2/3] riscv: add support for SBI Supervisor Software Events extension
@ 2023-10-26 14:31 ` Clément Léger
0 siblings, 0 replies; 25+ messages in thread
From: Clément Léger @ 2023-10-26 14:31 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Albert Ou, Atish Patra, Anup Patel,
linux-riscv, linux-kernel, linux-arm-kernel
Cc: Clément Léger, Himanshu Chauhan, Xu Lu
The SBI SSE extension allows the supervisor software to be notified by
the SBI of specific events that are not maskable. The context switch is
handled entirely by the firmware which will save all registers into a
supervisor provided context and restore a set of registers from this
same context. This context contains all the RV GP registers and thus
will also switch the stack.
Since each events (and each CPU for local events) have their own
context (and thus their own set of register), allocate a stack for each
event (and for each cpu for local events). This can be modified if
needed to use a single stack per cpu for SSE event but will require a
bit more additional plumbing.
When completing the event, if we were coming from kernel with interrupts
disabled, simply return there. If coming from userspace or kernel with
interrupts enabled, simulate an interrupt exception to allow delivery of
signal to user task. For instance this can happen, when a RAS event has
been generated and a SIGBUS has been sent to a task.
Signed-off-by: Clément Léger <cleger@rivosinc.com>
---
arch/riscv/include/asm/asm-prototypes.h | 5 +
arch/riscv/include/asm/sse.h | 94 +++++
arch/riscv/kernel/Makefile | 1 +
arch/riscv/kernel/asm-offsets.c | 17 +
arch/riscv/kernel/entry.S | 156 ++++++++
arch/riscv/kernel/sse.c | 97 +++++
arch/riscv/kernel/stacktrace.c | 13 +
arch/riscv/kernel/vmlinux.lds.S | 6 +
drivers/firmware/Kconfig | 10 +
drivers/firmware/Makefile | 1 +
drivers/firmware/riscv_sse.c | 496 ++++++++++++++++++++++++
include/linux/riscv_sse.h | 56 +++
12 files changed, 952 insertions(+)
create mode 100644 arch/riscv/include/asm/sse.h
create mode 100644 arch/riscv/kernel/sse.c
create mode 100644 drivers/firmware/riscv_sse.c
create mode 100644 include/linux/riscv_sse.h
diff --git a/arch/riscv/include/asm/asm-prototypes.h b/arch/riscv/include/asm/asm-prototypes.h
index 61ba8ed43d8f..c622ed1db33c 100644
--- a/arch/riscv/include/asm/asm-prototypes.h
+++ b/arch/riscv/include/asm/asm-prototypes.h
@@ -3,6 +3,8 @@
#define _ASM_RISCV_PROTOTYPES_H
#include <linux/ftrace.h>
+#include <linux/riscv_sse.h>
+#include <asm/sse.h>
#include <asm-generic/asm-prototypes.h>
long long __lshrti3(long long a, int b);
@@ -29,5 +31,8 @@ asmlinkage unsigned long get_overflow_stack(void);
asmlinkage void handle_bad_stack(struct pt_regs *regs);
asmlinkage void do_page_fault(struct pt_regs *regs);
asmlinkage void do_irq(struct pt_regs *regs);
+asmlinkage unsigned long do_sse(unsigned long evt,
+ struct sse_interrupted_state *i_state,
+ sse_event_handler *handler, void *arg);
#endif /* _ASM_RISCV_PROTOTYPES_H */
diff --git a/arch/riscv/include/asm/sse.h b/arch/riscv/include/asm/sse.h
new file mode 100644
index 000000000000..e663224296df
--- /dev/null
+++ b/arch/riscv/include/asm/sse.h
@@ -0,0 +1,94 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (C) 2023 Rivos Inc.
+ */
+#ifndef __ASM_SSE_H
+#define __ASM_SSE_H
+
+#include <linux/riscv_sse.h>
+
+
+struct sse_interrupted_state {
+ unsigned long pc;
+ unsigned long ra;
+ unsigned long sp;
+ unsigned long gp;
+ unsigned long tp;
+ unsigned long t0;
+ unsigned long t1;
+ unsigned long t2;
+ unsigned long s0;
+ unsigned long s1;
+ unsigned long a0;
+ unsigned long a1;
+ unsigned long a2;
+ unsigned long a3;
+ unsigned long a4;
+ unsigned long a5;
+ unsigned long a6;
+ unsigned long a7;
+ unsigned long s2;
+ unsigned long s3;
+ unsigned long s4;
+ unsigned long s5;
+ unsigned long s6;
+ unsigned long s7;
+ unsigned long s8;
+ unsigned long s9;
+ unsigned long s10;
+ unsigned long s11;
+ unsigned long t3;
+ unsigned long t4;
+ unsigned long t5;
+ unsigned long t6;
+ unsigned long exec_mode;
+};
+
+struct sse_entry_state {
+ unsigned long pc;
+ unsigned long ra;
+ unsigned long sp;
+ unsigned long gp;
+ unsigned long tp;
+ unsigned long t0;
+ unsigned long t1;
+ unsigned long t2;
+ unsigned long s0;
+ unsigned long s1;
+ unsigned long a0;
+ unsigned long a1;
+ unsigned long a2;
+ unsigned long a3;
+ unsigned long a4;
+ unsigned long a5;
+ unsigned long a6;
+ unsigned long a7;
+ unsigned long s2;
+ unsigned long s3;
+ unsigned long s4;
+ unsigned long s5;
+ unsigned long s6;
+ unsigned long s7;
+ unsigned long s8;
+ unsigned long s9;
+ unsigned long s10;
+ unsigned long s11;
+ unsigned long t3;
+ unsigned long t4;
+ unsigned long t5;
+ unsigned long t6;
+};
+
+struct sse_handler_context {
+ struct sse_entry_state e_state;
+ struct sse_interrupted_state i_state;
+};
+
+unsigned long *sse_stack_alloc(unsigned int cpu, unsigned int size);
+void sse_stack_free(unsigned long *stack);
+
+struct sse_handler_context;
+void sse_handler_context_init(struct sse_handler_context *ctx, void *stack,
+ u32 evt, sse_event_handler *handler, void *arg);
+
+#endif
diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile
index 95cf25d48405..571b2aaf669b 100644
--- a/arch/riscv/kernel/Makefile
+++ b/arch/riscv/kernel/Makefile
@@ -79,6 +79,7 @@ obj-$(CONFIG_DYNAMIC_FTRACE) += mcount-dyn.o
obj-$(CONFIG_PERF_EVENTS) += perf_callchain.o
obj-$(CONFIG_HAVE_PERF_REGS) += perf_regs.o
obj-$(CONFIG_RISCV_SBI) += sbi.o
+obj-$(CONFIG_RISCV_SSE) += sse.o
ifeq ($(CONFIG_RISCV_SBI), y)
obj-$(CONFIG_SMP) += sbi-ipi.o
obj-$(CONFIG_SMP) += cpu_ops_sbi.o
diff --git a/arch/riscv/kernel/asm-offsets.c b/arch/riscv/kernel/asm-offsets.c
index d6a75aac1d27..1776448e67d2 100644
--- a/arch/riscv/kernel/asm-offsets.c
+++ b/arch/riscv/kernel/asm-offsets.c
@@ -14,7 +14,10 @@
#include <asm/thread_info.h>
#include <asm/ptrace.h>
#include <asm/cpu_ops_sbi.h>
+#include <asm/sbi.h>
+#include <asm/sse.h>
#include <asm/suspend.h>
+#include <asm/stacktrace.h>
void asm_offsets(void);
@@ -479,4 +482,18 @@ void asm_offsets(void)
OFFSET(KERNEL_MAP_VIRT_ADDR, kernel_mapping, virt_addr);
OFFSET(SBI_HART_BOOT_TASK_PTR_OFFSET, sbi_hart_boot_data, task_ptr);
OFFSET(SBI_HART_BOOT_STACK_PTR_OFFSET, sbi_hart_boot_data, stack_ptr);
+
+ OFFSET(SSE_INTERRUPTED_EXEC_MODE, sse_interrupted_state, exec_mode);
+ OFFSET(SSE_INTERRUPTED_S0, sse_interrupted_state, s0);
+ OFFSET(SSE_INTERRUPTED_TP, sse_interrupted_state, tp);
+ OFFSET(SSE_INTERRUPTED_PC, sse_interrupted_state, pc);
+
+ DEFINE(STACKFRAME_SIZE_ON_STACK, ALIGN(sizeof(struct stackframe),
+ STACK_ALIGN));
+ OFFSET(STACKFRAME_FP, stackframe, fp);
+ OFFSET(STACKFRAME_RA, stackframe, ra);
+ DEFINE(SBI_EXT_SSE, SBI_EXT_SSE);
+ DEFINE(SBI_SSE_EVENT_COMPLETE, SBI_SSE_EVENT_COMPLETE);
+
+ DEFINE(ASM_PAGE_OFFSET, CONFIG_PAGE_OFFSET);
}
diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
index 143a2bb3e697..4adf7b21a568 100644
--- a/arch/riscv/kernel/entry.S
+++ b/arch/riscv/kernel/entry.S
@@ -14,17 +14,49 @@
#include <asm/asm-offsets.h>
#include <asm/errata_list.h>
+#ifdef CONFIG_RISCV_SSE
+#define IN_TP 0
+#define IN_SSCRATCH 1
+#define PRV_U_TASK_LOC(loc) (loc << 0)
+#define PRV_S_TASK_LOC(loc) (loc << 1)
+#define TASK_LOC(s_loc, u_loc) (PRV_U_TASK_LOC(u_loc) | PRV_S_TASK_LOC(s_loc))
+#define TASK_LOC_ENTRY_SIZE (RISCV_SZPTR + 1)
+
+/* __SSE_TASK_LOC - Annotate e'xception code with the location of current task
+ * struct when coming from S/U mode
+ *
+ * When entering handle_exception(), the current task struct is located either
+ * in tp or sscratch depending on interrupted mode. Since SSE handlers can be
+ * triggered at any time during the execution of the kernel, we need to be able
+ * to retrieve the current task struct even if in handle_exception. This macro
+ * create an entry into the __task_loc section that holds the location of
+ * the current task struct for the subsequent assembly located after that up to
+ * _ret_from_exception_end or the next __TASK_LOC.
+ */
+#define __SSE_TASK_LOC(s_loc, u_loc) \
+ .pushsection __task_loc,"a"; \
+ RISCV_PTR 99f; \
+ .byte TASK_LOC(s_loc, u_loc); \
+ .popsection; \
+ 99:
+#else
+#define __SSE_TASK_LOC(s_loc, u_loc)
+#endif
+
SYM_CODE_START(handle_exception)
/*
* If coming from userspace, preserve the user thread pointer and load
* the kernel thread pointer. If we came from the kernel, the scratch
* register will contain 0, and we should continue on the current TP.
*/
+__SSE_TASK_LOC(IN_TP, IN_SSCRATCH)
csrrw tp, CSR_SCRATCH, tp
+__SSE_TASK_LOC(IN_SSCRATCH, IN_TP)
bnez tp, _save_context
_restore_kernel_tpsp:
csrr tp, CSR_SCRATCH
+__SSE_TASK_LOC(IN_TP, IN_TP)
REG_S sp, TASK_TI_KERNEL_SP(tp)
#ifdef CONFIG_VMAP_STACK
@@ -151,6 +183,7 @@ SYM_CODE_START_NOALIGN(ret_from_exception)
REG_SC x0, a2, PT_EPC(sp)
csrw CSR_STATUS, a0
+__SSE_TASK_LOC(IN_TP, IN_SSCRATCH)
csrw CSR_EPC, a2
REG_L x1, PT_RA(sp)
@@ -166,6 +199,7 @@ SYM_CODE_START_NOALIGN(ret_from_exception)
#else
sret
#endif
+_ret_from_exception_end:
SYM_CODE_END(ret_from_exception)
#ifdef CONFIG_VMAP_STACK
@@ -268,6 +302,125 @@ SYM_CODE_START(ret_from_fork)
tail syscall_exit_to_user_mode
SYM_CODE_END(ret_from_fork)
+#ifdef CONFIG_RISCV_SSE
+SYM_CODE_START(handle_sse)
+ move s3, a0
+ /*
+ * Depending on where the sse event interrupted the kernel execution,
+ * sscratch content might be 0 or not even if kernel was interrupted.
+ * This might happen if the sse event was triggered while in
+ * handle_exception() right after entry. In that case, sscratch might
+ * contain 0 if coming from kernel. In order to handle that easily,
+ * simply save sscratch content and restore it when returning from sse
+ * event.
+ */
+ csrr s4, CSR_SSCRATCH
+ /*
+ * Save CAUSE and TVAL in case of nested exceptions, EPC/STATUS are
+ * already saved/restored by handle_exception() for nested exceptions.
+ */
+ csrr s5, CSR_SCAUSE
+ csrr s6, CSR_STVAL
+ REG_L t0, SSE_INTERRUPTED_EXEC_MODE(a1)
+ andi t1, t0, 0x1
+
+ /* Userspace was interrupted, simply restore TP from scratch */
+ bnez t1, _sse_from_kernel
+ move tp, s4
+ j _call_do_sse
+
+_sse_from_kernel:
+ REG_L t0, SSE_INTERRUPTED_PC(a1)
+#ifdef CONFIG_FRAME_POINTER
+ /*
+ * Else, kernel was interrupted and we will create a correct stack frame
+ * from interrupted context.
+ */
+ addi sp, sp, -(STACKFRAME_SIZE_ON_STACK)
+ REG_L t2, SSE_INTERRUPTED_S0(a1)
+ REG_S t2, STACKFRAME_FP(sp)
+ REG_S t0, STACKFRAME_RA(sp)
+ addi s0, sp, STACKFRAME_SIZE_ON_STACK
+#endif
+ REG_L tp, SSE_INTERRUPTED_TP(a1)
+
+ /*
+ * If interrupting the kernel during exception handling
+ * (see handle_exception), then, we might have tp either in SSCRATCH or
+ * in tp, this part is non regular and requires some more work to
+ * determine were is located the current task.
+ */
+ la t1, handle_exception
+ la t2, _ret_from_exception_end
+ bltu t0, t1, _call_do_sse
+ bltu t0, t2, _sse_exception_slow_path
+
+_call_do_sse:
+
+ /* TODO: remove this sanity check once sure everything works ! */
+ li t1, ASM_PAGE_OFFSET
+ bgeu tp, t1, _tp_ok
+ la tp, init_task
+ la a0, sse_panic_string
+ la t0, panic
+ jalr t0
+_tp_ok:
+
+ csrw CSR_SCRATCH, x0
+ la ra, ret_from_sse
+ tail do_sse
+
+.global ret_from_sse
+ret_from_sse:
+ /* Restore saved CSRs */
+ csrw CSR_SSCRATCH, s4
+ csrw CSR_SCAUSE, s5
+ csrw CSR_STVAL, s6
+ li a2, 0
+ move a1, a0
+ move a0, s3
+ li a7, SBI_EXT_SSE
+ li a6, SBI_SSE_EVENT_COMPLETE
+ ecall
+
+/*
+ * t0 contains interrupted pc
+ * t1 contains handle_exception address
+ * When called, t0 must be in [handle_exception, _ret_from_exception_end[
+ */
+_sse_exception_slow_path:
+ la t3, __stop___task_loc
+ add t3, t3, -TASK_LOC_ENTRY_SIZE
+ la t4, __start___task_loc
+ /*
+ * Reverse iterate the task location section to find where is located
+ * the task struct
+ */
+1:
+ REG_L t2, 0(t3)
+ bgeu t0, t2, 2f
+ add t3, t3, -TASK_LOC_ENTRY_SIZE
+ bgeu t3, t4, 1b
+
+2:
+ lbu t2, RISCV_SZPTR(t3)
+
+ /* Get the value of SR_SPP */
+ csrr t1, CSR_SSTATUS
+ andi t1, t1, SR_SPP
+ snez t1, t1
+
+ srl t2, t2, t1
+ andi t2, t2, 1
+ beqz t2, _call_do_sse
+
+_restore_tp_from_sscratch:
+ csrr tp, CSR_SCRATCH
+ j _call_do_sse
+
+SYM_CODE_END(handle_sse)
+#endif
+
/*
* Integer register context switch
* The callee-saved registers must be saved and restored.
@@ -351,3 +504,6 @@ SYM_CODE_START(__user_rt_sigreturn)
ecall
SYM_CODE_END(__user_rt_sigreturn)
#endif
+
+sse_panic_string:
+ .ascii "SSE TP is invalid, last state %px"
diff --git a/arch/riscv/kernel/sse.c b/arch/riscv/kernel/sse.c
new file mode 100644
index 000000000000..69c2607d9dbf
--- /dev/null
+++ b/arch/riscv/kernel/sse.c
@@ -0,0 +1,97 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2023 Rivos Inc.
+ */
+#include <linux/nmi.h>
+#include <linux/bitfield.h>
+#include <linux/riscv_sse.h>
+
+#include <asm/irq_stack.h>
+#include <asm/sbi.h>
+#include <asm/sse.h>
+
+#define SSE_PRIVILEGE_MODE_BIT BIT(0)
+#define SSE_SPIE_BIT BIT(2)
+
+#define sse_privilege_mode(exec_mode) FIELD_GET(SSE_PRIVILEGE_MODE_BIT, exec_mode)
+#define sse_spie(exec_mode) FIELD_GET(SSE_SPIE_BIT, exec_mode)
+
+register unsigned long gp_in_global __asm__("gp");
+
+extern asmlinkage void handle_exception(void);
+extern asmlinkage void handle_sse(unsigned long evt,
+ struct sse_interrupted_state *i_state,
+ sse_event_handler *handler, void *arg);
+
+static void sse_get_pt_regs(struct sse_interrupted_state *i_state,
+ struct pt_regs *regs)
+{
+ unsigned long sstatus = csr_read(CSR_SSTATUS);
+
+ memcpy(regs, i_state, offsetof(struct sse_interrupted_state, exec_mode));
+ regs->status = sstatus & ~SR_SPP;
+ regs->status |= FIELD_PREP(SR_SPP, sse_privilege_mode(i_state->exec_mode));
+}
+
+unsigned long do_sse(unsigned long evt, struct sse_interrupted_state *i_state,
+ sse_event_handler *handler, void *arg)
+{
+ int ret;
+ struct pt_regs regs;
+
+ nmi_enter();
+
+ sse_get_pt_regs(i_state, ®s);
+ ret = handler(evt, arg, ®s);
+ if (ret)
+ pr_warn("event %lx handler failed with error %d\n", evt, ret);
+
+ /* The SSE delivery path does not uses the "standard" exception path and
+ * thus does not process any pending signal/softirqs. Some drivers might
+ * enqueue pending work that needs to be handled as soon as possible.
+ * For that purpose, set the software interrupt pending bit
+ */
+ csr_set(CSR_IP, IE_SIE);
+
+ nmi_exit();
+
+ return ret ? SBI_SSE_HANDLER_FAILED : SBI_SSE_HANDLER_SUCCESS;
+}
+
+#ifdef CONFIG_VMAP_STACK
+unsigned long *sse_stack_alloc(unsigned int cpu, unsigned int size)
+{
+ return arch_alloc_vmap_stack(size, cpu_to_node(cpu));
+}
+
+void sse_stack_free(unsigned long *stack)
+{
+ vfree(stack);
+}
+#else /* CONFIG_VMAP_STACK */
+
+unsigned long *sse_stack_alloc(unsigned int cpu, unsigned int size)
+{
+ return kmalloc(size, GFP_KERNEL);
+}
+
+void sse_stack_free(unsigned long *stack)
+{
+ kfree(stack);
+}
+
+#endif /* CONFIG_VMAP_STACK */
+
+void sse_handler_context_init(struct sse_handler_context *ctx, void *stack,
+ u32 evt, sse_event_handler *handler, void *arg)
+{
+ ctx->e_state.pc = (unsigned long)handle_sse;
+ ctx->e_state.gp = gp_in_global;
+ ctx->e_state.sp = (unsigned long)stack + THREAD_SIZE;
+
+ /* This must match handle_sse expected parameter order */
+ ctx->e_state.a0 = evt;
+ ctx->e_state.a1 = (unsigned long)&ctx->i_state;
+ ctx->e_state.a2 = (unsigned long)handler;
+ ctx->e_state.a3 = (unsigned long)arg;
+}
diff --git a/arch/riscv/kernel/stacktrace.c b/arch/riscv/kernel/stacktrace.c
index 64a9c093aef9..a3d08792e3b5 100644
--- a/arch/riscv/kernel/stacktrace.c
+++ b/arch/riscv/kernel/stacktrace.c
@@ -17,6 +17,7 @@
#ifdef CONFIG_FRAME_POINTER
extern asmlinkage void ret_from_exception(void);
+extern asmlinkage void ret_from_sse(void);
void notrace walk_stackframe(struct task_struct *task, struct pt_regs *regs,
bool (*fn)(void *, unsigned long), void *arg)
@@ -68,6 +69,18 @@ void notrace walk_stackframe(struct task_struct *task, struct pt_regs *regs,
pc = ((struct pt_regs *)sp)->epc;
fp = ((struct pt_regs *)sp)->s0;
+ } else if (pc == (unsigned long)ret_from_sse) {
+ if (unlikely(!fn(arg, pc)))
+ break;
+ /* We don't have pt_regs when handling SSE
+ * events but we build a custom stackframe,
+ * moreover, the stack changes across boundaries
+ * so update it to avoid failing the checks above
+ */
+ frame = (struct stackframe *)fp - 1;
+ fp = frame->fp;
+ pc = frame->ra;
+ sp = fp - sizeof(struct stackframe);
}
}
diff --git a/arch/riscv/kernel/vmlinux.lds.S b/arch/riscv/kernel/vmlinux.lds.S
index 492dd4b8f3d6..f7e9cde2e832 100644
--- a/arch/riscv/kernel/vmlinux.lds.S
+++ b/arch/riscv/kernel/vmlinux.lds.S
@@ -120,6 +120,12 @@ SECTIONS
/* Start of data section */
_sdata = .;
RO_DATA(SECTION_ALIGN)
+
+#ifdef CONFIG_RISCV_SSE
+ __task_loc : {
+ BOUNDED_SECTION_BY(__task_loc, ___task_loc)
+ }
+#endif
.srodata : {
*(.srodata*)
}
diff --git a/drivers/firmware/Kconfig b/drivers/firmware/Kconfig
index b59e3041fd62..1749595b4150 100644
--- a/drivers/firmware/Kconfig
+++ b/drivers/firmware/Kconfig
@@ -226,6 +226,16 @@ config QCOM_SCM_DOWNLOAD_MODE_DEFAULT
Say Y here to enable "download mode" by default.
+config RISCV_SSE
+ bool
+ depends on RISCV_SBI
+ default y
+ help
+ The Supervisor Software Events support allow the SBI to deliver
+ NMI-like notifications to the supervisor mode software. When enable,
+ this option provides support to register callbacks on specific SSE
+ events.
+
config SYSFB
bool
select BOOT_VESA_SUPPORT
diff --git a/drivers/firmware/Makefile b/drivers/firmware/Makefile
index 28fcddcd688f..b0fcb0a74642 100644
--- a/drivers/firmware/Makefile
+++ b/drivers/firmware/Makefile
@@ -20,6 +20,7 @@ obj-$(CONFIG_RASPBERRYPI_FIRMWARE) += raspberrypi.o
obj-$(CONFIG_FW_CFG_SYSFS) += qemu_fw_cfg.o
obj-$(CONFIG_QCOM_SCM) += qcom-scm.o
qcom-scm-objs += qcom_scm.o qcom_scm-smc.o qcom_scm-legacy.o
+obj-$(CONFIG_RISCV_SSE) += riscv_sse.o
obj-$(CONFIG_SYSFB) += sysfb.o
obj-$(CONFIG_SYSFB_SIMPLEFB) += sysfb_simplefb.o
obj-$(CONFIG_TI_SCI_PROTOCOL) += ti_sci.o
diff --git a/drivers/firmware/riscv_sse.c b/drivers/firmware/riscv_sse.c
new file mode 100644
index 000000000000..5447ed0d3ae4
--- /dev/null
+++ b/drivers/firmware/riscv_sse.c
@@ -0,0 +1,496 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (C) 2023 Rivos Inc.
+ */
+
+#define pr_fmt(fmt) "sse: " fmt
+
+#include <linux/cpu.h>
+#include <linux/cpuhotplug.h>
+#include <linux/hardirq.h>
+#include <linux/list.h>
+#include <linux/percpu-defs.h>
+#include <linux/riscv_sse.h>
+#include <linux/slab.h>
+
+#include <asm/sbi.h>
+#include <asm/sse.h>
+
+struct sse_registered_event {
+ struct sse_handler_context ctx;
+ unsigned long *stack;
+};
+
+struct sse_event {
+ struct list_head list;
+ u32 evt;
+ u32 priority;
+ unsigned int cpu;
+
+ union {
+ struct sse_registered_event *global;
+ struct sse_registered_event __percpu *local;
+ };
+};
+
+static bool sse_available;
+static DEFINE_SPINLOCK(events_list_lock);
+static LIST_HEAD(events);
+static DEFINE_MUTEX(sse_mutex);
+
+static struct sbiret sbi_sse_ecall(int fid, unsigned long arg0,
+ unsigned long arg1, unsigned long arg2)
+{
+ return sbi_ecall(SBI_EXT_SSE, fid, arg0, arg1, arg2, 0, 0, 0);
+}
+
+static bool sse_event_is_global(u32 evt)
+{
+ return !!(evt & SBI_SSE_EVENT_GLOBAL);
+}
+
+static
+struct sse_event *sse_event_get(u32 evt)
+{
+ struct sse_event *sse_evt = NULL, *tmp;
+
+ spin_lock(&events_list_lock);
+ list_for_each_entry(tmp, &events, list) {
+ if (tmp->evt == evt) {
+ sse_evt = tmp;
+ break;
+ }
+ }
+ spin_unlock(&events_list_lock);
+
+ return sse_evt;
+}
+
+static int sse_event_set_target_cpu_nolock(struct sse_event *event, unsigned int cpu)
+{
+ unsigned int hart_id = cpuid_to_hartid_map(cpu);
+ u32 evt = event->evt;
+ struct sbiret sret;
+
+ if (!sse_event_is_global(evt))
+ return -EINVAL;
+
+ do {
+ sret = sbi_sse_ecall(SBI_SSE_EVENT_ATTR_SET, evt,
+ SBI_SSE_EVENT_ATTR_HART_ID, hart_id);
+ if (sret.error && sret.error != SBI_ERR_BUSY) {
+ pr_err("Failed to set event %x hart id, error %ld\n", evt,
+ sret.error);
+ return sbi_err_map_linux_errno(sret.error);
+ }
+ } while (sret.error);
+
+ event->cpu = cpu;
+
+ return 0;
+}
+
+
+int sse_event_set_target_cpu(struct sse_event *event, unsigned int cpu)
+{
+ int ret;
+
+ mutex_lock(&sse_mutex);
+ ret = sse_event_set_target_cpu_nolock(event, cpu);
+ mutex_unlock(&sse_mutex);
+
+ return ret;
+}
+
+static int sse_event_init_registered(unsigned int cpu, u32 evt,
+ struct sse_registered_event *reg_evt,
+ sse_event_handler *handler, void *arg)
+{
+ unsigned long *stack;
+
+ stack = sse_stack_alloc(cpu, THREAD_SIZE);
+ if (!stack)
+ return -ENOMEM;
+
+ reg_evt->stack = stack;
+
+ sse_handler_context_init(®_evt->ctx, stack, evt, handler, arg);
+
+ return 0;
+}
+
+static struct sse_event *sse_event_alloc(u32 evt,
+ u32 priority,
+ sse_event_handler *handler, void *arg)
+{
+ int err;
+ unsigned int cpu;
+ struct sse_event *event;
+ struct sse_registered_event __percpu *reg_evts;
+ struct sse_registered_event *reg_evt;
+
+ event = kzalloc(sizeof(*event), GFP_KERNEL);
+ if (!event)
+ return ERR_PTR(-ENOMEM);
+
+ event->evt = evt;
+ event->priority = priority;
+
+ if (sse_event_is_global(evt)) {
+ reg_evt = kzalloc(sizeof(*reg_evt), GFP_KERNEL);
+ if (!reg_evt) {
+ err = -ENOMEM;
+ goto err_alloc_reg_evt;
+ }
+
+ event->global = reg_evt;
+ err = sse_event_init_registered(smp_processor_id(), evt,
+ reg_evt, handler, arg);
+ if (err)
+ goto err_alloc_reg_evt;
+
+
+ } else {
+ reg_evts = alloc_percpu(struct sse_registered_event);
+ if (!reg_evts) {
+ err = -ENOMEM;
+ goto err_alloc_reg_evt;
+ }
+
+ event->local = reg_evts;
+
+ for_each_possible_cpu(cpu) {
+ reg_evt = per_cpu_ptr(reg_evts, cpu);
+
+ err = sse_event_init_registered(cpu, evt, reg_evt,
+ handler, arg);
+ if (err)
+ goto err_alloc_reg_evt;
+
+ }
+ }
+
+ return event;
+
+err_alloc_reg_evt:
+ kfree(event);
+
+ return ERR_PTR(err);
+}
+
+static int sse_sbi_register_event(struct sse_event *event,
+ struct sse_registered_event *reg_evt)
+{
+ struct sbiret ret;
+ phys_addr_t phys;
+ u32 evt = event->evt;
+
+ ret = sbi_sse_ecall(SBI_SSE_EVENT_ATTR_SET, evt,
+ SBI_SSE_EVENT_ATTR_PRIORITY, event->priority);
+ if (ret.error) {
+ pr_err("Failed to set event %x priority, error %ld\n", evt,
+ ret.error);
+ return sbi_err_map_linux_errno(ret.error);
+ }
+
+ if (sse_event_is_global(event->evt))
+ phys = virt_to_phys(®_evt->ctx);
+ else
+ phys = per_cpu_ptr_to_phys(®_evt->ctx);
+
+ ret = sbi_sse_ecall(SBI_SSE_EVENT_REGISTER, evt, phys, 0);
+ if (ret.error)
+ pr_err("Failed to register event %d, error %ld\n", evt,
+ ret.error);
+
+ return sbi_err_map_linux_errno(ret.error);
+}
+
+static int sse_sbi_event_func(struct sse_event *event, unsigned long func)
+{
+ struct sbiret ret;
+ u32 evt = event->evt;
+
+ ret = sbi_sse_ecall(func, evt, 0, 0);
+ if (ret.error)
+ pr_err("Failed to execute func %lx, event %d, error %ld\n", func,
+ evt, ret.error);
+
+ return sbi_err_map_linux_errno(ret.error);
+}
+
+static int sse_event_register_local(struct sse_event *event)
+{
+ int ret;
+ struct sse_registered_event *reg_evt = per_cpu_ptr(event->local,
+ smp_processor_id());
+
+ ret = sse_sbi_register_event(event, reg_evt);
+ if (ret)
+ pr_err("Failed to register event %x: err %d\n", event->evt,
+ ret);
+
+ return ret;
+}
+
+static int sse_sbi_disable_event(struct sse_event *event)
+{
+ return sse_sbi_event_func(event, SBI_SSE_EVENT_DISABLE);
+}
+
+static int sse_sbi_enable_event(struct sse_event *event)
+{
+ return sse_sbi_event_func(event, SBI_SSE_EVENT_ENABLE);
+}
+
+static int sse_sbi_unregister_event(struct sse_event *event)
+{
+ return sse_sbi_event_func(event, SBI_SSE_EVENT_UNREGISTER);
+}
+
+struct sse_per_cpu_evt {
+ struct sse_event *event;
+ unsigned long func;
+ int error;
+};
+
+static void sse_event_per_cpu_func(void *info)
+{
+ int ret;
+ struct sse_per_cpu_evt *cpu_evt = info;
+
+ if (cpu_evt->func == SBI_SSE_EVENT_REGISTER)
+ ret = sse_event_register_local(cpu_evt->event);
+ else
+ ret = sse_sbi_event_func(cpu_evt->event, cpu_evt->func);
+
+ if (ret)
+ WRITE_ONCE(cpu_evt->error, 1);
+}
+
+static void sse_event_free(struct sse_event *event)
+{
+ unsigned int cpu;
+ struct sse_registered_event *reg_evt;
+
+ if (sse_event_is_global(event->evt)) {
+ sse_stack_free(event->global->stack);
+ } else {
+ for_each_possible_cpu(cpu) {
+ reg_evt = per_cpu_ptr(event->local, cpu);
+ sse_stack_free(reg_evt->stack);
+ }
+ free_percpu(event->local);
+ }
+
+ kfree(event);
+}
+
+int sse_event_enable(struct sse_event *event)
+{
+ int ret = 0;
+ struct sse_per_cpu_evt cpu_evt;
+
+ mutex_lock(&sse_mutex);
+
+ cpus_read_lock();
+ if (sse_event_is_global(event->evt)) {
+ /* Global events can only be unregister from target hart */
+ ret = sse_event_set_target_cpu_nolock(event, smp_processor_id());
+ if (ret)
+ goto out;
+
+ ret = sse_sbi_enable_event(event);
+ if (ret)
+ goto out;
+
+ } else {
+ cpu_evt.event = event;
+ cpu_evt.error = 0;
+ cpu_evt.func = SBI_SSE_EVENT_ENABLE;
+ on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1);
+ if (READ_ONCE(cpu_evt.error)) {
+ cpu_evt.func = SBI_SSE_EVENT_DISABLE;
+ on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1);
+ goto out;
+ }
+ }
+ cpus_read_unlock();
+
+out:
+ mutex_unlock(&sse_mutex);
+
+ return ret;
+}
+
+void sse_event_disable(struct sse_event *event)
+{
+ struct sse_per_cpu_evt cpu_evt;
+
+ mutex_lock(&sse_mutex);
+
+ if (sse_event_is_global(event->evt)) {
+ sse_sbi_disable_event(event);
+ } else {
+ cpu_evt.event = event;
+ cpu_evt.func = SBI_SSE_EVENT_DISABLE;
+ on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1);
+ }
+
+ mutex_unlock(&sse_mutex);
+}
+
+struct sse_event *sse_event_register(u32 evt, u32 priority,
+ sse_event_handler *handler, void *arg)
+{
+ struct sse_per_cpu_evt cpu_evt;
+ struct sse_event *event;
+ int ret = 0;
+
+ mutex_lock(&sse_mutex);
+ if (sse_event_get(evt)) {
+ pr_err("Event %x already registered\n", evt);
+ ret = -EEXIST;
+ goto out_unlock;
+ }
+
+ event = sse_event_alloc(evt, priority, handler, arg);
+ if (IS_ERR(event)) {
+ ret = PTR_ERR(event);
+ goto out_unlock;
+ }
+
+ cpus_read_lock();
+ if (sse_event_is_global(evt)) {
+ /* SSE spec mandates that the CPU registering the global event be the
+ * one set as the target hart, plus we don't know initial value
+ */
+ ret = sse_event_set_target_cpu_nolock(event, smp_processor_id());
+ if (ret)
+ goto err_event_free;
+
+ ret = sse_sbi_register_event(event, event->global);
+ if (ret)
+ goto err_event_free;
+
+ } else {
+ cpu_evt.event = event;
+ cpu_evt.error = 0;
+ cpu_evt.func = SBI_SSE_EVENT_REGISTER;
+ on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1);
+ if (READ_ONCE(cpu_evt.error)) {
+ cpu_evt.func = SBI_SSE_EVENT_UNREGISTER;
+ on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1);
+ goto err_event_free;
+ }
+ }
+ cpus_read_unlock();
+
+ spin_lock(&events_list_lock);
+ list_add(&event->list, &events);
+ spin_unlock(&events_list_lock);
+
+ mutex_unlock(&sse_mutex);
+
+ return event;
+
+err_event_free:
+ cpus_read_unlock();
+ sse_event_free(event);
+out_unlock:
+ mutex_unlock(&sse_mutex);
+
+ return ERR_PTR(ret);
+}
+
+void sse_event_unregister(struct sse_event *event)
+{
+ int ret;
+ struct sse_per_cpu_evt cpu_evt;
+
+ mutex_lock(&sse_mutex);
+
+ if (sse_event_is_global(event->evt)) {
+ /* Global events can only be unregister from target hart */
+ ret = sse_event_set_target_cpu_nolock(event, smp_processor_id());
+ WARN_ON(ret);
+ sse_sbi_unregister_event(event);
+ } else {
+ cpu_evt.event = event;
+ cpu_evt.func = SBI_SSE_EVENT_UNREGISTER;
+ on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1);
+ }
+
+ spin_lock(&events_list_lock);
+ list_del(&event->list);
+ spin_unlock(&events_list_lock);
+
+ sse_event_free(event);
+
+ mutex_unlock(&sse_mutex);
+}
+
+static int sse_cpu_online(unsigned int cpu)
+{
+ struct sse_event *sse_evt;
+
+ spin_lock(&events_list_lock);
+ list_for_each_entry(sse_evt, &events, list) {
+ if (sse_event_is_global(sse_evt->evt))
+ continue;
+
+ sse_event_register_local(sse_evt);
+ }
+
+ spin_unlock(&events_list_lock);
+
+ return 0;
+}
+
+static int sse_cpu_teardown(unsigned int cpu)
+{
+ unsigned int next_cpu;
+ struct sse_event *sse_evt;
+
+ spin_lock(&events_list_lock);
+ list_for_each_entry(sse_evt, &events, list) {
+ if (!sse_event_is_global(sse_evt->evt)) {
+ sse_sbi_unregister_event(sse_evt);
+ continue;
+ }
+
+ if (sse_evt->cpu != smp_processor_id())
+ continue;
+
+ /* Update destination hart */
+ next_cpu = cpumask_any_but(cpu_online_mask, cpu);
+ sse_event_set_target_cpu(sse_evt, next_cpu);
+ }
+ spin_unlock(&events_list_lock);
+
+ return 0;
+}
+
+static int __init sse_init(void)
+{
+ int cpu, ret;
+
+ if (sbi_probe_extension(SBI_EXT_SSE) <= 0) {
+ pr_err("Missing SBI SSE extension\n");
+ return -EOPNOTSUPP;
+ }
+ pr_info("SBI SSE extension detected\n");
+
+ for_each_possible_cpu(cpu)
+ INIT_LIST_HEAD(&events);
+
+ ret = cpuhp_setup_state_nocalls(CPUHP_AP_ONLINE_DYN, "riscv/sse:online",
+ sse_cpu_online, sse_cpu_teardown);
+ if (ret < 0)
+ return ret;
+
+ sse_available = true;
+
+ return 0;
+}
+device_initcall(sse_init);
diff --git a/include/linux/riscv_sse.h b/include/linux/riscv_sse.h
new file mode 100644
index 000000000000..3ca145368c50
--- /dev/null
+++ b/include/linux/riscv_sse.h
@@ -0,0 +1,56 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2023 Rivos Inc.
+ */
+
+#ifndef __LINUX_RISCV_SSE_H
+#define __LINUX_RISCV_SSE_H
+
+#include <linux/types.h>
+#include <linux/linkage.h>
+
+struct sse_interrupted_state;
+struct sse_event;
+struct pt_regs;
+
+typedef int (sse_event_handler)(u32 event_num, void *arg, struct pt_regs *regs);
+
+#ifdef CONFIG_RISCV_SSE
+
+struct sse_event *sse_event_register(u32 event_num, u32 priority,
+ sse_event_handler *handler, void *arg);
+
+void sse_event_unregister(struct sse_event *evt);
+
+int sse_event_set_target_cpu(struct sse_event *sse_evt, unsigned int cpu);
+
+int sse_event_enable(struct sse_event *sse_evt);
+
+void sse_event_disable(struct sse_event *sse_evt);
+
+#else
+static inline int sse_event_register(struct sse_event *evt, u32 priority,
+ sse_event_handler *handler, void *arg)
+{
+ return -EOPNOTSUPP;
+}
+
+static inline void sse_event_unregister(struct sse_event *evt) {}
+
+static inline int sse_event_set_target_cpu(struct sse_event *sse_evt,
+ unsigned int cpu)
+{
+ return -EOPNOTSUPP;
+}
+
+static inline int sse_event_enable(struct sse_event *sse_evt)
+{
+ return -EOPNOTSUPP;
+}
+
+static inline void sse_event_disable(struct sse_event *sse_evt) {}
+
+
+#endif
+
+#endif /* __LINUX_RISCV_SSE_H */
--
2.42.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [RFC PATCH 3/3] perf: RISC-V: add support for SSE event
2023-10-26 14:31 ` Clément Léger
(?)
@ 2023-10-26 14:31 ` Clément Léger
-1 siblings, 0 replies; 25+ messages in thread
From: Clément Léger @ 2023-10-26 14:31 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Albert Ou, Atish Patra, Anup Patel,
linux-riscv, linux-kernel, linux-arm-kernel
Cc: Clément Léger, Himanshu Chauhan, Xu Lu
In order to use SSE within PMU drivers, register a SSE handler for the
local PMU event. There is not a lot of specific code needed to handle
the SSE event, just reuse the existing overlflow IRQ handler and pass
appropriate pt_regs.
Signed-off-by: Clément Léger <cleger@rivosinc.com>
---
arch/riscv/include/asm/sbi.h | 2 ++
drivers/perf/riscv_pmu_sbi.c | 51 +++++++++++++++++++++++++++++-------
2 files changed, 44 insertions(+), 9 deletions(-)
diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
index 2e99cafe7fed..13b01cd3a814 100644
--- a/arch/riscv/include/asm/sbi.h
+++ b/arch/riscv/include/asm/sbi.h
@@ -121,6 +121,8 @@ enum sbi_ext_pmu_fid {
SBI_EXT_PMU_COUNTER_START,
SBI_EXT_PMU_COUNTER_STOP,
SBI_EXT_PMU_COUNTER_FW_READ,
+ SBI_EXT_PMU_COUNTER_FW_READ_HI,
+ SBI_EXT_PMU_COUNTER_IRQ_CLEAR,
};
union sbi_pmu_ctr_info {
diff --git a/drivers/perf/riscv_pmu_sbi.c b/drivers/perf/riscv_pmu_sbi.c
index 96c7f670c8f0..3fca70b13304 100644
--- a/drivers/perf/riscv_pmu_sbi.c
+++ b/drivers/perf/riscv_pmu_sbi.c
@@ -17,6 +17,7 @@
#include <linux/irqdomain.h>
#include <linux/of_irq.h>
#include <linux/of.h>
+#include <linux/riscv_sse.h>
#include <linux/cpu_pm.h>
#include <linux/sched/clock.h>
@@ -625,6 +626,12 @@ static inline void pmu_sbi_stop_hw_ctrs(struct riscv_pmu *pmu)
cpu_hw_evt->used_hw_ctrs[0], 0, 0, 0, 0);
}
+static void pmu_sbi_irq_clear(void)
+{
+ /* No need to check the error here as we can't do anything about the error */
+ sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_IRQ_CLEAR, 0, 0, 0, 0, 0, 0);
+}
+
/*
* This function starts all the used counters in two step approach.
* Any counter that did not overflow can be start in a single step
@@ -670,10 +677,10 @@ static inline void pmu_sbi_start_overflow_mask(struct riscv_pmu *pmu,
}
}
-static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
+static irqreturn_t pmu_sbi_ovf_handler(struct cpu_hw_events *cpu_hw_evt,
+ struct pt_regs *regs, bool from_sse)
{
struct perf_sample_data data;
- struct pt_regs *regs;
struct hw_perf_event *hw_evt;
union sbi_pmu_ctr_info *info;
int lidx, hidx, fidx;
@@ -681,7 +688,6 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
struct perf_event *event;
unsigned long overflow;
unsigned long overflowed_ctrs = 0;
- struct cpu_hw_events *cpu_hw_evt = dev;
u64 start_clock = sched_clock();
if (WARN_ON_ONCE(!cpu_hw_evt))
@@ -691,7 +697,10 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
fidx = find_first_bit(cpu_hw_evt->used_hw_ctrs, RISCV_MAX_COUNTERS);
event = cpu_hw_evt->events[fidx];
if (!event) {
- csr_clear(CSR_SIP, BIT(riscv_pmu_irq_num));
+ if (from_sse)
+ pmu_sbi_irq_clear();
+ else
+ csr_clear(CSR_SIP, BIT(riscv_pmu_irq_num));
return IRQ_NONE;
}
@@ -703,16 +712,16 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
/*
* Overflow interrupt pending bit should only be cleared after stopping
- * all the counters to avoid any race condition.
+ * all the counters to avoid any race condition. When using SSE,
+ * interrupt is cleared when stopping counters.
*/
- csr_clear(CSR_SIP, BIT(riscv_pmu_irq_num));
+ if (!from_sse)
+ csr_clear(CSR_SIP, BIT(riscv_pmu_irq_num));
/* No overflow bit is set */
if (!overflow)
return IRQ_NONE;
- regs = get_irq_regs();
-
for_each_set_bit(lidx, cpu_hw_evt->used_hw_ctrs, RISCV_MAX_COUNTERS) {
struct perf_event *event = cpu_hw_evt->events[lidx];
@@ -758,6 +767,22 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
return IRQ_HANDLED;
}
+static irqreturn_t pmu_sbi_ovf_irq_handler(int irq, void *dev)
+{
+ return pmu_sbi_ovf_handler(dev, get_irq_regs(), false);
+}
+
+static int pmu_sbi_ovf_sse_handler(uint32_t evt, void *arg,
+ struct pt_regs *regs)
+{
+ struct cpu_hw_events __percpu *hw_events = arg;
+ struct cpu_hw_events *hw_event = raw_cpu_ptr(hw_events);
+
+ pmu_sbi_ovf_handler(hw_event, regs, true);
+
+ return 0;
+}
+
static int pmu_sbi_starting_cpu(unsigned int cpu, struct hlist_node *node)
{
struct riscv_pmu *pmu = hlist_entry_safe(node, struct riscv_pmu, node);
@@ -801,9 +826,17 @@ static int pmu_sbi_dying_cpu(unsigned int cpu, struct hlist_node *node)
static int pmu_sbi_setup_irqs(struct riscv_pmu *pmu, struct platform_device *pdev)
{
int ret;
+ struct sse_event *evt;
struct cpu_hw_events __percpu *hw_events = pmu->hw_events;
struct irq_domain *domain = NULL;
+ evt = sse_event_register(SBI_SSE_EVENT_LOCAL_PMU, 0,
+ pmu_sbi_ovf_sse_handler, hw_events);
+ if (!IS_ERR(evt)) {
+ sse_event_enable(evt);
+ return 0;
+ }
+
if (riscv_isa_extension_available(NULL, SSCOFPMF)) {
riscv_pmu_irq_num = RV_IRQ_PMU;
riscv_pmu_use_irq = true;
@@ -831,7 +864,7 @@ static int pmu_sbi_setup_irqs(struct riscv_pmu *pmu, struct platform_device *pde
return -ENODEV;
}
- ret = request_percpu_irq(riscv_pmu_irq, pmu_sbi_ovf_handler, "riscv-pmu", hw_events);
+ ret = request_percpu_irq(riscv_pmu_irq, pmu_sbi_ovf_irq_handler, "riscv-pmu", hw_events);
if (ret) {
pr_err("registering percpu irq failed [%d]\n", ret);
return ret;
--
2.42.0
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [RFC PATCH 3/3] perf: RISC-V: add support for SSE event
@ 2023-10-26 14:31 ` Clément Léger
0 siblings, 0 replies; 25+ messages in thread
From: Clément Léger @ 2023-10-26 14:31 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Albert Ou, Atish Patra, Anup Patel,
linux-riscv, linux-kernel, linux-arm-kernel
Cc: Clément Léger, Himanshu Chauhan, Xu Lu
In order to use SSE within PMU drivers, register a SSE handler for the
local PMU event. There is not a lot of specific code needed to handle
the SSE event, just reuse the existing overlflow IRQ handler and pass
appropriate pt_regs.
Signed-off-by: Clément Léger <cleger@rivosinc.com>
---
arch/riscv/include/asm/sbi.h | 2 ++
drivers/perf/riscv_pmu_sbi.c | 51 +++++++++++++++++++++++++++++-------
2 files changed, 44 insertions(+), 9 deletions(-)
diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
index 2e99cafe7fed..13b01cd3a814 100644
--- a/arch/riscv/include/asm/sbi.h
+++ b/arch/riscv/include/asm/sbi.h
@@ -121,6 +121,8 @@ enum sbi_ext_pmu_fid {
SBI_EXT_PMU_COUNTER_START,
SBI_EXT_PMU_COUNTER_STOP,
SBI_EXT_PMU_COUNTER_FW_READ,
+ SBI_EXT_PMU_COUNTER_FW_READ_HI,
+ SBI_EXT_PMU_COUNTER_IRQ_CLEAR,
};
union sbi_pmu_ctr_info {
diff --git a/drivers/perf/riscv_pmu_sbi.c b/drivers/perf/riscv_pmu_sbi.c
index 96c7f670c8f0..3fca70b13304 100644
--- a/drivers/perf/riscv_pmu_sbi.c
+++ b/drivers/perf/riscv_pmu_sbi.c
@@ -17,6 +17,7 @@
#include <linux/irqdomain.h>
#include <linux/of_irq.h>
#include <linux/of.h>
+#include <linux/riscv_sse.h>
#include <linux/cpu_pm.h>
#include <linux/sched/clock.h>
@@ -625,6 +626,12 @@ static inline void pmu_sbi_stop_hw_ctrs(struct riscv_pmu *pmu)
cpu_hw_evt->used_hw_ctrs[0], 0, 0, 0, 0);
}
+static void pmu_sbi_irq_clear(void)
+{
+ /* No need to check the error here as we can't do anything about the error */
+ sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_IRQ_CLEAR, 0, 0, 0, 0, 0, 0);
+}
+
/*
* This function starts all the used counters in two step approach.
* Any counter that did not overflow can be start in a single step
@@ -670,10 +677,10 @@ static inline void pmu_sbi_start_overflow_mask(struct riscv_pmu *pmu,
}
}
-static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
+static irqreturn_t pmu_sbi_ovf_handler(struct cpu_hw_events *cpu_hw_evt,
+ struct pt_regs *regs, bool from_sse)
{
struct perf_sample_data data;
- struct pt_regs *regs;
struct hw_perf_event *hw_evt;
union sbi_pmu_ctr_info *info;
int lidx, hidx, fidx;
@@ -681,7 +688,6 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
struct perf_event *event;
unsigned long overflow;
unsigned long overflowed_ctrs = 0;
- struct cpu_hw_events *cpu_hw_evt = dev;
u64 start_clock = sched_clock();
if (WARN_ON_ONCE(!cpu_hw_evt))
@@ -691,7 +697,10 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
fidx = find_first_bit(cpu_hw_evt->used_hw_ctrs, RISCV_MAX_COUNTERS);
event = cpu_hw_evt->events[fidx];
if (!event) {
- csr_clear(CSR_SIP, BIT(riscv_pmu_irq_num));
+ if (from_sse)
+ pmu_sbi_irq_clear();
+ else
+ csr_clear(CSR_SIP, BIT(riscv_pmu_irq_num));
return IRQ_NONE;
}
@@ -703,16 +712,16 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
/*
* Overflow interrupt pending bit should only be cleared after stopping
- * all the counters to avoid any race condition.
+ * all the counters to avoid any race condition. When using SSE,
+ * interrupt is cleared when stopping counters.
*/
- csr_clear(CSR_SIP, BIT(riscv_pmu_irq_num));
+ if (!from_sse)
+ csr_clear(CSR_SIP, BIT(riscv_pmu_irq_num));
/* No overflow bit is set */
if (!overflow)
return IRQ_NONE;
- regs = get_irq_regs();
-
for_each_set_bit(lidx, cpu_hw_evt->used_hw_ctrs, RISCV_MAX_COUNTERS) {
struct perf_event *event = cpu_hw_evt->events[lidx];
@@ -758,6 +767,22 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
return IRQ_HANDLED;
}
+static irqreturn_t pmu_sbi_ovf_irq_handler(int irq, void *dev)
+{
+ return pmu_sbi_ovf_handler(dev, get_irq_regs(), false);
+}
+
+static int pmu_sbi_ovf_sse_handler(uint32_t evt, void *arg,
+ struct pt_regs *regs)
+{
+ struct cpu_hw_events __percpu *hw_events = arg;
+ struct cpu_hw_events *hw_event = raw_cpu_ptr(hw_events);
+
+ pmu_sbi_ovf_handler(hw_event, regs, true);
+
+ return 0;
+}
+
static int pmu_sbi_starting_cpu(unsigned int cpu, struct hlist_node *node)
{
struct riscv_pmu *pmu = hlist_entry_safe(node, struct riscv_pmu, node);
@@ -801,9 +826,17 @@ static int pmu_sbi_dying_cpu(unsigned int cpu, struct hlist_node *node)
static int pmu_sbi_setup_irqs(struct riscv_pmu *pmu, struct platform_device *pdev)
{
int ret;
+ struct sse_event *evt;
struct cpu_hw_events __percpu *hw_events = pmu->hw_events;
struct irq_domain *domain = NULL;
+ evt = sse_event_register(SBI_SSE_EVENT_LOCAL_PMU, 0,
+ pmu_sbi_ovf_sse_handler, hw_events);
+ if (!IS_ERR(evt)) {
+ sse_event_enable(evt);
+ return 0;
+ }
+
if (riscv_isa_extension_available(NULL, SSCOFPMF)) {
riscv_pmu_irq_num = RV_IRQ_PMU;
riscv_pmu_use_irq = true;
@@ -831,7 +864,7 @@ static int pmu_sbi_setup_irqs(struct riscv_pmu *pmu, struct platform_device *pde
return -ENODEV;
}
- ret = request_percpu_irq(riscv_pmu_irq, pmu_sbi_ovf_handler, "riscv-pmu", hw_events);
+ ret = request_percpu_irq(riscv_pmu_irq, pmu_sbi_ovf_irq_handler, "riscv-pmu", hw_events);
if (ret) {
pr_err("registering percpu irq failed [%d]\n", ret);
return ret;
--
2.42.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 25+ messages in thread
* [RFC PATCH 3/3] perf: RISC-V: add support for SSE event
@ 2023-10-26 14:31 ` Clément Léger
0 siblings, 0 replies; 25+ messages in thread
From: Clément Léger @ 2023-10-26 14:31 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Albert Ou, Atish Patra, Anup Patel,
linux-riscv, linux-kernel, linux-arm-kernel
Cc: Clément Léger, Himanshu Chauhan, Xu Lu
In order to use SSE within PMU drivers, register a SSE handler for the
local PMU event. There is not a lot of specific code needed to handle
the SSE event, just reuse the existing overlflow IRQ handler and pass
appropriate pt_regs.
Signed-off-by: Clément Léger <cleger@rivosinc.com>
---
arch/riscv/include/asm/sbi.h | 2 ++
drivers/perf/riscv_pmu_sbi.c | 51 +++++++++++++++++++++++++++++-------
2 files changed, 44 insertions(+), 9 deletions(-)
diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
index 2e99cafe7fed..13b01cd3a814 100644
--- a/arch/riscv/include/asm/sbi.h
+++ b/arch/riscv/include/asm/sbi.h
@@ -121,6 +121,8 @@ enum sbi_ext_pmu_fid {
SBI_EXT_PMU_COUNTER_START,
SBI_EXT_PMU_COUNTER_STOP,
SBI_EXT_PMU_COUNTER_FW_READ,
+ SBI_EXT_PMU_COUNTER_FW_READ_HI,
+ SBI_EXT_PMU_COUNTER_IRQ_CLEAR,
};
union sbi_pmu_ctr_info {
diff --git a/drivers/perf/riscv_pmu_sbi.c b/drivers/perf/riscv_pmu_sbi.c
index 96c7f670c8f0..3fca70b13304 100644
--- a/drivers/perf/riscv_pmu_sbi.c
+++ b/drivers/perf/riscv_pmu_sbi.c
@@ -17,6 +17,7 @@
#include <linux/irqdomain.h>
#include <linux/of_irq.h>
#include <linux/of.h>
+#include <linux/riscv_sse.h>
#include <linux/cpu_pm.h>
#include <linux/sched/clock.h>
@@ -625,6 +626,12 @@ static inline void pmu_sbi_stop_hw_ctrs(struct riscv_pmu *pmu)
cpu_hw_evt->used_hw_ctrs[0], 0, 0, 0, 0);
}
+static void pmu_sbi_irq_clear(void)
+{
+ /* No need to check the error here as we can't do anything about the error */
+ sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_IRQ_CLEAR, 0, 0, 0, 0, 0, 0);
+}
+
/*
* This function starts all the used counters in two step approach.
* Any counter that did not overflow can be start in a single step
@@ -670,10 +677,10 @@ static inline void pmu_sbi_start_overflow_mask(struct riscv_pmu *pmu,
}
}
-static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
+static irqreturn_t pmu_sbi_ovf_handler(struct cpu_hw_events *cpu_hw_evt,
+ struct pt_regs *regs, bool from_sse)
{
struct perf_sample_data data;
- struct pt_regs *regs;
struct hw_perf_event *hw_evt;
union sbi_pmu_ctr_info *info;
int lidx, hidx, fidx;
@@ -681,7 +688,6 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
struct perf_event *event;
unsigned long overflow;
unsigned long overflowed_ctrs = 0;
- struct cpu_hw_events *cpu_hw_evt = dev;
u64 start_clock = sched_clock();
if (WARN_ON_ONCE(!cpu_hw_evt))
@@ -691,7 +697,10 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
fidx = find_first_bit(cpu_hw_evt->used_hw_ctrs, RISCV_MAX_COUNTERS);
event = cpu_hw_evt->events[fidx];
if (!event) {
- csr_clear(CSR_SIP, BIT(riscv_pmu_irq_num));
+ if (from_sse)
+ pmu_sbi_irq_clear();
+ else
+ csr_clear(CSR_SIP, BIT(riscv_pmu_irq_num));
return IRQ_NONE;
}
@@ -703,16 +712,16 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
/*
* Overflow interrupt pending bit should only be cleared after stopping
- * all the counters to avoid any race condition.
+ * all the counters to avoid any race condition. When using SSE,
+ * interrupt is cleared when stopping counters.
*/
- csr_clear(CSR_SIP, BIT(riscv_pmu_irq_num));
+ if (!from_sse)
+ csr_clear(CSR_SIP, BIT(riscv_pmu_irq_num));
/* No overflow bit is set */
if (!overflow)
return IRQ_NONE;
- regs = get_irq_regs();
-
for_each_set_bit(lidx, cpu_hw_evt->used_hw_ctrs, RISCV_MAX_COUNTERS) {
struct perf_event *event = cpu_hw_evt->events[lidx];
@@ -758,6 +767,22 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
return IRQ_HANDLED;
}
+static irqreturn_t pmu_sbi_ovf_irq_handler(int irq, void *dev)
+{
+ return pmu_sbi_ovf_handler(dev, get_irq_regs(), false);
+}
+
+static int pmu_sbi_ovf_sse_handler(uint32_t evt, void *arg,
+ struct pt_regs *regs)
+{
+ struct cpu_hw_events __percpu *hw_events = arg;
+ struct cpu_hw_events *hw_event = raw_cpu_ptr(hw_events);
+
+ pmu_sbi_ovf_handler(hw_event, regs, true);
+
+ return 0;
+}
+
static int pmu_sbi_starting_cpu(unsigned int cpu, struct hlist_node *node)
{
struct riscv_pmu *pmu = hlist_entry_safe(node, struct riscv_pmu, node);
@@ -801,9 +826,17 @@ static int pmu_sbi_dying_cpu(unsigned int cpu, struct hlist_node *node)
static int pmu_sbi_setup_irqs(struct riscv_pmu *pmu, struct platform_device *pdev)
{
int ret;
+ struct sse_event *evt;
struct cpu_hw_events __percpu *hw_events = pmu->hw_events;
struct irq_domain *domain = NULL;
+ evt = sse_event_register(SBI_SSE_EVENT_LOCAL_PMU, 0,
+ pmu_sbi_ovf_sse_handler, hw_events);
+ if (!IS_ERR(evt)) {
+ sse_event_enable(evt);
+ return 0;
+ }
+
if (riscv_isa_extension_available(NULL, SSCOFPMF)) {
riscv_pmu_irq_num = RV_IRQ_PMU;
riscv_pmu_use_irq = true;
@@ -831,7 +864,7 @@ static int pmu_sbi_setup_irqs(struct riscv_pmu *pmu, struct platform_device *pde
return -ENODEV;
}
- ret = request_percpu_irq(riscv_pmu_irq, pmu_sbi_ovf_handler, "riscv-pmu", hw_events);
+ ret = request_percpu_irq(riscv_pmu_irq, pmu_sbi_ovf_irq_handler, "riscv-pmu", hw_events);
if (ret) {
pr_err("registering percpu irq failed [%d]\n", ret);
return ret;
--
2.42.0
^ permalink raw reply related [flat|nested] 25+ messages in thread
* Re: [RFC PATCH 3/3] perf: RISC-V: add support for SSE event
2023-10-26 14:31 ` Clément Léger
(?)
@ 2023-10-26 19:52 ` Atish Patra
-1 siblings, 0 replies; 25+ messages in thread
From: Atish Patra @ 2023-10-26 19:52 UTC (permalink / raw)
To: Clément Léger
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Anup Patel, linux-riscv,
linux-kernel, linux-arm-kernel, Himanshu Chauhan, Xu Lu
On Thu, Oct 26, 2023 at 7:31 AM Clément Léger <cleger@rivosinc.com> wrote:
>
> In order to use SSE within PMU drivers, register a SSE handler for the
> local PMU event. There is not a lot of specific code needed to handle
> the SSE event, just reuse the existing overlflow IRQ handler and pass
> appropriate pt_regs.
>
> Signed-off-by: Clément Léger <cleger@rivosinc.com>
> ---
> arch/riscv/include/asm/sbi.h | 2 ++
> drivers/perf/riscv_pmu_sbi.c | 51 +++++++++++++++++++++++++++++-------
> 2 files changed, 44 insertions(+), 9 deletions(-)
>
> diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
> index 2e99cafe7fed..13b01cd3a814 100644
> --- a/arch/riscv/include/asm/sbi.h
> +++ b/arch/riscv/include/asm/sbi.h
> @@ -121,6 +121,8 @@ enum sbi_ext_pmu_fid {
> SBI_EXT_PMU_COUNTER_START,
> SBI_EXT_PMU_COUNTER_STOP,
> SBI_EXT_PMU_COUNTER_FW_READ,
> + SBI_EXT_PMU_COUNTER_FW_READ_HI,
> + SBI_EXT_PMU_COUNTER_IRQ_CLEAR,
FWIW: This is not part of the SBI v2.0 or any proposed improvements.
As the SSE spec is in the draft state,
we need to evaluate if this is required or if there are better way to
request the M-mode to clear the interrupt pending bit.
With counter delegation extensions in place, this requires a bit more
thought as it won't use the SBI PMU at all.
In addition to that, there may be other use cases where irq needs to
be cleared first thing in the handler
There are few options which we are looking at :
1. Move the irq processing to the workqueue and call SSE complete
which can clear the irq
2. Define a generic irq clear function in the SSE extension itself to
achieve this if we have to rely on the SBI call.
> };
>
> union sbi_pmu_ctr_info {
> diff --git a/drivers/perf/riscv_pmu_sbi.c b/drivers/perf/riscv_pmu_sbi.c
> index 96c7f670c8f0..3fca70b13304 100644
> --- a/drivers/perf/riscv_pmu_sbi.c
> +++ b/drivers/perf/riscv_pmu_sbi.c
> @@ -17,6 +17,7 @@
> #include <linux/irqdomain.h>
> #include <linux/of_irq.h>
> #include <linux/of.h>
> +#include <linux/riscv_sse.h>
> #include <linux/cpu_pm.h>
> #include <linux/sched/clock.h>
>
> @@ -625,6 +626,12 @@ static inline void pmu_sbi_stop_hw_ctrs(struct riscv_pmu *pmu)
> cpu_hw_evt->used_hw_ctrs[0], 0, 0, 0, 0);
> }
>
> +static void pmu_sbi_irq_clear(void)
> +{
> + /* No need to check the error here as we can't do anything about the error */
> + sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_IRQ_CLEAR, 0, 0, 0, 0, 0, 0);
> +}
> +
> /*
> * This function starts all the used counters in two step approach.
> * Any counter that did not overflow can be start in a single step
> @@ -670,10 +677,10 @@ static inline void pmu_sbi_start_overflow_mask(struct riscv_pmu *pmu,
> }
> }
>
> -static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
> +static irqreturn_t pmu_sbi_ovf_handler(struct cpu_hw_events *cpu_hw_evt,
> + struct pt_regs *regs, bool from_sse)
> {
> struct perf_sample_data data;
> - struct pt_regs *regs;
> struct hw_perf_event *hw_evt;
> union sbi_pmu_ctr_info *info;
> int lidx, hidx, fidx;
> @@ -681,7 +688,6 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
> struct perf_event *event;
> unsigned long overflow;
> unsigned long overflowed_ctrs = 0;
> - struct cpu_hw_events *cpu_hw_evt = dev;
> u64 start_clock = sched_clock();
>
> if (WARN_ON_ONCE(!cpu_hw_evt))
> @@ -691,7 +697,10 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
> fidx = find_first_bit(cpu_hw_evt->used_hw_ctrs, RISCV_MAX_COUNTERS);
> event = cpu_hw_evt->events[fidx];
> if (!event) {
> - csr_clear(CSR_SIP, BIT(riscv_pmu_irq_num));
> + if (from_sse)
> + pmu_sbi_irq_clear();
> + else
> + csr_clear(CSR_SIP, BIT(riscv_pmu_irq_num));
> return IRQ_NONE;
> }
>
> @@ -703,16 +712,16 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
>
> /*
> * Overflow interrupt pending bit should only be cleared after stopping
> - * all the counters to avoid any race condition.
> + * all the counters to avoid any race condition. When using SSE,
> + * interrupt is cleared when stopping counters.
> */
> - csr_clear(CSR_SIP, BIT(riscv_pmu_irq_num));
> + if (!from_sse)
> + csr_clear(CSR_SIP, BIT(riscv_pmu_irq_num));
>
> /* No overflow bit is set */
> if (!overflow)
> return IRQ_NONE;
>
> - regs = get_irq_regs();
> -
> for_each_set_bit(lidx, cpu_hw_evt->used_hw_ctrs, RISCV_MAX_COUNTERS) {
> struct perf_event *event = cpu_hw_evt->events[lidx];
>
> @@ -758,6 +767,22 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
> return IRQ_HANDLED;
> }
>
> +static irqreturn_t pmu_sbi_ovf_irq_handler(int irq, void *dev)
> +{
> + return pmu_sbi_ovf_handler(dev, get_irq_regs(), false);
> +}
> +
> +static int pmu_sbi_ovf_sse_handler(uint32_t evt, void *arg,
> + struct pt_regs *regs)
> +{
> + struct cpu_hw_events __percpu *hw_events = arg;
> + struct cpu_hw_events *hw_event = raw_cpu_ptr(hw_events);
> +
> + pmu_sbi_ovf_handler(hw_event, regs, true);
> +
> + return 0;
> +}
> +
> static int pmu_sbi_starting_cpu(unsigned int cpu, struct hlist_node *node)
> {
> struct riscv_pmu *pmu = hlist_entry_safe(node, struct riscv_pmu, node);
> @@ -801,9 +826,17 @@ static int pmu_sbi_dying_cpu(unsigned int cpu, struct hlist_node *node)
> static int pmu_sbi_setup_irqs(struct riscv_pmu *pmu, struct platform_device *pdev)
> {
> int ret;
> + struct sse_event *evt;
> struct cpu_hw_events __percpu *hw_events = pmu->hw_events;
> struct irq_domain *domain = NULL;
>
> + evt = sse_event_register(SBI_SSE_EVENT_LOCAL_PMU, 0,
> + pmu_sbi_ovf_sse_handler, hw_events);
> + if (!IS_ERR(evt)) {
> + sse_event_enable(evt);
> + return 0;
> + }
> +
> if (riscv_isa_extension_available(NULL, SSCOFPMF)) {
> riscv_pmu_irq_num = RV_IRQ_PMU;
> riscv_pmu_use_irq = true;
> @@ -831,7 +864,7 @@ static int pmu_sbi_setup_irqs(struct riscv_pmu *pmu, struct platform_device *pde
> return -ENODEV;
> }
>
> - ret = request_percpu_irq(riscv_pmu_irq, pmu_sbi_ovf_handler, "riscv-pmu", hw_events);
> + ret = request_percpu_irq(riscv_pmu_irq, pmu_sbi_ovf_irq_handler, "riscv-pmu", hw_events);
> if (ret) {
> pr_err("registering percpu irq failed [%d]\n", ret);
> return ret;
> --
> 2.42.0
>
--
Regards,
Atish
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC PATCH 3/3] perf: RISC-V: add support for SSE event
@ 2023-10-26 19:52 ` Atish Patra
0 siblings, 0 replies; 25+ messages in thread
From: Atish Patra @ 2023-10-26 19:52 UTC (permalink / raw)
To: Clément Léger
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Anup Patel, linux-riscv,
linux-kernel, linux-arm-kernel, Himanshu Chauhan, Xu Lu
On Thu, Oct 26, 2023 at 7:31 AM Clément Léger <cleger@rivosinc.com> wrote:
>
> In order to use SSE within PMU drivers, register a SSE handler for the
> local PMU event. There is not a lot of specific code needed to handle
> the SSE event, just reuse the existing overlflow IRQ handler and pass
> appropriate pt_regs.
>
> Signed-off-by: Clément Léger <cleger@rivosinc.com>
> ---
> arch/riscv/include/asm/sbi.h | 2 ++
> drivers/perf/riscv_pmu_sbi.c | 51 +++++++++++++++++++++++++++++-------
> 2 files changed, 44 insertions(+), 9 deletions(-)
>
> diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
> index 2e99cafe7fed..13b01cd3a814 100644
> --- a/arch/riscv/include/asm/sbi.h
> +++ b/arch/riscv/include/asm/sbi.h
> @@ -121,6 +121,8 @@ enum sbi_ext_pmu_fid {
> SBI_EXT_PMU_COUNTER_START,
> SBI_EXT_PMU_COUNTER_STOP,
> SBI_EXT_PMU_COUNTER_FW_READ,
> + SBI_EXT_PMU_COUNTER_FW_READ_HI,
> + SBI_EXT_PMU_COUNTER_IRQ_CLEAR,
FWIW: This is not part of the SBI v2.0 or any proposed improvements.
As the SSE spec is in the draft state,
we need to evaluate if this is required or if there are better way to
request the M-mode to clear the interrupt pending bit.
With counter delegation extensions in place, this requires a bit more
thought as it won't use the SBI PMU at all.
In addition to that, there may be other use cases where irq needs to
be cleared first thing in the handler
There are few options which we are looking at :
1. Move the irq processing to the workqueue and call SSE complete
which can clear the irq
2. Define a generic irq clear function in the SSE extension itself to
achieve this if we have to rely on the SBI call.
> };
>
> union sbi_pmu_ctr_info {
> diff --git a/drivers/perf/riscv_pmu_sbi.c b/drivers/perf/riscv_pmu_sbi.c
> index 96c7f670c8f0..3fca70b13304 100644
> --- a/drivers/perf/riscv_pmu_sbi.c
> +++ b/drivers/perf/riscv_pmu_sbi.c
> @@ -17,6 +17,7 @@
> #include <linux/irqdomain.h>
> #include <linux/of_irq.h>
> #include <linux/of.h>
> +#include <linux/riscv_sse.h>
> #include <linux/cpu_pm.h>
> #include <linux/sched/clock.h>
>
> @@ -625,6 +626,12 @@ static inline void pmu_sbi_stop_hw_ctrs(struct riscv_pmu *pmu)
> cpu_hw_evt->used_hw_ctrs[0], 0, 0, 0, 0);
> }
>
> +static void pmu_sbi_irq_clear(void)
> +{
> + /* No need to check the error here as we can't do anything about the error */
> + sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_IRQ_CLEAR, 0, 0, 0, 0, 0, 0);
> +}
> +
> /*
> * This function starts all the used counters in two step approach.
> * Any counter that did not overflow can be start in a single step
> @@ -670,10 +677,10 @@ static inline void pmu_sbi_start_overflow_mask(struct riscv_pmu *pmu,
> }
> }
>
> -static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
> +static irqreturn_t pmu_sbi_ovf_handler(struct cpu_hw_events *cpu_hw_evt,
> + struct pt_regs *regs, bool from_sse)
> {
> struct perf_sample_data data;
> - struct pt_regs *regs;
> struct hw_perf_event *hw_evt;
> union sbi_pmu_ctr_info *info;
> int lidx, hidx, fidx;
> @@ -681,7 +688,6 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
> struct perf_event *event;
> unsigned long overflow;
> unsigned long overflowed_ctrs = 0;
> - struct cpu_hw_events *cpu_hw_evt = dev;
> u64 start_clock = sched_clock();
>
> if (WARN_ON_ONCE(!cpu_hw_evt))
> @@ -691,7 +697,10 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
> fidx = find_first_bit(cpu_hw_evt->used_hw_ctrs, RISCV_MAX_COUNTERS);
> event = cpu_hw_evt->events[fidx];
> if (!event) {
> - csr_clear(CSR_SIP, BIT(riscv_pmu_irq_num));
> + if (from_sse)
> + pmu_sbi_irq_clear();
> + else
> + csr_clear(CSR_SIP, BIT(riscv_pmu_irq_num));
> return IRQ_NONE;
> }
>
> @@ -703,16 +712,16 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
>
> /*
> * Overflow interrupt pending bit should only be cleared after stopping
> - * all the counters to avoid any race condition.
> + * all the counters to avoid any race condition. When using SSE,
> + * interrupt is cleared when stopping counters.
> */
> - csr_clear(CSR_SIP, BIT(riscv_pmu_irq_num));
> + if (!from_sse)
> + csr_clear(CSR_SIP, BIT(riscv_pmu_irq_num));
>
> /* No overflow bit is set */
> if (!overflow)
> return IRQ_NONE;
>
> - regs = get_irq_regs();
> -
> for_each_set_bit(lidx, cpu_hw_evt->used_hw_ctrs, RISCV_MAX_COUNTERS) {
> struct perf_event *event = cpu_hw_evt->events[lidx];
>
> @@ -758,6 +767,22 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
> return IRQ_HANDLED;
> }
>
> +static irqreturn_t pmu_sbi_ovf_irq_handler(int irq, void *dev)
> +{
> + return pmu_sbi_ovf_handler(dev, get_irq_regs(), false);
> +}
> +
> +static int pmu_sbi_ovf_sse_handler(uint32_t evt, void *arg,
> + struct pt_regs *regs)
> +{
> + struct cpu_hw_events __percpu *hw_events = arg;
> + struct cpu_hw_events *hw_event = raw_cpu_ptr(hw_events);
> +
> + pmu_sbi_ovf_handler(hw_event, regs, true);
> +
> + return 0;
> +}
> +
> static int pmu_sbi_starting_cpu(unsigned int cpu, struct hlist_node *node)
> {
> struct riscv_pmu *pmu = hlist_entry_safe(node, struct riscv_pmu, node);
> @@ -801,9 +826,17 @@ static int pmu_sbi_dying_cpu(unsigned int cpu, struct hlist_node *node)
> static int pmu_sbi_setup_irqs(struct riscv_pmu *pmu, struct platform_device *pdev)
> {
> int ret;
> + struct sse_event *evt;
> struct cpu_hw_events __percpu *hw_events = pmu->hw_events;
> struct irq_domain *domain = NULL;
>
> + evt = sse_event_register(SBI_SSE_EVENT_LOCAL_PMU, 0,
> + pmu_sbi_ovf_sse_handler, hw_events);
> + if (!IS_ERR(evt)) {
> + sse_event_enable(evt);
> + return 0;
> + }
> +
> if (riscv_isa_extension_available(NULL, SSCOFPMF)) {
> riscv_pmu_irq_num = RV_IRQ_PMU;
> riscv_pmu_use_irq = true;
> @@ -831,7 +864,7 @@ static int pmu_sbi_setup_irqs(struct riscv_pmu *pmu, struct platform_device *pde
> return -ENODEV;
> }
>
> - ret = request_percpu_irq(riscv_pmu_irq, pmu_sbi_ovf_handler, "riscv-pmu", hw_events);
> + ret = request_percpu_irq(riscv_pmu_irq, pmu_sbi_ovf_irq_handler, "riscv-pmu", hw_events);
> if (ret) {
> pr_err("registering percpu irq failed [%d]\n", ret);
> return ret;
> --
> 2.42.0
>
--
Regards,
Atish
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC PATCH 3/3] perf: RISC-V: add support for SSE event
@ 2023-10-26 19:52 ` Atish Patra
0 siblings, 0 replies; 25+ messages in thread
From: Atish Patra @ 2023-10-26 19:52 UTC (permalink / raw)
To: Clément Léger
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Anup Patel, linux-riscv,
linux-kernel, linux-arm-kernel, Himanshu Chauhan, Xu Lu
On Thu, Oct 26, 2023 at 7:31 AM Clément Léger <cleger@rivosinc.com> wrote:
>
> In order to use SSE within PMU drivers, register a SSE handler for the
> local PMU event. There is not a lot of specific code needed to handle
> the SSE event, just reuse the existing overlflow IRQ handler and pass
> appropriate pt_regs.
>
> Signed-off-by: Clément Léger <cleger@rivosinc.com>
> ---
> arch/riscv/include/asm/sbi.h | 2 ++
> drivers/perf/riscv_pmu_sbi.c | 51 +++++++++++++++++++++++++++++-------
> 2 files changed, 44 insertions(+), 9 deletions(-)
>
> diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h
> index 2e99cafe7fed..13b01cd3a814 100644
> --- a/arch/riscv/include/asm/sbi.h
> +++ b/arch/riscv/include/asm/sbi.h
> @@ -121,6 +121,8 @@ enum sbi_ext_pmu_fid {
> SBI_EXT_PMU_COUNTER_START,
> SBI_EXT_PMU_COUNTER_STOP,
> SBI_EXT_PMU_COUNTER_FW_READ,
> + SBI_EXT_PMU_COUNTER_FW_READ_HI,
> + SBI_EXT_PMU_COUNTER_IRQ_CLEAR,
FWIW: This is not part of the SBI v2.0 or any proposed improvements.
As the SSE spec is in the draft state,
we need to evaluate if this is required or if there are better way to
request the M-mode to clear the interrupt pending bit.
With counter delegation extensions in place, this requires a bit more
thought as it won't use the SBI PMU at all.
In addition to that, there may be other use cases where irq needs to
be cleared first thing in the handler
There are few options which we are looking at :
1. Move the irq processing to the workqueue and call SSE complete
which can clear the irq
2. Define a generic irq clear function in the SSE extension itself to
achieve this if we have to rely on the SBI call.
> };
>
> union sbi_pmu_ctr_info {
> diff --git a/drivers/perf/riscv_pmu_sbi.c b/drivers/perf/riscv_pmu_sbi.c
> index 96c7f670c8f0..3fca70b13304 100644
> --- a/drivers/perf/riscv_pmu_sbi.c
> +++ b/drivers/perf/riscv_pmu_sbi.c
> @@ -17,6 +17,7 @@
> #include <linux/irqdomain.h>
> #include <linux/of_irq.h>
> #include <linux/of.h>
> +#include <linux/riscv_sse.h>
> #include <linux/cpu_pm.h>
> #include <linux/sched/clock.h>
>
> @@ -625,6 +626,12 @@ static inline void pmu_sbi_stop_hw_ctrs(struct riscv_pmu *pmu)
> cpu_hw_evt->used_hw_ctrs[0], 0, 0, 0, 0);
> }
>
> +static void pmu_sbi_irq_clear(void)
> +{
> + /* No need to check the error here as we can't do anything about the error */
> + sbi_ecall(SBI_EXT_PMU, SBI_EXT_PMU_COUNTER_IRQ_CLEAR, 0, 0, 0, 0, 0, 0);
> +}
> +
> /*
> * This function starts all the used counters in two step approach.
> * Any counter that did not overflow can be start in a single step
> @@ -670,10 +677,10 @@ static inline void pmu_sbi_start_overflow_mask(struct riscv_pmu *pmu,
> }
> }
>
> -static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
> +static irqreturn_t pmu_sbi_ovf_handler(struct cpu_hw_events *cpu_hw_evt,
> + struct pt_regs *regs, bool from_sse)
> {
> struct perf_sample_data data;
> - struct pt_regs *regs;
> struct hw_perf_event *hw_evt;
> union sbi_pmu_ctr_info *info;
> int lidx, hidx, fidx;
> @@ -681,7 +688,6 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
> struct perf_event *event;
> unsigned long overflow;
> unsigned long overflowed_ctrs = 0;
> - struct cpu_hw_events *cpu_hw_evt = dev;
> u64 start_clock = sched_clock();
>
> if (WARN_ON_ONCE(!cpu_hw_evt))
> @@ -691,7 +697,10 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
> fidx = find_first_bit(cpu_hw_evt->used_hw_ctrs, RISCV_MAX_COUNTERS);
> event = cpu_hw_evt->events[fidx];
> if (!event) {
> - csr_clear(CSR_SIP, BIT(riscv_pmu_irq_num));
> + if (from_sse)
> + pmu_sbi_irq_clear();
> + else
> + csr_clear(CSR_SIP, BIT(riscv_pmu_irq_num));
> return IRQ_NONE;
> }
>
> @@ -703,16 +712,16 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
>
> /*
> * Overflow interrupt pending bit should only be cleared after stopping
> - * all the counters to avoid any race condition.
> + * all the counters to avoid any race condition. When using SSE,
> + * interrupt is cleared when stopping counters.
> */
> - csr_clear(CSR_SIP, BIT(riscv_pmu_irq_num));
> + if (!from_sse)
> + csr_clear(CSR_SIP, BIT(riscv_pmu_irq_num));
>
> /* No overflow bit is set */
> if (!overflow)
> return IRQ_NONE;
>
> - regs = get_irq_regs();
> -
> for_each_set_bit(lidx, cpu_hw_evt->used_hw_ctrs, RISCV_MAX_COUNTERS) {
> struct perf_event *event = cpu_hw_evt->events[lidx];
>
> @@ -758,6 +767,22 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev)
> return IRQ_HANDLED;
> }
>
> +static irqreturn_t pmu_sbi_ovf_irq_handler(int irq, void *dev)
> +{
> + return pmu_sbi_ovf_handler(dev, get_irq_regs(), false);
> +}
> +
> +static int pmu_sbi_ovf_sse_handler(uint32_t evt, void *arg,
> + struct pt_regs *regs)
> +{
> + struct cpu_hw_events __percpu *hw_events = arg;
> + struct cpu_hw_events *hw_event = raw_cpu_ptr(hw_events);
> +
> + pmu_sbi_ovf_handler(hw_event, regs, true);
> +
> + return 0;
> +}
> +
> static int pmu_sbi_starting_cpu(unsigned int cpu, struct hlist_node *node)
> {
> struct riscv_pmu *pmu = hlist_entry_safe(node, struct riscv_pmu, node);
> @@ -801,9 +826,17 @@ static int pmu_sbi_dying_cpu(unsigned int cpu, struct hlist_node *node)
> static int pmu_sbi_setup_irqs(struct riscv_pmu *pmu, struct platform_device *pdev)
> {
> int ret;
> + struct sse_event *evt;
> struct cpu_hw_events __percpu *hw_events = pmu->hw_events;
> struct irq_domain *domain = NULL;
>
> + evt = sse_event_register(SBI_SSE_EVENT_LOCAL_PMU, 0,
> + pmu_sbi_ovf_sse_handler, hw_events);
> + if (!IS_ERR(evt)) {
> + sse_event_enable(evt);
> + return 0;
> + }
> +
> if (riscv_isa_extension_available(NULL, SSCOFPMF)) {
> riscv_pmu_irq_num = RV_IRQ_PMU;
> riscv_pmu_use_irq = true;
> @@ -831,7 +864,7 @@ static int pmu_sbi_setup_irqs(struct riscv_pmu *pmu, struct platform_device *pde
> return -ENODEV;
> }
>
> - ret = request_percpu_irq(riscv_pmu_irq, pmu_sbi_ovf_handler, "riscv-pmu", hw_events);
> + ret = request_percpu_irq(riscv_pmu_irq, pmu_sbi_ovf_irq_handler, "riscv-pmu", hw_events);
> if (ret) {
> pr_err("registering percpu irq failed [%d]\n", ret);
> return ret;
> --
> 2.42.0
>
--
Regards,
Atish
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC PATCH 2/3] riscv: add support for SBI Supervisor Software Events extension
2023-10-26 14:31 ` Clément Léger
@ 2023-10-31 2:23 ` kernel test robot
-1 siblings, 0 replies; 25+ messages in thread
From: kernel test robot @ 2023-10-30 13:47 UTC (permalink / raw)
To: oe-kbuild; +Cc: lkp
::::::
:::::: Manual check reason: "git am base is a link in commit message"
::::::
BCC: lkp@intel.com
CC: oe-kbuild-all@lists.linux.dev
In-Reply-To: <20231026143122.279437-3-cleger@rivosinc.com>
References: <20231026143122.279437-3-cleger@rivosinc.com>
TO: "Clément Léger" <cleger@rivosinc.com>
Hi Clément,
[This is a private test report for your RFC patch.]
kernel test robot noticed the following build warnings:
[auto build test WARNING on linus/master]
[also build test WARNING on v6.6]
[cannot apply to next-20231030]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Cl-ment-L-ger/riscv-add-SBI-SSE-extension-definitions/20231026-223410
base: linus/master
patch link: https://lore.kernel.org/r/20231026143122.279437-3-cleger%40rivosinc.com
patch subject: [RFC PATCH 2/3] riscv: add support for SBI Supervisor Software Events extension
:::::: branch date: 4 days ago
:::::: commit date: 4 days ago
config: riscv-allmodconfig (https://download.01.org/0day-ci/archive/20231030/202310302119.PQRASCF7-lkp@intel.com/config)
compiler: riscv64-linux-gcc (GCC) 13.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20231030/202310302119.PQRASCF7-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/r/202310302119.PQRASCF7-lkp@intel.com/
Note: functions only called from assembly code should be annotated with the asmlinkage attribute
All warnings (new ones prefixed by >>):
>> arch/riscv/kernel/sse.c:36:15: warning: no previous prototype for 'do_sse' [-Wmissing-prototypes]
36 | unsigned long do_sse(unsigned long evt, struct sse_interrupted_state *i_state,
| ^~~~~~
vim +/do_sse +36 arch/riscv/kernel/sse.c
a77c752ef9c2a1 Clément Léger 2023-10-26 35
a77c752ef9c2a1 Clément Léger 2023-10-26 @36 unsigned long do_sse(unsigned long evt, struct sse_interrupted_state *i_state,
a77c752ef9c2a1 Clément Léger 2023-10-26 37 sse_event_handler *handler, void *arg)
a77c752ef9c2a1 Clément Léger 2023-10-26 38 {
a77c752ef9c2a1 Clément Léger 2023-10-26 39 int ret;
a77c752ef9c2a1 Clément Léger 2023-10-26 40 struct pt_regs regs;
a77c752ef9c2a1 Clément Léger 2023-10-26 41
a77c752ef9c2a1 Clément Léger 2023-10-26 42 nmi_enter();
a77c752ef9c2a1 Clément Léger 2023-10-26 43
a77c752ef9c2a1 Clément Léger 2023-10-26 44 sse_get_pt_regs(i_state, ®s);
a77c752ef9c2a1 Clément Léger 2023-10-26 45 ret = handler(evt, arg, ®s);
a77c752ef9c2a1 Clément Léger 2023-10-26 46 if (ret)
a77c752ef9c2a1 Clément Léger 2023-10-26 47 pr_warn("event %lx handler failed with error %d\n", evt, ret);
a77c752ef9c2a1 Clément Léger 2023-10-26 48
a77c752ef9c2a1 Clément Léger 2023-10-26 49 /* The SSE delivery path does not uses the "standard" exception path and
a77c752ef9c2a1 Clément Léger 2023-10-26 50 * thus does not process any pending signal/softirqs. Some drivers might
a77c752ef9c2a1 Clément Léger 2023-10-26 51 * enqueue pending work that needs to be handled as soon as possible.
a77c752ef9c2a1 Clément Léger 2023-10-26 52 * For that purpose, set the software interrupt pending bit
a77c752ef9c2a1 Clément Léger 2023-10-26 53 */
a77c752ef9c2a1 Clément Léger 2023-10-26 54 csr_set(CSR_IP, IE_SIE);
a77c752ef9c2a1 Clément Léger 2023-10-26 55
a77c752ef9c2a1 Clément Léger 2023-10-26 56 nmi_exit();
a77c752ef9c2a1 Clément Léger 2023-10-26 57
a77c752ef9c2a1 Clément Léger 2023-10-26 58 return ret ? SBI_SSE_HANDLER_FAILED : SBI_SSE_HANDLER_SUCCESS;
a77c752ef9c2a1 Clément Léger 2023-10-26 59 }
a77c752ef9c2a1 Clément Léger 2023-10-26 60
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC PATCH 2/3] riscv: add support for SBI Supervisor Software Events extension
@ 2023-10-31 2:23 ` kernel test robot
0 siblings, 0 replies; 25+ messages in thread
From: kernel test robot @ 2023-10-31 2:23 UTC (permalink / raw)
To: Clément Léger; +Cc: oe-kbuild-all
Hi Clément,
[This is a private test report for your RFC patch.]
kernel test robot noticed the following build warnings:
[auto build test WARNING on linus/master]
[also build test WARNING on v6.6]
[cannot apply to next-20231030]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Cl-ment-L-ger/riscv-add-SBI-SSE-extension-definitions/20231026-223410
base: linus/master
patch link: https://lore.kernel.org/r/20231026143122.279437-3-cleger%40rivosinc.com
patch subject: [RFC PATCH 2/3] riscv: add support for SBI Supervisor Software Events extension
config: riscv-allmodconfig (https://download.01.org/0day-ci/archive/20231030/202310302119.PQRASCF7-lkp@intel.com/config)
compiler: riscv64-linux-gcc (GCC) 13.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20231030/202310302119.PQRASCF7-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <yujie.liu@intel.com>
| Closes: https://lore.kernel.org/r/202310302119.PQRASCF7-lkp@intel.com/
Note: functions only called from assembly code should be annotated with the asmlinkage attribute
All warnings (new ones prefixed by >>):
>> arch/riscv/kernel/sse.c:36:15: warning: no previous prototype for 'do_sse' [-Wmissing-prototypes]
36 | unsigned long do_sse(unsigned long evt, struct sse_interrupted_state *i_state,
| ^~~~~~
vim +/do_sse +36 arch/riscv/kernel/sse.c
a77c752ef9c2a1 Clément Léger 2023-10-26 35
a77c752ef9c2a1 Clément Léger 2023-10-26 @36 unsigned long do_sse(unsigned long evt, struct sse_interrupted_state *i_state,
a77c752ef9c2a1 Clément Léger 2023-10-26 37 sse_event_handler *handler, void *arg)
a77c752ef9c2a1 Clément Léger 2023-10-26 38 {
a77c752ef9c2a1 Clément Léger 2023-10-26 39 int ret;
a77c752ef9c2a1 Clément Léger 2023-10-26 40 struct pt_regs regs;
a77c752ef9c2a1 Clément Léger 2023-10-26 41
a77c752ef9c2a1 Clément Léger 2023-10-26 42 nmi_enter();
a77c752ef9c2a1 Clément Léger 2023-10-26 43
a77c752ef9c2a1 Clément Léger 2023-10-26 44 sse_get_pt_regs(i_state, ®s);
a77c752ef9c2a1 Clément Léger 2023-10-26 45 ret = handler(evt, arg, ®s);
a77c752ef9c2a1 Clément Léger 2023-10-26 46 if (ret)
a77c752ef9c2a1 Clément Léger 2023-10-26 47 pr_warn("event %lx handler failed with error %d\n", evt, ret);
a77c752ef9c2a1 Clément Léger 2023-10-26 48
a77c752ef9c2a1 Clément Léger 2023-10-26 49 /* The SSE delivery path does not uses the "standard" exception path and
a77c752ef9c2a1 Clément Léger 2023-10-26 50 * thus does not process any pending signal/softirqs. Some drivers might
a77c752ef9c2a1 Clément Léger 2023-10-26 51 * enqueue pending work that needs to be handled as soon as possible.
a77c752ef9c2a1 Clément Léger 2023-10-26 52 * For that purpose, set the software interrupt pending bit
a77c752ef9c2a1 Clément Léger 2023-10-26 53 */
a77c752ef9c2a1 Clément Léger 2023-10-26 54 csr_set(CSR_IP, IE_SIE);
a77c752ef9c2a1 Clément Léger 2023-10-26 55
a77c752ef9c2a1 Clément Léger 2023-10-26 56 nmi_exit();
a77c752ef9c2a1 Clément Léger 2023-10-26 57
a77c752ef9c2a1 Clément Léger 2023-10-26 58 return ret ? SBI_SSE_HANDLER_FAILED : SBI_SSE_HANDLER_SUCCESS;
a77c752ef9c2a1 Clément Léger 2023-10-26 59 }
a77c752ef9c2a1 Clément Léger 2023-10-26 60
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC PATCH 2/3] riscv: add support for SBI Supervisor Software Events extension
2023-10-26 14:31 ` Clément Léger
@ 2023-11-02 5:25 ` kernel test robot
-1 siblings, 0 replies; 25+ messages in thread
From: kernel test robot @ 2023-10-31 4:54 UTC (permalink / raw)
To: oe-kbuild; +Cc: lkp
::::::
:::::: Manual check reason: "git am base is a link in commit message"
::::::
BCC: lkp@intel.com
CC: oe-kbuild-all@lists.linux.dev
In-Reply-To: <20231026143122.279437-3-cleger@rivosinc.com>
References: <20231026143122.279437-3-cleger@rivosinc.com>
TO: "Clément Léger" <cleger@rivosinc.com>
Hi Clément,
[This is a private test report for your RFC patch.]
kernel test robot noticed the following build errors:
[auto build test ERROR on linus/master]
[also build test ERROR on v6.6]
[cannot apply to next-20231030]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Cl-ment-L-ger/riscv-add-SBI-SSE-extension-definitions/20231026-223410
base: linus/master
patch link: https://lore.kernel.org/r/20231026143122.279437-3-cleger%40rivosinc.com
patch subject: [RFC PATCH 2/3] riscv: add support for SBI Supervisor Software Events extension
:::::: branch date: 5 days ago
:::::: commit date: 5 days ago
config: riscv-randconfig-002-20231031 (https://download.01.org/0day-ci/archive/20231031/202310311237.W6U57ENT-lkp@intel.com/config)
compiler: riscv64-linux-gcc (GCC) 13.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20231031/202310311237.W6U57ENT-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/r/202310311237.W6U57ENT-lkp@intel.com/
All errors (new ones prefixed by >>):
In file included from arch/riscv/kernel/asm-offsets.c:9:
arch/riscv/kernel/asm-offsets.c: In function 'asm_offsets':
>> arch/riscv/kernel/asm-offsets.c:495:29: error: 'SBI_EXT_SSE' undeclared (first use in this function)
495 | DEFINE(SBI_EXT_SSE, SBI_EXT_SSE);
| ^~~~~~~~~~~
include/linux/kbuild.h:6:69: note: in definition of macro 'DEFINE'
6 | asm volatile("\n.ascii \"->" #sym " %0 " #val "\"" : : "i" (val))
| ^~~
arch/riscv/kernel/asm-offsets.c:495:29: note: each undeclared identifier is reported only once for each function it appears in
495 | DEFINE(SBI_EXT_SSE, SBI_EXT_SSE);
| ^~~~~~~~~~~
include/linux/kbuild.h:6:69: note: in definition of macro 'DEFINE'
6 | asm volatile("\n.ascii \"->" #sym " %0 " #val "\"" : : "i" (val))
| ^~~
>> arch/riscv/kernel/asm-offsets.c:496:40: error: 'SBI_SSE_EVENT_COMPLETE' undeclared (first use in this function); did you mean 'SB_FREEZE_COMPLETE'?
496 | DEFINE(SBI_SSE_EVENT_COMPLETE, SBI_SSE_EVENT_COMPLETE);
| ^~~~~~~~~~~~~~~~~~~~~~
include/linux/kbuild.h:6:69: note: in definition of macro 'DEFINE'
6 | asm volatile("\n.ascii \"->" #sym " %0 " #val "\"" : : "i" (val))
| ^~~
make[3]: *** [scripts/Makefile.build:116: arch/riscv/kernel/asm-offsets.s] Error 1 shuffle=3905066713
make[3]: Target 'prepare' not remade because of errors.
make[2]: *** [Makefile:1202: prepare0] Error 2 shuffle=3905066713
make[2]: Target 'prepare' not remade because of errors.
make[1]: *** [Makefile:234: __sub-make] Error 2 shuffle=3905066713
make[1]: Target 'prepare' not remade because of errors.
make: *** [Makefile:234: __sub-make] Error 2 shuffle=3905066713
make: Target 'prepare' not remade because of errors.
vim +/SBI_EXT_SSE +495 arch/riscv/kernel/asm-offsets.c
7db91e57a0acde Palmer Dabbelt 2017-07-10 81
7db91e57a0acde Palmer Dabbelt 2017-07-10 82 DEFINE(PT_SIZE, sizeof(struct pt_regs));
a4c3733d32a72f Christoph Hellwig 2019-10-28 83 OFFSET(PT_EPC, pt_regs, epc);
7db91e57a0acde Palmer Dabbelt 2017-07-10 84 OFFSET(PT_RA, pt_regs, ra);
7db91e57a0acde Palmer Dabbelt 2017-07-10 85 OFFSET(PT_FP, pt_regs, s0);
7db91e57a0acde Palmer Dabbelt 2017-07-10 86 OFFSET(PT_S0, pt_regs, s0);
7db91e57a0acde Palmer Dabbelt 2017-07-10 87 OFFSET(PT_S1, pt_regs, s1);
7db91e57a0acde Palmer Dabbelt 2017-07-10 88 OFFSET(PT_S2, pt_regs, s2);
7db91e57a0acde Palmer Dabbelt 2017-07-10 89 OFFSET(PT_S3, pt_regs, s3);
7db91e57a0acde Palmer Dabbelt 2017-07-10 90 OFFSET(PT_S4, pt_regs, s4);
7db91e57a0acde Palmer Dabbelt 2017-07-10 91 OFFSET(PT_S5, pt_regs, s5);
7db91e57a0acde Palmer Dabbelt 2017-07-10 92 OFFSET(PT_S6, pt_regs, s6);
7db91e57a0acde Palmer Dabbelt 2017-07-10 93 OFFSET(PT_S7, pt_regs, s7);
7db91e57a0acde Palmer Dabbelt 2017-07-10 94 OFFSET(PT_S8, pt_regs, s8);
7db91e57a0acde Palmer Dabbelt 2017-07-10 95 OFFSET(PT_S9, pt_regs, s9);
7db91e57a0acde Palmer Dabbelt 2017-07-10 96 OFFSET(PT_S10, pt_regs, s10);
7db91e57a0acde Palmer Dabbelt 2017-07-10 97 OFFSET(PT_S11, pt_regs, s11);
7db91e57a0acde Palmer Dabbelt 2017-07-10 98 OFFSET(PT_SP, pt_regs, sp);
7db91e57a0acde Palmer Dabbelt 2017-07-10 99 OFFSET(PT_TP, pt_regs, tp);
7db91e57a0acde Palmer Dabbelt 2017-07-10 100 OFFSET(PT_A0, pt_regs, a0);
7db91e57a0acde Palmer Dabbelt 2017-07-10 101 OFFSET(PT_A1, pt_regs, a1);
7db91e57a0acde Palmer Dabbelt 2017-07-10 102 OFFSET(PT_A2, pt_regs, a2);
7db91e57a0acde Palmer Dabbelt 2017-07-10 103 OFFSET(PT_A3, pt_regs, a3);
7db91e57a0acde Palmer Dabbelt 2017-07-10 104 OFFSET(PT_A4, pt_regs, a4);
7db91e57a0acde Palmer Dabbelt 2017-07-10 105 OFFSET(PT_A5, pt_regs, a5);
7db91e57a0acde Palmer Dabbelt 2017-07-10 106 OFFSET(PT_A6, pt_regs, a6);
7db91e57a0acde Palmer Dabbelt 2017-07-10 107 OFFSET(PT_A7, pt_regs, a7);
7db91e57a0acde Palmer Dabbelt 2017-07-10 108 OFFSET(PT_T0, pt_regs, t0);
7db91e57a0acde Palmer Dabbelt 2017-07-10 109 OFFSET(PT_T1, pt_regs, t1);
7db91e57a0acde Palmer Dabbelt 2017-07-10 110 OFFSET(PT_T2, pt_regs, t2);
7db91e57a0acde Palmer Dabbelt 2017-07-10 111 OFFSET(PT_T3, pt_regs, t3);
7db91e57a0acde Palmer Dabbelt 2017-07-10 112 OFFSET(PT_T4, pt_regs, t4);
7db91e57a0acde Palmer Dabbelt 2017-07-10 113 OFFSET(PT_T5, pt_regs, t5);
7db91e57a0acde Palmer Dabbelt 2017-07-10 114 OFFSET(PT_T6, pt_regs, t6);
7db91e57a0acde Palmer Dabbelt 2017-07-10 115 OFFSET(PT_GP, pt_regs, gp);
7db91e57a0acde Palmer Dabbelt 2017-07-10 116 OFFSET(PT_ORIG_A0, pt_regs, orig_a0);
a4c3733d32a72f Christoph Hellwig 2019-10-28 117 OFFSET(PT_STATUS, pt_regs, status);
a4c3733d32a72f Christoph Hellwig 2019-10-28 118 OFFSET(PT_BADADDR, pt_regs, badaddr);
a4c3733d32a72f Christoph Hellwig 2019-10-28 119 OFFSET(PT_CAUSE, pt_regs, cause);
7db91e57a0acde Palmer Dabbelt 2017-07-10 120
63b13e64a829e7 Anup Patel 2022-02-10 121 OFFSET(SUSPEND_CONTEXT_REGS, suspend_context, regs);
63b13e64a829e7 Anup Patel 2022-02-10 122
c0317210012e3b Sia Jee Heng 2023-03-30 123 OFFSET(HIBERN_PBE_ADDR, pbe, address);
c0317210012e3b Sia Jee Heng 2023-03-30 124 OFFSET(HIBERN_PBE_ORIG, pbe, orig_address);
c0317210012e3b Sia Jee Heng 2023-03-30 125 OFFSET(HIBERN_PBE_NEXT, pbe, next);
c0317210012e3b Sia Jee Heng 2023-03-30 126
34bde9d8b9e6e5 Anup Patel 2021-09-27 127 OFFSET(KVM_ARCH_GUEST_ZERO, kvm_vcpu_arch, guest_context.zero);
34bde9d8b9e6e5 Anup Patel 2021-09-27 128 OFFSET(KVM_ARCH_GUEST_RA, kvm_vcpu_arch, guest_context.ra);
34bde9d8b9e6e5 Anup Patel 2021-09-27 129 OFFSET(KVM_ARCH_GUEST_SP, kvm_vcpu_arch, guest_context.sp);
34bde9d8b9e6e5 Anup Patel 2021-09-27 130 OFFSET(KVM_ARCH_GUEST_GP, kvm_vcpu_arch, guest_context.gp);
34bde9d8b9e6e5 Anup Patel 2021-09-27 131 OFFSET(KVM_ARCH_GUEST_TP, kvm_vcpu_arch, guest_context.tp);
34bde9d8b9e6e5 Anup Patel 2021-09-27 132 OFFSET(KVM_ARCH_GUEST_T0, kvm_vcpu_arch, guest_context.t0);
34bde9d8b9e6e5 Anup Patel 2021-09-27 133 OFFSET(KVM_ARCH_GUEST_T1, kvm_vcpu_arch, guest_context.t1);
34bde9d8b9e6e5 Anup Patel 2021-09-27 134 OFFSET(KVM_ARCH_GUEST_T2, kvm_vcpu_arch, guest_context.t2);
34bde9d8b9e6e5 Anup Patel 2021-09-27 135 OFFSET(KVM_ARCH_GUEST_S0, kvm_vcpu_arch, guest_context.s0);
34bde9d8b9e6e5 Anup Patel 2021-09-27 136 OFFSET(KVM_ARCH_GUEST_S1, kvm_vcpu_arch, guest_context.s1);
34bde9d8b9e6e5 Anup Patel 2021-09-27 137 OFFSET(KVM_ARCH_GUEST_A0, kvm_vcpu_arch, guest_context.a0);
34bde9d8b9e6e5 Anup Patel 2021-09-27 138 OFFSET(KVM_ARCH_GUEST_A1, kvm_vcpu_arch, guest_context.a1);
34bde9d8b9e6e5 Anup Patel 2021-09-27 139 OFFSET(KVM_ARCH_GUEST_A2, kvm_vcpu_arch, guest_context.a2);
34bde9d8b9e6e5 Anup Patel 2021-09-27 140 OFFSET(KVM_ARCH_GUEST_A3, kvm_vcpu_arch, guest_context.a3);
34bde9d8b9e6e5 Anup Patel 2021-09-27 141 OFFSET(KVM_ARCH_GUEST_A4, kvm_vcpu_arch, guest_context.a4);
34bde9d8b9e6e5 Anup Patel 2021-09-27 142 OFFSET(KVM_ARCH_GUEST_A5, kvm_vcpu_arch, guest_context.a5);
34bde9d8b9e6e5 Anup Patel 2021-09-27 143 OFFSET(KVM_ARCH_GUEST_A6, kvm_vcpu_arch, guest_context.a6);
34bde9d8b9e6e5 Anup Patel 2021-09-27 144 OFFSET(KVM_ARCH_GUEST_A7, kvm_vcpu_arch, guest_context.a7);
34bde9d8b9e6e5 Anup Patel 2021-09-27 145 OFFSET(KVM_ARCH_GUEST_S2, kvm_vcpu_arch, guest_context.s2);
34bde9d8b9e6e5 Anup Patel 2021-09-27 146 OFFSET(KVM_ARCH_GUEST_S3, kvm_vcpu_arch, guest_context.s3);
34bde9d8b9e6e5 Anup Patel 2021-09-27 147 OFFSET(KVM_ARCH_GUEST_S4, kvm_vcpu_arch, guest_context.s4);
34bde9d8b9e6e5 Anup Patel 2021-09-27 148 OFFSET(KVM_ARCH_GUEST_S5, kvm_vcpu_arch, guest_context.s5);
34bde9d8b9e6e5 Anup Patel 2021-09-27 149 OFFSET(KVM_ARCH_GUEST_S6, kvm_vcpu_arch, guest_context.s6);
34bde9d8b9e6e5 Anup Patel 2021-09-27 150 OFFSET(KVM_ARCH_GUEST_S7, kvm_vcpu_arch, guest_context.s7);
34bde9d8b9e6e5 Anup Patel 2021-09-27 151 OFFSET(KVM_ARCH_GUEST_S8, kvm_vcpu_arch, guest_context.s8);
34bde9d8b9e6e5 Anup Patel 2021-09-27 152 OFFSET(KVM_ARCH_GUEST_S9, kvm_vcpu_arch, guest_context.s9);
34bde9d8b9e6e5 Anup Patel 2021-09-27 153 OFFSET(KVM_ARCH_GUEST_S10, kvm_vcpu_arch, guest_context.s10);
34bde9d8b9e6e5 Anup Patel 2021-09-27 154 OFFSET(KVM_ARCH_GUEST_S11, kvm_vcpu_arch, guest_context.s11);
34bde9d8b9e6e5 Anup Patel 2021-09-27 155 OFFSET(KVM_ARCH_GUEST_T3, kvm_vcpu_arch, guest_context.t3);
34bde9d8b9e6e5 Anup Patel 2021-09-27 156 OFFSET(KVM_ARCH_GUEST_T4, kvm_vcpu_arch, guest_context.t4);
34bde9d8b9e6e5 Anup Patel 2021-09-27 157 OFFSET(KVM_ARCH_GUEST_T5, kvm_vcpu_arch, guest_context.t5);
34bde9d8b9e6e5 Anup Patel 2021-09-27 158 OFFSET(KVM_ARCH_GUEST_T6, kvm_vcpu_arch, guest_context.t6);
34bde9d8b9e6e5 Anup Patel 2021-09-27 159 OFFSET(KVM_ARCH_GUEST_SEPC, kvm_vcpu_arch, guest_context.sepc);
34bde9d8b9e6e5 Anup Patel 2021-09-27 160 OFFSET(KVM_ARCH_GUEST_SSTATUS, kvm_vcpu_arch, guest_context.sstatus);
34bde9d8b9e6e5 Anup Patel 2021-09-27 161 OFFSET(KVM_ARCH_GUEST_HSTATUS, kvm_vcpu_arch, guest_context.hstatus);
34bde9d8b9e6e5 Anup Patel 2021-09-27 162 OFFSET(KVM_ARCH_GUEST_SCOUNTEREN, kvm_vcpu_arch, guest_csr.scounteren);
34bde9d8b9e6e5 Anup Patel 2021-09-27 163
34bde9d8b9e6e5 Anup Patel 2021-09-27 164 OFFSET(KVM_ARCH_HOST_ZERO, kvm_vcpu_arch, host_context.zero);
34bde9d8b9e6e5 Anup Patel 2021-09-27 165 OFFSET(KVM_ARCH_HOST_RA, kvm_vcpu_arch, host_context.ra);
34bde9d8b9e6e5 Anup Patel 2021-09-27 166 OFFSET(KVM_ARCH_HOST_SP, kvm_vcpu_arch, host_context.sp);
34bde9d8b9e6e5 Anup Patel 2021-09-27 167 OFFSET(KVM_ARCH_HOST_GP, kvm_vcpu_arch, host_context.gp);
34bde9d8b9e6e5 Anup Patel 2021-09-27 168 OFFSET(KVM_ARCH_HOST_TP, kvm_vcpu_arch, host_context.tp);
34bde9d8b9e6e5 Anup Patel 2021-09-27 169 OFFSET(KVM_ARCH_HOST_T0, kvm_vcpu_arch, host_context.t0);
34bde9d8b9e6e5 Anup Patel 2021-09-27 170 OFFSET(KVM_ARCH_HOST_T1, kvm_vcpu_arch, host_context.t1);
34bde9d8b9e6e5 Anup Patel 2021-09-27 171 OFFSET(KVM_ARCH_HOST_T2, kvm_vcpu_arch, host_context.t2);
34bde9d8b9e6e5 Anup Patel 2021-09-27 172 OFFSET(KVM_ARCH_HOST_S0, kvm_vcpu_arch, host_context.s0);
34bde9d8b9e6e5 Anup Patel 2021-09-27 173 OFFSET(KVM_ARCH_HOST_S1, kvm_vcpu_arch, host_context.s1);
34bde9d8b9e6e5 Anup Patel 2021-09-27 174 OFFSET(KVM_ARCH_HOST_A0, kvm_vcpu_arch, host_context.a0);
34bde9d8b9e6e5 Anup Patel 2021-09-27 175 OFFSET(KVM_ARCH_HOST_A1, kvm_vcpu_arch, host_context.a1);
34bde9d8b9e6e5 Anup Patel 2021-09-27 176 OFFSET(KVM_ARCH_HOST_A2, kvm_vcpu_arch, host_context.a2);
34bde9d8b9e6e5 Anup Patel 2021-09-27 177 OFFSET(KVM_ARCH_HOST_A3, kvm_vcpu_arch, host_context.a3);
34bde9d8b9e6e5 Anup Patel 2021-09-27 178 OFFSET(KVM_ARCH_HOST_A4, kvm_vcpu_arch, host_context.a4);
34bde9d8b9e6e5 Anup Patel 2021-09-27 179 OFFSET(KVM_ARCH_HOST_A5, kvm_vcpu_arch, host_context.a5);
34bde9d8b9e6e5 Anup Patel 2021-09-27 180 OFFSET(KVM_ARCH_HOST_A6, kvm_vcpu_arch, host_context.a6);
34bde9d8b9e6e5 Anup Patel 2021-09-27 181 OFFSET(KVM_ARCH_HOST_A7, kvm_vcpu_arch, host_context.a7);
34bde9d8b9e6e5 Anup Patel 2021-09-27 182 OFFSET(KVM_ARCH_HOST_S2, kvm_vcpu_arch, host_context.s2);
34bde9d8b9e6e5 Anup Patel 2021-09-27 183 OFFSET(KVM_ARCH_HOST_S3, kvm_vcpu_arch, host_context.s3);
34bde9d8b9e6e5 Anup Patel 2021-09-27 184 OFFSET(KVM_ARCH_HOST_S4, kvm_vcpu_arch, host_context.s4);
34bde9d8b9e6e5 Anup Patel 2021-09-27 185 OFFSET(KVM_ARCH_HOST_S5, kvm_vcpu_arch, host_context.s5);
34bde9d8b9e6e5 Anup Patel 2021-09-27 186 OFFSET(KVM_ARCH_HOST_S6, kvm_vcpu_arch, host_context.s6);
34bde9d8b9e6e5 Anup Patel 2021-09-27 187 OFFSET(KVM_ARCH_HOST_S7, kvm_vcpu_arch, host_context.s7);
34bde9d8b9e6e5 Anup Patel 2021-09-27 188 OFFSET(KVM_ARCH_HOST_S8, kvm_vcpu_arch, host_context.s8);
34bde9d8b9e6e5 Anup Patel 2021-09-27 189 OFFSET(KVM_ARCH_HOST_S9, kvm_vcpu_arch, host_context.s9);
34bde9d8b9e6e5 Anup Patel 2021-09-27 190 OFFSET(KVM_ARCH_HOST_S10, kvm_vcpu_arch, host_context.s10);
34bde9d8b9e6e5 Anup Patel 2021-09-27 191 OFFSET(KVM_ARCH_HOST_S11, kvm_vcpu_arch, host_context.s11);
34bde9d8b9e6e5 Anup Patel 2021-09-27 192 OFFSET(KVM_ARCH_HOST_T3, kvm_vcpu_arch, host_context.t3);
34bde9d8b9e6e5 Anup Patel 2021-09-27 193 OFFSET(KVM_ARCH_HOST_T4, kvm_vcpu_arch, host_context.t4);
34bde9d8b9e6e5 Anup Patel 2021-09-27 194 OFFSET(KVM_ARCH_HOST_T5, kvm_vcpu_arch, host_context.t5);
34bde9d8b9e6e5 Anup Patel 2021-09-27 195 OFFSET(KVM_ARCH_HOST_T6, kvm_vcpu_arch, host_context.t6);
34bde9d8b9e6e5 Anup Patel 2021-09-27 196 OFFSET(KVM_ARCH_HOST_SEPC, kvm_vcpu_arch, host_context.sepc);
34bde9d8b9e6e5 Anup Patel 2021-09-27 197 OFFSET(KVM_ARCH_HOST_SSTATUS, kvm_vcpu_arch, host_context.sstatus);
34bde9d8b9e6e5 Anup Patel 2021-09-27 198 OFFSET(KVM_ARCH_HOST_HSTATUS, kvm_vcpu_arch, host_context.hstatus);
34bde9d8b9e6e5 Anup Patel 2021-09-27 199 OFFSET(KVM_ARCH_HOST_SSCRATCH, kvm_vcpu_arch, host_sscratch);
34bde9d8b9e6e5 Anup Patel 2021-09-27 200 OFFSET(KVM_ARCH_HOST_STVEC, kvm_vcpu_arch, host_stvec);
34bde9d8b9e6e5 Anup Patel 2021-09-27 201 OFFSET(KVM_ARCH_HOST_SCOUNTEREN, kvm_vcpu_arch, host_scounteren);
34bde9d8b9e6e5 Anup Patel 2021-09-27 202
9f7013265112a9 Anup Patel 2021-09-27 203 OFFSET(KVM_ARCH_TRAP_SEPC, kvm_cpu_trap, sepc);
9f7013265112a9 Anup Patel 2021-09-27 204 OFFSET(KVM_ARCH_TRAP_SCAUSE, kvm_cpu_trap, scause);
9f7013265112a9 Anup Patel 2021-09-27 205 OFFSET(KVM_ARCH_TRAP_STVAL, kvm_cpu_trap, stval);
9f7013265112a9 Anup Patel 2021-09-27 206 OFFSET(KVM_ARCH_TRAP_HTVAL, kvm_cpu_trap, htval);
9f7013265112a9 Anup Patel 2021-09-27 207 OFFSET(KVM_ARCH_TRAP_HTINST, kvm_cpu_trap, htinst);
9f7013265112a9 Anup Patel 2021-09-27 208
5de52d4a23ad33 Atish Patra 2021-09-27 209 /* F extension */
5de52d4a23ad33 Atish Patra 2021-09-27 210
5de52d4a23ad33 Atish Patra 2021-09-27 211 OFFSET(KVM_ARCH_FP_F_F0, kvm_cpu_context, fp.f.f[0]);
5de52d4a23ad33 Atish Patra 2021-09-27 212 OFFSET(KVM_ARCH_FP_F_F1, kvm_cpu_context, fp.f.f[1]);
5de52d4a23ad33 Atish Patra 2021-09-27 213 OFFSET(KVM_ARCH_FP_F_F2, kvm_cpu_context, fp.f.f[2]);
5de52d4a23ad33 Atish Patra 2021-09-27 214 OFFSET(KVM_ARCH_FP_F_F3, kvm_cpu_context, fp.f.f[3]);
5de52d4a23ad33 Atish Patra 2021-09-27 215 OFFSET(KVM_ARCH_FP_F_F4, kvm_cpu_context, fp.f.f[4]);
5de52d4a23ad33 Atish Patra 2021-09-27 216 OFFSET(KVM_ARCH_FP_F_F5, kvm_cpu_context, fp.f.f[5]);
5de52d4a23ad33 Atish Patra 2021-09-27 217 OFFSET(KVM_ARCH_FP_F_F6, kvm_cpu_context, fp.f.f[6]);
5de52d4a23ad33 Atish Patra 2021-09-27 218 OFFSET(KVM_ARCH_FP_F_F7, kvm_cpu_context, fp.f.f[7]);
5de52d4a23ad33 Atish Patra 2021-09-27 219 OFFSET(KVM_ARCH_FP_F_F8, kvm_cpu_context, fp.f.f[8]);
5de52d4a23ad33 Atish Patra 2021-09-27 220 OFFSET(KVM_ARCH_FP_F_F9, kvm_cpu_context, fp.f.f[9]);
5de52d4a23ad33 Atish Patra 2021-09-27 221 OFFSET(KVM_ARCH_FP_F_F10, kvm_cpu_context, fp.f.f[10]);
5de52d4a23ad33 Atish Patra 2021-09-27 222 OFFSET(KVM_ARCH_FP_F_F11, kvm_cpu_context, fp.f.f[11]);
5de52d4a23ad33 Atish Patra 2021-09-27 223 OFFSET(KVM_ARCH_FP_F_F12, kvm_cpu_context, fp.f.f[12]);
5de52d4a23ad33 Atish Patra 2021-09-27 224 OFFSET(KVM_ARCH_FP_F_F13, kvm_cpu_context, fp.f.f[13]);
5de52d4a23ad33 Atish Patra 2021-09-27 225 OFFSET(KVM_ARCH_FP_F_F14, kvm_cpu_context, fp.f.f[14]);
5de52d4a23ad33 Atish Patra 2021-09-27 226 OFFSET(KVM_ARCH_FP_F_F15, kvm_cpu_context, fp.f.f[15]);
5de52d4a23ad33 Atish Patra 2021-09-27 227 OFFSET(KVM_ARCH_FP_F_F16, kvm_cpu_context, fp.f.f[16]);
5de52d4a23ad33 Atish Patra 2021-09-27 228 OFFSET(KVM_ARCH_FP_F_F17, kvm_cpu_context, fp.f.f[17]);
5de52d4a23ad33 Atish Patra 2021-09-27 229 OFFSET(KVM_ARCH_FP_F_F18, kvm_cpu_context, fp.f.f[18]);
5de52d4a23ad33 Atish Patra 2021-09-27 230 OFFSET(KVM_ARCH_FP_F_F19, kvm_cpu_context, fp.f.f[19]);
5de52d4a23ad33 Atish Patra 2021-09-27 231 OFFSET(KVM_ARCH_FP_F_F20, kvm_cpu_context, fp.f.f[20]);
5de52d4a23ad33 Atish Patra 2021-09-27 232 OFFSET(KVM_ARCH_FP_F_F21, kvm_cpu_context, fp.f.f[21]);
5de52d4a23ad33 Atish Patra 2021-09-27 233 OFFSET(KVM_ARCH_FP_F_F22, kvm_cpu_context, fp.f.f[22]);
5de52d4a23ad33 Atish Patra 2021-09-27 234 OFFSET(KVM_ARCH_FP_F_F23, kvm_cpu_context, fp.f.f[23]);
5de52d4a23ad33 Atish Patra 2021-09-27 235 OFFSET(KVM_ARCH_FP_F_F24, kvm_cpu_context, fp.f.f[24]);
5de52d4a23ad33 Atish Patra 2021-09-27 236 OFFSET(KVM_ARCH_FP_F_F25, kvm_cpu_context, fp.f.f[25]);
5de52d4a23ad33 Atish Patra 2021-09-27 237 OFFSET(KVM_ARCH_FP_F_F26, kvm_cpu_context, fp.f.f[26]);
5de52d4a23ad33 Atish Patra 2021-09-27 238 OFFSET(KVM_ARCH_FP_F_F27, kvm_cpu_context, fp.f.f[27]);
5de52d4a23ad33 Atish Patra 2021-09-27 239 OFFSET(KVM_ARCH_FP_F_F28, kvm_cpu_context, fp.f.f[28]);
5de52d4a23ad33 Atish Patra 2021-09-27 240 OFFSET(KVM_ARCH_FP_F_F29, kvm_cpu_context, fp.f.f[29]);
5de52d4a23ad33 Atish Patra 2021-09-27 241 OFFSET(KVM_ARCH_FP_F_F30, kvm_cpu_context, fp.f.f[30]);
5de52d4a23ad33 Atish Patra 2021-09-27 242 OFFSET(KVM_ARCH_FP_F_F31, kvm_cpu_context, fp.f.f[31]);
5de52d4a23ad33 Atish Patra 2021-09-27 243 OFFSET(KVM_ARCH_FP_F_FCSR, kvm_cpu_context, fp.f.fcsr);
5de52d4a23ad33 Atish Patra 2021-09-27 244
5de52d4a23ad33 Atish Patra 2021-09-27 245 /* D extension */
5de52d4a23ad33 Atish Patra 2021-09-27 246
5de52d4a23ad33 Atish Patra 2021-09-27 247 OFFSET(KVM_ARCH_FP_D_F0, kvm_cpu_context, fp.d.f[0]);
5de52d4a23ad33 Atish Patra 2021-09-27 248 OFFSET(KVM_ARCH_FP_D_F1, kvm_cpu_context, fp.d.f[1]);
5de52d4a23ad33 Atish Patra 2021-09-27 249 OFFSET(KVM_ARCH_FP_D_F2, kvm_cpu_context, fp.d.f[2]);
5de52d4a23ad33 Atish Patra 2021-09-27 250 OFFSET(KVM_ARCH_FP_D_F3, kvm_cpu_context, fp.d.f[3]);
5de52d4a23ad33 Atish Patra 2021-09-27 251 OFFSET(KVM_ARCH_FP_D_F4, kvm_cpu_context, fp.d.f[4]);
5de52d4a23ad33 Atish Patra 2021-09-27 252 OFFSET(KVM_ARCH_FP_D_F5, kvm_cpu_context, fp.d.f[5]);
5de52d4a23ad33 Atish Patra 2021-09-27 253 OFFSET(KVM_ARCH_FP_D_F6, kvm_cpu_context, fp.d.f[6]);
5de52d4a23ad33 Atish Patra 2021-09-27 254 OFFSET(KVM_ARCH_FP_D_F7, kvm_cpu_context, fp.d.f[7]);
5de52d4a23ad33 Atish Patra 2021-09-27 255 OFFSET(KVM_ARCH_FP_D_F8, kvm_cpu_context, fp.d.f[8]);
5de52d4a23ad33 Atish Patra 2021-09-27 256 OFFSET(KVM_ARCH_FP_D_F9, kvm_cpu_context, fp.d.f[9]);
5de52d4a23ad33 Atish Patra 2021-09-27 257 OFFSET(KVM_ARCH_FP_D_F10, kvm_cpu_context, fp.d.f[10]);
5de52d4a23ad33 Atish Patra 2021-09-27 258 OFFSET(KVM_ARCH_FP_D_F11, kvm_cpu_context, fp.d.f[11]);
5de52d4a23ad33 Atish Patra 2021-09-27 259 OFFSET(KVM_ARCH_FP_D_F12, kvm_cpu_context, fp.d.f[12]);
5de52d4a23ad33 Atish Patra 2021-09-27 260 OFFSET(KVM_ARCH_FP_D_F13, kvm_cpu_context, fp.d.f[13]);
5de52d4a23ad33 Atish Patra 2021-09-27 261 OFFSET(KVM_ARCH_FP_D_F14, kvm_cpu_context, fp.d.f[14]);
5de52d4a23ad33 Atish Patra 2021-09-27 262 OFFSET(KVM_ARCH_FP_D_F15, kvm_cpu_context, fp.d.f[15]);
5de52d4a23ad33 Atish Patra 2021-09-27 263 OFFSET(KVM_ARCH_FP_D_F16, kvm_cpu_context, fp.d.f[16]);
5de52d4a23ad33 Atish Patra 2021-09-27 264 OFFSET(KVM_ARCH_FP_D_F17, kvm_cpu_context, fp.d.f[17]);
5de52d4a23ad33 Atish Patra 2021-09-27 265 OFFSET(KVM_ARCH_FP_D_F18, kvm_cpu_context, fp.d.f[18]);
5de52d4a23ad33 Atish Patra 2021-09-27 266 OFFSET(KVM_ARCH_FP_D_F19, kvm_cpu_context, fp.d.f[19]);
5de52d4a23ad33 Atish Patra 2021-09-27 267 OFFSET(KVM_ARCH_FP_D_F20, kvm_cpu_context, fp.d.f[20]);
5de52d4a23ad33 Atish Patra 2021-09-27 268 OFFSET(KVM_ARCH_FP_D_F21, kvm_cpu_context, fp.d.f[21]);
5de52d4a23ad33 Atish Patra 2021-09-27 269 OFFSET(KVM_ARCH_FP_D_F22, kvm_cpu_context, fp.d.f[22]);
5de52d4a23ad33 Atish Patra 2021-09-27 270 OFFSET(KVM_ARCH_FP_D_F23, kvm_cpu_context, fp.d.f[23]);
5de52d4a23ad33 Atish Patra 2021-09-27 271 OFFSET(KVM_ARCH_FP_D_F24, kvm_cpu_context, fp.d.f[24]);
5de52d4a23ad33 Atish Patra 2021-09-27 272 OFFSET(KVM_ARCH_FP_D_F25, kvm_cpu_context, fp.d.f[25]);
5de52d4a23ad33 Atish Patra 2021-09-27 273 OFFSET(KVM_ARCH_FP_D_F26, kvm_cpu_context, fp.d.f[26]);
5de52d4a23ad33 Atish Patra 2021-09-27 274 OFFSET(KVM_ARCH_FP_D_F27, kvm_cpu_context, fp.d.f[27]);
5de52d4a23ad33 Atish Patra 2021-09-27 275 OFFSET(KVM_ARCH_FP_D_F28, kvm_cpu_context, fp.d.f[28]);
5de52d4a23ad33 Atish Patra 2021-09-27 276 OFFSET(KVM_ARCH_FP_D_F29, kvm_cpu_context, fp.d.f[29]);
5de52d4a23ad33 Atish Patra 2021-09-27 277 OFFSET(KVM_ARCH_FP_D_F30, kvm_cpu_context, fp.d.f[30]);
5de52d4a23ad33 Atish Patra 2021-09-27 278 OFFSET(KVM_ARCH_FP_D_F31, kvm_cpu_context, fp.d.f[31]);
5de52d4a23ad33 Atish Patra 2021-09-27 279 OFFSET(KVM_ARCH_FP_D_FCSR, kvm_cpu_context, fp.d.fcsr);
5de52d4a23ad33 Atish Patra 2021-09-27 280
7db91e57a0acde Palmer Dabbelt 2017-07-10 281 /*
7db91e57a0acde Palmer Dabbelt 2017-07-10 282 * THREAD_{F,X}* might be larger than a S-type offset can handle, but
7db91e57a0acde Palmer Dabbelt 2017-07-10 283 * these are used in performance-sensitive assembly so we can't resort
7db91e57a0acde Palmer Dabbelt 2017-07-10 284 * to loading the long immediate every time.
7db91e57a0acde Palmer Dabbelt 2017-07-10 285 */
7db91e57a0acde Palmer Dabbelt 2017-07-10 286 DEFINE(TASK_THREAD_RA_RA,
7db91e57a0acde Palmer Dabbelt 2017-07-10 287 offsetof(struct task_struct, thread.ra)
7db91e57a0acde Palmer Dabbelt 2017-07-10 288 - offsetof(struct task_struct, thread.ra)
7db91e57a0acde Palmer Dabbelt 2017-07-10 289 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 290 DEFINE(TASK_THREAD_SP_RA,
7db91e57a0acde Palmer Dabbelt 2017-07-10 291 offsetof(struct task_struct, thread.sp)
7db91e57a0acde Palmer Dabbelt 2017-07-10 292 - offsetof(struct task_struct, thread.ra)
7db91e57a0acde Palmer Dabbelt 2017-07-10 293 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 294 DEFINE(TASK_THREAD_S0_RA,
7db91e57a0acde Palmer Dabbelt 2017-07-10 295 offsetof(struct task_struct, thread.s[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 296 - offsetof(struct task_struct, thread.ra)
7db91e57a0acde Palmer Dabbelt 2017-07-10 297 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 298 DEFINE(TASK_THREAD_S1_RA,
7db91e57a0acde Palmer Dabbelt 2017-07-10 299 offsetof(struct task_struct, thread.s[1])
7db91e57a0acde Palmer Dabbelt 2017-07-10 300 - offsetof(struct task_struct, thread.ra)
7db91e57a0acde Palmer Dabbelt 2017-07-10 301 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 302 DEFINE(TASK_THREAD_S2_RA,
7db91e57a0acde Palmer Dabbelt 2017-07-10 303 offsetof(struct task_struct, thread.s[2])
7db91e57a0acde Palmer Dabbelt 2017-07-10 304 - offsetof(struct task_struct, thread.ra)
7db91e57a0acde Palmer Dabbelt 2017-07-10 305 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 306 DEFINE(TASK_THREAD_S3_RA,
7db91e57a0acde Palmer Dabbelt 2017-07-10 307 offsetof(struct task_struct, thread.s[3])
7db91e57a0acde Palmer Dabbelt 2017-07-10 308 - offsetof(struct task_struct, thread.ra)
7db91e57a0acde Palmer Dabbelt 2017-07-10 309 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 310 DEFINE(TASK_THREAD_S4_RA,
7db91e57a0acde Palmer Dabbelt 2017-07-10 311 offsetof(struct task_struct, thread.s[4])
7db91e57a0acde Palmer Dabbelt 2017-07-10 312 - offsetof(struct task_struct, thread.ra)
7db91e57a0acde Palmer Dabbelt 2017-07-10 313 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 314 DEFINE(TASK_THREAD_S5_RA,
7db91e57a0acde Palmer Dabbelt 2017-07-10 315 offsetof(struct task_struct, thread.s[5])
7db91e57a0acde Palmer Dabbelt 2017-07-10 316 - offsetof(struct task_struct, thread.ra)
7db91e57a0acde Palmer Dabbelt 2017-07-10 317 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 318 DEFINE(TASK_THREAD_S6_RA,
7db91e57a0acde Palmer Dabbelt 2017-07-10 319 offsetof(struct task_struct, thread.s[6])
7db91e57a0acde Palmer Dabbelt 2017-07-10 320 - offsetof(struct task_struct, thread.ra)
7db91e57a0acde Palmer Dabbelt 2017-07-10 321 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 322 DEFINE(TASK_THREAD_S7_RA,
7db91e57a0acde Palmer Dabbelt 2017-07-10 323 offsetof(struct task_struct, thread.s[7])
7db91e57a0acde Palmer Dabbelt 2017-07-10 324 - offsetof(struct task_struct, thread.ra)
7db91e57a0acde Palmer Dabbelt 2017-07-10 325 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 326 DEFINE(TASK_THREAD_S8_RA,
7db91e57a0acde Palmer Dabbelt 2017-07-10 327 offsetof(struct task_struct, thread.s[8])
7db91e57a0acde Palmer Dabbelt 2017-07-10 328 - offsetof(struct task_struct, thread.ra)
7db91e57a0acde Palmer Dabbelt 2017-07-10 329 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 330 DEFINE(TASK_THREAD_S9_RA,
7db91e57a0acde Palmer Dabbelt 2017-07-10 331 offsetof(struct task_struct, thread.s[9])
7db91e57a0acde Palmer Dabbelt 2017-07-10 332 - offsetof(struct task_struct, thread.ra)
7db91e57a0acde Palmer Dabbelt 2017-07-10 333 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 334 DEFINE(TASK_THREAD_S10_RA,
7db91e57a0acde Palmer Dabbelt 2017-07-10 335 offsetof(struct task_struct, thread.s[10])
7db91e57a0acde Palmer Dabbelt 2017-07-10 336 - offsetof(struct task_struct, thread.ra)
7db91e57a0acde Palmer Dabbelt 2017-07-10 337 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 338 DEFINE(TASK_THREAD_S11_RA,
7db91e57a0acde Palmer Dabbelt 2017-07-10 339 offsetof(struct task_struct, thread.s[11])
7db91e57a0acde Palmer Dabbelt 2017-07-10 340 - offsetof(struct task_struct, thread.ra)
7db91e57a0acde Palmer Dabbelt 2017-07-10 341 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 342
7db91e57a0acde Palmer Dabbelt 2017-07-10 343 DEFINE(TASK_THREAD_F0_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 344 offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 345 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 346 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 347 DEFINE(TASK_THREAD_F1_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 348 offsetof(struct task_struct, thread.fstate.f[1])
7db91e57a0acde Palmer Dabbelt 2017-07-10 349 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 350 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 351 DEFINE(TASK_THREAD_F2_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 352 offsetof(struct task_struct, thread.fstate.f[2])
7db91e57a0acde Palmer Dabbelt 2017-07-10 353 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 354 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 355 DEFINE(TASK_THREAD_F3_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 356 offsetof(struct task_struct, thread.fstate.f[3])
7db91e57a0acde Palmer Dabbelt 2017-07-10 357 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 358 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 359 DEFINE(TASK_THREAD_F4_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 360 offsetof(struct task_struct, thread.fstate.f[4])
7db91e57a0acde Palmer Dabbelt 2017-07-10 361 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 362 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 363 DEFINE(TASK_THREAD_F5_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 364 offsetof(struct task_struct, thread.fstate.f[5])
7db91e57a0acde Palmer Dabbelt 2017-07-10 365 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 366 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 367 DEFINE(TASK_THREAD_F6_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 368 offsetof(struct task_struct, thread.fstate.f[6])
7db91e57a0acde Palmer Dabbelt 2017-07-10 369 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 370 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 371 DEFINE(TASK_THREAD_F7_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 372 offsetof(struct task_struct, thread.fstate.f[7])
7db91e57a0acde Palmer Dabbelt 2017-07-10 373 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 374 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 375 DEFINE(TASK_THREAD_F8_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 376 offsetof(struct task_struct, thread.fstate.f[8])
7db91e57a0acde Palmer Dabbelt 2017-07-10 377 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 378 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 379 DEFINE(TASK_THREAD_F9_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 380 offsetof(struct task_struct, thread.fstate.f[9])
7db91e57a0acde Palmer Dabbelt 2017-07-10 381 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 382 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 383 DEFINE(TASK_THREAD_F10_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 384 offsetof(struct task_struct, thread.fstate.f[10])
7db91e57a0acde Palmer Dabbelt 2017-07-10 385 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 386 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 387 DEFINE(TASK_THREAD_F11_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 388 offsetof(struct task_struct, thread.fstate.f[11])
7db91e57a0acde Palmer Dabbelt 2017-07-10 389 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 390 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 391 DEFINE(TASK_THREAD_F12_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 392 offsetof(struct task_struct, thread.fstate.f[12])
7db91e57a0acde Palmer Dabbelt 2017-07-10 393 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 394 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 395 DEFINE(TASK_THREAD_F13_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 396 offsetof(struct task_struct, thread.fstate.f[13])
7db91e57a0acde Palmer Dabbelt 2017-07-10 397 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 398 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 399 DEFINE(TASK_THREAD_F14_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 400 offsetof(struct task_struct, thread.fstate.f[14])
7db91e57a0acde Palmer Dabbelt 2017-07-10 401 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 402 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 403 DEFINE(TASK_THREAD_F15_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 404 offsetof(struct task_struct, thread.fstate.f[15])
7db91e57a0acde Palmer Dabbelt 2017-07-10 405 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 406 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 407 DEFINE(TASK_THREAD_F16_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 408 offsetof(struct task_struct, thread.fstate.f[16])
7db91e57a0acde Palmer Dabbelt 2017-07-10 409 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 410 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 411 DEFINE(TASK_THREAD_F17_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 412 offsetof(struct task_struct, thread.fstate.f[17])
7db91e57a0acde Palmer Dabbelt 2017-07-10 413 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 414 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 415 DEFINE(TASK_THREAD_F18_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 416 offsetof(struct task_struct, thread.fstate.f[18])
7db91e57a0acde Palmer Dabbelt 2017-07-10 417 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 418 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 419 DEFINE(TASK_THREAD_F19_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 420 offsetof(struct task_struct, thread.fstate.f[19])
7db91e57a0acde Palmer Dabbelt 2017-07-10 421 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 422 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 423 DEFINE(TASK_THREAD_F20_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 424 offsetof(struct task_struct, thread.fstate.f[20])
7db91e57a0acde Palmer Dabbelt 2017-07-10 425 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 426 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 427 DEFINE(TASK_THREAD_F21_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 428 offsetof(struct task_struct, thread.fstate.f[21])
7db91e57a0acde Palmer Dabbelt 2017-07-10 429 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 430 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 431 DEFINE(TASK_THREAD_F22_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 432 offsetof(struct task_struct, thread.fstate.f[22])
7db91e57a0acde Palmer Dabbelt 2017-07-10 433 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 434 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 435 DEFINE(TASK_THREAD_F23_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 436 offsetof(struct task_struct, thread.fstate.f[23])
7db91e57a0acde Palmer Dabbelt 2017-07-10 437 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 438 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 439 DEFINE(TASK_THREAD_F24_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 440 offsetof(struct task_struct, thread.fstate.f[24])
7db91e57a0acde Palmer Dabbelt 2017-07-10 441 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 442 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 443 DEFINE(TASK_THREAD_F25_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 444 offsetof(struct task_struct, thread.fstate.f[25])
7db91e57a0acde Palmer Dabbelt 2017-07-10 445 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 446 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 447 DEFINE(TASK_THREAD_F26_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 448 offsetof(struct task_struct, thread.fstate.f[26])
7db91e57a0acde Palmer Dabbelt 2017-07-10 449 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 450 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 451 DEFINE(TASK_THREAD_F27_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 452 offsetof(struct task_struct, thread.fstate.f[27])
7db91e57a0acde Palmer Dabbelt 2017-07-10 453 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 454 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 455 DEFINE(TASK_THREAD_F28_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 456 offsetof(struct task_struct, thread.fstate.f[28])
7db91e57a0acde Palmer Dabbelt 2017-07-10 457 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 458 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 459 DEFINE(TASK_THREAD_F29_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 460 offsetof(struct task_struct, thread.fstate.f[29])
7db91e57a0acde Palmer Dabbelt 2017-07-10 461 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 462 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 463 DEFINE(TASK_THREAD_F30_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 464 offsetof(struct task_struct, thread.fstate.f[30])
7db91e57a0acde Palmer Dabbelt 2017-07-10 465 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 466 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 467 DEFINE(TASK_THREAD_F31_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 468 offsetof(struct task_struct, thread.fstate.f[31])
7db91e57a0acde Palmer Dabbelt 2017-07-10 469 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 470 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 471 DEFINE(TASK_THREAD_FCSR_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 472 offsetof(struct task_struct, thread.fstate.fcsr)
7db91e57a0acde Palmer Dabbelt 2017-07-10 473 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 474 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 475
7db91e57a0acde Palmer Dabbelt 2017-07-10 476 /*
7db91e57a0acde Palmer Dabbelt 2017-07-10 477 * We allocate a pt_regs on the stack when entering the kernel. This
7db91e57a0acde Palmer Dabbelt 2017-07-10 478 * ensures the alignment is sane.
7db91e57a0acde Palmer Dabbelt 2017-07-10 479 */
7db91e57a0acde Palmer Dabbelt 2017-07-10 480 DEFINE(PT_SIZE_ON_STACK, ALIGN(sizeof(struct pt_regs), STACK_ALIGN));
658e2c5125bbbc Alexandre Ghiti 2021-06-17 481
658e2c5125bbbc Alexandre Ghiti 2021-06-17 482 OFFSET(KERNEL_MAP_VIRT_ADDR, kernel_mapping, virt_addr);
9a2451f1866344 Atish Patra 2022-01-20 483 OFFSET(SBI_HART_BOOT_TASK_PTR_OFFSET, sbi_hart_boot_data, task_ptr);
9a2451f1866344 Atish Patra 2022-01-20 484 OFFSET(SBI_HART_BOOT_STACK_PTR_OFFSET, sbi_hart_boot_data, stack_ptr);
a77c752ef9c2a1 Clément Léger 2023-10-26 485
a77c752ef9c2a1 Clément Léger 2023-10-26 486 OFFSET(SSE_INTERRUPTED_EXEC_MODE, sse_interrupted_state, exec_mode);
a77c752ef9c2a1 Clément Léger 2023-10-26 487 OFFSET(SSE_INTERRUPTED_S0, sse_interrupted_state, s0);
a77c752ef9c2a1 Clément Léger 2023-10-26 488 OFFSET(SSE_INTERRUPTED_TP, sse_interrupted_state, tp);
a77c752ef9c2a1 Clément Léger 2023-10-26 489 OFFSET(SSE_INTERRUPTED_PC, sse_interrupted_state, pc);
a77c752ef9c2a1 Clément Léger 2023-10-26 490
a77c752ef9c2a1 Clément Léger 2023-10-26 491 DEFINE(STACKFRAME_SIZE_ON_STACK, ALIGN(sizeof(struct stackframe),
a77c752ef9c2a1 Clément Léger 2023-10-26 492 STACK_ALIGN));
a77c752ef9c2a1 Clément Léger 2023-10-26 493 OFFSET(STACKFRAME_FP, stackframe, fp);
a77c752ef9c2a1 Clément Léger 2023-10-26 494 OFFSET(STACKFRAME_RA, stackframe, ra);
a77c752ef9c2a1 Clément Léger 2023-10-26 @495 DEFINE(SBI_EXT_SSE, SBI_EXT_SSE);
a77c752ef9c2a1 Clément Léger 2023-10-26 @496 DEFINE(SBI_SSE_EVENT_COMPLETE, SBI_SSE_EVENT_COMPLETE);
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [RFC PATCH 2/3] riscv: add support for SBI Supervisor Software Events extension
@ 2023-11-02 5:25 ` kernel test robot
0 siblings, 0 replies; 25+ messages in thread
From: kernel test robot @ 2023-11-02 5:25 UTC (permalink / raw)
To: Clément Léger; +Cc: oe-kbuild-all
Hi Clément,
[This is a private test report for your RFC patch.]
kernel test robot noticed the following build errors:
[auto build test ERROR on linus/master]
[also build test ERROR on v6.6]
[cannot apply to next-20231030]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Cl-ment-L-ger/riscv-add-SBI-SSE-extension-definitions/20231026-223410
base: linus/master
patch link: https://lore.kernel.org/r/20231026143122.279437-3-cleger%40rivosinc.com
patch subject: [RFC PATCH 2/3] riscv: add support for SBI Supervisor Software Events extension
config: riscv-randconfig-002-20231031 (https://download.01.org/0day-ci/archive/20231031/202310311237.W6U57ENT-lkp@intel.com/config)
compiler: riscv64-linux-gcc (GCC) 13.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20231031/202310311237.W6U57ENT-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <yujie.liu@intel.com>
| Closes: https://lore.kernel.org/r/202310311237.W6U57ENT-lkp@intel.com/
All errors (new ones prefixed by >>):
In file included from arch/riscv/kernel/asm-offsets.c:9:
arch/riscv/kernel/asm-offsets.c: In function 'asm_offsets':
>> arch/riscv/kernel/asm-offsets.c:495:29: error: 'SBI_EXT_SSE' undeclared (first use in this function)
495 | DEFINE(SBI_EXT_SSE, SBI_EXT_SSE);
| ^~~~~~~~~~~
include/linux/kbuild.h:6:69: note: in definition of macro 'DEFINE'
6 | asm volatile("\n.ascii \"->" #sym " %0 " #val "\"" : : "i" (val))
| ^~~
arch/riscv/kernel/asm-offsets.c:495:29: note: each undeclared identifier is reported only once for each function it appears in
495 | DEFINE(SBI_EXT_SSE, SBI_EXT_SSE);
| ^~~~~~~~~~~
include/linux/kbuild.h:6:69: note: in definition of macro 'DEFINE'
6 | asm volatile("\n.ascii \"->" #sym " %0 " #val "\"" : : "i" (val))
| ^~~
>> arch/riscv/kernel/asm-offsets.c:496:40: error: 'SBI_SSE_EVENT_COMPLETE' undeclared (first use in this function); did you mean 'SB_FREEZE_COMPLETE'?
496 | DEFINE(SBI_SSE_EVENT_COMPLETE, SBI_SSE_EVENT_COMPLETE);
| ^~~~~~~~~~~~~~~~~~~~~~
include/linux/kbuild.h:6:69: note: in definition of macro 'DEFINE'
6 | asm volatile("\n.ascii \"->" #sym " %0 " #val "\"" : : "i" (val))
| ^~~
make[3]: *** [scripts/Makefile.build:116: arch/riscv/kernel/asm-offsets.s] Error 1 shuffle=3905066713
make[3]: Target 'prepare' not remade because of errors.
make[2]: *** [Makefile:1202: prepare0] Error 2 shuffle=3905066713
make[2]: Target 'prepare' not remade because of errors.
make[1]: *** [Makefile:234: __sub-make] Error 2 shuffle=3905066713
make[1]: Target 'prepare' not remade because of errors.
make: *** [Makefile:234: __sub-make] Error 2 shuffle=3905066713
make: Target 'prepare' not remade because of errors.
vim +/SBI_EXT_SSE +495 arch/riscv/kernel/asm-offsets.c
7db91e57a0acde Palmer Dabbelt 2017-07-10 81
7db91e57a0acde Palmer Dabbelt 2017-07-10 82 DEFINE(PT_SIZE, sizeof(struct pt_regs));
a4c3733d32a72f Christoph Hellwig 2019-10-28 83 OFFSET(PT_EPC, pt_regs, epc);
7db91e57a0acde Palmer Dabbelt 2017-07-10 84 OFFSET(PT_RA, pt_regs, ra);
7db91e57a0acde Palmer Dabbelt 2017-07-10 85 OFFSET(PT_FP, pt_regs, s0);
7db91e57a0acde Palmer Dabbelt 2017-07-10 86 OFFSET(PT_S0, pt_regs, s0);
7db91e57a0acde Palmer Dabbelt 2017-07-10 87 OFFSET(PT_S1, pt_regs, s1);
7db91e57a0acde Palmer Dabbelt 2017-07-10 88 OFFSET(PT_S2, pt_regs, s2);
7db91e57a0acde Palmer Dabbelt 2017-07-10 89 OFFSET(PT_S3, pt_regs, s3);
7db91e57a0acde Palmer Dabbelt 2017-07-10 90 OFFSET(PT_S4, pt_regs, s4);
7db91e57a0acde Palmer Dabbelt 2017-07-10 91 OFFSET(PT_S5, pt_regs, s5);
7db91e57a0acde Palmer Dabbelt 2017-07-10 92 OFFSET(PT_S6, pt_regs, s6);
7db91e57a0acde Palmer Dabbelt 2017-07-10 93 OFFSET(PT_S7, pt_regs, s7);
7db91e57a0acde Palmer Dabbelt 2017-07-10 94 OFFSET(PT_S8, pt_regs, s8);
7db91e57a0acde Palmer Dabbelt 2017-07-10 95 OFFSET(PT_S9, pt_regs, s9);
7db91e57a0acde Palmer Dabbelt 2017-07-10 96 OFFSET(PT_S10, pt_regs, s10);
7db91e57a0acde Palmer Dabbelt 2017-07-10 97 OFFSET(PT_S11, pt_regs, s11);
7db91e57a0acde Palmer Dabbelt 2017-07-10 98 OFFSET(PT_SP, pt_regs, sp);
7db91e57a0acde Palmer Dabbelt 2017-07-10 99 OFFSET(PT_TP, pt_regs, tp);
7db91e57a0acde Palmer Dabbelt 2017-07-10 100 OFFSET(PT_A0, pt_regs, a0);
7db91e57a0acde Palmer Dabbelt 2017-07-10 101 OFFSET(PT_A1, pt_regs, a1);
7db91e57a0acde Palmer Dabbelt 2017-07-10 102 OFFSET(PT_A2, pt_regs, a2);
7db91e57a0acde Palmer Dabbelt 2017-07-10 103 OFFSET(PT_A3, pt_regs, a3);
7db91e57a0acde Palmer Dabbelt 2017-07-10 104 OFFSET(PT_A4, pt_regs, a4);
7db91e57a0acde Palmer Dabbelt 2017-07-10 105 OFFSET(PT_A5, pt_regs, a5);
7db91e57a0acde Palmer Dabbelt 2017-07-10 106 OFFSET(PT_A6, pt_regs, a6);
7db91e57a0acde Palmer Dabbelt 2017-07-10 107 OFFSET(PT_A7, pt_regs, a7);
7db91e57a0acde Palmer Dabbelt 2017-07-10 108 OFFSET(PT_T0, pt_regs, t0);
7db91e57a0acde Palmer Dabbelt 2017-07-10 109 OFFSET(PT_T1, pt_regs, t1);
7db91e57a0acde Palmer Dabbelt 2017-07-10 110 OFFSET(PT_T2, pt_regs, t2);
7db91e57a0acde Palmer Dabbelt 2017-07-10 111 OFFSET(PT_T3, pt_regs, t3);
7db91e57a0acde Palmer Dabbelt 2017-07-10 112 OFFSET(PT_T4, pt_regs, t4);
7db91e57a0acde Palmer Dabbelt 2017-07-10 113 OFFSET(PT_T5, pt_regs, t5);
7db91e57a0acde Palmer Dabbelt 2017-07-10 114 OFFSET(PT_T6, pt_regs, t6);
7db91e57a0acde Palmer Dabbelt 2017-07-10 115 OFFSET(PT_GP, pt_regs, gp);
7db91e57a0acde Palmer Dabbelt 2017-07-10 116 OFFSET(PT_ORIG_A0, pt_regs, orig_a0);
a4c3733d32a72f Christoph Hellwig 2019-10-28 117 OFFSET(PT_STATUS, pt_regs, status);
a4c3733d32a72f Christoph Hellwig 2019-10-28 118 OFFSET(PT_BADADDR, pt_regs, badaddr);
a4c3733d32a72f Christoph Hellwig 2019-10-28 119 OFFSET(PT_CAUSE, pt_regs, cause);
7db91e57a0acde Palmer Dabbelt 2017-07-10 120
63b13e64a829e7 Anup Patel 2022-02-10 121 OFFSET(SUSPEND_CONTEXT_REGS, suspend_context, regs);
63b13e64a829e7 Anup Patel 2022-02-10 122
c0317210012e3b Sia Jee Heng 2023-03-30 123 OFFSET(HIBERN_PBE_ADDR, pbe, address);
c0317210012e3b Sia Jee Heng 2023-03-30 124 OFFSET(HIBERN_PBE_ORIG, pbe, orig_address);
c0317210012e3b Sia Jee Heng 2023-03-30 125 OFFSET(HIBERN_PBE_NEXT, pbe, next);
c0317210012e3b Sia Jee Heng 2023-03-30 126
34bde9d8b9e6e5 Anup Patel 2021-09-27 127 OFFSET(KVM_ARCH_GUEST_ZERO, kvm_vcpu_arch, guest_context.zero);
34bde9d8b9e6e5 Anup Patel 2021-09-27 128 OFFSET(KVM_ARCH_GUEST_RA, kvm_vcpu_arch, guest_context.ra);
34bde9d8b9e6e5 Anup Patel 2021-09-27 129 OFFSET(KVM_ARCH_GUEST_SP, kvm_vcpu_arch, guest_context.sp);
34bde9d8b9e6e5 Anup Patel 2021-09-27 130 OFFSET(KVM_ARCH_GUEST_GP, kvm_vcpu_arch, guest_context.gp);
34bde9d8b9e6e5 Anup Patel 2021-09-27 131 OFFSET(KVM_ARCH_GUEST_TP, kvm_vcpu_arch, guest_context.tp);
34bde9d8b9e6e5 Anup Patel 2021-09-27 132 OFFSET(KVM_ARCH_GUEST_T0, kvm_vcpu_arch, guest_context.t0);
34bde9d8b9e6e5 Anup Patel 2021-09-27 133 OFFSET(KVM_ARCH_GUEST_T1, kvm_vcpu_arch, guest_context.t1);
34bde9d8b9e6e5 Anup Patel 2021-09-27 134 OFFSET(KVM_ARCH_GUEST_T2, kvm_vcpu_arch, guest_context.t2);
34bde9d8b9e6e5 Anup Patel 2021-09-27 135 OFFSET(KVM_ARCH_GUEST_S0, kvm_vcpu_arch, guest_context.s0);
34bde9d8b9e6e5 Anup Patel 2021-09-27 136 OFFSET(KVM_ARCH_GUEST_S1, kvm_vcpu_arch, guest_context.s1);
34bde9d8b9e6e5 Anup Patel 2021-09-27 137 OFFSET(KVM_ARCH_GUEST_A0, kvm_vcpu_arch, guest_context.a0);
34bde9d8b9e6e5 Anup Patel 2021-09-27 138 OFFSET(KVM_ARCH_GUEST_A1, kvm_vcpu_arch, guest_context.a1);
34bde9d8b9e6e5 Anup Patel 2021-09-27 139 OFFSET(KVM_ARCH_GUEST_A2, kvm_vcpu_arch, guest_context.a2);
34bde9d8b9e6e5 Anup Patel 2021-09-27 140 OFFSET(KVM_ARCH_GUEST_A3, kvm_vcpu_arch, guest_context.a3);
34bde9d8b9e6e5 Anup Patel 2021-09-27 141 OFFSET(KVM_ARCH_GUEST_A4, kvm_vcpu_arch, guest_context.a4);
34bde9d8b9e6e5 Anup Patel 2021-09-27 142 OFFSET(KVM_ARCH_GUEST_A5, kvm_vcpu_arch, guest_context.a5);
34bde9d8b9e6e5 Anup Patel 2021-09-27 143 OFFSET(KVM_ARCH_GUEST_A6, kvm_vcpu_arch, guest_context.a6);
34bde9d8b9e6e5 Anup Patel 2021-09-27 144 OFFSET(KVM_ARCH_GUEST_A7, kvm_vcpu_arch, guest_context.a7);
34bde9d8b9e6e5 Anup Patel 2021-09-27 145 OFFSET(KVM_ARCH_GUEST_S2, kvm_vcpu_arch, guest_context.s2);
34bde9d8b9e6e5 Anup Patel 2021-09-27 146 OFFSET(KVM_ARCH_GUEST_S3, kvm_vcpu_arch, guest_context.s3);
34bde9d8b9e6e5 Anup Patel 2021-09-27 147 OFFSET(KVM_ARCH_GUEST_S4, kvm_vcpu_arch, guest_context.s4);
34bde9d8b9e6e5 Anup Patel 2021-09-27 148 OFFSET(KVM_ARCH_GUEST_S5, kvm_vcpu_arch, guest_context.s5);
34bde9d8b9e6e5 Anup Patel 2021-09-27 149 OFFSET(KVM_ARCH_GUEST_S6, kvm_vcpu_arch, guest_context.s6);
34bde9d8b9e6e5 Anup Patel 2021-09-27 150 OFFSET(KVM_ARCH_GUEST_S7, kvm_vcpu_arch, guest_context.s7);
34bde9d8b9e6e5 Anup Patel 2021-09-27 151 OFFSET(KVM_ARCH_GUEST_S8, kvm_vcpu_arch, guest_context.s8);
34bde9d8b9e6e5 Anup Patel 2021-09-27 152 OFFSET(KVM_ARCH_GUEST_S9, kvm_vcpu_arch, guest_context.s9);
34bde9d8b9e6e5 Anup Patel 2021-09-27 153 OFFSET(KVM_ARCH_GUEST_S10, kvm_vcpu_arch, guest_context.s10);
34bde9d8b9e6e5 Anup Patel 2021-09-27 154 OFFSET(KVM_ARCH_GUEST_S11, kvm_vcpu_arch, guest_context.s11);
34bde9d8b9e6e5 Anup Patel 2021-09-27 155 OFFSET(KVM_ARCH_GUEST_T3, kvm_vcpu_arch, guest_context.t3);
34bde9d8b9e6e5 Anup Patel 2021-09-27 156 OFFSET(KVM_ARCH_GUEST_T4, kvm_vcpu_arch, guest_context.t4);
34bde9d8b9e6e5 Anup Patel 2021-09-27 157 OFFSET(KVM_ARCH_GUEST_T5, kvm_vcpu_arch, guest_context.t5);
34bde9d8b9e6e5 Anup Patel 2021-09-27 158 OFFSET(KVM_ARCH_GUEST_T6, kvm_vcpu_arch, guest_context.t6);
34bde9d8b9e6e5 Anup Patel 2021-09-27 159 OFFSET(KVM_ARCH_GUEST_SEPC, kvm_vcpu_arch, guest_context.sepc);
34bde9d8b9e6e5 Anup Patel 2021-09-27 160 OFFSET(KVM_ARCH_GUEST_SSTATUS, kvm_vcpu_arch, guest_context.sstatus);
34bde9d8b9e6e5 Anup Patel 2021-09-27 161 OFFSET(KVM_ARCH_GUEST_HSTATUS, kvm_vcpu_arch, guest_context.hstatus);
34bde9d8b9e6e5 Anup Patel 2021-09-27 162 OFFSET(KVM_ARCH_GUEST_SCOUNTEREN, kvm_vcpu_arch, guest_csr.scounteren);
34bde9d8b9e6e5 Anup Patel 2021-09-27 163
34bde9d8b9e6e5 Anup Patel 2021-09-27 164 OFFSET(KVM_ARCH_HOST_ZERO, kvm_vcpu_arch, host_context.zero);
34bde9d8b9e6e5 Anup Patel 2021-09-27 165 OFFSET(KVM_ARCH_HOST_RA, kvm_vcpu_arch, host_context.ra);
34bde9d8b9e6e5 Anup Patel 2021-09-27 166 OFFSET(KVM_ARCH_HOST_SP, kvm_vcpu_arch, host_context.sp);
34bde9d8b9e6e5 Anup Patel 2021-09-27 167 OFFSET(KVM_ARCH_HOST_GP, kvm_vcpu_arch, host_context.gp);
34bde9d8b9e6e5 Anup Patel 2021-09-27 168 OFFSET(KVM_ARCH_HOST_TP, kvm_vcpu_arch, host_context.tp);
34bde9d8b9e6e5 Anup Patel 2021-09-27 169 OFFSET(KVM_ARCH_HOST_T0, kvm_vcpu_arch, host_context.t0);
34bde9d8b9e6e5 Anup Patel 2021-09-27 170 OFFSET(KVM_ARCH_HOST_T1, kvm_vcpu_arch, host_context.t1);
34bde9d8b9e6e5 Anup Patel 2021-09-27 171 OFFSET(KVM_ARCH_HOST_T2, kvm_vcpu_arch, host_context.t2);
34bde9d8b9e6e5 Anup Patel 2021-09-27 172 OFFSET(KVM_ARCH_HOST_S0, kvm_vcpu_arch, host_context.s0);
34bde9d8b9e6e5 Anup Patel 2021-09-27 173 OFFSET(KVM_ARCH_HOST_S1, kvm_vcpu_arch, host_context.s1);
34bde9d8b9e6e5 Anup Patel 2021-09-27 174 OFFSET(KVM_ARCH_HOST_A0, kvm_vcpu_arch, host_context.a0);
34bde9d8b9e6e5 Anup Patel 2021-09-27 175 OFFSET(KVM_ARCH_HOST_A1, kvm_vcpu_arch, host_context.a1);
34bde9d8b9e6e5 Anup Patel 2021-09-27 176 OFFSET(KVM_ARCH_HOST_A2, kvm_vcpu_arch, host_context.a2);
34bde9d8b9e6e5 Anup Patel 2021-09-27 177 OFFSET(KVM_ARCH_HOST_A3, kvm_vcpu_arch, host_context.a3);
34bde9d8b9e6e5 Anup Patel 2021-09-27 178 OFFSET(KVM_ARCH_HOST_A4, kvm_vcpu_arch, host_context.a4);
34bde9d8b9e6e5 Anup Patel 2021-09-27 179 OFFSET(KVM_ARCH_HOST_A5, kvm_vcpu_arch, host_context.a5);
34bde9d8b9e6e5 Anup Patel 2021-09-27 180 OFFSET(KVM_ARCH_HOST_A6, kvm_vcpu_arch, host_context.a6);
34bde9d8b9e6e5 Anup Patel 2021-09-27 181 OFFSET(KVM_ARCH_HOST_A7, kvm_vcpu_arch, host_context.a7);
34bde9d8b9e6e5 Anup Patel 2021-09-27 182 OFFSET(KVM_ARCH_HOST_S2, kvm_vcpu_arch, host_context.s2);
34bde9d8b9e6e5 Anup Patel 2021-09-27 183 OFFSET(KVM_ARCH_HOST_S3, kvm_vcpu_arch, host_context.s3);
34bde9d8b9e6e5 Anup Patel 2021-09-27 184 OFFSET(KVM_ARCH_HOST_S4, kvm_vcpu_arch, host_context.s4);
34bde9d8b9e6e5 Anup Patel 2021-09-27 185 OFFSET(KVM_ARCH_HOST_S5, kvm_vcpu_arch, host_context.s5);
34bde9d8b9e6e5 Anup Patel 2021-09-27 186 OFFSET(KVM_ARCH_HOST_S6, kvm_vcpu_arch, host_context.s6);
34bde9d8b9e6e5 Anup Patel 2021-09-27 187 OFFSET(KVM_ARCH_HOST_S7, kvm_vcpu_arch, host_context.s7);
34bde9d8b9e6e5 Anup Patel 2021-09-27 188 OFFSET(KVM_ARCH_HOST_S8, kvm_vcpu_arch, host_context.s8);
34bde9d8b9e6e5 Anup Patel 2021-09-27 189 OFFSET(KVM_ARCH_HOST_S9, kvm_vcpu_arch, host_context.s9);
34bde9d8b9e6e5 Anup Patel 2021-09-27 190 OFFSET(KVM_ARCH_HOST_S10, kvm_vcpu_arch, host_context.s10);
34bde9d8b9e6e5 Anup Patel 2021-09-27 191 OFFSET(KVM_ARCH_HOST_S11, kvm_vcpu_arch, host_context.s11);
34bde9d8b9e6e5 Anup Patel 2021-09-27 192 OFFSET(KVM_ARCH_HOST_T3, kvm_vcpu_arch, host_context.t3);
34bde9d8b9e6e5 Anup Patel 2021-09-27 193 OFFSET(KVM_ARCH_HOST_T4, kvm_vcpu_arch, host_context.t4);
34bde9d8b9e6e5 Anup Patel 2021-09-27 194 OFFSET(KVM_ARCH_HOST_T5, kvm_vcpu_arch, host_context.t5);
34bde9d8b9e6e5 Anup Patel 2021-09-27 195 OFFSET(KVM_ARCH_HOST_T6, kvm_vcpu_arch, host_context.t6);
34bde9d8b9e6e5 Anup Patel 2021-09-27 196 OFFSET(KVM_ARCH_HOST_SEPC, kvm_vcpu_arch, host_context.sepc);
34bde9d8b9e6e5 Anup Patel 2021-09-27 197 OFFSET(KVM_ARCH_HOST_SSTATUS, kvm_vcpu_arch, host_context.sstatus);
34bde9d8b9e6e5 Anup Patel 2021-09-27 198 OFFSET(KVM_ARCH_HOST_HSTATUS, kvm_vcpu_arch, host_context.hstatus);
34bde9d8b9e6e5 Anup Patel 2021-09-27 199 OFFSET(KVM_ARCH_HOST_SSCRATCH, kvm_vcpu_arch, host_sscratch);
34bde9d8b9e6e5 Anup Patel 2021-09-27 200 OFFSET(KVM_ARCH_HOST_STVEC, kvm_vcpu_arch, host_stvec);
34bde9d8b9e6e5 Anup Patel 2021-09-27 201 OFFSET(KVM_ARCH_HOST_SCOUNTEREN, kvm_vcpu_arch, host_scounteren);
34bde9d8b9e6e5 Anup Patel 2021-09-27 202
9f7013265112a9 Anup Patel 2021-09-27 203 OFFSET(KVM_ARCH_TRAP_SEPC, kvm_cpu_trap, sepc);
9f7013265112a9 Anup Patel 2021-09-27 204 OFFSET(KVM_ARCH_TRAP_SCAUSE, kvm_cpu_trap, scause);
9f7013265112a9 Anup Patel 2021-09-27 205 OFFSET(KVM_ARCH_TRAP_STVAL, kvm_cpu_trap, stval);
9f7013265112a9 Anup Patel 2021-09-27 206 OFFSET(KVM_ARCH_TRAP_HTVAL, kvm_cpu_trap, htval);
9f7013265112a9 Anup Patel 2021-09-27 207 OFFSET(KVM_ARCH_TRAP_HTINST, kvm_cpu_trap, htinst);
9f7013265112a9 Anup Patel 2021-09-27 208
5de52d4a23ad33 Atish Patra 2021-09-27 209 /* F extension */
5de52d4a23ad33 Atish Patra 2021-09-27 210
5de52d4a23ad33 Atish Patra 2021-09-27 211 OFFSET(KVM_ARCH_FP_F_F0, kvm_cpu_context, fp.f.f[0]);
5de52d4a23ad33 Atish Patra 2021-09-27 212 OFFSET(KVM_ARCH_FP_F_F1, kvm_cpu_context, fp.f.f[1]);
5de52d4a23ad33 Atish Patra 2021-09-27 213 OFFSET(KVM_ARCH_FP_F_F2, kvm_cpu_context, fp.f.f[2]);
5de52d4a23ad33 Atish Patra 2021-09-27 214 OFFSET(KVM_ARCH_FP_F_F3, kvm_cpu_context, fp.f.f[3]);
5de52d4a23ad33 Atish Patra 2021-09-27 215 OFFSET(KVM_ARCH_FP_F_F4, kvm_cpu_context, fp.f.f[4]);
5de52d4a23ad33 Atish Patra 2021-09-27 216 OFFSET(KVM_ARCH_FP_F_F5, kvm_cpu_context, fp.f.f[5]);
5de52d4a23ad33 Atish Patra 2021-09-27 217 OFFSET(KVM_ARCH_FP_F_F6, kvm_cpu_context, fp.f.f[6]);
5de52d4a23ad33 Atish Patra 2021-09-27 218 OFFSET(KVM_ARCH_FP_F_F7, kvm_cpu_context, fp.f.f[7]);
5de52d4a23ad33 Atish Patra 2021-09-27 219 OFFSET(KVM_ARCH_FP_F_F8, kvm_cpu_context, fp.f.f[8]);
5de52d4a23ad33 Atish Patra 2021-09-27 220 OFFSET(KVM_ARCH_FP_F_F9, kvm_cpu_context, fp.f.f[9]);
5de52d4a23ad33 Atish Patra 2021-09-27 221 OFFSET(KVM_ARCH_FP_F_F10, kvm_cpu_context, fp.f.f[10]);
5de52d4a23ad33 Atish Patra 2021-09-27 222 OFFSET(KVM_ARCH_FP_F_F11, kvm_cpu_context, fp.f.f[11]);
5de52d4a23ad33 Atish Patra 2021-09-27 223 OFFSET(KVM_ARCH_FP_F_F12, kvm_cpu_context, fp.f.f[12]);
5de52d4a23ad33 Atish Patra 2021-09-27 224 OFFSET(KVM_ARCH_FP_F_F13, kvm_cpu_context, fp.f.f[13]);
5de52d4a23ad33 Atish Patra 2021-09-27 225 OFFSET(KVM_ARCH_FP_F_F14, kvm_cpu_context, fp.f.f[14]);
5de52d4a23ad33 Atish Patra 2021-09-27 226 OFFSET(KVM_ARCH_FP_F_F15, kvm_cpu_context, fp.f.f[15]);
5de52d4a23ad33 Atish Patra 2021-09-27 227 OFFSET(KVM_ARCH_FP_F_F16, kvm_cpu_context, fp.f.f[16]);
5de52d4a23ad33 Atish Patra 2021-09-27 228 OFFSET(KVM_ARCH_FP_F_F17, kvm_cpu_context, fp.f.f[17]);
5de52d4a23ad33 Atish Patra 2021-09-27 229 OFFSET(KVM_ARCH_FP_F_F18, kvm_cpu_context, fp.f.f[18]);
5de52d4a23ad33 Atish Patra 2021-09-27 230 OFFSET(KVM_ARCH_FP_F_F19, kvm_cpu_context, fp.f.f[19]);
5de52d4a23ad33 Atish Patra 2021-09-27 231 OFFSET(KVM_ARCH_FP_F_F20, kvm_cpu_context, fp.f.f[20]);
5de52d4a23ad33 Atish Patra 2021-09-27 232 OFFSET(KVM_ARCH_FP_F_F21, kvm_cpu_context, fp.f.f[21]);
5de52d4a23ad33 Atish Patra 2021-09-27 233 OFFSET(KVM_ARCH_FP_F_F22, kvm_cpu_context, fp.f.f[22]);
5de52d4a23ad33 Atish Patra 2021-09-27 234 OFFSET(KVM_ARCH_FP_F_F23, kvm_cpu_context, fp.f.f[23]);
5de52d4a23ad33 Atish Patra 2021-09-27 235 OFFSET(KVM_ARCH_FP_F_F24, kvm_cpu_context, fp.f.f[24]);
5de52d4a23ad33 Atish Patra 2021-09-27 236 OFFSET(KVM_ARCH_FP_F_F25, kvm_cpu_context, fp.f.f[25]);
5de52d4a23ad33 Atish Patra 2021-09-27 237 OFFSET(KVM_ARCH_FP_F_F26, kvm_cpu_context, fp.f.f[26]);
5de52d4a23ad33 Atish Patra 2021-09-27 238 OFFSET(KVM_ARCH_FP_F_F27, kvm_cpu_context, fp.f.f[27]);
5de52d4a23ad33 Atish Patra 2021-09-27 239 OFFSET(KVM_ARCH_FP_F_F28, kvm_cpu_context, fp.f.f[28]);
5de52d4a23ad33 Atish Patra 2021-09-27 240 OFFSET(KVM_ARCH_FP_F_F29, kvm_cpu_context, fp.f.f[29]);
5de52d4a23ad33 Atish Patra 2021-09-27 241 OFFSET(KVM_ARCH_FP_F_F30, kvm_cpu_context, fp.f.f[30]);
5de52d4a23ad33 Atish Patra 2021-09-27 242 OFFSET(KVM_ARCH_FP_F_F31, kvm_cpu_context, fp.f.f[31]);
5de52d4a23ad33 Atish Patra 2021-09-27 243 OFFSET(KVM_ARCH_FP_F_FCSR, kvm_cpu_context, fp.f.fcsr);
5de52d4a23ad33 Atish Patra 2021-09-27 244
5de52d4a23ad33 Atish Patra 2021-09-27 245 /* D extension */
5de52d4a23ad33 Atish Patra 2021-09-27 246
5de52d4a23ad33 Atish Patra 2021-09-27 247 OFFSET(KVM_ARCH_FP_D_F0, kvm_cpu_context, fp.d.f[0]);
5de52d4a23ad33 Atish Patra 2021-09-27 248 OFFSET(KVM_ARCH_FP_D_F1, kvm_cpu_context, fp.d.f[1]);
5de52d4a23ad33 Atish Patra 2021-09-27 249 OFFSET(KVM_ARCH_FP_D_F2, kvm_cpu_context, fp.d.f[2]);
5de52d4a23ad33 Atish Patra 2021-09-27 250 OFFSET(KVM_ARCH_FP_D_F3, kvm_cpu_context, fp.d.f[3]);
5de52d4a23ad33 Atish Patra 2021-09-27 251 OFFSET(KVM_ARCH_FP_D_F4, kvm_cpu_context, fp.d.f[4]);
5de52d4a23ad33 Atish Patra 2021-09-27 252 OFFSET(KVM_ARCH_FP_D_F5, kvm_cpu_context, fp.d.f[5]);
5de52d4a23ad33 Atish Patra 2021-09-27 253 OFFSET(KVM_ARCH_FP_D_F6, kvm_cpu_context, fp.d.f[6]);
5de52d4a23ad33 Atish Patra 2021-09-27 254 OFFSET(KVM_ARCH_FP_D_F7, kvm_cpu_context, fp.d.f[7]);
5de52d4a23ad33 Atish Patra 2021-09-27 255 OFFSET(KVM_ARCH_FP_D_F8, kvm_cpu_context, fp.d.f[8]);
5de52d4a23ad33 Atish Patra 2021-09-27 256 OFFSET(KVM_ARCH_FP_D_F9, kvm_cpu_context, fp.d.f[9]);
5de52d4a23ad33 Atish Patra 2021-09-27 257 OFFSET(KVM_ARCH_FP_D_F10, kvm_cpu_context, fp.d.f[10]);
5de52d4a23ad33 Atish Patra 2021-09-27 258 OFFSET(KVM_ARCH_FP_D_F11, kvm_cpu_context, fp.d.f[11]);
5de52d4a23ad33 Atish Patra 2021-09-27 259 OFFSET(KVM_ARCH_FP_D_F12, kvm_cpu_context, fp.d.f[12]);
5de52d4a23ad33 Atish Patra 2021-09-27 260 OFFSET(KVM_ARCH_FP_D_F13, kvm_cpu_context, fp.d.f[13]);
5de52d4a23ad33 Atish Patra 2021-09-27 261 OFFSET(KVM_ARCH_FP_D_F14, kvm_cpu_context, fp.d.f[14]);
5de52d4a23ad33 Atish Patra 2021-09-27 262 OFFSET(KVM_ARCH_FP_D_F15, kvm_cpu_context, fp.d.f[15]);
5de52d4a23ad33 Atish Patra 2021-09-27 263 OFFSET(KVM_ARCH_FP_D_F16, kvm_cpu_context, fp.d.f[16]);
5de52d4a23ad33 Atish Patra 2021-09-27 264 OFFSET(KVM_ARCH_FP_D_F17, kvm_cpu_context, fp.d.f[17]);
5de52d4a23ad33 Atish Patra 2021-09-27 265 OFFSET(KVM_ARCH_FP_D_F18, kvm_cpu_context, fp.d.f[18]);
5de52d4a23ad33 Atish Patra 2021-09-27 266 OFFSET(KVM_ARCH_FP_D_F19, kvm_cpu_context, fp.d.f[19]);
5de52d4a23ad33 Atish Patra 2021-09-27 267 OFFSET(KVM_ARCH_FP_D_F20, kvm_cpu_context, fp.d.f[20]);
5de52d4a23ad33 Atish Patra 2021-09-27 268 OFFSET(KVM_ARCH_FP_D_F21, kvm_cpu_context, fp.d.f[21]);
5de52d4a23ad33 Atish Patra 2021-09-27 269 OFFSET(KVM_ARCH_FP_D_F22, kvm_cpu_context, fp.d.f[22]);
5de52d4a23ad33 Atish Patra 2021-09-27 270 OFFSET(KVM_ARCH_FP_D_F23, kvm_cpu_context, fp.d.f[23]);
5de52d4a23ad33 Atish Patra 2021-09-27 271 OFFSET(KVM_ARCH_FP_D_F24, kvm_cpu_context, fp.d.f[24]);
5de52d4a23ad33 Atish Patra 2021-09-27 272 OFFSET(KVM_ARCH_FP_D_F25, kvm_cpu_context, fp.d.f[25]);
5de52d4a23ad33 Atish Patra 2021-09-27 273 OFFSET(KVM_ARCH_FP_D_F26, kvm_cpu_context, fp.d.f[26]);
5de52d4a23ad33 Atish Patra 2021-09-27 274 OFFSET(KVM_ARCH_FP_D_F27, kvm_cpu_context, fp.d.f[27]);
5de52d4a23ad33 Atish Patra 2021-09-27 275 OFFSET(KVM_ARCH_FP_D_F28, kvm_cpu_context, fp.d.f[28]);
5de52d4a23ad33 Atish Patra 2021-09-27 276 OFFSET(KVM_ARCH_FP_D_F29, kvm_cpu_context, fp.d.f[29]);
5de52d4a23ad33 Atish Patra 2021-09-27 277 OFFSET(KVM_ARCH_FP_D_F30, kvm_cpu_context, fp.d.f[30]);
5de52d4a23ad33 Atish Patra 2021-09-27 278 OFFSET(KVM_ARCH_FP_D_F31, kvm_cpu_context, fp.d.f[31]);
5de52d4a23ad33 Atish Patra 2021-09-27 279 OFFSET(KVM_ARCH_FP_D_FCSR, kvm_cpu_context, fp.d.fcsr);
5de52d4a23ad33 Atish Patra 2021-09-27 280
7db91e57a0acde Palmer Dabbelt 2017-07-10 281 /*
7db91e57a0acde Palmer Dabbelt 2017-07-10 282 * THREAD_{F,X}* might be larger than a S-type offset can handle, but
7db91e57a0acde Palmer Dabbelt 2017-07-10 283 * these are used in performance-sensitive assembly so we can't resort
7db91e57a0acde Palmer Dabbelt 2017-07-10 284 * to loading the long immediate every time.
7db91e57a0acde Palmer Dabbelt 2017-07-10 285 */
7db91e57a0acde Palmer Dabbelt 2017-07-10 286 DEFINE(TASK_THREAD_RA_RA,
7db91e57a0acde Palmer Dabbelt 2017-07-10 287 offsetof(struct task_struct, thread.ra)
7db91e57a0acde Palmer Dabbelt 2017-07-10 288 - offsetof(struct task_struct, thread.ra)
7db91e57a0acde Palmer Dabbelt 2017-07-10 289 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 290 DEFINE(TASK_THREAD_SP_RA,
7db91e57a0acde Palmer Dabbelt 2017-07-10 291 offsetof(struct task_struct, thread.sp)
7db91e57a0acde Palmer Dabbelt 2017-07-10 292 - offsetof(struct task_struct, thread.ra)
7db91e57a0acde Palmer Dabbelt 2017-07-10 293 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 294 DEFINE(TASK_THREAD_S0_RA,
7db91e57a0acde Palmer Dabbelt 2017-07-10 295 offsetof(struct task_struct, thread.s[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 296 - offsetof(struct task_struct, thread.ra)
7db91e57a0acde Palmer Dabbelt 2017-07-10 297 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 298 DEFINE(TASK_THREAD_S1_RA,
7db91e57a0acde Palmer Dabbelt 2017-07-10 299 offsetof(struct task_struct, thread.s[1])
7db91e57a0acde Palmer Dabbelt 2017-07-10 300 - offsetof(struct task_struct, thread.ra)
7db91e57a0acde Palmer Dabbelt 2017-07-10 301 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 302 DEFINE(TASK_THREAD_S2_RA,
7db91e57a0acde Palmer Dabbelt 2017-07-10 303 offsetof(struct task_struct, thread.s[2])
7db91e57a0acde Palmer Dabbelt 2017-07-10 304 - offsetof(struct task_struct, thread.ra)
7db91e57a0acde Palmer Dabbelt 2017-07-10 305 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 306 DEFINE(TASK_THREAD_S3_RA,
7db91e57a0acde Palmer Dabbelt 2017-07-10 307 offsetof(struct task_struct, thread.s[3])
7db91e57a0acde Palmer Dabbelt 2017-07-10 308 - offsetof(struct task_struct, thread.ra)
7db91e57a0acde Palmer Dabbelt 2017-07-10 309 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 310 DEFINE(TASK_THREAD_S4_RA,
7db91e57a0acde Palmer Dabbelt 2017-07-10 311 offsetof(struct task_struct, thread.s[4])
7db91e57a0acde Palmer Dabbelt 2017-07-10 312 - offsetof(struct task_struct, thread.ra)
7db91e57a0acde Palmer Dabbelt 2017-07-10 313 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 314 DEFINE(TASK_THREAD_S5_RA,
7db91e57a0acde Palmer Dabbelt 2017-07-10 315 offsetof(struct task_struct, thread.s[5])
7db91e57a0acde Palmer Dabbelt 2017-07-10 316 - offsetof(struct task_struct, thread.ra)
7db91e57a0acde Palmer Dabbelt 2017-07-10 317 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 318 DEFINE(TASK_THREAD_S6_RA,
7db91e57a0acde Palmer Dabbelt 2017-07-10 319 offsetof(struct task_struct, thread.s[6])
7db91e57a0acde Palmer Dabbelt 2017-07-10 320 - offsetof(struct task_struct, thread.ra)
7db91e57a0acde Palmer Dabbelt 2017-07-10 321 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 322 DEFINE(TASK_THREAD_S7_RA,
7db91e57a0acde Palmer Dabbelt 2017-07-10 323 offsetof(struct task_struct, thread.s[7])
7db91e57a0acde Palmer Dabbelt 2017-07-10 324 - offsetof(struct task_struct, thread.ra)
7db91e57a0acde Palmer Dabbelt 2017-07-10 325 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 326 DEFINE(TASK_THREAD_S8_RA,
7db91e57a0acde Palmer Dabbelt 2017-07-10 327 offsetof(struct task_struct, thread.s[8])
7db91e57a0acde Palmer Dabbelt 2017-07-10 328 - offsetof(struct task_struct, thread.ra)
7db91e57a0acde Palmer Dabbelt 2017-07-10 329 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 330 DEFINE(TASK_THREAD_S9_RA,
7db91e57a0acde Palmer Dabbelt 2017-07-10 331 offsetof(struct task_struct, thread.s[9])
7db91e57a0acde Palmer Dabbelt 2017-07-10 332 - offsetof(struct task_struct, thread.ra)
7db91e57a0acde Palmer Dabbelt 2017-07-10 333 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 334 DEFINE(TASK_THREAD_S10_RA,
7db91e57a0acde Palmer Dabbelt 2017-07-10 335 offsetof(struct task_struct, thread.s[10])
7db91e57a0acde Palmer Dabbelt 2017-07-10 336 - offsetof(struct task_struct, thread.ra)
7db91e57a0acde Palmer Dabbelt 2017-07-10 337 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 338 DEFINE(TASK_THREAD_S11_RA,
7db91e57a0acde Palmer Dabbelt 2017-07-10 339 offsetof(struct task_struct, thread.s[11])
7db91e57a0acde Palmer Dabbelt 2017-07-10 340 - offsetof(struct task_struct, thread.ra)
7db91e57a0acde Palmer Dabbelt 2017-07-10 341 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 342
7db91e57a0acde Palmer Dabbelt 2017-07-10 343 DEFINE(TASK_THREAD_F0_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 344 offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 345 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 346 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 347 DEFINE(TASK_THREAD_F1_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 348 offsetof(struct task_struct, thread.fstate.f[1])
7db91e57a0acde Palmer Dabbelt 2017-07-10 349 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 350 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 351 DEFINE(TASK_THREAD_F2_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 352 offsetof(struct task_struct, thread.fstate.f[2])
7db91e57a0acde Palmer Dabbelt 2017-07-10 353 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 354 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 355 DEFINE(TASK_THREAD_F3_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 356 offsetof(struct task_struct, thread.fstate.f[3])
7db91e57a0acde Palmer Dabbelt 2017-07-10 357 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 358 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 359 DEFINE(TASK_THREAD_F4_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 360 offsetof(struct task_struct, thread.fstate.f[4])
7db91e57a0acde Palmer Dabbelt 2017-07-10 361 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 362 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 363 DEFINE(TASK_THREAD_F5_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 364 offsetof(struct task_struct, thread.fstate.f[5])
7db91e57a0acde Palmer Dabbelt 2017-07-10 365 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 366 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 367 DEFINE(TASK_THREAD_F6_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 368 offsetof(struct task_struct, thread.fstate.f[6])
7db91e57a0acde Palmer Dabbelt 2017-07-10 369 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 370 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 371 DEFINE(TASK_THREAD_F7_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 372 offsetof(struct task_struct, thread.fstate.f[7])
7db91e57a0acde Palmer Dabbelt 2017-07-10 373 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 374 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 375 DEFINE(TASK_THREAD_F8_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 376 offsetof(struct task_struct, thread.fstate.f[8])
7db91e57a0acde Palmer Dabbelt 2017-07-10 377 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 378 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 379 DEFINE(TASK_THREAD_F9_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 380 offsetof(struct task_struct, thread.fstate.f[9])
7db91e57a0acde Palmer Dabbelt 2017-07-10 381 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 382 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 383 DEFINE(TASK_THREAD_F10_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 384 offsetof(struct task_struct, thread.fstate.f[10])
7db91e57a0acde Palmer Dabbelt 2017-07-10 385 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 386 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 387 DEFINE(TASK_THREAD_F11_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 388 offsetof(struct task_struct, thread.fstate.f[11])
7db91e57a0acde Palmer Dabbelt 2017-07-10 389 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 390 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 391 DEFINE(TASK_THREAD_F12_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 392 offsetof(struct task_struct, thread.fstate.f[12])
7db91e57a0acde Palmer Dabbelt 2017-07-10 393 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 394 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 395 DEFINE(TASK_THREAD_F13_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 396 offsetof(struct task_struct, thread.fstate.f[13])
7db91e57a0acde Palmer Dabbelt 2017-07-10 397 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 398 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 399 DEFINE(TASK_THREAD_F14_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 400 offsetof(struct task_struct, thread.fstate.f[14])
7db91e57a0acde Palmer Dabbelt 2017-07-10 401 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 402 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 403 DEFINE(TASK_THREAD_F15_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 404 offsetof(struct task_struct, thread.fstate.f[15])
7db91e57a0acde Palmer Dabbelt 2017-07-10 405 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 406 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 407 DEFINE(TASK_THREAD_F16_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 408 offsetof(struct task_struct, thread.fstate.f[16])
7db91e57a0acde Palmer Dabbelt 2017-07-10 409 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 410 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 411 DEFINE(TASK_THREAD_F17_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 412 offsetof(struct task_struct, thread.fstate.f[17])
7db91e57a0acde Palmer Dabbelt 2017-07-10 413 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 414 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 415 DEFINE(TASK_THREAD_F18_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 416 offsetof(struct task_struct, thread.fstate.f[18])
7db91e57a0acde Palmer Dabbelt 2017-07-10 417 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 418 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 419 DEFINE(TASK_THREAD_F19_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 420 offsetof(struct task_struct, thread.fstate.f[19])
7db91e57a0acde Palmer Dabbelt 2017-07-10 421 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 422 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 423 DEFINE(TASK_THREAD_F20_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 424 offsetof(struct task_struct, thread.fstate.f[20])
7db91e57a0acde Palmer Dabbelt 2017-07-10 425 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 426 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 427 DEFINE(TASK_THREAD_F21_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 428 offsetof(struct task_struct, thread.fstate.f[21])
7db91e57a0acde Palmer Dabbelt 2017-07-10 429 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 430 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 431 DEFINE(TASK_THREAD_F22_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 432 offsetof(struct task_struct, thread.fstate.f[22])
7db91e57a0acde Palmer Dabbelt 2017-07-10 433 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 434 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 435 DEFINE(TASK_THREAD_F23_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 436 offsetof(struct task_struct, thread.fstate.f[23])
7db91e57a0acde Palmer Dabbelt 2017-07-10 437 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 438 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 439 DEFINE(TASK_THREAD_F24_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 440 offsetof(struct task_struct, thread.fstate.f[24])
7db91e57a0acde Palmer Dabbelt 2017-07-10 441 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 442 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 443 DEFINE(TASK_THREAD_F25_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 444 offsetof(struct task_struct, thread.fstate.f[25])
7db91e57a0acde Palmer Dabbelt 2017-07-10 445 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 446 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 447 DEFINE(TASK_THREAD_F26_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 448 offsetof(struct task_struct, thread.fstate.f[26])
7db91e57a0acde Palmer Dabbelt 2017-07-10 449 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 450 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 451 DEFINE(TASK_THREAD_F27_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 452 offsetof(struct task_struct, thread.fstate.f[27])
7db91e57a0acde Palmer Dabbelt 2017-07-10 453 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 454 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 455 DEFINE(TASK_THREAD_F28_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 456 offsetof(struct task_struct, thread.fstate.f[28])
7db91e57a0acde Palmer Dabbelt 2017-07-10 457 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 458 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 459 DEFINE(TASK_THREAD_F29_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 460 offsetof(struct task_struct, thread.fstate.f[29])
7db91e57a0acde Palmer Dabbelt 2017-07-10 461 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 462 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 463 DEFINE(TASK_THREAD_F30_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 464 offsetof(struct task_struct, thread.fstate.f[30])
7db91e57a0acde Palmer Dabbelt 2017-07-10 465 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 466 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 467 DEFINE(TASK_THREAD_F31_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 468 offsetof(struct task_struct, thread.fstate.f[31])
7db91e57a0acde Palmer Dabbelt 2017-07-10 469 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 470 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 471 DEFINE(TASK_THREAD_FCSR_F0,
7db91e57a0acde Palmer Dabbelt 2017-07-10 472 offsetof(struct task_struct, thread.fstate.fcsr)
7db91e57a0acde Palmer Dabbelt 2017-07-10 473 - offsetof(struct task_struct, thread.fstate.f[0])
7db91e57a0acde Palmer Dabbelt 2017-07-10 474 );
7db91e57a0acde Palmer Dabbelt 2017-07-10 475
7db91e57a0acde Palmer Dabbelt 2017-07-10 476 /*
7db91e57a0acde Palmer Dabbelt 2017-07-10 477 * We allocate a pt_regs on the stack when entering the kernel. This
7db91e57a0acde Palmer Dabbelt 2017-07-10 478 * ensures the alignment is sane.
7db91e57a0acde Palmer Dabbelt 2017-07-10 479 */
7db91e57a0acde Palmer Dabbelt 2017-07-10 480 DEFINE(PT_SIZE_ON_STACK, ALIGN(sizeof(struct pt_regs), STACK_ALIGN));
658e2c5125bbbc Alexandre Ghiti 2021-06-17 481
658e2c5125bbbc Alexandre Ghiti 2021-06-17 482 OFFSET(KERNEL_MAP_VIRT_ADDR, kernel_mapping, virt_addr);
9a2451f1866344 Atish Patra 2022-01-20 483 OFFSET(SBI_HART_BOOT_TASK_PTR_OFFSET, sbi_hart_boot_data, task_ptr);
9a2451f1866344 Atish Patra 2022-01-20 484 OFFSET(SBI_HART_BOOT_STACK_PTR_OFFSET, sbi_hart_boot_data, stack_ptr);
a77c752ef9c2a1 Clément Léger 2023-10-26 485
a77c752ef9c2a1 Clément Léger 2023-10-26 486 OFFSET(SSE_INTERRUPTED_EXEC_MODE, sse_interrupted_state, exec_mode);
a77c752ef9c2a1 Clément Léger 2023-10-26 487 OFFSET(SSE_INTERRUPTED_S0, sse_interrupted_state, s0);
a77c752ef9c2a1 Clément Léger 2023-10-26 488 OFFSET(SSE_INTERRUPTED_TP, sse_interrupted_state, tp);
a77c752ef9c2a1 Clément Léger 2023-10-26 489 OFFSET(SSE_INTERRUPTED_PC, sse_interrupted_state, pc);
a77c752ef9c2a1 Clément Léger 2023-10-26 490
a77c752ef9c2a1 Clément Léger 2023-10-26 491 DEFINE(STACKFRAME_SIZE_ON_STACK, ALIGN(sizeof(struct stackframe),
a77c752ef9c2a1 Clément Léger 2023-10-26 492 STACK_ALIGN));
a77c752ef9c2a1 Clément Léger 2023-10-26 493 OFFSET(STACKFRAME_FP, stackframe, fp);
a77c752ef9c2a1 Clément Léger 2023-10-26 494 OFFSET(STACKFRAME_RA, stackframe, ra);
a77c752ef9c2a1 Clément Léger 2023-10-26 @495 DEFINE(SBI_EXT_SSE, SBI_EXT_SSE);
a77c752ef9c2a1 Clément Léger 2023-10-26 @496 DEFINE(SBI_SSE_EVENT_COMPLETE, SBI_SSE_EVENT_COMPLETE);
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [External] [RFC PATCH 0/3] riscv: add support for SBI Supervisor Software Events
2023-10-26 14:31 ` Clément Léger
(?)
@ 2023-12-07 9:09 ` Xu Lu
-1 siblings, 0 replies; 25+ messages in thread
From: Xu Lu @ 2023-12-07 9:09 UTC (permalink / raw)
To: Clément Léger
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Atish Patra, Anup Patel,
linux-riscv, linux-kernel, linux-arm-kernel, Himanshu Chauhan
Pardon. It seems that the code in opensbi[1] is not complete for PMU NMI now.
For example, the pmu ovf irq is still delegated to supervisor mode and
thus can not really play a role as NMI. And neither the kernel nor
opensbi will inject a pmu event.
To complete the work, we think maybe 'enable_cb' and 'disable_cb'
functions can be supplied for sbi_sse_cb_ops.
When sbi_sse_enable is called to enable pmu event, the enable_cb will
be called to revoke the delegation of pmu ovf irq and enable this irq
in CSR_MIE.
When pmu ovf irq occurs, kernel traps into opensbi and opensbi injects
the event via sbi_sse_inject_event and eret back to kernel.
Please point it out if we have any misunderstanding.
By the way, how is SSE going now? We will appreciate it very much if
we can participate in some development work in kernel or opensbi and
be of any help.
Regards!
Link: https://github.com/rivosinc/opensbi/tree/dev/cleger/sse [1]
On Thu, Oct 26, 2023 at 10:31 PM Clément Léger <cleger@rivosinc.com> wrote:
>
> The SBI Supervisor Software Events (SSE) extensions provides a mechanism
> to inject software events from an SBI implementation to supervisor
> software such that it preempts all other supervisor level traps and
> interrupts [1].
>
> Various events are defined and can be send asynchronously to supervisor
> software (RAS, PMU, DEBUG, Asynchronous page fault) from SBI as well
> as platform specific events. Events can be either local (per-hart) or
> global. Events can be nested on top of each other based on priority and
> can interrupt the kernel at any time.
>
> First patch adds the SSE definitions. Second one adds support for SSE
> itself. Implementation is split between arch specific code and generic
> part (similarly to what is done for ARM SDEI). Finally, the last patch
> add support fro SSE event in the SBI PMU driver. If the SSE event is
> available from the SBI then, it will be used instead of the normal
> interrupt.
>
> Amongst the specific points that needs to be handle is the interruption
> at any point of the kernel execution and more specifically during
> exception handling. Due to the fact that the exception entry
> implementation uses the SCRATCH CSR as both the current task struct and
> as the temporary register to switch the stack and save register, it is
> difficult to reliably get the current task struct if we get interrupted
> at this specific moment. A fixup-like mechanism allows to mark the
> location of the current task struct depending on the entry level
> (user/kernel) and the location. This is then used in the SSE assembly to
> determine where is located the current task_struct.
>
> Contrary to pseudo NMI [2], SSE does not modifies the way interrupts are
> handled and does not adds any overhead to existing code. Moreover, it
> provides "true" NMI-like interrupts which can interrupt the kernel at
> any time (even in exception handling). This is particularly crucial for
> RAS errors which needs to be handled as fast as possible to avoid any
> fault propagation. Additionally, SSE event handling is faster that the
> standard IRQ handling path with almost half executed instruction (700 vs
> 1590). Some complementary tests/perf measurements will be done.
>
> For testing purpose, one can use the provided SBI implementation at [3].
> This series also needs patch [4] to fix a bug in the PMU driver.
>
> Link: https://lists.riscv.org/g/tech-prs/message/515 [1]
> Link: https://lore.kernel.org/lkml/20231023082911.23242-10-luxu.kernel@bytedance.com/T/ [2]
> Link: https://github.com/rivosinc/opensbi/tree/dev/cleger/sse [3]
> Link: https://lore.kernel.org/linux-arm-kernel/20231026084010.11888-1-alexghiti@rivosinc.com/ [4]
>
> ---
>
> Clément Léger (3):
> riscv: add SBI SSE extension definitions
> riscv: add support for SBI Supervisor Software Events extension
> perf: RISC-V: add support for SSE event
>
> arch/riscv/include/asm/asm-prototypes.h | 5 +
> arch/riscv/include/asm/sbi.h | 40 ++
> arch/riscv/include/asm/sse.h | 94 +++++
> arch/riscv/kernel/Makefile | 1 +
> arch/riscv/kernel/asm-offsets.c | 17 +
> arch/riscv/kernel/entry.S | 156 ++++++++
> arch/riscv/kernel/sbi.c | 4 +
> arch/riscv/kernel/sse.c | 97 +++++
> arch/riscv/kernel/stacktrace.c | 13 +
> arch/riscv/kernel/vmlinux.lds.S | 6 +
> drivers/firmware/Kconfig | 10 +
> drivers/firmware/Makefile | 1 +
> drivers/firmware/riscv_sse.c | 496 ++++++++++++++++++++++++
> drivers/perf/riscv_pmu_sbi.c | 51 ++-
> include/linux/riscv_sse.h | 56 +++
> 15 files changed, 1038 insertions(+), 9 deletions(-)
> create mode 100644 arch/riscv/include/asm/sse.h
> create mode 100644 arch/riscv/kernel/sse.c
> create mode 100644 drivers/firmware/riscv_sse.c
> create mode 100644 include/linux/riscv_sse.h
>
> --
> 2.42.0
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [External] [RFC PATCH 0/3] riscv: add support for SBI Supervisor Software Events
@ 2023-12-07 9:09 ` Xu Lu
0 siblings, 0 replies; 25+ messages in thread
From: Xu Lu @ 2023-12-07 9:09 UTC (permalink / raw)
To: Clément Léger
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Atish Patra, Anup Patel,
linux-riscv, linux-kernel, linux-arm-kernel, Himanshu Chauhan
Pardon. It seems that the code in opensbi[1] is not complete for PMU NMI now.
For example, the pmu ovf irq is still delegated to supervisor mode and
thus can not really play a role as NMI. And neither the kernel nor
opensbi will inject a pmu event.
To complete the work, we think maybe 'enable_cb' and 'disable_cb'
functions can be supplied for sbi_sse_cb_ops.
When sbi_sse_enable is called to enable pmu event, the enable_cb will
be called to revoke the delegation of pmu ovf irq and enable this irq
in CSR_MIE.
When pmu ovf irq occurs, kernel traps into opensbi and opensbi injects
the event via sbi_sse_inject_event and eret back to kernel.
Please point it out if we have any misunderstanding.
By the way, how is SSE going now? We will appreciate it very much if
we can participate in some development work in kernel or opensbi and
be of any help.
Regards!
Link: https://github.com/rivosinc/opensbi/tree/dev/cleger/sse [1]
On Thu, Oct 26, 2023 at 10:31 PM Clément Léger <cleger@rivosinc.com> wrote:
>
> The SBI Supervisor Software Events (SSE) extensions provides a mechanism
> to inject software events from an SBI implementation to supervisor
> software such that it preempts all other supervisor level traps and
> interrupts [1].
>
> Various events are defined and can be send asynchronously to supervisor
> software (RAS, PMU, DEBUG, Asynchronous page fault) from SBI as well
> as platform specific events. Events can be either local (per-hart) or
> global. Events can be nested on top of each other based on priority and
> can interrupt the kernel at any time.
>
> First patch adds the SSE definitions. Second one adds support for SSE
> itself. Implementation is split between arch specific code and generic
> part (similarly to what is done for ARM SDEI). Finally, the last patch
> add support fro SSE event in the SBI PMU driver. If the SSE event is
> available from the SBI then, it will be used instead of the normal
> interrupt.
>
> Amongst the specific points that needs to be handle is the interruption
> at any point of the kernel execution and more specifically during
> exception handling. Due to the fact that the exception entry
> implementation uses the SCRATCH CSR as both the current task struct and
> as the temporary register to switch the stack and save register, it is
> difficult to reliably get the current task struct if we get interrupted
> at this specific moment. A fixup-like mechanism allows to mark the
> location of the current task struct depending on the entry level
> (user/kernel) and the location. This is then used in the SSE assembly to
> determine where is located the current task_struct.
>
> Contrary to pseudo NMI [2], SSE does not modifies the way interrupts are
> handled and does not adds any overhead to existing code. Moreover, it
> provides "true" NMI-like interrupts which can interrupt the kernel at
> any time (even in exception handling). This is particularly crucial for
> RAS errors which needs to be handled as fast as possible to avoid any
> fault propagation. Additionally, SSE event handling is faster that the
> standard IRQ handling path with almost half executed instruction (700 vs
> 1590). Some complementary tests/perf measurements will be done.
>
> For testing purpose, one can use the provided SBI implementation at [3].
> This series also needs patch [4] to fix a bug in the PMU driver.
>
> Link: https://lists.riscv.org/g/tech-prs/message/515 [1]
> Link: https://lore.kernel.org/lkml/20231023082911.23242-10-luxu.kernel@bytedance.com/T/ [2]
> Link: https://github.com/rivosinc/opensbi/tree/dev/cleger/sse [3]
> Link: https://lore.kernel.org/linux-arm-kernel/20231026084010.11888-1-alexghiti@rivosinc.com/ [4]
>
> ---
>
> Clément Léger (3):
> riscv: add SBI SSE extension definitions
> riscv: add support for SBI Supervisor Software Events extension
> perf: RISC-V: add support for SSE event
>
> arch/riscv/include/asm/asm-prototypes.h | 5 +
> arch/riscv/include/asm/sbi.h | 40 ++
> arch/riscv/include/asm/sse.h | 94 +++++
> arch/riscv/kernel/Makefile | 1 +
> arch/riscv/kernel/asm-offsets.c | 17 +
> arch/riscv/kernel/entry.S | 156 ++++++++
> arch/riscv/kernel/sbi.c | 4 +
> arch/riscv/kernel/sse.c | 97 +++++
> arch/riscv/kernel/stacktrace.c | 13 +
> arch/riscv/kernel/vmlinux.lds.S | 6 +
> drivers/firmware/Kconfig | 10 +
> drivers/firmware/Makefile | 1 +
> drivers/firmware/riscv_sse.c | 496 ++++++++++++++++++++++++
> drivers/perf/riscv_pmu_sbi.c | 51 ++-
> include/linux/riscv_sse.h | 56 +++
> 15 files changed, 1038 insertions(+), 9 deletions(-)
> create mode 100644 arch/riscv/include/asm/sse.h
> create mode 100644 arch/riscv/kernel/sse.c
> create mode 100644 drivers/firmware/riscv_sse.c
> create mode 100644 include/linux/riscv_sse.h
>
> --
> 2.42.0
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [External] [RFC PATCH 0/3] riscv: add support for SBI Supervisor Software Events
@ 2023-12-07 9:09 ` Xu Lu
0 siblings, 0 replies; 25+ messages in thread
From: Xu Lu @ 2023-12-07 9:09 UTC (permalink / raw)
To: Clément Léger
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Atish Patra, Anup Patel,
linux-riscv, linux-kernel, linux-arm-kernel, Himanshu Chauhan
Pardon. It seems that the code in opensbi[1] is not complete for PMU NMI now.
For example, the pmu ovf irq is still delegated to supervisor mode and
thus can not really play a role as NMI. And neither the kernel nor
opensbi will inject a pmu event.
To complete the work, we think maybe 'enable_cb' and 'disable_cb'
functions can be supplied for sbi_sse_cb_ops.
When sbi_sse_enable is called to enable pmu event, the enable_cb will
be called to revoke the delegation of pmu ovf irq and enable this irq
in CSR_MIE.
When pmu ovf irq occurs, kernel traps into opensbi and opensbi injects
the event via sbi_sse_inject_event and eret back to kernel.
Please point it out if we have any misunderstanding.
By the way, how is SSE going now? We will appreciate it very much if
we can participate in some development work in kernel or opensbi and
be of any help.
Regards!
Link: https://github.com/rivosinc/opensbi/tree/dev/cleger/sse [1]
On Thu, Oct 26, 2023 at 10:31 PM Clément Léger <cleger@rivosinc.com> wrote:
>
> The SBI Supervisor Software Events (SSE) extensions provides a mechanism
> to inject software events from an SBI implementation to supervisor
> software such that it preempts all other supervisor level traps and
> interrupts [1].
>
> Various events are defined and can be send asynchronously to supervisor
> software (RAS, PMU, DEBUG, Asynchronous page fault) from SBI as well
> as platform specific events. Events can be either local (per-hart) or
> global. Events can be nested on top of each other based on priority and
> can interrupt the kernel at any time.
>
> First patch adds the SSE definitions. Second one adds support for SSE
> itself. Implementation is split between arch specific code and generic
> part (similarly to what is done for ARM SDEI). Finally, the last patch
> add support fro SSE event in the SBI PMU driver. If the SSE event is
> available from the SBI then, it will be used instead of the normal
> interrupt.
>
> Amongst the specific points that needs to be handle is the interruption
> at any point of the kernel execution and more specifically during
> exception handling. Due to the fact that the exception entry
> implementation uses the SCRATCH CSR as both the current task struct and
> as the temporary register to switch the stack and save register, it is
> difficult to reliably get the current task struct if we get interrupted
> at this specific moment. A fixup-like mechanism allows to mark the
> location of the current task struct depending on the entry level
> (user/kernel) and the location. This is then used in the SSE assembly to
> determine where is located the current task_struct.
>
> Contrary to pseudo NMI [2], SSE does not modifies the way interrupts are
> handled and does not adds any overhead to existing code. Moreover, it
> provides "true" NMI-like interrupts which can interrupt the kernel at
> any time (even in exception handling). This is particularly crucial for
> RAS errors which needs to be handled as fast as possible to avoid any
> fault propagation. Additionally, SSE event handling is faster that the
> standard IRQ handling path with almost half executed instruction (700 vs
> 1590). Some complementary tests/perf measurements will be done.
>
> For testing purpose, one can use the provided SBI implementation at [3].
> This series also needs patch [4] to fix a bug in the PMU driver.
>
> Link: https://lists.riscv.org/g/tech-prs/message/515 [1]
> Link: https://lore.kernel.org/lkml/20231023082911.23242-10-luxu.kernel@bytedance.com/T/ [2]
> Link: https://github.com/rivosinc/opensbi/tree/dev/cleger/sse [3]
> Link: https://lore.kernel.org/linux-arm-kernel/20231026084010.11888-1-alexghiti@rivosinc.com/ [4]
>
> ---
>
> Clément Léger (3):
> riscv: add SBI SSE extension definitions
> riscv: add support for SBI Supervisor Software Events extension
> perf: RISC-V: add support for SSE event
>
> arch/riscv/include/asm/asm-prototypes.h | 5 +
> arch/riscv/include/asm/sbi.h | 40 ++
> arch/riscv/include/asm/sse.h | 94 +++++
> arch/riscv/kernel/Makefile | 1 +
> arch/riscv/kernel/asm-offsets.c | 17 +
> arch/riscv/kernel/entry.S | 156 ++++++++
> arch/riscv/kernel/sbi.c | 4 +
> arch/riscv/kernel/sse.c | 97 +++++
> arch/riscv/kernel/stacktrace.c | 13 +
> arch/riscv/kernel/vmlinux.lds.S | 6 +
> drivers/firmware/Kconfig | 10 +
> drivers/firmware/Makefile | 1 +
> drivers/firmware/riscv_sse.c | 496 ++++++++++++++++++++++++
> drivers/perf/riscv_pmu_sbi.c | 51 ++-
> include/linux/riscv_sse.h | 56 +++
> 15 files changed, 1038 insertions(+), 9 deletions(-)
> create mode 100644 arch/riscv/include/asm/sse.h
> create mode 100644 arch/riscv/kernel/sse.c
> create mode 100644 drivers/firmware/riscv_sse.c
> create mode 100644 include/linux/riscv_sse.h
>
> --
> 2.42.0
>
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [External] [RFC PATCH 0/3] riscv: add support for SBI Supervisor Software Events
2023-12-07 9:09 ` Xu Lu
(?)
@ 2023-12-07 9:23 ` Clément Léger
-1 siblings, 0 replies; 25+ messages in thread
From: Clément Léger @ 2023-12-07 9:23 UTC (permalink / raw)
To: Xu Lu
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Atish Patra, Anup Patel,
linux-riscv, linux-kernel, linux-arm-kernel, Himanshu Chauhan
On 07/12/2023 10:09, Xu Lu wrote:
> Pardon. It seems that the code in opensbi[1] is not complete for PMU NMI now.
> For example, the pmu ovf irq is still delegated to supervisor mode and
> thus can not really play a role as NMI. And neither the kernel nor
> opensbi will inject a pmu event.
>
> To complete the work, we think maybe 'enable_cb' and 'disable_cb'
> functions can be supplied for sbi_sse_cb_ops.
> When sbi_sse_enable is called to enable pmu event, the enable_cb will
> be called to revoke the delegation of pmu ovf irq and enable this irq
> in CSR_MIE.
Hi Xu,
Indeed, this part has been developed but was left out for the RFC. But
your understanding is correct.
> When pmu ovf irq occurs, kernel traps into opensbi and opensbi injects
> the event via sbi_sse_inject_event and eret back to kernel.
>
> Please point it out if we have any misunderstanding.
>
> By the way, how is SSE going now? We will appreciate it very much if
> we can participate in some development work in kernel or opensbi and
> be of any help.
The development is almost complete, Anup/Himanchu will send a new
revision of the spec addressing various comments and I'll resend the RFC
following that spec update.
Regards,
Clément
>
> Regards!
>
> Link: https://github.com/rivosinc/opensbi/tree/dev/cleger/sse [1]
>
> On Thu, Oct 26, 2023 at 10:31 PM Clément Léger <cleger@rivosinc.com> wrote:
>>
>> The SBI Supervisor Software Events (SSE) extensions provides a mechanism
>> to inject software events from an SBI implementation to supervisor
>> software such that it preempts all other supervisor level traps and
>> interrupts [1].
>>
>> Various events are defined and can be send asynchronously to supervisor
>> software (RAS, PMU, DEBUG, Asynchronous page fault) from SBI as well
>> as platform specific events. Events can be either local (per-hart) or
>> global. Events can be nested on top of each other based on priority and
>> can interrupt the kernel at any time.
>>
>> First patch adds the SSE definitions. Second one adds support for SSE
>> itself. Implementation is split between arch specific code and generic
>> part (similarly to what is done for ARM SDEI). Finally, the last patch
>> add support fro SSE event in the SBI PMU driver. If the SSE event is
>> available from the SBI then, it will be used instead of the normal
>> interrupt.
>>
>> Amongst the specific points that needs to be handle is the interruption
>> at any point of the kernel execution and more specifically during
>> exception handling. Due to the fact that the exception entry
>> implementation uses the SCRATCH CSR as both the current task struct and
>> as the temporary register to switch the stack and save register, it is
>> difficult to reliably get the current task struct if we get interrupted
>> at this specific moment. A fixup-like mechanism allows to mark the
>> location of the current task struct depending on the entry level
>> (user/kernel) and the location. This is then used in the SSE assembly to
>> determine where is located the current task_struct.
>>
>> Contrary to pseudo NMI [2], SSE does not modifies the way interrupts are
>> handled and does not adds any overhead to existing code. Moreover, it
>> provides "true" NMI-like interrupts which can interrupt the kernel at
>> any time (even in exception handling). This is particularly crucial for
>> RAS errors which needs to be handled as fast as possible to avoid any
>> fault propagation. Additionally, SSE event handling is faster that the
>> standard IRQ handling path with almost half executed instruction (700 vs
>> 1590). Some complementary tests/perf measurements will be done.
>>
>> For testing purpose, one can use the provided SBI implementation at [3].
>> This series also needs patch [4] to fix a bug in the PMU driver.
>>
>> Link: https://lists.riscv.org/g/tech-prs/message/515 [1]
>> Link: https://lore.kernel.org/lkml/20231023082911.23242-10-luxu.kernel@bytedance.com/T/ [2]
>> Link: https://github.com/rivosinc/opensbi/tree/dev/cleger/sse [3]
>> Link: https://lore.kernel.org/linux-arm-kernel/20231026084010.11888-1-alexghiti@rivosinc.com/ [4]
>>
>> ---
>>
>> Clément Léger (3):
>> riscv: add SBI SSE extension definitions
>> riscv: add support for SBI Supervisor Software Events extension
>> perf: RISC-V: add support for SSE event
>>
>> arch/riscv/include/asm/asm-prototypes.h | 5 +
>> arch/riscv/include/asm/sbi.h | 40 ++
>> arch/riscv/include/asm/sse.h | 94 +++++
>> arch/riscv/kernel/Makefile | 1 +
>> arch/riscv/kernel/asm-offsets.c | 17 +
>> arch/riscv/kernel/entry.S | 156 ++++++++
>> arch/riscv/kernel/sbi.c | 4 +
>> arch/riscv/kernel/sse.c | 97 +++++
>> arch/riscv/kernel/stacktrace.c | 13 +
>> arch/riscv/kernel/vmlinux.lds.S | 6 +
>> drivers/firmware/Kconfig | 10 +
>> drivers/firmware/Makefile | 1 +
>> drivers/firmware/riscv_sse.c | 496 ++++++++++++++++++++++++
>> drivers/perf/riscv_pmu_sbi.c | 51 ++-
>> include/linux/riscv_sse.h | 56 +++
>> 15 files changed, 1038 insertions(+), 9 deletions(-)
>> create mode 100644 arch/riscv/include/asm/sse.h
>> create mode 100644 arch/riscv/kernel/sse.c
>> create mode 100644 drivers/firmware/riscv_sse.c
>> create mode 100644 include/linux/riscv_sse.h
>>
>> --
>> 2.42.0
>>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [External] [RFC PATCH 0/3] riscv: add support for SBI Supervisor Software Events
@ 2023-12-07 9:23 ` Clément Léger
0 siblings, 0 replies; 25+ messages in thread
From: Clément Léger @ 2023-12-07 9:23 UTC (permalink / raw)
To: Xu Lu
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Atish Patra, Anup Patel,
linux-riscv, linux-kernel, linux-arm-kernel, Himanshu Chauhan
On 07/12/2023 10:09, Xu Lu wrote:
> Pardon. It seems that the code in opensbi[1] is not complete for PMU NMI now.
> For example, the pmu ovf irq is still delegated to supervisor mode and
> thus can not really play a role as NMI. And neither the kernel nor
> opensbi will inject a pmu event.
>
> To complete the work, we think maybe 'enable_cb' and 'disable_cb'
> functions can be supplied for sbi_sse_cb_ops.
> When sbi_sse_enable is called to enable pmu event, the enable_cb will
> be called to revoke the delegation of pmu ovf irq and enable this irq
> in CSR_MIE.
Hi Xu,
Indeed, this part has been developed but was left out for the RFC. But
your understanding is correct.
> When pmu ovf irq occurs, kernel traps into opensbi and opensbi injects
> the event via sbi_sse_inject_event and eret back to kernel.
>
> Please point it out if we have any misunderstanding.
>
> By the way, how is SSE going now? We will appreciate it very much if
> we can participate in some development work in kernel or opensbi and
> be of any help.
The development is almost complete, Anup/Himanchu will send a new
revision of the spec addressing various comments and I'll resend the RFC
following that spec update.
Regards,
Clément
>
> Regards!
>
> Link: https://github.com/rivosinc/opensbi/tree/dev/cleger/sse [1]
>
> On Thu, Oct 26, 2023 at 10:31 PM Clément Léger <cleger@rivosinc.com> wrote:
>>
>> The SBI Supervisor Software Events (SSE) extensions provides a mechanism
>> to inject software events from an SBI implementation to supervisor
>> software such that it preempts all other supervisor level traps and
>> interrupts [1].
>>
>> Various events are defined and can be send asynchronously to supervisor
>> software (RAS, PMU, DEBUG, Asynchronous page fault) from SBI as well
>> as platform specific events. Events can be either local (per-hart) or
>> global. Events can be nested on top of each other based on priority and
>> can interrupt the kernel at any time.
>>
>> First patch adds the SSE definitions. Second one adds support for SSE
>> itself. Implementation is split between arch specific code and generic
>> part (similarly to what is done for ARM SDEI). Finally, the last patch
>> add support fro SSE event in the SBI PMU driver. If the SSE event is
>> available from the SBI then, it will be used instead of the normal
>> interrupt.
>>
>> Amongst the specific points that needs to be handle is the interruption
>> at any point of the kernel execution and more specifically during
>> exception handling. Due to the fact that the exception entry
>> implementation uses the SCRATCH CSR as both the current task struct and
>> as the temporary register to switch the stack and save register, it is
>> difficult to reliably get the current task struct if we get interrupted
>> at this specific moment. A fixup-like mechanism allows to mark the
>> location of the current task struct depending on the entry level
>> (user/kernel) and the location. This is then used in the SSE assembly to
>> determine where is located the current task_struct.
>>
>> Contrary to pseudo NMI [2], SSE does not modifies the way interrupts are
>> handled and does not adds any overhead to existing code. Moreover, it
>> provides "true" NMI-like interrupts which can interrupt the kernel at
>> any time (even in exception handling). This is particularly crucial for
>> RAS errors which needs to be handled as fast as possible to avoid any
>> fault propagation. Additionally, SSE event handling is faster that the
>> standard IRQ handling path with almost half executed instruction (700 vs
>> 1590). Some complementary tests/perf measurements will be done.
>>
>> For testing purpose, one can use the provided SBI implementation at [3].
>> This series also needs patch [4] to fix a bug in the PMU driver.
>>
>> Link: https://lists.riscv.org/g/tech-prs/message/515 [1]
>> Link: https://lore.kernel.org/lkml/20231023082911.23242-10-luxu.kernel@bytedance.com/T/ [2]
>> Link: https://github.com/rivosinc/opensbi/tree/dev/cleger/sse [3]
>> Link: https://lore.kernel.org/linux-arm-kernel/20231026084010.11888-1-alexghiti@rivosinc.com/ [4]
>>
>> ---
>>
>> Clément Léger (3):
>> riscv: add SBI SSE extension definitions
>> riscv: add support for SBI Supervisor Software Events extension
>> perf: RISC-V: add support for SSE event
>>
>> arch/riscv/include/asm/asm-prototypes.h | 5 +
>> arch/riscv/include/asm/sbi.h | 40 ++
>> arch/riscv/include/asm/sse.h | 94 +++++
>> arch/riscv/kernel/Makefile | 1 +
>> arch/riscv/kernel/asm-offsets.c | 17 +
>> arch/riscv/kernel/entry.S | 156 ++++++++
>> arch/riscv/kernel/sbi.c | 4 +
>> arch/riscv/kernel/sse.c | 97 +++++
>> arch/riscv/kernel/stacktrace.c | 13 +
>> arch/riscv/kernel/vmlinux.lds.S | 6 +
>> drivers/firmware/Kconfig | 10 +
>> drivers/firmware/Makefile | 1 +
>> drivers/firmware/riscv_sse.c | 496 ++++++++++++++++++++++++
>> drivers/perf/riscv_pmu_sbi.c | 51 ++-
>> include/linux/riscv_sse.h | 56 +++
>> 15 files changed, 1038 insertions(+), 9 deletions(-)
>> create mode 100644 arch/riscv/include/asm/sse.h
>> create mode 100644 arch/riscv/kernel/sse.c
>> create mode 100644 drivers/firmware/riscv_sse.c
>> create mode 100644 include/linux/riscv_sse.h
>>
>> --
>> 2.42.0
>>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [External] [RFC PATCH 0/3] riscv: add support for SBI Supervisor Software Events
@ 2023-12-07 9:23 ` Clément Léger
0 siblings, 0 replies; 25+ messages in thread
From: Clément Léger @ 2023-12-07 9:23 UTC (permalink / raw)
To: Xu Lu
Cc: Paul Walmsley, Palmer Dabbelt, Albert Ou, Atish Patra, Anup Patel,
linux-riscv, linux-kernel, linux-arm-kernel, Himanshu Chauhan
On 07/12/2023 10:09, Xu Lu wrote:
> Pardon. It seems that the code in opensbi[1] is not complete for PMU NMI now.
> For example, the pmu ovf irq is still delegated to supervisor mode and
> thus can not really play a role as NMI. And neither the kernel nor
> opensbi will inject a pmu event.
>
> To complete the work, we think maybe 'enable_cb' and 'disable_cb'
> functions can be supplied for sbi_sse_cb_ops.
> When sbi_sse_enable is called to enable pmu event, the enable_cb will
> be called to revoke the delegation of pmu ovf irq and enable this irq
> in CSR_MIE.
Hi Xu,
Indeed, this part has been developed but was left out for the RFC. But
your understanding is correct.
> When pmu ovf irq occurs, kernel traps into opensbi and opensbi injects
> the event via sbi_sse_inject_event and eret back to kernel.
>
> Please point it out if we have any misunderstanding.
>
> By the way, how is SSE going now? We will appreciate it very much if
> we can participate in some development work in kernel or opensbi and
> be of any help.
The development is almost complete, Anup/Himanchu will send a new
revision of the spec addressing various comments and I'll resend the RFC
following that spec update.
Regards,
Clément
>
> Regards!
>
> Link: https://github.com/rivosinc/opensbi/tree/dev/cleger/sse [1]
>
> On Thu, Oct 26, 2023 at 10:31 PM Clément Léger <cleger@rivosinc.com> wrote:
>>
>> The SBI Supervisor Software Events (SSE) extensions provides a mechanism
>> to inject software events from an SBI implementation to supervisor
>> software such that it preempts all other supervisor level traps and
>> interrupts [1].
>>
>> Various events are defined and can be send asynchronously to supervisor
>> software (RAS, PMU, DEBUG, Asynchronous page fault) from SBI as well
>> as platform specific events. Events can be either local (per-hart) or
>> global. Events can be nested on top of each other based on priority and
>> can interrupt the kernel at any time.
>>
>> First patch adds the SSE definitions. Second one adds support for SSE
>> itself. Implementation is split between arch specific code and generic
>> part (similarly to what is done for ARM SDEI). Finally, the last patch
>> add support fro SSE event in the SBI PMU driver. If the SSE event is
>> available from the SBI then, it will be used instead of the normal
>> interrupt.
>>
>> Amongst the specific points that needs to be handle is the interruption
>> at any point of the kernel execution and more specifically during
>> exception handling. Due to the fact that the exception entry
>> implementation uses the SCRATCH CSR as both the current task struct and
>> as the temporary register to switch the stack and save register, it is
>> difficult to reliably get the current task struct if we get interrupted
>> at this specific moment. A fixup-like mechanism allows to mark the
>> location of the current task struct depending on the entry level
>> (user/kernel) and the location. This is then used in the SSE assembly to
>> determine where is located the current task_struct.
>>
>> Contrary to pseudo NMI [2], SSE does not modifies the way interrupts are
>> handled and does not adds any overhead to existing code. Moreover, it
>> provides "true" NMI-like interrupts which can interrupt the kernel at
>> any time (even in exception handling). This is particularly crucial for
>> RAS errors which needs to be handled as fast as possible to avoid any
>> fault propagation. Additionally, SSE event handling is faster that the
>> standard IRQ handling path with almost half executed instruction (700 vs
>> 1590). Some complementary tests/perf measurements will be done.
>>
>> For testing purpose, one can use the provided SBI implementation at [3].
>> This series also needs patch [4] to fix a bug in the PMU driver.
>>
>> Link: https://lists.riscv.org/g/tech-prs/message/515 [1]
>> Link: https://lore.kernel.org/lkml/20231023082911.23242-10-luxu.kernel@bytedance.com/T/ [2]
>> Link: https://github.com/rivosinc/opensbi/tree/dev/cleger/sse [3]
>> Link: https://lore.kernel.org/linux-arm-kernel/20231026084010.11888-1-alexghiti@rivosinc.com/ [4]
>>
>> ---
>>
>> Clément Léger (3):
>> riscv: add SBI SSE extension definitions
>> riscv: add support for SBI Supervisor Software Events extension
>> perf: RISC-V: add support for SSE event
>>
>> arch/riscv/include/asm/asm-prototypes.h | 5 +
>> arch/riscv/include/asm/sbi.h | 40 ++
>> arch/riscv/include/asm/sse.h | 94 +++++
>> arch/riscv/kernel/Makefile | 1 +
>> arch/riscv/kernel/asm-offsets.c | 17 +
>> arch/riscv/kernel/entry.S | 156 ++++++++
>> arch/riscv/kernel/sbi.c | 4 +
>> arch/riscv/kernel/sse.c | 97 +++++
>> arch/riscv/kernel/stacktrace.c | 13 +
>> arch/riscv/kernel/vmlinux.lds.S | 6 +
>> drivers/firmware/Kconfig | 10 +
>> drivers/firmware/Makefile | 1 +
>> drivers/firmware/riscv_sse.c | 496 ++++++++++++++++++++++++
>> drivers/perf/riscv_pmu_sbi.c | 51 ++-
>> include/linux/riscv_sse.h | 56 +++
>> 15 files changed, 1038 insertions(+), 9 deletions(-)
>> create mode 100644 arch/riscv/include/asm/sse.h
>> create mode 100644 arch/riscv/kernel/sse.c
>> create mode 100644 drivers/firmware/riscv_sse.c
>> create mode 100644 include/linux/riscv_sse.h
>>
>> --
>> 2.42.0
>>
^ permalink raw reply [flat|nested] 25+ messages in thread
end of thread, other threads:[~2023-12-07 9:24 UTC | newest]
Thread overview: 25+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-10-31 4:54 [RFC PATCH 2/3] riscv: add support for SBI Supervisor Software Events extension kernel test robot
2023-11-02 5:25 ` kernel test robot
-- strict thread matches above, loose matches on Subject: below --
2023-10-30 13:47 kernel test robot
2023-10-31 2:23 ` kernel test robot
2023-10-26 14:31 [RFC PATCH 0/3] riscv: add support for SBI Supervisor Software Events Clément Léger
2023-10-26 14:31 ` Clément Léger
2023-10-26 14:31 ` Clément Léger
2023-10-26 14:31 ` [RFC PATCH 1/3] riscv: add SBI SSE extension definitions Clément Léger
2023-10-26 14:31 ` Clément Léger
2023-10-26 14:31 ` Clément Léger
2023-10-26 14:31 ` [RFC PATCH 2/3] riscv: add support for SBI Supervisor Software Events extension Clément Léger
2023-10-26 14:31 ` Clément Léger
2023-10-26 14:31 ` Clément Léger
2023-10-26 14:31 ` [RFC PATCH 3/3] perf: RISC-V: add support for SSE event Clément Léger
2023-10-26 14:31 ` Clément Léger
2023-10-26 14:31 ` Clément Léger
2023-10-26 19:52 ` Atish Patra
2023-10-26 19:52 ` Atish Patra
2023-10-26 19:52 ` Atish Patra
2023-12-07 9:09 ` [External] [RFC PATCH 0/3] riscv: add support for SBI Supervisor Software Events Xu Lu
2023-12-07 9:09 ` Xu Lu
2023-12-07 9:09 ` Xu Lu
2023-12-07 9:23 ` Clément Léger
2023-12-07 9:23 ` Clément Léger
2023-12-07 9:23 ` Clément Léger
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.