[kvm-unit-tests PATCH v5 0/5] riscv: sbi: Add support to test HSM extension

All of lore.kernel.org
 help / color / mirror / Atom feed

* [kvm-unit-tests PATCH v5 0/5] riscv: sbi: Add support to test HSM extension
@ 2024-09-21 10:08 ` James Raphael Tiovalen
  0 siblings, 0 replies; 18+ messages in thread
From: James Raphael Tiovalen @ 2024-09-21 10:08 UTC (permalink / raw)
  To: kvm-riscv

This patch series adds support for testing all 4 functions of the HSM
extension as defined in the RISC-V SBI specification. The first 4
patches add some helper routines to prepare for the HSM test, while
the last patch adds the actual test for the HSM extension.

v5:
- Addressed all of Andrew's comments.
- Added 2 new patches to clear on_cpu_info[cpu].func and to set the
  cpu_started mask, which are used to perform cleanup after running the
  HSM tests.
- Added some new tests to validate suspension on RV64 with the high
  bits set for suspend_type.
- Picked up the hartid_to_cpu rewrite patch from Andrew's branch.
- Moved the variables declared in riscv/sbi.c in patch 2 to group it
  together with the other HSM test variables declared in patch 5.

v4:
- Addressed all of Andrew's comments.
- Included the 2 patches from Andrew's branch that refactored some
  functions.
- Added timers to all of the waiting activities in the HSM tests.

v3:
- Addressed all of Andrew's comments.
- Split the report_prefix_pop patch into its own series.
- Added a new environment variable to specify the maximum number of
  CPUs supported by the SBI implementation.

v2:
- Addressed all of Andrew's comments.
- Added a new patch to add helper routines to clear multiple prefixes.
- Reworked the approach to test the HSM extension by using cpumask and
  on-cpus.

Andrew Jones (1):
  riscv: Rewrite hartid_to_cpu in assembly

James Raphael Tiovalen (4):
  riscv: sbi: Provide entry point for HSM tests
  lib/on-cpus: Add helper method to clear the function from on_cpu_info
  riscv: Add helper method to set cpu started mask
  riscv: sbi: Add tests for HSM extension

 riscv/Makefile          |   3 +-
 lib/riscv/asm/smp.h     |   2 +
 lib/on-cpus.h           |   1 +
 lib/on-cpus.c           |  11 +
 lib/riscv/asm-offsets.c |   5 +
 lib/riscv/setup.c       |  10 -
 lib/riscv/smp.c         |   8 +
 riscv/sbi-tests.h       |  10 +
 riscv/cstart.S          |  24 ++
 riscv/sbi-asm.S         |  71 +++++
 riscv/sbi.c             | 651 ++++++++++++++++++++++++++++++++++++++++
 11 files changed, 785 insertions(+), 11 deletions(-)
 create mode 100644 riscv/sbi-tests.h
 create mode 100644 riscv/sbi-asm.S

--
2.43.0



^ permalink raw reply	[flat|nested] 18+ messages in thread

* [kvm-unit-tests PATCH v5 0/5] riscv: sbi: Add support to test HSM extension
@ 2024-09-21 10:08 ` James Raphael Tiovalen
  0 siblings, 0 replies; 18+ messages in thread
From: James Raphael Tiovalen @ 2024-09-21 10:08 UTC (permalink / raw)
  To: kvm, kvm-riscv; +Cc: andrew.jones, atishp, cade.richard, James Raphael Tiovalen

This patch series adds support for testing all 4 functions of the HSM
extension as defined in the RISC-V SBI specification. The first 4
patches add some helper routines to prepare for the HSM test, while
the last patch adds the actual test for the HSM extension.

v5:
- Addressed all of Andrew's comments.
- Added 2 new patches to clear on_cpu_info[cpu].func and to set the
  cpu_started mask, which are used to perform cleanup after running the
  HSM tests.
- Added some new tests to validate suspension on RV64 with the high
  bits set for suspend_type.
- Picked up the hartid_to_cpu rewrite patch from Andrew's branch.
- Moved the variables declared in riscv/sbi.c in patch 2 to group it
  together with the other HSM test variables declared in patch 5.

v4:
- Addressed all of Andrew's comments.
- Included the 2 patches from Andrew's branch that refactored some
  functions.
- Added timers to all of the waiting activities in the HSM tests.

v3:
- Addressed all of Andrew's comments.
- Split the report_prefix_pop patch into its own series.
- Added a new environment variable to specify the maximum number of
  CPUs supported by the SBI implementation.

v2:
- Addressed all of Andrew's comments.
- Added a new patch to add helper routines to clear multiple prefixes.
- Reworked the approach to test the HSM extension by using cpumask and
  on-cpus.

Andrew Jones (1):
  riscv: Rewrite hartid_to_cpu in assembly

James Raphael Tiovalen (4):
  riscv: sbi: Provide entry point for HSM tests
  lib/on-cpus: Add helper method to clear the function from on_cpu_info
  riscv: Add helper method to set cpu started mask
  riscv: sbi: Add tests for HSM extension

 riscv/Makefile          |   3 +-
 lib/riscv/asm/smp.h     |   2 +
 lib/on-cpus.h           |   1 +
 lib/on-cpus.c           |  11 +
 lib/riscv/asm-offsets.c |   5 +
 lib/riscv/setup.c       |  10 -
 lib/riscv/smp.c         |   8 +
 riscv/sbi-tests.h       |  10 +
 riscv/cstart.S          |  24 ++
 riscv/sbi-asm.S         |  71 +++++
 riscv/sbi.c             | 651 ++++++++++++++++++++++++++++++++++++++++
 11 files changed, 785 insertions(+), 11 deletions(-)
 create mode 100644 riscv/sbi-tests.h
 create mode 100644 riscv/sbi-asm.S

--
2.43.0


^ permalink raw reply	[flat|nested] 18+ messages in thread

* [kvm-unit-tests PATCH v5 1/5] riscv: Rewrite hartid_to_cpu in assembly
  2024-09-21 10:08 ` James Raphael Tiovalen
@ 2024-09-21 10:08   ` James Raphael Tiovalen
  -1 siblings, 0 replies; 18+ messages in thread
From: James Raphael Tiovalen @ 2024-09-21 10:08 UTC (permalink / raw)
  To: kvm-riscv

From: Andrew Jones <andrew.jones@linux.dev>

Some SBI HSM tests run without a stack being setup so they can't
run C code. Those tests still need to know the corresponding cpuid
for the hartid on which they are running. Give those tests
hartid_to_cpu() by reimplementing it in assembly.

Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
Signed-off-by: James Raphael Tiovalen <jamestiotio@gmail.com>
---
 lib/riscv/asm-offsets.c |  5 +++++
 lib/riscv/setup.c       | 10 ----------
 riscv/cstart.S          | 24 ++++++++++++++++++++++++
 3 files changed, 29 insertions(+), 10 deletions(-)

diff --git a/lib/riscv/asm-offsets.c b/lib/riscv/asm-offsets.c
index a2a32438..6c511c14 100644
--- a/lib/riscv/asm-offsets.c
+++ b/lib/riscv/asm-offsets.c
@@ -1,6 +1,7 @@
 // SPDX-License-Identifier: GPL-2.0-only
 #include <kbuild.h>
 #include <elf.h>
+#include <asm/processor.h>
 #include <asm/ptrace.h>
 #include <asm/smp.h>
 
@@ -58,5 +59,9 @@ int main(void)
 	OFFSET(SECONDARY_FUNC, secondary_data, func);
 	DEFINE(SECONDARY_DATA_SIZE, sizeof(struct secondary_data));
 
+	OFFSET(THREAD_INFO_CPU, thread_info, cpu);
+	OFFSET(THREAD_INFO_HARTID, thread_info, hartid);
+	DEFINE(THREAD_INFO_SIZE, sizeof(struct thread_info));
+
 	return 0;
 }
diff --git a/lib/riscv/setup.c b/lib/riscv/setup.c
index 495db041..f347ad63 100644
--- a/lib/riscv/setup.c
+++ b/lib/riscv/setup.c
@@ -43,16 +43,6 @@ uint64_t timebase_frequency;
 
 static struct mem_region riscv_mem_regions[NR_MEM_REGIONS + 1];
 
-int hartid_to_cpu(unsigned long hartid)
-{
-	int cpu;
-
-	for_each_present_cpu(cpu)
-		if (cpus[cpu].hartid == hartid)
-			return cpu;
-	return -1;
-}
-
 static void cpu_set_fdt(int fdtnode __unused, u64 regval, void *info __unused)
 {
 	int cpu = nr_cpus++;
diff --git a/riscv/cstart.S b/riscv/cstart.S
index 8f269997..68717370 100644
--- a/riscv/cstart.S
+++ b/riscv/cstart.S
@@ -109,6 +109,30 @@ halt:
 1:	wfi
 	j	1b
 
+/*
+ * hartid_to_cpu
+ *   a0 is a hartid on entry
+ * Returns, in a0, the corresponding cpuid, or -1 if no
+ * thread_info struct with 'hartid' is found.
+ */
+.balign 4
+.global hartid_to_cpu
+hartid_to_cpu:
+	la	t0, cpus
+	la	t1, nr_cpus
+	lw	t1, 0(t1)
+	li	t2, 0
+1:	bne	t2, t1, 2f
+	li	a0, -1
+	ret
+2:	REG_L	t3, THREAD_INFO_HARTID(t0)
+	bne	a0, t3, 3f
+	lw	a0, THREAD_INFO_CPU(t0)
+	ret
+3:	addi	t0, t0, THREAD_INFO_SIZE
+	addi	t2, t2, 1
+	j	1b
+
 .balign 4
 .global secondary_entry
 secondary_entry:
-- 
2.43.0



^ permalink raw reply related	[flat|nested] 18+ messages in thread

* [kvm-unit-tests PATCH v5 1/5] riscv: Rewrite hartid_to_cpu in assembly
@ 2024-09-21 10:08   ` James Raphael Tiovalen
  0 siblings, 0 replies; 18+ messages in thread
From: James Raphael Tiovalen @ 2024-09-21 10:08 UTC (permalink / raw)
  To: kvm, kvm-riscv; +Cc: andrew.jones, atishp, cade.richard, James Raphael Tiovalen

From: Andrew Jones <andrew.jones@linux.dev>

Some SBI HSM tests run without a stack being setup so they can't
run C code. Those tests still need to know the corresponding cpuid
for the hartid on which they are running. Give those tests
hartid_to_cpu() by reimplementing it in assembly.

Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
Signed-off-by: James Raphael Tiovalen <jamestiotio@gmail.com>
---
 lib/riscv/asm-offsets.c |  5 +++++
 lib/riscv/setup.c       | 10 ----------
 riscv/cstart.S          | 24 ++++++++++++++++++++++++
 3 files changed, 29 insertions(+), 10 deletions(-)

diff --git a/lib/riscv/asm-offsets.c b/lib/riscv/asm-offsets.c
index a2a32438..6c511c14 100644
--- a/lib/riscv/asm-offsets.c
+++ b/lib/riscv/asm-offsets.c
@@ -1,6 +1,7 @@
 // SPDX-License-Identifier: GPL-2.0-only
 #include <kbuild.h>
 #include <elf.h>
+#include <asm/processor.h>
 #include <asm/ptrace.h>
 #include <asm/smp.h>
 
@@ -58,5 +59,9 @@ int main(void)
 	OFFSET(SECONDARY_FUNC, secondary_data, func);
 	DEFINE(SECONDARY_DATA_SIZE, sizeof(struct secondary_data));
 
+	OFFSET(THREAD_INFO_CPU, thread_info, cpu);
+	OFFSET(THREAD_INFO_HARTID, thread_info, hartid);
+	DEFINE(THREAD_INFO_SIZE, sizeof(struct thread_info));
+
 	return 0;
 }
diff --git a/lib/riscv/setup.c b/lib/riscv/setup.c
index 495db041..f347ad63 100644
--- a/lib/riscv/setup.c
+++ b/lib/riscv/setup.c
@@ -43,16 +43,6 @@ uint64_t timebase_frequency;
 
 static struct mem_region riscv_mem_regions[NR_MEM_REGIONS + 1];
 
-int hartid_to_cpu(unsigned long hartid)
-{
-	int cpu;
-
-	for_each_present_cpu(cpu)
-		if (cpus[cpu].hartid == hartid)
-			return cpu;
-	return -1;
-}
-
 static void cpu_set_fdt(int fdtnode __unused, u64 regval, void *info __unused)
 {
 	int cpu = nr_cpus++;
diff --git a/riscv/cstart.S b/riscv/cstart.S
index 8f269997..68717370 100644
--- a/riscv/cstart.S
+++ b/riscv/cstart.S
@@ -109,6 +109,30 @@ halt:
 1:	wfi
 	j	1b
 
+/*
+ * hartid_to_cpu
+ *   a0 is a hartid on entry
+ * Returns, in a0, the corresponding cpuid, or -1 if no
+ * thread_info struct with 'hartid' is found.
+ */
+.balign 4
+.global hartid_to_cpu
+hartid_to_cpu:
+	la	t0, cpus
+	la	t1, nr_cpus
+	lw	t1, 0(t1)
+	li	t2, 0
+1:	bne	t2, t1, 2f
+	li	a0, -1
+	ret
+2:	REG_L	t3, THREAD_INFO_HARTID(t0)
+	bne	a0, t3, 3f
+	lw	a0, THREAD_INFO_CPU(t0)
+	ret
+3:	addi	t0, t0, THREAD_INFO_SIZE
+	addi	t2, t2, 1
+	j	1b
+
 .balign 4
 .global secondary_entry
 secondary_entry:
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 18+ messages in thread

* [kvm-unit-tests PATCH v5 2/5] riscv: sbi: Provide entry point for HSM tests
  2024-09-21 10:08 ` James Raphael Tiovalen
@ 2024-09-21 10:08   ` James Raphael Tiovalen
  -1 siblings, 0 replies; 18+ messages in thread
From: James Raphael Tiovalen @ 2024-09-21 10:08 UTC (permalink / raw)
  To: kvm-riscv

The HSM tests will need to test HSM start and resumption from HSM
suspend. Provide an entry point written in assembly that doesn't
use a stack for this. Results of the test are written to global
per-hart arrays to be checked by the main SBI HSM test function. The
started/resumed hart does its checks and then just loops until it
gets a signal from the main SBI HSM test function to invoke HSM stop.

Signed-off-by: James Raphael Tiovalen <jamestiotio@gmail.com>
Co-developed-by: Andrew Jones <andrew.jones@linux.dev>
Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
---
 riscv/Makefile    |  3 +-
 riscv/sbi-tests.h | 10 +++++++
 riscv/sbi-asm.S   | 71 +++++++++++++++++++++++++++++++++++++++++++++++
 riscv/sbi.c       |  5 ++++
 4 files changed, 88 insertions(+), 1 deletion(-)
 create mode 100644 riscv/sbi-tests.h
 create mode 100644 riscv/sbi-asm.S

diff --git a/riscv/Makefile b/riscv/Makefile
index 2ee7c5bb..4676d262 100644
--- a/riscv/Makefile
+++ b/riscv/Makefile
@@ -43,6 +43,7 @@ cflatobjs += lib/riscv/timer.o
 ifeq ($(ARCH),riscv32)
 cflatobjs += lib/ldiv32.o
 endif
+cflatobjs += riscv/sbi-asm.o
 
 ########################################
 
@@ -80,7 +81,7 @@ CFLAGS += -mcmodel=medany
 CFLAGS += -std=gnu99
 CFLAGS += -ffreestanding
 CFLAGS += -O2
-CFLAGS += -I $(SRCDIR)/lib -I $(SRCDIR)/lib/libfdt -I lib
+CFLAGS += -I $(SRCDIR)/lib -I $(SRCDIR)/lib/libfdt -I lib -I $(SRCDIR)/riscv
 
 asm-offsets = lib/riscv/asm-offsets.h
 include $(SRCDIR)/scripts/asm-offsets.mak
diff --git a/riscv/sbi-tests.h b/riscv/sbi-tests.h
new file mode 100644
index 00000000..f5cc8635
--- /dev/null
+++ b/riscv/sbi-tests.h
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _RISCV_SBI_TESTS_H_
+#define _RISCV_SBI_TESTS_H_
+
+#define SBI_HSM_TEST_DONE	(1 << 0)
+#define SBI_HSM_TEST_HARTID_A1	(1 << 1)
+#define SBI_HSM_TEST_SATP	(1 << 2)
+#define SBI_HSM_TEST_SIE	(1 << 3)
+
+#endif /* _RISCV_SBI_TESTS_H_ */
diff --git a/riscv/sbi-asm.S b/riscv/sbi-asm.S
new file mode 100644
index 00000000..f165f9da
--- /dev/null
+++ b/riscv/sbi-asm.S
@@ -0,0 +1,71 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Helper assembly code routines for RISC-V SBI extension tests.
+ *
+ * Copyright (C) 2024, James Raphael Tiovalen <jamestiotio@gmail.com>
+ */
+#define __ASSEMBLY__
+#include <asm/csr.h>
+
+#include "sbi-tests.h"
+
+.section .text
+
+/*
+ * sbi_hsm_check
+ *   a0 and a1 are set by SBI HSM start/suspend
+ *   s1 is the address of the results array
+ * Doesn't return.
+ *
+ * This function is only called from HSM start and on resumption
+ * from HSM suspend which means we can do whatever we like with
+ * all registers. So, to avoid complicated register agreements with
+ * other assembly functions called, we just always use the saved
+ * registers for anything that should be maintained across calls.
+ */
+#define RESULTS_ARRAY	s1
+#define RESULTS_MAP	s2
+#define CPU_INDEX	s3
+.balign 4
+sbi_hsm_check:
+	li	RESULTS_MAP, 0
+	bne	a0, a1, 1f
+	ori	RESULTS_MAP, RESULTS_MAP, SBI_HSM_TEST_HARTID_A1
+1:	csrr	t0, CSR_SATP
+	bnez	t0, 2f
+	ori	RESULTS_MAP, RESULTS_MAP, SBI_HSM_TEST_SATP
+2:	csrr	t0, CSR_SSTATUS
+	andi	t0, t0, SR_SIE
+	bnez	t0, 3f
+	ori	RESULTS_MAP, RESULTS_MAP, SBI_HSM_TEST_SIE
+3:	call	hartid_to_cpu
+	mv	CPU_INDEX, a0
+	li	t0, -1
+	bne	CPU_INDEX, t0, 5f
+4:	pause
+	j	4b
+5:	ori	RESULTS_MAP, RESULTS_MAP, SBI_HSM_TEST_DONE
+	add	t0, RESULTS_ARRAY, CPU_INDEX
+	sb	RESULTS_MAP, 0(t0)
+	la	t1, sbi_hsm_stop_hart
+	add	t1, t1, CPU_INDEX
+6:	lb	t0, 0(t1)
+	pause
+	beqz	t0, 6b
+	li	a7, 0x48534d	/* SBI_EXT_HSM */
+	li	a6, 1		/* SBI_EXT_HSM_HART_STOP */
+	ecall
+7:	pause
+	j	7b
+
+.balign 4
+.global sbi_hsm_check_hart_start
+sbi_hsm_check_hart_start:
+	la	RESULTS_ARRAY, sbi_hsm_hart_start_checks
+	j	sbi_hsm_check
+
+.balign 4
+.global sbi_hsm_check_non_retentive_suspend
+sbi_hsm_check_non_retentive_suspend:
+	la	RESULTS_ARRAY, sbi_hsm_non_retentive_hart_suspend_checks
+	j	sbi_hsm_check
diff --git a/riscv/sbi.c b/riscv/sbi.c
index a7abc08c..b5147dee 100644
--- a/riscv/sbi.c
+++ b/riscv/sbi.c
@@ -18,6 +18,7 @@
 #include <asm/mmu.h>
 #include <asm/processor.h>
 #include <asm/sbi.h>
+#include <asm/setup.h>
 #include <asm/smp.h>
 #include <asm/timer.h>
 
@@ -429,6 +430,10 @@ static void check_dbcn(void)
 	report_prefix_popn(2);
 }
 
+unsigned char sbi_hsm_stop_hart[NR_CPUS];
+unsigned char sbi_hsm_hart_start_checks[NR_CPUS];
+unsigned char sbi_hsm_non_retentive_hart_suspend_checks[NR_CPUS];
+
 int main(int argc, char **argv)
 {
 	if (argc > 1 && !strcmp(argv[1], "-h")) {
-- 
2.43.0



^ permalink raw reply related	[flat|nested] 18+ messages in thread

* [kvm-unit-tests PATCH v5 2/5] riscv: sbi: Provide entry point for HSM tests
@ 2024-09-21 10:08   ` James Raphael Tiovalen
  0 siblings, 0 replies; 18+ messages in thread
From: James Raphael Tiovalen @ 2024-09-21 10:08 UTC (permalink / raw)
  To: kvm, kvm-riscv; +Cc: andrew.jones, atishp, cade.richard, James Raphael Tiovalen

The HSM tests will need to test HSM start and resumption from HSM
suspend. Provide an entry point written in assembly that doesn't
use a stack for this. Results of the test are written to global
per-hart arrays to be checked by the main SBI HSM test function. The
started/resumed hart does its checks and then just loops until it
gets a signal from the main SBI HSM test function to invoke HSM stop.

Signed-off-by: James Raphael Tiovalen <jamestiotio@gmail.com>
Co-developed-by: Andrew Jones <andrew.jones@linux.dev>
Signed-off-by: Andrew Jones <andrew.jones@linux.dev>
---
 riscv/Makefile    |  3 +-
 riscv/sbi-tests.h | 10 +++++++
 riscv/sbi-asm.S   | 71 +++++++++++++++++++++++++++++++++++++++++++++++
 riscv/sbi.c       |  5 ++++
 4 files changed, 88 insertions(+), 1 deletion(-)
 create mode 100644 riscv/sbi-tests.h
 create mode 100644 riscv/sbi-asm.S

diff --git a/riscv/Makefile b/riscv/Makefile
index 2ee7c5bb..4676d262 100644
--- a/riscv/Makefile
+++ b/riscv/Makefile
@@ -43,6 +43,7 @@ cflatobjs += lib/riscv/timer.o
 ifeq ($(ARCH),riscv32)
 cflatobjs += lib/ldiv32.o
 endif
+cflatobjs += riscv/sbi-asm.o
 
 ########################################
 
@@ -80,7 +81,7 @@ CFLAGS += -mcmodel=medany
 CFLAGS += -std=gnu99
 CFLAGS += -ffreestanding
 CFLAGS += -O2
-CFLAGS += -I $(SRCDIR)/lib -I $(SRCDIR)/lib/libfdt -I lib
+CFLAGS += -I $(SRCDIR)/lib -I $(SRCDIR)/lib/libfdt -I lib -I $(SRCDIR)/riscv
 
 asm-offsets = lib/riscv/asm-offsets.h
 include $(SRCDIR)/scripts/asm-offsets.mak
diff --git a/riscv/sbi-tests.h b/riscv/sbi-tests.h
new file mode 100644
index 00000000..f5cc8635
--- /dev/null
+++ b/riscv/sbi-tests.h
@@ -0,0 +1,10 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _RISCV_SBI_TESTS_H_
+#define _RISCV_SBI_TESTS_H_
+
+#define SBI_HSM_TEST_DONE	(1 << 0)
+#define SBI_HSM_TEST_HARTID_A1	(1 << 1)
+#define SBI_HSM_TEST_SATP	(1 << 2)
+#define SBI_HSM_TEST_SIE	(1 << 3)
+
+#endif /* _RISCV_SBI_TESTS_H_ */
diff --git a/riscv/sbi-asm.S b/riscv/sbi-asm.S
new file mode 100644
index 00000000..f165f9da
--- /dev/null
+++ b/riscv/sbi-asm.S
@@ -0,0 +1,71 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Helper assembly code routines for RISC-V SBI extension tests.
+ *
+ * Copyright (C) 2024, James Raphael Tiovalen <jamestiotio@gmail.com>
+ */
+#define __ASSEMBLY__
+#include <asm/csr.h>
+
+#include "sbi-tests.h"
+
+.section .text
+
+/*
+ * sbi_hsm_check
+ *   a0 and a1 are set by SBI HSM start/suspend
+ *   s1 is the address of the results array
+ * Doesn't return.
+ *
+ * This function is only called from HSM start and on resumption
+ * from HSM suspend which means we can do whatever we like with
+ * all registers. So, to avoid complicated register agreements with
+ * other assembly functions called, we just always use the saved
+ * registers for anything that should be maintained across calls.
+ */
+#define RESULTS_ARRAY	s1
+#define RESULTS_MAP	s2
+#define CPU_INDEX	s3
+.balign 4
+sbi_hsm_check:
+	li	RESULTS_MAP, 0
+	bne	a0, a1, 1f
+	ori	RESULTS_MAP, RESULTS_MAP, SBI_HSM_TEST_HARTID_A1
+1:	csrr	t0, CSR_SATP
+	bnez	t0, 2f
+	ori	RESULTS_MAP, RESULTS_MAP, SBI_HSM_TEST_SATP
+2:	csrr	t0, CSR_SSTATUS
+	andi	t0, t0, SR_SIE
+	bnez	t0, 3f
+	ori	RESULTS_MAP, RESULTS_MAP, SBI_HSM_TEST_SIE
+3:	call	hartid_to_cpu
+	mv	CPU_INDEX, a0
+	li	t0, -1
+	bne	CPU_INDEX, t0, 5f
+4:	pause
+	j	4b
+5:	ori	RESULTS_MAP, RESULTS_MAP, SBI_HSM_TEST_DONE
+	add	t0, RESULTS_ARRAY, CPU_INDEX
+	sb	RESULTS_MAP, 0(t0)
+	la	t1, sbi_hsm_stop_hart
+	add	t1, t1, CPU_INDEX
+6:	lb	t0, 0(t1)
+	pause
+	beqz	t0, 6b
+	li	a7, 0x48534d	/* SBI_EXT_HSM */
+	li	a6, 1		/* SBI_EXT_HSM_HART_STOP */
+	ecall
+7:	pause
+	j	7b
+
+.balign 4
+.global sbi_hsm_check_hart_start
+sbi_hsm_check_hart_start:
+	la	RESULTS_ARRAY, sbi_hsm_hart_start_checks
+	j	sbi_hsm_check
+
+.balign 4
+.global sbi_hsm_check_non_retentive_suspend
+sbi_hsm_check_non_retentive_suspend:
+	la	RESULTS_ARRAY, sbi_hsm_non_retentive_hart_suspend_checks
+	j	sbi_hsm_check
diff --git a/riscv/sbi.c b/riscv/sbi.c
index a7abc08c..b5147dee 100644
--- a/riscv/sbi.c
+++ b/riscv/sbi.c
@@ -18,6 +18,7 @@
 #include <asm/mmu.h>
 #include <asm/processor.h>
 #include <asm/sbi.h>
+#include <asm/setup.h>
 #include <asm/smp.h>
 #include <asm/timer.h>
 
@@ -429,6 +430,10 @@ static void check_dbcn(void)
 	report_prefix_popn(2);
 }
 
+unsigned char sbi_hsm_stop_hart[NR_CPUS];
+unsigned char sbi_hsm_hart_start_checks[NR_CPUS];
+unsigned char sbi_hsm_non_retentive_hart_suspend_checks[NR_CPUS];
+
 int main(int argc, char **argv)
 {
 	if (argc > 1 && !strcmp(argv[1], "-h")) {
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 18+ messages in thread

* [kvm-unit-tests PATCH v5 3/5] lib/on-cpus: Add helper method to clear the function from on_cpu_info
  2024-09-21 10:08 ` James Raphael Tiovalen
@ 2024-09-21 10:08   ` James Raphael Tiovalen
  -1 siblings, 0 replies; 18+ messages in thread
From: James Raphael Tiovalen @ 2024-09-21 10:08 UTC (permalink / raw)
  To: kvm-riscv

When a CPU abruptly stops during some test, the CPU will not have the
chance to go back to the do_idle() loop and set the
on_cpu_info[cpu].func variable to NULL. Add a helper method for some
test manager CPU to clear this function. This would re-enable
on_cpu_async and allow future tests to use the on-cpus API again.

Signed-off-by: James Raphael Tiovalen <jamestiotio@gmail.com>
---
 lib/on-cpus.h |  1 +
 lib/on-cpus.c | 11 +++++++++++
 2 files changed, 12 insertions(+)

diff --git a/lib/on-cpus.h b/lib/on-cpus.h
index 4bc6236d..497ff9d1 100644
--- a/lib/on-cpus.h
+++ b/lib/on-cpus.h
@@ -13,5 +13,6 @@ void on_cpu(int cpu, void (*func)(void *data), void *data);
 void on_cpus(void (*func)(void *data), void *data);
 void on_cpumask_async(const cpumask_t *mask, void (*func)(void *data), void *data);
 void on_cpumask(const cpumask_t *mask, void (*func)(void *data), void *data);
+void on_cpu_clear_func(int cpu);
 
 #endif /* _ON_CPUS_H_ */
diff --git a/lib/on-cpus.c b/lib/on-cpus.c
index 89214933..cc73690a 100644
--- a/lib/on-cpus.c
+++ b/lib/on-cpus.c
@@ -171,3 +171,14 @@ void on_cpus(void (*func)(void *data), void *data)
 {
 	on_cpumask(&cpu_present_mask, func, data);
 }
+
+void on_cpu_clear_func(int cpu)
+{
+	for (;;) {
+		if (get_on_cpu_info(cpu))
+			break;
+	}
+
+	on_cpu_info[cpu].func = NULL;
+	put_on_cpu_info(cpu);
+}
-- 
2.43.0



^ permalink raw reply related	[flat|nested] 18+ messages in thread

* [kvm-unit-tests PATCH v5 3/5] lib/on-cpus: Add helper method to clear the function from on_cpu_info
@ 2024-09-21 10:08   ` James Raphael Tiovalen
  0 siblings, 0 replies; 18+ messages in thread
From: James Raphael Tiovalen @ 2024-09-21 10:08 UTC (permalink / raw)
  To: kvm, kvm-riscv; +Cc: andrew.jones, atishp, cade.richard, James Raphael Tiovalen

When a CPU abruptly stops during some test, the CPU will not have the
chance to go back to the do_idle() loop and set the
on_cpu_info[cpu].func variable to NULL. Add a helper method for some
test manager CPU to clear this function. This would re-enable
on_cpu_async and allow future tests to use the on-cpus API again.

Signed-off-by: James Raphael Tiovalen <jamestiotio@gmail.com>
---
 lib/on-cpus.h |  1 +
 lib/on-cpus.c | 11 +++++++++++
 2 files changed, 12 insertions(+)

diff --git a/lib/on-cpus.h b/lib/on-cpus.h
index 4bc6236d..497ff9d1 100644
--- a/lib/on-cpus.h
+++ b/lib/on-cpus.h
@@ -13,5 +13,6 @@ void on_cpu(int cpu, void (*func)(void *data), void *data);
 void on_cpus(void (*func)(void *data), void *data);
 void on_cpumask_async(const cpumask_t *mask, void (*func)(void *data), void *data);
 void on_cpumask(const cpumask_t *mask, void (*func)(void *data), void *data);
+void on_cpu_clear_func(int cpu);
 
 #endif /* _ON_CPUS_H_ */
diff --git a/lib/on-cpus.c b/lib/on-cpus.c
index 89214933..cc73690a 100644
--- a/lib/on-cpus.c
+++ b/lib/on-cpus.c
@@ -171,3 +171,14 @@ void on_cpus(void (*func)(void *data), void *data)
 {
 	on_cpumask(&cpu_present_mask, func, data);
 }
+
+void on_cpu_clear_func(int cpu)
+{
+	for (;;) {
+		if (get_on_cpu_info(cpu))
+			break;
+	}
+
+	on_cpu_info[cpu].func = NULL;
+	put_on_cpu_info(cpu);
+}
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 18+ messages in thread

* [kvm-unit-tests PATCH v5 4/5] riscv: Add helper method to set cpu started mask
  2024-09-21 10:08 ` James Raphael Tiovalen
@ 2024-09-21 10:08   ` James Raphael Tiovalen
  -1 siblings, 0 replies; 18+ messages in thread
From: James Raphael Tiovalen @ 2024-09-21 10:08 UTC (permalink / raw)
  To: kvm-riscv

When a CPU abruptly stops during the RISC-V SBI hart stop tests, it is
considered to be offline. As such, it should be removed from the
cpu_started mask so that future tests can initiate another
smp_boot_secondary. Add a helper method to allow the RISC-V SBI
boot hart to remove a dead CPU from the mask.

Signed-off-by: James Raphael Tiovalen <jamestiotio@gmail.com>
---
 lib/riscv/asm/smp.h | 2 ++
 lib/riscv/smp.c     | 8 ++++++++
 2 files changed, 10 insertions(+)

diff --git a/lib/riscv/asm/smp.h b/lib/riscv/asm/smp.h
index b3ead4e8..5d379a7a 100644
--- a/lib/riscv/asm/smp.h
+++ b/lib/riscv/asm/smp.h
@@ -26,4 +26,6 @@ secondary_func_t secondary_cinit(struct secondary_data *data);
 void smp_boot_secondary(int cpu, void (*func)(void));
 void smp_boot_secondary_nofail(int cpu, void (*func)(void));
 
+void set_cpu_started(int cpu, bool started);
+
 #endif /* _ASMRISCV_SMP_H_ */
diff --git a/lib/riscv/smp.c b/lib/riscv/smp.c
index eb7061ab..eb7cfb72 100644
--- a/lib/riscv/smp.c
+++ b/lib/riscv/smp.c
@@ -74,3 +74,11 @@ void smp_boot_secondary_nofail(int cpu, void (*func)(void))
 	while (!cpu_online(cpu))
 		smp_wait_for_event();
 }
+
+void set_cpu_started(int cpu, bool started)
+{
+	if (started)
+		cpumask_set_cpu(cpu, &cpu_started);
+	else
+		cpumask_clear_cpu(cpu, &cpu_started);
+}
-- 
2.43.0



^ permalink raw reply related	[flat|nested] 18+ messages in thread

* [kvm-unit-tests PATCH v5 4/5] riscv: Add helper method to set cpu started mask
@ 2024-09-21 10:08   ` James Raphael Tiovalen
  0 siblings, 0 replies; 18+ messages in thread
From: James Raphael Tiovalen @ 2024-09-21 10:08 UTC (permalink / raw)
  To: kvm, kvm-riscv; +Cc: andrew.jones, atishp, cade.richard, James Raphael Tiovalen

When a CPU abruptly stops during the RISC-V SBI hart stop tests, it is
considered to be offline. As such, it should be removed from the
cpu_started mask so that future tests can initiate another
smp_boot_secondary. Add a helper method to allow the RISC-V SBI
boot hart to remove a dead CPU from the mask.

Signed-off-by: James Raphael Tiovalen <jamestiotio@gmail.com>
---
 lib/riscv/asm/smp.h | 2 ++
 lib/riscv/smp.c     | 8 ++++++++
 2 files changed, 10 insertions(+)

diff --git a/lib/riscv/asm/smp.h b/lib/riscv/asm/smp.h
index b3ead4e8..5d379a7a 100644
--- a/lib/riscv/asm/smp.h
+++ b/lib/riscv/asm/smp.h
@@ -26,4 +26,6 @@ secondary_func_t secondary_cinit(struct secondary_data *data);
 void smp_boot_secondary(int cpu, void (*func)(void));
 void smp_boot_secondary_nofail(int cpu, void (*func)(void));
 
+void set_cpu_started(int cpu, bool started);
+
 #endif /* _ASMRISCV_SMP_H_ */
diff --git a/lib/riscv/smp.c b/lib/riscv/smp.c
index eb7061ab..eb7cfb72 100644
--- a/lib/riscv/smp.c
+++ b/lib/riscv/smp.c
@@ -74,3 +74,11 @@ void smp_boot_secondary_nofail(int cpu, void (*func)(void))
 	while (!cpu_online(cpu))
 		smp_wait_for_event();
 }
+
+void set_cpu_started(int cpu, bool started)
+{
+	if (started)
+		cpumask_set_cpu(cpu, &cpu_started);
+	else
+		cpumask_clear_cpu(cpu, &cpu_started);
+}
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 18+ messages in thread

* [kvm-unit-tests PATCH v5 5/5] riscv: sbi: Add tests for HSM extension
  2024-09-21 10:08 ` James Raphael Tiovalen
@ 2024-09-21 10:08   ` James Raphael Tiovalen
  -1 siblings, 0 replies; 18+ messages in thread
From: James Raphael Tiovalen @ 2024-09-21 10:08 UTC (permalink / raw)
  To: kvm-riscv

Add some tests for all of the HSM extension functions. These tests
ensure that the HSM extension functions follow the behavior as described
in the SBI specification.

Signed-off-by: James Raphael Tiovalen <jamestiotio@gmail.com>
---
 riscv/sbi.c | 648 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 648 insertions(+)

diff --git a/riscv/sbi.c b/riscv/sbi.c
index b5147dee..cd1ed95b 100644
--- a/riscv/sbi.c
+++ b/riscv/sbi.c
@@ -6,6 +6,8 @@
  */
 #include <libcflat.h>
 #include <alloc_page.h>
+#include <cpumask.h>
+#include <on-cpus.h>
 #include <stdlib.h>
 #include <string.h>
 #include <limits.h>
@@ -16,12 +18,15 @@
 #include <asm/delay.h>
 #include <asm/io.h>
 #include <asm/mmu.h>
+#include <asm/page.h>
 #include <asm/processor.h>
 #include <asm/sbi.h>
 #include <asm/setup.h>
 #include <asm/smp.h>
 #include <asm/timer.h>
 
+#include "sbi-tests.h"
+
 #define	HIGH_ADDR_BOUNDARY	((phys_addr_t)1 << 32)
 
 static void help(void)
@@ -47,6 +52,11 @@ static struct sbiret sbi_dbcn_write_byte(uint8_t byte)
 	return sbi_ecall(SBI_EXT_DBCN, SBI_EXT_DBCN_CONSOLE_WRITE_BYTE, byte, 0, 0, 0, 0, 0);
 }
 
+static struct sbiret sbi_hart_suspend(uint32_t suspend_type, unsigned long resume_addr, unsigned long opaque)
+{
+	return sbi_ecall(SBI_EXT_HSM, SBI_EXT_HSM_HART_SUSPEND, suspend_type, resume_addr, opaque, 0, 0, 0);
+}
+
 static void split_phys_addr(phys_addr_t paddr, unsigned long *hi, unsigned long *lo)
 {
 	*lo = (unsigned long)paddr;
@@ -433,6 +443,643 @@ static void check_dbcn(void)
 unsigned char sbi_hsm_stop_hart[NR_CPUS];
 unsigned char sbi_hsm_hart_start_checks[NR_CPUS];
 unsigned char sbi_hsm_non_retentive_hart_suspend_checks[NR_CPUS];
+cpumask_t sbi_hsm_started_hart_checks;
+static bool sbi_hsm_invalid_hartid_check;
+static bool sbi_hsm_timer_fired;
+extern void sbi_hsm_check_hart_start(void);
+extern void sbi_hsm_check_non_retentive_suspend(void);
+
+static void hsm_timer_irq_handler(struct pt_regs *regs)
+{
+	sbi_hsm_timer_fired = true;
+	timer_stop();
+}
+
+static void hsm_timer_setup(void)
+{
+	install_irq_handler(IRQ_S_TIMER, hsm_timer_irq_handler);
+	local_irq_enable();
+	timer_irq_enable();
+}
+
+static void hsm_timer_teardown(void)
+{
+	timer_irq_disable();
+	local_irq_disable();
+	install_irq_handler(IRQ_S_TIMER, NULL);
+}
+
+static void hart_empty_fn(void *data) {}
+
+static void hart_execute(void *data)
+{
+	struct sbiret ret;
+	unsigned long hartid = current_thread_info()->hartid;
+	int me = smp_processor_id();
+
+	ret = sbi_hart_start(hartid, virt_to_phys(&hart_empty_fn), 0);
+
+	if (ret.error == SBI_ERR_ALREADY_AVAILABLE)
+		cpumask_set_cpu(me, &sbi_hsm_started_hart_checks);
+}
+
+static void hart_start_invalid_hartid(void *data)
+{
+	struct sbiret ret;
+
+	ret = sbi_hart_start(ULONG_MAX, virt_to_phys(&hart_empty_fn), 0);
+
+	if (ret.error == SBI_ERR_INVALID_PARAM)
+		sbi_hsm_invalid_hartid_check = true;
+}
+
+static void hart_stop(void *data)
+{
+	unsigned long hartid = current_thread_info()->hartid;
+	struct sbiret ret = sbi_hart_stop();
+
+	report_fail("failed to stop hart %ld (error=%ld)", hartid, ret.error);
+}
+
+static void hart_retentive_suspend(void *data)
+{
+	unsigned long hartid = current_thread_info()->hartid;
+	struct sbiret ret = sbi_hart_suspend(SBI_EXT_HSM_HART_SUSPEND_RETENTIVE, 0, 0);
+
+	if (ret.error)
+		report_fail("failed to retentive suspend hart %ld (error=%ld)", hartid, ret.error);
+}
+
+static void hart_non_retentive_suspend(void *data)
+{
+	unsigned long hartid = current_thread_info()->hartid;
+
+	/* Set opaque as hartid so that we can check a0 == a1, ensuring that a0 is hartid and a1 is opaque */
+	struct sbiret ret = sbi_hart_suspend(SBI_EXT_HSM_HART_SUSPEND_NON_RETENTIVE,
+					     virt_to_phys(&sbi_hsm_check_non_retentive_suspend), hartid);
+
+	report_fail("failed to non-retentive suspend hart %ld (error=%ld)", hartid, ret.error);
+}
+
+static void hart_retentive_suspend_with_msb_set(void *data)
+{
+	unsigned long hartid = current_thread_info()->hartid;
+	unsigned long suspend_type = SBI_EXT_HSM_HART_SUSPEND_RETENTIVE | (_AC(1, UL) << (__riscv_xlen - 1));
+	struct sbiret ret = sbi_ecall(SBI_EXT_HSM, SBI_EXT_HSM_HART_SUSPEND, suspend_type, 0, 0, 0, 0, 0);
+
+	if (ret.error)
+		report_fail("failed to retentive suspend hart %ld with MSB set (error=%ld)", hartid, ret.error);
+}
+
+static void hart_non_retentive_suspend_with_msb_set(void *data)
+{
+	unsigned long hartid = current_thread_info()->hartid;
+	unsigned long suspend_type = SBI_EXT_HSM_HART_SUSPEND_NON_RETENTIVE | (_AC(1, UL) << (__riscv_xlen - 1));
+
+	/* Set opaque as hartid so that we can check a0 == a1, ensuring that a0 is hartid and a1 is opaque */
+	struct sbiret ret = sbi_ecall(SBI_EXT_HSM, SBI_EXT_HSM_HART_SUSPEND, suspend_type,
+				      virt_to_phys(&sbi_hsm_check_non_retentive_suspend), hartid, 0, 0, 0);
+
+	report_fail("failed to non-retentive suspend hart %ld with MSB set (error=%ld)", hartid, ret.error);
+}
+
+static bool hart_wait_on_status(unsigned long hartid, enum sbi_ext_hsm_sid status, unsigned long duration)
+{
+	struct sbiret ret;
+
+	sbi_hsm_timer_fired = false;
+	timer_start(duration);
+
+	ret = sbi_hart_get_status(hartid);
+
+	while (!ret.error && ret.value == status && !sbi_hsm_timer_fired) {
+		cpu_relax();
+		ret = sbi_hart_get_status(hartid);
+	}
+
+	timer_stop();
+
+	if (sbi_hsm_timer_fired)
+		report_info("timer fired while waiting on status %u for hart %ld", status, hartid);
+	else if (ret.error)
+		report_fail("got %ld while waiting on status %u for hart %ld\n", ret.error, status, hartid);
+
+	return sbi_hsm_timer_fired || ret.error;
+}
+
+static void check_hsm(void)
+{
+	struct sbiret ret;
+	unsigned long hartid;
+	cpumask_t secondary_cpus_mask, hsm_start, hsm_stop, hsm_suspend, hsm_resume, hsm_check;
+	bool ipi_unavailable = false;
+	bool suspend_with_msb = false, resume_with_msb = false, check_with_msb = false, stop_with_msb = false;
+	int cpu, me = smp_processor_id();
+	int max_cpus = getenv("SBI_MAX_CPUS") ? strtol(getenv("SBI_MAX_CPUS"), NULL, 0) : nr_cpus;
+	unsigned long hsm_timer_duration = getenv("SBI_HSM_TIMER_DURATION")
+					 ? strtol(getenv("SBI_HSM_TIMER_DURATION"), NULL, 0) : 200000;
+
+	max_cpus = MIN(max_cpus, nr_cpus);
+
+	report_prefix_push("hsm");
+
+	if (!sbi_probe(SBI_EXT_HSM)) {
+		report_skip("hsm extension not available");
+		report_prefix_pop();
+		return;
+	}
+
+	report_prefix_push("hart_get_status");
+
+	hartid = current_thread_info()->hartid;
+	ret = sbi_hart_get_status(hartid);
+
+	if (ret.error) {
+		report_fail("failed to get status of current hart (error=%ld)", ret.error);
+		report_prefix_popn(2);
+		return;
+	} else if (ret.value != SBI_EXT_HSM_STARTED) {
+		report_fail("current hart is not started (ret.value=%ld)", ret.value);
+		report_prefix_popn(2);
+		return;
+	}
+
+	report_pass("status of current hart is started");
+
+	for_each_present_cpu(cpu) {
+		if (sbi_hart_get_status(cpus[cpu].hartid).error == SBI_ERR_INVALID_PARAM)
+			set_cpu_present(cpu, false);
+	}
+
+	report(cpumask_weight(&cpu_present_mask) == nr_cpus, "all present harts have valid hartids");
+
+	report_prefix_pop();
+
+	if (max_cpus < 2) {
+		report_skip("no other cpus to run the remaining hsm tests on");
+		report_prefix_pop();
+		return;
+	}
+
+	report_prefix_push("hart_start");
+
+	cpumask_copy(&secondary_cpus_mask, &cpu_present_mask);
+	cpumask_clear_cpu(me, &secondary_cpus_mask);
+	hsm_timer_setup();
+
+	cpumask_clear(&hsm_start);
+	cpumask_clear(&hsm_check);
+
+	for_each_cpu(cpu, &secondary_cpus_mask) {
+		hartid = cpus[cpu].hartid;
+		/* Set opaque as hartid so that we can check a0 == a1, ensuring that a0 is hartid and a1 is opaque */
+		ret = sbi_hart_start(hartid, virt_to_phys(&sbi_hsm_check_hart_start), hartid);
+		if (ret.error) {
+			report_fail("failed to start test hart %ld (error=%ld)", hartid, ret.error);
+			continue;
+		}
+
+		if (hart_wait_on_status(hartid, SBI_EXT_HSM_STOPPED, hsm_timer_duration))
+			continue;
+		if (hart_wait_on_status(hartid, SBI_EXT_HSM_START_PENDING, hsm_timer_duration))
+			continue;
+
+		ret = sbi_hart_get_status(hartid);
+		if (ret.error) {
+			report_info("hart %ld get status failed (error=%ld)", hartid, ret.error);
+			continue;
+		} else if (ret.value != SBI_EXT_HSM_STARTED) {
+			report_info("hart %ld status is not 'started' (ret.value=%ld)", hartid, ret.value);
+			continue;
+		} else {
+			cpumask_set_cpu(cpu, &hsm_start);
+		}
+
+		sbi_hsm_timer_fired = false;
+		timer_start(hsm_timer_duration);
+
+		while (!(READ_ONCE(sbi_hsm_hart_start_checks[cpu]) & SBI_HSM_TEST_DONE) && !sbi_hsm_timer_fired)
+			cpu_relax();
+
+		timer_stop();
+
+		if (sbi_hsm_timer_fired) {
+			report_info("hsm timer fired before hart %ld is done with start checks", hartid);
+			continue;
+		}
+
+		if (!(sbi_hsm_hart_start_checks[cpu] & SBI_HSM_TEST_SATP))
+			report_info("satp is not zero for test hart %ld", hartid);
+		else if (!(sbi_hsm_hart_start_checks[cpu] & SBI_HSM_TEST_SIE))
+			report_info("sstatus.SIE is not zero for test hart %ld", hartid);
+		else if (!(sbi_hsm_hart_start_checks[cpu] & SBI_HSM_TEST_HARTID_A1))
+			report_info("either a0 or a1 is not hartid for test hart %ld", hartid);
+		else
+			cpumask_set_cpu(cpu, &hsm_check);
+	}
+
+	report(cpumask_weight(&hsm_start) == max_cpus - 1, "all secondary harts started");
+	report(cpumask_weight(&hsm_check) == max_cpus - 1,
+	       "all secondary harts have expected register values after hart start");
+
+	report_prefix_pop();
+
+	report_prefix_push("hart_stop");
+
+	memset(sbi_hsm_stop_hart, 1, sizeof(sbi_hsm_stop_hart));
+
+	cpumask_clear(&hsm_stop);
+
+	for_each_cpu(cpu, &secondary_cpus_mask) {
+		hartid = cpus[cpu].hartid;
+		if (hart_wait_on_status(hartid, SBI_EXT_HSM_STARTED, hsm_timer_duration))
+			continue;
+		if (hart_wait_on_status(hartid, SBI_EXT_HSM_STOP_PENDING, hsm_timer_duration))
+			continue;
+
+		ret = sbi_hart_get_status(hartid);
+		if (ret.error)
+			report_info("hart %ld get status failed (error=%ld)", hartid, ret.error);
+		else if (ret.value != SBI_EXT_HSM_STOPPED)
+			report_info("hart %ld status is not 'stopped' (ret.value=%ld)", hartid, ret.value);
+		else
+			cpumask_set_cpu(cpu, &hsm_stop);
+	}
+
+	report(cpumask_weight(&hsm_stop) == max_cpus - 1, "all secondary harts stopped");
+
+	/* Reset the stop flags so that we can reuse them after suspension tests */
+	memset(sbi_hsm_stop_hart, 0, sizeof(sbi_hsm_stop_hart));
+
+	report_prefix_pop();
+
+	report_prefix_push("hart_start");
+
+	/* Select just one secondary cpu to run the invalid hartid test */
+	on_cpu(cpumask_next(-1, &secondary_cpus_mask), hart_start_invalid_hartid, NULL);
+
+	report(sbi_hsm_invalid_hartid_check, "secondary hart refuse to start with invalid hartid");
+
+	on_cpumask_async(&secondary_cpus_mask, hart_execute, NULL);
+
+	cpumask_clear(&hsm_start);
+
+	for_each_cpu(cpu, &secondary_cpus_mask) {
+		hartid = cpus[cpu].hartid;
+		if (hart_wait_on_status(hartid, SBI_EXT_HSM_STOPPED, hsm_timer_duration))
+			continue;
+		if (hart_wait_on_status(hartid, SBI_EXT_HSM_START_PENDING, hsm_timer_duration))
+			continue;
+
+		ret = sbi_hart_get_status(hartid);
+		if (ret.error)
+			report_info("hart %ld get status failed (error=%ld)", hartid, ret.error);
+		else if (ret.value != SBI_EXT_HSM_STARTED)
+			report_info("hart %ld status is not 'started' (ret.value=%ld)", hartid, ret.value);
+		else
+			cpumask_set_cpu(cpu, &hsm_start);
+	}
+
+	report(cpumask_weight(&hsm_start) == max_cpus - 1, "all secondary harts started");
+
+	sbi_hsm_timer_fired = false;
+	timer_start(hsm_timer_duration);
+
+	while (cpumask_weight(&cpu_idle_mask) != max_cpus - 1 && !sbi_hsm_timer_fired)
+		cpu_relax();
+
+	timer_stop();
+
+	if (sbi_hsm_timer_fired)
+		report_info("hsm timer fired before all secondary harts started");
+
+	report(cpumask_weight(&cpu_idle_mask) == max_cpus - 1,
+	       "all secondary harts successfully executed code after start");
+	report(cpumask_weight(&cpu_online_mask) == max_cpus, "all secondary harts online");
+	report(cpumask_weight(&sbi_hsm_started_hart_checks) == max_cpus - 1,
+	       "all secondary harts are already started");
+
+	report_prefix_pop();
+
+	report_prefix_push("hart_suspend");
+
+	if (!sbi_probe(SBI_EXT_IPI)) {
+		report_skip("skipping suspension tests since ipi extension is unavailable");
+		report_prefix_pop();
+		ipi_unavailable = true;
+		goto sbi_hsm_hart_stop_tests;
+	}
+
+	on_cpumask_async(&secondary_cpus_mask, hart_retentive_suspend, NULL);
+
+	cpumask_clear(&hsm_suspend);
+
+	for_each_cpu(cpu, &secondary_cpus_mask) {
+		hartid = cpus[cpu].hartid;
+		if (hart_wait_on_status(hartid, SBI_EXT_HSM_STARTED, hsm_timer_duration))
+			continue;
+		if (hart_wait_on_status(hartid, SBI_EXT_HSM_SUSPEND_PENDING, hsm_timer_duration))
+			continue;
+
+		ret = sbi_hart_get_status(hartid);
+		if (ret.error)
+			report_info("hart %ld get status failed (error=%ld)", hartid, ret.error);
+		else if (ret.value != SBI_EXT_HSM_SUSPENDED)
+			report_info("hart %ld status is not 'suspended' (ret.value=%ld)", hartid, ret.value);
+		else
+			cpumask_set_cpu(cpu, &hsm_suspend);
+	}
+
+	report(cpumask_weight(&hsm_suspend) == max_cpus - 1, "all secondary harts retentive suspended");
+
+	/* Ignore the return value since we check the status of each hart anyway */
+	sbi_send_ipi_cpumask(&secondary_cpus_mask);
+
+	cpumask_clear(&hsm_resume);
+
+	for_each_cpu(cpu, &secondary_cpus_mask) {
+		hartid = cpus[cpu].hartid;
+		if (hart_wait_on_status(hartid, SBI_EXT_HSM_SUSPENDED, hsm_timer_duration))
+			continue;
+		if (hart_wait_on_status(hartid, SBI_EXT_HSM_RESUME_PENDING, hsm_timer_duration))
+			continue;
+
+		ret = sbi_hart_get_status(hartid);
+		if (ret.error)
+			report_info("hart %ld get status failed (error=%ld)", hartid, ret.error);
+		else if (ret.value != SBI_EXT_HSM_STARTED)
+			report_info("hart %ld status is not 'started' (ret.value=%ld)", hartid, ret.value);
+		else
+			cpumask_set_cpu(cpu, &hsm_resume);
+	}
+
+	report(cpumask_weight(&hsm_resume) == max_cpus - 1, "all secondary harts retentive resumed");
+
+	sbi_hsm_timer_fired = false;
+	timer_start(hsm_timer_duration);
+
+	while (cpumask_weight(&cpu_idle_mask) != max_cpus - 1 && !sbi_hsm_timer_fired)
+		cpu_relax();
+
+	timer_stop();
+
+	if (sbi_hsm_timer_fired)
+		report_info("hsm timer fired before all secondary harts retentive resumed");
+
+	report(cpumask_weight(&cpu_idle_mask) == max_cpus - 1,
+	       "all secondary harts successfully executed code after retentive suspend");
+	report(cpumask_weight(&cpu_online_mask) == max_cpus,
+	       "all secondary harts online");
+
+	on_cpumask_async(&secondary_cpus_mask, hart_non_retentive_suspend, NULL);
+
+	cpumask_clear(&hsm_suspend);
+
+	for_each_cpu(cpu, &secondary_cpus_mask) {
+		hartid = cpus[cpu].hartid;
+		if (hart_wait_on_status(hartid, SBI_EXT_HSM_STARTED, hsm_timer_duration))
+			continue;
+		if (hart_wait_on_status(hartid, SBI_EXT_HSM_SUSPEND_PENDING, hsm_timer_duration))
+			continue;
+
+		ret = sbi_hart_get_status(hartid);
+		if (ret.error)
+			report_info("hart %ld get status failed (error=%ld)", hartid, ret.error);
+		else if (ret.value != SBI_EXT_HSM_SUSPENDED)
+			report_info("hart %ld status is not 'suspended' (ret.value=%ld)", hartid, ret.value);
+		else
+			cpumask_set_cpu(cpu, &hsm_suspend);
+	}
+
+	report(cpumask_weight(&hsm_suspend) == max_cpus - 1, "all secondary harts non-retentive suspended");
+
+	/* Ignore the return value since we check the status of each hart anyway */
+	sbi_send_ipi_cpumask(&secondary_cpus_mask);
+
+	cpumask_clear(&hsm_resume);
+	cpumask_clear(&hsm_check);
+
+	for_each_cpu(cpu, &secondary_cpus_mask) {
+		hartid = cpus[cpu].hartid;
+		if (hart_wait_on_status(hartid, SBI_EXT_HSM_SUSPENDED, hsm_timer_duration))
+			continue;
+		if (hart_wait_on_status(hartid, SBI_EXT_HSM_RESUME_PENDING, hsm_timer_duration))
+			continue;
+
+		ret = sbi_hart_get_status(hartid);
+		if (ret.error) {
+			report_info("hart %ld get status failed (error=%ld)", hartid, ret.error);
+			continue;
+		} else if (ret.value != SBI_EXT_HSM_STARTED) {
+			report_info("hart %ld status is not 'started' (ret.value=%ld)", hartid, ret.value);
+			continue;
+		} else {
+			cpumask_set_cpu(cpu, &hsm_resume);
+		}
+
+		sbi_hsm_timer_fired = false;
+		timer_start(hsm_timer_duration);
+
+		while (!((READ_ONCE(sbi_hsm_non_retentive_hart_suspend_checks[cpu])) & SBI_HSM_TEST_DONE)
+			&& !sbi_hsm_timer_fired)
+			cpu_relax();
+
+		timer_stop();
+
+		if (sbi_hsm_timer_fired) {
+			report_info("hsm timer fired before hart %ld is done with non-retentive resume checks",
+				    hartid);
+			continue;
+		}
+
+		if (!(sbi_hsm_non_retentive_hart_suspend_checks[cpu] & SBI_HSM_TEST_SATP))
+			report_info("satp is not zero for test hart %ld", hartid);
+		else if (!(sbi_hsm_non_retentive_hart_suspend_checks[cpu] & SBI_HSM_TEST_SIE))
+			report_info("sstatus.SIE is not zero for test hart %ld", hartid);
+		else if (!(sbi_hsm_non_retentive_hart_suspend_checks[cpu] & SBI_HSM_TEST_HARTID_A1))
+			report_info("either a0 or a1 is not hartid for test hart %ld", hartid);
+		else
+			cpumask_set_cpu(cpu, &hsm_check);
+	}
+
+	report(cpumask_weight(&hsm_resume) == max_cpus - 1, "all secondary harts non-retentive resumed");
+	report(cpumask_weight(&hsm_check) == max_cpus - 1,
+	       "all secondary harts have expected register values after non-retentive resume");
+
+	report_prefix_pop();
+
+sbi_hsm_hart_stop_tests:
+	report_prefix_push("hart_stop");
+
+	if (ipi_unavailable)
+		on_cpumask_async(&secondary_cpus_mask, hart_stop, NULL);
+	else
+		memset(sbi_hsm_stop_hart, 1, sizeof(sbi_hsm_stop_hart));
+
+	cpumask_clear(&hsm_stop);
+
+	for_each_cpu(cpu, &secondary_cpus_mask) {
+		hartid = cpus[cpu].hartid;
+		if (hart_wait_on_status(hartid, SBI_EXT_HSM_STARTED, hsm_timer_duration))
+			continue;
+		if (hart_wait_on_status(hartid, SBI_EXT_HSM_STOP_PENDING, hsm_timer_duration))
+			continue;
+
+		ret = sbi_hart_get_status(hartid);
+		if (ret.error)
+			report_info("hart %ld get status failed (error=%ld)", hartid, ret.error);
+		else if (ret.value != SBI_EXT_HSM_STOPPED)
+			report_info("hart %ld status is not 'stopped' (ret.value=%ld)", hartid, ret.value);
+		else
+			cpumask_set_cpu(cpu, &hsm_stop);
+	}
+
+	report(cpumask_weight(&hsm_stop) == max_cpus - 1, "all secondary harts stopped");
+
+	/* Reset the state of the secondary cpus since they did not have a chance to clean up after stopping */
+	for_each_cpu(cpu, &secondary_cpus_mask) {
+		on_cpu_clear_func(cpu);
+		set_cpu_online(cpu, false);
+		set_cpu_started(cpu, false);
+	}
+
+	if (__riscv_xlen == 32 || ipi_unavailable) {
+		hsm_timer_teardown();
+		report_prefix_popn(2);
+		return;
+	}
+
+	report_prefix_pop();
+
+	report_prefix_push("hart_suspend");
+
+	/* Select just one secondary cpu to run suspension tests with MSB of suspend type being set */
+	cpu = cpumask_next(-1, &secondary_cpus_mask);
+	hartid = cpus[cpu].hartid;
+
+	/* Boot up the secondary cpu and let it proceed to the idle loop */
+	on_cpu(cpu, hart_empty_fn, NULL);
+
+	on_cpu_async(cpu, hart_retentive_suspend_with_msb_set, NULL);
+
+	if (!hart_wait_on_status(hartid, SBI_EXT_HSM_STARTED, hsm_timer_duration) &&
+	    !hart_wait_on_status(hartid, SBI_EXT_HSM_SUSPEND_PENDING, hsm_timer_duration)) {
+		ret = sbi_hart_get_status(hartid);
+		if (ret.error)
+			report_info("hart %ld get status failed (error=%ld)", hartid, ret.error);
+		else if (ret.value != SBI_EXT_HSM_SUSPENDED)
+			report_info("hart %ld status is not 'suspended' (ret.value=%ld)", hartid, ret.value);
+		else
+			suspend_with_msb = true;
+	}
+
+	report(suspend_with_msb, "secondary hart retentive suspended with MSB set");
+
+	/* Ignore the return value since we manually validate the status of the hart anyway */
+	sbi_send_ipi_cpu(cpu);
+
+	if (!hart_wait_on_status(hartid, SBI_EXT_HSM_SUSPENDED, hsm_timer_duration) &&
+	    !hart_wait_on_status(hartid, SBI_EXT_HSM_RESUME_PENDING, hsm_timer_duration)) {
+		ret = sbi_hart_get_status(hartid);
+		if (ret.error)
+			report_info("hart %ld get status failed (error=%ld)", hartid, ret.error);
+		else if (ret.value != SBI_EXT_HSM_STARTED)
+			report_info("hart %ld status is not 'started' (ret.value=%ld)", hartid, ret.value);
+		else
+			resume_with_msb = true;
+	}
+
+	report(resume_with_msb, "secondary hart retentive resumed with MSB set");
+
+	/* Reset these flags so that we can reuse them for the non-retentive suspension test */
+	suspend_with_msb = false;
+	resume_with_msb = false;
+	sbi_hsm_stop_hart[cpu] = 0;
+	sbi_hsm_non_retentive_hart_suspend_checks[cpu] = 0;
+
+	on_cpu_async(cpu, hart_non_retentive_suspend_with_msb_set, NULL);
+
+	if (!hart_wait_on_status(hartid, SBI_EXT_HSM_STARTED, hsm_timer_duration) &&
+	    !hart_wait_on_status(hartid, SBI_EXT_HSM_SUSPEND_PENDING, hsm_timer_duration)) {
+		ret = sbi_hart_get_status(hartid);
+		if (ret.error)
+			report_info("hart %ld get status failed (error=%ld)", hartid, ret.error);
+		else if (ret.value != SBI_EXT_HSM_SUSPENDED)
+			report_info("hart %ld status is not 'suspended' (ret.value=%ld)", hartid, ret.value);
+		else
+			suspend_with_msb = true;
+	}
+
+	report(suspend_with_msb, "secondary hart non-retentive suspended with MSB set");
+
+	/* Ignore the return value since we manually validate the status of the hart anyway */
+	sbi_send_ipi_cpu(cpu);
+
+	if (!hart_wait_on_status(hartid, SBI_EXT_HSM_SUSPENDED, hsm_timer_duration) &&
+	    !hart_wait_on_status(hartid, SBI_EXT_HSM_RESUME_PENDING, hsm_timer_duration)) {
+		ret = sbi_hart_get_status(hartid);
+		if (ret.error)
+			report_info("hart %ld get status failed (error=%ld)", hartid, ret.error);
+		else if (ret.value != SBI_EXT_HSM_STARTED)
+			report_info("hart %ld status is not 'started' (ret.value=%ld)", hartid, ret.value);
+		else
+			resume_with_msb = true;
+
+		sbi_hsm_timer_fired = false;
+		timer_start(hsm_timer_duration);
+
+		while (!((READ_ONCE(sbi_hsm_non_retentive_hart_suspend_checks[cpu])) & SBI_HSM_TEST_DONE)
+			&& !sbi_hsm_timer_fired)
+			cpu_relax();
+
+		timer_stop();
+
+		if (sbi_hsm_timer_fired) {
+			report_info("hsm timer fired before hart %ld is done with non-retentive resume checks",
+				    hartid);
+		} else {
+			if (!(sbi_hsm_non_retentive_hart_suspend_checks[cpu] & SBI_HSM_TEST_SATP))
+				report_info("satp is not zero for test hart %ld", hartid);
+			else if (!(sbi_hsm_non_retentive_hart_suspend_checks[cpu] & SBI_HSM_TEST_SIE))
+				report_info("sstatus.SIE is not zero for test hart %ld", hartid);
+			else if (!(sbi_hsm_non_retentive_hart_suspend_checks[cpu] & SBI_HSM_TEST_HARTID_A1))
+				report_info("either a0 or a1 is not hartid for test hart %ld", hartid);
+			else
+				check_with_msb = true;
+		}
+	}
+
+	report(resume_with_msb, "secondary hart non-retentive resumed with MSB set");
+	report(check_with_msb,
+	       "secondary hart has expected register values after non-retentive resume with MSB set");
+
+	report_prefix_pop();
+
+	report_prefix_push("hart_stop");
+
+	sbi_hsm_stop_hart[cpu] = 1;
+
+	if (!hart_wait_on_status(hartid, SBI_EXT_HSM_STARTED, hsm_timer_duration) &&
+	    !hart_wait_on_status(hartid, SBI_EXT_HSM_STOP_PENDING, hsm_timer_duration)) {
+		ret = sbi_hart_get_status(hartid);
+		if (ret.error)
+			report_info("hart %ld get status failed (error=%ld)", hartid, ret.error);
+		else if (ret.value != SBI_EXT_HSM_STOPPED)
+			report_info("hart %ld status is not 'stopped' (ret.value=%ld)", hartid, ret.value);
+		else
+			stop_with_msb = true;
+	}
+
+	report(stop_with_msb, "secondary hart stopped after suspension tests with MSB set");
+
+	/* Reset the state of the secondary cpu since it did not have a chance to clean up after stopping */
+	on_cpu_clear_func(cpu);
+	set_cpu_online(cpu, false);
+	set_cpu_started(cpu, false);
+
+	hsm_timer_teardown();
+	report_prefix_popn(2);
+}
 
 int main(int argc, char **argv)
 {
@@ -444,6 +1091,7 @@ int main(int argc, char **argv)
 	report_prefix_push("sbi");
 	check_base();
 	check_time();
+	check_hsm();
 	check_dbcn();
 
 	return report_summary();
-- 
2.43.0



^ permalink raw reply related	[flat|nested] 18+ messages in thread

* [kvm-unit-tests PATCH v5 5/5] riscv: sbi: Add tests for HSM extension
@ 2024-09-21 10:08   ` James Raphael Tiovalen
  0 siblings, 0 replies; 18+ messages in thread
From: James Raphael Tiovalen @ 2024-09-21 10:08 UTC (permalink / raw)
  To: kvm, kvm-riscv; +Cc: andrew.jones, atishp, cade.richard, James Raphael Tiovalen

Add some tests for all of the HSM extension functions. These tests
ensure that the HSM extension functions follow the behavior as described
in the SBI specification.

Signed-off-by: James Raphael Tiovalen <jamestiotio@gmail.com>
---
 riscv/sbi.c | 648 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 648 insertions(+)

diff --git a/riscv/sbi.c b/riscv/sbi.c
index b5147dee..cd1ed95b 100644
--- a/riscv/sbi.c
+++ b/riscv/sbi.c
@@ -6,6 +6,8 @@
  */
 #include <libcflat.h>
 #include <alloc_page.h>
+#include <cpumask.h>
+#include <on-cpus.h>
 #include <stdlib.h>
 #include <string.h>
 #include <limits.h>
@@ -16,12 +18,15 @@
 #include <asm/delay.h>
 #include <asm/io.h>
 #include <asm/mmu.h>
+#include <asm/page.h>
 #include <asm/processor.h>
 #include <asm/sbi.h>
 #include <asm/setup.h>
 #include <asm/smp.h>
 #include <asm/timer.h>
 
+#include "sbi-tests.h"
+
 #define	HIGH_ADDR_BOUNDARY	((phys_addr_t)1 << 32)
 
 static void help(void)
@@ -47,6 +52,11 @@ static struct sbiret sbi_dbcn_write_byte(uint8_t byte)
 	return sbi_ecall(SBI_EXT_DBCN, SBI_EXT_DBCN_CONSOLE_WRITE_BYTE, byte, 0, 0, 0, 0, 0);
 }
 
+static struct sbiret sbi_hart_suspend(uint32_t suspend_type, unsigned long resume_addr, unsigned long opaque)
+{
+	return sbi_ecall(SBI_EXT_HSM, SBI_EXT_HSM_HART_SUSPEND, suspend_type, resume_addr, opaque, 0, 0, 0);
+}
+
 static void split_phys_addr(phys_addr_t paddr, unsigned long *hi, unsigned long *lo)
 {
 	*lo = (unsigned long)paddr;
@@ -433,6 +443,643 @@ static void check_dbcn(void)
 unsigned char sbi_hsm_stop_hart[NR_CPUS];
 unsigned char sbi_hsm_hart_start_checks[NR_CPUS];
 unsigned char sbi_hsm_non_retentive_hart_suspend_checks[NR_CPUS];
+cpumask_t sbi_hsm_started_hart_checks;
+static bool sbi_hsm_invalid_hartid_check;
+static bool sbi_hsm_timer_fired;
+extern void sbi_hsm_check_hart_start(void);
+extern void sbi_hsm_check_non_retentive_suspend(void);
+
+static void hsm_timer_irq_handler(struct pt_regs *regs)
+{
+	sbi_hsm_timer_fired = true;
+	timer_stop();
+}
+
+static void hsm_timer_setup(void)
+{
+	install_irq_handler(IRQ_S_TIMER, hsm_timer_irq_handler);
+	local_irq_enable();
+	timer_irq_enable();
+}
+
+static void hsm_timer_teardown(void)
+{
+	timer_irq_disable();
+	local_irq_disable();
+	install_irq_handler(IRQ_S_TIMER, NULL);
+}
+
+static void hart_empty_fn(void *data) {}
+
+static void hart_execute(void *data)
+{
+	struct sbiret ret;
+	unsigned long hartid = current_thread_info()->hartid;
+	int me = smp_processor_id();
+
+	ret = sbi_hart_start(hartid, virt_to_phys(&hart_empty_fn), 0);
+
+	if (ret.error == SBI_ERR_ALREADY_AVAILABLE)
+		cpumask_set_cpu(me, &sbi_hsm_started_hart_checks);
+}
+
+static void hart_start_invalid_hartid(void *data)
+{
+	struct sbiret ret;
+
+	ret = sbi_hart_start(ULONG_MAX, virt_to_phys(&hart_empty_fn), 0);
+
+	if (ret.error == SBI_ERR_INVALID_PARAM)
+		sbi_hsm_invalid_hartid_check = true;
+}
+
+static void hart_stop(void *data)
+{
+	unsigned long hartid = current_thread_info()->hartid;
+	struct sbiret ret = sbi_hart_stop();
+
+	report_fail("failed to stop hart %ld (error=%ld)", hartid, ret.error);
+}
+
+static void hart_retentive_suspend(void *data)
+{
+	unsigned long hartid = current_thread_info()->hartid;
+	struct sbiret ret = sbi_hart_suspend(SBI_EXT_HSM_HART_SUSPEND_RETENTIVE, 0, 0);
+
+	if (ret.error)
+		report_fail("failed to retentive suspend hart %ld (error=%ld)", hartid, ret.error);
+}
+
+static void hart_non_retentive_suspend(void *data)
+{
+	unsigned long hartid = current_thread_info()->hartid;
+
+	/* Set opaque as hartid so that we can check a0 == a1, ensuring that a0 is hartid and a1 is opaque */
+	struct sbiret ret = sbi_hart_suspend(SBI_EXT_HSM_HART_SUSPEND_NON_RETENTIVE,
+					     virt_to_phys(&sbi_hsm_check_non_retentive_suspend), hartid);
+
+	report_fail("failed to non-retentive suspend hart %ld (error=%ld)", hartid, ret.error);
+}
+
+static void hart_retentive_suspend_with_msb_set(void *data)
+{
+	unsigned long hartid = current_thread_info()->hartid;
+	unsigned long suspend_type = SBI_EXT_HSM_HART_SUSPEND_RETENTIVE | (_AC(1, UL) << (__riscv_xlen - 1));
+	struct sbiret ret = sbi_ecall(SBI_EXT_HSM, SBI_EXT_HSM_HART_SUSPEND, suspend_type, 0, 0, 0, 0, 0);
+
+	if (ret.error)
+		report_fail("failed to retentive suspend hart %ld with MSB set (error=%ld)", hartid, ret.error);
+}
+
+static void hart_non_retentive_suspend_with_msb_set(void *data)
+{
+	unsigned long hartid = current_thread_info()->hartid;
+	unsigned long suspend_type = SBI_EXT_HSM_HART_SUSPEND_NON_RETENTIVE | (_AC(1, UL) << (__riscv_xlen - 1));
+
+	/* Set opaque as hartid so that we can check a0 == a1, ensuring that a0 is hartid and a1 is opaque */
+	struct sbiret ret = sbi_ecall(SBI_EXT_HSM, SBI_EXT_HSM_HART_SUSPEND, suspend_type,
+				      virt_to_phys(&sbi_hsm_check_non_retentive_suspend), hartid, 0, 0, 0);
+
+	report_fail("failed to non-retentive suspend hart %ld with MSB set (error=%ld)", hartid, ret.error);
+}
+
+static bool hart_wait_on_status(unsigned long hartid, enum sbi_ext_hsm_sid status, unsigned long duration)
+{
+	struct sbiret ret;
+
+	sbi_hsm_timer_fired = false;
+	timer_start(duration);
+
+	ret = sbi_hart_get_status(hartid);
+
+	while (!ret.error && ret.value == status && !sbi_hsm_timer_fired) {
+		cpu_relax();
+		ret = sbi_hart_get_status(hartid);
+	}
+
+	timer_stop();
+
+	if (sbi_hsm_timer_fired)
+		report_info("timer fired while waiting on status %u for hart %ld", status, hartid);
+	else if (ret.error)
+		report_fail("got %ld while waiting on status %u for hart %ld\n", ret.error, status, hartid);
+
+	return sbi_hsm_timer_fired || ret.error;
+}
+
+static void check_hsm(void)
+{
+	struct sbiret ret;
+	unsigned long hartid;
+	cpumask_t secondary_cpus_mask, hsm_start, hsm_stop, hsm_suspend, hsm_resume, hsm_check;
+	bool ipi_unavailable = false;
+	bool suspend_with_msb = false, resume_with_msb = false, check_with_msb = false, stop_with_msb = false;
+	int cpu, me = smp_processor_id();
+	int max_cpus = getenv("SBI_MAX_CPUS") ? strtol(getenv("SBI_MAX_CPUS"), NULL, 0) : nr_cpus;
+	unsigned long hsm_timer_duration = getenv("SBI_HSM_TIMER_DURATION")
+					 ? strtol(getenv("SBI_HSM_TIMER_DURATION"), NULL, 0) : 200000;
+
+	max_cpus = MIN(max_cpus, nr_cpus);
+
+	report_prefix_push("hsm");
+
+	if (!sbi_probe(SBI_EXT_HSM)) {
+		report_skip("hsm extension not available");
+		report_prefix_pop();
+		return;
+	}
+
+	report_prefix_push("hart_get_status");
+
+	hartid = current_thread_info()->hartid;
+	ret = sbi_hart_get_status(hartid);
+
+	if (ret.error) {
+		report_fail("failed to get status of current hart (error=%ld)", ret.error);
+		report_prefix_popn(2);
+		return;
+	} else if (ret.value != SBI_EXT_HSM_STARTED) {
+		report_fail("current hart is not started (ret.value=%ld)", ret.value);
+		report_prefix_popn(2);
+		return;
+	}
+
+	report_pass("status of current hart is started");
+
+	for_each_present_cpu(cpu) {
+		if (sbi_hart_get_status(cpus[cpu].hartid).error == SBI_ERR_INVALID_PARAM)
+			set_cpu_present(cpu, false);
+	}
+
+	report(cpumask_weight(&cpu_present_mask) == nr_cpus, "all present harts have valid hartids");
+
+	report_prefix_pop();
+
+	if (max_cpus < 2) {
+		report_skip("no other cpus to run the remaining hsm tests on");
+		report_prefix_pop();
+		return;
+	}
+
+	report_prefix_push("hart_start");
+
+	cpumask_copy(&secondary_cpus_mask, &cpu_present_mask);
+	cpumask_clear_cpu(me, &secondary_cpus_mask);
+	hsm_timer_setup();
+
+	cpumask_clear(&hsm_start);
+	cpumask_clear(&hsm_check);
+
+	for_each_cpu(cpu, &secondary_cpus_mask) {
+		hartid = cpus[cpu].hartid;
+		/* Set opaque as hartid so that we can check a0 == a1, ensuring that a0 is hartid and a1 is opaque */
+		ret = sbi_hart_start(hartid, virt_to_phys(&sbi_hsm_check_hart_start), hartid);
+		if (ret.error) {
+			report_fail("failed to start test hart %ld (error=%ld)", hartid, ret.error);
+			continue;
+		}
+
+		if (hart_wait_on_status(hartid, SBI_EXT_HSM_STOPPED, hsm_timer_duration))
+			continue;
+		if (hart_wait_on_status(hartid, SBI_EXT_HSM_START_PENDING, hsm_timer_duration))
+			continue;
+
+		ret = sbi_hart_get_status(hartid);
+		if (ret.error) {
+			report_info("hart %ld get status failed (error=%ld)", hartid, ret.error);
+			continue;
+		} else if (ret.value != SBI_EXT_HSM_STARTED) {
+			report_info("hart %ld status is not 'started' (ret.value=%ld)", hartid, ret.value);
+			continue;
+		} else {
+			cpumask_set_cpu(cpu, &hsm_start);
+		}
+
+		sbi_hsm_timer_fired = false;
+		timer_start(hsm_timer_duration);
+
+		while (!(READ_ONCE(sbi_hsm_hart_start_checks[cpu]) & SBI_HSM_TEST_DONE) && !sbi_hsm_timer_fired)
+			cpu_relax();
+
+		timer_stop();
+
+		if (sbi_hsm_timer_fired) {
+			report_info("hsm timer fired before hart %ld is done with start checks", hartid);
+			continue;
+		}
+
+		if (!(sbi_hsm_hart_start_checks[cpu] & SBI_HSM_TEST_SATP))
+			report_info("satp is not zero for test hart %ld", hartid);
+		else if (!(sbi_hsm_hart_start_checks[cpu] & SBI_HSM_TEST_SIE))
+			report_info("sstatus.SIE is not zero for test hart %ld", hartid);
+		else if (!(sbi_hsm_hart_start_checks[cpu] & SBI_HSM_TEST_HARTID_A1))
+			report_info("either a0 or a1 is not hartid for test hart %ld", hartid);
+		else
+			cpumask_set_cpu(cpu, &hsm_check);
+	}
+
+	report(cpumask_weight(&hsm_start) == max_cpus - 1, "all secondary harts started");
+	report(cpumask_weight(&hsm_check) == max_cpus - 1,
+	       "all secondary harts have expected register values after hart start");
+
+	report_prefix_pop();
+
+	report_prefix_push("hart_stop");
+
+	memset(sbi_hsm_stop_hart, 1, sizeof(sbi_hsm_stop_hart));
+
+	cpumask_clear(&hsm_stop);
+
+	for_each_cpu(cpu, &secondary_cpus_mask) {
+		hartid = cpus[cpu].hartid;
+		if (hart_wait_on_status(hartid, SBI_EXT_HSM_STARTED, hsm_timer_duration))
+			continue;
+		if (hart_wait_on_status(hartid, SBI_EXT_HSM_STOP_PENDING, hsm_timer_duration))
+			continue;
+
+		ret = sbi_hart_get_status(hartid);
+		if (ret.error)
+			report_info("hart %ld get status failed (error=%ld)", hartid, ret.error);
+		else if (ret.value != SBI_EXT_HSM_STOPPED)
+			report_info("hart %ld status is not 'stopped' (ret.value=%ld)", hartid, ret.value);
+		else
+			cpumask_set_cpu(cpu, &hsm_stop);
+	}
+
+	report(cpumask_weight(&hsm_stop) == max_cpus - 1, "all secondary harts stopped");
+
+	/* Reset the stop flags so that we can reuse them after suspension tests */
+	memset(sbi_hsm_stop_hart, 0, sizeof(sbi_hsm_stop_hart));
+
+	report_prefix_pop();
+
+	report_prefix_push("hart_start");
+
+	/* Select just one secondary cpu to run the invalid hartid test */
+	on_cpu(cpumask_next(-1, &secondary_cpus_mask), hart_start_invalid_hartid, NULL);
+
+	report(sbi_hsm_invalid_hartid_check, "secondary hart refuse to start with invalid hartid");
+
+	on_cpumask_async(&secondary_cpus_mask, hart_execute, NULL);
+
+	cpumask_clear(&hsm_start);
+
+	for_each_cpu(cpu, &secondary_cpus_mask) {
+		hartid = cpus[cpu].hartid;
+		if (hart_wait_on_status(hartid, SBI_EXT_HSM_STOPPED, hsm_timer_duration))
+			continue;
+		if (hart_wait_on_status(hartid, SBI_EXT_HSM_START_PENDING, hsm_timer_duration))
+			continue;
+
+		ret = sbi_hart_get_status(hartid);
+		if (ret.error)
+			report_info("hart %ld get status failed (error=%ld)", hartid, ret.error);
+		else if (ret.value != SBI_EXT_HSM_STARTED)
+			report_info("hart %ld status is not 'started' (ret.value=%ld)", hartid, ret.value);
+		else
+			cpumask_set_cpu(cpu, &hsm_start);
+	}
+
+	report(cpumask_weight(&hsm_start) == max_cpus - 1, "all secondary harts started");
+
+	sbi_hsm_timer_fired = false;
+	timer_start(hsm_timer_duration);
+
+	while (cpumask_weight(&cpu_idle_mask) != max_cpus - 1 && !sbi_hsm_timer_fired)
+		cpu_relax();
+
+	timer_stop();
+
+	if (sbi_hsm_timer_fired)
+		report_info("hsm timer fired before all secondary harts started");
+
+	report(cpumask_weight(&cpu_idle_mask) == max_cpus - 1,
+	       "all secondary harts successfully executed code after start");
+	report(cpumask_weight(&cpu_online_mask) == max_cpus, "all secondary harts online");
+	report(cpumask_weight(&sbi_hsm_started_hart_checks) == max_cpus - 1,
+	       "all secondary harts are already started");
+
+	report_prefix_pop();
+
+	report_prefix_push("hart_suspend");
+
+	if (!sbi_probe(SBI_EXT_IPI)) {
+		report_skip("skipping suspension tests since ipi extension is unavailable");
+		report_prefix_pop();
+		ipi_unavailable = true;
+		goto sbi_hsm_hart_stop_tests;
+	}
+
+	on_cpumask_async(&secondary_cpus_mask, hart_retentive_suspend, NULL);
+
+	cpumask_clear(&hsm_suspend);
+
+	for_each_cpu(cpu, &secondary_cpus_mask) {
+		hartid = cpus[cpu].hartid;
+		if (hart_wait_on_status(hartid, SBI_EXT_HSM_STARTED, hsm_timer_duration))
+			continue;
+		if (hart_wait_on_status(hartid, SBI_EXT_HSM_SUSPEND_PENDING, hsm_timer_duration))
+			continue;
+
+		ret = sbi_hart_get_status(hartid);
+		if (ret.error)
+			report_info("hart %ld get status failed (error=%ld)", hartid, ret.error);
+		else if (ret.value != SBI_EXT_HSM_SUSPENDED)
+			report_info("hart %ld status is not 'suspended' (ret.value=%ld)", hartid, ret.value);
+		else
+			cpumask_set_cpu(cpu, &hsm_suspend);
+	}
+
+	report(cpumask_weight(&hsm_suspend) == max_cpus - 1, "all secondary harts retentive suspended");
+
+	/* Ignore the return value since we check the status of each hart anyway */
+	sbi_send_ipi_cpumask(&secondary_cpus_mask);
+
+	cpumask_clear(&hsm_resume);
+
+	for_each_cpu(cpu, &secondary_cpus_mask) {
+		hartid = cpus[cpu].hartid;
+		if (hart_wait_on_status(hartid, SBI_EXT_HSM_SUSPENDED, hsm_timer_duration))
+			continue;
+		if (hart_wait_on_status(hartid, SBI_EXT_HSM_RESUME_PENDING, hsm_timer_duration))
+			continue;
+
+		ret = sbi_hart_get_status(hartid);
+		if (ret.error)
+			report_info("hart %ld get status failed (error=%ld)", hartid, ret.error);
+		else if (ret.value != SBI_EXT_HSM_STARTED)
+			report_info("hart %ld status is not 'started' (ret.value=%ld)", hartid, ret.value);
+		else
+			cpumask_set_cpu(cpu, &hsm_resume);
+	}
+
+	report(cpumask_weight(&hsm_resume) == max_cpus - 1, "all secondary harts retentive resumed");
+
+	sbi_hsm_timer_fired = false;
+	timer_start(hsm_timer_duration);
+
+	while (cpumask_weight(&cpu_idle_mask) != max_cpus - 1 && !sbi_hsm_timer_fired)
+		cpu_relax();
+
+	timer_stop();
+
+	if (sbi_hsm_timer_fired)
+		report_info("hsm timer fired before all secondary harts retentive resumed");
+
+	report(cpumask_weight(&cpu_idle_mask) == max_cpus - 1,
+	       "all secondary harts successfully executed code after retentive suspend");
+	report(cpumask_weight(&cpu_online_mask) == max_cpus,
+	       "all secondary harts online");
+
+	on_cpumask_async(&secondary_cpus_mask, hart_non_retentive_suspend, NULL);
+
+	cpumask_clear(&hsm_suspend);
+
+	for_each_cpu(cpu, &secondary_cpus_mask) {
+		hartid = cpus[cpu].hartid;
+		if (hart_wait_on_status(hartid, SBI_EXT_HSM_STARTED, hsm_timer_duration))
+			continue;
+		if (hart_wait_on_status(hartid, SBI_EXT_HSM_SUSPEND_PENDING, hsm_timer_duration))
+			continue;
+
+		ret = sbi_hart_get_status(hartid);
+		if (ret.error)
+			report_info("hart %ld get status failed (error=%ld)", hartid, ret.error);
+		else if (ret.value != SBI_EXT_HSM_SUSPENDED)
+			report_info("hart %ld status is not 'suspended' (ret.value=%ld)", hartid, ret.value);
+		else
+			cpumask_set_cpu(cpu, &hsm_suspend);
+	}
+
+	report(cpumask_weight(&hsm_suspend) == max_cpus - 1, "all secondary harts non-retentive suspended");
+
+	/* Ignore the return value since we check the status of each hart anyway */
+	sbi_send_ipi_cpumask(&secondary_cpus_mask);
+
+	cpumask_clear(&hsm_resume);
+	cpumask_clear(&hsm_check);
+
+	for_each_cpu(cpu, &secondary_cpus_mask) {
+		hartid = cpus[cpu].hartid;
+		if (hart_wait_on_status(hartid, SBI_EXT_HSM_SUSPENDED, hsm_timer_duration))
+			continue;
+		if (hart_wait_on_status(hartid, SBI_EXT_HSM_RESUME_PENDING, hsm_timer_duration))
+			continue;
+
+		ret = sbi_hart_get_status(hartid);
+		if (ret.error) {
+			report_info("hart %ld get status failed (error=%ld)", hartid, ret.error);
+			continue;
+		} else if (ret.value != SBI_EXT_HSM_STARTED) {
+			report_info("hart %ld status is not 'started' (ret.value=%ld)", hartid, ret.value);
+			continue;
+		} else {
+			cpumask_set_cpu(cpu, &hsm_resume);
+		}
+
+		sbi_hsm_timer_fired = false;
+		timer_start(hsm_timer_duration);
+
+		while (!((READ_ONCE(sbi_hsm_non_retentive_hart_suspend_checks[cpu])) & SBI_HSM_TEST_DONE)
+			&& !sbi_hsm_timer_fired)
+			cpu_relax();
+
+		timer_stop();
+
+		if (sbi_hsm_timer_fired) {
+			report_info("hsm timer fired before hart %ld is done with non-retentive resume checks",
+				    hartid);
+			continue;
+		}
+
+		if (!(sbi_hsm_non_retentive_hart_suspend_checks[cpu] & SBI_HSM_TEST_SATP))
+			report_info("satp is not zero for test hart %ld", hartid);
+		else if (!(sbi_hsm_non_retentive_hart_suspend_checks[cpu] & SBI_HSM_TEST_SIE))
+			report_info("sstatus.SIE is not zero for test hart %ld", hartid);
+		else if (!(sbi_hsm_non_retentive_hart_suspend_checks[cpu] & SBI_HSM_TEST_HARTID_A1))
+			report_info("either a0 or a1 is not hartid for test hart %ld", hartid);
+		else
+			cpumask_set_cpu(cpu, &hsm_check);
+	}
+
+	report(cpumask_weight(&hsm_resume) == max_cpus - 1, "all secondary harts non-retentive resumed");
+	report(cpumask_weight(&hsm_check) == max_cpus - 1,
+	       "all secondary harts have expected register values after non-retentive resume");
+
+	report_prefix_pop();
+
+sbi_hsm_hart_stop_tests:
+	report_prefix_push("hart_stop");
+
+	if (ipi_unavailable)
+		on_cpumask_async(&secondary_cpus_mask, hart_stop, NULL);
+	else
+		memset(sbi_hsm_stop_hart, 1, sizeof(sbi_hsm_stop_hart));
+
+	cpumask_clear(&hsm_stop);
+
+	for_each_cpu(cpu, &secondary_cpus_mask) {
+		hartid = cpus[cpu].hartid;
+		if (hart_wait_on_status(hartid, SBI_EXT_HSM_STARTED, hsm_timer_duration))
+			continue;
+		if (hart_wait_on_status(hartid, SBI_EXT_HSM_STOP_PENDING, hsm_timer_duration))
+			continue;
+
+		ret = sbi_hart_get_status(hartid);
+		if (ret.error)
+			report_info("hart %ld get status failed (error=%ld)", hartid, ret.error);
+		else if (ret.value != SBI_EXT_HSM_STOPPED)
+			report_info("hart %ld status is not 'stopped' (ret.value=%ld)", hartid, ret.value);
+		else
+			cpumask_set_cpu(cpu, &hsm_stop);
+	}
+
+	report(cpumask_weight(&hsm_stop) == max_cpus - 1, "all secondary harts stopped");
+
+	/* Reset the state of the secondary cpus since they did not have a chance to clean up after stopping */
+	for_each_cpu(cpu, &secondary_cpus_mask) {
+		on_cpu_clear_func(cpu);
+		set_cpu_online(cpu, false);
+		set_cpu_started(cpu, false);
+	}
+
+	if (__riscv_xlen == 32 || ipi_unavailable) {
+		hsm_timer_teardown();
+		report_prefix_popn(2);
+		return;
+	}
+
+	report_prefix_pop();
+
+	report_prefix_push("hart_suspend");
+
+	/* Select just one secondary cpu to run suspension tests with MSB of suspend type being set */
+	cpu = cpumask_next(-1, &secondary_cpus_mask);
+	hartid = cpus[cpu].hartid;
+
+	/* Boot up the secondary cpu and let it proceed to the idle loop */
+	on_cpu(cpu, hart_empty_fn, NULL);
+
+	on_cpu_async(cpu, hart_retentive_suspend_with_msb_set, NULL);
+
+	if (!hart_wait_on_status(hartid, SBI_EXT_HSM_STARTED, hsm_timer_duration) &&
+	    !hart_wait_on_status(hartid, SBI_EXT_HSM_SUSPEND_PENDING, hsm_timer_duration)) {
+		ret = sbi_hart_get_status(hartid);
+		if (ret.error)
+			report_info("hart %ld get status failed (error=%ld)", hartid, ret.error);
+		else if (ret.value != SBI_EXT_HSM_SUSPENDED)
+			report_info("hart %ld status is not 'suspended' (ret.value=%ld)", hartid, ret.value);
+		else
+			suspend_with_msb = true;
+	}
+
+	report(suspend_with_msb, "secondary hart retentive suspended with MSB set");
+
+	/* Ignore the return value since we manually validate the status of the hart anyway */
+	sbi_send_ipi_cpu(cpu);
+
+	if (!hart_wait_on_status(hartid, SBI_EXT_HSM_SUSPENDED, hsm_timer_duration) &&
+	    !hart_wait_on_status(hartid, SBI_EXT_HSM_RESUME_PENDING, hsm_timer_duration)) {
+		ret = sbi_hart_get_status(hartid);
+		if (ret.error)
+			report_info("hart %ld get status failed (error=%ld)", hartid, ret.error);
+		else if (ret.value != SBI_EXT_HSM_STARTED)
+			report_info("hart %ld status is not 'started' (ret.value=%ld)", hartid, ret.value);
+		else
+			resume_with_msb = true;
+	}
+
+	report(resume_with_msb, "secondary hart retentive resumed with MSB set");
+
+	/* Reset these flags so that we can reuse them for the non-retentive suspension test */
+	suspend_with_msb = false;
+	resume_with_msb = false;
+	sbi_hsm_stop_hart[cpu] = 0;
+	sbi_hsm_non_retentive_hart_suspend_checks[cpu] = 0;
+
+	on_cpu_async(cpu, hart_non_retentive_suspend_with_msb_set, NULL);
+
+	if (!hart_wait_on_status(hartid, SBI_EXT_HSM_STARTED, hsm_timer_duration) &&
+	    !hart_wait_on_status(hartid, SBI_EXT_HSM_SUSPEND_PENDING, hsm_timer_duration)) {
+		ret = sbi_hart_get_status(hartid);
+		if (ret.error)
+			report_info("hart %ld get status failed (error=%ld)", hartid, ret.error);
+		else if (ret.value != SBI_EXT_HSM_SUSPENDED)
+			report_info("hart %ld status is not 'suspended' (ret.value=%ld)", hartid, ret.value);
+		else
+			suspend_with_msb = true;
+	}
+
+	report(suspend_with_msb, "secondary hart non-retentive suspended with MSB set");
+
+	/* Ignore the return value since we manually validate the status of the hart anyway */
+	sbi_send_ipi_cpu(cpu);
+
+	if (!hart_wait_on_status(hartid, SBI_EXT_HSM_SUSPENDED, hsm_timer_duration) &&
+	    !hart_wait_on_status(hartid, SBI_EXT_HSM_RESUME_PENDING, hsm_timer_duration)) {
+		ret = sbi_hart_get_status(hartid);
+		if (ret.error)
+			report_info("hart %ld get status failed (error=%ld)", hartid, ret.error);
+		else if (ret.value != SBI_EXT_HSM_STARTED)
+			report_info("hart %ld status is not 'started' (ret.value=%ld)", hartid, ret.value);
+		else
+			resume_with_msb = true;
+
+		sbi_hsm_timer_fired = false;
+		timer_start(hsm_timer_duration);
+
+		while (!((READ_ONCE(sbi_hsm_non_retentive_hart_suspend_checks[cpu])) & SBI_HSM_TEST_DONE)
+			&& !sbi_hsm_timer_fired)
+			cpu_relax();
+
+		timer_stop();
+
+		if (sbi_hsm_timer_fired) {
+			report_info("hsm timer fired before hart %ld is done with non-retentive resume checks",
+				    hartid);
+		} else {
+			if (!(sbi_hsm_non_retentive_hart_suspend_checks[cpu] & SBI_HSM_TEST_SATP))
+				report_info("satp is not zero for test hart %ld", hartid);
+			else if (!(sbi_hsm_non_retentive_hart_suspend_checks[cpu] & SBI_HSM_TEST_SIE))
+				report_info("sstatus.SIE is not zero for test hart %ld", hartid);
+			else if (!(sbi_hsm_non_retentive_hart_suspend_checks[cpu] & SBI_HSM_TEST_HARTID_A1))
+				report_info("either a0 or a1 is not hartid for test hart %ld", hartid);
+			else
+				check_with_msb = true;
+		}
+	}
+
+	report(resume_with_msb, "secondary hart non-retentive resumed with MSB set");
+	report(check_with_msb,
+	       "secondary hart has expected register values after non-retentive resume with MSB set");
+
+	report_prefix_pop();
+
+	report_prefix_push("hart_stop");
+
+	sbi_hsm_stop_hart[cpu] = 1;
+
+	if (!hart_wait_on_status(hartid, SBI_EXT_HSM_STARTED, hsm_timer_duration) &&
+	    !hart_wait_on_status(hartid, SBI_EXT_HSM_STOP_PENDING, hsm_timer_duration)) {
+		ret = sbi_hart_get_status(hartid);
+		if (ret.error)
+			report_info("hart %ld get status failed (error=%ld)", hartid, ret.error);
+		else if (ret.value != SBI_EXT_HSM_STOPPED)
+			report_info("hart %ld status is not 'stopped' (ret.value=%ld)", hartid, ret.value);
+		else
+			stop_with_msb = true;
+	}
+
+	report(stop_with_msb, "secondary hart stopped after suspension tests with MSB set");
+
+	/* Reset the state of the secondary cpu since it did not have a chance to clean up after stopping */
+	on_cpu_clear_func(cpu);
+	set_cpu_online(cpu, false);
+	set_cpu_started(cpu, false);
+
+	hsm_timer_teardown();
+	report_prefix_popn(2);
+}
 
 int main(int argc, char **argv)
 {
@@ -444,6 +1091,7 @@ int main(int argc, char **argv)
 	report_prefix_push("sbi");
 	check_base();
 	check_time();
+	check_hsm();
 	check_dbcn();
 
 	return report_summary();
-- 
2.43.0


^ permalink raw reply related	[flat|nested] 18+ messages in thread

* [kvm-unit-tests PATCH v5 0/5] riscv: sbi: Add support to test HSM extension
  2024-09-21 10:08 ` James Raphael Tiovalen
@ 2024-10-23 14:23   ` Andrew Jones
  -1 siblings, 0 replies; 18+ messages in thread
From: Andrew Jones @ 2024-10-23 14:23 UTC (permalink / raw)
  To: kvm-riscv

On Sat, Sep 21, 2024 at 06:08:18PM +0800, James Raphael Tiovalen wrote:
> This patch series adds support for testing all 4 functions of the HSM
> extension as defined in the RISC-V SBI specification. The first 4
> patches add some helper routines to prepare for the HSM test, while
> the last patch adds the actual test for the HSM extension.

Hi James,

Patch1 is now merged and I've applied patch2 to riscv/sbi[1]. I've also
applied [2] and [3] to riscv/sbi so patches 3 and 4 of this series
should no longer be necessary. Can you please rebase patch5 on
riscv/sbi and repost?

[1] https://gitlab.com/jones-drew/kvm-unit-tests/-/commits/riscv/sbi
[2] https://lore.kernel.org/all/20241023131718.117452-4-andrew.jones at linux.dev/
[3] https://lore.kernel.org/all/20241023132130.118073-6-andrew.jones at linux.dev/

Thanks,
drew

> 
> v5:
> - Addressed all of Andrew's comments.
> - Added 2 new patches to clear on_cpu_info[cpu].func and to set the
>   cpu_started mask, which are used to perform cleanup after running the
>   HSM tests.
> - Added some new tests to validate suspension on RV64 with the high
>   bits set for suspend_type.
> - Picked up the hartid_to_cpu rewrite patch from Andrew's branch.
> - Moved the variables declared in riscv/sbi.c in patch 2 to group it
>   together with the other HSM test variables declared in patch 5.
> 
> v4:
> - Addressed all of Andrew's comments.
> - Included the 2 patches from Andrew's branch that refactored some
>   functions.
> - Added timers to all of the waiting activities in the HSM tests.
> 
> v3:
> - Addressed all of Andrew's comments.
> - Split the report_prefix_pop patch into its own series.
> - Added a new environment variable to specify the maximum number of
>   CPUs supported by the SBI implementation.
> 
> v2:
> - Addressed all of Andrew's comments.
> - Added a new patch to add helper routines to clear multiple prefixes.
> - Reworked the approach to test the HSM extension by using cpumask and
>   on-cpus.
> 
> Andrew Jones (1):
>   riscv: Rewrite hartid_to_cpu in assembly
> 
> James Raphael Tiovalen (4):
>   riscv: sbi: Provide entry point for HSM tests
>   lib/on-cpus: Add helper method to clear the function from on_cpu_info
>   riscv: Add helper method to set cpu started mask
>   riscv: sbi: Add tests for HSM extension
> 
>  riscv/Makefile          |   3 +-
>  lib/riscv/asm/smp.h     |   2 +
>  lib/on-cpus.h           |   1 +
>  lib/on-cpus.c           |  11 +
>  lib/riscv/asm-offsets.c |   5 +
>  lib/riscv/setup.c       |  10 -
>  lib/riscv/smp.c         |   8 +
>  riscv/sbi-tests.h       |  10 +
>  riscv/cstart.S          |  24 ++
>  riscv/sbi-asm.S         |  71 +++++
>  riscv/sbi.c             | 651 ++++++++++++++++++++++++++++++++++++++++
>  11 files changed, 785 insertions(+), 11 deletions(-)
>  create mode 100644 riscv/sbi-tests.h
>  create mode 100644 riscv/sbi-asm.S
> 
> --
> 2.43.0
> 
> 
> -- 
> kvm-riscv mailing list
> kvm-riscv at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kvm-riscv


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [kvm-unit-tests PATCH v5 0/5] riscv: sbi: Add support to test HSM extension
@ 2024-10-23 14:23   ` Andrew Jones
  0 siblings, 0 replies; 18+ messages in thread
From: Andrew Jones @ 2024-10-23 14:23 UTC (permalink / raw)
  To: James Raphael Tiovalen; +Cc: kvm, kvm-riscv, atishp, cade.richard

On Sat, Sep 21, 2024 at 06:08:18PM +0800, James Raphael Tiovalen wrote:
> This patch series adds support for testing all 4 functions of the HSM
> extension as defined in the RISC-V SBI specification. The first 4
> patches add some helper routines to prepare for the HSM test, while
> the last patch adds the actual test for the HSM extension.

Hi James,

Patch1 is now merged and I've applied patch2 to riscv/sbi[1]. I've also
applied [2] and [3] to riscv/sbi so patches 3 and 4 of this series
should no longer be necessary. Can you please rebase patch5 on
riscv/sbi and repost?

[1] https://gitlab.com/jones-drew/kvm-unit-tests/-/commits/riscv/sbi
[2] https://lore.kernel.org/all/20241023131718.117452-4-andrew.jones@linux.dev/
[3] https://lore.kernel.org/all/20241023132130.118073-6-andrew.jones@linux.dev/

Thanks,
drew

> 
> v5:
> - Addressed all of Andrew's comments.
> - Added 2 new patches to clear on_cpu_info[cpu].func and to set the
>   cpu_started mask, which are used to perform cleanup after running the
>   HSM tests.
> - Added some new tests to validate suspension on RV64 with the high
>   bits set for suspend_type.
> - Picked up the hartid_to_cpu rewrite patch from Andrew's branch.
> - Moved the variables declared in riscv/sbi.c in patch 2 to group it
>   together with the other HSM test variables declared in patch 5.
> 
> v4:
> - Addressed all of Andrew's comments.
> - Included the 2 patches from Andrew's branch that refactored some
>   functions.
> - Added timers to all of the waiting activities in the HSM tests.
> 
> v3:
> - Addressed all of Andrew's comments.
> - Split the report_prefix_pop patch into its own series.
> - Added a new environment variable to specify the maximum number of
>   CPUs supported by the SBI implementation.
> 
> v2:
> - Addressed all of Andrew's comments.
> - Added a new patch to add helper routines to clear multiple prefixes.
> - Reworked the approach to test the HSM extension by using cpumask and
>   on-cpus.
> 
> Andrew Jones (1):
>   riscv: Rewrite hartid_to_cpu in assembly
> 
> James Raphael Tiovalen (4):
>   riscv: sbi: Provide entry point for HSM tests
>   lib/on-cpus: Add helper method to clear the function from on_cpu_info
>   riscv: Add helper method to set cpu started mask
>   riscv: sbi: Add tests for HSM extension
> 
>  riscv/Makefile          |   3 +-
>  lib/riscv/asm/smp.h     |   2 +
>  lib/on-cpus.h           |   1 +
>  lib/on-cpus.c           |  11 +
>  lib/riscv/asm-offsets.c |   5 +
>  lib/riscv/setup.c       |  10 -
>  lib/riscv/smp.c         |   8 +
>  riscv/sbi-tests.h       |  10 +
>  riscv/cstart.S          |  24 ++
>  riscv/sbi-asm.S         |  71 +++++
>  riscv/sbi.c             | 651 ++++++++++++++++++++++++++++++++++++++++
>  11 files changed, 785 insertions(+), 11 deletions(-)
>  create mode 100644 riscv/sbi-tests.h
>  create mode 100644 riscv/sbi-asm.S
> 
> --
> 2.43.0
> 
> 
> -- 
> kvm-riscv mailing list
> kvm-riscv@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kvm-riscv

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [kvm-unit-tests PATCH v5 0/5] riscv: sbi: Add support to test HSM extension
  2024-10-23 14:23   ` Andrew Jones
@ 2024-10-23 14:40     ` James R T
  -1 siblings, 0 replies; 18+ messages in thread
From: James R T @ 2024-10-23 14:40 UTC (permalink / raw)
  To: kvm-riscv

On Wed, Oct 23, 2024 at 10:23?PM Andrew Jones <andrew.jones@linux.dev> wrote:
>
> On Sat, Sep 21, 2024 at 06:08:18PM +0800, James Raphael Tiovalen wrote:
> > This patch series adds support for testing all 4 functions of the HSM
> > extension as defined in the RISC-V SBI specification. The first 4
> > patches add some helper routines to prepare for the HSM test, while
> > the last patch adds the actual test for the HSM extension.
>
> Hi James,
>
> Patch1 is now merged and I've applied patch2 to riscv/sbi[1]. I've also
> applied [2] and [3] to riscv/sbi so patches 3 and 4 of this series
> should no longer be necessary. Can you please rebase patch5 on
> riscv/sbi and repost?
>

Hi Andrew,

Sure, I will rebase patch 5 on the riscv/sbi branch and repost it.

Best regards,
James Raphael Tiovalen

> [1] https://gitlab.com/jones-drew/kvm-unit-tests/-/commits/riscv/sbi
> [2] https://lore.kernel.org/all/20241023131718.117452-4-andrew.jones at linux.dev/
> [3] https://lore.kernel.org/all/20241023132130.118073-6-andrew.jones at linux.dev/
>
> Thanks,
> drew
>
> >
> > v5:
> > - Addressed all of Andrew's comments.
> > - Added 2 new patches to clear on_cpu_info[cpu].func and to set the
> >   cpu_started mask, which are used to perform cleanup after running the
> >   HSM tests.
> > - Added some new tests to validate suspension on RV64 with the high
> >   bits set for suspend_type.
> > - Picked up the hartid_to_cpu rewrite patch from Andrew's branch.
> > - Moved the variables declared in riscv/sbi.c in patch 2 to group it
> >   together with the other HSM test variables declared in patch 5.
> >
> > v4:
> > - Addressed all of Andrew's comments.
> > - Included the 2 patches from Andrew's branch that refactored some
> >   functions.
> > - Added timers to all of the waiting activities in the HSM tests.
> >
> > v3:
> > - Addressed all of Andrew's comments.
> > - Split the report_prefix_pop patch into its own series.
> > - Added a new environment variable to specify the maximum number of
> >   CPUs supported by the SBI implementation.
> >
> > v2:
> > - Addressed all of Andrew's comments.
> > - Added a new patch to add helper routines to clear multiple prefixes.
> > - Reworked the approach to test the HSM extension by using cpumask and
> >   on-cpus.
> >
> > Andrew Jones (1):
> >   riscv: Rewrite hartid_to_cpu in assembly
> >
> > James Raphael Tiovalen (4):
> >   riscv: sbi: Provide entry point for HSM tests
> >   lib/on-cpus: Add helper method to clear the function from on_cpu_info
> >   riscv: Add helper method to set cpu started mask
> >   riscv: sbi: Add tests for HSM extension
> >
> >  riscv/Makefile          |   3 +-
> >  lib/riscv/asm/smp.h     |   2 +
> >  lib/on-cpus.h           |   1 +
> >  lib/on-cpus.c           |  11 +
> >  lib/riscv/asm-offsets.c |   5 +
> >  lib/riscv/setup.c       |  10 -
> >  lib/riscv/smp.c         |   8 +
> >  riscv/sbi-tests.h       |  10 +
> >  riscv/cstart.S          |  24 ++
> >  riscv/sbi-asm.S         |  71 +++++
> >  riscv/sbi.c             | 651 ++++++++++++++++++++++++++++++++++++++++
> >  11 files changed, 785 insertions(+), 11 deletions(-)
> >  create mode 100644 riscv/sbi-tests.h
> >  create mode 100644 riscv/sbi-asm.S
> >
> > --
> > 2.43.0
> >
> >
> > --
> > kvm-riscv mailing list
> > kvm-riscv at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/kvm-riscv


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [kvm-unit-tests PATCH v5 0/5] riscv: sbi: Add support to test HSM extension
@ 2024-10-23 14:40     ` James R T
  0 siblings, 0 replies; 18+ messages in thread
From: James R T @ 2024-10-23 14:40 UTC (permalink / raw)
  To: Andrew Jones; +Cc: kvm, kvm-riscv, atishp, cade.richard

On Wed, Oct 23, 2024 at 10:23 PM Andrew Jones <andrew.jones@linux.dev> wrote:
>
> On Sat, Sep 21, 2024 at 06:08:18PM +0800, James Raphael Tiovalen wrote:
> > This patch series adds support for testing all 4 functions of the HSM
> > extension as defined in the RISC-V SBI specification. The first 4
> > patches add some helper routines to prepare for the HSM test, while
> > the last patch adds the actual test for the HSM extension.
>
> Hi James,
>
> Patch1 is now merged and I've applied patch2 to riscv/sbi[1]. I've also
> applied [2] and [3] to riscv/sbi so patches 3 and 4 of this series
> should no longer be necessary. Can you please rebase patch5 on
> riscv/sbi and repost?
>

Hi Andrew,

Sure, I will rebase patch 5 on the riscv/sbi branch and repost it.

Best regards,
James Raphael Tiovalen

> [1] https://gitlab.com/jones-drew/kvm-unit-tests/-/commits/riscv/sbi
> [2] https://lore.kernel.org/all/20241023131718.117452-4-andrew.jones@linux.dev/
> [3] https://lore.kernel.org/all/20241023132130.118073-6-andrew.jones@linux.dev/
>
> Thanks,
> drew
>
> >
> > v5:
> > - Addressed all of Andrew's comments.
> > - Added 2 new patches to clear on_cpu_info[cpu].func and to set the
> >   cpu_started mask, which are used to perform cleanup after running the
> >   HSM tests.
> > - Added some new tests to validate suspension on RV64 with the high
> >   bits set for suspend_type.
> > - Picked up the hartid_to_cpu rewrite patch from Andrew's branch.
> > - Moved the variables declared in riscv/sbi.c in patch 2 to group it
> >   together with the other HSM test variables declared in patch 5.
> >
> > v4:
> > - Addressed all of Andrew's comments.
> > - Included the 2 patches from Andrew's branch that refactored some
> >   functions.
> > - Added timers to all of the waiting activities in the HSM tests.
> >
> > v3:
> > - Addressed all of Andrew's comments.
> > - Split the report_prefix_pop patch into its own series.
> > - Added a new environment variable to specify the maximum number of
> >   CPUs supported by the SBI implementation.
> >
> > v2:
> > - Addressed all of Andrew's comments.
> > - Added a new patch to add helper routines to clear multiple prefixes.
> > - Reworked the approach to test the HSM extension by using cpumask and
> >   on-cpus.
> >
> > Andrew Jones (1):
> >   riscv: Rewrite hartid_to_cpu in assembly
> >
> > James Raphael Tiovalen (4):
> >   riscv: sbi: Provide entry point for HSM tests
> >   lib/on-cpus: Add helper method to clear the function from on_cpu_info
> >   riscv: Add helper method to set cpu started mask
> >   riscv: sbi: Add tests for HSM extension
> >
> >  riscv/Makefile          |   3 +-
> >  lib/riscv/asm/smp.h     |   2 +
> >  lib/on-cpus.h           |   1 +
> >  lib/on-cpus.c           |  11 +
> >  lib/riscv/asm-offsets.c |   5 +
> >  lib/riscv/setup.c       |  10 -
> >  lib/riscv/smp.c         |   8 +
> >  riscv/sbi-tests.h       |  10 +
> >  riscv/cstart.S          |  24 ++
> >  riscv/sbi-asm.S         |  71 +++++
> >  riscv/sbi.c             | 651 ++++++++++++++++++++++++++++++++++++++++
> >  11 files changed, 785 insertions(+), 11 deletions(-)
> >  create mode 100644 riscv/sbi-tests.h
> >  create mode 100644 riscv/sbi-asm.S
> >
> > --
> > 2.43.0
> >
> >
> > --
> > kvm-riscv mailing list
> > kvm-riscv@lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/kvm-riscv

^ permalink raw reply	[flat|nested] 18+ messages in thread

* [kvm-unit-tests PATCH v5 0/5] riscv: sbi: Add support to test HSM extension
  2024-10-23 14:23   ` Andrew Jones
@ 2024-11-06  8:44     ` Andrew Jones
  -1 siblings, 0 replies; 18+ messages in thread
From: Andrew Jones @ 2024-11-06  8:44 UTC (permalink / raw)
  To: kvm-riscv

On Wed, Oct 23, 2024 at 04:23:10PM +0200, Andrew Jones wrote:
> On Sat, Sep 21, 2024 at 06:08:18PM +0800, James Raphael Tiovalen wrote:
> > This patch series adds support for testing all 4 functions of the HSM
> > extension as defined in the RISC-V SBI specification. The first 4
> > patches add some helper routines to prepare for the HSM test, while
> > the last patch adds the actual test for the HSM extension.
> 
> Hi James,
> 
> Patch1 is now merged and I've applied patch2 to riscv/sbi[1]. I've also

I've had some second thoughts on patch2, as pointed out in the review of
v6. I need some bits of that patch, though, for other tests. I'll post a
patch with just those bits, retaining your authorship.

Thanks,
drew

> applied [2] and [3] to riscv/sbi so patches 3 and 4 of this series
> should no longer be necessary. Can you please rebase patch5 on
> riscv/sbi and repost?
> 
> [1] https://gitlab.com/jones-drew/kvm-unit-tests/-/commits/riscv/sbi
> [2] https://lore.kernel.org/all/20241023131718.117452-4-andrew.jones at linux.dev/
> [3] https://lore.kernel.org/all/20241023132130.118073-6-andrew.jones at linux.dev/
> 
> Thanks,
> drew
> 
> > 
> > v5:
> > - Addressed all of Andrew's comments.
> > - Added 2 new patches to clear on_cpu_info[cpu].func and to set the
> >   cpu_started mask, which are used to perform cleanup after running the
> >   HSM tests.
> > - Added some new tests to validate suspension on RV64 with the high
> >   bits set for suspend_type.
> > - Picked up the hartid_to_cpu rewrite patch from Andrew's branch.
> > - Moved the variables declared in riscv/sbi.c in patch 2 to group it
> >   together with the other HSM test variables declared in patch 5.
> > 
> > v4:
> > - Addressed all of Andrew's comments.
> > - Included the 2 patches from Andrew's branch that refactored some
> >   functions.
> > - Added timers to all of the waiting activities in the HSM tests.
> > 
> > v3:
> > - Addressed all of Andrew's comments.
> > - Split the report_prefix_pop patch into its own series.
> > - Added a new environment variable to specify the maximum number of
> >   CPUs supported by the SBI implementation.
> > 
> > v2:
> > - Addressed all of Andrew's comments.
> > - Added a new patch to add helper routines to clear multiple prefixes.
> > - Reworked the approach to test the HSM extension by using cpumask and
> >   on-cpus.
> > 
> > Andrew Jones (1):
> >   riscv: Rewrite hartid_to_cpu in assembly
> > 
> > James Raphael Tiovalen (4):
> >   riscv: sbi: Provide entry point for HSM tests
> >   lib/on-cpus: Add helper method to clear the function from on_cpu_info
> >   riscv: Add helper method to set cpu started mask
> >   riscv: sbi: Add tests for HSM extension
> > 
> >  riscv/Makefile          |   3 +-
> >  lib/riscv/asm/smp.h     |   2 +
> >  lib/on-cpus.h           |   1 +
> >  lib/on-cpus.c           |  11 +
> >  lib/riscv/asm-offsets.c |   5 +
> >  lib/riscv/setup.c       |  10 -
> >  lib/riscv/smp.c         |   8 +
> >  riscv/sbi-tests.h       |  10 +
> >  riscv/cstart.S          |  24 ++
> >  riscv/sbi-asm.S         |  71 +++++
> >  riscv/sbi.c             | 651 ++++++++++++++++++++++++++++++++++++++++
> >  11 files changed, 785 insertions(+), 11 deletions(-)
> >  create mode 100644 riscv/sbi-tests.h
> >  create mode 100644 riscv/sbi-asm.S
> > 
> > --
> > 2.43.0
> > 
> > 
> > -- 
> > kvm-riscv mailing list
> > kvm-riscv at lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/kvm-riscv
> 
> -- 
> kvm-riscv mailing list
> kvm-riscv at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kvm-riscv


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [kvm-unit-tests PATCH v5 0/5] riscv: sbi: Add support to test HSM extension
@ 2024-11-06  8:44     ` Andrew Jones
  0 siblings, 0 replies; 18+ messages in thread
From: Andrew Jones @ 2024-11-06  8:44 UTC (permalink / raw)
  To: James Raphael Tiovalen; +Cc: kvm, kvm-riscv, atishp, cade.richard

On Wed, Oct 23, 2024 at 04:23:10PM +0200, Andrew Jones wrote:
> On Sat, Sep 21, 2024 at 06:08:18PM +0800, James Raphael Tiovalen wrote:
> > This patch series adds support for testing all 4 functions of the HSM
> > extension as defined in the RISC-V SBI specification. The first 4
> > patches add some helper routines to prepare for the HSM test, while
> > the last patch adds the actual test for the HSM extension.
> 
> Hi James,
> 
> Patch1 is now merged and I've applied patch2 to riscv/sbi[1]. I've also

I've had some second thoughts on patch2, as pointed out in the review of
v6. I need some bits of that patch, though, for other tests. I'll post a
patch with just those bits, retaining your authorship.

Thanks,
drew

> applied [2] and [3] to riscv/sbi so patches 3 and 4 of this series
> should no longer be necessary. Can you please rebase patch5 on
> riscv/sbi and repost?
> 
> [1] https://gitlab.com/jones-drew/kvm-unit-tests/-/commits/riscv/sbi
> [2] https://lore.kernel.org/all/20241023131718.117452-4-andrew.jones@linux.dev/
> [3] https://lore.kernel.org/all/20241023132130.118073-6-andrew.jones@linux.dev/
> 
> Thanks,
> drew
> 
> > 
> > v5:
> > - Addressed all of Andrew's comments.
> > - Added 2 new patches to clear on_cpu_info[cpu].func and to set the
> >   cpu_started mask, which are used to perform cleanup after running the
> >   HSM tests.
> > - Added some new tests to validate suspension on RV64 with the high
> >   bits set for suspend_type.
> > - Picked up the hartid_to_cpu rewrite patch from Andrew's branch.
> > - Moved the variables declared in riscv/sbi.c in patch 2 to group it
> >   together with the other HSM test variables declared in patch 5.
> > 
> > v4:
> > - Addressed all of Andrew's comments.
> > - Included the 2 patches from Andrew's branch that refactored some
> >   functions.
> > - Added timers to all of the waiting activities in the HSM tests.
> > 
> > v3:
> > - Addressed all of Andrew's comments.
> > - Split the report_prefix_pop patch into its own series.
> > - Added a new environment variable to specify the maximum number of
> >   CPUs supported by the SBI implementation.
> > 
> > v2:
> > - Addressed all of Andrew's comments.
> > - Added a new patch to add helper routines to clear multiple prefixes.
> > - Reworked the approach to test the HSM extension by using cpumask and
> >   on-cpus.
> > 
> > Andrew Jones (1):
> >   riscv: Rewrite hartid_to_cpu in assembly
> > 
> > James Raphael Tiovalen (4):
> >   riscv: sbi: Provide entry point for HSM tests
> >   lib/on-cpus: Add helper method to clear the function from on_cpu_info
> >   riscv: Add helper method to set cpu started mask
> >   riscv: sbi: Add tests for HSM extension
> > 
> >  riscv/Makefile          |   3 +-
> >  lib/riscv/asm/smp.h     |   2 +
> >  lib/on-cpus.h           |   1 +
> >  lib/on-cpus.c           |  11 +
> >  lib/riscv/asm-offsets.c |   5 +
> >  lib/riscv/setup.c       |  10 -
> >  lib/riscv/smp.c         |   8 +
> >  riscv/sbi-tests.h       |  10 +
> >  riscv/cstart.S          |  24 ++
> >  riscv/sbi-asm.S         |  71 +++++
> >  riscv/sbi.c             | 651 ++++++++++++++++++++++++++++++++++++++++
> >  11 files changed, 785 insertions(+), 11 deletions(-)
> >  create mode 100644 riscv/sbi-tests.h
> >  create mode 100644 riscv/sbi-asm.S
> > 
> > --
> > 2.43.0
> > 
> > 
> > -- 
> > kvm-riscv mailing list
> > kvm-riscv@lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/kvm-riscv
> 
> -- 
> kvm-riscv mailing list
> kvm-riscv@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/kvm-riscv

^ permalink raw reply	[flat|nested] 18+ messages in thread

end of thread, other threads:[~2024-11-06  8:45 UTC | newest]

Thread overview: 18+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-09-21 10:08 [kvm-unit-tests PATCH v5 0/5] riscv: sbi: Add support to test HSM extension James Raphael Tiovalen
2024-09-21 10:08 ` James Raphael Tiovalen
2024-09-21 10:08 ` [kvm-unit-tests PATCH v5 1/5] riscv: Rewrite hartid_to_cpu in assembly James Raphael Tiovalen
2024-09-21 10:08   ` James Raphael Tiovalen
2024-09-21 10:08 ` [kvm-unit-tests PATCH v5 2/5] riscv: sbi: Provide entry point for HSM tests James Raphael Tiovalen
2024-09-21 10:08   ` James Raphael Tiovalen
2024-09-21 10:08 ` [kvm-unit-tests PATCH v5 3/5] lib/on-cpus: Add helper method to clear the function from on_cpu_info James Raphael Tiovalen
2024-09-21 10:08   ` James Raphael Tiovalen
2024-09-21 10:08 ` [kvm-unit-tests PATCH v5 4/5] riscv: Add helper method to set cpu started mask James Raphael Tiovalen
2024-09-21 10:08   ` James Raphael Tiovalen
2024-09-21 10:08 ` [kvm-unit-tests PATCH v5 5/5] riscv: sbi: Add tests for HSM extension James Raphael Tiovalen
2024-09-21 10:08   ` James Raphael Tiovalen
2024-10-23 14:23 ` [kvm-unit-tests PATCH v5 0/5] riscv: sbi: Add support to test " Andrew Jones
2024-10-23 14:23   ` Andrew Jones
2024-10-23 14:40   ` James R T
2024-10-23 14:40     ` James R T
2024-11-06  8:44   ` Andrew Jones
2024-11-06  8:44     ` Andrew Jones

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.