[RFC PATCH v2 0/8] Preserve more architectural state across domain switches

[RFC PATCH v2 0/8] Preserve more architectural state across domain switches

[Theodorus Wensan Februanto]

This RFC series extends OpenSBI domain context switching to preserve
additional hypervisor, virtual supervisor, supervisor timer,
counter configuration, AIA priority, CTR, and vector state.

Changes since v1:
- Group hypervisor and VS state in hart_context_hext.
- Fold HCOUNTEREN and HTIMEDELTA preservation into the core patch.
- Group STIMECMP and VSTIMECMP in hart_context_sstc.
- Move Sstc after H-extension state preservation.
- Detect CTR capacity in sbi_hart.c and store it in hart features.
- Include vector VL and VTYPE preservation.

Link: https://github.com/riscv-software-src/opensbi/issues/433
Link: https://github.com/riscv-software-src/opensbi/issues/434

Theodorus Wensan Februanto (8):
  lib: sbi: Preserve hypervisor state across domain switches
  lib: sbi: Preserve optional H-extension state across domain switches
  lib: sbi: Preserve Sstc timer state across domain switches
  lib: sbi: Preserve supervisor count inhibit configuration
  lib: sbi: Preserve the supervisor indirect CSR selector
  lib: sbi: Preserve supervisor AIA interrupt priorities
  lib: sbi: Preserve supervisor CTR records across domain switches
  lib: sbi: Preserve vl and vtype during domain switches

 include/sbi/riscv_encoding.h |   2 +
 include/sbi/sbi_hart.h       |   2 +
 include/sbi/sbi_vector.h     |   2 +
 lib/sbi/sbi_domain_context.c | 239 +++++++++++++++++++++++++++++++++++
 lib/sbi/sbi_hart.c           |  11 ++
 lib/sbi/sbi_vector.c         |  11 ++
 6 files changed, 267 insertions(+)

-- 
2.43.0


-- 
opensbi mailing list
opensbi@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

[RFC PATCH v2 1/8] lib: sbi: Preserve hypervisor state across domain switches

[Theodorus Wensan Februanto]

The domain context switch saves several supervisor CSRs but leaves
hypervisor trap, delegation, interrupt, and translation state shared
between domains.

Exchange core writable H-extension CSRs when H is available. Use a
64-bit exchange for henvcfg and check Smaia before accessing RV32
high-half AIA registers.

The domain protection reconfiguration performs SFENCE/HFENCE after
the CSR exchange.

Link: https://github.com/riscv-software-src/opensbi/issues/433
Signed-off-by: Theodorus Wensan Februanto <theodoruswensanfebruanto025@gmail.com>
---
 lib/sbi/sbi_domain_context.c | 70 ++++++++++++++++++++++++++++++++++++
 1 file changed, 70 insertions(+)

diff --git a/lib/sbi/sbi_domain_context.c b/lib/sbi/sbi_domain_context.c
index 0861d541..4bafda5b 100644
--- a/lib/sbi/sbi_domain_context.c
+++ b/lib/sbi/sbi_domain_context.c
@@ -21,6 +21,34 @@
 #include <sbi/sbi_vector.h>
 #include <sbi/sbi_fp.h>
 
+struct hart_context_hext {
+	unsigned long hstatus;
+	unsigned long hedeleg;
+	unsigned long hideleg;
+	unsigned long hie;
+	unsigned long hgeie;
+	u64 henvcfg;
+	unsigned long htval;
+	unsigned long htinst;
+	unsigned long hvip;
+	unsigned long hgatp;
+#if __riscv_xlen == 32
+	unsigned long hidelegh;
+	unsigned long hviph;
+#endif
+	unsigned long hcounteren;
+	u64 htimedelta;
+	unsigned long vsstatus;
+	unsigned long vsie;
+	unsigned long vstvec;
+	unsigned long vsscratch;
+	unsigned long vsepc;
+	unsigned long vscause;
+	unsigned long vstval;
+	unsigned long vsip;
+	unsigned long vsatp;
+};
+
 /** Context representation for a hart within a domain */
 struct hart_context {
 	/** Trap-related states such as GPRs, mepc, and mstatus */
@@ -44,6 +72,8 @@ struct hart_context {
 	unsigned long sip;
 	/** Supervisor address translation and protection register */
 	unsigned long satp;
+
+	struct hart_context_hext hext;
 	/** Counter-enable register */
 	unsigned long scounteren;
 	/** Supervisor environment configuration register */
@@ -139,6 +169,46 @@ static int switch_to_next_domain_context(struct hart_context *ctx,
 	ctx->stval	= csr_swap(CSR_STVAL, dom_ctx->stval);
 	ctx->sip	= csr_swap(CSR_SIP, dom_ctx->sip);
 	ctx->satp	= csr_swap(CSR_SATP, dom_ctx->satp);
+
+	if (misa_extension('H')) {
+		ctx->hext.hstatus = csr_swap(CSR_HSTATUS, dom_ctx->hext.hstatus);
+		ctx->hext.hedeleg = csr_swap(CSR_HEDELEG, dom_ctx->hext.hedeleg);
+		ctx->hext.hideleg = csr_swap(CSR_HIDELEG, dom_ctx->hext.hideleg);
+		ctx->hext.hie = csr_swap(CSR_HIE, dom_ctx->hext.hie);
+		ctx->hext.hgeie = csr_swap(CSR_HGEIE, dom_ctx->hext.hgeie);
+		ctx->hext.henvcfg = csr_swap64(CSR_HENVCFG, dom_ctx->hext.henvcfg);
+		ctx->hext.htval = csr_swap(CSR_HTVAL, dom_ctx->hext.htval);
+		ctx->hext.htinst = csr_swap(CSR_HTINST, dom_ctx->hext.htinst);
+		ctx->hext.hvip = csr_swap(CSR_HVIP, dom_ctx->hext.hvip);
+		ctx->hext.hgatp = csr_swap(CSR_HGATP, dom_ctx->hext.hgatp);
+		ctx->hext.hcounteren = csr_swap(CSR_HCOUNTEREN, dom_ctx->hext.hcounteren);
+		ctx->hext.htimedelta = csr_swap64(CSR_HTIMEDELTA, dom_ctx->hext.htimedelta);
+		ctx->hext.vsstatus = csr_swap(CSR_VSSTATUS,
+					 dom_ctx->hext.vsstatus);
+		ctx->hext.vsie = csr_swap(CSR_VSIE,
+					 dom_ctx->hext.vsie);
+		ctx->hext.vstvec = csr_swap(CSR_VSTVEC,
+					 dom_ctx->hext.vstvec);
+		ctx->hext.vsscratch = csr_swap(CSR_VSSCRATCH,
+					 dom_ctx->hext.vsscratch);
+		ctx->hext.vsepc = csr_swap(CSR_VSEPC,
+					 dom_ctx->hext.vsepc);
+		ctx->hext.vscause = csr_swap(CSR_VSCAUSE,
+					 dom_ctx->hext.vscause);
+		ctx->hext.vstval = csr_swap(CSR_VSTVAL,
+					 dom_ctx->hext.vstval);
+		ctx->hext.vsip = csr_swap(CSR_VSIP,
+					 dom_ctx->hext.vsip);
+		ctx->hext.vsatp = csr_swap(CSR_VSATP,
+					 dom_ctx->hext.vsatp);
+#if __riscv_xlen == 32
+		if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SMAIA)) {
+			ctx->hext.hidelegh = csr_swap(CSR_HIDELEGH,
+						dom_ctx->hext.hidelegh);
+			ctx->hext.hviph = csr_swap(CSR_HVIPH, dom_ctx->hext.hviph);
+		}
+#endif
+	}
 	if (sbi_hart_priv_version(scratch) >= SBI_HART_PRIV_VER_1_10)
 		ctx->scounteren = csr_swap(CSR_SCOUNTEREN, dom_ctx->scounteren);
 	if (sbi_hart_priv_version(scratch) >= SBI_HART_PRIV_VER_1_12)
-- 
2.43.0


-- 
opensbi mailing list
opensbi@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

[RFC PATCH v2 2/8] lib: sbi: Preserve optional H-extension state across domain switches

[Theodorus Wensan Februanto]

Preserve hypervisor interrupt enable state provided by Smaia and
hypervisor state-enable CSRs provided by Smstateen.

Use extension checks so systems without these CSRs are unaffected.
The 64-bit helper preserves the corresponding high-half CSRs on RV32.

Link: https://github.com/riscv-software-src/opensbi/issues/433
Signed-off-by: Theodorus Wensan Februanto <theodoruswensanfebruanto025@gmail.com>
---
 lib/sbi/sbi_domain_context.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/lib/sbi/sbi_domain_context.c b/lib/sbi/sbi_domain_context.c
index 4bafda5b..79419edc 100644
--- a/lib/sbi/sbi_domain_context.c
+++ b/lib/sbi/sbi_domain_context.c
@@ -32,6 +32,9 @@ struct hart_context_hext {
 	unsigned long htinst;
 	unsigned long hvip;
 	unsigned long hgatp;
+	/** Optional hypervisor state */
+	u64 hvien;
+	u64 hstateen[4];
 #if __riscv_xlen == 32
 	unsigned long hidelegh;
 	unsigned long hviph;
@@ -208,6 +211,19 @@ static int switch_to_next_domain_context(struct hart_context *ctx,
 			ctx->hext.hviph = csr_swap(CSR_HVIPH, dom_ctx->hext.hviph);
 		}
 #endif
+		if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SMAIA))
+			ctx->hext.hvien = csr_swap64(CSR_HVIEN, dom_ctx->hext.hvien);
+
+		if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SMSTATEEN)) {
+			ctx->hext.hstateen[0] = csr_swap64(CSR_HSTATEEN0,
+						dom_ctx->hext.hstateen[0]);
+			ctx->hext.hstateen[1] = csr_swap64(CSR_HSTATEEN1,
+						dom_ctx->hext.hstateen[1]);
+			ctx->hext.hstateen[2] = csr_swap64(CSR_HSTATEEN2,
+						dom_ctx->hext.hstateen[2]);
+			ctx->hext.hstateen[3] = csr_swap64(CSR_HSTATEEN3,
+						dom_ctx->hext.hstateen[3]);
+		}
 	}
 	if (sbi_hart_priv_version(scratch) >= SBI_HART_PRIV_VER_1_10)
 		ctx->scounteren = csr_swap(CSR_SCOUNTEREN, dom_ctx->scounteren);
-- 
2.43.0


-- 
opensbi mailing list
opensbi@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

[RFC PATCH v2 3/8] lib: sbi: Preserve Sstc timer state across domain switches

[Theodorus Wensan Februanto]

The Sstc extension provides a supervisor timer comparison register.
Without preserving this value, another domain can replace the
previous domain's timer deadline.

Keep a per-domain timer comparison value and initialize new domains
to a future deadline to avoid immediate timer interrupts.

Link: https://github.com/riscv-software-src/opensbi/issues/433
Signed-off-by: Theodorus Wensan Februanto <theodoruswensanfebruanto025@gmail.com>
---
 include/sbi/riscv_encoding.h |  2 ++
 lib/sbi/sbi_domain_context.c | 15 +++++++++++++++
 2 files changed, 17 insertions(+)

diff --git a/include/sbi/riscv_encoding.h b/include/sbi/riscv_encoding.h
index ffe7666c..60a0c39e 100644
--- a/include/sbi/riscv_encoding.h
+++ b/include/sbi/riscv_encoding.h
@@ -448,6 +448,8 @@
 #define CSR_VSCAUSE			0x242
 #define CSR_VSTVAL			0x243
 #define CSR_VSIP			0x244
+#define CSR_VSTIMECMP		0x24D
+#define CSR_VSTIMECMPH		0x25D
 #define CSR_VSATP			0x280
 
 /* Virtual Interrupts and Interrupt Priorities (H-extension with AIA) */
diff --git a/lib/sbi/sbi_domain_context.c b/lib/sbi/sbi_domain_context.c
index 79419edc..33c19d04 100644
--- a/lib/sbi/sbi_domain_context.c
+++ b/lib/sbi/sbi_domain_context.c
@@ -52,6 +52,11 @@ struct hart_context_hext {
 	unsigned long vsatp;
 };
 
+struct hart_context_sstc {
+	u64 stimecmp;
+	u64 vstimecmp;
+};
+
 /** Context representation for a hart within a domain */
 struct hart_context {
 	/** Trap-related states such as GPRs, mepc, and mstatus */
@@ -83,6 +88,7 @@ struct hart_context {
 	unsigned long senvcfg;
 	/** Supervisor resource management configuration register */
 	unsigned long srmcfg;
+	struct hart_context_sstc sstc;
 
 	/** Float context state */
 	struct sbi_fp_context fp_ctx;
@@ -231,6 +237,12 @@ static int switch_to_next_domain_context(struct hart_context *ctx,
 		ctx->senvcfg	= csr_swap(CSR_SENVCFG, dom_ctx->senvcfg);
 	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SSQOSID))
 		ctx->srmcfg	= csr_swap(CSR_SRMCFG, dom_ctx->srmcfg);
+	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SSTC)) {
+		ctx->sstc.stimecmp = csr_swap64(CSR_STIMECMP, dom_ctx->sstc.stimecmp);
+		if (misa_extension('H'))
+			ctx->sstc.vstimecmp = csr_swap64(CSR_VSTIMECMP,
+						 dom_ctx->sstc.vstimecmp);
+	}
 
 	/* Eager context switch for float */
 	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_F) ||
@@ -292,6 +304,9 @@ static int hart_context_init(u32 hartindex)
 		if (!ctx)
 			return SBI_ENOMEM;
 
+		ctx->sstc.stimecmp = ~0ULL;
+		ctx->sstc.vstimecmp = ~0ULL;
+
 		if (sbi_hart_has_extension(sbi_scratch_thishart_ptr(),
 					   SBI_HART_EXT_V)) {
 			vec_size = sbi_vector_context_size();
-- 
2.43.0


-- 
opensbi mailing list
opensbi@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

[RFC PATCH v2 4/8] lib: sbi: Preserve supervisor count inhibit configuration

[Theodorus Wensan Februanto]

Preserve the delegated counter inhibit configuration across domain
switches when Ssccfg is available and menvcfg.CDE is enabled.

The extra CDE guard avoids accessing scountinhibit while counter
delegation is disabled. scountinhibit aliases delegated parts of
mcountinhibit, so this is not a complete PMU state virtualization fix.

Link: https://github.com/riscv-software-src/opensbi/issues/433
Signed-off-by: Theodorus Wensan Februanto <theodoruswensanfebruanto025@gmail.com>
---
 lib/sbi/sbi_domain_context.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/lib/sbi/sbi_domain_context.c b/lib/sbi/sbi_domain_context.c
index 33c19d04..9b4a4f5a 100644
--- a/lib/sbi/sbi_domain_context.c
+++ b/lib/sbi/sbi_domain_context.c
@@ -89,6 +89,8 @@ struct hart_context {
 	/** Supervisor resource management configuration register */
 	unsigned long srmcfg;
 	struct hart_context_sstc sstc;
+	/** Supervisor counter inhibit configuration */
+	unsigned long scountinhibit;
 
 	/** Float context state */
 	struct sbi_fp_context fp_ctx;
@@ -243,6 +245,10 @@ static int switch_to_next_domain_context(struct hart_context *ctx,
 			ctx->sstc.vstimecmp = csr_swap64(CSR_VSTIMECMP,
 						 dom_ctx->sstc.vstimecmp);
 	}
+	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SSCCFG) &&
+	    (csr_read64(CSR_MENVCFG) & ENVCFG_CDE))
+		ctx->scountinhibit = csr_swap(CSR_SCOUNTINHIBIT,
+						 dom_ctx->scountinhibit);
 
 	/* Eager context switch for float */
 	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_F) ||
-- 
2.43.0


-- 
opensbi mailing list
opensbi@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

[RFC PATCH v2 5/8] lib: sbi: Preserve the supervisor indirect CSR selector

[Theodorus Wensan Februanto]

Save the supervisor indirect CSR selector when Sscsrind is available.

The sireg windows refer to different backing state depending on
siselect. This change preserves the selector only and does not save
the indirect backing state, IMSIC files, or CTR records.

Link: https://github.com/riscv-software-src/opensbi/issues/433
Signed-off-by: Theodorus Wensan Februanto <theodoruswensanfebruanto025@gmail.com>
---
 lib/sbi/sbi_domain_context.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/lib/sbi/sbi_domain_context.c b/lib/sbi/sbi_domain_context.c
index 9b4a4f5a..961fd182 100644
--- a/lib/sbi/sbi_domain_context.c
+++ b/lib/sbi/sbi_domain_context.c
@@ -91,6 +91,8 @@ struct hart_context {
 	struct hart_context_sstc sstc;
 	/** Supervisor counter inhibit configuration */
 	unsigned long scountinhibit;
+	/** Supervisor indirect CSR selector */
+	unsigned long siselect;
 
 	/** Float context state */
 	struct sbi_fp_context fp_ctx;
@@ -249,6 +251,8 @@ static int switch_to_next_domain_context(struct hart_context *ctx,
 	    (csr_read64(CSR_MENVCFG) & ENVCFG_CDE))
 		ctx->scountinhibit = csr_swap(CSR_SCOUNTINHIBIT,
 						 dom_ctx->scountinhibit);
+	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SSCSRIND))
+		ctx->siselect = csr_swap(CSR_SISELECT, dom_ctx->siselect);
 
 	/* Eager context switch for float */
 	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_F) ||
-- 
2.43.0


-- 
opensbi mailing list
opensbi@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

[RFC PATCH v2 6/8] lib: sbi: Preserve supervisor AIA interrupt priorities

[Theodorus Wensan Februanto]

The siselect/sireg window exposes supervisor interrupt priority state
at selector values 0x30 through 0x3f.  Save and restore the implemented
bank across domain switches when Smaia is available.

Only sireg is accessed in this selector range.  The other sireg windows
can trap and must not be read.  IMSIC indirect registers at 0x70-0xff
are not handled in this change.

Link: https://github.com/riscv-software-src/opensbi/issues/433
Signed-off-by: Theodorus Wensan Februanto <theodoruswensanfebruanto025@gmail.com>
---
 lib/sbi/sbi_domain_context.c | 30 +++++++++++++++++++++++++++++-
 1 file changed, 29 insertions(+), 1 deletion(-)

diff --git a/lib/sbi/sbi_domain_context.c b/lib/sbi/sbi_domain_context.c
index 961fd182..34d1bc76 100644
--- a/lib/sbi/sbi_domain_context.c
+++ b/lib/sbi/sbi_domain_context.c
@@ -93,6 +93,8 @@ struct hart_context {
 	unsigned long scountinhibit;
 	/** Supervisor indirect CSR selector */
 	unsigned long siselect;
+	/** AIA supervisor interrupt priority register bank */
+	unsigned long siprio[16];
 
 	/** Float context state */
 	struct sbi_fp_context fp_ctx;
@@ -138,6 +140,26 @@ static void hart_context_set(struct sbi_domain *dom, u32 hartindex,
 	hart_context_get(sbi_domain_thishart_ptr(),			\
 			 current_hartindex())
 
+static void hart_aia_iprio_save(struct hart_context *ctx)
+{
+	unsigned int i;
+
+	for (i = 0; i < 16; i += __riscv_xlen / 32) {
+		csr_write(CSR_SISELECT, 0x30 + i);
+		ctx->siprio[i] = csr_read(CSR_SIREG);
+	}
+}
+
+static void hart_aia_iprio_restore(const struct hart_context *ctx)
+{
+	unsigned int i;
+
+	for (i = 0; i < 16; i += __riscv_xlen / 32) {
+		csr_write(CSR_SISELECT, 0x30 + i);
+		csr_write(CSR_SIREG, ctx->siprio[i]);
+	}
+}
+
 /**
  * Switches the HART context from the current domain to the target domain.
  * This includes changing domain assignments and reconfiguring PMP, as well
@@ -251,8 +273,14 @@ static int switch_to_next_domain_context(struct hart_context *ctx,
 	    (csr_read64(CSR_MENVCFG) & ENVCFG_CDE))
 		ctx->scountinhibit = csr_swap(CSR_SCOUNTINHIBIT,
 						 dom_ctx->scountinhibit);
-	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SSCSRIND))
+	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SSCSRIND) ||
+	    sbi_hart_has_extension(scratch, SBI_HART_EXT_SMAIA))
 		ctx->siselect = csr_swap(CSR_SISELECT, dom_ctx->siselect);
+	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SMAIA)) {
+		hart_aia_iprio_save(ctx);
+		hart_aia_iprio_restore(dom_ctx);
+		csr_write(CSR_SISELECT, dom_ctx->siselect);
+	}
 
 	/* Eager context switch for float */
 	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_F) ||
-- 
2.43.0


-- 
opensbi mailing list
opensbi@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

[RFC PATCH v2 7/8] lib: sbi: Preserve supervisor CTR records across domain switches

[Theodorus Wensan Februanto]

Save supervisor CTR control, depth, status, and logical record
entries when Ssctr and Sscsrind are available. This experimental
implementation freezes recording before saving and restores
status before logical records.

IMSIC and other indirect CSR state remain outside this change.

Link: https://github.com/riscv-software-src/opensbi/issues/433
Signed-off-by: Theodorus Wensan Februanto <theodoruswensanfebruanto025@gmail.com>
---
 include/sbi/sbi_hart.h       |   2 +
 lib/sbi/sbi_domain_context.c | 100 +++++++++++++++++++++++++++++++++++
 lib/sbi/sbi_hart.c           |  11 ++++
 3 files changed, 113 insertions(+)

diff --git a/include/sbi/sbi_hart.h b/include/sbi/sbi_hart.h
index 543393bb..171fe759 100644
--- a/include/sbi/sbi_hart.h
+++ b/include/sbi/sbi_hart.h
@@ -125,6 +125,8 @@ struct sbi_hart_features {
 	unsigned int pmp_log2gran;
 	unsigned int mhpm_mask;
 	unsigned int mhpm_bits;
+	unsigned long ctr_depth;
+	unsigned int ctr_num_entries;
 };
 
 extern unsigned long hart_features_offset;
diff --git a/lib/sbi/sbi_domain_context.c b/lib/sbi/sbi_domain_context.c
index 34d1bc76..b2918c9f 100644
--- a/lib/sbi/sbi_domain_context.c
+++ b/lib/sbi/sbi_domain_context.c
@@ -21,6 +21,24 @@
 #include <sbi/sbi_vector.h>
 #include <sbi/sbi_fp.h>
 
+#define HART_CTR_FIRST_SELECTOR 0x200
+#define HART_CTR_FROZEN (1UL << 31)
+
+struct hart_ctr_entry {
+	unsigned long source;
+	unsigned long target;
+	unsigned long data;
+};
+
+struct hart_ctr_context {
+	unsigned long ctl;
+	unsigned long depth;
+	unsigned long status;
+	unsigned long vsctl;
+	unsigned int num_entries;
+	struct hart_ctr_entry entries[];
+};
+
 struct hart_context_hext {
 	unsigned long hstatus;
 	unsigned long hedeleg;
@@ -95,6 +113,8 @@ struct hart_context {
 	unsigned long siselect;
 	/** AIA supervisor interrupt priority register bank */
 	unsigned long siprio[16];
+	/** Supervisor control transfer records */
+	struct hart_ctr_context *ctr_ctx;
 
 	/** Float context state */
 	struct sbi_fp_context fp_ctx;
@@ -160,6 +180,63 @@ static void hart_aia_iprio_restore(const struct hart_context *ctx)
 	}
 }
 
+#if __riscv_xlen == 64
+static unsigned int hart_ctr_entry_count(unsigned long depth)
+{
+	unsigned int encoding = depth & 7;
+
+	if (encoding > 4)
+		return 0;
+	return 16U << encoding;
+}
+
+static void hart_ctr_save(struct hart_ctr_context *ctx)
+{
+	unsigned int i, count;
+
+	ctx->status = csr_read_set(CSR_SCTRSTATUS, HART_CTR_FROZEN);
+	ctx->ctl = csr_read(CSR_SCTRCTL);
+	ctx->depth = csr_read(CSR_SCTRDEPTH);
+	if (misa_extension('H'))
+		ctx->vsctl = csr_read(CSR_VSCTRCTL);
+
+	count = hart_ctr_entry_count(ctx->depth);
+	if (count > ctx->num_entries)
+		count = ctx->num_entries;
+	for (i = 0; i < count; i++) {
+		csr_write(CSR_SISELECT, HART_CTR_FIRST_SELECTOR + i);
+		ctx->entries[i].source = csr_read(CSR_SIREG);
+		ctx->entries[i].target = csr_read(CSR_SIREG2);
+		ctx->entries[i].data = csr_read(CSR_SIREG3);
+	}
+}
+
+static void hart_ctr_restore(const struct hart_ctr_context *ctx)
+{
+	unsigned int i, count;
+
+	csr_write(CSR_SCTRCTL, 0);
+	csr_write(CSR_SCTRDEPTH, ctx->depth);
+	__asm__ __volatile__(".4byte 0x10400073" ::: "memory");
+	csr_write(CSR_SCTRSTATUS, ctx->status | HART_CTR_FROZEN);
+
+	count = hart_ctr_entry_count(ctx->depth);
+	if (count > ctx->num_entries)
+		count = ctx->num_entries;
+	for (i = 0; i < count; i++) {
+		csr_write(CSR_SISELECT, HART_CTR_FIRST_SELECTOR + i);
+		csr_write(CSR_SIREG2, ctx->entries[i].target);
+		csr_write(CSR_SIREG3, ctx->entries[i].data);
+		csr_write(CSR_SIREG, ctx->entries[i].source);
+	}
+
+	if (misa_extension('H'))
+		csr_write(CSR_VSCTRCTL, ctx->vsctl);
+	csr_write(CSR_SCTRCTL, ctx->ctl);
+	csr_write(CSR_SCTRSTATUS, ctx->status);
+}
+#endif
+
 /**
  * Switches the HART context from the current domain to the target domain.
  * This includes changing domain assignments and reconfiguring PMP, as well
@@ -281,6 +358,13 @@ static int switch_to_next_domain_context(struct hart_context *ctx,
 		hart_aia_iprio_restore(dom_ctx);
 		csr_write(CSR_SISELECT, dom_ctx->siselect);
 	}
+#if __riscv_xlen == 64
+	if (ctx->ctr_ctx && dom_ctx->ctr_ctx) {
+		hart_ctr_save(ctx->ctr_ctx);
+		hart_ctr_restore(dom_ctx->ctr_ctx);
+		csr_write(CSR_SISELECT, dom_ctx->siselect);
+	}
+#endif
 
 	/* Eager context switch for float */
 	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_F) ||
@@ -331,6 +415,8 @@ static int hart_context_init(u32 hartindex)
 {
 	size_t vec_size;
 	struct hart_context *ctx;
+	struct sbi_hart_features *hart_features =
+		sbi_hart_features_ptr(sbi_scratch_thishart_ptr());
 	struct sbi_domain *dom;
 
 	sbi_domain_for_each(dom) {
@@ -345,6 +431,19 @@ static int hart_context_init(u32 hartindex)
 		ctx->sstc.stimecmp = ~0ULL;
 		ctx->sstc.vstimecmp = ~0ULL;
 
+#if __riscv_xlen == 64
+		if (hart_features->ctr_num_entries) {
+			ctx->ctr_ctx = sbi_zalloc(sizeof(*ctx->ctr_ctx) +
+				hart_features->ctr_num_entries * sizeof(struct hart_ctr_entry));
+			if (!ctx->ctr_ctx) {
+				sbi_free(ctx);
+				return SBI_ENOMEM;
+			}
+			ctx->ctr_ctx->depth = hart_features->ctr_depth;
+			ctx->ctr_ctx->num_entries = hart_features->ctr_num_entries;
+		}
+#endif
+
 		if (sbi_hart_has_extension(sbi_scratch_thishart_ptr(),
 					   SBI_HART_EXT_V)) {
 			vec_size = sbi_vector_context_size();
@@ -352,6 +451,7 @@ static int hart_context_init(u32 hartindex)
 			/* Allocate the vector context pointer */
 			ctx->vec_ctx = sbi_zalloc(vec_size);
 			if (!ctx->vec_ctx) {
+				sbi_free(ctx->ctr_ctx);
 				sbi_free(ctx);
 				return SBI_ENOMEM;
 			}
diff --git a/lib/sbi/sbi_hart.c b/lib/sbi/sbi_hart.c
index bee88557..a4bbb750 100644
--- a/lib/sbi/sbi_hart.c
+++ b/lib/sbi/sbi_hart.c
@@ -672,6 +672,17 @@ __pmp_skip:
 		__sbi_hart_update_extension(hfeatures,
 					SBI_HART_EXT_ZIHPM, true);
 
+#if __riscv_xlen == 64
+	if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SSCTR) &&
+	    sbi_hart_has_extension(scratch, SBI_HART_EXT_SSCSRIND)) {
+		val = csr_read_allowed(CSR_SCTRDEPTH, &trap);
+		if (!trap.cause && (val & 7) <= 4) {
+			hfeatures->ctr_depth = val;
+			hfeatures->ctr_num_entries = 16U << (val & 7);
+		}
+	}
+#endif
+
 	/* Mark hart feature detection done */
 	hfeatures->detected = true;
 
-- 
2.43.0


-- 
opensbi mailing list
opensbi@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

[RFC PATCH v2 8/8] lib: sbi: Preserve vl and vtype during domain switches

[Theodorus Wensan Februanto]

Save vl and vtype as part of the Vector domain context.

Restore both CSRs using vsetvl after restoring the Vector register
file. Restore vstart after vsetvl because Vector configuration
instructions reset vstart.

Link: https://github.com/riscv-software-src/opensbi/issues/434
Signed-off-by: Theodorus Wensan Februanto <theodoruswensanfebruanto025@gmail.com>
---
 include/sbi/sbi_vector.h |  2 ++
 lib/sbi/sbi_vector.c     | 11 +++++++++++
 2 files changed, 13 insertions(+)

diff --git a/include/sbi/sbi_vector.h b/include/sbi/sbi_vector.h
index c14f3174..480f9ac3 100644
--- a/include/sbi/sbi_vector.h
+++ b/include/sbi/sbi_vector.h
@@ -15,6 +15,8 @@
 struct sbi_vector_context {
 	unsigned long vcsr;
 	unsigned long vstart;
+	unsigned long vl;
+	unsigned long vtype;
 
 	/* size depends on VLEN */
 	uint8_t vregs[];
diff --git a/lib/sbi/sbi_vector.c b/lib/sbi/sbi_vector.c
index 1f9e01cf..53c2e5a7 100644
--- a/lib/sbi/sbi_vector.c
+++ b/lib/sbi/sbi_vector.c
@@ -33,6 +33,8 @@ void sbi_vector_save(struct sbi_vector_context *dst)
 	/* Step 2: Save CSRs */
 	dst->vcsr = csr_read(vcsr);
 	dst->vstart = csr_read(vstart);
+	dst->vl = csr_read(vl);
+	dst->vtype = csr_read(vtype);
 
 	/* Step 3: Save vector registers */
 #define SAVE_VREG(i)							\
@@ -86,6 +88,15 @@ void sbi_vector_restore(const struct sbi_vector_context *src)
 
 	/* Step 3: Restore CSR's last */
 	/* Restore CSRs first */
+	asm volatile(
+		".option push\n\t"
+		".option arch, +v\n\t"
+		"vsetvl zero, %0, %1\n\t"
+		".option pop\n\t"
+		:
+		: "r"(src->vl), "r"(src->vtype)
+		: "memory");
+
 	csr_write(vcsr,   src->vcsr);
 	csr_write(vstart, src->vstart);
 
-- 
2.43.0


-- 
opensbi mailing list
opensbi@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

Re: [RFC PATCH v2 1/8] lib: sbi: Preserve hypervisor state across domain switches

[Radim Krcmar]

2026-10-23T00:09:18-07:00, Theodorus Wensan Februanto <theodoruswensanfebruanto025@gmail.com>:
> The domain context switch saves several supervisor CSRs but leaves
> hypervisor trap, delegation, interrupt, and translation state shared
> between domains.
>
> Exchange core writable H-extension CSRs when H is available. Use a
> 64-bit exchange for henvcfg and check Smaia before accessing RV32
> high-half AIA registers.
>
> The domain protection reconfiguration performs SFENCE/HFENCE after
> the CSR exchange.
>
> Link: https://github.com/riscv-software-src/opensbi/issues/433
> Signed-off-by: Theodorus Wensan Februanto <theodoruswensanfebruanto025@gmail.com>
> ---
> diff --git a/lib/sbi/sbi_domain_context.c b/lib/sbi/sbi_domain_context.c
> @@ -139,6 +169,46 @@ static int switch_to_next_domain_context(struct hart_context *ctx,
> +	if (misa_extension('H')) {
> +		ctx->hext.hideleg = csr_swap(CSR_HIDELEG, dom_ctx->hext.hideleg);
> +		ctx->hext.hie = csr_swap(CSR_HIE, dom_ctx->hext.hie);
> +		ctx->hext.hgeie = csr_swap(CSR_HGEIE, dom_ctx->hext.hgeie);
> +		ctx->hext.hvip = csr_swap(CSR_HVIP, dom_ctx->hext.hvip);
> +		ctx->hext.vsie = csr_swap(CSR_VSIE,
> +					 dom_ctx->hext.vsie);
> +		ctx->hext.vsip = csr_swap(CSR_VSIP,
> +					 dom_ctx->hext.vsip);
> +#if __riscv_xlen == 32
> +		if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SMAIA)) {
> +			ctx->hext.hidelegh = csr_swap(CSR_HIDELEGH,
> +						dom_ctx->hext.hidelegh);
> +			ctx->hext.hviph = csr_swap(CSR_HVIPH, dom_ctx->hext.hviph);
> +		}
> +#endif

The state of hie, hideleg, hvip, vsie, and vsip is connected, so the
context switch needs to take that into account;
the order should first save vs*, then swap h*, and finally restore vs*.

(Also, vsieh and vsiph are missing on RV32.)

Thanks.
-- 
opensbi mailing list
opensbi@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

Re: [RFC PATCH v2 2/8] lib: sbi: Preserve optional H-extension state across domain switches

[Radim Krcmar]

Btw. you're adding optional state in [1/8] as well, so why not squash at
least the interrupt parts?

2026-10-23T00:09:19-07:00, Theodorus Wensan Februanto <theodoruswensanfebruanto025@gmail.com>:
> Preserve hypervisor interrupt enable state provided by Smaia and
> hypervisor state-enable CSRs provided by Smstateen.
>
> Use extension checks so systems without these CSRs are unaffected.
> The 64-bit helper preserves the corresponding high-half CSRs on RV32.
>
> Link: https://github.com/riscv-software-src/opensbi/issues/433
> Signed-off-by: Theodorus Wensan Februanto <theodoruswensanfebruanto025@gmail.com>
> ---
> diff --git a/lib/sbi/sbi_domain_context.c b/lib/sbi/sbi_domain_context.c
> @@ -208,6 +211,19 @@ static int switch_to_next_domain_context(struct hart_context *ctx,
>  			ctx->hext.hviph = csr_swap(CSR_HVIPH, dom_ctx->hext.hviph);
>  		}
>  #endif
> +		if (sbi_hart_has_extension(scratch, SBI_HART_EXT_SMAIA))
> +			ctx->hext.hvien = csr_swap64(CSR_HVIEN, dom_ctx->hext.hvien);

hvien also needs to be carefully ordered because it has an effect on
vsip and vsie.

(hvictl, hviprio1, and hviprio2 are missing.)

Thanks.
-- 
opensbi mailing list
opensbi@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

Re: [RFC PATCH v2 6/8] lib: sbi: Preserve supervisor AIA interrupt priorities

[Radim Krcmar]

2026-10-23T00:09:23-07:00, Theodorus Wensan Februanto <theodoruswensanfebruanto025@gmail.com>:
> The siselect/sireg window exposes supervisor interrupt priority state
> at selector values 0x30 through 0x3f.  Save and restore the implemented
> bank across domain switches when Smaia is available.
>
> Only sireg is accessed in this selector range.  The other sireg windows
> can trap and must not be read.  IMSIC indirect registers at 0x70-0xff
> are not handled in this change.

Simply context switching IMSIC files through 0x70-0xff wouldn't bring us
much closer to implementing AIA for domains.

In systems without Smsdia, I think it would be better to give AIA only
to a single domain on each hart and prevent access from other domains,
because it's not like concurent domains can sanely share AIA anyway.

To prevent other domains from accessing AIA, we'd need machinery to
define the set of available features for each domain, and manipulate
mstateen.

> Link: https://github.com/riscv-software-src/opensbi/issues/433
> Signed-off-by: Theodorus Wensan Februanto <theodoruswensanfebruanto025@gmail.com>
> ---
> diff --git a/lib/sbi/sbi_domain_context.c b/lib/sbi/sbi_domain_context.c
> @@ -93,6 +93,8 @@ struct hart_context {
> +	/** AIA supervisor interrupt priority register bank */
> +	unsigned long siprio[16];

The priorities are u32[16] or u64[8], although the code wastes memory to
look very clean, so I'm not sure we want to optimize...

Thanks.
-- 
opensbi mailing list
opensbi@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi